2024-05-08T05:30:15.286 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T05:30:15.286 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name jcollin-2024-05-08_05:22:48-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi --archive /home/teuthworker/archive/jcollin-2024-05-08_05:22:48-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697310 --owner scheduled_jcollin@teuthology --description fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} -- /home/teuthworker/archive/jcollin-2024-05-08_05:22:48-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697310/orig.config.yaml 2024-05-08T05:30:15.333 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T05:30:15.423 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/jcollin-2024-05-08_05:22:48-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697310 branch: wip-jcollin-testing-07052024 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} email: jcollin@redhat.com first_in_suite: false job_id: '7697310' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: jcollin-2024-05-08_05:22:48-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-jcollin-testing-07052024 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: cd693b0b93278c2b128be2d6d93ae4df18c3ba1c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: cd693b0b93278c2b128be2d6d93ae4df18c3ba1c kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-jcollin-testing-07052024 sha1: cd693b0b93278c2b128be2d6d93ae4df18c3ba1c owner: scheduled_jcollin@teuthology priority: 2 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3880 sha1: cd693b0b93278c2b128be2d6d93ae4df18c3ba1c sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-jcollin-testing-07052024 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: cd693b0b93278c2b128be2d6d93ae4df18c3ba1c syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNg+jL2TFZLqEtYhSHQewN38uACCZFO7eXUwqrpHlx/FTXOWQj+xZ3w69kzHZOiA4tYvSJvJbpniXpITjUVOnWA= smithi063.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJLilVd7/rNRhWYw6MU9+3Ckro0GQceTbVPKn/c5mxyp28sk/YF12goWlIH7ePrfPdpZnK6SAh44avQEtMLye4g= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_05:22:48 tube: smithi user: jcollin verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T05:30:15.423 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/qa; will attempt to use it 2024-05-08T05:30:15.424 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/qa/tasks 2024-05-08T05:30:15.424 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T05:30:15.425 INFO:teuthology.task.internal:Checking packages... 2024-05-08T05:30:15.449 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c' 2024-05-08T05:30:15.449 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T05:30:15.449 INFO:teuthology.packaging:ref: None 2024-05-08T05:30:15.449 INFO:teuthology.packaging:tag: None 2024-05-08T05:30:15.449 INFO:teuthology.packaging:branch: wip-jcollin-testing-07052024 2024-05-08T05:30:15.450 INFO:teuthology.packaging:sha1: cd693b0b93278c2b128be2d6d93ae4df18c3ba1c 2024-05-08T05:30:15.450 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-jcollin-testing-07052024 2024-05-08T05:30:15.589 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3538.gcd693b0b 2024-05-08T05:30:15.590 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T05:30:15.597 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T05:30:15.597 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T05:30:15.619 INFO:teuthology.task.internal:Saving configuration 2024-05-08T05:30:15.636 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T05:30:15.642 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T05:30:15.667 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-05-08_05:22:48-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697310', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 05:25:06.792692', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNg+jL2TFZLqEtYhSHQewN38uACCZFO7eXUwqrpHlx/FTXOWQj+xZ3w69kzHZOiA4tYvSJvJbpniXpITjUVOnWA='} 2024-05-08T05:30:15.689 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi063.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-05-08_05:22:48-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697310', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 05:25:06.794254', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJLilVd7/rNRhWYw6MU9+3Ckro0GQceTbVPKn/c5mxyp28sk/YF12goWlIH7ePrfPdpZnK6SAh44avQEtMLye4g='} 2024-05-08T05:30:15.689 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T05:30:15.695 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-08T05:30:15.695 INFO:teuthology.task.internal:roles: ubuntu@smithi063.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-08T05:30:15.695 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T05:30:15.767 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7faad7cd9160>, signals=[15]) 2024-05-08T05:30:15.767 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T05:30:15.773 INFO:teuthology.task.internal:Opening connections... 2024-05-08T05:30:15.773 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:30:15.775 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:30:15.855 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi063.front.sepia.ceph.com 2024-05-08T05:30:15.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:30:15.933 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T05:30:15.939 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2024-05-08T05:30:15.957 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2024-05-08T05:30:15.958 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2024-05-08T05:30:16.016 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2024-05-08T05:30:16.016 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="9" 2024-05-08T05:30:16.016 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2024-05-08T05:30:16.016 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2024-05-08T05:30:16.016 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="9" 2024-05-08T05:30:16.016 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el9" 2024-05-08T05:30:16.016 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T05:30:16.016 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2024-05-08T05:30:16.016 INFO:teuthology.orchestra.run.smithi045.stdout:LOGO="fedora-logo-icon" 2024-05-08T05:30:16.016 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T05:30:16.017 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2024-05-08T05:30:16.017 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T05:30:16.017 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T05:30:16.017 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T05:30:16.017 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2024-05-08T05:30:16.045 DEBUG:teuthology.orchestra.run.smithi063:> uname -m 2024-05-08T05:30:16.064 INFO:teuthology.orchestra.run.smithi063.stdout:x86_64 2024-05-08T05:30:16.064 DEBUG:teuthology.orchestra.run.smithi063:> cat /etc/os-release 2024-05-08T05:30:16.122 INFO:teuthology.orchestra.run.smithi063.stdout:NAME="CentOS Stream" 2024-05-08T05:30:16.122 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION="9" 2024-05-08T05:30:16.122 INFO:teuthology.orchestra.run.smithi063.stdout:ID="centos" 2024-05-08T05:30:16.122 INFO:teuthology.orchestra.run.smithi063.stdout:ID_LIKE="rhel fedora" 2024-05-08T05:30:16.122 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_ID="9" 2024-05-08T05:30:16.122 INFO:teuthology.orchestra.run.smithi063.stdout:PLATFORM_ID="platform:el9" 2024-05-08T05:30:16.123 INFO:teuthology.orchestra.run.smithi063.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T05:30:16.123 INFO:teuthology.orchestra.run.smithi063.stdout:ANSI_COLOR="0;31" 2024-05-08T05:30:16.123 INFO:teuthology.orchestra.run.smithi063.stdout:LOGO="fedora-logo-icon" 2024-05-08T05:30:16.123 INFO:teuthology.orchestra.run.smithi063.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T05:30:16.123 INFO:teuthology.orchestra.run.smithi063.stdout:HOME_URL="https://centos.org/" 2024-05-08T05:30:16.123 INFO:teuthology.orchestra.run.smithi063.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T05:30:16.123 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T05:30:16.123 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T05:30:16.123 INFO:teuthology.lock.ops:Updating smithi063.front.sepia.ceph.com on lock server 2024-05-08T05:30:16.145 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T05:30:16.155 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T05:30:16.160 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T05:30:16.161 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2024-05-08T05:30:16.163 DEBUG:teuthology.orchestra.run.smithi063:> test '!' -e /home/ubuntu/cephtest 2024-05-08T05:30:16.180 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T05:30:16.186 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T05:30:16.186 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2024-05-08T05:30:16.220 DEBUG:teuthology.orchestra.run.smithi063:> test -z $(ls -A /var/lib/ceph) 2024-05-08T05:30:16.249 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T05:30:16.302 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T05:30:16.317 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-05-08T05:30:16.317 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-05-08T05:30:16.317 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-05-08T05:30:16.318 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T05:30:16.318 DEBUG:teuthology.orchestra.run.smithi063:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T05:30:16.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:30:16.336 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-05-08T05:30:16.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:30:16.337 DEBUG:teuthology.orchestra.run.smithi063:> uname -r 2024-05-08T05:30:16.393 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-437.el9.x86_64 2024-05-08T05:30:16.394 INFO:teuthology.task.kernel:Running kernel on smithi045: 5.14.0-437.el9.x86_64 2024-05-08T05:30:16.394 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum install -y kernel 2024-05-08T05:30:16.395 INFO:teuthology.orchestra.run.smithi063.stdout:5.14.0-437.el9.x86_64 2024-05-08T05:30:16.395 INFO:teuthology.task.kernel:Running kernel on smithi063: 5.14.0-437.el9.x86_64 2024-05-08T05:30:16.395 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum install -y kernel 2024-05-08T05:30:19.361 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 6.1 MB/s | 8.1 MB 00:01 2024-05-08T05:30:20.214 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - BaseOS 3.6 MB/s | 8.1 MB 00:02 2024-05-08T05:30:21.145 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 30 MB/s | 19 MB 00:00 2024-05-08T05:30:25.214 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - AppStream 4.9 MB/s | 19 MB 00:03 2024-05-08T05:30:25.312 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-05-08T05:30:26.796 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 16 kB 00:00 2024-05-08T05:30:28.208 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-05-08T05:30:31.323 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.2 MB 00:02 2024-05-08T05:30:32.790 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 16 kB 00:00 2024-05-08T05:30:33.672 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-05-08T05:30:34.133 INFO:teuthology.orchestra.run.smithi063.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-05-08T05:30:35.762 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T05:30:35.762 INFO:teuthology.orchestra.run.smithi045.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T05:30:35.860 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-08T05:30:35.865 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-08T05:30:35.865 INFO:teuthology.orchestra.run.smithi045.stdout: Package Architecture Version Repository Size 2024-05-08T05:30:35.865 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-08T05:30:35.865 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-08T05:30:35.865 INFO:teuthology.orchestra.run.smithi045.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T05:30:35.865 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-05-08T05:30:35.865 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T05:30:35.865 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T05:30:35.865 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T05:30:35.865 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:30:35.865 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-08T05:30:35.865 INFO:teuthology.orchestra.run.smithi045.stdout:================================================================================ 2024-05-08T05:30:35.866 INFO:teuthology.orchestra.run.smithi045.stdout:Install 4 Packages 2024-05-08T05:30:35.866 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:30:35.866 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 78 M 2024-05-08T05:30:35.866 INFO:teuthology.orchestra.run.smithi045.stdout:Installed size: 123 M 2024-05-08T05:30:35.866 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-08T05:30:36.815 INFO:teuthology.orchestra.run.smithi045.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 711 kB/s | 484 kB 00:00 2024-05-08T05:30:39.483 INFO:teuthology.orchestra.run.smithi063.stdout:lab-extras 51 kB/s | 1.7 kB 00:00 2024-05-08T05:30:41.539 INFO:teuthology.orchestra.run.smithi063.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T05:30:41.540 INFO:teuthology.orchestra.run.smithi063.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T05:30:41.638 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-08T05:30:41.644 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-08T05:30:41.644 INFO:teuthology.orchestra.run.smithi063.stdout: Package Architecture Version Repository Size 2024-05-08T05:30:41.644 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-08T05:30:41.644 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-08T05:30:41.644 INFO:teuthology.orchestra.run.smithi063.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T05:30:41.644 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-05-08T05:30:41.645 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T05:30:41.645 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T05:30:41.645 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T05:30:41.645 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:30:41.645 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-08T05:30:41.645 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-05-08T05:30:41.645 INFO:teuthology.orchestra.run.smithi063.stdout:Install 4 Packages 2024-05-08T05:30:41.645 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:30:41.646 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 78 M 2024-05-08T05:30:41.646 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 123 M 2024-05-08T05:30:41.646 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-08T05:30:41.893 INFO:teuthology.orchestra.run.smithi063.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 2.7 MB/s | 484 kB 00:00 2024-05-08T05:30:42.560 INFO:teuthology.orchestra.run.smithi063.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 18 MB/s | 15 MB 00:00 2024-05-08T05:30:42.943 INFO:teuthology.orchestra.run.smithi063.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 27 MB/s | 28 MB 00:01 2024-05-08T05:30:43.552 INFO:teuthology.orchestra.run.smithi063.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 18 MB/s | 34 MB 00:01 2024-05-08T05:30:43.555 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:30:43.555 INFO:teuthology.orchestra.run.smithi063.stdout:Total 41 MB/s | 78 MB 00:01 2024-05-08T05:30:44.267 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-08T05:30:44.663 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-08T05:30:44.663 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-08T05:30:45.593 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-08T05:30:45.593 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-08T05:30:48.171 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-08T05:30:48.825 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T05:30:48.910 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T05:30:49.181 INFO:teuthology.orchestra.run.smithi045.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 1.2 MB/s | 15 MB 00:13 2024-05-08T05:30:51.721 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T05:30:51.851 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T05:30:56.927 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T05:30:56.978 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:31:02.069 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:31:05.054 INFO:teuthology.orchestra.run.smithi045.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 1.0 MB/s | 28 MB 00:28 2024-05-08T05:31:16.820 INFO:teuthology.orchestra.run.smithi045.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 854 kB/s | 34 MB 00:40 2024-05-08T05:31:16.820 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:31:16.820 INFO:teuthology.orchestra.run.smithi045.stdout:Total 1.9 MB/s | 78 MB 00:40 2024-05-08T05:31:17.523 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-08T05:31:17.934 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-08T05:31:17.934 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-08T05:31:18.894 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-08T05:31:18.894 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-08T05:31:21.471 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-08T05:31:22.144 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T05:31:22.227 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T05:31:25.016 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T05:31:25.151 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T05:31:30.232 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T05:31:30.278 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:31:33.210 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:31:33.251 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:31:33.625 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:31:33.625 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T05:31:33.625 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T05:31:33.625 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T05:31:34.471 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:31:34.471 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:31:34.471 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-08T05:31:34.471 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T05:31:34.471 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T05:31:34.471 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T05:31:34.471 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T05:31:34.471 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:31:34.472 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-08T05:31:34.816 DEBUG:teuthology.orchestra.run.smithi063:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T05:31:34.888 INFO:teuthology.orchestra.run.smithi063.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T05:31:34.888 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T05:31:34.888 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T05:31:34.888 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T05:31:34.888 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T05:31:34.888 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-05-08T05:31:34.889 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi063.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T05:31:34.889 DEBUG:teuthology.orchestra.run.smithi063:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T05:31:34.940 INFO:teuthology.orchestra.run.smithi063.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T05:31:34.940 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T05:31:34.940 DEBUG:teuthology.orchestra.run.smithi063:> sudo rpm -qi grub2-tools 2024-05-08T05:31:34.971 INFO:teuthology.orchestra.run.smithi063.stdout:Name : grub2-tools 2024-05-08T05:31:34.971 INFO:teuthology.orchestra.run.smithi063.stdout:Epoch : 1 2024-05-08T05:31:34.971 INFO:teuthology.orchestra.run.smithi063.stdout:Version : 2.06 2024-05-08T05:31:34.971 INFO:teuthology.orchestra.run.smithi063.stdout:Release : 68.el9 2024-05-08T05:31:34.971 INFO:teuthology.orchestra.run.smithi063.stdout:Architecture: x86_64 2024-05-08T05:31:34.971 INFO:teuthology.orchestra.run.smithi063.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T05:31:34.972 INFO:teuthology.orchestra.run.smithi063.stdout:Group : Unspecified 2024-05-08T05:31:34.972 INFO:teuthology.orchestra.run.smithi063.stdout:Size : 8264254 2024-05-08T05:31:34.972 INFO:teuthology.orchestra.run.smithi063.stdout:License : GPLv3+ 2024-05-08T05:31:34.972 INFO:teuthology.orchestra.run.smithi063.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T05:31:34.972 INFO:teuthology.orchestra.run.smithi063.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T05:31:34.972 INFO:teuthology.orchestra.run.smithi063.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T05:31:34.972 INFO:teuthology.orchestra.run.smithi063.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T05:31:34.972 INFO:teuthology.orchestra.run.smithi063.stdout:Packager : builder@centos.org 2024-05-08T05:31:34.972 INFO:teuthology.orchestra.run.smithi063.stdout:Vendor : CentOS 2024-05-08T05:31:34.972 INFO:teuthology.orchestra.run.smithi063.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T05:31:34.972 INFO:teuthology.orchestra.run.smithi063.stdout:Summary : Support tools for GRUB. 2024-05-08T05:31:34.972 INFO:teuthology.orchestra.run.smithi063.stdout:Description : 2024-05-08T05:31:34.972 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:31:34.972 INFO:teuthology.orchestra.run.smithi063.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T05:31:34.973 INFO:teuthology.orchestra.run.smithi063.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T05:31:34.973 INFO:teuthology.orchestra.run.smithi063.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T05:31:34.973 INFO:teuthology.orchestra.run.smithi063.stdout:hardware devices. 2024-05-08T05:31:34.973 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:31:34.973 INFO:teuthology.orchestra.run.smithi063.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T05:31:34.974 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T05:31:34.974 INFO:teuthology.task.kernel:Updating grub on smithi063 to boot 5.14.0-444.el9.x86_64 2024-05-08T05:31:34.974 DEBUG:teuthology.orchestra.run.smithi063:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T05:31:35.378 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:31:35.601 INFO:teuthology.orchestra.run.smithi063.stderr:Generating grub configuration file ... 2024-05-08T05:31:36.969 INFO:teuthology.orchestra.run.smithi063.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T05:31:37.000 INFO:teuthology.orchestra.run.smithi063.stderr:done 2024-05-08T05:31:37.009 DEBUG:teuthology.orchestra.run.smithi063:> mktemp 2024-05-08T05:31:37.024 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/tmp.vjVrIZLFG7 2024-05-08T05:31:37.025 DEBUG:teuthology.orchestra.run.smithi063:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.vjVrIZLFG7 2024-05-08T05:31:37.091 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0666 /tmp/tmp.vjVrIZLFG7 2024-05-08T05:31:37.284 DEBUG:teuthology.orchestra.remote:smithi063:/tmp/tmp.vjVrIZLFG7 is 6KB 2024-05-08T05:31:37.332 DEBUG:teuthology.orchestra.run.smithi063:> rm -fr /tmp/tmp.vjVrIZLFG7 2024-05-08T05:31:37.346 DEBUG:teuthology.orchestra.run.smithi063:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T05:31:37.409 INFO:teuthology.orchestra.run.smithi063.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T05:31:37.409 INFO:teuthology.orchestra.run.smithi063.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T05:31:37.409 INFO:teuthology.orchestra.run.smithi063.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T05:31:37.410 INFO:teuthology.orchestra.run.smithi063.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T05:31:37.410 DEBUG:teuthology.orchestra.run.smithi063:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T05:31:37.548 DEBUG:teuthology.orchestra.run.smithi063:> sudo shutdown -r now 2024-05-08T05:32:06.348 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:32:06.387 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:32:06.740 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:32:06.740 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T05:32:06.740 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T05:32:06.740 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T05:32:07.549 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:32:07.549 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:32:07.549 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-08T05:32:07.549 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T05:32:07.550 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T05:32:07.550 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T05:32:07.550 INFO:teuthology.orchestra.run.smithi045.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T05:32:07.550 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:32:07.550 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-08T05:32:07.551 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-05-08T05:32:07.552 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-08T05:32:07.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:32:07.878 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T05:32:07.956 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T05:32:07.956 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T05:32:07.957 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T05:32:07.957 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T05:32:07.957 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T05:32:07.957 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-08T05:32:07.957 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi045.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T05:32:07.957 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T05:32:08.007 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T05:32:08.008 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T05:32:08.008 DEBUG:teuthology.orchestra.run.smithi045:> sudo rpm -qi grub2-tools 2024-05-08T05:32:08.045 INFO:teuthology.orchestra.run.smithi045.stdout:Name : grub2-tools 2024-05-08T05:32:08.045 INFO:teuthology.orchestra.run.smithi045.stdout:Epoch : 1 2024-05-08T05:32:08.045 INFO:teuthology.orchestra.run.smithi045.stdout:Version : 2.06 2024-05-08T05:32:08.045 INFO:teuthology.orchestra.run.smithi045.stdout:Release : 68.el9 2024-05-08T05:32:08.045 INFO:teuthology.orchestra.run.smithi045.stdout:Architecture: x86_64 2024-05-08T05:32:08.046 INFO:teuthology.orchestra.run.smithi045.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T05:32:08.046 INFO:teuthology.orchestra.run.smithi045.stdout:Group : Unspecified 2024-05-08T05:32:08.046 INFO:teuthology.orchestra.run.smithi045.stdout:Size : 8264254 2024-05-08T05:32:08.046 INFO:teuthology.orchestra.run.smithi045.stdout:License : GPLv3+ 2024-05-08T05:32:08.046 INFO:teuthology.orchestra.run.smithi045.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T05:32:08.046 INFO:teuthology.orchestra.run.smithi045.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T05:32:08.046 INFO:teuthology.orchestra.run.smithi045.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T05:32:08.046 INFO:teuthology.orchestra.run.smithi045.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T05:32:08.046 INFO:teuthology.orchestra.run.smithi045.stdout:Packager : builder@centos.org 2024-05-08T05:32:08.046 INFO:teuthology.orchestra.run.smithi045.stdout:Vendor : CentOS 2024-05-08T05:32:08.046 INFO:teuthology.orchestra.run.smithi045.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T05:32:08.046 INFO:teuthology.orchestra.run.smithi045.stdout:Summary : Support tools for GRUB. 2024-05-08T05:32:08.046 INFO:teuthology.orchestra.run.smithi045.stdout:Description : 2024-05-08T05:32:08.047 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:32:08.047 INFO:teuthology.orchestra.run.smithi045.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T05:32:08.047 INFO:teuthology.orchestra.run.smithi045.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T05:32:08.047 INFO:teuthology.orchestra.run.smithi045.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T05:32:08.047 INFO:teuthology.orchestra.run.smithi045.stdout:hardware devices. 2024-05-08T05:32:08.047 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:32:08.047 INFO:teuthology.orchestra.run.smithi045.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T05:32:08.048 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T05:32:08.048 INFO:teuthology.task.kernel:Updating grub on smithi045 to boot 5.14.0-444.el9.x86_64 2024-05-08T05:32:08.048 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T05:32:08.701 INFO:teuthology.orchestra.run.smithi045.stderr:Generating grub configuration file ... 2024-05-08T05:32:10.075 INFO:teuthology.orchestra.run.smithi045.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T05:32:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:done 2024-05-08T05:32:10.106 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-05-08T05:32:10.123 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.qWrwQaKD9J 2024-05-08T05:32:10.123 DEBUG:teuthology.orchestra.run.smithi045:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.qWrwQaKD9J 2024-05-08T05:32:10.189 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.qWrwQaKD9J 2024-05-08T05:32:10.326 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.qWrwQaKD9J is 6KB 2024-05-08T05:32:10.374 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.qWrwQaKD9J 2024-05-08T05:32:10.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T05:32:10.451 INFO:teuthology.orchestra.run.smithi045.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T05:32:10.451 INFO:teuthology.orchestra.run.smithi045.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T05:32:10.451 INFO:teuthology.orchestra.run.smithi045.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T05:32:10.452 INFO:teuthology.orchestra.run.smithi045.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T05:32:10.453 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T05:32:10.621 DEBUG:teuthology.orchestra.run.smithi045:> sudo shutdown -r now 2024-05-08T05:32:26.018 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-05-08T05:32:35.026 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-08T05:32:35.027 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:32:38.082 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-05-08T05:32:40.626 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-08T05:32:40.626 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-05-08T05:32:40.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:32:50.086 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-08T05:32:50.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:32:59.038 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-05-08T05:33:08.046 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-05-08T05:33:08.047 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:33:08.510 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-05-08T05:33:11.102 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-05-08T05:33:23.117 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-05-08T05:33:23.118 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:33:23.511 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-08T05:33:23.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:33:23.875 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-05-08T05:33:24.256 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-08T05:33:24.256 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-444.el9.x86_64"... 2024-05-08T05:33:24.256 DEBUG:teuthology.orchestra.run.smithi063:> uname -r 2024-05-08T05:33:24.271 INFO:teuthology.orchestra.run.smithi063.stdout:5.14.0-444.el9.x86_64 2024-05-08T05:33:24.271 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T05:33:24.272 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T05:33:24.272 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T05:33:25.272 DEBUG:teuthology.parallel:result is None 2024-05-08T05:33:41.538 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-05-08T05:33:56.540 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-05-08T05:33:56.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:33:56.910 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-05-08T05:33:57.275 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-05-08T05:33:57.275 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-444.el9.x86_64"... 2024-05-08T05:33:57.275 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-05-08T05:33:57.292 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-444.el9.x86_64 2024-05-08T05:33:57.292 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T05:33:57.292 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T05:33:57.292 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T05:33:58.293 DEBUG:teuthology.parallel:result is None 2024-05-08T05:33:58.293 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T05:33:58.301 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T05:33:58.301 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T05:33:58.304 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T05:33:58.325 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T05:33:58.335 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T05:33:58.343 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T05:33:58.344 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T05:33:58.364 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T05:33:58.400 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T05:33:58.407 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T05:33:58.407 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T05:33:58.432 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T05:33:58.506 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:33:58.532 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:33:58.548 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:33:58.569 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:33:58.571 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T05:33:58.578 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T05:33:58.579 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T05:33:58.582 DEBUG:teuthology.orchestra.run.smithi063:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T05:33:58.612 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T05:33:58.621 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T05:33:58.622 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T05:33:58.653 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T05:33:58.671 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:33:58.754 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:33:58.797 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:33:58.880 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:33:58.925 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:33:58.925 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T05:33:58.992 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:33:59.043 DEBUG:teuthology.orchestra.run.smithi063:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:33:59.094 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:33:59.180 DEBUG:teuthology.orchestra.run.smithi063:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:33:59.225 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:33:59.225 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T05:33:59.294 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2024-05-08T05:33:59.297 DEBUG:teuthology.orchestra.run.smithi063:> sudo service rsyslog restart 2024-05-08T05:33:59.334 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T05:33:59.393 INFO:teuthology.orchestra.run.smithi063.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T05:33:59.859 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T05:33:59.868 INFO:teuthology.task.internal:Starting timer... 2024-05-08T05:33:59.868 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T05:33:59.878 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T05:33:59.935 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-08T05:33:59.936 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2024-05-08T05:34:00.005 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2024-05-08T05:34:00.008 DEBUG:teuthology.orchestra.run.smithi063:> sudo service auditd rotate 2024-05-08T05:34:00.076 INFO:teuthology.orchestra.run.smithi063.stdout:Rotating logs: 2024-05-08T05:34:00.078 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T05:34:00.078 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2024-05-08T05:34:00.104 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2024-05-08T05:34:00.104 DEBUG:teuthology.orchestra.run.smithi063:> /usr/sbin/getenforce 2024-05-08T05:34:00.124 INFO:teuthology.orchestra.run.smithi063.stdout:Permissive 2024-05-08T05:34:00.124 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi063.front.sepia.ceph.com': 'permissive'} 2024-05-08T05:34:00.125 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T05:34:00.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:34:00.172 DEBUG:teuthology.orchestra.run.smithi063:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T05:34:00.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:34:00.203 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T05:34:00.203 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2024-05-08T05:34:00.240 DEBUG:teuthology.orchestra.run.smithi063:> sudo /usr/sbin/setenforce permissive 2024-05-08T05:34:00.270 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T05:34:00.280 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-08T05:34:00.286 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-08T05:34:00.419 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T05:34:00.434 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T05:34:00.435 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi045.front.sepia.ceph.com,smithi063.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T05:38:10.581 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi063.front.sepia.ceph.com')] 2024-05-08T05:38:10.582 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-05-08T05:38:10.584 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:38:10.662 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-05-08T05:38:10.738 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-05-08T05:38:10.738 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-08T05:38:10.739 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:38:10.814 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-05-08T05:38:10.897 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-08T05:38:10.897 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T05:38:10.919 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T05:38:10.920 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T05:38:10.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T05:38:10.923 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T05:38:10.923 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T05:38:10.952 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T05:38:10.967 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T05:38:10.972 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T05:38:10.988 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T05:38:10.993 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2024-05-08T05:38:11.005 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2024-05-08T05:38:11.014 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: ntpd: command not found 2024-05-08T05:38:11.020 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T05:38:11.026 INFO:teuthology.orchestra.run.smithi063.stdout:506 Cannot talk to daemon 2024-05-08T05:38:11.035 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T05:38:11.040 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T05:38:11.055 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T05:38:11.090 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-05-08T05:38:11.094 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T05:38:11.094 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-05-08T05:38:11.094 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:38:11.094 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:38:11.094 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:38:11.094 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:38:11.121 INFO:teuthology.orchestra.run.smithi063.stderr:bash: line 1: ntpq: command not found 2024-05-08T05:38:11.124 INFO:teuthology.orchestra.run.smithi063.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T05:38:11.124 INFO:teuthology.orchestra.run.smithi063.stdout:=============================================================================== 2024-05-08T05:38:11.124 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:38:11.124 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:38:11.124 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:38:11.124 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:38:11.125 INFO:teuthology.run_tasks:Running task install... 2024-05-08T05:38:11.134 DEBUG:teuthology.task.install:project ceph 2024-05-08T05:38:11.134 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c'}} 2024-05-08T05:38:11.134 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c'} 2024-05-08T05:38:11.134 INFO:teuthology.task.install:Using flavor: default 2024-05-08T05:38:11.140 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T05:38:11.140 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-08T05:38:11.140 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c', 'tag': None, 'wait_for_package': False} 2024-05-08T05:38:11.141 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=cd693b0b93278c2b128be2d6d93ae4df18c3ba1c 2024-05-08T05:38:11.143 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c', 'tag': None, 'wait_for_package': False} 2024-05-08T05:38:11.143 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=cd693b0b93278c2b128be2d6d93ae4df18c3ba1c 2024-05-08T05:38:11.377 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing-07052024/cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/centos/9/flavors/default/ 2024-05-08T05:38:11.377 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3538.gcd693b0b 2024-05-08T05:38:11.381 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing-07052024/cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/centos/9/flavors/default/ 2024-05-08T05:38:11.381 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3538.gcd693b0b 2024-05-08T05:38:11.488 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing-07052024/cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing-07052024/cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing-07052024/cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T05:38:11.488 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:38:11.488 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T05:38:11.516 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T05:38:11.516 DEBUG:teuthology.orchestra.run.smithi063:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T05:38:11.585 DEBUG:teuthology.orchestra.run.smithi063:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T05:38:11.662 DEBUG:teuthology.orchestra.run.smithi063:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T05:38:11.728 INFO:teuthology.orchestra.run.smithi063.stdout:check_obsoletes = 1 2024-05-08T05:38:11.729 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum clean all 2024-05-08T05:38:11.839 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing-07052024/cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing-07052024/cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-jcollin-testing-07052024/cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T05:38:11.839 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:38:11.839 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T05:38:11.871 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T05:38:11.871 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T05:38:11.943 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T05:38:11.968 INFO:teuthology.orchestra.run.smithi063.stdout:54 files removed 2024-05-08T05:38:11.996 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T05:38:12.025 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T05:38:12.093 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2024-05-08T05:38:12.095 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2024-05-08T05:38:12.342 INFO:teuthology.orchestra.run.smithi045.stdout:54 files removed 2024-05-08T05:38:12.370 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T05:38:12.697 INFO:teuthology.orchestra.run.smithi063.stdout:ceph packages for x86_64 203 kB/s | 84 kB 00:00 2024-05-08T05:38:13.105 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 189 kB/s | 84 kB 00:00 2024-05-08T05:38:13.139 INFO:teuthology.orchestra.run.smithi063.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-05-08T05:38:13.547 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-05-08T05:38:13.581 INFO:teuthology.orchestra.run.smithi063.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-05-08T05:38:14.140 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 3.5 kB/s | 1.9 kB 00:00 2024-05-08T05:38:14.315 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-05-08T05:38:15.106 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 8.7 MB/s | 8.1 MB 00:00 2024-05-08T05:38:19.158 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 6.5 MB/s | 19 MB 00:02 2024-05-08T05:38:21.082 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - AppStream 3.4 MB/s | 19 MB 00:05 2024-05-08T05:38:26.175 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.2 MB 00:01 2024-05-08T05:38:26.474 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 1.7 MB/s | 6.2 MB 00:03 2024-05-08T05:38:27.826 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-05-08T05:38:28.183 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-05-08T05:38:29.161 INFO:teuthology.orchestra.run.smithi063.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-05-08T05:38:31.387 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 6.9 MB/s | 21 MB 00:03 2024-05-08T05:38:34.494 INFO:teuthology.orchestra.run.smithi063.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-05-08T05:38:36.701 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-05-08T05:38:36.906 INFO:teuthology.orchestra.run.smithi063.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:38:36.906 INFO:teuthology.orchestra.run.smithi063.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:38:36.919 INFO:teuthology.orchestra.run.smithi063.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-08T05:38:36.920 INFO:teuthology.orchestra.run.smithi063.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-08T05:38:36.920 INFO:teuthology.orchestra.run.smithi063.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-08T05:38:36.921 INFO:teuthology.orchestra.run.smithi063.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-08T05:38:36.922 INFO:teuthology.orchestra.run.smithi063.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-08T05:38:36.924 INFO:teuthology.orchestra.run.smithi063.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-08T05:38:36.925 INFO:teuthology.orchestra.run.smithi063.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-08T05:38:36.992 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-05-08T05:38:37.013 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-05-08T05:38:37.013 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-05-08T05:38:37.013 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-05-08T05:38:37.013 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-05-08T05:38:37.013 INFO:teuthology.orchestra.run.smithi063.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-08T05:38:37.013 INFO:teuthology.orchestra.run.smithi063.stdout: ceph x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 6.6 k 2024-05-08T05:38:37.014 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 5.4 M 2024-05-08T05:38:37.014 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-fuse x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 891 k 2024-05-08T05:38:37.014 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 145 k 2024-05-08T05:38:37.014 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 1.5 M 2024-05-08T05:38:37.014 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 142 k 2024-05-08T05:38:37.014 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 3.5 M 2024-05-08T05:38:37.014 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 7.4 M 2024-05-08T05:38:37.014 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-rook noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 49 k 2024-05-08T05:38:37.014 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-radosgw x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 11 M 2024-05-08T05:38:37.014 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-test x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 49 M 2024-05-08T05:38:37.014 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-volume noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 273 k 2024-05-08T05:38:37.015 INFO:teuthology.orchestra.run.smithi063.stdout: cephadm noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 764 k 2024-05-08T05:38:37.015 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-mirror x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 225 k 2024-05-08T05:38:37.015 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-shell x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 27 k 2024-05-08T05:38:37.015 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-top noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 18 k 2024-05-08T05:38:37.015 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-08T05:38:37.015 INFO:teuthology.orchestra.run.smithi063.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-08T05:38:37.015 INFO:teuthology.orchestra.run.smithi063.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-08T05:38:37.015 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs-devel x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 32 k 2024-05-08T05:38:37.015 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs2 x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 746 k 2024-05-08T05:38:37.015 INFO:teuthology.orchestra.run.smithi063.stdout: librados-devel x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 127 k 2024-05-08T05:38:37.015 INFO:teuthology.orchestra.run.smithi063.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-08T05:38:37.015 INFO:teuthology.orchestra.run.smithi063.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-08T05:38:37.015 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-08T05:38:37.016 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-08T05:38:37.016 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-08T05:38:37.016 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-08T05:38:37.016 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cephfs x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 161 k 2024-05-08T05:38:37.016 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rados x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 320 k 2024-05-08T05:38:37.016 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rbd x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 301 k 2024-05-08T05:38:37.016 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rgw x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 99 k 2024-05-08T05:38:37.016 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-fuse x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 86 k 2024-05-08T05:38:37.016 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-mirror x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 3.1 M 2024-05-08T05:38:37.016 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-nbd x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 171 k 2024-05-08T05:38:37.016 INFO:teuthology.orchestra.run.smithi063.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-08T05:38:37.017 INFO:teuthology.orchestra.run.smithi063.stdout:Upgrading: 2024-05-08T05:38:37.017 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-08T05:38:37.017 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-08T05:38:37.017 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-08T05:38:37.017 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-08T05:38:37.017 INFO:teuthology.orchestra.run.smithi063.stdout: librados2 x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 3.4 M 2024-05-08T05:38:37.017 INFO:teuthology.orchestra.run.smithi063.stdout: librbd1 x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 3.1 M 2024-05-08T05:38:37.017 INFO:teuthology.orchestra.run.smithi063.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-08T05:38:37.017 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-08T05:38:37.017 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-05-08T05:38:37.017 INFO:teuthology.orchestra.run.smithi063.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T05:38:37.017 INFO:teuthology.orchestra.run.smithi063.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-08T05:38:37.018 INFO:teuthology.orchestra.run.smithi063.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T05:38:37.018 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-common x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 21 M 2024-05-08T05:38:37.018 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 34 k 2024-05-08T05:38:37.018 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mds x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 2.4 M 2024-05-08T05:38:37.018 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 274 k 2024-05-08T05:38:37.018 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mon x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 4.7 M 2024-05-08T05:38:37.018 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-osd x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 17 M 2024-05-08T05:38:37.018 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 16 k 2024-05-08T05:38:37.018 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-selinux x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 25 k 2024-05-08T05:38:37.019 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T05:38:37.019 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T05:38:37.019 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T05:38:37.019 INFO:teuthology.orchestra.run.smithi063.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T05:38:37.019 INFO:teuthology.orchestra.run.smithi063.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T05:38:37.019 INFO:teuthology.orchestra.run.smithi063.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T05:38:37.019 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T05:38:37.019 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T05:38:37.019 INFO:teuthology.orchestra.run.smithi063.stdout: libcephsqlite x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 163 k 2024-05-08T05:38:37.019 INFO:teuthology.orchestra.run.smithi063.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T05:38:37.019 INFO:teuthology.orchestra.run.smithi063.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T05:38:37.019 INFO:teuthology.orchestra.run.smithi063.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T05:38:37.020 INFO:teuthology.orchestra.run.smithi063.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T05:38:37.020 INFO:teuthology.orchestra.run.smithi063.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T05:38:37.020 INFO:teuthology.orchestra.run.smithi063.stdout: libradosstriper1 x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 500 k 2024-05-08T05:38:37.020 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T05:38:37.020 INFO:teuthology.orchestra.run.smithi063.stdout: librgw2 x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 5.4 M 2024-05-08T05:38:37.020 INFO:teuthology.orchestra.run.smithi063.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T05:38:37.020 INFO:teuthology.orchestra.run.smithi063.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-08T05:38:37.020 INFO:teuthology.orchestra.run.smithi063.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T05:38:37.020 INFO:teuthology.orchestra.run.smithi063.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T05:38:37.020 INFO:teuthology.orchestra.run.smithi063.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T05:38:37.020 INFO:teuthology.orchestra.run.smithi063.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T05:38:37.020 INFO:teuthology.orchestra.run.smithi063.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T05:38:37.020 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T05:38:37.021 INFO:teuthology.orchestra.run.smithi063.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T05:38:37.021 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-08T05:38:37.021 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T05:38:37.021 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T05:38:37.021 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T05:38:37.021 INFO:teuthology.orchestra.run.smithi063.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-08T05:38:37.021 INFO:teuthology.orchestra.run.smithi063.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T05:38:37.021 INFO:teuthology.orchestra.run.smithi063.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T05:38:37.021 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T05:38:37.021 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-argparse x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 45 k 2024-05-08T05:38:37.021 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-common x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 135 k 2024-05-08T05:38:37.021 INFO:teuthology.orchestra.run.smithi063.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T05:38:37.022 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T05:38:37.022 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T05:38:37.022 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T05:38:37.022 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-08T05:38:37.022 INFO:teuthology.orchestra.run.smithi063.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-08T05:38:37.022 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T05:38:37.022 INFO:teuthology.orchestra.run.smithi063.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T05:38:37.022 INFO:teuthology.orchestra.run.smithi063.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T05:38:37.022 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T05:38:37.022 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T05:38:37.022 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T05:38:37.022 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T05:38:37.023 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T05:38:37.023 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T05:38:37.023 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T05:38:37.023 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T05:38:37.023 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T05:38:37.023 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T05:38:37.023 INFO:teuthology.orchestra.run.smithi063.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T05:38:37.023 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T05:38:37.023 INFO:teuthology.orchestra.run.smithi063.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T05:38:37.023 INFO:teuthology.orchestra.run.smithi063.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T05:38:37.023 INFO:teuthology.orchestra.run.smithi063.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T05:38:37.023 INFO:teuthology.orchestra.run.smithi063.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T05:38:37.024 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T05:38:37.024 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T05:38:37.024 INFO:teuthology.orchestra.run.smithi063.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T05:38:37.024 INFO:teuthology.orchestra.run.smithi063.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T05:38:37.024 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T05:38:37.024 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T05:38:37.024 INFO:teuthology.orchestra.run.smithi063.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T05:38:37.024 INFO:teuthology.orchestra.run.smithi063.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T05:38:37.024 INFO:teuthology.orchestra.run.smithi063.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T05:38:37.024 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T05:38:37.024 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T05:38:37.024 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T05:38:37.025 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T05:38:37.025 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-08T05:38:37.025 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T05:38:37.025 INFO:teuthology.orchestra.run.smithi063.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T05:38:37.025 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T05:38:37.025 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T05:38:37.025 INFO:teuthology.orchestra.run.smithi063.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T05:38:37.025 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T05:38:37.025 INFO:teuthology.orchestra.run.smithi063.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T05:38:37.025 INFO:teuthology.orchestra.run.smithi063.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T05:38:37.025 INFO:teuthology.orchestra.run.smithi063.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T05:38:37.025 INFO:teuthology.orchestra.run.smithi063.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T05:38:37.025 INFO:teuthology.orchestra.run.smithi063.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T05:38:37.026 INFO:teuthology.orchestra.run.smithi063.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-08T05:38:37.026 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T05:38:37.026 INFO:teuthology.orchestra.run.smithi063.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T05:38:37.026 INFO:teuthology.orchestra.run.smithi063.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T05:38:37.026 INFO:teuthology.orchestra.run.smithi063.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T05:38:37.026 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T05:38:37.026 INFO:teuthology.orchestra.run.smithi063.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T05:38:37.026 INFO:teuthology.orchestra.run.smithi063.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T05:38:37.026 INFO:teuthology.orchestra.run.smithi063.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T05:38:37.026 INFO:teuthology.orchestra.run.smithi063.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T05:38:37.027 INFO:teuthology.orchestra.run.smithi063.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-08T05:38:37.027 INFO:teuthology.orchestra.run.smithi063.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T05:38:37.027 INFO:teuthology.orchestra.run.smithi063.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-08T05:38:37.027 INFO:teuthology.orchestra.run.smithi063.stdout:Installing weak dependencies: 2024-05-08T05:38:37.027 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T05:38:37.027 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:38:37.027 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-05-08T05:38:37.027 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-05-08T05:38:37.027 INFO:teuthology.orchestra.run.smithi063.stdout:Install 148 Packages 2024-05-08T05:38:37.027 INFO:teuthology.orchestra.run.smithi063.stdout:Upgrade 8 Packages 2024-05-08T05:38:37.027 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:38:37.028 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 226 M 2024-05-08T05:38:37.028 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-05-08T05:38:38.212 INFO:teuthology.orchestra.run.smithi063.stdout:(1/156): ceph-19.0.0-3538.gcd693b0b.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-05-08T05:38:38.946 INFO:teuthology.orchestra.run.smithi063.stdout:(2/156): ceph-fuse-19.0.0-3538.gcd693b0b.el9.x8 1.2 MB/s | 891 kB 00:00 2024-05-08T05:38:39.063 INFO:teuthology.orchestra.run.smithi063.stdout:(3/156): ceph-immutable-object-cache-19.0.0-353 1.2 MB/s | 145 kB 00:00 2024-05-08T05:38:39.129 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:38:39.130 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:38:39.141 INFO:teuthology.orchestra.run.smithi045.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-08T05:38:39.142 INFO:teuthology.orchestra.run.smithi045.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-08T05:38:39.143 INFO:teuthology.orchestra.run.smithi045.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-08T05:38:39.144 INFO:teuthology.orchestra.run.smithi045.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-08T05:38:39.145 INFO:teuthology.orchestra.run.smithi045.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-08T05:38:39.148 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-08T05:38:39.149 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-08T05:38:39.217 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-08T05:38:39.237 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-05-08T05:38:39.238 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-05-08T05:38:39.238 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-05-08T05:38:39.238 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-08T05:38:39.238 INFO:teuthology.orchestra.run.smithi045.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-08T05:38:39.238 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 6.6 k 2024-05-08T05:38:39.238 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 5.4 M 2024-05-08T05:38:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 891 k 2024-05-08T05:38:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 145 k 2024-05-08T05:38:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 1.5 M 2024-05-08T05:38:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 142 k 2024-05-08T05:38:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 3.5 M 2024-05-08T05:38:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 7.4 M 2024-05-08T05:38:39.239 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 49 k 2024-05-08T05:38:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 11 M 2024-05-08T05:38:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 49 M 2024-05-08T05:38:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 273 k 2024-05-08T05:38:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 764 k 2024-05-08T05:38:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 225 k 2024-05-08T05:38:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 27 k 2024-05-08T05:38:39.240 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 18 k 2024-05-08T05:38:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-08T05:38:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-08T05:38:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-08T05:38:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 32 k 2024-05-08T05:38:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 746 k 2024-05-08T05:38:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 127 k 2024-05-08T05:38:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-08T05:38:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-08T05:38:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-08T05:38:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-08T05:38:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-08T05:38:39.241 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-08T05:38:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 161 k 2024-05-08T05:38:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 320 k 2024-05-08T05:38:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 301 k 2024-05-08T05:38:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 99 k 2024-05-08T05:38:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 86 k 2024-05-08T05:38:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 3.1 M 2024-05-08T05:38:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 171 k 2024-05-08T05:38:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-08T05:38:39.242 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2024-05-08T05:38:39.242 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-08T05:38:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-08T05:38:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-08T05:38:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-08T05:38:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 3.4 M 2024-05-08T05:38:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 3.1 M 2024-05-08T05:38:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-08T05:38:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-08T05:38:39.243 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-05-08T05:38:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T05:38:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-08T05:38:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T05:38:39.243 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 21 M 2024-05-08T05:38:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 34 k 2024-05-08T05:38:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 2.4 M 2024-05-08T05:38:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 274 k 2024-05-08T05:38:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 4.7 M 2024-05-08T05:38:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 17 M 2024-05-08T05:38:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3538.gcd693b0b.el9 ceph-noarch 16 k 2024-05-08T05:38:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 25 k 2024-05-08T05:38:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T05:38:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T05:38:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T05:38:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T05:38:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T05:38:39.244 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T05:38:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T05:38:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T05:38:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 163 k 2024-05-08T05:38:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T05:38:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T05:38:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T05:38:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T05:38:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T05:38:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 500 k 2024-05-08T05:38:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T05:38:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 5.4 M 2024-05-08T05:38:39.245 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T05:38:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-08T05:38:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T05:38:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T05:38:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T05:38:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T05:38:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T05:38:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T05:38:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T05:38:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-08T05:38:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T05:38:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T05:38:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T05:38:39.246 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-08T05:38:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T05:38:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T05:38:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T05:38:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 45 k 2024-05-08T05:38:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:19.0.0-3538.gcd693b0b.el9 ceph 135 k 2024-05-08T05:38:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T05:38:39.247 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T05:38:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T05:38:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T05:38:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-08T05:38:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-08T05:38:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T05:38:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T05:38:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T05:38:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T05:38:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T05:38:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T05:38:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T05:38:39.248 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T05:38:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T05:38:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T05:38:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T05:38:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T05:38:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T05:38:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T05:38:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T05:38:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T05:38:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T05:38:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T05:38:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T05:38:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T05:38:39.249 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T05:38:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T05:38:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T05:38:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T05:38:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T05:38:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T05:38:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T05:38:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T05:38:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T05:38:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T05:38:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T05:38:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T05:38:39.250 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-08T05:38:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T05:38:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T05:38:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T05:38:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T05:38:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T05:38:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T05:38:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T05:38:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T05:38:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T05:38:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T05:38:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T05:38:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-08T05:38:39.251 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T05:38:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T05:38:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T05:38:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T05:38:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T05:38:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T05:38:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T05:38:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T05:38:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T05:38:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-08T05:38:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T05:38:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-08T05:38:39.252 INFO:teuthology.orchestra.run.smithi045.stdout:Installing weak dependencies: 2024-05-08T05:38:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T05:38:39.252 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:38:39.252 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-08T05:38:39.253 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-05-08T05:38:39.253 INFO:teuthology.orchestra.run.smithi045.stdout:Install 148 Packages 2024-05-08T05:38:39.253 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 8 Packages 2024-05-08T05:38:39.253 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:38:39.253 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 226 M 2024-05-08T05:38:39.254 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-08T05:38:40.113 INFO:teuthology.orchestra.run.smithi063.stdout:(4/156): ceph-mds-19.0.0-3538.gcd693b0b.el9.x86 2.3 MB/s | 2.4 MB 00:01 2024-05-08T05:38:40.186 INFO:teuthology.orchestra.run.smithi045.stdout:(1/156): ceph-19.0.0-3538.gcd693b0b.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-05-08T05:38:40.713 INFO:teuthology.orchestra.run.smithi063.stdout:(5/156): ceph-mgr-19.0.0-3538.gcd693b0b.el9.x86 2.5 MB/s | 1.5 MB 00:00 2024-05-08T05:38:41.419 INFO:teuthology.orchestra.run.smithi045.stdout:(2/156): ceph-fuse-19.0.0-3538.gcd693b0b.el9.x8 722 kB/s | 891 kB 00:01 2024-05-08T05:38:41.711 INFO:teuthology.orchestra.run.smithi045.stdout:(3/156): ceph-immutable-object-cache-19.0.0-353 496 kB/s | 145 kB 00:00 2024-05-08T05:38:42.486 INFO:teuthology.orchestra.run.smithi045.stdout:(4/156): ceph-base-19.0.0-3538.gcd693b0b.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-05-08T05:38:42.797 INFO:teuthology.orchestra.run.smithi063.stdout:(6/156): ceph-base-19.0.0-3538.gcd693b0b.el9.x8 1.1 MB/s | 5.4 MB 00:04 2024-05-08T05:38:42.953 INFO:teuthology.orchestra.run.smithi045.stdout:(5/156): ceph-mgr-19.0.0-3538.gcd693b0b.el9.x86 3.2 MB/s | 1.5 MB 00:00 2024-05-08T05:38:43.072 INFO:teuthology.orchestra.run.smithi063.stdout:(7/156): ceph-mon-19.0.0-3538.gcd693b0b.el9.x86 2.0 MB/s | 4.7 MB 00:02 2024-05-08T05:38:44.170 INFO:teuthology.orchestra.run.smithi045.stdout:(6/156): ceph-mds-19.0.0-3538.gcd693b0b.el9.x86 984 kB/s | 2.4 MB 00:02 2024-05-08T05:38:44.521 INFO:teuthology.orchestra.run.smithi045.stdout:(7/156): ceph-mon-19.0.0-3538.gcd693b0b.el9.x86 3.0 MB/s | 4.7 MB 00:01 2024-05-08T05:38:46.264 INFO:teuthology.orchestra.run.smithi063.stdout:(8/156): ceph-common-19.0.0-3538.gcd693b0b.el9. 2.6 MB/s | 21 MB 00:08 2024-05-08T05:38:46.372 INFO:teuthology.orchestra.run.smithi063.stdout:(9/156): ceph-selinux-19.0.0-3538.gcd693b0b.el9 232 kB/s | 25 kB 00:00 2024-05-08T05:38:47.562 INFO:teuthology.orchestra.run.smithi045.stdout:(8/156): ceph-radosgw-19.0.0-3538.gcd693b0b.el9 3.5 MB/s | 11 MB 00:03 2024-05-08T05:38:47.654 INFO:teuthology.orchestra.run.smithi045.stdout:(9/156): ceph-selinux-19.0.0-3538.gcd693b0b.el9 275 kB/s | 25 kB 00:00 2024-05-08T05:38:48.264 INFO:teuthology.orchestra.run.smithi063.stdout:(10/156): ceph-radosgw-19.0.0-3538.gcd693b0b.el 2.0 MB/s | 11 MB 00:05 2024-05-08T05:38:48.364 INFO:teuthology.orchestra.run.smithi063.stdout:(11/156): cephfs-mirror-19.0.0-3538.gcd693b0b.e 2.2 MB/s | 225 kB 00:00 2024-05-08T05:38:48.423 INFO:teuthology.orchestra.run.smithi063.stdout:(12/156): cephfs-shell-19.0.0-3538.gcd693b0b.el 464 kB/s | 27 kB 00:00 2024-05-08T05:38:48.481 INFO:teuthology.orchestra.run.smithi063.stdout:(13/156): libcephfs-devel-19.0.0-3538.gcd693b0b 544 kB/s | 32 kB 00:00 2024-05-08T05:38:49.132 INFO:teuthology.orchestra.run.smithi063.stdout:(14/156): libcephfs2-19.0.0-3538.gcd693b0b.el9. 1.1 MB/s | 746 kB 00:00 2024-05-08T05:38:49.265 INFO:teuthology.orchestra.run.smithi063.stdout:(15/156): libcephsqlite-19.0.0-3538.gcd693b0b.e 1.2 MB/s | 163 kB 00:00 2024-05-08T05:38:49.357 INFO:teuthology.orchestra.run.smithi063.stdout:(16/156): librados-devel-19.0.0-3538.gcd693b0b. 1.4 MB/s | 127 kB 00:00 2024-05-08T05:38:49.650 INFO:teuthology.orchestra.run.smithi063.stdout:(17/156): libradosstriper1-19.0.0-3538.gcd693b0 1.7 MB/s | 500 kB 00:00 2024-05-08T05:38:52.050 INFO:teuthology.orchestra.run.smithi063.stdout:(18/156): librgw2-19.0.0-3538.gcd693b0b.el9.x86 2.2 MB/s | 5.4 MB 00:02 2024-05-08T05:38:52.134 INFO:teuthology.orchestra.run.smithi063.stdout:(19/156): python3-ceph-argparse-19.0.0-3538.gcd 543 kB/s | 45 kB 00:00 2024-05-08T05:38:52.221 INFO:teuthology.orchestra.run.smithi063.stdout:(20/156): python3-ceph-common-19.0.0-3538.gcd69 1.6 MB/s | 135 kB 00:00 2024-05-08T05:38:52.352 INFO:teuthology.orchestra.run.smithi063.stdout:(21/156): python3-cephfs-19.0.0-3538.gcd693b0b. 1.2 MB/s | 161 kB 00:00 2024-05-08T05:38:52.559 INFO:teuthology.orchestra.run.smithi063.stdout:(22/156): python3-rados-19.0.0-3538.gcd693b0b.e 1.5 MB/s | 320 kB 00:00 2024-05-08T05:38:52.710 INFO:teuthology.orchestra.run.smithi063.stdout:(23/156): python3-rbd-19.0.0-3538.gcd693b0b.el9 2.0 MB/s | 301 kB 00:00 2024-05-08T05:38:52.777 INFO:teuthology.orchestra.run.smithi063.stdout:(24/156): python3-rgw-19.0.0-3538.gcd693b0b.el9 1.5 MB/s | 99 kB 00:00 2024-05-08T05:38:52.787 INFO:teuthology.orchestra.run.smithi045.stdout:(10/156): ceph-osd-19.0.0-3538.gcd693b0b.el9.x8 2.0 MB/s | 17 MB 00:08 2024-05-08T05:38:52.844 INFO:teuthology.orchestra.run.smithi063.stdout:(25/156): rbd-fuse-19.0.0-3538.gcd693b0b.el9.x8 1.3 MB/s | 86 kB 00:00 2024-05-08T05:38:52.887 INFO:teuthology.orchestra.run.smithi045.stdout:(11/156): cephfs-mirror-19.0.0-3538.gcd693b0b.e 2.2 MB/s | 225 kB 00:00 2024-05-08T05:38:52.954 INFO:teuthology.orchestra.run.smithi045.stdout:(12/156): cephfs-shell-19.0.0-3538.gcd693b0b.el 406 kB/s | 27 kB 00:00 2024-05-08T05:38:53.010 INFO:teuthology.orchestra.run.smithi063.stdout:(26/156): ceph-osd-19.0.0-3538.gcd693b0b.el9.x8 1.6 MB/s | 17 MB 00:10 2024-05-08T05:38:53.021 INFO:teuthology.orchestra.run.smithi045.stdout:(13/156): libcephfs-devel-19.0.0-3538.gcd693b0b 476 kB/s | 32 kB 00:00 2024-05-08T05:38:53.119 INFO:teuthology.orchestra.run.smithi063.stdout:(27/156): rbd-nbd-19.0.0-3538.gcd693b0b.el9.x86 1.5 MB/s | 171 kB 00:00 2024-05-08T05:38:53.238 INFO:teuthology.orchestra.run.smithi045.stdout:(14/156): libcephfs2-19.0.0-3538.gcd693b0b.el9. 3.4 MB/s | 746 kB 00:00 2024-05-08T05:38:53.330 INFO:teuthology.orchestra.run.smithi045.stdout:(15/156): libcephsqlite-19.0.0-3538.gcd693b0b.e 1.7 MB/s | 163 kB 00:00 2024-05-08T05:38:53.352 INFO:teuthology.orchestra.run.smithi063.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3538.g 147 kB/s | 34 kB 00:00 2024-05-08T05:38:53.405 INFO:teuthology.orchestra.run.smithi045.stdout:(16/156): librados-devel-19.0.0-3538.gcd693b0b. 1.7 MB/s | 127 kB 00:00 2024-05-08T05:38:53.461 INFO:teuthology.orchestra.run.smithi063.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3538.gcd693b0 1.3 MB/s | 142 kB 00:00 2024-05-08T05:38:53.672 INFO:teuthology.orchestra.run.smithi045.stdout:(17/156): ceph-common-19.0.0-3538.gcd693b0b.el9 1.5 MB/s | 21 MB 00:13 2024-05-08T05:38:53.698 INFO:teuthology.orchestra.run.smithi045.stdout:(18/156): libradosstriper1-19.0.0-3538.gcd693b0 1.7 MB/s | 500 kB 00:00 2024-05-08T05:38:53.761 INFO:teuthology.orchestra.run.smithi063.stdout:(30/156): rbd-mirror-19.0.0-3538.gcd693b0b.el9. 3.4 MB/s | 3.1 MB 00:00 2024-05-08T05:38:53.781 INFO:teuthology.orchestra.run.smithi045.stdout:(19/156): python3-ceph-argparse-19.0.0-3538.gcd 542 kB/s | 45 kB 00:00 2024-05-08T05:38:53.873 INFO:teuthology.orchestra.run.smithi045.stdout:(20/156): python3-ceph-common-19.0.0-3538.gcd69 1.4 MB/s | 135 kB 00:00 2024-05-08T05:38:53.965 INFO:teuthology.orchestra.run.smithi045.stdout:(21/156): python3-cephfs-19.0.0-3538.gcd693b0b. 1.7 MB/s | 161 kB 00:00 2024-05-08T05:38:54.182 INFO:teuthology.orchestra.run.smithi045.stdout:(22/156): python3-rados-19.0.0-3538.gcd693b0b.e 1.4 MB/s | 320 kB 00:00 2024-05-08T05:38:54.299 INFO:teuthology.orchestra.run.smithi045.stdout:(23/156): python3-rbd-19.0.0-3538.gcd693b0b.el9 2.5 MB/s | 301 kB 00:00 2024-05-08T05:38:54.399 INFO:teuthology.orchestra.run.smithi045.stdout:(24/156): python3-rgw-19.0.0-3538.gcd693b0b.el9 996 kB/s | 99 kB 00:00 2024-05-08T05:38:54.499 INFO:teuthology.orchestra.run.smithi045.stdout:(25/156): rbd-fuse-19.0.0-3538.gcd693b0b.el9.x8 859 kB/s | 86 kB 00:00 2024-05-08T05:38:54.670 INFO:teuthology.orchestra.run.smithi063.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3538.gcd693 2.9 MB/s | 3.5 MB 00:01 2024-05-08T05:38:54.837 INFO:teuthology.orchestra.run.smithi063.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3538.gcd 1.6 MB/s | 274 kB 00:00 2024-05-08T05:38:54.979 INFO:teuthology.orchestra.run.smithi063.stdout:(33/156): ceph-mgr-rook-19.0.0-3538.gcd693b0b.e 348 kB/s | 49 kB 00:00 2024-05-08T05:38:55.063 INFO:teuthology.orchestra.run.smithi063.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3538.gc 197 kB/s | 16 kB 00:00 2024-05-08T05:38:55.230 INFO:teuthology.orchestra.run.smithi063.stdout:(35/156): ceph-volume-19.0.0-3538.gcd693b0b.el9 1.6 MB/s | 273 kB 00:00 2024-05-08T05:38:55.233 INFO:teuthology.orchestra.run.smithi045.stdout:(26/156): rbd-mirror-19.0.0-3538.gcd693b0b.el9. 4.2 MB/s | 3.1 MB 00:00 2024-05-08T05:38:55.342 INFO:teuthology.orchestra.run.smithi045.stdout:(27/156): rbd-nbd-19.0.0-3538.gcd693b0b.el9.x86 1.5 MB/s | 171 kB 00:00 2024-05-08T05:38:55.434 INFO:teuthology.orchestra.run.smithi045.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3538.g 374 kB/s | 34 kB 00:00 2024-05-08T05:38:55.542 INFO:teuthology.orchestra.run.smithi045.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3538.gcd693b0 1.3 MB/s | 142 kB 00:00 2024-05-08T05:38:55.580 INFO:teuthology.orchestra.run.smithi063.stdout:(36/156): cephadm-19.0.0-3538.gcd693b0b.el9.noa 2.1 MB/s | 764 kB 00:00 2024-05-08T05:38:55.689 INFO:teuthology.orchestra.run.smithi063.stdout:(37/156): cephfs-top-19.0.0-3538.gcd693b0b.el9. 169 kB/s | 18 kB 00:00 2024-05-08T05:38:55.701 INFO:teuthology.orchestra.run.smithi045.stdout:(30/156): librgw2-19.0.0-3538.gcd693b0b.el9.x86 2.6 MB/s | 5.4 MB 00:02 2024-05-08T05:38:55.764 INFO:teuthology.orchestra.run.smithi063.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 1.1 MB/s | 80 kB 00:00 2024-05-08T05:38:55.997 INFO:teuthology.orchestra.run.smithi063.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-05-08T05:38:56.073 INFO:teuthology.orchestra.run.smithi063.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-05-08T05:38:56.148 INFO:teuthology.orchestra.run.smithi063.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-05-08T05:38:56.248 INFO:teuthology.orchestra.run.smithi063.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-05-08T05:38:56.315 INFO:teuthology.orchestra.run.smithi063.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-05-08T05:38:56.484 INFO:teuthology.orchestra.run.smithi045.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3538.gcd693 3.8 MB/s | 3.5 MB 00:00 2024-05-08T05:38:56.568 INFO:teuthology.orchestra.run.smithi045.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3538.gcd 3.2 MB/s | 274 kB 00:00 2024-05-08T05:38:56.643 INFO:teuthology.orchestra.run.smithi045.stdout:(33/156): ceph-mgr-rook-19.0.0-3538.gcd693b0b.e 659 kB/s | 49 kB 00:00 2024-05-08T05:38:56.710 INFO:teuthology.orchestra.run.smithi045.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3538.gc 247 kB/s | 16 kB 00:00 2024-05-08T05:38:56.757 INFO:teuthology.orchestra.run.smithi063.stdout:(44/156): ceph-test-19.0.0-3538.gcd693b0b.el9.x 4.7 MB/s | 49 MB 00:10 2024-05-08T05:38:56.786 INFO:teuthology.orchestra.run.smithi045.stdout:(35/156): ceph-volume-19.0.0-3538.gcd693b0b.el9 3.6 MB/s | 273 kB 00:00 2024-05-08T05:38:56.849 INFO:teuthology.orchestra.run.smithi063.stdout:(45/156): ceph-mgr-diskprediction-local-19.0.0- 2.4 MB/s | 7.4 MB 00:03 2024-05-08T05:38:56.883 INFO:teuthology.orchestra.run.smithi063.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 238 kB/s | 135 kB 00:00 2024-05-08T05:38:56.925 INFO:teuthology.orchestra.run.smithi063.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 748 kB/s | 125 kB 00:00 2024-05-08T05:38:56.950 INFO:teuthology.orchestra.run.smithi063.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-05-08T05:38:56.978 INFO:teuthology.orchestra.run.smithi045.stdout:(36/156): cephadm-19.0.0-3538.gcd693b0b.el9.noa 3.9 MB/s | 764 kB 00:00 2024-05-08T05:38:57.000 INFO:teuthology.orchestra.run.smithi063.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 5.7 MB/s | 689 kB 00:00 2024-05-08T05:38:57.034 INFO:teuthology.orchestra.run.smithi063.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-05-08T05:38:57.044 INFO:teuthology.orchestra.run.smithi045.stdout:(37/156): cephfs-top-19.0.0-3538.gcd693b0b.el9. 275 kB/s | 18 kB 00:00 2024-05-08T05:38:57.067 INFO:teuthology.orchestra.run.smithi063.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 323 kB/s | 21 kB 00:00 2024-05-08T05:38:57.092 INFO:teuthology.orchestra.run.smithi063.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 5.5 MB/s | 943 kB 00:00 2024-05-08T05:38:57.126 INFO:teuthology.orchestra.run.smithi063.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-05-08T05:38:57.151 INFO:teuthology.orchestra.run.smithi063.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-05-08T05:38:57.185 INFO:teuthology.orchestra.run.smithi063.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-05-08T05:38:57.202 INFO:teuthology.orchestra.run.smithi063.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-05-08T05:38:57.252 INFO:teuthology.orchestra.run.smithi063.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 908 kB/s | 45 kB 00:00 2024-05-08T05:38:57.277 INFO:teuthology.orchestra.run.smithi063.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-05-08T05:38:57.336 INFO:teuthology.orchestra.run.smithi063.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-05-08T05:38:57.378 INFO:teuthology.orchestra.run.smithi063.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 613 kB/s | 25 kB 00:00 2024-05-08T05:38:57.403 INFO:teuthology.orchestra.run.smithi063.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-05-08T05:38:57.429 INFO:teuthology.orchestra.run.smithi063.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-05-08T05:38:57.454 INFO:teuthology.orchestra.run.smithi063.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-05-08T05:38:57.514 INFO:teuthology.orchestra.run.smithi063.stdout:(64/156): openblas-0.3.21-2.el9.x86_64.rpm 602 kB/s | 35 kB 00:00 2024-05-08T05:38:57.547 INFO:teuthology.orchestra.run.smithi063.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-05-08T05:38:57.572 INFO:teuthology.orchestra.run.smithi063.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.3 MB/s | 188 kB 00:00 2024-05-08T05:38:57.711 INFO:teuthology.orchestra.run.smithi063.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-08T05:38:57.895 INFO:teuthology.orchestra.run.smithi045.stdout:(38/156): ceph-test-19.0.0-3538.gcd693b0b.el9.x 4.8 MB/s | 49 MB 00:10 2024-05-08T05:38:57.950 INFO:teuthology.orchestra.run.smithi063.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 11 MB/s | 4.7 MB 00:00 2024-05-08T05:38:57.987 INFO:teuthology.orchestra.run.smithi045.stdout:(39/156): ceph-mgr-diskprediction-local-19.0.0- 3.2 MB/s | 7.4 MB 00:02 2024-05-08T05:38:58.031 INFO:teuthology.orchestra.run.smithi063.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.6 MB/s | 4.4 MB 00:00 2024-05-08T05:38:58.081 INFO:teuthology.orchestra.run.smithi063.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 2.4 MB/s | 911 kB 00:00 2024-05-08T05:38:58.115 INFO:teuthology.orchestra.run.smithi063.stdout:(71/156): postgresql-private-libs-13.14-1.el9.x 809 kB/s | 133 kB 00:00 2024-05-08T05:38:58.182 INFO:teuthology.orchestra.run.smithi063.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-08T05:38:58.232 INFO:teuthology.orchestra.run.smithi063.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-05-08T05:38:58.349 INFO:teuthology.orchestra.run.smithi063.stdout:(74/156): postgresql-server-13.14-1.el9.x86_64. 18 MB/s | 5.8 MB 00:00 2024-05-08T05:38:58.362 INFO:teuthology.orchestra.run.smithi045.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 61 kB/s | 80 kB 00:01 2024-05-08T05:38:58.382 INFO:teuthology.orchestra.run.smithi063.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.6 MB/s | 244 kB 00:00 2024-05-08T05:38:58.449 INFO:teuthology.orchestra.run.smithi063.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 385 kB/s | 26 kB 00:00 2024-05-08T05:38:58.475 INFO:teuthology.orchestra.run.smithi063.stdout:(77/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-05-08T05:38:58.558 INFO:teuthology.orchestra.run.smithi063.stdout:(78/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-05-08T05:38:58.586 INFO:teuthology.orchestra.run.smithi063.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 143 kB/s | 19 kB 00:00 2024-05-08T05:38:58.625 INFO:teuthology.orchestra.run.smithi063.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 520 kB/s | 35 kB 00:00 2024-05-08T05:38:58.642 INFO:teuthology.orchestra.run.smithi063.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-05-08T05:38:58.718 INFO:teuthology.orchestra.run.smithi063.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-05-08T05:38:58.743 INFO:teuthology.orchestra.run.smithi063.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-08T05:38:58.801 INFO:teuthology.orchestra.run.smithi063.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 925 kB/s | 77 kB 00:00 2024-05-08T05:38:58.827 INFO:teuthology.orchestra.run.smithi063.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 506 kB/s | 42 kB 00:00 2024-05-08T05:38:58.902 INFO:teuthology.orchestra.run.smithi063.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-05-08T05:38:58.944 INFO:teuthology.orchestra.run.smithi063.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-05-08T05:38:58.969 INFO:teuthology.orchestra.run.smithi063.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 1.6 MB/s | 269 kB 00:00 2024-05-08T05:38:59.019 INFO:teuthology.orchestra.run.smithi063.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-05-08T05:38:59.045 INFO:teuthology.orchestra.run.smithi063.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 507 kB/s | 51 kB 00:00 2024-05-08T05:38:59.062 INFO:teuthology.orchestra.run.smithi045.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 182 kB/s | 195 kB 00:01 2024-05-08T05:38:59.070 INFO:teuthology.orchestra.run.smithi063.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 534 kB/s | 54 kB 00:00 2024-05-08T05:38:59.112 INFO:teuthology.orchestra.run.smithi063.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 624 kB/s | 42 kB 00:00 2024-05-08T05:38:59.137 INFO:teuthology.orchestra.run.smithi063.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 661 kB/s | 44 kB 00:00 2024-05-08T05:38:59.188 INFO:teuthology.orchestra.run.smithi063.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-05-08T05:38:59.213 INFO:teuthology.orchestra.run.smithi063.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-05-08T05:38:59.237 INFO:teuthology.orchestra.run.smithi045.stdout:(42/156): libgfortran-11.4.1-3.el9.x86_64.rpm 599 kB/s | 803 kB 00:01 2024-05-08T05:38:59.255 INFO:teuthology.orchestra.run.smithi063.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 711 kB/s | 47 kB 00:00 2024-05-08T05:38:59.297 INFO:teuthology.orchestra.run.smithi063.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-05-08T05:38:59.313 INFO:teuthology.orchestra.run.smithi063.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 987 kB/s | 58 kB 00:00 2024-05-08T05:38:59.372 INFO:teuthology.orchestra.run.smithi063.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-05-08T05:38:59.397 INFO:teuthology.orchestra.run.smithi063.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 762 kB/s | 64 kB 00:00 2024-05-08T05:38:59.438 INFO:teuthology.orchestra.run.smithi045.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 235 kB/s | 253 kB 00:01 2024-05-08T05:38:59.448 INFO:teuthology.orchestra.run.smithi063.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 595 kB/s | 45 kB 00:00 2024-05-08T05:38:59.481 INFO:teuthology.orchestra.run.smithi063.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-05-08T05:38:59.673 INFO:teuthology.orchestra.run.smithi063.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.7 MB/s | 863 kB 00:00 2024-05-08T05:38:59.874 INFO:teuthology.orchestra.run.smithi063.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-05-08T05:39:00.378 INFO:teuthology.orchestra.run.smithi063.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 41 kB/s | 19 kB 00:00 2024-05-08T05:39:00.446 INFO:teuthology.orchestra.run.smithi045.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 907 kB/s | 1.2 MB 00:01 2024-05-08T05:39:00.513 INFO:teuthology.orchestra.run.smithi045.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 83 kB/s | 106 kB 00:01 2024-05-08T05:39:00.666 INFO:teuthology.orchestra.run.smithi063.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 312 kB/s | 309 kB 00:00 2024-05-08T05:39:00.724 INFO:teuthology.orchestra.run.smithi063.stdout:(107/156): abseil-cpp-20211102.0-3.el9.x86_64.r 441 kB/s | 548 kB 00:01 2024-05-08T05:39:00.766 INFO:teuthology.orchestra.run.smithi063.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-05-08T05:39:00.833 INFO:teuthology.orchestra.run.smithi063.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 447 kB/s | 49 kB 00:00 2024-05-08T05:39:00.846 INFO:teuthology.orchestra.run.smithi045.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 96 kB/s | 135 kB 00:01 2024-05-08T05:39:00.950 INFO:teuthology.orchestra.run.smithi063.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 366 kB/s | 67 kB 00:00 2024-05-08T05:39:01.100 INFO:teuthology.orchestra.run.smithi063.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 566 kB/s | 151 kB 00:00 2024-05-08T05:39:01.405 INFO:teuthology.orchestra.run.smithi045.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 130 kB/s | 125 kB 00:00 2024-05-08T05:39:01.697 INFO:teuthology.orchestra.run.smithi045.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 839 kB/s | 689 kB 00:00 2024-05-08T05:39:01.806 INFO:teuthology.orchestra.run.smithi045.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 986 kB/s | 107 kB 00:00 2024-05-08T05:39:01.851 INFO:teuthology.orchestra.run.smithi063.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 731 kB/s | 548 kB 00:00 2024-05-08T05:39:01.906 INFO:teuthology.orchestra.run.smithi045.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 215 kB/s | 21 kB 00:00 2024-05-08T05:39:01.959 INFO:teuthology.orchestra.run.smithi063.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 403 kB/s | 43 kB 00:00 2024-05-08T05:39:02.010 INFO:teuthology.orchestra.run.smithi063.stdout:(114/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 792 kB/s | 839 kB 00:01 2024-05-08T05:39:02.014 INFO:teuthology.orchestra.run.smithi045.stdout:(51/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-05-08T05:39:02.040 INFO:teuthology.orchestra.run.smithi045.stdout:(52/156): python3-urllib3-1.26.5-5.el9.noarch.r 141 kB/s | 215 kB 00:01 2024-05-08T05:39:02.069 INFO:teuthology.orchestra.run.smithi063.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 295 kB/s | 32 kB 00:00 2024-05-08T05:39:02.115 INFO:teuthology.orchestra.run.smithi045.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-05-08T05:39:02.166 INFO:teuthology.orchestra.run.smithi045.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-05-08T05:39:02.210 INFO:teuthology.orchestra.run.smithi063.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 75 kB/s | 15 kB 00:00 2024-05-08T05:39:02.215 INFO:teuthology.orchestra.run.smithi045.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 181 kB/s | 18 kB 00:00 2024-05-08T05:39:02.274 INFO:teuthology.orchestra.run.smithi045.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-05-08T05:39:02.324 INFO:teuthology.orchestra.run.smithi045.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-05-08T05:39:02.377 INFO:teuthology.orchestra.run.smithi063.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 557 kB/s | 172 kB 00:00 2024-05-08T05:39:02.408 INFO:teuthology.orchestra.run.smithi045.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 339 kB/s | 45 kB 00:00 2024-05-08T05:39:02.517 INFO:teuthology.orchestra.run.smithi045.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-05-08T05:39:02.550 INFO:teuthology.orchestra.run.smithi045.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 4.0 MB/s | 582 kB 00:00 2024-05-08T05:39:02.617 INFO:teuthology.orchestra.run.smithi045.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 254 kB/s | 25 kB 00:00 2024-05-08T05:39:02.659 INFO:teuthology.orchestra.run.smithi045.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-05-08T05:39:02.719 INFO:teuthology.orchestra.run.smithi063.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 705 kB/s | 358 kB 00:00 2024-05-08T05:39:02.726 INFO:teuthology.orchestra.run.smithi045.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-08T05:39:02.768 INFO:teuthology.orchestra.run.smithi045.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-08T05:39:02.827 INFO:teuthology.orchestra.run.smithi045.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 351 kB/s | 35 kB 00:00 2024-05-08T05:39:02.827 INFO:teuthology.orchestra.run.smithi063.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 326 kB/s | 35 kB 00:00 2024-05-08T05:39:02.853 INFO:teuthology.orchestra.run.smithi063.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 638 kB/s | 303 kB 00:00 2024-05-08T05:39:03.244 INFO:teuthology.orchestra.run.smithi045.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 10 MB/s | 4.7 MB 00:00 2024-05-08T05:39:03.244 INFO:teuthology.orchestra.run.smithi063.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 493 kB/s | 206 kB 00:00 2024-05-08T05:39:03.302 INFO:teuthology.orchestra.run.smithi045.stdout:(67/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.4 MB/s | 3.0 MB 00:01 2024-05-08T05:39:03.444 INFO:teuthology.orchestra.run.smithi045.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 6.3 MB/s | 911 kB 00:00 2024-05-08T05:39:03.486 INFO:teuthology.orchestra.run.smithi045.stdout:(69/156): postgresql-13.14-1.el9.x86_64.rpm 6.6 MB/s | 1.6 MB 00:00 2024-05-08T05:39:03.570 INFO:teuthology.orchestra.run.smithi045.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-05-08T05:39:03.578 INFO:teuthology.orchestra.run.smithi063.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 433 kB/s | 144 kB 00:00 2024-05-08T05:39:03.637 INFO:teuthology.orchestra.run.smithi045.stdout:(71/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.5 MB/s | 4.4 MB 00:00 2024-05-08T05:39:03.687 INFO:teuthology.orchestra.run.smithi063.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-05-08T05:39:03.745 INFO:teuthology.orchestra.run.smithi045.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 809 kB/s | 87 kB 00:00 2024-05-08T05:39:03.804 INFO:teuthology.orchestra.run.smithi063.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-05-08T05:39:03.913 INFO:teuthology.orchestra.run.smithi063.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-05-08T05:39:04.021 INFO:teuthology.orchestra.run.smithi063.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 180 kB/s | 19 kB 00:00 2024-05-08T05:39:04.130 INFO:teuthology.orchestra.run.smithi063.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 182 kB/s | 20 kB 00:00 2024-05-08T05:39:04.971 INFO:teuthology.orchestra.run.smithi045.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 738 kB/s | 1.0 MB 00:01 2024-05-08T05:39:05.322 INFO:teuthology.orchestra.run.smithi045.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 698 kB/s | 244 kB 00:00 2024-05-08T05:39:05.438 INFO:teuthology.orchestra.run.smithi063.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 799 kB/s | 1.0 MB 00:01 2024-05-08T05:39:05.556 INFO:teuthology.orchestra.run.smithi063.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 397 kB/s | 46 kB 00:00 2024-05-08T05:39:05.672 INFO:teuthology.orchestra.run.smithi063.stdout:(130/156): python3-grpcio-1.46.7-10.el9.x86_64. 740 kB/s | 2.0 MB 00:02 2024-05-08T05:39:05.679 INFO:teuthology.orchestra.run.smithi045.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 692 kB/s | 248 kB 00:00 2024-05-08T05:39:05.731 INFO:teuthology.orchestra.run.smithi063.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 450 kB/s | 79 kB 00:00 2024-05-08T05:39:05.780 INFO:teuthology.orchestra.run.smithi045.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-05-08T05:39:05.856 INFO:teuthology.orchestra.run.smithi063.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 314 kB/s | 58 kB 00:00 2024-05-08T05:39:05.888 INFO:teuthology.orchestra.run.smithi045.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 177 kB/s | 19 kB 00:00 2024-05-08T05:39:05.965 INFO:teuthology.orchestra.run.smithi063.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-05-08T05:39:06.149 INFO:teuthology.orchestra.run.smithi063.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 653 kB/s | 272 kB 00:00 2024-05-08T05:39:06.155 INFO:teuthology.orchestra.run.smithi045.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 646 kB/s | 172 kB 00:00 2024-05-08T05:39:06.224 INFO:teuthology.orchestra.run.smithi063.stdout:(135/156): libarrow-9.0.0-11.el9.x86_64.rpm 773 kB/s | 4.4 MB 00:05 2024-05-08T05:39:06.249 INFO:teuthology.orchestra.run.smithi063.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 317 kB/s | 90 kB 00:00 2024-05-08T05:39:06.264 INFO:teuthology.orchestra.run.smithi045.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-05-08T05:39:06.266 INFO:teuthology.orchestra.run.smithi063.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 241 kB/s | 28 kB 00:00 2024-05-08T05:39:06.349 INFO:teuthology.orchestra.run.smithi063.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-05-08T05:39:06.450 INFO:teuthology.orchestra.run.smithi063.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 322 kB/s | 59 kB 00:00 2024-05-08T05:39:06.466 INFO:teuthology.orchestra.run.smithi063.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 306 kB/s | 36 kB 00:00 2024-05-08T05:39:06.517 INFO:teuthology.orchestra.run.smithi063.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 705 kB/s | 188 kB 00:00 2024-05-08T05:39:06.542 INFO:teuthology.orchestra.run.smithi063.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 554 kB/s | 51 kB 00:00 2024-05-08T05:39:06.701 INFO:teuthology.orchestra.run.smithi063.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 489 kB/s | 90 kB 00:00 2024-05-08T05:39:06.809 INFO:teuthology.orchestra.run.smithi063.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-05-08T05:39:06.834 INFO:teuthology.orchestra.run.smithi063.stdout:(145/156): python3-webob-1.8.7-6.el9.noarch.rpm 624 kB/s | 230 kB 00:00 2024-05-08T05:39:07.076 INFO:teuthology.orchestra.run.smithi063.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 718 kB/s | 192 kB 00:00 2024-05-08T05:39:07.127 INFO:teuthology.orchestra.run.smithi063.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 731 kB/s | 427 kB 00:00 2024-05-08T05:39:07.635 INFO:teuthology.orchestra.run.smithi063.stdout:(148/156): librbd1-19.0.0-3538.gcd693b0b.el9.x8 6.2 MB/s | 3.1 MB 00:00 2024-05-08T05:39:07.694 INFO:teuthology.orchestra.run.smithi063.stdout:(149/156): librados2-19.0.0-3538.gcd693b0b.el9. 5.5 MB/s | 3.4 MB 00:00 2024-05-08T05:39:07.819 INFO:teuthology.orchestra.run.smithi063.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 15 MB/s | 1.8 MB 00:00 2024-05-08T05:39:07.861 INFO:teuthology.orchestra.run.smithi063.stdout:(151/156): NetworkManager-1.47.5-1.el9.x86_64.r 10 MB/s | 2.3 MB 00:00 2024-05-08T05:39:07.903 INFO:teuthology.orchestra.run.smithi063.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 502 kB/s | 42 kB 00:00 2024-05-08T05:39:07.937 INFO:teuthology.orchestra.run.smithi063.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 3.2 MB/s | 248 kB 00:00 2024-05-08T05:39:08.095 INFO:teuthology.orchestra.run.smithi063.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 7.0 MB/s | 1.3 MB 00:00 2024-05-08T05:39:08.146 INFO:teuthology.orchestra.run.smithi063.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-05-08T05:39:08.422 INFO:teuthology.orchestra.run.smithi045.stdout:(80/156): python3-babel-2.9.1-2.el9.noarch.rpm 1.3 MB/s | 6.0 MB 00:04 2024-05-08T05:39:08.764 INFO:teuthology.orchestra.run.smithi045.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 734 kB/s | 250 kB 00:00 2024-05-08T05:39:08.913 INFO:teuthology.orchestra.run.smithi063.stdout:(156/156): thrift-0.15.0-2.el9.x86_64.rpm 779 kB/s | 1.6 MB 00:02 2024-05-08T05:39:08.919 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:39:08.919 INFO:teuthology.orchestra.run.smithi063.stdout:Total 7.1 MB/s | 226 MB 00:31 2024-05-08T05:39:09.106 INFO:teuthology.orchestra.run.smithi045.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 650 kB/s | 222 kB 00:00 2024-05-08T05:39:09.214 INFO:teuthology.orchestra.run.smithi045.stdout:(83/156): postgresql-server-13.14-1.el9.x86_64. 1.0 MB/s | 5.8 MB 00:05 2024-05-08T05:39:09.323 INFO:teuthology.orchestra.run.smithi045.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 356 kB/s | 77 kB 00:00 2024-05-08T05:39:09.340 INFO:teuthology.orchestra.run.smithi045.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 337 kB/s | 42 kB 00:00 2024-05-08T05:39:09.607 INFO:teuthology.orchestra.run.smithi045.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 595 kB/s | 159 kB 00:00 2024-05-08T05:39:09.757 INFO:teuthology.orchestra.run.smithi045.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 621 kB/s | 269 kB 00:00 2024-05-08T05:39:09.867 INFO:teuthology.orchestra.run.smithi045.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 464 kB/s | 51 kB 00:00 2024-05-08T05:39:09.983 INFO:teuthology.orchestra.run.smithi045.stdout:(89/156): python3-requests-oauthlib-1.3.0-12.el 467 kB/s | 54 kB 00:00 2024-05-08T05:39:10.050 INFO:teuthology.orchestra.run.smithi045.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 630 kB/s | 279 kB 00:00 2024-05-08T05:39:10.119 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-05-08T05:39:10.150 INFO:teuthology.orchestra.run.smithi045.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-05-08T05:39:10.251 INFO:teuthology.orchestra.run.smithi045.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-05-08T05:39:10.289 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-05-08T05:39:10.289 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-05-08T05:39:10.593 INFO:teuthology.orchestra.run.smithi045.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 646 kB/s | 221 kB 00:00 2024-05-08T05:39:10.701 INFO:teuthology.orchestra.run.smithi045.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 331 kB/s | 36 kB 00:00 2024-05-08T05:39:10.810 INFO:teuthology.orchestra.run.smithi045.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 438 kB/s | 47 kB 00:00 2024-05-08T05:39:11.235 INFO:teuthology.orchestra.run.smithi045.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 717 kB/s | 305 kB 00:00 2024-05-08T05:39:11.344 INFO:teuthology.orchestra.run.smithi045.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 534 kB/s | 58 kB 00:00 2024-05-08T05:39:11.527 INFO:teuthology.orchestra.run.smithi045.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 511 kB/s | 94 kB 00:00 2024-05-08T05:39:11.719 INFO:teuthology.orchestra.run.smithi045.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 332 kB/s | 64 kB 00:00 2024-05-08T05:39:11.819 INFO:teuthology.orchestra.run.smithi045.stdout:(100/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 448 kB/s | 45 kB 00:00 2024-05-08T05:39:11.920 INFO:teuthology.orchestra.run.smithi045.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-05-08T05:39:12.586 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-05-08T05:39:12.587 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-05-08T05:39:12.978 INFO:teuthology.orchestra.run.smithi045.stdout:(102/156): python3-numpy-1.20.1-5.el9.x86_64.rp 777 kB/s | 5.1 MB 00:06 2024-05-08T05:39:13.095 INFO:teuthology.orchestra.run.smithi045.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 735 kB/s | 863 kB 00:01 2024-05-08T05:39:13.212 INFO:teuthology.orchestra.run.smithi045.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.3 MB/s | 548 kB 00:00 2024-05-08T05:39:13.262 INFO:teuthology.orchestra.run.smithi045.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 391 kB/s | 19 kB 00:00 2024-05-08T05:39:13.304 INFO:teuthology.orchestra.run.smithi045.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.4 MB/s | 309 kB 00:00 2024-05-08T05:39:13.354 INFO:teuthology.orchestra.run.smithi045.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 522 kB/s | 26 kB 00:00 2024-05-08T05:39:13.405 INFO:teuthology.orchestra.run.smithi045.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 973 kB/s | 49 kB 00:00 2024-05-08T05:39:13.464 INFO:teuthology.orchestra.run.smithi045.stdout:(109/156): libarrow-9.0.0-11.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-05-08T05:39:13.490 INFO:teuthology.orchestra.run.smithi045.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 805 kB/s | 67 kB 00:00 2024-05-08T05:39:13.514 INFO:teuthology.orchestra.run.smithi045.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-05-08T05:39:13.572 INFO:teuthology.orchestra.run.smithi045.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-05-08T05:39:13.606 INFO:teuthology.orchestra.run.smithi045.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-05-08T05:39:13.623 INFO:teuthology.orchestra.run.smithi045.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 866 kB/s | 43 kB 00:00 2024-05-08T05:39:13.648 INFO:teuthology.orchestra.run.smithi045.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 771 kB/s | 32 kB 00:00 2024-05-08T05:39:13.665 INFO:teuthology.orchestra.run.smithi045.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 357 kB/s | 15 kB 00:00 2024-05-08T05:39:13.699 INFO:teuthology.orchestra.run.smithi045.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-05-08T05:39:13.724 INFO:teuthology.orchestra.run.smithi045.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-05-08T05:39:13.749 INFO:teuthology.orchestra.run.smithi045.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-05-08T05:39:13.775 INFO:teuthology.orchestra.run.smithi045.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 701 kB/s | 35 kB 00:00 2024-05-08T05:39:13.800 INFO:teuthology.orchestra.run.smithi045.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-05-08T05:39:13.850 INFO:teuthology.orchestra.run.smithi045.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-05-08T05:39:13.892 INFO:teuthology.orchestra.run.smithi045.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-05-08T05:39:13.909 INFO:teuthology.orchestra.run.smithi045.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 182 kB/s | 11 kB 00:00 2024-05-08T05:39:13.934 INFO:teuthology.orchestra.run.smithi045.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 422 kB/s | 18 kB 00:00 2024-05-08T05:39:13.951 INFO:teuthology.orchestra.run.smithi045.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 553 kB/s | 23 kB 00:00 2024-05-08T05:39:13.977 INFO:teuthology.orchestra.run.smithi045.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 464 kB/s | 19 kB 00:00 2024-05-08T05:39:13.993 INFO:teuthology.orchestra.run.smithi045.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 470 kB/s | 20 kB 00:00 2024-05-08T05:39:14.035 INFO:teuthology.orchestra.run.smithi045.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 17 MB/s | 1.0 MB 00:00 2024-05-08T05:39:14.052 INFO:teuthology.orchestra.run.smithi045.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 791 kB/s | 46 kB 00:00 2024-05-08T05:39:14.078 INFO:teuthology.orchestra.run.smithi045.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-05-08T05:39:14.095 INFO:teuthology.orchestra.run.smithi045.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-05-08T05:39:14.128 INFO:teuthology.orchestra.run.smithi045.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-05-08T05:39:14.145 INFO:teuthology.orchestra.run.smithi045.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-05-08T05:39:14.179 INFO:teuthology.orchestra.run.smithi045.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-05-08T05:39:14.196 INFO:teuthology.orchestra.run.smithi045.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-05-08T05:39:14.221 INFO:teuthology.orchestra.run.smithi045.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 736 kB/s | 31 kB 00:00 2024-05-08T05:39:14.246 INFO:teuthology.orchestra.run.smithi045.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-05-08T05:39:14.271 INFO:teuthology.orchestra.run.smithi045.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-05-08T05:39:14.297 INFO:teuthology.orchestra.run.smithi045.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-05-08T05:39:14.330 INFO:teuthology.orchestra.run.smithi045.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 869 kB/s | 51 kB 00:00 2024-05-08T05:39:14.372 INFO:teuthology.orchestra.run.smithi045.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-08T05:39:14.397 INFO:teuthology.orchestra.run.smithi045.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-05-08T05:39:14.423 INFO:teuthology.orchestra.run.smithi045.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 8.3 MB/s | 427 kB 00:00 2024-05-08T05:39:14.448 INFO:teuthology.orchestra.run.smithi045.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-05-08T05:39:14.473 INFO:teuthology.orchestra.run.smithi045.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-05-08T05:39:14.515 INFO:teuthology.orchestra.run.smithi045.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-05-08T05:39:15.507 INFO:teuthology.orchestra.run.smithi045.stdout:(148/156): librbd1-19.0.0-3538.gcd693b0b.el9.x8 3.2 MB/s | 3.1 MB 00:00 2024-05-08T05:39:15.666 INFO:teuthology.orchestra.run.smithi045.stdout:(149/156): librados2-19.0.0-3538.gcd693b0b.el9. 2.8 MB/s | 3.4 MB 00:01 2024-05-08T05:39:17.200 INFO:teuthology.orchestra.run.smithi045.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.2 MB/s | 1.8 MB 00:01 2024-05-08T05:39:18.008 INFO:teuthology.orchestra.run.smithi045.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 52 kB/s | 42 kB 00:00 2024-05-08T05:39:18.050 INFO:teuthology.orchestra.run.smithi045.stdout:(152/156): NetworkManager-1.47.5-1.el9.x86_64.r 930 kB/s | 2.3 MB 00:02 2024-05-08T05:39:18.908 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-05-08T05:39:19.001 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-08T05:39:19.039 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-08T05:39:19.097 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-08T05:39:19.134 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-08T05:39:19.150 INFO:teuthology.orchestra.run.smithi045.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 218 kB/s | 248 kB 00:01 2024-05-08T05:39:19.159 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-08T05:39:19.217 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-08T05:39:19.418 INFO:teuthology.orchestra.run.smithi045.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 1.0 MB/s | 1.3 MB 00:01 2024-05-08T05:39:19.558 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-08T05:39:19.668 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : librados2-2:19.0.0-3538.gcd693b0b.el9.x86_64 8/164 2024-05-08T05:39:19.801 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados2-2:19.0.0-3538.gcd693b0b.el9.x86_64 8/164 2024-05-08T05:39:19.828 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephfs2-2:19.0.0-3538.gcd693b0b.el9.x86_64 9/164 2024-05-08T05:39:19.926 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephfs2-2:19.0.0-3538.gcd693b0b.el9.x86_64 9/164 2024-05-08T05:39:19.958 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rados-2:19.0.0-3538.gcd693b0b.el9.x86_64 10/164 2024-05-08T05:39:20.160 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-08T05:39:20.216 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-08T05:39:20.240 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephsqlite-2:19.0.0-3538.gcd693b0b.el9.x86_64 13/164 2024-05-08T05:39:20.346 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3538.gcd693b0b.el9.x86_64 13/164 2024-05-08T05:39:20.374 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libradosstriper1-2:19.0.0-3538.gcd693b0b.el9.x86 14/164 2024-05-08T05:39:20.437 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3538.gcd693b0b.el9.x86 14/164 2024-05-08T05:39:20.459 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:39:20.486 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:39:20.654 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:39:20.702 INFO:teuthology.orchestra.run.smithi045.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.5 MB/s | 2.4 MB 00:01 2024-05-08T05:39:20.830 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:39:20.973 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:39:21.033 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:39:21.187 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:39:21.251 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:39:21.455 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:39:21.571 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-08T05:39:21.597 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-08T05:39:21.632 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-08T05:39:21.709 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-08T05:39:21.748 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-08T05:39:21.792 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-08T05:39:21.824 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-08T05:39:21.852 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-08T05:39:21.924 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-08T05:39:21.983 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-08T05:39:22.013 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-08T05:39:22.056 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-08T05:39:22.136 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-08T05:39:22.167 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ceph-common-2:19.0.0-3538.gcd693b0b.el9. 32/164 2024-05-08T05:39:22.211 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ceph-argparse-2:19.0.0-3538.gcd693b0b.el 33/164 2024-05-08T05:39:22.239 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cephfs-2:19.0.0-3538.gcd693b0b.el9.x86_6 34/164 2024-05-08T05:39:22.314 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-08T05:39:22.363 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-08T05:39:22.439 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-08T05:39:22.578 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-08T05:39:22.633 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-08T05:39:23.190 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-08T05:39:23.279 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-08T05:39:23.314 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-08T05:39:23.357 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-08T05:39:23.411 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librados-devel-2:19.0.0-3538.gcd693b0b.el9.x86_6 44/164 2024-05-08T05:39:23.455 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-08T05:39:23.608 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-08T05:39:23.664 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-08T05:39:23.759 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-08T05:39:23.830 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-08T05:39:23.904 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-08T05:39:23.940 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-08T05:39:23.977 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-08T05:39:24.025 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-08T05:39:24.060 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-08T05:39:24.104 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-08T05:39:24.146 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-08T05:39:24.289 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-08T05:39:24.342 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-08T05:39:24.372 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-08T05:39:24.414 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-08T05:39:24.511 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-08T05:39:24.547 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-08T05:39:24.588 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-08T05:39:24.629 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-08T05:39:24.653 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-08T05:39:24.747 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-08T05:39:24.876 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-08T05:39:24.910 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-08T05:39:24.939 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-08T05:39:24.973 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-08T05:39:25.012 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-08T05:39:25.052 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-08T05:39:25.082 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-08T05:39:25.116 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-08T05:39:25.149 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-08T05:39:25.191 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-08T05:39:25.225 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-08T05:39:25.315 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-08T05:39:25.365 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-08T05:39:25.395 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-08T05:39:25.443 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-08T05:39:25.545 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-08T05:39:25.597 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-08T05:39:26.355 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-08T05:39:26.402 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-08T05:39:26.444 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-08T05:39:26.555 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-08T05:39:26.611 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-08T05:39:26.701 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-08T05:39:26.729 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-08T05:39:26.907 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-08T05:39:26.927 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-08T05:39:27.008 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-08T05:39:27.688 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-08T05:39:27.811 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-08T05:39:29.126 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-08T05:39:29.204 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-08T05:39:29.232 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-08T05:39:29.265 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-08T05:39:29.602 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-08T05:39:29.636 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : librbd1-2:19.0.0-3538.gcd693b0b.el9.x86_64 101/164 2024-05-08T05:39:29.732 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librbd1-2:19.0.0-3538.gcd693b0b.el9.x86_64 101/164 2024-05-08T05:39:29.758 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 102/164 2024-05-08T05:39:29.794 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-08T05:39:30.301 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-08T05:39:30.331 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librgw2-2:19.0.0-3538.gcd693b0b.el9.x86_64 105/164 2024-05-08T05:39:30.386 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librgw2-2:19.0.0-3538.gcd693b0b.el9.x86_64 105/164 2024-05-08T05:39:30.416 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 106/164 2024-05-08T05:39:31.624 INFO:teuthology.orchestra.run.smithi045.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 769 kB/s | 16 MB 00:21 2024-05-08T05:39:31.628 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:39:31.628 INFO:teuthology.orchestra.run.smithi045.stdout:Total 4.3 MB/s | 226 MB 00:52 2024-05-08T05:39:32.792 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 107/164 2024-05-08T05:39:32.804 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-08T05:39:32.828 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 107/164 2024-05-08T05:39:32.972 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-08T05:39:32.972 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-08T05:39:33.408 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 107/164 2024-05-08T05:39:33.449 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-base-2:19.0.0-3538.gcd693b0b.el9.x86_64 108/164 2024-05-08T05:39:33.542 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-base-2:19.0.0-3538.gcd693b0b.el9.x86_64 108/164 2024-05-08T05:39:33.542 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T05:39:33.542 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T05:39:33.542 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:39:33.583 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-selinux-2:19.0.0-3538.gcd693b0b.el9.x86_64 109/164 2024-05-08T05:39:35.267 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-08T05:39:35.267 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-08T05:39:41.439 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-08T05:39:41.520 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-08T05:39:41.550 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-08T05:39:41.608 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-08T05:39:41.645 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-08T05:39:41.671 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-08T05:39:41.737 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-08T05:39:42.086 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-08T05:39:42.205 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:19.0.0-3538.gcd693b0b.el9.x86_64 8/164 2024-05-08T05:39:42.313 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:19.0.0-3538.gcd693b0b.el9.x86_64 8/164 2024-05-08T05:39:42.339 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:19.0.0-3538.gcd693b0b.el9.x86_64 9/164 2024-05-08T05:39:42.436 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:19.0.0-3538.gcd693b0b.el9.x86_64 9/164 2024-05-08T05:39:42.469 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:19.0.0-3538.gcd693b0b.el9.x86_64 10/164 2024-05-08T05:39:42.645 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-08T05:39:42.716 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-08T05:39:42.743 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:19.0.0-3538.gcd693b0b.el9.x86_64 13/164 2024-05-08T05:39:42.848 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3538.gcd693b0b.el9.x86_64 13/164 2024-05-08T05:39:42.877 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:19.0.0-3538.gcd693b0b.el9.x86 14/164 2024-05-08T05:39:42.940 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3538.gcd693b0b.el9.x86 14/164 2024-05-08T05:39:42.962 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:39:42.981 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:39:43.181 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:39:43.342 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:39:43.492 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:39:43.553 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:39:43.670 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:39:43.746 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:39:43.950 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:39:44.063 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-08T05:39:44.091 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-08T05:39:44.126 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-08T05:39:44.202 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-08T05:39:44.250 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-08T05:39:44.294 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-08T05:39:44.326 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-08T05:39:44.353 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-08T05:39:44.424 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-08T05:39:44.485 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-08T05:39:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-08T05:39:44.559 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-08T05:39:44.637 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-08T05:39:44.677 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:19.0.0-3538.gcd693b0b.el9. 32/164 2024-05-08T05:39:44.713 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:19.0.0-3538.gcd693b0b.el 33/164 2024-05-08T05:39:44.741 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:19.0.0-3538.gcd693b0b.el9.x86_6 34/164 2024-05-08T05:39:44.813 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-08T05:39:44.856 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-08T05:39:44.934 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-08T05:39:45.077 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-08T05:39:45.134 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-08T05:39:45.635 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3538.gcd693b0b.el9.x86_64 109/164 2024-05-08T05:39:45.635 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /sys 2024-05-08T05:39:45.635 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /proc 2024-05-08T05:39:45.635 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /mnt 2024-05-08T05:39:45.635 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /var/tmp 2024-05-08T05:39:45.635 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /home 2024-05-08T05:39:45.636 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /root 2024-05-08T05:39:45.636 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /tmp 2024-05-08T05:39:45.636 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:39:45.685 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-08T05:39:45.764 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-08T05:39:45.799 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-08T05:39:45.807 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-volume-2:19.0.0-3538.gcd693b0b.el9.noarch 110/164 2024-05-08T05:39:45.823 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-volume-2:19.0.0-3538.gcd693b0b.el9.noarch 110/164 2024-05-08T05:39:45.823 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:45.823 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T05:39:45.823 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:39:45.833 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-08T05:39:45.871 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:19.0.0-3538.gcd693b0b.el9.x86_6 44/164 2024-05-08T05:39:45.914 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-08T05:39:46.073 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-08T05:39:46.133 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-08T05:39:46.227 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-08T05:39:46.290 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-08T05:39:46.363 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-08T05:39:46.400 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-08T05:39:46.437 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-08T05:39:46.484 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-08T05:39:46.520 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-08T05:39:46.563 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-08T05:39:46.606 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-08T05:39:46.748 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-08T05:39:46.802 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-08T05:39:46.832 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-08T05:39:46.874 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-08T05:39:46.972 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-08T05:39:47.010 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-08T05:39:47.048 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-08T05:39:47.089 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-08T05:39:47.113 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-08T05:39:47.198 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-08T05:39:47.337 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-08T05:39:47.371 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-08T05:39:47.384 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-osd-2:19.0.0-3538.gcd693b0b.el9.x86_64 111/164 2024-05-08T05:39:47.399 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-08T05:39:47.412 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-osd-2:19.0.0-3538.gcd693b0b.el9.x86_64 111/164 2024-05-08T05:39:47.412 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:47.412 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T05:39:47.412 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:39:47.413 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:39:47.413 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:39:47.441 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-08T05:39:47.480 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-08T05:39:47.512 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-08T05:39:47.533 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-08T05:39:47.568 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-08T05:39:47.601 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-08T05:39:47.642 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-08T05:39:47.669 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mds-2:19.0.0-3538.gcd693b0b.el9.x86_64 112/164 2024-05-08T05:39:47.685 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-08T05:39:47.694 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mds-2:19.0.0-3538.gcd693b0b.el9.x86_64 112/164 2024-05-08T05:39:47.694 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:47.694 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T05:39:47.694 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:39:47.694 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:39:47.694 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:39:47.775 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-08T05:39:47.825 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-08T05:39:47.855 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-08T05:39:47.903 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-08T05:39:48.012 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-08T05:39:48.066 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-08T05:39:48.163 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mon-2:19.0.0-3538.gcd693b0b.el9.x86_64 113/164 2024-05-08T05:39:48.188 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mon-2:19.0.0-3538.gcd693b0b.el9.x86_64 113/164 2024-05-08T05:39:48.188 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:48.188 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T05:39:48.188 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:39:48.188 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:39:48.188 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:39:48.294 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-08T05:39:48.351 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-08T05:39:48.415 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-08T05:39:48.617 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-08T05:39:48.664 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-08T05:39:48.744 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-08T05:39:48.823 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-08T05:39:48.833 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-08T05:39:48.870 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-08T05:39:48.904 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-08T05:39:48.975 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-08T05:39:49.006 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-08T05:39:49.023 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-08T05:39:49.054 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-08T05:39:49.063 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-08T05:39:49.082 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-08T05:39:49.152 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-08T05:39:49.189 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-08T05:39:49.364 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-08T05:39:49.394 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-08T05:39:49.476 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-08T05:39:49.511 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: cephadm-2:19.0.0-3538.gcd693b0b.el9.noarch 125/164 2024-05-08T05:39:49.530 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephadm-2:19.0.0-3538.gcd693b0b.el9.noarch 125/164 2024-05-08T05:39:49.563 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3538.gcd693b0b.e 126/164 2024-05-08T05:39:49.626 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3538.gcd693b0b. 127/164 2024-05-08T05:39:49.653 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3538.gcd693b0b.el9.noa 128/164 2024-05-08T05:39:50.169 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-08T05:39:50.296 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-08T05:39:50.748 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3538.gcd693b0b.el9.noa 128/164 2024-05-08T05:39:50.780 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3538.gcd693b0b.el9.n 129/164 2024-05-08T05:39:51.618 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-08T05:39:51.705 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-08T05:39:51.725 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-08T05:39:51.758 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-08T05:39:51.796 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3538.gcd693b0b.el9.n 129/164 2024-05-08T05:39:51.822 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3538.gcd6 130/164 2024-05-08T05:39:51.938 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3538.gcd6 130/164 2024-05-08T05:39:52.087 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-08T05:39:52.100 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3538.gcd693b0b.el 131/164 2024-05-08T05:39:52.133 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-2:19.0.0-3538.gcd693b0b.el9.x86_64 132/164 2024-05-08T05:39:52.158 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3538.gcd693b0b.el9.x86_64 132/164 2024-05-08T05:39:52.159 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:52.159 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T05:39:52.159 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:39:52.159 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:39:52.159 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:39:52.209 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-rook-2:19.0.0-3538.gcd693b0b.el9.noarch 133/164 2024-05-08T05:39:52.220 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3538.gcd693b0b.el9.noarch 133/164 2024-05-08T05:39:52.221 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:19.0.0-3538.gcd693b0b.el9.x86_64 101/164 2024-05-08T05:39:52.313 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:19.0.0-3538.gcd693b0b.el9.x86_64 101/164 2024-05-08T05:39:52.322 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-2:19.0.0-3538.gcd693b0b.el9.x86_64 134/164 2024-05-08T05:39:52.343 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 102/164 2024-05-08T05:39:52.379 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-08T05:39:52.381 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 135/164 2024-05-08T05:39:52.884 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-08T05:39:52.907 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:19.0.0-3538.gcd693b0b.el9.x86_64 105/164 2024-05-08T05:39:52.978 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:19.0.0-3538.gcd693b0b.el9.x86_64 105/164 2024-05-08T05:39:53.008 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 106/164 2024-05-08T05:39:53.360 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-08T05:39:53.394 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-radosgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 137/164 2024-05-08T05:39:53.420 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 137/164 2024-05-08T05:39:53.420 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:53.420 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T05:39:53.420 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:39:53.420 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:39:53.420 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:39:53.462 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3538.gcd693 138/164 2024-05-08T05:39:53.486 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3538.gcd693 138/164 2024-05-08T05:39:53.486 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:53.486 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T05:39:53.486 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:39:53.538 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephfs-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 139/164 2024-05-08T05:39:53.562 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 139/164 2024-05-08T05:39:53.562 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:53.563 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-08T05:39:53.563 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:39:53.563 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:39:53.563 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:39:53.880 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 140/164 2024-05-08T05:39:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 140/164 2024-05-08T05:39:53.905 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:53.905 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T05:39:53.905 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:39:53.905 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:39:53.905 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:39:55.440 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 107/164 2024-05-08T05:39:55.479 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 107/164 2024-05-08T05:39:56.050 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 107/164 2024-05-08T05:39:56.091 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:19.0.0-3538.gcd693b0b.el9.x86_64 108/164 2024-05-08T05:39:56.188 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:19.0.0-3538.gcd693b0b.el9.x86_64 108/164 2024-05-08T05:39:56.189 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T05:39:56.189 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T05:39:56.189 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:39:56.225 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:19.0.0-3538.gcd693b0b.el9.x86_64 109/164 2024-05-08T05:39:58.793 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-test-2:19.0.0-3538.gcd693b0b.el9.x86_64 141/164 2024-05-08T05:39:58.849 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 142/164 2024-05-08T05:39:58.900 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-nbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 143/164 2024-05-08T05:39:58.932 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-08T05:39:58.959 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-08T05:39:59.162 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephfs-shell-2:19.0.0-3538.gcd693b0b.el9.x86_64 146/164 2024-05-08T05:39:59.205 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-08T05:39:59.235 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephfs-devel-2:19.0.0-3538.gcd693b0b.el9.x86_ 148/164 2024-05-08T05:39:59.265 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-08T05:39:59.311 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-08T05:39:59.966 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:40:00.009 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:40:00.043 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:40:00.945 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephfs-top-2:19.0.0-3538.gcd693b0b.el9.noarch 152/164 2024-05-08T05:40:01.023 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-08T05:40:01.071 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-08T05:40:01.166 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-08T05:40:01.201 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-08T05:40:01.235 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-08T05:40:01.235 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:40:01.312 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:40:01.344 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-08T05:40:01.345 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:40:01.429 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:40:01.454 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-08T05:40:01.459 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:40:01.459 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:40:01.664 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:40:01.664 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:40:01.741 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:40:01.816 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:40:07.210 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:40:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-2:19.0.0-3538.gcd693b0b.el9.x86_64 1/164 2024-05-08T05:40:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-base-2:19.0.0-3538.gcd693b0b.el9.x86_64 2/164 2024-05-08T05:40:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 3/164 2024-05-08T05:40:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 4/164 2024-05-08T05:40:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3538.gcd693 5/164 2024-05-08T05:40:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mds-2:19.0.0-3538.gcd693b0b.el9.x86_64 6/164 2024-05-08T05:40:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-2:19.0.0-3538.gcd693b0b.el9.x86_64 7/164 2024-05-08T05:40:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mon-2:19.0.0-3538.gcd693b0b.el9.x86_64 8/164 2024-05-08T05:40:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-osd-2:19.0.0-3538.gcd693b0b.el9.x86_64 9/164 2024-05-08T05:40:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-radosgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 10/164 2024-05-08T05:40:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-selinux-2:19.0.0-3538.gcd693b0b.el9.x86_64 11/164 2024-05-08T05:40:07.211 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-test-2:19.0.0-3538.gcd693b0b.el9.x86_64 12/164 2024-05-08T05:40:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephfs-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 13/164 2024-05-08T05:40:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephfs-shell-2:19.0.0-3538.gcd693b0b.el9.x86_64 14/164 2024-05-08T05:40:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephfs-devel-2:19.0.0-3538.gcd693b0b.el9.x86_ 15/164 2024-05-08T05:40:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephfs2-2:19.0.0-3538.gcd693b0b.el9.x86_64 16/164 2024-05-08T05:40:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephsqlite-2:19.0.0-3538.gcd693b0b.el9.x86_64 17/164 2024-05-08T05:40:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados-devel-2:19.0.0-3538.gcd693b0b.el9.x86_6 18/164 2024-05-08T05:40:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libradosstriper1-2:19.0.0-3538.gcd693b0b.el9.x86 19/164 2024-05-08T05:40:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librgw2-2:19.0.0-3538.gcd693b0b.el9.x86_64 20/164 2024-05-08T05:40:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3538.gcd693b0b.el 21/164 2024-05-08T05:40:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ceph-common-2:19.0.0-3538.gcd693b0b.el9. 22/164 2024-05-08T05:40:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cephfs-2:19.0.0-3538.gcd693b0b.el9.x86_6 23/164 2024-05-08T05:40:07.212 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rados-2:19.0.0-3538.gcd693b0b.el9.x86_64 24/164 2024-05-08T05:40:07.213 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 25/164 2024-05-08T05:40:07.213 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 26/164 2024-05-08T05:40:07.213 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 27/164 2024-05-08T05:40:07.213 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 28/164 2024-05-08T05:40:07.213 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-nbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 29/164 2024-05-08T05:40:07.213 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3538.gcd693b0b. 30/164 2024-05-08T05:40:07.225 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3538.gcd693b0b.el9.noa 31/164 2024-05-08T05:40:07.225 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3538.gcd693b0b.el9.n 32/164 2024-05-08T05:40:07.225 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3538.gcd6 33/164 2024-05-08T05:40:07.225 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3538.gcd693b0b.el 34/164 2024-05-08T05:40:07.225 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3538.gcd693b0b.el9.noarch 35/164 2024-05-08T05:40:07.225 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3538.gcd693b0b.e 36/164 2024-05-08T05:40:07.225 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-volume-2:19.0.0-3538.gcd693b0b.el9.noarch 37/164 2024-05-08T05:40:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephadm-2:19.0.0-3538.gcd693b0b.el9.noarch 38/164 2024-05-08T05:40:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephfs-top-2:19.0.0-3538.gcd693b0b.el9.noarch 39/164 2024-05-08T05:40:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-08T05:40:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-08T05:40:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-08T05:40:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-08T05:40:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-08T05:40:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-08T05:40:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-08T05:40:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-08T05:40:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-08T05:40:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-08T05:40:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-08T05:40:07.226 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-08T05:40:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-08T05:40:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-08T05:40:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-08T05:40:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-08T05:40:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-08T05:40:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-08T05:40:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-08T05:40:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-08T05:40:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-08T05:40:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-08T05:40:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-08T05:40:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-08T05:40:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-08T05:40:07.227 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-08T05:40:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-08T05:40:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-08T05:40:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-08T05:40:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-08T05:40:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-08T05:40:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-08T05:40:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-08T05:40:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-08T05:40:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-08T05:40:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-08T05:40:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-08T05:40:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-08T05:40:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-08T05:40:07.228 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-08T05:40:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-08T05:40:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-08T05:40:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-08T05:40:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-08T05:40:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-08T05:40:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-08T05:40:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-08T05:40:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-08T05:40:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-08T05:40:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-08T05:40:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-08T05:40:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-08T05:40:07.229 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-08T05:40:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-08T05:40:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-08T05:40:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-08T05:40:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-08T05:40:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-08T05:40:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-08T05:40:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-08T05:40:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-08T05:40:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-08T05:40:07.230 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-08T05:40:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-08T05:40:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-08T05:40:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-08T05:40:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-08T05:40:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-08T05:40:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-08T05:40:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-08T05:40:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-08T05:40:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-08T05:40:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-08T05:40:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-08T05:40:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-08T05:40:07.231 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-08T05:40:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-08T05:40:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-08T05:40:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-08T05:40:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-08T05:40:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-08T05:40:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-08T05:40:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-08T05:40:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-08T05:40:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-08T05:40:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-08T05:40:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-08T05:40:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-08T05:40:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-08T05:40:07.232 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-08T05:40:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-08T05:40:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-08T05:40:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-08T05:40:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-08T05:40:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-08T05:40:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-08T05:40:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-08T05:40:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-08T05:40:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-08T05:40:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-08T05:40:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-08T05:40:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-08T05:40:07.233 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-08T05:40:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-08T05:40:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-08T05:40:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-08T05:40:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-08T05:40:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-08T05:40:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-08T05:40:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados2-2:19.0.0-3538.gcd693b0b.el9.x86_64 149/164 2024-05-08T05:40:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-08T05:40:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librbd1-2:19.0.0-3538.gcd693b0b.el9.x86_64 151/164 2024-05-08T05:40:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-08T05:40:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-08T05:40:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-08T05:40:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-08T05:40:07.234 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-08T05:40:07.235 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-08T05:40:07.235 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-08T05:40:07.235 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-08T05:40:07.235 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-08T05:40:07.235 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-08T05:40:07.235 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-08T05:40:07.235 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-08T05:40:07.770 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3538.gcd693b0b.el9.x86_64 109/164 2024-05-08T05:40:07.770 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2024-05-08T05:40:07.770 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2024-05-08T05:40:07.770 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2024-05-08T05:40:07.770 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2024-05-08T05:40:07.770 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2024-05-08T05:40:07.770 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /root 2024-05-08T05:40:07.770 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2024-05-08T05:40:07.770 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:07.949 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:19.0.0-3538.gcd693b0b.el9.noarch 110/164 2024-05-08T05:40:07.965 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:19.0.0-3538.gcd693b0b.el9.noarch 110/164 2024-05-08T05:40:07.965 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:40:07.965 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T05:40:07.965 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:09.502 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:19.0.0-3538.gcd693b0b.el9.x86_64 111/164 2024-05-08T05:40:09.531 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:19.0.0-3538.gcd693b0b.el9.x86_64 111/164 2024-05-08T05:40:09.531 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:40:09.531 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T05:40:09.531 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:40:09.531 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:40:09.531 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:09.769 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:19.0.0-3538.gcd693b0b.el9.x86_64 112/164 2024-05-08T05:40:09.793 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:19.0.0-3538.gcd693b0b.el9.x86_64 112/164 2024-05-08T05:40:09.794 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:40:09.794 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T05:40:09.794 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:40:09.794 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:40:09.794 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:40:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:09.931 INFO:teuthology.orchestra.run.smithi063.stdout:Upgraded: 2024-05-08T05:40:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-08T05:40:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-08T05:40:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-08T05:40:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-08T05:40:09.931 INFO:teuthology.orchestra.run.smithi063.stdout: librados2-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: librbd1-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-08T05:40:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-08T05:40:09.932 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-05-08T05:40:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T05:40:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: automake-1.16.2-8.el9.noarch 2024-05-08T05:40:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: bison-3.7.4-5.el9.x86_64 2024-05-08T05:40:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T05:40:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-grafana-dashboards-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T05:40:09.932 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-immutable-object-cache-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mds-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-cephadm-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T05:40:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T05:40:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T05:40:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T05:40:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-rook-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T05:40:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mon-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-osd-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-prometheus-alerts-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T05:40:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-radosgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-selinux-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-test-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.933 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-volume-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T05:40:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: cephadm-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T05:40:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-shell-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: cephfs-top-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T05:40:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-08T05:40:09.934 INFO:teuthology.orchestra.run.smithi063.stdout: flex-2.6.4-9.el9.x86_64 2024-05-08T05:40:09.937 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T05:40:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T05:40:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T05:40:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T05:40:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T05:40:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T05:40:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-08T05:40:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T05:40:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T05:40:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs-devel-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs2-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: libcephsqlite-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T05:40:09.938 INFO:teuthology.orchestra.run.smithi063.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T05:40:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T05:40:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T05:40:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T05:40:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: librados-devel-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: libradosstriper1-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T05:40:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: librgw2-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-08T05:40:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T05:40:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-08T05:40:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-08T05:40:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T05:40:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T05:40:09.939 INFO:teuthology.orchestra.run.smithi063.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T05:40:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T05:40:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T05:40:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T05:40:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-08T05:40:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T05:40:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-13.14-1.el9.x86_64 2024-05-08T05:40:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-08T05:40:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-08T05:40:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-08T05:40:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T05:40:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T05:40:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T05:40:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-08T05:40:09.940 INFO:teuthology.orchestra.run.smithi063.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T05:40:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T05:40:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T05:40:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-argparse-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cephfs-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T05:40:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T05:40:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T05:40:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T05:40:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-08T05:40:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-08T05:40:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T05:40:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T05:40:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T05:40:09.941 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T05:40:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T05:40:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T05:40:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T05:40:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T05:40:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T05:40:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T05:40:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T05:40:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T05:40:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T05:40:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T05:40:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T05:40:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T05:40:09.942 INFO:teuthology.orchestra.run.smithi063.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T05:40:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T05:40:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T05:40:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T05:40:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T05:40:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T05:40:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T05:40:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T05:40:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T05:40:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T05:40:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T05:40:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T05:40:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T05:40:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T05:40:09.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T05:40:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T05:40:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-08T05:40:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T05:40:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rados-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T05:40:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T05:40:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T05:40:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T05:40:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T05:40:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T05:40:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T05:40:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T05:40:09.944 INFO:teuthology.orchestra.run.smithi063.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T05:40:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T05:40:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-08T05:40:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T05:40:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T05:40:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T05:40:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T05:40:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T05:40:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T05:40:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T05:40:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-nbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T05:40:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T05:40:09.945 INFO:teuthology.orchestra.run.smithi063.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T05:40:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-08T05:40:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-08T05:40:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T05:40:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-08T05:40:09.946 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:09.946 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-05-08T05:40:10.261 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:19.0.0-3538.gcd693b0b.el9.x86_64 113/164 2024-05-08T05:40:10.285 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:19.0.0-3538.gcd693b0b.el9.x86_64 113/164 2024-05-08T05:40:10.286 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:40:10.286 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T05:40:10.286 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:40:10.286 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:40:10.286 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:10.344 DEBUG:teuthology.parallel:result is None 2024-05-08T05:40:10.392 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-08T05:40:10.450 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-08T05:40:10.514 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-08T05:40:10.710 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-08T05:40:10.755 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-08T05:40:10.843 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-08T05:40:10.932 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-08T05:40:11.075 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-08T05:40:11.121 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-08T05:40:11.153 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-08T05:40:11.180 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-08T05:40:11.611 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:19.0.0-3538.gcd693b0b.el9.noarch 125/164 2024-05-08T05:40:11.637 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:19.0.0-3538.gcd693b0b.el9.noarch 125/164 2024-05-08T05:40:11.670 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3538.gcd693b0b.e 126/164 2024-05-08T05:40:11.734 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3538.gcd693b0b. 127/164 2024-05-08T05:40:11.760 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3538.gcd693b0b.el9.noa 128/164 2024-05-08T05:40:12.853 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3538.gcd693b0b.el9.noa 128/164 2024-05-08T05:40:12.887 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3538.gcd693b0b.el9.n 129/164 2024-05-08T05:40:13.900 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3538.gcd693b0b.el9.n 129/164 2024-05-08T05:40:13.929 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3538.gcd6 130/164 2024-05-08T05:40:14.049 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3538.gcd6 130/164 2024-05-08T05:40:14.222 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3538.gcd693b0b.el 131/164 2024-05-08T05:40:14.256 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:19.0.0-3538.gcd693b0b.el9.x86_64 132/164 2024-05-08T05:40:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3538.gcd693b0b.el9.x86_64 132/164 2024-05-08T05:40:14.281 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:40:14.281 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T05:40:14.281 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:40:14.281 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:40:14.281 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:14.332 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:19.0.0-3538.gcd693b0b.el9.noarch 133/164 2024-05-08T05:40:14.343 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3538.gcd693b0b.el9.noarch 133/164 2024-05-08T05:40:14.445 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:19.0.0-3538.gcd693b0b.el9.x86_64 134/164 2024-05-08T05:40:14.507 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 135/164 2024-05-08T05:40:15.486 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-08T05:40:15.517 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 137/164 2024-05-08T05:40:15.542 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 137/164 2024-05-08T05:40:15.542 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:40:15.542 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T05:40:15.542 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:40:15.542 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:40:15.542 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:15.585 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3538.gcd693 138/164 2024-05-08T05:40:15.608 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3538.gcd693 138/164 2024-05-08T05:40:15.608 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:40:15.608 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T05:40:15.608 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:15.661 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 139/164 2024-05-08T05:40:15.684 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 139/164 2024-05-08T05:40:15.684 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:40:15.685 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-08T05:40:15.685 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:40:15.685 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:40:15.685 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:16.011 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 140/164 2024-05-08T05:40:16.035 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 140/164 2024-05-08T05:40:16.035 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:40:16.036 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T05:40:16.036 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:40:16.036 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:40:16.036 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:20.907 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:19.0.0-3538.gcd693b0b.el9.x86_64 141/164 2024-05-08T05:40:20.957 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 142/164 2024-05-08T05:40:20.998 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 143/164 2024-05-08T05:40:21.029 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-08T05:40:21.065 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-08T05:40:21.271 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-shell-2:19.0.0-3538.gcd693b0b.el9.x86_64 146/164 2024-05-08T05:40:21.319 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-08T05:40:21.349 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:19.0.0-3538.gcd693b0b.el9.x86_ 148/164 2024-05-08T05:40:21.379 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-08T05:40:21.417 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-08T05:40:21.970 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:40:22.006 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:40:22.027 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:40:22.927 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-top-2:19.0.0-3538.gcd693b0b.el9.noarch 152/164 2024-05-08T05:40:23.020 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-08T05:40:23.070 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-08T05:40:23.168 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-08T05:40:23.198 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-08T05:40:23.223 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-08T05:40:23.224 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:40:23.300 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:40:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-08T05:40:23.333 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:40:23.409 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:40:23.434 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-08T05:40:23.439 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:40:23.440 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:40:23.631 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:40:23.632 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:40:23.721 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:40:23.798 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:40:29.311 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:40:29.311 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:19.0.0-3538.gcd693b0b.el9.x86_64 1/164 2024-05-08T05:40:29.311 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:19.0.0-3538.gcd693b0b.el9.x86_64 2/164 2024-05-08T05:40:29.311 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 3/164 2024-05-08T05:40:29.311 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 4/164 2024-05-08T05:40:29.312 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3538.gcd693 5/164 2024-05-08T05:40:29.312 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:19.0.0-3538.gcd693b0b.el9.x86_64 6/164 2024-05-08T05:40:29.312 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:19.0.0-3538.gcd693b0b.el9.x86_64 7/164 2024-05-08T05:40:29.312 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:19.0.0-3538.gcd693b0b.el9.x86_64 8/164 2024-05-08T05:40:29.312 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:19.0.0-3538.gcd693b0b.el9.x86_64 9/164 2024-05-08T05:40:29.312 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 10/164 2024-05-08T05:40:29.312 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:19.0.0-3538.gcd693b0b.el9.x86_64 11/164 2024-05-08T05:40:29.312 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:19.0.0-3538.gcd693b0b.el9.x86_64 12/164 2024-05-08T05:40:29.312 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 13/164 2024-05-08T05:40:29.312 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-shell-2:19.0.0-3538.gcd693b0b.el9.x86_64 14/164 2024-05-08T05:40:29.312 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:19.0.0-3538.gcd693b0b.el9.x86_ 15/164 2024-05-08T05:40:29.312 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:19.0.0-3538.gcd693b0b.el9.x86_64 16/164 2024-05-08T05:40:29.312 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:19.0.0-3538.gcd693b0b.el9.x86_64 17/164 2024-05-08T05:40:29.312 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:19.0.0-3538.gcd693b0b.el9.x86_6 18/164 2024-05-08T05:40:29.312 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:19.0.0-3538.gcd693b0b.el9.x86 19/164 2024-05-08T05:40:29.313 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:19.0.0-3538.gcd693b0b.el9.x86_64 20/164 2024-05-08T05:40:29.313 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3538.gcd693b0b.el 21/164 2024-05-08T05:40:29.313 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:19.0.0-3538.gcd693b0b.el9. 22/164 2024-05-08T05:40:29.313 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:19.0.0-3538.gcd693b0b.el9.x86_6 23/164 2024-05-08T05:40:29.313 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:19.0.0-3538.gcd693b0b.el9.x86_64 24/164 2024-05-08T05:40:29.313 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 25/164 2024-05-08T05:40:29.313 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 26/164 2024-05-08T05:40:29.313 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 27/164 2024-05-08T05:40:29.313 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 28/164 2024-05-08T05:40:29.313 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 29/164 2024-05-08T05:40:29.313 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3538.gcd693b0b. 30/164 2024-05-08T05:40:29.313 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3538.gcd693b0b.el9.noa 31/164 2024-05-08T05:40:29.313 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3538.gcd693b0b.el9.n 32/164 2024-05-08T05:40:29.313 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3538.gcd6 33/164 2024-05-08T05:40:29.314 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3538.gcd693b0b.el 34/164 2024-05-08T05:40:29.314 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3538.gcd693b0b.el9.noarch 35/164 2024-05-08T05:40:29.314 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3538.gcd693b0b.e 36/164 2024-05-08T05:40:29.314 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:19.0.0-3538.gcd693b0b.el9.noarch 37/164 2024-05-08T05:40:29.314 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:19.0.0-3538.gcd693b0b.el9.noarch 38/164 2024-05-08T05:40:29.314 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-top-2:19.0.0-3538.gcd693b0b.el9.noarch 39/164 2024-05-08T05:40:29.314 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-08T05:40:29.314 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-08T05:40:29.314 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-08T05:40:29.314 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-08T05:40:29.314 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-08T05:40:29.314 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-08T05:40:29.314 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-08T05:40:29.314 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-08T05:40:29.314 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-08T05:40:29.315 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-08T05:40:29.315 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-08T05:40:29.315 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-08T05:40:29.315 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-08T05:40:29.315 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-08T05:40:29.315 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-08T05:40:29.315 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-08T05:40:29.315 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-08T05:40:29.315 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-08T05:40:29.315 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-08T05:40:29.315 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-08T05:40:29.315 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-08T05:40:29.315 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-08T05:40:29.316 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-08T05:40:29.316 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-08T05:40:29.316 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-08T05:40:29.316 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-08T05:40:29.316 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-08T05:40:29.316 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-08T05:40:29.316 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-08T05:40:29.316 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-08T05:40:29.316 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-08T05:40:29.316 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-08T05:40:29.316 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-08T05:40:29.316 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-08T05:40:29.316 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-08T05:40:29.316 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-08T05:40:29.316 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-08T05:40:29.317 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-08T05:40:29.317 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-08T05:40:29.317 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-08T05:40:29.317 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-08T05:40:29.317 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-08T05:40:29.317 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-08T05:40:29.317 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-08T05:40:29.317 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-08T05:40:29.317 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-08T05:40:29.317 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-08T05:40:29.317 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-08T05:40:29.317 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-08T05:40:29.317 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-08T05:40:29.318 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-08T05:40:29.318 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-08T05:40:29.318 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-08T05:40:29.318 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-08T05:40:29.318 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-08T05:40:29.318 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-08T05:40:29.318 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-08T05:40:29.318 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-08T05:40:29.318 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-08T05:40:29.318 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-08T05:40:29.318 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-08T05:40:29.318 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-08T05:40:29.318 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-08T05:40:29.318 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-08T05:40:29.319 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-08T05:40:29.319 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-08T05:40:29.319 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-08T05:40:29.319 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-08T05:40:29.319 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-08T05:40:29.319 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-08T05:40:29.319 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-08T05:40:29.319 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-08T05:40:29.319 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-08T05:40:29.319 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-08T05:40:29.319 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-08T05:40:29.320 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-08T05:40:29.320 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-08T05:40:29.320 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-08T05:40:29.320 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-08T05:40:29.320 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-08T05:40:29.320 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-08T05:40:29.320 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-08T05:40:29.320 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-08T05:40:29.320 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-08T05:40:29.320 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-08T05:40:29.320 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-08T05:40:29.320 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-08T05:40:29.321 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-08T05:40:29.321 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-08T05:40:29.321 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-08T05:40:29.321 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-08T05:40:29.321 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-08T05:40:29.321 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-08T05:40:29.321 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-08T05:40:29.321 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-08T05:40:29.321 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-08T05:40:29.321 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-08T05:40:29.321 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-08T05:40:29.321 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-08T05:40:29.321 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-08T05:40:29.322 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-08T05:40:29.322 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-08T05:40:29.322 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-08T05:40:29.322 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-08T05:40:29.322 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-08T05:40:29.322 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-08T05:40:29.322 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-08T05:40:29.322 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-08T05:40:29.322 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-08T05:40:29.322 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:19.0.0-3538.gcd693b0b.el9.x86_64 149/164 2024-05-08T05:40:29.322 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-08T05:40:29.322 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:19.0.0-3538.gcd693b0b.el9.x86_64 151/164 2024-05-08T05:40:29.322 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-08T05:40:29.322 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-08T05:40:29.323 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-08T05:40:29.323 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-08T05:40:29.323 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-08T05:40:29.323 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-08T05:40:29.323 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-08T05:40:29.323 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-08T05:40:29.323 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-08T05:40:29.323 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-08T05:40:29.323 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-08T05:40:29.323 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-08T05:40:31.816 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:40:31.816 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:31.816 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2024-05-08T05:40:31.816 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-08T05:40:31.816 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-08T05:40:31.816 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-08T05:40:31.816 INFO:teuthology.orchestra.run.smithi045.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-08T05:40:31.816 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-08T05:40:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-08T05:40:31.817 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-08T05:40:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T05:40:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: automake-1.16.2-8.el9.noarch 2024-05-08T05:40:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: bison-3.7.4-5.el9.x86_64 2024-05-08T05:40:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T05:40:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T05:40:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.817 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T05:40:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T05:40:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T05:40:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T05:40:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T05:40:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T05:40:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T05:40:31.818 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T05:40:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top-2:19.0.0-3538.gcd693b0b.el9.noarch 2024-05-08T05:40:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-08T05:40:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: flex-2.6.4-9.el9.x86_64 2024-05-08T05:40:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T05:40:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T05:40:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T05:40:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T05:40:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T05:40:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T05:40:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-08T05:40:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T05:40:31.819 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T05:40:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T05:40:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T05:40:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T05:40:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T05:40:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T05:40:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T05:40:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-08T05:40:31.820 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T05:40:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-08T05:40:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-08T05:40:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T05:40:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T05:40:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T05:40:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T05:40:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T05:40:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T05:40:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-08T05:40:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T05:40:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-13.14-1.el9.x86_64 2024-05-08T05:40:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-08T05:40:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-08T05:40:31.821 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-08T05:40:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T05:40:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T05:40:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T05:40:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-08T05:40:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T05:40:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T05:40:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T05:40:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T05:40:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T05:40:31.822 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T05:40:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T05:40:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-08T05:40:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-08T05:40:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T05:40:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T05:40:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T05:40:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T05:40:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T05:40:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T05:40:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T05:40:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T05:40:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T05:40:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T05:40:31.823 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T05:40:31.824 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T05:40:31.824 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T05:40:31.824 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T05:40:31.824 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T05:40:31.824 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T05:40:31.824 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T05:40:31.824 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T05:40:31.824 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T05:40:31.824 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T05:40:31.824 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T05:40:31.824 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T05:40:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T05:40:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T05:40:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T05:40:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T05:40:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T05:40:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T05:40:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T05:40:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T05:40:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T05:40:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T05:40:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-08T05:40:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T05:40:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.825 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T05:40:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T05:40:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T05:40:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T05:40:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T05:40:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T05:40:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T05:40:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T05:40:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T05:40:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T05:40:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-08T05:40:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T05:40:31.826 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T05:40:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T05:40:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T05:40:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T05:40:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T05:40:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T05:40:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:19.0.0-3538.gcd693b0b.el9.x86_64 2024-05-08T05:40:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T05:40:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T05:40:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T05:40:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-08T05:40:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-08T05:40:31.827 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T05:40:31.828 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-08T05:40:31.828 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:31.828 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-08T05:40:32.241 DEBUG:teuthology.parallel:result is None 2024-05-08T05:40:32.242 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=cd693b0b93278c2b128be2d6d93ae4df18c3ba1c 2024-05-08T05:40:32.385 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T05:40:32.409 INFO:teuthology.orchestra.run.smithi045.stdout:19.0.0-3538.gcd693b0b.el9 2024-05-08T05:40:32.410 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3538.gcd693b0b.el9 2024-05-08T05:40:32.410 INFO:teuthology.task.install:The correct ceph version 19.0.0-3538.gcd693b0b is installed. 2024-05-08T05:40:32.412 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=cd693b0b93278c2b128be2d6d93ae4df18c3ba1c 2024-05-08T05:40:32.623 DEBUG:teuthology.orchestra.run.smithi063:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T05:40:32.650 INFO:teuthology.orchestra.run.smithi063.stdout:19.0.0-3538.gcd693b0b.el9 2024-05-08T05:40:32.650 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3538.gcd693b0b.el9 2024-05-08T05:40:32.651 INFO:teuthology.task.install:The correct ceph version 19.0.0-3538.gcd693b0b is installed. 2024-05-08T05:40:32.652 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T05:40:32.653 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:40:32.653 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T05:40:32.683 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:40:32.683 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T05:40:32.724 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T05:40:32.725 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:40:32.725 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T05:40:32.753 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T05:40:32.820 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:40:32.820 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T05:40:32.850 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T05:40:32.921 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T05:40:32.921 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:40:32.921 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T05:40:32.948 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T05:40:33.014 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:40:33.014 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T05:40:33.044 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T05:40:33.111 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T05:40:33.112 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:40:33.112 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T05:40:33.138 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T05:40:33.205 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:40:33.205 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T05:40:33.231 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T05:40:33.297 INFO:teuthology.run_tasks:Running task ceph... 2024-05-08T05:40:33.377 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-08T05:40:33.378 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /var/log/ceph 2024-05-08T05:40:33.380 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 777 /var/log/ceph 2024-05-08T05:40:33.405 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-08T05:40:33.405 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T05:40:33.447 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T05:40:33.472 INFO:tasks.ceph:Creating extra log directories... 2024-05-08T05:40:33.473 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T05:40:33.514 DEBUG:teuthology.orchestra.run.smithi063:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T05:40:33.545 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-08T05:40:33.546 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-08T05:40:33.546 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-08T05:40:33.546 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:40:33.546 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-08T05:40:33.594 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-08T05:40:33.709 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-08T05:40:33.735 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:40:33.735 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-08T05:40:33.751 DEBUG:teuthology.orchestra.run.smithi063:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-08T05:40:33.864 DEBUG:teuthology.orchestra.run.smithi063:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-08T05:40:33.896 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-08T05:40:33.896 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-08T05:40:33.896 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/jcollin-2024-05-08_05:22:48-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697310', 'branch': 'wip-jcollin-testing-07052024', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub}', 'email': 'jcollin@redhat.com', 'first_in_suite': False, 'job_id': '7697310', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'jcollin-2024-05-08_05:22:48-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-jcollin-testing-07052024'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-jcollin-testing-07052024', 'sha1': 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c'}}, 'owner': 'scheduled_jcollin@teuthology', 'priority': 2, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3880, 'sha1': 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-jcollin-testing-07052024', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_cd693b0b93278c2b128be2d6d93ae4df18c3ba1c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi045.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNg+jL2TFZLqEtYhSHQewN38uACCZFO7eXUwqrpHlx/FTXOWQj+xZ3w69kzHZOiA4tYvSJvJbpniXpITjUVOnWA=', 'smithi063.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJLilVd7/rNRhWYw6MU9+3Ckro0GQceTbVPKn/c5mxyp28sk/YF12goWlIH7ePrfPdpZnK6SAh44avQEtMLye4g='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'cd693b0b93278c2b128be2d6d93ae4df18c3ba1c', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-08_05:22:48', 'tube': 'smithi', 'user': 'jcollin', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-08T05:40:33.897 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T05:40:33.899 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T05:40:33.948 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T05:40:33.956 DEBUG:teuthology.orchestra.run.smithi063:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T05:40:34.014 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:40:34.014 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T05:40:34.036 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T05:40:34.036 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2024-05-08T05:40:34.094 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T05:40:34.094 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:40:34.094 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-08T05:40:34.095 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:40:34.095 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:40:34.095 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-05-08 05:40:24.143212902 +0000 2024-05-08T05:40:34.095 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-05-08 05:37:46.637239419 +0000 2024-05-08T05:40:34.095 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-05-08 05:37:46.637239419 +0000 2024-05-08T05:40:34.095 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-05-08 05:37:46.637239419 +0000 2024-05-08T05:40:34.095 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T05:40:34.159 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-05-08T05:40:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-05-08T05:40:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000105579 s, 4.8 MB/s 2024-05-08T05:40:34.161 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T05:40:34.218 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2024-05-08T05:40:34.274 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T05:40:34.274 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:40:34.274 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-08T05:40:34.274 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:40:34.274 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:40:34.274 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-05-08 05:40:24.144212888 +0000 2024-05-08T05:40:34.275 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-05-08 05:37:46.977234523 +0000 2024-05-08T05:40:34.275 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-05-08 05:37:46.977234523 +0000 2024-05-08T05:40:34.275 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-05-08 05:37:46.977234523 +0000 2024-05-08T05:40:34.275 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T05:40:34.338 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-05-08T05:40:34.338 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-05-08T05:40:34.338 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000103583 s, 4.9 MB/s 2024-05-08T05:40:34.340 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T05:40:34.397 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2024-05-08T05:40:34.453 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T05:40:34.453 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:40:34.454 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-08T05:40:34.454 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:40:34.454 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:40:34.454 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-05-08 05:40:24.144212888 +0000 2024-05-08T05:40:34.454 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-05-08 05:37:47.311229713 +0000 2024-05-08T05:40:34.454 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-05-08 05:37:47.311229713 +0000 2024-05-08T05:40:34.454 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-05-08 05:37:47.311229713 +0000 2024-05-08T05:40:34.454 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T05:40:34.518 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-05-08T05:40:34.519 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-05-08T05:40:34.519 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000103823 s, 4.9 MB/s 2024-05-08T05:40:34.520 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T05:40:34.577 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2024-05-08T05:40:34.633 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T05:40:34.633 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:40:34.633 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-08T05:40:34.633 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:40:34.633 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:40:34.633 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-05-08 05:40:24.144212888 +0000 2024-05-08T05:40:34.634 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-05-08 05:37:47.647224874 +0000 2024-05-08T05:40:34.634 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-05-08 05:37:47.647224874 +0000 2024-05-08T05:40:34.634 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-05-08 05:37:47.647224874 +0000 2024-05-08T05:40:34.634 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T05:40:34.697 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-05-08T05:40:34.697 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-05-08T05:40:34.697 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 9.3046e-05 s, 5.5 MB/s 2024-05-08T05:40:34.699 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T05:40:34.756 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T05:40:34.756 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:40:34.756 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T05:40:34.774 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T05:40:34.775 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_1 2024-05-08T05:40:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T05:40:34.832 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:40:34.832 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-08T05:40:34.832 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:40:34.832 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:40:34.832 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-08 05:40:02.166257383 +0000 2024-05-08T05:40:34.833 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-08 05:37:50.448403336 +0000 2024-05-08T05:40:34.833 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-08 05:37:50.448403336 +0000 2024-05-08T05:40:34.833 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-08 05:37:50.448403336 +0000 2024-05-08T05:40:34.833 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T05:40:34.898 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-08T05:40:34.898 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-08T05:40:34.898 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000225749 s, 2.3 MB/s 2024-05-08T05:40:34.899 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T05:40:34.957 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_2 2024-05-08T05:40:35.014 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T05:40:35.014 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:40:35.014 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-05-08T05:40:35.014 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:40:35.014 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:40:35.014 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-08 05:40:02.166257383 +0000 2024-05-08T05:40:35.014 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-08 05:37:50.800399899 +0000 2024-05-08T05:40:35.014 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-08 05:37:50.800399899 +0000 2024-05-08T05:40:35.014 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-08 05:37:50.800399899 +0000 2024-05-08T05:40:35.014 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T05:40:35.080 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-08T05:40:35.080 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-08T05:40:35.081 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000188654 s, 2.7 MB/s 2024-05-08T05:40:35.082 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T05:40:35.139 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_3 2024-05-08T05:40:35.196 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T05:40:35.196 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:40:35.197 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-08T05:40:35.197 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:40:35.197 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:40:35.197 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-08 05:40:02.167257374 +0000 2024-05-08T05:40:35.197 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-08 05:37:51.148396502 +0000 2024-05-08T05:40:35.197 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-08 05:37:51.148396502 +0000 2024-05-08T05:40:35.197 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-08 05:37:51.148396502 +0000 2024-05-08T05:40:35.197 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T05:40:35.262 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-08T05:40:35.262 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-08T05:40:35.262 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000163114 s, 3.1 MB/s 2024-05-08T05:40:35.263 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T05:40:35.323 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_4 2024-05-08T05:40:35.379 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T05:40:35.379 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:40:35.379 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-08T05:40:35.379 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:40:35.379 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:40:35.379 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-08 05:40:02.167257374 +0000 2024-05-08T05:40:35.379 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-08 05:37:51.502393045 +0000 2024-05-08T05:40:35.380 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-08 05:37:51.502393045 +0000 2024-05-08T05:40:35.380 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-05-08 05:37:51.502393045 +0000 2024-05-08T05:40:35.380 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T05:40:35.443 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-08T05:40:35.443 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-08T05:40:35.443 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000184723 s, 2.8 MB/s 2024-05-08T05:40:35.444 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T05:40:35.502 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T05:40:35.502 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi045.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi063.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T05:40:35.502 INFO:tasks.ceph:Generating config... 2024-05-08T05:40:35.503 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-08T05:40:35.503 INFO:tasks.ceph:[client] debug client = 20 2024-05-08T05:40:35.503 INFO:tasks.ceph:[client] debug ms = 1 2024-05-08T05:40:35.503 INFO:tasks.ceph:[client] pid file = 2024-05-08T05:40:35.503 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-08T05:40:35.503 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-08T05:40:35.503 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-05-08T05:40:35.504 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-08T05:40:35.504 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-08T05:40:35.504 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-08T05:40:35.504 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-08T05:40:35.504 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-08T05:40:35.504 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-08T05:40:35.504 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-08T05:40:35.504 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-08T05:40:35.504 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-08T05:40:35.504 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-08T05:40:35.504 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-08T05:40:35.504 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-08T05:40:35.504 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-08T05:40:35.504 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-08T05:40:35.504 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-08T05:40:35.504 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-08T05:40:35.505 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-08T05:40:35.505 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-08T05:40:35.505 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-08T05:40:35.505 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-08T05:40:35.505 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-05-08T05:40:35.505 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-05-08T05:40:35.505 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-08T05:40:35.505 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-08T05:40:35.505 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-05-08T05:40:35.505 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-08T05:40:35.505 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-08T05:40:35.505 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-08T05:40:35.505 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-08T05:40:35.505 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-08T05:40:35.505 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-08T05:40:35.506 INFO:tasks.ceph:Setting up mon.a... 2024-05-08T05:40:35.506 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-08T05:40:35.558 INFO:teuthology.orchestra.run.smithi045.stdout:creating /etc/ceph/ceph.keyring 2024-05-08T05:40:35.561 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-08T05:40:35.608 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T05:40:35.634 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.45'), ('mon.b', '172.21.15.63'), ('mon.c', '[v2:172.21.15.63:3301,v1:172.21.15.63:6790]')] 2024-05-08T05:40:35.635 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.45,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-08T05:40:35.636 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:40:35.636 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T05:40:35.692 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.45 --add b 172.21.15.63 --addv c '[v2:172.21.15.63:3301,v1:172.21.15.63:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-08T05:40:35.782 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-08T05:40:35.782 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: generated fsid e9e1170a-fd27-4625-97a9-5d5f104d13a2 2024-05-08T05:40:35.782 INFO:teuthology.orchestra.run.smithi045.stdout:setting min_mon_release = quincy 2024-05-08T05:40:35.782 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 0 2024-05-08T05:40:35.782 INFO:teuthology.orchestra.run.smithi045.stdout:fsid e9e1170a-fd27-4625-97a9-5d5f104d13a2 2024-05-08T05:40:35.782 INFO:teuthology.orchestra.run.smithi045.stdout:last_changed 2024-05-08T05:40:35.782506+0000 2024-05-08T05:40:35.782 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-05-08T05:40:35.782506+0000 2024-05-08T05:40:35.782 INFO:teuthology.orchestra.run.smithi045.stdout:min_mon_release 17 (quincy) 2024-05-08T05:40:35.782 INFO:teuthology.orchestra.run.smithi045.stdout:election_strategy: 1 2024-05-08T05:40:35.782 INFO:teuthology.orchestra.run.smithi045.stdout:0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-05-08T05:40:35.782 INFO:teuthology.orchestra.run.smithi045.stdout:1: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.b 2024-05-08T05:40:35.782 INFO:teuthology.orchestra.run.smithi045.stdout:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-08T05:40:35.783 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-08T05:40:35.784 DEBUG:teuthology.orchestra.run.smithi045:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T05:40:35.799 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e9e1170a-fd27-4625-97a9-5d5f104d13a2... 2024-05-08T05:40:35.800 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T05:40:35.842 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T05:40:35.884 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-05-08T05:40:35.884 INFO:teuthology.orchestra.run.smithi045.stdout: chdir = "" 2024-05-08T05:40:35.884 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T05:40:35.885 INFO:teuthology.orchestra.run.smithi045.stdout: auth supported = cephx 2024-05-08T05:40:35.885 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.885 INFO:teuthology.orchestra.run.smithi045.stdout: filestore xattr use omap = true 2024-05-08T05:40:35.885 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.885 INFO:teuthology.orchestra.run.smithi045.stdout: mon clock drift allowed = 1.000 2024-05-08T05:40:35.885 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.885 INFO:teuthology.orchestra.run.smithi045.stdout: osd crush chooseleaf type = 0 2024-05-08T05:40:35.885 INFO:teuthology.orchestra.run.smithi045.stdout: auth debug = true 2024-05-08T05:40:35.885 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.885 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on old message = true 2024-05-08T05:40:35.885 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on bug = true 2024-05-08T05:40:35.885 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.885 INFO:teuthology.orchestra.run.smithi045.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T05:40:35.885 INFO:teuthology.orchestra.run.smithi045.stdout: mon pg warn max object skew = 0 2024-05-08T05:40:35.886 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.886 INFO:teuthology.orchestra.run.smithi045.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T05:40:35.886 INFO:teuthology.orchestra.run.smithi045.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T05:40:35.886 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.886 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default size = 2 2024-05-08T05:40:35.886 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.886 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow primary affinity = true 2024-05-08T05:40:35.886 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow pg remap = true 2024-05-08T05:40:35.886 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on legacy crush tunables = false 2024-05-08T05:40:35.886 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on crush straw calc version zero = false 2024-05-08T05:40:35.886 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on no sortbitwise = false 2024-05-08T05:40:35.886 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on osd down out interval zero = false 2024-05-08T05:40:35.886 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on too few osds = false 2024-05-08T05:40:35.886 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T05:40:35.887 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T05:40:35.887 INFO:teuthology.orchestra.run.smithi045.stdout: mon_allow_pool_size_one = true 2024-05-08T05:40:35.887 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.887 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T05:40:35.887 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.887 INFO:teuthology.orchestra.run.smithi045.stdout: osd default data pool replay window = 5 2024-05-08T05:40:35.887 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.887 INFO:teuthology.orchestra.run.smithi045.stdout: mon allow pool delete = true 2024-05-08T05:40:35.887 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.887 INFO:teuthology.orchestra.run.smithi045.stdout: mon cluster log file level = debug 2024-05-08T05:40:35.887 INFO:teuthology.orchestra.run.smithi045.stdout: debug asserts on shutdown = true 2024-05-08T05:40:35.887 INFO:teuthology.orchestra.run.smithi045.stdout: mon health detail to clog = false 2024-05-08T05:40:35.887 INFO:teuthology.orchestra.run.smithi045.stdout: mon host = "172.21.15.45,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-08T05:40:35.887 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on skipped message = False 2024-05-08T05:40:35.888 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = e9e1170a-fd27-4625-97a9-5d5f104d13a2 2024-05-08T05:40:35.888 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.888 INFO:teuthology.orchestra.run.smithi045.stdout:[osd] 2024-05-08T05:40:35.888 INFO:teuthology.orchestra.run.smithi045.stdout: osd journal size = 100 2024-05-08T05:40:35.888 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.888 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub load threshold = 5.0 2024-05-08T05:40:35.888 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub max interval = 600 2024-05-08T05:40:35.888 INFO:teuthology.orchestra.run.smithi045.stdout: osd mclock profile = high_recovery_ops 2024-05-08T05:40:35.888 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.888 INFO:teuthology.orchestra.run.smithi045.stdout: osd recover clone overlap = true 2024-05-08T05:40:35.888 INFO:teuthology.orchestra.run.smithi045.stdout: osd recovery max chunk = 1048576 2024-05-08T05:40:35.888 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.888 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug shutdown = true 2024-05-08T05:40:35.888 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug op order = true 2024-05-08T05:40:35.888 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify stray on activate = true 2024-05-08T05:40:35.888 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug trim objects = true 2024-05-08T05:40:35.889 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.889 INFO:teuthology.orchestra.run.smithi045.stdout: osd open classes on start = true 2024-05-08T05:40:35.889 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug pg log writeout = true 2024-05-08T05:40:35.889 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.889 INFO:teuthology.orchestra.run.smithi045.stdout: osd deep scrub update digest min age = 30 2024-05-08T05:40:35.889 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.889 INFO:teuthology.orchestra.run.smithi045.stdout: osd map max advance = 10 2024-05-08T05:40:35.889 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.889 INFO:teuthology.orchestra.run.smithi045.stdout: journal zero on create = true 2024-05-08T05:40:35.889 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.889 INFO:teuthology.orchestra.run.smithi045.stdout: filestore ondisk finisher threads = 3 2024-05-08T05:40:35.889 INFO:teuthology.orchestra.run.smithi045.stdout: filestore apply finisher threads = 3 2024-05-08T05:40:35.889 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.889 INFO:teuthology.orchestra.run.smithi045.stdout: bdev debug aio = true 2024-05-08T05:40:35.889 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug misdirected ops = true 2024-05-08T05:40:35.890 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore block size = 96636764160 2024-05-08T05:40:35.890 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore fsck on mount = True 2024-05-08T05:40:35.890 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluefs = 20 2024-05-08T05:40:35.890 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluestore = 20 2024-05-08T05:40:35.890 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:40:35.890 INFO:teuthology.orchestra.run.smithi045.stdout: debug osd = 20 2024-05-08T05:40:35.890 INFO:teuthology.orchestra.run.smithi045.stdout: debug rocksdb = 10 2024-05-08T05:40:35.890 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T05:40:35.890 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd full ratio = 0.9 2024-05-08T05:40:35.890 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd nearfull ratio = 0.8 2024-05-08T05:40:35.890 INFO:teuthology.orchestra.run.smithi045.stdout: osd failsafe full ratio = 0.95 2024-05-08T05:40:35.890 INFO:teuthology.orchestra.run.smithi045.stdout: osd objectstore = bluestore 2024-05-08T05:40:35.890 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-05-08T05:40:35.890 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.891 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr] 2024-05-08T05:40:35.891 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:40:35.891 INFO:teuthology.orchestra.run.smithi045.stdout: debug mgr = 20 2024-05-08T05:40:35.891 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-05-08T05:40:35.891 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-05-08T05:40:35.891 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-05-08T05:40:35.891 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-05-08T05:40:35.891 INFO:teuthology.orchestra.run.smithi045.stdout: mgr/telemetry/nag = false 2024-05-08T05:40:35.891 INFO:teuthology.orchestra.run.smithi045.stdout: debug client = 20 2024-05-08T05:40:35.891 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.891 INFO:teuthology.orchestra.run.smithi045.stdout:[mon] 2024-05-08T05:40:35.891 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:40:35.891 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-05-08T05:40:35.891 INFO:teuthology.orchestra.run.smithi045.stdout: debug paxos = 20 2024-05-08T05:40:35.892 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-05-08T05:40:35.892 INFO:teuthology.orchestra.run.smithi045.stdout: mon data avail warn = 5 2024-05-08T05:40:35.892 INFO:teuthology.orchestra.run.smithi045.stdout: mon mgr mkfs grace = 240 2024-05-08T05:40:35.892 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-05-08T05:40:35.892 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd reporter subtree level = osd 2024-05-08T05:40:35.892 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd prime pg temp = true 2024-05-08T05:40:35.892 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-05-08T05:40:35.892 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.892 INFO:teuthology.orchestra.run.smithi045.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T05:40:35.892 INFO:teuthology.orchestra.run.smithi045.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T05:40:35.892 INFO:teuthology.orchestra.run.smithi045.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T05:40:35.892 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.892 INFO:teuthology.orchestra.run.smithi045.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T05:40:35.892 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T05:40:35.892 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T05:40:35.893 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.893 INFO:teuthology.orchestra.run.smithi045.stdout: # 1m isn't quite enough 2024-05-08T05:40:35.893 INFO:teuthology.orchestra.run.smithi045.stdout: mon_down_mkfs_grace = 2m 2024-05-08T05:40:35.893 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.893 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_filestore_osds = false 2024-05-08T05:40:35.893 INFO:teuthology.orchestra.run.smithi045.stdout: mon down mkfs grace = 300 2024-05-08T05:40:35.893 INFO:teuthology.orchestra.run.smithi045.stdout: mon op complaint time = 120 2024-05-08T05:40:35.893 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:40:35.893 INFO:teuthology.orchestra.run.smithi045.stdout:[client] 2024-05-08T05:40:35.893 INFO:teuthology.orchestra.run.smithi045.stdout: rgw cache enabled = true 2024-05-08T05:40:35.893 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable ops log = true 2024-05-08T05:40:35.893 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable usage log = true 2024-05-08T05:40:35.893 INFO:teuthology.orchestra.run.smithi045.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T05:40:35.893 INFO:teuthology.orchestra.run.smithi045.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T05:40:35.894 INFO:teuthology.orchestra.run.smithi045.stdout: client mount timeout = 600 2024-05-08T05:40:35.894 INFO:teuthology.orchestra.run.smithi045.stdout: debug client = 20 2024-05-08T05:40:35.894 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:40:35.894 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = "" 2024-05-08T05:40:35.894 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-05-08T05:40:35.894 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-05-08T05:40:35.894 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.a] 2024-05-08T05:40:35.894 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.b] 2024-05-08T05:40:35.894 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.c] 2024-05-08T05:40:35.894 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a] 2024-05-08T05:40:35.894 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c] 2024-05-08T05:40:35.894 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b] 2024-05-08T05:40:35.894 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d] 2024-05-08T05:40:35.894 INFO:teuthology.orchestra.run.smithi045.stdout:[mds] 2024-05-08T05:40:35.894 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds = 20 2024-05-08T05:40:35.895 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds balancer = 20 2024-05-08T05:40:35.895 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:40:35.895 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug frag = True 2024-05-08T05:40:35.895 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug scatterstat = True 2024-05-08T05:40:35.895 INFO:teuthology.orchestra.run.smithi045.stdout: mds op complaint time = 180 2024-05-08T05:40:35.895 INFO:teuthology.orchestra.run.smithi045.stdout: mds verify scatter = True 2024-05-08T05:40:35.895 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-05-08T05:40:35.895 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-05-08T05:40:35.895 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-05-08T05:40:35.895 INFO:teuthology.orchestra.run.smithi063.stdout:[global] 2024-05-08T05:40:35.896 INFO:teuthology.orchestra.run.smithi063.stdout: chdir = "" 2024-05-08T05:40:35.896 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T05:40:35.896 INFO:teuthology.orchestra.run.smithi063.stdout: auth supported = cephx 2024-05-08T05:40:35.896 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.896 INFO:teuthology.orchestra.run.smithi063.stdout: filestore xattr use omap = true 2024-05-08T05:40:35.896 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.896 INFO:teuthology.orchestra.run.smithi063.stdout: mon clock drift allowed = 1.000 2024-05-08T05:40:35.896 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.896 INFO:teuthology.orchestra.run.smithi063.stdout: osd crush chooseleaf type = 0 2024-05-08T05:40:35.896 INFO:teuthology.orchestra.run.smithi063.stdout: auth debug = true 2024-05-08T05:40:35.896 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.896 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on old message = true 2024-05-08T05:40:35.896 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on bug = true 2024-05-08T05:40:35.897 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.897 INFO:teuthology.orchestra.run.smithi063.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T05:40:35.897 INFO:teuthology.orchestra.run.smithi063.stdout: mon pg warn max object skew = 0 2024-05-08T05:40:35.897 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.897 INFO:teuthology.orchestra.run.smithi063.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T05:40:35.897 INFO:teuthology.orchestra.run.smithi063.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T05:40:35.897 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.897 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default size = 2 2024-05-08T05:40:35.897 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.897 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow primary affinity = true 2024-05-08T05:40:35.897 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow pg remap = true 2024-05-08T05:40:35.897 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on legacy crush tunables = false 2024-05-08T05:40:35.897 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on crush straw calc version zero = false 2024-05-08T05:40:35.897 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on no sortbitwise = false 2024-05-08T05:40:35.898 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on osd down out interval zero = false 2024-05-08T05:40:35.898 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on too few osds = false 2024-05-08T05:40:35.898 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T05:40:35.898 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T05:40:35.898 INFO:teuthology.orchestra.run.smithi063.stdout: mon_allow_pool_size_one = true 2024-05-08T05:40:35.898 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.898 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T05:40:35.898 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.898 INFO:teuthology.orchestra.run.smithi063.stdout: osd default data pool replay window = 5 2024-05-08T05:40:35.898 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.898 INFO:teuthology.orchestra.run.smithi063.stdout: mon allow pool delete = true 2024-05-08T05:40:35.898 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.898 INFO:teuthology.orchestra.run.smithi063.stdout: mon cluster log file level = debug 2024-05-08T05:40:35.898 INFO:teuthology.orchestra.run.smithi063.stdout: debug asserts on shutdown = true 2024-05-08T05:40:35.899 INFO:teuthology.orchestra.run.smithi063.stdout: mon health detail to clog = false 2024-05-08T05:40:35.899 INFO:teuthology.orchestra.run.smithi063.stdout: mon host = "172.21.15.45,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-08T05:40:35.899 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on skipped message = False 2024-05-08T05:40:35.899 INFO:teuthology.orchestra.run.smithi063.stdout: fsid = e9e1170a-fd27-4625-97a9-5d5f104d13a2 2024-05-08T05:40:35.899 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.899 INFO:teuthology.orchestra.run.smithi063.stdout:[osd] 2024-05-08T05:40:35.899 INFO:teuthology.orchestra.run.smithi063.stdout: osd journal size = 100 2024-05-08T05:40:35.899 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.899 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub load threshold = 5.0 2024-05-08T05:40:35.899 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub max interval = 600 2024-05-08T05:40:35.899 INFO:teuthology.orchestra.run.smithi063.stdout: osd mclock profile = high_recovery_ops 2024-05-08T05:40:35.900 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.900 INFO:teuthology.orchestra.run.smithi063.stdout: osd recover clone overlap = true 2024-05-08T05:40:35.900 INFO:teuthology.orchestra.run.smithi063.stdout: osd recovery max chunk = 1048576 2024-05-08T05:40:35.900 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.900 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug shutdown = true 2024-05-08T05:40:35.900 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug op order = true 2024-05-08T05:40:35.900 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug verify stray on activate = true 2024-05-08T05:40:35.900 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug trim objects = true 2024-05-08T05:40:35.900 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.900 INFO:teuthology.orchestra.run.smithi063.stdout: osd open classes on start = true 2024-05-08T05:40:35.900 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug pg log writeout = true 2024-05-08T05:40:35.900 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.900 INFO:teuthology.orchestra.run.smithi063.stdout: osd deep scrub update digest min age = 30 2024-05-08T05:40:35.900 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.901 INFO:teuthology.orchestra.run.smithi063.stdout: osd map max advance = 10 2024-05-08T05:40:35.901 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.901 INFO:teuthology.orchestra.run.smithi063.stdout: journal zero on create = true 2024-05-08T05:40:35.901 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.901 INFO:teuthology.orchestra.run.smithi063.stdout: filestore ondisk finisher threads = 3 2024-05-08T05:40:35.901 INFO:teuthology.orchestra.run.smithi063.stdout: filestore apply finisher threads = 3 2024-05-08T05:40:35.901 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.901 INFO:teuthology.orchestra.run.smithi063.stdout: bdev debug aio = true 2024-05-08T05:40:35.901 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug misdirected ops = true 2024-05-08T05:40:35.901 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore block size = 96636764160 2024-05-08T05:40:35.901 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore fsck on mount = True 2024-05-08T05:40:35.901 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluefs = 20 2024-05-08T05:40:35.901 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluestore = 20 2024-05-08T05:40:35.901 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:40:35.901 INFO:teuthology.orchestra.run.smithi063.stdout: debug osd = 20 2024-05-08T05:40:35.902 INFO:teuthology.orchestra.run.smithi063.stdout: debug rocksdb = 10 2024-05-08T05:40:35.902 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T05:40:35.902 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd full ratio = 0.9 2024-05-08T05:40:35.902 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd nearfull ratio = 0.8 2024-05-08T05:40:35.902 INFO:teuthology.orchestra.run.smithi063.stdout: osd failsafe full ratio = 0.95 2024-05-08T05:40:35.902 INFO:teuthology.orchestra.run.smithi063.stdout: osd objectstore = bluestore 2024-05-08T05:40:35.902 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-08T05:40:35.902 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.902 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr] 2024-05-08T05:40:35.902 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:40:35.902 INFO:teuthology.orchestra.run.smithi063.stdout: debug mgr = 20 2024-05-08T05:40:35.902 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-08T05:40:35.902 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-08T05:40:35.903 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-08T05:40:35.903 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-08T05:40:35.903 INFO:teuthology.orchestra.run.smithi063.stdout: mgr/telemetry/nag = false 2024-05-08T05:40:35.903 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-08T05:40:35.903 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.903 INFO:teuthology.orchestra.run.smithi063.stdout:[mon] 2024-05-08T05:40:35.903 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:40:35.903 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-08T05:40:35.903 INFO:teuthology.orchestra.run.smithi063.stdout: debug paxos = 20 2024-05-08T05:40:35.903 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-08T05:40:35.903 INFO:teuthology.orchestra.run.smithi063.stdout: mon data avail warn = 5 2024-05-08T05:40:35.903 INFO:teuthology.orchestra.run.smithi063.stdout: mon mgr mkfs grace = 240 2024-05-08T05:40:35.903 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-08T05:40:35.903 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd reporter subtree level = osd 2024-05-08T05:40:35.904 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd prime pg temp = true 2024-05-08T05:40:35.904 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-08T05:40:35.904 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.904 INFO:teuthology.orchestra.run.smithi063.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T05:40:35.904 INFO:teuthology.orchestra.run.smithi063.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T05:40:35.904 INFO:teuthology.orchestra.run.smithi063.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T05:40:35.904 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.904 INFO:teuthology.orchestra.run.smithi063.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T05:40:35.904 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T05:40:35.904 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T05:40:35.904 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.904 INFO:teuthology.orchestra.run.smithi063.stdout: # 1m isn't quite enough 2024-05-08T05:40:35.904 INFO:teuthology.orchestra.run.smithi063.stdout: mon_down_mkfs_grace = 2m 2024-05-08T05:40:35.904 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.905 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_filestore_osds = false 2024-05-08T05:40:35.905 INFO:teuthology.orchestra.run.smithi063.stdout: mon down mkfs grace = 300 2024-05-08T05:40:35.905 INFO:teuthology.orchestra.run.smithi063.stdout: mon op complaint time = 120 2024-05-08T05:40:35.905 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:40:35.905 INFO:teuthology.orchestra.run.smithi063.stdout:[client] 2024-05-08T05:40:35.905 INFO:teuthology.orchestra.run.smithi063.stdout: rgw cache enabled = true 2024-05-08T05:40:35.905 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable ops log = true 2024-05-08T05:40:35.905 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable usage log = true 2024-05-08T05:40:35.905 INFO:teuthology.orchestra.run.smithi063.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T05:40:35.905 INFO:teuthology.orchestra.run.smithi063.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T05:40:35.905 INFO:teuthology.orchestra.run.smithi063.stdout: client mount timeout = 600 2024-05-08T05:40:35.905 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-08T05:40:35.905 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:40:35.905 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = "" 2024-05-08T05:40:35.905 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-08T05:40:35.906 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-08T05:40:35.906 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.a] 2024-05-08T05:40:35.906 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.b] 2024-05-08T05:40:35.906 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.c] 2024-05-08T05:40:35.906 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.a] 2024-05-08T05:40:35.906 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.c] 2024-05-08T05:40:35.906 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.b] 2024-05-08T05:40:35.906 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.d] 2024-05-08T05:40:35.906 INFO:teuthology.orchestra.run.smithi063.stdout:[mds] 2024-05-08T05:40:35.906 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds = 20 2024-05-08T05:40:35.906 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds balancer = 20 2024-05-08T05:40:35.906 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:40:35.906 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug frag = True 2024-05-08T05:40:35.906 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug scatterstat = True 2024-05-08T05:40:35.907 INFO:teuthology.orchestra.run.smithi063.stdout: mds op complaint time = 180 2024-05-08T05:40:35.907 INFO:teuthology.orchestra.run.smithi063.stdout: mds verify scatter = True 2024-05-08T05:40:35.907 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-08T05:40:35.907 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-08T05:40:35.907 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-08T05:40:35.908 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-08T05:40:35.908 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-08T05:40:36.005 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-08T05:40:36.005 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:40:36.005 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-08T05:40:36.021 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:40:36.021 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-08T05:40:36.077 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:40:36.077 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:40:36.077 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T05:40:36.077 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T05:40:36.153 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:40:36.153 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T05:40:36.209 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi063.front.sepia.ceph.com 2024-05-08T05:40:36.209 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:40:36.209 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T05:40:36.209 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T05:40:36.246 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:40:36.247 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T05:40:36.303 INFO:tasks.ceph:Setting up mon nodes... 2024-05-08T05:40:36.303 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-08T05:40:36.304 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T05:40:36.356 INFO:teuthology.orchestra.run.smithi045.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T05:40:36.359 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T05:40:36.415 INFO:teuthology.orchestra.run.smithi063.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T05:40:36.417 INFO:tasks.ceph:Setting up mds nodes... 2024-05-08T05:40:36.417 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-08T05:40:36.470 INFO:teuthology.orchestra.run.smithi045.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-08T05:40:36.473 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-08T05:40:36.498 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-08T05:40:36.590 INFO:teuthology.orchestra.run.smithi045.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-08T05:40:36.593 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-08T05:40:36.618 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-08T05:40:36.671 INFO:teuthology.orchestra.run.smithi063.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-08T05:40:36.674 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-08T05:40:36.699 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-08T05:40:36.792 INFO:teuthology.orchestra.run.smithi063.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-08T05:40:36.795 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-08T05:40:36.820 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-08T05:40:36.820 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T05:40:36.863 INFO:teuthology.orchestra.run.smithi045.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-08T05:40:36.876 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-08T05:40:36.960 INFO:teuthology.orchestra.run.smithi045.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-08T05:40:36.973 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-08T05:40:37.016 INFO:teuthology.orchestra.run.smithi063.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-08T05:40:37.029 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-08T05:40:37.111 INFO:teuthology.orchestra.run.smithi063.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-08T05:40:37.124 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-08T05:40:37.125 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi045.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi063.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T05:40:37.125 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-08T05:40:37.150 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T05:40:37.150 INFO:tasks.ceph:role: osd.0 2024-05-08T05:40:37.150 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:40:37.150 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T05:40:37.215 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T05:40:37.215 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T05:40:37.215 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T05:40:37.215 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T05:40:37.215 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T05:40:37.215 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-05-08T05:40:37.215 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T05:40:37.215 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T05:40:37.215 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T05:40:37.215 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T05:40:37.247 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-05-08T05:40:37.249 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-05-08T05:40:37.249 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-08T05:40:37.330 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-08T05:40:37.400 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-08T05:40:37.466 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T05:40:37.466 INFO:tasks.ceph:role: osd.1 2024-05-08T05:40:37.466 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:40:37.466 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T05:40:37.532 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T05:40:37.532 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T05:40:37.532 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T05:40:37.533 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T05:40:37.533 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T05:40:37.533 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-05-08T05:40:37.533 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T05:40:37.533 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T05:40:37.533 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T05:40:37.533 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T05:40:37.565 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-05-08T05:40:37.567 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-05-08T05:40:37.568 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-08T05:40:37.638 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-08T05:40:37.708 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-08T05:40:37.776 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T05:40:37.776 INFO:tasks.ceph:role: osd.2 2024-05-08T05:40:37.776 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:40:37.776 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T05:40:37.841 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T05:40:37.842 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T05:40:37.842 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T05:40:37.842 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T05:40:37.842 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T05:40:37.842 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-05-08T05:40:37.842 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T05:40:37.842 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T05:40:37.842 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T05:40:37.842 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T05:40:37.878 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-05-08T05:40:37.881 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-05-08T05:40:37.881 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-08T05:40:37.950 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-08T05:40:38.020 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-08T05:40:38.087 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T05:40:38.087 INFO:tasks.ceph:role: osd.3 2024-05-08T05:40:38.087 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:40:38.087 DEBUG:teuthology.orchestra.run.smithi045:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T05:40:38.152 INFO:teuthology.orchestra.run.smithi045.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T05:40:38.152 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T05:40:38.152 INFO:teuthology.orchestra.run.smithi045.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T05:40:38.152 INFO:teuthology.orchestra.run.smithi045.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T05:40:38.152 INFO:teuthology.orchestra.run.smithi045.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T05:40:38.152 INFO:teuthology.orchestra.run.smithi045.stdout: = sunit=0 swidth=0 blks 2024-05-08T05:40:38.152 INFO:teuthology.orchestra.run.smithi045.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T05:40:38.153 INFO:teuthology.orchestra.run.smithi045.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T05:40:38.153 INFO:teuthology.orchestra.run.smithi045.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T05:40:38.153 INFO:teuthology.orchestra.run.smithi045.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T05:40:38.185 INFO:teuthology.orchestra.run.smithi045.stdout:Discarding blocks...Done. 2024-05-08T05:40:38.187 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi045.front.sepia.ceph.com -o noatime 2024-05-08T05:40:38.188 DEBUG:teuthology.orchestra.run.smithi045:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-08T05:40:38.267 DEBUG:teuthology.orchestra.run.smithi045:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-08T05:40:38.338 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T05:40:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:38.431+0000 7f42013b6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-08T05:40:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:38.431+0000 7f42013b6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-08T05:40:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:38.431+0000 7f42013b6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T05:40:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:38.431+0000 7f42013b6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T05:40:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:38.431+0000 7f42013b6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T05:40:38.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:38.432+0000 7f42013b6640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-08T05:40:41.282 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-08T05:40:41.313 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T05:40:41.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:41.406+0000 7fa65ee1a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-08T05:40:41.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:41.406+0000 7fa65ee1a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-08T05:40:41.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:41.406+0000 7fa65ee1a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T05:40:41.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:41.406+0000 7fa65ee1a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T05:40:41.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:41.406+0000 7fa65ee1a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T05:40:41.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:41.406+0000 7fa65ee1a640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-08T05:40:44.259 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-08T05:40:44.290 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T05:40:44.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:44.380+0000 7f0339a9d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-08T05:40:44.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:44.381+0000 7f0339a9d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-08T05:40:44.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:44.381+0000 7f0339a9d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T05:40:44.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:44.381+0000 7f0339a9d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T05:40:44.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:44.381+0000 7f0339a9d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T05:40:44.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:44.381+0000 7f0339a9d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-08T05:40:47.233 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-08T05:40:47.263 DEBUG:teuthology.orchestra.run.smithi045:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T05:40:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:47.356+0000 7f8757ead640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-08T05:40:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:47.356+0000 7f8757ead640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-08T05:40:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:47.357+0000 7f8757ead640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T05:40:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:47.357+0000 7f8757ead640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T05:40:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:47.357+0000 7f8757ead640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T05:40:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:40:47.357+0000 7f8757ead640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-08T05:40:50.204 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-08T05:40:50.235 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-08T05:40:50.267 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T05:40:50.267 INFO:tasks.ceph:role: osd.4 2024-05-08T05:40:50.267 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-08T05:40:50.267 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T05:40:50.345 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T05:40:50.345 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T05:40:50.345 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T05:40:50.345 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T05:40:50.345 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T05:40:50.345 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-05-08T05:40:50.345 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T05:40:50.346 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T05:40:50.346 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T05:40:50.346 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T05:40:50.861 INFO:teuthology.orchestra.run.smithi063.stdout:Discarding blocks...Done. 2024-05-08T05:40:50.863 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-05-08T05:40:50.864 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-08T05:40:50.992 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-08T05:40:51.029 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-08T05:40:51.099 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T05:40:51.099 INFO:tasks.ceph:role: osd.5 2024-05-08T05:40:51.099 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-08T05:40:51.099 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T05:40:51.176 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T05:40:51.176 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T05:40:51.176 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T05:40:51.176 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T05:40:51.176 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T05:40:51.176 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-05-08T05:40:51.176 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T05:40:51.176 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T05:40:51.176 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T05:40:51.176 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T05:40:51.622 INFO:teuthology.orchestra.run.smithi063.stdout:Discarding blocks...Done. 2024-05-08T05:40:51.625 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-05-08T05:40:51.625 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-08T05:40:51.722 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-08T05:40:51.755 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-08T05:40:51.824 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T05:40:51.824 INFO:tasks.ceph:role: osd.6 2024-05-08T05:40:51.824 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-08T05:40:51.824 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T05:40:51.899 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T05:40:51.900 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T05:40:51.900 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T05:40:51.900 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T05:40:51.900 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T05:40:51.900 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-05-08T05:40:51.900 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T05:40:51.900 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T05:40:51.900 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T05:40:51.900 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T05:40:52.392 INFO:teuthology.orchestra.run.smithi063.stdout:Discarding blocks...Done. 2024-05-08T05:40:52.394 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-05-08T05:40:52.394 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-08T05:40:52.528 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-08T05:40:52.562 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-08T05:40:52.633 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T05:40:52.633 INFO:tasks.ceph:role: osd.7 2024-05-08T05:40:52.633 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-08T05:40:52.633 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T05:40:52.710 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T05:40:52.710 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T05:40:52.710 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T05:40:52.710 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T05:40:52.710 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T05:40:52.710 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-05-08T05:40:52.710 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T05:40:52.710 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T05:40:52.710 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T05:40:52.710 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T05:40:53.199 INFO:teuthology.orchestra.run.smithi063.stdout:Discarding blocks...Done. 2024-05-08T05:40:53.201 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-05-08T05:40:53.201 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-08T05:40:53.330 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-08T05:40:53.367 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T05:40:53.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:40:53.467+0000 7f981ece1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-08T05:40:53.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:40:53.467+0000 7f981ece1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-08T05:40:53.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:40:53.467+0000 7f981ece1640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T05:40:53.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:40:53.467+0000 7f981ece1640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T05:40:53.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:40:53.467+0000 7f981ece1640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T05:40:53.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:40:53.468+0000 7f981ece1640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-08T05:40:56.322 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-08T05:40:56.353 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T05:40:56.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:40:56.447+0000 7fd9accea640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-08T05:40:56.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:40:56.447+0000 7fd9accea640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-08T05:40:56.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:40:56.447+0000 7fd9accea640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T05:40:56.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:40:56.447+0000 7fd9accea640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T05:40:56.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:40:56.447+0000 7fd9accea640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T05:40:56.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:40:56.447+0000 7fd9accea640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-08T05:40:59.301 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-08T05:40:59.332 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T05:40:59.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:40:59.424+0000 7fc87c6f0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-08T05:40:59.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:40:59.424+0000 7fc87c6f0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-08T05:40:59.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:40:59.424+0000 7fc87c6f0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T05:40:59.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:40:59.424+0000 7fc87c6f0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T05:40:59.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:40:59.424+0000 7fc87c6f0640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T05:40:59.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:40:59.425+0000 7fc87c6f0640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-08T05:41:02.304 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-08T05:41:02.334 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T05:41:02.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:02.427+0000 7ff9fa2fd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-08T05:41:02.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:02.427+0000 7ff9fa2fd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-08T05:41:02.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:02.427+0000 7ff9fa2fd640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T05:41:02.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:02.427+0000 7ff9fa2fd640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T05:41:02.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:02.427+0000 7ff9fa2fd640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T05:41:02.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:02.427+0000 7ff9fa2fd640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-08T05:41:03.898 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:41:03.901 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:41:05.283 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-08T05:41:05.314 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-08T05:41:05.314 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:41:05.314 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-08T05:41:05.345 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:41:05.345 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-08T05:41:05.415 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:41:05.415 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-08T05:41:05.484 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:41:05.484 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-08T05:41:05.552 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:41:05.552 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-08T05:41:05.619 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:41:05.620 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-08T05:41:05.687 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:41:05.688 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-08T05:41:05.755 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:41:05.755 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-08T05:41:05.786 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:41:05.786 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-08T05:41:05.856 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:41:05.856 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-08T05:41:05.925 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:41:05.926 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-08T05:41:05.994 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:41:05.994 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-08T05:41:06.061 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:41:06.062 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-08T05:41:06.129 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:41:06.129 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-08T05:41:06.196 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:41:06.196 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-08T05:41:06.215 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:41:06.215 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-08T05:41:06.273 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:41:06.274 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-08T05:41:06.290 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:41:06.290 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-08T05:41:06.347 INFO:tasks.ceph:Adding keys to all mons... 2024-05-08T05:41:06.347 DEBUG:teuthology.orchestra.run.smithi045:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T05:41:06.349 DEBUG:teuthology.orchestra.run.smithi063:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T05:41:06.391 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr.y] 2024-05-08T05:41:06.391 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBUEDtmfM5OFRAAgdJPUfqmnKU8vRrjGMNwrw== 2024-05-08T05:41:06.391 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a] 2024-05-08T05:41:06.391 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBUEDtmrRgXHBAAX1mgdmOD9JC4S1vJNyLSQw== 2024-05-08T05:41:06.391 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c] 2024-05-08T05:41:06.391 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBUEDtmeiJCIxAAr3Br1fJVgR9TW2Q+4tMQBQ== 2024-05-08T05:41:06.391 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.0] 2024-05-08T05:41:06.391 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBWEDtmLdHHGRAAA1o4Z/lTGERIZhulqVtybw== 2024-05-08T05:41:06.391 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.1] 2024-05-08T05:41:06.391 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBZEDtmMrRVGBAAeKBtcf5fSQMFE6IStUvokQ== 2024-05-08T05:41:06.391 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.2] 2024-05-08T05:41:06.391 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBcEDtmpozQFhAAS6AF6X592Wh/QiBipcOVew== 2024-05-08T05:41:06.392 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.3] 2024-05-08T05:41:06.392 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBfEDtm47lSFRAAt5tvpRCljxGtAmHQqcFj2A== 2024-05-08T05:41:06.392 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr.x] 2024-05-08T05:41:06.392 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBUEDtmL0XLGBAA1BSFdyiDYV9Je+igOVBvlQ== 2024-05-08T05:41:06.392 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b] 2024-05-08T05:41:06.392 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBUEDtmvn8SKBAAVkXYBbTfu98gl0AptOfJEg== 2024-05-08T05:41:06.392 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d] 2024-05-08T05:41:06.392 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBUEDtmXdFILxAAxteIxUYeFdQKd/oxHLpeOQ== 2024-05-08T05:41:06.392 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.4] 2024-05-08T05:41:06.392 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBlEDtmDQ7mGxAA5z7rHZRf7X5g7ZiqEoO2DA== 2024-05-08T05:41:06.392 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.5] 2024-05-08T05:41:06.392 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBoEDtmBdi0GhAAVMfg+KEJHRvaiS49PEboOg== 2024-05-08T05:41:06.392 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.6] 2024-05-08T05:41:06.392 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBrEDtmG2daGRAANQgJFAT6gNE/vPofCv6veA== 2024-05-08T05:41:06.392 INFO:teuthology.orchestra.run.smithi045.stdout:[osd.7] 2024-05-08T05:41:06.393 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBuEDtm3tmDGRAARsAzm9KIzgKAs4ZtfCVdaQ== 2024-05-08T05:41:06.393 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-05-08T05:41:06.393 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBUEDtmDcqGMxAALgQaBYp4Zd8uMqswCXvGaw== 2024-05-08T05:41:06.393 INFO:teuthology.orchestra.run.smithi045.stdout:[client.2] 2024-05-08T05:41:06.393 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBUEDtmAMtIORAAJf8oSwHqSzgjsTuqxtQqHQ== 2024-05-08T05:41:06.393 INFO:teuthology.orchestra.run.smithi045.stdout:[client.1] 2024-05-08T05:41:06.393 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBVEDtm6z0GARAARgRma5hPpH6X95pE5NNrQw== 2024-05-08T05:41:06.393 INFO:teuthology.orchestra.run.smithi045.stdout:[client.3] 2024-05-08T05:41:06.393 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBVEDtm9qK1BhAAokvkJZmNxT5WER9/+EKIpA== 2024-05-08T05:41:06.412 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr.y] 2024-05-08T05:41:06.412 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBUEDtmfM5OFRAAgdJPUfqmnKU8vRrjGMNwrw== 2024-05-08T05:41:06.412 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.a] 2024-05-08T05:41:06.412 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBUEDtmrRgXHBAAX1mgdmOD9JC4S1vJNyLSQw== 2024-05-08T05:41:06.412 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.c] 2024-05-08T05:41:06.412 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBUEDtmeiJCIxAAr3Br1fJVgR9TW2Q+4tMQBQ== 2024-05-08T05:41:06.412 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.0] 2024-05-08T05:41:06.412 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBWEDtmLdHHGRAAA1o4Z/lTGERIZhulqVtybw== 2024-05-08T05:41:06.412 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.1] 2024-05-08T05:41:06.412 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBZEDtmMrRVGBAAeKBtcf5fSQMFE6IStUvokQ== 2024-05-08T05:41:06.413 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.2] 2024-05-08T05:41:06.413 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBcEDtmpozQFhAAS6AF6X592Wh/QiBipcOVew== 2024-05-08T05:41:06.413 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.3] 2024-05-08T05:41:06.413 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBfEDtm47lSFRAAt5tvpRCljxGtAmHQqcFj2A== 2024-05-08T05:41:06.413 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr.x] 2024-05-08T05:41:06.413 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBUEDtmL0XLGBAA1BSFdyiDYV9Je+igOVBvlQ== 2024-05-08T05:41:06.413 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.b] 2024-05-08T05:41:06.413 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBUEDtmvn8SKBAAVkXYBbTfu98gl0AptOfJEg== 2024-05-08T05:41:06.413 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.d] 2024-05-08T05:41:06.413 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBUEDtmXdFILxAAxteIxUYeFdQKd/oxHLpeOQ== 2024-05-08T05:41:06.413 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.4] 2024-05-08T05:41:06.413 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBlEDtmDQ7mGxAA5z7rHZRf7X5g7ZiqEoO2DA== 2024-05-08T05:41:06.413 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.5] 2024-05-08T05:41:06.413 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBoEDtmBdi0GhAAVMfg+KEJHRvaiS49PEboOg== 2024-05-08T05:41:06.413 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.6] 2024-05-08T05:41:06.414 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBrEDtmG2daGRAANQgJFAT6gNE/vPofCv6veA== 2024-05-08T05:41:06.414 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.7] 2024-05-08T05:41:06.414 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBuEDtm3tmDGRAARsAzm9KIzgKAs4ZtfCVdaQ== 2024-05-08T05:41:06.414 INFO:teuthology.orchestra.run.smithi063.stdout:[client.0] 2024-05-08T05:41:06.414 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBUEDtmDcqGMxAALgQaBYp4Zd8uMqswCXvGaw== 2024-05-08T05:41:06.414 INFO:teuthology.orchestra.run.smithi063.stdout:[client.2] 2024-05-08T05:41:06.414 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBUEDtmAMtIORAAJf8oSwHqSzgjsTuqxtQqHQ== 2024-05-08T05:41:06.414 INFO:teuthology.orchestra.run.smithi063.stdout:[client.1] 2024-05-08T05:41:06.414 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBVEDtm6z0GARAARgRma5hPpH6X95pE5NNrQw== 2024-05-08T05:41:06.414 INFO:teuthology.orchestra.run.smithi063.stdout:[client.3] 2024-05-08T05:41:06.414 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBVEDtm9qK1BhAAokvkJZmNxT5WER9/+EKIpA== 2024-05-08T05:41:06.415 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T05:41:06.437 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T05:41:06.509 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T05:41:06.511 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T05:41:06.567 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T05:41:06.569 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T05:41:06.624 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T05:41:06.626 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T05:41:06.678 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T05:41:06.681 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T05:41:06.733 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T05:41:06.735 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T05:41:06.787 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T05:41:06.791 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T05:41:06.843 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T05:41:06.845 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T05:41:06.898 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T05:41:06.900 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T05:41:06.955 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T05:41:06.957 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-08T05:41:07.012 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T05:41:07.015 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T05:41:07.068 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T05:41:07.070 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T05:41:07.122 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T05:41:07.124 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T05:41:07.176 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T05:41:07.178 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T05:41:07.230 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T05:41:07.232 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T05:41:07.288 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T05:41:07.290 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T05:41:07.345 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T05:41:07.347 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T05:41:07.402 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T05:41:07.405 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T05:41:07.460 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-08T05:41:07.460 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-08T05:41:07.486 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T05:41:07.584 DEBUG:teuthology.orchestra.run.smithi045:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-08T05:41:07.610 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-08T05:41:07.636 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T05:41:07.733 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-08T05:41:07.759 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-08T05:41:07.825 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T05:41:07.922 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-08T05:41:07.948 DEBUG:teuthology.orchestra.run.smithi045:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T05:41:07.950 DEBUG:teuthology.orchestra.run.smithi063:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T05:41:08.004 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-08T05:41:08.004 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T05:41:08.004 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T05:41:08.007 INFO:tasks.ceph.mon.a:Started 2024-05-08T05:41:08.007 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-08T05:41:08.007 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-08T05:41:08.047 INFO:tasks.ceph.mon.b:Started 2024-05-08T05:41:08.047 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T05:41:08.047 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T05:41:08.049 INFO:tasks.ceph.mon.c:Started 2024-05-08T05:41:08.049 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-08T05:41:08.049 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-08T05:41:08.049 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-08T05:41:08.053 INFO:tasks.ceph.mgr.y:Started 2024-05-08T05:41:08.053 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-08T05:41:08.053 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-08T05:41:08.055 INFO:tasks.ceph.mgr.x:Started 2024-05-08T05:41:08.055 INFO:tasks.ceph:Setting crush tunables to default 2024-05-08T05:41:08.056 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph osd crush tunables default 2024-05-08T05:41:08.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.160+0000 7fc77a0c4640 1 Processor -- start 2024-05-08T05:41:08.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.160+0000 7fc77a0c4640 1 -- start start 2024-05-08T05:41:08.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.160+0000 7fc77a0c4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7740b1050 0x7fc774071720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:08.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.160+0000 7fc77a0c4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc77415f680 0x7fc7740b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:08.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.161+0000 7fc77a0c4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc77415fae0 con 0x7fc7740b5bf0 2024-05-08T05:41:08.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.161+0000 7fc77a0c4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc774160600 con 0x7fc77415f680 2024-05-08T05:41:08.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.161+0000 7fc77a0c4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc77415fd60 con 0x7fc7740b1050 2024-05-08T05:41:08.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.161+0000 7fc773fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc7740b5bf0 0x7fc77406c3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50282/0 (socket says 172.21.15.45:50282) 2024-05-08T05:41:08.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.161+0000 7fc773fff640 1 -- 172.21.15.45:0/2078471497 learned_addr learned my addr 172.21.15.45:0/2078471497 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:08.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.161+0000 7fc7737fe640 1 --2- 172.21.15.45:0/2078471497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7740b1050 0x7fc774071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:08.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.161+0000 7fc772ffd640 1 --2- 172.21.15.45:0/2078471497 >> v2:172.21.15.63:3300/0 conn(0x7fc77415f680 0x7fc7740b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:08.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.161+0000 7fc7737fe640 1 -- 172.21.15.45:0/2078471497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7740b1050 msgr2=0x7fc774071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T05:41:08.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.161+0000 7fc7737fe640 1 -- 172.21.15.45:0/2078471497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7740b1050 msgr2=0x7fc774071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T05:41:08.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.161+0000 7fc7737fe640 1 --2- 172.21.15.45:0/2078471497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7740b1050 0x7fc774071720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T05:41:08.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.161+0000 7fc7737fe640 1 --2- 172.21.15.45:0/2078471497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7740b1050 0x7fc774071720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:41:08.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.161+0000 7fc772ffd640 1 -- 172.21.15.45:0/2078471497 >> v2:172.21.15.63:3300/0 conn(0x7fc77415f680 msgr2=0x7fc7740b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T05:41:08.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.161+0000 7fc772ffd640 1 -- 172.21.15.45:0/2078471497 >> v2:172.21.15.63:3300/0 conn(0x7fc77415f680 msgr2=0x7fc7740b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T05:41:08.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.161+0000 7fc772ffd640 1 --2- 172.21.15.45:0/2078471497 >> v2:172.21.15.63:3300/0 conn(0x7fc77415f680 0x7fc7740b56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T05:41:08.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.161+0000 7fc772ffd640 1 --2- 172.21.15.45:0/2078471497 >> v2:172.21.15.63:3300/0 conn(0x7fc77415f680 0x7fc7740b56b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:41:08.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.317+0000 7fc7727fc640 1 -- 172.21.15.45:0/2078471497 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4199364874 0 0) 0x7fc76c001490 con 0x7fc7740b5bf0 2024-05-08T05:41:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.317+0000 7fc7727fc640 1 -- 172.21.15.45:0/2078471497 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc75c003520 con 0x7fc7740b5bf0 2024-05-08T05:41:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.317+0000 7fc7727fc640 1 -- 172.21.15.45:0/2078471497 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3910594143 0 0) 0x7fc75c003520 con 0x7fc7740b5bf0 2024-05-08T05:41:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.317+0000 7fc7727fc640 1 -- 172.21.15.45:0/2078471497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7740b1050 msgr2=0x7fc774071720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T05:41:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.317+0000 7fc7727fc640 1 --2- 172.21.15.45:0/2078471497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7740b1050 0x7fc774071720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.317+0000 7fc7727fc640 1 -- 172.21.15.45:0/2078471497 >> v2:172.21.15.63:3300/0 conn(0x7fc77415f680 msgr2=0x7fc7740b56b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T05:41:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.317+0000 7fc7727fc640 1 --2- 172.21.15.45:0/2078471497 >> v2:172.21.15.63:3300/0 conn(0x7fc77415f680 0x7fc7740b56b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.317+0000 7fc7727fc640 1 -- 172.21.15.45:0/2078471497 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc774160040 con 0x7fc7740b5bf0 2024-05-08T05:41:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.317+0000 7fc7727fc640 1 -- 172.21.15.45:0/2078471497 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fc76c001c40 con 0x7fc7740b5bf0 2024-05-08T05:41:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.318+0000 7fc7727fc640 1 -- 172.21.15.45:0/2078471497 >> v1:172.21.15.45:6789/0 conn(0x7fc7740b5bf0 legacy=0x7fc77406c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.318+0000 7fc7727fc640 1 --2- 172.21.15.45:0/2078471497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7740b1050 0x7fc75c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.318+0000 7fc7727fc640 1 --2- 172.21.15.45:0/2078471497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc75c004080 0x7fc75c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.318+0000 7fc7727fc640 1 --2- 172.21.15.45:0/2078471497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc75c004d40 0x7fc75c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.318+0000 7fc7727fc640 1 -- 172.21.15.45:0/2078471497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc774160600 con 0x7fc75c004080 2024-05-08T05:41:08.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.318+0000 7fc7727fc640 1 -- 172.21.15.45:0/2078471497 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc77415fd60 con 0x7fc75c004d40 2024-05-08T05:41:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.318+0000 7fc7727fc640 1 -- 172.21.15.45:0/2078471497 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc75c00bb80 con 0x7fc7740b1050 2024-05-08T05:41:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.318+0000 7fc772ffd640 1 --2- 172.21.15.45:0/2078471497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc75c004080 0x7fc75c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.318+0000 7fc7737fe640 1 --2- 172.21.15.45:0/2078471497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7740b1050 0x7fc75c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.318+0000 7fc773fff640 1 --2- 172.21.15.45:0/2078471497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc75c004d40 0x7fc75c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.318+0000 7fc772ffd640 1 -- 172.21.15.45:0/2078471497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7740b1050 msgr2=0x7fc75c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.318+0000 7fc772ffd640 1 --2- 172.21.15.45:0/2078471497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7740b1050 0x7fc75c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.318+0000 7fc772ffd640 1 -- 172.21.15.45:0/2078471497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc75c004d40 msgr2=0x7fc75c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.318+0000 7fc772ffd640 1 --2- 172.21.15.45:0/2078471497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc75c004d40 0x7fc75c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.318+0000 7fc772ffd640 1 -- 172.21.15.45:0/2078471497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75c003520 con 0x7fc75c004080 2024-05-08T05:41:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.318+0000 7fc773fff640 1 --2- 172.21.15.45:0/2078471497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc75c004d40 0x7fc75c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.318+0000 7fc7737fe640 1 --2- 172.21.15.45:0/2078471497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7740b1050 0x7fc75c003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.319+0000 7fc772ffd640 1 --2- 172.21.15.45:0/2078471497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc75c004080 0x7fc75c004800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc768006fd0 tx=0x7fc75c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cfb2c396840f09 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.319+0000 7fc7727fc640 1 -- 172.21.15.45:0/2078471497 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc768030070 con 0x7fc75c004080 2024-05-08T05:41:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.319+0000 7fc7727fc640 1 -- 172.21.15.45:0/2078471497 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc76802acc0 con 0x7fc75c004080 2024-05-08T05:41:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.319+0000 7fc7727fc640 1 -- 172.21.15.45:0/2078471497 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76800b5e0 con 0x7fc75c004080 2024-05-08T05:41:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.320+0000 7fc77a0c4640 1 -- 172.21.15.45:0/2078471497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc75c004080 msgr2=0x7fc75c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.320+0000 7fc77a0c4640 1 --2- 172.21.15.45:0/2078471497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc75c004080 0x7fc75c004800 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc768006fd0 tx=0x7fc75c005830 comp rx=0 tx=0).stop 2024-05-08T05:41:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.320+0000 7fc773fff640 1 -- 172.21.15.45:0/2078471497 reap_dead start 2024-05-08T05:41:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.320+0000 7fc77a0c4640 1 -- 172.21.15.45:0/2078471497 shutdown_connections 2024-05-08T05:41:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.320+0000 7fc77a0c4640 1 -- 172.21.15.45:0/2078471497 >> 172.21.15.45:0/2078471497 conn(0x7fc7740b3c40 msgr2=0x7fc7740b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.320+0000 7fc77a0c4640 1 -- 172.21.15.45:0/2078471497 shutdown_connections 2024-05-08T05:41:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.320+0000 7fc77a0c4640 1 -- 172.21.15.45:0/2078471497 wait complete. 2024-05-08T05:41:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.321+0000 7fc77a0c4640 1 Processor -- start 2024-05-08T05:41:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.321+0000 7fc77a0c4640 1 -- start start 2024-05-08T05:41:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.322+0000 7fc77a0c4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc75c004080 0x7fc77414bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.322+0000 7fc77a0c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7741f6300 0x7fc77414edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.322+0000 7fc77a0c4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7741f6780 0x7fc77414f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.322+0000 7fc77a0c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc7740701e0 con 0x7fc7741f6300 2024-05-08T05:41:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.322+0000 7fc77a0c4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc774162d10 con 0x7fc75c004080 2024-05-08T05:41:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.322+0000 7fc77a0c4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc7740b1430 con 0x7fc7741f6780 2024-05-08T05:41:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.322+0000 7fc772ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7741f6300 0x7fc77414edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.322+0000 7fc772ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7741f6300 0x7fc77414edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34912/0 (socket says 172.21.15.45:34912) 2024-05-08T05:41:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.322+0000 7fc772ffd640 1 -- 172.21.15.45:0/81356226 learned_addr learned my addr 172.21.15.45:0/81356226 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.322+0000 7fc7737fe640 1 --2- 172.21.15.45:0/81356226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc75c004080 0x7fc77414bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.322+0000 7fc773fff640 1 --2- 172.21.15.45:0/81356226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7741f6780 0x7fc77414f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.322+0000 7fc772ffd640 1 -- 172.21.15.45:0/81356226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7741f6780 msgr2=0x7fc77414f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.322+0000 7fc772ffd640 1 --2- 172.21.15.45:0/81356226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7741f6780 0x7fc77414f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.322+0000 7fc772ffd640 1 -- 172.21.15.45:0/81356226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc75c004080 msgr2=0x7fc77414bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.322+0000 7fc772ffd640 1 --2- 172.21.15.45:0/81356226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc75c004080 0x7fc77414bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.322+0000 7fc772ffd640 1 -- 172.21.15.45:0/81356226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77414c170 con 0x7fc7741f6300 2024-05-08T05:41:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.322+0000 7fc772ffd640 1 --2- 172.21.15.45:0/81356226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7741f6300 0x7fc77414edf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc768036040 tx=0x7fc768004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.323+0000 7fc75bfff640 1 -- 172.21.15.45:0/81356226 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc768043020 con 0x7fc7741f6300 2024-05-08T05:41:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.323+0000 7fc75bfff640 1 -- 172.21.15.45:0/81356226 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc76803f680 con 0x7fc7741f6300 2024-05-08T05:41:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.323+0000 7fc77a0c4640 1 -- 172.21.15.45:0/81356226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc77414c750 con 0x7fc7741f6300 2024-05-08T05:41:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.323+0000 7fc75bfff640 1 -- 172.21.15.45:0/81356226 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76803fa30 con 0x7fc7741f6300 2024-05-08T05:41:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.323+0000 7fc77a0c4640 1 -- 172.21.15.45:0/81356226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc77414cd10 con 0x7fc7741f6300 2024-05-08T05:41:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.323+0000 7fc75bfff640 1 -- 172.21.15.45:0/81356226 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc768046020 con 0x7fc7741f6300 2024-05-08T05:41:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.323+0000 7fc75bfff640 1 -- 172.21.15.45:0/81356226 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc768048400 con 0x7fc7741f6300 2024-05-08T05:41:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.324+0000 7fc77a0c4640 1 -- 172.21.15.45:0/81356226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7740507e0 con 0x7fc7741f6300 2024-05-08T05:41:08.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.326+0000 7fc75bfff640 1 -- 172.21.15.45:0/81356226 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc7740507e0 con 0x7fc7741f6300 2024-05-08T05:41:08.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.394+0000 7fc77a0c4640 1 -- 172.21.15.45:0/81356226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fc774069b00 con 0x7fc7741f6300 2024-05-08T05:41:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.396+0000 7fc75bfff640 1 -- 172.21.15.45:0/81356226 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fc76804d030 con 0x7fc7741f6300 2024-05-08T05:41:08.396 INFO:teuthology.orchestra.run.smithi045.stderr:adjusted tunables profile to default 2024-05-08T05:41:08.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.397+0000 7fc77a0c4640 1 -- 172.21.15.45:0/81356226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7741f6300 msgr2=0x7fc77414edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:08.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.397+0000 7fc77a0c4640 1 --2- 172.21.15.45:0/81356226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7741f6300 0x7fc77414edf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc768036040 tx=0x7fc768004730 comp rx=0 tx=0).stop 2024-05-08T05:41:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.397+0000 7fc77a0c4640 1 -- 172.21.15.45:0/81356226 shutdown_connections 2024-05-08T05:41:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.397+0000 7fc77a0c4640 1 --2- 172.21.15.45:0/81356226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7741f6780 0x7fc77414f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.397+0000 7fc77a0c4640 1 --2- 172.21.15.45:0/81356226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7741f6300 0x7fc77414edf0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.397+0000 7fc77a0c4640 1 --2- 172.21.15.45:0/81356226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc75c004080 0x7fc77414bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.397+0000 7fc77a0c4640 1 -- 172.21.15.45:0/81356226 >> 172.21.15.45:0/81356226 conn(0x7fc7740b3c40 msgr2=0x7fc7740b5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.397+0000 7fc77a0c4640 1 -- 172.21.15.45:0/81356226 shutdown_connections 2024-05-08T05:41:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:08.397+0000 7fc77a0c4640 1 -- 172.21.15.45:0/81356226 wait complete. 2024-05-08T05:41:08.408 INFO:tasks.ceph:check_enable_crimson: False 2024-05-08T05:41:08.408 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-08T05:41:08.409 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:41:08.409 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-08T05:41:08.431 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:08.431+0000 7fba6e130040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:41:08.434 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:41:08.434 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-08T05:41:08.445 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:08.445+0000 7f9bc0327040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:41:08.471 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:41:08.471 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-08T05:41:08.537 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:41:08.537 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-08T05:41:08.589 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:08.589+0000 7fba6e130040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:41:08.604 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:41:08.604 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-08T05:41:08.620 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:08.620+0000 7f9bc0327040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:41:08.630 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:41:08.631 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-08T05:41:08.646 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:08.646+0000 7fba6e130040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:41:08.671 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:41:08.672 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-08T05:41:08.676 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:08.676+0000 7f9bc0327040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:41:08.738 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:41:08.738 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-08T05:41:08.804 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 402a8cef-9d73-41d2-b171-5a8140c79a49 0 2024-05-08T05:41:08.926 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:08.926+0000 7fba6e130040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:41:08.935 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:08.935+0000 7f9bc0327040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:41:08.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.948+0000 7fa86cca0640 1 Processor -- start 2024-05-08T05:41:08.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.948+0000 7fa86cca0640 1 -- start start 2024-05-08T05:41:08.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.949+0000 7fa86cca0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa86815f4d0 0x7fa86806ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:08.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.949+0000 7fa86cca0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8680d2760 0x7fa8680d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:08.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.949+0000 7fa86cca0640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8680d1720 con 0x7fa8680d1310 2024-05-08T05:41:08.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.949+0000 7fa86cca0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa86806db10 con 0x7fa86815f4d0 2024-05-08T05:41:08.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.949+0000 7fa86cca0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa8680d2bf0 con 0x7fa8680d2760 2024-05-08T05:41:08.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.949+0000 7fa866d9d640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fa8680d1310 0x7fa868162f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.63:43946/0 (socket says 172.21.15.63:43946) 2024-05-08T05:41:08.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.949+0000 7fa866d9d640 1 -- 172.21.15.63:0/1853055594 learned_addr learned my addr 172.21.15.63:0/1853055594 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-05-08T05:41:08.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.949+0000 7fa85ffff640 1 --2- 172.21.15.63:0/1853055594 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8680d2760 0x7fa8680d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:08.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.949+0000 7fa86659c640 1 --2- 172.21.15.63:0/1853055594 >> v2:172.21.15.45:3300/0 conn(0x7fa86815f4d0 0x7fa86806ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:08.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.949+0000 7fa865d9b640 1 -- 172.21.15.63:0/1853055594 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 808105949 0 0) 0x7fa860001490 con 0x7fa8680d1310 2024-05-08T05:41:08.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.949+0000 7fa865d9b640 1 -- 172.21.15.63:0/1853055594 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa854003520 con 0x7fa8680d1310 2024-05-08T05:41:08.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.950+0000 7fa85ffff640 1 -- 172.21.15.63:0/1853055594 >> v2:172.21.15.45:3300/0 conn(0x7fa86815f4d0 msgr2=0x7fa86806ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:08.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.950+0000 7fa85ffff640 1 --2- 172.21.15.63:0/1853055594 >> v2:172.21.15.45:3300/0 conn(0x7fa86815f4d0 0x7fa86806ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:08.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.950+0000 7fa85ffff640 1 -- 172.21.15.63:0/1853055594 >> v1:172.21.15.63:6789/0 conn(0x7fa8680d1310 legacy=0x7fa868162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:08.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.950+0000 7fa85ffff640 1 -- 172.21.15.63:0/1853055594 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86806d550 con 0x7fa8680d2760 2024-05-08T05:41:08.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.950+0000 7fa86659c640 1 --2- 172.21.15.63:0/1853055594 >> v2:172.21.15.45:3300/0 conn(0x7fa86815f4d0 0x7fa86806ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:08.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.950+0000 7fa85ffff640 1 --2- 172.21.15.63:0/1853055594 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8680d2760 0x7fa8680d0dd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa8500097a0 tx=0x7fa85002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f2f27cb25000b8f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:08.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.950+0000 7fa865d9b640 1 -- 172.21.15.63:0/1853055594 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850037070 con 0x7fa8680d2760 2024-05-08T05:41:08.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.951+0000 7fa865d9b640 1 -- 172.21.15.63:0/1853055594 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa850032440 con 0x7fa8680d2760 2024-05-08T05:41:08.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.951+0000 7fa865d9b640 1 -- 172.21.15.63:0/1853055594 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850032810 con 0x7fa8680d2760 2024-05-08T05:41:08.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.951+0000 7fa86cca0640 1 -- 172.21.15.63:0/1853055594 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8680d2760 msgr2=0x7fa8680d0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:08.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.951+0000 7fa86cca0640 1 --2- 172.21.15.63:0/1853055594 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8680d2760 0x7fa8680d0dd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa8500097a0 tx=0x7fa85002a180 comp rx=0 tx=0).stop 2024-05-08T05:41:08.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.951+0000 7fa86cca0640 1 -- 172.21.15.63:0/1853055594 shutdown_connections 2024-05-08T05:41:08.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.951+0000 7fa86cca0640 1 --2- 172.21.15.63:0/1853055594 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8680d2760 0x7fa8680d0dd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:08.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.951+0000 7fa86cca0640 1 --2- 172.21.15.63:0/1853055594 >> v2:172.21.15.45:3300/0 conn(0x7fa86815f4d0 0x7fa86806ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:08.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.951+0000 7fa86cca0640 1 -- 172.21.15.63:0/1853055594 >> 172.21.15.63:0/1853055594 conn(0x7fa86809ba40 msgr2=0x7fa86809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:08.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.951+0000 7fa86cca0640 1 -- 172.21.15.63:0/1853055594 shutdown_connections 2024-05-08T05:41:08.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.951+0000 7fa86cca0640 1 -- 172.21.15.63:0/1853055594 wait complete. 2024-05-08T05:41:08.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.953+0000 7fa86cca0640 1 Processor -- start 2024-05-08T05:41:08.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.953+0000 7fa86cca0640 1 -- start start 2024-05-08T05:41:08.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.953+0000 7fa86cca0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8680d1310 0x7fa868146760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:08.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.953+0000 7fa86cca0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa86815f4d0 0x7fa868157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:08.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.953+0000 7fa86cca0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8681569a0 0x7fa868159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:08.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.953+0000 7fa86cca0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa8680b5d40 con 0x7fa8680d1310 2024-05-08T05:41:08.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.953+0000 7fa86cca0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa8680d0460 con 0x7fa8681569a0 2024-05-08T05:41:08.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.953+0000 7fa86cca0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa8680d1a50 con 0x7fa86815f4d0 2024-05-08T05:41:08.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.953+0000 7fa85ffff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa86815f4d0 0x7fa868157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:08.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.953+0000 7fa85ffff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa86815f4d0 0x7fa868157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:47360/0 (socket says 172.21.15.63:47360) 2024-05-08T05:41:08.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.953+0000 7fa85ffff640 1 -- 172.21.15.63:0/1274800966 learned_addr learned my addr 172.21.15.63:0/1274800966 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-08T05:41:08.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.953+0000 7fa866d9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8681569a0 0x7fa868159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:08.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.954+0000 7fa86659c640 1 --2- 172.21.15.63:0/1274800966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8680d1310 0x7fa868146760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:08.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.954+0000 7fa85ffff640 1 -- 172.21.15.63:0/1274800966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8681569a0 msgr2=0x7fa868159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:08.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.954+0000 7fa85ffff640 1 --2- 172.21.15.63:0/1274800966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8681569a0 0x7fa868159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:08.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.954+0000 7fa85ffff640 1 -- 172.21.15.63:0/1274800966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8680d1310 msgr2=0x7fa868146760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:08.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.954+0000 7fa85ffff640 1 --2- 172.21.15.63:0/1274800966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8680d1310 0x7fa868146760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:08.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.954+0000 7fa85ffff640 1 -- 172.21.15.63:0/1274800966 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa868146ca0 con 0x7fa86815f4d0 2024-05-08T05:41:08.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.954+0000 7fa86659c640 1 --2- 172.21.15.63:0/1274800966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8680d1310 0x7fa868146760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:08.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.954+0000 7fa85ffff640 1 --2- 172.21.15.63:0/1274800966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa86815f4d0 0x7fa868157f90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa8500098c0 tx=0x7fa8500046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:08.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.954+0000 7fa85effd640 1 -- 172.21.15.63:0/1274800966 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850041030 con 0x7fa86815f4d0 2024-05-08T05:41:08.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.954+0000 7fa85effd640 1 -- 172.21.15.63:0/1274800966 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa85003d4e0 con 0x7fa86815f4d0 2024-05-08T05:41:08.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.954+0000 7fa85effd640 1 -- 172.21.15.63:0/1274800966 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850037040 con 0x7fa86815f4d0 2024-05-08T05:41:08.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.954+0000 7fa86cca0640 1 -- 172.21.15.63:0/1274800966 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa868156ef0 con 0x7fa86815f4d0 2024-05-08T05:41:08.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.954+0000 7fa86cca0640 1 -- 172.21.15.63:0/1274800966 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8681574b0 con 0x7fa86815f4d0 2024-05-08T05:41:08.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.954+0000 7fa85effd640 1 -- 172.21.15.63:0/1274800966 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa850044070 con 0x7fa86815f4d0 2024-05-08T05:41:08.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.954+0000 7fa85effd640 1 -- 172.21.15.63:0/1274800966 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fa850046b00 con 0x7fa86815f4d0 2024-05-08T05:41:08.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.955+0000 7fa86cca0640 1 -- 172.21.15.63:0/1274800966 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa868069b00 con 0x7fa86815f4d0 2024-05-08T05:41:08.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:08.959+0000 7fa85effd640 1 -- 172.21.15.63:0/1274800966 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa850046dd0 con 0x7fa86815f4d0 2024-05-08T05:41:08.977 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:08.977+0000 7fba6e130040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:41:08.983 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:08.982+0000 7f9bc0327040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:41:09.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.033+0000 7fa86cca0640 1 -- 172.21.15.63:0/1274800966 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd new", "uuid": "402a8cef-9d73-41d2-b171-5a8140c79a49", "id": 0} v 0) -- 0x7fa8680718f0 con 0x7fa86815f4d0 2024-05-08T05:41:09.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.036+0000 7fa85effd640 1 -- 172.21.15.63:0/1274800966 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "402a8cef-9d73-41d2-b171-5a8140c79a49", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fa85003d8b0 con 0x7fa86815f4d0 2024-05-08T05:41:09.037 INFO:teuthology.orchestra.run.smithi063.stdout:0 2024-05-08T05:41:09.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.037+0000 7fa86cca0640 1 -- 172.21.15.63:0/1274800966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa86815f4d0 msgr2=0x7fa868157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.037+0000 7fa86cca0640 1 --2- 172.21.15.63:0/1274800966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa86815f4d0 0x7fa868157f90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa8500098c0 tx=0x7fa8500046c0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.037+0000 7fa86cca0640 1 -- 172.21.15.63:0/1274800966 shutdown_connections 2024-05-08T05:41:09.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.037+0000 7fa86cca0640 1 --2- 172.21.15.63:0/1274800966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa86815f4d0 0x7fa868157f90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.037+0000 7fa86cca0640 1 --2- 172.21.15.63:0/1274800966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8681569a0 0x7fa868159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.037+0000 7fa86cca0640 1 --2- 172.21.15.63:0/1274800966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8680d1310 0x7fa868146760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.037+0000 7fa86cca0640 1 -- 172.21.15.63:0/1274800966 >> 172.21.15.63:0/1274800966 conn(0x7fa86809ba40 msgr2=0x7fa86809be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:09.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.037+0000 7fa86cca0640 1 -- 172.21.15.63:0/1274800966 shutdown_connections 2024-05-08T05:41:09.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.037+0000 7fa86cca0640 1 -- 172.21.15.63:0/1274800966 wait complete. 2024-05-08T05:41:09.048 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 7cf7f6a4-d457-432d-b5d9-1a69592abc90 1 2024-05-08T05:41:09.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.150+0000 7f8090a28640 1 Processor -- start 2024-05-08T05:41:09.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.150+0000 7f8090a28640 1 -- start start 2024-05-08T05:41:09.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.151+0000 7f8090a28640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f808c0d2760 0x7f808c06ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.151+0000 7f8090a28640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f808c0d14e0 0x7f808c0d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.151+0000 7f8090a28640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f808c0d2bf0 con 0x7f808c15f4d0 2024-05-08T05:41:09.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.151+0000 7f8090a28640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f808c06db10 con 0x7f808c0d14e0 2024-05-08T05:41:09.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.151+0000 7f8090a28640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f808c06d550 con 0x7f808c0d2760 2024-05-08T05:41:09.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.152+0000 7f8089d9b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f808c0d2760 0x7f808c06ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.152+0000 7f8089d9b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f808c0d2760 0x7f808c06ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:47368/0 (socket says 172.21.15.63:47368) 2024-05-08T05:41:09.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.152+0000 7f8089d9b640 1 -- 172.21.15.63:0/169994600 learned_addr learned my addr 172.21.15.63:0/169994600 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-08T05:41:09.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.152+0000 7f808ad9d640 1 --2- 172.21.15.63:0/169994600 >> v2:172.21.15.45:3300/0 conn(0x7f808c0d14e0 0x7f808c0d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.152+0000 7f8089d9b640 1 -- 172.21.15.63:0/169994600 >> v2:172.21.15.45:3300/0 conn(0x7f808c0d14e0 msgr2=0x7f808c0d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:09.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.152+0000 7f8089d9b640 1 --2- 172.21.15.63:0/169994600 >> v2:172.21.15.45:3300/0 conn(0x7f808c0d14e0 0x7f808c0d2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.152+0000 7f8089d9b640 1 -- 172.21.15.63:0/169994600 >> v1:172.21.15.45:6789/0 conn(0x7f808c15f4d0 legacy=0x7f808c0d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.152+0000 7f8089d9b640 1 -- 172.21.15.63:0/169994600 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808c06d830 con 0x7f808c0d2760 2024-05-08T05:41:09.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.152+0000 7f808959a640 1 -- 172.21.15.63:0/169994600 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2898829408 0 0) 0x7f808c0d2bf0 con 0x7f808c15f4d0 2024-05-08T05:41:09.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.153+0000 7f8089d9b640 1 --2- 172.21.15.63:0/169994600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f808c0d2760 0x7f808c06ef30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8080009870 tx=0x7f808002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=657953226a9e5fce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:09.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.153+0000 7f808959a640 1 -- 172.21.15.63:0/169994600 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080037070 con 0x7f808c0d2760 2024-05-08T05:41:09.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.153+0000 7f808959a640 1 -- 172.21.15.63:0/169994600 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8080032440 con 0x7f808c0d2760 2024-05-08T05:41:09.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.153+0000 7f808959a640 1 -- 172.21.15.63:0/169994600 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080032810 con 0x7f808c0d2760 2024-05-08T05:41:09.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.153+0000 7f8090a28640 1 -- 172.21.15.63:0/169994600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f808c0d2760 msgr2=0x7f808c06ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.153+0000 7f8090a28640 1 --2- 172.21.15.63:0/169994600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f808c0d2760 0x7f808c06ef30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8080009870 tx=0x7f808002a270 comp rx=0 tx=0).stop 2024-05-08T05:41:09.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.154+0000 7f8090a28640 1 -- 172.21.15.63:0/169994600 shutdown_connections 2024-05-08T05:41:09.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.154+0000 7f8090a28640 1 --2- 172.21.15.63:0/169994600 >> v2:172.21.15.45:3300/0 conn(0x7f808c0d14e0 0x7f808c0d2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.154+0000 7f8090a28640 1 --2- 172.21.15.63:0/169994600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f808c0d2760 0x7f808c06ef30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.154+0000 7f8090a28640 1 -- 172.21.15.63:0/169994600 >> 172.21.15.63:0/169994600 conn(0x7f808c09ba40 msgr2=0x7f808c09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:09.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.154+0000 7f8090a28640 1 -- 172.21.15.63:0/169994600 shutdown_connections 2024-05-08T05:41:09.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.154+0000 7f8090a28640 1 -- 172.21.15.63:0/169994600 wait complete. 2024-05-08T05:41:09.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.155+0000 7f8090a28640 1 Processor -- start 2024-05-08T05:41:09.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.155+0000 7f8090a28640 1 -- start start 2024-05-08T05:41:09.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.156+0000 7f8090a28640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f808c0d14e0 0x7f808c149590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.156+0000 7f8090a28640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f808c15f4d0 0x7f808c149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.156+0000 7f8090a28640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f808c15cac0 0x7f808c15cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.156+0000 7f8090a28640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f808c0d1a50 con 0x7f808c0d14e0 2024-05-08T05:41:09.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.156+0000 7f8090a28640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f808c0b5d40 con 0x7f808c15cac0 2024-05-08T05:41:09.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.156+0000 7f8090a28640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f808c0d1bd0 con 0x7f808c15f4d0 2024-05-08T05:41:09.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.156+0000 7f8089d9b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f808c15f4d0 0x7f808c149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.156+0000 7f8089d9b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f808c15f4d0 0x7f808c149ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:47380/0 (socket says 172.21.15.63:47380) 2024-05-08T05:41:09.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.156+0000 7f8089d9b640 1 -- 172.21.15.63:0/4135927426 learned_addr learned my addr 172.21.15.63:0/4135927426 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-08T05:41:09.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.156+0000 7f808a59c640 1 --2- 172.21.15.63:0/4135927426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f808c0d14e0 0x7f808c149590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.156+0000 7f808ad9d640 1 --2- 172.21.15.63:0/4135927426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f808c15cac0 0x7f808c15cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.156+0000 7f8089d9b640 1 -- 172.21.15.63:0/4135927426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f808c15cac0 msgr2=0x7f808c15cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.156+0000 7f8089d9b640 1 --2- 172.21.15.63:0/4135927426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f808c15cac0 0x7f808c15cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.156+0000 7f8089d9b640 1 -- 172.21.15.63:0/4135927426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f808c0d14e0 msgr2=0x7f808c149590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.156+0000 7f8089d9b640 1 --2- 172.21.15.63:0/4135927426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f808c0d14e0 0x7f808c149590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.156+0000 7f8089d9b640 1 -- 172.21.15.63:0/4135927426 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808c15c8b0 con 0x7f808c15f4d0 2024-05-08T05:41:09.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.156+0000 7f8089d9b640 1 --2- 172.21.15.63:0/4135927426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f808c15f4d0 0x7f808c149ad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f808002a2b0 tx=0x7f80800046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:09.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.156+0000 7f807affd640 1 -- 172.21.15.63:0/4135927426 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080037040 con 0x7f808c15f4d0 2024-05-08T05:41:09.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.157+0000 7f807affd640 1 -- 172.21.15.63:0/4135927426 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8080042070 con 0x7f808c15f4d0 2024-05-08T05:41:09.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.157+0000 7f8090a28640 1 -- 172.21.15.63:0/4135927426 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808c1e9070 con 0x7f808c15f4d0 2024-05-08T05:41:09.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.157+0000 7f807affd640 1 -- 172.21.15.63:0/4135927426 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080030880 con 0x7f808c15f4d0 2024-05-08T05:41:09.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.157+0000 7f8090a28640 1 -- 172.21.15.63:0/4135927426 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f808c1e9310 con 0x7f808c15f4d0 2024-05-08T05:41:09.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.157+0000 7f807affd640 1 -- 172.21.15.63:0/4135927426 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8080030a20 con 0x7f808c15f4d0 2024-05-08T05:41:09.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.157+0000 7f807affd640 1 -- 172.21.15.63:0/4135927426 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f8080047c10 con 0x7f808c15f4d0 2024-05-08T05:41:09.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.158+0000 7f8090a28640 1 -- 172.21.15.63:0/4135927426 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f808c069b00 con 0x7f808c15f4d0 2024-05-08T05:41:09.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.160+0000 7f807affd640 1 -- 172.21.15.63:0/4135927426 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8080047460 con 0x7f808c15f4d0 2024-05-08T05:41:09.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.236+0000 7f8090a28640 1 -- 172.21.15.63:0/4135927426 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7cf7f6a4-d457-432d-b5d9-1a69592abc90", "id": 1} v 0) -- 0x7f808c0d10b0 con 0x7f808c15f4d0 2024-05-08T05:41:09.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.239+0000 7f807affd640 1 -- 172.21.15.63:0/4135927426 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7cf7f6a4-d457-432d-b5d9-1a69592abc90", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f8080047640 con 0x7f808c15f4d0 2024-05-08T05:41:09.240 INFO:teuthology.orchestra.run.smithi063.stdout:1 2024-05-08T05:41:09.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.240+0000 7f8090a28640 1 -- 172.21.15.63:0/4135927426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f808c15f4d0 msgr2=0x7f808c149ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.240+0000 7f8090a28640 1 --2- 172.21.15.63:0/4135927426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f808c15f4d0 0x7f808c149ad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f808002a2b0 tx=0x7f80800046c0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.240+0000 7f8090a28640 1 -- 172.21.15.63:0/4135927426 shutdown_connections 2024-05-08T05:41:09.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.240+0000 7f8090a28640 1 --2- 172.21.15.63:0/4135927426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f808c15f4d0 0x7f808c149ad0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.240+0000 7f8090a28640 1 --2- 172.21.15.63:0/4135927426 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f808c15cac0 0x7f808c15cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.240+0000 7f8090a28640 1 --2- 172.21.15.63:0/4135927426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f808c0d14e0 0x7f808c149590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.240+0000 7f8090a28640 1 -- 172.21.15.63:0/4135927426 >> 172.21.15.63:0/4135927426 conn(0x7f808c09ba40 msgr2=0x7f808c0b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:09.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.240+0000 7f8090a28640 1 -- 172.21.15.63:0/4135927426 shutdown_connections 2024-05-08T05:41:09.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.241+0000 7f8090a28640 1 -- 172.21.15.63:0/4135927426 wait complete. 2024-05-08T05:41:09.251 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 68484847-6fbd-4594-8374-a1a3e959f247 2 2024-05-08T05:41:09.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.355+0000 7f91ce9da640 1 Processor -- start 2024-05-08T05:41:09.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.355+0000 7f91ce9da640 1 -- start start 2024-05-08T05:41:09.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.356+0000 7f91ce9da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c8156530 0x7f91c80aef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.356+0000 7f91ce9da640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f91c806cab0 0x7f91c8162fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.356+0000 7f91ce9da640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91c80b57c0 con 0x7f91c80b6280 2024-05-08T05:41:09.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.356+0000 7f91ce9da640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f91c80b5df0 con 0x7f91c806cab0 2024-05-08T05:41:09.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.356+0000 7f91ce9da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f91c80afd10 con 0x7f91c8156530 2024-05-08T05:41:09.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.356+0000 7f91ccf77640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f91c806cab0 0x7f91c8162fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.356+0000 7f91ccf77640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f91c806cab0 0x7f91c8162fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35732/0 (socket says 172.21.15.63:35732) 2024-05-08T05:41:09.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.356+0000 7f91c77fe640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f91c80b6280 0x7f91c806c3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.63:43952/0 (socket says 172.21.15.63:43952) 2024-05-08T05:41:09.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.356+0000 7f91ccf77640 1 -- 172.21.15.63:0/704210114 learned_addr learned my addr 172.21.15.63:0/704210114 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-08T05:41:09.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.356+0000 7f91c7fff640 1 --2- 172.21.15.63:0/704210114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c8156530 0x7f91c80aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.357+0000 7f91ccf77640 1 -- 172.21.15.63:0/704210114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c8156530 msgr2=0x7f91c80aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:09.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.357+0000 7f91ccf77640 1 --2- 172.21.15.63:0/704210114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c8156530 0x7f91c80aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.357+0000 7f91ccf77640 1 -- 172.21.15.63:0/704210114 >> v1:172.21.15.63:6789/0 conn(0x7f91c80b6280 legacy=0x7f91c806c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.357+0000 7f91ccf77640 1 -- 172.21.15.63:0/704210114 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c80afa30 con 0x7f91c806cab0 2024-05-08T05:41:09.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.357+0000 7f91c6ffd640 1 -- 172.21.15.63:0/704210114 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1059354752 0 0) 0x7f91c80b57c0 con 0x7f91c80b6280 2024-05-08T05:41:09.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.357+0000 7f91c7fff640 1 --2- 172.21.15.63:0/704210114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c8156530 0x7f91c80aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:09.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.358+0000 7f91ccf77640 1 --2- 172.21.15.63:0/704210114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91c806cab0 0x7f91c8162fe0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f91c80b5f70 tx=0x7f91c002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81f5c48fd4a290d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:09.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.358+0000 7f91c6ffd640 1 -- 172.21.15.63:0/704210114 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0036070 con 0x7f91c806cab0 2024-05-08T05:41:09.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.358+0000 7f91c6ffd640 1 -- 172.21.15.63:0/704210114 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c0031600 con 0x7f91c806cab0 2024-05-08T05:41:09.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.358+0000 7f91c6ffd640 1 -- 172.21.15.63:0/704210114 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c00319d0 con 0x7f91c806cab0 2024-05-08T05:41:09.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.358+0000 7f91ce9da640 1 -- 172.21.15.63:0/704210114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91c806cab0 msgr2=0x7f91c8162fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.358+0000 7f91ce9da640 1 --2- 172.21.15.63:0/704210114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91c806cab0 0x7f91c8162fe0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f91c80b5f70 tx=0x7f91c002a710 comp rx=0 tx=0).stop 2024-05-08T05:41:09.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.359+0000 7f91ce9da640 1 -- 172.21.15.63:0/704210114 shutdown_connections 2024-05-08T05:41:09.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.359+0000 7f91ce9da640 1 --2- 172.21.15.63:0/704210114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91c806cab0 0x7f91c8162fe0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.359+0000 7f91ce9da640 1 --2- 172.21.15.63:0/704210114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c8156530 0x7f91c80aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.359+0000 7f91ce9da640 1 -- 172.21.15.63:0/704210114 >> 172.21.15.63:0/704210114 conn(0x7f91c80b39a0 msgr2=0x7f91c80b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:09.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.359+0000 7f91ce9da640 1 -- 172.21.15.63:0/704210114 shutdown_connections 2024-05-08T05:41:09.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.359+0000 7f91ce9da640 1 -- 172.21.15.63:0/704210114 wait complete. 2024-05-08T05:41:09.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.360+0000 7f91ce9da640 1 Processor -- start 2024-05-08T05:41:09.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.360+0000 7f91ce9da640 1 -- start start 2024-05-08T05:41:09.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.361+0000 7f91ce9da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91c806cab0 0x7f91c81f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.361+0000 7f91ce9da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c80b6280 0x7f91c814d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.361+0000 7f91ce9da640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91c8156530 0x7f91c815f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.361+0000 7f91ce9da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f91c80af4d0 con 0x7f91c806cab0 2024-05-08T05:41:09.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.361+0000 7f91ce9da640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f91c80b14d0 con 0x7f91c8156530 2024-05-08T05:41:09.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.361+0000 7f91ce9da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f91c80af650 con 0x7f91c80b6280 2024-05-08T05:41:09.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.361+0000 7f91c77fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c80b6280 0x7f91c814d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.361+0000 7f91ccf77640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91c8156530 0x7f91c815f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.361+0000 7f91ccf77640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91c8156530 0x7f91c815f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35748/0 (socket says 172.21.15.63:35748) 2024-05-08T05:41:09.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.361+0000 7f91c77fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c80b6280 0x7f91c814d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:47398/0 (socket says 172.21.15.63:47398) 2024-05-08T05:41:09.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.361+0000 7f91ccf77640 1 -- 172.21.15.63:0/1750532284 learned_addr learned my addr 172.21.15.63:0/1750532284 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-08T05:41:09.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.361+0000 7f91c7fff640 1 --2- 172.21.15.63:0/1750532284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91c806cab0 0x7f91c81f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.361+0000 7f91ccf77640 1 -- 172.21.15.63:0/1750532284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c80b6280 msgr2=0x7f91c814d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.361+0000 7f91ccf77640 1 --2- 172.21.15.63:0/1750532284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c80b6280 0x7f91c814d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.361+0000 7f91ccf77640 1 -- 172.21.15.63:0/1750532284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91c806cab0 msgr2=0x7f91c81f5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.361+0000 7f91ccf77640 1 --2- 172.21.15.63:0/1750532284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91c806cab0 0x7f91c81f5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.361+0000 7f91ccf77640 1 -- 172.21.15.63:0/1750532284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c814e4a0 con 0x7f91c8156530 2024-05-08T05:41:09.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.361+0000 7f91c77fe640 1 --2- 172.21.15.63:0/1750532284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c80b6280 0x7f91c814d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:09.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.361+0000 7f91c7fff640 1 --2- 172.21.15.63:0/1750532284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91c806cab0 0x7f91c81f5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:09.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.361+0000 7f91ccf77640 1 --2- 172.21.15.63:0/1750532284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91c8156530 0x7f91c815f860 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009f40 tx=0x7f91c0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:09.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.362+0000 7f91c4ff9640 1 -- 172.21.15.63:0/1750532284 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0036050 con 0x7f91c8156530 2024-05-08T05:41:09.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.362+0000 7f91c4ff9640 1 -- 172.21.15.63:0/1750532284 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c0042070 con 0x7f91c8156530 2024-05-08T05:41:09.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.362+0000 7f91ce9da640 1 -- 172.21.15.63:0/1750532284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c814e780 con 0x7f91c8156530 2024-05-08T05:41:09.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.362+0000 7f91c4ff9640 1 -- 172.21.15.63:0/1750532284 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0009510 con 0x7f91c8156530 2024-05-08T05:41:09.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.362+0000 7f91ce9da640 1 -- 172.21.15.63:0/1750532284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c814ed40 con 0x7f91c8156530 2024-05-08T05:41:09.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.362+0000 7f91c4ff9640 1 -- 172.21.15.63:0/1750532284 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f91c0007bf0 con 0x7f91c8156530 2024-05-08T05:41:09.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.362+0000 7f91c4ff9640 1 -- 172.21.15.63:0/1750532284 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f91c003d450 con 0x7f91c8156530 2024-05-08T05:41:09.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.363+0000 7f91ce9da640 1 -- 172.21.15.63:0/1750532284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91c8069b00 con 0x7f91c8156530 2024-05-08T05:41:09.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.365+0000 7f91c4ff9640 1 -- 172.21.15.63:0/1750532284 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f91c0047460 con 0x7f91c8156530 2024-05-08T05:41:09.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.438+0000 7f91ce9da640 1 -- 172.21.15.63:0/1750532284 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd new", "uuid": "68484847-6fbd-4594-8374-a1a3e959f247", "id": 2} v 0) -- 0x7f91c80b0730 con 0x7f91c8156530 2024-05-08T05:41:09.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.441+0000 7f91c4ff9640 1 -- 172.21.15.63:0/1750532284 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "68484847-6fbd-4594-8374-a1a3e959f247", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f91c0047af0 con 0x7f91c8156530 2024-05-08T05:41:09.442 INFO:teuthology.orchestra.run.smithi063.stdout:2 2024-05-08T05:41:09.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.442+0000 7f91ce9da640 1 -- 172.21.15.63:0/1750532284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91c8156530 msgr2=0x7f91c815f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.442+0000 7f91ce9da640 1 --2- 172.21.15.63:0/1750532284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91c8156530 0x7f91c815f860 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009f40 tx=0x7f91c0004a40 comp rx=0 tx=0).stop 2024-05-08T05:41:09.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.442+0000 7f91ce9da640 1 -- 172.21.15.63:0/1750532284 shutdown_connections 2024-05-08T05:41:09.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.442+0000 7f91ce9da640 1 --2- 172.21.15.63:0/1750532284 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91c80b6280 0x7f91c814d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.442+0000 7f91ce9da640 1 --2- 172.21.15.63:0/1750532284 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91c8156530 0x7f91c815f860 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.442+0000 7f91ce9da640 1 --2- 172.21.15.63:0/1750532284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91c806cab0 0x7f91c81f5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.442+0000 7f91ce9da640 1 -- 172.21.15.63:0/1750532284 >> 172.21.15.63:0/1750532284 conn(0x7f91c80b39a0 msgr2=0x7f91c806c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:09.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.442+0000 7f91ce9da640 1 -- 172.21.15.63:0/1750532284 shutdown_connections 2024-05-08T05:41:09.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.442+0000 7f91ce9da640 1 -- 172.21.15.63:0/1750532284 wait complete. 2024-05-08T05:41:09.453 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 663351b5-c96a-470b-b9c8-3d1f345c75a7 3 2024-05-08T05:41:09.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.556+0000 7fd9ce9f0640 1 Processor -- start 2024-05-08T05:41:09.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.556+0000 7fd9ce9f0640 1 -- start start 2024-05-08T05:41:09.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.556+0000 7fd9ce9f0640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd9c806c3a0 0x7fd9c815d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.556+0000 7fd9ce9f0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9c806f5e0 0x7fd9c806c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.557+0000 7fd9ce9f0640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9c806ccc0 con 0x7fd9c8161490 2024-05-08T05:41:09.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.557+0000 7fd9ce9f0640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd9c8160d20 con 0x7fd9c806c3a0 2024-05-08T05:41:09.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.557+0000 7fd9ce9f0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd9c8161000 con 0x7fd9c806f5e0 2024-05-08T05:41:09.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.557+0000 7fd9c77fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9c806f5e0 0x7fd9c806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.557+0000 7fd9c77fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9c806f5e0 0x7fd9c806c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:47400/0 (socket says 172.21.15.63:47400) 2024-05-08T05:41:09.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.557+0000 7fd9c77fe640 1 -- 172.21.15.63:0/2115453836 learned_addr learned my addr 172.21.15.63:0/2115453836 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-08T05:41:09.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.557+0000 7fd9c7fff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd9c806c3a0 0x7fd9c815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.557+0000 7fd9c77fe640 1 -- 172.21.15.63:0/2115453836 >> v2:172.21.15.63:3300/0 conn(0x7fd9c806c3a0 msgr2=0x7fd9c815d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:09.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.557+0000 7fd9c77fe640 1 --2- 172.21.15.63:0/2115453836 >> v2:172.21.15.63:3300/0 conn(0x7fd9c806c3a0 0x7fd9c815d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.557+0000 7fd9c77fe640 1 -- 172.21.15.63:0/2115453836 >> v1:172.21.15.63:6789/0 conn(0x7fd9c8161490 legacy=0x7fd9c815eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.557+0000 7fd9c77fe640 1 -- 172.21.15.63:0/2115453836 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c806fa60 con 0x7fd9c806f5e0 2024-05-08T05:41:09.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.558+0000 7fd9c77fe640 1 --2- 172.21.15.63:0/2115453836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9c806f5e0 0x7fd9c806c780 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009870 tx=0x7fd9b002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bb19a74a3624bf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:09.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.558+0000 7fd9c6ffd640 1 -- 172.21.15.63:0/2115453836 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0037070 con 0x7fd9c806f5e0 2024-05-08T05:41:09.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.558+0000 7fd9c6ffd640 1 -- 172.21.15.63:0/2115453836 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b0032440 con 0x7fd9c806f5e0 2024-05-08T05:41:09.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.558+0000 7fd9c6ffd640 1 -- 172.21.15.63:0/2115453836 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0032810 con 0x7fd9c806f5e0 2024-05-08T05:41:09.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.559+0000 7fd9ce9f0640 1 -- 172.21.15.63:0/2115453836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9c806f5e0 msgr2=0x7fd9c806c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.559+0000 7fd9ce9f0640 1 --2- 172.21.15.63:0/2115453836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9c806f5e0 0x7fd9c806c780 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009870 tx=0x7fd9b002a270 comp rx=0 tx=0).stop 2024-05-08T05:41:09.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.559+0000 7fd9ce9f0640 1 -- 172.21.15.63:0/2115453836 shutdown_connections 2024-05-08T05:41:09.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.559+0000 7fd9ce9f0640 1 --2- 172.21.15.63:0/2115453836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9c806f5e0 0x7fd9c806c780 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.559+0000 7fd9ce9f0640 1 --2- 172.21.15.63:0/2115453836 >> v2:172.21.15.63:3300/0 conn(0x7fd9c806c3a0 0x7fd9c815d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.559+0000 7fd9ce9f0640 1 -- 172.21.15.63:0/2115453836 >> 172.21.15.63:0/2115453836 conn(0x7fd9c80b39a0 msgr2=0x7fd9c80b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:09.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.559+0000 7fd9ce9f0640 1 -- 172.21.15.63:0/2115453836 shutdown_connections 2024-05-08T05:41:09.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.559+0000 7fd9ce9f0640 1 -- 172.21.15.63:0/2115453836 wait complete. 2024-05-08T05:41:09.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.560+0000 7fd9ce9f0640 1 Processor -- start 2024-05-08T05:41:09.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.560+0000 7fd9ce9f0640 1 -- start start 2024-05-08T05:41:09.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.561+0000 7fd9ce9f0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9c806c3a0 0x7fd9c81f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.561+0000 7fd9ce9f0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9c806f5e0 0x7fd9c81f6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.561+0000 7fd9ce9f0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9c8161490 0x7fd9c81f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.561+0000 7fd9ce9f0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c815e480 con 0x7fd9c806f5e0 2024-05-08T05:41:09.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.561+0000 7fd9ce9f0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c815e1a0 con 0x7fd9c8161490 2024-05-08T05:41:09.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.561+0000 7fd9ce9f0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd9c80d1390 con 0x7fd9c806c3a0 2024-05-08T05:41:09.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.561+0000 7fd9c7fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9c806c3a0 0x7fd9c81f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.561+0000 7fd9c7fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9c806c3a0 0x7fd9c81f4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:47406/0 (socket says 172.21.15.63:47406) 2024-05-08T05:41:09.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.561+0000 7fd9ccf8d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9c8161490 0x7fd9c81f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.561+0000 7fd9c7fff640 1 -- 172.21.15.63:0/3926219312 learned_addr learned my addr 172.21.15.63:0/3926219312 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-08T05:41:09.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.561+0000 7fd9c77fe640 1 --2- 172.21.15.63:0/3926219312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9c806f5e0 0x7fd9c81f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.561+0000 7fd9c7fff640 1 -- 172.21.15.63:0/3926219312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9c8161490 msgr2=0x7fd9c81f37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.561+0000 7fd9c7fff640 1 --2- 172.21.15.63:0/3926219312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9c8161490 0x7fd9c81f37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.561+0000 7fd9c7fff640 1 -- 172.21.15.63:0/3926219312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9c806f5e0 msgr2=0x7fd9c81f6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.561+0000 7fd9c7fff640 1 --2- 172.21.15.63:0/3926219312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9c806f5e0 0x7fd9c81f6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.561+0000 7fd9c7fff640 1 -- 172.21.15.63:0/3926219312 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c81e6b40 con 0x7fd9c806c3a0 2024-05-08T05:41:09.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.561+0000 7fd9ccf8d640 1 --2- 172.21.15.63:0/3926219312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9c8161490 0x7fd9c81f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:09.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.561+0000 7fd9c7fff640 1 --2- 172.21.15.63:0/3926219312 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9c806c3a0 0x7fd9c81f4a10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800e9b0 tx=0x7fd9b800ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:09.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.562+0000 7fd9c4ff9640 1 -- 172.21.15.63:0/3926219312 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8017070 con 0x7fd9c806c3a0 2024-05-08T05:41:09.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.562+0000 7fd9c4ff9640 1 -- 172.21.15.63:0/3926219312 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b800c9a0 con 0x7fd9c806c3a0 2024-05-08T05:41:09.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.562+0000 7fd9ce9f0640 1 -- 172.21.15.63:0/3926219312 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c81f4110 con 0x7fd9c806c3a0 2024-05-08T05:41:09.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.562+0000 7fd9c4ff9640 1 -- 172.21.15.63:0/3926219312 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b800cd90 con 0x7fd9c806c3a0 2024-05-08T05:41:09.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.562+0000 7fd9ce9f0640 1 -- 172.21.15.63:0/3926219312 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c81f46d0 con 0x7fd9c806c3a0 2024-05-08T05:41:09.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.562+0000 7fd9c4ff9640 1 -- 172.21.15.63:0/3926219312 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd9b801a070 con 0x7fd9c806c3a0 2024-05-08T05:41:09.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.562+0000 7fd9c4ff9640 1 -- 172.21.15.63:0/3926219312 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fd9b801c9d0 con 0x7fd9c806c3a0 2024-05-08T05:41:09.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.563+0000 7fd9ce9f0640 1 -- 172.21.15.63:0/3926219312 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9c8069b00 con 0x7fd9c806c3a0 2024-05-08T05:41:09.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.565+0000 7fd9c4ff9640 1 -- 172.21.15.63:0/3926219312 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd9b801ccd0 con 0x7fd9c806c3a0 2024-05-08T05:41:09.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.640+0000 7fd9ce9f0640 1 -- 172.21.15.63:0/3926219312 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd new", "uuid": "663351b5-c96a-470b-b9c8-3d1f345c75a7", "id": 3} v 0) -- 0x7fd9c8142f40 con 0x7fd9c806c3a0 2024-05-08T05:41:09.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.643+0000 7fd9c4ff9640 1 -- 172.21.15.63:0/3926219312 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "663351b5-c96a-470b-b9c8-3d1f345c75a7", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fd9b801c540 con 0x7fd9c806c3a0 2024-05-08T05:41:09.646 INFO:teuthology.orchestra.run.smithi063.stdout:3 2024-05-08T05:41:09.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.644+0000 7fd9ce9f0640 1 -- 172.21.15.63:0/3926219312 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9c806c3a0 msgr2=0x7fd9c81f4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.644+0000 7fd9ce9f0640 1 --2- 172.21.15.63:0/3926219312 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9c806c3a0 0x7fd9c81f4a10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800e9b0 tx=0x7fd9b800ee70 comp rx=0 tx=0).stop 2024-05-08T05:41:09.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.644+0000 7fd9ce9f0640 1 -- 172.21.15.63:0/3926219312 shutdown_connections 2024-05-08T05:41:09.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.644+0000 7fd9ce9f0640 1 --2- 172.21.15.63:0/3926219312 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd9c8161490 0x7fd9c81f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.644+0000 7fd9ce9f0640 1 --2- 172.21.15.63:0/3926219312 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd9c806f5e0 0x7fd9c81f6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.644+0000 7fd9ce9f0640 1 --2- 172.21.15.63:0/3926219312 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd9c806c3a0 0x7fd9c81f4a10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.644+0000 7fd9ce9f0640 1 -- 172.21.15.63:0/3926219312 >> 172.21.15.63:0/3926219312 conn(0x7fd9c80b39a0 msgr2=0x7fd9c815d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:09.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.644+0000 7fd9ce9f0640 1 -- 172.21.15.63:0/3926219312 shutdown_connections 2024-05-08T05:41:09.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.644+0000 7fd9ce9f0640 1 -- 172.21.15.63:0/3926219312 wait complete. 2024-05-08T05:41:09.655 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 92decff9-1aa1-4089-9bb4-0c24366e8def 4 2024-05-08T05:41:09.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.757+0000 7fe86c8da640 1 Processor -- start 2024-05-08T05:41:09.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.757+0000 7fe86c8da640 1 -- start start 2024-05-08T05:41:09.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.758+0000 7fe86c8da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe868071720 0x7fe86815f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.758+0000 7fe86c8da640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe86806c3a0 0x7fe86806c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.758+0000 7fe86c8da640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe868160600 con 0x7fe8680b1050 2024-05-08T05:41:09.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.758+0000 7fe86c8da640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe868071b80 con 0x7fe86806c3a0 2024-05-08T05:41:09.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.758+0000 7fe86c8da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe86806cd50 con 0x7fe868071720 2024-05-08T05:41:09.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.758+0000 7fe866d9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe86806c3a0 0x7fe86806c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.758+0000 7fe865d9b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe868071720 0x7fe86815f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.758+0000 7fe865d9b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe868071720 0x7fe86815f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:47410/0 (socket says 172.21.15.63:47410) 2024-05-08T05:41:09.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.758+0000 7fe866d9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe86806c3a0 0x7fe86806c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35768/0 (socket says 172.21.15.63:35768) 2024-05-08T05:41:09.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.758+0000 7fe865d9b640 1 -- 172.21.15.63:0/545519910 learned_addr learned my addr 172.21.15.63:0/545519910 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-08T05:41:09.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.758+0000 7fe866d9d640 1 -- 172.21.15.63:0/545519910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe868071720 msgr2=0x7fe86815f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:09.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.758+0000 7fe866d9d640 1 --2- 172.21.15.63:0/545519910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe868071720 0x7fe86815f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.758+0000 7fe866d9d640 1 -- 172.21.15.63:0/545519910 >> v1:172.21.15.45:6789/0 conn(0x7fe8680b1050 legacy=0x7fe8680b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.758+0000 7fe866d9d640 1 -- 172.21.15.63:0/545519910 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86815fd60 con 0x7fe86806c3a0 2024-05-08T05:41:09.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.759+0000 7fe865d9b640 1 --2- 172.21.15.63:0/545519910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe868071720 0x7fe86815f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:09.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.759+0000 7fe866d9d640 1 --2- 172.21.15.63:0/545519910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe86806c3a0 0x7fe86806c810 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe860009d90 tx=0x7fe86002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce2b152ba44c837 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:09.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.759+0000 7fe86559a640 1 -- 172.21.15.63:0/545519910 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860007740 con 0x7fe86806c3a0 2024-05-08T05:41:09.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.759+0000 7fe86559a640 1 -- 172.21.15.63:0/545519910 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8600078e0 con 0x7fe86806c3a0 2024-05-08T05:41:09.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.759+0000 7fe86559a640 1 -- 172.21.15.63:0/545519910 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860007cb0 con 0x7fe86806c3a0 2024-05-08T05:41:09.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.760+0000 7fe86c8da640 1 -- 172.21.15.63:0/545519910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe86806c3a0 msgr2=0x7fe86806c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.760+0000 7fe86c8da640 1 --2- 172.21.15.63:0/545519910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe86806c3a0 0x7fe86806c810 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe860009d90 tx=0x7fe86002a660 comp rx=0 tx=0).stop 2024-05-08T05:41:09.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.760+0000 7fe86c8da640 1 -- 172.21.15.63:0/545519910 shutdown_connections 2024-05-08T05:41:09.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.760+0000 7fe86c8da640 1 --2- 172.21.15.63:0/545519910 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe86806c3a0 0x7fe86806c810 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.760+0000 7fe86c8da640 1 --2- 172.21.15.63:0/545519910 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe868071720 0x7fe86815f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.760+0000 7fe86c8da640 1 -- 172.21.15.63:0/545519910 >> 172.21.15.63:0/545519910 conn(0x7fe8680b3c40 msgr2=0x7fe8680b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:09.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.760+0000 7fe86c8da640 1 -- 172.21.15.63:0/545519910 shutdown_connections 2024-05-08T05:41:09.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.760+0000 7fe86c8da640 1 -- 172.21.15.63:0/545519910 wait complete. 2024-05-08T05:41:09.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.761+0000 7fe86c8da640 1 Processor -- start 2024-05-08T05:41:09.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.762+0000 7fe86c8da640 1 -- start start 2024-05-08T05:41:09.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.762+0000 7fe86c8da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe86806c3a0 0x7fe86814d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.762+0000 7fe86c8da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe868071720 0x7fe868150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.762+0000 7fe86c8da640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8680b1050 0x7fe86814bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.762+0000 7fe86c8da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe868162d10 con 0x7fe868071720 2024-05-08T05:41:09.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.762+0000 7fe86c8da640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe868160320 con 0x7fe8680b1050 2024-05-08T05:41:09.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.762+0000 7fe86c8da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe8680b5dc0 con 0x7fe86806c3a0 2024-05-08T05:41:09.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.762+0000 7fe866d9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8680b1050 0x7fe86814bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.762+0000 7fe86659c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe86806c3a0 0x7fe86814d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.762+0000 7fe866d9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8680b1050 0x7fe86814bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35784/0 (socket says 172.21.15.63:35784) 2024-05-08T05:41:09.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.762+0000 7fe866d9d640 1 -- 172.21.15.63:0/1773227921 learned_addr learned my addr 172.21.15.63:0/1773227921 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-08T05:41:09.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.762+0000 7fe865d9b640 1 --2- 172.21.15.63:0/1773227921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe868071720 0x7fe868150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.763+0000 7fe86659c640 1 -- 172.21.15.63:0/1773227921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8680b1050 msgr2=0x7fe86814bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.763+0000 7fe86659c640 1 --2- 172.21.15.63:0/1773227921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8680b1050 0x7fe86814bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.763+0000 7fe86659c640 1 -- 172.21.15.63:0/1773227921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe868071720 msgr2=0x7fe868150da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.763+0000 7fe86659c640 1 --2- 172.21.15.63:0/1773227921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe868071720 0x7fe868150da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.763+0000 7fe86659c640 1 -- 172.21.15.63:0/1773227921 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86814d7b0 con 0x7fe86806c3a0 2024-05-08T05:41:09.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.763+0000 7fe866d9d640 1 --2- 172.21.15.63:0/1773227921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8680b1050 0x7fe86814bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:09.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.763+0000 7fe86659c640 1 --2- 172.21.15.63:0/1773227921 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe86806c3a0 0x7fe86814d0d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe85800c460 tx=0x7fe85800c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:09.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.763+0000 7fe85effd640 1 -- 172.21.15.63:0/1773227921 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8580052d0 con 0x7fe86806c3a0 2024-05-08T05:41:09.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.763+0000 7fe85effd640 1 -- 172.21.15.63:0/1773227921 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe858005470 con 0x7fe86806c3a0 2024-05-08T05:41:09.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.763+0000 7fe86c8da640 1 -- 172.21.15.63:0/1773227921 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe86814da90 con 0x7fe86806c3a0 2024-05-08T05:41:09.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.763+0000 7fe85effd640 1 -- 172.21.15.63:0/1773227921 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe858012850 con 0x7fe86806c3a0 2024-05-08T05:41:09.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.763+0000 7fe86c8da640 1 -- 172.21.15.63:0/1773227921 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe868151e10 con 0x7fe86806c3a0 2024-05-08T05:41:09.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.763+0000 7fe85effd640 1 -- 172.21.15.63:0/1773227921 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe8580129f0 con 0x7fe86806c3a0 2024-05-08T05:41:09.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.763+0000 7fe85effd640 1 -- 172.21.15.63:0/1773227921 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fe85801b910 con 0x7fe86806c3a0 2024-05-08T05:41:09.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.764+0000 7fe86c8da640 1 -- 172.21.15.63:0/1773227921 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe868157550 con 0x7fe86806c3a0 2024-05-08T05:41:09.767 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:09.765+0000 7fba6e130040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:41:09.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.766+0000 7fe85effd640 1 -- 172.21.15.63:0/1773227921 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe85801b400 con 0x7fe86806c3a0 2024-05-08T05:41:09.775 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:09.775+0000 7f9bc0327040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:41:09.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.840+0000 7fe86c8da640 1 -- 172.21.15.63:0/1773227921 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd new", "uuid": "92decff9-1aa1-4089-9bb4-0c24366e8def", "id": 4} v 0) -- 0x7fe868069b00 con 0x7fe86806c3a0 2024-05-08T05:41:09.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.843+0000 7fe85effd640 1 -- 172.21.15.63:0/1773227921 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "92decff9-1aa1-4089-9bb4-0c24366e8def", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fe85801b5e0 con 0x7fe86806c3a0 2024-05-08T05:41:09.844 INFO:teuthology.orchestra.run.smithi063.stdout:4 2024-05-08T05:41:09.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.844+0000 7fe86c8da640 1 -- 172.21.15.63:0/1773227921 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe86806c3a0 msgr2=0x7fe86814d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.844+0000 7fe86c8da640 1 --2- 172.21.15.63:0/1773227921 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe86806c3a0 0x7fe86814d0d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe85800c460 tx=0x7fe85800c920 comp rx=0 tx=0).stop 2024-05-08T05:41:09.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.844+0000 7fe86c8da640 1 -- 172.21.15.63:0/1773227921 shutdown_connections 2024-05-08T05:41:09.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.844+0000 7fe86c8da640 1 --2- 172.21.15.63:0/1773227921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8680b1050 0x7fe86814bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.844+0000 7fe86c8da640 1 --2- 172.21.15.63:0/1773227921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe868071720 0x7fe868150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.844+0000 7fe86c8da640 1 --2- 172.21.15.63:0/1773227921 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe86806c3a0 0x7fe86814d0d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.844+0000 7fe86c8da640 1 -- 172.21.15.63:0/1773227921 >> 172.21.15.63:0/1773227921 conn(0x7fe8680b3c40 msgr2=0x7fe8680af360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:09.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.844+0000 7fe86c8da640 1 -- 172.21.15.63:0/1773227921 shutdown_connections 2024-05-08T05:41:09.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.844+0000 7fe86c8da640 1 -- 172.21.15.63:0/1773227921 wait complete. 2024-05-08T05:41:09.855 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 19f59f65-8cab-4bc4-a68d-07aab3fa695a 5 2024-05-08T05:41:09.859 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:09.859+0000 7fba6e130040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:41:09.867 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:09.867+0000 7f9bc0327040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:41:09.906 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:09.905+0000 7fba6e130040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:41:09.911 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:09.911+0000 7f9bc0327040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:41:09.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.958+0000 7f11d5189640 1 Processor -- start 2024-05-08T05:41:09.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.958+0000 7f11d5189640 1 -- start start 2024-05-08T05:41:09.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.959+0000 7f11d5189640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f11d0162750 0x7f11d00af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.959+0000 7f11d5189640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11d00ae8b0 0x7f11d0070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.959+0000 7f11d5189640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11d0162b30 con 0x7f11d0071100 2024-05-08T05:41:09.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.959+0000 7f11d5189640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f11d0070730 con 0x7f11d0162750 2024-05-08T05:41:09.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.959+0000 7f11d5189640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f11d006fb20 con 0x7f11d00ae8b0 2024-05-08T05:41:09.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.959+0000 7f11ce59c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11d00ae8b0 0x7f11d0070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.959+0000 7f11ce59c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11d00ae8b0 0x7f11d0070bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:47428/0 (socket says 172.21.15.63:47428) 2024-05-08T05:41:09.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.959+0000 7f11ce59c640 1 -- 172.21.15.63:0/1890809660 learned_addr learned my addr 172.21.15.63:0/1890809660 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-08T05:41:09.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.959+0000 7f11ced9d640 1 --2- 172.21.15.63:0/1890809660 >> v2:172.21.15.45:3300/0 conn(0x7f11d0162750 0x7f11d00af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.959+0000 7f11ce59c640 1 -- 172.21.15.63:0/1890809660 >> v2:172.21.15.45:3300/0 conn(0x7f11d0162750 msgr2=0x7f11d00af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:09.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.959+0000 7f11ce59c640 1 --2- 172.21.15.63:0/1890809660 >> v2:172.21.15.45:3300/0 conn(0x7f11d0162750 0x7f11d00af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.959+0000 7f11ce59c640 1 -- 172.21.15.63:0/1890809660 >> v1:172.21.15.45:6789/0 conn(0x7f11d0071100 legacy=0x7f11d00b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.959+0000 7f11ce59c640 1 -- 172.21.15.63:0/1890809660 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d006fe00 con 0x7f11d00ae8b0 2024-05-08T05:41:09.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.960+0000 7f11ce59c640 1 --2- 172.21.15.63:0/1890809660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11d00ae8b0 0x7f11d0070bc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009870 tx=0x7f11c402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73ad7e79b7e0bc2b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:09.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.960+0000 7f11cdd9b640 1 -- 172.21.15.63:0/1890809660 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4037070 con 0x7f11d00ae8b0 2024-05-08T05:41:09.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.960+0000 7f11cdd9b640 1 -- 172.21.15.63:0/1890809660 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11c4032440 con 0x7f11d00ae8b0 2024-05-08T05:41:09.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.960+0000 7f11cdd9b640 1 -- 172.21.15.63:0/1890809660 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4032810 con 0x7f11d00ae8b0 2024-05-08T05:41:09.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.961+0000 7f11d5189640 1 -- 172.21.15.63:0/1890809660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11d00ae8b0 msgr2=0x7f11d0070bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.961+0000 7f11d5189640 1 --2- 172.21.15.63:0/1890809660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11d00ae8b0 0x7f11d0070bc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009870 tx=0x7f11c402a270 comp rx=0 tx=0).stop 2024-05-08T05:41:09.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.961+0000 7f11d5189640 1 -- 172.21.15.63:0/1890809660 shutdown_connections 2024-05-08T05:41:09.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.961+0000 7f11d5189640 1 --2- 172.21.15.63:0/1890809660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11d00ae8b0 0x7f11d0070bc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.961+0000 7f11d5189640 1 --2- 172.21.15.63:0/1890809660 >> v2:172.21.15.45:3300/0 conn(0x7f11d0162750 0x7f11d00af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.961+0000 7f11d5189640 1 -- 172.21.15.63:0/1890809660 >> 172.21.15.63:0/1890809660 conn(0x7f11d00738e0 msgr2=0x7f11d006c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:09.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.961+0000 7f11d5189640 1 -- 172.21.15.63:0/1890809660 shutdown_connections 2024-05-08T05:41:09.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.961+0000 7f11d5189640 1 -- 172.21.15.63:0/1890809660 wait complete. 2024-05-08T05:41:09.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.963+0000 7f11d5189640 1 Processor -- start 2024-05-08T05:41:09.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.963+0000 7f11d5189640 1 -- start start 2024-05-08T05:41:09.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.963+0000 7f11d5189640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11d00ae8b0 0x7f11d014b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.963+0000 7f11d5189640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11d0162750 0x7f11d015eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.963+0000 7f11d5189640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11d01607a0 0x7f11d01615a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.963+0000 7f11d5189640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f11d00b07b0 con 0x7f11d0162750 2024-05-08T05:41:09.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.963+0000 7f11d5189640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f11d00703c0 con 0x7f11d00ae8b0 2024-05-08T05:41:09.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.963+0000 7f11d5189640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f11d00b0930 con 0x7f11d01607a0 2024-05-08T05:41:09.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.963+0000 7f11cf59e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11d01607a0 0x7f11d01615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.964+0000 7f11ced9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11d00ae8b0 0x7f11d014b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.964+0000 7f11cf59e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11d01607a0 0x7f11d01615a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:47434/0 (socket says 172.21.15.63:47434) 2024-05-08T05:41:09.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.964+0000 7f11cf59e640 1 -- 172.21.15.63:0/684825744 learned_addr learned my addr 172.21.15.63:0/684825744 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-08T05:41:09.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.964+0000 7f11ce59c640 1 --2- 172.21.15.63:0/684825744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11d0162750 0x7f11d015eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.964+0000 7f11ced9d640 1 -- 172.21.15.63:0/684825744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11d01607a0 msgr2=0x7f11d01615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.964+0000 7f11ced9d640 1 --2- 172.21.15.63:0/684825744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11d01607a0 0x7f11d01615a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.964+0000 7f11ced9d640 1 -- 172.21.15.63:0/684825744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11d0162750 msgr2=0x7f11d015eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.964+0000 7f11ced9d640 1 --2- 172.21.15.63:0/684825744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11d0162750 0x7f11d015eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.964+0000 7f11ced9d640 1 -- 172.21.15.63:0/684825744 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d015f700 con 0x7f11d00ae8b0 2024-05-08T05:41:09.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.964+0000 7f11cf59e640 1 --2- 172.21.15.63:0/684825744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11d01607a0 0x7f11d01615a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:09.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.964+0000 7f11ce59c640 1 --2- 172.21.15.63:0/684825744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11d0162750 0x7f11d015eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:09.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.964+0000 7f11ced9d640 1 --2- 172.21.15.63:0/684825744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11d00ae8b0 0x7f11d014b020 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f11c000e870 tx=0x7f11c000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:09.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.964+0000 7f11b77fe640 1 -- 172.21.15.63:0/684825744 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c0017070 con 0x7f11d00ae8b0 2024-05-08T05:41:09.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.964+0000 7f11b77fe640 1 -- 172.21.15.63:0/684825744 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11c000ee60 con 0x7f11d00ae8b0 2024-05-08T05:41:09.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.964+0000 7f11d5189640 1 -- 172.21.15.63:0/684825744 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d015f9e0 con 0x7f11d00ae8b0 2024-05-08T05:41:09.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.964+0000 7f11b77fe640 1 -- 172.21.15.63:0/684825744 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c000cb60 con 0x7f11d00ae8b0 2024-05-08T05:41:09.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.964+0000 7f11d5189640 1 -- 172.21.15.63:0/684825744 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d015ffa0 con 0x7f11d00ae8b0 2024-05-08T05:41:09.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.964+0000 7f11b77fe640 1 -- 172.21.15.63:0/684825744 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f11c001a070 con 0x7f11d00ae8b0 2024-05-08T05:41:09.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.964+0000 7f11b77fe640 1 -- 172.21.15.63:0/684825744 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f11c001c9d0 con 0x7f11d00ae8b0 2024-05-08T05:41:09.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.965+0000 7f11d5189640 1 -- 172.21.15.63:0/684825744 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d0069b00 con 0x7f11d00ae8b0 2024-05-08T05:41:09.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:09.967+0000 7f11b77fe640 1 -- 172.21.15.63:0/684825744 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f11c000cd00 con 0x7f11d00ae8b0 2024-05-08T05:41:10.041 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.041+0000 7f11d5189640 1 -- 172.21.15.63:0/684825744 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd new", "uuid": "19f59f65-8cab-4bc4-a68d-07aab3fa695a", "id": 5} v 0) -- 0x7f11d00b1ca0 con 0x7f11d00ae8b0 2024-05-08T05:41:10.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.044+0000 7f11b77fe640 1 -- 172.21.15.63:0/684825744 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "19f59f65-8cab-4bc4-a68d-07aab3fa695a", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f11c001c450 con 0x7f11d00ae8b0 2024-05-08T05:41:10.045 INFO:teuthology.orchestra.run.smithi063.stdout:5 2024-05-08T05:41:10.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.045+0000 7f11d5189640 1 -- 172.21.15.63:0/684825744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11d00ae8b0 msgr2=0x7f11d014b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:10.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.045+0000 7f11d5189640 1 --2- 172.21.15.63:0/684825744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11d00ae8b0 0x7f11d014b020 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f11c000e870 tx=0x7f11c000ed30 comp rx=0 tx=0).stop 2024-05-08T05:41:10.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.045+0000 7f11d5189640 1 -- 172.21.15.63:0/684825744 shutdown_connections 2024-05-08T05:41:10.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.045+0000 7f11d5189640 1 --2- 172.21.15.63:0/684825744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f11d01607a0 0x7f11d01615a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.045+0000 7f11d5189640 1 --2- 172.21.15.63:0/684825744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11d0162750 0x7f11d015eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.045+0000 7f11d5189640 1 --2- 172.21.15.63:0/684825744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11d00ae8b0 0x7f11d014b020 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.045+0000 7f11d5189640 1 -- 172.21.15.63:0/684825744 >> 172.21.15.63:0/684825744 conn(0x7f11d00738e0 msgr2=0x7f11d00b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:10.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.045+0000 7f11d5189640 1 -- 172.21.15.63:0/684825744 shutdown_connections 2024-05-08T05:41:10.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.045+0000 7f11d5189640 1 -- 172.21.15.63:0/684825744 wait complete. 2024-05-08T05:41:10.056 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new 37dfc950-fffc-42d3-8d74-c27574554eee 6 2024-05-08T05:41:10.084 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:10.084+0000 7fba6e130040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:41:10.086 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:10.086+0000 7f9bc0327040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:41:10.136 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:10.135+0000 7f9bc0327040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:41:10.136 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:10.136+0000 7fba6e130040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:41:10.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.188+0000 7f1c8b7b6640 1 Processor -- start 2024-05-08T05:41:10.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.188+0000 7f1c8b7b6640 1 -- start start 2024-05-08T05:41:10.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.189+0000 7f1c8b7b6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1c84162750 0x7f1c840af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:10.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.189+0000 7f1c8b7b6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c84162b30 con 0x7f1c840ae8b0 2024-05-08T05:41:10.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.189+0000 7f1c8b7b6640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c840712d0 con 0x7f1c840b00a0 2024-05-08T05:41:10.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.189+0000 7f1c8b7b6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f1c84070730 con 0x7f1c84162750 2024-05-08T05:41:10.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.189+0000 7f1c89d53640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f1c840b00a0 0x7f1c840b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.63:43968/0 (socket says 172.21.15.63:43968) 2024-05-08T05:41:10.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.189+0000 7f1c89d53640 1 -- 172.21.15.63:0/1967567756 learned_addr learned my addr 172.21.15.63:0/1967567756 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-05-08T05:41:10.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.189+0000 7f1c89552640 1 --2- 172.21.15.63:0/1967567756 >> v2:172.21.15.45:3300/0 conn(0x7f1c84162750 0x7f1c840af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:10.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.189+0000 7f1c7bfff640 1 -- 172.21.15.63:0/1967567756 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3194104429 0 0) 0x7f1c840712d0 con 0x7f1c840b00a0 2024-05-08T05:41:10.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.189+0000 7f1c7bfff640 1 -- 172.21.15.63:0/1967567756 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c70003520 con 0x7f1c840b00a0 2024-05-08T05:41:10.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.190+0000 7f1c89552640 1 -- 172.21.15.63:0/1967567756 >> v1:172.21.15.63:6789/0 conn(0x7f1c840b00a0 legacy=0x7f1c840b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:10.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.190+0000 7f1c89552640 1 -- 172.21.15.63:0/1967567756 >> v1:172.21.15.45:6789/0 conn(0x7f1c840ae8b0 legacy=0x7f1c84070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:10.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.190+0000 7f1c89552640 1 -- 172.21.15.63:0/1967567756 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8406fb20 con 0x7f1c84162750 2024-05-08T05:41:10.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.190+0000 7f1c7bfff640 1 -- 172.21.15.63:0/1967567756 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1641475121 0 0) 0x7f1c84162b30 con 0x7f1c840ae8b0 2024-05-08T05:41:10.192 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:10.190+0000 7f9bc0327040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:41:10.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.190+0000 7f1c89552640 1 --2- 172.21.15.63:0/1967567756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c84162750 0x7f1c840af7a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1c800097a0 tx=0x7f1c8002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c789e5d93e7dd8e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:10.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.191+0000 7f1c7bfff640 1 -- 172.21.15.63:0/1967567756 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80037070 con 0x7f1c84162750 2024-05-08T05:41:10.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.191+0000 7f1c7bfff640 1 -- 172.21.15.63:0/1967567756 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c80032440 con 0x7f1c84162750 2024-05-08T05:41:10.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.191+0000 7f1c7bfff640 1 -- 172.21.15.63:0/1967567756 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80032810 con 0x7f1c84162750 2024-05-08T05:41:10.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.191+0000 7f1c8b7b6640 1 -- 172.21.15.63:0/1967567756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c84162750 msgr2=0x7f1c840af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:10.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.191+0000 7f1c8b7b6640 1 --2- 172.21.15.63:0/1967567756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c84162750 0x7f1c840af7a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1c800097a0 tx=0x7f1c8002a230 comp rx=0 tx=0).stop 2024-05-08T05:41:10.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.191+0000 7f1c8b7b6640 1 -- 172.21.15.63:0/1967567756 shutdown_connections 2024-05-08T05:41:10.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.191+0000 7f1c8b7b6640 1 --2- 172.21.15.63:0/1967567756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c84162750 0x7f1c840af7a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.191+0000 7f1c8b7b6640 1 -- 172.21.15.63:0/1967567756 >> 172.21.15.63:0/1967567756 conn(0x7f1c840738e0 msgr2=0x7f1c8406c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:10.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.191+0000 7f1c8b7b6640 1 -- 172.21.15.63:0/1967567756 shutdown_connections 2024-05-08T05:41:10.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.191+0000 7f1c8b7b6640 1 -- 172.21.15.63:0/1967567756 wait complete. 2024-05-08T05:41:10.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.193+0000 7f1c8b7b6640 1 Processor -- start 2024-05-08T05:41:10.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.193+0000 7f1c8b7b6640 1 -- start start 2024-05-08T05:41:10.194 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:10.193+0000 7fba6e130040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:41:10.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.193+0000 7f1c8b7b6640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c840ae8b0 0x7f1c84160f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:10.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.193+0000 7f1c8b7b6640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c840b00a0 0x7f1c8414aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:10.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.193+0000 7f1c8b7b6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c84162750 0x7f1c8415d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:10.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.193+0000 7f1c8b7b6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1c840703c0 con 0x7f1c84162750 2024-05-08T05:41:10.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.193+0000 7f1c8b7b6640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1c840700e0 con 0x7f1c840ae8b0 2024-05-08T05:41:10.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.193+0000 7f1c8b7b6640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1c840b4d60 con 0x7f1c840b00a0 2024-05-08T05:41:10.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.193+0000 7f1c89552640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c840ae8b0 0x7f1c84160f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:10.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.194+0000 7f1c89552640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c840ae8b0 0x7f1c84160f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35806/0 (socket says 172.21.15.63:35806) 2024-05-08T05:41:10.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.194+0000 7f1c89552640 1 -- 172.21.15.63:0/1887334671 learned_addr learned my addr 172.21.15.63:0/1887334671 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-08T05:41:10.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.194+0000 7f1c88d51640 1 --2- 172.21.15.63:0/1887334671 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c840b00a0 0x7f1c8414aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:10.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.194+0000 7f1c89d53640 1 --2- 172.21.15.63:0/1887334671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c84162750 0x7f1c8415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:10.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.194+0000 7f1c89552640 1 -- 172.21.15.63:0/1887334671 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c840b00a0 msgr2=0x7f1c8414aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:10.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.194+0000 7f1c89552640 1 --2- 172.21.15.63:0/1887334671 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c840b00a0 0x7f1c8414aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.194+0000 7f1c89552640 1 -- 172.21.15.63:0/1887334671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c84162750 msgr2=0x7f1c8415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:10.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.194+0000 7f1c89552640 1 --2- 172.21.15.63:0/1887334671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c84162750 0x7f1c8415d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.194+0000 7f1c89552640 1 -- 172.21.15.63:0/1887334671 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8414b3f0 con 0x7f1c840ae8b0 2024-05-08T05:41:10.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.194+0000 7f1c89552640 1 --2- 172.21.15.63:0/1887334671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c840ae8b0 0x7f1c84160f00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1c80035040 tx=0x7f1c800046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:10.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.194+0000 7f1c79ffb640 1 -- 172.21.15.63:0/1887334671 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80041020 con 0x7f1c840ae8b0 2024-05-08T05:41:10.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.194+0000 7f1c79ffb640 1 -- 172.21.15.63:0/1887334671 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c8003d6e0 con 0x7f1c840ae8b0 2024-05-08T05:41:10.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.194+0000 7f1c8b7b6640 1 -- 172.21.15.63:0/1887334671 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c8415f420 con 0x7f1c840ae8b0 2024-05-08T05:41:10.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.194+0000 7f1c79ffb640 1 -- 172.21.15.63:0/1887334671 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80037040 con 0x7f1c840ae8b0 2024-05-08T05:41:10.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.194+0000 7f1c8b7b6640 1 -- 172.21.15.63:0/1887334671 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c84160420 con 0x7f1c840ae8b0 2024-05-08T05:41:10.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.195+0000 7f1c79ffb640 1 -- 172.21.15.63:0/1887334671 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1c80044020 con 0x7f1c840ae8b0 2024-05-08T05:41:10.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.195+0000 7f1c79ffb640 1 -- 172.21.15.63:0/1887334671 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f1c80046bd0 con 0x7f1c840ae8b0 2024-05-08T05:41:10.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.195+0000 7f1c8b7b6640 1 -- 172.21.15.63:0/1887334671 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c84069b00 con 0x7f1c840ae8b0 2024-05-08T05:41:10.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.197+0000 7f1c79ffb640 1 -- 172.21.15.63:0/1887334671 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1c8004b030 con 0x7f1c840ae8b0 2024-05-08T05:41:10.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.272+0000 7f1c8b7b6640 1 -- 172.21.15.63:0/1887334671 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd new", "uuid": "37dfc950-fffc-42d3-8d74-c27574554eee", "id": 6} v 0) -- 0x7f1c8406d6c0 con 0x7f1c840ae8b0 2024-05-08T05:41:10.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.275+0000 7f1c79ffb640 1 -- 172.21.15.63:0/1887334671 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "37dfc950-fffc-42d3-8d74-c27574554eee", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f1c80046400 con 0x7f1c840ae8b0 2024-05-08T05:41:10.277 INFO:teuthology.orchestra.run.smithi063.stdout:6 2024-05-08T05:41:10.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.276+0000 7f1c8b7b6640 1 -- 172.21.15.63:0/1887334671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c840ae8b0 msgr2=0x7f1c84160f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:10.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.276+0000 7f1c8b7b6640 1 --2- 172.21.15.63:0/1887334671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c840ae8b0 0x7f1c84160f00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1c80035040 tx=0x7f1c800046c0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.276+0000 7f1c8b7b6640 1 -- 172.21.15.63:0/1887334671 shutdown_connections 2024-05-08T05:41:10.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.276+0000 7f1c8b7b6640 1 --2- 172.21.15.63:0/1887334671 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c840b00a0 0x7f1c8414aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.276+0000 7f1c8b7b6640 1 --2- 172.21.15.63:0/1887334671 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c84162750 0x7f1c8415d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.276+0000 7f1c8b7b6640 1 --2- 172.21.15.63:0/1887334671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c840ae8b0 0x7f1c84160f00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.276+0000 7f1c8b7b6640 1 -- 172.21.15.63:0/1887334671 >> 172.21.15.63:0/1887334671 conn(0x7f1c840738e0 msgr2=0x7f1c840b2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:10.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.276+0000 7f1c8b7b6640 1 -- 172.21.15.63:0/1887334671 shutdown_connections 2024-05-08T05:41:10.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.276+0000 7f1c8b7b6640 1 -- 172.21.15.63:0/1887334671 wait complete. 2024-05-08T05:41:10.286 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd new adf18b78-2e46-4ad0-84ce-296c5c431ebb 7 2024-05-08T05:41:10.309 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:10.309+0000 7f9bc0327040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:41:10.314 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:10.314+0000 7fba6e130040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:41:10.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.390+0000 7fa2a8a83640 1 Processor -- start 2024-05-08T05:41:10.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.390+0000 7fa2a8a83640 1 -- start start 2024-05-08T05:41:10.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.391+0000 7fa2a8a83640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa2a4071720 0x7fa2a415f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:10.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.391+0000 7fa2a8a83640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2a406c3a0 0x7fa2a406c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:10.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.391+0000 7fa2a8a83640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a4160600 con 0x7fa2a40b1050 2024-05-08T05:41:10.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.391+0000 7fa2a8a83640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa2a4071b80 con 0x7fa2a4071720 2024-05-08T05:41:10.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.391+0000 7fa2a8a83640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a406cd30 con 0x7fa2a406c3a0 2024-05-08T05:41:10.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.391+0000 7fa2a1d9b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa2a4071720 0x7fa2a415f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:10.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.391+0000 7fa2a259c640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fa2a40b1050 0x7fa2a40b56b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.63:43980/0 (socket says 172.21.15.63:43980) 2024-05-08T05:41:10.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.391+0000 7fa2a2d9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2a406c3a0 0x7fa2a406c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:10.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.391+0000 7fa2a259c640 1 -- 172.21.15.63:0/2415019282 learned_addr learned my addr 172.21.15.63:0/2415019282 (peer_addr_for_me v1:172.21.15.63:0/0) 2024-05-08T05:41:10.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.391+0000 7fa2a2d9d640 1 -- 172.21.15.63:0/2415019282 >> v2:172.21.15.63:3300/0 conn(0x7fa2a4071720 msgr2=0x7fa2a415f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:10.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.391+0000 7fa2a2d9d640 1 --2- 172.21.15.63:0/2415019282 >> v2:172.21.15.63:3300/0 conn(0x7fa2a4071720 0x7fa2a415f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.391+0000 7fa2a2d9d640 1 -- 172.21.15.63:0/2415019282 >> v1:172.21.15.63:6789/0 conn(0x7fa2a40b1050 legacy=0x7fa2a40b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:10.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.391+0000 7fa2a2d9d640 1 -- 172.21.15.63:0/2415019282 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a415fd60 con 0x7fa2a406c3a0 2024-05-08T05:41:10.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.391+0000 7fa2a1d9b640 1 --2- 172.21.15.63:0/2415019282 >> v2:172.21.15.63:3300/0 conn(0x7fa2a4071720 0x7fa2a415f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:10.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.392+0000 7fa2a2d9d640 1 --2- 172.21.15.63:0/2415019282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2a406c3a0 0x7fa2a406c7f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009d90 tx=0x7fa29c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91a6aee2e5af1790 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:10.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.392+0000 7fa2a159a640 1 -- 172.21.15.63:0/2415019282 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c007740 con 0x7fa2a406c3a0 2024-05-08T05:41:10.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.392+0000 7fa2a159a640 1 -- 172.21.15.63:0/2415019282 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa29c0078e0 con 0x7fa2a406c3a0 2024-05-08T05:41:10.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.392+0000 7fa2a159a640 1 -- 172.21.15.63:0/2415019282 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c007cb0 con 0x7fa2a406c3a0 2024-05-08T05:41:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.393+0000 7fa2a8a83640 1 -- 172.21.15.63:0/2415019282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2a406c3a0 msgr2=0x7fa2a406c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.393+0000 7fa2a8a83640 1 --2- 172.21.15.63:0/2415019282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2a406c3a0 0x7fa2a406c7f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009d90 tx=0x7fa29c02a660 comp rx=0 tx=0).stop 2024-05-08T05:41:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.393+0000 7fa2a8a83640 1 -- 172.21.15.63:0/2415019282 shutdown_connections 2024-05-08T05:41:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.393+0000 7fa2a8a83640 1 --2- 172.21.15.63:0/2415019282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2a406c3a0 0x7fa2a406c7f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.393+0000 7fa2a8a83640 1 --2- 172.21.15.63:0/2415019282 >> v2:172.21.15.63:3300/0 conn(0x7fa2a4071720 0x7fa2a415f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.393+0000 7fa2a8a83640 1 -- 172.21.15.63:0/2415019282 >> 172.21.15.63:0/2415019282 conn(0x7fa2a40b3c40 msgr2=0x7fa2a40b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.393+0000 7fa2a8a83640 1 -- 172.21.15.63:0/2415019282 shutdown_connections 2024-05-08T05:41:10.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.393+0000 7fa2a8a83640 1 -- 172.21.15.63:0/2415019282 wait complete. 2024-05-08T05:41:10.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.394+0000 7fa2a8a83640 1 Processor -- start 2024-05-08T05:41:10.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.394+0000 7fa2a8a83640 1 -- start start 2024-05-08T05:41:10.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.395+0000 7fa2a8a83640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2a406c3a0 0x7fa2a41577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:10.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.395+0000 7fa2a8a83640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2a4071720 0x7fa2a4159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:10.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.395+0000 7fa2a8a83640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2a40b1050 0x7fa2a413c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:10.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.395+0000 7fa2a8a83640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a4162d10 con 0x7fa2a4071720 2024-05-08T05:41:10.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.395+0000 7fa2a8a83640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a4160320 con 0x7fa2a40b1050 2024-05-08T05:41:10.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.395+0000 7fa2a8a83640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a40b5dc0 con 0x7fa2a406c3a0 2024-05-08T05:41:10.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.395+0000 7fa2a259c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2a406c3a0 0x7fa2a41577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:10.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.395+0000 7fa2a2d9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2a40b1050 0x7fa2a413c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:10.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.395+0000 7fa2a259c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2a406c3a0 0x7fa2a41577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.63:47466/0 (socket says 172.21.15.63:47466) 2024-05-08T05:41:10.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.395+0000 7fa2a2d9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2a40b1050 0x7fa2a413c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35828/0 (socket says 172.21.15.63:35828) 2024-05-08T05:41:10.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.395+0000 7fa2a259c640 1 -- 172.21.15.63:0/3089992298 learned_addr learned my addr 172.21.15.63:0/3089992298 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-05-08T05:41:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.396+0000 7fa2a1d9b640 1 --2- 172.21.15.63:0/3089992298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2a4071720 0x7fa2a4159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.396+0000 7fa2a2d9d640 1 -- 172.21.15.63:0/3089992298 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2a406c3a0 msgr2=0x7fa2a41577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.396+0000 7fa2a2d9d640 1 --2- 172.21.15.63:0/3089992298 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2a406c3a0 0x7fa2a41577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.396+0000 7fa2a2d9d640 1 -- 172.21.15.63:0/3089992298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2a4071720 msgr2=0x7fa2a4159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.396+0000 7fa2a2d9d640 1 --2- 172.21.15.63:0/3089992298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2a4071720 0x7fa2a4159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.396+0000 7fa2a2d9d640 1 -- 172.21.15.63:0/3089992298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a4157dd0 con 0x7fa2a40b1050 2024-05-08T05:41:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.396+0000 7fa2a2d9d640 1 --2- 172.21.15.63:0/3089992298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2a40b1050 0x7fa2a413c200 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa29c031000 tx=0x7fa29c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.396+0000 7fa292ffd640 1 -- 172.21.15.63:0/3089992298 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c03f070 con 0x7fa2a40b1050 2024-05-08T05:41:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.396+0000 7fa292ffd640 1 -- 172.21.15.63:0/3089992298 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa29c031c00 con 0x7fa2a40b1050 2024-05-08T05:41:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.396+0000 7fa2a8a83640 1 -- 172.21.15.63:0/3089992298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a41580b0 con 0x7fa2a40b1050 2024-05-08T05:41:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.396+0000 7fa292ffd640 1 -- 172.21.15.63:0/3089992298 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c009450 con 0x7fa2a40b1050 2024-05-08T05:41:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.396+0000 7fa2a8a83640 1 -- 172.21.15.63:0/3089992298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a4158670 con 0x7fa2a40b1050 2024-05-08T05:41:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.396+0000 7fa292ffd640 1 -- 172.21.15.63:0/3089992298 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa29c044460 con 0x7fa2a40b1050 2024-05-08T05:41:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.396+0000 7fa292ffd640 1 -- 172.21.15.63:0/3089992298 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fa29c03ba70 con 0x7fa2a40b1050 2024-05-08T05:41:10.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.397+0000 7fa2a8a83640 1 -- 172.21.15.63:0/3089992298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a4069b00 con 0x7fa2a40b1050 2024-05-08T05:41:10.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.399+0000 7fa292ffd640 1 -- 172.21.15.63:0/3089992298 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa29c03bd10 con 0x7fa2a40b1050 2024-05-08T05:41:10.474 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.474+0000 7fa2a8a83640 1 -- 172.21.15.63:0/3089992298 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd new", "uuid": "adf18b78-2e46-4ad0-84ce-296c5c431ebb", "id": 7} v 0) -- 0x7fa2a406d290 con 0x7fa2a40b1050 2024-05-08T05:41:10.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.479+0000 7fa292ffd640 1 -- 172.21.15.63:0/3089992298 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "adf18b78-2e46-4ad0-84ce-296c5c431ebb", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fa29c03b410 con 0x7fa2a40b1050 2024-05-08T05:41:10.480 INFO:teuthology.orchestra.run.smithi063.stdout:7 2024-05-08T05:41:10.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.480+0000 7fa2a8a83640 1 -- 172.21.15.63:0/3089992298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2a40b1050 msgr2=0x7fa2a413c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:10.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.480+0000 7fa2a8a83640 1 --2- 172.21.15.63:0/3089992298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2a40b1050 0x7fa2a413c200 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa29c031000 tx=0x7fa29c004a40 comp rx=0 tx=0).stop 2024-05-08T05:41:10.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.480+0000 7fa2a8a83640 1 -- 172.21.15.63:0/3089992298 shutdown_connections 2024-05-08T05:41:10.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.480+0000 7fa2a8a83640 1 --2- 172.21.15.63:0/3089992298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2a40b1050 0x7fa2a413c200 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.480+0000 7fa2a8a83640 1 --2- 172.21.15.63:0/3089992298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2a4071720 0x7fa2a4159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.480+0000 7fa2a8a83640 1 --2- 172.21.15.63:0/3089992298 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2a406c3a0 0x7fa2a41577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.480+0000 7fa2a8a83640 1 -- 172.21.15.63:0/3089992298 >> 172.21.15.63:0/3089992298 conn(0x7fa2a40b3c40 msgr2=0x7fa2a40af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:10.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.480+0000 7fa2a8a83640 1 -- 172.21.15.63:0/3089992298 shutdown_connections 2024-05-08T05:41:10.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:10.480+0000 7fa2a8a83640 1 -- 172.21.15.63:0/3089992298 wait complete. 2024-05-08T05:41:10.491 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-08T05:41:10.491 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-08T05:41:10.494 INFO:tasks.ceph.osd.0:Started 2024-05-08T05:41:10.494 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-08T05:41:10.494 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-08T05:41:10.496 INFO:tasks.ceph.osd.1:Started 2024-05-08T05:41:10.496 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-08T05:41:10.496 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-08T05:41:10.498 INFO:tasks.ceph.osd.2:Started 2024-05-08T05:41:10.498 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-08T05:41:10.499 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-08T05:41:10.501 INFO:tasks.ceph.osd.3:Started 2024-05-08T05:41:10.501 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-08T05:41:10.501 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-08T05:41:10.533 INFO:tasks.ceph.osd.4:Started 2024-05-08T05:41:10.534 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-08T05:41:10.534 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-08T05:41:10.536 INFO:tasks.ceph.osd.5:Started 2024-05-08T05:41:10.536 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-08T05:41:10.536 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-08T05:41:10.538 INFO:tasks.ceph.osd.6:Started 2024-05-08T05:41:10.538 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-08T05:41:10.539 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-08T05:41:10.541 INFO:tasks.ceph.osd.7:Started 2024-05-08T05:41:10.541 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:41:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.663+0000 7f941a899640 1 Processor -- start 2024-05-08T05:41:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.663+0000 7f941a899640 1 -- start start 2024-05-08T05:41:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.664+0000 7f941a899640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f94140cfea0 0x7f94141633e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.664+0000 7f941a899640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94140d0500 con 0x7f9414162f50 2024-05-08T05:41:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.664+0000 7f9418e36640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f94140cfea0 0x7f94141633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.664+0000 7f940bfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9414162f50 0x7f94140cf790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35102/0 (socket says 172.21.15.45:35102) 2024-05-08T05:41:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.664+0000 7f940bfff640 1 -- 172.21.15.45:0/304383358 learned_addr learned my addr 172.21.15.45:0/304383358 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.664+0000 7f941a899640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f941406da60 con 0x7f94140b21f0 2024-05-08T05:41:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.664+0000 7f941a899640 1 -- 172.21.15.45:0/304383358 --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f941406d810 con 0x7f94140cfea0 2024-05-08T05:41:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.664+0000 7f94137fe640 1 -- 172.21.15.45:0/304383358 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1511745539 0 0) 0x7f941406da60 con 0x7f94140b21f0 2024-05-08T05:41:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.664+0000 7f94137fe640 1 -- 172.21.15.45:0/304383358 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9400003520 con 0x7f94140b21f0 2024-05-08T05:41:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.664+0000 7f9418e36640 1 -- 172.21.15.45:0/304383358 >> v1:172.21.15.63:6789/0 conn(0x7f94140b21f0 legacy=0x7f941406e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.664+0000 7f9418e36640 1 -- 172.21.15.45:0/304383358 >> v1:172.21.15.45:6789/0 conn(0x7f9414162f50 legacy=0x7f94140cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.664+0000 7f94137fe640 1 -- 172.21.15.45:0/304383358 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 147797831 0 0) 0x7f94140d0500 con 0x7f9414162f50 2024-05-08T05:41:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.664+0000 7f9418e36640 1 -- 172.21.15.45:0/304383358 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f941406dd40 con 0x7f94140cfea0 2024-05-08T05:41:10.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.665+0000 7f9418e36640 1 --2- 172.21.15.45:0/304383358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94140cfea0 0x7f94141633e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f940c009d90 tx=0x7f940c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8336d2b4fbadd975 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:10.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.665+0000 7f94137fe640 1 -- 172.21.15.45:0/304383358 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c031600 con 0x7f94140cfea0 2024-05-08T05:41:10.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.665+0000 7f94137fe640 1 -- 172.21.15.45:0/304383358 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f940c0317a0 con 0x7f94140cfea0 2024-05-08T05:41:10.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.665+0000 7f94137fe640 1 -- 172.21.15.45:0/304383358 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c031b70 con 0x7f94140cfea0 2024-05-08T05:41:10.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.666+0000 7f941a899640 1 -- 172.21.15.45:0/304383358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94140cfea0 msgr2=0x7f94141633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:10.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.666+0000 7f941a899640 1 --2- 172.21.15.45:0/304383358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94140cfea0 0x7f94141633e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f940c009d90 tx=0x7f940c02a5d0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.666+0000 7f941a899640 1 -- 172.21.15.45:0/304383358 shutdown_connections 2024-05-08T05:41:10.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.666+0000 7f941a899640 1 --2- 172.21.15.45:0/304383358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94140cfea0 0x7f94141633e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.666+0000 7f941a899640 1 -- 172.21.15.45:0/304383358 >> 172.21.15.45:0/304383358 conn(0x7f941409ba40 msgr2=0x7f941409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:10.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.666+0000 7f941a899640 1 -- 172.21.15.45:0/304383358 shutdown_connections 2024-05-08T05:41:10.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.666+0000 7f941a899640 1 -- 172.21.15.45:0/304383358 wait complete. 2024-05-08T05:41:10.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.667+0000 7f941a899640 1 Processor -- start 2024-05-08T05:41:10.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.667+0000 7f941a899640 1 -- start start 2024-05-08T05:41:10.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.668+0000 7f941a899640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94140b21f0 0x7f94141f8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:10.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.668+0000 7f941a899640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94140cfea0 0x7f94140c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:10.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.668+0000 7f941a899640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9414162f50 0x7f94140cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:10.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.668+0000 7f941a899640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f94140cd480 con 0x7f94140b21f0 2024-05-08T05:41:10.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.668+0000 7f941a899640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f941406e0b0 con 0x7f9414162f50 2024-05-08T05:41:10.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.668+0000 7f941a899640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f94140cd600 con 0x7f94140cfea0 2024-05-08T05:41:10.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.668+0000 7f9413fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94140b21f0 0x7f94141f8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:10.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.668+0000 7f9413fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94140b21f0 0x7f94141f8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35546/0 (socket says 172.21.15.45:35546) 2024-05-08T05:41:10.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.668+0000 7f9413fff640 1 -- 172.21.15.45:0/2626656108 learned_addr learned my addr 172.21.15.45:0/2626656108 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:10.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.668+0000 7f9418e36640 1 --2- 172.21.15.45:0/2626656108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9414162f50 0x7f94140cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:10.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.668+0000 7f940bfff640 1 --2- 172.21.15.45:0/2626656108 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94140cfea0 0x7f94140c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:10.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.668+0000 7f9413fff640 1 -- 172.21.15.45:0/2626656108 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94140cfea0 msgr2=0x7f94140c6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:10.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.668+0000 7f9413fff640 1 --2- 172.21.15.45:0/2626656108 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94140cfea0 0x7f94140c6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.668+0000 7f9413fff640 1 -- 172.21.15.45:0/2626656108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9414162f50 msgr2=0x7f94140cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:10.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.668+0000 7f9413fff640 1 --2- 172.21.15.45:0/2626656108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9414162f50 0x7f94140cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.668+0000 7f9413fff640 1 -- 172.21.15.45:0/2626656108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94140c7310 con 0x7f94140b21f0 2024-05-08T05:41:10.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.669+0000 7f9413fff640 1 --2- 172.21.15.45:0/2626656108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94140b21f0 0x7f94141f8f30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f940400c490 tx=0x7f940400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:10.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.669+0000 7f94117fa640 1 -- 172.21.15.45:0/2626656108 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94040052d0 con 0x7f94140b21f0 2024-05-08T05:41:10.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.669+0000 7f94117fa640 1 -- 172.21.15.45:0/2626656108 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9404005490 con 0x7f94140b21f0 2024-05-08T05:41:10.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.669+0000 7f941a899640 1 -- 172.21.15.45:0/2626656108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94141f4f40 con 0x7f94140b21f0 2024-05-08T05:41:10.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.669+0000 7f94117fa640 1 -- 172.21.15.45:0/2626656108 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9404012870 con 0x7f94140b21f0 2024-05-08T05:41:10.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.669+0000 7f941a899640 1 -- 172.21.15.45:0/2626656108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94140c75f0 con 0x7f94140b21f0 2024-05-08T05:41:10.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.669+0000 7f94117fa640 1 -- 172.21.15.45:0/2626656108 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9404012a10 con 0x7f94140b21f0 2024-05-08T05:41:10.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.669+0000 7f94117fa640 1 -- 172.21.15.45:0/2626656108 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f940401bb70 con 0x7f94140b21f0 2024-05-08T05:41:10.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.670+0000 7f941a899640 1 -- 172.21.15.45:0/2626656108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9414069b00 con 0x7f94140b21f0 2024-05-08T05:41:10.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.672+0000 7f94117fa640 1 -- 172.21.15.45:0/2626656108 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f940401b4a0 con 0x7f94140b21f0 2024-05-08T05:41:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.734+0000 7f941a899640 1 -- 172.21.15.45:0/2626656108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9414070a30 con 0x7f94140b21f0 2024-05-08T05:41:10.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.735+0000 7f94117fa640 1 -- 172.21.15.45:0/2626656108 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f940401b680 con 0x7f94140b21f0 2024-05-08T05:41:10.735 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:10.736 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":10,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:41:10.475782+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:41:10.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.736+0000 7f941a899640 1 -- 172.21.15.45:0/2626656108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94140b21f0 msgr2=0x7f94141f8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.736+0000 7f941a899640 1 --2- 172.21.15.45:0/2626656108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94140b21f0 0x7f94141f8f30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f940400c490 tx=0x7f940400c950 comp rx=0 tx=0).stop 2024-05-08T05:41:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.736+0000 7f941a899640 1 -- 172.21.15.45:0/2626656108 shutdown_connections 2024-05-08T05:41:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.736+0000 7f941a899640 1 --2- 172.21.15.45:0/2626656108 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f94140cfea0 0x7f94140c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.736+0000 7f941a899640 1 --2- 172.21.15.45:0/2626656108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9414162f50 0x7f94140cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.736+0000 7f941a899640 1 --2- 172.21.15.45:0/2626656108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94140b21f0 0x7f94141f8f30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.736+0000 7f941a899640 1 -- 172.21.15.45:0/2626656108 >> 172.21.15.45:0/2626656108 conn(0x7f941409ba40 msgr2=0x7f94140714d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.736+0000 7f941a899640 1 -- 172.21.15.45:0/2626656108 shutdown_connections 2024-05-08T05:41:10.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:10.736+0000 7f941a899640 1 -- 172.21.15.45:0/2626656108 wait complete. 2024-05-08T05:41:10.747 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-08T05:41:10.748 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-08T05:41:10.748 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:10.747+0000 7fba6e130040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:41:10.751 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:10.751+0000 7f9bc0327040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:41:10.894 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:10.894+0000 7fba6e130040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:41:10.897 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:10.897+0000 7f9bc0327040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:41:11.108 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-05-08T05:41:11.107+0000 7fc24061f640 -1 Falling back to public interface 2024-05-08T05:41:11.117 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-05-08T05:41:11.117+0000 7f78a440d640 -1 Falling back to public interface 2024-05-08T05:41:11.121 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-05-08T05:41:11.121+0000 7f709d6e8640 -1 Falling back to public interface 2024-05-08T05:41:11.124 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-05-08T05:41:11.124+0000 7f2a4e501640 -1 Falling back to public interface 2024-05-08T05:41:11.149 INFO:tasks.ceph.osd.6.smithi063.stderr:2024-05-08T05:41:11.148+0000 7f4f8a686640 -1 Falling back to public interface 2024-05-08T05:41:11.154 INFO:tasks.ceph.osd.5.smithi063.stderr:2024-05-08T05:41:11.154+0000 7fa33620d640 -1 Falling back to public interface 2024-05-08T05:41:11.167 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:11.167+0000 7fba6e130040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:41:11.167 INFO:tasks.ceph.osd.7.smithi063.stderr:2024-05-08T05:41:11.167+0000 7fd40d6db640 -1 Falling back to public interface 2024-05-08T05:41:11.174 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:11.174+0000 7f9bc0327040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:41:11.174 INFO:tasks.ceph.osd.4.smithi063.stderr:2024-05-08T05:41:11.175+0000 7fd376c13640 -1 Falling back to public interface 2024-05-08T05:41:11.227 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:11.227+0000 7fba6e130040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:41:11.229 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:11.229+0000 7f9bc0327040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:41:11.351 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:11.351+0000 7fba6e130040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:41:11.352 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:11.352+0000 7f9bc0327040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:41:11.553 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T05:41:11.649 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.649+0000 7fb2bd0a0640 1 Processor -- start 2024-05-08T05:41:11.650 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.649+0000 7fb2bd0a0640 1 -- start start 2024-05-08T05:41:11.653 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2bd0a0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2b815cee0 0x7fb2b8160460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:11.653 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2bd0a0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2b8070ca0 con 0x7fb2b8070250 2024-05-08T05:41:11.653 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2bd0a0640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2b815f980 con 0x7fb2b80b5480 2024-05-08T05:41:11.653 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2bd0a0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b815fc60 con 0x7fb2b815cee0 2024-05-08T05:41:11.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2b659c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb2b8070250 0x7fb2b80b4d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35112/0 (socket says 172.21.15.45:35112) 2024-05-08T05:41:11.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2b659c640 1 -- 172.21.15.45:0/1720456322 learned_addr learned my addr 172.21.15.45:0/1720456322 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:11.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2b6d9d640 1 --2- 172.21.15.45:0/1720456322 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2b815cee0 0x7fb2b8160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:11.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2b5d9b640 1 -- 172.21.15.45:0/1720456322 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2833584488 0 0) 0x7fb2b8070ca0 con 0x7fb2b8070250 2024-05-08T05:41:11.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2b5d9b640 1 -- 172.21.15.45:0/1720456322 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2a0003520 con 0x7fb2b8070250 2024-05-08T05:41:11.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2b5d9b640 1 -- 172.21.15.45:0/1720456322 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2461226569 0 0) 0x7fb2a0003520 con 0x7fb2b8070250 2024-05-08T05:41:11.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2b5d9b640 1 -- 172.21.15.45:0/1720456322 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2b815cee0 msgr2=0x7fb2b8160460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:11.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2b5d9b640 1 --2- 172.21.15.45:0/1720456322 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2b815cee0 0x7fb2b8160460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:11.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2b5d9b640 1 -- 172.21.15.45:0/1720456322 >> v1:172.21.15.63:6789/0 conn(0x7fb2b80b5480 legacy=0x7fb2b806ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:11.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2b5d9b640 1 -- 172.21.15.45:0/1720456322 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b815ff40 con 0x7fb2b8070250 2024-05-08T05:41:11.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2b5d9b640 1 -- 172.21.15.45:0/1720456322 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fb2ac0016f0 con 0x7fb2b8070250 2024-05-08T05:41:11.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2b5d9b640 1 -- 172.21.15.45:0/1720456322 >> v1:172.21.15.45:6789/0 conn(0x7fb2b8070250 legacy=0x7fb2b80b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:11.655 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2b5d9b640 1 --2- 172.21.15.45:0/1720456322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2a00039e0 0x7fb2a0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:11.655 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2b5d9b640 1 --2- 172.21.15.45:0/1720456322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2a0004800 0x7fb2a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:11.655 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2b5d9b640 1 --2- 172.21.15.45:0/1720456322 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2b815cee0 0x7fb2a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:11.655 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2b5d9b640 1 -- 172.21.15.45:0/1720456322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b815fc60 con 0x7fb2a0004800 2024-05-08T05:41:11.655 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2b5d9b640 1 -- 172.21.15.45:0/1720456322 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a000ba70 con 0x7fb2a00039e0 2024-05-08T05:41:11.655 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.650+0000 7fb2b5d9b640 1 -- 172.21.15.45:0/1720456322 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a000bc70 con 0x7fb2b815cee0 2024-05-08T05:41:11.655 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.651+0000 7fb2b659c640 1 --2- 172.21.15.45:0/1720456322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2a0004800 0x7fb2a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:11.655 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.651+0000 7fb2b6d9d640 1 --2- 172.21.15.45:0/1720456322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2a00039e0 0x7fb2a0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:11.655 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.651+0000 7fb2b759e640 1 --2- 172.21.15.45:0/1720456322 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2b815cee0 0x7fb2a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:11.655 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.651+0000 7fb2b659c640 1 -- 172.21.15.45:0/1720456322 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2b815cee0 msgr2=0x7fb2a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:11.655 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.651+0000 7fb2b659c640 1 --2- 172.21.15.45:0/1720456322 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2b815cee0 0x7fb2a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:11.655 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.651+0000 7fb2b659c640 1 -- 172.21.15.45:0/1720456322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2a00039e0 msgr2=0x7fb2a0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:11.656 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.651+0000 7fb2b659c640 1 --2- 172.21.15.45:0/1720456322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2a00039e0 0x7fb2a0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:11.656 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.651+0000 7fb2b659c640 1 -- 172.21.15.45:0/1720456322 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a0003520 con 0x7fb2a0004800 2024-05-08T05:41:11.656 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.651+0000 7fb2b659c640 1 --2- 172.21.15.45:0/1720456322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2a0004800 0x7fb2a0004c80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac001eb0 tx=0x7fb2a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d986171dfe9a7849 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:11.656 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.652+0000 7fb2b5d9b640 1 -- 172.21.15.45:0/1720456322 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac007d10 con 0x7fb2a0004800 2024-05-08T05:41:11.656 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.652+0000 7fb2b5d9b640 1 -- 172.21.15.45:0/1720456322 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2ac005ad0 con 0x7fb2a0004800 2024-05-08T05:41:11.656 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.652+0000 7fb2b5d9b640 1 -- 172.21.15.45:0/1720456322 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac007070 con 0x7fb2a0004800 2024-05-08T05:41:11.656 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.652+0000 7fb2bd0a0640 1 -- 172.21.15.45:0/1720456322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2a0004800 msgr2=0x7fb2a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:11.656 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.652+0000 7fb2bd0a0640 1 --2- 172.21.15.45:0/1720456322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2a0004800 0x7fb2a0004c80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac001eb0 tx=0x7fb2a0005830 comp rx=0 tx=0).stop 2024-05-08T05:41:11.656 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.652+0000 7fb2b759e640 1 -- 172.21.15.45:0/1720456322 reap_dead start 2024-05-08T05:41:11.656 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.652+0000 7fb2bd0a0640 1 -- 172.21.15.45:0/1720456322 shutdown_connections 2024-05-08T05:41:11.656 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.652+0000 7fb2bd0a0640 1 -- 172.21.15.45:0/1720456322 >> 172.21.15.45:0/1720456322 conn(0x7fb2b80b3a70 msgr2=0x7fb2b80b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:11.656 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.652+0000 7fb2bd0a0640 1 -- 172.21.15.45:0/1720456322 shutdown_connections 2024-05-08T05:41:11.656 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.652+0000 7fb2bd0a0640 1 -- 172.21.15.45:0/1720456322 wait complete. 2024-05-08T05:41:11.659 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.654+0000 7fb2bd0a0640 1 Processor -- start 2024-05-08T05:41:11.659 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.654+0000 7fb2bd0a0640 1 -- start start 2024-05-08T05:41:11.659 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.654+0000 7fb2bd0a0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2a0004800 0x7fb2b80aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:11.659 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.654+0000 7fb2bd0a0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2b8153ee0 0x7fb2b80af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:11.659 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.655+0000 7fb2bd0a0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2b81613b0 0x7fb2b8161830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:11.659 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.655+0000 7fb2bd0a0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b80b7020 con 0x7fb2b81613b0 2024-05-08T05:41:11.659 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.655+0000 7fb2bd0a0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b80b6ea0 con 0x7fb2b8153ee0 2024-05-08T05:41:11.659 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.655+0000 7fb2bd0a0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b8160460 con 0x7fb2a0004800 2024-05-08T05:41:11.660 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.655+0000 7fb2b759e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2b81613b0 0x7fb2b8161830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:11.660 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.655+0000 7fb2b759e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2b81613b0 0x7fb2b8161830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35566/0 (socket says 172.21.15.45:35566) 2024-05-08T05:41:11.660 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.655+0000 7fb2b759e640 1 -- 172.21.15.45:0/371969946 learned_addr learned my addr 172.21.15.45:0/371969946 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:11.660 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.655+0000 7fb2b659c640 1 --2- 172.21.15.45:0/371969946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2b8153ee0 0x7fb2b80af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:11.660 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.655+0000 7fb2b6d9d640 1 --2- 172.21.15.45:0/371969946 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2a0004800 0x7fb2b80aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:11.660 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.655+0000 7fb2b759e640 1 -- 172.21.15.45:0/371969946 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2a0004800 msgr2=0x7fb2b80aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:11.660 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.655+0000 7fb2b759e640 1 --2- 172.21.15.45:0/371969946 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2a0004800 0x7fb2b80aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:11.660 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.655+0000 7fb2b759e640 1 -- 172.21.15.45:0/371969946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2b8153ee0 msgr2=0x7fb2b80af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:11.660 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.655+0000 7fb2b759e640 1 --2- 172.21.15.45:0/371969946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2b8153ee0 0x7fb2b80af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:11.660 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.655+0000 7fb2b759e640 1 -- 172.21.15.45:0/371969946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b80ceea0 con 0x7fb2b81613b0 2024-05-08T05:41:11.660 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.655+0000 7fb2b759e640 1 --2- 172.21.15.45:0/371969946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2b81613b0 0x7fb2b8161830 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0001520 tx=0x7fb2b000ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:11.660 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.655+0000 7fb29f7fe640 1 -- 172.21.15.45:0/371969946 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0003b80 con 0x7fb2b81613b0 2024-05-08T05:41:11.660 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.655+0000 7fb29f7fe640 1 -- 172.21.15.45:0/371969946 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b0018070 con 0x7fb2b81613b0 2024-05-08T05:41:11.660 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.655+0000 7fb29f7fe640 1 -- 172.21.15.45:0/371969946 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b000d770 con 0x7fb2b81613b0 2024-05-08T05:41:11.661 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.655+0000 7fb2bd0a0640 1 -- 172.21.15.45:0/371969946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b81629f0 con 0x7fb2b81613b0 2024-05-08T05:41:11.661 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.655+0000 7fb2bd0a0640 1 -- 172.21.15.45:0/371969946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b8162430 con 0x7fb2b81613b0 2024-05-08T05:41:11.661 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.656+0000 7fb29f7fe640 1 -- 172.21.15.45:0/371969946 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb2b000d910 con 0x7fb2b81613b0 2024-05-08T05:41:11.661 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.656+0000 7fb29f7fe640 1 -- 172.21.15.45:0/371969946 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fb2b001db30 con 0x7fb2b81613b0 2024-05-08T05:41:11.661 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.656+0000 7fb2bd0a0640 1 -- 172.21.15.45:0/371969946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b8076720 con 0x7fb2b81613b0 2024-05-08T05:41:11.661 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.658+0000 7fb29f7fe640 1 -- 172.21.15.45:0/371969946 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb2b80ceea0 con 0x7fb2b81613b0 2024-05-08T05:41:11.661 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:11.661+0000 7fba6e130040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:41:11.669 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:11.669+0000 7f9bc0327040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:41:11.711 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:11.711+0000 7fba6e130040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:41:11.719 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:11.719+0000 7f9bc0327040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:41:11.721 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.721+0000 7fb2bd0a0640 1 -- 172.21.15.45:0/371969946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2b806d200 con 0x7fb2b81613b0 2024-05-08T05:41:11.722 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.722+0000 7fb29f7fe640 1 -- 172.21.15.45:0/371969946 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fb2b000dd90 con 0x7fb2b81613b0 2024-05-08T05:41:11.722 INFO:teuthology.misc.health.smithi045.stdout: 2024-05-08T05:41:11.722 INFO:teuthology.misc.health.smithi045.stdout:{"epoch":10,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:41:10.475782+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:41:11.723 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.723+0000 7fb2bd0a0640 1 -- 172.21.15.45:0/371969946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2b81613b0 msgr2=0x7fb2b8161830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:11.723 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.723+0000 7fb2bd0a0640 1 --2- 172.21.15.45:0/371969946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2b81613b0 0x7fb2b8161830 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0001520 tx=0x7fb2b000ff70 comp rx=0 tx=0).stop 2024-05-08T05:41:11.723 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.723+0000 7fb2bd0a0640 1 -- 172.21.15.45:0/371969946 shutdown_connections 2024-05-08T05:41:11.724 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.723+0000 7fb2bd0a0640 1 --2- 172.21.15.45:0/371969946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2b81613b0 0x7fb2b8161830 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:11.724 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.723+0000 7fb2bd0a0640 1 --2- 172.21.15.45:0/371969946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2b8153ee0 0x7fb2b80af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:11.724 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.723+0000 7fb2bd0a0640 1 --2- 172.21.15.45:0/371969946 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2a0004800 0x7fb2b80aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:11.724 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.723+0000 7fb2bd0a0640 1 -- 172.21.15.45:0/371969946 >> 172.21.15.45:0/371969946 conn(0x7fb2b80b3a70 msgr2=0x7fb2b80b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:11.724 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.723+0000 7fb2bd0a0640 1 -- 172.21.15.45:0/371969946 shutdown_connections 2024-05-08T05:41:11.724 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:11.723+0000 7fb2bd0a0640 1 -- 172.21.15.45:0/371969946 wait complete. 2024-05-08T05:41:11.733 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-08T05:41:11.876 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:11.875+0000 7fba6e130040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:41:11.884 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:11.884+0000 7f9bc0327040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:41:12.020 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:12.020+0000 7fba6e130040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:41:12.029 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:12.029+0000 7f9bc0327040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:41:12.072 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:12.072+0000 7fba6e130040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:41:12.081 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:12.081+0000 7f9bc0327040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:41:12.146 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:12.146+0000 7fba6e130040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:41:12.154 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:12.154+0000 7f9bc0327040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:41:12.193 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:12.193+0000 7fba6e130040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:41:12.202 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:12.202+0000 7f9bc0327040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:41:12.698 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:12.697+0000 7fba6e130040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:41:12.706 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:12.706+0000 7f9bc0327040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:41:12.820 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:12.819+0000 7fba6e130040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:41:12.828 INFO:tasks.ceph.mgr.y.smithi045.stderr:2024-05-08T05:41:12.828+0000 7f9bc0327040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:41:13.836 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:13.836+0000 7fba0e4d3640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-08T05:41:13.837 INFO:tasks.ceph.mgr.x.smithi063.stderr:2024-05-08T05:41:13.836+0000 7fba0e4d3640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-08T05:41:14.597 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-05-08T05:41:14.597+0000 7fc24061f640 -1 osd.0 0 log_to_monitors true 2024-05-08T05:41:14.641 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-05-08T05:41:14.641+0000 7f2a4e501640 -1 osd.1 0 log_to_monitors true 2024-05-08T05:41:14.641 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-05-08T05:41:14.641+0000 7f78a440d640 -1 osd.2 0 log_to_monitors true 2024-05-08T05:41:14.642 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-05-08T05:41:14.642+0000 7f709d6e8640 -1 osd.3 0 log_to_monitors true 2024-05-08T05:41:14.654 INFO:tasks.ceph.osd.6.smithi063.stderr:2024-05-08T05:41:14.654+0000 7f4f8a686640 -1 osd.6 0 log_to_monitors true 2024-05-08T05:41:14.661 INFO:tasks.ceph.osd.5.smithi063.stderr:2024-05-08T05:41:14.661+0000 7fa33620d640 -1 osd.5 0 log_to_monitors true 2024-05-08T05:41:14.668 INFO:tasks.ceph.osd.4.smithi063.stderr:2024-05-08T05:41:14.668+0000 7fd376c13640 -1 osd.4 0 log_to_monitors true 2024-05-08T05:41:14.671 INFO:tasks.ceph.osd.7.smithi063.stderr:2024-05-08T05:41:14.671+0000 7fd40d6db640 -1 osd.7 0 log_to_monitors true 2024-05-08T05:41:17.213 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-05-08T05:41:17.213+0000 7f789fd04640 -1 osd.2 0 waiting for initial osdmap 2024-05-08T05:41:17.216 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-05-08T05:41:17.216+0000 7fc23c5b5640 -1 osd.0 0 waiting for initial osdmap 2024-05-08T05:41:17.227 INFO:tasks.ceph.osd.0.smithi045.stderr:2024-05-08T05:41:17.227+0000 7fc2343d5640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:41:17.230 INFO:tasks.ceph.osd.2.smithi045.stderr:2024-05-08T05:41:17.230+0000 7f7897b24640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:41:17.314 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-05-08T05:41:17.314+0000 7f2a4a497640 -1 osd.1 0 waiting for initial osdmap 2024-05-08T05:41:17.325 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-05-08T05:41:17.325+0000 7f709967e640 -1 osd.3 0 waiting for initial osdmap 2024-05-08T05:41:17.327 INFO:tasks.ceph.osd.1.smithi045.stderr:2024-05-08T05:41:17.327+0000 7f2a422b7640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:41:17.343 INFO:tasks.ceph.osd.3.smithi045.stderr:2024-05-08T05:41:17.343+0000 7f709149e640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:41:17.745 INFO:tasks.ceph.osd.4.smithi063.stderr:2024-05-08T05:41:17.745+0000 7fd372d38640 -1 osd.4 0 waiting for initial osdmap 2024-05-08T05:41:17.771 INFO:tasks.ceph.osd.4.smithi063.stderr:2024-05-08T05:41:17.771+0000 7fd36a344640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:41:17.980 INFO:tasks.ceph.osd.6.smithi063.stderr:2024-05-08T05:41:17.980+0000 7f4f8661c640 -1 osd.6 0 waiting for initial osdmap 2024-05-08T05:41:17.989 INFO:tasks.ceph.osd.7.smithi063.stderr:2024-05-08T05:41:17.989+0000 7fd409671640 -1 osd.7 0 waiting for initial osdmap 2024-05-08T05:41:17.990 INFO:tasks.ceph.osd.5.smithi063.stderr:2024-05-08T05:41:17.989+0000 7fa331c38640 -1 osd.5 0 waiting for initial osdmap 2024-05-08T05:41:17.994 INFO:tasks.ceph.osd.6.smithi063.stderr:2024-05-08T05:41:17.994+0000 7f4f7e43c640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:41:18.008 INFO:tasks.ceph.osd.5.smithi063.stderr:2024-05-08T05:41:18.008+0000 7fa329a58640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:41:18.008 INFO:tasks.ceph.osd.7.smithi063.stderr:2024-05-08T05:41:18.008+0000 7fd401491640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:41:18.539 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T05:41:18.645 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.644+0000 7f1dbfbc9640 1 Processor -- start 2024-05-08T05:41:18.645 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.644+0000 7f1dbfbc9640 1 -- start start 2024-05-08T05:41:18.646 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.645+0000 7f1dbfbc9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1db815b690 0x7f1db80b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:18.646 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.645+0000 7f1dbfbc9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1db80b5f30 0x7f1db80cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:18.646 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.645+0000 7f1dbfbc9640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1db8072a70 con 0x7f1db80b65a0 2024-05-08T05:41:18.646 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.645+0000 7f1dbfbc9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1db815ba70 con 0x7f1db815b690 2024-05-08T05:41:18.646 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.645+0000 7f1dbfbc9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1db8072d50 con 0x7f1db80b5f30 2024-05-08T05:41:18.646 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.646+0000 7f1dbd164640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1db80b5f30 0x7f1db80cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:18.646 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.646+0000 7f1dbd965640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1db815b690 0x7f1db80b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:18.646 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.646+0000 7f1dbd965640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1db815b690 0x7f1db80b4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:47022/0 (socket says 172.21.15.45:47022) 2024-05-08T05:41:18.647 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.646+0000 7f1dbe166640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f1db80b65a0 0x7f1db8073830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:36330/0 (socket says 172.21.15.45:36330) 2024-05-08T05:41:18.647 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.646+0000 7f1dbd164640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1db80b5f30 0x7f1db80cee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:47090/0 (socket says 172.21.15.45:47090) 2024-05-08T05:41:18.647 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.646+0000 7f1dbd965640 1 -- 172.21.15.45:0/2354363125 learned_addr learned my addr 172.21.15.45:0/2354363125 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:18.648 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.646+0000 7f1dbd965640 1 -- 172.21.15.45:0/2354363125 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1db80b5f30 msgr2=0x7f1db80cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:18.648 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.646+0000 7f1dbd965640 1 --2- 172.21.15.45:0/2354363125 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1db80b5f30 0x7f1db80cee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:18.648 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.646+0000 7f1dbd965640 1 -- 172.21.15.45:0/2354363125 >> v1:172.21.15.63:6789/0 conn(0x7f1db80b65a0 legacy=0x7f1db8073830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:18.648 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.646+0000 7f1dbd965640 1 -- 172.21.15.45:0/2354363125 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db8073030 con 0x7f1db815b690 2024-05-08T05:41:18.649 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.646+0000 7f1dbd164640 1 --2- 172.21.15.45:0/2354363125 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1db80b5f30 0x7f1db80cee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:18.649 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.647+0000 7f1dbd965640 1 --2- 172.21.15.45:0/2354363125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1db815b690 0x7f1db80b4c90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1da800af60 tx=0x7f1da802b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c1c36cff036a363 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:18.649 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.647+0000 7f1dbc963640 1 -- 172.21.15.45:0/2354363125 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8037070 con 0x7f1db815b690 2024-05-08T05:41:18.649 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.647+0000 7f1dbc963640 1 -- 172.21.15.45:0/2354363125 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da8008500 con 0x7f1db815b690 2024-05-08T05:41:18.649 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.647+0000 7f1dbc963640 1 -- 172.21.15.45:0/2354363125 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da80088d0 con 0x7f1db815b690 2024-05-08T05:41:18.649 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.648+0000 7f1dbfbc9640 1 -- 172.21.15.45:0/2354363125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1db815b690 msgr2=0x7f1db80b4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:18.649 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.648+0000 7f1dbfbc9640 1 --2- 172.21.15.45:0/2354363125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1db815b690 0x7f1db80b4c90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1da800af60 tx=0x7f1da802b7a0 comp rx=0 tx=0).stop 2024-05-08T05:41:18.649 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.648+0000 7f1dbfbc9640 1 -- 172.21.15.45:0/2354363125 shutdown_connections 2024-05-08T05:41:18.649 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.648+0000 7f1dbfbc9640 1 --2- 172.21.15.45:0/2354363125 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1db80b5f30 0x7f1db80cee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:18.649 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.648+0000 7f1dbfbc9640 1 --2- 172.21.15.45:0/2354363125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1db815b690 0x7f1db80b4c90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:18.649 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.648+0000 7f1dbfbc9640 1 -- 172.21.15.45:0/2354363125 >> 172.21.15.45:0/2354363125 conn(0x7f1db806e3e0 msgr2=0x7f1db806e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:18.649 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.648+0000 7f1dbfbc9640 1 -- 172.21.15.45:0/2354363125 shutdown_connections 2024-05-08T05:41:18.649 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.648+0000 7f1dbfbc9640 1 -- 172.21.15.45:0/2354363125 wait complete. 2024-05-08T05:41:18.651 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.649+0000 7f1dbfbc9640 1 Processor -- start 2024-05-08T05:41:18.651 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.649+0000 7f1dbfbc9640 1 -- start start 2024-05-08T05:41:18.651 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.650+0000 7f1dbfbc9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1db80b5f30 0x7f1db8148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:18.653 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.650+0000 7f1dbfbc9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1db80b65a0 0x7f1db8146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:18.653 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.650+0000 7f1dbfbc9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1db815b690 0x7f1db814b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:18.653 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.650+0000 7f1dbfbc9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1db80b54c0 con 0x7f1db815b690 2024-05-08T05:41:18.653 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.650+0000 7f1dbfbc9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1db80b51d0 con 0x7f1db80b65a0 2024-05-08T05:41:18.653 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.650+0000 7f1dbfbc9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1db80cf360 con 0x7f1db80b5f30 2024-05-08T05:41:18.653 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.650+0000 7f1dbe166640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1db815b690 0x7f1db814b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:18.653 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.650+0000 7f1dbd965640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1db80b5f30 0x7f1db8148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:18.653 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.650+0000 7f1dbe166640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1db815b690 0x7f1db814b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35612/0 (socket says 172.21.15.45:35612) 2024-05-08T05:41:18.653 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.650+0000 7f1dbe166640 1 -- 172.21.15.45:0/1945184612 learned_addr learned my addr 172.21.15.45:0/1945184612 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:18.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.650+0000 7f1dbd164640 1 --2- 172.21.15.45:0/1945184612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1db80b65a0 0x7f1db8146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:18.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.650+0000 7f1dbe166640 1 -- 172.21.15.45:0/1945184612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1db80b5f30 msgr2=0x7f1db8148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:18.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.650+0000 7f1dbe166640 1 --2- 172.21.15.45:0/1945184612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1db80b5f30 0x7f1db8148050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:18.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.650+0000 7f1dbe166640 1 -- 172.21.15.45:0/1945184612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1db80b65a0 msgr2=0x7f1db8146f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:18.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.650+0000 7f1dbe166640 1 --2- 172.21.15.45:0/1945184612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1db80b65a0 0x7f1db8146f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:18.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.650+0000 7f1dbe166640 1 -- 172.21.15.45:0/1945184612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db81f4ec0 con 0x7f1db815b690 2024-05-08T05:41:18.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.651+0000 7f1dbe166640 1 --2- 172.21.15.45:0/1945184612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1db815b690 0x7f1db814b5d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1db000c4d0 tx=0x7f1db000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:18.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.651+0000 7f1db67fc640 1 -- 172.21.15.45:0/1945184612 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0018070 con 0x7f1db815b690 2024-05-08T05:41:18.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.651+0000 7f1db67fc640 1 -- 172.21.15.45:0/1945184612 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db0007480 con 0x7f1db815b690 2024-05-08T05:41:18.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.651+0000 7f1db67fc640 1 -- 172.21.15.45:0/1945184612 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db00047e0 con 0x7f1db815b690 2024-05-08T05:41:18.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.651+0000 7f1dbfbc9640 1 -- 172.21.15.45:0/1945184612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db81e5150 con 0x7f1db815b690 2024-05-08T05:41:18.654 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.651+0000 7f1dbfbc9640 1 -- 172.21.15.45:0/1945184612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db81e53f0 con 0x7f1db815b690 2024-05-08T05:41:18.655 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.652+0000 7f1dbfbc9640 1 -- 172.21.15.45:0/1945184612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db80693e0 con 0x7f1db815b690 2024-05-08T05:41:18.655 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.652+0000 7f1db67fc640 1 -- 172.21.15.45:0/1945184612 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) ==== 98145+0+0 (secure 0 0 0) 0x7f1db001b070 con 0x7f1db815b690 2024-05-08T05:41:18.655 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.653+0000 7f1db67fc640 1 --2- 172.21.15.45:0/1945184612 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1da4076240 0x7f1da4078700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:18.655 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.653+0000 7f1db67fc640 1 -- 172.21.15.45:0/1945184612 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 3631+0+0 (secure 0 0 0) 0x7f1db0095e20 con 0x7f1db815b690 2024-05-08T05:41:18.656 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.653+0000 7f1dbd965640 1 --2- 172.21.15.45:0/1945184612 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1da4076240 0x7f1da4078700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:18.656 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.654+0000 7f1dbd965640 1 --2- 172.21.15.45:0/1945184612 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1da4076240 0x7f1da4078700 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1da8034000 tx=0x7f1da800a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:18.658 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.658+0000 7f1db67fc640 1 -- 172.21.15.45:0/1945184612 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1db005fac0 con 0x7f1db815b690 2024-05-08T05:41:18.851 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.851+0000 7f1dbfbc9640 1 -- 172.21.15.45:0/1945184612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1db815eee0 con 0x7f1db815b690 2024-05-08T05:41:18.852 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.851+0000 7f1db67fc640 1 -- 172.21.15.45:0/1945184612 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+9439 (secure 0 0 0) 0x7f1db0068100 con 0x7f1db815b690 2024-05-08T05:41:18.852 INFO:teuthology.misc.health.smithi045.stdout: 2024-05-08T05:41:18.852 INFO:teuthology.misc.health.smithi045.stdout:{"epoch":13,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:41:17.957187+0000","last_up_change":"2024-05-08T05:41:17.957187+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:41:18.857 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.854+0000 7f1dbfbc9640 1 -- 172.21.15.45:0/1945184612 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1da4076240 msgr2=0x7f1da4078700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:18.857 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.854+0000 7f1dbfbc9640 1 --2- 172.21.15.45:0/1945184612 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1da4076240 0x7f1da4078700 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1da8034000 tx=0x7f1da800a000 comp rx=0 tx=0).stop 2024-05-08T05:41:18.857 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.854+0000 7f1dbfbc9640 1 -- 172.21.15.45:0/1945184612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1db815b690 msgr2=0x7f1db814b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:18.857 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.854+0000 7f1dbfbc9640 1 --2- 172.21.15.45:0/1945184612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1db815b690 0x7f1db814b5d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1db000c4d0 tx=0x7f1db000c990 comp rx=0 tx=0).stop 2024-05-08T05:41:18.858 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.854+0000 7f1dbfbc9640 1 -- 172.21.15.45:0/1945184612 shutdown_connections 2024-05-08T05:41:18.858 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.854+0000 7f1dbfbc9640 1 --2- 172.21.15.45:0/1945184612 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1da4076240 0x7f1da4078700 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:18.858 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.854+0000 7f1dbfbc9640 1 --2- 172.21.15.45:0/1945184612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1db815b690 0x7f1db814b5d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:18.858 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.854+0000 7f1dbfbc9640 1 --2- 172.21.15.45:0/1945184612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1db80b65a0 0x7f1db8146f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:18.858 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.854+0000 7f1dbfbc9640 1 --2- 172.21.15.45:0/1945184612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1db80b5f30 0x7f1db8148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:18.858 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.854+0000 7f1dbfbc9640 1 -- 172.21.15.45:0/1945184612 >> 172.21.15.45:0/1945184612 conn(0x7f1db806e3e0 msgr2=0x7f1db8151920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:18.858 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.854+0000 7f1dbfbc9640 1 -- 172.21.15.45:0/1945184612 shutdown_connections 2024-05-08T05:41:18.858 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:18.854+0000 7f1dbfbc9640 1 -- 172.21.15.45:0/1945184612 wait complete. 2024-05-08T05:41:18.864 DEBUG:teuthology.misc:5 of 8 OSDs are up 2024-05-08T05:41:25.670 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T05:41:25.777 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.777+0000 7faa9d3c8640 1 Processor -- start 2024-05-08T05:41:25.777 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.777+0000 7faa9d3c8640 1 -- start start 2024-05-08T05:41:25.779 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.778+0000 7faa9d3c8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa980b7f60 0x7faa980d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:25.779 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.778+0000 7faa9d3c8640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faa980b64b0 0x7faa980d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:25.779 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.778+0000 7faa9d3c8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa980d07c0 con 0x7faa981629a0 2024-05-08T05:41:25.779 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.778+0000 7faa9d3c8640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7faa980d2e10 con 0x7faa980b64b0 2024-05-08T05:41:25.779 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.778+0000 7faa9d3c8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faa980d0aa0 con 0x7faa980b7f60 2024-05-08T05:41:25.779 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.778+0000 7faa8e7fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa980b7f60 0x7faa980d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:25.779 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.778+0000 7faa977fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faa980b64b0 0x7faa980d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:25.779 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.778+0000 7faa8e7fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa980b7f60 0x7faa980d19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:49160/0 (socket says 172.21.15.45:49160) 2024-05-08T05:41:25.779 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.778+0000 7faa96ffd640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7faa981629a0 0x7faa980cfd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:51962/0 (socket says 172.21.15.45:51962) 2024-05-08T05:41:25.779 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.778+0000 7faa96ffd640 1 -- 172.21.15.45:0/4001126294 learned_addr learned my addr 172.21.15.45:0/4001126294 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:25.780 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.778+0000 7faa967fc640 1 -- 172.21.15.45:0/4001126294 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4266958482 0 0) 0x7faa980d07c0 con 0x7faa981629a0 2024-05-08T05:41:25.780 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.778+0000 7faa967fc640 1 -- 172.21.15.45:0/4001126294 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa84003520 con 0x7faa981629a0 2024-05-08T05:41:25.780 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.778+0000 7faa8e7fc640 1 -- 172.21.15.45:0/4001126294 >> v2:172.21.15.63:3300/0 conn(0x7faa980b64b0 msgr2=0x7faa980d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:25.780 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.778+0000 7faa8e7fc640 1 --2- 172.21.15.45:0/4001126294 >> v2:172.21.15.63:3300/0 conn(0x7faa980b64b0 0x7faa980d1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:25.780 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.778+0000 7faa8e7fc640 1 -- 172.21.15.45:0/4001126294 >> v1:172.21.15.63:6789/0 conn(0x7faa981629a0 legacy=0x7faa980cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:25.780 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.778+0000 7faa8e7fc640 1 -- 172.21.15.45:0/4001126294 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa980d0d80 con 0x7faa980b7f60 2024-05-08T05:41:25.780 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.778+0000 7faa977fe640 1 --2- 172.21.15.45:0/4001126294 >> v2:172.21.15.63:3300/0 conn(0x7faa980b64b0 0x7faa980d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:25.781 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.779+0000 7faa8e7fc640 1 --2- 172.21.15.45:0/4001126294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa980b7f60 0x7faa980d19e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faa800097a0 tx=0x7faa8002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bba662cae2a93d5a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:25.781 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.779+0000 7faa967fc640 1 -- 172.21.15.45:0/4001126294 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80037070 con 0x7faa980b7f60 2024-05-08T05:41:25.781 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.779+0000 7faa967fc640 1 -- 172.21.15.45:0/4001126294 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa80032440 con 0x7faa980b7f60 2024-05-08T05:41:25.781 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.779+0000 7faa967fc640 1 -- 172.21.15.45:0/4001126294 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80032810 con 0x7faa980b7f60 2024-05-08T05:41:25.781 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.780+0000 7faa9d3c8640 1 -- 172.21.15.45:0/4001126294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa980b7f60 msgr2=0x7faa980d19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:25.781 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.780+0000 7faa9d3c8640 1 --2- 172.21.15.45:0/4001126294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa980b7f60 0x7faa980d19e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faa800097a0 tx=0x7faa8002a180 comp rx=0 tx=0).stop 2024-05-08T05:41:25.781 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.780+0000 7faa9d3c8640 1 -- 172.21.15.45:0/4001126294 shutdown_connections 2024-05-08T05:41:25.782 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.780+0000 7faa9d3c8640 1 --2- 172.21.15.45:0/4001126294 >> v2:172.21.15.63:3300/0 conn(0x7faa980b64b0 0x7faa980d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:25.782 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.780+0000 7faa9d3c8640 1 --2- 172.21.15.45:0/4001126294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa980b7f60 0x7faa980d19e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:25.782 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.780+0000 7faa9d3c8640 1 -- 172.21.15.45:0/4001126294 >> 172.21.15.45:0/4001126294 conn(0x7faa9809e400 msgr2=0x7faa980d5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:25.782 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.780+0000 7faa9d3c8640 1 -- 172.21.15.45:0/4001126294 shutdown_connections 2024-05-08T05:41:25.782 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.780+0000 7faa9d3c8640 1 -- 172.21.15.45:0/4001126294 wait complete. 2024-05-08T05:41:25.784 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.781+0000 7faa9d3c8640 1 Processor -- start 2024-05-08T05:41:25.784 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.782+0000 7faa9d3c8640 1 -- start start 2024-05-08T05:41:25.784 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.782+0000 7faa9d3c8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa980b64b0 0x7faa98155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:25.784 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.782+0000 7faa9d3c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa980b7f60 0x7faa980c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:25.784 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.782+0000 7faa9d3c8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa981629a0 0x7faa981577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:25.784 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.782+0000 7faa9d3c8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faa98162d80 con 0x7faa980b7f60 2024-05-08T05:41:25.784 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.782+0000 7faa9d3c8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faa980d0430 con 0x7faa981629a0 2024-05-08T05:41:25.784 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.782+0000 7faa9d3c8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faa98162f00 con 0x7faa980b64b0 2024-05-08T05:41:25.784 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.783+0000 7faa8e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa980b7f60 0x7faa980c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:25.785 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.783+0000 7faa8e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa980b7f60 0x7faa980c9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35946/0 (socket says 172.21.15.45:35946) 2024-05-08T05:41:25.785 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.783+0000 7faa8e7fc640 1 -- 172.21.15.45:0/2071969577 learned_addr learned my addr 172.21.15.45:0/2071969577 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:25.785 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.783+0000 7faa96ffd640 1 --2- 172.21.15.45:0/2071969577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa980b64b0 0x7faa98155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:25.785 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.783+0000 7faa977fe640 1 --2- 172.21.15.45:0/2071969577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa981629a0 0x7faa981577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:25.785 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.783+0000 7faa8e7fc640 1 -- 172.21.15.45:0/2071969577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa980b64b0 msgr2=0x7faa98155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:25.785 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.783+0000 7faa8e7fc640 1 --2- 172.21.15.45:0/2071969577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa980b64b0 0x7faa98155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:25.785 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.783+0000 7faa8e7fc640 1 -- 172.21.15.45:0/2071969577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa981629a0 msgr2=0x7faa981577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:25.785 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.783+0000 7faa8e7fc640 1 --2- 172.21.15.45:0/2071969577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa981629a0 0x7faa981577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:25.785 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.783+0000 7faa8e7fc640 1 -- 172.21.15.45:0/2071969577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa98156330 con 0x7faa980b7f60 2024-05-08T05:41:25.785 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.783+0000 7faa8e7fc640 1 --2- 172.21.15.45:0/2071969577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa980b7f60 0x7faa980c9c00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faa80035040 tx=0x7faa800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:25.785 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.783+0000 7faa8ffff640 1 -- 172.21.15.45:0/2071969577 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80041030 con 0x7faa980b7f60 2024-05-08T05:41:25.785 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.783+0000 7faa8ffff640 1 -- 172.21.15.45:0/2071969577 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa8003d420 con 0x7faa980b7f60 2024-05-08T05:41:25.785 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.783+0000 7faa8ffff640 1 -- 172.21.15.45:0/2071969577 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80037040 con 0x7faa980b7f60 2024-05-08T05:41:25.785 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.783+0000 7faa9d3c8640 1 -- 172.21.15.45:0/2071969577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa980cd670 con 0x7faa980b7f60 2024-05-08T05:41:25.786 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.784+0000 7faa9d3c8640 1 -- 172.21.15.45:0/2071969577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa980cdc30 con 0x7faa980b7f60 2024-05-08T05:41:25.786 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.785+0000 7faa9d3c8640 1 -- 172.21.15.45:0/2071969577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa980693e0 con 0x7faa980b7f60 2024-05-08T05:41:25.786 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.785+0000 7faa8ffff640 1 -- 172.21.15.45:0/2071969577 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7faa80044070 con 0x7faa980b7f60 2024-05-08T05:41:25.787 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.786+0000 7faa8ffff640 1 --2- 172.21.15.45:0/2071969577 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faa78076300 0x7faa780787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:25.787 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.786+0000 7faa8ffff640 1 -- 172.21.15.45:0/2071969577 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7faa800b7800 con 0x7faa980b7f60 2024-05-08T05:41:25.787 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.786+0000 7faa96ffd640 1 --2- 172.21.15.45:0/2071969577 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faa78076300 0x7faa780787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:25.787 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.787+0000 7faa96ffd640 1 --2- 172.21.15.45:0/2071969577 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faa78076300 0x7faa780787c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faa88005880 tx=0x7faa88006aa0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:25.791 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.791+0000 7faa8ffff640 1 -- 172.21.15.45:0/2071969577 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7faa800814c0 con 0x7faa980b7f60 2024-05-08T05:41:25.984 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.984+0000 7faa9d3c8640 1 -- 172.21.15.45:0/2071969577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa980d2240 con 0x7faa980b7f60 2024-05-08T05:41:25.985 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.985+0000 7faa8ffff640 1 -- 172.21.15.45:0/2071969577 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12836 (secure 0 0 0) 0x7faa80080dc0 con 0x7faa980b7f60 2024-05-08T05:41:25.985 INFO:teuthology.misc.health.smithi045.stdout: 2024-05-08T05:41:25.985 INFO:teuthology.misc.health.smithi045.stdout:{"epoch":16,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:41:21.236078+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:41:25.987 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.986+0000 7faa9d3c8640 1 -- 172.21.15.45:0/2071969577 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faa78076300 msgr2=0x7faa780787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:25.987 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.986+0000 7faa9d3c8640 1 --2- 172.21.15.45:0/2071969577 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faa78076300 0x7faa780787c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faa88005880 tx=0x7faa88006aa0 comp rx=0 tx=0).stop 2024-05-08T05:41:25.988 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.986+0000 7faa9d3c8640 1 -- 172.21.15.45:0/2071969577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa980b7f60 msgr2=0x7faa980c9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:25.988 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.986+0000 7faa9d3c8640 1 --2- 172.21.15.45:0/2071969577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa980b7f60 0x7faa980c9c00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faa80035040 tx=0x7faa800046c0 comp rx=0 tx=0).stop 2024-05-08T05:41:25.988 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.987+0000 7faa9d3c8640 1 -- 172.21.15.45:0/2071969577 shutdown_connections 2024-05-08T05:41:25.988 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.987+0000 7faa9d3c8640 1 --2- 172.21.15.45:0/2071969577 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faa78076300 0x7faa780787c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:25.988 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.987+0000 7faa9d3c8640 1 --2- 172.21.15.45:0/2071969577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa981629a0 0x7faa981577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:25.988 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.987+0000 7faa9d3c8640 1 --2- 172.21.15.45:0/2071969577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa980b7f60 0x7faa980c9c00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:25.988 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.987+0000 7faa9d3c8640 1 --2- 172.21.15.45:0/2071969577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa980b64b0 0x7faa98155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:25.988 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.987+0000 7faa9d3c8640 1 -- 172.21.15.45:0/2071969577 >> 172.21.15.45:0/2071969577 conn(0x7faa9809e400 msgr2=0x7faa980b8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:25.988 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.987+0000 7faa9d3c8640 1 -- 172.21.15.45:0/2071969577 shutdown_connections 2024-05-08T05:41:25.988 INFO:teuthology.misc.health.smithi045.stderr:2024-05-08T05:41:25.987+0000 7faa9d3c8640 1 -- 172.21.15.45:0/2071969577 wait complete. 2024-05-08T05:41:25.997 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-08T05:41:25.997 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-08T05:41:25.997 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T05:41:25.997 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T05:41:25.999 INFO:tasks.ceph.mds.a:Started 2024-05-08T05:41:26.000 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T05:41:26.000 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T05:41:26.002 INFO:tasks.ceph.mds.c:Started 2024-05-08T05:41:26.002 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T05:41:26.002 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T05:41:26.005 INFO:tasks.ceph.mds.b:Started 2024-05-08T05:41:26.005 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T05:41:26.005 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T05:41:26.007 INFO:tasks.ceph.mds.d:Started 2024-05-08T05:41:26.008 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-08T05:41:26.008 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:41:26.077 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-05-08T05:41:26.078 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-05-08T05:41:26.095 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-08T05:41:26.095 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-08T05:41:26.095 INFO:tasks.ceph.mds.d.smithi063.stderr:2024-05-08T05:41:26.095+0000 7f6a4e57d640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-05-08T05:41:26.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.116+0000 7f2dd7946640 1 Processor -- start 2024-05-08T05:41:26.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.116+0000 7f2dd7946640 1 -- start start 2024-05-08T05:41:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.117+0000 7f2dd7946640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2dd015b7a0 0x7f2dd006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.117+0000 7f2dd7946640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2dd006d8f0 0x7f2dd006fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.117+0000 7f2dd7946640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dd00b4780 con 0x7f2dd006ffd0 2024-05-08T05:41:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.117+0000 7f2dd7946640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f2dd00b5820 con 0x7f2dd015b7a0 2024-05-08T05:41:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.117+0000 7f2dd7946640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2dd00b4a60 con 0x7f2dd006d8f0 2024-05-08T05:41:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.118+0000 7f2dd56e2640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2dd015b7a0 0x7f2dd006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.118+0000 7f2dd56e2640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2dd015b7a0 0x7f2dd006d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35962/0 (socket says 172.21.15.45:35962) 2024-05-08T05:41:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.118+0000 7f2dd56e2640 1 -- 172.21.15.45:0/1553174363 learned_addr learned my addr 172.21.15.45:0/1553174363 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.118+0000 7f2dd4ee1640 1 --2- 172.21.15.45:0/1553174363 >> v2:172.21.15.63:3300/0 conn(0x7f2dd006d8f0 0x7f2dd006fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.118+0000 7f2dd56e2640 1 -- 172.21.15.45:0/1553174363 >> v2:172.21.15.63:3300/0 conn(0x7f2dd006d8f0 msgr2=0x7f2dd006fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.118+0000 7f2dd56e2640 1 --2- 172.21.15.45:0/1553174363 >> v2:172.21.15.63:3300/0 conn(0x7f2dd006d8f0 0x7f2dd006fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.118+0000 7f2dd56e2640 1 -- 172.21.15.45:0/1553174363 >> v1:172.21.15.63:6789/0 conn(0x7f2dd006ffd0 legacy=0x7f2dd006e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.118+0000 7f2dd56e2640 1 -- 172.21.15.45:0/1553174363 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd00b4d40 con 0x7f2dd015b7a0 2024-05-08T05:41:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.118+0000 7f2dd4ee1640 1 --2- 172.21.15.45:0/1553174363 >> v2:172.21.15.63:3300/0 conn(0x7f2dd006d8f0 0x7f2dd006fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.119+0000 7f2dd56e2640 1 --2- 172.21.15.45:0/1553174363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2dd015b7a0 0x7f2dd006d3b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0076820 tx=0x7f2dc002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf72915af7997b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.119+0000 7f2dcffff640 1 -- 172.21.15.45:0/1553174363 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0036070 con 0x7f2dd015b7a0 2024-05-08T05:41:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.119+0000 7f2dcffff640 1 -- 172.21.15.45:0/1553174363 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dc0031600 con 0x7f2dd015b7a0 2024-05-08T05:41:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.119+0000 7f2dcffff640 1 -- 172.21.15.45:0/1553174363 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc00319d0 con 0x7f2dd015b7a0 2024-05-08T05:41:26.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.119+0000 7f2dd7946640 1 -- 172.21.15.45:0/1553174363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2dd015b7a0 msgr2=0x7f2dd006d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:26.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.119+0000 7f2dd7946640 1 --2- 172.21.15.45:0/1553174363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2dd015b7a0 0x7f2dd006d3b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0076820 tx=0x7f2dc002a6a0 comp rx=0 tx=0).stop 2024-05-08T05:41:26.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.119+0000 7f2dd7946640 1 -- 172.21.15.45:0/1553174363 shutdown_connections 2024-05-08T05:41:26.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.119+0000 7f2dd7946640 1 --2- 172.21.15.45:0/1553174363 >> v2:172.21.15.63:3300/0 conn(0x7f2dd006d8f0 0x7f2dd006fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:26.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.119+0000 7f2dd7946640 1 --2- 172.21.15.45:0/1553174363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2dd015b7a0 0x7f2dd006d3b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:26.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.119+0000 7f2dd7946640 1 -- 172.21.15.45:0/1553174363 >> 172.21.15.45:0/1553174363 conn(0x7f2dd0072960 msgr2=0x7f2dd0072d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:26.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.119+0000 7f2dd7946640 1 -- 172.21.15.45:0/1553174363 shutdown_connections 2024-05-08T05:41:26.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.119+0000 7f2dd7946640 1 -- 172.21.15.45:0/1553174363 wait complete. 2024-05-08T05:41:26.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.121+0000 7f2dd7946640 1 Processor -- start 2024-05-08T05:41:26.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.121+0000 7f2dd7946640 1 -- start start 2024-05-08T05:41:26.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.121+0000 7f2dd7946640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2dd006d8f0 0x7f2dd01f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:26.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.121+0000 7f2dd7946640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2dd006ffd0 0x7f2dd00aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.122+0000 7f2dd7946640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2dd015b7a0 0x7f2dd00ab910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.122+0000 7f2dd7946640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd0070400 con 0x7f2dd015b7a0 2024-05-08T05:41:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.122+0000 7f2dd7946640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd00b5300 con 0x7f2dd006d8f0 2024-05-08T05:41:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.122+0000 7f2dd7946640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2dd00cf600 con 0x7f2dd006ffd0 2024-05-08T05:41:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.122+0000 7f2dd56e2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2dd006d8f0 0x7f2dd01f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.122+0000 7f2dd4ee1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2dd006ffd0 0x7f2dd00aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.122+0000 7f2dd56e2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2dd006d8f0 0x7f2dd01f58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:50926/0 (socket says 172.21.15.45:50926) 2024-05-08T05:41:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.122+0000 7f2dd56e2640 1 -- 172.21.15.45:0/1006031356 learned_addr learned my addr 172.21.15.45:0/1006031356 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.122+0000 7f2dd5ee3640 1 --2- 172.21.15.45:0/1006031356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2dd015b7a0 0x7f2dd00ab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.122+0000 7f2dd5ee3640 1 -- 172.21.15.45:0/1006031356 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2dd006ffd0 msgr2=0x7f2dd00aaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.122+0000 7f2dd5ee3640 1 --2- 172.21.15.45:0/1006031356 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2dd006ffd0 0x7f2dd00aaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.122+0000 7f2dd5ee3640 1 -- 172.21.15.45:0/1006031356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2dd006d8f0 msgr2=0x7f2dd01f58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:26.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.122+0000 7f2dd5ee3640 1 --2- 172.21.15.45:0/1006031356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2dd006d8f0 0x7f2dd01f58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:26.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.122+0000 7f2dd5ee3640 1 -- 172.21.15.45:0/1006031356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd01e6ca0 con 0x7f2dd015b7a0 2024-05-08T05:41:26.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.122+0000 7f2dd5ee3640 1 --2- 172.21.15.45:0/1006031356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2dd015b7a0 0x7f2dd00ab910 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800c3c0 tx=0x7f2dc800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:26.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.122+0000 7f2dcdffb640 1 -- 172.21.15.45:0/1006031356 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8017070 con 0x7f2dd015b7a0 2024-05-08T05:41:26.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.122+0000 7f2dcdffb640 1 -- 172.21.15.45:0/1006031356 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dc80052d0 con 0x7f2dd015b7a0 2024-05-08T05:41:26.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.122+0000 7f2dd7946640 1 -- 172.21.15.45:0/1006031356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd01e6f80 con 0x7f2dd015b7a0 2024-05-08T05:41:26.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.122+0000 7f2dcdffb640 1 -- 172.21.15.45:0/1006031356 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc80037f0 con 0x7f2dd015b7a0 2024-05-08T05:41:26.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.123+0000 7f2dd7946640 1 -- 172.21.15.45:0/1006031356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd01e7540 con 0x7f2dd015b7a0 2024-05-08T05:41:26.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.124+0000 7f2dd7946640 1 -- 172.21.15.45:0/1006031356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dd0069b00 con 0x7f2dd015b7a0 2024-05-08T05:41:26.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.124+0000 7f2dcdffb640 1 -- 172.21.15.45:0/1006031356 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2dc801a070 con 0x7f2dd015b7a0 2024-05-08T05:41:26.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.125+0000 7f2dcdffb640 1 --2- 172.21.15.45:0/1006031356 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2dbc0762d0 0x7f2dbc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:26.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.125+0000 7f2dcdffb640 1 -- 172.21.15.45:0/1006031356 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f2dc80964e0 con 0x7f2dd015b7a0 2024-05-08T05:41:26.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.125+0000 7f2dd56e2640 1 --2- 172.21.15.45:0/1006031356 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2dbc0762d0 0x7f2dbc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:26.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.126+0000 7f2dd56e2640 1 --2- 172.21.15.45:0/1006031356 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2dbc0762d0 0x7f2dbc078790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009850 tx=0x7f2dc0009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.129+0000 7f2dcdffb640 1 -- 172.21.15.45:0/1006031356 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2dc805fb30 con 0x7f2dd015b7a0 2024-05-08T05:41:26.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.359+0000 7f2dd7946640 1 -- 172.21.15.45:0/1006031356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2dd0159e40 con 0x7f2dd015b7a0 2024-05-08T05:41:26.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.359+0000 7f2dcdffb640 1 -- 172.21.15.45:0/1006031356 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1389 (secure 0 0 0) 0x7f2dc8068170 con 0x7f2dd015b7a0 2024-05-08T05:41:26.360 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:26.360 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/709809719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":709809719},{"type":"v1","addr":"172.21.15.45:6833","nonce":709809719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-08T05:41:26.360 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 2 2024-05-08T05:41:26.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.362+0000 7f2dd7946640 1 -- 172.21.15.45:0/1006031356 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2dbc0762d0 msgr2=0x7f2dbc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:26.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.362+0000 7f2dd7946640 1 --2- 172.21.15.45:0/1006031356 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2dbc0762d0 0x7f2dbc078790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009850 tx=0x7f2dc0009000 comp rx=0 tx=0).stop 2024-05-08T05:41:26.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.362+0000 7f2dd7946640 1 -- 172.21.15.45:0/1006031356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2dd015b7a0 msgr2=0x7f2dd00ab910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:26.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.362+0000 7f2dd7946640 1 --2- 172.21.15.45:0/1006031356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2dd015b7a0 0x7f2dd00ab910 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800c3c0 tx=0x7f2dc800c880 comp rx=0 tx=0).stop 2024-05-08T05:41:26.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.362+0000 7f2dd7946640 1 -- 172.21.15.45:0/1006031356 shutdown_connections 2024-05-08T05:41:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.362+0000 7f2dd7946640 1 --2- 172.21.15.45:0/1006031356 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2dbc0762d0 0x7f2dbc078790 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.362+0000 7f2dd7946640 1 --2- 172.21.15.45:0/1006031356 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2dd006ffd0 0x7f2dd00aaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.362+0000 7f2dd7946640 1 --2- 172.21.15.45:0/1006031356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2dd015b7a0 0x7f2dd00ab910 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.362+0000 7f2dd7946640 1 --2- 172.21.15.45:0/1006031356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2dd006d8f0 0x7f2dd01f58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.362+0000 7f2dd7946640 1 -- 172.21.15.45:0/1006031356 >> 172.21.15.45:0/1006031356 conn(0x7f2dd0072960 msgr2=0x7f2dd015dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.362+0000 7f2dd7946640 1 -- 172.21.15.45:0/1006031356 shutdown_connections 2024-05-08T05:41:26.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:26.362+0000 7f2dd7946640 1 -- 172.21.15.45:0/1006031356 wait complete. 2024-05-08T05:41:28.374 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:41:28.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.489+0000 7fee139c8640 1 Processor -- start 2024-05-08T05:41:28.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.489+0000 7fee139c8640 1 -- start start 2024-05-08T05:41:28.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.490+0000 7fee139c8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee0c072840 0x7fee0c15c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:28.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.490+0000 7fee139c8640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fee0c06e4d0 0x7fee0c0aec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:28.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.490+0000 7fee139c8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee0c15c920 con 0x7fee0c06e0f0 2024-05-08T05:41:28.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.490+0000 7fee139c8640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fee0c15cc00 con 0x7fee0c06e4d0 2024-05-08T05:41:28.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.490+0000 7fee139c8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fee0c15cee0 con 0x7fee0c072840 2024-05-08T05:41:28.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.490+0000 7fee10f63640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee0c072840 0x7fee0c15c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:28.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.490+0000 7fee11764640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fee0c06e0f0 0x7fee0c0d0f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:51996/0 (socket says 172.21.15.45:51996) 2024-05-08T05:41:28.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.490+0000 7fee11f65640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fee0c06e4d0 0x7fee0c0aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:28.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.490+0000 7fee11764640 1 -- 172.21.15.45:0/4001473463 learned_addr learned my addr 172.21.15.45:0/4001473463 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:28.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.491+0000 7fee03fff640 1 -- 172.21.15.45:0/4001473463 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3003478821 0 0) 0x7fee0c15c920 con 0x7fee0c06e0f0 2024-05-08T05:41:28.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.491+0000 7fee03fff640 1 -- 172.21.15.45:0/4001473463 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedf8003520 con 0x7fee0c06e0f0 2024-05-08T05:41:28.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.491+0000 7fee10f63640 1 -- 172.21.15.45:0/4001473463 >> v2:172.21.15.63:3300/0 conn(0x7fee0c06e4d0 msgr2=0x7fee0c0aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:28.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.491+0000 7fee10f63640 1 --2- 172.21.15.45:0/4001473463 >> v2:172.21.15.63:3300/0 conn(0x7fee0c06e4d0 0x7fee0c0aec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:28.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.491+0000 7fee10f63640 1 -- 172.21.15.45:0/4001473463 >> v1:172.21.15.63:6789/0 conn(0x7fee0c06e0f0 legacy=0x7fee0c0d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:28.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.491+0000 7fee10f63640 1 -- 172.21.15.45:0/4001473463 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0c15d1c0 con 0x7fee0c072840 2024-05-08T05:41:28.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.492+0000 7fee10f63640 1 --2- 172.21.15.45:0/4001473463 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee0c072840 0x7fee0c15c340 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0097a0 tx=0x7fedfc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d921c9cf8e0cf3b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:28.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.492+0000 7fee03fff640 1 -- 172.21.15.45:0/4001473463 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc037070 con 0x7fee0c072840 2024-05-08T05:41:28.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.492+0000 7fee03fff640 1 -- 172.21.15.45:0/4001473463 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedfc032440 con 0x7fee0c072840 2024-05-08T05:41:28.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.492+0000 7fee03fff640 1 -- 172.21.15.45:0/4001473463 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc032810 con 0x7fee0c072840 2024-05-08T05:41:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.492+0000 7fee139c8640 1 -- 172.21.15.45:0/4001473463 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee0c072840 msgr2=0x7fee0c15c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.492+0000 7fee139c8640 1 --2- 172.21.15.45:0/4001473463 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee0c072840 0x7fee0c15c340 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0097a0 tx=0x7fedfc02a180 comp rx=0 tx=0).stop 2024-05-08T05:41:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.492+0000 7fee139c8640 1 -- 172.21.15.45:0/4001473463 shutdown_connections 2024-05-08T05:41:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.492+0000 7fee139c8640 1 --2- 172.21.15.45:0/4001473463 >> v2:172.21.15.63:3300/0 conn(0x7fee0c06e4d0 0x7fee0c0aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.492+0000 7fee139c8640 1 --2- 172.21.15.45:0/4001473463 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee0c072840 0x7fee0c15c340 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.492+0000 7fee139c8640 1 -- 172.21.15.45:0/4001473463 >> 172.21.15.45:0/4001473463 conn(0x7fee0c06c3a0 msgr2=0x7fee0c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.493+0000 7fee139c8640 1 -- 172.21.15.45:0/4001473463 shutdown_connections 2024-05-08T05:41:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.493+0000 7fee139c8640 1 -- 172.21.15.45:0/4001473463 wait complete. 2024-05-08T05:41:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.494+0000 7fee139c8640 1 Processor -- start 2024-05-08T05:41:28.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.494+0000 7fee139c8640 1 -- start start 2024-05-08T05:41:28.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.494+0000 7fee139c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee0c06e0f0 0x7fee0c09cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:28.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee139c8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee0c06e4d0 0x7fee0c1f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:28.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee139c8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee0c072840 0x7fee0c1f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:28.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee139c8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c0ae7d0 con 0x7fee0c06e0f0 2024-05-08T05:41:28.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee139c8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c0ae650 con 0x7fee0c072840 2024-05-08T05:41:28.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee139c8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fee0c1530b0 con 0x7fee0c06e4d0 2024-05-08T05:41:28.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee11764640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee0c06e0f0 0x7fee0c09cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:28.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee11764640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee0c06e0f0 0x7fee0c09cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35974/0 (socket says 172.21.15.45:35974) 2024-05-08T05:41:28.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee11764640 1 -- 172.21.15.45:0/2668396790 learned_addr learned my addr 172.21.15.45:0/2668396790 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee11f65640 1 --2- 172.21.15.45:0/2668396790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee0c072840 0x7fee0c1f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee10f63640 1 --2- 172.21.15.45:0/2668396790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee0c06e4d0 0x7fee0c1f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee11764640 1 -- 172.21.15.45:0/2668396790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee0c06e4d0 msgr2=0x7fee0c1f5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee11764640 1 --2- 172.21.15.45:0/2668396790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee0c06e4d0 0x7fee0c1f5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee11764640 1 -- 172.21.15.45:0/2668396790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee0c072840 msgr2=0x7fee0c1f63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee11764640 1 --2- 172.21.15.45:0/2668396790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee0c072840 0x7fee0c1f63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee11764640 1 -- 172.21.15.45:0/2668396790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0c1580d0 con 0x7fee0c06e0f0 2024-05-08T05:41:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee11f65640 1 --2- 172.21.15.45:0/2668396790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee0c072840 0x7fee0c1f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee11764640 1 --2- 172.21.15.45:0/2668396790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee0c06e0f0 0x7fee0c09cff0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fee0800c530 tx=0x7fee0800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee01ffb640 1 -- 172.21.15.45:0/2668396790 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee08017070 con 0x7fee0c06e0f0 2024-05-08T05:41:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee01ffb640 1 -- 172.21.15.45:0/2668396790 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee08005ce0 con 0x7fee0c06e0f0 2024-05-08T05:41:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee01ffb640 1 -- 172.21.15.45:0/2668396790 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee08007270 con 0x7fee0c06e0f0 2024-05-08T05:41:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.495+0000 7fee139c8640 1 -- 172.21.15.45:0/2668396790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0c1583b0 con 0x7fee0c06e0f0 2024-05-08T05:41:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.496+0000 7fee139c8640 1 -- 172.21.15.45:0/2668396790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee0c158970 con 0x7fee0c06e0f0 2024-05-08T05:41:28.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.497+0000 7fee139c8640 1 -- 172.21.15.45:0/2668396790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee0c0693f0 con 0x7fee0c06e0f0 2024-05-08T05:41:28.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.497+0000 7fee01ffb640 1 -- 172.21.15.45:0/2668396790 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fee0801a070 con 0x7fee0c06e0f0 2024-05-08T05:41:28.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.498+0000 7fee01ffb640 1 --2- 172.21.15.45:0/2668396790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fedf00762d0 0x7fedf0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:28.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.498+0000 7fee01ffb640 1 -- 172.21.15.45:0/2668396790 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fee080965d0 con 0x7fee0c06e0f0 2024-05-08T05:41:28.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.498+0000 7fee10f63640 1 --2- 172.21.15.45:0/2668396790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fedf00762d0 0x7fedf0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:28.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.498+0000 7fee10f63640 1 --2- 172.21.15.45:0/2668396790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fedf00762d0 0x7fedf0078790 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fedfc034000 tx=0x7fedfc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:28.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.502+0000 7fee01ffb640 1 -- 172.21.15.45:0/2668396790 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fee0805fb90 con 0x7fee0c06e0f0 2024-05-08T05:41:28.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.731+0000 7fee139c8640 1 -- 172.21.15.45:0/2668396790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee0c0765b0 con 0x7fee0c06e0f0 2024-05-08T05:41:28.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.731+0000 7fee01ffb640 1 -- 172.21.15.45:0/2668396790 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3861 (secure 0 0 0) 0x7fee08063e90 con 0x7fee0c06e0f0 2024-05-08T05:41:28.732 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:28.732 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2346958825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2346958825},{"type":"v1","addr":"172.21.15.45:6835","nonce":2346958825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/709809719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":709809719},{"type":"v1","addr":"172.21.15.45:6833","nonce":709809719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3208180302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3208180302},{"type":"v1","addr":"172.21.15.63:6836","nonce":3208180302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3653698989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":3653698989},{"type":"v1","addr":"172.21.15.63:6837","nonce":3653698989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-08T05:41:28.732 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 3 2024-05-08T05:41:28.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.733+0000 7fee139c8640 1 -- 172.21.15.45:0/2668396790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fedf00762d0 msgr2=0x7fedf0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:28.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.733+0000 7fee139c8640 1 --2- 172.21.15.45:0/2668396790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fedf00762d0 0x7fedf0078790 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fedfc034000 tx=0x7fedfc02e000 comp rx=0 tx=0).stop 2024-05-08T05:41:28.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.733+0000 7fee139c8640 1 -- 172.21.15.45:0/2668396790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee0c06e0f0 msgr2=0x7fee0c09cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:28.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.733+0000 7fee139c8640 1 --2- 172.21.15.45:0/2668396790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee0c06e0f0 0x7fee0c09cff0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fee0800c530 tx=0x7fee0800c9f0 comp rx=0 tx=0).stop 2024-05-08T05:41:28.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.733+0000 7fee139c8640 1 -- 172.21.15.45:0/2668396790 shutdown_connections 2024-05-08T05:41:28.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.733+0000 7fee139c8640 1 --2- 172.21.15.45:0/2668396790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fedf00762d0 0x7fedf0078790 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:28.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.733+0000 7fee139c8640 1 --2- 172.21.15.45:0/2668396790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee0c06e4d0 0x7fee0c1f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:28.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.733+0000 7fee139c8640 1 --2- 172.21.15.45:0/2668396790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee0c072840 0x7fee0c1f63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:28.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.733+0000 7fee139c8640 1 --2- 172.21.15.45:0/2668396790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee0c06e0f0 0x7fee0c09cff0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:28.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.733+0000 7fee139c8640 1 -- 172.21.15.45:0/2668396790 >> 172.21.15.45:0/2668396790 conn(0x7fee0c06c3a0 msgr2=0x7fee0c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:28.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.733+0000 7fee139c8640 1 -- 172.21.15.45:0/2668396790 shutdown_connections 2024-05-08T05:41:28.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.733+0000 7fee139c8640 1 -- 172.21.15.45:0/2668396790 wait complete. 2024-05-08T05:41:28.745 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T05:41:28.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.849+0000 7fa18ffff640 1 Processor -- start 2024-05-08T05:41:28.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.849+0000 7fa18ffff640 1 -- start start 2024-05-08T05:41:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.850+0000 7fa18ffff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa190162970 0x7fa1900b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.850+0000 7fa18ffff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa1900d1820 0x7fa1900d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.850+0000 7fa18ffff640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1900d3810 con 0x7fa1900d32c0 2024-05-08T05:41:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.850+0000 7fa18ffff640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa1900cff60 con 0x7fa190162970 2024-05-08T05:41:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.850+0000 7fa18ffff640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa1900d3af0 con 0x7fa1900d1820 2024-05-08T05:41:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.850+0000 7fa18effd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa190162970 0x7fa1900b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.850+0000 7fa187fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa1900d32c0 0x7fa1900b6170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45012/0 (socket says 172.21.15.45:45012) 2024-05-08T05:41:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.850+0000 7fa187fff640 1 -- 172.21.15.45:0/3501109949 learned_addr learned my addr 172.21.15.45:0/3501109949 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.850+0000 7fa18effd640 1 -- 172.21.15.45:0/3501109949 >> v2:172.21.15.63:3300/0 conn(0x7fa1900d1820 msgr2=0x7fa1900d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.850+0000 7fa18f7fe640 1 --2- 172.21.15.45:0/3501109949 >> v2:172.21.15.63:3300/0 conn(0x7fa1900d1820 0x7fa1900d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.850+0000 7fa18e7fc640 1 -- 172.21.15.45:0/3501109949 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 644793159 0 0) 0x7fa1900d3810 con 0x7fa1900d32c0 2024-05-08T05:41:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.850+0000 7fa18effd640 1 --2- 172.21.15.45:0/3501109949 >> v2:172.21.15.63:3300/0 conn(0x7fa1900d1820 0x7fa1900d1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.850+0000 7fa18effd640 1 -- 172.21.15.45:0/3501109949 >> v1:172.21.15.45:6789/0 conn(0x7fa1900d32c0 legacy=0x7fa1900b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.850+0000 7fa18effd640 1 -- 172.21.15.45:0/3501109949 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1900d3dd0 con 0x7fa190162970 2024-05-08T05:41:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.851+0000 7fa18effd640 1 --2- 172.21.15.45:0/3501109949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa190162970 0x7fa1900b4730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa19008d7c0 tx=0x7fa18002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bce4671f7ffd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.851+0000 7fa18e7fc640 1 -- 172.21.15.45:0/3501109949 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180037070 con 0x7fa190162970 2024-05-08T05:41:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.851+0000 7fa18e7fc640 1 -- 172.21.15.45:0/3501109949 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa180032440 con 0x7fa190162970 2024-05-08T05:41:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.851+0000 7fa18e7fc640 1 -- 172.21.15.45:0/3501109949 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180032810 con 0x7fa190162970 2024-05-08T05:41:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.852+0000 7fa18ffff640 1 -- 172.21.15.45:0/3501109949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa190162970 msgr2=0x7fa1900b4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.852+0000 7fa18ffff640 1 --2- 172.21.15.45:0/3501109949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa190162970 0x7fa1900b4730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa19008d7c0 tx=0x7fa18002a1a0 comp rx=0 tx=0).stop 2024-05-08T05:41:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.852+0000 7fa18ffff640 1 -- 172.21.15.45:0/3501109949 shutdown_connections 2024-05-08T05:41:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.852+0000 7fa18ffff640 1 --2- 172.21.15.45:0/3501109949 >> v2:172.21.15.63:3300/0 conn(0x7fa1900d1820 0x7fa1900d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.852+0000 7fa18ffff640 1 --2- 172.21.15.45:0/3501109949 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa190162970 0x7fa1900b4730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.852+0000 7fa18ffff640 1 -- 172.21.15.45:0/3501109949 >> 172.21.15.45:0/3501109949 conn(0x7fa19009e1a0 msgr2=0x7fa1900d4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.852+0000 7fa18ffff640 1 -- 172.21.15.45:0/3501109949 shutdown_connections 2024-05-08T05:41:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.852+0000 7fa18ffff640 1 -- 172.21.15.45:0/3501109949 wait complete. 2024-05-08T05:41:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.853+0000 7fa18ffff640 1 Processor -- start 2024-05-08T05:41:28.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.853+0000 7fa18ffff640 1 -- start start 2024-05-08T05:41:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.854+0000 7fa18ffff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1900d1820 0x7fa19014e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.854+0000 7fa18ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1900d32c0 0x7fa19015fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.854+0000 7fa18ffff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa190162970 0x7fa1900cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.854+0000 7fa18ffff640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa1900cfc70 con 0x7fa1900d32c0 2024-05-08T05:41:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.854+0000 7fa18ffff640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1900cfaf0 con 0x7fa190162970 2024-05-08T05:41:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.854+0000 7fa18ffff640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa1900d03f0 con 0x7fa1900d1820 2024-05-08T05:41:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.854+0000 7fa187fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1900d32c0 0x7fa19015fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.854+0000 7fa187fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1900d32c0 0x7fa19015fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35984/0 (socket says 172.21.15.45:35984) 2024-05-08T05:41:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.854+0000 7fa187fff640 1 -- 172.21.15.45:0/3790618090 learned_addr learned my addr 172.21.15.45:0/3790618090 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.854+0000 7fa18f7fe640 1 --2- 172.21.15.45:0/3790618090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa190162970 0x7fa1900cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.854+0000 7fa18effd640 1 --2- 172.21.15.45:0/3790618090 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1900d1820 0x7fa19014e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.854+0000 7fa187fff640 1 -- 172.21.15.45:0/3790618090 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1900d1820 msgr2=0x7fa19014e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.854+0000 7fa187fff640 1 --2- 172.21.15.45:0/3790618090 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1900d1820 0x7fa19014e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.854+0000 7fa187fff640 1 -- 172.21.15.45:0/3790618090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa190162970 msgr2=0x7fa1900cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.854+0000 7fa187fff640 1 --2- 172.21.15.45:0/3790618090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa190162970 0x7fa1900cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.854+0000 7fa187fff640 1 -- 172.21.15.45:0/3790618090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa190160250 con 0x7fa1900d32c0 2024-05-08T05:41:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.854+0000 7fa18effd640 1 --2- 172.21.15.45:0/3790618090 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1900d1820 0x7fa19014e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:28.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.854+0000 7fa18f7fe640 1 --2- 172.21.15.45:0/3790618090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa190162970 0x7fa1900cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:28.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.854+0000 7fa187fff640 1 --2- 172.21.15.45:0/3790618090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1900d32c0 0x7fa19015fac0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa17800c490 tx=0x7fa17800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:28.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.855+0000 7fa1877fe640 1 -- 172.21.15.45:0/3790618090 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa178018070 con 0x7fa1900d32c0 2024-05-08T05:41:28.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.855+0000 7fa1877fe640 1 -- 172.21.15.45:0/3790618090 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa178007500 con 0x7fa1900d32c0 2024-05-08T05:41:28.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.855+0000 7fa18ffff640 1 -- 172.21.15.45:0/3790618090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa190160530 con 0x7fa1900d32c0 2024-05-08T05:41:28.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.855+0000 7fa1877fe640 1 -- 172.21.15.45:0/3790618090 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa178004a20 con 0x7fa1900d32c0 2024-05-08T05:41:28.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.855+0000 7fa18ffff640 1 -- 172.21.15.45:0/3790618090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa190160af0 con 0x7fa1900d32c0 2024-05-08T05:41:28.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.856+0000 7fa18ffff640 1 -- 172.21.15.45:0/3790618090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa190050740 con 0x7fa1900d32c0 2024-05-08T05:41:28.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.856+0000 7fa1877fe640 1 -- 172.21.15.45:0/3790618090 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa17801b070 con 0x7fa1900d32c0 2024-05-08T05:41:28.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.857+0000 7fa1877fe640 1 --2- 172.21.15.45:0/3790618090 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1700762d0 0x7fa170078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:28.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.857+0000 7fa1877fe640 1 -- 172.21.15.45:0/3790618090 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa178096690 con 0x7fa1900d32c0 2024-05-08T05:41:28.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.857+0000 7fa18effd640 1 --2- 172.21.15.45:0/3790618090 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1700762d0 0x7fa170078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:28.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.857+0000 7fa18effd640 1 --2- 172.21.15.45:0/3790618090 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1700762d0 0x7fa170078790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa180034000 tx=0x7fa18002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:28.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:28.861+0000 7fa1877fe640 1 -- 172.21.15.45:0/3790618090 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa1780602a0 con 0x7fa1900d32c0 2024-05-08T05:41:29.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.053+0000 7fa18ffff640 1 -- 172.21.15.45:0/3790618090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa19009e6c0 con 0x7fa1900d32c0 2024-05-08T05:41:29.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.054+0000 7fa1877fe640 1 -- 172.21.15.45:0/3790618090 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7fa17805fba0 con 0x7fa1900d32c0 2024-05-08T05:41:29.054 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:29.054 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-05-08T05:41:29.054 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:41:29.054 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-05-08T05:41:29.055 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-05-08T05:41:29.055 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:41:29.055 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-05-08T05:41:29.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.055+0000 7fa18ffff640 1 -- 172.21.15.45:0/3790618090 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1700762d0 msgr2=0x7fa170078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:29.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.055+0000 7fa18ffff640 1 --2- 172.21.15.45:0/3790618090 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1700762d0 0x7fa170078790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa180034000 tx=0x7fa18002e000 comp rx=0 tx=0).stop 2024-05-08T05:41:29.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.055+0000 7fa18ffff640 1 -- 172.21.15.45:0/3790618090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1900d32c0 msgr2=0x7fa19015fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:29.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.055+0000 7fa18ffff640 1 --2- 172.21.15.45:0/3790618090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1900d32c0 0x7fa19015fac0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa17800c490 tx=0x7fa17800c950 comp rx=0 tx=0).stop 2024-05-08T05:41:29.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.056+0000 7fa18ffff640 1 -- 172.21.15.45:0/3790618090 shutdown_connections 2024-05-08T05:41:29.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.056+0000 7fa18ffff640 1 --2- 172.21.15.45:0/3790618090 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1700762d0 0x7fa170078790 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:29.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.056+0000 7fa18ffff640 1 --2- 172.21.15.45:0/3790618090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa190162970 0x7fa1900cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:29.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.056+0000 7fa18ffff640 1 --2- 172.21.15.45:0/3790618090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1900d32c0 0x7fa19015fac0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:29.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.056+0000 7fa18ffff640 1 --2- 172.21.15.45:0/3790618090 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1900d1820 0x7fa19014e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:29.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.056+0000 7fa18ffff640 1 -- 172.21.15.45:0/3790618090 >> 172.21.15.45:0/3790618090 conn(0x7fa19009e1a0 msgr2=0x7fa19015dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:29.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.056+0000 7fa18ffff640 1 -- 172.21.15.45:0/3790618090 shutdown_connections 2024-05-08T05:41:29.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.056+0000 7fa18ffff640 1 -- 172.21.15.45:0/3790618090 wait complete. 2024-05-08T05:41:29.066 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T05:41:29.067 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T05:41:29.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.173+0000 7f4b5c4c8640 1 Processor -- start 2024-05-08T05:41:29.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.173+0000 7f4b5c4c8640 1 -- start start 2024-05-08T05:41:29.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.174+0000 7f4b5c4c8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4b54162b70 0x7f4b540d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:29.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.174+0000 7f4b5c4c8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b540d3480 0x7f4b540d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:29.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.174+0000 7f4b5c4c8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b54162f50 con 0x7f4b540b4a40 2024-05-08T05:41:29.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.174+0000 7f4b5c4c8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4b540d1f60 con 0x7f4b54162b70 2024-05-08T05:41:29.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.174+0000 7f4b5c4c8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b540d3ee0 con 0x7f4b540d3480 2024-05-08T05:41:29.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.175+0000 7f4b5a264640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4b54162b70 0x7f4b540d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:29.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.175+0000 7f4b5aa65640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4b540b4a40 0x7f4b540b4ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45024/0 (socket says 172.21.15.45:45024) 2024-05-08T05:41:29.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.175+0000 7f4b5aa65640 1 -- 172.21.15.45:0/482591405 learned_addr learned my addr 172.21.15.45:0/482591405 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:29.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.175+0000 7f4b59a63640 1 --2- 172.21.15.45:0/482591405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b540d3480 0x7f4b540d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:29.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.175+0000 7f4b5a264640 1 -- 172.21.15.45:0/482591405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b540d3480 msgr2=0x7f4b540d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:29.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.175+0000 7f4b5a264640 1 --2- 172.21.15.45:0/482591405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b540d3480 0x7f4b540d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:29.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.175+0000 7f4b5a264640 1 -- 172.21.15.45:0/482591405 >> v1:172.21.15.45:6789/0 conn(0x7f4b540b4a40 legacy=0x7f4b540b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:29.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.175+0000 7f4b59262640 1 -- 172.21.15.45:0/482591405 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1523985152 0 0) 0x7f4b4c001490 con 0x7f4b540b4a40 2024-05-08T05:41:29.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.175+0000 7f4b5a264640 1 -- 172.21.15.45:0/482591405 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b540d21b0 con 0x7f4b54162b70 2024-05-08T05:41:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.176+0000 7f4b5a264640 1 --2- 172.21.15.45:0/482591405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b54162b70 0x7f4b540d2f40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4b500097a0 tx=0x7f4b5002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93af23914ffea41 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.176+0000 7f4b59262640 1 -- 172.21.15.45:0/482591405 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50037070 con 0x7f4b54162b70 2024-05-08T05:41:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.176+0000 7f4b59262640 1 -- 172.21.15.45:0/482591405 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b50032440 con 0x7f4b54162b70 2024-05-08T05:41:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.176+0000 7f4b59262640 1 -- 172.21.15.45:0/482591405 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50032810 con 0x7f4b54162b70 2024-05-08T05:41:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.176+0000 7f4b5c4c8640 1 -- 172.21.15.45:0/482591405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b54162b70 msgr2=0x7f4b540d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.176+0000 7f4b5c4c8640 1 --2- 172.21.15.45:0/482591405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b54162b70 0x7f4b540d2f40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4b500097a0 tx=0x7f4b5002a180 comp rx=0 tx=0).stop 2024-05-08T05:41:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.176+0000 7f4b5c4c8640 1 -- 172.21.15.45:0/482591405 shutdown_connections 2024-05-08T05:41:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.176+0000 7f4b5c4c8640 1 --2- 172.21.15.45:0/482591405 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b540d3480 0x7f4b540d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.176+0000 7f4b5c4c8640 1 --2- 172.21.15.45:0/482591405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b54162b70 0x7f4b540d2f40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.176+0000 7f4b5c4c8640 1 -- 172.21.15.45:0/482591405 >> 172.21.15.45:0/482591405 conn(0x7f4b540d5040 msgr2=0x7f4b540d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.176+0000 7f4b5c4c8640 1 -- 172.21.15.45:0/482591405 shutdown_connections 2024-05-08T05:41:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.176+0000 7f4b5c4c8640 1 -- 172.21.15.45:0/482591405 wait complete. 2024-05-08T05:41:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.178+0000 7f4b5c4c8640 1 Processor -- start 2024-05-08T05:41:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.178+0000 7f4b5c4c8640 1 -- start start 2024-05-08T05:41:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.178+0000 7f4b5c4c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b540b4a40 0x7f4b540c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.178+0000 7f4b5c4c8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b540d3480 0x7f4b540ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.178+0000 7f4b5c4c8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b54162b70 0x7f4b5406dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.178+0000 7f4b5c4c8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b540b42f0 con 0x7f4b540b4a40 2024-05-08T05:41:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.178+0000 7f4b5c4c8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b540b4170 con 0x7f4b54162b70 2024-05-08T05:41:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.178+0000 7f4b5c4c8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b540d0c60 con 0x7f4b540d3480 2024-05-08T05:41:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.178+0000 7f4b5a264640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b540b4a40 0x7f4b540c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.178+0000 7f4b5a264640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b540b4a40 0x7f4b540c86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35998/0 (socket says 172.21.15.45:35998) 2024-05-08T05:41:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.178+0000 7f4b5a264640 1 -- 172.21.15.45:0/2664403546 learned_addr learned my addr 172.21.15.45:0/2664403546 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.179+0000 7f4b59a63640 1 --2- 172.21.15.45:0/2664403546 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b540d3480 0x7f4b540ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.179+0000 7f4b5aa65640 1 --2- 172.21.15.45:0/2664403546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b54162b70 0x7f4b5406dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.179+0000 7f4b5a264640 1 -- 172.21.15.45:0/2664403546 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b540d3480 msgr2=0x7f4b540ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.179+0000 7f4b5a264640 1 --2- 172.21.15.45:0/2664403546 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b540d3480 0x7f4b540ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.179+0000 7f4b5a264640 1 -- 172.21.15.45:0/2664403546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b54162b70 msgr2=0x7f4b5406dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.179+0000 7f4b5a264640 1 --2- 172.21.15.45:0/2664403546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b54162b70 0x7f4b5406dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:29.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.179+0000 7f4b5a264640 1 -- 172.21.15.45:0/2664403546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b540ba200 con 0x7f4b540b4a40 2024-05-08T05:41:29.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.179+0000 7f4b59a63640 1 --2- 172.21.15.45:0/2664403546 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b540d3480 0x7f4b540ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:29.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.179+0000 7f4b5a264640 1 --2- 172.21.15.45:0/2664403546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b540b4a40 0x7f4b540c86e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4b500097a0 tx=0x7f4b500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:29.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.179+0000 7f4b4affd640 1 -- 172.21.15.45:0/2664403546 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50037050 con 0x7f4b540b4a40 2024-05-08T05:41:29.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.179+0000 7f4b4affd640 1 -- 172.21.15.45:0/2664403546 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b50038040 con 0x7f4b540b4a40 2024-05-08T05:41:29.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.179+0000 7f4b5c4c8640 1 -- 172.21.15.45:0/2664403546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b540c8cc0 con 0x7f4b540b4a40 2024-05-08T05:41:29.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.179+0000 7f4b4affd640 1 -- 172.21.15.45:0/2664403546 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50004470 con 0x7f4b540b4a40 2024-05-08T05:41:29.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.179+0000 7f4b5c4c8640 1 -- 172.21.15.45:0/2664403546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b540c9280 con 0x7f4b540b4a40 2024-05-08T05:41:29.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.180+0000 7f4b5c4c8640 1 -- 172.21.15.45:0/2664403546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b540693f0 con 0x7f4b540b4a40 2024-05-08T05:41:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.181+0000 7f4b4affd640 1 -- 172.21.15.45:0/2664403546 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4b50030900 con 0x7f4b540b4a40 2024-05-08T05:41:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.181+0000 7f4b4affd640 1 --2- 172.21.15.45:0/2664403546 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b380762d0 0x7f4b38078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.181+0000 7f4b4affd640 1 -- 172.21.15.45:0/2664403546 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f4b500b7dd0 con 0x7f4b540b4a40 2024-05-08T05:41:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.182+0000 7f4b59a63640 1 --2- 172.21.15.45:0/2664403546 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b380762d0 0x7f4b38078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.182+0000 7f4b59a63640 1 --2- 172.21.15.45:0/2664403546 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b380762d0 0x7f4b38078790 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4b54069c90 tx=0x7f4b44008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:29.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.186+0000 7f4b4affd640 1 -- 172.21.15.45:0/2664403546 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4b50081a70 con 0x7f4b540b4a40 2024-05-08T05:41:29.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:29.377+0000 7f4b5c4c8640 1 -- 172.21.15.45:0/2664403546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4b540b1f60 con 0x7f4b540b4a40 2024-05-08T05:41:30.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.096+0000 7f4b4affd640 1 -- 172.21.15.45:0/2664403546 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f4b50081370 con 0x7f4b540b4a40 2024-05-08T05:41:30.096 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-05-08T05:41:30.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.099+0000 7f4b5c4c8640 1 -- 172.21.15.45:0/2664403546 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b380762d0 msgr2=0x7f4b38078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:30.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.099+0000 7f4b5c4c8640 1 --2- 172.21.15.45:0/2664403546 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b380762d0 0x7f4b38078790 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4b54069c90 tx=0x7f4b44008040 comp rx=0 tx=0).stop 2024-05-08T05:41:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.099+0000 7f4b5c4c8640 1 -- 172.21.15.45:0/2664403546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b540b4a40 msgr2=0x7f4b540c86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.099+0000 7f4b5c4c8640 1 --2- 172.21.15.45:0/2664403546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b540b4a40 0x7f4b540c86e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4b500097a0 tx=0x7f4b500046c0 comp rx=0 tx=0).stop 2024-05-08T05:41:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.099+0000 7f4b5c4c8640 1 -- 172.21.15.45:0/2664403546 shutdown_connections 2024-05-08T05:41:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.099+0000 7f4b5c4c8640 1 --2- 172.21.15.45:0/2664403546 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b380762d0 0x7f4b38078790 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.099+0000 7f4b5c4c8640 1 --2- 172.21.15.45:0/2664403546 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b540d3480 0x7f4b540ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.099+0000 7f4b5c4c8640 1 --2- 172.21.15.45:0/2664403546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b54162b70 0x7f4b5406dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.099+0000 7f4b5c4c8640 1 --2- 172.21.15.45:0/2664403546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b540b4a40 0x7f4b540c86e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.099+0000 7f4b5c4c8640 1 -- 172.21.15.45:0/2664403546 >> 172.21.15.45:0/2664403546 conn(0x7f4b540d5040 msgr2=0x7f4b5406e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:30.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.100+0000 7f4b5c4c8640 1 -- 172.21.15.45:0/2664403546 shutdown_connections 2024-05-08T05:41:30.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.100+0000 7f4b5c4c8640 1 -- 172.21.15.45:0/2664403546 wait complete. 2024-05-08T05:41:30.118 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-08T05:41:30.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.233+0000 7f063fddd640 1 Processor -- start 2024-05-08T05:41:30.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.234+0000 7f063fddd640 1 -- start start 2024-05-08T05:41:30.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.234+0000 7f063fddd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f06380d56b0 0x7f063816a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:30.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.234+0000 7f063fddd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06380d0320 con 0x7f06380cfcd0 2024-05-08T05:41:30.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.234+0000 7f063fddd640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06380d0600 con 0x7f063815f4e0 2024-05-08T05:41:30.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.234+0000 7f063fddd640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f06380d41b0 con 0x7f06380d56b0 2024-05-08T05:41:30.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.235+0000 7f063d378640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f06380cfcd0 0x7f06380d4fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45034/0 (socket says 172.21.15.45:45034) 2024-05-08T05:41:30.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.235+0000 7f063d378640 1 -- 172.21.15.45:0/552130886 learned_addr learned my addr 172.21.15.45:0/552130886 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:30.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.235+0000 7f063e37a640 1 --2- 172.21.15.45:0/552130886 >> v2:172.21.15.63:3300/0 conn(0x7f06380d56b0 0x7f063816a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:30.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.235+0000 7f063cb77640 1 -- 172.21.15.45:0/552130886 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3220468105 0 0) 0x7f06380d0320 con 0x7f06380cfcd0 2024-05-08T05:41:30.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.235+0000 7f063cb77640 1 -- 172.21.15.45:0/552130886 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0624003520 con 0x7f06380cfcd0 2024-05-08T05:41:30.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.235+0000 7f063cb77640 1 -- 172.21.15.45:0/552130886 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 432540345 0 0) 0x7f06380d0600 con 0x7f063815f4e0 2024-05-08T05:41:30.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.235+0000 7f063cb77640 1 -- 172.21.15.45:0/552130886 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06380d0320 con 0x7f063815f4e0 2024-05-08T05:41:30.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.235+0000 7f063e37a640 1 -- 172.21.15.45:0/552130886 >> v1:172.21.15.63:6789/0 conn(0x7f063815f4e0 legacy=0x7f06380d3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:30.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.235+0000 7f063e37a640 1 -- 172.21.15.45:0/552130886 >> v1:172.21.15.45:6789/0 conn(0x7f06380cfcd0 legacy=0x7f06380d4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:30.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.235+0000 7f063cb77640 1 -- 172.21.15.45:0/552130886 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 459080129 0 0) 0x7f0624003520 con 0x7f06380cfcd0 2024-05-08T05:41:30.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.235+0000 7f063e37a640 1 -- 172.21.15.45:0/552130886 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06380d08e0 con 0x7f06380d56b0 2024-05-08T05:41:30.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.236+0000 7f063e37a640 1 --2- 172.21.15.45:0/552130886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06380d56b0 0x7f063816a080 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0630009d90 tx=0x7f063002a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4967f1d068cae9aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:30.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.236+0000 7f063cb77640 1 -- 172.21.15.45:0/552130886 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06280016f0 con 0x7f06380d56b0 2024-05-08T05:41:30.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.236+0000 7f063cb77640 1 -- 172.21.15.45:0/552130886 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0630007540 con 0x7f06380d56b0 2024-05-08T05:41:30.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.236+0000 7f063cb77640 1 -- 172.21.15.45:0/552130886 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630007910 con 0x7f06380d56b0 2024-05-08T05:41:30.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.237+0000 7f063fddd640 1 -- 172.21.15.45:0/552130886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06380d56b0 msgr2=0x7f063816a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:30.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.237+0000 7f063fddd640 1 --2- 172.21.15.45:0/552130886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06380d56b0 0x7f063816a080 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0630009d90 tx=0x7f063002a680 comp rx=0 tx=0).stop 2024-05-08T05:41:30.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.237+0000 7f063fddd640 1 -- 172.21.15.45:0/552130886 shutdown_connections 2024-05-08T05:41:30.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.237+0000 7f063fddd640 1 --2- 172.21.15.45:0/552130886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06380d56b0 0x7f063816a080 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:30.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.237+0000 7f063fddd640 1 -- 172.21.15.45:0/552130886 >> 172.21.15.45:0/552130886 conn(0x7f063808eac0 msgr2=0x7f063809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:30.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.237+0000 7f063fddd640 1 -- 172.21.15.45:0/552130886 shutdown_connections 2024-05-08T05:41:30.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.237+0000 7f063fddd640 1 -- 172.21.15.45:0/552130886 wait complete. 2024-05-08T05:41:30.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.238+0000 7f063fddd640 1 Processor -- start 2024-05-08T05:41:30.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.238+0000 7f063fddd640 1 -- start start 2024-05-08T05:41:30.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f063fddd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06380cfcd0 0x7f0638150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:30.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f063fddd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06380d56b0 0x7f06381f8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:30.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f063fddd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f063815f4e0 0x7f063815d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:30.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f063fddd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f063815f8c0 con 0x7f06380cfcd0 2024-05-08T05:41:30.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f063fddd640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f06380d0ea0 con 0x7f063815f4e0 2024-05-08T05:41:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f063fddd640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f06380d23d0 con 0x7f06380d56b0 2024-05-08T05:41:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f063db79640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06380cfcd0 0x7f0638150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f063db79640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06380cfcd0 0x7f0638150360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36014/0 (socket says 172.21.15.45:36014) 2024-05-08T05:41:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f063db79640 1 -- 172.21.15.45:0/546005962 learned_addr learned my addr 172.21.15.45:0/546005962 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f063e37a640 1 --2- 172.21.15.45:0/546005962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f063815f4e0 0x7f063815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f063d378640 1 --2- 172.21.15.45:0/546005962 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06380d56b0 0x7f06381f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f063db79640 1 -- 172.21.15.45:0/546005962 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06380d56b0 msgr2=0x7f06381f8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f063db79640 1 --2- 172.21.15.45:0/546005962 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06380d56b0 0x7f06381f8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f063db79640 1 -- 172.21.15.45:0/546005962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f063815f4e0 msgr2=0x7f063815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f063db79640 1 --2- 172.21.15.45:0/546005962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f063815f4e0 0x7f063815d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f063db79640 1 -- 172.21.15.45:0/546005962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06381f8b30 con 0x7f06380cfcd0 2024-05-08T05:41:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f063db79640 1 --2- 172.21.15.45:0/546005962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06380cfcd0 0x7f0638150360 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f063400c430 tx=0x7f063400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f062e7fc640 1 -- 172.21.15.45:0/546005962 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0634017070 con 0x7f06380cfcd0 2024-05-08T05:41:30.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f062e7fc640 1 -- 172.21.15.45:0/546005962 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06340052f0 con 0x7f06380cfcd0 2024-05-08T05:41:30.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f063fddd640 1 -- 172.21.15.45:0/546005962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06381f8e10 con 0x7f06380cfcd0 2024-05-08T05:41:30.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.239+0000 7f062e7fc640 1 -- 172.21.15.45:0/546005962 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0634003830 con 0x7f06380cfcd0 2024-05-08T05:41:30.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.240+0000 7f063fddd640 1 -- 172.21.15.45:0/546005962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06381f90f0 con 0x7f06380cfcd0 2024-05-08T05:41:30.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.240+0000 7f063fddd640 1 -- 172.21.15.45:0/546005962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06380693f0 con 0x7f06380cfcd0 2024-05-08T05:41:30.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.241+0000 7f062e7fc640 1 -- 172.21.15.45:0/546005962 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f063401a070 con 0x7f06380cfcd0 2024-05-08T05:41:30.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.242+0000 7f062e7fc640 1 --2- 172.21.15.45:0/546005962 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f061c0762d0 0x7f061c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:30.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.242+0000 7f062e7fc640 1 -- 172.21.15.45:0/546005962 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f06340966f0 con 0x7f06380cfcd0 2024-05-08T05:41:30.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.242+0000 7f063d378640 1 --2- 172.21.15.45:0/546005962 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f061c0762d0 0x7f061c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:30.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.243+0000 7f063d378640 1 --2- 172.21.15.45:0/546005962 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f061c0762d0 0x7f061c078790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f06380bb0c0 tx=0x7f06280016a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:30.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.246+0000 7f062e7fc640 1 -- 172.21.15.45:0/546005962 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f063405fbe0 con 0x7f06380cfcd0 2024-05-08T05:41:30.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:30.439+0000 7f063fddd640 1 -- 172.21.15.45:0/546005962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f06380b3790 con 0x7f06380cfcd0 2024-05-08T05:41:31.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.104+0000 7f062e7fc640 1 -- 172.21.15.45:0/546005962 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7f0634068220 con 0x7f06380cfcd0 2024-05-08T05:41:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-05-08T05:41:31.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.109+0000 7f063fddd640 1 -- 172.21.15.45:0/546005962 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f061c0762d0 msgr2=0x7f061c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:31.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.109+0000 7f063fddd640 1 --2- 172.21.15.45:0/546005962 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f061c0762d0 0x7f061c078790 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f06380bb0c0 tx=0x7f06280016a0 comp rx=0 tx=0).stop 2024-05-08T05:41:31.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.109+0000 7f063fddd640 1 -- 172.21.15.45:0/546005962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06380cfcd0 msgr2=0x7f0638150360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:31.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.109+0000 7f063fddd640 1 --2- 172.21.15.45:0/546005962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06380cfcd0 0x7f0638150360 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f063400c430 tx=0x7f063400c8f0 comp rx=0 tx=0).stop 2024-05-08T05:41:31.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.110+0000 7f063fddd640 1 -- 172.21.15.45:0/546005962 shutdown_connections 2024-05-08T05:41:31.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.110+0000 7f063fddd640 1 --2- 172.21.15.45:0/546005962 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f061c0762d0 0x7f061c078790 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:31.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.110+0000 7f063fddd640 1 --2- 172.21.15.45:0/546005962 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06380d56b0 0x7f06381f8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:31.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.110+0000 7f063fddd640 1 --2- 172.21.15.45:0/546005962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f063815f4e0 0x7f063815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:31.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.110+0000 7f063fddd640 1 --2- 172.21.15.45:0/546005962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06380cfcd0 0x7f0638150360 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:31.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.110+0000 7f063fddd640 1 -- 172.21.15.45:0/546005962 >> 172.21.15.45:0/546005962 conn(0x7f063808eac0 msgr2=0x7f06380d3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:31.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.113+0000 7f063fddd640 1 -- 172.21.15.45:0/546005962 shutdown_connections 2024-05-08T05:41:31.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.115+0000 7f063fddd640 1 -- 172.21.15.45:0/546005962 wait complete. 2024-05-08T05:41:31.138 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T05:41:31.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.249+0000 7f29ef623640 1 Processor -- start 2024-05-08T05:41:31.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.249+0000 7f29ef623640 1 -- start start 2024-05-08T05:41:31.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.250+0000 7f29ef623640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29e815f380 0x7f29e80d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:31.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.250+0000 7f29ef623640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29e80b3f30 0x7f29e80d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:31.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.250+0000 7f29ef623640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29e80b1c90 con 0x7f29e80b2420 2024-05-08T05:41:31.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.250+0000 7f29ef623640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f29e80d0a50 con 0x7f29e815f380 2024-05-08T05:41:31.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.250+0000 7f29ef623640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f29e80d0d30 con 0x7f29e80b3f30 2024-05-08T05:41:31.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.250+0000 7f29ed3bf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29e815f380 0x7f29e80d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:31.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.250+0000 7f29ed3bf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29e815f380 0x7f29e80d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33210/0 (socket says 172.21.15.45:33210) 2024-05-08T05:41:31.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.250+0000 7f29ed3bf640 1 -- 172.21.15.45:0/3136751819 learned_addr learned my addr 172.21.15.45:0/3136751819 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:31.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.250+0000 7f29ecbbe640 1 --2- 172.21.15.45:0/3136751819 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29e80b3f30 0x7f29e80d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:31.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.250+0000 7f29ed3bf640 1 -- 172.21.15.45:0/3136751819 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29e80b3f30 msgr2=0x7f29e80d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:31.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.250+0000 7f29ed3bf640 1 --2- 172.21.15.45:0/3136751819 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29e80b3f30 0x7f29e80d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:31.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.250+0000 7f29ed3bf640 1 -- 172.21.15.45:0/3136751819 >> v1:172.21.15.63:6789/0 conn(0x7f29e80b2420 legacy=0x7f29e80b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:31.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.250+0000 7f29ed3bf640 1 -- 172.21.15.45:0/3136751819 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e80b1f00 con 0x7f29e815f380 2024-05-08T05:41:31.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.250+0000 7f29ecbbe640 1 --2- 172.21.15.45:0/3136751819 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29e80b3f30 0x7f29e80d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:31.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.251+0000 7f29ed3bf640 1 --2- 172.21.15.45:0/3136751819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e815f380 0x7f29e80d35c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009e50 tx=0x7f29d802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b34f70d77bbaff6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:31.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.251+0000 7f29e7fff640 1 -- 172.21.15.45:0/3136751819 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8036070 con 0x7f29e815f380 2024-05-08T05:41:31.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.251+0000 7f29e7fff640 1 -- 172.21.15.45:0/3136751819 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29d8031600 con 0x7f29e815f380 2024-05-08T05:41:31.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.251+0000 7f29e7fff640 1 -- 172.21.15.45:0/3136751819 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d80319d0 con 0x7f29e815f380 2024-05-08T05:41:31.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.252+0000 7f29ef623640 1 -- 172.21.15.45:0/3136751819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e815f380 msgr2=0x7f29e80d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:31.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.252+0000 7f29ef623640 1 --2- 172.21.15.45:0/3136751819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e815f380 0x7f29e80d35c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009e50 tx=0x7f29d802a710 comp rx=0 tx=0).stop 2024-05-08T05:41:31.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.252+0000 7f29ef623640 1 -- 172.21.15.45:0/3136751819 shutdown_connections 2024-05-08T05:41:31.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.252+0000 7f29ef623640 1 --2- 172.21.15.45:0/3136751819 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29e80b3f30 0x7f29e80d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:31.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.252+0000 7f29ef623640 1 --2- 172.21.15.45:0/3136751819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e815f380 0x7f29e80d35c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:31.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.252+0000 7f29ef623640 1 -- 172.21.15.45:0/3136751819 >> 172.21.15.45:0/3136751819 conn(0x7f29e80d4fe0 msgr2=0x7f29e80d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:31.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.252+0000 7f29ef623640 1 -- 172.21.15.45:0/3136751819 shutdown_connections 2024-05-08T05:41:31.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.252+0000 7f29ef623640 1 -- 172.21.15.45:0/3136751819 wait complete. 2024-05-08T05:41:31.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.253+0000 7f29ef623640 1 Processor -- start 2024-05-08T05:41:31.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.253+0000 7f29ef623640 1 -- start start 2024-05-08T05:41:31.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.254+0000 7f29ef623640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29e80b2420 0x7f29e80b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:31.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.254+0000 7f29ef623640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e80b3f30 0x7f29e81f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:31.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.254+0000 7f29ef623640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29e815f380 0x7f29e81f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:31.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.254+0000 7f29ef623640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f29e80d06e0 con 0x7f29e80b3f30 2024-05-08T05:41:31.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.254+0000 7f29ef623640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29e80d1010 con 0x7f29e80b2420 2024-05-08T05:41:31.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.254+0000 7f29ef623640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f29e80d2600 con 0x7f29e815f380 2024-05-08T05:41:31.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.254+0000 7f29ecbbe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e80b3f30 0x7f29e81f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:31.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.254+0000 7f29ecbbe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e80b3f30 0x7f29e81f4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33214/0 (socket says 172.21.15.45:33214) 2024-05-08T05:41:31.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.254+0000 7f29ecbbe640 1 -- 172.21.15.45:0/3284990718 learned_addr learned my addr 172.21.15.45:0/3284990718 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:31.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.254+0000 7f29ed3bf640 1 --2- 172.21.15.45:0/3284990718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29e80b2420 0x7f29e80b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:31.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.254+0000 7f29edbc0640 1 --2- 172.21.15.45:0/3284990718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29e815f380 0x7f29e81f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:31.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.254+0000 7f29ecbbe640 1 -- 172.21.15.45:0/3284990718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29e815f380 msgr2=0x7f29e81f5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:31.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.254+0000 7f29ecbbe640 1 --2- 172.21.15.45:0/3284990718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29e815f380 0x7f29e81f5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:31.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.254+0000 7f29ecbbe640 1 -- 172.21.15.45:0/3284990718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29e80b2420 msgr2=0x7f29e80b7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:31.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.254+0000 7f29ecbbe640 1 --2- 172.21.15.45:0/3284990718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29e80b2420 0x7f29e80b7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:31.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.254+0000 7f29ecbbe640 1 -- 172.21.15.45:0/3284990718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e80b85d0 con 0x7f29e80b3f30 2024-05-08T05:41:31.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.254+0000 7f29edbc0640 1 --2- 172.21.15.45:0/3284990718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29e815f380 0x7f29e81f5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:31.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.254+0000 7f29ecbbe640 1 --2- 172.21.15.45:0/3284990718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e80b3f30 0x7f29e81f4b30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00e840 tx=0x7f29dc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:31.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.254+0000 7f29e5ffb640 1 -- 172.21.15.45:0/3284990718 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc00f040 con 0x7f29e80b3f30 2024-05-08T05:41:31.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.254+0000 7f29e5ffb640 1 -- 172.21.15.45:0/3284990718 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29dc004230 con 0x7f29e80b3f30 2024-05-08T05:41:31.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.255+0000 7f29ef623640 1 -- 172.21.15.45:0/3284990718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e80b88b0 con 0x7f29e80b3f30 2024-05-08T05:41:31.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.255+0000 7f29e5ffb640 1 -- 172.21.15.45:0/3284990718 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc00c670 con 0x7f29e80b3f30 2024-05-08T05:41:31.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.255+0000 7f29ef623640 1 -- 172.21.15.45:0/3284990718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e80b8e70 con 0x7f29e80b3f30 2024-05-08T05:41:31.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.256+0000 7f29ef623640 1 -- 172.21.15.45:0/3284990718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e80693f0 con 0x7f29e80b3f30 2024-05-08T05:41:31.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.256+0000 7f29e5ffb640 1 -- 172.21.15.45:0/3284990718 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f29dc00c810 con 0x7f29e80b3f30 2024-05-08T05:41:31.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.257+0000 7f29e5ffb640 1 --2- 172.21.15.45:0/3284990718 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f29d40762d0 0x7f29d4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:31.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.257+0000 7f29e5ffb640 1 -- 172.21.15.45:0/3284990718 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f29dc097190 con 0x7f29e80b3f30 2024-05-08T05:41:31.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.257+0000 7f29ed3bf640 1 --2- 172.21.15.45:0/3284990718 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f29d40762d0 0x7f29d4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:31.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.258+0000 7f29ed3bf640 1 --2- 172.21.15.45:0/3284990718 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f29d40762d0 0x7f29d4078790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f29d8033000 tx=0x7f29d8009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:31.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.261+0000 7f29e5ffb640 1 -- 172.21.15.45:0/3284990718 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f29dc017020 con 0x7f29e80b3f30 2024-05-08T05:41:31.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:31.493+0000 7f29ef623640 1 -- 172.21.15.45:0/3284990718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f29e80763f0 con 0x7f29e80b3f30 2024-05-08T05:41:32.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.106+0000 7f29e5ffb640 1 -- 172.21.15.45:0/3284990718 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f29dc060500 con 0x7f29e80b3f30 2024-05-08T05:41:32.106 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 2 and data pool 3 2024-05-08T05:41:32.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.110+0000 7f29ef623640 1 -- 172.21.15.45:0/3284990718 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f29d40762d0 msgr2=0x7f29d4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:32.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.110+0000 7f29ef623640 1 --2- 172.21.15.45:0/3284990718 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f29d40762d0 0x7f29d4078790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f29d8033000 tx=0x7f29d8009000 comp rx=0 tx=0).stop 2024-05-08T05:41:32.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.110+0000 7f29ef623640 1 -- 172.21.15.45:0/3284990718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e80b3f30 msgr2=0x7f29e81f4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:32.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.110+0000 7f29ef623640 1 --2- 172.21.15.45:0/3284990718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e80b3f30 0x7f29e81f4b30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00e840 tx=0x7f29dc00ed00 comp rx=0 tx=0).stop 2024-05-08T05:41:32.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.111+0000 7f29ef623640 1 -- 172.21.15.45:0/3284990718 shutdown_connections 2024-05-08T05:41:32.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.111+0000 7f29ef623640 1 --2- 172.21.15.45:0/3284990718 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f29d40762d0 0x7f29d4078790 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:32.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.111+0000 7f29ef623640 1 --2- 172.21.15.45:0/3284990718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29e815f380 0x7f29e81f5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:32.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.111+0000 7f29ef623640 1 --2- 172.21.15.45:0/3284990718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29e80b3f30 0x7f29e81f4b30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:32.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.111+0000 7f29ef623640 1 --2- 172.21.15.45:0/3284990718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29e80b2420 0x7f29e80b7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:32.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.111+0000 7f29ef623640 1 -- 172.21.15.45:0/3284990718 >> 172.21.15.45:0/3284990718 conn(0x7f29e80d4fe0 msgr2=0x7f29e806e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:32.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.112+0000 7f29ef623640 1 -- 172.21.15.45:0/3284990718 shutdown_connections 2024-05-08T05:41:32.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.112+0000 7f29ef623640 1 -- 172.21.15.45:0/3284990718 wait complete. 2024-05-08T05:41:32.141 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-08T05:41:32.141 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-08T05:41:32.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.254+0000 7ff9ae828640 1 Processor -- start 2024-05-08T05:41:32.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.254+0000 7ff9ae828640 1 -- start start 2024-05-08T05:41:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.255+0000 7ff9ae828640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9a80cf8f0 0x7ff9a8070500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.255+0000 7ff9ae828640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9a80d2030 con 0x7ff9a80cef40 2024-05-08T05:41:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.255+0000 7ff9ae828640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9a815f8d0 con 0x7ff9a80d3920 2024-05-08T05:41:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.255+0000 7ff9ae828640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a80d22b0 con 0x7ff9a80cf8f0 2024-05-08T05:41:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.255+0000 7ff9acdc5640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff9a80cef40 0x7ff9a8169b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47394/0 (socket says 172.21.15.45:47394) 2024-05-08T05:41:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.255+0000 7ff9acdc5640 1 -- 172.21.15.45:0/3082260782 learned_addr learned my addr 172.21.15.45:0/3082260782 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:32.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a7fff640 1 --2- 172.21.15.45:0/3082260782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9a80cf8f0 0x7ff9a8070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:32.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a6ffd640 1 -- 172.21.15.45:0/3082260782 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 593500815 0 0) 0x7ff9a0001490 con 0x7ff9a80cef40 2024-05-08T05:41:32.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a6ffd640 1 -- 172.21.15.45:0/3082260782 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff990003520 con 0x7ff9a80cef40 2024-05-08T05:41:32.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a6ffd640 1 -- 172.21.15.45:0/3082260782 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 697360587 0 0) 0x7ff990003520 con 0x7ff9a80cef40 2024-05-08T05:41:32.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a6ffd640 1 -- 172.21.15.45:0/3082260782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9a80cf8f0 msgr2=0x7ff9a8070500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:32.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a6ffd640 1 --2- 172.21.15.45:0/3082260782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9a80cf8f0 0x7ff9a8070500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:32.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a6ffd640 1 -- 172.21.15.45:0/3082260782 >> v1:172.21.15.63:6789/0 conn(0x7ff9a80d3920 legacy=0x7ff9a8070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:32.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a6ffd640 1 -- 172.21.15.45:0/3082260782 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a80d2590 con 0x7ff9a80cef40 2024-05-08T05:41:32.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a6ffd640 1 -- 172.21.15.45:0/3082260782 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7ff9a0001c40 con 0x7ff9a80cef40 2024-05-08T05:41:32.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a6ffd640 1 -- 172.21.15.45:0/3082260782 >> v1:172.21.15.45:6789/0 conn(0x7ff9a80cef40 legacy=0x7ff9a8169b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a6ffd640 1 --2- 172.21.15.45:0/3082260782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff990003a30 0x7ff990003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a6ffd640 1 --2- 172.21.15.45:0/3082260782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff990004800 0x7ff990004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a6ffd640 1 --2- 172.21.15.45:0/3082260782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9a80cf8f0 0x7ff990005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a6ffd640 1 -- 172.21.15.45:0/3082260782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a80d22b0 con 0x7ff990003a30 2024-05-08T05:41:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a6ffd640 1 -- 172.21.15.45:0/3082260782 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff99000bb80 con 0x7ff990004800 2024-05-08T05:41:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a6ffd640 1 -- 172.21.15.45:0/3082260782 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff99000bd80 con 0x7ff9a80cf8f0 2024-05-08T05:41:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a7fff640 1 --2- 172.21.15.45:0/3082260782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff990003a30 0x7ff990003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9acdc5640 1 --2- 172.21.15.45:0/3082260782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9a80cf8f0 0x7ff990005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a77fe640 1 --2- 172.21.15.45:0/3082260782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff990004800 0x7ff990004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a7fff640 1 -- 172.21.15.45:0/3082260782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9a80cf8f0 msgr2=0x7ff990005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a7fff640 1 --2- 172.21.15.45:0/3082260782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9a80cf8f0 0x7ff990005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a7fff640 1 -- 172.21.15.45:0/3082260782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff990004800 msgr2=0x7ff990004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a7fff640 1 --2- 172.21.15.45:0/3082260782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff990004800 0x7ff990004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.256+0000 7ff9a7fff640 1 -- 172.21.15.45:0/3082260782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff990003520 con 0x7ff990003a30 2024-05-08T05:41:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.257+0000 7ff9a7fff640 1 --2- 172.21.15.45:0/3082260782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff990003a30 0x7ff990003e50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff998009ac0 tx=0x7ff990005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5b0206905a42d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.257+0000 7ff9a6ffd640 1 -- 172.21.15.45:0/3082260782 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998036070 con 0x7ff990003a30 2024-05-08T05:41:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.257+0000 7ff9a6ffd640 1 -- 172.21.15.45:0/3082260782 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff998031400 con 0x7ff990003a30 2024-05-08T05:41:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.257+0000 7ff9a6ffd640 1 -- 172.21.15.45:0/3082260782 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998009690 con 0x7ff990003a30 2024-05-08T05:41:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.258+0000 7ff9ae828640 1 -- 172.21.15.45:0/3082260782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff990003a30 msgr2=0x7ff990003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.258+0000 7ff9ae828640 1 --2- 172.21.15.45:0/3082260782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff990003a30 0x7ff990003e50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff998009ac0 tx=0x7ff990005830 comp rx=0 tx=0).stop 2024-05-08T05:41:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.258+0000 7ff9acdc5640 1 -- 172.21.15.45:0/3082260782 reap_dead start 2024-05-08T05:41:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.258+0000 7ff9ae828640 1 -- 172.21.15.45:0/3082260782 shutdown_connections 2024-05-08T05:41:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.258+0000 7ff9ae828640 1 -- 172.21.15.45:0/3082260782 >> 172.21.15.45:0/3082260782 conn(0x7ff9a80d2e00 msgr2=0x7ff9a80d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.258+0000 7ff9ae828640 1 -- 172.21.15.45:0/3082260782 shutdown_connections 2024-05-08T05:41:32.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.258+0000 7ff9ae828640 1 -- 172.21.15.45:0/3082260782 wait complete. 2024-05-08T05:41:32.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.259+0000 7ff9ae828640 1 Processor -- start 2024-05-08T05:41:32.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.259+0000 7ff9ae828640 1 -- start start 2024-05-08T05:41:32.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.260+0000 7ff9ae828640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff990003a30 0x7ff9a80cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.260+0000 7ff9ae828640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9a81f51b0 0x7ff9a8157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.260+0000 7ff9ae828640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9a80cc7a0 0x7ff9a80c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.260+0000 7ff9ae828640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a80cda10 con 0x7ff990003a30 2024-05-08T05:41:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.260+0000 7ff9ae828640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a80cd730 con 0x7ff9a81f51b0 2024-05-08T05:41:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.260+0000 7ff9ae828640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a8071610 con 0x7ff9a80cc7a0 2024-05-08T05:41:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.260+0000 7ff9a7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff990003a30 0x7ff9a80cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.260+0000 7ff9a7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff990003a30 0x7ff9a80cb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33224/0 (socket says 172.21.15.45:33224) 2024-05-08T05:41:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.260+0000 7ff9a7fff640 1 -- 172.21.15.45:0/4258433922 learned_addr learned my addr 172.21.15.45:0/4258433922 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.260+0000 7ff9a77fe640 1 --2- 172.21.15.45:0/4258433922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9a81f51b0 0x7ff9a8157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.260+0000 7ff9acdc5640 1 --2- 172.21.15.45:0/4258433922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9a80cc7a0 0x7ff9a80c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.260+0000 7ff9a7fff640 1 -- 172.21.15.45:0/4258433922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9a80cc7a0 msgr2=0x7ff9a80c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.260+0000 7ff9a7fff640 1 --2- 172.21.15.45:0/4258433922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9a80cc7a0 0x7ff9a80c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.260+0000 7ff9a7fff640 1 -- 172.21.15.45:0/4258433922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9a81f51b0 msgr2=0x7ff9a8157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.260+0000 7ff9a7fff640 1 --2- 172.21.15.45:0/4258433922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9a81f51b0 0x7ff9a8157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.260+0000 7ff9a7fff640 1 -- 172.21.15.45:0/4258433922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a81582a0 con 0x7ff990003a30 2024-05-08T05:41:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.260+0000 7ff9a7fff640 1 --2- 172.21.15.45:0/4258433922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff990003a30 0x7ff9a80cb660 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff998009ac0 tx=0x7ff998004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.261+0000 7ff9a4ff9640 1 -- 172.21.15.45:0/4258433922 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998040020 con 0x7ff990003a30 2024-05-08T05:41:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.261+0000 7ff9a4ff9640 1 -- 172.21.15.45:0/4258433922 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff998003bc0 con 0x7ff990003a30 2024-05-08T05:41:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.261+0000 7ff9ae828640 1 -- 172.21.15.45:0/4258433922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a8158580 con 0x7ff990003a30 2024-05-08T05:41:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.261+0000 7ff9a4ff9640 1 -- 172.21.15.45:0/4258433922 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998036040 con 0x7ff990003a30 2024-05-08T05:41:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.261+0000 7ff9ae828640 1 -- 172.21.15.45:0/4258433922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a8158b40 con 0x7ff990003a30 2024-05-08T05:41:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.262+0000 7ff9ae828640 1 -- 172.21.15.45:0/4258433922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a80c6700 con 0x7ff990003a30 2024-05-08T05:41:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.262+0000 7ff9a4ff9640 1 -- 172.21.15.45:0/4258433922 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff998043020 con 0x7ff990003a30 2024-05-08T05:41:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.263+0000 7ff9a4ff9640 1 --2- 172.21.15.45:0/4258433922 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9940762d0 0x7ff994078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.263+0000 7ff9a4ff9640 1 -- 172.21.15.45:0/4258433922 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7ff9980b76d0 con 0x7ff990003a30 2024-05-08T05:41:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.263+0000 7ff9a77fe640 1 --2- 172.21.15.45:0/4258433922 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9940762d0 0x7ff994078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.264+0000 7ff9a77fe640 1 --2- 172.21.15.45:0/4258433922 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9940762d0 0x7ff994078790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff99c007fd0 tx=0x7ff99c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.267+0000 7ff9a4ff9640 1 -- 172.21.15.45:0/4258433922 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff9a80c6700 con 0x7ff990003a30 2024-05-08T05:41:32.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:32.466+0000 7ff9ae828640 1 -- 172.21.15.45:0/4258433922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7ff9a80693f0 con 0x7ff990003a30 2024-05-08T05:41:33.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.121+0000 7ff9a4ff9640 1 -- 172.21.15.45:0/4258433922 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7ff99804a070 con 0x7ff990003a30 2024-05-08T05:41:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.124+0000 7ff9ae828640 1 -- 172.21.15.45:0/4258433922 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9940762d0 msgr2=0x7ff994078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.124+0000 7ff9ae828640 1 --2- 172.21.15.45:0/4258433922 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9940762d0 0x7ff994078790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff99c007fd0 tx=0x7ff99c009040 comp rx=0 tx=0).stop 2024-05-08T05:41:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.124+0000 7ff9ae828640 1 -- 172.21.15.45:0/4258433922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff990003a30 msgr2=0x7ff9a80cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.124+0000 7ff9ae828640 1 --2- 172.21.15.45:0/4258433922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff990003a30 0x7ff9a80cb660 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff998009ac0 tx=0x7ff998004730 comp rx=0 tx=0).stop 2024-05-08T05:41:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.124+0000 7ff9ae828640 1 -- 172.21.15.45:0/4258433922 shutdown_connections 2024-05-08T05:41:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.124+0000 7ff9ae828640 1 --2- 172.21.15.45:0/4258433922 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9940762d0 0x7ff994078790 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.124+0000 7ff9ae828640 1 --2- 172.21.15.45:0/4258433922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9a80cc7a0 0x7ff9a80c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.124+0000 7ff9ae828640 1 --2- 172.21.15.45:0/4258433922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9a81f51b0 0x7ff9a8157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.124+0000 7ff9ae828640 1 --2- 172.21.15.45:0/4258433922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff990003a30 0x7ff9a80cb660 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.124+0000 7ff9ae828640 1 -- 172.21.15.45:0/4258433922 >> 172.21.15.45:0/4258433922 conn(0x7ff9a80d2e00 msgr2=0x7ff9a80c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.125+0000 7ff9ae828640 1 -- 172.21.15.45:0/4258433922 shutdown_connections 2024-05-08T05:41:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.125+0000 7ff9ae828640 1 -- 172.21.15.45:0/4258433922 wait complete. 2024-05-08T05:41:33.138 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T05:41:33.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.256+0000 7f420f1da640 1 Processor -- start 2024-05-08T05:41:33.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.256+0000 7f420f1da640 1 -- start start 2024-05-08T05:41:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.257+0000 7f420f1da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f420815d4c0 0x7f420815d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.257+0000 7f420f1da640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42080b3c20 con 0x7f42080af740 2024-05-08T05:41:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.257+0000 7f420f1da640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f420815dff0 con 0x7f4208157510 2024-05-08T05:41:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.257+0000 7f420f1da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f420815e2d0 con 0x7f420815d4c0 2024-05-08T05:41:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.257+0000 7f420d777640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f42080af740 0x7f42080afbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47406/0 (socket says 172.21.15.45:47406) 2024-05-08T05:41:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.257+0000 7f420d777640 1 -- 172.21.15.45:0/2723430008 learned_addr learned my addr 172.21.15.45:0/2723430008 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.257+0000 7f41fffff640 1 --2- 172.21.15.45:0/2723430008 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f420815d4c0 0x7f420815d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:33.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.257+0000 7f41ff7fe640 1 -- 172.21.15.45:0/2723430008 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 797905267 0 0) 0x7f42080b3c20 con 0x7f42080af740 2024-05-08T05:41:33.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.257+0000 7f41ff7fe640 1 -- 172.21.15.45:0/2723430008 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41f0003520 con 0x7f42080af740 2024-05-08T05:41:33.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.257+0000 7f41ff7fe640 1 -- 172.21.15.45:0/2723430008 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2114026623 0 0) 0x7f420815dff0 con 0x7f4208157510 2024-05-08T05:41:33.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.257+0000 7f41ff7fe640 1 -- 172.21.15.45:0/2723430008 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42080b3c20 con 0x7f4208157510 2024-05-08T05:41:33.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.257+0000 7f41fffff640 1 -- 172.21.15.45:0/2723430008 >> v1:172.21.15.63:6789/0 conn(0x7f4208157510 legacy=0x7f42080b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:33.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.257+0000 7f41fffff640 1 -- 172.21.15.45:0/2723430008 >> v1:172.21.15.45:6789/0 conn(0x7f42080af740 legacy=0x7f42080afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:33.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.257+0000 7f41fffff640 1 -- 172.21.15.45:0/2723430008 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42080b3ee0 con 0x7f420815d4c0 2024-05-08T05:41:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.258+0000 7f41fffff640 1 --2- 172.21.15.45:0/2723430008 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f420815d4c0 0x7f420815d920 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f41f80097d0 tx=0x7f41f802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acd06aab1ad714e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.258+0000 7f41ff7fe640 1 -- 172.21.15.45:0/2723430008 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8037070 con 0x7f420815d4c0 2024-05-08T05:41:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.258+0000 7f41ff7fe640 1 -- 172.21.15.45:0/2723430008 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f8032440 con 0x7f420815d4c0 2024-05-08T05:41:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.258+0000 7f41ff7fe640 1 -- 172.21.15.45:0/2723430008 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8032810 con 0x7f420815d4c0 2024-05-08T05:41:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.259+0000 7f420f1da640 1 -- 172.21.15.45:0/2723430008 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f420815d4c0 msgr2=0x7f420815d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.259+0000 7f420f1da640 1 --2- 172.21.15.45:0/2723430008 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f420815d4c0 0x7f420815d920 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f41f80097d0 tx=0x7f41f802a180 comp rx=0 tx=0).stop 2024-05-08T05:41:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.259+0000 7f420f1da640 1 -- 172.21.15.45:0/2723430008 shutdown_connections 2024-05-08T05:41:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.259+0000 7f420f1da640 1 --2- 172.21.15.45:0/2723430008 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f420815d4c0 0x7f420815d920 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:33.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.259+0000 7f420f1da640 1 -- 172.21.15.45:0/2723430008 >> 172.21.15.45:0/2723430008 conn(0x7f420806c3a0 msgr2=0x7f420806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:33.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.259+0000 7f420f1da640 1 -- 172.21.15.45:0/2723430008 shutdown_connections 2024-05-08T05:41:33.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.259+0000 7f420f1da640 1 -- 172.21.15.45:0/2723430008 wait complete. 2024-05-08T05:41:33.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.260+0000 7f420f1da640 1 Processor -- start 2024-05-08T05:41:33.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.260+0000 7f420f1da640 1 -- start start 2024-05-08T05:41:33.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.261+0000 7f420f1da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42080af740 0x7f4208148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:33.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.261+0000 7f420f1da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4208157510 0x7f420815b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:33.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.261+0000 7f420f1da640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420815d4c0 0x7f420815b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:33.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.261+0000 7f420f1da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f42080b0580 con 0x7f4208157510 2024-05-08T05:41:33.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.261+0000 7f420f1da640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f42081578f0 con 0x7f420815d4c0 2024-05-08T05:41:33.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.261+0000 7f420f1da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f42081e6fd0 con 0x7f42080af740 2024-05-08T05:41:33.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.261+0000 7f41fffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4208157510 0x7f420815b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:33.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.261+0000 7f41fffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4208157510 0x7f420815b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33230/0 (socket says 172.21.15.45:33230) 2024-05-08T05:41:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.261+0000 7f41fffff640 1 -- 172.21.15.45:0/818673742 learned_addr learned my addr 172.21.15.45:0/818673742 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.261+0000 7f420d777640 1 --2- 172.21.15.45:0/818673742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420815d4c0 0x7f420815b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.261+0000 7f420cf76640 1 --2- 172.21.15.45:0/818673742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42080af740 0x7f4208148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.262+0000 7f41fffff640 1 -- 172.21.15.45:0/818673742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42080af740 msgr2=0x7f4208148950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.262+0000 7f41fffff640 1 --2- 172.21.15.45:0/818673742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42080af740 0x7f4208148950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.262+0000 7f41fffff640 1 -- 172.21.15.45:0/818673742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420815d4c0 msgr2=0x7f420815b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.262+0000 7f41fffff640 1 --2- 172.21.15.45:0/818673742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420815d4c0 0x7f420815b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.262+0000 7f41fffff640 1 -- 172.21.15.45:0/818673742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4208073030 con 0x7f4208157510 2024-05-08T05:41:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.262+0000 7f420cf76640 1 --2- 172.21.15.45:0/818673742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42080af740 0x7f4208148950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.262+0000 7f420d777640 1 --2- 172.21.15.45:0/818673742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420815d4c0 0x7f420815b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.262+0000 7f41fffff640 1 --2- 172.21.15.45:0/818673742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4208157510 0x7f420815b050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f41f8035040 tx=0x7f41f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.262+0000 7f41fd7fa640 1 -- 172.21.15.45:0/818673742 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8041020 con 0x7f4208157510 2024-05-08T05:41:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.262+0000 7f41fd7fa640 1 -- 172.21.15.45:0/818673742 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f803d6e0 con 0x7f4208157510 2024-05-08T05:41:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.262+0000 7f420f1da640 1 -- 172.21.15.45:0/818673742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4208073310 con 0x7f4208157510 2024-05-08T05:41:33.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.262+0000 7f41fd7fa640 1 -- 172.21.15.45:0/818673742 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8037040 con 0x7f4208157510 2024-05-08T05:41:33.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.262+0000 7f420f1da640 1 -- 172.21.15.45:0/818673742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42080738d0 con 0x7f4208157510 2024-05-08T05:41:33.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.263+0000 7f420f1da640 1 -- 172.21.15.45:0/818673742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42080693f0 con 0x7f4208157510 2024-05-08T05:41:33.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.264+0000 7f41fd7fa640 1 -- 172.21.15.45:0/818673742 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f41f8044020 con 0x7f4208157510 2024-05-08T05:41:33.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.264+0000 7f41fd7fa640 1 --2- 172.21.15.45:0/818673742 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f41f40762d0 0x7f41f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:33.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.264+0000 7f41fd7fa640 1 -- 172.21.15.45:0/818673742 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7f41f80b7f20 con 0x7f4208157510 2024-05-08T05:41:33.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.265+0000 7f420cf76640 1 --2- 172.21.15.45:0/818673742 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f41f40762d0 0x7f41f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:33.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.265+0000 7f420cf76640 1 --2- 172.21.15.45:0/818673742 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f41f40762d0 0x7f41f4078790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4204001710 tx=0x7f4204009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:33.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.269+0000 7f41fd7fa640 1 -- 172.21.15.45:0/818673742 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f41f804b030 con 0x7f4208157510 2024-05-08T05:41:33.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:33.460+0000 7f420f1da640 1 -- 172.21.15.45:0/818673742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f42081607a0 con 0x7f4208157510 2024-05-08T05:41:33.984 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:41:33.987 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:41:35.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.131+0000 7f41fd7fa640 1 -- 172.21.15.45:0/818673742 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7f41f80816b0 con 0x7f4208157510 2024-05-08T05:41:35.132 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data_ec' created 2024-05-08T05:41:35.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.135+0000 7f420f1da640 1 -- 172.21.15.45:0/818673742 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f41f40762d0 msgr2=0x7f41f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:35.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.135+0000 7f420f1da640 1 --2- 172.21.15.45:0/818673742 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f41f40762d0 0x7f41f4078790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4204001710 tx=0x7f4204009040 comp rx=0 tx=0).stop 2024-05-08T05:41:35.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.135+0000 7f420f1da640 1 -- 172.21.15.45:0/818673742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4208157510 msgr2=0x7f420815b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:35.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.135+0000 7f420f1da640 1 --2- 172.21.15.45:0/818673742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4208157510 0x7f420815b050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f41f8035040 tx=0x7f41f80046c0 comp rx=0 tx=0).stop 2024-05-08T05:41:35.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.135+0000 7f420f1da640 1 -- 172.21.15.45:0/818673742 shutdown_connections 2024-05-08T05:41:35.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.135+0000 7f420f1da640 1 --2- 172.21.15.45:0/818673742 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f41f40762d0 0x7f41f4078790 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4204001710 tx=0x7f4204009040 comp rx=0 tx=0).stop 2024-05-08T05:41:35.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.135+0000 7f420f1da640 1 --2- 172.21.15.45:0/818673742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f420815d4c0 0x7f420815b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:35.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.135+0000 7f420f1da640 1 --2- 172.21.15.45:0/818673742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4208157510 0x7f420815b050 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:35.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.135+0000 7f420f1da640 1 --2- 172.21.15.45:0/818673742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42080af740 0x7f4208148950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:35.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.135+0000 7f420f1da640 1 -- 172.21.15.45:0/818673742 >> 172.21.15.45:0/818673742 conn(0x7f420806c3a0 msgr2=0x7f42080b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:35.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.135+0000 7f420f1da640 1 -- 172.21.15.45:0/818673742 shutdown_connections 2024-05-08T05:41:35.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.135+0000 7f420f1da640 1 -- 172.21.15.45:0/818673742 wait complete. 2024-05-08T05:41:35.155 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-08T05:41:35.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.268+0000 7f62cf6fc640 1 Processor -- start 2024-05-08T05:41:35.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.268+0000 7f62cf6fc640 1 -- start start 2024-05-08T05:41:35.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.269+0000 7f62cf6fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f62c81629f0 0x7f62c80d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:35.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.269+0000 7f62cf6fc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f62c80d22a0 0x7f62c80d1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:35.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.269+0000 7f62cf6fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62c80d1ba0 0x7f62c80ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:35.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.269+0000 7f62cf6fc640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f62c80d1ff0 con 0x7f62c81629f0 2024-05-08T05:41:35.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.269+0000 7f62cf6fc640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f62c80cde80 con 0x7f62c80d22a0 2024-05-08T05:41:35.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.269+0000 7f62cf6fc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f62c80ce120 con 0x7f62c80d1ba0 2024-05-08T05:41:35.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.269+0000 7f62cd498640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f62c81629f0 0x7f62c80d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:35.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.269+0000 7f62cd498640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f62c81629f0 0x7f62c80d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33234/0 (socket says 172.21.15.45:33234) 2024-05-08T05:41:35.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.269+0000 7f62cd498640 1 -- 172.21.15.45:0/125513919 learned_addr learned my addr 172.21.15.45:0/125513919 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:35.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.269+0000 7f62cdc99640 1 --2- 172.21.15.45:0/125513919 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62c80d1ba0 0x7f62c80ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:35.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.269+0000 7f62ccc97640 1 --2- 172.21.15.45:0/125513919 >> v2:172.21.15.63:3300/0 conn(0x7f62c80d22a0 0x7f62c80d1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:35.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.269+0000 7f62cd498640 1 -- 172.21.15.45:0/125513919 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62c80d1ba0 msgr2=0x7f62c80ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:35.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.269+0000 7f62cd498640 1 --2- 172.21.15.45:0/125513919 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62c80d1ba0 0x7f62c80ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:35.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.269+0000 7f62cd498640 1 -- 172.21.15.45:0/125513919 >> v2:172.21.15.63:3300/0 conn(0x7f62c80d22a0 msgr2=0x7f62c80d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:35.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.269+0000 7f62cd498640 1 --2- 172.21.15.45:0/125513919 >> v2:172.21.15.63:3300/0 conn(0x7f62c80d22a0 0x7f62c80d1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:35.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.269+0000 7f62cd498640 1 -- 172.21.15.45:0/125513919 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c80ce400 con 0x7f62c81629f0 2024-05-08T05:41:35.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.270+0000 7f62cd498640 1 --2- 172.21.15.45:0/125513919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62c81629f0 0x7f62c80d1120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f62c8073140 tx=0x7f62c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3fd19546b2203ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:35.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.270+0000 7f62bffff640 1 -- 172.21.15.45:0/125513919 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4037070 con 0x7f62c81629f0 2024-05-08T05:41:35.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.270+0000 7f62bffff640 1 -- 172.21.15.45:0/125513919 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62c4032440 con 0x7f62c81629f0 2024-05-08T05:41:35.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.270+0000 7f62bffff640 1 -- 172.21.15.45:0/125513919 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4032810 con 0x7f62c81629f0 2024-05-08T05:41:35.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.271+0000 7f62cf6fc640 1 -- 172.21.15.45:0/125513919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62c81629f0 msgr2=0x7f62c80d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:35.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.271+0000 7f62cf6fc640 1 --2- 172.21.15.45:0/125513919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62c81629f0 0x7f62c80d1120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f62c8073140 tx=0x7f62c402a1e0 comp rx=0 tx=0).stop 2024-05-08T05:41:35.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.271+0000 7f62cf6fc640 1 -- 172.21.15.45:0/125513919 shutdown_connections 2024-05-08T05:41:35.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.271+0000 7f62cf6fc640 1 --2- 172.21.15.45:0/125513919 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62c80d1ba0 0x7f62c80ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:35.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.271+0000 7f62cf6fc640 1 --2- 172.21.15.45:0/125513919 >> v2:172.21.15.63:3300/0 conn(0x7f62c80d22a0 0x7f62c80d1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:35.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.271+0000 7f62cf6fc640 1 --2- 172.21.15.45:0/125513919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62c81629f0 0x7f62c80d1120 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:35.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.271+0000 7f62cf6fc640 1 -- 172.21.15.45:0/125513919 >> 172.21.15.45:0/125513919 conn(0x7f62c80d2d80 msgr2=0x7f62c80d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:35.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.271+0000 7f62cf6fc640 1 -- 172.21.15.45:0/125513919 shutdown_connections 2024-05-08T05:41:35.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.271+0000 7f62cf6fc640 1 -- 172.21.15.45:0/125513919 wait complete. 2024-05-08T05:41:35.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.272+0000 7f62cf6fc640 1 Processor -- start 2024-05-08T05:41:35.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.272+0000 7f62cf6fc640 1 -- start start 2024-05-08T05:41:35.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.273+0000 7f62cf6fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62c80d1ba0 0x7f62c8150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:35.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.273+0000 7f62cf6fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62c80d22a0 0x7f62c80caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:35.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.273+0000 7f62cf6fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62c81629f0 0x7f62c80cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:35.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.273+0000 7f62cf6fc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f62c80cf420 con 0x7f62c81629f0 2024-05-08T05:41:35.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.273+0000 7f62cf6fc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62c80ce9c0 con 0x7f62c80d22a0 2024-05-08T05:41:35.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.273+0000 7f62cf6fc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f62c80cf5a0 con 0x7f62c80d1ba0 2024-05-08T05:41:35.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.273+0000 7f62cdc99640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62c81629f0 0x7f62c80cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:35.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.273+0000 7f62cdc99640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62c81629f0 0x7f62c80cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33250/0 (socket says 172.21.15.45:33250) 2024-05-08T05:41:35.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.273+0000 7f62cdc99640 1 -- 172.21.15.45:0/469372474 learned_addr learned my addr 172.21.15.45:0/469372474 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:35.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.273+0000 7f62cd498640 1 --2- 172.21.15.45:0/469372474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62c80d1ba0 0x7f62c8150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:35.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.273+0000 7f62ccc97640 1 --2- 172.21.15.45:0/469372474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62c80d22a0 0x7f62c80caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:35.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.273+0000 7f62cdc99640 1 -- 172.21.15.45:0/469372474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62c80d1ba0 msgr2=0x7f62c8150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:35.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.273+0000 7f62cdc99640 1 --2- 172.21.15.45:0/469372474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62c80d1ba0 0x7f62c8150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:35.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.273+0000 7f62cdc99640 1 -- 172.21.15.45:0/469372474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62c80d22a0 msgr2=0x7f62c80caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:35.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.273+0000 7f62cdc99640 1 --2- 172.21.15.45:0/469372474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62c80d22a0 0x7f62c80caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:35.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.273+0000 7f62cdc99640 1 -- 172.21.15.45:0/469372474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c80b2a80 con 0x7f62c81629f0 2024-05-08T05:41:35.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.273+0000 7f62cdc99640 1 --2- 172.21.15.45:0/469372474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62c81629f0 0x7f62c80cb3e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f62c0007e70 tx=0x7f62c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:35.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.274+0000 7f62bdffb640 1 -- 172.21.15.45:0/469372474 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0014620 con 0x7f62c81629f0 2024-05-08T05:41:35.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.274+0000 7f62bdffb640 1 -- 172.21.15.45:0/469372474 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62c0019070 con 0x7f62c81629f0 2024-05-08T05:41:35.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.274+0000 7f62cf6fc640 1 -- 172.21.15.45:0/469372474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c80b2d60 con 0x7f62c81629f0 2024-05-08T05:41:35.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.274+0000 7f62bdffb640 1 -- 172.21.15.45:0/469372474 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0014a20 con 0x7f62c81629f0 2024-05-08T05:41:35.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.274+0000 7f62cf6fc640 1 -- 172.21.15.45:0/469372474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c80b3320 con 0x7f62c81629f0 2024-05-08T05:41:35.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.275+0000 7f62cf6fc640 1 -- 172.21.15.45:0/469372474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62c80693f0 con 0x7f62c81629f0 2024-05-08T05:41:35.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.276+0000 7f62bdffb640 1 -- 172.21.15.45:0/469372474 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f62c0014bc0 con 0x7f62c81629f0 2024-05-08T05:41:35.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.276+0000 7f62bdffb640 1 --2- 172.21.15.45:0/469372474 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f62ac0762d0 0x7f62ac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:35.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.276+0000 7f62bdffb640 1 -- 172.21.15.45:0/469372474 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7f62c0098320 con 0x7f62c81629f0 2024-05-08T05:41:35.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.277+0000 7f62cd498640 1 --2- 172.21.15.45:0/469372474 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f62ac0762d0 0x7f62ac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:35.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.277+0000 7f62cd498640 1 --2- 172.21.15.45:0/469372474 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f62ac0762d0 0x7f62ac078790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f62c4034000 tx=0x7f62c402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:35.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.281+0000 7f62bdffb640 1 -- 172.21.15.45:0/469372474 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f62c0061830 con 0x7f62c81629f0 2024-05-08T05:41:35.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:35.470+0000 7f62cf6fc640 1 -- 172.21.15.45:0/469372474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f62c80d2ad0 con 0x7f62c81629f0 2024-05-08T05:41:36.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.133+0000 7f62bdffb640 1 -- 172.21.15.45:0/469372474 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7f62c0061130 con 0x7f62c81629f0 2024-05-08T05:41:36.133 INFO:teuthology.orchestra.run.smithi045.stderr:set pool 4 allow_ec_overwrites to true 2024-05-08T05:41:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.138+0000 7f62cf6fc640 1 -- 172.21.15.45:0/469372474 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f62ac0762d0 msgr2=0x7f62ac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.138+0000 7f62cf6fc640 1 --2- 172.21.15.45:0/469372474 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f62ac0762d0 0x7f62ac078790 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f62c4034000 tx=0x7f62c402e000 comp rx=0 tx=0).stop 2024-05-08T05:41:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.138+0000 7f62cf6fc640 1 -- 172.21.15.45:0/469372474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62c81629f0 msgr2=0x7f62c80cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:36.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.138+0000 7f62cf6fc640 1 --2- 172.21.15.45:0/469372474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62c81629f0 0x7f62c80cb3e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f62c0007e70 tx=0x7f62c000e4b0 comp rx=0 tx=0).stop 2024-05-08T05:41:36.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.138+0000 7f62cf6fc640 1 -- 172.21.15.45:0/469372474 shutdown_connections 2024-05-08T05:41:36.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.138+0000 7f62cf6fc640 1 --2- 172.21.15.45:0/469372474 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f62ac0762d0 0x7f62ac078790 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:36.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.138+0000 7f62cf6fc640 1 --2- 172.21.15.45:0/469372474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62c81629f0 0x7f62c80cb3e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:36.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.139+0000 7f62cf6fc640 1 --2- 172.21.15.45:0/469372474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62c80d22a0 0x7f62c80caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:36.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.139+0000 7f62cf6fc640 1 --2- 172.21.15.45:0/469372474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62c80d1ba0 0x7f62c8150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:36.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.139+0000 7f62cf6fc640 1 -- 172.21.15.45:0/469372474 >> 172.21.15.45:0/469372474 conn(0x7f62c80d2d80 msgr2=0x7f62c8161ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:36.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.143+0000 7f62cf6fc640 1 -- 172.21.15.45:0/469372474 shutdown_connections 2024-05-08T05:41:36.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.148+0000 7f62cf6fc640 1 -- 172.21.15.45:0/469372474 wait complete. 2024-05-08T05:41:36.170 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-08T05:41:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.280+0000 7f984d779640 1 Processor -- start 2024-05-08T05:41:36.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.280+0000 7f984d779640 1 -- start start 2024-05-08T05:41:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.280+0000 7f984d779640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f98481629a0 0x7f98480d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.280+0000 7f984d779640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98480d2dc0 0x7f98480d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.280+0000 7f984d779640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98480c8ed0 0x7f98480c9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.280+0000 7f984d779640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f98480d3330 con 0x7f98480c8ed0 2024-05-08T05:41:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.280+0000 7f984d779640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f98480d2170 con 0x7f98481629a0 2024-05-08T05:41:36.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.280+0000 7f984d779640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f98480d2450 con 0x7f98480d2dc0 2024-05-08T05:41:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.281+0000 7f98477fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98480c8ed0 0x7f98480c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.281+0000 7f98477fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98480c8ed0 0x7f98480c9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33260/0 (socket says 172.21.15.45:33260) 2024-05-08T05:41:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.281+0000 7f98477fe640 1 -- 172.21.15.45:0/4187132987 learned_addr learned my addr 172.21.15.45:0/4187132987 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.281+0000 7f9846ffd640 1 --2- 172.21.15.45:0/4187132987 >> v2:172.21.15.63:3300/0 conn(0x7f98481629a0 0x7f98480d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.281+0000 7f98467fc640 1 --2- 172.21.15.45:0/4187132987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98480d2dc0 0x7f98480d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.281+0000 7f98477fe640 1 -- 172.21.15.45:0/4187132987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98480d2dc0 msgr2=0x7f98480d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.281+0000 7f98477fe640 1 --2- 172.21.15.45:0/4187132987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98480d2dc0 0x7f98480d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.281+0000 7f98477fe640 1 -- 172.21.15.45:0/4187132987 >> v2:172.21.15.63:3300/0 conn(0x7f98481629a0 msgr2=0x7f98480d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.281+0000 7f98477fe640 1 --2- 172.21.15.45:0/4187132987 >> v2:172.21.15.63:3300/0 conn(0x7f98481629a0 0x7f98480d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.281+0000 7f98477fe640 1 -- 172.21.15.45:0/4187132987 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98480d2730 con 0x7f98480c8ed0 2024-05-08T05:41:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.281+0000 7f98467fc640 1 --2- 172.21.15.45:0/4187132987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98480d2dc0 0x7f98480d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.282+0000 7f98477fe640 1 --2- 172.21.15.45:0/4187132987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98480c8ed0 0x7f98480c9360 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f98480768a0 tx=0x7f984002a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb48177106d866f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:36.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.282+0000 7f9845ffb640 1 -- 172.21.15.45:0/4187132987 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840036070 con 0x7f98480c8ed0 2024-05-08T05:41:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.282+0000 7f9845ffb640 1 -- 172.21.15.45:0/4187132987 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9840031600 con 0x7f98480c8ed0 2024-05-08T05:41:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.282+0000 7f9845ffb640 1 -- 172.21.15.45:0/4187132987 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98400319d0 con 0x7f98480c8ed0 2024-05-08T05:41:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.282+0000 7f984d779640 1 -- 172.21.15.45:0/4187132987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98480c8ed0 msgr2=0x7f98480c9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.282+0000 7f984d779640 1 --2- 172.21.15.45:0/4187132987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98480c8ed0 0x7f98480c9360 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f98480768a0 tx=0x7f984002a570 comp rx=0 tx=0).stop 2024-05-08T05:41:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.282+0000 7f984d779640 1 -- 172.21.15.45:0/4187132987 shutdown_connections 2024-05-08T05:41:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.282+0000 7f984d779640 1 --2- 172.21.15.45:0/4187132987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98480c8ed0 0x7f98480c9360 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.282+0000 7f984d779640 1 --2- 172.21.15.45:0/4187132987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98480d2dc0 0x7f98480d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.282+0000 7f984d779640 1 --2- 172.21.15.45:0/4187132987 >> v2:172.21.15.63:3300/0 conn(0x7f98481629a0 0x7f98480d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.282+0000 7f984d779640 1 -- 172.21.15.45:0/4187132987 >> 172.21.15.45:0/4187132987 conn(0x7f98480d4fe0 msgr2=0x7f98480d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.282+0000 7f984d779640 1 -- 172.21.15.45:0/4187132987 shutdown_connections 2024-05-08T05:41:36.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.283+0000 7f984d779640 1 -- 172.21.15.45:0/4187132987 wait complete. 2024-05-08T05:41:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.284+0000 7f984d779640 1 Processor -- start 2024-05-08T05:41:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.284+0000 7f984d779640 1 -- start start 2024-05-08T05:41:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f984d779640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98480c8ed0 0x7f98480cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f984d779640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98480d2dc0 0x7f98481f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f984d779640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98481629a0 0x7f9848157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f984d779640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f98480cf440 con 0x7f98480d2dc0 2024-05-08T05:41:36.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f984d779640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f98480cf2c0 con 0x7f98481629a0 2024-05-08T05:41:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f984d779640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f98480c98a0 con 0x7f98480c8ed0 2024-05-08T05:41:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f98467fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98480d2dc0 0x7f98481f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f98467fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98480d2dc0 0x7f98481f5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33264/0 (socket says 172.21.15.45:33264) 2024-05-08T05:41:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f98467fc640 1 -- 172.21.15.45:0/819280133 learned_addr learned my addr 172.21.15.45:0/819280133 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f9846ffd640 1 --2- 172.21.15.45:0/819280133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98480c8ed0 0x7f98480cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f98477fe640 1 --2- 172.21.15.45:0/819280133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98481629a0 0x7f9848157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f98467fc640 1 -- 172.21.15.45:0/819280133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98480c8ed0 msgr2=0x7f98480cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f98467fc640 1 --2- 172.21.15.45:0/819280133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98480c8ed0 0x7f98480cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f98467fc640 1 -- 172.21.15.45:0/819280133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98481629a0 msgr2=0x7f9848157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f98467fc640 1 --2- 172.21.15.45:0/819280133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98481629a0 0x7f9848157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f98467fc640 1 -- 172.21.15.45:0/819280133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98480cc670 con 0x7f98480d2dc0 2024-05-08T05:41:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f98467fc640 1 --2- 172.21.15.45:0/819280133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98480d2dc0 0x7f98481f5a50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f983000e810 tx=0x7f983000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f983f7fe640 1 -- 172.21.15.45:0/819280133 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983000f040 con 0x7f98480d2dc0 2024-05-08T05:41:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f983f7fe640 1 -- 172.21.15.45:0/819280133 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9830004030 con 0x7f98480d2dc0 2024-05-08T05:41:36.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f984d779640 1 -- 172.21.15.45:0/819280133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98480cc950 con 0x7f98480d2dc0 2024-05-08T05:41:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f983f7fe640 1 -- 172.21.15.45:0/819280133 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830004400 con 0x7f98480d2dc0 2024-05-08T05:41:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.285+0000 7f984d779640 1 -- 172.21.15.45:0/819280133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98480ccf10 con 0x7f98480d2dc0 2024-05-08T05:41:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.287+0000 7f984d779640 1 -- 172.21.15.45:0/819280133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98480693f0 con 0x7f98480d2dc0 2024-05-08T05:41:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.287+0000 7f983f7fe640 1 -- 172.21.15.45:0/819280133 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f983000c6d0 con 0x7f98480d2dc0 2024-05-08T05:41:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.288+0000 7f983f7fe640 1 --2- 172.21.15.45:0/819280133 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f98280762d0 0x7f9828078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.288+0000 7f983f7fe640 1 -- 172.21.15.45:0/819280133 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f9830097690 con 0x7f98480d2dc0 2024-05-08T05:41:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.288+0000 7f9846ffd640 1 --2- 172.21.15.45:0/819280133 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f98280762d0 0x7f9828078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:36.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.288+0000 7f9846ffd640 1 --2- 172.21.15.45:0/819280133 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f98280762d0 0x7f9828078790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9838009aa0 tx=0x7f9838009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:36.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.292+0000 7f983f7fe640 1 -- 172.21.15.45:0/819280133 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9830017020 con 0x7f98480d2dc0 2024-05-08T05:41:36.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:36.530+0000 7f984d779640 1 -- 172.21.15.45:0/819280133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f98480cb4e0 con 0x7f98480d2dc0 2024-05-08T05:41:37.092 INFO:tasks.ceph.mds.d.smithi063.stderr:2024-05-08T05:41:37.092+0000 7f6a48571640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:41:37.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.144+0000 7f983f7fe640 1 -- 172.21.15.45:0/819280133 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) ==== 137+0+0 (secure 0 0 0) 0x7f9830060560 con 0x7f98480d2dc0 2024-05-08T05:41:37.144 INFO:teuthology.orchestra.run.smithi045.stderr:added data pool 4 to fsmap 2024-05-08T05:41:37.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.147+0000 7f984d779640 1 -- 172.21.15.45:0/819280133 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f98280762d0 msgr2=0x7f9828078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.147+0000 7f984d779640 1 --2- 172.21.15.45:0/819280133 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f98280762d0 0x7f9828078790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9838009aa0 tx=0x7f9838009190 comp rx=0 tx=0).stop 2024-05-08T05:41:37.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.147+0000 7f984d779640 1 -- 172.21.15.45:0/819280133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98480d2dc0 msgr2=0x7f98481f5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.147+0000 7f984d779640 1 --2- 172.21.15.45:0/819280133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98480d2dc0 0x7f98481f5a50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f983000e810 tx=0x7f983000ecd0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.147+0000 7f984d779640 1 -- 172.21.15.45:0/819280133 shutdown_connections 2024-05-08T05:41:37.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.147+0000 7f984d779640 1 --2- 172.21.15.45:0/819280133 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f98280762d0 0x7f9828078790 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.147+0000 7f984d779640 1 --2- 172.21.15.45:0/819280133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98481629a0 0x7f9848157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.147+0000 7f984d779640 1 --2- 172.21.15.45:0/819280133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98480d2dc0 0x7f98481f5a50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.147+0000 7f984d779640 1 --2- 172.21.15.45:0/819280133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98480c8ed0 0x7f98480cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.147+0000 7f984d779640 1 -- 172.21.15.45:0/819280133 >> 172.21.15.45:0/819280133 conn(0x7f98480d4fe0 msgr2=0x7f98480d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:37.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.148+0000 7f984d779640 1 -- 172.21.15.45:0/819280133 shutdown_connections 2024-05-08T05:41:37.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.148+0000 7f984d779640 1 -- 172.21.15.45:0/819280133 wait complete. 2024-05-08T05:41:37.161 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:41:37.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.271+0000 7f1ce7fa4640 1 Processor -- start 2024-05-08T05:41:37.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.271+0000 7f1ce7fa4640 1 -- start start 2024-05-08T05:41:37.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.272+0000 7f1ce7fa4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1ce00d1970 0x7f1ce00d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.272+0000 7f1ce7fa4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ce01630b0 con 0x7f1ce015f420 2024-05-08T05:41:37.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.272+0000 7f1ce7fa4640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ce00d3e70 con 0x7f1ce00d4cc0 2024-05-08T05:41:37.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.272+0000 7f1ce7fa4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1ce00d2550 con 0x7f1ce00d1970 2024-05-08T05:41:37.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.272+0000 7f1ce5d40640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1ce015f420 0x7f1ce00d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47420/0 (socket says 172.21.15.45:47420) 2024-05-08T05:41:37.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.272+0000 7f1ce5d40640 1 -- 172.21.15.45:0/3857475122 learned_addr learned my addr 172.21.15.45:0/3857475122 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:37.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.272+0000 7f1ce553f640 1 --2- 172.21.15.45:0/3857475122 >> v2:172.21.15.63:3300/0 conn(0x7f1ce00d1970 0x7f1ce00d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.272+0000 7f1ce4d3e640 1 -- 172.21.15.45:0/3857475122 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 52573603 0 0) 0x7f1cdc001110 con 0x7f1ce015f420 2024-05-08T05:41:37.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.272+0000 7f1ce4d3e640 1 -- 172.21.15.45:0/3857475122 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ccc003520 con 0x7f1ce015f420 2024-05-08T05:41:37.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.273+0000 7f1ce4d3e640 1 -- 172.21.15.45:0/3857475122 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 448283872 0 0) 0x7f1ccc003520 con 0x7f1ce015f420 2024-05-08T05:41:37.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.273+0000 7f1ce553f640 1 -- 172.21.15.45:0/3857475122 >> v1:172.21.15.63:6789/0 conn(0x7f1ce00d4cc0 legacy=0x7f1ce00d50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.273+0000 7f1ce553f640 1 -- 172.21.15.45:0/3857475122 >> v1:172.21.15.45:6789/0 conn(0x7f1ce015f420 legacy=0x7f1ce00d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.273+0000 7f1ce553f640 1 -- 172.21.15.45:0/3857475122 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce00d2c80 con 0x7f1ce00d1970 2024-05-08T05:41:37.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.273+0000 7f1ce553f640 1 --2- 172.21.15.45:0/3857475122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ce00d1970 0x7f1ce00d1e20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00097a0 tx=0x7f1cd002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3487516fedca938e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:37.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.274+0000 7f1ce4d3e640 1 -- 172.21.15.45:0/3857475122 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0037070 con 0x7f1ce00d1970 2024-05-08T05:41:37.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.274+0000 7f1ce4d3e640 1 -- 172.21.15.45:0/3857475122 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cd002ac50 con 0x7f1ce00d1970 2024-05-08T05:41:37.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.274+0000 7f1ce4d3e640 1 -- 172.21.15.45:0/3857475122 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc0018c0 con 0x7f1ce00d1970 2024-05-08T05:41:37.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.274+0000 7f1ce7fa4640 1 -- 172.21.15.45:0/3857475122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ce00d1970 msgr2=0x7f1ce00d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.274+0000 7f1ce7fa4640 1 --2- 172.21.15.45:0/3857475122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ce00d1970 0x7f1ce00d1e20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00097a0 tx=0x7f1cd002a1c0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.274+0000 7f1ce7fa4640 1 -- 172.21.15.45:0/3857475122 shutdown_connections 2024-05-08T05:41:37.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.274+0000 7f1ce7fa4640 1 --2- 172.21.15.45:0/3857475122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ce00d1970 0x7f1ce00d1e20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.274+0000 7f1ce7fa4640 1 -- 172.21.15.45:0/3857475122 >> 172.21.15.45:0/3857475122 conn(0x7f1ce008eac0 msgr2=0x7f1ce009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:37.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.274+0000 7f1ce7fa4640 1 -- 172.21.15.45:0/3857475122 shutdown_connections 2024-05-08T05:41:37.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.274+0000 7f1ce7fa4640 1 -- 172.21.15.45:0/3857475122 wait complete. 2024-05-08T05:41:37.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.275+0000 7f1ce7fa4640 1 Processor -- start 2024-05-08T05:41:37.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.275+0000 7f1ce7fa4640 1 -- start start 2024-05-08T05:41:37.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.276+0000 7f1ce7fa4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ce00d1970 0x7f1ce015d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.276+0000 7f1ce7fa4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ce00d4cc0 0x7f1ce00cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.276+0000 7f1ce7fa4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ce015e010 0x7f1ce00cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.276+0000 7f1ce7fa4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce00d2830 con 0x7f1ce00d4cc0 2024-05-08T05:41:37.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.276+0000 7f1ce7fa4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce00d3240 con 0x7f1ce015e010 2024-05-08T05:41:37.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.276+0000 7f1ce7fa4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce00cfb20 con 0x7f1ce00d1970 2024-05-08T05:41:37.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.276+0000 7f1ce553f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ce00d4cc0 0x7f1ce00cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.276+0000 7f1ce553f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ce00d4cc0 0x7f1ce00cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33278/0 (socket says 172.21.15.45:33278) 2024-05-08T05:41:37.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.276+0000 7f1ce553f640 1 -- 172.21.15.45:0/705340912 learned_addr learned my addr 172.21.15.45:0/705340912 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:37.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.276+0000 7f1ce6541640 1 --2- 172.21.15.45:0/705340912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ce015e010 0x7f1ce00cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.276+0000 7f1ce5d40640 1 --2- 172.21.15.45:0/705340912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ce00d1970 0x7f1ce015d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.277+0000 7f1ce553f640 1 -- 172.21.15.45:0/705340912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ce00d1970 msgr2=0x7f1ce015d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.277+0000 7f1ce553f640 1 --2- 172.21.15.45:0/705340912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ce00d1970 0x7f1ce015d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.277+0000 7f1ce553f640 1 -- 172.21.15.45:0/705340912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ce015e010 msgr2=0x7f1ce00cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.277+0000 7f1ce553f640 1 --2- 172.21.15.45:0/705340912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ce015e010 0x7f1ce00cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.277+0000 7f1ce553f640 1 -- 172.21.15.45:0/705340912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce00b9f30 con 0x7f1ce00d4cc0 2024-05-08T05:41:37.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.277+0000 7f1ce6541640 1 --2- 172.21.15.45:0/705340912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ce015e010 0x7f1ce00cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:37.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.277+0000 7f1ce553f640 1 --2- 172.21.15.45:0/705340912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ce00d4cc0 0x7f1ce00cc160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0035040 tx=0x7f1cd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:37.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.277+0000 7f1cd67fc640 1 -- 172.21.15.45:0/705340912 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0042020 con 0x7f1ce00d4cc0 2024-05-08T05:41:37.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.277+0000 7f1cd67fc640 1 -- 172.21.15.45:0/705340912 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cd003d470 con 0x7f1ce00d4cc0 2024-05-08T05:41:37.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.277+0000 7f1ce7fa4640 1 -- 172.21.15.45:0/705340912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce00ba210 con 0x7f1ce00d4cc0 2024-05-08T05:41:37.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.277+0000 7f1cd67fc640 1 -- 172.21.15.45:0/705340912 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0037040 con 0x7f1ce00d4cc0 2024-05-08T05:41:37.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.277+0000 7f1ce7fa4640 1 -- 172.21.15.45:0/705340912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce01e90b0 con 0x7f1ce00d4cc0 2024-05-08T05:41:37.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.278+0000 7f1ce7fa4640 1 -- 172.21.15.45:0/705340912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ce01e92b0 con 0x7f1ce00d4cc0 2024-05-08T05:41:37.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.279+0000 7f1cd67fc640 1 -- 172.21.15.45:0/705340912 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1cd0045020 con 0x7f1ce00d4cc0 2024-05-08T05:41:37.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.279+0000 7f1cd67fc640 1 --2- 172.21.15.45:0/705340912 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1cc40762d0 0x7f1cc4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.279+0000 7f1cd67fc640 1 -- 172.21.15.45:0/705340912 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1cd00b8070 con 0x7f1ce00d4cc0 2024-05-08T05:41:37.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.280+0000 7f1ce5d40640 1 --2- 172.21.15.45:0/705340912 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1cc40762d0 0x7f1cc4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.280+0000 7f1ce5d40640 1 --2- 172.21.15.45:0/705340912 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1cc40762d0 0x7f1cc4078790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc001590 tx=0x7f1cdc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:37.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.283+0000 7f1cd67fc640 1 -- 172.21.15.45:0/705340912 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1ce01e92b0 con 0x7f1ce00d4cc0 2024-05-08T05:41:37.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.513+0000 7f1ce7fa4640 1 -- 172.21.15.45:0/705340912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ce015eab0 con 0x7f1ce00d4cc0 2024-05-08T05:41:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.514+0000 7f1cd67fc640 1 -- 172.21.15.45:0/705340912 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5174 (secure 0 0 0) 0x7f1cd004c030 con 0x7f1ce00d4cc0 2024-05-08T05:41:37.514 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:37.514 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2346958825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2346958825},{"type":"v1","addr":"172.21.15.45:6835","nonce":2346958825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/709809719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":709809719},{"type":"v1","addr":"172.21.15.45:6833","nonce":709809719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3208180302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3208180302},{"type":"v1","addr":"172.21.15.63:6836","nonce":3208180302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:41:31.494380+0000","modified":"2024-05-08T05:41:36.532197+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6837/3653698989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":3653698989},{"type":"v1","addr":"172.21.15.63:6837","nonce":3653698989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4303,"qdb_cluster":[4303]},"id":1}]} 2024-05-08T05:41:37.515 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 6 2024-05-08T05:41:37.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.516+0000 7f1ce7fa4640 1 -- 172.21.15.45:0/705340912 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1cc40762d0 msgr2=0x7f1cc4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.516+0000 7f1ce7fa4640 1 --2- 172.21.15.45:0/705340912 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1cc40762d0 0x7f1cc4078790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc001590 tx=0x7f1cdc009040 comp rx=0 tx=0).stop 2024-05-08T05:41:37.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.516+0000 7f1ce7fa4640 1 -- 172.21.15.45:0/705340912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ce00d4cc0 msgr2=0x7f1ce00cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.516+0000 7f1ce7fa4640 1 --2- 172.21.15.45:0/705340912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ce00d4cc0 0x7f1ce00cc160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0035040 tx=0x7f1cd00046c0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.516+0000 7f1ce7fa4640 1 -- 172.21.15.45:0/705340912 shutdown_connections 2024-05-08T05:41:37.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.516+0000 7f1ce7fa4640 1 --2- 172.21.15.45:0/705340912 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1cc40762d0 0x7f1cc4078790 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.516+0000 7f1ce7fa4640 1 --2- 172.21.15.45:0/705340912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1ce015e010 0x7f1ce00cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.516+0000 7f1ce7fa4640 1 --2- 172.21.15.45:0/705340912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ce00d4cc0 0x7f1ce00cc160 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.516+0000 7f1ce7fa4640 1 --2- 172.21.15.45:0/705340912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1ce00d1970 0x7f1ce015d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.516+0000 7f1ce7fa4640 1 -- 172.21.15.45:0/705340912 >> 172.21.15.45:0/705340912 conn(0x7f1ce008eac0 msgr2=0x7f1ce00cfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.516+0000 7f1ce7fa4640 1 -- 172.21.15.45:0/705340912 shutdown_connections 2024-05-08T05:41:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.516+0000 7f1ce7fa4640 1 -- 172.21.15.45:0/705340912 wait complete. 2024-05-08T05:41:37.527 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:41:37.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.632+0000 7f6d22459640 1 Processor -- start 2024-05-08T05:41:37.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.632+0000 7f6d22459640 1 -- start start 2024-05-08T05:41:37.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.633+0000 7f6d22459640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6d1c15f2d0 0x7f6d1c0b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.633+0000 7f6d22459640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d1c0d2310 0x7f6d1c0d5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.633+0000 7f6d22459640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d1c1606c0 con 0x7f6d1c0d4fc0 2024-05-08T05:41:37.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.633+0000 7f6d22459640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6d1c0d2780 con 0x7f6d1c15f2d0 2024-05-08T05:41:37.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.633+0000 7f6d22459640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6d1c0d5960 con 0x7f6d1c0d2310 2024-05-08T05:41:37.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.633+0000 7f6d1b7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6d1c0d4fc0 0x7f6d1c0d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47428/0 (socket says 172.21.15.45:47428) 2024-05-08T05:41:37.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.633+0000 7f6d1b7fe640 1 -- 172.21.15.45:0/4260360519 learned_addr learned my addr 172.21.15.45:0/4260360519 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:37.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1bfff640 1 --2- 172.21.15.45:0/4260360519 >> v2:172.21.15.63:3300/0 conn(0x7f6d1c15f2d0 0x7f6d1c0b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d209f6640 1 --2- 172.21.15.45:0/4260360519 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d1c0d2310 0x7f6d1c0d5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1affd640 1 -- 172.21.15.45:0/4260360519 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1277820221 0 0) 0x7f6d1c1606c0 con 0x7f6d1c0d4fc0 2024-05-08T05:41:37.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1affd640 1 -- 172.21.15.45:0/4260360519 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d04003520 con 0x7f6d1c0d4fc0 2024-05-08T05:41:37.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1affd640 1 -- 172.21.15.45:0/4260360519 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2872057634 0 0) 0x7f6d04003520 con 0x7f6d1c0d4fc0 2024-05-08T05:41:37.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1affd640 1 -- 172.21.15.45:0/4260360519 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d1c0d2310 msgr2=0x7f6d1c0d5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:37.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1affd640 1 --2- 172.21.15.45:0/4260360519 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d1c0d2310 0x7f6d1c0d5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1affd640 1 -- 172.21.15.45:0/4260360519 >> v2:172.21.15.63:3300/0 conn(0x7f6d1c15f2d0 msgr2=0x7f6d1c0b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:37.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1affd640 1 --2- 172.21.15.45:0/4260360519 >> v2:172.21.15.63:3300/0 conn(0x7f6d1c15f2d0 0x7f6d1c0b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1affd640 1 -- 172.21.15.45:0/4260360519 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d1c0b4f00 con 0x7f6d1c0d4fc0 2024-05-08T05:41:37.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1affd640 1 -- 172.21.15.45:0/4260360519 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f6d100016f0 con 0x7f6d1c0d4fc0 2024-05-08T05:41:37.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1bfff640 1 --2- 172.21.15.45:0/4260360519 >> v2:172.21.15.63:3300/0 conn(0x7f6d1c15f2d0 0x7f6d1c0b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1affd640 1 -- 172.21.15.45:0/4260360519 >> v1:172.21.15.45:6789/0 conn(0x7f6d1c0d4fc0 legacy=0x7f6d1c0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1affd640 1 --2- 172.21.15.45:0/4260360519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d04003aa0 0x7f6d04003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1affd640 1 --2- 172.21.15.45:0/4260360519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d04004800 0x7f6d04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1affd640 1 --2- 172.21.15.45:0/4260360519 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d1c0d2310 0x7f6d040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1affd640 1 -- 172.21.15.45:0/4260360519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c0d2780 con 0x7f6d04003aa0 2024-05-08T05:41:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1affd640 1 -- 172.21.15.45:0/4260360519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0400b9c0 con 0x7f6d04004800 2024-05-08T05:41:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1affd640 1 -- 172.21.15.45:0/4260360519 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6d0400bbc0 con 0x7f6d1c0d2310 2024-05-08T05:41:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1bfff640 1 --2- 172.21.15.45:0/4260360519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d04003aa0 0x7f6d04003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1b7fe640 1 --2- 172.21.15.45:0/4260360519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d04004800 0x7f6d04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d209f6640 1 --2- 172.21.15.45:0/4260360519 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d1c0d2310 0x7f6d040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1bfff640 1 -- 172.21.15.45:0/4260360519 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d1c0d2310 msgr2=0x7f6d040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:37.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1bfff640 1 --2- 172.21.15.45:0/4260360519 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d1c0d2310 0x7f6d040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1bfff640 1 -- 172.21.15.45:0/4260360519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d04004800 msgr2=0x7f6d04004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1bfff640 1 --2- 172.21.15.45:0/4260360519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d04004800 0x7f6d04004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.634+0000 7f6d1bfff640 1 -- 172.21.15.45:0/4260360519 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d04003520 con 0x7f6d04003aa0 2024-05-08T05:41:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.635+0000 7f6d1bfff640 1 --2- 172.21.15.45:0/4260360519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d04003aa0 0x7f6d04003ec0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009c90 tx=0x7f6d04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85710c9d6c60943 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.635+0000 7f6d1affd640 1 -- 172.21.15.45:0/4260360519 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c036070 con 0x7f6d04003aa0 2024-05-08T05:41:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.635+0000 7f6d1affd640 1 -- 172.21.15.45:0/4260360519 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d0c031400 con 0x7f6d04003aa0 2024-05-08T05:41:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.635+0000 7f6d1affd640 1 -- 172.21.15.45:0/4260360519 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c009860 con 0x7f6d04003aa0 2024-05-08T05:41:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.636+0000 7f6d22459640 1 -- 172.21.15.45:0/4260360519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d04003aa0 msgr2=0x7f6d04003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.636+0000 7f6d22459640 1 --2- 172.21.15.45:0/4260360519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d04003aa0 0x7f6d04003ec0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009c90 tx=0x7f6d04005830 comp rx=0 tx=0).stop 2024-05-08T05:41:37.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.636+0000 7f6d209f6640 1 -- 172.21.15.45:0/4260360519 reap_dead start 2024-05-08T05:41:37.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.636+0000 7f6d22459640 1 -- 172.21.15.45:0/4260360519 shutdown_connections 2024-05-08T05:41:37.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.636+0000 7f6d22459640 1 -- 172.21.15.45:0/4260360519 >> 172.21.15.45:0/4260360519 conn(0x7f6d1c0746d0 msgr2=0x7f6d1c09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:37.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.636+0000 7f6d22459640 1 -- 172.21.15.45:0/4260360519 shutdown_connections 2024-05-08T05:41:37.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.636+0000 7f6d22459640 1 -- 172.21.15.45:0/4260360519 wait complete. 2024-05-08T05:41:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.637+0000 7f6d22459640 1 Processor -- start 2024-05-08T05:41:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.637+0000 7f6d22459640 1 -- start start 2024-05-08T05:41:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.638+0000 7f6d22459640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d04003aa0 0x7f6d1c152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.638+0000 7f6d22459640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d1c0b3a60 0x7f6d1c0cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.638+0000 7f6d22459640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d1c0cfbe0 0x7f6d1c0d0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.638+0000 7f6d22459640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c0b54a0 con 0x7f6d1c0cfbe0 2024-05-08T05:41:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.638+0000 7f6d22459640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1c0b5810 con 0x7f6d1c0b3a60 2024-05-08T05:41:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.638+0000 7f6d22459640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6d1c0d5420 con 0x7f6d04003aa0 2024-05-08T05:41:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.638+0000 7f6d209f6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d1c0cfbe0 0x7f6d1c0d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.638+0000 7f6d209f6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d1c0cfbe0 0x7f6d1c0d0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33308/0 (socket says 172.21.15.45:33308) 2024-05-08T05:41:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.638+0000 7f6d209f6640 1 -- 172.21.15.45:0/1248013145 learned_addr learned my addr 172.21.15.45:0/1248013145 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:37.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.638+0000 7f6d1bfff640 1 --2- 172.21.15.45:0/1248013145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d04003aa0 0x7f6d1c152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.638+0000 7f6d1b7fe640 1 --2- 172.21.15.45:0/1248013145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d1c0b3a60 0x7f6d1c0cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.638+0000 7f6d209f6640 1 -- 172.21.15.45:0/1248013145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d04003aa0 msgr2=0x7f6d1c152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.638+0000 7f6d209f6640 1 --2- 172.21.15.45:0/1248013145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d04003aa0 0x7f6d1c152530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.638+0000 7f6d209f6640 1 -- 172.21.15.45:0/1248013145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d1c0b3a60 msgr2=0x7f6d1c0cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.638+0000 7f6d209f6640 1 --2- 172.21.15.45:0/1248013145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d1c0b3a60 0x7f6d1c0cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.638+0000 7f6d209f6640 1 -- 172.21.15.45:0/1248013145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d1c0ccb10 con 0x7f6d1c0cfbe0 2024-05-08T05:41:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.638+0000 7f6d1bfff640 1 --2- 172.21.15.45:0/1248013145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d04003aa0 0x7f6d1c152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.638+0000 7f6d1b7fe640 1 --2- 172.21.15.45:0/1248013145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d1c0b3a60 0x7f6d1c0cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.639+0000 7f6d209f6640 1 --2- 172.21.15.45:0/1248013145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d1c0cfbe0 0x7f6d1c0d0060 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6d14011bf0 tx=0x7f6d1400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.639+0000 7f6d18ff9640 1 -- 172.21.15.45:0/1248013145 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1400fdb0 con 0x7f6d1c0cfbe0 2024-05-08T05:41:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.639+0000 7f6d18ff9640 1 -- 172.21.15.45:0/1248013145 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d1401d070 con 0x7f6d1c0cfbe0 2024-05-08T05:41:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.639+0000 7f6d18ff9640 1 -- 172.21.15.45:0/1248013145 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d14015670 con 0x7f6d1c0cfbe0 2024-05-08T05:41:37.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.639+0000 7f6d22459640 1 -- 172.21.15.45:0/1248013145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d1c0ccdb0 con 0x7f6d1c0cfbe0 2024-05-08T05:41:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.639+0000 7f6d22459640 1 -- 172.21.15.45:0/1248013145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d1c0cd370 con 0x7f6d1c0cfbe0 2024-05-08T05:41:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.640+0000 7f6d22459640 1 -- 172.21.15.45:0/1248013145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d1c069b00 con 0x7f6d1c0cfbe0 2024-05-08T05:41:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.640+0000 7f6d18ff9640 1 -- 172.21.15.45:0/1248013145 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6d14015810 con 0x7f6d1c0cfbe0 2024-05-08T05:41:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.641+0000 7f6d18ff9640 1 --2- 172.21.15.45:0/1248013145 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6d080762d0 0x7f6d08078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.641+0000 7f6d18ff9640 1 -- 172.21.15.45:0/1248013145 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6d140981f0 con 0x7f6d1c0cfbe0 2024-05-08T05:41:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.641+0000 7f6d1bfff640 1 --2- 172.21.15.45:0/1248013145 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6d080762d0 0x7f6d08078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.642+0000 7f6d1bfff640 1 --2- 172.21.15.45:0/1248013145 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6d080762d0 0x7f6d08078790 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c033000 tx=0x7f6d0c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:37.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.645+0000 7f6d18ff9640 1 -- 172.21.15.45:0/1248013145 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6d14010040 con 0x7f6d1c0cfbe0 2024-05-08T05:41:37.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.836+0000 7f6d22459640 1 -- 172.21.15.45:0/1248013145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d1c0b4b60 con 0x7f6d1c0cfbe0 2024-05-08T05:41:37.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.837+0000 7f6d18ff9640 1 -- 172.21.15.45:0/1248013145 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18466 (secure 0 0 0) 0x7f6d14061810 con 0x7f6d1c0cfbe0 2024-05-08T05:41:37.838 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:37.838 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:41:36.532045+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:41:29.381274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:41:30.443612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:41:34.132285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:41:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.839+0000 7f6d22459640 1 -- 172.21.15.45:0/1248013145 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6d080762d0 msgr2=0x7f6d08078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.839+0000 7f6d22459640 1 --2- 172.21.15.45:0/1248013145 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6d080762d0 0x7f6d08078790 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c033000 tx=0x7f6d0c008030 comp rx=0 tx=0).stop 2024-05-08T05:41:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.839+0000 7f6d22459640 1 -- 172.21.15.45:0/1248013145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d1c0cfbe0 msgr2=0x7f6d1c0d0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.839+0000 7f6d22459640 1 --2- 172.21.15.45:0/1248013145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d1c0cfbe0 0x7f6d1c0d0060 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6d14011bf0 tx=0x7f6d1400f580 comp rx=0 tx=0).stop 2024-05-08T05:41:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.839+0000 7f6d22459640 1 -- 172.21.15.45:0/1248013145 shutdown_connections 2024-05-08T05:41:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.839+0000 7f6d22459640 1 --2- 172.21.15.45:0/1248013145 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6d080762d0 0x7f6d08078790 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.839+0000 7f6d22459640 1 --2- 172.21.15.45:0/1248013145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d1c0cfbe0 0x7f6d1c0d0060 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.839+0000 7f6d22459640 1 --2- 172.21.15.45:0/1248013145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d1c0b3a60 0x7f6d1c0cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.839+0000 7f6d22459640 1 --2- 172.21.15.45:0/1248013145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d04003aa0 0x7f6d1c152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.839+0000 7f6d22459640 1 -- 172.21.15.45:0/1248013145 >> 172.21.15.45:0/1248013145 conn(0x7f6d1c0746d0 msgr2=0x7f6d1c0b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:37.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.839+0000 7f6d22459640 1 -- 172.21.15.45:0/1248013145 shutdown_connections 2024-05-08T05:41:37.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.839+0000 7f6d22459640 1 -- 172.21.15.45:0/1248013145 wait complete. 2024-05-08T05:41:37.851 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:41:37.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.955+0000 7fa995809640 1 Processor -- start 2024-05-08T05:41:37.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.955+0000 7fa995809640 1 -- start start 2024-05-08T05:41:37.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.956+0000 7fa995809640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9900d0ec0 0x7fa9900d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.956+0000 7fa995809640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa9900d3b00 0x7fa9900b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.956+0000 7fa995809640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9900d1760 con 0x7fa99015f380 2024-05-08T05:41:37.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.956+0000 7fa995809640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa9900d30f0 con 0x7fa9900d3b00 2024-05-08T05:41:37.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.956+0000 7fa995809640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa9900d2370 con 0x7fa9900d0ec0 2024-05-08T05:41:37.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.956+0000 7fa98f7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa9900d3b00 0x7fa9900b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.956+0000 7fa98e7fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9900d0ec0 0x7fa9900d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.956+0000 7fa98effd640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fa99015f380 0x7fa9900d07b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:48072/0 (socket says 172.21.15.45:48072) 2024-05-08T05:41:37.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.956+0000 7fa98f7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa9900d3b00 0x7fa9900b7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:51902/0 (socket says 172.21.15.45:51902) 2024-05-08T05:41:37.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.956+0000 7fa98e7fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9900d0ec0 0x7fa9900d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:49114/0 (socket says 172.21.15.45:49114) 2024-05-08T05:41:37.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.956+0000 7fa98effd640 1 -- 172.21.15.45:0/1322337961 learned_addr learned my addr 172.21.15.45:0/1322337961 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:37.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.957+0000 7fa98dffb640 1 -- 172.21.15.45:0/1322337961 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2850092382 0 0) 0x7fa9900d1760 con 0x7fa99015f380 2024-05-08T05:41:37.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.957+0000 7fa98dffb640 1 -- 172.21.15.45:0/1322337961 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa978003520 con 0x7fa99015f380 2024-05-08T05:41:37.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.957+0000 7fa98f7fe640 1 -- 172.21.15.45:0/1322337961 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9900d0ec0 msgr2=0x7fa9900d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:37.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.957+0000 7fa98f7fe640 1 --2- 172.21.15.45:0/1322337961 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9900d0ec0 0x7fa9900d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.957+0000 7fa98f7fe640 1 -- 172.21.15.45:0/1322337961 >> v1:172.21.15.63:6789/0 conn(0x7fa99015f380 legacy=0x7fa9900d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.957+0000 7fa98f7fe640 1 -- 172.21.15.45:0/1322337961 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9900d1a40 con 0x7fa9900d3b00 2024-05-08T05:41:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.958+0000 7fa98f7fe640 1 --2- 172.21.15.45:0/1322337961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9900d3b00 0x7fa9900b7e30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9880097a0 tx=0x7fa98802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fec724670d3553e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.958+0000 7fa98dffb640 1 -- 172.21.15.45:0/1322337961 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988037070 con 0x7fa9900d3b00 2024-05-08T05:41:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.958+0000 7fa98dffb640 1 -- 172.21.15.45:0/1322337961 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa988032440 con 0x7fa9900d3b00 2024-05-08T05:41:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.958+0000 7fa98dffb640 1 -- 172.21.15.45:0/1322337961 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9880327f0 con 0x7fa9900d3b00 2024-05-08T05:41:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.958+0000 7fa995809640 1 -- 172.21.15.45:0/1322337961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9900d3b00 msgr2=0x7fa9900b7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.958+0000 7fa995809640 1 --2- 172.21.15.45:0/1322337961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9900d3b00 0x7fa9900b7e30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9880097a0 tx=0x7fa98802a180 comp rx=0 tx=0).stop 2024-05-08T05:41:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.958+0000 7fa995809640 1 -- 172.21.15.45:0/1322337961 shutdown_connections 2024-05-08T05:41:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.958+0000 7fa995809640 1 --2- 172.21.15.45:0/1322337961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9900d3b00 0x7fa9900b7e30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.958+0000 7fa995809640 1 --2- 172.21.15.45:0/1322337961 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9900d0ec0 0x7fa9900d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.958+0000 7fa995809640 1 -- 172.21.15.45:0/1322337961 >> 172.21.15.45:0/1322337961 conn(0x7fa9900d4fe0 msgr2=0x7fa9900d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:37.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.958+0000 7fa995809640 1 -- 172.21.15.45:0/1322337961 shutdown_connections 2024-05-08T05:41:37.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.958+0000 7fa995809640 1 -- 172.21.15.45:0/1322337961 wait complete. 2024-05-08T05:41:37.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.960+0000 7fa995809640 1 Processor -- start 2024-05-08T05:41:37.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.960+0000 7fa995809640 1 -- start start 2024-05-08T05:41:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.960+0000 7fa995809640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9900d0ec0 0x7fa9901f5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.960+0000 7fa995809640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9900d3b00 0x7fa99015b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.960+0000 7fa995809640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa99015f380 0x7fa9900c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.961+0000 7fa995809640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa99015f760 con 0x7fa9900d3b00 2024-05-08T05:41:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.961+0000 7fa995809640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa9900d2000 con 0x7fa9900d0ec0 2024-05-08T05:41:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.961+0000 7fa995809640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa9900d2800 con 0x7fa99015f380 2024-05-08T05:41:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.961+0000 7fa98e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9900d3b00 0x7fa99015b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.961+0000 7fa98e7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9900d3b00 0x7fa99015b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33314/0 (socket says 172.21.15.45:33314) 2024-05-08T05:41:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.961+0000 7fa98e7fc640 1 -- 172.21.15.45:0/1060113411 learned_addr learned my addr 172.21.15.45:0/1060113411 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.961+0000 7fa98effd640 1 --2- 172.21.15.45:0/1060113411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9900d0ec0 0x7fa9901f5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.961+0000 7fa98f7fe640 1 --2- 172.21.15.45:0/1060113411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa99015f380 0x7fa9900c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.961+0000 7fa98e7fc640 1 -- 172.21.15.45:0/1060113411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa99015f380 msgr2=0x7fa9900c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.961+0000 7fa98e7fc640 1 --2- 172.21.15.45:0/1060113411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa99015f380 0x7fa9900c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.961+0000 7fa98e7fc640 1 -- 172.21.15.45:0/1060113411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9900d0ec0 msgr2=0x7fa9901f5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.961+0000 7fa98e7fc640 1 --2- 172.21.15.45:0/1060113411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9900d0ec0 0x7fa9901f5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.961+0000 7fa98e7fc640 1 -- 172.21.15.45:0/1060113411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9901f6180 con 0x7fa9900d3b00 2024-05-08T05:41:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.961+0000 7fa98effd640 1 --2- 172.21.15.45:0/1060113411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9900d0ec0 0x7fa9901f5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.961+0000 7fa98e7fc640 1 --2- 172.21.15.45:0/1060113411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9900d3b00 0x7fa99015b4e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa98400e9b0 tx=0x7fa98400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.961+0000 7fa977fff640 1 -- 172.21.15.45:0/1060113411 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa984017070 con 0x7fa9900d3b00 2024-05-08T05:41:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.961+0000 7fa977fff640 1 -- 172.21.15.45:0/1060113411 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa98400c9a0 con 0x7fa9900d3b00 2024-05-08T05:41:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.961+0000 7fa995809640 1 -- 172.21.15.45:0/1060113411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa990163410 con 0x7fa9900d3b00 2024-05-08T05:41:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.961+0000 7fa977fff640 1 -- 172.21.15.45:0/1060113411 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98400cd70 con 0x7fa9900d3b00 2024-05-08T05:41:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.961+0000 7fa995809640 1 -- 172.21.15.45:0/1060113411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9900cc7c0 con 0x7fa9900d3b00 2024-05-08T05:41:37.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.962+0000 7fa995809640 1 -- 172.21.15.45:0/1060113411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9900693f0 con 0x7fa9900d3b00 2024-05-08T05:41:37.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.963+0000 7fa977fff640 1 -- 172.21.15.45:0/1060113411 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa98401a070 con 0x7fa9900d3b00 2024-05-08T05:41:37.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.963+0000 7fa977fff640 1 --2- 172.21.15.45:0/1060113411 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa97c0762d0 0x7fa97c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.964+0000 7fa977fff640 1 -- 172.21.15.45:0/1060113411 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa9840978f0 con 0x7fa9900d3b00 2024-05-08T05:41:37.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.964+0000 7fa98effd640 1 --2- 172.21.15.45:0/1060113411 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa97c0762d0 0x7fa97c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.964+0000 7fa98effd640 1 --2- 172.21.15.45:0/1060113411 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa97c0762d0 0x7fa97c078790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa980005c00 tx=0x7fa980009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:37.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:37.968+0000 7fa977fff640 1 -- 172.21.15.45:0/1060113411 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa9901f6180 con 0x7fa9900d3b00 2024-05-08T05:41:38.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.161+0000 7fa995809640 1 -- 172.21.15.45:0/1060113411 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa99008cfd0 con 0x7fa9900d3b00 2024-05-08T05:41:38.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.162+0000 7fa977fff640 1 -- 172.21.15.45:0/1060113411 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18466 (secure 0 0 0) 0x7fa984067e20 con 0x7fa9900d3b00 2024-05-08T05:41:38.163 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:38.163 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:41:36.532045+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:41:29.381274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:41:30.443612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:41:34.132285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:41:38.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.164+0000 7fa995809640 1 -- 172.21.15.45:0/1060113411 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa97c0762d0 msgr2=0x7fa97c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:38.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.164+0000 7fa995809640 1 --2- 172.21.15.45:0/1060113411 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa97c0762d0 0x7fa97c078790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa980005c00 tx=0x7fa980009040 comp rx=0 tx=0).stop 2024-05-08T05:41:38.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.164+0000 7fa995809640 1 -- 172.21.15.45:0/1060113411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9900d3b00 msgr2=0x7fa99015b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:38.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.164+0000 7fa995809640 1 --2- 172.21.15.45:0/1060113411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9900d3b00 0x7fa99015b4e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa98400e9b0 tx=0x7fa98400ee70 comp rx=0 tx=0).stop 2024-05-08T05:41:38.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.164+0000 7fa995809640 1 -- 172.21.15.45:0/1060113411 shutdown_connections 2024-05-08T05:41:38.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.164+0000 7fa995809640 1 --2- 172.21.15.45:0/1060113411 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa97c0762d0 0x7fa97c078790 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.164+0000 7fa995809640 1 --2- 172.21.15.45:0/1060113411 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa99015f380 0x7fa9900c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.164+0000 7fa995809640 1 --2- 172.21.15.45:0/1060113411 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9900d3b00 0x7fa99015b4e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.164+0000 7fa995809640 1 --2- 172.21.15.45:0/1060113411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9900d0ec0 0x7fa9901f5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.164+0000 7fa995809640 1 -- 172.21.15.45:0/1060113411 >> 172.21.15.45:0/1060113411 conn(0x7fa9900d4fe0 msgr2=0x7fa9900ba440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:38.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.164+0000 7fa995809640 1 -- 172.21.15.45:0/1060113411 shutdown_connections 2024-05-08T05:41:38.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.164+0000 7fa995809640 1 -- 172.21.15.45:0/1060113411 wait complete. 2024-05-08T05:41:38.216 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:41:38.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.319+0000 7f7e12119640 1 Processor -- start 2024-05-08T05:41:38.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.320+0000 7f7e12119640 1 -- start start 2024-05-08T05:41:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.320+0000 7f7e12119640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7e0c0b21f0 0x7f7e0c06e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.320+0000 7f7e12119640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e0c0d0500 con 0x7f7e0c162f50 2024-05-08T05:41:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.320+0000 7f7e12119640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e0c06da60 con 0x7f7e0c0cfea0 2024-05-08T05:41:38.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.320+0000 7f7e12119640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f7e0c06d810 con 0x7f7e0c0b21f0 2024-05-08T05:41:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.320+0000 7f7e0b7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7e0c0b21f0 0x7f7e0c06e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.320+0000 7f7e0affd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7e0c162f50 0x7f7e0c0cf790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47440/0 (socket says 172.21.15.45:47440) 2024-05-08T05:41:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.320+0000 7f7e0affd640 1 -- 172.21.15.45:0/2130756510 learned_addr learned my addr 172.21.15.45:0/2130756510 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.321+0000 7f7e0b7fe640 1 -- 172.21.15.45:0/2130756510 >> v1:172.21.15.63:6789/0 conn(0x7f7e0c0cfea0 legacy=0x7f7e0c0cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.321+0000 7f7e0b7fe640 1 -- 172.21.15.45:0/2130756510 >> v1:172.21.15.45:6789/0 conn(0x7f7e0c162f50 legacy=0x7f7e0c0cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.321+0000 7f7e0a7fc640 1 -- 172.21.15.45:0/2130756510 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1525666701 0 0) 0x7f7e0c0d0500 con 0x7f7e0c162f50 2024-05-08T05:41:38.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.321+0000 7f7e0b7fe640 1 -- 172.21.15.45:0/2130756510 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e0c06dd40 con 0x7f7e0c0b21f0 2024-05-08T05:41:38.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.322+0000 7f7e0b7fe640 1 --2- 172.21.15.45:0/2130756510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e0c0b21f0 0x7f7e0c06e540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc0097a0 tx=0x7f7dfc02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d42a072cd70732 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:38.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.322+0000 7f7e0a7fc640 1 -- 172.21.15.45:0/2130756510 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc037070 con 0x7f7e0c0b21f0 2024-05-08T05:41:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.322+0000 7f7e0a7fc640 1 -- 172.21.15.45:0/2130756510 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dfc02adb0 con 0x7f7e0c0b21f0 2024-05-08T05:41:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.322+0000 7f7e0a7fc640 1 -- 172.21.15.45:0/2130756510 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc032660 con 0x7f7e0c0b21f0 2024-05-08T05:41:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.322+0000 7f7e12119640 1 -- 172.21.15.45:0/2130756510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e0c0b21f0 msgr2=0x7f7e0c06e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.322+0000 7f7e12119640 1 --2- 172.21.15.45:0/2130756510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e0c0b21f0 0x7f7e0c06e540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc0097a0 tx=0x7f7dfc02a060 comp rx=0 tx=0).stop 2024-05-08T05:41:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.322+0000 7f7e12119640 1 -- 172.21.15.45:0/2130756510 shutdown_connections 2024-05-08T05:41:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.322+0000 7f7e12119640 1 --2- 172.21.15.45:0/2130756510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e0c0b21f0 0x7f7e0c06e540 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.322+0000 7f7e12119640 1 -- 172.21.15.45:0/2130756510 >> 172.21.15.45:0/2130756510 conn(0x7f7e0c09ba40 msgr2=0x7f7e0c09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.322+0000 7f7e12119640 1 -- 172.21.15.45:0/2130756510 shutdown_connections 2024-05-08T05:41:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.322+0000 7f7e12119640 1 -- 172.21.15.45:0/2130756510 wait complete. 2024-05-08T05:41:38.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.323+0000 7f7e12119640 1 Processor -- start 2024-05-08T05:41:38.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.324+0000 7f7e12119640 1 -- start start 2024-05-08T05:41:38.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.324+0000 7f7e12119640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e0c0b21f0 0x7f7e0c1f9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:38.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.324+0000 7f7e12119640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e0c0cfea0 0x7f7e0c1541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.324+0000 7f7e12119640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e0c162f50 0x7f7e0c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.324+0000 7f7e12119640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c0b25d0 con 0x7f7e0c0cfea0 2024-05-08T05:41:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.324+0000 7f7e12119640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c06e0b0 con 0x7f7e0c162f50 2024-05-08T05:41:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.324+0000 7f7e12119640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7e0c0cdb90 con 0x7f7e0c0b21f0 2024-05-08T05:41:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.325+0000 7f7e0affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e0c0cfea0 0x7f7e0c1541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.325+0000 7f7e0affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e0c0cfea0 0x7f7e0c1541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33342/0 (socket says 172.21.15.45:33342) 2024-05-08T05:41:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.325+0000 7f7e0affd640 1 -- 172.21.15.45:0/1458656854 learned_addr learned my addr 172.21.15.45:0/1458656854 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.325+0000 7f7e0b7fe640 1 --2- 172.21.15.45:0/1458656854 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e0c0b21f0 0x7f7e0c1f9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.325+0000 7f7e0bfff640 1 --2- 172.21.15.45:0/1458656854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e0c162f50 0x7f7e0c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:38.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.325+0000 7f7e0affd640 1 -- 172.21.15.45:0/1458656854 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e0c0b21f0 msgr2=0x7f7e0c1f9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:38.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.325+0000 7f7e0affd640 1 --2- 172.21.15.45:0/1458656854 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e0c0b21f0 0x7f7e0c1f9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.325+0000 7f7e0affd640 1 -- 172.21.15.45:0/1458656854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e0c162f50 msgr2=0x7f7e0c0b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:38.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.325+0000 7f7e0affd640 1 --2- 172.21.15.45:0/1458656854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e0c162f50 0x7f7e0c0b4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.325+0000 7f7e0affd640 1 -- 172.21.15.45:0/1458656854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e0c1f9890 con 0x7f7e0c0cfea0 2024-05-08T05:41:38.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.325+0000 7f7e0affd640 1 --2- 172.21.15.45:0/1458656854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e0c0cfea0 0x7f7e0c1541b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000c3e0 tx=0x7f7e0000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:38.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.325+0000 7f7df3fff640 1 -- 172.21.15.45:0/1458656854 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e000074e0 con 0x7f7e0c0cfea0 2024-05-08T05:41:38.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.325+0000 7f7df3fff640 1 -- 172.21.15.45:0/1458656854 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e000047e0 con 0x7f7e0c0cfea0 2024-05-08T05:41:38.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.325+0000 7f7df3fff640 1 -- 172.21.15.45:0/1458656854 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00004bd0 con 0x7f7e0c0cfea0 2024-05-08T05:41:38.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.325+0000 7f7e12119640 1 -- 172.21.15.45:0/1458656854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e0c0b2a60 con 0x7f7e0c0cfea0 2024-05-08T05:41:38.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.325+0000 7f7e12119640 1 -- 172.21.15.45:0/1458656854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e0c0b4730 con 0x7f7e0c0cfea0 2024-05-08T05:41:38.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.327+0000 7f7e12119640 1 -- 172.21.15.45:0/1458656854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e0c0b27b0 con 0x7f7e0c0cfea0 2024-05-08T05:41:38.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.327+0000 7f7df3fff640 1 -- 172.21.15.45:0/1458656854 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7e00011070 con 0x7f7e0c0cfea0 2024-05-08T05:41:38.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.328+0000 7f7df3fff640 1 --2- 172.21.15.45:0/1458656854 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7df80762d0 0x7f7df8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:38.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.328+0000 7f7df3fff640 1 -- 172.21.15.45:0/1458656854 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7e00097520 con 0x7f7e0c0cfea0 2024-05-08T05:41:38.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.328+0000 7f7e0b7fe640 1 --2- 172.21.15.45:0/1458656854 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7df80762d0 0x7f7df8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:38.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.328+0000 7f7e0b7fe640 1 --2- 172.21.15.45:0/1458656854 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7df80762d0 0x7f7df8078790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc034000 tx=0x7f7dfc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:38.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.332+0000 7f7df3fff640 1 -- 172.21.15.45:0/1458656854 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7e0c1f9890 con 0x7f7e0c0cfea0 2024-05-08T05:41:38.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.564+0000 7f7e12119640 1 -- 172.21.15.45:0/1458656854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e0c070a30 con 0x7f7e0c0cfea0 2024-05-08T05:41:38.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.565+0000 7f7df3fff640 1 -- 172.21.15.45:0/1458656854 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5174 (secure 0 0 0) 0x7f7e00060b40 con 0x7f7e0c0cfea0 2024-05-08T05:41:38.565 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:38.565 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2346958825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2346958825},{"type":"v1","addr":"172.21.15.45:6835","nonce":2346958825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/709809719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":709809719},{"type":"v1","addr":"172.21.15.45:6833","nonce":709809719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3208180302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3208180302},{"type":"v1","addr":"172.21.15.63:6836","nonce":3208180302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:41:31.494380+0000","modified":"2024-05-08T05:41:36.532197+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6837/3653698989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":3653698989},{"type":"v1","addr":"172.21.15.63:6837","nonce":3653698989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4303,"qdb_cluster":[4303]},"id":1}]} 2024-05-08T05:41:38.566 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 6 2024-05-08T05:41:38.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.567+0000 7f7e12119640 1 -- 172.21.15.45:0/1458656854 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7df80762d0 msgr2=0x7f7df8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:38.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.567+0000 7f7e12119640 1 --2- 172.21.15.45:0/1458656854 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7df80762d0 0x7f7df8078790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc034000 tx=0x7f7dfc02e000 comp rx=0 tx=0).stop 2024-05-08T05:41:38.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.567+0000 7f7e12119640 1 -- 172.21.15.45:0/1458656854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e0c0cfea0 msgr2=0x7f7e0c1541b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:38.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.567+0000 7f7e12119640 1 --2- 172.21.15.45:0/1458656854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e0c0cfea0 0x7f7e0c1541b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000c3e0 tx=0x7f7e0000c8a0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.567+0000 7f7e12119640 1 -- 172.21.15.45:0/1458656854 shutdown_connections 2024-05-08T05:41:38.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.567+0000 7f7e12119640 1 --2- 172.21.15.45:0/1458656854 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7df80762d0 0x7f7df8078790 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.567+0000 7f7e12119640 1 --2- 172.21.15.45:0/1458656854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e0c162f50 0x7f7e0c0b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.567+0000 7f7e12119640 1 --2- 172.21.15.45:0/1458656854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e0c0cfea0 0x7f7e0c1541b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.567+0000 7f7e12119640 1 --2- 172.21.15.45:0/1458656854 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e0c0b21f0 0x7f7e0c1f9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.567+0000 7f7e12119640 1 -- 172.21.15.45:0/1458656854 >> 172.21.15.45:0/1458656854 conn(0x7f7e0c09ba40 msgr2=0x7f7e0c0afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:38.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.567+0000 7f7e12119640 1 -- 172.21.15.45:0/1458656854 shutdown_connections 2024-05-08T05:41:38.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.567+0000 7f7e12119640 1 -- 172.21.15.45:0/1458656854 wait complete. 2024-05-08T05:41:38.579 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:41:38.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.681+0000 7fc0851af640 1 Processor -- start 2024-05-08T05:41:38.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.681+0000 7fc0851af640 1 -- start start 2024-05-08T05:41:38.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.682+0000 7fc0851af640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc08015bc50 0x7fc0800b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:38.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.682+0000 7fc0851af640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc08006f540 0x7fc080070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:38.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.682+0000 7fc0851af640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc08006fa40 con 0x7fc080071020 2024-05-08T05:41:38.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.682+0000 7fc0851af640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc08006fd20 con 0x7fc08006f540 2024-05-08T05:41:38.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.682+0000 7fc0851af640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc080070000 con 0x7fc08015bc50 2024-05-08T05:41:38.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.683+0000 7fc07e59c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc08006f540 0x7fc080070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:38.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.683+0000 7fc07ed9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc08015bc50 0x7fc0800b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:38.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.683+0000 7fc07f59e640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fc080071020 0x7fc0800732c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:48076/0 (socket says 172.21.15.45:48076) 2024-05-08T05:41:38.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.683+0000 7fc07f59e640 1 -- 172.21.15.45:0/914341257 learned_addr learned my addr 172.21.15.45:0/914341257 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:38.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.683+0000 7fc07e59c640 1 --2- 172.21.15.45:0/914341257 >> v2:172.21.15.63:3300/0 conn(0x7fc08006f540 0x7fc080070ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:51926/0 (socket says 172.21.15.45:51926) 2024-05-08T05:41:38.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.683+0000 7fc07ed9d640 1 --2- 172.21.15.45:0/914341257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc08015bc50 0x7fc0800b5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:49142/0 (socket says 172.21.15.45:49142) 2024-05-08T05:41:38.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.683+0000 7fc07e59c640 1 -- 172.21.15.45:0/914341257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc08015bc50 msgr2=0x7fc0800b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:38.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.683+0000 7fc07e59c640 1 --2- 172.21.15.45:0/914341257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc08015bc50 0x7fc0800b5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.683+0000 7fc07e59c640 1 -- 172.21.15.45:0/914341257 >> v1:172.21.15.63:6789/0 conn(0x7fc080071020 legacy=0x7fc0800732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:38.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.683+0000 7fc07e59c640 1 -- 172.21.15.45:0/914341257 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0800702e0 con 0x7fc08006f540 2024-05-08T05:41:38.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.683+0000 7fc07ed9d640 1 --2- 172.21.15.45:0/914341257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc08015bc50 0x7fc0800b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:38.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.684+0000 7fc07e59c640 1 --2- 172.21.15.45:0/914341257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc08006f540 0x7fc080070ae0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc0740097a0 tx=0x7fc07402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db7dcd49d26da706 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:38.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.684+0000 7fc07dd9b640 1 -- 172.21.15.45:0/914341257 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074037070 con 0x7fc08006f540 2024-05-08T05:41:38.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.684+0000 7fc07dd9b640 1 -- 172.21.15.45:0/914341257 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc074032440 con 0x7fc08006f540 2024-05-08T05:41:38.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.684+0000 7fc07dd9b640 1 -- 172.21.15.45:0/914341257 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074032810 con 0x7fc08006f540 2024-05-08T05:41:38.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.684+0000 7fc0851af640 1 -- 172.21.15.45:0/914341257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc08006f540 msgr2=0x7fc080070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:38.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.684+0000 7fc0851af640 1 --2- 172.21.15.45:0/914341257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc08006f540 0x7fc080070ae0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc0740097a0 tx=0x7fc07402a180 comp rx=0 tx=0).stop 2024-05-08T05:41:38.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.685+0000 7fc0851af640 1 -- 172.21.15.45:0/914341257 shutdown_connections 2024-05-08T05:41:38.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.685+0000 7fc0851af640 1 --2- 172.21.15.45:0/914341257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc08006f540 0x7fc080070ae0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.685+0000 7fc0851af640 1 --2- 172.21.15.45:0/914341257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc08015bc50 0x7fc0800b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.685+0000 7fc0851af640 1 -- 172.21.15.45:0/914341257 >> 172.21.15.45:0/914341257 conn(0x7fc0800b3c40 msgr2=0x7fc0800b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:38.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.685+0000 7fc0851af640 1 -- 172.21.15.45:0/914341257 shutdown_connections 2024-05-08T05:41:38.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.685+0000 7fc0851af640 1 -- 172.21.15.45:0/914341257 wait complete. 2024-05-08T05:41:38.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.686+0000 7fc0851af640 1 Processor -- start 2024-05-08T05:41:38.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.686+0000 7fc0851af640 1 -- start start 2024-05-08T05:41:38.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.687+0000 7fc0851af640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc08006f540 0x7fc0801f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:38.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.687+0000 7fc0851af640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc08015bc50 0x7fc0801540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:38.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.687+0000 7fc0851af640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0801572d0 0x7fc080157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:38.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.687+0000 7fc0851af640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc0800b6260 con 0x7fc0801572d0 2024-05-08T05:41:38.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.687+0000 7fc0851af640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc0800b60e0 con 0x7fc08006f540 2024-05-08T05:41:38.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.687+0000 7fc0851af640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc08015c030 con 0x7fc08015bc50 2024-05-08T05:41:38.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.687+0000 7fc07f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0801572d0 0x7fc080157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:38.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.687+0000 7fc07f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0801572d0 0x7fc080157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33348/0 (socket says 172.21.15.45:33348) 2024-05-08T05:41:38.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.687+0000 7fc07f59e640 1 -- 172.21.15.45:0/2869234356 learned_addr learned my addr 172.21.15.45:0/2869234356 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:38.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.687+0000 7fc07e59c640 1 --2- 172.21.15.45:0/2869234356 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc08015bc50 0x7fc0801540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:38.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.687+0000 7fc07ed9d640 1 --2- 172.21.15.45:0/2869234356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc08006f540 0x7fc0801f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:38.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.687+0000 7fc07f59e640 1 -- 172.21.15.45:0/2869234356 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc08015bc50 msgr2=0x7fc0801540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:38.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.687+0000 7fc07f59e640 1 --2- 172.21.15.45:0/2869234356 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc08015bc50 0x7fc0801540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.687+0000 7fc07f59e640 1 -- 172.21.15.45:0/2869234356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc08006f540 msgr2=0x7fc0801f5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:38.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.687+0000 7fc07f59e640 1 --2- 172.21.15.45:0/2869234356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc08006f540 0x7fc0801f5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.687+0000 7fc07f59e640 1 -- 172.21.15.45:0/2869234356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0801f6350 con 0x7fc0801572d0 2024-05-08T05:41:38.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.687+0000 7fc07f59e640 1 --2- 172.21.15.45:0/2869234356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0801572d0 0x7fc080157750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc07800caf0 tx=0x7fc07800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:38.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.688+0000 7fc0677fe640 1 -- 172.21.15.45:0/2869234356 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc078017070 con 0x7fc0801572d0 2024-05-08T05:41:38.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.688+0000 7fc0677fe640 1 -- 172.21.15.45:0/2869234356 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc07800aca0 con 0x7fc0801572d0 2024-05-08T05:41:38.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.688+0000 7fc0851af640 1 -- 172.21.15.45:0/2869234356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc080153450 con 0x7fc0801572d0 2024-05-08T05:41:38.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.688+0000 7fc0677fe640 1 -- 172.21.15.45:0/2869234356 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc078012600 con 0x7fc0801572d0 2024-05-08T05:41:38.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.688+0000 7fc0851af640 1 -- 172.21.15.45:0/2869234356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08015db80 con 0x7fc0801572d0 2024-05-08T05:41:38.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.689+0000 7fc0851af640 1 -- 172.21.15.45:0/2869234356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc080069b00 con 0x7fc0801572d0 2024-05-08T05:41:38.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.689+0000 7fc0677fe640 1 -- 172.21.15.45:0/2869234356 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc07801a070 con 0x7fc0801572d0 2024-05-08T05:41:38.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.690+0000 7fc0677fe640 1 --2- 172.21.15.45:0/2869234356 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc06c0762d0 0x7fc06c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:38.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.690+0000 7fc0677fe640 1 -- 172.21.15.45:0/2869234356 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc078097660 con 0x7fc0801572d0 2024-05-08T05:41:38.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.690+0000 7fc07ed9d640 1 --2- 172.21.15.45:0/2869234356 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc06c0762d0 0x7fc06c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:38.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.691+0000 7fc07ed9d640 1 --2- 172.21.15.45:0/2869234356 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc06c0762d0 0x7fc06c078790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc070009560 tx=0x7fc070009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:38.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.694+0000 7fc0677fe640 1 -- 172.21.15.45:0/2869234356 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc0801f6350 con 0x7fc0801572d0 2024-05-08T05:41:38.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.885+0000 7fc0851af640 1 -- 172.21.15.45:0/2869234356 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc08015b200 con 0x7fc0801572d0 2024-05-08T05:41:38.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.886+0000 7fc0677fe640 1 -- 172.21.15.45:0/2869234356 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18466 (secure 0 0 0) 0x7fc078060590 con 0x7fc0801572d0 2024-05-08T05:41:38.887 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:38.887 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:41:36.532045+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:41:29.381274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:41:30.443612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:41:34.132285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:41:38.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.889+0000 7fc0851af640 1 -- 172.21.15.45:0/2869234356 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc06c0762d0 msgr2=0x7fc06c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:38.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.889+0000 7fc0851af640 1 --2- 172.21.15.45:0/2869234356 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc06c0762d0 0x7fc06c078790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc070009560 tx=0x7fc070009190 comp rx=0 tx=0).stop 2024-05-08T05:41:38.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.889+0000 7fc0851af640 1 -- 172.21.15.45:0/2869234356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0801572d0 msgr2=0x7fc080157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:38.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.889+0000 7fc0851af640 1 --2- 172.21.15.45:0/2869234356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0801572d0 0x7fc080157750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc07800caf0 tx=0x7fc07800a060 comp rx=0 tx=0).stop 2024-05-08T05:41:38.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.889+0000 7fc0851af640 1 -- 172.21.15.45:0/2869234356 shutdown_connections 2024-05-08T05:41:38.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.889+0000 7fc0851af640 1 --2- 172.21.15.45:0/2869234356 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc06c0762d0 0x7fc06c078790 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.889+0000 7fc0851af640 1 --2- 172.21.15.45:0/2869234356 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc08015bc50 0x7fc0801540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.889+0000 7fc0851af640 1 --2- 172.21.15.45:0/2869234356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0801572d0 0x7fc080157750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.889+0000 7fc0851af640 1 --2- 172.21.15.45:0/2869234356 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc08006f540 0x7fc0801f5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:38.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.889+0000 7fc0851af640 1 -- 172.21.15.45:0/2869234356 >> 172.21.15.45:0/2869234356 conn(0x7fc0800b3c40 msgr2=0x7fc0800b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:38.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.889+0000 7fc0851af640 1 -- 172.21.15.45:0/2869234356 shutdown_connections 2024-05-08T05:41:38.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:38.889+0000 7fc0851af640 1 -- 172.21.15.45:0/2869234356 wait complete. 2024-05-08T05:41:39.303 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:41:31.494380+0000', 'modified': '2024-05-08T05:41:36.532197+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4303}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4303': {'gid': 4303, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.63:6837/3653698989', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6835', 'nonce': 3653698989}, {'type': 'v1', 'addr': '172.21.15.63:6837', 'nonce': 3653698989}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4303, 'qdb_cluster': [4303]} 2024-05-08T05:41:39.303 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T05:41:39.304 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T05:41:39.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.415+0000 7f33af493640 1 Processor -- start 2024-05-08T05:41:39.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.415+0000 7f33af493640 1 -- start start 2024-05-08T05:41:39.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.416+0000 7f33af493640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33a80d1970 0x7f33a80d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.416+0000 7f33af493640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33a81630b0 con 0x7f33a815f420 2024-05-08T05:41:39.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.416+0000 7f33af493640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33a80d3e70 con 0x7f33a80d4cc0 2024-05-08T05:41:39.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.416+0000 7f33af493640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f33a80d2330 con 0x7f33a80d1970 2024-05-08T05:41:39.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.416+0000 7f33ad22f640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f33a815f420 0x7f33a80d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47442/0 (socket says 172.21.15.45:47442) 2024-05-08T05:41:39.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.416+0000 7f33ad22f640 1 -- 172.21.15.45:0/3144651125 learned_addr learned my addr 172.21.15.45:0/3144651125 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:39.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.416+0000 7f339ffff640 1 -- 172.21.15.45:0/3144651125 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2876005958 0 0) 0x7f33a4001110 con 0x7f33a815f420 2024-05-08T05:41:39.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.416+0000 7f33aca2e640 1 --2- 172.21.15.45:0/3144651125 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33a80d1970 0x7f33a80d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.416+0000 7f339ffff640 1 -- 172.21.15.45:0/3144651125 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3394003520 con 0x7f33a815f420 2024-05-08T05:41:39.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.416+0000 7f339ffff640 1 -- 172.21.15.45:0/3144651125 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 751960129 0 0) 0x7f3394003520 con 0x7f33a815f420 2024-05-08T05:41:39.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.416+0000 7f339ffff640 1 -- 172.21.15.45:0/3144651125 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33a80d1970 msgr2=0x7f33a80d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:39.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.416+0000 7f339ffff640 1 --2- 172.21.15.45:0/3144651125 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33a80d1970 0x7f33a80d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.416+0000 7f339ffff640 1 -- 172.21.15.45:0/3144651125 >> v1:172.21.15.63:6789/0 conn(0x7f33a80d4cc0 legacy=0x7f33a80d5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:39.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.416+0000 7f339ffff640 1 -- 172.21.15.45:0/3144651125 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a80d2550 con 0x7f33a815f420 2024-05-08T05:41:39.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.416+0000 7f339ffff640 1 -- 172.21.15.45:0/3144651125 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f33a40018c0 con 0x7f33a815f420 2024-05-08T05:41:39.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.416+0000 7f339ffff640 1 -- 172.21.15.45:0/3144651125 >> v1:172.21.15.45:6789/0 conn(0x7f33a815f420 legacy=0x7f33a80d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:39.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.416+0000 7f339ffff640 1 --2- 172.21.15.45:0/3144651125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33940039e0 0x7f3394003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.417+0000 7f339ffff640 1 --2- 172.21.15.45:0/3144651125 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33a80d1970 0x7f3394004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.417+0000 7f339ffff640 1 --2- 172.21.15.45:0/3144651125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3394004d40 0x7f33940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.417+0000 7f339ffff640 1 -- 172.21.15.45:0/3144651125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f33a80d2330 con 0x7f33940039e0 2024-05-08T05:41:39.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.417+0000 7f339ffff640 1 -- 172.21.15.45:0/3144651125 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f339400ba70 con 0x7f3394004d40 2024-05-08T05:41:39.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.417+0000 7f339ffff640 1 -- 172.21.15.45:0/3144651125 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f339400bc70 con 0x7f33a80d1970 2024-05-08T05:41:39.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.417+0000 7f33ad22f640 1 --2- 172.21.15.45:0/3144651125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33940039e0 0x7f3394003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.417+0000 7f33aca2e640 1 --2- 172.21.15.45:0/3144651125 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33a80d1970 0x7f3394004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.417+0000 7f33ada30640 1 --2- 172.21.15.45:0/3144651125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3394004d40 0x7f33940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.417+0000 7f33ad22f640 1 -- 172.21.15.45:0/3144651125 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33a80d1970 msgr2=0x7f3394004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:39.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.417+0000 7f33ad22f640 1 --2- 172.21.15.45:0/3144651125 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33a80d1970 0x7f3394004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.417+0000 7f33ad22f640 1 -- 172.21.15.45:0/3144651125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3394004d40 msgr2=0x7f33940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:39.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.417+0000 7f33ad22f640 1 --2- 172.21.15.45:0/3144651125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3394004d40 0x7f33940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.417+0000 7f33ad22f640 1 -- 172.21.15.45:0/3144651125 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3394003520 con 0x7f33940039e0 2024-05-08T05:41:39.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.417+0000 7f33ada30640 1 --2- 172.21.15.45:0/3144651125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3394004d40 0x7f33940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:39.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.418+0000 7f33ad22f640 1 --2- 172.21.15.45:0/3144651125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33940039e0 0x7f3394003dc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f33a400c490 tx=0x7f3394005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd1d550b2d195b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:39.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.418+0000 7f339ffff640 1 -- 172.21.15.45:0/3144651125 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4036070 con 0x7f33940039e0 2024-05-08T05:41:39.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.418+0000 7f339ffff640 1 -- 172.21.15.45:0/3144651125 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a4005af0 con 0x7f33940039e0 2024-05-08T05:41:39.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.418+0000 7f339ffff640 1 -- 172.21.15.45:0/3144651125 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a400c0e0 con 0x7f33940039e0 2024-05-08T05:41:39.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.418+0000 7f33af493640 1 -- 172.21.15.45:0/3144651125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33940039e0 msgr2=0x7f3394003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:39.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.418+0000 7f33af493640 1 --2- 172.21.15.45:0/3144651125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33940039e0 0x7f3394003dc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f33a400c490 tx=0x7f3394005830 comp rx=0 tx=0).stop 2024-05-08T05:41:39.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.418+0000 7f33ada30640 1 -- 172.21.15.45:0/3144651125 reap_dead start 2024-05-08T05:41:39.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.418+0000 7f33af493640 1 -- 172.21.15.45:0/3144651125 shutdown_connections 2024-05-08T05:41:39.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.418+0000 7f33af493640 1 -- 172.21.15.45:0/3144651125 >> 172.21.15.45:0/3144651125 conn(0x7f33a808eac0 msgr2=0x7f33a809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:39.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.418+0000 7f33af493640 1 -- 172.21.15.45:0/3144651125 shutdown_connections 2024-05-08T05:41:39.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.418+0000 7f33af493640 1 -- 172.21.15.45:0/3144651125 wait complete. 2024-05-08T05:41:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.420+0000 7f33af493640 1 Processor -- start 2024-05-08T05:41:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.420+0000 7f33af493640 1 -- start start 2024-05-08T05:41:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.420+0000 7f33af493640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33940039e0 0x7f33a80b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.420+0000 7f33af493640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33a80b2c90 0x7f33a8153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.420+0000 7f33af493640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33a80cc0e0 0x7f33a80cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.420+0000 7f33af493640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f33a80d3240 con 0x7f33a80cc0e0 2024-05-08T05:41:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.420+0000 7f33af493640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f33a80d2f60 con 0x7f33a80b2c90 2024-05-08T05:41:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.420+0000 7f33af493640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f33a80d2830 con 0x7f33940039e0 2024-05-08T05:41:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.420+0000 7f33ada30640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33a80cc0e0 0x7f33a80cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.420+0000 7f33ada30640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33a80cc0e0 0x7f33a80cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33364/0 (socket says 172.21.15.45:33364) 2024-05-08T05:41:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.420+0000 7f33ada30640 1 -- 172.21.15.45:0/349006632 learned_addr learned my addr 172.21.15.45:0/349006632 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.421+0000 7f33aca2e640 1 --2- 172.21.15.45:0/349006632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33a80b2c90 0x7f33a8153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.421+0000 7f33ad22f640 1 --2- 172.21.15.45:0/349006632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33940039e0 0x7f33a80b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.421+0000 7f33ada30640 1 -- 172.21.15.45:0/349006632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33940039e0 msgr2=0x7f33a80b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.421+0000 7f33ada30640 1 --2- 172.21.15.45:0/349006632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33940039e0 0x7f33a80b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.421+0000 7f33ada30640 1 -- 172.21.15.45:0/349006632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33a80b2c90 msgr2=0x7f33a8153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.421+0000 7f33ada30640 1 --2- 172.21.15.45:0/349006632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33a80b2c90 0x7f33a8153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.421+0000 7f33ada30640 1 -- 172.21.15.45:0/349006632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a80b3120 con 0x7f33a80cc0e0 2024-05-08T05:41:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.421+0000 7f33aca2e640 1 --2- 172.21.15.45:0/349006632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33a80b2c90 0x7f33a8153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.421+0000 7f33ada30640 1 --2- 172.21.15.45:0/349006632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33a80cc0e0 0x7f33a80cc570 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f33a0001520 tx=0x7f33a000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.421+0000 7f339dffb640 1 -- 172.21.15.45:0/349006632 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0003970 con 0x7f33a80cc0e0 2024-05-08T05:41:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.421+0000 7f339dffb640 1 -- 172.21.15.45:0/349006632 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a0018070 con 0x7f33a80cc0e0 2024-05-08T05:41:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.421+0000 7f33af493640 1 -- 172.21.15.45:0/349006632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a80b3400 con 0x7f33a80cc0e0 2024-05-08T05:41:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.421+0000 7f339dffb640 1 -- 172.21.15.45:0/349006632 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0003d50 con 0x7f33a80cc0e0 2024-05-08T05:41:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.421+0000 7f33af493640 1 -- 172.21.15.45:0/349006632 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a80b39c0 con 0x7f33a80cc0e0 2024-05-08T05:41:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.423+0000 7f339dffb640 1 -- 172.21.15.45:0/349006632 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f33a00050d0 con 0x7f33a80cc0e0 2024-05-08T05:41:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.424+0000 7f339dffb640 1 --2- 172.21.15.45:0/349006632 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f338c076300 0x7f338c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.424+0000 7f339dffb640 1 -- 172.21.15.45:0/349006632 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f33a0097790 con 0x7f33a80cc0e0 2024-05-08T05:41:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.424+0000 7f33ad22f640 1 --2- 172.21.15.45:0/349006632 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f338c076300 0x7f338c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.425+0000 7f33b21b5740 1 Processor -- start 2024-05-08T05:41:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.425+0000 7f33b21b5740 1 -- start start 2024-05-08T05:41:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.425+0000 7f33ad22f640 1 --2- 172.21.15.45:0/349006632 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f338c076300 0x7f338c0787c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f33a400c490 tx=0x7f33a4009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.425+0000 7f33b21b5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dec3a9ef50 0x55dec3aa1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.425+0000 7f33b21b5740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dec3a9f330 0x55dec3aa1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.425+0000 7f33b21b5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a71bb0 0x55dec3a753c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.425+0000 7f33b21b5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dec39db8d0 con 0x55dec3a71bb0 2024-05-08T05:41:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.425+0000 7f33b21b5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dec3994c10 con 0x55dec3a9ef50 2024-05-08T05:41:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.425+0000 7f33b21b5740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55dec39d7890 con 0x55dec3a9f330 2024-05-08T05:41:39.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.426+0000 7f33ada30640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a71bb0 0x55dec3a753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.426+0000 7f33ada30640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a71bb0 0x55dec3a753c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33374/0 (socket says 172.21.15.45:33374) 2024-05-08T05:41:39.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.426+0000 7f33ada30640 1 -- 172.21.15.45:0/1589004303 learned_addr learned my addr 172.21.15.45:0/1589004303 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:39.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.426+0000 7f33ad22f640 1 --2- 172.21.15.45:0/1589004303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dec3a9ef50 0x55dec3aa1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.426+0000 7f33aca2e640 1 --2- 172.21.15.45:0/1589004303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dec3a9f330 0x55dec3aa1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.426+0000 7f33ada30640 1 -- 172.21.15.45:0/1589004303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dec3a9f330 msgr2=0x55dec3aa1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:39.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.426+0000 7f33ada30640 1 --2- 172.21.15.45:0/1589004303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dec3a9f330 0x55dec3aa1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.426+0000 7f33ada30640 1 -- 172.21.15.45:0/1589004303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dec3a9ef50 msgr2=0x55dec3aa1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:39.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.426+0000 7f33ada30640 1 --2- 172.21.15.45:0/1589004303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dec3a9ef50 0x55dec3aa1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.426+0000 7f33ada30640 1 -- 172.21.15.45:0/1589004303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dec399c5d0 con 0x55dec3a71bb0 2024-05-08T05:41:39.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.426+0000 7f33aca2e640 1 --2- 172.21.15.45:0/1589004303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dec3a9f330 0x55dec3aa1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:39.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.426+0000 7f33ada30640 1 --2- 172.21.15.45:0/1589004303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a71bb0 0x55dec3a753c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f33a0060350 tx=0x7f33a0060550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bed2e25112cf60d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:39.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.426+0000 7f3392ffd640 1 -- 172.21.15.45:0/1589004303 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0060650 con 0x55dec3a71bb0 2024-05-08T05:41:39.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.426+0000 7f3392ffd640 1 -- 172.21.15.45:0/1589004303 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a0064a00 con 0x55dec3a71bb0 2024-05-08T05:41:39.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.426+0000 7f3392ffd640 1 -- 172.21.15.45:0/1589004303 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0065db0 con 0x55dec3a71bb0 2024-05-08T05:41:39.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.427+0000 7f33b21b5740 1 -- 172.21.15.45:0/1589004303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a71bb0 msgr2=0x55dec3a753c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:39.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.427+0000 7f33b21b5740 1 --2- 172.21.15.45:0/1589004303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a71bb0 0x55dec3a753c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f33a0060350 tx=0x7f33a0060550 comp rx=0 tx=0).stop 2024-05-08T05:41:39.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.427+0000 7f33b21b5740 1 -- 172.21.15.45:0/1589004303 shutdown_connections 2024-05-08T05:41:39.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.427+0000 7f33b21b5740 1 --2- 172.21.15.45:0/1589004303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dec3a9f330 0x55dec3aa1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.427+0000 7f33b21b5740 1 --2- 172.21.15.45:0/1589004303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a71bb0 0x55dec3a753c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.427+0000 7f33b21b5740 1 --2- 172.21.15.45:0/1589004303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dec3a9ef50 0x55dec3aa1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.427+0000 7f33b21b5740 1 -- 172.21.15.45:0/1589004303 >> 172.21.15.45:0/1589004303 conn(0x55dec3a73e20 msgr2=0x55dec3abc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:39.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.427+0000 7f33b21b5740 1 -- 172.21.15.45:0/1589004303 shutdown_connections 2024-05-08T05:41:39.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.427+0000 7f33b21b5740 1 -- 172.21.15.45:0/1589004303 wait complete. 2024-05-08T05:41:39.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.427+0000 7f33b21b5740 1 -- start start 2024-05-08T05:41:39.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.428+0000 7f33b21b5740 1 Processor -- start 2024-05-08T05:41:39.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.428+0000 7f33b21b5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 0x55dec3aa5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.428+0000 7f33b21b5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dec3a9f330 0x55dec3aa37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.428+0000 7f33b21b5740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dec3a946e0 0x55dec3a94b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.428+0000 7f33b21b5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dec3a76fd0 con 0x55dec3a9ef50 2024-05-08T05:41:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.428+0000 7f33b21b5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dec3998760 con 0x55dec3a9f330 2024-05-08T05:41:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.428+0000 7f33b21b5740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55dec3a772b0 con 0x55dec3a946e0 2024-05-08T05:41:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.429+0000 7f33b21b5740 10 client.0 _ll_register_callbacks cb 0x55dec39d1cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:41:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.429+0000 7f33ad22f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 0x55dec3aa5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.429+0000 7f33ad22f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 0x55dec3aa5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33390/0 (socket says 172.21.15.45:33390) 2024-05-08T05:41:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.429+0000 7f33ad22f640 1 -- 172.21.15.45:0/3044459879 learned_addr learned my addr 172.21.15.45:0/3044459879 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.429+0000 7f33aca2e640 1 --2- 172.21.15.45:0/3044459879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dec3a9f330 0x55dec3aa37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.429+0000 7f33ada30640 1 --2- 172.21.15.45:0/3044459879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dec3a946e0 0x55dec3a94b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.429+0000 7f33ad22f640 1 -- 172.21.15.45:0/3044459879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dec3a946e0 msgr2=0x55dec3a94b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.429+0000 7f33ad22f640 1 --2- 172.21.15.45:0/3044459879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dec3a946e0 0x55dec3a94b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.429+0000 7f33ad22f640 1 -- 172.21.15.45:0/3044459879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dec3a9f330 msgr2=0x55dec3aa37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.429+0000 7f33ad22f640 1 --2- 172.21.15.45:0/3044459879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dec3a9f330 0x55dec3aa37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.429+0000 7f33ad22f640 1 -- 172.21.15.45:0/3044459879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dec36ed2d0 con 0x55dec3a9ef50 2024-05-08T05:41:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.429+0000 7f33ad22f640 1 --2- 172.21.15.45:0/3044459879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 0x55dec3aa5560 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f33a4004a60 tx=0x7f33a80d5cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d37eae48f7ab0c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.429+0000 7f3387fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:41:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.429+0000 7f3387fff640 1 -- 172.21.15.45:0/3044459879 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4050060 con 0x55dec3a9ef50 2024-05-08T05:41:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.429+0000 7f33b21b5740 1 -- 172.21.15.45:0/3044459879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33a0064a00 con 0x55dec3a9ef50 2024-05-08T05:41:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.429+0000 7f3387fff640 1 -- 172.21.15.45:0/3044459879 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f33a403e470 con 0x55dec3a9ef50 2024-05-08T05:41:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.429+0000 7f3387fff640 1 -- 172.21.15.45:0/3044459879 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f33a4036040 con 0x55dec3a9ef50 2024-05-08T05:41:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.429+0000 7f33b21b5740 10 client.4380 fetch_fsmap learned FSMap version 6 2024-05-08T05:41:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.429+0000 7f33b21b5740 1 -- 172.21.15.45:0/3044459879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55dec37b76b0 con 0x55dec3a9ef50 2024-05-08T05:41:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.430+0000 7f3387fff640 1 -- 172.21.15.45:0/3044459879 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a0064a00 con 0x55dec3a9ef50 2024-05-08T05:41:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.430+0000 7f3387fff640 1 -- 172.21.15.45:0/3044459879 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 6) ==== 3502+0+0 (secure 0 0 0) 0x7f33a403c040 con 0x55dec3a9ef50 2024-05-08T05:41:39.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.430+0000 7f33b21b5740 10 client.4380 fetch_fsmap finished waiting for FSMap version 6 2024-05-08T05:41:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.430+0000 7f33b21b5740 10 client.4380 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T05:41:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.430+0000 7f33b21b5740 20 client.4380 populate_metadata read hostname 'smithi045' 2024-05-08T05:41:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.430+0000 7f33b21b5740 1 --2- 172.21.15.45:0/3044459879 >> [v2:172.21.15.63:6835/3653698989,v1:172.21.15.63:6837/3653698989] conn(0x55dec3ad02b0 0x55dec3ad0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.430+0000 7f33b21b5740 4 client.4380 mds_command: new command op to 4303 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:41:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.430+0000 7f33b21b5740 1 -- 172.21.15.45:0/3044459879 --> [v2:172.21.15.63:6835/3653698989,v1:172.21.15.63:6837/3653698989] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dec395f760 con 0x55dec3ad02b0 2024-05-08T05:41:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.430+0000 7f33aca2e640 1 --2- 172.21.15.45:0/3044459879 >> [v2:172.21.15.63:6835/3653698989,v1:172.21.15.63:6837/3653698989] conn(0x55dec3ad02b0 0x55dec3ad0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.430+0000 7f3387fff640 10 client.4380 ms_handle_connect on v2:172.21.15.63:6835/3653698989 2024-05-08T05:41:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.430+0000 7f33aca2e640 1 --2- 172.21.15.45:0/3044459879 >> [v2:172.21.15.63:6835/3653698989,v1:172.21.15.63:6837/3653698989] conn(0x55dec3ad02b0 0x55dec3ad0690 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d55076a7d621136 server_cookie=1074a23c70bc2d85 in_seq=0 out_seq=0 2024-05-08T05:41:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.431+0000 7f3387fff640 1 -- 172.21.15.45:0/3044459879 <== mds.0 v2:172.21.15.63:6835/3653698989 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dec395f760 con 0x55dec3ad02b0 2024-05-08T05:41:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.431+0000 7f3387fff640 10 client.4380 handle_command_reply: tid=0 2024-05-08T05:41:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.431+0000 7f33b21b5740 1 client.4380 shutdown 2024-05-08T05:41:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.431+0000 7f33b21b5740 1 -- 172.21.15.45:0/3044459879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 msgr2=0x55dec3aa5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.431+0000 7f33b21b5740 1 --2- 172.21.15.45:0/3044459879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 0x55dec3aa5560 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f33a4004a60 tx=0x7f33a80d5cd0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.431+0000 7f33b21b5740 1 -- 172.21.15.45:0/3044459879 shutdown_connections 2024-05-08T05:41:39.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.431+0000 7f33b21b5740 1 --2- 172.21.15.45:0/3044459879 >> [v2:172.21.15.63:6835/3653698989,v1:172.21.15.63:6837/3653698989] conn(0x55dec3ad02b0 0x55dec3ad0690 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.431+0000 7f33b21b5740 1 --2- 172.21.15.45:0/3044459879 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dec3a946e0 0x55dec3a94b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.431+0000 7f33b21b5740 1 --2- 172.21.15.45:0/3044459879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dec3a9f330 0x55dec3aa37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.431+0000 7f3387fff640 0 client.4380 ms_handle_reset on v2:172.21.15.63:6835/3653698989 2024-05-08T05:41:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.431+0000 7f33b21b5740 1 --2- 172.21.15.45:0/3044459879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 0x55dec3aa5560 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.431+0000 7f33b21b5740 1 -- 172.21.15.45:0/3044459879 >> 172.21.15.45:0/3044459879 conn(0x55dec3a71bb0 msgr2=0x55dec3aa1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.431+0000 7f33b21b5740 1 -- 172.21.15.45:0/3044459879 shutdown_connections 2024-05-08T05:41:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.431+0000 7f33b21b5740 1 -- 172.21.15.45:0/3044459879 wait complete. 2024-05-08T05:41:39.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.432+0000 7f33b21b5740 20 client.4380 trim_cache size 0 max 16384 2024-05-08T05:41:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.450+0000 7f33b21b5740 1 Processor -- start 2024-05-08T05:41:39.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.450+0000 7f33b21b5740 1 -- start start 2024-05-08T05:41:39.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.451+0000 7f33b21b5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dec3a946e0 0x55dec3abc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.451+0000 7f33b21b5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 0x55dec3acf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.451+0000 7f33b21b5740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dec3a9f330 0x55dec3acf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.451+0000 7f33b21b5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f33a82035b0 con 0x55dec3a9ef50 2024-05-08T05:41:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.451+0000 7f33b21b5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f33a82025b0 con 0x55dec3a946e0 2024-05-08T05:41:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.451+0000 7f33b21b5740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f33a8201910 con 0x55dec3a9f330 2024-05-08T05:41:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.451+0000 7f33aca2e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 0x55dec3acf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.451+0000 7f33aca2e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 0x55dec3acf0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33398/0 (socket says 172.21.15.45:33398) 2024-05-08T05:41:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.451+0000 7f33aca2e640 1 -- 172.21.15.45:0/2762903871 learned_addr learned my addr 172.21.15.45:0/2762903871 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.451+0000 7f33aca2e640 1 -- 172.21.15.45:0/2762903871 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dec3a9f330 msgr2=0x55dec3acf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:39.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.451+0000 7f33aca2e640 1 --2- 172.21.15.45:0/2762903871 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dec3a9f330 0x55dec3acf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.451+0000 7f33aca2e640 1 -- 172.21.15.45:0/2762903871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dec3a946e0 msgr2=0x55dec3abc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:39.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.451+0000 7f33aca2e640 1 --2- 172.21.15.45:0/2762903871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dec3a946e0 0x55dec3abc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.451+0000 7f33aca2e640 1 -- 172.21.15.45:0/2762903871 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dec3aa4f70 con 0x55dec3a9ef50 2024-05-08T05:41:39.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.451+0000 7f33aca2e640 1 --2- 172.21.15.45:0/2762903871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 0x55dec3acf0d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f33a81578a0 tx=0x7f3398009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c880722db2324c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:39.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.451+0000 7f33927fc640 1 -- 172.21.15.45:0/2762903871 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398019110 con 0x55dec3a9ef50 2024-05-08T05:41:39.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.451+0000 7f33927fc640 1 -- 172.21.15.45:0/2762903871 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33980192b0 con 0x55dec3a9ef50 2024-05-08T05:41:39.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.451+0000 7f33927fc640 1 -- 172.21.15.45:0/2762903871 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398004270 con 0x55dec3a9ef50 2024-05-08T05:41:39.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.452+0000 7f33b21b5740 1 -- 172.21.15.45:0/2762903871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 msgr2=0x55dec3acf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:39.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.452+0000 7f33b21b5740 1 --2- 172.21.15.45:0/2762903871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 0x55dec3acf0d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f33a81578a0 tx=0x7f3398009190 comp rx=0 tx=0).stop 2024-05-08T05:41:39.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.452+0000 7f33b21b5740 1 -- 172.21.15.45:0/2762903871 shutdown_connections 2024-05-08T05:41:39.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.452+0000 7f33b21b5740 1 --2- 172.21.15.45:0/2762903871 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dec3a9f330 0x55dec3acf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.452+0000 7f33b21b5740 1 --2- 172.21.15.45:0/2762903871 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 0x55dec3acf0d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.452+0000 7f33b21b5740 1 --2- 172.21.15.45:0/2762903871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dec3a946e0 0x55dec3abc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.452+0000 7f33b21b5740 1 -- 172.21.15.45:0/2762903871 >> 172.21.15.45:0/2762903871 conn(0x55dec3a71bb0 msgr2=0x55dec3aa1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:39.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.452+0000 7f33b21b5740 1 -- 172.21.15.45:0/2762903871 shutdown_connections 2024-05-08T05:41:39.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.452+0000 7f33b21b5740 1 -- 172.21.15.45:0/2762903871 wait complete. 2024-05-08T05:41:39.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.452+0000 7f33b21b5740 1 -- start start 2024-05-08T05:41:39.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.453+0000 7f33b21b5740 1 Processor -- start 2024-05-08T05:41:39.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.453+0000 7f33b21b5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dec3a946e0 0x55dec3ab20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.453+0000 7f33b21b5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 0x55dec3ab2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.453+0000 7f33b21b5740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dec3a9f330 0x55dec3adf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.453+0000 7f33b21b5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dec3aa5aa0 con 0x55dec3a9ef50 2024-05-08T05:41:39.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f33b21b5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f33a8201f60 con 0x55dec3a946e0 2024-05-08T05:41:39.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f33b21b5740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55dec3abec60 con 0x55dec3a9f330 2024-05-08T05:41:39.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f33b21b5740 10 client.0 _ll_register_callbacks cb 0x7f33940036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:41:39.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f33aca2e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 0x55dec3ab2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f33aca2e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 0x55dec3ab2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33400/0 (socket says 172.21.15.45:33400) 2024-05-08T05:41:39.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f33aca2e640 1 -- 172.21.15.45:0/658804375 learned_addr learned my addr 172.21.15.45:0/658804375 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:39.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f33ad22f640 1 --2- 172.21.15.45:0/658804375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dec3a946e0 0x55dec3ab20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f33ada30640 1 --2- 172.21.15.45:0/658804375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dec3a9f330 0x55dec3adf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f33aca2e640 1 -- 172.21.15.45:0/658804375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dec3a9f330 msgr2=0x55dec3adf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:39.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f33aca2e640 1 --2- 172.21.15.45:0/658804375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dec3a9f330 0x55dec3adf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f33aca2e640 1 -- 172.21.15.45:0/658804375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dec3a946e0 msgr2=0x55dec3ab20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:39.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f33aca2e640 1 --2- 172.21.15.45:0/658804375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dec3a946e0 0x55dec3ab20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f33aca2e640 1 -- 172.21.15.45:0/658804375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dec3ad0690 con 0x55dec3a9ef50 2024-05-08T05:41:39.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f33aca2e640 1 --2- 172.21.15.45:0/658804375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 0x55dec3ab2630 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f33a815f8f0 tx=0x7f33980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f9cbdf458a0a43 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:39.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f3387fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:41:39.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f3387fff640 1 -- 172.21.15.45:0/658804375 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398004020 con 0x55dec3a9ef50 2024-05-08T05:41:39.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f3387fff640 1 -- 172.21.15.45:0/658804375 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3398036de0 con 0x55dec3a9ef50 2024-05-08T05:41:39.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f33b21b5740 1 -- 172.21.15.45:0/658804375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33980192b0 con 0x55dec3a9ef50 2024-05-08T05:41:39.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f3387fff640 1 -- 172.21.15.45:0/658804375 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f33980041c0 con 0x55dec3a9ef50 2024-05-08T05:41:39.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f33b21b5740 10 client.4386 fetch_fsmap learned FSMap version 6 2024-05-08T05:41:39.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.454+0000 7f33b21b5740 1 -- 172.21.15.45:0/658804375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55dec3b3b1c0 con 0x55dec3a9ef50 2024-05-08T05:41:39.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.455+0000 7f3387fff640 1 -- 172.21.15.45:0/658804375 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33980192b0 con 0x55dec3a9ef50 2024-05-08T05:41:39.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.455+0000 7f3387fff640 1 -- 172.21.15.45:0/658804375 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 6) ==== 3502+0+0 (secure 0 0 0) 0x7f3398034070 con 0x55dec3a9ef50 2024-05-08T05:41:39.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.455+0000 7f33b21b5740 10 client.4386 fetch_fsmap finished waiting for FSMap version 6 2024-05-08T05:41:39.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.455+0000 7f33b21b5740 10 client.4386 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T05:41:39.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.455+0000 7f33b21b5740 20 client.4386 populate_metadata read hostname 'smithi045' 2024-05-08T05:41:39.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.455+0000 7f33b21b5740 1 --2- 172.21.15.45:0/658804375 >> [v2:172.21.15.63:6835/3653698989,v1:172.21.15.63:6837/3653698989] conn(0x55dec3adfa80 0x55dec3abd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.455+0000 7f33b21b5740 4 client.4386 mds_command: new command op to 4303 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:41:39.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.455+0000 7f33b21b5740 1 -- 172.21.15.45:0/658804375 --> [v2:172.21.15.63:6835/3653698989,v1:172.21.15.63:6837/3653698989] -- command(tid 0: {"prefix": "status"}) -- 0x55dec39e94e0 con 0x55dec3adfa80 2024-05-08T05:41:39.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.455+0000 7f33ad22f640 1 --2- 172.21.15.45:0/658804375 >> [v2:172.21.15.63:6835/3653698989,v1:172.21.15.63:6837/3653698989] conn(0x55dec3adfa80 0x55dec3abd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.455+0000 7f3387fff640 10 client.4386 ms_handle_connect on v2:172.21.15.63:6835/3653698989 2024-05-08T05:41:39.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.455+0000 7f33ad22f640 1 --2- 172.21.15.45:0/658804375 >> [v2:172.21.15.63:6835/3653698989,v1:172.21.15.63:6837/3653698989] conn(0x55dec3adfa80 0x55dec3abd890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1510474c6b3118df server_cookie=b5e898561b32d2a4 in_seq=0 out_seq=0 2024-05-08T05:41:39.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.455+0000 7f3387fff640 1 -- 172.21.15.45:0/658804375 <== mds.0 v2:172.21.15.63:6835/3653698989 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55dec39e94e0 con 0x55dec3adfa80 2024-05-08T05:41:39.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.455+0000 7f3387fff640 10 client.4386 handle_command_reply: tid=0 2024-05-08T05:41:39.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.455+0000 7f33b21b5740 1 client.4386 shutdown 2024-05-08T05:41:39.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.455+0000 7f33b21b5740 1 -- 172.21.15.45:0/658804375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 msgr2=0x55dec3ab2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:39.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.455+0000 7f33b21b5740 1 --2- 172.21.15.45:0/658804375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 0x55dec3ab2630 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f33a815f8f0 tx=0x7f33980046c0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.455+0000 7f33b21b5740 1 -- 172.21.15.45:0/658804375 shutdown_connections 2024-05-08T05:41:39.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f33b21b5740 1 --2- 172.21.15.45:0/658804375 >> [v2:172.21.15.63:6835/3653698989,v1:172.21.15.63:6837/3653698989] conn(0x55dec3adfa80 0x55dec3abd890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f33b21b5740 1 --2- 172.21.15.45:0/658804375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dec3a9f330 0x55dec3adf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f3387fff640 0 client.4386 ms_handle_reset on v2:172.21.15.63:6835/3653698989 2024-05-08T05:41:39.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f33b21b5740 1 --2- 172.21.15.45:0/658804375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dec3a9ef50 0x55dec3ab2630 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f33b21b5740 1 --2- 172.21.15.45:0/658804375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dec3a946e0 0x55dec3ab20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f33b21b5740 1 -- 172.21.15.45:0/658804375 >> 172.21.15.45:0/658804375 conn(0x55dec3a71bb0 msgr2=0x55dec3a9e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:39.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f33b21b5740 1 -- 172.21.15.45:0/658804375 shutdown_connections 2024-05-08T05:41:39.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f33b21b5740 1 -- 172.21.15.45:0/658804375 wait complete. 2024-05-08T05:41:39.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f33b21b5740 20 client.4386 trim_cache size 0 max 16384 2024-05-08T05:41:39.466 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:41:39.466 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "e9e1170a-fd27-4625-97a9-5d5f104d13a2", 2024-05-08T05:41:39.466 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-05-08T05:41:39.466 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 4303, 2024-05-08T05:41:39.466 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-05-08T05:41:39.466 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-05-08T05:41:39.466 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-05-08T05:41:39.467 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 7.3482382770000001, 2024-05-08T05:41:39.467 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 6, 2024-05-08T05:41:39.467 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 24, 2024-05-08T05:41:39.467 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 23, 2024-05-08T05:41:39.467 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 13.358931674999999 2024-05-08T05:41:39.467 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:41:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f33af493640 1 -- 172.21.15.45:0/349006632 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f338c076300 msgr2=0x7f338c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f33af493640 1 --2- 172.21.15.45:0/349006632 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f338c076300 0x7f338c0787c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f33a400c490 tx=0x7f33a4009030 comp rx=0 tx=0).stop 2024-05-08T05:41:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f33af493640 1 -- 172.21.15.45:0/349006632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33a80cc0e0 msgr2=0x7f33a80cc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f33af493640 1 --2- 172.21.15.45:0/349006632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33a80cc0e0 0x7f33a80cc570 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f33a0001520 tx=0x7f33a000eb90 comp rx=0 tx=0).stop 2024-05-08T05:41:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f33af493640 1 -- 172.21.15.45:0/349006632 shutdown_connections 2024-05-08T05:41:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f33af493640 1 --2- 172.21.15.45:0/349006632 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f338c076300 0x7f338c0787c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f33af493640 1 --2- 172.21.15.45:0/349006632 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33a80cc0e0 0x7f33a80cc570 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f33af493640 1 --2- 172.21.15.45:0/349006632 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33a80b2c90 0x7f33a8153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f33af493640 1 --2- 172.21.15.45:0/349006632 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33940039e0 0x7f33a80b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f33af493640 1 -- 172.21.15.45:0/349006632 >> 172.21.15.45:0/349006632 conn(0x7f33a808eac0 msgr2=0x7f33a80b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f33af493640 1 -- 172.21.15.45:0/349006632 shutdown_connections 2024-05-08T05:41:39.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.456+0000 7f33af493640 1 -- 172.21.15.45:0/349006632 wait complete. 2024-05-08T05:41:39.469 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T05:41:39.470 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:41:39.470 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T05:41:39.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.573+0000 7fc4e9262640 1 Processor -- start 2024-05-08T05:41:39.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.573+0000 7fc4e9262640 1 -- start start 2024-05-08T05:41:39.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.574+0000 7fc4e9262640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4e40d3cd0 0x7fc4e40cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.574+0000 7fc4e9262640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc4e40c9ee0 0x7fc4e40d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.574+0000 7fc4e9262640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4e40d3330 con 0x7fc4e41629a0 2024-05-08T05:41:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.574+0000 7fc4e9262640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc4e40d2170 con 0x7fc4e40d3cd0 2024-05-08T05:41:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.574+0000 7fc4e9262640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc4e40d2450 con 0x7fc4e40c9ee0 2024-05-08T05:41:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.574+0000 7fc4e2ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc4e41629a0 0x7fc4e40d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47448/0 (socket says 172.21.15.45:47448) 2024-05-08T05:41:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.574+0000 7fc4e27fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4e40d3cd0 0x7fc4e40cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.574+0000 7fc4e2ffd640 1 -- 172.21.15.45:0/2449822492 learned_addr learned my addr 172.21.15.45:0/2449822492 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.574+0000 7fc4e37fe640 1 --2- 172.21.15.45:0/2449822492 >> v2:172.21.15.63:3300/0 conn(0x7fc4e40c9ee0 0x7fc4e40d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.574+0000 7fc4e27fc640 1 -- 172.21.15.45:0/2449822492 >> v2:172.21.15.63:3300/0 conn(0x7fc4e40c9ee0 msgr2=0x7fc4e40d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.574+0000 7fc4e27fc640 1 --2- 172.21.15.45:0/2449822492 >> v2:172.21.15.63:3300/0 conn(0x7fc4e40c9ee0 0x7fc4e40d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.574+0000 7fc4e27fc640 1 -- 172.21.15.45:0/2449822492 >> v1:172.21.15.45:6789/0 conn(0x7fc4e41629a0 legacy=0x7fc4e40d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.574+0000 7fc4e27fc640 1 -- 172.21.15.45:0/2449822492 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e40d2730 con 0x7fc4e40d3cd0 2024-05-08T05:41:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.574+0000 7fc4e37fe640 1 --2- 172.21.15.45:0/2449822492 >> v2:172.21.15.63:3300/0 conn(0x7fc4e40c9ee0 0x7fc4e40d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.575+0000 7fc4e27fc640 1 --2- 172.21.15.45:0/2449822492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e40d3cd0 0x7fc4e40cfca0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc4e408e4c0 tx=0x7fc4cc02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911aebb5b8686359 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.575+0000 7fc4e1ffb640 1 -- 172.21.15.45:0/2449822492 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc037070 con 0x7fc4e40d3cd0 2024-05-08T05:41:39.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.575+0000 7fc4e1ffb640 1 -- 172.21.15.45:0/2449822492 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4cc02add0 con 0x7fc4e40d3cd0 2024-05-08T05:41:39.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.575+0000 7fc4e1ffb640 1 -- 172.21.15.45:0/2449822492 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc032660 con 0x7fc4e40d3cd0 2024-05-08T05:41:39.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.576+0000 7fc4e9262640 1 -- 172.21.15.45:0/2449822492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e40d3cd0 msgr2=0x7fc4e40cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:39.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.576+0000 7fc4e9262640 1 --2- 172.21.15.45:0/2449822492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e40d3cd0 0x7fc4e40cfca0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc4e408e4c0 tx=0x7fc4cc02a080 comp rx=0 tx=0).stop 2024-05-08T05:41:39.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.576+0000 7fc4e9262640 1 -- 172.21.15.45:0/2449822492 shutdown_connections 2024-05-08T05:41:39.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.576+0000 7fc4e9262640 1 --2- 172.21.15.45:0/2449822492 >> v2:172.21.15.63:3300/0 conn(0x7fc4e40c9ee0 0x7fc4e40d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.576+0000 7fc4e9262640 1 --2- 172.21.15.45:0/2449822492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e40d3cd0 0x7fc4e40cfca0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.576+0000 7fc4e9262640 1 -- 172.21.15.45:0/2449822492 >> 172.21.15.45:0/2449822492 conn(0x7fc4e40d4fe0 msgr2=0x7fc4e40d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:39.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.576+0000 7fc4e9262640 1 -- 172.21.15.45:0/2449822492 shutdown_connections 2024-05-08T05:41:39.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.576+0000 7fc4e9262640 1 -- 172.21.15.45:0/2449822492 wait complete. 2024-05-08T05:41:39.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.577+0000 7fc4e9262640 1 Processor -- start 2024-05-08T05:41:39.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.577+0000 7fc4e9262640 1 -- start start 2024-05-08T05:41:39.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4e9262640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4e40c9ee0 0x7fc4e4156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4e9262640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4e40d3cd0 0x7fc4e40b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4e9262640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e41629a0 0x7fc4e40b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4e9262640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e40cf440 con 0x7fc4e41629a0 2024-05-08T05:41:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4e9262640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e40cf2c0 con 0x7fc4e40d3cd0 2024-05-08T05:41:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4e9262640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e40d2dc0 con 0x7fc4e40c9ee0 2024-05-08T05:41:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4e37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e41629a0 0x7fc4e40b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4e37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e41629a0 0x7fc4e40b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:33424/0 (socket says 172.21.15.45:33424) 2024-05-08T05:41:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4e37fe640 1 -- 172.21.15.45:0/737567514 learned_addr learned my addr 172.21.15.45:0/737567514 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4e27fc640 1 --2- 172.21.15.45:0/737567514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4e40d3cd0 0x7fc4e40b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4e2ffd640 1 --2- 172.21.15.45:0/737567514 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4e40c9ee0 0x7fc4e4156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4e37fe640 1 -- 172.21.15.45:0/737567514 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4e40c9ee0 msgr2=0x7fc4e4156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4e37fe640 1 --2- 172.21.15.45:0/737567514 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4e40c9ee0 0x7fc4e4156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4e37fe640 1 -- 172.21.15.45:0/737567514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4e40d3cd0 msgr2=0x7fc4e40b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4e37fe640 1 --2- 172.21.15.45:0/737567514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4e40d3cd0 0x7fc4e40b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4e37fe640 1 -- 172.21.15.45:0/737567514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e40b2560 con 0x7fc4e41629a0 2024-05-08T05:41:39.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4e2ffd640 1 --2- 172.21.15.45:0/737567514 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4e40c9ee0 0x7fc4e4156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:39.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4e37fe640 1 --2- 172.21.15.45:0/737567514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e41629a0 0x7fc4e40b1bf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400c7f0 tx=0x7fc4d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:39.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4db7fe640 1 -- 172.21.15.45:0/737567514 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4015070 con 0x7fc4e41629a0 2024-05-08T05:41:39.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4db7fe640 1 -- 172.21.15.45:0/737567514 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d4007820 con 0x7fc4e41629a0 2024-05-08T05:41:39.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.578+0000 7fc4e9262640 1 -- 172.21.15.45:0/737567514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e40b2840 con 0x7fc4e41629a0 2024-05-08T05:41:39.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.579+0000 7fc4db7fe640 1 -- 172.21.15.45:0/737567514 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4007c50 con 0x7fc4e41629a0 2024-05-08T05:41:39.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.579+0000 7fc4e9262640 1 -- 172.21.15.45:0/737567514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e40b30e0 con 0x7fc4e41629a0 2024-05-08T05:41:39.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.580+0000 7fc4e9262640 1 -- 172.21.15.45:0/737567514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e40b2c30 con 0x7fc4e41629a0 2024-05-08T05:41:39.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.580+0000 7fc4db7fe640 1 -- 172.21.15.45:0/737567514 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc4d4018070 con 0x7fc4e41629a0 2024-05-08T05:41:39.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.581+0000 7fc4db7fe640 1 --2- 172.21.15.45:0/737567514 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc4c40762d0 0x7fc4c4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:39.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.581+0000 7fc4db7fe640 1 -- 172.21.15.45:0/737567514 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc4d4097b20 con 0x7fc4e41629a0 2024-05-08T05:41:39.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.581+0000 7fc4e2ffd640 1 --2- 172.21.15.45:0/737567514 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc4c40762d0 0x7fc4c4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:39.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.581+0000 7fc4e2ffd640 1 --2- 172.21.15.45:0/737567514 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc4c40762d0 0x7fc4c4078790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0012e0 tx=0x7fc4dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:39.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.585+0000 7fc4db7fe640 1 -- 172.21.15.45:0/737567514 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc4d4061130 con 0x7fc4e41629a0 2024-05-08T05:41:39.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.816+0000 7fc4e9262640 1 -- 172.21.15.45:0/737567514 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc4e40693f0 con 0x7fc4e41629a0 2024-05-08T05:41:39.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.816+0000 7fc4db7fe640 1 -- 172.21.15.45:0/737567514 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+88 (secure 0 0 0) 0x7fc4d4060a30 con 0x7fc4e41629a0 2024-05-08T05:41:39.817 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T05:41:39.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.818+0000 7fc4e9262640 1 -- 172.21.15.45:0/737567514 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc4c40762d0 msgr2=0x7fc4c4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:39.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.818+0000 7fc4e9262640 1 --2- 172.21.15.45:0/737567514 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc4c40762d0 0x7fc4c4078790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0012e0 tx=0x7fc4dc009040 comp rx=0 tx=0).stop 2024-05-08T05:41:39.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.818+0000 7fc4e9262640 1 -- 172.21.15.45:0/737567514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e41629a0 msgr2=0x7fc4e40b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:39.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.818+0000 7fc4e9262640 1 --2- 172.21.15.45:0/737567514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e41629a0 0x7fc4e40b1bf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400c7f0 tx=0x7fc4d400ccb0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.818+0000 7fc4e9262640 1 -- 172.21.15.45:0/737567514 shutdown_connections 2024-05-08T05:41:39.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.818+0000 7fc4e9262640 1 --2- 172.21.15.45:0/737567514 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc4c40762d0 0x7fc4c4078790 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.818+0000 7fc4e9262640 1 --2- 172.21.15.45:0/737567514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4e41629a0 0x7fc4e40b1bf0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.818+0000 7fc4e9262640 1 --2- 172.21.15.45:0/737567514 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4e40d3cd0 0x7fc4e40b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.818+0000 7fc4e9262640 1 --2- 172.21.15.45:0/737567514 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4e40c9ee0 0x7fc4e4156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:39.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.818+0000 7fc4e9262640 1 -- 172.21.15.45:0/737567514 >> 172.21.15.45:0/737567514 conn(0x7fc4e40d4fe0 msgr2=0x7fc4e40cffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:39.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.818+0000 7fc4e9262640 1 -- 172.21.15.45:0/737567514 shutdown_connections 2024-05-08T05:41:39.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39.818+0000 7fc4e9262640 1 -- 172.21.15.45:0/737567514 wait complete. 2024-05-08T05:41:39.829 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T05:41:39.829 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:41:39.829 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-05-08T05:41:39.829 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-08T05:41:39.829 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:41:39.830 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T05:41:39.830 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T05:41:39.830 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:41:39.830 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-08T05:41:39.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:39.830 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-05-08T05:41:39.846 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:41:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:41:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:41:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-05-08T05:41:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:39.847 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:41:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:41:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-05-08T05:41:39.847 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:41:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:41:39.847 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-05-08T05:41:39.848 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:41:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-05-08T05:41:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-05-08T05:41:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:41:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42725sec preferred_lft 42725sec 2024-05-08T05:41:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-05-08T05:41:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:39.848 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:41:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-05-08T05:41:39.848 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-05-08T05:41:39.849 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T05:41:39.849 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:41:39.849 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:41:39.849 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2024-05-08T05:41:39.849 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2024-05-08T05:41:39.849 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2024-05-08T05:41:39.849 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T05:41:39.850 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:41:39.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:40.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:40.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:40.045 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:41:40.071 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:41:40.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:40.073 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-05-08T05:41:40.146 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-05-08T05:41:40.146 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:41:40.146 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:41:40.146 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:41:40.146 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:41:40.147 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:41:40.147 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:41:40.147 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:41:40.147 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:41:40.147 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:41:40.147 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:41:40.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:40.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:40.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:40.456 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:41:40.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:40.472 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2024-05-08T05:41:40.529 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:41:40.529 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:41:40.529 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T05:41:40.529 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-08T05:41:40.529 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:41:40.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:40.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:40.651 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-08T05:41:40.651 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:41:40.651 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:41:40.651 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-08T05:41:40.651 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T05:41:40.651 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-08T05:41:40.652 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-08T05:41:40.652 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-08T05:41:40.652 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:41:40.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:40.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:40.785 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:41:40.785 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:41:40.785 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-05-08T05:41:40.785 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T05:41:40.785 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:41:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:40.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:40.868 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-08T05:41:40.868 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:41:40.868 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:41:40.884 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-08T05:41:40.884 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:41:40.884 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:41:40.940 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2024-05-08T05:41:41.007 DEBUG:teuthology.orchestra.run.smithi045:> cat /proc/self/mounts | awk '{print $2}' 2024-05-08T05:41:41.064 INFO:teuthology.orchestra.run.smithi045.stdout:/proc 2024-05-08T05:41:41.064 INFO:teuthology.orchestra.run.smithi045.stdout:/sys 2024-05-08T05:41:41.064 INFO:teuthology.orchestra.run.smithi045.stdout:/dev 2024-05-08T05:41:41.064 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/security 2024-05-08T05:41:41.064 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/shm 2024-05-08T05:41:41.064 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/pts 2024-05-08T05:41:41.064 INFO:teuthology.orchestra.run.smithi045.stdout:/run 2024-05-08T05:41:41.064 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/cgroup 2024-05-08T05:41:41.064 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/pstore 2024-05-08T05:41:41.064 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/bpf 2024-05-08T05:41:41.064 INFO:teuthology.orchestra.run.smithi045.stdout:/ 2024-05-08T05:41:41.064 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/selinux 2024-05-08T05:41:41.064 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-05-08T05:41:41.065 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/mqueue 2024-05-08T05:41:41.065 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/hugepages 2024-05-08T05:41:41.065 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/debug 2024-05-08T05:41:41.065 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/tracing 2024-05-08T05:41:41.065 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/fuse/connections 2024-05-08T05:41:41.065 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/config 2024-05-08T05:41:41.065 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-sysctl.service 2024-05-08T05:41:41.065 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-08T05:41:41.065 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-08T05:41:41.065 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/nfs/rpc_pipefs 2024-05-08T05:41:41.065 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/1000 2024-05-08T05:41:41.065 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph 2024-05-08T05:41:41.065 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph/osd/ceph-0 2024-05-08T05:41:41.065 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph/osd/ceph-1 2024-05-08T05:41:41.065 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph/osd/ceph-2 2024-05-08T05:41:41.066 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph/osd/ceph-3 2024-05-08T05:41:41.066 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns 2024-05-08T05:41:41.066 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T05:41:41.066 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T05:41:41.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:41.066 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2024-05-08T05:41:41.120 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T05:41:41.120 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-08T05:41:41.162 DEBUG:teuthology.orchestra.run.smithi045:> sudo modprobe fuse 2024-05-08T05:41:41.188 DEBUG:teuthology.orchestra.run.smithi045:> cat /proc/self/mounts | awk '{print $2}' 2024-05-08T05:41:41.246 INFO:teuthology.orchestra.run.smithi045.stdout:/proc 2024-05-08T05:41:41.247 INFO:teuthology.orchestra.run.smithi045.stdout:/sys 2024-05-08T05:41:41.247 INFO:teuthology.orchestra.run.smithi045.stdout:/dev 2024-05-08T05:41:41.247 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/security 2024-05-08T05:41:41.247 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/shm 2024-05-08T05:41:41.247 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/pts 2024-05-08T05:41:41.247 INFO:teuthology.orchestra.run.smithi045.stdout:/run 2024-05-08T05:41:41.247 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/cgroup 2024-05-08T05:41:41.247 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/pstore 2024-05-08T05:41:41.247 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/bpf 2024-05-08T05:41:41.247 INFO:teuthology.orchestra.run.smithi045.stdout:/ 2024-05-08T05:41:41.247 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/selinux 2024-05-08T05:41:41.247 INFO:teuthology.orchestra.run.smithi045.stdout:/proc/sys/fs/binfmt_misc 2024-05-08T05:41:41.247 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/mqueue 2024-05-08T05:41:41.247 INFO:teuthology.orchestra.run.smithi045.stdout:/dev/hugepages 2024-05-08T05:41:41.248 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/debug 2024-05-08T05:41:41.248 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/tracing 2024-05-08T05:41:41.248 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/fs/fuse/connections 2024-05-08T05:41:41.248 INFO:teuthology.orchestra.run.smithi045.stdout:/sys/kernel/config 2024-05-08T05:41:41.248 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-sysctl.service 2024-05-08T05:41:41.248 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-08T05:41:41.248 INFO:teuthology.orchestra.run.smithi045.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-08T05:41:41.248 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/nfs/rpc_pipefs 2024-05-08T05:41:41.248 INFO:teuthology.orchestra.run.smithi045.stdout:/run/user/1000 2024-05-08T05:41:41.248 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph 2024-05-08T05:41:41.248 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph/osd/ceph-0 2024-05-08T05:41:41.248 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph/osd/ceph-1 2024-05-08T05:41:41.248 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph/osd/ceph-2 2024-05-08T05:41:41.248 INFO:teuthology.orchestra.run.smithi045.stdout:/var/lib/ceph/osd/ceph-3 2024-05-08T05:41:41.249 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns 2024-05-08T05:41:41.249 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T05:41:41.249 INFO:teuthology.orchestra.run.smithi045.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T05:41:41.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:41.249 DEBUG:teuthology.orchestra.run.smithi045:> ls /sys/fs/fuse/connections 2024-05-08T05:41:41.273 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:2024-05-08T05:41:41.272+0000 7f314afc8480 -1 init, newargv = 0x56363ceecd10 newargc=15 2024-05-08T05:41:41.273 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[42319]: starting ceph client 2024-05-08T05:41:41.280 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[42319]: starting fuse 2024-05-08T05:41:41.289 INFO:teuthology.orchestra.run.smithi045.stdout:40 2024-05-08T05:41:41.290 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-05-08T05:41:41.290 DEBUG:teuthology.orchestra.run.smithi045:> sudo stdin-killer -- python3 -c ' 2024-05-08T05:41:41.290 DEBUG:teuthology.orchestra.run.smithi045:> import glob 2024-05-08T05:41:41.290 DEBUG:teuthology.orchestra.run.smithi045:> import re 2024-05-08T05:41:41.290 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:41:41.290 DEBUG:teuthology.orchestra.run.smithi045:> import subprocess 2024-05-08T05:41:41.290 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:41:41.290 DEBUG:teuthology.orchestra.run.smithi045:> def _find_admin_socket(client_name): 2024-05-08T05:41:41.290 DEBUG:teuthology.orchestra.run.smithi045:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-08T05:41:41.290 DEBUG:teuthology.orchestra.run.smithi045:> files = glob.glob(asok_path) 2024-05-08T05:41:41.291 DEBUG:teuthology.orchestra.run.smithi045:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-08T05:41:41.291 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:41:41.291 DEBUG:teuthology.orchestra.run.smithi045:> # Given a non-glob path, it better be there 2024-05-08T05:41:41.291 DEBUG:teuthology.orchestra.run.smithi045:> if "*" not in asok_path: 2024-05-08T05:41:41.291 DEBUG:teuthology.orchestra.run.smithi045:> assert(len(files) == 1) 2024-05-08T05:41:41.291 DEBUG:teuthology.orchestra.run.smithi045:> return files[0] 2024-05-08T05:41:41.291 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:41:41.291 DEBUG:teuthology.orchestra.run.smithi045:> for f in files: 2024-05-08T05:41:41.291 DEBUG:teuthology.orchestra.run.smithi045:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T05:41:41.291 DEBUG:teuthology.orchestra.run.smithi045:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T05:41:41.291 DEBUG:teuthology.orchestra.run.smithi045:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T05:41:41.291 DEBUG:teuthology.orchestra.run.smithi045:> contents = proc_f.read() 2024-05-08T05:41:41.291 DEBUG:teuthology.orchestra.run.smithi045:> if mountpoint in contents: 2024-05-08T05:41:41.291 DEBUG:teuthology.orchestra.run.smithi045:> return f 2024-05-08T05:41:41.291 DEBUG:teuthology.orchestra.run.smithi045:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T05:41:41.291 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:41:41.292 DEBUG:teuthology.orchestra.run.smithi045:> print(_find_admin_socket("client.admin")) 2024-05-08T05:41:41.292 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:41:41.412 INFO:teuthology.orchestra.run.smithi045.stdout:/var/run/ceph/ceph-client.admin.42319.asok 2024-05-08T05:41:41.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:41.423 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.42319.asok 2024-05-08T05:41:41.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:41.423 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.42319.asok status 2024-05-08T05:41:41.562 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:41:41.562 INFO:teuthology.orchestra.run.smithi045.stdout: "metadata": { 2024-05-08T05:41:41.562 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_sha1": "cd693b0b93278c2b128be2d6d93ae4df18c3ba1c", 2024-05-08T05:41:41.562 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 19.0.0-3538-gcd693b0b (cd693b0b93278c2b128be2d6d93ae4df18c3ba1c) squid (dev)", 2024-05-08T05:41:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "entity_id": "admin", 2024-05-08T05:41:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": "smithi045", 2024-05-08T05:41:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-08T05:41:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "pid": "42319", 2024-05-08T05:41:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "root": "/" 2024-05-08T05:41:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:41:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_count": 0, 2024-05-08T05:41:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "dentry_pinned_count": 0, 2024-05-08T05:41:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 4404, 2024-05-08T05:41:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "inst": { 2024-05-08T05:41:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "name": { 2024-05-08T05:41:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "client", 2024-05-08T05:41:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "num": 4404 2024-05-08T05:41:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:41:41.563 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2024-05-08T05:41:41.564 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-05-08T05:41:41.564 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2024-05-08T05:41:41.564 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3437096518 2024-05-08T05:41:41.564 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:41:41.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:41:41.564 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": { 2024-05-08T05:41:41.564 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-05-08T05:41:41.564 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "192.168.0.1:0", 2024-05-08T05:41:41.564 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3437096518 2024-05-08T05:41:41.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:41:41.564 INFO:teuthology.orchestra.run.smithi045.stdout: "inst_str": "client.4404 192.168.0.1:0/3437096518", 2024-05-08T05:41:41.565 INFO:teuthology.orchestra.run.smithi045.stdout: "addr_str": "192.168.0.1:0/3437096518", 2024-05-08T05:41:41.565 INFO:teuthology.orchestra.run.smithi045.stdout: "inode_count": 1, 2024-05-08T05:41:41.565 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_epoch": 6, 2024-05-08T05:41:41.565 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch": 24, 2024-05-08T05:41:41.565 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_epoch_barrier": 0, 2024-05-08T05:41:41.565 INFO:teuthology.orchestra.run.smithi045.stdout: "blocklisted": false, 2024-05-08T05:41:41.565 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs" 2024-05-08T05:41:41.565 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:41:41.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:41.571 DEBUG:teuthology.orchestra.run.smithi045:> stat --file-system '--printf=%T 2024-05-08T05:41:41.571 DEBUG:teuthology.orchestra.run.smithi045:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:41:41.589 INFO:teuthology.orchestra.run.smithi045.stdout:fuseblk 2024-05-08T05:41:41.590 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:41:41.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:41.590 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:41:41.661 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-08T05:41:41.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:41:41.771 INFO:teuthology.orchestra.run.smithi045.stdout:# file: . 2024-05-08T05:41:41.771 INFO:teuthology.orchestra.run.smithi045.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-08T05:41:41.771 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:41.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:41.777 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:41:41.778 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:41:41.795 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:41:41.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.939+0000 7fefc046d640 1 Processor -- start 2024-05-08T05:41:41.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.939+0000 7fefc046d640 1 -- start start 2024-05-08T05:41:41.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.940+0000 7fefc046d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefb8072840 0x7fefb815c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:41.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.940+0000 7fefc046d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fefb806e4d0 0x7fefb80aec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:41.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.940+0000 7fefc046d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefb815c920 con 0x7fefb806e0f0 2024-05-08T05:41:41.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.940+0000 7fefc046d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fefb815cc00 con 0x7fefb806e4d0 2024-05-08T05:41:41.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.940+0000 7fefc046d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fefb815cee0 con 0x7fefb8072840 2024-05-08T05:41:41.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.940+0000 7fefbda08640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefb8072840 0x7fefb815c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:41.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.940+0000 7fefbea0a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fefb806e4d0 0x7fefb80aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:41.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.940+0000 7fefbda08640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefb8072840 0x7fefb815c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:32928/0 (socket says 172.21.15.45:32928) 2024-05-08T05:41:41.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.940+0000 7fefbda08640 1 -- 172.21.15.45:0/3716001322 learned_addr learned my addr 172.21.15.45:0/3716001322 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:41.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.940+0000 7fefbea0a640 1 -- 172.21.15.45:0/3716001322 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefb8072840 msgr2=0x7fefb815c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:41.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.940+0000 7fefbea0a640 1 --2- 172.21.15.45:0/3716001322 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefb8072840 0x7fefb815c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:41.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.940+0000 7fefbea0a640 1 -- 172.21.15.45:0/3716001322 >> v1:172.21.15.63:6789/0 conn(0x7fefb806e0f0 legacy=0x7fefb80d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:41.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.940+0000 7fefbea0a640 1 -- 172.21.15.45:0/3716001322 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb815d1c0 con 0x7fefb806e4d0 2024-05-08T05:41:41.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.940+0000 7fefbd207640 1 -- 172.21.15.45:0/3716001322 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1530851433 0 0) 0x7fefb815c920 con 0x7fefb806e0f0 2024-05-08T05:41:41.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.941+0000 7fefbea0a640 1 --2- 172.21.15.45:0/3716001322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fefb806e4d0 0x7fefb80aec30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009d90 tx=0x7fefb002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d653ab46c4e078a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:41.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.941+0000 7fefbd207640 1 -- 172.21.15.45:0/3716001322 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb0007740 con 0x7fefb806e4d0 2024-05-08T05:41:41.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.941+0000 7fefbd207640 1 -- 172.21.15.45:0/3716001322 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb00078e0 con 0x7fefb806e4d0 2024-05-08T05:41:41.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.941+0000 7fefbd207640 1 -- 172.21.15.45:0/3716001322 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb0007cb0 con 0x7fefb806e4d0 2024-05-08T05:41:41.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.942+0000 7fefc046d640 1 -- 172.21.15.45:0/3716001322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fefb806e4d0 msgr2=0x7fefb80aec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:41.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.942+0000 7fefc046d640 1 --2- 172.21.15.45:0/3716001322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fefb806e4d0 0x7fefb80aec30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009d90 tx=0x7fefb002a660 comp rx=0 tx=0).stop 2024-05-08T05:41:41.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.942+0000 7fefc046d640 1 -- 172.21.15.45:0/3716001322 shutdown_connections 2024-05-08T05:41:41.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.942+0000 7fefc046d640 1 --2- 172.21.15.45:0/3716001322 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fefb806e4d0 0x7fefb80aec30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:41.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.942+0000 7fefc046d640 1 --2- 172.21.15.45:0/3716001322 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefb8072840 0x7fefb815c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:41.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.942+0000 7fefc046d640 1 -- 172.21.15.45:0/3716001322 >> 172.21.15.45:0/3716001322 conn(0x7fefb806c3a0 msgr2=0x7fefb806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:41.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.942+0000 7fefc046d640 1 -- 172.21.15.45:0/3716001322 shutdown_connections 2024-05-08T05:41:41.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.942+0000 7fefc046d640 1 -- 172.21.15.45:0/3716001322 wait complete. 2024-05-08T05:41:41.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.943+0000 7fefc046d640 1 Processor -- start 2024-05-08T05:41:41.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.943+0000 7fefc046d640 1 -- start start 2024-05-08T05:41:41.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.944+0000 7fefc046d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fefb806e0f0 0x7fefb8145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:41.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.944+0000 7fefc046d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefb806e4d0 0x7fefb809c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:41.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.944+0000 7fefc046d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefb8072840 0x7fefb80cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:41.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.944+0000 7fefc046d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fefb80ae7d0 con 0x7fefb8072840 2024-05-08T05:41:41.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.944+0000 7fefc046d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fefb80ae650 con 0x7fefb806e0f0 2024-05-08T05:41:41.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.944+0000 7fefc046d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fefb81530b0 con 0x7fefb806e4d0 2024-05-08T05:41:41.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.944+0000 7fefbea0a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefb8072840 0x7fefb80cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:41.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.944+0000 7fefbea0a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefb8072840 0x7fefb80cfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51048/0 (socket says 172.21.15.45:51048) 2024-05-08T05:41:41.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.944+0000 7fefbea0a640 1 -- 172.21.15.45:0/1976035963 learned_addr learned my addr 172.21.15.45:0/1976035963 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:41.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.944+0000 7fefbda08640 1 --2- 172.21.15.45:0/1976035963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefb806e4d0 0x7fefb809c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:41.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.944+0000 7fefbe209640 1 --2- 172.21.15.45:0/1976035963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fefb806e0f0 0x7fefb8145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:41.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.945+0000 7fefbea0a640 1 -- 172.21.15.45:0/1976035963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefb806e4d0 msgr2=0x7fefb809c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:41.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.945+0000 7fefbea0a640 1 --2- 172.21.15.45:0/1976035963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefb806e4d0 0x7fefb809c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:41.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.945+0000 7fefbea0a640 1 -- 172.21.15.45:0/1976035963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fefb806e0f0 msgr2=0x7fefb8145970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:41.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.945+0000 7fefbea0a640 1 --2- 172.21.15.45:0/1976035963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fefb806e0f0 0x7fefb8145970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:41.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.945+0000 7fefbea0a640 1 -- 172.21.15.45:0/1976035963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb80cd850 con 0x7fefb8072840 2024-05-08T05:41:41.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.945+0000 7fefbea0a640 1 --2- 172.21.15.45:0/1976035963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefb8072840 0x7fefb80cfa60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009eb0 tx=0x7fefb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:41.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.945+0000 7fefaeffd640 1 -- 172.21.15.45:0/1976035963 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb0040070 con 0x7fefb8072840 2024-05-08T05:41:41.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.945+0000 7fefaeffd640 1 -- 172.21.15.45:0/1976035963 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefb0031c40 con 0x7fefb8072840 2024-05-08T05:41:41.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.945+0000 7fefaeffd640 1 -- 172.21.15.45:0/1976035963 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb0009450 con 0x7fefb8072840 2024-05-08T05:41:41.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.945+0000 7fefc046d640 1 -- 172.21.15.45:0/1976035963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb80cdb30 con 0x7fefb8072840 2024-05-08T05:41:41.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.945+0000 7fefc046d640 1 -- 172.21.15.45:0/1976035963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb8145eb0 con 0x7fefb8072840 2024-05-08T05:41:41.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.946+0000 7fefc046d640 1 -- 172.21.15.45:0/1976035963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefb80693f0 con 0x7fefb8072840 2024-05-08T05:41:41.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.947+0000 7fefaeffd640 1 -- 172.21.15.45:0/1976035963 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fefb0039040 con 0x7fefb8072840 2024-05-08T05:41:41.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.947+0000 7fefaeffd640 1 --2- 172.21.15.45:0/1976035963 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef9c0762d0 0x7fef9c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:41.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.947+0000 7fefaeffd640 1 -- 172.21.15.45:0/1976035963 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fefb00b86a0 con 0x7fefb8072840 2024-05-08T05:41:41.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.947+0000 7fefbe209640 1 --2- 172.21.15.45:0/1976035963 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef9c0762d0 0x7fef9c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:41.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.948+0000 7fefbe209640 1 --2- 172.21.15.45:0/1976035963 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef9c0762d0 0x7fef9c078790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fefb4000f60 tx=0x7fefb4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:41.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:41.952+0000 7fefaeffd640 1 -- 172.21.15.45:0/1976035963 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fefb0081d50 con 0x7fefb8072840 2024-05-08T05:41:42.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.163+0000 7fefc046d640 1 -- 172.21.15.45:0/1976035963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fefb80765b0 con 0x7fefb8072840 2024-05-08T05:41:42.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.164+0000 7fefaeffd640 1 -- 172.21.15.45:0/1976035963 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fefb0081650 con 0x7fefb8072840 2024-05-08T05:41:42.164 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-05-08T05:41:42.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.166+0000 7fefc046d640 1 -- 172.21.15.45:0/1976035963 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef9c0762d0 msgr2=0x7fef9c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:42.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.166+0000 7fefc046d640 1 --2- 172.21.15.45:0/1976035963 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef9c0762d0 0x7fef9c078790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fefb4000f60 tx=0x7fefb4009040 comp rx=0 tx=0).stop 2024-05-08T05:41:42.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.166+0000 7fefc046d640 1 -- 172.21.15.45:0/1976035963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefb8072840 msgr2=0x7fefb80cfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:42.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.166+0000 7fefc046d640 1 --2- 172.21.15.45:0/1976035963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefb8072840 0x7fefb80cfa60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009eb0 tx=0x7fefb0004a40 comp rx=0 tx=0).stop 2024-05-08T05:41:42.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.166+0000 7fefc046d640 1 -- 172.21.15.45:0/1976035963 shutdown_connections 2024-05-08T05:41:42.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.166+0000 7fefc046d640 1 --2- 172.21.15.45:0/1976035963 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef9c0762d0 0x7fef9c078790 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.166+0000 7fefc046d640 1 --2- 172.21.15.45:0/1976035963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fefb806e4d0 0x7fefb809c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.166+0000 7fefc046d640 1 --2- 172.21.15.45:0/1976035963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefb8072840 0x7fefb80cfa60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.166+0000 7fefc046d640 1 --2- 172.21.15.45:0/1976035963 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fefb806e0f0 0x7fefb8145970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.166+0000 7fefc046d640 1 -- 172.21.15.45:0/1976035963 >> 172.21.15.45:0/1976035963 conn(0x7fefb806c3a0 msgr2=0x7fefb80af3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:42.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.166+0000 7fefc046d640 1 -- 172.21.15.45:0/1976035963 shutdown_connections 2024-05-08T05:41:42.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.166+0000 7fefc046d640 1 -- 172.21.15.45:0/1976035963 wait complete. 2024-05-08T05:41:42.177 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:41:42.177 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:41:42.194 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:41:42.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.338+0000 7f3f2755f640 1 Processor -- start 2024-05-08T05:41:42.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.338+0000 7f3f2755f640 1 -- start start 2024-05-08T05:41:42.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.339+0000 7f3f2755f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3f201629a0 0x7f3f200d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:42.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.339+0000 7f3f2755f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f200d3330 con 0x7f3f200d0cb0 2024-05-08T05:41:42.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.339+0000 7f3f2755f640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f200d2170 con 0x7f3f200d2dc0 2024-05-08T05:41:42.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.339+0000 7f3f2755f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3f200d2450 con 0x7f3f201629a0 2024-05-08T05:41:42.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.339+0000 7f3f25afc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3f200d0cb0 0x7f3f200c95e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43886/0 (socket says 172.21.15.45:43886) 2024-05-08T05:41:42.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.339+0000 7f3f25afc640 1 -- 172.21.15.45:0/3481850992 learned_addr learned my addr 172.21.15.45:0/3481850992 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:42.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.339+0000 7f3f252fb640 1 --2- 172.21.15.45:0/3481850992 >> v2:172.21.15.63:3300/0 conn(0x7f3f201629a0 0x7f3f200d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:42.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.339+0000 7f3f1ffff640 1 -- 172.21.15.45:0/3481850992 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3914843816 0 0) 0x7f3f200d3330 con 0x7f3f200d0cb0 2024-05-08T05:41:42.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.339+0000 7f3f1ffff640 1 -- 172.21.15.45:0/3481850992 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f08003520 con 0x7f3f200d0cb0 2024-05-08T05:41:42.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f1ffff640 1 -- 172.21.15.45:0/3481850992 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3194730804 0 0) 0x7f3f08003520 con 0x7f3f200d0cb0 2024-05-08T05:41:42.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f1ffff640 1 -- 172.21.15.45:0/3481850992 >> v2:172.21.15.63:3300/0 conn(0x7f3f201629a0 msgr2=0x7f3f200d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:42.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f1ffff640 1 --2- 172.21.15.45:0/3481850992 >> v2:172.21.15.63:3300/0 conn(0x7f3f201629a0 0x7f3f200d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f1ffff640 1 -- 172.21.15.45:0/3481850992 >> v1:172.21.15.63:6789/0 conn(0x7f3f200d2dc0 legacy=0x7f3f200c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:42.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f1ffff640 1 -- 172.21.15.45:0/3481850992 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f200d2730 con 0x7f3f200d0cb0 2024-05-08T05:41:42.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f1ffff640 1 -- 172.21.15.45:0/3481850992 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f3f10001a70 con 0x7f3f200d0cb0 2024-05-08T05:41:42.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f1ffff640 1 -- 172.21.15.45:0/3481850992 >> v1:172.21.15.45:6789/0 conn(0x7f3f200d0cb0 legacy=0x7f3f200c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:42.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f1ffff640 1 --2- 172.21.15.45:0/3481850992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f08003a50 0x7f3f08003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:42.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f1ffff640 1 --2- 172.21.15.45:0/3481850992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f08004800 0x7f3f08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:42.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f1ffff640 1 --2- 172.21.15.45:0/3481850992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f080053c0 0x7f3f0800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:42.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f1ffff640 1 -- 172.21.15.45:0/3481850992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3f200d2450 con 0x7f3f08004800 2024-05-08T05:41:42.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f1ffff640 1 -- 172.21.15.45:0/3481850992 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0800c230 con 0x7f3f080053c0 2024-05-08T05:41:42.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f1ffff640 1 -- 172.21.15.45:0/3481850992 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3f0800c430 con 0x7f3f08003a50 2024-05-08T05:41:42.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f24afa640 1 --2- 172.21.15.45:0/3481850992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f08004800 0x7f3f08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:42.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f25afc640 1 --2- 172.21.15.45:0/3481850992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f080053c0 0x7f3f0800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:42.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f252fb640 1 --2- 172.21.15.45:0/3481850992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f08003a50 0x7f3f08003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:42.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f24afa640 1 -- 172.21.15.45:0/3481850992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f08003a50 msgr2=0x7f3f08003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:42.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f24afa640 1 --2- 172.21.15.45:0/3481850992 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f08003a50 0x7f3f08003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f24afa640 1 -- 172.21.15.45:0/3481850992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f080053c0 msgr2=0x7f3f0800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:42.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f24afa640 1 --2- 172.21.15.45:0/3481850992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f080053c0 0x7f3f0800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f24afa640 1 -- 172.21.15.45:0/3481850992 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f08003520 con 0x7f3f08004800 2024-05-08T05:41:42.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f25afc640 1 --2- 172.21.15.45:0/3481850992 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f080053c0 0x7f3f0800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:42.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.340+0000 7f3f25afc640 1 -- 172.21.15.45:0/3481850992 reap_dead start 2024-05-08T05:41:42.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.341+0000 7f3f24afa640 1 --2- 172.21.15.45:0/3481850992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f08004800 0x7f3f08004c80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3f14001790 tx=0x7f3f08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc92d898274b7434 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.341+0000 7f3f1ffff640 1 -- 172.21.15.45:0/3481850992 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f140013b0 con 0x7f3f08004800 2024-05-08T05:41:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.341+0000 7f3f1ffff640 1 -- 172.21.15.45:0/3481850992 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f140052d0 con 0x7f3f08004800 2024-05-08T05:41:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.341+0000 7f3f1ffff640 1 -- 172.21.15.45:0/3481850992 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1402aac0 con 0x7f3f08004800 2024-05-08T05:41:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.341+0000 7f3f2755f640 1 -- 172.21.15.45:0/3481850992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f08004800 msgr2=0x7f3f08004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.341+0000 7f3f2755f640 1 --2- 172.21.15.45:0/3481850992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f08004800 0x7f3f08004c80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3f14001790 tx=0x7f3f08005830 comp rx=0 tx=0).stop 2024-05-08T05:41:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.342+0000 7f3f2755f640 1 -- 172.21.15.45:0/3481850992 shutdown_connections 2024-05-08T05:41:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.342+0000 7f3f2755f640 1 --2- 172.21.15.45:0/3481850992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f08004800 0x7f3f08004c80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.342+0000 7f3f2755f640 1 -- 172.21.15.45:0/3481850992 >> 172.21.15.45:0/3481850992 conn(0x7f3f200d4fe0 msgr2=0x7f3f200d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.342+0000 7f3f2755f640 1 -- 172.21.15.45:0/3481850992 shutdown_connections 2024-05-08T05:41:42.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.342+0000 7f3f2755f640 1 -- 172.21.15.45:0/3481850992 wait complete. 2024-05-08T05:41:42.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.343+0000 7f3f2755f640 1 Processor -- start 2024-05-08T05:41:42.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.343+0000 7f3f2755f640 1 -- start start 2024-05-08T05:41:42.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.343+0000 7f3f2755f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f08004800 0x7f3f200a2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:42.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.343+0000 7f3f2755f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f200a1000 0x7f3f20156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:42.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.344+0000 7f3f2755f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f201f52e0 0x7f3f200ce430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:42.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.344+0000 7f3f2755f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3f200cf440 con 0x7f3f200a1000 2024-05-08T05:41:42.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.344+0000 7f3f2755f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3f200cf2c0 con 0x7f3f201f52e0 2024-05-08T05:41:42.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.344+0000 7f3f2755f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3f200d35c0 con 0x7f3f08004800 2024-05-08T05:41:42.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.344+0000 7f3f24afa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f200a1000 0x7f3f20156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:42.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.344+0000 7f3f24afa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f200a1000 0x7f3f20156360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51080/0 (socket says 172.21.15.45:51080) 2024-05-08T05:41:42.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.344+0000 7f3f24afa640 1 -- 172.21.15.45:0/668245872 learned_addr learned my addr 172.21.15.45:0/668245872 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:42.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.344+0000 7f3f252fb640 1 --2- 172.21.15.45:0/668245872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f08004800 0x7f3f200a2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:42.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.344+0000 7f3f25afc640 1 --2- 172.21.15.45:0/668245872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f201f52e0 0x7f3f200ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:42.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.344+0000 7f3f24afa640 1 -- 172.21.15.45:0/668245872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f08004800 msgr2=0x7f3f200a2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:42.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.344+0000 7f3f24afa640 1 --2- 172.21.15.45:0/668245872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f08004800 0x7f3f200a2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.344+0000 7f3f24afa640 1 -- 172.21.15.45:0/668245872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f201f52e0 msgr2=0x7f3f200ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:42.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.344+0000 7f3f24afa640 1 --2- 172.21.15.45:0/668245872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f201f52e0 0x7f3f200ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.344+0000 7f3f24afa640 1 -- 172.21.15.45:0/668245872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f201568a0 con 0x7f3f200a1000 2024-05-08T05:41:42.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.344+0000 7f3f25afc640 1 --2- 172.21.15.45:0/668245872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f201f52e0 0x7f3f200ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:42.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.344+0000 7f3f24afa640 1 --2- 172.21.15.45:0/668245872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f200a1000 0x7f3f20156360 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3f14001790 tx=0x7f3f14001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:42.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.344+0000 7f3f1dffb640 1 -- 172.21.15.45:0/668245872 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14042070 con 0x7f3f200a1000 2024-05-08T05:41:42.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.344+0000 7f3f1dffb640 1 -- 172.21.15.45:0/668245872 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f14005480 con 0x7f3f200a1000 2024-05-08T05:41:42.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.344+0000 7f3f2755f640 1 -- 172.21.15.45:0/668245872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f200cc0b0 con 0x7f3f200a1000 2024-05-08T05:41:42.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.344+0000 7f3f1dffb640 1 -- 172.21.15.45:0/668245872 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14037040 con 0x7f3f200a1000 2024-05-08T05:41:42.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.345+0000 7f3f2755f640 1 -- 172.21.15.45:0/668245872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f200cc670 con 0x7f3f200a1000 2024-05-08T05:41:42.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.345+0000 7f3f2755f640 1 -- 172.21.15.45:0/668245872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f200693f0 con 0x7f3f200a1000 2024-05-08T05:41:42.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.346+0000 7f3f1dffb640 1 -- 172.21.15.45:0/668245872 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3f14045070 con 0x7f3f200a1000 2024-05-08T05:41:42.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.347+0000 7f3f1dffb640 1 --2- 172.21.15.45:0/668245872 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f0c0762d0 0x7f3f0c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:42.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.347+0000 7f3f1dffb640 1 -- 172.21.15.45:0/668245872 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3f140b7230 con 0x7f3f200a1000 2024-05-08T05:41:42.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.347+0000 7f3f252fb640 1 --2- 172.21.15.45:0/668245872 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f0c0762d0 0x7f3f0c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:42.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.347+0000 7f3f252fb640 1 --2- 172.21.15.45:0/668245872 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f0c0762d0 0x7f3f0c078790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3f18009b40 tx=0x7f3f18008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:42.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.351+0000 7f3f1dffb640 1 -- 172.21.15.45:0/668245872 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3f14080170 con 0x7f3f200a1000 2024-05-08T05:41:42.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.553+0000 7f3f2755f640 1 -- 172.21.15.45:0/668245872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3f200bada0 con 0x7f3f200a1000 2024-05-08T05:41:42.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.554+0000 7f3f1dffb640 1 -- 172.21.15.45:0/668245872 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f3f14084470 con 0x7f3f200a1000 2024-05-08T05:41:42.554 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-05-08T05:41:42.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.556+0000 7f3f2755f640 1 -- 172.21.15.45:0/668245872 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f0c0762d0 msgr2=0x7f3f0c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:42.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.556+0000 7f3f2755f640 1 --2- 172.21.15.45:0/668245872 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f0c0762d0 0x7f3f0c078790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3f18009b40 tx=0x7f3f18008040 comp rx=0 tx=0).stop 2024-05-08T05:41:42.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.556+0000 7f3f2755f640 1 -- 172.21.15.45:0/668245872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f200a1000 msgr2=0x7f3f20156360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:42.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.556+0000 7f3f2755f640 1 --2- 172.21.15.45:0/668245872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f200a1000 0x7f3f20156360 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3f14001790 tx=0x7f3f14001120 comp rx=0 tx=0).stop 2024-05-08T05:41:42.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.556+0000 7f3f2755f640 1 -- 172.21.15.45:0/668245872 shutdown_connections 2024-05-08T05:41:42.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.556+0000 7f3f2755f640 1 --2- 172.21.15.45:0/668245872 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f0c0762d0 0x7f3f0c078790 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.556+0000 7f3f2755f640 1 --2- 172.21.15.45:0/668245872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f201f52e0 0x7f3f200ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.556+0000 7f3f2755f640 1 --2- 172.21.15.45:0/668245872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f200a1000 0x7f3f20156360 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.556+0000 7f3f2755f640 1 --2- 172.21.15.45:0/668245872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f08004800 0x7f3f200a2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.556+0000 7f3f2755f640 1 -- 172.21.15.45:0/668245872 >> 172.21.15.45:0/668245872 conn(0x7f3f200d4fe0 msgr2=0x7f3f200b71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:42.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.556+0000 7f3f2755f640 1 -- 172.21.15.45:0/668245872 shutdown_connections 2024-05-08T05:41:42.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.556+0000 7f3f2755f640 1 -- 172.21.15.45:0/668245872 wait complete. 2024-05-08T05:41:42.567 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi045.front.sepia.ceph.com... 2024-05-08T05:41:42.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:42.567 DEBUG:teuthology.orchestra.run.smithi045:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:41:42.592 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi045.stderr:ceph-fuse[42319]: fuse finished with error 0 and tester_r 0 2024-05-08T05:41:42.593 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:41:42.661 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:41:42.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:42.661 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-08T05:41:42.686 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:41:42.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.830+0000 7f5ea9319640 1 Processor -- start 2024-05-08T05:41:42.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.830+0000 7f5ea9319640 1 -- start start 2024-05-08T05:41:42.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.831+0000 7f5ea9319640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5ea415f420 0x7f5ea40d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:42.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.831+0000 7f5ea9319640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5ea40d3ca0 0x7f5ea40d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:42.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.831+0000 7f5ea9319640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ea41630b0 con 0x7f5ea40d1eb0 2024-05-08T05:41:42.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.831+0000 7f5ea9319640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f5ea40d2550 con 0x7f5ea415f420 2024-05-08T05:41:42.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.831+0000 7f5ea9319640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5ea40d2c80 con 0x7f5ea40d3ca0 2024-05-08T05:41:42.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.831+0000 7f5ea2ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5ea415f420 0x7f5ea40d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:42.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.831+0000 7f5ea37fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5ea40d1eb0 0x7f5ea40d4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43898/0 (socket says 172.21.15.45:43898) 2024-05-08T05:41:42.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.831+0000 7f5ea37fe640 1 -- 172.21.15.45:0/1879379704 learned_addr learned my addr 172.21.15.45:0/1879379704 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:42.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.831+0000 7f5e9bfff640 1 --2- 172.21.15.45:0/1879379704 >> v2:172.21.15.63:3300/0 conn(0x7f5ea40d3ca0 0x7f5ea40d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:42.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.831+0000 7f5ea27fc640 1 -- 172.21.15.45:0/1879379704 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3971527159 0 0) 0x7f5e9c001490 con 0x7f5ea40d1eb0 2024-05-08T05:41:42.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.831+0000 7f5ea2ffd640 1 -- 172.21.15.45:0/1879379704 >> v2:172.21.15.63:3300/0 conn(0x7f5ea40d3ca0 msgr2=0x7f5ea40d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:42.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.831+0000 7f5ea2ffd640 1 --2- 172.21.15.45:0/1879379704 >> v2:172.21.15.63:3300/0 conn(0x7f5ea40d3ca0 0x7f5ea40d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.831+0000 7f5ea2ffd640 1 -- 172.21.15.45:0/1879379704 >> v1:172.21.15.45:6789/0 conn(0x7f5ea40d1eb0 legacy=0x7f5ea40d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:42.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.831+0000 7f5ea2ffd640 1 -- 172.21.15.45:0/1879379704 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea40d2f60 con 0x7f5ea415f420 2024-05-08T05:41:42.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.832+0000 7f5ea2ffd640 1 --2- 172.21.15.45:0/1879379704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ea415f420 0x7f5ea40d3760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5ea408f380 tx=0x7f5e9402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d87e3b452eaec6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:42.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.832+0000 7f5ea27fc640 1 -- 172.21.15.45:0/1879379704 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e94037070 con 0x7f5ea415f420 2024-05-08T05:41:42.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.832+0000 7f5ea27fc640 1 -- 172.21.15.45:0/1879379704 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e94032440 con 0x7f5ea415f420 2024-05-08T05:41:42.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.832+0000 7f5ea27fc640 1 -- 172.21.15.45:0/1879379704 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e94032810 con 0x7f5ea415f420 2024-05-08T05:41:42.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.833+0000 7f5ea9319640 1 -- 172.21.15.45:0/1879379704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ea415f420 msgr2=0x7f5ea40d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:42.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.833+0000 7f5ea9319640 1 --2- 172.21.15.45:0/1879379704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ea415f420 0x7f5ea40d3760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5ea408f380 tx=0x7f5e9402a1a0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.833+0000 7f5ea9319640 1 -- 172.21.15.45:0/1879379704 shutdown_connections 2024-05-08T05:41:42.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.833+0000 7f5ea9319640 1 --2- 172.21.15.45:0/1879379704 >> v2:172.21.15.63:3300/0 conn(0x7f5ea40d3ca0 0x7f5ea40d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.833+0000 7f5ea9319640 1 --2- 172.21.15.45:0/1879379704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ea415f420 0x7f5ea40d3760 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.833+0000 7f5ea9319640 1 -- 172.21.15.45:0/1879379704 >> 172.21.15.45:0/1879379704 conn(0x7f5ea408eac0 msgr2=0x7f5ea409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:42.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.833+0000 7f5ea9319640 1 -- 172.21.15.45:0/1879379704 shutdown_connections 2024-05-08T05:41:42.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.833+0000 7f5ea9319640 1 -- 172.21.15.45:0/1879379704 wait complete. 2024-05-08T05:41:42.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.834+0000 7f5ea9319640 1 Processor -- start 2024-05-08T05:41:42.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.834+0000 7f5ea9319640 1 -- start start 2024-05-08T05:41:42.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.835+0000 7f5ea9319640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ea40d1eb0 0x7f5ea4158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:42.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.835+0000 7f5ea9319640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ea40c9a20 0x7f5ea415f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:42.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.835+0000 7f5ea9319640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ea40c9ea0 0x7f5ea40caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:42.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.835+0000 7f5ea9319640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea40d53d0 con 0x7f5ea40c9ea0 2024-05-08T05:41:42.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.835+0000 7f5ea9319640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea40d2830 con 0x7f5ea40c9a20 2024-05-08T05:41:42.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.835+0000 7f5ea9319640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea40d5550 con 0x7f5ea40d1eb0 2024-05-08T05:41:42.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.835+0000 7f5ea37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ea40c9ea0 0x7f5ea40caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:42.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.835+0000 7f5ea37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ea40c9ea0 0x7f5ea40caee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51094/0 (socket says 172.21.15.45:51094) 2024-05-08T05:41:42.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.835+0000 7f5ea37fe640 1 -- 172.21.15.45:0/454564211 learned_addr learned my addr 172.21.15.45:0/454564211 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:42.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.835+0000 7f5e9bfff640 1 --2- 172.21.15.45:0/454564211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ea40c9a20 0x7f5ea415f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:42.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.835+0000 7f5ea2ffd640 1 --2- 172.21.15.45:0/454564211 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ea40d1eb0 0x7f5ea4158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:42.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.835+0000 7f5ea37fe640 1 -- 172.21.15.45:0/454564211 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ea40d1eb0 msgr2=0x7f5ea4158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:42.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.835+0000 7f5ea37fe640 1 --2- 172.21.15.45:0/454564211 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ea40d1eb0 0x7f5ea4158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.835+0000 7f5ea37fe640 1 -- 172.21.15.45:0/454564211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ea40c9a20 msgr2=0x7f5ea415f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:42.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.835+0000 7f5ea37fe640 1 --2- 172.21.15.45:0/454564211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ea40c9a20 0x7f5ea415f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.835+0000 7f5ea37fe640 1 -- 172.21.15.45:0/454564211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea40cc280 con 0x7f5ea40c9ea0 2024-05-08T05:41:42.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.835+0000 7f5ea37fe640 1 --2- 172.21.15.45:0/454564211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ea40c9ea0 0x7f5ea40caee0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00c7d0 tx=0x7f5e9c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:42.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.836+0000 7f5e9b7fe640 1 -- 172.21.15.45:0/454564211 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c018070 con 0x7f5ea40c9ea0 2024-05-08T05:41:42.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.836+0000 7f5e9b7fe640 1 -- 172.21.15.45:0/454564211 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e9c00a5e0 con 0x7f5ea40c9ea0 2024-05-08T05:41:42.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.836+0000 7f5ea9319640 1 -- 172.21.15.45:0/454564211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea40cc560 con 0x7f5ea40c9ea0 2024-05-08T05:41:42.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.836+0000 7f5e9b7fe640 1 -- 172.21.15.45:0/454564211 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c00a9d0 con 0x7f5ea40c9ea0 2024-05-08T05:41:42.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.836+0000 7f5ea9319640 1 -- 172.21.15.45:0/454564211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea40ccb20 con 0x7f5ea40c9ea0 2024-05-08T05:41:42.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.837+0000 7f5ea9319640 1 -- 172.21.15.45:0/454564211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea40693f0 con 0x7f5ea40c9ea0 2024-05-08T05:41:42.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.837+0000 7f5e9b7fe640 1 -- 172.21.15.45:0/454564211 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5e9c01b070 con 0x7f5ea40c9ea0 2024-05-08T05:41:42.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.838+0000 7f5e9b7fe640 1 --2- 172.21.15.45:0/454564211 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5e840762d0 0x7f5e84078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:42.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.838+0000 7f5e9b7fe640 1 -- 172.21.15.45:0/454564211 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5e9c096dd0 con 0x7f5ea40c9ea0 2024-05-08T05:41:42.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.838+0000 7f5ea2ffd640 1 --2- 172.21.15.45:0/454564211 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5e840762d0 0x7f5e84078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:42.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.839+0000 7f5ea2ffd640 1 --2- 172.21.15.45:0/454564211 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5e840762d0 0x7f5e84078790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5e94034000 tx=0x7f5e9402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:42.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:42.843+0000 7f5e9b7fe640 1 -- 172.21.15.45:0/454564211 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5ea41630b0 con 0x7f5ea40c9ea0 2024-05-08T05:41:43.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.036+0000 7f5ea9319640 1 -- 172.21.15.45:0/454564211 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ea40d15b0 con 0x7f5ea40c9ea0 2024-05-08T05:41:43.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.037+0000 7f5e9b7fe640 1 -- 172.21.15.45:0/454564211 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18466 (secure 0 0 0) 0x7f5e9c05fe10 con 0x7f5ea40c9ea0 2024-05-08T05:41:43.038 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:43.038 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:41:36.532045+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:41:29.381274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:41:30.443612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:41:34.132285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:41:43.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.040+0000 7f5ea9319640 1 -- 172.21.15.45:0/454564211 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5e840762d0 msgr2=0x7f5e84078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:43.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.040+0000 7f5ea9319640 1 --2- 172.21.15.45:0/454564211 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5e840762d0 0x7f5e84078790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5e94034000 tx=0x7f5e9402e000 comp rx=0 tx=0).stop 2024-05-08T05:41:43.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.040+0000 7f5ea9319640 1 -- 172.21.15.45:0/454564211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ea40c9ea0 msgr2=0x7f5ea40caee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:43.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.040+0000 7f5ea9319640 1 --2- 172.21.15.45:0/454564211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ea40c9ea0 0x7f5ea40caee0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00c7d0 tx=0x7f5e9c00cc90 comp rx=0 tx=0).stop 2024-05-08T05:41:43.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.040+0000 7f5ea9319640 1 -- 172.21.15.45:0/454564211 shutdown_connections 2024-05-08T05:41:43.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.040+0000 7f5ea9319640 1 --2- 172.21.15.45:0/454564211 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5e840762d0 0x7f5e84078790 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:43.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.040+0000 7f5ea9319640 1 --2- 172.21.15.45:0/454564211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ea40c9ea0 0x7f5ea40caee0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:43.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.040+0000 7f5ea9319640 1 --2- 172.21.15.45:0/454564211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ea40c9a20 0x7f5ea415f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:43.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.040+0000 7f5ea9319640 1 --2- 172.21.15.45:0/454564211 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ea40d1eb0 0x7f5ea4158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:43.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.040+0000 7f5ea9319640 1 -- 172.21.15.45:0/454564211 >> 172.21.15.45:0/454564211 conn(0x7f5ea408eac0 msgr2=0x7f5ea406cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:43.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.040+0000 7f5ea9319640 1 -- 172.21.15.45:0/454564211 shutdown_connections 2024-05-08T05:41:43.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.040+0000 7f5ea9319640 1 -- 172.21.15.45:0/454564211 wait complete. 2024-05-08T05:41:43.052 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:41:43.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.154+0000 7f510db40640 1 Processor -- start 2024-05-08T05:41:43.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.154+0000 7f510db40640 1 -- start start 2024-05-08T05:41:43.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.155+0000 7f510db40640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f51080d33f0 0x7f510815d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:43.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.155+0000 7f510db40640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5108070b90 0x7f5108071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:43.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.155+0000 7f510db40640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51080718c0 con 0x7f510815ccb0 2024-05-08T05:41:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.155+0000 7f510db40640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f51080afba0 con 0x7f51080d33f0 2024-05-08T05:41:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.155+0000 7f510db40640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5108071ba0 con 0x7f5108070b90 2024-05-08T05:41:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.155+0000 7f51077fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f510815ccb0 0x7f51080d2ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43902/0 (socket says 172.21.15.45:43902) 2024-05-08T05:41:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.155+0000 7f51077fe640 1 -- 172.21.15.45:0/2704019576 learned_addr learned my addr 172.21.15.45:0/2704019576 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.156+0000 7f5107fff640 1 --2- 172.21.15.45:0/2704019576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5108070b90 0x7f5108071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.156+0000 7f5106ffd640 1 --2- 172.21.15.45:0/2704019576 >> v2:172.21.15.63:3300/0 conn(0x7f51080d33f0 0x7f510815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.156+0000 7f51067fc640 1 -- 172.21.15.45:0/2704019576 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 912900354 0 0) 0x7f51080718c0 con 0x7f510815ccb0 2024-05-08T05:41:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.156+0000 7f51067fc640 1 -- 172.21.15.45:0/2704019576 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50f0003520 con 0x7f510815ccb0 2024-05-08T05:41:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.156+0000 7f5106ffd640 1 -- 172.21.15.45:0/2704019576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5108070b90 msgr2=0x7f5108071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.156+0000 7f5106ffd640 1 --2- 172.21.15.45:0/2704019576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5108070b90 0x7f5108071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.156+0000 7f51067fc640 1 -- 172.21.15.45:0/2704019576 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 105345271 0 0) 0x7f50f0003520 con 0x7f510815ccb0 2024-05-08T05:41:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.156+0000 7f5106ffd640 1 -- 172.21.15.45:0/2704019576 >> v1:172.21.15.45:6789/0 conn(0x7f510815ccb0 legacy=0x7f51080d2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.156+0000 7f5106ffd640 1 -- 172.21.15.45:0/2704019576 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5108071e80 con 0x7f51080d33f0 2024-05-08T05:41:43.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.157+0000 7f5106ffd640 1 --2- 172.21.15.45:0/2704019576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51080d33f0 0x7f510815d090 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0097a0 tx=0x7f50fc02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83fd0e9ba5a21e61 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:43.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.157+0000 7f51067fc640 1 -- 172.21.15.45:0/2704019576 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc037070 con 0x7f51080d33f0 2024-05-08T05:41:43.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.157+0000 7f51067fc640 1 -- 172.21.15.45:0/2704019576 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50fc02aba0 con 0x7f51080d33f0 2024-05-08T05:41:43.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.157+0000 7f51067fc640 1 -- 172.21.15.45:0/2704019576 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f80016f0 con 0x7f51080d33f0 2024-05-08T05:41:43.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.157+0000 7f510db40640 1 -- 172.21.15.45:0/2704019576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51080d33f0 msgr2=0x7f510815d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:43.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.157+0000 7f510db40640 1 --2- 172.21.15.45:0/2704019576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51080d33f0 0x7f510815d090 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f50fc0097a0 tx=0x7f50fc02a110 comp rx=0 tx=0).stop 2024-05-08T05:41:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.157+0000 7f510db40640 1 -- 172.21.15.45:0/2704019576 shutdown_connections 2024-05-08T05:41:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.157+0000 7f510db40640 1 --2- 172.21.15.45:0/2704019576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5108070b90 0x7f5108071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.157+0000 7f510db40640 1 --2- 172.21.15.45:0/2704019576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51080d33f0 0x7f510815d090 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.157+0000 7f510db40640 1 -- 172.21.15.45:0/2704019576 >> 172.21.15.45:0/2704019576 conn(0x7f510809ba40 msgr2=0x7f510809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.158+0000 7f510db40640 1 -- 172.21.15.45:0/2704019576 shutdown_connections 2024-05-08T05:41:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.158+0000 7f510db40640 1 -- 172.21.15.45:0/2704019576 wait complete. 2024-05-08T05:41:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.159+0000 7f510db40640 1 Processor -- start 2024-05-08T05:41:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.159+0000 7f510db40640 1 -- start start 2024-05-08T05:41:43.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f510db40640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5108070b90 0x7f5108157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:43.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f510db40640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51080d33f0 0x7f51080b7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:43.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f510db40640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510815ccb0 0x7f51080b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:43.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f510db40640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f51080cf980 con 0x7f510815ccb0 2024-05-08T05:41:43.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f510db40640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5108071560 con 0x7f51080d33f0 2024-05-08T05:41:43.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f510db40640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f51080cfb00 con 0x7f5108070b90 2024-05-08T05:41:43.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f5107fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510815ccb0 0x7f51080b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:43.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f5107fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510815ccb0 0x7f51080b4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51110/0 (socket says 172.21.15.45:51110) 2024-05-08T05:41:43.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f5107fff640 1 -- 172.21.15.45:0/1312278523 learned_addr learned my addr 172.21.15.45:0/1312278523 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:43.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f51077fe640 1 --2- 172.21.15.45:0/1312278523 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5108070b90 0x7f5108157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:43.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f5106ffd640 1 --2- 172.21.15.45:0/1312278523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51080d33f0 0x7f51080b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:43.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f5107fff640 1 -- 172.21.15.45:0/1312278523 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5108070b90 msgr2=0x7f5108157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:43.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f5107fff640 1 --2- 172.21.15.45:0/1312278523 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5108070b90 0x7f5108157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:43.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f5107fff640 1 -- 172.21.15.45:0/1312278523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51080d33f0 msgr2=0x7f51080b7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:43.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f5107fff640 1 --2- 172.21.15.45:0/1312278523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51080d33f0 0x7f51080b7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:43.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f5107fff640 1 -- 172.21.15.45:0/1312278523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51080b8440 con 0x7f510815ccb0 2024-05-08T05:41:43.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f5107fff640 1 --2- 172.21.15.45:0/1312278523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510815ccb0 0x7f51080b4530 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5100009d30 tx=0x7f510000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:43.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f50effff640 1 -- 172.21.15.45:0/1312278523 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100018020 con 0x7f510815ccb0 2024-05-08T05:41:43.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f50effff640 1 -- 172.21.15.45:0/1312278523 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5100014480 con 0x7f510815ccb0 2024-05-08T05:41:43.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f510db40640 1 -- 172.21.15.45:0/1312278523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51080b8720 con 0x7f510815ccb0 2024-05-08T05:41:43.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f50effff640 1 -- 172.21.15.45:0/1312278523 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100014870 con 0x7f510815ccb0 2024-05-08T05:41:43.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.160+0000 7f510db40640 1 -- 172.21.15.45:0/1312278523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510815e6e0 con 0x7f510815ccb0 2024-05-08T05:41:43.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.161+0000 7f510db40640 1 -- 172.21.15.45:0/1312278523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5108069b00 con 0x7f510815ccb0 2024-05-08T05:41:43.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.162+0000 7f50effff640 1 -- 172.21.15.45:0/1312278523 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5100005d10 con 0x7f510815ccb0 2024-05-08T05:41:43.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.163+0000 7f50effff640 1 --2- 172.21.15.45:0/1312278523 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50f40762d0 0x7f50f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:43.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.163+0000 7f50effff640 1 -- 172.21.15.45:0/1312278523 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5100098260 con 0x7f510815ccb0 2024-05-08T05:41:43.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.163+0000 7f51077fe640 1 --2- 172.21.15.45:0/1312278523 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50f40762d0 0x7f50f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:43.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.163+0000 7f51077fe640 1 --2- 172.21.15.45:0/1312278523 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50f40762d0 0x7f50f4078790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f50f8005880 tx=0x7f50f8001db0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:43.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.170+0000 7f50effff640 1 -- 172.21.15.45:0/1312278523 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5100022070 con 0x7f510815ccb0 2024-05-08T05:41:43.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.363+0000 7f510db40640 1 -- 172.21.15.45:0/1312278523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5108070380 con 0x7f510815ccb0 2024-05-08T05:41:43.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.364+0000 7f50effff640 1 -- 172.21.15.45:0/1312278523 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18466 (secure 0 0 0) 0x7f5100010040 con 0x7f510815ccb0 2024-05-08T05:41:43.364 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:43.365 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:41:36.532045+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:41:29.381274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:41:30.443612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:41:34.132285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:41:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.366+0000 7f510db40640 1 -- 172.21.15.45:0/1312278523 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50f40762d0 msgr2=0x7f50f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.366+0000 7f510db40640 1 --2- 172.21.15.45:0/1312278523 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50f40762d0 0x7f50f4078790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f50f8005880 tx=0x7f50f8001db0 comp rx=0 tx=0).stop 2024-05-08T05:41:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.366+0000 7f510db40640 1 -- 172.21.15.45:0/1312278523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510815ccb0 msgr2=0x7f51080b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.366+0000 7f510db40640 1 --2- 172.21.15.45:0/1312278523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510815ccb0 0x7f51080b4530 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5100009d30 tx=0x7f510000c490 comp rx=0 tx=0).stop 2024-05-08T05:41:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.366+0000 7f510db40640 1 -- 172.21.15.45:0/1312278523 shutdown_connections 2024-05-08T05:41:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.366+0000 7f510db40640 1 --2- 172.21.15.45:0/1312278523 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50f40762d0 0x7f50f4078790 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.366+0000 7f510db40640 1 --2- 172.21.15.45:0/1312278523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f510815ccb0 0x7f51080b4530 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.366+0000 7f510db40640 1 --2- 172.21.15.45:0/1312278523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51080d33f0 0x7f51080b7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.366+0000 7f510db40640 1 --2- 172.21.15.45:0/1312278523 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5108070b90 0x7f5108157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.366+0000 7f510db40640 1 -- 172.21.15.45:0/1312278523 >> 172.21.15.45:0/1312278523 conn(0x7f510809ba40 msgr2=0x7f51080b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:43.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.366+0000 7f510db40640 1 -- 172.21.15.45:0/1312278523 shutdown_connections 2024-05-08T05:41:43.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.366+0000 7f510db40640 1 -- 172.21.15.45:0/1312278523 wait complete. 2024-05-08T05:41:43.378 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T05:41:43.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.484+0000 7f00ec94d640 1 Processor -- start 2024-05-08T05:41:43.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.484+0000 7f00ec94d640 1 -- start start 2024-05-08T05:41:43.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.485+0000 7f00ec94d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f00e80b6590 0x7f00e80d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:43.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.485+0000 7f00ec94d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00e8163460 0x7f00e815ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:43.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.485+0000 7f00ec94d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00e80d3320 con 0x7f00e80d2bb0 2024-05-08T05:41:43.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.485+0000 7f00ec94d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f00e80d3600 con 0x7f00e80b6590 2024-05-08T05:41:43.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.485+0000 7f00ec94d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f00e80d38e0 con 0x7f00e8163460 2024-05-08T05:41:43.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.485+0000 7f00e659c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f00e80b6590 0x7f00e80d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:43.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.485+0000 7f00e659c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f00e80b6590 0x7f00e80d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51118/0 (socket says 172.21.15.45:51118) 2024-05-08T05:41:43.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.485+0000 7f00e659c640 1 -- 172.21.15.45:0/977254044 learned_addr learned my addr 172.21.15.45:0/977254044 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:43.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.485+0000 7f00e6d9d640 1 --2- 172.21.15.45:0/977254044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00e8163460 0x7f00e815ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:43.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.486+0000 7f00e659c640 1 -- 172.21.15.45:0/977254044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00e8163460 msgr2=0x7f00e815ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:43.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.486+0000 7f00e659c640 1 --2- 172.21.15.45:0/977254044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00e8163460 0x7f00e815ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:43.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.486+0000 7f00e659c640 1 -- 172.21.15.45:0/977254044 >> v1:172.21.15.63:6789/0 conn(0x7f00e80d2bb0 legacy=0x7f00e8162d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:43.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.486+0000 7f00e659c640 1 -- 172.21.15.45:0/977254044 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e80d3bc0 con 0x7f00e80b6590 2024-05-08T05:41:43.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.486+0000 7f00e659c640 1 --2- 172.21.15.45:0/977254044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00e80b6590 0x7f00e80d0bd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009e50 tx=0x7f00d802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c96285e3a2ced3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:43.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.486+0000 7f00e559a640 1 -- 172.21.15.45:0/977254044 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8036070 con 0x7f00e80b6590 2024-05-08T05:41:43.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.486+0000 7f00e559a640 1 -- 172.21.15.45:0/977254044 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d8031600 con 0x7f00e80b6590 2024-05-08T05:41:43.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.486+0000 7f00e559a640 1 -- 172.21.15.45:0/977254044 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d80319d0 con 0x7f00e80b6590 2024-05-08T05:41:43.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.487+0000 7f00ec94d640 1 -- 172.21.15.45:0/977254044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00e80b6590 msgr2=0x7f00e80d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:43.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.487+0000 7f00ec94d640 1 --2- 172.21.15.45:0/977254044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00e80b6590 0x7f00e80d0bd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009e50 tx=0x7f00d802a680 comp rx=0 tx=0).stop 2024-05-08T05:41:43.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.487+0000 7f00ec94d640 1 -- 172.21.15.45:0/977254044 shutdown_connections 2024-05-08T05:41:43.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.487+0000 7f00ec94d640 1 --2- 172.21.15.45:0/977254044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00e8163460 0x7f00e815ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:43.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.487+0000 7f00ec94d640 1 --2- 172.21.15.45:0/977254044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00e80b6590 0x7f00e80d0bd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:43.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.487+0000 7f00ec94d640 1 -- 172.21.15.45:0/977254044 >> 172.21.15.45:0/977254044 conn(0x7f00e80d4fe0 msgr2=0x7f00e80d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:43.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.487+0000 7f00ec94d640 1 -- 172.21.15.45:0/977254044 shutdown_connections 2024-05-08T05:41:43.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.487+0000 7f00ec94d640 1 -- 172.21.15.45:0/977254044 wait complete. 2024-05-08T05:41:43.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.488+0000 7f00ec94d640 1 Processor -- start 2024-05-08T05:41:43.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.488+0000 7f00ec94d640 1 -- start start 2024-05-08T05:41:43.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.489+0000 7f00ec94d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00e80b6590 0x7f00e80cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:43.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.489+0000 7f00ec94d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00e80d2bb0 0x7f00e81533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:43.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.489+0000 7f00ec94d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00e8163460 0x7f00e80cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:43.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.489+0000 7f00ec94d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f00e815f170 con 0x7f00e80b6590 2024-05-08T05:41:43.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.489+0000 7f00ec94d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f00e80d3010 con 0x7f00e8163460 2024-05-08T05:41:43.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.489+0000 7f00ec94d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f00e815f2f0 con 0x7f00e80d2bb0 2024-05-08T05:41:43.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.489+0000 7f00e659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00e80b6590 0x7f00e80cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:43.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.489+0000 7f00e659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00e80b6590 0x7f00e80cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51132/0 (socket says 172.21.15.45:51132) 2024-05-08T05:41:43.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.489+0000 7f00e659c640 1 -- 172.21.15.45:0/487405685 learned_addr learned my addr 172.21.15.45:0/487405685 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:43.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.490+0000 7f00e5d9b640 1 --2- 172.21.15.45:0/487405685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00e80d2bb0 0x7f00e81533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:43.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.490+0000 7f00e6d9d640 1 --2- 172.21.15.45:0/487405685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00e8163460 0x7f00e80cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:43.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.490+0000 7f00e659c640 1 -- 172.21.15.45:0/487405685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00e80d2bb0 msgr2=0x7f00e81533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:43.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.490+0000 7f00e659c640 1 --2- 172.21.15.45:0/487405685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00e80d2bb0 0x7f00e81533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:43.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.490+0000 7f00e659c640 1 -- 172.21.15.45:0/487405685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00e8163460 msgr2=0x7f00e80cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:43.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.490+0000 7f00e659c640 1 --2- 172.21.15.45:0/487405685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00e8163460 0x7f00e80cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:43.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.490+0000 7f00e659c640 1 -- 172.21.15.45:0/487405685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e80cc8e0 con 0x7f00e80b6590 2024-05-08T05:41:43.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.490+0000 7f00e5d9b640 1 --2- 172.21.15.45:0/487405685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00e80d2bb0 0x7f00e81533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:43.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.490+0000 7f00e6d9d640 1 --2- 172.21.15.45:0/487405685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00e8163460 0x7f00e80cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:43.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.490+0000 7f00e659c640 1 --2- 172.21.15.45:0/487405685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00e80b6590 0x7f00e80cc0b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009e50 tx=0x7f00d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:43.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.490+0000 7f00d6ffd640 1 -- 172.21.15.45:0/487405685 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8036050 con 0x7f00e80b6590 2024-05-08T05:41:43.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.490+0000 7f00d6ffd640 1 -- 172.21.15.45:0/487405685 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d8042070 con 0x7f00e80b6590 2024-05-08T05:41:43.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.490+0000 7f00ec94d640 1 -- 172.21.15.45:0/487405685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e80ccbc0 con 0x7f00e80b6590 2024-05-08T05:41:43.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.490+0000 7f00d6ffd640 1 -- 172.21.15.45:0/487405685 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8009510 con 0x7f00e80b6590 2024-05-08T05:41:43.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.490+0000 7f00ec94d640 1 -- 172.21.15.45:0/487405685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e80cd180 con 0x7f00e80b6590 2024-05-08T05:41:43.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.491+0000 7f00ec94d640 1 -- 172.21.15.45:0/487405685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e80693f0 con 0x7f00e80b6590 2024-05-08T05:41:43.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.491+0000 7f00d6ffd640 1 -- 172.21.15.45:0/487405685 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f00d8007c50 con 0x7f00e80b6590 2024-05-08T05:41:43.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.492+0000 7f00d6ffd640 1 --2- 172.21.15.45:0/487405685 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f00c80762d0 0x7f00c8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:43.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.492+0000 7f00d6ffd640 1 -- 172.21.15.45:0/487405685 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f00d80b88c0 con 0x7f00e80b6590 2024-05-08T05:41:43.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.492+0000 7f00e5d9b640 1 --2- 172.21.15.45:0/487405685 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f00c80762d0 0x7f00c8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:43.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.493+0000 7f00e5d9b640 1 --2- 172.21.15.45:0/487405685 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f00c80762d0 0x7f00c8078790 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f00e80b5e70 tx=0x7f00dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:43.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.496+0000 7f00d6ffd640 1 -- 172.21.15.45:0/487405685 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f00d8081ed0 con 0x7f00e80b6590 2024-05-08T05:41:43.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:43.725+0000 7f00ec94d640 1 -- 172.21.15.45:0/487405685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f00e80cc5f0 con 0x7f00e80b6590 2024-05-08T05:41:44.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.171+0000 7f00d6ffd640 1 -- 172.21.15.45:0/487405685 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f00d80817d0 con 0x7f00e80b6590 2024-05-08T05:41:44.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.173+0000 7f00ec94d640 1 -- 172.21.15.45:0/487405685 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f00c80762d0 msgr2=0x7f00c8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:44.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.173+0000 7f00ec94d640 1 --2- 172.21.15.45:0/487405685 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f00c80762d0 0x7f00c8078790 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f00e80b5e70 tx=0x7f00dc009040 comp rx=0 tx=0).stop 2024-05-08T05:41:44.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.173+0000 7f00ec94d640 1 -- 172.21.15.45:0/487405685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00e80b6590 msgr2=0x7f00e80cc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:44.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.173+0000 7f00ec94d640 1 --2- 172.21.15.45:0/487405685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00e80b6590 0x7f00e80cc0b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009e50 tx=0x7f00d8004a40 comp rx=0 tx=0).stop 2024-05-08T05:41:44.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.173+0000 7f00ec94d640 1 -- 172.21.15.45:0/487405685 shutdown_connections 2024-05-08T05:41:44.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.173+0000 7f00ec94d640 1 --2- 172.21.15.45:0/487405685 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f00c80762d0 0x7f00c8078790 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:44.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.173+0000 7f00ec94d640 1 --2- 172.21.15.45:0/487405685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f00e80d2bb0 0x7f00e81533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:44.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.173+0000 7f00ec94d640 1 --2- 172.21.15.45:0/487405685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00e8163460 0x7f00e80cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:44.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.173+0000 7f00ec94d640 1 --2- 172.21.15.45:0/487405685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00e80b6590 0x7f00e80cc0b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:44.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.173+0000 7f00ec94d640 1 -- 172.21.15.45:0/487405685 >> 172.21.15.45:0/487405685 conn(0x7f00e80d4fe0 msgr2=0x7f00e80d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:44.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.173+0000 7f00ec94d640 1 -- 172.21.15.45:0/487405685 shutdown_connections 2024-05-08T05:41:44.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.173+0000 7f00ec94d640 1 -- 172.21.15.45:0/487405685 wait complete. 2024-05-08T05:41:44.186 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-05-08T05:41:44.186 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T05:41:44.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.297+0000 7effa64c8640 1 Processor -- start 2024-05-08T05:41:44.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.297+0000 7effa64c8640 1 -- start start 2024-05-08T05:41:44.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.298+0000 7effa64c8640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7effa01639e0 0x7effa006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:44.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.298+0000 7effa64c8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa0160720 con 0x7effa00d2130 2024-05-08T05:41:44.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.298+0000 7effa64c8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa00d10d0 con 0x7effa0070d30 2024-05-08T05:41:44.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.298+0000 7effa64c8640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7effa0163e10 con 0x7effa01639e0 2024-05-08T05:41:44.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.298+0000 7eff9f7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7effa00d2130 0x7effa0071110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43908/0 (socket says 172.21.15.45:43908) 2024-05-08T05:41:44.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.298+0000 7eff9f7fe640 1 -- 172.21.15.45:0/458323705 learned_addr learned my addr 172.21.15.45:0/458323705 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:44.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.298+0000 7effa4a65640 1 --2- 172.21.15.45:0/458323705 >> v2:172.21.15.63:3300/0 conn(0x7effa01639e0 0x7effa006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:44.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.298+0000 7eff9effd640 1 -- 172.21.15.45:0/458323705 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1336068537 0 0) 0x7effa0160720 con 0x7effa00d2130 2024-05-08T05:41:44.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.298+0000 7eff9effd640 1 -- 172.21.15.45:0/458323705 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff88003520 con 0x7effa00d2130 2024-05-08T05:41:44.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.299+0000 7eff9effd640 1 -- 172.21.15.45:0/458323705 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3446845606 0 0) 0x7effa00d10d0 con 0x7effa0070d30 2024-05-08T05:41:44.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.299+0000 7eff9effd640 1 -- 172.21.15.45:0/458323705 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effa0160720 con 0x7effa0070d30 2024-05-08T05:41:44.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.299+0000 7effa4a65640 1 -- 172.21.15.45:0/458323705 >> v1:172.21.15.63:6789/0 conn(0x7effa0070d30 legacy=0x7effa00703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:44.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.299+0000 7effa4a65640 1 -- 172.21.15.45:0/458323705 >> v1:172.21.15.45:6789/0 conn(0x7effa00d2130 legacy=0x7effa0071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:44.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.299+0000 7eff9effd640 1 -- 172.21.15.45:0/458323705 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4230981535 0 0) 0x7eff88003520 con 0x7effa00d2130 2024-05-08T05:41:44.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.299+0000 7effa4a65640 1 -- 172.21.15.45:0/458323705 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa00d1370 con 0x7effa01639e0 2024-05-08T05:41:44.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.299+0000 7effa4a65640 1 --2- 172.21.15.45:0/458323705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7effa01639e0 0x7effa006d4b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7eff98009d90 tx=0x7eff9802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4383f21c094206a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:44.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.300+0000 7eff9effd640 1 -- 172.21.15.45:0/458323705 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff940016f0 con 0x7effa01639e0 2024-05-08T05:41:44.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.300+0000 7eff9effd640 1 -- 172.21.15.45:0/458323705 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff98007540 con 0x7effa01639e0 2024-05-08T05:41:44.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.300+0000 7eff9effd640 1 -- 172.21.15.45:0/458323705 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98007910 con 0x7effa01639e0 2024-05-08T05:41:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.300+0000 7effa64c8640 1 -- 172.21.15.45:0/458323705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7effa01639e0 msgr2=0x7effa006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.300+0000 7effa64c8640 1 --2- 172.21.15.45:0/458323705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7effa01639e0 0x7effa006d4b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7eff98009d90 tx=0x7eff9802a680 comp rx=0 tx=0).stop 2024-05-08T05:41:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.300+0000 7effa64c8640 1 -- 172.21.15.45:0/458323705 shutdown_connections 2024-05-08T05:41:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.300+0000 7effa64c8640 1 --2- 172.21.15.45:0/458323705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7effa01639e0 0x7effa006d4b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.300+0000 7effa64c8640 1 -- 172.21.15.45:0/458323705 >> 172.21.15.45:0/458323705 conn(0x7effa00d2e90 msgr2=0x7effa00d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.300+0000 7effa64c8640 1 -- 172.21.15.45:0/458323705 shutdown_connections 2024-05-08T05:41:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.300+0000 7effa64c8640 1 -- 172.21.15.45:0/458323705 wait complete. 2024-05-08T05:41:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.301+0000 7effa64c8640 1 Processor -- start 2024-05-08T05:41:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.302+0000 7effa64c8640 1 -- start start 2024-05-08T05:41:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.302+0000 7effa64c8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7effa0070d30 0x7effa00cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.302+0000 7effa64c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effa00d2130 0x7effa014b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:44.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.302+0000 7effa64c8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7effa01639e0 0x7effa014bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:44.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.302+0000 7effa64c8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7effa00d1930 con 0x7effa00d2130 2024-05-08T05:41:44.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.302+0000 7effa64c8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7effa00d1ca0 con 0x7effa01639e0 2024-05-08T05:41:44.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.302+0000 7effa64c8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7effa006d9f0 con 0x7effa0070d30 2024-05-08T05:41:44.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.302+0000 7eff9f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effa00d2130 0x7effa014b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:44.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.302+0000 7eff9f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effa00d2130 0x7effa014b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51142/0 (socket says 172.21.15.45:51142) 2024-05-08T05:41:44.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.302+0000 7eff9f7fe640 1 -- 172.21.15.45:0/2068621987 learned_addr learned my addr 172.21.15.45:0/2068621987 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:44.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.303+0000 7effa4a65640 1 --2- 172.21.15.45:0/2068621987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7effa01639e0 0x7effa014bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:44.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.303+0000 7eff9ffff640 1 --2- 172.21.15.45:0/2068621987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7effa0070d30 0x7effa00cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:44.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.303+0000 7eff9f7fe640 1 -- 172.21.15.45:0/2068621987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7effa0070d30 msgr2=0x7effa00cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:44.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.303+0000 7eff9f7fe640 1 --2- 172.21.15.45:0/2068621987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7effa0070d30 0x7effa00cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:44.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.303+0000 7eff9f7fe640 1 -- 172.21.15.45:0/2068621987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7effa01639e0 msgr2=0x7effa014bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:44.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.303+0000 7eff9f7fe640 1 --2- 172.21.15.45:0/2068621987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7effa01639e0 0x7effa014bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:44.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.303+0000 7eff9f7fe640 1 -- 172.21.15.45:0/2068621987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa014b7f0 con 0x7effa00d2130 2024-05-08T05:41:44.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.303+0000 7eff9ffff640 1 --2- 172.21.15.45:0/2068621987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7effa0070d30 0x7effa00cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:44.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.303+0000 7effa4a65640 1 --2- 172.21.15.45:0/2068621987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7effa01639e0 0x7effa014bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:44.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.303+0000 7eff9f7fe640 1 --2- 172.21.15.45:0/2068621987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effa00d2130 0x7effa014b2b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7eff9400c560 tx=0x7eff9400ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:44.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.303+0000 7eff9cff9640 1 -- 172.21.15.45:0/2068621987 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff94017070 con 0x7effa00d2130 2024-05-08T05:41:44.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.303+0000 7eff9cff9640 1 -- 172.21.15.45:0/2068621987 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff94005ce0 con 0x7effa00d2130 2024-05-08T05:41:44.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.303+0000 7effa64c8640 1 -- 172.21.15.45:0/2068621987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa01ea050 con 0x7effa00d2130 2024-05-08T05:41:44.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.303+0000 7eff9cff9640 1 -- 172.21.15.45:0/2068621987 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff94007290 con 0x7effa00d2130 2024-05-08T05:41:44.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.303+0000 7effa64c8640 1 -- 172.21.15.45:0/2068621987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa00ce210 con 0x7effa00d2130 2024-05-08T05:41:44.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.304+0000 7effa64c8640 1 -- 172.21.15.45:0/2068621987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa00693f0 con 0x7effa00d2130 2024-05-08T05:41:44.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.305+0000 7eff9cff9640 1 -- 172.21.15.45:0/2068621987 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7eff9401a070 con 0x7effa00d2130 2024-05-08T05:41:44.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.305+0000 7eff9cff9640 1 --2- 172.21.15.45:0/2068621987 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7eff8c0762d0 0x7eff8c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:44.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.305+0000 7eff9cff9640 1 -- 172.21.15.45:0/2068621987 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7eff94096e60 con 0x7effa00d2130 2024-05-08T05:41:44.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.305+0000 7eff9ffff640 1 --2- 172.21.15.45:0/2068621987 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7eff8c0762d0 0x7eff8c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:44.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.306+0000 7eff9ffff640 1 --2- 172.21.15.45:0/2068621987 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7eff8c0762d0 0x7eff8c078790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff90001710 tx=0x7eff90009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:44.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.310+0000 7eff9cff9640 1 -- 172.21.15.45:0/2068621987 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7effa014b7f0 con 0x7effa00d2130 2024-05-08T05:41:44.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:44.542+0000 7effa64c8640 1 -- 172.21.15.45:0/2068621987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7effa00cd1c0 con 0x7effa00d2130 2024-05-08T05:41:45.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.183+0000 7eff9cff9640 1 -- 172.21.15.45:0/2068621987 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7eff94060490 con 0x7effa00d2130 2024-05-08T05:41:45.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.185+0000 7effa64c8640 1 -- 172.21.15.45:0/2068621987 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7eff8c0762d0 msgr2=0x7eff8c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:45.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.185+0000 7effa64c8640 1 --2- 172.21.15.45:0/2068621987 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7eff8c0762d0 0x7eff8c078790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff90001710 tx=0x7eff90009040 comp rx=0 tx=0).stop 2024-05-08T05:41:45.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.185+0000 7effa64c8640 1 -- 172.21.15.45:0/2068621987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effa00d2130 msgr2=0x7effa014b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:45.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.185+0000 7effa64c8640 1 --2- 172.21.15.45:0/2068621987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effa00d2130 0x7effa014b2b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7eff9400c560 tx=0x7eff9400ca20 comp rx=0 tx=0).stop 2024-05-08T05:41:45.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.185+0000 7effa64c8640 1 -- 172.21.15.45:0/2068621987 shutdown_connections 2024-05-08T05:41:45.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.185+0000 7effa64c8640 1 --2- 172.21.15.45:0/2068621987 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7eff8c0762d0 0x7eff8c078790 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.185+0000 7effa64c8640 1 --2- 172.21.15.45:0/2068621987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7effa01639e0 0x7effa014bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.185+0000 7effa64c8640 1 --2- 172.21.15.45:0/2068621987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effa00d2130 0x7effa014b2b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.185+0000 7effa64c8640 1 --2- 172.21.15.45:0/2068621987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7effa0070d30 0x7effa00cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.186+0000 7effa64c8640 1 -- 172.21.15.45:0/2068621987 >> 172.21.15.45:0/2068621987 conn(0x7effa00d2e90 msgr2=0x7effa006ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:45.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.186+0000 7effa64c8640 1 -- 172.21.15.45:0/2068621987 shutdown_connections 2024-05-08T05:41:45.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.186+0000 7effa64c8640 1 -- 172.21.15.45:0/2068621987 wait complete. 2024-05-08T05:41:45.199 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T05:41:45.200 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:41:45.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.309+0000 7f89ab4c4640 1 Processor -- start 2024-05-08T05:41:45.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.309+0000 7f89ab4c4640 1 -- start start 2024-05-08T05:41:45.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.310+0000 7f89ab4c4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f89a4070d30 0x7f89a40d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:45.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.310+0000 7f89ab4c4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89a4163920 0x7f89a40703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:45.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.310+0000 7f89ab4c4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f89a4071110 0x7f89a406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:45.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.310+0000 7f89ab4c4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f89a40d10d0 con 0x7f89a4071110 2024-05-08T05:41:45.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.310+0000 7f89ab4c4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f89a4070b80 con 0x7f89a4070d30 2024-05-08T05:41:45.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.310+0000 7f89ab4c4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f89a40d1c30 con 0x7f89a4163920 2024-05-08T05:41:45.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.310+0000 7f89a9a61640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f89a4071110 0x7f89a406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:45.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.310+0000 7f89a9a61640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f89a4071110 0x7f89a406d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51158/0 (socket says 172.21.15.45:51158) 2024-05-08T05:41:45.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.310+0000 7f89a9a61640 1 -- 172.21.15.45:0/1419480590 learned_addr learned my addr 172.21.15.45:0/1419480590 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:45.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.311+0000 7f89a9a61640 1 -- 172.21.15.45:0/1419480590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89a4163920 msgr2=0x7f89a40703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:45.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.311+0000 7f89a9a61640 1 --2- 172.21.15.45:0/1419480590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89a4163920 0x7f89a40703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.311+0000 7f89a9a61640 1 -- 172.21.15.45:0/1419480590 >> v2:172.21.15.63:3300/0 conn(0x7f89a4070d30 msgr2=0x7f89a40d20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:45.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.311+0000 7f89a9a61640 1 --2- 172.21.15.45:0/1419480590 >> v2:172.21.15.63:3300/0 conn(0x7f89a4070d30 0x7f89a40d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.311+0000 7f89a9a61640 1 -- 172.21.15.45:0/1419480590 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a4163d80 con 0x7f89a4071110 2024-05-08T05:41:45.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.311+0000 7f89a9a61640 1 --2- 172.21.15.45:0/1419480590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a4071110 0x7f89a406d4b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f89a40d3f90 tx=0x7f899c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff5b32ab7b59056 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:45.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.311+0000 7f899bfff640 1 -- 172.21.15.45:0/1419480590 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c036070 con 0x7f89a4071110 2024-05-08T05:41:45.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.311+0000 7f899bfff640 1 -- 172.21.15.45:0/1419480590 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f899c031600 con 0x7f89a4071110 2024-05-08T05:41:45.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.311+0000 7f899bfff640 1 -- 172.21.15.45:0/1419480590 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c0319d0 con 0x7f89a4071110 2024-05-08T05:41:45.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.312+0000 7f89ab4c4640 1 -- 172.21.15.45:0/1419480590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a4071110 msgr2=0x7f89a406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:45.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.312+0000 7f89ab4c4640 1 --2- 172.21.15.45:0/1419480590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a4071110 0x7f89a406d4b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f89a40d3f90 tx=0x7f899c02a6a0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.312+0000 7f89ab4c4640 1 -- 172.21.15.45:0/1419480590 shutdown_connections 2024-05-08T05:41:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.312+0000 7f89ab4c4640 1 --2- 172.21.15.45:0/1419480590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a4071110 0x7f89a406d4b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.312+0000 7f89ab4c4640 1 --2- 172.21.15.45:0/1419480590 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89a4163920 0x7f89a40703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.312+0000 7f89ab4c4640 1 --2- 172.21.15.45:0/1419480590 >> v2:172.21.15.63:3300/0 conn(0x7f89a4070d30 0x7f89a40d20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.312+0000 7f89ab4c4640 1 -- 172.21.15.45:0/1419480590 >> 172.21.15.45:0/1419480590 conn(0x7f89a40d2e90 msgr2=0x7f89a40d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.312+0000 7f89ab4c4640 1 -- 172.21.15.45:0/1419480590 shutdown_connections 2024-05-08T05:41:45.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.312+0000 7f89ab4c4640 1 -- 172.21.15.45:0/1419480590 wait complete. 2024-05-08T05:41:45.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.313+0000 7f89ab4c4640 1 Processor -- start 2024-05-08T05:41:45.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.313+0000 7f89ab4c4640 1 -- start start 2024-05-08T05:41:45.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.314+0000 7f89ab4c4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89a4070d30 0x7f89a40b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:45.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.314+0000 7f89ab4c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a4071110 0x7f89a4154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:45.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.314+0000 7f89ab4c4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89a4163920 0x7f89a40b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:45.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.314+0000 7f89ab4c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f89a40d15e0 con 0x7f89a4071110 2024-05-08T05:41:45.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.314+0000 7f89ab4c4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f89a40708e0 con 0x7f89a4070d30 2024-05-08T05:41:45.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.314+0000 7f89ab4c4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f89a40d18c0 con 0x7f89a4163920 2024-05-08T05:41:45.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.314+0000 7f89a8a5f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a4071110 0x7f89a4154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:45.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.314+0000 7f89a8a5f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a4071110 0x7f89a4154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51174/0 (socket says 172.21.15.45:51174) 2024-05-08T05:41:45.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.314+0000 7f89a8a5f640 1 -- 172.21.15.45:0/2148541586 learned_addr learned my addr 172.21.15.45:0/2148541586 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:45.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.314+0000 7f89a9a61640 1 --2- 172.21.15.45:0/2148541586 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89a4163920 0x7f89a40b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:45.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.314+0000 7f89a9260640 1 --2- 172.21.15.45:0/2148541586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89a4070d30 0x7f89a40b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:45.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.314+0000 7f89a8a5f640 1 -- 172.21.15.45:0/2148541586 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89a4163920 msgr2=0x7f89a40b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:45.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.314+0000 7f89a8a5f640 1 --2- 172.21.15.45:0/2148541586 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89a4163920 0x7f89a40b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.315+0000 7f89a8a5f640 1 -- 172.21.15.45:0/2148541586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89a4070d30 msgr2=0x7f89a40b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:45.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.315+0000 7f89a8a5f640 1 --2- 172.21.15.45:0/2148541586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89a4070d30 0x7f89a40b2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.315+0000 7f89a8a5f640 1 -- 172.21.15.45:0/2148541586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a40b8320 con 0x7f89a4071110 2024-05-08T05:41:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.315+0000 7f89a9260640 1 --2- 172.21.15.45:0/2148541586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89a4070d30 0x7f89a40b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.315+0000 7f89a8a5f640 1 --2- 172.21.15.45:0/2148541586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a4071110 0x7f89a4154010 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f899400b4a0 tx=0x7f899400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.315+0000 7f8999ffb640 1 -- 172.21.15.45:0/2148541586 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8994014070 con 0x7f89a4071110 2024-05-08T05:41:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.315+0000 7f8999ffb640 1 -- 172.21.15.45:0/2148541586 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8994010610 con 0x7f89a4071110 2024-05-08T05:41:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.315+0000 7f89ab4c4640 1 -- 172.21.15.45:0/2148541586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a40b8600 con 0x7f89a4071110 2024-05-08T05:41:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.315+0000 7f8999ffb640 1 -- 172.21.15.45:0/2148541586 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8994010a00 con 0x7f89a4071110 2024-05-08T05:41:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.315+0000 7f89ab4c4640 1 -- 172.21.15.45:0/2148541586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a40cb0e0 con 0x7f89a4071110 2024-05-08T05:41:45.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.316+0000 7f89ab4c4640 1 -- 172.21.15.45:0/2148541586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a40d2800 con 0x7f89a4071110 2024-05-08T05:41:45.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.317+0000 7f8999ffb640 1 -- 172.21.15.45:0/2148541586 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8994017070 con 0x7f89a4071110 2024-05-08T05:41:45.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.317+0000 7f8999ffb640 1 --2- 172.21.15.45:0/2148541586 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f89880762d0 0x7f8988078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:45.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.317+0000 7f8999ffb640 1 -- 172.21.15.45:0/2148541586 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8994097200 con 0x7f89a4071110 2024-05-08T05:41:45.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.317+0000 7f89a9260640 1 --2- 172.21.15.45:0/2148541586 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f89880762d0 0x7f8988078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:45.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.318+0000 7f89a9260640 1 --2- 172.21.15.45:0/2148541586 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f89880762d0 0x7f8988078790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009890 tx=0x7f89a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:45.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.322+0000 7f8999ffb640 1 -- 172.21.15.45:0/2148541586 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f89a40d2800 con 0x7f89a4071110 2024-05-08T05:41:45.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.551+0000 7f89ab4c4640 1 -- 172.21.15.45:0/2148541586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89a40693f0 con 0x7f89a4071110 2024-05-08T05:41:45.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.552+0000 7f8999ffb640 1 -- 172.21.15.45:0/2148541586 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5174 (secure 0 0 0) 0x7f8994060260 con 0x7f89a4071110 2024-05-08T05:41:45.553 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:45.553 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2346958825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2346958825},{"type":"v1","addr":"172.21.15.45:6835","nonce":2346958825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/709809719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":709809719},{"type":"v1","addr":"172.21.15.45:6833","nonce":709809719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3208180302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3208180302},{"type":"v1","addr":"172.21.15.63:6836","nonce":3208180302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:41:31.494380+0000","modified":"2024-05-08T05:41:44.544082+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6837/3653698989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":3653698989},{"type":"v1","addr":"172.21.15.63:6837","nonce":3653698989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4303,"qdb_cluster":[4303]},"id":1}]} 2024-05-08T05:41:45.553 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-05-08T05:41:45.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.554+0000 7f89ab4c4640 1 -- 172.21.15.45:0/2148541586 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f89880762d0 msgr2=0x7f8988078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:45.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.554+0000 7f89ab4c4640 1 --2- 172.21.15.45:0/2148541586 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f89880762d0 0x7f8988078790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009890 tx=0x7f89a0008040 comp rx=0 tx=0).stop 2024-05-08T05:41:45.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.554+0000 7f89ab4c4640 1 -- 172.21.15.45:0/2148541586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a4071110 msgr2=0x7f89a4154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:45.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.554+0000 7f89ab4c4640 1 --2- 172.21.15.45:0/2148541586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a4071110 0x7f89a4154010 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f899400b4a0 tx=0x7f899400b960 comp rx=0 tx=0).stop 2024-05-08T05:41:45.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.555+0000 7f89ab4c4640 1 -- 172.21.15.45:0/2148541586 shutdown_connections 2024-05-08T05:41:45.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.555+0000 7f89ab4c4640 1 --2- 172.21.15.45:0/2148541586 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f89880762d0 0x7f8988078790 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.555+0000 7f89ab4c4640 1 --2- 172.21.15.45:0/2148541586 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89a4163920 0x7f89a40b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.555+0000 7f89ab4c4640 1 --2- 172.21.15.45:0/2148541586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89a4071110 0x7f89a4154010 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.555+0000 7f89ab4c4640 1 --2- 172.21.15.45:0/2148541586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89a4070d30 0x7f89a40b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.555+0000 7f89ab4c4640 1 -- 172.21.15.45:0/2148541586 >> 172.21.15.45:0/2148541586 conn(0x7f89a40d2e90 msgr2=0x7f89a4162f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:45.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.555+0000 7f89ab4c4640 1 -- 172.21.15.45:0/2148541586 shutdown_connections 2024-05-08T05:41:45.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.555+0000 7f89ab4c4640 1 -- 172.21.15.45:0/2148541586 wait complete. 2024-05-08T05:41:45.566 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:41:45.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.670+0000 7f76b7ec8640 1 Processor -- start 2024-05-08T05:41:45.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.670+0000 7f76b7ec8640 1 -- start start 2024-05-08T05:41:45.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.671+0000 7f76b7ec8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76b015f420 0x7f76b00d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:45.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.671+0000 7f76b7ec8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f76b00d3ca0 0x7f76b00d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:45.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.671+0000 7f76b7ec8640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f76b00d1eb0 0x7f76b00d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:45.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.671+0000 7f76b7ec8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f76b00d22f0 con 0x7f76b00d3ca0 2024-05-08T05:41:45.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.671+0000 7f76b7ec8640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f76b01630b0 con 0x7f76b00d1eb0 2024-05-08T05:41:45.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.671+0000 7f76b7ec8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f76b00d2550 con 0x7f76b015f420 2024-05-08T05:41:45.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.671+0000 7f76b5463640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f76b00d3ca0 0x7f76b00d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:45.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.671+0000 7f76b5463640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f76b00d3ca0 0x7f76b00d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51184/0 (socket says 172.21.15.45:51184) 2024-05-08T05:41:45.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.671+0000 7f76b5463640 1 -- 172.21.15.45:0/809037758 learned_addr learned my addr 172.21.15.45:0/809037758 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:45.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.672+0000 7f76b6465640 1 --2- 172.21.15.45:0/809037758 >> v2:172.21.15.63:3300/0 conn(0x7f76b00d1eb0 0x7f76b00d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:45.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.672+0000 7f76b5c64640 1 --2- 172.21.15.45:0/809037758 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76b015f420 0x7f76b00d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:45.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.672+0000 7f76b5463640 1 -- 172.21.15.45:0/809037758 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76b015f420 msgr2=0x7f76b00d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:45.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.672+0000 7f76b5463640 1 --2- 172.21.15.45:0/809037758 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76b015f420 0x7f76b00d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.672+0000 7f76b5463640 1 -- 172.21.15.45:0/809037758 >> v2:172.21.15.63:3300/0 conn(0x7f76b00d1eb0 msgr2=0x7f76b00d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:45.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.672+0000 7f76b5463640 1 --2- 172.21.15.45:0/809037758 >> v2:172.21.15.63:3300/0 conn(0x7f76b00d1eb0 0x7f76b00d4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.672+0000 7f76b5463640 1 -- 172.21.15.45:0/809037758 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b00d2c80 con 0x7f76b00d3ca0 2024-05-08T05:41:45.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.672+0000 7f76b5c64640 1 --2- 172.21.15.45:0/809037758 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76b015f420 0x7f76b00d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:45.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.672+0000 7f76b5463640 1 --2- 172.21.15.45:0/809037758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76b00d3ca0 0x7f76b00d1970 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f76b00d17a0 tx=0x7f76a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6deb0d410f40d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:45.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.673+0000 7f76b4c62640 1 -- 172.21.15.45:0/809037758 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a4037070 con 0x7f76b00d3ca0 2024-05-08T05:41:45.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.673+0000 7f76b4c62640 1 -- 172.21.15.45:0/809037758 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a4032440 con 0x7f76b00d3ca0 2024-05-08T05:41:45.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.673+0000 7f76b4c62640 1 -- 172.21.15.45:0/809037758 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a4032810 con 0x7f76b00d3ca0 2024-05-08T05:41:45.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.673+0000 7f76b7ec8640 1 -- 172.21.15.45:0/809037758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76b00d3ca0 msgr2=0x7f76b00d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:45.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.673+0000 7f76b7ec8640 1 --2- 172.21.15.45:0/809037758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76b00d3ca0 0x7f76b00d1970 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f76b00d17a0 tx=0x7f76a402a1e0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.673+0000 7f76b7ec8640 1 -- 172.21.15.45:0/809037758 shutdown_connections 2024-05-08T05:41:45.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.673+0000 7f76b7ec8640 1 --2- 172.21.15.45:0/809037758 >> v2:172.21.15.63:3300/0 conn(0x7f76b00d1eb0 0x7f76b00d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.673+0000 7f76b7ec8640 1 --2- 172.21.15.45:0/809037758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76b00d3ca0 0x7f76b00d1970 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.673+0000 7f76b7ec8640 1 --2- 172.21.15.45:0/809037758 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76b015f420 0x7f76b00d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.673+0000 7f76b7ec8640 1 -- 172.21.15.45:0/809037758 >> 172.21.15.45:0/809037758 conn(0x7f76b008eac0 msgr2=0x7f76b009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.673+0000 7f76b7ec8640 1 -- 172.21.15.45:0/809037758 shutdown_connections 2024-05-08T05:41:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.673+0000 7f76b7ec8640 1 -- 172.21.15.45:0/809037758 wait complete. 2024-05-08T05:41:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.675+0000 7f76b7ec8640 1 Processor -- start 2024-05-08T05:41:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.675+0000 7f76b7ec8640 1 -- start start 2024-05-08T05:41:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.675+0000 7f76b7ec8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76b00d1eb0 0x7f76b0159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.675+0000 7f76b7ec8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b015a7a0 0x7f76b0148290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.675+0000 7f76b7ec8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76b01487d0 0x7f76b0148c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.675+0000 7f76b7ec8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76b00d2830 con 0x7f76b01487d0 2024-05-08T05:41:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.675+0000 7f76b7ec8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76b00d3240 con 0x7f76b015a7a0 2024-05-08T05:41:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.675+0000 7f76b7ec8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f76b00d5200 con 0x7f76b00d1eb0 2024-05-08T05:41:45.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.676+0000 7f76b6465640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76b01487d0 0x7f76b0148c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:45.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.676+0000 7f76b6465640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76b01487d0 0x7f76b0148c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51198/0 (socket says 172.21.15.45:51198) 2024-05-08T05:41:45.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.676+0000 7f76b5463640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b015a7a0 0x7f76b0148290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:45.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.676+0000 7f76b6465640 1 -- 172.21.15.45:0/1100156023 learned_addr learned my addr 172.21.15.45:0/1100156023 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:45.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.676+0000 7f76b5c64640 1 --2- 172.21.15.45:0/1100156023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76b00d1eb0 0x7f76b0159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:45.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.676+0000 7f76b6465640 1 -- 172.21.15.45:0/1100156023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76b00d1eb0 msgr2=0x7f76b0159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:45.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.676+0000 7f76b6465640 1 --2- 172.21.15.45:0/1100156023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76b00d1eb0 0x7f76b0159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.676+0000 7f76b6465640 1 -- 172.21.15.45:0/1100156023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b015a7a0 msgr2=0x7f76b0148290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:45.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.676+0000 7f76b6465640 1 --2- 172.21.15.45:0/1100156023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b015a7a0 0x7f76b0148290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.676+0000 7f76b6465640 1 -- 172.21.15.45:0/1100156023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b015c0e0 con 0x7f76b01487d0 2024-05-08T05:41:45.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.676+0000 7f76b6465640 1 --2- 172.21.15.45:0/1100156023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76b01487d0 0x7f76b0148c50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f76a000ef00 tx=0x7f76a000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:45.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.676+0000 7f76ae7fc640 1 -- 172.21.15.45:0/1100156023 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0018020 con 0x7f76b01487d0 2024-05-08T05:41:45.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.676+0000 7f76ae7fc640 1 -- 172.21.15.45:0/1100156023 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a0014400 con 0x7f76b01487d0 2024-05-08T05:41:45.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.676+0000 7f76b7ec8640 1 -- 172.21.15.45:0/1100156023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b015c3c0 con 0x7f76b01487d0 2024-05-08T05:41:45.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.676+0000 7f76ae7fc640 1 -- 172.21.15.45:0/1100156023 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00147f0 con 0x7f76b01487d0 2024-05-08T05:41:45.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.676+0000 7f76b7ec8640 1 -- 172.21.15.45:0/1100156023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b015c980 con 0x7f76b01487d0 2024-05-08T05:41:45.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.677+0000 7f76b7ec8640 1 -- 172.21.15.45:0/1100156023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b00693f0 con 0x7f76b01487d0 2024-05-08T05:41:45.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.678+0000 7f76ae7fc640 1 -- 172.21.15.45:0/1100156023 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f76a0005d10 con 0x7f76b01487d0 2024-05-08T05:41:45.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.678+0000 7f76ae7fc640 1 --2- 172.21.15.45:0/1100156023 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f769c0762d0 0x7f769c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:45.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.679+0000 7f76ae7fc640 1 -- 172.21.15.45:0/1100156023 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f76a0098140 con 0x7f76b01487d0 2024-05-08T05:41:45.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.679+0000 7f76b5c64640 1 --2- 172.21.15.45:0/1100156023 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f769c0762d0 0x7f769c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:45.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.679+0000 7f76b5c64640 1 --2- 172.21.15.45:0/1100156023 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f769c0762d0 0x7f769c078790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f76a8006fd0 tx=0x7f76a8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:45.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.683+0000 7f76ae7fc640 1 -- 172.21.15.45:0/1100156023 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f76a0022030 con 0x7f76b01487d0 2024-05-08T05:41:45.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.874+0000 7f76b7ec8640 1 -- 172.21.15.45:0/1100156023 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76b00d1c90 con 0x7f76b01487d0 2024-05-08T05:41:45.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.876+0000 7f76ae7fc640 1 -- 172.21.15.45:0/1100156023 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18466 (secure 0 0 0) 0x7f76a0061100 con 0x7f76b01487d0 2024-05-08T05:41:45.876 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:45.876 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:41:36.532045+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:41:29.381274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:41:30.443612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:41:34.132285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:41:45.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.878+0000 7f76b7ec8640 1 -- 172.21.15.45:0/1100156023 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f769c0762d0 msgr2=0x7f769c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:45.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.878+0000 7f76b7ec8640 1 --2- 172.21.15.45:0/1100156023 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f769c0762d0 0x7f769c078790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f76a8006fd0 tx=0x7f76a8008040 comp rx=0 tx=0).stop 2024-05-08T05:41:45.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.878+0000 7f76b7ec8640 1 -- 172.21.15.45:0/1100156023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76b01487d0 msgr2=0x7f76b0148c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:45.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.878+0000 7f76b7ec8640 1 --2- 172.21.15.45:0/1100156023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76b01487d0 0x7f76b0148c50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f76a000ef00 tx=0x7f76a000b480 comp rx=0 tx=0).stop 2024-05-08T05:41:45.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.878+0000 7f76b7ec8640 1 -- 172.21.15.45:0/1100156023 shutdown_connections 2024-05-08T05:41:45.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.878+0000 7f76b7ec8640 1 --2- 172.21.15.45:0/1100156023 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f769c0762d0 0x7f769c078790 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.878+0000 7f76b7ec8640 1 --2- 172.21.15.45:0/1100156023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76b01487d0 0x7f76b0148c50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.878+0000 7f76b7ec8640 1 --2- 172.21.15.45:0/1100156023 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76b015a7a0 0x7f76b0148290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.878+0000 7f76b7ec8640 1 --2- 172.21.15.45:0/1100156023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76b00d1eb0 0x7f76b0159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.878+0000 7f76b7ec8640 1 -- 172.21.15.45:0/1100156023 >> 172.21.15.45:0/1100156023 conn(0x7f76b008eac0 msgr2=0x7f76b00b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:45.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.878+0000 7f76b7ec8640 1 -- 172.21.15.45:0/1100156023 shutdown_connections 2024-05-08T05:41:45.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.878+0000 7f76b7ec8640 1 -- 172.21.15.45:0/1100156023 wait complete. 2024-05-08T05:41:45.889 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T05:41:45.890 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:41:45.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.992+0000 7fb6e3989640 1 Processor -- start 2024-05-08T05:41:45.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.993+0000 7fb6e3989640 1 -- start start 2024-05-08T05:41:45.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.993+0000 7fb6e3989640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb6dc0d0ec0 0x7fb6dc0d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:45.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.993+0000 7fb6e3989640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb6dc0d3b00 0x7fb6dc0b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:45.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.993+0000 7fb6e3989640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6dc0d1760 con 0x7fb6dc15f380 2024-05-08T05:41:45.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.993+0000 7fb6e3989640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb6dc0d30f0 con 0x7fb6dc0d0ec0 2024-05-08T05:41:45.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.993+0000 7fb6e3989640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb6dc0d2370 con 0x7fb6dc0d3b00 2024-05-08T05:41:45.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.993+0000 7fb6e0f24640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb6dc0d0ec0 0x7fb6dc0d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:45.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.993+0000 7fb6e1725640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb6dc15f380 0x7fb6dc0d07b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43910/0 (socket says 172.21.15.45:43910) 2024-05-08T05:41:45.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.994+0000 7fb6e1725640 1 -- 172.21.15.45:0/444173389 learned_addr learned my addr 172.21.15.45:0/444173389 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:45.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.994+0000 7fb6e1f26640 1 --2- 172.21.15.45:0/444173389 >> v2:172.21.15.63:3300/0 conn(0x7fb6dc0d3b00 0x7fb6dc0b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:45.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.994+0000 7fb6e0f24640 1 -- 172.21.15.45:0/444173389 >> v2:172.21.15.63:3300/0 conn(0x7fb6dc0d3b00 msgr2=0x7fb6dc0b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:45.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.994+0000 7fb6e0f24640 1 --2- 172.21.15.45:0/444173389 >> v2:172.21.15.63:3300/0 conn(0x7fb6dc0d3b00 0x7fb6dc0b7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.994+0000 7fb6e0f24640 1 -- 172.21.15.45:0/444173389 >> v1:172.21.15.45:6789/0 conn(0x7fb6dc15f380 legacy=0x7fb6dc0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:45.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.994+0000 7fb6cffff640 1 -- 172.21.15.45:0/444173389 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1939085517 0 0) 0x7fb6dc0d1760 con 0x7fb6dc15f380 2024-05-08T05:41:45.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.994+0000 7fb6e0f24640 1 -- 172.21.15.45:0/444173389 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6dc0d1a40 con 0x7fb6dc0d0ec0 2024-05-08T05:41:45.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.994+0000 7fb6e1f26640 1 --2- 172.21.15.45:0/444173389 >> v2:172.21.15.63:3300/0 conn(0x7fb6dc0d3b00 0x7fb6dc0b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:45.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.995+0000 7fb6e0f24640 1 --2- 172.21.15.45:0/444173389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6dc0d0ec0 0x7fb6dc0d35c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc163f60 tx=0x7fb6d002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b3a9fd9defefb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:45.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.995+0000 7fb6cffff640 1 -- 172.21.15.45:0/444173389 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0037070 con 0x7fb6dc0d0ec0 2024-05-08T05:41:45.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.995+0000 7fb6cffff640 1 -- 172.21.15.45:0/444173389 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6d0032440 con 0x7fb6dc0d0ec0 2024-05-08T05:41:45.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.995+0000 7fb6cffff640 1 -- 172.21.15.45:0/444173389 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0032810 con 0x7fb6dc0d0ec0 2024-05-08T05:41:45.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.995+0000 7fb6e3989640 1 -- 172.21.15.45:0/444173389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6dc0d0ec0 msgr2=0x7fb6dc0d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:45.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.995+0000 7fb6e3989640 1 --2- 172.21.15.45:0/444173389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6dc0d0ec0 0x7fb6dc0d35c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc163f60 tx=0x7fb6d002a1a0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.995+0000 7fb6e3989640 1 -- 172.21.15.45:0/444173389 shutdown_connections 2024-05-08T05:41:45.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.995+0000 7fb6e3989640 1 --2- 172.21.15.45:0/444173389 >> v2:172.21.15.63:3300/0 conn(0x7fb6dc0d3b00 0x7fb6dc0b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.995+0000 7fb6e3989640 1 --2- 172.21.15.45:0/444173389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6dc0d0ec0 0x7fb6dc0d35c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:45.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.995+0000 7fb6e3989640 1 -- 172.21.15.45:0/444173389 >> 172.21.15.45:0/444173389 conn(0x7fb6dc0d4fe0 msgr2=0x7fb6dc0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:45.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.995+0000 7fb6e3989640 1 -- 172.21.15.45:0/444173389 shutdown_connections 2024-05-08T05:41:45.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.995+0000 7fb6e3989640 1 -- 172.21.15.45:0/444173389 wait complete. 2024-05-08T05:41:45.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.996+0000 7fb6e3989640 1 Processor -- start 2024-05-08T05:41:45.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.996+0000 7fb6e3989640 1 -- start start 2024-05-08T05:41:46.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.997+0000 7fb6e3989640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6dc0d0ec0 0x7fb6dc0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:46.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.997+0000 7fb6e3989640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6dc0d3b00 0x7fb6dc15f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:46.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.997+0000 7fb6e3989640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6dc15f380 0x7fb6dc14e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:46.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.997+0000 7fb6e3989640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc0d2000 con 0x7fb6dc15f380 2024-05-08T05:41:46.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.997+0000 7fb6e3989640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc0d3f30 con 0x7fb6dc0d3b00 2024-05-08T05:41:46.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.997+0000 7fb6e3989640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb6dc15f760 con 0x7fb6dc0d0ec0 2024-05-08T05:41:46.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.997+0000 7fb6e1f26640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6dc15f380 0x7fb6dc14e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:46.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.997+0000 7fb6e1f26640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6dc15f380 0x7fb6dc14e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51212/0 (socket says 172.21.15.45:51212) 2024-05-08T05:41:46.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.997+0000 7fb6e1f26640 1 -- 172.21.15.45:0/273439775 learned_addr learned my addr 172.21.15.45:0/273439775 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:46.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.997+0000 7fb6e0f24640 1 --2- 172.21.15.45:0/273439775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6dc0d3b00 0x7fb6dc15f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:46.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.998+0000 7fb6e1725640 1 --2- 172.21.15.45:0/273439775 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6dc0d0ec0 0x7fb6dc0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:46.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.998+0000 7fb6e1f26640 1 -- 172.21.15.45:0/273439775 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6dc0d0ec0 msgr2=0x7fb6dc0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:46.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.998+0000 7fb6e1f26640 1 --2- 172.21.15.45:0/273439775 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6dc0d0ec0 0x7fb6dc0ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.998+0000 7fb6e1f26640 1 -- 172.21.15.45:0/273439775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6dc0d3b00 msgr2=0x7fb6dc15f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:46.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.998+0000 7fb6e1f26640 1 --2- 172.21.15.45:0/273439775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6dc0d3b00 0x7fb6dc15f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.998+0000 7fb6e1f26640 1 -- 172.21.15.45:0/273439775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6dc09fda0 con 0x7fb6dc15f380 2024-05-08T05:41:46.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.998+0000 7fb6e0f24640 1 --2- 172.21.15.45:0/273439775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6dc0d3b00 0x7fb6dc15f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:46.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.998+0000 7fb6e1f26640 1 --2- 172.21.15.45:0/273439775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6dc15f380 0x7fb6dc14e000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4007ef0 tx=0x7fb6d400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:46.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.998+0000 7fb6cdffb640 1 -- 172.21.15.45:0/273439775 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d401a070 con 0x7fb6dc15f380 2024-05-08T05:41:46.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.998+0000 7fb6cdffb640 1 -- 172.21.15.45:0/273439775 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6d4003b90 con 0x7fb6dc15f380 2024-05-08T05:41:46.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.998+0000 7fb6e3989640 1 -- 172.21.15.45:0/273439775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6dc09ed00 con 0x7fb6dc15f380 2024-05-08T05:41:46.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.998+0000 7fb6cdffb640 1 -- 172.21.15.45:0/273439775 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4004030 con 0x7fb6dc15f380 2024-05-08T05:41:46.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.998+0000 7fb6e3989640 1 -- 172.21.15.45:0/273439775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6dc09f2c0 con 0x7fb6dc15f380 2024-05-08T05:41:46.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:45.999+0000 7fb6e3989640 1 -- 172.21.15.45:0/273439775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6dc0693f0 con 0x7fb6dc15f380 2024-05-08T05:41:46.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.000+0000 7fb6cdffb640 1 -- 172.21.15.45:0/273439775 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb6d401d070 con 0x7fb6dc15f380 2024-05-08T05:41:46.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.000+0000 7fb6cdffb640 1 --2- 172.21.15.45:0/273439775 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb6c00762d0 0x7fb6c0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:46.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.000+0000 7fb6cdffb640 1 -- 172.21.15.45:0/273439775 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb6d4097b50 con 0x7fb6dc15f380 2024-05-08T05:41:46.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.000+0000 7fb6e1725640 1 --2- 172.21.15.45:0/273439775 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb6c00762d0 0x7fb6c0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:46.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.001+0000 7fb6e1725640 1 --2- 172.21.15.45:0/273439775 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb6c00762d0 0x7fb6c0078790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8000f60 tx=0x7fb6d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:46.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.005+0000 7fb6cdffb640 1 -- 172.21.15.45:0/273439775 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb6dc09fda0 con 0x7fb6dc15f380 2024-05-08T05:41:46.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.194+0000 7fb6e3989640 1 -- 172.21.15.45:0/273439775 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb6dc0d37d0 con 0x7fb6c00762d0 2024-05-08T05:41:46.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.205+0000 7fb6cdffb640 1 -- 172.21.15.45:0/273439775 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332563 (secure 0 0 0) 0x7fb6dc0d37d0 con 0x7fb6c00762d0 2024-05-08T05:41:46.209 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:46.211 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:41:46.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.208+0000 7fb6e3989640 1 -- 172.21.15.45:0/273439775 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb6c00762d0 msgr2=0x7fb6c0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:46.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.208+0000 7fb6e3989640 1 --2- 172.21.15.45:0/273439775 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb6c00762d0 0x7fb6c0078790 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8000f60 tx=0x7fb6d8009040 comp rx=0 tx=0).stop 2024-05-08T05:41:46.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.208+0000 7fb6e3989640 1 -- 172.21.15.45:0/273439775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6dc15f380 msgr2=0x7fb6dc14e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:46.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.208+0000 7fb6e3989640 1 --2- 172.21.15.45:0/273439775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6dc15f380 0x7fb6dc14e000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4007ef0 tx=0x7fb6d400d4a0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.208+0000 7fb6e3989640 1 -- 172.21.15.45:0/273439775 shutdown_connections 2024-05-08T05:41:46.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.208+0000 7fb6e3989640 1 --2- 172.21.15.45:0/273439775 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb6c00762d0 0x7fb6c0078790 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.208+0000 7fb6e3989640 1 --2- 172.21.15.45:0/273439775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6dc15f380 0x7fb6dc14e000 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.208+0000 7fb6e3989640 1 --2- 172.21.15.45:0/273439775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6dc0d3b00 0x7fb6dc15f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.208+0000 7fb6e3989640 1 --2- 172.21.15.45:0/273439775 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6dc0d0ec0 0x7fb6dc0ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.208+0000 7fb6e3989640 1 -- 172.21.15.45:0/273439775 >> 172.21.15.45:0/273439775 conn(0x7fb6dc0d4fe0 msgr2=0x7fb6dc06e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:46.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.208+0000 7fb6e3989640 1 -- 172.21.15.45:0/273439775 shutdown_connections 2024-05-08T05:41:46.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.208+0000 7fb6e3989640 1 -- 172.21.15.45:0/273439775 wait complete. 2024-05-08T05:41:46.220 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-05-08T05:41:44.836570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":308,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236012,"kb_used_data":3176,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738708,"statfs":{"total":773094113280,"available":772852436992,"internally_reserved":0,"allocated":3252224,"data_stored":1983114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.701500"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575491+0000","last_change":"2024-05-08T05:41:31.127376+0000","last_active":"2024-05-08T05:41:37.575491+0000","last_peered":"2024-05-08T05:41:37.575491+0000","last_clean":"2024-05-08T05:41:37.575491+0000","last_became_active":"2024-05-08T05:41:31.127083+0000","last_became_peered":"2024-05-08T05:41:31.127083+0000","last_unstale":"2024-05-08T05:41:37.575491+0000","last_undegraded":"2024-05-08T05:41:37.575491+0000","last_fullsized":"2024-05-08T05:41:37.575491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:53.557101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576109+0000","last_change":"2024-05-08T05:41:32.133658+0000","last_active":"2024-05-08T05:41:37.576109+0000","last_peered":"2024-05-08T05:41:37.576109+0000","last_clean":"2024-05-08T05:41:37.576109+0000","last_became_active":"2024-05-08T05:41:32.133241+0000","last_became_peered":"2024-05-08T05:41:32.133241+0000","last_unstale":"2024-05-08T05:41:37.576109+0000","last_undegraded":"2024-05-08T05:41:37.576109+0000","last_fullsized":"2024-05-08T05:41:37.576109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:07:26.071605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576413+0000","last_change":"2024-05-08T05:41:36.385551+0000","last_active":"2024-05-08T05:41:37.576413+0000","last_peered":"2024-05-08T05:41:37.576413+0000","last_clean":"2024-05-08T05:41:37.576413+0000","last_became_active":"2024-05-08T05:41:36.385135+0000","last_became_peered":"2024-05-08T05:41:36.385135+0000","last_unstale":"2024-05-08T05:41:37.576413+0000","last_undegraded":"2024-05-08T05:41:37.576413+0000","last_fullsized":"2024-05-08T05:41:37.576413+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:39:51.537856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576344+0000","last_change":"2024-05-08T05:41:31.125574+0000","last_active":"2024-05-08T05:41:37.576344+0000","last_peered":"2024-05-08T05:41:37.576344+0000","last_clean":"2024-05-08T05:41:37.576344+0000","last_became_active":"2024-05-08T05:41:31.125302+0000","last_became_peered":"2024-05-08T05:41:31.125302+0000","last_unstale":"2024-05-08T05:41:37.576344+0000","last_undegraded":"2024-05-08T05:41:37.576344+0000","last_fullsized":"2024-05-08T05:41:37.576344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:39.672951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575165+0000","last_change":"2024-05-08T05:41:32.140528+0000","last_active":"2024-05-08T05:41:37.575165+0000","last_peered":"2024-05-08T05:41:37.575165+0000","last_clean":"2024-05-08T05:41:37.575165+0000","last_became_active":"2024-05-08T05:41:32.140227+0000","last_became_peered":"2024-05-08T05:41:32.140227+0000","last_unstale":"2024-05-08T05:41:37.575165+0000","last_undegraded":"2024-05-08T05:41:37.575165+0000","last_fullsized":"2024-05-08T05:41:37.575165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:30:07.184101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153167+0000","last_change":"2024-05-08T05:41:36.149551+0000","last_active":"2024-05-08T05:41:37.153167+0000","last_peered":"2024-05-08T05:41:37.153167+0000","last_clean":"2024-05-08T05:41:37.153167+0000","last_became_active":"2024-05-08T05:41:36.149115+0000","last_became_peered":"2024-05-08T05:41:36.149115+0000","last_unstale":"2024-05-08T05:41:37.153167+0000","last_undegraded":"2024-05-08T05:41:37.153167+0000","last_fullsized":"2024-05-08T05:41:37.153167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:57.593969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577133+0000","last_change":"2024-05-08T05:41:31.128105+0000","last_active":"2024-05-08T05:41:37.577133+0000","last_peered":"2024-05-08T05:41:37.577133+0000","last_clean":"2024-05-08T05:41:37.577133+0000","last_became_active":"2024-05-08T05:41:31.127782+0000","last_became_peered":"2024-05-08T05:41:31.127782+0000","last_unstale":"2024-05-08T05:41:37.577133+0000","last_undegraded":"2024-05-08T05:41:37.577133+0000","last_fullsized":"2024-05-08T05:41:37.577133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:46.418806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576628+0000","last_change":"2024-05-08T05:41:32.140667+0000","last_active":"2024-05-08T05:41:37.576628+0000","last_peered":"2024-05-08T05:41:37.576628+0000","last_clean":"2024-05-08T05:41:37.576628+0000","last_became_active":"2024-05-08T05:41:32.140365+0000","last_became_peered":"2024-05-08T05:41:32.140365+0000","last_unstale":"2024-05-08T05:41:37.576628+0000","last_undegraded":"2024-05-08T05:41:37.576628+0000","last_fullsized":"2024-05-08T05:41:37.576628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:02.517245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153785+0000","last_change":"2024-05-08T05:41:36.157609+0000","last_active":"2024-05-08T05:41:37.153785+0000","last_peered":"2024-05-08T05:41:37.153785+0000","last_clean":"2024-05-08T05:41:37.153785+0000","last_became_active":"2024-05-08T05:41:36.157214+0000","last_became_peered":"2024-05-08T05:41:36.157214+0000","last_unstale":"2024-05-08T05:41:37.153785+0000","last_undegraded":"2024-05-08T05:41:37.153785+0000","last_fullsized":"2024-05-08T05:41:37.153785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:46.769826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576228+0000","last_change":"2024-05-08T05:41:31.131582+0000","last_active":"2024-05-08T05:41:37.576228+0000","last_peered":"2024-05-08T05:41:37.576228+0000","last_clean":"2024-05-08T05:41:37.576228+0000","last_became_active":"2024-05-08T05:41:31.131302+0000","last_became_peered":"2024-05-08T05:41:31.131302+0000","last_unstale":"2024-05-08T05:41:37.576228+0000","last_undegraded":"2024-05-08T05:41:37.576228+0000","last_fullsized":"2024-05-08T05:41:37.576228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:07.648895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577648+0000","last_change":"2024-05-08T05:41:32.136792+0000","last_active":"2024-05-08T05:41:37.577648+0000","last_peered":"2024-05-08T05:41:37.577648+0000","last_clean":"2024-05-08T05:41:37.577648+0000","last_became_active":"2024-05-08T05:41:32.136285+0000","last_became_peered":"2024-05-08T05:41:32.136285+0000","last_unstale":"2024-05-08T05:41:37.577648+0000","last_undegraded":"2024-05-08T05:41:37.577648+0000","last_fullsized":"2024-05-08T05:41:37.577648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:53.826185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153415+0000","last_change":"2024-05-08T05:41:36.151839+0000","last_active":"2024-05-08T05:41:37.153415+0000","last_peered":"2024-05-08T05:41:37.153415+0000","last_clean":"2024-05-08T05:41:37.153415+0000","last_became_active":"2024-05-08T05:41:36.151387+0000","last_became_peered":"2024-05-08T05:41:36.151387+0000","last_unstale":"2024-05-08T05:41:37.153415+0000","last_undegraded":"2024-05-08T05:41:37.153415+0000","last_fullsized":"2024-05-08T05:41:37.153415+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:01:36.105428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576169+0000","last_change":"2024-05-08T05:41:32.136438+0000","last_active":"2024-05-08T05:41:37.576169+0000","last_peered":"2024-05-08T05:41:37.576169+0000","last_clean":"2024-05-08T05:41:37.576169+0000","last_became_active":"2024-05-08T05:41:32.136145+0000","last_became_peered":"2024-05-08T05:41:32.136145+0000","last_unstale":"2024-05-08T05:41:37.576169+0000","last_undegraded":"2024-05-08T05:41:37.576169+0000","last_fullsized":"2024-05-08T05:41:37.576169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:47.493805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576520+0000","last_change":"2024-05-08T05:41:31.124904+0000","last_active":"2024-05-08T05:41:37.576520+0000","last_peered":"2024-05-08T05:41:37.576520+0000","last_clean":"2024-05-08T05:41:37.576520+0000","last_became_active":"2024-05-08T05:41:31.124621+0000","last_became_peered":"2024-05-08T05:41:31.124621+0000","last_unstale":"2024-05-08T05:41:37.576520+0000","last_undegraded":"2024-05-08T05:41:37.576520+0000","last_fullsized":"2024-05-08T05:41:37.576520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:41.591349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154482+0000","last_change":"2024-05-08T05:41:36.154069+0000","last_active":"2024-05-08T05:41:37.154482+0000","last_peered":"2024-05-08T05:41:37.154482+0000","last_clean":"2024-05-08T05:41:37.154482+0000","last_became_active":"2024-05-08T05:41:36.153717+0000","last_became_peered":"2024-05-08T05:41:36.153717+0000","last_unstale":"2024-05-08T05:41:37.154482+0000","last_undegraded":"2024-05-08T05:41:37.154482+0000","last_fullsized":"2024-05-08T05:41:37.154482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:57.856276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577086+0000","last_change":"2024-05-08T05:41:32.137987+0000","last_active":"2024-05-08T05:41:37.577086+0000","last_peered":"2024-05-08T05:41:37.577086+0000","last_clean":"2024-05-08T05:41:37.577086+0000","last_became_active":"2024-05-08T05:41:32.137711+0000","last_became_peered":"2024-05-08T05:41:32.137711+0000","last_unstale":"2024-05-08T05:41:37.577086+0000","last_undegraded":"2024-05-08T05:41:37.577086+0000","last_fullsized":"2024-05-08T05:41:37.577086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:01:32.075337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576626+0000","last_change":"2024-05-08T05:41:31.121790+0000","last_active":"2024-05-08T05:41:37.576626+0000","last_peered":"2024-05-08T05:41:37.576626+0000","last_clean":"2024-05-08T05:41:37.576626+0000","last_became_active":"2024-05-08T05:41:31.121476+0000","last_became_peered":"2024-05-08T05:41:31.121476+0000","last_unstale":"2024-05-08T05:41:37.576626+0000","last_undegraded":"2024-05-08T05:41:37.576626+0000","last_fullsized":"2024-05-08T05:41:37.576626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:11:56.288788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154013+0000","last_change":"2024-05-08T05:41:36.157615+0000","last_active":"2024-05-08T05:41:37.154013+0000","last_peered":"2024-05-08T05:41:37.154013+0000","last_clean":"2024-05-08T05:41:37.154013+0000","last_became_active":"2024-05-08T05:41:36.157156+0000","last_became_peered":"2024-05-08T05:41:36.157156+0000","last_unstale":"2024-05-08T05:41:37.154013+0000","last_undegraded":"2024-05-08T05:41:37.154013+0000","last_fullsized":"2024-05-08T05:41:37.154013+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:46.115781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576120+0000","last_change":"2024-05-08T05:41:32.119867+0000","last_active":"2024-05-08T05:41:37.576120+0000","last_peered":"2024-05-08T05:41:37.576120+0000","last_clean":"2024-05-08T05:41:37.576120+0000","last_became_active":"2024-05-08T05:41:32.119186+0000","last_became_peered":"2024-05-08T05:41:32.119186+0000","last_unstale":"2024-05-08T05:41:37.576120+0000","last_undegraded":"2024-05-08T05:41:37.576120+0000","last_fullsized":"2024-05-08T05:41:37.576120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:47.493805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575164+0000","last_change":"2024-05-08T05:41:31.128486+0000","last_active":"2024-05-08T05:41:37.575164+0000","last_peered":"2024-05-08T05:41:37.575164+0000","last_clean":"2024-05-08T05:41:37.575164+0000","last_became_active":"2024-05-08T05:41:31.128207+0000","last_became_peered":"2024-05-08T05:41:31.128207+0000","last_unstale":"2024-05-08T05:41:37.575164+0000","last_undegraded":"2024-05-08T05:41:37.575164+0000","last_fullsized":"2024-05-08T05:41:37.575164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:46.418806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576817+0000","last_change":"2024-05-08T05:41:36.381823+0000","last_active":"2024-05-08T05:41:37.576817+0000","last_peered":"2024-05-08T05:41:37.576817+0000","last_clean":"2024-05-08T05:41:37.576817+0000","last_became_active":"2024-05-08T05:41:36.381421+0000","last_became_peered":"2024-05-08T05:41:36.381421+0000","last_unstale":"2024-05-08T05:41:37.576817+0000","last_undegraded":"2024-05-08T05:41:37.576817+0000","last_fullsized":"2024-05-08T05:41:37.576817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:21.321405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153397+0000","last_change":"2024-05-08T05:41:32.117529+0000","last_active":"2024-05-08T05:41:37.153397+0000","last_peered":"2024-05-08T05:41:37.153397+0000","last_clean":"2024-05-08T05:41:37.153397+0000","last_became_active":"2024-05-08T05:41:32.117205+0000","last_became_peered":"2024-05-08T05:41:32.117205+0000","last_unstale":"2024-05-08T05:41:37.153397+0000","last_undegraded":"2024-05-08T05:41:37.153397+0000","last_fullsized":"2024-05-08T05:41:37.153397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:08.654224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575429+0000","last_change":"2024-05-08T05:41:31.126184+0000","last_active":"2024-05-08T05:41:37.575429+0000","last_peered":"2024-05-08T05:41:37.575429+0000","last_clean":"2024-05-08T05:41:37.575429+0000","last_became_active":"2024-05-08T05:41:31.125860+0000","last_became_peered":"2024-05-08T05:41:31.125860+0000","last_unstale":"2024-05-08T05:41:37.575429+0000","last_undegraded":"2024-05-08T05:41:37.575429+0000","last_fullsized":"2024-05-08T05:41:37.575429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:13.465598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154197+0000","last_change":"2024-05-08T05:41:36.151020+0000","last_active":"2024-05-08T05:41:37.154197+0000","last_peered":"2024-05-08T05:41:37.154197+0000","last_clean":"2024-05-08T05:41:37.154197+0000","last_became_active":"2024-05-08T05:41:36.150320+0000","last_became_peered":"2024-05-08T05:41:36.150320+0000","last_unstale":"2024-05-08T05:41:37.154197+0000","last_undegraded":"2024-05-08T05:41:37.154197+0000","last_fullsized":"2024-05-08T05:41:37.154197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:06.547337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576056+0000","last_change":"2024-05-08T05:41:32.121088+0000","last_active":"2024-05-08T05:41:37.576056+0000","last_peered":"2024-05-08T05:41:37.576056+0000","last_clean":"2024-05-08T05:41:37.576056+0000","last_became_active":"2024-05-08T05:41:32.119585+0000","last_became_peered":"2024-05-08T05:41:32.119585+0000","last_unstale":"2024-05-08T05:41:37.576056+0000","last_undegraded":"2024-05-08T05:41:37.576056+0000","last_fullsized":"2024-05-08T05:41:37.576056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:18:52.796967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577379+0000","last_change":"2024-05-08T05:41:31.129202+0000","last_active":"2024-05-08T05:41:37.577379+0000","last_peered":"2024-05-08T05:41:37.577379+0000","last_clean":"2024-05-08T05:41:37.577379+0000","last_became_active":"2024-05-08T05:41:31.128911+0000","last_became_peered":"2024-05-08T05:41:31.128911+0000","last_unstale":"2024-05-08T05:41:37.577379+0000","last_undegraded":"2024-05-08T05:41:37.577379+0000","last_fullsized":"2024-05-08T05:41:37.577379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:13.465598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153283+0000","last_change":"2024-05-08T05:41:36.384181+0000","last_active":"2024-05-08T05:41:37.153283+0000","last_peered":"2024-05-08T05:41:37.153283+0000","last_clean":"2024-05-08T05:41:37.153283+0000","last_became_active":"2024-05-08T05:41:36.383800+0000","last_became_peered":"2024-05-08T05:41:36.383800+0000","last_unstale":"2024-05-08T05:41:37.153283+0000","last_undegraded":"2024-05-08T05:41:37.153283+0000","last_fullsized":"2024-05-08T05:41:37.153283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:33.503988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576885+0000","last_change":"2024-05-08T05:41:32.121597+0000","last_active":"2024-05-08T05:41:37.576885+0000","last_peered":"2024-05-08T05:41:37.576885+0000","last_clean":"2024-05-08T05:41:37.576885+0000","last_became_active":"2024-05-08T05:41:32.121321+0000","last_became_peered":"2024-05-08T05:41:32.121321+0000","last_unstale":"2024-05-08T05:41:37.576885+0000","last_undegraded":"2024-05-08T05:41:37.576885+0000","last_fullsized":"2024-05-08T05:41:37.576885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:42.739734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575444+0000","last_change":"2024-05-08T05:41:31.126359+0000","last_active":"2024-05-08T05:41:37.575444+0000","last_peered":"2024-05-08T05:41:37.575444+0000","last_clean":"2024-05-08T05:41:37.575444+0000","last_became_active":"2024-05-08T05:41:31.126069+0000","last_became_peered":"2024-05-08T05:41:31.126069+0000","last_unstale":"2024-05-08T05:41:37.575444+0000","last_undegraded":"2024-05-08T05:41:37.575444+0000","last_fullsized":"2024-05-08T05:41:37.575444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:24.722618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575700+0000","last_change":"2024-05-08T05:41:36.381909+0000","last_active":"2024-05-08T05:41:37.575700+0000","last_peered":"2024-05-08T05:41:37.575700+0000","last_clean":"2024-05-08T05:41:37.575700+0000","last_became_active":"2024-05-08T05:41:36.381251+0000","last_became_peered":"2024-05-08T05:41:36.381251+0000","last_unstale":"2024-05-08T05:41:37.575700+0000","last_undegraded":"2024-05-08T05:41:37.575700+0000","last_fullsized":"2024-05-08T05:41:37.575700+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:06:13.980953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576829+0000","last_change":"2024-05-08T05:41:32.133545+0000","last_active":"2024-05-08T05:41:37.576829+0000","last_peered":"2024-05-08T05:41:37.576829+0000","last_clean":"2024-05-08T05:41:37.576829+0000","last_became_active":"2024-05-08T05:41:32.132677+0000","last_became_peered":"2024-05-08T05:41:32.132677+0000","last_unstale":"2024-05-08T05:41:37.576829+0000","last_undegraded":"2024-05-08T05:41:37.576829+0000","last_fullsized":"2024-05-08T05:41:37.576829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:42.041915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577263+0000","last_change":"2024-05-08T05:41:31.127131+0000","last_active":"2024-05-08T05:41:37.577263+0000","last_peered":"2024-05-08T05:41:37.577263+0000","last_clean":"2024-05-08T05:41:37.577263+0000","last_became_active":"2024-05-08T05:41:31.126789+0000","last_became_peered":"2024-05-08T05:41:31.126789+0000","last_unstale":"2024-05-08T05:41:37.577263+0000","last_undegraded":"2024-05-08T05:41:37.577263+0000","last_fullsized":"2024-05-08T05:41:37.577263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:19:12.221462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575923+0000","last_change":"2024-05-08T05:41:36.381721+0000","last_active":"2024-05-08T05:41:37.575923+0000","last_peered":"2024-05-08T05:41:37.575923+0000","last_clean":"2024-05-08T05:41:37.575923+0000","last_became_active":"2024-05-08T05:41:36.381329+0000","last_became_peered":"2024-05-08T05:41:36.381329+0000","last_unstale":"2024-05-08T05:41:37.575923+0000","last_undegraded":"2024-05-08T05:41:37.575923+0000","last_fullsized":"2024-05-08T05:41:37.575923+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:17.202706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577782+0000","last_change":"2024-05-08T05:41:32.136558+0000","last_active":"2024-05-08T05:41:37.577782+0000","last_peered":"2024-05-08T05:41:37.577782+0000","last_clean":"2024-05-08T05:41:37.577782+0000","last_became_active":"2024-05-08T05:41:32.133354+0000","last_became_peered":"2024-05-08T05:41:32.133354+0000","last_unstale":"2024-05-08T05:41:37.577782+0000","last_undegraded":"2024-05-08T05:41:37.577782+0000","last_fullsized":"2024-05-08T05:41:37.577782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:13.172615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576762+0000","last_change":"2024-05-08T05:41:31.125582+0000","last_active":"2024-05-08T05:41:37.576762+0000","last_peered":"2024-05-08T05:41:37.576762+0000","last_clean":"2024-05-08T05:41:37.576762+0000","last_became_active":"2024-05-08T05:41:31.125252+0000","last_became_peered":"2024-05-08T05:41:31.125252+0000","last_unstale":"2024-05-08T05:41:37.576762+0000","last_undegraded":"2024-05-08T05:41:37.576762+0000","last_fullsized":"2024-05-08T05:41:37.576762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:38.174964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576435+0000","last_change":"2024-05-08T05:41:36.154344+0000","last_active":"2024-05-08T05:41:37.576435+0000","last_peered":"2024-05-08T05:41:37.576435+0000","last_clean":"2024-05-08T05:41:37.576435+0000","last_became_active":"2024-05-08T05:41:36.153930+0000","last_became_peered":"2024-05-08T05:41:36.153930+0000","last_unstale":"2024-05-08T05:41:37.576435+0000","last_undegraded":"2024-05-08T05:41:37.576435+0000","last_fullsized":"2024-05-08T05:41:37.576435+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:20.531197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154018+0000","last_change":"2024-05-08T05:41:32.136504+0000","last_active":"2024-05-08T05:41:37.154018+0000","last_peered":"2024-05-08T05:41:37.154018+0000","last_clean":"2024-05-08T05:41:37.154018+0000","last_became_active":"2024-05-08T05:41:32.135721+0000","last_became_peered":"2024-05-08T05:41:32.135721+0000","last_unstale":"2024-05-08T05:41:37.154018+0000","last_undegraded":"2024-05-08T05:41:37.154018+0000","last_fullsized":"2024-05-08T05:41:37.154018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:42.596678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576189+0000","last_change":"2024-05-08T05:41:31.131574+0000","last_active":"2024-05-08T05:41:37.576189+0000","last_peered":"2024-05-08T05:41:37.576189+0000","last_clean":"2024-05-08T05:41:37.576189+0000","last_became_active":"2024-05-08T05:41:31.131158+0000","last_became_peered":"2024-05-08T05:41:31.131158+0000","last_unstale":"2024-05-08T05:41:37.576189+0000","last_undegraded":"2024-05-08T05:41:37.576189+0000","last_fullsized":"2024-05-08T05:41:37.576189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:13.465598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576707+0000","last_change":"2024-05-08T05:41:36.159573+0000","last_active":"2024-05-08T05:41:37.576707+0000","last_peered":"2024-05-08T05:41:37.576707+0000","last_clean":"2024-05-08T05:41:37.576707+0000","last_became_active":"2024-05-08T05:41:36.159150+0000","last_became_peered":"2024-05-08T05:41:36.159150+0000","last_unstale":"2024-05-08T05:41:37.576707+0000","last_undegraded":"2024-05-08T05:41:37.576707+0000","last_fullsized":"2024-05-08T05:41:37.576707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:17.202706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576057+0000","last_change":"2024-05-08T05:41:32.121923+0000","last_active":"2024-05-08T05:41:37.576057+0000","last_peered":"2024-05-08T05:41:37.576057+0000","last_clean":"2024-05-08T05:41:37.576057+0000","last_became_active":"2024-05-08T05:41:32.121213+0000","last_became_peered":"2024-05-08T05:41:32.121213+0000","last_unstale":"2024-05-08T05:41:37.576057+0000","last_undegraded":"2024-05-08T05:41:37.576057+0000","last_fullsized":"2024-05-08T05:41:37.576057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:03:20.127059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.147846+0000","last_change":"2024-05-08T05:41:31.122162+0000","last_active":"2024-05-08T05:41:37.147846+0000","last_peered":"2024-05-08T05:41:37.147846+0000","last_clean":"2024-05-08T05:41:37.147846+0000","last_became_active":"2024-05-08T05:41:31.121865+0000","last_became_peered":"2024-05-08T05:41:31.121865+0000","last_unstale":"2024-05-08T05:41:37.147846+0000","last_undegraded":"2024-05-08T05:41:37.147846+0000","last_fullsized":"2024-05-08T05:41:37.147846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:24.722618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576949+0000","last_change":"2024-05-08T05:41:36.155633+0000","last_active":"2024-05-08T05:41:37.576949+0000","last_peered":"2024-05-08T05:41:37.576949+0000","last_clean":"2024-05-08T05:41:37.576949+0000","last_became_active":"2024-05-08T05:41:36.155125+0000","last_became_peered":"2024-05-08T05:41:36.155125+0000","last_unstale":"2024-05-08T05:41:37.576949+0000","last_undegraded":"2024-05-08T05:41:37.576949+0000","last_fullsized":"2024-05-08T05:41:37.576949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:50:36.282661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.574963+0000","last_change":"2024-05-08T05:41:32.119953+0000","last_active":"2024-05-08T05:41:37.574963+0000","last_peered":"2024-05-08T05:41:37.574963+0000","last_clean":"2024-05-08T05:41:37.574963+0000","last_became_active":"2024-05-08T05:41:32.119532+0000","last_became_peered":"2024-05-08T05:41:32.119532+0000","last_unstale":"2024-05-08T05:41:37.574963+0000","last_undegraded":"2024-05-08T05:41:37.574963+0000","last_fullsized":"2024-05-08T05:41:37.574963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:16:33.854897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575557+0000","last_change":"2024-05-08T05:41:31.125004+0000","last_active":"2024-05-08T05:41:37.575557+0000","last_peered":"2024-05-08T05:41:37.575557+0000","last_clean":"2024-05-08T05:41:37.575557+0000","last_became_active":"2024-05-08T05:41:31.124678+0000","last_became_peered":"2024-05-08T05:41:31.124678+0000","last_unstale":"2024-05-08T05:41:37.575557+0000","last_undegraded":"2024-05-08T05:41:37.575557+0000","last_fullsized":"2024-05-08T05:41:37.575557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:38.174964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575882+0000","last_change":"2024-05-08T05:41:36.385360+0000","last_active":"2024-05-08T05:41:37.575882+0000","last_peered":"2024-05-08T05:41:37.575882+0000","last_clean":"2024-05-08T05:41:37.575882+0000","last_became_active":"2024-05-08T05:41:36.384992+0000","last_became_peered":"2024-05-08T05:41:36.384992+0000","last_unstale":"2024-05-08T05:41:37.575882+0000","last_undegraded":"2024-05-08T05:41:37.575882+0000","last_fullsized":"2024-05-08T05:41:37.575882+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:20.531197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575341+0000","last_change":"2024-05-08T05:41:32.138842+0000","last_active":"2024-05-08T05:41:37.575341+0000","last_peered":"2024-05-08T05:41:37.575341+0000","last_clean":"2024-05-08T05:41:37.575341+0000","last_became_active":"2024-05-08T05:41:32.138583+0000","last_became_peered":"2024-05-08T05:41:32.138583+0000","last_unstale":"2024-05-08T05:41:37.575341+0000","last_undegraded":"2024-05-08T05:41:37.575341+0000","last_fullsized":"2024-05-08T05:41:37.575341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:53.563878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576353+0000","last_change":"2024-05-08T05:41:31.132616+0000","last_active":"2024-05-08T05:41:37.576353+0000","last_peered":"2024-05-08T05:41:37.576353+0000","last_clean":"2024-05-08T05:41:37.576353+0000","last_became_active":"2024-05-08T05:41:31.131462+0000","last_became_peered":"2024-05-08T05:41:31.131462+0000","last_unstale":"2024-05-08T05:41:37.576353+0000","last_undegraded":"2024-05-08T05:41:37.576353+0000","last_fullsized":"2024-05-08T05:41:37.576353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:24.722618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":118,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575672+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:41:37.575672+0000","last_peered":"2024-05-08T05:41:37.575672+0000","last_clean":"2024-05-08T05:41:37.575672+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:41:37.575672+0000","last_undegraded":"2024-05-08T05:41:37.575672+0000","last_fullsized":"2024-05-08T05:41:37.575672+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576785+0000","last_change":"2024-05-08T05:41:36.381947+0000","last_active":"2024-05-08T05:41:37.576785+0000","last_peered":"2024-05-08T05:41:37.576785+0000","last_clean":"2024-05-08T05:41:37.576785+0000","last_became_active":"2024-05-08T05:41:36.381491+0000","last_became_peered":"2024-05-08T05:41:36.381491+0000","last_unstale":"2024-05-08T05:41:37.576785+0000","last_undegraded":"2024-05-08T05:41:37.576785+0000","last_fullsized":"2024-05-08T05:41:37.576785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:57.856276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576983+0000","last_change":"2024-05-08T05:41:32.134397+0000","last_active":"2024-05-08T05:41:37.576983+0000","last_peered":"2024-05-08T05:41:37.576983+0000","last_clean":"2024-05-08T05:41:37.576983+0000","last_became_active":"2024-05-08T05:41:32.134098+0000","last_became_peered":"2024-05-08T05:41:32.134098+0000","last_unstale":"2024-05-08T05:41:37.576983+0000","last_undegraded":"2024-05-08T05:41:37.576983+0000","last_fullsized":"2024-05-08T05:41:37.576983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:16.501105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.148234+0000","last_change":"2024-05-08T05:41:31.133531+0000","last_active":"2024-05-08T05:41:37.148234+0000","last_peered":"2024-05-08T05:41:37.148234+0000","last_clean":"2024-05-08T05:41:37.148234+0000","last_became_active":"2024-05-08T05:41:31.133208+0000","last_became_peered":"2024-05-08T05:41:31.133208+0000","last_unstale":"2024-05-08T05:41:37.148234+0000","last_undegraded":"2024-05-08T05:41:37.148234+0000","last_fullsized":"2024-05-08T05:41:37.148234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:22.751500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576104+0000","last_change":"2024-05-08T05:41:36.381903+0000","last_active":"2024-05-08T05:41:37.576104+0000","last_peered":"2024-05-08T05:41:37.576104+0000","last_clean":"2024-05-08T05:41:37.576104+0000","last_became_active":"2024-05-08T05:41:36.381397+0000","last_became_peered":"2024-05-08T05:41:36.381397+0000","last_unstale":"2024-05-08T05:41:37.576104+0000","last_undegraded":"2024-05-08T05:41:37.576104+0000","last_fullsized":"2024-05-08T05:41:37.576104+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:06.547337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576725+0000","last_change":"2024-05-08T05:41:32.128554+0000","last_active":"2024-05-08T05:41:37.576725+0000","last_peered":"2024-05-08T05:41:37.576725+0000","last_clean":"2024-05-08T05:41:37.576725+0000","last_became_active":"2024-05-08T05:41:32.128280+0000","last_became_peered":"2024-05-08T05:41:32.128280+0000","last_unstale":"2024-05-08T05:41:37.576725+0000","last_undegraded":"2024-05-08T05:41:37.576725+0000","last_fullsized":"2024-05-08T05:41:37.576725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:42.085689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576546+0000","last_change":"2024-05-08T05:41:31.122071+0000","last_active":"2024-05-08T05:41:37.576546+0000","last_peered":"2024-05-08T05:41:37.576546+0000","last_clean":"2024-05-08T05:41:37.576546+0000","last_became_active":"2024-05-08T05:41:31.121137+0000","last_became_peered":"2024-05-08T05:41:31.121137+0000","last_unstale":"2024-05-08T05:41:37.576546+0000","last_undegraded":"2024-05-08T05:41:37.576546+0000","last_fullsized":"2024-05-08T05:41:37.576546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:13.465598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576176+0000","last_change":"2024-05-08T05:41:36.160917+0000","last_active":"2024-05-08T05:41:37.576176+0000","last_peered":"2024-05-08T05:41:37.576176+0000","last_clean":"2024-05-08T05:41:37.576176+0000","last_became_active":"2024-05-08T05:41:36.160457+0000","last_became_peered":"2024-05-08T05:41:36.160457+0000","last_unstale":"2024-05-08T05:41:37.576176+0000","last_undegraded":"2024-05-08T05:41:37.576176+0000","last_fullsized":"2024-05-08T05:41:37.576176+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:21.321405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577384+0000","last_change":"2024-05-08T05:41:32.131012+0000","last_active":"2024-05-08T05:41:37.577384+0000","last_peered":"2024-05-08T05:41:37.577384+0000","last_clean":"2024-05-08T05:41:37.577384+0000","last_became_active":"2024-05-08T05:41:32.130682+0000","last_became_peered":"2024-05-08T05:41:32.130682+0000","last_unstale":"2024-05-08T05:41:37.577384+0000","last_undegraded":"2024-05-08T05:41:37.577384+0000","last_fullsized":"2024-05-08T05:41:37.577384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:06:09.950861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.148282+0000","last_change":"2024-05-08T05:41:31.122329+0000","last_active":"2024-05-08T05:41:37.148282+0000","last_peered":"2024-05-08T05:41:37.148282+0000","last_clean":"2024-05-08T05:41:37.148282+0000","last_became_active":"2024-05-08T05:41:31.122021+0000","last_became_peered":"2024-05-08T05:41:31.122021+0000","last_unstale":"2024-05-08T05:41:37.148282+0000","last_undegraded":"2024-05-08T05:41:37.148282+0000","last_fullsized":"2024-05-08T05:41:37.148282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:11:56.288788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575492+0000","last_change":"2024-05-08T05:41:36.382122+0000","last_active":"2024-05-08T05:41:37.575492+0000","last_peered":"2024-05-08T05:41:37.575492+0000","last_clean":"2024-05-08T05:41:37.575492+0000","last_became_active":"2024-05-08T05:41:36.381385+0000","last_became_peered":"2024-05-08T05:41:36.381385+0000","last_unstale":"2024-05-08T05:41:37.575492+0000","last_undegraded":"2024-05-08T05:41:37.575492+0000","last_fullsized":"2024-05-08T05:41:37.575492+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:02.479497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576083+0000","last_change":"2024-05-08T05:41:32.131255+0000","last_active":"2024-05-08T05:41:37.576083+0000","last_peered":"2024-05-08T05:41:37.576083+0000","last_clean":"2024-05-08T05:41:37.576083+0000","last_became_active":"2024-05-08T05:41:32.130252+0000","last_became_peered":"2024-05-08T05:41:32.130252+0000","last_unstale":"2024-05-08T05:41:37.576083+0000","last_undegraded":"2024-05-08T05:41:37.576083+0000","last_fullsized":"2024-05-08T05:41:37.576083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:17.291314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154349+0000","last_change":"2024-05-08T05:41:31.125770+0000","last_active":"2024-05-08T05:41:37.154349+0000","last_peered":"2024-05-08T05:41:37.154349+0000","last_clean":"2024-05-08T05:41:37.154349+0000","last_became_active":"2024-05-08T05:41:31.125442+0000","last_became_peered":"2024-05-08T05:41:31.125442+0000","last_unstale":"2024-05-08T05:41:37.154349+0000","last_undegraded":"2024-05-08T05:41:37.154349+0000","last_fullsized":"2024-05-08T05:41:37.154349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:53.557101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153167+0000","last_change":"2024-05-08T05:41:36.151074+0000","last_active":"2024-05-08T05:41:37.153167+0000","last_peered":"2024-05-08T05:41:37.153167+0000","last_clean":"2024-05-08T05:41:37.153167+0000","last_became_active":"2024-05-08T05:41:36.150613+0000","last_became_peered":"2024-05-08T05:41:36.150613+0000","last_unstale":"2024-05-08T05:41:37.153167+0000","last_undegraded":"2024-05-08T05:41:37.153167+0000","last_fullsized":"2024-05-08T05:41:37.153167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:46.072007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576628+0000","last_change":"2024-05-08T05:41:32.132197+0000","last_active":"2024-05-08T05:41:37.576628+0000","last_peered":"2024-05-08T05:41:37.576628+0000","last_clean":"2024-05-08T05:41:37.576628+0000","last_became_active":"2024-05-08T05:41:32.131086+0000","last_became_peered":"2024-05-08T05:41:32.131086+0000","last_unstale":"2024-05-08T05:41:37.576628+0000","last_undegraded":"2024-05-08T05:41:37.576628+0000","last_fullsized":"2024-05-08T05:41:37.576628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:02.517245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.147676+0000","last_change":"2024-05-08T05:41:31.120099+0000","last_active":"2024-05-08T05:41:37.147676+0000","last_peered":"2024-05-08T05:41:37.147676+0000","last_clean":"2024-05-08T05:41:37.147676+0000","last_became_active":"2024-05-08T05:41:31.119750+0000","last_became_peered":"2024-05-08T05:41:31.119750+0000","last_unstale":"2024-05-08T05:41:37.147676+0000","last_undegraded":"2024-05-08T05:41:37.147676+0000","last_fullsized":"2024-05-08T05:41:37.147676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:49.171947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575735+0000","last_change":"2024-05-08T05:41:36.162020+0000","last_active":"2024-05-08T05:41:37.575735+0000","last_peered":"2024-05-08T05:41:37.575735+0000","last_clean":"2024-05-08T05:41:37.575735+0000","last_became_active":"2024-05-08T05:41:36.161613+0000","last_became_peered":"2024-05-08T05:41:36.161613+0000","last_unstale":"2024-05-08T05:41:37.575735+0000","last_undegraded":"2024-05-08T05:41:37.575735+0000","last_fullsized":"2024-05-08T05:41:37.575735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:57.856276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575424+0000","last_change":"2024-05-08T05:41:32.137565+0000","last_active":"2024-05-08T05:41:37.575424+0000","last_peered":"2024-05-08T05:41:37.575424+0000","last_clean":"2024-05-08T05:41:37.575424+0000","last_became_active":"2024-05-08T05:41:32.137078+0000","last_became_peered":"2024-05-08T05:41:32.137078+0000","last_unstale":"2024-05-08T05:41:37.575424+0000","last_undegraded":"2024-05-08T05:41:37.575424+0000","last_fullsized":"2024-05-08T05:41:37.575424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:44.038894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576417+0000","last_change":"2024-05-08T05:41:31.127117+0000","last_active":"2024-05-08T05:41:37.576417+0000","last_peered":"2024-05-08T05:41:37.576417+0000","last_clean":"2024-05-08T05:41:37.576417+0000","last_became_active":"2024-05-08T05:41:31.126826+0000","last_became_peered":"2024-05-08T05:41:31.126826+0000","last_unstale":"2024-05-08T05:41:37.576417+0000","last_undegraded":"2024-05-08T05:41:37.576417+0000","last_fullsized":"2024-05-08T05:41:37.576417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:22.751500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576723+0000","last_change":"2024-05-08T05:41:36.383650+0000","last_active":"2024-05-08T05:41:37.576723+0000","last_peered":"2024-05-08T05:41:37.576723+0000","last_clean":"2024-05-08T05:41:37.576723+0000","last_became_active":"2024-05-08T05:41:36.382759+0000","last_became_peered":"2024-05-08T05:41:36.382759+0000","last_unstale":"2024-05-08T05:41:37.576723+0000","last_undegraded":"2024-05-08T05:41:37.576723+0000","last_fullsized":"2024-05-08T05:41:37.576723+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:43:10.562469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576336+0000","last_change":"2024-05-08T05:41:32.130461+0000","last_active":"2024-05-08T05:41:37.576336+0000","last_peered":"2024-05-08T05:41:37.576336+0000","last_clean":"2024-05-08T05:41:37.576336+0000","last_became_active":"2024-05-08T05:41:32.130160+0000","last_became_peered":"2024-05-08T05:41:32.130160+0000","last_unstale":"2024-05-08T05:41:37.576336+0000","last_undegraded":"2024-05-08T05:41:37.576336+0000","last_fullsized":"2024-05-08T05:41:37.576336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:42.085689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576077+0000","last_change":"2024-05-08T05:41:31.132478+0000","last_active":"2024-05-08T05:41:37.576077+0000","last_peered":"2024-05-08T05:41:37.576077+0000","last_clean":"2024-05-08T05:41:37.576077+0000","last_became_active":"2024-05-08T05:41:31.131093+0000","last_became_peered":"2024-05-08T05:41:31.131093+0000","last_unstale":"2024-05-08T05:41:37.576077+0000","last_undegraded":"2024-05-08T05:41:37.576077+0000","last_fullsized":"2024-05-08T05:41:37.576077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:07.648895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577230+0000","last_change":"2024-05-08T05:41:36.381669+0000","last_active":"2024-05-08T05:41:37.577230+0000","last_peered":"2024-05-08T05:41:37.577230+0000","last_clean":"2024-05-08T05:41:37.577230+0000","last_became_active":"2024-05-08T05:41:36.381233+0000","last_became_peered":"2024-05-08T05:41:36.381233+0000","last_unstale":"2024-05-08T05:41:37.577230+0000","last_undegraded":"2024-05-08T05:41:37.577230+0000","last_fullsized":"2024-05-08T05:41:37.577230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:07.864670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575368+0000","last_change":"2024-05-08T05:41:32.131789+0000","last_active":"2024-05-08T05:41:37.575368+0000","last_peered":"2024-05-08T05:41:37.575368+0000","last_clean":"2024-05-08T05:41:37.575368+0000","last_became_active":"2024-05-08T05:41:32.131515+0000","last_became_peered":"2024-05-08T05:41:32.131515+0000","last_unstale":"2024-05-08T05:41:37.575368+0000","last_undegraded":"2024-05-08T05:41:37.575368+0000","last_fullsized":"2024-05-08T05:41:37.575368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:03:20.127059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576417+0000","last_change":"2024-05-08T05:41:31.126864+0000","last_active":"2024-05-08T05:41:37.576417+0000","last_peered":"2024-05-08T05:41:37.576417+0000","last_clean":"2024-05-08T05:41:37.576417+0000","last_became_active":"2024-05-08T05:41:31.126568+0000","last_became_peered":"2024-05-08T05:41:31.126568+0000","last_unstale":"2024-05-08T05:41:37.576417+0000","last_undegraded":"2024-05-08T05:41:37.576417+0000","last_fullsized":"2024-05-08T05:41:37.576417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:39.672951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577692+0000","last_change":"2024-05-08T05:41:36.382511+0000","last_active":"2024-05-08T05:41:37.577692+0000","last_peered":"2024-05-08T05:41:37.577692+0000","last_clean":"2024-05-08T05:41:37.577692+0000","last_became_active":"2024-05-08T05:41:36.382110+0000","last_became_peered":"2024-05-08T05:41:36.382110+0000","last_unstale":"2024-05-08T05:41:37.577692+0000","last_undegraded":"2024-05-08T05:41:37.577692+0000","last_fullsized":"2024-05-08T05:41:37.577692+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:39:51.537856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576868+0000","last_change":"2024-05-08T05:41:32.127674+0000","last_active":"2024-05-08T05:41:37.576868+0000","last_peered":"2024-05-08T05:41:37.576868+0000","last_clean":"2024-05-08T05:41:37.576868+0000","last_became_active":"2024-05-08T05:41:32.127374+0000","last_became_peered":"2024-05-08T05:41:32.127374+0000","last_unstale":"2024-05-08T05:41:37.576868+0000","last_undegraded":"2024-05-08T05:41:37.576868+0000","last_fullsized":"2024-05-08T05:41:37.576868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:53.826185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":28,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577094+0000","last_change":"2024-05-08T05:41:31.126507+0000","last_active":"2024-05-08T05:41:37.577094+0000","last_peered":"2024-05-08T05:41:37.577094+0000","last_clean":"2024-05-08T05:41:37.577094+0000","last_became_active":"2024-05-08T05:41:31.126218+0000","last_became_peered":"2024-05-08T05:41:31.126218+0000","last_unstale":"2024-05-08T05:41:37.577094+0000","last_undegraded":"2024-05-08T05:41:37.577094+0000","last_fullsized":"2024-05-08T05:41:37.577094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:11:56.288788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576865+0000","last_change":"2024-05-08T05:41:36.153303+0000","last_active":"2024-05-08T05:41:37.576865+0000","last_peered":"2024-05-08T05:41:37.576865+0000","last_clean":"2024-05-08T05:41:37.576865+0000","last_became_active":"2024-05-08T05:41:36.152849+0000","last_became_peered":"2024-05-08T05:41:36.152849+0000","last_unstale":"2024-05-08T05:41:37.576865+0000","last_undegraded":"2024-05-08T05:41:37.576865+0000","last_fullsized":"2024-05-08T05:41:37.576865+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:07:30.112175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154093+0000","last_change":"2024-05-08T05:41:32.121425+0000","last_active":"2024-05-08T05:41:37.154093+0000","last_peered":"2024-05-08T05:41:37.154093+0000","last_clean":"2024-05-08T05:41:37.154093+0000","last_became_active":"2024-05-08T05:41:32.121121+0000","last_became_peered":"2024-05-08T05:41:32.121121+0000","last_unstale":"2024-05-08T05:41:37.154093+0000","last_undegraded":"2024-05-08T05:41:37.154093+0000","last_fullsized":"2024-05-08T05:41:37.154093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:47.424136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576434+0000","last_change":"2024-05-08T05:41:31.132128+0000","last_active":"2024-05-08T05:41:37.576434+0000","last_peered":"2024-05-08T05:41:37.576434+0000","last_clean":"2024-05-08T05:41:37.576434+0000","last_became_active":"2024-05-08T05:41:31.131776+0000","last_became_peered":"2024-05-08T05:41:31.131776+0000","last_unstale":"2024-05-08T05:41:37.576434+0000","last_undegraded":"2024-05-08T05:41:37.576434+0000","last_fullsized":"2024-05-08T05:41:37.576434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:07.648895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153781+0000","last_change":"2024-05-08T05:41:36.152157+0000","last_active":"2024-05-08T05:41:37.153781+0000","last_peered":"2024-05-08T05:41:37.153781+0000","last_clean":"2024-05-08T05:41:37.153781+0000","last_became_active":"2024-05-08T05:41:36.151495+0000","last_became_peered":"2024-05-08T05:41:36.151495+0000","last_unstale":"2024-05-08T05:41:37.153781+0000","last_undegraded":"2024-05-08T05:41:37.153781+0000","last_fullsized":"2024-05-08T05:41:37.153781+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:34.172975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575332+0000","last_change":"2024-05-08T05:41:32.137651+0000","last_active":"2024-05-08T05:41:37.575332+0000","last_peered":"2024-05-08T05:41:37.575332+0000","last_clean":"2024-05-08T05:41:37.575332+0000","last_became_active":"2024-05-08T05:41:32.137380+0000","last_became_peered":"2024-05-08T05:41:32.137380+0000","last_unstale":"2024-05-08T05:41:37.575332+0000","last_undegraded":"2024-05-08T05:41:37.575332+0000","last_fullsized":"2024-05-08T05:41:37.575332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:44.038894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577757+0000","last_change":"2024-05-08T05:41:31.132598+0000","last_active":"2024-05-08T05:41:37.577757+0000","last_peered":"2024-05-08T05:41:37.577757+0000","last_clean":"2024-05-08T05:41:37.577757+0000","last_became_active":"2024-05-08T05:41:31.131053+0000","last_became_peered":"2024-05-08T05:41:31.131053+0000","last_unstale":"2024-05-08T05:41:37.577757+0000","last_undegraded":"2024-05-08T05:41:37.577757+0000","last_fullsized":"2024-05-08T05:41:37.577757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:11:56.288788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575991+0000","last_change":"2024-05-08T05:41:36.159470+0000","last_active":"2024-05-08T05:41:37.575991+0000","last_peered":"2024-05-08T05:41:37.575991+0000","last_clean":"2024-05-08T05:41:37.575991+0000","last_became_active":"2024-05-08T05:41:36.158246+0000","last_became_peered":"2024-05-08T05:41:36.158246+0000","last_unstale":"2024-05-08T05:41:37.575991+0000","last_undegraded":"2024-05-08T05:41:37.575991+0000","last_fullsized":"2024-05-08T05:41:37.575991+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:02.479497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577667+0000","last_change":"2024-05-08T05:41:32.139258+0000","last_active":"2024-05-08T05:41:37.577667+0000","last_peered":"2024-05-08T05:41:37.577667+0000","last_clean":"2024-05-08T05:41:37.577667+0000","last_became_active":"2024-05-08T05:41:32.138975+0000","last_became_peered":"2024-05-08T05:41:32.138975+0000","last_unstale":"2024-05-08T05:41:37.577667+0000","last_undegraded":"2024-05-08T05:41:37.577667+0000","last_fullsized":"2024-05-08T05:41:37.577667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:16.501105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577263+0000","last_change":"2024-05-08T05:41:31.133643+0000","last_active":"2024-05-08T05:41:37.577263+0000","last_peered":"2024-05-08T05:41:37.577263+0000","last_clean":"2024-05-08T05:41:37.577263+0000","last_became_active":"2024-05-08T05:41:31.133256+0000","last_became_peered":"2024-05-08T05:41:31.133256+0000","last_unstale":"2024-05-08T05:41:37.577263+0000","last_undegraded":"2024-05-08T05:41:37.577263+0000","last_fullsized":"2024-05-08T05:41:37.577263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:07.648895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576551+0000","last_change":"2024-05-08T05:41:36.383032+0000","last_active":"2024-05-08T05:41:37.576551+0000","last_peered":"2024-05-08T05:41:37.576551+0000","last_clean":"2024-05-08T05:41:37.576551+0000","last_became_active":"2024-05-08T05:41:36.382632+0000","last_became_peered":"2024-05-08T05:41:36.382632+0000","last_unstale":"2024-05-08T05:41:37.576551+0000","last_undegraded":"2024-05-08T05:41:37.576551+0000","last_fullsized":"2024-05-08T05:41:37.576551+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:07:08.116447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575585+0000","last_change":"2024-05-08T05:41:36.385447+0000","last_active":"2024-05-08T05:41:37.575585+0000","last_peered":"2024-05-08T05:41:37.575585+0000","last_clean":"2024-05-08T05:41:37.575585+0000","last_became_active":"2024-05-08T05:41:36.385062+0000","last_became_peered":"2024-05-08T05:41:36.385062+0000","last_unstale":"2024-05-08T05:41:37.575585+0000","last_undegraded":"2024-05-08T05:41:37.575585+0000","last_fullsized":"2024-05-08T05:41:37.575585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:06.547337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.574973+0000","last_change":"2024-05-08T05:41:32.127321+0000","last_active":"2024-05-08T05:41:37.574973+0000","last_peered":"2024-05-08T05:41:37.574973+0000","last_clean":"2024-05-08T05:41:37.574973+0000","last_became_active":"2024-05-08T05:41:32.127028+0000","last_became_peered":"2024-05-08T05:41:32.127028+0000","last_unstale":"2024-05-08T05:41:37.574973+0000","last_undegraded":"2024-05-08T05:41:37.574973+0000","last_fullsized":"2024-05-08T05:41:37.574973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:47.493805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575819+0000","last_change":"2024-05-08T05:41:31.131618+0000","last_active":"2024-05-08T05:41:37.575819+0000","last_peered":"2024-05-08T05:41:37.575819+0000","last_clean":"2024-05-08T05:41:37.575819+0000","last_became_active":"2024-05-08T05:41:31.131012+0000","last_became_peered":"2024-05-08T05:41:31.131012+0000","last_unstale":"2024-05-08T05:41:37.575819+0000","last_undegraded":"2024-05-08T05:41:37.575819+0000","last_fullsized":"2024-05-08T05:41:37.575819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:22.751500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577121+0000","last_change":"2024-05-08T05:41:36.158506+0000","last_active":"2024-05-08T05:41:37.577121+0000","last_peered":"2024-05-08T05:41:37.577121+0000","last_clean":"2024-05-08T05:41:37.577121+0000","last_became_active":"2024-05-08T05:41:36.158020+0000","last_became_peered":"2024-05-08T05:41:36.158020+0000","last_unstale":"2024-05-08T05:41:37.577121+0000","last_undegraded":"2024-05-08T05:41:37.577121+0000","last_fullsized":"2024-05-08T05:41:37.577121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:20.531197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576038+0000","last_change":"2024-05-08T05:41:32.132554+0000","last_active":"2024-05-08T05:41:37.576038+0000","last_peered":"2024-05-08T05:41:37.576038+0000","last_clean":"2024-05-08T05:41:37.576038+0000","last_became_active":"2024-05-08T05:41:32.132257+0000","last_became_peered":"2024-05-08T05:41:32.132257+0000","last_unstale":"2024-05-08T05:41:37.576038+0000","last_undegraded":"2024-05-08T05:41:37.576038+0000","last_fullsized":"2024-05-08T05:41:37.576038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:16:33.854897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576780+0000","last_change":"2024-05-08T05:41:31.124710+0000","last_active":"2024-05-08T05:41:37.576780+0000","last_peered":"2024-05-08T05:41:37.576780+0000","last_clean":"2024-05-08T05:41:37.576780+0000","last_became_active":"2024-05-08T05:41:31.124304+0000","last_became_peered":"2024-05-08T05:41:31.124304+0000","last_unstale":"2024-05-08T05:41:37.576780+0000","last_undegraded":"2024-05-08T05:41:37.576780+0000","last_fullsized":"2024-05-08T05:41:37.576780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:49.171947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576513+0000","last_change":"2024-05-08T05:41:31.123731+0000","last_active":"2024-05-08T05:41:37.576513+0000","last_peered":"2024-05-08T05:41:37.576513+0000","last_clean":"2024-05-08T05:41:37.576513+0000","last_became_active":"2024-05-08T05:41:31.123423+0000","last_became_peered":"2024-05-08T05:41:31.123423+0000","last_unstale":"2024-05-08T05:41:37.576513+0000","last_undegraded":"2024-05-08T05:41:37.576513+0000","last_fullsized":"2024-05-08T05:41:37.576513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:24.722618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576325+0000","last_change":"2024-05-08T05:41:32.135788+0000","last_active":"2024-05-08T05:41:37.576325+0000","last_peered":"2024-05-08T05:41:37.576325+0000","last_clean":"2024-05-08T05:41:37.576325+0000","last_became_active":"2024-05-08T05:41:32.135522+0000","last_became_peered":"2024-05-08T05:41:32.135522+0000","last_unstale":"2024-05-08T05:41:37.576325+0000","last_undegraded":"2024-05-08T05:41:37.576325+0000","last_fullsized":"2024-05-08T05:41:37.576325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:42.041915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.148134+0000","last_change":"2024-05-08T05:41:36.155967+0000","last_active":"2024-05-08T05:41:37.148134+0000","last_peered":"2024-05-08T05:41:37.148134+0000","last_clean":"2024-05-08T05:41:37.148134+0000","last_became_active":"2024-05-08T05:41:36.155132+0000","last_became_peered":"2024-05-08T05:41:36.155132+0000","last_unstale":"2024-05-08T05:41:37.148134+0000","last_undegraded":"2024-05-08T05:41:37.148134+0000","last_fullsized":"2024-05-08T05:41:37.148134+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:58.729110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577152+0000","last_change":"2024-05-08T05:41:31.127645+0000","last_active":"2024-05-08T05:41:37.577152+0000","last_peered":"2024-05-08T05:41:37.577152+0000","last_clean":"2024-05-08T05:41:37.577152+0000","last_became_active":"2024-05-08T05:41:31.127330+0000","last_became_peered":"2024-05-08T05:41:31.127330+0000","last_unstale":"2024-05-08T05:41:37.577152+0000","last_undegraded":"2024-05-08T05:41:37.577152+0000","last_fullsized":"2024-05-08T05:41:37.577152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:22.751500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153479+0000","last_change":"2024-05-08T05:41:32.125718+0000","last_active":"2024-05-08T05:41:37.153479+0000","last_peered":"2024-05-08T05:41:37.153479+0000","last_clean":"2024-05-08T05:41:37.153479+0000","last_became_active":"2024-05-08T05:41:32.125435+0000","last_became_peered":"2024-05-08T05:41:32.125435+0000","last_unstale":"2024-05-08T05:41:37.153479+0000","last_undegraded":"2024-05-08T05:41:37.153479+0000","last_fullsized":"2024-05-08T05:41:37.153479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:39.180293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575823+0000","last_change":"2024-05-08T05:41:36.158103+0000","last_active":"2024-05-08T05:41:37.575823+0000","last_peered":"2024-05-08T05:41:37.575823+0000","last_clean":"2024-05-08T05:41:37.575823+0000","last_became_active":"2024-05-08T05:41:36.157668+0000","last_became_peered":"2024-05-08T05:41:36.157668+0000","last_unstale":"2024-05-08T05:41:37.575823+0000","last_undegraded":"2024-05-08T05:41:37.575823+0000","last_fullsized":"2024-05-08T05:41:37.575823+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:45:24.742821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575306+0000","last_change":"2024-05-08T05:41:32.137321+0000","last_active":"2024-05-08T05:41:37.575306+0000","last_peered":"2024-05-08T05:41:37.575306+0000","last_clean":"2024-05-08T05:41:37.575306+0000","last_became_active":"2024-05-08T05:41:32.136738+0000","last_became_peered":"2024-05-08T05:41:32.136738+0000","last_unstale":"2024-05-08T05:41:37.575306+0000","last_undegraded":"2024-05-08T05:41:37.575306+0000","last_fullsized":"2024-05-08T05:41:37.575306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:16:33.854897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.147790+0000","last_change":"2024-05-08T05:41:32.127130+0000","last_active":"2024-05-08T05:41:37.147790+0000","last_peered":"2024-05-08T05:41:37.147790+0000","last_clean":"2024-05-08T05:41:37.147790+0000","last_became_active":"2024-05-08T05:41:32.125853+0000","last_became_peered":"2024-05-08T05:41:32.125853+0000","last_unstale":"2024-05-08T05:41:37.147790+0000","last_undegraded":"2024-05-08T05:41:37.147790+0000","last_fullsized":"2024-05-08T05:41:37.147790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:47.424136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575968+0000","last_change":"2024-05-08T05:41:32.139558+0000","last_active":"2024-05-08T05:41:37.575968+0000","last_peered":"2024-05-08T05:41:37.575968+0000","last_clean":"2024-05-08T05:41:37.575968+0000","last_became_active":"2024-05-08T05:41:32.139243+0000","last_became_peered":"2024-05-08T05:41:32.139243+0000","last_unstale":"2024-05-08T05:41:37.575968+0000","last_undegraded":"2024-05-08T05:41:37.575968+0000","last_fullsized":"2024-05-08T05:41:37.575968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:07:26.082084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576235+0000","last_change":"2024-05-08T05:41:32.126388+0000","last_active":"2024-05-08T05:41:37.576235+0000","last_peered":"2024-05-08T05:41:37.576235+0000","last_clean":"2024-05-08T05:41:37.576235+0000","last_became_active":"2024-05-08T05:41:32.126078+0000","last_became_peered":"2024-05-08T05:41:32.126078+0000","last_unstale":"2024-05-08T05:41:37.576235+0000","last_undegraded":"2024-05-08T05:41:37.576235+0000","last_fullsized":"2024-05-08T05:41:37.576235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:53.563878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577307+0000","last_change":"2024-05-08T05:41:32.135523+0000","last_active":"2024-05-08T05:41:37.577307+0000","last_peered":"2024-05-08T05:41:37.577307+0000","last_clean":"2024-05-08T05:41:37.577307+0000","last_became_active":"2024-05-08T05:41:32.135248+0000","last_became_peered":"2024-05-08T05:41:32.135248+0000","last_unstale":"2024-05-08T05:41:37.577307+0000","last_undegraded":"2024-05-08T05:41:37.577307+0000","last_fullsized":"2024-05-08T05:41:37.577307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:42.041915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576262+0000","last_change":"2024-05-08T05:41:32.130227+0000","last_active":"2024-05-08T05:41:37.576262+0000","last_peered":"2024-05-08T05:41:37.576262+0000","last_clean":"2024-05-08T05:41:37.576262+0000","last_became_active":"2024-05-08T05:41:32.129175+0000","last_became_peered":"2024-05-08T05:41:32.129175+0000","last_unstale":"2024-05-08T05:41:37.576262+0000","last_undegraded":"2024-05-08T05:41:37.576262+0000","last_fullsized":"2024-05-08T05:41:37.576262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:53.563878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577357+0000","last_change":"2024-05-08T05:41:32.143284+0000","last_active":"2024-05-08T05:41:37.577357+0000","last_peered":"2024-05-08T05:41:37.577357+0000","last_clean":"2024-05-08T05:41:37.577357+0000","last_became_active":"2024-05-08T05:41:32.142815+0000","last_became_peered":"2024-05-08T05:41:32.142815+0000","last_unstale":"2024-05-08T05:41:37.577357+0000","last_undegraded":"2024-05-08T05:41:37.577357+0000","last_fullsized":"2024-05-08T05:41:37.577357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:53.563878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576778+0000","last_change":"2024-05-08T05:41:32.134965+0000","last_active":"2024-05-08T05:41:37.576778+0000","last_peered":"2024-05-08T05:41:37.576778+0000","last_clean":"2024-05-08T05:41:37.576778+0000","last_became_active":"2024-05-08T05:41:32.134641+0000","last_became_peered":"2024-05-08T05:41:32.134641+0000","last_unstale":"2024-05-08T05:41:37.576778+0000","last_undegraded":"2024-05-08T05:41:37.576778+0000","last_fullsized":"2024-05-08T05:41:37.576778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:30.142883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575749+0000","last_change":"2024-05-08T05:41:32.135139+0000","last_active":"2024-05-08T05:41:37.575749+0000","last_peered":"2024-05-08T05:41:37.575749+0000","last_clean":"2024-05-08T05:41:37.575749+0000","last_became_active":"2024-05-08T05:41:32.134839+0000","last_became_peered":"2024-05-08T05:41:32.134839+0000","last_unstale":"2024-05-08T05:41:37.575749+0000","last_undegraded":"2024-05-08T05:41:37.575749+0000","last_fullsized":"2024-05-08T05:41:37.575749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:53.563878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576660+0000","last_change":"2024-05-08T05:41:32.134673+0000","last_active":"2024-05-08T05:41:37.576660+0000","last_peered":"2024-05-08T05:41:37.576660+0000","last_clean":"2024-05-08T05:41:37.576660+0000","last_became_active":"2024-05-08T05:41:32.134402+0000","last_became_peered":"2024-05-08T05:41:32.134402+0000","last_unstale":"2024-05-08T05:41:37.576660+0000","last_undegraded":"2024-05-08T05:41:37.576660+0000","last_fullsized":"2024-05-08T05:41:37.576660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:03:20.127059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575846+0000","last_change":"2024-05-08T05:41:32.140242+0000","last_active":"2024-05-08T05:41:37.575846+0000","last_peered":"2024-05-08T05:41:37.575846+0000","last_clean":"2024-05-08T05:41:37.575846+0000","last_became_active":"2024-05-08T05:41:32.139932+0000","last_became_peered":"2024-05-08T05:41:32.139932+0000","last_unstale":"2024-05-08T05:41:37.575846+0000","last_undegraded":"2024-05-08T05:41:37.575846+0000","last_fullsized":"2024-05-08T05:41:37.575846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:42.739734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575975+0000","last_change":"2024-05-08T05:41:32.132509+0000","last_active":"2024-05-08T05:41:37.575975+0000","last_peered":"2024-05-08T05:41:37.575975+0000","last_clean":"2024-05-08T05:41:37.575975+0000","last_became_active":"2024-05-08T05:41:32.132240+0000","last_became_peered":"2024-05-08T05:41:32.132240+0000","last_unstale":"2024-05-08T05:41:37.575975+0000","last_undegraded":"2024-05-08T05:41:37.575975+0000","last_fullsized":"2024-05-08T05:41:37.575975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:07:26.071605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576973+0000","last_change":"2024-05-08T05:41:32.140982+0000","last_active":"2024-05-08T05:41:37.576973+0000","last_peered":"2024-05-08T05:41:37.576973+0000","last_clean":"2024-05-08T05:41:37.576973+0000","last_became_active":"2024-05-08T05:41:32.140698+0000","last_became_peered":"2024-05-08T05:41:32.140698+0000","last_unstale":"2024-05-08T05:41:37.576973+0000","last_undegraded":"2024-05-08T05:41:37.576973+0000","last_fullsized":"2024-05-08T05:41:37.576973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:07:26.071605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577173+0000","last_change":"2024-05-08T05:41:32.134190+0000","last_active":"2024-05-08T05:41:37.577173+0000","last_peered":"2024-05-08T05:41:37.577173+0000","last_clean":"2024-05-08T05:41:37.577173+0000","last_became_active":"2024-05-08T05:41:32.132164+0000","last_became_peered":"2024-05-08T05:41:32.132164+0000","last_unstale":"2024-05-08T05:41:37.577173+0000","last_undegraded":"2024-05-08T05:41:37.577173+0000","last_fullsized":"2024-05-08T05:41:37.577173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:13.172615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.148157+0000","last_change":"2024-05-08T05:41:32.131091+0000","last_active":"2024-05-08T05:41:37.148157+0000","last_peered":"2024-05-08T05:41:37.148157+0000","last_clean":"2024-05-08T05:41:37.148157+0000","last_became_active":"2024-05-08T05:41:32.130767+0000","last_became_peered":"2024-05-08T05:41:32.130767+0000","last_unstale":"2024-05-08T05:41:37.148157+0000","last_undegraded":"2024-05-08T05:41:37.148157+0000","last_fullsized":"2024-05-08T05:41:37.148157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:42.596678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577931+0000","last_change":"2024-05-08T05:41:32.137943+0000","last_active":"2024-05-08T05:41:37.577931+0000","last_peered":"2024-05-08T05:41:37.577931+0000","last_clean":"2024-05-08T05:41:37.577931+0000","last_became_active":"2024-05-08T05:41:32.137662+0000","last_became_peered":"2024-05-08T05:41:32.137662+0000","last_unstale":"2024-05-08T05:41:37.577931+0000","last_undegraded":"2024-05-08T05:41:37.577931+0000","last_fullsized":"2024-05-08T05:41:37.577931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:17.291314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575635+0000","last_change":"2024-05-08T05:41:32.129374+0000","last_active":"2024-05-08T05:41:37.575635+0000","last_peered":"2024-05-08T05:41:37.575635+0000","last_clean":"2024-05-08T05:41:37.575635+0000","last_became_active":"2024-05-08T05:41:32.128594+0000","last_became_peered":"2024-05-08T05:41:32.128594+0000","last_unstale":"2024-05-08T05:41:37.575635+0000","last_undegraded":"2024-05-08T05:41:37.575635+0000","last_fullsized":"2024-05-08T05:41:37.575635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:01:32.075337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575072+0000","last_change":"2024-05-08T05:41:32.131159+0000","last_active":"2024-05-08T05:41:37.575072+0000","last_peered":"2024-05-08T05:41:37.575072+0000","last_clean":"2024-05-08T05:41:37.575072+0000","last_became_active":"2024-05-08T05:41:32.130766+0000","last_became_peered":"2024-05-08T05:41:32.130766+0000","last_unstale":"2024-05-08T05:41:37.575072+0000","last_undegraded":"2024-05-08T05:41:37.575072+0000","last_fullsized":"2024-05-08T05:41:37.575072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:16:33.854897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575282+0000","last_change":"2024-05-08T05:41:32.119797+0000","last_active":"2024-05-08T05:41:37.575282+0000","last_peered":"2024-05-08T05:41:37.575282+0000","last_clean":"2024-05-08T05:41:37.575282+0000","last_became_active":"2024-05-08T05:41:32.119134+0000","last_became_peered":"2024-05-08T05:41:32.119134+0000","last_unstale":"2024-05-08T05:41:37.575282+0000","last_undegraded":"2024-05-08T05:41:37.575282+0000","last_fullsized":"2024-05-08T05:41:37.575282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:47.493805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577070+0000","last_change":"2024-05-08T05:41:32.135351+0000","last_active":"2024-05-08T05:41:37.577070+0000","last_peered":"2024-05-08T05:41:37.577070+0000","last_clean":"2024-05-08T05:41:37.577070+0000","last_became_active":"2024-05-08T05:41:32.133812+0000","last_became_peered":"2024-05-08T05:41:32.133812+0000","last_unstale":"2024-05-08T05:41:37.577070+0000","last_undegraded":"2024-05-08T05:41:37.577070+0000","last_fullsized":"2024-05-08T05:41:37.577070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:29.473897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154048+0000","last_change":"2024-05-08T05:41:32.136108+0000","last_active":"2024-05-08T05:41:37.154048+0000","last_peered":"2024-05-08T05:41:37.154048+0000","last_clean":"2024-05-08T05:41:37.154048+0000","last_became_active":"2024-05-08T05:41:32.135622+0000","last_became_peered":"2024-05-08T05:41:32.135622+0000","last_unstale":"2024-05-08T05:41:37.154048+0000","last_undegraded":"2024-05-08T05:41:37.154048+0000","last_fullsized":"2024-05-08T05:41:37.154048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:14.470927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576751+0000","last_change":"2024-05-08T05:41:32.138499+0000","last_active":"2024-05-08T05:41:37.576751+0000","last_peered":"2024-05-08T05:41:37.576751+0000","last_clean":"2024-05-08T05:41:37.576751+0000","last_became_active":"2024-05-08T05:41:32.138049+0000","last_became_peered":"2024-05-08T05:41:32.138049+0000","last_unstale":"2024-05-08T05:41:37.576751+0000","last_undegraded":"2024-05-08T05:41:37.576751+0000","last_fullsized":"2024-05-08T05:41:37.576751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:03:20.127059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577193+0000","last_change":"2024-05-08T05:41:32.138310+0000","last_active":"2024-05-08T05:41:37.577193+0000","last_peered":"2024-05-08T05:41:37.577193+0000","last_clean":"2024-05-08T05:41:37.577193+0000","last_became_active":"2024-05-08T05:41:32.137983+0000","last_became_peered":"2024-05-08T05:41:32.137983+0000","last_unstale":"2024-05-08T05:41:37.577193+0000","last_undegraded":"2024-05-08T05:41:37.577193+0000","last_fullsized":"2024-05-08T05:41:37.577193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:42.041915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577267+0000","last_change":"2024-05-08T05:41:32.144208+0000","last_active":"2024-05-08T05:41:37.577267+0000","last_peered":"2024-05-08T05:41:37.577267+0000","last_clean":"2024-05-08T05:41:37.577267+0000","last_became_active":"2024-05-08T05:41:32.143884+0000","last_became_peered":"2024-05-08T05:41:32.143884+0000","last_unstale":"2024-05-08T05:41:37.577267+0000","last_undegraded":"2024-05-08T05:41:37.577267+0000","last_fullsized":"2024-05-08T05:41:37.577267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:07:26.082084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575800+0000","last_change":"2024-05-08T05:41:32.136185+0000","last_active":"2024-05-08T05:41:37.575800+0000","last_peered":"2024-05-08T05:41:37.575800+0000","last_clean":"2024-05-08T05:41:37.575800+0000","last_became_active":"2024-05-08T05:41:32.135878+0000","last_became_peered":"2024-05-08T05:41:32.135878+0000","last_unstale":"2024-05-08T05:41:37.575800+0000","last_undegraded":"2024-05-08T05:41:37.575800+0000","last_fullsized":"2024-05-08T05:41:37.575800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:44.038894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576529+0000","last_change":"2024-05-08T05:41:32.137009+0000","last_active":"2024-05-08T05:41:37.576529+0000","last_peered":"2024-05-08T05:41:37.576529+0000","last_clean":"2024-05-08T05:41:37.576529+0000","last_became_active":"2024-05-08T05:41:32.136708+0000","last_became_peered":"2024-05-08T05:41:32.136708+0000","last_unstale":"2024-05-08T05:41:37.576529+0000","last_undegraded":"2024-05-08T05:41:37.576529+0000","last_fullsized":"2024-05-08T05:41:37.576529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:30.142883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575732+0000","last_change":"2024-05-08T05:41:32.137351+0000","last_active":"2024-05-08T05:41:37.575732+0000","last_peered":"2024-05-08T05:41:37.575732+0000","last_clean":"2024-05-08T05:41:37.575732+0000","last_became_active":"2024-05-08T05:41:32.135445+0000","last_became_peered":"2024-05-08T05:41:32.135445+0000","last_unstale":"2024-05-08T05:41:37.575732+0000","last_undegraded":"2024-05-08T05:41:37.575732+0000","last_fullsized":"2024-05-08T05:41:37.575732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:30.142883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575746+0000","last_change":"2024-05-08T05:41:32.132566+0000","last_active":"2024-05-08T05:41:37.575746+0000","last_peered":"2024-05-08T05:41:37.575746+0000","last_clean":"2024-05-08T05:41:37.575746+0000","last_became_active":"2024-05-08T05:41:32.131787+0000","last_became_peered":"2024-05-08T05:41:32.131787+0000","last_unstale":"2024-05-08T05:41:37.575746+0000","last_undegraded":"2024-05-08T05:41:37.575746+0000","last_fullsized":"2024-05-08T05:41:37.575746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:42.739734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576521+0000","last_change":"2024-05-08T05:41:32.135611+0000","last_active":"2024-05-08T05:41:37.576521+0000","last_peered":"2024-05-08T05:41:37.576521+0000","last_clean":"2024-05-08T05:41:37.576521+0000","last_became_active":"2024-05-08T05:41:32.135346+0000","last_became_peered":"2024-05-08T05:41:32.135346+0000","last_unstale":"2024-05-08T05:41:37.576521+0000","last_undegraded":"2024-05-08T05:41:37.576521+0000","last_fullsized":"2024-05-08T05:41:37.576521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:29.473897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575102+0000","last_change":"2024-05-08T05:41:32.118779+0000","last_active":"2024-05-08T05:41:37.575102+0000","last_peered":"2024-05-08T05:41:37.575102+0000","last_clean":"2024-05-08T05:41:37.575102+0000","last_became_active":"2024-05-08T05:41:32.118481+0000","last_became_peered":"2024-05-08T05:41:32.118481+0000","last_unstale":"2024-05-08T05:41:37.575102+0000","last_undegraded":"2024-05-08T05:41:37.575102+0000","last_fullsized":"2024-05-08T05:41:37.575102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:54.699019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575394+0000","last_change":"2024-05-08T05:41:32.128168+0000","last_active":"2024-05-08T05:41:37.575394+0000","last_peered":"2024-05-08T05:41:37.575394+0000","last_clean":"2024-05-08T05:41:37.575394+0000","last_became_active":"2024-05-08T05:41:32.127869+0000","last_became_peered":"2024-05-08T05:41:32.127869+0000","last_unstale":"2024-05-08T05:41:37.575394+0000","last_undegraded":"2024-05-08T05:41:37.575394+0000","last_fullsized":"2024-05-08T05:41:37.575394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:07:26.071605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577048+0000","last_change":"2024-05-08T05:41:32.134768+0000","last_active":"2024-05-08T05:41:37.577048+0000","last_peered":"2024-05-08T05:41:37.577048+0000","last_clean":"2024-05-08T05:41:37.577048+0000","last_became_active":"2024-05-08T05:41:32.132540+0000","last_became_peered":"2024-05-08T05:41:32.132540+0000","last_unstale":"2024-05-08T05:41:37.577048+0000","last_undegraded":"2024-05-08T05:41:37.577048+0000","last_fullsized":"2024-05-08T05:41:37.577048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:50:32.252569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":847872,"data_stored":690366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":101028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":99506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":99542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":262144,"data_stored":103258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":249856,"data_stored":99416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":258048,"data_stored":99576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":856064,"data_stored":690422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:41:46.224 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:41:46.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.363+0000 7fee4aca0640 1 Processor -- start 2024-05-08T05:41:46.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.363+0000 7fee4aca0640 1 -- start start 2024-05-08T05:41:46.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.364+0000 7fee4aca0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fee4415f450 0x7fee4406d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:46.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.364+0000 7fee4aca0640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fee4415f830 0x7fee4406d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:46.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.364+0000 7fee4aca0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee440d0cb0 con 0x7fee440d0700 2024-05-08T05:41:46.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.364+0000 7fee4aca0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fee4406d690 con 0x7fee4415f450 2024-05-08T05:41:46.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.364+0000 7fee4aca0640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fee440d0f90 con 0x7fee4415f830 2024-05-08T05:41:46.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.364+0000 7fee48a3c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fee4415f450 0x7fee4406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:46.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.364+0000 7fee4923d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fee440d0700 0x7fee440d17e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43912/0 (socket says 172.21.15.45:43912) 2024-05-08T05:41:46.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.364+0000 7fee4923d640 1 -- 172.21.15.45:0/3142516860 learned_addr learned my addr 172.21.15.45:0/3142516860 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:46.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.364+0000 7fee43fff640 1 --2- 172.21.15.45:0/3142516860 >> v2:172.21.15.63:3300/0 conn(0x7fee4415f830 0x7fee4406d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:46.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.365+0000 7fee48a3c640 1 -- 172.21.15.45:0/3142516860 >> v2:172.21.15.63:3300/0 conn(0x7fee4415f830 msgr2=0x7fee4406d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:46.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.365+0000 7fee48a3c640 1 --2- 172.21.15.45:0/3142516860 >> v2:172.21.15.63:3300/0 conn(0x7fee4415f830 0x7fee4406d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.365+0000 7fee48a3c640 1 -- 172.21.15.45:0/3142516860 >> v1:172.21.15.45:6789/0 conn(0x7fee440d0700 legacy=0x7fee440d17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:46.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.365+0000 7fee48a3c640 1 -- 172.21.15.45:0/3142516860 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee440d2fe0 con 0x7fee4415f450 2024-05-08T05:41:46.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.365+0000 7fee437fe640 1 -- 172.21.15.45:0/3142516860 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1110982246 0 0) 0x7fee440d0cb0 con 0x7fee440d0700 2024-05-08T05:41:46.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.365+0000 7fee43fff640 1 --2- 172.21.15.45:0/3142516860 >> v2:172.21.15.63:3300/0 conn(0x7fee4415f830 0x7fee4406d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:46.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.365+0000 7fee48a3c640 1 --2- 172.21.15.45:0/3142516860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee4415f450 0x7fee4406d150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fee4406f170 tx=0x7fee3402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d771341b3810e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:46.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.365+0000 7fee437fe640 1 -- 172.21.15.45:0/3142516860 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee34037070 con 0x7fee4415f450 2024-05-08T05:41:46.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.366+0000 7fee437fe640 1 -- 172.21.15.45:0/3142516860 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee34032440 con 0x7fee4415f450 2024-05-08T05:41:46.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.366+0000 7fee437fe640 1 -- 172.21.15.45:0/3142516860 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee34032810 con 0x7fee4415f450 2024-05-08T05:41:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.366+0000 7fee4aca0640 1 -- 172.21.15.45:0/3142516860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee4415f450 msgr2=0x7fee4406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.366+0000 7fee4aca0640 1 --2- 172.21.15.45:0/3142516860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee4415f450 0x7fee4406d150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fee4406f170 tx=0x7fee3402a1a0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.366+0000 7fee4aca0640 1 -- 172.21.15.45:0/3142516860 shutdown_connections 2024-05-08T05:41:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.366+0000 7fee4aca0640 1 --2- 172.21.15.45:0/3142516860 >> v2:172.21.15.63:3300/0 conn(0x7fee4415f830 0x7fee4406d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.366+0000 7fee4aca0640 1 --2- 172.21.15.45:0/3142516860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee4415f450 0x7fee4406d150 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.366+0000 7fee4aca0640 1 -- 172.21.15.45:0/3142516860 >> 172.21.15.45:0/3142516860 conn(0x7fee440d5040 msgr2=0x7fee440d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.366+0000 7fee4aca0640 1 -- 172.21.15.45:0/3142516860 shutdown_connections 2024-05-08T05:41:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.366+0000 7fee4aca0640 1 -- 172.21.15.45:0/3142516860 wait complete. 2024-05-08T05:41:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.367+0000 7fee4aca0640 1 Processor -- start 2024-05-08T05:41:46.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.367+0000 7fee4aca0640 1 -- start start 2024-05-08T05:41:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.368+0000 7fee4aca0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee440d0700 0x7fee44161240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.368+0000 7fee4aca0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee4415f450 0x7fee44152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.368+0000 7fee4aca0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee4415f830 0x7fee440d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.368+0000 7fee4aca0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fee440d4c50 con 0x7fee440d0700 2024-05-08T05:41:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.368+0000 7fee4aca0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fee440d4ad0 con 0x7fee4415f450 2024-05-08T05:41:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.368+0000 7fee4aca0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fee440d4dd0 con 0x7fee4415f830 2024-05-08T05:41:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.368+0000 7fee48a3c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee440d0700 0x7fee44161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.368+0000 7fee48a3c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee440d0700 0x7fee44161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51220/0 (socket says 172.21.15.45:51220) 2024-05-08T05:41:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.368+0000 7fee48a3c640 1 -- 172.21.15.45:0/364093600 learned_addr learned my addr 172.21.15.45:0/364093600 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.369+0000 7fee43fff640 1 --2- 172.21.15.45:0/364093600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee4415f450 0x7fee44152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.369+0000 7fee4923d640 1 --2- 172.21.15.45:0/364093600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee4415f830 0x7fee440d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.369+0000 7fee48a3c640 1 -- 172.21.15.45:0/364093600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee4415f830 msgr2=0x7fee440d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.369+0000 7fee48a3c640 1 --2- 172.21.15.45:0/364093600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee4415f830 0x7fee440d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.369+0000 7fee48a3c640 1 -- 172.21.15.45:0/364093600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee4415f450 msgr2=0x7fee44152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:46.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.369+0000 7fee48a3c640 1 --2- 172.21.15.45:0/364093600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee4415f450 0x7fee44152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.369+0000 7fee48a3c640 1 -- 172.21.15.45:0/364093600 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee440bae80 con 0x7fee440d0700 2024-05-08T05:41:46.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.369+0000 7fee43fff640 1 --2- 172.21.15.45:0/364093600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee4415f450 0x7fee44152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:46.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.369+0000 7fee48a3c640 1 --2- 172.21.15.45:0/364093600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee440d0700 0x7fee44161240 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fee34000c00 tx=0x7fee340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:46.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.369+0000 7fee417fa640 1 -- 172.21.15.45:0/364093600 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee34037050 con 0x7fee440d0700 2024-05-08T05:41:46.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.369+0000 7fee417fa640 1 -- 172.21.15.45:0/364093600 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee34038040 con 0x7fee440d0700 2024-05-08T05:41:46.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.369+0000 7fee417fa640 1 -- 172.21.15.45:0/364093600 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee34006e10 con 0x7fee440d0700 2024-05-08T05:41:46.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.369+0000 7fee4aca0640 1 -- 172.21.15.45:0/364093600 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee441618d0 con 0x7fee440d0700 2024-05-08T05:41:46.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.369+0000 7fee4aca0640 1 -- 172.21.15.45:0/364093600 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee44161e90 con 0x7fee440d0700 2024-05-08T05:41:46.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.370+0000 7fee4aca0640 1 -- 172.21.15.45:0/364093600 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee440693f0 con 0x7fee440d0700 2024-05-08T05:41:46.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.371+0000 7fee417fa640 1 -- 172.21.15.45:0/364093600 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fee34030980 con 0x7fee440d0700 2024-05-08T05:41:46.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.371+0000 7fee417fa640 1 --2- 172.21.15.45:0/364093600 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fee240762d0 0x7fee24078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:46.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.371+0000 7fee417fa640 1 -- 172.21.15.45:0/364093600 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fee340b83c0 con 0x7fee440d0700 2024-05-08T05:41:46.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.371+0000 7fee43fff640 1 --2- 172.21.15.45:0/364093600 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fee240762d0 0x7fee24078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:46.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.372+0000 7fee43fff640 1 --2- 172.21.15.45:0/364093600 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fee240762d0 0x7fee24078790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fee441611d0 tx=0x7fee2c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:46.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.376+0000 7fee417fa640 1 -- 172.21.15.45:0/364093600 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fee34081a70 con 0x7fee440d0700 2024-05-08T05:41:46.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.565+0000 7fee4aca0640 1 -- 172.21.15.45:0/364093600 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fee440c9a20 con 0x7fee240762d0 2024-05-08T05:41:46.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.574+0000 7fee417fa640 1 -- 172.21.15.45:0/364093600 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332563 (secure 0 0 0) 0x7fee440c9a20 con 0x7fee240762d0 2024-05-08T05:41:46.578 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:46.579 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:41:46.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.577+0000 7fee4aca0640 1 -- 172.21.15.45:0/364093600 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fee240762d0 msgr2=0x7fee24078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:46.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.577+0000 7fee4aca0640 1 --2- 172.21.15.45:0/364093600 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fee240762d0 0x7fee24078790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fee441611d0 tx=0x7fee2c008040 comp rx=0 tx=0).stop 2024-05-08T05:41:46.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.577+0000 7fee4aca0640 1 -- 172.21.15.45:0/364093600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee440d0700 msgr2=0x7fee44161240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:46.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.577+0000 7fee4aca0640 1 --2- 172.21.15.45:0/364093600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee440d0700 0x7fee44161240 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fee34000c00 tx=0x7fee340046c0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.577+0000 7fee4aca0640 1 -- 172.21.15.45:0/364093600 shutdown_connections 2024-05-08T05:41:46.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.577+0000 7fee4aca0640 1 --2- 172.21.15.45:0/364093600 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fee240762d0 0x7fee24078790 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.577+0000 7fee4aca0640 1 --2- 172.21.15.45:0/364093600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee4415f830 0x7fee440d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.577+0000 7fee4aca0640 1 --2- 172.21.15.45:0/364093600 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee4415f450 0x7fee44152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.577+0000 7fee4aca0640 1 --2- 172.21.15.45:0/364093600 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee440d0700 0x7fee44161240 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.577+0000 7fee4aca0640 1 -- 172.21.15.45:0/364093600 >> 172.21.15.45:0/364093600 conn(0x7fee440d5040 msgr2=0x7fee440ca2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:46.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.577+0000 7fee4aca0640 1 -- 172.21.15.45:0/364093600 shutdown_connections 2024-05-08T05:41:46.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.577+0000 7fee4aca0640 1 -- 172.21.15.45:0/364093600 wait complete. 2024-05-08T05:41:46.589 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-05-08T05:41:44.836570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":308,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236012,"kb_used_data":3176,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738708,"statfs":{"total":773094113280,"available":772852436992,"internally_reserved":0,"allocated":3252224,"data_stored":1983114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.701500"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575491+0000","last_change":"2024-05-08T05:41:31.127376+0000","last_active":"2024-05-08T05:41:37.575491+0000","last_peered":"2024-05-08T05:41:37.575491+0000","last_clean":"2024-05-08T05:41:37.575491+0000","last_became_active":"2024-05-08T05:41:31.127083+0000","last_became_peered":"2024-05-08T05:41:31.127083+0000","last_unstale":"2024-05-08T05:41:37.575491+0000","last_undegraded":"2024-05-08T05:41:37.575491+0000","last_fullsized":"2024-05-08T05:41:37.575491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:53.557101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576109+0000","last_change":"2024-05-08T05:41:32.133658+0000","last_active":"2024-05-08T05:41:37.576109+0000","last_peered":"2024-05-08T05:41:37.576109+0000","last_clean":"2024-05-08T05:41:37.576109+0000","last_became_active":"2024-05-08T05:41:32.133241+0000","last_became_peered":"2024-05-08T05:41:32.133241+0000","last_unstale":"2024-05-08T05:41:37.576109+0000","last_undegraded":"2024-05-08T05:41:37.576109+0000","last_fullsized":"2024-05-08T05:41:37.576109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:07:26.071605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576413+0000","last_change":"2024-05-08T05:41:36.385551+0000","last_active":"2024-05-08T05:41:37.576413+0000","last_peered":"2024-05-08T05:41:37.576413+0000","last_clean":"2024-05-08T05:41:37.576413+0000","last_became_active":"2024-05-08T05:41:36.385135+0000","last_became_peered":"2024-05-08T05:41:36.385135+0000","last_unstale":"2024-05-08T05:41:37.576413+0000","last_undegraded":"2024-05-08T05:41:37.576413+0000","last_fullsized":"2024-05-08T05:41:37.576413+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:39:51.537856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576344+0000","last_change":"2024-05-08T05:41:31.125574+0000","last_active":"2024-05-08T05:41:37.576344+0000","last_peered":"2024-05-08T05:41:37.576344+0000","last_clean":"2024-05-08T05:41:37.576344+0000","last_became_active":"2024-05-08T05:41:31.125302+0000","last_became_peered":"2024-05-08T05:41:31.125302+0000","last_unstale":"2024-05-08T05:41:37.576344+0000","last_undegraded":"2024-05-08T05:41:37.576344+0000","last_fullsized":"2024-05-08T05:41:37.576344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:39.672951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575165+0000","last_change":"2024-05-08T05:41:32.140528+0000","last_active":"2024-05-08T05:41:37.575165+0000","last_peered":"2024-05-08T05:41:37.575165+0000","last_clean":"2024-05-08T05:41:37.575165+0000","last_became_active":"2024-05-08T05:41:32.140227+0000","last_became_peered":"2024-05-08T05:41:32.140227+0000","last_unstale":"2024-05-08T05:41:37.575165+0000","last_undegraded":"2024-05-08T05:41:37.575165+0000","last_fullsized":"2024-05-08T05:41:37.575165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:30:07.184101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153167+0000","last_change":"2024-05-08T05:41:36.149551+0000","last_active":"2024-05-08T05:41:37.153167+0000","last_peered":"2024-05-08T05:41:37.153167+0000","last_clean":"2024-05-08T05:41:37.153167+0000","last_became_active":"2024-05-08T05:41:36.149115+0000","last_became_peered":"2024-05-08T05:41:36.149115+0000","last_unstale":"2024-05-08T05:41:37.153167+0000","last_undegraded":"2024-05-08T05:41:37.153167+0000","last_fullsized":"2024-05-08T05:41:37.153167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:57.593969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577133+0000","last_change":"2024-05-08T05:41:31.128105+0000","last_active":"2024-05-08T05:41:37.577133+0000","last_peered":"2024-05-08T05:41:37.577133+0000","last_clean":"2024-05-08T05:41:37.577133+0000","last_became_active":"2024-05-08T05:41:31.127782+0000","last_became_peered":"2024-05-08T05:41:31.127782+0000","last_unstale":"2024-05-08T05:41:37.577133+0000","last_undegraded":"2024-05-08T05:41:37.577133+0000","last_fullsized":"2024-05-08T05:41:37.577133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:46.418806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576628+0000","last_change":"2024-05-08T05:41:32.140667+0000","last_active":"2024-05-08T05:41:37.576628+0000","last_peered":"2024-05-08T05:41:37.576628+0000","last_clean":"2024-05-08T05:41:37.576628+0000","last_became_active":"2024-05-08T05:41:32.140365+0000","last_became_peered":"2024-05-08T05:41:32.140365+0000","last_unstale":"2024-05-08T05:41:37.576628+0000","last_undegraded":"2024-05-08T05:41:37.576628+0000","last_fullsized":"2024-05-08T05:41:37.576628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:02.517245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153785+0000","last_change":"2024-05-08T05:41:36.157609+0000","last_active":"2024-05-08T05:41:37.153785+0000","last_peered":"2024-05-08T05:41:37.153785+0000","last_clean":"2024-05-08T05:41:37.153785+0000","last_became_active":"2024-05-08T05:41:36.157214+0000","last_became_peered":"2024-05-08T05:41:36.157214+0000","last_unstale":"2024-05-08T05:41:37.153785+0000","last_undegraded":"2024-05-08T05:41:37.153785+0000","last_fullsized":"2024-05-08T05:41:37.153785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:46.769826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576228+0000","last_change":"2024-05-08T05:41:31.131582+0000","last_active":"2024-05-08T05:41:37.576228+0000","last_peered":"2024-05-08T05:41:37.576228+0000","last_clean":"2024-05-08T05:41:37.576228+0000","last_became_active":"2024-05-08T05:41:31.131302+0000","last_became_peered":"2024-05-08T05:41:31.131302+0000","last_unstale":"2024-05-08T05:41:37.576228+0000","last_undegraded":"2024-05-08T05:41:37.576228+0000","last_fullsized":"2024-05-08T05:41:37.576228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:07.648895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577648+0000","last_change":"2024-05-08T05:41:32.136792+0000","last_active":"2024-05-08T05:41:37.577648+0000","last_peered":"2024-05-08T05:41:37.577648+0000","last_clean":"2024-05-08T05:41:37.577648+0000","last_became_active":"2024-05-08T05:41:32.136285+0000","last_became_peered":"2024-05-08T05:41:32.136285+0000","last_unstale":"2024-05-08T05:41:37.577648+0000","last_undegraded":"2024-05-08T05:41:37.577648+0000","last_fullsized":"2024-05-08T05:41:37.577648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:53.826185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153415+0000","last_change":"2024-05-08T05:41:36.151839+0000","last_active":"2024-05-08T05:41:37.153415+0000","last_peered":"2024-05-08T05:41:37.153415+0000","last_clean":"2024-05-08T05:41:37.153415+0000","last_became_active":"2024-05-08T05:41:36.151387+0000","last_became_peered":"2024-05-08T05:41:36.151387+0000","last_unstale":"2024-05-08T05:41:37.153415+0000","last_undegraded":"2024-05-08T05:41:37.153415+0000","last_fullsized":"2024-05-08T05:41:37.153415+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:01:36.105428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576169+0000","last_change":"2024-05-08T05:41:32.136438+0000","last_active":"2024-05-08T05:41:37.576169+0000","last_peered":"2024-05-08T05:41:37.576169+0000","last_clean":"2024-05-08T05:41:37.576169+0000","last_became_active":"2024-05-08T05:41:32.136145+0000","last_became_peered":"2024-05-08T05:41:32.136145+0000","last_unstale":"2024-05-08T05:41:37.576169+0000","last_undegraded":"2024-05-08T05:41:37.576169+0000","last_fullsized":"2024-05-08T05:41:37.576169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:47.493805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576520+0000","last_change":"2024-05-08T05:41:31.124904+0000","last_active":"2024-05-08T05:41:37.576520+0000","last_peered":"2024-05-08T05:41:37.576520+0000","last_clean":"2024-05-08T05:41:37.576520+0000","last_became_active":"2024-05-08T05:41:31.124621+0000","last_became_peered":"2024-05-08T05:41:31.124621+0000","last_unstale":"2024-05-08T05:41:37.576520+0000","last_undegraded":"2024-05-08T05:41:37.576520+0000","last_fullsized":"2024-05-08T05:41:37.576520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:41.591349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154482+0000","last_change":"2024-05-08T05:41:36.154069+0000","last_active":"2024-05-08T05:41:37.154482+0000","last_peered":"2024-05-08T05:41:37.154482+0000","last_clean":"2024-05-08T05:41:37.154482+0000","last_became_active":"2024-05-08T05:41:36.153717+0000","last_became_peered":"2024-05-08T05:41:36.153717+0000","last_unstale":"2024-05-08T05:41:37.154482+0000","last_undegraded":"2024-05-08T05:41:37.154482+0000","last_fullsized":"2024-05-08T05:41:37.154482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:57.856276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577086+0000","last_change":"2024-05-08T05:41:32.137987+0000","last_active":"2024-05-08T05:41:37.577086+0000","last_peered":"2024-05-08T05:41:37.577086+0000","last_clean":"2024-05-08T05:41:37.577086+0000","last_became_active":"2024-05-08T05:41:32.137711+0000","last_became_peered":"2024-05-08T05:41:32.137711+0000","last_unstale":"2024-05-08T05:41:37.577086+0000","last_undegraded":"2024-05-08T05:41:37.577086+0000","last_fullsized":"2024-05-08T05:41:37.577086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:01:32.075337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576626+0000","last_change":"2024-05-08T05:41:31.121790+0000","last_active":"2024-05-08T05:41:37.576626+0000","last_peered":"2024-05-08T05:41:37.576626+0000","last_clean":"2024-05-08T05:41:37.576626+0000","last_became_active":"2024-05-08T05:41:31.121476+0000","last_became_peered":"2024-05-08T05:41:31.121476+0000","last_unstale":"2024-05-08T05:41:37.576626+0000","last_undegraded":"2024-05-08T05:41:37.576626+0000","last_fullsized":"2024-05-08T05:41:37.576626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:11:56.288788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154013+0000","last_change":"2024-05-08T05:41:36.157615+0000","last_active":"2024-05-08T05:41:37.154013+0000","last_peered":"2024-05-08T05:41:37.154013+0000","last_clean":"2024-05-08T05:41:37.154013+0000","last_became_active":"2024-05-08T05:41:36.157156+0000","last_became_peered":"2024-05-08T05:41:36.157156+0000","last_unstale":"2024-05-08T05:41:37.154013+0000","last_undegraded":"2024-05-08T05:41:37.154013+0000","last_fullsized":"2024-05-08T05:41:37.154013+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:46.115781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576120+0000","last_change":"2024-05-08T05:41:32.119867+0000","last_active":"2024-05-08T05:41:37.576120+0000","last_peered":"2024-05-08T05:41:37.576120+0000","last_clean":"2024-05-08T05:41:37.576120+0000","last_became_active":"2024-05-08T05:41:32.119186+0000","last_became_peered":"2024-05-08T05:41:32.119186+0000","last_unstale":"2024-05-08T05:41:37.576120+0000","last_undegraded":"2024-05-08T05:41:37.576120+0000","last_fullsized":"2024-05-08T05:41:37.576120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:47.493805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575164+0000","last_change":"2024-05-08T05:41:31.128486+0000","last_active":"2024-05-08T05:41:37.575164+0000","last_peered":"2024-05-08T05:41:37.575164+0000","last_clean":"2024-05-08T05:41:37.575164+0000","last_became_active":"2024-05-08T05:41:31.128207+0000","last_became_peered":"2024-05-08T05:41:31.128207+0000","last_unstale":"2024-05-08T05:41:37.575164+0000","last_undegraded":"2024-05-08T05:41:37.575164+0000","last_fullsized":"2024-05-08T05:41:37.575164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:46.418806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576817+0000","last_change":"2024-05-08T05:41:36.381823+0000","last_active":"2024-05-08T05:41:37.576817+0000","last_peered":"2024-05-08T05:41:37.576817+0000","last_clean":"2024-05-08T05:41:37.576817+0000","last_became_active":"2024-05-08T05:41:36.381421+0000","last_became_peered":"2024-05-08T05:41:36.381421+0000","last_unstale":"2024-05-08T05:41:37.576817+0000","last_undegraded":"2024-05-08T05:41:37.576817+0000","last_fullsized":"2024-05-08T05:41:37.576817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:21.321405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153397+0000","last_change":"2024-05-08T05:41:32.117529+0000","last_active":"2024-05-08T05:41:37.153397+0000","last_peered":"2024-05-08T05:41:37.153397+0000","last_clean":"2024-05-08T05:41:37.153397+0000","last_became_active":"2024-05-08T05:41:32.117205+0000","last_became_peered":"2024-05-08T05:41:32.117205+0000","last_unstale":"2024-05-08T05:41:37.153397+0000","last_undegraded":"2024-05-08T05:41:37.153397+0000","last_fullsized":"2024-05-08T05:41:37.153397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:08.654224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575429+0000","last_change":"2024-05-08T05:41:31.126184+0000","last_active":"2024-05-08T05:41:37.575429+0000","last_peered":"2024-05-08T05:41:37.575429+0000","last_clean":"2024-05-08T05:41:37.575429+0000","last_became_active":"2024-05-08T05:41:31.125860+0000","last_became_peered":"2024-05-08T05:41:31.125860+0000","last_unstale":"2024-05-08T05:41:37.575429+0000","last_undegraded":"2024-05-08T05:41:37.575429+0000","last_fullsized":"2024-05-08T05:41:37.575429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:13.465598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154197+0000","last_change":"2024-05-08T05:41:36.151020+0000","last_active":"2024-05-08T05:41:37.154197+0000","last_peered":"2024-05-08T05:41:37.154197+0000","last_clean":"2024-05-08T05:41:37.154197+0000","last_became_active":"2024-05-08T05:41:36.150320+0000","last_became_peered":"2024-05-08T05:41:36.150320+0000","last_unstale":"2024-05-08T05:41:37.154197+0000","last_undegraded":"2024-05-08T05:41:37.154197+0000","last_fullsized":"2024-05-08T05:41:37.154197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:06.547337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576056+0000","last_change":"2024-05-08T05:41:32.121088+0000","last_active":"2024-05-08T05:41:37.576056+0000","last_peered":"2024-05-08T05:41:37.576056+0000","last_clean":"2024-05-08T05:41:37.576056+0000","last_became_active":"2024-05-08T05:41:32.119585+0000","last_became_peered":"2024-05-08T05:41:32.119585+0000","last_unstale":"2024-05-08T05:41:37.576056+0000","last_undegraded":"2024-05-08T05:41:37.576056+0000","last_fullsized":"2024-05-08T05:41:37.576056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:18:52.796967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577379+0000","last_change":"2024-05-08T05:41:31.129202+0000","last_active":"2024-05-08T05:41:37.577379+0000","last_peered":"2024-05-08T05:41:37.577379+0000","last_clean":"2024-05-08T05:41:37.577379+0000","last_became_active":"2024-05-08T05:41:31.128911+0000","last_became_peered":"2024-05-08T05:41:31.128911+0000","last_unstale":"2024-05-08T05:41:37.577379+0000","last_undegraded":"2024-05-08T05:41:37.577379+0000","last_fullsized":"2024-05-08T05:41:37.577379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:13.465598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153283+0000","last_change":"2024-05-08T05:41:36.384181+0000","last_active":"2024-05-08T05:41:37.153283+0000","last_peered":"2024-05-08T05:41:37.153283+0000","last_clean":"2024-05-08T05:41:37.153283+0000","last_became_active":"2024-05-08T05:41:36.383800+0000","last_became_peered":"2024-05-08T05:41:36.383800+0000","last_unstale":"2024-05-08T05:41:37.153283+0000","last_undegraded":"2024-05-08T05:41:37.153283+0000","last_fullsized":"2024-05-08T05:41:37.153283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:33.503988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576885+0000","last_change":"2024-05-08T05:41:32.121597+0000","last_active":"2024-05-08T05:41:37.576885+0000","last_peered":"2024-05-08T05:41:37.576885+0000","last_clean":"2024-05-08T05:41:37.576885+0000","last_became_active":"2024-05-08T05:41:32.121321+0000","last_became_peered":"2024-05-08T05:41:32.121321+0000","last_unstale":"2024-05-08T05:41:37.576885+0000","last_undegraded":"2024-05-08T05:41:37.576885+0000","last_fullsized":"2024-05-08T05:41:37.576885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:42.739734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575444+0000","last_change":"2024-05-08T05:41:31.126359+0000","last_active":"2024-05-08T05:41:37.575444+0000","last_peered":"2024-05-08T05:41:37.575444+0000","last_clean":"2024-05-08T05:41:37.575444+0000","last_became_active":"2024-05-08T05:41:31.126069+0000","last_became_peered":"2024-05-08T05:41:31.126069+0000","last_unstale":"2024-05-08T05:41:37.575444+0000","last_undegraded":"2024-05-08T05:41:37.575444+0000","last_fullsized":"2024-05-08T05:41:37.575444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:24.722618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575700+0000","last_change":"2024-05-08T05:41:36.381909+0000","last_active":"2024-05-08T05:41:37.575700+0000","last_peered":"2024-05-08T05:41:37.575700+0000","last_clean":"2024-05-08T05:41:37.575700+0000","last_became_active":"2024-05-08T05:41:36.381251+0000","last_became_peered":"2024-05-08T05:41:36.381251+0000","last_unstale":"2024-05-08T05:41:37.575700+0000","last_undegraded":"2024-05-08T05:41:37.575700+0000","last_fullsized":"2024-05-08T05:41:37.575700+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:06:13.980953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576829+0000","last_change":"2024-05-08T05:41:32.133545+0000","last_active":"2024-05-08T05:41:37.576829+0000","last_peered":"2024-05-08T05:41:37.576829+0000","last_clean":"2024-05-08T05:41:37.576829+0000","last_became_active":"2024-05-08T05:41:32.132677+0000","last_became_peered":"2024-05-08T05:41:32.132677+0000","last_unstale":"2024-05-08T05:41:37.576829+0000","last_undegraded":"2024-05-08T05:41:37.576829+0000","last_fullsized":"2024-05-08T05:41:37.576829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:42.041915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577263+0000","last_change":"2024-05-08T05:41:31.127131+0000","last_active":"2024-05-08T05:41:37.577263+0000","last_peered":"2024-05-08T05:41:37.577263+0000","last_clean":"2024-05-08T05:41:37.577263+0000","last_became_active":"2024-05-08T05:41:31.126789+0000","last_became_peered":"2024-05-08T05:41:31.126789+0000","last_unstale":"2024-05-08T05:41:37.577263+0000","last_undegraded":"2024-05-08T05:41:37.577263+0000","last_fullsized":"2024-05-08T05:41:37.577263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:19:12.221462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575923+0000","last_change":"2024-05-08T05:41:36.381721+0000","last_active":"2024-05-08T05:41:37.575923+0000","last_peered":"2024-05-08T05:41:37.575923+0000","last_clean":"2024-05-08T05:41:37.575923+0000","last_became_active":"2024-05-08T05:41:36.381329+0000","last_became_peered":"2024-05-08T05:41:36.381329+0000","last_unstale":"2024-05-08T05:41:37.575923+0000","last_undegraded":"2024-05-08T05:41:37.575923+0000","last_fullsized":"2024-05-08T05:41:37.575923+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:17.202706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577782+0000","last_change":"2024-05-08T05:41:32.136558+0000","last_active":"2024-05-08T05:41:37.577782+0000","last_peered":"2024-05-08T05:41:37.577782+0000","last_clean":"2024-05-08T05:41:37.577782+0000","last_became_active":"2024-05-08T05:41:32.133354+0000","last_became_peered":"2024-05-08T05:41:32.133354+0000","last_unstale":"2024-05-08T05:41:37.577782+0000","last_undegraded":"2024-05-08T05:41:37.577782+0000","last_fullsized":"2024-05-08T05:41:37.577782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:13.172615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576762+0000","last_change":"2024-05-08T05:41:31.125582+0000","last_active":"2024-05-08T05:41:37.576762+0000","last_peered":"2024-05-08T05:41:37.576762+0000","last_clean":"2024-05-08T05:41:37.576762+0000","last_became_active":"2024-05-08T05:41:31.125252+0000","last_became_peered":"2024-05-08T05:41:31.125252+0000","last_unstale":"2024-05-08T05:41:37.576762+0000","last_undegraded":"2024-05-08T05:41:37.576762+0000","last_fullsized":"2024-05-08T05:41:37.576762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:38.174964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576435+0000","last_change":"2024-05-08T05:41:36.154344+0000","last_active":"2024-05-08T05:41:37.576435+0000","last_peered":"2024-05-08T05:41:37.576435+0000","last_clean":"2024-05-08T05:41:37.576435+0000","last_became_active":"2024-05-08T05:41:36.153930+0000","last_became_peered":"2024-05-08T05:41:36.153930+0000","last_unstale":"2024-05-08T05:41:37.576435+0000","last_undegraded":"2024-05-08T05:41:37.576435+0000","last_fullsized":"2024-05-08T05:41:37.576435+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:20.531197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154018+0000","last_change":"2024-05-08T05:41:32.136504+0000","last_active":"2024-05-08T05:41:37.154018+0000","last_peered":"2024-05-08T05:41:37.154018+0000","last_clean":"2024-05-08T05:41:37.154018+0000","last_became_active":"2024-05-08T05:41:32.135721+0000","last_became_peered":"2024-05-08T05:41:32.135721+0000","last_unstale":"2024-05-08T05:41:37.154018+0000","last_undegraded":"2024-05-08T05:41:37.154018+0000","last_fullsized":"2024-05-08T05:41:37.154018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:42.596678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576189+0000","last_change":"2024-05-08T05:41:31.131574+0000","last_active":"2024-05-08T05:41:37.576189+0000","last_peered":"2024-05-08T05:41:37.576189+0000","last_clean":"2024-05-08T05:41:37.576189+0000","last_became_active":"2024-05-08T05:41:31.131158+0000","last_became_peered":"2024-05-08T05:41:31.131158+0000","last_unstale":"2024-05-08T05:41:37.576189+0000","last_undegraded":"2024-05-08T05:41:37.576189+0000","last_fullsized":"2024-05-08T05:41:37.576189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:13.465598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576707+0000","last_change":"2024-05-08T05:41:36.159573+0000","last_active":"2024-05-08T05:41:37.576707+0000","last_peered":"2024-05-08T05:41:37.576707+0000","last_clean":"2024-05-08T05:41:37.576707+0000","last_became_active":"2024-05-08T05:41:36.159150+0000","last_became_peered":"2024-05-08T05:41:36.159150+0000","last_unstale":"2024-05-08T05:41:37.576707+0000","last_undegraded":"2024-05-08T05:41:37.576707+0000","last_fullsized":"2024-05-08T05:41:37.576707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:17.202706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576057+0000","last_change":"2024-05-08T05:41:32.121923+0000","last_active":"2024-05-08T05:41:37.576057+0000","last_peered":"2024-05-08T05:41:37.576057+0000","last_clean":"2024-05-08T05:41:37.576057+0000","last_became_active":"2024-05-08T05:41:32.121213+0000","last_became_peered":"2024-05-08T05:41:32.121213+0000","last_unstale":"2024-05-08T05:41:37.576057+0000","last_undegraded":"2024-05-08T05:41:37.576057+0000","last_fullsized":"2024-05-08T05:41:37.576057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:03:20.127059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.147846+0000","last_change":"2024-05-08T05:41:31.122162+0000","last_active":"2024-05-08T05:41:37.147846+0000","last_peered":"2024-05-08T05:41:37.147846+0000","last_clean":"2024-05-08T05:41:37.147846+0000","last_became_active":"2024-05-08T05:41:31.121865+0000","last_became_peered":"2024-05-08T05:41:31.121865+0000","last_unstale":"2024-05-08T05:41:37.147846+0000","last_undegraded":"2024-05-08T05:41:37.147846+0000","last_fullsized":"2024-05-08T05:41:37.147846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:24.722618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576949+0000","last_change":"2024-05-08T05:41:36.155633+0000","last_active":"2024-05-08T05:41:37.576949+0000","last_peered":"2024-05-08T05:41:37.576949+0000","last_clean":"2024-05-08T05:41:37.576949+0000","last_became_active":"2024-05-08T05:41:36.155125+0000","last_became_peered":"2024-05-08T05:41:36.155125+0000","last_unstale":"2024-05-08T05:41:37.576949+0000","last_undegraded":"2024-05-08T05:41:37.576949+0000","last_fullsized":"2024-05-08T05:41:37.576949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:50:36.282661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.574963+0000","last_change":"2024-05-08T05:41:32.119953+0000","last_active":"2024-05-08T05:41:37.574963+0000","last_peered":"2024-05-08T05:41:37.574963+0000","last_clean":"2024-05-08T05:41:37.574963+0000","last_became_active":"2024-05-08T05:41:32.119532+0000","last_became_peered":"2024-05-08T05:41:32.119532+0000","last_unstale":"2024-05-08T05:41:37.574963+0000","last_undegraded":"2024-05-08T05:41:37.574963+0000","last_fullsized":"2024-05-08T05:41:37.574963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:16:33.854897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575557+0000","last_change":"2024-05-08T05:41:31.125004+0000","last_active":"2024-05-08T05:41:37.575557+0000","last_peered":"2024-05-08T05:41:37.575557+0000","last_clean":"2024-05-08T05:41:37.575557+0000","last_became_active":"2024-05-08T05:41:31.124678+0000","last_became_peered":"2024-05-08T05:41:31.124678+0000","last_unstale":"2024-05-08T05:41:37.575557+0000","last_undegraded":"2024-05-08T05:41:37.575557+0000","last_fullsized":"2024-05-08T05:41:37.575557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:38.174964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575882+0000","last_change":"2024-05-08T05:41:36.385360+0000","last_active":"2024-05-08T05:41:37.575882+0000","last_peered":"2024-05-08T05:41:37.575882+0000","last_clean":"2024-05-08T05:41:37.575882+0000","last_became_active":"2024-05-08T05:41:36.384992+0000","last_became_peered":"2024-05-08T05:41:36.384992+0000","last_unstale":"2024-05-08T05:41:37.575882+0000","last_undegraded":"2024-05-08T05:41:37.575882+0000","last_fullsized":"2024-05-08T05:41:37.575882+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:20.531197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575341+0000","last_change":"2024-05-08T05:41:32.138842+0000","last_active":"2024-05-08T05:41:37.575341+0000","last_peered":"2024-05-08T05:41:37.575341+0000","last_clean":"2024-05-08T05:41:37.575341+0000","last_became_active":"2024-05-08T05:41:32.138583+0000","last_became_peered":"2024-05-08T05:41:32.138583+0000","last_unstale":"2024-05-08T05:41:37.575341+0000","last_undegraded":"2024-05-08T05:41:37.575341+0000","last_fullsized":"2024-05-08T05:41:37.575341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:53.563878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576353+0000","last_change":"2024-05-08T05:41:31.132616+0000","last_active":"2024-05-08T05:41:37.576353+0000","last_peered":"2024-05-08T05:41:37.576353+0000","last_clean":"2024-05-08T05:41:37.576353+0000","last_became_active":"2024-05-08T05:41:31.131462+0000","last_became_peered":"2024-05-08T05:41:31.131462+0000","last_unstale":"2024-05-08T05:41:37.576353+0000","last_undegraded":"2024-05-08T05:41:37.576353+0000","last_fullsized":"2024-05-08T05:41:37.576353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:24.722618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":118,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575672+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:41:37.575672+0000","last_peered":"2024-05-08T05:41:37.575672+0000","last_clean":"2024-05-08T05:41:37.575672+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:41:37.575672+0000","last_undegraded":"2024-05-08T05:41:37.575672+0000","last_fullsized":"2024-05-08T05:41:37.575672+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576785+0000","last_change":"2024-05-08T05:41:36.381947+0000","last_active":"2024-05-08T05:41:37.576785+0000","last_peered":"2024-05-08T05:41:37.576785+0000","last_clean":"2024-05-08T05:41:37.576785+0000","last_became_active":"2024-05-08T05:41:36.381491+0000","last_became_peered":"2024-05-08T05:41:36.381491+0000","last_unstale":"2024-05-08T05:41:37.576785+0000","last_undegraded":"2024-05-08T05:41:37.576785+0000","last_fullsized":"2024-05-08T05:41:37.576785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:57.856276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576983+0000","last_change":"2024-05-08T05:41:32.134397+0000","last_active":"2024-05-08T05:41:37.576983+0000","last_peered":"2024-05-08T05:41:37.576983+0000","last_clean":"2024-05-08T05:41:37.576983+0000","last_became_active":"2024-05-08T05:41:32.134098+0000","last_became_peered":"2024-05-08T05:41:32.134098+0000","last_unstale":"2024-05-08T05:41:37.576983+0000","last_undegraded":"2024-05-08T05:41:37.576983+0000","last_fullsized":"2024-05-08T05:41:37.576983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:16.501105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.148234+0000","last_change":"2024-05-08T05:41:31.133531+0000","last_active":"2024-05-08T05:41:37.148234+0000","last_peered":"2024-05-08T05:41:37.148234+0000","last_clean":"2024-05-08T05:41:37.148234+0000","last_became_active":"2024-05-08T05:41:31.133208+0000","last_became_peered":"2024-05-08T05:41:31.133208+0000","last_unstale":"2024-05-08T05:41:37.148234+0000","last_undegraded":"2024-05-08T05:41:37.148234+0000","last_fullsized":"2024-05-08T05:41:37.148234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:22.751500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576104+0000","last_change":"2024-05-08T05:41:36.381903+0000","last_active":"2024-05-08T05:41:37.576104+0000","last_peered":"2024-05-08T05:41:37.576104+0000","last_clean":"2024-05-08T05:41:37.576104+0000","last_became_active":"2024-05-08T05:41:36.381397+0000","last_became_peered":"2024-05-08T05:41:36.381397+0000","last_unstale":"2024-05-08T05:41:37.576104+0000","last_undegraded":"2024-05-08T05:41:37.576104+0000","last_fullsized":"2024-05-08T05:41:37.576104+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:06.547337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576725+0000","last_change":"2024-05-08T05:41:32.128554+0000","last_active":"2024-05-08T05:41:37.576725+0000","last_peered":"2024-05-08T05:41:37.576725+0000","last_clean":"2024-05-08T05:41:37.576725+0000","last_became_active":"2024-05-08T05:41:32.128280+0000","last_became_peered":"2024-05-08T05:41:32.128280+0000","last_unstale":"2024-05-08T05:41:37.576725+0000","last_undegraded":"2024-05-08T05:41:37.576725+0000","last_fullsized":"2024-05-08T05:41:37.576725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:42.085689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576546+0000","last_change":"2024-05-08T05:41:31.122071+0000","last_active":"2024-05-08T05:41:37.576546+0000","last_peered":"2024-05-08T05:41:37.576546+0000","last_clean":"2024-05-08T05:41:37.576546+0000","last_became_active":"2024-05-08T05:41:31.121137+0000","last_became_peered":"2024-05-08T05:41:31.121137+0000","last_unstale":"2024-05-08T05:41:37.576546+0000","last_undegraded":"2024-05-08T05:41:37.576546+0000","last_fullsized":"2024-05-08T05:41:37.576546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:13.465598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576176+0000","last_change":"2024-05-08T05:41:36.160917+0000","last_active":"2024-05-08T05:41:37.576176+0000","last_peered":"2024-05-08T05:41:37.576176+0000","last_clean":"2024-05-08T05:41:37.576176+0000","last_became_active":"2024-05-08T05:41:36.160457+0000","last_became_peered":"2024-05-08T05:41:36.160457+0000","last_unstale":"2024-05-08T05:41:37.576176+0000","last_undegraded":"2024-05-08T05:41:37.576176+0000","last_fullsized":"2024-05-08T05:41:37.576176+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:21.321405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577384+0000","last_change":"2024-05-08T05:41:32.131012+0000","last_active":"2024-05-08T05:41:37.577384+0000","last_peered":"2024-05-08T05:41:37.577384+0000","last_clean":"2024-05-08T05:41:37.577384+0000","last_became_active":"2024-05-08T05:41:32.130682+0000","last_became_peered":"2024-05-08T05:41:32.130682+0000","last_unstale":"2024-05-08T05:41:37.577384+0000","last_undegraded":"2024-05-08T05:41:37.577384+0000","last_fullsized":"2024-05-08T05:41:37.577384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:06:09.950861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.148282+0000","last_change":"2024-05-08T05:41:31.122329+0000","last_active":"2024-05-08T05:41:37.148282+0000","last_peered":"2024-05-08T05:41:37.148282+0000","last_clean":"2024-05-08T05:41:37.148282+0000","last_became_active":"2024-05-08T05:41:31.122021+0000","last_became_peered":"2024-05-08T05:41:31.122021+0000","last_unstale":"2024-05-08T05:41:37.148282+0000","last_undegraded":"2024-05-08T05:41:37.148282+0000","last_fullsized":"2024-05-08T05:41:37.148282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:11:56.288788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575492+0000","last_change":"2024-05-08T05:41:36.382122+0000","last_active":"2024-05-08T05:41:37.575492+0000","last_peered":"2024-05-08T05:41:37.575492+0000","last_clean":"2024-05-08T05:41:37.575492+0000","last_became_active":"2024-05-08T05:41:36.381385+0000","last_became_peered":"2024-05-08T05:41:36.381385+0000","last_unstale":"2024-05-08T05:41:37.575492+0000","last_undegraded":"2024-05-08T05:41:37.575492+0000","last_fullsized":"2024-05-08T05:41:37.575492+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:02.479497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576083+0000","last_change":"2024-05-08T05:41:32.131255+0000","last_active":"2024-05-08T05:41:37.576083+0000","last_peered":"2024-05-08T05:41:37.576083+0000","last_clean":"2024-05-08T05:41:37.576083+0000","last_became_active":"2024-05-08T05:41:32.130252+0000","last_became_peered":"2024-05-08T05:41:32.130252+0000","last_unstale":"2024-05-08T05:41:37.576083+0000","last_undegraded":"2024-05-08T05:41:37.576083+0000","last_fullsized":"2024-05-08T05:41:37.576083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:17.291314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154349+0000","last_change":"2024-05-08T05:41:31.125770+0000","last_active":"2024-05-08T05:41:37.154349+0000","last_peered":"2024-05-08T05:41:37.154349+0000","last_clean":"2024-05-08T05:41:37.154349+0000","last_became_active":"2024-05-08T05:41:31.125442+0000","last_became_peered":"2024-05-08T05:41:31.125442+0000","last_unstale":"2024-05-08T05:41:37.154349+0000","last_undegraded":"2024-05-08T05:41:37.154349+0000","last_fullsized":"2024-05-08T05:41:37.154349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:53.557101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153167+0000","last_change":"2024-05-08T05:41:36.151074+0000","last_active":"2024-05-08T05:41:37.153167+0000","last_peered":"2024-05-08T05:41:37.153167+0000","last_clean":"2024-05-08T05:41:37.153167+0000","last_became_active":"2024-05-08T05:41:36.150613+0000","last_became_peered":"2024-05-08T05:41:36.150613+0000","last_unstale":"2024-05-08T05:41:37.153167+0000","last_undegraded":"2024-05-08T05:41:37.153167+0000","last_fullsized":"2024-05-08T05:41:37.153167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:46.072007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576628+0000","last_change":"2024-05-08T05:41:32.132197+0000","last_active":"2024-05-08T05:41:37.576628+0000","last_peered":"2024-05-08T05:41:37.576628+0000","last_clean":"2024-05-08T05:41:37.576628+0000","last_became_active":"2024-05-08T05:41:32.131086+0000","last_became_peered":"2024-05-08T05:41:32.131086+0000","last_unstale":"2024-05-08T05:41:37.576628+0000","last_undegraded":"2024-05-08T05:41:37.576628+0000","last_fullsized":"2024-05-08T05:41:37.576628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:02.517245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.147676+0000","last_change":"2024-05-08T05:41:31.120099+0000","last_active":"2024-05-08T05:41:37.147676+0000","last_peered":"2024-05-08T05:41:37.147676+0000","last_clean":"2024-05-08T05:41:37.147676+0000","last_became_active":"2024-05-08T05:41:31.119750+0000","last_became_peered":"2024-05-08T05:41:31.119750+0000","last_unstale":"2024-05-08T05:41:37.147676+0000","last_undegraded":"2024-05-08T05:41:37.147676+0000","last_fullsized":"2024-05-08T05:41:37.147676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:49.171947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575735+0000","last_change":"2024-05-08T05:41:36.162020+0000","last_active":"2024-05-08T05:41:37.575735+0000","last_peered":"2024-05-08T05:41:37.575735+0000","last_clean":"2024-05-08T05:41:37.575735+0000","last_became_active":"2024-05-08T05:41:36.161613+0000","last_became_peered":"2024-05-08T05:41:36.161613+0000","last_unstale":"2024-05-08T05:41:37.575735+0000","last_undegraded":"2024-05-08T05:41:37.575735+0000","last_fullsized":"2024-05-08T05:41:37.575735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:57.856276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575424+0000","last_change":"2024-05-08T05:41:32.137565+0000","last_active":"2024-05-08T05:41:37.575424+0000","last_peered":"2024-05-08T05:41:37.575424+0000","last_clean":"2024-05-08T05:41:37.575424+0000","last_became_active":"2024-05-08T05:41:32.137078+0000","last_became_peered":"2024-05-08T05:41:32.137078+0000","last_unstale":"2024-05-08T05:41:37.575424+0000","last_undegraded":"2024-05-08T05:41:37.575424+0000","last_fullsized":"2024-05-08T05:41:37.575424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:44.038894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576417+0000","last_change":"2024-05-08T05:41:31.127117+0000","last_active":"2024-05-08T05:41:37.576417+0000","last_peered":"2024-05-08T05:41:37.576417+0000","last_clean":"2024-05-08T05:41:37.576417+0000","last_became_active":"2024-05-08T05:41:31.126826+0000","last_became_peered":"2024-05-08T05:41:31.126826+0000","last_unstale":"2024-05-08T05:41:37.576417+0000","last_undegraded":"2024-05-08T05:41:37.576417+0000","last_fullsized":"2024-05-08T05:41:37.576417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:22.751500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576723+0000","last_change":"2024-05-08T05:41:36.383650+0000","last_active":"2024-05-08T05:41:37.576723+0000","last_peered":"2024-05-08T05:41:37.576723+0000","last_clean":"2024-05-08T05:41:37.576723+0000","last_became_active":"2024-05-08T05:41:36.382759+0000","last_became_peered":"2024-05-08T05:41:36.382759+0000","last_unstale":"2024-05-08T05:41:37.576723+0000","last_undegraded":"2024-05-08T05:41:37.576723+0000","last_fullsized":"2024-05-08T05:41:37.576723+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:43:10.562469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576336+0000","last_change":"2024-05-08T05:41:32.130461+0000","last_active":"2024-05-08T05:41:37.576336+0000","last_peered":"2024-05-08T05:41:37.576336+0000","last_clean":"2024-05-08T05:41:37.576336+0000","last_became_active":"2024-05-08T05:41:32.130160+0000","last_became_peered":"2024-05-08T05:41:32.130160+0000","last_unstale":"2024-05-08T05:41:37.576336+0000","last_undegraded":"2024-05-08T05:41:37.576336+0000","last_fullsized":"2024-05-08T05:41:37.576336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:42.085689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576077+0000","last_change":"2024-05-08T05:41:31.132478+0000","last_active":"2024-05-08T05:41:37.576077+0000","last_peered":"2024-05-08T05:41:37.576077+0000","last_clean":"2024-05-08T05:41:37.576077+0000","last_became_active":"2024-05-08T05:41:31.131093+0000","last_became_peered":"2024-05-08T05:41:31.131093+0000","last_unstale":"2024-05-08T05:41:37.576077+0000","last_undegraded":"2024-05-08T05:41:37.576077+0000","last_fullsized":"2024-05-08T05:41:37.576077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:07.648895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577230+0000","last_change":"2024-05-08T05:41:36.381669+0000","last_active":"2024-05-08T05:41:37.577230+0000","last_peered":"2024-05-08T05:41:37.577230+0000","last_clean":"2024-05-08T05:41:37.577230+0000","last_became_active":"2024-05-08T05:41:36.381233+0000","last_became_peered":"2024-05-08T05:41:36.381233+0000","last_unstale":"2024-05-08T05:41:37.577230+0000","last_undegraded":"2024-05-08T05:41:37.577230+0000","last_fullsized":"2024-05-08T05:41:37.577230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:07.864670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575368+0000","last_change":"2024-05-08T05:41:32.131789+0000","last_active":"2024-05-08T05:41:37.575368+0000","last_peered":"2024-05-08T05:41:37.575368+0000","last_clean":"2024-05-08T05:41:37.575368+0000","last_became_active":"2024-05-08T05:41:32.131515+0000","last_became_peered":"2024-05-08T05:41:32.131515+0000","last_unstale":"2024-05-08T05:41:37.575368+0000","last_undegraded":"2024-05-08T05:41:37.575368+0000","last_fullsized":"2024-05-08T05:41:37.575368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:03:20.127059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576417+0000","last_change":"2024-05-08T05:41:31.126864+0000","last_active":"2024-05-08T05:41:37.576417+0000","last_peered":"2024-05-08T05:41:37.576417+0000","last_clean":"2024-05-08T05:41:37.576417+0000","last_became_active":"2024-05-08T05:41:31.126568+0000","last_became_peered":"2024-05-08T05:41:31.126568+0000","last_unstale":"2024-05-08T05:41:37.576417+0000","last_undegraded":"2024-05-08T05:41:37.576417+0000","last_fullsized":"2024-05-08T05:41:37.576417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:39.672951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577692+0000","last_change":"2024-05-08T05:41:36.382511+0000","last_active":"2024-05-08T05:41:37.577692+0000","last_peered":"2024-05-08T05:41:37.577692+0000","last_clean":"2024-05-08T05:41:37.577692+0000","last_became_active":"2024-05-08T05:41:36.382110+0000","last_became_peered":"2024-05-08T05:41:36.382110+0000","last_unstale":"2024-05-08T05:41:37.577692+0000","last_undegraded":"2024-05-08T05:41:37.577692+0000","last_fullsized":"2024-05-08T05:41:37.577692+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:39:51.537856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576868+0000","last_change":"2024-05-08T05:41:32.127674+0000","last_active":"2024-05-08T05:41:37.576868+0000","last_peered":"2024-05-08T05:41:37.576868+0000","last_clean":"2024-05-08T05:41:37.576868+0000","last_became_active":"2024-05-08T05:41:32.127374+0000","last_became_peered":"2024-05-08T05:41:32.127374+0000","last_unstale":"2024-05-08T05:41:37.576868+0000","last_undegraded":"2024-05-08T05:41:37.576868+0000","last_fullsized":"2024-05-08T05:41:37.576868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:53.826185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":28,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577094+0000","last_change":"2024-05-08T05:41:31.126507+0000","last_active":"2024-05-08T05:41:37.577094+0000","last_peered":"2024-05-08T05:41:37.577094+0000","last_clean":"2024-05-08T05:41:37.577094+0000","last_became_active":"2024-05-08T05:41:31.126218+0000","last_became_peered":"2024-05-08T05:41:31.126218+0000","last_unstale":"2024-05-08T05:41:37.577094+0000","last_undegraded":"2024-05-08T05:41:37.577094+0000","last_fullsized":"2024-05-08T05:41:37.577094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:11:56.288788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576865+0000","last_change":"2024-05-08T05:41:36.153303+0000","last_active":"2024-05-08T05:41:37.576865+0000","last_peered":"2024-05-08T05:41:37.576865+0000","last_clean":"2024-05-08T05:41:37.576865+0000","last_became_active":"2024-05-08T05:41:36.152849+0000","last_became_peered":"2024-05-08T05:41:36.152849+0000","last_unstale":"2024-05-08T05:41:37.576865+0000","last_undegraded":"2024-05-08T05:41:37.576865+0000","last_fullsized":"2024-05-08T05:41:37.576865+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:07:30.112175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154093+0000","last_change":"2024-05-08T05:41:32.121425+0000","last_active":"2024-05-08T05:41:37.154093+0000","last_peered":"2024-05-08T05:41:37.154093+0000","last_clean":"2024-05-08T05:41:37.154093+0000","last_became_active":"2024-05-08T05:41:32.121121+0000","last_became_peered":"2024-05-08T05:41:32.121121+0000","last_unstale":"2024-05-08T05:41:37.154093+0000","last_undegraded":"2024-05-08T05:41:37.154093+0000","last_fullsized":"2024-05-08T05:41:37.154093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:47.424136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576434+0000","last_change":"2024-05-08T05:41:31.132128+0000","last_active":"2024-05-08T05:41:37.576434+0000","last_peered":"2024-05-08T05:41:37.576434+0000","last_clean":"2024-05-08T05:41:37.576434+0000","last_became_active":"2024-05-08T05:41:31.131776+0000","last_became_peered":"2024-05-08T05:41:31.131776+0000","last_unstale":"2024-05-08T05:41:37.576434+0000","last_undegraded":"2024-05-08T05:41:37.576434+0000","last_fullsized":"2024-05-08T05:41:37.576434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:07.648895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153781+0000","last_change":"2024-05-08T05:41:36.152157+0000","last_active":"2024-05-08T05:41:37.153781+0000","last_peered":"2024-05-08T05:41:37.153781+0000","last_clean":"2024-05-08T05:41:37.153781+0000","last_became_active":"2024-05-08T05:41:36.151495+0000","last_became_peered":"2024-05-08T05:41:36.151495+0000","last_unstale":"2024-05-08T05:41:37.153781+0000","last_undegraded":"2024-05-08T05:41:37.153781+0000","last_fullsized":"2024-05-08T05:41:37.153781+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:34.172975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575332+0000","last_change":"2024-05-08T05:41:32.137651+0000","last_active":"2024-05-08T05:41:37.575332+0000","last_peered":"2024-05-08T05:41:37.575332+0000","last_clean":"2024-05-08T05:41:37.575332+0000","last_became_active":"2024-05-08T05:41:32.137380+0000","last_became_peered":"2024-05-08T05:41:32.137380+0000","last_unstale":"2024-05-08T05:41:37.575332+0000","last_undegraded":"2024-05-08T05:41:37.575332+0000","last_fullsized":"2024-05-08T05:41:37.575332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:44.038894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577757+0000","last_change":"2024-05-08T05:41:31.132598+0000","last_active":"2024-05-08T05:41:37.577757+0000","last_peered":"2024-05-08T05:41:37.577757+0000","last_clean":"2024-05-08T05:41:37.577757+0000","last_became_active":"2024-05-08T05:41:31.131053+0000","last_became_peered":"2024-05-08T05:41:31.131053+0000","last_unstale":"2024-05-08T05:41:37.577757+0000","last_undegraded":"2024-05-08T05:41:37.577757+0000","last_fullsized":"2024-05-08T05:41:37.577757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:11:56.288788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575991+0000","last_change":"2024-05-08T05:41:36.159470+0000","last_active":"2024-05-08T05:41:37.575991+0000","last_peered":"2024-05-08T05:41:37.575991+0000","last_clean":"2024-05-08T05:41:37.575991+0000","last_became_active":"2024-05-08T05:41:36.158246+0000","last_became_peered":"2024-05-08T05:41:36.158246+0000","last_unstale":"2024-05-08T05:41:37.575991+0000","last_undegraded":"2024-05-08T05:41:37.575991+0000","last_fullsized":"2024-05-08T05:41:37.575991+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:02.479497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577667+0000","last_change":"2024-05-08T05:41:32.139258+0000","last_active":"2024-05-08T05:41:37.577667+0000","last_peered":"2024-05-08T05:41:37.577667+0000","last_clean":"2024-05-08T05:41:37.577667+0000","last_became_active":"2024-05-08T05:41:32.138975+0000","last_became_peered":"2024-05-08T05:41:32.138975+0000","last_unstale":"2024-05-08T05:41:37.577667+0000","last_undegraded":"2024-05-08T05:41:37.577667+0000","last_fullsized":"2024-05-08T05:41:37.577667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:16.501105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577263+0000","last_change":"2024-05-08T05:41:31.133643+0000","last_active":"2024-05-08T05:41:37.577263+0000","last_peered":"2024-05-08T05:41:37.577263+0000","last_clean":"2024-05-08T05:41:37.577263+0000","last_became_active":"2024-05-08T05:41:31.133256+0000","last_became_peered":"2024-05-08T05:41:31.133256+0000","last_unstale":"2024-05-08T05:41:37.577263+0000","last_undegraded":"2024-05-08T05:41:37.577263+0000","last_fullsized":"2024-05-08T05:41:37.577263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:07.648895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576551+0000","last_change":"2024-05-08T05:41:36.383032+0000","last_active":"2024-05-08T05:41:37.576551+0000","last_peered":"2024-05-08T05:41:37.576551+0000","last_clean":"2024-05-08T05:41:37.576551+0000","last_became_active":"2024-05-08T05:41:36.382632+0000","last_became_peered":"2024-05-08T05:41:36.382632+0000","last_unstale":"2024-05-08T05:41:37.576551+0000","last_undegraded":"2024-05-08T05:41:37.576551+0000","last_fullsized":"2024-05-08T05:41:37.576551+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:07:08.116447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575585+0000","last_change":"2024-05-08T05:41:36.385447+0000","last_active":"2024-05-08T05:41:37.575585+0000","last_peered":"2024-05-08T05:41:37.575585+0000","last_clean":"2024-05-08T05:41:37.575585+0000","last_became_active":"2024-05-08T05:41:36.385062+0000","last_became_peered":"2024-05-08T05:41:36.385062+0000","last_unstale":"2024-05-08T05:41:37.575585+0000","last_undegraded":"2024-05-08T05:41:37.575585+0000","last_fullsized":"2024-05-08T05:41:37.575585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:06.547337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.574973+0000","last_change":"2024-05-08T05:41:32.127321+0000","last_active":"2024-05-08T05:41:37.574973+0000","last_peered":"2024-05-08T05:41:37.574973+0000","last_clean":"2024-05-08T05:41:37.574973+0000","last_became_active":"2024-05-08T05:41:32.127028+0000","last_became_peered":"2024-05-08T05:41:32.127028+0000","last_unstale":"2024-05-08T05:41:37.574973+0000","last_undegraded":"2024-05-08T05:41:37.574973+0000","last_fullsized":"2024-05-08T05:41:37.574973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:47.493805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575819+0000","last_change":"2024-05-08T05:41:31.131618+0000","last_active":"2024-05-08T05:41:37.575819+0000","last_peered":"2024-05-08T05:41:37.575819+0000","last_clean":"2024-05-08T05:41:37.575819+0000","last_became_active":"2024-05-08T05:41:31.131012+0000","last_became_peered":"2024-05-08T05:41:31.131012+0000","last_unstale":"2024-05-08T05:41:37.575819+0000","last_undegraded":"2024-05-08T05:41:37.575819+0000","last_fullsized":"2024-05-08T05:41:37.575819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:22.751500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577121+0000","last_change":"2024-05-08T05:41:36.158506+0000","last_active":"2024-05-08T05:41:37.577121+0000","last_peered":"2024-05-08T05:41:37.577121+0000","last_clean":"2024-05-08T05:41:37.577121+0000","last_became_active":"2024-05-08T05:41:36.158020+0000","last_became_peered":"2024-05-08T05:41:36.158020+0000","last_unstale":"2024-05-08T05:41:37.577121+0000","last_undegraded":"2024-05-08T05:41:37.577121+0000","last_fullsized":"2024-05-08T05:41:37.577121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:20.531197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576038+0000","last_change":"2024-05-08T05:41:32.132554+0000","last_active":"2024-05-08T05:41:37.576038+0000","last_peered":"2024-05-08T05:41:37.576038+0000","last_clean":"2024-05-08T05:41:37.576038+0000","last_became_active":"2024-05-08T05:41:32.132257+0000","last_became_peered":"2024-05-08T05:41:32.132257+0000","last_unstale":"2024-05-08T05:41:37.576038+0000","last_undegraded":"2024-05-08T05:41:37.576038+0000","last_fullsized":"2024-05-08T05:41:37.576038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:16:33.854897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576780+0000","last_change":"2024-05-08T05:41:31.124710+0000","last_active":"2024-05-08T05:41:37.576780+0000","last_peered":"2024-05-08T05:41:37.576780+0000","last_clean":"2024-05-08T05:41:37.576780+0000","last_became_active":"2024-05-08T05:41:31.124304+0000","last_became_peered":"2024-05-08T05:41:31.124304+0000","last_unstale":"2024-05-08T05:41:37.576780+0000","last_undegraded":"2024-05-08T05:41:37.576780+0000","last_fullsized":"2024-05-08T05:41:37.576780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:49.171947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576513+0000","last_change":"2024-05-08T05:41:31.123731+0000","last_active":"2024-05-08T05:41:37.576513+0000","last_peered":"2024-05-08T05:41:37.576513+0000","last_clean":"2024-05-08T05:41:37.576513+0000","last_became_active":"2024-05-08T05:41:31.123423+0000","last_became_peered":"2024-05-08T05:41:31.123423+0000","last_unstale":"2024-05-08T05:41:37.576513+0000","last_undegraded":"2024-05-08T05:41:37.576513+0000","last_fullsized":"2024-05-08T05:41:37.576513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:24.722618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576325+0000","last_change":"2024-05-08T05:41:32.135788+0000","last_active":"2024-05-08T05:41:37.576325+0000","last_peered":"2024-05-08T05:41:37.576325+0000","last_clean":"2024-05-08T05:41:37.576325+0000","last_became_active":"2024-05-08T05:41:32.135522+0000","last_became_peered":"2024-05-08T05:41:32.135522+0000","last_unstale":"2024-05-08T05:41:37.576325+0000","last_undegraded":"2024-05-08T05:41:37.576325+0000","last_fullsized":"2024-05-08T05:41:37.576325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:42.041915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.148134+0000","last_change":"2024-05-08T05:41:36.155967+0000","last_active":"2024-05-08T05:41:37.148134+0000","last_peered":"2024-05-08T05:41:37.148134+0000","last_clean":"2024-05-08T05:41:37.148134+0000","last_became_active":"2024-05-08T05:41:36.155132+0000","last_became_peered":"2024-05-08T05:41:36.155132+0000","last_unstale":"2024-05-08T05:41:37.148134+0000","last_undegraded":"2024-05-08T05:41:37.148134+0000","last_fullsized":"2024-05-08T05:41:37.148134+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:58.729110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577152+0000","last_change":"2024-05-08T05:41:31.127645+0000","last_active":"2024-05-08T05:41:37.577152+0000","last_peered":"2024-05-08T05:41:37.577152+0000","last_clean":"2024-05-08T05:41:37.577152+0000","last_became_active":"2024-05-08T05:41:31.127330+0000","last_became_peered":"2024-05-08T05:41:31.127330+0000","last_unstale":"2024-05-08T05:41:37.577152+0000","last_undegraded":"2024-05-08T05:41:37.577152+0000","last_fullsized":"2024-05-08T05:41:37.577152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:22.751500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153479+0000","last_change":"2024-05-08T05:41:32.125718+0000","last_active":"2024-05-08T05:41:37.153479+0000","last_peered":"2024-05-08T05:41:37.153479+0000","last_clean":"2024-05-08T05:41:37.153479+0000","last_became_active":"2024-05-08T05:41:32.125435+0000","last_became_peered":"2024-05-08T05:41:32.125435+0000","last_unstale":"2024-05-08T05:41:37.153479+0000","last_undegraded":"2024-05-08T05:41:37.153479+0000","last_fullsized":"2024-05-08T05:41:37.153479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:39.180293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575823+0000","last_change":"2024-05-08T05:41:36.158103+0000","last_active":"2024-05-08T05:41:37.575823+0000","last_peered":"2024-05-08T05:41:37.575823+0000","last_clean":"2024-05-08T05:41:37.575823+0000","last_became_active":"2024-05-08T05:41:36.157668+0000","last_became_peered":"2024-05-08T05:41:36.157668+0000","last_unstale":"2024-05-08T05:41:37.575823+0000","last_undegraded":"2024-05-08T05:41:37.575823+0000","last_fullsized":"2024-05-08T05:41:37.575823+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:45:24.742821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575306+0000","last_change":"2024-05-08T05:41:32.137321+0000","last_active":"2024-05-08T05:41:37.575306+0000","last_peered":"2024-05-08T05:41:37.575306+0000","last_clean":"2024-05-08T05:41:37.575306+0000","last_became_active":"2024-05-08T05:41:32.136738+0000","last_became_peered":"2024-05-08T05:41:32.136738+0000","last_unstale":"2024-05-08T05:41:37.575306+0000","last_undegraded":"2024-05-08T05:41:37.575306+0000","last_fullsized":"2024-05-08T05:41:37.575306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:16:33.854897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.147790+0000","last_change":"2024-05-08T05:41:32.127130+0000","last_active":"2024-05-08T05:41:37.147790+0000","last_peered":"2024-05-08T05:41:37.147790+0000","last_clean":"2024-05-08T05:41:37.147790+0000","last_became_active":"2024-05-08T05:41:32.125853+0000","last_became_peered":"2024-05-08T05:41:32.125853+0000","last_unstale":"2024-05-08T05:41:37.147790+0000","last_undegraded":"2024-05-08T05:41:37.147790+0000","last_fullsized":"2024-05-08T05:41:37.147790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:47.424136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575968+0000","last_change":"2024-05-08T05:41:32.139558+0000","last_active":"2024-05-08T05:41:37.575968+0000","last_peered":"2024-05-08T05:41:37.575968+0000","last_clean":"2024-05-08T05:41:37.575968+0000","last_became_active":"2024-05-08T05:41:32.139243+0000","last_became_peered":"2024-05-08T05:41:32.139243+0000","last_unstale":"2024-05-08T05:41:37.575968+0000","last_undegraded":"2024-05-08T05:41:37.575968+0000","last_fullsized":"2024-05-08T05:41:37.575968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:07:26.082084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576235+0000","last_change":"2024-05-08T05:41:32.126388+0000","last_active":"2024-05-08T05:41:37.576235+0000","last_peered":"2024-05-08T05:41:37.576235+0000","last_clean":"2024-05-08T05:41:37.576235+0000","last_became_active":"2024-05-08T05:41:32.126078+0000","last_became_peered":"2024-05-08T05:41:32.126078+0000","last_unstale":"2024-05-08T05:41:37.576235+0000","last_undegraded":"2024-05-08T05:41:37.576235+0000","last_fullsized":"2024-05-08T05:41:37.576235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:53.563878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577307+0000","last_change":"2024-05-08T05:41:32.135523+0000","last_active":"2024-05-08T05:41:37.577307+0000","last_peered":"2024-05-08T05:41:37.577307+0000","last_clean":"2024-05-08T05:41:37.577307+0000","last_became_active":"2024-05-08T05:41:32.135248+0000","last_became_peered":"2024-05-08T05:41:32.135248+0000","last_unstale":"2024-05-08T05:41:37.577307+0000","last_undegraded":"2024-05-08T05:41:37.577307+0000","last_fullsized":"2024-05-08T05:41:37.577307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:42.041915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576262+0000","last_change":"2024-05-08T05:41:32.130227+0000","last_active":"2024-05-08T05:41:37.576262+0000","last_peered":"2024-05-08T05:41:37.576262+0000","last_clean":"2024-05-08T05:41:37.576262+0000","last_became_active":"2024-05-08T05:41:32.129175+0000","last_became_peered":"2024-05-08T05:41:32.129175+0000","last_unstale":"2024-05-08T05:41:37.576262+0000","last_undegraded":"2024-05-08T05:41:37.576262+0000","last_fullsized":"2024-05-08T05:41:37.576262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:53.563878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577357+0000","last_change":"2024-05-08T05:41:32.143284+0000","last_active":"2024-05-08T05:41:37.577357+0000","last_peered":"2024-05-08T05:41:37.577357+0000","last_clean":"2024-05-08T05:41:37.577357+0000","last_became_active":"2024-05-08T05:41:32.142815+0000","last_became_peered":"2024-05-08T05:41:32.142815+0000","last_unstale":"2024-05-08T05:41:37.577357+0000","last_undegraded":"2024-05-08T05:41:37.577357+0000","last_fullsized":"2024-05-08T05:41:37.577357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:53.563878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576778+0000","last_change":"2024-05-08T05:41:32.134965+0000","last_active":"2024-05-08T05:41:37.576778+0000","last_peered":"2024-05-08T05:41:37.576778+0000","last_clean":"2024-05-08T05:41:37.576778+0000","last_became_active":"2024-05-08T05:41:32.134641+0000","last_became_peered":"2024-05-08T05:41:32.134641+0000","last_unstale":"2024-05-08T05:41:37.576778+0000","last_undegraded":"2024-05-08T05:41:37.576778+0000","last_fullsized":"2024-05-08T05:41:37.576778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:30.142883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575749+0000","last_change":"2024-05-08T05:41:32.135139+0000","last_active":"2024-05-08T05:41:37.575749+0000","last_peered":"2024-05-08T05:41:37.575749+0000","last_clean":"2024-05-08T05:41:37.575749+0000","last_became_active":"2024-05-08T05:41:32.134839+0000","last_became_peered":"2024-05-08T05:41:32.134839+0000","last_unstale":"2024-05-08T05:41:37.575749+0000","last_undegraded":"2024-05-08T05:41:37.575749+0000","last_fullsized":"2024-05-08T05:41:37.575749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:53.563878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576660+0000","last_change":"2024-05-08T05:41:32.134673+0000","last_active":"2024-05-08T05:41:37.576660+0000","last_peered":"2024-05-08T05:41:37.576660+0000","last_clean":"2024-05-08T05:41:37.576660+0000","last_became_active":"2024-05-08T05:41:32.134402+0000","last_became_peered":"2024-05-08T05:41:32.134402+0000","last_unstale":"2024-05-08T05:41:37.576660+0000","last_undegraded":"2024-05-08T05:41:37.576660+0000","last_fullsized":"2024-05-08T05:41:37.576660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:03:20.127059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575846+0000","last_change":"2024-05-08T05:41:32.140242+0000","last_active":"2024-05-08T05:41:37.575846+0000","last_peered":"2024-05-08T05:41:37.575846+0000","last_clean":"2024-05-08T05:41:37.575846+0000","last_became_active":"2024-05-08T05:41:32.139932+0000","last_became_peered":"2024-05-08T05:41:32.139932+0000","last_unstale":"2024-05-08T05:41:37.575846+0000","last_undegraded":"2024-05-08T05:41:37.575846+0000","last_fullsized":"2024-05-08T05:41:37.575846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:42.739734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575975+0000","last_change":"2024-05-08T05:41:32.132509+0000","last_active":"2024-05-08T05:41:37.575975+0000","last_peered":"2024-05-08T05:41:37.575975+0000","last_clean":"2024-05-08T05:41:37.575975+0000","last_became_active":"2024-05-08T05:41:32.132240+0000","last_became_peered":"2024-05-08T05:41:32.132240+0000","last_unstale":"2024-05-08T05:41:37.575975+0000","last_undegraded":"2024-05-08T05:41:37.575975+0000","last_fullsized":"2024-05-08T05:41:37.575975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:07:26.071605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576973+0000","last_change":"2024-05-08T05:41:32.140982+0000","last_active":"2024-05-08T05:41:37.576973+0000","last_peered":"2024-05-08T05:41:37.576973+0000","last_clean":"2024-05-08T05:41:37.576973+0000","last_became_active":"2024-05-08T05:41:32.140698+0000","last_became_peered":"2024-05-08T05:41:32.140698+0000","last_unstale":"2024-05-08T05:41:37.576973+0000","last_undegraded":"2024-05-08T05:41:37.576973+0000","last_fullsized":"2024-05-08T05:41:37.576973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:07:26.071605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577173+0000","last_change":"2024-05-08T05:41:32.134190+0000","last_active":"2024-05-08T05:41:37.577173+0000","last_peered":"2024-05-08T05:41:37.577173+0000","last_clean":"2024-05-08T05:41:37.577173+0000","last_became_active":"2024-05-08T05:41:32.132164+0000","last_became_peered":"2024-05-08T05:41:32.132164+0000","last_unstale":"2024-05-08T05:41:37.577173+0000","last_undegraded":"2024-05-08T05:41:37.577173+0000","last_fullsized":"2024-05-08T05:41:37.577173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:13.172615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.148157+0000","last_change":"2024-05-08T05:41:32.131091+0000","last_active":"2024-05-08T05:41:37.148157+0000","last_peered":"2024-05-08T05:41:37.148157+0000","last_clean":"2024-05-08T05:41:37.148157+0000","last_became_active":"2024-05-08T05:41:32.130767+0000","last_became_peered":"2024-05-08T05:41:32.130767+0000","last_unstale":"2024-05-08T05:41:37.148157+0000","last_undegraded":"2024-05-08T05:41:37.148157+0000","last_fullsized":"2024-05-08T05:41:37.148157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:42.596678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577931+0000","last_change":"2024-05-08T05:41:32.137943+0000","last_active":"2024-05-08T05:41:37.577931+0000","last_peered":"2024-05-08T05:41:37.577931+0000","last_clean":"2024-05-08T05:41:37.577931+0000","last_became_active":"2024-05-08T05:41:32.137662+0000","last_became_peered":"2024-05-08T05:41:32.137662+0000","last_unstale":"2024-05-08T05:41:37.577931+0000","last_undegraded":"2024-05-08T05:41:37.577931+0000","last_fullsized":"2024-05-08T05:41:37.577931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:17.291314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575635+0000","last_change":"2024-05-08T05:41:32.129374+0000","last_active":"2024-05-08T05:41:37.575635+0000","last_peered":"2024-05-08T05:41:37.575635+0000","last_clean":"2024-05-08T05:41:37.575635+0000","last_became_active":"2024-05-08T05:41:32.128594+0000","last_became_peered":"2024-05-08T05:41:32.128594+0000","last_unstale":"2024-05-08T05:41:37.575635+0000","last_undegraded":"2024-05-08T05:41:37.575635+0000","last_fullsized":"2024-05-08T05:41:37.575635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:01:32.075337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575072+0000","last_change":"2024-05-08T05:41:32.131159+0000","last_active":"2024-05-08T05:41:37.575072+0000","last_peered":"2024-05-08T05:41:37.575072+0000","last_clean":"2024-05-08T05:41:37.575072+0000","last_became_active":"2024-05-08T05:41:32.130766+0000","last_became_peered":"2024-05-08T05:41:32.130766+0000","last_unstale":"2024-05-08T05:41:37.575072+0000","last_undegraded":"2024-05-08T05:41:37.575072+0000","last_fullsized":"2024-05-08T05:41:37.575072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:16:33.854897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575282+0000","last_change":"2024-05-08T05:41:32.119797+0000","last_active":"2024-05-08T05:41:37.575282+0000","last_peered":"2024-05-08T05:41:37.575282+0000","last_clean":"2024-05-08T05:41:37.575282+0000","last_became_active":"2024-05-08T05:41:32.119134+0000","last_became_peered":"2024-05-08T05:41:32.119134+0000","last_unstale":"2024-05-08T05:41:37.575282+0000","last_undegraded":"2024-05-08T05:41:37.575282+0000","last_fullsized":"2024-05-08T05:41:37.575282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:47.493805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577070+0000","last_change":"2024-05-08T05:41:32.135351+0000","last_active":"2024-05-08T05:41:37.577070+0000","last_peered":"2024-05-08T05:41:37.577070+0000","last_clean":"2024-05-08T05:41:37.577070+0000","last_became_active":"2024-05-08T05:41:32.133812+0000","last_became_peered":"2024-05-08T05:41:32.133812+0000","last_unstale":"2024-05-08T05:41:37.577070+0000","last_undegraded":"2024-05-08T05:41:37.577070+0000","last_fullsized":"2024-05-08T05:41:37.577070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:29.473897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154048+0000","last_change":"2024-05-08T05:41:32.136108+0000","last_active":"2024-05-08T05:41:37.154048+0000","last_peered":"2024-05-08T05:41:37.154048+0000","last_clean":"2024-05-08T05:41:37.154048+0000","last_became_active":"2024-05-08T05:41:32.135622+0000","last_became_peered":"2024-05-08T05:41:32.135622+0000","last_unstale":"2024-05-08T05:41:37.154048+0000","last_undegraded":"2024-05-08T05:41:37.154048+0000","last_fullsized":"2024-05-08T05:41:37.154048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:14.470927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576751+0000","last_change":"2024-05-08T05:41:32.138499+0000","last_active":"2024-05-08T05:41:37.576751+0000","last_peered":"2024-05-08T05:41:37.576751+0000","last_clean":"2024-05-08T05:41:37.576751+0000","last_became_active":"2024-05-08T05:41:32.138049+0000","last_became_peered":"2024-05-08T05:41:32.138049+0000","last_unstale":"2024-05-08T05:41:37.576751+0000","last_undegraded":"2024-05-08T05:41:37.576751+0000","last_fullsized":"2024-05-08T05:41:37.576751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:03:20.127059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577193+0000","last_change":"2024-05-08T05:41:32.138310+0000","last_active":"2024-05-08T05:41:37.577193+0000","last_peered":"2024-05-08T05:41:37.577193+0000","last_clean":"2024-05-08T05:41:37.577193+0000","last_became_active":"2024-05-08T05:41:32.137983+0000","last_became_peered":"2024-05-08T05:41:32.137983+0000","last_unstale":"2024-05-08T05:41:37.577193+0000","last_undegraded":"2024-05-08T05:41:37.577193+0000","last_fullsized":"2024-05-08T05:41:37.577193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:42.041915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577267+0000","last_change":"2024-05-08T05:41:32.144208+0000","last_active":"2024-05-08T05:41:37.577267+0000","last_peered":"2024-05-08T05:41:37.577267+0000","last_clean":"2024-05-08T05:41:37.577267+0000","last_became_active":"2024-05-08T05:41:32.143884+0000","last_became_peered":"2024-05-08T05:41:32.143884+0000","last_unstale":"2024-05-08T05:41:37.577267+0000","last_undegraded":"2024-05-08T05:41:37.577267+0000","last_fullsized":"2024-05-08T05:41:37.577267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:07:26.082084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575800+0000","last_change":"2024-05-08T05:41:32.136185+0000","last_active":"2024-05-08T05:41:37.575800+0000","last_peered":"2024-05-08T05:41:37.575800+0000","last_clean":"2024-05-08T05:41:37.575800+0000","last_became_active":"2024-05-08T05:41:32.135878+0000","last_became_peered":"2024-05-08T05:41:32.135878+0000","last_unstale":"2024-05-08T05:41:37.575800+0000","last_undegraded":"2024-05-08T05:41:37.575800+0000","last_fullsized":"2024-05-08T05:41:37.575800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:44.038894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576529+0000","last_change":"2024-05-08T05:41:32.137009+0000","last_active":"2024-05-08T05:41:37.576529+0000","last_peered":"2024-05-08T05:41:37.576529+0000","last_clean":"2024-05-08T05:41:37.576529+0000","last_became_active":"2024-05-08T05:41:32.136708+0000","last_became_peered":"2024-05-08T05:41:32.136708+0000","last_unstale":"2024-05-08T05:41:37.576529+0000","last_undegraded":"2024-05-08T05:41:37.576529+0000","last_fullsized":"2024-05-08T05:41:37.576529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:30.142883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575732+0000","last_change":"2024-05-08T05:41:32.137351+0000","last_active":"2024-05-08T05:41:37.575732+0000","last_peered":"2024-05-08T05:41:37.575732+0000","last_clean":"2024-05-08T05:41:37.575732+0000","last_became_active":"2024-05-08T05:41:32.135445+0000","last_became_peered":"2024-05-08T05:41:32.135445+0000","last_unstale":"2024-05-08T05:41:37.575732+0000","last_undegraded":"2024-05-08T05:41:37.575732+0000","last_fullsized":"2024-05-08T05:41:37.575732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:30.142883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575746+0000","last_change":"2024-05-08T05:41:32.132566+0000","last_active":"2024-05-08T05:41:37.575746+0000","last_peered":"2024-05-08T05:41:37.575746+0000","last_clean":"2024-05-08T05:41:37.575746+0000","last_became_active":"2024-05-08T05:41:32.131787+0000","last_became_peered":"2024-05-08T05:41:32.131787+0000","last_unstale":"2024-05-08T05:41:37.575746+0000","last_undegraded":"2024-05-08T05:41:37.575746+0000","last_fullsized":"2024-05-08T05:41:37.575746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:42.739734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576521+0000","last_change":"2024-05-08T05:41:32.135611+0000","last_active":"2024-05-08T05:41:37.576521+0000","last_peered":"2024-05-08T05:41:37.576521+0000","last_clean":"2024-05-08T05:41:37.576521+0000","last_became_active":"2024-05-08T05:41:32.135346+0000","last_became_peered":"2024-05-08T05:41:32.135346+0000","last_unstale":"2024-05-08T05:41:37.576521+0000","last_undegraded":"2024-05-08T05:41:37.576521+0000","last_fullsized":"2024-05-08T05:41:37.576521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:29.473897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575102+0000","last_change":"2024-05-08T05:41:32.118779+0000","last_active":"2024-05-08T05:41:37.575102+0000","last_peered":"2024-05-08T05:41:37.575102+0000","last_clean":"2024-05-08T05:41:37.575102+0000","last_became_active":"2024-05-08T05:41:32.118481+0000","last_became_peered":"2024-05-08T05:41:32.118481+0000","last_unstale":"2024-05-08T05:41:37.575102+0000","last_undegraded":"2024-05-08T05:41:37.575102+0000","last_fullsized":"2024-05-08T05:41:37.575102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:54.699019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575394+0000","last_change":"2024-05-08T05:41:32.128168+0000","last_active":"2024-05-08T05:41:37.575394+0000","last_peered":"2024-05-08T05:41:37.575394+0000","last_clean":"2024-05-08T05:41:37.575394+0000","last_became_active":"2024-05-08T05:41:32.127869+0000","last_became_peered":"2024-05-08T05:41:32.127869+0000","last_unstale":"2024-05-08T05:41:37.575394+0000","last_undegraded":"2024-05-08T05:41:37.575394+0000","last_fullsized":"2024-05-08T05:41:37.575394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:07:26.071605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577048+0000","last_change":"2024-05-08T05:41:32.134768+0000","last_active":"2024-05-08T05:41:37.577048+0000","last_peered":"2024-05-08T05:41:37.577048+0000","last_clean":"2024-05-08T05:41:37.577048+0000","last_became_active":"2024-05-08T05:41:32.132540+0000","last_became_peered":"2024-05-08T05:41:32.132540+0000","last_unstale":"2024-05-08T05:41:37.577048+0000","last_undegraded":"2024-05-08T05:41:37.577048+0000","last_fullsized":"2024-05-08T05:41:37.577048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:50:32.252569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":847872,"data_stored":690366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":101028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":99506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":99542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":262144,"data_stored":103258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":249856,"data_stored":99416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":258048,"data_stored":99576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":856064,"data_stored":690422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:41:46.593 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T05:41:46.594 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T05:41:46.594 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-08T05:41:46.594 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-08T05:41:46.594 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-08T05:41:46.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.733+0000 7f2eb1f46640 1 Processor -- start 2024-05-08T05:41:46.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.733+0000 7f2eb1f46640 1 -- start start 2024-05-08T05:41:46.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.733+0000 7f2eb1f46640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2eac0d33f0 0x7f2eac15d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:46.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.733+0000 7f2eb1f46640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2eac070b90 0x7f2eac071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:46.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.733+0000 7f2eb1f46640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eac0718c0 con 0x7f2eac15ccb0 2024-05-08T05:41:46.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.733+0000 7f2eb1f46640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2eac0afba0 con 0x7f2eac070b90 2024-05-08T05:41:46.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.733+0000 7f2eb1f46640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2eac071ba0 con 0x7f2eac0d33f0 2024-05-08T05:41:46.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.734+0000 7f2eabfff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2eac070b90 0x7f2eac071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:46.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.734+0000 7f2eaaffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2eac0d33f0 0x7f2eac15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:46.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.734+0000 7f2eab7fe640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f2eac15ccb0 0x7f2eac0d2ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:53624/0 (socket says 172.21.15.45:53624) 2024-05-08T05:41:46.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.734+0000 7f2eab7fe640 1 -- 172.21.15.45:0/1766100124 learned_addr learned my addr 172.21.15.45:0/1766100124 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:46.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.734+0000 7f2eaa7fc640 1 -- 172.21.15.45:0/1766100124 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2738195704 0 0) 0x7f2eac0718c0 con 0x7f2eac15ccb0 2024-05-08T05:41:46.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.734+0000 7f2eaa7fc640 1 -- 172.21.15.45:0/1766100124 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e98003520 con 0x7f2eac15ccb0 2024-05-08T05:41:46.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.734+0000 7f2eaaffd640 1 -- 172.21.15.45:0/1766100124 >> v2:172.21.15.63:3300/0 conn(0x7f2eac070b90 msgr2=0x7f2eac071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:46.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.734+0000 7f2eaaffd640 1 --2- 172.21.15.45:0/1766100124 >> v2:172.21.15.63:3300/0 conn(0x7f2eac070b90 0x7f2eac071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.734+0000 7f2eaaffd640 1 -- 172.21.15.45:0/1766100124 >> v1:172.21.15.63:6789/0 conn(0x7f2eac15ccb0 legacy=0x7f2eac0d2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:46.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.734+0000 7f2eaaffd640 1 -- 172.21.15.45:0/1766100124 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eac071e80 con 0x7f2eac0d33f0 2024-05-08T05:41:46.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.735+0000 7f2eaaffd640 1 --2- 172.21.15.45:0/1766100124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2eac0d33f0 0x7f2eac15d090 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2e940097a0 tx=0x7f2e9402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9811252abb29077 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:46.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.735+0000 7f2eaa7fc640 1 -- 172.21.15.45:0/1766100124 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94037070 con 0x7f2eac0d33f0 2024-05-08T05:41:46.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.735+0000 7f2eaa7fc640 1 -- 172.21.15.45:0/1766100124 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e9402ae50 con 0x7f2eac0d33f0 2024-05-08T05:41:46.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.735+0000 7f2eaa7fc640 1 -- 172.21.15.45:0/1766100124 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94032640 con 0x7f2eac0d33f0 2024-05-08T05:41:46.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.736+0000 7f2eb1f46640 1 -- 172.21.15.45:0/1766100124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2eac0d33f0 msgr2=0x7f2eac15d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:46.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.736+0000 7f2eb1f46640 1 --2- 172.21.15.45:0/1766100124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2eac0d33f0 0x7f2eac15d090 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2e940097a0 tx=0x7f2e9402a100 comp rx=0 tx=0).stop 2024-05-08T05:41:46.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.736+0000 7f2eb1f46640 1 -- 172.21.15.45:0/1766100124 shutdown_connections 2024-05-08T05:41:46.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.736+0000 7f2eb1f46640 1 --2- 172.21.15.45:0/1766100124 >> v2:172.21.15.63:3300/0 conn(0x7f2eac070b90 0x7f2eac071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.736+0000 7f2eb1f46640 1 --2- 172.21.15.45:0/1766100124 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2eac0d33f0 0x7f2eac15d090 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.736+0000 7f2eb1f46640 1 -- 172.21.15.45:0/1766100124 >> 172.21.15.45:0/1766100124 conn(0x7f2eac09ba40 msgr2=0x7f2eac09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:46.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.736+0000 7f2eb1f46640 1 -- 172.21.15.45:0/1766100124 shutdown_connections 2024-05-08T05:41:46.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.736+0000 7f2eb1f46640 1 -- 172.21.15.45:0/1766100124 wait complete. 2024-05-08T05:41:46.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.737+0000 7f2eb1f46640 1 Processor -- start 2024-05-08T05:41:46.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.737+0000 7f2eb1f46640 1 -- start start 2024-05-08T05:41:46.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.738+0000 7f2eb1f46640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2eac070b90 0x7f2eac0ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:46.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.738+0000 7f2eb1f46640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2eac0d33f0 0x7f2eac15d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:46.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.738+0000 7f2eb1f46640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eac15ccb0 0x7f2eac15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:46.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.738+0000 7f2eb1f46640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac0cf980 con 0x7f2eac15ccb0 2024-05-08T05:41:46.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.738+0000 7f2eb1f46640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac071560 con 0x7f2eac0d33f0 2024-05-08T05:41:46.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.738+0000 7f2eb1f46640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2eac0cfb00 con 0x7f2eac070b90 2024-05-08T05:41:46.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.738+0000 7f2eabfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eac15ccb0 0x7f2eac15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:46.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.738+0000 7f2eabfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eac15ccb0 0x7f2eac15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51232/0 (socket says 172.21.15.45:51232) 2024-05-08T05:41:46.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.738+0000 7f2eabfff640 1 -- 172.21.15.45:0/4106095240 learned_addr learned my addr 172.21.15.45:0/4106095240 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:46.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.738+0000 7f2eab7fe640 1 --2- 172.21.15.45:0/4106095240 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2eac070b90 0x7f2eac0ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:46.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.738+0000 7f2eaaffd640 1 --2- 172.21.15.45:0/4106095240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2eac0d33f0 0x7f2eac15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:46.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.738+0000 7f2eabfff640 1 -- 172.21.15.45:0/4106095240 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2eac070b90 msgr2=0x7f2eac0ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:46.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.738+0000 7f2eabfff640 1 --2- 172.21.15.45:0/4106095240 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2eac070b90 0x7f2eac0ca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.738+0000 7f2eabfff640 1 -- 172.21.15.45:0/4106095240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2eac0d33f0 msgr2=0x7f2eac15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:46.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.738+0000 7f2eabfff640 1 --2- 172.21.15.45:0/4106095240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2eac0d33f0 0x7f2eac15d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.738+0000 7f2eabfff640 1 -- 172.21.15.45:0/4106095240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eac15f010 con 0x7f2eac15ccb0 2024-05-08T05:41:46.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.738+0000 7f2eaaffd640 1 --2- 172.21.15.45:0/4106095240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2eac0d33f0 0x7f2eac15d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:46.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.739+0000 7f2eabfff640 1 --2- 172.21.15.45:0/4106095240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eac15ccb0 0x7f2eac15d5d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400efc0 tx=0x7f2ea400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:46.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.739+0000 7f2ea3fff640 1 -- 172.21.15.45:0/4106095240 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4018020 con 0x7f2eac15ccb0 2024-05-08T05:41:46.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.739+0000 7f2ea3fff640 1 -- 172.21.15.45:0/4106095240 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea4014480 con 0x7f2eac15ccb0 2024-05-08T05:41:46.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.739+0000 7f2ea3fff640 1 -- 172.21.15.45:0/4106095240 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4014870 con 0x7f2eac15ccb0 2024-05-08T05:41:46.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.739+0000 7f2eb1f46640 1 -- 172.21.15.45:0/4106095240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eac15f2b0 con 0x7f2eac15ccb0 2024-05-08T05:41:46.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.739+0000 7f2eb1f46640 1 -- 172.21.15.45:0/4106095240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eac15f870 con 0x7f2eac15ccb0 2024-05-08T05:41:46.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.740+0000 7f2eb1f46640 1 -- 172.21.15.45:0/4106095240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eac069b00 con 0x7f2eac15ccb0 2024-05-08T05:41:46.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.740+0000 7f2ea3fff640 1 -- 172.21.15.45:0/4106095240 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2ea4005d10 con 0x7f2eac15ccb0 2024-05-08T05:41:46.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.741+0000 7f2ea3fff640 1 --2- 172.21.15.45:0/4106095240 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2e8c0762d0 0x7f2e8c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:46.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.741+0000 7f2ea3fff640 1 -- 172.21.15.45:0/4106095240 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2ea40983d0 con 0x7f2eac15ccb0 2024-05-08T05:41:46.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.741+0000 7f2eab7fe640 1 --2- 172.21.15.45:0/4106095240 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2e8c0762d0 0x7f2e8c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:46.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.742+0000 7f2eab7fe640 1 --2- 172.21.15.45:0/4106095240 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2e8c0762d0 0x7f2e8c078790 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c007a60 tx=0x7f2e9c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:46.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.745+0000 7f2ea3fff640 1 -- 172.21.15.45:0/4106095240 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2ea4022070 con 0x7f2eac15ccb0 2024-05-08T05:41:46.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.988+0000 7f2eb1f46640 1 -- 172.21.15.45:0/4106095240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f2eac070380 con 0x7f2eac15ccb0 2024-05-08T05:41:46.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.991+0000 7f2ea3fff640 1 -- 172.21.15.45:0/4106095240 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+189521 (secure 0 0 0) 0x7f2ea4010040 con 0x7f2eac15ccb0 2024-05-08T05:41:46.994 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:46.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.994+0000 7f2eb1f46640 1 -- 172.21.15.45:0/4106095240 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2e8c0762d0 msgr2=0x7f2e8c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:46.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.994+0000 7f2eb1f46640 1 --2- 172.21.15.45:0/4106095240 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2e8c0762d0 0x7f2e8c078790 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c007a60 tx=0x7f2e9c009040 comp rx=0 tx=0).stop 2024-05-08T05:41:46.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.994+0000 7f2eb1f46640 1 -- 172.21.15.45:0/4106095240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eac15ccb0 msgr2=0x7f2eac15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:46.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.994+0000 7f2eb1f46640 1 --2- 172.21.15.45:0/4106095240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eac15ccb0 0x7f2eac15d5d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400efc0 tx=0x7f2ea400c490 comp rx=0 tx=0).stop 2024-05-08T05:41:46.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.994+0000 7f2eb1f46640 1 -- 172.21.15.45:0/4106095240 shutdown_connections 2024-05-08T05:41:46.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.994+0000 7f2eb1f46640 1 --2- 172.21.15.45:0/4106095240 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2e8c0762d0 0x7f2e8c078790 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.994+0000 7f2eb1f46640 1 --2- 172.21.15.45:0/4106095240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eac15ccb0 0x7f2eac15d5d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.994+0000 7f2eb1f46640 1 --2- 172.21.15.45:0/4106095240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2eac0d33f0 0x7f2eac15d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.994+0000 7f2eb1f46640 1 --2- 172.21.15.45:0/4106095240 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2eac070b90 0x7f2eac0ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:46.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.994+0000 7f2eb1f46640 1 -- 172.21.15.45:0/4106095240 >> 172.21.15.45:0/4106095240 conn(0x7f2eac09ba40 msgr2=0x7f2eac0b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:46.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.994+0000 7f2eb1f46640 1 -- 172.21.15.45:0/4106095240 shutdown_connections 2024-05-08T05:41:46.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:46.994+0000 7f2eb1f46640 1 -- 172.21.15.45:0/4106095240 wait complete. 2024-05-08T05:41:47.006 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":2860796916},{"type":"v1","addr":"172.21.15.63:6833","nonce":2860796916}]},"active_addr":"172.21.15.63:6833/2860796916","active_change":"2024-05-08T05:41:12.822660+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":4181761907}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":1487953711}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":3308558014}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":2697607777}]}]} 2024-05-08T05:41:47.009 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-08T05:41:47.010 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-08T05:41:47.010 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:41:47.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.113+0000 7f6ff1f9e640 1 Processor -- start 2024-05-08T05:41:47.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.113+0000 7f6ff1f9e640 1 -- start start 2024-05-08T05:41:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.114+0000 7f6ff1f9e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6fec0c9aa0 0x7f6fec16a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.114+0000 7f6ff1f9e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fec0b6330 con 0x7f6fec15f480 2024-05-08T05:41:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.114+0000 7f6ff1f9e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fec0b6610 con 0x7f6fec0d5a60 2024-05-08T05:41:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.114+0000 7f6ff1f9e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f6fec0d0c40 con 0x7f6fec0c9aa0 2024-05-08T05:41:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.114+0000 7f6ff179d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6fec0c9aa0 0x7f6fec16a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.114+0000 7f6ff0f9c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6fec15f480 0x7f6fec0d1570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43926/0 (socket says 172.21.15.45:43926) 2024-05-08T05:41:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.114+0000 7f6ff0f9c640 1 -- 172.21.15.45:0/3355801619 learned_addr learned my addr 172.21.15.45:0/3355801619 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.114+0000 7f6ff179d640 1 -- 172.21.15.45:0/3355801619 >> v1:172.21.15.63:6789/0 conn(0x7f6fec0d5a60 legacy=0x7f6fec0d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.114+0000 7f6ff179d640 1 -- 172.21.15.45:0/3355801619 >> v1:172.21.15.45:6789/0 conn(0x7f6fec15f480 legacy=0x7f6fec0d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.114+0000 7f6ff179d640 1 -- 172.21.15.45:0/3355801619 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fec0d05f0 con 0x7f6fec0c9aa0 2024-05-08T05:41:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.114+0000 7f6fe37fe640 1 -- 172.21.15.45:0/3355801619 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1652488432 0 0) 0x7f6fec0b6330 con 0x7f6fec15f480 2024-05-08T05:41:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.115+0000 7f6ff179d640 1 --2- 172.21.15.45:0/3355801619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec0c9aa0 0x7f6fec16a150 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009dc0 tx=0x7f6fe402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527e01e25364ce30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.115+0000 7f6fe37fe640 1 -- 172.21.15.45:0/3355801619 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4007740 con 0x7f6fec0c9aa0 2024-05-08T05:41:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.115+0000 7f6fe37fe640 1 -- 172.21.15.45:0/3355801619 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fe40078e0 con 0x7f6fec0c9aa0 2024-05-08T05:41:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.115+0000 7f6fe37fe640 1 -- 172.21.15.45:0/3355801619 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4007cb0 con 0x7f6fec0c9aa0 2024-05-08T05:41:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.116+0000 7f6ff1f9e640 1 -- 172.21.15.45:0/3355801619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec0c9aa0 msgr2=0x7f6fec16a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.116+0000 7f6ff1f9e640 1 --2- 172.21.15.45:0/3355801619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec0c9aa0 0x7f6fec16a150 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009dc0 tx=0x7f6fe402a710 comp rx=0 tx=0).stop 2024-05-08T05:41:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.116+0000 7f6ff1f9e640 1 -- 172.21.15.45:0/3355801619 shutdown_connections 2024-05-08T05:41:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.116+0000 7f6ff1f9e640 1 --2- 172.21.15.45:0/3355801619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec0c9aa0 0x7f6fec16a150 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.116+0000 7f6ff1f9e640 1 -- 172.21.15.45:0/3355801619 >> 172.21.15.45:0/3355801619 conn(0x7f6fec0d4f40 msgr2=0x7f6fec0d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.116+0000 7f6ff1f9e640 1 -- 172.21.15.45:0/3355801619 shutdown_connections 2024-05-08T05:41:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.116+0000 7f6ff1f9e640 1 -- 172.21.15.45:0/3355801619 wait complete. 2024-05-08T05:41:47.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.117+0000 7f6ff1f9e640 1 Processor -- start 2024-05-08T05:41:47.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.117+0000 7f6ff1f9e640 1 -- start start 2024-05-08T05:41:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.118+0000 7f6ff1f9e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec0c9aa0 0x7f6fec0ba800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.118+0000 7f6ff1f9e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fec0d5a60 0x7f6fec162600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.118+0000 7f6ff1f9e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fec15f480 0x7f6fec0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.118+0000 7f6ff1f9e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec15f860 con 0x7f6fec0c9aa0 2024-05-08T05:41:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.118+0000 7f6ff1f9e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec0b6000 con 0x7f6fec15f480 2024-05-08T05:41:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.118+0000 7f6ff1f9e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6fec15f9e0 con 0x7f6fec0d5a60 2024-05-08T05:41:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.118+0000 7f6ff0f9c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec0c9aa0 0x7f6fec0ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.118+0000 7f6ff0f9c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec0c9aa0 0x7f6fec0ba800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51240/0 (socket says 172.21.15.45:51240) 2024-05-08T05:41:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.118+0000 7f6ff0f9c640 1 -- 172.21.15.45:0/3226092882 learned_addr learned my addr 172.21.15.45:0/3226092882 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.118+0000 7f6ff179d640 1 --2- 172.21.15.45:0/3226092882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fec15f480 0x7f6fec0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.118+0000 7f6fe3fff640 1 --2- 172.21.15.45:0/3226092882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fec0d5a60 0x7f6fec162600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.118+0000 7f6ff0f9c640 1 -- 172.21.15.45:0/3226092882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fec0d5a60 msgr2=0x7f6fec162600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.118+0000 7f6ff0f9c640 1 --2- 172.21.15.45:0/3226092882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fec0d5a60 0x7f6fec162600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.118+0000 7f6ff0f9c640 1 -- 172.21.15.45:0/3226092882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fec15f480 msgr2=0x7f6fec0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.118+0000 7f6ff0f9c640 1 --2- 172.21.15.45:0/3226092882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fec15f480 0x7f6fec0cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.118+0000 7f6ff0f9c640 1 -- 172.21.15.45:0/3226092882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fec160b00 con 0x7f6fec0c9aa0 2024-05-08T05:41:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.118+0000 7f6ff179d640 1 --2- 172.21.15.45:0/3226092882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fec15f480 0x7f6fec0cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.119+0000 7f6ff0f9c640 1 --2- 172.21.15.45:0/3226092882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec0c9aa0 0x7f6fec0ba800 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6fe800c3e0 tx=0x7f6fe800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.119+0000 7f6fe17fa640 1 -- 172.21.15.45:0/3226092882 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe80074e0 con 0x7f6fec0c9aa0 2024-05-08T05:41:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.119+0000 7f6fe17fa640 1 -- 172.21.15.45:0/3226092882 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fe80047e0 con 0x7f6fec0c9aa0 2024-05-08T05:41:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.119+0000 7f6ff1f9e640 1 -- 172.21.15.45:0/3226092882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fec160de0 con 0x7f6fec0c9aa0 2024-05-08T05:41:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.119+0000 7f6fe17fa640 1 -- 172.21.15.45:0/3226092882 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8004bd0 con 0x7f6fec0c9aa0 2024-05-08T05:41:47.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.119+0000 7f6ff1f9e640 1 -- 172.21.15.45:0/3226092882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fec1613a0 con 0x7f6fec0c9aa0 2024-05-08T05:41:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.120+0000 7f6ff1f9e640 1 -- 172.21.15.45:0/3226092882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fec050740 con 0x7f6fec0c9aa0 2024-05-08T05:41:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.121+0000 7f6fe17fa640 1 -- 172.21.15.45:0/3226092882 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6fe8011070 con 0x7f6fec0c9aa0 2024-05-08T05:41:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.121+0000 7f6fe17fa640 1 --2- 172.21.15.45:0/3226092882 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6fd80762d0 0x7f6fd8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.121+0000 7f6fe17fa640 1 -- 172.21.15.45:0/3226092882 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6fe8097590 con 0x7f6fec0c9aa0 2024-05-08T05:41:47.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.121+0000 7f6fe3fff640 1 --2- 172.21.15.45:0/3226092882 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6fd80762d0 0x7f6fd8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.122+0000 7f6fe3fff640 1 --2- 172.21.15.45:0/3226092882 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6fd80762d0 0x7f6fd8078790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0d00c0 tx=0x7f6fdc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.126+0000 7f6fe17fa640 1 -- 172.21.15.45:0/3226092882 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6fe8060bb0 con 0x7f6fec0c9aa0 2024-05-08T05:41:47.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.322+0000 7f6ff1f9e640 1 -- 172.21.15.45:0/3226092882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fec0b8a00 con 0x7f6fec0c9aa0 2024-05-08T05:41:47.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.323+0000 7f6fe17fa640 1 -- 172.21.15.45:0/3226092882 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18466 (secure 0 0 0) 0x7f6fe80604b0 con 0x7f6fec0c9aa0 2024-05-08T05:41:47.323 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:47.323 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:41:36.532045+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:41:29.381274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:41:30.443612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:41:34.132285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:41:47.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.325+0000 7f6ff1f9e640 1 -- 172.21.15.45:0/3226092882 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6fd80762d0 msgr2=0x7f6fd8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.325+0000 7f6ff1f9e640 1 --2- 172.21.15.45:0/3226092882 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6fd80762d0 0x7f6fd8078790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0d00c0 tx=0x7f6fdc009040 comp rx=0 tx=0).stop 2024-05-08T05:41:47.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.325+0000 7f6ff1f9e640 1 -- 172.21.15.45:0/3226092882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec0c9aa0 msgr2=0x7f6fec0ba800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.325+0000 7f6ff1f9e640 1 --2- 172.21.15.45:0/3226092882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec0c9aa0 0x7f6fec0ba800 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6fe800c3e0 tx=0x7f6fe800c8a0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.325+0000 7f6ff1f9e640 1 -- 172.21.15.45:0/3226092882 shutdown_connections 2024-05-08T05:41:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.325+0000 7f6ff1f9e640 1 --2- 172.21.15.45:0/3226092882 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6fd80762d0 0x7f6fd8078790 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.325+0000 7f6ff1f9e640 1 --2- 172.21.15.45:0/3226092882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6fec0d5a60 0x7f6fec162600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.325+0000 7f6ff1f9e640 1 --2- 172.21.15.45:0/3226092882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6fec15f480 0x7f6fec0cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.325+0000 7f6ff1f9e640 1 --2- 172.21.15.45:0/3226092882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec0c9aa0 0x7f6fec0ba800 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.325+0000 7f6ff1f9e640 1 -- 172.21.15.45:0/3226092882 >> 172.21.15.45:0/3226092882 conn(0x7f6fec0d4f40 msgr2=0x7f6fec15e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.325+0000 7f6ff1f9e640 1 -- 172.21.15.45:0/3226092882 shutdown_connections 2024-05-08T05:41:47.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.325+0000 7f6ff1f9e640 1 -- 172.21.15.45:0/3226092882 wait complete. 2024-05-08T05:41:47.337 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-08T05:41:47.337 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:41:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.440+0000 7f2cb81f9640 1 Processor -- start 2024-05-08T05:41:47.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.440+0000 7f2cb81f9640 1 -- start start 2024-05-08T05:41:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.441+0000 7f2cb81f9640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2cb015f450 0x7f2cb006d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.441+0000 7f2cb81f9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cb015f830 0x7f2cb006d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.441+0000 7f2cb81f9640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cb00d0cb0 con 0x7f2cb00d0700 2024-05-08T05:41:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.441+0000 7f2cb81f9640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f2cb006d690 con 0x7f2cb015f450 2024-05-08T05:41:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.441+0000 7f2cb81f9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb00d0f90 con 0x7f2cb015f830 2024-05-08T05:41:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.441+0000 7f2cb6796640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2cb00d0700 0x7f2cb00d17e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43932/0 (socket says 172.21.15.45:43932) 2024-05-08T05:41:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.441+0000 7f2cb5f95640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2cb015f450 0x7f2cb006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.441+0000 7f2cb6796640 1 -- 172.21.15.45:0/2089977991 learned_addr learned my addr 172.21.15.45:0/2089977991 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.441+0000 7f2cb5794640 1 --2- 172.21.15.45:0/2089977991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cb015f830 0x7f2cb006d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.441+0000 7f2cb5f95640 1 -- 172.21.15.45:0/2089977991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cb015f830 msgr2=0x7f2cb006d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.441+0000 7f2cb5f95640 1 --2- 172.21.15.45:0/2089977991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cb015f830 0x7f2cb006d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.441+0000 7f2cb5f95640 1 -- 172.21.15.45:0/2089977991 >> v1:172.21.15.45:6789/0 conn(0x7f2cb00d0700 legacy=0x7f2cb00d17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.441+0000 7f2cb5f95640 1 -- 172.21.15.45:0/2089977991 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb00d2fe0 con 0x7f2cb015f450 2024-05-08T05:41:47.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.441+0000 7f2cb4f93640 1 -- 172.21.15.45:0/2089977991 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4218301639 0 0) 0x7f2cb00d0cb0 con 0x7f2cb00d0700 2024-05-08T05:41:47.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.442+0000 7f2cb5f95640 1 --2- 172.21.15.45:0/2089977991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cb015f450 0x7f2cb006d150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2cac0097a0 tx=0x7f2cac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb50e9ea6d20886 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.442+0000 7f2cb4f93640 1 -- 172.21.15.45:0/2089977991 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac037070 con 0x7f2cb015f450 2024-05-08T05:41:47.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.442+0000 7f2cb4f93640 1 -- 172.21.15.45:0/2089977991 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cac032440 con 0x7f2cb015f450 2024-05-08T05:41:47.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.442+0000 7f2cb4f93640 1 -- 172.21.15.45:0/2089977991 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cac032810 con 0x7f2cb015f450 2024-05-08T05:41:47.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.443+0000 7f2cb81f9640 1 -- 172.21.15.45:0/2089977991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cb015f450 msgr2=0x7f2cb006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.443+0000 7f2cb81f9640 1 --2- 172.21.15.45:0/2089977991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cb015f450 0x7f2cb006d150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2cac0097a0 tx=0x7f2cac02a180 comp rx=0 tx=0).stop 2024-05-08T05:41:47.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.443+0000 7f2cb81f9640 1 -- 172.21.15.45:0/2089977991 shutdown_connections 2024-05-08T05:41:47.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.443+0000 7f2cb81f9640 1 --2- 172.21.15.45:0/2089977991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cb015f830 0x7f2cb006d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.443+0000 7f2cb81f9640 1 --2- 172.21.15.45:0/2089977991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cb015f450 0x7f2cb006d150 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.443+0000 7f2cb81f9640 1 -- 172.21.15.45:0/2089977991 >> 172.21.15.45:0/2089977991 conn(0x7f2cb00d5040 msgr2=0x7f2cb00d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:47.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.443+0000 7f2cb81f9640 1 -- 172.21.15.45:0/2089977991 shutdown_connections 2024-05-08T05:41:47.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.443+0000 7f2cb81f9640 1 -- 172.21.15.45:0/2089977991 wait complete. 2024-05-08T05:41:47.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.444+0000 7f2cb81f9640 1 Processor -- start 2024-05-08T05:41:47.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.444+0000 7f2cb81f9640 1 -- start start 2024-05-08T05:41:47.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2cb81f9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cb00d0700 0x7f2cb0161540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2cb81f9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cb015f450 0x7f2cb015b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2cb81f9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cb015f830 0x7f2cb0161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2cb81f9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb00d4c50 con 0x7f2cb015f830 2024-05-08T05:41:47.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2cb81f9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb00d4ad0 con 0x7f2cb00d0700 2024-05-08T05:41:47.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2cb81f9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb00d4dd0 con 0x7f2cb015f450 2024-05-08T05:41:47.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2cb6796640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cb015f830 0x7f2cb0161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2cb6796640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cb015f830 0x7f2cb0161eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51254/0 (socket says 172.21.15.45:51254) 2024-05-08T05:41:47.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2cb6796640 1 -- 172.21.15.45:0/2154500096 learned_addr learned my addr 172.21.15.45:0/2154500096 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:47.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2cb5794640 1 --2- 172.21.15.45:0/2154500096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cb015f450 0x7f2cb015b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2cb5f95640 1 --2- 172.21.15.45:0/2154500096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cb00d0700 0x7f2cb0161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2cb6796640 1 -- 172.21.15.45:0/2154500096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cb015f450 msgr2=0x7f2cb015b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2cb6796640 1 --2- 172.21.15.45:0/2154500096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cb015f450 0x7f2cb015b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2cb6796640 1 -- 172.21.15.45:0/2154500096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cb00d0700 msgr2=0x7f2cb0161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2cb6796640 1 --2- 172.21.15.45:0/2154500096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cb00d0700 0x7f2cb0161540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2cb6796640 1 -- 172.21.15.45:0/2154500096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb015b580 con 0x7f2cb015f830 2024-05-08T05:41:47.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2cb6796640 1 --2- 172.21.15.45:0/2154500096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cb015f830 0x7f2cb0161eb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800c6e0 tx=0x7f2ca800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2ca27fc640 1 -- 172.21.15.45:0/2154500096 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8018070 con 0x7f2cb015f830 2024-05-08T05:41:47.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2ca27fc640 1 -- 172.21.15.45:0/2154500096 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ca8004d60 con 0x7f2cb015f830 2024-05-08T05:41:47.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2cb81f9640 1 -- 172.21.15.45:0/2154500096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb01623f0 con 0x7f2cb015f830 2024-05-08T05:41:47.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2ca27fc640 1 -- 172.21.15.45:0/2154500096 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800a880 con 0x7f2cb015f830 2024-05-08T05:41:47.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.445+0000 7f2cb81f9640 1 -- 172.21.15.45:0/2154500096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb00cc360 con 0x7f2cb015f830 2024-05-08T05:41:47.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.446+0000 7f2cb81f9640 1 -- 172.21.15.45:0/2154500096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb0068c40 con 0x7f2cb015f830 2024-05-08T05:41:47.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.447+0000 7f2ca27fc640 1 -- 172.21.15.45:0/2154500096 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2ca801b070 con 0x7f2cb015f830 2024-05-08T05:41:47.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.448+0000 7f2ca27fc640 1 --2- 172.21.15.45:0/2154500096 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2c940762d0 0x7f2c94078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.448+0000 7f2ca27fc640 1 -- 172.21.15.45:0/2154500096 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2ca8096df0 con 0x7f2cb015f830 2024-05-08T05:41:47.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.448+0000 7f2cb5f95640 1 --2- 172.21.15.45:0/2154500096 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2c940762d0 0x7f2c94078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.448+0000 7f2cb5f95640 1 --2- 172.21.15.45:0/2154500096 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2c940762d0 0x7f2c94078790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2cac034000 tx=0x7f2cac02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.452+0000 7f2ca27fc640 1 -- 172.21.15.45:0/2154500096 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2cb0068c40 con 0x7f2cb015f830 2024-05-08T05:41:47.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.645+0000 7f2cb81f9640 1 -- 172.21.15.45:0/2154500096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2cb00cc170 con 0x7f2cb015f830 2024-05-08T05:41:47.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.646+0000 7f2ca27fc640 1 -- 172.21.15.45:0/2154500096 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18466 (secure 0 0 0) 0x7f2cb015b580 con 0x7f2cb015f830 2024-05-08T05:41:47.647 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:47.647 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:41:36.532045+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:41:29.381274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:41:30.443612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:41:34.132285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:41:47.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.648+0000 7f2cb81f9640 1 -- 172.21.15.45:0/2154500096 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2c940762d0 msgr2=0x7f2c94078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.648+0000 7f2cb81f9640 1 --2- 172.21.15.45:0/2154500096 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2c940762d0 0x7f2c94078790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2cac034000 tx=0x7f2cac02e000 comp rx=0 tx=0).stop 2024-05-08T05:41:47.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.648+0000 7f2cb81f9640 1 -- 172.21.15.45:0/2154500096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cb015f830 msgr2=0x7f2cb0161eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.648+0000 7f2cb81f9640 1 --2- 172.21.15.45:0/2154500096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cb015f830 0x7f2cb0161eb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800c6e0 tx=0x7f2ca800cba0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.648+0000 7f2cb81f9640 1 -- 172.21.15.45:0/2154500096 shutdown_connections 2024-05-08T05:41:47.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.648+0000 7f2cb81f9640 1 --2- 172.21.15.45:0/2154500096 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2c940762d0 0x7f2c94078790 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.648+0000 7f2cb81f9640 1 --2- 172.21.15.45:0/2154500096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cb015f450 0x7f2cb015b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.648+0000 7f2cb81f9640 1 --2- 172.21.15.45:0/2154500096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cb015f830 0x7f2cb0161eb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.648+0000 7f2cb81f9640 1 --2- 172.21.15.45:0/2154500096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cb00d0700 0x7f2cb0161540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.648+0000 7f2cb81f9640 1 -- 172.21.15.45:0/2154500096 >> 172.21.15.45:0/2154500096 conn(0x7f2cb00d5040 msgr2=0x7f2cb015e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:47.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.648+0000 7f2cb81f9640 1 -- 172.21.15.45:0/2154500096 shutdown_connections 2024-05-08T05:41:47.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.649+0000 7f2cb81f9640 1 -- 172.21.15.45:0/2154500096 wait complete. 2024-05-08T05:41:47.662 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-08T05:41:47.663 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-08T05:41:47.663 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-08T05:41:47.663 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-08T05:41:47.664 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-08T05:41:47.664 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-08T05:41:47.665 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-08T05:41:47.665 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-08T05:41:47.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.843+0000 7f48a3fff640 1 Processor -- start 2024-05-08T05:41:47.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.843+0000 7f48a3fff640 1 -- start start 2024-05-08T05:41:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.843+0000 7fc564a1d640 1 Processor -- start 2024-05-08T05:41:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.843+0000 7fc564a1d640 1 -- start start 2024-05-08T05:41:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.843+0000 7f823bed1640 1 Processor -- start 2024-05-08T05:41:47.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.843+0000 7f823bed1640 1 -- start start 2024-05-08T05:41:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.844+0000 7f48a3fff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f489c03afd0 0x7f489c007c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.844+0000 7f48a3fff640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f489c00cb30 con 0x7f489c0e0020 2024-05-08T05:41:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.844+0000 7f48a3fff640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f489c0565d0 con 0x7f489c00b1d0 2024-05-08T05:41:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.844+0000 7f48a3fff640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f489c0568b0 con 0x7f489c03afd0 2024-05-08T05:41:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.844+0000 7fc564a1d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc5600cff10 0x7fc56006f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.844+0000 7fc564a1d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc56006fba0 0x7fc560070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.844+0000 7f48a2ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f489c0e0020 0x7f489c008f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43944/0 (socket says 172.21.15.45:43944) 2024-05-08T05:41:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.844+0000 7f48a2ffd640 1 -- 172.21.15.45:0/2248297589 learned_addr learned my addr 172.21.15.45:0/2248297589 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.844+0000 7fc564a1d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc56006e8e0 con 0x7fc560068110 2024-05-08T05:41:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.844+0000 7fc564a1d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc56006eb80 con 0x7fc56006fba0 2024-05-08T05:41:47.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.844+0000 7fc564a1d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc56006ee60 con 0x7fc5600cff10 2024-05-08T05:41:47.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.845+0000 7f48a37fe640 1 --2- 172.21.15.45:0/2248297589 >> v2:172.21.15.63:3300/0 conn(0x7f489c03afd0 0x7f489c007c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.845+0000 7f823bed1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82340d1bb0 0x7f82340d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.845+0000 7f823bed1640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f82340b7e30 0x7f82340b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.845+0000 7f823bed1640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82340d2810 con 0x7f82340c8460 2024-05-08T05:41:47.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.845+0000 7f823bed1640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f82340d0a90 con 0x7f82340b7e30 2024-05-08T05:41:47.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.845+0000 7f823bed1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f82340d07b0 con 0x7f82340d1bb0 2024-05-08T05:41:47.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.845+0000 7f823a46e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f82340b7e30 0x7f82340b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.845+0000 7ff854ff0640 1 Processor -- start 2024-05-08T05:41:47.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.845+0000 7f48a37fe640 1 -- 172.21.15.45:0/2248297589 >> v1:172.21.15.63:6789/0 conn(0x7f489c00b1d0 legacy=0x7f489c03a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:47.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.845+0000 7f823946c640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f82340c8460 0x7f82340d1f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:53648/0 (socket says 172.21.15.45:53648) 2024-05-08T05:41:47.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.845+0000 7f823946c640 1 -- 172.21.15.45:0/1141753613 learned_addr learned my addr 172.21.15.45:0/1141753613 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:47.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.845+0000 7f823a46e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f82340b7e30 0x7f82340b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:47240/0 (socket says 172.21.15.45:47240) 2024-05-08T05:41:47.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.845+0000 7fc55ed9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc56006fba0 0x7fc560070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.845+0000 7fc55ed9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc56006fba0 0x7fc560070010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51260/0 (socket says 172.21.15.45:51260) 2024-05-08T05:41:47.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.845+0000 7fc55ed9d640 1 -- 172.21.15.45:0/1058018627 learned_addr learned my addr 172.21.15.45:0/1058018627 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:47.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.845+0000 7fc55e59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc560068110 0x7fc56015bde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43948/0 (socket says 172.21.15.45:43948) 2024-05-08T05:41:47.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.845+0000 7fc555d9b640 1 --2- 172.21.15.45:0/1058018627 >> v2:172.21.15.63:3300/0 conn(0x7fc5600cff10 0x7fc56006f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.846+0000 7ff854ff0640 1 -- start start 2024-05-08T05:41:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.846+0000 7fc55dd9b640 1 -- 172.21.15.45:0/1058018627 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2008973760 0 0) 0x7fc56006e8e0 con 0x7fc560068110 2024-05-08T05:41:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.846+0000 7fc55dd9b640 1 -- 172.21.15.45:0/1058018627 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc54c003520 con 0x7fc560068110 2024-05-08T05:41:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.846+0000 7fc55ed9d640 1 -- 172.21.15.45:0/1058018627 >> v2:172.21.15.63:3300/0 conn(0x7fc5600cff10 msgr2=0x7fc56006f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.846+0000 7fc55ed9d640 1 --2- 172.21.15.45:0/1058018627 >> v2:172.21.15.63:3300/0 conn(0x7fc5600cff10 0x7fc56006f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.846+0000 7fc55ed9d640 1 -- 172.21.15.45:0/1058018627 >> v1:172.21.15.45:6789/0 conn(0x7fc560068110 legacy=0x7fc56015bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.846+0000 7fc55ed9d640 1 -- 172.21.15.45:0/1058018627 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc56006f140 con 0x7fc56006fba0 2024-05-08T05:41:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.846+0000 7f8238c6b640 1 -- 172.21.15.45:0/1141753613 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2975898630 0 0) 0x7f82340d2810 con 0x7f82340c8460 2024-05-08T05:41:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.846+0000 7f823a46e640 1 -- 172.21.15.45:0/1141753613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82340d1bb0 msgr2=0x7f82340d0f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T05:41:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.846+0000 7f823a46e640 1 --2- 172.21.15.45:0/1141753613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82340d1bb0 0x7f82340d0f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.846+0000 7f823a46e640 1 -- 172.21.15.45:0/1141753613 >> v1:172.21.15.63:6789/0 conn(0x7f82340c8460 legacy=0x7f82340d1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.846+0000 7f823a46e640 1 -- 172.21.15.45:0/1141753613 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82340b64b0 con 0x7f82340b7e30 2024-05-08T05:41:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.847+0000 7f48a1ffb640 1 -- 172.21.15.45:0/2248297589 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1989907171 0 0) 0x7f4894001110 con 0x7f489c0e0020 2024-05-08T05:41:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.847+0000 7ff854ff0640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff850163920 0x7ff8500b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.847+0000 7f48a37fe640 1 -- 172.21.15.45:0/2248297589 >> v1:172.21.15.45:6789/0 conn(0x7f489c0e0020 legacy=0x7f489c008f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.847+0000 7f48a37fe640 1 -- 172.21.15.45:0/2248297589 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489c00cdb0 con 0x7f489c03afd0 2024-05-08T05:41:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.847+0000 7ff854ff0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff8500b4650 0x7ff8500c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.847+0000 7ff854ff0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8500b5560 0x7ff8500c6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.847+0000 7ff854ff0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff850163d00 con 0x7ff8500b4650 2024-05-08T05:41:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.847+0000 7ff854ff0640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff8500b23a0 con 0x7ff850163920 2024-05-08T05:41:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.847+0000 7ff854ff0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff8500b25f0 con 0x7ff8500b5560 2024-05-08T05:41:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.847+0000 7ff84e59c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff850163920 0x7ff8500b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.847+0000 7ff84e59c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff850163920 0x7ff8500b33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:47248/0 (socket says 172.21.15.45:47248) 2024-05-08T05:41:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.847+0000 7ff84e59c640 1 -- 172.21.15.45:0/1423360016 learned_addr learned my addr 172.21.15.45:0/1423360016 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.847+0000 7ff845d9b640 1 --2- 172.21.15.45:0/1423360016 >> v2:172.21.15.45:3300/0 conn(0x7ff8500b4650 0x7ff8500c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.847+0000 7fc55ed9d640 1 --2- 172.21.15.45:0/1058018627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56006fba0 0x7fc560070010 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc5600cf6c0 tx=0x7fc55802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e9e512ddd54485 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.847+0000 7ffb24ea0640 1 Processor -- start 2024-05-08T05:41:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.847+0000 7ffb24ea0640 1 -- start start 2024-05-08T05:41:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.848+0000 7ff84e59c640 1 -- 172.21.15.45:0/1423360016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8500b5560 msgr2=0x7ff8500c6440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T05:41:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.848+0000 7ff84e59c640 1 --2- 172.21.15.45:0/1423360016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8500b5560 0x7ff8500c6440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.848+0000 7ff84e59c640 1 -- 172.21.15.45:0/1423360016 >> v2:172.21.15.45:3300/0 conn(0x7ff8500b4650 msgr2=0x7ff8500c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.848+0000 7ff84e59c640 1 --2- 172.21.15.45:0/1423360016 >> v2:172.21.15.45:3300/0 conn(0x7ff8500b4650 0x7ff8500c5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.848+0000 7ff84e59c640 1 -- 172.21.15.45:0/1423360016 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8500b28d0 con 0x7ff850163920 2024-05-08T05:41:47.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.848+0000 7fc55dd9b640 1 -- 172.21.15.45:0/1058018627 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc558037070 con 0x7fc56006fba0 2024-05-08T05:41:47.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.848+0000 7fc55dd9b640 1 -- 172.21.15.45:0/1058018627 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc558032440 con 0x7fc56006fba0 2024-05-08T05:41:47.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.848+0000 7f48a37fe640 1 --2- 172.21.15.45:0/2248297589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f489c03afd0 0x7f489c007c10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f48a406b320 tx=0x7f48a40a6750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dee34d66be6600a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.848+0000 7fc55dd9b640 1 -- 172.21.15.45:0/1058018627 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc558032810 con 0x7fc56006fba0 2024-05-08T05:41:47.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.848+0000 7fe50dc64640 1 Processor -- start 2024-05-08T05:41:47.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.848+0000 7f823a46e640 1 --2- 172.21.15.45:0/1141753613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82340b7e30 0x7f82340b82c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f822c009d90 tx=0x7f822c02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2eab2b8114f3a06 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.848+0000 7fe50dc64640 1 -- start start 2024-05-08T05:41:47.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.848+0000 7f8238c6b640 1 -- 172.21.15.45:0/1141753613 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c031600 con 0x7f82340b7e30 2024-05-08T05:41:47.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.849+0000 7f8238c6b640 1 -- 172.21.15.45:0/1141753613 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f822c0317a0 con 0x7f82340b7e30 2024-05-08T05:41:47.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.848+0000 7f48a1ffb640 1 -- 172.21.15.45:0/2248297589 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a406e620 con 0x7f489c03afd0 2024-05-08T05:41:47.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.849+0000 7f48a1ffb640 1 -- 172.21.15.45:0/2248297589 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48a406e7c0 con 0x7f489c03afd0 2024-05-08T05:41:47.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.849+0000 7f48a1ffb640 1 -- 172.21.15.45:0/2248297589 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a406ebb0 con 0x7f489c03afd0 2024-05-08T05:41:47.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.849+0000 7ff84e59c640 1 --2- 172.21.15.45:0/1423360016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff850163920 0x7ff8500b33b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff8500c6a30 tx=0x7ff83802a1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92689c995a623298 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.849+0000 7fc564a1d640 1 -- 172.21.15.45:0/1058018627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56006fba0 msgr2=0x7fc560070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.849+0000 7fc564a1d640 1 --2- 172.21.15.45:0/1058018627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56006fba0 0x7fc560070010 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc5600cf6c0 tx=0x7fc55802a270 comp rx=0 tx=0).stop 2024-05-08T05:41:47.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.849+0000 7fc564a1d640 1 -- 172.21.15.45:0/1058018627 shutdown_connections 2024-05-08T05:41:47.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7fc564a1d640 1 --2- 172.21.15.45:0/1058018627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc56006fba0 0x7fc560070010 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7fc564a1d640 1 --2- 172.21.15.45:0/1058018627 >> v2:172.21.15.63:3300/0 conn(0x7fc5600cff10 0x7fc56006f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7fc564a1d640 1 -- 172.21.15.45:0/1058018627 >> 172.21.15.45:0/1058018627 conn(0x7fc5600738e0 msgr2=0x7fc56006c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:47.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7fc564a1d640 1 -- 172.21.15.45:0/1058018627 shutdown_connections 2024-05-08T05:41:47.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7fc564a1d640 1 -- 172.21.15.45:0/1058018627 wait complete. 2024-05-08T05:41:47.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.849+0000 7ffb24ea0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffb20163500 0x7ffb200af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.849+0000 7ffb24ea0640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ffb2006ff00 0x7ffb2006e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.849+0000 7ffb24ea0640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb200ae750 con 0x7ffb200d08b0 2024-05-08T05:41:47.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.849+0000 7ffb24ea0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ffb201638e0 con 0x7ffb20163500 2024-05-08T05:41:47.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.849+0000 7ffb24ea0640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ffb200ae9d0 con 0x7ffb2006ff00 2024-05-08T05:41:47.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.849+0000 7ffb15d9b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ffb2006ff00 0x7ffb2006e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.849+0000 7ffb15d9b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ffb2006ff00 0x7ffb2006e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:47264/0 (socket says 172.21.15.45:47264) 2024-05-08T05:41:47.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.849+0000 7ffb15d9b640 1 -- 172.21.15.45:0/778144143 learned_addr learned my addr 172.21.15.45:0/778144143 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:47.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.849+0000 7ffb1e59c640 1 --2- 172.21.15.45:0/778144143 >> v2:172.21.15.45:3300/0 conn(0x7ffb20163500 0x7ffb200af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7ffb1e59c640 1 -- 172.21.15.45:0/778144143 >> v2:172.21.15.63:3300/0 conn(0x7ffb2006ff00 msgr2=0x7ffb2006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:47.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7ffb1e59c640 1 --2- 172.21.15.45:0/778144143 >> v2:172.21.15.63:3300/0 conn(0x7ffb2006ff00 0x7ffb2006e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7ffb1e59c640 1 -- 172.21.15.45:0/778144143 >> v1:172.21.15.63:6789/0 conn(0x7ffb200d08b0 legacy=0x7ffb200b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:47.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7ffb1e59c640 1 -- 172.21.15.45:0/778144143 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb200aecb0 con 0x7ffb20163500 2024-05-08T05:41:47.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7ffb1e59c640 1 --2- 172.21.15.45:0/778144143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb20163500 0x7ffb200af790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb20070580 tx=0x7ffb0803b120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a69963c2a94a6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7f823bed1640 1 -- 172.21.15.45:0/1141753613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82340b7e30 msgr2=0x7f82340b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7f8238c6b640 1 -- 172.21.15.45:0/1141753613 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c031600 con 0x7f82340b7e30 2024-05-08T05:41:47.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7f823bed1640 1 --2- 172.21.15.45:0/1141753613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82340b7e30 0x7f82340b82c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f822c009d90 tx=0x7f822c02a5d0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7f823bed1640 1 -- 172.21.15.45:0/1141753613 shutdown_connections 2024-05-08T05:41:47.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7f823bed1640 1 --2- 172.21.15.45:0/1141753613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82340b7e30 0x7f82340b82c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7f823bed1640 1 --2- 172.21.15.45:0/1141753613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82340d1bb0 0x7f82340d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7f823bed1640 1 -- 172.21.15.45:0/1141753613 >> 172.21.15.45:0/1141753613 conn(0x7f82340d4fe0 msgr2=0x7f82340d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:47.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7f823bed1640 1 -- 172.21.15.45:0/1141753613 shutdown_connections 2024-05-08T05:41:47.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7f48a3fff640 1 -- 172.21.15.45:0/2248297589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f489c03afd0 msgr2=0x7f489c007c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7f48a3fff640 1 --2- 172.21.15.45:0/2248297589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f489c03afd0 0x7f489c007c10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f48a406b320 tx=0x7f48a40a6750 comp rx=0 tx=0).stop 2024-05-08T05:41:47.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7f48a3fff640 1 -- 172.21.15.45:0/2248297589 shutdown_connections 2024-05-08T05:41:47.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7f48a3fff640 1 --2- 172.21.15.45:0/2248297589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f489c03afd0 0x7f489c007c10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7f48a3fff640 1 -- 172.21.15.45:0/2248297589 >> 172.21.15.45:0/2248297589 conn(0x7f489c01ef70 msgr2=0x7f489c055c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:47.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7f48a3fff640 1 -- 172.21.15.45:0/2248297589 shutdown_connections 2024-05-08T05:41:47.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7f48a3fff640 1 -- 172.21.15.45:0/2248297589 wait complete. 2024-05-08T05:41:47.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.849+0000 7ff84dd9b640 1 -- 172.21.15.45:0/1423360016 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838037070 con 0x7ff850163920 2024-05-08T05:41:47.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.849+0000 7ff84dd9b640 1 -- 172.21.15.45:0/1423360016 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff838032440 con 0x7ff850163920 2024-05-08T05:41:47.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.849+0000 7ff84dd9b640 1 -- 172.21.15.45:0/1423360016 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838032810 con 0x7ff850163920 2024-05-08T05:41:47.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7ff854ff0640 1 -- 172.21.15.45:0/1423360016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff850163920 msgr2=0x7ff8500b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7ff854ff0640 1 --2- 172.21.15.45:0/1423360016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff850163920 0x7ff8500b33b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff8500c6a30 tx=0x7ff83802a1f0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7ff854ff0640 1 -- 172.21.15.45:0/1423360016 shutdown_connections 2024-05-08T05:41:47.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7ff854ff0640 1 --2- 172.21.15.45:0/1423360016 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8500b5560 0x7ff8500c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7ff854ff0640 1 --2- 172.21.15.45:0/1423360016 >> v2:172.21.15.45:3300/0 conn(0x7ff8500b4650 0x7ff8500c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7ff854ff0640 1 --2- 172.21.15.45:0/1423360016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff850163920 0x7ff8500b33b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7ff854ff0640 1 -- 172.21.15.45:0/1423360016 >> 172.21.15.45:0/1423360016 conn(0x7ff8500b4a40 msgr2=0x7ff8500b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:47.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7ff854ff0640 1 -- 172.21.15.45:0/1423360016 shutdown_connections 2024-05-08T05:41:47.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7ff854ff0640 1 -- 172.21.15.45:0/1423360016 wait complete. 2024-05-08T05:41:47.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7fe50dc64640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe50815f2d0 0x7fe5080b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7fe50dc64640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5080d2310 0x7fe5080d5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7fe506ffd640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fe5080d4fc0 0x7fe5080d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:53680/0 (socket says 172.21.15.45:53680) 2024-05-08T05:41:47.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7fe5077fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe50815f2d0 0x7fe5080b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7fe5077fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe50815f2d0 0x7fe5080b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:33144/0 (socket says 172.21.15.45:33144) 2024-05-08T05:41:47.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7fe506ffd640 1 -- 172.21.15.45:0/1946727761 learned_addr learned my addr 172.21.15.45:0/1946727761 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:47.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7fe50dc64640 1 -- 172.21.15.45:0/1946727761 --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5081606c0 con 0x7fe5080d4fc0 2024-05-08T05:41:47.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7fe50dc64640 1 -- 172.21.15.45:0/1946727761 --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe5080d5960 con 0x7fe5080d2310 2024-05-08T05:41:47.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7fe507fff640 1 --2- 172.21.15.45:0/1946727761 >> v2:172.21.15.45:3300/0 conn(0x7fe5080d2310 0x7fe5080d5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7fe50dc64640 1 -- 172.21.15.45:0/1946727761 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe5080b5810 con 0x7fe50815f2d0 2024-05-08T05:41:47.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7fe5067fc640 1 -- 172.21.15.45:0/1946727761 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2151786228 0 0) 0x7fe5081606c0 con 0x7fe5080d4fc0 2024-05-08T05:41:47.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7fe5067fc640 1 -- 172.21.15.45:0/1946727761 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4f0003520 con 0x7fe5080d4fc0 2024-05-08T05:41:47.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe5067fc640 1 -- 172.21.15.45:0/1946727761 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 869487884 0 0) 0x7fe4f0003520 con 0x7fe5080d4fc0 2024-05-08T05:41:47.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe5067fc640 1 -- 172.21.15.45:0/1946727761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe50815f2d0 msgr2=0x7fe5080b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:47.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe5067fc640 1 --2- 172.21.15.45:0/1946727761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe50815f2d0 0x7fe5080b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe5067fc640 1 -- 172.21.15.45:0/1946727761 >> v2:172.21.15.45:3300/0 conn(0x7fe5080d2310 msgr2=0x7fe5080d5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:47.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe5077fe640 1 --2- 172.21.15.45:0/1946727761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe50815f2d0 0x7fe5080b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:47.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe5067fc640 1 --2- 172.21.15.45:0/1946727761 >> v2:172.21.15.45:3300/0 conn(0x7fe5080d2310 0x7fe5080d5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe5067fc640 1 -- 172.21.15.45:0/1946727761 --> v1:172.21.15.63:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5080b4f00 con 0x7fe5080d4fc0 2024-05-08T05:41:47.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe5067fc640 1 -- 172.21.15.45:0/1946727761 <== mon.1 v1:172.21.15.63:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fe4f80016f0 con 0x7fe5080d4fc0 2024-05-08T05:41:47.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe5067fc640 1 -- 172.21.15.45:0/1946727761 >> v1:172.21.15.63:6789/0 conn(0x7fe5080d4fc0 legacy=0x7fe5080d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe5067fc640 1 --2- 172.21.15.45:0/1946727761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe50815f2d0 0x7fe4f0003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe5067fc640 1 --2- 172.21.15.45:0/1946727761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4f0004030 0x7fe4f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe5067fc640 1 --2- 172.21.15.45:0/1946727761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4f0004d40 0x7fe4f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe5067fc640 1 -- 172.21.15.45:0/1946727761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe5080b5810 con 0x7fe4f0004030 2024-05-08T05:41:47.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe5067fc640 1 -- 172.21.15.45:0/1946727761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f000baa0 con 0x7fe4f0004d40 2024-05-08T05:41:47.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe5067fc640 1 -- 172.21.15.45:0/1946727761 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe4f000bca0 con 0x7fe50815f2d0 2024-05-08T05:41:47.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe507fff640 1 --2- 172.21.15.45:0/1946727761 >> v2:172.21.15.45:3300/0 conn(0x7fe5080d2310 0x7fe5080d5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:47.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe506ffd640 1 --2- 172.21.15.45:0/1946727761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4f0004030 0x7fe4f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe506ffd640 1 -- 172.21.15.45:0/1946727761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe50815f2d0 msgr2=0x7fe4f0003aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T05:41:47.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe506ffd640 1 --2- 172.21.15.45:0/1946727761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe50815f2d0 0x7fe4f0003aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe506ffd640 1 -- 172.21.15.45:0/1946727761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4f0004d40 msgr2=0x7fe4f00051c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T05:41:47.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe506ffd640 1 --2- 172.21.15.45:0/1946727761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4f0004d40 0x7fe4f00051c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.851+0000 7fe506ffd640 1 -- 172.21.15.45:0/1946727761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f0003520 con 0x7fe4f0004030 2024-05-08T05:41:47.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.850+0000 7f823bed1640 1 -- 172.21.15.45:0/1141753613 wait complete. 2024-05-08T05:41:47.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.852+0000 7f823bed1640 1 Processor -- start 2024-05-08T05:41:47.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.852+0000 7ffb1dd9b640 1 -- 172.21.15.45:0/778144143 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08048070 con 0x7ffb20163500 2024-05-08T05:41:47.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.852+0000 7ffb1dd9b640 1 -- 172.21.15.45:0/778144143 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb08043440 con 0x7ffb20163500 2024-05-08T05:41:47.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.852+0000 7f48a3fff640 1 Processor -- start 2024-05-08T05:41:47.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.852+0000 7ffb1dd9b640 1 -- 172.21.15.45:0/778144143 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08043810 con 0x7ffb20163500 2024-05-08T05:41:47.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.852+0000 7f823bed1640 1 -- start start 2024-05-08T05:41:47.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.852+0000 7f48a3fff640 1 -- start start 2024-05-08T05:41:47.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.853+0000 7fc564a1d640 1 Processor -- start 2024-05-08T05:41:47.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.853+0000 7fc564a1d640 1 -- start start 2024-05-08T05:41:47.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.853+0000 7f3f8e9e4640 1 Processor -- start 2024-05-08T05:41:47.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.853+0000 7fe506ffd640 1 --2- 172.21.15.45:0/1946727761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4f0004030 0x7fe4f0004800 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800c190 tx=0x7fe4f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c8da6311d076c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.853+0000 7f3f8e9e4640 1 -- start start 2024-05-08T05:41:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.853+0000 7f823bed1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82340b7e30 0x7f8234160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.853+0000 7f823bed1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82340c8460 0x7f8234165830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.853+0000 7f823bed1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82340d1bb0 0x7f82340c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.853+0000 7f823bed1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82340b6020 con 0x7f82340b7e30 2024-05-08T05:41:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.853+0000 7f823bed1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82340b5ea0 con 0x7f82340d1bb0 2024-05-08T05:41:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.853+0000 7f823bed1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f82340b8800 con 0x7f82340c8460 2024-05-08T05:41:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.853+0000 7ffb24ea0640 1 -- 172.21.15.45:0/778144143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb20163500 msgr2=0x7ffb200af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.853+0000 7ffb24ea0640 1 --2- 172.21.15.45:0/778144143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb20163500 0x7ffb200af790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb20070580 tx=0x7ffb0803b120 comp rx=0 tx=0).stop 2024-05-08T05:41:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.853+0000 7ffb24ea0640 1 -- 172.21.15.45:0/778144143 shutdown_connections 2024-05-08T05:41:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.853+0000 7ffb24ea0640 1 --2- 172.21.15.45:0/778144143 >> v2:172.21.15.63:3300/0 conn(0x7ffb2006ff00 0x7ffb2006e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.853+0000 7ffb24ea0640 1 --2- 172.21.15.45:0/778144143 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb20163500 0x7ffb200af790 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb20070580 tx=0x7ffb0803b120 comp rx=0 tx=0).stop 2024-05-08T05:41:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.853+0000 7ffb24ea0640 1 -- 172.21.15.45:0/778144143 >> 172.21.15.45:0/778144143 conn(0x7ffb2006c3a0 msgr2=0x7ffb2006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.853+0000 7f8239c6d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82340b7e30 0x7f8234160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.853+0000 7fe5067fc640 1 -- 172.21.15.45:0/1946727761 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8007cc0 con 0x7fe4f0004030 2024-05-08T05:41:47.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.853+0000 7f823a46e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82340d1bb0 0x7f82340c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f823a46e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82340d1bb0 0x7f82340c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:47266/0 (socket says 172.21.15.45:47266) 2024-05-08T05:41:47.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f823a46e640 1 -- 172.21.15.45:0/2602459025 learned_addr learned my addr 172.21.15.45:0/2602459025 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:47.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7fe5067fc640 1 -- 172.21.15.45:0/1946727761 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4f8005af0 con 0x7fe4f0004030 2024-05-08T05:41:47.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f48a3fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f489c00b1d0 0x7f489c0d4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7fe5067fc640 1 -- 172.21.15.45:0/1946727761 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8007070 con 0x7fe4f0004030 2024-05-08T05:41:47.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f48a3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f489c03afd0 0x7f489c048110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f48a2ffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f489c00b1d0 0x7f489c0d4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f48a3fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f489c0e0020 0x7f489c048650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f48a2ffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f489c00b1d0 0x7f489c0d4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:33166/0 (socket says 172.21.15.45:33166) 2024-05-08T05:41:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f823a46e640 1 -- 172.21.15.45:0/2602459025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82340c8460 msgr2=0x7f8234165830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f823a46e640 1 --2- 172.21.15.45:0/2602459025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82340c8460 0x7f8234165830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f823a46e640 1 -- 172.21.15.45:0/2602459025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82340b7e30 msgr2=0x7f8234160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f823a46e640 1 --2- 172.21.15.45:0/2602459025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82340b7e30 0x7f8234160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f823a46e640 1 -- 172.21.15.45:0/2602459025 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82341661d0 con 0x7f82340d1bb0 2024-05-08T05:41:47.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7fc564a1d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc560068110 0x7fc560146fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f3f8e9e4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3f880d0fc0 0x7f3f880af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f3f8e9e4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f880ae750 con 0x7f3f8806ff00 2024-05-08T05:41:47.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f3f8e9e4640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f881638e0 con 0x7f3f88163500 2024-05-08T05:41:47.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f3f8e9e4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3f880ae9d0 con 0x7f3f880d0fc0 2024-05-08T05:41:47.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f48a3fff640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f489c0e0400 con 0x7f489c03afd0 2024-05-08T05:41:47.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f48a3fff640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f489c0534a0 con 0x7f489c0e0020 2024-05-08T05:41:47.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f48a3fff640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f489c03b610 con 0x7f489c00b1d0 2024-05-08T05:41:47.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f48a27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f489c03afd0 0x7f489c048110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f48a27fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f489c03afd0 0x7f489c048110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51324/0 (socket says 172.21.15.45:51324) 2024-05-08T05:41:47.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f48a27fc640 1 -- 172.21.15.45:0/95129841 learned_addr learned my addr 172.21.15.45:0/95129841 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:47.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7ff854ff0640 1 Processor -- start 2024-05-08T05:41:47.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7ff854ff0640 1 -- start start 2024-05-08T05:41:47.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7fc564a1d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc56006fba0 0x7fc560154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f823a46e640 1 --2- 172.21.15.45:0/2602459025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82340d1bb0 0x7f82340c8fd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f822c009d90 tx=0x7f822c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f3f87fff640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f3f88163500 0x7f3f8806e0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:53688/0 (socket says 172.21.15.45:53688) 2024-05-08T05:41:47.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f3f87fff640 1 -- 172.21.15.45:0/3087507190 learned_addr learned my addr 172.21.15.45:0/3087507190 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:47.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.854+0000 7f48a37fe640 1 --2- 172.21.15.45:0/95129841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f489c0e0020 0x7f489c048650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7f3f8cf81640 1 --2- 172.21.15.45:0/3087507190 >> v2:172.21.15.63:3300/0 conn(0x7f3f880d0fc0 0x7f3f880af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7fc564a1d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5600cff10 0x7fc560154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7ffb24ea0640 1 -- 172.21.15.45:0/778144143 shutdown_connections 2024-05-08T05:41:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7ffb24ea0640 1 -- 172.21.15.45:0/778144143 wait complete. 2024-05-08T05:41:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7fc555d9b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc56006fba0 0x7fc560154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7fc555d9b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc56006fba0 0x7fc560154aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:33170/0 (socket says 172.21.15.45:33170) 2024-05-08T05:41:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7f48a27fc640 1 -- 172.21.15.45:0/95129841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f489c00b1d0 msgr2=0x7f489c0d4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7f48a27fc640 1 --2- 172.21.15.45:0/95129841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f489c00b1d0 0x7f489c0d4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7f48a27fc640 1 -- 172.21.15.45:0/95129841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f489c0e0020 msgr2=0x7f489c048650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7f48a27fc640 1 --2- 172.21.15.45:0/95129841 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f489c0e0020 0x7f489c048650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7f48a27fc640 1 -- 172.21.15.45:0/95129841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489c0d2090 con 0x7f489c03afd0 2024-05-08T05:41:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7fc55e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc560068110 0x7fc560146fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7fc55e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc560068110 0x7fc560146fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51330/0 (socket says 172.21.15.45:51330) 2024-05-08T05:41:47.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7f3f8cf81640 1 -- 172.21.15.45:0/3087507190 >> v1:172.21.15.63:6789/0 conn(0x7f3f88163500 legacy=0x7f3f8806e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7f3f8cf81640 1 -- 172.21.15.45:0/3087507190 >> v1:172.21.15.45:6789/0 conn(0x7f3f8806ff00 legacy=0x7f3f880d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:47.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7f3f8cf81640 1 -- 172.21.15.45:0/3087507190 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f880aecb0 con 0x7f3f880d0fc0 2024-05-08T05:41:47.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7f822a7fc640 1 -- 172.21.15.45:0/2602459025 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c040070 con 0x7f82340d1bb0 2024-05-08T05:41:47.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7f822a7fc640 1 -- 172.21.15.45:0/2602459025 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f822c004240 con 0x7f82340d1bb0 2024-05-08T05:41:47.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7f822a7fc640 1 -- 172.21.15.45:0/2602459025 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c009450 con 0x7f82340d1bb0 2024-05-08T05:41:47.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7f823bed1640 1 -- 172.21.15.45:0/2602459025 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82341664b0 con 0x7f82340d1bb0 2024-05-08T05:41:47.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7ff854ff0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8500b5560 0x7ff8501f9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7ff854ff0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff850163920 0x7ff85015fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7fc555d9b640 1 -- 172.21.15.45:0/67583542 learned_addr learned my addr 172.21.15.45:0/67583542 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:47.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7fc564a1d640 1 -- 172.21.15.45:0/67583542 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc5600d0f30 con 0x7fc560068110 2024-05-08T05:41:47.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7fe50dc64640 1 -- 172.21.15.45:0/1946727761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4f0004030 msgr2=0x7fe4f0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7fe50dc64640 1 --2- 172.21.15.45:0/1946727761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4f0004030 0x7fe4f0004800 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe4f800c190 tx=0x7fe4f0005830 comp rx=0 tx=0).stop 2024-05-08T05:41:47.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7fe507fff640 1 -- 172.21.15.45:0/1946727761 reap_dead start 2024-05-08T05:41:47.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7fe2dbb22640 1 Processor -- start 2024-05-08T05:41:47.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7fe50dc64640 1 -- 172.21.15.45:0/1946727761 shutdown_connections 2024-05-08T05:41:47.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7fe50dc64640 1 -- 172.21.15.45:0/1946727761 >> 172.21.15.45:0/1946727761 conn(0x7fe5080746d0 msgr2=0x7fe50809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:47.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7ff854ff0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8500d3460 0x7ff8500d3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.855+0000 7f48a27fc640 1 --2- 172.21.15.45:0/95129841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f489c03afd0 0x7f489c048110 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f489800c430 tx=0x7f489800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7fc55ed9d640 1 --2- 172.21.15.45:0/67583542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5600cff10 0x7fc560154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7fe50dc64640 1 -- 172.21.15.45:0/1946727761 shutdown_connections 2024-05-08T05:41:47.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7fc564a1d640 1 -- 172.21.15.45:0/67583542 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc56015c4f0 con 0x7fc5600cff10 2024-05-08T05:41:47.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7fe50dc64640 1 -- 172.21.15.45:0/1946727761 wait complete. 2024-05-08T05:41:47.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7fc564a1d640 1 -- 172.21.15.45:0/67583542 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc560070550 con 0x7fc56006fba0 2024-05-08T05:41:47.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7ff845d9b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff850163920 0x7ff85015fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7ff845d9b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff850163920 0x7ff85015fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:47282/0 (socket says 172.21.15.45:47282) 2024-05-08T05:41:47.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7ff84e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8500b5560 0x7ff8501f9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7ff84e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8500b5560 0x7ff8501f9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51332/0 (socket says 172.21.15.45:51332) 2024-05-08T05:41:47.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7ff84ed9d640 1 --2- 172.21.15.45:0/333134463 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8500d3460 0x7ff8500d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7ff845d9b640 1 -- 172.21.15.45:0/333134463 learned_addr learned my addr 172.21.15.45:0/333134463 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:47.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7ff854ff0640 1 -- 172.21.15.45:0/333134463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff8500b38f0 con 0x7ff8500b5560 2024-05-08T05:41:47.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7ff854ff0640 1 -- 172.21.15.45:0/333134463 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff8500b2e90 con 0x7ff850163920 2024-05-08T05:41:47.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7ff854ff0640 1 -- 172.21.15.45:0/333134463 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff8500b3be0 con 0x7ff8500d3460 2024-05-08T05:41:47.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7fe2dbb22640 1 -- start start 2024-05-08T05:41:47.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7f488bfff640 1 -- 172.21.15.45:0/95129841 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898011790 con 0x7f489c03afd0 2024-05-08T05:41:47.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7f823bed1640 1 -- 172.21.15.45:0/2602459025 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8234166a70 con 0x7f82340d1bb0 2024-05-08T05:41:47.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7fc555d9b640 1 -- 172.21.15.45:0/67583542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5600cff10 msgr2=0x7fc560154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7fc555d9b640 1 --2- 172.21.15.45:0/67583542 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5600cff10 0x7fc560154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7fc555d9b640 1 -- 172.21.15.45:0/67583542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc560068110 msgr2=0x7fc560146fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.857+0000 7ff84ed9d640 1 -- 172.21.15.45:0/333134463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff850163920 msgr2=0x7ff85015fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.857+0000 7ff84ed9d640 1 --2- 172.21.15.45:0/333134463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff850163920 0x7ff85015fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.857+0000 7ff84ed9d640 1 -- 172.21.15.45:0/333134463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8500b5560 msgr2=0x7ff8501f9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.857+0000 7ff84ed9d640 1 --2- 172.21.15.45:0/333134463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8500b5560 0x7ff8501f9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.857+0000 7ff84ed9d640 1 -- 172.21.15.45:0/333134463 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8501f9c40 con 0x7ff8500d3460 2024-05-08T05:41:47.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7f488bfff640 1 -- 172.21.15.45:0/95129841 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4898017070 con 0x7f489c03afd0 2024-05-08T05:41:47.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7f488bfff640 1 -- 172.21.15.45:0/95129841 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898011bb0 con 0x7f489c03afd0 2024-05-08T05:41:47.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7f48a3fff640 1 -- 172.21.15.45:0/95129841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489c0d15e0 con 0x7f489c03afd0 2024-05-08T05:41:47.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.856+0000 7f48a3fff640 1 -- 172.21.15.45:0/95129841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f489c037340 con 0x7f489c03afd0 2024-05-08T05:41:47.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.857+0000 7ff84e59c640 1 --2- 172.21.15.45:0/333134463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8500b5560 0x7ff8501f9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:47.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.857+0000 7ff84ed9d640 1 --2- 172.21.15.45:0/333134463 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8500d3460 0x7ff8500d3910 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff8500b5990 tx=0x7ff84800b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.857+0000 7ffb24ea0640 1 Processor -- start 2024-05-08T05:41:47.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.857+0000 7ff845d9b640 1 --2- 172.21.15.45:0/333134463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff850163920 0x7ff85015fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:47.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.857+0000 7ffb24ea0640 1 -- start start 2024-05-08T05:41:47.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.857+0000 7fe2dbb22640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe2d40b33f0 0x7fe2d40ae030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.857+0000 7fe2dbb22640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe2d4163af0 0x7fe2d40ae570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.857+0000 7fe2dbb22640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2d406e730 con 0x7fe2d4163710 2024-05-08T05:41:47.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7fe2dbb22640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe2d40afd70 con 0x7fe2d4163af0 2024-05-08T05:41:47.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7fe2dbb22640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe2d40afa90 con 0x7fe2d40b33f0 2024-05-08T05:41:47.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7fc555d9b640 1 --2- 172.21.15.45:0/67583542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc560068110 0x7fc560146fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7fc555d9b640 1 -- 172.21.15.45:0/67583542 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5601467c0 con 0x7fc56006fba0 2024-05-08T05:41:47.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7fe2da0bf640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe2d4163af0 0x7fe2d40ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7fe2da0bf640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe2d4163af0 0x7fe2d40ae570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:47296/0 (socket says 172.21.15.45:47296) 2024-05-08T05:41:47.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7fe2da0bf640 1 -- 172.21.15.45:0/1349740447 learned_addr learned my addr 172.21.15.45:0/1349740447 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:47.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7fc55e59c640 1 --2- 172.21.15.45:0/67583542 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc560068110 0x7fc560146fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:47.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7fe2d90bd640 1 --2- 172.21.15.45:0/1349740447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe2d40b33f0 0x7fe2d40ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7ff8477fe640 1 -- 172.21.15.45:0/333134463 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848015020 con 0x7ff8500d3460 2024-05-08T05:41:47.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7f48a3fff640 1 -- 172.21.15.45:0/95129841 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f489c005710 con 0x7f489c03afd0 2024-05-08T05:41:47.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7f3f8cf81640 1 --2- 172.21.15.45:0/3087507190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f880d0fc0 0x7f3f880af790 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009d90 tx=0x7f3f8002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d97bb45d2b1a64d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7fe50dc64640 1 Processor -- start 2024-05-08T05:41:47.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7fc555d9b640 1 --2- 172.21.15.45:0/67583542 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc56006fba0 0x7fc560154aa0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc54800e8a0 tx=0x7fc54800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7f8217fff640 1 -- 172.21.15.45:0/2602459025 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8210001030 con 0x7f82340d1bb0 2024-05-08T05:41:47.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7ff854ff0640 1 -- 172.21.15.45:0/333134463 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8500c6b10 con 0x7ff8500d3460 2024-05-08T05:41:47.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7ff854ff0640 1 -- 172.21.15.45:0/333134463 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8500c70d0 con 0x7ff8500d3460 2024-05-08T05:41:47.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7ff8477fe640 1 -- 172.21.15.45:0/333134463 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff848011830 con 0x7ff8500d3460 2024-05-08T05:41:47.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7fe50dc64640 1 -- start start 2024-05-08T05:41:47.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7fe2da0bf640 1 -- 172.21.15.45:0/1349740447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe2d40b33f0 msgr2=0x7fe2d40ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:47.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7fe2da0bf640 1 --2- 172.21.15.45:0/1349740447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe2d40b33f0 0x7fe2d40ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7fe2da0bf640 1 -- 172.21.15.45:0/1349740447 >> v1:172.21.15.63:6789/0 conn(0x7fe2d4163710 legacy=0x7fe2d40b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7fe2da0bf640 1 -- 172.21.15.45:0/1349740447 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d406e9d0 con 0x7fe2d4163af0 2024-05-08T05:41:47.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7ffb24ea0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb2006ff00 0x7ffb2009e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7ffb24ea0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb200d08b0 0x7ffb20150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.858+0000 7ffb24ea0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb201649e0 0x7ffb20164e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.859+0000 7ffb24ea0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2006e630 con 0x7ffb2006ff00 2024-05-08T05:41:47.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.859+0000 7ffb24ea0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffb200af270 con 0x7ffb201649e0 2024-05-08T05:41:47.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.859+0000 7ffb24ea0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffb2006e7b0 con 0x7ffb200d08b0 2024-05-08T05:41:47.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.859+0000 7ffb1e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb2006ff00 0x7ffb2009e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.859+0000 7ffb1e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb2006ff00 0x7ffb2009e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51334/0 (socket says 172.21.15.45:51334) 2024-05-08T05:41:47.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.859+0000 7ffb1e59c640 1 -- 172.21.15.45:0/1232729636 learned_addr learned my addr 172.21.15.45:0/1232729636 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:47.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.859+0000 7fc5577fe640 1 -- 172.21.15.45:0/67583542 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc548017070 con 0x7fc56006fba0 2024-05-08T05:41:47.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.859+0000 7fc5577fe640 1 -- 172.21.15.45:0/67583542 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc54800c940 con 0x7fc56006fba0 2024-05-08T05:41:47.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.859+0000 7fc5577fe640 1 -- 172.21.15.45:0/67583542 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54800cd30 con 0x7fc56006fba0 2024-05-08T05:41:47.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.860+0000 7fc564a1d640 1 -- 172.21.15.45:0/67583542 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc560145720 con 0x7fc56006fba0 2024-05-08T05:41:47.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.860+0000 7f3f86ffd640 1 -- 172.21.15.45:0/3087507190 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80007740 con 0x7f3f880d0fc0 2024-05-08T05:41:47.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.860+0000 7f3f86ffd640 1 -- 172.21.15.45:0/3087507190 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f800078e0 con 0x7f3f880d0fc0 2024-05-08T05:41:47.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.860+0000 7fc564a1d640 1 -- 172.21.15.45:0/67583542 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc560145ce0 con 0x7fc56006fba0 2024-05-08T05:41:47.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.859+0000 7ffb1ed9d640 1 --2- 172.21.15.45:0/1232729636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb201649e0 0x7ffb20164e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.860+0000 7ffb15d9b640 1 --2- 172.21.15.45:0/1232729636 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb200d08b0 0x7ffb20150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.860+0000 7ffb1e59c640 1 -- 172.21.15.45:0/1232729636 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb200d08b0 msgr2=0x7ffb20150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.860+0000 7ffb1e59c640 1 --2- 172.21.15.45:0/1232729636 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb200d08b0 0x7ffb20150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.860+0000 7ffb1e59c640 1 -- 172.21.15.45:0/1232729636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb201649e0 msgr2=0x7ffb20164e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.860+0000 7ffb1e59c640 1 --2- 172.21.15.45:0/1232729636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb201649e0 0x7ffb20164e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.860+0000 7ffb1e59c640 1 -- 172.21.15.45:0/1232729636 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2009eb80 con 0x7ffb2006ff00 2024-05-08T05:41:47.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.860+0000 7ffb15d9b640 1 --2- 172.21.15.45:0/1232729636 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb200d08b0 0x7ffb20150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:47.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.860+0000 7ffb1e59c640 1 --2- 172.21.15.45:0/1232729636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb2006ff00 0x7ffb2009e640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffb0803b710 tx=0x7ffb080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.860+0000 7f822a7fc640 1 -- 172.21.15.45:0/2602459025 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f822c039040 con 0x7f82340d1bb0 2024-05-08T05:41:47.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.860+0000 7ffb177fe640 1 -- 172.21.15.45:0/1232729636 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08051020 con 0x7ffb2006ff00 2024-05-08T05:41:47.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7ffb24ea0640 1 -- 172.21.15.45:0/1232729636 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb20163af0 con 0x7ffb2006ff00 2024-05-08T05:41:47.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7f3f8e9e4640 1 -- 172.21.15.45:0/3087507190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f880d0fc0 msgr2=0x7f3f880af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7f3f8e9e4640 1 --2- 172.21.15.45:0/3087507190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f880d0fc0 0x7f3f880af790 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009d90 tx=0x7f3f8002a660 comp rx=0 tx=0).stop 2024-05-08T05:41:47.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7f3f8e9e4640 1 -- 172.21.15.45:0/3087507190 shutdown_connections 2024-05-08T05:41:47.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7f3f8e9e4640 1 --2- 172.21.15.45:0/3087507190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f880d0fc0 0x7f3f880af790 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7f3f8e9e4640 1 -- 172.21.15.45:0/3087507190 >> 172.21.15.45:0/3087507190 conn(0x7f3f8806c3a0 msgr2=0x7f3f8806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:47.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.859+0000 7ff8477fe640 1 -- 172.21.15.45:0/333134463 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848011c20 con 0x7ff8500d3460 2024-05-08T05:41:47.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7fe50dc64640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4f0004030 0x7fe50815a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7fe50dc64640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5080cb310 0x7fe5081e9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7fe2da0bf640 1 --2- 172.21.15.45:0/1349740447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe2d4163af0 0x7fe2d40ae570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009e50 tx=0x7fe2cc02a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d0fe6d529b86a17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7ffb24ea0640 1 -- 172.21.15.45:0/1232729636 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb201640b0 con 0x7ffb2006ff00 2024-05-08T05:41:47.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7fe5077fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4f0004030 0x7fe50815a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7fe5077fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4f0004030 0x7fe50815a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:47312/0 (socket says 172.21.15.45:47312) 2024-05-08T05:41:47.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7f822a7fc640 1 --2- 172.21.15.45:0/2602459025 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8218076350 0x7f8218078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7ff8477fe640 1 -- 172.21.15.45:0/333134463 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff848018020 con 0x7ff8500d3460 2024-05-08T05:41:47.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7ff854ff0640 1 -- 172.21.15.45:0/333134463 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff834001030 con 0x7ff8500d3460 2024-05-08T05:41:47.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7fe50dc64640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5080c9d00 0x7fe5080cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7fc564a1d640 1 -- 172.21.15.45:0/67583542 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc560069b00 con 0x7fc56006fba0 2024-05-08T05:41:47.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7fe50dc64640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe5080b5ca0 con 0x7fe5080cb310 2024-05-08T05:41:47.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7f822a7fc640 1 -- 172.21.15.45:0/2602459025 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f822c0b92c0 con 0x7f82340d1bb0 2024-05-08T05:41:47.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7fe50dc64640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe5080b54a0 con 0x7fe4f0004030 2024-05-08T05:41:47.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7fe50dc64640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe5080b5e20 con 0x7fe5080c9d00 2024-05-08T05:41:47.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7fe5077fe640 1 -- 172.21.15.45:0/3361262883 learned_addr learned my addr 172.21.15.45:0/3361262883 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:47.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7f822a7fc640 1 --2- 172.21.15.45:0/2602459025 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x7f821807fac0 0x7f8218081f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7f822a7fc640 1 -- 172.21.15.45:0/2602459025 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f82180825d0 con 0x7f821807fac0 2024-05-08T05:41:47.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7f822a7fc640 1 -- 172.21.15.45:0/2602459025 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f822c0b9670 con 0x7f82340d1bb0 2024-05-08T05:41:47.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7f3f8e9e4640 1 -- 172.21.15.45:0/3087507190 shutdown_connections 2024-05-08T05:41:47.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7f3f8e9e4640 1 -- 172.21.15.45:0/3087507190 wait complete. 2024-05-08T05:41:47.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.862+0000 7fe2d88bc640 1 -- 172.21.15.45:0/1349740447 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc036070 con 0x7fe2d4163af0 2024-05-08T05:41:47.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.862+0000 7fe2d88bc640 1 -- 172.21.15.45:0/1349740447 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2cc031600 con 0x7fe2d4163af0 2024-05-08T05:41:47.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.862+0000 7fe2d88bc640 1 -- 172.21.15.45:0/1349740447 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc0319d0 con 0x7fe2d4163af0 2024-05-08T05:41:47.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.862+0000 7fe507fff640 1 --2- 172.21.15.45:0/3361262883 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5080c9d00 0x7fe5080cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.862+0000 7ff8477fe640 1 --2- 172.21.15.45:0/333134463 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff830076320 0x7ff8300787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.862+0000 7fe5077fe640 1 -- 172.21.15.45:0/3361262883 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5080c9d00 msgr2=0x7fe5080cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7ffb177fe640 1 -- 172.21.15.45:0/1232729636 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb08049040 con 0x7ffb2006ff00 2024-05-08T05:41:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7ffb177fe640 1 -- 172.21.15.45:0/1232729636 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08048070 con 0x7ffb2006ff00 2024-05-08T05:41:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.862+0000 7fe5077fe640 1 --2- 172.21.15.45:0/3361262883 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5080c9d00 0x7fe5080cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.862+0000 7fe5077fe640 1 -- 172.21.15.45:0/3361262883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5080cb310 msgr2=0x7fe5081e9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.862+0000 7fe5077fe640 1 --2- 172.21.15.45:0/3361262883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5080cb310 0x7fe5081e9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.861+0000 7f488bfff640 1 -- 172.21.15.45:0/95129841 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f489801c460 con 0x7f489c03afd0 2024-05-08T05:41:47.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.862+0000 7fe5077fe640 1 -- 172.21.15.45:0/3361262883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50815a730 con 0x7fe4f0004030 2024-05-08T05:41:47.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.862+0000 7ffb14d99640 1 -- 172.21.15.45:0/1232729636 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffaf8001030 con 0x7ffb2006ff00 2024-05-08T05:41:47.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.862+0000 7fe507fff640 1 --2- 172.21.15.45:0/3361262883 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5080c9d00 0x7fe5080cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:47.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.862+0000 7ff84e59c640 1 --2- 172.21.15.45:0/333134463 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff830076320 0x7ff8300787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.862+0000 7ff8477fe640 1 -- 172.21.15.45:0/333134463 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff848097760 con 0x7ff8500d3460 2024-05-08T05:41:47.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.863+0000 7f8239c6d640 1 --2- 172.21.15.45:0/2602459025 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8218076350 0x7f8218078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.862+0000 7f488bfff640 1 --2- 172.21.15.45:0/95129841 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f48900763a0 0x7f4890078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.863+0000 7ff8477fe640 1 --2- 172.21.15.45:0/333134463 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x7ff83007faa0 0x7ff830081ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.863+0000 7ff8477fe640 1 -- 172.21.15.45:0/333134463 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff848013ec0 con 0x7ff83007faa0 2024-05-08T05:41:47.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.863+0000 7ff8477fe640 1 -- 172.21.15.45:0/333134463 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff84809e050 con 0x7ff8500d3460 2024-05-08T05:41:47.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.863+0000 7fc5577fe640 1 -- 172.21.15.45:0/67583542 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc54801a070 con 0x7fc56006fba0 2024-05-08T05:41:47.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.862+0000 7f488bfff640 1 -- 172.21.15.45:0/95129841 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4898097770 con 0x7f489c03afd0 2024-05-08T05:41:47.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.863+0000 7f488bfff640 1 --2- 172.21.15.45:0/95129841 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x7f489007fa90 0x7f4890081ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.863+0000 7f488bfff640 1 -- 172.21.15.45:0/95129841 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48900825a0 con 0x7f489007fa90 2024-05-08T05:41:47.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.863+0000 7fe5077fe640 1 --2- 172.21.15.45:0/3361262883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4f0004030 0x7fe50815a1f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00c770 tx=0x7fe4fc00cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.863+0000 7f48a37fe640 1 --2- 172.21.15.45:0/95129841 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x7f489007fa90 0x7f4890081ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.863+0000 7ff845d9b640 1 --2- 172.21.15.45:0/333134463 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x7ff83007faa0 0x7ff830081ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.863+0000 7f3f8e9e4640 1 Processor -- start 2024-05-08T05:41:47.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.863+0000 7f488bfff640 1 -- 172.21.15.45:0/95129841 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f489809e050 con 0x7f489c03afd0 2024-05-08T05:41:47.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.863+0000 7fe2dbb22640 1 -- 172.21.15.45:0/1349740447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe2d4163af0 msgr2=0x7fe2d40ae570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.863+0000 7fe2dbb22640 1 --2- 172.21.15.45:0/1349740447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe2d4163af0 0x7fe2d40ae570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009e50 tx=0x7fe2cc02a710 comp rx=0 tx=0).stop 2024-05-08T05:41:47.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.863+0000 7f3f8e9e4640 1 -- start start 2024-05-08T05:41:47.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.863+0000 7f823946c640 1 --2- 172.21.15.45:0/2602459025 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x7f821807fac0 0x7f8218081f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.863+0000 7f48a2ffd640 1 --2- 172.21.15.45:0/95129841 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f48900763a0 0x7f4890078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.863+0000 7fe4effff640 1 -- 172.21.15.45:0/3361262883 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc015070 con 0x7fe4f0004030 2024-05-08T05:41:47.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.864+0000 7ff84e59c640 1 --2- 172.21.15.45:0/333134463 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff830076320 0x7ff8300787e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff838034000 tx=0x7ff83802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.864+0000 7f48a37fe640 1 --2- 172.21.15.45:0/95129841 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x7f489007fa90 0x7f4890081ed0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.864+0000 7fc5577fe640 1 --2- 172.21.15.45:0/67583542 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc5400762d0 0x7fc540078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.864+0000 7f823946c640 1 --2- 172.21.15.45:0/2602459025 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x7f821807fac0 0x7f8218081f00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.864+0000 7f48a2ffd640 1 --2- 172.21.15.45:0/95129841 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f48900763a0 0x7f4890078860 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4894000f60 tx=0x7f4894007d00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.864+0000 7fc5577fe640 1 -- 172.21.15.45:0/67583542 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc548097af0 con 0x7fc56006fba0 2024-05-08T05:41:47.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.864+0000 7fc5577fe640 1 --2- 172.21.15.45:0/67583542 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x7fc54007fa40 0x7fc540081e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.864+0000 7fc5577fe640 1 -- 172.21.15.45:0/67583542 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc540082550 con 0x7fc54007fa40 2024-05-08T05:41:47.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.864+0000 7fc5577fe640 1 -- 172.21.15.45:0/67583542 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc54809c2a0 con 0x7fc56006fba0 2024-05-08T05:41:47.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.864+0000 7fe2dbb22640 1 -- 172.21.15.45:0/1349740447 shutdown_connections 2024-05-08T05:41:47.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.864+0000 7fe2dbb22640 1 --2- 172.21.15.45:0/1349740447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe2d4163af0 0x7fe2d40ae570 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.864+0000 7fe2dbb22640 1 --2- 172.21.15.45:0/1349740447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe2d40b33f0 0x7fe2d40ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.864+0000 7fe2dbb22640 1 -- 172.21.15.45:0/1349740447 >> 172.21.15.45:0/1349740447 conn(0x7fe2d40b6570 msgr2=0x7fe2d40b6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:47.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.865+0000 7fc55e59c640 1 --2- 172.21.15.45:0/67583542 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc5400762d0 0x7fc540078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.864+0000 7f3f8e9e4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f8806ff00 0x7f3f8815e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.864+0000 7f3f8e9e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f880d0fc0 0x7f3f8814ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.864+0000 7f3f8e9e4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f88163500 0x7f3f8815c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.864+0000 7f3f8e9e4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3f880b3c20 con 0x7f3f880d0fc0 2024-05-08T05:41:47.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.864+0000 7f3f8e9e4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3f880af270 con 0x7f3f88163500 2024-05-08T05:41:47.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.865+0000 7fc55ed9d640 1 --2- 172.21.15.45:0/67583542 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x7fc54007fa40 0x7fc540081e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.865+0000 7ff845d9b640 1 --2- 172.21.15.45:0/333134463 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x7ff83007faa0 0x7ff830081ee0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.865+0000 7fc55ed9d640 1 --2- 172.21.15.45:0/67583542 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x7fc54007fa40 0x7fc540081e80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.866+0000 7fc55e59c640 1 --2- 172.21.15.45:0/67583542 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc5400762d0 0x7fc540078790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc550001970 tx=0x7fc550001900 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.865+0000 7ffb177fe640 1 -- 172.21.15.45:0/1232729636 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ffb08054020 con 0x7ffb2006ff00 2024-05-08T05:41:47.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.866+0000 7ffb177fe640 1 --2- 172.21.15.45:0/1232729636 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ffb000763d0 0x7ffb00078890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.864+0000 7f8239c6d640 1 --2- 172.21.15.45:0/2602459025 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8218076350 0x7f8218078810 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8230006fd0 tx=0x7f8230009060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.865+0000 7f822a7fc640 1 -- 172.21.15.45:0/2602459025 <== osd.2 v2:172.21.15.45:6808/785834340 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f82180825d0 con 0x7f821807fac0 2024-05-08T05:41:47.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.865+0000 7f488bfff640 1 -- 172.21.15.45:0/95129841 <== osd.3 v2:172.21.15.45:6816/1259225446 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f48900825a0 con 0x7f489007fa90 2024-05-08T05:41:47.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.865+0000 7f3f87fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f8806ff00 0x7f3f8815e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.865+0000 7f3f8e9e4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3f880b3da0 con 0x7f3f8806ff00 2024-05-08T05:41:47.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.865+0000 7f3f877fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f880d0fc0 0x7f3f8814ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.865+0000 7f3f877fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f880d0fc0 0x7f3f8814ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51356/0 (socket says 172.21.15.45:51356) 2024-05-08T05:41:47.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.865+0000 7f3f877fe640 1 -- 172.21.15.45:0/2173613819 learned_addr learned my addr 172.21.15.45:0/2173613819 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:47.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.865+0000 7f3f8cf81640 1 --2- 172.21.15.45:0/2173613819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f88163500 0x7f3f8815c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.866+0000 7f3f877fe640 1 -- 172.21.15.45:0/2173613819 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f8806ff00 msgr2=0x7f3f8815e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.866+0000 7f3f877fe640 1 --2- 172.21.15.45:0/2173613819 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f8806ff00 0x7f3f8815e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.866+0000 7f3f877fe640 1 -- 172.21.15.45:0/2173613819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f88163500 msgr2=0x7f3f8815c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.866+0000 7f3f877fe640 1 --2- 172.21.15.45:0/2173613819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f88163500 0x7f3f8815c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.866+0000 7f3f877fe640 1 -- 172.21.15.45:0/2173613819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8815ed80 con 0x7f3f880d0fc0 2024-05-08T05:41:47.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.866+0000 7f3f877fe640 1 --2- 172.21.15.45:0/2173613819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f880d0fc0 0x7f3f8814ecb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00c410 tx=0x7f3f7c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.865+0000 7fe2dbb22640 1 -- 172.21.15.45:0/1349740447 shutdown_connections 2024-05-08T05:41:47.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.865+0000 7fe2dbb22640 1 -- 172.21.15.45:0/1349740447 wait complete. 2024-05-08T05:41:47.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.866+0000 7ff8477fe640 1 -- 172.21.15.45:0/333134463 <== osd.6 v2:172.21.15.63:6800/2926759052 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff848013ec0 con 0x7ff83007faa0 2024-05-08T05:41:47.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.863+0000 7fe4effff640 1 -- 172.21.15.45:0/3361262883 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4fc007a30 con 0x7fe4f0004030 2024-05-08T05:41:47.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.865+0000 7fe50dc64640 1 -- 172.21.15.45:0/3361262883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5080ca270 con 0x7fe4f0004030 2024-05-08T05:41:47.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.865+0000 7fe50dc64640 1 -- 172.21.15.45:0/3361262883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5080ca830 con 0x7fe4f0004030 2024-05-08T05:41:47.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.865+0000 7fe4effff640 1 -- 172.21.15.45:0/3361262883 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc015070 con 0x7fe4f0004030 2024-05-08T05:41:47.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.866+0000 7ffb177fe640 1 -- 172.21.15.45:0/1232729636 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ffb080c92e0 con 0x7ffb2006ff00 2024-05-08T05:41:47.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.866+0000 7ffb177fe640 1 --2- 172.21.15.45:0/1232729636 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x7ffb0007fb50 0x7ffb00081f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.867+0000 7fc5577fe640 1 -- 172.21.15.45:0/67583542 <== osd.0 v2:172.21.15.45:6800/3734454393 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc540082550 con 0x7fc54007fa40 2024-05-08T05:41:47.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.867+0000 7f3f84ff9640 1 -- 172.21.15.45:0/2173613819 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c017070 con 0x7f3f880d0fc0 2024-05-08T05:41:47.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.867+0000 7f3f84ff9640 1 -- 172.21.15.45:0/2173613819 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f7c0035f0 con 0x7f3f880d0fc0 2024-05-08T05:41:47.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.867+0000 7f3f84ff9640 1 -- 172.21.15.45:0/2173613819 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c003a00 con 0x7f3f880d0fc0 2024-05-08T05:41:47.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.867+0000 7ffb177fe640 1 -- 172.21.15.45:0/1232729636 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffb0803de50 con 0x7ffb0007fb50 2024-05-08T05:41:47.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.867+0000 7ffb177fe640 1 -- 172.21.15.45:0/1232729636 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ffb080c96d0 con 0x7ffb2006ff00 2024-05-08T05:41:47.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.867+0000 7fe2dbb22640 1 Processor -- start 2024-05-08T05:41:47.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.866+0000 7fe50dc64640 1 -- 172.21.15.45:0/3361262883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe5080caef0 con 0x7fe4f0004030 2024-05-08T05:41:47.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.867+0000 7fe2dbb22640 1 -- start start 2024-05-08T05:41:47.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.867+0000 7ffb1ed9d640 1 --2- 172.21.15.45:0/1232729636 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x7ffb0007fb50 0x7ffb00081f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.868+0000 7f3f8e9e4640 1 -- 172.21.15.45:0/2173613819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8809e0a0 con 0x7f3f880d0fc0 2024-05-08T05:41:47.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.868+0000 7f3f8e9e4640 1 -- 172.21.15.45:0/2173613819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8809e660 con 0x7f3f880d0fc0 2024-05-08T05:41:47.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.868+0000 7ffb1ed9d640 1 --2- 172.21.15.45:0/1232729636 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x7ffb0007fb50 0x7ffb00081f90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.869+0000 7fe4effff640 1 -- 172.21.15.45:0/3361262883 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe4fc018070 con 0x7fe4f0004030 2024-05-08T05:41:47.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.869+0000 7fe2dbb22640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe2d40b33f0 0x7fe2d4166210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.869+0000 7ffb15d9b640 1 --2- 172.21.15.45:0/1232729636 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ffb000763d0 0x7ffb00078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.869+0000 7fe2dbb22640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2d4163710 0x7fe2d4146760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.869+0000 7fe4effff640 1 --2- 172.21.15.45:0/3361262883 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe4f4076700 0x7fe4f4078bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.870+0000 7fe2d98be640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe2d40b33f0 0x7fe2d4166210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.870+0000 7fe4effff640 1 -- 172.21.15.45:0/3361262883 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe4fc007830 con 0x7fe4f0004030 2024-05-08T05:41:47.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.870+0000 7fe506ffd640 1 --2- 172.21.15.45:0/3361262883 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe4f4076700 0x7fe4f4078bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.870+0000 7ffb177fe640 1 -- 172.21.15.45:0/1232729636 <== osd.1 v2:172.21.15.45:6824/1337179184 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffb0803de50 con 0x7ffb0007fb50 2024-05-08T05:41:47.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.870+0000 7ffb15d9b640 1 --2- 172.21.15.45:0/1232729636 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ffb000763d0 0x7ffb00078890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffb200cffc0 tx=0x7ffb10008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.870+0000 7fe4effff640 1 --2- 172.21.15.45:0/3361262883 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x7fe4f40809d0 0x7fe4f4082e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.870+0000 7fe4effff640 1 -- 172.21.15.45:0/3361262883 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe4f4083540 con 0x7fe4f40809d0 2024-05-08T05:41:47.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.870+0000 7f3f8e9e4640 1 -- 172.21.15.45:0/2173613819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3f880693f0 con 0x7f3f880d0fc0 2024-05-08T05:41:47.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.870+0000 7fe2d98be640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe2d40b33f0 0x7fe2d4166210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:47320/0 (socket says 172.21.15.45:47320) 2024-05-08T05:41:47.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.870+0000 7fe507fff640 1 --2- 172.21.15.45:0/3361262883 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x7fe4f40809d0 0x7fe4f4082e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.870+0000 7fe2dbb22640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe2d4163af0 0x7fe2d4156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.870+0000 7fe2dbb22640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d40b0990 con 0x7fe2d4163710 2024-05-08T05:41:47.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.871+0000 7fe506ffd640 1 --2- 172.21.15.45:0/3361262883 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe4f4076700 0x7fe4f4078bc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8032000 tx=0x7fe4f8007000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.871+0000 7fe507fff640 1 --2- 172.21.15.45:0/3361262883 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x7fe4f40809d0 0x7fe4f4082e70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.870+0000 7fe2dbb22640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d40b0810 con 0x7fe2d40b33f0 2024-05-08T05:41:47.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.870+0000 7fe2dbb22640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe2d40701e0 con 0x7fe2d4163af0 2024-05-08T05:41:47.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.870+0000 7f3f84ff9640 1 -- 172.21.15.45:0/2173613819 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3f7c01a070 con 0x7f3f880d0fc0 2024-05-08T05:41:47.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.871+0000 7f3f84ff9640 1 --2- 172.21.15.45:0/2173613819 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f74076320 0x7f3f740787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.870+0000 7fe2d90bd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2d4163710 0x7fe2d4146760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.870+0000 7fe2d90bd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2d4163710 0x7fe2d4146760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51362/0 (socket says 172.21.15.45:51362) 2024-05-08T05:41:47.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.870+0000 7fe2d90bd640 1 -- 172.21.15.45:0/3772331330 learned_addr learned my addr 172.21.15.45:0/3772331330 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:47.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.871+0000 7fe2d90bd640 1 -- 172.21.15.45:0/3772331330 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe2d4163af0 msgr2=0x7fe2d4156940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T05:41:47.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.871+0000 7fe2d90bd640 1 --2- 172.21.15.45:0/3772331330 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe2d4163af0 0x7fe2d4156940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.871+0000 7fe2d90bd640 1 -- 172.21.15.45:0/3772331330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe2d40b33f0 msgr2=0x7fe2d4166210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.871+0000 7fe2d90bd640 1 --2- 172.21.15.45:0/3772331330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe2d40b33f0 0x7fe2d4166210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.871+0000 7fe2d90bd640 1 -- 172.21.15.45:0/3772331330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d4165170 con 0x7fe2d4163710 2024-05-08T05:41:47.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.871+0000 7fe2d90bd640 1 --2- 172.21.15.45:0/3772331330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2d4163710 0x7fe2d4146760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400d800 tx=0x7fe2c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.871+0000 7fe2d98be640 1 --2- 172.21.15.45:0/3772331330 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe2d40b33f0 0x7fe2d4166210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-08T05:41:47.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.871+0000 7fe4effff640 1 -- 172.21.15.45:0/3361262883 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe4fc05c7a0 con 0x7fe4f0004030 2024-05-08T05:41:47.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.872+0000 7f3f84ff9640 1 -- 172.21.15.45:0/2173613819 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3f7c05c1e0 con 0x7f3f880d0fc0 2024-05-08T05:41:47.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.872+0000 7f3f84ff9640 1 --2- 172.21.15.45:0/2173613819 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x7f3f740806f0 0x7f3f74082b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.872+0000 7f3f84ff9640 1 -- 172.21.15.45:0/2173613819 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f74083260 con 0x7f3f740806f0 2024-05-08T05:41:47.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.872+0000 7fe2ca7fc640 1 -- 172.21.15.45:0/3772331330 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c400b6c0 con 0x7fe2d4163710 2024-05-08T05:41:47.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.872+0000 7fe2ca7fc640 1 -- 172.21.15.45:0/3772331330 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c4019070 con 0x7fe2d4163710 2024-05-08T05:41:47.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.872+0000 7fe4effff640 1 -- 172.21.15.45:0/3361262883 <== osd.7 v2:172.21.15.63:6816/1163469437 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4f4083540 con 0x7fe4f40809d0 2024-05-08T05:41:47.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.872+0000 7fe2dbb22640 1 -- 172.21.15.45:0/3772331330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d4165450 con 0x7fe2d4163710 2024-05-08T05:41:47.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.872+0000 7fe2ca7fc640 1 -- 172.21.15.45:0/3772331330 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c400ba60 con 0x7fe2d4163710 2024-05-08T05:41:47.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.872+0000 7fe2dbb22640 1 -- 172.21.15.45:0/3772331330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d4165a10 con 0x7fe2d4163710 2024-05-08T05:41:47.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.872+0000 7f3f84ff9640 1 -- 172.21.15.45:0/2173613819 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3f7c05c3e0 con 0x7f3f880d0fc0 2024-05-08T05:41:47.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.872+0000 7f3f8cf81640 1 --2- 172.21.15.45:0/2173613819 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x7f3f740806f0 0x7f3f74082b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.873+0000 7f3f87fff640 1 --2- 172.21.15.45:0/2173613819 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f74076320 0x7f3f740787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.874+0000 7f3f8cf81640 1 --2- 172.21.15.45:0/2173613819 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x7f3f740806f0 0x7f3f74082b90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.874+0000 7f3f87fff640 1 --2- 172.21.15.45:0/2173613819 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f74076320 0x7f3f740787e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3f78000f60 tx=0x7f3f78009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.875+0000 7fe2dbb22640 1 -- 172.21.15.45:0/3772331330 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe2bc001030 con 0x7fe2d4163710 2024-05-08T05:41:47.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.875+0000 7fe2ca7fc640 1 -- 172.21.15.45:0/3772331330 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe2c400bc00 con 0x7fe2d4163710 2024-05-08T05:41:47.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.875+0000 7f3f84ff9640 1 -- 172.21.15.45:0/2173613819 <== osd.4 v2:172.21.15.63:6824/2212879151 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f74083260 con 0x7f3f740806f0 2024-05-08T05:41:47.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.876+0000 7fe2ca7fc640 1 --2- 172.21.15.45:0/3772331330 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe2b80763a0 0x7fe2b8078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.876+0000 7fe2d98be640 1 --2- 172.21.15.45:0/3772331330 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe2b80763a0 0x7fe2b8078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.876+0000 7fe2ca7fc640 1 -- 172.21.15.45:0/3772331330 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe2c4097d30 con 0x7fe2d4163710 2024-05-08T05:41:47.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.877+0000 7fe2ca7fc640 1 --2- 172.21.15.45:0/3772331330 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x7fe2b807fa90 0x7fe2b8081ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.877+0000 7fe2d98be640 1 --2- 172.21.15.45:0/3772331330 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe2b80763a0 0x7fe2b8078860 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4166b50 tx=0x7fe2d0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.877+0000 7fe2ca7fc640 1 -- 172.21.15.45:0/3772331330 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe2c4013e50 con 0x7fe2b807fa90 2024-05-08T05:41:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.877+0000 7fe2ca7fc640 1 -- 172.21.15.45:0/3772331330 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe2c405d280 con 0x7fe2d4163710 2024-05-08T05:41:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.878+0000 7fe2da0bf640 1 --2- 172.21.15.45:0/3772331330 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x7fe2b807fa90 0x7fe2b8081ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.878+0000 7fe2da0bf640 1 --2- 172.21.15.45:0/3772331330 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x7fe2b807fa90 0x7fe2b8081ed0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.879+0000 7fe2ca7fc640 1 -- 172.21.15.45:0/3772331330 <== osd.5 v2:172.21.15.63:6808/2395798255 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2c4013e50 con 0x7fe2b807fa90 2024-05-08T05:41:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.896+0000 7f8217fff640 1 -- 172.21.15.45:0/2602459025 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8210002dd0 con 0x7f821807fac0 2024-05-08T05:41:47.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.897+0000 7f822a7fc640 1 -- 172.21.15.45:0/2602459025 <== osd.2 v2:172.21.15.45:6808/785834340 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8210002dd0 con 0x7f821807fac0 2024-05-08T05:41:47.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.897+0000 7f48a3fff640 1 -- 172.21.15.45:0/95129841 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f489c03b930 con 0x7f489007fa90 2024-05-08T05:41:47.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.898+0000 7f823bed1640 1 -- 172.21.15.45:0/2602459025 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x7f821807fac0 msgr2=0x7f8218081f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.898+0000 7f823bed1640 1 --2- 172.21.15.45:0/2602459025 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x7f821807fac0 0x7f8218081f00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.898+0000 7f823bed1640 1 -- 172.21.15.45:0/2602459025 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8218076350 msgr2=0x7f8218078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.898+0000 7f823bed1640 1 --2- 172.21.15.45:0/2602459025 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8218076350 0x7f8218078810 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8230006fd0 tx=0x7f8230009060 comp rx=0 tx=0).stop 2024-05-08T05:41:47.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.898+0000 7f823bed1640 1 -- 172.21.15.45:0/2602459025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82340d1bb0 msgr2=0x7f82340c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.898+0000 7f823bed1640 1 --2- 172.21.15.45:0/2602459025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82340d1bb0 0x7f82340c8fd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f822c009d90 tx=0x7f822c004a40 comp rx=0 tx=0).stop 2024-05-08T05:41:47.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.898+0000 7f823a46e640 1 -- 172.21.15.45:0/2602459025 reap_dead start 2024-05-08T05:41:47.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.898+0000 7f488bfff640 1 -- 172.21.15.45:0/95129841 <== osd.3 v2:172.21.15.45:6816/1259225446 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f489c03b930 con 0x7f489007fa90 2024-05-08T05:41:47.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.898+0000 7f823bed1640 1 -- 172.21.15.45:0/2602459025 shutdown_connections 2024-05-08T05:41:47.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.898+0000 7f823bed1640 1 -- 172.21.15.45:0/2602459025 >> 172.21.15.45:0/2602459025 conn(0x7f82340d4fe0 msgr2=0x7f8234162460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:47.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.899+0000 7f823bed1640 1 -- 172.21.15.45:0/2602459025 shutdown_connections 2024-05-08T05:41:47.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.899+0000 7f823bed1640 1 -- 172.21.15.45:0/2602459025 wait complete. 2024-05-08T05:41:47.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.899+0000 7f4889ffb640 1 -- 172.21.15.45:0/95129841 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x7f489007fa90 msgr2=0x7f4890081ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.899+0000 7f4889ffb640 1 --2- 172.21.15.45:0/95129841 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x7f489007fa90 0x7f4890081ed0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.899+0000 7f4889ffb640 1 -- 172.21.15.45:0/95129841 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f48900763a0 msgr2=0x7f4890078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.900+0000 7f4889ffb640 1 --2- 172.21.15.45:0/95129841 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f48900763a0 0x7f4890078860 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4894000f60 tx=0x7f4894007d00 comp rx=0 tx=0).stop 2024-05-08T05:41:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.900+0000 7f4889ffb640 1 -- 172.21.15.45:0/95129841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f489c03afd0 msgr2=0x7f489c048110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.900+0000 7f4889ffb640 1 --2- 172.21.15.45:0/95129841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f489c03afd0 0x7f489c048110 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f489800c430 tx=0x7f489800c8f0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.900+0000 7f48a37fe640 1 -- 172.21.15.45:0/95129841 reap_dead start 2024-05-08T05:41:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.900+0000 7f4889ffb640 1 -- 172.21.15.45:0/95129841 shutdown_connections 2024-05-08T05:41:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.900+0000 7f4889ffb640 1 -- 172.21.15.45:0/95129841 >> 172.21.15.45:0/95129841 conn(0x7f489c01ef70 msgr2=0x7f489c0080d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.900+0000 7f4889ffb640 1 -- 172.21.15.45:0/95129841 shutdown_connections 2024-05-08T05:41:47.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.901+0000 7f4889ffb640 1 -- 172.21.15.45:0/95129841 wait complete. 2024-05-08T05:41:47.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.901+0000 7ff854ff0640 1 -- 172.21.15.45:0/333134463 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff834002dd0 con 0x7ff83007faa0 2024-05-08T05:41:47.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.902+0000 7ff8477fe640 1 -- 172.21.15.45:0/333134463 <== osd.6 v2:172.21.15.63:6800/2926759052 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff834002dd0 con 0x7ff83007faa0 2024-05-08T05:41:47.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.902+0000 7fc564a1d640 1 -- 172.21.15.45:0/67583542 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc5600b6570 con 0x7fc54007fa40 2024-05-08T05:41:47.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.903+0000 7ff844d99640 1 -- 172.21.15.45:0/333134463 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x7ff83007faa0 msgr2=0x7ff830081ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.903+0000 7ff844d99640 1 --2- 172.21.15.45:0/333134463 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x7ff83007faa0 0x7ff830081ee0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.903+0000 7fc5577fe640 1 -- 172.21.15.45:0/67583542 <== osd.0 v2:172.21.15.45:6800/3734454393 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc5600b6570 con 0x7fc54007fa40 2024-05-08T05:41:47.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.903+0000 7ff844d99640 1 -- 172.21.15.45:0/333134463 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff830076320 msgr2=0x7ff8300787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.903+0000 7ff844d99640 1 --2- 172.21.15.45:0/333134463 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff830076320 0x7ff8300787e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff838034000 tx=0x7ff83802e000 comp rx=0 tx=0).stop 2024-05-08T05:41:47.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.903+0000 7ff844d99640 1 -- 172.21.15.45:0/333134463 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8500d3460 msgr2=0x7ff8500d3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.903+0000 7ff844d99640 1 --2- 172.21.15.45:0/333134463 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8500d3460 0x7ff8500d3910 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff8500b5990 tx=0x7ff84800b870 comp rx=0 tx=0).stop 2024-05-08T05:41:47.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.903+0000 7ff84ed9d640 1 -- 172.21.15.45:0/333134463 reap_dead start 2024-05-08T05:41:47.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.903+0000 7ff844d99640 1 -- 172.21.15.45:0/333134463 shutdown_connections 2024-05-08T05:41:47.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.903+0000 7ff844d99640 1 -- 172.21.15.45:0/333134463 >> 172.21.15.45:0/333134463 conn(0x7ff8500b4a40 msgr2=0x7ff8500b7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:47.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.903+0000 7ffb14d99640 1 -- 172.21.15.45:0/1232729636 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffaf8002dd0 con 0x7ffb0007fb50 2024-05-08T05:41:47.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.904+0000 7ff844d99640 1 -- 172.21.15.45:0/333134463 shutdown_connections 2024-05-08T05:41:47.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.904+0000 7ff844d99640 1 -- 172.21.15.45:0/333134463 wait complete. 2024-05-08T05:41:47.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.904+0000 7ffb177fe640 1 -- 172.21.15.45:0/1232729636 <== osd.1 v2:172.21.15.45:6824/1337179184 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffaf8002dd0 con 0x7ffb0007fb50 2024-05-08T05:41:47.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.905+0000 7fc554d99640 1 -- 172.21.15.45:0/67583542 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x7fc54007fa40 msgr2=0x7fc540081e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.905+0000 7fc554d99640 1 --2- 172.21.15.45:0/67583542 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x7fc54007fa40 0x7fc540081e80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.905+0000 7fe50dc64640 1 -- 172.21.15.45:0/3361262883 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe508069b00 con 0x7fe4f40809d0 2024-05-08T05:41:47.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.905+0000 7ffb14d99640 1 -- 172.21.15.45:0/1232729636 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x7ffb0007fb50 msgr2=0x7ffb00081f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.905+0000 7ffb14d99640 1 --2- 172.21.15.45:0/1232729636 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x7ffb0007fb50 0x7ffb00081f90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.905+0000 7ffb14d99640 1 -- 172.21.15.45:0/1232729636 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ffb000763d0 msgr2=0x7ffb00078890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.905+0000 7ffb14d99640 1 --2- 172.21.15.45:0/1232729636 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ffb000763d0 0x7ffb00078890 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffb200cffc0 tx=0x7ffb10008040 comp rx=0 tx=0).stop 2024-05-08T05:41:47.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.905+0000 7ffb14d99640 1 -- 172.21.15.45:0/1232729636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb2006ff00 msgr2=0x7ffb2009e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.905+0000 7ffb14d99640 1 --2- 172.21.15.45:0/1232729636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb2006ff00 0x7ffb2009e640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffb0803b710 tx=0x7ffb080046c0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.905+0000 7fc554d99640 1 -- 172.21.15.45:0/67583542 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc5400762d0 msgr2=0x7fc540078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.905+0000 7ffb1ed9d640 1 -- 172.21.15.45:0/1232729636 reap_dead start 2024-05-08T05:41:47.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.905+0000 7fc554d99640 1 --2- 172.21.15.45:0/67583542 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc5400762d0 0x7fc540078790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc550001970 tx=0x7fc550001900 comp rx=0 tx=0).stop 2024-05-08T05:41:47.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.905+0000 7fc554d99640 1 -- 172.21.15.45:0/67583542 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc56006fba0 msgr2=0x7fc560154aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.905+0000 7fc554d99640 1 --2- 172.21.15.45:0/67583542 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc56006fba0 0x7fc560154aa0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc54800e8a0 tx=0x7fc54800ed60 comp rx=0 tx=0).stop 2024-05-08T05:41:47.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.905+0000 7fc55ed9d640 1 -- 172.21.15.45:0/67583542 reap_dead start 2024-05-08T05:41:47.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.905+0000 7ffb14d99640 1 -- 172.21.15.45:0/1232729636 shutdown_connections 2024-05-08T05:41:47.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.905+0000 7ffb14d99640 1 -- 172.21.15.45:0/1232729636 >> 172.21.15.45:0/1232729636 conn(0x7ffb2006c3a0 msgr2=0x7ffb20160a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:47.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.906+0000 7fc554d99640 1 -- 172.21.15.45:0/67583542 shutdown_connections 2024-05-08T05:41:47.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.906+0000 7fc554d99640 1 -- 172.21.15.45:0/67583542 >> 172.21.15.45:0/67583542 conn(0x7fc5600738e0 msgr2=0x7fc5601575a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:47.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.906+0000 7ffb14d99640 1 -- 172.21.15.45:0/1232729636 shutdown_connections 2024-05-08T05:41:47.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.906+0000 7ffb14d99640 1 -- 172.21.15.45:0/1232729636 wait complete. 2024-05-08T05:41:47.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.906+0000 7fc554d99640 1 -- 172.21.15.45:0/67583542 shutdown_connections 2024-05-08T05:41:47.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.906+0000 7fc554d99640 1 -- 172.21.15.45:0/67583542 wait complete. 2024-05-08T05:41:47.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.906+0000 7fe4effff640 1 -- 172.21.15.45:0/3361262883 <== osd.7 v2:172.21.15.63:6816/1163469437 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe508069b00 con 0x7fe4f40809d0 2024-05-08T05:41:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.907+0000 7fe50dc64640 1 -- 172.21.15.45:0/3361262883 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x7fe4f40809d0 msgr2=0x7fe4f4082e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.907+0000 7fe50dc64640 1 --2- 172.21.15.45:0/3361262883 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x7fe4f40809d0 0x7fe4f4082e70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.907+0000 7fe50dc64640 1 -- 172.21.15.45:0/3361262883 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe4f4076700 msgr2=0x7fe4f4078bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.907+0000 7fe50dc64640 1 --2- 172.21.15.45:0/3361262883 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe4f4076700 0x7fe4f4078bc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8032000 tx=0x7fe4f8007000 comp rx=0 tx=0).stop 2024-05-08T05:41:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.907+0000 7fe50dc64640 1 -- 172.21.15.45:0/3361262883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4f0004030 msgr2=0x7fe50815a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.907+0000 7fe50dc64640 1 --2- 172.21.15.45:0/3361262883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4f0004030 0x7fe50815a1f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00c770 tx=0x7fe4fc00cc30 comp rx=0 tx=0).stop 2024-05-08T05:41:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.907+0000 7fe507fff640 1 -- 172.21.15.45:0/3361262883 reap_dead start 2024-05-08T05:41:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.907+0000 7fe50dc64640 1 -- 172.21.15.45:0/3361262883 shutdown_connections 2024-05-08T05:41:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.907+0000 7fe50dc64640 1 -- 172.21.15.45:0/3361262883 >> 172.21.15.45:0/3361262883 conn(0x7fe5080746d0 msgr2=0x7fe5080b67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.908+0000 7fe50dc64640 1 -- 172.21.15.45:0/3361262883 shutdown_connections 2024-05-08T05:41:47.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.908+0000 7fe50dc64640 1 -- 172.21.15.45:0/3361262883 wait complete. 2024-05-08T05:41:47.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.908+0000 7f3f8e9e4640 1 -- 172.21.15.45:0/2173613819 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3f88165f40 con 0x7f3f740806f0 2024-05-08T05:41:47.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.910+0000 7f3f84ff9640 1 -- 172.21.15.45:0/2173613819 <== osd.4 v2:172.21.15.63:6824/2212879151 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3f88165f40 con 0x7f3f740806f0 2024-05-08T05:41:47.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.910+0000 7fe2dbb22640 1 -- 172.21.15.45:0/3772331330 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe2bc002dd0 con 0x7fe2b807fa90 2024-05-08T05:41:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.910+0000 7fe2ca7fc640 1 -- 172.21.15.45:0/3772331330 <== osd.5 v2:172.21.15.63:6808/2395798255 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe2bc002dd0 con 0x7fe2b807fa90 2024-05-08T05:41:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.911+0000 7f3f8e9e4640 1 -- 172.21.15.45:0/2173613819 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x7f3f740806f0 msgr2=0x7f3f74082b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.911+0000 7f3f8e9e4640 1 --2- 172.21.15.45:0/2173613819 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x7f3f740806f0 0x7f3f74082b90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.911+0000 7f3f8e9e4640 1 -- 172.21.15.45:0/2173613819 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f74076320 msgr2=0x7f3f740787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.911+0000 7f3f8e9e4640 1 --2- 172.21.15.45:0/2173613819 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f74076320 0x7f3f740787e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3f78000f60 tx=0x7f3f78009040 comp rx=0 tx=0).stop 2024-05-08T05:41:47.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.911+0000 7f3f8e9e4640 1 -- 172.21.15.45:0/2173613819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f880d0fc0 msgr2=0x7f3f8814ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.911+0000 7f3f8e9e4640 1 --2- 172.21.15.45:0/2173613819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f880d0fc0 0x7f3f8814ecb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00c410 tx=0x7f3f7c00c8d0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.912+0000 7f3f8cf81640 1 -- 172.21.15.45:0/2173613819 reap_dead start 2024-05-08T05:41:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.912+0000 7fe2b7fff640 1 -- 172.21.15.45:0/3772331330 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x7fe2b807fa90 msgr2=0x7fe2b8081ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.912+0000 7fe2b7fff640 1 --2- 172.21.15.45:0/3772331330 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x7fe2b807fa90 0x7fe2b8081ed0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.912+0000 7f3f8e9e4640 1 -- 172.21.15.45:0/2173613819 shutdown_connections 2024-05-08T05:41:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.912+0000 7f3f8e9e4640 1 -- 172.21.15.45:0/2173613819 >> 172.21.15.45:0/2173613819 conn(0x7f3f8806c3a0 msgr2=0x7f3f880d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.912+0000 7fe2b7fff640 1 -- 172.21.15.45:0/3772331330 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe2b80763a0 msgr2=0x7fe2b8078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.912+0000 7fe2b7fff640 1 --2- 172.21.15.45:0/3772331330 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe2b80763a0 0x7fe2b8078860 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4166b50 tx=0x7fe2d0009040 comp rx=0 tx=0).stop 2024-05-08T05:41:47.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.912+0000 7fe2b7fff640 1 -- 172.21.15.45:0/3772331330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2d4163710 msgr2=0x7fe2d4146760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.912+0000 7fe2b7fff640 1 --2- 172.21.15.45:0/3772331330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2d4163710 0x7fe2d4146760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400d800 tx=0x7fe2c400dcc0 comp rx=0 tx=0).stop 2024-05-08T05:41:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.912+0000 7fe2da0bf640 1 -- 172.21.15.45:0/3772331330 reap_dead start 2024-05-08T05:41:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.912+0000 7f3f8e9e4640 1 -- 172.21.15.45:0/2173613819 shutdown_connections 2024-05-08T05:41:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.912+0000 7f3f8e9e4640 1 -- 172.21.15.45:0/2173613819 wait complete. 2024-05-08T05:41:47.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.914+0000 7fe2b7fff640 1 -- 172.21.15.45:0/3772331330 shutdown_connections 2024-05-08T05:41:47.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.914+0000 7fe2b7fff640 1 -- 172.21.15.45:0/3772331330 >> 172.21.15.45:0/3772331330 conn(0x7fe2d40b6570 msgr2=0x7fe2d40ae030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:47.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.915+0000 7fe2b7fff640 1 -- 172.21.15.45:0/3772331330 shutdown_connections 2024-05-08T05:41:47.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:47.915+0000 7fe2b7fff640 1 -- 172.21.15.45:0/3772331330 wait complete. 2024-05-08T05:41:47.966 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2024-05-08T05:41:47.967 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T05:41:47.968 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2024-05-08T05:41:47.968 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T05:41:47.969 INFO:teuthology.orchestra.run.smithi045.stdout:60129542154 2024-05-08T05:41:47.969 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T05:41:47.970 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2024-05-08T05:41:47.970 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T05:41:47.971 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2024-05-08T05:41:47.971 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T05:41:47.972 INFO:teuthology.orchestra.run.smithi045.stdout:60129542153 2024-05-08T05:41:47.972 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T05:41:47.973 INFO:teuthology.orchestra.run.smithi045.stdout:55834574856 2024-05-08T05:41:47.973 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T05:41:47.974 INFO:teuthology.orchestra.run.smithi045.stdout:60129542153 2024-05-08T05:41:47.974 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T05:41:48.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.186+0000 7f1acad90640 1 Processor -- start 2024-05-08T05:41:48.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.187+0000 7f1acad90640 1 -- start start 2024-05-08T05:41:48.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.188+0000 7f1acad90640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1abc0e7550 0x7f1abc005ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.188+0000 7f1acad90640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1abc047c90 0x7f1abc006410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.188+0000 7f1acad90640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1abc04f450 con 0x7f1abc006950 2024-05-08T05:41:48.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.188+0000 7f1acad90640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f1abc04f730 con 0x7f1abc047c90 2024-05-08T05:41:48.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.188+0000 7f1acad90640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1abc04fbb0 con 0x7f1abc0e7550 2024-05-08T05:41:48.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.188+0000 7f1aca58f640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1abc006950 0x7f1abc0eb160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43956/0 (socket says 172.21.15.45:43956) 2024-05-08T05:41:48.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.188+0000 7f1aca58f640 1 -- 172.21.15.45:0/342731712 learned_addr learned my addr 172.21.15.45:0/342731712 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:48.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.188+0000 7f1ac9d8e640 1 --2- 172.21.15.45:0/342731712 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1abc0e7550 0x7f1abc005ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.189+0000 7f1ac958d640 1 --2- 172.21.15.45:0/342731712 >> v2:172.21.15.45:3300/0 conn(0x7f1abc047c90 0x7f1abc006410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.189+0000 7f1ac8d8c640 1 -- 172.21.15.45:0/342731712 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 960174062 0 0) 0x7f1abc04f450 con 0x7f1abc006950 2024-05-08T05:41:48.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.189+0000 7f1ac8d8c640 1 -- 172.21.15.45:0/342731712 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ab0003520 con 0x7f1abc006950 2024-05-08T05:41:48.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.190+0000 7f1ac958d640 1 -- 172.21.15.45:0/342731712 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1abc0e7550 msgr2=0x7f1abc005ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:48.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.190+0000 7f1ac958d640 1 --2- 172.21.15.45:0/342731712 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1abc0e7550 0x7f1abc005ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.190+0000 7f1ac958d640 1 -- 172.21.15.45:0/342731712 >> v1:172.21.15.45:6789/0 conn(0x7f1abc006950 legacy=0x7f1abc0eb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.190+0000 7f1ac8d8c640 1 -- 172.21.15.45:0/342731712 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3977614955 0 0) 0x7f1ab0003520 con 0x7f1abc006950 2024-05-08T05:41:48.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.190+0000 7f1ac958d640 1 -- 172.21.15.45:0/342731712 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc04fe90 con 0x7f1abc047c90 2024-05-08T05:41:48.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.190+0000 7f1ac9d8e640 1 --2- 172.21.15.45:0/342731712 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1abc0e7550 0x7f1abc005ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:48.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.191+0000 7f1ac958d640 1 --2- 172.21.15.45:0/342731712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc047c90 0x7f1abc006410 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009a00 tx=0x7f1ac002a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff926eb8cf0084c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.191+0000 7f9e8f59e640 1 Processor -- start 2024-05-08T05:41:48.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.191+0000 7f4427028640 1 Processor -- start 2024-05-08T05:41:48.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.191+0000 7f4427028640 1 -- start start 2024-05-08T05:41:48.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.191+0000 7f279955f640 1 Processor -- start 2024-05-08T05:41:48.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.192+0000 7f279955f640 1 -- start start 2024-05-08T05:41:48.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.191+0000 7f9e8f59e640 1 -- start start 2024-05-08T05:41:48.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f4427028640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f442006f060 0x7f442006e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f4427028640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f44200d0530 0x7f44200b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f4427028640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f442015d2a0 con 0x7f442006fe30 2024-05-08T05:41:48.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f4427028640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f442006f440 con 0x7f44200d0530 2024-05-08T05:41:48.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f4427028640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f44201609b0 con 0x7f442006f060 2024-05-08T05:41:48.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f279955f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f27940b42c0 0x7f279415ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f4424dc4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f442006f060 0x7f442006e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f279955f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f279415f0c0 con 0x7f2794070050 2024-05-08T05:41:48.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f279955f640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f279415f3a0 con 0x7f279415bf80 2024-05-08T05:41:48.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f4424dc4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f442006f060 0x7f442006e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:33242/0 (socket says 172.21.15.45:33242) 2024-05-08T05:41:48.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f9e8f59e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e880e7970 0x7f9e88053790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f9e8f59e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9e88007180 0x7f9e88053cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f9e8f59e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e88050550 con 0x7f9e880e7590 2024-05-08T05:41:48.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f279955f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f279415fd40 con 0x7f27940b42c0 2024-05-08T05:41:48.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f9e8f59e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9e880507f0 con 0x7f9e88007180 2024-05-08T05:41:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f9e8f59e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9e88050ad0 con 0x7f9e880e7970 2024-05-08T05:41:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f4424dc4640 1 -- 172.21.15.45:0/1163763499 learned_addr learned my addr 172.21.15.45:0/1163763499 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f9e8dd9b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e880e7970 0x7f9e88053790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f44255c5640 1 --2- 172.21.15.45:0/1163763499 >> v2:172.21.15.63:3300/0 conn(0x7f44200d0530 0x7f44200b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.192+0000 7f141d872640 1 Processor -- start 2024-05-08T05:41:48.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.192+0000 7f141d872640 1 -- start start 2024-05-08T05:41:48.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f9e8dd9b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e880e7970 0x7f9e88053790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:33246/0 (socket says 172.21.15.45:33246) 2024-05-08T05:41:48.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f9e8dd9b640 1 -- 172.21.15.45:0/2985550326 learned_addr learned my addr 172.21.15.45:0/2985550326 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:48.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f4424dc4640 1 -- 172.21.15.45:0/1163763499 >> v2:172.21.15.63:3300/0 conn(0x7f44200d0530 msgr2=0x7f44200b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:48.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f4424dc4640 1 --2- 172.21.15.45:0/1163763499 >> v2:172.21.15.63:3300/0 conn(0x7f44200d0530 0x7f44200b3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f4424dc4640 1 -- 172.21.15.45:0/1163763499 >> v1:172.21.15.45:6789/0 conn(0x7f442006fe30 legacy=0x7f44200cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:48.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f4424dc4640 1 -- 172.21.15.45:0/1163763499 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4420160c90 con 0x7f442006f060 2024-05-08T05:41:48.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.194+0000 7f141d872640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f141815bc50 0x7f14180b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.194+0000 7f141d872640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f14180732c0 0x7f1418073750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f2798d5e640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f279415bf80 0x7f27940cece0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:53718/0 (socket says 172.21.15.45:53718) 2024-05-08T05:41:48.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f2798d5e640 1 -- 172.21.15.45:0/1930723754 learned_addr learned my addr 172.21.15.45:0/1930723754 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:48.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.193+0000 7f9e8ed9d640 1 --2- 172.21.15.45:0/2985550326 >> v2:172.21.15.63:3300/0 conn(0x7f9e88007180 0x7f9e88053cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.194+0000 7f9e8dd9b640 1 -- 172.21.15.45:0/2985550326 >> v2:172.21.15.63:3300/0 conn(0x7f9e88007180 msgr2=0x7f9e88053cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:48.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.194+0000 7f9e8dd9b640 1 --2- 172.21.15.45:0/2985550326 >> v2:172.21.15.63:3300/0 conn(0x7f9e88007180 0x7f9e88053cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.194+0000 7f9e8dd9b640 1 -- 172.21.15.45:0/2985550326 >> v1:172.21.15.63:6789/0 conn(0x7f9e880e7590 legacy=0x7f9e88006a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.194+0000 7f9e8dd9b640 1 -- 172.21.15.45:0/2985550326 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e88050db0 con 0x7f9e880e7970 2024-05-08T05:41:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.194+0000 7f1ac8d8c640 1 -- 172.21.15.45:0/342731712 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0037070 con 0x7f1abc047c90 2024-05-08T05:41:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.194+0000 7f1ac8d8c640 1 -- 172.21.15.45:0/342731712 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ac002aab0 con 0x7f1abc047c90 2024-05-08T05:41:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.195+0000 7f4424dc4640 1 --2- 172.21.15.45:0/1163763499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f442006f060 0x7f442006e0f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4410009870 tx=0x7f441002a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82161d6feb76e1b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.195+0000 7f27937fe640 1 --2- 172.21.15.45:0/1930723754 >> v2:172.21.15.45:3300/0 conn(0x7f27940b42c0 0x7f279415ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.194+0000 7f141d872640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14180711f0 con 0x7f141806f540 2024-05-08T05:41:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.194+0000 7f141d872640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f141806fa40 con 0x7f14180732c0 2024-05-08T05:41:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.194+0000 7f141d872640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f141806fd20 con 0x7f141815bc50 2024-05-08T05:41:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.194+0000 7f1416ffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f141815bc50 0x7f14180b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.194+0000 7f1416ffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f141815bc50 0x7f14180b5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:33262/0 (socket says 172.21.15.45:33262) 2024-05-08T05:41:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.194+0000 7f1416ffd640 1 -- 172.21.15.45:0/121693877 learned_addr learned my addr 172.21.15.45:0/121693877 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:48.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.195+0000 7f1acad90640 1 -- 172.21.15.45:0/342731712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc047c90 msgr2=0x7f1abc006410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.195+0000 7f9e8dd9b640 1 --2- 172.21.15.45:0/2985550326 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e880e7970 0x7f9e88053790 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9e90000b60 tx=0x7f9e900a6770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c561f5df2f258a6a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.195+0000 7f1acad90640 1 --2- 172.21.15.45:0/342731712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc047c90 0x7f1abc006410 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009a00 tx=0x7f1ac002a1f0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.195+0000 7f62eff78640 1 Processor -- start 2024-05-08T05:41:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.195+0000 7f14177fe640 1 --2- 172.21.15.45:0/121693877 >> v2:172.21.15.45:3300/0 conn(0x7f14180732c0 0x7f1418073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.195+0000 7f1416ffd640 1 -- 172.21.15.45:0/121693877 >> v2:172.21.15.45:3300/0 conn(0x7f14180732c0 msgr2=0x7f1418073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.195+0000 7f1416ffd640 1 --2- 172.21.15.45:0/121693877 >> v2:172.21.15.45:3300/0 conn(0x7f14180732c0 0x7f1418073750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.195+0000 7f1416ffd640 1 -- 172.21.15.45:0/121693877 >> v1:172.21.15.45:6789/0 conn(0x7f141806f540 legacy=0x7f1418070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:48.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.195+0000 7f1416ffd640 1 -- 172.21.15.45:0/121693877 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1418070000 con 0x7f141815bc50 2024-05-08T05:41:48.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.195+0000 7f1acad90640 1 -- 172.21.15.45:0/342731712 shutdown_connections 2024-05-08T05:41:48.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.195+0000 7f1acad90640 1 --2- 172.21.15.45:0/342731712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc047c90 0x7f1abc006410 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.195+0000 7f1acad90640 1 --2- 172.21.15.45:0/342731712 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1abc0e7550 0x7f1abc005ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.195+0000 7f1acad90640 1 -- 172.21.15.45:0/342731712 >> 172.21.15.45:0/342731712 conn(0x7f1abc009fa0 msgr2=0x7f1abc00a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:48.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.195+0000 7f62eff78640 1 -- start start 2024-05-08T05:41:48.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.196+0000 7f1acad90640 1 -- 172.21.15.45:0/342731712 shutdown_connections 2024-05-08T05:41:48.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.196+0000 7f1acad90640 1 -- 172.21.15.45:0/342731712 wait complete. 2024-05-08T05:41:48.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.196+0000 7f27937fe640 1 -- 172.21.15.45:0/1930723754 >> v1:172.21.15.63:6789/0 conn(0x7f279415bf80 legacy=0x7f27940cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.196+0000 7f27937fe640 1 -- 172.21.15.45:0/1930723754 >> v1:172.21.15.45:6789/0 conn(0x7f2794070050 legacy=0x7f27940b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.196+0000 7f27937fe640 1 -- 172.21.15.45:0/1930723754 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279415f680 con 0x7f27940b42c0 2024-05-08T05:41:48.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.196+0000 7f1416ffd640 1 --2- 172.21.15.45:0/121693877 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f141815bc50 0x7f14180b5ba0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1408009e50 tx=0x7f140802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d52933a3b37eb54 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.197+0000 7f62eff78640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62e00e3070 0x7f62e003c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.197+0000 7f62eff78640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62e00589b0 con 0x7f62e00e3450 2024-05-08T05:41:48.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.197+0000 7f62eff78640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62e0058c90 con 0x7f62e00e7900 2024-05-08T05:41:48.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.197+0000 7f62eff78640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f62e0006890 con 0x7f62e00e3070 2024-05-08T05:41:48.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.197+0000 7fa48bf0a640 1 Processor -- start 2024-05-08T05:41:48.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.197+0000 7f62ed513640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f62e00e3450 0x7f62e0058160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43994/0 (socket says 172.21.15.45:43994) 2024-05-08T05:41:48.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.197+0000 7f62ed513640 1 -- 172.21.15.45:0/1696799324 learned_addr learned my addr 172.21.15.45:0/1696799324 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:48.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.197+0000 7f62edd14640 1 --2- 172.21.15.45:0/1696799324 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62e00e3070 0x7f62e003c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.197+0000 7fa48bf0a640 1 -- start start 2024-05-08T05:41:48.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.197+0000 7f27937fe640 1 --2- 172.21.15.45:0/1930723754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27940b42c0 0x7f279415ba40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f27880097a0 tx=0x7f278802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75c9abb23f61a19 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.198+0000 7fc051fa7640 1 Processor -- start 2024-05-08T05:41:48.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.198+0000 7f1acad90640 1 Processor -- start 2024-05-08T05:41:48.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.198+0000 7fc051fa7640 1 -- start start 2024-05-08T05:41:48.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.198+0000 7f2792ffd640 1 -- 172.21.15.45:0/1930723754 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788037070 con 0x7f27940b42c0 2024-05-08T05:41:48.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.198+0000 7f2792ffd640 1 -- 172.21.15.45:0/1930723754 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2788032440 con 0x7f27940b42c0 2024-05-08T05:41:48.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.198+0000 7f2792ffd640 1 -- 172.21.15.45:0/1930723754 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788032810 con 0x7f27940b42c0 2024-05-08T05:41:48.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.198+0000 7f1acad90640 1 -- start start 2024-05-08T05:41:48.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.198+0000 7f62edd14640 1 -- 172.21.15.45:0/1696799324 >> v1:172.21.15.63:6789/0 conn(0x7f62e00e7900 legacy=0x7f62e0053dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.198+0000 7f62edd14640 1 -- 172.21.15.45:0/1696799324 >> v1:172.21.15.45:6789/0 conn(0x7f62e00e3450 legacy=0x7f62e0058160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.198+0000 7f62ecd12640 1 -- 172.21.15.45:0/1696799324 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3988711516 0 0) 0x7f62e00589b0 con 0x7f62e00e3450 2024-05-08T05:41:48.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.198+0000 7f62edd14640 1 -- 172.21.15.45:0/1696799324 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e0006240 con 0x7f62e00e3070 2024-05-08T05:41:48.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.198+0000 7f279955f640 1 -- 172.21.15.45:0/1930723754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27940b42c0 msgr2=0x7f279415ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.198+0000 7f279955f640 1 --2- 172.21.15.45:0/1930723754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27940b42c0 0x7f279415ba40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f27880097a0 tx=0x7f278802a230 comp rx=0 tx=0).stop 2024-05-08T05:41:48.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f279955f640 1 -- 172.21.15.45:0/1930723754 shutdown_connections 2024-05-08T05:41:48.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f279955f640 1 --2- 172.21.15.45:0/1930723754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27940b42c0 0x7f279415ba40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f279955f640 1 -- 172.21.15.45:0/1930723754 >> 172.21.15.45:0/1930723754 conn(0x7f27940b28b0 msgr2=0x7f27940b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:48.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f9e8d59a640 1 -- 172.21.15.45:0/2985550326 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9006e620 con 0x7f9e880e7970 2024-05-08T05:41:48.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f1acad90640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1abc006950 0x7f1abc0485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f1acad90640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1abc047c90 0x7f1abc04b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f1acad90640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc0e7550 0x7f1abc04bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f1acad90640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc0e7ab0 con 0x7f1abc0e7550 2024-05-08T05:41:48.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f1acad90640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc0e7930 con 0x7f1abc047c90 2024-05-08T05:41:48.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f1acad90640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1abc04f070 con 0x7f1abc006950 2024-05-08T05:41:48.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f9e8d59a640 1 -- 172.21.15.45:0/2985550326 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e9006e7c0 con 0x7f9e880e7970 2024-05-08T05:41:48.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f9e8d59a640 1 -- 172.21.15.45:0/2985550326 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9006ebf0 con 0x7f9e880e7970 2024-05-08T05:41:48.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f279955f640 1 -- 172.21.15.45:0/1930723754 shutdown_connections 2024-05-08T05:41:48.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7fa48bf0a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa4840d53d0 0x7fa4840d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7fa489ca6640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fa48415f420 0x7fa4840d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:53744/0 (socket says 172.21.15.45:53744) 2024-05-08T05:41:48.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7fa48bf0a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4841630b0 con 0x7fa4840d1970 2024-05-08T05:41:48.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7fa48bf0a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4840d3e70 con 0x7fa48415f420 2024-05-08T05:41:48.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7fa48bf0a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa4840d2550 con 0x7fa4840d53d0 2024-05-08T05:41:48.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7fa4894a5640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa4840d1970 0x7fa4840d4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:43998/0 (socket says 172.21.15.45:43998) 2024-05-08T05:41:48.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7fa4894a5640 1 -- 172.21.15.45:0/2636060330 learned_addr learned my addr 172.21.15.45:0/2636060330 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:48.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7fc051fa7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc04404f4d0 0x7fc04404f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f62edd14640 1 --2- 172.21.15.45:0/1696799324 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62e00e3070 0x7f62e003c1b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f62e8052040 tx=0x7f62e80a6700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8dc4d95ab6600c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7fa48a4a7640 1 --2- 172.21.15.45:0/2636060330 >> v2:172.21.15.45:3300/0 conn(0x7fa4840d53d0 0x7fa4840d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7fc051fa7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc044052a50 con 0x7fc044038ee0 2024-05-08T05:41:48.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f1ac958d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1abc047c90 0x7f1abc04b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f1aca58f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc0e7550 0x7f1abc04bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f1aca58f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc0e7550 0x7f1abc04bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51396/0 (socket says 172.21.15.45:51396) 2024-05-08T05:41:48.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f1aca58f640 1 -- 172.21.15.45:0/1968405017 learned_addr learned my addr 172.21.15.45:0/1968405017 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:48.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f1ac9d8e640 1 --2- 172.21.15.45:0/1968405017 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1abc006950 0x7f1abc0485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f279955f640 1 -- 172.21.15.45:0/1930723754 wait complete. 2024-05-08T05:41:48.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f1415ffb640 1 -- 172.21.15.45:0/121693877 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1408036070 con 0x7f141815bc50 2024-05-08T05:41:48.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f1415ffb640 1 -- 172.21.15.45:0/121693877 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1408031600 con 0x7f141815bc50 2024-05-08T05:41:48.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7fa488ca4640 1 -- 172.21.15.45:0/2636060330 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2332877306 0 0) 0x7fa4840d3e70 con 0x7fa48415f420 2024-05-08T05:41:48.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7fa488ca4640 1 -- 172.21.15.45:0/2636060330 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa470003520 con 0x7fa48415f420 2024-05-08T05:41:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f441f7fe640 1 -- 172.21.15.45:0/1163763499 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410037070 con 0x7f442006f060 2024-05-08T05:41:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f441f7fe640 1 -- 172.21.15.45:0/1163763499 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4410032440 con 0x7f442006f060 2024-05-08T05:41:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f441f7fe640 1 -- 172.21.15.45:0/1163763499 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44100327f0 con 0x7f442006f060 2024-05-08T05:41:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7fc04bfff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc04404f4d0 0x7fc04404f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7fc050fa5640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fc0440dfdc0 0x7fc044053dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:53754/0 (socket says 172.21.15.45:53754) 2024-05-08T05:41:48.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7f62ecd12640 1 -- 172.21.15.45:0/1696799324 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e806e620 con 0x7f62e00e3070 2024-05-08T05:41:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f62ecd12640 1 -- 172.21.15.45:0/1696799324 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e806e7c0 con 0x7f62e00e3070 2024-05-08T05:41:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f1415ffb640 1 -- 172.21.15.45:0/121693877 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14080319d0 con 0x7f141815bc50 2024-05-08T05:41:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7fc050fa5640 1 -- 172.21.15.45:0/417279806 learned_addr learned my addr 172.21.15.45:0/417279806 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:48.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.199+0000 7fc0517a6640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc044038ee0 0x7fc044039350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44004/0 (socket says 172.21.15.45:44004) 2024-05-08T05:41:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7fc051fa7640 1 -- 172.21.15.45:0/417279806 --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc044052d30 con 0x7fc0440dfdc0 2024-05-08T05:41:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f1ac958d640 1 -- 172.21.15.45:0/1968405017 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1abc006950 msgr2=0x7f1abc0485e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7fc051fa7640 1 -- 172.21.15.45:0/417279806 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc044053010 con 0x7fc04404f4d0 2024-05-08T05:41:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f1ac958d640 1 --2- 172.21.15.45:0/1968405017 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1abc006950 0x7f1abc0485e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f1ac958d640 1 -- 172.21.15.45:0/1968405017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc0e7550 msgr2=0x7f1abc04bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f1ac958d640 1 --2- 172.21.15.45:0/1968405017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc0e7550 0x7f1abc04bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f1ac958d640 1 -- 172.21.15.45:0/1968405017 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc038e70 con 0x7f1abc047c90 2024-05-08T05:41:48.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f9e8f59e640 1 -- 172.21.15.45:0/2985550326 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e880e7970 msgr2=0x7f9e88053790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f9e8f59e640 1 --2- 172.21.15.45:0/2985550326 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e880e7970 0x7f9e88053790 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9e90000b60 tx=0x7f9e900a6770 comp rx=0 tx=0).stop 2024-05-08T05:41:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f9e8f59e640 1 -- 172.21.15.45:0/2985550326 shutdown_connections 2024-05-08T05:41:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f9e8f59e640 1 --2- 172.21.15.45:0/2985550326 >> v2:172.21.15.63:3300/0 conn(0x7f9e88007180 0x7f9e88053cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f9e8f59e640 1 --2- 172.21.15.45:0/2985550326 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e880e7970 0x7f9e88053790 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f9e8f59e640 1 -- 172.21.15.45:0/2985550326 >> 172.21.15.45:0/2985550326 conn(0x7f9e88055bc0 msgr2=0x7f9e88055fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f141d872640 1 -- 172.21.15.45:0/121693877 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f141815bc50 msgr2=0x7f14180b5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f141d872640 1 --2- 172.21.15.45:0/121693877 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f141815bc50 0x7f14180b5ba0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1408009e50 tx=0x7f140802a680 comp rx=0 tx=0).stop 2024-05-08T05:41:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f141d872640 1 -- 172.21.15.45:0/121693877 shutdown_connections 2024-05-08T05:41:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f141d872640 1 --2- 172.21.15.45:0/121693877 >> v2:172.21.15.45:3300/0 conn(0x7f14180732c0 0x7f1418073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f141d872640 1 --2- 172.21.15.45:0/121693877 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f141815bc50 0x7f14180b5ba0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f141d872640 1 -- 172.21.15.45:0/121693877 >> 172.21.15.45:0/121693877 conn(0x7f14180b3c40 msgr2=0x7f14180b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f1ac9d8e640 1 --2- 172.21.15.45:0/1968405017 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1abc006950 0x7f1abc0485e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7fa48a4a7640 1 -- 172.21.15.45:0/2636060330 >> v1:172.21.15.63:6789/0 conn(0x7fa48415f420 legacy=0x7fa4840d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7fa48a4a7640 1 -- 172.21.15.45:0/2636060330 >> v1:172.21.15.45:6789/0 conn(0x7fa4840d1970 legacy=0x7fa4840d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7fa48a4a7640 1 -- 172.21.15.45:0/2636060330 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4840d2c80 con 0x7fa4840d53d0 2024-05-08T05:41:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f1ac958d640 1 --2- 172.21.15.45:0/1968405017 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1abc047c90 0x7f1abc04b0c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00dad0 tx=0x7f1ac00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f62ecd12640 1 -- 172.21.15.45:0/1696799324 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e806e620 con 0x7f62e00e3070 2024-05-08T05:41:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f9e8f59e640 1 -- 172.21.15.45:0/2985550326 shutdown_connections 2024-05-08T05:41:48.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f9e8f59e640 1 -- 172.21.15.45:0/2985550326 wait complete. 2024-05-08T05:41:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.201+0000 7f62eff78640 1 -- 172.21.15.45:0/1696799324 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62e00e3070 msgr2=0x7f62e003c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.201+0000 7f62eff78640 1 --2- 172.21.15.45:0/1696799324 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62e00e3070 0x7f62e003c1b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f62e8052040 tx=0x7f62e80a6700 comp rx=0 tx=0).stop 2024-05-08T05:41:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.201+0000 7f62eff78640 1 -- 172.21.15.45:0/1696799324 shutdown_connections 2024-05-08T05:41:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.201+0000 7f62eff78640 1 --2- 172.21.15.45:0/1696799324 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62e00e3070 0x7f62e003c1b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.201+0000 7f62eff78640 1 -- 172.21.15.45:0/1696799324 >> 172.21.15.45:0/1696799324 conn(0x7f62e003aa70 msgr2=0x7f62e003ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.201+0000 7fc04bfff640 1 -- 172.21.15.45:0/417279806 >> v1:172.21.15.63:6789/0 conn(0x7fc0440dfdc0 legacy=0x7fc044053dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.201+0000 7fc04bfff640 1 -- 172.21.15.45:0/417279806 >> v1:172.21.15.45:6789/0 conn(0x7fc044038ee0 legacy=0x7fc044039350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:48.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.201+0000 7fc04bfff640 1 -- 172.21.15.45:0/417279806 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0440532f0 con 0x7fc04404f4d0 2024-05-08T05:41:48.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.201+0000 7f1aba7fc640 1 -- 172.21.15.45:0/1968405017 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0040020 con 0x7f1abc047c90 2024-05-08T05:41:48.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.201+0000 7f279955f640 1 Processor -- start 2024-05-08T05:41:48.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.201+0000 7f1acad90640 1 -- 172.21.15.45:0/1968405017 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1abc039150 con 0x7f1abc047c90 2024-05-08T05:41:48.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f141d872640 1 -- 172.21.15.45:0/121693877 shutdown_connections 2024-05-08T05:41:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.200+0000 7f141d872640 1 -- 172.21.15.45:0/121693877 wait complete. 2024-05-08T05:41:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.201+0000 7f1aba7fc640 1 -- 172.21.15.45:0/1968405017 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ac0038040 con 0x7f1abc047c90 2024-05-08T05:41:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.201+0000 7f1acad90640 1 -- 172.21.15.45:0/1968405017 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1abc039710 con 0x7f1abc047c90 2024-05-08T05:41:48.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.201+0000 7f279955f640 1 -- start start 2024-05-08T05:41:48.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.201+0000 7f4427028640 1 -- 172.21.15.45:0/1163763499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f442006f060 msgr2=0x7f442006e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.201+0000 7f4427028640 1 --2- 172.21.15.45:0/1163763499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f442006f060 0x7f442006e0f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4410009870 tx=0x7f441002a1e0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.201+0000 7f1aba7fc640 1 -- 172.21.15.45:0/1968405017 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0037070 con 0x7f1abc047c90 2024-05-08T05:41:48.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7f9e8f59e640 1 Processor -- start 2024-05-08T05:41:48.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7f141d872640 1 Processor -- start 2024-05-08T05:41:48.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.201+0000 7fa48a4a7640 1 --2- 172.21.15.45:0/2636060330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4840d53d0 0x7fa4840d1e20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009720 tx=0x7fa47c02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9533e8bcb11c6239 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7fa488ca4640 1 -- 172.21.15.45:0/2636060330 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c037070 con 0x7fa4840d53d0 2024-05-08T05:41:48.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7fa488ca4640 1 -- 172.21.15.45:0/2636060330 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47c02ac60 con 0x7fa4840d53d0 2024-05-08T05:41:48.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7fa488ca4640 1 -- 172.21.15.45:0/2636060330 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480001a70 con 0x7fa4840d53d0 2024-05-08T05:41:48.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7f4427028640 1 -- 172.21.15.45:0/1163763499 shutdown_connections 2024-05-08T05:41:48.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7f4427028640 1 --2- 172.21.15.45:0/1163763499 >> v2:172.21.15.63:3300/0 conn(0x7f44200d0530 0x7f44200b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7f4427028640 1 --2- 172.21.15.45:0/1163763499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f442006f060 0x7f442006e0f0 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4410009870 tx=0x7f441002a1e0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7f4427028640 1 -- 172.21.15.45:0/1163763499 >> 172.21.15.45:0/1163763499 conn(0x7f442006c3a0 msgr2=0x7f442006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7fc04bfff640 1 --2- 172.21.15.45:0/417279806 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc04404f4d0 0x7fc04404f910 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc0380097d0 tx=0x7fc03802a1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d311555ce1b4ace0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7f62eff78640 1 -- 172.21.15.45:0/1696799324 shutdown_connections 2024-05-08T05:41:48.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7f9e8f59e640 1 -- start start 2024-05-08T05:41:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7f279955f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2794070050 0x7f2794155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7f279955f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27940b42c0 0x7f2794155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7f279955f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f279415bf80 0x7f279406e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7f279955f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f279406f990 con 0x7f27940b42c0 2024-05-08T05:41:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7f279955f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f279415ed90 con 0x7f2794070050 2024-05-08T05:41:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7f279955f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f279406fb10 con 0x7f279415bf80 2024-05-08T05:41:48.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7f62eff78640 1 -- 172.21.15.45:0/1696799324 wait complete. 2024-05-08T05:41:48.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.203+0000 7f2798d5e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f279415bf80 0x7f279406e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.203+0000 7f1acad90640 1 -- 172.21.15.45:0/1968405017 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1abc0052b0 con 0x7f1abc047c90 2024-05-08T05:41:48.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.203+0000 7f2793fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2794070050 0x7f2794155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.203+0000 7f2793fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2794070050 0x7f2794155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:47348/0 (socket says 172.21.15.45:47348) 2024-05-08T05:41:48.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.203+0000 7f2793fff640 1 -- 172.21.15.45:0/4126468900 learned_addr learned my addr 172.21.15.45:0/4126468900 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:48.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7f141d872640 1 -- start start 2024-05-08T05:41:48.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f9e8f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e88007180 0x7f9e88037df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f9e8f59e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e880e7590 0x7f9e881709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f9e8f59e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e88039520 0x7f9e8803aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f9e8f59e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9e880515b0 con 0x7f9e88007180 2024-05-08T05:41:48.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f9e8f59e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88055800 con 0x7f9e880e7590 2024-05-08T05:41:48.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f9e8f59e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9e88170f20 con 0x7f9e88039520 2024-05-08T05:41:48.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f9e8e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e88007180 0x7f9e88037df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f9e8e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e88007180 0x7f9e88037df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51422/0 (socket says 172.21.15.45:51422) 2024-05-08T05:41:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f9e8e59c640 1 -- 172.21.15.45:0/1492625572 learned_addr learned my addr 172.21.15.45:0/1492625572 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f9e8ed9d640 1 --2- 172.21.15.45:0/1492625572 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e88039520 0x7f9e8803aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f9e8dd9b640 1 --2- 172.21.15.45:0/1492625572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e880e7590 0x7f9e881709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.203+0000 7f2793fff640 1 -- 172.21.15.45:0/4126468900 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f279415bf80 msgr2=0x7f279406e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.203+0000 7f2793fff640 1 --2- 172.21.15.45:0/4126468900 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f279415bf80 0x7f279406e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.203+0000 7f2793fff640 1 -- 172.21.15.45:0/4126468900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27940b42c0 msgr2=0x7f2794155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.203+0000 7f2793fff640 1 --2- 172.21.15.45:0/4126468900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27940b42c0 0x7f2794155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.203+0000 7f2793fff640 1 -- 172.21.15.45:0/4126468900 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2794156360 con 0x7f2794070050 2024-05-08T05:41:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f2793fff640 1 --2- 172.21.15.45:0/4126468900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2794070050 0x7f2794155630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f278c00c780 tx=0x7f278c00cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f2790ff9640 1 -- 172.21.15.45:0/4126468900 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c003b70 con 0x7f2794070050 2024-05-08T05:41:48.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f2790ff9640 1 -- 172.21.15.45:0/4126468900 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f278c003d10 con 0x7f2794070050 2024-05-08T05:41:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f2790ff9640 1 -- 172.21.15.45:0/4126468900 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c00aa70 con 0x7f2794070050 2024-05-08T05:41:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f279955f640 1 -- 172.21.15.45:0/4126468900 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279409e620 con 0x7f2794070050 2024-05-08T05:41:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f279955f640 1 -- 172.21.15.45:0/4126468900 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2794159590 con 0x7f2794070050 2024-05-08T05:41:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f141d872640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f141806f540 0x7f141815d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f141d872640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14180732c0 0x7f141814caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f141d872640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f141815bc50 0x7f1418152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f141d872640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14180b60e0 con 0x7f14180732c0 2024-05-08T05:41:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f141d872640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f14180705c0 con 0x7f141815bc50 2024-05-08T05:41:48.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f141d872640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f14180b6260 con 0x7f141806f540 2024-05-08T05:41:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f14167fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14180732c0 0x7f141814caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f1416ffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f141806f540 0x7f141815d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f14167fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14180732c0 0x7f141814caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51428/0 (socket says 172.21.15.45:51428) 2024-05-08T05:41:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f14167fc640 1 -- 172.21.15.45:0/3449107450 learned_addr learned my addr 172.21.15.45:0/3449107450 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f14177fe640 1 --2- 172.21.15.45:0/3449107450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f141815bc50 0x7f1418152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.203+0000 7fa48bf0a640 1 -- 172.21.15.45:0/2636060330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4840d53d0 msgr2=0x7fa4840d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.203+0000 7fa48bf0a640 1 --2- 172.21.15.45:0/2636060330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4840d53d0 0x7fa4840d1e20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009720 tx=0x7fa47c02a150 comp rx=0 tx=0).stop 2024-05-08T05:41:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7fa48bf0a640 1 -- 172.21.15.45:0/2636060330 shutdown_connections 2024-05-08T05:41:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7fa48bf0a640 1 --2- 172.21.15.45:0/2636060330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4840d53d0 0x7fa4840d1e20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7fa48bf0a640 1 -- 172.21.15.45:0/2636060330 >> 172.21.15.45:0/2636060330 conn(0x7fa48408eac0 msgr2=0x7fa48409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.203+0000 7f4427028640 1 -- 172.21.15.45:0/1163763499 shutdown_connections 2024-05-08T05:41:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7fc04b7fe640 1 -- 172.21.15.45:0/417279806 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038037070 con 0x7fc04404f4d0 2024-05-08T05:41:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7fc04b7fe640 1 -- 172.21.15.45:0/417279806 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc038032440 con 0x7fc04404f4d0 2024-05-08T05:41:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.202+0000 7fc04b7fe640 1 -- 172.21.15.45:0/417279806 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038032810 con 0x7fc04404f4d0 2024-05-08T05:41:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.203+0000 7fc051fa7640 1 -- 172.21.15.45:0/417279806 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc04404f4d0 msgr2=0x7fc04404f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.203+0000 7fc051fa7640 1 --2- 172.21.15.45:0/417279806 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc04404f4d0 0x7fc04404f910 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc0380097d0 tx=0x7fc03802a1f0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7fc051fa7640 1 -- 172.21.15.45:0/417279806 shutdown_connections 2024-05-08T05:41:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7fc051fa7640 1 --2- 172.21.15.45:0/417279806 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc04404f4d0 0x7fc04404f910 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7fc051fa7640 1 -- 172.21.15.45:0/417279806 >> 172.21.15.45:0/417279806 conn(0x7fc04400a400 msgr2=0x7fc04400a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.203+0000 7f62eff78640 1 Processor -- start 2024-05-08T05:41:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.203+0000 7f62eff78640 1 -- start start 2024-05-08T05:41:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7fc051fa7640 1 -- 172.21.15.45:0/417279806 shutdown_connections 2024-05-08T05:41:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f9e8e59c640 1 -- 172.21.15.45:0/1492625572 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e88039520 msgr2=0x7f9e8803aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f9e8e59c640 1 --2- 172.21.15.45:0/1492625572 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e88039520 0x7f9e8803aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f9e8e59c640 1 -- 172.21.15.45:0/1492625572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e880e7590 msgr2=0x7f9e881709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f9e8e59c640 1 --2- 172.21.15.45:0/1492625572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e880e7590 0x7f9e881709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f9e8e59c640 1 -- 172.21.15.45:0/1492625572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e88038480 con 0x7f9e88007180 2024-05-08T05:41:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f9e8dd9b640 1 --2- 172.21.15.45:0/1492625572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e880e7590 0x7f9e881709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7fc051fa7640 1 -- 172.21.15.45:0/417279806 wait complete. 2024-05-08T05:41:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f14167fc640 1 -- 172.21.15.45:0/3449107450 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f141806f540 msgr2=0x7f141815d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f14167fc640 1 --2- 172.21.15.45:0/3449107450 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f141806f540 0x7f141815d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f14167fc640 1 -- 172.21.15.45:0/3449107450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f141815bc50 msgr2=0x7f1418152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f14167fc640 1 --2- 172.21.15.45:0/3449107450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f141815bc50 0x7f1418152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f14167fc640 1 -- 172.21.15.45:0/3449107450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141815c1b0 con 0x7f14180732c0 2024-05-08T05:41:48.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.204+0000 7f1aba7fc640 1 -- 172.21.15.45:0/1968405017 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1ac0043020 con 0x7f1abc047c90 2024-05-08T05:41:48.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f9e8e59c640 1 --2- 172.21.15.45:0/1492625572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e88007180 0x7f9e88037df0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400c530 tx=0x7f9e8400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f62eff78640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62e00e3070 0x7f62e00d9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f14167fc640 1 --2- 172.21.15.45:0/3449107450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14180732c0 0x7f141814caa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f140000c410 tx=0x7f140000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.203+0000 7f4427028640 1 -- 172.21.15.45:0/1163763499 wait complete. 2024-05-08T05:41:48.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f62eff78640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62e00e3450 0x7f62e0050a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f62eff78640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62e00e7900 0x7f62e0050fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f62eff78640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f62e0054650 con 0x7f62e00e3070 2024-05-08T05:41:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f1aba7fc640 1 --2- 172.21.15.45:0/1968405017 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1aa80763d0 0x7f1aa8078890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f1aba7fc640 1 -- 172.21.15.45:0/1968405017 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1ac00b76a0 con 0x7f1abc047c90 2024-05-08T05:41:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f4427028640 1 Processor -- start 2024-05-08T05:41:48.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f62ed513640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62e00e3450 0x7f62e0050a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f62ed513640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62e00e3450 0x7f62e0050a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:33324/0 (socket says 172.21.15.45:33324) 2024-05-08T05:41:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f62ed513640 1 -- 172.21.15.45:0/2910444559 learned_addr learned my addr 172.21.15.45:0/2910444559 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f62eff78640 1 -- 172.21.15.45:0/2910444559 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f62e00544d0 con 0x7f62e00e7900 2024-05-08T05:41:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f62eff78640 1 -- 172.21.15.45:0/2910444559 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f62e00547d0 con 0x7f62e00e3450 2024-05-08T05:41:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f9e7effd640 1 -- 172.21.15.45:0/1492625572 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e840052d0 con 0x7f9e88007180 2024-05-08T05:41:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f9e8f59e640 1 -- 172.21.15.45:0/1492625572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e88038760 con 0x7f9e88007180 2024-05-08T05:41:48.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f9e8f59e640 1 -- 172.21.15.45:0/1492625572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e88038d20 con 0x7f9e88007180 2024-05-08T05:41:48.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.206+0000 7f1ac9d8e640 1 --2- 172.21.15.45:0/1968405017 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1aa80763d0 0x7f1aa8078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.206+0000 7f140ffff640 1 -- 172.21.15.45:0/3449107450 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400017070 con 0x7f14180732c0 2024-05-08T05:41:48.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.206+0000 7fa48bf0a640 1 -- 172.21.15.45:0/2636060330 shutdown_connections 2024-05-08T05:41:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.206+0000 7fa48bf0a640 1 -- 172.21.15.45:0/2636060330 wait complete. 2024-05-08T05:41:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.205+0000 7f4427028640 1 -- start start 2024-05-08T05:41:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f62edd14640 1 --2- 172.21.15.45:0/2910444559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62e00e3070 0x7f62e00d9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f62ed513640 1 -- 172.21.15.45:0/2910444559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62e00e7900 msgr2=0x7f62e0050fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f62ee515640 1 --2- 172.21.15.45:0/2910444559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62e00e7900 0x7f62e0050fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f62ed513640 1 --2- 172.21.15.45:0/2910444559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62e00e7900 0x7f62e0050fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f62ed513640 1 -- 172.21.15.45:0/2910444559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62e00e3070 msgr2=0x7f62e00d9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f62ed513640 1 --2- 172.21.15.45:0/2910444559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62e00e3070 0x7f62e00d9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f62ed513640 1 -- 172.21.15.45:0/2910444559 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e0051920 con 0x7f62e00e3450 2024-05-08T05:41:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f279955f640 1 -- 172.21.15.45:0/4126468900 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2774005290 con 0x7f2794070050 2024-05-08T05:41:48.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f62edd14640 1 --2- 172.21.15.45:0/2910444559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62e00e3070 0x7f62e00d9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:48.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7fc051fa7640 1 Processor -- start 2024-05-08T05:41:48.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f62ed513640 1 --2- 172.21.15.45:0/2910444559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62e00e3450 0x7f62e0050a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f62e400c440 tx=0x7f62e400c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.206+0000 7f9e7effd640 1 -- 172.21.15.45:0/1492625572 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e84005470 con 0x7f9e88007180 2024-05-08T05:41:48.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.206+0000 7f9e7effd640 1 -- 172.21.15.45:0/1492625572 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e840052d0 con 0x7f9e88007180 2024-05-08T05:41:48.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f9e8f59e640 1 -- 172.21.15.45:0/1492625572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e74005290 con 0x7f9e88007180 2024-05-08T05:41:48.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.206+0000 7f141d872640 1 -- 172.21.15.45:0/3449107450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141815c490 con 0x7f14180732c0 2024-05-08T05:41:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.206+0000 7f141d872640 1 -- 172.21.15.45:0/3449107450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f141815ca50 con 0x7f14180732c0 2024-05-08T05:41:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f140ffff640 1 -- 172.21.15.45:0/3449107450 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14000035f0 con 0x7f14180732c0 2024-05-08T05:41:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7fc051fa7640 1 -- start start 2024-05-08T05:41:48.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f4427028640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f442006fe30 0x7f44200b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f62de7fc640 1 -- 172.21.15.45:0/2910444559 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4018070 con 0x7f62e00e3450 2024-05-08T05:41:48.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f62de7fc640 1 -- 172.21.15.45:0/2910444559 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e4007500 con 0x7f62e00e3450 2024-05-08T05:41:48.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f4427028640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44200d0530 0x7f44201f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f4427028640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f442015d490 0x7f44200ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f4427028640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4420161530 con 0x7f44200d0530 2024-05-08T05:41:48.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f4427028640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4420161250 con 0x7f442015d490 2024-05-08T05:41:48.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f4427028640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4420161810 con 0x7f442006fe30 2024-05-08T05:41:48.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.208+0000 7f441ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44200d0530 0x7f44201f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.208+0000 7f441ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44200d0530 0x7f44201f4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51456/0 (socket says 172.21.15.45:51456) 2024-05-08T05:41:48.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.208+0000 7f441ffff640 1 -- 172.21.15.45:0/76616607 learned_addr learned my addr 172.21.15.45:0/76616607 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:48.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.208+0000 7fa48bf0a640 1 Processor -- start 2024-05-08T05:41:48.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.208+0000 7f4424dc4640 1 --2- 172.21.15.45:0/76616607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f442006fe30 0x7f44200b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.208+0000 7f441ffff640 1 -- 172.21.15.45:0/76616607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f442006fe30 msgr2=0x7f44200b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.208+0000 7f441ffff640 1 --2- 172.21.15.45:0/76616607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f442006fe30 0x7f44200b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.208+0000 7f441ffff640 1 -- 172.21.15.45:0/76616607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f442015d490 msgr2=0x7f44200ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.208+0000 7f441ffff640 1 --2- 172.21.15.45:0/76616607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f442015d490 0x7f44200ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.208+0000 7f441ffff640 1 -- 172.21.15.45:0/76616607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44201f4f80 con 0x7f44200d0530 2024-05-08T05:41:48.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.208+0000 7fa48bf0a640 1 -- start start 2024-05-08T05:41:48.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.208+0000 7f441ffff640 1 --2- 172.21.15.45:0/76616607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44200d0530 0x7f44201f4810 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f441400c410 tx=0x7f441400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.207+0000 7f140dffb640 1 -- 172.21.15.45:0/3449107450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13fc005290 con 0x7f14180732c0 2024-05-08T05:41:48.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.208+0000 7f140ffff640 1 -- 172.21.15.45:0/3449107450 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400003a20 con 0x7f14180732c0 2024-05-08T05:41:48.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.208+0000 7f62de7fc640 1 -- 172.21.15.45:0/2910444559 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4004a20 con 0x7f62e00e3450 2024-05-08T05:41:48.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.209+0000 7f62eff78640 1 -- 172.21.15.45:0/2910444559 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e0051c00 con 0x7f62e00e3450 2024-05-08T05:41:48.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.208+0000 7f1ac9d8e640 1 --2- 172.21.15.45:0/1968405017 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1aa80763d0 0x7f1aa8078890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009560 tx=0x7f1ab4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.209+0000 7f62eff78640 1 -- 172.21.15.45:0/2910444559 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e00521c0 con 0x7f62e00e3450 2024-05-08T05:41:48.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.209+0000 7fc051fa7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc044038ee0 0x7fc0440d7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.209+0000 7fc051fa7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc04404f4d0 0x7fc0440e0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.209+0000 7fc051fa7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0440dfdc0 0x7fc0440da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.209+0000 7f140ffff640 1 -- 172.21.15.45:0/3449107450 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f140001a070 con 0x7f14180732c0 2024-05-08T05:41:48.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.209+0000 7fc051fa7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc0440e01a0 con 0x7fc0440dfdc0 2024-05-08T05:41:48.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.209+0000 7fc051fa7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc0440538b0 con 0x7fc04404f4d0 2024-05-08T05:41:48.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.209+0000 7fc051fa7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc0440e0320 con 0x7fc044038ee0 2024-05-08T05:41:48.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7fc0517a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0440dfdc0 0x7fc0440da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7f140ffff640 1 --2- 172.21.15.45:0/3449107450 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f13f80763a0 0x7f13f8078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7fc0517a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0440dfdc0 0x7fc0440da560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51468/0 (socket says 172.21.15.45:51468) 2024-05-08T05:41:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7fc0517a6640 1 -- 172.21.15.45:0/3082158240 learned_addr learned my addr 172.21.15.45:0/3082158240 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:48.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7fc050fa5640 1 --2- 172.21.15.45:0/3082158240 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc044038ee0 0x7fc0440d7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7f140ffff640 1 -- 172.21.15.45:0/3449107450 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1400096da0 con 0x7f14180732c0 2024-05-08T05:41:48.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7fa48bf0a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4840d1970 0x7fa4841e8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7fc04bfff640 1 --2- 172.21.15.45:0/3082158240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc04404f4d0 0x7fc0440e0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7fa489ca6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4840d1970 0x7fa4841e8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7fc0517a6640 1 -- 172.21.15.45:0/3082158240 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc044038ee0 msgr2=0x7fc0440d7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7fc0517a6640 1 --2- 172.21.15.45:0/3082158240 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc044038ee0 0x7fc0440d7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7fc0517a6640 1 -- 172.21.15.45:0/3082158240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc04404f4d0 msgr2=0x7fc0440e0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7fc0517a6640 1 --2- 172.21.15.45:0/3082158240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc04404f4d0 0x7fc0440e0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7fc0517a6640 1 -- 172.21.15.45:0/3082158240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0440e2050 con 0x7fc0440dfdc0 2024-05-08T05:41:48.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7fa489ca6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4840d1970 0x7fa4841e8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51476/0 (socket says 172.21.15.45:51476) 2024-05-08T05:41:48.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7fa489ca6640 1 -- 172.21.15.45:0/1448268523 learned_addr learned my addr 172.21.15.45:0/1448268523 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:48.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7fa48bf0a640 1 --2- 172.21.15.45:0/1448268523 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4840d53d0 0x7fa4841e90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7fa48bf0a640 1 --2- 172.21.15.45:0/1448268523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4840c9ff0 0x7fa4840ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7f62eff78640 1 -- 172.21.15.45:0/2910444559 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e00052f0 con 0x7f62e00e3450 2024-05-08T05:41:48.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7fc0517a6640 1 --2- 172.21.15.45:0/3082158240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0440dfdc0 0x7fc0440da560 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc04c02e0e0 tx=0x7fc04c0756c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7fa48bf0a640 1 -- 172.21.15.45:0/1448268523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa4840d2830 con 0x7fa4840d1970 2024-05-08T05:41:48.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.211+0000 7fa4894a5640 1 --2- 172.21.15.45:0/1448268523 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4840d53d0 0x7fa4841e90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.210+0000 7f9e7effd640 1 -- 172.21.15.45:0/1492625572 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9e84012630 con 0x7f9e88007180 2024-05-08T05:41:48.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.211+0000 7f1416ffd640 1 --2- 172.21.15.45:0/3449107450 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f13f80763a0 0x7f13f8078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.211+0000 7fa48a4a7640 1 --2- 172.21.15.45:0/1448268523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4840c9ff0 0x7fa4840ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.211+0000 7fa48bf0a640 1 -- 172.21.15.45:0/1448268523 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa4840d3240 con 0x7fa4840c9ff0 2024-05-08T05:41:48.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.211+0000 7fa48bf0a640 1 -- 172.21.15.45:0/1448268523 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa4840cfb20 con 0x7fa4840d53d0 2024-05-08T05:41:48.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.211+0000 7f1aba7fc640 1 -- 172.21.15.45:0/1968405017 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1ac004a030 con 0x7f1abc047c90 2024-05-08T05:41:48.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.211+0000 7f9e7effd640 1 --2- 172.21.15.45:0/1492625572 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9e70076500 0x7f9e700789c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.211+0000 7f62de7fc640 1 -- 172.21.15.45:0/2910444559 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f62e401b070 con 0x7f62e00e3450 2024-05-08T05:41:48.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.211+0000 7f9e8dd9b640 1 --2- 172.21.15.45:0/1492625572 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9e70076500 0x7f9e700789c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.211+0000 7f1416ffd640 1 --2- 172.21.15.45:0/3449107450 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f13f80763a0 0x7f13f8078860 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1408033000 tx=0x7f1408009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.211+0000 7f9e7effd640 1 -- 172.21.15.45:0/1492625572 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9e84097b90 con 0x7f9e88007180 2024-05-08T05:41:48.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.211+0000 7fa489ca6640 1 -- 172.21.15.45:0/1448268523 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4840d53d0 msgr2=0x7fa4841e90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.211+0000 7fa489ca6640 1 --2- 172.21.15.45:0/1448268523 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4840d53d0 0x7fa4841e90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.211+0000 7fa489ca6640 1 -- 172.21.15.45:0/1448268523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4840c9ff0 msgr2=0x7fa4840ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.211+0000 7fa489ca6640 1 --2- 172.21.15.45:0/1448268523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4840c9ff0 0x7fa4840ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.212+0000 7fa489ca6640 1 -- 172.21.15.45:0/1448268523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4840cafb0 con 0x7fa4840d1970 2024-05-08T05:41:48.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.212+0000 7fa4894a5640 1 --2- 172.21.15.45:0/1448268523 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4840d53d0 0x7fa4841e90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:48.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.212+0000 7f62de7fc640 1 --2- 172.21.15.45:0/2910444559 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f62cc076350 0x7f62cc078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.212+0000 7f9e8dd9b640 1 --2- 172.21.15.45:0/1492625572 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9e70076500 0x7f9e700789c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9e88050ca0 tx=0x7f9e900aa000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.212+0000 7f62de7fc640 1 -- 172.21.15.45:0/2910444559 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f62e4096c40 con 0x7f62e00e3450 2024-05-08T05:41:48.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.212+0000 7f62edd14640 1 --2- 172.21.15.45:0/2910444559 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f62cc076350 0x7f62cc078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.212+0000 7fa489ca6640 1 --2- 172.21.15.45:0/1448268523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4840d1970 0x7fa4841e8b80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa48000cdc0 tx=0x7fa48000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.212+0000 7f62edd14640 1 --2- 172.21.15.45:0/2910444559 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f62cc076350 0x7f62cc078810 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f62e0055940 tx=0x7f62e80aa000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.213+0000 7f2790ff9640 1 -- 172.21.15.45:0/4126468900 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f278c00ac10 con 0x7f2794070050 2024-05-08T05:41:48.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.213+0000 7f2790ff9640 1 --2- 172.21.15.45:0/4126468900 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2780076320 0x7f27800787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.214+0000 7f2790ff9640 1 -- 172.21.15.45:0/4126468900 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f278c098200 con 0x7f2794070050 2024-05-08T05:41:48.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.214+0000 7f27937fe640 1 --2- 172.21.15.45:0/4126468900 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2780076320 0x7f27800787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.214+0000 7f27937fe640 1 --2- 172.21.15.45:0/4126468900 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2780076320 0x7f27800787e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2788034000 tx=0x7f278802c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.216+0000 7f441d7fa640 1 -- 172.21.15.45:0/76616607 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414017070 con 0x7f44200d0530 2024-05-08T05:41:48.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.216+0000 7fc0497fa640 1 -- 172.21.15.45:0/3082158240 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c074720 con 0x7fc0440dfdc0 2024-05-08T05:41:48.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.216+0000 7f4427028640 1 -- 172.21.15.45:0/76616607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44201f5260 con 0x7f44200d0530 2024-05-08T05:41:48.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.216+0000 7fc051fa7640 1 -- 172.21.15.45:0/3082158240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0440e2330 con 0x7fc0440dfdc0 2024-05-08T05:41:48.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.216+0000 7fa47a7fc640 1 -- 172.21.15.45:0/1448268523 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480018070 con 0x7fa4840d1970 2024-05-08T05:41:48.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.216+0000 7fc051fa7640 1 -- 172.21.15.45:0/3082158240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0440e28f0 con 0x7fc0440dfdc0 2024-05-08T05:41:48.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.216+0000 7f4427028640 1 -- 172.21.15.45:0/76616607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f442015f2d0 con 0x7f44200d0530 2024-05-08T05:41:48.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.216+0000 7fa48bf0a640 1 -- 172.21.15.45:0/1448268523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4840cb290 con 0x7fa4840d1970 2024-05-08T05:41:48.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.216+0000 7f9e7effd640 1 -- 172.21.15.45:0/1492625572 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9e84061290 con 0x7f9e88007180 2024-05-08T05:41:48.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.216+0000 7fa48bf0a640 1 -- 172.21.15.45:0/1448268523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa484149900 con 0x7fa4840d1970 2024-05-08T05:41:48.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.217+0000 7f62de7fc640 1 -- 172.21.15.45:0/2910444559 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f62e40602d0 con 0x7f62e00e3450 2024-05-08T05:41:48.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.217+0000 7fc051fa7640 1 -- 172.21.15.45:0/3082158240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc044005710 con 0x7fc0440dfdc0 2024-05-08T05:41:48.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.217+0000 7f4427028640 1 -- 172.21.15.45:0/76616607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f442015f4d0 con 0x7f44200d0530 2024-05-08T05:41:48.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.217+0000 7f2790ff9640 1 -- 172.21.15.45:0/4126468900 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f278c061890 con 0x7f2794070050 2024-05-08T05:41:48.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.218+0000 7fa48bf0a640 1 -- 172.21.15.45:0/1448268523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa484149b00 con 0x7fa4840d1970 2024-05-08T05:41:48.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.225+0000 7f140ffff640 1 -- 172.21.15.45:0/3449107450 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f140005fe60 con 0x7f14180732c0 2024-05-08T05:41:48.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.225+0000 7fc0497fa640 1 -- 172.21.15.45:0/3082158240 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc04c0748c0 con 0x7fc0440dfdc0 2024-05-08T05:41:48.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.225+0000 7fc0497fa640 1 -- 172.21.15.45:0/3082158240 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c094650 con 0x7fc0440dfdc0 2024-05-08T05:41:48.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.225+0000 7f441d7fa640 1 -- 172.21.15.45:0/76616607 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44140035f0 con 0x7f44200d0530 2024-05-08T05:41:48.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.225+0000 7f441d7fa640 1 -- 172.21.15.45:0/76616607 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414003a40 con 0x7f44200d0530 2024-05-08T05:41:48.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.228+0000 7fa47a7fc640 1 -- 172.21.15.45:0/1448268523 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa480004d60 con 0x7fa4840d1970 2024-05-08T05:41:48.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.230+0000 7fa47a7fc640 1 -- 172.21.15.45:0/1448268523 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48000ad40 con 0x7fa4840d1970 2024-05-08T05:41:48.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.230+0000 7fa47a7fc640 1 -- 172.21.15.45:0/1448268523 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa48001b070 con 0x7fa4840d1970 2024-05-08T05:41:48.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.229+0000 7f441d7fa640 1 -- 172.21.15.45:0/76616607 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f441401a070 con 0x7f44200d0530 2024-05-08T05:41:48.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.230+0000 7f441d7fa640 1 --2- 172.21.15.45:0/76616607 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f440c0762d0 0x7f440c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.230+0000 7f441d7fa640 1 -- 172.21.15.45:0/76616607 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4414097290 con 0x7f44200d0530 2024-05-08T05:41:48.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.229+0000 7fc0497fa640 1 -- 172.21.15.45:0/3082158240 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc04c0947f0 con 0x7fc0440dfdc0 2024-05-08T05:41:48.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.230+0000 7fc0497fa640 1 --2- 172.21.15.45:0/3082158240 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc0300762d0 0x7fc030078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.230+0000 7fc0497fa640 1 -- 172.21.15.45:0/3082158240 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc04c113c50 con 0x7fc0440dfdc0 2024-05-08T05:41:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.231+0000 7fa47a7fc640 1 --2- 172.21.15.45:0/1448268523 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa468076330 0x7fa4680787f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.231+0000 7fc050fa5640 1 --2- 172.21.15.45:0/3082158240 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc0300762d0 0x7fc030078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.231+0000 7fa4894a5640 1 --2- 172.21.15.45:0/1448268523 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa468076330 0x7fa4680787f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.231+0000 7f4424dc4640 1 --2- 172.21.15.45:0/76616607 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f440c0762d0 0x7f440c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:48.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.231+0000 7fa47a7fc640 1 -- 172.21.15.45:0/1448268523 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa480097490 con 0x7fa4840d1970 2024-05-08T05:41:48.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.232+0000 7fc050fa5640 1 --2- 172.21.15.45:0/3082158240 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc0300762d0 0x7fc030078790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc040000f60 tx=0x7fc040009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.232+0000 7fa4894a5640 1 --2- 172.21.15.45:0/1448268523 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa468076330 0x7fa4680787f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa4840cbb60 tx=0x7fa474009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.232+0000 7f4424dc4640 1 --2- 172.21.15.45:0/76616607 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f440c0762d0 0x7f440c078790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4410034000 tx=0x7f441002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:48.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.235+0000 7f441d7fa640 1 -- 172.21.15.45:0/76616607 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f442015f4d0 con 0x7f44200d0530 2024-05-08T05:41:48.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.243+0000 7fc0497fa640 1 -- 172.21.15.45:0/3082158240 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc04c09a020 con 0x7fc0440dfdc0 2024-05-08T05:41:48.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.252+0000 7fa47a7fc640 1 -- 172.21.15.45:0/1448268523 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa484149b00 con 0x7fa4840d1970 2024-05-08T05:41:48.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.644+0000 7f1acad90640 1 -- 172.21.15.45:0/1968405017 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1abc0d4770 con 0x7f1abc047c90 2024-05-08T05:41:48.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.644+0000 7f1aba7fc640 1 -- 172.21.15.45:0/1968405017 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1ac00805c0 con 0x7f1abc047c90 2024-05-08T05:41:48.645 INFO:teuthology.orchestra.run.smithi045.stdout:55834574854 2024-05-08T05:41:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.648+0000 7f1aa7fff640 1 -- 172.21.15.45:0/1968405017 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1aa80763d0 msgr2=0x7f1aa8078890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.648+0000 7f1aa7fff640 1 --2- 172.21.15.45:0/1968405017 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1aa80763d0 0x7f1aa8078890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009560 tx=0x7f1ab4009190 comp rx=0 tx=0).stop 2024-05-08T05:41:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.648+0000 7f1aa7fff640 1 -- 172.21.15.45:0/1968405017 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1abc047c90 msgr2=0x7f1abc04b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.648+0000 7f1aa7fff640 1 --2- 172.21.15.45:0/1968405017 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1abc047c90 0x7f1abc04b0c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00dad0 tx=0x7f1ac00046c0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.648+0000 7f1aa7fff640 1 -- 172.21.15.45:0/1968405017 shutdown_connections 2024-05-08T05:41:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.648+0000 7f1aa7fff640 1 --2- 172.21.15.45:0/1968405017 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1aa80763d0 0x7f1aa8078890 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.649+0000 7f1aa7fff640 1 --2- 172.21.15.45:0/1968405017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc0e7550 0x7f1abc04bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.649+0000 7f1aa7fff640 1 --2- 172.21.15.45:0/1968405017 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1abc047c90 0x7f1abc04b0c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.649+0000 7f1aa7fff640 1 --2- 172.21.15.45:0/1968405017 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1abc006950 0x7f1abc0485e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.649+0000 7f1aa7fff640 1 -- 172.21.15.45:0/1968405017 >> 172.21.15.45:0/1968405017 conn(0x7f1abc009fa0 msgr2=0x7f1abc0e8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.649+0000 7f1aa7fff640 1 -- 172.21.15.45:0/1968405017 shutdown_connections 2024-05-08T05:41:48.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.649+0000 7f1aa7fff640 1 -- 172.21.15.45:0/1968405017 wait complete. 2024-05-08T05:41:48.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.657+0000 7f62eff78640 1 -- 172.21.15.45:0/2910444559 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f62e003eb30 con 0x7f62e00e3450 2024-05-08T05:41:48.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.657+0000 7f62de7fc640 1 -- 172.21.15.45:0/2910444559 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f62e40682c0 con 0x7f62e00e3450 2024-05-08T05:41:48.658 INFO:teuthology.orchestra.run.smithi045.stdout:60129542152 2024-05-08T05:41:48.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.661+0000 7f62cbfff640 1 -- 172.21.15.45:0/2910444559 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f62cc076350 msgr2=0x7f62cc078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.661+0000 7f62cbfff640 1 --2- 172.21.15.45:0/2910444559 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f62cc076350 0x7f62cc078810 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f62e0055940 tx=0x7f62e80aa000 comp rx=0 tx=0).stop 2024-05-08T05:41:48.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.661+0000 7f62cbfff640 1 -- 172.21.15.45:0/2910444559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62e00e3450 msgr2=0x7f62e0050a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.661+0000 7f62cbfff640 1 --2- 172.21.15.45:0/2910444559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62e00e3450 0x7f62e0050a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f62e400c440 tx=0x7f62e400c900 comp rx=0 tx=0).stop 2024-05-08T05:41:48.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.661+0000 7f62cbfff640 1 -- 172.21.15.45:0/2910444559 shutdown_connections 2024-05-08T05:41:48.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.661+0000 7f62cbfff640 1 --2- 172.21.15.45:0/2910444559 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f62cc076350 0x7f62cc078810 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.661+0000 7f62cbfff640 1 --2- 172.21.15.45:0/2910444559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f62e00e3450 0x7f62e0050a60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.661+0000 7f62cbfff640 1 --2- 172.21.15.45:0/2910444559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62e00e7900 0x7f62e0050fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.661+0000 7f62cbfff640 1 --2- 172.21.15.45:0/2910444559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62e00e3070 0x7f62e00d9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.661+0000 7f62cbfff640 1 -- 172.21.15.45:0/2910444559 >> 172.21.15.45:0/2910444559 conn(0x7f62e003aa70 msgr2=0x7f62e00db780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:48.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.662+0000 7f62cbfff640 1 -- 172.21.15.45:0/2910444559 shutdown_connections 2024-05-08T05:41:48.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.662+0000 7f62cbfff640 1 -- 172.21.15.45:0/2910444559 wait complete. 2024-05-08T05:41:48.667 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-05-08T05:41:48.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.668+0000 7f140dffb640 1 -- 172.21.15.45:0/3449107450 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f13fc005580 con 0x7f14180732c0 2024-05-08T05:41:48.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.669+0000 7f140ffff640 1 -- 172.21.15.45:0/3449107450 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1400064160 con 0x7f14180732c0 2024-05-08T05:41:48.669 INFO:teuthology.orchestra.run.smithi045.stdout:55834574854 2024-05-08T05:41:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.671+0000 7f140dffb640 1 -- 172.21.15.45:0/3449107450 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f13f80763a0 msgr2=0x7f13f8078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.671+0000 7f140dffb640 1 --2- 172.21.15.45:0/3449107450 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f13f80763a0 0x7f13f8078860 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1408033000 tx=0x7f1408009000 comp rx=0 tx=0).stop 2024-05-08T05:41:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.671+0000 7f140dffb640 1 -- 172.21.15.45:0/3449107450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14180732c0 msgr2=0x7f141814caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.671+0000 7f140dffb640 1 --2- 172.21.15.45:0/3449107450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14180732c0 0x7f141814caa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f140000c410 tx=0x7f140000c8d0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.671+0000 7f140dffb640 1 -- 172.21.15.45:0/3449107450 shutdown_connections 2024-05-08T05:41:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.671+0000 7f140dffb640 1 --2- 172.21.15.45:0/3449107450 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f13f80763a0 0x7f13f8078860 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.671+0000 7f140dffb640 1 --2- 172.21.15.45:0/3449107450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f141815bc50 0x7f1418152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.671+0000 7f140dffb640 1 --2- 172.21.15.45:0/3449107450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14180732c0 0x7f141814caa0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.671+0000 7f140dffb640 1 --2- 172.21.15.45:0/3449107450 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f141806f540 0x7f141815d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.671+0000 7f140dffb640 1 -- 172.21.15.45:0/3449107450 >> 172.21.15.45:0/3449107450 conn(0x7f14180b3c40 msgr2=0x7f14180d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.671+0000 7f9e8f59e640 1 -- 172.21.15.45:0/1492625572 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f9e74005850 con 0x7f9e88007180 2024-05-08T05:41:48.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.671+0000 7f140dffb640 1 -- 172.21.15.45:0/3449107450 shutdown_connections 2024-05-08T05:41:48.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.671+0000 7f140dffb640 1 -- 172.21.15.45:0/3449107450 wait complete. 2024-05-08T05:41:48.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.672+0000 7f9e7effd640 1 -- 172.21.15.45:0/1492625572 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9e84069280 con 0x7f9e88007180 2024-05-08T05:41:48.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.672+0000 7f279955f640 1 -- 172.21.15.45:0/4126468900 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f2774005850 con 0x7f2794070050 2024-05-08T05:41:48.673 INFO:teuthology.orchestra.run.smithi045.stdout:55834574854 2024-05-08T05:41:48.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.672+0000 7f2790ff9640 1 -- 172.21.15.45:0/4126468900 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f278c061190 con 0x7f2794070050 2024-05-08T05:41:48.674 INFO:teuthology.orchestra.run.smithi045.stdout:60129542153 2024-05-08T05:41:48.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.675+0000 7f9e8f59e640 1 -- 172.21.15.45:0/1492625572 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9e70076500 msgr2=0x7f9e700789c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.675+0000 7f9e8f59e640 1 --2- 172.21.15.45:0/1492625572 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9e70076500 0x7f9e700789c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9e88050ca0 tx=0x7f9e900aa000 comp rx=0 tx=0).stop 2024-05-08T05:41:48.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.675+0000 7f9e8f59e640 1 -- 172.21.15.45:0/1492625572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e88007180 msgr2=0x7f9e88037df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.675+0000 7f9e8f59e640 1 --2- 172.21.15.45:0/1492625572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e88007180 0x7f9e88037df0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400c530 tx=0x7f9e8400c9f0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.675+0000 7f9e8f59e640 1 -- 172.21.15.45:0/1492625572 shutdown_connections 2024-05-08T05:41:48.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.675+0000 7f9e8f59e640 1 --2- 172.21.15.45:0/1492625572 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9e70076500 0x7f9e700789c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.675+0000 7f9e8f59e640 1 --2- 172.21.15.45:0/1492625572 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e88039520 0x7f9e8803aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.675+0000 7f9e8f59e640 1 --2- 172.21.15.45:0/1492625572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e880e7590 0x7f9e881709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.675+0000 7f9e8f59e640 1 --2- 172.21.15.45:0/1492625572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e88007180 0x7f9e88037df0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.675+0000 7f9e8f59e640 1 -- 172.21.15.45:0/1492625572 >> 172.21.15.45:0/1492625572 conn(0x7f9e88055bc0 msgr2=0x7f9e880e6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:48.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.675+0000 7f279955f640 1 -- 172.21.15.45:0/4126468900 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2780076320 msgr2=0x7f27800787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.675+0000 7f279955f640 1 --2- 172.21.15.45:0/4126468900 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2780076320 0x7f27800787e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2788034000 tx=0x7f278802c000 comp rx=0 tx=0).stop 2024-05-08T05:41:48.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.676+0000 7f279955f640 1 -- 172.21.15.45:0/4126468900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2794070050 msgr2=0x7f2794155630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.676+0000 7f279955f640 1 --2- 172.21.15.45:0/4126468900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2794070050 0x7f2794155630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f278c00c780 tx=0x7f278c00cc40 comp rx=0 tx=0).stop 2024-05-08T05:41:48.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.676+0000 7f9e8f59e640 1 -- 172.21.15.45:0/1492625572 shutdown_connections 2024-05-08T05:41:48.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.676+0000 7f9e8f59e640 1 -- 172.21.15.45:0/1492625572 wait complete. 2024-05-08T05:41:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.676+0000 7f279955f640 1 -- 172.21.15.45:0/4126468900 shutdown_connections 2024-05-08T05:41:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.676+0000 7f279955f640 1 --2- 172.21.15.45:0/4126468900 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2780076320 0x7f27800787e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.676+0000 7f279955f640 1 --2- 172.21.15.45:0/4126468900 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f279415bf80 0x7f279406e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.676+0000 7f279955f640 1 --2- 172.21.15.45:0/4126468900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27940b42c0 0x7f2794155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.676+0000 7f279955f640 1 --2- 172.21.15.45:0/4126468900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2794070050 0x7f2794155630 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.676+0000 7f279955f640 1 -- 172.21.15.45:0/4126468900 >> 172.21.15.45:0/4126468900 conn(0x7f27940b28b0 msgr2=0x7f2794071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.676+0000 7f279955f640 1 -- 172.21.15.45:0/4126468900 shutdown_connections 2024-05-08T05:41:48.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.676+0000 7f279955f640 1 -- 172.21.15.45:0/4126468900 wait complete. 2024-05-08T05:41:48.683 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.5 2024-05-08T05:41:48.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.682+0000 7fc051fa7640 1 -- 172.21.15.45:0/3082158240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc0440482e0 con 0x7fc0440dfdc0 2024-05-08T05:41:48.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.683+0000 7fc0497fa640 1 -- 172.21.15.45:0/3082158240 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc04c090030 con 0x7fc0440dfdc0 2024-05-08T05:41:48.683 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2024-05-08T05:41:48.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.685+0000 7fc051fa7640 1 -- 172.21.15.45:0/3082158240 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc0300762d0 msgr2=0x7fc030078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.685+0000 7fc051fa7640 1 --2- 172.21.15.45:0/3082158240 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc0300762d0 0x7fc030078790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc040000f60 tx=0x7fc040009040 comp rx=0 tx=0).stop 2024-05-08T05:41:48.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.685+0000 7fc051fa7640 1 -- 172.21.15.45:0/3082158240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0440dfdc0 msgr2=0x7fc0440da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.685+0000 7fc051fa7640 1 --2- 172.21.15.45:0/3082158240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0440dfdc0 0x7fc0440da560 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc04c02e0e0 tx=0x7fc04c0756c0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.685+0000 7fc051fa7640 1 -- 172.21.15.45:0/3082158240 shutdown_connections 2024-05-08T05:41:48.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.685+0000 7fc051fa7640 1 --2- 172.21.15.45:0/3082158240 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc0300762d0 0x7fc030078790 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.685+0000 7fc051fa7640 1 --2- 172.21.15.45:0/3082158240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0440dfdc0 0x7fc0440da560 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.685+0000 7fc051fa7640 1 --2- 172.21.15.45:0/3082158240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc04404f4d0 0x7fc0440e0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.685+0000 7fc051fa7640 1 --2- 172.21.15.45:0/3082158240 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc044038ee0 0x7fc0440d7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.685+0000 7fc051fa7640 1 -- 172.21.15.45:0/3082158240 >> 172.21.15.45:0/3082158240 conn(0x7fc04400a400 msgr2=0x7fc044039350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:48.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.685+0000 7fc051fa7640 1 -- 172.21.15.45:0/3082158240 shutdown_connections 2024-05-08T05:41:48.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.685+0000 7fc051fa7640 1 -- 172.21.15.45:0/3082158240 wait complete. 2024-05-08T05:41:48.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.690+0000 7f4427028640 1 -- 172.21.15.45:0/76616607 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f44200693f0 con 0x7f44200d0530 2024-05-08T05:41:48.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-05-08T05:41:48.692 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.6 2024-05-08T05:41:48.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.690+0000 7f441d7fa640 1 -- 172.21.15.45:0/76616607 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4414021020 con 0x7f44200d0530 2024-05-08T05:41:48.692 INFO:teuthology.orchestra.run.smithi045.stdout:60129542152 2024-05-08T05:41:48.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.693+0000 7f4427028640 1 -- 172.21.15.45:0/76616607 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f440c0762d0 msgr2=0x7f440c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.693+0000 7f4427028640 1 --2- 172.21.15.45:0/76616607 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f440c0762d0 0x7f440c078790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4410034000 tx=0x7f441002e000 comp rx=0 tx=0).stop 2024-05-08T05:41:48.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.693+0000 7f4427028640 1 -- 172.21.15.45:0/76616607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44200d0530 msgr2=0x7f44201f4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.693+0000 7f4427028640 1 --2- 172.21.15.45:0/76616607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44200d0530 0x7f44201f4810 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f441400c410 tx=0x7f441400c8d0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.693+0000 7f4427028640 1 -- 172.21.15.45:0/76616607 shutdown_connections 2024-05-08T05:41:48.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.693+0000 7f4427028640 1 --2- 172.21.15.45:0/76616607 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f440c0762d0 0x7f440c078790 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.693+0000 7f4427028640 1 --2- 172.21.15.45:0/76616607 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f442015d490 0x7f44200ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.693+0000 7f4427028640 1 --2- 172.21.15.45:0/76616607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44200d0530 0x7f44201f4810 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.693+0000 7f4427028640 1 --2- 172.21.15.45:0/76616607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f442006fe30 0x7f44200b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.693+0000 7f4427028640 1 -- 172.21.15.45:0/76616607 >> 172.21.15.45:0/76616607 conn(0x7f442006c3a0 msgr2=0x7f442006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:48.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.693+0000 7f4427028640 1 -- 172.21.15.45:0/76616607 shutdown_connections 2024-05-08T05:41:48.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.693+0000 7f4427028640 1 -- 172.21.15.45:0/76616607 wait complete. 2024-05-08T05:41:48.695 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2024-05-08T05:41:48.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.696+0000 7fa48bf0a640 1 -- 172.21.15.45:0/1448268523 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa4840693f0 con 0x7fa4840d1970 2024-05-08T05:41:48.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.697+0000 7fa47a7fc640 1 -- 172.21.15.45:0/1448268523 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa480023080 con 0x7fa4840d1970 2024-05-08T05:41:48.697 INFO:teuthology.orchestra.run.smithi045.stdout:55834574854 2024-05-08T05:41:48.699 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2024-05-08T05:41:48.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.700+0000 7fa48bf0a640 1 -- 172.21.15.45:0/1448268523 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa468076330 msgr2=0x7fa4680787f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.700+0000 7fa48bf0a640 1 --2- 172.21.15.45:0/1448268523 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa468076330 0x7fa4680787f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa4840cbb60 tx=0x7fa474009040 comp rx=0 tx=0).stop 2024-05-08T05:41:48.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.700+0000 7fa48bf0a640 1 -- 172.21.15.45:0/1448268523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4840d1970 msgr2=0x7fa4841e8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:48.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.700+0000 7fa48bf0a640 1 --2- 172.21.15.45:0/1448268523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4840d1970 0x7fa4841e8b80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa48000cdc0 tx=0x7fa48000a430 comp rx=0 tx=0).stop 2024-05-08T05:41:48.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.700+0000 7fa48bf0a640 1 -- 172.21.15.45:0/1448268523 shutdown_connections 2024-05-08T05:41:48.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.700+0000 7fa48bf0a640 1 --2- 172.21.15.45:0/1448268523 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa468076330 0x7fa4680787f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.700+0000 7fa48bf0a640 1 --2- 172.21.15.45:0/1448268523 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4840d53d0 0x7fa4841e90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.700+0000 7fa48bf0a640 1 --2- 172.21.15.45:0/1448268523 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4840c9ff0 0x7fa4840ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.700+0000 7fa48bf0a640 1 --2- 172.21.15.45:0/1448268523 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4840d1970 0x7fa4841e8b80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:48.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.700+0000 7fa48bf0a640 1 -- 172.21.15.45:0/1448268523 >> 172.21.15.45:0/1448268523 conn(0x7fa48408eac0 msgr2=0x7fa48406d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:48.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.700+0000 7fa48bf0a640 1 -- 172.21.15.45:0/1448268523 shutdown_connections 2024-05-08T05:41:48.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:48.700+0000 7fa48bf0a640 1 -- 172.21.15.45:0/1448268523 wait complete. 2024-05-08T05:41:48.708 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.7 2024-05-08T05:41:48.713 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-05-08T05:41:49.667 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T05:41:49.683 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T05:41:49.692 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T05:41:49.692 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T05:41:49.695 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T05:41:49.699 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T05:41:49.709 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T05:41:49.714 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T05:41:49.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.838+0000 7fe5ac1f0640 1 Processor -- start 2024-05-08T05:41:49.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.839+0000 7fe5ac1f0640 1 -- start start 2024-05-08T05:41:49.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.840+0000 7fe5ac1f0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe5a40b53b0 0x7fe5a40b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.840+0000 7fe5ac1f0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5a408c980 0x7fe5a40720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.840+0000 7fe5ac1f0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5a406e4b0 con 0x7fe5a40adf80 2024-05-08T05:41:49.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.840+0000 7fe5ac1f0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe5a40b06f0 con 0x7fe5a40b53b0 2024-05-08T05:41:49.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.840+0000 7fe5ac1f0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe5a406e790 con 0x7fe5a408c980 2024-05-08T05:41:49.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.841+0000 7fe5aa78d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe5a40adf80 0x7fe5a406f550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44012/0 (socket says 172.21.15.45:44012) 2024-05-08T05:41:49.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.841+0000 7fe5aa78d640 1 -- 172.21.15.45:0/2561365283 learned_addr learned my addr 172.21.15.45:0/2561365283 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:49.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.841+0000 7fe5a978b640 1 --2- 172.21.15.45:0/2561365283 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5a408c980 0x7fe5a40720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.841+0000 7fe5a9f8c640 1 --2- 172.21.15.45:0/2561365283 >> v2:172.21.15.45:3300/0 conn(0x7fe5a40b53b0 0x7fe5a40b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.841+0000 7fe5a8f8a640 1 -- 172.21.15.45:0/2561365283 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 821834290 0 0) 0x7fe5a406e4b0 con 0x7fe5a40adf80 2024-05-08T05:41:49.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.841+0000 7fe5a8f8a640 1 -- 172.21.15.45:0/2561365283 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe590003420 con 0x7fe5a40adf80 2024-05-08T05:41:49.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.842+0000 7fe5a8f8a640 1 -- 172.21.15.45:0/2561365283 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 924295196 0 0) 0x7fe590003420 con 0x7fe5a40adf80 2024-05-08T05:41:49.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.842+0000 7fe5a8f8a640 1 -- 172.21.15.45:0/2561365283 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5a408c980 msgr2=0x7fe5a40720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:49.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.842+0000 7fe5a8f8a640 1 --2- 172.21.15.45:0/2561365283 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5a408c980 0x7fe5a40720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.842+0000 7fe5a8f8a640 1 -- 172.21.15.45:0/2561365283 >> v2:172.21.15.45:3300/0 conn(0x7fe5a40b53b0 msgr2=0x7fe5a40b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:49.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.851+0000 7fe5a8f8a640 1 --2- 172.21.15.45:0/2561365283 >> v2:172.21.15.45:3300/0 conn(0x7fe5a40b53b0 0x7fe5a40b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.851+0000 7fe5a8f8a640 1 -- 172.21.15.45:0/2561365283 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a406ea70 con 0x7fe5a40adf80 2024-05-08T05:41:49.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.851+0000 7fe5a9f8c640 1 --2- 172.21.15.45:0/2561365283 >> v2:172.21.15.45:3300/0 conn(0x7fe5a40b53b0 0x7fe5a40b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:49.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.851+0000 7fe5a8f8a640 1 -- 172.21.15.45:0/2561365283 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fe59c001a70 con 0x7fe5a40adf80 2024-05-08T05:41:49.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.851+0000 7fe5a8f8a640 1 -- 172.21.15.45:0/2561365283 >> v1:172.21.15.45:6789/0 conn(0x7fe5a40adf80 legacy=0x7fe5a406f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.851+0000 7fe5a8f8a640 1 --2- 172.21.15.45:0/2561365283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe590004ae0 0x7fe590004f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.851+0000 7fe5a8f8a640 1 --2- 172.21.15.45:0/2561365283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5900036a0 0x7fe590003b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.851+0000 7fe5a8f8a640 1 --2- 172.21.15.45:0/2561365283 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5a408c980 0x7fe59000fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.851+0000 7fe5a8f8a640 1 -- 172.21.15.45:0/2561365283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a40b06f0 con 0x7fe5900036a0 2024-05-08T05:41:49.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.851+0000 7fe5a8f8a640 1 -- 172.21.15.45:0/2561365283 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a406e790 con 0x7fe590004ae0 2024-05-08T05:41:49.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.851+0000 7fe5a8f8a640 1 -- 172.21.15.45:0/2561365283 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe5900100b0 con 0x7fe5a408c980 2024-05-08T05:41:49.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.851+0000 7fe5aa78d640 1 --2- 172.21.15.45:0/2561365283 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5a408c980 0x7fe59000fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.851+0000 7fe5a978b640 1 --2- 172.21.15.45:0/2561365283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5900036a0 0x7fe590003b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.852+0000 7fe5a978b640 1 -- 172.21.15.45:0/2561365283 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5a408c980 msgr2=0x7fe59000fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:49.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.852+0000 7fe5a978b640 1 --2- 172.21.15.45:0/2561365283 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5a408c980 0x7fe59000fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.852+0000 7fe5a978b640 1 -- 172.21.15.45:0/2561365283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe590004ae0 msgr2=0x7fe590004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:49.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.852+0000 7fe5a978b640 1 --2- 172.21.15.45:0/2561365283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe590004ae0 0x7fe590004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.852+0000 7fe5a978b640 1 -- 172.21.15.45:0/2561365283 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe590003420 con 0x7fe5900036a0 2024-05-08T05:41:49.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.853+0000 7fe5a978b640 1 --2- 172.21.15.45:0/2561365283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5900036a0 0x7fe590003b20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0015e60 tx=0x7fe590004720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33f48405246fb43 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.854+0000 7fe5a8f8a640 1 -- 172.21.15.45:0/2561365283 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a003f070 con 0x7fe5900036a0 2024-05-08T05:41:49.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.854+0000 7fe5a8f8a640 1 -- 172.21.15.45:0/2561365283 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a0005b90 con 0x7fe5900036a0 2024-05-08T05:41:49.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.855+0000 7fe5ac1f0640 1 -- 172.21.15.45:0/2561365283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5900036a0 msgr2=0x7fe590003b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.855+0000 7fe5ac1f0640 1 --2- 172.21.15.45:0/2561365283 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5900036a0 0x7fe590003b20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0015e60 tx=0x7fe590004720 comp rx=0 tx=0).stop 2024-05-08T05:41:49.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.855+0000 7fe5aa78d640 1 -- 172.21.15.45:0/2561365283 reap_dead start 2024-05-08T05:41:49.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.855+0000 7fe5ac1f0640 1 -- 172.21.15.45:0/2561365283 shutdown_connections 2024-05-08T05:41:49.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.855+0000 7fe5ac1f0640 1 -- 172.21.15.45:0/2561365283 >> 172.21.15.45:0/2561365283 conn(0x7fe5a40b2920 msgr2=0x7fe5a40b2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:49.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.856+0000 7fe5ac1f0640 1 -- 172.21.15.45:0/2561365283 shutdown_connections 2024-05-08T05:41:49.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.856+0000 7fe5ac1f0640 1 -- 172.21.15.45:0/2561365283 wait complete. 2024-05-08T05:41:49.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.858+0000 7fe5ac1f0640 1 Processor -- start 2024-05-08T05:41:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.858+0000 7fe5ac1f0640 1 -- start start 2024-05-08T05:41:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.859+0000 7fe5ac1f0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5900036a0 0x7fe5a4168380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.859+0000 7fe5ac1f0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a409ca90 0x7fe5a41656c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.859+0000 7fe5ac1f0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5a4160d90 0x7fe5a4161210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.859+0000 7fe5ac1f0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a40b5790 con 0x7fe5a409ca90 2024-05-08T05:41:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.859+0000 7fe5ac1f0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a406f030 con 0x7fe5900036a0 2024-05-08T05:41:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.859+0000 7fe5ac1f0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe5a406e0f0 con 0x7fe5a4160d90 2024-05-08T05:41:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.860+0000 7fe5a978b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a409ca90 0x7fe5a41656c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.860+0000 7fe5a978b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a409ca90 0x7fe5a41656c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51510/0 (socket says 172.21.15.45:51510) 2024-05-08T05:41:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.860+0000 7fe5a978b640 1 -- 172.21.15.45:0/1403534104 learned_addr learned my addr 172.21.15.45:0/1403534104 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.860+0000 7fe5a978b640 1 -- 172.21.15.45:0/1403534104 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5a4160d90 msgr2=0x7fe5a4161210 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T05:41:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.860+0000 7fe5a978b640 1 --2- 172.21.15.45:0/1403534104 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5a4160d90 0x7fe5a4161210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.860+0000 7fe5a978b640 1 -- 172.21.15.45:0/1403534104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5900036a0 msgr2=0x7fe5a4168380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:41:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.860+0000 7fe5a978b640 1 --2- 172.21.15.45:0/1403534104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5900036a0 0x7fe5a4168380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.860+0000 7fe5a978b640 1 -- 172.21.15.45:0/1403534104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a409e380 con 0x7fe5a409ca90 2024-05-08T05:41:49.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.861+0000 7fe5a978b640 1 --2- 172.21.15.45:0/1403534104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a409ca90 0x7fe5a41656c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000a120 tx=0x7fe5a0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.862+0000 7fe59a7fc640 1 -- 172.21.15.45:0/1403534104 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a003f070 con 0x7fe5a409ca90 2024-05-08T05:41:49.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.862+0000 7fe5ac1f0640 1 -- 172.21.15.45:0/1403534104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a409d2e0 con 0x7fe5a409ca90 2024-05-08T05:41:49.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.862+0000 7fe5ac1f0640 1 -- 172.21.15.45:0/1403534104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a409d8a0 con 0x7fe5a409ca90 2024-05-08T05:41:49.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.867+0000 7fe59a7fc640 1 -- 172.21.15.45:0/1403534104 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a00153d0 con 0x7fe5a409ca90 2024-05-08T05:41:49.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.868+0000 7fe59a7fc640 1 -- 172.21.15.45:0/1403534104 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0015a30 con 0x7fe5a409ca90 2024-05-08T05:41:49.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.868+0000 7fe59a7fc640 1 -- 172.21.15.45:0/1403534104 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe5a0042070 con 0x7fe5a409ca90 2024-05-08T05:41:49.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.869+0000 7fe59a7fc640 1 --2- 172.21.15.45:0/1403534104 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe588076400 0x7fe5880788c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.869+0000 7fe59a7fc640 1 -- 172.21.15.45:0/1403534104 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe5a00b7a90 con 0x7fe5a409ca90 2024-05-08T05:41:49.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.869+0000 7fe5a9f8c640 1 --2- 172.21.15.45:0/1403534104 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe588076400 0x7fe5880788c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.870+0000 7fe5ac1f0640 1 -- 172.21.15.45:0/1403534104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe58c005290 con 0x7fe5a409ca90 2024-05-08T05:41:49.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.876+0000 7fe5a9f8c640 1 --2- 172.21.15.45:0/1403534104 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe588076400 0x7fe5880788c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe5940079c0 tx=0x7fe594008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.879+0000 7fe59a7fc640 1 -- 172.21.15.45:0/1403534104 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe5a0080a40 con 0x7fe5a409ca90 2024-05-08T05:41:49.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.891+0000 7fcfd66f8640 1 Processor -- start 2024-05-08T05:41:49.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.891+0000 7fcfd66f8640 1 -- start start 2024-05-08T05:41:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.891+0000 7f59058e4640 1 Processor -- start 2024-05-08T05:41:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.891+0000 7f59058e4640 1 -- start start 2024-05-08T05:41:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.893+0000 7fcfd66f8640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcfd00b3410 0x7fcfd00b5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.893+0000 7fcfd66f8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcfd00b3c20 0x7fcfd00aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.893+0000 7fcfd66f8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfd0162040 con 0x7fcfd00aef70 2024-05-08T05:41:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.893+0000 7fcfd66f8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fcfd00b45a0 con 0x7fcfd00b3c20 2024-05-08T05:41:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.893+0000 7fcfd66f8640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fcfd00b4880 con 0x7fcfd00b3410 2024-05-08T05:41:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.893+0000 7fcfcf7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcfd00b3c20 0x7fcfd00aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.893+0000 7f59058e4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5900070050 0x7f59000b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.893+0000 7f59058e4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f590006e900 0x7f59000cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.893+0000 7f59058e4640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f590015f3a0 con 0x7f59000b40f0 2024-05-08T05:41:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.893+0000 7f59058e4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f590015fd40 con 0x7f5900070050 2024-05-08T05:41:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.893+0000 7f59058e4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f590015f680 con 0x7f590006e900 2024-05-08T05:41:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.893+0000 7f59048e2640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5900070050 0x7f59000b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.893+0000 7f59048e2640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5900070050 0x7f59000b3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51528/0 (socket says 172.21.15.45:51528) 2024-05-08T05:41:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.893+0000 7f59048e2640 1 -- 172.21.15.45:0/1774363902 learned_addr learned my addr 172.21.15.45:0/1774363902 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.893+0000 7fcfcf7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcfd00b3c20 0x7fcfd00aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51520/0 (socket says 172.21.15.45:51520) 2024-05-08T05:41:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.893+0000 7fcfcf7fe640 1 -- 172.21.15.45:0/3125436148 learned_addr learned my addr 172.21.15.45:0/3125436148 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.894+0000 7f59050e3640 1 --2- 172.21.15.45:0/1774363902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f590006e900 0x7f59000cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.894+0000 7fcfcf7fe640 1 -- 172.21.15.45:0/3125436148 >> v2:172.21.15.63:3300/0 conn(0x7fcfd00b3410 msgr2=0x7fcfd00b5360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T05:41:49.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.894+0000 7fcfcf7fe640 1 --2- 172.21.15.45:0/3125436148 >> v2:172.21.15.63:3300/0 conn(0x7fcfd00b3410 0x7fcfd00b5360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.894+0000 7fcfcf7fe640 1 -- 172.21.15.45:0/3125436148 >> v1:172.21.15.63:6789/0 conn(0x7fcfd00aef70 legacy=0x7fcfd015ecd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T05:41:49.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.894+0000 7fcfcf7fe640 1 -- 172.21.15.45:0/3125436148 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd00b4b60 con 0x7fcfd00b3c20 2024-05-08T05:41:49.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.894+0000 7f59048e2640 1 -- 172.21.15.45:0/1774363902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f590006e900 msgr2=0x7f59000cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:49.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.894+0000 7f59048e2640 1 --2- 172.21.15.45:0/1774363902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f590006e900 0x7f59000cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.894+0000 7f59048e2640 1 -- 172.21.15.45:0/1774363902 >> v1:172.21.15.63:6789/0 conn(0x7f59000b40f0 legacy=0x7f590006e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:49.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.894+0000 7f59048e2640 1 -- 172.21.15.45:0/1774363902 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590015f960 con 0x7f5900070050 2024-05-08T05:41:49.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.895+0000 7fcfcf7fe640 1 --2- 172.21.15.45:0/3125436148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfd00b3c20 0x7fcfd00aea30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcfd006d350 tx=0x7fcfc402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e3ae9d162e9ebd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.895+0000 7f7443fff640 1 Processor -- start 2024-05-08T05:41:49.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.895+0000 7f7443fff640 1 -- start start 2024-05-08T05:41:49.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.895+0000 7f59048e2640 1 --2- 172.21.15.45:0/1774363902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5900070050 0x7f59000b3bb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f590006a0b0 tx=0x7f58f802a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210676065a43d4e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.896+0000 7f58ff7fe640 1 -- 172.21.15.45:0/1774363902 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8036070 con 0x7f5900070050 2024-05-08T05:41:49.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.896+0000 7fcfceffd640 1 -- 172.21.15.45:0/3125436148 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4037070 con 0x7fcfd00b3c20 2024-05-08T05:41:49.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.896+0000 7fcfceffd640 1 -- 172.21.15.45:0/3125436148 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfc4032440 con 0x7fcfd00b3c20 2024-05-08T05:41:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.896+0000 7fcfceffd640 1 -- 172.21.15.45:0/3125436148 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4032810 con 0x7fcfd00b3c20 2024-05-08T05:41:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.896+0000 7f58ff7fe640 1 -- 172.21.15.45:0/1774363902 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f8031600 con 0x7f5900070050 2024-05-08T05:41:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.896+0000 7f58ff7fe640 1 -- 172.21.15.45:0/1774363902 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f802ae60 con 0x7f5900070050 2024-05-08T05:41:49.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.896+0000 7f6a08f67640 1 Processor -- start 2024-05-08T05:41:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.896+0000 7f6a08f67640 1 -- start start 2024-05-08T05:41:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7f7443fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7444161080 0x7f7444163a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7f7443fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f74440b5880 0x7f74441677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7f7443fff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7444167ce0 0x7f74440cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7f7443fff640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f7444073290 con 0x7f7444161080 2024-05-08T05:41:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7f7443fff640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7444161460 con 0x7f7444167ce0 2024-05-08T05:41:49.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7f7443fff640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f74441642d0 con 0x7f74440b5880 2024-05-08T05:41:49.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7f59058e4640 1 -- 172.21.15.45:0/1774363902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5900070050 msgr2=0x7f59000b3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7f59058e4640 1 --2- 172.21.15.45:0/1774363902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5900070050 0x7f59000b3bb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f590006a0b0 tx=0x7f58f802a640 comp rx=0 tx=0).stop 2024-05-08T05:41:49.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7fcfd66f8640 1 -- 172.21.15.45:0/3125436148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfd00b3c20 msgr2=0x7fcfd00aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7f7442ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7444161080 0x7f7444163a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7f74437fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7444167ce0 0x7f74440cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7f59058e4640 1 -- 172.21.15.45:0/1774363902 shutdown_connections 2024-05-08T05:41:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7f7442ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7444161080 0x7f7444163a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51534/0 (socket says 172.21.15.45:51534) 2024-05-08T05:41:49.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7f7442ffd640 1 -- 172.21.15.45:0/1621908714 learned_addr learned my addr 172.21.15.45:0/1621908714 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7fcfd66f8640 1 --2- 172.21.15.45:0/3125436148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfd00b3c20 0x7fcfd00aea30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcfd006d350 tx=0x7fcfc402a270 comp rx=0 tx=0).stop 2024-05-08T05:41:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7f59058e4640 1 --2- 172.21.15.45:0/1774363902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f590006e900 0x7f59000cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7f59058e4640 1 --2- 172.21.15.45:0/1774363902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5900070050 0x7f59000b3bb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7f59058e4640 1 -- 172.21.15.45:0/1774363902 >> 172.21.15.45:0/1774363902 conn(0x7f59000b28b0 msgr2=0x7f59000b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:49.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7f74427fc640 1 --2- 172.21.15.45:0/1621908714 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f74440b5880 0x7f74441677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7fcfd66f8640 1 -- 172.21.15.45:0/3125436148 shutdown_connections 2024-05-08T05:41:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7fcfd66f8640 1 --2- 172.21.15.45:0/3125436148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfd00b3c20 0x7fcfd00aea30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7fcfd66f8640 1 --2- 172.21.15.45:0/3125436148 >> v2:172.21.15.63:3300/0 conn(0x7fcfd00b3410 0x7fcfd00b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7fcfd66f8640 1 -- 172.21.15.45:0/3125436148 >> 172.21.15.45:0/3125436148 conn(0x7fcfd00b2920 msgr2=0x7fcfd00b2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:49.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7f07488f7640 1 Processor -- start 2024-05-08T05:41:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.898+0000 7f74437fe640 1 -- 172.21.15.45:0/1621908714 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f74440b5880 msgr2=0x7f74441677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:49.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.897+0000 7f07488f7640 1 -- start start 2024-05-08T05:41:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.898+0000 7f74437fe640 1 --2- 172.21.15.45:0/1621908714 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f74440b5880 0x7f74441677a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.898+0000 7f74437fe640 1 -- 172.21.15.45:0/1621908714 >> v2:172.21.15.45:3300/0 conn(0x7f7444161080 msgr2=0x7f7444163a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.898+0000 7f74437fe640 1 --2- 172.21.15.45:0/1621908714 >> v2:172.21.15.45:3300/0 conn(0x7f7444161080 0x7f7444163a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.898+0000 7f74437fe640 1 -- 172.21.15.45:0/1621908714 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74441645b0 con 0x7f7444167ce0 2024-05-08T05:41:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.898+0000 7f7442ffd640 1 --2- 172.21.15.45:0/1621908714 >> v2:172.21.15.45:3300/0 conn(0x7f7444161080 0x7f7444163a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:49.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.898+0000 7f6a08f67640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a04158b70 0x7f6a0415f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.898+0000 7f6a08f67640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6a0415c290 0x7f6a040b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.898+0000 7fcfd66f8640 1 -- 172.21.15.45:0/3125436148 shutdown_connections 2024-05-08T05:41:49.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.898+0000 7f59058e4640 1 -- 172.21.15.45:0/1774363902 shutdown_connections 2024-05-08T05:41:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.898+0000 7f6a08f67640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a0415ecf0 con 0x7f6a0406f6b0 2024-05-08T05:41:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.899+0000 7f6a03fff640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f6a0406f6b0 0x7f6a04072110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:53772/0 (socket says 172.21.15.45:53772) 2024-05-08T05:41:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.899+0000 7f6a03fff640 1 -- 172.21.15.45:0/2323275234 learned_addr learned my addr 172.21.15.45:0/2323275234 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.899+0000 7f6a08f67640 1 -- 172.21.15.45:0/2323275234 --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f6a0415f2d0 con 0x7f6a0415c290 2024-05-08T05:41:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.899+0000 7f6a037fe640 1 --2- 172.21.15.45:0/2323275234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a04158b70 0x7f6a0415f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.898+0000 7f59058e4640 1 -- 172.21.15.45:0/1774363902 wait complete. 2024-05-08T05:41:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.899+0000 7f69faffd640 1 --2- 172.21.15.45:0/2323275234 >> v2:172.21.15.45:3300/0 conn(0x7f6a0415c290 0x7f6a040b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.899+0000 7f6a08f67640 1 -- 172.21.15.45:0/2323275234 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6a040b5890 con 0x7f6a04158b70 2024-05-08T05:41:49.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.898+0000 7fcfd66f8640 1 -- 172.21.15.45:0/3125436148 wait complete. 2024-05-08T05:41:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.899+0000 7f07488f7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f073c0e0020 0x7f073c00b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.899+0000 7f74437fe640 1 --2- 172.21.15.45:0/1621908714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7444167ce0 0x7f74440cea00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f74440d08f0 tx=0x7f743c02a540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f72c0672fa0a85ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.899+0000 7f6a02ffd640 1 -- 172.21.15.45:0/2323275234 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1716846530 0 0) 0x7f6a0415ecf0 con 0x7f6a0406f6b0 2024-05-08T05:41:49.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.899+0000 7f6a02ffd640 1 -- 172.21.15.45:0/2323275234 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69f0003520 con 0x7f6a0406f6b0 2024-05-08T05:41:49.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.900+0000 7f59058e4640 1 Processor -- start 2024-05-08T05:41:49.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.900+0000 7f6a02ffd640 1 -- 172.21.15.45:0/2323275234 <== mon.1 v1:172.21.15.63:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1604595123 0 0) 0x7f69f0003520 con 0x7f6a0406f6b0 2024-05-08T05:41:49.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.900+0000 7f6a037fe640 1 -- 172.21.15.45:0/2323275234 >> v2:172.21.15.45:3300/0 conn(0x7f6a0415c290 msgr2=0x7f6a040b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:49.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.900+0000 7f6a037fe640 1 --2- 172.21.15.45:0/2323275234 >> v2:172.21.15.45:3300/0 conn(0x7f6a0415c290 0x7f6a040b4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.900+0000 7f6a037fe640 1 -- 172.21.15.45:0/2323275234 >> v1:172.21.15.63:6789/0 conn(0x7f6a0406f6b0 legacy=0x7f6a04072110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.900+0000 7f6a037fe640 1 -- 172.21.15.45:0/2323275234 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0415ef60 con 0x7f6a04158b70 2024-05-08T05:41:49.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.900+0000 7f69faffd640 1 --2- 172.21.15.45:0/2323275234 >> v2:172.21.15.45:3300/0 conn(0x7f6a0415c290 0x7f6a040b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:49.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.899+0000 7f07488f7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f073c03afd0 0x7f073c007c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.900+0000 7f07488f7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f073c00cb30 con 0x7f073c008f50 2024-05-08T05:41:49.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.900+0000 7f07488f7640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f073c00b710 con 0x7f073c03afd0 2024-05-08T05:41:49.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.900+0000 7f0742d9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f073c0e0020 0x7f073c00b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.900+0000 7f07488f7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f073c0565d0 con 0x7f073c0e0020 2024-05-08T05:41:49.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.900+0000 7f074359e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f073c03afd0 0x7f073c007c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.900+0000 7f074259c640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f073c008f50 0x7f073c03a8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:53784/0 (socket says 172.21.15.45:53784) 2024-05-08T05:41:49.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.900+0000 7f074259c640 1 -- 172.21.15.45:0/3255580101 learned_addr learned my addr 172.21.15.45:0/3255580101 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:49.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.900+0000 7fcfd66f8640 1 Processor -- start 2024-05-08T05:41:49.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.900+0000 7fcfd66f8640 1 -- start start 2024-05-08T05:41:49.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.900+0000 7f0742d9d640 1 -- 172.21.15.45:0/3255580101 >> v2:172.21.15.45:3300/0 conn(0x7f073c03afd0 msgr2=0x7f073c007c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:49.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.900+0000 7f0742d9d640 1 --2- 172.21.15.45:0/3255580101 >> v2:172.21.15.45:3300/0 conn(0x7f073c03afd0 0x7f073c007c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.900+0000 7f0742d9d640 1 -- 172.21.15.45:0/3255580101 >> v1:172.21.15.63:6789/0 conn(0x7f073c008f50 legacy=0x7f073c03a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.901+0000 7f0742d9d640 1 -- 172.21.15.45:0/3255580101 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073c0568b0 con 0x7f073c0e0020 2024-05-08T05:41:49.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.900+0000 7f59058e4640 1 -- start start 2024-05-08T05:41:49.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.901+0000 7f7441ffb640 1 -- 172.21.15.45:0/1621908714 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c007740 con 0x7f7444167ce0 2024-05-08T05:41:49.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.901+0000 7f7441ffb640 1 -- 172.21.15.45:0/1621908714 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f743c0078e0 con 0x7f7444167ce0 2024-05-08T05:41:49.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.901+0000 7f7441ffb640 1 -- 172.21.15.45:0/1621908714 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c007cb0 con 0x7f7444167ce0 2024-05-08T05:41:49.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.901+0000 7f6a037fe640 1 --2- 172.21.15.45:0/2323275234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a04158b70 0x7f6a0415f760 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f69f40097a0 tx=0x7f69f402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afed60bc53212da3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.901+0000 7f6a02ffd640 1 -- 172.21.15.45:0/2323275234 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4037070 con 0x7f6a04158b70 2024-05-08T05:41:49.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.901+0000 7f6a02ffd640 1 -- 172.21.15.45:0/2323275234 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69f402aba0 con 0x7f6a04158b70 2024-05-08T05:41:49.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.901+0000 7f6a02ffd640 1 -- 172.21.15.45:0/2323275234 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4037070 con 0x7f6a04158b70 2024-05-08T05:41:49.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7f0742d9d640 1 --2- 172.21.15.45:0/3255580101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f073c0e0020 0x7f073c00b1d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0744052040 tx=0x7f07440a6790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef252dc5272ed162 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7fcfd66f8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfd00aef70 0x7fcfd015ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7f7443fff640 1 -- 172.21.15.45:0/1621908714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7444167ce0 msgr2=0x7f74440cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7f7443fff640 1 --2- 172.21.15.45:0/1621908714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7444167ce0 0x7f74440cea00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f74440d08f0 tx=0x7f743c02a540 comp rx=0 tx=0).stop 2024-05-08T05:41:49.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7f7443fff640 1 -- 172.21.15.45:0/1621908714 shutdown_connections 2024-05-08T05:41:49.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7f7443fff640 1 --2- 172.21.15.45:0/1621908714 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7444167ce0 0x7f74440cea00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7f7443fff640 1 --2- 172.21.15.45:0/1621908714 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f74440b5880 0x7f74441677a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7f7443fff640 1 --2- 172.21.15.45:0/1621908714 >> v2:172.21.15.45:3300/0 conn(0x7f7444161080 0x7f7444163a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7f7443fff640 1 -- 172.21.15.45:0/1621908714 >> 172.21.15.45:0/1621908714 conn(0x7f74440b28b0 msgr2=0x7f74440b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:49.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7fcfd66f8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfd00b3410 0x7fcfd01454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7fcfd66f8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfd00b3c20 0x7fcfd015e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7fcfd66f8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd015f560 con 0x7fcfd00b3c20 2024-05-08T05:41:49.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7fcfd66f8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd015f3e0 con 0x7fcfd00aef70 2024-05-08T05:41:49.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7fcfd66f8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcfd015c100 con 0x7fcfd00b3410 2024-05-08T05:41:49.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7f0741d9b640 1 -- 172.21.15.45:0/3255580101 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074406e620 con 0x7f073c0e0020 2024-05-08T05:41:49.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7f7443fff640 1 -- 172.21.15.45:0/1621908714 shutdown_connections 2024-05-08T05:41:49.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7fdbf203e640 1 Processor -- start 2024-05-08T05:41:49.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7f59058e4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f590006e900 0x7f59001493b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7f59058e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5900070050 0x7f59001498f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7f59058e4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59000b40f0 0x7f5900149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7f59058e4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f590006fb10 con 0x7f5900070050 2024-05-08T05:41:49.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7f59058e4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f590006f990 con 0x7f590006e900 2024-05-08T05:41:49.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7f59058e4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f590006ed50 con 0x7f59000b40f0 2024-05-08T05:41:49.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7f0741d9b640 1 -- 172.21.15.45:0/3255580101 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f074406e7c0 con 0x7f073c0e0020 2024-05-08T05:41:49.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7f0741d9b640 1 -- 172.21.15.45:0/3255580101 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074406ebf0 con 0x7f073c0e0020 2024-05-08T05:41:49.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7fcfcf7fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfd00b3410 0x7fcfd01454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7f7443fff640 1 -- 172.21.15.45:0/1621908714 wait complete. 2024-05-08T05:41:49.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7fcfcf7fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfd00b3410 0x7fcfd01454a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:33428/0 (socket says 172.21.15.45:33428) 2024-05-08T05:41:49.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7fcfcf7fe640 1 -- 172.21.15.45:0/163018979 learned_addr learned my addr 172.21.15.45:0/163018979 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:49.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7fcfd4c95640 1 --2- 172.21.15.45:0/163018979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfd00b3c20 0x7fcfd015e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7fcfcffff640 1 --2- 172.21.15.45:0/163018979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfd00aef70 0x7fcfd015ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.903+0000 7f6a08f67640 1 -- 172.21.15.45:0/2323275234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a04158b70 msgr2=0x7f6a0415f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.903+0000 7f6a08f67640 1 --2- 172.21.15.45:0/2323275234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a04158b70 0x7f6a0415f760 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f69f40097a0 tx=0x7f69f402a110 comp rx=0 tx=0).stop 2024-05-08T05:41:49.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.903+0000 7fcfcf7fe640 1 -- 172.21.15.45:0/163018979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfd00aef70 msgr2=0x7fcfd015ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.903+0000 7fcfcf7fe640 1 --2- 172.21.15.45:0/163018979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfd00aef70 0x7fcfd015ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.903+0000 7fcfcf7fe640 1 -- 172.21.15.45:0/163018979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfd00b3c20 msgr2=0x7fcfd015e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.903+0000 7fcfcf7fe640 1 --2- 172.21.15.45:0/163018979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfd00b3c20 0x7fcfd015e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.903+0000 7fcfcf7fe640 1 -- 172.21.15.45:0/163018979 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd015d4b0 con 0x7fcfd00b3410 2024-05-08T05:41:49.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.902+0000 7f59050e3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59000b40f0 0x7f5900149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.903+0000 7fcfcf7fe640 1 --2- 172.21.15.45:0/163018979 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfd00b3410 0x7fcfd01454a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009950 tx=0x7fcfc40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.903+0000 7f58fffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5900070050 0x7f59001498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.903+0000 7f59050e3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59000b40f0 0x7f5900149f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:33444/0 (socket says 172.21.15.45:33444) 2024-05-08T05:41:49.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.903+0000 7f59050e3640 1 -- 172.21.15.45:0/1181734092 learned_addr learned my addr 172.21.15.45:0/1181734092 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:49.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.903+0000 7f59048e2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f590006e900 0x7f59001493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.903+0000 7f58fffff640 1 -- 172.21.15.45:0/1181734092 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59000b40f0 msgr2=0x7f5900149f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.903+0000 7f58fffff640 1 --2- 172.21.15.45:0/1181734092 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59000b40f0 0x7f5900149f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.903+0000 7f58fffff640 1 -- 172.21.15.45:0/1181734092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f590006e900 msgr2=0x7f59001493b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.903+0000 7f58fffff640 1 --2- 172.21.15.45:0/1181734092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f590006e900 0x7f59001493b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.904+0000 7f58fffff640 1 -- 172.21.15.45:0/1181734092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590013f670 con 0x7f5900070050 2024-05-08T05:41:49.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.904+0000 7f59050e3640 1 --2- 172.21.15.45:0/1181734092 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59000b40f0 0x7f5900149f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:49.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.904+0000 7f07488f7640 1 -- 172.21.15.45:0/3255580101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f073c0e0020 msgr2=0x7f073c00b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.904+0000 7fcfccff9640 1 -- 172.21.15.45:0/163018979 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4041070 con 0x7fcfd00b3410 2024-05-08T05:41:49.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.904+0000 7f07488f7640 1 --2- 172.21.15.45:0/3255580101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f073c0e0020 0x7f073c00b1d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0744052040 tx=0x7f07440a6790 comp rx=0 tx=0).stop 2024-05-08T05:41:49.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.904+0000 7f07488f7640 1 -- 172.21.15.45:0/3255580101 shutdown_connections 2024-05-08T05:41:49.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.904+0000 7f07488f7640 1 --2- 172.21.15.45:0/3255580101 >> v2:172.21.15.45:3300/0 conn(0x7f073c03afd0 0x7f073c007c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.904+0000 7f07488f7640 1 --2- 172.21.15.45:0/3255580101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f073c0e0020 0x7f073c00b1d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.904+0000 7f07488f7640 1 -- 172.21.15.45:0/3255580101 >> 172.21.15.45:0/3255580101 conn(0x7f073c01ef70 msgr2=0x7f073c055c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:49.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.904+0000 7fcfd66f8640 1 -- 172.21.15.45:0/163018979 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd015d790 con 0x7fcfd00b3410 2024-05-08T05:41:49.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.904+0000 7fcfd66f8640 1 -- 172.21.15.45:0/163018979 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd0142480 con 0x7fcfd00b3410 2024-05-08T05:41:49.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.904+0000 7f7443fff640 1 Processor -- start 2024-05-08T05:41:49.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.904+0000 7f58fffff640 1 --2- 172.21.15.45:0/1181734092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5900070050 0x7f59001498f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f58f000c410 tx=0x7f58f000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.904+0000 7f7443fff640 1 -- start start 2024-05-08T05:41:49.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.903+0000 7fdbf203e640 1 -- start start 2024-05-08T05:41:49.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.904+0000 7fdbf203e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdbec163710 0x7fdbec071f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.904+0000 7fdbf203e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdbec0b53e0 0x7fdbec0724a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7fdbf083b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdbec0b53e0 0x7fdbec0724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7fcfccff9640 1 -- 172.21.15.45:0/163018979 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfc403c930 con 0x7fcfd00b3410 2024-05-08T05:41:49.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7fcfccff9640 1 -- 172.21.15.45:0/163018979 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4037040 con 0x7fcfd00b3410 2024-05-08T05:41:49.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7fdbf103c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdbec163710 0x7fdbec071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7f07488f7640 1 -- 172.21.15.45:0/3255580101 shutdown_connections 2024-05-08T05:41:49.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7fdbf083b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdbec0b53e0 0x7fdbec0724a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:33454/0 (socket says 172.21.15.45:33454) 2024-05-08T05:41:49.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7fdbf083b640 1 -- 172.21.15.45:0/3654814516 learned_addr learned my addr 172.21.15.45:0/3654814516 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:49.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7fdbf203e640 1 -- 172.21.15.45:0/3654814516 --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbec0d0460 con 0x7fdbec0729e0 2024-05-08T05:41:49.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7fdbf203e640 1 -- 172.21.15.45:0/3654814516 --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fdbec0cea00 con 0x7fdbec163710 2024-05-08T05:41:49.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7f7443fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74440b5880 0x7f7444152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7f7443fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7444161080 0x7f744406def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7f7443fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7444167ce0 0x7f74440cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7f7443fff640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7444163fc0 con 0x7f74440b5880 2024-05-08T05:41:49.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7f7443fff640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7444167280 con 0x7f7444167ce0 2024-05-08T05:41:49.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7f7443fff640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f74440cef40 con 0x7f7444161080 2024-05-08T05:41:49.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7f58fd7fa640 1 -- 172.21.15.45:0/1181734092 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0011790 con 0x7f5900070050 2024-05-08T05:41:49.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.906+0000 7fdbf203e640 1 -- 172.21.15.45:0/3654814516 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdbec0cefc0 con 0x7fdbec0b53e0 2024-05-08T05:41:49.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7f07488f7640 1 -- 172.21.15.45:0/3255580101 wait complete. 2024-05-08T05:41:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7f6a08f67640 1 -- 172.21.15.45:0/2323275234 shutdown_connections 2024-05-08T05:41:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7f6a08f67640 1 --2- 172.21.15.45:0/2323275234 >> v2:172.21.15.45:3300/0 conn(0x7f6a0415c290 0x7f6a040b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7f6a08f67640 1 --2- 172.21.15.45:0/2323275234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a04158b70 0x7f6a0415f760 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.905+0000 7f6a08f67640 1 -- 172.21.15.45:0/2323275234 >> 172.21.15.45:0/2323275234 conn(0x7f6a040b2da0 msgr2=0x7f6a040b3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:49.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.906+0000 7fdbdffff640 1 -- 172.21.15.45:0/3654814516 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1433813847 0 0) 0x7fdbec0d0460 con 0x7fdbec0729e0 2024-05-08T05:41:49.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.906+0000 7f59058e4640 1 -- 172.21.15.45:0/1181734092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f590009e6e0 con 0x7f5900070050 2024-05-08T05:41:49.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.906+0000 7f59058e4640 1 -- 172.21.15.45:0/1181734092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f590009c940 con 0x7f5900070050 2024-05-08T05:41:49.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.906+0000 7fdbdffff640 1 -- 172.21.15.45:0/3654814516 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbd8003520 con 0x7fdbec0729e0 2024-05-08T05:41:49.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.906+0000 7f74427fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7444161080 0x7f744406def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.906+0000 7f74437fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7444167ce0 0x7f74440cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.906+0000 7f74437fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7444167ce0 0x7f74440cf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:47446/0 (socket says 172.21.15.45:47446) 2024-05-08T05:41:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.906+0000 7f74437fe640 1 -- 172.21.15.45:0/217418665 learned_addr learned my addr 172.21.15.45:0/217418665 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.906+0000 7f7442ffd640 1 --2- 172.21.15.45:0/217418665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74440b5880 0x7f7444152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.906+0000 7f74437fe640 1 -- 172.21.15.45:0/217418665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7444161080 msgr2=0x7f744406def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.906+0000 7f74437fe640 1 --2- 172.21.15.45:0/217418665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7444161080 0x7f744406def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.906+0000 7f74437fe640 1 -- 172.21.15.45:0/217418665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74440b5880 msgr2=0x7f7444152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.906+0000 7f74437fe640 1 --2- 172.21.15.45:0/217418665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74440b5880 0x7f7444152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.906+0000 7f74437fe640 1 -- 172.21.15.45:0/217418665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744406e980 con 0x7f7444167ce0 2024-05-08T05:41:49.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.907+0000 7f74437fe640 1 --2- 172.21.15.45:0/217418665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7444167ce0 0x7f74440cf4e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f743c02ab30 tx=0x7f743c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.907+0000 7f07488f7640 1 Processor -- start 2024-05-08T05:41:49.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.907+0000 7f742b7fe640 1 -- 172.21.15.45:0/217418665 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c03f070 con 0x7f7444167ce0 2024-05-08T05:41:49.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.907+0000 7f742b7fe640 1 -- 172.21.15.45:0/217418665 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f743c031b40 con 0x7f7444167ce0 2024-05-08T05:41:49.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.907+0000 7f742b7fe640 1 -- 172.21.15.45:0/217418665 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c044460 con 0x7f7444167ce0 2024-05-08T05:41:49.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.907+0000 7f7443fff640 1 -- 172.21.15.45:0/217418665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f744406ec60 con 0x7f7444167ce0 2024-05-08T05:41:49.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.907+0000 7fcfccff9640 1 -- 172.21.15.45:0/163018979 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcfc4044070 con 0x7fcfd00b3410 2024-05-08T05:41:49.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.907+0000 7fcfd66f8640 1 -- 172.21.15.45:0/163018979 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfd00693f0 con 0x7fcfd00b3410 2024-05-08T05:41:49.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.907+0000 7f58fd7fa640 1 -- 172.21.15.45:0/1181734092 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f0017070 con 0x7f5900070050 2024-05-08T05:41:49.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.907+0000 7f58fd7fa640 1 -- 172.21.15.45:0/1181734092 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0011b70 con 0x7f5900070050 2024-05-08T05:41:49.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.907+0000 7f58f6ffd640 1 -- 172.21.15.45:0/1181734092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f590009c450 con 0x7f5900070050 2024-05-08T05:41:49.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.907+0000 7fcfccff9640 1 --2- 172.21.15.45:0/163018979 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcfbc076320 0x7fcfbc0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.907+0000 7fdbf083b640 1 -- 172.21.15.45:0/3654814516 >> v2:172.21.15.45:3300/0 conn(0x7fdbec163710 msgr2=0x7fdbec071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:49.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.907+0000 7fdbf083b640 1 --2- 172.21.15.45:0/3654814516 >> v2:172.21.15.45:3300/0 conn(0x7fdbec163710 0x7fdbec071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.907+0000 7fdbf083b640 1 -- 172.21.15.45:0/3654814516 >> v1:172.21.15.63:6789/0 conn(0x7fdbec0729e0 legacy=0x7fdbec0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.907+0000 7fdbf083b640 1 -- 172.21.15.45:0/3654814516 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbec167c20 con 0x7fdbec0b53e0 2024-05-08T05:41:49.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.908+0000 7f7443fff640 1 -- 172.21.15.45:0/217418665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f744406f220 con 0x7f7444167ce0 2024-05-08T05:41:49.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.907+0000 7f6a08f67640 1 -- 172.21.15.45:0/2323275234 shutdown_connections 2024-05-08T05:41:49.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.908+0000 7fcfccff9640 1 -- 172.21.15.45:0/163018979 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcfc40b82f0 con 0x7fcfd00b3410 2024-05-08T05:41:49.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.908+0000 7f07488f7640 1 -- start start 2024-05-08T05:41:49.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.908+0000 7fcfcffff640 1 --2- 172.21.15.45:0/163018979 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcfbc076320 0x7fcfbc0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.908+0000 7f6a08f67640 1 -- 172.21.15.45:0/2323275234 wait complete. 2024-05-08T05:41:49.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.908+0000 7fdbf083b640 1 --2- 172.21.15.45:0/3654814516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdbec0b53e0 0x7fdbec0724a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009e50 tx=0x7fdbe402a610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb662cfff492cf6a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.909+0000 7f07488f7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f073c008f50 0x7f073c0e2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.909+0000 7f07488f7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f073c03afd0 0x7f073c0d0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.909+0000 7fcfcffff640 1 --2- 172.21.15.45:0/163018979 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcfbc076320 0x7fcfbc0787e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0006fd0 tx=0x7fcfc0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.909+0000 7f07488f7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f073c0e0020 0x7f073c0338b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.909+0000 7f07488f7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f073c0e0400 con 0x7f073c0e0020 2024-05-08T05:41:49.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.909+0000 7f07488f7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f073c0534a0 con 0x7f073c008f50 2024-05-08T05:41:49.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.909+0000 7f07488f7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f073c03b610 con 0x7f073c03afd0 2024-05-08T05:41:49.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.909+0000 7f58fd7fa640 1 -- 172.21.15.45:0/1181734092 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f58f0011d10 con 0x7f5900070050 2024-05-08T05:41:49.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.910+0000 7f074259c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f073c03afd0 0x7f073c0d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.910+0000 7f074259c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f073c03afd0 0x7f073c0d0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:33474/0 (socket says 172.21.15.45:33474) 2024-05-08T05:41:49.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.910+0000 7f074259c640 1 -- 172.21.15.45:0/1747544851 learned_addr learned my addr 172.21.15.45:0/1747544851 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:49.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.910+0000 7f0742d9d640 1 --2- 172.21.15.45:0/1747544851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f073c008f50 0x7f073c0e2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.910+0000 7f6a08f67640 1 Processor -- start 2024-05-08T05:41:49.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.910+0000 7f074359e640 1 --2- 172.21.15.45:0/1747544851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f073c0e0020 0x7f073c0338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.910+0000 7f7443fff640 1 -- 172.21.15.45:0/217418665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7424005290 con 0x7f7444167ce0 2024-05-08T05:41:49.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.910+0000 7f6a08f67640 1 -- start start 2024-05-08T05:41:49.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.911+0000 7f58fd7fa640 1 --2- 172.21.15.45:0/1181734092 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f58e0076320 0x7f58e00787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.911+0000 7f742b7fe640 1 -- 172.21.15.45:0/217418665 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f743c03b410 con 0x7f7444167ce0 2024-05-08T05:41:49.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.910+0000 7f074259c640 1 -- 172.21.15.45:0/1747544851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f073c008f50 msgr2=0x7f073c0e2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.911+0000 7f074259c640 1 --2- 172.21.15.45:0/1747544851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f073c008f50 0x7f073c0e2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.911+0000 7f074259c640 1 -- 172.21.15.45:0/1747544851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f073c0e0020 msgr2=0x7f073c0338b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.911+0000 7f074259c640 1 --2- 172.21.15.45:0/1747544851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f073c0e0020 0x7f073c0338b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.911+0000 7f074259c640 1 -- 172.21.15.45:0/1747544851 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073c0e34f0 con 0x7f073c03afd0 2024-05-08T05:41:49.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.911+0000 7f0742d9d640 1 --2- 172.21.15.45:0/1747544851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f073c008f50 0x7f073c0e2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:49.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.911+0000 7f074259c640 1 --2- 172.21.15.45:0/1747544851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f073c03afd0 0x7f073c0d0470 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f073400c500 tx=0x7f073400c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.911+0000 7f6a08f67640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a0406f6b0 0x7f6a041544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.911+0000 7f58fd7fa640 1 -- 172.21.15.45:0/1181734092 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f58f0097540 con 0x7f5900070050 2024-05-08T05:41:49.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.911+0000 7f59048e2640 1 --2- 172.21.15.45:0/1181734092 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f58e0076320 0x7f58e00787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.911+0000 7f6a08f67640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a04158b70 0x7f6a040aad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.912+0000 7f6a08f67640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a0415c290 0x7f6a040ab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.912+0000 7f6a08f67640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6a040b5670 con 0x7f6a0415c290 2024-05-08T05:41:49.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.912+0000 7f6a08f67640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6a040b54f0 con 0x7f6a04158b70 2024-05-08T05:41:49.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.912+0000 7f6a08f67640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6a040b5b70 con 0x7f6a0406f6b0 2024-05-08T05:41:49.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.912+0000 7f59048e2640 1 --2- 172.21.15.45:0/1181734092 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f58e0076320 0x7f58e00787e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f58f8033000 tx=0x7f58f8009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.912+0000 7f6a037fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a0406f6b0 0x7f6a041544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.912+0000 7f6a037fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a0406f6b0 0x7f6a041544d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:33482/0 (socket says 172.21.15.45:33482) 2024-05-08T05:41:49.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.912+0000 7f6a037fe640 1 -- 172.21.15.45:0/3151422062 learned_addr learned my addr 172.21.15.45:0/3151422062 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:49.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.912+0000 7f6a03fff640 1 --2- 172.21.15.45:0/3151422062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a0415c290 0x7f6a040ab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.912+0000 7f69faffd640 1 --2- 172.21.15.45:0/3151422062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a04158b70 0x7f6a040aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.912+0000 7f742b7fe640 1 --2- 172.21.15.45:0/217418665 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f74300763a0 0x7f7430078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.912+0000 7f742b7fe640 1 -- 172.21.15.45:0/217418665 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f743c0b8f00 con 0x7f7444167ce0 2024-05-08T05:41:49.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.912+0000 7f7442ffd640 1 --2- 172.21.15.45:0/217418665 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f74300763a0 0x7f7430078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.912+0000 7f6a03fff640 1 -- 172.21.15.45:0/3151422062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a0406f6b0 msgr2=0x7f6a041544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.912+0000 7f6a03fff640 1 --2- 172.21.15.45:0/3151422062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a0406f6b0 0x7f6a041544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.912+0000 7f6a03fff640 1 -- 172.21.15.45:0/3151422062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a04158b70 msgr2=0x7f6a040aad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.912+0000 7f6a03fff640 1 --2- 172.21.15.45:0/3151422062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a04158b70 0x7f6a040aad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.912+0000 7f6a03fff640 1 -- 172.21.15.45:0/3151422062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a04154a10 con 0x7f6a0415c290 2024-05-08T05:41:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.912+0000 7f6a037fe640 1 --2- 172.21.15.45:0/3151422062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a0406f6b0 0x7f6a041544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.912+0000 7f6a03fff640 1 --2- 172.21.15.45:0/3151422062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a0415c290 0x7f6a040ab2b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f69fc008ad0 tx=0x7f69fc008f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.913+0000 7f7442ffd640 1 --2- 172.21.15.45:0/217418665 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f74300763a0 0x7f7430078860 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7434009630 tx=0x7f7434009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.913+0000 7fdbdffff640 1 -- 172.21.15.45:0/3654814516 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4036070 con 0x7fdbec0b53e0 2024-05-08T05:41:49.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.913+0000 7f072b7fe640 1 -- 172.21.15.45:0/1747544851 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734017070 con 0x7f073c03afd0 2024-05-08T05:41:49.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.913+0000 7fdbdffff640 1 -- 172.21.15.45:0/3654814516 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbe4031600 con 0x7fdbec0b53e0 2024-05-08T05:41:49.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.913+0000 7fdbdffff640 1 -- 172.21.15.45:0/3654814516 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe40319d0 con 0x7fdbec0b53e0 2024-05-08T05:41:49.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.913+0000 7fcfccff9640 1 -- 172.21.15.45:0/163018979 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcfc4081330 con 0x7fcfd00b3410 2024-05-08T05:41:49.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.913+0000 7f07488f7640 1 -- 172.21.15.45:0/1747544851 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f073c0e1f10 con 0x7f073c03afd0 2024-05-08T05:41:49.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.913+0000 7f072b7fe640 1 -- 172.21.15.45:0/1747544851 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0734005ce0 con 0x7f073c03afd0 2024-05-08T05:41:49.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.914+0000 7fdbf203e640 1 -- 172.21.15.45:0/3654814516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdbec0b53e0 msgr2=0x7fdbec0724a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.914+0000 7fdbf203e640 1 --2- 172.21.15.45:0/3654814516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdbec0b53e0 0x7fdbec0724a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009e50 tx=0x7fdbe402a610 comp rx=0 tx=0).stop 2024-05-08T05:41:49.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.913+0000 7f072b7fe640 1 -- 172.21.15.45:0/1747544851 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07340072f0 con 0x7f073c03afd0 2024-05-08T05:41:49.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.913+0000 7f07488f7640 1 -- 172.21.15.45:0/1747544851 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f073c0e24d0 con 0x7f073c03afd0 2024-05-08T05:41:49.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.914+0000 7f07488f7640 1 -- 172.21.15.45:0/1747544851 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0724005290 con 0x7f073c03afd0 2024-05-08T05:41:49.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.914+0000 7fdbf203e640 1 -- 172.21.15.45:0/3654814516 shutdown_connections 2024-05-08T05:41:49.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.914+0000 7fdbf203e640 1 --2- 172.21.15.45:0/3654814516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdbec0b53e0 0x7fdbec0724a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.914+0000 7fdbf203e640 1 --2- 172.21.15.45:0/3654814516 >> v2:172.21.15.45:3300/0 conn(0x7fdbec163710 0x7fdbec071f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.914+0000 7fdbf203e640 1 -- 172.21.15.45:0/3654814516 >> 172.21.15.45:0/3654814516 conn(0x7fdbec0b28b0 msgr2=0x7fdbec0b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:49.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.914+0000 7fdbf203e640 1 -- 172.21.15.45:0/3654814516 shutdown_connections 2024-05-08T05:41:49.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.915+0000 7fdbf203e640 1 -- 172.21.15.45:0/3654814516 wait complete. 2024-05-08T05:41:49.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.916+0000 7fdbf203e640 1 Processor -- start 2024-05-08T05:41:49.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.916+0000 7fdbf203e640 1 -- start start 2024-05-08T05:41:49.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.916+0000 7f742b7fe640 1 -- 172.21.15.45:0/217418665 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f743c082590 con 0x7f7444167ce0 2024-05-08T05:41:49.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.917+0000 7f072b7fe640 1 -- 172.21.15.45:0/1747544851 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f073401a070 con 0x7f073c03afd0 2024-05-08T05:41:49.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.918+0000 7f6a00ff9640 1 -- 172.21.15.45:0/3151422062 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc018070 con 0x7f6a0415c290 2024-05-08T05:41:49.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.918+0000 7f58fd7fa640 1 -- 172.21.15.45:0/1181734092 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f58f0060c40 con 0x7f5900070050 2024-05-08T05:41:49.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.918+0000 7f6a00ff9640 1 -- 172.21.15.45:0/3151422062 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69fc00d8a0 con 0x7f6a0415c290 2024-05-08T05:41:49.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.918+0000 7f6a08f67640 1 -- 172.21.15.45:0/3151422062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a041fa950 con 0x7f6a0415c290 2024-05-08T05:41:49.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.918+0000 7f072b7fe640 1 --2- 172.21.15.45:0/1747544851 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07300762d0 0x7f0730078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.918+0000 7f072b7fe640 1 -- 172.21.15.45:0/1747544851 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f0734097410 con 0x7f073c03afd0 2024-05-08T05:41:49.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.918+0000 7f0742d9d640 1 --2- 172.21.15.45:0/1747544851 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07300762d0 0x7f0730078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.918+0000 7fdbf203e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbec0729e0 0x7fdbec09e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.918+0000 7fdbf203e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbec0b53e0 0x7fdbec164a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.918+0000 7fdbf203e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdbec163710 0x7fdbec164f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.918+0000 7fdbf083b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbec0b53e0 0x7fdbec164a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.918+0000 7fdbf083b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbec0b53e0 0x7fdbec164a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:47476/0 (socket says 172.21.15.45:47476) 2024-05-08T05:41:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.918+0000 7fdbf083b640 1 -- 172.21.15.45:0/4191256256 learned_addr learned my addr 172.21.15.45:0/4191256256 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.918+0000 7fdbf183d640 1 --2- 172.21.15.45:0/4191256256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdbec163710 0x7fdbec164f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.918+0000 7fdbf203e640 1 -- 172.21.15.45:0/4191256256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec06dce0 con 0x7fdbec0729e0 2024-05-08T05:41:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.918+0000 7fdbf203e640 1 -- 172.21.15.45:0/4191256256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec0cec50 con 0x7fdbec0b53e0 2024-05-08T05:41:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.918+0000 7fdbf203e640 1 -- 172.21.15.45:0/4191256256 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdbec0b5ae0 con 0x7fdbec163710 2024-05-08T05:41:49.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.919+0000 7f6a00ff9640 1 -- 172.21.15.45:0/3151422062 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc00dc70 con 0x7f6a0415c290 2024-05-08T05:41:49.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.919+0000 7f0742d9d640 1 --2- 172.21.15.45:0/1747544851 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07300762d0 0x7f0730078790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f073c0567a0 tx=0x7f07440aa000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.919+0000 7fdbf083b640 1 -- 172.21.15.45:0/4191256256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdbec163710 msgr2=0x7fdbec164f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.919+0000 7fdbf083b640 1 --2- 172.21.15.45:0/4191256256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdbec163710 0x7fdbec164f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.919+0000 7fdbf083b640 1 -- 172.21.15.45:0/4191256256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbec0729e0 msgr2=0x7fdbec09e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.919+0000 7fdbf083b640 1 --2- 172.21.15.45:0/4191256256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbec0729e0 0x7fdbec09e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.919+0000 7fdbf083b640 1 -- 172.21.15.45:0/4191256256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbec09e790 con 0x7fdbec0b53e0 2024-05-08T05:41:49.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.919+0000 7fdbf083b640 1 --2- 172.21.15.45:0/4191256256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbec0b53e0 0x7fdbec164a10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009e50 tx=0x7fdbe4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.919+0000 7f6a08f67640 1 -- 172.21.15.45:0/3151422062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a041faf10 con 0x7f6a0415c290 2024-05-08T05:41:49.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.920+0000 7f072b7fe640 1 -- 172.21.15.45:0/1747544851 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f073c00cb30 con 0x7f073c03afd0 2024-05-08T05:41:49.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.921+0000 7fdbddffb640 1 -- 172.21.15.45:0/4191256256 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4036050 con 0x7fdbec0b53e0 2024-05-08T05:41:49.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.921+0000 7fdbddffb640 1 -- 172.21.15.45:0/4191256256 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbe4042070 con 0x7fdbec0b53e0 2024-05-08T05:41:49.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.922+0000 7fdbf203e640 1 -- 172.21.15.45:0/4191256256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbec163f30 con 0x7fdbec0b53e0 2024-05-08T05:41:49.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.922+0000 7fdbf203e640 1 -- 172.21.15.45:0/4191256256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbec0add70 con 0x7fdbec0b53e0 2024-05-08T05:41:49.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.923+0000 7fdbddffb640 1 -- 172.21.15.45:0/4191256256 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4009510 con 0x7fdbec0b53e0 2024-05-08T05:41:49.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.923+0000 7f6a00ff9640 1 -- 172.21.15.45:0/3151422062 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f69fc01b070 con 0x7f6a0415c290 2024-05-08T05:41:49.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.925+0000 7f6a08f67640 1 -- 172.21.15.45:0/3151422062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a0415a080 con 0x7f6a0415c290 2024-05-08T05:41:49.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.925+0000 7f6a00ff9640 1 --2- 172.21.15.45:0/3151422062 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f69e40763a0 0x7f69e4078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.925+0000 7f6a00ff9640 1 -- 172.21.15.45:0/3151422062 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f69fc097440 con 0x7f6a0415c290 2024-05-08T05:41:49.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.925+0000 7fdbddffb640 1 -- 172.21.15.45:0/4191256256 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdbe4047460 con 0x7fdbec0b53e0 2024-05-08T05:41:49.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.926+0000 7fdbddffb640 1 --2- 172.21.15.45:0/4191256256 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdbd0076350 0x7fdbd0078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.926+0000 7fdbddffb640 1 -- 172.21.15.45:0/4191256256 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdbe40b88e0 con 0x7fdbec0b53e0 2024-05-08T05:41:49.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.927+0000 7fdbf203e640 1 -- 172.21.15.45:0/4191256256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbec074440 con 0x7fdbec0b53e0 2024-05-08T05:41:49.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.927+0000 7fdbf103c640 1 --2- 172.21.15.45:0/4191256256 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdbd0076350 0x7fdbd0078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.927+0000 7f6a037fe640 1 --2- 172.21.15.45:0/3151422062 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f69e40763a0 0x7f69e4078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.928+0000 7fdbf103c640 1 --2- 172.21.15.45:0/4191256256 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdbd0076350 0x7fdbd0078810 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8006fd0 tx=0x7fdbe8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.928+0000 7f6a037fe640 1 --2- 172.21.15.45:0/3151422062 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f69e40763a0 0x7f69e4078860 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6a041f06e0 tx=0x7f69f402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.932+0000 7fdbddffb640 1 -- 172.21.15.45:0/4191256256 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdbec074440 con 0x7fdbec0b53e0 2024-05-08T05:41:49.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.936+0000 7f6a00ff9640 1 -- 172.21.15.45:0/3151422062 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6a0415a080 con 0x7f6a0415c290 2024-05-08T05:41:49.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.942+0000 7f824d0a7640 1 Processor -- start 2024-05-08T05:41:49.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.943+0000 7f824d0a7640 1 -- start start 2024-05-08T05:41:49.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.944+0000 7f824d0a7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f82480d1970 0x7f82480d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.944+0000 7f824d0a7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f82480d4cc0 0x7f82480d50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.944+0000 7f824d0a7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82481630b0 con 0x7f824815f420 2024-05-08T05:41:49.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.944+0000 7f824d0a7640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f82480d3e70 con 0x7f82480d1970 2024-05-08T05:41:49.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.944+0000 7f824d0a7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f82480d2550 con 0x7f82480d4cc0 2024-05-08T05:41:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.944+0000 7f824759e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f82480d4cc0 0x7f82480d50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.945+0000 7f824759e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f82480d4cc0 0x7f82480d50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:47488/0 (socket says 172.21.15.45:47488) 2024-05-08T05:41:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.945+0000 7f824759e640 1 -- 172.21.15.45:0/556607902 learned_addr learned my addr 172.21.15.45:0/556607902 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.945+0000 7f824759e640 1 -- 172.21.15.45:0/556607902 >> v2:172.21.15.45:3300/0 conn(0x7f82480d1970 msgr2=0x7f82480d1e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T05:41:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.945+0000 7f824759e640 1 --2- 172.21.15.45:0/556607902 >> v2:172.21.15.45:3300/0 conn(0x7f82480d1970 0x7f82480d1e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.945+0000 7f824759e640 1 -- 172.21.15.45:0/556607902 >> v1:172.21.15.45:6789/0 conn(0x7f824815f420 legacy=0x7f82480d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.945+0000 7f824759e640 1 -- 172.21.15.45:0/556607902 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82480d2c80 con 0x7f82480d4cc0 2024-05-08T05:41:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.946+0000 7f824759e640 1 --2- 172.21.15.45:0/556607902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82480d4cc0 0x7f82480d50e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f82480d17a0 tx=0x7f8240032ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eea8eb906d62d036 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.947+0000 7f8245d9b640 1 -- 172.21.15.45:0/556607902 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240033a10 con 0x7f82480d4cc0 2024-05-08T05:41:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.947+0000 7f8245d9b640 1 -- 172.21.15.45:0/556607902 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8240033bb0 con 0x7f82480d4cc0 2024-05-08T05:41:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.948+0000 7f824d0a7640 1 -- 172.21.15.45:0/556607902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82480d4cc0 msgr2=0x7f82480d50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.948+0000 7f824d0a7640 1 --2- 172.21.15.45:0/556607902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82480d4cc0 0x7f82480d50e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f82480d17a0 tx=0x7f8240032ed0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.948+0000 7f824d0a7640 1 -- 172.21.15.45:0/556607902 shutdown_connections 2024-05-08T05:41:49.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.948+0000 7f824d0a7640 1 --2- 172.21.15.45:0/556607902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82480d4cc0 0x7f82480d50e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.948+0000 7f824d0a7640 1 --2- 172.21.15.45:0/556607902 >> v2:172.21.15.45:3300/0 conn(0x7f82480d1970 0x7f82480d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.948+0000 7f824d0a7640 1 -- 172.21.15.45:0/556607902 >> 172.21.15.45:0/556607902 conn(0x7f824808eac0 msgr2=0x7f824809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:49.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.949+0000 7f824d0a7640 1 -- 172.21.15.45:0/556607902 shutdown_connections 2024-05-08T05:41:49.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.949+0000 7f824d0a7640 1 -- 172.21.15.45:0/556607902 wait complete. 2024-05-08T05:41:49.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.951+0000 7f824d0a7640 1 Processor -- start 2024-05-08T05:41:49.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.951+0000 7f824d0a7640 1 -- start start 2024-05-08T05:41:49.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.952+0000 7f824d0a7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82480d1970 0x7f824815af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.953+0000 7f824d0a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82480ca770 0x7f8248161260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.953+0000 7f8246d9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82480d1970 0x7f824815af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.953+0000 7f8246d9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82480d1970 0x7f824815af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:47504/0 (socket says 172.21.15.45:47504) 2024-05-08T05:41:49.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.953+0000 7f824d0a7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f824815d420 0x7f824815d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.953+0000 7f8246d9d640 1 -- 172.21.15.45:0/3373607171 learned_addr learned my addr 172.21.15.45:0/3373607171 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:49.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.953+0000 7f824d0a7640 1 -- 172.21.15.45:0/3373607171 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82480d2830 con 0x7f82480ca770 2024-05-08T05:41:49.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.953+0000 7f824759e640 1 --2- 172.21.15.45:0/3373607171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f824815d420 0x7f824815d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.954+0000 7f824d0a7640 1 -- 172.21.15.45:0/3373607171 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82480d3240 con 0x7f82480d1970 2024-05-08T05:41:49.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.954+0000 7f824d0a7640 1 -- 172.21.15.45:0/3373607171 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f82480d5620 con 0x7f824815d420 2024-05-08T05:41:49.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.954+0000 7f8246d9d640 1 -- 172.21.15.45:0/3373607171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f824815d420 msgr2=0x7f824815d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.954+0000 7f8246d9d640 1 --2- 172.21.15.45:0/3373607171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f824815d420 0x7f824815d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.954+0000 7f8246d9d640 1 -- 172.21.15.45:0/3373607171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82480ca770 msgr2=0x7f8248161260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:49.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.954+0000 7f8246d9d640 1 --2- 172.21.15.45:0/3373607171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82480ca770 0x7f8248161260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:49.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.954+0000 7f8246d9d640 1 -- 172.21.15.45:0/3373607171 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82480c96d0 con 0x7f82480d1970 2024-05-08T05:41:49.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.955+0000 7f8246d9d640 1 --2- 172.21.15.45:0/3373607171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82480d1970 0x7f824815af30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f823800b410 tx=0x7f823800b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.955+0000 7f822f7fe640 1 -- 172.21.15.45:0/3373607171 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238015020 con 0x7f82480d1970 2024-05-08T05:41:49.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.955+0000 7f822f7fe640 1 -- 172.21.15.45:0/3373607171 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8238011830 con 0x7f82480d1970 2024-05-08T05:41:49.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.955+0000 7f824d0a7640 1 -- 172.21.15.45:0/3373607171 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82480c99b0 con 0x7f82480d1970 2024-05-08T05:41:49.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.955+0000 7f822f7fe640 1 -- 172.21.15.45:0/3373607171 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238011be0 con 0x7f82480d1970 2024-05-08T05:41:49.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.955+0000 7f824d0a7640 1 -- 172.21.15.45:0/3373607171 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82480c9f70 con 0x7f82480d1970 2024-05-08T05:41:49.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.959+0000 7f822f7fe640 1 -- 172.21.15.45:0/3373607171 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8238018020 con 0x7f82480d1970 2024-05-08T05:41:49.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.959+0000 7f822f7fe640 1 --2- 172.21.15.45:0/3373607171 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8234076300 0x7f82340787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:49.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.960+0000 7f822f7fe640 1 -- 172.21.15.45:0/3373607171 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8238097830 con 0x7f82480d1970 2024-05-08T05:41:49.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.960+0000 7f822d7fa640 1 -- 172.21.15.45:0/3373607171 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82480693f0 con 0x7f82480d1970 2024-05-08T05:41:49.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.960+0000 7f824659c640 1 --2- 172.21.15.45:0/3373607171 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8234076300 0x7f82340787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:49.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.961+0000 7f824659c640 1 --2- 172.21.15.45:0/3373607171 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8234076300 0x7f82340787c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f824815bd00 tx=0x7f823c006c40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:49.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:49.965+0000 7f822f7fe640 1 -- 172.21.15.45:0/3373607171 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f823801f020 con 0x7f82480d1970 2024-05-08T05:41:50.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.319+0000 7fe5ac1f0640 1 -- 172.21.15.45:0/1403534104 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe58c005580 con 0x7fe5a409ca90 2024-05-08T05:41:50.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.319+0000 7fe59a7fc640 1 -- 172.21.15.45:0/1403534104 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe5a0084d40 con 0x7fe5a409ca90 2024-05-08T05:41:50.321 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2024-05-08T05:41:50.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.323+0000 7fe587fff640 1 -- 172.21.15.45:0/1403534104 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe588076400 msgr2=0x7fe5880788c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.323+0000 7fe587fff640 1 --2- 172.21.15.45:0/1403534104 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe588076400 0x7fe5880788c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe5940079c0 tx=0x7fe594008040 comp rx=0 tx=0).stop 2024-05-08T05:41:50.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.323+0000 7fe587fff640 1 -- 172.21.15.45:0/1403534104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a409ca90 msgr2=0x7fe5a41656c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.323+0000 7fe587fff640 1 --2- 172.21.15.45:0/1403534104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a409ca90 0x7fe5a41656c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000a120 tx=0x7fe5a0008040 comp rx=0 tx=0).stop 2024-05-08T05:41:50.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.323+0000 7fe587fff640 1 -- 172.21.15.45:0/1403534104 shutdown_connections 2024-05-08T05:41:50.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.324+0000 7fe587fff640 1 --2- 172.21.15.45:0/1403534104 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe588076400 0x7fe5880788c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.324+0000 7fe587fff640 1 --2- 172.21.15.45:0/1403534104 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5a4160d90 0x7fe5a4161210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.324+0000 7fe587fff640 1 --2- 172.21.15.45:0/1403534104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5a409ca90 0x7fe5a41656c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.324+0000 7fe587fff640 1 --2- 172.21.15.45:0/1403534104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5900036a0 0x7fe5a4168380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.324+0000 7fe587fff640 1 -- 172.21.15.45:0/1403534104 >> 172.21.15.45:0/1403534104 conn(0x7fe5a40b2920 msgr2=0x7fe5a4073df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:50.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.324+0000 7fe587fff640 1 -- 172.21.15.45:0/1403534104 shutdown_connections 2024-05-08T05:41:50.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.324+0000 7fe587fff640 1 -- 172.21.15.45:0/1403534104 wait complete. 2024-05-08T05:41:50.343 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-05-08T05:41:50.343 DEBUG:teuthology.parallel:result is None 2024-05-08T05:41:50.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.348+0000 7f7443fff640 1 -- 172.21.15.45:0/217418665 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f7424005580 con 0x7f7444167ce0 2024-05-08T05:41:50.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.349+0000 7f742b7fe640 1 -- 172.21.15.45:0/217418665 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f743c081e90 con 0x7f7444167ce0 2024-05-08T05:41:50.350 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2024-05-08T05:41:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.351+0000 7f7443fff640 1 -- 172.21.15.45:0/217418665 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f74300763a0 msgr2=0x7f7430078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.351+0000 7f7443fff640 1 --2- 172.21.15.45:0/217418665 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f74300763a0 0x7f7430078860 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7434009630 tx=0x7f7434009240 comp rx=0 tx=0).stop 2024-05-08T05:41:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.351+0000 7f7443fff640 1 -- 172.21.15.45:0/217418665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7444167ce0 msgr2=0x7f74440cf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.351+0000 7f7443fff640 1 --2- 172.21.15.45:0/217418665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7444167ce0 0x7f74440cf4e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f743c02ab30 tx=0x7f743c004a40 comp rx=0 tx=0).stop 2024-05-08T05:41:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.351+0000 7f7443fff640 1 -- 172.21.15.45:0/217418665 shutdown_connections 2024-05-08T05:41:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.351+0000 7f7443fff640 1 --2- 172.21.15.45:0/217418665 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f74300763a0 0x7f7430078860 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.351+0000 7f7443fff640 1 --2- 172.21.15.45:0/217418665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7444161080 0x7f744406def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.351+0000 7f7443fff640 1 --2- 172.21.15.45:0/217418665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7444167ce0 0x7f74440cf4e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.351+0000 7f7443fff640 1 --2- 172.21.15.45:0/217418665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74440b5880 0x7f7444152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.351+0000 7f7443fff640 1 -- 172.21.15.45:0/217418665 >> 172.21.15.45:0/217418665 conn(0x7f74440b28b0 msgr2=0x7f744415ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.351+0000 7f7443fff640 1 -- 172.21.15.45:0/217418665 shutdown_connections 2024-05-08T05:41:50.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.351+0000 7f7443fff640 1 -- 172.21.15.45:0/217418665 wait complete. 2024-05-08T05:41:50.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.356+0000 7f07488f7640 1 -- 172.21.15.45:0/1747544851 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f0724005850 con 0x7f073c03afd0 2024-05-08T05:41:50.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.357+0000 7f072b7fe640 1 -- 172.21.15.45:0/1747544851 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0734060450 con 0x7f073c03afd0 2024-05-08T05:41:50.358 INFO:teuthology.orchestra.run.smithi045.stdout:60129542154 2024-05-08T05:41:50.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.360+0000 7f07488f7640 1 -- 172.21.15.45:0/1747544851 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07300762d0 msgr2=0x7f0730078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.360+0000 7f07488f7640 1 --2- 172.21.15.45:0/1747544851 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07300762d0 0x7f0730078790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f073c0567a0 tx=0x7f07440aa000 comp rx=0 tx=0).stop 2024-05-08T05:41:50.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.361+0000 7f07488f7640 1 -- 172.21.15.45:0/1747544851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f073c03afd0 msgr2=0x7f073c0d0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.361+0000 7f07488f7640 1 --2- 172.21.15.45:0/1747544851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f073c03afd0 0x7f073c0d0470 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f073400c500 tx=0x7f073400c9c0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.361+0000 7f07488f7640 1 -- 172.21.15.45:0/1747544851 shutdown_connections 2024-05-08T05:41:50.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.361+0000 7f07488f7640 1 --2- 172.21.15.45:0/1747544851 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07300762d0 0x7f0730078790 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.361+0000 7f07488f7640 1 --2- 172.21.15.45:0/1747544851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f073c03afd0 0x7f073c0d0470 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.361+0000 7f07488f7640 1 --2- 172.21.15.45:0/1747544851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f073c0e0020 0x7f073c0338b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.361+0000 7f07488f7640 1 --2- 172.21.15.45:0/1747544851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f073c008f50 0x7f073c0e2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.361+0000 7f07488f7640 1 -- 172.21.15.45:0/1747544851 >> 172.21.15.45:0/1747544851 conn(0x7f073c01ef70 msgr2=0x7f073c04c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:50.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.361+0000 7f07488f7640 1 -- 172.21.15.45:0/1747544851 shutdown_connections 2024-05-08T05:41:50.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.362+0000 7f07488f7640 1 -- 172.21.15.45:0/1747544851 wait complete. 2024-05-08T05:41:50.366 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-05-08T05:41:50.366 DEBUG:teuthology.parallel:result is None 2024-05-08T05:41:50.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.367+0000 7f58f6ffd640 1 -- 172.21.15.45:0/1181734092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f5900050740 con 0x7f5900070050 2024-05-08T05:41:50.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.367+0000 7f58fd7fa640 1 -- 172.21.15.45:0/1181734092 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f58f0060c40 con 0x7f5900070050 2024-05-08T05:41:50.369 INFO:teuthology.orchestra.run.smithi045.stdout:60129542153 2024-05-08T05:41:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.371+0000 7f58f6ffd640 1 -- 172.21.15.45:0/1181734092 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f58e0076320 msgr2=0x7f58e00787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.371+0000 7f58f6ffd640 1 --2- 172.21.15.45:0/1181734092 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f58e0076320 0x7f58e00787e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f58f8033000 tx=0x7f58f8009000 comp rx=0 tx=0).stop 2024-05-08T05:41:50.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.371+0000 7f58f6ffd640 1 -- 172.21.15.45:0/1181734092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5900070050 msgr2=0x7f59001498f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.371+0000 7f58f6ffd640 1 --2- 172.21.15.45:0/1181734092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5900070050 0x7f59001498f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f58f000c410 tx=0x7f58f000c8d0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.371+0000 7f58f6ffd640 1 -- 172.21.15.45:0/1181734092 shutdown_connections 2024-05-08T05:41:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.371+0000 7f58f6ffd640 1 --2- 172.21.15.45:0/1181734092 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f58e0076320 0x7f58e00787e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.371+0000 7f58f6ffd640 1 --2- 172.21.15.45:0/1181734092 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f59000b40f0 0x7f5900149f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.371+0000 7f58f6ffd640 1 --2- 172.21.15.45:0/1181734092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5900070050 0x7f59001498f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.371+0000 7f58f6ffd640 1 --2- 172.21.15.45:0/1181734092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f590006e900 0x7f59001493b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.371+0000 7f58f6ffd640 1 -- 172.21.15.45:0/1181734092 >> 172.21.15.45:0/1181734092 conn(0x7f59000b28b0 msgr2=0x7f5900072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:50.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.371+0000 7f58f6ffd640 1 -- 172.21.15.45:0/1181734092 shutdown_connections 2024-05-08T05:41:50.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.372+0000 7f58f6ffd640 1 -- 172.21.15.45:0/1181734092 wait complete. 2024-05-08T05:41:50.378 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.6 2024-05-08T05:41:50.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.375+0000 7fcfd66f8640 1 -- 172.21.15.45:0/163018979 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fcfd00724c0 con 0x7fcfd00b3410 2024-05-08T05:41:50.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.376+0000 7fcfccff9640 1 -- 172.21.15.45:0/163018979 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcfc4085630 con 0x7fcfd00b3410 2024-05-08T05:41:50.379 DEBUG:teuthology.parallel:result is None 2024-05-08T05:41:50.379 INFO:teuthology.orchestra.run.smithi045.stdout:55834574856 2024-05-08T05:41:50.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.377+0000 7fdbf203e640 1 -- 172.21.15.45:0/4191256256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fdbec050740 con 0x7fdbec0b53e0 2024-05-08T05:41:50.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.378+0000 7fdbddffb640 1 -- 172.21.15.45:0/4191256256 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdbe4081e60 con 0x7fdbec0b53e0 2024-05-08T05:41:50.380 INFO:teuthology.orchestra.run.smithi045.stdout:60129542153 2024-05-08T05:41:50.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.378+0000 7fcfd66f8640 1 -- 172.21.15.45:0/163018979 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcfbc076320 msgr2=0x7fcfbc0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.379+0000 7fcfd66f8640 1 --2- 172.21.15.45:0/163018979 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcfbc076320 0x7fcfbc0787e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0006fd0 tx=0x7fcfc0008040 comp rx=0 tx=0).stop 2024-05-08T05:41:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.379+0000 7fcfd66f8640 1 -- 172.21.15.45:0/163018979 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfd00b3410 msgr2=0x7fcfd01454a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.379+0000 7fcfd66f8640 1 --2- 172.21.15.45:0/163018979 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfd00b3410 0x7fcfd01454a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009950 tx=0x7fcfc40046c0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.379+0000 7fcfd66f8640 1 -- 172.21.15.45:0/163018979 shutdown_connections 2024-05-08T05:41:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.379+0000 7fcfd66f8640 1 --2- 172.21.15.45:0/163018979 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcfbc076320 0x7fcfbc0787e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.379+0000 7fcfd66f8640 1 --2- 172.21.15.45:0/163018979 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcfd00b3410 0x7fcfd01454a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.379+0000 7fcfd66f8640 1 --2- 172.21.15.45:0/163018979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfd00b3c20 0x7fcfd015e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.379+0000 7fcfd66f8640 1 --2- 172.21.15.45:0/163018979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcfd00aef70 0x7fcfd015ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.379+0000 7fcfd66f8640 1 -- 172.21.15.45:0/163018979 >> 172.21.15.45:0/163018979 conn(0x7fcfd00b2920 msgr2=0x7fcfd014a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:50.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.379+0000 7fcfd66f8640 1 -- 172.21.15.45:0/163018979 shutdown_connections 2024-05-08T05:41:50.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.379+0000 7fcfd66f8640 1 -- 172.21.15.45:0/163018979 wait complete. 2024-05-08T05:41:50.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.381+0000 7fdbf203e640 1 -- 172.21.15.45:0/4191256256 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdbd0076350 msgr2=0x7fdbd0078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.381+0000 7fdbf203e640 1 --2- 172.21.15.45:0/4191256256 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdbd0076350 0x7fdbd0078810 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8006fd0 tx=0x7fdbe8008040 comp rx=0 tx=0).stop 2024-05-08T05:41:50.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.381+0000 7fdbf203e640 1 -- 172.21.15.45:0/4191256256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbec0b53e0 msgr2=0x7fdbec164a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.381+0000 7fdbf203e640 1 --2- 172.21.15.45:0/4191256256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbec0b53e0 0x7fdbec164a10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009e50 tx=0x7fdbe4004a40 comp rx=0 tx=0).stop 2024-05-08T05:41:50.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.381+0000 7fdbf203e640 1 -- 172.21.15.45:0/4191256256 shutdown_connections 2024-05-08T05:41:50.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.381+0000 7fdbf203e640 1 --2- 172.21.15.45:0/4191256256 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdbd0076350 0x7fdbd0078810 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.381+0000 7fdbf203e640 1 --2- 172.21.15.45:0/4191256256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdbec163710 0x7fdbec164f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.381+0000 7fdbf203e640 1 --2- 172.21.15.45:0/4191256256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbec0b53e0 0x7fdbec164a10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.381+0000 7fdbf203e640 1 --2- 172.21.15.45:0/4191256256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbec0729e0 0x7fdbec09e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.381+0000 7fdbf203e640 1 -- 172.21.15.45:0/4191256256 >> 172.21.15.45:0/4191256256 conn(0x7fdbec0b28b0 msgr2=0x7fdbec0cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:50.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.381+0000 7fdbf203e640 1 -- 172.21.15.45:0/4191256256 shutdown_connections 2024-05-08T05:41:50.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.381+0000 7fdbf203e640 1 -- 172.21.15.45:0/4191256256 wait complete. 2024-05-08T05:41:50.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.385+0000 7f822d7fa640 1 -- 172.21.15.45:0/3373607171 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f82480b4c50 con 0x7f82480d1970 2024-05-08T05:41:50.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.385+0000 7f6a08f67640 1 -- 172.21.15.45:0/3151422062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f6a04050740 con 0x7f6a0415c290 2024-05-08T05:41:50.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.385+0000 7f822f7fe640 1 -- 172.21.15.45:0/3373607171 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8238060db0 con 0x7f82480d1970 2024-05-08T05:41:50.387 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2024-05-08T05:41:50.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.386+0000 7f6a00ff9640 1 -- 172.21.15.45:0/3151422062 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f69fc0603f0 con 0x7f6a0415c290 2024-05-08T05:41:50.388 INFO:teuthology.orchestra.run.smithi045.stdout:55834574855 2024-05-08T05:41:50.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.388+0000 7f822d7fa640 1 -- 172.21.15.45:0/3373607171 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8234076300 msgr2=0x7f82340787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.388+0000 7f822d7fa640 1 --2- 172.21.15.45:0/3373607171 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8234076300 0x7f82340787c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f824815bd00 tx=0x7f823c006c40 comp rx=0 tx=0).stop 2024-05-08T05:41:50.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.388+0000 7f822d7fa640 1 -- 172.21.15.45:0/3373607171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82480d1970 msgr2=0x7f824815af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.388+0000 7f822d7fa640 1 --2- 172.21.15.45:0/3373607171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82480d1970 0x7f824815af30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f823800b410 tx=0x7f823800b8d0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.388+0000 7f822d7fa640 1 -- 172.21.15.45:0/3373607171 shutdown_connections 2024-05-08T05:41:50.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.388+0000 7f822d7fa640 1 --2- 172.21.15.45:0/3373607171 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8234076300 0x7f82340787c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.388+0000 7f822d7fa640 1 --2- 172.21.15.45:0/3373607171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f824815d420 0x7f824815d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.388+0000 7f822d7fa640 1 --2- 172.21.15.45:0/3373607171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82480ca770 0x7f8248161260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.388+0000 7f822d7fa640 1 --2- 172.21.15.45:0/3373607171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82480d1970 0x7f824815af30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.388+0000 7f822d7fa640 1 -- 172.21.15.45:0/3373607171 >> 172.21.15.45:0/3373607171 conn(0x7f824808eac0 msgr2=0x7f824806e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:50.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.389+0000 7f822d7fa640 1 -- 172.21.15.45:0/3373607171 shutdown_connections 2024-05-08T05:41:50.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.389+0000 7f822d7fa640 1 -- 172.21.15.45:0/3373607171 wait complete. 2024-05-08T05:41:50.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.389+0000 7f6a08f67640 1 -- 172.21.15.45:0/3151422062 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f69e40763a0 msgr2=0x7f69e4078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.389+0000 7f6a08f67640 1 --2- 172.21.15.45:0/3151422062 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f69e40763a0 0x7f69e4078860 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6a041f06e0 tx=0x7f69f402e000 comp rx=0 tx=0).stop 2024-05-08T05:41:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.389+0000 7f6a08f67640 1 -- 172.21.15.45:0/3151422062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a0415c290 msgr2=0x7f6a040ab2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.389+0000 7f6a08f67640 1 --2- 172.21.15.45:0/3151422062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a0415c290 0x7f6a040ab2b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f69fc008ad0 tx=0x7f69fc008f90 comp rx=0 tx=0).stop 2024-05-08T05:41:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.389+0000 7f6a08f67640 1 -- 172.21.15.45:0/3151422062 shutdown_connections 2024-05-08T05:41:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.389+0000 7f6a08f67640 1 --2- 172.21.15.45:0/3151422062 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f69e40763a0 0x7f69e4078860 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.389+0000 7f6a08f67640 1 --2- 172.21.15.45:0/3151422062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6a0415c290 0x7f6a040ab2b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.389+0000 7f6a08f67640 1 --2- 172.21.15.45:0/3151422062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6a04158b70 0x7f6a040aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.389+0000 7f6a08f67640 1 --2- 172.21.15.45:0/3151422062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6a0406f6b0 0x7f6a041544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.389+0000 7f6a08f67640 1 -- 172.21.15.45:0/3151422062 >> 172.21.15.45:0/3151422062 conn(0x7f6a040b2da0 msgr2=0x7f6a0406ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.389+0000 7f6a08f67640 1 -- 172.21.15.45:0/3151422062 shutdown_connections 2024-05-08T05:41:50.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.389+0000 7f6a08f67640 1 -- 172.21.15.45:0/3151422062 wait complete. 2024-05-08T05:41:50.393 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.5 2024-05-08T05:41:50.393 DEBUG:teuthology.parallel:result is None 2024-05-08T05:41:50.399 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2024-05-08T05:41:50.399 DEBUG:teuthology.parallel:result is None 2024-05-08T05:41:50.401 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.7 2024-05-08T05:41:50.401 DEBUG:teuthology.parallel:result is None 2024-05-08T05:41:50.403 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-05-08T05:41:50.404 DEBUG:teuthology.parallel:result is None 2024-05-08T05:41:50.404 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-05-08T05:41:50.404 DEBUG:teuthology.parallel:result is None 2024-05-08T05:41:50.404 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T05:41:50.404 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:41:50.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.546+0000 7f4e979da640 1 Processor -- start 2024-05-08T05:41:50.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.546+0000 7f4e979da640 1 -- start start 2024-05-08T05:41:50.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.547+0000 7f4e979da640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4e9015ccb0 0x7f4e9015d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:50.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.547+0000 7f4e979da640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4e90070b90 0x7f4e90071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:50.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.547+0000 7f4e979da640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e900718c0 con 0x7f4e900d2ce0 2024-05-08T05:41:50.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.547+0000 7f4e979da640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4e900afba0 con 0x7f4e90070b90 2024-05-08T05:41:50.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.547+0000 7f4e979da640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4e90071ba0 con 0x7f4e9015ccb0 2024-05-08T05:41:50.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.547+0000 7f4e95f77640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4e90070b90 0x7f4e90071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.547+0000 7f4e94f75640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4e900d2ce0 0x7f4e900d3120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44028/0 (socket says 172.21.15.45:44028) 2024-05-08T05:41:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.547+0000 7f4e94f75640 1 -- 172.21.15.45:0/1198458710 learned_addr learned my addr 172.21.15.45:0/1198458710 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.548+0000 7f4e95776640 1 --2- 172.21.15.45:0/1198458710 >> v2:172.21.15.63:3300/0 conn(0x7f4e9015ccb0 0x7f4e9015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.548+0000 7f4e95f77640 1 -- 172.21.15.45:0/1198458710 >> v2:172.21.15.63:3300/0 conn(0x7f4e9015ccb0 msgr2=0x7f4e9015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.548+0000 7f4e95f77640 1 --2- 172.21.15.45:0/1198458710 >> v2:172.21.15.63:3300/0 conn(0x7f4e9015ccb0 0x7f4e9015d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.548+0000 7f4e95f77640 1 -- 172.21.15.45:0/1198458710 >> v1:172.21.15.45:6789/0 conn(0x7f4e900d2ce0 legacy=0x7f4e900d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.548+0000 7f4e95f77640 1 -- 172.21.15.45:0/1198458710 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e90071e80 con 0x7f4e90070b90 2024-05-08T05:41:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.548+0000 7f4e95f77640 1 --2- 172.21.15.45:0/1198458710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e90070b90 0x7f4e90071000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4e9006ff90 tx=0x7f4e8802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721834de5e0ee118 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.549+0000 7f4e8ffff640 1 -- 172.21.15.45:0/1198458710 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e88037070 con 0x7f4e90070b90 2024-05-08T05:41:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.549+0000 7f4e8ffff640 1 -- 172.21.15.45:0/1198458710 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e88032440 con 0x7f4e90070b90 2024-05-08T05:41:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.549+0000 7f4e8ffff640 1 -- 172.21.15.45:0/1198458710 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8802ae60 con 0x7f4e90070b90 2024-05-08T05:41:50.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.549+0000 7f4e979da640 1 -- 172.21.15.45:0/1198458710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e90070b90 msgr2=0x7f4e90071000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.549+0000 7f4e979da640 1 --2- 172.21.15.45:0/1198458710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e90070b90 0x7f4e90071000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4e9006ff90 tx=0x7f4e8802a110 comp rx=0 tx=0).stop 2024-05-08T05:41:50.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.549+0000 7f4e979da640 1 -- 172.21.15.45:0/1198458710 shutdown_connections 2024-05-08T05:41:50.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.549+0000 7f4e979da640 1 --2- 172.21.15.45:0/1198458710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e90070b90 0x7f4e90071000 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.549+0000 7f4e979da640 1 --2- 172.21.15.45:0/1198458710 >> v2:172.21.15.63:3300/0 conn(0x7f4e9015ccb0 0x7f4e9015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.549+0000 7f4e979da640 1 -- 172.21.15.45:0/1198458710 >> 172.21.15.45:0/1198458710 conn(0x7f4e9009ba40 msgr2=0x7f4e9009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:50.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.549+0000 7f4e979da640 1 -- 172.21.15.45:0/1198458710 shutdown_connections 2024-05-08T05:41:50.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.549+0000 7f4e979da640 1 -- 172.21.15.45:0/1198458710 wait complete. 2024-05-08T05:41:50.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.550+0000 7f4e979da640 1 Processor -- start 2024-05-08T05:41:50.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.551+0000 7f4e979da640 1 -- start start 2024-05-08T05:41:50.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.551+0000 7f4e979da640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e90070b90 0x7f4e901532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:50.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.551+0000 7f4e979da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e900d2ce0 0x7f4e9015d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:50.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.551+0000 7f4e979da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e9015ccb0 0x7f4e900b4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:50.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.551+0000 7f4e979da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4e900cf980 con 0x7f4e9015ccb0 2024-05-08T05:41:50.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.551+0000 7f4e979da640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4e90071540 con 0x7f4e90070b90 2024-05-08T05:41:50.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.551+0000 7f4e979da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4e900cfb00 con 0x7f4e900d2ce0 2024-05-08T05:41:50.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.551+0000 7f4e95f77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e9015ccb0 0x7f4e900b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:50.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.551+0000 7f4e95f77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e9015ccb0 0x7f4e900b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51604/0 (socket says 172.21.15.45:51604) 2024-05-08T05:41:50.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.551+0000 7f4e95f77640 1 -- 172.21.15.45:0/2423476927 learned_addr learned my addr 172.21.15.45:0/2423476927 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:50.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.552+0000 7f4e94f75640 1 --2- 172.21.15.45:0/2423476927 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e900d2ce0 0x7f4e9015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:50.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.552+0000 7f4e95776640 1 --2- 172.21.15.45:0/2423476927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e90070b90 0x7f4e901532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:50.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.552+0000 7f4e95f77640 1 -- 172.21.15.45:0/2423476927 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e900d2ce0 msgr2=0x7f4e9015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.552+0000 7f4e95f77640 1 --2- 172.21.15.45:0/2423476927 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e900d2ce0 0x7f4e9015d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.552+0000 7f4e95f77640 1 -- 172.21.15.45:0/2423476927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e90070b90 msgr2=0x7f4e901532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.552+0000 7f4e95f77640 1 --2- 172.21.15.45:0/2423476927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e90070b90 0x7f4e901532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.552+0000 7f4e95f77640 1 -- 172.21.15.45:0/2423476927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e900b5ad0 con 0x7f4e9015ccb0 2024-05-08T05:41:50.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.552+0000 7f4e95776640 1 --2- 172.21.15.45:0/2423476927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e90070b90 0x7f4e901532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:50.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.552+0000 7f4e94f75640 1 --2- 172.21.15.45:0/2423476927 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e900d2ce0 0x7f4e9015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:50.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.552+0000 7f4e95f77640 1 --2- 172.21.15.45:0/2423476927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e9015ccb0 0x7f4e900b4970 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e88035040 tx=0x7f4e880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:50.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.552+0000 7f4e8dffb640 1 -- 172.21.15.45:0/2423476927 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e88037040 con 0x7f4e9015ccb0 2024-05-08T05:41:50.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.552+0000 7f4e8dffb640 1 -- 172.21.15.45:0/2423476927 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e88042070 con 0x7f4e9015ccb0 2024-05-08T05:41:50.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.552+0000 7f4e979da640 1 -- 172.21.15.45:0/2423476927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e900b12d0 con 0x7f4e9015ccb0 2024-05-08T05:41:50.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.552+0000 7f4e8dffb640 1 -- 172.21.15.45:0/2423476927 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e88030b30 con 0x7f4e9015ccb0 2024-05-08T05:41:50.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.552+0000 7f4e979da640 1 -- 172.21.15.45:0/2423476927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e900b1890 con 0x7f4e9015ccb0 2024-05-08T05:41:50.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.553+0000 7f4e979da640 1 -- 172.21.15.45:0/2423476927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e90069b00 con 0x7f4e9015ccb0 2024-05-08T05:41:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.554+0000 7f4e8dffb640 1 -- 172.21.15.45:0/2423476927 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4e88030cd0 con 0x7f4e9015ccb0 2024-05-08T05:41:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.554+0000 7f4e8dffb640 1 --2- 172.21.15.45:0/2423476927 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4e7c0762d0 0x7f4e7c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.555+0000 7f4e8dffb640 1 -- 172.21.15.45:0/2423476927 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4e880b8380 con 0x7f4e9015ccb0 2024-05-08T05:41:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.555+0000 7f4e95776640 1 --2- 172.21.15.45:0/2423476927 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4e7c0762d0 0x7f4e7c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:50.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.555+0000 7f4e95776640 1 --2- 172.21.15.45:0/2423476927 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4e7c0762d0 0x7f4e7c078790 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4e80007330 tx=0x7f4e8000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:50.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.559+0000 7f4e8dffb640 1 -- 172.21.15.45:0/2423476927 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4e88032e20 con 0x7f4e9015ccb0 2024-05-08T05:41:50.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.746+0000 7f4e979da640 1 -- 172.21.15.45:0/2423476927 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e900b5710 con 0x7f4e7c0762d0 2024-05-08T05:41:50.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.755+0000 7f4e8dffb640 1 -- 172.21.15.45:0/2423476927 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332519 (secure 0 0 0) 0x7f4e8006c300 con 0x7f4e7c0762d0 2024-05-08T05:41:50.760 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:50.762 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:41:50.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.758+0000 7f4e979da640 1 -- 172.21.15.45:0/2423476927 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4e7c0762d0 msgr2=0x7f4e7c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.758+0000 7f4e979da640 1 --2- 172.21.15.45:0/2423476927 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4e7c0762d0 0x7f4e7c078790 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4e80007330 tx=0x7f4e8000b040 comp rx=0 tx=0).stop 2024-05-08T05:41:50.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.758+0000 7f4e979da640 1 -- 172.21.15.45:0/2423476927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e9015ccb0 msgr2=0x7f4e900b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.758+0000 7f4e979da640 1 --2- 172.21.15.45:0/2423476927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e9015ccb0 0x7f4e900b4970 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e88035040 tx=0x7f4e880046c0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.758+0000 7f4e979da640 1 -- 172.21.15.45:0/2423476927 shutdown_connections 2024-05-08T05:41:50.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.758+0000 7f4e979da640 1 --2- 172.21.15.45:0/2423476927 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4e7c0762d0 0x7f4e7c078790 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.758+0000 7f4e979da640 1 --2- 172.21.15.45:0/2423476927 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4e900d2ce0 0x7f4e9015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.758+0000 7f4e979da640 1 --2- 172.21.15.45:0/2423476927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e9015ccb0 0x7f4e900b4970 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.758+0000 7f4e979da640 1 --2- 172.21.15.45:0/2423476927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e90070b90 0x7f4e901532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.758+0000 7f4e979da640 1 -- 172.21.15.45:0/2423476927 >> 172.21.15.45:0/2423476927 conn(0x7f4e9009ba40 msgr2=0x7f4e900c8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:50.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.758+0000 7f4e979da640 1 -- 172.21.15.45:0/2423476927 shutdown_connections 2024-05-08T05:41:50.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.758+0000 7f4e979da640 1 -- 172.21.15.45:0/2423476927 wait complete. 2024-05-08T05:41:50.771 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-05-08T05:41:48.837552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236000,"kb_used_data":3176,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738720,"statfs":{"total":773094113280,"available":772852449280,"internally_reserved":0,"allocated":3252224,"data_stored":1985344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.686123"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575491+0000","last_change":"2024-05-08T05:41:31.127376+0000","last_active":"2024-05-08T05:41:37.575491+0000","last_peered":"2024-05-08T05:41:37.575491+0000","last_clean":"2024-05-08T05:41:37.575491+0000","last_became_active":"2024-05-08T05:41:31.127083+0000","last_became_peered":"2024-05-08T05:41:31.127083+0000","last_unstale":"2024-05-08T05:41:37.575491+0000","last_undegraded":"2024-05-08T05:41:37.575491+0000","last_fullsized":"2024-05-08T05:41:37.575491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:53.557101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576109+0000","last_change":"2024-05-08T05:41:32.133658+0000","last_active":"2024-05-08T05:41:37.576109+0000","last_peered":"2024-05-08T05:41:37.576109+0000","last_clean":"2024-05-08T05:41:37.576109+0000","last_became_active":"2024-05-08T05:41:32.133241+0000","last_became_peered":"2024-05-08T05:41:32.133241+0000","last_unstale":"2024-05-08T05:41:37.576109+0000","last_undegraded":"2024-05-08T05:41:37.576109+0000","last_fullsized":"2024-05-08T05:41:37.576109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:07:26.071605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576413+0000","last_change":"2024-05-08T05:41:36.385551+0000","last_active":"2024-05-08T05:41:37.576413+0000","last_peered":"2024-05-08T05:41:37.576413+0000","last_clean":"2024-05-08T05:41:37.576413+0000","last_became_active":"2024-05-08T05:41:36.385135+0000","last_became_peered":"2024-05-08T05:41:36.385135+0000","last_unstale":"2024-05-08T05:41:37.576413+0000","last_undegraded":"2024-05-08T05:41:37.576413+0000","last_fullsized":"2024-05-08T05:41:37.576413+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:39:51.537856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576344+0000","last_change":"2024-05-08T05:41:31.125574+0000","last_active":"2024-05-08T05:41:37.576344+0000","last_peered":"2024-05-08T05:41:37.576344+0000","last_clean":"2024-05-08T05:41:37.576344+0000","last_became_active":"2024-05-08T05:41:31.125302+0000","last_became_peered":"2024-05-08T05:41:31.125302+0000","last_unstale":"2024-05-08T05:41:37.576344+0000","last_undegraded":"2024-05-08T05:41:37.576344+0000","last_fullsized":"2024-05-08T05:41:37.576344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:39.672951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575165+0000","last_change":"2024-05-08T05:41:32.140528+0000","last_active":"2024-05-08T05:41:37.575165+0000","last_peered":"2024-05-08T05:41:37.575165+0000","last_clean":"2024-05-08T05:41:37.575165+0000","last_became_active":"2024-05-08T05:41:32.140227+0000","last_became_peered":"2024-05-08T05:41:32.140227+0000","last_unstale":"2024-05-08T05:41:37.575165+0000","last_undegraded":"2024-05-08T05:41:37.575165+0000","last_fullsized":"2024-05-08T05:41:37.575165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:30:07.184101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153167+0000","last_change":"2024-05-08T05:41:36.149551+0000","last_active":"2024-05-08T05:41:37.153167+0000","last_peered":"2024-05-08T05:41:37.153167+0000","last_clean":"2024-05-08T05:41:37.153167+0000","last_became_active":"2024-05-08T05:41:36.149115+0000","last_became_peered":"2024-05-08T05:41:36.149115+0000","last_unstale":"2024-05-08T05:41:37.153167+0000","last_undegraded":"2024-05-08T05:41:37.153167+0000","last_fullsized":"2024-05-08T05:41:37.153167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:57.593969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577133+0000","last_change":"2024-05-08T05:41:31.128105+0000","last_active":"2024-05-08T05:41:37.577133+0000","last_peered":"2024-05-08T05:41:37.577133+0000","last_clean":"2024-05-08T05:41:37.577133+0000","last_became_active":"2024-05-08T05:41:31.127782+0000","last_became_peered":"2024-05-08T05:41:31.127782+0000","last_unstale":"2024-05-08T05:41:37.577133+0000","last_undegraded":"2024-05-08T05:41:37.577133+0000","last_fullsized":"2024-05-08T05:41:37.577133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:46.418806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576628+0000","last_change":"2024-05-08T05:41:32.140667+0000","last_active":"2024-05-08T05:41:37.576628+0000","last_peered":"2024-05-08T05:41:37.576628+0000","last_clean":"2024-05-08T05:41:37.576628+0000","last_became_active":"2024-05-08T05:41:32.140365+0000","last_became_peered":"2024-05-08T05:41:32.140365+0000","last_unstale":"2024-05-08T05:41:37.576628+0000","last_undegraded":"2024-05-08T05:41:37.576628+0000","last_fullsized":"2024-05-08T05:41:37.576628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:02.517245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153785+0000","last_change":"2024-05-08T05:41:36.157609+0000","last_active":"2024-05-08T05:41:37.153785+0000","last_peered":"2024-05-08T05:41:37.153785+0000","last_clean":"2024-05-08T05:41:37.153785+0000","last_became_active":"2024-05-08T05:41:36.157214+0000","last_became_peered":"2024-05-08T05:41:36.157214+0000","last_unstale":"2024-05-08T05:41:37.153785+0000","last_undegraded":"2024-05-08T05:41:37.153785+0000","last_fullsized":"2024-05-08T05:41:37.153785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:46.769826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576228+0000","last_change":"2024-05-08T05:41:31.131582+0000","last_active":"2024-05-08T05:41:37.576228+0000","last_peered":"2024-05-08T05:41:37.576228+0000","last_clean":"2024-05-08T05:41:37.576228+0000","last_became_active":"2024-05-08T05:41:31.131302+0000","last_became_peered":"2024-05-08T05:41:31.131302+0000","last_unstale":"2024-05-08T05:41:37.576228+0000","last_undegraded":"2024-05-08T05:41:37.576228+0000","last_fullsized":"2024-05-08T05:41:37.576228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:07.648895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577648+0000","last_change":"2024-05-08T05:41:32.136792+0000","last_active":"2024-05-08T05:41:37.577648+0000","last_peered":"2024-05-08T05:41:37.577648+0000","last_clean":"2024-05-08T05:41:37.577648+0000","last_became_active":"2024-05-08T05:41:32.136285+0000","last_became_peered":"2024-05-08T05:41:32.136285+0000","last_unstale":"2024-05-08T05:41:37.577648+0000","last_undegraded":"2024-05-08T05:41:37.577648+0000","last_fullsized":"2024-05-08T05:41:37.577648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:53.826185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153415+0000","last_change":"2024-05-08T05:41:36.151839+0000","last_active":"2024-05-08T05:41:37.153415+0000","last_peered":"2024-05-08T05:41:37.153415+0000","last_clean":"2024-05-08T05:41:37.153415+0000","last_became_active":"2024-05-08T05:41:36.151387+0000","last_became_peered":"2024-05-08T05:41:36.151387+0000","last_unstale":"2024-05-08T05:41:37.153415+0000","last_undegraded":"2024-05-08T05:41:37.153415+0000","last_fullsized":"2024-05-08T05:41:37.153415+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:01:36.105428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576169+0000","last_change":"2024-05-08T05:41:32.136438+0000","last_active":"2024-05-08T05:41:37.576169+0000","last_peered":"2024-05-08T05:41:37.576169+0000","last_clean":"2024-05-08T05:41:37.576169+0000","last_became_active":"2024-05-08T05:41:32.136145+0000","last_became_peered":"2024-05-08T05:41:32.136145+0000","last_unstale":"2024-05-08T05:41:37.576169+0000","last_undegraded":"2024-05-08T05:41:37.576169+0000","last_fullsized":"2024-05-08T05:41:37.576169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:47.493805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576520+0000","last_change":"2024-05-08T05:41:31.124904+0000","last_active":"2024-05-08T05:41:37.576520+0000","last_peered":"2024-05-08T05:41:37.576520+0000","last_clean":"2024-05-08T05:41:37.576520+0000","last_became_active":"2024-05-08T05:41:31.124621+0000","last_became_peered":"2024-05-08T05:41:31.124621+0000","last_unstale":"2024-05-08T05:41:37.576520+0000","last_undegraded":"2024-05-08T05:41:37.576520+0000","last_fullsized":"2024-05-08T05:41:37.576520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:41.591349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154482+0000","last_change":"2024-05-08T05:41:36.154069+0000","last_active":"2024-05-08T05:41:37.154482+0000","last_peered":"2024-05-08T05:41:37.154482+0000","last_clean":"2024-05-08T05:41:37.154482+0000","last_became_active":"2024-05-08T05:41:36.153717+0000","last_became_peered":"2024-05-08T05:41:36.153717+0000","last_unstale":"2024-05-08T05:41:37.154482+0000","last_undegraded":"2024-05-08T05:41:37.154482+0000","last_fullsized":"2024-05-08T05:41:37.154482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:57.856276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577086+0000","last_change":"2024-05-08T05:41:32.137987+0000","last_active":"2024-05-08T05:41:37.577086+0000","last_peered":"2024-05-08T05:41:37.577086+0000","last_clean":"2024-05-08T05:41:37.577086+0000","last_became_active":"2024-05-08T05:41:32.137711+0000","last_became_peered":"2024-05-08T05:41:32.137711+0000","last_unstale":"2024-05-08T05:41:37.577086+0000","last_undegraded":"2024-05-08T05:41:37.577086+0000","last_fullsized":"2024-05-08T05:41:37.577086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:01:32.075337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576626+0000","last_change":"2024-05-08T05:41:31.121790+0000","last_active":"2024-05-08T05:41:37.576626+0000","last_peered":"2024-05-08T05:41:37.576626+0000","last_clean":"2024-05-08T05:41:37.576626+0000","last_became_active":"2024-05-08T05:41:31.121476+0000","last_became_peered":"2024-05-08T05:41:31.121476+0000","last_unstale":"2024-05-08T05:41:37.576626+0000","last_undegraded":"2024-05-08T05:41:37.576626+0000","last_fullsized":"2024-05-08T05:41:37.576626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:11:56.288788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154013+0000","last_change":"2024-05-08T05:41:36.157615+0000","last_active":"2024-05-08T05:41:37.154013+0000","last_peered":"2024-05-08T05:41:37.154013+0000","last_clean":"2024-05-08T05:41:37.154013+0000","last_became_active":"2024-05-08T05:41:36.157156+0000","last_became_peered":"2024-05-08T05:41:36.157156+0000","last_unstale":"2024-05-08T05:41:37.154013+0000","last_undegraded":"2024-05-08T05:41:37.154013+0000","last_fullsized":"2024-05-08T05:41:37.154013+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:46.115781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576120+0000","last_change":"2024-05-08T05:41:32.119867+0000","last_active":"2024-05-08T05:41:37.576120+0000","last_peered":"2024-05-08T05:41:37.576120+0000","last_clean":"2024-05-08T05:41:37.576120+0000","last_became_active":"2024-05-08T05:41:32.119186+0000","last_became_peered":"2024-05-08T05:41:32.119186+0000","last_unstale":"2024-05-08T05:41:37.576120+0000","last_undegraded":"2024-05-08T05:41:37.576120+0000","last_fullsized":"2024-05-08T05:41:37.576120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:47.493805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575164+0000","last_change":"2024-05-08T05:41:31.128486+0000","last_active":"2024-05-08T05:41:37.575164+0000","last_peered":"2024-05-08T05:41:37.575164+0000","last_clean":"2024-05-08T05:41:37.575164+0000","last_became_active":"2024-05-08T05:41:31.128207+0000","last_became_peered":"2024-05-08T05:41:31.128207+0000","last_unstale":"2024-05-08T05:41:37.575164+0000","last_undegraded":"2024-05-08T05:41:37.575164+0000","last_fullsized":"2024-05-08T05:41:37.575164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:46.418806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576817+0000","last_change":"2024-05-08T05:41:36.381823+0000","last_active":"2024-05-08T05:41:37.576817+0000","last_peered":"2024-05-08T05:41:37.576817+0000","last_clean":"2024-05-08T05:41:37.576817+0000","last_became_active":"2024-05-08T05:41:36.381421+0000","last_became_peered":"2024-05-08T05:41:36.381421+0000","last_unstale":"2024-05-08T05:41:37.576817+0000","last_undegraded":"2024-05-08T05:41:37.576817+0000","last_fullsized":"2024-05-08T05:41:37.576817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:21.321405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153397+0000","last_change":"2024-05-08T05:41:32.117529+0000","last_active":"2024-05-08T05:41:37.153397+0000","last_peered":"2024-05-08T05:41:37.153397+0000","last_clean":"2024-05-08T05:41:37.153397+0000","last_became_active":"2024-05-08T05:41:32.117205+0000","last_became_peered":"2024-05-08T05:41:32.117205+0000","last_unstale":"2024-05-08T05:41:37.153397+0000","last_undegraded":"2024-05-08T05:41:37.153397+0000","last_fullsized":"2024-05-08T05:41:37.153397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:08.654224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575429+0000","last_change":"2024-05-08T05:41:31.126184+0000","last_active":"2024-05-08T05:41:37.575429+0000","last_peered":"2024-05-08T05:41:37.575429+0000","last_clean":"2024-05-08T05:41:37.575429+0000","last_became_active":"2024-05-08T05:41:31.125860+0000","last_became_peered":"2024-05-08T05:41:31.125860+0000","last_unstale":"2024-05-08T05:41:37.575429+0000","last_undegraded":"2024-05-08T05:41:37.575429+0000","last_fullsized":"2024-05-08T05:41:37.575429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:13.465598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154197+0000","last_change":"2024-05-08T05:41:36.151020+0000","last_active":"2024-05-08T05:41:37.154197+0000","last_peered":"2024-05-08T05:41:37.154197+0000","last_clean":"2024-05-08T05:41:37.154197+0000","last_became_active":"2024-05-08T05:41:36.150320+0000","last_became_peered":"2024-05-08T05:41:36.150320+0000","last_unstale":"2024-05-08T05:41:37.154197+0000","last_undegraded":"2024-05-08T05:41:37.154197+0000","last_fullsized":"2024-05-08T05:41:37.154197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:06.547337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576056+0000","last_change":"2024-05-08T05:41:32.121088+0000","last_active":"2024-05-08T05:41:37.576056+0000","last_peered":"2024-05-08T05:41:37.576056+0000","last_clean":"2024-05-08T05:41:37.576056+0000","last_became_active":"2024-05-08T05:41:32.119585+0000","last_became_peered":"2024-05-08T05:41:32.119585+0000","last_unstale":"2024-05-08T05:41:37.576056+0000","last_undegraded":"2024-05-08T05:41:37.576056+0000","last_fullsized":"2024-05-08T05:41:37.576056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:18:52.796967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577379+0000","last_change":"2024-05-08T05:41:31.129202+0000","last_active":"2024-05-08T05:41:37.577379+0000","last_peered":"2024-05-08T05:41:37.577379+0000","last_clean":"2024-05-08T05:41:37.577379+0000","last_became_active":"2024-05-08T05:41:31.128911+0000","last_became_peered":"2024-05-08T05:41:31.128911+0000","last_unstale":"2024-05-08T05:41:37.577379+0000","last_undegraded":"2024-05-08T05:41:37.577379+0000","last_fullsized":"2024-05-08T05:41:37.577379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:13.465598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153283+0000","last_change":"2024-05-08T05:41:36.384181+0000","last_active":"2024-05-08T05:41:37.153283+0000","last_peered":"2024-05-08T05:41:37.153283+0000","last_clean":"2024-05-08T05:41:37.153283+0000","last_became_active":"2024-05-08T05:41:36.383800+0000","last_became_peered":"2024-05-08T05:41:36.383800+0000","last_unstale":"2024-05-08T05:41:37.153283+0000","last_undegraded":"2024-05-08T05:41:37.153283+0000","last_fullsized":"2024-05-08T05:41:37.153283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:33.503988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576885+0000","last_change":"2024-05-08T05:41:32.121597+0000","last_active":"2024-05-08T05:41:37.576885+0000","last_peered":"2024-05-08T05:41:37.576885+0000","last_clean":"2024-05-08T05:41:37.576885+0000","last_became_active":"2024-05-08T05:41:32.121321+0000","last_became_peered":"2024-05-08T05:41:32.121321+0000","last_unstale":"2024-05-08T05:41:37.576885+0000","last_undegraded":"2024-05-08T05:41:37.576885+0000","last_fullsized":"2024-05-08T05:41:37.576885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:42.739734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575444+0000","last_change":"2024-05-08T05:41:31.126359+0000","last_active":"2024-05-08T05:41:37.575444+0000","last_peered":"2024-05-08T05:41:37.575444+0000","last_clean":"2024-05-08T05:41:37.575444+0000","last_became_active":"2024-05-08T05:41:31.126069+0000","last_became_peered":"2024-05-08T05:41:31.126069+0000","last_unstale":"2024-05-08T05:41:37.575444+0000","last_undegraded":"2024-05-08T05:41:37.575444+0000","last_fullsized":"2024-05-08T05:41:37.575444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:24.722618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575700+0000","last_change":"2024-05-08T05:41:36.381909+0000","last_active":"2024-05-08T05:41:37.575700+0000","last_peered":"2024-05-08T05:41:37.575700+0000","last_clean":"2024-05-08T05:41:37.575700+0000","last_became_active":"2024-05-08T05:41:36.381251+0000","last_became_peered":"2024-05-08T05:41:36.381251+0000","last_unstale":"2024-05-08T05:41:37.575700+0000","last_undegraded":"2024-05-08T05:41:37.575700+0000","last_fullsized":"2024-05-08T05:41:37.575700+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:06:13.980953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576829+0000","last_change":"2024-05-08T05:41:32.133545+0000","last_active":"2024-05-08T05:41:37.576829+0000","last_peered":"2024-05-08T05:41:37.576829+0000","last_clean":"2024-05-08T05:41:37.576829+0000","last_became_active":"2024-05-08T05:41:32.132677+0000","last_became_peered":"2024-05-08T05:41:32.132677+0000","last_unstale":"2024-05-08T05:41:37.576829+0000","last_undegraded":"2024-05-08T05:41:37.576829+0000","last_fullsized":"2024-05-08T05:41:37.576829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:42.041915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577263+0000","last_change":"2024-05-08T05:41:31.127131+0000","last_active":"2024-05-08T05:41:37.577263+0000","last_peered":"2024-05-08T05:41:37.577263+0000","last_clean":"2024-05-08T05:41:37.577263+0000","last_became_active":"2024-05-08T05:41:31.126789+0000","last_became_peered":"2024-05-08T05:41:31.126789+0000","last_unstale":"2024-05-08T05:41:37.577263+0000","last_undegraded":"2024-05-08T05:41:37.577263+0000","last_fullsized":"2024-05-08T05:41:37.577263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:19:12.221462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575923+0000","last_change":"2024-05-08T05:41:36.381721+0000","last_active":"2024-05-08T05:41:37.575923+0000","last_peered":"2024-05-08T05:41:37.575923+0000","last_clean":"2024-05-08T05:41:37.575923+0000","last_became_active":"2024-05-08T05:41:36.381329+0000","last_became_peered":"2024-05-08T05:41:36.381329+0000","last_unstale":"2024-05-08T05:41:37.575923+0000","last_undegraded":"2024-05-08T05:41:37.575923+0000","last_fullsized":"2024-05-08T05:41:37.575923+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:17.202706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577782+0000","last_change":"2024-05-08T05:41:32.136558+0000","last_active":"2024-05-08T05:41:37.577782+0000","last_peered":"2024-05-08T05:41:37.577782+0000","last_clean":"2024-05-08T05:41:37.577782+0000","last_became_active":"2024-05-08T05:41:32.133354+0000","last_became_peered":"2024-05-08T05:41:32.133354+0000","last_unstale":"2024-05-08T05:41:37.577782+0000","last_undegraded":"2024-05-08T05:41:37.577782+0000","last_fullsized":"2024-05-08T05:41:37.577782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:13.172615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576762+0000","last_change":"2024-05-08T05:41:31.125582+0000","last_active":"2024-05-08T05:41:37.576762+0000","last_peered":"2024-05-08T05:41:37.576762+0000","last_clean":"2024-05-08T05:41:37.576762+0000","last_became_active":"2024-05-08T05:41:31.125252+0000","last_became_peered":"2024-05-08T05:41:31.125252+0000","last_unstale":"2024-05-08T05:41:37.576762+0000","last_undegraded":"2024-05-08T05:41:37.576762+0000","last_fullsized":"2024-05-08T05:41:37.576762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:38.174964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576435+0000","last_change":"2024-05-08T05:41:36.154344+0000","last_active":"2024-05-08T05:41:37.576435+0000","last_peered":"2024-05-08T05:41:37.576435+0000","last_clean":"2024-05-08T05:41:37.576435+0000","last_became_active":"2024-05-08T05:41:36.153930+0000","last_became_peered":"2024-05-08T05:41:36.153930+0000","last_unstale":"2024-05-08T05:41:37.576435+0000","last_undegraded":"2024-05-08T05:41:37.576435+0000","last_fullsized":"2024-05-08T05:41:37.576435+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:20.531197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154018+0000","last_change":"2024-05-08T05:41:32.136504+0000","last_active":"2024-05-08T05:41:37.154018+0000","last_peered":"2024-05-08T05:41:37.154018+0000","last_clean":"2024-05-08T05:41:37.154018+0000","last_became_active":"2024-05-08T05:41:32.135721+0000","last_became_peered":"2024-05-08T05:41:32.135721+0000","last_unstale":"2024-05-08T05:41:37.154018+0000","last_undegraded":"2024-05-08T05:41:37.154018+0000","last_fullsized":"2024-05-08T05:41:37.154018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:42.596678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576189+0000","last_change":"2024-05-08T05:41:31.131574+0000","last_active":"2024-05-08T05:41:37.576189+0000","last_peered":"2024-05-08T05:41:37.576189+0000","last_clean":"2024-05-08T05:41:37.576189+0000","last_became_active":"2024-05-08T05:41:31.131158+0000","last_became_peered":"2024-05-08T05:41:31.131158+0000","last_unstale":"2024-05-08T05:41:37.576189+0000","last_undegraded":"2024-05-08T05:41:37.576189+0000","last_fullsized":"2024-05-08T05:41:37.576189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:13.465598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576707+0000","last_change":"2024-05-08T05:41:36.159573+0000","last_active":"2024-05-08T05:41:37.576707+0000","last_peered":"2024-05-08T05:41:37.576707+0000","last_clean":"2024-05-08T05:41:37.576707+0000","last_became_active":"2024-05-08T05:41:36.159150+0000","last_became_peered":"2024-05-08T05:41:36.159150+0000","last_unstale":"2024-05-08T05:41:37.576707+0000","last_undegraded":"2024-05-08T05:41:37.576707+0000","last_fullsized":"2024-05-08T05:41:37.576707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:17.202706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576057+0000","last_change":"2024-05-08T05:41:32.121923+0000","last_active":"2024-05-08T05:41:37.576057+0000","last_peered":"2024-05-08T05:41:37.576057+0000","last_clean":"2024-05-08T05:41:37.576057+0000","last_became_active":"2024-05-08T05:41:32.121213+0000","last_became_peered":"2024-05-08T05:41:32.121213+0000","last_unstale":"2024-05-08T05:41:37.576057+0000","last_undegraded":"2024-05-08T05:41:37.576057+0000","last_fullsized":"2024-05-08T05:41:37.576057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:03:20.127059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.147846+0000","last_change":"2024-05-08T05:41:31.122162+0000","last_active":"2024-05-08T05:41:37.147846+0000","last_peered":"2024-05-08T05:41:37.147846+0000","last_clean":"2024-05-08T05:41:37.147846+0000","last_became_active":"2024-05-08T05:41:31.121865+0000","last_became_peered":"2024-05-08T05:41:31.121865+0000","last_unstale":"2024-05-08T05:41:37.147846+0000","last_undegraded":"2024-05-08T05:41:37.147846+0000","last_fullsized":"2024-05-08T05:41:37.147846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:24.722618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576949+0000","last_change":"2024-05-08T05:41:36.155633+0000","last_active":"2024-05-08T05:41:37.576949+0000","last_peered":"2024-05-08T05:41:37.576949+0000","last_clean":"2024-05-08T05:41:37.576949+0000","last_became_active":"2024-05-08T05:41:36.155125+0000","last_became_peered":"2024-05-08T05:41:36.155125+0000","last_unstale":"2024-05-08T05:41:37.576949+0000","last_undegraded":"2024-05-08T05:41:37.576949+0000","last_fullsized":"2024-05-08T05:41:37.576949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:50:36.282661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.574963+0000","last_change":"2024-05-08T05:41:32.119953+0000","last_active":"2024-05-08T05:41:37.574963+0000","last_peered":"2024-05-08T05:41:37.574963+0000","last_clean":"2024-05-08T05:41:37.574963+0000","last_became_active":"2024-05-08T05:41:32.119532+0000","last_became_peered":"2024-05-08T05:41:32.119532+0000","last_unstale":"2024-05-08T05:41:37.574963+0000","last_undegraded":"2024-05-08T05:41:37.574963+0000","last_fullsized":"2024-05-08T05:41:37.574963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:16:33.854897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575557+0000","last_change":"2024-05-08T05:41:31.125004+0000","last_active":"2024-05-08T05:41:37.575557+0000","last_peered":"2024-05-08T05:41:37.575557+0000","last_clean":"2024-05-08T05:41:37.575557+0000","last_became_active":"2024-05-08T05:41:31.124678+0000","last_became_peered":"2024-05-08T05:41:31.124678+0000","last_unstale":"2024-05-08T05:41:37.575557+0000","last_undegraded":"2024-05-08T05:41:37.575557+0000","last_fullsized":"2024-05-08T05:41:37.575557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:38.174964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575882+0000","last_change":"2024-05-08T05:41:36.385360+0000","last_active":"2024-05-08T05:41:37.575882+0000","last_peered":"2024-05-08T05:41:37.575882+0000","last_clean":"2024-05-08T05:41:37.575882+0000","last_became_active":"2024-05-08T05:41:36.384992+0000","last_became_peered":"2024-05-08T05:41:36.384992+0000","last_unstale":"2024-05-08T05:41:37.575882+0000","last_undegraded":"2024-05-08T05:41:37.575882+0000","last_fullsized":"2024-05-08T05:41:37.575882+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:20.531197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575341+0000","last_change":"2024-05-08T05:41:32.138842+0000","last_active":"2024-05-08T05:41:37.575341+0000","last_peered":"2024-05-08T05:41:37.575341+0000","last_clean":"2024-05-08T05:41:37.575341+0000","last_became_active":"2024-05-08T05:41:32.138583+0000","last_became_peered":"2024-05-08T05:41:32.138583+0000","last_unstale":"2024-05-08T05:41:37.575341+0000","last_undegraded":"2024-05-08T05:41:37.575341+0000","last_fullsized":"2024-05-08T05:41:37.575341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:53.563878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576353+0000","last_change":"2024-05-08T05:41:31.132616+0000","last_active":"2024-05-08T05:41:37.576353+0000","last_peered":"2024-05-08T05:41:37.576353+0000","last_clean":"2024-05-08T05:41:37.576353+0000","last_became_active":"2024-05-08T05:41:31.131462+0000","last_became_peered":"2024-05-08T05:41:31.131462+0000","last_unstale":"2024-05-08T05:41:37.576353+0000","last_undegraded":"2024-05-08T05:41:37.576353+0000","last_fullsized":"2024-05-08T05:41:37.576353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:24.722618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":118,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575672+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:41:37.575672+0000","last_peered":"2024-05-08T05:41:37.575672+0000","last_clean":"2024-05-08T05:41:37.575672+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:41:37.575672+0000","last_undegraded":"2024-05-08T05:41:37.575672+0000","last_fullsized":"2024-05-08T05:41:37.575672+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576785+0000","last_change":"2024-05-08T05:41:36.381947+0000","last_active":"2024-05-08T05:41:37.576785+0000","last_peered":"2024-05-08T05:41:37.576785+0000","last_clean":"2024-05-08T05:41:37.576785+0000","last_became_active":"2024-05-08T05:41:36.381491+0000","last_became_peered":"2024-05-08T05:41:36.381491+0000","last_unstale":"2024-05-08T05:41:37.576785+0000","last_undegraded":"2024-05-08T05:41:37.576785+0000","last_fullsized":"2024-05-08T05:41:37.576785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:57.856276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576983+0000","last_change":"2024-05-08T05:41:32.134397+0000","last_active":"2024-05-08T05:41:37.576983+0000","last_peered":"2024-05-08T05:41:37.576983+0000","last_clean":"2024-05-08T05:41:37.576983+0000","last_became_active":"2024-05-08T05:41:32.134098+0000","last_became_peered":"2024-05-08T05:41:32.134098+0000","last_unstale":"2024-05-08T05:41:37.576983+0000","last_undegraded":"2024-05-08T05:41:37.576983+0000","last_fullsized":"2024-05-08T05:41:37.576983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:16.501105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.148234+0000","last_change":"2024-05-08T05:41:31.133531+0000","last_active":"2024-05-08T05:41:37.148234+0000","last_peered":"2024-05-08T05:41:37.148234+0000","last_clean":"2024-05-08T05:41:37.148234+0000","last_became_active":"2024-05-08T05:41:31.133208+0000","last_became_peered":"2024-05-08T05:41:31.133208+0000","last_unstale":"2024-05-08T05:41:37.148234+0000","last_undegraded":"2024-05-08T05:41:37.148234+0000","last_fullsized":"2024-05-08T05:41:37.148234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:22.751500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576104+0000","last_change":"2024-05-08T05:41:36.381903+0000","last_active":"2024-05-08T05:41:37.576104+0000","last_peered":"2024-05-08T05:41:37.576104+0000","last_clean":"2024-05-08T05:41:37.576104+0000","last_became_active":"2024-05-08T05:41:36.381397+0000","last_became_peered":"2024-05-08T05:41:36.381397+0000","last_unstale":"2024-05-08T05:41:37.576104+0000","last_undegraded":"2024-05-08T05:41:37.576104+0000","last_fullsized":"2024-05-08T05:41:37.576104+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:06.547337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576725+0000","last_change":"2024-05-08T05:41:32.128554+0000","last_active":"2024-05-08T05:41:37.576725+0000","last_peered":"2024-05-08T05:41:37.576725+0000","last_clean":"2024-05-08T05:41:37.576725+0000","last_became_active":"2024-05-08T05:41:32.128280+0000","last_became_peered":"2024-05-08T05:41:32.128280+0000","last_unstale":"2024-05-08T05:41:37.576725+0000","last_undegraded":"2024-05-08T05:41:37.576725+0000","last_fullsized":"2024-05-08T05:41:37.576725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:42.085689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576546+0000","last_change":"2024-05-08T05:41:31.122071+0000","last_active":"2024-05-08T05:41:37.576546+0000","last_peered":"2024-05-08T05:41:37.576546+0000","last_clean":"2024-05-08T05:41:37.576546+0000","last_became_active":"2024-05-08T05:41:31.121137+0000","last_became_peered":"2024-05-08T05:41:31.121137+0000","last_unstale":"2024-05-08T05:41:37.576546+0000","last_undegraded":"2024-05-08T05:41:37.576546+0000","last_fullsized":"2024-05-08T05:41:37.576546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:13.465598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576176+0000","last_change":"2024-05-08T05:41:36.160917+0000","last_active":"2024-05-08T05:41:37.576176+0000","last_peered":"2024-05-08T05:41:37.576176+0000","last_clean":"2024-05-08T05:41:37.576176+0000","last_became_active":"2024-05-08T05:41:36.160457+0000","last_became_peered":"2024-05-08T05:41:36.160457+0000","last_unstale":"2024-05-08T05:41:37.576176+0000","last_undegraded":"2024-05-08T05:41:37.576176+0000","last_fullsized":"2024-05-08T05:41:37.576176+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:21.321405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577384+0000","last_change":"2024-05-08T05:41:32.131012+0000","last_active":"2024-05-08T05:41:37.577384+0000","last_peered":"2024-05-08T05:41:37.577384+0000","last_clean":"2024-05-08T05:41:37.577384+0000","last_became_active":"2024-05-08T05:41:32.130682+0000","last_became_peered":"2024-05-08T05:41:32.130682+0000","last_unstale":"2024-05-08T05:41:37.577384+0000","last_undegraded":"2024-05-08T05:41:37.577384+0000","last_fullsized":"2024-05-08T05:41:37.577384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:06:09.950861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.148282+0000","last_change":"2024-05-08T05:41:31.122329+0000","last_active":"2024-05-08T05:41:37.148282+0000","last_peered":"2024-05-08T05:41:37.148282+0000","last_clean":"2024-05-08T05:41:37.148282+0000","last_became_active":"2024-05-08T05:41:31.122021+0000","last_became_peered":"2024-05-08T05:41:31.122021+0000","last_unstale":"2024-05-08T05:41:37.148282+0000","last_undegraded":"2024-05-08T05:41:37.148282+0000","last_fullsized":"2024-05-08T05:41:37.148282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:11:56.288788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575492+0000","last_change":"2024-05-08T05:41:36.382122+0000","last_active":"2024-05-08T05:41:37.575492+0000","last_peered":"2024-05-08T05:41:37.575492+0000","last_clean":"2024-05-08T05:41:37.575492+0000","last_became_active":"2024-05-08T05:41:36.381385+0000","last_became_peered":"2024-05-08T05:41:36.381385+0000","last_unstale":"2024-05-08T05:41:37.575492+0000","last_undegraded":"2024-05-08T05:41:37.575492+0000","last_fullsized":"2024-05-08T05:41:37.575492+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:02.479497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576083+0000","last_change":"2024-05-08T05:41:32.131255+0000","last_active":"2024-05-08T05:41:37.576083+0000","last_peered":"2024-05-08T05:41:37.576083+0000","last_clean":"2024-05-08T05:41:37.576083+0000","last_became_active":"2024-05-08T05:41:32.130252+0000","last_became_peered":"2024-05-08T05:41:32.130252+0000","last_unstale":"2024-05-08T05:41:37.576083+0000","last_undegraded":"2024-05-08T05:41:37.576083+0000","last_fullsized":"2024-05-08T05:41:37.576083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:17.291314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154349+0000","last_change":"2024-05-08T05:41:31.125770+0000","last_active":"2024-05-08T05:41:37.154349+0000","last_peered":"2024-05-08T05:41:37.154349+0000","last_clean":"2024-05-08T05:41:37.154349+0000","last_became_active":"2024-05-08T05:41:31.125442+0000","last_became_peered":"2024-05-08T05:41:31.125442+0000","last_unstale":"2024-05-08T05:41:37.154349+0000","last_undegraded":"2024-05-08T05:41:37.154349+0000","last_fullsized":"2024-05-08T05:41:37.154349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:53.557101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153167+0000","last_change":"2024-05-08T05:41:36.151074+0000","last_active":"2024-05-08T05:41:37.153167+0000","last_peered":"2024-05-08T05:41:37.153167+0000","last_clean":"2024-05-08T05:41:37.153167+0000","last_became_active":"2024-05-08T05:41:36.150613+0000","last_became_peered":"2024-05-08T05:41:36.150613+0000","last_unstale":"2024-05-08T05:41:37.153167+0000","last_undegraded":"2024-05-08T05:41:37.153167+0000","last_fullsized":"2024-05-08T05:41:37.153167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:46.072007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576628+0000","last_change":"2024-05-08T05:41:32.132197+0000","last_active":"2024-05-08T05:41:37.576628+0000","last_peered":"2024-05-08T05:41:37.576628+0000","last_clean":"2024-05-08T05:41:37.576628+0000","last_became_active":"2024-05-08T05:41:32.131086+0000","last_became_peered":"2024-05-08T05:41:32.131086+0000","last_unstale":"2024-05-08T05:41:37.576628+0000","last_undegraded":"2024-05-08T05:41:37.576628+0000","last_fullsized":"2024-05-08T05:41:37.576628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:02.517245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.147676+0000","last_change":"2024-05-08T05:41:31.120099+0000","last_active":"2024-05-08T05:41:37.147676+0000","last_peered":"2024-05-08T05:41:37.147676+0000","last_clean":"2024-05-08T05:41:37.147676+0000","last_became_active":"2024-05-08T05:41:31.119750+0000","last_became_peered":"2024-05-08T05:41:31.119750+0000","last_unstale":"2024-05-08T05:41:37.147676+0000","last_undegraded":"2024-05-08T05:41:37.147676+0000","last_fullsized":"2024-05-08T05:41:37.147676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:49.171947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575735+0000","last_change":"2024-05-08T05:41:36.162020+0000","last_active":"2024-05-08T05:41:37.575735+0000","last_peered":"2024-05-08T05:41:37.575735+0000","last_clean":"2024-05-08T05:41:37.575735+0000","last_became_active":"2024-05-08T05:41:36.161613+0000","last_became_peered":"2024-05-08T05:41:36.161613+0000","last_unstale":"2024-05-08T05:41:37.575735+0000","last_undegraded":"2024-05-08T05:41:37.575735+0000","last_fullsized":"2024-05-08T05:41:37.575735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:57.856276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575424+0000","last_change":"2024-05-08T05:41:32.137565+0000","last_active":"2024-05-08T05:41:37.575424+0000","last_peered":"2024-05-08T05:41:37.575424+0000","last_clean":"2024-05-08T05:41:37.575424+0000","last_became_active":"2024-05-08T05:41:32.137078+0000","last_became_peered":"2024-05-08T05:41:32.137078+0000","last_unstale":"2024-05-08T05:41:37.575424+0000","last_undegraded":"2024-05-08T05:41:37.575424+0000","last_fullsized":"2024-05-08T05:41:37.575424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:44.038894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576417+0000","last_change":"2024-05-08T05:41:31.127117+0000","last_active":"2024-05-08T05:41:37.576417+0000","last_peered":"2024-05-08T05:41:37.576417+0000","last_clean":"2024-05-08T05:41:37.576417+0000","last_became_active":"2024-05-08T05:41:31.126826+0000","last_became_peered":"2024-05-08T05:41:31.126826+0000","last_unstale":"2024-05-08T05:41:37.576417+0000","last_undegraded":"2024-05-08T05:41:37.576417+0000","last_fullsized":"2024-05-08T05:41:37.576417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:22.751500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576723+0000","last_change":"2024-05-08T05:41:36.383650+0000","last_active":"2024-05-08T05:41:37.576723+0000","last_peered":"2024-05-08T05:41:37.576723+0000","last_clean":"2024-05-08T05:41:37.576723+0000","last_became_active":"2024-05-08T05:41:36.382759+0000","last_became_peered":"2024-05-08T05:41:36.382759+0000","last_unstale":"2024-05-08T05:41:37.576723+0000","last_undegraded":"2024-05-08T05:41:37.576723+0000","last_fullsized":"2024-05-08T05:41:37.576723+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:43:10.562469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576336+0000","last_change":"2024-05-08T05:41:32.130461+0000","last_active":"2024-05-08T05:41:37.576336+0000","last_peered":"2024-05-08T05:41:37.576336+0000","last_clean":"2024-05-08T05:41:37.576336+0000","last_became_active":"2024-05-08T05:41:32.130160+0000","last_became_peered":"2024-05-08T05:41:32.130160+0000","last_unstale":"2024-05-08T05:41:37.576336+0000","last_undegraded":"2024-05-08T05:41:37.576336+0000","last_fullsized":"2024-05-08T05:41:37.576336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:42.085689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576077+0000","last_change":"2024-05-08T05:41:31.132478+0000","last_active":"2024-05-08T05:41:37.576077+0000","last_peered":"2024-05-08T05:41:37.576077+0000","last_clean":"2024-05-08T05:41:37.576077+0000","last_became_active":"2024-05-08T05:41:31.131093+0000","last_became_peered":"2024-05-08T05:41:31.131093+0000","last_unstale":"2024-05-08T05:41:37.576077+0000","last_undegraded":"2024-05-08T05:41:37.576077+0000","last_fullsized":"2024-05-08T05:41:37.576077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:07.648895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577230+0000","last_change":"2024-05-08T05:41:36.381669+0000","last_active":"2024-05-08T05:41:37.577230+0000","last_peered":"2024-05-08T05:41:37.577230+0000","last_clean":"2024-05-08T05:41:37.577230+0000","last_became_active":"2024-05-08T05:41:36.381233+0000","last_became_peered":"2024-05-08T05:41:36.381233+0000","last_unstale":"2024-05-08T05:41:37.577230+0000","last_undegraded":"2024-05-08T05:41:37.577230+0000","last_fullsized":"2024-05-08T05:41:37.577230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:07.864670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575368+0000","last_change":"2024-05-08T05:41:32.131789+0000","last_active":"2024-05-08T05:41:37.575368+0000","last_peered":"2024-05-08T05:41:37.575368+0000","last_clean":"2024-05-08T05:41:37.575368+0000","last_became_active":"2024-05-08T05:41:32.131515+0000","last_became_peered":"2024-05-08T05:41:32.131515+0000","last_unstale":"2024-05-08T05:41:37.575368+0000","last_undegraded":"2024-05-08T05:41:37.575368+0000","last_fullsized":"2024-05-08T05:41:37.575368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:03:20.127059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576417+0000","last_change":"2024-05-08T05:41:31.126864+0000","last_active":"2024-05-08T05:41:37.576417+0000","last_peered":"2024-05-08T05:41:37.576417+0000","last_clean":"2024-05-08T05:41:37.576417+0000","last_became_active":"2024-05-08T05:41:31.126568+0000","last_became_peered":"2024-05-08T05:41:31.126568+0000","last_unstale":"2024-05-08T05:41:37.576417+0000","last_undegraded":"2024-05-08T05:41:37.576417+0000","last_fullsized":"2024-05-08T05:41:37.576417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:39.672951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577692+0000","last_change":"2024-05-08T05:41:36.382511+0000","last_active":"2024-05-08T05:41:37.577692+0000","last_peered":"2024-05-08T05:41:37.577692+0000","last_clean":"2024-05-08T05:41:37.577692+0000","last_became_active":"2024-05-08T05:41:36.382110+0000","last_became_peered":"2024-05-08T05:41:36.382110+0000","last_unstale":"2024-05-08T05:41:37.577692+0000","last_undegraded":"2024-05-08T05:41:37.577692+0000","last_fullsized":"2024-05-08T05:41:37.577692+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:39:51.537856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576868+0000","last_change":"2024-05-08T05:41:32.127674+0000","last_active":"2024-05-08T05:41:37.576868+0000","last_peered":"2024-05-08T05:41:37.576868+0000","last_clean":"2024-05-08T05:41:37.576868+0000","last_became_active":"2024-05-08T05:41:32.127374+0000","last_became_peered":"2024-05-08T05:41:32.127374+0000","last_unstale":"2024-05-08T05:41:37.576868+0000","last_undegraded":"2024-05-08T05:41:37.576868+0000","last_fullsized":"2024-05-08T05:41:37.576868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:53.826185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:41.770215+0000","last_change":"2024-05-08T05:41:31.126507+0000","last_active":"2024-05-08T05:41:41.770215+0000","last_peered":"2024-05-08T05:41:41.770215+0000","last_clean":"2024-05-08T05:41:41.770215+0000","last_became_active":"2024-05-08T05:41:31.126218+0000","last_became_peered":"2024-05-08T05:41:31.126218+0000","last_unstale":"2024-05-08T05:41:41.770215+0000","last_undegraded":"2024-05-08T05:41:41.770215+0000","last_fullsized":"2024-05-08T05:41:41.770215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:11:56.288788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576865+0000","last_change":"2024-05-08T05:41:36.153303+0000","last_active":"2024-05-08T05:41:37.576865+0000","last_peered":"2024-05-08T05:41:37.576865+0000","last_clean":"2024-05-08T05:41:37.576865+0000","last_became_active":"2024-05-08T05:41:36.152849+0000","last_became_peered":"2024-05-08T05:41:36.152849+0000","last_unstale":"2024-05-08T05:41:37.576865+0000","last_undegraded":"2024-05-08T05:41:37.576865+0000","last_fullsized":"2024-05-08T05:41:37.576865+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:07:30.112175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154093+0000","last_change":"2024-05-08T05:41:32.121425+0000","last_active":"2024-05-08T05:41:37.154093+0000","last_peered":"2024-05-08T05:41:37.154093+0000","last_clean":"2024-05-08T05:41:37.154093+0000","last_became_active":"2024-05-08T05:41:32.121121+0000","last_became_peered":"2024-05-08T05:41:32.121121+0000","last_unstale":"2024-05-08T05:41:37.154093+0000","last_undegraded":"2024-05-08T05:41:37.154093+0000","last_fullsized":"2024-05-08T05:41:37.154093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:47.424136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576434+0000","last_change":"2024-05-08T05:41:31.132128+0000","last_active":"2024-05-08T05:41:37.576434+0000","last_peered":"2024-05-08T05:41:37.576434+0000","last_clean":"2024-05-08T05:41:37.576434+0000","last_became_active":"2024-05-08T05:41:31.131776+0000","last_became_peered":"2024-05-08T05:41:31.131776+0000","last_unstale":"2024-05-08T05:41:37.576434+0000","last_undegraded":"2024-05-08T05:41:37.576434+0000","last_fullsized":"2024-05-08T05:41:37.576434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:07.648895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153781+0000","last_change":"2024-05-08T05:41:36.152157+0000","last_active":"2024-05-08T05:41:37.153781+0000","last_peered":"2024-05-08T05:41:37.153781+0000","last_clean":"2024-05-08T05:41:37.153781+0000","last_became_active":"2024-05-08T05:41:36.151495+0000","last_became_peered":"2024-05-08T05:41:36.151495+0000","last_unstale":"2024-05-08T05:41:37.153781+0000","last_undegraded":"2024-05-08T05:41:37.153781+0000","last_fullsized":"2024-05-08T05:41:37.153781+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:34.172975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575332+0000","last_change":"2024-05-08T05:41:32.137651+0000","last_active":"2024-05-08T05:41:37.575332+0000","last_peered":"2024-05-08T05:41:37.575332+0000","last_clean":"2024-05-08T05:41:37.575332+0000","last_became_active":"2024-05-08T05:41:32.137380+0000","last_became_peered":"2024-05-08T05:41:32.137380+0000","last_unstale":"2024-05-08T05:41:37.575332+0000","last_undegraded":"2024-05-08T05:41:37.575332+0000","last_fullsized":"2024-05-08T05:41:37.575332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:44.038894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577757+0000","last_change":"2024-05-08T05:41:31.132598+0000","last_active":"2024-05-08T05:41:37.577757+0000","last_peered":"2024-05-08T05:41:37.577757+0000","last_clean":"2024-05-08T05:41:37.577757+0000","last_became_active":"2024-05-08T05:41:31.131053+0000","last_became_peered":"2024-05-08T05:41:31.131053+0000","last_unstale":"2024-05-08T05:41:37.577757+0000","last_undegraded":"2024-05-08T05:41:37.577757+0000","last_fullsized":"2024-05-08T05:41:37.577757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:11:56.288788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575991+0000","last_change":"2024-05-08T05:41:36.159470+0000","last_active":"2024-05-08T05:41:37.575991+0000","last_peered":"2024-05-08T05:41:37.575991+0000","last_clean":"2024-05-08T05:41:37.575991+0000","last_became_active":"2024-05-08T05:41:36.158246+0000","last_became_peered":"2024-05-08T05:41:36.158246+0000","last_unstale":"2024-05-08T05:41:37.575991+0000","last_undegraded":"2024-05-08T05:41:37.575991+0000","last_fullsized":"2024-05-08T05:41:37.575991+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:02.479497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577667+0000","last_change":"2024-05-08T05:41:32.139258+0000","last_active":"2024-05-08T05:41:37.577667+0000","last_peered":"2024-05-08T05:41:37.577667+0000","last_clean":"2024-05-08T05:41:37.577667+0000","last_became_active":"2024-05-08T05:41:32.138975+0000","last_became_peered":"2024-05-08T05:41:32.138975+0000","last_unstale":"2024-05-08T05:41:37.577667+0000","last_undegraded":"2024-05-08T05:41:37.577667+0000","last_fullsized":"2024-05-08T05:41:37.577667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:16.501105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577263+0000","last_change":"2024-05-08T05:41:31.133643+0000","last_active":"2024-05-08T05:41:37.577263+0000","last_peered":"2024-05-08T05:41:37.577263+0000","last_clean":"2024-05-08T05:41:37.577263+0000","last_became_active":"2024-05-08T05:41:31.133256+0000","last_became_peered":"2024-05-08T05:41:31.133256+0000","last_unstale":"2024-05-08T05:41:37.577263+0000","last_undegraded":"2024-05-08T05:41:37.577263+0000","last_fullsized":"2024-05-08T05:41:37.577263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:07.648895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576551+0000","last_change":"2024-05-08T05:41:36.383032+0000","last_active":"2024-05-08T05:41:37.576551+0000","last_peered":"2024-05-08T05:41:37.576551+0000","last_clean":"2024-05-08T05:41:37.576551+0000","last_became_active":"2024-05-08T05:41:36.382632+0000","last_became_peered":"2024-05-08T05:41:36.382632+0000","last_unstale":"2024-05-08T05:41:37.576551+0000","last_undegraded":"2024-05-08T05:41:37.576551+0000","last_fullsized":"2024-05-08T05:41:37.576551+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:07:08.116447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575585+0000","last_change":"2024-05-08T05:41:36.385447+0000","last_active":"2024-05-08T05:41:37.575585+0000","last_peered":"2024-05-08T05:41:37.575585+0000","last_clean":"2024-05-08T05:41:37.575585+0000","last_became_active":"2024-05-08T05:41:36.385062+0000","last_became_peered":"2024-05-08T05:41:36.385062+0000","last_unstale":"2024-05-08T05:41:37.575585+0000","last_undegraded":"2024-05-08T05:41:37.575585+0000","last_fullsized":"2024-05-08T05:41:37.575585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:06.547337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.574973+0000","last_change":"2024-05-08T05:41:32.127321+0000","last_active":"2024-05-08T05:41:37.574973+0000","last_peered":"2024-05-08T05:41:37.574973+0000","last_clean":"2024-05-08T05:41:37.574973+0000","last_became_active":"2024-05-08T05:41:32.127028+0000","last_became_peered":"2024-05-08T05:41:32.127028+0000","last_unstale":"2024-05-08T05:41:37.574973+0000","last_undegraded":"2024-05-08T05:41:37.574973+0000","last_fullsized":"2024-05-08T05:41:37.574973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:47.493805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575819+0000","last_change":"2024-05-08T05:41:31.131618+0000","last_active":"2024-05-08T05:41:37.575819+0000","last_peered":"2024-05-08T05:41:37.575819+0000","last_clean":"2024-05-08T05:41:37.575819+0000","last_became_active":"2024-05-08T05:41:31.131012+0000","last_became_peered":"2024-05-08T05:41:31.131012+0000","last_unstale":"2024-05-08T05:41:37.575819+0000","last_undegraded":"2024-05-08T05:41:37.575819+0000","last_fullsized":"2024-05-08T05:41:37.575819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:22.751500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577121+0000","last_change":"2024-05-08T05:41:36.158506+0000","last_active":"2024-05-08T05:41:37.577121+0000","last_peered":"2024-05-08T05:41:37.577121+0000","last_clean":"2024-05-08T05:41:37.577121+0000","last_became_active":"2024-05-08T05:41:36.158020+0000","last_became_peered":"2024-05-08T05:41:36.158020+0000","last_unstale":"2024-05-08T05:41:37.577121+0000","last_undegraded":"2024-05-08T05:41:37.577121+0000","last_fullsized":"2024-05-08T05:41:37.577121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:20.531197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576038+0000","last_change":"2024-05-08T05:41:32.132554+0000","last_active":"2024-05-08T05:41:37.576038+0000","last_peered":"2024-05-08T05:41:37.576038+0000","last_clean":"2024-05-08T05:41:37.576038+0000","last_became_active":"2024-05-08T05:41:32.132257+0000","last_became_peered":"2024-05-08T05:41:32.132257+0000","last_unstale":"2024-05-08T05:41:37.576038+0000","last_undegraded":"2024-05-08T05:41:37.576038+0000","last_fullsized":"2024-05-08T05:41:37.576038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:16:33.854897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576780+0000","last_change":"2024-05-08T05:41:31.124710+0000","last_active":"2024-05-08T05:41:37.576780+0000","last_peered":"2024-05-08T05:41:37.576780+0000","last_clean":"2024-05-08T05:41:37.576780+0000","last_became_active":"2024-05-08T05:41:31.124304+0000","last_became_peered":"2024-05-08T05:41:31.124304+0000","last_unstale":"2024-05-08T05:41:37.576780+0000","last_undegraded":"2024-05-08T05:41:37.576780+0000","last_fullsized":"2024-05-08T05:41:37.576780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:49.171947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576513+0000","last_change":"2024-05-08T05:41:31.123731+0000","last_active":"2024-05-08T05:41:37.576513+0000","last_peered":"2024-05-08T05:41:37.576513+0000","last_clean":"2024-05-08T05:41:37.576513+0000","last_became_active":"2024-05-08T05:41:31.123423+0000","last_became_peered":"2024-05-08T05:41:31.123423+0000","last_unstale":"2024-05-08T05:41:37.576513+0000","last_undegraded":"2024-05-08T05:41:37.576513+0000","last_fullsized":"2024-05-08T05:41:37.576513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:24.722618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576325+0000","last_change":"2024-05-08T05:41:32.135788+0000","last_active":"2024-05-08T05:41:37.576325+0000","last_peered":"2024-05-08T05:41:37.576325+0000","last_clean":"2024-05-08T05:41:37.576325+0000","last_became_active":"2024-05-08T05:41:32.135522+0000","last_became_peered":"2024-05-08T05:41:32.135522+0000","last_unstale":"2024-05-08T05:41:37.576325+0000","last_undegraded":"2024-05-08T05:41:37.576325+0000","last_fullsized":"2024-05-08T05:41:37.576325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:42.041915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.148134+0000","last_change":"2024-05-08T05:41:36.155967+0000","last_active":"2024-05-08T05:41:37.148134+0000","last_peered":"2024-05-08T05:41:37.148134+0000","last_clean":"2024-05-08T05:41:37.148134+0000","last_became_active":"2024-05-08T05:41:36.155132+0000","last_became_peered":"2024-05-08T05:41:36.155132+0000","last_unstale":"2024-05-08T05:41:37.148134+0000","last_undegraded":"2024-05-08T05:41:37.148134+0000","last_fullsized":"2024-05-08T05:41:37.148134+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:58.729110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577152+0000","last_change":"2024-05-08T05:41:31.127645+0000","last_active":"2024-05-08T05:41:37.577152+0000","last_peered":"2024-05-08T05:41:37.577152+0000","last_clean":"2024-05-08T05:41:37.577152+0000","last_became_active":"2024-05-08T05:41:31.127330+0000","last_became_peered":"2024-05-08T05:41:31.127330+0000","last_unstale":"2024-05-08T05:41:37.577152+0000","last_undegraded":"2024-05-08T05:41:37.577152+0000","last_fullsized":"2024-05-08T05:41:37.577152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:22.751500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153479+0000","last_change":"2024-05-08T05:41:32.125718+0000","last_active":"2024-05-08T05:41:37.153479+0000","last_peered":"2024-05-08T05:41:37.153479+0000","last_clean":"2024-05-08T05:41:37.153479+0000","last_became_active":"2024-05-08T05:41:32.125435+0000","last_became_peered":"2024-05-08T05:41:32.125435+0000","last_unstale":"2024-05-08T05:41:37.153479+0000","last_undegraded":"2024-05-08T05:41:37.153479+0000","last_fullsized":"2024-05-08T05:41:37.153479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:39.180293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575823+0000","last_change":"2024-05-08T05:41:36.158103+0000","last_active":"2024-05-08T05:41:37.575823+0000","last_peered":"2024-05-08T05:41:37.575823+0000","last_clean":"2024-05-08T05:41:37.575823+0000","last_became_active":"2024-05-08T05:41:36.157668+0000","last_became_peered":"2024-05-08T05:41:36.157668+0000","last_unstale":"2024-05-08T05:41:37.575823+0000","last_undegraded":"2024-05-08T05:41:37.575823+0000","last_fullsized":"2024-05-08T05:41:37.575823+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:45:24.742821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575306+0000","last_change":"2024-05-08T05:41:32.137321+0000","last_active":"2024-05-08T05:41:37.575306+0000","last_peered":"2024-05-08T05:41:37.575306+0000","last_clean":"2024-05-08T05:41:37.575306+0000","last_became_active":"2024-05-08T05:41:32.136738+0000","last_became_peered":"2024-05-08T05:41:32.136738+0000","last_unstale":"2024-05-08T05:41:37.575306+0000","last_undegraded":"2024-05-08T05:41:37.575306+0000","last_fullsized":"2024-05-08T05:41:37.575306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:16:33.854897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.147790+0000","last_change":"2024-05-08T05:41:32.127130+0000","last_active":"2024-05-08T05:41:37.147790+0000","last_peered":"2024-05-08T05:41:37.147790+0000","last_clean":"2024-05-08T05:41:37.147790+0000","last_became_active":"2024-05-08T05:41:32.125853+0000","last_became_peered":"2024-05-08T05:41:32.125853+0000","last_unstale":"2024-05-08T05:41:37.147790+0000","last_undegraded":"2024-05-08T05:41:37.147790+0000","last_fullsized":"2024-05-08T05:41:37.147790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:47.424136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575968+0000","last_change":"2024-05-08T05:41:32.139558+0000","last_active":"2024-05-08T05:41:37.575968+0000","last_peered":"2024-05-08T05:41:37.575968+0000","last_clean":"2024-05-08T05:41:37.575968+0000","last_became_active":"2024-05-08T05:41:32.139243+0000","last_became_peered":"2024-05-08T05:41:32.139243+0000","last_unstale":"2024-05-08T05:41:37.575968+0000","last_undegraded":"2024-05-08T05:41:37.575968+0000","last_fullsized":"2024-05-08T05:41:37.575968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:07:26.082084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576235+0000","last_change":"2024-05-08T05:41:32.126388+0000","last_active":"2024-05-08T05:41:37.576235+0000","last_peered":"2024-05-08T05:41:37.576235+0000","last_clean":"2024-05-08T05:41:37.576235+0000","last_became_active":"2024-05-08T05:41:32.126078+0000","last_became_peered":"2024-05-08T05:41:32.126078+0000","last_unstale":"2024-05-08T05:41:37.576235+0000","last_undegraded":"2024-05-08T05:41:37.576235+0000","last_fullsized":"2024-05-08T05:41:37.576235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:53.563878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577307+0000","last_change":"2024-05-08T05:41:32.135523+0000","last_active":"2024-05-08T05:41:37.577307+0000","last_peered":"2024-05-08T05:41:37.577307+0000","last_clean":"2024-05-08T05:41:37.577307+0000","last_became_active":"2024-05-08T05:41:32.135248+0000","last_became_peered":"2024-05-08T05:41:32.135248+0000","last_unstale":"2024-05-08T05:41:37.577307+0000","last_undegraded":"2024-05-08T05:41:37.577307+0000","last_fullsized":"2024-05-08T05:41:37.577307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:42.041915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576262+0000","last_change":"2024-05-08T05:41:32.130227+0000","last_active":"2024-05-08T05:41:37.576262+0000","last_peered":"2024-05-08T05:41:37.576262+0000","last_clean":"2024-05-08T05:41:37.576262+0000","last_became_active":"2024-05-08T05:41:32.129175+0000","last_became_peered":"2024-05-08T05:41:32.129175+0000","last_unstale":"2024-05-08T05:41:37.576262+0000","last_undegraded":"2024-05-08T05:41:37.576262+0000","last_fullsized":"2024-05-08T05:41:37.576262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:53.563878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577357+0000","last_change":"2024-05-08T05:41:32.143284+0000","last_active":"2024-05-08T05:41:37.577357+0000","last_peered":"2024-05-08T05:41:37.577357+0000","last_clean":"2024-05-08T05:41:37.577357+0000","last_became_active":"2024-05-08T05:41:32.142815+0000","last_became_peered":"2024-05-08T05:41:32.142815+0000","last_unstale":"2024-05-08T05:41:37.577357+0000","last_undegraded":"2024-05-08T05:41:37.577357+0000","last_fullsized":"2024-05-08T05:41:37.577357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:53.563878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576778+0000","last_change":"2024-05-08T05:41:32.134965+0000","last_active":"2024-05-08T05:41:37.576778+0000","last_peered":"2024-05-08T05:41:37.576778+0000","last_clean":"2024-05-08T05:41:37.576778+0000","last_became_active":"2024-05-08T05:41:32.134641+0000","last_became_peered":"2024-05-08T05:41:32.134641+0000","last_unstale":"2024-05-08T05:41:37.576778+0000","last_undegraded":"2024-05-08T05:41:37.576778+0000","last_fullsized":"2024-05-08T05:41:37.576778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:30.142883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575749+0000","last_change":"2024-05-08T05:41:32.135139+0000","last_active":"2024-05-08T05:41:37.575749+0000","last_peered":"2024-05-08T05:41:37.575749+0000","last_clean":"2024-05-08T05:41:37.575749+0000","last_became_active":"2024-05-08T05:41:32.134839+0000","last_became_peered":"2024-05-08T05:41:32.134839+0000","last_unstale":"2024-05-08T05:41:37.575749+0000","last_undegraded":"2024-05-08T05:41:37.575749+0000","last_fullsized":"2024-05-08T05:41:37.575749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:53.563878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576660+0000","last_change":"2024-05-08T05:41:32.134673+0000","last_active":"2024-05-08T05:41:37.576660+0000","last_peered":"2024-05-08T05:41:37.576660+0000","last_clean":"2024-05-08T05:41:37.576660+0000","last_became_active":"2024-05-08T05:41:32.134402+0000","last_became_peered":"2024-05-08T05:41:32.134402+0000","last_unstale":"2024-05-08T05:41:37.576660+0000","last_undegraded":"2024-05-08T05:41:37.576660+0000","last_fullsized":"2024-05-08T05:41:37.576660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:03:20.127059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575846+0000","last_change":"2024-05-08T05:41:32.140242+0000","last_active":"2024-05-08T05:41:37.575846+0000","last_peered":"2024-05-08T05:41:37.575846+0000","last_clean":"2024-05-08T05:41:37.575846+0000","last_became_active":"2024-05-08T05:41:32.139932+0000","last_became_peered":"2024-05-08T05:41:32.139932+0000","last_unstale":"2024-05-08T05:41:37.575846+0000","last_undegraded":"2024-05-08T05:41:37.575846+0000","last_fullsized":"2024-05-08T05:41:37.575846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:42.739734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575975+0000","last_change":"2024-05-08T05:41:32.132509+0000","last_active":"2024-05-08T05:41:37.575975+0000","last_peered":"2024-05-08T05:41:37.575975+0000","last_clean":"2024-05-08T05:41:37.575975+0000","last_became_active":"2024-05-08T05:41:32.132240+0000","last_became_peered":"2024-05-08T05:41:32.132240+0000","last_unstale":"2024-05-08T05:41:37.575975+0000","last_undegraded":"2024-05-08T05:41:37.575975+0000","last_fullsized":"2024-05-08T05:41:37.575975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:07:26.071605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576973+0000","last_change":"2024-05-08T05:41:32.140982+0000","last_active":"2024-05-08T05:41:37.576973+0000","last_peered":"2024-05-08T05:41:37.576973+0000","last_clean":"2024-05-08T05:41:37.576973+0000","last_became_active":"2024-05-08T05:41:32.140698+0000","last_became_peered":"2024-05-08T05:41:32.140698+0000","last_unstale":"2024-05-08T05:41:37.576973+0000","last_undegraded":"2024-05-08T05:41:37.576973+0000","last_fullsized":"2024-05-08T05:41:37.576973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:07:26.071605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577173+0000","last_change":"2024-05-08T05:41:32.134190+0000","last_active":"2024-05-08T05:41:37.577173+0000","last_peered":"2024-05-08T05:41:37.577173+0000","last_clean":"2024-05-08T05:41:37.577173+0000","last_became_active":"2024-05-08T05:41:32.132164+0000","last_became_peered":"2024-05-08T05:41:32.132164+0000","last_unstale":"2024-05-08T05:41:37.577173+0000","last_undegraded":"2024-05-08T05:41:37.577173+0000","last_fullsized":"2024-05-08T05:41:37.577173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:13.172615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.148157+0000","last_change":"2024-05-08T05:41:32.131091+0000","last_active":"2024-05-08T05:41:37.148157+0000","last_peered":"2024-05-08T05:41:37.148157+0000","last_clean":"2024-05-08T05:41:37.148157+0000","last_became_active":"2024-05-08T05:41:32.130767+0000","last_became_peered":"2024-05-08T05:41:32.130767+0000","last_unstale":"2024-05-08T05:41:37.148157+0000","last_undegraded":"2024-05-08T05:41:37.148157+0000","last_fullsized":"2024-05-08T05:41:37.148157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:42.596678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577931+0000","last_change":"2024-05-08T05:41:32.137943+0000","last_active":"2024-05-08T05:41:37.577931+0000","last_peered":"2024-05-08T05:41:37.577931+0000","last_clean":"2024-05-08T05:41:37.577931+0000","last_became_active":"2024-05-08T05:41:32.137662+0000","last_became_peered":"2024-05-08T05:41:32.137662+0000","last_unstale":"2024-05-08T05:41:37.577931+0000","last_undegraded":"2024-05-08T05:41:37.577931+0000","last_fullsized":"2024-05-08T05:41:37.577931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:17.291314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575635+0000","last_change":"2024-05-08T05:41:32.129374+0000","last_active":"2024-05-08T05:41:37.575635+0000","last_peered":"2024-05-08T05:41:37.575635+0000","last_clean":"2024-05-08T05:41:37.575635+0000","last_became_active":"2024-05-08T05:41:32.128594+0000","last_became_peered":"2024-05-08T05:41:32.128594+0000","last_unstale":"2024-05-08T05:41:37.575635+0000","last_undegraded":"2024-05-08T05:41:37.575635+0000","last_fullsized":"2024-05-08T05:41:37.575635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:01:32.075337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575072+0000","last_change":"2024-05-08T05:41:32.131159+0000","last_active":"2024-05-08T05:41:37.575072+0000","last_peered":"2024-05-08T05:41:37.575072+0000","last_clean":"2024-05-08T05:41:37.575072+0000","last_became_active":"2024-05-08T05:41:32.130766+0000","last_became_peered":"2024-05-08T05:41:32.130766+0000","last_unstale":"2024-05-08T05:41:37.575072+0000","last_undegraded":"2024-05-08T05:41:37.575072+0000","last_fullsized":"2024-05-08T05:41:37.575072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:16:33.854897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575282+0000","last_change":"2024-05-08T05:41:32.119797+0000","last_active":"2024-05-08T05:41:37.575282+0000","last_peered":"2024-05-08T05:41:37.575282+0000","last_clean":"2024-05-08T05:41:37.575282+0000","last_became_active":"2024-05-08T05:41:32.119134+0000","last_became_peered":"2024-05-08T05:41:32.119134+0000","last_unstale":"2024-05-08T05:41:37.575282+0000","last_undegraded":"2024-05-08T05:41:37.575282+0000","last_fullsized":"2024-05-08T05:41:37.575282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:47.493805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577070+0000","last_change":"2024-05-08T05:41:32.135351+0000","last_active":"2024-05-08T05:41:37.577070+0000","last_peered":"2024-05-08T05:41:37.577070+0000","last_clean":"2024-05-08T05:41:37.577070+0000","last_became_active":"2024-05-08T05:41:32.133812+0000","last_became_peered":"2024-05-08T05:41:32.133812+0000","last_unstale":"2024-05-08T05:41:37.577070+0000","last_undegraded":"2024-05-08T05:41:37.577070+0000","last_fullsized":"2024-05-08T05:41:37.577070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:29.473897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154048+0000","last_change":"2024-05-08T05:41:32.136108+0000","last_active":"2024-05-08T05:41:37.154048+0000","last_peered":"2024-05-08T05:41:37.154048+0000","last_clean":"2024-05-08T05:41:37.154048+0000","last_became_active":"2024-05-08T05:41:32.135622+0000","last_became_peered":"2024-05-08T05:41:32.135622+0000","last_unstale":"2024-05-08T05:41:37.154048+0000","last_undegraded":"2024-05-08T05:41:37.154048+0000","last_fullsized":"2024-05-08T05:41:37.154048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:14.470927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576751+0000","last_change":"2024-05-08T05:41:32.138499+0000","last_active":"2024-05-08T05:41:37.576751+0000","last_peered":"2024-05-08T05:41:37.576751+0000","last_clean":"2024-05-08T05:41:37.576751+0000","last_became_active":"2024-05-08T05:41:32.138049+0000","last_became_peered":"2024-05-08T05:41:32.138049+0000","last_unstale":"2024-05-08T05:41:37.576751+0000","last_undegraded":"2024-05-08T05:41:37.576751+0000","last_fullsized":"2024-05-08T05:41:37.576751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:03:20.127059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577193+0000","last_change":"2024-05-08T05:41:32.138310+0000","last_active":"2024-05-08T05:41:37.577193+0000","last_peered":"2024-05-08T05:41:37.577193+0000","last_clean":"2024-05-08T05:41:37.577193+0000","last_became_active":"2024-05-08T05:41:32.137983+0000","last_became_peered":"2024-05-08T05:41:32.137983+0000","last_unstale":"2024-05-08T05:41:37.577193+0000","last_undegraded":"2024-05-08T05:41:37.577193+0000","last_fullsized":"2024-05-08T05:41:37.577193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:42.041915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577267+0000","last_change":"2024-05-08T05:41:32.144208+0000","last_active":"2024-05-08T05:41:37.577267+0000","last_peered":"2024-05-08T05:41:37.577267+0000","last_clean":"2024-05-08T05:41:37.577267+0000","last_became_active":"2024-05-08T05:41:32.143884+0000","last_became_peered":"2024-05-08T05:41:32.143884+0000","last_unstale":"2024-05-08T05:41:37.577267+0000","last_undegraded":"2024-05-08T05:41:37.577267+0000","last_fullsized":"2024-05-08T05:41:37.577267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:07:26.082084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575800+0000","last_change":"2024-05-08T05:41:32.136185+0000","last_active":"2024-05-08T05:41:37.575800+0000","last_peered":"2024-05-08T05:41:37.575800+0000","last_clean":"2024-05-08T05:41:37.575800+0000","last_became_active":"2024-05-08T05:41:32.135878+0000","last_became_peered":"2024-05-08T05:41:32.135878+0000","last_unstale":"2024-05-08T05:41:37.575800+0000","last_undegraded":"2024-05-08T05:41:37.575800+0000","last_fullsized":"2024-05-08T05:41:37.575800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:44.038894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576529+0000","last_change":"2024-05-08T05:41:32.137009+0000","last_active":"2024-05-08T05:41:37.576529+0000","last_peered":"2024-05-08T05:41:37.576529+0000","last_clean":"2024-05-08T05:41:37.576529+0000","last_became_active":"2024-05-08T05:41:32.136708+0000","last_became_peered":"2024-05-08T05:41:32.136708+0000","last_unstale":"2024-05-08T05:41:37.576529+0000","last_undegraded":"2024-05-08T05:41:37.576529+0000","last_fullsized":"2024-05-08T05:41:37.576529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:30.142883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575732+0000","last_change":"2024-05-08T05:41:32.137351+0000","last_active":"2024-05-08T05:41:37.575732+0000","last_peered":"2024-05-08T05:41:37.575732+0000","last_clean":"2024-05-08T05:41:37.575732+0000","last_became_active":"2024-05-08T05:41:32.135445+0000","last_became_peered":"2024-05-08T05:41:32.135445+0000","last_unstale":"2024-05-08T05:41:37.575732+0000","last_undegraded":"2024-05-08T05:41:37.575732+0000","last_fullsized":"2024-05-08T05:41:37.575732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:30.142883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575746+0000","last_change":"2024-05-08T05:41:32.132566+0000","last_active":"2024-05-08T05:41:37.575746+0000","last_peered":"2024-05-08T05:41:37.575746+0000","last_clean":"2024-05-08T05:41:37.575746+0000","last_became_active":"2024-05-08T05:41:32.131787+0000","last_became_peered":"2024-05-08T05:41:32.131787+0000","last_unstale":"2024-05-08T05:41:37.575746+0000","last_undegraded":"2024-05-08T05:41:37.575746+0000","last_fullsized":"2024-05-08T05:41:37.575746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:42.739734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576521+0000","last_change":"2024-05-08T05:41:32.135611+0000","last_active":"2024-05-08T05:41:37.576521+0000","last_peered":"2024-05-08T05:41:37.576521+0000","last_clean":"2024-05-08T05:41:37.576521+0000","last_became_active":"2024-05-08T05:41:32.135346+0000","last_became_peered":"2024-05-08T05:41:32.135346+0000","last_unstale":"2024-05-08T05:41:37.576521+0000","last_undegraded":"2024-05-08T05:41:37.576521+0000","last_fullsized":"2024-05-08T05:41:37.576521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:29.473897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575102+0000","last_change":"2024-05-08T05:41:32.118779+0000","last_active":"2024-05-08T05:41:37.575102+0000","last_peered":"2024-05-08T05:41:37.575102+0000","last_clean":"2024-05-08T05:41:37.575102+0000","last_became_active":"2024-05-08T05:41:32.118481+0000","last_became_peered":"2024-05-08T05:41:32.118481+0000","last_unstale":"2024-05-08T05:41:37.575102+0000","last_undegraded":"2024-05-08T05:41:37.575102+0000","last_fullsized":"2024-05-08T05:41:37.575102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:54.699019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575394+0000","last_change":"2024-05-08T05:41:32.128168+0000","last_active":"2024-05-08T05:41:37.575394+0000","last_peered":"2024-05-08T05:41:37.575394+0000","last_clean":"2024-05-08T05:41:37.575394+0000","last_became_active":"2024-05-08T05:41:32.127869+0000","last_became_peered":"2024-05-08T05:41:32.127869+0000","last_unstale":"2024-05-08T05:41:37.575394+0000","last_undegraded":"2024-05-08T05:41:37.575394+0000","last_fullsized":"2024-05-08T05:41:37.575394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:07:26.071605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577048+0000","last_change":"2024-05-08T05:41:32.134768+0000","last_active":"2024-05-08T05:41:37.577048+0000","last_peered":"2024-05-08T05:41:37.577048+0000","last_clean":"2024-05-08T05:41:37.577048+0000","last_became_active":"2024-05-08T05:41:32.132540+0000","last_became_peered":"2024-05-08T05:41:32.132540+0000","last_unstale":"2024-05-08T05:41:37.577048+0000","last_undegraded":"2024-05-08T05:41:37.577048+0000","last_fullsized":"2024-05-08T05:41:37.577048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:50:32.252569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":847872,"data_stored":690366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":262144,"data_stored":103258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":99506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":99542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":262144,"data_stored":103258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":249856,"data_stored":99416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":258048,"data_stored":99576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":856064,"data_stored":690422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:41:50.776 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:41:50.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.915+0000 7f4b28fe4640 1 Processor -- start 2024-05-08T05:41:50.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.915+0000 7f4b28fe4640 1 -- start start 2024-05-08T05:41:50.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.916+0000 7f4b28fe4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b240d1970 0x7f4b240d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:50.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.916+0000 7f4b28fe4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4b240d4cc0 0x7f4b240d5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:50.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.916+0000 7f4b28fe4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b241630b0 con 0x7f4b2415f420 2024-05-08T05:41:50.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.916+0000 7f4b28fe4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4b240d2330 con 0x7f4b240d4cc0 2024-05-08T05:41:50.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.916+0000 7f4b28fe4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b240d3e70 con 0x7f4b240d1970 2024-05-08T05:41:50.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.916+0000 7f4b2259c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4b2415f420 0x7f4b240d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49634/0 (socket says 172.21.15.45:49634) 2024-05-08T05:41:50.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.916+0000 7f4b2259c640 1 -- 172.21.15.45:0/3131760382 learned_addr learned my addr 172.21.15.45:0/3131760382 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:50.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b22d9d640 1 --2- 172.21.15.45:0/3131760382 >> v2:172.21.15.63:3300/0 conn(0x7f4b240d4cc0 0x7f4b240d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:50.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b21d9b640 1 --2- 172.21.15.45:0/3131760382 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b240d1970 0x7f4b240d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:50.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b2159a640 1 -- 172.21.15.45:0/3131760382 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1931730236 0 0) 0x7f4b14001490 con 0x7f4b2415f420 2024-05-08T05:41:50.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b2159a640 1 -- 172.21.15.45:0/3131760382 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b10003520 con 0x7f4b2415f420 2024-05-08T05:41:50.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b2159a640 1 -- 172.21.15.45:0/3131760382 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3067521504 0 0) 0x7f4b10003520 con 0x7f4b2415f420 2024-05-08T05:41:50.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b2159a640 1 -- 172.21.15.45:0/3131760382 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b240d1970 msgr2=0x7f4b240d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:50.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b2159a640 1 --2- 172.21.15.45:0/3131760382 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b240d1970 0x7f4b240d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b2159a640 1 -- 172.21.15.45:0/3131760382 >> v2:172.21.15.63:3300/0 conn(0x7f4b240d4cc0 msgr2=0x7f4b240d5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:50.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b2159a640 1 --2- 172.21.15.45:0/3131760382 >> v2:172.21.15.63:3300/0 conn(0x7f4b240d4cc0 0x7f4b240d5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b2159a640 1 -- 172.21.15.45:0/3131760382 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b240d5670 con 0x7f4b2415f420 2024-05-08T05:41:50.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b2159a640 1 -- 172.21.15.45:0/3131760382 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f4b14001c60 con 0x7f4b2415f420 2024-05-08T05:41:50.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b22d9d640 1 --2- 172.21.15.45:0/3131760382 >> v2:172.21.15.63:3300/0 conn(0x7f4b240d4cc0 0x7f4b240d5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:41:50.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b2159a640 1 -- 172.21.15.45:0/3131760382 >> v1:172.21.15.45:6789/0 conn(0x7f4b2415f420 legacy=0x7f4b240d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b2159a640 1 --2- 172.21.15.45:0/3131760382 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b240d1970 0x7f4b10003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:50.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b2159a640 1 --2- 172.21.15.45:0/3131760382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b10003fd0 0x7f4b10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:50.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b2159a640 1 --2- 172.21.15.45:0/3131760382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b10004d40 0x7f4b100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:50.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b2159a640 1 -- 172.21.15.45:0/3131760382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b240d2330 con 0x7f4b10003fd0 2024-05-08T05:41:50.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b2159a640 1 -- 172.21.15.45:0/3131760382 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b240d3e70 con 0x7f4b10004d40 2024-05-08T05:41:50.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b2159a640 1 -- 172.21.15.45:0/3131760382 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b1000ba10 con 0x7f4b240d1970 2024-05-08T05:41:50.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b21d9b640 1 --2- 172.21.15.45:0/3131760382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b10003fd0 0x7f4b10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:50.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b2259c640 1 --2- 172.21.15.45:0/3131760382 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b240d1970 0x7f4b10003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:50.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b22d9d640 1 --2- 172.21.15.45:0/3131760382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b10004d40 0x7f4b100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:50.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b21d9b640 1 -- 172.21.15.45:0/3131760382 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b240d1970 msgr2=0x7f4b10003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:50.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b21d9b640 1 --2- 172.21.15.45:0/3131760382 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b240d1970 0x7f4b10003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b21d9b640 1 -- 172.21.15.45:0/3131760382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b10004d40 msgr2=0x7f4b100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:50.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b21d9b640 1 --2- 172.21.15.45:0/3131760382 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b10004d40 0x7f4b100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.917+0000 7f4b21d9b640 1 -- 172.21.15.45:0/3131760382 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b10003520 con 0x7f4b10003fd0 2024-05-08T05:41:50.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.918+0000 7f4b21d9b640 1 --2- 172.21.15.45:0/3131760382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b10003fd0 0x7f4b10004800 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009af0 tx=0x7f4b10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a15605bda7f47c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:50.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.918+0000 7f4b2159a640 1 -- 172.21.15.45:0/3131760382 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c036070 con 0x7f4b10003fd0 2024-05-08T05:41:50.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.918+0000 7f4b2159a640 1 -- 172.21.15.45:0/3131760382 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b0c031400 con 0x7f4b10003fd0 2024-05-08T05:41:50.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.918+0000 7f4b2159a640 1 -- 172.21.15.45:0/3131760382 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c009690 con 0x7f4b10003fd0 2024-05-08T05:41:50.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.919+0000 7f4b28fe4640 1 -- 172.21.15.45:0/3131760382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b10003fd0 msgr2=0x7f4b10004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.919+0000 7f4b28fe4640 1 --2- 172.21.15.45:0/3131760382 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b10003fd0 0x7f4b10004800 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009af0 tx=0x7f4b10005830 comp rx=0 tx=0).stop 2024-05-08T05:41:50.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.919+0000 7f4b22d9d640 1 -- 172.21.15.45:0/3131760382 reap_dead start 2024-05-08T05:41:50.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.919+0000 7f4b28fe4640 1 -- 172.21.15.45:0/3131760382 shutdown_connections 2024-05-08T05:41:50.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.919+0000 7f4b28fe4640 1 -- 172.21.15.45:0/3131760382 >> 172.21.15.45:0/3131760382 conn(0x7f4b2408eac0 msgr2=0x7f4b2409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:50.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.919+0000 7f4b28fe4640 1 -- 172.21.15.45:0/3131760382 shutdown_connections 2024-05-08T05:41:50.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.919+0000 7f4b28fe4640 1 -- 172.21.15.45:0/3131760382 wait complete. 2024-05-08T05:41:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.920+0000 7f4b28fe4640 1 Processor -- start 2024-05-08T05:41:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.920+0000 7f4b28fe4640 1 -- start start 2024-05-08T05:41:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.921+0000 7f4b28fe4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b10003fd0 0x7f4b24155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.921+0000 7f4b28fe4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b240ba190 0x7f4b240ba610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.921+0000 7f4b28fe4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b240b5210 0x7f4b240b5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.921+0000 7f4b28fe4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b240d2f60 con 0x7f4b240ba190 2024-05-08T05:41:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.921+0000 7f4b28fe4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b240d2c80 con 0x7f4b10003fd0 2024-05-08T05:41:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.921+0000 7f4b28fe4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b240d3240 con 0x7f4b240b5210 2024-05-08T05:41:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.921+0000 7f4b21d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b240ba190 0x7f4b240ba610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.921+0000 7f4b21d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b240ba190 0x7f4b240ba610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34666/0 (socket says 172.21.15.45:34666) 2024-05-08T05:41:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.921+0000 7f4b21d9b640 1 -- 172.21.15.45:0/455125603 learned_addr learned my addr 172.21.15.45:0/455125603 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.921+0000 7f4b2259c640 1 --2- 172.21.15.45:0/455125603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b10003fd0 0x7f4b24155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.921+0000 7f4b22d9d640 1 --2- 172.21.15.45:0/455125603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b240b5210 0x7f4b240b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.922+0000 7f4b21d9b640 1 -- 172.21.15.45:0/455125603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b240b5210 msgr2=0x7f4b240b5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.922+0000 7f4b21d9b640 1 --2- 172.21.15.45:0/455125603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b240b5210 0x7f4b240b5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.922+0000 7f4b21d9b640 1 -- 172.21.15.45:0/455125603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b10003fd0 msgr2=0x7f4b24155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.922+0000 7f4b21d9b640 1 --2- 172.21.15.45:0/455125603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b10003fd0 0x7f4b24155280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.922+0000 7f4b21d9b640 1 -- 172.21.15.45:0/455125603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b240b7dd0 con 0x7f4b240ba190 2024-05-08T05:41:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.922+0000 7f4b21d9b640 1 --2- 172.21.15.45:0/455125603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b240ba190 0x7f4b240ba610 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c034040 tx=0x7f4b0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.922+0000 7f4b1affd640 1 -- 172.21.15.45:0/455125603 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c040020 con 0x7f4b240ba190 2024-05-08T05:41:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.922+0000 7f4b1affd640 1 -- 172.21.15.45:0/455125603 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b0c003c60 con 0x7f4b240ba190 2024-05-08T05:41:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.922+0000 7f4b28fe4640 1 -- 172.21.15.45:0/455125603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b240b80b0 con 0x7f4b240ba190 2024-05-08T05:41:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.922+0000 7f4b1affd640 1 -- 172.21.15.45:0/455125603 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c036040 con 0x7f4b240ba190 2024-05-08T05:41:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.922+0000 7f4b28fe4640 1 -- 172.21.15.45:0/455125603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b240b8670 con 0x7f4b240ba190 2024-05-08T05:41:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.923+0000 7f4b28fe4640 1 -- 172.21.15.45:0/455125603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b240b5bd0 con 0x7f4b240ba190 2024-05-08T05:41:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.924+0000 7f4b1affd640 1 -- 172.21.15.45:0/455125603 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4b0c043020 con 0x7f4b240ba190 2024-05-08T05:41:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.924+0000 7f4b1affd640 1 --2- 172.21.15.45:0/455125603 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b04076300 0x7f4b040787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:50.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.924+0000 7f4b1affd640 1 -- 172.21.15.45:0/455125603 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4b0c0b7a60 con 0x7f4b240ba190 2024-05-08T05:41:50.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.924+0000 7f4b2259c640 1 --2- 172.21.15.45:0/455125603 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b04076300 0x7f4b040787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:50.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.925+0000 7f4b2259c640 1 --2- 172.21.15.45:0/455125603 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b04076300 0x7f4b040787c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4b14001390 tx=0x7f4b14006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:50.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:50.928+0000 7f4b1affd640 1 -- 172.21.15.45:0/455125603 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4b240b5bd0 con 0x7f4b240ba190 2024-05-08T05:41:51.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.116+0000 7f4b28fe4640 1 -- 172.21.15.45:0/455125603 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b240693f0 con 0x7f4b04076300 2024-05-08T05:41:51.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.125+0000 7f4b1affd640 1 -- 172.21.15.45:0/455125603 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332519 (secure 0 0 0) 0x7f4b240693f0 con 0x7f4b04076300 2024-05-08T05:41:51.132 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:51.133 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:41:51.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.128+0000 7f4b28fe4640 1 -- 172.21.15.45:0/455125603 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b04076300 msgr2=0x7f4b040787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:51.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.128+0000 7f4b28fe4640 1 --2- 172.21.15.45:0/455125603 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b04076300 0x7f4b040787c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4b14001390 tx=0x7f4b14006040 comp rx=0 tx=0).stop 2024-05-08T05:41:51.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.128+0000 7f4b28fe4640 1 -- 172.21.15.45:0/455125603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b240ba190 msgr2=0x7f4b240ba610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:51.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.128+0000 7f4b28fe4640 1 --2- 172.21.15.45:0/455125603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b240ba190 0x7f4b240ba610 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c034040 tx=0x7f4b0c004730 comp rx=0 tx=0).stop 2024-05-08T05:41:51.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.128+0000 7f4b28fe4640 1 -- 172.21.15.45:0/455125603 shutdown_connections 2024-05-08T05:41:51.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.128+0000 7f4b28fe4640 1 --2- 172.21.15.45:0/455125603 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b04076300 0x7f4b040787c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.128+0000 7f4b28fe4640 1 --2- 172.21.15.45:0/455125603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b240b5210 0x7f4b240b5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.128+0000 7f4b28fe4640 1 --2- 172.21.15.45:0/455125603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b240ba190 0x7f4b240ba610 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.128+0000 7f4b28fe4640 1 --2- 172.21.15.45:0/455125603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b10003fd0 0x7f4b24155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.128+0000 7f4b28fe4640 1 -- 172.21.15.45:0/455125603 >> 172.21.15.45:0/455125603 conn(0x7f4b2408eac0 msgr2=0x7f4b2415e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:51.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.128+0000 7f4b28fe4640 1 -- 172.21.15.45:0/455125603 shutdown_connections 2024-05-08T05:41:51.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.128+0000 7f4b28fe4640 1 -- 172.21.15.45:0/455125603 wait complete. 2024-05-08T05:41:51.141 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-05-08T05:41:50.838165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236000,"kb_used_data":3176,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738720,"statfs":{"total":773094113280,"available":772852449280,"internally_reserved":0,"allocated":3252224,"data_stored":1985344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003128"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575491+0000","last_change":"2024-05-08T05:41:31.127376+0000","last_active":"2024-05-08T05:41:37.575491+0000","last_peered":"2024-05-08T05:41:37.575491+0000","last_clean":"2024-05-08T05:41:37.575491+0000","last_became_active":"2024-05-08T05:41:31.127083+0000","last_became_peered":"2024-05-08T05:41:31.127083+0000","last_unstale":"2024-05-08T05:41:37.575491+0000","last_undegraded":"2024-05-08T05:41:37.575491+0000","last_fullsized":"2024-05-08T05:41:37.575491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:53.557101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576109+0000","last_change":"2024-05-08T05:41:32.133658+0000","last_active":"2024-05-08T05:41:37.576109+0000","last_peered":"2024-05-08T05:41:37.576109+0000","last_clean":"2024-05-08T05:41:37.576109+0000","last_became_active":"2024-05-08T05:41:32.133241+0000","last_became_peered":"2024-05-08T05:41:32.133241+0000","last_unstale":"2024-05-08T05:41:37.576109+0000","last_undegraded":"2024-05-08T05:41:37.576109+0000","last_fullsized":"2024-05-08T05:41:37.576109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:07:26.071605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576413+0000","last_change":"2024-05-08T05:41:36.385551+0000","last_active":"2024-05-08T05:41:37.576413+0000","last_peered":"2024-05-08T05:41:37.576413+0000","last_clean":"2024-05-08T05:41:37.576413+0000","last_became_active":"2024-05-08T05:41:36.385135+0000","last_became_peered":"2024-05-08T05:41:36.385135+0000","last_unstale":"2024-05-08T05:41:37.576413+0000","last_undegraded":"2024-05-08T05:41:37.576413+0000","last_fullsized":"2024-05-08T05:41:37.576413+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:39:51.537856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576344+0000","last_change":"2024-05-08T05:41:31.125574+0000","last_active":"2024-05-08T05:41:37.576344+0000","last_peered":"2024-05-08T05:41:37.576344+0000","last_clean":"2024-05-08T05:41:37.576344+0000","last_became_active":"2024-05-08T05:41:31.125302+0000","last_became_peered":"2024-05-08T05:41:31.125302+0000","last_unstale":"2024-05-08T05:41:37.576344+0000","last_undegraded":"2024-05-08T05:41:37.576344+0000","last_fullsized":"2024-05-08T05:41:37.576344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:39.672951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575165+0000","last_change":"2024-05-08T05:41:32.140528+0000","last_active":"2024-05-08T05:41:37.575165+0000","last_peered":"2024-05-08T05:41:37.575165+0000","last_clean":"2024-05-08T05:41:37.575165+0000","last_became_active":"2024-05-08T05:41:32.140227+0000","last_became_peered":"2024-05-08T05:41:32.140227+0000","last_unstale":"2024-05-08T05:41:37.575165+0000","last_undegraded":"2024-05-08T05:41:37.575165+0000","last_fullsized":"2024-05-08T05:41:37.575165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:30:07.184101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153167+0000","last_change":"2024-05-08T05:41:36.149551+0000","last_active":"2024-05-08T05:41:37.153167+0000","last_peered":"2024-05-08T05:41:37.153167+0000","last_clean":"2024-05-08T05:41:37.153167+0000","last_became_active":"2024-05-08T05:41:36.149115+0000","last_became_peered":"2024-05-08T05:41:36.149115+0000","last_unstale":"2024-05-08T05:41:37.153167+0000","last_undegraded":"2024-05-08T05:41:37.153167+0000","last_fullsized":"2024-05-08T05:41:37.153167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:57.593969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577133+0000","last_change":"2024-05-08T05:41:31.128105+0000","last_active":"2024-05-08T05:41:37.577133+0000","last_peered":"2024-05-08T05:41:37.577133+0000","last_clean":"2024-05-08T05:41:37.577133+0000","last_became_active":"2024-05-08T05:41:31.127782+0000","last_became_peered":"2024-05-08T05:41:31.127782+0000","last_unstale":"2024-05-08T05:41:37.577133+0000","last_undegraded":"2024-05-08T05:41:37.577133+0000","last_fullsized":"2024-05-08T05:41:37.577133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:46.418806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576628+0000","last_change":"2024-05-08T05:41:32.140667+0000","last_active":"2024-05-08T05:41:37.576628+0000","last_peered":"2024-05-08T05:41:37.576628+0000","last_clean":"2024-05-08T05:41:37.576628+0000","last_became_active":"2024-05-08T05:41:32.140365+0000","last_became_peered":"2024-05-08T05:41:32.140365+0000","last_unstale":"2024-05-08T05:41:37.576628+0000","last_undegraded":"2024-05-08T05:41:37.576628+0000","last_fullsized":"2024-05-08T05:41:37.576628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:02.517245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153785+0000","last_change":"2024-05-08T05:41:36.157609+0000","last_active":"2024-05-08T05:41:37.153785+0000","last_peered":"2024-05-08T05:41:37.153785+0000","last_clean":"2024-05-08T05:41:37.153785+0000","last_became_active":"2024-05-08T05:41:36.157214+0000","last_became_peered":"2024-05-08T05:41:36.157214+0000","last_unstale":"2024-05-08T05:41:37.153785+0000","last_undegraded":"2024-05-08T05:41:37.153785+0000","last_fullsized":"2024-05-08T05:41:37.153785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:46.769826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576228+0000","last_change":"2024-05-08T05:41:31.131582+0000","last_active":"2024-05-08T05:41:37.576228+0000","last_peered":"2024-05-08T05:41:37.576228+0000","last_clean":"2024-05-08T05:41:37.576228+0000","last_became_active":"2024-05-08T05:41:31.131302+0000","last_became_peered":"2024-05-08T05:41:31.131302+0000","last_unstale":"2024-05-08T05:41:37.576228+0000","last_undegraded":"2024-05-08T05:41:37.576228+0000","last_fullsized":"2024-05-08T05:41:37.576228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:07.648895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577648+0000","last_change":"2024-05-08T05:41:32.136792+0000","last_active":"2024-05-08T05:41:37.577648+0000","last_peered":"2024-05-08T05:41:37.577648+0000","last_clean":"2024-05-08T05:41:37.577648+0000","last_became_active":"2024-05-08T05:41:32.136285+0000","last_became_peered":"2024-05-08T05:41:32.136285+0000","last_unstale":"2024-05-08T05:41:37.577648+0000","last_undegraded":"2024-05-08T05:41:37.577648+0000","last_fullsized":"2024-05-08T05:41:37.577648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:53.826185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153415+0000","last_change":"2024-05-08T05:41:36.151839+0000","last_active":"2024-05-08T05:41:37.153415+0000","last_peered":"2024-05-08T05:41:37.153415+0000","last_clean":"2024-05-08T05:41:37.153415+0000","last_became_active":"2024-05-08T05:41:36.151387+0000","last_became_peered":"2024-05-08T05:41:36.151387+0000","last_unstale":"2024-05-08T05:41:37.153415+0000","last_undegraded":"2024-05-08T05:41:37.153415+0000","last_fullsized":"2024-05-08T05:41:37.153415+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:01:36.105428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576169+0000","last_change":"2024-05-08T05:41:32.136438+0000","last_active":"2024-05-08T05:41:37.576169+0000","last_peered":"2024-05-08T05:41:37.576169+0000","last_clean":"2024-05-08T05:41:37.576169+0000","last_became_active":"2024-05-08T05:41:32.136145+0000","last_became_peered":"2024-05-08T05:41:32.136145+0000","last_unstale":"2024-05-08T05:41:37.576169+0000","last_undegraded":"2024-05-08T05:41:37.576169+0000","last_fullsized":"2024-05-08T05:41:37.576169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:47.493805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576520+0000","last_change":"2024-05-08T05:41:31.124904+0000","last_active":"2024-05-08T05:41:37.576520+0000","last_peered":"2024-05-08T05:41:37.576520+0000","last_clean":"2024-05-08T05:41:37.576520+0000","last_became_active":"2024-05-08T05:41:31.124621+0000","last_became_peered":"2024-05-08T05:41:31.124621+0000","last_unstale":"2024-05-08T05:41:37.576520+0000","last_undegraded":"2024-05-08T05:41:37.576520+0000","last_fullsized":"2024-05-08T05:41:37.576520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:41.591349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154482+0000","last_change":"2024-05-08T05:41:36.154069+0000","last_active":"2024-05-08T05:41:37.154482+0000","last_peered":"2024-05-08T05:41:37.154482+0000","last_clean":"2024-05-08T05:41:37.154482+0000","last_became_active":"2024-05-08T05:41:36.153717+0000","last_became_peered":"2024-05-08T05:41:36.153717+0000","last_unstale":"2024-05-08T05:41:37.154482+0000","last_undegraded":"2024-05-08T05:41:37.154482+0000","last_fullsized":"2024-05-08T05:41:37.154482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:57.856276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577086+0000","last_change":"2024-05-08T05:41:32.137987+0000","last_active":"2024-05-08T05:41:37.577086+0000","last_peered":"2024-05-08T05:41:37.577086+0000","last_clean":"2024-05-08T05:41:37.577086+0000","last_became_active":"2024-05-08T05:41:32.137711+0000","last_became_peered":"2024-05-08T05:41:32.137711+0000","last_unstale":"2024-05-08T05:41:37.577086+0000","last_undegraded":"2024-05-08T05:41:37.577086+0000","last_fullsized":"2024-05-08T05:41:37.577086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:01:32.075337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576626+0000","last_change":"2024-05-08T05:41:31.121790+0000","last_active":"2024-05-08T05:41:37.576626+0000","last_peered":"2024-05-08T05:41:37.576626+0000","last_clean":"2024-05-08T05:41:37.576626+0000","last_became_active":"2024-05-08T05:41:31.121476+0000","last_became_peered":"2024-05-08T05:41:31.121476+0000","last_unstale":"2024-05-08T05:41:37.576626+0000","last_undegraded":"2024-05-08T05:41:37.576626+0000","last_fullsized":"2024-05-08T05:41:37.576626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:11:56.288788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154013+0000","last_change":"2024-05-08T05:41:36.157615+0000","last_active":"2024-05-08T05:41:37.154013+0000","last_peered":"2024-05-08T05:41:37.154013+0000","last_clean":"2024-05-08T05:41:37.154013+0000","last_became_active":"2024-05-08T05:41:36.157156+0000","last_became_peered":"2024-05-08T05:41:36.157156+0000","last_unstale":"2024-05-08T05:41:37.154013+0000","last_undegraded":"2024-05-08T05:41:37.154013+0000","last_fullsized":"2024-05-08T05:41:37.154013+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:46.115781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576120+0000","last_change":"2024-05-08T05:41:32.119867+0000","last_active":"2024-05-08T05:41:37.576120+0000","last_peered":"2024-05-08T05:41:37.576120+0000","last_clean":"2024-05-08T05:41:37.576120+0000","last_became_active":"2024-05-08T05:41:32.119186+0000","last_became_peered":"2024-05-08T05:41:32.119186+0000","last_unstale":"2024-05-08T05:41:37.576120+0000","last_undegraded":"2024-05-08T05:41:37.576120+0000","last_fullsized":"2024-05-08T05:41:37.576120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:47.493805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575164+0000","last_change":"2024-05-08T05:41:31.128486+0000","last_active":"2024-05-08T05:41:37.575164+0000","last_peered":"2024-05-08T05:41:37.575164+0000","last_clean":"2024-05-08T05:41:37.575164+0000","last_became_active":"2024-05-08T05:41:31.128207+0000","last_became_peered":"2024-05-08T05:41:31.128207+0000","last_unstale":"2024-05-08T05:41:37.575164+0000","last_undegraded":"2024-05-08T05:41:37.575164+0000","last_fullsized":"2024-05-08T05:41:37.575164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:46.418806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576817+0000","last_change":"2024-05-08T05:41:36.381823+0000","last_active":"2024-05-08T05:41:37.576817+0000","last_peered":"2024-05-08T05:41:37.576817+0000","last_clean":"2024-05-08T05:41:37.576817+0000","last_became_active":"2024-05-08T05:41:36.381421+0000","last_became_peered":"2024-05-08T05:41:36.381421+0000","last_unstale":"2024-05-08T05:41:37.576817+0000","last_undegraded":"2024-05-08T05:41:37.576817+0000","last_fullsized":"2024-05-08T05:41:37.576817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:21.321405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153397+0000","last_change":"2024-05-08T05:41:32.117529+0000","last_active":"2024-05-08T05:41:37.153397+0000","last_peered":"2024-05-08T05:41:37.153397+0000","last_clean":"2024-05-08T05:41:37.153397+0000","last_became_active":"2024-05-08T05:41:32.117205+0000","last_became_peered":"2024-05-08T05:41:32.117205+0000","last_unstale":"2024-05-08T05:41:37.153397+0000","last_undegraded":"2024-05-08T05:41:37.153397+0000","last_fullsized":"2024-05-08T05:41:37.153397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:08.654224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575429+0000","last_change":"2024-05-08T05:41:31.126184+0000","last_active":"2024-05-08T05:41:37.575429+0000","last_peered":"2024-05-08T05:41:37.575429+0000","last_clean":"2024-05-08T05:41:37.575429+0000","last_became_active":"2024-05-08T05:41:31.125860+0000","last_became_peered":"2024-05-08T05:41:31.125860+0000","last_unstale":"2024-05-08T05:41:37.575429+0000","last_undegraded":"2024-05-08T05:41:37.575429+0000","last_fullsized":"2024-05-08T05:41:37.575429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:13.465598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154197+0000","last_change":"2024-05-08T05:41:36.151020+0000","last_active":"2024-05-08T05:41:37.154197+0000","last_peered":"2024-05-08T05:41:37.154197+0000","last_clean":"2024-05-08T05:41:37.154197+0000","last_became_active":"2024-05-08T05:41:36.150320+0000","last_became_peered":"2024-05-08T05:41:36.150320+0000","last_unstale":"2024-05-08T05:41:37.154197+0000","last_undegraded":"2024-05-08T05:41:37.154197+0000","last_fullsized":"2024-05-08T05:41:37.154197+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:06.547337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576056+0000","last_change":"2024-05-08T05:41:32.121088+0000","last_active":"2024-05-08T05:41:37.576056+0000","last_peered":"2024-05-08T05:41:37.576056+0000","last_clean":"2024-05-08T05:41:37.576056+0000","last_became_active":"2024-05-08T05:41:32.119585+0000","last_became_peered":"2024-05-08T05:41:32.119585+0000","last_unstale":"2024-05-08T05:41:37.576056+0000","last_undegraded":"2024-05-08T05:41:37.576056+0000","last_fullsized":"2024-05-08T05:41:37.576056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:18:52.796967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577379+0000","last_change":"2024-05-08T05:41:31.129202+0000","last_active":"2024-05-08T05:41:37.577379+0000","last_peered":"2024-05-08T05:41:37.577379+0000","last_clean":"2024-05-08T05:41:37.577379+0000","last_became_active":"2024-05-08T05:41:31.128911+0000","last_became_peered":"2024-05-08T05:41:31.128911+0000","last_unstale":"2024-05-08T05:41:37.577379+0000","last_undegraded":"2024-05-08T05:41:37.577379+0000","last_fullsized":"2024-05-08T05:41:37.577379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:13.465598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153283+0000","last_change":"2024-05-08T05:41:36.384181+0000","last_active":"2024-05-08T05:41:37.153283+0000","last_peered":"2024-05-08T05:41:37.153283+0000","last_clean":"2024-05-08T05:41:37.153283+0000","last_became_active":"2024-05-08T05:41:36.383800+0000","last_became_peered":"2024-05-08T05:41:36.383800+0000","last_unstale":"2024-05-08T05:41:37.153283+0000","last_undegraded":"2024-05-08T05:41:37.153283+0000","last_fullsized":"2024-05-08T05:41:37.153283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:33.503988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576885+0000","last_change":"2024-05-08T05:41:32.121597+0000","last_active":"2024-05-08T05:41:37.576885+0000","last_peered":"2024-05-08T05:41:37.576885+0000","last_clean":"2024-05-08T05:41:37.576885+0000","last_became_active":"2024-05-08T05:41:32.121321+0000","last_became_peered":"2024-05-08T05:41:32.121321+0000","last_unstale":"2024-05-08T05:41:37.576885+0000","last_undegraded":"2024-05-08T05:41:37.576885+0000","last_fullsized":"2024-05-08T05:41:37.576885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:42.739734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575444+0000","last_change":"2024-05-08T05:41:31.126359+0000","last_active":"2024-05-08T05:41:37.575444+0000","last_peered":"2024-05-08T05:41:37.575444+0000","last_clean":"2024-05-08T05:41:37.575444+0000","last_became_active":"2024-05-08T05:41:31.126069+0000","last_became_peered":"2024-05-08T05:41:31.126069+0000","last_unstale":"2024-05-08T05:41:37.575444+0000","last_undegraded":"2024-05-08T05:41:37.575444+0000","last_fullsized":"2024-05-08T05:41:37.575444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:24.722618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575700+0000","last_change":"2024-05-08T05:41:36.381909+0000","last_active":"2024-05-08T05:41:37.575700+0000","last_peered":"2024-05-08T05:41:37.575700+0000","last_clean":"2024-05-08T05:41:37.575700+0000","last_became_active":"2024-05-08T05:41:36.381251+0000","last_became_peered":"2024-05-08T05:41:36.381251+0000","last_unstale":"2024-05-08T05:41:37.575700+0000","last_undegraded":"2024-05-08T05:41:37.575700+0000","last_fullsized":"2024-05-08T05:41:37.575700+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:06:13.980953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576829+0000","last_change":"2024-05-08T05:41:32.133545+0000","last_active":"2024-05-08T05:41:37.576829+0000","last_peered":"2024-05-08T05:41:37.576829+0000","last_clean":"2024-05-08T05:41:37.576829+0000","last_became_active":"2024-05-08T05:41:32.132677+0000","last_became_peered":"2024-05-08T05:41:32.132677+0000","last_unstale":"2024-05-08T05:41:37.576829+0000","last_undegraded":"2024-05-08T05:41:37.576829+0000","last_fullsized":"2024-05-08T05:41:37.576829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:42.041915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577263+0000","last_change":"2024-05-08T05:41:31.127131+0000","last_active":"2024-05-08T05:41:37.577263+0000","last_peered":"2024-05-08T05:41:37.577263+0000","last_clean":"2024-05-08T05:41:37.577263+0000","last_became_active":"2024-05-08T05:41:31.126789+0000","last_became_peered":"2024-05-08T05:41:31.126789+0000","last_unstale":"2024-05-08T05:41:37.577263+0000","last_undegraded":"2024-05-08T05:41:37.577263+0000","last_fullsized":"2024-05-08T05:41:37.577263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:19:12.221462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575923+0000","last_change":"2024-05-08T05:41:36.381721+0000","last_active":"2024-05-08T05:41:37.575923+0000","last_peered":"2024-05-08T05:41:37.575923+0000","last_clean":"2024-05-08T05:41:37.575923+0000","last_became_active":"2024-05-08T05:41:36.381329+0000","last_became_peered":"2024-05-08T05:41:36.381329+0000","last_unstale":"2024-05-08T05:41:37.575923+0000","last_undegraded":"2024-05-08T05:41:37.575923+0000","last_fullsized":"2024-05-08T05:41:37.575923+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:17.202706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577782+0000","last_change":"2024-05-08T05:41:32.136558+0000","last_active":"2024-05-08T05:41:37.577782+0000","last_peered":"2024-05-08T05:41:37.577782+0000","last_clean":"2024-05-08T05:41:37.577782+0000","last_became_active":"2024-05-08T05:41:32.133354+0000","last_became_peered":"2024-05-08T05:41:32.133354+0000","last_unstale":"2024-05-08T05:41:37.577782+0000","last_undegraded":"2024-05-08T05:41:37.577782+0000","last_fullsized":"2024-05-08T05:41:37.577782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:13.172615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576762+0000","last_change":"2024-05-08T05:41:31.125582+0000","last_active":"2024-05-08T05:41:37.576762+0000","last_peered":"2024-05-08T05:41:37.576762+0000","last_clean":"2024-05-08T05:41:37.576762+0000","last_became_active":"2024-05-08T05:41:31.125252+0000","last_became_peered":"2024-05-08T05:41:31.125252+0000","last_unstale":"2024-05-08T05:41:37.576762+0000","last_undegraded":"2024-05-08T05:41:37.576762+0000","last_fullsized":"2024-05-08T05:41:37.576762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:38.174964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576435+0000","last_change":"2024-05-08T05:41:36.154344+0000","last_active":"2024-05-08T05:41:37.576435+0000","last_peered":"2024-05-08T05:41:37.576435+0000","last_clean":"2024-05-08T05:41:37.576435+0000","last_became_active":"2024-05-08T05:41:36.153930+0000","last_became_peered":"2024-05-08T05:41:36.153930+0000","last_unstale":"2024-05-08T05:41:37.576435+0000","last_undegraded":"2024-05-08T05:41:37.576435+0000","last_fullsized":"2024-05-08T05:41:37.576435+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:20.531197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154018+0000","last_change":"2024-05-08T05:41:32.136504+0000","last_active":"2024-05-08T05:41:37.154018+0000","last_peered":"2024-05-08T05:41:37.154018+0000","last_clean":"2024-05-08T05:41:37.154018+0000","last_became_active":"2024-05-08T05:41:32.135721+0000","last_became_peered":"2024-05-08T05:41:32.135721+0000","last_unstale":"2024-05-08T05:41:37.154018+0000","last_undegraded":"2024-05-08T05:41:37.154018+0000","last_fullsized":"2024-05-08T05:41:37.154018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:42.596678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576189+0000","last_change":"2024-05-08T05:41:31.131574+0000","last_active":"2024-05-08T05:41:37.576189+0000","last_peered":"2024-05-08T05:41:37.576189+0000","last_clean":"2024-05-08T05:41:37.576189+0000","last_became_active":"2024-05-08T05:41:31.131158+0000","last_became_peered":"2024-05-08T05:41:31.131158+0000","last_unstale":"2024-05-08T05:41:37.576189+0000","last_undegraded":"2024-05-08T05:41:37.576189+0000","last_fullsized":"2024-05-08T05:41:37.576189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:13.465598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576707+0000","last_change":"2024-05-08T05:41:36.159573+0000","last_active":"2024-05-08T05:41:37.576707+0000","last_peered":"2024-05-08T05:41:37.576707+0000","last_clean":"2024-05-08T05:41:37.576707+0000","last_became_active":"2024-05-08T05:41:36.159150+0000","last_became_peered":"2024-05-08T05:41:36.159150+0000","last_unstale":"2024-05-08T05:41:37.576707+0000","last_undegraded":"2024-05-08T05:41:37.576707+0000","last_fullsized":"2024-05-08T05:41:37.576707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:17.202706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576057+0000","last_change":"2024-05-08T05:41:32.121923+0000","last_active":"2024-05-08T05:41:37.576057+0000","last_peered":"2024-05-08T05:41:37.576057+0000","last_clean":"2024-05-08T05:41:37.576057+0000","last_became_active":"2024-05-08T05:41:32.121213+0000","last_became_peered":"2024-05-08T05:41:32.121213+0000","last_unstale":"2024-05-08T05:41:37.576057+0000","last_undegraded":"2024-05-08T05:41:37.576057+0000","last_fullsized":"2024-05-08T05:41:37.576057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:03:20.127059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.147846+0000","last_change":"2024-05-08T05:41:31.122162+0000","last_active":"2024-05-08T05:41:37.147846+0000","last_peered":"2024-05-08T05:41:37.147846+0000","last_clean":"2024-05-08T05:41:37.147846+0000","last_became_active":"2024-05-08T05:41:31.121865+0000","last_became_peered":"2024-05-08T05:41:31.121865+0000","last_unstale":"2024-05-08T05:41:37.147846+0000","last_undegraded":"2024-05-08T05:41:37.147846+0000","last_fullsized":"2024-05-08T05:41:37.147846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:24.722618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576949+0000","last_change":"2024-05-08T05:41:36.155633+0000","last_active":"2024-05-08T05:41:37.576949+0000","last_peered":"2024-05-08T05:41:37.576949+0000","last_clean":"2024-05-08T05:41:37.576949+0000","last_became_active":"2024-05-08T05:41:36.155125+0000","last_became_peered":"2024-05-08T05:41:36.155125+0000","last_unstale":"2024-05-08T05:41:37.576949+0000","last_undegraded":"2024-05-08T05:41:37.576949+0000","last_fullsized":"2024-05-08T05:41:37.576949+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:50:36.282661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.574963+0000","last_change":"2024-05-08T05:41:32.119953+0000","last_active":"2024-05-08T05:41:37.574963+0000","last_peered":"2024-05-08T05:41:37.574963+0000","last_clean":"2024-05-08T05:41:37.574963+0000","last_became_active":"2024-05-08T05:41:32.119532+0000","last_became_peered":"2024-05-08T05:41:32.119532+0000","last_unstale":"2024-05-08T05:41:37.574963+0000","last_undegraded":"2024-05-08T05:41:37.574963+0000","last_fullsized":"2024-05-08T05:41:37.574963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:16:33.854897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575557+0000","last_change":"2024-05-08T05:41:31.125004+0000","last_active":"2024-05-08T05:41:37.575557+0000","last_peered":"2024-05-08T05:41:37.575557+0000","last_clean":"2024-05-08T05:41:37.575557+0000","last_became_active":"2024-05-08T05:41:31.124678+0000","last_became_peered":"2024-05-08T05:41:31.124678+0000","last_unstale":"2024-05-08T05:41:37.575557+0000","last_undegraded":"2024-05-08T05:41:37.575557+0000","last_fullsized":"2024-05-08T05:41:37.575557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:38.174964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575882+0000","last_change":"2024-05-08T05:41:36.385360+0000","last_active":"2024-05-08T05:41:37.575882+0000","last_peered":"2024-05-08T05:41:37.575882+0000","last_clean":"2024-05-08T05:41:37.575882+0000","last_became_active":"2024-05-08T05:41:36.384992+0000","last_became_peered":"2024-05-08T05:41:36.384992+0000","last_unstale":"2024-05-08T05:41:37.575882+0000","last_undegraded":"2024-05-08T05:41:37.575882+0000","last_fullsized":"2024-05-08T05:41:37.575882+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:20.531197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575341+0000","last_change":"2024-05-08T05:41:32.138842+0000","last_active":"2024-05-08T05:41:37.575341+0000","last_peered":"2024-05-08T05:41:37.575341+0000","last_clean":"2024-05-08T05:41:37.575341+0000","last_became_active":"2024-05-08T05:41:32.138583+0000","last_became_peered":"2024-05-08T05:41:32.138583+0000","last_unstale":"2024-05-08T05:41:37.575341+0000","last_undegraded":"2024-05-08T05:41:37.575341+0000","last_fullsized":"2024-05-08T05:41:37.575341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:53.563878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576353+0000","last_change":"2024-05-08T05:41:31.132616+0000","last_active":"2024-05-08T05:41:37.576353+0000","last_peered":"2024-05-08T05:41:37.576353+0000","last_clean":"2024-05-08T05:41:37.576353+0000","last_became_active":"2024-05-08T05:41:31.131462+0000","last_became_peered":"2024-05-08T05:41:31.131462+0000","last_unstale":"2024-05-08T05:41:37.576353+0000","last_undegraded":"2024-05-08T05:41:37.576353+0000","last_fullsized":"2024-05-08T05:41:37.576353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:24.722618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":118,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575672+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:41:37.575672+0000","last_peered":"2024-05-08T05:41:37.575672+0000","last_clean":"2024-05-08T05:41:37.575672+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:41:37.575672+0000","last_undegraded":"2024-05-08T05:41:37.575672+0000","last_fullsized":"2024-05-08T05:41:37.575672+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576785+0000","last_change":"2024-05-08T05:41:36.381947+0000","last_active":"2024-05-08T05:41:37.576785+0000","last_peered":"2024-05-08T05:41:37.576785+0000","last_clean":"2024-05-08T05:41:37.576785+0000","last_became_active":"2024-05-08T05:41:36.381491+0000","last_became_peered":"2024-05-08T05:41:36.381491+0000","last_unstale":"2024-05-08T05:41:37.576785+0000","last_undegraded":"2024-05-08T05:41:37.576785+0000","last_fullsized":"2024-05-08T05:41:37.576785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:57.856276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576983+0000","last_change":"2024-05-08T05:41:32.134397+0000","last_active":"2024-05-08T05:41:37.576983+0000","last_peered":"2024-05-08T05:41:37.576983+0000","last_clean":"2024-05-08T05:41:37.576983+0000","last_became_active":"2024-05-08T05:41:32.134098+0000","last_became_peered":"2024-05-08T05:41:32.134098+0000","last_unstale":"2024-05-08T05:41:37.576983+0000","last_undegraded":"2024-05-08T05:41:37.576983+0000","last_fullsized":"2024-05-08T05:41:37.576983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:16.501105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.148234+0000","last_change":"2024-05-08T05:41:31.133531+0000","last_active":"2024-05-08T05:41:37.148234+0000","last_peered":"2024-05-08T05:41:37.148234+0000","last_clean":"2024-05-08T05:41:37.148234+0000","last_became_active":"2024-05-08T05:41:31.133208+0000","last_became_peered":"2024-05-08T05:41:31.133208+0000","last_unstale":"2024-05-08T05:41:37.148234+0000","last_undegraded":"2024-05-08T05:41:37.148234+0000","last_fullsized":"2024-05-08T05:41:37.148234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:22.751500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576104+0000","last_change":"2024-05-08T05:41:36.381903+0000","last_active":"2024-05-08T05:41:37.576104+0000","last_peered":"2024-05-08T05:41:37.576104+0000","last_clean":"2024-05-08T05:41:37.576104+0000","last_became_active":"2024-05-08T05:41:36.381397+0000","last_became_peered":"2024-05-08T05:41:36.381397+0000","last_unstale":"2024-05-08T05:41:37.576104+0000","last_undegraded":"2024-05-08T05:41:37.576104+0000","last_fullsized":"2024-05-08T05:41:37.576104+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:06.547337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576725+0000","last_change":"2024-05-08T05:41:32.128554+0000","last_active":"2024-05-08T05:41:37.576725+0000","last_peered":"2024-05-08T05:41:37.576725+0000","last_clean":"2024-05-08T05:41:37.576725+0000","last_became_active":"2024-05-08T05:41:32.128280+0000","last_became_peered":"2024-05-08T05:41:32.128280+0000","last_unstale":"2024-05-08T05:41:37.576725+0000","last_undegraded":"2024-05-08T05:41:37.576725+0000","last_fullsized":"2024-05-08T05:41:37.576725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:42.085689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576546+0000","last_change":"2024-05-08T05:41:31.122071+0000","last_active":"2024-05-08T05:41:37.576546+0000","last_peered":"2024-05-08T05:41:37.576546+0000","last_clean":"2024-05-08T05:41:37.576546+0000","last_became_active":"2024-05-08T05:41:31.121137+0000","last_became_peered":"2024-05-08T05:41:31.121137+0000","last_unstale":"2024-05-08T05:41:37.576546+0000","last_undegraded":"2024-05-08T05:41:37.576546+0000","last_fullsized":"2024-05-08T05:41:37.576546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:13.465598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576176+0000","last_change":"2024-05-08T05:41:36.160917+0000","last_active":"2024-05-08T05:41:37.576176+0000","last_peered":"2024-05-08T05:41:37.576176+0000","last_clean":"2024-05-08T05:41:37.576176+0000","last_became_active":"2024-05-08T05:41:36.160457+0000","last_became_peered":"2024-05-08T05:41:36.160457+0000","last_unstale":"2024-05-08T05:41:37.576176+0000","last_undegraded":"2024-05-08T05:41:37.576176+0000","last_fullsized":"2024-05-08T05:41:37.576176+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:21.321405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577384+0000","last_change":"2024-05-08T05:41:32.131012+0000","last_active":"2024-05-08T05:41:37.577384+0000","last_peered":"2024-05-08T05:41:37.577384+0000","last_clean":"2024-05-08T05:41:37.577384+0000","last_became_active":"2024-05-08T05:41:32.130682+0000","last_became_peered":"2024-05-08T05:41:32.130682+0000","last_unstale":"2024-05-08T05:41:37.577384+0000","last_undegraded":"2024-05-08T05:41:37.577384+0000","last_fullsized":"2024-05-08T05:41:37.577384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:06:09.950861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.148282+0000","last_change":"2024-05-08T05:41:31.122329+0000","last_active":"2024-05-08T05:41:37.148282+0000","last_peered":"2024-05-08T05:41:37.148282+0000","last_clean":"2024-05-08T05:41:37.148282+0000","last_became_active":"2024-05-08T05:41:31.122021+0000","last_became_peered":"2024-05-08T05:41:31.122021+0000","last_unstale":"2024-05-08T05:41:37.148282+0000","last_undegraded":"2024-05-08T05:41:37.148282+0000","last_fullsized":"2024-05-08T05:41:37.148282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:11:56.288788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575492+0000","last_change":"2024-05-08T05:41:36.382122+0000","last_active":"2024-05-08T05:41:37.575492+0000","last_peered":"2024-05-08T05:41:37.575492+0000","last_clean":"2024-05-08T05:41:37.575492+0000","last_became_active":"2024-05-08T05:41:36.381385+0000","last_became_peered":"2024-05-08T05:41:36.381385+0000","last_unstale":"2024-05-08T05:41:37.575492+0000","last_undegraded":"2024-05-08T05:41:37.575492+0000","last_fullsized":"2024-05-08T05:41:37.575492+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:02.479497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576083+0000","last_change":"2024-05-08T05:41:32.131255+0000","last_active":"2024-05-08T05:41:37.576083+0000","last_peered":"2024-05-08T05:41:37.576083+0000","last_clean":"2024-05-08T05:41:37.576083+0000","last_became_active":"2024-05-08T05:41:32.130252+0000","last_became_peered":"2024-05-08T05:41:32.130252+0000","last_unstale":"2024-05-08T05:41:37.576083+0000","last_undegraded":"2024-05-08T05:41:37.576083+0000","last_fullsized":"2024-05-08T05:41:37.576083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:17.291314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154349+0000","last_change":"2024-05-08T05:41:31.125770+0000","last_active":"2024-05-08T05:41:37.154349+0000","last_peered":"2024-05-08T05:41:37.154349+0000","last_clean":"2024-05-08T05:41:37.154349+0000","last_became_active":"2024-05-08T05:41:31.125442+0000","last_became_peered":"2024-05-08T05:41:31.125442+0000","last_unstale":"2024-05-08T05:41:37.154349+0000","last_undegraded":"2024-05-08T05:41:37.154349+0000","last_fullsized":"2024-05-08T05:41:37.154349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:53.557101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153167+0000","last_change":"2024-05-08T05:41:36.151074+0000","last_active":"2024-05-08T05:41:37.153167+0000","last_peered":"2024-05-08T05:41:37.153167+0000","last_clean":"2024-05-08T05:41:37.153167+0000","last_became_active":"2024-05-08T05:41:36.150613+0000","last_became_peered":"2024-05-08T05:41:36.150613+0000","last_unstale":"2024-05-08T05:41:37.153167+0000","last_undegraded":"2024-05-08T05:41:37.153167+0000","last_fullsized":"2024-05-08T05:41:37.153167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:46.072007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576628+0000","last_change":"2024-05-08T05:41:32.132197+0000","last_active":"2024-05-08T05:41:37.576628+0000","last_peered":"2024-05-08T05:41:37.576628+0000","last_clean":"2024-05-08T05:41:37.576628+0000","last_became_active":"2024-05-08T05:41:32.131086+0000","last_became_peered":"2024-05-08T05:41:32.131086+0000","last_unstale":"2024-05-08T05:41:37.576628+0000","last_undegraded":"2024-05-08T05:41:37.576628+0000","last_fullsized":"2024-05-08T05:41:37.576628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:02.517245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.147676+0000","last_change":"2024-05-08T05:41:31.120099+0000","last_active":"2024-05-08T05:41:37.147676+0000","last_peered":"2024-05-08T05:41:37.147676+0000","last_clean":"2024-05-08T05:41:37.147676+0000","last_became_active":"2024-05-08T05:41:31.119750+0000","last_became_peered":"2024-05-08T05:41:31.119750+0000","last_unstale":"2024-05-08T05:41:37.147676+0000","last_undegraded":"2024-05-08T05:41:37.147676+0000","last_fullsized":"2024-05-08T05:41:37.147676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:49.171947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575735+0000","last_change":"2024-05-08T05:41:36.162020+0000","last_active":"2024-05-08T05:41:37.575735+0000","last_peered":"2024-05-08T05:41:37.575735+0000","last_clean":"2024-05-08T05:41:37.575735+0000","last_became_active":"2024-05-08T05:41:36.161613+0000","last_became_peered":"2024-05-08T05:41:36.161613+0000","last_unstale":"2024-05-08T05:41:37.575735+0000","last_undegraded":"2024-05-08T05:41:37.575735+0000","last_fullsized":"2024-05-08T05:41:37.575735+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:57.856276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575424+0000","last_change":"2024-05-08T05:41:32.137565+0000","last_active":"2024-05-08T05:41:37.575424+0000","last_peered":"2024-05-08T05:41:37.575424+0000","last_clean":"2024-05-08T05:41:37.575424+0000","last_became_active":"2024-05-08T05:41:32.137078+0000","last_became_peered":"2024-05-08T05:41:32.137078+0000","last_unstale":"2024-05-08T05:41:37.575424+0000","last_undegraded":"2024-05-08T05:41:37.575424+0000","last_fullsized":"2024-05-08T05:41:37.575424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:44.038894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576417+0000","last_change":"2024-05-08T05:41:31.127117+0000","last_active":"2024-05-08T05:41:37.576417+0000","last_peered":"2024-05-08T05:41:37.576417+0000","last_clean":"2024-05-08T05:41:37.576417+0000","last_became_active":"2024-05-08T05:41:31.126826+0000","last_became_peered":"2024-05-08T05:41:31.126826+0000","last_unstale":"2024-05-08T05:41:37.576417+0000","last_undegraded":"2024-05-08T05:41:37.576417+0000","last_fullsized":"2024-05-08T05:41:37.576417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:22.751500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576723+0000","last_change":"2024-05-08T05:41:36.383650+0000","last_active":"2024-05-08T05:41:37.576723+0000","last_peered":"2024-05-08T05:41:37.576723+0000","last_clean":"2024-05-08T05:41:37.576723+0000","last_became_active":"2024-05-08T05:41:36.382759+0000","last_became_peered":"2024-05-08T05:41:36.382759+0000","last_unstale":"2024-05-08T05:41:37.576723+0000","last_undegraded":"2024-05-08T05:41:37.576723+0000","last_fullsized":"2024-05-08T05:41:37.576723+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:43:10.562469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576336+0000","last_change":"2024-05-08T05:41:32.130461+0000","last_active":"2024-05-08T05:41:37.576336+0000","last_peered":"2024-05-08T05:41:37.576336+0000","last_clean":"2024-05-08T05:41:37.576336+0000","last_became_active":"2024-05-08T05:41:32.130160+0000","last_became_peered":"2024-05-08T05:41:32.130160+0000","last_unstale":"2024-05-08T05:41:37.576336+0000","last_undegraded":"2024-05-08T05:41:37.576336+0000","last_fullsized":"2024-05-08T05:41:37.576336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:42.085689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576077+0000","last_change":"2024-05-08T05:41:31.132478+0000","last_active":"2024-05-08T05:41:37.576077+0000","last_peered":"2024-05-08T05:41:37.576077+0000","last_clean":"2024-05-08T05:41:37.576077+0000","last_became_active":"2024-05-08T05:41:31.131093+0000","last_became_peered":"2024-05-08T05:41:31.131093+0000","last_unstale":"2024-05-08T05:41:37.576077+0000","last_undegraded":"2024-05-08T05:41:37.576077+0000","last_fullsized":"2024-05-08T05:41:37.576077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:07.648895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577230+0000","last_change":"2024-05-08T05:41:36.381669+0000","last_active":"2024-05-08T05:41:37.577230+0000","last_peered":"2024-05-08T05:41:37.577230+0000","last_clean":"2024-05-08T05:41:37.577230+0000","last_became_active":"2024-05-08T05:41:36.381233+0000","last_became_peered":"2024-05-08T05:41:36.381233+0000","last_unstale":"2024-05-08T05:41:37.577230+0000","last_undegraded":"2024-05-08T05:41:37.577230+0000","last_fullsized":"2024-05-08T05:41:37.577230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:07.864670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575368+0000","last_change":"2024-05-08T05:41:32.131789+0000","last_active":"2024-05-08T05:41:37.575368+0000","last_peered":"2024-05-08T05:41:37.575368+0000","last_clean":"2024-05-08T05:41:37.575368+0000","last_became_active":"2024-05-08T05:41:32.131515+0000","last_became_peered":"2024-05-08T05:41:32.131515+0000","last_unstale":"2024-05-08T05:41:37.575368+0000","last_undegraded":"2024-05-08T05:41:37.575368+0000","last_fullsized":"2024-05-08T05:41:37.575368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:03:20.127059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576417+0000","last_change":"2024-05-08T05:41:31.126864+0000","last_active":"2024-05-08T05:41:37.576417+0000","last_peered":"2024-05-08T05:41:37.576417+0000","last_clean":"2024-05-08T05:41:37.576417+0000","last_became_active":"2024-05-08T05:41:31.126568+0000","last_became_peered":"2024-05-08T05:41:31.126568+0000","last_unstale":"2024-05-08T05:41:37.576417+0000","last_undegraded":"2024-05-08T05:41:37.576417+0000","last_fullsized":"2024-05-08T05:41:37.576417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:39.672951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577692+0000","last_change":"2024-05-08T05:41:36.382511+0000","last_active":"2024-05-08T05:41:37.577692+0000","last_peered":"2024-05-08T05:41:37.577692+0000","last_clean":"2024-05-08T05:41:37.577692+0000","last_became_active":"2024-05-08T05:41:36.382110+0000","last_became_peered":"2024-05-08T05:41:36.382110+0000","last_unstale":"2024-05-08T05:41:37.577692+0000","last_undegraded":"2024-05-08T05:41:37.577692+0000","last_fullsized":"2024-05-08T05:41:37.577692+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:39:51.537856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576868+0000","last_change":"2024-05-08T05:41:32.127674+0000","last_active":"2024-05-08T05:41:37.576868+0000","last_peered":"2024-05-08T05:41:37.576868+0000","last_clean":"2024-05-08T05:41:37.576868+0000","last_became_active":"2024-05-08T05:41:32.127374+0000","last_became_peered":"2024-05-08T05:41:32.127374+0000","last_unstale":"2024-05-08T05:41:37.576868+0000","last_undegraded":"2024-05-08T05:41:37.576868+0000","last_fullsized":"2024-05-08T05:41:37.576868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:25:53.826185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:41.770215+0000","last_change":"2024-05-08T05:41:31.126507+0000","last_active":"2024-05-08T05:41:41.770215+0000","last_peered":"2024-05-08T05:41:41.770215+0000","last_clean":"2024-05-08T05:41:41.770215+0000","last_became_active":"2024-05-08T05:41:31.126218+0000","last_became_peered":"2024-05-08T05:41:31.126218+0000","last_unstale":"2024-05-08T05:41:41.770215+0000","last_undegraded":"2024-05-08T05:41:41.770215+0000","last_fullsized":"2024-05-08T05:41:41.770215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:11:56.288788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576865+0000","last_change":"2024-05-08T05:41:36.153303+0000","last_active":"2024-05-08T05:41:37.576865+0000","last_peered":"2024-05-08T05:41:37.576865+0000","last_clean":"2024-05-08T05:41:37.576865+0000","last_became_active":"2024-05-08T05:41:36.152849+0000","last_became_peered":"2024-05-08T05:41:36.152849+0000","last_unstale":"2024-05-08T05:41:37.576865+0000","last_undegraded":"2024-05-08T05:41:37.576865+0000","last_fullsized":"2024-05-08T05:41:37.576865+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:07:30.112175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154093+0000","last_change":"2024-05-08T05:41:32.121425+0000","last_active":"2024-05-08T05:41:37.154093+0000","last_peered":"2024-05-08T05:41:37.154093+0000","last_clean":"2024-05-08T05:41:37.154093+0000","last_became_active":"2024-05-08T05:41:32.121121+0000","last_became_peered":"2024-05-08T05:41:32.121121+0000","last_unstale":"2024-05-08T05:41:37.154093+0000","last_undegraded":"2024-05-08T05:41:37.154093+0000","last_fullsized":"2024-05-08T05:41:37.154093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:47.424136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576434+0000","last_change":"2024-05-08T05:41:31.132128+0000","last_active":"2024-05-08T05:41:37.576434+0000","last_peered":"2024-05-08T05:41:37.576434+0000","last_clean":"2024-05-08T05:41:37.576434+0000","last_became_active":"2024-05-08T05:41:31.131776+0000","last_became_peered":"2024-05-08T05:41:31.131776+0000","last_unstale":"2024-05-08T05:41:37.576434+0000","last_undegraded":"2024-05-08T05:41:37.576434+0000","last_fullsized":"2024-05-08T05:41:37.576434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:07.648895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153781+0000","last_change":"2024-05-08T05:41:36.152157+0000","last_active":"2024-05-08T05:41:37.153781+0000","last_peered":"2024-05-08T05:41:37.153781+0000","last_clean":"2024-05-08T05:41:37.153781+0000","last_became_active":"2024-05-08T05:41:36.151495+0000","last_became_peered":"2024-05-08T05:41:36.151495+0000","last_unstale":"2024-05-08T05:41:37.153781+0000","last_undegraded":"2024-05-08T05:41:37.153781+0000","last_fullsized":"2024-05-08T05:41:37.153781+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:34.172975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575332+0000","last_change":"2024-05-08T05:41:32.137651+0000","last_active":"2024-05-08T05:41:37.575332+0000","last_peered":"2024-05-08T05:41:37.575332+0000","last_clean":"2024-05-08T05:41:37.575332+0000","last_became_active":"2024-05-08T05:41:32.137380+0000","last_became_peered":"2024-05-08T05:41:32.137380+0000","last_unstale":"2024-05-08T05:41:37.575332+0000","last_undegraded":"2024-05-08T05:41:37.575332+0000","last_fullsized":"2024-05-08T05:41:37.575332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:44.038894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577757+0000","last_change":"2024-05-08T05:41:31.132598+0000","last_active":"2024-05-08T05:41:37.577757+0000","last_peered":"2024-05-08T05:41:37.577757+0000","last_clean":"2024-05-08T05:41:37.577757+0000","last_became_active":"2024-05-08T05:41:31.131053+0000","last_became_peered":"2024-05-08T05:41:31.131053+0000","last_unstale":"2024-05-08T05:41:37.577757+0000","last_undegraded":"2024-05-08T05:41:37.577757+0000","last_fullsized":"2024-05-08T05:41:37.577757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:11:56.288788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575991+0000","last_change":"2024-05-08T05:41:36.159470+0000","last_active":"2024-05-08T05:41:37.575991+0000","last_peered":"2024-05-08T05:41:37.575991+0000","last_clean":"2024-05-08T05:41:37.575991+0000","last_became_active":"2024-05-08T05:41:36.158246+0000","last_became_peered":"2024-05-08T05:41:36.158246+0000","last_unstale":"2024-05-08T05:41:37.575991+0000","last_undegraded":"2024-05-08T05:41:37.575991+0000","last_fullsized":"2024-05-08T05:41:37.575991+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:02.479497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577667+0000","last_change":"2024-05-08T05:41:32.139258+0000","last_active":"2024-05-08T05:41:37.577667+0000","last_peered":"2024-05-08T05:41:37.577667+0000","last_clean":"2024-05-08T05:41:37.577667+0000","last_became_active":"2024-05-08T05:41:32.138975+0000","last_became_peered":"2024-05-08T05:41:32.138975+0000","last_unstale":"2024-05-08T05:41:37.577667+0000","last_undegraded":"2024-05-08T05:41:37.577667+0000","last_fullsized":"2024-05-08T05:41:37.577667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:16.501105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577263+0000","last_change":"2024-05-08T05:41:31.133643+0000","last_active":"2024-05-08T05:41:37.577263+0000","last_peered":"2024-05-08T05:41:37.577263+0000","last_clean":"2024-05-08T05:41:37.577263+0000","last_became_active":"2024-05-08T05:41:31.133256+0000","last_became_peered":"2024-05-08T05:41:31.133256+0000","last_unstale":"2024-05-08T05:41:37.577263+0000","last_undegraded":"2024-05-08T05:41:37.577263+0000","last_fullsized":"2024-05-08T05:41:37.577263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:07.648895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576551+0000","last_change":"2024-05-08T05:41:36.383032+0000","last_active":"2024-05-08T05:41:37.576551+0000","last_peered":"2024-05-08T05:41:37.576551+0000","last_clean":"2024-05-08T05:41:37.576551+0000","last_became_active":"2024-05-08T05:41:36.382632+0000","last_became_peered":"2024-05-08T05:41:36.382632+0000","last_unstale":"2024-05-08T05:41:37.576551+0000","last_undegraded":"2024-05-08T05:41:37.576551+0000","last_fullsized":"2024-05-08T05:41:37.576551+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:07:08.116447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575585+0000","last_change":"2024-05-08T05:41:36.385447+0000","last_active":"2024-05-08T05:41:37.575585+0000","last_peered":"2024-05-08T05:41:37.575585+0000","last_clean":"2024-05-08T05:41:37.575585+0000","last_became_active":"2024-05-08T05:41:36.385062+0000","last_became_peered":"2024-05-08T05:41:36.385062+0000","last_unstale":"2024-05-08T05:41:37.575585+0000","last_undegraded":"2024-05-08T05:41:37.575585+0000","last_fullsized":"2024-05-08T05:41:37.575585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:06.547337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.574973+0000","last_change":"2024-05-08T05:41:32.127321+0000","last_active":"2024-05-08T05:41:37.574973+0000","last_peered":"2024-05-08T05:41:37.574973+0000","last_clean":"2024-05-08T05:41:37.574973+0000","last_became_active":"2024-05-08T05:41:32.127028+0000","last_became_peered":"2024-05-08T05:41:32.127028+0000","last_unstale":"2024-05-08T05:41:37.574973+0000","last_undegraded":"2024-05-08T05:41:37.574973+0000","last_fullsized":"2024-05-08T05:41:37.574973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:47.493805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575819+0000","last_change":"2024-05-08T05:41:31.131618+0000","last_active":"2024-05-08T05:41:37.575819+0000","last_peered":"2024-05-08T05:41:37.575819+0000","last_clean":"2024-05-08T05:41:37.575819+0000","last_became_active":"2024-05-08T05:41:31.131012+0000","last_became_peered":"2024-05-08T05:41:31.131012+0000","last_unstale":"2024-05-08T05:41:37.575819+0000","last_undegraded":"2024-05-08T05:41:37.575819+0000","last_fullsized":"2024-05-08T05:41:37.575819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:22.751500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577121+0000","last_change":"2024-05-08T05:41:36.158506+0000","last_active":"2024-05-08T05:41:37.577121+0000","last_peered":"2024-05-08T05:41:37.577121+0000","last_clean":"2024-05-08T05:41:37.577121+0000","last_became_active":"2024-05-08T05:41:36.158020+0000","last_became_peered":"2024-05-08T05:41:36.158020+0000","last_unstale":"2024-05-08T05:41:37.577121+0000","last_undegraded":"2024-05-08T05:41:37.577121+0000","last_fullsized":"2024-05-08T05:41:37.577121+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:20.531197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576038+0000","last_change":"2024-05-08T05:41:32.132554+0000","last_active":"2024-05-08T05:41:37.576038+0000","last_peered":"2024-05-08T05:41:37.576038+0000","last_clean":"2024-05-08T05:41:37.576038+0000","last_became_active":"2024-05-08T05:41:32.132257+0000","last_became_peered":"2024-05-08T05:41:32.132257+0000","last_unstale":"2024-05-08T05:41:37.576038+0000","last_undegraded":"2024-05-08T05:41:37.576038+0000","last_fullsized":"2024-05-08T05:41:37.576038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:16:33.854897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576780+0000","last_change":"2024-05-08T05:41:31.124710+0000","last_active":"2024-05-08T05:41:37.576780+0000","last_peered":"2024-05-08T05:41:37.576780+0000","last_clean":"2024-05-08T05:41:37.576780+0000","last_became_active":"2024-05-08T05:41:31.124304+0000","last_became_peered":"2024-05-08T05:41:31.124304+0000","last_unstale":"2024-05-08T05:41:37.576780+0000","last_undegraded":"2024-05-08T05:41:37.576780+0000","last_fullsized":"2024-05-08T05:41:37.576780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:49.171947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576513+0000","last_change":"2024-05-08T05:41:31.123731+0000","last_active":"2024-05-08T05:41:37.576513+0000","last_peered":"2024-05-08T05:41:37.576513+0000","last_clean":"2024-05-08T05:41:37.576513+0000","last_became_active":"2024-05-08T05:41:31.123423+0000","last_became_peered":"2024-05-08T05:41:31.123423+0000","last_unstale":"2024-05-08T05:41:37.576513+0000","last_undegraded":"2024-05-08T05:41:37.576513+0000","last_fullsized":"2024-05-08T05:41:37.576513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:24.722618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576325+0000","last_change":"2024-05-08T05:41:32.135788+0000","last_active":"2024-05-08T05:41:37.576325+0000","last_peered":"2024-05-08T05:41:37.576325+0000","last_clean":"2024-05-08T05:41:37.576325+0000","last_became_active":"2024-05-08T05:41:32.135522+0000","last_became_peered":"2024-05-08T05:41:32.135522+0000","last_unstale":"2024-05-08T05:41:37.576325+0000","last_undegraded":"2024-05-08T05:41:37.576325+0000","last_fullsized":"2024-05-08T05:41:37.576325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:42.041915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.148134+0000","last_change":"2024-05-08T05:41:36.155967+0000","last_active":"2024-05-08T05:41:37.148134+0000","last_peered":"2024-05-08T05:41:37.148134+0000","last_clean":"2024-05-08T05:41:37.148134+0000","last_became_active":"2024-05-08T05:41:36.155132+0000","last_became_peered":"2024-05-08T05:41:36.155132+0000","last_unstale":"2024-05-08T05:41:37.148134+0000","last_undegraded":"2024-05-08T05:41:37.148134+0000","last_fullsized":"2024-05-08T05:41:37.148134+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:58.729110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577152+0000","last_change":"2024-05-08T05:41:31.127645+0000","last_active":"2024-05-08T05:41:37.577152+0000","last_peered":"2024-05-08T05:41:37.577152+0000","last_clean":"2024-05-08T05:41:37.577152+0000","last_became_active":"2024-05-08T05:41:31.127330+0000","last_became_peered":"2024-05-08T05:41:31.127330+0000","last_unstale":"2024-05-08T05:41:37.577152+0000","last_undegraded":"2024-05-08T05:41:37.577152+0000","last_fullsized":"2024-05-08T05:41:37.577152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:30.091972+0000","last_clean_scrub_stamp":"2024-05-08T05:41:30.091972+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:09:22.751500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.153479+0000","last_change":"2024-05-08T05:41:32.125718+0000","last_active":"2024-05-08T05:41:37.153479+0000","last_peered":"2024-05-08T05:41:37.153479+0000","last_clean":"2024-05-08T05:41:37.153479+0000","last_became_active":"2024-05-08T05:41:32.125435+0000","last_became_peered":"2024-05-08T05:41:32.125435+0000","last_unstale":"2024-05-08T05:41:37.153479+0000","last_undegraded":"2024-05-08T05:41:37.153479+0000","last_fullsized":"2024-05-08T05:41:37.153479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:39.180293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575823+0000","last_change":"2024-05-08T05:41:36.158103+0000","last_active":"2024-05-08T05:41:37.575823+0000","last_peered":"2024-05-08T05:41:37.575823+0000","last_clean":"2024-05-08T05:41:37.575823+0000","last_became_active":"2024-05-08T05:41:36.157668+0000","last_became_peered":"2024-05-08T05:41:36.157668+0000","last_unstale":"2024-05-08T05:41:37.575823+0000","last_undegraded":"2024-05-08T05:41:37.575823+0000","last_fullsized":"2024-05-08T05:41:37.575823+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:35.127393+0000","last_clean_scrub_stamp":"2024-05-08T05:41:35.127393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:45:24.742821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575306+0000","last_change":"2024-05-08T05:41:32.137321+0000","last_active":"2024-05-08T05:41:37.575306+0000","last_peered":"2024-05-08T05:41:37.575306+0000","last_clean":"2024-05-08T05:41:37.575306+0000","last_became_active":"2024-05-08T05:41:32.136738+0000","last_became_peered":"2024-05-08T05:41:32.136738+0000","last_unstale":"2024-05-08T05:41:37.575306+0000","last_undegraded":"2024-05-08T05:41:37.575306+0000","last_fullsized":"2024-05-08T05:41:37.575306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:16:33.854897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.147790+0000","last_change":"2024-05-08T05:41:32.127130+0000","last_active":"2024-05-08T05:41:37.147790+0000","last_peered":"2024-05-08T05:41:37.147790+0000","last_clean":"2024-05-08T05:41:37.147790+0000","last_became_active":"2024-05-08T05:41:32.125853+0000","last_became_peered":"2024-05-08T05:41:32.125853+0000","last_unstale":"2024-05-08T05:41:37.147790+0000","last_undegraded":"2024-05-08T05:41:37.147790+0000","last_fullsized":"2024-05-08T05:41:37.147790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:47.424136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575968+0000","last_change":"2024-05-08T05:41:32.139558+0000","last_active":"2024-05-08T05:41:37.575968+0000","last_peered":"2024-05-08T05:41:37.575968+0000","last_clean":"2024-05-08T05:41:37.575968+0000","last_became_active":"2024-05-08T05:41:32.139243+0000","last_became_peered":"2024-05-08T05:41:32.139243+0000","last_unstale":"2024-05-08T05:41:37.575968+0000","last_undegraded":"2024-05-08T05:41:37.575968+0000","last_fullsized":"2024-05-08T05:41:37.575968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:07:26.082084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576235+0000","last_change":"2024-05-08T05:41:32.126388+0000","last_active":"2024-05-08T05:41:37.576235+0000","last_peered":"2024-05-08T05:41:37.576235+0000","last_clean":"2024-05-08T05:41:37.576235+0000","last_became_active":"2024-05-08T05:41:32.126078+0000","last_became_peered":"2024-05-08T05:41:32.126078+0000","last_unstale":"2024-05-08T05:41:37.576235+0000","last_undegraded":"2024-05-08T05:41:37.576235+0000","last_fullsized":"2024-05-08T05:41:37.576235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:53.563878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577307+0000","last_change":"2024-05-08T05:41:32.135523+0000","last_active":"2024-05-08T05:41:37.577307+0000","last_peered":"2024-05-08T05:41:37.577307+0000","last_clean":"2024-05-08T05:41:37.577307+0000","last_became_active":"2024-05-08T05:41:32.135248+0000","last_became_peered":"2024-05-08T05:41:32.135248+0000","last_unstale":"2024-05-08T05:41:37.577307+0000","last_undegraded":"2024-05-08T05:41:37.577307+0000","last_fullsized":"2024-05-08T05:41:37.577307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:42.041915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576262+0000","last_change":"2024-05-08T05:41:32.130227+0000","last_active":"2024-05-08T05:41:37.576262+0000","last_peered":"2024-05-08T05:41:37.576262+0000","last_clean":"2024-05-08T05:41:37.576262+0000","last_became_active":"2024-05-08T05:41:32.129175+0000","last_became_peered":"2024-05-08T05:41:32.129175+0000","last_unstale":"2024-05-08T05:41:37.576262+0000","last_undegraded":"2024-05-08T05:41:37.576262+0000","last_fullsized":"2024-05-08T05:41:37.576262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:53.563878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577357+0000","last_change":"2024-05-08T05:41:32.143284+0000","last_active":"2024-05-08T05:41:37.577357+0000","last_peered":"2024-05-08T05:41:37.577357+0000","last_clean":"2024-05-08T05:41:37.577357+0000","last_became_active":"2024-05-08T05:41:32.142815+0000","last_became_peered":"2024-05-08T05:41:32.142815+0000","last_unstale":"2024-05-08T05:41:37.577357+0000","last_undegraded":"2024-05-08T05:41:37.577357+0000","last_fullsized":"2024-05-08T05:41:37.577357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:53.563878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576778+0000","last_change":"2024-05-08T05:41:32.134965+0000","last_active":"2024-05-08T05:41:37.576778+0000","last_peered":"2024-05-08T05:41:37.576778+0000","last_clean":"2024-05-08T05:41:37.576778+0000","last_became_active":"2024-05-08T05:41:32.134641+0000","last_became_peered":"2024-05-08T05:41:32.134641+0000","last_unstale":"2024-05-08T05:41:37.576778+0000","last_undegraded":"2024-05-08T05:41:37.576778+0000","last_fullsized":"2024-05-08T05:41:37.576778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:30.142883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575749+0000","last_change":"2024-05-08T05:41:32.135139+0000","last_active":"2024-05-08T05:41:37.575749+0000","last_peered":"2024-05-08T05:41:37.575749+0000","last_clean":"2024-05-08T05:41:37.575749+0000","last_became_active":"2024-05-08T05:41:32.134839+0000","last_became_peered":"2024-05-08T05:41:32.134839+0000","last_unstale":"2024-05-08T05:41:37.575749+0000","last_undegraded":"2024-05-08T05:41:37.575749+0000","last_fullsized":"2024-05-08T05:41:37.575749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:53.563878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576660+0000","last_change":"2024-05-08T05:41:32.134673+0000","last_active":"2024-05-08T05:41:37.576660+0000","last_peered":"2024-05-08T05:41:37.576660+0000","last_clean":"2024-05-08T05:41:37.576660+0000","last_became_active":"2024-05-08T05:41:32.134402+0000","last_became_peered":"2024-05-08T05:41:32.134402+0000","last_unstale":"2024-05-08T05:41:37.576660+0000","last_undegraded":"2024-05-08T05:41:37.576660+0000","last_fullsized":"2024-05-08T05:41:37.576660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:03:20.127059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575846+0000","last_change":"2024-05-08T05:41:32.140242+0000","last_active":"2024-05-08T05:41:37.575846+0000","last_peered":"2024-05-08T05:41:37.575846+0000","last_clean":"2024-05-08T05:41:37.575846+0000","last_became_active":"2024-05-08T05:41:32.139932+0000","last_became_peered":"2024-05-08T05:41:32.139932+0000","last_unstale":"2024-05-08T05:41:37.575846+0000","last_undegraded":"2024-05-08T05:41:37.575846+0000","last_fullsized":"2024-05-08T05:41:37.575846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:42.739734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575975+0000","last_change":"2024-05-08T05:41:32.132509+0000","last_active":"2024-05-08T05:41:37.575975+0000","last_peered":"2024-05-08T05:41:37.575975+0000","last_clean":"2024-05-08T05:41:37.575975+0000","last_became_active":"2024-05-08T05:41:32.132240+0000","last_became_peered":"2024-05-08T05:41:32.132240+0000","last_unstale":"2024-05-08T05:41:37.575975+0000","last_undegraded":"2024-05-08T05:41:37.575975+0000","last_fullsized":"2024-05-08T05:41:37.575975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:07:26.071605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576973+0000","last_change":"2024-05-08T05:41:32.140982+0000","last_active":"2024-05-08T05:41:37.576973+0000","last_peered":"2024-05-08T05:41:37.576973+0000","last_clean":"2024-05-08T05:41:37.576973+0000","last_became_active":"2024-05-08T05:41:32.140698+0000","last_became_peered":"2024-05-08T05:41:32.140698+0000","last_unstale":"2024-05-08T05:41:37.576973+0000","last_undegraded":"2024-05-08T05:41:37.576973+0000","last_fullsized":"2024-05-08T05:41:37.576973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:07:26.071605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577173+0000","last_change":"2024-05-08T05:41:32.134190+0000","last_active":"2024-05-08T05:41:37.577173+0000","last_peered":"2024-05-08T05:41:37.577173+0000","last_clean":"2024-05-08T05:41:37.577173+0000","last_became_active":"2024-05-08T05:41:32.132164+0000","last_became_peered":"2024-05-08T05:41:32.132164+0000","last_unstale":"2024-05-08T05:41:37.577173+0000","last_undegraded":"2024-05-08T05:41:37.577173+0000","last_fullsized":"2024-05-08T05:41:37.577173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:13.172615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.148157+0000","last_change":"2024-05-08T05:41:32.131091+0000","last_active":"2024-05-08T05:41:37.148157+0000","last_peered":"2024-05-08T05:41:37.148157+0000","last_clean":"2024-05-08T05:41:37.148157+0000","last_became_active":"2024-05-08T05:41:32.130767+0000","last_became_peered":"2024-05-08T05:41:32.130767+0000","last_unstale":"2024-05-08T05:41:37.148157+0000","last_undegraded":"2024-05-08T05:41:37.148157+0000","last_fullsized":"2024-05-08T05:41:37.148157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:42.596678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577931+0000","last_change":"2024-05-08T05:41:32.137943+0000","last_active":"2024-05-08T05:41:37.577931+0000","last_peered":"2024-05-08T05:41:37.577931+0000","last_clean":"2024-05-08T05:41:37.577931+0000","last_became_active":"2024-05-08T05:41:32.137662+0000","last_became_peered":"2024-05-08T05:41:32.137662+0000","last_unstale":"2024-05-08T05:41:37.577931+0000","last_undegraded":"2024-05-08T05:41:37.577931+0000","last_fullsized":"2024-05-08T05:41:37.577931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:17.291314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575635+0000","last_change":"2024-05-08T05:41:32.129374+0000","last_active":"2024-05-08T05:41:37.575635+0000","last_peered":"2024-05-08T05:41:37.575635+0000","last_clean":"2024-05-08T05:41:37.575635+0000","last_became_active":"2024-05-08T05:41:32.128594+0000","last_became_peered":"2024-05-08T05:41:32.128594+0000","last_unstale":"2024-05-08T05:41:37.575635+0000","last_undegraded":"2024-05-08T05:41:37.575635+0000","last_fullsized":"2024-05-08T05:41:37.575635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:01:32.075337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575072+0000","last_change":"2024-05-08T05:41:32.131159+0000","last_active":"2024-05-08T05:41:37.575072+0000","last_peered":"2024-05-08T05:41:37.575072+0000","last_clean":"2024-05-08T05:41:37.575072+0000","last_became_active":"2024-05-08T05:41:32.130766+0000","last_became_peered":"2024-05-08T05:41:32.130766+0000","last_unstale":"2024-05-08T05:41:37.575072+0000","last_undegraded":"2024-05-08T05:41:37.575072+0000","last_fullsized":"2024-05-08T05:41:37.575072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:16:33.854897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575282+0000","last_change":"2024-05-08T05:41:32.119797+0000","last_active":"2024-05-08T05:41:37.575282+0000","last_peered":"2024-05-08T05:41:37.575282+0000","last_clean":"2024-05-08T05:41:37.575282+0000","last_became_active":"2024-05-08T05:41:32.119134+0000","last_became_peered":"2024-05-08T05:41:32.119134+0000","last_unstale":"2024-05-08T05:41:37.575282+0000","last_undegraded":"2024-05-08T05:41:37.575282+0000","last_fullsized":"2024-05-08T05:41:37.575282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:47.493805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577070+0000","last_change":"2024-05-08T05:41:32.135351+0000","last_active":"2024-05-08T05:41:37.577070+0000","last_peered":"2024-05-08T05:41:37.577070+0000","last_clean":"2024-05-08T05:41:37.577070+0000","last_became_active":"2024-05-08T05:41:32.133812+0000","last_became_peered":"2024-05-08T05:41:32.133812+0000","last_unstale":"2024-05-08T05:41:37.577070+0000","last_undegraded":"2024-05-08T05:41:37.577070+0000","last_fullsized":"2024-05-08T05:41:37.577070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:29.473897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.154048+0000","last_change":"2024-05-08T05:41:32.136108+0000","last_active":"2024-05-08T05:41:37.154048+0000","last_peered":"2024-05-08T05:41:37.154048+0000","last_clean":"2024-05-08T05:41:37.154048+0000","last_became_active":"2024-05-08T05:41:32.135622+0000","last_became_peered":"2024-05-08T05:41:32.135622+0000","last_unstale":"2024-05-08T05:41:37.154048+0000","last_undegraded":"2024-05-08T05:41:37.154048+0000","last_fullsized":"2024-05-08T05:41:37.154048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:14.470927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576751+0000","last_change":"2024-05-08T05:41:32.138499+0000","last_active":"2024-05-08T05:41:37.576751+0000","last_peered":"2024-05-08T05:41:37.576751+0000","last_clean":"2024-05-08T05:41:37.576751+0000","last_became_active":"2024-05-08T05:41:32.138049+0000","last_became_peered":"2024-05-08T05:41:32.138049+0000","last_unstale":"2024-05-08T05:41:37.576751+0000","last_undegraded":"2024-05-08T05:41:37.576751+0000","last_fullsized":"2024-05-08T05:41:37.576751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:03:20.127059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577193+0000","last_change":"2024-05-08T05:41:32.138310+0000","last_active":"2024-05-08T05:41:37.577193+0000","last_peered":"2024-05-08T05:41:37.577193+0000","last_clean":"2024-05-08T05:41:37.577193+0000","last_became_active":"2024-05-08T05:41:32.137983+0000","last_became_peered":"2024-05-08T05:41:32.137983+0000","last_unstale":"2024-05-08T05:41:37.577193+0000","last_undegraded":"2024-05-08T05:41:37.577193+0000","last_fullsized":"2024-05-08T05:41:37.577193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:42.041915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577267+0000","last_change":"2024-05-08T05:41:32.144208+0000","last_active":"2024-05-08T05:41:37.577267+0000","last_peered":"2024-05-08T05:41:37.577267+0000","last_clean":"2024-05-08T05:41:37.577267+0000","last_became_active":"2024-05-08T05:41:32.143884+0000","last_became_peered":"2024-05-08T05:41:32.143884+0000","last_unstale":"2024-05-08T05:41:37.577267+0000","last_undegraded":"2024-05-08T05:41:37.577267+0000","last_fullsized":"2024-05-08T05:41:37.577267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:07:26.082084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575800+0000","last_change":"2024-05-08T05:41:32.136185+0000","last_active":"2024-05-08T05:41:37.575800+0000","last_peered":"2024-05-08T05:41:37.575800+0000","last_clean":"2024-05-08T05:41:37.575800+0000","last_became_active":"2024-05-08T05:41:32.135878+0000","last_became_peered":"2024-05-08T05:41:32.135878+0000","last_unstale":"2024-05-08T05:41:37.575800+0000","last_undegraded":"2024-05-08T05:41:37.575800+0000","last_fullsized":"2024-05-08T05:41:37.575800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:44.038894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576529+0000","last_change":"2024-05-08T05:41:32.137009+0000","last_active":"2024-05-08T05:41:37.576529+0000","last_peered":"2024-05-08T05:41:37.576529+0000","last_clean":"2024-05-08T05:41:37.576529+0000","last_became_active":"2024-05-08T05:41:32.136708+0000","last_became_peered":"2024-05-08T05:41:32.136708+0000","last_unstale":"2024-05-08T05:41:37.576529+0000","last_undegraded":"2024-05-08T05:41:37.576529+0000","last_fullsized":"2024-05-08T05:41:37.576529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:30.142883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575732+0000","last_change":"2024-05-08T05:41:32.137351+0000","last_active":"2024-05-08T05:41:37.575732+0000","last_peered":"2024-05-08T05:41:37.575732+0000","last_clean":"2024-05-08T05:41:37.575732+0000","last_became_active":"2024-05-08T05:41:32.135445+0000","last_became_peered":"2024-05-08T05:41:32.135445+0000","last_unstale":"2024-05-08T05:41:37.575732+0000","last_undegraded":"2024-05-08T05:41:37.575732+0000","last_fullsized":"2024-05-08T05:41:37.575732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:30.142883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575746+0000","last_change":"2024-05-08T05:41:32.132566+0000","last_active":"2024-05-08T05:41:37.575746+0000","last_peered":"2024-05-08T05:41:37.575746+0000","last_clean":"2024-05-08T05:41:37.575746+0000","last_became_active":"2024-05-08T05:41:32.131787+0000","last_became_peered":"2024-05-08T05:41:32.131787+0000","last_unstale":"2024-05-08T05:41:37.575746+0000","last_undegraded":"2024-05-08T05:41:37.575746+0000","last_fullsized":"2024-05-08T05:41:37.575746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:42.739734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.576521+0000","last_change":"2024-05-08T05:41:32.135611+0000","last_active":"2024-05-08T05:41:37.576521+0000","last_peered":"2024-05-08T05:41:37.576521+0000","last_clean":"2024-05-08T05:41:37.576521+0000","last_became_active":"2024-05-08T05:41:32.135346+0000","last_became_peered":"2024-05-08T05:41:32.135346+0000","last_unstale":"2024-05-08T05:41:37.576521+0000","last_undegraded":"2024-05-08T05:41:37.576521+0000","last_fullsized":"2024-05-08T05:41:37.576521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:29.473897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575102+0000","last_change":"2024-05-08T05:41:32.118779+0000","last_active":"2024-05-08T05:41:37.575102+0000","last_peered":"2024-05-08T05:41:37.575102+0000","last_clean":"2024-05-08T05:41:37.575102+0000","last_became_active":"2024-05-08T05:41:32.118481+0000","last_became_peered":"2024-05-08T05:41:32.118481+0000","last_unstale":"2024-05-08T05:41:37.575102+0000","last_undegraded":"2024-05-08T05:41:37.575102+0000","last_fullsized":"2024-05-08T05:41:37.575102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:54.699019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.575394+0000","last_change":"2024-05-08T05:41:32.128168+0000","last_active":"2024-05-08T05:41:37.575394+0000","last_peered":"2024-05-08T05:41:37.575394+0000","last_clean":"2024-05-08T05:41:37.575394+0000","last_became_active":"2024-05-08T05:41:32.127869+0000","last_became_peered":"2024-05-08T05:41:32.127869+0000","last_unstale":"2024-05-08T05:41:37.575394+0000","last_undegraded":"2024-05-08T05:41:37.575394+0000","last_fullsized":"2024-05-08T05:41:37.575394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:07:26.071605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-05-08T05:41:37.577048+0000","last_change":"2024-05-08T05:41:32.134768+0000","last_active":"2024-05-08T05:41:37.577048+0000","last_peered":"2024-05-08T05:41:37.577048+0000","last_clean":"2024-05-08T05:41:37.577048+0000","last_became_active":"2024-05-08T05:41:32.132540+0000","last_became_peered":"2024-05-08T05:41:32.132540+0000","last_unstale":"2024-05-08T05:41:37.577048+0000","last_undegraded":"2024-05-08T05:41:37.577048+0000","last_fullsized":"2024-05-08T05:41:37.577048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:31.097302+0000","last_clean_scrub_stamp":"2024-05-08T05:41:31.097302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:50:32.252569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":847872,"data_stored":690366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542155,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":262144,"data_stored":103258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542154,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":99506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":99542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":262144,"data_stored":103258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":249856,"data_stored":99416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":258048,"data_stored":99576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":856064,"data_stored":690422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:41:51.145 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T05:41:51.145 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T05:41:51.146 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-08T05:41:51.146 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T05:41:51.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.283+0000 7fdd6886d640 1 Processor -- start 2024-05-08T05:41:51.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.283+0000 7fdd6886d640 1 -- start start 2024-05-08T05:41:51.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.284+0000 7fdd6886d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd640d3cd0 0x7fdd6416a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:51.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.284+0000 7fdd6886d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd640d1760 con 0x7fdd640d0ec0 2024-05-08T05:41:51.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.284+0000 7fdd6886d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd640d1a40 con 0x7fdd6415f380 2024-05-08T05:41:51.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.284+0000 7fdd6886d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdd640d30f0 con 0x7fdd640d3cd0 2024-05-08T05:41:51.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.284+0000 7fdd61d9b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fdd640d0ec0 0x7fdd640d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49640/0 (socket says 172.21.15.45:49640) 2024-05-08T05:41:51.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.284+0000 7fdd61d9b640 1 -- 172.21.15.45:0/480912227 learned_addr learned my addr 172.21.15.45:0/480912227 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:51.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.284+0000 7fdd62d9d640 1 --2- 172.21.15.45:0/480912227 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd640d3cd0 0x7fdd6416a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:51.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6159a640 1 -- 172.21.15.45:0/480912227 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1445499248 0 0) 0x7fdd640d1760 con 0x7fdd640d0ec0 2024-05-08T05:41:51.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6159a640 1 -- 172.21.15.45:0/480912227 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd4c003520 con 0x7fdd640d0ec0 2024-05-08T05:41:51.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6159a640 1 -- 172.21.15.45:0/480912227 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1257551655 0 0) 0x7fdd640d1a40 con 0x7fdd6415f380 2024-05-08T05:41:51.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6159a640 1 -- 172.21.15.45:0/480912227 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd640d1760 con 0x7fdd6415f380 2024-05-08T05:41:51.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6159a640 1 -- 172.21.15.45:0/480912227 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1863705728 0 0) 0x7fdd4c003520 con 0x7fdd640d0ec0 2024-05-08T05:41:51.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6159a640 1 -- 172.21.15.45:0/480912227 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd640d3cd0 msgr2=0x7fdd6416a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:51.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6159a640 1 --2- 172.21.15.45:0/480912227 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd640d3cd0 0x7fdd6416a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6159a640 1 -- 172.21.15.45:0/480912227 >> v1:172.21.15.63:6789/0 conn(0x7fdd6415f380 legacy=0x7fdd640d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6159a640 1 -- 172.21.15.45:0/480912227 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd640d1d20 con 0x7fdd640d0ec0 2024-05-08T05:41:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6159a640 1 -- 172.21.15.45:0/480912227 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fdd58001a70 con 0x7fdd640d0ec0 2024-05-08T05:41:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6159a640 1 -- 172.21.15.45:0/480912227 >> v1:172.21.15.45:6789/0 conn(0x7fdd640d0ec0 legacy=0x7fdd640d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6159a640 1 --2- 172.21.15.45:0/480912227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd4c003c10 0x7fdd4c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6159a640 1 --2- 172.21.15.45:0/480912227 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd640d3cd0 0x7fdd4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6159a640 1 --2- 172.21.15.45:0/480912227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd4c004d40 0x7fdd4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6159a640 1 -- 172.21.15.45:0/480912227 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdd640d30f0 con 0x7fdd4c003c10 2024-05-08T05:41:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6159a640 1 -- 172.21.15.45:0/480912227 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c00bb50 con 0x7fdd4c004d40 2024-05-08T05:41:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6159a640 1 -- 172.21.15.45:0/480912227 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdd4c00bd50 con 0x7fdd640d3cd0 2024-05-08T05:41:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6259c640 1 --2- 172.21.15.45:0/480912227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd4c003c10 0x7fdd4c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd62d9d640 1 --2- 172.21.15.45:0/480912227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd4c004d40 0x7fdd4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd61d9b640 1 --2- 172.21.15.45:0/480912227 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd640d3cd0 0x7fdd4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:51.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6259c640 1 -- 172.21.15.45:0/480912227 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd640d3cd0 msgr2=0x7fdd4c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:51.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6259c640 1 --2- 172.21.15.45:0/480912227 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd640d3cd0 0x7fdd4c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6259c640 1 -- 172.21.15.45:0/480912227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd4c004d40 msgr2=0x7fdd4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:51.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6259c640 1 --2- 172.21.15.45:0/480912227 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd4c004d40 0x7fdd4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.285+0000 7fdd6259c640 1 -- 172.21.15.45:0/480912227 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd4c003520 con 0x7fdd4c003c10 2024-05-08T05:41:51.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.286+0000 7fdd6259c640 1 --2- 172.21.15.45:0/480912227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd4c003c10 0x7fdd4c004010 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdd540018a0 tx=0x7fdd4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae45ddc220af2007 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:51.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.286+0000 7fdd6159a640 1 -- 172.21.15.45:0/480912227 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd540014c0 con 0x7fdd4c003c10 2024-05-08T05:41:51.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.286+0000 7fdd6159a640 1 -- 172.21.15.45:0/480912227 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd5402acf0 con 0x7fdd4c003c10 2024-05-08T05:41:51.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.286+0000 7fdd6159a640 1 -- 172.21.15.45:0/480912227 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54005310 con 0x7fdd4c003c10 2024-05-08T05:41:51.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.287+0000 7fdd6886d640 1 -- 172.21.15.45:0/480912227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd4c003c10 msgr2=0x7fdd4c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:51.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.287+0000 7fdd6886d640 1 --2- 172.21.15.45:0/480912227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd4c003c10 0x7fdd4c004010 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdd540018a0 tx=0x7fdd4c005830 comp rx=0 tx=0).stop 2024-05-08T05:41:51.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.287+0000 7fdd62d9d640 1 -- 172.21.15.45:0/480912227 reap_dead start 2024-05-08T05:41:51.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.287+0000 7fdd6886d640 1 -- 172.21.15.45:0/480912227 shutdown_connections 2024-05-08T05:41:51.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.287+0000 7fdd6886d640 1 -- 172.21.15.45:0/480912227 >> 172.21.15.45:0/480912227 conn(0x7fdd640d4fe0 msgr2=0x7fdd640d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:51.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.287+0000 7fdd6886d640 1 -- 172.21.15.45:0/480912227 shutdown_connections 2024-05-08T05:41:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.287+0000 7fdd6886d640 1 -- 172.21.15.45:0/480912227 wait complete. 2024-05-08T05:41:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.288+0000 7fdd6886d640 1 Processor -- start 2024-05-08T05:41:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.288+0000 7fdd6886d640 1 -- start start 2024-05-08T05:41:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.289+0000 7fdd6886d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd4c003c10 0x7fdd641504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.289+0000 7fdd6886d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd64162610 0x7fdd641607f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.289+0000 7fdd6886d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd64160d30 0x7fdd640a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.289+0000 7fdd6886d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdd640b8e40 con 0x7fdd4c003c10 2024-05-08T05:41:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.289+0000 7fdd6886d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdd640d2370 con 0x7fdd64162610 2024-05-08T05:41:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.289+0000 7fdd6886d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdd6415f760 con 0x7fdd64160d30 2024-05-08T05:41:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.289+0000 7fdd6259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd4c003c10 0x7fdd641504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.289+0000 7fdd6259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd4c003c10 0x7fdd641504c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34680/0 (socket says 172.21.15.45:34680) 2024-05-08T05:41:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.289+0000 7fdd6259c640 1 -- 172.21.15.45:0/1495151193 learned_addr learned my addr 172.21.15.45:0/1495151193 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:51.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.289+0000 7fdd61d9b640 1 --2- 172.21.15.45:0/1495151193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd64162610 0x7fdd641607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.289+0000 7fdd6259c640 1 -- 172.21.15.45:0/1495151193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd64160d30 msgr2=0x7fdd640a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.289+0000 7fdd62d9d640 1 --2- 172.21.15.45:0/1495151193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd64160d30 0x7fdd640a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.289+0000 7fdd6259c640 1 --2- 172.21.15.45:0/1495151193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd64160d30 0x7fdd640a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.289+0000 7fdd6259c640 1 -- 172.21.15.45:0/1495151193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd64162610 msgr2=0x7fdd641607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.289+0000 7fdd6259c640 1 --2- 172.21.15.45:0/1495151193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd64162610 0x7fdd641607f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.289+0000 7fdd6259c640 1 -- 172.21.15.45:0/1495151193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd64161570 con 0x7fdd4c003c10 2024-05-08T05:41:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.289+0000 7fdd61d9b640 1 --2- 172.21.15.45:0/1495151193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd64162610 0x7fdd641607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.289+0000 7fdd62d9d640 1 --2- 172.21.15.45:0/1495151193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd64160d30 0x7fdd640a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.290+0000 7fdd6259c640 1 --2- 172.21.15.45:0/1495151193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd4c003c10 0x7fdd641504c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdd540019c0 tx=0x7fdd540012f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.290+0000 7fdd52ffd640 1 -- 172.21.15.45:0/1495151193 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54007a10 con 0x7fdd4c003c10 2024-05-08T05:41:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.290+0000 7fdd52ffd640 1 -- 172.21.15.45:0/1495151193 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd540075e0 con 0x7fdd4c003c10 2024-05-08T05:41:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.290+0000 7fdd6886d640 1 -- 172.21.15.45:0/1495151193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd64161850 con 0x7fdd4c003c10 2024-05-08T05:41:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.290+0000 7fdd52ffd640 1 -- 172.21.15.45:0/1495151193 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd540014c0 con 0x7fdd4c003c10 2024-05-08T05:41:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.290+0000 7fdd6886d640 1 -- 172.21.15.45:0/1495151193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd64161e10 con 0x7fdd4c003c10 2024-05-08T05:41:51.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.291+0000 7fdd6886d640 1 -- 172.21.15.45:0/1495151193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd640693f0 con 0x7fdd4c003c10 2024-05-08T05:41:51.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.291+0000 7fdd52ffd640 1 -- 172.21.15.45:0/1495151193 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdd540069d0 con 0x7fdd4c003c10 2024-05-08T05:41:51.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.292+0000 7fdd52ffd640 1 --2- 172.21.15.45:0/1495151193 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdd44076300 0x7fdd440787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:51.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.292+0000 7fdd52ffd640 1 -- 172.21.15.45:0/1495151193 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fdd540b7220 con 0x7fdd4c003c10 2024-05-08T05:41:51.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.292+0000 7fdd61d9b640 1 --2- 172.21.15.45:0/1495151193 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdd44076300 0x7fdd440787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:51.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.293+0000 7fdd61d9b640 1 --2- 172.21.15.45:0/1495151193 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdd44076300 0x7fdd440787c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdd640cbb20 tx=0x7fdd58006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:51.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.296+0000 7fdd52ffd640 1 -- 172.21.15.45:0/1495151193 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdd54080900 con 0x7fdd4c003c10 2024-05-08T05:41:51.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.528+0000 7fdd6886d640 1 -- 172.21.15.45:0/1495151193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fdd6408cfd0 con 0x7fdd4c003c10 2024-05-08T05:41:51.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.528+0000 7fdd52ffd640 1 -- 172.21.15.45:0/1495151193 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fdd54080200 con 0x7fdd4c003c10 2024-05-08T05:41:51.529 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:51.530 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T05:41:51.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.530+0000 7fdd6886d640 1 -- 172.21.15.45:0/1495151193 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdd44076300 msgr2=0x7fdd440787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:51.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.530+0000 7fdd6886d640 1 --2- 172.21.15.45:0/1495151193 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdd44076300 0x7fdd440787c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdd640cbb20 tx=0x7fdd58006040 comp rx=0 tx=0).stop 2024-05-08T05:41:51.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.530+0000 7fdd6886d640 1 -- 172.21.15.45:0/1495151193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd4c003c10 msgr2=0x7fdd641504c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:51.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.530+0000 7fdd6886d640 1 --2- 172.21.15.45:0/1495151193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd4c003c10 0x7fdd641504c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdd540019c0 tx=0x7fdd540012f0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.530+0000 7fdd6886d640 1 -- 172.21.15.45:0/1495151193 shutdown_connections 2024-05-08T05:41:51.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.530+0000 7fdd6886d640 1 --2- 172.21.15.45:0/1495151193 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdd44076300 0x7fdd440787c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.530+0000 7fdd6886d640 1 --2- 172.21.15.45:0/1495151193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdd64160d30 0x7fdd640a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.530+0000 7fdd6886d640 1 --2- 172.21.15.45:0/1495151193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdd64162610 0x7fdd641607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.530+0000 7fdd6886d640 1 --2- 172.21.15.45:0/1495151193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd4c003c10 0x7fdd641504c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.530+0000 7fdd6886d640 1 -- 172.21.15.45:0/1495151193 >> 172.21.15.45:0/1495151193 conn(0x7fdd640d4fe0 msgr2=0x7fdd640b7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:51.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.530+0000 7fdd6886d640 1 -- 172.21.15.45:0/1495151193 shutdown_connections 2024-05-08T05:41:51.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.530+0000 7fdd6886d640 1 -- 172.21.15.45:0/1495151193 wait complete. 2024-05-08T05:41:51.544 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-08T05:41:51.544 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:41:51.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.646+0000 7f70c5f9d640 1 Processor -- start 2024-05-08T05:41:51.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.646+0000 7f70c5f9d640 1 -- start start 2024-05-08T05:41:51.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.647+0000 7f70c5f9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f70c00d1970 0x7f70c00d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:51.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.647+0000 7f70c5f9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f70c00d4cc0 0x7f70c00d50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:51.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.647+0000 7f70c5f9d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70c01630b0 con 0x7f70c015f420 2024-05-08T05:41:51.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.647+0000 7f70c5f9d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f70c00d3e70 con 0x7f70c00d1970 2024-05-08T05:41:51.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.647+0000 7f70c5f9d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f70c00d2550 con 0x7f70c00d4cc0 2024-05-08T05:41:51.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.647+0000 7f70beffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f70c00d1970 0x7f70c00d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:51.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.647+0000 7f70beffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f70c00d1970 0x7f70c00d1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34686/0 (socket says 172.21.15.45:34686) 2024-05-08T05:41:51.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.647+0000 7f70beffd640 1 -- 172.21.15.45:0/2331970505 learned_addr learned my addr 172.21.15.45:0/2331970505 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:51.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.648+0000 7f70bffff640 1 --2- 172.21.15.45:0/2331970505 >> v2:172.21.15.63:3300/0 conn(0x7f70c00d4cc0 0x7f70c00d50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:51.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.648+0000 7f70beffd640 1 -- 172.21.15.45:0/2331970505 >> v2:172.21.15.63:3300/0 conn(0x7f70c00d4cc0 msgr2=0x7f70c00d50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:51.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.648+0000 7f70beffd640 1 --2- 172.21.15.45:0/2331970505 >> v2:172.21.15.63:3300/0 conn(0x7f70c00d4cc0 0x7f70c00d50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.648+0000 7f70beffd640 1 -- 172.21.15.45:0/2331970505 >> v1:172.21.15.63:6789/0 conn(0x7f70c015f420 legacy=0x7f70c00d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:51.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.648+0000 7f70beffd640 1 -- 172.21.15.45:0/2331970505 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c00d2c80 con 0x7f70c00d1970 2024-05-08T05:41:51.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.648+0000 7f70beffd640 1 --2- 172.21.15.45:0/2331970505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c00d1970 0x7f70c00d1e20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f70c00d17a0 tx=0x7f70b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03f343349ff39f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:51.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.649+0000 7f70be7fc640 1 -- 172.21.15.45:0/2331970505 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4037070 con 0x7f70c00d1970 2024-05-08T05:41:51.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.649+0000 7f70be7fc640 1 -- 172.21.15.45:0/2331970505 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b4032440 con 0x7f70c00d1970 2024-05-08T05:41:51.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.649+0000 7f70be7fc640 1 -- 172.21.15.45:0/2331970505 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4032810 con 0x7f70c00d1970 2024-05-08T05:41:51.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.649+0000 7f70c5f9d640 1 -- 172.21.15.45:0/2331970505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c00d1970 msgr2=0x7f70c00d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:51.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.649+0000 7f70c5f9d640 1 --2- 172.21.15.45:0/2331970505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c00d1970 0x7f70c00d1e20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f70c00d17a0 tx=0x7f70b402a270 comp rx=0 tx=0).stop 2024-05-08T05:41:51.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.649+0000 7f70c5f9d640 1 -- 172.21.15.45:0/2331970505 shutdown_connections 2024-05-08T05:41:51.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.649+0000 7f70c5f9d640 1 --2- 172.21.15.45:0/2331970505 >> v2:172.21.15.63:3300/0 conn(0x7f70c00d4cc0 0x7f70c00d50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.649+0000 7f70c5f9d640 1 --2- 172.21.15.45:0/2331970505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c00d1970 0x7f70c00d1e20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.649+0000 7f70c5f9d640 1 -- 172.21.15.45:0/2331970505 >> 172.21.15.45:0/2331970505 conn(0x7f70c008eac0 msgr2=0x7f70c009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:51.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.649+0000 7f70c5f9d640 1 -- 172.21.15.45:0/2331970505 shutdown_connections 2024-05-08T05:41:51.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.649+0000 7f70c5f9d640 1 -- 172.21.15.45:0/2331970505 wait complete. 2024-05-08T05:41:51.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.650+0000 7f70c5f9d640 1 Processor -- start 2024-05-08T05:41:51.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.651+0000 7f70c5f9d640 1 -- start start 2024-05-08T05:41:51.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.651+0000 7f70c5f9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c00d1970 0x7f70c01f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:51.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.651+0000 7f70c5f9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70c00d4cc0 0x7f70c00ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:51.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.651+0000 7f70c5f9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70c00b21b0 0x7f70c00b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.651+0000 7f70c5f9d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f70c00d2830 con 0x7f70c00d1970 2024-05-08T05:41:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.651+0000 7f70c5f9d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f70c00d3240 con 0x7f70c00b21b0 2024-05-08T05:41:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.651+0000 7f70c5f9d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f70c00d5620 con 0x7f70c00d4cc0 2024-05-08T05:41:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.651+0000 7f70bf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c00d1970 0x7f70c01f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.651+0000 7f70bf7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c00d1970 0x7f70c01f5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34696/0 (socket says 172.21.15.45:34696) 2024-05-08T05:41:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.651+0000 7f70bf7fe640 1 -- 172.21.15.45:0/1053559975 learned_addr learned my addr 172.21.15.45:0/1053559975 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.652+0000 7f70beffd640 1 --2- 172.21.15.45:0/1053559975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70c00d4cc0 0x7f70c00ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.652+0000 7f70bffff640 1 --2- 172.21.15.45:0/1053559975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70c00b21b0 0x7f70c00b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.652+0000 7f70bf7fe640 1 -- 172.21.15.45:0/1053559975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70c00d4cc0 msgr2=0x7f70c00ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.652+0000 7f70bf7fe640 1 --2- 172.21.15.45:0/1053559975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70c00d4cc0 0x7f70c00ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.652+0000 7f70bf7fe640 1 -- 172.21.15.45:0/1053559975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70c00b21b0 msgr2=0x7f70c00b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.652+0000 7f70bf7fe640 1 --2- 172.21.15.45:0/1053559975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70c00b21b0 0x7f70c00b2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.652+0000 7f70bf7fe640 1 -- 172.21.15.45:0/1053559975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c00cf2e0 con 0x7f70c00d1970 2024-05-08T05:41:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.652+0000 7f70bf7fe640 1 --2- 172.21.15.45:0/1053559975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c00d1970 0x7f70c01f5300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f70b000c3c0 tx=0x7f70b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:51.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.652+0000 7f70a7fff640 1 -- 172.21.15.45:0/1053559975 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b00052d0 con 0x7f70c00d1970 2024-05-08T05:41:51.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.652+0000 7f70a7fff640 1 -- 172.21.15.45:0/1053559975 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b0005490 con 0x7f70c00d1970 2024-05-08T05:41:51.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.652+0000 7f70c5f9d640 1 -- 172.21.15.45:0/1053559975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c01489e0 con 0x7f70c00d1970 2024-05-08T05:41:51.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.652+0000 7f70a7fff640 1 -- 172.21.15.45:0/1053559975 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0007260 con 0x7f70c00d1970 2024-05-08T05:41:51.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.652+0000 7f70c5f9d640 1 -- 172.21.15.45:0/1053559975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c00ce5a0 con 0x7f70c00d1970 2024-05-08T05:41:51.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.653+0000 7f70c5f9d640 1 -- 172.21.15.45:0/1053559975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c00693f0 con 0x7f70c00d1970 2024-05-08T05:41:51.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.654+0000 7f70a7fff640 1 -- 172.21.15.45:0/1053559975 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f70b0007400 con 0x7f70c00d1970 2024-05-08T05:41:51.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.654+0000 7f70a7fff640 1 --2- 172.21.15.45:0/1053559975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f70ac076300 0x7f70ac0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:51.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.654+0000 7f70a7fff640 1 -- 172.21.15.45:0/1053559975 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f70b0096d80 con 0x7f70c00d1970 2024-05-08T05:41:51.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.654+0000 7f70beffd640 1 --2- 172.21.15.45:0/1053559975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f70ac076300 0x7f70ac0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:51.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.655+0000 7f70beffd640 1 --2- 172.21.15.45:0/1053559975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f70ac076300 0x7f70ac0787c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f70b4034000 tx=0x7f70b402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:51.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.658+0000 7f70a7fff640 1 -- 172.21.15.45:0/1053559975 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f70c01630b0 con 0x7f70c00d1970 2024-05-08T05:41:51.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.889+0000 7f70c5f9d640 1 -- 172.21.15.45:0/1053559975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70c0162c60 con 0x7f70c00d1970 2024-05-08T05:41:51.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.890+0000 7f70a7fff640 1 -- 172.21.15.45:0/1053559975 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5174 (secure 0 0 0) 0x7f70b0068390 con 0x7f70c00d1970 2024-05-08T05:41:51.892 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:51.892 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2346958825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2346958825},{"type":"v1","addr":"172.21.15.45:6835","nonce":2346958825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/709809719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":709809719},{"type":"v1","addr":"172.21.15.45:6833","nonce":709809719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3208180302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3208180302},{"type":"v1","addr":"172.21.15.63:6836","nonce":3208180302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:41:31.494380+0000","modified":"2024-05-08T05:41:44.544082+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6837/3653698989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":3653698989},{"type":"v1","addr":"172.21.15.63:6837","nonce":3653698989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4303,"qdb_cluster":[4303]},"id":1}]} 2024-05-08T05:41:51.892 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-05-08T05:41:51.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.892+0000 7f70c5f9d640 1 -- 172.21.15.45:0/1053559975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f70ac076300 msgr2=0x7f70ac0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:51.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.892+0000 7f70c5f9d640 1 --2- 172.21.15.45:0/1053559975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f70ac076300 0x7f70ac0787c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f70b4034000 tx=0x7f70b402e000 comp rx=0 tx=0).stop 2024-05-08T05:41:51.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.892+0000 7f70c5f9d640 1 -- 172.21.15.45:0/1053559975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c00d1970 msgr2=0x7f70c01f5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:51.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.892+0000 7f70c5f9d640 1 --2- 172.21.15.45:0/1053559975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c00d1970 0x7f70c01f5300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f70b000c3c0 tx=0x7f70b000c880 comp rx=0 tx=0).stop 2024-05-08T05:41:51.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.892+0000 7f70c5f9d640 1 -- 172.21.15.45:0/1053559975 shutdown_connections 2024-05-08T05:41:51.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.892+0000 7f70c5f9d640 1 --2- 172.21.15.45:0/1053559975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f70ac076300 0x7f70ac0787c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.892+0000 7f70c5f9d640 1 --2- 172.21.15.45:0/1053559975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70c00d4cc0 0x7f70c00ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.892+0000 7f70c5f9d640 1 --2- 172.21.15.45:0/1053559975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70c00b21b0 0x7f70c00b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.892+0000 7f70c5f9d640 1 --2- 172.21.15.45:0/1053559975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70c00d1970 0x7f70c01f5300 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:51.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.892+0000 7f70c5f9d640 1 -- 172.21.15.45:0/1053559975 >> 172.21.15.45:0/1053559975 conn(0x7f70c008eac0 msgr2=0x7f70c006d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:51.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.892+0000 7f70c5f9d640 1 -- 172.21.15.45:0/1053559975 shutdown_connections 2024-05-08T05:41:51.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:51.892+0000 7f70c5f9d640 1 -- 172.21.15.45:0/1053559975 wait complete. 2024-05-08T05:41:51.905 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:41:52.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.007+0000 7f2b15c59640 1 Processor -- start 2024-05-08T05:41:52.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.007+0000 7f2b15c59640 1 -- start start 2024-05-08T05:41:52.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.008+0000 7f2b15c59640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b100d3cd0 0x7f2b100cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:52.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.008+0000 7f2b15c59640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2b100c9ee0 0x7f2b100d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:52.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.008+0000 7f2b15c59640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b100d3330 con 0x7f2b101629a0 2024-05-08T05:41:52.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.008+0000 7f2b15c59640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2b100d2170 con 0x7f2b100c9ee0 2024-05-08T05:41:52.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.008+0000 7f2b15c59640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b100d2450 con 0x7f2b100d3cd0 2024-05-08T05:41:52.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.008+0000 7f2b0f7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2b101629a0 0x7f2b100d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49650/0 (socket says 172.21.15.45:49650) 2024-05-08T05:41:52.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.008+0000 7f2b0f7fe640 1 -- 172.21.15.45:0/3634033228 learned_addr learned my addr 172.21.15.45:0/3634033228 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:52.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.008+0000 7f2b0effd640 1 --2- 172.21.15.45:0/3634033228 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b100d3cd0 0x7f2b100cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:52.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.008+0000 7f2b0ffff640 1 --2- 172.21.15.45:0/3634033228 >> v2:172.21.15.63:3300/0 conn(0x7f2b100c9ee0 0x7f2b100d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:52.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.008+0000 7f2b0e7fc640 1 -- 172.21.15.45:0/3634033228 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1984343512 0 0) 0x7f2b100d3330 con 0x7f2b101629a0 2024-05-08T05:41:52.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.008+0000 7f2b0e7fc640 1 -- 172.21.15.45:0/3634033228 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2af8003520 con 0x7f2b101629a0 2024-05-08T05:41:52.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.009+0000 7f2b0ffff640 1 -- 172.21.15.45:0/3634033228 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b100d3cd0 msgr2=0x7f2b100cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:52.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.009+0000 7f2b0ffff640 1 --2- 172.21.15.45:0/3634033228 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b100d3cd0 0x7f2b100cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.009+0000 7f2b0e7fc640 1 -- 172.21.15.45:0/3634033228 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3478335437 0 0) 0x7f2af8003520 con 0x7f2b101629a0 2024-05-08T05:41:52.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.009+0000 7f2b0ffff640 1 -- 172.21.15.45:0/3634033228 >> v1:172.21.15.45:6789/0 conn(0x7f2b101629a0 legacy=0x7f2b100d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:52.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.009+0000 7f2b0ffff640 1 -- 172.21.15.45:0/3634033228 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b100d2730 con 0x7f2b100c9ee0 2024-05-08T05:41:52.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.009+0000 7f2b0ffff640 1 --2- 172.21.15.45:0/3634033228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b100c9ee0 0x7f2b100d01e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009d90 tx=0x7f2b0802a4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32a1cb3c6dc56c39 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:52.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.010+0000 7f2b0e7fc640 1 -- 172.21.15.45:0/3634033228 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b000016f0 con 0x7f2b100c9ee0 2024-05-08T05:41:52.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.010+0000 7f2b0e7fc640 1 -- 172.21.15.45:0/3634033228 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b08007540 con 0x7f2b100c9ee0 2024-05-08T05:41:52.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.010+0000 7f2b0e7fc640 1 -- 172.21.15.45:0/3634033228 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b080078f0 con 0x7f2b100c9ee0 2024-05-08T05:41:52.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.010+0000 7f2b15c59640 1 -- 172.21.15.45:0/3634033228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b100c9ee0 msgr2=0x7f2b100d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:52.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.010+0000 7f2b15c59640 1 --2- 172.21.15.45:0/3634033228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b100c9ee0 0x7f2b100d01e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009d90 tx=0x7f2b0802a4d0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.010+0000 7f2b15c59640 1 -- 172.21.15.45:0/3634033228 shutdown_connections 2024-05-08T05:41:52.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.010+0000 7f2b15c59640 1 --2- 172.21.15.45:0/3634033228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b100c9ee0 0x7f2b100d01e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.010+0000 7f2b15c59640 1 --2- 172.21.15.45:0/3634033228 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b100d3cd0 0x7f2b100cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.010+0000 7f2b15c59640 1 -- 172.21.15.45:0/3634033228 >> 172.21.15.45:0/3634033228 conn(0x7f2b100d4fe0 msgr2=0x7f2b100d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:52.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.010+0000 7f2b15c59640 1 -- 172.21.15.45:0/3634033228 shutdown_connections 2024-05-08T05:41:52.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.010+0000 7f2b15c59640 1 -- 172.21.15.45:0/3634033228 wait complete. 2024-05-08T05:41:52.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.011+0000 7f2b15c59640 1 Processor -- start 2024-05-08T05:41:52.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.012+0000 7f2b15c59640 1 -- start start 2024-05-08T05:41:52.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.012+0000 7f2b15c59640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b100c9ee0 0x7f2b1015ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:52.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.012+0000 7f2b15c59640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b100d3cd0 0x7f2b100ce400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:52.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.012+0000 7f2b15c59640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b101629a0 0x7f2b1006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:52.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.012+0000 7f2b15c59640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2b100cf440 con 0x7f2b101629a0 2024-05-08T05:41:52.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.012+0000 7f2b15c59640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b100cf2c0 con 0x7f2b100c9ee0 2024-05-08T05:41:52.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.012+0000 7f2b15c59640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b100d2dc0 con 0x7f2b100d3cd0 2024-05-08T05:41:52.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.012+0000 7f2b0ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b101629a0 0x7f2b1006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:52.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.012+0000 7f2b0ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b101629a0 0x7f2b1006e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34706/0 (socket says 172.21.15.45:34706) 2024-05-08T05:41:52.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.012+0000 7f2b0ffff640 1 -- 172.21.15.45:0/3862760666 learned_addr learned my addr 172.21.15.45:0/3862760666 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:52.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.013+0000 7f2b0f7fe640 1 --2- 172.21.15.45:0/3862760666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b100c9ee0 0x7f2b1015ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:52.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.013+0000 7f2b0effd640 1 --2- 172.21.15.45:0/3862760666 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b100d3cd0 0x7f2b100ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:52.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.013+0000 7f2b0ffff640 1 -- 172.21.15.45:0/3862760666 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b100d3cd0 msgr2=0x7f2b100ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:52.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.013+0000 7f2b0ffff640 1 --2- 172.21.15.45:0/3862760666 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b100d3cd0 0x7f2b100ce400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.013+0000 7f2b0ffff640 1 -- 172.21.15.45:0/3862760666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b100c9ee0 msgr2=0x7f2b1015ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:52.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.013+0000 7f2b0ffff640 1 --2- 172.21.15.45:0/3862760666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b100c9ee0 0x7f2b1015ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.013+0000 7f2b0ffff640 1 -- 172.21.15.45:0/3862760666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b100ceec0 con 0x7f2b101629a0 2024-05-08T05:41:52.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.013+0000 7f2b0f7fe640 1 --2- 172.21.15.45:0/3862760666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b100c9ee0 0x7f2b1015ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:52.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.013+0000 7f2b0ffff640 1 --2- 172.21.15.45:0/3862760666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b101629a0 0x7f2b1006e160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009eb0 tx=0x7f2b08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:52.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.013+0000 7f2af7fff640 1 -- 172.21.15.45:0/3862760666 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b08040070 con 0x7f2b101629a0 2024-05-08T05:41:52.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.013+0000 7f2af7fff640 1 -- 172.21.15.45:0/3862760666 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b08031c40 con 0x7f2b101629a0 2024-05-08T05:41:52.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.013+0000 7f2b15c59640 1 -- 172.21.15.45:0/3862760666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b101f4fd0 con 0x7f2b101629a0 2024-05-08T05:41:52.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.013+0000 7f2af7fff640 1 -- 172.21.15.45:0/3862760666 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b08009450 con 0x7f2b101629a0 2024-05-08T05:41:52.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.013+0000 7f2b15c59640 1 -- 172.21.15.45:0/3862760666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b101f5590 con 0x7f2b101629a0 2024-05-08T05:41:52.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.014+0000 7f2b15c59640 1 -- 172.21.15.45:0/3862760666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b101f4de0 con 0x7f2b101629a0 2024-05-08T05:41:52.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.015+0000 7f2af7fff640 1 -- 172.21.15.45:0/3862760666 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2b08039040 con 0x7f2b101629a0 2024-05-08T05:41:52.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.016+0000 7f2af7fff640 1 --2- 172.21.15.45:0/3862760666 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2afc0762d0 0x7f2afc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:52.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.016+0000 7f2af7fff640 1 -- 172.21.15.45:0/3862760666 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2b080b8bf0 con 0x7f2b101629a0 2024-05-08T05:41:52.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.016+0000 7f2b0f7fe640 1 --2- 172.21.15.45:0/3862760666 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2afc0762d0 0x7f2afc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:52.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.016+0000 7f2b0f7fe640 1 --2- 172.21.15.45:0/3862760666 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2afc0762d0 0x7f2afc078790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2b000078f0 tx=0x7f2b00009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:52.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.020+0000 7f2af7fff640 1 -- 172.21.15.45:0/3862760666 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2b080822a0 con 0x7f2b101629a0 2024-05-08T05:41:52.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.257+0000 7f2b15c59640 1 -- 172.21.15.45:0/3862760666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b100693f0 con 0x7f2b101629a0 2024-05-08T05:41:52.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.257+0000 7f2af7fff640 1 -- 172.21.15.45:0/3862760666 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5174 (secure 0 0 0) 0x7f2b08081ba0 con 0x7f2b101629a0 2024-05-08T05:41:52.259 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:52.259 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2346958825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2346958825},{"type":"v1","addr":"172.21.15.45:6835","nonce":2346958825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/709809719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":709809719},{"type":"v1","addr":"172.21.15.45:6833","nonce":709809719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3208180302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3208180302},{"type":"v1","addr":"172.21.15.63:6836","nonce":3208180302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:41:31.494380+0000","modified":"2024-05-08T05:41:44.544082+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6837/3653698989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":3653698989},{"type":"v1","addr":"172.21.15.63:6837","nonce":3653698989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4303,"qdb_cluster":[4303]},"id":1}]} 2024-05-08T05:41:52.259 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-05-08T05:41:52.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.260+0000 7f2b15c59640 1 -- 172.21.15.45:0/3862760666 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2afc0762d0 msgr2=0x7f2afc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:52.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.260+0000 7f2b15c59640 1 --2- 172.21.15.45:0/3862760666 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2afc0762d0 0x7f2afc078790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2b000078f0 tx=0x7f2b00009040 comp rx=0 tx=0).stop 2024-05-08T05:41:52.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.260+0000 7f2b15c59640 1 -- 172.21.15.45:0/3862760666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b101629a0 msgr2=0x7f2b1006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:52.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.260+0000 7f2b15c59640 1 --2- 172.21.15.45:0/3862760666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b101629a0 0x7f2b1006e160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2b08009eb0 tx=0x7f2b08004a40 comp rx=0 tx=0).stop 2024-05-08T05:41:52.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.260+0000 7f2b15c59640 1 -- 172.21.15.45:0/3862760666 shutdown_connections 2024-05-08T05:41:52.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.260+0000 7f2b15c59640 1 --2- 172.21.15.45:0/3862760666 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2afc0762d0 0x7f2afc078790 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.260+0000 7f2b15c59640 1 --2- 172.21.15.45:0/3862760666 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b100d3cd0 0x7f2b100ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.260+0000 7f2b15c59640 1 --2- 172.21.15.45:0/3862760666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b101629a0 0x7f2b1006e160 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.260+0000 7f2b15c59640 1 --2- 172.21.15.45:0/3862760666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b100c9ee0 0x7f2b1015ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.260+0000 7f2b15c59640 1 -- 172.21.15.45:0/3862760666 >> 172.21.15.45:0/3862760666 conn(0x7f2b100d4fe0 msgr2=0x7f2b100b72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:52.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.260+0000 7f2b15c59640 1 -- 172.21.15.45:0/3862760666 shutdown_connections 2024-05-08T05:41:52.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.260+0000 7f2b15c59640 1 -- 172.21.15.45:0/3862760666 wait complete. 2024-05-08T05:41:52.273 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:41:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.375+0000 7f2e10e09640 1 Processor -- start 2024-05-08T05:41:52.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.375+0000 7f2e10e09640 1 -- start start 2024-05-08T05:41:52.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.376+0000 7f2e10e09640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2e0c06d620 0x7f2e0c073800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:52.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.376+0000 7f2e10e09640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e0c06da00 0x7f2e0c0af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:52.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.376+0000 7f2e10e09640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e0c0af800 con 0x7f2e0c0b5ee0 2024-05-08T05:41:52.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.376+0000 7f2e10e09640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f2e0c0afaa0 con 0x7f2e0c06d620 2024-05-08T05:41:52.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.376+0000 7f2e10e09640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2e0c0b5a50 con 0x7f2e0c06da00 2024-05-08T05:41:52.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.376+0000 7f2e0a59c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2e0c06d620 0x7f2e0c073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:52.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.376+0000 7f2e0a59c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2e0c06d620 0x7f2e0c073800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34722/0 (socket says 172.21.15.45:34722) 2024-05-08T05:41:52.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.376+0000 7f2e0a59c640 1 -- 172.21.15.45:0/4070020209 learned_addr learned my addr 172.21.15.45:0/4070020209 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:52.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.376+0000 7f2e09d9b640 1 --2- 172.21.15.45:0/4070020209 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e0c06da00 0x7f2e0c0af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:52.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.377+0000 7f2e0a59c640 1 -- 172.21.15.45:0/4070020209 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e0c06da00 msgr2=0x7f2e0c0af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:52.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.377+0000 7f2e0a59c640 1 --2- 172.21.15.45:0/4070020209 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e0c06da00 0x7f2e0c0af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.377+0000 7f2e0a59c640 1 -- 172.21.15.45:0/4070020209 >> v1:172.21.15.45:6789/0 conn(0x7f2e0c0b5ee0 legacy=0x7f2e0c06e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:52.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.377+0000 7f2e0a59c640 1 -- 172.21.15.45:0/4070020209 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0c0b5120 con 0x7f2e0c06d620 2024-05-08T05:41:52.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.377+0000 7f2e0a59c640 1 --2- 172.21.15.45:0/4070020209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e0c06d620 0x7f2e0c073800 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009e50 tx=0x7f2dfc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a09508bcae948d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:52.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.377+0000 7f2e0959a640 1 -- 172.21.15.45:0/4070020209 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc036070 con 0x7f2e0c06d620 2024-05-08T05:41:52.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.377+0000 7f2e0959a640 1 -- 172.21.15.45:0/4070020209 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dfc031600 con 0x7f2e0c06d620 2024-05-08T05:41:52.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.377+0000 7f2e0959a640 1 -- 172.21.15.45:0/4070020209 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc0319d0 con 0x7f2e0c06d620 2024-05-08T05:41:52.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.378+0000 7f2e10e09640 1 -- 172.21.15.45:0/4070020209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e0c06d620 msgr2=0x7f2e0c073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:52.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.378+0000 7f2e10e09640 1 --2- 172.21.15.45:0/4070020209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e0c06d620 0x7f2e0c073800 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009e50 tx=0x7f2dfc02a5f0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.378+0000 7f2e10e09640 1 -- 172.21.15.45:0/4070020209 shutdown_connections 2024-05-08T05:41:52.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.378+0000 7f2e10e09640 1 --2- 172.21.15.45:0/4070020209 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e0c06da00 0x7f2e0c0af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.378+0000 7f2e10e09640 1 --2- 172.21.15.45:0/4070020209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e0c06d620 0x7f2e0c073800 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.378+0000 7f2e10e09640 1 -- 172.21.15.45:0/4070020209 >> 172.21.15.45:0/4070020209 conn(0x7f2e0c0728c0 msgr2=0x7f2e0c072cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:52.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.378+0000 7f2e10e09640 1 -- 172.21.15.45:0/4070020209 shutdown_connections 2024-05-08T05:41:52.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.378+0000 7f2e10e09640 1 -- 172.21.15.45:0/4070020209 wait complete. 2024-05-08T05:41:52.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.380+0000 7f2e10e09640 1 Processor -- start 2024-05-08T05:41:52.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.380+0000 7f2e10e09640 1 -- start start 2024-05-08T05:41:52.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.381+0000 7f2e10e09640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e0c06d620 0x7f2e0c0cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.381+0000 7f2e10e09640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e0c06da00 0x7f2e0c0d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.381+0000 7f2e10e09640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e0c0b5ee0 0x7f2e0c0aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.381+0000 7f2e10e09640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c06f030 con 0x7f2e0c0b5ee0 2024-05-08T05:41:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.381+0000 7f2e10e09640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c0b56e0 con 0x7f2e0c06d620 2024-05-08T05:41:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.381+0000 7f2e10e09640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2e0c06f1b0 con 0x7f2e0c06da00 2024-05-08T05:41:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.381+0000 7f2e0ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e0c0b5ee0 0x7f2e0c0aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.381+0000 7f2e0ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e0c0b5ee0 0x7f2e0c0aa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34732/0 (socket says 172.21.15.45:34732) 2024-05-08T05:41:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.381+0000 7f2e0a59c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e0c06d620 0x7f2e0c0cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.381+0000 7f2e0ad9d640 1 -- 172.21.15.45:0/2779869497 learned_addr learned my addr 172.21.15.45:0/2779869497 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.381+0000 7f2e09d9b640 1 --2- 172.21.15.45:0/2779869497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e0c06da00 0x7f2e0c0d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.381+0000 7f2e0ad9d640 1 -- 172.21.15.45:0/2779869497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e0c06da00 msgr2=0x7f2e0c0d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.381+0000 7f2e0ad9d640 1 --2- 172.21.15.45:0/2779869497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e0c06da00 0x7f2e0c0d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.381+0000 7f2e0ad9d640 1 -- 172.21.15.45:0/2779869497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e0c06d620 msgr2=0x7f2e0c0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.381+0000 7f2e0ad9d640 1 --2- 172.21.15.45:0/2779869497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e0c06d620 0x7f2e0c0cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.381+0000 7f2e0ad9d640 1 -- 172.21.15.45:0/2779869497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0c0ce740 con 0x7f2e0c0b5ee0 2024-05-08T05:41:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.382+0000 7f2e0ad9d640 1 --2- 172.21.15.45:0/2779869497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e0c0b5ee0 0x7f2e0c0aa4a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400c400 tx=0x7f2e0400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.382+0000 7f2e02ffd640 1 -- 172.21.15.45:0/2779869497 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e04017070 con 0x7f2e0c0b5ee0 2024-05-08T05:41:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.382+0000 7f2e02ffd640 1 -- 172.21.15.45:0/2779869497 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e040052d0 con 0x7f2e0c0b5ee0 2024-05-08T05:41:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.382+0000 7f2e10e09640 1 -- 172.21.15.45:0/2779869497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e0c0cea20 con 0x7f2e0c0b5ee0 2024-05-08T05:41:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.382+0000 7f2e02ffd640 1 -- 172.21.15.45:0/2779869497 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e040037f0 con 0x7f2e0c0b5ee0 2024-05-08T05:41:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.382+0000 7f2e10e09640 1 -- 172.21.15.45:0/2779869497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e0c0cefe0 con 0x7f2e0c0b5ee0 2024-05-08T05:41:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.383+0000 7f2e10e09640 1 -- 172.21.15.45:0/2779869497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e0c0693f0 con 0x7f2e0c0b5ee0 2024-05-08T05:41:52.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.384+0000 7f2e02ffd640 1 -- 172.21.15.45:0/2779869497 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2e0401a070 con 0x7f2e0c0b5ee0 2024-05-08T05:41:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.385+0000 7f2e02ffd640 1 --2- 172.21.15.45:0/2779869497 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2dec0762d0 0x7f2dec078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.385+0000 7f2e02ffd640 1 -- 172.21.15.45:0/2779869497 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f2e04096b90 con 0x7f2e0c0b5ee0 2024-05-08T05:41:52.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.385+0000 7f2e0a59c640 1 --2- 172.21.15.45:0/2779869497 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2dec0762d0 0x7f2dec078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:52.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.385+0000 7f2e0a59c640 1 --2- 172.21.15.45:0/2779869497 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2dec0762d0 0x7f2dec078790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc033000 tx=0x7f2dfc009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:52.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.389+0000 7f2e02ffd640 1 -- 172.21.15.45:0/2779869497 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2e0405fb60 con 0x7f2e0c0b5ee0 2024-05-08T05:41:52.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.586+0000 7f2e10e09640 1 -- 172.21.15.45:0/2779869497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e0c068ba0 con 0x7f2e0c0b5ee0 2024-05-08T05:41:52.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.588+0000 7f2e02ffd640 1 -- 172.21.15.45:0/2779869497 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18466 (secure 0 0 0) 0x7f2e040681a0 con 0x7f2e0c0b5ee0 2024-05-08T05:41:52.589 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:52.589 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:41:36.532045+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:41:29.381274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:41:30.443612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:41:34.132285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:41:52.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.589+0000 7f2e10e09640 1 -- 172.21.15.45:0/2779869497 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2dec0762d0 msgr2=0x7f2dec078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:52.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.589+0000 7f2e10e09640 1 --2- 172.21.15.45:0/2779869497 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2dec0762d0 0x7f2dec078790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc033000 tx=0x7f2dfc009000 comp rx=0 tx=0).stop 2024-05-08T05:41:52.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.589+0000 7f2e10e09640 1 -- 172.21.15.45:0/2779869497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e0c0b5ee0 msgr2=0x7f2e0c0aa4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:52.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.589+0000 7f2e10e09640 1 --2- 172.21.15.45:0/2779869497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e0c0b5ee0 0x7f2e0c0aa4a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400c400 tx=0x7f2e0400c8c0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.589+0000 7f2e10e09640 1 -- 172.21.15.45:0/2779869497 shutdown_connections 2024-05-08T05:41:52.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.589+0000 7f2e10e09640 1 --2- 172.21.15.45:0/2779869497 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2dec0762d0 0x7f2dec078790 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.589+0000 7f2e10e09640 1 --2- 172.21.15.45:0/2779869497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e0c06da00 0x7f2e0c0d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.589+0000 7f2e10e09640 1 --2- 172.21.15.45:0/2779869497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e0c0b5ee0 0x7f2e0c0aa4a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.589+0000 7f2e10e09640 1 --2- 172.21.15.45:0/2779869497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e0c06d620 0x7f2e0c0cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.589+0000 7f2e10e09640 1 -- 172.21.15.45:0/2779869497 >> 172.21.15.45:0/2779869497 conn(0x7f2e0c0728c0 msgr2=0x7f2e0c15e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:52.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.590+0000 7f2e10e09640 1 -- 172.21.15.45:0/2779869497 shutdown_connections 2024-05-08T05:41:52.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.590+0000 7f2e10e09640 1 -- 172.21.15.45:0/2779869497 wait complete. 2024-05-08T05:41:52.602 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:41:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.704+0000 7f66c4132640 1 Processor -- start 2024-05-08T05:41:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.705+0000 7f66c4132640 1 -- start start 2024-05-08T05:41:52.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.705+0000 7f66c4132640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f66bc0b1050 0x7f66bc071720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:52.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.705+0000 7f66c4132640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f66bc06c3a0 0x7f66bc06c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:52.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.705+0000 7f66c4132640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66bc15fae0 con 0x7f66bc15f680 2024-05-08T05:41:52.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.705+0000 7f66c4132640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f66bc06cd10 con 0x7f66bc0b1050 2024-05-08T05:41:52.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.705+0000 7f66c4132640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f66bc160600 con 0x7f66bc06c3a0 2024-05-08T05:41:52.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.705+0000 7f66c1ece640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f66bc0b1050 0x7f66bc071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:52.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.706+0000 7f66c16cd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f66bc15f680 0x7f66bc0b56b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49670/0 (socket says 172.21.15.45:49670) 2024-05-08T05:41:52.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.706+0000 7f66c16cd640 1 -- 172.21.15.45:0/1656162759 learned_addr learned my addr 172.21.15.45:0/1656162759 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:52.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.706+0000 7f66c26cf640 1 --2- 172.21.15.45:0/1656162759 >> v2:172.21.15.63:3300/0 conn(0x7f66bc06c3a0 0x7f66bc06c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:52.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.706+0000 7f66c1ece640 1 -- 172.21.15.45:0/1656162759 >> v2:172.21.15.63:3300/0 conn(0x7f66bc06c3a0 msgr2=0x7f66bc06c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:52.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.706+0000 7f66c1ece640 1 --2- 172.21.15.45:0/1656162759 >> v2:172.21.15.63:3300/0 conn(0x7f66bc06c3a0 0x7f66bc06c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.706+0000 7f66c1ece640 1 -- 172.21.15.45:0/1656162759 >> v1:172.21.15.45:6789/0 conn(0x7f66bc15f680 legacy=0x7f66bc0b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:52.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.706+0000 7f66c1ece640 1 -- 172.21.15.45:0/1656162759 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc15fd60 con 0x7f66bc0b1050 2024-05-08T05:41:52.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.706+0000 7f66c0ecc640 1 -- 172.21.15.45:0/1656162759 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2297839093 0 0) 0x7f66b0001110 con 0x7f66bc15f680 2024-05-08T05:41:52.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.707+0000 7f66c1ece640 1 --2- 172.21.15.45:0/1656162759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc0b1050 0x7f66bc071720 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0b55c0 tx=0x7f66b402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17a64104e97cf73 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:52.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.707+0000 7f66c0ecc640 1 -- 172.21.15.45:0/1656162759 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b4037070 con 0x7f66bc0b1050 2024-05-08T05:41:52.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.707+0000 7f66c0ecc640 1 -- 172.21.15.45:0/1656162759 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b4032440 con 0x7f66bc0b1050 2024-05-08T05:41:52.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.707+0000 7f66c0ecc640 1 -- 172.21.15.45:0/1656162759 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b4032810 con 0x7f66bc0b1050 2024-05-08T05:41:52.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.707+0000 7f66c4132640 1 -- 172.21.15.45:0/1656162759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc0b1050 msgr2=0x7f66bc071720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:52.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.707+0000 7f66c4132640 1 --2- 172.21.15.45:0/1656162759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc0b1050 0x7f66bc071720 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0b55c0 tx=0x7f66b402a1a0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.707+0000 7f66c4132640 1 -- 172.21.15.45:0/1656162759 shutdown_connections 2024-05-08T05:41:52.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.707+0000 7f66c4132640 1 --2- 172.21.15.45:0/1656162759 >> v2:172.21.15.63:3300/0 conn(0x7f66bc06c3a0 0x7f66bc06c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.707+0000 7f66c4132640 1 --2- 172.21.15.45:0/1656162759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc0b1050 0x7f66bc071720 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.707+0000 7f66c4132640 1 -- 172.21.15.45:0/1656162759 >> 172.21.15.45:0/1656162759 conn(0x7f66bc0b3c40 msgr2=0x7f66bc0b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:52.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.707+0000 7f66c4132640 1 -- 172.21.15.45:0/1656162759 shutdown_connections 2024-05-08T05:41:52.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.707+0000 7f66c4132640 1 -- 172.21.15.45:0/1656162759 wait complete. 2024-05-08T05:41:52.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.709+0000 7f66c4132640 1 Processor -- start 2024-05-08T05:41:52.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.709+0000 7f66c4132640 1 -- start start 2024-05-08T05:41:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.709+0000 7f66c4132640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66bc06c3a0 0x7f66bc152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.709+0000 7f66c4132640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f66bc0b1050 0x7f66bc13f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.709+0000 7f66c4132640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc15f680 0x7f66bc157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.709+0000 7f66c4132640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc162d10 con 0x7f66bc15f680 2024-05-08T05:41:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.709+0000 7f66c4132640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc160320 con 0x7f66bc06c3a0 2024-05-08T05:41:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.709+0000 7f66c4132640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f66bc0b5dc0 con 0x7f66bc0b1050 2024-05-08T05:41:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.709+0000 7f66c26cf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc15f680 0x7f66bc157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.710+0000 7f66c26cf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc15f680 0x7f66bc157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34750/0 (socket says 172.21.15.45:34750) 2024-05-08T05:41:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.710+0000 7f66c26cf640 1 -- 172.21.15.45:0/3362007950 learned_addr learned my addr 172.21.15.45:0/3362007950 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.710+0000 7f66c16cd640 1 --2- 172.21.15.45:0/3362007950 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f66bc0b1050 0x7f66bc13f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.710+0000 7f66c1ece640 1 --2- 172.21.15.45:0/3362007950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66bc06c3a0 0x7f66bc152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:52.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.710+0000 7f66c26cf640 1 -- 172.21.15.45:0/3362007950 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f66bc0b1050 msgr2=0x7f66bc13f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:52.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.710+0000 7f66c26cf640 1 --2- 172.21.15.45:0/3362007950 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f66bc0b1050 0x7f66bc13f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.710+0000 7f66c26cf640 1 -- 172.21.15.45:0/3362007950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66bc06c3a0 msgr2=0x7f66bc152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:52.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.710+0000 7f66c26cf640 1 --2- 172.21.15.45:0/3362007950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66bc06c3a0 0x7f66bc152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.710+0000 7f66c26cf640 1 -- 172.21.15.45:0/3362007950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc1533d0 con 0x7f66bc15f680 2024-05-08T05:41:52.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.710+0000 7f66c1ece640 1 --2- 172.21.15.45:0/3362007950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66bc06c3a0 0x7f66bc152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:52.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.710+0000 7f66c26cf640 1 --2- 172.21.15.45:0/3362007950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc15f680 0x7f66bc157500 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009d30 tx=0x7f66ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:52.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.710+0000 7f66ba7fc640 1 -- 172.21.15.45:0/3362007950 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac018020 con 0x7f66bc15f680 2024-05-08T05:41:52.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.710+0000 7f66ba7fc640 1 -- 172.21.15.45:0/3362007950 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66ac014480 con 0x7f66bc15f680 2024-05-08T05:41:52.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.710+0000 7f66c4132640 1 -- 172.21.15.45:0/3362007950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66bc14cd10 con 0x7f66bc15f680 2024-05-08T05:41:52.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.710+0000 7f66ba7fc640 1 -- 172.21.15.45:0/3362007950 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac014870 con 0x7f66bc15f680 2024-05-08T05:41:52.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.710+0000 7f66c4132640 1 -- 172.21.15.45:0/3362007950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66bc14d2d0 con 0x7f66bc15f680 2024-05-08T05:41:52.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.711+0000 7f66c4132640 1 -- 172.21.15.45:0/3362007950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66bc071020 con 0x7f66bc15f680 2024-05-08T05:41:52.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.712+0000 7f66ba7fc640 1 -- 172.21.15.45:0/3362007950 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f66ac005d10 con 0x7f66bc15f680 2024-05-08T05:41:52.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.712+0000 7f66ba7fc640 1 --2- 172.21.15.45:0/3362007950 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f66a80762d0 0x7f66a8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:52.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.713+0000 7f66ba7fc640 1 -- 172.21.15.45:0/3362007950 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f66ac0986a0 con 0x7f66bc15f680 2024-05-08T05:41:52.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.713+0000 7f66c1ece640 1 --2- 172.21.15.45:0/3362007950 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f66a80762d0 0x7f66a8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:52.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.713+0000 7f66c1ece640 1 --2- 172.21.15.45:0/3362007950 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f66a80762d0 0x7f66a8078790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f66b4034000 tx=0x7f66b402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:52.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.717+0000 7f66ba7fc640 1 -- 172.21.15.45:0/3362007950 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f66bc071020 con 0x7f66bc15f680 2024-05-08T05:41:52.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.946+0000 7f66c4132640 1 -- 172.21.15.45:0/3362007950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66bc069b00 con 0x7f66bc15f680 2024-05-08T05:41:52.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.946+0000 7f66ba7fc640 1 -- 172.21.15.45:0/3362007950 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5174 (secure 0 0 0) 0x7f66ac022030 con 0x7f66bc15f680 2024-05-08T05:41:52.948 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:41:52.949 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2346958825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2346958825},{"type":"v1","addr":"172.21.15.45:6835","nonce":2346958825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/709809719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":709809719},{"type":"v1","addr":"172.21.15.45:6833","nonce":709809719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3208180302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3208180302},{"type":"v1","addr":"172.21.15.63:6836","nonce":3208180302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:41:31.494380+0000","modified":"2024-05-08T05:41:44.544082+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6837/3653698989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":3653698989},{"type":"v1","addr":"172.21.15.63:6837","nonce":3653698989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4303,"qdb_cluster":[4303]},"id":1}]} 2024-05-08T05:41:52.950 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-05-08T05:41:52.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.948+0000 7f66c4132640 1 -- 172.21.15.45:0/3362007950 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f66a80762d0 msgr2=0x7f66a8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:52.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.948+0000 7f66c4132640 1 --2- 172.21.15.45:0/3362007950 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f66a80762d0 0x7f66a8078790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f66b4034000 tx=0x7f66b402e000 comp rx=0 tx=0).stop 2024-05-08T05:41:52.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.948+0000 7f66c4132640 1 -- 172.21.15.45:0/3362007950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc15f680 msgr2=0x7f66bc157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:52.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.948+0000 7f66c4132640 1 --2- 172.21.15.45:0/3362007950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc15f680 0x7f66bc157500 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009d30 tx=0x7f66ac00c490 comp rx=0 tx=0).stop 2024-05-08T05:41:52.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.948+0000 7f66c4132640 1 -- 172.21.15.45:0/3362007950 shutdown_connections 2024-05-08T05:41:52.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.948+0000 7f66c4132640 1 --2- 172.21.15.45:0/3362007950 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f66a80762d0 0x7f66a8078790 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.948+0000 7f66c4132640 1 --2- 172.21.15.45:0/3362007950 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f66bc0b1050 0x7f66bc13f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.948+0000 7f66c4132640 1 --2- 172.21.15.45:0/3362007950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc15f680 0x7f66bc157500 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.948+0000 7f66c4132640 1 --2- 172.21.15.45:0/3362007950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66bc06c3a0 0x7f66bc152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:52.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.948+0000 7f66c4132640 1 -- 172.21.15.45:0/3362007950 >> 172.21.15.45:0/3362007950 conn(0x7f66bc0b3c40 msgr2=0x7f66bc161eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:52.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.948+0000 7f66c4132640 1 -- 172.21.15.45:0/3362007950 shutdown_connections 2024-05-08T05:41:52.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:52.948+0000 7f66c4132640 1 -- 172.21.15.45:0/3362007950 wait complete. 2024-05-08T05:41:53.363 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:41:31.494380+0000', 'modified': '2024-05-08T05:41:44.544082+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4303}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4303': {'gid': 4303, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.63:6837/3653698989', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6835', 'nonce': 3653698989}, {'type': 'v1', 'addr': '172.21.15.63:6837', 'nonce': 3653698989}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4303, 'qdb_cluster': [4303]} 2024-05-08T05:41:53.364 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T05:41:53.364 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T05:41:53.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.469+0000 7fb75d6cf640 1 Processor -- start 2024-05-08T05:41:53.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.469+0000 7fb75d6cf640 1 -- start start 2024-05-08T05:41:53.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.470+0000 7fb75d6cf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb758070d30 0x7fb7580d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.470+0000 7fb75d6cf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb758071110 0x7fb75806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.470+0000 7fb75d6cf640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7580d1300 con 0x7fb758163920 2024-05-08T05:41:53.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.470+0000 7fb75d6cf640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb7580d10d0 con 0x7fb758070d30 2024-05-08T05:41:53.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.470+0000 7fb75d6cf640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb758163db0 con 0x7fb758071110 2024-05-08T05:41:53.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.470+0000 7fb756ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb758070d30 0x7fb7580d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.470+0000 7fb7567fc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb758163920 0x7fb7580703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49676/0 (socket says 172.21.15.45:49676) 2024-05-08T05:41:53.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.470+0000 7fb756ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb758070d30 0x7fb7580d20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34760/0 (socket says 172.21.15.45:34760) 2024-05-08T05:41:53.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.470+0000 7fb7567fc640 1 -- 172.21.15.45:0/2772497459 learned_addr learned my addr 172.21.15.45:0/2772497459 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:53.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.470+0000 7fb756ffd640 1 -- 172.21.15.45:0/2772497459 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb758071110 msgr2=0x7fb75806d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:53.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.470+0000 7fb7577fe640 1 --2- 172.21.15.45:0/2772497459 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb758071110 0x7fb75806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.471+0000 7fb755ffb640 1 -- 172.21.15.45:0/2772497459 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3837562356 0 0) 0x7fb7580d1300 con 0x7fb758163920 2024-05-08T05:41:53.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.471+0000 7fb756ffd640 1 --2- 172.21.15.45:0/2772497459 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb758071110 0x7fb75806d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.471+0000 7fb756ffd640 1 -- 172.21.15.45:0/2772497459 >> v1:172.21.15.45:6789/0 conn(0x7fb758163920 legacy=0x7fb7580703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:53.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.471+0000 7fb756ffd640 1 -- 172.21.15.45:0/2772497459 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7580d15e0 con 0x7fb758070d30 2024-05-08T05:41:53.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.471+0000 7fb7577fe640 1 --2- 172.21.15.45:0/2772497459 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb758071110 0x7fb75806d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:53.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.471+0000 7fb756ffd640 1 --2- 172.21.15.45:0/2772497459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb758070d30 0x7fb7580d20c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb7500097a0 tx=0x7fb75002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0867a0bb755dbac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:53.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.471+0000 7fb755ffb640 1 -- 172.21.15.45:0/2772497459 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750037070 con 0x7fb758070d30 2024-05-08T05:41:53.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.471+0000 7fb755ffb640 1 -- 172.21.15.45:0/2772497459 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb750032440 con 0x7fb758070d30 2024-05-08T05:41:53.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.471+0000 7fb755ffb640 1 -- 172.21.15.45:0/2772497459 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750032810 con 0x7fb758070d30 2024-05-08T05:41:53.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.472+0000 7fb75d6cf640 1 -- 172.21.15.45:0/2772497459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb758070d30 msgr2=0x7fb7580d20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:53.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.472+0000 7fb75d6cf640 1 --2- 172.21.15.45:0/2772497459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb758070d30 0x7fb7580d20c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb7500097a0 tx=0x7fb75002a180 comp rx=0 tx=0).stop 2024-05-08T05:41:53.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.472+0000 7fb75d6cf640 1 -- 172.21.15.45:0/2772497459 shutdown_connections 2024-05-08T05:41:53.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.472+0000 7fb75d6cf640 1 --2- 172.21.15.45:0/2772497459 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb758071110 0x7fb75806d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.472+0000 7fb75d6cf640 1 --2- 172.21.15.45:0/2772497459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb758070d30 0x7fb7580d20c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.472+0000 7fb75d6cf640 1 -- 172.21.15.45:0/2772497459 >> 172.21.15.45:0/2772497459 conn(0x7fb7580d2e90 msgr2=0x7fb7580d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:53.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.472+0000 7fb75d6cf640 1 -- 172.21.15.45:0/2772497459 shutdown_connections 2024-05-08T05:41:53.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.472+0000 7fb75d6cf640 1 -- 172.21.15.45:0/2772497459 wait complete. 2024-05-08T05:41:53.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.473+0000 7fb75d6cf640 1 Processor -- start 2024-05-08T05:41:53.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.473+0000 7fb75d6cf640 1 -- start start 2024-05-08T05:41:53.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.474+0000 7fb75d6cf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb758070d30 0x7fb7580b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.474+0000 7fb75d6cf640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb758071110 0x7fb7580c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.474+0000 7fb75d6cf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb758163920 0x7fb7580caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.474+0000 7fb75d6cf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb7580d1c30 con 0x7fb758163920 2024-05-08T05:41:53.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.474+0000 7fb75d6cf640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb758070b80 con 0x7fb758071110 2024-05-08T05:41:53.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.474+0000 7fb75d6cf640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb7580715a0 con 0x7fb758070d30 2024-05-08T05:41:53.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.474+0000 7fb7577fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb758163920 0x7fb7580caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.474+0000 7fb7577fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb758163920 0x7fb7580caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34762/0 (socket says 172.21.15.45:34762) 2024-05-08T05:41:53.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.474+0000 7fb7577fe640 1 -- 172.21.15.45:0/3228459648 learned_addr learned my addr 172.21.15.45:0/3228459648 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:53.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.474+0000 7fb7567fc640 1 --2- 172.21.15.45:0/3228459648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb758071110 0x7fb7580c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.474+0000 7fb7577fe640 1 -- 172.21.15.45:0/3228459648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb758070d30 msgr2=0x7fb7580b6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:53.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.474+0000 7fb756ffd640 1 --2- 172.21.15.45:0/3228459648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb758070d30 0x7fb7580b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.475+0000 7fb7577fe640 1 --2- 172.21.15.45:0/3228459648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb758070d30 0x7fb7580b6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.475+0000 7fb7577fe640 1 -- 172.21.15.45:0/3228459648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb758071110 msgr2=0x7fb7580c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:53.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.475+0000 7fb7577fe640 1 --2- 172.21.15.45:0/3228459648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb758071110 0x7fb7580c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.475+0000 7fb7577fe640 1 -- 172.21.15.45:0/3228459648 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7580c97e0 con 0x7fb758163920 2024-05-08T05:41:53.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.475+0000 7fb7577fe640 1 --2- 172.21.15.45:0/3228459648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb758163920 0x7fb7580caec0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb74800cd50 tx=0x7fb74800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:53.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.475+0000 7fb74f7fe640 1 -- 172.21.15.45:0/3228459648 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74800f680 con 0x7fb758163920 2024-05-08T05:41:53.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.475+0000 7fb74f7fe640 1 -- 172.21.15.45:0/3228459648 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb74800f840 con 0x7fb758163920 2024-05-08T05:41:53.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.475+0000 7fb74f7fe640 1 -- 172.21.15.45:0/3228459648 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74800fc30 con 0x7fb758163920 2024-05-08T05:41:53.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.475+0000 7fb75d6cf640 1 -- 172.21.15.45:0/3228459648 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7580cb400 con 0x7fb758163920 2024-05-08T05:41:53.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.475+0000 7fb75d6cf640 1 -- 172.21.15.45:0/3228459648 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7580c6dd0 con 0x7fb758163920 2024-05-08T05:41:53.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.477+0000 7fb74f7fe640 1 -- 172.21.15.45:0/3228459648 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb7480213f0 con 0x7fb758163920 2024-05-08T05:41:53.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.477+0000 7fb74f7fe640 1 --2- 172.21.15.45:0/3228459648 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb7380762d0 0x7fb738078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.477+0000 7fb74f7fe640 1 -- 172.21.15.45:0/3228459648 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb7480985e0 con 0x7fb758163920 2024-05-08T05:41:53.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.477+0000 7fb756ffd640 1 --2- 172.21.15.45:0/3228459648 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb7380762d0 0x7fb738078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.478+0000 7fb7603e1740 1 Processor -- start 2024-05-08T05:41:53.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.478+0000 7fb7603e1740 1 -- start start 2024-05-08T05:41:53.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.478+0000 7fb756ffd640 1 --2- 172.21.15.45:0/3228459648 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb7380762d0 0x7fb738078790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb750034000 tx=0x7fb75002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:53.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb7603e1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ebf50 0x5589944ee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb7603e1740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5589944ec330 0x5589944eeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb7603e1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589944bebb0 0x5589944c23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb7603e1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5589944288d0 con 0x5589944ebf50 2024-05-08T05:41:53.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb7603e1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5589943e1c10 con 0x5589944bebb0 2024-05-08T05:41:53.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb7603e1740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x558994424890 con 0x5589944ec330 2024-05-08T05:41:53.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb756ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ebf50 0x5589944ee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb756ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ebf50 0x5589944ee760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34772/0 (socket says 172.21.15.45:34772) 2024-05-08T05:41:53.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb756ffd640 1 -- 172.21.15.45:0/1861910190 learned_addr learned my addr 172.21.15.45:0/1861910190 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:53.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb7577fe640 1 --2- 172.21.15.45:0/1861910190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589944bebb0 0x5589944c23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb7567fc640 1 --2- 172.21.15.45:0/1861910190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5589944ec330 0x5589944eeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb756ffd640 1 -- 172.21.15.45:0/1861910190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5589944ec330 msgr2=0x5589944eeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:53.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb756ffd640 1 --2- 172.21.15.45:0/1861910190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5589944ec330 0x5589944eeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb756ffd640 1 -- 172.21.15.45:0/1861910190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589944bebb0 msgr2=0x5589944c23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:53.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb756ffd640 1 --2- 172.21.15.45:0/1861910190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589944bebb0 0x5589944c23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb756ffd640 1 -- 172.21.15.45:0/1861910190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589943e95d0 con 0x5589944ebf50 2024-05-08T05:41:53.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb7577fe640 1 --2- 172.21.15.45:0/1861910190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589944bebb0 0x5589944c23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:53.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb756ffd640 1 --2- 172.21.15.45:0/1861910190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ebf50 0x5589944ee760 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb750000c00 tx=0x7fb75002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518ebe001e7e4641 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:53.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb74cff9640 1 -- 172.21.15.45:0/1861910190 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75004c070 con 0x5589944ebf50 2024-05-08T05:41:53.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb74cff9640 1 -- 172.21.15.45:0/1861910190 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb750032e20 con 0x5589944ebf50 2024-05-08T05:41:53.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.479+0000 7fb74cff9640 1 -- 172.21.15.45:0/1861910190 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750037040 con 0x5589944ebf50 2024-05-08T05:41:53.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.480+0000 7fb7603e1740 1 -- 172.21.15.45:0/1861910190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ebf50 msgr2=0x5589944ee760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:53.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.480+0000 7fb7603e1740 1 --2- 172.21.15.45:0/1861910190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ebf50 0x5589944ee760 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb750000c00 tx=0x7fb75002c000 comp rx=0 tx=0).stop 2024-05-08T05:41:53.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.480+0000 7fb7603e1740 1 -- 172.21.15.45:0/1861910190 shutdown_connections 2024-05-08T05:41:53.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.480+0000 7fb7603e1740 1 --2- 172.21.15.45:0/1861910190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5589944ec330 0x5589944eeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.480+0000 7fb7603e1740 1 --2- 172.21.15.45:0/1861910190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589944bebb0 0x5589944c23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.480+0000 7fb7603e1740 1 --2- 172.21.15.45:0/1861910190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ebf50 0x5589944ee760 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.480+0000 7fb7603e1740 1 -- 172.21.15.45:0/1861910190 >> 172.21.15.45:0/1861910190 conn(0x5589944c0e20 msgr2=0x558994509760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:53.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.480+0000 7fb7603e1740 1 -- 172.21.15.45:0/1861910190 shutdown_connections 2024-05-08T05:41:53.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.480+0000 7fb7603e1740 1 -- 172.21.15.45:0/1861910190 wait complete. 2024-05-08T05:41:53.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.481+0000 7fb7603e1740 1 -- start start 2024-05-08T05:41:53.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.481+0000 7fb7603e1740 1 Processor -- start 2024-05-08T05:41:53.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb7603e1740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5589944ebf50 0x55899450a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb7603e1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589944ec330 0x5589944f17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb7603e1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55899451caf0 0x5589944f1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb7603e1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5589944c3fd0 con 0x55899451caf0 2024-05-08T05:41:53.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb7603e1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5589943e5760 con 0x5589944ec330 2024-05-08T05:41:53.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb7603e1740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5589944c42b0 con 0x5589944ebf50 2024-05-08T05:41:53.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb7603e1740 10 client.0 _ll_register_callbacks cb 0x55899441ecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:41:53.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb7577fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55899451caf0 0x5589944f1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb7577fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55899451caf0 0x5589944f1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34778/0 (socket says 172.21.15.45:34778) 2024-05-08T05:41:53.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb7577fe640 1 -- 172.21.15.45:0/3750257147 learned_addr learned my addr 172.21.15.45:0/3750257147 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:53.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb756ffd640 1 --2- 172.21.15.45:0/3750257147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5589944ebf50 0x55899450a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb7567fc640 1 --2- 172.21.15.45:0/3750257147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589944ec330 0x5589944f17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb7577fe640 1 -- 172.21.15.45:0/3750257147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5589944ebf50 msgr2=0x55899450a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:53.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb7577fe640 1 --2- 172.21.15.45:0/3750257147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5589944ebf50 0x55899450a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb7577fe640 1 -- 172.21.15.45:0/3750257147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589944ec330 msgr2=0x5589944f17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:53.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb7577fe640 1 --2- 172.21.15.45:0/3750257147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589944ec330 0x5589944f17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb7577fe640 1 -- 172.21.15.45:0/3750257147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55899413a2d0 con 0x55899451caf0 2024-05-08T05:41:53.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb7577fe640 1 --2- 172.21.15.45:0/3750257147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55899451caf0 0x5589944f1d00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb748061520 tx=0x7fb7480611d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0854614797bf1f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:53.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb735ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:41:53.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb735ffb640 1 -- 172.21.15.45:0/3750257147 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748065940 con 0x55899451caf0 2024-05-08T05:41:53.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb735ffb640 1 -- 172.21.15.45:0/3750257147 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb748069ed0 con 0x55899451caf0 2024-05-08T05:41:53.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.482+0000 7fb7603e1740 1 -- 172.21.15.45:0/3750257147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb750032e20 con 0x55899451caf0 2024-05-08T05:41:53.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.483+0000 7fb735ffb640 1 -- 172.21.15.45:0/3750257147 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fb748096050 con 0x55899451caf0 2024-05-08T05:41:53.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.483+0000 7fb7603e1740 10 client.4665 fetch_fsmap learned FSMap version 8 2024-05-08T05:41:53.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.483+0000 7fb7603e1740 1 -- 172.21.15.45:0/3750257147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5589942046b0 con 0x55899451caf0 2024-05-08T05:41:53.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.483+0000 7fb735ffb640 1 -- 172.21.15.45:0/3750257147 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb750032e20 con 0x55899451caf0 2024-05-08T05:41:53.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.483+0000 7fb735ffb640 1 -- 172.21.15.45:0/3750257147 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7fb7480607f0 con 0x55899451caf0 2024-05-08T05:41:53.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.483+0000 7fb7603e1740 10 client.4665 fetch_fsmap finished waiting for FSMap version 8 2024-05-08T05:41:53.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.483+0000 7fb7603e1740 10 client.4665 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T05:41:53.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.483+0000 7fb7603e1740 20 client.4665 populate_metadata read hostname 'smithi045' 2024-05-08T05:41:53.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.483+0000 7fb7603e1740 1 --2- 172.21.15.45:0/3750257147 >> [v2:172.21.15.63:6835/3653698989,v1:172.21.15.63:6837/3653698989] conn(0x55899451a310 0x55899451a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.483+0000 7fb7603e1740 4 client.4665 mds_command: new command op to 4303 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:41:53.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.483+0000 7fb7603e1740 1 -- 172.21.15.45:0/3750257147 --> [v2:172.21.15.63:6835/3653698989,v1:172.21.15.63:6837/3653698989] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589943ac760 con 0x55899451a310 2024-05-08T05:41:53.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.483+0000 7fb756ffd640 1 --2- 172.21.15.45:0/3750257147 >> [v2:172.21.15.63:6835/3653698989,v1:172.21.15.63:6837/3653698989] conn(0x55899451a310 0x55899451a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.484+0000 7fb735ffb640 10 client.4665 ms_handle_connect on v2:172.21.15.63:6835/3653698989 2024-05-08T05:41:53.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.484+0000 7fb756ffd640 1 --2- 172.21.15.45:0/3750257147 >> [v2:172.21.15.63:6835/3653698989,v1:172.21.15.63:6837/3653698989] conn(0x55899451a310 0x55899451a6f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ab67e649b70a270 server_cookie=416449753895647e in_seq=0 out_seq=0 2024-05-08T05:41:53.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.484+0000 7fb735ffb640 1 -- 172.21.15.45:0/3750257147 <== mds.0 v2:172.21.15.63:6835/3653698989 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5589943ac760 con 0x55899451a310 2024-05-08T05:41:53.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.484+0000 7fb735ffb640 10 client.4665 handle_command_reply: tid=0 2024-05-08T05:41:53.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.484+0000 7fb7603e1740 1 client.4665 shutdown 2024-05-08T05:41:53.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.484+0000 7fb7603e1740 1 -- 172.21.15.45:0/3750257147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55899451caf0 msgr2=0x5589944f1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:53.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.484+0000 7fb7603e1740 1 --2- 172.21.15.45:0/3750257147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55899451caf0 0x5589944f1d00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb748061520 tx=0x7fb7480611d0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.485+0000 7fb7603e1740 1 -- 172.21.15.45:0/3750257147 shutdown_connections 2024-05-08T05:41:53.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.485+0000 7fb7603e1740 1 --2- 172.21.15.45:0/3750257147 >> [v2:172.21.15.63:6835/3653698989,v1:172.21.15.63:6837/3653698989] conn(0x55899451a310 0x55899451a6f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.485+0000 7fb7603e1740 1 --2- 172.21.15.45:0/3750257147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55899451caf0 0x5589944f1d00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.485+0000 7fb735ffb640 0 client.4665 ms_handle_reset on v2:172.21.15.63:6835/3653698989 2024-05-08T05:41:53.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.485+0000 7fb7603e1740 1 --2- 172.21.15.45:0/3750257147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589944ec330 0x5589944f17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.485+0000 7fb7603e1740 1 --2- 172.21.15.45:0/3750257147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5589944ebf50 0x55899450a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.485+0000 7fb7603e1740 1 -- 172.21.15.45:0/3750257147 >> 172.21.15.45:0/3750257147 conn(0x5589944bebb0 msgr2=0x5589944c23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:53.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.485+0000 7fb7603e1740 1 -- 172.21.15.45:0/3750257147 shutdown_connections 2024-05-08T05:41:53.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.485+0000 7fb7603e1740 1 -- 172.21.15.45:0/3750257147 wait complete. 2024-05-08T05:41:53.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.485+0000 7fb7603e1740 20 client.4665 trim_cache size 0 max 16384 2024-05-08T05:41:53.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.504+0000 7fb7603e1740 1 Processor -- start 2024-05-08T05:41:53.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.504+0000 7fb7603e1740 1 -- start start 2024-05-08T05:41:53.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.504+0000 7fb7603e1740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5589944ebf50 0x5589944c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.504+0000 7fb7603e1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ec330 0x5589944c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.504+0000 7fb7603e1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55899451caf0 0x558994509000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.504+0000 7fb7603e1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb758202a40 con 0x5589944ec330 2024-05-08T05:41:53.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.504+0000 7fb7603e1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb758201ac0 con 0x55899451caf0 2024-05-08T05:41:53.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.504+0000 7fb7603e1740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb758200e50 con 0x5589944ebf50 2024-05-08T05:41:53.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.504+0000 7fb7567fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ec330 0x5589944c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.504+0000 7fb7567fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ec330 0x5589944c2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34784/0 (socket says 172.21.15.45:34784) 2024-05-08T05:41:53.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.504+0000 7fb7567fc640 1 -- 172.21.15.45:0/3353301192 learned_addr learned my addr 172.21.15.45:0/3353301192 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:53.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.505+0000 7fb756ffd640 1 --2- 172.21.15.45:0/3353301192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5589944ebf50 0x5589944c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.505+0000 7fb7577fe640 1 --2- 172.21.15.45:0/3353301192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55899451caf0 0x558994509000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.505+0000 7fb7567fc640 1 -- 172.21.15.45:0/3353301192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5589944ebf50 msgr2=0x5589944c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:53.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.505+0000 7fb7567fc640 1 --2- 172.21.15.45:0/3353301192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5589944ebf50 0x5589944c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.505+0000 7fb7567fc640 1 -- 172.21.15.45:0/3353301192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55899451caf0 msgr2=0x558994509000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:53.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.505+0000 7fb7567fc640 1 --2- 172.21.15.45:0/3353301192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55899451caf0 0x558994509000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.505+0000 7fb7567fc640 1 -- 172.21.15.45:0/3353301192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55899450ac00 con 0x5589944ec330 2024-05-08T05:41:53.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.505+0000 7fb7567fc640 1 --2- 172.21.15.45:0/3353301192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ec330 0x5589944c2a00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb740007580 tx=0x7fb740004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267bb9a0efc1b650 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:53.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.505+0000 7fb737fff640 1 -- 172.21.15.45:0/3353301192 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74001a030 con 0x5589944ec330 2024-05-08T05:41:53.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.505+0000 7fb737fff640 1 -- 172.21.15.45:0/3353301192 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb740004ef0 con 0x5589944ec330 2024-05-08T05:41:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.505+0000 7fb737fff640 1 -- 172.21.15.45:0/3353301192 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7400052c0 con 0x5589944ec330 2024-05-08T05:41:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.505+0000 7fb7603e1740 1 -- 172.21.15.45:0/3353301192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ec330 msgr2=0x5589944c2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.505+0000 7fb7603e1740 1 --2- 172.21.15.45:0/3353301192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ec330 0x5589944c2a00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb740007580 tx=0x7fb740004bb0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.505+0000 7fb7603e1740 1 -- 172.21.15.45:0/3353301192 shutdown_connections 2024-05-08T05:41:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.505+0000 7fb7603e1740 1 --2- 172.21.15.45:0/3353301192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55899451caf0 0x558994509000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.505+0000 7fb7603e1740 1 --2- 172.21.15.45:0/3353301192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ec330 0x5589944c2a00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.505+0000 7fb7603e1740 1 --2- 172.21.15.45:0/3353301192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5589944ebf50 0x5589944c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.505+0000 7fb7603e1740 1 -- 172.21.15.45:0/3353301192 >> 172.21.15.45:0/3353301192 conn(0x5589944bebb0 msgr2=0x5589944eea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.506+0000 7fb7603e1740 1 -- 172.21.15.45:0/3353301192 shutdown_connections 2024-05-08T05:41:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.506+0000 7fb7603e1740 1 -- 172.21.15.45:0/3353301192 wait complete. 2024-05-08T05:41:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.506+0000 7fb7603e1740 1 -- start start 2024-05-08T05:41:53.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.506+0000 7fb7603e1740 1 Processor -- start 2024-05-08T05:41:53.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.507+0000 7fb7603e1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ebf50 0x5589945877e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.507+0000 7fb7603e1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589944ec330 0x55899450a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.507+0000 7fb7603e1740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55899451caf0 0x558994510710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.507+0000 7fb7603e1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55899450bcf0 con 0x5589944ebf50 2024-05-08T05:41:53.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.507+0000 7fb7603e1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb758201470 con 0x5589944ec330 2024-05-08T05:41:53.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.507+0000 7fb7603e1740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55899450add0 con 0x55899451caf0 2024-05-08T05:41:53.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.507+0000 7fb7603e1740 10 client.0 _ll_register_callbacks cb 0x7fb748061360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:41:53.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.507+0000 7fb756ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ebf50 0x5589945877e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.507+0000 7fb756ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ebf50 0x5589945877e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34796/0 (socket says 172.21.15.45:34796) 2024-05-08T05:41:53.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.507+0000 7fb756ffd640 1 -- 172.21.15.45:0/42250601 learned_addr learned my addr 172.21.15.45:0/42250601 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:53.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.507+0000 7fb7577fe640 1 --2- 172.21.15.45:0/42250601 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55899451caf0 0x558994510710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.507+0000 7fb7567fc640 1 --2- 172.21.15.45:0/42250601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589944ec330 0x55899450a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.507+0000 7fb756ffd640 1 -- 172.21.15.45:0/42250601 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55899451caf0 msgr2=0x558994510710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:53.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.507+0000 7fb756ffd640 1 --2- 172.21.15.45:0/42250601 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55899451caf0 0x558994510710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.507+0000 7fb756ffd640 1 -- 172.21.15.45:0/42250601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589944ec330 msgr2=0x55899450a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:53.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.507+0000 7fb756ffd640 1 --2- 172.21.15.45:0/42250601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589944ec330 0x55899450a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.507+0000 7fb756ffd640 1 -- 172.21.15.45:0/42250601 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55899451aa60 con 0x5589944ebf50 2024-05-08T05:41:53.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.507+0000 7fb7567fc640 1 --2- 172.21.15.45:0/42250601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589944ec330 0x55899450a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:53.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.507+0000 7fb756ffd640 1 --2- 172.21.15.45:0/42250601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ebf50 0x5589945877e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb750009890 tx=0x7fb75002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8af3493aa8fc9e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:53.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.507+0000 7fb735ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:41:53.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.508+0000 7fb735ffb640 1 -- 172.21.15.45:0/42250601 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750057060 con 0x5589944ebf50 2024-05-08T05:41:53.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.508+0000 7fb735ffb640 1 -- 172.21.15.45:0/42250601 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb750004350 con 0x5589944ebf50 2024-05-08T05:41:53.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.508+0000 7fb7603e1740 1 -- 172.21.15.45:0/42250601 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb740004ef0 con 0x5589944ebf50 2024-05-08T05:41:53.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.508+0000 7fb735ffb640 1 -- 172.21.15.45:0/42250601 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fb75005d050 con 0x5589944ebf50 2024-05-08T05:41:53.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.508+0000 7fb7603e1740 10 client.4671 fetch_fsmap learned FSMap version 8 2024-05-08T05:41:53.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.508+0000 7fb7603e1740 1 -- 172.21.15.45:0/42250601 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x558994510c50 con 0x5589944ebf50 2024-05-08T05:41:53.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.508+0000 7fb735ffb640 1 -- 172.21.15.45:0/42250601 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb740004ef0 con 0x5589944ebf50 2024-05-08T05:41:53.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.508+0000 7fb735ffb640 1 -- 172.21.15.45:0/42250601 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7fb750060020 con 0x5589944ebf50 2024-05-08T05:41:53.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.508+0000 7fb7603e1740 10 client.4671 fetch_fsmap finished waiting for FSMap version 8 2024-05-08T05:41:53.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.508+0000 7fb7603e1740 10 client.4671 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T05:41:53.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.508+0000 7fb7603e1740 20 client.4671 populate_metadata read hostname 'smithi045' 2024-05-08T05:41:53.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.508+0000 7fb7603e1740 1 --2- 172.21.15.45:0/42250601 >> [v2:172.21.15.63:6835/3653698989,v1:172.21.15.63:6837/3653698989] conn(0x5589944f1c70 0x55899452c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.508+0000 7fb7603e1740 4 client.4671 mds_command: new command op to 4303 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:41:53.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.508+0000 7fb7603e1740 1 -- 172.21.15.45:0/42250601 --> [v2:172.21.15.63:6835/3653698989,v1:172.21.15.63:6837/3653698989] -- command(tid 0: {"prefix": "status"}) -- 0x5589944364e0 con 0x5589944f1c70 2024-05-08T05:41:53.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.508+0000 7fb7567fc640 1 --2- 172.21.15.45:0/42250601 >> [v2:172.21.15.63:6835/3653698989,v1:172.21.15.63:6837/3653698989] conn(0x5589944f1c70 0x55899452c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.509+0000 7fb735ffb640 10 client.4671 ms_handle_connect on v2:172.21.15.63:6835/3653698989 2024-05-08T05:41:53.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.509+0000 7fb7567fc640 1 --2- 172.21.15.45:0/42250601 >> [v2:172.21.15.63:6835/3653698989,v1:172.21.15.63:6837/3653698989] conn(0x5589944f1c70 0x55899452c370 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f7b366406743d08 server_cookie=e4d846a052ac071f in_seq=0 out_seq=0 2024-05-08T05:41:53.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.509+0000 7fb735ffb640 1 -- 172.21.15.45:0/42250601 <== mds.0 v2:172.21.15.63:6835/3653698989 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5589944364e0 con 0x5589944f1c70 2024-05-08T05:41:53.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.509+0000 7fb735ffb640 10 client.4671 handle_command_reply: tid=0 2024-05-08T05:41:53.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.509+0000 7fb7603e1740 1 client.4671 shutdown 2024-05-08T05:41:53.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.509+0000 7fb7603e1740 1 -- 172.21.15.45:0/42250601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ebf50 msgr2=0x5589945877e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:53.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.509+0000 7fb7603e1740 1 --2- 172.21.15.45:0/42250601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ebf50 0x5589945877e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb750009890 tx=0x7fb75002c070 comp rx=0 tx=0).stop 2024-05-08T05:41:53.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.509+0000 7fb7603e1740 1 -- 172.21.15.45:0/42250601 shutdown_connections 2024-05-08T05:41:53.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.509+0000 7fb7603e1740 1 --2- 172.21.15.45:0/42250601 >> [v2:172.21.15.63:6835/3653698989,v1:172.21.15.63:6837/3653698989] conn(0x5589944f1c70 0x55899452c370 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.509+0000 7fb7603e1740 1 --2- 172.21.15.45:0/42250601 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55899451caf0 0x558994510710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.509+0000 7fb735ffb640 0 client.4671 ms_handle_reset on v2:172.21.15.63:6835/3653698989 2024-05-08T05:41:53.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.509+0000 7fb7603e1740 1 --2- 172.21.15.45:0/42250601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5589944ec330 0x55899450a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.509+0000 7fb7603e1740 1 --2- 172.21.15.45:0/42250601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5589944ebf50 0x5589945877e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.509+0000 7fb7603e1740 1 -- 172.21.15.45:0/42250601 >> 172.21.15.45:0/42250601 conn(0x5589944bebb0 msgr2=0x5589944e97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:53.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.509+0000 7fb7603e1740 1 -- 172.21.15.45:0/42250601 shutdown_connections 2024-05-08T05:41:53.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.509+0000 7fb7603e1740 1 -- 172.21.15.45:0/42250601 wait complete. 2024-05-08T05:41:53.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.509+0000 7fb7603e1740 20 client.4671 trim_cache size 0 max 16384 2024-05-08T05:41:53.517 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:41:53.517 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "e9e1170a-fd27-4625-97a9-5d5f104d13a2", 2024-05-08T05:41:53.517 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-05-08T05:41:53.517 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 4303, 2024-05-08T05:41:53.517 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-05-08T05:41:53.517 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-05-08T05:41:53.517 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-05-08T05:41:53.517 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 21.402749121999999, 2024-05-08T05:41:53.517 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 8, 2024-05-08T05:41:53.517 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 24, 2024-05-08T05:41:53.517 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 24, 2024-05-08T05:41:53.517 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 27.413448144 2024-05-08T05:41:53.517 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:41:53.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.510+0000 7fb75d6cf640 1 -- 172.21.15.45:0/3228459648 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb7380762d0 msgr2=0x7fb738078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:53.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.510+0000 7fb75d6cf640 1 --2- 172.21.15.45:0/3228459648 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb7380762d0 0x7fb738078790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb750034000 tx=0x7fb75002e000 comp rx=0 tx=0).stop 2024-05-08T05:41:53.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.510+0000 7fb75d6cf640 1 -- 172.21.15.45:0/3228459648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb758163920 msgr2=0x7fb7580caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:53.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.510+0000 7fb75d6cf640 1 --2- 172.21.15.45:0/3228459648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb758163920 0x7fb7580caec0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb74800cd50 tx=0x7fb74800a650 comp rx=0 tx=0).stop 2024-05-08T05:41:53.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.510+0000 7fb75d6cf640 1 -- 172.21.15.45:0/3228459648 shutdown_connections 2024-05-08T05:41:53.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.510+0000 7fb75d6cf640 1 --2- 172.21.15.45:0/3228459648 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb7380762d0 0x7fb738078790 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.510+0000 7fb75d6cf640 1 --2- 172.21.15.45:0/3228459648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb758163920 0x7fb7580caec0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.510+0000 7fb75d6cf640 1 --2- 172.21.15.45:0/3228459648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb758071110 0x7fb7580c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.510+0000 7fb75d6cf640 1 --2- 172.21.15.45:0/3228459648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb758070d30 0x7fb7580b6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.510+0000 7fb75d6cf640 1 -- 172.21.15.45:0/3228459648 >> 172.21.15.45:0/3228459648 conn(0x7fb7580d2e90 msgr2=0x7fb758161bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:53.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.510+0000 7fb75d6cf640 1 -- 172.21.15.45:0/3228459648 shutdown_connections 2024-05-08T05:41:53.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:53.510+0000 7fb75d6cf640 1 -- 172.21.15.45:0/3228459648 wait complete. 2024-05-08T05:41:53.522 INFO:teuthology.run_tasks:Running task sequential... 2024-05-08T05:41:53.532 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-08T05:41:53.532 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-08T05:41:53.532 INFO:teuthology.task.print:Enabling mgr modules 2024-05-08T05:41:53.532 INFO:teuthology.run_tasks:Running task kclient... 2024-05-08T05:41:53.556 INFO:tasks.kclient:Mounting kernel clients... 2024-05-08T05:41:53.556 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-08T05:41:53.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:53.557 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:41:53.573 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-08T05:41:53.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:53.573 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T05:41:53.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:53.639 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-05-08T05:41:53.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:53.743 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:41:53.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:53.759 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-05-08T05:41:53.829 INFO:teuthology.orchestra.run.smithi045.stderr:Cannot find device "ceph-brx" 2024-05-08T05:41:53.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:41:53.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:53.831 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-08T05:41:53.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:53.851 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link delete ceph-brx 2024-05-08T05:41:53.925 INFO:teuthology.orchestra.run.smithi063.stderr:Cannot find device "ceph-brx" 2024-05-08T05:41:53.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:41:53.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:53.927 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-08T05:41:53.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:53.986 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link delete ceph-brx 2024-05-08T05:41:54.057 INFO:teuthology.orchestra.run.smithi063.stderr:Cannot find device "ceph-brx" 2024-05-08T05:41:54.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:41:54.059 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-05-08T05:41:54.059 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T05:41:54.059 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:41:54.059 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T05:41:54.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.163+0000 7fea08059640 1 Processor -- start 2024-05-08T05:41:54.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.163+0000 7fea08059640 1 -- start start 2024-05-08T05:41:54.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.164+0000 7fea08059640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0015bc80 0x7fea0006e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:54.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.164+0000 7fea08059640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fea0006e8d0 0x7fea00073700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:54.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.164+0000 7fea08059640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea000b4590 con 0x7fea001524f0 2024-05-08T05:41:54.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.164+0000 7fea08059640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fea000b51a0 con 0x7fea0006e8d0 2024-05-08T05:41:54.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.164+0000 7fea08059640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea000b4330 con 0x7fea0015bc80 2024-05-08T05:41:54.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.164+0000 7fea065f6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fea0006e8d0 0x7fea00073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:54.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.164+0000 7fea065f6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fea0006e8d0 0x7fea00073700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34800/0 (socket says 172.21.15.45:34800) 2024-05-08T05:41:54.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.164+0000 7fea065f6640 1 -- 172.21.15.45:0/1249466335 learned_addr learned my addr 172.21.15.45:0/1249466335 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:54.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.164+0000 7fea055f4640 1 --2- 172.21.15.45:0/1249466335 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0015bc80 0x7fea0006e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:54.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.164+0000 7fea065f6640 1 -- 172.21.15.45:0/1249466335 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0015bc80 msgr2=0x7fea0006e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:54.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.164+0000 7fea065f6640 1 --2- 172.21.15.45:0/1249466335 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0015bc80 0x7fea0006e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:54.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.164+0000 7fea065f6640 1 -- 172.21.15.45:0/1249466335 >> v1:172.21.15.63:6789/0 conn(0x7fea001524f0 legacy=0x7fea000b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:54.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.164+0000 7fea065f6640 1 -- 172.21.15.45:0/1249466335 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea000b4870 con 0x7fea0006e8d0 2024-05-08T05:41:54.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.164+0000 7fea055f4640 1 --2- 172.21.15.45:0/1249466335 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0015bc80 0x7fea0006e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:41:54.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.165+0000 7fea065f6640 1 --2- 172.21.15.45:0/1249466335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea0006e8d0 0x7fea00073700 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009e50 tx=0x7fe9f802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d00856446c121 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:54.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.165+0000 7fea04df3640 1 -- 172.21.15.45:0/1249466335 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8036070 con 0x7fea0006e8d0 2024-05-08T05:41:54.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.165+0000 7fea04df3640 1 -- 172.21.15.45:0/1249466335 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f8031600 con 0x7fea0006e8d0 2024-05-08T05:41:54.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.165+0000 7fea04df3640 1 -- 172.21.15.45:0/1249466335 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f80319d0 con 0x7fea0006e8d0 2024-05-08T05:41:54.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.166+0000 7fea08059640 1 -- 172.21.15.45:0/1249466335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea0006e8d0 msgr2=0x7fea00073700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:54.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.166+0000 7fea08059640 1 --2- 172.21.15.45:0/1249466335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea0006e8d0 0x7fea00073700 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009e50 tx=0x7fe9f802a680 comp rx=0 tx=0).stop 2024-05-08T05:41:54.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.166+0000 7fea08059640 1 -- 172.21.15.45:0/1249466335 shutdown_connections 2024-05-08T05:41:54.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.166+0000 7fea08059640 1 --2- 172.21.15.45:0/1249466335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea0006e8d0 0x7fea00073700 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:54.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.166+0000 7fea08059640 1 --2- 172.21.15.45:0/1249466335 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0015bc80 0x7fea0006e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:54.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.166+0000 7fea08059640 1 -- 172.21.15.45:0/1249466335 >> 172.21.15.45:0/1249466335 conn(0x7fea0006c3a0 msgr2=0x7fea0006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:54.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.166+0000 7fea08059640 1 -- 172.21.15.45:0/1249466335 shutdown_connections 2024-05-08T05:41:54.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.166+0000 7fea08059640 1 -- 172.21.15.45:0/1249466335 wait complete. 2024-05-08T05:41:54.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.167+0000 7fea08059640 1 Processor -- start 2024-05-08T05:41:54.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.167+0000 7fea08059640 1 -- start start 2024-05-08T05:41:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.168+0000 7fea08059640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0006e8d0 0x7fea00148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.168+0000 7fea08059640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea001524f0 0x7fea00157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.168+0000 7fea08059640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea0015bc80 0x7fea00156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.168+0000 7fea08059640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea001528d0 con 0x7fea0015bc80 2024-05-08T05:41:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.168+0000 7fea08059640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea000b4e30 con 0x7fea001524f0 2024-05-08T05:41:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.168+0000 7fea08059640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea000b5630 con 0x7fea0006e8d0 2024-05-08T05:41:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.168+0000 7fea065f6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea0015bc80 0x7fea00156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.168+0000 7fea065f6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea0015bc80 0x7fea00156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34814/0 (socket says 172.21.15.45:34814) 2024-05-08T05:41:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.168+0000 7fea065f6640 1 -- 172.21.15.45:0/1735354261 learned_addr learned my addr 172.21.15.45:0/1735354261 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.168+0000 7fea055f4640 1 --2- 172.21.15.45:0/1735354261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea001524f0 0x7fea00157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.168+0000 7fea05df5640 1 --2- 172.21.15.45:0/1735354261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0006e8d0 0x7fea00148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:54.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.168+0000 7fea065f6640 1 -- 172.21.15.45:0/1735354261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0006e8d0 msgr2=0x7fea00148810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.168+0000 7fea065f6640 1 --2- 172.21.15.45:0/1735354261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0006e8d0 0x7fea00148810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.168+0000 7fea065f6640 1 -- 172.21.15.45:0/1735354261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea001524f0 msgr2=0x7fea00157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.168+0000 7fea065f6640 1 --2- 172.21.15.45:0/1735354261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea001524f0 0x7fea00157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.168+0000 7fea065f6640 1 -- 172.21.15.45:0/1735354261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea00156cd0 con 0x7fea0015bc80 2024-05-08T05:41:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.168+0000 7fea055f4640 1 --2- 172.21.15.45:0/1735354261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea001524f0 0x7fea00157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.168+0000 7fea065f6640 1 --2- 172.21.15.45:0/1735354261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea0015bc80 0x7fea00156160 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8034040 tx=0x7fe9f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.169+0000 7fe9f67fc640 1 -- 172.21.15.45:0/1735354261 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8036050 con 0x7fea0015bc80 2024-05-08T05:41:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.169+0000 7fe9f67fc640 1 -- 172.21.15.45:0/1735354261 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f8042070 con 0x7fea0015bc80 2024-05-08T05:41:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.169+0000 7fea08059640 1 -- 172.21.15.45:0/1735354261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea00156fb0 con 0x7fea0015bc80 2024-05-08T05:41:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.169+0000 7fe9f67fc640 1 -- 172.21.15.45:0/1735354261 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8009510 con 0x7fea0015bc80 2024-05-08T05:41:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.169+0000 7fea08059640 1 -- 172.21.15.45:0/1735354261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea00157570 con 0x7fea0015bc80 2024-05-08T05:41:54.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.170+0000 7fea08059640 1 -- 172.21.15.45:0/1735354261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea000693f0 con 0x7fea0015bc80 2024-05-08T05:41:54.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.170+0000 7fe9f67fc640 1 -- 172.21.15.45:0/1735354261 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe9f8007c70 con 0x7fea0015bc80 2024-05-08T05:41:54.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.171+0000 7fe9f67fc640 1 --2- 172.21.15.45:0/1735354261 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe9e40762d0 0x7fe9e4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:54.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.171+0000 7fe9f67fc640 1 -- 172.21.15.45:0/1735354261 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe9f80b8850 con 0x7fea0015bc80 2024-05-08T05:41:54.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.171+0000 7fea05df5640 1 --2- 172.21.15.45:0/1735354261 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe9e40762d0 0x7fe9e4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:54.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.172+0000 7fea05df5640 1 --2- 172.21.15.45:0/1735354261 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe9e40762d0 0x7fe9e4078790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc004f60 tx=0x7fe9fc007e30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:54.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.176+0000 7fe9f67fc640 1 -- 172.21.15.45:0/1735354261 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe9f8081f00 con 0x7fea0015bc80 2024-05-08T05:41:54.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.406+0000 7fea08059640 1 -- 172.21.15.45:0/1735354261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fea0015ca20 con 0x7fea0015bc80 2024-05-08T05:41:54.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.406+0000 7fe9f67fc640 1 -- 172.21.15.45:0/1735354261 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7fe9f8081800 con 0x7fea0015bc80 2024-05-08T05:41:54.408 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T05:41:54.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.408+0000 7fea08059640 1 -- 172.21.15.45:0/1735354261 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe9e40762d0 msgr2=0x7fe9e4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:54.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.408+0000 7fea08059640 1 --2- 172.21.15.45:0/1735354261 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe9e40762d0 0x7fe9e4078790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc004f60 tx=0x7fe9fc007e30 comp rx=0 tx=0).stop 2024-05-08T05:41:54.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.408+0000 7fea08059640 1 -- 172.21.15.45:0/1735354261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea0015bc80 msgr2=0x7fea00156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:54.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.408+0000 7fea08059640 1 --2- 172.21.15.45:0/1735354261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea0015bc80 0x7fea00156160 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8034040 tx=0x7fe9f8004a40 comp rx=0 tx=0).stop 2024-05-08T05:41:54.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.408+0000 7fea08059640 1 -- 172.21.15.45:0/1735354261 shutdown_connections 2024-05-08T05:41:54.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.408+0000 7fea08059640 1 --2- 172.21.15.45:0/1735354261 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe9e40762d0 0x7fe9e4078790 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:54.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.408+0000 7fea08059640 1 --2- 172.21.15.45:0/1735354261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea0015bc80 0x7fea00156160 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:54.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.408+0000 7fea08059640 1 --2- 172.21.15.45:0/1735354261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea001524f0 0x7fea00157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:54.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.408+0000 7fea08059640 1 --2- 172.21.15.45:0/1735354261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea0006e8d0 0x7fea00148810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:54.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.408+0000 7fea08059640 1 -- 172.21.15.45:0/1735354261 >> 172.21.15.45:0/1735354261 conn(0x7fea0006c3a0 msgr2=0x7fea0015d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:54.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.408+0000 7fea08059640 1 -- 172.21.15.45:0/1735354261 shutdown_connections 2024-05-08T05:41:54.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54.408+0000 7fea08059640 1 -- 172.21.15.45:0/1735354261 wait complete. 2024-05-08T05:41:54.421 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T05:41:54.421 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:41:54.421 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-05-08T05:41:54.421 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T05:41:54.421 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:41:54.421 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T05:41:54.421 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T05:41:54.421 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:41:54.421 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:41:54.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:54.422 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-05-08T05:41:54.438 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:41:54.438 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:41:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:41:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-05-08T05:41:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:54.439 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:41:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:41:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-05-08T05:41:54.439 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:41:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:41:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-05-08T05:41:54.439 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:41:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-05-08T05:41:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-05-08T05:41:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:41:54.439 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42711sec preferred_lft 42711sec 2024-05-08T05:41:54.440 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-05-08T05:41:54.440 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:54.440 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:41:54.440 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-05-08T05:41:54.440 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-05-08T05:41:54.440 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T05:41:54.440 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:41:54.440 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:41:54.440 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2024-05-08T05:41:54.440 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2024-05-08T05:41:54.440 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2024-05-08T05:41:54.441 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T05:41:54.441 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:41:54.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:54.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:54.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:54.582 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:41:54.606 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:41:54.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:54.608 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-05-08T05:41:54.665 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-05-08T05:41:54.665 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:41:54.665 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:41:54.665 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:41:54.665 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:41:54.666 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:41:54.666 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:41:54.666 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:41:54.666 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:41:54.666 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:41:54.666 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:41:54.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:54.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:54.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:54.795 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:41:54.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:54.811 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2024-05-08T05:41:54.867 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:41:54.867 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:41:54.867 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:41:54.867 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-08T05:41:54.868 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:41:54.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:54.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:54.987 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-08T05:41:54.987 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:41:54.987 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:41:54.987 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-08T05:41:54.987 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T05:41:54.987 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-08T05:41:54.987 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-08T05:41:54.987 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-08T05:41:54.987 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:41:55.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:55.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:55.107 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:41:55.107 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:41:55.107 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-05-08T05:41:55.107 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T05:41:55.107 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:41:55.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:55.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:55.188 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:41:55.188 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T05:41:55.204 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:41:55.204 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:41:55.204 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:41:55.260 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T05:41:55.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:55.261 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:41:55.345 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:41:55.345 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:41:55.455 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:41:55.455 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:41:55.455 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-08T05:41:55.455 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@e9e1170a-fd27-4625-97a9-5d5f104d13a2.cephfs=/ 2024-05-08T05:41:55.456 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.63:3300/172.21.15.63:3301" will pass to kernel 2024-05-08T05:41:55.456 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T05:41:55.456 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:41:55.456 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:41:55.456 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:41:55.456 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:41:55.458 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:41:55.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:55.458 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:41:55.484 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:41:55.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:41:55.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:55.583 INFO:teuthology.orchestra.run.smithi045.stdout:client4520 2024-05-08T05:41:55.583 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:41:55.583 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:41:55.600 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:41:55.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:41:55.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:55.698 INFO:teuthology.orchestra.run.smithi045.stdout:client4520 2024-05-08T05:41:55.699 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:41:55.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:41:55.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:55.757 INFO:teuthology.orchestra.run.smithi045.stdout:client4520 2024-05-08T05:41:55.757 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e9e1170a-fd27-4625-97a9-5d5f104d13a2.client4520/status') 2024-05-08T05:41:55.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:55.818 INFO:teuthology.orchestra.run.smithi045.stdout:instance: client.4520 (3)172.21.15.45:0/2729840346 2024-05-08T05:41:55.818 INFO:teuthology.orchestra.run.smithi045.stdout:blocklisted: false 2024-05-08T05:41:55.818 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records in 2024-05-08T05:41:55.819 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records out 2024-05-08T05:41:55.819 INFO:teuthology.orchestra.run.smithi045.stderr:70 bytes copied, 2.4956e-05 s, 2.8 MB/s 2024-05-08T05:41:55.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:55.825 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-05-08T05:41:55.826 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T05:41:55.826 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-08T05:41:55.826 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T05:41:55.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.927+0000 7f7d97c6b640 1 Processor -- start 2024-05-08T05:41:55.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.928+0000 7f7d97c6b640 1 -- start start 2024-05-08T05:41:55.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.928+0000 7f7d97c6b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d900b21f0 0x7f7d9006e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:55.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.928+0000 7f7d97c6b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d900cfea0 0x7f7d901633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:55.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.928+0000 7f7d97c6b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d900d0500 con 0x7f7d90162f50 2024-05-08T05:41:55.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.928+0000 7f7d97c6b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f7d901638f0 con 0x7f7d900cfea0 2024-05-08T05:41:55.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.928+0000 7f7d97c6b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7d9006d810 con 0x7f7d900b21f0 2024-05-08T05:41:55.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.929+0000 7f7d96208640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d900cfea0 0x7f7d901633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:55.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.929+0000 7f7d96208640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d900cfea0 0x7f7d901633b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34818/0 (socket says 172.21.15.45:34818) 2024-05-08T05:41:55.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.929+0000 7f7d96208640 1 -- 172.21.15.45:0/2363112233 learned_addr learned my addr 172.21.15.45:0/2363112233 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:55.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.929+0000 7f7d95a07640 1 --2- 172.21.15.45:0/2363112233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d900b21f0 0x7f7d9006e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:55.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.929+0000 7f7d96208640 1 -- 172.21.15.45:0/2363112233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d900b21f0 msgr2=0x7f7d9006e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:55.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.929+0000 7f7d96208640 1 --2- 172.21.15.45:0/2363112233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d900b21f0 0x7f7d9006e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:55.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.929+0000 7f7d96208640 1 -- 172.21.15.45:0/2363112233 >> v1:172.21.15.63:6789/0 conn(0x7f7d90162f50 legacy=0x7f7d900cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:55.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.929+0000 7f7d96208640 1 -- 172.21.15.45:0/2363112233 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9006da60 con 0x7f7d900cfea0 2024-05-08T05:41:55.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.929+0000 7f7d95a07640 1 --2- 172.21.15.45:0/2363112233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d900b21f0 0x7f7d9006e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:55.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.930+0000 7f7d96208640 1 --2- 172.21.15.45:0/2363112233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d900cfea0 0x7f7d901633b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009e50 tx=0x7f7d8802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770816ce0c03c3e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:55.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.930+0000 7f7d94a05640 1 -- 172.21.15.45:0/2363112233 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d88036070 con 0x7f7d900cfea0 2024-05-08T05:41:55.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.930+0000 7f7d94a05640 1 -- 172.21.15.45:0/2363112233 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d88031600 con 0x7f7d900cfea0 2024-05-08T05:41:55.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.930+0000 7f7d94a05640 1 -- 172.21.15.45:0/2363112233 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d880319d0 con 0x7f7d900cfea0 2024-05-08T05:41:55.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.930+0000 7f7d97c6b640 1 -- 172.21.15.45:0/2363112233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d900cfea0 msgr2=0x7f7d901633b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:55.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.930+0000 7f7d97c6b640 1 --2- 172.21.15.45:0/2363112233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d900cfea0 0x7f7d901633b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009e50 tx=0x7f7d8802a610 comp rx=0 tx=0).stop 2024-05-08T05:41:55.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.930+0000 7f7d97c6b640 1 -- 172.21.15.45:0/2363112233 shutdown_connections 2024-05-08T05:41:55.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.930+0000 7f7d97c6b640 1 --2- 172.21.15.45:0/2363112233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d900cfea0 0x7f7d901633b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:55.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.930+0000 7f7d97c6b640 1 --2- 172.21.15.45:0/2363112233 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d900b21f0 0x7f7d9006e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:55.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.930+0000 7f7d97c6b640 1 -- 172.21.15.45:0/2363112233 >> 172.21.15.45:0/2363112233 conn(0x7f7d9009ba40 msgr2=0x7f7d9009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:55.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.930+0000 7f7d97c6b640 1 -- 172.21.15.45:0/2363112233 shutdown_connections 2024-05-08T05:41:55.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.930+0000 7f7d97c6b640 1 -- 172.21.15.45:0/2363112233 wait complete. 2024-05-08T05:41:55.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.932+0000 7f7d97c6b640 1 Processor -- start 2024-05-08T05:41:55.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.932+0000 7f7d97c6b640 1 -- start start 2024-05-08T05:41:55.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.932+0000 7f7d97c6b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d900b21f0 0x7f7d900c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:55.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.932+0000 7f7d97c6b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d900cfea0 0x7f7d90154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:55.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.932+0000 7f7d97c6b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d90162f50 0x7f7d900c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:55.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.932+0000 7f7d97c6b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9006ea80 con 0x7f7d900cfea0 2024-05-08T05:41:55.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.932+0000 7f7d97c6b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9006e0b0 con 0x7f7d900b21f0 2024-05-08T05:41:55.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.932+0000 7f7d97c6b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7d900b25d0 con 0x7f7d90162f50 2024-05-08T05:41:55.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.932+0000 7f7d95206640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d900cfea0 0x7f7d90154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:55.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.933+0000 7f7d95206640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d900cfea0 0x7f7d90154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34824/0 (socket says 172.21.15.45:34824) 2024-05-08T05:41:55.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.933+0000 7f7d95206640 1 -- 172.21.15.45:0/1313338788 learned_addr learned my addr 172.21.15.45:0/1313338788 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:55.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.933+0000 7f7d96208640 1 --2- 172.21.15.45:0/1313338788 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d90162f50 0x7f7d900c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:55.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.933+0000 7f7d95a07640 1 --2- 172.21.15.45:0/1313338788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d900b21f0 0x7f7d900c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:55.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.933+0000 7f7d95206640 1 -- 172.21.15.45:0/1313338788 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d90162f50 msgr2=0x7f7d900c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:55.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.933+0000 7f7d95206640 1 --2- 172.21.15.45:0/1313338788 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d90162f50 0x7f7d900c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:55.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.933+0000 7f7d95206640 1 -- 172.21.15.45:0/1313338788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d900b21f0 msgr2=0x7f7d900c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:55.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.933+0000 7f7d95206640 1 --2- 172.21.15.45:0/1313338788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d900b21f0 0x7f7d900c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:55.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.933+0000 7f7d95206640 1 -- 172.21.15.45:0/1313338788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d900c6e60 con 0x7f7d900cfea0 2024-05-08T05:41:55.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.933+0000 7f7d95a07640 1 --2- 172.21.15.45:0/1313338788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d900b21f0 0x7f7d900c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:41:55.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.933+0000 7f7d95206640 1 --2- 172.21.15.45:0/1313338788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d900cfea0 0x7f7d90154250 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7d8000c3a0 tx=0x7f7d8000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:55.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.933+0000 7f7d867fc640 1 -- 172.21.15.45:0/1313338788 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80017070 con 0x7f7d900cfea0 2024-05-08T05:41:55.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.933+0000 7f7d867fc640 1 -- 172.21.15.45:0/1313338788 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d800052d0 con 0x7f7d900cfea0 2024-05-08T05:41:55.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.933+0000 7f7d867fc640 1 -- 172.21.15.45:0/1313338788 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d80003830 con 0x7f7d900cfea0 2024-05-08T05:41:55.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.933+0000 7f7d97c6b640 1 -- 172.21.15.45:0/1313338788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d900c7140 con 0x7f7d900cfea0 2024-05-08T05:41:55.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.933+0000 7f7d97c6b640 1 -- 172.21.15.45:0/1313338788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d900c7700 con 0x7f7d900cfea0 2024-05-08T05:41:55.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.934+0000 7f7d97c6b640 1 -- 172.21.15.45:0/1313338788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d90069b00 con 0x7f7d900cfea0 2024-05-08T05:41:55.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.935+0000 7f7d867fc640 1 -- 172.21.15.45:0/1313338788 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7d8001a070 con 0x7f7d900cfea0 2024-05-08T05:41:55.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.935+0000 7f7d867fc640 1 --2- 172.21.15.45:0/1313338788 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7d740762d0 0x7f7d74078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:55.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.935+0000 7f7d867fc640 1 -- 172.21.15.45:0/1313338788 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7d80096bc0 con 0x7f7d900cfea0 2024-05-08T05:41:55.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.936+0000 7f7d95a07640 1 --2- 172.21.15.45:0/1313338788 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7d740762d0 0x7f7d74078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:55.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.936+0000 7f7d95a07640 1 --2- 172.21.15.45:0/1313338788 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7d740762d0 0x7f7d74078790 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7d901f86f0 tx=0x7f7d8c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:55.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:55.941+0000 7f7d867fc640 1 -- 172.21.15.45:0/1313338788 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7d8005fb90 con 0x7f7d900cfea0 2024-05-08T05:41:56.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56.176+0000 7f7d97c6b640 1 -- 172.21.15.45:0/1313338788 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7d90070a30 con 0x7f7d900cfea0 2024-05-08T05:41:56.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56.177+0000 7f7d867fc640 1 -- 172.21.15.45:0/1313338788 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f7d800681d0 con 0x7f7d900cfea0 2024-05-08T05:41:56.178 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T05:41:56.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56.179+0000 7f7d97c6b640 1 -- 172.21.15.45:0/1313338788 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7d740762d0 msgr2=0x7f7d74078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:56.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56.179+0000 7f7d97c6b640 1 --2- 172.21.15.45:0/1313338788 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7d740762d0 0x7f7d74078790 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7d901f86f0 tx=0x7f7d8c008040 comp rx=0 tx=0).stop 2024-05-08T05:41:56.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56.179+0000 7f7d97c6b640 1 -- 172.21.15.45:0/1313338788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d900cfea0 msgr2=0x7f7d90154250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56.179+0000 7f7d97c6b640 1 --2- 172.21.15.45:0/1313338788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d900cfea0 0x7f7d90154250 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7d8000c3a0 tx=0x7f7d8000c860 comp rx=0 tx=0).stop 2024-05-08T05:41:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56.179+0000 7f7d97c6b640 1 -- 172.21.15.45:0/1313338788 shutdown_connections 2024-05-08T05:41:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56.179+0000 7f7d97c6b640 1 --2- 172.21.15.45:0/1313338788 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7d740762d0 0x7f7d74078790 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56.179+0000 7f7d97c6b640 1 --2- 172.21.15.45:0/1313338788 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d90162f50 0x7f7d900c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56.179+0000 7f7d97c6b640 1 --2- 172.21.15.45:0/1313338788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d900cfea0 0x7f7d90154250 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56.179+0000 7f7d97c6b640 1 --2- 172.21.15.45:0/1313338788 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d900b21f0 0x7f7d900c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56.179+0000 7f7d97c6b640 1 -- 172.21.15.45:0/1313338788 >> 172.21.15.45:0/1313338788 conn(0x7f7d9009ba40 msgr2=0x7f7d900afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56.179+0000 7f7d97c6b640 1 -- 172.21.15.45:0/1313338788 shutdown_connections 2024-05-08T05:41:56.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56.179+0000 7f7d97c6b640 1 -- 172.21.15.45:0/1313338788 wait complete. 2024-05-08T05:41:56.191 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T05:41:56.191 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:41:56.191 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-05-08T05:41:56.191 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-08T05:41:56.191 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-08T05:41:56.191 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T05:41:56.191 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T05:41:56.191 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:41:56.191 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-08T05:41:56.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:56.192 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-05-08T05:41:56.208 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:41:56.208 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:41:56.208 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:41:56.208 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:56.208 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-05-08T05:41:56.208 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:56.208 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:41:56.208 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:41:56.208 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-05-08T05:41:56.209 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:41:56.209 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:41:56.209 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-05-08T05:41:56.209 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:41:56.209 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-05-08T05:41:56.209 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-05-08T05:41:56.209 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:41:56.209 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42709sec preferred_lft 42709sec 2024-05-08T05:41:56.209 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-05-08T05:41:56.209 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:56.209 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:41:56.209 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-05-08T05:41:56.209 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-05-08T05:41:56.209 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:41:56.209 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 82:f3:eb:6a:57:ba brd ff:ff:ff:ff:ff:ff 2024-05-08T05:41:56.210 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T05:41:56.210 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:56.210 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c80a:aff:fe94:c77/64 scope link 2024-05-08T05:41:56.210 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:56.210 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:41:56.210 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 82:f3:eb:6a:57:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:41:56.210 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::80f3:ebff:fe6a:57ba/64 scope link tentative 2024-05-08T05:41:56.210 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:56.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:56.210 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:41:56.273 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:41:56.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:56.274 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-05-08T05:41:56.330 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-05-08T05:41:56.330 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:41:56.331 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:41:56.331 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:41:56.331 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:41:56.331 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:41:56.331 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:41:56.331 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:41:56.332 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:41:56.332 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:41:56.332 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:41:56.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:56.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:56.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:56.461 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:41:56.477 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T05:41:56.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:56.478 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2024-05-08T05:41:56.533 INFO:teuthology.orchestra.run.smithi045.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-08T05:41:56.533 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:41:56.533 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:41:56.534 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T05:41:56.534 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-08T05:41:56.534 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:41:56.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:56.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:56.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:56.653 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-08T05:41:56.678 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:41:56.678 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:41:56.678 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:41:56.678 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:56.678 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-05-08T05:41:56.678 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:56.679 INFO:teuthology.orchestra.run.smithi045.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:41:56.679 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 32:6a:07:43:b7:0a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T05:41:56.679 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T05:41:56.679 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:56.679 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::306a:7ff:fe43:b70a/64 scope link tentative 2024-05-08T05:41:56.679 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:56.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:56.680 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-08T05:41:56.746 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:41:56.746 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:41:56.746 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:41:56.746 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:56.746 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-05-08T05:41:56.746 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:56.746 INFO:teuthology.orchestra.run.smithi045.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:41:56.746 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 32:6a:07:43:b7:0a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T05:41:56.746 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T05:41:56.747 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:56.747 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::306a:7ff:fe43:b70a/64 scope link tentative 2024-05-08T05:41:56.747 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:56.748 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-08T05:41:56.748 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:41:56.748 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:41:56.748 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-08T05:41:56.748 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-08T05:41:56.748 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-08T05:41:56.749 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-08T05:41:56.749 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-08T05:41:56.749 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:41:56.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:56.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:56.944 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:41:56.944 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:41:56.944 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.1 up 2024-05-08T05:41:56.944 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.1 master ceph-brx 2024-05-08T05:41:56.944 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:41:56.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:57.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:57.024 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:41:57.024 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-08T05:41:57.040 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-08T05:41:57.040 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:41:57.040 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-08T05:41:57.097 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-05-08T05:41:57.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:57.097 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:41:57.182 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:41:57.182 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:41:57.190 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:41:57.190 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:41:57.190 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-08T05:41:57.190 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 2@e9e1170a-fd27-4625-97a9-5d5f104d13a2.cephfs=/ 2024-05-08T05:41:57.190 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.45:3300/172.21.15.63:3300/172.21.15.63:3301" will pass to kernel 2024-05-08T05:41:57.190 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-05-08T05:41:57.190 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:41:57.191 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:41:57.191 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:41:57.191 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:41:57.192 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:41:57.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:57.193 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-08T05:41:57.218 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:41:57.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:41:57.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:57.320 INFO:teuthology.orchestra.run.smithi045.stdout:client4529 2024-05-08T05:41:57.320 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:41:57.320 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:41:57.337 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:41:57.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:41:57.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:57.435 INFO:teuthology.orchestra.run.smithi045.stdout:client4529 2024-05-08T05:41:57.435 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:41:57.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:41:57.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:57.492 INFO:teuthology.orchestra.run.smithi045.stdout:client4529 2024-05-08T05:41:57.492 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e9e1170a-fd27-4625-97a9-5d5f104d13a2.client4529/status') 2024-05-08T05:41:57.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:57.552 INFO:teuthology.orchestra.run.smithi045.stdout:instance: client.4529 (3)172.21.15.45:0/2958743641 2024-05-08T05:41:57.552 INFO:teuthology.orchestra.run.smithi045.stdout:blocklisted: false 2024-05-08T05:41:57.552 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records in 2024-05-08T05:41:57.553 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records out 2024-05-08T05:41:57.553 INFO:teuthology.orchestra.run.smithi045.stderr:70 bytes copied, 2.2973e-05 s, 3.0 MB/s 2024-05-08T05:41:57.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:57.560 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-05-08T05:41:57.560 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T05:41:57.560 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-08T05:41:57.560 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T05:41:57.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.663+0000 7f5a34e28640 1 Processor -- start 2024-05-08T05:41:57.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.663+0000 7f5a34e28640 1 -- start start 2024-05-08T05:41:57.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.664+0000 7f5a34e28640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a300b2490 0x7f5a30070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:57.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.664+0000 7f5a34e28640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5a30070d20 0x7f5a300711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.664+0000 7f5a34e28640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a3006e110 con 0x7f5a301629b0 2024-05-08T05:41:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.664+0000 7f5a34e28640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5a3006dec0 con 0x7f5a30070d20 2024-05-08T05:41:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.664+0000 7f5a34e28640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5a3006e3f0 con 0x7f5a300b2490 2024-05-08T05:41:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.665+0000 7f5a2dd9b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a300b2490 0x7f5a30070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.665+0000 7f5a2e59c640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f5a301629b0 0x7f5a3006eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:49330/0 (socket says 172.21.15.45:49330) 2024-05-08T05:41:57.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.665+0000 7f5a2ed9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5a30070d20 0x7f5a300711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:57.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.665+0000 7f5a2e59c640 1 -- 172.21.15.45:0/2555273360 learned_addr learned my addr 172.21.15.45:0/2555273360 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:41:57.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.665+0000 7f5a2d59a640 1 -- 172.21.15.45:0/2555273360 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1417805430 0 0) 0x7f5a3006e110 con 0x7f5a301629b0 2024-05-08T05:41:57.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.665+0000 7f5a2d59a640 1 -- 172.21.15.45:0/2555273360 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a1c003520 con 0x7f5a301629b0 2024-05-08T05:41:57.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.665+0000 7f5a2ed9d640 1 -- 172.21.15.45:0/2555273360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a300b2490 msgr2=0x7f5a30070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:57.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.665+0000 7f5a2ed9d640 1 --2- 172.21.15.45:0/2555273360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a300b2490 0x7f5a30070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:57.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.665+0000 7f5a2ed9d640 1 -- 172.21.15.45:0/2555273360 >> v1:172.21.15.63:6789/0 conn(0x7f5a301629b0 legacy=0x7f5a3006eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:57.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.665+0000 7f5a2ed9d640 1 -- 172.21.15.45:0/2555273360 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3006e6d0 con 0x7f5a30070d20 2024-05-08T05:41:57.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.666+0000 7f5a2ed9d640 1 --2- 172.21.15.45:0/2555273360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a30070d20 0x7f5a300711a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009d90 tx=0x7f5a2002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e601cc62e81f7baf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:57.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.666+0000 7f5a2d59a640 1 -- 172.21.15.45:0/2555273360 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20007740 con 0x7f5a30070d20 2024-05-08T05:41:57.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.666+0000 7f5a2d59a640 1 -- 172.21.15.45:0/2555273360 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a200078e0 con 0x7f5a30070d20 2024-05-08T05:41:57.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.666+0000 7f5a2d59a640 1 -- 172.21.15.45:0/2555273360 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20007c90 con 0x7f5a30070d20 2024-05-08T05:41:57.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.666+0000 7f5a34e28640 1 -- 172.21.15.45:0/2555273360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a30070d20 msgr2=0x7f5a300711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:57.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.666+0000 7f5a34e28640 1 --2- 172.21.15.45:0/2555273360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a30070d20 0x7f5a300711a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009d90 tx=0x7f5a2002a660 comp rx=0 tx=0).stop 2024-05-08T05:41:57.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.667+0000 7f5a34e28640 1 -- 172.21.15.45:0/2555273360 shutdown_connections 2024-05-08T05:41:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.667+0000 7f5a34e28640 1 --2- 172.21.15.45:0/2555273360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a30070d20 0x7f5a300711a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.667+0000 7f5a34e28640 1 --2- 172.21.15.45:0/2555273360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a300b2490 0x7f5a30070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.667+0000 7f5a34e28640 1 -- 172.21.15.45:0/2555273360 >> 172.21.15.45:0/2555273360 conn(0x7f5a300b49d0 msgr2=0x7f5a300b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.667+0000 7f5a34e28640 1 -- 172.21.15.45:0/2555273360 shutdown_connections 2024-05-08T05:41:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.667+0000 7f5a34e28640 1 -- 172.21.15.45:0/2555273360 wait complete. 2024-05-08T05:41:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.668+0000 7f5a34e28640 1 Processor -- start 2024-05-08T05:41:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.668+0000 7f5a34e28640 1 -- start start 2024-05-08T05:41:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.669+0000 7f5a34e28640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a30070d20 0x7f5a3015da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.669+0000 7f5a34e28640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a300b2490 0x7f5a300cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.669+0000 7f5a34e28640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a301629b0 0x7f5a300cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.669+0000 7f5a34e28640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5a300709f0 con 0x7f5a30070d20 2024-05-08T05:41:57.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.669+0000 7f5a34e28640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5a30070870 con 0x7f5a300b2490 2024-05-08T05:41:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.669+0000 7f5a34e28640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5a300afcb0 con 0x7f5a301629b0 2024-05-08T05:41:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.669+0000 7f5a2e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a30070d20 0x7f5a3015da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.669+0000 7f5a2e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a30070d20 0x7f5a3015da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34830/0 (socket says 172.21.15.45:34830) 2024-05-08T05:41:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.669+0000 7f5a2e59c640 1 -- 172.21.15.45:0/311611263 learned_addr learned my addr 172.21.15.45:0/311611263 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.669+0000 7f5a2ed9d640 1 --2- 172.21.15.45:0/311611263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a301629b0 0x7f5a300cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.669+0000 7f5a2dd9b640 1 --2- 172.21.15.45:0/311611263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a300b2490 0x7f5a300cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.669+0000 7f5a2e59c640 1 -- 172.21.15.45:0/311611263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a301629b0 msgr2=0x7f5a300cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.669+0000 7f5a2e59c640 1 --2- 172.21.15.45:0/311611263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a301629b0 0x7f5a300cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.669+0000 7f5a2e59c640 1 -- 172.21.15.45:0/311611263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a300b2490 msgr2=0x7f5a300cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.669+0000 7f5a2e59c640 1 --2- 172.21.15.45:0/311611263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a300b2490 0x7f5a300cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:57.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.669+0000 7f5a2e59c640 1 -- 172.21.15.45:0/311611263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a301faa90 con 0x7f5a30070d20 2024-05-08T05:41:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.669+0000 7f5a2e59c640 1 --2- 172.21.15.45:0/311611263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a30070d20 0x7f5a3015da60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800c440 tx=0x7f5a2800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.670+0000 7f5a26ffd640 1 -- 172.21.15.45:0/311611263 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a28017070 con 0x7f5a30070d20 2024-05-08T05:41:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.670+0000 7f5a26ffd640 1 -- 172.21.15.45:0/311611263 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a28005ce0 con 0x7f5a30070d20 2024-05-08T05:41:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.670+0000 7f5a34e28640 1 -- 172.21.15.45:0/311611263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a300ce6b0 con 0x7f5a30070d20 2024-05-08T05:41:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.670+0000 7f5a26ffd640 1 -- 172.21.15.45:0/311611263 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a28007270 con 0x7f5a30070d20 2024-05-08T05:41:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.670+0000 7f5a34e28640 1 -- 172.21.15.45:0/311611263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a301fb010 con 0x7f5a30070d20 2024-05-08T05:41:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.671+0000 7f5a34e28640 1 -- 172.21.15.45:0/311611263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a30068c40 con 0x7f5a30070d20 2024-05-08T05:41:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.671+0000 7f5a26ffd640 1 -- 172.21.15.45:0/311611263 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5a2801a070 con 0x7f5a30070d20 2024-05-08T05:41:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.672+0000 7f5a26ffd640 1 --2- 172.21.15.45:0/311611263 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5a100762d0 0x7f5a10078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.672+0000 7f5a26ffd640 1 -- 172.21.15.45:0/311611263 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f5a28096ab0 con 0x7f5a30070d20 2024-05-08T05:41:57.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.672+0000 7f5a2dd9b640 1 --2- 172.21.15.45:0/311611263 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5a100762d0 0x7f5a10078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:57.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.673+0000 7f5a2dd9b640 1 --2- 172.21.15.45:0/311611263 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5a100762d0 0x7f5a10078790 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5a300bd5d0 tx=0x7f5a18009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:57.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.677+0000 7f5a26ffd640 1 -- 172.21.15.45:0/311611263 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5a30068c40 con 0x7f5a30070d20 2024-05-08T05:41:57.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.906+0000 7f5a34e28640 1 -- 172.21.15.45:0/311611263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5a300693f0 con 0x7f5a30070d20 2024-05-08T05:41:57.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.906+0000 7f5a26ffd640 1 -- 172.21.15.45:0/311611263 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f5a2805fa80 con 0x7f5a30070d20 2024-05-08T05:41:57.907 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T05:41:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.908+0000 7f5a34e28640 1 -- 172.21.15.45:0/311611263 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5a100762d0 msgr2=0x7f5a10078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.908+0000 7f5a34e28640 1 --2- 172.21.15.45:0/311611263 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5a100762d0 0x7f5a10078790 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5a300bd5d0 tx=0x7f5a18009190 comp rx=0 tx=0).stop 2024-05-08T05:41:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.908+0000 7f5a34e28640 1 -- 172.21.15.45:0/311611263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a30070d20 msgr2=0x7f5a3015da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.908+0000 7f5a34e28640 1 --2- 172.21.15.45:0/311611263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a30070d20 0x7f5a3015da60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800c440 tx=0x7f5a2800c900 comp rx=0 tx=0).stop 2024-05-08T05:41:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.908+0000 7f5a34e28640 1 -- 172.21.15.45:0/311611263 shutdown_connections 2024-05-08T05:41:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.908+0000 7f5a34e28640 1 --2- 172.21.15.45:0/311611263 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5a100762d0 0x7f5a10078790 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.908+0000 7f5a34e28640 1 --2- 172.21.15.45:0/311611263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5a301629b0 0x7f5a300cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.908+0000 7f5a34e28640 1 --2- 172.21.15.45:0/311611263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5a300b2490 0x7f5a300cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.908+0000 7f5a34e28640 1 --2- 172.21.15.45:0/311611263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5a30070d20 0x7f5a3015da60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.908+0000 7f5a34e28640 1 -- 172.21.15.45:0/311611263 >> 172.21.15.45:0/311611263 conn(0x7f5a300b49d0 msgr2=0x7f5a300b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.908+0000 7f5a34e28640 1 -- 172.21.15.45:0/311611263 shutdown_connections 2024-05-08T05:41:57.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:57.908+0000 7f5a34e28640 1 -- 172.21.15.45:0/311611263 wait complete. 2024-05-08T05:41:57.919 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T05:41:57.919 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:41:57.919 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi063.front.sepia.ceph.com 2024-05-08T05:41:57.920 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-08T05:41:57.920 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-08T05:41:57.920 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T05:41:57.920 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T05:41:57.920 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:41:57.920 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-08T05:41:57.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:57.920 DEBUG:teuthology.orchestra.run.smithi063:> ip addr 2024-05-08T05:41:57.940 INFO:teuthology.orchestra.run.smithi063.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:41:57.940 INFO:teuthology.orchestra.run.smithi063.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:41:57.940 INFO:teuthology.orchestra.run.smithi063.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:41:57.940 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:57.940 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 ::1/128 scope host 2024-05-08T05:41:57.941 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:57.941 INFO:teuthology.orchestra.run.smithi063.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:41:57.941 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:76 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:41:57.941 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f0 2024-05-08T05:41:57.941 INFO:teuthology.orchestra.run.smithi063.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:41:57.941 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:77 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:41:57.941 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f1 2024-05-08T05:41:57.941 INFO:teuthology.orchestra.run.smithi063.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:41:57.941 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d6 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:41:57.941 INFO:teuthology.orchestra.run.smithi063.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:41:57.941 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d7 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:41:57.941 INFO:teuthology.orchestra.run.smithi063.stdout: inet 172.21.15.63/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-08T05:41:57.941 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft 42682sec preferred_lft 42682sec 2024-05-08T05:41:57.941 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::ec4:7aff:feb6:f8d7/64 scope link 2024-05-08T05:41:57.942 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:41:57.942 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T05:41:57.942 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:41:57.942 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-08T05:41:57.942 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add name ceph-brx type bridge 2024-05-08T05:41:57.942 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip addr flush dev ceph-brx 2024-05-08T05:41:57.942 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set ceph-brx up 2024-05-08T05:41:57.942 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T05:41:57.942 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-08T05:41:58.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:58.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:58.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:58.139 DEBUG:teuthology.orchestra.run.smithi063:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:41:58.165 INFO:teuthology.orchestra.run.smithi063.stdout:1 2024-05-08T05:41:58.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:58.167 DEBUG:teuthology.orchestra.run.smithi063:> route 2024-05-08T05:41:58.233 INFO:teuthology.orchestra.run.smithi063.stdout:Kernel IP routing table 2024-05-08T05:41:58.233 INFO:teuthology.orchestra.run.smithi063.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:41:58.233 INFO:teuthology.orchestra.run.smithi063.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T05:41:58.233 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T05:41:58.233 INFO:teuthology.orchestra.run.smithi063.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:41:58.234 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:41:58.234 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-08T05:41:58.234 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T05:41:58.234 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T05:41:58.234 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T05:41:58.234 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-08T05:41:58.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:58.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:58.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:58.533 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-08T05:41:58.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:58.550 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list-id 2024-05-08T05:41:58.606 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:41:58.606 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-08T05:41:58.606 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T05:41:58.606 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-08T05:41:58.606 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-08T05:41:58.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:58.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:58.730 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-08T05:41:58.730 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:41:58.730 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-08T05:41:58.730 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-08T05:41:58.730 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T05:41:58.730 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-08T05:41:58.730 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-08T05:41:58.731 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-08T05:41:58.731 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-08T05:41:58.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:58.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:58.861 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:41:58.861 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-08T05:41:58.861 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set brx.0 up 2024-05-08T05:41:58.861 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T05:41:58.861 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-08T05:41:58.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:58.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:58.941 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:41:58.941 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-08T05:41:58.957 INFO:teuthology.orchestra.run.smithi063.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-08T05:41:58.958 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:41:58.958 DEBUG:teuthology.orchestra.run.smithi063:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-08T05:41:59.014 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-05-08T05:41:59.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:59.014 DEBUG:teuthology.orchestra.run.smithi063:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:41:59.220 INFO:teuthology.orchestra.run.smithi063.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:41:59.220 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:41:59.403 INFO:teuthology.orchestra.run.smithi063.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:41:59.403 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:41:59.403 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-08T05:41:59.403 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: trying mount with new device syntax: 1@e9e1170a-fd27-4625-97a9-5d5f104d13a2.cephfs=/ 2024-05-08T05:41:59.403 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.45:3300/172.21.15.63:3300/172.21.15.63:3301" will pass to kernel 2024-05-08T05:41:59.404 INFO:teuthology.orchestra.run.smithi063.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-08T05:41:59.404 INFO:teuthology.orchestra.run.smithi063.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:41:59.404 INFO:teuthology.orchestra.run.smithi063.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:41:59.404 INFO:teuthology.orchestra.run.smithi063.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:41:59.404 INFO:teuthology.orchestra.run.smithi063.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:41:59.406 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:41:59.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:41:59.406 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-08T05:41:59.432 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:41:59.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:41:59.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:59.530 INFO:teuthology.orchestra.run.smithi063.stdout:client4538 2024-05-08T05:41:59.530 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:41:59.530 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:41:59.546 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:41:59.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:41:59.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:59.645 INFO:teuthology.orchestra.run.smithi063.stdout:client4538 2024-05-08T05:41:59.645 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:41:59.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:41:59.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:59.702 INFO:teuthology.orchestra.run.smithi063.stdout:client4538 2024-05-08T05:41:59.703 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e9e1170a-fd27-4625-97a9-5d5f104d13a2.client4538/status') 2024-05-08T05:41:59.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:41:59.764 INFO:teuthology.orchestra.run.smithi063.stdout:instance: client.4538 (3)192.168.0.1:0/3050394840 2024-05-08T05:41:59.764 INFO:teuthology.orchestra.run.smithi063.stdout:blocklisted: false 2024-05-08T05:41:59.764 INFO:teuthology.orchestra.run.smithi063.stderr:0+1 records in 2024-05-08T05:41:59.764 INFO:teuthology.orchestra.run.smithi063.stderr:0+1 records out 2024-05-08T05:41:59.764 INFO:teuthology.orchestra.run.smithi063.stderr:69 bytes copied, 2.3977e-05 s, 2.9 MB/s 2024-05-08T05:41:59.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:41:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:41:59.771 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-05-08T05:41:59.771 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T05:41:59.771 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-08T05:41:59.772 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T05:41:59.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.888+0000 7f7a2ab04640 1 Processor -- start 2024-05-08T05:41:59.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.888+0000 7f7a2ab04640 1 -- start start 2024-05-08T05:41:59.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.889+0000 7f7a2ab04640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7a24157540 0x7f7a2415e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:59.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.889+0000 7f7a2ab04640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7a2406fc30 0x7f7a240af740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:59.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.889+0000 7f7a2ab04640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a2415e550 con 0x7f7a240b48a0 2024-05-08T05:41:59.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.889+0000 7f7a2ab04640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f7a240701d0 con 0x7f7a24157540 2024-05-08T05:41:59.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.889+0000 7f7a2ab04640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7a240b3dc0 con 0x7f7a2406fc30 2024-05-08T05:41:59.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.889+0000 7f7a288a0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7a24157540 0x7f7a2415e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:59.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.890+0000 7f7a288a0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7a24157540 0x7f7a2415e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34836/0 (socket says 172.21.15.45:34836) 2024-05-08T05:41:59.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.890+0000 7f7a288a0640 1 -- 172.21.15.45:0/3756876694 learned_addr learned my addr 172.21.15.45:0/3756876694 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:59.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.890+0000 7f7a23fff640 1 --2- 172.21.15.45:0/3756876694 >> v2:172.21.15.63:3300/0 conn(0x7f7a2406fc30 0x7f7a240af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:59.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.890+0000 7f7a288a0640 1 -- 172.21.15.45:0/3756876694 >> v2:172.21.15.63:3300/0 conn(0x7f7a2406fc30 msgr2=0x7f7a240af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:41:59.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.890+0000 7f7a288a0640 1 --2- 172.21.15.45:0/3756876694 >> v2:172.21.15.63:3300/0 conn(0x7f7a2406fc30 0x7f7a240af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:59.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.890+0000 7f7a288a0640 1 -- 172.21.15.45:0/3756876694 >> v1:172.21.15.63:6789/0 conn(0x7f7a240b48a0 legacy=0x7f7a240ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:59.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.890+0000 7f7a288a0640 1 -- 172.21.15.45:0/3756876694 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a240b40a0 con 0x7f7a24157540 2024-05-08T05:41:59.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.891+0000 7f7a288a0640 1 --2- 172.21.15.45:0/3756876694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a24157540 0x7f7a2415e010 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7a24160ac0 tx=0x7f7a1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8169d1268ea28cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:59.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.891+0000 7f7a237fe640 1 -- 172.21.15.45:0/3756876694 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a14037070 con 0x7f7a24157540 2024-05-08T05:41:59.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.891+0000 7f7a237fe640 1 -- 172.21.15.45:0/3756876694 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a14032440 con 0x7f7a24157540 2024-05-08T05:41:59.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.891+0000 7f7a237fe640 1 -- 172.21.15.45:0/3756876694 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a14032810 con 0x7f7a24157540 2024-05-08T05:41:59.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.891+0000 7f7a2ab04640 1 -- 172.21.15.45:0/3756876694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a24157540 msgr2=0x7f7a2415e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.891+0000 7f7a2ab04640 1 --2- 172.21.15.45:0/3756876694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a24157540 0x7f7a2415e010 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7a24160ac0 tx=0x7f7a1402a270 comp rx=0 tx=0).stop 2024-05-08T05:41:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.891+0000 7f7a2ab04640 1 -- 172.21.15.45:0/3756876694 shutdown_connections 2024-05-08T05:41:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.891+0000 7f7a2ab04640 1 --2- 172.21.15.45:0/3756876694 >> v2:172.21.15.63:3300/0 conn(0x7f7a2406fc30 0x7f7a240af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.891+0000 7f7a2ab04640 1 --2- 172.21.15.45:0/3756876694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a24157540 0x7f7a2415e010 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.891+0000 7f7a2ab04640 1 -- 172.21.15.45:0/3756876694 >> 172.21.15.45:0/3756876694 conn(0x7f7a2406c3a0 msgr2=0x7f7a2406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.891+0000 7f7a2ab04640 1 -- 172.21.15.45:0/3756876694 shutdown_connections 2024-05-08T05:41:59.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.891+0000 7f7a2ab04640 1 -- 172.21.15.45:0/3756876694 wait complete. 2024-05-08T05:41:59.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.893+0000 7f7a2ab04640 1 Processor -- start 2024-05-08T05:41:59.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.893+0000 7f7a2ab04640 1 -- start start 2024-05-08T05:41:59.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.894+0000 7f7a2ab04640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a2406fc30 0x7f7a24151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:59.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.894+0000 7f7a2ab04640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a240b48a0 0x7f7a24151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:59.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.894+0000 7f7a2ab04640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a24157540 0x7f7a241f2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:59.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.894+0000 7f7a2ab04640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7a24157aa0 con 0x7f7a240b48a0 2024-05-08T05:41:59.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.894+0000 7f7a2ab04640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7a24157920 con 0x7f7a24157540 2024-05-08T05:41:59.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.894+0000 7f7a2ab04640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7a240afc80 con 0x7f7a2406fc30 2024-05-08T05:41:59.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.894+0000 7f7a23fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a240b48a0 0x7f7a24151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:59.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.894+0000 7f7a23fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a240b48a0 0x7f7a24151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34846/0 (socket says 172.21.15.45:34846) 2024-05-08T05:41:59.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.894+0000 7f7a23fff640 1 -- 172.21.15.45:0/1181256223 learned_addr learned my addr 172.21.15.45:0/1181256223 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:41:59.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.894+0000 7f7a290a1640 1 --2- 172.21.15.45:0/1181256223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a24157540 0x7f7a241f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:59.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.894+0000 7f7a288a0640 1 --2- 172.21.15.45:0/1181256223 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a2406fc30 0x7f7a24151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:59.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.894+0000 7f7a23fff640 1 -- 172.21.15.45:0/1181256223 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a2406fc30 msgr2=0x7f7a24151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:59.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.894+0000 7f7a23fff640 1 --2- 172.21.15.45:0/1181256223 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a2406fc30 0x7f7a24151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:59.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.894+0000 7f7a23fff640 1 -- 172.21.15.45:0/1181256223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a24157540 msgr2=0x7f7a241f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:59.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.894+0000 7f7a23fff640 1 --2- 172.21.15.45:0/1181256223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a24157540 0x7f7a241f2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:59.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.894+0000 7f7a23fff640 1 -- 172.21.15.45:0/1181256223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a24159640 con 0x7f7a240b48a0 2024-05-08T05:41:59.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.894+0000 7f7a23fff640 1 --2- 172.21.15.45:0/1181256223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a240b48a0 0x7f7a24151fd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800e9b0 tx=0x7f7a1800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:59.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.894+0000 7f7a217fa640 1 -- 172.21.15.45:0/1181256223 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18017070 con 0x7f7a240b48a0 2024-05-08T05:41:59.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.894+0000 7f7a217fa640 1 -- 172.21.15.45:0/1181256223 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a1800c9a0 con 0x7f7a240b48a0 2024-05-08T05:41:59.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.894+0000 7f7a2ab04640 1 -- 172.21.15.45:0/1181256223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a24152840 con 0x7f7a240b48a0 2024-05-08T05:41:59.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.895+0000 7f7a217fa640 1 -- 172.21.15.45:0/1181256223 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1800cd90 con 0x7f7a240b48a0 2024-05-08T05:41:59.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.895+0000 7f7a2ab04640 1 -- 172.21.15.45:0/1181256223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a24152e00 con 0x7f7a240b48a0 2024-05-08T05:41:59.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.895+0000 7f7a2ab04640 1 -- 172.21.15.45:0/1181256223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a240693f0 con 0x7f7a240b48a0 2024-05-08T05:41:59.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.896+0000 7f7a217fa640 1 -- 172.21.15.45:0/1181256223 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7a1801a070 con 0x7f7a240b48a0 2024-05-08T05:41:59.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.897+0000 7f7a217fa640 1 --2- 172.21.15.45:0/1181256223 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7a100762d0 0x7f7a10078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:59.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.897+0000 7f7a217fa640 1 -- 172.21.15.45:0/1181256223 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f7a18097880 con 0x7f7a240b48a0 2024-05-08T05:41:59.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.897+0000 7f7a288a0640 1 --2- 172.21.15.45:0/1181256223 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7a100762d0 0x7f7a10078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:59.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.897+0000 7f7a288a0640 1 --2- 172.21.15.45:0/1181256223 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7a100762d0 0x7f7a10078790 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7a14034000 tx=0x7f7a1402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:59.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:41:59.901+0000 7f7a217fa640 1 -- 172.21.15.45:0/1181256223 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7a18060850 con 0x7f7a240b48a0 2024-05-08T05:42:00.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:00.129+0000 7f7a2ab04640 1 -- 172.21.15.45:0/1181256223 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7a240008d0 con 0x7f7a240b48a0 2024-05-08T05:42:00.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:00.130+0000 7f7a217fa640 1 -- 172.21.15.45:0/1181256223 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f7a18064b50 con 0x7f7a240b48a0 2024-05-08T05:42:00.130 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T05:42:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:00.132+0000 7f7a2ab04640 1 -- 172.21.15.45:0/1181256223 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7a100762d0 msgr2=0x7f7a10078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:00.132+0000 7f7a2ab04640 1 --2- 172.21.15.45:0/1181256223 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7a100762d0 0x7f7a10078790 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7a14034000 tx=0x7f7a1402e000 comp rx=0 tx=0).stop 2024-05-08T05:42:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:00.132+0000 7f7a2ab04640 1 -- 172.21.15.45:0/1181256223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a240b48a0 msgr2=0x7f7a24151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:00.132+0000 7f7a2ab04640 1 --2- 172.21.15.45:0/1181256223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a240b48a0 0x7f7a24151fd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800e9b0 tx=0x7f7a1800ee70 comp rx=0 tx=0).stop 2024-05-08T05:42:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:00.132+0000 7f7a2ab04640 1 -- 172.21.15.45:0/1181256223 shutdown_connections 2024-05-08T05:42:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:00.132+0000 7f7a2ab04640 1 --2- 172.21.15.45:0/1181256223 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7a100762d0 0x7f7a10078790 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:00.132+0000 7f7a2ab04640 1 --2- 172.21.15.45:0/1181256223 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a24157540 0x7f7a241f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:00.132+0000 7f7a2ab04640 1 --2- 172.21.15.45:0/1181256223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a240b48a0 0x7f7a24151fd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:00.132+0000 7f7a2ab04640 1 --2- 172.21.15.45:0/1181256223 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a2406fc30 0x7f7a24151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:00.132+0000 7f7a2ab04640 1 -- 172.21.15.45:0/1181256223 >> 172.21.15.45:0/1181256223 conn(0x7f7a2406c3a0 msgr2=0x7f7a2415b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:00.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:00.132+0000 7f7a2ab04640 1 -- 172.21.15.45:0/1181256223 shutdown_connections 2024-05-08T05:42:00.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:00.132+0000 7f7a2ab04640 1 -- 172.21.15.45:0/1181256223 wait complete. 2024-05-08T05:42:00.143 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T05:42:00.144 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:42:00.144 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi063.front.sepia.ceph.com 2024-05-08T05:42:00.144 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-08T05:42:00.144 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-08T05:42:00.144 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T05:42:00.144 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T05:42:00.144 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:42:00.144 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-08T05:42:00.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:00.144 DEBUG:teuthology.orchestra.run.smithi063:> ip addr 2024-05-08T05:42:00.163 INFO:teuthology.orchestra.run.smithi063.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:42:00.163 INFO:teuthology.orchestra.run.smithi063.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:42:00.163 INFO:teuthology.orchestra.run.smithi063.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:42:00.163 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:00.163 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 ::1/128 scope host 2024-05-08T05:42:00.163 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:00.163 INFO:teuthology.orchestra.run.smithi063.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:42:00.163 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:76 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:42:00.163 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f0 2024-05-08T05:42:00.163 INFO:teuthology.orchestra.run.smithi063.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:42:00.163 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:77 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:42:00.163 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f1 2024-05-08T05:42:00.164 INFO:teuthology.orchestra.run.smithi063.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:42:00.164 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d6 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:42:00.164 INFO:teuthology.orchestra.run.smithi063.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:42:00.164 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d7 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:42:00.164 INFO:teuthology.orchestra.run.smithi063.stdout: inet 172.21.15.63/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-08T05:42:00.164 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft 42680sec preferred_lft 42680sec 2024-05-08T05:42:00.164 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::ec4:7aff:feb6:f8d7/64 scope link 2024-05-08T05:42:00.164 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:00.164 INFO:teuthology.orchestra.run.smithi063.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:42:00.164 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether b2:b5:31:22:15:df brd ff:ff:ff:ff:ff:ff 2024-05-08T05:42:00.164 INFO:teuthology.orchestra.run.smithi063.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T05:42:00.164 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:00.164 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::207d:92ff:fe7a:da45/64 scope link 2024-05-08T05:42:00.164 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:00.165 INFO:teuthology.orchestra.run.smithi063.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:42:00.165 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether b2:b5:31:22:15:df brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T05:42:00.165 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::b0b5:31ff:fe22:15df/64 scope link tentative 2024-05-08T05:42:00.165 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:00.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:00.165 DEBUG:teuthology.orchestra.run.smithi063:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:42:00.230 INFO:teuthology.orchestra.run.smithi063.stdout:1 2024-05-08T05:42:00.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:00.232 DEBUG:teuthology.orchestra.run.smithi063:> route 2024-05-08T05:42:00.290 INFO:teuthology.orchestra.run.smithi063.stdout:Kernel IP routing table 2024-05-08T05:42:00.290 INFO:teuthology.orchestra.run.smithi063.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:42:00.290 INFO:teuthology.orchestra.run.smithi063.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-08T05:42:00.290 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-08T05:42:00.290 INFO:teuthology.orchestra.run.smithi063.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:42:00.290 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:42:00.290 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-08T05:42:00.291 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-08T05:42:00.291 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-08T05:42:00.291 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-08T05:42:00.291 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-08T05:42:00.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:42:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:42:00.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:42:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:42:00.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:00.426 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-08T05:42:00.441 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-08T05:42:00.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:00.442 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list-id 2024-05-08T05:42:00.498 INFO:teuthology.orchestra.run.smithi063.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-08T05:42:00.498 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:42:00.498 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-08T05:42:00.498 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-08T05:42:00.498 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-08T05:42:00.498 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-08T05:42:00.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:42:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:42:00.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:42:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:42:00.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:00.619 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-08T05:42:00.644 INFO:teuthology.orchestra.run.smithi063.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:42:00.644 INFO:teuthology.orchestra.run.smithi063.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:42:00.644 INFO:teuthology.orchestra.run.smithi063.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:42:00.644 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:00.644 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 ::1/128 scope host 2024-05-08T05:42:00.645 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:00.645 INFO:teuthology.orchestra.run.smithi063.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:42:00.645 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether a2:d7:13:dc:95:39 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T05:42:00.645 INFO:teuthology.orchestra.run.smithi063.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T05:42:00.645 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:00.645 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::a0d7:13ff:fedc:9539/64 scope link tentative 2024-05-08T05:42:00.645 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:00.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:00.646 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-08T05:42:00.711 INFO:teuthology.orchestra.run.smithi063.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:42:00.711 INFO:teuthology.orchestra.run.smithi063.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:42:00.711 INFO:teuthology.orchestra.run.smithi063.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:42:00.711 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:00.711 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 ::1/128 scope host 2024-05-08T05:42:00.712 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:00.712 INFO:teuthology.orchestra.run.smithi063.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:42:00.712 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether a2:d7:13:dc:95:39 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-08T05:42:00.712 INFO:teuthology.orchestra.run.smithi063.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-08T05:42:00.712 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:00.712 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::a0d7:13ff:fedc:9539/64 scope link tentative 2024-05-08T05:42:00.712 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:00.713 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-08T05:42:00.713 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:42:00.713 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-08T05:42:00.713 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-08T05:42:00.713 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-08T05:42:00.713 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-08T05:42:00.713 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-08T05:42:00.713 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-08T05:42:00.713 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-08T05:42:00.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:42:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:42:00.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:42:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:42:00.871 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:42:00.871 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-05-08T05:42:00.871 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set brx.1 up 2024-05-08T05:42:00.871 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set dev brx.1 master ceph-brx 2024-05-08T05:42:00.871 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-05-08T05:42:00.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:42:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:42:00.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:42:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:42:00.950 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:42:00.950 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-08T05:42:00.966 INFO:teuthology.orchestra.run.smithi063.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-08T05:42:00.966 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:42:00.966 DEBUG:teuthology.orchestra.run.smithi063:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-08T05:42:01.021 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-05-08T05:42:01.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:01.022 DEBUG:teuthology.orchestra.run.smithi063:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:42:01.107 INFO:teuthology.orchestra.run.smithi063.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:42:01.107 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:42:01.114 INFO:teuthology.orchestra.run.smithi063.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:42:01.115 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:42:01.115 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-08T05:42:01.115 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: trying mount with new device syntax: 3@e9e1170a-fd27-4625-97a9-5d5f104d13a2.cephfs=/ 2024-05-08T05:42:01.115 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.45:3300/172.21.15.63:3300/172.21.15.63:3301" will pass to kernel 2024-05-08T05:42:01.115 INFO:teuthology.orchestra.run.smithi063.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-05-08T05:42:01.116 INFO:teuthology.orchestra.run.smithi063.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:42:01.116 INFO:teuthology.orchestra.run.smithi063.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:42:01.116 INFO:teuthology.orchestra.run.smithi063.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:42:01.116 INFO:teuthology.orchestra.run.smithi063.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:42:01.117 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:42:01.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:01.118 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-08T05:42:01.143 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:42:01.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:42:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:42:01.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:42:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:42:01.241 INFO:teuthology.orchestra.run.smithi063.stdout:client4651 2024-05-08T05:42:01.242 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:42:01.242 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:01.258 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:42:01.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:42:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:42:01.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:42:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:42:01.358 INFO:teuthology.orchestra.run.smithi063.stdout:client4651 2024-05-08T05:42:01.358 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:42:01.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:42:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:42:01.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:42:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:42:01.416 INFO:teuthology.orchestra.run.smithi063.stdout:client4651 2024-05-08T05:42:01.416 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e9e1170a-fd27-4625-97a9-5d5f104d13a2.client4651/status') 2024-05-08T05:42:01.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:42:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:42:01.476 INFO:teuthology.orchestra.run.smithi063.stdout:instance: client.4651 (3)192.168.0.2:0/569973341 2024-05-08T05:42:01.477 INFO:teuthology.orchestra.run.smithi063.stdout:blocklisted: false 2024-05-08T05:42:01.477 INFO:teuthology.orchestra.run.smithi063.stderr:0+1 records in 2024-05-08T05:42:01.477 INFO:teuthology.orchestra.run.smithi063.stderr:0+1 records out 2024-05-08T05:42:01.477 INFO:teuthology.orchestra.run.smithi063.stderr:68 bytes copied, 2.458e-05 s, 2.8 MB/s 2024-05-08T05:42:01.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-08T05:42:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:42:01.484 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-08T05:42:01.511 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-08T05:42:01.511 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:42:01.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.627+0000 7f850afb6640 1 Processor -- start 2024-05-08T05:42:01.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.627+0000 7f850afb6640 1 -- start start 2024-05-08T05:42:01.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.628+0000 7f850afb6640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f850415f420 0x7f85040d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.628+0000 7f850afb6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f85040d3ca0 0x7f85040d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.628+0000 7f850afb6640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85041630b0 con 0x7f85040d1eb0 2024-05-08T05:42:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.628+0000 7f850afb6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f85040d2550 con 0x7f85040d3ca0 2024-05-08T05:42:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.628+0000 7f850afb6640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f85040d2c80 con 0x7f850415f420 2024-05-08T05:42:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.628+0000 7f84fbfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f85040d3ca0 0x7f85040d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.628+0000 7f84fbfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f85040d3ca0 0x7f85040d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48916/0 (socket says 172.21.15.45:48916) 2024-05-08T05:42:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.628+0000 7f84fbfff640 1 -- 172.21.15.45:0/109187688 learned_addr learned my addr 172.21.15.45:0/109187688 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.628+0000 7f8508d52640 1 --2- 172.21.15.45:0/109187688 >> v2:172.21.15.63:3300/0 conn(0x7f850415f420 0x7f85040d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.628+0000 7f84fbfff640 1 -- 172.21.15.45:0/109187688 >> v2:172.21.15.63:3300/0 conn(0x7f850415f420 msgr2=0x7f85040d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.628+0000 7f84fbfff640 1 --2- 172.21.15.45:0/109187688 >> v2:172.21.15.63:3300/0 conn(0x7f850415f420 0x7f85040d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.628+0000 7f84fbfff640 1 -- 172.21.15.45:0/109187688 >> v1:172.21.15.63:6789/0 conn(0x7f85040d1eb0 legacy=0x7f85040d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.628+0000 7f84fbfff640 1 -- 172.21.15.45:0/109187688 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85040d2f60 con 0x7f85040d3ca0 2024-05-08T05:42:01.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.628+0000 7f8508d52640 1 --2- 172.21.15.45:0/109187688 >> v2:172.21.15.63:3300/0 conn(0x7f850415f420 0x7f85040d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.629+0000 7f84fbfff640 1 --2- 172.21.15.45:0/109187688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85040d3ca0 0x7f85040d1970 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f85040d5ec0 tx=0x7f84f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfc4691ef0cf0d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.629+0000 7f84fb7fe640 1 -- 172.21.15.45:0/109187688 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4037070 con 0x7f85040d3ca0 2024-05-08T05:42:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.629+0000 7f84fb7fe640 1 -- 172.21.15.45:0/109187688 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f4032440 con 0x7f85040d3ca0 2024-05-08T05:42:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.629+0000 7f84fb7fe640 1 -- 172.21.15.45:0/109187688 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4032810 con 0x7f85040d3ca0 2024-05-08T05:42:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.630+0000 7f850afb6640 1 -- 172.21.15.45:0/109187688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85040d3ca0 msgr2=0x7f85040d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.630+0000 7f850afb6640 1 --2- 172.21.15.45:0/109187688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85040d3ca0 0x7f85040d1970 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f85040d5ec0 tx=0x7f84f402a1e0 comp rx=0 tx=0).stop 2024-05-08T05:42:01.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.630+0000 7f850afb6640 1 -- 172.21.15.45:0/109187688 shutdown_connections 2024-05-08T05:42:01.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.630+0000 7f850afb6640 1 --2- 172.21.15.45:0/109187688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85040d3ca0 0x7f85040d1970 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:01.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.630+0000 7f850afb6640 1 --2- 172.21.15.45:0/109187688 >> v2:172.21.15.63:3300/0 conn(0x7f850415f420 0x7f85040d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:01.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.630+0000 7f850afb6640 1 -- 172.21.15.45:0/109187688 >> 172.21.15.45:0/109187688 conn(0x7f850408eac0 msgr2=0x7f850409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:01.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.630+0000 7f850afb6640 1 -- 172.21.15.45:0/109187688 shutdown_connections 2024-05-08T05:42:01.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.630+0000 7f850afb6640 1 -- 172.21.15.45:0/109187688 wait complete. 2024-05-08T05:42:01.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.631+0000 7f850afb6640 1 Processor -- start 2024-05-08T05:42:01.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.631+0000 7f850afb6640 1 -- start start 2024-05-08T05:42:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.632+0000 7f850afb6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85040d1eb0 0x7f85040cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.632+0000 7f850afb6640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8504158090 0x7f8504161890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.632+0000 7f850afb6640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8504161dd0 0x7f85040c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.632+0000 7f850afb6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f85040d53d0 con 0x7f85040d1eb0 2024-05-08T05:42:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.632+0000 7f850afb6640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f85040d2830 con 0x7f8504161dd0 2024-05-08T05:42:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.632+0000 7f850afb6640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f85040d5550 con 0x7f8504158090 2024-05-08T05:42:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.632+0000 7f8508d52640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85040d1eb0 0x7f85040cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.632+0000 7f8508d52640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85040d1eb0 0x7f85040cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48922/0 (socket says 172.21.15.45:48922) 2024-05-08T05:42:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.632+0000 7f8508d52640 1 -- 172.21.15.45:0/1416824077 learned_addr learned my addr 172.21.15.45:0/1416824077 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.632+0000 7f8509553640 1 --2- 172.21.15.45:0/1416824077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8504161dd0 0x7f85040c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.632+0000 7f84fbfff640 1 --2- 172.21.15.45:0/1416824077 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8504158090 0x7f8504161890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.633+0000 7f8508d52640 1 -- 172.21.15.45:0/1416824077 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8504158090 msgr2=0x7f8504161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.633+0000 7f8508d52640 1 --2- 172.21.15.45:0/1416824077 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8504158090 0x7f8504161890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.633+0000 7f8508d52640 1 -- 172.21.15.45:0/1416824077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8504161dd0 msgr2=0x7f85040c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.633+0000 7f8508d52640 1 --2- 172.21.15.45:0/1416824077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8504161dd0 0x7f85040c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.633+0000 7f8508d52640 1 -- 172.21.15.45:0/1416824077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8504161150 con 0x7f85040d1eb0 2024-05-08T05:42:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.633+0000 7f8508d52640 1 --2- 172.21.15.45:0/1416824077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85040d1eb0 0x7f85040cc0e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f850000c7f0 tx=0x7f850000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.633+0000 7f84f97fa640 1 -- 172.21.15.45:0/1416824077 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500015070 con 0x7f85040d1eb0 2024-05-08T05:42:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.633+0000 7f84f97fa640 1 -- 172.21.15.45:0/1416824077 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8500007820 con 0x7f85040d1eb0 2024-05-08T05:42:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.633+0000 7f850afb6640 1 -- 172.21.15.45:0/1416824077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8504158b20 con 0x7f85040d1eb0 2024-05-08T05:42:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.633+0000 7f84f97fa640 1 -- 172.21.15.45:0/1416824077 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500007bd0 con 0x7f85040d1eb0 2024-05-08T05:42:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.633+0000 7f850afb6640 1 -- 172.21.15.45:0/1416824077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85040cf080 con 0x7f85040d1eb0 2024-05-08T05:42:01.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.634+0000 7f850afb6640 1 -- 172.21.15.45:0/1416824077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85041621e0 con 0x7f85040d1eb0 2024-05-08T05:42:01.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.635+0000 7f84f97fa640 1 -- 172.21.15.45:0/1416824077 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8500018070 con 0x7f85040d1eb0 2024-05-08T05:42:01.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.635+0000 7f84f97fa640 1 --2- 172.21.15.45:0/1416824077 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f84f00762d0 0x7f84f0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:01.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.635+0000 7f84f97fa640 1 -- 172.21.15.45:0/1416824077 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8500097900 con 0x7f85040d1eb0 2024-05-08T05:42:01.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.636+0000 7f84fbfff640 1 --2- 172.21.15.45:0/1416824077 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f84f00762d0 0x7f84f0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:01.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.636+0000 7f84fbfff640 1 --2- 172.21.15.45:0/1416824077 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f84f00762d0 0x7f84f0078790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f84f4034000 tx=0x7f84f402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:01.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.640+0000 7f84f97fa640 1 -- 172.21.15.45:0/1416824077 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f85041621e0 con 0x7f85040d1eb0 2024-05-08T05:42:01.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.834+0000 7f850afb6640 1 -- 172.21.15.45:0/1416824077 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85040b4c50 con 0x7f85040d1eb0 2024-05-08T05:42:01.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.836+0000 7f84f97fa640 1 -- 172.21.15.45:0/1416824077 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18466 (secure 0 0 0) 0x7f8500060960 con 0x7f85040d1eb0 2024-05-08T05:42:01.836 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:01.836 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:41:36.532045+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:41:29.381274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:41:30.443612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:41:34.132285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:42:01.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.838+0000 7f850afb6640 1 -- 172.21.15.45:0/1416824077 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f84f00762d0 msgr2=0x7f84f0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:01.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.838+0000 7f850afb6640 1 --2- 172.21.15.45:0/1416824077 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f84f00762d0 0x7f84f0078790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f84f4034000 tx=0x7f84f402e000 comp rx=0 tx=0).stop 2024-05-08T05:42:01.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.838+0000 7f850afb6640 1 -- 172.21.15.45:0/1416824077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85040d1eb0 msgr2=0x7f85040cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:01.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.838+0000 7f850afb6640 1 --2- 172.21.15.45:0/1416824077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85040d1eb0 0x7f85040cc0e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f850000c7f0 tx=0x7f850000ccb0 comp rx=0 tx=0).stop 2024-05-08T05:42:01.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.838+0000 7f850afb6640 1 -- 172.21.15.45:0/1416824077 shutdown_connections 2024-05-08T05:42:01.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.838+0000 7f850afb6640 1 --2- 172.21.15.45:0/1416824077 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f84f00762d0 0x7f84f0078790 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:01.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.838+0000 7f850afb6640 1 --2- 172.21.15.45:0/1416824077 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8504158090 0x7f8504161890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:01.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.838+0000 7f850afb6640 1 --2- 172.21.15.45:0/1416824077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8504161dd0 0x7f85040c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:01.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.838+0000 7f850afb6640 1 --2- 172.21.15.45:0/1416824077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85040d1eb0 0x7f85040cc0e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:01.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.838+0000 7f850afb6640 1 -- 172.21.15.45:0/1416824077 >> 172.21.15.45:0/1416824077 conn(0x7f850408eac0 msgr2=0x7f85040b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:01.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.838+0000 7f850afb6640 1 -- 172.21.15.45:0/1416824077 shutdown_connections 2024-05-08T05:42:01.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.838+0000 7f850afb6640 1 -- 172.21.15.45:0/1416824077 wait complete. 2024-05-08T05:42:01.849 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T05:41:18.847604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T05:41:29.381274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T05:41:30.443612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T05:41:34.132285+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T05:42:01.850 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T05:42:01.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.956+0000 7fc9fd119640 1 Processor -- start 2024-05-08T05:42:01.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.956+0000 7fc9fd119640 1 -- start start 2024-05-08T05:42:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.957+0000 7fc9fd119640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc9f80d3cd0 0x7fc9f80d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.957+0000 7fc9fd119640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9f80b3f30 0x7fc9f80d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:01.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.957+0000 7fc9fd119640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9f80b1c90 con 0x7fc9f815f380 2024-05-08T05:42:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.957+0000 7fc9fd119640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc9f80d0a50 con 0x7fc9f80d3cd0 2024-05-08T05:42:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.957+0000 7fc9fd119640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc9f80d0d30 con 0x7fc9f80b3f30 2024-05-08T05:42:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.957+0000 7fc9f659c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc9f80d3cd0 0x7fc9f80d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.957+0000 7fc9f6d9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc9f815f380 0x7fc9f80d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53152/0 (socket says 172.21.15.45:53152) 2024-05-08T05:42:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.957+0000 7fc9f6d9d640 1 -- 172.21.15.45:0/3086898114 learned_addr learned my addr 172.21.15.45:0/3086898114 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.957+0000 7fc9f759e640 1 --2- 172.21.15.45:0/3086898114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9f80b3f30 0x7fc9f80d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.957+0000 7fc9f659c640 1 -- 172.21.15.45:0/3086898114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9f80b3f30 msgr2=0x7fc9f80d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.957+0000 7fc9f659c640 1 --2- 172.21.15.45:0/3086898114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9f80b3f30 0x7fc9f80d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.957+0000 7fc9f659c640 1 -- 172.21.15.45:0/3086898114 >> v1:172.21.15.45:6789/0 conn(0x7fc9f815f380 legacy=0x7fc9f80d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.957+0000 7fc9f5d9b640 1 -- 172.21.15.45:0/3086898114 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4235924114 0 0) 0x7fc9f80b1c90 con 0x7fc9f815f380 2024-05-08T05:42:01.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.957+0000 7fc9f659c640 1 -- 172.21.15.45:0/3086898114 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f80b1f00 con 0x7fc9f80d3cd0 2024-05-08T05:42:01.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.958+0000 7fc9f659c640 1 --2- 172.21.15.45:0/3086898114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f80d3cd0 0x7fc9f80d15f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0097a0 tx=0x7fc9ec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3212f6b22db91c1d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:01.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.958+0000 7fc9f5d9b640 1 -- 172.21.15.45:0/3086898114 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec037070 con 0x7fc9f80d3cd0 2024-05-08T05:42:01.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.958+0000 7fc9f5d9b640 1 -- 172.21.15.45:0/3086898114 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9ec032440 con 0x7fc9f80d3cd0 2024-05-08T05:42:01.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.958+0000 7fc9f5d9b640 1 -- 172.21.15.45:0/3086898114 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec032810 con 0x7fc9f80d3cd0 2024-05-08T05:42:01.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.958+0000 7fc9fd119640 1 -- 172.21.15.45:0/3086898114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f80d3cd0 msgr2=0x7fc9f80d15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:01.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.958+0000 7fc9fd119640 1 --2- 172.21.15.45:0/3086898114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f80d3cd0 0x7fc9f80d15f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0097a0 tx=0x7fc9ec02a180 comp rx=0 tx=0).stop 2024-05-08T05:42:01.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.958+0000 7fc9fd119640 1 -- 172.21.15.45:0/3086898114 shutdown_connections 2024-05-08T05:42:01.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.958+0000 7fc9fd119640 1 --2- 172.21.15.45:0/3086898114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9f80b3f30 0x7fc9f80d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:01.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.958+0000 7fc9fd119640 1 --2- 172.21.15.45:0/3086898114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f80d3cd0 0x7fc9f80d15f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:01.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.958+0000 7fc9fd119640 1 -- 172.21.15.45:0/3086898114 >> 172.21.15.45:0/3086898114 conn(0x7fc9f80d4fe0 msgr2=0x7fc9f80d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:01.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.959+0000 7fc9fd119640 1 -- 172.21.15.45:0/3086898114 shutdown_connections 2024-05-08T05:42:01.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.959+0000 7fc9fd119640 1 -- 172.21.15.45:0/3086898114 wait complete. 2024-05-08T05:42:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.960+0000 7fc9fd119640 1 Processor -- start 2024-05-08T05:42:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.960+0000 7fc9fd119640 1 -- start start 2024-05-08T05:42:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.961+0000 7fc9fd119640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9f80b3f30 0x7fc9f8158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.961+0000 7fc9fd119640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f80d3cd0 0x7fc9f80b9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.961+0000 7fc9fd119640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9f815f380 0x7fc9f815b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.961+0000 7fc9fd119640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f80d06e0 con 0x7fc9f80d3cd0 2024-05-08T05:42:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.961+0000 7fc9fd119640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f80d1010 con 0x7fc9f80b3f30 2024-05-08T05:42:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.961+0000 7fc9fd119640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc9f80b3430 con 0x7fc9f815f380 2024-05-08T05:42:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.961+0000 7fc9f659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f80d3cd0 0x7fc9f80b9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:01.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.961+0000 7fc9f659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f80d3cd0 0x7fc9f80b9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48942/0 (socket says 172.21.15.45:48942) 2024-05-08T05:42:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.961+0000 7fc9f659c640 1 -- 172.21.15.45:0/3651907323 learned_addr learned my addr 172.21.15.45:0/3651907323 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.961+0000 7fc9f6d9d640 1 --2- 172.21.15.45:0/3651907323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9f80b3f30 0x7fc9f8158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.961+0000 7fc9f759e640 1 --2- 172.21.15.45:0/3651907323 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9f815f380 0x7fc9f815b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.961+0000 7fc9f659c640 1 -- 172.21.15.45:0/3651907323 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9f815f380 msgr2=0x7fc9f815b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.961+0000 7fc9f659c640 1 --2- 172.21.15.45:0/3651907323 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9f815f380 0x7fc9f815b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.961+0000 7fc9f659c640 1 -- 172.21.15.45:0/3651907323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9f80b3f30 msgr2=0x7fc9f8158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.961+0000 7fc9f659c640 1 --2- 172.21.15.45:0/3651907323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9f80b3f30 0x7fc9f8158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.961+0000 7fc9f659c640 1 -- 172.21.15.45:0/3651907323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f80bad30 con 0x7fc9f80d3cd0 2024-05-08T05:42:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.961+0000 7fc9f759e640 1 --2- 172.21.15.45:0/3651907323 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9f815f380 0x7fc9f815b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.962+0000 7fc9f659c640 1 --2- 172.21.15.45:0/3651907323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f80d3cd0 0x7fc9f80b9fb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0098c0 tx=0x7fc9ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.962+0000 7fc9df7fe640 1 -- 172.21.15.45:0/3651907323 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec037040 con 0x7fc9f80d3cd0 2024-05-08T05:42:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.962+0000 7fc9df7fe640 1 -- 172.21.15.45:0/3651907323 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9ec030770 con 0x7fc9f80d3cd0 2024-05-08T05:42:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.962+0000 7fc9fd119640 1 -- 172.21.15.45:0/3651907323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f80baa50 con 0x7fc9f80d3cd0 2024-05-08T05:42:01.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.962+0000 7fc9df7fe640 1 -- 172.21.15.45:0/3651907323 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec030b40 con 0x7fc9f80d3cd0 2024-05-08T05:42:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.962+0000 7fc9fd119640 1 -- 172.21.15.45:0/3651907323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f80b7ed0 con 0x7fc9f80d3cd0 2024-05-08T05:42:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.963+0000 7fc9fd119640 1 -- 172.21.15.45:0/3651907323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f80693f0 con 0x7fc9f80d3cd0 2024-05-08T05:42:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.963+0000 7fc9df7fe640 1 -- 172.21.15.45:0/3651907323 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc9ec044020 con 0x7fc9f80d3cd0 2024-05-08T05:42:01.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.964+0000 7fc9df7fe640 1 --2- 172.21.15.45:0/3651907323 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc9e40762d0 0x7fc9e4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.964+0000 7fc9df7fe640 1 -- 172.21.15.45:0/3651907323 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fc9ec0b7f50 con 0x7fc9f80d3cd0 2024-05-08T05:42:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.964+0000 7fc9f6d9d640 1 --2- 172.21.15.45:0/3651907323 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc9e40762d0 0x7fc9e4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.964+0000 7fc9f6d9d640 1 --2- 172.21.15.45:0/3651907323 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc9e40762d0 0x7fc9e4078790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8000f60 tx=0x7fc9e8007c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:01.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:01.968+0000 7fc9df7fe640 1 -- 172.21.15.45:0/3651907323 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc9ec032e20 con 0x7fc9f80d3cd0 2024-05-08T05:42:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.158+0000 7fc9fd119640 1 -- 172.21.15.45:0/3651907323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc9f80d36f0 con 0x7fc9f80d3cd0 2024-05-08T05:42:02.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.158+0000 7fc9df7fe640 1 -- 172.21.15.45:0/3651907323 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7fc9ec081570 con 0x7fc9f80d3cd0 2024-05-08T05:42:02.159 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-05-08T05:42:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.160+0000 7fc9fd119640 1 -- 172.21.15.45:0/3651907323 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc9e40762d0 msgr2=0x7fc9e4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.161+0000 7fc9fd119640 1 --2- 172.21.15.45:0/3651907323 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc9e40762d0 0x7fc9e4078790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8000f60 tx=0x7fc9e8007c00 comp rx=0 tx=0).stop 2024-05-08T05:42:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.161+0000 7fc9fd119640 1 -- 172.21.15.45:0/3651907323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f80d3cd0 msgr2=0x7fc9f80b9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.161+0000 7fc9fd119640 1 --2- 172.21.15.45:0/3651907323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f80d3cd0 0x7fc9f80b9fb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0098c0 tx=0x7fc9ec0046c0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.161+0000 7fc9fd119640 1 -- 172.21.15.45:0/3651907323 shutdown_connections 2024-05-08T05:42:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.161+0000 7fc9fd119640 1 --2- 172.21.15.45:0/3651907323 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc9e40762d0 0x7fc9e4078790 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.161+0000 7fc9fd119640 1 --2- 172.21.15.45:0/3651907323 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9f815f380 0x7fc9f815b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.161+0000 7fc9fd119640 1 --2- 172.21.15.45:0/3651907323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9f80d3cd0 0x7fc9f80b9fb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.161+0000 7fc9fd119640 1 --2- 172.21.15.45:0/3651907323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9f80b3f30 0x7fc9f8158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.161+0000 7fc9fd119640 1 -- 172.21.15.45:0/3651907323 >> 172.21.15.45:0/3651907323 conn(0x7fc9f80d4fe0 msgr2=0x7fc9f80d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.161+0000 7fc9fd119640 1 -- 172.21.15.45:0/3651907323 shutdown_connections 2024-05-08T05:42:02.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.161+0000 7fc9fd119640 1 -- 172.21.15.45:0/3651907323 wait complete. 2024-05-08T05:42:02.175 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T05:42:02.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.283+0000 7f4fdbbe4640 1 Processor -- start 2024-05-08T05:42:02.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.283+0000 7f4fdbbe4640 1 -- start start 2024-05-08T05:42:02.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.284+0000 7f4fdbbe4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4fd41629b0 0x7f4fd406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:02.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.284+0000 7f4fdbbe4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fd406e110 con 0x7f4fd4070d20 2024-05-08T05:42:02.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.284+0000 7f4fdbbe4640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fd406e3f0 con 0x7f4fd40b2490 2024-05-08T05:42:02.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.284+0000 7f4fdbbe4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4fd4070a40 con 0x7f4fd41629b0 2024-05-08T05:42:02.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.284+0000 7f4fd9980640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4fd41629b0 0x7f4fd406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:02.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.284+0000 7f4fda181640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4fd4070d20 0x7f4fd40afcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53158/0 (socket says 172.21.15.45:53158) 2024-05-08T05:42:02.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.284+0000 7f4fda181640 1 -- 172.21.15.45:0/1640617091 learned_addr learned my addr 172.21.15.45:0/1640617091 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:02.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.284+0000 7f4fd9980640 1 -- 172.21.15.45:0/1640617091 >> v1:172.21.15.63:6789/0 conn(0x7f4fd40b2490 legacy=0x7f4fd4070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.284+0000 7f4fd9980640 1 -- 172.21.15.45:0/1640617091 >> v1:172.21.15.45:6789/0 conn(0x7f4fd4070d20 legacy=0x7f4fd40afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.284+0000 7f4fd9980640 1 -- 172.21.15.45:0/1640617091 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd406e6d0 con 0x7f4fd41629b0 2024-05-08T05:42:02.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.285+0000 7f4fd9980640 1 --2- 172.21.15.45:0/1640617091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fd41629b0 0x7f4fd406eed0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00097a0 tx=0x7f4fd002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593bc1cd5f739008 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:02.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.285+0000 7f4fd897e640 1 -- 172.21.15.45:0/1640617091 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0037070 con 0x7f4fd41629b0 2024-05-08T05:42:02.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.285+0000 7f4fd897e640 1 -- 172.21.15.45:0/1640617091 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd0032440 con 0x7f4fd41629b0 2024-05-08T05:42:02.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.285+0000 7f4fd897e640 1 -- 172.21.15.45:0/1640617091 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0032810 con 0x7f4fd41629b0 2024-05-08T05:42:02.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.285+0000 7f4fdbbe4640 1 -- 172.21.15.45:0/1640617091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fd41629b0 msgr2=0x7f4fd406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.285+0000 7f4fdbbe4640 1 --2- 172.21.15.45:0/1640617091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fd41629b0 0x7f4fd406eed0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00097a0 tx=0x7f4fd002a230 comp rx=0 tx=0).stop 2024-05-08T05:42:02.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.285+0000 7f4fdbbe4640 1 -- 172.21.15.45:0/1640617091 shutdown_connections 2024-05-08T05:42:02.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.285+0000 7f4fdbbe4640 1 --2- 172.21.15.45:0/1640617091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fd41629b0 0x7f4fd406eed0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.285+0000 7f4fdbbe4640 1 -- 172.21.15.45:0/1640617091 >> 172.21.15.45:0/1640617091 conn(0x7f4fd40b49d0 msgr2=0x7f4fd40b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:02.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.286+0000 7f4fdbbe4640 1 -- 172.21.15.45:0/1640617091 shutdown_connections 2024-05-08T05:42:02.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.286+0000 7f4fdbbe4640 1 -- 172.21.15.45:0/1640617091 wait complete. 2024-05-08T05:42:02.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.287+0000 7f4fdbbe4640 1 Processor -- start 2024-05-08T05:42:02.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.287+0000 7f4fdbbe4640 1 -- start start 2024-05-08T05:42:02.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.287+0000 7f4fdbbe4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4fd4070d20 0x7f4fd40c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:02.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.287+0000 7f4fdbbe4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fd40b2490 0x7f4fd41f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:02.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.287+0000 7f4fdbbe4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fd41629b0 0x7f4fd41f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:02.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.287+0000 7f4fdbbe4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd406f410 con 0x7f4fd40b2490 2024-05-08T05:42:02.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.287+0000 7f4fdbbe4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd406dec0 con 0x7f4fd41629b0 2024-05-08T05:42:02.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.287+0000 7f4fdbbe4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd40b03c0 con 0x7f4fd4070d20 2024-05-08T05:42:02.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.288+0000 7f4fd917f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fd40b2490 0x7f4fd41f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:02.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.288+0000 7f4fd917f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fd40b2490 0x7f4fd41f4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48970/0 (socket says 172.21.15.45:48970) 2024-05-08T05:42:02.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.288+0000 7f4fd917f640 1 -- 172.21.15.45:0/2335774868 learned_addr learned my addr 172.21.15.45:0/2335774868 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:02.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.288+0000 7f4fd9980640 1 --2- 172.21.15.45:0/2335774868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4fd4070d20 0x7f4fd40c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:02.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.288+0000 7f4fda181640 1 --2- 172.21.15.45:0/2335774868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fd41629b0 0x7f4fd41f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:02.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.288+0000 7f4fd917f640 1 -- 172.21.15.45:0/2335774868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4fd4070d20 msgr2=0x7f4fd40c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.288+0000 7f4fd917f640 1 --2- 172.21.15.45:0/2335774868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4fd4070d20 0x7f4fd40c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.288+0000 7f4fd917f640 1 -- 172.21.15.45:0/2335774868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fd41629b0 msgr2=0x7f4fd41f5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.288+0000 7f4fd917f640 1 --2- 172.21.15.45:0/2335774868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fd41629b0 0x7f4fd41f5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.288+0000 7f4fd917f640 1 -- 172.21.15.45:0/2335774868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd40c79a0 con 0x7f4fd40b2490 2024-05-08T05:42:02.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.288+0000 7f4fd917f640 1 --2- 172.21.15.45:0/2335774868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fd40b2490 0x7f4fd41f4dc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400c450 tx=0x7f4fc400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:02.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.288+0000 7f4fca7fc640 1 -- 172.21.15.45:0/2335774868 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc40052d0 con 0x7f4fd40b2490 2024-05-08T05:42:02.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.288+0000 7f4fca7fc640 1 -- 172.21.15.45:0/2335774868 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fc4005470 con 0x7f4fd40b2490 2024-05-08T05:42:02.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.288+0000 7f4fdbbe4640 1 -- 172.21.15.45:0/2335774868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd40c7ce0 con 0x7f4fd40b2490 2024-05-08T05:42:02.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.288+0000 7f4fca7fc640 1 -- 172.21.15.45:0/2335774868 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4012870 con 0x7f4fd40b2490 2024-05-08T05:42:02.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.288+0000 7f4fdbbe4640 1 -- 172.21.15.45:0/2335774868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd40c8520 con 0x7f4fd40b2490 2024-05-08T05:42:02.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.289+0000 7f4fdbbe4640 1 -- 172.21.15.45:0/2335774868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd40693f0 con 0x7f4fd40b2490 2024-05-08T05:42:02.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.290+0000 7f4fca7fc640 1 -- 172.21.15.45:0/2335774868 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4fc4012a10 con 0x7f4fd40b2490 2024-05-08T05:42:02.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.291+0000 7f4fca7fc640 1 --2- 172.21.15.45:0/2335774868 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4fb80762d0 0x7f4fb8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:02.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.291+0000 7f4fca7fc640 1 -- 172.21.15.45:0/2335774868 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f4fc4097350 con 0x7f4fd40b2490 2024-05-08T05:42:02.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.291+0000 7f4fd9980640 1 --2- 172.21.15.45:0/2335774868 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4fb80762d0 0x7f4fb8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:02.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.291+0000 7f4fd9980640 1 --2- 172.21.15.45:0/2335774868 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4fb80762d0 0x7f4fb8078790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0034000 tx=0x7f4fd002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:02.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.295+0000 7f4fca7fc640 1 -- 172.21.15.45:0/2335774868 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4fd40c7ce0 con 0x7f4fd40b2490 2024-05-08T05:42:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.486+0000 7f4fdbbe4640 1 -- 172.21.15.45:0/2335774868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4fd40c7ec0 con 0x7f4fd40b2490 2024-05-08T05:42:02.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.486+0000 7f4fca7fc640 1 -- 172.21.15.45:0/2335774868 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) ==== 104+0+11 (secure 0 0 0) 0x7f4fc40603b0 con 0x7f4fd40b2490 2024-05-08T05:42:02.486 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-05-08T05:42:02.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.488+0000 7f4fdbbe4640 1 -- 172.21.15.45:0/2335774868 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4fb80762d0 msgr2=0x7f4fb8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.488+0000 7f4fdbbe4640 1 --2- 172.21.15.45:0/2335774868 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4fb80762d0 0x7f4fb8078790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0034000 tx=0x7f4fd002e000 comp rx=0 tx=0).stop 2024-05-08T05:42:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.488+0000 7f4fdbbe4640 1 -- 172.21.15.45:0/2335774868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fd40b2490 msgr2=0x7f4fd41f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.488+0000 7f4fdbbe4640 1 --2- 172.21.15.45:0/2335774868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fd40b2490 0x7f4fd41f4dc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400c450 tx=0x7f4fc400c910 comp rx=0 tx=0).stop 2024-05-08T05:42:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.488+0000 7f4fdbbe4640 1 -- 172.21.15.45:0/2335774868 shutdown_connections 2024-05-08T05:42:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.488+0000 7f4fdbbe4640 1 --2- 172.21.15.45:0/2335774868 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4fb80762d0 0x7f4fb8078790 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.488+0000 7f4fdbbe4640 1 --2- 172.21.15.45:0/2335774868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4fd41629b0 0x7f4fd41f5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.488+0000 7f4fdbbe4640 1 --2- 172.21.15.45:0/2335774868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4fd40b2490 0x7f4fd41f4dc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.488+0000 7f4fdbbe4640 1 --2- 172.21.15.45:0/2335774868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4fd4070d20 0x7f4fd40c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.488+0000 7f4fdbbe4640 1 -- 172.21.15.45:0/2335774868 >> 172.21.15.45:0/2335774868 conn(0x7f4fd40b49d0 msgr2=0x7f4fd40afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.488+0000 7f4fdbbe4640 1 -- 172.21.15.45:0/2335774868 shutdown_connections 2024-05-08T05:42:02.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.488+0000 7f4fdbbe4640 1 -- 172.21.15.45:0/2335774868 wait complete. 2024-05-08T05:42:02.499 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T05:42:02.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.605+0000 7fde9395f640 1 Processor -- start 2024-05-08T05:42:02.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.606+0000 7fde9395f640 1 -- start start 2024-05-08T05:42:02.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.606+0000 7fde9395f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde8c15f4e0 0x7fde8c0cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:02.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.606+0000 7fde9395f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fde8c0d56b0 0x7fde8c0d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:02.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.606+0000 7fde9395f640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde8c0d0320 con 0x7fde8c0d3aa0 2024-05-08T05:42:02.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.606+0000 7fde9395f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fde8c0d0600 con 0x7fde8c0d56b0 2024-05-08T05:42:02.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.606+0000 7fde9395f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fde8c0d08e0 con 0x7fde8c15f4e0 2024-05-08T05:42:02.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.607+0000 7fde91efc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fde8c0d56b0 0x7fde8c0d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:02.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.607+0000 7fde916fb640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde8c15f4e0 0x7fde8c0cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:02.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.607+0000 7fde91efc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fde8c0d56b0 0x7fde8c0d13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:41300/0 (socket says 172.21.15.45:41300) 2024-05-08T05:42:02.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.607+0000 7fde91efc640 1 -- 172.21.15.45:0/3072119068 learned_addr learned my addr 172.21.15.45:0/3072119068 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:02.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.607+0000 7fde91efc640 1 -- 172.21.15.45:0/3072119068 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde8c15f4e0 msgr2=0x7fde8c0cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:02.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.607+0000 7fde91efc640 1 --2- 172.21.15.45:0/3072119068 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde8c15f4e0 0x7fde8c0cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.607+0000 7fde91efc640 1 -- 172.21.15.45:0/3072119068 >> v1:172.21.15.63:6789/0 conn(0x7fde8c0d3aa0 legacy=0x7fde8c0d4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.607+0000 7fde91efc640 1 -- 172.21.15.45:0/3072119068 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8c0d0bc0 con 0x7fde8c0d56b0 2024-05-08T05:42:02.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.608+0000 7fde91efc640 1 --2- 172.21.15.45:0/3072119068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde8c0d56b0 0x7fde8c0d13c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fde84009e50 tx=0x7fde8402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6523fbeb2651c2eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:02.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.608+0000 7fde83fff640 1 -- 172.21.15.45:0/3072119068 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84036070 con 0x7fde8c0d56b0 2024-05-08T05:42:02.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.608+0000 7fde83fff640 1 -- 172.21.15.45:0/3072119068 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde84031600 con 0x7fde8c0d56b0 2024-05-08T05:42:02.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.608+0000 7fde83fff640 1 -- 172.21.15.45:0/3072119068 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde840319d0 con 0x7fde8c0d56b0 2024-05-08T05:42:02.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.609+0000 7fde9395f640 1 -- 172.21.15.45:0/3072119068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde8c0d56b0 msgr2=0x7fde8c0d13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.609+0000 7fde9395f640 1 --2- 172.21.15.45:0/3072119068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde8c0d56b0 0x7fde8c0d13c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fde84009e50 tx=0x7fde8402a710 comp rx=0 tx=0).stop 2024-05-08T05:42:02.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.609+0000 7fde9395f640 1 -- 172.21.15.45:0/3072119068 shutdown_connections 2024-05-08T05:42:02.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.609+0000 7fde9395f640 1 --2- 172.21.15.45:0/3072119068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde8c0d56b0 0x7fde8c0d13c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.609+0000 7fde9395f640 1 --2- 172.21.15.45:0/3072119068 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde8c15f4e0 0x7fde8c0cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.609+0000 7fde9395f640 1 -- 172.21.15.45:0/3072119068 >> 172.21.15.45:0/3072119068 conn(0x7fde8c08eac0 msgr2=0x7fde8c09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:02.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.609+0000 7fde9395f640 1 -- 172.21.15.45:0/3072119068 shutdown_connections 2024-05-08T05:42:02.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.609+0000 7fde9395f640 1 -- 172.21.15.45:0/3072119068 wait complete. 2024-05-08T05:42:02.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.610+0000 7fde9395f640 1 Processor -- start 2024-05-08T05:42:02.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.610+0000 7fde9395f640 1 -- start start 2024-05-08T05:42:02.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.611+0000 7fde9395f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde8c0d3aa0 0x7fde8c156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:02.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.611+0000 7fde9395f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde8c0d56b0 0x7fde8c156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:02.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.611+0000 7fde9395f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde8c15f4e0 0x7fde8c0d1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:02.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.611+0000 7fde9395f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c0d3f00 con 0x7fde8c15f4e0 2024-05-08T05:42:02.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.611+0000 7fde9395f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c15f8c0 con 0x7fde8c0d3aa0 2024-05-08T05:42:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.611+0000 7fde9395f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fde8c0d4080 con 0x7fde8c0d56b0 2024-05-08T05:42:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.611+0000 7fde91efc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde8c15f4e0 0x7fde8c0d1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.611+0000 7fde916fb640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde8c0d3aa0 0x7fde8c156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.611+0000 7fde91efc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde8c15f4e0 0x7fde8c0d1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48980/0 (socket says 172.21.15.45:48980) 2024-05-08T05:42:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.611+0000 7fde91efc640 1 -- 172.21.15.45:0/4146424199 learned_addr learned my addr 172.21.15.45:0/4146424199 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.611+0000 7fde90efa640 1 --2- 172.21.15.45:0/4146424199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde8c0d56b0 0x7fde8c156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.611+0000 7fde91efc640 1 -- 172.21.15.45:0/4146424199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde8c0d56b0 msgr2=0x7fde8c156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.611+0000 7fde91efc640 1 --2- 172.21.15.45:0/4146424199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde8c0d56b0 0x7fde8c156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.611+0000 7fde91efc640 1 -- 172.21.15.45:0/4146424199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde8c0d3aa0 msgr2=0x7fde8c156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.611+0000 7fde91efc640 1 --2- 172.21.15.45:0/4146424199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde8c0d3aa0 0x7fde8c156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.611+0000 7fde91efc640 1 -- 172.21.15.45:0/4146424199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8c157520 con 0x7fde8c15f4e0 2024-05-08T05:42:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.611+0000 7fde916fb640 1 --2- 172.21.15.45:0/4146424199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde8c0d3aa0 0x7fde8c156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:42:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.611+0000 7fde91efc640 1 --2- 172.21.15.45:0/4146424199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde8c15f4e0 0x7fde8c0d1c60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fde84034040 tx=0x7fde84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.612+0000 7fde81ffb640 1 -- 172.21.15.45:0/4146424199 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84041020 con 0x7fde8c15f4e0 2024-05-08T05:42:02.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.612+0000 7fde81ffb640 1 -- 172.21.15.45:0/4146424199 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde840077a0 con 0x7fde8c15f4e0 2024-05-08T05:42:02.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.612+0000 7fde81ffb640 1 -- 172.21.15.45:0/4146424199 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84036040 con 0x7fde8c15f4e0 2024-05-08T05:42:02.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.612+0000 7fde9395f640 1 -- 172.21.15.45:0/4146424199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde8c15fc00 con 0x7fde8c15f4e0 2024-05-08T05:42:02.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.612+0000 7fde9395f640 1 -- 172.21.15.45:0/4146424199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde8c1601c0 con 0x7fde8c15f4e0 2024-05-08T05:42:02.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.613+0000 7fde9395f640 1 -- 172.21.15.45:0/4146424199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde8c0693f0 con 0x7fde8c15f4e0 2024-05-08T05:42:02.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.613+0000 7fde81ffb640 1 -- 172.21.15.45:0/4146424199 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fde84044020 con 0x7fde8c15f4e0 2024-05-08T05:42:02.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.614+0000 7fde81ffb640 1 --2- 172.21.15.45:0/4146424199 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fde700762d0 0x7fde70078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:02.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.614+0000 7fde81ffb640 1 -- 172.21.15.45:0/4146424199 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fde840b8420 con 0x7fde8c15f4e0 2024-05-08T05:42:02.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.615+0000 7fde916fb640 1 --2- 172.21.15.45:0/4146424199 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fde700762d0 0x7fde70078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:02.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.615+0000 7fde916fb640 1 --2- 172.21.15.45:0/4146424199 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fde700762d0 0x7fde70078790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fde88009e90 tx=0x7fde88009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:02.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.619+0000 7fde81ffb640 1 -- 172.21.15.45:0/4146424199 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fde8404b030 con 0x7fde8c15f4e0 2024-05-08T05:42:02.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.808+0000 7fde9395f640 1 -- 172.21.15.45:0/4146424199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fde8c0b1de0 con 0x7fde8c15f4e0 2024-05-08T05:42:02.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.809+0000 7fde81ffb640 1 -- 172.21.15.45:0/4146424199 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) ==== 100+0+11 (secure 0 0 0) 0x7fde8c157520 con 0x7fde8c15f4e0 2024-05-08T05:42:02.809 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-05-08T05:42:02.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.811+0000 7fde9395f640 1 -- 172.21.15.45:0/4146424199 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fde700762d0 msgr2=0x7fde70078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.811+0000 7fde9395f640 1 --2- 172.21.15.45:0/4146424199 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fde700762d0 0x7fde70078790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fde88009e90 tx=0x7fde88009190 comp rx=0 tx=0).stop 2024-05-08T05:42:02.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.811+0000 7fde9395f640 1 -- 172.21.15.45:0/4146424199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde8c15f4e0 msgr2=0x7fde8c0d1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.811+0000 7fde9395f640 1 --2- 172.21.15.45:0/4146424199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde8c15f4e0 0x7fde8c0d1c60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fde84034040 tx=0x7fde84004a40 comp rx=0 tx=0).stop 2024-05-08T05:42:02.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.811+0000 7fde9395f640 1 -- 172.21.15.45:0/4146424199 shutdown_connections 2024-05-08T05:42:02.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.811+0000 7fde9395f640 1 --2- 172.21.15.45:0/4146424199 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fde700762d0 0x7fde70078790 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.811+0000 7fde9395f640 1 --2- 172.21.15.45:0/4146424199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fde8c0d56b0 0x7fde8c156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.811+0000 7fde9395f640 1 --2- 172.21.15.45:0/4146424199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde8c15f4e0 0x7fde8c0d1c60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.811+0000 7fde9395f640 1 --2- 172.21.15.45:0/4146424199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde8c0d3aa0 0x7fde8c156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.811+0000 7fde9395f640 1 -- 172.21.15.45:0/4146424199 >> 172.21.15.45:0/4146424199 conn(0x7fde8c08eac0 msgr2=0x7fde8c06df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:02.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.811+0000 7fde9395f640 1 -- 172.21.15.45:0/4146424199 shutdown_connections 2024-05-08T05:42:02.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.811+0000 7fde9395f640 1 -- 172.21.15.45:0/4146424199 wait complete. 2024-05-08T05:42:02.822 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T05:42:02.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.928+0000 7f685707e640 1 Processor -- start 2024-05-08T05:42:02.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.929+0000 7f685707e640 1 -- start start 2024-05-08T05:42:02.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.929+0000 7f685707e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f68500cfcd0 0x7f68500d4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:02.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.929+0000 7f685707e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68500d54e0 0x7f68500d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:02.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.929+0000 7f685707e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68500d0320 con 0x7f685015f4e0 2024-05-08T05:42:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.929+0000 7f685707e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f68500d41b0 con 0x7f68500cfcd0 2024-05-08T05:42:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.929+0000 7f685707e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f68500d0600 con 0x7f68500d54e0 2024-05-08T05:42:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.929+0000 7f6854e1a640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f685015f4e0 0x7f68500d3aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53170/0 (socket says 172.21.15.45:53170) 2024-05-08T05:42:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.930+0000 7f6854e1a640 1 -- 172.21.15.45:0/1862710516 learned_addr learned my addr 172.21.15.45:0/1862710516 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.929+0000 7f684ffff640 1 --2- 172.21.15.45:0/1862710516 >> v2:172.21.15.45:3300/0 conn(0x7f68500cfcd0 0x7f68500d4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.930+0000 7f685561b640 1 --2- 172.21.15.45:0/1862710516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68500d54e0 0x7f68500d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.930+0000 7f684f7fe640 1 -- 172.21.15.45:0/1862710516 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4200251588 0 0) 0x7f68500d0320 con 0x7f685015f4e0 2024-05-08T05:42:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.930+0000 7f684f7fe640 1 -- 172.21.15.45:0/1862710516 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6838003520 con 0x7f685015f4e0 2024-05-08T05:42:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.930+0000 7f684ffff640 1 -- 172.21.15.45:0/1862710516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68500d54e0 msgr2=0x7f68500d13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.930+0000 7f684ffff640 1 --2- 172.21.15.45:0/1862710516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68500d54e0 0x7f68500d13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.930+0000 7f684ffff640 1 -- 172.21.15.45:0/1862710516 >> v1:172.21.15.45:6789/0 conn(0x7f685015f4e0 legacy=0x7f68500d3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.930+0000 7f684ffff640 1 -- 172.21.15.45:0/1862710516 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68500d08e0 con 0x7f68500cfcd0 2024-05-08T05:42:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.931+0000 7f684ffff640 1 --2- 172.21.15.45:0/1862710516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68500cfcd0 0x7f68500d4fa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f68440097a0 tx=0x7f684402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb616d2f650ca1cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.931+0000 7f684f7fe640 1 -- 172.21.15.45:0/1862710516 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6844037070 con 0x7f68500cfcd0 2024-05-08T05:42:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.931+0000 7f684f7fe640 1 -- 172.21.15.45:0/1862710516 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6844032440 con 0x7f68500cfcd0 2024-05-08T05:42:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.931+0000 7f684f7fe640 1 -- 172.21.15.45:0/1862710516 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68440327f0 con 0x7f68500cfcd0 2024-05-08T05:42:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.931+0000 7f685707e640 1 -- 172.21.15.45:0/1862710516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68500cfcd0 msgr2=0x7f68500d4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.931+0000 7f685707e640 1 --2- 172.21.15.45:0/1862710516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68500cfcd0 0x7f68500d4fa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f68440097a0 tx=0x7f684402a180 comp rx=0 tx=0).stop 2024-05-08T05:42:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.931+0000 7f685707e640 1 -- 172.21.15.45:0/1862710516 shutdown_connections 2024-05-08T05:42:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.931+0000 7f685707e640 1 --2- 172.21.15.45:0/1862710516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68500d54e0 0x7f68500d13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.931+0000 7f685707e640 1 --2- 172.21.15.45:0/1862710516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68500cfcd0 0x7f68500d4fa0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.931+0000 7f685707e640 1 -- 172.21.15.45:0/1862710516 >> 172.21.15.45:0/1862710516 conn(0x7f685008eac0 msgr2=0x7f685009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.931+0000 7f685707e640 1 -- 172.21.15.45:0/1862710516 shutdown_connections 2024-05-08T05:42:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.931+0000 7f685707e640 1 -- 172.21.15.45:0/1862710516 wait complete. 2024-05-08T05:42:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.933+0000 7f685707e640 1 Processor -- start 2024-05-08T05:42:02.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.933+0000 7f685707e640 1 -- start start 2024-05-08T05:42:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.933+0000 7f685707e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68500cfcd0 0x7f685015d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.933+0000 7f685707e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68500d54e0 0x7f6850162510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.933+0000 7f685707e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f685015f4e0 0x7f68500b9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.933+0000 7f685707e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f685015f8c0 con 0x7f685015f4e0 2024-05-08T05:42:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.933+0000 7f685707e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68500d0ea0 con 0x7f68500cfcd0 2024-05-08T05:42:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.933+0000 7f685707e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f68500d1900 con 0x7f68500d54e0 2024-05-08T05:42:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.934+0000 7f685561b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f685015f4e0 0x7f68500b9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.934+0000 7f685561b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f685015f4e0 0x7f68500b9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48998/0 (socket says 172.21.15.45:48998) 2024-05-08T05:42:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.934+0000 7f685561b640 1 -- 172.21.15.45:0/663613980 learned_addr learned my addr 172.21.15.45:0/663613980 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.934+0000 7f6854e1a640 1 --2- 172.21.15.45:0/663613980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68500cfcd0 0x7f685015d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.934+0000 7f685561b640 1 -- 172.21.15.45:0/663613980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68500d54e0 msgr2=0x7f6850162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.934+0000 7f685561b640 1 --2- 172.21.15.45:0/663613980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68500d54e0 0x7f6850162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.934+0000 7f685561b640 1 -- 172.21.15.45:0/663613980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68500cfcd0 msgr2=0x7f685015d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.934+0000 7f685561b640 1 --2- 172.21.15.45:0/663613980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68500cfcd0 0x7f685015d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.934+0000 7f685561b640 1 -- 172.21.15.45:0/663613980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6850162a50 con 0x7f685015f4e0 2024-05-08T05:42:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.934+0000 7f685561b640 1 --2- 172.21.15.45:0/663613980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f685015f4e0 0x7f68500b9bb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f684000e960 tx=0x7f684000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.934+0000 7f684d7fa640 1 -- 172.21.15.45:0/663613980 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840017070 con 0x7f685015f4e0 2024-05-08T05:42:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.934+0000 7f684d7fa640 1 -- 172.21.15.45:0/663613980 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f684000c980 con 0x7f685015f4e0 2024-05-08T05:42:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.934+0000 7f684d7fa640 1 -- 172.21.15.45:0/663613980 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684000cd70 con 0x7f685015f4e0 2024-05-08T05:42:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.934+0000 7f685707e640 1 -- 172.21.15.45:0/663613980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6850160e70 con 0x7f685015f4e0 2024-05-08T05:42:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.934+0000 7f685707e640 1 -- 172.21.15.45:0/663613980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6850161430 con 0x7f685015f4e0 2024-05-08T05:42:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.935+0000 7f685707e640 1 -- 172.21.15.45:0/663613980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6850068600 con 0x7f685015f4e0 2024-05-08T05:42:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.936+0000 7f684d7fa640 1 -- 172.21.15.45:0/663613980 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f684001a070 con 0x7f685015f4e0 2024-05-08T05:42:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.936+0000 7f684d7fa640 1 --2- 172.21.15.45:0/663613980 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f683c076300 0x7f683c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.936+0000 7f684d7fa640 1 -- 172.21.15.45:0/663613980 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6840097830 con 0x7f685015f4e0 2024-05-08T05:42:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.936+0000 7f6854e1a640 1 --2- 172.21.15.45:0/663613980 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f683c076300 0x7f683c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.937+0000 7f6854e1a640 1 --2- 172.21.15.45:0/663613980 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f683c076300 0x7f683c0787c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f68480015a0 tx=0x7f6848009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:02.941+0000 7f684d7fa640 1 -- 172.21.15.45:0/663613980 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6850068600 con 0x7f685015f4e0 2024-05-08T05:42:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.130+0000 7f685707e640 1 -- 172.21.15.45:0/663613980 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f68500d1330 con 0x7f685015f4e0 2024-05-08T05:42:03.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.131+0000 7f684d7fa640 1 -- 172.21.15.45:0/663613980 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) ==== 103+0+11 (secure 0 0 0) 0x7f6840060ee0 con 0x7f685015f4e0 2024-05-08T05:42:03.131 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-05-08T05:42:03.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.133+0000 7f685707e640 1 -- 172.21.15.45:0/663613980 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f683c076300 msgr2=0x7f683c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:03.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.133+0000 7f685707e640 1 --2- 172.21.15.45:0/663613980 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f683c076300 0x7f683c0787c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f68480015a0 tx=0x7f6848009040 comp rx=0 tx=0).stop 2024-05-08T05:42:03.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.133+0000 7f685707e640 1 -- 172.21.15.45:0/663613980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f685015f4e0 msgr2=0x7f68500b9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:03.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.133+0000 7f685707e640 1 --2- 172.21.15.45:0/663613980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f685015f4e0 0x7f68500b9bb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f684000e960 tx=0x7f684000ee20 comp rx=0 tx=0).stop 2024-05-08T05:42:03.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.133+0000 7f685707e640 1 -- 172.21.15.45:0/663613980 shutdown_connections 2024-05-08T05:42:03.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.133+0000 7f685707e640 1 --2- 172.21.15.45:0/663613980 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f683c076300 0x7f683c0787c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:03.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.133+0000 7f685707e640 1 --2- 172.21.15.45:0/663613980 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68500d54e0 0x7f6850162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:03.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.133+0000 7f685707e640 1 --2- 172.21.15.45:0/663613980 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f685015f4e0 0x7f68500b9bb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:03.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.133+0000 7f685707e640 1 --2- 172.21.15.45:0/663613980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68500cfcd0 0x7f685015d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:03.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.133+0000 7f685707e640 1 -- 172.21.15.45:0/663613980 >> 172.21.15.45:0/663613980 conn(0x7f685008eac0 msgr2=0x7f6850075840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:03.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.133+0000 7f685707e640 1 -- 172.21.15.45:0/663613980 shutdown_connections 2024-05-08T05:42:03.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.133+0000 7f685707e640 1 -- 172.21.15.45:0/663613980 wait complete. 2024-05-08T05:42:03.144 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-05-08T05:42:03.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.248+0000 7fef675c8640 1 Processor -- start 2024-05-08T05:42:03.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.248+0000 7fef675c8640 1 -- start start 2024-05-08T05:42:03.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.249+0000 7fef675c8640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fef600d1970 0x7fef600d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:03.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.249+0000 7fef675c8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef600d4cc0 0x7fef600d5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:03.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.249+0000 7fef675c8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef601630b0 con 0x7fef6015f420 2024-05-08T05:42:03.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.249+0000 7fef675c8640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fef600d3e70 con 0x7fef600d1970 2024-05-08T05:42:03.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.249+0000 7fef675c8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fef600d5670 con 0x7fef600d4cc0 2024-05-08T05:42:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.249+0000 7fef65364640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fef6015f420 0x7fef600d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53186/0 (socket says 172.21.15.45:53186) 2024-05-08T05:42:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.249+0000 7fef65364640 1 -- 172.21.15.45:0/1537681919 learned_addr learned my addr 172.21.15.45:0/1537681919 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.250+0000 7fef65b65640 1 --2- 172.21.15.45:0/1537681919 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef600d4cc0 0x7fef600d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.250+0000 7fef64b63640 1 --2- 172.21.15.45:0/1537681919 >> v2:172.21.15.63:3300/0 conn(0x7fef600d1970 0x7fef600d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.250+0000 7fef5ffff640 1 -- 172.21.15.45:0/1537681919 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1320390282 0 0) 0x7fef58001110 con 0x7fef6015f420 2024-05-08T05:42:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.250+0000 7fef5ffff640 1 -- 172.21.15.45:0/1537681919 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef48003520 con 0x7fef6015f420 2024-05-08T05:42:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.250+0000 7fef5ffff640 1 -- 172.21.15.45:0/1537681919 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3065401484 0 0) 0x7fef48003520 con 0x7fef6015f420 2024-05-08T05:42:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.250+0000 7fef65b65640 1 -- 172.21.15.45:0/1537681919 >> v2:172.21.15.63:3300/0 conn(0x7fef600d1970 msgr2=0x7fef600d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.250+0000 7fef65b65640 1 --2- 172.21.15.45:0/1537681919 >> v2:172.21.15.63:3300/0 conn(0x7fef600d1970 0x7fef600d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.250+0000 7fef65b65640 1 -- 172.21.15.45:0/1537681919 >> v1:172.21.15.45:6789/0 conn(0x7fef6015f420 legacy=0x7fef600d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.250+0000 7fef65b65640 1 -- 172.21.15.45:0/1537681919 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef600d2550 con 0x7fef600d4cc0 2024-05-08T05:42:03.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.250+0000 7fef64b63640 1 --2- 172.21.15.45:0/1537681919 >> v2:172.21.15.63:3300/0 conn(0x7fef600d1970 0x7fef600d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:42:03.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.251+0000 7fef65b65640 1 --2- 172.21.15.45:0/1537681919 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef600d4cc0 0x7fef600d5130 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fef50009d90 tx=0x7fef5002a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6f1ef960f8cc263 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:03.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.251+0000 7fef5ffff640 1 -- 172.21.15.45:0/1537681919 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef580018e0 con 0x7fef600d4cc0 2024-05-08T05:42:03.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.251+0000 7fef5ffff640 1 -- 172.21.15.45:0/1537681919 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef50007540 con 0x7fef600d4cc0 2024-05-08T05:42:03.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.251+0000 7fef5ffff640 1 -- 172.21.15.45:0/1537681919 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef500078f0 con 0x7fef600d4cc0 2024-05-08T05:42:03.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.251+0000 7fef675c8640 1 -- 172.21.15.45:0/1537681919 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef600d4cc0 msgr2=0x7fef600d5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:03.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.251+0000 7fef675c8640 1 --2- 172.21.15.45:0/1537681919 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef600d4cc0 0x7fef600d5130 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fef50009d90 tx=0x7fef5002a5f0 comp rx=0 tx=0).stop 2024-05-08T05:42:03.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.251+0000 7fef675c8640 1 -- 172.21.15.45:0/1537681919 shutdown_connections 2024-05-08T05:42:03.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.252+0000 7fef675c8640 1 --2- 172.21.15.45:0/1537681919 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef600d4cc0 0x7fef600d5130 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:03.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.252+0000 7fef675c8640 1 --2- 172.21.15.45:0/1537681919 >> v2:172.21.15.63:3300/0 conn(0x7fef600d1970 0x7fef600d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:03.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.252+0000 7fef675c8640 1 -- 172.21.15.45:0/1537681919 >> 172.21.15.45:0/1537681919 conn(0x7fef6008eac0 msgr2=0x7fef6009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:03.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.252+0000 7fef675c8640 1 -- 172.21.15.45:0/1537681919 shutdown_connections 2024-05-08T05:42:03.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.252+0000 7fef675c8640 1 -- 172.21.15.45:0/1537681919 wait complete. 2024-05-08T05:42:03.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.253+0000 7fef675c8640 1 Processor -- start 2024-05-08T05:42:03.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.253+0000 7fef675c8640 1 -- start start 2024-05-08T05:42:03.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef675c8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef600d1970 0x7fef600c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:03.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef675c8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef600d4cc0 0x7fef60159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:03.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef675c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef600cb6f0 0x7fef600b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef675c8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fef600d3240 con 0x7fef600cb6f0 2024-05-08T05:42:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef675c8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef600d2f60 con 0x7fef600d1970 2024-05-08T05:42:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef675c8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fef600d2830 con 0x7fef600d4cc0 2024-05-08T05:42:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef65b65640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef600cb6f0 0x7fef600b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef65b65640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef600cb6f0 0x7fef600b2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49006/0 (socket says 172.21.15.45:49006) 2024-05-08T05:42:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef65b65640 1 -- 172.21.15.45:0/27411346 learned_addr learned my addr 172.21.15.45:0/27411346 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef65364640 1 --2- 172.21.15.45:0/27411346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef600d1970 0x7fef600c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef64b63640 1 --2- 172.21.15.45:0/27411346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef600d4cc0 0x7fef60159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef65b65640 1 -- 172.21.15.45:0/27411346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef600d4cc0 msgr2=0x7fef60159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef65b65640 1 --2- 172.21.15.45:0/27411346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef600d4cc0 0x7fef60159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef65b65640 1 -- 172.21.15.45:0/27411346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef600d1970 msgr2=0x7fef600c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef65b65640 1 --2- 172.21.15.45:0/27411346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef600d1970 0x7fef600c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef65b65640 1 -- 172.21.15.45:0/27411346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6015a4d0 con 0x7fef600cb6f0 2024-05-08T05:42:03.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef65364640 1 --2- 172.21.15.45:0/27411346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef600d1970 0x7fef600c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:03.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef65b65640 1 --2- 172.21.15.45:0/27411346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef600cb6f0 0x7fef600b2c90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fef50009eb0 tx=0x7fef50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:03.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef5dffb640 1 -- 172.21.15.45:0/27411346 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50040070 con 0x7fef600cb6f0 2024-05-08T05:42:03.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef5dffb640 1 -- 172.21.15.45:0/27411346 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef50031c40 con 0x7fef600cb6f0 2024-05-08T05:42:03.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef675c8640 1 -- 172.21.15.45:0/27411346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef600ca650 con 0x7fef600cb6f0 2024-05-08T05:42:03.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.254+0000 7fef5dffb640 1 -- 172.21.15.45:0/27411346 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50009450 con 0x7fef600cb6f0 2024-05-08T05:42:03.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.255+0000 7fef675c8640 1 -- 172.21.15.45:0/27411346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef600cac10 con 0x7fef600cb6f0 2024-05-08T05:42:03.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.256+0000 7fef675c8640 1 -- 172.21.15.45:0/27411346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef600693f0 con 0x7fef600cb6f0 2024-05-08T05:42:03.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.256+0000 7fef5dffb640 1 -- 172.21.15.45:0/27411346 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fef50039040 con 0x7fef600cb6f0 2024-05-08T05:42:03.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.257+0000 7fef5dffb640 1 --2- 172.21.15.45:0/27411346 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef4c0762d0 0x7fef4c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:03.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.257+0000 7fef5dffb640 1 -- 172.21.15.45:0/27411346 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fef500b8d80 con 0x7fef600cb6f0 2024-05-08T05:42:03.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.257+0000 7fef65364640 1 --2- 172.21.15.45:0/27411346 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef4c0762d0 0x7fef4c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:03.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.258+0000 7fef65364640 1 --2- 172.21.15.45:0/27411346 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef4c0762d0 0x7fef4c078790 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fef58001410 tx=0x7fef58001200 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:03.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.261+0000 7fef5dffb640 1 -- 172.21.15.45:0/27411346 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fef6015a4d0 con 0x7fef600cb6f0 2024-05-08T05:42:03.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:03.531+0000 7fef675c8640 1 -- 172.21.15.45:0/27411346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7fef6015ebd0 con 0x7fef600cb6f0 2024-05-08T05:42:04.050 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:42:04.053 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:42:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.230+0000 7fef5dffb640 1 -- 172.21.15.45:0/27411346 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v51) ==== 145+0+0 (secure 0 0 0) 0x7fef50082310 con 0x7fef600cb6f0 2024-05-08T05:42:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.233+0000 7fef675c8640 1 -- 172.21.15.45:0/27411346 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef4c0762d0 msgr2=0x7fef4c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.233+0000 7fef675c8640 1 --2- 172.21.15.45:0/27411346 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef4c0762d0 0x7fef4c078790 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fef58001410 tx=0x7fef58001200 comp rx=0 tx=0).stop 2024-05-08T05:42:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.233+0000 7fef675c8640 1 -- 172.21.15.45:0/27411346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef600cb6f0 msgr2=0x7fef600b2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.233+0000 7fef675c8640 1 --2- 172.21.15.45:0/27411346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef600cb6f0 0x7fef600b2c90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fef50009eb0 tx=0x7fef50004a40 comp rx=0 tx=0).stop 2024-05-08T05:42:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.233+0000 7fef675c8640 1 -- 172.21.15.45:0/27411346 shutdown_connections 2024-05-08T05:42:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.233+0000 7fef675c8640 1 --2- 172.21.15.45:0/27411346 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef4c0762d0 0x7fef4c078790 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.233+0000 7fef675c8640 1 --2- 172.21.15.45:0/27411346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef600d4cc0 0x7fef60159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.233+0000 7fef675c8640 1 --2- 172.21.15.45:0/27411346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef600cb6f0 0x7fef600b2c90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.233+0000 7fef675c8640 1 --2- 172.21.15.45:0/27411346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef600d1970 0x7fef600c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.233+0000 7fef675c8640 1 -- 172.21.15.45:0/27411346 >> 172.21.15.45:0/27411346 conn(0x7fef6008eac0 msgr2=0x7fef6015a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.233+0000 7fef675c8640 1 -- 172.21.15.45:0/27411346 shutdown_connections 2024-05-08T05:42:04.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.233+0000 7fef675c8640 1 -- 172.21.15.45:0/27411346 wait complete. 2024-05-08T05:42:04.247 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T05:42:04.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.360+0000 7fe94f8c5640 1 Processor -- start 2024-05-08T05:42:04.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.360+0000 7fe94f8c5640 1 -- start start 2024-05-08T05:42:04.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.361+0000 7fe94f8c5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe9480b2490 0x7fe948070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:04.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.361+0000 7fe94f8c5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe948070d20 0x7fe9480711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:04.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.361+0000 7fe94f8c5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe94806e110 con 0x7fe9481629b0 2024-05-08T05:42:04.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.361+0000 7fe94f8c5640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe94806dec0 con 0x7fe9480b2490 2024-05-08T05:42:04.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.361+0000 7fe94f8c5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe94806e3f0 con 0x7fe948070d20 2024-05-08T05:42:04.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.361+0000 7fe94ce60640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe9480b2490 0x7fe948070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:04.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.361+0000 7fe94ce60640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe9480b2490 0x7fe948070330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49012/0 (socket says 172.21.15.45:49012) 2024-05-08T05:42:04.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.361+0000 7fe94ce60640 1 -- 172.21.15.45:0/1495091278 learned_addr learned my addr 172.21.15.45:0/1495091278 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:04.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.361+0000 7fe94de62640 1 --2- 172.21.15.45:0/1495091278 >> v2:172.21.15.63:3300/0 conn(0x7fe948070d20 0x7fe9480711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:04.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.361+0000 7fe94ce60640 1 -- 172.21.15.45:0/1495091278 >> v2:172.21.15.63:3300/0 conn(0x7fe948070d20 msgr2=0x7fe9480711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:04.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.361+0000 7fe94ce60640 1 --2- 172.21.15.45:0/1495091278 >> v2:172.21.15.63:3300/0 conn(0x7fe948070d20 0x7fe9480711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.361+0000 7fe94ce60640 1 -- 172.21.15.45:0/1495091278 >> v1:172.21.15.63:6789/0 conn(0x7fe9481629b0 legacy=0x7fe94806eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:04.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.361+0000 7fe94ce60640 1 -- 172.21.15.45:0/1495091278 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94806e6d0 con 0x7fe9480b2490 2024-05-08T05:42:04.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.362+0000 7fe94ce60640 1 --2- 172.21.15.45:0/1495091278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9480b2490 0x7fe948070330 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe948073f60 tx=0x7fe93802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2516662daec37afe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:04.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.362+0000 7fe93ffff640 1 -- 172.21.15.45:0/1495091278 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938037070 con 0x7fe9480b2490 2024-05-08T05:42:04.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.362+0000 7fe93ffff640 1 -- 172.21.15.45:0/1495091278 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe938032440 con 0x7fe9480b2490 2024-05-08T05:42:04.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.362+0000 7fe93ffff640 1 -- 172.21.15.45:0/1495091278 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938032810 con 0x7fe9480b2490 2024-05-08T05:42:04.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.362+0000 7fe94f8c5640 1 -- 172.21.15.45:0/1495091278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9480b2490 msgr2=0x7fe948070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:04.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.362+0000 7fe94f8c5640 1 --2- 172.21.15.45:0/1495091278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9480b2490 0x7fe948070330 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe948073f60 tx=0x7fe93802a270 comp rx=0 tx=0).stop 2024-05-08T05:42:04.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.363+0000 7fe94f8c5640 1 -- 172.21.15.45:0/1495091278 shutdown_connections 2024-05-08T05:42:04.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.363+0000 7fe94f8c5640 1 --2- 172.21.15.45:0/1495091278 >> v2:172.21.15.63:3300/0 conn(0x7fe948070d20 0x7fe9480711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.363+0000 7fe94f8c5640 1 --2- 172.21.15.45:0/1495091278 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9480b2490 0x7fe948070330 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.363+0000 7fe94f8c5640 1 -- 172.21.15.45:0/1495091278 >> 172.21.15.45:0/1495091278 conn(0x7fe9480b49d0 msgr2=0x7fe9480b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:04.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.363+0000 7fe94f8c5640 1 -- 172.21.15.45:0/1495091278 shutdown_connections 2024-05-08T05:42:04.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.363+0000 7fe94f8c5640 1 -- 172.21.15.45:0/1495091278 wait complete. 2024-05-08T05:42:04.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.364+0000 7fe94f8c5640 1 Processor -- start 2024-05-08T05:42:04.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.364+0000 7fe94f8c5640 1 -- start start 2024-05-08T05:42:04.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.365+0000 7fe94f8c5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe948070d20 0x7fe9480b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:04.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.365+0000 7fe94f8c5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9480b2490 0x7fe9480cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:04.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.365+0000 7fe94f8c5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9481629b0 0x7fe9480cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:04.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.365+0000 7fe94f8c5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe9480709f0 con 0x7fe9480b2490 2024-05-08T05:42:04.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.365+0000 7fe94f8c5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe948070870 con 0x7fe9481629b0 2024-05-08T05:42:04.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.365+0000 7fe94f8c5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe9480afcb0 con 0x7fe948070d20 2024-05-08T05:42:04.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.365+0000 7fe94ce60640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9480b2490 0x7fe9480cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:04.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.365+0000 7fe94ce60640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9480b2490 0x7fe9480cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49026/0 (socket says 172.21.15.45:49026) 2024-05-08T05:42:04.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.365+0000 7fe94ce60640 1 -- 172.21.15.45:0/2771417983 learned_addr learned my addr 172.21.15.45:0/2771417983 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:04.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.365+0000 7fe94de62640 1 --2- 172.21.15.45:0/2771417983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9481629b0 0x7fe9480cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:04.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.365+0000 7fe94d661640 1 --2- 172.21.15.45:0/2771417983 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe948070d20 0x7fe9480b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:04.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.365+0000 7fe94ce60640 1 -- 172.21.15.45:0/2771417983 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe948070d20 msgr2=0x7fe9480b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:04.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.365+0000 7fe94ce60640 1 --2- 172.21.15.45:0/2771417983 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe948070d20 0x7fe9480b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.365+0000 7fe94ce60640 1 -- 172.21.15.45:0/2771417983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9481629b0 msgr2=0x7fe9480cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:04.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.365+0000 7fe94ce60640 1 --2- 172.21.15.45:0/2771417983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9481629b0 0x7fe9480cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.365+0000 7fe94ce60640 1 -- 172.21.15.45:0/2771417983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9480d0460 con 0x7fe9480b2490 2024-05-08T05:42:04.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.365+0000 7fe94ce60640 1 --2- 172.21.15.45:0/2771417983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9480b2490 0x7fe9480cd730 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe938000c00 tx=0x7fe9380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:04.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.365+0000 7fe93dffb640 1 -- 172.21.15.45:0/2771417983 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938041070 con 0x7fe9480b2490 2024-05-08T05:42:04.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.366+0000 7fe93dffb640 1 -- 172.21.15.45:0/2771417983 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe93803c870 con 0x7fe9480b2490 2024-05-08T05:42:04.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.366+0000 7fe93dffb640 1 -- 172.21.15.45:0/2771417983 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938037040 con 0x7fe9480b2490 2024-05-08T05:42:04.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.366+0000 7fe94f8c5640 1 -- 172.21.15.45:0/2771417983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9480b6520 con 0x7fe9480b2490 2024-05-08T05:42:04.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.366+0000 7fe94f8c5640 1 -- 172.21.15.45:0/2771417983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9480d0180 con 0x7fe9480b2490 2024-05-08T05:42:04.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.367+0000 7fe94f8c5640 1 -- 172.21.15.45:0/2771417983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9480693f0 con 0x7fe9480b2490 2024-05-08T05:42:04.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.367+0000 7fe93dffb640 1 -- 172.21.15.45:0/2771417983 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe938044070 con 0x7fe9480b2490 2024-05-08T05:42:04.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.368+0000 7fe93dffb640 1 --2- 172.21.15.45:0/2771417983 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe92c0762d0 0x7fe92c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:04.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.368+0000 7fe93dffb640 1 -- 172.21.15.45:0/2771417983 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fe9380b8270 con 0x7fe9480b2490 2024-05-08T05:42:04.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.368+0000 7fe94d661640 1 --2- 172.21.15.45:0/2771417983 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe92c0762d0 0x7fe92c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:04.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.369+0000 7fe94d661640 1 --2- 172.21.15.45:0/2771417983 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe92c0762d0 0x7fe92c078790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe944001720 tx=0x7fe944009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:04.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.372+0000 7fe93dffb640 1 -- 172.21.15.45:0/2771417983 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe9380812d0 con 0x7fe9480b2490 2024-05-08T05:42:04.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.564+0000 7fe94f8c5640 1 -- 172.21.15.45:0/2771417983 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe94815ee60 con 0x7fe9480b2490 2024-05-08T05:42:04.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.568+0000 7fe93dffb640 1 -- 172.21.15.45:0/2771417983 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fe9380855d0 con 0x7fe9480b2490 2024-05-08T05:42:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.570+0000 7fe94f8c5640 1 -- 172.21.15.45:0/2771417983 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe92c0762d0 msgr2=0x7fe92c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.570+0000 7fe94f8c5640 1 --2- 172.21.15.45:0/2771417983 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe92c0762d0 0x7fe92c078790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe944001720 tx=0x7fe944009040 comp rx=0 tx=0).stop 2024-05-08T05:42:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.570+0000 7fe94f8c5640 1 -- 172.21.15.45:0/2771417983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9480b2490 msgr2=0x7fe9480cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:04.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.570+0000 7fe94f8c5640 1 --2- 172.21.15.45:0/2771417983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9480b2490 0x7fe9480cd730 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe938000c00 tx=0x7fe9380046c0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.570+0000 7fe94f8c5640 1 -- 172.21.15.45:0/2771417983 shutdown_connections 2024-05-08T05:42:04.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.570+0000 7fe94f8c5640 1 --2- 172.21.15.45:0/2771417983 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe92c0762d0 0x7fe92c078790 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.570+0000 7fe94f8c5640 1 --2- 172.21.15.45:0/2771417983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9481629b0 0x7fe9480cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.570+0000 7fe94f8c5640 1 --2- 172.21.15.45:0/2771417983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9480b2490 0x7fe9480cd730 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.570+0000 7fe94f8c5640 1 --2- 172.21.15.45:0/2771417983 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe948070d20 0x7fe9480b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.570+0000 7fe94f8c5640 1 -- 172.21.15.45:0/2771417983 >> 172.21.15.45:0/2771417983 conn(0x7fe9480b49d0 msgr2=0x7fe9480b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:04.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.570+0000 7fe94f8c5640 1 -- 172.21.15.45:0/2771417983 shutdown_connections 2024-05-08T05:42:04.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.570+0000 7fe94f8c5640 1 -- 172.21.15.45:0/2771417983 wait complete. 2024-05-08T05:42:04.581 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T05:42:04.581 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:42:04.623 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:42:04.666 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T05:42:04.666 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:42:04.666 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:04.683 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:42:04.683 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:04.739 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:42:04.739 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:04.796 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:42:04.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.940+0000 7ff7f0864640 1 Processor -- start 2024-05-08T05:42:04.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.940+0000 7ff7f0864640 1 -- start start 2024-05-08T05:42:04.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.941+0000 7ff7f0864640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff7ec0b2490 0x7ff7ec070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:04.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.941+0000 7ff7f0864640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff7ec070d20 0x7ff7ec0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:04.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.941+0000 7ff7f0864640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7ec06e110 con 0x7ff7ec1629b0 2024-05-08T05:42:04.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.941+0000 7ff7f0864640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff7ec06dec0 con 0x7ff7ec070d20 2024-05-08T05:42:04.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.941+0000 7ff7f0864640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff7ec06e3f0 con 0x7ff7ec0b2490 2024-05-08T05:42:04.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.941+0000 7ff7ead9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff7ec070d20 0x7ff7ec0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:04.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.941+0000 7ff7ead9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff7ec070d20 0x7ff7ec0711d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49034/0 (socket says 172.21.15.45:49034) 2024-05-08T05:42:04.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.941+0000 7ff7ead9d640 1 -- 172.21.15.45:0/1299903369 learned_addr learned my addr 172.21.15.45:0/1299903369 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:04.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.941+0000 7ff7e9d9b640 1 --2- 172.21.15.45:0/1299903369 >> v2:172.21.15.63:3300/0 conn(0x7ff7ec0b2490 0x7ff7ec070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:04.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.942+0000 7ff7ead9d640 1 -- 172.21.15.45:0/1299903369 >> v2:172.21.15.63:3300/0 conn(0x7ff7ec0b2490 msgr2=0x7ff7ec070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:04.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.942+0000 7ff7ead9d640 1 --2- 172.21.15.45:0/1299903369 >> v2:172.21.15.63:3300/0 conn(0x7ff7ec0b2490 0x7ff7ec070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.942+0000 7ff7ead9d640 1 -- 172.21.15.45:0/1299903369 >> v1:172.21.15.63:6789/0 conn(0x7ff7ec1629b0 legacy=0x7ff7ec06eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:04.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.942+0000 7ff7ead9d640 1 -- 172.21.15.45:0/1299903369 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ec06e6d0 con 0x7ff7ec070d20 2024-05-08T05:42:04.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.942+0000 7ff7ead9d640 1 --2- 172.21.15.45:0/1299903369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7ec070d20 0x7ff7ec0711d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0b8fb0 tx=0x7ff7e402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4297dabef4d60178 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:04.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.942+0000 7ff7e959a640 1 -- 172.21.15.45:0/1299903369 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4036070 con 0x7ff7ec070d20 2024-05-08T05:42:04.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.942+0000 7ff7e959a640 1 -- 172.21.15.45:0/1299903369 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e4031600 con 0x7ff7ec070d20 2024-05-08T05:42:04.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.943+0000 7ff7e959a640 1 -- 172.21.15.45:0/1299903369 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e40319d0 con 0x7ff7ec070d20 2024-05-08T05:42:04.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.943+0000 7ff7f0864640 1 -- 172.21.15.45:0/1299903369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7ec070d20 msgr2=0x7ff7ec0711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:04.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.943+0000 7ff7f0864640 1 --2- 172.21.15.45:0/1299903369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7ec070d20 0x7ff7ec0711d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0b8fb0 tx=0x7ff7e402a6a0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.943+0000 7ff7f0864640 1 -- 172.21.15.45:0/1299903369 shutdown_connections 2024-05-08T05:42:04.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.943+0000 7ff7f0864640 1 --2- 172.21.15.45:0/1299903369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7ec070d20 0x7ff7ec0711d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.943+0000 7ff7f0864640 1 --2- 172.21.15.45:0/1299903369 >> v2:172.21.15.63:3300/0 conn(0x7ff7ec0b2490 0x7ff7ec070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.943+0000 7ff7f0864640 1 -- 172.21.15.45:0/1299903369 >> 172.21.15.45:0/1299903369 conn(0x7ff7ec0b49d0 msgr2=0x7ff7ec0b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:04.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.943+0000 7ff7f0864640 1 -- 172.21.15.45:0/1299903369 shutdown_connections 2024-05-08T05:42:04.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.943+0000 7ff7f0864640 1 -- 172.21.15.45:0/1299903369 wait complete. 2024-05-08T05:42:04.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.944+0000 7ff7f0864640 1 Processor -- start 2024-05-08T05:42:04.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.945+0000 7ff7f0864640 1 -- start start 2024-05-08T05:42:04.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.945+0000 7ff7f0864640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7ec070d20 0x7ff7ec1f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:04.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.945+0000 7ff7f0864640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7ec0b2490 0x7ff7ec0c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:04.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.945+0000 7ff7f0864640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7ec1629b0 0x7ff7ec1531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:04.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.945+0000 7ff7f0864640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec0709f0 con 0x7ff7ec1629b0 2024-05-08T05:42:04.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.945+0000 7ff7f0864640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec070870 con 0x7ff7ec0b2490 2024-05-08T05:42:04.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.945+0000 7ff7f0864640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff7ec0afcb0 con 0x7ff7ec070d20 2024-05-08T05:42:04.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.945+0000 7ff7ead9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7ec1629b0 0x7ff7ec1531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:04.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.945+0000 7ff7ead9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7ec1629b0 0x7ff7ec1531e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49040/0 (socket says 172.21.15.45:49040) 2024-05-08T05:42:04.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.945+0000 7ff7ead9d640 1 -- 172.21.15.45:0/3005620145 learned_addr learned my addr 172.21.15.45:0/3005620145 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:04.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.946+0000 7ff7e9d9b640 1 --2- 172.21.15.45:0/3005620145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7ec0b2490 0x7ff7ec0c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:04.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.946+0000 7ff7ea59c640 1 --2- 172.21.15.45:0/3005620145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7ec070d20 0x7ff7ec1f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:04.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.946+0000 7ff7ead9d640 1 -- 172.21.15.45:0/3005620145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7ec070d20 msgr2=0x7ff7ec1f4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:04.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.946+0000 7ff7ead9d640 1 --2- 172.21.15.45:0/3005620145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7ec070d20 0x7ff7ec1f4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.946+0000 7ff7ead9d640 1 -- 172.21.15.45:0/3005620145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7ec0b2490 msgr2=0x7ff7ec0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:04.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.946+0000 7ff7ead9d640 1 --2- 172.21.15.45:0/3005620145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7ec0b2490 0x7ff7ec0c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:04.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.946+0000 7ff7ead9d640 1 -- 172.21.15.45:0/3005620145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ec0cf7a0 con 0x7ff7ec1629b0 2024-05-08T05:42:04.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.946+0000 7ff7ea59c640 1 --2- 172.21.15.45:0/3005620145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7ec070d20 0x7ff7ec1f4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:04.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.946+0000 7ff7e9d9b640 1 --2- 172.21.15.45:0/3005620145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7ec0b2490 0x7ff7ec0c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:04.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.946+0000 7ff7ead9d640 1 --2- 172.21.15.45:0/3005620145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7ec1629b0 0x7ff7ec1531e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009f40 tx=0x7ff7e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:04.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.946+0000 7ff7daffd640 1 -- 172.21.15.45:0/3005620145 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4041020 con 0x7ff7ec1629b0 2024-05-08T05:42:04.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.946+0000 7ff7daffd640 1 -- 172.21.15.45:0/3005620145 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e40077a0 con 0x7ff7ec1629b0 2024-05-08T05:42:04.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.946+0000 7ff7f0864640 1 -- 172.21.15.45:0/3005620145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7ec0c9640 con 0x7ff7ec1629b0 2024-05-08T05:42:04.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.946+0000 7ff7daffd640 1 -- 172.21.15.45:0/3005620145 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4036040 con 0x7ff7ec1629b0 2024-05-08T05:42:04.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.946+0000 7ff7f0864640 1 -- 172.21.15.45:0/3005620145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7ec0cec20 con 0x7ff7ec1629b0 2024-05-08T05:42:04.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.947+0000 7ff7f0864640 1 -- 172.21.15.45:0/3005620145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7ec0693f0 con 0x7ff7ec1629b0 2024-05-08T05:42:04.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.948+0000 7ff7daffd640 1 -- 172.21.15.45:0/3005620145 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff7e4044020 con 0x7ff7ec1629b0 2024-05-08T05:42:04.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.948+0000 7ff7daffd640 1 --2- 172.21.15.45:0/3005620145 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff7cc0762d0 0x7ff7cc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:04.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.948+0000 7ff7daffd640 1 -- 172.21.15.45:0/3005620145 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7ff7e40b8a20 con 0x7ff7ec1629b0 2024-05-08T05:42:04.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.949+0000 7ff7ea59c640 1 --2- 172.21.15.45:0/3005620145 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff7cc0762d0 0x7ff7cc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:04.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.949+0000 7ff7ea59c640 1 --2- 172.21.15.45:0/3005620145 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff7cc0762d0 0x7ff7cc078790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00a830 tx=0x7ff7dc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:04.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:04.953+0000 7ff7daffd640 1 -- 172.21.15.45:0/3005620145 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff7e404b020 con 0x7ff7ec1629b0 2024-05-08T05:42:05.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.156+0000 7ff7f0864640 1 -- 172.21.15.45:0/3005620145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff7ec0b5200 con 0x7ff7ec1629b0 2024-05-08T05:42:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.156+0000 7ff7daffd640 1 -- 172.21.15.45:0/3005620145 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7ff7e4081fa0 con 0x7ff7ec1629b0 2024-05-08T05:42:05.157 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-05-08T05:42:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.158+0000 7ff7f0864640 1 -- 172.21.15.45:0/3005620145 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff7cc0762d0 msgr2=0x7ff7cc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.158+0000 7ff7f0864640 1 --2- 172.21.15.45:0/3005620145 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff7cc0762d0 0x7ff7cc078790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00a830 tx=0x7ff7dc008040 comp rx=0 tx=0).stop 2024-05-08T05:42:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.158+0000 7ff7f0864640 1 -- 172.21.15.45:0/3005620145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7ec1629b0 msgr2=0x7ff7ec1531e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.158+0000 7ff7f0864640 1 --2- 172.21.15.45:0/3005620145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7ec1629b0 0x7ff7ec1531e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009f40 tx=0x7ff7e4004a40 comp rx=0 tx=0).stop 2024-05-08T05:42:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.158+0000 7ff7f0864640 1 -- 172.21.15.45:0/3005620145 shutdown_connections 2024-05-08T05:42:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.158+0000 7ff7f0864640 1 --2- 172.21.15.45:0/3005620145 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff7cc0762d0 0x7ff7cc078790 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.158+0000 7ff7f0864640 1 --2- 172.21.15.45:0/3005620145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7ec1629b0 0x7ff7ec1531e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.158+0000 7ff7f0864640 1 --2- 172.21.15.45:0/3005620145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7ec0b2490 0x7ff7ec0c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:05.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.158+0000 7ff7f0864640 1 --2- 172.21.15.45:0/3005620145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7ec070d20 0x7ff7ec1f4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.158+0000 7ff7f0864640 1 -- 172.21.15.45:0/3005620145 >> 172.21.15.45:0/3005620145 conn(0x7ff7ec0b49d0 msgr2=0x7ff7ec0d1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.158+0000 7ff7f0864640 1 -- 172.21.15.45:0/3005620145 shutdown_connections 2024-05-08T05:42:05.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.158+0000 7ff7f0864640 1 -- 172.21.15.45:0/3005620145 wait complete. 2024-05-08T05:42:05.170 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T05:42:05.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:05.170 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-05-08T05:42:05.198 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:42:05.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:05.198 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T05:42:05.255 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:42:05.255 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:05.276 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:42:05.276 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:05.335 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:42:05.336 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:05.396 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:42:05.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.501+0000 7f9582919640 1 Processor -- start 2024-05-08T05:42:05.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.501+0000 7f9582919640 1 -- start start 2024-05-08T05:42:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.502+0000 7f9582919640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f957c0d3cd0 0x7f957c0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.502+0000 7f9582919640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f957c0d3330 con 0x7f957c1629a0 2024-05-08T05:42:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.502+0000 7f9582919640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f957c0d2170 con 0x7f957c0c9ee0 2024-05-08T05:42:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.502+0000 7f9582919640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f957c0d2450 con 0x7f957c0d3cd0 2024-05-08T05:42:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.502+0000 7f957b7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f957c0d3cd0 0x7f957c0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.502+0000 7f957bfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f957c1629a0 0x7f957c0d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53194/0 (socket says 172.21.15.45:53194) 2024-05-08T05:42:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.502+0000 7f957bfff640 1 -- 172.21.15.45:0/1943228589 learned_addr learned my addr 172.21.15.45:0/1943228589 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.502+0000 7f957b7fe640 1 -- 172.21.15.45:0/1943228589 >> v1:172.21.15.63:6789/0 conn(0x7f957c0c9ee0 legacy=0x7f957c0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.502+0000 7f957b7fe640 1 -- 172.21.15.45:0/1943228589 >> v1:172.21.15.45:6789/0 conn(0x7f957c1629a0 legacy=0x7f957c0d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.502+0000 7f957b7fe640 1 -- 172.21.15.45:0/1943228589 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957c0d2730 con 0x7f957c0d3cd0 2024-05-08T05:42:05.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.502+0000 7f957affd640 1 -- 172.21.15.45:0/1943228589 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 827251673 0 0) 0x7f957c0d3330 con 0x7f957c1629a0 2024-05-08T05:42:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.503+0000 7f957b7fe640 1 --2- 172.21.15.45:0/1943228589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957c0d3cd0 0x7f957c0cfca0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f95700097a0 tx=0x7f957002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baeceb5cc4ef6cd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.503+0000 7f957affd640 1 -- 172.21.15.45:0/1943228589 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570037070 con 0x7f957c0d3cd0 2024-05-08T05:42:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.503+0000 7f957affd640 1 -- 172.21.15.45:0/1943228589 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f957002adb0 con 0x7f957c0d3cd0 2024-05-08T05:42:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.503+0000 7f957affd640 1 -- 172.21.15.45:0/1943228589 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570032660 con 0x7f957c0d3cd0 2024-05-08T05:42:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.504+0000 7f9582919640 1 -- 172.21.15.45:0/1943228589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957c0d3cd0 msgr2=0x7f957c0cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.504+0000 7f9582919640 1 --2- 172.21.15.45:0/1943228589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957c0d3cd0 0x7f957c0cfca0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f95700097a0 tx=0x7f957002a060 comp rx=0 tx=0).stop 2024-05-08T05:42:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.504+0000 7f9582919640 1 -- 172.21.15.45:0/1943228589 shutdown_connections 2024-05-08T05:42:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.504+0000 7f9582919640 1 --2- 172.21.15.45:0/1943228589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957c0d3cd0 0x7f957c0cfca0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.504+0000 7f9582919640 1 -- 172.21.15.45:0/1943228589 >> 172.21.15.45:0/1943228589 conn(0x7f957c0d4fe0 msgr2=0x7f957c0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.504+0000 7f9582919640 1 -- 172.21.15.45:0/1943228589 shutdown_connections 2024-05-08T05:42:05.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.504+0000 7f9582919640 1 -- 172.21.15.45:0/1943228589 wait complete. 2024-05-08T05:42:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.505+0000 7f9582919640 1 Processor -- start 2024-05-08T05:42:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.505+0000 7f9582919640 1 -- start start 2024-05-08T05:42:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.506+0000 7f9582919640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f957c0c9ee0 0x7f957c15d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.506+0000 7f9582919640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f957c0d3cd0 0x7f957c0b8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.506+0000 7f9582919640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957c1629a0 0x7f957c09fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.506+0000 7f9582919640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f957c0cf440 con 0x7f957c1629a0 2024-05-08T05:42:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.506+0000 7f9582919640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f957c0cf2c0 con 0x7f957c0c9ee0 2024-05-08T05:42:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.506+0000 7f9582919640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f957c0d2dc0 con 0x7f957c0d3cd0 2024-05-08T05:42:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.506+0000 7f9580eb6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957c1629a0 0x7f957c09fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.506+0000 7f9580eb6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957c1629a0 0x7f957c09fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49056/0 (socket says 172.21.15.45:49056) 2024-05-08T05:42:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.506+0000 7f9580eb6640 1 -- 172.21.15.45:0/3294007090 learned_addr learned my addr 172.21.15.45:0/3294007090 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:05.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.506+0000 7f957bfff640 1 --2- 172.21.15.45:0/3294007090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f957c0c9ee0 0x7f957c15d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.506+0000 7f957b7fe640 1 --2- 172.21.15.45:0/3294007090 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f957c0d3cd0 0x7f957c0b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.506+0000 7f9580eb6640 1 -- 172.21.15.45:0/3294007090 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f957c0d3cd0 msgr2=0x7f957c0b8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.506+0000 7f9580eb6640 1 --2- 172.21.15.45:0/3294007090 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f957c0d3cd0 0x7f957c0b8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.506+0000 7f9580eb6640 1 -- 172.21.15.45:0/3294007090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f957c0c9ee0 msgr2=0x7f957c15d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.506+0000 7f9580eb6640 1 --2- 172.21.15.45:0/3294007090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f957c0c9ee0 0x7f957c15d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.506+0000 7f9580eb6640 1 -- 172.21.15.45:0/3294007090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957c0cc300 con 0x7f957c1629a0 2024-05-08T05:42:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.506+0000 7f957b7fe640 1 --2- 172.21.15.45:0/3294007090 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f957c0d3cd0 0x7f957c0b8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.507+0000 7f9580eb6640 1 --2- 172.21.15.45:0/3294007090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957c1629a0 0x7f957c09fe80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f957400c780 tx=0x7f957400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.507+0000 7f9578ff9640 1 -- 172.21.15.45:0/3294007090 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9574017070 con 0x7f957c1629a0 2024-05-08T05:42:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.507+0000 7f9578ff9640 1 -- 172.21.15.45:0/3294007090 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9574003da0 con 0x7f957c1629a0 2024-05-08T05:42:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.507+0000 7f9578ff9640 1 -- 172.21.15.45:0/3294007090 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95740052b0 con 0x7f957c1629a0 2024-05-08T05:42:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.507+0000 7f9582919640 1 -- 172.21.15.45:0/3294007090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957c0cc5e0 con 0x7f957c1629a0 2024-05-08T05:42:05.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.507+0000 7f9582919640 1 -- 172.21.15.45:0/3294007090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f957c0ccba0 con 0x7f957c1629a0 2024-05-08T05:42:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.508+0000 7f9582919640 1 -- 172.21.15.45:0/3294007090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f957c0693f0 con 0x7f957c1629a0 2024-05-08T05:42:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.508+0000 7f9578ff9640 1 -- 172.21.15.45:0/3294007090 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9574005450 con 0x7f957c1629a0 2024-05-08T05:42:05.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.509+0000 7f9578ff9640 1 --2- 172.21.15.45:0/3294007090 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f95680762d0 0x7f9568078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.509+0000 7f9578ff9640 1 -- 172.21.15.45:0/3294007090 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f9574096e60 con 0x7f957c1629a0 2024-05-08T05:42:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.509+0000 7f957bfff640 1 --2- 172.21.15.45:0/3294007090 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f95680762d0 0x7f9568078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:05.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.510+0000 7f957bfff640 1 --2- 172.21.15.45:0/3294007090 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f95680762d0 0x7f9568078790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f956c007fd0 tx=0x7f956c007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:05.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.513+0000 7f9578ff9640 1 -- 172.21.15.45:0/3294007090 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f957405fe30 con 0x7f957c1629a0 2024-05-08T05:42:05.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.718+0000 7f9582919640 1 -- 172.21.15.45:0/3294007090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f957c0c8670 con 0x7f957c1629a0 2024-05-08T05:42:05.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.719+0000 7f9578ff9640 1 -- 172.21.15.45:0/3294007090 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f9574064130 con 0x7f957c1629a0 2024-05-08T05:42:05.720 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-05-08T05:42:05.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.721+0000 7f9582919640 1 -- 172.21.15.45:0/3294007090 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f95680762d0 msgr2=0x7f9568078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:05.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.721+0000 7f9582919640 1 --2- 172.21.15.45:0/3294007090 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f95680762d0 0x7f9568078790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f956c007fd0 tx=0x7f956c007de0 comp rx=0 tx=0).stop 2024-05-08T05:42:05.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.721+0000 7f9582919640 1 -- 172.21.15.45:0/3294007090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957c1629a0 msgr2=0x7f957c09fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:05.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.721+0000 7f9582919640 1 --2- 172.21.15.45:0/3294007090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957c1629a0 0x7f957c09fe80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f957400c780 tx=0x7f957400cc40 comp rx=0 tx=0).stop 2024-05-08T05:42:05.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.721+0000 7f9582919640 1 -- 172.21.15.45:0/3294007090 shutdown_connections 2024-05-08T05:42:05.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.721+0000 7f9582919640 1 --2- 172.21.15.45:0/3294007090 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f95680762d0 0x7f9568078790 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:05.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.721+0000 7f9582919640 1 --2- 172.21.15.45:0/3294007090 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f957c0d3cd0 0x7f957c0b8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:05.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.721+0000 7f9582919640 1 --2- 172.21.15.45:0/3294007090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f957c1629a0 0x7f957c09fe80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:05.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.721+0000 7f9582919640 1 --2- 172.21.15.45:0/3294007090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f957c0c9ee0 0x7f957c15d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:05.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.721+0000 7f9582919640 1 -- 172.21.15.45:0/3294007090 >> 172.21.15.45:0/3294007090 conn(0x7f957c0d4fe0 msgr2=0x7f957c0d04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:05.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.721+0000 7f9582919640 1 -- 172.21.15.45:0/3294007090 shutdown_connections 2024-05-08T05:42:05.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:05.721+0000 7f9582919640 1 -- 172.21.15.45:0/3294007090 wait complete. 2024-05-08T05:42:05.732 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-08T05:42:05.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:05.733 DEBUG:teuthology.orchestra.run.smithi063:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-05-08T05:42:05.766 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-08T05:42:05.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:05.766 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-08T05:42:05.836 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:42:05.836 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:05.852 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:42:05.852 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:05.908 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:42:05.909 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:05.965 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:42:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.109+0000 7fb2cfc54640 1 Processor -- start 2024-05-08T05:42:06.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.110+0000 7fb2cfc54640 1 -- start start 2024-05-08T05:42:06.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.110+0000 7fb2cfc54640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb2c815f420 0x7fb2c80d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:06.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.110+0000 7fb2cfc54640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2c81630b0 con 0x7fb2c80d3ca0 2024-05-08T05:42:06.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.110+0000 7fb2cfc54640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2c80d2550 con 0x7fb2c80d2080 2024-05-08T05:42:06.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.110+0000 7fb2cfc54640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb2c80d2c80 con 0x7fb2c815f420 2024-05-08T05:42:06.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.111+0000 7fb2cd9f0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb2c815f420 0x7fb2c80d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:06.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.111+0000 7fb2cd1ef640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb2c80d3ca0 0x7fb2c80d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53206/0 (socket says 172.21.15.45:53206) 2024-05-08T05:42:06.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.111+0000 7fb2cd9f0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb2c815f420 0x7fb2c80d3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49064/0 (socket says 172.21.15.45:49064) 2024-05-08T05:42:06.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.111+0000 7fb2cd1ef640 1 -- 172.21.15.45:0/2523480435 learned_addr learned my addr 172.21.15.45:0/2523480435 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:06.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.111+0000 7fb2cd9f0640 1 -- 172.21.15.45:0/2523480435 >> v1:172.21.15.63:6789/0 conn(0x7fb2c80d2080 legacy=0x7fb2c80d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:06.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.111+0000 7fb2cd9f0640 1 -- 172.21.15.45:0/2523480435 >> v1:172.21.15.45:6789/0 conn(0x7fb2c80d3ca0 legacy=0x7fb2c80d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:06.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.111+0000 7fb2cc9ee640 1 -- 172.21.15.45:0/2523480435 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4132338531 0 0) 0x7fb2c4001110 con 0x7fb2c80d3ca0 2024-05-08T05:42:06.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.111+0000 7fb2cd9f0640 1 -- 172.21.15.45:0/2523480435 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c80d2f60 con 0x7fb2c815f420 2024-05-08T05:42:06.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.112+0000 7fb2cd9f0640 1 --2- 172.21.15.45:0/2523480435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c815f420 0x7fb2c80d3760 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009d90 tx=0x7fb2bc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8bd9d356e449ca7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:06.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.112+0000 7fb2cc9ee640 1 -- 172.21.15.45:0/2523480435 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc007740 con 0x7fb2c815f420 2024-05-08T05:42:06.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.112+0000 7fb2cc9ee640 1 -- 172.21.15.45:0/2523480435 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2bc0078e0 con 0x7fb2c815f420 2024-05-08T05:42:06.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.112+0000 7fb2cc9ee640 1 -- 172.21.15.45:0/2523480435 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc007cb0 con 0x7fb2c815f420 2024-05-08T05:42:06.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.112+0000 7fb2cfc54640 1 -- 172.21.15.45:0/2523480435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c815f420 msgr2=0x7fb2c80d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:06.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.112+0000 7fb2cfc54640 1 --2- 172.21.15.45:0/2523480435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c815f420 0x7fb2c80d3760 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009d90 tx=0x7fb2bc02a660 comp rx=0 tx=0).stop 2024-05-08T05:42:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.112+0000 7fb2cfc54640 1 -- 172.21.15.45:0/2523480435 shutdown_connections 2024-05-08T05:42:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.112+0000 7fb2cfc54640 1 --2- 172.21.15.45:0/2523480435 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c815f420 0x7fb2c80d3760 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.112+0000 7fb2cfc54640 1 -- 172.21.15.45:0/2523480435 >> 172.21.15.45:0/2523480435 conn(0x7fb2c808eac0 msgr2=0x7fb2c809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.112+0000 7fb2cfc54640 1 -- 172.21.15.45:0/2523480435 shutdown_connections 2024-05-08T05:42:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.113+0000 7fb2cfc54640 1 -- 172.21.15.45:0/2523480435 wait complete. 2024-05-08T05:42:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.114+0000 7fb2cfc54640 1 Processor -- start 2024-05-08T05:42:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.114+0000 7fb2cfc54640 1 -- start start 2024-05-08T05:42:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.114+0000 7fb2cfc54640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2c80d2080 0x7fb2c80ba4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.114+0000 7fb2cfc54640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c8154bc0 0x7fb2c80b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.114+0000 7fb2cfc54640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2c80b8100 0x7fb2c80b8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:06.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.114+0000 7fb2cfc54640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c80d53d0 con 0x7fb2c8154bc0 2024-05-08T05:42:06.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.114+0000 7fb2cfc54640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c80d2830 con 0x7fb2c80d2080 2024-05-08T05:42:06.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.114+0000 7fb2cfc54640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c80d5550 con 0x7fb2c80b8100 2024-05-08T05:42:06.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.115+0000 7fb2cd1ef640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c8154bc0 0x7fb2c80b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:06.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.115+0000 7fb2cd1ef640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c8154bc0 0x7fb2c80b7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49076/0 (socket says 172.21.15.45:49076) 2024-05-08T05:42:06.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.115+0000 7fb2cd1ef640 1 -- 172.21.15.45:0/1781077973 learned_addr learned my addr 172.21.15.45:0/1781077973 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:06.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.115+0000 7fb2cd9f0640 1 --2- 172.21.15.45:0/1781077973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2c80d2080 0x7fb2c80ba4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:06.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.115+0000 7fb2cd1ef640 1 -- 172.21.15.45:0/1781077973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2c80b8100 msgr2=0x7fb2c80b8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:06.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.115+0000 7fb2cd1ef640 1 --2- 172.21.15.45:0/1781077973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2c80b8100 0x7fb2c80b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:06.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.115+0000 7fb2cd1ef640 1 -- 172.21.15.45:0/1781077973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2c80d2080 msgr2=0x7fb2c80ba4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:06.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.115+0000 7fb2cd1ef640 1 --2- 172.21.15.45:0/1781077973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2c80d2080 0x7fb2c80ba4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.115+0000 7fb2cd1ef640 1 -- 172.21.15.45:0/1781077973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c80baa20 con 0x7fb2c8154bc0 2024-05-08T05:42:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.115+0000 7fb2cd1ef640 1 --2- 172.21.15.45:0/1781077973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c8154bc0 0x7fb2c80b7bc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400c730 tx=0x7fb2c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.115+0000 7fb2ba7fc640 1 -- 172.21.15.45:0/1781077973 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4018070 con 0x7fb2c8154bc0 2024-05-08T05:42:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.115+0000 7fb2ba7fc640 1 -- 172.21.15.45:0/1781077973 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c40072d0 con 0x7fb2c8154bc0 2024-05-08T05:42:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.115+0000 7fb2ba7fc640 1 -- 172.21.15.45:0/1781077973 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4007e20 con 0x7fb2c8154bc0 2024-05-08T05:42:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.115+0000 7fb2cfc54640 1 -- 172.21.15.45:0/1781077973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c8161c60 con 0x7fb2c8154bc0 2024-05-08T05:42:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.115+0000 7fb2cfc54640 1 -- 172.21.15.45:0/1781077973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c80c8f40 con 0x7fb2c8154bc0 2024-05-08T05:42:06.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.116+0000 7fb2cfc54640 1 -- 172.21.15.45:0/1781077973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c80c8490 con 0x7fb2c8154bc0 2024-05-08T05:42:06.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.117+0000 7fb2ba7fc640 1 -- 172.21.15.45:0/1781077973 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb2c401b070 con 0x7fb2c8154bc0 2024-05-08T05:42:06.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.117+0000 7fb2ba7fc640 1 --2- 172.21.15.45:0/1781077973 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb2ac076300 0x7fb2ac0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:06.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.117+0000 7fb2ba7fc640 1 -- 172.21.15.45:0/1781077973 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fb2c4096c70 con 0x7fb2c8154bc0 2024-05-08T05:42:06.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.118+0000 7fb2cd9f0640 1 --2- 172.21.15.45:0/1781077973 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb2ac076300 0x7fb2ac0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:06.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.118+0000 7fb2cd9f0640 1 --2- 172.21.15.45:0/1781077973 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb2ac076300 0x7fb2ac0787c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc034000 tx=0x7fb2bc02f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:06.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.122+0000 7fb2ba7fc640 1 -- 172.21.15.45:0/1781077973 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb2c80c8490 con 0x7fb2c8154bc0 2024-05-08T05:42:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.325+0000 7fb2cfc54640 1 -- 172.21.15.45:0/1781077973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb2c80693f0 con 0x7fb2c8154bc0 2024-05-08T05:42:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.325+0000 7fb2ba7fc640 1 -- 172.21.15.45:0/1781077973 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7fb2c81630b0 con 0x7fb2c8154bc0 2024-05-08T05:42:06.325 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-05-08T05:42:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.327+0000 7fb2cfc54640 1 -- 172.21.15.45:0/1781077973 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb2ac076300 msgr2=0x7fb2ac0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.327+0000 7fb2cfc54640 1 --2- 172.21.15.45:0/1781077973 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb2ac076300 0x7fb2ac0787c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc034000 tx=0x7fb2bc02f000 comp rx=0 tx=0).stop 2024-05-08T05:42:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.327+0000 7fb2cfc54640 1 -- 172.21.15.45:0/1781077973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c8154bc0 msgr2=0x7fb2c80b7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.327+0000 7fb2cfc54640 1 --2- 172.21.15.45:0/1781077973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c8154bc0 0x7fb2c80b7bc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400c730 tx=0x7fb2c400cbf0 comp rx=0 tx=0).stop 2024-05-08T05:42:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.327+0000 7fb2cfc54640 1 -- 172.21.15.45:0/1781077973 shutdown_connections 2024-05-08T05:42:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.327+0000 7fb2cfc54640 1 --2- 172.21.15.45:0/1781077973 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb2ac076300 0x7fb2ac0787c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.327+0000 7fb2cfc54640 1 --2- 172.21.15.45:0/1781077973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb2c80b8100 0x7fb2c80b8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.327+0000 7fb2cfc54640 1 --2- 172.21.15.45:0/1781077973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb2c8154bc0 0x7fb2c80b7bc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.327+0000 7fb2cfc54640 1 --2- 172.21.15.45:0/1781077973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2c80d2080 0x7fb2c80ba4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:06.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.327+0000 7fb2cfc54640 1 -- 172.21.15.45:0/1781077973 >> 172.21.15.45:0/1781077973 conn(0x7fb2c808eac0 msgr2=0x7fb2c806d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.327+0000 7fb2cfc54640 1 -- 172.21.15.45:0/1781077973 shutdown_connections 2024-05-08T05:42:06.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.327+0000 7fb2cfc54640 1 -- 172.21.15.45:0/1781077973 wait complete. 2024-05-08T05:42:06.339 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-05-08T05:42:06.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:06.339 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-05-08T05:42:06.365 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:42:06.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:06.366 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-08T05:42:06.423 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:42:06.423 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:06.444 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:42:06.444 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:06.503 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:42:06.503 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:06.562 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:42:06.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.666+0000 7f88f8474640 1 Processor -- start 2024-05-08T05:42:06.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.666+0000 7f88f8474640 1 -- start start 2024-05-08T05:42:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.667+0000 7f88f8474640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f88f00d3cd0 0x7f88f00cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.667+0000 7f88f8474640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88f00d3330 con 0x7f88f01629a0 2024-05-08T05:42:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.667+0000 7f88f8474640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88f00d2170 con 0x7f88f00c9ee0 2024-05-08T05:42:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.667+0000 7f88f8474640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f88f00d2450 con 0x7f88f00d3cd0 2024-05-08T05:42:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.667+0000 7f88f5a0f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f88f00d3cd0 0x7f88f00cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.667+0000 7f88f6210640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f88f01629a0 0x7f88f00d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53220/0 (socket says 172.21.15.45:53220) 2024-05-08T05:42:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.667+0000 7f88f5a0f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f88f00d3cd0 0x7f88f00cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49080/0 (socket says 172.21.15.45:49080) 2024-05-08T05:42:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.667+0000 7f88f6210640 1 -- 172.21.15.45:0/2610865391 learned_addr learned my addr 172.21.15.45:0/2610865391 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.668+0000 7f88f5a0f640 1 -- 172.21.15.45:0/2610865391 >> v1:172.21.15.63:6789/0 conn(0x7f88f00c9ee0 legacy=0x7f88f00d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.668+0000 7f88f5a0f640 1 -- 172.21.15.45:0/2610865391 >> v1:172.21.15.45:6789/0 conn(0x7f88f01629a0 legacy=0x7f88f00d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.668+0000 7f88f5a0f640 1 -- 172.21.15.45:0/2610865391 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f00d2730 con 0x7f88f00d3cd0 2024-05-08T05:42:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.668+0000 7f88f5a0f640 1 --2- 172.21.15.45:0/2610865391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88f00d3cd0 0x7f88f00cfca0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f88e40097a0 tx=0x7f88e402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b438ed213b16f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.668+0000 7f88f520e640 1 -- 172.21.15.45:0/2610865391 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4037070 con 0x7f88f00d3cd0 2024-05-08T05:42:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.668+0000 7f88f520e640 1 -- 172.21.15.45:0/2610865391 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88e402adb0 con 0x7f88f00d3cd0 2024-05-08T05:42:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.668+0000 7f88f520e640 1 -- 172.21.15.45:0/2610865391 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4032660 con 0x7f88f00d3cd0 2024-05-08T05:42:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.669+0000 7f88f8474640 1 -- 172.21.15.45:0/2610865391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88f00d3cd0 msgr2=0x7f88f00cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.669+0000 7f88f8474640 1 --2- 172.21.15.45:0/2610865391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88f00d3cd0 0x7f88f00cfca0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f88e40097a0 tx=0x7f88e402a060 comp rx=0 tx=0).stop 2024-05-08T05:42:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.669+0000 7f88f8474640 1 -- 172.21.15.45:0/2610865391 shutdown_connections 2024-05-08T05:42:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.669+0000 7f88f8474640 1 --2- 172.21.15.45:0/2610865391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88f00d3cd0 0x7f88f00cfca0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.669+0000 7f88f8474640 1 -- 172.21.15.45:0/2610865391 >> 172.21.15.45:0/2610865391 conn(0x7f88f00d4fe0 msgr2=0x7f88f00d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.669+0000 7f88f8474640 1 -- 172.21.15.45:0/2610865391 shutdown_connections 2024-05-08T05:42:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.669+0000 7f88f8474640 1 -- 172.21.15.45:0/2610865391 wait complete. 2024-05-08T05:42:06.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.670+0000 7f88f8474640 1 Processor -- start 2024-05-08T05:42:06.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.670+0000 7f88f8474640 1 -- start start 2024-05-08T05:42:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.671+0000 7f88f8474640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88f00c9ee0 0x7f88f00cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.671+0000 7f88f8474640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88f00d3cd0 0x7f88f015d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.671+0000 7f88f8474640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88f01629a0 0x7f88f01e7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.671+0000 7f88f8474640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f88f00cf440 con 0x7f88f00d3cd0 2024-05-08T05:42:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.671+0000 7f88f8474640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f88f00cf2c0 con 0x7f88f01629a0 2024-05-08T05:42:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.671+0000 7f88f8474640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f88f00d2dc0 con 0x7f88f00c9ee0 2024-05-08T05:42:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.671+0000 7f88f5a0f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88f00d3cd0 0x7f88f015d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.671+0000 7f88f5a0f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88f00d3cd0 0x7f88f015d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49092/0 (socket says 172.21.15.45:49092) 2024-05-08T05:42:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.671+0000 7f88f5a0f640 1 -- 172.21.15.45:0/3522595978 learned_addr learned my addr 172.21.15.45:0/3522595978 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.671+0000 7f88f5a0f640 1 -- 172.21.15.45:0/3522595978 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88f00c9ee0 msgr2=0x7f88f00cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.671+0000 7f88f5a0f640 1 --2- 172.21.15.45:0/3522595978 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88f00c9ee0 0x7f88f00cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.671+0000 7f88f5a0f640 1 -- 172.21.15.45:0/3522595978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88f01629a0 msgr2=0x7f88f01e7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.671+0000 7f88f5a0f640 1 --2- 172.21.15.45:0/3522595978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88f01629a0 0x7f88f01e7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:06.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.671+0000 7f88f5a0f640 1 -- 172.21.15.45:0/3522595978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f00ce0d0 con 0x7f88f00d3cd0 2024-05-08T05:42:06.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.672+0000 7f88f5a0f640 1 --2- 172.21.15.45:0/3522595978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88f00d3cd0 0x7f88f015d570 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f88e40098c0 tx=0x7f88e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:06.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.672+0000 7f88eeffd640 1 -- 172.21.15.45:0/3522595978 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4041070 con 0x7f88f00d3cd0 2024-05-08T05:42:06.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.672+0000 7f88eeffd640 1 -- 172.21.15.45:0/3522595978 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88e403c670 con 0x7f88f00d3cd0 2024-05-08T05:42:06.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.672+0000 7f88eeffd640 1 -- 172.21.15.45:0/3522595978 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e4037040 con 0x7f88f00d3cd0 2024-05-08T05:42:06.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.672+0000 7f88f8474640 1 -- 172.21.15.45:0/3522595978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f00ce3b0 con 0x7f88f00d3cd0 2024-05-08T05:42:06.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.672+0000 7f88f8474640 1 -- 172.21.15.45:0/3522595978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f00ce970 con 0x7f88f00d3cd0 2024-05-08T05:42:06.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.673+0000 7f88f8474640 1 -- 172.21.15.45:0/3522595978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88f015db10 con 0x7f88f00d3cd0 2024-05-08T05:42:06.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.674+0000 7f88eeffd640 1 -- 172.21.15.45:0/3522595978 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f88e4044070 con 0x7f88f00d3cd0 2024-05-08T05:42:06.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.674+0000 7f88eeffd640 1 --2- 172.21.15.45:0/3522595978 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f88dc0762d0 0x7f88dc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:06.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.674+0000 7f88eeffd640 1 -- 172.21.15.45:0/3522595978 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f88e40b8260 con 0x7f88f00d3cd0 2024-05-08T05:42:06.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.674+0000 7f88f6210640 1 --2- 172.21.15.45:0/3522595978 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f88dc0762d0 0x7f88dc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.675+0000 7f88f6210640 1 --2- 172.21.15.45:0/3522595978 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f88dc0762d0 0x7f88dc078790 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f88e0001fc0 tx=0x7f88e00015b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:06.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.678+0000 7f88eeffd640 1 -- 172.21.15.45:0/3522595978 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f88e4081880 con 0x7f88f00d3cd0 2024-05-08T05:42:06.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.882+0000 7f88f8474640 1 -- 172.21.15.45:0/3522595978 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f88f00d54c0 con 0x7f88f00d3cd0 2024-05-08T05:42:06.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.882+0000 7f88eeffd640 1 -- 172.21.15.45:0/3522595978 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) ==== 80+0+0 (secure 0 0 0) 0x7f88e4081180 con 0x7f88f00d3cd0 2024-05-08T05:42:06.882 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-05-08T05:42:06.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.884+0000 7f88f8474640 1 -- 172.21.15.45:0/3522595978 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f88dc0762d0 msgr2=0x7f88dc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.884+0000 7f88f8474640 1 --2- 172.21.15.45:0/3522595978 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f88dc0762d0 0x7f88dc078790 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f88e0001fc0 tx=0x7f88e00015b0 comp rx=0 tx=0).stop 2024-05-08T05:42:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.884+0000 7f88f8474640 1 -- 172.21.15.45:0/3522595978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88f00d3cd0 msgr2=0x7f88f015d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.884+0000 7f88f8474640 1 --2- 172.21.15.45:0/3522595978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88f00d3cd0 0x7f88f015d570 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f88e40098c0 tx=0x7f88e40046c0 comp rx=0 tx=0).stop 2024-05-08T05:42:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.884+0000 7f88f8474640 1 -- 172.21.15.45:0/3522595978 shutdown_connections 2024-05-08T05:42:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.884+0000 7f88f8474640 1 --2- 172.21.15.45:0/3522595978 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f88dc0762d0 0x7f88dc078790 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.884+0000 7f88f8474640 1 --2- 172.21.15.45:0/3522595978 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88f01629a0 0x7f88f01e7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.884+0000 7f88f8474640 1 --2- 172.21.15.45:0/3522595978 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88f00d3cd0 0x7f88f015d570 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.884+0000 7f88f8474640 1 --2- 172.21.15.45:0/3522595978 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88f00c9ee0 0x7f88f00cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.884+0000 7f88f8474640 1 -- 172.21.15.45:0/3522595978 >> 172.21.15.45:0/3522595978 conn(0x7f88f00d4fe0 msgr2=0x7f88f01612d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.884+0000 7f88f8474640 1 -- 172.21.15.45:0/3522595978 shutdown_connections 2024-05-08T05:42:06.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:06.884+0000 7f88f8474640 1 -- 172.21.15.45:0/3522595978 wait complete. 2024-05-08T05:42:06.895 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-05-08T05:42:06.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:06.896 DEBUG:teuthology.orchestra.run.smithi063:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-05-08T05:42:06.929 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-05-08T05:42:06.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:06.929 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-08T05:42:06.989 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T05:42:06.989 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:42:07.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.092+0000 7f97381af640 1 Processor -- start 2024-05-08T05:42:07.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.092+0000 7f97381af640 1 -- start start 2024-05-08T05:42:07.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.093+0000 7f97381af640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97300d20c0 0x7f9730163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.093+0000 7f97381af640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9730071110 0x7f973006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.093+0000 7f97381af640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97300d1300 con 0x7f9730070d30 2024-05-08T05:42:07.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.093+0000 7f97381af640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f97300d10d0 con 0x7f9730071110 2024-05-08T05:42:07.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.093+0000 7f97381af640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9730070b80 con 0x7f97300d20c0 2024-05-08T05:42:07.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.093+0000 7f9735f4b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9730070d30 0x7f97300703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53234/0 (socket says 172.21.15.45:53234) 2024-05-08T05:42:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.093+0000 7f9735f4b640 1 -- 172.21.15.45:0/2058113940 learned_addr learned my addr 172.21.15.45:0/2058113940 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.093+0000 7f973674c640 1 --2- 172.21.15.45:0/2058113940 >> v2:172.21.15.63:3300/0 conn(0x7f9730071110 0x7f973006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.093+0000 7f973574a640 1 --2- 172.21.15.45:0/2058113940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97300d20c0 0x7f9730163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.094+0000 7f9734f49640 1 -- 172.21.15.45:0/2058113940 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1400731513 0 0) 0x7f97300d1300 con 0x7f9730070d30 2024-05-08T05:42:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.094+0000 7f9734f49640 1 -- 172.21.15.45:0/2058113940 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9718003520 con 0x7f9730070d30 2024-05-08T05:42:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.094+0000 7f973674c640 1 -- 172.21.15.45:0/2058113940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97300d20c0 msgr2=0x7f9730163920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.094+0000 7f973674c640 1 --2- 172.21.15.45:0/2058113940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97300d20c0 0x7f9730163920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.094+0000 7f973674c640 1 -- 172.21.15.45:0/2058113940 >> v1:172.21.15.45:6789/0 conn(0x7f9730070d30 legacy=0x7f97300703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.094+0000 7f973674c640 1 -- 172.21.15.45:0/2058113940 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97300d15e0 con 0x7f9730071110 2024-05-08T05:42:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.094+0000 7f973574a640 1 --2- 172.21.15.45:0/2058113940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97300d20c0 0x7f9730163920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:42:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.095+0000 7f973674c640 1 --2- 172.21.15.45:0/2058113940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9730071110 0x7f973006d4b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9720009d90 tx=0x7f972002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bec5c8035a886c63 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.095+0000 7f9734f49640 1 -- 172.21.15.45:0/2058113940 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720007740 con 0x7f9730071110 2024-05-08T05:42:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.095+0000 7f9734f49640 1 -- 172.21.15.45:0/2058113940 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97200078e0 con 0x7f9730071110 2024-05-08T05:42:07.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.095+0000 7f9734f49640 1 -- 172.21.15.45:0/2058113940 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720007cb0 con 0x7f9730071110 2024-05-08T05:42:07.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.095+0000 7f97381af640 1 -- 172.21.15.45:0/2058113940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9730071110 msgr2=0x7f973006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.095+0000 7f97381af640 1 --2- 172.21.15.45:0/2058113940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9730071110 0x7f973006d4b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9720009d90 tx=0x7f972002a660 comp rx=0 tx=0).stop 2024-05-08T05:42:07.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.095+0000 7f97381af640 1 -- 172.21.15.45:0/2058113940 shutdown_connections 2024-05-08T05:42:07.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.095+0000 7f97381af640 1 --2- 172.21.15.45:0/2058113940 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9730071110 0x7f973006d4b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.095+0000 7f97381af640 1 --2- 172.21.15.45:0/2058113940 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f97300d20c0 0x7f9730163920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.095+0000 7f97381af640 1 -- 172.21.15.45:0/2058113940 >> 172.21.15.45:0/2058113940 conn(0x7f97300d2e90 msgr2=0x7f97300d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:07.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.096+0000 7f97381af640 1 -- 172.21.15.45:0/2058113940 shutdown_connections 2024-05-08T05:42:07.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.096+0000 7f97381af640 1 -- 172.21.15.45:0/2058113940 wait complete. 2024-05-08T05:42:07.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.097+0000 7f97381af640 1 Processor -- start 2024-05-08T05:42:07.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.097+0000 7f97381af640 1 -- start start 2024-05-08T05:42:07.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.097+0000 7f97381af640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9730070d30 0x7f973015c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.097+0000 7f97381af640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9730071110 0x7f97300c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.097+0000 7f97381af640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97300d20c0 0x7f97301571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.097+0000 7f97381af640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9730071580 con 0x7f97300d20c0 2024-05-08T05:42:07.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.097+0000 7f97381af640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f97300d1c30 con 0x7f9730070d30 2024-05-08T05:42:07.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.097+0000 7f97381af640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9730071700 con 0x7f9730071110 2024-05-08T05:42:07.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.098+0000 7f973674c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97300d20c0 0x7f97301571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.098+0000 7f973674c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97300d20c0 0x7f97301571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49102/0 (socket says 172.21.15.45:49102) 2024-05-08T05:42:07.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.098+0000 7f973674c640 1 -- 172.21.15.45:0/781478613 learned_addr learned my addr 172.21.15.45:0/781478613 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:07.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.098+0000 7f9735f4b640 1 --2- 172.21.15.45:0/781478613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9730070d30 0x7f973015c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.098+0000 7f973574a640 1 --2- 172.21.15.45:0/781478613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9730071110 0x7f97300c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.098+0000 7f973674c640 1 -- 172.21.15.45:0/781478613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9730071110 msgr2=0x7f97300c69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.098+0000 7f973674c640 1 --2- 172.21.15.45:0/781478613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9730071110 0x7f97300c69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.098+0000 7f973674c640 1 -- 172.21.15.45:0/781478613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9730070d30 msgr2=0x7f973015c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.098+0000 7f973674c640 1 --2- 172.21.15.45:0/781478613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9730070d30 0x7f973015c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.098+0000 7f973674c640 1 -- 172.21.15.45:0/781478613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9730157c10 con 0x7f97300d20c0 2024-05-08T05:42:07.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.098+0000 7f973674c640 1 --2- 172.21.15.45:0/781478613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97300d20c0 0x7f97301571b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9720009eb0 tx=0x7f9720004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:07.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.098+0000 7f972e7fc640 1 -- 172.21.15.45:0/781478613 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720040070 con 0x7f97300d20c0 2024-05-08T05:42:07.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.098+0000 7f972e7fc640 1 -- 172.21.15.45:0/781478613 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9720031bc0 con 0x7f97300d20c0 2024-05-08T05:42:07.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.098+0000 7f97381af640 1 -- 172.21.15.45:0/781478613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f973015cc20 con 0x7f97300d20c0 2024-05-08T05:42:07.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.098+0000 7f972e7fc640 1 -- 172.21.15.45:0/781478613 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720009450 con 0x7f97300d20c0 2024-05-08T05:42:07.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.098+0000 7f97381af640 1 -- 172.21.15.45:0/781478613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97300b75e0 con 0x7f97300d20c0 2024-05-08T05:42:07.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.099+0000 7f97381af640 1 -- 172.21.15.45:0/781478613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9730050830 con 0x7f97300d20c0 2024-05-08T05:42:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.100+0000 7f972e7fc640 1 -- 172.21.15.45:0/781478613 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9720039040 con 0x7f97300d20c0 2024-05-08T05:42:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.100+0000 7f972e7fc640 1 --2- 172.21.15.45:0/781478613 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f971c0762d0 0x7f971c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.100+0000 7f972e7fc640 1 -- 172.21.15.45:0/781478613 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f97200b8fa0 con 0x7f97300d20c0 2024-05-08T05:42:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.101+0000 7f9735f4b640 1 --2- 172.21.15.45:0/781478613 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f971c0762d0 0x7f971c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.101+0000 7f9735f4b640 1 --2- 172.21.15.45:0/781478613 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f971c0762d0 0x7f971c078790 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9728005880 tx=0x7f9728006aa0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:07.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.105+0000 7f972e7fc640 1 -- 172.21.15.45:0/781478613 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9730050830 con 0x7f97300d20c0 2024-05-08T05:42:07.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.334+0000 7f97381af640 1 -- 172.21.15.45:0/781478613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f973015efc0 con 0x7f97300d20c0 2024-05-08T05:42:07.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.335+0000 7f972e7fc640 1 -- 172.21.15.45:0/781478613 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5174 (secure 0 0 0) 0x7f9720081f60 con 0x7f97300d20c0 2024-05-08T05:42:07.335 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:07.336 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2346958825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2346958825},{"type":"v1","addr":"172.21.15.45:6835","nonce":2346958825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/709809719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":709809719},{"type":"v1","addr":"172.21.15.45:6833","nonce":709809719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3208180302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3208180302},{"type":"v1","addr":"172.21.15.63:6836","nonce":3208180302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:41:31.494380+0000","modified":"2024-05-08T05:41:44.544082+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6837/3653698989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":3653698989},{"type":"v1","addr":"172.21.15.63:6837","nonce":3653698989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4303,"qdb_cluster":[4303]},"id":1}]} 2024-05-08T05:42:07.336 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-05-08T05:42:07.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.337+0000 7f97381af640 1 -- 172.21.15.45:0/781478613 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f971c0762d0 msgr2=0x7f971c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.337+0000 7f97381af640 1 --2- 172.21.15.45:0/781478613 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f971c0762d0 0x7f971c078790 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9728005880 tx=0x7f9728006aa0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.337+0000 7f97381af640 1 -- 172.21.15.45:0/781478613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97300d20c0 msgr2=0x7f97301571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.337+0000 7f97381af640 1 --2- 172.21.15.45:0/781478613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97300d20c0 0x7f97301571b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9720009eb0 tx=0x7f9720004a40 comp rx=0 tx=0).stop 2024-05-08T05:42:07.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.337+0000 7f97381af640 1 -- 172.21.15.45:0/781478613 shutdown_connections 2024-05-08T05:42:07.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.337+0000 7f97381af640 1 --2- 172.21.15.45:0/781478613 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f971c0762d0 0x7f971c078790 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.337+0000 7f97381af640 1 --2- 172.21.15.45:0/781478613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9730071110 0x7f97300c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.337+0000 7f97381af640 1 --2- 172.21.15.45:0/781478613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f97300d20c0 0x7f97301571b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.337+0000 7f97381af640 1 --2- 172.21.15.45:0/781478613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9730070d30 0x7f973015c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.337+0000 7f97381af640 1 -- 172.21.15.45:0/781478613 >> 172.21.15.45:0/781478613 conn(0x7f97300d2e90 msgr2=0x7f97301609f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:07.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.337+0000 7f97381af640 1 -- 172.21.15.45:0/781478613 shutdown_connections 2024-05-08T05:42:07.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.337+0000 7f97381af640 1 -- 172.21.15.45:0/781478613 wait complete. 2024-05-08T05:42:07.348 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:42:07.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.451+0000 7f1d2fd14640 1 Processor -- start 2024-05-08T05:42:07.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.452+0000 7f1d2fd14640 1 -- start start 2024-05-08T05:42:07.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.452+0000 7f1d2fd14640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1d2815f420 0x7f1d280d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.452+0000 7f1d2fd14640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1d280d2080 0x7f1d280d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.452+0000 7f1d2fd14640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d281630b0 con 0x7f1d280d3ca0 2024-05-08T05:42:07.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.452+0000 7f1d2fd14640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f1d280d2550 con 0x7f1d280d2080 2024-05-08T05:42:07.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.452+0000 7f1d2fd14640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1d280d2c80 con 0x7f1d2815f420 2024-05-08T05:42:07.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.453+0000 7f1d2e2b1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1d280d2080 0x7f1d280d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.453+0000 7f1d2d2af640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1d280d3ca0 0x7f1d280d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53242/0 (socket says 172.21.15.45:53242) 2024-05-08T05:42:07.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.453+0000 7f1d2e2b1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1d280d2080 0x7f1d280d4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49110/0 (socket says 172.21.15.45:49110) 2024-05-08T05:42:07.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.453+0000 7f1d2d2af640 1 -- 172.21.15.45:0/3664383539 learned_addr learned my addr 172.21.15.45:0/3664383539 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:07.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.453+0000 7f1d2dab0640 1 --2- 172.21.15.45:0/3664383539 >> v2:172.21.15.63:3300/0 conn(0x7f1d2815f420 0x7f1d280d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.453+0000 7f1d2e2b1640 1 -- 172.21.15.45:0/3664383539 >> v2:172.21.15.63:3300/0 conn(0x7f1d2815f420 msgr2=0x7f1d280d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:07.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.453+0000 7f1d2e2b1640 1 --2- 172.21.15.45:0/3664383539 >> v2:172.21.15.63:3300/0 conn(0x7f1d2815f420 0x7f1d280d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.453+0000 7f1d2e2b1640 1 -- 172.21.15.45:0/3664383539 >> v1:172.21.15.45:6789/0 conn(0x7f1d280d3ca0 legacy=0x7f1d280d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.453+0000 7f1d2caae640 1 -- 172.21.15.45:0/3664383539 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1768455353 0 0) 0x7f1d18001110 con 0x7f1d280d3ca0 2024-05-08T05:42:07.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.453+0000 7f1d2e2b1640 1 -- 172.21.15.45:0/3664383539 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d280d2f60 con 0x7f1d280d2080 2024-05-08T05:42:07.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.453+0000 7f1d2dab0640 1 --2- 172.21.15.45:0/3664383539 >> v2:172.21.15.63:3300/0 conn(0x7f1d2815f420 0x7f1d280d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:07.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.454+0000 7f1d2e2b1640 1 --2- 172.21.15.45:0/3664383539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d280d2080 0x7f1d280d4cc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1d280d5ec0 tx=0x7f1d2002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926acaeef8b957d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:07.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.454+0000 7f1d2caae640 1 -- 172.21.15.45:0/3664383539 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20007740 con 0x7f1d280d2080 2024-05-08T05:42:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.454+0000 7f1d2caae640 1 -- 172.21.15.45:0/3664383539 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d200078e0 con 0x7f1d280d2080 2024-05-08T05:42:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.454+0000 7f1d2caae640 1 -- 172.21.15.45:0/3664383539 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20007cb0 con 0x7f1d280d2080 2024-05-08T05:42:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.454+0000 7f1d2fd14640 1 -- 172.21.15.45:0/3664383539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d280d2080 msgr2=0x7f1d280d4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.454+0000 7f1d2fd14640 1 --2- 172.21.15.45:0/3664383539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d280d2080 0x7f1d280d4cc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1d280d5ec0 tx=0x7f1d2002a680 comp rx=0 tx=0).stop 2024-05-08T05:42:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.454+0000 7f1d2fd14640 1 -- 172.21.15.45:0/3664383539 shutdown_connections 2024-05-08T05:42:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.454+0000 7f1d2fd14640 1 --2- 172.21.15.45:0/3664383539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d280d2080 0x7f1d280d4cc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.454+0000 7f1d2fd14640 1 --2- 172.21.15.45:0/3664383539 >> v2:172.21.15.63:3300/0 conn(0x7f1d2815f420 0x7f1d280d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.454+0000 7f1d2fd14640 1 -- 172.21.15.45:0/3664383539 >> 172.21.15.45:0/3664383539 conn(0x7f1d2808eac0 msgr2=0x7f1d2809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.455+0000 7f1d2fd14640 1 -- 172.21.15.45:0/3664383539 shutdown_connections 2024-05-08T05:42:07.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.455+0000 7f1d2fd14640 1 -- 172.21.15.45:0/3664383539 wait complete. 2024-05-08T05:42:07.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.456+0000 7f1d2fd14640 1 Processor -- start 2024-05-08T05:42:07.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.456+0000 7f1d2fd14640 1 -- start start 2024-05-08T05:42:07.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.456+0000 7f1d2fd14640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d280d2080 0x7f1d2815a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d2fd14640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d280c9d30 0x7f1d280cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d2fd14640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d280ceae0 0x7f1d280ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d2fd14640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1d280d5200 con 0x7f1d280ceae0 2024-05-08T05:42:07.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d2fd14640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1d280d2830 con 0x7f1d280c9d30 2024-05-08T05:42:07.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d2fd14640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1d280d5380 con 0x7f1d280d2080 2024-05-08T05:42:07.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d2e2b1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d280ceae0 0x7f1d280ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d2e2b1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d280ceae0 0x7f1d280ccd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49126/0 (socket says 172.21.15.45:49126) 2024-05-08T05:42:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d2e2b1640 1 -- 172.21.15.45:0/2407459018 learned_addr learned my addr 172.21.15.45:0/2407459018 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d2d2af640 1 --2- 172.21.15.45:0/2407459018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d280c9d30 0x7f1d280cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d2dab0640 1 --2- 172.21.15.45:0/2407459018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d280d2080 0x7f1d2815a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d2e2b1640 1 -- 172.21.15.45:0/2407459018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d280d2080 msgr2=0x7f1d2815a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d2e2b1640 1 --2- 172.21.15.45:0/2407459018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d280d2080 0x7f1d2815a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d2e2b1640 1 -- 172.21.15.45:0/2407459018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d280c9d30 msgr2=0x7f1d280cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d2e2b1640 1 --2- 172.21.15.45:0/2407459018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d280c9d30 0x7f1d280cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d2e2b1640 1 -- 172.21.15.45:0/2407459018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2815a660 con 0x7f1d280ceae0 2024-05-08T05:42:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d2d2af640 1 --2- 172.21.15.45:0/2407459018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d280c9d30 0x7f1d280cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d2e2b1640 1 --2- 172.21.15.45:0/2407459018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d280ceae0 0x7f1d280ccd80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009e50 tx=0x7f1d20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d1e7fc640 1 -- 172.21.15.45:0/2407459018 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20040070 con 0x7f1d280ceae0 2024-05-08T05:42:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d1e7fc640 1 -- 172.21.15.45:0/2407459018 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d20031bf0 con 0x7f1d280ceae0 2024-05-08T05:42:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d2fd14640 1 -- 172.21.15.45:0/2407459018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d280b2b70 con 0x7f1d280ceae0 2024-05-08T05:42:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d1e7fc640 1 -- 172.21.15.45:0/2407459018 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20045460 con 0x7f1d280ceae0 2024-05-08T05:42:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.457+0000 7f1d2fd14640 1 -- 172.21.15.45:0/2407459018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d280b3130 con 0x7f1d280ceae0 2024-05-08T05:42:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.458+0000 7f1d2fd14640 1 -- 172.21.15.45:0/2407459018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d280693f0 con 0x7f1d280ceae0 2024-05-08T05:42:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.459+0000 7f1d1e7fc640 1 -- 172.21.15.45:0/2407459018 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1d20039040 con 0x7f1d280ceae0 2024-05-08T05:42:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.460+0000 7f1d1e7fc640 1 --2- 172.21.15.45:0/2407459018 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1d0c076300 0x7f1d0c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.460+0000 7f1d1e7fc640 1 -- 172.21.15.45:0/2407459018 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1d200b9250 con 0x7f1d280ceae0 2024-05-08T05:42:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.460+0000 7f1d2dab0640 1 --2- 172.21.15.45:0/2407459018 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1d0c076300 0x7f1d0c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.460+0000 7f1d2dab0640 1 --2- 172.21.15.45:0/2407459018 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1d0c076300 0x7f1d0c0787c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1d280b3e10 tx=0x7f1d24008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.464+0000 7f1d1e7fc640 1 -- 172.21.15.45:0/2407459018 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1d2815a660 con 0x7f1d280ceae0 2024-05-08T05:42:07.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.692+0000 7f1d2fd14640 1 -- 172.21.15.45:0/2407459018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d280b4c50 con 0x7f1d280ceae0 2024-05-08T05:42:07.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.693+0000 7f1d1e7fc640 1 -- 172.21.15.45:0/2407459018 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5174 (secure 0 0 0) 0x7f1d20082900 con 0x7f1d280ceae0 2024-05-08T05:42:07.694 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:07.694 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2346958825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2346958825},{"type":"v1","addr":"172.21.15.45:6835","nonce":2346958825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/709809719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":709809719},{"type":"v1","addr":"172.21.15.45:6833","nonce":709809719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3208180302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3208180302},{"type":"v1","addr":"172.21.15.63:6836","nonce":3208180302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:41:31.494380+0000","modified":"2024-05-08T05:41:44.544082+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6837/3653698989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":3653698989},{"type":"v1","addr":"172.21.15.63:6837","nonce":3653698989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4303,"qdb_cluster":[4303]},"id":1}]} 2024-05-08T05:42:07.694 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-05-08T05:42:07.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.695+0000 7f1d2fd14640 1 -- 172.21.15.45:0/2407459018 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1d0c076300 msgr2=0x7f1d0c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.695+0000 7f1d2fd14640 1 --2- 172.21.15.45:0/2407459018 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1d0c076300 0x7f1d0c0787c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1d280b3e10 tx=0x7f1d24008040 comp rx=0 tx=0).stop 2024-05-08T05:42:07.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.695+0000 7f1d2fd14640 1 -- 172.21.15.45:0/2407459018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d280ceae0 msgr2=0x7f1d280ccd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.695+0000 7f1d2fd14640 1 --2- 172.21.15.45:0/2407459018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d280ceae0 0x7f1d280ccd80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009e50 tx=0x7f1d20004a40 comp rx=0 tx=0).stop 2024-05-08T05:42:07.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.695+0000 7f1d2fd14640 1 -- 172.21.15.45:0/2407459018 shutdown_connections 2024-05-08T05:42:07.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.695+0000 7f1d2fd14640 1 --2- 172.21.15.45:0/2407459018 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1d0c076300 0x7f1d0c0787c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.695+0000 7f1d2fd14640 1 --2- 172.21.15.45:0/2407459018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d280ceae0 0x7f1d280ccd80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.695+0000 7f1d2fd14640 1 --2- 172.21.15.45:0/2407459018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d280c9d30 0x7f1d280cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.695+0000 7f1d2fd14640 1 --2- 172.21.15.45:0/2407459018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d280d2080 0x7f1d2815a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.695+0000 7f1d2fd14640 1 -- 172.21.15.45:0/2407459018 >> 172.21.15.45:0/2407459018 conn(0x7f1d2808eac0 msgr2=0x7f1d2806d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:07.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.695+0000 7f1d2fd14640 1 -- 172.21.15.45:0/2407459018 shutdown_connections 2024-05-08T05:42:07.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.695+0000 7f1d2fd14640 1 -- 172.21.15.45:0/2407459018 wait complete. 2024-05-08T05:42:07.706 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:42:07.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.810+0000 7f1f1167b640 1 Processor -- start 2024-05-08T05:42:07.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.810+0000 7f1f1167b640 1 -- start start 2024-05-08T05:42:07.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.811+0000 7f1f1167b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1f0c0d0ec0 0x7f1f0c0d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.811+0000 7f1f1167b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f0c0d3b00 0x7f1f0c0b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.811+0000 7f1f1167b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f0c0d1760 con 0x7f1f0c15f380 2024-05-08T05:42:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.811+0000 7f1f1167b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f1f0c0d30f0 con 0x7f1f0c0d0ec0 2024-05-08T05:42:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.811+0000 7f1f1167b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1f0c0d3f70 con 0x7f1f0c0d3b00 2024-05-08T05:42:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.811+0000 7f1f0a7fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1f0c0d0ec0 0x7f1f0c0d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.811+0000 7f1f0affd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1f0c15f380 0x7f1f0c0d07b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53254/0 (socket says 172.21.15.45:53254) 2024-05-08T05:42:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.811+0000 7f1f0affd640 1 -- 172.21.15.45:0/1846425239 learned_addr learned my addr 172.21.15.45:0/1846425239 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.811+0000 7f1f0b7fe640 1 --2- 172.21.15.45:0/1846425239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f0c0d3b00 0x7f1f0c0b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.811+0000 7f1f0a7fc640 1 -- 172.21.15.45:0/1846425239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f0c0d3b00 msgr2=0x7f1f0c0b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.811+0000 7f1f0a7fc640 1 --2- 172.21.15.45:0/1846425239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f0c0d3b00 0x7f1f0c0b7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.811+0000 7f1f0a7fc640 1 -- 172.21.15.45:0/1846425239 >> v1:172.21.15.45:6789/0 conn(0x7f1f0c15f380 legacy=0x7f1f0c0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.811+0000 7f1f0a7fc640 1 -- 172.21.15.45:0/1846425239 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0c0d1a40 con 0x7f1f0c0d0ec0 2024-05-08T05:42:07.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.811+0000 7f1f09ffb640 1 -- 172.21.15.45:0/1846425239 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 870065142 0 0) 0x7f1f0c0d1760 con 0x7f1f0c15f380 2024-05-08T05:42:07.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.812+0000 7f1f0a7fc640 1 --2- 172.21.15.45:0/1846425239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f0c0d0ec0 0x7f1f0c0d35c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1f000097a0 tx=0x7f1f0002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cf8aa223ef4dc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.812+0000 7f1f09ffb640 1 -- 172.21.15.45:0/1846425239 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00037070 con 0x7f1f0c0d0ec0 2024-05-08T05:42:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.812+0000 7f1f09ffb640 1 -- 172.21.15.45:0/1846425239 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f00032440 con 0x7f1f0c0d0ec0 2024-05-08T05:42:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.812+0000 7f1f09ffb640 1 -- 172.21.15.45:0/1846425239 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00032810 con 0x7f1f0c0d0ec0 2024-05-08T05:42:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.812+0000 7f1f1167b640 1 -- 172.21.15.45:0/1846425239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f0c0d0ec0 msgr2=0x7f1f0c0d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.812+0000 7f1f1167b640 1 --2- 172.21.15.45:0/1846425239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f0c0d0ec0 0x7f1f0c0d35c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1f000097a0 tx=0x7f1f0002a180 comp rx=0 tx=0).stop 2024-05-08T05:42:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.812+0000 7f1f1167b640 1 -- 172.21.15.45:0/1846425239 shutdown_connections 2024-05-08T05:42:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.812+0000 7f1f1167b640 1 --2- 172.21.15.45:0/1846425239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f0c0d3b00 0x7f1f0c0b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.812+0000 7f1f1167b640 1 --2- 172.21.15.45:0/1846425239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f0c0d0ec0 0x7f1f0c0d35c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.812+0000 7f1f1167b640 1 -- 172.21.15.45:0/1846425239 >> 172.21.15.45:0/1846425239 conn(0x7f1f0c0d4fe0 msgr2=0x7f1f0c0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.813+0000 7f1f1167b640 1 -- 172.21.15.45:0/1846425239 shutdown_connections 2024-05-08T05:42:07.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.813+0000 7f1f1167b640 1 -- 172.21.15.45:0/1846425239 wait complete. 2024-05-08T05:42:07.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.814+0000 7f1f1167b640 1 Processor -- start 2024-05-08T05:42:07.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.814+0000 7f1f1167b640 1 -- start start 2024-05-08T05:42:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1f1167b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f0c0d0ec0 0x7f1f0c0b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1f1167b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f0c0d3b00 0x7f1f0c0b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1f1167b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f0c15f380 0x7f1f0c1f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1f1167b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c0d2000 con 0x7f1f0c15f380 2024-05-08T05:42:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1f1167b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c0d2370 con 0x7f1f0c0d0ec0 2024-05-08T05:42:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1f1167b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1f0c15f760 con 0x7f1f0c0d3b00 2024-05-08T05:42:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1f0b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f0c15f380 0x7f1f0c1f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1f0b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f0c15f380 0x7f1f0c1f50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49148/0 (socket says 172.21.15.45:49148) 2024-05-08T05:42:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1f0b7fe640 1 -- 172.21.15.45:0/193282764 learned_addr learned my addr 172.21.15.45:0/193282764 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1f0a7fc640 1 --2- 172.21.15.45:0/193282764 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f0c0d3b00 0x7f1f0c0b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1f0affd640 1 --2- 172.21.15.45:0/193282764 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f0c0d0ec0 0x7f1f0c0b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1f0b7fe640 1 -- 172.21.15.45:0/193282764 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f0c0d3b00 msgr2=0x7f1f0c0b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1f0b7fe640 1 --2- 172.21.15.45:0/193282764 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f0c0d3b00 0x7f1f0c0b2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1f0b7fe640 1 -- 172.21.15.45:0/193282764 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f0c0d0ec0 msgr2=0x7f1f0c0b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1f0b7fe640 1 --2- 172.21.15.45:0/193282764 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f0c0d0ec0 0x7f1f0c0b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1f0b7fe640 1 -- 172.21.15.45:0/193282764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0c0b3280 con 0x7f1f0c15f380 2024-05-08T05:42:07.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1f0affd640 1 --2- 172.21.15.45:0/193282764 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f0c0d0ec0 0x7f1f0c0b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:07.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1f0b7fe640 1 --2- 172.21.15.45:0/193282764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f0c15f380 0x7f1f0c1f50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1f04007e40 tx=0x7f1f0400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:07.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1ef37fe640 1 -- 172.21.15.45:0/193282764 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04014620 con 0x7f1f0c15f380 2024-05-08T05:42:07.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1ef37fe640 1 -- 172.21.15.45:0/193282764 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f04019070 con 0x7f1f0c15f380 2024-05-08T05:42:07.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1f1167b640 1 -- 172.21.15.45:0/193282764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f0c0b3560 con 0x7f1f0c15f380 2024-05-08T05:42:07.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.815+0000 7f1ef37fe640 1 -- 172.21.15.45:0/193282764 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f04014a20 con 0x7f1f0c15f380 2024-05-08T05:42:07.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.816+0000 7f1f1167b640 1 -- 172.21.15.45:0/193282764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f0c0b3b20 con 0x7f1f0c15f380 2024-05-08T05:42:07.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.817+0000 7f1f1167b640 1 -- 172.21.15.45:0/193282764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f0c0693f0 con 0x7f1f0c15f380 2024-05-08T05:42:07.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.817+0000 7f1ef37fe640 1 -- 172.21.15.45:0/193282764 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1f04014bc0 con 0x7f1f0c15f380 2024-05-08T05:42:07.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.818+0000 7f1ef37fe640 1 --2- 172.21.15.45:0/193282764 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1ef80762d0 0x7f1ef8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.818+0000 7f1ef37fe640 1 -- 172.21.15.45:0/193282764 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f1f040981a0 con 0x7f1f0c15f380 2024-05-08T05:42:07.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.818+0000 7f1f0affd640 1 --2- 172.21.15.45:0/193282764 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1ef80762d0 0x7f1ef8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.819+0000 7f1f0affd640 1 --2- 172.21.15.45:0/193282764 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1ef80762d0 0x7f1ef8078790 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1efc000f60 tx=0x7f1efc007bd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:07.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:07.822+0000 7f1ef37fe640 1 -- 172.21.15.45:0/193282764 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1f04061850 con 0x7f1f0c15f380 2024-05-08T05:42:08.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.013+0000 7f1f1167b640 1 -- 172.21.15.45:0/193282764 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f0c08cfd0 con 0x7f1f0c15f380 2024-05-08T05:42:08.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.014+0000 7f1ef37fe640 1 -- 172.21.15.45:0/193282764 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18466 (secure 0 0 0) 0x7f1f04061150 con 0x7f1f0c15f380 2024-05-08T05:42:08.015 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:08.015 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:41:36.532045+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:41:29.381274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:41:30.443612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:41:34.132285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:42:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.016+0000 7f1f1167b640 1 -- 172.21.15.45:0/193282764 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1ef80762d0 msgr2=0x7f1ef8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.016+0000 7f1f1167b640 1 --2- 172.21.15.45:0/193282764 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1ef80762d0 0x7f1ef8078790 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1efc000f60 tx=0x7f1efc007bd0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.016+0000 7f1f1167b640 1 -- 172.21.15.45:0/193282764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f0c15f380 msgr2=0x7f1f0c1f50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.016+0000 7f1f1167b640 1 --2- 172.21.15.45:0/193282764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f0c15f380 0x7f1f0c1f50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1f04007e40 tx=0x7f1f0400e4b0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.016+0000 7f1f1167b640 1 -- 172.21.15.45:0/193282764 shutdown_connections 2024-05-08T05:42:08.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.016+0000 7f1f1167b640 1 --2- 172.21.15.45:0/193282764 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1ef80762d0 0x7f1ef8078790 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.016+0000 7f1f1167b640 1 --2- 172.21.15.45:0/193282764 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f0c0d3b00 0x7f1f0c0b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.016+0000 7f1f1167b640 1 --2- 172.21.15.45:0/193282764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f0c15f380 0x7f1f0c1f50e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.016+0000 7f1f1167b640 1 --2- 172.21.15.45:0/193282764 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f0c0d0ec0 0x7f1f0c0b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.016+0000 7f1f1167b640 1 -- 172.21.15.45:0/193282764 >> 172.21.15.45:0/193282764 conn(0x7f1f0c0d4fe0 msgr2=0x7f1f0c0d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.016+0000 7f1f1167b640 1 -- 172.21.15.45:0/193282764 shutdown_connections 2024-05-08T05:42:08.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.016+0000 7f1f1167b640 1 -- 172.21.15.45:0/193282764 wait complete. 2024-05-08T05:42:08.028 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T05:42:08.029 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:42:08.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.133+0000 7f6cdbf12640 1 Processor -- start 2024-05-08T05:42:08.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.133+0000 7f6cdbf12640 1 -- start start 2024-05-08T05:42:08.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.134+0000 7f6cdbf12640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6cd40d3cd0 0x7f6cd416a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:08.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.134+0000 7f6cdbf12640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cd40d1760 con 0x7f6cd40d0ec0 2024-05-08T05:42:08.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.134+0000 7f6cdbf12640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cd40d1a40 con 0x7f6cd415f380 2024-05-08T05:42:08.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.134+0000 7f6cdbf12640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f6cd40d30f0 con 0x7f6cd40d3cd0 2024-05-08T05:42:08.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.134+0000 7f6cda4af640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6cd40d3cd0 0x7f6cd416a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:08.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.134+0000 7f6cd94ad640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6cd40d0ec0 0x7f6cd40d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53260/0 (socket says 172.21.15.45:53260) 2024-05-08T05:42:08.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.134+0000 7f6cd94ad640 1 -- 172.21.15.45:0/983050329 learned_addr learned my addr 172.21.15.45:0/983050329 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:08.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.134+0000 7f6cda4af640 1 -- 172.21.15.45:0/983050329 >> v1:172.21.15.63:6789/0 conn(0x7f6cd415f380 legacy=0x7f6cd40d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:08.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.134+0000 7f6cda4af640 1 -- 172.21.15.45:0/983050329 >> v1:172.21.15.45:6789/0 conn(0x7f6cd40d0ec0 legacy=0x7f6cd40d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:08.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.134+0000 7f6cda4af640 1 -- 172.21.15.45:0/983050329 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd40d1d20 con 0x7f6cd40d3cd0 2024-05-08T05:42:08.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.135+0000 7f6cda4af640 1 --2- 172.21.15.45:0/983050329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cd40d3cd0 0x7f6cd416a1f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009d90 tx=0x7f6ccc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d659c3f5e53f840 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:08.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.135+0000 7f6cd8cac640 1 -- 172.21.15.45:0/983050329 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc007740 con 0x7f6cd40d3cd0 2024-05-08T05:42:08.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.135+0000 7f6cd8cac640 1 -- 172.21.15.45:0/983050329 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ccc0078e0 con 0x7f6cd40d3cd0 2024-05-08T05:42:08.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.135+0000 7f6cd8cac640 1 -- 172.21.15.45:0/983050329 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc007cb0 con 0x7f6cd40d3cd0 2024-05-08T05:42:08.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.136+0000 7f6cdbf12640 1 -- 172.21.15.45:0/983050329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cd40d3cd0 msgr2=0x7f6cd416a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:08.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.136+0000 7f6cdbf12640 1 --2- 172.21.15.45:0/983050329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cd40d3cd0 0x7f6cd416a1f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009d90 tx=0x7f6ccc02a6f0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.136+0000 7f6cdbf12640 1 -- 172.21.15.45:0/983050329 shutdown_connections 2024-05-08T05:42:08.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.136+0000 7f6cdbf12640 1 --2- 172.21.15.45:0/983050329 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cd40d3cd0 0x7f6cd416a1f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.136+0000 7f6cdbf12640 1 -- 172.21.15.45:0/983050329 >> 172.21.15.45:0/983050329 conn(0x7f6cd40d4fe0 msgr2=0x7f6cd40d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:08.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.136+0000 7f6cdbf12640 1 -- 172.21.15.45:0/983050329 shutdown_connections 2024-05-08T05:42:08.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.136+0000 7f6cdbf12640 1 -- 172.21.15.45:0/983050329 wait complete. 2024-05-08T05:42:08.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.137+0000 7f6cdbf12640 1 Processor -- start 2024-05-08T05:42:08.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.137+0000 7f6cdbf12640 1 -- start start 2024-05-08T05:42:08.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cdbf12640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cd40d0ec0 0x7f6cd40a2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:08.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cdbf12640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cd40d3cd0 0x7f6cd40b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:08.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cdbf12640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cd415f380 0x7f6cd40b8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:08.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cdbf12640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd40b8e40 con 0x7f6cd40d0ec0 2024-05-08T05:42:08.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cdbf12640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd40d2370 con 0x7f6cd40d3cd0 2024-05-08T05:42:08.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cdbf12640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6cd415f760 con 0x7f6cd415f380 2024-05-08T05:42:08.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cd9cae640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cd40d0ec0 0x7f6cd40a2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:08.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cd9cae640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cd40d0ec0 0x7f6cd40a2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49164/0 (socket says 172.21.15.45:49164) 2024-05-08T05:42:08.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cd9cae640 1 -- 172.21.15.45:0/3913516433 learned_addr learned my addr 172.21.15.45:0/3913516433 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:08.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cda4af640 1 --2- 172.21.15.45:0/3913516433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cd415f380 0x7f6cd40b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:08.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cd94ad640 1 --2- 172.21.15.45:0/3913516433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cd40d3cd0 0x7f6cd40b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:08.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cd9cae640 1 -- 172.21.15.45:0/3913516433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cd415f380 msgr2=0x7f6cd40b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:08.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cd9cae640 1 --2- 172.21.15.45:0/3913516433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cd415f380 0x7f6cd40b8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cd9cae640 1 -- 172.21.15.45:0/3913516433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cd40d3cd0 msgr2=0x7f6cd40b7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:08.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cd9cae640 1 --2- 172.21.15.45:0/3913516433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cd40d3cd0 0x7f6cd40b7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cd9cae640 1 -- 172.21.15.45:0/3913516433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd4157c10 con 0x7f6cd40d0ec0 2024-05-08T05:42:08.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cd94ad640 1 --2- 172.21.15.45:0/3913516433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cd40d3cd0 0x7f6cd40b7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:08.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cd9cae640 1 --2- 172.21.15.45:0/3913516433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cd40d0ec0 0x7f6cd40a2340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000c450 tx=0x7f6cd000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:08.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cca7fc640 1 -- 172.21.15.45:0/3913516433 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd00052d0 con 0x7f6cd40d0ec0 2024-05-08T05:42:08.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cca7fc640 1 -- 172.21.15.45:0/3913516433 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cd0005470 con 0x7f6cd40d0ec0 2024-05-08T05:42:08.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cdbf12640 1 -- 172.21.15.45:0/3913516433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd4156b70 con 0x7f6cd40d0ec0 2024-05-08T05:42:08.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cca7fc640 1 -- 172.21.15.45:0/3913516433 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0012870 con 0x7f6cd40d0ec0 2024-05-08T05:42:08.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.138+0000 7f6cdbf12640 1 -- 172.21.15.45:0/3913516433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd4157130 con 0x7f6cd40d0ec0 2024-05-08T05:42:08.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.139+0000 7f6cdbf12640 1 -- 172.21.15.45:0/3913516433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd40693f0 con 0x7f6cd40d0ec0 2024-05-08T05:42:08.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.140+0000 7f6cca7fc640 1 -- 172.21.15.45:0/3913516433 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6cd0012a10 con 0x7f6cd40d0ec0 2024-05-08T05:42:08.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.141+0000 7f6cca7fc640 1 --2- 172.21.15.45:0/3913516433 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6cb80762d0 0x7f6cb8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:08.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.141+0000 7f6cca7fc640 1 -- 172.21.15.45:0/3913516433 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f6cd0097550 con 0x7f6cd40d0ec0 2024-05-08T05:42:08.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.141+0000 7f6cd94ad640 1 --2- 172.21.15.45:0/3913516433 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6cb80762d0 0x7f6cb8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:08.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.141+0000 7f6cd94ad640 1 --2- 172.21.15.45:0/3913516433 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6cb80762d0 0x7f6cb8078790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6cd415e4d0 tx=0x7f6cc4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:08.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.145+0000 7f6cca7fc640 1 -- 172.21.15.45:0/3913516433 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6cd0060b70 con 0x7f6cd40d0ec0 2024-05-08T05:42:08.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.378+0000 7f6cdbf12640 1 -- 172.21.15.45:0/3913516433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cd408cfd0 con 0x7f6cd40d0ec0 2024-05-08T05:42:08.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.379+0000 7f6cca7fc640 1 -- 172.21.15.45:0/3913516433 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5174 (secure 0 0 0) 0x7f6cd0060470 con 0x7f6cd40d0ec0 2024-05-08T05:42:08.379 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:08.379 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2346958825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2346958825},{"type":"v1","addr":"172.21.15.45:6835","nonce":2346958825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/709809719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":709809719},{"type":"v1","addr":"172.21.15.45:6833","nonce":709809719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3208180302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3208180302},{"type":"v1","addr":"172.21.15.63:6836","nonce":3208180302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:41:31.494380+0000","modified":"2024-05-08T05:41:44.544082+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6837/3653698989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":3653698989},{"type":"v1","addr":"172.21.15.63:6837","nonce":3653698989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4303,"qdb_cluster":[4303]},"id":1}]} 2024-05-08T05:42:08.380 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-05-08T05:42:08.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.381+0000 7f6cdbf12640 1 -- 172.21.15.45:0/3913516433 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6cb80762d0 msgr2=0x7f6cb8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:08.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.381+0000 7f6cdbf12640 1 --2- 172.21.15.45:0/3913516433 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6cb80762d0 0x7f6cb8078790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6cd415e4d0 tx=0x7f6cc4009040 comp rx=0 tx=0).stop 2024-05-08T05:42:08.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.381+0000 7f6cdbf12640 1 -- 172.21.15.45:0/3913516433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cd40d0ec0 msgr2=0x7f6cd40a2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:08.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.381+0000 7f6cdbf12640 1 --2- 172.21.15.45:0/3913516433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cd40d0ec0 0x7f6cd40a2340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6cd000c450 tx=0x7f6cd000c910 comp rx=0 tx=0).stop 2024-05-08T05:42:08.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.381+0000 7f6cdbf12640 1 -- 172.21.15.45:0/3913516433 shutdown_connections 2024-05-08T05:42:08.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.381+0000 7f6cdbf12640 1 --2- 172.21.15.45:0/3913516433 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6cb80762d0 0x7f6cb8078790 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.381+0000 7f6cdbf12640 1 --2- 172.21.15.45:0/3913516433 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cd415f380 0x7f6cd40b8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.381+0000 7f6cdbf12640 1 --2- 172.21.15.45:0/3913516433 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cd40d3cd0 0x7f6cd40b7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.381+0000 7f6cdbf12640 1 --2- 172.21.15.45:0/3913516433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cd40d0ec0 0x7f6cd40a2340 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.381+0000 7f6cdbf12640 1 -- 172.21.15.45:0/3913516433 >> 172.21.15.45:0/3913516433 conn(0x7f6cd40d4fe0 msgr2=0x7f6cd406e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:08.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.382+0000 7f6cdbf12640 1 -- 172.21.15.45:0/3913516433 shutdown_connections 2024-05-08T05:42:08.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.382+0000 7f6cdbf12640 1 -- 172.21.15.45:0/3913516433 wait complete. 2024-05-08T05:42:08.394 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:42:08.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.497+0000 7f3527378640 1 Processor -- start 2024-05-08T05:42:08.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.497+0000 7f3527378640 1 -- start start 2024-05-08T05:42:08.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.498+0000 7f3527378640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35200d0ec0 0x7f35200d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:08.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.498+0000 7f3527378640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35200d1760 con 0x7f35200d3b00 2024-05-08T05:42:08.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.498+0000 7f3527378640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35200d1a40 con 0x7f352015f380 2024-05-08T05:42:08.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.498+0000 7f3527378640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f35200d30f0 con 0x7f35200d0ec0 2024-05-08T05:42:08.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.498+0000 7f3524913640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35200d0ec0 0x7f35200d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:08.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.498+0000 7f3525915640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f35200d3b00 0x7f35200b7e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53262/0 (socket says 172.21.15.45:53262) 2024-05-08T05:42:08.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.498+0000 7f3525915640 1 -- 172.21.15.45:0/3518385993 learned_addr learned my addr 172.21.15.45:0/3518385993 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:08.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.498+0000 7f3524913640 1 -- 172.21.15.45:0/3518385993 >> v1:172.21.15.63:6789/0 conn(0x7f352015f380 legacy=0x7f35200d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:08.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.498+0000 7f3524913640 1 -- 172.21.15.45:0/3518385993 >> v1:172.21.15.45:6789/0 conn(0x7f35200d3b00 legacy=0x7f35200b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:08.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.498+0000 7f3524913640 1 -- 172.21.15.45:0/3518385993 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35200d1d20 con 0x7f35200d0ec0 2024-05-08T05:42:08.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.498+0000 7f351ffff640 1 -- 172.21.15.45:0/3518385993 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1586853577 0 0) 0x7f35200d1760 con 0x7f35200d3b00 2024-05-08T05:42:08.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.499+0000 7f3524913640 1 --2- 172.21.15.45:0/3518385993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35200d0ec0 0x7f35200d35c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f35140097a0 tx=0x7f351402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05ca91dc556b34d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:08.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.499+0000 7f351ffff640 1 -- 172.21.15.45:0/3518385993 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514037070 con 0x7f35200d0ec0 2024-05-08T05:42:08.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.499+0000 7f351ffff640 1 -- 172.21.15.45:0/3518385993 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3514032440 con 0x7f35200d0ec0 2024-05-08T05:42:08.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.499+0000 7f351ffff640 1 -- 172.21.15.45:0/3518385993 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514032810 con 0x7f35200d0ec0 2024-05-08T05:42:08.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.500+0000 7f3527378640 1 -- 172.21.15.45:0/3518385993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35200d0ec0 msgr2=0x7f35200d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:08.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.500+0000 7f3527378640 1 --2- 172.21.15.45:0/3518385993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35200d0ec0 0x7f35200d35c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f35140097a0 tx=0x7f351402a180 comp rx=0 tx=0).stop 2024-05-08T05:42:08.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.500+0000 7f3527378640 1 -- 172.21.15.45:0/3518385993 shutdown_connections 2024-05-08T05:42:08.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.500+0000 7f3527378640 1 --2- 172.21.15.45:0/3518385993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35200d0ec0 0x7f35200d35c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.500+0000 7f3527378640 1 -- 172.21.15.45:0/3518385993 >> 172.21.15.45:0/3518385993 conn(0x7f35200d4fe0 msgr2=0x7f35200d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:08.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.500+0000 7f3527378640 1 -- 172.21.15.45:0/3518385993 shutdown_connections 2024-05-08T05:42:08.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.500+0000 7f3527378640 1 -- 172.21.15.45:0/3518385993 wait complete. 2024-05-08T05:42:08.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.501+0000 7f3527378640 1 Processor -- start 2024-05-08T05:42:08.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.501+0000 7f3527378640 1 -- start start 2024-05-08T05:42:08.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.502+0000 7f3527378640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35200d0ec0 0x7f35201629a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:08.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.502+0000 7f3527378640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35200d3b00 0x7f3520162ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:08.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.502+0000 7f3527378640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f352015f380 0x7f3520150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:08.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.502+0000 7f3527378640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f35200d3f30 con 0x7f352015f380 2024-05-08T05:42:08.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.502+0000 7f3527378640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f35200d2370 con 0x7f35200d0ec0 2024-05-08T05:42:08.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.502+0000 7f3527378640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f352015f760 con 0x7f35200d3b00 2024-05-08T05:42:08.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.502+0000 7f3525915640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f352015f380 0x7f3520150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:08.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.502+0000 7f3525915640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f352015f380 0x7f3520150810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49176/0 (socket says 172.21.15.45:49176) 2024-05-08T05:42:08.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.502+0000 7f3525915640 1 -- 172.21.15.45:0/3249523067 learned_addr learned my addr 172.21.15.45:0/3249523067 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:08.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.502+0000 7f3525915640 1 -- 172.21.15.45:0/3249523067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35200d3b00 msgr2=0x7f3520162ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:08.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.502+0000 7f3525915640 1 --2- 172.21.15.45:0/3249523067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35200d3b00 0x7f3520162ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.502+0000 7f3525915640 1 -- 172.21.15.45:0/3249523067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35200d0ec0 msgr2=0x7f35201629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:08.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.502+0000 7f3525114640 1 --2- 172.21.15.45:0/3249523067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35200d0ec0 0x7f35201629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:08.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.502+0000 7f3525915640 1 --2- 172.21.15.45:0/3249523067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35200d0ec0 0x7f35201629a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.502+0000 7f3525915640 1 -- 172.21.15.45:0/3249523067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35200b6810 con 0x7f352015f380 2024-05-08T05:42:08.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.502+0000 7f3525915640 1 --2- 172.21.15.45:0/3249523067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f352015f380 0x7f3520150810 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f351000c7a0 tx=0x7f351000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:08.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.503+0000 7f351dffb640 1 -- 172.21.15.45:0/3249523067 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510018070 con 0x7f352015f380 2024-05-08T05:42:08.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.503+0000 7f351dffb640 1 -- 172.21.15.45:0/3249523067 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f351000a5e0 con 0x7f352015f380 2024-05-08T05:42:08.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.503+0000 7f351dffb640 1 -- 172.21.15.45:0/3249523067 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351000aa50 con 0x7f352015f380 2024-05-08T05:42:08.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.503+0000 7f3527378640 1 -- 172.21.15.45:0/3249523067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35200b6af0 con 0x7f352015f380 2024-05-08T05:42:08.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.503+0000 7f3527378640 1 -- 172.21.15.45:0/3249523067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f352014e580 con 0x7f352015f380 2024-05-08T05:42:08.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.504+0000 7f3527378640 1 -- 172.21.15.45:0/3249523067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35200693f0 con 0x7f352015f380 2024-05-08T05:42:08.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.504+0000 7f351dffb640 1 -- 172.21.15.45:0/3249523067 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f351000abf0 con 0x7f352015f380 2024-05-08T05:42:08.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.505+0000 7f351dffb640 1 --2- 172.21.15.45:0/3249523067 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f350c0762d0 0x7f350c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:08.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.505+0000 7f351dffb640 1 -- 172.21.15.45:0/3249523067 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f3510096f30 con 0x7f352015f380 2024-05-08T05:42:08.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.505+0000 7f3525114640 1 --2- 172.21.15.45:0/3249523067 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f350c0762d0 0x7f350c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:08.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.506+0000 7f3525114640 1 --2- 172.21.15.45:0/3249523067 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f350c0762d0 0x7f350c078790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f351800a750 tx=0x7f3518008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:08.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.510+0000 7f351dffb640 1 -- 172.21.15.45:0/3249523067 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3510060550 con 0x7f352015f380 2024-05-08T05:42:08.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.740+0000 7f3527378640 1 -- 172.21.15.45:0/3249523067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35200ba6f0 con 0x7f352015f380 2024-05-08T05:42:08.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.741+0000 7f351dffb640 1 -- 172.21.15.45:0/3249523067 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5174 (secure 0 0 0) 0x7f351005fe50 con 0x7f352015f380 2024-05-08T05:42:08.742 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:08.742 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2346958825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2346958825},{"type":"v1","addr":"172.21.15.45:6835","nonce":2346958825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/709809719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":709809719},{"type":"v1","addr":"172.21.15.45:6833","nonce":709809719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3208180302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3208180302},{"type":"v1","addr":"172.21.15.63:6836","nonce":3208180302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:41:31.494380+0000","modified":"2024-05-08T05:41:44.544082+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6837/3653698989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":3653698989},{"type":"v1","addr":"172.21.15.63:6837","nonce":3653698989}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4303,"qdb_cluster":[4303]},"id":1}]} 2024-05-08T05:42:08.742 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 8 2024-05-08T05:42:08.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.743+0000 7f3527378640 1 -- 172.21.15.45:0/3249523067 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f350c0762d0 msgr2=0x7f350c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:08.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.743+0000 7f3527378640 1 --2- 172.21.15.45:0/3249523067 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f350c0762d0 0x7f350c078790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f351800a750 tx=0x7f3518008040 comp rx=0 tx=0).stop 2024-05-08T05:42:08.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.743+0000 7f3527378640 1 -- 172.21.15.45:0/3249523067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f352015f380 msgr2=0x7f3520150810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:08.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.743+0000 7f3527378640 1 --2- 172.21.15.45:0/3249523067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f352015f380 0x7f3520150810 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f351000c7a0 tx=0x7f351000cc60 comp rx=0 tx=0).stop 2024-05-08T05:42:08.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.743+0000 7f3527378640 1 -- 172.21.15.45:0/3249523067 shutdown_connections 2024-05-08T05:42:08.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.743+0000 7f3527378640 1 --2- 172.21.15.45:0/3249523067 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f350c0762d0 0x7f350c078790 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.743+0000 7f3527378640 1 --2- 172.21.15.45:0/3249523067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35200d3b00 0x7f3520162ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.743+0000 7f3527378640 1 --2- 172.21.15.45:0/3249523067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f352015f380 0x7f3520150810 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.743+0000 7f3527378640 1 --2- 172.21.15.45:0/3249523067 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35200d0ec0 0x7f35201629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.743+0000 7f3527378640 1 -- 172.21.15.45:0/3249523067 >> 172.21.15.45:0/3249523067 conn(0x7f35200d4fe0 msgr2=0x7f35200cb390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:08.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.744+0000 7f3527378640 1 -- 172.21.15.45:0/3249523067 shutdown_connections 2024-05-08T05:42:08.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.744+0000 7f3527378640 1 -- 172.21.15.45:0/3249523067 wait complete. 2024-05-08T05:42:08.755 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:42:08.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.859+0000 7fa1cc4d1640 1 Processor -- start 2024-05-08T05:42:08.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.859+0000 7fa1cc4d1640 1 -- start start 2024-05-08T05:42:08.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.860+0000 7fa1cc4d1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1c4070d30 0x7fa1c40d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:08.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.860+0000 7fa1cc4d1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1c4163920 0x7fa1c40703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:08.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.860+0000 7fa1cc4d1640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1c4163d80 con 0x7fa1c4071110 2024-05-08T05:42:08.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.860+0000 7fa1cc4d1640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa1c40d10d0 con 0x7fa1c4163920 2024-05-08T05:42:08.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.860+0000 7fa1cc4d1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c4070b80 con 0x7fa1c4070d30 2024-05-08T05:42:08.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.860+0000 7fa1c9a6c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1c4163920 0x7fa1c40703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:08.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.860+0000 7fa1c9a6c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1c4163920 0x7fa1c40703a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49192/0 (socket says 172.21.15.45:49192) 2024-05-08T05:42:08.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.860+0000 7fa1c9a6c640 1 -- 172.21.15.45:0/1218516171 learned_addr learned my addr 172.21.15.45:0/1218516171 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:08.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.861+0000 7fa1ca26d640 1 --2- 172.21.15.45:0/1218516171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1c4070d30 0x7fa1c40d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:08.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.861+0000 7fa1c9a6c640 1 -- 172.21.15.45:0/1218516171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1c4070d30 msgr2=0x7fa1c40d20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:08.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.861+0000 7fa1c9a6c640 1 --2- 172.21.15.45:0/1218516171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1c4070d30 0x7fa1c40d20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.861+0000 7fa1c9a6c640 1 -- 172.21.15.45:0/1218516171 >> v1:172.21.15.63:6789/0 conn(0x7fa1c4071110 legacy=0x7fa1c406d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:08.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.861+0000 7fa1c9a6c640 1 -- 172.21.15.45:0/1218516171 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c40708e0 con 0x7fa1c4163920 2024-05-08T05:42:08.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.861+0000 7fa1c9a6c640 1 --2- 172.21.15.45:0/1218516171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1c4163920 0x7fa1c40703a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009870 tx=0x7fa1b402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4ffa199a1c18e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:08.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.862+0000 7fa1c926b640 1 -- 172.21.15.45:0/1218516171 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4037070 con 0x7fa1c4163920 2024-05-08T05:42:08.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.862+0000 7fa1c926b640 1 -- 172.21.15.45:0/1218516171 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1b4032440 con 0x7fa1c4163920 2024-05-08T05:42:08.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.862+0000 7fa1c926b640 1 -- 172.21.15.45:0/1218516171 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4032810 con 0x7fa1c4163920 2024-05-08T05:42:08.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.862+0000 7fa1cc4d1640 1 -- 172.21.15.45:0/1218516171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1c4163920 msgr2=0x7fa1c40703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:08.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.862+0000 7fa1cc4d1640 1 --2- 172.21.15.45:0/1218516171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1c4163920 0x7fa1c40703a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4009870 tx=0x7fa1b402a1c0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.862+0000 7fa1cc4d1640 1 -- 172.21.15.45:0/1218516171 shutdown_connections 2024-05-08T05:42:08.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.862+0000 7fa1cc4d1640 1 --2- 172.21.15.45:0/1218516171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1c4163920 0x7fa1c40703a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.862+0000 7fa1cc4d1640 1 --2- 172.21.15.45:0/1218516171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1c4070d30 0x7fa1c40d20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.862+0000 7fa1cc4d1640 1 -- 172.21.15.45:0/1218516171 >> 172.21.15.45:0/1218516171 conn(0x7fa1c40d2e90 msgr2=0x7fa1c40d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:08.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.862+0000 7fa1cc4d1640 1 -- 172.21.15.45:0/1218516171 shutdown_connections 2024-05-08T05:42:08.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.862+0000 7fa1cc4d1640 1 -- 172.21.15.45:0/1218516171 wait complete. 2024-05-08T05:42:08.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.863+0000 7fa1cc4d1640 1 Processor -- start 2024-05-08T05:42:08.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.864+0000 7fa1cc4d1640 1 -- start start 2024-05-08T05:42:08.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.864+0000 7fa1cc4d1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1c4070d30 0x7fa1c415e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:08.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.864+0000 7fa1cc4d1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1c4071110 0x7fa1c41574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:08.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.864+0000 7fa1cc4d1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1c4163920 0x7fa1c40cf460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:08.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.864+0000 7fa1cc4d1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c40d15e0 con 0x7fa1c4070d30 2024-05-08T05:42:08.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.864+0000 7fa1cc4d1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c40d1c30 con 0x7fa1c4071110 2024-05-08T05:42:08.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.864+0000 7fa1cc4d1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c40d18c0 con 0x7fa1c4163920 2024-05-08T05:42:08.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.864+0000 7fa1ca26d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1c4070d30 0x7fa1c415e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:08.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.864+0000 7fa1ca26d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1c4070d30 0x7fa1c415e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49204/0 (socket says 172.21.15.45:49204) 2024-05-08T05:42:08.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.864+0000 7fa1ca26d640 1 -- 172.21.15.45:0/2081171423 learned_addr learned my addr 172.21.15.45:0/2081171423 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:08.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.865+0000 7fa1caa6e640 1 --2- 172.21.15.45:0/2081171423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1c4163920 0x7fa1c40cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:08.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.865+0000 7fa1c9a6c640 1 --2- 172.21.15.45:0/2081171423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1c4071110 0x7fa1c41574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:08.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.865+0000 7fa1ca26d640 1 -- 172.21.15.45:0/2081171423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1c4163920 msgr2=0x7fa1c40cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:08.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.865+0000 7fa1ca26d640 1 --2- 172.21.15.45:0/2081171423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1c4163920 0x7fa1c40cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.865+0000 7fa1ca26d640 1 -- 172.21.15.45:0/2081171423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1c4071110 msgr2=0x7fa1c41574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:08.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.865+0000 7fa1ca26d640 1 --2- 172.21.15.45:0/2081171423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1c4071110 0x7fa1c41574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:08.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.865+0000 7fa1ca26d640 1 -- 172.21.15.45:0/2081171423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c40ce2b0 con 0x7fa1c4070d30 2024-05-08T05:42:08.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.865+0000 7fa1ca26d640 1 --2- 172.21.15.45:0/2081171423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1c4070d30 0x7fa1c415e4b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0007e40 tx=0x7fa1c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:08.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.865+0000 7fa1baffd640 1 -- 172.21.15.45:0/2081171423 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0014620 con 0x7fa1c4070d30 2024-05-08T05:42:08.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.865+0000 7fa1baffd640 1 -- 172.21.15.45:0/2081171423 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c0019070 con 0x7fa1c4070d30 2024-05-08T05:42:08.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.865+0000 7fa1cc4d1640 1 -- 172.21.15.45:0/2081171423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c40ce590 con 0x7fa1c4070d30 2024-05-08T05:42:08.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.865+0000 7fa1baffd640 1 -- 172.21.15.45:0/2081171423 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0014a00 con 0x7fa1c4070d30 2024-05-08T05:42:08.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.865+0000 7fa1cc4d1640 1 -- 172.21.15.45:0/2081171423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c40b3c40 con 0x7fa1c4070d30 2024-05-08T05:42:08.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.866+0000 7fa1cc4d1640 1 -- 172.21.15.45:0/2081171423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c406fc30 con 0x7fa1c4070d30 2024-05-08T05:42:08.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.867+0000 7fa1baffd640 1 -- 172.21.15.45:0/2081171423 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa1c0014ba0 con 0x7fa1c4070d30 2024-05-08T05:42:08.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.867+0000 7fa1baffd640 1 --2- 172.21.15.45:0/2081171423 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1a80762d0 0x7fa1a8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:08.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.867+0000 7fa1baffd640 1 -- 172.21.15.45:0/2081171423 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fa1c0098190 con 0x7fa1c4070d30 2024-05-08T05:42:08.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.867+0000 7fa1c9a6c640 1 --2- 172.21.15.45:0/2081171423 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1a80762d0 0x7fa1a8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:08.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.868+0000 7fa1c9a6c640 1 --2- 172.21.15.45:0/2081171423 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1a80762d0 0x7fa1a8078790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4034000 tx=0x7fa1b402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:08.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:08.872+0000 7fa1baffd640 1 -- 172.21.15.45:0/2081171423 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa1c406fc30 con 0x7fa1c4070d30 2024-05-08T05:42:09.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.066+0000 7fa1cc4d1640 1 -- 172.21.15.45:0/2081171423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1c40693f0 con 0x7fa1c4070d30 2024-05-08T05:42:09.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.067+0000 7fa1baffd640 1 -- 172.21.15.45:0/2081171423 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18466 (secure 0 0 0) 0x7fa1c00617b0 con 0x7fa1c4070d30 2024-05-08T05:42:09.067 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:09.068 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":24,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:41:36.532045+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:41:29.381274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:41:30.443612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:41:34.132285+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:42:09.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.069+0000 7fa1cc4d1640 1 -- 172.21.15.45:0/2081171423 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1a80762d0 msgr2=0x7fa1a8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:09.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.069+0000 7fa1cc4d1640 1 --2- 172.21.15.45:0/2081171423 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1a80762d0 0x7fa1a8078790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4034000 tx=0x7fa1b402e000 comp rx=0 tx=0).stop 2024-05-08T05:42:09.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.069+0000 7fa1cc4d1640 1 -- 172.21.15.45:0/2081171423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1c4070d30 msgr2=0x7fa1c415e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:09.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.069+0000 7fa1cc4d1640 1 --2- 172.21.15.45:0/2081171423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1c4070d30 0x7fa1c415e4b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0007e40 tx=0x7fa1c000e4b0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.069+0000 7fa1cc4d1640 1 -- 172.21.15.45:0/2081171423 shutdown_connections 2024-05-08T05:42:09.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.069+0000 7fa1cc4d1640 1 --2- 172.21.15.45:0/2081171423 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1a80762d0 0x7fa1a8078790 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.069+0000 7fa1cc4d1640 1 --2- 172.21.15.45:0/2081171423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1c4163920 0x7fa1c40cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.069+0000 7fa1cc4d1640 1 --2- 172.21.15.45:0/2081171423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1c4071110 0x7fa1c41574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.069+0000 7fa1cc4d1640 1 --2- 172.21.15.45:0/2081171423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1c4070d30 0x7fa1c415e4b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.069+0000 7fa1cc4d1640 1 -- 172.21.15.45:0/2081171423 >> 172.21.15.45:0/2081171423 conn(0x7fa1c40d2e90 msgr2=0x7fa1c40b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:09.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.069+0000 7fa1cc4d1640 1 -- 172.21.15.45:0/2081171423 shutdown_connections 2024-05-08T05:42:09.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.069+0000 7fa1cc4d1640 1 -- 172.21.15.45:0/2081171423 wait complete. 2024-05-08T05:42:09.081 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T05:42:09.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.185+0000 7f805fe1c640 1 Processor -- start 2024-05-08T05:42:09.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.186+0000 7f805fe1c640 1 -- start start 2024-05-08T05:42:09.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.186+0000 7f805fe1c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f805815f420 0x7f80580d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:09.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.186+0000 7f805fe1c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80580d3ca0 0x7f80580d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:09.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.186+0000 7f805fe1c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f80580d1eb0 0x7f80580d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:09.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.186+0000 7f805fe1c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f80580d22f0 con 0x7f80580d1eb0 2024-05-08T05:42:09.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.186+0000 7f805fe1c640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f80581630b0 con 0x7f805815f420 2024-05-08T05:42:09.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.186+0000 7f805fe1c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f80580d2550 con 0x7f80580d3ca0 2024-05-08T05:42:09.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.187+0000 7f805e3b9640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f80580d1eb0 0x7f80580d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:09.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.187+0000 7f805e3b9640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f80580d1eb0 0x7f80580d4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49208/0 (socket says 172.21.15.45:49208) 2024-05-08T05:42:09.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.187+0000 7f805e3b9640 1 -- 172.21.15.45:0/1869151867 learned_addr learned my addr 172.21.15.45:0/1869151867 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:09.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.187+0000 7f805dbb8640 1 --2- 172.21.15.45:0/1869151867 >> v2:172.21.15.63:3300/0 conn(0x7f805815f420 0x7f80580d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:09.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.187+0000 7f805d3b7640 1 --2- 172.21.15.45:0/1869151867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80580d3ca0 0x7f80580d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:09.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.187+0000 7f805e3b9640 1 -- 172.21.15.45:0/1869151867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80580d3ca0 msgr2=0x7f80580d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:09.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.187+0000 7f805e3b9640 1 --2- 172.21.15.45:0/1869151867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80580d3ca0 0x7f80580d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.187+0000 7f805e3b9640 1 -- 172.21.15.45:0/1869151867 >> v2:172.21.15.63:3300/0 conn(0x7f805815f420 msgr2=0x7f80580d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:09.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.187+0000 7f805e3b9640 1 --2- 172.21.15.45:0/1869151867 >> v2:172.21.15.63:3300/0 conn(0x7f805815f420 0x7f80580d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.187+0000 7f805e3b9640 1 -- 172.21.15.45:0/1869151867 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80580d2c80 con 0x7f80580d1eb0 2024-05-08T05:42:09.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.187+0000 7f805d3b7640 1 --2- 172.21.15.45:0/1869151867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80580d3ca0 0x7f80580d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:09.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.188+0000 7f805e3b9640 1 --2- 172.21.15.45:0/1869151867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80580d1eb0 0x7f80580d4cc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f80580d5ec0 tx=0x7f805002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fff8ffc66c0ed0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:09.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.188+0000 7f805cbb6640 1 -- 172.21.15.45:0/1869151867 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8050036070 con 0x7f80580d1eb0 2024-05-08T05:42:09.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.188+0000 7f805cbb6640 1 -- 172.21.15.45:0/1869151867 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8050031600 con 0x7f80580d1eb0 2024-05-08T05:42:09.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.188+0000 7f805cbb6640 1 -- 172.21.15.45:0/1869151867 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80500319d0 con 0x7f80580d1eb0 2024-05-08T05:42:09.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.188+0000 7f805fe1c640 1 -- 172.21.15.45:0/1869151867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80580d1eb0 msgr2=0x7f80580d4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:09.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.188+0000 7f805fe1c640 1 --2- 172.21.15.45:0/1869151867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80580d1eb0 0x7f80580d4cc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f80580d5ec0 tx=0x7f805002a6a0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.188+0000 7f805fe1c640 1 -- 172.21.15.45:0/1869151867 shutdown_connections 2024-05-08T05:42:09.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.188+0000 7f805fe1c640 1 --2- 172.21.15.45:0/1869151867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80580d1eb0 0x7f80580d4cc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.188+0000 7f805fe1c640 1 --2- 172.21.15.45:0/1869151867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80580d3ca0 0x7f80580d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.188+0000 7f805fe1c640 1 --2- 172.21.15.45:0/1869151867 >> v2:172.21.15.63:3300/0 conn(0x7f805815f420 0x7f80580d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.188+0000 7f805fe1c640 1 -- 172.21.15.45:0/1869151867 >> 172.21.15.45:0/1869151867 conn(0x7f805808eac0 msgr2=0x7f805809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:09.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.188+0000 7f805fe1c640 1 -- 172.21.15.45:0/1869151867 shutdown_connections 2024-05-08T05:42:09.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.188+0000 7f805fe1c640 1 -- 172.21.15.45:0/1869151867 wait complete. 2024-05-08T05:42:09.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.190+0000 7f805fe1c640 1 Processor -- start 2024-05-08T05:42:09.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.190+0000 7f805fe1c640 1 -- start start 2024-05-08T05:42:09.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.190+0000 7f805fe1c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80580d1eb0 0x7f80580c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:09.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.190+0000 7f805fe1c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80580cb690 0x7f8058162200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:09.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.190+0000 7f805fe1c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8058150330 0x7f805815ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:09.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.190+0000 7f805fe1c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f80580d2830 con 0x7f80580cb690 2024-05-08T05:42:09.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.190+0000 7f805fe1c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f80580d3240 con 0x7f8058150330 2024-05-08T05:42:09.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.190+0000 7f805fe1c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f80580d5200 con 0x7f80580d1eb0 2024-05-08T05:42:09.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.190+0000 7f805d3b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80580cb690 0x7f8058162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:09.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.190+0000 7f805d3b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80580cb690 0x7f8058162200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49220/0 (socket says 172.21.15.45:49220) 2024-05-08T05:42:09.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.190+0000 7f805d3b7640 1 -- 172.21.15.45:0/873479619 learned_addr learned my addr 172.21.15.45:0/873479619 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:09.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.191+0000 7f805e3b9640 1 --2- 172.21.15.45:0/873479619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8058150330 0x7f805815ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:09.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.191+0000 7f805dbb8640 1 --2- 172.21.15.45:0/873479619 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80580d1eb0 0x7f80580c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:09.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.191+0000 7f805d3b7640 1 -- 172.21.15.45:0/873479619 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80580d1eb0 msgr2=0x7f80580c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.191+0000 7f805d3b7640 1 --2- 172.21.15.45:0/873479619 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80580d1eb0 0x7f80580c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.191+0000 7f805d3b7640 1 -- 172.21.15.45:0/873479619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8058150330 msgr2=0x7f805815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.191+0000 7f805d3b7640 1 --2- 172.21.15.45:0/873479619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8058150330 0x7f805815ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.191+0000 7f805d3b7640 1 -- 172.21.15.45:0/873479619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8058162740 con 0x7f80580cb690 2024-05-08T05:42:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.191+0000 7f805d3b7640 1 --2- 172.21.15.45:0/873479619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80580cb690 0x7f8058162200 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f804800e840 tx=0x7f804800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.191+0000 7f804e7fc640 1 -- 172.21.15.45:0/873479619 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804800f040 con 0x7f80580cb690 2024-05-08T05:42:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.191+0000 7f804e7fc640 1 -- 172.21.15.45:0/873479619 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8048004030 con 0x7f80580cb690 2024-05-08T05:42:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.191+0000 7f805fe1c640 1 -- 172.21.15.45:0/873479619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80580ca5f0 con 0x7f80580cb690 2024-05-08T05:42:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.191+0000 7f804e7fc640 1 -- 172.21.15.45:0/873479619 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8048004400 con 0x7f80580cb690 2024-05-08T05:42:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.191+0000 7f805fe1c640 1 -- 172.21.15.45:0/873479619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80580cabb0 con 0x7f80580cb690 2024-05-08T05:42:09.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.192+0000 7f805fe1c640 1 -- 172.21.15.45:0/873479619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80580693f0 con 0x7f80580cb690 2024-05-08T05:42:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.193+0000 7f804e7fc640 1 -- 172.21.15.45:0/873479619 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f804800c6d0 con 0x7f80580cb690 2024-05-08T05:42:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.193+0000 7f804e7fc640 1 --2- 172.21.15.45:0/873479619 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f803c0762d0 0x7f803c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.193+0000 7f804e7fc640 1 -- 172.21.15.45:0/873479619 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f8048097540 con 0x7f80580cb690 2024-05-08T05:42:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.194+0000 7f805dbb8640 1 --2- 172.21.15.45:0/873479619 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f803c0762d0 0x7f803c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.194+0000 7f805dbb8640 1 --2- 172.21.15.45:0/873479619 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f803c0762d0 0x7f803c078790 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8054009a30 tx=0x7f8054008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:09.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.198+0000 7f804e7fc640 1 -- 172.21.15.45:0/873479619 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8048017020 con 0x7f80580cb690 2024-05-08T05:42:09.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.432+0000 7f805fe1c640 1 -- 172.21.15.45:0/873479619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f80580d1ab0 con 0x7f80580cb690 2024-05-08T05:42:09.433 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-05-08T05:42:09.432+0000 7f124de12640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T05:42:09.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.434+0000 7f804e7fc640 1 -- 172.21.15.45:0/873479619 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f8048060bf0 con 0x7f80580cb690 2024-05-08T05:42:09.434 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T05:42:09.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.437+0000 7f805fe1c640 1 -- 172.21.15.45:0/873479619 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f803c0762d0 msgr2=0x7f803c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:09.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.437+0000 7f805fe1c640 1 --2- 172.21.15.45:0/873479619 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f803c0762d0 0x7f803c078790 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8054009a30 tx=0x7f8054008040 comp rx=0 tx=0).stop 2024-05-08T05:42:09.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.437+0000 7f805fe1c640 1 -- 172.21.15.45:0/873479619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80580cb690 msgr2=0x7f8058162200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:09.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.437+0000 7f805fe1c640 1 --2- 172.21.15.45:0/873479619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80580cb690 0x7f8058162200 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f804800e840 tx=0x7f804800ed00 comp rx=0 tx=0).stop 2024-05-08T05:42:09.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.437+0000 7f805fe1c640 1 -- 172.21.15.45:0/873479619 shutdown_connections 2024-05-08T05:42:09.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.437+0000 7f805fe1c640 1 --2- 172.21.15.45:0/873479619 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f803c0762d0 0x7f803c078790 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.437+0000 7f805fe1c640 1 --2- 172.21.15.45:0/873479619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8058150330 0x7f805815ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.437+0000 7f805fe1c640 1 --2- 172.21.15.45:0/873479619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80580cb690 0x7f8058162200 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.437+0000 7f805fe1c640 1 --2- 172.21.15.45:0/873479619 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80580d1eb0 0x7f80580c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.437+0000 7f805fe1c640 1 -- 172.21.15.45:0/873479619 >> 172.21.15.45:0/873479619 conn(0x7f805808eac0 msgr2=0x7f805806e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:09.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.437+0000 7f805fe1c640 1 -- 172.21.15.45:0/873479619 shutdown_connections 2024-05-08T05:42:09.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.437+0000 7f805fe1c640 1 -- 172.21.15.45:0/873479619 wait complete. 2024-05-08T05:42:09.439 INFO:tasks.ceph.mds.d.smithi063.stderr: -5350> 2024-05-08T05:41:26.095+0000 7f6a4e57d640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-05-08T05:42:09.441 INFO:tasks.ceph.mds.d.smithi063.stderr: -4083> 2024-05-08T05:41:37.092+0000 7f6a48571640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:42:09.448 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T05:42:09.474 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-08T05:42:09.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.554+0000 7fe3823d0640 1 Processor -- start 2024-05-08T05:42:09.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.554+0000 7fe3823d0640 1 -- start start 2024-05-08T05:42:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.555+0000 7fe3823d0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe37c1629a0 0x7fe37c0d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.555+0000 7fe3823d0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe37c0d2dc0 0x7fe37c0d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:09.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.555+0000 7fe3823d0640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe37c0d3330 con 0x7fe37c0c8ed0 2024-05-08T05:42:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.555+0000 7fe3823d0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe37c0d2170 con 0x7fe37c1629a0 2024-05-08T05:42:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.555+0000 7fe3823d0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe37c0d2450 con 0x7fe37c0d2dc0 2024-05-08T05:42:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.555+0000 7fe37bfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe37c1629a0 0x7fe37c0d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.555+0000 7fe37bfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe37c1629a0 0x7fe37c0d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49232/0 (socket says 172.21.15.45:49232) 2024-05-08T05:42:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.555+0000 7fe37bfff640 1 -- 172.21.15.45:0/1556406100 learned_addr learned my addr 172.21.15.45:0/1556406100 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.555+0000 7fe37b7fe640 1 --2- 172.21.15.45:0/1556406100 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe37c0d2dc0 0x7fe37c0d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.555+0000 7fe37bfff640 1 -- 172.21.15.45:0/1556406100 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe37c0d2dc0 msgr2=0x7fe37c0d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.555+0000 7fe37bfff640 1 --2- 172.21.15.45:0/1556406100 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe37c0d2dc0 0x7fe37c0d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.555+0000 7fe37bfff640 1 -- 172.21.15.45:0/1556406100 >> v1:172.21.15.63:6789/0 conn(0x7fe37c0c8ed0 legacy=0x7fe37c0c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.555+0000 7fe37bfff640 1 -- 172.21.15.45:0/1556406100 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c0d2730 con 0x7fe37c1629a0 2024-05-08T05:42:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.556+0000 7fe37bfff640 1 --2- 172.21.15.45:0/1556406100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c1629a0 0x7fe37c0d35c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009870 tx=0x7fe36c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c99deec21a53b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.556+0000 7fe37affd640 1 -- 172.21.15.45:0/1556406100 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c037070 con 0x7fe37c1629a0 2024-05-08T05:42:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.556+0000 7fe37affd640 1 -- 172.21.15.45:0/1556406100 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe36c032440 con 0x7fe37c1629a0 2024-05-08T05:42:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.556+0000 7fe37affd640 1 -- 172.21.15.45:0/1556406100 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c032810 con 0x7fe37c1629a0 2024-05-08T05:42:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.557+0000 7fe3823d0640 1 -- 172.21.15.45:0/1556406100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c1629a0 msgr2=0x7fe37c0d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.557+0000 7fe3823d0640 1 --2- 172.21.15.45:0/1556406100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c1629a0 0x7fe37c0d35c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009870 tx=0x7fe36c02a130 comp rx=0 tx=0).stop 2024-05-08T05:42:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.557+0000 7fe3823d0640 1 -- 172.21.15.45:0/1556406100 shutdown_connections 2024-05-08T05:42:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.557+0000 7fe3823d0640 1 --2- 172.21.15.45:0/1556406100 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe37c0d2dc0 0x7fe37c0d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.557+0000 7fe3823d0640 1 --2- 172.21.15.45:0/1556406100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c1629a0 0x7fe37c0d35c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.557+0000 7fe3823d0640 1 -- 172.21.15.45:0/1556406100 >> 172.21.15.45:0/1556406100 conn(0x7fe37c0d4fe0 msgr2=0x7fe37c0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:09.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.557+0000 7fe3823d0640 1 -- 172.21.15.45:0/1556406100 shutdown_connections 2024-05-08T05:42:09.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.557+0000 7fe3823d0640 1 -- 172.21.15.45:0/1556406100 wait complete. 2024-05-08T05:42:09.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.558+0000 7fe3823d0640 1 Processor -- start 2024-05-08T05:42:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.558+0000 7fe3823d0640 1 -- start start 2024-05-08T05:42:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.559+0000 7fe3823d0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c0c8ed0 0x7fe37c152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.559+0000 7fe3823d0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe37c0d2dc0 0x7fe37c152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.559+0000 7fe3823d0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37c1629a0 0x7fe37c1508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.559+0000 7fe3823d0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c0cf440 con 0x7fe37c0c8ed0 2024-05-08T05:42:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.559+0000 7fe3823d0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c0cf2c0 con 0x7fe37c1629a0 2024-05-08T05:42:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.559+0000 7fe3823d0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe37c0d0cb0 con 0x7fe37c0d2dc0 2024-05-08T05:42:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.559+0000 7fe37bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c0c8ed0 0x7fe37c152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:09.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.559+0000 7fe37bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c0c8ed0 0x7fe37c152340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49242/0 (socket says 172.21.15.45:49242) 2024-05-08T05:42:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.559+0000 7fe37bfff640 1 -- 172.21.15.45:0/312683207 learned_addr learned my addr 172.21.15.45:0/312683207 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.560+0000 7fe37bfff640 1 -- 172.21.15.45:0/312683207 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe37c0d2dc0 msgr2=0x7fe37c152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.560+0000 7fe37bfff640 1 --2- 172.21.15.45:0/312683207 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe37c0d2dc0 0x7fe37c152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.560+0000 7fe37bfff640 1 -- 172.21.15.45:0/312683207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37c1629a0 msgr2=0x7fe37c1508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.560+0000 7fe37bfff640 1 --2- 172.21.15.45:0/312683207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37c1629a0 0x7fe37c1508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.560+0000 7fe37bfff640 1 -- 172.21.15.45:0/312683207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c1512a0 con 0x7fe37c0c8ed0 2024-05-08T05:42:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.560+0000 7fe37bfff640 1 --2- 172.21.15.45:0/312683207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c0c8ed0 0x7fe37c152340 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe36c000c00 tx=0x7fe36c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.560+0000 7fe378ff9640 1 -- 172.21.15.45:0/312683207 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c041020 con 0x7fe37c0c8ed0 2024-05-08T05:42:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.560+0000 7fe378ff9640 1 -- 172.21.15.45:0/312683207 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe36c0306c0 con 0x7fe37c0c8ed0 2024-05-08T05:42:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.560+0000 7fe378ff9640 1 -- 172.21.15.45:0/312683207 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c037040 con 0x7fe37c0c8ed0 2024-05-08T05:42:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.560+0000 7fe3823d0640 1 -- 172.21.15.45:0/312683207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c151580 con 0x7fe37c0c8ed0 2024-05-08T05:42:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.560+0000 7fe3823d0640 1 -- 172.21.15.45:0/312683207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c151b40 con 0x7fe37c0c8ed0 2024-05-08T05:42:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.561+0000 7fe3823d0640 1 -- 172.21.15.45:0/312683207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe37c0693f0 con 0x7fe37c0c8ed0 2024-05-08T05:42:09.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.562+0000 7fe378ff9640 1 -- 172.21.15.45:0/312683207 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe36c044020 con 0x7fe37c0c8ed0 2024-05-08T05:42:09.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.562+0000 7fe378ff9640 1 --2- 172.21.15.45:0/312683207 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe3680762d0 0x7fe368078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:09.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.562+0000 7fe378ff9640 1 -- 172.21.15.45:0/312683207 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6618+0+0 (secure 0 0 0) 0x7fe36c0b8140 con 0x7fe37c0c8ed0 2024-05-08T05:42:09.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.562+0000 7fe37b7fe640 1 --2- 172.21.15.45:0/312683207 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe3680762d0 0x7fe368078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:09.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.563+0000 7fe37b7fe640 1 --2- 172.21.15.45:0/312683207 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe3680762d0 0x7fe368078790 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0baa70 tx=0x7fe370008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:09.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.567+0000 7fe378ff9640 1 -- 172.21.15.45:0/312683207 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe36c04b030 con 0x7fe37c0c8ed0 2024-05-08T05:42:09.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:09.798+0000 7fe3823d0640 1 -- 172.21.15.45:0/312683207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe37c0d0870 con 0x7fe37c0c8ed0 2024-05-08T05:42:10.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.435+0000 7fe378ff9640 1 -- 172.21.15.45:0/312683207 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7fe36c0817a0 con 0x7fe37c0c8ed0 2024-05-08T05:42:10.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.438+0000 7fe3823d0640 1 -- 172.21.15.45:0/312683207 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe3680762d0 msgr2=0x7fe368078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.438+0000 7fe3823d0640 1 --2- 172.21.15.45:0/312683207 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe3680762d0 0x7fe368078790 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0baa70 tx=0x7fe370008040 comp rx=0 tx=0).stop 2024-05-08T05:42:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.438+0000 7fe3823d0640 1 -- 172.21.15.45:0/312683207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c0c8ed0 msgr2=0x7fe37c152340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.438+0000 7fe3823d0640 1 --2- 172.21.15.45:0/312683207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c0c8ed0 0x7fe37c152340 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe36c000c00 tx=0x7fe36c0046c0 comp rx=0 tx=0).stop 2024-05-08T05:42:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.438+0000 7fe3823d0640 1 -- 172.21.15.45:0/312683207 shutdown_connections 2024-05-08T05:42:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.438+0000 7fe3823d0640 1 --2- 172.21.15.45:0/312683207 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe3680762d0 0x7fe368078790 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.438+0000 7fe3823d0640 1 --2- 172.21.15.45:0/312683207 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe37c0d2dc0 0x7fe37c152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.438+0000 7fe3823d0640 1 --2- 172.21.15.45:0/312683207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe37c1629a0 0x7fe37c1508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.438+0000 7fe3823d0640 1 --2- 172.21.15.45:0/312683207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c0c8ed0 0x7fe37c152340 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.438+0000 7fe3823d0640 1 -- 172.21.15.45:0/312683207 >> 172.21.15.45:0/312683207 conn(0x7fe37c0d4fe0 msgr2=0x7fe37c0ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:10.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.438+0000 7fe3823d0640 1 -- 172.21.15.45:0/312683207 shutdown_connections 2024-05-08T05:42:10.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.438+0000 7fe3823d0640 1 -- 172.21.15.45:0/312683207 wait complete. 2024-05-08T05:42:10.452 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T05:42:10.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.565+0000 7f5bfcbc1640 1 Processor -- start 2024-05-08T05:42:10.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.565+0000 7f5bfcbc1640 1 -- start start 2024-05-08T05:42:10.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.566+0000 7f5bfcbc1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5bf80d3670 0x7f5bf80b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:10.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.566+0000 7f5bfcbc1640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5bf80d3a50 0x7f5bf80d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:10.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.566+0000 7f5bfcbc1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5bf80d0e80 0x7f5bf80d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:10.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.566+0000 7f5bfcbc1640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f5bf80d2a10 con 0x7f5bf80d3670 2024-05-08T05:42:10.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.566+0000 7f5bfcbc1640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5bf80d04b0 con 0x7f5bf80d3a50 2024-05-08T05:42:10.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.566+0000 7f5bfcbc1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf80cfc00 con 0x7f5bf80d0e80 2024-05-08T05:42:10.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.566+0000 7f5bf659c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5bf80d3670 0x7f5bf80b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:10.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.566+0000 7f5bf659c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5bf80d3670 0x7f5bf80b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49248/0 (socket says 172.21.15.45:49248) 2024-05-08T05:42:10.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.566+0000 7f5bf659c640 1 -- 172.21.15.45:0/2116831038 learned_addr learned my addr 172.21.15.45:0/2116831038 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:10.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.566+0000 7f5bf6d9d640 1 --2- 172.21.15.45:0/2116831038 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5bf80d0e80 0x7f5bf80d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:10.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.566+0000 7f5bedd9b640 1 --2- 172.21.15.45:0/2116831038 >> v2:172.21.15.63:3300/0 conn(0x7f5bf80d3a50 0x7f5bf80d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:10.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.566+0000 7f5bf659c640 1 -- 172.21.15.45:0/2116831038 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5bf80d0e80 msgr2=0x7f5bf80d19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:10.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.566+0000 7f5bf659c640 1 --2- 172.21.15.45:0/2116831038 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5bf80d0e80 0x7f5bf80d19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:10.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.566+0000 7f5bf659c640 1 -- 172.21.15.45:0/2116831038 >> v2:172.21.15.63:3300/0 conn(0x7f5bf80d3a50 msgr2=0x7f5bf80d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:10.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.566+0000 7f5bf659c640 1 --2- 172.21.15.45:0/2116831038 >> v2:172.21.15.63:3300/0 conn(0x7f5bf80d3a50 0x7f5bf80d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:10.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.566+0000 7f5bf659c640 1 -- 172.21.15.45:0/2116831038 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf80cfea0 con 0x7f5bf80d3670 2024-05-08T05:42:10.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.567+0000 7f5bf659c640 1 --2- 172.21.15.45:0/2116831038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf80d3670 0x7f5bf80b49c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5bf808f550 tx=0x7f5be802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd7bc7a9bd08054 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:10.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.567+0000 7f5bf5d9b640 1 -- 172.21.15.45:0/2116831038 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8037070 con 0x7f5bf80d3670 2024-05-08T05:42:10.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.567+0000 7f5bf5d9b640 1 -- 172.21.15.45:0/2116831038 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be8032440 con 0x7f5bf80d3670 2024-05-08T05:42:10.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.567+0000 7f5bf5d9b640 1 -- 172.21.15.45:0/2116831038 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8032810 con 0x7f5bf80d3670 2024-05-08T05:42:10.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.568+0000 7f5bfcbc1640 1 -- 172.21.15.45:0/2116831038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf80d3670 msgr2=0x7f5bf80b49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:10.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.568+0000 7f5bfcbc1640 1 --2- 172.21.15.45:0/2116831038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf80d3670 0x7f5bf80b49c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5bf808f550 tx=0x7f5be802a1e0 comp rx=0 tx=0).stop 2024-05-08T05:42:10.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.568+0000 7f5bfcbc1640 1 -- 172.21.15.45:0/2116831038 shutdown_connections 2024-05-08T05:42:10.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.568+0000 7f5bfcbc1640 1 --2- 172.21.15.45:0/2116831038 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5bf80d0e80 0x7f5bf80d19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:10.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.568+0000 7f5bfcbc1640 1 --2- 172.21.15.45:0/2116831038 >> v2:172.21.15.63:3300/0 conn(0x7f5bf80d3a50 0x7f5bf80d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:10.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.568+0000 7f5bfcbc1640 1 --2- 172.21.15.45:0/2116831038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf80d3670 0x7f5bf80b49c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:10.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.568+0000 7f5bfcbc1640 1 -- 172.21.15.45:0/2116831038 >> 172.21.15.45:0/2116831038 conn(0x7f5bf809e390 msgr2=0x7f5bf80d5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:10.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.568+0000 7f5bfcbc1640 1 -- 172.21.15.45:0/2116831038 shutdown_connections 2024-05-08T05:42:10.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.568+0000 7f5bfcbc1640 1 -- 172.21.15.45:0/2116831038 wait complete. 2024-05-08T05:42:10.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.569+0000 7f5bfcbc1640 1 Processor -- start 2024-05-08T05:42:10.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.569+0000 7f5bfcbc1640 1 -- start start 2024-05-08T05:42:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.570+0000 7f5bfcbc1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bf80d0e80 0x7f5bf80ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.570+0000 7f5bfcbc1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5bf80d3670 0x7f5bf80b7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.570+0000 7f5bfcbc1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf80d3a50 0x7f5bf8158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.570+0000 7f5bfcbc1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf80d0140 con 0x7f5bf80d3a50 2024-05-08T05:42:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.570+0000 7f5bfcbc1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf80d2730 con 0x7f5bf80d0e80 2024-05-08T05:42:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.570+0000 7f5bfcbc1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf80b4f00 con 0x7f5bf80d3670 2024-05-08T05:42:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.570+0000 7f5bf6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf80d3a50 0x7f5bf8158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.570+0000 7f5bf6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf80d3a50 0x7f5bf8158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49254/0 (socket says 172.21.15.45:49254) 2024-05-08T05:42:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.570+0000 7f5bf6d9d640 1 -- 172.21.15.45:0/2841453713 learned_addr learned my addr 172.21.15.45:0/2841453713 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.570+0000 7f5bf659c640 1 --2- 172.21.15.45:0/2841453713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bf80d0e80 0x7f5bf80ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.570+0000 7f5bedd9b640 1 --2- 172.21.15.45:0/2841453713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5bf80d3670 0x7f5bf80b7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:10.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.570+0000 7f5bf6d9d640 1 -- 172.21.15.45:0/2841453713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5bf80d3670 msgr2=0x7f5bf80b7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.570+0000 7f5bf6d9d640 1 --2- 172.21.15.45:0/2841453713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5bf80d3670 0x7f5bf80b7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.570+0000 7f5bf6d9d640 1 -- 172.21.15.45:0/2841453713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bf80d0e80 msgr2=0x7f5bf80ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.570+0000 7f5bf6d9d640 1 --2- 172.21.15.45:0/2841453713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bf80d0e80 0x7f5bf80ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.570+0000 7f5bf6d9d640 1 -- 172.21.15.45:0/2841453713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf80ba9b0 con 0x7f5bf80d3a50 2024-05-08T05:42:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.571+0000 7f5bedd9b640 1 --2- 172.21.15.45:0/2841453713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5bf80d3670 0x7f5bf80b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.571+0000 7f5bf6d9d640 1 --2- 172.21.15.45:0/2841453713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf80d3a50 0x7f5bf8158ca0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000ef00 tx=0x7f5bf000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.571+0000 7f5bef7fe640 1 -- 172.21.15.45:0/2841453713 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0018020 con 0x7f5bf80d3a50 2024-05-08T05:42:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.571+0000 7f5bef7fe640 1 -- 172.21.15.45:0/2841453713 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf0014400 con 0x7f5bf80d3a50 2024-05-08T05:42:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.571+0000 7f5bfcbc1640 1 -- 172.21.15.45:0/2841453713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf80cdfd0 con 0x7f5bf80d3a50 2024-05-08T05:42:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.571+0000 7f5bef7fe640 1 -- 172.21.15.45:0/2841453713 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf00147f0 con 0x7f5bf80d3a50 2024-05-08T05:42:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.571+0000 7f5bfcbc1640 1 -- 172.21.15.45:0/2841453713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf81591e0 con 0x7f5bf80d3a50 2024-05-08T05:42:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.572+0000 7f5bfcbc1640 1 -- 172.21.15.45:0/2841453713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf80693f0 con 0x7f5bf80d3a50 2024-05-08T05:42:10.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.573+0000 7f5bef7fe640 1 -- 172.21.15.45:0/2841453713 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5bf0005d10 con 0x7f5bf80d3a50 2024-05-08T05:42:10.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.573+0000 7f5bef7fe640 1 --2- 172.21.15.45:0/2841453713 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5bd80762d0 0x7f5bd8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:10.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.573+0000 7f5bef7fe640 1 -- 172.21.15.45:0/2841453713 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6618+0+0 (secure 0 0 0) 0x7f5bf0098020 con 0x7f5bf80d3a50 2024-05-08T05:42:10.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.573+0000 7f5bf659c640 1 --2- 172.21.15.45:0/2841453713 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5bd80762d0 0x7f5bd8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:10.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.574+0000 7f5bf659c640 1 --2- 172.21.15.45:0/2841453713 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5bd80762d0 0x7f5bd8078790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5be8034000 tx=0x7f5be802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:10.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.577+0000 7f5bef7fe640 1 -- 172.21.15.45:0/2841453713 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5bf0022070 con 0x7f5bf80d3a50 2024-05-08T05:42:10.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:10.771+0000 7f5bfcbc1640 1 -- 172.21.15.45:0/2841453713 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5bf806e320 con 0x7f5bf80d3a50 2024-05-08T05:42:11.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.443+0000 7f5bef7fe640 1 -- 172.21.15.45:0/2841453713 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) ==== 181+0+0 (secure 0 0 0) 0x7f5bf0061570 con 0x7f5bf80d3a50 2024-05-08T05:42:11.443 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-05-08T05:42:11.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.446+0000 7f5bfcbc1640 1 -- 172.21.15.45:0/2841453713 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5bd80762d0 msgr2=0x7f5bd8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:11.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.446+0000 7f5bfcbc1640 1 --2- 172.21.15.45:0/2841453713 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5bd80762d0 0x7f5bd8078790 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5be8034000 tx=0x7f5be802e000 comp rx=0 tx=0).stop 2024-05-08T05:42:11.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.446+0000 7f5bfcbc1640 1 -- 172.21.15.45:0/2841453713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf80d3a50 msgr2=0x7f5bf8158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:11.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.446+0000 7f5bfcbc1640 1 --2- 172.21.15.45:0/2841453713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf80d3a50 0x7f5bf8158ca0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000ef00 tx=0x7f5bf000b480 comp rx=0 tx=0).stop 2024-05-08T05:42:11.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.446+0000 7f5bfcbc1640 1 -- 172.21.15.45:0/2841453713 shutdown_connections 2024-05-08T05:42:11.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.446+0000 7f5bfcbc1640 1 --2- 172.21.15.45:0/2841453713 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5bd80762d0 0x7f5bd8078790 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:11.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.446+0000 7f5bfcbc1640 1 --2- 172.21.15.45:0/2841453713 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5bf80d3670 0x7f5bf80b7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:11.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.446+0000 7f5bfcbc1640 1 --2- 172.21.15.45:0/2841453713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bf80d3a50 0x7f5bf8158ca0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:11.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.446+0000 7f5bfcbc1640 1 --2- 172.21.15.45:0/2841453713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bf80d0e80 0x7f5bf80ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:11.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.447+0000 7f5bfcbc1640 1 -- 172.21.15.45:0/2841453713 >> 172.21.15.45:0/2841453713 conn(0x7f5bf809e390 msgr2=0x7f5bf80b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:11.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.447+0000 7f5bfcbc1640 1 -- 172.21.15.45:0/2841453713 shutdown_connections 2024-05-08T05:42:11.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.447+0000 7f5bfcbc1640 1 -- 172.21.15.45:0/2841453713 wait complete. 2024-05-08T05:42:11.461 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T05:42:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.574+0000 7f2afe189640 1 Processor -- start 2024-05-08T05:42:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.574+0000 7f2afe189640 1 -- start start 2024-05-08T05:42:11.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.575+0000 7f2afe189640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2af80d22a0 0x7f2af8169f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:11.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.575+0000 7f2afe189640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2af80cde80 con 0x7f2af81629f0 2024-05-08T05:42:11.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.575+0000 7f2afe189640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2af80ce120 con 0x7f2af80cfef0 2024-05-08T05:42:11.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.575+0000 7f2afe189640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2af80ce400 con 0x7f2af80d22a0 2024-05-08T05:42:11.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.575+0000 7f2af77fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2af81629f0 0x7f2af80d1120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36742/0 (socket says 172.21.15.45:36742) 2024-05-08T05:42:11.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.575+0000 7f2af77fe640 1 -- 172.21.15.45:0/1047971434 learned_addr learned my addr 172.21.15.45:0/1047971434 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:11.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af7fff640 1 --2- 172.21.15.45:0/1047971434 >> v2:172.21.15.63:3300/0 conn(0x7f2af80d22a0 0x7f2af8169f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:11.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af67fc640 1 -- 172.21.15.45:0/1047971434 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 547448038 0 0) 0x7f2af80cde80 con 0x7f2af81629f0 2024-05-08T05:42:11.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af67fc640 1 -- 172.21.15.45:0/1047971434 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ae0003520 con 0x7f2af81629f0 2024-05-08T05:42:11.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af67fc640 1 -- 172.21.15.45:0/1047971434 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3277000652 0 0) 0x7f2ae0003520 con 0x7f2af81629f0 2024-05-08T05:42:11.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af67fc640 1 -- 172.21.15.45:0/1047971434 >> v2:172.21.15.63:3300/0 conn(0x7f2af80d22a0 msgr2=0x7f2af8169f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:11.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af67fc640 1 --2- 172.21.15.45:0/1047971434 >> v2:172.21.15.63:3300/0 conn(0x7f2af80d22a0 0x7f2af8169f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:11.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af67fc640 1 -- 172.21.15.45:0/1047971434 >> v1:172.21.15.63:6789/0 conn(0x7f2af80cfef0 legacy=0x7f2af80d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:11.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af67fc640 1 -- 172.21.15.45:0/1047971434 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af80ce6e0 con 0x7f2af81629f0 2024-05-08T05:42:11.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af67fc640 1 -- 172.21.15.45:0/1047971434 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f2ae80016f0 con 0x7f2af81629f0 2024-05-08T05:42:11.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af67fc640 1 -- 172.21.15.45:0/1047971434 >> v1:172.21.15.45:6789/0 conn(0x7f2af81629f0 legacy=0x7f2af80d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:11.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af67fc640 1 --2- 172.21.15.45:0/1047971434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ae0003a50 0x7f2ae0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:11.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af67fc640 1 --2- 172.21.15.45:0/1047971434 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae0004800 0x7f2ae0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:11.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af67fc640 1 --2- 172.21.15.45:0/1047971434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae00051c0 0x7f2ae000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:11.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af67fc640 1 -- 172.21.15.45:0/1047971434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2af80ce400 con 0x7f2ae00051c0 2024-05-08T05:42:11.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af67fc640 1 -- 172.21.15.45:0/1047971434 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae000c160 con 0x7f2ae0003a50 2024-05-08T05:42:11.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af67fc640 1 -- 172.21.15.45:0/1047971434 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae000c360 con 0x7f2ae0004800 2024-05-08T05:42:11.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af7fff640 1 --2- 172.21.15.45:0/1047971434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae00051c0 0x7f2ae000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:11.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af77fe640 1 --2- 172.21.15.45:0/1047971434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ae0003a50 0x7f2ae0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:11.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af6ffd640 1 --2- 172.21.15.45:0/1047971434 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae0004800 0x7f2ae0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:11.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af7fff640 1 -- 172.21.15.45:0/1047971434 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae0004800 msgr2=0x7f2ae0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af7fff640 1 --2- 172.21.15.45:0/1047971434 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae0004800 0x7f2ae0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af7fff640 1 -- 172.21.15.45:0/1047971434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ae0003a50 msgr2=0x7f2ae0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af7fff640 1 --2- 172.21.15.45:0/1047971434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ae0003a50 0x7f2ae0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af7fff640 1 -- 172.21.15.45:0/1047971434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae0003520 con 0x7f2ae00051c0 2024-05-08T05:42:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.576+0000 7f2af77fe640 1 --2- 172.21.15.45:0/1047971434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ae0003a50 0x7f2ae0003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.577+0000 7f2af7fff640 1 -- 172.21.15.45:0/1047971434 reap_dead start 2024-05-08T05:42:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.577+0000 7f2af7fff640 1 --2- 172.21.15.45:0/1047971434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae00051c0 0x7f2ae000b9c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009fc0 tx=0x7f2ae0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630f011e4fea5a86 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.577+0000 7f2af67fc640 1 -- 172.21.15.45:0/1047971434 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0035020 con 0x7f2ae00051c0 2024-05-08T05:42:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.577+0000 7f2af67fc640 1 -- 172.21.15.45:0/1047971434 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af0003bd0 con 0x7f2ae00051c0 2024-05-08T05:42:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.577+0000 7f2af67fc640 1 -- 172.21.15.45:0/1047971434 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0009c10 con 0x7f2ae00051c0 2024-05-08T05:42:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.578+0000 7f2afe189640 1 -- 172.21.15.45:0/1047971434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae00051c0 msgr2=0x7f2ae000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.578+0000 7f2afe189640 1 --2- 172.21.15.45:0/1047971434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae00051c0 0x7f2ae000b9c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009fc0 tx=0x7f2ae0005830 comp rx=0 tx=0).stop 2024-05-08T05:42:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.578+0000 7f2afe189640 1 -- 172.21.15.45:0/1047971434 shutdown_connections 2024-05-08T05:42:11.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.578+0000 7f2afe189640 1 --2- 172.21.15.45:0/1047971434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ae00051c0 0x7f2ae000b9c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:11.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.578+0000 7f2afe189640 1 -- 172.21.15.45:0/1047971434 >> 172.21.15.45:0/1047971434 conn(0x7f2af80d2d80 msgr2=0x7f2af80d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:11.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.578+0000 7f2afe189640 1 -- 172.21.15.45:0/1047971434 shutdown_connections 2024-05-08T05:42:11.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.578+0000 7f2afe189640 1 -- 172.21.15.45:0/1047971434 wait complete. 2024-05-08T05:42:11.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.579+0000 7f2afe189640 1 Processor -- start 2024-05-08T05:42:11.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.579+0000 7f2afe189640 1 -- start start 2024-05-08T05:42:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.580+0000 7f2afe189640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae00051c0 0x7f2af809f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.580+0000 7f2afe189640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2af814cd70 0x7f2af809f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.580+0000 7f2afe189640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2af80cade0 0x7f2af80cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.580+0000 7f2afe189640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2af80cf060 con 0x7f2af80cade0 2024-05-08T05:42:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.580+0000 7f2afe189640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2af80ceee0 con 0x7f2af814cd70 2024-05-08T05:42:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.580+0000 7f2afe189640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2af80cf1e0 con 0x7f2ae00051c0 2024-05-08T05:42:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.580+0000 7f2af7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2af80cade0 0x7f2af80cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.580+0000 7f2af7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2af80cade0 0x7f2af80cb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44928/0 (socket says 172.21.15.45:44928) 2024-05-08T05:42:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.580+0000 7f2af7fff640 1 -- 172.21.15.45:0/3135941360 learned_addr learned my addr 172.21.15.45:0/3135941360 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.580+0000 7f2af77fe640 1 --2- 172.21.15.45:0/3135941360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae00051c0 0x7f2af809f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.580+0000 7f2af6ffd640 1 --2- 172.21.15.45:0/3135941360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2af814cd70 0x7f2af809f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.580+0000 7f2af7fff640 1 -- 172.21.15.45:0/3135941360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae00051c0 msgr2=0x7f2af809f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:11.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.580+0000 7f2af7fff640 1 --2- 172.21.15.45:0/3135941360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae00051c0 0x7f2af809f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:11.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.581+0000 7f2af7fff640 1 -- 172.21.15.45:0/3135941360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2af814cd70 msgr2=0x7f2af809f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:11.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.581+0000 7f2af7fff640 1 --2- 172.21.15.45:0/3135941360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2af814cd70 0x7f2af809f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:11.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.581+0000 7f2af7fff640 1 -- 172.21.15.45:0/3135941360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af815c710 con 0x7f2af80cade0 2024-05-08T05:42:11.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.581+0000 7f2af7fff640 1 --2- 172.21.15.45:0/3135941360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2af80cade0 0x7f2af80cb260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2af002af10 tx=0x7f2af0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:11.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.581+0000 7f2adffff640 1 -- 172.21.15.45:0/3135941360 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0035030 con 0x7f2af80cade0 2024-05-08T05:42:11.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.581+0000 7f2adffff640 1 -- 172.21.15.45:0/3135941360 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af0043070 con 0x7f2af80cade0 2024-05-08T05:42:11.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.581+0000 7f2afe189640 1 -- 172.21.15.45:0/3135941360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af80cbb80 con 0x7f2af80cade0 2024-05-08T05:42:11.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.581+0000 7f2adffff640 1 -- 172.21.15.45:0/3135941360 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af002d070 con 0x7f2af80cade0 2024-05-08T05:42:11.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.581+0000 7f2afe189640 1 -- 172.21.15.45:0/3135941360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af814d320 con 0x7f2af80cade0 2024-05-08T05:42:11.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.582+0000 7f2afe189640 1 -- 172.21.15.45:0/3135941360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af80693f0 con 0x7f2af80cade0 2024-05-08T05:42:11.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.583+0000 7f2adffff640 1 -- 172.21.15.45:0/3135941360 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2af0005d10 con 0x7f2af80cade0 2024-05-08T05:42:11.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.583+0000 7f2adffff640 1 --2- 172.21.15.45:0/3135941360 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2ae40762d0 0x7f2ae4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:11.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.583+0000 7f2adffff640 1 -- 172.21.15.45:0/3135941360 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6180+0+0 (secure 0 0 0) 0x7f2af00b7f70 con 0x7f2af80cade0 2024-05-08T05:42:11.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.583+0000 7f2af77fe640 1 --2- 172.21.15.45:0/3135941360 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2ae40762d0 0x7f2ae4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:11.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.584+0000 7f2af77fe640 1 --2- 172.21.15.45:0/3135941360 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2ae40762d0 0x7f2ae4078790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8000fb0 tx=0x7f2ae8007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:11.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.588+0000 7f2adffff640 1 -- 172.21.15.45:0/3135941360 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2af004c020 con 0x7f2af80cade0 2024-05-08T05:42:11.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:11.777+0000 7f2afe189640 1 -- 172.21.15.45:0/3135941360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2af806d4b0 con 0x7f2af80cade0 2024-05-08T05:42:12.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.444+0000 7f2adffff640 1 -- 172.21.15.45:0/3135941360 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) ==== 169+0+0 (secure 0 0 0) 0x7f2af815c710 con 0x7f2af80cade0 2024-05-08T05:42:12.445 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-05-08T05:42:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.448+0000 7f2afe189640 1 -- 172.21.15.45:0/3135941360 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2ae40762d0 msgr2=0x7f2ae4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.448+0000 7f2afe189640 1 --2- 172.21.15.45:0/3135941360 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2ae40762d0 0x7f2ae4078790 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8000fb0 tx=0x7f2ae8007040 comp rx=0 tx=0).stop 2024-05-08T05:42:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.448+0000 7f2afe189640 1 -- 172.21.15.45:0/3135941360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2af80cade0 msgr2=0x7f2af80cb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.448+0000 7f2afe189640 1 --2- 172.21.15.45:0/3135941360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2af80cade0 0x7f2af80cb260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2af002af10 tx=0x7f2af0009750 comp rx=0 tx=0).stop 2024-05-08T05:42:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.448+0000 7f2afe189640 1 -- 172.21.15.45:0/3135941360 shutdown_connections 2024-05-08T05:42:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.448+0000 7f2afe189640 1 --2- 172.21.15.45:0/3135941360 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2ae40762d0 0x7f2ae4078790 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.448+0000 7f2afe189640 1 --2- 172.21.15.45:0/3135941360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2af80cade0 0x7f2af80cb260 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.448+0000 7f2afe189640 1 --2- 172.21.15.45:0/3135941360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2af814cd70 0x7f2af809f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.448+0000 7f2afe189640 1 --2- 172.21.15.45:0/3135941360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae00051c0 0x7f2af809f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:12.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.448+0000 7f2afe189640 1 -- 172.21.15.45:0/3135941360 >> 172.21.15.45:0/3135941360 conn(0x7f2af80d2d80 msgr2=0x7f2af8070280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:12.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.449+0000 7f2afe189640 1 -- 172.21.15.45:0/3135941360 shutdown_connections 2024-05-08T05:42:12.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.449+0000 7f2afe189640 1 -- 172.21.15.45:0/3135941360 wait complete. 2024-05-08T05:42:12.463 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-08T05:42:12.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.577+0000 7f541302a640 1 Processor -- start 2024-05-08T05:42:12.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.577+0000 7f541302a640 1 -- start start 2024-05-08T05:42:12.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.578+0000 7f541302a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f540c0d3670 0x7f540c0b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:12.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.578+0000 7f541302a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f540c0d1050 0x7f540c162920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:12.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.578+0000 7f541302a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f540c0cfc00 con 0x7f540c0d3a50 2024-05-08T05:42:12.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.578+0000 7f541302a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f540c0d2a10 con 0x7f540c0d1050 2024-05-08T05:42:12.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.578+0000 7f541302a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f540c0d04b0 con 0x7f540c0d3670 2024-05-08T05:42:12.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.578+0000 7f54115c7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f540c0d1050 0x7f540c162920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:12.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.578+0000 7f540bfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f540c0d3a50 0x7f540c0d0940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36758/0 (socket says 172.21.15.45:36758) 2024-05-08T05:42:12.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.578+0000 7f540bfff640 1 -- 172.21.15.45:0/3314678132 learned_addr learned my addr 172.21.15.45:0/3314678132 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:12.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.578+0000 7f54115c7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f540c0d1050 0x7f540c162920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44930/0 (socket says 172.21.15.45:44930) 2024-05-08T05:42:12.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.578+0000 7f54115c7640 1 -- 172.21.15.45:0/3314678132 >> v2:172.21.15.63:3300/0 conn(0x7f540c0d3670 msgr2=0x7f540c0b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:12.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.578+0000 7f54115c7640 1 --2- 172.21.15.45:0/3314678132 >> v2:172.21.15.63:3300/0 conn(0x7f540c0d3670 0x7f540c0b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:12.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.578+0000 7f54115c7640 1 -- 172.21.15.45:0/3314678132 >> v1:172.21.15.45:6789/0 conn(0x7f540c0d3a50 legacy=0x7f540c0d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:12.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.579+0000 7f54115c7640 1 -- 172.21.15.45:0/3314678132 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c0cfea0 con 0x7f540c0d1050 2024-05-08T05:42:12.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.579+0000 7f54115c7640 1 --2- 172.21.15.45:0/3314678132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f540c0d1050 0x7f540c162920 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f540c08f7c0 tx=0x7f540402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907d8421fee93fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:12.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.579+0000 7f540b7fe640 1 -- 172.21.15.45:0/3314678132 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404007740 con 0x7f540c0d1050 2024-05-08T05:42:12.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.579+0000 7f540b7fe640 1 -- 172.21.15.45:0/3314678132 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54040078e0 con 0x7f540c0d1050 2024-05-08T05:42:12.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.579+0000 7f540b7fe640 1 -- 172.21.15.45:0/3314678132 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404007cb0 con 0x7f540c0d1050 2024-05-08T05:42:12.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.580+0000 7f541302a640 1 -- 172.21.15.45:0/3314678132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f540c0d1050 msgr2=0x7f540c162920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:12.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.580+0000 7f541302a640 1 --2- 172.21.15.45:0/3314678132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f540c0d1050 0x7f540c162920 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f540c08f7c0 tx=0x7f540402a680 comp rx=0 tx=0).stop 2024-05-08T05:42:12.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.580+0000 7f541302a640 1 -- 172.21.15.45:0/3314678132 shutdown_connections 2024-05-08T05:42:12.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.580+0000 7f541302a640 1 --2- 172.21.15.45:0/3314678132 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f540c0d1050 0x7f540c162920 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:12.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.580+0000 7f541302a640 1 --2- 172.21.15.45:0/3314678132 >> v2:172.21.15.63:3300/0 conn(0x7f540c0d3670 0x7f540c0b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:12.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.580+0000 7f541302a640 1 -- 172.21.15.45:0/3314678132 >> 172.21.15.45:0/3314678132 conn(0x7f540c09e390 msgr2=0x7f540c0d5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:12.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.580+0000 7f541302a640 1 -- 172.21.15.45:0/3314678132 shutdown_connections 2024-05-08T05:42:12.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.580+0000 7f541302a640 1 -- 172.21.15.45:0/3314678132 wait complete. 2024-05-08T05:42:12.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.581+0000 7f541302a640 1 Processor -- start 2024-05-08T05:42:12.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.581+0000 7f541302a640 1 -- start start 2024-05-08T05:42:12.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.582+0000 7f541302a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f540c0d1050 0x7f540c0cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:12.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.582+0000 7f541302a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f540c0d3670 0x7f540c15ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:12.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.582+0000 7f541302a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f540c0d3a50 0x7f540c15d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:12.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.582+0000 7f541302a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f540c162e60 con 0x7f540c0d1050 2024-05-08T05:42:12.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.582+0000 7f541302a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f540c0d0140 con 0x7f540c0d3a50 2024-05-08T05:42:12.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.582+0000 7f541302a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f540c0b4f00 con 0x7f540c0d3670 2024-05-08T05:42:12.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.582+0000 7f5410dc6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f540c0d1050 0x7f540c0cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:12.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.582+0000 7f5410dc6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f540c0d1050 0x7f540c0cd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44942/0 (socket says 172.21.15.45:44942) 2024-05-08T05:42:12.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.582+0000 7f5410dc6640 1 -- 172.21.15.45:0/3416715920 learned_addr learned my addr 172.21.15.45:0/3416715920 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:12.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.583+0000 7f5410dc6640 1 -- 172.21.15.45:0/3416715920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f540c0d3670 msgr2=0x7f540c15ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:12.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.583+0000 7f540bfff640 1 --2- 172.21.15.45:0/3416715920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f540c0d3670 0x7f540c15ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:12.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.583+0000 7f5410dc6640 1 --2- 172.21.15.45:0/3416715920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f540c0d3670 0x7f540c15ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:12.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.583+0000 7f54115c7640 1 --2- 172.21.15.45:0/3416715920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f540c0d3a50 0x7f540c15d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:12.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.583+0000 7f5410dc6640 1 -- 172.21.15.45:0/3416715920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f540c0d3a50 msgr2=0x7f540c15d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.583+0000 7f5410dc6640 1 --2- 172.21.15.45:0/3416715920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f540c0d3a50 0x7f540c15d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.583+0000 7f5410dc6640 1 -- 172.21.15.45:0/3416715920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c0ce1b0 con 0x7f540c0d1050 2024-05-08T05:42:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.583+0000 7f54115c7640 1 --2- 172.21.15.45:0/3416715920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f540c0d3a50 0x7f540c15d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.583+0000 7f5410dc6640 1 --2- 172.21.15.45:0/3416715920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f540c0d1050 0x7f540c0cd980 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00b4a0 tx=0x7f53fc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.583+0000 7f54097fa640 1 -- 172.21.15.45:0/3416715920 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc014070 con 0x7f540c0d1050 2024-05-08T05:42:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.583+0000 7f54097fa640 1 -- 172.21.15.45:0/3416715920 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53fc010610 con 0x7f540c0d1050 2024-05-08T05:42:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.583+0000 7f54097fa640 1 -- 172.21.15.45:0/3416715920 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc010a00 con 0x7f540c0d1050 2024-05-08T05:42:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.583+0000 7f541302a640 1 -- 172.21.15.45:0/3416715920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540c0ce490 con 0x7f540c0d1050 2024-05-08T05:42:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.583+0000 7f541302a640 1 -- 172.21.15.45:0/3416715920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540c0ced30 con 0x7f540c0d1050 2024-05-08T05:42:12.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.584+0000 7f541302a640 1 -- 172.21.15.45:0/3416715920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f540c0ce660 con 0x7f540c0d1050 2024-05-08T05:42:12.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.585+0000 7f54097fa640 1 -- 172.21.15.45:0/3416715920 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f53fc017070 con 0x7f540c0d1050 2024-05-08T05:42:12.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.585+0000 7f54097fa640 1 --2- 172.21.15.45:0/3416715920 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f53f80762d0 0x7f53f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:12.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.585+0000 7f54097fa640 1 -- 172.21.15.45:0/3416715920 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5766+0+0 (secure 0 0 0) 0x7f53fc0970b0 con 0x7f540c0d1050 2024-05-08T05:42:12.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.585+0000 7f540bfff640 1 --2- 172.21.15.45:0/3416715920 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f53f80762d0 0x7f53f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:12.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.586+0000 7f540bfff640 1 --2- 172.21.15.45:0/3416715920 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f53f80762d0 0x7f53f8078790 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f540c0d25a0 tx=0x7f5400009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:12.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.590+0000 7f54097fa640 1 -- 172.21.15.45:0/3416715920 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f53fc0609d0 con 0x7f540c0d1050 2024-05-08T05:42:12.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:12.784+0000 7f541302a640 1 -- 172.21.15.45:0/3416715920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f540c0693f0 con 0x7f540c0d1050 2024-05-08T05:42:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.449+0000 7f54097fa640 1 -- 172.21.15.45:0/3416715920 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v28) ==== 178+0+0 (secure 0 0 0) 0x7f53fc0602d0 con 0x7f540c0d1050 2024-05-08T05:42:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data_ec' removed 2024-05-08T05:42:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.454+0000 7f541302a640 1 -- 172.21.15.45:0/3416715920 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f53f80762d0 msgr2=0x7f53f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.454+0000 7f541302a640 1 --2- 172.21.15.45:0/3416715920 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f53f80762d0 0x7f53f8078790 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f540c0d25a0 tx=0x7f5400009040 comp rx=0 tx=0).stop 2024-05-08T05:42:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.454+0000 7f541302a640 1 -- 172.21.15.45:0/3416715920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f540c0d1050 msgr2=0x7f540c0cd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.454+0000 7f541302a640 1 --2- 172.21.15.45:0/3416715920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f540c0d1050 0x7f540c0cd980 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00b4a0 tx=0x7f53fc00b960 comp rx=0 tx=0).stop 2024-05-08T05:42:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.455+0000 7f541302a640 1 -- 172.21.15.45:0/3416715920 shutdown_connections 2024-05-08T05:42:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.455+0000 7f541302a640 1 --2- 172.21.15.45:0/3416715920 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f53f80762d0 0x7f53f8078790 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.455+0000 7f541302a640 1 --2- 172.21.15.45:0/3416715920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f540c0d3670 0x7f540c15ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.455+0000 7f541302a640 1 --2- 172.21.15.45:0/3416715920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f540c0d3a50 0x7f540c15d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.455+0000 7f541302a640 1 --2- 172.21.15.45:0/3416715920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f540c0d1050 0x7f540c0cd980 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.455+0000 7f541302a640 1 -- 172.21.15.45:0/3416715920 >> 172.21.15.45:0/3416715920 conn(0x7f540c09e390 msgr2=0x7f540c0d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.455+0000 7f541302a640 1 -- 172.21.15.45:0/3416715920 shutdown_connections 2024-05-08T05:42:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.455+0000 7f541302a640 1 -- 172.21.15.45:0/3416715920 wait complete. 2024-05-08T05:42:13.472 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T05:42:13.472 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T05:42:13.472 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T05:42:13.472 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:42:13.472 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T05:42:13.472 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T05:42:13.473 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T05:42:13.473 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:42:13.473 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T05:42:13.473 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T05:42:13.473 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T05:42:13.473 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:42:13.473 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T05:42:13.473 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T05:42:13.474 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T05:42:13.474 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:42:13.483 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T05:42:13.483 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T05:42:13.485 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T05:42:13.485 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T05:42:13.488 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T05:42:13.488 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T05:42:13.490 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T05:42:13.490 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T05:42:13.527 INFO:tasks.ceph.mds.a:Started 2024-05-08T05:42:13.528 DEBUG:teuthology.parallel:result is None 2024-05-08T05:42:13.529 INFO:tasks.ceph.mds.c:Started 2024-05-08T05:42:13.529 DEBUG:teuthology.parallel:result is None 2024-05-08T05:42:13.532 INFO:tasks.ceph.mds.b:Started 2024-05-08T05:42:13.532 DEBUG:teuthology.parallel:result is None 2024-05-08T05:42:13.533 INFO:tasks.ceph.mds.d:Started 2024-05-08T05:42:13.534 DEBUG:teuthology.parallel:result is None 2024-05-08T05:42:13.534 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T05:42:13.610 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-05-08T05:42:13.611 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-05-08T05:42:13.618 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-08T05:42:13.618 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-08T05:42:13.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.649+0000 7f0b9cc46640 1 Processor -- start 2024-05-08T05:42:13.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.649+0000 7f0b9cc46640 1 -- start start 2024-05-08T05:42:13.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.650+0000 7f0b9cc46640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b980d53d0 0x7f0b980d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:13.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.650+0000 7f0b9cc46640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b981630b0 con 0x7f0b9815f420 2024-05-08T05:42:13.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.650+0000 7f0b9cc46640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b980d3e70 con 0x7f0b980d1970 2024-05-08T05:42:13.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.650+0000 7f0b9cc46640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0b980d2550 con 0x7f0b980d53d0 2024-05-08T05:42:13.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.650+0000 7f0b9659c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0b9815f420 0x7f0b980d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36764/0 (socket says 172.21.15.45:36764) 2024-05-08T05:42:13.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.650+0000 7f0b9659c640 1 -- 172.21.15.45:0/3513919199 learned_addr learned my addr 172.21.15.45:0/3513919199 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:13.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.650+0000 7f0b96d9d640 1 --2- 172.21.15.45:0/3513919199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b980d53d0 0x7f0b980d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:13.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.650+0000 7f0b95d9b640 1 -- 172.21.15.45:0/3513919199 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1673588653 0 0) 0x7f0b88001110 con 0x7f0b9815f420 2024-05-08T05:42:13.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.650+0000 7f0b95d9b640 1 -- 172.21.15.45:0/3513919199 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b84003520 con 0x7f0b9815f420 2024-05-08T05:42:13.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.650+0000 7f0b95d9b640 1 -- 172.21.15.45:0/3513919199 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3308405774 0 0) 0x7f0b980d3e70 con 0x7f0b980d1970 2024-05-08T05:42:13.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.650+0000 7f0b95d9b640 1 -- 172.21.15.45:0/3513919199 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b88001110 con 0x7f0b980d1970 2024-05-08T05:42:13.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.650+0000 7f0b96d9d640 1 -- 172.21.15.45:0/3513919199 >> v1:172.21.15.63:6789/0 conn(0x7f0b980d1970 legacy=0x7f0b980d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:13.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.650+0000 7f0b95d9b640 1 -- 172.21.15.45:0/3513919199 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3646825210 0 0) 0x7f0b84003520 con 0x7f0b9815f420 2024-05-08T05:42:13.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.650+0000 7f0b96d9d640 1 -- 172.21.15.45:0/3513919199 >> v1:172.21.15.45:6789/0 conn(0x7f0b9815f420 legacy=0x7f0b980d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:13.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.650+0000 7f0b96d9d640 1 -- 172.21.15.45:0/3513919199 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b980d2c80 con 0x7f0b980d53d0 2024-05-08T05:42:13.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.651+0000 7f0b96d9d640 1 --2- 172.21.15.45:0/3513919199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b980d53d0 0x7f0b980d1e20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009dc0 tx=0x7f0b9002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=420a1c8ac38869df server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.651+0000 7f0b95d9b640 1 -- 172.21.15.45:0/3513919199 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b880018c0 con 0x7f0b980d53d0 2024-05-08T05:42:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.651+0000 7f0b95d9b640 1 -- 172.21.15.45:0/3513919199 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b90007540 con 0x7f0b980d53d0 2024-05-08T05:42:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.651+0000 7f0b95d9b640 1 -- 172.21.15.45:0/3513919199 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90007910 con 0x7f0b980d53d0 2024-05-08T05:42:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.652+0000 7f0b9cc46640 1 -- 172.21.15.45:0/3513919199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b980d53d0 msgr2=0x7f0b980d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.652+0000 7f0b9cc46640 1 --2- 172.21.15.45:0/3513919199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b980d53d0 0x7f0b980d1e20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009dc0 tx=0x7f0b9002a660 comp rx=0 tx=0).stop 2024-05-08T05:42:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.652+0000 7f0b9cc46640 1 -- 172.21.15.45:0/3513919199 shutdown_connections 2024-05-08T05:42:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.652+0000 7f0b9cc46640 1 --2- 172.21.15.45:0/3513919199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b980d53d0 0x7f0b980d1e20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.652+0000 7f0b9cc46640 1 -- 172.21.15.45:0/3513919199 >> 172.21.15.45:0/3513919199 conn(0x7f0b9808eac0 msgr2=0x7f0b9809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.652+0000 7f0b9cc46640 1 -- 172.21.15.45:0/3513919199 shutdown_connections 2024-05-08T05:42:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.652+0000 7f0b9cc46640 1 -- 172.21.15.45:0/3513919199 wait complete. 2024-05-08T05:42:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.653+0000 7f0b9cc46640 1 Processor -- start 2024-05-08T05:42:13.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.653+0000 7f0b9cc46640 1 -- start start 2024-05-08T05:42:13.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.654+0000 7f0b9cc46640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b980d1970 0x7f0b9815a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:13.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.654+0000 7f0b9cc46640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b980d53d0 0x7f0b981612d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:13.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.654+0000 7f0b9cc46640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b980b7ec0 0x7f0b980cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:13.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.654+0000 7f0b9cc46640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0b980d2830 con 0x7f0b980d53d0 2024-05-08T05:42:13.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.654+0000 7f0b9cc46640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0b980d3240 con 0x7f0b980b7ec0 2024-05-08T05:42:13.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.654+0000 7f0b9cc46640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0b980cfb20 con 0x7f0b980d1970 2024-05-08T05:42:13.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.654+0000 7f0b8ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b980d53d0 0x7f0b981612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:13.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.654+0000 7f0b8ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b980d53d0 0x7f0b981612d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44998/0 (socket says 172.21.15.45:44998) 2024-05-08T05:42:13.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.654+0000 7f0b8ffff640 1 -- 172.21.15.45:0/2611446497 learned_addr learned my addr 172.21.15.45:0/2611446497 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:13.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.654+0000 7f0b96d9d640 1 --2- 172.21.15.45:0/2611446497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b980b7ec0 0x7f0b980cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:13.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.654+0000 7f0b9659c640 1 --2- 172.21.15.45:0/2611446497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b980d1970 0x7f0b9815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:13.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.654+0000 7f0b8ffff640 1 -- 172.21.15.45:0/2611446497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b980d1970 msgr2=0x7f0b9815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:13.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.654+0000 7f0b8ffff640 1 --2- 172.21.15.45:0/2611446497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b980d1970 0x7f0b9815a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:13.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.654+0000 7f0b8ffff640 1 -- 172.21.15.45:0/2611446497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b980b7ec0 msgr2=0x7f0b980cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:13.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.654+0000 7f0b8ffff640 1 --2- 172.21.15.45:0/2611446497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b980b7ec0 0x7f0b980cfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:13.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.654+0000 7f0b8ffff640 1 -- 172.21.15.45:0/2611446497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b980c9070 con 0x7f0b980d53d0 2024-05-08T05:42:13.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.654+0000 7f0b96d9d640 1 --2- 172.21.15.45:0/2611446497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b980b7ec0 0x7f0b980cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:42:13.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.655+0000 7f0b8ffff640 1 --2- 172.21.15.45:0/2611446497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b980d53d0 0x7f0b981612d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000c4c0 tx=0x7f0b8000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:13.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.655+0000 7f0b8effd640 1 -- 172.21.15.45:0/2611446497 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b800052d0 con 0x7f0b980d53d0 2024-05-08T05:42:13.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.655+0000 7f0b8effd640 1 -- 172.21.15.45:0/2611446497 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b80005490 con 0x7f0b980d53d0 2024-05-08T05:42:13.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.655+0000 7f0b9cc46640 1 -- 172.21.15.45:0/2611446497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b980c9350 con 0x7f0b980d53d0 2024-05-08T05:42:13.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.655+0000 7f0b8effd640 1 -- 172.21.15.45:0/2611446497 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80012870 con 0x7f0b980d53d0 2024-05-08T05:42:13.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.655+0000 7f0b9cc46640 1 -- 172.21.15.45:0/2611446497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b980c9910 con 0x7f0b980d53d0 2024-05-08T05:42:13.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.656+0000 7f0b9cc46640 1 -- 172.21.15.45:0/2611446497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b980c9b10 con 0x7f0b980d53d0 2024-05-08T05:42:13.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.657+0000 7f0b8effd640 1 -- 172.21.15.45:0/2611446497 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0b80012a10 con 0x7f0b980d53d0 2024-05-08T05:42:13.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.657+0000 7f0b8effd640 1 --2- 172.21.15.45:0/2611446497 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0b780762d0 0x7f0b78078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:13.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.657+0000 7f0b8effd640 1 -- 172.21.15.45:0/2611446497 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5335+0+0 (secure 0 0 0) 0x7f0b80096fb0 con 0x7f0b980d53d0 2024-05-08T05:42:13.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.657+0000 7f0b9659c640 1 --2- 172.21.15.45:0/2611446497 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0b780762d0 0x7f0b78078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:13.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.658+0000 7f0b9659c640 1 --2- 172.21.15.45:0/2611446497 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0b780762d0 0x7f0b78078790 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0b980cbb60 tx=0x7f0b880057f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:13.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.661+0000 7f0b8effd640 1 -- 172.21.15.45:0/2611446497 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0b800603b0 con 0x7f0b980d53d0 2024-05-08T05:42:13.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:13.864+0000 7f0b9cc46640 1 -- 172.21.15.45:0/2611446497 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f0b980693f0 con 0x7f0b980d53d0 2024-05-08T05:42:14.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.459+0000 7f0b8effd640 1 -- 172.21.15.45:0/2611446497 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) ==== 97+0+0 (secure 0 0 0) 0x7f0b800646b0 con 0x7f0b980d53d0 2024-05-08T05:42:14.460 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2024-05-08T05:42:14.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.462+0000 7f0b9cc46640 1 -- 172.21.15.45:0/2611446497 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0b780762d0 msgr2=0x7f0b78078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:14.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.462+0000 7f0b9cc46640 1 --2- 172.21.15.45:0/2611446497 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0b780762d0 0x7f0b78078790 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0b980cbb60 tx=0x7f0b880057f0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.462+0000 7f0b9cc46640 1 -- 172.21.15.45:0/2611446497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b980d53d0 msgr2=0x7f0b981612d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:14.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.462+0000 7f0b9cc46640 1 --2- 172.21.15.45:0/2611446497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b980d53d0 0x7f0b981612d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000c4c0 tx=0x7f0b8000c980 comp rx=0 tx=0).stop 2024-05-08T05:42:14.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.462+0000 7f0b9cc46640 1 -- 172.21.15.45:0/2611446497 shutdown_connections 2024-05-08T05:42:14.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.462+0000 7f0b9cc46640 1 --2- 172.21.15.45:0/2611446497 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0b780762d0 0x7f0b78078790 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.462+0000 7f0b9cc46640 1 --2- 172.21.15.45:0/2611446497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b980b7ec0 0x7f0b980cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.462+0000 7f0b9cc46640 1 --2- 172.21.15.45:0/2611446497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b980d53d0 0x7f0b981612d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.462+0000 7f0b9cc46640 1 --2- 172.21.15.45:0/2611446497 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b980d1970 0x7f0b9815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.462+0000 7f0b9cc46640 1 -- 172.21.15.45:0/2611446497 >> 172.21.15.45:0/2611446497 conn(0x7f0b9808eac0 msgr2=0x7f0b98075840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:14.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.462+0000 7f0b9cc46640 1 -- 172.21.15.45:0/2611446497 shutdown_connections 2024-05-08T05:42:14.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.462+0000 7f0b9cc46640 1 -- 172.21.15.45:0/2611446497 wait complete. 2024-05-08T05:42:14.477 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T05:42:14.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.590+0000 7fe4dd464640 1 Processor -- start 2024-05-08T05:42:14.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.590+0000 7fe4dd464640 1 -- start start 2024-05-08T05:42:14.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.591+0000 7fe4dd464640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe4d815ce30 0x7fe4d806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:14.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.591+0000 7fe4dd464640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe4d8161630 0x7fe4d80ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:14.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.591+0000 7fe4dd464640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d80715b0 con 0x7fe4d806c3a0 2024-05-08T05:42:14.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.591+0000 7fe4dd464640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe4d80d0840 con 0x7fe4d815ce30 2024-05-08T05:42:14.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.591+0000 7fe4dd464640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe4d8071890 con 0x7fe4d8161630 2024-05-08T05:42:14.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.591+0000 7fe4d6ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe4d815ce30 0x7fe4d806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:14.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.591+0000 7fe4d77fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe4d806c3a0 0x7fe4d80d0cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36766/0 (socket says 172.21.15.45:36766) 2024-05-08T05:42:14.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.591+0000 7fe4d77fe640 1 -- 172.21.15.45:0/259347531 learned_addr learned my addr 172.21.15.45:0/259347531 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.591+0000 7fe4d67fc640 1 --2- 172.21.15.45:0/259347531 >> v2:172.21.15.63:3300/0 conn(0x7fe4d8161630 0x7fe4d80ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.591+0000 7fe4d6ffd640 1 -- 172.21.15.45:0/259347531 >> v2:172.21.15.63:3300/0 conn(0x7fe4d8161630 msgr2=0x7fe4d80ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.591+0000 7fe4d6ffd640 1 --2- 172.21.15.45:0/259347531 >> v2:172.21.15.63:3300/0 conn(0x7fe4d8161630 0x7fe4d80ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.591+0000 7fe4d6ffd640 1 -- 172.21.15.45:0/259347531 >> v1:172.21.15.45:6789/0 conn(0x7fe4d806c3a0 legacy=0x7fe4d80d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.591+0000 7fe4d6ffd640 1 -- 172.21.15.45:0/259347531 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d8071b70 con 0x7fe4d815ce30 2024-05-08T05:42:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.591+0000 7fe4d5ffb640 1 -- 172.21.15.45:0/259347531 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4061161501 0 0) 0x7fe4d80715b0 con 0x7fe4d806c3a0 2024-05-08T05:42:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.592+0000 7fe4d6ffd640 1 --2- 172.21.15.45:0/259347531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d815ce30 0x7fe4d806f5d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8070d60 tx=0x7fe4c802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e616c2d6284b9202 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:14.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.592+0000 7fe4d5ffb640 1 -- 172.21.15.45:0/259347531 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8037070 con 0x7fe4d815ce30 2024-05-08T05:42:14.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.592+0000 7fe4d5ffb640 1 -- 172.21.15.45:0/259347531 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c8032440 con 0x7fe4d815ce30 2024-05-08T05:42:14.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.592+0000 7fe4d5ffb640 1 -- 172.21.15.45:0/259347531 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8032810 con 0x7fe4d815ce30 2024-05-08T05:42:14.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.593+0000 7fe4dd464640 1 -- 172.21.15.45:0/259347531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d815ce30 msgr2=0x7fe4d806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:14.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.593+0000 7fe4dd464640 1 --2- 172.21.15.45:0/259347531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d815ce30 0x7fe4d806f5d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8070d60 tx=0x7fe4c802a1a0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.593+0000 7fe4dd464640 1 -- 172.21.15.45:0/259347531 shutdown_connections 2024-05-08T05:42:14.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.593+0000 7fe4dd464640 1 --2- 172.21.15.45:0/259347531 >> v2:172.21.15.63:3300/0 conn(0x7fe4d8161630 0x7fe4d80ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.593+0000 7fe4dd464640 1 --2- 172.21.15.45:0/259347531 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d815ce30 0x7fe4d806f5d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.593+0000 7fe4dd464640 1 -- 172.21.15.45:0/259347531 >> 172.21.15.45:0/259347531 conn(0x7fe4d80b3c40 msgr2=0x7fe4d80b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:14.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.593+0000 7fe4dd464640 1 -- 172.21.15.45:0/259347531 shutdown_connections 2024-05-08T05:42:14.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.593+0000 7fe4dd464640 1 -- 172.21.15.45:0/259347531 wait complete. 2024-05-08T05:42:14.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.594+0000 7fe4dd464640 1 Processor -- start 2024-05-08T05:42:14.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.594+0000 7fe4dd464640 1 -- start start 2024-05-08T05:42:14.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.595+0000 7fe4dd464640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d806c3a0 0x7fe4d815bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:14.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.595+0000 7fe4dd464640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4d815ce30 0x7fe4d8154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:14.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.595+0000 7fe4dd464640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4d8161630 0x7fe4d814d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:14.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.595+0000 7fe4dd464640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d80cee60 con 0x7fe4d806c3a0 2024-05-08T05:42:14.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.595+0000 7fe4dd464640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d80d0460 con 0x7fe4d8161630 2024-05-08T05:42:14.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.595+0000 7fe4dd464640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d80cefe0 con 0x7fe4d815ce30 2024-05-08T05:42:14.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.595+0000 7fe4d6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d806c3a0 0x7fe4d815bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:14.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.595+0000 7fe4d6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d806c3a0 0x7fe4d815bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45022/0 (socket says 172.21.15.45:45022) 2024-05-08T05:42:14.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.595+0000 7fe4d6ffd640 1 -- 172.21.15.45:0/910390244 learned_addr learned my addr 172.21.15.45:0/910390244 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:14.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.595+0000 7fe4d77fe640 1 --2- 172.21.15.45:0/910390244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4d8161630 0x7fe4d814d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:14.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.595+0000 7fe4d67fc640 1 --2- 172.21.15.45:0/910390244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4d815ce30 0x7fe4d8154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:14.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.596+0000 7fe4d6ffd640 1 -- 172.21.15.45:0/910390244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4d815ce30 msgr2=0x7fe4d8154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:14.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.596+0000 7fe4d6ffd640 1 --2- 172.21.15.45:0/910390244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4d815ce30 0x7fe4d8154350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.596+0000 7fe4d6ffd640 1 -- 172.21.15.45:0/910390244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4d8161630 msgr2=0x7fe4d814d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:14.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.596+0000 7fe4d6ffd640 1 --2- 172.21.15.45:0/910390244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4d8161630 0x7fe4d814d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.596+0000 7fe4d6ffd640 1 -- 172.21.15.45:0/910390244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d815ace0 con 0x7fe4d806c3a0 2024-05-08T05:42:14.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.596+0000 7fe4d6ffd640 1 --2- 172.21.15.45:0/910390244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d806c3a0 0x7fe4d815bd80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8035040 tx=0x7fe4c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:14.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.596+0000 7fe4bf7fe640 1 -- 172.21.15.45:0/910390244 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8037040 con 0x7fe4d806c3a0 2024-05-08T05:42:14.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.596+0000 7fe4bf7fe640 1 -- 172.21.15.45:0/910390244 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c8004230 con 0x7fe4d806c3a0 2024-05-08T05:42:14.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.596+0000 7fe4dd464640 1 -- 172.21.15.45:0/910390244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d815afc0 con 0x7fe4d806c3a0 2024-05-08T05:42:14.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.596+0000 7fe4bf7fe640 1 -- 172.21.15.45:0/910390244 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8003610 con 0x7fe4d806c3a0 2024-05-08T05:42:14.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.596+0000 7fe4dd464640 1 -- 172.21.15.45:0/910390244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d815b580 con 0x7fe4d806c3a0 2024-05-08T05:42:14.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.597+0000 7fe4dd464640 1 -- 172.21.15.45:0/910390244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d8069b00 con 0x7fe4d806c3a0 2024-05-08T05:42:14.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.598+0000 7fe4bf7fe640 1 -- 172.21.15.45:0/910390244 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe4c80037b0 con 0x7fe4d806c3a0 2024-05-08T05:42:14.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.598+0000 7fe4bf7fe640 1 --2- 172.21.15.45:0/910390244 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe4c40762d0 0x7fe4c4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:14.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.598+0000 7fe4bf7fe640 1 -- 172.21.15.45:0/910390244 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5249+0+0 (secure 0 0 0) 0x7fe4c80b84f0 con 0x7fe4d806c3a0 2024-05-08T05:42:14.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.598+0000 7fe4d67fc640 1 --2- 172.21.15.45:0/910390244 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe4c40762d0 0x7fe4c4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:14.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.599+0000 7fe4d67fc640 1 --2- 172.21.15.45:0/910390244 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe4c40762d0 0x7fe4c4078790 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe4d81f60f0 tx=0x7fe4cc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:14.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.602+0000 7fe4bf7fe640 1 -- 172.21.15.45:0/910390244 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe4c80820a0 con 0x7fe4d806c3a0 2024-05-08T05:42:14.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.868+0000 7fe4dd464640 1 -- 172.21.15.45:0/910390244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe4d80b4870 con 0x7fe4d806c3a0 2024-05-08T05:42:14.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.869+0000 7fe4bf7fe640 1 -- 172.21.15.45:0/910390244 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7fe4c80819a0 con 0x7fe4d806c3a0 2024-05-08T05:42:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:14.869 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:42:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2024-05-08T05:42:14.869 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2024-05-08T05:42:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmrRgXHBAAX1mgdmOD9JC4S1vJNyLSQw==", 2024-05-08T05:42:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:42:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:42:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:42:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2024-05-08T05:42:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmvn8SKBAAVkXYBbTfu98gl0AptOfJEg==", 2024-05-08T05:42:14.870 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:42:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:42:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:42:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2024-05-08T05:42:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmeiJCIxAAr3Br1fJVgR9TW2Q+4tMQBQ==", 2024-05-08T05:42:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:42:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:42:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:42:14.871 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2024-05-08T05:42:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmXdFILxAAxteIxUYeFdQKd/oxHLpeOQ==", 2024-05-08T05:42:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:42:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:42:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:42:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.872 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2024-05-08T05:42:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBWEDtmLdHHGRAAA1o4Z/lTGERIZhulqVtybw==", 2024-05-08T05:42:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:42:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:42:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2024-05-08T05:42:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBZEDtmMrRVGBAAeKBtcf5fSQMFE6IStUvokQ==", 2024-05-08T05:42:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:42:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:42:14.873 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2024-05-08T05:42:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBcEDtmpozQFhAAS6AF6X592Wh/QiBipcOVew==", 2024-05-08T05:42:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:42:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:42:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2024-05-08T05:42:14.874 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBfEDtm47lSFRAAt5tvpRCljxGtAmHQqcFj2A==", 2024-05-08T05:42:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:42:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:42:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2024-05-08T05:42:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBlEDtmDQ7mGxAA5z7rHZRf7X5g7ZiqEoO2DA==", 2024-05-08T05:42:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:42:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:42:14.875 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2024-05-08T05:42:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBoEDtmBdi0GhAAVMfg+KEJHRvaiS49PEboOg==", 2024-05-08T05:42:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:42:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:42:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2024-05-08T05:42:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBrEDtmG2daGRAANQgJFAT6gNE/vPofCv6veA==", 2024-05-08T05:42:14.876 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:42:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:42:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2024-05-08T05:42:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBuEDtm3tmDGRAARsAzm9KIzgKAs4ZtfCVdaQ==", 2024-05-08T05:42:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:42:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:42:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2024-05-08T05:42:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmDcqGMxAALgQaBYp4Zd8uMqswCXvGaw==", 2024-05-08T05:42:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:42:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:42:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:42:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:42:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2024-05-08T05:42:14.878 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBVEDtm6z0GARAARgRma5hPpH6X95pE5NNrQw==", 2024-05-08T05:42:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:42:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:42:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:42:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:42:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2024-05-08T05:42:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmAMtIORAAJf8oSwHqSzgjsTuqxtQqHQ==", 2024-05-08T05:42:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:42:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:42:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:42:14.879 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:42:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2024-05-08T05:42:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBVEDtm9qK1BhAAokvkJZmNxT5WER9/+EKIpA==", 2024-05-08T05:42:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:42:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:42:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:42:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:42:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.880 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2024-05-08T05:42:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmZ4Q3ABAANcqmwOmt+dNJ5wOiPHHO4Q==", 2024-05-08T05:42:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:42:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:42:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:42:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-mds", 2024-05-08T05:42:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQB0EDtmLwVsEhAALz3MRXOgovM+HzPo3K3xhQ==", 2024-05-08T05:42:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-mds" 2024-05-08T05:42:14.881 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-mgr", 2024-05-08T05:42:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQB0EDtm9Q1sEhAAVPXYchiH4yMgULmOH68HGQ==", 2024-05-08T05:42:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-08T05:42:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-osd", 2024-05-08T05:42:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQB0EDtmlxZsEhAAMDs+imrlncNZI6SYlLeeXg==", 2024-05-08T05:42:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.882 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-osd" 2024-05-08T05:42:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-rbd", 2024-05-08T05:42:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQB0EDtmCh5sEhAA7n3k5P/uSvlDVr4TQmSlfg==", 2024-05-08T05:42:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-08T05:42:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-08T05:42:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQB0EDtmqSVsEhAAEgaiU3gXxFIs2jb3UPtYBA==", 2024-05-08T05:42:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.883 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-08T05:42:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.bootstrap-rgw", 2024-05-08T05:42:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQB0EDtmjC1sEhAA5VoEUo4TZC8Zc4C3tkSOkQ==", 2024-05-08T05:42:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-08T05:42:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2024-05-08T05:42:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmL0XLGBAA1BSFdyiDYV9Je+igOVBvlQ==", 2024-05-08T05:42:14.884 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:42:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-05-08T05:42:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2024-05-08T05:42:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmfM5OFRAAgdJPUfqmnKU8vRrjGMNwrw==", 2024-05-08T05:42:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:42:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-05-08T05:42:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.885 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:14.886 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-05-08T05:42:14.886 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:42:14.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.871+0000 7fe4dd464640 1 -- 172.21.15.45:0/910390244 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe4c40762d0 msgr2=0x7fe4c4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:14.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.871+0000 7fe4dd464640 1 --2- 172.21.15.45:0/910390244 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe4c40762d0 0x7fe4c4078790 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe4d81f60f0 tx=0x7fe4cc008040 comp rx=0 tx=0).stop 2024-05-08T05:42:14.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.871+0000 7fe4dd464640 1 -- 172.21.15.45:0/910390244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d806c3a0 msgr2=0x7fe4d815bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:14.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.871+0000 7fe4dd464640 1 --2- 172.21.15.45:0/910390244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d806c3a0 0x7fe4d815bd80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8035040 tx=0x7fe4c80046c0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.871+0000 7fe4dd464640 1 -- 172.21.15.45:0/910390244 shutdown_connections 2024-05-08T05:42:14.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.871+0000 7fe4dd464640 1 --2- 172.21.15.45:0/910390244 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe4c40762d0 0x7fe4c4078790 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.871+0000 7fe4dd464640 1 --2- 172.21.15.45:0/910390244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4d815ce30 0x7fe4d8154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.871+0000 7fe4dd464640 1 --2- 172.21.15.45:0/910390244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4d8161630 0x7fe4d814d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.871+0000 7fe4dd464640 1 --2- 172.21.15.45:0/910390244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4d806c3a0 0x7fe4d815bd80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.871+0000 7fe4dd464640 1 -- 172.21.15.45:0/910390244 >> 172.21.15.45:0/910390244 conn(0x7fe4d80b3c40 msgr2=0x7fe4d8070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:14.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.871+0000 7fe4dd464640 1 -- 172.21.15.45:0/910390244 shutdown_connections 2024-05-08T05:42:14.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.871+0000 7fe4dd464640 1 -- 172.21.15.45:0/910390244 wait complete. 2024-05-08T05:42:14.888 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-08T05:42:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.993+0000 7fc284e64640 1 Processor -- start 2024-05-08T05:42:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.993+0000 7fc284e64640 1 -- start start 2024-05-08T05:42:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.994+0000 7fc284e64640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc2800d0ec0 0x7fc2800d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.994+0000 7fc284e64640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc2800d3b00 0x7fc2800b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.994+0000 7fc284e64640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2800d1760 con 0x7fc28015f380 2024-05-08T05:42:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.994+0000 7fc284e64640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc2800d30f0 con 0x7fc2800d0ec0 2024-05-08T05:42:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.994+0000 7fc284e64640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc2800d2370 con 0x7fc2800d3b00 2024-05-08T05:42:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.994+0000 7fc27e59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc28015f380 0x7fc2800d07b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36770/0 (socket says 172.21.15.45:36770) 2024-05-08T05:42:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.994+0000 7fc27e59c640 1 -- 172.21.15.45:0/4088426669 learned_addr learned my addr 172.21.15.45:0/4088426669 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.994+0000 7fc27dd9b640 1 --2- 172.21.15.45:0/4088426669 >> v2:172.21.15.45:3300/0 conn(0x7fc2800d0ec0 0x7fc2800d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.994+0000 7fc27ed9d640 1 --2- 172.21.15.45:0/4088426669 >> v2:172.21.15.63:3300/0 conn(0x7fc2800d3b00 0x7fc2800b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.994+0000 7fc27d59a640 1 -- 172.21.15.45:0/4088426669 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 260732523 0 0) 0x7fc2800d1760 con 0x7fc28015f380 2024-05-08T05:42:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.994+0000 7fc27d59a640 1 -- 172.21.15.45:0/4088426669 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc268003520 con 0x7fc28015f380 2024-05-08T05:42:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.995+0000 7fc27dd9b640 1 -- 172.21.15.45:0/4088426669 >> v2:172.21.15.63:3300/0 conn(0x7fc2800d3b00 msgr2=0x7fc2800b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.995+0000 7fc27dd9b640 1 --2- 172.21.15.45:0/4088426669 >> v2:172.21.15.63:3300/0 conn(0x7fc2800d3b00 0x7fc2800b7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.995+0000 7fc27dd9b640 1 -- 172.21.15.45:0/4088426669 >> v1:172.21.15.45:6789/0 conn(0x7fc28015f380 legacy=0x7fc2800d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.995+0000 7fc27dd9b640 1 -- 172.21.15.45:0/4088426669 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2800d1a40 con 0x7fc2800d0ec0 2024-05-08T05:42:14.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.995+0000 7fc27ed9d640 1 --2- 172.21.15.45:0/4088426669 >> v2:172.21.15.63:3300/0 conn(0x7fc2800d3b00 0x7fc2800b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.995+0000 7fc27dd9b640 1 --2- 172.21.15.45:0/4088426669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2800d0ec0 0x7fc2800d35c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc280163f60 tx=0x7fc27402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cde2a2547e6981 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.995+0000 7fc27d59a640 1 -- 172.21.15.45:0/4088426669 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274037070 con 0x7fc2800d0ec0 2024-05-08T05:42:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.995+0000 7fc27d59a640 1 -- 172.21.15.45:0/4088426669 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc274032440 con 0x7fc2800d0ec0 2024-05-08T05:42:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.995+0000 7fc27d59a640 1 -- 172.21.15.45:0/4088426669 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2740327f0 con 0x7fc2800d0ec0 2024-05-08T05:42:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.996+0000 7fc284e64640 1 -- 172.21.15.45:0/4088426669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2800d0ec0 msgr2=0x7fc2800d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.996+0000 7fc284e64640 1 --2- 172.21.15.45:0/4088426669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2800d0ec0 0x7fc2800d35c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc280163f60 tx=0x7fc27402a1a0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.996+0000 7fc284e64640 1 -- 172.21.15.45:0/4088426669 shutdown_connections 2024-05-08T05:42:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.996+0000 7fc284e64640 1 --2- 172.21.15.45:0/4088426669 >> v2:172.21.15.63:3300/0 conn(0x7fc2800d3b00 0x7fc2800b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.996+0000 7fc284e64640 1 --2- 172.21.15.45:0/4088426669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2800d0ec0 0x7fc2800d35c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.996+0000 7fc284e64640 1 -- 172.21.15.45:0/4088426669 >> 172.21.15.45:0/4088426669 conn(0x7fc2800d4fe0 msgr2=0x7fc2800d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.996+0000 7fc284e64640 1 -- 172.21.15.45:0/4088426669 shutdown_connections 2024-05-08T05:42:14.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.996+0000 7fc284e64640 1 -- 172.21.15.45:0/4088426669 wait complete. 2024-05-08T05:42:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.997+0000 7fc284e64640 1 Processor -- start 2024-05-08T05:42:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.997+0000 7fc284e64640 1 -- start start 2024-05-08T05:42:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.998+0000 7fc284e64640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2800d0ec0 0x7fc28015b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.998+0000 7fc284e64640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2800d3b00 0x7fc2800a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.998+0000 7fc284e64640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc28015f380 0x7fc2800a2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.998+0000 7fc284e64640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc2800d2000 con 0x7fc2800d0ec0 2024-05-08T05:42:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.998+0000 7fc284e64640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc2800d3f30 con 0x7fc28015f380 2024-05-08T05:42:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.998+0000 7fc284e64640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc28015f760 con 0x7fc2800d3b00 2024-05-08T05:42:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.998+0000 7fc27e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2800d0ec0 0x7fc28015b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.998+0000 7fc27e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2800d0ec0 0x7fc28015b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45034/0 (socket says 172.21.15.45:45034) 2024-05-08T05:42:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.998+0000 7fc27e59c640 1 -- 172.21.15.45:0/2592153373 learned_addr learned my addr 172.21.15.45:0/2592153373 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.999+0000 7fc27e59c640 1 -- 172.21.15.45:0/2592153373 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2800d3b00 msgr2=0x7fc2800a1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.999+0000 7fc27e59c640 1 --2- 172.21.15.45:0/2592153373 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2800d3b00 0x7fc2800a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.999+0000 7fc27ed9d640 1 --2- 172.21.15.45:0/2592153373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc28015f380 0x7fc2800a2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.999+0000 7fc27e59c640 1 -- 172.21.15.45:0/2592153373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc28015f380 msgr2=0x7fc2800a2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.999+0000 7fc27e59c640 1 --2- 172.21.15.45:0/2592153373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc28015f380 0x7fc2800a2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.999+0000 7fc27e59c640 1 -- 172.21.15.45:0/2592153373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2800a2780 con 0x7fc2800d0ec0 2024-05-08T05:42:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.999+0000 7fc27e59c640 1 --2- 172.21.15.45:0/2592153373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2800d0ec0 0x7fc28015b900 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc27000c500 tx=0x7fc27000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.999+0000 7fc26effd640 1 -- 172.21.15.45:0/2592153373 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc270017070 con 0x7fc2800d0ec0 2024-05-08T05:42:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.999+0000 7fc26effd640 1 -- 172.21.15.45:0/2592153373 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc270005ce0 con 0x7fc2800d0ec0 2024-05-08T05:42:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.999+0000 7fc284e64640 1 -- 172.21.15.45:0/2592153373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2800cd590 con 0x7fc2800d0ec0 2024-05-08T05:42:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.999+0000 7fc26effd640 1 -- 172.21.15.45:0/2592153373 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc270007270 con 0x7fc2800d0ec0 2024-05-08T05:42:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:14.999+0000 7fc284e64640 1 -- 172.21.15.45:0/2592153373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc28009f9b0 con 0x7fc2800d0ec0 2024-05-08T05:42:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.000+0000 7fc284e64640 1 -- 172.21.15.45:0/2592153373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2800693f0 con 0x7fc2800d0ec0 2024-05-08T05:42:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.001+0000 7fc26effd640 1 -- 172.21.15.45:0/2592153373 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc27001a070 con 0x7fc2800d0ec0 2024-05-08T05:42:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.001+0000 7fc26effd640 1 --2- 172.21.15.45:0/2592153373 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc2600762d0 0x7fc260078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.001+0000 7fc26effd640 1 -- 172.21.15.45:0/2592153373 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5249+0+0 (secure 0 0 0) 0x7fc270096740 con 0x7fc2800d0ec0 2024-05-08T05:42:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.002+0000 7fc27dd9b640 1 --2- 172.21.15.45:0/2592153373 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc2600762d0 0x7fc260078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.002+0000 7fc27dd9b640 1 --2- 172.21.15.45:0/2592153373 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc2600762d0 0x7fc260078790 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc274034000 tx=0x7fc27402c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:15.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.006+0000 7fc26effd640 1 -- 172.21.15.45:0/2592153373 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc27005fc00 con 0x7fc2800d0ec0 2024-05-08T05:42:15.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.274+0000 7fc284e64640 1 -- 172.21.15.45:0/2592153373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fc2800cd340 con 0x7fc2800d0ec0 2024-05-08T05:42:15.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.276+0000 7fc26effd640 1 -- 172.21.15.45:0/2592153373 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7fc270096020 con 0x7fc2800d0ec0 2024-05-08T05:42:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.278+0000 7fc284e64640 1 -- 172.21.15.45:0/2592153373 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc2600762d0 msgr2=0x7fc260078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.278+0000 7fc284e64640 1 --2- 172.21.15.45:0/2592153373 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc2600762d0 0x7fc260078790 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc274034000 tx=0x7fc27402c000 comp rx=0 tx=0).stop 2024-05-08T05:42:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.278+0000 7fc284e64640 1 -- 172.21.15.45:0/2592153373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2800d0ec0 msgr2=0x7fc28015b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.278+0000 7fc284e64640 1 --2- 172.21.15.45:0/2592153373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2800d0ec0 0x7fc28015b900 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc27000c500 tx=0x7fc27000c9c0 comp rx=0 tx=0).stop 2024-05-08T05:42:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.278+0000 7fc284e64640 1 -- 172.21.15.45:0/2592153373 shutdown_connections 2024-05-08T05:42:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.278+0000 7fc284e64640 1 --2- 172.21.15.45:0/2592153373 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc2600762d0 0x7fc260078790 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.278+0000 7fc284e64640 1 --2- 172.21.15.45:0/2592153373 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2800d3b00 0x7fc2800a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.278+0000 7fc284e64640 1 --2- 172.21.15.45:0/2592153373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc28015f380 0x7fc2800a2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.278+0000 7fc284e64640 1 --2- 172.21.15.45:0/2592153373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2800d0ec0 0x7fc28015b900 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.278+0000 7fc284e64640 1 -- 172.21.15.45:0/2592153373 >> 172.21.15.45:0/2592153373 conn(0x7fc2800d4fe0 msgr2=0x7fc28006cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.278+0000 7fc284e64640 1 -- 172.21.15.45:0/2592153373 shutdown_connections 2024-05-08T05:42:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.278+0000 7fc284e64640 1 -- 172.21.15.45:0/2592153373 wait complete. 2024-05-08T05:42:15.290 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-08T05:42:15.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.395+0000 7fd890d86640 1 Processor -- start 2024-05-08T05:42:15.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.395+0000 7fd890d86640 1 -- start start 2024-05-08T05:42:15.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.395+0000 7fd890d86640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd88c162f50 0x7fd88c1633e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:15.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.395+0000 7fd890d86640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd88c0d0500 con 0x7fd88c0cf790 2024-05-08T05:42:15.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.395+0000 7fd890d86640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd88c06da60 con 0x7fd88c0b21f0 2024-05-08T05:42:15.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.395+0000 7fd890d86640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fd88c06d810 con 0x7fd88c162f50 2024-05-08T05:42:15.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.396+0000 7fd883fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd88c162f50 0x7fd88c1633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:15.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.396+0000 7fd883fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd88c162f50 0x7fd88c1633e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45046/0 (socket says 172.21.15.45:45046) 2024-05-08T05:42:15.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.396+0000 7fd883fff640 1 -- 172.21.15.45:0/2020874937 learned_addr learned my addr 172.21.15.45:0/2020874937 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:15.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.396+0000 7fd88ad9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd88c0cf790 0x7fd88c0cfc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36784/0 (socket says 172.21.15.45:36784) 2024-05-08T05:42:15.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.396+0000 7fd883fff640 1 -- 172.21.15.45:0/2020874937 >> v1:172.21.15.63:6789/0 conn(0x7fd88c0b21f0 legacy=0x7fd88c06e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:15.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.396+0000 7fd883fff640 1 -- 172.21.15.45:0/2020874937 >> v1:172.21.15.45:6789/0 conn(0x7fd88c0cf790 legacy=0x7fd88c0cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:15.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.396+0000 7fd883fff640 1 -- 172.21.15.45:0/2020874937 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88c06dd40 con 0x7fd88c162f50 2024-05-08T05:42:15.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.397+0000 7fd883fff640 1 --2- 172.21.15.45:0/2020874937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c162f50 0x7fd88c1633e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd874009870 tx=0x7fd87402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d1733f9a87999f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:15.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.397+0000 7fd889d9b640 1 -- 172.21.15.45:0/2020874937 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874037070 con 0x7fd88c162f50 2024-05-08T05:42:15.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.397+0000 7fd889d9b640 1 -- 172.21.15.45:0/2020874937 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd874032440 con 0x7fd88c162f50 2024-05-08T05:42:15.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.397+0000 7fd889d9b640 1 -- 172.21.15.45:0/2020874937 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874032810 con 0x7fd88c162f50 2024-05-08T05:42:15.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.397+0000 7fd890d86640 1 -- 172.21.15.45:0/2020874937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c162f50 msgr2=0x7fd88c1633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:15.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.397+0000 7fd890d86640 1 --2- 172.21.15.45:0/2020874937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c162f50 0x7fd88c1633e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd874009870 tx=0x7fd87402a1e0 comp rx=0 tx=0).stop 2024-05-08T05:42:15.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.397+0000 7fd890d86640 1 -- 172.21.15.45:0/2020874937 shutdown_connections 2024-05-08T05:42:15.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.397+0000 7fd890d86640 1 --2- 172.21.15.45:0/2020874937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c162f50 0x7fd88c1633e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:15.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.397+0000 7fd890d86640 1 -- 172.21.15.45:0/2020874937 >> 172.21.15.45:0/2020874937 conn(0x7fd88c09ba40 msgr2=0x7fd88c09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:15.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.397+0000 7fd890d86640 1 -- 172.21.15.45:0/2020874937 shutdown_connections 2024-05-08T05:42:15.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.397+0000 7fd890d86640 1 -- 172.21.15.45:0/2020874937 wait complete. 2024-05-08T05:42:15.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.399+0000 7fd890d86640 1 Processor -- start 2024-05-08T05:42:15.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.399+0000 7fd890d86640 1 -- start start 2024-05-08T05:42:15.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.399+0000 7fd890d86640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd88c0b21f0 0x7fd88c1500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:15.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.399+0000 7fd890d86640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd88c0cf790 0x7fd88c0d2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:15.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.399+0000 7fd890d86640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c162f50 0x7fd88c0d2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:15.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.399+0000 7fd890d86640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c0cd480 con 0x7fd88c162f50 2024-05-08T05:42:15.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.399+0000 7fd890d86640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c06e0b0 con 0x7fd88c0b21f0 2024-05-08T05:42:15.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.400+0000 7fd890d86640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd88c0cd600 con 0x7fd88c0cf790 2024-05-08T05:42:15.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.400+0000 7fd88ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c162f50 0x7fd88c0d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:15.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.400+0000 7fd88ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c162f50 0x7fd88c0d2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45062/0 (socket says 172.21.15.45:45062) 2024-05-08T05:42:15.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.400+0000 7fd88ad9d640 1 -- 172.21.15.45:0/1285301651 learned_addr learned my addr 172.21.15.45:0/1285301651 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:15.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.400+0000 7fd88a59c640 1 --2- 172.21.15.45:0/1285301651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd88c0b21f0 0x7fd88c1500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:15.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.400+0000 7fd883fff640 1 --2- 172.21.15.45:0/1285301651 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd88c0cf790 0x7fd88c0d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:15.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.400+0000 7fd88ad9d640 1 -- 172.21.15.45:0/1285301651 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd88c0cf790 msgr2=0x7fd88c0d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:15.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.400+0000 7fd88ad9d640 1 --2- 172.21.15.45:0/1285301651 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd88c0cf790 0x7fd88c0d2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:15.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.400+0000 7fd88ad9d640 1 -- 172.21.15.45:0/1285301651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd88c0b21f0 msgr2=0x7fd88c1500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:15.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.400+0000 7fd88ad9d640 1 --2- 172.21.15.45:0/1285301651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd88c0b21f0 0x7fd88c1500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:15.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.400+0000 7fd88ad9d640 1 -- 172.21.15.45:0/1285301651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88c0b2e10 con 0x7fd88c162f50 2024-05-08T05:42:15.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.400+0000 7fd88ad9d640 1 --2- 172.21.15.45:0/1285301651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c162f50 0x7fd88c0d2770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd88400c7c0 tx=0x7fd88400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:15.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.400+0000 7fd882ffd640 1 -- 172.21.15.45:0/1285301651 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884017070 con 0x7fd88c162f50 2024-05-08T05:42:15.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.400+0000 7fd882ffd640 1 -- 172.21.15.45:0/1285301651 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd88400a640 con 0x7fd88c162f50 2024-05-08T05:42:15.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.400+0000 7fd890d86640 1 -- 172.21.15.45:0/1285301651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88c0b30f0 con 0x7fd88c162f50 2024-05-08T05:42:15.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.400+0000 7fd882ffd640 1 -- 172.21.15.45:0/1285301651 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88400aa30 con 0x7fd88c162f50 2024-05-08T05:42:15.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.400+0000 7fd890d86640 1 -- 172.21.15.45:0/1285301651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88c0b36b0 con 0x7fd88c162f50 2024-05-08T05:42:15.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.401+0000 7fd890d86640 1 -- 172.21.15.45:0/1285301651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd88c069b00 con 0x7fd88c162f50 2024-05-08T05:42:15.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.402+0000 7fd882ffd640 1 -- 172.21.15.45:0/1285301651 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd88401a070 con 0x7fd88c162f50 2024-05-08T05:42:15.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.403+0000 7fd882ffd640 1 --2- 172.21.15.45:0/1285301651 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd86c0762d0 0x7fd86c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:15.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.403+0000 7fd882ffd640 1 -- 172.21.15.45:0/1285301651 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5249+0+0 (secure 0 0 0) 0x7fd884096a30 con 0x7fd88c162f50 2024-05-08T05:42:15.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.403+0000 7fd88a59c640 1 --2- 172.21.15.45:0/1285301651 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd86c0762d0 0x7fd86c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:15.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.403+0000 7fd88a59c640 1 --2- 172.21.15.45:0/1285301651 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd86c0762d0 0x7fd86c078790 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0017c0 tx=0x7fd87c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:15.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.407+0000 7fd882ffd640 1 -- 172.21.15.45:0/1285301651 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd88405ff00 con 0x7fd88c162f50 2024-05-08T05:42:15.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.671+0000 7fd890d86640 1 -- 172.21.15.45:0/1285301651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fd88c070a30 con 0x7fd88c162f50 2024-05-08T05:42:15.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.673+0000 7fd882ffd640 1 -- 172.21.15.45:0/1285301651 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7fd884064200 con 0x7fd88c162f50 2024-05-08T05:42:15.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.675+0000 7fd890d86640 1 -- 172.21.15.45:0/1285301651 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd86c0762d0 msgr2=0x7fd86c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.675+0000 7fd890d86640 1 --2- 172.21.15.45:0/1285301651 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd86c0762d0 0x7fd86c078790 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0017c0 tx=0x7fd87c009040 comp rx=0 tx=0).stop 2024-05-08T05:42:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.675+0000 7fd890d86640 1 -- 172.21.15.45:0/1285301651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c162f50 msgr2=0x7fd88c0d2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.675+0000 7fd890d86640 1 --2- 172.21.15.45:0/1285301651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c162f50 0x7fd88c0d2770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd88400c7c0 tx=0x7fd88400cc80 comp rx=0 tx=0).stop 2024-05-08T05:42:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.675+0000 7fd890d86640 1 -- 172.21.15.45:0/1285301651 shutdown_connections 2024-05-08T05:42:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.675+0000 7fd890d86640 1 --2- 172.21.15.45:0/1285301651 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd86c0762d0 0x7fd86c078790 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.675+0000 7fd890d86640 1 --2- 172.21.15.45:0/1285301651 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd88c0cf790 0x7fd88c0d2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.675+0000 7fd890d86640 1 --2- 172.21.15.45:0/1285301651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd88c162f50 0x7fd88c0d2770 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.675+0000 7fd890d86640 1 --2- 172.21.15.45:0/1285301651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd88c0b21f0 0x7fd88c1500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.675+0000 7fd890d86640 1 -- 172.21.15.45:0/1285301651 >> 172.21.15.45:0/1285301651 conn(0x7fd88c09ba40 msgr2=0x7fd88c0aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.675+0000 7fd890d86640 1 -- 172.21.15.45:0/1285301651 shutdown_connections 2024-05-08T05:42:15.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.675+0000 7fd890d86640 1 -- 172.21.15.45:0/1285301651 wait complete. 2024-05-08T05:42:15.686 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-08T05:42:15.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.791+0000 7f23165da640 1 Processor -- start 2024-05-08T05:42:15.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.791+0000 7f23165da640 1 -- start start 2024-05-08T05:42:15.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.792+0000 7f23165da640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f231006e0f0 0x7f23100cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.792+0000 7f23165da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23100d0360 0x7f23100b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.792+0000 7f23165da640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f231015d2a0 con 0x7f231006f060 2024-05-08T05:42:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.792+0000 7f23165da640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f23100d07c0 con 0x7f231006e0f0 2024-05-08T05:42:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.792+0000 7f23165da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f231006f440 con 0x7f23100d0360 2024-05-08T05:42:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.792+0000 7f230f7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f231006e0f0 0x7f23100cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.792+0000 7f230ffff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f231006f060 0x7f231006fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36790/0 (socket says 172.21.15.45:36790) 2024-05-08T05:42:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.792+0000 7f230ffff640 1 -- 172.21.15.45:0/3207429100 learned_addr learned my addr 172.21.15.45:0/3207429100 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.792+0000 7f2314b77640 1 --2- 172.21.15.45:0/3207429100 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23100d0360 0x7f23100b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.792+0000 7f230effd640 1 -- 172.21.15.45:0/3207429100 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2276260872 0 0) 0x7f231015d2a0 con 0x7f231006f060 2024-05-08T05:42:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.792+0000 7f230effd640 1 -- 172.21.15.45:0/3207429100 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22f8003520 con 0x7f231006f060 2024-05-08T05:42:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.792+0000 7f230f7fe640 1 -- 172.21.15.45:0/3207429100 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23100d0360 msgr2=0x7f23100b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.792+0000 7f230f7fe640 1 --2- 172.21.15.45:0/3207429100 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23100d0360 0x7f23100b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:15.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.792+0000 7f230f7fe640 1 -- 172.21.15.45:0/3207429100 >> v1:172.21.15.45:6789/0 conn(0x7f231006f060 legacy=0x7f231006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:15.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.792+0000 7f230f7fe640 1 -- 172.21.15.45:0/3207429100 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23101609b0 con 0x7f231006e0f0 2024-05-08T05:42:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.793+0000 7f230f7fe640 1 --2- 172.21.15.45:0/3207429100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f231006e0f0 0x7f23100cfe20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f23040097a0 tx=0x7f230402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b141f69a7e451b2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.793+0000 7f230effd640 1 -- 172.21.15.45:0/3207429100 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304037070 con 0x7f231006e0f0 2024-05-08T05:42:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.793+0000 7f230effd640 1 -- 172.21.15.45:0/3207429100 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2304032440 con 0x7f231006e0f0 2024-05-08T05:42:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.793+0000 7f230effd640 1 -- 172.21.15.45:0/3207429100 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304032810 con 0x7f231006e0f0 2024-05-08T05:42:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.794+0000 7f23165da640 1 -- 172.21.15.45:0/3207429100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f231006e0f0 msgr2=0x7f23100cfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:15.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.794+0000 7f23165da640 1 --2- 172.21.15.45:0/3207429100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f231006e0f0 0x7f23100cfe20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f23040097a0 tx=0x7f230402a180 comp rx=0 tx=0).stop 2024-05-08T05:42:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.794+0000 7f23165da640 1 -- 172.21.15.45:0/3207429100 shutdown_connections 2024-05-08T05:42:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.794+0000 7f23165da640 1 --2- 172.21.15.45:0/3207429100 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f23100d0360 0x7f23100b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.794+0000 7f23165da640 1 --2- 172.21.15.45:0/3207429100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f231006e0f0 0x7f23100cfe20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.794+0000 7f23165da640 1 -- 172.21.15.45:0/3207429100 >> 172.21.15.45:0/3207429100 conn(0x7f231006c3a0 msgr2=0x7f231006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.794+0000 7f23165da640 1 -- 172.21.15.45:0/3207429100 shutdown_connections 2024-05-08T05:42:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.794+0000 7f23165da640 1 -- 172.21.15.45:0/3207429100 wait complete. 2024-05-08T05:42:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.795+0000 7f23165da640 1 Processor -- start 2024-05-08T05:42:15.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.795+0000 7f23165da640 1 -- start start 2024-05-08T05:42:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.796+0000 7f23165da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231006e0f0 0x7f23100b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.796+0000 7f23165da640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f231006f060 0x7f231014a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.796+0000 7f23165da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23100d0360 0x7f23100af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.796+0000 7f23165da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2310161250 con 0x7f23100d0360 2024-05-08T05:42:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.796+0000 7f23165da640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2310160f70 con 0x7f231006f060 2024-05-08T05:42:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.796+0000 7f23165da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2310161530 con 0x7f231006e0f0 2024-05-08T05:42:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.796+0000 7f2314b77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23100d0360 0x7f23100af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.796+0000 7f2314b77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23100d0360 0x7f23100af850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45074/0 (socket says 172.21.15.45:45074) 2024-05-08T05:42:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.796+0000 7f2314b77640 1 -- 172.21.15.45:0/454336468 learned_addr learned my addr 172.21.15.45:0/454336468 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:15.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.796+0000 7f230f7fe640 1 --2- 172.21.15.45:0/454336468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f231006f060 0x7f231014a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:15.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.796+0000 7f230ffff640 1 --2- 172.21.15.45:0/454336468 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231006e0f0 0x7f23100b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:15.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.796+0000 7f2314b77640 1 -- 172.21.15.45:0/454336468 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231006e0f0 msgr2=0x7f23100b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:15.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.796+0000 7f2314b77640 1 --2- 172.21.15.45:0/454336468 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231006e0f0 0x7f23100b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:15.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.796+0000 7f2314b77640 1 -- 172.21.15.45:0/454336468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f231006f060 msgr2=0x7f231014a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:15.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.796+0000 7f2314b77640 1 --2- 172.21.15.45:0/454336468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f231006f060 0x7f231014a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:15.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.796+0000 7f2314b77640 1 -- 172.21.15.45:0/454336468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231014abe0 con 0x7f23100d0360 2024-05-08T05:42:15.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.796+0000 7f2314b77640 1 --2- 172.21.15.45:0/454336468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23100d0360 0x7f23100af850 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2308009d30 tx=0x7f230800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:15.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.797+0000 7f230cff9640 1 -- 172.21.15.45:0/454336468 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308018020 con 0x7f23100d0360 2024-05-08T05:42:15.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.797+0000 7f230cff9640 1 -- 172.21.15.45:0/454336468 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2308014480 con 0x7f23100d0360 2024-05-08T05:42:15.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.797+0000 7f230cff9640 1 -- 172.21.15.45:0/454336468 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308014870 con 0x7f23100d0360 2024-05-08T05:42:15.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.797+0000 7f23165da640 1 -- 172.21.15.45:0/454336468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23100b05a0 con 0x7f23100d0360 2024-05-08T05:42:15.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.797+0000 7f23165da640 1 -- 172.21.15.45:0/454336468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23100b0b60 con 0x7f23100d0360 2024-05-08T05:42:15.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.798+0000 7f23165da640 1 -- 172.21.15.45:0/454336468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23100693f0 con 0x7f23100d0360 2024-05-08T05:42:15.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.798+0000 7f230cff9640 1 -- 172.21.15.45:0/454336468 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2308005d10 con 0x7f23100d0360 2024-05-08T05:42:15.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.799+0000 7f230cff9640 1 --2- 172.21.15.45:0/454336468 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f22fc0762d0 0x7f22fc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:15.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.799+0000 7f230cff9640 1 -- 172.21.15.45:0/454336468 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5249+0+0 (secure 0 0 0) 0x7f2308097e60 con 0x7f23100d0360 2024-05-08T05:42:15.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.799+0000 7f230ffff640 1 --2- 172.21.15.45:0/454336468 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f22fc0762d0 0x7f22fc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:15.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.800+0000 7f230ffff640 1 --2- 172.21.15.45:0/454336468 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f22fc0762d0 0x7f22fc078790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2300007a60 tx=0x7f2300009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:15.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:15.803+0000 7f230cff9640 1 -- 172.21.15.45:0/454336468 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2308022070 con 0x7f23100d0360 2024-05-08T05:42:16.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.068+0000 7f23165da640 1 -- 172.21.15.45:0/454336468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f23100765b0 con 0x7f23100d0360 2024-05-08T05:42:16.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.070+0000 7f230cff9640 1 -- 172.21.15.45:0/454336468 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f231014abe0 con 0x7f23100d0360 2024-05-08T05:42:16.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.072+0000 7f23165da640 1 -- 172.21.15.45:0/454336468 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f22fc0762d0 msgr2=0x7f22fc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:16.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.072+0000 7f23165da640 1 --2- 172.21.15.45:0/454336468 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f22fc0762d0 0x7f22fc078790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2300007a60 tx=0x7f2300009040 comp rx=0 tx=0).stop 2024-05-08T05:42:16.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.072+0000 7f23165da640 1 -- 172.21.15.45:0/454336468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23100d0360 msgr2=0x7f23100af850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:16.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.072+0000 7f23165da640 1 --2- 172.21.15.45:0/454336468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23100d0360 0x7f23100af850 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2308009d30 tx=0x7f230800c490 comp rx=0 tx=0).stop 2024-05-08T05:42:16.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.072+0000 7f23165da640 1 -- 172.21.15.45:0/454336468 shutdown_connections 2024-05-08T05:42:16.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.072+0000 7f23165da640 1 --2- 172.21.15.45:0/454336468 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f22fc0762d0 0x7f22fc078790 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.072+0000 7f23165da640 1 --2- 172.21.15.45:0/454336468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23100d0360 0x7f23100af850 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.072+0000 7f23165da640 1 --2- 172.21.15.45:0/454336468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f231006f060 0x7f231014a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.072+0000 7f23165da640 1 --2- 172.21.15.45:0/454336468 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f231006e0f0 0x7f23100b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.072+0000 7f23165da640 1 -- 172.21.15.45:0/454336468 >> 172.21.15.45:0/454336468 conn(0x7f231006c3a0 msgr2=0x7f231015d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:16.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.072+0000 7f23165da640 1 -- 172.21.15.45:0/454336468 shutdown_connections 2024-05-08T05:42:16.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.072+0000 7f23165da640 1 -- 172.21.15.45:0/454336468 wait complete. 2024-05-08T05:42:16.083 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-08T05:42:16.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.188+0000 7fb62b2c5640 1 Processor -- start 2024-05-08T05:42:16.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.188+0000 7fb62b2c5640 1 -- start start 2024-05-08T05:42:16.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.189+0000 7fb62b2c5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb62415f420 0x7fb6240d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:16.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.189+0000 7fb62b2c5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb6240d3ca0 0x7fb6240d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:16.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.189+0000 7fb62b2c5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6241630b0 con 0x7fb6240d1eb0 2024-05-08T05:42:16.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.189+0000 7fb62b2c5640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb6240d22f0 con 0x7fb6240d3ca0 2024-05-08T05:42:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.189+0000 7fb62b2c5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb6240d2550 con 0x7fb62415f420 2024-05-08T05:42:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.189+0000 7fb628860640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb6240d3ca0 0x7fb6240d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.189+0000 7fb628860640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb6240d3ca0 0x7fb6240d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45086/0 (socket says 172.21.15.45:45086) 2024-05-08T05:42:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.189+0000 7fb628860640 1 -- 172.21.15.45:0/781707034 learned_addr learned my addr 172.21.15.45:0/781707034 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.189+0000 7fb629061640 1 --2- 172.21.15.45:0/781707034 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb62415f420 0x7fb6240d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.189+0000 7fb628860640 1 -- 172.21.15.45:0/781707034 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb62415f420 msgr2=0x7fb6240d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.189+0000 7fb628860640 1 --2- 172.21.15.45:0/781707034 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb62415f420 0x7fb6240d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.189+0000 7fb628860640 1 -- 172.21.15.45:0/781707034 >> v1:172.21.15.63:6789/0 conn(0x7fb6240d1eb0 legacy=0x7fb6240d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.189+0000 7fb628860640 1 -- 172.21.15.45:0/781707034 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6240d2c80 con 0x7fb6240d3ca0 2024-05-08T05:42:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.189+0000 7fb629061640 1 --2- 172.21.15.45:0/781707034 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb62415f420 0x7fb6240d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.190+0000 7fb628860640 1 --2- 172.21.15.45:0/781707034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6240d3ca0 0x7fb6240d1970 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb614009870 tx=0x7fb61402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36087371f1bf644 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.190+0000 7fb61bfff640 1 -- 172.21.15.45:0/781707034 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb614037070 con 0x7fb6240d3ca0 2024-05-08T05:42:16.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.190+0000 7fb61bfff640 1 -- 172.21.15.45:0/781707034 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb614032440 con 0x7fb6240d3ca0 2024-05-08T05:42:16.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.190+0000 7fb61bfff640 1 -- 172.21.15.45:0/781707034 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb614032810 con 0x7fb6240d3ca0 2024-05-08T05:42:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.191+0000 7fb62b2c5640 1 -- 172.21.15.45:0/781707034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6240d3ca0 msgr2=0x7fb6240d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.191+0000 7fb62b2c5640 1 --2- 172.21.15.45:0/781707034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6240d3ca0 0x7fb6240d1970 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb614009870 tx=0x7fb61402a270 comp rx=0 tx=0).stop 2024-05-08T05:42:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.191+0000 7fb62b2c5640 1 -- 172.21.15.45:0/781707034 shutdown_connections 2024-05-08T05:42:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.191+0000 7fb62b2c5640 1 --2- 172.21.15.45:0/781707034 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6240d3ca0 0x7fb6240d1970 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.191+0000 7fb62b2c5640 1 --2- 172.21.15.45:0/781707034 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb62415f420 0x7fb6240d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.191+0000 7fb62b2c5640 1 -- 172.21.15.45:0/781707034 >> 172.21.15.45:0/781707034 conn(0x7fb62408eac0 msgr2=0x7fb62409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.191+0000 7fb62b2c5640 1 -- 172.21.15.45:0/781707034 shutdown_connections 2024-05-08T05:42:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.191+0000 7fb62b2c5640 1 -- 172.21.15.45:0/781707034 wait complete. 2024-05-08T05:42:16.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.192+0000 7fb62b2c5640 1 Processor -- start 2024-05-08T05:42:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.192+0000 7fb62b2c5640 1 -- start start 2024-05-08T05:42:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.193+0000 7fb62b2c5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6240d1eb0 0x7fb62415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.193+0000 7fb62b2c5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6240c9b50 0x7fb6240b8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.193+0000 7fb62b2c5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6240b8b50 0x7fb6240b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.193+0000 7fb62b2c5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb6240d2830 con 0x7fb6240d1eb0 2024-05-08T05:42:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.193+0000 7fb62b2c5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb6240d3240 con 0x7fb6240b8b50 2024-05-08T05:42:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.193+0000 7fb62b2c5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb6240d53d0 con 0x7fb6240c9b50 2024-05-08T05:42:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.193+0000 7fb629061640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6240d1eb0 0x7fb62415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.193+0000 7fb629061640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6240d1eb0 0x7fb62415d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45090/0 (socket says 172.21.15.45:45090) 2024-05-08T05:42:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.193+0000 7fb629061640 1 -- 172.21.15.45:0/1660514279 learned_addr learned my addr 172.21.15.45:0/1660514279 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.193+0000 7fb628860640 1 --2- 172.21.15.45:0/1660514279 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6240c9b50 0x7fb6240b8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.193+0000 7fb629862640 1 --2- 172.21.15.45:0/1660514279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6240b8b50 0x7fb6240b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.193+0000 7fb629061640 1 -- 172.21.15.45:0/1660514279 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6240c9b50 msgr2=0x7fb6240b8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:16.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.193+0000 7fb629061640 1 --2- 172.21.15.45:0/1660514279 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6240c9b50 0x7fb6240b8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.193+0000 7fb629061640 1 -- 172.21.15.45:0/1660514279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6240b8b50 msgr2=0x7fb6240b5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:16.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.193+0000 7fb629061640 1 --2- 172.21.15.45:0/1660514279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6240b8b50 0x7fb6240b5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.193+0000 7fb629061640 1 -- 172.21.15.45:0/1660514279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6240ca0c0 con 0x7fb6240d1eb0 2024-05-08T05:42:16.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.193+0000 7fb629061640 1 --2- 172.21.15.45:0/1660514279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6240d1eb0 0x7fb62415d6c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb62000d800 tx=0x7fb62000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:16.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.193+0000 7fb619ffb640 1 -- 172.21.15.45:0/1660514279 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620004030 con 0x7fb6240d1eb0 2024-05-08T05:42:16.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.194+0000 7fb619ffb640 1 -- 172.21.15.45:0/1660514279 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6200041d0 con 0x7fb6240d1eb0 2024-05-08T05:42:16.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.194+0000 7fb62b2c5640 1 -- 172.21.15.45:0/1660514279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6240ca3a0 con 0x7fb6240d1eb0 2024-05-08T05:42:16.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.194+0000 7fb619ffb640 1 -- 172.21.15.45:0/1660514279 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6200096a0 con 0x7fb6240d1eb0 2024-05-08T05:42:16.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.194+0000 7fb62b2c5640 1 -- 172.21.15.45:0/1660514279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6240ca960 con 0x7fb6240d1eb0 2024-05-08T05:42:16.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.195+0000 7fb62b2c5640 1 -- 172.21.15.45:0/1660514279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6240693f0 con 0x7fb6240d1eb0 2024-05-08T05:42:16.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.195+0000 7fb619ffb640 1 -- 172.21.15.45:0/1660514279 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb620009840 con 0x7fb6240d1eb0 2024-05-08T05:42:16.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.196+0000 7fb619ffb640 1 --2- 172.21.15.45:0/1660514279 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb6080762d0 0x7fb608078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:16.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.196+0000 7fb619ffb640 1 -- 172.21.15.45:0/1660514279 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5249+0+0 (secure 0 0 0) 0x7fb6200977b0 con 0x7fb6240d1eb0 2024-05-08T05:42:16.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.196+0000 7fb628860640 1 --2- 172.21.15.45:0/1660514279 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb6080762d0 0x7fb608078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:16.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.197+0000 7fb628860640 1 --2- 172.21.15.45:0/1660514279 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb6080762d0 0x7fb608078790 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb614034000 tx=0x7fb61402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:16.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.200+0000 7fb619ffb640 1 -- 172.21.15.45:0/1660514279 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb620061230 con 0x7fb6240d1eb0 2024-05-08T05:42:16.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.466+0000 7fb62b2c5640 1 -- 172.21.15.45:0/1660514279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fb6240cd8a0 con 0x7fb6240d1eb0 2024-05-08T05:42:16.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.468+0000 7fb619ffb640 1 -- 172.21.15.45:0/1660514279 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fb620060b30 con 0x7fb6240d1eb0 2024-05-08T05:42:16.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.470+0000 7fb62b2c5640 1 -- 172.21.15.45:0/1660514279 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb6080762d0 msgr2=0x7fb608078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:16.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.470+0000 7fb62b2c5640 1 --2- 172.21.15.45:0/1660514279 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb6080762d0 0x7fb608078790 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb614034000 tx=0x7fb61402e000 comp rx=0 tx=0).stop 2024-05-08T05:42:16.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.470+0000 7fb62b2c5640 1 -- 172.21.15.45:0/1660514279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6240d1eb0 msgr2=0x7fb62415d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:16.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.470+0000 7fb62b2c5640 1 --2- 172.21.15.45:0/1660514279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6240d1eb0 0x7fb62415d6c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb62000d800 tx=0x7fb62000dcc0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.470+0000 7fb62b2c5640 1 -- 172.21.15.45:0/1660514279 shutdown_connections 2024-05-08T05:42:16.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.470+0000 7fb62b2c5640 1 --2- 172.21.15.45:0/1660514279 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb6080762d0 0x7fb608078790 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.470+0000 7fb62b2c5640 1 --2- 172.21.15.45:0/1660514279 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb6240c9b50 0x7fb6240b8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.470+0000 7fb62b2c5640 1 --2- 172.21.15.45:0/1660514279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6240b8b50 0x7fb6240b5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.470+0000 7fb62b2c5640 1 --2- 172.21.15.45:0/1660514279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6240d1eb0 0x7fb62415d6c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.470+0000 7fb62b2c5640 1 -- 172.21.15.45:0/1660514279 >> 172.21.15.45:0/1660514279 conn(0x7fb62408eac0 msgr2=0x7fb6240cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:16.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.470+0000 7fb62b2c5640 1 -- 172.21.15.45:0/1660514279 shutdown_connections 2024-05-08T05:42:16.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.470+0000 7fb62b2c5640 1 -- 172.21.15.45:0/1660514279 wait complete. 2024-05-08T05:42:16.481 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-08T05:42:16.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.585+0000 7fef34938640 1 Processor -- start 2024-05-08T05:42:16.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.585+0000 7fef34938640 1 -- start start 2024-05-08T05:42:16.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.586+0000 7fef34938640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef3015f0c0 0x7fef300b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:16.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.586+0000 7fef34938640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fef301638e0 0x7fef300b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:16.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.586+0000 7fef34938640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef300b1260 con 0x7fef3006c3a0 2024-05-08T05:42:16.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.586+0000 7fef34938640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fef30163450 con 0x7fef3015f0c0 2024-05-08T05:42:16.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.586+0000 7fef34938640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fef301604b0 con 0x7fef301638e0 2024-05-08T05:42:16.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.586+0000 7fef2e59c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef3015f0c0 0x7fef300b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:16.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.586+0000 7fef2e59c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef3015f0c0 0x7fef300b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45102/0 (socket says 172.21.15.45:45102) 2024-05-08T05:42:16.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.586+0000 7fef2ed9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fef3006c3a0 0x7fef30073170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36802/0 (socket says 172.21.15.45:36802) 2024-05-08T05:42:16.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.586+0000 7fef2e59c640 1 -- 172.21.15.45:0/3588684928 learned_addr learned my addr 172.21.15.45:0/3588684928 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:16.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.587+0000 7fef2dd9b640 1 --2- 172.21.15.45:0/3588684928 >> v2:172.21.15.63:3300/0 conn(0x7fef301638e0 0x7fef300b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.587+0000 7fef2e59c640 1 -- 172.21.15.45:0/3588684928 >> v2:172.21.15.63:3300/0 conn(0x7fef301638e0 msgr2=0x7fef300b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.587+0000 7fef2e59c640 1 --2- 172.21.15.45:0/3588684928 >> v2:172.21.15.63:3300/0 conn(0x7fef301638e0 0x7fef300b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.587+0000 7fef2e59c640 1 -- 172.21.15.45:0/3588684928 >> v1:172.21.15.45:6789/0 conn(0x7fef3006c3a0 legacy=0x7fef30073170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.587+0000 7fef2e59c640 1 -- 172.21.15.45:0/3588684928 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef300b14e0 con 0x7fef3015f0c0 2024-05-08T05:42:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.587+0000 7fef2e59c640 1 --2- 172.21.15.45:0/3588684928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef3015f0c0 0x7fef300b6f40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fef3006ea70 tx=0x7fef2002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c49f35164e56581 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.588+0000 7fef2d59a640 1 -- 172.21.15.45:0/3588684928 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20037070 con 0x7fef3015f0c0 2024-05-08T05:42:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.588+0000 7fef2d59a640 1 -- 172.21.15.45:0/3588684928 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef20032440 con 0x7fef3015f0c0 2024-05-08T05:42:16.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.588+0000 7fef2d59a640 1 -- 172.21.15.45:0/3588684928 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20032810 con 0x7fef3015f0c0 2024-05-08T05:42:16.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.588+0000 7fef34938640 1 -- 172.21.15.45:0/3588684928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef3015f0c0 msgr2=0x7fef300b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:16.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.588+0000 7fef34938640 1 --2- 172.21.15.45:0/3588684928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef3015f0c0 0x7fef300b6f40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fef3006ea70 tx=0x7fef2002a270 comp rx=0 tx=0).stop 2024-05-08T05:42:16.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.588+0000 7fef34938640 1 -- 172.21.15.45:0/3588684928 shutdown_connections 2024-05-08T05:42:16.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.588+0000 7fef34938640 1 --2- 172.21.15.45:0/3588684928 >> v2:172.21.15.63:3300/0 conn(0x7fef301638e0 0x7fef300b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.588+0000 7fef34938640 1 --2- 172.21.15.45:0/3588684928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef3015f0c0 0x7fef300b6f40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.588+0000 7fef34938640 1 -- 172.21.15.45:0/3588684928 >> 172.21.15.45:0/3588684928 conn(0x7fef300b3b10 msgr2=0x7fef300b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:16.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.588+0000 7fef34938640 1 -- 172.21.15.45:0/3588684928 shutdown_connections 2024-05-08T05:42:16.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.588+0000 7fef34938640 1 -- 172.21.15.45:0/3588684928 wait complete. 2024-05-08T05:42:16.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.590+0000 7fef34938640 1 Processor -- start 2024-05-08T05:42:16.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.590+0000 7fef34938640 1 -- start start 2024-05-08T05:42:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.590+0000 7fef34938640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef3006c3a0 0x7fef301f8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.590+0000 7fef34938640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef3015f0c0 0x7fef3015cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.590+0000 7fef34938640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef301638e0 0x7fef300c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.590+0000 7fef34938640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fef300b17c0 con 0x7fef3015f0c0 2024-05-08T05:42:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.590+0000 7fef34938640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef301630e0 con 0x7fef3006c3a0 2024-05-08T05:42:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.590+0000 7fef34938640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fef3006c7e0 con 0x7fef301638e0 2024-05-08T05:42:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.590+0000 7fef2dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef3015f0c0 0x7fef3015cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.591+0000 7fef2dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef3015f0c0 0x7fef3015cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45114/0 (socket says 172.21.15.45:45114) 2024-05-08T05:42:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.591+0000 7fef2dd9b640 1 -- 172.21.15.45:0/2026837570 learned_addr learned my addr 172.21.15.45:0/2026837570 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.591+0000 7fef2e59c640 1 --2- 172.21.15.45:0/2026837570 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef3006c3a0 0x7fef301f8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:16.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.591+0000 7fef2ed9d640 1 --2- 172.21.15.45:0/2026837570 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef301638e0 0x7fef300c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.591+0000 7fef2dd9b640 1 -- 172.21.15.45:0/2026837570 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef301638e0 msgr2=0x7fef300c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.591+0000 7fef2dd9b640 1 --2- 172.21.15.45:0/2026837570 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef301638e0 0x7fef300c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.591+0000 7fef2dd9b640 1 -- 172.21.15.45:0/2026837570 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef3006c3a0 msgr2=0x7fef301f8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.591+0000 7fef2dd9b640 1 --2- 172.21.15.45:0/2026837570 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef3006c3a0 0x7fef301f8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.591+0000 7fef2dd9b640 1 -- 172.21.15.45:0/2026837570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef301f94f0 con 0x7fef3015f0c0 2024-05-08T05:42:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.591+0000 7fef2e59c640 1 --2- 172.21.15.45:0/2026837570 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef3006c3a0 0x7fef301f8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.591+0000 7fef2dd9b640 1 --2- 172.21.15.45:0/2026837570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef3015f0c0 0x7fef3015cd80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fef2400d620 tx=0x7fef2400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.591+0000 7fef1effd640 1 -- 172.21.15.45:0/2026837570 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef240125f0 con 0x7fef3015f0c0 2024-05-08T05:42:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.591+0000 7fef1effd640 1 -- 172.21.15.45:0/2026837570 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef24019070 con 0x7fef3015f0c0 2024-05-08T05:42:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.591+0000 7fef34938640 1 -- 172.21.15.45:0/2026837570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef300c70b0 con 0x7fef3015f0c0 2024-05-08T05:42:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.591+0000 7fef1effd640 1 -- 172.21.15.45:0/2026837570 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef240129d0 con 0x7fef3015f0c0 2024-05-08T05:42:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.591+0000 7fef34938640 1 -- 172.21.15.45:0/2026837570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef300c7670 con 0x7fef3015f0c0 2024-05-08T05:42:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.592+0000 7fef34938640 1 -- 172.21.15.45:0/2026837570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef300693f0 con 0x7fef3015f0c0 2024-05-08T05:42:16.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.593+0000 7fef1effd640 1 -- 172.21.15.45:0/2026837570 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fef24012b70 con 0x7fef3015f0c0 2024-05-08T05:42:16.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.593+0000 7fef1effd640 1 --2- 172.21.15.45:0/2026837570 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef100762d0 0x7fef10078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:16.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.593+0000 7fef1effd640 1 -- 172.21.15.45:0/2026837570 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5249+0+0 (secure 0 0 0) 0x7fef240978c0 con 0x7fef3015f0c0 2024-05-08T05:42:16.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.594+0000 7fef2e59c640 1 --2- 172.21.15.45:0/2026837570 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef100762d0 0x7fef10078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:16.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.594+0000 7fef2e59c640 1 --2- 172.21.15.45:0/2026837570 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef100762d0 0x7fef10078790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fef20034000 tx=0x7fef2002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:16.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.598+0000 7fef1effd640 1 -- 172.21.15.45:0/2026837570 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fef301f94f0 con 0x7fef3015f0c0 2024-05-08T05:42:16.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.862+0000 7fef34938640 1 -- 172.21.15.45:0/2026837570 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fef300d2b10 con 0x7fef3015f0c0 2024-05-08T05:42:16.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.863+0000 7fef1effd640 1 -- 172.21.15.45:0/2026837570 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7fef24061340 con 0x7fef3015f0c0 2024-05-08T05:42:16.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.865+0000 7fef34938640 1 -- 172.21.15.45:0/2026837570 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef100762d0 msgr2=0x7fef10078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:16.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.865+0000 7fef34938640 1 --2- 172.21.15.45:0/2026837570 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef100762d0 0x7fef10078790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fef20034000 tx=0x7fef2002e000 comp rx=0 tx=0).stop 2024-05-08T05:42:16.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.865+0000 7fef34938640 1 -- 172.21.15.45:0/2026837570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef3015f0c0 msgr2=0x7fef3015cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:16.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.865+0000 7fef34938640 1 --2- 172.21.15.45:0/2026837570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef3015f0c0 0x7fef3015cd80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fef2400d620 tx=0x7fef2400dae0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.865+0000 7fef34938640 1 -- 172.21.15.45:0/2026837570 shutdown_connections 2024-05-08T05:42:16.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.865+0000 7fef34938640 1 --2- 172.21.15.45:0/2026837570 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef100762d0 0x7fef10078790 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.865+0000 7fef34938640 1 --2- 172.21.15.45:0/2026837570 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef301638e0 0x7fef300c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.865+0000 7fef34938640 1 --2- 172.21.15.45:0/2026837570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef3015f0c0 0x7fef3015cd80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.865+0000 7fef34938640 1 --2- 172.21.15.45:0/2026837570 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef3006c3a0 0x7fef301f8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.865+0000 7fef34938640 1 -- 172.21.15.45:0/2026837570 >> 172.21.15.45:0/2026837570 conn(0x7fef300b3b10 msgr2=0x7fef30075840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:16.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.866+0000 7fef34938640 1 -- 172.21.15.45:0/2026837570 shutdown_connections 2024-05-08T05:42:16.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.866+0000 7fef34938640 1 -- 172.21.15.45:0/2026837570 wait complete. 2024-05-08T05:42:16.877 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-08T05:42:16.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.981+0000 7fa97f7da640 1 Processor -- start 2024-05-08T05:42:16.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.981+0000 7fa97f7da640 1 -- start start 2024-05-08T05:42:16.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.982+0000 7fa97f7da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9780d0ec0 0x7fa9780d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:16.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.982+0000 7fa97f7da640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa9780d3b00 0x7fa9780b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:16.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.982+0000 7fa97f7da640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9780d1760 con 0x7fa97815f380 2024-05-08T05:42:16.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.982+0000 7fa97f7da640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa9780d30f0 con 0x7fa9780d3b00 2024-05-08T05:42:16.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.982+0000 7fa97f7da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa9780d2370 con 0x7fa9780d0ec0 2024-05-08T05:42:16.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.983+0000 7fa97cd75640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9780d0ec0 0x7fa9780d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:16.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.983+0000 7fa97d576640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fa97815f380 0x7fa9780d07b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:36694/0 (socket says 172.21.15.45:36694) 2024-05-08T05:42:16.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.983+0000 7fa97dd77640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa9780d3b00 0x7fa9780b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:16.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.983+0000 7fa97d576640 1 -- 172.21.15.45:0/4028477857 learned_addr learned my addr 172.21.15.45:0/4028477857 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:16.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.983+0000 7fa97cd75640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9780d0ec0 0x7fa9780d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:40978/0 (socket says 172.21.15.45:40978) 2024-05-08T05:42:16.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.983+0000 7fa977fff640 1 -- 172.21.15.45:0/4028477857 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2573178983 0 0) 0x7fa9780d1760 con 0x7fa97815f380 2024-05-08T05:42:16.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.983+0000 7fa977fff640 1 -- 172.21.15.45:0/4028477857 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa960003520 con 0x7fa97815f380 2024-05-08T05:42:16.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.983+0000 7fa97cd75640 1 -- 172.21.15.45:0/4028477857 >> v2:172.21.15.63:3300/0 conn(0x7fa9780d3b00 msgr2=0x7fa9780b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:16.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.983+0000 7fa97cd75640 1 --2- 172.21.15.45:0/4028477857 >> v2:172.21.15.63:3300/0 conn(0x7fa9780d3b00 0x7fa9780b7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.983+0000 7fa97cd75640 1 -- 172.21.15.45:0/4028477857 >> v1:172.21.15.63:6789/0 conn(0x7fa97815f380 legacy=0x7fa9780d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:16.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.983+0000 7fa97cd75640 1 -- 172.21.15.45:0/4028477857 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9780d1a40 con 0x7fa9780d0ec0 2024-05-08T05:42:16.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.983+0000 7fa97dd77640 1 --2- 172.21.15.45:0/4028477857 >> v2:172.21.15.63:3300/0 conn(0x7fa9780d3b00 0x7fa9780b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:42:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.984+0000 7fa97cd75640 1 --2- 172.21.15.45:0/4028477857 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9780d0ec0 0x7fa9780d35c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0097a0 tx=0x7fa96c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df8ec47eadd74cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.984+0000 7fa977fff640 1 -- 172.21.15.45:0/4028477857 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c037070 con 0x7fa9780d0ec0 2024-05-08T05:42:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.984+0000 7fa977fff640 1 -- 172.21.15.45:0/4028477857 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa96c032440 con 0x7fa9780d0ec0 2024-05-08T05:42:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.984+0000 7fa977fff640 1 -- 172.21.15.45:0/4028477857 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c032810 con 0x7fa9780d0ec0 2024-05-08T05:42:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.985+0000 7fa97f7da640 1 -- 172.21.15.45:0/4028477857 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9780d0ec0 msgr2=0x7fa9780d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.985+0000 7fa97f7da640 1 --2- 172.21.15.45:0/4028477857 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9780d0ec0 0x7fa9780d35c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa96c0097a0 tx=0x7fa96c02a180 comp rx=0 tx=0).stop 2024-05-08T05:42:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.985+0000 7fa97f7da640 1 -- 172.21.15.45:0/4028477857 shutdown_connections 2024-05-08T05:42:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.985+0000 7fa97f7da640 1 --2- 172.21.15.45:0/4028477857 >> v2:172.21.15.63:3300/0 conn(0x7fa9780d3b00 0x7fa9780b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.985+0000 7fa97f7da640 1 --2- 172.21.15.45:0/4028477857 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa9780d0ec0 0x7fa9780d35c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.985+0000 7fa97f7da640 1 -- 172.21.15.45:0/4028477857 >> 172.21.15.45:0/4028477857 conn(0x7fa9780d4fe0 msgr2=0x7fa9780d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.985+0000 7fa97f7da640 1 -- 172.21.15.45:0/4028477857 shutdown_connections 2024-05-08T05:42:16.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.985+0000 7fa97f7da640 1 -- 172.21.15.45:0/4028477857 wait complete. 2024-05-08T05:42:16.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.986+0000 7fa97f7da640 1 Processor -- start 2024-05-08T05:42:16.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.986+0000 7fa97f7da640 1 -- start start 2024-05-08T05:42:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.987+0000 7fa97f7da640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9780d0ec0 0x7fa97815de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.987+0000 7fa97f7da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9780d3b00 0x7fa97806df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.987+0000 7fa97f7da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa97815f380 0x7fa97806e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.987+0000 7fa97f7da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa97815f760 con 0x7fa9780d3b00 2024-05-08T05:42:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.987+0000 7fa97f7da640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa9780d2000 con 0x7fa9780d0ec0 2024-05-08T05:42:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.987+0000 7fa97f7da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa9780d2800 con 0x7fa97815f380 2024-05-08T05:42:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.987+0000 7fa97cd75640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9780d3b00 0x7fa97806df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.987+0000 7fa97cd75640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9780d3b00 0x7fa97806df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45120/0 (socket says 172.21.15.45:45120) 2024-05-08T05:42:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.987+0000 7fa97cd75640 1 -- 172.21.15.45:0/2156839887 learned_addr learned my addr 172.21.15.45:0/2156839887 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.987+0000 7fa97d576640 1 --2- 172.21.15.45:0/2156839887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9780d0ec0 0x7fa97815de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.987+0000 7fa97dd77640 1 --2- 172.21.15.45:0/2156839887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa97815f380 0x7fa97806e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.987+0000 7fa97cd75640 1 -- 172.21.15.45:0/2156839887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa97815f380 msgr2=0x7fa97806e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:16.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.987+0000 7fa97cd75640 1 --2- 172.21.15.45:0/2156839887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa97815f380 0x7fa97806e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.987+0000 7fa97cd75640 1 -- 172.21.15.45:0/2156839887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9780d0ec0 msgr2=0x7fa97815de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.987+0000 7fa97cd75640 1 --2- 172.21.15.45:0/2156839887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9780d0ec0 0x7fa97815de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.987+0000 7fa97cd75640 1 -- 172.21.15.45:0/2156839887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97815e3b0 con 0x7fa9780d3b00 2024-05-08T05:42:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.987+0000 7fa97cd75640 1 --2- 172.21.15.45:0/2156839887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9780d3b00 0x7fa97806df70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa96c035040 tx=0x7fa96c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.988+0000 7fa975ffb640 1 -- 172.21.15.45:0/2156839887 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c041030 con 0x7fa9780d3b00 2024-05-08T05:42:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.988+0000 7fa975ffb640 1 -- 172.21.15.45:0/2156839887 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa96c03d420 con 0x7fa9780d3b00 2024-05-08T05:42:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.988+0000 7fa975ffb640 1 -- 172.21.15.45:0/2156839887 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c037040 con 0x7fa9780d3b00 2024-05-08T05:42:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.988+0000 7fa97f7da640 1 -- 172.21.15.45:0/2156839887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa978161140 con 0x7fa9780d3b00 2024-05-08T05:42:16.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.988+0000 7fa97f7da640 1 -- 172.21.15.45:0/2156839887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa978161700 con 0x7fa9780d3b00 2024-05-08T05:42:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.989+0000 7fa97f7da640 1 -- 172.21.15.45:0/2156839887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9780693f0 con 0x7fa9780d3b00 2024-05-08T05:42:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.990+0000 7fa975ffb640 1 -- 172.21.15.45:0/2156839887 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa96c044070 con 0x7fa9780d3b00 2024-05-08T05:42:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.990+0000 7fa975ffb640 1 --2- 172.21.15.45:0/2156839887 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa964076300 0x7fa9640787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:16.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.990+0000 7fa975ffb640 1 -- 172.21.15.45:0/2156839887 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5249+0+0 (secure 0 0 0) 0x7fa96c0b7990 con 0x7fa9780d3b00 2024-05-08T05:42:16.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.990+0000 7fa97d576640 1 --2- 172.21.15.45:0/2156839887 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa964076300 0x7fa9640787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:16.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.991+0000 7fa97d576640 1 --2- 172.21.15.45:0/2156839887 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa964076300 0x7fa9640787c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa970005880 tx=0x7fa970006aa0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:16.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:16.994+0000 7fa975ffb640 1 -- 172.21.15.45:0/2156839887 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa96c081420 con 0x7fa9780d3b00 2024-05-08T05:42:17.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.263+0000 7fa97f7da640 1 -- 172.21.15.45:0/2156839887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fa97808cfd0 con 0x7fa9780d3b00 2024-05-08T05:42:17.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.265+0000 7fa975ffb640 1 -- 172.21.15.45:0/2156839887 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fa96c080d20 con 0x7fa9780d3b00 2024-05-08T05:42:17.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.267+0000 7fa97f7da640 1 -- 172.21.15.45:0/2156839887 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa964076300 msgr2=0x7fa9640787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:17.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.267+0000 7fa97f7da640 1 --2- 172.21.15.45:0/2156839887 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa964076300 0x7fa9640787c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa970005880 tx=0x7fa970006aa0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.267+0000 7fa97f7da640 1 -- 172.21.15.45:0/2156839887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9780d3b00 msgr2=0x7fa97806df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:17.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.267+0000 7fa97f7da640 1 --2- 172.21.15.45:0/2156839887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9780d3b00 0x7fa97806df70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa96c035040 tx=0x7fa96c0046c0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.267+0000 7fa97f7da640 1 -- 172.21.15.45:0/2156839887 shutdown_connections 2024-05-08T05:42:17.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.267+0000 7fa97f7da640 1 --2- 172.21.15.45:0/2156839887 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa964076300 0x7fa9640787c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.267+0000 7fa97f7da640 1 --2- 172.21.15.45:0/2156839887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa97815f380 0x7fa97806e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.267+0000 7fa97f7da640 1 --2- 172.21.15.45:0/2156839887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9780d3b00 0x7fa97806df70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.267+0000 7fa97f7da640 1 --2- 172.21.15.45:0/2156839887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa9780d0ec0 0x7fa97815de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.267+0000 7fa97f7da640 1 -- 172.21.15.45:0/2156839887 >> 172.21.15.45:0/2156839887 conn(0x7fa9780d4fe0 msgr2=0x7fa9780cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:17.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.267+0000 7fa97f7da640 1 -- 172.21.15.45:0/2156839887 shutdown_connections 2024-05-08T05:42:17.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.267+0000 7fa97f7da640 1 -- 172.21.15.45:0/2156839887 wait complete. 2024-05-08T05:42:17.279 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T05:42:17.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.382+0000 7fb880b6a640 1 Processor -- start 2024-05-08T05:42:17.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.382+0000 7fb880b6a640 1 -- start start 2024-05-08T05:42:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.383+0000 7fb880b6a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb87c06e0f0 0x7fb87c15c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.383+0000 7fb880b6a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb87c072840 0x7fb87c0d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.383+0000 7fb880b6a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb87c0d14b0 con 0x7fb87c06e4d0 2024-05-08T05:42:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.383+0000 7fb880b6a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb87c15c920 con 0x7fb87c06e0f0 2024-05-08T05:42:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.383+0000 7fb880b6a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb87c15cc00 con 0x7fb87c072840 2024-05-08T05:42:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.383+0000 7fb87a59c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb87c06e0f0 0x7fb87c15c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.383+0000 7fb87a59c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb87c06e0f0 0x7fb87c15c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45126/0 (socket says 172.21.15.45:45126) 2024-05-08T05:42:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.383+0000 7fb87a59c640 1 -- 172.21.15.45:0/3894772178 learned_addr learned my addr 172.21.15.45:0/3894772178 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:17.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.383+0000 7fb879d9b640 1 --2- 172.21.15.45:0/3894772178 >> v2:172.21.15.63:3300/0 conn(0x7fb87c072840 0x7fb87c0d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.383+0000 7fb87a59c640 1 -- 172.21.15.45:0/3894772178 >> v2:172.21.15.63:3300/0 conn(0x7fb87c072840 msgr2=0x7fb87c0d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.383+0000 7fb87a59c640 1 --2- 172.21.15.45:0/3894772178 >> v2:172.21.15.63:3300/0 conn(0x7fb87c072840 0x7fb87c0d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.383+0000 7fb87a59c640 1 -- 172.21.15.45:0/3894772178 >> v1:172.21.15.63:6789/0 conn(0x7fb87c06e4d0 legacy=0x7fb87c0b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.383+0000 7fb87a59c640 1 -- 172.21.15.45:0/3894772178 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87c15cee0 con 0x7fb87c06e0f0 2024-05-08T05:42:17.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.383+0000 7fb879d9b640 1 --2- 172.21.15.45:0/3894772178 >> v2:172.21.15.63:3300/0 conn(0x7fb87c072840 0x7fb87c0d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.384+0000 7fb87a59c640 1 --2- 172.21.15.45:0/3894772178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb87c06e0f0 0x7fb87c15c340 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0727d0 tx=0x7fb86c02c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b04ad68543277e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.384+0000 7fb87959a640 1 -- 172.21.15.45:0/3894772178 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c036020 con 0x7fb87c06e0f0 2024-05-08T05:42:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.384+0000 7fb87959a640 1 -- 172.21.15.45:0/3894772178 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb86c032640 con 0x7fb87c06e0f0 2024-05-08T05:42:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.384+0000 7fb87959a640 1 -- 172.21.15.45:0/3894772178 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c032a10 con 0x7fb87c06e0f0 2024-05-08T05:42:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.385+0000 7fb880b6a640 1 -- 172.21.15.45:0/3894772178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb87c06e0f0 msgr2=0x7fb87c15c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.385+0000 7fb880b6a640 1 --2- 172.21.15.45:0/3894772178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb87c06e0f0 0x7fb87c15c340 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0727d0 tx=0x7fb86c02c940 comp rx=0 tx=0).stop 2024-05-08T05:42:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.385+0000 7fb880b6a640 1 -- 172.21.15.45:0/3894772178 shutdown_connections 2024-05-08T05:42:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.385+0000 7fb880b6a640 1 --2- 172.21.15.45:0/3894772178 >> v2:172.21.15.63:3300/0 conn(0x7fb87c072840 0x7fb87c0d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.385+0000 7fb880b6a640 1 --2- 172.21.15.45:0/3894772178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb87c06e0f0 0x7fb87c15c340 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.385+0000 7fb880b6a640 1 -- 172.21.15.45:0/3894772178 >> 172.21.15.45:0/3894772178 conn(0x7fb87c06c3a0 msgr2=0x7fb87c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.385+0000 7fb880b6a640 1 -- 172.21.15.45:0/3894772178 shutdown_connections 2024-05-08T05:42:17.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.385+0000 7fb880b6a640 1 -- 172.21.15.45:0/3894772178 wait complete. 2024-05-08T05:42:17.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.386+0000 7fb880b6a640 1 Processor -- start 2024-05-08T05:42:17.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.387+0000 7fb880b6a640 1 -- start start 2024-05-08T05:42:17.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.387+0000 7fb880b6a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb87c06e0f0 0x7fb87c1e4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:17.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.387+0000 7fb880b6a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb87c06e4d0 0x7fb87c1e5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:17.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.387+0000 7fb880b6a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb87c072840 0x7fb87c0ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.387+0000 7fb880b6a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c0ae650 con 0x7fb87c072840 2024-05-08T05:42:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.387+0000 7fb880b6a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c06e960 con 0x7fb87c06e4d0 2024-05-08T05:42:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.387+0000 7fb880b6a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb87c0ae7d0 con 0x7fb87c06e0f0 2024-05-08T05:42:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.387+0000 7fb87ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb87c072840 0x7fb87c0ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.387+0000 7fb87ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb87c072840 0x7fb87c0ceb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45130/0 (socket says 172.21.15.45:45130) 2024-05-08T05:42:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.388+0000 7fb87ad9d640 1 -- 172.21.15.45:0/2410083882 learned_addr learned my addr 172.21.15.45:0/2410083882 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.388+0000 7fb87a59c640 1 --2- 172.21.15.45:0/2410083882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb87c06e0f0 0x7fb87c1e4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.388+0000 7fb879d9b640 1 --2- 172.21.15.45:0/2410083882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb87c06e4d0 0x7fb87c1e5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.388+0000 7fb87ad9d640 1 -- 172.21.15.45:0/2410083882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb87c06e0f0 msgr2=0x7fb87c1e4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.388+0000 7fb87ad9d640 1 --2- 172.21.15.45:0/2410083882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb87c06e0f0 0x7fb87c1e4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.388+0000 7fb87ad9d640 1 -- 172.21.15.45:0/2410083882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb87c06e4d0 msgr2=0x7fb87c1e5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.388+0000 7fb87ad9d640 1 --2- 172.21.15.45:0/2410083882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb87c06e4d0 0x7fb87c1e5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.388+0000 7fb87ad9d640 1 -- 172.21.15.45:0/2410083882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87c1e4c40 con 0x7fb87c072840 2024-05-08T05:42:17.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.388+0000 7fb87ad9d640 1 --2- 172.21.15.45:0/2410083882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb87c072840 0x7fb87c0ceb70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb87400c7a0 tx=0x7fb87400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.388+0000 7fb86affd640 1 -- 172.21.15.45:0/2410083882 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb874003b70 con 0x7fb87c072840 2024-05-08T05:42:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.388+0000 7fb86affd640 1 -- 172.21.15.45:0/2410083882 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb874003d30 con 0x7fb87c072840 2024-05-08T05:42:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.388+0000 7fb880b6a640 1 -- 172.21.15.45:0/2410083882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb87c1e4f20 con 0x7fb87c072840 2024-05-08T05:42:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.388+0000 7fb86affd640 1 -- 172.21.15.45:0/2410083882 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87400aa90 con 0x7fb87c072840 2024-05-08T05:42:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.388+0000 7fb880b6a640 1 -- 172.21.15.45:0/2410083882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb87c1e54e0 con 0x7fb87c072840 2024-05-08T05:42:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.389+0000 7fb880b6a640 1 -- 172.21.15.45:0/2410083882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb87c0693f0 con 0x7fb87c072840 2024-05-08T05:42:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.390+0000 7fb86affd640 1 -- 172.21.15.45:0/2410083882 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb87400ac30 con 0x7fb87c072840 2024-05-08T05:42:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.390+0000 7fb86affd640 1 --2- 172.21.15.45:0/2410083882 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb85c076300 0x7fb85c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.390+0000 7fb86affd640 1 -- 172.21.15.45:0/2410083882 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5249+0+0 (secure 0 0 0) 0x7fb874097190 con 0x7fb87c072840 2024-05-08T05:42:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.390+0000 7fb87a59c640 1 --2- 172.21.15.45:0/2410083882 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb85c076300 0x7fb85c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:17.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.391+0000 7fb87a59c640 1 --2- 172.21.15.45:0/2410083882 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb85c076300 0x7fb85c0787c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb86c037000 tx=0x7fb86c03d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:17.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.394+0000 7fb86affd640 1 -- 172.21.15.45:0/2410083882 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb8740606f0 con 0x7fb87c072840 2024-05-08T05:42:17.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.586+0000 7fb880b6a640 1 -- 172.21.15.45:0/2410083882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb87c0ccfb0 con 0x7fb87c072840 2024-05-08T05:42:17.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.586+0000 7fb86affd640 1 -- 172.21.15.45:0/2410083882 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) ==== 84+0+65 (secure 0 0 0) 0x7fb8740649f0 con 0x7fb87c072840 2024-05-08T05:42:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:17.587 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-05-08T05:42:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-05-08T05:42:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-05-08T05:42:17.587 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:17.587 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-05-08T05:42:17.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.588+0000 7fb880b6a640 1 -- 172.21.15.45:0/2410083882 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb85c076300 msgr2=0x7fb85c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:17.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.588+0000 7fb880b6a640 1 --2- 172.21.15.45:0/2410083882 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb85c076300 0x7fb85c0787c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb86c037000 tx=0x7fb86c03d040 comp rx=0 tx=0).stop 2024-05-08T05:42:17.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.588+0000 7fb880b6a640 1 -- 172.21.15.45:0/2410083882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb87c072840 msgr2=0x7fb87c0ceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:17.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.588+0000 7fb880b6a640 1 --2- 172.21.15.45:0/2410083882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb87c072840 0x7fb87c0ceb70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb87400c7a0 tx=0x7fb87400cc60 comp rx=0 tx=0).stop 2024-05-08T05:42:17.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.588+0000 7fb880b6a640 1 -- 172.21.15.45:0/2410083882 shutdown_connections 2024-05-08T05:42:17.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.588+0000 7fb880b6a640 1 --2- 172.21.15.45:0/2410083882 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb85c076300 0x7fb85c0787c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.588+0000 7fb880b6a640 1 --2- 172.21.15.45:0/2410083882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb87c072840 0x7fb87c0ceb70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.588+0000 7fb880b6a640 1 --2- 172.21.15.45:0/2410083882 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb87c06e4d0 0x7fb87c1e5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.588+0000 7fb880b6a640 1 --2- 172.21.15.45:0/2410083882 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb87c06e0f0 0x7fb87c1e4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.588+0000 7fb880b6a640 1 -- 172.21.15.45:0/2410083882 >> 172.21.15.45:0/2410083882 conn(0x7fb87c06c3a0 msgr2=0x7fb87c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:17.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.588+0000 7fb880b6a640 1 -- 172.21.15.45:0/2410083882 shutdown_connections 2024-05-08T05:42:17.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.588+0000 7fb880b6a640 1 -- 172.21.15.45:0/2410083882 wait complete. 2024-05-08T05:42:17.599 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T05:42:17.600 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T05:42:17.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.707+0000 7f9c05946640 1 Processor -- start 2024-05-08T05:42:17.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.707+0000 7f9c05946640 1 -- start start 2024-05-08T05:42:17.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.708+0000 7f9c05946640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9c0015d3c0 0x7f9c000d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:17.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.708+0000 7f9c05946640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9c000cfe50 0x7f9c000d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:17.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.708+0000 7f9c05946640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c000d18a0 0x7f9c000cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:17.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.708+0000 7f9c05946640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f9c000b2e60 con 0x7f9c000cfe50 2024-05-08T05:42:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.708+0000 7f9c05946640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9c000d0b60 con 0x7f9c0015d3c0 2024-05-08T05:42:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.708+0000 7f9c05946640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9c000d0e40 con 0x7f9c000d18a0 2024-05-08T05:42:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.708+0000 7f9bf67fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9c000cfe50 0x7f9c000d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.708+0000 7f9bf67fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9c000cfe50 0x7f9c000d1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45134/0 (socket says 172.21.15.45:45134) 2024-05-08T05:42:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.708+0000 7f9bf67fc640 1 -- 172.21.15.45:0/92601383 learned_addr learned my addr 172.21.15.45:0/92601383 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.708+0000 7f9bfeffd640 1 --2- 172.21.15.45:0/92601383 >> v2:172.21.15.63:3300/0 conn(0x7f9c0015d3c0 0x7f9c000d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.708+0000 7f9bff7fe640 1 --2- 172.21.15.45:0/92601383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c000d18a0 0x7f9c000cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.708+0000 7f9bf67fc640 1 -- 172.21.15.45:0/92601383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c000d18a0 msgr2=0x7f9c000cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.708+0000 7f9bf67fc640 1 --2- 172.21.15.45:0/92601383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c000d18a0 0x7f9c000cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.708+0000 7f9bf67fc640 1 -- 172.21.15.45:0/92601383 >> v2:172.21.15.63:3300/0 conn(0x7f9c0015d3c0 msgr2=0x7f9c000d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.708+0000 7f9bf67fc640 1 --2- 172.21.15.45:0/92601383 >> v2:172.21.15.63:3300/0 conn(0x7f9c0015d3c0 0x7f9c000d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.708+0000 7f9bf67fc640 1 -- 172.21.15.45:0/92601383 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c000b2810 con 0x7f9c000cfe50 2024-05-08T05:42:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.708+0000 7f9bfeffd640 1 --2- 172.21.15.45:0/92601383 >> v2:172.21.15.63:3300/0 conn(0x7f9c0015d3c0 0x7f9c000d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.708+0000 7f9bff7fe640 1 --2- 172.21.15.45:0/92601383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c000d18a0 0x7f9c000cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.709+0000 7f9bf67fc640 1 --2- 172.21.15.45:0/92601383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c000cfe50 0x7f9c000d1360 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9c00068c10 tx=0x7f9be802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a18a3de0706935 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.709+0000 7f9bfe7fc640 1 -- 172.21.15.45:0/92601383 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8037070 con 0x7f9c000cfe50 2024-05-08T05:42:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.709+0000 7f9bfe7fc640 1 -- 172.21.15.45:0/92601383 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be8032440 con 0x7f9c000cfe50 2024-05-08T05:42:17.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.709+0000 7f9bfe7fc640 1 -- 172.21.15.45:0/92601383 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8032810 con 0x7f9c000cfe50 2024-05-08T05:42:17.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.709+0000 7f9c05946640 1 -- 172.21.15.45:0/92601383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c000cfe50 msgr2=0x7f9c000d1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.709+0000 7f9c05946640 1 --2- 172.21.15.45:0/92601383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c000cfe50 0x7f9c000d1360 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9c00068c10 tx=0x7f9be802a1e0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.710+0000 7f9c05946640 1 -- 172.21.15.45:0/92601383 shutdown_connections 2024-05-08T05:42:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.710+0000 7f9c05946640 1 --2- 172.21.15.45:0/92601383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c000d18a0 0x7f9c000cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.710+0000 7f9c05946640 1 --2- 172.21.15.45:0/92601383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c000cfe50 0x7f9c000d1360 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.710+0000 7f9c05946640 1 --2- 172.21.15.45:0/92601383 >> v2:172.21.15.63:3300/0 conn(0x7f9c0015d3c0 0x7f9c000d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.710+0000 7f9c05946640 1 -- 172.21.15.45:0/92601383 >> 172.21.15.45:0/92601383 conn(0x7f9c000d2d80 msgr2=0x7f9c000d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.710+0000 7f9c05946640 1 -- 172.21.15.45:0/92601383 shutdown_connections 2024-05-08T05:42:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.710+0000 7f9c05946640 1 -- 172.21.15.45:0/92601383 wait complete. 2024-05-08T05:42:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.711+0000 7f9c05946640 1 Processor -- start 2024-05-08T05:42:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.711+0000 7f9c05946640 1 -- start start 2024-05-08T05:42:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.712+0000 7f9c05946640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c000cfe50 0x7f9c0014c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.712+0000 7f9c05946640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c000d18a0 0x7f9c000b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.712+0000 7f9c05946640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c0015d3c0 0x7f9c000c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:17.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.712+0000 7f9c05946640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9c000ce130 con 0x7f9c0015d3c0 2024-05-08T05:42:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.712+0000 7f9c05946640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9c000cdfb0 con 0x7f9c000cfe50 2024-05-08T05:42:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.712+0000 7f9c05946640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9c000ce2b0 con 0x7f9c000d18a0 2024-05-08T05:42:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.712+0000 7f9bff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c0015d3c0 0x7f9c000c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.712+0000 7f9bff7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c0015d3c0 0x7f9c000c66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45136/0 (socket says 172.21.15.45:45136) 2024-05-08T05:42:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.712+0000 7f9bff7fe640 1 -- 172.21.15.45:0/2275031425 learned_addr learned my addr 172.21.15.45:0/2275031425 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.712+0000 7f9bf67fc640 1 --2- 172.21.15.45:0/2275031425 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c000d18a0 0x7f9c000b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.712+0000 7f9bfeffd640 1 --2- 172.21.15.45:0/2275031425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c000cfe50 0x7f9c0014c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.712+0000 7f9bff7fe640 1 -- 172.21.15.45:0/2275031425 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c000d18a0 msgr2=0x7f9c000b6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.712+0000 7f9bff7fe640 1 --2- 172.21.15.45:0/2275031425 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c000d18a0 0x7f9c000b6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.712+0000 7f9bff7fe640 1 -- 172.21.15.45:0/2275031425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c000cfe50 msgr2=0x7f9c0014c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.712+0000 7f9bff7fe640 1 --2- 172.21.15.45:0/2275031425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c000cfe50 0x7f9c0014c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.712+0000 7f9bff7fe640 1 -- 172.21.15.45:0/2275031425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c000b78c0 con 0x7f9c0015d3c0 2024-05-08T05:42:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.712+0000 7f9bfeffd640 1 --2- 172.21.15.45:0/2275031425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c000cfe50 0x7f9c0014c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:17.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.712+0000 7f9bff7fe640 1 --2- 172.21.15.45:0/2275031425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c0015d3c0 0x7f9c000c66e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8007e40 tx=0x7f9bf800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.712+0000 7f9bfcff9640 1 -- 172.21.15.45:0/2275031425 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf801a070 con 0x7f9c0015d3c0 2024-05-08T05:42:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.713+0000 7f9bfcff9640 1 -- 172.21.15.45:0/2275031425 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf8003b90 con 0x7f9c0015d3c0 2024-05-08T05:42:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.713+0000 7f9bfcff9640 1 -- 172.21.15.45:0/2275031425 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf800de80 con 0x7f9c0015d3c0 2024-05-08T05:42:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.713+0000 7f9c05946640 1 -- 172.21.15.45:0/2275031425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c000b7ba0 con 0x7f9c0015d3c0 2024-05-08T05:42:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.713+0000 7f9c05946640 1 -- 172.21.15.45:0/2275031425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c000b8160 con 0x7f9c0015d3c0 2024-05-08T05:42:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.714+0000 7f9c05946640 1 -- 172.21.15.45:0/2275031425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c000693f0 con 0x7f9c0015d3c0 2024-05-08T05:42:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.714+0000 7f9bfcff9640 1 -- 172.21.15.45:0/2275031425 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9bf801d070 con 0x7f9c0015d3c0 2024-05-08T05:42:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.715+0000 7f9bfcff9640 1 --2- 172.21.15.45:0/2275031425 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9be00762d0 0x7f9be0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.715+0000 7f9bfcff9640 1 -- 172.21.15.45:0/2275031425 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5249+0+0 (secure 0 0 0) 0x7f9bf8097660 con 0x7f9c0015d3c0 2024-05-08T05:42:17.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.715+0000 7f9bfeffd640 1 --2- 172.21.15.45:0/2275031425 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9be00762d0 0x7f9be0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:17.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.716+0000 7f9bfeffd640 1 --2- 172.21.15.45:0/2275031425 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9be00762d0 0x7f9be0078790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0006fd0 tx=0x7f9bf0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:17.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.719+0000 7f9bfcff9640 1 -- 172.21.15.45:0/2275031425 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9bf8010070 con 0x7f9c0015d3c0 2024-05-08T05:42:17.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:17.913+0000 7f9c05946640 1 -- 172.21.15.45:0/2275031425 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9c00159980 con 0x7f9c0015d3c0 2024-05-08T05:42:18.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.470+0000 7f9bfcff9640 1 -- 172.21.15.45:0/2275031425 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) ==== 138+0+0 (secure 0 0 0) 0x7f9bf8060b30 con 0x7f9c0015d3c0 2024-05-08T05:42:18.471 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-05-08T05:42:18.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.474+0000 7f9c05946640 1 -- 172.21.15.45:0/2275031425 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9be00762d0 msgr2=0x7f9be0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:18.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.474+0000 7f9c05946640 1 --2- 172.21.15.45:0/2275031425 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9be00762d0 0x7f9be0078790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0006fd0 tx=0x7f9bf0008040 comp rx=0 tx=0).stop 2024-05-08T05:42:18.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.474+0000 7f9c05946640 1 -- 172.21.15.45:0/2275031425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c0015d3c0 msgr2=0x7f9c000c66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:18.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.474+0000 7f9c05946640 1 --2- 172.21.15.45:0/2275031425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c0015d3c0 0x7f9c000c66e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8007e40 tx=0x7f9bf800d4a0 comp rx=0 tx=0).stop 2024-05-08T05:42:18.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.474+0000 7f9c05946640 1 -- 172.21.15.45:0/2275031425 shutdown_connections 2024-05-08T05:42:18.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.474+0000 7f9c05946640 1 --2- 172.21.15.45:0/2275031425 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9be00762d0 0x7f9be0078790 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:18.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.474+0000 7f9c05946640 1 --2- 172.21.15.45:0/2275031425 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9c000d18a0 0x7f9c000b6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:18.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.474+0000 7f9c05946640 1 --2- 172.21.15.45:0/2275031425 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9c0015d3c0 0x7f9c000c66e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:18.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.474+0000 7f9c05946640 1 --2- 172.21.15.45:0/2275031425 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c000cfe50 0x7f9c0014c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:18.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.475+0000 7f9c05946640 1 -- 172.21.15.45:0/2275031425 >> 172.21.15.45:0/2275031425 conn(0x7f9c000d2d80 msgr2=0x7f9c000b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:18.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.475+0000 7f9c05946640 1 -- 172.21.15.45:0/2275031425 shutdown_connections 2024-05-08T05:42:18.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.475+0000 7f9c05946640 1 -- 172.21.15.45:0/2275031425 wait complete. 2024-05-08T05:42:18.492 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T05:42:18.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.609+0000 7f754134d640 1 Processor -- start 2024-05-08T05:42:18.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.609+0000 7f754134d640 1 -- start start 2024-05-08T05:42:18.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.609+0000 7f754134d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f753c0b6590 0x7f753c0d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.609+0000 7f754134d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f753c0d2bb0 0x7f753c162d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.609+0000 7f754134d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f753c0d3320 con 0x7f753c163290 2024-05-08T05:42:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.609+0000 7f754134d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f753c0d3600 con 0x7f753c0d2bb0 2024-05-08T05:42:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.609+0000 7f754134d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f753c0d38e0 con 0x7f753c0b6590 2024-05-08T05:42:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.610+0000 7f753a7fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f753c0d2bb0 0x7f753c162d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.610+0000 7f753b7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f753c163290 0x7f753c15ec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36806/0 (socket says 172.21.15.45:36806) 2024-05-08T05:42:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.610+0000 7f753b7fe640 1 -- 172.21.15.45:0/1420738322 learned_addr learned my addr 172.21.15.45:0/1420738322 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.610+0000 7f753affd640 1 --2- 172.21.15.45:0/1420738322 >> v2:172.21.15.63:3300/0 conn(0x7f753c0b6590 0x7f753c0d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.610+0000 7f753a7fc640 1 -- 172.21.15.45:0/1420738322 >> v2:172.21.15.63:3300/0 conn(0x7f753c0b6590 msgr2=0x7f753c0d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.610+0000 7f753a7fc640 1 --2- 172.21.15.45:0/1420738322 >> v2:172.21.15.63:3300/0 conn(0x7f753c0b6590 0x7f753c0d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.610+0000 7f753a7fc640 1 -- 172.21.15.45:0/1420738322 >> v1:172.21.15.45:6789/0 conn(0x7f753c163290 legacy=0x7f753c15ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.610+0000 7f7539ffb640 1 -- 172.21.15.45:0/1420738322 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2619220383 0 0) 0x7f753c0d3320 con 0x7f753c163290 2024-05-08T05:42:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.610+0000 7f753a7fc640 1 -- 172.21.15.45:0/1420738322 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753c0d3bc0 con 0x7f753c0d2bb0 2024-05-08T05:42:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.610+0000 7f753affd640 1 --2- 172.21.15.45:0/1420738322 >> v2:172.21.15.63:3300/0 conn(0x7f753c0b6590 0x7f753c0d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:18.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.611+0000 7f753a7fc640 1 --2- 172.21.15.45:0/1420738322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f753c0d2bb0 0x7f753c162d50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f753c0d4f40 tx=0x7f753002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbafbfc1aff18862 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:18.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.611+0000 7f7539ffb640 1 -- 172.21.15.45:0/1420738322 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530037070 con 0x7f753c0d2bb0 2024-05-08T05:42:18.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.611+0000 7f7539ffb640 1 -- 172.21.15.45:0/1420738322 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7530032440 con 0x7f753c0d2bb0 2024-05-08T05:42:18.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.611+0000 7f7539ffb640 1 -- 172.21.15.45:0/1420738322 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530032810 con 0x7f753c0d2bb0 2024-05-08T05:42:18.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.611+0000 7f754134d640 1 -- 172.21.15.45:0/1420738322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f753c0d2bb0 msgr2=0x7f753c162d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:18.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.611+0000 7f754134d640 1 --2- 172.21.15.45:0/1420738322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f753c0d2bb0 0x7f753c162d50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f753c0d4f40 tx=0x7f753002a1a0 comp rx=0 tx=0).stop 2024-05-08T05:42:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.611+0000 7f754134d640 1 -- 172.21.15.45:0/1420738322 shutdown_connections 2024-05-08T05:42:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.611+0000 7f754134d640 1 --2- 172.21.15.45:0/1420738322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f753c0d2bb0 0x7f753c162d50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.611+0000 7f754134d640 1 --2- 172.21.15.45:0/1420738322 >> v2:172.21.15.63:3300/0 conn(0x7f753c0b6590 0x7f753c0d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.611+0000 7f754134d640 1 -- 172.21.15.45:0/1420738322 >> 172.21.15.45:0/1420738322 conn(0x7f753c0d4fe0 msgr2=0x7f753c0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.612+0000 7f754134d640 1 -- 172.21.15.45:0/1420738322 shutdown_connections 2024-05-08T05:42:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.612+0000 7f754134d640 1 -- 172.21.15.45:0/1420738322 wait complete. 2024-05-08T05:42:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.613+0000 7f754134d640 1 Processor -- start 2024-05-08T05:42:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.613+0000 7f754134d640 1 -- start start 2024-05-08T05:42:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f754134d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f753c0b6590 0x7f753c0d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f754134d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f753c0d2bb0 0x7f753c157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f754134d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f753c163290 0x7f753c161c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f754134d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f753c15f340 con 0x7f753c0b6590 2024-05-08T05:42:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f754134d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f753c0d3010 con 0x7f753c0d2bb0 2024-05-08T05:42:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f754134d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f753c15f4c0 con 0x7f753c163290 2024-05-08T05:42:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f753affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f753c0b6590 0x7f753c0d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f753affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f753c0b6590 0x7f753c0d25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45154/0 (socket says 172.21.15.45:45154) 2024-05-08T05:42:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f753affd640 1 -- 172.21.15.45:0/2610397442 learned_addr learned my addr 172.21.15.45:0/2610397442 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f753a7fc640 1 --2- 172.21.15.45:0/2610397442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f753c0d2bb0 0x7f753c157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f753b7fe640 1 --2- 172.21.15.45:0/2610397442 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f753c163290 0x7f753c161c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f753affd640 1 -- 172.21.15.45:0/2610397442 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f753c163290 msgr2=0x7f753c161c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f753affd640 1 --2- 172.21.15.45:0/2610397442 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f753c163290 0x7f753c161c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f753affd640 1 -- 172.21.15.45:0/2610397442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f753c0d2bb0 msgr2=0x7f753c157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f753affd640 1 --2- 172.21.15.45:0/2610397442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f753c0d2bb0 0x7f753c157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f753affd640 1 -- 172.21.15.45:0/2610397442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753c0ca190 con 0x7f753c0b6590 2024-05-08T05:42:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f753b7fe640 1 --2- 172.21.15.45:0/2610397442 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f753c163290 0x7f753c161c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f753affd640 1 --2- 172.21.15.45:0/2610397442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f753c0b6590 0x7f753c0d25c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f752c00e730 tx=0x7f752c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f75237fe640 1 -- 172.21.15.45:0/2610397442 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c00f040 con 0x7f753c0b6590 2024-05-08T05:42:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f75237fe640 1 -- 172.21.15.45:0/2610397442 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f752c004230 con 0x7f753c0b6590 2024-05-08T05:42:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f754134d640 1 -- 172.21.15.45:0/2610397442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753c0ca470 con 0x7f753c0b6590 2024-05-08T05:42:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f75237fe640 1 -- 172.21.15.45:0/2610397442 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c00ee40 con 0x7f753c0b6590 2024-05-08T05:42:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.614+0000 7f754134d640 1 -- 172.21.15.45:0/2610397442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753c0caa30 con 0x7f753c0b6590 2024-05-08T05:42:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.615+0000 7f754134d640 1 -- 172.21.15.45:0/2610397442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f753c0693f0 con 0x7f753c0b6590 2024-05-08T05:42:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.616+0000 7f75237fe640 1 -- 172.21.15.45:0/2610397442 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f752c00c6d0 con 0x7f753c0b6590 2024-05-08T05:42:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.617+0000 7f75237fe640 1 --2- 172.21.15.45:0/2610397442 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f75280762d0 0x7f7528078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.617+0000 7f75237fe640 1 -- 172.21.15.45:0/2610397442 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5619+0+0 (secure 0 0 0) 0x7f752c097240 con 0x7f753c0b6590 2024-05-08T05:42:18.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.617+0000 7f753a7fc640 1 --2- 172.21.15.45:0/2610397442 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f75280762d0 0x7f7528078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:18.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.617+0000 7f753a7fc640 1 --2- 172.21.15.45:0/2610397442 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f75280762d0 0x7f7528078790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7530034000 tx=0x7f753002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:18.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.621+0000 7f75237fe640 1 -- 172.21.15.45:0/2610397442 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f752c017020 con 0x7f753c0b6590 2024-05-08T05:42:18.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:18.811+0000 7f754134d640 1 -- 172.21.15.45:0/2610397442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f753c0baba0 con 0x7f753c0b6590 2024-05-08T05:42:19.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.473+0000 7f75237fe640 1 -- 172.21.15.45:0/2610397442 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) ==== 170+0+0 (secure 0 0 0) 0x7f752c060c80 con 0x7f753c0b6590 2024-05-08T05:42:19.474 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-05-08T05:42:19.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.476+0000 7f754134d640 1 -- 172.21.15.45:0/2610397442 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f75280762d0 msgr2=0x7f7528078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:19.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.476+0000 7f754134d640 1 --2- 172.21.15.45:0/2610397442 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f75280762d0 0x7f7528078790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7530034000 tx=0x7f753002e000 comp rx=0 tx=0).stop 2024-05-08T05:42:19.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.476+0000 7f754134d640 1 -- 172.21.15.45:0/2610397442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f753c0b6590 msgr2=0x7f753c0d25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:19.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.476+0000 7f754134d640 1 --2- 172.21.15.45:0/2610397442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f753c0b6590 0x7f753c0d25c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f752c00e730 tx=0x7f752c00ebf0 comp rx=0 tx=0).stop 2024-05-08T05:42:19.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.476+0000 7f754134d640 1 -- 172.21.15.45:0/2610397442 shutdown_connections 2024-05-08T05:42:19.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.476+0000 7f754134d640 1 --2- 172.21.15.45:0/2610397442 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f75280762d0 0x7f7528078790 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:19.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.476+0000 7f754134d640 1 --2- 172.21.15.45:0/2610397442 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f753c163290 0x7f753c161c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:19.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.476+0000 7f754134d640 1 --2- 172.21.15.45:0/2610397442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f753c0d2bb0 0x7f753c157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:19.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.476+0000 7f754134d640 1 --2- 172.21.15.45:0/2610397442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f753c0b6590 0x7f753c0d25c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:19.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.476+0000 7f754134d640 1 -- 172.21.15.45:0/2610397442 >> 172.21.15.45:0/2610397442 conn(0x7f753c0d4fe0 msgr2=0x7f753c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:19.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.476+0000 7f754134d640 1 -- 172.21.15.45:0/2610397442 shutdown_connections 2024-05-08T05:42:19.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.476+0000 7f754134d640 1 -- 172.21.15.45:0/2610397442 wait complete. 2024-05-08T05:42:19.505 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T05:42:19.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.616+0000 7f32f1e72640 1 Processor -- start 2024-05-08T05:42:19.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.616+0000 7f32f1e72640 1 -- start start 2024-05-08T05:42:19.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.617+0000 7f32f1e72640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f32ec06d620 0x7f32ec073800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:19.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.617+0000 7f32f1e72640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32ec0b65f0 con 0x7f32ec06da00 2024-05-08T05:42:19.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.617+0000 7f32f1e72640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32ec0b5120 con 0x7f32ec1637b0 2024-05-08T05:42:19.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.617+0000 7f32f1e72640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f32ec0afaa0 con 0x7f32ec06d620 2024-05-08T05:42:19.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.617+0000 7f32eaffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f32ec06da00 0x7f32ec0af2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36818/0 (socket says 172.21.15.45:36818) 2024-05-08T05:42:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.617+0000 7f32eaffd640 1 -- 172.21.15.45:0/4033873943 learned_addr learned my addr 172.21.15.45:0/4033873943 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.617+0000 7f32eb7fe640 1 --2- 172.21.15.45:0/4033873943 >> v2:172.21.15.63:3300/0 conn(0x7f32ec06d620 0x7f32ec073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.617+0000 7f32ea7fc640 1 -- 172.21.15.45:0/4033873943 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2915795002 0 0) 0x7f32ec0b65f0 con 0x7f32ec06da00 2024-05-08T05:42:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.617+0000 7f32ea7fc640 1 -- 172.21.15.45:0/4033873943 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32d4003520 con 0x7f32ec06da00 2024-05-08T05:42:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.618+0000 7f32ea7fc640 1 -- 172.21.15.45:0/4033873943 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1817948838 0 0) 0x7f32d4003520 con 0x7f32ec06da00 2024-05-08T05:42:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.618+0000 7f32eb7fe640 1 -- 172.21.15.45:0/4033873943 >> v1:172.21.15.63:6789/0 conn(0x7f32ec1637b0 legacy=0x7f32ec0b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.618+0000 7f32eb7fe640 1 -- 172.21.15.45:0/4033873943 >> v1:172.21.15.45:6789/0 conn(0x7f32ec06da00 legacy=0x7f32ec0af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:19.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.618+0000 7f32eb7fe640 1 -- 172.21.15.45:0/4033873943 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ec0b5400 con 0x7f32ec06d620 2024-05-08T05:42:19.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.619+0000 7f32eb7fe640 1 --2- 172.21.15.45:0/4033873943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32ec06d620 0x7f32ec073800 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0097a0 tx=0x7f32dc029fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7653d0b79809fd29 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:19.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.619+0000 7f32ea7fc640 1 -- 172.21.15.45:0/4033873943 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc037070 con 0x7f32ec06d620 2024-05-08T05:42:19.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.619+0000 7f32ea7fc640 1 -- 172.21.15.45:0/4033873943 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32dc02aa40 con 0x7f32ec06d620 2024-05-08T05:42:19.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.619+0000 7f32ea7fc640 1 -- 172.21.15.45:0/4033873943 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e00016f0 con 0x7f32ec06d620 2024-05-08T05:42:19.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.619+0000 7f32f1e72640 1 -- 172.21.15.45:0/4033873943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32ec06d620 msgr2=0x7f32ec073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:19.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.619+0000 7f32f1e72640 1 --2- 172.21.15.45:0/4033873943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32ec06d620 0x7f32ec073800 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0097a0 tx=0x7f32dc029fb0 comp rx=0 tx=0).stop 2024-05-08T05:42:19.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.619+0000 7f32f1e72640 1 -- 172.21.15.45:0/4033873943 shutdown_connections 2024-05-08T05:42:19.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.619+0000 7f32f1e72640 1 --2- 172.21.15.45:0/4033873943 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32ec06d620 0x7f32ec073800 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:19.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.619+0000 7f32f1e72640 1 -- 172.21.15.45:0/4033873943 >> 172.21.15.45:0/4033873943 conn(0x7f32ec0728c0 msgr2=0x7f32ec072cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:19.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.619+0000 7f32f1e72640 1 -- 172.21.15.45:0/4033873943 shutdown_connections 2024-05-08T05:42:19.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.619+0000 7f32f1e72640 1 -- 172.21.15.45:0/4033873943 wait complete. 2024-05-08T05:42:19.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.620+0000 7f32f1e72640 1 Processor -- start 2024-05-08T05:42:19.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.621+0000 7f32f1e72640 1 -- start start 2024-05-08T05:42:19.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.621+0000 7f32f1e72640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32ec06d620 0x7f32ec157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:19.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.621+0000 7f32f1e72640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec06da00 0x7f32ec0d0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:19.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.621+0000 7f32f1e72640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32ec1637b0 0x7f32ec145d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:19.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.621+0000 7f32f1e72640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec06e920 con 0x7f32ec06da00 2024-05-08T05:42:19.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.621+0000 7f32f1e72640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec0b5a50 con 0x7f32ec06d620 2024-05-08T05:42:19.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.621+0000 7f32f1e72640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f32ec06eaa0 con 0x7f32ec1637b0 2024-05-08T05:42:19.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.622+0000 7f32eaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec06da00 0x7f32ec0d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:19.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.622+0000 7f32eaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec06da00 0x7f32ec0d0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45160/0 (socket says 172.21.15.45:45160) 2024-05-08T05:42:19.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.622+0000 7f32eaffd640 1 -- 172.21.15.45:0/654937306 learned_addr learned my addr 172.21.15.45:0/654937306 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:19.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.622+0000 7f32eb7fe640 1 --2- 172.21.15.45:0/654937306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32ec06d620 0x7f32ec157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:19.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.622+0000 7f32ebfff640 1 --2- 172.21.15.45:0/654937306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32ec1637b0 0x7f32ec145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:19.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.622+0000 7f32eaffd640 1 -- 172.21.15.45:0/654937306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32ec1637b0 msgr2=0x7f32ec145d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:19.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.622+0000 7f32eaffd640 1 --2- 172.21.15.45:0/654937306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32ec1637b0 0x7f32ec145d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:19.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.622+0000 7f32eaffd640 1 -- 172.21.15.45:0/654937306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32ec06d620 msgr2=0x7f32ec157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:19.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.622+0000 7f32eaffd640 1 --2- 172.21.15.45:0/654937306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32ec06d620 0x7f32ec157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:19.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.622+0000 7f32eaffd640 1 -- 172.21.15.45:0/654937306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ec0d0650 con 0x7f32ec06da00 2024-05-08T05:42:19.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.622+0000 7f32ebfff640 1 --2- 172.21.15.45:0/654937306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32ec1637b0 0x7f32ec145d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:19.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.622+0000 7f32eaffd640 1 --2- 172.21.15.45:0/654937306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec06da00 0x7f32ec0d0110 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f32e000c700 tx=0x7f32e000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:19.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.622+0000 7f32d3fff640 1 -- 172.21.15.45:0/654937306 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0017070 con 0x7f32ec06da00 2024-05-08T05:42:19.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.622+0000 7f32f1e72640 1 -- 172.21.15.45:0/654937306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ec157cd0 con 0x7f32ec06da00 2024-05-08T05:42:19.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.622+0000 7f32d3fff640 1 -- 172.21.15.45:0/654937306 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32e0005ad0 con 0x7f32ec06da00 2024-05-08T05:42:19.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.622+0000 7f32d3fff640 1 -- 172.21.15.45:0/654937306 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0007070 con 0x7f32ec06da00 2024-05-08T05:42:19.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.622+0000 7f32f1e72640 1 -- 172.21.15.45:0/654937306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ec0cf910 con 0x7f32ec06da00 2024-05-08T05:42:19.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.623+0000 7f32f1e72640 1 -- 172.21.15.45:0/654937306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32ec1581f0 con 0x7f32ec06da00 2024-05-08T05:42:19.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.624+0000 7f32d3fff640 1 -- 172.21.15.45:0/654937306 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f32e001a070 con 0x7f32ec06da00 2024-05-08T05:42:19.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.624+0000 7f32d3fff640 1 --2- 172.21.15.45:0/654937306 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f32d80762d0 0x7f32d8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:19.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.624+0000 7f32d3fff640 1 -- 172.21.15.45:0/654937306 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 6001+0+0 (secure 0 0 0) 0x7f32e0096bf0 con 0x7f32ec06da00 2024-05-08T05:42:19.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.625+0000 7f32eb7fe640 1 --2- 172.21.15.45:0/654937306 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f32d80762d0 0x7f32d8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:19.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.625+0000 7f32eb7fe640 1 --2- 172.21.15.45:0/654937306 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f32d80762d0 0x7f32d8078790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f32dc034000 tx=0x7f32dc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:19.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.629+0000 7f32d3fff640 1 -- 172.21.15.45:0/654937306 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f32ec0d0650 con 0x7f32ec06da00 2024-05-08T05:42:19.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:19.859+0000 7f32f1e72640 1 -- 172.21.15.45:0/654937306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f32ec0cf700 con 0x7f32ec06da00 2024-05-08T05:42:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.499+0000 7f32d3fff640 1 -- 172.21.15.45:0/654937306 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) ==== 172+0+0 (secure 0 0 0) 0x7f32e005fd50 con 0x7f32ec06da00 2024-05-08T05:42:20.500 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 5 and data pool 6 2024-05-08T05:42:20.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.505+0000 7f32f1e72640 1 -- 172.21.15.45:0/654937306 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f32d80762d0 msgr2=0x7f32d8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:20.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.505+0000 7f32f1e72640 1 --2- 172.21.15.45:0/654937306 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f32d80762d0 0x7f32d8078790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f32dc034000 tx=0x7f32dc02e000 comp rx=0 tx=0).stop 2024-05-08T05:42:20.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.505+0000 7f32f1e72640 1 -- 172.21.15.45:0/654937306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec06da00 msgr2=0x7f32ec0d0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:20.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.505+0000 7f32f1e72640 1 --2- 172.21.15.45:0/654937306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec06da00 0x7f32ec0d0110 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f32e000c700 tx=0x7f32e000cbc0 comp rx=0 tx=0).stop 2024-05-08T05:42:20.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.506+0000 7f32f1e72640 1 -- 172.21.15.45:0/654937306 shutdown_connections 2024-05-08T05:42:20.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.506+0000 7f32f1e72640 1 --2- 172.21.15.45:0/654937306 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f32d80762d0 0x7f32d8078790 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:20.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.506+0000 7f32f1e72640 1 --2- 172.21.15.45:0/654937306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f32ec1637b0 0x7f32ec145d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:20.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.506+0000 7f32f1e72640 1 --2- 172.21.15.45:0/654937306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec06da00 0x7f32ec0d0110 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:20.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.506+0000 7f32f1e72640 1 --2- 172.21.15.45:0/654937306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32ec06d620 0x7f32ec157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:20.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.506+0000 7f32f1e72640 1 -- 172.21.15.45:0/654937306 >> 172.21.15.45:0/654937306 conn(0x7f32ec0728c0 msgr2=0x7f32ec162b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:20.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.508+0000 7f32f1e72640 1 -- 172.21.15.45:0/654937306 shutdown_connections 2024-05-08T05:42:20.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.509+0000 7f32f1e72640 1 -- 172.21.15.45:0/654937306 wait complete. 2024-05-08T05:42:20.538 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:42:20.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.645+0000 7f09832d1640 1 Processor -- start 2024-05-08T05:42:20.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.646+0000 7f09832d1640 1 -- start start 2024-05-08T05:42:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.646+0000 7f09832d1640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f097c15f420 0x7f097c0d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:20.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.646+0000 7f09832d1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f097c0d3ca0 0x7f097c0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.646+0000 7f09832d1640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f097c1630b0 con 0x7f097c0d1eb0 2024-05-08T05:42:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.646+0000 7f09832d1640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f097c0d22f0 con 0x7f097c15f420 2024-05-08T05:42:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.646+0000 7f09832d1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f097c0d2550 con 0x7f097c0d3ca0 2024-05-08T05:42:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.647+0000 7f098086c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f097c0d3ca0 0x7f097c0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.647+0000 7f098186e640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f097c0d1eb0 0x7f097c0d4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:37534/0 (socket says 172.21.15.45:37534) 2024-05-08T05:42:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.647+0000 7f098186e640 1 -- 172.21.15.45:0/1310978900 learned_addr learned my addr 172.21.15.45:0/1310978900 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.647+0000 7f098106d640 1 --2- 172.21.15.45:0/1310978900 >> v2:172.21.15.63:3300/0 conn(0x7f097c15f420 0x7f097c0d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.647+0000 7f097bfff640 1 -- 172.21.15.45:0/1310978900 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3671809485 0 0) 0x7f096c001490 con 0x7f097c0d1eb0 2024-05-08T05:42:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.647+0000 7f097bfff640 1 -- 172.21.15.45:0/1310978900 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0964003520 con 0x7f097c0d1eb0 2024-05-08T05:42:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.647+0000 7f098086c640 1 -- 172.21.15.45:0/1310978900 >> v2:172.21.15.63:3300/0 conn(0x7f097c15f420 msgr2=0x7f097c0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.647+0000 7f098086c640 1 --2- 172.21.15.45:0/1310978900 >> v2:172.21.15.63:3300/0 conn(0x7f097c15f420 0x7f097c0d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.647+0000 7f098086c640 1 -- 172.21.15.45:0/1310978900 >> v1:172.21.15.63:6789/0 conn(0x7f097c0d1eb0 legacy=0x7f097c0d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:20.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.647+0000 7f098086c640 1 -- 172.21.15.45:0/1310978900 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097c0d2c80 con 0x7f097c0d3ca0 2024-05-08T05:42:20.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.647+0000 7f098106d640 1 --2- 172.21.15.45:0/1310978900 >> v2:172.21.15.63:3300/0 conn(0x7f097c15f420 0x7f097c0d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:42:20.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.648+0000 7f098086c640 1 --2- 172.21.15.45:0/1310978900 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f097c0d3ca0 0x7f097c0d1970 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f09700097a0 tx=0x7f097002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92a186530a7073f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:20.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.648+0000 7f097bfff640 1 -- 172.21.15.45:0/1310978900 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970037070 con 0x7f097c0d3ca0 2024-05-08T05:42:20.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.648+0000 7f097bfff640 1 -- 172.21.15.45:0/1310978900 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0970032440 con 0x7f097c0d3ca0 2024-05-08T05:42:20.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.648+0000 7f097bfff640 1 -- 172.21.15.45:0/1310978900 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09700327f0 con 0x7f097c0d3ca0 2024-05-08T05:42:20.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.649+0000 7f09832d1640 1 -- 172.21.15.45:0/1310978900 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f097c0d3ca0 msgr2=0x7f097c0d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:20.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.649+0000 7f09832d1640 1 --2- 172.21.15.45:0/1310978900 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f097c0d3ca0 0x7f097c0d1970 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f09700097a0 tx=0x7f097002a180 comp rx=0 tx=0).stop 2024-05-08T05:42:20.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.649+0000 7f09832d1640 1 -- 172.21.15.45:0/1310978900 shutdown_connections 2024-05-08T05:42:20.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.649+0000 7f09832d1640 1 --2- 172.21.15.45:0/1310978900 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f097c0d3ca0 0x7f097c0d1970 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:20.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.649+0000 7f09832d1640 1 --2- 172.21.15.45:0/1310978900 >> v2:172.21.15.63:3300/0 conn(0x7f097c15f420 0x7f097c0d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:20.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.649+0000 7f09832d1640 1 -- 172.21.15.45:0/1310978900 >> 172.21.15.45:0/1310978900 conn(0x7f097c08eac0 msgr2=0x7f097c09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:20.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.649+0000 7f09832d1640 1 -- 172.21.15.45:0/1310978900 shutdown_connections 2024-05-08T05:42:20.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.649+0000 7f09832d1640 1 -- 172.21.15.45:0/1310978900 wait complete. 2024-05-08T05:42:20.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.650+0000 7f09832d1640 1 Processor -- start 2024-05-08T05:42:20.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.650+0000 7f09832d1640 1 -- start start 2024-05-08T05:42:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.651+0000 7f09832d1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f097c0d1eb0 0x7f097c161490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.651+0000 7f09832d1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f097c162070 0x7f097c15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.651+0000 7f09832d1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f097c0d0150 0x7f097c0d0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.651+0000 7f09832d1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f097c0d2830 con 0x7f097c0d0150 2024-05-08T05:42:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.651+0000 7f09832d1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f097c0d3240 con 0x7f097c162070 2024-05-08T05:42:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.651+0000 7f09832d1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f097c0d53d0 con 0x7f097c0d1eb0 2024-05-08T05:42:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.651+0000 7f098186e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f097c0d0150 0x7f097c0d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.651+0000 7f098186e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f097c0d0150 0x7f097c0d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44102/0 (socket says 172.21.15.45:44102) 2024-05-08T05:42:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.651+0000 7f098186e640 1 -- 172.21.15.45:0/697808389 learned_addr learned my addr 172.21.15.45:0/697808389 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:20.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.651+0000 7f098086c640 1 --2- 172.21.15.45:0/697808389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f097c162070 0x7f097c15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.651+0000 7f098106d640 1 --2- 172.21.15.45:0/697808389 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f097c0d1eb0 0x7f097c161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.651+0000 7f098186e640 1 -- 172.21.15.45:0/697808389 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f097c0d1eb0 msgr2=0x7f097c161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.651+0000 7f098186e640 1 --2- 172.21.15.45:0/697808389 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f097c0d1eb0 0x7f097c161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.651+0000 7f098186e640 1 -- 172.21.15.45:0/697808389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f097c162070 msgr2=0x7f097c15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.651+0000 7f098186e640 1 --2- 172.21.15.45:0/697808389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f097c162070 0x7f097c15dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.651+0000 7f098186e640 1 -- 172.21.15.45:0/697808389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097c0c8ba0 con 0x7f097c0d0150 2024-05-08T05:42:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.651+0000 7f098186e640 1 --2- 172.21.15.45:0/697808389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f097c0d0150 0x7f097c0d0600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f096c00cb40 tx=0x7f096c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.652+0000 7f0979ffb640 1 -- 172.21.15.45:0/697808389 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c00aca0 con 0x7f097c0d0150 2024-05-08T05:42:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.652+0000 7f0979ffb640 1 -- 172.21.15.45:0/697808389 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f096c00ae40 con 0x7f097c0d0150 2024-05-08T05:42:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.652+0000 7f09832d1640 1 -- 172.21.15.45:0/697808389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f097c0c8e80 con 0x7f097c0d0150 2024-05-08T05:42:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.652+0000 7f0979ffb640 1 -- 172.21.15.45:0/697808389 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c012600 con 0x7f097c0d0150 2024-05-08T05:42:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.652+0000 7f09832d1640 1 -- 172.21.15.45:0/697808389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f097c0c9440 con 0x7f097c0d0150 2024-05-08T05:42:20.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.653+0000 7f09832d1640 1 -- 172.21.15.45:0/697808389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f097c0693f0 con 0x7f097c0d0150 2024-05-08T05:42:20.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.653+0000 7f0979ffb640 1 -- 172.21.15.45:0/697808389 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f096c0127a0 con 0x7f097c0d0150 2024-05-08T05:42:20.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.654+0000 7f0979ffb640 1 --2- 172.21.15.45:0/697808389 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0968076300 0x7f09680787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:20.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.654+0000 7f0979ffb640 1 -- 172.21.15.45:0/697808389 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6101+0+0 (secure 0 0 0) 0x7f096c097da0 con 0x7f097c0d0150 2024-05-08T05:42:20.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.654+0000 7f098106d640 1 --2- 172.21.15.45:0/697808389 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0968076300 0x7f09680787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:20.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.655+0000 7f098106d640 1 --2- 172.21.15.45:0/697808389 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0968076300 0x7f09680787c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f097c161e40 tx=0x7f0974008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:20.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.658+0000 7f0979ffb640 1 -- 172.21.15.45:0/697808389 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f097c1630b0 con 0x7f097c0d0150 2024-05-08T05:42:20.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.851+0000 7f09832d1640 1 -- 172.21.15.45:0/697808389 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f097c0b4c50 con 0x7f097c0d0150 2024-05-08T05:42:20.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.852+0000 7f0979ffb640 1 -- 172.21.15.45:0/697808389 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16907 (secure 0 0 0) 0x7f096c061600 con 0x7f097c0d0150 2024-05-08T05:42:20.852 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:20.852 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":35,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:42:19.860379+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:42:17.918020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-08T05:42:18.816243+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:42:20.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.855+0000 7f09832d1640 1 -- 172.21.15.45:0/697808389 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0968076300 msgr2=0x7f09680787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:20.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.855+0000 7f09832d1640 1 --2- 172.21.15.45:0/697808389 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0968076300 0x7f09680787c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f097c161e40 tx=0x7f0974008040 comp rx=0 tx=0).stop 2024-05-08T05:42:20.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.855+0000 7f09832d1640 1 -- 172.21.15.45:0/697808389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f097c0d0150 msgr2=0x7f097c0d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:20.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.855+0000 7f09832d1640 1 --2- 172.21.15.45:0/697808389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f097c0d0150 0x7f097c0d0600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f096c00cb40 tx=0x7f096c00a060 comp rx=0 tx=0).stop 2024-05-08T05:42:20.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.855+0000 7f09832d1640 1 -- 172.21.15.45:0/697808389 shutdown_connections 2024-05-08T05:42:20.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.855+0000 7f09832d1640 1 --2- 172.21.15.45:0/697808389 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0968076300 0x7f09680787c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:20.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.855+0000 7f09832d1640 1 --2- 172.21.15.45:0/697808389 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f097c0d0150 0x7f097c0d0600 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:20.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.855+0000 7f09832d1640 1 --2- 172.21.15.45:0/697808389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f097c162070 0x7f097c15dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:20.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.855+0000 7f09832d1640 1 --2- 172.21.15.45:0/697808389 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f097c0d1eb0 0x7f097c161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:20.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.855+0000 7f09832d1640 1 -- 172.21.15.45:0/697808389 >> 172.21.15.45:0/697808389 conn(0x7f097c08eac0 msgr2=0x7f097c06e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:20.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.855+0000 7f09832d1640 1 -- 172.21.15.45:0/697808389 shutdown_connections 2024-05-08T05:42:20.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.855+0000 7f09832d1640 1 -- 172.21.15.45:0/697808389 wait complete. 2024-05-08T05:42:20.867 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:42:20.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.972+0000 7f6d9c99e640 1 Processor -- start 2024-05-08T05:42:20.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.973+0000 7f6d9c99e640 1 -- start start 2024-05-08T05:42:20.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.973+0000 7f6d9c99e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9806cab0 0x7f6d98071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:20.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.973+0000 7f6d9c99e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6d980af190 0x7f6d980af620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:20.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.973+0000 7f6d9c99e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d980b1500 con 0x7f6d9815cf50 2024-05-08T05:42:20.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.974+0000 7f6d9c99e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f6d98071ba0 con 0x7f6d980af190 2024-05-08T05:42:20.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.974+0000 7f6d9c99e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6d98160b20 con 0x7f6d9806cab0 2024-05-08T05:42:20.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.974+0000 7f6d97fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6d980af190 0x7f6d980af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:20.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.974+0000 7f6d97fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6d980af190 0x7f6d980af620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44106/0 (socket says 172.21.15.45:44106) 2024-05-08T05:42:20.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.974+0000 7f6d97fff640 1 -- 172.21.15.45:0/93884208 learned_addr learned my addr 172.21.15.45:0/93884208 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:20.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.974+0000 7f6d8effd640 1 --2- 172.21.15.45:0/93884208 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9806cab0 0x7f6d98071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:20.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.974+0000 7f6d97fff640 1 -- 172.21.15.45:0/93884208 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9806cab0 msgr2=0x7f6d98071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:20.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.974+0000 7f6d97fff640 1 --2- 172.21.15.45:0/93884208 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9806cab0 0x7f6d98071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:20.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.974+0000 7f6d97fff640 1 -- 172.21.15.45:0/93884208 >> v1:172.21.15.63:6789/0 conn(0x7f6d9815cf50 legacy=0x7f6d9806c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:20.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.974+0000 7f6d97fff640 1 -- 172.21.15.45:0/93884208 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d98160e00 con 0x7f6d980af190 2024-05-08T05:42:20.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.975+0000 7f6d97fff640 1 --2- 172.21.15.45:0/93884208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d980af190 0x7f6d980af620 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009e50 tx=0x7f6d9002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127dad2127ec4328 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:20.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.975+0000 7f6d96ffd640 1 -- 172.21.15.45:0/93884208 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90036070 con 0x7f6d980af190 2024-05-08T05:42:20.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.975+0000 7f6d96ffd640 1 -- 172.21.15.45:0/93884208 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d90031600 con 0x7f6d980af190 2024-05-08T05:42:20.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.975+0000 7f6d96ffd640 1 -- 172.21.15.45:0/93884208 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d900319d0 con 0x7f6d980af190 2024-05-08T05:42:20.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.976+0000 7f6d9c99e640 1 -- 172.21.15.45:0/93884208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d980af190 msgr2=0x7f6d980af620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:20.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.976+0000 7f6d9c99e640 1 --2- 172.21.15.45:0/93884208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d980af190 0x7f6d980af620 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009e50 tx=0x7f6d9002a710 comp rx=0 tx=0).stop 2024-05-08T05:42:20.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.976+0000 7f6d9c99e640 1 -- 172.21.15.45:0/93884208 shutdown_connections 2024-05-08T05:42:20.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.976+0000 7f6d9c99e640 1 --2- 172.21.15.45:0/93884208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d980af190 0x7f6d980af620 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:20.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.976+0000 7f6d9c99e640 1 --2- 172.21.15.45:0/93884208 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9806cab0 0x7f6d98071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:20.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.976+0000 7f6d9c99e640 1 -- 172.21.15.45:0/93884208 >> 172.21.15.45:0/93884208 conn(0x7f6d980b39a0 msgr2=0x7f6d980b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:20.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.976+0000 7f6d9c99e640 1 -- 172.21.15.45:0/93884208 shutdown_connections 2024-05-08T05:42:20.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.976+0000 7f6d9c99e640 1 -- 172.21.15.45:0/93884208 wait complete. 2024-05-08T05:42:20.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.977+0000 7f6d9c99e640 1 Processor -- start 2024-05-08T05:42:20.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.978+0000 7f6d9c99e640 1 -- start start 2024-05-08T05:42:20.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.978+0000 7f6d9c99e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d9806cab0 0x7f6d981594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:20.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.978+0000 7f6d9c99e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d980af190 0x7f6d9815eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:20.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.978+0000 7f6d9c99e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9815cf50 0x7f6d9815f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:20.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.978+0000 7f6d9c99e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9806f300 con 0x7f6d9806cab0 2024-05-08T05:42:20.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.978+0000 7f6d9c99e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6d980b1190 con 0x7f6d980af190 2024-05-08T05:42:20.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.978+0000 7f6d9c99e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6d9806f480 con 0x7f6d9815cf50 2024-05-08T05:42:20.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.978+0000 7f6d977fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d9806cab0 0x7f6d981594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:20.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.979+0000 7f6d977fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d9806cab0 0x7f6d981594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44122/0 (socket says 172.21.15.45:44122) 2024-05-08T05:42:20.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.979+0000 7f6d977fe640 1 -- 172.21.15.45:0/3032239038 learned_addr learned my addr 172.21.15.45:0/3032239038 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:20.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.979+0000 7f6d8effd640 1 --2- 172.21.15.45:0/3032239038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d980af190 0x7f6d9815eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:20.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.979+0000 7f6d97fff640 1 --2- 172.21.15.45:0/3032239038 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9815cf50 0x7f6d9815f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:20.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.979+0000 7f6d977fe640 1 -- 172.21.15.45:0/3032239038 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9815cf50 msgr2=0x7f6d9815f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:20.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.979+0000 7f6d977fe640 1 --2- 172.21.15.45:0/3032239038 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9815cf50 0x7f6d9815f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:20.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.979+0000 7f6d977fe640 1 -- 172.21.15.45:0/3032239038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d980af190 msgr2=0x7f6d9815eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:20.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.979+0000 7f6d977fe640 1 --2- 172.21.15.45:0/3032239038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d980af190 0x7f6d9815eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:20.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.979+0000 7f6d977fe640 1 -- 172.21.15.45:0/3032239038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9815fac0 con 0x7f6d9806cab0 2024-05-08T05:42:20.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.979+0000 7f6d8effd640 1 --2- 172.21.15.45:0/3032239038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d980af190 0x7f6d9815eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:20.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.979+0000 7f6d977fe640 1 --2- 172.21.15.45:0/3032239038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d9806cab0 0x7f6d981594c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800c3d0 tx=0x7f6d8800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:20.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.979+0000 7f6d94ff9640 1 -- 172.21.15.45:0/3032239038 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d88017070 con 0x7f6d9806cab0 2024-05-08T05:42:20.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.979+0000 7f6d94ff9640 1 -- 172.21.15.45:0/3032239038 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d880052d0 con 0x7f6d9806cab0 2024-05-08T05:42:20.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.979+0000 7f6d9c99e640 1 -- 172.21.15.45:0/3032239038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d9815fda0 con 0x7f6d9806cab0 2024-05-08T05:42:20.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.979+0000 7f6d94ff9640 1 -- 172.21.15.45:0/3032239038 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d88003830 con 0x7f6d9806cab0 2024-05-08T05:42:20.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.979+0000 7f6d9c99e640 1 -- 172.21.15.45:0/3032239038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d98159a00 con 0x7f6d9806cab0 2024-05-08T05:42:20.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.980+0000 7f6d9c99e640 1 -- 172.21.15.45:0/3032239038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d9813e460 con 0x7f6d9806cab0 2024-05-08T05:42:20.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.981+0000 7f6d94ff9640 1 -- 172.21.15.45:0/3032239038 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6d8801a070 con 0x7f6d9806cab0 2024-05-08T05:42:20.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.981+0000 7f6d94ff9640 1 --2- 172.21.15.45:0/3032239038 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6d780762d0 0x7f6d78078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:20.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.982+0000 7f6d94ff9640 1 -- 172.21.15.45:0/3032239038 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6101+0+0 (secure 0 0 0) 0x7f6d88096b60 con 0x7f6d9806cab0 2024-05-08T05:42:20.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.982+0000 7f6d8effd640 1 --2- 172.21.15.45:0/3032239038 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6d780762d0 0x7f6d78078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:20.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.982+0000 7f6d8effd640 1 --2- 172.21.15.45:0/3032239038 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6d780762d0 0x7f6d78078790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6d9815e7b0 tx=0x7f6d80008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:20.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:20.986+0000 7f6d94ff9640 1 -- 172.21.15.45:0/3032239038 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6d9813e460 con 0x7f6d9806cab0 2024-05-08T05:42:21.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.179+0000 7f6d9c99e640 1 -- 172.21.15.45:0/3032239038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d98050740 con 0x7f6d9806cab0 2024-05-08T05:42:21.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.181+0000 7f6d94ff9640 1 -- 172.21.15.45:0/3032239038 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16907 (secure 0 0 0) 0x7f6d88060290 con 0x7f6d9806cab0 2024-05-08T05:42:21.182 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:21.182 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":35,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:42:19.860379+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:42:17.918020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-08T05:42:18.816243+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:42:21.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.183+0000 7f6d9c99e640 1 -- 172.21.15.45:0/3032239038 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6d780762d0 msgr2=0x7f6d78078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:21.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.183+0000 7f6d9c99e640 1 --2- 172.21.15.45:0/3032239038 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6d780762d0 0x7f6d78078790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6d9815e7b0 tx=0x7f6d80008040 comp rx=0 tx=0).stop 2024-05-08T05:42:21.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.183+0000 7f6d9c99e640 1 -- 172.21.15.45:0/3032239038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d9806cab0 msgr2=0x7f6d981594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:21.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.183+0000 7f6d9c99e640 1 --2- 172.21.15.45:0/3032239038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d9806cab0 0x7f6d981594c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800c3d0 tx=0x7f6d8800c890 comp rx=0 tx=0).stop 2024-05-08T05:42:21.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.183+0000 7f6d9c99e640 1 -- 172.21.15.45:0/3032239038 shutdown_connections 2024-05-08T05:42:21.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.183+0000 7f6d9c99e640 1 --2- 172.21.15.45:0/3032239038 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6d780762d0 0x7f6d78078790 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:21.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.183+0000 7f6d9c99e640 1 --2- 172.21.15.45:0/3032239038 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6d9815cf50 0x7f6d9815f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:21.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.183+0000 7f6d9c99e640 1 --2- 172.21.15.45:0/3032239038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d980af190 0x7f6d9815eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:21.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.183+0000 7f6d9c99e640 1 --2- 172.21.15.45:0/3032239038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d9806cab0 0x7f6d981594c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:21.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.183+0000 7f6d9c99e640 1 -- 172.21.15.45:0/3032239038 >> 172.21.15.45:0/3032239038 conn(0x7f6d980b39a0 msgr2=0x7f6d9806c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:21.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.183+0000 7f6d9c99e640 1 -- 172.21.15.45:0/3032239038 shutdown_connections 2024-05-08T05:42:21.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.183+0000 7f6d9c99e640 1 -- 172.21.15.45:0/3032239038 wait complete. 2024-05-08T05:42:21.195 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T05:42:21.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.304+0000 7fa525acf640 1 Processor -- start 2024-05-08T05:42:21.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.304+0000 7fa525acf640 1 -- start start 2024-05-08T05:42:21.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.305+0000 7fa525acf640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa52006e0f0 0x7fa5200cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:21.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.305+0000 7fa525acf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5200d0360 0x7fa5200b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:21.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.305+0000 7fa525acf640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa52015d2a0 con 0x7fa52006f060 2024-05-08T05:42:21.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.305+0000 7fa525acf640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa5200d07c0 con 0x7fa5200d0360 2024-05-08T05:42:21.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.305+0000 7fa525acf640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa52006f440 con 0x7fa52006e0f0 2024-05-08T05:42:21.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.305+0000 7fa51ffff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5200d0360 0x7fa5200b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:21.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.305+0000 7fa51ffff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa5200d0360 0x7fa5200b3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44132/0 (socket says 172.21.15.45:44132) 2024-05-08T05:42:21.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.305+0000 7fa51ffff640 1 -- 172.21.15.45:0/1534485892 learned_addr learned my addr 172.21.15.45:0/1534485892 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:21.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.305+0000 7fa51ffff640 1 -- 172.21.15.45:0/1534485892 >> v2:172.21.15.63:3300/0 conn(0x7fa52006e0f0 msgr2=0x7fa5200cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:21.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.305+0000 7fa51effd640 1 --2- 172.21.15.45:0/1534485892 >> v2:172.21.15.63:3300/0 conn(0x7fa52006e0f0 0x7fa5200cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:21.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.305+0000 7fa51ffff640 1 --2- 172.21.15.45:0/1534485892 >> v2:172.21.15.63:3300/0 conn(0x7fa52006e0f0 0x7fa5200cfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:21.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.305+0000 7fa51ffff640 1 -- 172.21.15.45:0/1534485892 >> v1:172.21.15.63:6789/0 conn(0x7fa52006f060 legacy=0x7fa52006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:21.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.305+0000 7fa51ffff640 1 -- 172.21.15.45:0/1534485892 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5201609b0 con 0x7fa5200d0360 2024-05-08T05:42:21.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.305+0000 7fa51effd640 1 --2- 172.21.15.45:0/1534485892 >> v2:172.21.15.63:3300/0 conn(0x7fa52006e0f0 0x7fa5200cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:21.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.306+0000 7fa51ffff640 1 --2- 172.21.15.45:0/1534485892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5200d0360 0x7fa5200b3f00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa5200d1b40 tx=0x7fa51802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168801959c204e7a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:21.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.306+0000 7fa51e7fc640 1 -- 172.21.15.45:0/1534485892 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518036070 con 0x7fa5200d0360 2024-05-08T05:42:21.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.306+0000 7fa51e7fc640 1 -- 172.21.15.45:0/1534485892 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa518031600 con 0x7fa5200d0360 2024-05-08T05:42:21.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.306+0000 7fa51e7fc640 1 -- 172.21.15.45:0/1534485892 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5180319d0 con 0x7fa5200d0360 2024-05-08T05:42:21.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.307+0000 7fa525acf640 1 -- 172.21.15.45:0/1534485892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5200d0360 msgr2=0x7fa5200b3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:21.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.307+0000 7fa525acf640 1 --2- 172.21.15.45:0/1534485892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5200d0360 0x7fa5200b3f00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa5200d1b40 tx=0x7fa51802a6a0 comp rx=0 tx=0).stop 2024-05-08T05:42:21.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.307+0000 7fa525acf640 1 -- 172.21.15.45:0/1534485892 shutdown_connections 2024-05-08T05:42:21.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.307+0000 7fa525acf640 1 --2- 172.21.15.45:0/1534485892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5200d0360 0x7fa5200b3f00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:21.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.307+0000 7fa525acf640 1 --2- 172.21.15.45:0/1534485892 >> v2:172.21.15.63:3300/0 conn(0x7fa52006e0f0 0x7fa5200cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:21.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.307+0000 7fa525acf640 1 -- 172.21.15.45:0/1534485892 >> 172.21.15.45:0/1534485892 conn(0x7fa52006c3a0 msgr2=0x7fa52006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:21.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.307+0000 7fa525acf640 1 -- 172.21.15.45:0/1534485892 shutdown_connections 2024-05-08T05:42:21.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.307+0000 7fa525acf640 1 -- 172.21.15.45:0/1534485892 wait complete. 2024-05-08T05:42:21.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.308+0000 7fa525acf640 1 Processor -- start 2024-05-08T05:42:21.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.308+0000 7fa525acf640 1 -- start start 2024-05-08T05:42:21.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.309+0000 7fa525acf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa52006e0f0 0x7fa5201f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:21.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.309+0000 7fa525acf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa52006f060 0x7fa52009caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:21.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.309+0000 7fa525acf640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5200d0360 0x7fa52014a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:21.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.309+0000 7fa525acf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa520161250 con 0x7fa52006e0f0 2024-05-08T05:42:21.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.309+0000 7fa525acf640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa520160f70 con 0x7fa5200d0360 2024-05-08T05:42:21.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.309+0000 7fa525acf640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa520161530 con 0x7fa52006f060 2024-05-08T05:42:21.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.309+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa52006e0f0 0x7fa5201f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:21.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.309+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa52006e0f0 0x7fa5201f5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44144/0 (socket says 172.21.15.45:44144) 2024-05-08T05:42:21.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.309+0000 7fa51f7fe640 1 -- 172.21.15.45:0/150424652 learned_addr learned my addr 172.21.15.45:0/150424652 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:21.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.309+0000 7fa51ffff640 1 --2- 172.21.15.45:0/150424652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5200d0360 0x7fa52014a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:21.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.310+0000 7fa51effd640 1 --2- 172.21.15.45:0/150424652 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa52006f060 0x7fa52009caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:21.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.310+0000 7fa51f7fe640 1 -- 172.21.15.45:0/150424652 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa52006f060 msgr2=0x7fa52009caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:21.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.310+0000 7fa51f7fe640 1 --2- 172.21.15.45:0/150424652 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa52006f060 0x7fa52009caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:21.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.310+0000 7fa51f7fe640 1 -- 172.21.15.45:0/150424652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5200d0360 msgr2=0x7fa52014a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:21.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.310+0000 7fa51f7fe640 1 --2- 172.21.15.45:0/150424652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5200d0360 0x7fa52014a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:21.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.310+0000 7fa51f7fe640 1 -- 172.21.15.45:0/150424652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5201f66a0 con 0x7fa52006e0f0 2024-05-08T05:42:21.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.310+0000 7fa51ffff640 1 --2- 172.21.15.45:0/150424652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5200d0360 0x7fa52014a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:21.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.310+0000 7fa51f7fe640 1 --2- 172.21.15.45:0/150424652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa52006e0f0 0x7fa5201f5fa0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa51000c460 tx=0x7fa51000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:21.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.310+0000 7fa507fff640 1 -- 172.21.15.45:0/150424652 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510016020 con 0x7fa52006e0f0 2024-05-08T05:42:21.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.310+0000 7fa507fff640 1 -- 172.21.15.45:0/150424652 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa510012870 con 0x7fa52006e0f0 2024-05-08T05:42:21.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.310+0000 7fa525acf640 1 -- 172.21.15.45:0/150424652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5201f6980 con 0x7fa52006e0f0 2024-05-08T05:42:21.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.310+0000 7fa507fff640 1 -- 172.21.15.45:0/150424652 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510012c60 con 0x7fa52006e0f0 2024-05-08T05:42:21.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.310+0000 7fa525acf640 1 -- 172.21.15.45:0/150424652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5201f2bc0 con 0x7fa52006e0f0 2024-05-08T05:42:21.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.311+0000 7fa525acf640 1 -- 172.21.15.45:0/150424652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5200693f0 con 0x7fa52006e0f0 2024-05-08T05:42:21.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.312+0000 7fa507fff640 1 -- 172.21.15.45:0/150424652 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa510019020 con 0x7fa52006e0f0 2024-05-08T05:42:21.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.312+0000 7fa507fff640 1 --2- 172.21.15.45:0/150424652 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa50c0762d0 0x7fa50c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:21.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.313+0000 7fa507fff640 1 -- 172.21.15.45:0/150424652 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6101+0+0 (secure 0 0 0) 0x7fa510097290 con 0x7fa52006e0f0 2024-05-08T05:42:21.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.313+0000 7fa51effd640 1 --2- 172.21.15.45:0/150424652 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa50c0762d0 0x7fa50c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:21.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.313+0000 7fa51effd640 1 --2- 172.21.15.45:0/150424652 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa50c0762d0 0x7fa50c078790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa5200b1a30 tx=0x7fa514008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:21.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.321+0000 7fa507fff640 1 -- 172.21.15.45:0/150424652 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa510020070 con 0x7fa52006e0f0 2024-05-08T05:42:21.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:21.555+0000 7fa525acf640 1 -- 172.21.15.45:0/150424652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa5201f29a0 con 0x7fa52006e0f0 2024-05-08T05:42:22.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.520+0000 7fa507fff640 1 -- 172.21.15.45:0/150424652 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fa5100609d0 con 0x7fa52006e0f0 2024-05-08T05:42:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.522+0000 7fa525acf640 1 -- 172.21.15.45:0/150424652 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa50c0762d0 msgr2=0x7fa50c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.522+0000 7fa525acf640 1 --2- 172.21.15.45:0/150424652 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa50c0762d0 0x7fa50c078790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa5200b1a30 tx=0x7fa514008040 comp rx=0 tx=0).stop 2024-05-08T05:42:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.522+0000 7fa525acf640 1 -- 172.21.15.45:0/150424652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa52006e0f0 msgr2=0x7fa5201f5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.522+0000 7fa525acf640 1 --2- 172.21.15.45:0/150424652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa52006e0f0 0x7fa5201f5fa0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa51000c460 tx=0x7fa51000c920 comp rx=0 tx=0).stop 2024-05-08T05:42:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.522+0000 7fa525acf640 1 -- 172.21.15.45:0/150424652 shutdown_connections 2024-05-08T05:42:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.522+0000 7fa525acf640 1 --2- 172.21.15.45:0/150424652 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa50c0762d0 0x7fa50c078790 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.522+0000 7fa525acf640 1 --2- 172.21.15.45:0/150424652 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa52006f060 0x7fa52009caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.522+0000 7fa525acf640 1 --2- 172.21.15.45:0/150424652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5200d0360 0x7fa52014a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.522+0000 7fa525acf640 1 --2- 172.21.15.45:0/150424652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa52006e0f0 0x7fa5201f5fa0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:22.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.522+0000 7fa525acf640 1 -- 172.21.15.45:0/150424652 >> 172.21.15.45:0/150424652 conn(0x7fa52006c3a0 msgr2=0x7fa5200cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.522+0000 7fa525acf640 1 -- 172.21.15.45:0/150424652 shutdown_connections 2024-05-08T05:42:22.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.523+0000 7fa525acf640 1 -- 172.21.15.45:0/150424652 wait complete. 2024-05-08T05:42:22.535 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T05:42:22.535 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T05:42:22.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.647+0000 7fae5fc1c640 1 Processor -- start 2024-05-08T05:42:22.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.647+0000 7fae5fc1c640 1 -- start start 2024-05-08T05:42:22.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.648+0000 7fae5fc1c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fae58157540 0x7fae5815e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:22.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.648+0000 7fae5fc1c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fae5806fc30 0x7fae580af740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:22.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.648+0000 7fae5fc1c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae5815e550 con 0x7fae580b48a0 2024-05-08T05:42:22.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.648+0000 7fae5fc1c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fae580701d0 con 0x7fae5806fc30 2024-05-08T05:42:22.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.648+0000 7fae5fc1c640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fae580b3dc0 con 0x7fae58157540 2024-05-08T05:42:22.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.648+0000 7fae5d1b7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fae5806fc30 0x7fae580af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:22.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.648+0000 7fae5e1b9640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fae580b48a0 0x7fae580ced20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56048/0 (socket says 172.21.15.45:56048) 2024-05-08T05:42:22.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.648+0000 7fae5d1b7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fae5806fc30 0x7fae580af740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44152/0 (socket says 172.21.15.45:44152) 2024-05-08T05:42:22.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.648+0000 7fae5e1b9640 1 -- 172.21.15.45:0/733725510 learned_addr learned my addr 172.21.15.45:0/733725510 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:22.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.648+0000 7fae5d9b8640 1 --2- 172.21.15.45:0/733725510 >> v2:172.21.15.63:3300/0 conn(0x7fae58157540 0x7fae5815e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:22.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.648+0000 7fae5d1b7640 1 -- 172.21.15.45:0/733725510 >> v2:172.21.15.63:3300/0 conn(0x7fae58157540 msgr2=0x7fae5815e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:22.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.648+0000 7fae5d1b7640 1 --2- 172.21.15.45:0/733725510 >> v2:172.21.15.63:3300/0 conn(0x7fae58157540 0x7fae5815e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:22.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.648+0000 7fae5d1b7640 1 -- 172.21.15.45:0/733725510 >> v1:172.21.15.45:6789/0 conn(0x7fae580b48a0 legacy=0x7fae580ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:22.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.648+0000 7fae5d1b7640 1 -- 172.21.15.45:0/733725510 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae580b40a0 con 0x7fae5806fc30 2024-05-08T05:42:22.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.649+0000 7fae5d1b7640 1 --2- 172.21.15.45:0/733725510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae5806fc30 0x7fae580af740 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fae580d17e0 tx=0x7fae4802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a7a972f02f98b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:22.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.649+0000 7fae5c9b6640 1 -- 172.21.15.45:0/733725510 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48037070 con 0x7fae5806fc30 2024-05-08T05:42:22.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.649+0000 7fae5c9b6640 1 -- 172.21.15.45:0/733725510 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae48032440 con 0x7fae5806fc30 2024-05-08T05:42:22.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.649+0000 7fae5c9b6640 1 -- 172.21.15.45:0/733725510 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48032810 con 0x7fae5806fc30 2024-05-08T05:42:22.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.650+0000 7fae5fc1c640 1 -- 172.21.15.45:0/733725510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae5806fc30 msgr2=0x7fae580af740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:22.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.650+0000 7fae5fc1c640 1 --2- 172.21.15.45:0/733725510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae5806fc30 0x7fae580af740 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fae580d17e0 tx=0x7fae4802a1a0 comp rx=0 tx=0).stop 2024-05-08T05:42:22.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.650+0000 7fae5fc1c640 1 -- 172.21.15.45:0/733725510 shutdown_connections 2024-05-08T05:42:22.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.650+0000 7fae5fc1c640 1 --2- 172.21.15.45:0/733725510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae5806fc30 0x7fae580af740 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:22.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.650+0000 7fae5fc1c640 1 --2- 172.21.15.45:0/733725510 >> v2:172.21.15.63:3300/0 conn(0x7fae58157540 0x7fae5815e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:22.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.650+0000 7fae5fc1c640 1 -- 172.21.15.45:0/733725510 >> 172.21.15.45:0/733725510 conn(0x7fae5806c3a0 msgr2=0x7fae5806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:22.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.650+0000 7fae5fc1c640 1 -- 172.21.15.45:0/733725510 shutdown_connections 2024-05-08T05:42:22.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.650+0000 7fae5fc1c640 1 -- 172.21.15.45:0/733725510 wait complete. 2024-05-08T05:42:22.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.651+0000 7fae5fc1c640 1 Processor -- start 2024-05-08T05:42:22.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.651+0000 7fae5fc1c640 1 -- start start 2024-05-08T05:42:22.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.652+0000 7fae5fc1c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae5806fc30 0x7fae58158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:22.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.652+0000 7fae5fc1c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae580b48a0 0x7fae581555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:22.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.652+0000 7fae5fc1c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae58157540 0x7fae5809c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:22.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.652+0000 7fae5fc1c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fae58157aa0 con 0x7fae58157540 2024-05-08T05:42:22.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.652+0000 7fae5fc1c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fae58157920 con 0x7fae580b48a0 2024-05-08T05:42:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.652+0000 7fae5fc1c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fae580afc80 con 0x7fae5806fc30 2024-05-08T05:42:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.652+0000 7fae5e1b9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae58157540 0x7fae5809c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.652+0000 7fae5e1b9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae58157540 0x7fae5809c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44162/0 (socket says 172.21.15.45:44162) 2024-05-08T05:42:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.652+0000 7fae5e1b9640 1 -- 172.21.15.45:0/3405516456 learned_addr learned my addr 172.21.15.45:0/3405516456 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.652+0000 7fae5d1b7640 1 --2- 172.21.15.45:0/3405516456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae580b48a0 0x7fae581555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.652+0000 7fae5d9b8640 1 --2- 172.21.15.45:0/3405516456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae5806fc30 0x7fae58158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.652+0000 7fae5e1b9640 1 -- 172.21.15.45:0/3405516456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae5806fc30 msgr2=0x7fae58158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.652+0000 7fae5e1b9640 1 --2- 172.21.15.45:0/3405516456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae5806fc30 0x7fae58158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.652+0000 7fae5e1b9640 1 -- 172.21.15.45:0/3405516456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae580b48a0 msgr2=0x7fae581555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.652+0000 7fae5e1b9640 1 --2- 172.21.15.45:0/3405516456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae580b48a0 0x7fae581555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.652+0000 7fae5e1b9640 1 -- 172.21.15.45:0/3405516456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae580722d0 con 0x7fae58157540 2024-05-08T05:42:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.652+0000 7fae5d9b8640 1 --2- 172.21.15.45:0/3405516456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae5806fc30 0x7fae58158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.653+0000 7fae5e1b9640 1 --2- 172.21.15.45:0/3405516456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae58157540 0x7fae5809c8b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fae5000cb50 tx=0x7fae5000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:22.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.653+0000 7fae4e7fc640 1 -- 172.21.15.45:0/3405516456 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50017070 con 0x7fae58157540 2024-05-08T05:42:22.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.653+0000 7fae4e7fc640 1 -- 172.21.15.45:0/3405516456 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae5000ad40 con 0x7fae58157540 2024-05-08T05:42:22.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.653+0000 7fae5fc1c640 1 -- 172.21.15.45:0/3405516456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae581593a0 con 0x7fae58157540 2024-05-08T05:42:22.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.653+0000 7fae4e7fc640 1 -- 172.21.15.45:0/3405516456 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50012600 con 0x7fae58157540 2024-05-08T05:42:22.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.653+0000 7fae5fc1c640 1 -- 172.21.15.45:0/3405516456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae58159960 con 0x7fae58157540 2024-05-08T05:42:22.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.654+0000 7fae5fc1c640 1 -- 172.21.15.45:0/3405516456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae580693f0 con 0x7fae58157540 2024-05-08T05:42:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.654+0000 7fae4e7fc640 1 -- 172.21.15.45:0/3405516456 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fae5001a070 con 0x7fae58157540 2024-05-08T05:42:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.655+0000 7fae4e7fc640 1 --2- 172.21.15.45:0/3405516456 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fae3c0762d0 0x7fae3c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.655+0000 7fae4e7fc640 1 -- 172.21.15.45:0/3405516456 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7fae50097330 con 0x7fae58157540 2024-05-08T05:42:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.655+0000 7fae5d9b8640 1 --2- 172.21.15.45:0/3405516456 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fae3c0762d0 0x7fae3c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:22.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.656+0000 7fae5d9b8640 1 --2- 172.21.15.45:0/3405516456 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fae3c0762d0 0x7fae3c078790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fae54009a20 tx=0x7fae54008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:22.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.659+0000 7fae4e7fc640 1 -- 172.21.15.45:0/3405516456 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fae580722d0 con 0x7fae58157540 2024-05-08T05:42:22.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:22.889+0000 7fae5fc1c640 1 -- 172.21.15.45:0/3405516456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fae580765b0 con 0x7fae58157540 2024-05-08T05:42:23.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.525+0000 7fae4e7fc640 1 -- 172.21.15.45:0/3405516456 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fae5815e550 con 0x7fae58157540 2024-05-08T05:42:23.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.527+0000 7fae5fc1c640 1 -- 172.21.15.45:0/3405516456 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fae3c0762d0 msgr2=0x7fae3c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:23.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.527+0000 7fae5fc1c640 1 --2- 172.21.15.45:0/3405516456 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fae3c0762d0 0x7fae3c078790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fae54009a20 tx=0x7fae54008040 comp rx=0 tx=0).stop 2024-05-08T05:42:23.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.527+0000 7fae5fc1c640 1 -- 172.21.15.45:0/3405516456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae58157540 msgr2=0x7fae5809c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:23.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.527+0000 7fae5fc1c640 1 --2- 172.21.15.45:0/3405516456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae58157540 0x7fae5809c8b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fae5000cb50 tx=0x7fae5000a430 comp rx=0 tx=0).stop 2024-05-08T05:42:23.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.528+0000 7fae5fc1c640 1 -- 172.21.15.45:0/3405516456 shutdown_connections 2024-05-08T05:42:23.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.528+0000 7fae5fc1c640 1 --2- 172.21.15.45:0/3405516456 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fae3c0762d0 0x7fae3c078790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:23.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.528+0000 7fae5fc1c640 1 --2- 172.21.15.45:0/3405516456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae58157540 0x7fae5809c8b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:23.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.528+0000 7fae5fc1c640 1 --2- 172.21.15.45:0/3405516456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fae580b48a0 0x7fae581555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:23.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.528+0000 7fae5fc1c640 1 --2- 172.21.15.45:0/3405516456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae5806fc30 0x7fae58158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:23.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.528+0000 7fae5fc1c640 1 -- 172.21.15.45:0/3405516456 >> 172.21.15.45:0/3405516456 conn(0x7fae5806c3a0 msgr2=0x7fae580b53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:23.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.528+0000 7fae5fc1c640 1 -- 172.21.15.45:0/3405516456 shutdown_connections 2024-05-08T05:42:23.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.528+0000 7fae5fc1c640 1 -- 172.21.15.45:0/3405516456 wait complete. 2024-05-08T05:42:23.541 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T05:42:23.541 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:42:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.653+0000 7f1518faf640 1 Processor -- start 2024-05-08T05:42:23.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.654+0000 7f1518faf640 1 -- start start 2024-05-08T05:42:23.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.654+0000 7f1518faf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1514162f50 0x7f15141633e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.655+0000 7f1518faf640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f15140cf790 0x7f15140cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.655+0000 7f1518faf640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15140d0500 con 0x7f15140b21f0 2024-05-08T05:42:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.655+0000 7f1518faf640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f151406d810 con 0x7f1514162f50 2024-05-08T05:42:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.655+0000 7f1518faf640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f151406e0b0 con 0x7f15140cf790 2024-05-08T05:42:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.655+0000 7f1511d9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1514162f50 0x7f15141633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.655+0000 7f1511d9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1514162f50 0x7f15141633e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44176/0 (socket says 172.21.15.45:44176) 2024-05-08T05:42:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.655+0000 7f1511d9b640 1 -- 172.21.15.45:0/4116696818 learned_addr learned my addr 172.21.15.45:0/4116696818 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.655+0000 7f1512d9d640 1 --2- 172.21.15.45:0/4116696818 >> v2:172.21.15.63:3300/0 conn(0x7f15140cf790 0x7f15140cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.655+0000 7f1511d9b640 1 -- 172.21.15.45:0/4116696818 >> v2:172.21.15.63:3300/0 conn(0x7f15140cf790 msgr2=0x7f15140cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.655+0000 7f1511d9b640 1 --2- 172.21.15.45:0/4116696818 >> v2:172.21.15.63:3300/0 conn(0x7f15140cf790 0x7f15140cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.655+0000 7f1511d9b640 1 -- 172.21.15.45:0/4116696818 >> v1:172.21.15.63:6789/0 conn(0x7f15140b21f0 legacy=0x7f151406e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:23.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.655+0000 7f1511d9b640 1 -- 172.21.15.45:0/4116696818 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151406da60 con 0x7f1514162f50 2024-05-08T05:42:23.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.656+0000 7f1511d9b640 1 --2- 172.21.15.45:0/4116696818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1514162f50 0x7f15141633e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f15140cf470 tx=0x7f14fc02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fc8d01992bdf6e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.656+0000 7f151159a640 1 -- 172.21.15.45:0/4116696818 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc037070 con 0x7f1514162f50 2024-05-08T05:42:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.656+0000 7f151159a640 1 -- 172.21.15.45:0/4116696818 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14fc032440 con 0x7f1514162f50 2024-05-08T05:42:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.656+0000 7f151159a640 1 -- 172.21.15.45:0/4116696818 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc032810 con 0x7f1514162f50 2024-05-08T05:42:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.656+0000 7f1518faf640 1 -- 172.21.15.45:0/4116696818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1514162f50 msgr2=0x7f15141633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.656+0000 7f1518faf640 1 --2- 172.21.15.45:0/4116696818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1514162f50 0x7f15141633e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f15140cf470 tx=0x7f14fc02a150 comp rx=0 tx=0).stop 2024-05-08T05:42:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.657+0000 7f1518faf640 1 -- 172.21.15.45:0/4116696818 shutdown_connections 2024-05-08T05:42:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.657+0000 7f1518faf640 1 --2- 172.21.15.45:0/4116696818 >> v2:172.21.15.63:3300/0 conn(0x7f15140cf790 0x7f15140cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.657+0000 7f1518faf640 1 --2- 172.21.15.45:0/4116696818 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1514162f50 0x7f15141633e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.657+0000 7f1518faf640 1 -- 172.21.15.45:0/4116696818 >> 172.21.15.45:0/4116696818 conn(0x7f151409ba40 msgr2=0x7f151409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.657+0000 7f1518faf640 1 -- 172.21.15.45:0/4116696818 shutdown_connections 2024-05-08T05:42:23.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.657+0000 7f1518faf640 1 -- 172.21.15.45:0/4116696818 wait complete. 2024-05-08T05:42:23.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.658+0000 7f1518faf640 1 Processor -- start 2024-05-08T05:42:23.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.658+0000 7f1518faf640 1 -- start start 2024-05-08T05:42:23.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.659+0000 7f1518faf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15140b21f0 0x7f15140c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.659+0000 7f1518faf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15140cf790 0x7f1514152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.659+0000 7f1518faf640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1514162f50 0x7f15140b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.659+0000 7f1518faf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f15140d0140 con 0x7f15140b21f0 2024-05-08T05:42:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.659+0000 7f1518faf640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f15140b25d0 con 0x7f1514162f50 2024-05-08T05:42:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.659+0000 7f1518faf640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f15140d02c0 con 0x7f15140cf790 2024-05-08T05:42:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.659+0000 7f151259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15140b21f0 0x7f15140c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.659+0000 7f151259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15140b21f0 0x7f15140c7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44180/0 (socket says 172.21.15.45:44180) 2024-05-08T05:42:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.659+0000 7f151259c640 1 -- 172.21.15.45:0/2743071272 learned_addr learned my addr 172.21.15.45:0/2743071272 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.659+0000 7f1512d9d640 1 --2- 172.21.15.45:0/2743071272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1514162f50 0x7f15140b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.659+0000 7f1511d9b640 1 --2- 172.21.15.45:0/2743071272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15140cf790 0x7f1514152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.659+0000 7f151259c640 1 -- 172.21.15.45:0/2743071272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15140cf790 msgr2=0x7f1514152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.659+0000 7f151259c640 1 --2- 172.21.15.45:0/2743071272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15140cf790 0x7f1514152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.659+0000 7f151259c640 1 -- 172.21.15.45:0/2743071272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1514162f50 msgr2=0x7f15140b81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:23.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.659+0000 7f151259c640 1 --2- 172.21.15.45:0/2743071272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1514162f50 0x7f15140b81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:23.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.659+0000 7f151259c640 1 -- 172.21.15.45:0/2743071272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15140b5c80 con 0x7f15140b21f0 2024-05-08T05:42:23.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.659+0000 7f151259c640 1 --2- 172.21.15.45:0/2743071272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15140b21f0 0x7f15140c7010 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f150400c870 tx=0x7f150400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:23.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.660+0000 7f150affd640 1 -- 172.21.15.45:0/2743071272 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504003b70 con 0x7f15140b21f0 2024-05-08T05:42:23.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.660+0000 7f150affd640 1 -- 172.21.15.45:0/2743071272 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1504003d10 con 0x7f15140b21f0 2024-05-08T05:42:23.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.660+0000 7f1518faf640 1 -- 172.21.15.45:0/2743071272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15140b5f60 con 0x7f15140b21f0 2024-05-08T05:42:23.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.660+0000 7f150affd640 1 -- 172.21.15.45:0/2743071272 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150400aa90 con 0x7f15140b21f0 2024-05-08T05:42:23.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.660+0000 7f1518faf640 1 -- 172.21.15.45:0/2743071272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15140b79d0 con 0x7f15140b21f0 2024-05-08T05:42:23.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.661+0000 7f1518faf640 1 -- 172.21.15.45:0/2743071272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1514069b00 con 0x7f15140b21f0 2024-05-08T05:42:23.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.661+0000 7f150affd640 1 -- 172.21.15.45:0/2743071272 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f150400ac30 con 0x7f15140b21f0 2024-05-08T05:42:23.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.662+0000 7f150affd640 1 --2- 172.21.15.45:0/2743071272 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f14f40762d0 0x7f14f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:23.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.662+0000 7f150affd640 1 -- 172.21.15.45:0/2743071272 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f1504097720 con 0x7f15140b21f0 2024-05-08T05:42:23.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.662+0000 7f1511d9b640 1 --2- 172.21.15.45:0/2743071272 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f14f40762d0 0x7f14f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:23.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.663+0000 7f1511d9b640 1 --2- 172.21.15.45:0/2743071272 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f14f40762d0 0x7f14f4078790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14fc034000 tx=0x7f14fc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:23.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.666+0000 7f150affd640 1 -- 172.21.15.45:0/2743071272 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1504060e50 con 0x7f15140b21f0 2024-05-08T05:42:23.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.897+0000 7f1518faf640 1 -- 172.21.15.45:0/2743071272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1514070a30 con 0x7f15140b21f0 2024-05-08T05:42:23.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.897+0000 7f150affd640 1 -- 172.21.15.45:0/2743071272 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5172 (secure 0 0 0) 0x7f1504060750 con 0x7f15140b21f0 2024-05-08T05:42:23.898 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:23.898 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3295938044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3295938044},{"type":"v1","addr":"172.21.15.63:6836","nonce":3295938044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4024530715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4024530715},{"type":"v1","addr":"172.21.15.63:6837","nonce":4024530715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/97258541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":97258541},{"type":"v1","addr":"172.21.15.45:6834","nonce":97258541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:42:19.860331+0000","modified":"2024-05-08T05:42:22.891006+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4884":{"gid":4884,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/173436628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":173436628},{"type":"v1","addr":"172.21.15.45:6835","nonce":173436628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4884,"qdb_cluster":[4884]},"id":2}]} 2024-05-08T05:42:23.898 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2024-05-08T05:42:23.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.899+0000 7f1518faf640 1 -- 172.21.15.45:0/2743071272 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f14f40762d0 msgr2=0x7f14f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:23.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.899+0000 7f1518faf640 1 --2- 172.21.15.45:0/2743071272 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f14f40762d0 0x7f14f4078790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14fc034000 tx=0x7f14fc02e000 comp rx=0 tx=0).stop 2024-05-08T05:42:23.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.899+0000 7f1518faf640 1 -- 172.21.15.45:0/2743071272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15140b21f0 msgr2=0x7f15140c7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:23.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.899+0000 7f1518faf640 1 --2- 172.21.15.45:0/2743071272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15140b21f0 0x7f15140c7010 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f150400c870 tx=0x7f150400cd30 comp rx=0 tx=0).stop 2024-05-08T05:42:23.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.899+0000 7f1518faf640 1 -- 172.21.15.45:0/2743071272 shutdown_connections 2024-05-08T05:42:23.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.899+0000 7f1518faf640 1 --2- 172.21.15.45:0/2743071272 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f14f40762d0 0x7f14f4078790 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:23.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.899+0000 7f1518faf640 1 --2- 172.21.15.45:0/2743071272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15140cf790 0x7f1514152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:23.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.899+0000 7f1518faf640 1 --2- 172.21.15.45:0/2743071272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1514162f50 0x7f15140b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:23.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.899+0000 7f1518faf640 1 --2- 172.21.15.45:0/2743071272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15140b21f0 0x7f15140c7010 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:23.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.899+0000 7f1518faf640 1 -- 172.21.15.45:0/2743071272 >> 172.21.15.45:0/2743071272 conn(0x7f151409ba40 msgr2=0x7f15140d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:23.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.899+0000 7f1518faf640 1 -- 172.21.15.45:0/2743071272 shutdown_connections 2024-05-08T05:42:23.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:23.899+0000 7f1518faf640 1 -- 172.21.15.45:0/2743071272 wait complete. 2024-05-08T05:42:23.911 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:42:24.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.014+0000 7fe602be4640 1 Processor -- start 2024-05-08T05:42:24.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.014+0000 7fe602be4640 1 -- start start 2024-05-08T05:42:24.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.015+0000 7fe602be4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe5fc162f50 0x7fe5fc1633e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.015+0000 7fe602be4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5fc0cf790 0x7fe5fc0cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.015+0000 7fe602be4640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5fc0d0500 con 0x7fe5fc0b21f0 2024-05-08T05:42:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.015+0000 7fe602be4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe5fc06d810 con 0x7fe5fc162f50 2024-05-08T05:42:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.015+0000 7fe602be4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe5fc06e0b0 con 0x7fe5fc0cf790 2024-05-08T05:42:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.016+0000 7fe601181640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5fc0cf790 0x7fe5fc0cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.016+0000 7fe5fbfff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe5fc162f50 0x7fe5fc1633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.016+0000 7fe600980640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fe5fc0b21f0 0x7fe5fc06e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:37582/0 (socket says 172.21.15.45:37582) 2024-05-08T05:42:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.016+0000 7fe5fbfff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe5fc162f50 0x7fe5fc1633e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:57176/0 (socket says 172.21.15.45:57176) 2024-05-08T05:42:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.016+0000 7fe601181640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5fc0cf790 0x7fe5fc0cfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:39078/0 (socket says 172.21.15.45:39078) 2024-05-08T05:42:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.016+0000 7fe600980640 1 -- 172.21.15.45:0/971940908 learned_addr learned my addr 172.21.15.45:0/971940908 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.016+0000 7fe5fbfff640 1 -- 172.21.15.45:0/971940908 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5fc0cf790 msgr2=0x7fe5fc0cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.016+0000 7fe5fbfff640 1 --2- 172.21.15.45:0/971940908 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5fc0cf790 0x7fe5fc0cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.016+0000 7fe5fbfff640 1 -- 172.21.15.45:0/971940908 >> v1:172.21.15.63:6789/0 conn(0x7fe5fc0b21f0 legacy=0x7fe5fc06e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.016+0000 7fe5fbfff640 1 -- 172.21.15.45:0/971940908 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5fc06da60 con 0x7fe5fc162f50 2024-05-08T05:42:24.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.017+0000 7fe5fbfff640 1 --2- 172.21.15.45:0/971940908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5fc162f50 0x7fe5fc1633e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00097a0 tx=0x7fe5f002a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe3271154a768798 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:24.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.017+0000 7fe5fb7fe640 1 -- 172.21.15.45:0/971940908 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0037070 con 0x7fe5fc162f50 2024-05-08T05:42:24.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.017+0000 7fe5fb7fe640 1 -- 172.21.15.45:0/971940908 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f002adb0 con 0x7fe5fc162f50 2024-05-08T05:42:24.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.017+0000 7fe5fb7fe640 1 -- 172.21.15.45:0/971940908 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0032660 con 0x7fe5fc162f50 2024-05-08T05:42:24.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.018+0000 7fe602be4640 1 -- 172.21.15.45:0/971940908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5fc162f50 msgr2=0x7fe5fc1633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.018+0000 7fe602be4640 1 --2- 172.21.15.45:0/971940908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5fc162f50 0x7fe5fc1633e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00097a0 tx=0x7fe5f002a060 comp rx=0 tx=0).stop 2024-05-08T05:42:24.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.018+0000 7fe602be4640 1 -- 172.21.15.45:0/971940908 shutdown_connections 2024-05-08T05:42:24.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.018+0000 7fe602be4640 1 --2- 172.21.15.45:0/971940908 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5fc0cf790 0x7fe5fc0cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.018+0000 7fe602be4640 1 --2- 172.21.15.45:0/971940908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5fc162f50 0x7fe5fc1633e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.018+0000 7fe602be4640 1 -- 172.21.15.45:0/971940908 >> 172.21.15.45:0/971940908 conn(0x7fe5fc09ba40 msgr2=0x7fe5fc09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:24.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.018+0000 7fe602be4640 1 -- 172.21.15.45:0/971940908 shutdown_connections 2024-05-08T05:42:24.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.018+0000 7fe602be4640 1 -- 172.21.15.45:0/971940908 wait complete. 2024-05-08T05:42:24.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.019+0000 7fe602be4640 1 Processor -- start 2024-05-08T05:42:24.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.019+0000 7fe602be4640 1 -- start start 2024-05-08T05:42:24.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.020+0000 7fe602be4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5fc0b21f0 0x7fe5fc156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.020+0000 7fe602be4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5fc0cf790 0x7fe5fc0c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.020+0000 7fe602be4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5fc162f50 0x7fe5fc0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.020+0000 7fe602be4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc0d0140 con 0x7fe5fc162f50 2024-05-08T05:42:24.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.020+0000 7fe602be4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc0b25d0 con 0x7fe5fc0cf790 2024-05-08T05:42:24.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.020+0000 7fe602be4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe5fc0d02c0 con 0x7fe5fc0b21f0 2024-05-08T05:42:24.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.020+0000 7fe601181640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5fc162f50 0x7fe5fc0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.020+0000 7fe601181640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5fc162f50 0x7fe5fc0c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44192/0 (socket says 172.21.15.45:44192) 2024-05-08T05:42:24.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.020+0000 7fe601181640 1 -- 172.21.15.45:0/1309521181 learned_addr learned my addr 172.21.15.45:0/1309521181 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:24.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.020+0000 7fe5fbfff640 1 --2- 172.21.15.45:0/1309521181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5fc0cf790 0x7fe5fc0c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.020+0000 7fe600980640 1 --2- 172.21.15.45:0/1309521181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5fc0b21f0 0x7fe5fc156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.020+0000 7fe601181640 1 -- 172.21.15.45:0/1309521181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5fc0b21f0 msgr2=0x7fe5fc156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.020+0000 7fe601181640 1 --2- 172.21.15.45:0/1309521181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5fc0b21f0 0x7fe5fc156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.020+0000 7fe601181640 1 -- 172.21.15.45:0/1309521181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5fc0cf790 msgr2=0x7fe5fc0c67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.020+0000 7fe601181640 1 --2- 172.21.15.45:0/1309521181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5fc0cf790 0x7fe5fc0c67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.020+0000 7fe601181640 1 -- 172.21.15.45:0/1309521181 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5fc0b52d0 con 0x7fe5fc162f50 2024-05-08T05:42:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.020+0000 7fe601181640 1 --2- 172.21.15.45:0/1309521181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5fc162f50 0x7fe5fc0c9670 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00e930 tx=0x7fe5ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.021+0000 7fe5f97fa640 1 -- 172.21.15.45:0/1309521181 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec017070 con 0x7fe5fc162f50 2024-05-08T05:42:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.021+0000 7fe5f97fa640 1 -- 172.21.15.45:0/1309521181 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5ec00c980 con 0x7fe5fc162f50 2024-05-08T05:42:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.021+0000 7fe602be4640 1 -- 172.21.15.45:0/1309521181 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5fc0b55b0 con 0x7fe5fc162f50 2024-05-08T05:42:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.021+0000 7fe5f97fa640 1 -- 172.21.15.45:0/1309521181 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec00cd70 con 0x7fe5fc162f50 2024-05-08T05:42:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.021+0000 7fe602be4640 1 -- 172.21.15.45:0/1309521181 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5fc0b4fc0 con 0x7fe5fc162f50 2024-05-08T05:42:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.022+0000 7fe602be4640 1 -- 172.21.15.45:0/1309521181 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5fc069b00 con 0x7fe5fc162f50 2024-05-08T05:42:24.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.022+0000 7fe5f97fa640 1 -- 172.21.15.45:0/1309521181 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe5ec01a070 con 0x7fe5fc162f50 2024-05-08T05:42:24.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.023+0000 7fe5f97fa640 1 --2- 172.21.15.45:0/1309521181 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe5e80762d0 0x7fe5e8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.023+0000 7fe5f97fa640 1 -- 172.21.15.45:0/1309521181 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7fe5ec097a10 con 0x7fe5fc162f50 2024-05-08T05:42:24.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.023+0000 7fe600980640 1 --2- 172.21.15.45:0/1309521181 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe5e80762d0 0x7fe5e8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.024+0000 7fe600980640 1 --2- 172.21.15.45:0/1309521181 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe5e80762d0 0x7fe5e8078790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40012e0 tx=0x7fe5f4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:24.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.027+0000 7fe5f97fa640 1 -- 172.21.15.45:0/1309521181 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe5ec060ce0 con 0x7fe5fc162f50 2024-05-08T05:42:24.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.218+0000 7fe602be4640 1 -- 172.21.15.45:0/1309521181 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5fc070a30 con 0x7fe5fc162f50 2024-05-08T05:42:24.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.219+0000 7fe5f97fa640 1 -- 172.21.15.45:0/1309521181 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16894 (secure 0 0 0) 0x7fe5ec064fe0 con 0x7fe5fc162f50 2024-05-08T05:42:24.220 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:24.220 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":36,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:42:21.508281+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:42:17.918020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-08T05:42:18.816243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:42:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.221+0000 7fe602be4640 1 -- 172.21.15.45:0/1309521181 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe5e80762d0 msgr2=0x7fe5e8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.221+0000 7fe602be4640 1 --2- 172.21.15.45:0/1309521181 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe5e80762d0 0x7fe5e8078790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40012e0 tx=0x7fe5f4009040 comp rx=0 tx=0).stop 2024-05-08T05:42:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.221+0000 7fe602be4640 1 -- 172.21.15.45:0/1309521181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5fc162f50 msgr2=0x7fe5fc0c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.221+0000 7fe602be4640 1 --2- 172.21.15.45:0/1309521181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5fc162f50 0x7fe5fc0c9670 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00e930 tx=0x7fe5ec00edf0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.222+0000 7fe602be4640 1 -- 172.21.15.45:0/1309521181 shutdown_connections 2024-05-08T05:42:24.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.222+0000 7fe602be4640 1 --2- 172.21.15.45:0/1309521181 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe5e80762d0 0x7fe5e8078790 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.222+0000 7fe602be4640 1 --2- 172.21.15.45:0/1309521181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe5fc162f50 0x7fe5fc0c9670 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.222+0000 7fe602be4640 1 --2- 172.21.15.45:0/1309521181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5fc0cf790 0x7fe5fc0c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.222+0000 7fe602be4640 1 --2- 172.21.15.45:0/1309521181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe5fc0b21f0 0x7fe5fc156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.222+0000 7fe602be4640 1 -- 172.21.15.45:0/1309521181 >> 172.21.15.45:0/1309521181 conn(0x7fe5fc09ba40 msgr2=0x7fe5fc163380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:24.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.222+0000 7fe602be4640 1 -- 172.21.15.45:0/1309521181 shutdown_connections 2024-05-08T05:42:24.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.222+0000 7fe602be4640 1 -- 172.21.15.45:0/1309521181 wait complete. 2024-05-08T05:42:24.233 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T05:42:24.233 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:42:24.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.338+0000 7fc827d9d640 1 Processor -- start 2024-05-08T05:42:24.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.338+0000 7fc827d9d640 1 -- start start 2024-05-08T05:42:24.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.339+0000 7fc827d9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8201629a0 0x7fc8200d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.339+0000 7fc827d9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc8200d0cb0 0x7fc8200d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.339+0000 7fc827d9d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8200d3330 con 0x7fc8200d2dc0 2024-05-08T05:42:24.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.339+0000 7fc827d9d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc8200d2170 con 0x7fc8200d0cb0 2024-05-08T05:42:24.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.339+0000 7fc827d9d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc8200d2450 con 0x7fc8201629a0 2024-05-08T05:42:24.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.339+0000 7fc825b39640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8201629a0 0x7fc8200d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.339+0000 7fc82633a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc8200d0cb0 0x7fc8200d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.339+0000 7fc825b39640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8201629a0 0x7fc8200d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:39100/0 (socket says 172.21.15.45:39100) 2024-05-08T05:42:24.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.339+0000 7fc825b39640 1 -- 172.21.15.45:0/442024085 learned_addr learned my addr 172.21.15.45:0/442024085 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:24.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.339+0000 7fc825338640 1 --1- 172.21.15.45:0/442024085 >> v1:172.21.15.63:6789/0 conn(0x7fc8200d2dc0 0x7fc8200c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:37588/0 (socket says 172.21.15.45:37588) 2024-05-08T05:42:24.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.339+0000 7fc825b39640 1 -- 172.21.15.45:0/442024085 >> v2:172.21.15.63:3300/0 conn(0x7fc8200d0cb0 msgr2=0x7fc8200d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:24.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.339+0000 7fc825b39640 1 --2- 172.21.15.45:0/442024085 >> v2:172.21.15.63:3300/0 conn(0x7fc8200d0cb0 0x7fc8200d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.339+0000 7fc825b39640 1 -- 172.21.15.45:0/442024085 >> v1:172.21.15.63:6789/0 conn(0x7fc8200d2dc0 legacy=0x7fc8200c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.339+0000 7fc825b39640 1 -- 172.21.15.45:0/442024085 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8200d2730 con 0x7fc8201629a0 2024-05-08T05:42:24.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.340+0000 7fc825b39640 1 --2- 172.21.15.45:0/442024085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8201629a0 0x7fc8200d35c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc810009870 tx=0x7fc81002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11674c94e40dfde server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:24.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.340+0000 7fc824b37640 1 -- 172.21.15.45:0/442024085 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc810037070 con 0x7fc8201629a0 2024-05-08T05:42:24.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.340+0000 7fc824b37640 1 -- 172.21.15.45:0/442024085 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc810032440 con 0x7fc8201629a0 2024-05-08T05:42:24.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.340+0000 7fc824b37640 1 -- 172.21.15.45:0/442024085 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc810032810 con 0x7fc8201629a0 2024-05-08T05:42:24.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.341+0000 7fc827d9d640 1 -- 172.21.15.45:0/442024085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8201629a0 msgr2=0x7fc8200d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.341+0000 7fc827d9d640 1 --2- 172.21.15.45:0/442024085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8201629a0 0x7fc8200d35c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc810009870 tx=0x7fc81002a130 comp rx=0 tx=0).stop 2024-05-08T05:42:24.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.341+0000 7fc827d9d640 1 -- 172.21.15.45:0/442024085 shutdown_connections 2024-05-08T05:42:24.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.341+0000 7fc827d9d640 1 --2- 172.21.15.45:0/442024085 >> v2:172.21.15.63:3300/0 conn(0x7fc8200d0cb0 0x7fc8200d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.341+0000 7fc827d9d640 1 --2- 172.21.15.45:0/442024085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8201629a0 0x7fc8200d35c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.341+0000 7fc827d9d640 1 -- 172.21.15.45:0/442024085 >> 172.21.15.45:0/442024085 conn(0x7fc8200d4fe0 msgr2=0x7fc8200d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:24.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.341+0000 7fc827d9d640 1 -- 172.21.15.45:0/442024085 shutdown_connections 2024-05-08T05:42:24.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.341+0000 7fc827d9d640 1 -- 172.21.15.45:0/442024085 wait complete. 2024-05-08T05:42:24.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.342+0000 7fc827d9d640 1 Processor -- start 2024-05-08T05:42:24.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.342+0000 7fc827d9d640 1 -- start start 2024-05-08T05:42:24.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.343+0000 7fc827d9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8200d0cb0 0x7fc8200b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.343+0000 7fc827d9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8200d2dc0 0x7fc8201f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.343+0000 7fc827d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8201629a0 0x7fc820156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.343+0000 7fc827d9d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc8200cf440 con 0x7fc8201629a0 2024-05-08T05:42:24.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.343+0000 7fc827d9d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc8200cf2c0 con 0x7fc8200d0cb0 2024-05-08T05:42:24.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.343+0000 7fc827d9d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc8200c95e0 con 0x7fc8200d2dc0 2024-05-08T05:42:24.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.343+0000 7fc82633a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8201629a0 0x7fc820156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.343+0000 7fc82633a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8201629a0 0x7fc820156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44208/0 (socket says 172.21.15.45:44208) 2024-05-08T05:42:24.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.343+0000 7fc82633a640 1 -- 172.21.15.45:0/3405727066 learned_addr learned my addr 172.21.15.45:0/3405727066 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:24.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.343+0000 7fc825338640 1 --2- 172.21.15.45:0/3405727066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8200d2dc0 0x7fc8201f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.343+0000 7fc825b39640 1 --2- 172.21.15.45:0/3405727066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8200d0cb0 0x7fc8200b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.343+0000 7fc82633a640 1 -- 172.21.15.45:0/3405727066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8200d2dc0 msgr2=0x7fc8201f4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.343+0000 7fc82633a640 1 --2- 172.21.15.45:0/3405727066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8200d2dc0 0x7fc8201f4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.343+0000 7fc82633a640 1 -- 172.21.15.45:0/3405727066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8200d0cb0 msgr2=0x7fc8200b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.343+0000 7fc82633a640 1 --2- 172.21.15.45:0/3405727066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8200d0cb0 0x7fc8200b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.343+0000 7fc82633a640 1 -- 172.21.15.45:0/3405727066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8201f5170 con 0x7fc8201629a0 2024-05-08T05:42:24.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.344+0000 7fc82633a640 1 --2- 172.21.15.45:0/3405727066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8201629a0 0x7fc820156520 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc81800efc0 tx=0x7fc81800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:24.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.344+0000 7fc8167fc640 1 -- 172.21.15.45:0/3405727066 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818018020 con 0x7fc8201629a0 2024-05-08T05:42:24.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.344+0000 7fc8167fc640 1 -- 172.21.15.45:0/3405727066 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc818014480 con 0x7fc8201629a0 2024-05-08T05:42:24.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.344+0000 7fc827d9d640 1 -- 172.21.15.45:0/3405727066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8201f53f0 con 0x7fc8201629a0 2024-05-08T05:42:24.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.344+0000 7fc8167fc640 1 -- 172.21.15.45:0/3405727066 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818014870 con 0x7fc8201629a0 2024-05-08T05:42:24.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.344+0000 7fc827d9d640 1 -- 172.21.15.45:0/3405727066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8201f59b0 con 0x7fc8201629a0 2024-05-08T05:42:24.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.345+0000 7fc827d9d640 1 -- 172.21.15.45:0/3405727066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8200693f0 con 0x7fc8201629a0 2024-05-08T05:42:24.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.345+0000 7fc8167fc640 1 -- 172.21.15.45:0/3405727066 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc818005d10 con 0x7fc8201629a0 2024-05-08T05:42:24.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.346+0000 7fc8167fc640 1 --2- 172.21.15.45:0/3405727066 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc8040762d0 0x7fc804078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.346+0000 7fc8167fc640 1 -- 172.21.15.45:0/3405727066 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7fc818098110 con 0x7fc8201629a0 2024-05-08T05:42:24.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.346+0000 7fc825b39640 1 --2- 172.21.15.45:0/3405727066 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc8040762d0 0x7fc804078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.347+0000 7fc825b39640 1 --2- 172.21.15.45:0/3405727066 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc8040762d0 0x7fc804078790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc810034000 tx=0x7fc81002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:24.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.350+0000 7fc8167fc640 1 -- 172.21.15.45:0/3405727066 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc818022070 con 0x7fc8201629a0 2024-05-08T05:42:24.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.540+0000 7fc827d9d640 1 -- 172.21.15.45:0/3405727066 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc8200d1e00 con 0x7fc8040762d0 2024-05-08T05:42:24.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.547+0000 7fc8167fc640 1 -- 172.21.15.45:0/3405727066 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242603 (secure 0 0 0) 0x7fc8200d1e00 con 0x7fc8040762d0 2024-05-08T05:42:24.550 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:24.551 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:42:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.550+0000 7fc827d9d640 1 -- 172.21.15.45:0/3405727066 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc8040762d0 msgr2=0x7fc804078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.550+0000 7fc827d9d640 1 --2- 172.21.15.45:0/3405727066 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc8040762d0 0x7fc804078790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc810034000 tx=0x7fc81002e000 comp rx=0 tx=0).stop 2024-05-08T05:42:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.550+0000 7fc827d9d640 1 -- 172.21.15.45:0/3405727066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8201629a0 msgr2=0x7fc820156520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.550+0000 7fc827d9d640 1 --2- 172.21.15.45:0/3405727066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8201629a0 0x7fc820156520 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc81800efc0 tx=0x7fc81800c490 comp rx=0 tx=0).stop 2024-05-08T05:42:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.550+0000 7fc827d9d640 1 -- 172.21.15.45:0/3405727066 shutdown_connections 2024-05-08T05:42:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.550+0000 7fc827d9d640 1 --2- 172.21.15.45:0/3405727066 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc8040762d0 0x7fc804078790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.550+0000 7fc827d9d640 1 --2- 172.21.15.45:0/3405727066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8200d2dc0 0x7fc8201f4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.550+0000 7fc827d9d640 1 --2- 172.21.15.45:0/3405727066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8201629a0 0x7fc820156520 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.550+0000 7fc827d9d640 1 --2- 172.21.15.45:0/3405727066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8200d0cb0 0x7fc8200b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.550+0000 7fc827d9d640 1 -- 172.21.15.45:0/3405727066 >> 172.21.15.45:0/3405727066 conn(0x7fc8200d4fe0 msgr2=0x7fc82006cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.550+0000 7fc827d9d640 1 -- 172.21.15.45:0/3405727066 shutdown_connections 2024-05-08T05:42:24.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.550+0000 7fc827d9d640 1 -- 172.21.15.45:0/3405727066 wait complete. 2024-05-08T05:42:24.561 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-05-08T05:42:22.845226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":120,"acting":120,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236872,"kb_used_data":4048,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737848,"statfs":{"total":773094113280,"available":772851556352,"internally_reserved":0,"allocated":4145152,"data_stored":2400336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.366890"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.494063+0000","last_change":"2024-05-08T05:42:19.494063+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.494063+0000","last_undegraded":"2024-05-08T05:42:19.494063+0000","last_fullsized":"2024-05-08T05:42:19.494063+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.492151+0000","last_change":"2024-05-08T05:42:19.492151+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.492151+0000","last_undegraded":"2024-05-08T05:42:19.492151+0000","last_fullsized":"2024-05-08T05:42:19.492151+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.489300+0000","last_change":"2024-05-08T05:42:19.489300+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.489300+0000","last_undegraded":"2024-05-08T05:42:19.489300+0000","last_fullsized":"2024-05-08T05:42:19.489300+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.488571+0000","last_change":"2024-05-08T05:42:19.488571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.488571+0000","last_undegraded":"2024-05-08T05:42:19.488571+0000","last_fullsized":"2024-05-08T05:42:19.488571+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.491652+0000","last_change":"2024-05-08T05:42:19.491652+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.491652+0000","last_undegraded":"2024-05-08T05:42:19.491652+0000","last_fullsized":"2024-05-08T05:42:19.491652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.483214+0000","last_change":"2024-05-08T05:42:19.483214+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.483214+0000","last_undegraded":"2024-05-08T05:42:19.483214+0000","last_fullsized":"2024-05-08T05:42:19.483214+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.482113+0000","last_change":"2024-05-08T05:42:19.482113+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.482113+0000","last_undegraded":"2024-05-08T05:42:19.482113+0000","last_fullsized":"2024-05-08T05:42:19.482113+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.486278+0000","last_change":"2024-05-08T05:42:19.486278+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.486278+0000","last_undegraded":"2024-05-08T05:42:19.486278+0000","last_fullsized":"2024-05-08T05:42:19.486278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.489219+0000","last_change":"2024-05-08T05:42:19.489219+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.489219+0000","last_undegraded":"2024-05-08T05:42:19.489219+0000","last_fullsized":"2024-05-08T05:42:19.489219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.489042+0000","last_change":"2024-05-08T05:42:19.489042+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.489042+0000","last_undegraded":"2024-05-08T05:42:19.489042+0000","last_fullsized":"2024-05-08T05:42:19.489042+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.481502+0000","last_change":"2024-05-08T05:42:19.481502+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.481502+0000","last_undegraded":"2024-05-08T05:42:19.481502+0000","last_fullsized":"2024-05-08T05:42:19.481502+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.485694+0000","last_change":"2024-05-08T05:42:19.485694+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.485694+0000","last_undegraded":"2024-05-08T05:42:19.485694+0000","last_fullsized":"2024-05-08T05:42:19.485694+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.486709+0000","last_change":"2024-05-08T05:42:19.486709+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.486709+0000","last_undegraded":"2024-05-08T05:42:19.486709+0000","last_fullsized":"2024-05-08T05:42:19.486709+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.488095+0000","last_change":"2024-05-08T05:42:19.488095+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.488095+0000","last_undegraded":"2024-05-08T05:42:19.488095+0000","last_fullsized":"2024-05-08T05:42:19.488095+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.488425+0000","last_change":"2024-05-08T05:42:19.488425+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.488425+0000","last_undegraded":"2024-05-08T05:42:19.488425+0000","last_fullsized":"2024-05-08T05:42:19.488425+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.485913+0000","last_change":"2024-05-08T05:42:19.485913+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.485913+0000","last_undegraded":"2024-05-08T05:42:19.485913+0000","last_fullsized":"2024-05-08T05:42:19.485913+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.481029+0000","last_change":"2024-05-08T05:42:19.481029+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.481029+0000","last_undegraded":"2024-05-08T05:42:19.481029+0000","last_fullsized":"2024-05-08T05:42:19.481029+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.481092+0000","last_change":"2024-05-08T05:42:19.481092+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.481092+0000","last_undegraded":"2024-05-08T05:42:19.481092+0000","last_fullsized":"2024-05-08T05:42:19.481092+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.497640+0000","last_change":"2024-05-08T05:42:19.493864+0000","last_active":"2024-05-08T05:42:19.497640+0000","last_peered":"2024-05-08T05:42:19.497640+0000","last_clean":"2024-05-08T05:42:19.497640+0000","last_became_active":"2024-05-08T05:42:19.493061+0000","last_became_peered":"2024-05-08T05:42:19.493061+0000","last_unstale":"2024-05-08T05:42:19.497640+0000","last_undegraded":"2024-05-08T05:42:19.497640+0000","last_fullsized":"2024-05-08T05:42:19.497640+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:47:27.293889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-08T05:42:18.476419+0000","last_change":"2024-05-08T05:42:18.476419+0000","last_active":"2024-05-08T05:42:18.469813+0000","last_peered":"2024-05-08T05:42:18.469813+0000","last_clean":"2024-05-08T05:42:18.469813+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:18.476419+0000","last_undegraded":"2024-05-08T05:42:18.476419+0000","last_fullsized":"2024-05-08T05:42:18.476419+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-08T05:42:18.478245+0000","last_change":"2024-05-08T05:42:18.478245+0000","last_active":"2024-05-08T05:42:18.469813+0000","last_peered":"2024-05-08T05:42:18.469813+0000","last_clean":"2024-05-08T05:42:18.469813+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:18.478245+0000","last_undegraded":"2024-05-08T05:42:18.478245+0000","last_fullsized":"2024-05-08T05:42:18.478245+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-08T05:42:18.481858+0000","last_change":"2024-05-08T05:42:18.481858+0000","last_active":"2024-05-08T05:42:18.469813+0000","last_peered":"2024-05-08T05:42:18.469813+0000","last_clean":"2024-05-08T05:42:18.469813+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:18.481858+0000","last_undegraded":"2024-05-08T05:42:18.481858+0000","last_fullsized":"2024-05-08T05:42:18.481858+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.501668+0000","last_change":"2024-05-08T05:42:19.499533+0000","last_active":"2024-05-08T05:42:19.501668+0000","last_peered":"2024-05-08T05:42:19.501668+0000","last_clean":"2024-05-08T05:42:19.501668+0000","last_became_active":"2024-05-08T05:42:19.498447+0000","last_became_peered":"2024-05-08T05:42:19.498447+0000","last_unstale":"2024-05-08T05:42:19.501668+0000","last_undegraded":"2024-05-08T05:42:19.501668+0000","last_fullsized":"2024-05-08T05:42:19.501668+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:37.041564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.488852+0000","last_change":"2024-05-08T05:42:19.488852+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.488852+0000","last_undegraded":"2024-05-08T05:42:19.488852+0000","last_fullsized":"2024-05-08T05:42:19.488852+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.497089+0000","last_change":"2024-05-08T05:42:19.495816+0000","last_active":"2024-05-08T05:42:19.497089+0000","last_peered":"2024-05-08T05:42:19.497089+0000","last_clean":"2024-05-08T05:42:19.497089+0000","last_became_active":"2024-05-08T05:42:19.495546+0000","last_became_peered":"2024-05-08T05:42:19.495546+0000","last_unstale":"2024-05-08T05:42:19.497089+0000","last_undegraded":"2024-05-08T05:42:19.497089+0000","last_fullsized":"2024-05-08T05:42:19.497089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:26.053156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-08T05:42:18.478240+0000","last_change":"2024-05-08T05:42:18.478240+0000","last_active":"2024-05-08T05:42:18.469813+0000","last_peered":"2024-05-08T05:42:18.469813+0000","last_clean":"2024-05-08T05:42:18.469813+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:18.478240+0000","last_undegraded":"2024-05-08T05:42:18.478240+0000","last_fullsized":"2024-05-08T05:42:18.478240+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-08T05:42:18.481141+0000","last_change":"2024-05-08T05:42:18.481141+0000","last_active":"2024-05-08T05:42:18.469813+0000","last_peered":"2024-05-08T05:42:18.469813+0000","last_clean":"2024-05-08T05:42:18.469813+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:18.481141+0000","last_undegraded":"2024-05-08T05:42:18.481141+0000","last_fullsized":"2024-05-08T05:42:18.481141+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-08T05:42:18.484430+0000","last_change":"2024-05-08T05:42:18.484430+0000","last_active":"2024-05-08T05:42:18.469813+0000","last_peered":"2024-05-08T05:42:18.469813+0000","last_clean":"2024-05-08T05:42:18.469813+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:18.484430+0000","last_undegraded":"2024-05-08T05:42:18.484430+0000","last_fullsized":"2024-05-08T05:42:18.484430+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.481950+0000","last_change":"2024-05-08T05:42:19.481950+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.481950+0000","last_undegraded":"2024-05-08T05:42:19.481950+0000","last_fullsized":"2024-05-08T05:42:19.481950+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.501312+0000","last_change":"2024-05-08T05:42:19.498339+0000","last_active":"2024-05-08T05:42:19.501312+0000","last_peered":"2024-05-08T05:42:19.501312+0000","last_clean":"2024-05-08T05:42:19.501312+0000","last_became_active":"2024-05-08T05:42:19.498000+0000","last_became_peered":"2024-05-08T05:42:19.498000+0000","last_unstale":"2024-05-08T05:42:19.501312+0000","last_undegraded":"2024-05-08T05:42:19.501312+0000","last_fullsized":"2024-05-08T05:42:19.501312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:22.668414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.501275+0000","last_change":"2024-05-08T05:42:19.499267+0000","last_active":"2024-05-08T05:42:19.501275+0000","last_peered":"2024-05-08T05:42:19.501275+0000","last_clean":"2024-05-08T05:42:19.501275+0000","last_became_active":"2024-05-08T05:42:19.498537+0000","last_became_peered":"2024-05-08T05:42:19.498537+0000","last_unstale":"2024-05-08T05:42:19.501275+0000","last_undegraded":"2024-05-08T05:42:19.501275+0000","last_fullsized":"2024-05-08T05:42:19.501275+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:53:50.777397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-08T05:42:18.484907+0000","last_change":"2024-05-08T05:42:18.484907+0000","last_active":"2024-05-08T05:42:18.469813+0000","last_peered":"2024-05-08T05:42:18.469813+0000","last_clean":"2024-05-08T05:42:18.469813+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:18.484907+0000","last_undegraded":"2024-05-08T05:42:18.484907+0000","last_fullsized":"2024-05-08T05:42:18.484907+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-08T05:42:18.484098+0000","last_change":"2024-05-08T05:42:18.484098+0000","last_active":"2024-05-08T05:42:18.469813+0000","last_peered":"2024-05-08T05:42:18.469813+0000","last_clean":"2024-05-08T05:42:18.469813+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:18.484098+0000","last_undegraded":"2024-05-08T05:42:18.484098+0000","last_fullsized":"2024-05-08T05:42:18.484098+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-08T05:42:18.478237+0000","last_change":"2024-05-08T05:42:18.478237+0000","last_active":"2024-05-08T05:42:18.469813+0000","last_peered":"2024-05-08T05:42:18.469813+0000","last_clean":"2024-05-08T05:42:18.469813+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:18.478237+0000","last_undegraded":"2024-05-08T05:42:18.478237+0000","last_fullsized":"2024-05-08T05:42:18.478237+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.480740+0000","last_change":"2024-05-08T05:42:19.480740+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.480740+0000","last_undegraded":"2024-05-08T05:42:19.480740+0000","last_fullsized":"2024-05-08T05:42:19.480740+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.498152+0000","last_change":"2024-05-08T05:42:19.496693+0000","last_active":"2024-05-08T05:42:19.498152+0000","last_peered":"2024-05-08T05:42:19.498152+0000","last_clean":"2024-05-08T05:42:19.498152+0000","last_became_active":"2024-05-08T05:42:19.496276+0000","last_became_peered":"2024-05-08T05:42:19.496276+0000","last_unstale":"2024-05-08T05:42:19.498152+0000","last_undegraded":"2024-05-08T05:42:19.498152+0000","last_fullsized":"2024-05-08T05:42:19.498152+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:50:26.559921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.495293+0000","last_change":"2024-05-08T05:42:19.493145+0000","last_active":"2024-05-08T05:42:19.495293+0000","last_peered":"2024-05-08T05:42:19.495293+0000","last_clean":"2024-05-08T05:42:19.495293+0000","last_became_active":"2024-05-08T05:42:19.492821+0000","last_became_peered":"2024-05-08T05:42:19.492821+0000","last_unstale":"2024-05-08T05:42:19.495293+0000","last_undegraded":"2024-05-08T05:42:19.495293+0000","last_fullsized":"2024-05-08T05:42:19.495293+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:20:51.731816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.501664+0000","last_change":"2024-05-08T05:42:19.499523+0000","last_active":"2024-05-08T05:42:19.501664+0000","last_peered":"2024-05-08T05:42:19.501664+0000","last_clean":"2024-05-08T05:42:19.501664+0000","last_became_active":"2024-05-08T05:42:19.499064+0000","last_became_peered":"2024-05-08T05:42:19.499064+0000","last_unstale":"2024-05-08T05:42:19.501664+0000","last_undegraded":"2024-05-08T05:42:19.501664+0000","last_fullsized":"2024-05-08T05:42:19.501664+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:28:34.310847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.488517+0000","last_change":"2024-05-08T05:42:19.488517+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.488517+0000","last_undegraded":"2024-05-08T05:42:19.488517+0000","last_fullsized":"2024-05-08T05:42:19.488517+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-08T05:42:18.482035+0000","last_change":"2024-05-08T05:42:18.482035+0000","last_active":"2024-05-08T05:42:18.469813+0000","last_peered":"2024-05-08T05:42:18.469813+0000","last_clean":"2024-05-08T05:42:18.469813+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:18.482035+0000","last_undegraded":"2024-05-08T05:42:18.482035+0000","last_fullsized":"2024-05-08T05:42:18.482035+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-08T05:42:18.481499+0000","last_change":"2024-05-08T05:42:18.481499+0000","last_active":"2024-05-08T05:42:18.469813+0000","last_peered":"2024-05-08T05:42:18.469813+0000","last_clean":"2024-05-08T05:42:18.469813+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:18.481499+0000","last_undegraded":"2024-05-08T05:42:18.481499+0000","last_fullsized":"2024-05-08T05:42:18.481499+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-08T05:42:18.478250+0000","last_change":"2024-05-08T05:42:18.478250+0000","last_active":"2024-05-08T05:42:18.469813+0000","last_peered":"2024-05-08T05:42:18.469813+0000","last_clean":"2024-05-08T05:42:18.469813+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:18.478250+0000","last_undegraded":"2024-05-08T05:42:18.478250+0000","last_fullsized":"2024-05-08T05:42:18.478250+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.480258+0000","last_change":"2024-05-08T05:42:19.480258+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.480258+0000","last_undegraded":"2024-05-08T05:42:19.480258+0000","last_fullsized":"2024-05-08T05:42:19.480258+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-08T05:42:18.481438+0000","last_change":"2024-05-08T05:42:18.481438+0000","last_active":"2024-05-08T05:42:18.469813+0000","last_peered":"2024-05-08T05:42:18.469813+0000","last_clean":"2024-05-08T05:42:18.469813+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:18.481438+0000","last_undegraded":"2024-05-08T05:42:18.481438+0000","last_fullsized":"2024-05-08T05:42:18.481438+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-08T05:42:18.481426+0000","last_change":"2024-05-08T05:42:18.481426+0000","last_active":"2024-05-08T05:42:18.469813+0000","last_peered":"2024-05-08T05:42:18.469813+0000","last_clean":"2024-05-08T05:42:18.469813+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:18.481426+0000","last_undegraded":"2024-05-08T05:42:18.481426+0000","last_fullsized":"2024-05-08T05:42:18.481426+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.490421+0000","last_change":"2024-05-08T05:42:19.490421+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.490421+0000","last_undegraded":"2024-05-08T05:42:19.490421+0000","last_fullsized":"2024-05-08T05:42:19.490421+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.495945+0000","last_change":"2024-05-08T05:42:19.494546+0000","last_active":"2024-05-08T05:42:19.495945+0000","last_peered":"2024-05-08T05:42:19.495945+0000","last_clean":"2024-05-08T05:42:19.495945+0000","last_became_active":"2024-05-08T05:42:19.494232+0000","last_became_peered":"2024-05-08T05:42:19.494232+0000","last_unstale":"2024-05-08T05:42:19.495945+0000","last_undegraded":"2024-05-08T05:42:19.495945+0000","last_fullsized":"2024-05-08T05:42:19.495945+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:22.668414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-08T05:42:18.481367+0000","last_change":"2024-05-08T05:42:18.481367+0000","last_active":"2024-05-08T05:42:18.469813+0000","last_peered":"2024-05-08T05:42:18.469813+0000","last_clean":"2024-05-08T05:42:18.469813+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:18.481367+0000","last_undegraded":"2024-05-08T05:42:18.481367+0000","last_fullsized":"2024-05-08T05:42:18.481367+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.501230+0000","last_change":"2024-05-08T05:42:19.498985+0000","last_active":"2024-05-08T05:42:19.501230+0000","last_peered":"2024-05-08T05:42:19.501230+0000","last_clean":"2024-05-08T05:42:19.501230+0000","last_became_active":"2024-05-08T05:42:19.498663+0000","last_became_peered":"2024-05-08T05:42:19.498663+0000","last_unstale":"2024-05-08T05:42:19.501230+0000","last_undegraded":"2024-05-08T05:42:19.501230+0000","last_fullsized":"2024-05-08T05:42:19.501230+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:26.053156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.481486+0000","last_change":"2024-05-08T05:42:19.481486+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.481486+0000","last_undegraded":"2024-05-08T05:42:19.481486+0000","last_fullsized":"2024-05-08T05:42:19.481486+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-08T05:42:18.481507+0000","last_change":"2024-05-08T05:42:18.481507+0000","last_active":"2024-05-08T05:42:18.469813+0000","last_peered":"2024-05-08T05:42:18.469813+0000","last_clean":"2024-05-08T05:42:18.469813+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:18.481507+0000","last_undegraded":"2024-05-08T05:42:18.481507+0000","last_fullsized":"2024-05-08T05:42:18.481507+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-08T05:42:18.476401+0000","last_change":"2024-05-08T05:42:18.476401+0000","last_active":"2024-05-08T05:42:18.469813+0000","last_peered":"2024-05-08T05:42:18.469813+0000","last_clean":"2024-05-08T05:42:18.469813+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:18.476401+0000","last_undegraded":"2024-05-08T05:42:18.476401+0000","last_fullsized":"2024-05-08T05:42:18.476401+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-08T05:42:18.476331+0000","last_change":"2024-05-08T05:42:18.476331+0000","last_active":"2024-05-08T05:42:18.469813+0000","last_peered":"2024-05-08T05:42:18.469813+0000","last_clean":"2024-05-08T05:42:18.469813+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:18.476331+0000","last_undegraded":"2024-05-08T05:42:18.476331+0000","last_fullsized":"2024-05-08T05:42:18.476331+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.499479+0000","last_change":"2024-05-08T05:42:19.497480+0000","last_active":"2024-05-08T05:42:19.499479+0000","last_peered":"2024-05-08T05:42:19.499479+0000","last_clean":"2024-05-08T05:42:19.499479+0000","last_became_active":"2024-05-08T05:42:19.497177+0000","last_became_peered":"2024-05-08T05:42:19.497177+0000","last_unstale":"2024-05-08T05:42:19.499479+0000","last_undegraded":"2024-05-08T05:42:19.499479+0000","last_fullsized":"2024-05-08T05:42:19.499479+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:58:57.734024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:42:19.471571+0000","last_change":"2024-05-08T05:42:19.471571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.471571+0000","last_undegraded":"2024-05-08T05:42:19.471571+0000","last_fullsized":"2024-05-08T05:42:19.471571+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-08T05:42:18.481294+0000","last_change":"2024-05-08T05:42:18.481294+0000","last_active":"2024-05-08T05:42:18.469813+0000","last_peered":"2024-05-08T05:42:18.469813+0000","last_clean":"2024-05-08T05:42:18.469813+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:18.481294+0000","last_undegraded":"2024-05-08T05:42:18.481294+0000","last_fullsized":"2024-05-08T05:42:18.481294+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.483121+0000","last_change":"2024-05-08T05:42:19.483121+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.483121+0000","last_undegraded":"2024-05-08T05:42:19.483121+0000","last_fullsized":"2024-05-08T05:42:19.483121+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-05-08T05:42:18.483913+0000","last_change":"2024-05-08T05:42:18.483913+0000","last_active":"2024-05-08T05:42:18.469813+0000","last_peered":"2024-05-08T05:42:18.469813+0000","last_clean":"2024-05-08T05:42:18.469813+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:18.483913+0000","last_undegraded":"2024-05-08T05:42:18.483913+0000","last_fullsized":"2024-05-08T05:42:18.483913+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.489194+0000","last_change":"2024-05-08T05:42:19.489194+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.489194+0000","last_undegraded":"2024-05-08T05:42:19.489194+0000","last_fullsized":"2024-05-08T05:42:19.489194+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.489865+0000","last_change":"2024-05-08T05:42:19.487846+0000","last_active":"2024-05-08T05:42:19.489865+0000","last_peered":"2024-05-08T05:42:19.489865+0000","last_clean":"2024-05-08T05:42:19.489865+0000","last_became_active":"2024-05-08T05:42:19.487570+0000","last_became_peered":"2024-05-08T05:42:19.487570+0000","last_unstale":"2024-05-08T05:42:19.489865+0000","last_undegraded":"2024-05-08T05:42:19.489865+0000","last_fullsized":"2024-05-08T05:42:19.489865+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:25.075141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":135,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.483776+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:42:19.483776+0000","last_peered":"2024-05-08T05:42:19.483776+0000","last_clean":"2024-05-08T05:42:19.483776+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:42:19.483776+0000","last_undegraded":"2024-05-08T05:42:19.483776+0000","last_fullsized":"2024-05-08T05:42:19.483776+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":54,"acting":54,"num_store_stats":3},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":966656,"data_stored":744313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":372736,"data_stored":153945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542160,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":160612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":372736,"data_stored":153945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":372736,"data_stored":153945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":360448,"data_stored":147736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":360448,"data_stored":147736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":954368,"data_stored":738104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:42:24.566 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:42:24.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.706+0000 7f2862daf640 1 Processor -- start 2024-05-08T05:42:24.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.706+0000 7f2862daf640 1 -- start start 2024-05-08T05:42:24.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.707+0000 7f2862daf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f285c0b2490 0x7f285c070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.707+0000 7f2862daf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f285c070d20 0x7f285c0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.707+0000 7f2862daf640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f285c06e110 con 0x7f285c1629b0 2024-05-08T05:42:24.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.707+0000 7f2862daf640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f285c06dec0 con 0x7f285c0b2490 2024-05-08T05:42:24.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.707+0000 7f2862daf640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f285c06e3f0 con 0x7f285c070d20 2024-05-08T05:42:24.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.707+0000 7f285bfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f285c0b2490 0x7f285c070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.707+0000 7f2860b4b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f285c1629b0 0x7f285c06eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56052/0 (socket says 172.21.15.45:56052) 2024-05-08T05:42:24.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.707+0000 7f2860b4b640 1 -- 172.21.15.45:0/3475422443 learned_addr learned my addr 172.21.15.45:0/3475422443 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:24.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.707+0000 7f285bfff640 1 -- 172.21.15.45:0/3475422443 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f285c070d20 msgr2=0x7f285c0711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:24.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.707+0000 7f286134c640 1 --2- 172.21.15.45:0/3475422443 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f285c070d20 0x7f285c0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.707+0000 7f285b7fe640 1 -- 172.21.15.45:0/3475422443 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3874215764 0 0) 0x7f285c06e110 con 0x7f285c1629b0 2024-05-08T05:42:24.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.707+0000 7f285bfff640 1 --2- 172.21.15.45:0/3475422443 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f285c070d20 0x7f285c0711d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.707+0000 7f285bfff640 1 -- 172.21.15.45:0/3475422443 >> v1:172.21.15.45:6789/0 conn(0x7f285c1629b0 legacy=0x7f285c06eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.707+0000 7f285bfff640 1 -- 172.21.15.45:0/3475422443 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285c06e6d0 con 0x7f285c0b2490 2024-05-08T05:42:24.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.708+0000 7f285bfff640 1 --2- 172.21.15.45:0/3475422443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285c0b2490 0x7f285c070330 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f28480097a0 tx=0x7f284802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8a5a9a0c5d6aa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:24.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.708+0000 7f285b7fe640 1 -- 172.21.15.45:0/3475422443 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848037070 con 0x7f285c0b2490 2024-05-08T05:42:24.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.708+0000 7f285b7fe640 1 -- 172.21.15.45:0/3475422443 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2848032440 con 0x7f285c0b2490 2024-05-08T05:42:24.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.708+0000 7f285b7fe640 1 -- 172.21.15.45:0/3475422443 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848032810 con 0x7f285c0b2490 2024-05-08T05:42:24.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.709+0000 7f2862daf640 1 -- 172.21.15.45:0/3475422443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285c0b2490 msgr2=0x7f285c070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.709+0000 7f2862daf640 1 --2- 172.21.15.45:0/3475422443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285c0b2490 0x7f285c070330 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f28480097a0 tx=0x7f284802a180 comp rx=0 tx=0).stop 2024-05-08T05:42:24.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.709+0000 7f2862daf640 1 -- 172.21.15.45:0/3475422443 shutdown_connections 2024-05-08T05:42:24.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.709+0000 7f2862daf640 1 --2- 172.21.15.45:0/3475422443 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f285c070d20 0x7f285c0711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.709+0000 7f2862daf640 1 --2- 172.21.15.45:0/3475422443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285c0b2490 0x7f285c070330 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.709+0000 7f2862daf640 1 -- 172.21.15.45:0/3475422443 >> 172.21.15.45:0/3475422443 conn(0x7f285c0b49d0 msgr2=0x7f285c0b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:24.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.709+0000 7f2862daf640 1 -- 172.21.15.45:0/3475422443 shutdown_connections 2024-05-08T05:42:24.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.709+0000 7f2862daf640 1 -- 172.21.15.45:0/3475422443 wait complete. 2024-05-08T05:42:24.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.710+0000 7f2862daf640 1 Processor -- start 2024-05-08T05:42:24.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.710+0000 7f2862daf640 1 -- start start 2024-05-08T05:42:24.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.711+0000 7f2862daf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285c070d20 0x7f285c0b8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.711+0000 7f2862daf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f285c0b2490 0x7f285c0b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.711+0000 7f2862daf640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f285c1629b0 0x7f285c0b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.711+0000 7f2862daf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f285c0709f0 con 0x7f285c070d20 2024-05-08T05:42:24.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.711+0000 7f2862daf640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f285c070870 con 0x7f285c1629b0 2024-05-08T05:42:24.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.711+0000 7f2862daf640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f285c0afcb0 con 0x7f285c0b2490 2024-05-08T05:42:24.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.711+0000 7f2860b4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285c070d20 0x7f285c0b8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.711+0000 7f2860b4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285c070d20 0x7f285c0b8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44216/0 (socket says 172.21.15.45:44216) 2024-05-08T05:42:24.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.711+0000 7f2860b4b640 1 -- 172.21.15.45:0/3307229047 learned_addr learned my addr 172.21.15.45:0/3307229047 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:24.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.711+0000 7f285bfff640 1 --2- 172.21.15.45:0/3307229047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f285c0b2490 0x7f285c0b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.711+0000 7f286134c640 1 --2- 172.21.15.45:0/3307229047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f285c1629b0 0x7f285c0b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.711+0000 7f2860b4b640 1 -- 172.21.15.45:0/3307229047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f285c0b2490 msgr2=0x7f285c0b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.711+0000 7f2860b4b640 1 --2- 172.21.15.45:0/3307229047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f285c0b2490 0x7f285c0b3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.711+0000 7f2860b4b640 1 -- 172.21.15.45:0/3307229047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f285c1629b0 msgr2=0x7f285c0b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.711+0000 7f2860b4b640 1 --2- 172.21.15.45:0/3307229047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f285c1629b0 0x7f285c0b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.711+0000 7f2860b4b640 1 -- 172.21.15.45:0/3307229047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285c0b8750 con 0x7f285c070d20 2024-05-08T05:42:24.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.712+0000 7f2860b4b640 1 --2- 172.21.15.45:0/3307229047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285c070d20 0x7f285c0b8210 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f285000c3e0 tx=0x7f285000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:24.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.712+0000 7f28597fa640 1 -- 172.21.15.45:0/3307229047 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28500074e0 con 0x7f285c070d20 2024-05-08T05:42:24.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.712+0000 7f28597fa640 1 -- 172.21.15.45:0/3307229047 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28500047e0 con 0x7f285c070d20 2024-05-08T05:42:24.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.712+0000 7f2862daf640 1 -- 172.21.15.45:0/3307229047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285c0d0090 con 0x7f285c070d20 2024-05-08T05:42:24.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.712+0000 7f28597fa640 1 -- 172.21.15.45:0/3307229047 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850004bd0 con 0x7f285c070d20 2024-05-08T05:42:24.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.712+0000 7f2862daf640 1 -- 172.21.15.45:0/3307229047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f285c0ca7f0 con 0x7f285c070d20 2024-05-08T05:42:24.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.713+0000 7f2862daf640 1 -- 172.21.15.45:0/3307229047 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f285c0693f0 con 0x7f285c070d20 2024-05-08T05:42:24.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.713+0000 7f28597fa640 1 -- 172.21.15.45:0/3307229047 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2850011070 con 0x7f285c070d20 2024-05-08T05:42:24.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.714+0000 7f28597fa640 1 --2- 172.21.15.45:0/3307229047 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f28400762d0 0x7f2840078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.714+0000 7f28597fa640 1 -- 172.21.15.45:0/3307229047 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f2850097300 con 0x7f285c070d20 2024-05-08T05:42:24.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.714+0000 7f285bfff640 1 --2- 172.21.15.45:0/3307229047 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f28400762d0 0x7f2840078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.715+0000 7f285bfff640 1 --2- 172.21.15.45:0/3307229047 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f28400762d0 0x7f2840078790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2848034000 tx=0x7f284802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:24.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.718+0000 7f28597fa640 1 -- 172.21.15.45:0/3307229047 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2850060b60 con 0x7f285c070d20 2024-05-08T05:42:24.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.905+0000 7f2862daf640 1 -- 172.21.15.45:0/3307229047 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f285c15f880 con 0x7f28400762d0 2024-05-08T05:42:24.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.912+0000 7f28597fa640 1 -- 172.21.15.45:0/3307229047 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249708 (secure 0 0 0) 0x7f285c15f880 con 0x7f28400762d0 2024-05-08T05:42:24.915 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:24.916 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:42:24.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.915+0000 7f2862daf640 1 -- 172.21.15.45:0/3307229047 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f28400762d0 msgr2=0x7f2840078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.915+0000 7f2862daf640 1 --2- 172.21.15.45:0/3307229047 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f28400762d0 0x7f2840078790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2848034000 tx=0x7f284802e000 comp rx=0 tx=0).stop 2024-05-08T05:42:24.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.915+0000 7f2862daf640 1 -- 172.21.15.45:0/3307229047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285c070d20 msgr2=0x7f285c0b8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.915+0000 7f2862daf640 1 --2- 172.21.15.45:0/3307229047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285c070d20 0x7f285c0b8210 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f285000c3e0 tx=0x7f285000c8a0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.915+0000 7f2862daf640 1 -- 172.21.15.45:0/3307229047 shutdown_connections 2024-05-08T05:42:24.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.915+0000 7f2862daf640 1 --2- 172.21.15.45:0/3307229047 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f28400762d0 0x7f2840078790 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.915+0000 7f2862daf640 1 --2- 172.21.15.45:0/3307229047 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f285c0b2490 0x7f285c0b3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.915+0000 7f2862daf640 1 --2- 172.21.15.45:0/3307229047 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f285c1629b0 0x7f285c0b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.915+0000 7f2862daf640 1 --2- 172.21.15.45:0/3307229047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f285c070d20 0x7f285c0b8210 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.915+0000 7f2862daf640 1 -- 172.21.15.45:0/3307229047 >> 172.21.15.45:0/3307229047 conn(0x7f285c0b49d0 msgr2=0x7f285c0d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:24.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.915+0000 7f2862daf640 1 -- 172.21.15.45:0/3307229047 shutdown_connections 2024-05-08T05:42:24.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:24.915+0000 7f2862daf640 1 -- 172.21.15.45:0/3307229047 wait complete. 2024-05-08T05:42:24.926 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-05-08T05:42:24.845538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592084,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":122,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237088,"kb_used_data":4276,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737632,"statfs":{"total":773094113280,"available":772851335168,"internally_reserved":0,"allocated":4378624,"data_stored":2510653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1716,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000980"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569700+0000","last_change":"2024-05-08T05:42:20.521354+0000","last_active":"2024-05-08T05:42:23.569700+0000","last_peered":"2024-05-08T05:42:23.569700+0000","last_clean":"2024-05-08T05:42:23.569700+0000","last_became_active":"2024-05-08T05:42:20.521070+0000","last_became_peered":"2024-05-08T05:42:20.521070+0000","last_unstale":"2024-05-08T05:42:23.569700+0000","last_undegraded":"2024-05-08T05:42:23.569700+0000","last_fullsized":"2024-05-08T05:42:23.569700+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:08.311556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568750+0000","last_change":"2024-05-08T05:42:20.525189+0000","last_active":"2024-05-08T05:42:23.568750+0000","last_peered":"2024-05-08T05:42:23.568750+0000","last_clean":"2024-05-08T05:42:23.568750+0000","last_became_active":"2024-05-08T05:42:20.524906+0000","last_became_peered":"2024-05-08T05:42:20.524906+0000","last_unstale":"2024-05-08T05:42:23.568750+0000","last_undegraded":"2024-05-08T05:42:23.568750+0000","last_fullsized":"2024-05-08T05:42:23.568750+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:22:33.614866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569160+0000","last_change":"2024-05-08T05:42:20.532374+0000","last_active":"2024-05-08T05:42:23.569160+0000","last_peered":"2024-05-08T05:42:23.569160+0000","last_clean":"2024-05-08T05:42:23.569160+0000","last_became_active":"2024-05-08T05:42:20.532079+0000","last_became_peered":"2024-05-08T05:42:20.532079+0000","last_unstale":"2024-05-08T05:42:23.569160+0000","last_undegraded":"2024-05-08T05:42:23.569160+0000","last_fullsized":"2024-05-08T05:42:23.569160+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:23.089039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517311+0000","last_change":"2024-05-08T05:42:20.530407+0000","last_active":"2024-05-08T05:42:21.517311+0000","last_peered":"2024-05-08T05:42:21.517311+0000","last_clean":"2024-05-08T05:42:21.517311+0000","last_became_active":"2024-05-08T05:42:20.530148+0000","last_became_peered":"2024-05-08T05:42:20.530148+0000","last_unstale":"2024-05-08T05:42:21.517311+0000","last_undegraded":"2024-05-08T05:42:21.517311+0000","last_fullsized":"2024-05-08T05:42:21.517311+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:30.266997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517173+0000","last_change":"2024-05-08T05:42:20.528145+0000","last_active":"2024-05-08T05:42:21.517173+0000","last_peered":"2024-05-08T05:42:21.517173+0000","last_clean":"2024-05-08T05:42:21.517173+0000","last_became_active":"2024-05-08T05:42:20.527841+0000","last_became_peered":"2024-05-08T05:42:20.527841+0000","last_unstale":"2024-05-08T05:42:21.517173+0000","last_undegraded":"2024-05-08T05:42:21.517173+0000","last_fullsized":"2024-05-08T05:42:21.517173+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:15.132265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569078+0000","last_change":"2024-05-08T05:42:20.518897+0000","last_active":"2024-05-08T05:42:23.569078+0000","last_peered":"2024-05-08T05:42:23.569078+0000","last_clean":"2024-05-08T05:42:23.569078+0000","last_became_active":"2024-05-08T05:42:20.518613+0000","last_became_peered":"2024-05-08T05:42:20.518613+0000","last_unstale":"2024-05-08T05:42:23.569078+0000","last_undegraded":"2024-05-08T05:42:23.569078+0000","last_fullsized":"2024-05-08T05:42:23.569078+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:23.089039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.494063+0000","last_change":"2024-05-08T05:42:19.494063+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.494063+0000","last_undegraded":"2024-05-08T05:42:19.494063+0000","last_fullsized":"2024-05-08T05:42:19.494063+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568409+0000","last_change":"2024-05-08T05:42:20.515418+0000","last_active":"2024-05-08T05:42:23.568409+0000","last_peered":"2024-05-08T05:42:23.568409+0000","last_clean":"2024-05-08T05:42:23.568409+0000","last_became_active":"2024-05-08T05:42:20.514961+0000","last_became_peered":"2024-05-08T05:42:20.514961+0000","last_unstale":"2024-05-08T05:42:23.568409+0000","last_undegraded":"2024-05-08T05:42:23.568409+0000","last_fullsized":"2024-05-08T05:42:23.568409+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:54:21.060834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569512+0000","last_change":"2024-05-08T05:42:20.533002+0000","last_active":"2024-05-08T05:42:23.569512+0000","last_peered":"2024-05-08T05:42:23.569512+0000","last_clean":"2024-05-08T05:42:23.569512+0000","last_became_active":"2024-05-08T05:42:20.532713+0000","last_became_peered":"2024-05-08T05:42:20.532713+0000","last_unstale":"2024-05-08T05:42:23.569512+0000","last_undegraded":"2024-05-08T05:42:23.569512+0000","last_fullsized":"2024-05-08T05:42:23.569512+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:34:08.047335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.492151+0000","last_change":"2024-05-08T05:42:19.492151+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.492151+0000","last_undegraded":"2024-05-08T05:42:19.492151+0000","last_fullsized":"2024-05-08T05:42:19.492151+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.489300+0000","last_change":"2024-05-08T05:42:19.489300+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.489300+0000","last_undegraded":"2024-05-08T05:42:19.489300+0000","last_fullsized":"2024-05-08T05:42:19.489300+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568992+0000","last_change":"2024-05-08T05:42:20.527823+0000","last_active":"2024-05-08T05:42:23.568992+0000","last_peered":"2024-05-08T05:42:23.568992+0000","last_clean":"2024-05-08T05:42:23.568992+0000","last_became_active":"2024-05-08T05:42:20.527531+0000","last_became_peered":"2024-05-08T05:42:20.527531+0000","last_unstale":"2024-05-08T05:42:23.568992+0000","last_undegraded":"2024-05-08T05:42:23.568992+0000","last_fullsized":"2024-05-08T05:42:23.568992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:06.429659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.488571+0000","last_change":"2024-05-08T05:42:19.488571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.488571+0000","last_undegraded":"2024-05-08T05:42:19.488571+0000","last_fullsized":"2024-05-08T05:42:19.488571+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568056+0000","last_change":"2024-05-08T05:42:20.520112+0000","last_active":"2024-05-08T05:42:23.568056+0000","last_peered":"2024-05-08T05:42:23.568056+0000","last_clean":"2024-05-08T05:42:23.568056+0000","last_became_active":"2024-05-08T05:42:20.519660+0000","last_became_peered":"2024-05-08T05:42:20.519660+0000","last_unstale":"2024-05-08T05:42:23.568056+0000","last_undegraded":"2024-05-08T05:42:23.568056+0000","last_fullsized":"2024-05-08T05:42:23.568056+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:25:23.199079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.491652+0000","last_change":"2024-05-08T05:42:19.491652+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.491652+0000","last_undegraded":"2024-05-08T05:42:19.491652+0000","last_fullsized":"2024-05-08T05:42:19.491652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.483214+0000","last_change":"2024-05-08T05:42:19.483214+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.483214+0000","last_undegraded":"2024-05-08T05:42:19.483214+0000","last_fullsized":"2024-05-08T05:42:19.483214+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.482113+0000","last_change":"2024-05-08T05:42:19.482113+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.482113+0000","last_undegraded":"2024-05-08T05:42:19.482113+0000","last_fullsized":"2024-05-08T05:42:19.482113+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.486278+0000","last_change":"2024-05-08T05:42:19.486278+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.486278+0000","last_undegraded":"2024-05-08T05:42:19.486278+0000","last_fullsized":"2024-05-08T05:42:19.486278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517514+0000","last_change":"2024-05-08T05:42:20.533970+0000","last_active":"2024-05-08T05:42:21.517514+0000","last_peered":"2024-05-08T05:42:21.517514+0000","last_clean":"2024-05-08T05:42:21.517514+0000","last_became_active":"2024-05-08T05:42:20.531975+0000","last_became_peered":"2024-05-08T05:42:20.531975+0000","last_unstale":"2024-05-08T05:42:21.517514+0000","last_undegraded":"2024-05-08T05:42:21.517514+0000","last_fullsized":"2024-05-08T05:42:21.517514+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:20:52.733574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.489219+0000","last_change":"2024-05-08T05:42:19.489219+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.489219+0000","last_undegraded":"2024-05-08T05:42:19.489219+0000","last_fullsized":"2024-05-08T05:42:19.489219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.489042+0000","last_change":"2024-05-08T05:42:19.489042+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.489042+0000","last_undegraded":"2024-05-08T05:42:19.489042+0000","last_fullsized":"2024-05-08T05:42:19.489042+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568464+0000","last_change":"2024-05-08T05:42:20.525789+0000","last_active":"2024-05-08T05:42:23.568464+0000","last_peered":"2024-05-08T05:42:23.568464+0000","last_clean":"2024-05-08T05:42:23.568464+0000","last_became_active":"2024-05-08T05:42:20.525492+0000","last_became_peered":"2024-05-08T05:42:20.525492+0000","last_unstale":"2024-05-08T05:42:23.568464+0000","last_undegraded":"2024-05-08T05:42:23.568464+0000","last_fullsized":"2024-05-08T05:42:23.568464+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:44.235500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.481502+0000","last_change":"2024-05-08T05:42:19.481502+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.481502+0000","last_undegraded":"2024-05-08T05:42:19.481502+0000","last_fullsized":"2024-05-08T05:42:19.481502+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517037+0000","last_change":"2024-05-08T05:42:20.534832+0000","last_active":"2024-05-08T05:42:21.517037+0000","last_peered":"2024-05-08T05:42:21.517037+0000","last_clean":"2024-05-08T05:42:21.517037+0000","last_became_active":"2024-05-08T05:42:20.534586+0000","last_became_peered":"2024-05-08T05:42:20.534586+0000","last_unstale":"2024-05-08T05:42:21.517037+0000","last_undegraded":"2024-05-08T05:42:21.517037+0000","last_fullsized":"2024-05-08T05:42:21.517037+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:30.319294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.485694+0000","last_change":"2024-05-08T05:42:19.485694+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.485694+0000","last_undegraded":"2024-05-08T05:42:19.485694+0000","last_fullsized":"2024-05-08T05:42:19.485694+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569431+0000","last_change":"2024-05-08T05:42:20.509600+0000","last_active":"2024-05-08T05:42:23.569431+0000","last_peered":"2024-05-08T05:42:23.569431+0000","last_clean":"2024-05-08T05:42:23.569431+0000","last_became_active":"2024-05-08T05:42:20.509045+0000","last_became_peered":"2024-05-08T05:42:20.509045+0000","last_unstale":"2024-05-08T05:42:23.569431+0000","last_undegraded":"2024-05-08T05:42:23.569431+0000","last_fullsized":"2024-05-08T05:42:23.569431+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:39.499482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.486709+0000","last_change":"2024-05-08T05:42:19.486709+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.486709+0000","last_undegraded":"2024-05-08T05:42:19.486709+0000","last_fullsized":"2024-05-08T05:42:19.486709+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.488095+0000","last_change":"2024-05-08T05:42:19.488095+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.488095+0000","last_undegraded":"2024-05-08T05:42:19.488095+0000","last_fullsized":"2024-05-08T05:42:19.488095+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.488425+0000","last_change":"2024-05-08T05:42:19.488425+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.488425+0000","last_undegraded":"2024-05-08T05:42:19.488425+0000","last_fullsized":"2024-05-08T05:42:19.488425+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.485913+0000","last_change":"2024-05-08T05:42:19.485913+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.485913+0000","last_undegraded":"2024-05-08T05:42:19.485913+0000","last_fullsized":"2024-05-08T05:42:19.485913+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.481029+0000","last_change":"2024-05-08T05:42:19.481029+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.481029+0000","last_undegraded":"2024-05-08T05:42:19.481029+0000","last_fullsized":"2024-05-08T05:42:19.481029+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569182+0000","last_change":"2024-05-08T05:42:20.524631+0000","last_active":"2024-05-08T05:42:23.569182+0000","last_peered":"2024-05-08T05:42:23.569182+0000","last_clean":"2024-05-08T05:42:23.569182+0000","last_became_active":"2024-05-08T05:42:20.524354+0000","last_became_peered":"2024-05-08T05:42:20.524354+0000","last_unstale":"2024-05-08T05:42:23.569182+0000","last_undegraded":"2024-05-08T05:42:23.569182+0000","last_fullsized":"2024-05-08T05:42:23.569182+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:34:08.047335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.481092+0000","last_change":"2024-05-08T05:42:19.481092+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.481092+0000","last_undegraded":"2024-05-08T05:42:19.481092+0000","last_fullsized":"2024-05-08T05:42:19.481092+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.497640+0000","last_change":"2024-05-08T05:42:19.493864+0000","last_active":"2024-05-08T05:42:19.497640+0000","last_peered":"2024-05-08T05:42:19.497640+0000","last_clean":"2024-05-08T05:42:19.497640+0000","last_became_active":"2024-05-08T05:42:19.493061+0000","last_became_peered":"2024-05-08T05:42:19.493061+0000","last_unstale":"2024-05-08T05:42:19.497640+0000","last_undegraded":"2024-05-08T05:42:19.497640+0000","last_fullsized":"2024-05-08T05:42:19.497640+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:47:27.293889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569291+0000","last_change":"2024-05-08T05:42:20.512807+0000","last_active":"2024-05-08T05:42:23.569291+0000","last_peered":"2024-05-08T05:42:23.569291+0000","last_clean":"2024-05-08T05:42:23.569291+0000","last_became_active":"2024-05-08T05:42:20.512501+0000","last_became_peered":"2024-05-08T05:42:20.512501+0000","last_unstale":"2024-05-08T05:42:23.569291+0000","last_undegraded":"2024-05-08T05:42:23.569291+0000","last_fullsized":"2024-05-08T05:42:23.569291+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:32:33.781763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569814+0000","last_change":"2024-05-08T05:42:19.493675+0000","last_active":"2024-05-08T05:42:23.569814+0000","last_peered":"2024-05-08T05:42:23.569814+0000","last_clean":"2024-05-08T05:42:23.569814+0000","last_became_active":"2024-05-08T05:42:19.493339+0000","last_became_peered":"2024-05-08T05:42:19.493339+0000","last_unstale":"2024-05-08T05:42:23.569814+0000","last_undegraded":"2024-05-08T05:42:23.569814+0000","last_fullsized":"2024-05-08T05:42:23.569814+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:51:24.291839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568501+0000","last_change":"2024-05-08T05:42:20.521416+0000","last_active":"2024-05-08T05:42:23.568501+0000","last_peered":"2024-05-08T05:42:23.568501+0000","last_clean":"2024-05-08T05:42:23.568501+0000","last_became_active":"2024-05-08T05:42:20.520422+0000","last_became_peered":"2024-05-08T05:42:20.520422+0000","last_unstale":"2024-05-08T05:42:23.568501+0000","last_undegraded":"2024-05-08T05:42:23.568501+0000","last_fullsized":"2024-05-08T05:42:23.568501+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:18.431907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569440+0000","last_change":"2024-05-08T05:42:19.491376+0000","last_active":"2024-05-08T05:42:23.569440+0000","last_peered":"2024-05-08T05:42:23.569440+0000","last_clean":"2024-05-08T05:42:23.569440+0000","last_became_active":"2024-05-08T05:42:19.491075+0000","last_became_peered":"2024-05-08T05:42:19.491075+0000","last_unstale":"2024-05-08T05:42:23.569440+0000","last_undegraded":"2024-05-08T05:42:23.569440+0000","last_fullsized":"2024-05-08T05:42:23.569440+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:28:34.310847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569384+0000","last_change":"2024-05-08T05:42:20.529711+0000","last_active":"2024-05-08T05:42:23.569384+0000","last_peered":"2024-05-08T05:42:23.569384+0000","last_clean":"2024-05-08T05:42:23.569384+0000","last_became_active":"2024-05-08T05:42:20.529354+0000","last_became_peered":"2024-05-08T05:42:20.529354+0000","last_unstale":"2024-05-08T05:42:23.569384+0000","last_undegraded":"2024-05-08T05:42:23.569384+0000","last_fullsized":"2024-05-08T05:42:23.569384+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:43:42.826524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568253+0000","last_change":"2024-05-08T05:42:19.495593+0000","last_active":"2024-05-08T05:42:23.568253+0000","last_peered":"2024-05-08T05:42:23.568253+0000","last_clean":"2024-05-08T05:42:23.568253+0000","last_became_active":"2024-05-08T05:42:19.495297+0000","last_became_peered":"2024-05-08T05:42:19.495297+0000","last_unstale":"2024-05-08T05:42:23.568253+0000","last_undegraded":"2024-05-08T05:42:23.568253+0000","last_fullsized":"2024-05-08T05:42:23.568253+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:29.317536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568467+0000","last_change":"2024-05-08T05:42:20.525554+0000","last_active":"2024-05-08T05:42:23.568467+0000","last_peered":"2024-05-08T05:42:23.568467+0000","last_clean":"2024-05-08T05:42:23.568467+0000","last_became_active":"2024-05-08T05:42:20.525027+0000","last_became_peered":"2024-05-08T05:42:20.525027+0000","last_unstale":"2024-05-08T05:42:23.568467+0000","last_undegraded":"2024-05-08T05:42:23.568467+0000","last_fullsized":"2024-05-08T05:42:23.568467+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:44:05.274392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.501668+0000","last_change":"2024-05-08T05:42:19.499533+0000","last_active":"2024-05-08T05:42:19.501668+0000","last_peered":"2024-05-08T05:42:19.501668+0000","last_clean":"2024-05-08T05:42:19.501668+0000","last_became_active":"2024-05-08T05:42:19.498447+0000","last_became_peered":"2024-05-08T05:42:19.498447+0000","last_unstale":"2024-05-08T05:42:19.501668+0000","last_undegraded":"2024-05-08T05:42:19.501668+0000","last_fullsized":"2024-05-08T05:42:19.501668+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:37.041564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.488852+0000","last_change":"2024-05-08T05:42:19.488852+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.488852+0000","last_undegraded":"2024-05-08T05:42:19.488852+0000","last_fullsized":"2024-05-08T05:42:19.488852+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.497089+0000","last_change":"2024-05-08T05:42:19.495816+0000","last_active":"2024-05-08T05:42:19.497089+0000","last_peered":"2024-05-08T05:42:19.497089+0000","last_clean":"2024-05-08T05:42:19.497089+0000","last_became_active":"2024-05-08T05:42:19.495546+0000","last_became_peered":"2024-05-08T05:42:19.495546+0000","last_unstale":"2024-05-08T05:42:19.497089+0000","last_undegraded":"2024-05-08T05:42:19.497089+0000","last_fullsized":"2024-05-08T05:42:19.497089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:26.053156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568299+0000","last_change":"2024-05-08T05:42:20.512851+0000","last_active":"2024-05-08T05:42:23.568299+0000","last_peered":"2024-05-08T05:42:23.568299+0000","last_clean":"2024-05-08T05:42:23.568299+0000","last_became_active":"2024-05-08T05:42:20.512412+0000","last_became_peered":"2024-05-08T05:42:20.512412+0000","last_unstale":"2024-05-08T05:42:23.568299+0000","last_undegraded":"2024-05-08T05:42:23.568299+0000","last_fullsized":"2024-05-08T05:42:23.568299+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:23.447125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569070+0000","last_change":"2024-05-08T05:42:19.499539+0000","last_active":"2024-05-08T05:42:23.569070+0000","last_peered":"2024-05-08T05:42:23.569070+0000","last_clean":"2024-05-08T05:42:23.569070+0000","last_became_active":"2024-05-08T05:42:19.499241+0000","last_became_peered":"2024-05-08T05:42:19.499241+0000","last_unstale":"2024-05-08T05:42:23.569070+0000","last_undegraded":"2024-05-08T05:42:23.569070+0000","last_fullsized":"2024-05-08T05:42:23.569070+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:22.668414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569402+0000","last_change":"2024-05-08T05:42:20.533418+0000","last_active":"2024-05-08T05:42:23.569402+0000","last_peered":"2024-05-08T05:42:23.569402+0000","last_clean":"2024-05-08T05:42:23.569402+0000","last_became_active":"2024-05-08T05:42:20.532766+0000","last_became_peered":"2024-05-08T05:42:20.532766+0000","last_unstale":"2024-05-08T05:42:23.569402+0000","last_undegraded":"2024-05-08T05:42:23.569402+0000","last_fullsized":"2024-05-08T05:42:23.569402+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:44.462590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.567873+0000","last_change":"2024-05-08T05:42:19.491786+0000","last_active":"2024-05-08T05:42:23.567873+0000","last_peered":"2024-05-08T05:42:23.567873+0000","last_clean":"2024-05-08T05:42:23.567873+0000","last_became_active":"2024-05-08T05:42:19.491455+0000","last_became_peered":"2024-05-08T05:42:19.491455+0000","last_unstale":"2024-05-08T05:42:23.567873+0000","last_undegraded":"2024-05-08T05:42:23.567873+0000","last_fullsized":"2024-05-08T05:42:23.567873+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:51:24.291839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517387+0000","last_change":"2024-05-08T05:42:20.529431+0000","last_active":"2024-05-08T05:42:21.517387+0000","last_peered":"2024-05-08T05:42:21.517387+0000","last_clean":"2024-05-08T05:42:21.517387+0000","last_became_active":"2024-05-08T05:42:20.529130+0000","last_became_peered":"2024-05-08T05:42:20.529130+0000","last_unstale":"2024-05-08T05:42:21.517387+0000","last_undegraded":"2024-05-08T05:42:21.517387+0000","last_fullsized":"2024-05-08T05:42:21.517387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:51:25.293597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569722+0000","last_change":"2024-05-08T05:42:19.487759+0000","last_active":"2024-05-08T05:42:23.569722+0000","last_peered":"2024-05-08T05:42:23.569722+0000","last_clean":"2024-05-08T05:42:23.569722+0000","last_became_active":"2024-05-08T05:42:19.487492+0000","last_became_peered":"2024-05-08T05:42:19.487492+0000","last_unstale":"2024-05-08T05:42:23.569722+0000","last_undegraded":"2024-05-08T05:42:23.569722+0000","last_fullsized":"2024-05-08T05:42:23.569722+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:22.445367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.481950+0000","last_change":"2024-05-08T05:42:19.481950+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.481950+0000","last_undegraded":"2024-05-08T05:42:19.481950+0000","last_fullsized":"2024-05-08T05:42:19.481950+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.501312+0000","last_change":"2024-05-08T05:42:19.498339+0000","last_active":"2024-05-08T05:42:19.501312+0000","last_peered":"2024-05-08T05:42:19.501312+0000","last_clean":"2024-05-08T05:42:19.501312+0000","last_became_active":"2024-05-08T05:42:19.498000+0000","last_became_peered":"2024-05-08T05:42:19.498000+0000","last_unstale":"2024-05-08T05:42:19.501312+0000","last_undegraded":"2024-05-08T05:42:19.501312+0000","last_fullsized":"2024-05-08T05:42:19.501312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:22.668414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569759+0000","last_change":"2024-05-08T05:42:20.525378+0000","last_active":"2024-05-08T05:42:23.569759+0000","last_peered":"2024-05-08T05:42:23.569759+0000","last_clean":"2024-05-08T05:42:23.569759+0000","last_became_active":"2024-05-08T05:42:20.525089+0000","last_became_peered":"2024-05-08T05:42:20.525089+0000","last_unstale":"2024-05-08T05:42:23.569759+0000","last_undegraded":"2024-05-08T05:42:23.569759+0000","last_fullsized":"2024-05-08T05:42:23.569759+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:32:33.781763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.501275+0000","last_change":"2024-05-08T05:42:19.499267+0000","last_active":"2024-05-08T05:42:19.501275+0000","last_peered":"2024-05-08T05:42:19.501275+0000","last_clean":"2024-05-08T05:42:19.501275+0000","last_became_active":"2024-05-08T05:42:19.498537+0000","last_became_peered":"2024-05-08T05:42:19.498537+0000","last_unstale":"2024-05-08T05:42:19.501275+0000","last_undegraded":"2024-05-08T05:42:19.501275+0000","last_fullsized":"2024-05-08T05:42:19.501275+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:53:50.777397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568542+0000","last_change":"2024-05-08T05:42:20.525088+0000","last_active":"2024-05-08T05:42:23.568542+0000","last_peered":"2024-05-08T05:42:23.568542+0000","last_clean":"2024-05-08T05:42:23.568542+0000","last_became_active":"2024-05-08T05:42:20.524526+0000","last_became_peered":"2024-05-08T05:42:20.524526+0000","last_unstale":"2024-05-08T05:42:23.568542+0000","last_undegraded":"2024-05-08T05:42:23.568542+0000","last_fullsized":"2024-05-08T05:42:23.568542+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:06.429659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569781+0000","last_change":"2024-05-08T05:42:19.488838+0000","last_active":"2024-05-08T05:42:23.569781+0000","last_peered":"2024-05-08T05:42:23.569781+0000","last_clean":"2024-05-08T05:42:23.569781+0000","last_became_active":"2024-05-08T05:42:19.488090+0000","last_became_peered":"2024-05-08T05:42:19.488090+0000","last_unstale":"2024-05-08T05:42:23.569781+0000","last_undegraded":"2024-05-08T05:42:23.569781+0000","last_fullsized":"2024-05-08T05:42:23.569781+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:17.430149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568617+0000","last_change":"2024-05-08T05:42:20.519358+0000","last_active":"2024-05-08T05:42:23.568617+0000","last_peered":"2024-05-08T05:42:23.568617+0000","last_clean":"2024-05-08T05:42:23.568617+0000","last_became_active":"2024-05-08T05:42:20.519067+0000","last_became_peered":"2024-05-08T05:42:20.519067+0000","last_unstale":"2024-05-08T05:42:23.568617+0000","last_undegraded":"2024-05-08T05:42:23.568617+0000","last_fullsized":"2024-05-08T05:42:23.568617+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:44.235500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569519+0000","last_change":"2024-05-08T05:42:19.486715+0000","last_active":"2024-05-08T05:42:23.569519+0000","last_peered":"2024-05-08T05:42:23.569519+0000","last_clean":"2024-05-08T05:42:23.569519+0000","last_became_active":"2024-05-08T05:42:19.486403+0000","last_became_peered":"2024-05-08T05:42:19.486403+0000","last_unstale":"2024-05-08T05:42:23.569519+0000","last_undegraded":"2024-05-08T05:42:23.569519+0000","last_fullsized":"2024-05-08T05:42:23.569519+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:43.233742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568767+0000","last_change":"2024-05-08T05:42:20.518993+0000","last_active":"2024-05-08T05:42:23.568767+0000","last_peered":"2024-05-08T05:42:23.568767+0000","last_clean":"2024-05-08T05:42:23.568767+0000","last_became_active":"2024-05-08T05:42:20.518710+0000","last_became_peered":"2024-05-08T05:42:20.518710+0000","last_unstale":"2024-05-08T05:42:23.568767+0000","last_undegraded":"2024-05-08T05:42:23.568767+0000","last_fullsized":"2024-05-08T05:42:23.568767+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:30:06.045575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569189+0000","last_change":"2024-05-08T05:42:19.500123+0000","last_active":"2024-05-08T05:42:23.569189+0000","last_peered":"2024-05-08T05:42:23.569189+0000","last_clean":"2024-05-08T05:42:23.569189+0000","last_became_active":"2024-05-08T05:42:19.499811+0000","last_became_peered":"2024-05-08T05:42:19.499811+0000","last_unstale":"2024-05-08T05:42:23.569189+0000","last_undegraded":"2024-05-08T05:42:23.569189+0000","last_fullsized":"2024-05-08T05:42:23.569189+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:26.053156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.480740+0000","last_change":"2024-05-08T05:42:19.480740+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.480740+0000","last_undegraded":"2024-05-08T05:42:19.480740+0000","last_fullsized":"2024-05-08T05:42:19.480740+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.498152+0000","last_change":"2024-05-08T05:42:19.496693+0000","last_active":"2024-05-08T05:42:19.498152+0000","last_peered":"2024-05-08T05:42:19.498152+0000","last_clean":"2024-05-08T05:42:19.498152+0000","last_became_active":"2024-05-08T05:42:19.496276+0000","last_became_peered":"2024-05-08T05:42:19.496276+0000","last_unstale":"2024-05-08T05:42:19.498152+0000","last_undegraded":"2024-05-08T05:42:19.498152+0000","last_fullsized":"2024-05-08T05:42:19.498152+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:50:26.559921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569458+0000","last_change":"2024-05-08T05:42:20.521385+0000","last_active":"2024-05-08T05:42:23.569458+0000","last_peered":"2024-05-08T05:42:23.569458+0000","last_clean":"2024-05-08T05:42:23.569458+0000","last_became_active":"2024-05-08T05:42:20.521089+0000","last_became_peered":"2024-05-08T05:42:20.521089+0000","last_unstale":"2024-05-08T05:42:23.569458+0000","last_undegraded":"2024-05-08T05:42:23.569458+0000","last_fullsized":"2024-05-08T05:42:23.569458+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:01.706249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.495293+0000","last_change":"2024-05-08T05:42:19.493145+0000","last_active":"2024-05-08T05:42:19.495293+0000","last_peered":"2024-05-08T05:42:19.495293+0000","last_clean":"2024-05-08T05:42:19.495293+0000","last_became_active":"2024-05-08T05:42:19.492821+0000","last_became_peered":"2024-05-08T05:42:19.492821+0000","last_unstale":"2024-05-08T05:42:19.495293+0000","last_undegraded":"2024-05-08T05:42:19.495293+0000","last_fullsized":"2024-05-08T05:42:19.495293+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:20:51.731816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569723+0000","last_change":"2024-05-08T05:42:20.529928+0000","last_active":"2024-05-08T05:42:23.569723+0000","last_peered":"2024-05-08T05:42:23.569723+0000","last_clean":"2024-05-08T05:42:23.569723+0000","last_became_active":"2024-05-08T05:42:20.529618+0000","last_became_peered":"2024-05-08T05:42:20.529618+0000","last_unstale":"2024-05-08T05:42:23.569723+0000","last_undegraded":"2024-05-08T05:42:23.569723+0000","last_fullsized":"2024-05-08T05:42:23.569723+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:33:26.883953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.501664+0000","last_change":"2024-05-08T05:42:19.499523+0000","last_active":"2024-05-08T05:42:19.501664+0000","last_peered":"2024-05-08T05:42:19.501664+0000","last_clean":"2024-05-08T05:42:19.501664+0000","last_became_active":"2024-05-08T05:42:19.499064+0000","last_became_peered":"2024-05-08T05:42:19.499064+0000","last_unstale":"2024-05-08T05:42:19.501664+0000","last_undegraded":"2024-05-08T05:42:19.501664+0000","last_fullsized":"2024-05-08T05:42:19.501664+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:28:34.310847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.488517+0000","last_change":"2024-05-08T05:42:19.488517+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.488517+0000","last_undegraded":"2024-05-08T05:42:19.488517+0000","last_fullsized":"2024-05-08T05:42:19.488517+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568515+0000","last_change":"2024-05-08T05:42:19.500389+0000","last_active":"2024-05-08T05:42:23.568515+0000","last_peered":"2024-05-08T05:42:23.568515+0000","last_clean":"2024-05-08T05:42:23.568515+0000","last_became_active":"2024-05-08T05:42:19.500111+0000","last_became_peered":"2024-05-08T05:42:19.500111+0000","last_unstale":"2024-05-08T05:42:23.568515+0000","last_undegraded":"2024-05-08T05:42:23.568515+0000","last_fullsized":"2024-05-08T05:42:23.568515+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:14.130507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568430+0000","last_change":"2024-05-08T05:42:20.522305+0000","last_active":"2024-05-08T05:42:23.568430+0000","last_peered":"2024-05-08T05:42:23.568430+0000","last_clean":"2024-05-08T05:42:23.568430+0000","last_became_active":"2024-05-08T05:42:20.521820+0000","last_became_peered":"2024-05-08T05:42:20.521820+0000","last_unstale":"2024-05-08T05:42:23.568430+0000","last_undegraded":"2024-05-08T05:42:23.568430+0000","last_fullsized":"2024-05-08T05:42:23.568430+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:44:05.274392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517447+0000","last_change":"2024-05-08T05:42:19.491582+0000","last_active":"2024-05-08T05:42:21.517447+0000","last_peered":"2024-05-08T05:42:21.517447+0000","last_clean":"2024-05-08T05:42:21.517447+0000","last_became_active":"2024-05-08T05:42:19.491291+0000","last_became_peered":"2024-05-08T05:42:19.491291+0000","last_unstale":"2024-05-08T05:42:21.517447+0000","last_undegraded":"2024-05-08T05:42:21.517447+0000","last_fullsized":"2024-05-08T05:42:21.517447+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:43:53.904889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517451+0000","last_change":"2024-05-08T05:42:20.527117+0000","last_active":"2024-05-08T05:42:21.517451+0000","last_peered":"2024-05-08T05:42:21.517451+0000","last_clean":"2024-05-08T05:42:21.517451+0000","last_became_active":"2024-05-08T05:42:20.526717+0000","last_became_peered":"2024-05-08T05:42:20.526717+0000","last_unstale":"2024-05-08T05:42:21.517451+0000","last_undegraded":"2024-05-08T05:42:21.517451+0000","last_fullsized":"2024-05-08T05:42:21.517451+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:26.076899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568839+0000","last_change":"2024-05-08T05:42:19.491505+0000","last_active":"2024-05-08T05:42:23.568839+0000","last_peered":"2024-05-08T05:42:23.568839+0000","last_clean":"2024-05-08T05:42:23.568839+0000","last_became_active":"2024-05-08T05:42:19.491206+0000","last_became_peered":"2024-05-08T05:42:19.491206+0000","last_unstale":"2024-05-08T05:42:23.568839+0000","last_undegraded":"2024-05-08T05:42:23.568839+0000","last_fullsized":"2024-05-08T05:42:23.568839+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:53:50.777397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.480258+0000","last_change":"2024-05-08T05:42:19.480258+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.480258+0000","last_undegraded":"2024-05-08T05:42:19.480258+0000","last_fullsized":"2024-05-08T05:42:19.480258+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517236+0000","last_change":"2024-05-08T05:42:19.494577+0000","last_active":"2024-05-08T05:42:21.517236+0000","last_peered":"2024-05-08T05:42:21.517236+0000","last_clean":"2024-05-08T05:42:21.517236+0000","last_became_active":"2024-05-08T05:42:19.494261+0000","last_became_peered":"2024-05-08T05:42:19.494261+0000","last_unstale":"2024-05-08T05:42:21.517236+0000","last_undegraded":"2024-05-08T05:42:21.517236+0000","last_fullsized":"2024-05-08T05:42:21.517236+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:07:51.458867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568433+0000","last_change":"2024-05-08T05:42:20.524296+0000","last_active":"2024-05-08T05:42:23.568433+0000","last_peered":"2024-05-08T05:42:23.568433+0000","last_clean":"2024-05-08T05:42:23.568433+0000","last_became_active":"2024-05-08T05:42:20.523971+0000","last_became_peered":"2024-05-08T05:42:20.523971+0000","last_unstale":"2024-05-08T05:42:23.568433+0000","last_undegraded":"2024-05-08T05:42:23.568433+0000","last_fullsized":"2024-05-08T05:42:23.568433+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:23.447125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517421+0000","last_change":"2024-05-08T05:42:19.495869+0000","last_active":"2024-05-08T05:42:21.517421+0000","last_peered":"2024-05-08T05:42:21.517421+0000","last_clean":"2024-05-08T05:42:21.517421+0000","last_became_active":"2024-05-08T05:42:19.495494+0000","last_became_peered":"2024-05-08T05:42:19.495494+0000","last_unstale":"2024-05-08T05:42:21.517421+0000","last_undegraded":"2024-05-08T05:42:21.517421+0000","last_fullsized":"2024-05-08T05:42:21.517421+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:08.085241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.490421+0000","last_change":"2024-05-08T05:42:19.490421+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.490421+0000","last_undegraded":"2024-05-08T05:42:19.490421+0000","last_fullsized":"2024-05-08T05:42:19.490421+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.495945+0000","last_change":"2024-05-08T05:42:19.494546+0000","last_active":"2024-05-08T05:42:19.495945+0000","last_peered":"2024-05-08T05:42:19.495945+0000","last_clean":"2024-05-08T05:42:19.495945+0000","last_became_active":"2024-05-08T05:42:19.494232+0000","last_became_peered":"2024-05-08T05:42:19.494232+0000","last_unstale":"2024-05-08T05:42:19.495945+0000","last_undegraded":"2024-05-08T05:42:19.495945+0000","last_fullsized":"2024-05-08T05:42:19.495945+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:22.668414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569563+0000","last_change":"2024-05-08T05:42:20.533065+0000","last_active":"2024-05-08T05:42:23.569563+0000","last_peered":"2024-05-08T05:42:23.569563+0000","last_clean":"2024-05-08T05:42:23.569563+0000","last_became_active":"2024-05-08T05:42:20.532437+0000","last_became_peered":"2024-05-08T05:42:20.532437+0000","last_unstale":"2024-05-08T05:42:23.569563+0000","last_undegraded":"2024-05-08T05:42:23.569563+0000","last_fullsized":"2024-05-08T05:42:23.569563+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:50.284596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517058+0000","last_change":"2024-05-08T05:42:19.494499+0000","last_active":"2024-05-08T05:42:21.517058+0000","last_peered":"2024-05-08T05:42:21.517058+0000","last_clean":"2024-05-08T05:42:21.517058+0000","last_became_active":"2024-05-08T05:42:19.494193+0000","last_became_peered":"2024-05-08T05:42:19.494193+0000","last_unstale":"2024-05-08T05:42:21.517058+0000","last_undegraded":"2024-05-08T05:42:21.517058+0000","last_fullsized":"2024-05-08T05:42:21.517058+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:34.880276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568778+0000","last_change":"2024-05-08T05:42:20.518085+0000","last_active":"2024-05-08T05:42:23.568778+0000","last_peered":"2024-05-08T05:42:23.568778+0000","last_clean":"2024-05-08T05:42:23.568778+0000","last_became_active":"2024-05-08T05:42:20.517771+0000","last_became_peered":"2024-05-08T05:42:20.517771+0000","last_unstale":"2024-05-08T05:42:23.568778+0000","last_undegraded":"2024-05-08T05:42:23.568778+0000","last_fullsized":"2024-05-08T05:42:23.568778+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:18.431907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.501230+0000","last_change":"2024-05-08T05:42:19.498985+0000","last_active":"2024-05-08T05:42:19.501230+0000","last_peered":"2024-05-08T05:42:19.501230+0000","last_clean":"2024-05-08T05:42:19.501230+0000","last_became_active":"2024-05-08T05:42:19.498663+0000","last_became_peered":"2024-05-08T05:42:19.498663+0000","last_unstale":"2024-05-08T05:42:19.501230+0000","last_undegraded":"2024-05-08T05:42:19.501230+0000","last_fullsized":"2024-05-08T05:42:19.501230+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:26.053156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.481486+0000","last_change":"2024-05-08T05:42:19.481486+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.481486+0000","last_undegraded":"2024-05-08T05:42:19.481486+0000","last_fullsized":"2024-05-08T05:42:19.481486+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"36'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517320+0000","last_change":"2024-05-08T05:42:19.499917+0000","last_active":"2024-05-08T05:42:21.517320+0000","last_peered":"2024-05-08T05:42:21.517320+0000","last_clean":"2024-05-08T05:42:21.517320+0000","last_became_active":"2024-05-08T05:42:19.499614+0000","last_became_peered":"2024-05-08T05:42:19.499614+0000","last_unstale":"2024-05-08T05:42:21.517320+0000","last_undegraded":"2024-05-08T05:42:21.517320+0000","last_fullsized":"2024-05-08T05:42:21.517320+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:45.821917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569509+0000","last_change":"2024-05-08T05:42:20.529586+0000","last_active":"2024-05-08T05:42:23.569509+0000","last_peered":"2024-05-08T05:42:23.569509+0000","last_clean":"2024-05-08T05:42:23.569509+0000","last_became_active":"2024-05-08T05:42:20.529176+0000","last_became_peered":"2024-05-08T05:42:20.529176+0000","last_unstale":"2024-05-08T05:42:23.569509+0000","last_undegraded":"2024-05-08T05:42:23.569509+0000","last_fullsized":"2024-05-08T05:42:23.569509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:53.621950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569217+0000","last_change":"2024-05-08T05:42:19.492651+0000","last_active":"2024-05-08T05:42:23.569217+0000","last_peered":"2024-05-08T05:42:23.569217+0000","last_clean":"2024-05-08T05:42:23.569217+0000","last_became_active":"2024-05-08T05:42:19.491268+0000","last_became_peered":"2024-05-08T05:42:19.491268+0000","last_unstale":"2024-05-08T05:42:23.569217+0000","last_undegraded":"2024-05-08T05:42:23.569217+0000","last_fullsized":"2024-05-08T05:42:23.569217+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:29.265240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568690+0000","last_change":"2024-05-08T05:42:20.525312+0000","last_active":"2024-05-08T05:42:23.568690+0000","last_peered":"2024-05-08T05:42:23.568690+0000","last_clean":"2024-05-08T05:42:23.568690+0000","last_became_active":"2024-05-08T05:42:20.524640+0000","last_became_peered":"2024-05-08T05:42:20.524640+0000","last_unstale":"2024-05-08T05:42:23.568690+0000","last_undegraded":"2024-05-08T05:42:23.568690+0000","last_fullsized":"2024-05-08T05:42:23.568690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:18.431907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569450+0000","last_change":"2024-05-08T05:42:19.494149+0000","last_active":"2024-05-08T05:42:23.569450+0000","last_peered":"2024-05-08T05:42:23.569450+0000","last_clean":"2024-05-08T05:42:23.569450+0000","last_became_active":"2024-05-08T05:42:19.493848+0000","last_became_peered":"2024-05-08T05:42:19.493848+0000","last_unstale":"2024-05-08T05:42:23.569450+0000","last_undegraded":"2024-05-08T05:42:23.569450+0000","last_fullsized":"2024-05-08T05:42:23.569450+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:25.075141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569011+0000","last_change":"2024-05-08T05:42:20.518816+0000","last_active":"2024-05-08T05:42:23.569011+0000","last_peered":"2024-05-08T05:42:23.569011+0000","last_clean":"2024-05-08T05:42:23.569011+0000","last_became_active":"2024-05-08T05:42:20.518514+0000","last_became_peered":"2024-05-08T05:42:20.518514+0000","last_unstale":"2024-05-08T05:42:23.569011+0000","last_undegraded":"2024-05-08T05:42:23.569011+0000","last_fullsized":"2024-05-08T05:42:23.569011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:06.429659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.499479+0000","last_change":"2024-05-08T05:42:19.497480+0000","last_active":"2024-05-08T05:42:19.499479+0000","last_peered":"2024-05-08T05:42:19.499479+0000","last_clean":"2024-05-08T05:42:19.499479+0000","last_became_active":"2024-05-08T05:42:19.497177+0000","last_became_peered":"2024-05-08T05:42:19.497177+0000","last_unstale":"2024-05-08T05:42:19.499479+0000","last_undegraded":"2024-05-08T05:42:19.499479+0000","last_fullsized":"2024-05-08T05:42:19.499479+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:58:57.734024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568064+0000","last_change":"2024-05-08T05:42:20.530479+0000","last_active":"2024-05-08T05:42:23.568064+0000","last_peered":"2024-05-08T05:42:23.568064+0000","last_clean":"2024-05-08T05:42:23.568064+0000","last_became_active":"2024-05-08T05:42:20.530210+0000","last_became_peered":"2024-05-08T05:42:20.530210+0000","last_unstale":"2024-05-08T05:42:23.568064+0000","last_undegraded":"2024-05-08T05:42:23.568064+0000","last_fullsized":"2024-05-08T05:42:23.568064+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:58.041468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.567873+0000","last_change":"2024-05-08T05:42:19.495013+0000","last_active":"2024-05-08T05:42:23.567873+0000","last_peered":"2024-05-08T05:42:23.567873+0000","last_clean":"2024-05-08T05:42:23.567873+0000","last_became_active":"2024-05-08T05:42:19.494694+0000","last_became_peered":"2024-05-08T05:42:19.494694+0000","last_unstale":"2024-05-08T05:42:23.567873+0000","last_undegraded":"2024-05-08T05:42:23.567873+0000","last_fullsized":"2024-05-08T05:42:23.567873+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:29.265240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.483121+0000","last_change":"2024-05-08T05:42:19.483121+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.483121+0000","last_undegraded":"2024-05-08T05:42:19.483121+0000","last_fullsized":"2024-05-08T05:42:19.483121+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568473+0000","last_change":"2024-05-08T05:42:19.487951+0000","last_active":"2024-05-08T05:42:23.568473+0000","last_peered":"2024-05-08T05:42:23.568473+0000","last_clean":"2024-05-08T05:42:23.568473+0000","last_became_active":"2024-05-08T05:42:19.487613+0000","last_became_peered":"2024-05-08T05:42:19.487613+0000","last_unstale":"2024-05-08T05:42:23.568473+0000","last_undegraded":"2024-05-08T05:42:23.568473+0000","last_fullsized":"2024-05-08T05:42:23.568473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:57.039710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.489194+0000","last_change":"2024-05-08T05:42:19.489194+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.489194+0000","last_undegraded":"2024-05-08T05:42:19.489194+0000","last_fullsized":"2024-05-08T05:42:19.489194+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.489865+0000","last_change":"2024-05-08T05:42:19.487846+0000","last_active":"2024-05-08T05:42:19.489865+0000","last_peered":"2024-05-08T05:42:19.489865+0000","last_clean":"2024-05-08T05:42:19.489865+0000","last_became_active":"2024-05-08T05:42:19.487570+0000","last_became_peered":"2024-05-08T05:42:19.487570+0000","last_unstale":"2024-05-08T05:42:19.489865+0000","last_undegraded":"2024-05-08T05:42:19.489865+0000","last_fullsized":"2024-05-08T05:42:19.489865+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:25.075141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":135,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.483776+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:42:19.483776+0000","last_peered":"2024-05-08T05:42:19.483776+0000","last_clean":"2024-05-08T05:42:19.483776+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:42:19.483776+0000","last_undegraded":"2024-05-08T05:42:19.483776+0000","last_fullsized":"2024-05-08T05:42:19.483776+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1716,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":966656,"data_stored":744313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":372736,"data_stored":153945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542160,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":160612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":175075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":425984,"data_stored":176117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":413696,"data_stored":174541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":167869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":995328,"data_stored":758181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:42:24.929 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:42:25.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.069+0000 7f08a7ae8640 1 Processor -- start 2024-05-08T05:42:25.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.070+0000 7f08a7ae8640 1 -- start start 2024-05-08T05:42:25.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a7ae8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08a00b5ca0 0x7f08a00d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a7ae8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08a01606c0 con 0x7f08a015f2d0 2024-05-08T05:42:25.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a7ae8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08a00b4f00 con 0x7f08a00d56d0 2024-05-08T05:42:25.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a7ae8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f08a00b5810 con 0x7f08a00b5ca0 2024-05-08T05:42:25.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a5884640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f08a015f2d0 0x7f08a00d4fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56066/0 (socket says 172.21.15.45:56066) 2024-05-08T05:42:25.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a5884640 1 -- 172.21.15.45:0/3112815905 learned_addr learned my addr 172.21.15.45:0/3112815905 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:25.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a6085640 1 --2- 172.21.15.45:0/3112815905 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08a00b5ca0 0x7f08a00d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a4882640 1 -- 172.21.15.45:0/3112815905 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3557339761 0 0) 0x7f08a01606c0 con 0x7f08a015f2d0 2024-05-08T05:42:25.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a4882640 1 -- 172.21.15.45:0/3112815905 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f088c003520 con 0x7f08a015f2d0 2024-05-08T05:42:25.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a4882640 1 -- 172.21.15.45:0/3112815905 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2562114183 0 0) 0x7f088c003520 con 0x7f08a015f2d0 2024-05-08T05:42:25.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a4882640 1 -- 172.21.15.45:0/3112815905 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08a00b5ca0 msgr2=0x7f08a00d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:25.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a4882640 1 --2- 172.21.15.45:0/3112815905 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08a00b5ca0 0x7f08a00d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a4882640 1 -- 172.21.15.45:0/3112815905 >> v1:172.21.15.63:6789/0 conn(0x7f08a00d56d0 legacy=0x7f08a00d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:25.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a4882640 1 -- 172.21.15.45:0/3112815905 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a00b51c0 con 0x7f08a015f2d0 2024-05-08T05:42:25.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a4882640 1 -- 172.21.15.45:0/3112815905 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f089c0016f0 con 0x7f08a015f2d0 2024-05-08T05:42:25.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a4882640 1 -- 172.21.15.45:0/3112815905 >> v1:172.21.15.45:6789/0 conn(0x7f08a015f2d0 legacy=0x7f08a00d4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:25.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a4882640 1 --2- 172.21.15.45:0/3112815905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f088c0039e0 0x7f088c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a4882640 1 --2- 172.21.15.45:0/3112815905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f088c004800 0x7f088c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a4882640 1 --2- 172.21.15.45:0/3112815905 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08a00b5ca0 0x7f088c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a4882640 1 -- 172.21.15.45:0/3112815905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f08a00b5810 con 0x7f088c004800 2024-05-08T05:42:25.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a4882640 1 -- 172.21.15.45:0/3112815905 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f088c00baa0 con 0x7f088c0039e0 2024-05-08T05:42:25.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a4882640 1 -- 172.21.15.45:0/3112815905 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f088c00bca0 con 0x7f08a00b5ca0 2024-05-08T05:42:25.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.071+0000 7f08a5083640 1 --2- 172.21.15.45:0/3112815905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f088c004800 0x7f088c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.072+0000 7f08a5884640 1 --2- 172.21.15.45:0/3112815905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f088c0039e0 0x7f088c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.072+0000 7f08a6085640 1 --2- 172.21.15.45:0/3112815905 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08a00b5ca0 0x7f088c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.072+0000 7f08a5083640 1 -- 172.21.15.45:0/3112815905 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08a00b5ca0 msgr2=0x7f088c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:25.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.072+0000 7f08a5083640 1 --2- 172.21.15.45:0/3112815905 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08a00b5ca0 0x7f088c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.072+0000 7f08a5083640 1 -- 172.21.15.45:0/3112815905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f088c0039e0 msgr2=0x7f088c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:25.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.072+0000 7f08a5083640 1 --2- 172.21.15.45:0/3112815905 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f088c0039e0 0x7f088c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.072+0000 7f08a5083640 1 -- 172.21.15.45:0/3112815905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088c003520 con 0x7f088c004800 2024-05-08T05:42:25.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.073+0000 7f08a5083640 1 --2- 172.21.15.45:0/3112815905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f088c004800 0x7f088c004c80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f08900017d0 tx=0x7f088c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc874729b887deb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:25.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.073+0000 7f08a4882640 1 -- 172.21.15.45:0/3112815905 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890037070 con 0x7f088c004800 2024-05-08T05:42:25.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.073+0000 7f08a4882640 1 -- 172.21.15.45:0/3112815905 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f089002ac00 con 0x7f088c004800 2024-05-08T05:42:25.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.073+0000 7f08a4882640 1 -- 172.21.15.45:0/3112815905 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08900013d0 con 0x7f088c004800 2024-05-08T05:42:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.073+0000 7f08a7ae8640 1 -- 172.21.15.45:0/3112815905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f088c004800 msgr2=0x7f088c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.073+0000 7f08a7ae8640 1 --2- 172.21.15.45:0/3112815905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f088c004800 0x7f088c004c80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f08900017d0 tx=0x7f088c005830 comp rx=0 tx=0).stop 2024-05-08T05:42:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.073+0000 7f08a6085640 1 -- 172.21.15.45:0/3112815905 reap_dead start 2024-05-08T05:42:25.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.073+0000 7f08a7ae8640 1 -- 172.21.15.45:0/3112815905 shutdown_connections 2024-05-08T05:42:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.073+0000 7f08a7ae8640 1 -- 172.21.15.45:0/3112815905 >> 172.21.15.45:0/3112815905 conn(0x7f08a00746d0 msgr2=0x7f08a009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.073+0000 7f08a7ae8640 1 -- 172.21.15.45:0/3112815905 shutdown_connections 2024-05-08T05:42:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.073+0000 7f08a7ae8640 1 -- 172.21.15.45:0/3112815905 wait complete. 2024-05-08T05:42:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.074+0000 7f08a7ae8640 1 Processor -- start 2024-05-08T05:42:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.075+0000 7f08a7ae8640 1 -- start start 2024-05-08T05:42:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.075+0000 7f08a7ae8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f088c004800 0x7f08a0148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.075+0000 7f08a7ae8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08a015ab00 0x7f08a0149670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.075+0000 7f08a7ae8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08a015af60 0x7f08a0161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.075+0000 7f08a7ae8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f08a00d2490 con 0x7f08a015ab00 2024-05-08T05:42:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.075+0000 7f08a7ae8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f08a00d2310 con 0x7f088c004800 2024-05-08T05:42:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.075+0000 7f08a7ae8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f08a00d2610 con 0x7f08a015af60 2024-05-08T05:42:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.075+0000 7f08a5083640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08a015ab00 0x7f08a0149670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.075+0000 7f08a5083640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08a015ab00 0x7f08a0149670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44244/0 (socket says 172.21.15.45:44244) 2024-05-08T05:42:25.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.075+0000 7f08a5083640 1 -- 172.21.15.45:0/2325527162 learned_addr learned my addr 172.21.15.45:0/2325527162 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.076+0000 7f08a5083640 1 -- 172.21.15.45:0/2325527162 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08a015af60 msgr2=0x7f08a0161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.076+0000 7f08a5083640 1 --2- 172.21.15.45:0/2325527162 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08a015af60 0x7f08a0161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.076+0000 7f08a5884640 1 --2- 172.21.15.45:0/2325527162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f088c004800 0x7f08a0148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.076+0000 7f08a5083640 1 -- 172.21.15.45:0/2325527162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f088c004800 msgr2=0x7f08a0148d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.076+0000 7f08a5083640 1 --2- 172.21.15.45:0/2325527162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f088c004800 0x7f08a0148d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.076+0000 7f08a5083640 1 -- 172.21.15.45:0/2325527162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a00cdad0 con 0x7f08a015ab00 2024-05-08T05:42:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.076+0000 7f08a5884640 1 --2- 172.21.15.45:0/2325527162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f088c004800 0x7f08a0148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.076+0000 7f08a5083640 1 --2- 172.21.15.45:0/2325527162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08a015ab00 0x7f08a0149670 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f08900018f0 tx=0x7f0890001110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.076+0000 7f08967fc640 1 -- 172.21.15.45:0/2325527162 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089003f070 con 0x7f08a015ab00 2024-05-08T05:42:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.076+0000 7f08967fc640 1 -- 172.21.15.45:0/2325527162 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0890003bf0 con 0x7f08a015ab00 2024-05-08T05:42:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.076+0000 7f08a7ae8640 1 -- 172.21.15.45:0/2325527162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a00cc3b0 con 0x7f08a015ab00 2024-05-08T05:42:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.076+0000 7f08967fc640 1 -- 172.21.15.45:0/2325527162 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0890037040 con 0x7f08a015ab00 2024-05-08T05:42:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.076+0000 7f08a7ae8640 1 -- 172.21.15.45:0/2325527162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a00ce050 con 0x7f08a015ab00 2024-05-08T05:42:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.077+0000 7f08a7ae8640 1 -- 172.21.15.45:0/2325527162 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08a0069b00 con 0x7f08a015ab00 2024-05-08T05:42:25.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.078+0000 7f08967fc640 1 -- 172.21.15.45:0/2325527162 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0890032440 con 0x7f08a015ab00 2024-05-08T05:42:25.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.078+0000 7f08967fc640 1 --2- 172.21.15.45:0/2325527162 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f08840762d0 0x7f0884078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.078+0000 7f08967fc640 1 -- 172.21.15.45:0/2325527162 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f08900b7150 con 0x7f08a015ab00 2024-05-08T05:42:25.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.078+0000 7f08a5884640 1 --2- 172.21.15.45:0/2325527162 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f08840762d0 0x7f0884078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.079+0000 7f08a5884640 1 --2- 172.21.15.45:0/2325527162 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f08840762d0 0x7f0884078790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f089c001eb0 tx=0x7f089c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:25.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.082+0000 7f08967fc640 1 -- 172.21.15.45:0/2325527162 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f08900802d0 con 0x7f08a015ab00 2024-05-08T05:42:25.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.270+0000 7f08a7ae8640 1 -- 172.21.15.45:0/2325527162 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f08a00baeb0 con 0x7f08840762d0 2024-05-08T05:42:25.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.277+0000 7f08967fc640 1 -- 172.21.15.45:0/2325527162 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249708 (secure 0 0 0) 0x7f08a00baeb0 con 0x7f08840762d0 2024-05-08T05:42:25.280 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:25.281 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:42:25.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.280+0000 7f08a7ae8640 1 -- 172.21.15.45:0/2325527162 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f08840762d0 msgr2=0x7f0884078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:25.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.280+0000 7f08a7ae8640 1 --2- 172.21.15.45:0/2325527162 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f08840762d0 0x7f0884078790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f089c001eb0 tx=0x7f089c009040 comp rx=0 tx=0).stop 2024-05-08T05:42:25.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.280+0000 7f08a7ae8640 1 -- 172.21.15.45:0/2325527162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08a015ab00 msgr2=0x7f08a0149670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:25.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.280+0000 7f08a7ae8640 1 --2- 172.21.15.45:0/2325527162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08a015ab00 0x7f08a0149670 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f08900018f0 tx=0x7f0890001110 comp rx=0 tx=0).stop 2024-05-08T05:42:25.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.280+0000 7f08a7ae8640 1 -- 172.21.15.45:0/2325527162 shutdown_connections 2024-05-08T05:42:25.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.280+0000 7f08a7ae8640 1 --2- 172.21.15.45:0/2325527162 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f08840762d0 0x7f0884078790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.280+0000 7f08a7ae8640 1 --2- 172.21.15.45:0/2325527162 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08a015af60 0x7f08a0161eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.280+0000 7f08a7ae8640 1 --2- 172.21.15.45:0/2325527162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08a015ab00 0x7f08a0149670 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.280+0000 7f08a7ae8640 1 --2- 172.21.15.45:0/2325527162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f088c004800 0x7f08a0148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.280+0000 7f08a7ae8640 1 -- 172.21.15.45:0/2325527162 >> 172.21.15.45:0/2325527162 conn(0x7f08a00746d0 msgr2=0x7f08a00cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:25.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.280+0000 7f08a7ae8640 1 -- 172.21.15.45:0/2325527162 shutdown_connections 2024-05-08T05:42:25.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.281+0000 7f08a7ae8640 1 -- 172.21.15.45:0/2325527162 wait complete. 2024-05-08T05:42:25.292 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-05-08T05:42:24.845538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592084,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":122,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237088,"kb_used_data":4276,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737632,"statfs":{"total":773094113280,"available":772851335168,"internally_reserved":0,"allocated":4378624,"data_stored":2510653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1716,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000980"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569700+0000","last_change":"2024-05-08T05:42:20.521354+0000","last_active":"2024-05-08T05:42:23.569700+0000","last_peered":"2024-05-08T05:42:23.569700+0000","last_clean":"2024-05-08T05:42:23.569700+0000","last_became_active":"2024-05-08T05:42:20.521070+0000","last_became_peered":"2024-05-08T05:42:20.521070+0000","last_unstale":"2024-05-08T05:42:23.569700+0000","last_undegraded":"2024-05-08T05:42:23.569700+0000","last_fullsized":"2024-05-08T05:42:23.569700+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:08.311556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568750+0000","last_change":"2024-05-08T05:42:20.525189+0000","last_active":"2024-05-08T05:42:23.568750+0000","last_peered":"2024-05-08T05:42:23.568750+0000","last_clean":"2024-05-08T05:42:23.568750+0000","last_became_active":"2024-05-08T05:42:20.524906+0000","last_became_peered":"2024-05-08T05:42:20.524906+0000","last_unstale":"2024-05-08T05:42:23.568750+0000","last_undegraded":"2024-05-08T05:42:23.568750+0000","last_fullsized":"2024-05-08T05:42:23.568750+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:22:33.614866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569160+0000","last_change":"2024-05-08T05:42:20.532374+0000","last_active":"2024-05-08T05:42:23.569160+0000","last_peered":"2024-05-08T05:42:23.569160+0000","last_clean":"2024-05-08T05:42:23.569160+0000","last_became_active":"2024-05-08T05:42:20.532079+0000","last_became_peered":"2024-05-08T05:42:20.532079+0000","last_unstale":"2024-05-08T05:42:23.569160+0000","last_undegraded":"2024-05-08T05:42:23.569160+0000","last_fullsized":"2024-05-08T05:42:23.569160+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:23.089039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517311+0000","last_change":"2024-05-08T05:42:20.530407+0000","last_active":"2024-05-08T05:42:21.517311+0000","last_peered":"2024-05-08T05:42:21.517311+0000","last_clean":"2024-05-08T05:42:21.517311+0000","last_became_active":"2024-05-08T05:42:20.530148+0000","last_became_peered":"2024-05-08T05:42:20.530148+0000","last_unstale":"2024-05-08T05:42:21.517311+0000","last_undegraded":"2024-05-08T05:42:21.517311+0000","last_fullsized":"2024-05-08T05:42:21.517311+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:30.266997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517173+0000","last_change":"2024-05-08T05:42:20.528145+0000","last_active":"2024-05-08T05:42:21.517173+0000","last_peered":"2024-05-08T05:42:21.517173+0000","last_clean":"2024-05-08T05:42:21.517173+0000","last_became_active":"2024-05-08T05:42:20.527841+0000","last_became_peered":"2024-05-08T05:42:20.527841+0000","last_unstale":"2024-05-08T05:42:21.517173+0000","last_undegraded":"2024-05-08T05:42:21.517173+0000","last_fullsized":"2024-05-08T05:42:21.517173+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:15.132265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569078+0000","last_change":"2024-05-08T05:42:20.518897+0000","last_active":"2024-05-08T05:42:23.569078+0000","last_peered":"2024-05-08T05:42:23.569078+0000","last_clean":"2024-05-08T05:42:23.569078+0000","last_became_active":"2024-05-08T05:42:20.518613+0000","last_became_peered":"2024-05-08T05:42:20.518613+0000","last_unstale":"2024-05-08T05:42:23.569078+0000","last_undegraded":"2024-05-08T05:42:23.569078+0000","last_fullsized":"2024-05-08T05:42:23.569078+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:23.089039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.494063+0000","last_change":"2024-05-08T05:42:19.494063+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.494063+0000","last_undegraded":"2024-05-08T05:42:19.494063+0000","last_fullsized":"2024-05-08T05:42:19.494063+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568409+0000","last_change":"2024-05-08T05:42:20.515418+0000","last_active":"2024-05-08T05:42:23.568409+0000","last_peered":"2024-05-08T05:42:23.568409+0000","last_clean":"2024-05-08T05:42:23.568409+0000","last_became_active":"2024-05-08T05:42:20.514961+0000","last_became_peered":"2024-05-08T05:42:20.514961+0000","last_unstale":"2024-05-08T05:42:23.568409+0000","last_undegraded":"2024-05-08T05:42:23.568409+0000","last_fullsized":"2024-05-08T05:42:23.568409+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:54:21.060834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569512+0000","last_change":"2024-05-08T05:42:20.533002+0000","last_active":"2024-05-08T05:42:23.569512+0000","last_peered":"2024-05-08T05:42:23.569512+0000","last_clean":"2024-05-08T05:42:23.569512+0000","last_became_active":"2024-05-08T05:42:20.532713+0000","last_became_peered":"2024-05-08T05:42:20.532713+0000","last_unstale":"2024-05-08T05:42:23.569512+0000","last_undegraded":"2024-05-08T05:42:23.569512+0000","last_fullsized":"2024-05-08T05:42:23.569512+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:34:08.047335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.492151+0000","last_change":"2024-05-08T05:42:19.492151+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.492151+0000","last_undegraded":"2024-05-08T05:42:19.492151+0000","last_fullsized":"2024-05-08T05:42:19.492151+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.489300+0000","last_change":"2024-05-08T05:42:19.489300+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.489300+0000","last_undegraded":"2024-05-08T05:42:19.489300+0000","last_fullsized":"2024-05-08T05:42:19.489300+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568992+0000","last_change":"2024-05-08T05:42:20.527823+0000","last_active":"2024-05-08T05:42:23.568992+0000","last_peered":"2024-05-08T05:42:23.568992+0000","last_clean":"2024-05-08T05:42:23.568992+0000","last_became_active":"2024-05-08T05:42:20.527531+0000","last_became_peered":"2024-05-08T05:42:20.527531+0000","last_unstale":"2024-05-08T05:42:23.568992+0000","last_undegraded":"2024-05-08T05:42:23.568992+0000","last_fullsized":"2024-05-08T05:42:23.568992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:06.429659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.488571+0000","last_change":"2024-05-08T05:42:19.488571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.488571+0000","last_undegraded":"2024-05-08T05:42:19.488571+0000","last_fullsized":"2024-05-08T05:42:19.488571+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568056+0000","last_change":"2024-05-08T05:42:20.520112+0000","last_active":"2024-05-08T05:42:23.568056+0000","last_peered":"2024-05-08T05:42:23.568056+0000","last_clean":"2024-05-08T05:42:23.568056+0000","last_became_active":"2024-05-08T05:42:20.519660+0000","last_became_peered":"2024-05-08T05:42:20.519660+0000","last_unstale":"2024-05-08T05:42:23.568056+0000","last_undegraded":"2024-05-08T05:42:23.568056+0000","last_fullsized":"2024-05-08T05:42:23.568056+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:25:23.199079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.491652+0000","last_change":"2024-05-08T05:42:19.491652+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.491652+0000","last_undegraded":"2024-05-08T05:42:19.491652+0000","last_fullsized":"2024-05-08T05:42:19.491652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.483214+0000","last_change":"2024-05-08T05:42:19.483214+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.483214+0000","last_undegraded":"2024-05-08T05:42:19.483214+0000","last_fullsized":"2024-05-08T05:42:19.483214+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.482113+0000","last_change":"2024-05-08T05:42:19.482113+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.482113+0000","last_undegraded":"2024-05-08T05:42:19.482113+0000","last_fullsized":"2024-05-08T05:42:19.482113+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.486278+0000","last_change":"2024-05-08T05:42:19.486278+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.486278+0000","last_undegraded":"2024-05-08T05:42:19.486278+0000","last_fullsized":"2024-05-08T05:42:19.486278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517514+0000","last_change":"2024-05-08T05:42:20.533970+0000","last_active":"2024-05-08T05:42:21.517514+0000","last_peered":"2024-05-08T05:42:21.517514+0000","last_clean":"2024-05-08T05:42:21.517514+0000","last_became_active":"2024-05-08T05:42:20.531975+0000","last_became_peered":"2024-05-08T05:42:20.531975+0000","last_unstale":"2024-05-08T05:42:21.517514+0000","last_undegraded":"2024-05-08T05:42:21.517514+0000","last_fullsized":"2024-05-08T05:42:21.517514+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:20:52.733574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.489219+0000","last_change":"2024-05-08T05:42:19.489219+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.489219+0000","last_undegraded":"2024-05-08T05:42:19.489219+0000","last_fullsized":"2024-05-08T05:42:19.489219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.489042+0000","last_change":"2024-05-08T05:42:19.489042+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.489042+0000","last_undegraded":"2024-05-08T05:42:19.489042+0000","last_fullsized":"2024-05-08T05:42:19.489042+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568464+0000","last_change":"2024-05-08T05:42:20.525789+0000","last_active":"2024-05-08T05:42:23.568464+0000","last_peered":"2024-05-08T05:42:23.568464+0000","last_clean":"2024-05-08T05:42:23.568464+0000","last_became_active":"2024-05-08T05:42:20.525492+0000","last_became_peered":"2024-05-08T05:42:20.525492+0000","last_unstale":"2024-05-08T05:42:23.568464+0000","last_undegraded":"2024-05-08T05:42:23.568464+0000","last_fullsized":"2024-05-08T05:42:23.568464+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:44.235500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.481502+0000","last_change":"2024-05-08T05:42:19.481502+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.481502+0000","last_undegraded":"2024-05-08T05:42:19.481502+0000","last_fullsized":"2024-05-08T05:42:19.481502+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517037+0000","last_change":"2024-05-08T05:42:20.534832+0000","last_active":"2024-05-08T05:42:21.517037+0000","last_peered":"2024-05-08T05:42:21.517037+0000","last_clean":"2024-05-08T05:42:21.517037+0000","last_became_active":"2024-05-08T05:42:20.534586+0000","last_became_peered":"2024-05-08T05:42:20.534586+0000","last_unstale":"2024-05-08T05:42:21.517037+0000","last_undegraded":"2024-05-08T05:42:21.517037+0000","last_fullsized":"2024-05-08T05:42:21.517037+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:30.319294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.485694+0000","last_change":"2024-05-08T05:42:19.485694+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.485694+0000","last_undegraded":"2024-05-08T05:42:19.485694+0000","last_fullsized":"2024-05-08T05:42:19.485694+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569431+0000","last_change":"2024-05-08T05:42:20.509600+0000","last_active":"2024-05-08T05:42:23.569431+0000","last_peered":"2024-05-08T05:42:23.569431+0000","last_clean":"2024-05-08T05:42:23.569431+0000","last_became_active":"2024-05-08T05:42:20.509045+0000","last_became_peered":"2024-05-08T05:42:20.509045+0000","last_unstale":"2024-05-08T05:42:23.569431+0000","last_undegraded":"2024-05-08T05:42:23.569431+0000","last_fullsized":"2024-05-08T05:42:23.569431+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:39.499482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.486709+0000","last_change":"2024-05-08T05:42:19.486709+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.486709+0000","last_undegraded":"2024-05-08T05:42:19.486709+0000","last_fullsized":"2024-05-08T05:42:19.486709+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.488095+0000","last_change":"2024-05-08T05:42:19.488095+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.488095+0000","last_undegraded":"2024-05-08T05:42:19.488095+0000","last_fullsized":"2024-05-08T05:42:19.488095+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.488425+0000","last_change":"2024-05-08T05:42:19.488425+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.488425+0000","last_undegraded":"2024-05-08T05:42:19.488425+0000","last_fullsized":"2024-05-08T05:42:19.488425+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.485913+0000","last_change":"2024-05-08T05:42:19.485913+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.485913+0000","last_undegraded":"2024-05-08T05:42:19.485913+0000","last_fullsized":"2024-05-08T05:42:19.485913+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.481029+0000","last_change":"2024-05-08T05:42:19.481029+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.481029+0000","last_undegraded":"2024-05-08T05:42:19.481029+0000","last_fullsized":"2024-05-08T05:42:19.481029+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569182+0000","last_change":"2024-05-08T05:42:20.524631+0000","last_active":"2024-05-08T05:42:23.569182+0000","last_peered":"2024-05-08T05:42:23.569182+0000","last_clean":"2024-05-08T05:42:23.569182+0000","last_became_active":"2024-05-08T05:42:20.524354+0000","last_became_peered":"2024-05-08T05:42:20.524354+0000","last_unstale":"2024-05-08T05:42:23.569182+0000","last_undegraded":"2024-05-08T05:42:23.569182+0000","last_fullsized":"2024-05-08T05:42:23.569182+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:34:08.047335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.481092+0000","last_change":"2024-05-08T05:42:19.481092+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.481092+0000","last_undegraded":"2024-05-08T05:42:19.481092+0000","last_fullsized":"2024-05-08T05:42:19.481092+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.497640+0000","last_change":"2024-05-08T05:42:19.493864+0000","last_active":"2024-05-08T05:42:19.497640+0000","last_peered":"2024-05-08T05:42:19.497640+0000","last_clean":"2024-05-08T05:42:19.497640+0000","last_became_active":"2024-05-08T05:42:19.493061+0000","last_became_peered":"2024-05-08T05:42:19.493061+0000","last_unstale":"2024-05-08T05:42:19.497640+0000","last_undegraded":"2024-05-08T05:42:19.497640+0000","last_fullsized":"2024-05-08T05:42:19.497640+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:47:27.293889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569291+0000","last_change":"2024-05-08T05:42:20.512807+0000","last_active":"2024-05-08T05:42:23.569291+0000","last_peered":"2024-05-08T05:42:23.569291+0000","last_clean":"2024-05-08T05:42:23.569291+0000","last_became_active":"2024-05-08T05:42:20.512501+0000","last_became_peered":"2024-05-08T05:42:20.512501+0000","last_unstale":"2024-05-08T05:42:23.569291+0000","last_undegraded":"2024-05-08T05:42:23.569291+0000","last_fullsized":"2024-05-08T05:42:23.569291+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:32:33.781763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569814+0000","last_change":"2024-05-08T05:42:19.493675+0000","last_active":"2024-05-08T05:42:23.569814+0000","last_peered":"2024-05-08T05:42:23.569814+0000","last_clean":"2024-05-08T05:42:23.569814+0000","last_became_active":"2024-05-08T05:42:19.493339+0000","last_became_peered":"2024-05-08T05:42:19.493339+0000","last_unstale":"2024-05-08T05:42:23.569814+0000","last_undegraded":"2024-05-08T05:42:23.569814+0000","last_fullsized":"2024-05-08T05:42:23.569814+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:51:24.291839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568501+0000","last_change":"2024-05-08T05:42:20.521416+0000","last_active":"2024-05-08T05:42:23.568501+0000","last_peered":"2024-05-08T05:42:23.568501+0000","last_clean":"2024-05-08T05:42:23.568501+0000","last_became_active":"2024-05-08T05:42:20.520422+0000","last_became_peered":"2024-05-08T05:42:20.520422+0000","last_unstale":"2024-05-08T05:42:23.568501+0000","last_undegraded":"2024-05-08T05:42:23.568501+0000","last_fullsized":"2024-05-08T05:42:23.568501+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:18.431907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569440+0000","last_change":"2024-05-08T05:42:19.491376+0000","last_active":"2024-05-08T05:42:23.569440+0000","last_peered":"2024-05-08T05:42:23.569440+0000","last_clean":"2024-05-08T05:42:23.569440+0000","last_became_active":"2024-05-08T05:42:19.491075+0000","last_became_peered":"2024-05-08T05:42:19.491075+0000","last_unstale":"2024-05-08T05:42:23.569440+0000","last_undegraded":"2024-05-08T05:42:23.569440+0000","last_fullsized":"2024-05-08T05:42:23.569440+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:28:34.310847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569384+0000","last_change":"2024-05-08T05:42:20.529711+0000","last_active":"2024-05-08T05:42:23.569384+0000","last_peered":"2024-05-08T05:42:23.569384+0000","last_clean":"2024-05-08T05:42:23.569384+0000","last_became_active":"2024-05-08T05:42:20.529354+0000","last_became_peered":"2024-05-08T05:42:20.529354+0000","last_unstale":"2024-05-08T05:42:23.569384+0000","last_undegraded":"2024-05-08T05:42:23.569384+0000","last_fullsized":"2024-05-08T05:42:23.569384+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:43:42.826524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568253+0000","last_change":"2024-05-08T05:42:19.495593+0000","last_active":"2024-05-08T05:42:23.568253+0000","last_peered":"2024-05-08T05:42:23.568253+0000","last_clean":"2024-05-08T05:42:23.568253+0000","last_became_active":"2024-05-08T05:42:19.495297+0000","last_became_peered":"2024-05-08T05:42:19.495297+0000","last_unstale":"2024-05-08T05:42:23.568253+0000","last_undegraded":"2024-05-08T05:42:23.568253+0000","last_fullsized":"2024-05-08T05:42:23.568253+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:29.317536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568467+0000","last_change":"2024-05-08T05:42:20.525554+0000","last_active":"2024-05-08T05:42:23.568467+0000","last_peered":"2024-05-08T05:42:23.568467+0000","last_clean":"2024-05-08T05:42:23.568467+0000","last_became_active":"2024-05-08T05:42:20.525027+0000","last_became_peered":"2024-05-08T05:42:20.525027+0000","last_unstale":"2024-05-08T05:42:23.568467+0000","last_undegraded":"2024-05-08T05:42:23.568467+0000","last_fullsized":"2024-05-08T05:42:23.568467+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:44:05.274392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.501668+0000","last_change":"2024-05-08T05:42:19.499533+0000","last_active":"2024-05-08T05:42:19.501668+0000","last_peered":"2024-05-08T05:42:19.501668+0000","last_clean":"2024-05-08T05:42:19.501668+0000","last_became_active":"2024-05-08T05:42:19.498447+0000","last_became_peered":"2024-05-08T05:42:19.498447+0000","last_unstale":"2024-05-08T05:42:19.501668+0000","last_undegraded":"2024-05-08T05:42:19.501668+0000","last_fullsized":"2024-05-08T05:42:19.501668+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:37.041564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.488852+0000","last_change":"2024-05-08T05:42:19.488852+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.488852+0000","last_undegraded":"2024-05-08T05:42:19.488852+0000","last_fullsized":"2024-05-08T05:42:19.488852+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.497089+0000","last_change":"2024-05-08T05:42:19.495816+0000","last_active":"2024-05-08T05:42:19.497089+0000","last_peered":"2024-05-08T05:42:19.497089+0000","last_clean":"2024-05-08T05:42:19.497089+0000","last_became_active":"2024-05-08T05:42:19.495546+0000","last_became_peered":"2024-05-08T05:42:19.495546+0000","last_unstale":"2024-05-08T05:42:19.497089+0000","last_undegraded":"2024-05-08T05:42:19.497089+0000","last_fullsized":"2024-05-08T05:42:19.497089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:26.053156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568299+0000","last_change":"2024-05-08T05:42:20.512851+0000","last_active":"2024-05-08T05:42:23.568299+0000","last_peered":"2024-05-08T05:42:23.568299+0000","last_clean":"2024-05-08T05:42:23.568299+0000","last_became_active":"2024-05-08T05:42:20.512412+0000","last_became_peered":"2024-05-08T05:42:20.512412+0000","last_unstale":"2024-05-08T05:42:23.568299+0000","last_undegraded":"2024-05-08T05:42:23.568299+0000","last_fullsized":"2024-05-08T05:42:23.568299+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:23.447125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569070+0000","last_change":"2024-05-08T05:42:19.499539+0000","last_active":"2024-05-08T05:42:23.569070+0000","last_peered":"2024-05-08T05:42:23.569070+0000","last_clean":"2024-05-08T05:42:23.569070+0000","last_became_active":"2024-05-08T05:42:19.499241+0000","last_became_peered":"2024-05-08T05:42:19.499241+0000","last_unstale":"2024-05-08T05:42:23.569070+0000","last_undegraded":"2024-05-08T05:42:23.569070+0000","last_fullsized":"2024-05-08T05:42:23.569070+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:22.668414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569402+0000","last_change":"2024-05-08T05:42:20.533418+0000","last_active":"2024-05-08T05:42:23.569402+0000","last_peered":"2024-05-08T05:42:23.569402+0000","last_clean":"2024-05-08T05:42:23.569402+0000","last_became_active":"2024-05-08T05:42:20.532766+0000","last_became_peered":"2024-05-08T05:42:20.532766+0000","last_unstale":"2024-05-08T05:42:23.569402+0000","last_undegraded":"2024-05-08T05:42:23.569402+0000","last_fullsized":"2024-05-08T05:42:23.569402+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:44.462590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.567873+0000","last_change":"2024-05-08T05:42:19.491786+0000","last_active":"2024-05-08T05:42:23.567873+0000","last_peered":"2024-05-08T05:42:23.567873+0000","last_clean":"2024-05-08T05:42:23.567873+0000","last_became_active":"2024-05-08T05:42:19.491455+0000","last_became_peered":"2024-05-08T05:42:19.491455+0000","last_unstale":"2024-05-08T05:42:23.567873+0000","last_undegraded":"2024-05-08T05:42:23.567873+0000","last_fullsized":"2024-05-08T05:42:23.567873+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:51:24.291839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517387+0000","last_change":"2024-05-08T05:42:20.529431+0000","last_active":"2024-05-08T05:42:21.517387+0000","last_peered":"2024-05-08T05:42:21.517387+0000","last_clean":"2024-05-08T05:42:21.517387+0000","last_became_active":"2024-05-08T05:42:20.529130+0000","last_became_peered":"2024-05-08T05:42:20.529130+0000","last_unstale":"2024-05-08T05:42:21.517387+0000","last_undegraded":"2024-05-08T05:42:21.517387+0000","last_fullsized":"2024-05-08T05:42:21.517387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:51:25.293597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569722+0000","last_change":"2024-05-08T05:42:19.487759+0000","last_active":"2024-05-08T05:42:23.569722+0000","last_peered":"2024-05-08T05:42:23.569722+0000","last_clean":"2024-05-08T05:42:23.569722+0000","last_became_active":"2024-05-08T05:42:19.487492+0000","last_became_peered":"2024-05-08T05:42:19.487492+0000","last_unstale":"2024-05-08T05:42:23.569722+0000","last_undegraded":"2024-05-08T05:42:23.569722+0000","last_fullsized":"2024-05-08T05:42:23.569722+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:22.445367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.481950+0000","last_change":"2024-05-08T05:42:19.481950+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.481950+0000","last_undegraded":"2024-05-08T05:42:19.481950+0000","last_fullsized":"2024-05-08T05:42:19.481950+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.501312+0000","last_change":"2024-05-08T05:42:19.498339+0000","last_active":"2024-05-08T05:42:19.501312+0000","last_peered":"2024-05-08T05:42:19.501312+0000","last_clean":"2024-05-08T05:42:19.501312+0000","last_became_active":"2024-05-08T05:42:19.498000+0000","last_became_peered":"2024-05-08T05:42:19.498000+0000","last_unstale":"2024-05-08T05:42:19.501312+0000","last_undegraded":"2024-05-08T05:42:19.501312+0000","last_fullsized":"2024-05-08T05:42:19.501312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:22.668414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569759+0000","last_change":"2024-05-08T05:42:20.525378+0000","last_active":"2024-05-08T05:42:23.569759+0000","last_peered":"2024-05-08T05:42:23.569759+0000","last_clean":"2024-05-08T05:42:23.569759+0000","last_became_active":"2024-05-08T05:42:20.525089+0000","last_became_peered":"2024-05-08T05:42:20.525089+0000","last_unstale":"2024-05-08T05:42:23.569759+0000","last_undegraded":"2024-05-08T05:42:23.569759+0000","last_fullsized":"2024-05-08T05:42:23.569759+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:32:33.781763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.501275+0000","last_change":"2024-05-08T05:42:19.499267+0000","last_active":"2024-05-08T05:42:19.501275+0000","last_peered":"2024-05-08T05:42:19.501275+0000","last_clean":"2024-05-08T05:42:19.501275+0000","last_became_active":"2024-05-08T05:42:19.498537+0000","last_became_peered":"2024-05-08T05:42:19.498537+0000","last_unstale":"2024-05-08T05:42:19.501275+0000","last_undegraded":"2024-05-08T05:42:19.501275+0000","last_fullsized":"2024-05-08T05:42:19.501275+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:53:50.777397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568542+0000","last_change":"2024-05-08T05:42:20.525088+0000","last_active":"2024-05-08T05:42:23.568542+0000","last_peered":"2024-05-08T05:42:23.568542+0000","last_clean":"2024-05-08T05:42:23.568542+0000","last_became_active":"2024-05-08T05:42:20.524526+0000","last_became_peered":"2024-05-08T05:42:20.524526+0000","last_unstale":"2024-05-08T05:42:23.568542+0000","last_undegraded":"2024-05-08T05:42:23.568542+0000","last_fullsized":"2024-05-08T05:42:23.568542+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:06.429659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569781+0000","last_change":"2024-05-08T05:42:19.488838+0000","last_active":"2024-05-08T05:42:23.569781+0000","last_peered":"2024-05-08T05:42:23.569781+0000","last_clean":"2024-05-08T05:42:23.569781+0000","last_became_active":"2024-05-08T05:42:19.488090+0000","last_became_peered":"2024-05-08T05:42:19.488090+0000","last_unstale":"2024-05-08T05:42:23.569781+0000","last_undegraded":"2024-05-08T05:42:23.569781+0000","last_fullsized":"2024-05-08T05:42:23.569781+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:17.430149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568617+0000","last_change":"2024-05-08T05:42:20.519358+0000","last_active":"2024-05-08T05:42:23.568617+0000","last_peered":"2024-05-08T05:42:23.568617+0000","last_clean":"2024-05-08T05:42:23.568617+0000","last_became_active":"2024-05-08T05:42:20.519067+0000","last_became_peered":"2024-05-08T05:42:20.519067+0000","last_unstale":"2024-05-08T05:42:23.568617+0000","last_undegraded":"2024-05-08T05:42:23.568617+0000","last_fullsized":"2024-05-08T05:42:23.568617+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:44.235500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569519+0000","last_change":"2024-05-08T05:42:19.486715+0000","last_active":"2024-05-08T05:42:23.569519+0000","last_peered":"2024-05-08T05:42:23.569519+0000","last_clean":"2024-05-08T05:42:23.569519+0000","last_became_active":"2024-05-08T05:42:19.486403+0000","last_became_peered":"2024-05-08T05:42:19.486403+0000","last_unstale":"2024-05-08T05:42:23.569519+0000","last_undegraded":"2024-05-08T05:42:23.569519+0000","last_fullsized":"2024-05-08T05:42:23.569519+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:43.233742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568767+0000","last_change":"2024-05-08T05:42:20.518993+0000","last_active":"2024-05-08T05:42:23.568767+0000","last_peered":"2024-05-08T05:42:23.568767+0000","last_clean":"2024-05-08T05:42:23.568767+0000","last_became_active":"2024-05-08T05:42:20.518710+0000","last_became_peered":"2024-05-08T05:42:20.518710+0000","last_unstale":"2024-05-08T05:42:23.568767+0000","last_undegraded":"2024-05-08T05:42:23.568767+0000","last_fullsized":"2024-05-08T05:42:23.568767+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:30:06.045575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569189+0000","last_change":"2024-05-08T05:42:19.500123+0000","last_active":"2024-05-08T05:42:23.569189+0000","last_peered":"2024-05-08T05:42:23.569189+0000","last_clean":"2024-05-08T05:42:23.569189+0000","last_became_active":"2024-05-08T05:42:19.499811+0000","last_became_peered":"2024-05-08T05:42:19.499811+0000","last_unstale":"2024-05-08T05:42:23.569189+0000","last_undegraded":"2024-05-08T05:42:23.569189+0000","last_fullsized":"2024-05-08T05:42:23.569189+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:26.053156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.480740+0000","last_change":"2024-05-08T05:42:19.480740+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.480740+0000","last_undegraded":"2024-05-08T05:42:19.480740+0000","last_fullsized":"2024-05-08T05:42:19.480740+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.498152+0000","last_change":"2024-05-08T05:42:19.496693+0000","last_active":"2024-05-08T05:42:19.498152+0000","last_peered":"2024-05-08T05:42:19.498152+0000","last_clean":"2024-05-08T05:42:19.498152+0000","last_became_active":"2024-05-08T05:42:19.496276+0000","last_became_peered":"2024-05-08T05:42:19.496276+0000","last_unstale":"2024-05-08T05:42:19.498152+0000","last_undegraded":"2024-05-08T05:42:19.498152+0000","last_fullsized":"2024-05-08T05:42:19.498152+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:50:26.559921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569458+0000","last_change":"2024-05-08T05:42:20.521385+0000","last_active":"2024-05-08T05:42:23.569458+0000","last_peered":"2024-05-08T05:42:23.569458+0000","last_clean":"2024-05-08T05:42:23.569458+0000","last_became_active":"2024-05-08T05:42:20.521089+0000","last_became_peered":"2024-05-08T05:42:20.521089+0000","last_unstale":"2024-05-08T05:42:23.569458+0000","last_undegraded":"2024-05-08T05:42:23.569458+0000","last_fullsized":"2024-05-08T05:42:23.569458+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:01.706249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.495293+0000","last_change":"2024-05-08T05:42:19.493145+0000","last_active":"2024-05-08T05:42:19.495293+0000","last_peered":"2024-05-08T05:42:19.495293+0000","last_clean":"2024-05-08T05:42:19.495293+0000","last_became_active":"2024-05-08T05:42:19.492821+0000","last_became_peered":"2024-05-08T05:42:19.492821+0000","last_unstale":"2024-05-08T05:42:19.495293+0000","last_undegraded":"2024-05-08T05:42:19.495293+0000","last_fullsized":"2024-05-08T05:42:19.495293+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:20:51.731816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569723+0000","last_change":"2024-05-08T05:42:20.529928+0000","last_active":"2024-05-08T05:42:23.569723+0000","last_peered":"2024-05-08T05:42:23.569723+0000","last_clean":"2024-05-08T05:42:23.569723+0000","last_became_active":"2024-05-08T05:42:20.529618+0000","last_became_peered":"2024-05-08T05:42:20.529618+0000","last_unstale":"2024-05-08T05:42:23.569723+0000","last_undegraded":"2024-05-08T05:42:23.569723+0000","last_fullsized":"2024-05-08T05:42:23.569723+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:33:26.883953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.501664+0000","last_change":"2024-05-08T05:42:19.499523+0000","last_active":"2024-05-08T05:42:19.501664+0000","last_peered":"2024-05-08T05:42:19.501664+0000","last_clean":"2024-05-08T05:42:19.501664+0000","last_became_active":"2024-05-08T05:42:19.499064+0000","last_became_peered":"2024-05-08T05:42:19.499064+0000","last_unstale":"2024-05-08T05:42:19.501664+0000","last_undegraded":"2024-05-08T05:42:19.501664+0000","last_fullsized":"2024-05-08T05:42:19.501664+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:28:34.310847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.488517+0000","last_change":"2024-05-08T05:42:19.488517+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.488517+0000","last_undegraded":"2024-05-08T05:42:19.488517+0000","last_fullsized":"2024-05-08T05:42:19.488517+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568515+0000","last_change":"2024-05-08T05:42:19.500389+0000","last_active":"2024-05-08T05:42:23.568515+0000","last_peered":"2024-05-08T05:42:23.568515+0000","last_clean":"2024-05-08T05:42:23.568515+0000","last_became_active":"2024-05-08T05:42:19.500111+0000","last_became_peered":"2024-05-08T05:42:19.500111+0000","last_unstale":"2024-05-08T05:42:23.568515+0000","last_undegraded":"2024-05-08T05:42:23.568515+0000","last_fullsized":"2024-05-08T05:42:23.568515+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:14.130507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568430+0000","last_change":"2024-05-08T05:42:20.522305+0000","last_active":"2024-05-08T05:42:23.568430+0000","last_peered":"2024-05-08T05:42:23.568430+0000","last_clean":"2024-05-08T05:42:23.568430+0000","last_became_active":"2024-05-08T05:42:20.521820+0000","last_became_peered":"2024-05-08T05:42:20.521820+0000","last_unstale":"2024-05-08T05:42:23.568430+0000","last_undegraded":"2024-05-08T05:42:23.568430+0000","last_fullsized":"2024-05-08T05:42:23.568430+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:44:05.274392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517447+0000","last_change":"2024-05-08T05:42:19.491582+0000","last_active":"2024-05-08T05:42:21.517447+0000","last_peered":"2024-05-08T05:42:21.517447+0000","last_clean":"2024-05-08T05:42:21.517447+0000","last_became_active":"2024-05-08T05:42:19.491291+0000","last_became_peered":"2024-05-08T05:42:19.491291+0000","last_unstale":"2024-05-08T05:42:21.517447+0000","last_undegraded":"2024-05-08T05:42:21.517447+0000","last_fullsized":"2024-05-08T05:42:21.517447+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:43:53.904889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517451+0000","last_change":"2024-05-08T05:42:20.527117+0000","last_active":"2024-05-08T05:42:21.517451+0000","last_peered":"2024-05-08T05:42:21.517451+0000","last_clean":"2024-05-08T05:42:21.517451+0000","last_became_active":"2024-05-08T05:42:20.526717+0000","last_became_peered":"2024-05-08T05:42:20.526717+0000","last_unstale":"2024-05-08T05:42:21.517451+0000","last_undegraded":"2024-05-08T05:42:21.517451+0000","last_fullsized":"2024-05-08T05:42:21.517451+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:26.076899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568839+0000","last_change":"2024-05-08T05:42:19.491505+0000","last_active":"2024-05-08T05:42:23.568839+0000","last_peered":"2024-05-08T05:42:23.568839+0000","last_clean":"2024-05-08T05:42:23.568839+0000","last_became_active":"2024-05-08T05:42:19.491206+0000","last_became_peered":"2024-05-08T05:42:19.491206+0000","last_unstale":"2024-05-08T05:42:23.568839+0000","last_undegraded":"2024-05-08T05:42:23.568839+0000","last_fullsized":"2024-05-08T05:42:23.568839+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:53:50.777397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.480258+0000","last_change":"2024-05-08T05:42:19.480258+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.480258+0000","last_undegraded":"2024-05-08T05:42:19.480258+0000","last_fullsized":"2024-05-08T05:42:19.480258+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517236+0000","last_change":"2024-05-08T05:42:19.494577+0000","last_active":"2024-05-08T05:42:21.517236+0000","last_peered":"2024-05-08T05:42:21.517236+0000","last_clean":"2024-05-08T05:42:21.517236+0000","last_became_active":"2024-05-08T05:42:19.494261+0000","last_became_peered":"2024-05-08T05:42:19.494261+0000","last_unstale":"2024-05-08T05:42:21.517236+0000","last_undegraded":"2024-05-08T05:42:21.517236+0000","last_fullsized":"2024-05-08T05:42:21.517236+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:07:51.458867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568433+0000","last_change":"2024-05-08T05:42:20.524296+0000","last_active":"2024-05-08T05:42:23.568433+0000","last_peered":"2024-05-08T05:42:23.568433+0000","last_clean":"2024-05-08T05:42:23.568433+0000","last_became_active":"2024-05-08T05:42:20.523971+0000","last_became_peered":"2024-05-08T05:42:20.523971+0000","last_unstale":"2024-05-08T05:42:23.568433+0000","last_undegraded":"2024-05-08T05:42:23.568433+0000","last_fullsized":"2024-05-08T05:42:23.568433+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:23.447125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517421+0000","last_change":"2024-05-08T05:42:19.495869+0000","last_active":"2024-05-08T05:42:21.517421+0000","last_peered":"2024-05-08T05:42:21.517421+0000","last_clean":"2024-05-08T05:42:21.517421+0000","last_became_active":"2024-05-08T05:42:19.495494+0000","last_became_peered":"2024-05-08T05:42:19.495494+0000","last_unstale":"2024-05-08T05:42:21.517421+0000","last_undegraded":"2024-05-08T05:42:21.517421+0000","last_fullsized":"2024-05-08T05:42:21.517421+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:08.085241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.490421+0000","last_change":"2024-05-08T05:42:19.490421+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.490421+0000","last_undegraded":"2024-05-08T05:42:19.490421+0000","last_fullsized":"2024-05-08T05:42:19.490421+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.495945+0000","last_change":"2024-05-08T05:42:19.494546+0000","last_active":"2024-05-08T05:42:19.495945+0000","last_peered":"2024-05-08T05:42:19.495945+0000","last_clean":"2024-05-08T05:42:19.495945+0000","last_became_active":"2024-05-08T05:42:19.494232+0000","last_became_peered":"2024-05-08T05:42:19.494232+0000","last_unstale":"2024-05-08T05:42:19.495945+0000","last_undegraded":"2024-05-08T05:42:19.495945+0000","last_fullsized":"2024-05-08T05:42:19.495945+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:22.668414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569563+0000","last_change":"2024-05-08T05:42:20.533065+0000","last_active":"2024-05-08T05:42:23.569563+0000","last_peered":"2024-05-08T05:42:23.569563+0000","last_clean":"2024-05-08T05:42:23.569563+0000","last_became_active":"2024-05-08T05:42:20.532437+0000","last_became_peered":"2024-05-08T05:42:20.532437+0000","last_unstale":"2024-05-08T05:42:23.569563+0000","last_undegraded":"2024-05-08T05:42:23.569563+0000","last_fullsized":"2024-05-08T05:42:23.569563+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:50.284596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517058+0000","last_change":"2024-05-08T05:42:19.494499+0000","last_active":"2024-05-08T05:42:21.517058+0000","last_peered":"2024-05-08T05:42:21.517058+0000","last_clean":"2024-05-08T05:42:21.517058+0000","last_became_active":"2024-05-08T05:42:19.494193+0000","last_became_peered":"2024-05-08T05:42:19.494193+0000","last_unstale":"2024-05-08T05:42:21.517058+0000","last_undegraded":"2024-05-08T05:42:21.517058+0000","last_fullsized":"2024-05-08T05:42:21.517058+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:34.880276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568778+0000","last_change":"2024-05-08T05:42:20.518085+0000","last_active":"2024-05-08T05:42:23.568778+0000","last_peered":"2024-05-08T05:42:23.568778+0000","last_clean":"2024-05-08T05:42:23.568778+0000","last_became_active":"2024-05-08T05:42:20.517771+0000","last_became_peered":"2024-05-08T05:42:20.517771+0000","last_unstale":"2024-05-08T05:42:23.568778+0000","last_undegraded":"2024-05-08T05:42:23.568778+0000","last_fullsized":"2024-05-08T05:42:23.568778+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:18.431907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.501230+0000","last_change":"2024-05-08T05:42:19.498985+0000","last_active":"2024-05-08T05:42:19.501230+0000","last_peered":"2024-05-08T05:42:19.501230+0000","last_clean":"2024-05-08T05:42:19.501230+0000","last_became_active":"2024-05-08T05:42:19.498663+0000","last_became_peered":"2024-05-08T05:42:19.498663+0000","last_unstale":"2024-05-08T05:42:19.501230+0000","last_undegraded":"2024-05-08T05:42:19.501230+0000","last_fullsized":"2024-05-08T05:42:19.501230+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:26.053156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.481486+0000","last_change":"2024-05-08T05:42:19.481486+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.481486+0000","last_undegraded":"2024-05-08T05:42:19.481486+0000","last_fullsized":"2024-05-08T05:42:19.481486+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"36'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517320+0000","last_change":"2024-05-08T05:42:19.499917+0000","last_active":"2024-05-08T05:42:21.517320+0000","last_peered":"2024-05-08T05:42:21.517320+0000","last_clean":"2024-05-08T05:42:21.517320+0000","last_became_active":"2024-05-08T05:42:19.499614+0000","last_became_peered":"2024-05-08T05:42:19.499614+0000","last_unstale":"2024-05-08T05:42:21.517320+0000","last_undegraded":"2024-05-08T05:42:21.517320+0000","last_fullsized":"2024-05-08T05:42:21.517320+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:45.821917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569509+0000","last_change":"2024-05-08T05:42:20.529586+0000","last_active":"2024-05-08T05:42:23.569509+0000","last_peered":"2024-05-08T05:42:23.569509+0000","last_clean":"2024-05-08T05:42:23.569509+0000","last_became_active":"2024-05-08T05:42:20.529176+0000","last_became_peered":"2024-05-08T05:42:20.529176+0000","last_unstale":"2024-05-08T05:42:23.569509+0000","last_undegraded":"2024-05-08T05:42:23.569509+0000","last_fullsized":"2024-05-08T05:42:23.569509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:53.621950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569217+0000","last_change":"2024-05-08T05:42:19.492651+0000","last_active":"2024-05-08T05:42:23.569217+0000","last_peered":"2024-05-08T05:42:23.569217+0000","last_clean":"2024-05-08T05:42:23.569217+0000","last_became_active":"2024-05-08T05:42:19.491268+0000","last_became_peered":"2024-05-08T05:42:19.491268+0000","last_unstale":"2024-05-08T05:42:23.569217+0000","last_undegraded":"2024-05-08T05:42:23.569217+0000","last_fullsized":"2024-05-08T05:42:23.569217+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:29.265240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568690+0000","last_change":"2024-05-08T05:42:20.525312+0000","last_active":"2024-05-08T05:42:23.568690+0000","last_peered":"2024-05-08T05:42:23.568690+0000","last_clean":"2024-05-08T05:42:23.568690+0000","last_became_active":"2024-05-08T05:42:20.524640+0000","last_became_peered":"2024-05-08T05:42:20.524640+0000","last_unstale":"2024-05-08T05:42:23.568690+0000","last_undegraded":"2024-05-08T05:42:23.568690+0000","last_fullsized":"2024-05-08T05:42:23.568690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:18.431907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569450+0000","last_change":"2024-05-08T05:42:19.494149+0000","last_active":"2024-05-08T05:42:23.569450+0000","last_peered":"2024-05-08T05:42:23.569450+0000","last_clean":"2024-05-08T05:42:23.569450+0000","last_became_active":"2024-05-08T05:42:19.493848+0000","last_became_peered":"2024-05-08T05:42:19.493848+0000","last_unstale":"2024-05-08T05:42:23.569450+0000","last_undegraded":"2024-05-08T05:42:23.569450+0000","last_fullsized":"2024-05-08T05:42:23.569450+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:25.075141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569011+0000","last_change":"2024-05-08T05:42:20.518816+0000","last_active":"2024-05-08T05:42:23.569011+0000","last_peered":"2024-05-08T05:42:23.569011+0000","last_clean":"2024-05-08T05:42:23.569011+0000","last_became_active":"2024-05-08T05:42:20.518514+0000","last_became_peered":"2024-05-08T05:42:20.518514+0000","last_unstale":"2024-05-08T05:42:23.569011+0000","last_undegraded":"2024-05-08T05:42:23.569011+0000","last_fullsized":"2024-05-08T05:42:23.569011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:06.429659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.499479+0000","last_change":"2024-05-08T05:42:19.497480+0000","last_active":"2024-05-08T05:42:19.499479+0000","last_peered":"2024-05-08T05:42:19.499479+0000","last_clean":"2024-05-08T05:42:19.499479+0000","last_became_active":"2024-05-08T05:42:19.497177+0000","last_became_peered":"2024-05-08T05:42:19.497177+0000","last_unstale":"2024-05-08T05:42:19.499479+0000","last_undegraded":"2024-05-08T05:42:19.499479+0000","last_fullsized":"2024-05-08T05:42:19.499479+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:58:57.734024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568064+0000","last_change":"2024-05-08T05:42:20.530479+0000","last_active":"2024-05-08T05:42:23.568064+0000","last_peered":"2024-05-08T05:42:23.568064+0000","last_clean":"2024-05-08T05:42:23.568064+0000","last_became_active":"2024-05-08T05:42:20.530210+0000","last_became_peered":"2024-05-08T05:42:20.530210+0000","last_unstale":"2024-05-08T05:42:23.568064+0000","last_undegraded":"2024-05-08T05:42:23.568064+0000","last_fullsized":"2024-05-08T05:42:23.568064+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:58.041468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.567873+0000","last_change":"2024-05-08T05:42:19.495013+0000","last_active":"2024-05-08T05:42:23.567873+0000","last_peered":"2024-05-08T05:42:23.567873+0000","last_clean":"2024-05-08T05:42:23.567873+0000","last_became_active":"2024-05-08T05:42:19.494694+0000","last_became_peered":"2024-05-08T05:42:19.494694+0000","last_unstale":"2024-05-08T05:42:23.567873+0000","last_undegraded":"2024-05-08T05:42:23.567873+0000","last_fullsized":"2024-05-08T05:42:23.567873+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:29.265240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.483121+0000","last_change":"2024-05-08T05:42:19.483121+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.483121+0000","last_undegraded":"2024-05-08T05:42:19.483121+0000","last_fullsized":"2024-05-08T05:42:19.483121+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568473+0000","last_change":"2024-05-08T05:42:19.487951+0000","last_active":"2024-05-08T05:42:23.568473+0000","last_peered":"2024-05-08T05:42:23.568473+0000","last_clean":"2024-05-08T05:42:23.568473+0000","last_became_active":"2024-05-08T05:42:19.487613+0000","last_became_peered":"2024-05-08T05:42:19.487613+0000","last_unstale":"2024-05-08T05:42:23.568473+0000","last_undegraded":"2024-05-08T05:42:23.568473+0000","last_fullsized":"2024-05-08T05:42:23.568473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:57.039710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.489194+0000","last_change":"2024-05-08T05:42:19.489194+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.489194+0000","last_undegraded":"2024-05-08T05:42:19.489194+0000","last_fullsized":"2024-05-08T05:42:19.489194+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.489865+0000","last_change":"2024-05-08T05:42:19.487846+0000","last_active":"2024-05-08T05:42:19.489865+0000","last_peered":"2024-05-08T05:42:19.489865+0000","last_clean":"2024-05-08T05:42:19.489865+0000","last_became_active":"2024-05-08T05:42:19.487570+0000","last_became_peered":"2024-05-08T05:42:19.487570+0000","last_unstale":"2024-05-08T05:42:19.489865+0000","last_undegraded":"2024-05-08T05:42:19.489865+0000","last_fullsized":"2024-05-08T05:42:19.489865+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:25.075141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":135,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.483776+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:42:19.483776+0000","last_peered":"2024-05-08T05:42:19.483776+0000","last_clean":"2024-05-08T05:42:19.483776+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:42:19.483776+0000","last_undegraded":"2024-05-08T05:42:19.483776+0000","last_fullsized":"2024-05-08T05:42:19.483776+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1716,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":966656,"data_stored":744313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":372736,"data_stored":153945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542160,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":160612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":175075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":425984,"data_stored":176117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":413696,"data_stored":174541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":167869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":995328,"data_stored":758181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:42:25.295 INFO:tasks.ceph.ceph_manager.ceph:PG 6.39 is not active+clean 2024-05-08T05:42:25.295 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.494063+0000', 'last_change': '2024-05-08T05:42:19.494063+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.494063+0000', 'last_undegraded': '2024-05-08T05:42:19.494063+0000', 'last_fullsized': '2024-05-08T05:42:19.494063+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:42:25.296 INFO:tasks.ceph.ceph_manager.ceph:PG 6.36 is not active+clean 2024-05-08T05:42:25.296 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.492151+0000', 'last_change': '2024-05-08T05:42:19.492151+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.492151+0000', 'last_undegraded': '2024-05-08T05:42:19.492151+0000', 'last_fullsized': '2024-05-08T05:42:19.492151+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:42:25.296 INFO:tasks.ceph.ceph_manager.ceph:PG 6.35 is not active+clean 2024-05-08T05:42:25.296 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.489300+0000', 'last_change': '2024-05-08T05:42:19.489300+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.489300+0000', 'last_undegraded': '2024-05-08T05:42:19.489300+0000', 'last_fullsized': '2024-05-08T05:42:19.489300+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:42:25.296 INFO:tasks.ceph.ceph_manager.ceph:PG 6.33 is not active+clean 2024-05-08T05:42:25.296 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.488571+0000', 'last_change': '2024-05-08T05:42:19.488571+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.488571+0000', 'last_undegraded': '2024-05-08T05:42:19.488571+0000', 'last_fullsized': '2024-05-08T05:42:19.488571+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:42:25.296 INFO:tasks.ceph.ceph_manager.ceph:PG 6.31 is not active+clean 2024-05-08T05:42:25.296 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.491652+0000', 'last_change': '2024-05-08T05:42:19.491652+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.491652+0000', 'last_undegraded': '2024-05-08T05:42:19.491652+0000', 'last_fullsized': '2024-05-08T05:42:19.491652+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:42:25.296 INFO:tasks.ceph.ceph_manager.ceph:PG 6.30 is not active+clean 2024-05-08T05:42:25.296 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.483214+0000', 'last_change': '2024-05-08T05:42:19.483214+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.483214+0000', 'last_undegraded': '2024-05-08T05:42:19.483214+0000', 'last_fullsized': '2024-05-08T05:42:19.483214+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T05:42:25.297 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2f is not active+clean 2024-05-08T05:42:25.297 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.482113+0000', 'last_change': '2024-05-08T05:42:19.482113+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.482113+0000', 'last_undegraded': '2024-05-08T05:42:19.482113+0000', 'last_fullsized': '2024-05-08T05:42:19.482113+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T05:42:25.297 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2e is not active+clean 2024-05-08T05:42:25.297 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.486278+0000', 'last_change': '2024-05-08T05:42:19.486278+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.486278+0000', 'last_undegraded': '2024-05-08T05:42:19.486278+0000', 'last_fullsized': '2024-05-08T05:42:19.486278+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T05:42:25.297 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2c is not active+clean 2024-05-08T05:42:25.297 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.489219+0000', 'last_change': '2024-05-08T05:42:19.489219+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.489219+0000', 'last_undegraded': '2024-05-08T05:42:19.489219+0000', 'last_fullsized': '2024-05-08T05:42:19.489219+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T05:42:25.297 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2b is not active+clean 2024-05-08T05:42:25.297 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.489042+0000', 'last_change': '2024-05-08T05:42:19.489042+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.489042+0000', 'last_undegraded': '2024-05-08T05:42:19.489042+0000', 'last_fullsized': '2024-05-08T05:42:19.489042+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:42:25.297 INFO:tasks.ceph.ceph_manager.ceph:PG 6.29 is not active+clean 2024-05-08T05:42:25.297 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.481502+0000', 'last_change': '2024-05-08T05:42:19.481502+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.481502+0000', 'last_undegraded': '2024-05-08T05:42:19.481502+0000', 'last_fullsized': '2024-05-08T05:42:19.481502+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T05:42:25.297 INFO:tasks.ceph.ceph_manager.ceph:PG 6.27 is not active+clean 2024-05-08T05:42:25.298 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.485694+0000', 'last_change': '2024-05-08T05:42:19.485694+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.485694+0000', 'last_undegraded': '2024-05-08T05:42:19.485694+0000', 'last_fullsized': '2024-05-08T05:42:19.485694+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T05:42:25.298 INFO:tasks.ceph.ceph_manager.ceph:PG 6.25 is not active+clean 2024-05-08T05:42:25.298 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.486709+0000', 'last_change': '2024-05-08T05:42:19.486709+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.486709+0000', 'last_undegraded': '2024-05-08T05:42:19.486709+0000', 'last_fullsized': '2024-05-08T05:42:19.486709+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T05:42:25.298 INFO:tasks.ceph.ceph_manager.ceph:PG 6.24 is not active+clean 2024-05-08T05:42:25.298 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.488095+0000', 'last_change': '2024-05-08T05:42:19.488095+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.488095+0000', 'last_undegraded': '2024-05-08T05:42:19.488095+0000', 'last_fullsized': '2024-05-08T05:42:19.488095+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:42:25.298 INFO:tasks.ceph.ceph_manager.ceph:PG 6.23 is not active+clean 2024-05-08T05:42:25.298 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.488425+0000', 'last_change': '2024-05-08T05:42:19.488425+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.488425+0000', 'last_undegraded': '2024-05-08T05:42:19.488425+0000', 'last_fullsized': '2024-05-08T05:42:19.488425+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T05:42:25.298 INFO:tasks.ceph.ceph_manager.ceph:PG 6.22 is not active+clean 2024-05-08T05:42:25.298 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.485913+0000', 'last_change': '2024-05-08T05:42:19.485913+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.485913+0000', 'last_undegraded': '2024-05-08T05:42:19.485913+0000', 'last_fullsized': '2024-05-08T05:42:19.485913+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T05:42:25.298 INFO:tasks.ceph.ceph_manager.ceph:PG 6.21 is not active+clean 2024-05-08T05:42:25.298 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.481029+0000', 'last_change': '2024-05-08T05:42:19.481029+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.481029+0000', 'last_undegraded': '2024-05-08T05:42:19.481029+0000', 'last_fullsized': '2024-05-08T05:42:19.481029+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T05:42:25.299 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1c is not active+clean 2024-05-08T05:42:25.299 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.481092+0000', 'last_change': '2024-05-08T05:42:19.481092+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.481092+0000', 'last_undegraded': '2024-05-08T05:42:19.481092+0000', 'last_fullsized': '2024-05-08T05:42:19.481092+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T05:42:25.299 INFO:tasks.ceph.ceph_manager.ceph:PG 6.19 is not active+clean 2024-05-08T05:42:25.299 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.488852+0000', 'last_change': '2024-05-08T05:42:19.488852+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.488852+0000', 'last_undegraded': '2024-05-08T05:42:19.488852+0000', 'last_fullsized': '2024-05-08T05:42:19.488852+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:42:25.299 INFO:tasks.ceph.ceph_manager.ceph:PG 6.15 is not active+clean 2024-05-08T05:42:25.299 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.481950+0000', 'last_change': '2024-05-08T05:42:19.481950+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.481950+0000', 'last_undegraded': '2024-05-08T05:42:19.481950+0000', 'last_fullsized': '2024-05-08T05:42:19.481950+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T05:42:25.299 INFO:tasks.ceph.ceph_manager.ceph:PG 6.12 is not active+clean 2024-05-08T05:42:25.299 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.480740+0000', 'last_change': '2024-05-08T05:42:19.480740+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.480740+0000', 'last_undegraded': '2024-05-08T05:42:19.480740+0000', 'last_fullsized': '2024-05-08T05:42:19.480740+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T05:42:25.299 INFO:tasks.ceph.ceph_manager.ceph:PG 6.d is not active+clean 2024-05-08T05:42:25.299 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.488517+0000', 'last_change': '2024-05-08T05:42:19.488517+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.488517+0000', 'last_undegraded': '2024-05-08T05:42:19.488517+0000', 'last_fullsized': '2024-05-08T05:42:19.488517+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:42:25.300 INFO:tasks.ceph.ceph_manager.ceph:PG 6.8 is not active+clean 2024-05-08T05:42:25.300 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.480258+0000', 'last_change': '2024-05-08T05:42:19.480258+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.480258+0000', 'last_undegraded': '2024-05-08T05:42:19.480258+0000', 'last_fullsized': '2024-05-08T05:42:19.480258+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T05:42:25.300 INFO:tasks.ceph.ceph_manager.ceph:PG 6.a is not active+clean 2024-05-08T05:42:25.300 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.490421+0000', 'last_change': '2024-05-08T05:42:19.490421+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.490421+0000', 'last_undegraded': '2024-05-08T05:42:19.490421+0000', 'last_fullsized': '2024-05-08T05:42:19.490421+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:42:25.300 INFO:tasks.ceph.ceph_manager.ceph:PG 6.5 is not active+clean 2024-05-08T05:42:25.300 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.481486+0000', 'last_change': '2024-05-08T05:42:19.481486+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.481486+0000', 'last_undegraded': '2024-05-08T05:42:19.481486+0000', 'last_fullsized': '2024-05-08T05:42:19.481486+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T05:42:25.300 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3 is not active+clean 2024-05-08T05:42:25.300 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.483121+0000', 'last_change': '2024-05-08T05:42:19.483121+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.483121+0000', 'last_undegraded': '2024-05-08T05:42:19.483121+0000', 'last_fullsized': '2024-05-08T05:42:19.483121+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T05:42:25.300 INFO:tasks.ceph.ceph_manager.ceph:PG 6.7 is not active+clean 2024-05-08T05:42:25.300 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:19.489194+0000', 'last_change': '2024-05-08T05:42:19.489194+0000', 'last_active': '2024-05-08T05:42:19.471571+0000', 'last_peered': '2024-05-08T05:42:19.471571+0000', 'last_clean': '2024-05-08T05:42:19.471571+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:19.489194+0000', 'last_undegraded': '2024-05-08T05:42:19.489194+0000', 'last_fullsized': '2024-05-08T05:42:19.489194+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:19.471571+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:42:25.301 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T05:42:25.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.435+0000 7f80ab075640 1 Processor -- start 2024-05-08T05:42:25.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.435+0000 7f80ab075640 1 -- start start 2024-05-08T05:42:25.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.436+0000 7f80ab075640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a415e180 0x7f80a40ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.436+0000 7f80ab075640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80a415e610 con 0x7f80a40d39b0 2024-05-08T05:42:25.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.436+0000 7f80ab075640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80a4160d10 con 0x7f80a40cf8a0 2024-05-08T05:42:25.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.436+0000 7f80ab075640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f80a415d5f0 con 0x7f80a415e180 2024-05-08T05:42:25.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.436+0000 7f80a3fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f80a40d39b0 0x7f80a40cd9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56074/0 (socket says 172.21.15.45:56074) 2024-05-08T05:42:25.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.436+0000 7f80a3fff640 1 -- 172.21.15.45:0/779518976 learned_addr learned my addr 172.21.15.45:0/779518976 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:25.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.436+0000 7f80a9612640 1 --2- 172.21.15.45:0/779518976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a415e180 0x7f80a40ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.436+0000 7f80a37fe640 1 -- 172.21.15.45:0/779518976 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 86802752 0 0) 0x7f80a415e610 con 0x7f80a40d39b0 2024-05-08T05:42:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.436+0000 7f80a37fe640 1 -- 172.21.15.45:0/779518976 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8094003520 con 0x7f80a40d39b0 2024-05-08T05:42:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.436+0000 7f80a37fe640 1 -- 172.21.15.45:0/779518976 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 991760502 0 0) 0x7f8094003520 con 0x7f80a40d39b0 2024-05-08T05:42:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.436+0000 7f80a37fe640 1 -- 172.21.15.45:0/779518976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a415e180 msgr2=0x7f80a40ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.436+0000 7f80a37fe640 1 --2- 172.21.15.45:0/779518976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a415e180 0x7f80a40ce0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.436+0000 7f80a37fe640 1 -- 172.21.15.45:0/779518976 >> v1:172.21.15.63:6789/0 conn(0x7f80a40cf8a0 legacy=0x7f80a40cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.436+0000 7f80a37fe640 1 -- 172.21.15.45:0/779518976 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a4160ff0 con 0x7f80a40d39b0 2024-05-08T05:42:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.436+0000 7f80a37fe640 1 -- 172.21.15.45:0/779518976 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f80900016f0 con 0x7f80a40d39b0 2024-05-08T05:42:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.437+0000 7f80a37fe640 1 -- 172.21.15.45:0/779518976 >> v1:172.21.15.45:6789/0 conn(0x7f80a40d39b0 legacy=0x7f80a40cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.437+0000 7f80a37fe640 1 --2- 172.21.15.45:0/779518976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a415e180 0x7f8094003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.437+0000 7f80a37fe640 1 --2- 172.21.15.45:0/779518976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8094004000 0x7f8094004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.437+0000 7f80a37fe640 1 --2- 172.21.15.45:0/779518976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8094004d40 0x7f80940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.437+0000 7f80a37fe640 1 -- 172.21.15.45:0/779518976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f80a415d5f0 con 0x7f8094004d40 2024-05-08T05:42:25.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.437+0000 7f80a37fe640 1 -- 172.21.15.45:0/779518976 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f809400bb50 con 0x7f8094004000 2024-05-08T05:42:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.437+0000 7f80a37fe640 1 -- 172.21.15.45:0/779518976 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f809400bd50 con 0x7f80a415e180 2024-05-08T05:42:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.437+0000 7f80a9612640 1 --2- 172.21.15.45:0/779518976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8094004d40 0x7f80940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.437+0000 7f80a3fff640 1 --2- 172.21.15.45:0/779518976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8094004000 0x7f8094004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.437+0000 7f80a8e11640 1 --2- 172.21.15.45:0/779518976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a415e180 0x7f8094003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.437+0000 7f80a9612640 1 -- 172.21.15.45:0/779518976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a415e180 msgr2=0x7f8094003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.437+0000 7f80a9612640 1 --2- 172.21.15.45:0/779518976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a415e180 0x7f8094003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.437+0000 7f80a9612640 1 -- 172.21.15.45:0/779518976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8094004000 msgr2=0x7f8094004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.437+0000 7f80a9612640 1 --2- 172.21.15.45:0/779518976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8094004000 0x7f8094004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.437+0000 7f80a9612640 1 -- 172.21.15.45:0/779518976 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8094003520 con 0x7f8094004d40 2024-05-08T05:42:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.438+0000 7f80a9612640 1 --2- 172.21.15.45:0/779518976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8094004d40 0x7f80940051c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f809c009fc0 tx=0x7f8094005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73030737cbbd492 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.438+0000 7f80a37fe640 1 -- 172.21.15.45:0/779518976 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c009c10 con 0x7f8094004d40 2024-05-08T05:42:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.438+0000 7f80a37fe640 1 -- 172.21.15.45:0/779518976 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f809c036070 con 0x7f8094004d40 2024-05-08T05:42:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.438+0000 7f80a37fe640 1 -- 172.21.15.45:0/779518976 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c031880 con 0x7f8094004d40 2024-05-08T05:42:25.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.438+0000 7f80ab075640 1 -- 172.21.15.45:0/779518976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8094004d40 msgr2=0x7f80940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:25.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.438+0000 7f80ab075640 1 --2- 172.21.15.45:0/779518976 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8094004d40 0x7f80940051c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f809c009fc0 tx=0x7f8094005830 comp rx=0 tx=0).stop 2024-05-08T05:42:25.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.438+0000 7f80a9612640 1 -- 172.21.15.45:0/779518976 reap_dead start 2024-05-08T05:42:25.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.438+0000 7f80ab075640 1 -- 172.21.15.45:0/779518976 shutdown_connections 2024-05-08T05:42:25.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.438+0000 7f80ab075640 1 -- 172.21.15.45:0/779518976 >> 172.21.15.45:0/779518976 conn(0x7f80a40d2e90 msgr2=0x7f80a40d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:25.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.438+0000 7f80ab075640 1 -- 172.21.15.45:0/779518976 shutdown_connections 2024-05-08T05:42:25.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.438+0000 7f80ab075640 1 -- 172.21.15.45:0/779518976 wait complete. 2024-05-08T05:42:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.440+0000 7f80ab075640 1 Processor -- start 2024-05-08T05:42:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.440+0000 7f80ab075640 1 -- start start 2024-05-08T05:42:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80ab075640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8094004d40 0x7f80a41f4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80ab075640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80a406ed90 0x7f80a40b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80ab075640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a406f210 0x7f80a40b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80ab075640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f80a40d0700 con 0x7f80a406ed90 2024-05-08T05:42:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80ab075640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f80a41615b0 con 0x7f8094004d40 2024-05-08T05:42:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80ab075640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f80a40ce0b0 con 0x7f80a406f210 2024-05-08T05:42:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80a3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80a406ed90 0x7f80a40b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80a3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80a406ed90 0x7f80a40b0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44258/0 (socket says 172.21.15.45:44258) 2024-05-08T05:42:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80a3fff640 1 -- 172.21.15.45:0/1564195139 learned_addr learned my addr 172.21.15.45:0/1564195139 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:25.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80a8e11640 1 --2- 172.21.15.45:0/1564195139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8094004d40 0x7f80a41f4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80a9612640 1 --2- 172.21.15.45:0/1564195139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a406f210 0x7f80a40b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80a3fff640 1 -- 172.21.15.45:0/1564195139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a406f210 msgr2=0x7f80a40b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80a3fff640 1 --2- 172.21.15.45:0/1564195139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a406f210 0x7f80a40b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80a3fff640 1 -- 172.21.15.45:0/1564195139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8094004d40 msgr2=0x7f80a41f4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80a3fff640 1 --2- 172.21.15.45:0/1564195139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8094004d40 0x7f80a41f4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80a3fff640 1 -- 172.21.15.45:0/1564195139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a41f8dd0 con 0x7f80a406ed90 2024-05-08T05:42:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80a9612640 1 --2- 172.21.15.45:0/1564195139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a406f210 0x7f80a40b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80a8e11640 1 --2- 172.21.15.45:0/1564195139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8094004d40 0x7f80a41f4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80a3fff640 1 --2- 172.21.15.45:0/1564195139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80a406ed90 0x7f80a40b0680 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8090007310 tx=0x7f80900077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80a17fa640 1 -- 172.21.15.45:0/1564195139 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809001a070 con 0x7f80a406ed90 2024-05-08T05:42:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80a17fa640 1 -- 172.21.15.45:0/1564195139 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f809000f9c0 con 0x7f80a406ed90 2024-05-08T05:42:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80ab075640 1 -- 172.21.15.45:0/1564195139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a41f90b0 con 0x7f80a406ed90 2024-05-08T05:42:25.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80a17fa640 1 -- 172.21.15.45:0/1564195139 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809000fdb0 con 0x7f80a406ed90 2024-05-08T05:42:25.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.441+0000 7f80ab075640 1 -- 172.21.15.45:0/1564195139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a41f9670 con 0x7f80a406ed90 2024-05-08T05:42:25.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.442+0000 7f80ab075640 1 -- 172.21.15.45:0/1564195139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a40693f0 con 0x7f80a406ed90 2024-05-08T05:42:25.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.443+0000 7f80a17fa640 1 -- 172.21.15.45:0/1564195139 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f809001d070 con 0x7f80a406ed90 2024-05-08T05:42:25.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.444+0000 7f80a17fa640 1 --2- 172.21.15.45:0/1564195139 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f80880762d0 0x7f8088078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.444+0000 7f80a17fa640 1 -- 172.21.15.45:0/1564195139 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f8090097260 con 0x7f80a406ed90 2024-05-08T05:42:25.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.444+0000 7f80a8e11640 1 --2- 172.21.15.45:0/1564195139 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f80880762d0 0x7f8088078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.444+0000 7f80a8e11640 1 --2- 172.21.15.45:0/1564195139 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f80880762d0 0x7f8088078790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8098000c00 tx=0x7f8098009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:25.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.448+0000 7f80a17fa640 1 -- 172.21.15.45:0/1564195139 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8090060a30 con 0x7f80a406ed90 2024-05-08T05:42:25.463 INFO:tasks.ceph.mds.c.smithi045.stderr:2024-05-08T05:42:25.463+0000 7f37c7991640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:42:25.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.715+0000 7f80ab075640 1 -- 172.21.15.45:0/1564195139 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f80a40cf680 con 0x7f80a406ed90 2024-05-08T05:42:25.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.715+0000 7f80a17fa640 1 -- 172.21.15.45:0/1564195139 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1646 (secure 0 0 0) 0x7f8090060330 con 0x7f80a406ed90 2024-05-08T05:42:25.716 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:25.716 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":77,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":8,"num_up_osds":8,"osd_up_since":1715146878,"num_in_osds":8,"osd_in_since":1715146870,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":70},{"state_name":"creating+peering","count":27}],"num_pgs":97,"num_pools":3,"num_objects":14,"data_bytes":592084,"bytes_used":242778112,"bytes_avail":772851335168,"bytes_total":773094113280,"inactive_pgs_ratio":0.27835050225257874,"write_bytes_sec":1877,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":17,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":4884}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-05-08T05:42:20.844943+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T05:42:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.717+0000 7f80ab075640 1 -- 172.21.15.45:0/1564195139 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f80880762d0 msgr2=0x7f8088078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.717+0000 7f80ab075640 1 --2- 172.21.15.45:0/1564195139 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f80880762d0 0x7f8088078790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8098000c00 tx=0x7f8098009040 comp rx=0 tx=0).stop 2024-05-08T05:42:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.717+0000 7f80ab075640 1 -- 172.21.15.45:0/1564195139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80a406ed90 msgr2=0x7f80a40b0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.717+0000 7f80ab075640 1 --2- 172.21.15.45:0/1564195139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80a406ed90 0x7f80a40b0680 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8090007310 tx=0x7f80900077d0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.717+0000 7f80ab075640 1 -- 172.21.15.45:0/1564195139 shutdown_connections 2024-05-08T05:42:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.717+0000 7f80ab075640 1 --2- 172.21.15.45:0/1564195139 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f80880762d0 0x7f8088078790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.717+0000 7f80ab075640 1 --2- 172.21.15.45:0/1564195139 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80a406f210 0x7f80a40b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.717+0000 7f80ab075640 1 --2- 172.21.15.45:0/1564195139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80a406ed90 0x7f80a40b0680 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.717+0000 7f80ab075640 1 --2- 172.21.15.45:0/1564195139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8094004d40 0x7f80a41f4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.717+0000 7f80ab075640 1 -- 172.21.15.45:0/1564195139 >> 172.21.15.45:0/1564195139 conn(0x7f80a40d2e90 msgr2=0x7f80a415eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:25.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.717+0000 7f80ab075640 1 -- 172.21.15.45:0/1564195139 shutdown_connections 2024-05-08T05:42:25.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.717+0000 7f80ab075640 1 -- 172.21.15.45:0/1564195139 wait complete. 2024-05-08T05:42:25.729 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-08T05:42:25.730 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:42:25.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.834+0000 7f69a38da640 1 Processor -- start 2024-05-08T05:42:25.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.834+0000 7f69a38da640 1 -- start start 2024-05-08T05:42:25.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.835+0000 7f69a38da640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f699c1638e0 0x7f699c0b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.835+0000 7f69a38da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f699c06c3a0 0x7f699c06c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.835+0000 7f69a38da640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f699c0b14e0 con 0x7f699c15f0c0 2024-05-08T05:42:25.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.835+0000 7f69a38da640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f699c163450 con 0x7f699c1638e0 2024-05-08T05:42:25.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.835+0000 7f69a38da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f699c1604b0 con 0x7f699c06c3a0 2024-05-08T05:42:25.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.835+0000 7f69a0e75640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f699c1638e0 0x7f699c0b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.835+0000 7f69a0e75640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f699c1638e0 0x7f699c0b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44262/0 (socket says 172.21.15.45:44262) 2024-05-08T05:42:25.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.835+0000 7f69a0e75640 1 -- 172.21.15.45:0/1360243609 learned_addr learned my addr 172.21.15.45:0/1360243609 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:25.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.835+0000 7f69a1e77640 1 --2- 172.21.15.45:0/1360243609 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f699c06c3a0 0x7f699c06c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.835+0000 7f69a0e75640 1 -- 172.21.15.45:0/1360243609 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f699c06c3a0 msgr2=0x7f699c06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:25.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.835+0000 7f69a0e75640 1 --2- 172.21.15.45:0/1360243609 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f699c06c3a0 0x7f699c06c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.835+0000 7f69a0e75640 1 -- 172.21.15.45:0/1360243609 >> v1:172.21.15.63:6789/0 conn(0x7f699c15f0c0 legacy=0x7f699c0b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:25.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.835+0000 7f69a0e75640 1 -- 172.21.15.45:0/1360243609 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c162e00 con 0x7f699c1638e0 2024-05-08T05:42:25.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.836+0000 7f69a0e75640 1 --2- 172.21.15.45:0/1360243609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f699c1638e0 0x7f699c0b6f40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f698c009870 tx=0x7f698c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1108d02eef148db6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:25.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.836+0000 7f6993fff640 1 -- 172.21.15.45:0/1360243609 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c037070 con 0x7f699c1638e0 2024-05-08T05:42:25.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.836+0000 7f6993fff640 1 -- 172.21.15.45:0/1360243609 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f698c032440 con 0x7f699c1638e0 2024-05-08T05:42:25.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.836+0000 7f6993fff640 1 -- 172.21.15.45:0/1360243609 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c032810 con 0x7f699c1638e0 2024-05-08T05:42:25.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.836+0000 7f69a38da640 1 -- 172.21.15.45:0/1360243609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f699c1638e0 msgr2=0x7f699c0b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:25.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.836+0000 7f69a38da640 1 --2- 172.21.15.45:0/1360243609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f699c1638e0 0x7f699c0b6f40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f698c009870 tx=0x7f698c02a270 comp rx=0 tx=0).stop 2024-05-08T05:42:25.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.837+0000 7f69a38da640 1 -- 172.21.15.45:0/1360243609 shutdown_connections 2024-05-08T05:42:25.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.837+0000 7f69a38da640 1 --2- 172.21.15.45:0/1360243609 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f699c06c3a0 0x7f699c06c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.837+0000 7f69a38da640 1 --2- 172.21.15.45:0/1360243609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f699c1638e0 0x7f699c0b6f40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.837+0000 7f69a38da640 1 -- 172.21.15.45:0/1360243609 >> 172.21.15.45:0/1360243609 conn(0x7f699c0b3b10 msgr2=0x7f699c0b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:25.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.837+0000 7f69a38da640 1 -- 172.21.15.45:0/1360243609 shutdown_connections 2024-05-08T05:42:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.837+0000 7f69a38da640 1 -- 172.21.15.45:0/1360243609 wait complete. 2024-05-08T05:42:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.838+0000 7f69a38da640 1 Processor -- start 2024-05-08T05:42:25.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.838+0000 7f69a38da640 1 -- start start 2024-05-08T05:42:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f69a38da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f699c06c3a0 0x7f699c159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f69a38da640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f699c15f0c0 0x7f699c0af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f69a38da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f699c1638e0 0x7f699c0d0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f69a38da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f699c15f4a0 con 0x7f699c1638e0 2024-05-08T05:42:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f69a38da640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f699c0b17c0 con 0x7f699c15f0c0 2024-05-08T05:42:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f69a38da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f699c15f620 con 0x7f699c06c3a0 2024-05-08T05:42:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f69a1e77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f699c1638e0 0x7f699c0d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f69a1e77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f699c1638e0 0x7f699c0d0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44270/0 (socket says 172.21.15.45:44270) 2024-05-08T05:42:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f69a1e77640 1 -- 172.21.15.45:0/232142919 learned_addr learned my addr 172.21.15.45:0/232142919 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f69a0e75640 1 --2- 172.21.15.45:0/232142919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f699c15f0c0 0x7f699c0af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f69a1676640 1 --2- 172.21.15.45:0/232142919 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f699c06c3a0 0x7f699c159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f69a1e77640 1 -- 172.21.15.45:0/232142919 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f699c06c3a0 msgr2=0x7f699c159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f69a1e77640 1 --2- 172.21.15.45:0/232142919 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f699c06c3a0 0x7f699c159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f69a1e77640 1 -- 172.21.15.45:0/232142919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f699c15f0c0 msgr2=0x7f699c0af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f69a1e77640 1 --2- 172.21.15.45:0/232142919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f699c15f0c0 0x7f699c0af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f69a1e77640 1 -- 172.21.15.45:0/232142919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c15a2e0 con 0x7f699c1638e0 2024-05-08T05:42:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f69a1e77640 1 --2- 172.21.15.45:0/232142919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f699c1638e0 0x7f699c0d0540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6994007e40 tx=0x7f699400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:25.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f6991ffb640 1 -- 172.21.15.45:0/232142919 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994014620 con 0x7f699c1638e0 2024-05-08T05:42:25.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f6991ffb640 1 -- 172.21.15.45:0/232142919 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6994019070 con 0x7f699c1638e0 2024-05-08T05:42:25.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f69a38da640 1 -- 172.21.15.45:0/232142919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699c0b0050 con 0x7f699c1638e0 2024-05-08T05:42:25.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.839+0000 7f6991ffb640 1 -- 172.21.15.45:0/232142919 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994014a20 con 0x7f699c1638e0 2024-05-08T05:42:25.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.840+0000 7f69a38da640 1 -- 172.21.15.45:0/232142919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699c0b0610 con 0x7f699c1638e0 2024-05-08T05:42:25.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.840+0000 7f69a38da640 1 -- 172.21.15.45:0/232142919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f699c0c6d80 con 0x7f699c1638e0 2024-05-08T05:42:25.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.841+0000 7f6991ffb640 1 -- 172.21.15.45:0/232142919 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6994014bc0 con 0x7f699c1638e0 2024-05-08T05:42:25.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.841+0000 7f6991ffb640 1 --2- 172.21.15.45:0/232142919 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f69800762d0 0x7f6980078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:25.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.842+0000 7f6991ffb640 1 -- 172.21.15.45:0/232142919 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f6994097e00 con 0x7f699c1638e0 2024-05-08T05:42:25.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.842+0000 7f69a1676640 1 --2- 172.21.15.45:0/232142919 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f69800762d0 0x7f6980078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:25.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.842+0000 7f69a1676640 1 --2- 172.21.15.45:0/232142919 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f69800762d0 0x7f6980078790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f699c0746a0 tx=0x7f6998009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:25.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:25.846+0000 7f6991ffb640 1 -- 172.21.15.45:0/232142919 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6994061010 con 0x7f699c1638e0 2024-05-08T05:42:26.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:26.032+0000 7f69a38da640 1 -- 172.21.15.45:0/232142919 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f699c06cf30 con 0x7f69800762d0 2024-05-08T05:42:26.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:26.038+0000 7f6991ffb640 1 -- 172.21.15.45:0/232142919 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249708 (secure 0 0 0) 0x7f699c06cf30 con 0x7f69800762d0 2024-05-08T05:42:26.041 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:26.042 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:42:26.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:26.041+0000 7f69a38da640 1 -- 172.21.15.45:0/232142919 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f69800762d0 msgr2=0x7f6980078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:26.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:26.041+0000 7f69a38da640 1 --2- 172.21.15.45:0/232142919 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f69800762d0 0x7f6980078790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f699c0746a0 tx=0x7f6998009040 comp rx=0 tx=0).stop 2024-05-08T05:42:26.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:26.041+0000 7f69a38da640 1 -- 172.21.15.45:0/232142919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f699c1638e0 msgr2=0x7f699c0d0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:26.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:26.041+0000 7f69a38da640 1 --2- 172.21.15.45:0/232142919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f699c1638e0 0x7f699c0d0540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6994007e40 tx=0x7f699400e4b0 comp rx=0 tx=0).stop 2024-05-08T05:42:26.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:26.041+0000 7f69a38da640 1 -- 172.21.15.45:0/232142919 shutdown_connections 2024-05-08T05:42:26.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:26.041+0000 7f69a38da640 1 --2- 172.21.15.45:0/232142919 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f69800762d0 0x7f6980078790 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:26.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:26.041+0000 7f69a38da640 1 --2- 172.21.15.45:0/232142919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f699c1638e0 0x7f699c0d0540 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:26.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:26.041+0000 7f69a38da640 1 --2- 172.21.15.45:0/232142919 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f699c15f0c0 0x7f699c0af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:26.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:26.041+0000 7f69a38da640 1 --2- 172.21.15.45:0/232142919 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f699c06c3a0 0x7f699c159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:26.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:26.041+0000 7f69a38da640 1 -- 172.21.15.45:0/232142919 >> 172.21.15.45:0/232142919 conn(0x7f699c0b3b10 msgr2=0x7f699c0b4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:26.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:26.041+0000 7f69a38da640 1 -- 172.21.15.45:0/232142919 shutdown_connections 2024-05-08T05:42:26.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:26.041+0000 7f69a38da640 1 -- 172.21.15.45:0/232142919 wait complete. 2024-05-08T05:42:26.052 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-05-08T05:42:24.845538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592084,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":122,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237088,"kb_used_data":4276,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737632,"statfs":{"total":773094113280,"available":772851335168,"internally_reserved":0,"allocated":4378624,"data_stored":2510653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1716,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000980"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569700+0000","last_change":"2024-05-08T05:42:20.521354+0000","last_active":"2024-05-08T05:42:23.569700+0000","last_peered":"2024-05-08T05:42:23.569700+0000","last_clean":"2024-05-08T05:42:23.569700+0000","last_became_active":"2024-05-08T05:42:20.521070+0000","last_became_peered":"2024-05-08T05:42:20.521070+0000","last_unstale":"2024-05-08T05:42:23.569700+0000","last_undegraded":"2024-05-08T05:42:23.569700+0000","last_fullsized":"2024-05-08T05:42:23.569700+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:08.311556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568750+0000","last_change":"2024-05-08T05:42:20.525189+0000","last_active":"2024-05-08T05:42:23.568750+0000","last_peered":"2024-05-08T05:42:23.568750+0000","last_clean":"2024-05-08T05:42:23.568750+0000","last_became_active":"2024-05-08T05:42:20.524906+0000","last_became_peered":"2024-05-08T05:42:20.524906+0000","last_unstale":"2024-05-08T05:42:23.568750+0000","last_undegraded":"2024-05-08T05:42:23.568750+0000","last_fullsized":"2024-05-08T05:42:23.568750+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:22:33.614866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569160+0000","last_change":"2024-05-08T05:42:20.532374+0000","last_active":"2024-05-08T05:42:23.569160+0000","last_peered":"2024-05-08T05:42:23.569160+0000","last_clean":"2024-05-08T05:42:23.569160+0000","last_became_active":"2024-05-08T05:42:20.532079+0000","last_became_peered":"2024-05-08T05:42:20.532079+0000","last_unstale":"2024-05-08T05:42:23.569160+0000","last_undegraded":"2024-05-08T05:42:23.569160+0000","last_fullsized":"2024-05-08T05:42:23.569160+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:23.089039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517311+0000","last_change":"2024-05-08T05:42:20.530407+0000","last_active":"2024-05-08T05:42:21.517311+0000","last_peered":"2024-05-08T05:42:21.517311+0000","last_clean":"2024-05-08T05:42:21.517311+0000","last_became_active":"2024-05-08T05:42:20.530148+0000","last_became_peered":"2024-05-08T05:42:20.530148+0000","last_unstale":"2024-05-08T05:42:21.517311+0000","last_undegraded":"2024-05-08T05:42:21.517311+0000","last_fullsized":"2024-05-08T05:42:21.517311+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:30.266997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517173+0000","last_change":"2024-05-08T05:42:20.528145+0000","last_active":"2024-05-08T05:42:21.517173+0000","last_peered":"2024-05-08T05:42:21.517173+0000","last_clean":"2024-05-08T05:42:21.517173+0000","last_became_active":"2024-05-08T05:42:20.527841+0000","last_became_peered":"2024-05-08T05:42:20.527841+0000","last_unstale":"2024-05-08T05:42:21.517173+0000","last_undegraded":"2024-05-08T05:42:21.517173+0000","last_fullsized":"2024-05-08T05:42:21.517173+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:15.132265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569078+0000","last_change":"2024-05-08T05:42:20.518897+0000","last_active":"2024-05-08T05:42:23.569078+0000","last_peered":"2024-05-08T05:42:23.569078+0000","last_clean":"2024-05-08T05:42:23.569078+0000","last_became_active":"2024-05-08T05:42:20.518613+0000","last_became_peered":"2024-05-08T05:42:20.518613+0000","last_unstale":"2024-05-08T05:42:23.569078+0000","last_undegraded":"2024-05-08T05:42:23.569078+0000","last_fullsized":"2024-05-08T05:42:23.569078+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:23.089039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.494063+0000","last_change":"2024-05-08T05:42:19.494063+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.494063+0000","last_undegraded":"2024-05-08T05:42:19.494063+0000","last_fullsized":"2024-05-08T05:42:19.494063+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568409+0000","last_change":"2024-05-08T05:42:20.515418+0000","last_active":"2024-05-08T05:42:23.568409+0000","last_peered":"2024-05-08T05:42:23.568409+0000","last_clean":"2024-05-08T05:42:23.568409+0000","last_became_active":"2024-05-08T05:42:20.514961+0000","last_became_peered":"2024-05-08T05:42:20.514961+0000","last_unstale":"2024-05-08T05:42:23.568409+0000","last_undegraded":"2024-05-08T05:42:23.568409+0000","last_fullsized":"2024-05-08T05:42:23.568409+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:54:21.060834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569512+0000","last_change":"2024-05-08T05:42:20.533002+0000","last_active":"2024-05-08T05:42:23.569512+0000","last_peered":"2024-05-08T05:42:23.569512+0000","last_clean":"2024-05-08T05:42:23.569512+0000","last_became_active":"2024-05-08T05:42:20.532713+0000","last_became_peered":"2024-05-08T05:42:20.532713+0000","last_unstale":"2024-05-08T05:42:23.569512+0000","last_undegraded":"2024-05-08T05:42:23.569512+0000","last_fullsized":"2024-05-08T05:42:23.569512+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:34:08.047335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.492151+0000","last_change":"2024-05-08T05:42:19.492151+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.492151+0000","last_undegraded":"2024-05-08T05:42:19.492151+0000","last_fullsized":"2024-05-08T05:42:19.492151+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.489300+0000","last_change":"2024-05-08T05:42:19.489300+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.489300+0000","last_undegraded":"2024-05-08T05:42:19.489300+0000","last_fullsized":"2024-05-08T05:42:19.489300+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568992+0000","last_change":"2024-05-08T05:42:20.527823+0000","last_active":"2024-05-08T05:42:23.568992+0000","last_peered":"2024-05-08T05:42:23.568992+0000","last_clean":"2024-05-08T05:42:23.568992+0000","last_became_active":"2024-05-08T05:42:20.527531+0000","last_became_peered":"2024-05-08T05:42:20.527531+0000","last_unstale":"2024-05-08T05:42:23.568992+0000","last_undegraded":"2024-05-08T05:42:23.568992+0000","last_fullsized":"2024-05-08T05:42:23.568992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:06.429659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.488571+0000","last_change":"2024-05-08T05:42:19.488571+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.488571+0000","last_undegraded":"2024-05-08T05:42:19.488571+0000","last_fullsized":"2024-05-08T05:42:19.488571+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568056+0000","last_change":"2024-05-08T05:42:20.520112+0000","last_active":"2024-05-08T05:42:23.568056+0000","last_peered":"2024-05-08T05:42:23.568056+0000","last_clean":"2024-05-08T05:42:23.568056+0000","last_became_active":"2024-05-08T05:42:20.519660+0000","last_became_peered":"2024-05-08T05:42:20.519660+0000","last_unstale":"2024-05-08T05:42:23.568056+0000","last_undegraded":"2024-05-08T05:42:23.568056+0000","last_fullsized":"2024-05-08T05:42:23.568056+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:25:23.199079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.491652+0000","last_change":"2024-05-08T05:42:19.491652+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.491652+0000","last_undegraded":"2024-05-08T05:42:19.491652+0000","last_fullsized":"2024-05-08T05:42:19.491652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.483214+0000","last_change":"2024-05-08T05:42:19.483214+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.483214+0000","last_undegraded":"2024-05-08T05:42:19.483214+0000","last_fullsized":"2024-05-08T05:42:19.483214+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.482113+0000","last_change":"2024-05-08T05:42:19.482113+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.482113+0000","last_undegraded":"2024-05-08T05:42:19.482113+0000","last_fullsized":"2024-05-08T05:42:19.482113+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.486278+0000","last_change":"2024-05-08T05:42:19.486278+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.486278+0000","last_undegraded":"2024-05-08T05:42:19.486278+0000","last_fullsized":"2024-05-08T05:42:19.486278+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517514+0000","last_change":"2024-05-08T05:42:20.533970+0000","last_active":"2024-05-08T05:42:21.517514+0000","last_peered":"2024-05-08T05:42:21.517514+0000","last_clean":"2024-05-08T05:42:21.517514+0000","last_became_active":"2024-05-08T05:42:20.531975+0000","last_became_peered":"2024-05-08T05:42:20.531975+0000","last_unstale":"2024-05-08T05:42:21.517514+0000","last_undegraded":"2024-05-08T05:42:21.517514+0000","last_fullsized":"2024-05-08T05:42:21.517514+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:20:52.733574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.489219+0000","last_change":"2024-05-08T05:42:19.489219+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.489219+0000","last_undegraded":"2024-05-08T05:42:19.489219+0000","last_fullsized":"2024-05-08T05:42:19.489219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.489042+0000","last_change":"2024-05-08T05:42:19.489042+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.489042+0000","last_undegraded":"2024-05-08T05:42:19.489042+0000","last_fullsized":"2024-05-08T05:42:19.489042+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568464+0000","last_change":"2024-05-08T05:42:20.525789+0000","last_active":"2024-05-08T05:42:23.568464+0000","last_peered":"2024-05-08T05:42:23.568464+0000","last_clean":"2024-05-08T05:42:23.568464+0000","last_became_active":"2024-05-08T05:42:20.525492+0000","last_became_peered":"2024-05-08T05:42:20.525492+0000","last_unstale":"2024-05-08T05:42:23.568464+0000","last_undegraded":"2024-05-08T05:42:23.568464+0000","last_fullsized":"2024-05-08T05:42:23.568464+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:44.235500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.481502+0000","last_change":"2024-05-08T05:42:19.481502+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.481502+0000","last_undegraded":"2024-05-08T05:42:19.481502+0000","last_fullsized":"2024-05-08T05:42:19.481502+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517037+0000","last_change":"2024-05-08T05:42:20.534832+0000","last_active":"2024-05-08T05:42:21.517037+0000","last_peered":"2024-05-08T05:42:21.517037+0000","last_clean":"2024-05-08T05:42:21.517037+0000","last_became_active":"2024-05-08T05:42:20.534586+0000","last_became_peered":"2024-05-08T05:42:20.534586+0000","last_unstale":"2024-05-08T05:42:21.517037+0000","last_undegraded":"2024-05-08T05:42:21.517037+0000","last_fullsized":"2024-05-08T05:42:21.517037+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:30.319294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.485694+0000","last_change":"2024-05-08T05:42:19.485694+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.485694+0000","last_undegraded":"2024-05-08T05:42:19.485694+0000","last_fullsized":"2024-05-08T05:42:19.485694+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569431+0000","last_change":"2024-05-08T05:42:20.509600+0000","last_active":"2024-05-08T05:42:23.569431+0000","last_peered":"2024-05-08T05:42:23.569431+0000","last_clean":"2024-05-08T05:42:23.569431+0000","last_became_active":"2024-05-08T05:42:20.509045+0000","last_became_peered":"2024-05-08T05:42:20.509045+0000","last_unstale":"2024-05-08T05:42:23.569431+0000","last_undegraded":"2024-05-08T05:42:23.569431+0000","last_fullsized":"2024-05-08T05:42:23.569431+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:39.499482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.486709+0000","last_change":"2024-05-08T05:42:19.486709+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.486709+0000","last_undegraded":"2024-05-08T05:42:19.486709+0000","last_fullsized":"2024-05-08T05:42:19.486709+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.488095+0000","last_change":"2024-05-08T05:42:19.488095+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.488095+0000","last_undegraded":"2024-05-08T05:42:19.488095+0000","last_fullsized":"2024-05-08T05:42:19.488095+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.488425+0000","last_change":"2024-05-08T05:42:19.488425+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.488425+0000","last_undegraded":"2024-05-08T05:42:19.488425+0000","last_fullsized":"2024-05-08T05:42:19.488425+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.485913+0000","last_change":"2024-05-08T05:42:19.485913+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.485913+0000","last_undegraded":"2024-05-08T05:42:19.485913+0000","last_fullsized":"2024-05-08T05:42:19.485913+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.481029+0000","last_change":"2024-05-08T05:42:19.481029+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.481029+0000","last_undegraded":"2024-05-08T05:42:19.481029+0000","last_fullsized":"2024-05-08T05:42:19.481029+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569182+0000","last_change":"2024-05-08T05:42:20.524631+0000","last_active":"2024-05-08T05:42:23.569182+0000","last_peered":"2024-05-08T05:42:23.569182+0000","last_clean":"2024-05-08T05:42:23.569182+0000","last_became_active":"2024-05-08T05:42:20.524354+0000","last_became_peered":"2024-05-08T05:42:20.524354+0000","last_unstale":"2024-05-08T05:42:23.569182+0000","last_undegraded":"2024-05-08T05:42:23.569182+0000","last_fullsized":"2024-05-08T05:42:23.569182+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:34:08.047335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.481092+0000","last_change":"2024-05-08T05:42:19.481092+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.481092+0000","last_undegraded":"2024-05-08T05:42:19.481092+0000","last_fullsized":"2024-05-08T05:42:19.481092+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.497640+0000","last_change":"2024-05-08T05:42:19.493864+0000","last_active":"2024-05-08T05:42:19.497640+0000","last_peered":"2024-05-08T05:42:19.497640+0000","last_clean":"2024-05-08T05:42:19.497640+0000","last_became_active":"2024-05-08T05:42:19.493061+0000","last_became_peered":"2024-05-08T05:42:19.493061+0000","last_unstale":"2024-05-08T05:42:19.497640+0000","last_undegraded":"2024-05-08T05:42:19.497640+0000","last_fullsized":"2024-05-08T05:42:19.497640+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:47:27.293889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569291+0000","last_change":"2024-05-08T05:42:20.512807+0000","last_active":"2024-05-08T05:42:23.569291+0000","last_peered":"2024-05-08T05:42:23.569291+0000","last_clean":"2024-05-08T05:42:23.569291+0000","last_became_active":"2024-05-08T05:42:20.512501+0000","last_became_peered":"2024-05-08T05:42:20.512501+0000","last_unstale":"2024-05-08T05:42:23.569291+0000","last_undegraded":"2024-05-08T05:42:23.569291+0000","last_fullsized":"2024-05-08T05:42:23.569291+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:32:33.781763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569814+0000","last_change":"2024-05-08T05:42:19.493675+0000","last_active":"2024-05-08T05:42:23.569814+0000","last_peered":"2024-05-08T05:42:23.569814+0000","last_clean":"2024-05-08T05:42:23.569814+0000","last_became_active":"2024-05-08T05:42:19.493339+0000","last_became_peered":"2024-05-08T05:42:19.493339+0000","last_unstale":"2024-05-08T05:42:23.569814+0000","last_undegraded":"2024-05-08T05:42:23.569814+0000","last_fullsized":"2024-05-08T05:42:23.569814+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:51:24.291839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568501+0000","last_change":"2024-05-08T05:42:20.521416+0000","last_active":"2024-05-08T05:42:23.568501+0000","last_peered":"2024-05-08T05:42:23.568501+0000","last_clean":"2024-05-08T05:42:23.568501+0000","last_became_active":"2024-05-08T05:42:20.520422+0000","last_became_peered":"2024-05-08T05:42:20.520422+0000","last_unstale":"2024-05-08T05:42:23.568501+0000","last_undegraded":"2024-05-08T05:42:23.568501+0000","last_fullsized":"2024-05-08T05:42:23.568501+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:18.431907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569440+0000","last_change":"2024-05-08T05:42:19.491376+0000","last_active":"2024-05-08T05:42:23.569440+0000","last_peered":"2024-05-08T05:42:23.569440+0000","last_clean":"2024-05-08T05:42:23.569440+0000","last_became_active":"2024-05-08T05:42:19.491075+0000","last_became_peered":"2024-05-08T05:42:19.491075+0000","last_unstale":"2024-05-08T05:42:23.569440+0000","last_undegraded":"2024-05-08T05:42:23.569440+0000","last_fullsized":"2024-05-08T05:42:23.569440+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:28:34.310847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569384+0000","last_change":"2024-05-08T05:42:20.529711+0000","last_active":"2024-05-08T05:42:23.569384+0000","last_peered":"2024-05-08T05:42:23.569384+0000","last_clean":"2024-05-08T05:42:23.569384+0000","last_became_active":"2024-05-08T05:42:20.529354+0000","last_became_peered":"2024-05-08T05:42:20.529354+0000","last_unstale":"2024-05-08T05:42:23.569384+0000","last_undegraded":"2024-05-08T05:42:23.569384+0000","last_fullsized":"2024-05-08T05:42:23.569384+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:43:42.826524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568253+0000","last_change":"2024-05-08T05:42:19.495593+0000","last_active":"2024-05-08T05:42:23.568253+0000","last_peered":"2024-05-08T05:42:23.568253+0000","last_clean":"2024-05-08T05:42:23.568253+0000","last_became_active":"2024-05-08T05:42:19.495297+0000","last_became_peered":"2024-05-08T05:42:19.495297+0000","last_unstale":"2024-05-08T05:42:23.568253+0000","last_undegraded":"2024-05-08T05:42:23.568253+0000","last_fullsized":"2024-05-08T05:42:23.568253+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:29.317536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568467+0000","last_change":"2024-05-08T05:42:20.525554+0000","last_active":"2024-05-08T05:42:23.568467+0000","last_peered":"2024-05-08T05:42:23.568467+0000","last_clean":"2024-05-08T05:42:23.568467+0000","last_became_active":"2024-05-08T05:42:20.525027+0000","last_became_peered":"2024-05-08T05:42:20.525027+0000","last_unstale":"2024-05-08T05:42:23.568467+0000","last_undegraded":"2024-05-08T05:42:23.568467+0000","last_fullsized":"2024-05-08T05:42:23.568467+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:44:05.274392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.501668+0000","last_change":"2024-05-08T05:42:19.499533+0000","last_active":"2024-05-08T05:42:19.501668+0000","last_peered":"2024-05-08T05:42:19.501668+0000","last_clean":"2024-05-08T05:42:19.501668+0000","last_became_active":"2024-05-08T05:42:19.498447+0000","last_became_peered":"2024-05-08T05:42:19.498447+0000","last_unstale":"2024-05-08T05:42:19.501668+0000","last_undegraded":"2024-05-08T05:42:19.501668+0000","last_fullsized":"2024-05-08T05:42:19.501668+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:37.041564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.488852+0000","last_change":"2024-05-08T05:42:19.488852+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.488852+0000","last_undegraded":"2024-05-08T05:42:19.488852+0000","last_fullsized":"2024-05-08T05:42:19.488852+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.497089+0000","last_change":"2024-05-08T05:42:19.495816+0000","last_active":"2024-05-08T05:42:19.497089+0000","last_peered":"2024-05-08T05:42:19.497089+0000","last_clean":"2024-05-08T05:42:19.497089+0000","last_became_active":"2024-05-08T05:42:19.495546+0000","last_became_peered":"2024-05-08T05:42:19.495546+0000","last_unstale":"2024-05-08T05:42:19.497089+0000","last_undegraded":"2024-05-08T05:42:19.497089+0000","last_fullsized":"2024-05-08T05:42:19.497089+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:26.053156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568299+0000","last_change":"2024-05-08T05:42:20.512851+0000","last_active":"2024-05-08T05:42:23.568299+0000","last_peered":"2024-05-08T05:42:23.568299+0000","last_clean":"2024-05-08T05:42:23.568299+0000","last_became_active":"2024-05-08T05:42:20.512412+0000","last_became_peered":"2024-05-08T05:42:20.512412+0000","last_unstale":"2024-05-08T05:42:23.568299+0000","last_undegraded":"2024-05-08T05:42:23.568299+0000","last_fullsized":"2024-05-08T05:42:23.568299+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:23.447125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569070+0000","last_change":"2024-05-08T05:42:19.499539+0000","last_active":"2024-05-08T05:42:23.569070+0000","last_peered":"2024-05-08T05:42:23.569070+0000","last_clean":"2024-05-08T05:42:23.569070+0000","last_became_active":"2024-05-08T05:42:19.499241+0000","last_became_peered":"2024-05-08T05:42:19.499241+0000","last_unstale":"2024-05-08T05:42:23.569070+0000","last_undegraded":"2024-05-08T05:42:23.569070+0000","last_fullsized":"2024-05-08T05:42:23.569070+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:22.668414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569402+0000","last_change":"2024-05-08T05:42:20.533418+0000","last_active":"2024-05-08T05:42:23.569402+0000","last_peered":"2024-05-08T05:42:23.569402+0000","last_clean":"2024-05-08T05:42:23.569402+0000","last_became_active":"2024-05-08T05:42:20.532766+0000","last_became_peered":"2024-05-08T05:42:20.532766+0000","last_unstale":"2024-05-08T05:42:23.569402+0000","last_undegraded":"2024-05-08T05:42:23.569402+0000","last_fullsized":"2024-05-08T05:42:23.569402+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:44.462590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.567873+0000","last_change":"2024-05-08T05:42:19.491786+0000","last_active":"2024-05-08T05:42:23.567873+0000","last_peered":"2024-05-08T05:42:23.567873+0000","last_clean":"2024-05-08T05:42:23.567873+0000","last_became_active":"2024-05-08T05:42:19.491455+0000","last_became_peered":"2024-05-08T05:42:19.491455+0000","last_unstale":"2024-05-08T05:42:23.567873+0000","last_undegraded":"2024-05-08T05:42:23.567873+0000","last_fullsized":"2024-05-08T05:42:23.567873+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:51:24.291839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517387+0000","last_change":"2024-05-08T05:42:20.529431+0000","last_active":"2024-05-08T05:42:21.517387+0000","last_peered":"2024-05-08T05:42:21.517387+0000","last_clean":"2024-05-08T05:42:21.517387+0000","last_became_active":"2024-05-08T05:42:20.529130+0000","last_became_peered":"2024-05-08T05:42:20.529130+0000","last_unstale":"2024-05-08T05:42:21.517387+0000","last_undegraded":"2024-05-08T05:42:21.517387+0000","last_fullsized":"2024-05-08T05:42:21.517387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:51:25.293597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569722+0000","last_change":"2024-05-08T05:42:19.487759+0000","last_active":"2024-05-08T05:42:23.569722+0000","last_peered":"2024-05-08T05:42:23.569722+0000","last_clean":"2024-05-08T05:42:23.569722+0000","last_became_active":"2024-05-08T05:42:19.487492+0000","last_became_peered":"2024-05-08T05:42:19.487492+0000","last_unstale":"2024-05-08T05:42:23.569722+0000","last_undegraded":"2024-05-08T05:42:23.569722+0000","last_fullsized":"2024-05-08T05:42:23.569722+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:22.445367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.481950+0000","last_change":"2024-05-08T05:42:19.481950+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.481950+0000","last_undegraded":"2024-05-08T05:42:19.481950+0000","last_fullsized":"2024-05-08T05:42:19.481950+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.501312+0000","last_change":"2024-05-08T05:42:19.498339+0000","last_active":"2024-05-08T05:42:19.501312+0000","last_peered":"2024-05-08T05:42:19.501312+0000","last_clean":"2024-05-08T05:42:19.501312+0000","last_became_active":"2024-05-08T05:42:19.498000+0000","last_became_peered":"2024-05-08T05:42:19.498000+0000","last_unstale":"2024-05-08T05:42:19.501312+0000","last_undegraded":"2024-05-08T05:42:19.501312+0000","last_fullsized":"2024-05-08T05:42:19.501312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:22.668414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569759+0000","last_change":"2024-05-08T05:42:20.525378+0000","last_active":"2024-05-08T05:42:23.569759+0000","last_peered":"2024-05-08T05:42:23.569759+0000","last_clean":"2024-05-08T05:42:23.569759+0000","last_became_active":"2024-05-08T05:42:20.525089+0000","last_became_peered":"2024-05-08T05:42:20.525089+0000","last_unstale":"2024-05-08T05:42:23.569759+0000","last_undegraded":"2024-05-08T05:42:23.569759+0000","last_fullsized":"2024-05-08T05:42:23.569759+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:32:33.781763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.501275+0000","last_change":"2024-05-08T05:42:19.499267+0000","last_active":"2024-05-08T05:42:19.501275+0000","last_peered":"2024-05-08T05:42:19.501275+0000","last_clean":"2024-05-08T05:42:19.501275+0000","last_became_active":"2024-05-08T05:42:19.498537+0000","last_became_peered":"2024-05-08T05:42:19.498537+0000","last_unstale":"2024-05-08T05:42:19.501275+0000","last_undegraded":"2024-05-08T05:42:19.501275+0000","last_fullsized":"2024-05-08T05:42:19.501275+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:53:50.777397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568542+0000","last_change":"2024-05-08T05:42:20.525088+0000","last_active":"2024-05-08T05:42:23.568542+0000","last_peered":"2024-05-08T05:42:23.568542+0000","last_clean":"2024-05-08T05:42:23.568542+0000","last_became_active":"2024-05-08T05:42:20.524526+0000","last_became_peered":"2024-05-08T05:42:20.524526+0000","last_unstale":"2024-05-08T05:42:23.568542+0000","last_undegraded":"2024-05-08T05:42:23.568542+0000","last_fullsized":"2024-05-08T05:42:23.568542+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:06.429659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569781+0000","last_change":"2024-05-08T05:42:19.488838+0000","last_active":"2024-05-08T05:42:23.569781+0000","last_peered":"2024-05-08T05:42:23.569781+0000","last_clean":"2024-05-08T05:42:23.569781+0000","last_became_active":"2024-05-08T05:42:19.488090+0000","last_became_peered":"2024-05-08T05:42:19.488090+0000","last_unstale":"2024-05-08T05:42:23.569781+0000","last_undegraded":"2024-05-08T05:42:23.569781+0000","last_fullsized":"2024-05-08T05:42:23.569781+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:17.430149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568617+0000","last_change":"2024-05-08T05:42:20.519358+0000","last_active":"2024-05-08T05:42:23.568617+0000","last_peered":"2024-05-08T05:42:23.568617+0000","last_clean":"2024-05-08T05:42:23.568617+0000","last_became_active":"2024-05-08T05:42:20.519067+0000","last_became_peered":"2024-05-08T05:42:20.519067+0000","last_unstale":"2024-05-08T05:42:23.568617+0000","last_undegraded":"2024-05-08T05:42:23.568617+0000","last_fullsized":"2024-05-08T05:42:23.568617+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:44.235500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569519+0000","last_change":"2024-05-08T05:42:19.486715+0000","last_active":"2024-05-08T05:42:23.569519+0000","last_peered":"2024-05-08T05:42:23.569519+0000","last_clean":"2024-05-08T05:42:23.569519+0000","last_became_active":"2024-05-08T05:42:19.486403+0000","last_became_peered":"2024-05-08T05:42:19.486403+0000","last_unstale":"2024-05-08T05:42:23.569519+0000","last_undegraded":"2024-05-08T05:42:23.569519+0000","last_fullsized":"2024-05-08T05:42:23.569519+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:43.233742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568767+0000","last_change":"2024-05-08T05:42:20.518993+0000","last_active":"2024-05-08T05:42:23.568767+0000","last_peered":"2024-05-08T05:42:23.568767+0000","last_clean":"2024-05-08T05:42:23.568767+0000","last_became_active":"2024-05-08T05:42:20.518710+0000","last_became_peered":"2024-05-08T05:42:20.518710+0000","last_unstale":"2024-05-08T05:42:23.568767+0000","last_undegraded":"2024-05-08T05:42:23.568767+0000","last_fullsized":"2024-05-08T05:42:23.568767+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:30:06.045575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569189+0000","last_change":"2024-05-08T05:42:19.500123+0000","last_active":"2024-05-08T05:42:23.569189+0000","last_peered":"2024-05-08T05:42:23.569189+0000","last_clean":"2024-05-08T05:42:23.569189+0000","last_became_active":"2024-05-08T05:42:19.499811+0000","last_became_peered":"2024-05-08T05:42:19.499811+0000","last_unstale":"2024-05-08T05:42:23.569189+0000","last_undegraded":"2024-05-08T05:42:23.569189+0000","last_fullsized":"2024-05-08T05:42:23.569189+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:26.053156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.480740+0000","last_change":"2024-05-08T05:42:19.480740+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.480740+0000","last_undegraded":"2024-05-08T05:42:19.480740+0000","last_fullsized":"2024-05-08T05:42:19.480740+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.498152+0000","last_change":"2024-05-08T05:42:19.496693+0000","last_active":"2024-05-08T05:42:19.498152+0000","last_peered":"2024-05-08T05:42:19.498152+0000","last_clean":"2024-05-08T05:42:19.498152+0000","last_became_active":"2024-05-08T05:42:19.496276+0000","last_became_peered":"2024-05-08T05:42:19.496276+0000","last_unstale":"2024-05-08T05:42:19.498152+0000","last_undegraded":"2024-05-08T05:42:19.498152+0000","last_fullsized":"2024-05-08T05:42:19.498152+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:50:26.559921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569458+0000","last_change":"2024-05-08T05:42:20.521385+0000","last_active":"2024-05-08T05:42:23.569458+0000","last_peered":"2024-05-08T05:42:23.569458+0000","last_clean":"2024-05-08T05:42:23.569458+0000","last_became_active":"2024-05-08T05:42:20.521089+0000","last_became_peered":"2024-05-08T05:42:20.521089+0000","last_unstale":"2024-05-08T05:42:23.569458+0000","last_undegraded":"2024-05-08T05:42:23.569458+0000","last_fullsized":"2024-05-08T05:42:23.569458+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:01.706249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.495293+0000","last_change":"2024-05-08T05:42:19.493145+0000","last_active":"2024-05-08T05:42:19.495293+0000","last_peered":"2024-05-08T05:42:19.495293+0000","last_clean":"2024-05-08T05:42:19.495293+0000","last_became_active":"2024-05-08T05:42:19.492821+0000","last_became_peered":"2024-05-08T05:42:19.492821+0000","last_unstale":"2024-05-08T05:42:19.495293+0000","last_undegraded":"2024-05-08T05:42:19.495293+0000","last_fullsized":"2024-05-08T05:42:19.495293+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:20:51.731816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569723+0000","last_change":"2024-05-08T05:42:20.529928+0000","last_active":"2024-05-08T05:42:23.569723+0000","last_peered":"2024-05-08T05:42:23.569723+0000","last_clean":"2024-05-08T05:42:23.569723+0000","last_became_active":"2024-05-08T05:42:20.529618+0000","last_became_peered":"2024-05-08T05:42:20.529618+0000","last_unstale":"2024-05-08T05:42:23.569723+0000","last_undegraded":"2024-05-08T05:42:23.569723+0000","last_fullsized":"2024-05-08T05:42:23.569723+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:33:26.883953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.501664+0000","last_change":"2024-05-08T05:42:19.499523+0000","last_active":"2024-05-08T05:42:19.501664+0000","last_peered":"2024-05-08T05:42:19.501664+0000","last_clean":"2024-05-08T05:42:19.501664+0000","last_became_active":"2024-05-08T05:42:19.499064+0000","last_became_peered":"2024-05-08T05:42:19.499064+0000","last_unstale":"2024-05-08T05:42:19.501664+0000","last_undegraded":"2024-05-08T05:42:19.501664+0000","last_fullsized":"2024-05-08T05:42:19.501664+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:28:34.310847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.488517+0000","last_change":"2024-05-08T05:42:19.488517+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.488517+0000","last_undegraded":"2024-05-08T05:42:19.488517+0000","last_fullsized":"2024-05-08T05:42:19.488517+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568515+0000","last_change":"2024-05-08T05:42:19.500389+0000","last_active":"2024-05-08T05:42:23.568515+0000","last_peered":"2024-05-08T05:42:23.568515+0000","last_clean":"2024-05-08T05:42:23.568515+0000","last_became_active":"2024-05-08T05:42:19.500111+0000","last_became_peered":"2024-05-08T05:42:19.500111+0000","last_unstale":"2024-05-08T05:42:23.568515+0000","last_undegraded":"2024-05-08T05:42:23.568515+0000","last_fullsized":"2024-05-08T05:42:23.568515+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:14.130507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568430+0000","last_change":"2024-05-08T05:42:20.522305+0000","last_active":"2024-05-08T05:42:23.568430+0000","last_peered":"2024-05-08T05:42:23.568430+0000","last_clean":"2024-05-08T05:42:23.568430+0000","last_became_active":"2024-05-08T05:42:20.521820+0000","last_became_peered":"2024-05-08T05:42:20.521820+0000","last_unstale":"2024-05-08T05:42:23.568430+0000","last_undegraded":"2024-05-08T05:42:23.568430+0000","last_fullsized":"2024-05-08T05:42:23.568430+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:44:05.274392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517447+0000","last_change":"2024-05-08T05:42:19.491582+0000","last_active":"2024-05-08T05:42:21.517447+0000","last_peered":"2024-05-08T05:42:21.517447+0000","last_clean":"2024-05-08T05:42:21.517447+0000","last_became_active":"2024-05-08T05:42:19.491291+0000","last_became_peered":"2024-05-08T05:42:19.491291+0000","last_unstale":"2024-05-08T05:42:21.517447+0000","last_undegraded":"2024-05-08T05:42:21.517447+0000","last_fullsized":"2024-05-08T05:42:21.517447+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:43:53.904889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517451+0000","last_change":"2024-05-08T05:42:20.527117+0000","last_active":"2024-05-08T05:42:21.517451+0000","last_peered":"2024-05-08T05:42:21.517451+0000","last_clean":"2024-05-08T05:42:21.517451+0000","last_became_active":"2024-05-08T05:42:20.526717+0000","last_became_peered":"2024-05-08T05:42:20.526717+0000","last_unstale":"2024-05-08T05:42:21.517451+0000","last_undegraded":"2024-05-08T05:42:21.517451+0000","last_fullsized":"2024-05-08T05:42:21.517451+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:26.076899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568839+0000","last_change":"2024-05-08T05:42:19.491505+0000","last_active":"2024-05-08T05:42:23.568839+0000","last_peered":"2024-05-08T05:42:23.568839+0000","last_clean":"2024-05-08T05:42:23.568839+0000","last_became_active":"2024-05-08T05:42:19.491206+0000","last_became_peered":"2024-05-08T05:42:19.491206+0000","last_unstale":"2024-05-08T05:42:23.568839+0000","last_undegraded":"2024-05-08T05:42:23.568839+0000","last_fullsized":"2024-05-08T05:42:23.568839+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:53:50.777397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.480258+0000","last_change":"2024-05-08T05:42:19.480258+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.480258+0000","last_undegraded":"2024-05-08T05:42:19.480258+0000","last_fullsized":"2024-05-08T05:42:19.480258+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517236+0000","last_change":"2024-05-08T05:42:19.494577+0000","last_active":"2024-05-08T05:42:21.517236+0000","last_peered":"2024-05-08T05:42:21.517236+0000","last_clean":"2024-05-08T05:42:21.517236+0000","last_became_active":"2024-05-08T05:42:19.494261+0000","last_became_peered":"2024-05-08T05:42:19.494261+0000","last_unstale":"2024-05-08T05:42:21.517236+0000","last_undegraded":"2024-05-08T05:42:21.517236+0000","last_fullsized":"2024-05-08T05:42:21.517236+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:07:51.458867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568433+0000","last_change":"2024-05-08T05:42:20.524296+0000","last_active":"2024-05-08T05:42:23.568433+0000","last_peered":"2024-05-08T05:42:23.568433+0000","last_clean":"2024-05-08T05:42:23.568433+0000","last_became_active":"2024-05-08T05:42:20.523971+0000","last_became_peered":"2024-05-08T05:42:20.523971+0000","last_unstale":"2024-05-08T05:42:23.568433+0000","last_undegraded":"2024-05-08T05:42:23.568433+0000","last_fullsized":"2024-05-08T05:42:23.568433+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:23.447125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517421+0000","last_change":"2024-05-08T05:42:19.495869+0000","last_active":"2024-05-08T05:42:21.517421+0000","last_peered":"2024-05-08T05:42:21.517421+0000","last_clean":"2024-05-08T05:42:21.517421+0000","last_became_active":"2024-05-08T05:42:19.495494+0000","last_became_peered":"2024-05-08T05:42:19.495494+0000","last_unstale":"2024-05-08T05:42:21.517421+0000","last_undegraded":"2024-05-08T05:42:21.517421+0000","last_fullsized":"2024-05-08T05:42:21.517421+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:08.085241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.490421+0000","last_change":"2024-05-08T05:42:19.490421+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.490421+0000","last_undegraded":"2024-05-08T05:42:19.490421+0000","last_fullsized":"2024-05-08T05:42:19.490421+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.495945+0000","last_change":"2024-05-08T05:42:19.494546+0000","last_active":"2024-05-08T05:42:19.495945+0000","last_peered":"2024-05-08T05:42:19.495945+0000","last_clean":"2024-05-08T05:42:19.495945+0000","last_became_active":"2024-05-08T05:42:19.494232+0000","last_became_peered":"2024-05-08T05:42:19.494232+0000","last_unstale":"2024-05-08T05:42:19.495945+0000","last_undegraded":"2024-05-08T05:42:19.495945+0000","last_fullsized":"2024-05-08T05:42:19.495945+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:22.668414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569563+0000","last_change":"2024-05-08T05:42:20.533065+0000","last_active":"2024-05-08T05:42:23.569563+0000","last_peered":"2024-05-08T05:42:23.569563+0000","last_clean":"2024-05-08T05:42:23.569563+0000","last_became_active":"2024-05-08T05:42:20.532437+0000","last_became_peered":"2024-05-08T05:42:20.532437+0000","last_unstale":"2024-05-08T05:42:23.569563+0000","last_undegraded":"2024-05-08T05:42:23.569563+0000","last_fullsized":"2024-05-08T05:42:23.569563+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:50.284596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517058+0000","last_change":"2024-05-08T05:42:19.494499+0000","last_active":"2024-05-08T05:42:21.517058+0000","last_peered":"2024-05-08T05:42:21.517058+0000","last_clean":"2024-05-08T05:42:21.517058+0000","last_became_active":"2024-05-08T05:42:19.494193+0000","last_became_peered":"2024-05-08T05:42:19.494193+0000","last_unstale":"2024-05-08T05:42:21.517058+0000","last_undegraded":"2024-05-08T05:42:21.517058+0000","last_fullsized":"2024-05-08T05:42:21.517058+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:34.880276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568778+0000","last_change":"2024-05-08T05:42:20.518085+0000","last_active":"2024-05-08T05:42:23.568778+0000","last_peered":"2024-05-08T05:42:23.568778+0000","last_clean":"2024-05-08T05:42:23.568778+0000","last_became_active":"2024-05-08T05:42:20.517771+0000","last_became_peered":"2024-05-08T05:42:20.517771+0000","last_unstale":"2024-05-08T05:42:23.568778+0000","last_undegraded":"2024-05-08T05:42:23.568778+0000","last_fullsized":"2024-05-08T05:42:23.568778+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:18.431907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.501230+0000","last_change":"2024-05-08T05:42:19.498985+0000","last_active":"2024-05-08T05:42:19.501230+0000","last_peered":"2024-05-08T05:42:19.501230+0000","last_clean":"2024-05-08T05:42:19.501230+0000","last_became_active":"2024-05-08T05:42:19.498663+0000","last_became_peered":"2024-05-08T05:42:19.498663+0000","last_unstale":"2024-05-08T05:42:19.501230+0000","last_undegraded":"2024-05-08T05:42:19.501230+0000","last_fullsized":"2024-05-08T05:42:19.501230+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:26.053156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.481486+0000","last_change":"2024-05-08T05:42:19.481486+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.481486+0000","last_undegraded":"2024-05-08T05:42:19.481486+0000","last_fullsized":"2024-05-08T05:42:19.481486+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"36'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517320+0000","last_change":"2024-05-08T05:42:19.499917+0000","last_active":"2024-05-08T05:42:21.517320+0000","last_peered":"2024-05-08T05:42:21.517320+0000","last_clean":"2024-05-08T05:42:21.517320+0000","last_became_active":"2024-05-08T05:42:19.499614+0000","last_became_peered":"2024-05-08T05:42:19.499614+0000","last_unstale":"2024-05-08T05:42:21.517320+0000","last_undegraded":"2024-05-08T05:42:21.517320+0000","last_fullsized":"2024-05-08T05:42:21.517320+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:45.821917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569509+0000","last_change":"2024-05-08T05:42:20.529586+0000","last_active":"2024-05-08T05:42:23.569509+0000","last_peered":"2024-05-08T05:42:23.569509+0000","last_clean":"2024-05-08T05:42:23.569509+0000","last_became_active":"2024-05-08T05:42:20.529176+0000","last_became_peered":"2024-05-08T05:42:20.529176+0000","last_unstale":"2024-05-08T05:42:23.569509+0000","last_undegraded":"2024-05-08T05:42:23.569509+0000","last_fullsized":"2024-05-08T05:42:23.569509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:53.621950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569217+0000","last_change":"2024-05-08T05:42:19.492651+0000","last_active":"2024-05-08T05:42:23.569217+0000","last_peered":"2024-05-08T05:42:23.569217+0000","last_clean":"2024-05-08T05:42:23.569217+0000","last_became_active":"2024-05-08T05:42:19.491268+0000","last_became_peered":"2024-05-08T05:42:19.491268+0000","last_unstale":"2024-05-08T05:42:23.569217+0000","last_undegraded":"2024-05-08T05:42:23.569217+0000","last_fullsized":"2024-05-08T05:42:23.569217+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:29.265240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568690+0000","last_change":"2024-05-08T05:42:20.525312+0000","last_active":"2024-05-08T05:42:23.568690+0000","last_peered":"2024-05-08T05:42:23.568690+0000","last_clean":"2024-05-08T05:42:23.568690+0000","last_became_active":"2024-05-08T05:42:20.524640+0000","last_became_peered":"2024-05-08T05:42:20.524640+0000","last_unstale":"2024-05-08T05:42:23.568690+0000","last_undegraded":"2024-05-08T05:42:23.568690+0000","last_fullsized":"2024-05-08T05:42:23.568690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:18.431907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569450+0000","last_change":"2024-05-08T05:42:19.494149+0000","last_active":"2024-05-08T05:42:23.569450+0000","last_peered":"2024-05-08T05:42:23.569450+0000","last_clean":"2024-05-08T05:42:23.569450+0000","last_became_active":"2024-05-08T05:42:19.493848+0000","last_became_peered":"2024-05-08T05:42:19.493848+0000","last_unstale":"2024-05-08T05:42:23.569450+0000","last_undegraded":"2024-05-08T05:42:23.569450+0000","last_fullsized":"2024-05-08T05:42:23.569450+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:25.075141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569011+0000","last_change":"2024-05-08T05:42:20.518816+0000","last_active":"2024-05-08T05:42:23.569011+0000","last_peered":"2024-05-08T05:42:23.569011+0000","last_clean":"2024-05-08T05:42:23.569011+0000","last_became_active":"2024-05-08T05:42:20.518514+0000","last_became_peered":"2024-05-08T05:42:20.518514+0000","last_unstale":"2024-05-08T05:42:23.569011+0000","last_undegraded":"2024-05-08T05:42:23.569011+0000","last_fullsized":"2024-05-08T05:42:23.569011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:06.429659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.499479+0000","last_change":"2024-05-08T05:42:19.497480+0000","last_active":"2024-05-08T05:42:19.499479+0000","last_peered":"2024-05-08T05:42:19.499479+0000","last_clean":"2024-05-08T05:42:19.499479+0000","last_became_active":"2024-05-08T05:42:19.497177+0000","last_became_peered":"2024-05-08T05:42:19.497177+0000","last_unstale":"2024-05-08T05:42:19.499479+0000","last_undegraded":"2024-05-08T05:42:19.499479+0000","last_fullsized":"2024-05-08T05:42:19.499479+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:58:57.734024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568064+0000","last_change":"2024-05-08T05:42:20.530479+0000","last_active":"2024-05-08T05:42:23.568064+0000","last_peered":"2024-05-08T05:42:23.568064+0000","last_clean":"2024-05-08T05:42:23.568064+0000","last_became_active":"2024-05-08T05:42:20.530210+0000","last_became_peered":"2024-05-08T05:42:20.530210+0000","last_unstale":"2024-05-08T05:42:23.568064+0000","last_undegraded":"2024-05-08T05:42:23.568064+0000","last_fullsized":"2024-05-08T05:42:23.568064+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:58.041468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.567873+0000","last_change":"2024-05-08T05:42:19.495013+0000","last_active":"2024-05-08T05:42:23.567873+0000","last_peered":"2024-05-08T05:42:23.567873+0000","last_clean":"2024-05-08T05:42:23.567873+0000","last_became_active":"2024-05-08T05:42:19.494694+0000","last_became_peered":"2024-05-08T05:42:19.494694+0000","last_unstale":"2024-05-08T05:42:23.567873+0000","last_undegraded":"2024-05-08T05:42:23.567873+0000","last_fullsized":"2024-05-08T05:42:23.567873+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:29.265240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.483121+0000","last_change":"2024-05-08T05:42:19.483121+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.483121+0000","last_undegraded":"2024-05-08T05:42:19.483121+0000","last_fullsized":"2024-05-08T05:42:19.483121+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568473+0000","last_change":"2024-05-08T05:42:19.487951+0000","last_active":"2024-05-08T05:42:23.568473+0000","last_peered":"2024-05-08T05:42:23.568473+0000","last_clean":"2024-05-08T05:42:23.568473+0000","last_became_active":"2024-05-08T05:42:19.487613+0000","last_became_peered":"2024-05-08T05:42:19.487613+0000","last_unstale":"2024-05-08T05:42:23.568473+0000","last_undegraded":"2024-05-08T05:42:23.568473+0000","last_fullsized":"2024-05-08T05:42:23.568473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:57.039710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2024-05-08T05:42:19.489194+0000","last_change":"2024-05-08T05:42:19.489194+0000","last_active":"2024-05-08T05:42:19.471571+0000","last_peered":"2024-05-08T05:42:19.471571+0000","last_clean":"2024-05-08T05:42:19.471571+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:19.489194+0000","last_undegraded":"2024-05-08T05:42:19.489194+0000","last_fullsized":"2024-05-08T05:42:19.489194+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.489865+0000","last_change":"2024-05-08T05:42:19.487846+0000","last_active":"2024-05-08T05:42:19.489865+0000","last_peered":"2024-05-08T05:42:19.489865+0000","last_clean":"2024-05-08T05:42:19.489865+0000","last_became_active":"2024-05-08T05:42:19.487570+0000","last_became_peered":"2024-05-08T05:42:19.487570+0000","last_unstale":"2024-05-08T05:42:19.489865+0000","last_undegraded":"2024-05-08T05:42:19.489865+0000","last_fullsized":"2024-05-08T05:42:19.489865+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:25.075141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":135,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-05-08T05:42:19.483776+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:42:19.483776+0000","last_peered":"2024-05-08T05:42:19.483776+0000","last_clean":"2024-05-08T05:42:19.483776+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:42:19.483776+0000","last_undegraded":"2024-05-08T05:42:19.483776+0000","last_fullsized":"2024-05-08T05:42:19.483776+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1716,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":966656,"data_stored":744313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":372736,"data_stored":153945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542160,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":160612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":175075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":425984,"data_stored":176117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":413696,"data_stored":174541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":167869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":995328,"data_stored":758181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:42:29.057 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:42:29.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.174+0000 7f33b7264640 1 Processor -- start 2024-05-08T05:42:29.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.174+0000 7f33b7264640 1 -- start start 2024-05-08T05:42:29.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.175+0000 7f33b7264640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f33b0070d30 0x7f33b00d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.175+0000 7f33b7264640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f33b0163920 0x7f33b00703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.175+0000 7f33b7264640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33b00708e0 con 0x7f33b0071110 2024-05-08T05:42:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.175+0000 7f33b7264640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f33b00d10d0 con 0x7f33b0070d30 2024-05-08T05:42:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.175+0000 7f33b7264640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f33b0163db0 con 0x7f33b0163920 2024-05-08T05:42:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.175+0000 7f33b5000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f33b0070d30 0x7f33b00d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.175+0000 7f33b5000640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f33b0070d30 0x7f33b00d20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44274/0 (socket says 172.21.15.45:44274) 2024-05-08T05:42:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.175+0000 7f33b5000640 1 -- 172.21.15.45:0/1615391942 learned_addr learned my addr 172.21.15.45:0/1615391942 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.175+0000 7f33affff640 1 --2- 172.21.15.45:0/1615391942 >> v2:172.21.15.63:3300/0 conn(0x7f33b0163920 0x7f33b00703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.175+0000 7f33b5000640 1 -- 172.21.15.45:0/1615391942 >> v2:172.21.15.63:3300/0 conn(0x7f33b0163920 msgr2=0x7f33b00703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.175+0000 7f33b5000640 1 --2- 172.21.15.45:0/1615391942 >> v2:172.21.15.63:3300/0 conn(0x7f33b0163920 0x7f33b00703a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.175+0000 7f33b5000640 1 -- 172.21.15.45:0/1615391942 >> v1:172.21.15.63:6789/0 conn(0x7f33b0071110 legacy=0x7f33b006d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.175+0000 7f33b5000640 1 -- 172.21.15.45:0/1615391942 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b00d1300 con 0x7f33b0070d30 2024-05-08T05:42:29.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.176+0000 7f33b5000640 1 --2- 172.21.15.45:0/1615391942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33b0070d30 0x7f33b00d20c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f33b0073710 tx=0x7f33a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40c2ce866830ca5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.176+0000 7f33af7fe640 1 -- 172.21.15.45:0/1615391942 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4037070 con 0x7f33b0070d30 2024-05-08T05:42:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.176+0000 7f33af7fe640 1 -- 172.21.15.45:0/1615391942 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a4032440 con 0x7f33b0070d30 2024-05-08T05:42:29.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.176+0000 7f33af7fe640 1 -- 172.21.15.45:0/1615391942 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4032810 con 0x7f33b0070d30 2024-05-08T05:42:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.177+0000 7f33b7264640 1 -- 172.21.15.45:0/1615391942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33b0070d30 msgr2=0x7f33b00d20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.177+0000 7f33b7264640 1 --2- 172.21.15.45:0/1615391942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33b0070d30 0x7f33b00d20c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f33b0073710 tx=0x7f33a402a1e0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.177+0000 7f33b7264640 1 -- 172.21.15.45:0/1615391942 shutdown_connections 2024-05-08T05:42:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.177+0000 7f33b7264640 1 --2- 172.21.15.45:0/1615391942 >> v2:172.21.15.63:3300/0 conn(0x7f33b0163920 0x7f33b00703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.177+0000 7f33b7264640 1 --2- 172.21.15.45:0/1615391942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33b0070d30 0x7f33b00d20c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.177+0000 7f33b7264640 1 -- 172.21.15.45:0/1615391942 >> 172.21.15.45:0/1615391942 conn(0x7f33b00d2e90 msgr2=0x7f33b00d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.177+0000 7f33b7264640 1 -- 172.21.15.45:0/1615391942 shutdown_connections 2024-05-08T05:42:29.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.177+0000 7f33b7264640 1 -- 172.21.15.45:0/1615391942 wait complete. 2024-05-08T05:42:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.178+0000 7f33b7264640 1 Processor -- start 2024-05-08T05:42:29.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.178+0000 7f33b7264640 1 -- start start 2024-05-08T05:42:29.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.179+0000 7f33b7264640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33b0070d30 0x7f33b00b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:29.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.179+0000 7f33b7264640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33b0071110 0x7f33b00cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:29.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.179+0000 7f33b7264640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33b0163920 0x7f33b00cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:29.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.179+0000 7f33b7264640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f33b00d1c30 con 0x7f33b0163920 2024-05-08T05:42:29.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.179+0000 7f33b7264640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f33b0070b80 con 0x7f33b0070d30 2024-05-08T05:42:29.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.179+0000 7f33b7264640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f33b00d18c0 con 0x7f33b0071110 2024-05-08T05:42:29.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.179+0000 7f33b5801640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33b0163920 0x7f33b00cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:29.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.179+0000 7f33b5801640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33b0163920 0x7f33b00cd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44282/0 (socket says 172.21.15.45:44282) 2024-05-08T05:42:29.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.179+0000 7f33b5801640 1 -- 172.21.15.45:0/2780904965 learned_addr learned my addr 172.21.15.45:0/2780904965 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.179+0000 7f33b5000640 1 --2- 172.21.15.45:0/2780904965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33b0070d30 0x7f33b00b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.180+0000 7f33affff640 1 --2- 172.21.15.45:0/2780904965 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33b0071110 0x7f33b00cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.180+0000 7f33b5801640 1 -- 172.21.15.45:0/2780904965 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33b0071110 msgr2=0x7f33b00cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.180+0000 7f33b5801640 1 --2- 172.21.15.45:0/2780904965 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33b0071110 0x7f33b00cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.180+0000 7f33b5801640 1 -- 172.21.15.45:0/2780904965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33b0070d30 msgr2=0x7f33b00b72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.180+0000 7f33b5801640 1 --2- 172.21.15.45:0/2780904965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33b0070d30 0x7f33b00b72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.180+0000 7f33b5801640 1 -- 172.21.15.45:0/2780904965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b00cb460 con 0x7f33b0163920 2024-05-08T05:42:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.180+0000 7f33b5000640 1 --2- 172.21.15.45:0/2780904965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33b0070d30 0x7f33b00b72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.180+0000 7f33b5801640 1 --2- 172.21.15.45:0/2780904965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33b0163920 0x7f33b00cd4c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f33a800b9c0 tx=0x7f33a800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.180+0000 7f33ad7fa640 1 -- 172.21.15.45:0/2780904965 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8019070 con 0x7f33b0163920 2024-05-08T05:42:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.180+0000 7f33ad7fa640 1 -- 172.21.15.45:0/2780904965 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a800cba0 con 0x7f33b0163920 2024-05-08T05:42:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.180+0000 7f33ad7fa640 1 -- 172.21.15.45:0/2780904965 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8014440 con 0x7f33b0163920 2024-05-08T05:42:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.180+0000 7f33b7264640 1 -- 172.21.15.45:0/2780904965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b00cb720 con 0x7f33b0163920 2024-05-08T05:42:29.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.180+0000 7f33b7264640 1 -- 172.21.15.45:0/2780904965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b00cbce0 con 0x7f33b0163920 2024-05-08T05:42:29.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.181+0000 7f33b7264640 1 -- 172.21.15.45:0/2780904965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b00693f0 con 0x7f33b0163920 2024-05-08T05:42:29.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.182+0000 7f33ad7fa640 1 -- 172.21.15.45:0/2780904965 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f33a801c070 con 0x7f33b0163920 2024-05-08T05:42:29.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.183+0000 7f33ad7fa640 1 --2- 172.21.15.45:0/2780904965 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f33940762d0 0x7f3394078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:29.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.183+0000 7f33ad7fa640 1 -- 172.21.15.45:0/2780904965 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f33a8097440 con 0x7f33b0163920 2024-05-08T05:42:29.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.183+0000 7f33b5000640 1 --2- 172.21.15.45:0/2780904965 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f33940762d0 0x7f3394078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:29.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.183+0000 7f33b5000640 1 --2- 172.21.15.45:0/2780904965 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f33940762d0 0x7f3394078790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f33a4034000 tx=0x7f33a402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:29.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.187+0000 7f33ad7fa640 1 -- 172.21.15.45:0/2780904965 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f33a8060ca0 con 0x7f33b0163920 2024-05-08T05:42:29.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.376+0000 7f33b7264640 1 -- 172.21.15.45:0/2780904965 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f33b00d3d70 con 0x7f33940762d0 2024-05-08T05:42:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.383+0000 7f33ad7fa640 1 -- 172.21.15.45:0/2780904965 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251945 (secure 0 0 0) 0x7f33a407f840 con 0x7f33940762d0 2024-05-08T05:42:29.386 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:29.387 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:42:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.386+0000 7f33b7264640 1 -- 172.21.15.45:0/2780904965 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f33940762d0 msgr2=0x7f3394078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.386+0000 7f33b7264640 1 --2- 172.21.15.45:0/2780904965 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f33940762d0 0x7f3394078790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f33a4034000 tx=0x7f33a402e000 comp rx=0 tx=0).stop 2024-05-08T05:42:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.386+0000 7f33b7264640 1 -- 172.21.15.45:0/2780904965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33b0163920 msgr2=0x7f33b00cd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.386+0000 7f33b7264640 1 --2- 172.21.15.45:0/2780904965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33b0163920 0x7f33b00cd4c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f33a800b9c0 tx=0x7f33a800be80 comp rx=0 tx=0).stop 2024-05-08T05:42:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.386+0000 7f33b7264640 1 -- 172.21.15.45:0/2780904965 shutdown_connections 2024-05-08T05:42:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.386+0000 7f33b7264640 1 --2- 172.21.15.45:0/2780904965 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f33940762d0 0x7f3394078790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.386+0000 7f33b7264640 1 --2- 172.21.15.45:0/2780904965 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f33b0071110 0x7f33b00cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.386+0000 7f33b7264640 1 --2- 172.21.15.45:0/2780904965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33b0163920 0x7f33b00cd4c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.386+0000 7f33b7264640 1 --2- 172.21.15.45:0/2780904965 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f33b0070d30 0x7f33b00b72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.386+0000 7f33b7264640 1 -- 172.21.15.45:0/2780904965 >> 172.21.15.45:0/2780904965 conn(0x7f33b00d2e90 msgr2=0x7f33b0161130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.386+0000 7f33b7264640 1 -- 172.21.15.45:0/2780904965 shutdown_connections 2024-05-08T05:42:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.386+0000 7f33b7264640 1 -- 172.21.15.45:0/2780904965 wait complete. 2024-05-08T05:42:29.397 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-05-08T05:42:28.846398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237216,"kb_used_data":4400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737504,"statfs":{"total":773094113280,"available":772851204096,"internally_reserved":0,"allocated":4505600,"data_stored":2567940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.344396"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569700+0000","last_change":"2024-05-08T05:42:20.521354+0000","last_active":"2024-05-08T05:42:23.569700+0000","last_peered":"2024-05-08T05:42:23.569700+0000","last_clean":"2024-05-08T05:42:23.569700+0000","last_became_active":"2024-05-08T05:42:20.521070+0000","last_became_peered":"2024-05-08T05:42:20.521070+0000","last_unstale":"2024-05-08T05:42:23.569700+0000","last_undegraded":"2024-05-08T05:42:23.569700+0000","last_fullsized":"2024-05-08T05:42:23.569700+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:08.311556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568750+0000","last_change":"2024-05-08T05:42:20.525189+0000","last_active":"2024-05-08T05:42:23.568750+0000","last_peered":"2024-05-08T05:42:23.568750+0000","last_clean":"2024-05-08T05:42:23.568750+0000","last_became_active":"2024-05-08T05:42:20.524906+0000","last_became_peered":"2024-05-08T05:42:20.524906+0000","last_unstale":"2024-05-08T05:42:23.568750+0000","last_undegraded":"2024-05-08T05:42:23.568750+0000","last_fullsized":"2024-05-08T05:42:23.568750+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:22:33.614866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569160+0000","last_change":"2024-05-08T05:42:20.532374+0000","last_active":"2024-05-08T05:42:23.569160+0000","last_peered":"2024-05-08T05:42:23.569160+0000","last_clean":"2024-05-08T05:42:23.569160+0000","last_became_active":"2024-05-08T05:42:20.532079+0000","last_became_peered":"2024-05-08T05:42:20.532079+0000","last_unstale":"2024-05-08T05:42:23.569160+0000","last_undegraded":"2024-05-08T05:42:23.569160+0000","last_fullsized":"2024-05-08T05:42:23.569160+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:23.089039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517311+0000","last_change":"2024-05-08T05:42:20.530407+0000","last_active":"2024-05-08T05:42:21.517311+0000","last_peered":"2024-05-08T05:42:21.517311+0000","last_clean":"2024-05-08T05:42:21.517311+0000","last_became_active":"2024-05-08T05:42:20.530148+0000","last_became_peered":"2024-05-08T05:42:20.530148+0000","last_unstale":"2024-05-08T05:42:21.517311+0000","last_undegraded":"2024-05-08T05:42:21.517311+0000","last_fullsized":"2024-05-08T05:42:21.517311+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:30.266997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517173+0000","last_change":"2024-05-08T05:42:20.528145+0000","last_active":"2024-05-08T05:42:21.517173+0000","last_peered":"2024-05-08T05:42:21.517173+0000","last_clean":"2024-05-08T05:42:21.517173+0000","last_became_active":"2024-05-08T05:42:20.527841+0000","last_became_peered":"2024-05-08T05:42:20.527841+0000","last_unstale":"2024-05-08T05:42:21.517173+0000","last_undegraded":"2024-05-08T05:42:21.517173+0000","last_fullsized":"2024-05-08T05:42:21.517173+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:15.132265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569078+0000","last_change":"2024-05-08T05:42:20.518897+0000","last_active":"2024-05-08T05:42:23.569078+0000","last_peered":"2024-05-08T05:42:23.569078+0000","last_clean":"2024-05-08T05:42:23.569078+0000","last_became_active":"2024-05-08T05:42:20.518613+0000","last_became_peered":"2024-05-08T05:42:20.518613+0000","last_unstale":"2024-05-08T05:42:23.569078+0000","last_undegraded":"2024-05-08T05:42:23.569078+0000","last_fullsized":"2024-05-08T05:42:23.569078+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:23.089039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:22.509168+0000","last_change":"2024-05-08T05:42:20.511405+0000","last_active":"2024-05-08T05:42:22.509168+0000","last_peered":"2024-05-08T05:42:22.509168+0000","last_clean":"2024-05-08T05:42:22.509168+0000","last_became_active":"2024-05-08T05:42:20.509946+0000","last_became_peered":"2024-05-08T05:42:20.509946+0000","last_unstale":"2024-05-08T05:42:22.509168+0000","last_undegraded":"2024-05-08T05:42:22.509168+0000","last_fullsized":"2024-05-08T05:42:22.509168+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:54:21.060834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568409+0000","last_change":"2024-05-08T05:42:20.515418+0000","last_active":"2024-05-08T05:42:23.568409+0000","last_peered":"2024-05-08T05:42:23.568409+0000","last_clean":"2024-05-08T05:42:23.568409+0000","last_became_active":"2024-05-08T05:42:20.514961+0000","last_became_peered":"2024-05-08T05:42:20.514961+0000","last_unstale":"2024-05-08T05:42:23.568409+0000","last_undegraded":"2024-05-08T05:42:23.568409+0000","last_fullsized":"2024-05-08T05:42:23.568409+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:54:21.060834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569512+0000","last_change":"2024-05-08T05:42:20.533002+0000","last_active":"2024-05-08T05:42:23.569512+0000","last_peered":"2024-05-08T05:42:23.569512+0000","last_clean":"2024-05-08T05:42:23.569512+0000","last_became_active":"2024-05-08T05:42:20.532713+0000","last_became_peered":"2024-05-08T05:42:20.532713+0000","last_unstale":"2024-05-08T05:42:23.569512+0000","last_undegraded":"2024-05-08T05:42:23.569512+0000","last_fullsized":"2024-05-08T05:42:23.569512+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:34:08.047335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:22.509750+0000","last_change":"2024-05-08T05:42:20.516489+0000","last_active":"2024-05-08T05:42:22.509750+0000","last_peered":"2024-05-08T05:42:22.509750+0000","last_clean":"2024-05-08T05:42:22.509750+0000","last_became_active":"2024-05-08T05:42:20.516204+0000","last_became_peered":"2024-05-08T05:42:20.516204+0000","last_unstale":"2024-05-08T05:42:22.509750+0000","last_undegraded":"2024-05-08T05:42:22.509750+0000","last_fullsized":"2024-05-08T05:42:22.509750+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:08.311556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:22.509742+0000","last_change":"2024-05-08T05:42:20.514788+0000","last_active":"2024-05-08T05:42:22.509742+0000","last_peered":"2024-05-08T05:42:22.509742+0000","last_clean":"2024-05-08T05:42:22.509742+0000","last_became_active":"2024-05-08T05:42:20.514520+0000","last_became_peered":"2024-05-08T05:42:20.514520+0000","last_unstale":"2024-05-08T05:42:22.509742+0000","last_undegraded":"2024-05-08T05:42:22.509742+0000","last_fullsized":"2024-05-08T05:42:22.509742+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:33:26.883953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568992+0000","last_change":"2024-05-08T05:42:20.527823+0000","last_active":"2024-05-08T05:42:23.568992+0000","last_peered":"2024-05-08T05:42:23.568992+0000","last_clean":"2024-05-08T05:42:23.568992+0000","last_became_active":"2024-05-08T05:42:20.527531+0000","last_became_peered":"2024-05-08T05:42:20.527531+0000","last_unstale":"2024-05-08T05:42:23.568992+0000","last_undegraded":"2024-05-08T05:42:23.568992+0000","last_fullsized":"2024-05-08T05:42:23.568992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:06.429659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:22.509532+0000","last_change":"2024-05-08T05:42:20.517054+0000","last_active":"2024-05-08T05:42:22.509532+0000","last_peered":"2024-05-08T05:42:22.509532+0000","last_clean":"2024-05-08T05:42:22.509532+0000","last_became_active":"2024-05-08T05:42:20.516735+0000","last_became_peered":"2024-05-08T05:42:20.516735+0000","last_unstale":"2024-05-08T05:42:22.509532+0000","last_undegraded":"2024-05-08T05:42:22.509532+0000","last_fullsized":"2024-05-08T05:42:22.509532+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:53.621950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568056+0000","last_change":"2024-05-08T05:42:20.520112+0000","last_active":"2024-05-08T05:42:23.568056+0000","last_peered":"2024-05-08T05:42:23.568056+0000","last_clean":"2024-05-08T05:42:23.568056+0000","last_became_active":"2024-05-08T05:42:20.519660+0000","last_became_peered":"2024-05-08T05:42:20.519660+0000","last_unstale":"2024-05-08T05:42:23.568056+0000","last_undegraded":"2024-05-08T05:42:23.568056+0000","last_fullsized":"2024-05-08T05:42:23.568056+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:25:23.199079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:22.509211+0000","last_change":"2024-05-08T05:42:20.530131+0000","last_active":"2024-05-08T05:42:22.509211+0000","last_peered":"2024-05-08T05:42:22.509211+0000","last_clean":"2024-05-08T05:42:22.509211+0000","last_became_active":"2024-05-08T05:42:20.529843+0000","last_became_peered":"2024-05-08T05:42:20.529843+0000","last_unstale":"2024-05-08T05:42:22.509211+0000","last_undegraded":"2024-05-08T05:42:22.509211+0000","last_fullsized":"2024-05-08T05:42:22.509211+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:22:33.614866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.570623+0000","last_change":"2024-05-08T05:42:20.518516+0000","last_active":"2024-05-08T05:42:23.570623+0000","last_peered":"2024-05-08T05:42:23.570623+0000","last_clean":"2024-05-08T05:42:23.570623+0000","last_became_active":"2024-05-08T05:42:20.518140+0000","last_became_peered":"2024-05-08T05:42:20.518140+0000","last_unstale":"2024-05-08T05:42:23.570623+0000","last_undegraded":"2024-05-08T05:42:23.570623+0000","last_fullsized":"2024-05-08T05:42:23.570623+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:09.216819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.571257+0000","last_change":"2024-05-08T05:42:20.525872+0000","last_active":"2024-05-08T05:42:23.571257+0000","last_peered":"2024-05-08T05:42:23.571257+0000","last_clean":"2024-05-08T05:42:23.571257+0000","last_became_active":"2024-05-08T05:42:20.525588+0000","last_became_peered":"2024-05-08T05:42:20.525588+0000","last_unstale":"2024-05-08T05:42:23.571257+0000","last_undegraded":"2024-05-08T05:42:23.571257+0000","last_fullsized":"2024-05-08T05:42:23.571257+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:44.462590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.518775+0000","last_change":"2024-05-08T05:42:20.521346+0000","last_active":"2024-05-08T05:42:21.518775+0000","last_peered":"2024-05-08T05:42:21.518775+0000","last_clean":"2024-05-08T05:42:21.518775+0000","last_became_active":"2024-05-08T05:42:20.521045+0000","last_became_peered":"2024-05-08T05:42:20.521045+0000","last_unstale":"2024-05-08T05:42:21.518775+0000","last_undegraded":"2024-05-08T05:42:21.518775+0000","last_fullsized":"2024-05-08T05:42:21.518775+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:38.043321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517514+0000","last_change":"2024-05-08T05:42:20.533970+0000","last_active":"2024-05-08T05:42:21.517514+0000","last_peered":"2024-05-08T05:42:21.517514+0000","last_clean":"2024-05-08T05:42:21.517514+0000","last_became_active":"2024-05-08T05:42:20.531975+0000","last_became_peered":"2024-05-08T05:42:20.531975+0000","last_unstale":"2024-05-08T05:42:21.517514+0000","last_undegraded":"2024-05-08T05:42:21.517514+0000","last_fullsized":"2024-05-08T05:42:21.517514+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:20:52.733574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.518794+0000","last_change":"2024-05-08T05:42:20.527256+0000","last_active":"2024-05-08T05:42:21.518794+0000","last_peered":"2024-05-08T05:42:21.518794+0000","last_clean":"2024-05-08T05:42:21.518794+0000","last_became_active":"2024-05-08T05:42:20.524099+0000","last_became_peered":"2024-05-08T05:42:20.524099+0000","last_unstale":"2024-05-08T05:42:21.518794+0000","last_undegraded":"2024-05-08T05:42:21.518794+0000","last_fullsized":"2024-05-08T05:42:21.518794+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:25:23.199079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:22.509404+0000","last_change":"2024-05-08T05:42:20.509620+0000","last_active":"2024-05-08T05:42:22.509404+0000","last_peered":"2024-05-08T05:42:22.509404+0000","last_clean":"2024-05-08T05:42:22.509404+0000","last_became_active":"2024-05-08T05:42:20.509287+0000","last_became_peered":"2024-05-08T05:42:20.509287+0000","last_unstale":"2024-05-08T05:42:22.509404+0000","last_undegraded":"2024-05-08T05:42:22.509404+0000","last_fullsized":"2024-05-08T05:42:22.509404+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:43:42.826524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568464+0000","last_change":"2024-05-08T05:42:20.525789+0000","last_active":"2024-05-08T05:42:23.568464+0000","last_peered":"2024-05-08T05:42:23.568464+0000","last_clean":"2024-05-08T05:42:23.568464+0000","last_became_active":"2024-05-08T05:42:20.525492+0000","last_became_peered":"2024-05-08T05:42:20.525492+0000","last_unstale":"2024-05-08T05:42:23.568464+0000","last_undegraded":"2024-05-08T05:42:23.568464+0000","last_fullsized":"2024-05-08T05:42:23.568464+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:44.235500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.570742+0000","last_change":"2024-05-08T05:42:20.530885+0000","last_active":"2024-05-08T05:42:23.570742+0000","last_peered":"2024-05-08T05:42:23.570742+0000","last_clean":"2024-05-08T05:42:23.570742+0000","last_became_active":"2024-05-08T05:42:20.530595+0000","last_became_peered":"2024-05-08T05:42:20.530595+0000","last_unstale":"2024-05-08T05:42:23.570742+0000","last_undegraded":"2024-05-08T05:42:23.570742+0000","last_fullsized":"2024-05-08T05:42:23.570742+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:49:12.236778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517037+0000","last_change":"2024-05-08T05:42:20.534832+0000","last_active":"2024-05-08T05:42:21.517037+0000","last_peered":"2024-05-08T05:42:21.517037+0000","last_clean":"2024-05-08T05:42:21.517037+0000","last_became_active":"2024-05-08T05:42:20.534586+0000","last_became_peered":"2024-05-08T05:42:20.534586+0000","last_unstale":"2024-05-08T05:42:21.517037+0000","last_undegraded":"2024-05-08T05:42:21.517037+0000","last_fullsized":"2024-05-08T05:42:21.517037+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:30.319294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.518812+0000","last_change":"2024-05-08T05:42:20.525101+0000","last_active":"2024-05-08T05:42:21.518812+0000","last_peered":"2024-05-08T05:42:21.518812+0000","last_clean":"2024-05-08T05:42:21.518812+0000","last_became_active":"2024-05-08T05:42:20.524735+0000","last_became_peered":"2024-05-08T05:42:20.524735+0000","last_unstale":"2024-05-08T05:42:21.518812+0000","last_undegraded":"2024-05-08T05:42:21.518812+0000","last_fullsized":"2024-05-08T05:42:21.518812+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:58.041468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569431+0000","last_change":"2024-05-08T05:42:20.509600+0000","last_active":"2024-05-08T05:42:23.569431+0000","last_peered":"2024-05-08T05:42:23.569431+0000","last_clean":"2024-05-08T05:42:23.569431+0000","last_became_active":"2024-05-08T05:42:20.509045+0000","last_became_peered":"2024-05-08T05:42:20.509045+0000","last_unstale":"2024-05-08T05:42:23.569431+0000","last_undegraded":"2024-05-08T05:42:23.569431+0000","last_fullsized":"2024-05-08T05:42:23.569431+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:39.499482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.519004+0000","last_change":"2024-05-08T05:42:20.526215+0000","last_active":"2024-05-08T05:42:21.519004+0000","last_peered":"2024-05-08T05:42:21.519004+0000","last_clean":"2024-05-08T05:42:21.519004+0000","last_became_active":"2024-05-08T05:42:20.525892+0000","last_became_peered":"2024-05-08T05:42:20.525892+0000","last_unstale":"2024-05-08T05:42:21.519004+0000","last_undegraded":"2024-05-08T05:42:21.519004+0000","last_fullsized":"2024-05-08T05:42:21.519004+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:23.447125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:22.509504+0000","last_change":"2024-05-08T05:42:20.528975+0000","last_active":"2024-05-08T05:42:22.509504+0000","last_peered":"2024-05-08T05:42:22.509504+0000","last_clean":"2024-05-08T05:42:22.509504+0000","last_became_active":"2024-05-08T05:42:20.527765+0000","last_became_peered":"2024-05-08T05:42:20.527765+0000","last_unstale":"2024-05-08T05:42:22.509504+0000","last_undegraded":"2024-05-08T05:42:22.509504+0000","last_fullsized":"2024-05-08T05:42:22.509504+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:01.706249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.518735+0000","last_change":"2024-05-08T05:42:20.524106+0000","last_active":"2024-05-08T05:42:21.518735+0000","last_peered":"2024-05-08T05:42:21.518735+0000","last_clean":"2024-05-08T05:42:21.518735+0000","last_became_active":"2024-05-08T05:42:20.523665+0000","last_became_peered":"2024-05-08T05:42:20.523665+0000","last_unstale":"2024-05-08T05:42:21.518735+0000","last_undegraded":"2024-05-08T05:42:21.518735+0000","last_fullsized":"2024-05-08T05:42:21.518735+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:28:35.312605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.518540+0000","last_change":"2024-05-08T05:42:20.530462+0000","last_active":"2024-05-08T05:42:21.518540+0000","last_peered":"2024-05-08T05:42:21.518540+0000","last_clean":"2024-05-08T05:42:21.518540+0000","last_became_active":"2024-05-08T05:42:20.530169+0000","last_became_peered":"2024-05-08T05:42:20.530169+0000","last_unstale":"2024-05-08T05:42:21.518540+0000","last_undegraded":"2024-05-08T05:42:21.518540+0000","last_fullsized":"2024-05-08T05:42:21.518540+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:53:51.779155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.570907+0000","last_change":"2024-05-08T05:42:20.530228+0000","last_active":"2024-05-08T05:42:23.570907+0000","last_peered":"2024-05-08T05:42:23.570907+0000","last_clean":"2024-05-08T05:42:23.570907+0000","last_became_active":"2024-05-08T05:42:20.529972+0000","last_became_peered":"2024-05-08T05:42:20.529972+0000","last_unstale":"2024-05-08T05:42:23.570907+0000","last_undegraded":"2024-05-08T05:42:23.570907+0000","last_fullsized":"2024-05-08T05:42:23.570907+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:26.781364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569182+0000","last_change":"2024-05-08T05:42:20.524631+0000","last_active":"2024-05-08T05:42:23.569182+0000","last_peered":"2024-05-08T05:42:23.569182+0000","last_clean":"2024-05-08T05:42:23.569182+0000","last_became_active":"2024-05-08T05:42:20.524354+0000","last_became_peered":"2024-05-08T05:42:20.524354+0000","last_unstale":"2024-05-08T05:42:23.569182+0000","last_undegraded":"2024-05-08T05:42:23.569182+0000","last_fullsized":"2024-05-08T05:42:23.569182+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:34:08.047335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.571117+0000","last_change":"2024-05-08T05:42:20.515896+0000","last_active":"2024-05-08T05:42:23.571117+0000","last_peered":"2024-05-08T05:42:23.571117+0000","last_clean":"2024-05-08T05:42:23.571117+0000","last_became_active":"2024-05-08T05:42:20.515364+0000","last_became_peered":"2024-05-08T05:42:20.515364+0000","last_unstale":"2024-05-08T05:42:23.571117+0000","last_undegraded":"2024-05-08T05:42:23.571117+0000","last_fullsized":"2024-05-08T05:42:23.571117+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:30:06.045575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'4","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.518908+0000","last_change":"2024-05-08T05:42:19.493864+0000","last_active":"2024-05-08T05:42:21.518908+0000","last_peered":"2024-05-08T05:42:21.518908+0000","last_clean":"2024-05-08T05:42:21.518908+0000","last_became_active":"2024-05-08T05:42:19.493061+0000","last_became_peered":"2024-05-08T05:42:19.493061+0000","last_unstale":"2024-05-08T05:42:21.518908+0000","last_undegraded":"2024-05-08T05:42:21.518908+0000","last_fullsized":"2024-05-08T05:42:21.518908+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:29.317536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569291+0000","last_change":"2024-05-08T05:42:20.512807+0000","last_active":"2024-05-08T05:42:23.569291+0000","last_peered":"2024-05-08T05:42:23.569291+0000","last_clean":"2024-05-08T05:42:23.569291+0000","last_became_active":"2024-05-08T05:42:20.512501+0000","last_became_peered":"2024-05-08T05:42:20.512501+0000","last_unstale":"2024-05-08T05:42:23.569291+0000","last_undegraded":"2024-05-08T05:42:23.569291+0000","last_fullsized":"2024-05-08T05:42:23.569291+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:32:33.781763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569814+0000","last_change":"2024-05-08T05:42:19.493675+0000","last_active":"2024-05-08T05:42:23.569814+0000","last_peered":"2024-05-08T05:42:23.569814+0000","last_clean":"2024-05-08T05:42:23.569814+0000","last_became_active":"2024-05-08T05:42:19.493339+0000","last_became_peered":"2024-05-08T05:42:19.493339+0000","last_unstale":"2024-05-08T05:42:23.569814+0000","last_undegraded":"2024-05-08T05:42:23.569814+0000","last_fullsized":"2024-05-08T05:42:23.569814+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:51:24.291839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568501+0000","last_change":"2024-05-08T05:42:20.521416+0000","last_active":"2024-05-08T05:42:23.568501+0000","last_peered":"2024-05-08T05:42:23.568501+0000","last_clean":"2024-05-08T05:42:23.568501+0000","last_became_active":"2024-05-08T05:42:20.520422+0000","last_became_peered":"2024-05-08T05:42:20.520422+0000","last_unstale":"2024-05-08T05:42:23.568501+0000","last_undegraded":"2024-05-08T05:42:23.568501+0000","last_fullsized":"2024-05-08T05:42:23.568501+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:18.431907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569440+0000","last_change":"2024-05-08T05:42:19.491376+0000","last_active":"2024-05-08T05:42:23.569440+0000","last_peered":"2024-05-08T05:42:23.569440+0000","last_clean":"2024-05-08T05:42:23.569440+0000","last_became_active":"2024-05-08T05:42:19.491075+0000","last_became_peered":"2024-05-08T05:42:19.491075+0000","last_unstale":"2024-05-08T05:42:23.569440+0000","last_undegraded":"2024-05-08T05:42:23.569440+0000","last_fullsized":"2024-05-08T05:42:23.569440+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:28:34.310847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569384+0000","last_change":"2024-05-08T05:42:20.529711+0000","last_active":"2024-05-08T05:42:23.569384+0000","last_peered":"2024-05-08T05:42:23.569384+0000","last_clean":"2024-05-08T05:42:23.569384+0000","last_became_active":"2024-05-08T05:42:20.529354+0000","last_became_peered":"2024-05-08T05:42:20.529354+0000","last_unstale":"2024-05-08T05:42:23.569384+0000","last_undegraded":"2024-05-08T05:42:23.569384+0000","last_fullsized":"2024-05-08T05:42:23.569384+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:43:42.826524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568253+0000","last_change":"2024-05-08T05:42:19.495593+0000","last_active":"2024-05-08T05:42:23.568253+0000","last_peered":"2024-05-08T05:42:23.568253+0000","last_clean":"2024-05-08T05:42:23.568253+0000","last_became_active":"2024-05-08T05:42:19.495297+0000","last_became_peered":"2024-05-08T05:42:19.495297+0000","last_unstale":"2024-05-08T05:42:23.568253+0000","last_undegraded":"2024-05-08T05:42:23.568253+0000","last_fullsized":"2024-05-08T05:42:23.568253+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:25:29.317536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568467+0000","last_change":"2024-05-08T05:42:20.525554+0000","last_active":"2024-05-08T05:42:23.568467+0000","last_peered":"2024-05-08T05:42:23.568467+0000","last_clean":"2024-05-08T05:42:23.568467+0000","last_became_active":"2024-05-08T05:42:20.525027+0000","last_became_peered":"2024-05-08T05:42:20.525027+0000","last_unstale":"2024-05-08T05:42:23.568467+0000","last_undegraded":"2024-05-08T05:42:23.568467+0000","last_fullsized":"2024-05-08T05:42:23.568467+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:44:05.274392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:22.509552+0000","last_change":"2024-05-08T05:42:19.499533+0000","last_active":"2024-05-08T05:42:22.509552+0000","last_peered":"2024-05-08T05:42:22.509552+0000","last_clean":"2024-05-08T05:42:22.509552+0000","last_became_active":"2024-05-08T05:42:19.498447+0000","last_became_peered":"2024-05-08T05:42:19.498447+0000","last_unstale":"2024-05-08T05:42:22.509552+0000","last_undegraded":"2024-05-08T05:42:22.509552+0000","last_fullsized":"2024-05-08T05:42:22.509552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:25:22.197321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:22.509468+0000","last_change":"2024-05-08T05:42:20.522929+0000","last_active":"2024-05-08T05:42:22.509468+0000","last_peered":"2024-05-08T05:42:22.509468+0000","last_clean":"2024-05-08T05:42:22.509468+0000","last_became_active":"2024-05-08T05:42:20.522215+0000","last_became_peered":"2024-05-08T05:42:20.522215+0000","last_unstale":"2024-05-08T05:42:22.509468+0000","last_undegraded":"2024-05-08T05:42:22.509468+0000","last_fullsized":"2024-05-08T05:42:22.509468+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:44.462590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.570739+0000","last_change":"2024-05-08T05:42:19.495816+0000","last_active":"2024-05-08T05:42:23.570739+0000","last_peered":"2024-05-08T05:42:23.570739+0000","last_clean":"2024-05-08T05:42:23.570739+0000","last_became_active":"2024-05-08T05:42:19.495546+0000","last_became_peered":"2024-05-08T05:42:19.495546+0000","last_unstale":"2024-05-08T05:42:23.570739+0000","last_undegraded":"2024-05-08T05:42:23.570739+0000","last_fullsized":"2024-05-08T05:42:23.570739+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:28:34.310847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568299+0000","last_change":"2024-05-08T05:42:20.512851+0000","last_active":"2024-05-08T05:42:23.568299+0000","last_peered":"2024-05-08T05:42:23.568299+0000","last_clean":"2024-05-08T05:42:23.568299+0000","last_became_active":"2024-05-08T05:42:20.512412+0000","last_became_peered":"2024-05-08T05:42:20.512412+0000","last_unstale":"2024-05-08T05:42:23.568299+0000","last_undegraded":"2024-05-08T05:42:23.568299+0000","last_fullsized":"2024-05-08T05:42:23.568299+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:23.447125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569070+0000","last_change":"2024-05-08T05:42:19.499539+0000","last_active":"2024-05-08T05:42:23.569070+0000","last_peered":"2024-05-08T05:42:23.569070+0000","last_clean":"2024-05-08T05:42:23.569070+0000","last_became_active":"2024-05-08T05:42:19.499241+0000","last_became_peered":"2024-05-08T05:42:19.499241+0000","last_unstale":"2024-05-08T05:42:23.569070+0000","last_undegraded":"2024-05-08T05:42:23.569070+0000","last_fullsized":"2024-05-08T05:42:23.569070+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:22.668414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569402+0000","last_change":"2024-05-08T05:42:20.533418+0000","last_active":"2024-05-08T05:42:23.569402+0000","last_peered":"2024-05-08T05:42:23.569402+0000","last_clean":"2024-05-08T05:42:23.569402+0000","last_became_active":"2024-05-08T05:42:20.532766+0000","last_became_peered":"2024-05-08T05:42:20.532766+0000","last_unstale":"2024-05-08T05:42:23.569402+0000","last_undegraded":"2024-05-08T05:42:23.569402+0000","last_fullsized":"2024-05-08T05:42:23.569402+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:44.462590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.567873+0000","last_change":"2024-05-08T05:42:19.491786+0000","last_active":"2024-05-08T05:42:23.567873+0000","last_peered":"2024-05-08T05:42:23.567873+0000","last_clean":"2024-05-08T05:42:23.567873+0000","last_became_active":"2024-05-08T05:42:19.491455+0000","last_became_peered":"2024-05-08T05:42:19.491455+0000","last_unstale":"2024-05-08T05:42:23.567873+0000","last_undegraded":"2024-05-08T05:42:23.567873+0000","last_fullsized":"2024-05-08T05:42:23.567873+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:51:24.291839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517387+0000","last_change":"2024-05-08T05:42:20.529431+0000","last_active":"2024-05-08T05:42:21.517387+0000","last_peered":"2024-05-08T05:42:21.517387+0000","last_clean":"2024-05-08T05:42:21.517387+0000","last_became_active":"2024-05-08T05:42:20.529130+0000","last_became_peered":"2024-05-08T05:42:20.529130+0000","last_unstale":"2024-05-08T05:42:21.517387+0000","last_undegraded":"2024-05-08T05:42:21.517387+0000","last_fullsized":"2024-05-08T05:42:21.517387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:51:25.293597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569722+0000","last_change":"2024-05-08T05:42:19.487759+0000","last_active":"2024-05-08T05:42:23.569722+0000","last_peered":"2024-05-08T05:42:23.569722+0000","last_clean":"2024-05-08T05:42:23.569722+0000","last_became_active":"2024-05-08T05:42:19.487492+0000","last_became_peered":"2024-05-08T05:42:19.487492+0000","last_unstale":"2024-05-08T05:42:23.569722+0000","last_undegraded":"2024-05-08T05:42:23.569722+0000","last_fullsized":"2024-05-08T05:42:23.569722+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:22.445367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.570998+0000","last_change":"2024-05-08T05:42:20.528756+0000","last_active":"2024-05-08T05:42:23.570998+0000","last_peered":"2024-05-08T05:42:23.570998+0000","last_clean":"2024-05-08T05:42:23.570998+0000","last_became_active":"2024-05-08T05:42:20.528177+0000","last_became_peered":"2024-05-08T05:42:20.528177+0000","last_unstale":"2024-05-08T05:42:23.570998+0000","last_undegraded":"2024-05-08T05:42:23.570998+0000","last_fullsized":"2024-05-08T05:42:23.570998+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:54:21.060834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:22.509896+0000","last_change":"2024-05-08T05:42:19.498339+0000","last_active":"2024-05-08T05:42:22.509896+0000","last_peered":"2024-05-08T05:42:22.509896+0000","last_clean":"2024-05-08T05:42:22.509896+0000","last_became_active":"2024-05-08T05:42:19.498000+0000","last_became_peered":"2024-05-08T05:42:19.498000+0000","last_unstale":"2024-05-08T05:42:22.509896+0000","last_undegraded":"2024-05-08T05:42:22.509896+0000","last_fullsized":"2024-05-08T05:42:22.509896+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:22.445367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569759+0000","last_change":"2024-05-08T05:42:20.525378+0000","last_active":"2024-05-08T05:42:23.569759+0000","last_peered":"2024-05-08T05:42:23.569759+0000","last_clean":"2024-05-08T05:42:23.569759+0000","last_became_active":"2024-05-08T05:42:20.525089+0000","last_became_peered":"2024-05-08T05:42:20.525089+0000","last_unstale":"2024-05-08T05:42:23.569759+0000","last_undegraded":"2024-05-08T05:42:23.569759+0000","last_fullsized":"2024-05-08T05:42:23.569759+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:32:33.781763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"35'3","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:22.509814+0000","last_change":"2024-05-08T05:42:19.499267+0000","last_active":"2024-05-08T05:42:22.509814+0000","last_peered":"2024-05-08T05:42:22.509814+0000","last_clean":"2024-05-08T05:42:22.509814+0000","last_became_active":"2024-05-08T05:42:19.498537+0000","last_became_peered":"2024-05-08T05:42:19.498537+0000","last_unstale":"2024-05-08T05:42:22.509814+0000","last_undegraded":"2024-05-08T05:42:22.509814+0000","last_fullsized":"2024-05-08T05:42:22.509814+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:57.039710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568542+0000","last_change":"2024-05-08T05:42:20.525088+0000","last_active":"2024-05-08T05:42:23.568542+0000","last_peered":"2024-05-08T05:42:23.568542+0000","last_clean":"2024-05-08T05:42:23.568542+0000","last_became_active":"2024-05-08T05:42:20.524526+0000","last_became_peered":"2024-05-08T05:42:20.524526+0000","last_unstale":"2024-05-08T05:42:23.568542+0000","last_undegraded":"2024-05-08T05:42:23.568542+0000","last_fullsized":"2024-05-08T05:42:23.568542+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:06.429659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569781+0000","last_change":"2024-05-08T05:42:19.488838+0000","last_active":"2024-05-08T05:42:23.569781+0000","last_peered":"2024-05-08T05:42:23.569781+0000","last_clean":"2024-05-08T05:42:23.569781+0000","last_became_active":"2024-05-08T05:42:19.488090+0000","last_became_peered":"2024-05-08T05:42:19.488090+0000","last_unstale":"2024-05-08T05:42:23.569781+0000","last_undegraded":"2024-05-08T05:42:23.569781+0000","last_fullsized":"2024-05-08T05:42:23.569781+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:17.430149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568617+0000","last_change":"2024-05-08T05:42:20.519358+0000","last_active":"2024-05-08T05:42:23.568617+0000","last_peered":"2024-05-08T05:42:23.568617+0000","last_clean":"2024-05-08T05:42:23.568617+0000","last_became_active":"2024-05-08T05:42:20.519067+0000","last_became_peered":"2024-05-08T05:42:20.519067+0000","last_unstale":"2024-05-08T05:42:23.568617+0000","last_undegraded":"2024-05-08T05:42:23.568617+0000","last_fullsized":"2024-05-08T05:42:23.568617+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:44.235500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569519+0000","last_change":"2024-05-08T05:42:19.486715+0000","last_active":"2024-05-08T05:42:23.569519+0000","last_peered":"2024-05-08T05:42:23.569519+0000","last_clean":"2024-05-08T05:42:23.569519+0000","last_became_active":"2024-05-08T05:42:19.486403+0000","last_became_peered":"2024-05-08T05:42:19.486403+0000","last_unstale":"2024-05-08T05:42:23.569519+0000","last_undegraded":"2024-05-08T05:42:23.569519+0000","last_fullsized":"2024-05-08T05:42:23.569519+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:43.233742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568767+0000","last_change":"2024-05-08T05:42:20.518993+0000","last_active":"2024-05-08T05:42:23.568767+0000","last_peered":"2024-05-08T05:42:23.568767+0000","last_clean":"2024-05-08T05:42:23.568767+0000","last_became_active":"2024-05-08T05:42:20.518710+0000","last_became_peered":"2024-05-08T05:42:20.518710+0000","last_unstale":"2024-05-08T05:42:23.568767+0000","last_undegraded":"2024-05-08T05:42:23.568767+0000","last_fullsized":"2024-05-08T05:42:23.568767+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:30:06.045575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569189+0000","last_change":"2024-05-08T05:42:19.500123+0000","last_active":"2024-05-08T05:42:23.569189+0000","last_peered":"2024-05-08T05:42:23.569189+0000","last_clean":"2024-05-08T05:42:23.569189+0000","last_became_active":"2024-05-08T05:42:19.499811+0000","last_became_peered":"2024-05-08T05:42:19.499811+0000","last_unstale":"2024-05-08T05:42:23.569189+0000","last_undegraded":"2024-05-08T05:42:23.569189+0000","last_fullsized":"2024-05-08T05:42:23.569189+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:26.053156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.570824+0000","last_change":"2024-05-08T05:42:20.529243+0000","last_active":"2024-05-08T05:42:23.570824+0000","last_peered":"2024-05-08T05:42:23.570824+0000","last_clean":"2024-05-08T05:42:23.570824+0000","last_became_active":"2024-05-08T05:42:20.528946+0000","last_became_peered":"2024-05-08T05:42:20.528946+0000","last_unstale":"2024-05-08T05:42:23.570824+0000","last_undegraded":"2024-05-08T05:42:23.570824+0000","last_fullsized":"2024-05-08T05:42:23.570824+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:14:25.524759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.518270+0000","last_change":"2024-05-08T05:42:19.496693+0000","last_active":"2024-05-08T05:42:21.518270+0000","last_peered":"2024-05-08T05:42:21.518270+0000","last_clean":"2024-05-08T05:42:21.518270+0000","last_became_active":"2024-05-08T05:42:19.496276+0000","last_became_peered":"2024-05-08T05:42:19.496276+0000","last_unstale":"2024-05-08T05:42:21.518270+0000","last_undegraded":"2024-05-08T05:42:21.518270+0000","last_fullsized":"2024-05-08T05:42:21.518270+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:40.235502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569458+0000","last_change":"2024-05-08T05:42:20.521385+0000","last_active":"2024-05-08T05:42:23.569458+0000","last_peered":"2024-05-08T05:42:23.569458+0000","last_clean":"2024-05-08T05:42:23.569458+0000","last_became_active":"2024-05-08T05:42:20.521089+0000","last_became_peered":"2024-05-08T05:42:20.521089+0000","last_unstale":"2024-05-08T05:42:23.569458+0000","last_undegraded":"2024-05-08T05:42:23.569458+0000","last_fullsized":"2024-05-08T05:42:23.569458+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:01.706249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'8","reported_seq":22,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.570657+0000","last_change":"2024-05-08T05:42:19.493145+0000","last_active":"2024-05-08T05:42:23.570657+0000","last_peered":"2024-05-08T05:42:23.570657+0000","last_clean":"2024-05-08T05:42:23.570657+0000","last_became_active":"2024-05-08T05:42:19.492821+0000","last_became_peered":"2024-05-08T05:42:19.492821+0000","last_unstale":"2024-05-08T05:42:23.570657+0000","last_undegraded":"2024-05-08T05:42:23.570657+0000","last_fullsized":"2024-05-08T05:42:23.570657+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:57.039710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569723+0000","last_change":"2024-05-08T05:42:20.529928+0000","last_active":"2024-05-08T05:42:23.569723+0000","last_peered":"2024-05-08T05:42:23.569723+0000","last_clean":"2024-05-08T05:42:23.569723+0000","last_became_active":"2024-05-08T05:42:20.529618+0000","last_became_peered":"2024-05-08T05:42:20.529618+0000","last_unstale":"2024-05-08T05:42:23.569723+0000","last_undegraded":"2024-05-08T05:42:23.569723+0000","last_fullsized":"2024-05-08T05:42:23.569723+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:33:26.883953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:22.509972+0000","last_change":"2024-05-08T05:42:19.499523+0000","last_active":"2024-05-08T05:42:22.509972+0000","last_peered":"2024-05-08T05:42:22.509972+0000","last_clean":"2024-05-08T05:42:22.509972+0000","last_became_active":"2024-05-08T05:42:19.499064+0000","last_became_peered":"2024-05-08T05:42:19.499064+0000","last_unstale":"2024-05-08T05:42:22.509972+0000","last_undegraded":"2024-05-08T05:42:22.509972+0000","last_fullsized":"2024-05-08T05:42:22.509972+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:44:04.272634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:22.509705+0000","last_change":"2024-05-08T05:42:20.524483+0000","last_active":"2024-05-08T05:42:22.509705+0000","last_peered":"2024-05-08T05:42:22.509705+0000","last_clean":"2024-05-08T05:42:22.509705+0000","last_became_active":"2024-05-08T05:42:20.524167+0000","last_became_peered":"2024-05-08T05:42:20.524167+0000","last_unstale":"2024-05-08T05:42:22.509705+0000","last_undegraded":"2024-05-08T05:42:22.509705+0000","last_fullsized":"2024-05-08T05:42:22.509705+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:50.284596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568515+0000","last_change":"2024-05-08T05:42:19.500389+0000","last_active":"2024-05-08T05:42:23.568515+0000","last_peered":"2024-05-08T05:42:23.568515+0000","last_clean":"2024-05-08T05:42:23.568515+0000","last_became_active":"2024-05-08T05:42:19.500111+0000","last_became_peered":"2024-05-08T05:42:19.500111+0000","last_unstale":"2024-05-08T05:42:23.568515+0000","last_undegraded":"2024-05-08T05:42:23.568515+0000","last_fullsized":"2024-05-08T05:42:23.568515+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:14.130507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568430+0000","last_change":"2024-05-08T05:42:20.522305+0000","last_active":"2024-05-08T05:42:23.568430+0000","last_peered":"2024-05-08T05:42:23.568430+0000","last_clean":"2024-05-08T05:42:23.568430+0000","last_became_active":"2024-05-08T05:42:20.521820+0000","last_became_peered":"2024-05-08T05:42:20.521820+0000","last_unstale":"2024-05-08T05:42:23.568430+0000","last_undegraded":"2024-05-08T05:42:23.568430+0000","last_fullsized":"2024-05-08T05:42:23.568430+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:44:05.274392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517447+0000","last_change":"2024-05-08T05:42:19.491582+0000","last_active":"2024-05-08T05:42:21.517447+0000","last_peered":"2024-05-08T05:42:21.517447+0000","last_clean":"2024-05-08T05:42:21.517447+0000","last_became_active":"2024-05-08T05:42:19.491291+0000","last_became_peered":"2024-05-08T05:42:19.491291+0000","last_unstale":"2024-05-08T05:42:21.517447+0000","last_undegraded":"2024-05-08T05:42:21.517447+0000","last_fullsized":"2024-05-08T05:42:21.517447+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:43:53.904889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517451+0000","last_change":"2024-05-08T05:42:20.527117+0000","last_active":"2024-05-08T05:42:21.517451+0000","last_peered":"2024-05-08T05:42:21.517451+0000","last_clean":"2024-05-08T05:42:21.517451+0000","last_became_active":"2024-05-08T05:42:20.526717+0000","last_became_peered":"2024-05-08T05:42:20.526717+0000","last_unstale":"2024-05-08T05:42:21.517451+0000","last_undegraded":"2024-05-08T05:42:21.517451+0000","last_fullsized":"2024-05-08T05:42:21.517451+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:26.076899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568839+0000","last_change":"2024-05-08T05:42:19.491505+0000","last_active":"2024-05-08T05:42:23.568839+0000","last_peered":"2024-05-08T05:42:23.568839+0000","last_clean":"2024-05-08T05:42:23.568839+0000","last_became_active":"2024-05-08T05:42:19.491206+0000","last_became_peered":"2024-05-08T05:42:19.491206+0000","last_unstale":"2024-05-08T05:42:23.568839+0000","last_undegraded":"2024-05-08T05:42:23.568839+0000","last_fullsized":"2024-05-08T05:42:23.568839+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:53:50.777397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.571172+0000","last_change":"2024-05-08T05:42:20.528732+0000","last_active":"2024-05-08T05:42:23.571172+0000","last_peered":"2024-05-08T05:42:23.571172+0000","last_clean":"2024-05-08T05:42:23.571172+0000","last_became_active":"2024-05-08T05:42:20.528444+0000","last_became_peered":"2024-05-08T05:42:20.528444+0000","last_unstale":"2024-05-08T05:42:23.571172+0000","last_undegraded":"2024-05-08T05:42:23.571172+0000","last_fullsized":"2024-05-08T05:42:23.571172+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:43:42.826524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517236+0000","last_change":"2024-05-08T05:42:19.494577+0000","last_active":"2024-05-08T05:42:21.517236+0000","last_peered":"2024-05-08T05:42:21.517236+0000","last_clean":"2024-05-08T05:42:21.517236+0000","last_became_active":"2024-05-08T05:42:19.494261+0000","last_became_peered":"2024-05-08T05:42:19.494261+0000","last_unstale":"2024-05-08T05:42:21.517236+0000","last_undegraded":"2024-05-08T05:42:21.517236+0000","last_fullsized":"2024-05-08T05:42:21.517236+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:07:51.458867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568433+0000","last_change":"2024-05-08T05:42:20.524296+0000","last_active":"2024-05-08T05:42:23.568433+0000","last_peered":"2024-05-08T05:42:23.568433+0000","last_clean":"2024-05-08T05:42:23.568433+0000","last_became_active":"2024-05-08T05:42:20.523971+0000","last_became_peered":"2024-05-08T05:42:20.523971+0000","last_unstale":"2024-05-08T05:42:23.568433+0000","last_undegraded":"2024-05-08T05:42:23.568433+0000","last_fullsized":"2024-05-08T05:42:23.568433+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:23.447125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517421+0000","last_change":"2024-05-08T05:42:19.495869+0000","last_active":"2024-05-08T05:42:21.517421+0000","last_peered":"2024-05-08T05:42:21.517421+0000","last_clean":"2024-05-08T05:42:21.517421+0000","last_became_active":"2024-05-08T05:42:19.495494+0000","last_became_peered":"2024-05-08T05:42:19.495494+0000","last_unstale":"2024-05-08T05:42:21.517421+0000","last_undegraded":"2024-05-08T05:42:21.517421+0000","last_fullsized":"2024-05-08T05:42:21.517421+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:08.085241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:22.509297+0000","last_change":"2024-05-08T05:42:20.519113+0000","last_active":"2024-05-08T05:42:22.509297+0000","last_peered":"2024-05-08T05:42:22.509297+0000","last_clean":"2024-05-08T05:42:22.509297+0000","last_became_active":"2024-05-08T05:42:20.518840+0000","last_became_peered":"2024-05-08T05:42:20.518840+0000","last_unstale":"2024-05-08T05:42:22.509297+0000","last_undegraded":"2024-05-08T05:42:22.509297+0000","last_fullsized":"2024-05-08T05:42:22.509297+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:30:06.045575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.570889+0000","last_change":"2024-05-08T05:42:19.494546+0000","last_active":"2024-05-08T05:42:23.570889+0000","last_peered":"2024-05-08T05:42:23.570889+0000","last_clean":"2024-05-08T05:42:23.570889+0000","last_became_active":"2024-05-08T05:42:19.494232+0000","last_became_peered":"2024-05-08T05:42:19.494232+0000","last_unstale":"2024-05-08T05:42:23.570889+0000","last_undegraded":"2024-05-08T05:42:23.570889+0000","last_fullsized":"2024-05-08T05:42:23.570889+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:37.041564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569563+0000","last_change":"2024-05-08T05:42:20.533065+0000","last_active":"2024-05-08T05:42:23.569563+0000","last_peered":"2024-05-08T05:42:23.569563+0000","last_clean":"2024-05-08T05:42:23.569563+0000","last_became_active":"2024-05-08T05:42:20.532437+0000","last_became_peered":"2024-05-08T05:42:20.532437+0000","last_unstale":"2024-05-08T05:42:23.569563+0000","last_undegraded":"2024-05-08T05:42:23.569563+0000","last_fullsized":"2024-05-08T05:42:23.569563+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:50.284596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517058+0000","last_change":"2024-05-08T05:42:19.494499+0000","last_active":"2024-05-08T05:42:21.517058+0000","last_peered":"2024-05-08T05:42:21.517058+0000","last_clean":"2024-05-08T05:42:21.517058+0000","last_became_active":"2024-05-08T05:42:19.494193+0000","last_became_peered":"2024-05-08T05:42:19.494193+0000","last_unstale":"2024-05-08T05:42:21.517058+0000","last_undegraded":"2024-05-08T05:42:21.517058+0000","last_fullsized":"2024-05-08T05:42:21.517058+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:34.880276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568778+0000","last_change":"2024-05-08T05:42:20.518085+0000","last_active":"2024-05-08T05:42:23.568778+0000","last_peered":"2024-05-08T05:42:23.568778+0000","last_clean":"2024-05-08T05:42:23.568778+0000","last_became_active":"2024-05-08T05:42:20.517771+0000","last_became_peered":"2024-05-08T05:42:20.517771+0000","last_unstale":"2024-05-08T05:42:23.568778+0000","last_undegraded":"2024-05-08T05:42:23.568778+0000","last_fullsized":"2024-05-08T05:42:23.568778+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:18.431907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:22.510060+0000","last_change":"2024-05-08T05:42:19.498985+0000","last_active":"2024-05-08T05:42:22.510060+0000","last_peered":"2024-05-08T05:42:22.510060+0000","last_clean":"2024-05-08T05:42:22.510060+0000","last_became_active":"2024-05-08T05:42:19.498663+0000","last_became_peered":"2024-05-08T05:42:19.498663+0000","last_unstale":"2024-05-08T05:42:22.510060+0000","last_undegraded":"2024-05-08T05:42:22.510060+0000","last_fullsized":"2024-05-08T05:42:22.510060+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:43.233742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.571032+0000","last_change":"2024-05-08T05:42:20.529587+0000","last_active":"2024-05-08T05:42:23.571032+0000","last_peered":"2024-05-08T05:42:23.571032+0000","last_clean":"2024-05-08T05:42:23.571032+0000","last_became_active":"2024-05-08T05:42:20.529151+0000","last_became_peered":"2024-05-08T05:42:20.529151+0000","last_unstale":"2024-05-08T05:42:23.571032+0000","last_undegraded":"2024-05-08T05:42:23.571032+0000","last_fullsized":"2024-05-08T05:42:23.571032+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:22:33.614866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"36'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.517320+0000","last_change":"2024-05-08T05:42:19.499917+0000","last_active":"2024-05-08T05:42:21.517320+0000","last_peered":"2024-05-08T05:42:21.517320+0000","last_clean":"2024-05-08T05:42:21.517320+0000","last_became_active":"2024-05-08T05:42:19.499614+0000","last_became_peered":"2024-05-08T05:42:19.499614+0000","last_unstale":"2024-05-08T05:42:21.517320+0000","last_undegraded":"2024-05-08T05:42:21.517320+0000","last_fullsized":"2024-05-08T05:42:21.517320+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:45.821917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569509+0000","last_change":"2024-05-08T05:42:20.529586+0000","last_active":"2024-05-08T05:42:23.569509+0000","last_peered":"2024-05-08T05:42:23.569509+0000","last_clean":"2024-05-08T05:42:23.569509+0000","last_became_active":"2024-05-08T05:42:20.529176+0000","last_became_peered":"2024-05-08T05:42:20.529176+0000","last_unstale":"2024-05-08T05:42:23.569509+0000","last_undegraded":"2024-05-08T05:42:23.569509+0000","last_fullsized":"2024-05-08T05:42:23.569509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:53.621950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569217+0000","last_change":"2024-05-08T05:42:19.492651+0000","last_active":"2024-05-08T05:42:23.569217+0000","last_peered":"2024-05-08T05:42:23.569217+0000","last_clean":"2024-05-08T05:42:23.569217+0000","last_became_active":"2024-05-08T05:42:19.491268+0000","last_became_peered":"2024-05-08T05:42:19.491268+0000","last_unstale":"2024-05-08T05:42:23.569217+0000","last_undegraded":"2024-05-08T05:42:23.569217+0000","last_fullsized":"2024-05-08T05:42:23.569217+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:29.265240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568690+0000","last_change":"2024-05-08T05:42:20.525312+0000","last_active":"2024-05-08T05:42:23.568690+0000","last_peered":"2024-05-08T05:42:23.568690+0000","last_clean":"2024-05-08T05:42:23.568690+0000","last_became_active":"2024-05-08T05:42:20.524640+0000","last_became_peered":"2024-05-08T05:42:20.524640+0000","last_unstale":"2024-05-08T05:42:23.568690+0000","last_undegraded":"2024-05-08T05:42:23.568690+0000","last_fullsized":"2024-05-08T05:42:23.568690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:18.431907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569450+0000","last_change":"2024-05-08T05:42:19.494149+0000","last_active":"2024-05-08T05:42:23.569450+0000","last_peered":"2024-05-08T05:42:23.569450+0000","last_clean":"2024-05-08T05:42:23.569450+0000","last_became_active":"2024-05-08T05:42:19.493848+0000","last_became_peered":"2024-05-08T05:42:19.493848+0000","last_unstale":"2024-05-08T05:42:23.569450+0000","last_undegraded":"2024-05-08T05:42:23.569450+0000","last_fullsized":"2024-05-08T05:42:23.569450+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:25.075141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.569011+0000","last_change":"2024-05-08T05:42:20.518816+0000","last_active":"2024-05-08T05:42:23.569011+0000","last_peered":"2024-05-08T05:42:23.569011+0000","last_clean":"2024-05-08T05:42:23.569011+0000","last_became_active":"2024-05-08T05:42:20.518514+0000","last_became_peered":"2024-05-08T05:42:20.518514+0000","last_unstale":"2024-05-08T05:42:23.569011+0000","last_undegraded":"2024-05-08T05:42:23.569011+0000","last_fullsized":"2024-05-08T05:42:23.569011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:06.429659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:21.518505+0000","last_change":"2024-05-08T05:42:19.497480+0000","last_active":"2024-05-08T05:42:21.518505+0000","last_peered":"2024-05-08T05:42:21.518505+0000","last_clean":"2024-05-08T05:42:21.518505+0000","last_became_active":"2024-05-08T05:42:19.497177+0000","last_became_peered":"2024-05-08T05:42:19.497177+0000","last_unstale":"2024-05-08T05:42:21.518505+0000","last_undegraded":"2024-05-08T05:42:21.518505+0000","last_fullsized":"2024-05-08T05:42:21.518505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:14.130507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568064+0000","last_change":"2024-05-08T05:42:20.530479+0000","last_active":"2024-05-08T05:42:23.568064+0000","last_peered":"2024-05-08T05:42:23.568064+0000","last_clean":"2024-05-08T05:42:23.568064+0000","last_became_active":"2024-05-08T05:42:20.530210+0000","last_became_peered":"2024-05-08T05:42:20.530210+0000","last_unstale":"2024-05-08T05:42:23.568064+0000","last_undegraded":"2024-05-08T05:42:23.568064+0000","last_fullsized":"2024-05-08T05:42:23.568064+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:58.041468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.567873+0000","last_change":"2024-05-08T05:42:19.495013+0000","last_active":"2024-05-08T05:42:23.567873+0000","last_peered":"2024-05-08T05:42:23.567873+0000","last_clean":"2024-05-08T05:42:23.567873+0000","last_became_active":"2024-05-08T05:42:19.494694+0000","last_became_peered":"2024-05-08T05:42:19.494694+0000","last_unstale":"2024-05-08T05:42:23.567873+0000","last_undegraded":"2024-05-08T05:42:23.567873+0000","last_fullsized":"2024-05-08T05:42:23.567873+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:53:29.265240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.571318+0000","last_change":"2024-05-08T05:42:20.523727+0000","last_active":"2024-05-08T05:42:23.571318+0000","last_peered":"2024-05-08T05:42:23.571318+0000","last_clean":"2024-05-08T05:42:23.571318+0000","last_became_active":"2024-05-08T05:42:20.523460+0000","last_became_peered":"2024-05-08T05:42:20.523460+0000","last_unstale":"2024-05-08T05:42:23.571318+0000","last_undegraded":"2024-05-08T05:42:23.571318+0000","last_fullsized":"2024-05-08T05:42:23.571318+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:01.706249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.568473+0000","last_change":"2024-05-08T05:42:19.487951+0000","last_active":"2024-05-08T05:42:23.568473+0000","last_peered":"2024-05-08T05:42:23.568473+0000","last_clean":"2024-05-08T05:42:23.568473+0000","last_became_active":"2024-05-08T05:42:19.487613+0000","last_became_peered":"2024-05-08T05:42:19.487613+0000","last_unstale":"2024-05-08T05:42:23.568473+0000","last_undegraded":"2024-05-08T05:42:23.568473+0000","last_fullsized":"2024-05-08T05:42:23.568473+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:57.039710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:22.509903+0000","last_change":"2024-05-08T05:42:20.525352+0000","last_active":"2024-05-08T05:42:22.509903+0000","last_peered":"2024-05-08T05:42:22.509903+0000","last_clean":"2024-05-08T05:42:22.509903+0000","last_became_active":"2024-05-08T05:42:20.524836+0000","last_became_peered":"2024-05-08T05:42:20.524836+0000","last_unstale":"2024-05-08T05:42:22.509903+0000","last_undegraded":"2024-05-08T05:42:22.509903+0000","last_fullsized":"2024-05-08T05:42:22.509903+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:19.471571+0000","last_clean_scrub_stamp":"2024-05-08T05:42:19.471571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:47:54.483217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.571081+0000","last_change":"2024-05-08T05:42:19.487846+0000","last_active":"2024-05-08T05:42:23.571081+0000","last_peered":"2024-05-08T05:42:23.571081+0000","last_clean":"2024-05-08T05:42:23.571081+0000","last_became_active":"2024-05-08T05:42:19.487570+0000","last_became_peered":"2024-05-08T05:42:19.487570+0000","last_unstale":"2024-05-08T05:42:23.571081+0000","last_undegraded":"2024-05-08T05:42:23.571081+0000","last_fullsized":"2024-05-08T05:42:23.571081+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:18.469813+0000","last_clean_scrub_stamp":"2024-05-08T05:42:18.469813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:25:22.197321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":139,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-08T05:42:23.571093+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:42:23.571093+0000","last_peered":"2024-05-08T05:42:23.571093+0000","last_clean":"2024-05-08T05:42:23.571093+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:42:23.571093+0000","last_undegraded":"2024-05-08T05:42:23.571093+0000","last_fullsized":"2024-05-08T05:42:23.571093+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":766521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":175053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":174583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":175075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":425984,"data_stored":176117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":413696,"data_stored":174541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31572,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340268,"statfs":{"total":96636764160,"available":96604434432,"internally_reserved":0,"allocated":401408,"data_stored":167869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":995328,"data_stored":758181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:42:29.402 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T05:42:29.402 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T05:42:29.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.549+0000 7f847f86d640 1 Processor -- start 2024-05-08T05:42:29.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.549+0000 7f847f86d640 1 -- start start 2024-05-08T05:42:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.550+0000 7f847f86d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f84780cfcd0 0x7f84780d4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.550+0000 7f847f86d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f84780d54e0 0x7f84780d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.550+0000 7f847f86d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84780d0320 con 0x7f847815f4e0 2024-05-08T05:42:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.550+0000 7f847f86d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f84780d41b0 con 0x7f84780cfcd0 2024-05-08T05:42:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.550+0000 7f847f86d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f84780d0600 con 0x7f84780d54e0 2024-05-08T05:42:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.550+0000 7f847ce08640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f84780cfcd0 0x7f84780d4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.550+0000 7f847ce08640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f84780cfcd0 0x7f84780d4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44286/0 (socket says 172.21.15.45:44286) 2024-05-08T05:42:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.550+0000 7f847ce08640 1 -- 172.21.15.45:0/434729963 learned_addr learned my addr 172.21.15.45:0/434729963 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.550+0000 7f847de0a640 1 --2- 172.21.15.45:0/434729963 >> v2:172.21.15.63:3300/0 conn(0x7f84780d54e0 0x7f84780d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.550+0000 7f847ce08640 1 -- 172.21.15.45:0/434729963 >> v2:172.21.15.63:3300/0 conn(0x7f84780d54e0 msgr2=0x7f84780d13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.550+0000 7f847ce08640 1 --2- 172.21.15.45:0/434729963 >> v2:172.21.15.63:3300/0 conn(0x7f84780d54e0 0x7f84780d13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.550+0000 7f847ce08640 1 -- 172.21.15.45:0/434729963 >> v1:172.21.15.63:6789/0 conn(0x7f847815f4e0 legacy=0x7f84780d3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.550+0000 7f847ce08640 1 -- 172.21.15.45:0/434729963 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84780d08e0 con 0x7f84780cfcd0 2024-05-08T05:42:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.551+0000 7f847ce08640 1 --2- 172.21.15.45:0/434729963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84780cfcd0 0x7f84780d4fa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f847808e9d0 tx=0x7f846802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81e50849bf9cd70 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.551+0000 7f846ffff640 1 -- 172.21.15.45:0/434729963 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468037070 con 0x7f84780cfcd0 2024-05-08T05:42:29.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.551+0000 7f846ffff640 1 -- 172.21.15.45:0/434729963 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8468032440 con 0x7f84780cfcd0 2024-05-08T05:42:29.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.551+0000 7f846ffff640 1 -- 172.21.15.45:0/434729963 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468032810 con 0x7f84780cfcd0 2024-05-08T05:42:29.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.552+0000 7f847f86d640 1 -- 172.21.15.45:0/434729963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84780cfcd0 msgr2=0x7f84780d4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:29.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.552+0000 7f847f86d640 1 --2- 172.21.15.45:0/434729963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84780cfcd0 0x7f84780d4fa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f847808e9d0 tx=0x7f846802a270 comp rx=0 tx=0).stop 2024-05-08T05:42:29.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.552+0000 7f847f86d640 1 -- 172.21.15.45:0/434729963 shutdown_connections 2024-05-08T05:42:29.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.552+0000 7f847f86d640 1 --2- 172.21.15.45:0/434729963 >> v2:172.21.15.63:3300/0 conn(0x7f84780d54e0 0x7f84780d13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.552+0000 7f847f86d640 1 --2- 172.21.15.45:0/434729963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84780cfcd0 0x7f84780d4fa0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.552+0000 7f847f86d640 1 -- 172.21.15.45:0/434729963 >> 172.21.15.45:0/434729963 conn(0x7f847808eac0 msgr2=0x7f847809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.552+0000 7f847f86d640 1 -- 172.21.15.45:0/434729963 shutdown_connections 2024-05-08T05:42:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.552+0000 7f847f86d640 1 -- 172.21.15.45:0/434729963 wait complete. 2024-05-08T05:42:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.553+0000 7f847f86d640 1 Processor -- start 2024-05-08T05:42:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.553+0000 7f847f86d640 1 -- start start 2024-05-08T05:42:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.554+0000 7f847f86d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84780cfcd0 0x7f84781617d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.554+0000 7f847f86d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84780d54e0 0x7f84781590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.554+0000 7f847f86d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f847815f4e0 0x7f84780ce820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.554+0000 7f847f86d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f847815f8c0 con 0x7f847815f4e0 2024-05-08T05:42:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.554+0000 7f847f86d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f84780d0ea0 con 0x7f84780cfcd0 2024-05-08T05:42:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.554+0000 7f847f86d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f84780d1900 con 0x7f84780d54e0 2024-05-08T05:42:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.554+0000 7f847de0a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f847815f4e0 0x7f84780ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.554+0000 7f847de0a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f847815f4e0 0x7f84780ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44294/0 (socket says 172.21.15.45:44294) 2024-05-08T05:42:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.554+0000 7f847de0a640 1 -- 172.21.15.45:0/4209072194 learned_addr learned my addr 172.21.15.45:0/4209072194 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.554+0000 7f847d609640 1 --2- 172.21.15.45:0/4209072194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84780cfcd0 0x7f84781617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.554+0000 7f847ce08640 1 --2- 172.21.15.45:0/4209072194 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84780d54e0 0x7f84781590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.554+0000 7f847de0a640 1 -- 172.21.15.45:0/4209072194 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84780d54e0 msgr2=0x7f84781590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.554+0000 7f847de0a640 1 --2- 172.21.15.45:0/4209072194 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84780d54e0 0x7f84781590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.555+0000 7f847de0a640 1 -- 172.21.15.45:0/4209072194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84780cfcd0 msgr2=0x7f84781617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.555+0000 7f847de0a640 1 --2- 172.21.15.45:0/4209072194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84780cfcd0 0x7f84781617d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.555+0000 7f847de0a640 1 -- 172.21.15.45:0/4209072194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8478160730 con 0x7f847815f4e0 2024-05-08T05:42:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.555+0000 7f847de0a640 1 --2- 172.21.15.45:0/4209072194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f847815f4e0 0x7f84780ce820 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8470007e40 tx=0x7f847000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.555+0000 7f846dffb640 1 -- 172.21.15.45:0/4209072194 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470014620 con 0x7f847815f4e0 2024-05-08T05:42:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.555+0000 7f846dffb640 1 -- 172.21.15.45:0/4209072194 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8470019070 con 0x7f847815f4e0 2024-05-08T05:42:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.555+0000 7f847f86d640 1 -- 172.21.15.45:0/4209072194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8478160a10 con 0x7f847815f4e0 2024-05-08T05:42:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.555+0000 7f846dffb640 1 -- 172.21.15.45:0/4209072194 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470014a20 con 0x7f847815f4e0 2024-05-08T05:42:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.555+0000 7f847f86d640 1 -- 172.21.15.45:0/4209072194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8478160fd0 con 0x7f847815f4e0 2024-05-08T05:42:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.556+0000 7f847f86d640 1 -- 172.21.15.45:0/4209072194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84780693f0 con 0x7f847815f4e0 2024-05-08T05:42:29.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.557+0000 7f846dffb640 1 -- 172.21.15.45:0/4209072194 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8470014bc0 con 0x7f847815f4e0 2024-05-08T05:42:29.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.557+0000 7f846dffb640 1 --2- 172.21.15.45:0/4209072194 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f845c0762d0 0x7f845c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:29.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.557+0000 7f846dffb640 1 -- 172.21.15.45:0/4209072194 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f8470097f40 con 0x7f847815f4e0 2024-05-08T05:42:29.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.558+0000 7f847d609640 1 --2- 172.21.15.45:0/4209072194 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f845c0762d0 0x7f845c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:29.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.558+0000 7f847d609640 1 --2- 172.21.15.45:0/4209072194 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f845c0762d0 0x7f845c078790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8474001720 tx=0x7f8474009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.562+0000 7f846dffb640 1 -- 172.21.15.45:0/4209072194 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f84700610c0 con 0x7f847815f4e0 2024-05-08T05:42:29.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.827+0000 7f847f86d640 1 -- 172.21.15.45:0/4209072194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f84780d31b0 con 0x7f847815f4e0 2024-05-08T05:42:29.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.829+0000 7f846dffb640 1 -- 172.21.15.45:0/4209072194 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7f84700653c0 con 0x7f847815f4e0 2024-05-08T05:42:29.830 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-05-08T05:42:29.830 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBUEDtmDcqGMxAALgQaBYp4Zd8uMqswCXvGaw== 2024-05-08T05:42:29.830 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2024-05-08T05:42:29.830 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2024-05-08T05:42:29.830 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-08T05:42:29.830 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2024-05-08T05:42:29.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.831+0000 7f847f86d640 1 -- 172.21.15.45:0/4209072194 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f845c0762d0 msgr2=0x7f845c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:29.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.831+0000 7f847f86d640 1 --2- 172.21.15.45:0/4209072194 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f845c0762d0 0x7f845c078790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8474001720 tx=0x7f8474009040 comp rx=0 tx=0).stop 2024-05-08T05:42:29.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.831+0000 7f847f86d640 1 -- 172.21.15.45:0/4209072194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f847815f4e0 msgr2=0x7f84780ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:29.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.831+0000 7f847f86d640 1 --2- 172.21.15.45:0/4209072194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f847815f4e0 0x7f84780ce820 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8470007e40 tx=0x7f847000e4b0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.831+0000 7f847f86d640 1 -- 172.21.15.45:0/4209072194 shutdown_connections 2024-05-08T05:42:29.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.832+0000 7f847f86d640 1 --2- 172.21.15.45:0/4209072194 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f845c0762d0 0x7f845c078790 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.832+0000 7f847f86d640 1 --2- 172.21.15.45:0/4209072194 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84780d54e0 0x7f84781590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.832+0000 7f847f86d640 1 --2- 172.21.15.45:0/4209072194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f847815f4e0 0x7f84780ce820 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.832+0000 7f847f86d640 1 --2- 172.21.15.45:0/4209072194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84780cfcd0 0x7f84781617d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.832+0000 7f847f86d640 1 -- 172.21.15.45:0/4209072194 >> 172.21.15.45:0/4209072194 conn(0x7f847808eac0 msgr2=0x7f84780d3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:29.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.832+0000 7f847f86d640 1 -- 172.21.15.45:0/4209072194 shutdown_connections 2024-05-08T05:42:29.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.832+0000 7f847f86d640 1 -- 172.21.15.45:0/4209072194 wait complete. 2024-05-08T05:42:29.843 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:42:29.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.947+0000 7feb14299640 1 Processor -- start 2024-05-08T05:42:29.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.947+0000 7feb14299640 1 -- start start 2024-05-08T05:42:29.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.948+0000 7feb14299640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7feb0c1629a0 0x7feb0c0d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:29.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.948+0000 7feb14299640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feb0c0d0cb0 0x7feb0c0d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:29.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.948+0000 7feb14299640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb0c0d3330 con 0x7feb0c0d2dc0 2024-05-08T05:42:29.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.948+0000 7feb14299640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7feb0c0d2170 con 0x7feb0c0d0cb0 2024-05-08T05:42:29.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.948+0000 7feb14299640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7feb0c0d2450 con 0x7feb0c1629a0 2024-05-08T05:42:29.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.948+0000 7feb12836640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feb0c0d0cb0 0x7feb0c0d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:29.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.948+0000 7feb11834640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7feb0c0d2dc0 0x7feb0c0c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:56088/0 (socket says 172.21.15.45:56088) 2024-05-08T05:42:29.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.948+0000 7feb11834640 1 -- 172.21.15.45:0/3833204563 learned_addr learned my addr 172.21.15.45:0/3833204563 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:29.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.948+0000 7feb12035640 1 --2- 172.21.15.45:0/3833204563 >> v2:172.21.15.63:3300/0 conn(0x7feb0c1629a0 0x7feb0c0d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:29.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.948+0000 7feb12836640 1 -- 172.21.15.45:0/3833204563 >> v2:172.21.15.63:3300/0 conn(0x7feb0c1629a0 msgr2=0x7feb0c0d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:29.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.948+0000 7feb12836640 1 --2- 172.21.15.45:0/3833204563 >> v2:172.21.15.63:3300/0 conn(0x7feb0c1629a0 0x7feb0c0d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.948+0000 7feb12836640 1 -- 172.21.15.45:0/3833204563 >> v1:172.21.15.45:6789/0 conn(0x7feb0c0d2dc0 legacy=0x7feb0c0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:29.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.948+0000 7feb12836640 1 -- 172.21.15.45:0/3833204563 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0c0d2730 con 0x7feb0c0d0cb0 2024-05-08T05:42:29.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.948+0000 7feb11033640 1 -- 172.21.15.45:0/3833204563 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3912071887 0 0) 0x7feb0c0d3330 con 0x7feb0c0d2dc0 2024-05-08T05:42:29.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.949+0000 7feb12836640 1 --2- 172.21.15.45:0/3833204563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c0d0cb0 0x7feb0c0d3b00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0768a0 tx=0x7feb0402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270d9141bb412cca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:29.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.949+0000 7feb11033640 1 -- 172.21.15.45:0/3833204563 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04007740 con 0x7feb0c0d0cb0 2024-05-08T05:42:29.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.949+0000 7feb11033640 1 -- 172.21.15.45:0/3833204563 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb040078e0 con 0x7feb0c0d0cb0 2024-05-08T05:42:29.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.949+0000 7feb11033640 1 -- 172.21.15.45:0/3833204563 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04007cb0 con 0x7feb0c0d0cb0 2024-05-08T05:42:29.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.950+0000 7feb14299640 1 -- 172.21.15.45:0/3833204563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c0d0cb0 msgr2=0x7feb0c0d3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:29.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.950+0000 7feb14299640 1 --2- 172.21.15.45:0/3833204563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c0d0cb0 0x7feb0c0d3b00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0768a0 tx=0x7feb0402a560 comp rx=0 tx=0).stop 2024-05-08T05:42:29.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.950+0000 7feb14299640 1 -- 172.21.15.45:0/3833204563 shutdown_connections 2024-05-08T05:42:29.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.950+0000 7feb14299640 1 --2- 172.21.15.45:0/3833204563 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c0d0cb0 0x7feb0c0d3b00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.950+0000 7feb14299640 1 --2- 172.21.15.45:0/3833204563 >> v2:172.21.15.63:3300/0 conn(0x7feb0c1629a0 0x7feb0c0d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.950+0000 7feb14299640 1 -- 172.21.15.45:0/3833204563 >> 172.21.15.45:0/3833204563 conn(0x7feb0c0d4fe0 msgr2=0x7feb0c0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:29.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.950+0000 7feb14299640 1 -- 172.21.15.45:0/3833204563 shutdown_connections 2024-05-08T05:42:29.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.950+0000 7feb14299640 1 -- 172.21.15.45:0/3833204563 wait complete. 2024-05-08T05:42:29.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.951+0000 7feb14299640 1 Processor -- start 2024-05-08T05:42:29.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.951+0000 7feb14299640 1 -- start start 2024-05-08T05:42:29.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.952+0000 7feb14299640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb0c0d0cb0 0x7feb0c15b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:29.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.952+0000 7feb14299640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb0c0d2dc0 0x7feb0c1f8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:29.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.952+0000 7feb14299640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c1629a0 0x7feb0c1f8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:29.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.952+0000 7feb14299640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c0cf440 con 0x7feb0c1629a0 2024-05-08T05:42:29.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.952+0000 7feb14299640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c0cf2c0 con 0x7feb0c0d0cb0 2024-05-08T05:42:29.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.952+0000 7feb14299640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feb0c0c95e0 con 0x7feb0c0d2dc0 2024-05-08T05:42:29.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.952+0000 7feb12836640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c1629a0 0x7feb0c1f8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:29.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.952+0000 7feb12836640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c1629a0 0x7feb0c1f8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44310/0 (socket says 172.21.15.45:44310) 2024-05-08T05:42:29.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.952+0000 7feb12836640 1 -- 172.21.15.45:0/1140328238 learned_addr learned my addr 172.21.15.45:0/1140328238 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:29.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.952+0000 7feb12035640 1 --2- 172.21.15.45:0/1140328238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb0c0d0cb0 0x7feb0c15b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:29.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.952+0000 7feb11834640 1 --2- 172.21.15.45:0/1140328238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb0c0d2dc0 0x7feb0c1f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:29.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.952+0000 7feb12836640 1 -- 172.21.15.45:0/1140328238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb0c0d2dc0 msgr2=0x7feb0c1f8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:29.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.952+0000 7feb12836640 1 --2- 172.21.15.45:0/1140328238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb0c0d2dc0 0x7feb0c1f8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.952+0000 7feb12836640 1 -- 172.21.15.45:0/1140328238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb0c0d0cb0 msgr2=0x7feb0c15b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:29.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.952+0000 7feb12836640 1 --2- 172.21.15.45:0/1140328238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb0c0d0cb0 0x7feb0c15b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:29.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.952+0000 7feb12836640 1 -- 172.21.15.45:0/1140328238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0c0b7d30 con 0x7feb0c1629a0 2024-05-08T05:42:29.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.952+0000 7feb11834640 1 --2- 172.21.15.45:0/1140328238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb0c0d2dc0 0x7feb0c1f8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:29.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.952+0000 7feb12836640 1 --2- 172.21.15.45:0/1140328238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c1629a0 0x7feb0c1f8b50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7feb0402ab50 tx=0x7feb04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:29.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.953+0000 7feb02ffd640 1 -- 172.21.15.45:0/1140328238 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04040070 con 0x7feb0c1629a0 2024-05-08T05:42:29.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.953+0000 7feb02ffd640 1 -- 172.21.15.45:0/1140328238 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb04031bc0 con 0x7feb0c1629a0 2024-05-08T05:42:29.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.953+0000 7feb14299640 1 -- 172.21.15.45:0/1140328238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0c0b8010 con 0x7feb0c1629a0 2024-05-08T05:42:29.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.953+0000 7feb02ffd640 1 -- 172.21.15.45:0/1140328238 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04009450 con 0x7feb0c1629a0 2024-05-08T05:42:29.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.953+0000 7feb14299640 1 -- 172.21.15.45:0/1140328238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb0c0b85d0 con 0x7feb0c1629a0 2024-05-08T05:42:29.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.954+0000 7feb14299640 1 -- 172.21.15.45:0/1140328238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb0c0693f0 con 0x7feb0c1629a0 2024-05-08T05:42:29.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.954+0000 7feb02ffd640 1 -- 172.21.15.45:0/1140328238 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7feb04039040 con 0x7feb0c1629a0 2024-05-08T05:42:29.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.955+0000 7feb02ffd640 1 --2- 172.21.15.45:0/1140328238 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feaf00762d0 0x7feaf0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:29.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.955+0000 7feb02ffd640 1 -- 172.21.15.45:0/1140328238 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7feb040b8f40 con 0x7feb0c1629a0 2024-05-08T05:42:29.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.955+0000 7feb12035640 1 --2- 172.21.15.45:0/1140328238 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feaf00762d0 0x7feaf0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:29.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.956+0000 7feb12035640 1 --2- 172.21.15.45:0/1140328238 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feaf00762d0 0x7feaf0078790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feb08009aa0 tx=0x7feb08009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:29.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:29.959+0000 7feb02ffd640 1 -- 172.21.15.45:0/1140328238 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7feb04082710 con 0x7feb0c1629a0 2024-05-08T05:42:30.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.188+0000 7feb14299640 1 -- 172.21.15.45:0/1140328238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb0c0b76a0 con 0x7feb0c1629a0 2024-05-08T05:42:30.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.189+0000 7feb02ffd640 1 -- 172.21.15.45:0/1140328238 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5172 (secure 0 0 0) 0x7feb04082010 con 0x7feb0c1629a0 2024-05-08T05:42:30.189 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:30.189 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3295938044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3295938044},{"type":"v1","addr":"172.21.15.63:6836","nonce":3295938044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4024530715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4024530715},{"type":"v1","addr":"172.21.15.63:6837","nonce":4024530715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/97258541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":97258541},{"type":"v1","addr":"172.21.15.45:6834","nonce":97258541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:42:19.860331+0000","modified":"2024-05-08T05:42:22.891006+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4884":{"gid":4884,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/173436628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":173436628},{"type":"v1","addr":"172.21.15.45:6835","nonce":173436628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4884,"qdb_cluster":[4884]},"id":2}]} 2024-05-08T05:42:30.189 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2024-05-08T05:42:30.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.190+0000 7feb14299640 1 -- 172.21.15.45:0/1140328238 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feaf00762d0 msgr2=0x7feaf0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:30.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.190+0000 7feb14299640 1 --2- 172.21.15.45:0/1140328238 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feaf00762d0 0x7feaf0078790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feb08009aa0 tx=0x7feb08009190 comp rx=0 tx=0).stop 2024-05-08T05:42:30.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.190+0000 7feb14299640 1 -- 172.21.15.45:0/1140328238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c1629a0 msgr2=0x7feb0c1f8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:30.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.190+0000 7feb14299640 1 --2- 172.21.15.45:0/1140328238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c1629a0 0x7feb0c1f8b50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7feb0402ab50 tx=0x7feb04004a40 comp rx=0 tx=0).stop 2024-05-08T05:42:30.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.190+0000 7feb14299640 1 -- 172.21.15.45:0/1140328238 shutdown_connections 2024-05-08T05:42:30.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.191+0000 7feb14299640 1 --2- 172.21.15.45:0/1140328238 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feaf00762d0 0x7feaf0078790 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.191+0000 7feb14299640 1 --2- 172.21.15.45:0/1140328238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb0c0d2dc0 0x7feb0c1f8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.191+0000 7feb14299640 1 --2- 172.21.15.45:0/1140328238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb0c1629a0 0x7feb0c1f8b50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.191+0000 7feb14299640 1 --2- 172.21.15.45:0/1140328238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb0c0d0cb0 0x7feb0c15b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.191+0000 7feb14299640 1 -- 172.21.15.45:0/1140328238 >> 172.21.15.45:0/1140328238 conn(0x7feb0c0d4fe0 msgr2=0x7feb0c0b6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:30.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.191+0000 7feb14299640 1 -- 172.21.15.45:0/1140328238 shutdown_connections 2024-05-08T05:42:30.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.191+0000 7feb14299640 1 -- 172.21.15.45:0/1140328238 wait complete. 2024-05-08T05:42:30.604 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:42:19.860331+0000', 'modified': '2024-05-08T05:42:22.891006+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4884}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4884': {'gid': 4884, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6835/173436628', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 173436628}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 173436628}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4884, 'qdb_cluster': [4884]} 2024-05-08T05:42:30.604 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T05:42:30.604 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T05:42:30.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.713+0000 7f6c58608640 1 Processor -- start 2024-05-08T05:42:30.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.713+0000 7f6c58608640 1 -- start start 2024-05-08T05:42:30.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.714+0000 7f6c58608640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c5015f380 0x7f6c500d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.714+0000 7f6c58608640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6c500d3cd0 0x7f6c500b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.714+0000 7f6c58608640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c500d1760 con 0x7f6c500d0cf0 2024-05-08T05:42:30.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.714+0000 7f6c58608640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6c500d30f0 con 0x7f6c500d3cd0 2024-05-08T05:42:30.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.714+0000 7f6c58608640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6c500d1150 con 0x7f6c5015f380 2024-05-08T05:42:30.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.714+0000 7f6c563a4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c5015f380 0x7f6c500d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.714+0000 7f6c56ba5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6c500d3cd0 0x7f6c500b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.715+0000 7f6c55ba3640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f6c500d0cf0 0x7f6c500d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:49522/0 (socket says 172.21.15.45:49522) 2024-05-08T05:42:30.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.715+0000 7f6c56ba5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6c500d3cd0 0x7f6c500b7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:53506/0 (socket says 172.21.15.45:53506) 2024-05-08T05:42:30.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.715+0000 7f6c563a4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c5015f380 0x7f6c500d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:40682/0 (socket says 172.21.15.45:40682) 2024-05-08T05:42:30.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.715+0000 7f6c55ba3640 1 -- 172.21.15.45:0/235450818 learned_addr learned my addr 172.21.15.45:0/235450818 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:30.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.715+0000 7f6c56ba5640 1 -- 172.21.15.45:0/235450818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c5015f380 msgr2=0x7f6c500d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:30.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.715+0000 7f6c56ba5640 1 --2- 172.21.15.45:0/235450818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c5015f380 0x7f6c500d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.715+0000 7f6c56ba5640 1 -- 172.21.15.45:0/235450818 >> v1:172.21.15.63:6789/0 conn(0x7f6c500d0cf0 legacy=0x7f6c500d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:30.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.715+0000 7f6c56ba5640 1 -- 172.21.15.45:0/235450818 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c500d1a40 con 0x7f6c500d3cd0 2024-05-08T05:42:30.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.716+0000 7f6c56ba5640 1 --2- 172.21.15.45:0/235450818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c500d3cd0 0x7f6c500b7e30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6c480097a0 tx=0x7f6c4802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df3e580b6f16cb4e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:30.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.716+0000 7f6c553a2640 1 -- 172.21.15.45:0/235450818 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48037070 con 0x7f6c500d3cd0 2024-05-08T05:42:30.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.716+0000 7f6c553a2640 1 -- 172.21.15.45:0/235450818 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c48032440 con 0x7f6c500d3cd0 2024-05-08T05:42:30.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.716+0000 7f6c553a2640 1 -- 172.21.15.45:0/235450818 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48032810 con 0x7f6c500d3cd0 2024-05-08T05:42:30.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.716+0000 7f6c58608640 1 -- 172.21.15.45:0/235450818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c500d3cd0 msgr2=0x7f6c500b7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:30.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.716+0000 7f6c58608640 1 --2- 172.21.15.45:0/235450818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c500d3cd0 0x7f6c500b7e30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6c480097a0 tx=0x7f6c4802a180 comp rx=0 tx=0).stop 2024-05-08T05:42:30.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.716+0000 7f6c58608640 1 -- 172.21.15.45:0/235450818 shutdown_connections 2024-05-08T05:42:30.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.716+0000 7f6c58608640 1 --2- 172.21.15.45:0/235450818 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c500d3cd0 0x7f6c500b7e30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.716+0000 7f6c58608640 1 --2- 172.21.15.45:0/235450818 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c5015f380 0x7f6c500d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.716+0000 7f6c58608640 1 -- 172.21.15.45:0/235450818 >> 172.21.15.45:0/235450818 conn(0x7f6c500d4fe0 msgr2=0x7f6c500d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:30.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.716+0000 7f6c58608640 1 -- 172.21.15.45:0/235450818 shutdown_connections 2024-05-08T05:42:30.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.717+0000 7f6c58608640 1 -- 172.21.15.45:0/235450818 wait complete. 2024-05-08T05:42:30.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.718+0000 7f6c58608640 1 Processor -- start 2024-05-08T05:42:30.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.718+0000 7f6c58608640 1 -- start start 2024-05-08T05:42:30.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.718+0000 7f6c58608640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c500d0cf0 0x7f6c500a26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.718+0000 7f6c58608640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c500d3cd0 0x7f6c500a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.718+0000 7f6c58608640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c5015f380 0x7f6c500a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.718+0000 7f6c58608640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6c500d2000 con 0x7f6c500d3cd0 2024-05-08T05:42:30.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.718+0000 7f6c58608640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6c500d2370 con 0x7f6c5015f380 2024-05-08T05:42:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.718+0000 7f6c58608640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6c5015f760 con 0x7f6c500d0cf0 2024-05-08T05:42:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.719+0000 7f6c55ba3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c500d3cd0 0x7f6c500a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.719+0000 7f6c55ba3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c500d3cd0 0x7f6c500a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57314/0 (socket says 172.21.15.45:57314) 2024-05-08T05:42:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.719+0000 7f6c55ba3640 1 -- 172.21.15.45:0/1427062114 learned_addr learned my addr 172.21.15.45:0/1427062114 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.719+0000 7f6c56ba5640 1 --2- 172.21.15.45:0/1427062114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c5015f380 0x7f6c500a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.719+0000 7f6c563a4640 1 --2- 172.21.15.45:0/1427062114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c500d0cf0 0x7f6c500a26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.719+0000 7f6c55ba3640 1 -- 172.21.15.45:0/1427062114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c500d0cf0 msgr2=0x7f6c500a26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.719+0000 7f6c55ba3640 1 --2- 172.21.15.45:0/1427062114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c500d0cf0 0x7f6c500a26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.719+0000 7f6c55ba3640 1 -- 172.21.15.45:0/1427062114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c5015f380 msgr2=0x7f6c500a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.719+0000 7f6c55ba3640 1 --2- 172.21.15.45:0/1427062114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c5015f380 0x7f6c500a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.719+0000 7f6c55ba3640 1 -- 172.21.15.45:0/1427062114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c500a13d0 con 0x7f6c500d3cd0 2024-05-08T05:42:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.719+0000 7f6c563a4640 1 --2- 172.21.15.45:0/1427062114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c500d0cf0 0x7f6c500a26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.719+0000 7f6c55ba3640 1 --2- 172.21.15.45:0/1427062114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c500d3cd0 0x7f6c500a0950 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000c4d0 tx=0x7f6c4000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.719+0000 7f6c46ffd640 1 -- 172.21.15.45:0/1427062114 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40017070 con 0x7f6c500d3cd0 2024-05-08T05:42:30.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.719+0000 7f6c46ffd640 1 -- 172.21.15.45:0/1427062114 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c40005d00 con 0x7f6c500d3cd0 2024-05-08T05:42:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.719+0000 7f6c46ffd640 1 -- 172.21.15.45:0/1427062114 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c400072b0 con 0x7f6c500d3cd0 2024-05-08T05:42:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.719+0000 7f6c58608640 1 -- 172.21.15.45:0/1427062114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c500a1640 con 0x7f6c500d3cd0 2024-05-08T05:42:30.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.719+0000 7f6c58608640 1 -- 172.21.15.45:0/1427062114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c500a1c00 con 0x7f6c500d3cd0 2024-05-08T05:42:30.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.721+0000 7f6c46ffd640 1 -- 172.21.15.45:0/1427062114 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6c4001a070 con 0x7f6c500d3cd0 2024-05-08T05:42:30.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.721+0000 7f6c46ffd640 1 --2- 172.21.15.45:0/1427062114 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6c340762d0 0x7f6c34078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.722+0000 7f6c46ffd640 1 -- 172.21.15.45:0/1427062114 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f6c40096b40 con 0x7f6c500d3cd0 2024-05-08T05:42:30.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.722+0000 7f6c563a4640 1 --2- 172.21.15.45:0/1427062114 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6c340762d0 0x7f6c34078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.722+0000 7f6c563a4640 1 --2- 172.21.15.45:0/1427062114 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6c340762d0 0x7f6c34078790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c007440 tx=0x7f6c4c00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:30.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.722+0000 7f6c5b3f4740 1 Processor -- start 2024-05-08T05:42:30.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.722+0000 7f6c5b3f4740 1 -- start start 2024-05-08T05:42:30.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.723+0000 7f6c5b3f4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6cb1f50 0x55b0a6cb4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.723+0000 7f6c5b3f4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 0x55b0a6cb4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.723+0000 7f6c5b3f4740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b0a6c84bb0 0x55b0a6c883c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.723+0000 7f6c5b3f4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b0a6bee8d0 con 0x55b0a6cb2330 2024-05-08T05:42:30.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.723+0000 7f6c5b3f4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b0a6ba7c10 con 0x55b0a6cb1f50 2024-05-08T05:42:30.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.723+0000 7f6c5b3f4740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55b0a6bea890 con 0x55b0a6c84bb0 2024-05-08T05:42:30.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.723+0000 7f6c55ba3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 0x55b0a6cb4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.723+0000 7f6c55ba3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 0x55b0a6cb4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57322/0 (socket says 172.21.15.45:57322) 2024-05-08T05:42:30.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.723+0000 7f6c55ba3640 1 -- 172.21.15.45:0/557237625 learned_addr learned my addr 172.21.15.45:0/557237625 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:30.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.723+0000 7f6c563a4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6cb1f50 0x55b0a6cb4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.723+0000 7f6c56ba5640 1 --2- 172.21.15.45:0/557237625 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b0a6c84bb0 0x55b0a6c883c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.724+0000 7f6c55ba3640 1 -- 172.21.15.45:0/557237625 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b0a6c84bb0 msgr2=0x55b0a6c883c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.724+0000 7f6c55ba3640 1 --2- 172.21.15.45:0/557237625 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b0a6c84bb0 0x55b0a6c883c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.724+0000 7f6c55ba3640 1 -- 172.21.15.45:0/557237625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6cb1f50 msgr2=0x55b0a6cb4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.724+0000 7f6c55ba3640 1 --2- 172.21.15.45:0/557237625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6cb1f50 0x55b0a6cb4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.724+0000 7f6c55ba3640 1 -- 172.21.15.45:0/557237625 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0a6baf5d0 con 0x55b0a6cb2330 2024-05-08T05:42:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.724+0000 7f6c563a4640 1 --2- 172.21.15.45:0/557237625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6cb1f50 0x55b0a6cb4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:42:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.724+0000 7f6c55ba3640 1 --2- 172.21.15.45:0/557237625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 0x55b0a6cb4ca0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6c4005cdf0 tx=0x7f6c4005f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7430a9c07cf05292 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.724+0000 7f6c33fff640 1 -- 172.21.15.45:0/557237625 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c400682a0 con 0x55b0a6cb2330 2024-05-08T05:42:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.724+0000 7f6c33fff640 1 -- 172.21.15.45:0/557237625 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c4005fb80 con 0x55b0a6cb2330 2024-05-08T05:42:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.724+0000 7f6c33fff640 1 -- 172.21.15.45:0/557237625 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c400812d0 con 0x55b0a6cb2330 2024-05-08T05:42:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.725+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/557237625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 msgr2=0x55b0a6cb4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.725+0000 7f6c5b3f4740 1 --2- 172.21.15.45:0/557237625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 0x55b0a6cb4ca0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6c4005cdf0 tx=0x7f6c4005f690 comp rx=0 tx=0).stop 2024-05-08T05:42:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.725+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/557237625 shutdown_connections 2024-05-08T05:42:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.725+0000 7f6c5b3f4740 1 --2- 172.21.15.45:0/557237625 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b0a6c84bb0 0x55b0a6c883c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.725+0000 7f6c5b3f4740 1 --2- 172.21.15.45:0/557237625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 0x55b0a6cb4ca0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.725+0000 7f6c5b3f4740 1 --2- 172.21.15.45:0/557237625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6cb1f50 0x55b0a6cb4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.725+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/557237625 >> 172.21.15.45:0/557237625 conn(0x55b0a6c86e20 msgr2=0x55b0a6ccf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:30.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.725+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/557237625 shutdown_connections 2024-05-08T05:42:30.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.725+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/557237625 wait complete. 2024-05-08T05:42:30.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.725+0000 7f6c5b3f4740 1 -- start start 2024-05-08T05:42:30.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.725+0000 7f6c5b3f4740 1 Processor -- start 2024-05-08T05:42:30.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.726+0000 7f6c5b3f4740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b0a6cb1f50 0x55b0a6ca89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.726+0000 7f6c5b3f4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 0x55b0a6cb39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.726+0000 7f6c5b3f4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6ca8fd0 0x55b0a6cb3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.726+0000 7f6c5b3f4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b0a6c89fd0 con 0x55b0a6cb2330 2024-05-08T05:42:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.726+0000 7f6c5b3f4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b0a6bab760 con 0x55b0a6ca8fd0 2024-05-08T05:42:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.726+0000 7f6c5b3f4740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55b0a6c8a2b0 con 0x55b0a6cb1f50 2024-05-08T05:42:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.726+0000 7f6c5b3f4740 10 client.0 _ll_register_callbacks cb 0x55b0a6be4cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:42:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.726+0000 7f6c55ba3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 0x55b0a6cb39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.726+0000 7f6c55ba3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 0x55b0a6cb39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57324/0 (socket says 172.21.15.45:57324) 2024-05-08T05:42:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.726+0000 7f6c55ba3640 1 -- 172.21.15.45:0/1558420195 learned_addr learned my addr 172.21.15.45:0/1558420195 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c56ba5640 1 --2- 172.21.15.45:0/1558420195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6ca8fd0 0x55b0a6cb3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c563a4640 1 --2- 172.21.15.45:0/1558420195 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b0a6cb1f50 0x55b0a6ca89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c55ba3640 1 -- 172.21.15.45:0/1558420195 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b0a6cb1f50 msgr2=0x55b0a6ca89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c55ba3640 1 --2- 172.21.15.45:0/1558420195 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b0a6cb1f50 0x55b0a6ca89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c55ba3640 1 -- 172.21.15.45:0/1558420195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6ca8fd0 msgr2=0x55b0a6cb3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:30.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c55ba3640 1 --2- 172.21.15.45:0/1558420195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6ca8fd0 0x55b0a6cb3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c55ba3640 1 -- 172.21.15.45:0/1558420195 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0a69002d0 con 0x55b0a6cb2330 2024-05-08T05:42:30.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c56ba5640 1 --2- 172.21.15.45:0/1558420195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6ca8fd0 0x55b0a6cb3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:30.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c55ba3640 1 --2- 172.21.15.45:0/1558420195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 0x55b0a6cb39e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6c4005cdf0 tx=0x7f6c40000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e978c5864a142b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:30.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c317fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:42:30.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c317fa640 1 -- 172.21.15.45:0/1558420195 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c400682a0 con 0x55b0a6cb2330 2024-05-08T05:42:30.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c317fa640 1 -- 172.21.15.45:0/1558420195 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f6c4007dc70 con 0x55b0a6cb2330 2024-05-08T05:42:30.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/1558420195 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c4005fb80 con 0x55b0a6cb2330 2024-05-08T05:42:30.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c317fa640 1 -- 172.21.15.45:0/1558420195 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f6c40093050 con 0x55b0a6cb2330 2024-05-08T05:42:30.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c5b3f4740 10 client.5070 fetch_fsmap learned FSMap version 17 2024-05-08T05:42:30.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/1558420195 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55b0a69ca6b0 con 0x55b0a6cb2330 2024-05-08T05:42:30.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c317fa640 1 -- 172.21.15.45:0/1558420195 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c4005fb80 con 0x55b0a6cb2330 2024-05-08T05:42:30.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c317fa640 1 -- 172.21.15.45:0/1558420195 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f6c40007450 con 0x55b0a6cb2330 2024-05-08T05:42:30.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c5b3f4740 10 client.5070 fetch_fsmap finished waiting for FSMap version 17 2024-05-08T05:42:30.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c5b3f4740 10 client.5070 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T05:42:30.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c5b3f4740 20 client.5070 populate_metadata read hostname 'smithi045' 2024-05-08T05:42:30.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c5b3f4740 1 --2- 172.21.15.45:0/1558420195 >> [v2:172.21.15.45:6833/173436628,v1:172.21.15.45:6835/173436628] conn(0x55b0a6cb6370 0x55b0a6cb6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c5b3f4740 4 client.5070 mds_command: new command op to 4884 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:42:30.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.727+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/1558420195 --> [v2:172.21.15.45:6833/173436628,v1:172.21.15.45:6835/173436628] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0a6b72760 con 0x55b0a6cb6370 2024-05-08T05:42:30.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.728+0000 7f6c563a4640 1 --2- 172.21.15.45:0/1558420195 >> [v2:172.21.15.45:6833/173436628,v1:172.21.15.45:6835/173436628] conn(0x55b0a6cb6370 0x55b0a6cb6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.728+0000 7f6c317fa640 10 client.5070 ms_handle_connect on v2:172.21.15.45:6833/173436628 2024-05-08T05:42:30.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.728+0000 7f6c563a4640 1 --2- 172.21.15.45:0/1558420195 >> [v2:172.21.15.45:6833/173436628,v1:172.21.15.45:6835/173436628] conn(0x55b0a6cb6370 0x55b0a6cb6750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31c6260848be0e50 server_cookie=768a441c04d3c5ee in_seq=0 out_seq=0 2024-05-08T05:42:30.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.728+0000 7f6c317fa640 1 -- 172.21.15.45:0/1558420195 <== mds.0 v2:172.21.15.45:6833/173436628 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b0a6b72760 con 0x55b0a6cb6370 2024-05-08T05:42:30.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.728+0000 7f6c317fa640 10 client.5070 handle_command_reply: tid=0 2024-05-08T05:42:30.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.728+0000 7f6c5b3f4740 1 client.5070 shutdown 2024-05-08T05:42:30.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.729+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/1558420195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 msgr2=0x55b0a6cb39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.729+0000 7f6c5b3f4740 1 --2- 172.21.15.45:0/1558420195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 0x55b0a6cb39e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6c4005cdf0 tx=0x7f6c40000bc0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.729+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/1558420195 shutdown_connections 2024-05-08T05:42:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.729+0000 7f6c5b3f4740 1 --2- 172.21.15.45:0/1558420195 >> [v2:172.21.15.45:6833/173436628,v1:172.21.15.45:6835/173436628] conn(0x55b0a6cb6370 0x55b0a6cb6750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.729+0000 7f6c5b3f4740 1 --2- 172.21.15.45:0/1558420195 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6ca8fd0 0x55b0a6cb3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.729+0000 7f6c5b3f4740 1 --2- 172.21.15.45:0/1558420195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 0x55b0a6cb39e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.729+0000 7f6c317fa640 0 client.5070 ms_handle_reset on v2:172.21.15.45:6833/173436628 2024-05-08T05:42:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.729+0000 7f6c5b3f4740 1 --2- 172.21.15.45:0/1558420195 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b0a6cb1f50 0x55b0a6ca89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.729+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/1558420195 >> 172.21.15.45:0/1558420195 conn(0x55b0a6c84bb0 msgr2=0x55b0a6c85860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.729+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/1558420195 shutdown_connections 2024-05-08T05:42:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.729+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/1558420195 wait complete. 2024-05-08T05:42:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.729+0000 7f6c5b3f4740 20 client.5070 trim_cache size 0 max 16384 2024-05-08T05:42:30.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.748+0000 7f6c5b3f4740 1 Processor -- start 2024-05-08T05:42:30.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.748+0000 7f6c5b3f4740 1 -- start start 2024-05-08T05:42:30.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.749+0000 7f6c5b3f4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6ca8fd0 0x55b0a6cd1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.749+0000 7f6c5b3f4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6cb1f50 0x55b0a6cd24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.749+0000 7f6c5b3f4740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b0a6cb2330 0x55b0a6cb3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.749+0000 7f6c5b3f4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6c50200f00 con 0x55b0a6ca8fd0 2024-05-08T05:42:30.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.749+0000 7f6c5b3f4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6c50200260 con 0x55b0a6cb1f50 2024-05-08T05:42:30.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.749+0000 7f6c5b3f4740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6c502008b0 con 0x55b0a6cb2330 2024-05-08T05:42:30.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.749+0000 7f6c563a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6ca8fd0 0x55b0a6cd1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.749+0000 7f6c563a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6ca8fd0 0x55b0a6cd1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57330/0 (socket says 172.21.15.45:57330) 2024-05-08T05:42:30.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.749+0000 7f6c563a4640 1 -- 172.21.15.45:0/3726640747 learned_addr learned my addr 172.21.15.45:0/3726640747 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:30.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.749+0000 7f6c563a4640 1 -- 172.21.15.45:0/3726640747 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b0a6cb2330 msgr2=0x55b0a6cb3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:30.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.749+0000 7f6c563a4640 1 --2- 172.21.15.45:0/3726640747 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b0a6cb2330 0x55b0a6cb3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.749+0000 7f6c563a4640 1 -- 172.21.15.45:0/3726640747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6cb1f50 msgr2=0x55b0a6cd24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:30.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.749+0000 7f6c563a4640 1 --2- 172.21.15.45:0/3726640747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6cb1f50 0x55b0a6cd24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.749+0000 7f6c563a4640 1 -- 172.21.15.45:0/3726640747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0a6cd2b60 con 0x55b0a6ca8fd0 2024-05-08T05:42:30.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.749+0000 7f6c563a4640 1 --2- 172.21.15.45:0/3726640747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6ca8fd0 0x55b0a6cd1fb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55b0a6cb5b30 tx=0x7f6c4c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5850d118625d5bc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:30.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.749+0000 7f6c337fe640 1 -- 172.21.15.45:0/3726640747 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c018db0 con 0x55b0a6ca8fd0 2024-05-08T05:42:30.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.749+0000 7f6c337fe640 1 -- 172.21.15.45:0/3726640747 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c4c029030 con 0x55b0a6ca8fd0 2024-05-08T05:42:30.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.749+0000 7f6c337fe640 1 -- 172.21.15.45:0/3726640747 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c018350 con 0x55b0a6ca8fd0 2024-05-08T05:42:30.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.750+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/3726640747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6ca8fd0 msgr2=0x55b0a6cd1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:30.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.750+0000 7f6c5b3f4740 1 --2- 172.21.15.45:0/3726640747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6ca8fd0 0x55b0a6cd1fb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55b0a6cb5b30 tx=0x7f6c4c012000 comp rx=0 tx=0).stop 2024-05-08T05:42:30.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.750+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/3726640747 shutdown_connections 2024-05-08T05:42:30.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.750+0000 7f6c5b3f4740 1 --2- 172.21.15.45:0/3726640747 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b0a6cb2330 0x55b0a6cb3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.750+0000 7f6c5b3f4740 1 --2- 172.21.15.45:0/3726640747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6cb1f50 0x55b0a6cd24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.750+0000 7f6c5b3f4740 1 --2- 172.21.15.45:0/3726640747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6ca8fd0 0x55b0a6cd1fb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.750+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/3726640747 >> 172.21.15.45:0/3726640747 conn(0x55b0a6c84bb0 msgr2=0x55b0a6ccc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:30.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.750+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/3726640747 shutdown_connections 2024-05-08T05:42:30.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.750+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/3726640747 wait complete. 2024-05-08T05:42:30.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.750+0000 7f6c5b3f4740 1 -- start start 2024-05-08T05:42:30.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.750+0000 7f6c5b3f4740 1 Processor -- start 2024-05-08T05:42:30.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.751+0000 7f6c5b3f4740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b0a6ca8fd0 0x55b0a6d50170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.751+0000 7f6c5b3f4740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6cb1f50 0x55b0a6cc5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.751+0000 7f6c5b3f4740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 0x55b0a6cc8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.751+0000 7f6c5b3f4740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b0a6c89fd0 con 0x55b0a6cb2330 2024-05-08T05:42:30.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.751+0000 7f6c5b3f4740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b0a6c8a2b0 con 0x55b0a6cb1f50 2024-05-08T05:42:30.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.751+0000 7f6c5b3f4740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55b0a6bab760 con 0x55b0a6ca8fd0 2024-05-08T05:42:30.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.751+0000 7f6c5b3f4740 10 client.0 _ll_register_callbacks cb 0x7f6c4c009950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:42:30.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.751+0000 7f6c56ba5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 0x55b0a6cc8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.752+0000 7f6c56ba5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 0x55b0a6cc8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57346/0 (socket says 172.21.15.45:57346) 2024-05-08T05:42:30.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.752+0000 7f6c56ba5640 1 -- 172.21.15.45:0/17168000 learned_addr learned my addr 172.21.15.45:0/17168000 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:30.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.752+0000 7f6c55ba3640 1 --2- 172.21.15.45:0/17168000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6cb1f50 0x55b0a6cc5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.752+0000 7f6c563a4640 1 --2- 172.21.15.45:0/17168000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b0a6ca8fd0 0x55b0a6d50170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.752+0000 7f6c56ba5640 1 -- 172.21.15.45:0/17168000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b0a6ca8fd0 msgr2=0x55b0a6d50170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:30.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.752+0000 7f6c56ba5640 1 --2- 172.21.15.45:0/17168000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b0a6ca8fd0 0x55b0a6d50170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.752+0000 7f6c56ba5640 1 -- 172.21.15.45:0/17168000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6cb1f50 msgr2=0x55b0a6cc5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:30.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.752+0000 7f6c56ba5640 1 --2- 172.21.15.45:0/17168000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6cb1f50 0x55b0a6cc5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.752+0000 7f6c56ba5640 1 -- 172.21.15.45:0/17168000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0a6ca8930 con 0x55b0a6cb2330 2024-05-08T05:42:30.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.752+0000 7f6c56ba5640 1 --2- 172.21.15.45:0/17168000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 0x55b0a6cc8dd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6c480097a0 tx=0x7f6c480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cb99c1a48b2161 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:30.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.752+0000 7f6c317fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:42:30.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.752+0000 7f6c317fa640 1 -- 172.21.15.45:0/17168000 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48037040 con 0x55b0a6cb2330 2024-05-08T05:42:30.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.752+0000 7f6c317fa640 1 -- 172.21.15.45:0/17168000 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f6c48046440 con 0x55b0a6cb2330 2024-05-08T05:42:30.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.752+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/17168000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c4c029030 con 0x55b0a6cb2330 2024-05-08T05:42:30.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.752+0000 7f6c317fa640 1 -- 172.21.15.45:0/17168000 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f6c4804e050 con 0x55b0a6cb2330 2024-05-08T05:42:30.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.752+0000 7f6c5b3f4740 10 client.5076 fetch_fsmap learned FSMap version 17 2024-05-08T05:42:30.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.752+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/17168000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55b0a6c89500 con 0x55b0a6cb2330 2024-05-08T05:42:30.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.752+0000 7f6c317fa640 1 -- 172.21.15.45:0/17168000 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c4c029030 con 0x55b0a6cb2330 2024-05-08T05:42:30.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.753+0000 7f6c317fa640 1 -- 172.21.15.45:0/17168000 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f6c48051020 con 0x55b0a6cb2330 2024-05-08T05:42:30.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.753+0000 7f6c5b3f4740 10 client.5076 fetch_fsmap finished waiting for FSMap version 17 2024-05-08T05:42:30.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.753+0000 7f6c5b3f4740 10 client.5076 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T05:42:30.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.753+0000 7f6c5b3f4740 20 client.5076 populate_metadata read hostname 'smithi045' 2024-05-08T05:42:30.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.753+0000 7f6c5b3f4740 1 --2- 172.21.15.45:0/17168000 >> [v2:172.21.15.45:6833/173436628,v1:172.21.15.45:6835/173436628] conn(0x55b0a6cf2b50 0x55b0a6cf2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.753+0000 7f6c5b3f4740 4 client.5076 mds_command: new command op to 4884 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:42:30.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.753+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/17168000 --> [v2:172.21.15.45:6833/173436628,v1:172.21.15.45:6835/173436628] -- command(tid 0: {"prefix": "status"}) -- 0x55b0a6bfc4e0 con 0x55b0a6cf2b50 2024-05-08T05:42:30.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.753+0000 7f6c563a4640 1 --2- 172.21.15.45:0/17168000 >> [v2:172.21.15.45:6833/173436628,v1:172.21.15.45:6835/173436628] conn(0x55b0a6cf2b50 0x55b0a6cf2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.753+0000 7f6c563a4640 1 --2- 172.21.15.45:0/17168000 >> [v2:172.21.15.45:6833/173436628,v1:172.21.15.45:6835/173436628] conn(0x55b0a6cf2b50 0x55b0a6cf2f30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67323ac1089770e3 server_cookie=5c5788af7c40b68c in_seq=0 out_seq=0 2024-05-08T05:42:30.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.753+0000 7f6c317fa640 10 client.5076 ms_handle_connect on v2:172.21.15.45:6833/173436628 2024-05-08T05:42:30.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.753+0000 7f6c317fa640 1 -- 172.21.15.45:0/17168000 <== mds.0 v2:172.21.15.45:6833/173436628 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55b0a6bfc4e0 con 0x55b0a6cf2b50 2024-05-08T05:42:30.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.753+0000 7f6c317fa640 10 client.5076 handle_command_reply: tid=0 2024-05-08T05:42:30.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.753+0000 7f6c5b3f4740 1 client.5076 shutdown 2024-05-08T05:42:30.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.753+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/17168000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 msgr2=0x55b0a6cc8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:30.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.753+0000 7f6c5b3f4740 1 --2- 172.21.15.45:0/17168000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 0x55b0a6cc8dd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6c480097a0 tx=0x7f6c480046c0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/17168000 shutdown_connections 2024-05-08T05:42:30.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c5b3f4740 1 --2- 172.21.15.45:0/17168000 >> [v2:172.21.15.45:6833/173436628,v1:172.21.15.45:6835/173436628] conn(0x55b0a6cf2b50 0x55b0a6cf2f30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c5b3f4740 1 --2- 172.21.15.45:0/17168000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b0a6cb2330 0x55b0a6cc8dd0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c5b3f4740 1 --2- 172.21.15.45:0/17168000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b0a6cb1f50 0x55b0a6cc5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c5b3f4740 1 --2- 172.21.15.45:0/17168000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b0a6ca8fd0 0x55b0a6d50170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c317fa640 0 client.5076 ms_handle_reset on v2:172.21.15.45:6833/173436628 2024-05-08T05:42:30.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/17168000 >> 172.21.15.45:0/17168000 conn(0x55b0a6c84bb0 msgr2=0x55b0a6cb16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:30.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/17168000 shutdown_connections 2024-05-08T05:42:30.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c5b3f4740 1 -- 172.21.15.45:0/17168000 wait complete. 2024-05-08T05:42:30.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c5b3f4740 20 client.5076 trim_cache size 0 max 16384 2024-05-08T05:42:30.760 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:42:30.760 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "e9e1170a-fd27-4625-97a9-5d5f104d13a2", 2024-05-08T05:42:30.760 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-05-08T05:42:30.760 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 4884, 2024-05-08T05:42:30.760 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-05-08T05:42:30.760 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-05-08T05:42:30.760 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-05-08T05:42:30.760 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 10.253515842000001, 2024-05-08T05:42:30.760 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 17, 2024-05-08T05:42:30.760 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 36, 2024-05-08T05:42:30.760 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 36, 2024-05-08T05:42:30.761 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 17.141030909000001 2024-05-08T05:42:30.761 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:42:30.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c58608640 1 -- 172.21.15.45:0/1427062114 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6c340762d0 msgr2=0x7f6c34078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:30.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c58608640 1 --2- 172.21.15.45:0/1427062114 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6c340762d0 0x7f6c34078790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c007440 tx=0x7f6c4c00b040 comp rx=0 tx=0).stop 2024-05-08T05:42:30.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c58608640 1 -- 172.21.15.45:0/1427062114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c500d3cd0 msgr2=0x7f6c500a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:30.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c58608640 1 --2- 172.21.15.45:0/1427062114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c500d3cd0 0x7f6c500a0950 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000c4d0 tx=0x7f6c4000c990 comp rx=0 tx=0).stop 2024-05-08T05:42:30.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c58608640 1 -- 172.21.15.45:0/1427062114 shutdown_connections 2024-05-08T05:42:30.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c58608640 1 --2- 172.21.15.45:0/1427062114 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6c340762d0 0x7f6c34078790 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c58608640 1 --2- 172.21.15.45:0/1427062114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c5015f380 0x7f6c500a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c58608640 1 --2- 172.21.15.45:0/1427062114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c500d3cd0 0x7f6c500a0950 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c58608640 1 --2- 172.21.15.45:0/1427062114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c500d0cf0 0x7f6c500a26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c58608640 1 -- 172.21.15.45:0/1427062114 >> 172.21.15.45:0/1427062114 conn(0x7f6c500d4fe0 msgr2=0x7f6c500b6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:30.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c58608640 1 -- 172.21.15.45:0/1427062114 shutdown_connections 2024-05-08T05:42:30.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.754+0000 7f6c58608640 1 -- 172.21.15.45:0/1427062114 wait complete. 2024-05-08T05:42:30.766 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T05:42:30.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.870+0000 7f72ae474640 1 Processor -- start 2024-05-08T05:42:30.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.870+0000 7f72ae474640 1 -- start start 2024-05-08T05:42:30.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.871+0000 7f72ae474640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f72a806fc30 0x7f72a80af740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.871+0000 7f72ae474640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f72a80b48a0 0x7f72a80afc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.871+0000 7f72ae474640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72a80701d0 con 0x7f72a8157540 2024-05-08T05:42:30.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.871+0000 7f72ae474640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f72a80b01c0 con 0x7f72a80b48a0 2024-05-08T05:42:30.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.871+0000 7f72ae474640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f72a80b3dc0 con 0x7f72a806fc30 2024-05-08T05:42:30.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.871+0000 7f72aca11640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f72a80b48a0 0x7f72a80afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.871+0000 7f72a7fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f72a8157540 0x7f72a815e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60580/0 (socket says 172.21.15.45:60580) 2024-05-08T05:42:30.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.871+0000 7f72a7fff640 1 -- 172.21.15.45:0/3165987474 learned_addr learned my addr 172.21.15.45:0/3165987474 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:30.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.871+0000 7f729ffff640 1 --2- 172.21.15.45:0/3165987474 >> v2:172.21.15.63:3300/0 conn(0x7f72a806fc30 0x7f72a80af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.871+0000 7f72aca11640 1 -- 172.21.15.45:0/3165987474 >> v2:172.21.15.63:3300/0 conn(0x7f72a806fc30 msgr2=0x7f72a80af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:30.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.871+0000 7f72aca11640 1 --2- 172.21.15.45:0/3165987474 >> v2:172.21.15.63:3300/0 conn(0x7f72a806fc30 0x7f72a80af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.871+0000 7f72aca11640 1 -- 172.21.15.45:0/3165987474 >> v1:172.21.15.45:6789/0 conn(0x7f72a8157540 legacy=0x7f72a815e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:30.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.871+0000 7f72a77fe640 1 -- 172.21.15.45:0/3165987474 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 152826662 0 0) 0x7f72a80701d0 con 0x7f72a8157540 2024-05-08T05:42:30.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.871+0000 7f72aca11640 1 -- 172.21.15.45:0/3165987474 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a80b40a0 con 0x7f72a80b48a0 2024-05-08T05:42:30.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.872+0000 7f72aca11640 1 --2- 172.21.15.45:0/3165987474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72a80b48a0 0x7f72a80afc80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f72a8160ac0 tx=0x7f72a002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18adc2900cca4897 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:30.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.872+0000 7f72a77fe640 1 -- 172.21.15.45:0/3165987474 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0007740 con 0x7f72a80b48a0 2024-05-08T05:42:30.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.872+0000 7f72a77fe640 1 -- 172.21.15.45:0/3165987474 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72a00078e0 con 0x7f72a80b48a0 2024-05-08T05:42:30.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.872+0000 7f72a77fe640 1 -- 172.21.15.45:0/3165987474 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0007cb0 con 0x7f72a80b48a0 2024-05-08T05:42:30.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.873+0000 7f72ae474640 1 -- 172.21.15.45:0/3165987474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72a80b48a0 msgr2=0x7f72a80afc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:30.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.873+0000 7f72ae474640 1 --2- 172.21.15.45:0/3165987474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72a80b48a0 0x7f72a80afc80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f72a8160ac0 tx=0x7f72a002a680 comp rx=0 tx=0).stop 2024-05-08T05:42:30.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.873+0000 7f72ae474640 1 -- 172.21.15.45:0/3165987474 shutdown_connections 2024-05-08T05:42:30.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.873+0000 7f72ae474640 1 --2- 172.21.15.45:0/3165987474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72a80b48a0 0x7f72a80afc80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.873+0000 7f72ae474640 1 --2- 172.21.15.45:0/3165987474 >> v2:172.21.15.63:3300/0 conn(0x7f72a806fc30 0x7f72a80af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.873+0000 7f72ae474640 1 -- 172.21.15.45:0/3165987474 >> 172.21.15.45:0/3165987474 conn(0x7f72a806c3a0 msgr2=0x7f72a806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:30.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.873+0000 7f72ae474640 1 -- 172.21.15.45:0/3165987474 shutdown_connections 2024-05-08T05:42:30.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.873+0000 7f72ae474640 1 -- 172.21.15.45:0/3165987474 wait complete. 2024-05-08T05:42:30.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.874+0000 7f72ae474640 1 Processor -- start 2024-05-08T05:42:30.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.874+0000 7f72ae474640 1 -- start start 2024-05-08T05:42:30.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.875+0000 7f72ae474640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72a806fc30 0x7f72a809ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.875+0000 7f72ae474640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72a80b48a0 0x7f72a8145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.875+0000 7f72ae474640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72a8157540 0x7f72a809c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.875+0000 7f72ae474640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f72a8157aa0 con 0x7f72a80b48a0 2024-05-08T05:42:30.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.875+0000 7f72ae474640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f72a8157920 con 0x7f72a806fc30 2024-05-08T05:42:30.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.875+0000 7f72ae474640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f72a80ced20 con 0x7f72a8157540 2024-05-08T05:42:30.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.875+0000 7f729ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72a80b48a0 0x7f72a8145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.875+0000 7f729ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72a80b48a0 0x7f72a8145630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57366/0 (socket says 172.21.15.45:57366) 2024-05-08T05:42:30.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.875+0000 7f729ffff640 1 -- 172.21.15.45:0/3167841761 learned_addr learned my addr 172.21.15.45:0/3167841761 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:30.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.875+0000 7f72a7fff640 1 --2- 172.21.15.45:0/3167841761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72a806fc30 0x7f72a809ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.875+0000 7f72aca11640 1 --2- 172.21.15.45:0/3167841761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72a8157540 0x7f72a809c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.875+0000 7f729ffff640 1 -- 172.21.15.45:0/3167841761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72a8157540 msgr2=0x7f72a809c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:30.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.875+0000 7f729ffff640 1 --2- 172.21.15.45:0/3167841761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72a8157540 0x7f72a809c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.875+0000 7f729ffff640 1 -- 172.21.15.45:0/3167841761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72a806fc30 msgr2=0x7f72a809ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:30.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.875+0000 7f729ffff640 1 --2- 172.21.15.45:0/3167841761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72a806fc30 0x7f72a809ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:30.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.875+0000 7f729ffff640 1 -- 172.21.15.45:0/3167841761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a809e9a0 con 0x7f72a80b48a0 2024-05-08T05:42:30.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.875+0000 7f72aca11640 1 --2- 172.21.15.45:0/3167841761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72a8157540 0x7f72a809c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:30.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.876+0000 7f729ffff640 1 --2- 172.21.15.45:0/3167841761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72a80b48a0 0x7f72a8145630 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f729000e9b0 tx=0x7f729000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:30.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.876+0000 7f72a57fa640 1 -- 172.21.15.45:0/3167841761 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290017070 con 0x7f72a80b48a0 2024-05-08T05:42:30.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.876+0000 7f72a57fa640 1 -- 172.21.15.45:0/3167841761 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f729000c9a0 con 0x7f72a80b48a0 2024-05-08T05:42:30.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.876+0000 7f72ae474640 1 -- 172.21.15.45:0/3167841761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a8145b70 con 0x7f72a80b48a0 2024-05-08T05:42:30.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.876+0000 7f72a57fa640 1 -- 172.21.15.45:0/3167841761 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729000cd90 con 0x7f72a80b48a0 2024-05-08T05:42:30.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.876+0000 7f72ae474640 1 -- 172.21.15.45:0/3167841761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a81f5d40 con 0x7f72a80b48a0 2024-05-08T05:42:30.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.877+0000 7f72ae474640 1 -- 172.21.15.45:0/3167841761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a80693f0 con 0x7f72a80b48a0 2024-05-08T05:42:30.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.877+0000 7f72a57fa640 1 -- 172.21.15.45:0/3167841761 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f729001a070 con 0x7f72a80b48a0 2024-05-08T05:42:30.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.878+0000 7f72a57fa640 1 --2- 172.21.15.45:0/3167841761 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f72880762d0 0x7f7288078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:30.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.878+0000 7f72a57fa640 1 -- 172.21.15.45:0/3167841761 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f72900977a0 con 0x7f72a80b48a0 2024-05-08T05:42:30.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.878+0000 7f72a7fff640 1 --2- 172.21.15.45:0/3167841761 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f72880762d0 0x7f7288078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:30.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.879+0000 7f72a7fff640 1 --2- 172.21.15.45:0/3167841761 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f72880762d0 0x7f7288078790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7298007fd0 tx=0x7f7298007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:30.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:30.882+0000 7f72a57fa640 1 -- 172.21.15.45:0/3167841761 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7290060920 con 0x7f72a80b48a0 2024-05-08T05:42:31.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:31.113+0000 7f72ae474640 1 -- 172.21.15.45:0/3167841761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f72a81f6670 con 0x7f72a80b48a0 2024-05-08T05:42:31.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:31.114+0000 7f72a57fa640 1 -- 172.21.15.45:0/3167841761 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f7290064c20 con 0x7f72a80b48a0 2024-05-08T05:42:31.114 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T05:42:31.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:31.116+0000 7f72ae474640 1 -- 172.21.15.45:0/3167841761 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f72880762d0 msgr2=0x7f7288078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:31.116+0000 7f72ae474640 1 --2- 172.21.15.45:0/3167841761 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f72880762d0 0x7f7288078790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7298007fd0 tx=0x7f7298007de0 comp rx=0 tx=0).stop 2024-05-08T05:42:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:31.116+0000 7f72ae474640 1 -- 172.21.15.45:0/3167841761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72a80b48a0 msgr2=0x7f72a8145630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:31.116+0000 7f72ae474640 1 --2- 172.21.15.45:0/3167841761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72a80b48a0 0x7f72a8145630 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f729000e9b0 tx=0x7f729000ee70 comp rx=0 tx=0).stop 2024-05-08T05:42:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:31.116+0000 7f72ae474640 1 -- 172.21.15.45:0/3167841761 shutdown_connections 2024-05-08T05:42:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:31.116+0000 7f72ae474640 1 --2- 172.21.15.45:0/3167841761 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f72880762d0 0x7f7288078790 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:31.116+0000 7f72ae474640 1 --2- 172.21.15.45:0/3167841761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f72a8157540 0x7f72a809c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:31.116+0000 7f72ae474640 1 --2- 172.21.15.45:0/3167841761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72a80b48a0 0x7f72a8145630 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:31.116+0000 7f72ae474640 1 --2- 172.21.15.45:0/3167841761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f72a806fc30 0x7f72a809ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:31.116+0000 7f72ae474640 1 -- 172.21.15.45:0/3167841761 >> 172.21.15.45:0/3167841761 conn(0x7f72a806c3a0 msgr2=0x7f72a80b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:31.116+0000 7f72ae474640 1 -- 172.21.15.45:0/3167841761 shutdown_connections 2024-05-08T05:42:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:31.116+0000 7f72ae474640 1 -- 172.21.15.45:0/3167841761 wait complete. 2024-05-08T05:42:31.127 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T05:42:31.127 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:42:31.127 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-05-08T05:42:31.127 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T05:42:31.127 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:42:31.127 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T05:42:31.128 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T05:42:31.128 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:42:31.128 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:42:31.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:31.128 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-05-08T05:42:31.145 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:42:31.145 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:42:31.145 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:42:31.145 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:31.145 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-05-08T05:42:31.145 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:31.145 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:42:31.145 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:42:31.145 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-05-08T05:42:31.145 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:42:31.145 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:42:31.145 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-05-08T05:42:31.145 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:42:31.146 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-05-08T05:42:31.146 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-05-08T05:42:31.146 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:42:31.146 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42673sec preferred_lft 42673sec 2024-05-08T05:42:31.146 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-05-08T05:42:31.146 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:31.146 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:42:31.146 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-05-08T05:42:31.146 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-05-08T05:42:31.146 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:42:31.146 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T05:42:31.146 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T05:42:31.146 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:31.146 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c80a:aff:fe94:c77/64 scope link 2024-05-08T05:42:31.147 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:31.147 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:42:31.147 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 82:f3:eb:6a:57:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:42:31.147 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::80f3:ebff:fe6a:57ba/64 scope link 2024-05-08T05:42:31.147 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:31.147 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:42:31.147 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T05:42:31.147 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::14a2:64ff:feda:3a9c/64 scope link 2024-05-08T05:42:31.147 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:42:31.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:31.147 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:42:31.209 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:42:31.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:31.211 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-05-08T05:42:31.268 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-05-08T05:42:31.268 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:42:31.268 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:42:31.268 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:42:31.268 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:42:31.268 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:42:31.268 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:42:31.268 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:42:31.268 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:42:31.269 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:42:31.269 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:42:31.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:42:31.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:42:31.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:31.399 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:42:31.415 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T05:42:31.415 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T05:42:31.415 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:42:31.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:31.416 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-05-08T05:42:31.481 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:42:31.481 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T05:42:31.537 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:42:31.537 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:42:31.537 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:42:31.593 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T05:42:31.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:31.594 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:42:31.679 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:42:31.679 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:42:31.687 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:42:31.687 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:42:31.687 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-08T05:42:31.688 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@e9e1170a-fd27-4625-97a9-5d5f104d13a2.cephfs=/ 2024-05-08T05:42:31.688 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.63:3300/172.21.15.63:3301" will pass to kernel 2024-05-08T05:42:31.689 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T05:42:31.689 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:42:31.689 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:42:31.689 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:42:31.689 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:42:31.691 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:42:31.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:31.691 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:42:31.721 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:42:31.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:42:31.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:42:31.823 INFO:teuthology.orchestra.run.smithi045.stdout:client4742 2024-05-08T05:42:31.824 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:42:31.824 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:31.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:31.841 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-05-08T05:42:31.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:31.902 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-05-08T05:42:31.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:31.960 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-05-08T05:42:32.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:32.017 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-05-08T05:42:32.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:32.079 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-05-08T05:42:32.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:32.136 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-08T05:42:32.196 INFO:teuthology.orchestra.run.smithi045.stdout:./ 2024-05-08T05:42:32.197 INFO:teuthology.orchestra.run.smithi045.stdout:./rfile 2024-05-08T05:42:32.198 INFO:teuthology.orchestra.run.smithi045.stdout:./parentdir 2024-05-08T05:42:32.198 INFO:teuthology.orchestra.run.smithi045.stdout:./parentdir/pfile 2024-05-08T05:42:32.198 INFO:teuthology.orchestra.run.smithi045.stdout:./parentdir/childdir 2024-05-08T05:42:32.198 INFO:teuthology.orchestra.run.smithi045.stdout:./parentdir/childdir/cfile 2024-05-08T05:42:32.199 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:42:32.199 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:42:32.199 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:42:32.199 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:42:32.200 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-08T05:42:32.200 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:42:32.307 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:42:32.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:42:32.316 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:42:32.316 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:42:32.316 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:42:32.316 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:42:32.316 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-05-08T05:42:32.316 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:42:32.424 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627778 2024-05-08T05:42:32.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:42:32.433 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:42:32.433 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:42:32.433 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:42:32.433 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:42:32.433 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-05-08T05:42:32.433 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:42:32.541 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627776 2024-05-08T05:42:32.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:42:32.550 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:42:32.550 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:42:32.550 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:42:32.550 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:42:32.550 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-05-08T05:42:32.550 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:42:32.658 INFO:teuthology.orchestra.run.smithi045.stdout:1099511628279 2024-05-08T05:42:32.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:42:32.667 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:42:32.667 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:42:32.667 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:42:32.668 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:42:32.668 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-05-08T05:42:32.668 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:42:32.775 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627777 2024-05-08T05:42:32.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:42:32.785 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:42:32.785 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:42:32.785 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:42:32.785 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:42:32.785 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-05-08T05:42:32.785 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:42:32.905 INFO:teuthology.orchestra.run.smithi045.stdout:1099511628280 2024-05-08T05:42:32.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:42:32.915 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:42:32.916 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:32.978 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:42:32.978 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:33.041 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:42:33.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.192+0000 7f86648da640 1 Processor -- start 2024-05-08T05:42:33.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.192+0000 7f86648da640 1 -- start start 2024-05-08T05:42:33.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.193+0000 7f86648da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86600d0530 0x7f86600b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.193+0000 7f86648da640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f866015d2a0 con 0x7f866006e0f0 2024-05-08T05:42:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.193+0000 7f86648da640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86601609b0 con 0x7f866006f060 2024-05-08T05:42:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.193+0000 7f86648da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f866006f440 con 0x7f86600d0530 2024-05-08T05:42:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.193+0000 7f865dd9b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f866006e0f0 0x7f86600cfe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60586/0 (socket says 172.21.15.45:60586) 2024-05-08T05:42:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.193+0000 7f865dd9b640 1 -- 172.21.15.45:0/2668170847 learned_addr learned my addr 172.21.15.45:0/2668170847 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.193+0000 7f865ed9d640 1 --2- 172.21.15.45:0/2668170847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86600d0530 0x7f86600b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865d59a640 1 -- 172.21.15.45:0/2668170847 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3956520062 0 0) 0x7f866015d2a0 con 0x7f866006e0f0 2024-05-08T05:42:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865d59a640 1 -- 172.21.15.45:0/2668170847 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8648003520 con 0x7f866006e0f0 2024-05-08T05:42:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865d59a640 1 -- 172.21.15.45:0/2668170847 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3591517259 0 0) 0x7f86601609b0 con 0x7f866006f060 2024-05-08T05:42:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865d59a640 1 -- 172.21.15.45:0/2668170847 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f866015d2a0 con 0x7f866006f060 2024-05-08T05:42:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865d59a640 1 -- 172.21.15.45:0/2668170847 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1606993795 0 0) 0x7f8648003520 con 0x7f866006e0f0 2024-05-08T05:42:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865d59a640 1 -- 172.21.15.45:0/2668170847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86600d0530 msgr2=0x7f86600b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:33.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865d59a640 1 --2- 172.21.15.45:0/2668170847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86600d0530 0x7f86600b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865d59a640 1 -- 172.21.15.45:0/2668170847 >> v1:172.21.15.63:6789/0 conn(0x7f866006f060 legacy=0x7f866006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865d59a640 1 -- 172.21.15.45:0/2668170847 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8660160c90 con 0x7f866006e0f0 2024-05-08T05:42:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865d59a640 1 -- 172.21.15.45:0/2668170847 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f86540016f0 con 0x7f866006e0f0 2024-05-08T05:42:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865d59a640 1 -- 172.21.15.45:0/2668170847 >> v1:172.21.15.45:6789/0 conn(0x7f866006e0f0 legacy=0x7f86600cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865d59a640 1 --2- 172.21.15.45:0/2668170847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8648003c10 0x7f8648004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865d59a640 1 --2- 172.21.15.45:0/2668170847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8648004800 0x7f8648004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865d59a640 1 --2- 172.21.15.45:0/2668170847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86600d0530 0x7f8648005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865d59a640 1 -- 172.21.15.45:0/2668170847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f866006f440 con 0x7f8648003c10 2024-05-08T05:42:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865d59a640 1 -- 172.21.15.45:0/2668170847 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f864800bb50 con 0x7f8648004800 2024-05-08T05:42:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865d59a640 1 -- 172.21.15.45:0/2668170847 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f864800bd20 con 0x7f86600d0530 2024-05-08T05:42:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865e59c640 1 --2- 172.21.15.45:0/2668170847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8648003c10 0x7f8648004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865dd9b640 1 --2- 172.21.15.45:0/2668170847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8648004800 0x7f8648004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865ed9d640 1 --2- 172.21.15.45:0/2668170847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86600d0530 0x7f8648005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:33.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865e59c640 1 -- 172.21.15.45:0/2668170847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86600d0530 msgr2=0x7f8648005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865e59c640 1 --2- 172.21.15.45:0/2668170847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86600d0530 0x7f8648005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865e59c640 1 -- 172.21.15.45:0/2668170847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8648004800 msgr2=0x7f8648004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865e59c640 1 --2- 172.21.15.45:0/2668170847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8648004800 0x7f8648004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.194+0000 7f865e59c640 1 -- 172.21.15.45:0/2668170847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8648003520 con 0x7f8648003c10 2024-05-08T05:42:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.195+0000 7f865e59c640 1 --2- 172.21.15.45:0/2668170847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8648003c10 0x7f8648004010 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8650001930 tx=0x7f8648005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470ec3691847176 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.195+0000 7f865d59a640 1 -- 172.21.15.45:0/2668170847 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650001520 con 0x7f8648003c10 2024-05-08T05:42:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.195+0000 7f865d59a640 1 -- 172.21.15.45:0/2668170847 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f865002ace0 con 0x7f8648003c10 2024-05-08T05:42:33.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.195+0000 7f865d59a640 1 -- 172.21.15.45:0/2668170847 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650005310 con 0x7f8648003c10 2024-05-08T05:42:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.196+0000 7f86648da640 1 -- 172.21.15.45:0/2668170847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8648003c10 msgr2=0x7f8648004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.196+0000 7f86648da640 1 --2- 172.21.15.45:0/2668170847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8648003c10 0x7f8648004010 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8650001930 tx=0x7f8648005830 comp rx=0 tx=0).stop 2024-05-08T05:42:33.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.196+0000 7f865ed9d640 1 -- 172.21.15.45:0/2668170847 reap_dead start 2024-05-08T05:42:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.196+0000 7f86648da640 1 -- 172.21.15.45:0/2668170847 shutdown_connections 2024-05-08T05:42:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.196+0000 7f86648da640 1 -- 172.21.15.45:0/2668170847 >> 172.21.15.45:0/2668170847 conn(0x7f866006c3a0 msgr2=0x7f866006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.196+0000 7f86648da640 1 -- 172.21.15.45:0/2668170847 shutdown_connections 2024-05-08T05:42:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.196+0000 7f86648da640 1 -- 172.21.15.45:0/2668170847 wait complete. 2024-05-08T05:42:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.197+0000 7f86648da640 1 Processor -- start 2024-05-08T05:42:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.197+0000 7f86648da640 1 -- start start 2024-05-08T05:42:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.198+0000 7f86648da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8648003c10 0x7f86600cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.198+0000 7f86648da640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86600ccf90 0x7f866014ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.198+0000 7f86648da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f866014e310 0x7f86600b2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.198+0000 7f86648da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8660161530 con 0x7f866014e310 2024-05-08T05:42:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.198+0000 7f86648da640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8660161250 con 0x7f86600ccf90 2024-05-08T05:42:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.198+0000 7f86648da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8660161810 con 0x7f8648003c10 2024-05-08T05:42:33.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.198+0000 7f865ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f866014e310 0x7f86600b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.198+0000 7f865ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f866014e310 0x7f86600b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57384/0 (socket says 172.21.15.45:57384) 2024-05-08T05:42:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.198+0000 7f865ed9d640 1 -- 172.21.15.45:0/444896686 learned_addr learned my addr 172.21.15.45:0/444896686 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.198+0000 7f865e59c640 1 --2- 172.21.15.45:0/444896686 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8648003c10 0x7f86600cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.198+0000 7f865dd9b640 1 --2- 172.21.15.45:0/444896686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86600ccf90 0x7f866014ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.198+0000 7f865ed9d640 1 -- 172.21.15.45:0/444896686 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8648003c10 msgr2=0x7f86600cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.198+0000 7f865ed9d640 1 --2- 172.21.15.45:0/444896686 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8648003c10 0x7f86600cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.198+0000 7f865ed9d640 1 -- 172.21.15.45:0/444896686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86600ccf90 msgr2=0x7f866014ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.198+0000 7f865ed9d640 1 --2- 172.21.15.45:0/444896686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86600ccf90 0x7f866014ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.198+0000 7f865ed9d640 1 -- 172.21.15.45:0/444896686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f866009e9c0 con 0x7f866014e310 2024-05-08T05:42:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.198+0000 7f865dd9b640 1 --2- 172.21.15.45:0/444896686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86600ccf90 0x7f866014ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.199+0000 7f865ed9d640 1 --2- 172.21.15.45:0/444896686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f866014e310 0x7f86600b2540 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f86601608e0 tx=0x7f865800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.199+0000 7f864effd640 1 -- 172.21.15.45:0/444896686 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8658015430 con 0x7f866014e310 2024-05-08T05:42:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.199+0000 7f864effd640 1 -- 172.21.15.45:0/444896686 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f865801d070 con 0x7f866014e310 2024-05-08T05:42:33.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.199+0000 7f86648da640 1 -- 172.21.15.45:0/444896686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f866009e6e0 con 0x7f866014e310 2024-05-08T05:42:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.199+0000 7f864effd640 1 -- 172.21.15.45:0/444896686 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8658015810 con 0x7f866014e310 2024-05-08T05:42:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.199+0000 7f86648da640 1 -- 172.21.15.45:0/444896686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8660148c40 con 0x7f866014e310 2024-05-08T05:42:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.200+0000 7f86648da640 1 -- 172.21.15.45:0/444896686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86600693f0 con 0x7f866014e310 2024-05-08T05:42:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.201+0000 7f864effd640 1 -- 172.21.15.45:0/444896686 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f86580159b0 con 0x7f866014e310 2024-05-08T05:42:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.201+0000 7f864effd640 1 --2- 172.21.15.45:0/444896686 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f86400762d0 0x7f8640078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.201+0000 7f864effd640 1 -- 172.21.15.45:0/444896686 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f8658098120 con 0x7f866014e310 2024-05-08T05:42:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.201+0000 7f865e59c640 1 --2- 172.21.15.45:0/444896686 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f86400762d0 0x7f8640078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:33.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.202+0000 7f865e59c640 1 --2- 172.21.15.45:0/444896686 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f86400762d0 0x7f8640078790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8650031000 tx=0x7f8650036040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:33.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.206+0000 7f864effd640 1 -- 172.21.15.45:0/444896686 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8658010040 con 0x7f866014e310 2024-05-08T05:42:33.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.410+0000 7f86648da640 1 -- 172.21.15.45:0/444896686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f86600cd370 con 0x7f866014e310 2024-05-08T05:42:33.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.411+0000 7f864effd640 1 -- 172.21.15.45:0/444896686 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) ==== 80+0+0 (secure 0 0 0) 0x7f86580618f0 con 0x7f866014e310 2024-05-08T05:42:33.411 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-05-08T05:42:33.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.413+0000 7f86648da640 1 -- 172.21.15.45:0/444896686 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f86400762d0 msgr2=0x7f8640078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.413+0000 7f86648da640 1 --2- 172.21.15.45:0/444896686 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f86400762d0 0x7f8640078790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8650031000 tx=0x7f8650036040 comp rx=0 tx=0).stop 2024-05-08T05:42:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.413+0000 7f86648da640 1 -- 172.21.15.45:0/444896686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f866014e310 msgr2=0x7f86600b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.413+0000 7f86648da640 1 --2- 172.21.15.45:0/444896686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f866014e310 0x7f86600b2540 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f86601608e0 tx=0x7f865800f580 comp rx=0 tx=0).stop 2024-05-08T05:42:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.413+0000 7f86648da640 1 -- 172.21.15.45:0/444896686 shutdown_connections 2024-05-08T05:42:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.413+0000 7f86648da640 1 --2- 172.21.15.45:0/444896686 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f86400762d0 0x7f8640078790 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.413+0000 7f86648da640 1 --2- 172.21.15.45:0/444896686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f866014e310 0x7f86600b2540 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.413+0000 7f86648da640 1 --2- 172.21.15.45:0/444896686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86600ccf90 0x7f866014ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.413+0000 7f86648da640 1 --2- 172.21.15.45:0/444896686 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8648003c10 0x7f86600cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.413+0000 7f86648da640 1 -- 172.21.15.45:0/444896686 >> 172.21.15.45:0/444896686 conn(0x7f866006c3a0 msgr2=0x7f866006f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:33.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.413+0000 7f86648da640 1 -- 172.21.15.45:0/444896686 shutdown_connections 2024-05-08T05:42:33.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.413+0000 7f86648da640 1 -- 172.21.15.45:0/444896686 wait complete. 2024-05-08T05:42:33.424 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T05:42:33.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:33.425 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T05:42:33.477 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:42:33.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:42:33.478 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T05:42:33.504 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:42:33.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.650+0000 7f6eb26e8640 1 Processor -- start 2024-05-08T05:42:33.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.650+0000 7f6eb26e8640 1 -- start start 2024-05-08T05:42:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.650+0000 7f6eb26e8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6eac1629a0 0x7f6eac0d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:33.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.650+0000 7f6eb26e8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6eac0d3330 con 0x7f6eac0d2dc0 2024-05-08T05:42:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.650+0000 7f6eb26e8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6eac0d2170 con 0x7f6eac0d0cb0 2024-05-08T05:42:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.650+0000 7f6eb26e8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6eac0d2450 con 0x7f6eac1629a0 2024-05-08T05:42:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eab7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6eac0d2dc0 0x7f6eac0c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60588/0 (socket says 172.21.15.45:60588) 2024-05-08T05:42:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eab7fe640 1 -- 172.21.15.45:0/4246241810 learned_addr learned my addr 172.21.15.45:0/4246241810 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eabfff640 1 --2- 172.21.15.45:0/4246241810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6eac1629a0 0x7f6eac0d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eaaffd640 1 -- 172.21.15.45:0/4246241810 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2701471650 0 0) 0x7f6eac0d3330 con 0x7f6eac0d2dc0 2024-05-08T05:42:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eaaffd640 1 -- 172.21.15.45:0/4246241810 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e94003520 con 0x7f6eac0d2dc0 2024-05-08T05:42:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eaaffd640 1 -- 172.21.15.45:0/4246241810 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1992479488 0 0) 0x7f6eac0d2170 con 0x7f6eac0d0cb0 2024-05-08T05:42:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eaaffd640 1 -- 172.21.15.45:0/4246241810 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6eac0d3330 con 0x7f6eac0d0cb0 2024-05-08T05:42:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eaaffd640 1 -- 172.21.15.45:0/4246241810 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 200686863 0 0) 0x7f6e94003520 con 0x7f6eac0d2dc0 2024-05-08T05:42:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eaaffd640 1 -- 172.21.15.45:0/4246241810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6eac1629a0 msgr2=0x7f6eac0d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eaaffd640 1 --2- 172.21.15.45:0/4246241810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6eac1629a0 0x7f6eac0d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eaaffd640 1 -- 172.21.15.45:0/4246241810 >> v1:172.21.15.63:6789/0 conn(0x7f6eac0d0cb0 legacy=0x7f6eac0c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eaaffd640 1 -- 172.21.15.45:0/4246241810 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eac0d2730 con 0x7f6eac0d2dc0 2024-05-08T05:42:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eaaffd640 1 -- 172.21.15.45:0/4246241810 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f6ea00016f0 con 0x7f6eac0d2dc0 2024-05-08T05:42:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eabfff640 1 --2- 172.21.15.45:0/4246241810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6eac1629a0 0x7f6eac0d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:42:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eaaffd640 1 -- 172.21.15.45:0/4246241810 >> v1:172.21.15.45:6789/0 conn(0x7f6eac0d2dc0 legacy=0x7f6eac0c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eaaffd640 1 --2- 172.21.15.45:0/4246241810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e94003c10 0x7f6e94004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eaaffd640 1 --2- 172.21.15.45:0/4246241810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e94004800 0x7f6e94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eaaffd640 1 --2- 172.21.15.45:0/4246241810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6eac1629a0 0x7f6e9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eaaffd640 1 -- 172.21.15.45:0/4246241810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac0d2450 con 0x7f6e94004800 2024-05-08T05:42:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eaaffd640 1 -- 172.21.15.45:0/4246241810 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6e94005590 con 0x7f6e94003c10 2024-05-08T05:42:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eaaffd640 1 -- 172.21.15.45:0/4246241810 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6e9400bf00 con 0x7f6eac1629a0 2024-05-08T05:42:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eab7fe640 1 --2- 172.21.15.45:0/4246241810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e94004800 0x7f6e94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.651+0000 7f6eabfff640 1 --2- 172.21.15.45:0/4246241810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e94003c10 0x7f6e94004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:33.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.652+0000 7f6eb0c85640 1 --2- 172.21.15.45:0/4246241810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6eac1629a0 0x7f6e9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.652+0000 7f6eab7fe640 1 -- 172.21.15.45:0/4246241810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6eac1629a0 msgr2=0x7f6e9400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.652+0000 7f6eab7fe640 1 --2- 172.21.15.45:0/4246241810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6eac1629a0 0x7f6e9400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.652+0000 7f6eab7fe640 1 -- 172.21.15.45:0/4246241810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e94003c10 msgr2=0x7f6e94004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.652+0000 7f6eab7fe640 1 --2- 172.21.15.45:0/4246241810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e94003c10 0x7f6e94004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.652+0000 7f6eab7fe640 1 -- 172.21.15.45:0/4246241810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e94003520 con 0x7f6e94004800 2024-05-08T05:42:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.652+0000 7f6eb0c85640 1 --2- 172.21.15.45:0/4246241810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6eac1629a0 0x7f6e9400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.652+0000 7f6eab7fe640 1 --2- 172.21.15.45:0/4246241810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e94004800 0x7f6e94004c80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000c160 tx=0x7f6e94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9119fda43e873cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.653+0000 7f6eaaffd640 1 -- 172.21.15.45:0/4246241810 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0007ca0 con 0x7f6e94004800 2024-05-08T05:42:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.653+0000 7f6eaaffd640 1 -- 172.21.15.45:0/4246241810 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea0005af0 con 0x7f6e94004800 2024-05-08T05:42:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.653+0000 7f6eaaffd640 1 -- 172.21.15.45:0/4246241810 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0007070 con 0x7f6e94004800 2024-05-08T05:42:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.653+0000 7f6eb26e8640 1 -- 172.21.15.45:0/4246241810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e94004800 msgr2=0x7f6e94004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.653+0000 7f6eb26e8640 1 --2- 172.21.15.45:0/4246241810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e94004800 0x7f6e94004c80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000c160 tx=0x7f6e94005830 comp rx=0 tx=0).stop 2024-05-08T05:42:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.653+0000 7f6eb0c85640 1 -- 172.21.15.45:0/4246241810 reap_dead start 2024-05-08T05:42:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.653+0000 7f6eb26e8640 1 -- 172.21.15.45:0/4246241810 shutdown_connections 2024-05-08T05:42:33.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.653+0000 7f6eb26e8640 1 -- 172.21.15.45:0/4246241810 >> 172.21.15.45:0/4246241810 conn(0x7f6eac0d4fe0 msgr2=0x7f6eac0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:33.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.653+0000 7f6eb26e8640 1 -- 172.21.15.45:0/4246241810 shutdown_connections 2024-05-08T05:42:33.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.653+0000 7f6eb26e8640 1 -- 172.21.15.45:0/4246241810 wait complete. 2024-05-08T05:42:33.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.654+0000 7f6eb26e8640 1 Processor -- start 2024-05-08T05:42:33.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.655+0000 7f6eb26e8640 1 -- start start 2024-05-08T05:42:33.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.655+0000 7f6eb26e8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e94004800 0x7f6eac154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:33.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.655+0000 7f6eb26e8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6eac0b81b0 0x7f6eac0b8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:33.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.655+0000 7f6eb26e8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6eac0b8b70 0x7f6eac0a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:33.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.655+0000 7f6eb26e8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac0cf440 con 0x7f6eac0b8b70 2024-05-08T05:42:33.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.655+0000 7f6eb26e8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6eac0cf2c0 con 0x7f6e94004800 2024-05-08T05:42:33.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.655+0000 7f6eb26e8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6eac0c9cf0 con 0x7f6eac0b81b0 2024-05-08T05:42:33.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.655+0000 7f6eb0c85640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6eac0b8b70 0x7f6eac0a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:33.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.655+0000 7f6eb0c85640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6eac0b8b70 0x7f6eac0a0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57400/0 (socket says 172.21.15.45:57400) 2024-05-08T05:42:33.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.655+0000 7f6eb0c85640 1 -- 172.21.15.45:0/1875581962 learned_addr learned my addr 172.21.15.45:0/1875581962 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:33.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.656+0000 7f6eb0c85640 1 -- 172.21.15.45:0/1875581962 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6eac0b81b0 msgr2=0x7f6eac0b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:33.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.656+0000 7f6eb0c85640 1 --2- 172.21.15.45:0/1875581962 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6eac0b81b0 0x7f6eac0b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.656+0000 7f6eb0c85640 1 -- 172.21.15.45:0/1875581962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e94004800 msgr2=0x7f6eac154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:33.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.656+0000 7f6eb0c85640 1 --2- 172.21.15.45:0/1875581962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e94004800 0x7f6eac154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.656+0000 7f6eb0c85640 1 -- 172.21.15.45:0/1875581962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eac1549b0 con 0x7f6eac0b8b70 2024-05-08T05:42:33.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.656+0000 7f6eb0c85640 1 --2- 172.21.15.45:0/1875581962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6eac0b8b70 0x7f6eac0a0550 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0768a0 tx=0x7f6ea400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:33.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.656+0000 7f6ea8ff9640 1 -- 172.21.15.45:0/1875581962 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4003970 con 0x7f6eac0b8b70 2024-05-08T05:42:33.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.656+0000 7f6ea8ff9640 1 -- 172.21.15.45:0/1875581962 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea4018070 con 0x7f6eac0b8b70 2024-05-08T05:42:33.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.656+0000 7f6ea8ff9640 1 -- 172.21.15.45:0/1875581962 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4003d90 con 0x7f6eac0b8b70 2024-05-08T05:42:33.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.656+0000 7f6eb26e8640 1 -- 172.21.15.45:0/1875581962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eac0ce880 con 0x7f6eac0b8b70 2024-05-08T05:42:33.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.656+0000 7f6eb26e8640 1 -- 172.21.15.45:0/1875581962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eac0b9800 con 0x7f6eac0b8b70 2024-05-08T05:42:33.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.657+0000 7f6eb26e8640 1 -- 172.21.15.45:0/1875581962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eac0693f0 con 0x7f6eac0b8b70 2024-05-08T05:42:33.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.658+0000 7f6ea8ff9640 1 -- 172.21.15.45:0/1875581962 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6ea40050d0 con 0x7f6eac0b8b70 2024-05-08T05:42:33.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.658+0000 7f6ea8ff9640 1 --2- 172.21.15.45:0/1875581962 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6e980762d0 0x7f6e98078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:33.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.658+0000 7f6ea8ff9640 1 -- 172.21.15.45:0/1875581962 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f6ea40973e0 con 0x7f6eac0b8b70 2024-05-08T05:42:33.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.658+0000 7f6eabfff640 1 --2- 172.21.15.45:0/1875581962 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6e980762d0 0x7f6e98078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:33.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.659+0000 7f6eabfff640 1 --2- 172.21.15.45:0/1875581962 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6e980762d0 0x7f6e98078790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0099d0 tx=0x7f6e9c0092e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:33.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.662+0000 7f6ea8ff9640 1 -- 172.21.15.45:0/1875581962 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6eac1549b0 con 0x7f6eac0b8b70 2024-05-08T05:42:33.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.891+0000 7f6eb26e8640 1 -- 172.21.15.45:0/1875581962 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6eac0d3ab0 con 0x7f6eac0b8b70 2024-05-08T05:42:33.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.892+0000 7f6ea8ff9640 1 -- 172.21.15.45:0/1875581962 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5172 (secure 0 0 0) 0x7f6ea4060c40 con 0x7f6eac0b8b70 2024-05-08T05:42:33.893 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:33.893 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3295938044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3295938044},{"type":"v1","addr":"172.21.15.63:6836","nonce":3295938044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4024530715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4024530715},{"type":"v1","addr":"172.21.15.63:6837","nonce":4024530715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/97258541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":97258541},{"type":"v1","addr":"172.21.15.45:6834","nonce":97258541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:42:19.860331+0000","modified":"2024-05-08T05:42:22.891006+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4884":{"gid":4884,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/173436628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":173436628},{"type":"v1","addr":"172.21.15.45:6835","nonce":173436628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4884,"qdb_cluster":[4884]},"id":2}]} 2024-05-08T05:42:33.893 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2024-05-08T05:42:33.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.895+0000 7f6eb26e8640 1 -- 172.21.15.45:0/1875581962 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6e980762d0 msgr2=0x7f6e98078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:33.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.895+0000 7f6eb26e8640 1 --2- 172.21.15.45:0/1875581962 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6e980762d0 0x7f6e98078790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c0099d0 tx=0x7f6e9c0092e0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.895+0000 7f6eb26e8640 1 -- 172.21.15.45:0/1875581962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6eac0b8b70 msgr2=0x7f6eac0a0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:33.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.895+0000 7f6eb26e8640 1 --2- 172.21.15.45:0/1875581962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6eac0b8b70 0x7f6eac0a0550 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0768a0 tx=0x7f6ea400eb90 comp rx=0 tx=0).stop 2024-05-08T05:42:33.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.895+0000 7f6eb26e8640 1 -- 172.21.15.45:0/1875581962 shutdown_connections 2024-05-08T05:42:33.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.895+0000 7f6eb26e8640 1 --2- 172.21.15.45:0/1875581962 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6e980762d0 0x7f6e98078790 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.895+0000 7f6eb26e8640 1 --2- 172.21.15.45:0/1875581962 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6eac0b81b0 0x7f6eac0b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.895+0000 7f6eb26e8640 1 --2- 172.21.15.45:0/1875581962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6eac0b8b70 0x7f6eac0a0550 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.895+0000 7f6eb26e8640 1 --2- 172.21.15.45:0/1875581962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e94004800 0x7f6eac154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:33.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.895+0000 7f6eb26e8640 1 -- 172.21.15.45:0/1875581962 >> 172.21.15.45:0/1875581962 conn(0x7f6eac0d4fe0 msgr2=0x7f6eac0d3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:33.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.895+0000 7f6eb26e8640 1 -- 172.21.15.45:0/1875581962 shutdown_connections 2024-05-08T05:42:33.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:33.895+0000 7f6eb26e8640 1 -- 172.21.15.45:0/1875581962 wait complete. 2024-05-08T05:42:33.906 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-08T05:42:34.043 INFO:teuthology.orchestra.run.smithi045.stdout:{"message":"","return_code":0} 2024-05-08T05:42:34.043 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-08T05:42:34.043 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:42:34.109 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:42:34.112 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:42:34.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.147+0000 7f3ee3971640 1 Processor -- start 2024-05-08T05:42:34.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.147+0000 7f3ee3971640 1 -- start start 2024-05-08T05:42:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.148+0000 7f3ee3971640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3edc0d1970 0x7f3edc0d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.148+0000 7f3ee3971640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3edc1630b0 con 0x7f3edc0d4cc0 2024-05-08T05:42:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.148+0000 7f3ee3971640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3edc0d3e70 con 0x7f3edc15f420 2024-05-08T05:42:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.148+0000 7f3ee3971640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f3edc0d2550 con 0x7f3edc0d1970 2024-05-08T05:42:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.148+0000 7f3ee0f0c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3edc0d1970 0x7f3edc0d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.148+0000 7f3ee1f0e640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3edc0d4cc0 0x7f3edc0d50e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60598/0 (socket says 172.21.15.45:60598) 2024-05-08T05:42:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.148+0000 7f3ee1f0e640 1 -- 172.21.15.45:0/1010247074 learned_addr learned my addr 172.21.15.45:0/1010247074 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.149+0000 7f3ee0f0c640 1 -- 172.21.15.45:0/1010247074 >> v1:172.21.15.63:6789/0 conn(0x7f3edc15f420 legacy=0x7f3edc0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.149+0000 7f3ee0f0c640 1 -- 172.21.15.45:0/1010247074 >> v1:172.21.15.45:6789/0 conn(0x7f3edc0d4cc0 legacy=0x7f3edc0d50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:34.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.149+0000 7f3ed3fff640 1 -- 172.21.15.45:0/1010247074 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 734869582 0 0) 0x7f3ed4001490 con 0x7f3edc0d4cc0 2024-05-08T05:42:34.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.149+0000 7f3ee0f0c640 1 -- 172.21.15.45:0/1010247074 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3edc0d2c80 con 0x7f3edc0d1970 2024-05-08T05:42:34.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.149+0000 7f3ee0f0c640 1 --2- 172.21.15.45:0/1010247074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3edc0d1970 0x7f3edc0d1e20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0097a0 tx=0x7f3ecc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7d6ff782905eab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:34.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.149+0000 7f3ed3fff640 1 -- 172.21.15.45:0/1010247074 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc037070 con 0x7f3edc0d1970 2024-05-08T05:42:34.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.150+0000 7f3ed3fff640 1 -- 172.21.15.45:0/1010247074 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ecc032440 con 0x7f3edc0d1970 2024-05-08T05:42:34.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.150+0000 7f3ed3fff640 1 -- 172.21.15.45:0/1010247074 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc032810 con 0x7f3edc0d1970 2024-05-08T05:42:34.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.150+0000 7f3ee3971640 1 -- 172.21.15.45:0/1010247074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3edc0d1970 msgr2=0x7f3edc0d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:34.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.150+0000 7f3ee3971640 1 --2- 172.21.15.45:0/1010247074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3edc0d1970 0x7f3edc0d1e20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc0097a0 tx=0x7f3ecc02a230 comp rx=0 tx=0).stop 2024-05-08T05:42:34.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.150+0000 7f3ee3971640 1 -- 172.21.15.45:0/1010247074 shutdown_connections 2024-05-08T05:42:34.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.150+0000 7f3ee3971640 1 --2- 172.21.15.45:0/1010247074 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3edc0d1970 0x7f3edc0d1e20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:34.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.150+0000 7f3ee3971640 1 -- 172.21.15.45:0/1010247074 >> 172.21.15.45:0/1010247074 conn(0x7f3edc08eac0 msgr2=0x7f3edc09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:34.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.150+0000 7f3ee3971640 1 -- 172.21.15.45:0/1010247074 shutdown_connections 2024-05-08T05:42:34.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.150+0000 7f3ee3971640 1 -- 172.21.15.45:0/1010247074 wait complete. 2024-05-08T05:42:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.151+0000 7f3ee3971640 1 Processor -- start 2024-05-08T05:42:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.151+0000 7f3ee3971640 1 -- start start 2024-05-08T05:42:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.152+0000 7f3ee3971640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3edc0d1970 0x7f3edc15e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.152+0000 7f3ee3971640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3edc0d4cc0 0x7f3edc15d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.152+0000 7f3ee3971640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3edc15e7d0 0x7f3edc0d0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.152+0000 7f3ee3971640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc0d2830 con 0x7f3edc0d1970 2024-05-08T05:42:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.152+0000 7f3ee3971640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc0d3240 con 0x7f3edc0d4cc0 2024-05-08T05:42:34.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.152+0000 7f3ee3971640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3edc0cfb20 con 0x7f3edc15e7d0 2024-05-08T05:42:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.152+0000 7f3ee170d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3edc0d1970 0x7f3edc15e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.152+0000 7f3ee170d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3edc0d1970 0x7f3edc15e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57414/0 (socket says 172.21.15.45:57414) 2024-05-08T05:42:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.152+0000 7f3ee170d640 1 -- 172.21.15.45:0/575278808 learned_addr learned my addr 172.21.15.45:0/575278808 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.153+0000 7f3ee0f0c640 1 --2- 172.21.15.45:0/575278808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3edc0d4cc0 0x7f3edc15d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.153+0000 7f3ee1f0e640 1 --2- 172.21.15.45:0/575278808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3edc15e7d0 0x7f3edc0d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.153+0000 7f3ee170d640 1 -- 172.21.15.45:0/575278808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3edc15e7d0 msgr2=0x7f3edc0d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.153+0000 7f3ee170d640 1 --2- 172.21.15.45:0/575278808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3edc15e7d0 0x7f3edc0d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.153+0000 7f3ee170d640 1 -- 172.21.15.45:0/575278808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3edc0d4cc0 msgr2=0x7f3edc15d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.153+0000 7f3ee170d640 1 --2- 172.21.15.45:0/575278808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3edc0d4cc0 0x7f3edc15d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.153+0000 7f3ee170d640 1 -- 172.21.15.45:0/575278808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3edc15df30 con 0x7f3edc0d1970 2024-05-08T05:42:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.153+0000 7f3ee0f0c640 1 --2- 172.21.15.45:0/575278808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3edc0d4cc0 0x7f3edc15d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.153+0000 7f3ee170d640 1 --2- 172.21.15.45:0/575278808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3edc0d1970 0x7f3edc15e160 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3ed800c430 tx=0x7f3ed800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.153+0000 7f3ed1ffb640 1 -- 172.21.15.45:0/575278808 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8017070 con 0x7f3edc0d1970 2024-05-08T05:42:34.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.153+0000 7f3ed1ffb640 1 -- 172.21.15.45:0/575278808 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ed80052d0 con 0x7f3edc0d1970 2024-05-08T05:42:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.153+0000 7f3ee3971640 1 -- 172.21.15.45:0/575278808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3edc0cc570 con 0x7f3edc0d1970 2024-05-08T05:42:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.153+0000 7f3ed1ffb640 1 -- 172.21.15.45:0/575278808 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8003830 con 0x7f3edc0d1970 2024-05-08T05:42:34.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.153+0000 7f3ee3971640 1 -- 172.21.15.45:0/575278808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3edc0ccb30 con 0x7f3edc0d1970 2024-05-08T05:42:34.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.155+0000 7f3ee3971640 1 -- 172.21.15.45:0/575278808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3edc0693f0 con 0x7f3edc0d1970 2024-05-08T05:42:34.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.155+0000 7f3ed1ffb640 1 -- 172.21.15.45:0/575278808 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3ed801a070 con 0x7f3edc0d1970 2024-05-08T05:42:34.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.156+0000 7f3ed1ffb640 1 --2- 172.21.15.45:0/575278808 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3ec00762d0 0x7f3ec0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:34.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.156+0000 7f3ed1ffb640 1 -- 172.21.15.45:0/575278808 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f3ed8096bd0 con 0x7f3edc0d1970 2024-05-08T05:42:34.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.156+0000 7f3ee0f0c640 1 --2- 172.21.15.45:0/575278808 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3ec00762d0 0x7f3ec0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:34.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.156+0000 7f3ee0f0c640 1 --2- 172.21.15.45:0/575278808 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3ec00762d0 0x7f3ec0078790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc034000 tx=0x7f3ecc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:34.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.160+0000 7f3ed1ffb640 1 -- 172.21.15.45:0/575278808 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3ed8060300 con 0x7f3edc0d1970 2024-05-08T05:42:34.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.389+0000 7f3ee3971640 1 -- 172.21.15.45:0/575278808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3edc0684e0 con 0x7f3edc0d1970 2024-05-08T05:42:34.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.389+0000 7f3ed1ffb640 1 -- 172.21.15.45:0/575278808 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5172 (secure 0 0 0) 0x7f3ed80682f0 con 0x7f3edc0d1970 2024-05-08T05:42:34.390 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:34.390 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3295938044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3295938044},{"type":"v1","addr":"172.21.15.63:6836","nonce":3295938044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4024530715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4024530715},{"type":"v1","addr":"172.21.15.63:6837","nonce":4024530715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/97258541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":97258541},{"type":"v1","addr":"172.21.15.45:6834","nonce":97258541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:42:19.860331+0000","modified":"2024-05-08T05:42:22.891006+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4884":{"gid":4884,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/173436628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":173436628},{"type":"v1","addr":"172.21.15.45:6835","nonce":173436628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4884,"qdb_cluster":[4884]},"id":2}]} 2024-05-08T05:42:34.390 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2024-05-08T05:42:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.391+0000 7f3ee3971640 1 -- 172.21.15.45:0/575278808 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3ec00762d0 msgr2=0x7f3ec0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.391+0000 7f3ee3971640 1 --2- 172.21.15.45:0/575278808 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3ec00762d0 0x7f3ec0078790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc034000 tx=0x7f3ecc02e000 comp rx=0 tx=0).stop 2024-05-08T05:42:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.391+0000 7f3ee3971640 1 -- 172.21.15.45:0/575278808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3edc0d1970 msgr2=0x7f3edc15e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.391+0000 7f3ee3971640 1 --2- 172.21.15.45:0/575278808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3edc0d1970 0x7f3edc15e160 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3ed800c430 tx=0x7f3ed800c8f0 comp rx=0 tx=0).stop 2024-05-08T05:42:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.391+0000 7f3ee3971640 1 -- 172.21.15.45:0/575278808 shutdown_connections 2024-05-08T05:42:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.391+0000 7f3ee3971640 1 --2- 172.21.15.45:0/575278808 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3ec00762d0 0x7f3ec0078790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.391+0000 7f3ee3971640 1 --2- 172.21.15.45:0/575278808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3edc15e7d0 0x7f3edc0d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.391+0000 7f3ee3971640 1 --2- 172.21.15.45:0/575278808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3edc0d4cc0 0x7f3edc15d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.391+0000 7f3ee3971640 1 --2- 172.21.15.45:0/575278808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3edc0d1970 0x7f3edc15e160 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.391+0000 7f3ee3971640 1 -- 172.21.15.45:0/575278808 >> 172.21.15.45:0/575278808 conn(0x7f3edc08eac0 msgr2=0x7f3edc06cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.391+0000 7f3ee3971640 1 -- 172.21.15.45:0/575278808 shutdown_connections 2024-05-08T05:42:34.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:34.391+0000 7f3ee3971640 1 -- 172.21.15.45:0/575278808 wait complete. 2024-05-08T05:42:34.403 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parentdir mytag 2024-05-08T05:42:34.567 INFO:teuthology.orchestra.run.smithi045.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-05-08T05:42:34.568 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-05-08T05:42:44.569 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-05-08T05:42:44.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.621+0000 7fb79731df40 1 Processor -- start 2024-05-08T05:42:44.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.621+0000 7fb79731df40 1 -- start start 2024-05-08T05:42:44.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.622+0000 7fb79731df40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5618f0179a20 0x5618f01c28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.622+0000 7fb79731df40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618f01c2200 0x5618f0247940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.622+0000 7fb79731df40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618f01c3980 0x5618f01aa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.622+0000 7fb79731df40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5618f0122f50 con 0x5618f01c3980 2024-05-08T05:42:44.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.622+0000 7fb79731df40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5618f0126f90 con 0x5618f01c2200 2024-05-08T05:42:44.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.622+0000 7fb79731df40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5618f01c3d60 con 0x5618f0179a20 2024-05-08T05:42:44.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.622+0000 7fb795618640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618f01c3980 0x5618f01aa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.622+0000 7fb795618640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618f01c3980 0x5618f01aa190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41120/0 (socket says 172.21.15.45:41120) 2024-05-08T05:42:44.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.622+0000 7fb795618640 1 -- 172.21.15.45:0/937330661 learned_addr learned my addr 172.21.15.45:0/937330661 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:44.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.622+0000 7fb794616640 1 --2- 172.21.15.45:0/937330661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618f01c2200 0x5618f0247940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.622+0000 7fb794e17640 1 --2- 172.21.15.45:0/937330661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5618f0179a20 0x5618f01c28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.622+0000 7fb795618640 1 -- 172.21.15.45:0/937330661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5618f0179a20 msgr2=0x5618f01c28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:44.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.622+0000 7fb795618640 1 --2- 172.21.15.45:0/937330661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5618f0179a20 0x5618f01c28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.622+0000 7fb795618640 1 -- 172.21.15.45:0/937330661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618f01c2200 msgr2=0x5618f0247940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:44.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.622+0000 7fb795618640 1 --2- 172.21.15.45:0/937330661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618f01c2200 0x5618f0247940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.622+0000 7fb795618640 1 -- 172.21.15.45:0/937330661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618f02456a0 con 0x5618f01c3980 2024-05-08T05:42:44.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.622+0000 7fb795618640 1 --2- 172.21.15.45:0/937330661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618f01c3980 0x5618f01aa190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb78c006590 tx=0x7fb78c006a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d50674bbe1b36b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:44.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.623+0000 7fb793614640 1 -- 172.21.15.45:0/937330661 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c016070 con 0x5618f01c3980 2024-05-08T05:42:44.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.623+0000 7fb793614640 1 -- 172.21.15.45:0/937330661 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb78c00b790 con 0x5618f01c3980 2024-05-08T05:42:44.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.623+0000 7fb793614640 1 -- 172.21.15.45:0/937330661 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c00bb80 con 0x5618f01c3980 2024-05-08T05:42:44.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.623+0000 7fb79731df40 1 -- 172.21.15.45:0/937330661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618f01c3980 msgr2=0x5618f01aa190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.623+0000 7fb79731df40 1 --2- 172.21.15.45:0/937330661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618f01c3980 0x5618f01aa190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb78c006590 tx=0x7fb78c006a50 comp rx=0 tx=0).stop 2024-05-08T05:42:44.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.623+0000 7fb79731df40 1 -- 172.21.15.45:0/937330661 shutdown_connections 2024-05-08T05:42:44.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.623+0000 7fb79731df40 1 --2- 172.21.15.45:0/937330661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618f01c3980 0x5618f01aa190 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.623+0000 7fb79731df40 1 --2- 172.21.15.45:0/937330661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618f01c2200 0x5618f0247940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.623+0000 7fb79731df40 1 --2- 172.21.15.45:0/937330661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5618f0179a20 0x5618f01c28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.623+0000 7fb79731df40 1 -- 172.21.15.45:0/937330661 >> 172.21.15.45:0/937330661 conn(0x5618f017ab30 msgr2=0x5618f01a94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:44.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.623+0000 7fb79731df40 1 -- 172.21.15.45:0/937330661 shutdown_connections 2024-05-08T05:42:44.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.623+0000 7fb79731df40 1 -- 172.21.15.45:0/937330661 wait complete. 2024-05-08T05:42:44.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.624+0000 7fb79731df40 1 Processor -- start 2024-05-08T05:42:44.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.624+0000 7fb79731df40 1 -- start start 2024-05-08T05:42:44.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.625+0000 7fb79731df40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618f0179a20 0x5618f023ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.625+0000 7fb79731df40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618f01c2200 0x5618f0252c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.625+0000 7fb79731df40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5618f01c3980 0x5618f02531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.625+0000 7fb79731df40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5618f01a8b70 con 0x5618f0179a20 2024-05-08T05:42:44.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.625+0000 7fb79731df40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5618f02452d0 con 0x5618f01c2200 2024-05-08T05:42:44.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.625+0000 7fb79731df40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5618f01a5400 con 0x5618f01c3980 2024-05-08T05:42:44.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.625+0000 7fb794e17640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618f0179a20 0x5618f023ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.625+0000 7fb794e17640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618f0179a20 0x5618f023ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41134/0 (socket says 172.21.15.45:41134) 2024-05-08T05:42:44.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.625+0000 7fb794e17640 1 -- 172.21.15.45:0/757166548 learned_addr learned my addr 172.21.15.45:0/757166548 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:44.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.625+0000 7fb794616640 1 --2- 172.21.15.45:0/757166548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618f01c2200 0x5618f0252c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.625+0000 7fb795618640 1 --2- 172.21.15.45:0/757166548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5618f01c3980 0x5618f02531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.625+0000 7fb794e17640 1 -- 172.21.15.45:0/757166548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5618f01c3980 msgr2=0x5618f02531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.625+0000 7fb794e17640 1 --2- 172.21.15.45:0/757166548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5618f01c3980 0x5618f02531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.625+0000 7fb794e17640 1 -- 172.21.15.45:0/757166548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618f01c2200 msgr2=0x5618f0252c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.625+0000 7fb794e17640 1 --2- 172.21.15.45:0/757166548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618f01c2200 0x5618f0252c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.625+0000 7fb794e17640 1 -- 172.21.15.45:0/757166548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618f0241980 con 0x5618f0179a20 2024-05-08T05:42:44.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.625+0000 7fb794616640 1 --2- 172.21.15.45:0/757166548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5618f01c2200 0x5618f0252c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:44.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.625+0000 7fb795618640 1 --2- 172.21.15.45:0/757166548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5618f01c3980 0x5618f02531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:44.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.625+0000 7fb794e17640 1 --2- 172.21.15.45:0/757166548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618f0179a20 0x5618f023ca50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb78400c530 tx=0x7fb78400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:44.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.626+0000 7fb792612640 1 -- 172.21.15.45:0/757166548 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784008070 con 0x5618f0179a20 2024-05-08T05:42:44.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.626+0000 7fb792612640 1 -- 172.21.15.45:0/757166548 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7840106a0 con 0x5618f0179a20 2024-05-08T05:42:44.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.626+0000 7fb79731df40 1 -- 172.21.15.45:0/757166548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5618f0240ee0 con 0x5618f0179a20 2024-05-08T05:42:44.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.626+0000 7fb792612640 1 -- 172.21.15.45:0/757166548 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784010a50 con 0x5618f0179a20 2024-05-08T05:42:44.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.626+0000 7fb79731df40 1 -- 172.21.15.45:0/757166548 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618f0241bd0 con 0x5618f0179a20 2024-05-08T05:42:44.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.627+0000 7fb792612640 1 -- 172.21.15.45:0/757166548 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb784010bf0 con 0x5618f0179a20 2024-05-08T05:42:44.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.628+0000 7fb792612640 1 --2- 172.21.15.45:0/757166548 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb780074cb0 0x7fb780077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.628+0000 7fb792612640 1 -- 172.21.15.45:0/757166548 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7fb784096cd0 con 0x5618f0179a20 2024-05-08T05:42:44.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.628+0000 7fb79731df40 1 --2- 172.21.15.45:0/757166548 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x5618f0243c20 0x5618f0244020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.628+0000 7fb79731df40 1 -- 172.21.15.45:0/757166548 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) -- 0x5618f0244560 con 0x5618f0243c20 2024-05-08T05:42:44.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.628+0000 7fb794616640 1 --2- 172.21.15.45:0/757166548 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb780074cb0 0x7fb780077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.628+0000 7fb795618640 1 --2- 172.21.15.45:0/757166548 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x5618f0243c20 0x5618f0244020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.629+0000 7fb795618640 1 --2- 172.21.15.45:0/757166548 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x5618f0243c20 0x5618f0244020 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:44.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.629+0000 7fb794616640 1 --2- 172.21.15.45:0/757166548 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb780074cb0 0x7fb780077170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb788056000 tx=0x7fb788050000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:44.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.629+0000 7fb795618640 1 -- 172.21.15.45:0/757166548 <== osd.7 v2:172.21.15.63:6816/1163469437 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fb78c00f070 con 0x5618f0243c20 2024-05-08T05:42:44.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.629+0000 7fb79731df40 1 -- 172.21.15.45:0/757166548 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x5618f0243c20 msgr2=0x5618f0244020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.629+0000 7fb79731df40 1 --2- 172.21.15.45:0/757166548 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x5618f0243c20 0x5618f0244020 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.629+0000 7fb79731df40 1 -- 172.21.15.45:0/757166548 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb780074cb0 msgr2=0x7fb780077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.629+0000 7fb79731df40 1 --2- 172.21.15.45:0/757166548 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb780074cb0 0x7fb780077170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb788056000 tx=0x7fb788050000 comp rx=0 tx=0).stop 2024-05-08T05:42:44.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.629+0000 7fb79731df40 1 -- 172.21.15.45:0/757166548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618f0179a20 msgr2=0x5618f023ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.629+0000 7fb79731df40 1 --2- 172.21.15.45:0/757166548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5618f0179a20 0x5618f023ca50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb78400c530 tx=0x7fb78400c930 comp rx=0 tx=0).stop 2024-05-08T05:42:44.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.629+0000 7fb795618640 1 -- 172.21.15.45:0/757166548 reap_dead start 2024-05-08T05:42:44.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.629+0000 7fb79731df40 1 -- 172.21.15.45:0/757166548 shutdown_connections 2024-05-08T05:42:44.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.629+0000 7fb79731df40 1 -- 172.21.15.45:0/757166548 >> 172.21.15.45:0/757166548 conn(0x5618f017ab30 msgr2=0x5618f0178ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:44.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.629+0000 7fb79731df40 1 -- 172.21.15.45:0/757166548 shutdown_connections 2024-05-08T05:42:44.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.630+0000 7fb79731df40 1 -- 172.21.15.45:0/757166548 wait complete. 2024-05-08T05:42:44.634 INFO:teuthology.orchestra.run.smithi045.stdout:mytag 2024-05-08T05:42:44.634 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-05-08T05:42:44.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.681+0000 7f9ed2925f40 1 Processor -- start 2024-05-08T05:42:44.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.682+0000 7f9ed2925f40 1 -- start start 2024-05-08T05:42:44.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.682+0000 7f9ed2925f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ddd198fea0 0x55ddd195df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.682+0000 7f9ed2925f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ddd1992440 0x55ddd1a28a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.682+0000 7f9ed2925f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ddd1a36b10 0x55ddd1a28f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.682+0000 7f9ed2925f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ddd19903f0 con 0x55ddd198fea0 2024-05-08T05:42:44.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.682+0000 7f9ed2925f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ddd1906f50 con 0x55ddd1a36b10 2024-05-08T05:42:44.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.682+0000 7f9ed2925f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55ddd195f710 con 0x55ddd1992440 2024-05-08T05:42:44.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.682+0000 7f9ecf955640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ddd198fea0 0x55ddd195df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.682+0000 7f9ecf955640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ddd198fea0 0x55ddd195df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41144/0 (socket says 172.21.15.45:41144) 2024-05-08T05:42:44.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.682+0000 7f9ecf955640 1 -- 172.21.15.45:0/2433862146 learned_addr learned my addr 172.21.15.45:0/2433862146 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:44.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.683+0000 7f9ed0156640 1 --2- 172.21.15.45:0/2433862146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ddd1a36b10 0x55ddd1a28f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.683+0000 7f9ecf154640 1 --2- 172.21.15.45:0/2433862146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ddd1992440 0x55ddd1a28a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.683+0000 7f9ecf955640 1 -- 172.21.15.45:0/2433862146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ddd1992440 msgr2=0x55ddd1a28a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:44.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.683+0000 7f9ecf955640 1 --2- 172.21.15.45:0/2433862146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ddd1992440 0x55ddd1a28a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.683+0000 7f9ecf955640 1 -- 172.21.15.45:0/2433862146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ddd1a36b10 msgr2=0x55ddd1a28f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:44.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.683+0000 7f9ecf955640 1 --2- 172.21.15.45:0/2433862146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ddd1a36b10 0x55ddd1a28f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.683+0000 7f9ecf955640 1 -- 172.21.15.45:0/2433862146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddd1a30e90 con 0x55ddd198fea0 2024-05-08T05:42:44.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.683+0000 7f9ed0156640 1 --2- 172.21.15.45:0/2433862146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ddd1a36b10 0x55ddd1a28f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:44.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.683+0000 7f9ecf955640 1 --2- 172.21.15.45:0/2433862146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ddd198fea0 0x55ddd195df40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0001720 tx=0x7f9ec0001be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a88b04c1dd7f61 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:44.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.683+0000 7f9ece152640 1 -- 172.21.15.45:0/2433862146 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec00062e0 con 0x55ddd198fea0 2024-05-08T05:42:44.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.683+0000 7f9ece152640 1 -- 172.21.15.45:0/2433862146 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec0003650 con 0x55ddd198fea0 2024-05-08T05:42:44.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.683+0000 7f9ece152640 1 -- 172.21.15.45:0/2433862146 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0003a60 con 0x55ddd198fea0 2024-05-08T05:42:44.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.684+0000 7f9ed2925f40 1 -- 172.21.15.45:0/2433862146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ddd198fea0 msgr2=0x55ddd195df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.684+0000 7f9ed2925f40 1 --2- 172.21.15.45:0/2433862146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ddd198fea0 0x55ddd195df40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0001720 tx=0x7f9ec0001be0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.684+0000 7f9ed2925f40 1 -- 172.21.15.45:0/2433862146 shutdown_connections 2024-05-08T05:42:44.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.684+0000 7f9ed2925f40 1 --2- 172.21.15.45:0/2433862146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ddd1992440 0x55ddd1a28a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.684+0000 7f9ed2925f40 1 --2- 172.21.15.45:0/2433862146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ddd1a36b10 0x55ddd1a28f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.684+0000 7f9ed2925f40 1 --2- 172.21.15.45:0/2433862146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ddd198fea0 0x55ddd195df40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.684+0000 7f9ed2925f40 1 -- 172.21.15.45:0/2433862146 >> 172.21.15.45:0/2433862146 conn(0x55ddd198eba0 msgr2=0x55ddd1a32800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:44.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.684+0000 7f9ed2925f40 1 -- 172.21.15.45:0/2433862146 shutdown_connections 2024-05-08T05:42:44.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.684+0000 7f9ed2925f40 1 -- 172.21.15.45:0/2433862146 wait complete. 2024-05-08T05:42:44.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.684+0000 7f9ed2925f40 1 Processor -- start 2024-05-08T05:42:44.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.684+0000 7f9ed2925f40 1 -- start start 2024-05-08T05:42:44.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.685+0000 7f9ed2925f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ddd198fea0 0x55ddd198e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.685+0000 7f9ed2925f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ddd1992440 0x55ddd1a26920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.685+0000 7f9ed2925f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ddd1a36b10 0x55ddd1ac53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.685+0000 7f9ed2925f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ddd1a31800 con 0x55ddd1992440 2024-05-08T05:42:44.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.685+0000 7f9ed2925f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ddd1a31170 con 0x55ddd1a36b10 2024-05-08T05:42:44.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.685+0000 7f9ed2925f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55ddd1a31ae0 con 0x55ddd198fea0 2024-05-08T05:42:44.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.685+0000 7f9ecf154640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ddd1992440 0x55ddd1a26920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.685+0000 7f9ecf154640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ddd1992440 0x55ddd1a26920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41150/0 (socket says 172.21.15.45:41150) 2024-05-08T05:42:44.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.685+0000 7f9ecf154640 1 -- 172.21.15.45:0/3066929826 learned_addr learned my addr 172.21.15.45:0/3066929826 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:44.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.685+0000 7f9ed0156640 1 --2- 172.21.15.45:0/3066929826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ddd1a36b10 0x55ddd1ac53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.685+0000 7f9ecf955640 1 --2- 172.21.15.45:0/3066929826 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ddd198fea0 0x55ddd198e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.685+0000 7f9ecf154640 1 -- 172.21.15.45:0/3066929826 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ddd198fea0 msgr2=0x55ddd198e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.685+0000 7f9ecf154640 1 --2- 172.21.15.45:0/3066929826 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ddd198fea0 0x55ddd198e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.685+0000 7f9ecf154640 1 -- 172.21.15.45:0/3066929826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ddd1a36b10 msgr2=0x55ddd1ac53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.685+0000 7f9ecf154640 1 --2- 172.21.15.45:0/3066929826 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ddd1a36b10 0x55ddd1ac53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.685+0000 7f9ecf154640 1 -- 172.21.15.45:0/3066929826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddd1a25900 con 0x55ddd1992440 2024-05-08T05:42:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.686+0000 7f9ecf154640 1 --2- 172.21.15.45:0/3066929826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ddd1992440 0x55ddd1a26920 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4057040 tx=0x7f9ec4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.686+0000 7f9ecd150640 1 -- 172.21.15.45:0/3066929826 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4063020 con 0x55ddd1992440 2024-05-08T05:42:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.686+0000 7f9ecd150640 1 -- 172.21.15.45:0/3066929826 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec4054ab0 con 0x55ddd1992440 2024-05-08T05:42:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.686+0000 7f9ed2925f40 1 -- 172.21.15.45:0/3066929826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ddd1a25340 con 0x55ddd1992440 2024-05-08T05:42:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.686+0000 7f9ecd150640 1 -- 172.21.15.45:0/3066929826 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4059040 con 0x55ddd1992440 2024-05-08T05:42:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.686+0000 7f9ed2925f40 1 -- 172.21.15.45:0/3066929826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddd1a2def0 con 0x55ddd1992440 2024-05-08T05:42:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.688+0000 7f9ecd150640 1 -- 172.21.15.45:0/3066929826 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9ec4066020 con 0x55ddd1992440 2024-05-08T05:42:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.688+0000 7f9ecd150640 1 --2- 172.21.15.45:0/3066929826 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ebc074cb0 0x7f9ebc077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.689+0000 7f9ecd150640 1 -- 172.21.15.45:0/3066929826 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f9ec40d9c70 con 0x55ddd1992440 2024-05-08T05:42:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.689+0000 7f9ecf955640 1 --2- 172.21.15.45:0/3066929826 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ebc074cb0 0x7f9ebc077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.689+0000 7f9ed2925f40 1 --2- 172.21.15.45:0/3066929826 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55ddd1a26e60 0x55ddd1ac5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.689+0000 7f9ed2925f40 1 -- 172.21.15.45:0/3066929826 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) -- 0x55ddd1a2e6d0 con 0x55ddd1a26e60 2024-05-08T05:42:44.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.689+0000 7f9ed0156640 1 --2- 172.21.15.45:0/3066929826 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55ddd1a26e60 0x55ddd1ac5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.689+0000 7f9ecf955640 1 --2- 172.21.15.45:0/3066929826 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ebc074cb0 0x7f9ebc077170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0001720 tx=0x7f9ec0008070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:44.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.689+0000 7f9ed0156640 1 --2- 172.21.15.45:0/3066929826 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55ddd1a26e60 0x55ddd1ac5e20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:44.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.690+0000 7f9ed0156640 1 -- 172.21.15.45:0/3066929826 <== osd.5 v2:172.21.15.63:6808/2395798255 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f9ec8009a10 con 0x55ddd1a26e60 2024-05-08T05:42:44.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.690+0000 7f9ed2925f40 1 -- 172.21.15.45:0/3066929826 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55ddd1a26e60 msgr2=0x55ddd1ac5e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.690+0000 7f9ed2925f40 1 --2- 172.21.15.45:0/3066929826 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55ddd1a26e60 0x55ddd1ac5e20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.690+0000 7f9ed2925f40 1 -- 172.21.15.45:0/3066929826 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ebc074cb0 msgr2=0x7f9ebc077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.690+0000 7f9ed2925f40 1 --2- 172.21.15.45:0/3066929826 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ebc074cb0 0x7f9ebc077170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0001720 tx=0x7f9ec0008070 comp rx=0 tx=0).stop 2024-05-08T05:42:44.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.690+0000 7f9ed2925f40 1 -- 172.21.15.45:0/3066929826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ddd1992440 msgr2=0x55ddd1a26920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.690+0000 7f9ed2925f40 1 --2- 172.21.15.45:0/3066929826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ddd1992440 0x55ddd1a26920 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4057040 tx=0x7f9ec4002fb0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.690+0000 7f9ed0156640 1 -- 172.21.15.45:0/3066929826 reap_dead start 2024-05-08T05:42:44.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.690+0000 7f9ed2925f40 1 -- 172.21.15.45:0/3066929826 shutdown_connections 2024-05-08T05:42:44.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.690+0000 7f9ed2925f40 1 -- 172.21.15.45:0/3066929826 >> 172.21.15.45:0/3066929826 conn(0x55ddd198eba0 msgr2=0x55ddd1991b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:44.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.690+0000 7f9ed2925f40 1 -- 172.21.15.45:0/3066929826 shutdown_connections 2024-05-08T05:42:44.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.690+0000 7f9ed2925f40 1 -- 172.21.15.45:0/3066929826 wait complete. 2024-05-08T05:42:44.694 INFO:teuthology.orchestra.run.smithi045.stdout:mytag 2024-05-08T05:42:44.695 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-05-08T05:42:44.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.740+0000 7f0501e0ef40 1 Processor -- start 2024-05-08T05:42:44.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.740+0000 7f0501e0ef40 1 -- start start 2024-05-08T05:42:44.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.740+0000 7f0501e0ef40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f68b966a20 0x55f68ba32b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.740+0000 7f0501e0ef40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b9b14d0 0x55f68b996300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.740+0000 7f0501e0ef40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f68ba3f6d0 0x55f68b996840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.740+0000 7f0501e0ef40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f68b969800 con 0x55f68b9b14d0 2024-05-08T05:42:44.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.740+0000 7f0501e0ef40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f68b90ff50 con 0x55f68ba3f6d0 2024-05-08T05:42:44.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.740+0000 7f0501e0ef40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f68b969ae0 con 0x55f68b966a20 2024-05-08T05:42:44.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.741+0000 7f04fe4b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b9b14d0 0x55f68b996300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.741+0000 7f04fe4b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b9b14d0 0x55f68b996300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41162/0 (socket says 172.21.15.45:41162) 2024-05-08T05:42:44.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.741+0000 7f04fe4b7640 1 -- 172.21.15.45:0/318944692 learned_addr learned my addr 172.21.15.45:0/318944692 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:44.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.741+0000 7f04ff4b9640 1 --2- 172.21.15.45:0/318944692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f68ba3f6d0 0x55f68b996840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.741+0000 7f04fecb8640 1 --2- 172.21.15.45:0/318944692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f68b966a20 0x55f68ba32b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.741+0000 7f04fe4b7640 1 -- 172.21.15.45:0/318944692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f68b966a20 msgr2=0x55f68ba32b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:44.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.741+0000 7f04fe4b7640 1 --2- 172.21.15.45:0/318944692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f68b966a20 0x55f68ba32b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.741+0000 7f04fe4b7640 1 -- 172.21.15.45:0/318944692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f68ba3f6d0 msgr2=0x55f68b996840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:44.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.741+0000 7f04fe4b7640 1 --2- 172.21.15.45:0/318944692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f68ba3f6d0 0x55f68b996840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.741+0000 7f04fe4b7640 1 -- 172.21.15.45:0/318944692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f68b9b28c0 con 0x55f68b9b14d0 2024-05-08T05:42:44.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.741+0000 7f04ff4b9640 1 --2- 172.21.15.45:0/318944692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f68ba3f6d0 0x55f68b996840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:44.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.741+0000 7f04fecb8640 1 --2- 172.21.15.45:0/318944692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f68b966a20 0x55f68ba32b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:44.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.741+0000 7f04fe4b7640 1 --2- 172.21.15.45:0/318944692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b9b14d0 0x55f68b996300 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f04f400a770 tx=0x7f04f400a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47720ed704919f08 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:44.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.741+0000 7f04fd4b5640 1 -- 172.21.15.45:0/318944692 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4018070 con 0x55f68b9b14d0 2024-05-08T05:42:44.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.741+0000 7f04fd4b5640 1 -- 172.21.15.45:0/318944692 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f4007600 con 0x55f68b9b14d0 2024-05-08T05:42:44.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.741+0000 7f04fd4b5640 1 -- 172.21.15.45:0/318944692 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f40079b0 con 0x55f68b9b14d0 2024-05-08T05:42:44.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.742+0000 7f0501e0ef40 1 -- 172.21.15.45:0/318944692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b9b14d0 msgr2=0x55f68b996300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.742+0000 7f0501e0ef40 1 --2- 172.21.15.45:0/318944692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b9b14d0 0x55f68b996300 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f04f400a770 tx=0x7f04f400a860 comp rx=0 tx=0).stop 2024-05-08T05:42:44.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.742+0000 7f0501e0ef40 1 -- 172.21.15.45:0/318944692 shutdown_connections 2024-05-08T05:42:44.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.742+0000 7f0501e0ef40 1 --2- 172.21.15.45:0/318944692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f68ba3f6d0 0x55f68b996840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.742+0000 7f0501e0ef40 1 --2- 172.21.15.45:0/318944692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b9b14d0 0x55f68b996300 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.742+0000 7f0501e0ef40 1 --2- 172.21.15.45:0/318944692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f68b966a20 0x55f68ba32b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.742+0000 7f0501e0ef40 1 -- 172.21.15.45:0/318944692 >> 172.21.15.45:0/318944692 conn(0x55f68b967b30 msgr2=0x55f68b9ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:44.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.742+0000 7f0501e0ef40 1 -- 172.21.15.45:0/318944692 shutdown_connections 2024-05-08T05:42:44.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.742+0000 7f0501e0ef40 1 -- 172.21.15.45:0/318944692 wait complete. 2024-05-08T05:42:44.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.742+0000 7f0501e0ef40 1 Processor -- start 2024-05-08T05:42:44.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.742+0000 7f0501e0ef40 1 -- start start 2024-05-08T05:42:44.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.743+0000 7f0501e0ef40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f68b966a20 0x55f68bad1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.743+0000 7f0501e0ef40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b9b14d0 0x55f68ba42010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.743+0000 7f0501e0ef40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f68ba3f6d0 0x55f68ba341e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.743+0000 7f0501e0ef40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f68b993fb0 con 0x55f68b9b14d0 2024-05-08T05:42:44.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.743+0000 7f0501e0ef40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f68b994290 con 0x55f68b966a20 2024-05-08T05:42:44.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.743+0000 7f0501e0ef40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f68b994570 con 0x55f68ba3f6d0 2024-05-08T05:42:44.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.743+0000 7f04fe4b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b9b14d0 0x55f68ba42010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.743+0000 7f04fe4b7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b9b14d0 0x55f68ba42010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41168/0 (socket says 172.21.15.45:41168) 2024-05-08T05:42:44.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.743+0000 7f04fe4b7640 1 -- 172.21.15.45:0/488388689 learned_addr learned my addr 172.21.15.45:0/488388689 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:44.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.743+0000 7f04fecb8640 1 --2- 172.21.15.45:0/488388689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f68b966a20 0x55f68bad1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.743+0000 7f04ff4b9640 1 --2- 172.21.15.45:0/488388689 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f68ba3f6d0 0x55f68ba341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.743+0000 7f04fe4b7640 1 -- 172.21.15.45:0/488388689 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f68ba3f6d0 msgr2=0x55f68ba341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.743+0000 7f04fe4b7640 1 --2- 172.21.15.45:0/488388689 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f68ba3f6d0 0x55f68ba341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.743+0000 7f04fe4b7640 1 -- 172.21.15.45:0/488388689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f68b966a20 msgr2=0x55f68bad1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.743+0000 7f04fe4b7640 1 --2- 172.21.15.45:0/488388689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f68b966a20 0x55f68bad1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.743+0000 7f04fe4b7640 1 -- 172.21.15.45:0/488388689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f68ba40cd0 con 0x55f68b9b14d0 2024-05-08T05:42:44.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.743+0000 7f04ff4b9640 1 --2- 172.21.15.45:0/488388689 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f68ba3f6d0 0x55f68ba341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:44.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.744+0000 7f04fe4b7640 1 --2- 172.21.15.45:0/488388689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b9b14d0 0x55f68ba42010 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f04f4016040 tx=0x7f04f4003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:44.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.744+0000 7f04e7fff640 1 -- 172.21.15.45:0/488388689 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4018040 con 0x55f68b9b14d0 2024-05-08T05:42:44.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.744+0000 7f04e7fff640 1 -- 172.21.15.45:0/488388689 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f4022070 con 0x55f68b9b14d0 2024-05-08T05:42:44.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.744+0000 7f0501e0ef40 1 -- 172.21.15.45:0/488388689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f68ba33690 con 0x55f68b9b14d0 2024-05-08T05:42:44.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.744+0000 7f04e7fff640 1 -- 172.21.15.45:0/488388689 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4007e10 con 0x55f68b9b14d0 2024-05-08T05:42:44.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.744+0000 7f0501e0ef40 1 -- 172.21.15.45:0/488388689 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f68ba33c50 con 0x55f68b9b14d0 2024-05-08T05:42:44.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.745+0000 7f04e7fff640 1 -- 172.21.15.45:0/488388689 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f04f4013410 con 0x55f68b9b14d0 2024-05-08T05:42:44.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.746+0000 7f04e7fff640 1 --2- 172.21.15.45:0/488388689 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f04ec074ce0 0x7f04ec0771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.746+0000 7f04e7fff640 1 -- 172.21.15.45:0/488388689 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f04f4096df0 con 0x55f68b9b14d0 2024-05-08T05:42:44.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.746+0000 7f0501e0ef40 1 --2- 172.21.15.45:0/488388689 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55f68ba42550 0x55f68ba3db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.746+0000 7f0501e0ef40 1 -- 172.21.15.45:0/488388689 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) -- 0x55f68ba3e0b0 con 0x55f68ba42550 2024-05-08T05:42:44.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.746+0000 7f04fecb8640 1 --2- 172.21.15.45:0/488388689 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f04ec074ce0 0x7f04ec0771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.746+0000 7f04ff4b9640 1 --2- 172.21.15.45:0/488388689 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55f68ba42550 0x55f68ba3db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.747+0000 7f04fecb8640 1 --2- 172.21.15.45:0/488388689 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f04ec074ce0 0x7f04ec0771a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f04f000f4e0 tx=0x7f04f00017e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:44.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.747+0000 7f04ff4b9640 1 --2- 172.21.15.45:0/488388689 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55f68ba42550 0x55f68ba3db70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:44.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.747+0000 7f04ff4b9640 1 -- 172.21.15.45:0/488388689 <== osd.5 v2:172.21.15.63:6808/2395798255 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f04f8051900 con 0x55f68ba42550 2024-05-08T05:42:44.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.747+0000 7f0501e0ef40 1 -- 172.21.15.45:0/488388689 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55f68ba42550 msgr2=0x55f68ba3db70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.747+0000 7f0501e0ef40 1 --2- 172.21.15.45:0/488388689 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55f68ba42550 0x55f68ba3db70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.747+0000 7f0501e0ef40 1 -- 172.21.15.45:0/488388689 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f04ec074ce0 msgr2=0x7f04ec0771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.747+0000 7f0501e0ef40 1 --2- 172.21.15.45:0/488388689 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f04ec074ce0 0x7f04ec0771a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f04f000f4e0 tx=0x7f04f00017e0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.747+0000 7f0501e0ef40 1 -- 172.21.15.45:0/488388689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b9b14d0 msgr2=0x55f68ba42010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.747+0000 7f0501e0ef40 1 --2- 172.21.15.45:0/488388689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f68b9b14d0 0x55f68ba42010 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f04f4016040 tx=0x7f04f4003020 comp rx=0 tx=0).stop 2024-05-08T05:42:44.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.747+0000 7f04ff4b9640 1 -- 172.21.15.45:0/488388689 reap_dead start 2024-05-08T05:42:44.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.747+0000 7f0501e0ef40 1 -- 172.21.15.45:0/488388689 shutdown_connections 2024-05-08T05:42:44.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.747+0000 7f0501e0ef40 1 -- 172.21.15.45:0/488388689 >> 172.21.15.45:0/488388689 conn(0x55f68b967b30 msgr2=0x55f68b9ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:44.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.747+0000 7f0501e0ef40 1 -- 172.21.15.45:0/488388689 shutdown_connections 2024-05-08T05:42:44.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.747+0000 7f0501e0ef40 1 -- 172.21.15.45:0/488388689 wait complete. 2024-05-08T05:42:44.753 INFO:teuthology.orchestra.run.smithi045.stdout:mytag 2024-05-08T05:42:44.753 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-05-08T05:42:44.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.798+0000 7fae42762f40 1 Processor -- start 2024-05-08T05:42:44.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.798+0000 7fae42762f40 1 -- start start 2024-05-08T05:42:44.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae42762f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae2c0022f0 0x5633fd22f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae42762f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633fd2cab00 0x5633fd22fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae42762f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5633fd2ce9e0 0x5633fd2cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae42762f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5633fd203de0 con 0x5633fd2cab00 2024-05-08T05:42:44.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae42762f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5633fd1a7f50 con 0x5633fd2ce9e0 2024-05-08T05:42:44.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae42762f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5633fd203230 con 0x7fae2c0022f0 2024-05-08T05:42:44.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae37fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633fd2cab00 0x5633fd22fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae37fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633fd2cab00 0x5633fd22fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41174/0 (socket says 172.21.15.45:41174) 2024-05-08T05:42:44.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae37fff640 1 -- 172.21.15.45:0/2776057492 learned_addr learned my addr 172.21.15.45:0/2776057492 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:44.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae40a5d640 1 --2- 172.21.15.45:0/2776057492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5633fd2ce9e0 0x5633fd2cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae4025c640 1 --2- 172.21.15.45:0/2776057492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae2c0022f0 0x5633fd22f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae37fff640 1 -- 172.21.15.45:0/2776057492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae2c0022f0 msgr2=0x5633fd22f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:44.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae37fff640 1 --2- 172.21.15.45:0/2776057492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae2c0022f0 0x5633fd22f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae37fff640 1 -- 172.21.15.45:0/2776057492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5633fd2ce9e0 msgr2=0x5633fd2cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:44.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae37fff640 1 --2- 172.21.15.45:0/2776057492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5633fd2ce9e0 0x5633fd2cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae37fff640 1 -- 172.21.15.45:0/2776057492 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633fd2d8880 con 0x5633fd2cab00 2024-05-08T05:42:44.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae4025c640 1 --2- 172.21.15.45:0/2776057492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae2c0022f0 0x5633fd22f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:44.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae37fff640 1 --2- 172.21.15.45:0/2776057492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633fd2cab00 0x5633fd22fa00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fae28009780 tx=0x7fae28009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcacaa4b6510ab0c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:44.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae3f25a640 1 -- 172.21.15.45:0/2776057492 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28019070 con 0x5633fd2cab00 2024-05-08T05:42:44.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae3f25a640 1 -- 172.21.15.45:0/2776057492 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae280088d0 con 0x5633fd2cab00 2024-05-08T05:42:44.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.799+0000 7fae3f25a640 1 -- 172.21.15.45:0/2776057492 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28008ca0 con 0x5633fd2cab00 2024-05-08T05:42:44.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.800+0000 7fae42762f40 1 -- 172.21.15.45:0/2776057492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633fd2cab00 msgr2=0x5633fd22fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.800+0000 7fae42762f40 1 --2- 172.21.15.45:0/2776057492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633fd2cab00 0x5633fd22fa00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fae28009780 tx=0x7fae28009870 comp rx=0 tx=0).stop 2024-05-08T05:42:44.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.800+0000 7fae42762f40 1 -- 172.21.15.45:0/2776057492 shutdown_connections 2024-05-08T05:42:44.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.800+0000 7fae42762f40 1 --2- 172.21.15.45:0/2776057492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5633fd2ce9e0 0x5633fd2cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.800+0000 7fae42762f40 1 --2- 172.21.15.45:0/2776057492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633fd2cab00 0x5633fd22fa00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.800+0000 7fae42762f40 1 --2- 172.21.15.45:0/2776057492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae2c0022f0 0x5633fd22f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.800+0000 7fae42762f40 1 -- 172.21.15.45:0/2776057492 >> 172.21.15.45:0/2776057492 conn(0x5633fd22db40 msgr2=0x5633fd22d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:44.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.800+0000 7fae42762f40 1 -- 172.21.15.45:0/2776057492 shutdown_connections 2024-05-08T05:42:44.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.800+0000 7fae42762f40 1 -- 172.21.15.45:0/2776057492 wait complete. 2024-05-08T05:42:44.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.800+0000 7fae42762f40 1 Processor -- start 2024-05-08T05:42:44.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.801+0000 7fae42762f40 1 -- start start 2024-05-08T05:42:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.801+0000 7fae42762f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633fd2cab00 0x5633fd2d7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.801+0000 7fae42762f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5633fd2ce9e0 0x5633fd2c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.801+0000 7fae42762f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae2c0022f0 0x5633fd2c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.801+0000 7fae42762f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5633fd2d85a0 con 0x5633fd2cab00 2024-05-08T05:42:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.801+0000 7fae42762f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5633fd2430b0 con 0x5633fd2ce9e0 2024-05-08T05:42:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.801+0000 7fae42762f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5633fd2425d0 con 0x7fae2c0022f0 2024-05-08T05:42:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.801+0000 7fae4025c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633fd2cab00 0x5633fd2d7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.801+0000 7fae4025c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633fd2cab00 0x5633fd2d7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41182/0 (socket says 172.21.15.45:41182) 2024-05-08T05:42:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.801+0000 7fae4025c640 1 -- 172.21.15.45:0/106849018 learned_addr learned my addr 172.21.15.45:0/106849018 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.801+0000 7fae37fff640 1 --2- 172.21.15.45:0/106849018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5633fd2ce9e0 0x5633fd2c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.801+0000 7fae40a5d640 1 --2- 172.21.15.45:0/106849018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae2c0022f0 0x5633fd2c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.802+0000 7fae4025c640 1 -- 172.21.15.45:0/106849018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae2c0022f0 msgr2=0x5633fd2c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.802+0000 7fae4025c640 1 --2- 172.21.15.45:0/106849018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fae2c0022f0 0x5633fd2c5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.802+0000 7fae4025c640 1 -- 172.21.15.45:0/106849018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5633fd2ce9e0 msgr2=0x5633fd2c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.802+0000 7fae4025c640 1 --2- 172.21.15.45:0/106849018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5633fd2ce9e0 0x5633fd2c60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.802+0000 7fae4025c640 1 -- 172.21.15.45:0/106849018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633fd2c76c0 con 0x5633fd2cab00 2024-05-08T05:42:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.802+0000 7fae4025c640 1 --2- 172.21.15.45:0/106849018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633fd2cab00 0x5633fd2d7970 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fae30026410 tx=0x7fae30001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.802+0000 7fae3e258640 1 -- 172.21.15.45:0/106849018 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30065070 con 0x5633fd2cab00 2024-05-08T05:42:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.802+0000 7fae3e258640 1 -- 172.21.15.45:0/106849018 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae30061620 con 0x5633fd2cab00 2024-05-08T05:42:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.802+0000 7fae42762f40 1 -- 172.21.15.45:0/106849018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5633fd2c9190 con 0x5633fd2cab00 2024-05-08T05:42:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.802+0000 7fae3e258640 1 -- 172.21.15.45:0/106849018 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30009760 con 0x5633fd2cab00 2024-05-08T05:42:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.802+0000 7fae42762f40 1 -- 172.21.15.45:0/106849018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633fd22d640 con 0x5633fd2cab00 2024-05-08T05:42:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.804+0000 7fae3e258640 1 -- 172.21.15.45:0/106849018 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fae30068070 con 0x5633fd2cab00 2024-05-08T05:42:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.804+0000 7fae3e258640 1 --2- 172.21.15.45:0/106849018 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fae20074cb0 0x7fae20077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.804+0000 7fae3e258640 1 -- 172.21.15.45:0/106849018 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7fae300db0b0 con 0x5633fd2cab00 2024-05-08T05:42:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.804+0000 7fae37fff640 1 --2- 172.21.15.45:0/106849018 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fae20074cb0 0x7fae20077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.804+0000 7fae42762f40 1 --2- 172.21.15.45:0/106849018 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x5633fd2cd020 0x5633fd2c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.804+0000 7fae42762f40 1 -- 172.21.15.45:0/106849018 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) -- 0x5633fd2c9bf0 con 0x5633fd2cd020 2024-05-08T05:42:44.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.805+0000 7fae40a5d640 1 --2- 172.21.15.45:0/106849018 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x5633fd2cd020 0x5633fd2c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.805+0000 7fae37fff640 1 --2- 172.21.15.45:0/106849018 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fae20074cb0 0x7fae20077170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fae28009780 tx=0x7fae2800c030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:44.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.805+0000 7fae40a5d640 1 --2- 172.21.15.45:0/106849018 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x5633fd2cd020 0x5633fd2c96b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:44.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.805+0000 7fae40a5d640 1 -- 172.21.15.45:0/106849018 <== osd.5 v2:172.21.15.63:6808/2395798255 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fae38011040 con 0x5633fd2cd020 2024-05-08T05:42:44.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.805+0000 7fae42762f40 1 -- 172.21.15.45:0/106849018 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x5633fd2cd020 msgr2=0x5633fd2c96b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.805+0000 7fae42762f40 1 --2- 172.21.15.45:0/106849018 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x5633fd2cd020 0x5633fd2c96b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.805+0000 7fae42762f40 1 -- 172.21.15.45:0/106849018 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fae20074cb0 msgr2=0x7fae20077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.805+0000 7fae42762f40 1 --2- 172.21.15.45:0/106849018 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fae20074cb0 0x7fae20077170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fae28009780 tx=0x7fae2800c030 comp rx=0 tx=0).stop 2024-05-08T05:42:44.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.805+0000 7fae42762f40 1 -- 172.21.15.45:0/106849018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633fd2cab00 msgr2=0x5633fd2d7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.805+0000 7fae42762f40 1 --2- 172.21.15.45:0/106849018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5633fd2cab00 0x5633fd2d7970 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fae30026410 tx=0x7fae30001a30 comp rx=0 tx=0).stop 2024-05-08T05:42:44.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.805+0000 7fae40a5d640 1 -- 172.21.15.45:0/106849018 reap_dead start 2024-05-08T05:42:44.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.806+0000 7fae42762f40 1 -- 172.21.15.45:0/106849018 shutdown_connections 2024-05-08T05:42:44.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.806+0000 7fae42762f40 1 -- 172.21.15.45:0/106849018 >> 172.21.15.45:0/106849018 conn(0x5633fd22db40 msgr2=0x5633fd241970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:44.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.806+0000 7fae42762f40 1 -- 172.21.15.45:0/106849018 shutdown_connections 2024-05-08T05:42:44.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.806+0000 7fae42762f40 1 -- 172.21.15.45:0/106849018 wait complete. 2024-05-08T05:42:44.810 INFO:teuthology.orchestra.run.smithi045.stdout:mytag 2024-05-08T05:42:44.811 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-05-08T05:42:44.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.856+0000 7f1cadd0ff40 1 Processor -- start 2024-05-08T05:42:44.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.856+0000 7f1cadd0ff40 1 -- start start 2024-05-08T05:42:44.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.856+0000 7f1cadd0ff40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560dce386110 0x560dce2de5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.856+0000 7f1cadd0ff40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dce3864f0 0x560dce384f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.856+0000 7f1cadd0ff40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560dce3868d0 0x560dce3854a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.856+0000 7f1cadd0ff40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560dce2daf20 con 0x560dce3864f0 2024-05-08T05:42:44.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.856+0000 7f1cadd0ff40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560dce258f50 con 0x560dce386110 2024-05-08T05:42:44.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.856+0000 7f1cadd0ff40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x560dce2df9c0 con 0x560dce3868d0 2024-05-08T05:42:44.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.857+0000 7f1ca3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dce3864f0 0x560dce384f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.857+0000 7f1ca3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dce3864f0 0x560dce384f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41206/0 (socket says 172.21.15.45:41206) 2024-05-08T05:42:44.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.857+0000 7f1ca3fff640 1 -- 172.21.15.45:0/974324892 learned_addr learned my addr 172.21.15.45:0/974324892 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:44.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.857+0000 7f1cab80b640 1 --2- 172.21.15.45:0/974324892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560dce386110 0x560dce2de5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.857+0000 7f1ca3fff640 1 -- 172.21.15.45:0/974324892 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560dce3868d0 msgr2=0x560dce3854a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.857+0000 7f1ca3fff640 1 --2- 172.21.15.45:0/974324892 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560dce3868d0 0x560dce3854a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.857+0000 7f1ca3fff640 1 -- 172.21.15.45:0/974324892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560dce386110 msgr2=0x560dce2de5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.857+0000 7f1ca3fff640 1 --2- 172.21.15.45:0/974324892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560dce386110 0x560dce2de5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.857+0000 7f1ca3fff640 1 -- 172.21.15.45:0/974324892 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dce37f7d0 con 0x560dce3864f0 2024-05-08T05:42:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.857+0000 7f1ca3fff640 1 --2- 172.21.15.45:0/974324892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dce3864f0 0x560dce384f60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1c940098b0 tx=0x7f1c94005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262ba7c15f76eb3d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.857+0000 7f1caa809640 1 -- 172.21.15.45:0/974324892 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c94008930 con 0x560dce3864f0 2024-05-08T05:42:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.857+0000 7f1caa809640 1 -- 172.21.15.45:0/974324892 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c94018070 con 0x560dce3864f0 2024-05-08T05:42:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.857+0000 7f1caa809640 1 -- 172.21.15.45:0/974324892 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c94008cd0 con 0x560dce3864f0 2024-05-08T05:42:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.858+0000 7f1cadd0ff40 1 -- 172.21.15.45:0/974324892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dce3864f0 msgr2=0x560dce384f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.858+0000 7f1cadd0ff40 1 --2- 172.21.15.45:0/974324892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dce3864f0 0x560dce384f60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1c940098b0 tx=0x7f1c94005a00 comp rx=0 tx=0).stop 2024-05-08T05:42:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.858+0000 7f1cadd0ff40 1 -- 172.21.15.45:0/974324892 shutdown_connections 2024-05-08T05:42:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.858+0000 7f1cadd0ff40 1 --2- 172.21.15.45:0/974324892 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560dce3868d0 0x560dce3854a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.858+0000 7f1cadd0ff40 1 --2- 172.21.15.45:0/974324892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dce3864f0 0x560dce384f60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.858+0000 7f1cadd0ff40 1 --2- 172.21.15.45:0/974324892 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560dce386110 0x560dce2de5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.858+0000 7f1cadd0ff40 1 -- 172.21.15.45:0/974324892 >> 172.21.15.45:0/974324892 conn(0x560dce2b07e0 msgr2=0x560dce2dc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.858+0000 7f1cadd0ff40 1 -- 172.21.15.45:0/974324892 shutdown_connections 2024-05-08T05:42:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.858+0000 7f1cadd0ff40 1 -- 172.21.15.45:0/974324892 wait complete. 2024-05-08T05:42:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.859+0000 7f1cadd0ff40 1 Processor -- start 2024-05-08T05:42:44.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.859+0000 7f1cadd0ff40 1 -- start start 2024-05-08T05:42:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.859+0000 7f1cadd0ff40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560dce386110 0x560dce384d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.859+0000 7f1cadd0ff40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560dce3864f0 0x560dce385290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.859+0000 7f1cadd0ff40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dce3868d0 0x560dce37b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.859+0000 7f1cadd0ff40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560dce38a0c0 con 0x560dce3868d0 2024-05-08T05:42:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.859+0000 7f1cadd0ff40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560dce37eab0 con 0x560dce386110 2024-05-08T05:42:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.859+0000 7f1cadd0ff40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x560dce2b3410 con 0x560dce3864f0 2024-05-08T05:42:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.859+0000 7f1cac00c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dce3868d0 0x560dce37b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.859+0000 7f1cac00c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dce3868d0 0x560dce37b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41210/0 (socket says 172.21.15.45:41210) 2024-05-08T05:42:44.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.860+0000 7f1cac00c640 1 -- 172.21.15.45:0/925759387 learned_addr learned my addr 172.21.15.45:0/925759387 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.860+0000 7f1ca3fff640 1 --2- 172.21.15.45:0/925759387 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560dce3864f0 0x560dce385290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.860+0000 7f1cab80b640 1 --2- 172.21.15.45:0/925759387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560dce386110 0x560dce384d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.860+0000 7f1cac00c640 1 -- 172.21.15.45:0/925759387 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560dce3864f0 msgr2=0x560dce385290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.860+0000 7f1cac00c640 1 --2- 172.21.15.45:0/925759387 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560dce3864f0 0x560dce385290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.860+0000 7f1cac00c640 1 -- 172.21.15.45:0/925759387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560dce386110 msgr2=0x560dce384d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.860+0000 7f1cac00c640 1 --2- 172.21.15.45:0/925759387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560dce386110 0x560dce384d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.860+0000 7f1cac00c640 1 -- 172.21.15.45:0/925759387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dce37fcd0 con 0x560dce3868d0 2024-05-08T05:42:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.860+0000 7f1ca3fff640 1 --2- 172.21.15.45:0/925759387 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560dce3864f0 0x560dce385290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.860+0000 7f1cab80b640 1 --2- 172.21.15.45:0/925759387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560dce386110 0x560dce384d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.860+0000 7f1cac00c640 1 --2- 172.21.15.45:0/925759387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dce3868d0 0x560dce37b310 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4006f70 tx=0x7f1ca400b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.860+0000 7f1ca9807640 1 -- 172.21.15.45:0/925759387 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca40062e0 con 0x560dce3868d0 2024-05-08T05:42:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.860+0000 7f1ca9807640 1 -- 172.21.15.45:0/925759387 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca400be40 con 0x560dce3868d0 2024-05-08T05:42:44.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.860+0000 7f1cadd0ff40 1 -- 172.21.15.45:0/925759387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560dce37e120 con 0x560dce3868d0 2024-05-08T05:42:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.860+0000 7f1ca9807640 1 -- 172.21.15.45:0/925759387 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4003690 con 0x560dce3868d0 2024-05-08T05:42:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.860+0000 7f1cadd0ff40 1 -- 172.21.15.45:0/925759387 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dce382920 con 0x560dce3868d0 2024-05-08T05:42:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.862+0000 7f1ca9807640 1 -- 172.21.15.45:0/925759387 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1ca4010070 con 0x560dce3868d0 2024-05-08T05:42:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.862+0000 7f1ca9807640 1 --2- 172.21.15.45:0/925759387 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1c8c074ce0 0x7f1c8c0771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.862+0000 7f1ca9807640 1 -- 172.21.15.45:0/925759387 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f1ca4095dd0 con 0x560dce3868d0 2024-05-08T05:42:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.863+0000 7f1cadd0ff40 1 --2- 172.21.15.45:0/925759387 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x560dce375480 0x560dce426e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.863+0000 7f1cadd0ff40 1 -- 172.21.15.45:0/925759387 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) -- 0x560dce2df240 con 0x560dce375480 2024-05-08T05:42:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.863+0000 7f1cab80b640 1 --2- 172.21.15.45:0/925759387 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1c8c074ce0 0x7f1c8c0771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.863+0000 7f1ca3fff640 1 --2- 172.21.15.45:0/925759387 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x560dce375480 0x560dce426e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.863+0000 7f1cab80b640 1 --2- 172.21.15.45:0/925759387 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1c8c074ce0 0x7f1c8c0771a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c058000 tx=0x7f1c9c00a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.863+0000 7f1ca3fff640 1 --2- 172.21.15.45:0/925759387 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x560dce375480 0x560dce426e70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.863+0000 7f1ca3fff640 1 -- 172.21.15.45:0/925759387 <== osd.7 v2:172.21.15.63:6816/1163469437 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f1c94008a90 con 0x560dce375480 2024-05-08T05:42:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-05-08T05:42:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.864+0000 7f1cadd0ff40 1 -- 172.21.15.45:0/925759387 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x560dce375480 msgr2=0x560dce426e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.864+0000 7f1cadd0ff40 1 --2- 172.21.15.45:0/925759387 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x560dce375480 0x560dce426e70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.864+0000 7f1cadd0ff40 1 -- 172.21.15.45:0/925759387 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1c8c074ce0 msgr2=0x7f1c8c0771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.864+0000 7f1cadd0ff40 1 --2- 172.21.15.45:0/925759387 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1c8c074ce0 0x7f1c8c0771a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c058000 tx=0x7f1c9c00a000 comp rx=0 tx=0).stop 2024-05-08T05:42:44.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.864+0000 7f1cadd0ff40 1 -- 172.21.15.45:0/925759387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dce3868d0 msgr2=0x560dce37b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:44.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.864+0000 7f1cadd0ff40 1 --2- 172.21.15.45:0/925759387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560dce3868d0 0x560dce37b310 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4006f70 tx=0x7f1ca400b4b0 comp rx=0 tx=0).stop 2024-05-08T05:42:44.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.864+0000 7f1cac00c640 1 -- 172.21.15.45:0/925759387 reap_dead start 2024-05-08T05:42:44.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.864+0000 7f1cadd0ff40 1 -- 172.21.15.45:0/925759387 shutdown_connections 2024-05-08T05:42:44.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.864+0000 7f1cadd0ff40 1 -- 172.21.15.45:0/925759387 >> 172.21.15.45:0/925759387 conn(0x560dce2b07e0 msgr2=0x560dce2de5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:44.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.864+0000 7f1cadd0ff40 1 -- 172.21.15.45:0/925759387 shutdown_connections 2024-05-08T05:42:44.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:44.864+0000 7f1cadd0ff40 1 -- 172.21.15.45:0/925759387 wait complete. 2024-05-08T05:42:44.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:42:44.869 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T05:42:44.869 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:42:44.906 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:42:44.958 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T05:42:44.958 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:42:44.958 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:42:44.959 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:44.975 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:42:44.975 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:42:44.976 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:42:44.976 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:44.994 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:42:44.994 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:42:44.994 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:42:44.995 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:45.032 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:42:45.032 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:42:45.032 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:42:45.032 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:45.052 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:42:45.052 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:42:45.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.175+0000 7fbcebe72640 1 Processor -- start 2024-05-08T05:42:45.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.175+0000 7fbcebe72640 1 -- start start 2024-05-08T05:42:45.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.176+0000 7fbcebe72640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbce40d3cd0 0x7fbce40cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.176+0000 7fbcebe72640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbce40c9ee0 0x7fbce40d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.176+0000 7fbcebe72640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbce40d3330 con 0x7fbce41629a0 2024-05-08T05:42:45.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.176+0000 7fbcebe72640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fbce40d2170 con 0x7fbce40d3cd0 2024-05-08T05:42:45.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.176+0000 7fbcebe72640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbce40d2450 con 0x7fbce40c9ee0 2024-05-08T05:42:45.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.176+0000 7fbce940d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbce40d3cd0 0x7fbce40cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.176+0000 7fbce940d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbce40d3cd0 0x7fbce40cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41218/0 (socket says 172.21.15.45:41218) 2024-05-08T05:42:45.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.176+0000 7fbce940d640 1 -- 172.21.15.45:0/2076311508 learned_addr learned my addr 172.21.15.45:0/2076311508 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:45.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.176+0000 7fbcea40f640 1 --2- 172.21.15.45:0/2076311508 >> v2:172.21.15.63:3300/0 conn(0x7fbce40c9ee0 0x7fbce40d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.176+0000 7fbce940d640 1 -- 172.21.15.45:0/2076311508 >> v2:172.21.15.63:3300/0 conn(0x7fbce40c9ee0 msgr2=0x7fbce40d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:45.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.176+0000 7fbce940d640 1 --2- 172.21.15.45:0/2076311508 >> v2:172.21.15.63:3300/0 conn(0x7fbce40c9ee0 0x7fbce40d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.176+0000 7fbce940d640 1 -- 172.21.15.45:0/2076311508 >> v1:172.21.15.63:6789/0 conn(0x7fbce41629a0 legacy=0x7fbce40d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:45.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.176+0000 7fbce940d640 1 -- 172.21.15.45:0/2076311508 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce40d2730 con 0x7fbce40d3cd0 2024-05-08T05:42:45.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.177+0000 7fbce940d640 1 --2- 172.21.15.45:0/2076311508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbce40d3cd0 0x7fbce40cfca0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbce408e4c0 tx=0x7fbcd402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04fc174c70f8cdd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:45.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.177+0000 7fbce8c0c640 1 -- 172.21.15.45:0/2076311508 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4037070 con 0x7fbce40d3cd0 2024-05-08T05:42:45.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.177+0000 7fbce8c0c640 1 -- 172.21.15.45:0/2076311508 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcd402ae00 con 0x7fbce40d3cd0 2024-05-08T05:42:45.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.177+0000 7fbce8c0c640 1 -- 172.21.15.45:0/2076311508 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4032660 con 0x7fbce40d3cd0 2024-05-08T05:42:45.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.178+0000 7fbcebe72640 1 -- 172.21.15.45:0/2076311508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbce40d3cd0 msgr2=0x7fbce40cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:45.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.178+0000 7fbcebe72640 1 --2- 172.21.15.45:0/2076311508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbce40d3cd0 0x7fbce40cfca0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbce408e4c0 tx=0x7fbcd402a0b0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.178+0000 7fbcebe72640 1 -- 172.21.15.45:0/2076311508 shutdown_connections 2024-05-08T05:42:45.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.178+0000 7fbcebe72640 1 --2- 172.21.15.45:0/2076311508 >> v2:172.21.15.63:3300/0 conn(0x7fbce40c9ee0 0x7fbce40d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.178+0000 7fbcebe72640 1 --2- 172.21.15.45:0/2076311508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbce40d3cd0 0x7fbce40cfca0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.178+0000 7fbcebe72640 1 -- 172.21.15.45:0/2076311508 >> 172.21.15.45:0/2076311508 conn(0x7fbce40d4fe0 msgr2=0x7fbce40d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:45.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.178+0000 7fbcebe72640 1 -- 172.21.15.45:0/2076311508 shutdown_connections 2024-05-08T05:42:45.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.178+0000 7fbcebe72640 1 -- 172.21.15.45:0/2076311508 wait complete. 2024-05-08T05:42:45.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.179+0000 7fbcebe72640 1 Processor -- start 2024-05-08T05:42:45.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.179+0000 7fbcebe72640 1 -- start start 2024-05-08T05:42:45.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.180+0000 7fbcebe72640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbce40c9ee0 0x7fbce415d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.180+0000 7fbcebe72640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbce40d3cd0 0x7fbce40b8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.180+0000 7fbcebe72640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbce41629a0 0x7fbce415f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.180+0000 7fbcebe72640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbce40cf440 con 0x7fbce40d3cd0 2024-05-08T05:42:45.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.180+0000 7fbcebe72640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbce40cf2c0 con 0x7fbce41629a0 2024-05-08T05:42:45.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.180+0000 7fbcebe72640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbce40d2dc0 con 0x7fbce40c9ee0 2024-05-08T05:42:45.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.180+0000 7fbce940d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbce40d3cd0 0x7fbce40b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.180+0000 7fbce940d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbce40d3cd0 0x7fbce40b8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41230/0 (socket says 172.21.15.45:41230) 2024-05-08T05:42:45.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.180+0000 7fbce940d640 1 -- 172.21.15.45:0/3008884020 learned_addr learned my addr 172.21.15.45:0/3008884020 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:45.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.180+0000 7fbcea40f640 1 --2- 172.21.15.45:0/3008884020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbce41629a0 0x7fbce415f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.180+0000 7fbce9c0e640 1 --2- 172.21.15.45:0/3008884020 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbce40c9ee0 0x7fbce415d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.180+0000 7fbce940d640 1 -- 172.21.15.45:0/3008884020 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbce40c9ee0 msgr2=0x7fbce415d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:45.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.180+0000 7fbce940d640 1 --2- 172.21.15.45:0/3008884020 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbce40c9ee0 0x7fbce415d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.180+0000 7fbce940d640 1 -- 172.21.15.45:0/3008884020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbce41629a0 msgr2=0x7fbce415f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:45.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.180+0000 7fbce940d640 1 --2- 172.21.15.45:0/3008884020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbce41629a0 0x7fbce415f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.180+0000 7fbce940d640 1 -- 172.21.15.45:0/3008884020 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce40b9d70 con 0x7fbce40d3cd0 2024-05-08T05:42:45.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.181+0000 7fbce940d640 1 --2- 172.21.15.45:0/3008884020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbce40d3cd0 0x7fbce40b8470 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4000c00 tx=0x7fbcd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:45.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.181+0000 7fbcda7fc640 1 -- 172.21.15.45:0/3008884020 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4041070 con 0x7fbce40d3cd0 2024-05-08T05:42:45.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.181+0000 7fbcda7fc640 1 -- 172.21.15.45:0/3008884020 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcd403c670 con 0x7fbce40d3cd0 2024-05-08T05:42:45.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.181+0000 7fbcebe72640 1 -- 172.21.15.45:0/3008884020 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce40ba050 con 0x7fbce40d3cd0 2024-05-08T05:42:45.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.181+0000 7fbcda7fc640 1 -- 172.21.15.45:0/3008884020 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4037040 con 0x7fbce40d3cd0 2024-05-08T05:42:45.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.181+0000 7fbcebe72640 1 -- 172.21.15.45:0/3008884020 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce40ba610 con 0x7fbce40d3cd0 2024-05-08T05:42:45.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.182+0000 7fbcebe72640 1 -- 172.21.15.45:0/3008884020 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce40693f0 con 0x7fbce40d3cd0 2024-05-08T05:42:45.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.182+0000 7fbcda7fc640 1 -- 172.21.15.45:0/3008884020 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbcd403a070 con 0x7fbce40d3cd0 2024-05-08T05:42:45.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.183+0000 7fbcda7fc640 1 --2- 172.21.15.45:0/3008884020 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbcc80762d0 0x7fbcc8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.183+0000 7fbcda7fc640 1 -- 172.21.15.45:0/3008884020 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7fbcd40b7ae0 con 0x7fbce40d3cd0 2024-05-08T05:42:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.183+0000 7fbce9c0e640 1 --2- 172.21.15.45:0/3008884020 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbcc80762d0 0x7fbcc8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.184+0000 7fbce9c0e640 1 --2- 172.21.15.45:0/3008884020 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbcc80762d0 0x7fbcc8078790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009fd0 tx=0x7fbce000d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:45.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.188+0000 7fbcda7fc640 1 -- 172.21.15.45:0/3008884020 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbcd4080cf0 con 0x7fbce40d3cd0 2024-05-08T05:42:45.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.419+0000 7fbcebe72640 1 -- 172.21.15.45:0/3008884020 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbce40b6fa0 con 0x7fbce40d3cd0 2024-05-08T05:42:45.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.419+0000 7fbcda7fc640 1 -- 172.21.15.45:0/3008884020 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5172 (secure 0 0 0) 0x7fbcd4084ff0 con 0x7fbce40d3cd0 2024-05-08T05:42:45.420 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:45.420 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3295938044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3295938044},{"type":"v1","addr":"172.21.15.63:6836","nonce":3295938044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4024530715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4024530715},{"type":"v1","addr":"172.21.15.63:6837","nonce":4024530715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/97258541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":97258541},{"type":"v1","addr":"172.21.15.45:6834","nonce":97258541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:42:19.860331+0000","modified":"2024-05-08T05:42:22.891006+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4884":{"gid":4884,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/173436628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":173436628},{"type":"v1","addr":"172.21.15.45:6835","nonce":173436628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4884,"qdb_cluster":[4884]},"id":2}]} 2024-05-08T05:42:45.420 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2024-05-08T05:42:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.421+0000 7fbcebe72640 1 -- 172.21.15.45:0/3008884020 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbcc80762d0 msgr2=0x7fbcc8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.421+0000 7fbcebe72640 1 --2- 172.21.15.45:0/3008884020 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbcc80762d0 0x7fbcc8078790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009fd0 tx=0x7fbce000d040 comp rx=0 tx=0).stop 2024-05-08T05:42:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.421+0000 7fbcebe72640 1 -- 172.21.15.45:0/3008884020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbce40d3cd0 msgr2=0x7fbce40b8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.421+0000 7fbcebe72640 1 --2- 172.21.15.45:0/3008884020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbce40d3cd0 0x7fbce40b8470 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4000c00 tx=0x7fbcd40046c0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.421+0000 7fbcebe72640 1 -- 172.21.15.45:0/3008884020 shutdown_connections 2024-05-08T05:42:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.421+0000 7fbcebe72640 1 --2- 172.21.15.45:0/3008884020 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbcc80762d0 0x7fbcc8078790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.421+0000 7fbcebe72640 1 --2- 172.21.15.45:0/3008884020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbce41629a0 0x7fbce415f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.421+0000 7fbcebe72640 1 --2- 172.21.15.45:0/3008884020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbce40d3cd0 0x7fbce40b8470 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.421+0000 7fbcebe72640 1 --2- 172.21.15.45:0/3008884020 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbce40c9ee0 0x7fbce415d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.421+0000 7fbcebe72640 1 -- 172.21.15.45:0/3008884020 >> 172.21.15.45:0/3008884020 conn(0x7fbce40d4fe0 msgr2=0x7fbce40d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:45.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.421+0000 7fbcebe72640 1 -- 172.21.15.45:0/3008884020 shutdown_connections 2024-05-08T05:42:45.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.421+0000 7fbcebe72640 1 -- 172.21.15.45:0/3008884020 wait complete. 2024-05-08T05:42:45.433 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:42:45.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.537+0000 7f893c55f640 1 Processor -- start 2024-05-08T05:42:45.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.537+0000 7f893c55f640 1 -- start start 2024-05-08T05:42:45.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.538+0000 7f893c55f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89340c9ee0 0x7f89340d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.538+0000 7f893c55f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89340d3330 con 0x7f89341629a0 2024-05-08T05:42:45.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.538+0000 7f893c55f640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89340d2170 con 0x7f89340d3cd0 2024-05-08T05:42:45.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.538+0000 7f893c55f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f89340d2450 con 0x7f89340c9ee0 2024-05-08T05:42:45.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.538+0000 7f893a2fb640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f89341629a0 0x7f89340d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52760/0 (socket says 172.21.15.45:52760) 2024-05-08T05:42:45.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.538+0000 7f893a2fb640 1 -- 172.21.15.45:0/915744502 learned_addr learned my addr 172.21.15.45:0/915744502 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:45.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.538+0000 7f893aafc640 1 --2- 172.21.15.45:0/915744502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89340c9ee0 0x7f89340d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.538+0000 7f89392f9640 1 -- 172.21.15.45:0/915744502 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 300053877 0 0) 0x7f89340d3330 con 0x7f89341629a0 2024-05-08T05:42:45.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.538+0000 7f89392f9640 1 -- 172.21.15.45:0/915744502 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8920003520 con 0x7f89341629a0 2024-05-08T05:42:45.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.538+0000 7f89392f9640 1 -- 172.21.15.45:0/915744502 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 432919373 0 0) 0x7f89340d2170 con 0x7f89340d3cd0 2024-05-08T05:42:45.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.538+0000 7f89392f9640 1 -- 172.21.15.45:0/915744502 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89340d3330 con 0x7f89340d3cd0 2024-05-08T05:42:45.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.538+0000 7f89392f9640 1 -- 172.21.15.45:0/915744502 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 305303892 0 0) 0x7f8920003520 con 0x7f89341629a0 2024-05-08T05:42:45.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.538+0000 7f89392f9640 1 -- 172.21.15.45:0/915744502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89340c9ee0 msgr2=0x7f89340d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:45.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.538+0000 7f89392f9640 1 --2- 172.21.15.45:0/915744502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89340c9ee0 0x7f89340d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.538+0000 7f89392f9640 1 -- 172.21.15.45:0/915744502 >> v1:172.21.15.63:6789/0 conn(0x7f89340d3cd0 legacy=0x7f89340cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:45.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.538+0000 7f89392f9640 1 -- 172.21.15.45:0/915744502 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89340d2730 con 0x7f89341629a0 2024-05-08T05:42:45.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.538+0000 7f89392f9640 1 -- 172.21.15.45:0/915744502 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f8930001a70 con 0x7f89341629a0 2024-05-08T05:42:45.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.538+0000 7f89392f9640 1 -- 172.21.15.45:0/915744502 >> v1:172.21.15.45:6789/0 conn(0x7f89341629a0 legacy=0x7f89340d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:45.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.539+0000 7f89392f9640 1 --2- 172.21.15.45:0/915744502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8920003c60 0x7f8920004060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.539+0000 7f89392f9640 1 --2- 172.21.15.45:0/915744502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8920004800 0x7f8920004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.539+0000 7f89392f9640 1 --2- 172.21.15.45:0/915744502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8920005340 0x7f892000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.539+0000 7f89392f9640 1 -- 172.21.15.45:0/915744502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f89340d2450 con 0x7f8920005340 2024-05-08T05:42:45.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.539+0000 7f89392f9640 1 -- 172.21.15.45:0/915744502 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f892000c230 con 0x7f8920003c60 2024-05-08T05:42:45.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.539+0000 7f89392f9640 1 -- 172.21.15.45:0/915744502 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f892000c430 con 0x7f8920004800 2024-05-08T05:42:45.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.539+0000 7f893aafc640 1 --2- 172.21.15.45:0/915744502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f89340c9ee0 0x7f89340d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:42:45.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.539+0000 7f893aafc640 1 --2- 172.21.15.45:0/915744502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8920005340 0x7f892000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.539+0000 7f8939afa640 1 --2- 172.21.15.45:0/915744502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8920004800 0x7f8920004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.539+0000 7f893a2fb640 1 --2- 172.21.15.45:0/915744502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8920003c60 0x7f8920004060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.539+0000 7f893aafc640 1 -- 172.21.15.45:0/915744502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8920004800 msgr2=0x7f8920004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:45.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.539+0000 7f893aafc640 1 --2- 172.21.15.45:0/915744502 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8920004800 0x7f8920004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.539+0000 7f893aafc640 1 -- 172.21.15.45:0/915744502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8920003c60 msgr2=0x7f8920004060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:45.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.539+0000 7f893aafc640 1 --2- 172.21.15.45:0/915744502 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8920003c60 0x7f8920004060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.539+0000 7f893aafc640 1 -- 172.21.15.45:0/915744502 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8920003520 con 0x7f8920005340 2024-05-08T05:42:45.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.540+0000 7f893aafc640 1 --2- 172.21.15.45:0/915744502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8920005340 0x7f892000b9c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f89340ca2c0 tx=0x7f8920005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7d6dcd6b7451f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:45.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.540+0000 7f89392f9640 1 -- 172.21.15.45:0/915744502 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c036070 con 0x7f8920005340 2024-05-08T05:42:45.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.540+0000 7f89392f9640 1 -- 172.21.15.45:0/915744502 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892c004030 con 0x7f8920005340 2024-05-08T05:42:45.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.540+0000 7f89392f9640 1 -- 172.21.15.45:0/915744502 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c0097d0 con 0x7f8920005340 2024-05-08T05:42:45.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.540+0000 7f893c55f640 1 -- 172.21.15.45:0/915744502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8920005340 msgr2=0x7f892000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:45.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.540+0000 7f893c55f640 1 --2- 172.21.15.45:0/915744502 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8920005340 0x7f892000b9c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f89340ca2c0 tx=0x7f8920005830 comp rx=0 tx=0).stop 2024-05-08T05:42:45.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.540+0000 7f893aafc640 1 -- 172.21.15.45:0/915744502 reap_dead start 2024-05-08T05:42:45.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.540+0000 7f893c55f640 1 -- 172.21.15.45:0/915744502 shutdown_connections 2024-05-08T05:42:45.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.540+0000 7f893c55f640 1 -- 172.21.15.45:0/915744502 >> 172.21.15.45:0/915744502 conn(0x7f89340d4fe0 msgr2=0x7f89340d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:45.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.541+0000 7f893c55f640 1 -- 172.21.15.45:0/915744502 shutdown_connections 2024-05-08T05:42:45.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.541+0000 7f893c55f640 1 -- 172.21.15.45:0/915744502 wait complete. 2024-05-08T05:42:45.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.542+0000 7f893c55f640 1 Processor -- start 2024-05-08T05:42:45.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.542+0000 7f893c55f640 1 -- start start 2024-05-08T05:42:45.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.542+0000 7f893c55f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8920005340 0x7f89341f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.542+0000 7f893c55f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f893414f490 0x7f89340cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.542+0000 7f893c55f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89340ce290 0x7f89340d0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.542+0000 7f893c55f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f89340cf440 con 0x7f89340ce290 2024-05-08T05:42:45.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.542+0000 7f893c55f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f89340cf2c0 con 0x7f8920005340 2024-05-08T05:42:45.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.542+0000 7f893c55f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f89340d2dc0 con 0x7f893414f490 2024-05-08T05:42:45.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.543+0000 7f893aafc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89340ce290 0x7f89340d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.543+0000 7f893aafc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89340ce290 0x7f89340d0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41248/0 (socket says 172.21.15.45:41248) 2024-05-08T05:42:45.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.543+0000 7f893aafc640 1 -- 172.21.15.45:0/414041279 learned_addr learned my addr 172.21.15.45:0/414041279 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:45.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.543+0000 7f893a2fb640 1 --2- 172.21.15.45:0/414041279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8920005340 0x7f89341f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.543+0000 7f8939afa640 1 --2- 172.21.15.45:0/414041279 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f893414f490 0x7f89340cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.543+0000 7f893aafc640 1 -- 172.21.15.45:0/414041279 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f893414f490 msgr2=0x7f89340cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:45.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.543+0000 7f893aafc640 1 --2- 172.21.15.45:0/414041279 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f893414f490 0x7f89340cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.543+0000 7f893aafc640 1 -- 172.21.15.45:0/414041279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8920005340 msgr2=0x7f89341f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:45.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.543+0000 7f893aafc640 1 --2- 172.21.15.45:0/414041279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8920005340 0x7f89341f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.543+0000 7f893aafc640 1 -- 172.21.15.45:0/414041279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893414f910 con 0x7f89340ce290 2024-05-08T05:42:45.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.543+0000 7f893aafc640 1 --2- 172.21.15.45:0/414041279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89340ce290 0x7f89340d0340 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f892c034040 tx=0x7f892c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:45.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.543+0000 7f892affd640 1 -- 172.21.15.45:0/414041279 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c036040 con 0x7f89340ce290 2024-05-08T05:42:45.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.543+0000 7f892affd640 1 -- 172.21.15.45:0/414041279 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892c043070 con 0x7f89340ce290 2024-05-08T05:42:45.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.543+0000 7f893c55f640 1 -- 172.21.15.45:0/414041279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89341f8510 con 0x7f89340ce290 2024-05-08T05:42:45.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.543+0000 7f892affd640 1 -- 172.21.15.45:0/414041279 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c0097d0 con 0x7f89340ce290 2024-05-08T05:42:45.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.543+0000 7f893c55f640 1 -- 172.21.15.45:0/414041279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89341f8ad0 con 0x7f89340ce290 2024-05-08T05:42:45.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.544+0000 7f893c55f640 1 -- 172.21.15.45:0/414041279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89341f9190 con 0x7f89340ce290 2024-05-08T05:42:45.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.545+0000 7f892affd640 1 -- 172.21.15.45:0/414041279 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f892c03c470 con 0x7f89340ce290 2024-05-08T05:42:45.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.545+0000 7f892affd640 1 --2- 172.21.15.45:0/414041279 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f89180762d0 0x7f8918078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.545+0000 7f892affd640 1 -- 172.21.15.45:0/414041279 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f892c0b7e50 con 0x7f89340ce290 2024-05-08T05:42:45.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.546+0000 7f893a2fb640 1 --2- 172.21.15.45:0/414041279 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f89180762d0 0x7f8918078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.546+0000 7f893a2fb640 1 --2- 172.21.15.45:0/414041279 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f89180762d0 0x7f8918078790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8930004c20 tx=0x7f8930006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:45.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.550+0000 7f892affd640 1 -- 172.21.15.45:0/414041279 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f89341f9190 con 0x7f89340ce290 2024-05-08T05:42:45.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.781+0000 7f893c55f640 1 -- 172.21.15.45:0/414041279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89340693f0 con 0x7f89340ce290 2024-05-08T05:42:45.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.781+0000 7f892affd640 1 -- 172.21.15.45:0/414041279 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5172 (secure 0 0 0) 0x7f892c081590 con 0x7f89340ce290 2024-05-08T05:42:45.782 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:45.782 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3295938044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3295938044},{"type":"v1","addr":"172.21.15.63:6836","nonce":3295938044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4024530715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4024530715},{"type":"v1","addr":"172.21.15.63:6837","nonce":4024530715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/97258541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":97258541},{"type":"v1","addr":"172.21.15.45:6834","nonce":97258541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:42:19.860331+0000","modified":"2024-05-08T05:42:22.891006+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4884":{"gid":4884,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/173436628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":173436628},{"type":"v1","addr":"172.21.15.45:6835","nonce":173436628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4884,"qdb_cluster":[4884]},"id":2}]} 2024-05-08T05:42:45.782 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2024-05-08T05:42:45.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.784+0000 7f893c55f640 1 -- 172.21.15.45:0/414041279 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f89180762d0 msgr2=0x7f8918078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:45.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.784+0000 7f893c55f640 1 --2- 172.21.15.45:0/414041279 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f89180762d0 0x7f8918078790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8930004c20 tx=0x7f8930006040 comp rx=0 tx=0).stop 2024-05-08T05:42:45.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.784+0000 7f893c55f640 1 -- 172.21.15.45:0/414041279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89340ce290 msgr2=0x7f89340d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:45.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.784+0000 7f893c55f640 1 --2- 172.21.15.45:0/414041279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89340ce290 0x7f89340d0340 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f892c034040 tx=0x7f892c004730 comp rx=0 tx=0).stop 2024-05-08T05:42:45.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.784+0000 7f893c55f640 1 -- 172.21.15.45:0/414041279 shutdown_connections 2024-05-08T05:42:45.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.784+0000 7f893c55f640 1 --2- 172.21.15.45:0/414041279 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f89180762d0 0x7f8918078790 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.784+0000 7f893c55f640 1 --2- 172.21.15.45:0/414041279 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f893414f490 0x7f89340cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.784+0000 7f893c55f640 1 --2- 172.21.15.45:0/414041279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f89340ce290 0x7f89340d0340 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.784+0000 7f893c55f640 1 --2- 172.21.15.45:0/414041279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8920005340 0x7f89341f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.784+0000 7f893c55f640 1 -- 172.21.15.45:0/414041279 >> 172.21.15.45:0/414041279 conn(0x7f89340d4fe0 msgr2=0x7f89340d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:45.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.784+0000 7f893c55f640 1 -- 172.21.15.45:0/414041279 shutdown_connections 2024-05-08T05:42:45.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.784+0000 7f893c55f640 1 -- 172.21.15.45:0/414041279 wait complete. 2024-05-08T05:42:45.795 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:42:45.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.898+0000 7f20261f0640 1 Processor -- start 2024-05-08T05:42:45.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.898+0000 7f20261f0640 1 -- start start 2024-05-08T05:42:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.899+0000 7f20261f0640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f202015f420 0x7f20200d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.899+0000 7f20261f0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f20200d3ca0 0x7f20200d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.899+0000 7f20261f0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f20200d1eb0 0x7f20200d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.899+0000 7f20261f0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f20200d22f0 con 0x7f20200d1eb0 2024-05-08T05:42:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.899+0000 7f20261f0640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f20201630b0 con 0x7f202015f420 2024-05-08T05:42:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.899+0000 7f20261f0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f20200d2550 con 0x7f20200d3ca0 2024-05-08T05:42:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.899+0000 7f201ffff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f20200d1eb0 0x7f20200d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.899+0000 7f201ffff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f20200d1eb0 0x7f20200d4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41262/0 (socket says 172.21.15.45:41262) 2024-05-08T05:42:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.899+0000 7f201ffff640 1 -- 172.21.15.45:0/3299512806 learned_addr learned my addr 172.21.15.45:0/3299512806 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:45.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.900+0000 7f201f7fe640 1 --2- 172.21.15.45:0/3299512806 >> v2:172.21.15.63:3300/0 conn(0x7f202015f420 0x7f20200d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.900+0000 7f201effd640 1 --2- 172.21.15.45:0/3299512806 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f20200d3ca0 0x7f20200d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.900+0000 7f201ffff640 1 -- 172.21.15.45:0/3299512806 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f20200d3ca0 msgr2=0x7f20200d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:45.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.900+0000 7f201ffff640 1 --2- 172.21.15.45:0/3299512806 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f20200d3ca0 0x7f20200d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.900+0000 7f201ffff640 1 -- 172.21.15.45:0/3299512806 >> v2:172.21.15.63:3300/0 conn(0x7f202015f420 msgr2=0x7f20200d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:45.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.900+0000 7f201ffff640 1 --2- 172.21.15.45:0/3299512806 >> v2:172.21.15.63:3300/0 conn(0x7f202015f420 0x7f20200d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.900+0000 7f201ffff640 1 -- 172.21.15.45:0/3299512806 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20200d2c80 con 0x7f20200d1eb0 2024-05-08T05:42:45.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.900+0000 7f201ffff640 1 --2- 172.21.15.45:0/3299512806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20200d1eb0 0x7f20200d4cc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f20200d5ec0 tx=0x7f201802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ab842e403b1671 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:45.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.901+0000 7f201e7fc640 1 -- 172.21.15.45:0/3299512806 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018036070 con 0x7f20200d1eb0 2024-05-08T05:42:45.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.901+0000 7f201e7fc640 1 -- 172.21.15.45:0/3299512806 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2018031600 con 0x7f20200d1eb0 2024-05-08T05:42:45.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.901+0000 7f201e7fc640 1 -- 172.21.15.45:0/3299512806 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20180319d0 con 0x7f20200d1eb0 2024-05-08T05:42:45.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.901+0000 7f20261f0640 1 -- 172.21.15.45:0/3299512806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20200d1eb0 msgr2=0x7f20200d4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:45.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.901+0000 7f20261f0640 1 --2- 172.21.15.45:0/3299512806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20200d1eb0 0x7f20200d4cc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f20200d5ec0 tx=0x7f201802a6a0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.901+0000 7f20261f0640 1 -- 172.21.15.45:0/3299512806 shutdown_connections 2024-05-08T05:42:45.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.901+0000 7f20261f0640 1 --2- 172.21.15.45:0/3299512806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20200d1eb0 0x7f20200d4cc0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.901+0000 7f20261f0640 1 --2- 172.21.15.45:0/3299512806 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f20200d3ca0 0x7f20200d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.901+0000 7f20261f0640 1 --2- 172.21.15.45:0/3299512806 >> v2:172.21.15.63:3300/0 conn(0x7f202015f420 0x7f20200d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.901+0000 7f20261f0640 1 -- 172.21.15.45:0/3299512806 >> 172.21.15.45:0/3299512806 conn(0x7f202008eac0 msgr2=0x7f202009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.901+0000 7f20261f0640 1 -- 172.21.15.45:0/3299512806 shutdown_connections 2024-05-08T05:42:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.901+0000 7f20261f0640 1 -- 172.21.15.45:0/3299512806 wait complete. 2024-05-08T05:42:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.902+0000 7f20261f0640 1 Processor -- start 2024-05-08T05:42:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.903+0000 7f20261f0640 1 -- start start 2024-05-08T05:42:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.903+0000 7f20261f0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20200d1eb0 0x7f202015f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.903+0000 7f20261f0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f202015a570 0x7f202015bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.903+0000 7f20261f0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f202015c4d0 0x7f20200c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.903+0000 7f20261f0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f20200d2830 con 0x7f202015c4d0 2024-05-08T05:42:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.903+0000 7f20261f0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f20200d3240 con 0x7f20200d1eb0 2024-05-08T05:42:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.903+0000 7f20261f0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f20200d5200 con 0x7f202015a570 2024-05-08T05:42:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.903+0000 7f201ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f202015c4d0 0x7f20200c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.904+0000 7f201ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f202015c4d0 0x7f20200c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41278/0 (socket says 172.21.15.45:41278) 2024-05-08T05:42:45.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.904+0000 7f201ffff640 1 -- 172.21.15.45:0/3810888000 learned_addr learned my addr 172.21.15.45:0/3810888000 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.904+0000 7f201effd640 1 --2- 172.21.15.45:0/3810888000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f202015a570 0x7f202015bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.904+0000 7f201f7fe640 1 --2- 172.21.15.45:0/3810888000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20200d1eb0 0x7f202015f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.904+0000 7f201ffff640 1 -- 172.21.15.45:0/3810888000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f202015a570 msgr2=0x7f202015bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.904+0000 7f201ffff640 1 --2- 172.21.15.45:0/3810888000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f202015a570 0x7f202015bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.904+0000 7f201ffff640 1 -- 172.21.15.45:0/3810888000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20200d1eb0 msgr2=0x7f202015f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.904+0000 7f201ffff640 1 --2- 172.21.15.45:0/3810888000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20200d1eb0 0x7f202015f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.904+0000 7f201ffff640 1 -- 172.21.15.45:0/3810888000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202015fd50 con 0x7f202015c4d0 2024-05-08T05:42:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.904+0000 7f201effd640 1 --2- 172.21.15.45:0/3810888000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f202015a570 0x7f202015bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.904+0000 7f201ffff640 1 --2- 172.21.15.45:0/3810888000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f202015c4d0 0x7f20200c8d50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2018009f40 tx=0x7f2018004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.904+0000 7f2007fff640 1 -- 172.21.15.45:0/3810888000 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018036050 con 0x7f202015c4d0 2024-05-08T05:42:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.904+0000 7f2007fff640 1 -- 172.21.15.45:0/3810888000 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2018042070 con 0x7f202015c4d0 2024-05-08T05:42:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.904+0000 7f20261f0640 1 -- 172.21.15.45:0/3810888000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20200cc180 con 0x7f202015c4d0 2024-05-08T05:42:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.904+0000 7f2007fff640 1 -- 172.21.15.45:0/3810888000 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018009510 con 0x7f202015c4d0 2024-05-08T05:42:45.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.904+0000 7f20261f0640 1 -- 172.21.15.45:0/3810888000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20200cc740 con 0x7f202015c4d0 2024-05-08T05:42:45.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.905+0000 7f20261f0640 1 -- 172.21.15.45:0/3810888000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20200693f0 con 0x7f202015c4d0 2024-05-08T05:42:45.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.906+0000 7f2007fff640 1 -- 172.21.15.45:0/3810888000 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2018007bf0 con 0x7f202015c4d0 2024-05-08T05:42:45.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.906+0000 7f2007fff640 1 --2- 172.21.15.45:0/3810888000 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f200c0762d0 0x7f200c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:45.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.907+0000 7f2007fff640 1 -- 172.21.15.45:0/3810888000 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f20180b84c0 con 0x7f202015c4d0 2024-05-08T05:42:45.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.907+0000 7f201f7fe640 1 --2- 172.21.15.45:0/3810888000 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f200c0762d0 0x7f200c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:45.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.907+0000 7f201f7fe640 1 --2- 172.21.15.45:0/3810888000 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f200c0762d0 0x7f200c078790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2010009aa0 tx=0x7f2010009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:45.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:45.911+0000 7f2007fff640 1 -- 172.21.15.45:0/3810888000 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2018081bf0 con 0x7f202015c4d0 2024-05-08T05:42:46.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.104+0000 7f20261f0640 1 -- 172.21.15.45:0/3810888000 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f202006e760 con 0x7f202015c4d0 2024-05-08T05:42:46.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.105+0000 7f2007fff640 1 -- 172.21.15.45:0/3810888000 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16894 (secure 0 0 0) 0x7f20180814f0 con 0x7f202015c4d0 2024-05-08T05:42:46.105 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:46.106 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":36,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:42:21.508281+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:42:17.918020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-08T05:42:18.816243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:42:46.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.107+0000 7f20261f0640 1 -- 172.21.15.45:0/3810888000 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f200c0762d0 msgr2=0x7f200c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.107+0000 7f20261f0640 1 --2- 172.21.15.45:0/3810888000 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f200c0762d0 0x7f200c078790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2010009aa0 tx=0x7f2010009190 comp rx=0 tx=0).stop 2024-05-08T05:42:46.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.107+0000 7f20261f0640 1 -- 172.21.15.45:0/3810888000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f202015c4d0 msgr2=0x7f20200c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.107+0000 7f20261f0640 1 --2- 172.21.15.45:0/3810888000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f202015c4d0 0x7f20200c8d50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2018009f40 tx=0x7f2018004a40 comp rx=0 tx=0).stop 2024-05-08T05:42:46.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.107+0000 7f20261f0640 1 -- 172.21.15.45:0/3810888000 shutdown_connections 2024-05-08T05:42:46.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.107+0000 7f20261f0640 1 --2- 172.21.15.45:0/3810888000 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f200c0762d0 0x7f200c078790 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.107+0000 7f20261f0640 1 --2- 172.21.15.45:0/3810888000 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f202015a570 0x7f202015bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.107+0000 7f20261f0640 1 --2- 172.21.15.45:0/3810888000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f202015c4d0 0x7f20200c8d50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.107+0000 7f20261f0640 1 --2- 172.21.15.45:0/3810888000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20200d1eb0 0x7f202015f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.107+0000 7f20261f0640 1 -- 172.21.15.45:0/3810888000 >> 172.21.15.45:0/3810888000 conn(0x7f202008eac0 msgr2=0x7f202006d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:46.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.107+0000 7f20261f0640 1 -- 172.21.15.45:0/3810888000 shutdown_connections 2024-05-08T05:42:46.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.107+0000 7f20261f0640 1 -- 172.21.15.45:0/3810888000 wait complete. 2024-05-08T05:42:46.119 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T05:42:46.119 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:42:46.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.222+0000 7ff9c6615640 1 Processor -- start 2024-05-08T05:42:46.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.222+0000 7ff9c6615640 1 -- start start 2024-05-08T05:42:46.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.223+0000 7ff9c6615640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff9c006e0f0 0x7ff9c015c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.223+0000 7ff9c6615640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9c0072840 0x7ff9c00d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.223+0000 7ff9c6615640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff9c006e4d0 0x7ff9c00b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.223+0000 7ff9c6615640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff9c00d14b0 con 0x7ff9c006e0f0 2024-05-08T05:42:46.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.223+0000 7ff9c6615640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff9c015c920 con 0x7ff9c006e4d0 2024-05-08T05:42:46.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.223+0000 7ff9c6615640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff9c015cc00 con 0x7ff9c0072840 2024-05-08T05:42:46.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.223+0000 7ff9bffff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff9c006e0f0 0x7ff9c015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.223+0000 7ff9bffff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff9c006e0f0 0x7ff9c015c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41280/0 (socket says 172.21.15.45:41280) 2024-05-08T05:42:46.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.223+0000 7ff9bffff640 1 -- 172.21.15.45:0/1577652224 learned_addr learned my addr 172.21.15.45:0/1577652224 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:46.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.224+0000 7ff9bffff640 1 -- 172.21.15.45:0/1577652224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9c0072840 msgr2=0x7ff9c00d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:46.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.224+0000 7ff9bf7fe640 1 --2- 172.21.15.45:0/1577652224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9c0072840 0x7ff9c00d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.224+0000 7ff9c4bb2640 1 --2- 172.21.15.45:0/1577652224 >> v2:172.21.15.63:3300/0 conn(0x7ff9c006e4d0 0x7ff9c00b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.224+0000 7ff9bffff640 1 --2- 172.21.15.45:0/1577652224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9c0072840 0x7ff9c00d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.224+0000 7ff9bffff640 1 -- 172.21.15.45:0/1577652224 >> v2:172.21.15.63:3300/0 conn(0x7ff9c006e4d0 msgr2=0x7ff9c00b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:46.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.224+0000 7ff9bffff640 1 --2- 172.21.15.45:0/1577652224 >> v2:172.21.15.63:3300/0 conn(0x7ff9c006e4d0 0x7ff9c00b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.224+0000 7ff9bffff640 1 -- 172.21.15.45:0/1577652224 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c015cee0 con 0x7ff9c006e0f0 2024-05-08T05:42:46.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.224+0000 7ff9bf7fe640 1 --2- 172.21.15.45:0/1577652224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9c0072840 0x7ff9c00d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:46.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.224+0000 7ff9c4bb2640 1 --2- 172.21.15.45:0/1577652224 >> v2:172.21.15.63:3300/0 conn(0x7ff9c006e4d0 0x7ff9c00b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:46.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.224+0000 7ff9bffff640 1 --2- 172.21.15.45:0/1577652224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c006e0f0 0x7ff9c015c340 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0070580 tx=0x7ff9b002c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb6e8c1c3fcfd73 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:46.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.225+0000 7ff9beffd640 1 -- 172.21.15.45:0/1577652224 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0032790 con 0x7ff9c006e0f0 2024-05-08T05:42:46.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.225+0000 7ff9beffd640 1 -- 172.21.15.45:0/1577652224 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9b0038070 con 0x7ff9c006e0f0 2024-05-08T05:42:46.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.225+0000 7ff9beffd640 1 -- 172.21.15.45:0/1577652224 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0032b50 con 0x7ff9c006e0f0 2024-05-08T05:42:46.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.225+0000 7ff9c6615640 1 -- 172.21.15.45:0/1577652224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c006e0f0 msgr2=0x7ff9c015c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.225+0000 7ff9c6615640 1 --2- 172.21.15.45:0/1577652224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c006e0f0 0x7ff9c015c340 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0070580 tx=0x7ff9b002c890 comp rx=0 tx=0).stop 2024-05-08T05:42:46.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.225+0000 7ff9c6615640 1 -- 172.21.15.45:0/1577652224 shutdown_connections 2024-05-08T05:42:46.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.225+0000 7ff9c6615640 1 --2- 172.21.15.45:0/1577652224 >> v2:172.21.15.63:3300/0 conn(0x7ff9c006e4d0 0x7ff9c00b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.225+0000 7ff9c6615640 1 --2- 172.21.15.45:0/1577652224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9c0072840 0x7ff9c00d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.225+0000 7ff9c6615640 1 --2- 172.21.15.45:0/1577652224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c006e0f0 0x7ff9c015c340 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.225+0000 7ff9c6615640 1 -- 172.21.15.45:0/1577652224 >> 172.21.15.45:0/1577652224 conn(0x7ff9c006c3a0 msgr2=0x7ff9c006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:46.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.225+0000 7ff9c6615640 1 -- 172.21.15.45:0/1577652224 shutdown_connections 2024-05-08T05:42:46.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.225+0000 7ff9c6615640 1 -- 172.21.15.45:0/1577652224 wait complete. 2024-05-08T05:42:46.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.226+0000 7ff9c6615640 1 Processor -- start 2024-05-08T05:42:46.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.227+0000 7ff9c6615640 1 -- start start 2024-05-08T05:42:46.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.227+0000 7ff9c6615640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c006e0f0 0x7ff9c00b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.227+0000 7ff9c6615640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9c006e4d0 0x7ff9c00cf340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.227+0000 7ff9c6615640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9c0072840 0x7ff9c014a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.227+0000 7ff9c6615640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c00ae650 con 0x7ff9c006e0f0 2024-05-08T05:42:46.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.227+0000 7ff9c6615640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c006e980 con 0x7ff9c006e4d0 2024-05-08T05:42:46.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.227+0000 7ff9c6615640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff9c00ae7d0 con 0x7ff9c0072840 2024-05-08T05:42:46.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.227+0000 7ff9bffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c006e0f0 0x7ff9c00b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.228+0000 7ff9bffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c006e0f0 0x7ff9c00b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41282/0 (socket says 172.21.15.45:41282) 2024-05-08T05:42:46.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.228+0000 7ff9bffff640 1 -- 172.21.15.45:0/3111180306 learned_addr learned my addr 172.21.15.45:0/3111180306 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:46.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.228+0000 7ff9bf7fe640 1 --2- 172.21.15.45:0/3111180306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9c006e4d0 0x7ff9c00cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.228+0000 7ff9c4bb2640 1 --2- 172.21.15.45:0/3111180306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9c0072840 0x7ff9c014a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.228+0000 7ff9bffff640 1 -- 172.21.15.45:0/3111180306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9c0072840 msgr2=0x7ff9c014a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.228+0000 7ff9bffff640 1 --2- 172.21.15.45:0/3111180306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9c0072840 0x7ff9c014a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.228+0000 7ff9bffff640 1 -- 172.21.15.45:0/3111180306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9c006e4d0 msgr2=0x7ff9c00cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.228+0000 7ff9bffff640 1 --2- 172.21.15.45:0/3111180306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9c006e4d0 0x7ff9c00cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.228+0000 7ff9bffff640 1 -- 172.21.15.45:0/3111180306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c00b0ed0 con 0x7ff9c006e0f0 2024-05-08T05:42:46.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.228+0000 7ff9bffff640 1 --2- 172.21.15.45:0/3111180306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c006e0f0 0x7ff9c00b2cd0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff9b002c8d0 tx=0x7ff9b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:46.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.228+0000 7ff9bcff9640 1 -- 172.21.15.45:0/3111180306 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0043070 con 0x7ff9c006e0f0 2024-05-08T05:42:46.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.228+0000 7ff9bcff9640 1 -- 172.21.15.45:0/3111180306 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9b003c850 con 0x7ff9c006e0f0 2024-05-08T05:42:46.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.228+0000 7ff9c6615640 1 -- 172.21.15.45:0/3111180306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c00b11b0 con 0x7ff9c006e0f0 2024-05-08T05:42:46.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.228+0000 7ff9bcff9640 1 -- 172.21.15.45:0/3111180306 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b003cc40 con 0x7ff9c006e0f0 2024-05-08T05:42:46.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.228+0000 7ff9c6615640 1 -- 172.21.15.45:0/3111180306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c00b1770 con 0x7ff9c006e0f0 2024-05-08T05:42:46.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.229+0000 7ff9c6615640 1 -- 172.21.15.45:0/3111180306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c00693f0 con 0x7ff9c006e0f0 2024-05-08T05:42:46.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.230+0000 7ff9bcff9640 1 -- 172.21.15.45:0/3111180306 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff9b0046070 con 0x7ff9c006e0f0 2024-05-08T05:42:46.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.230+0000 7ff9bcff9640 1 --2- 172.21.15.45:0/3111180306 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9ac0762d0 0x7ff9ac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.230+0000 7ff9bcff9640 1 -- 172.21.15.45:0/3111180306 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7ff9b00b7d40 con 0x7ff9c006e0f0 2024-05-08T05:42:46.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.230+0000 7ff9bf7fe640 1 --2- 172.21.15.45:0/3111180306 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9ac0762d0 0x7ff9ac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.231+0000 7ff9bf7fe640 1 --2- 172.21.15.45:0/3111180306 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9ac0762d0 0x7ff9ac078790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00b2690 tx=0x7ff9b4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:46.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.235+0000 7ff9bcff9640 1 -- 172.21.15.45:0/3111180306 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff9b0080f50 con 0x7ff9c006e0f0 2024-05-08T05:42:46.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.465+0000 7ff9c6615640 1 -- 172.21.15.45:0/3111180306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9c01563f0 con 0x7ff9c006e0f0 2024-05-08T05:42:46.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.465+0000 7ff9bcff9640 1 -- 172.21.15.45:0/3111180306 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5172 (secure 0 0 0) 0x7ff9b0085250 con 0x7ff9c006e0f0 2024-05-08T05:42:46.466 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:46.466 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3295938044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3295938044},{"type":"v1","addr":"172.21.15.63:6836","nonce":3295938044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4024530715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4024530715},{"type":"v1","addr":"172.21.15.63:6837","nonce":4024530715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/97258541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":97258541},{"type":"v1","addr":"172.21.15.45:6834","nonce":97258541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:42:19.860331+0000","modified":"2024-05-08T05:42:22.891006+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4884":{"gid":4884,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/173436628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":173436628},{"type":"v1","addr":"172.21.15.45:6835","nonce":173436628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4884,"qdb_cluster":[4884]},"id":2}]} 2024-05-08T05:42:46.466 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2024-05-08T05:42:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.467+0000 7ff9c6615640 1 -- 172.21.15.45:0/3111180306 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9ac0762d0 msgr2=0x7ff9ac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.467+0000 7ff9c6615640 1 --2- 172.21.15.45:0/3111180306 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9ac0762d0 0x7ff9ac078790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00b2690 tx=0x7ff9b4008040 comp rx=0 tx=0).stop 2024-05-08T05:42:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.467+0000 7ff9c6615640 1 -- 172.21.15.45:0/3111180306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c006e0f0 msgr2=0x7ff9c00b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.467+0000 7ff9c6615640 1 --2- 172.21.15.45:0/3111180306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c006e0f0 0x7ff9c00b2cd0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff9b002c8d0 tx=0x7ff9b0004730 comp rx=0 tx=0).stop 2024-05-08T05:42:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.467+0000 7ff9c6615640 1 -- 172.21.15.45:0/3111180306 shutdown_connections 2024-05-08T05:42:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.467+0000 7ff9c6615640 1 --2- 172.21.15.45:0/3111180306 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9ac0762d0 0x7ff9ac078790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.467+0000 7ff9c6615640 1 --2- 172.21.15.45:0/3111180306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9c0072840 0x7ff9c014a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.467+0000 7ff9c6615640 1 --2- 172.21.15.45:0/3111180306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9c006e4d0 0x7ff9c00cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.467+0000 7ff9c6615640 1 --2- 172.21.15.45:0/3111180306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9c006e0f0 0x7ff9c00b2cd0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.467+0000 7ff9c6615640 1 -- 172.21.15.45:0/3111180306 >> 172.21.15.45:0/3111180306 conn(0x7ff9c006c3a0 msgr2=0x7ff9c006f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.467+0000 7ff9c6615640 1 -- 172.21.15.45:0/3111180306 shutdown_connections 2024-05-08T05:42:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.467+0000 7ff9c6615640 1 -- 172.21.15.45:0/3111180306 wait complete. 2024-05-08T05:42:46.479 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:42:46.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.582+0000 7fe1020e1640 1 Processor -- start 2024-05-08T05:42:46.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.582+0000 7fe1020e1640 1 -- start start 2024-05-08T05:42:46.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.583+0000 7fe1020e1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0fc0d0ec0 0x7fe0fc0d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.583+0000 7fe1020e1640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0fc0d1760 con 0x7fe0fc0d3b00 2024-05-08T05:42:46.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.583+0000 7fe1020e1640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0fc0d1a40 con 0x7fe0fc15f380 2024-05-08T05:42:46.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.583+0000 7fe1020e1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe0fc0d30f0 con 0x7fe0fc0d0ec0 2024-05-08T05:42:46.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.583+0000 7fe0fbfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe0fc0d3b00 0x7fe0fc0b7e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52772/0 (socket says 172.21.15.45:52772) 2024-05-08T05:42:46.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.583+0000 7fe0fbfff640 1 -- 172.21.15.45:0/1717533539 learned_addr learned my addr 172.21.15.45:0/1717533539 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:46.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0faffd640 1 --2- 172.21.15.45:0/1717533539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0fc0d0ec0 0x7fe0fc0d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fa7fc640 1 -- 172.21.15.45:0/1717533539 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4286186753 0 0) 0x7fe0fc0d1760 con 0x7fe0fc0d3b00 2024-05-08T05:42:46.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fa7fc640 1 -- 172.21.15.45:0/1717533539 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0e4003520 con 0x7fe0fc0d3b00 2024-05-08T05:42:46.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fa7fc640 1 -- 172.21.15.45:0/1717533539 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1902090387 0 0) 0x7fe0e4003520 con 0x7fe0fc0d3b00 2024-05-08T05:42:46.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fa7fc640 1 -- 172.21.15.45:0/1717533539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0fc0d0ec0 msgr2=0x7fe0fc0d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:46.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fa7fc640 1 --2- 172.21.15.45:0/1717533539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0fc0d0ec0 0x7fe0fc0d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fa7fc640 1 -- 172.21.15.45:0/1717533539 >> v1:172.21.15.63:6789/0 conn(0x7fe0fc15f380 legacy=0x7fe0fc0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fa7fc640 1 -- 172.21.15.45:0/1717533539 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0fc0d1d20 con 0x7fe0fc0d3b00 2024-05-08T05:42:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fa7fc640 1 -- 172.21.15.45:0/1717533539 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fe0f4001a70 con 0x7fe0fc0d3b00 2024-05-08T05:42:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fa7fc640 1 -- 172.21.15.45:0/1717533539 >> v1:172.21.15.45:6789/0 conn(0x7fe0fc0d3b00 legacy=0x7fe0fc0b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fa7fc640 1 --2- 172.21.15.45:0/1717533539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0e4003a80 0x7fe0e4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fa7fc640 1 --2- 172.21.15.45:0/1717533539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0fc0d0ec0 0x7fe0e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fa7fc640 1 --2- 172.21.15.45:0/1717533539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e4004d40 0x7fe0e40051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fa7fc640 1 -- 172.21.15.45:0/1717533539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc0d30f0 con 0x7fe0e4004d40 2024-05-08T05:42:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fa7fc640 1 -- 172.21.15.45:0/1717533539 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e400bb50 con 0x7fe0e4003a80 2024-05-08T05:42:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fa7fc640 1 -- 172.21.15.45:0/1717533539 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e400bd50 con 0x7fe0fc0d0ec0 2024-05-08T05:42:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fbfff640 1 --2- 172.21.15.45:0/1717533539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e4004d40 0x7fe0e40051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fb7fe640 1 --2- 172.21.15.45:0/1717533539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0e4003a80 0x7fe0e4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0faffd640 1 --2- 172.21.15.45:0/1717533539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0fc0d0ec0 0x7fe0e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fbfff640 1 -- 172.21.15.45:0/1717533539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0fc0d0ec0 msgr2=0x7fe0e4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fbfff640 1 --2- 172.21.15.45:0/1717533539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0fc0d0ec0 0x7fe0e4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fbfff640 1 -- 172.21.15.45:0/1717533539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0e4003a80 msgr2=0x7fe0e4003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:46.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fbfff640 1 --2- 172.21.15.45:0/1717533539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0e4003a80 0x7fe0e4003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.584+0000 7fe0fbfff640 1 -- 172.21.15.45:0/1717533539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e4003520 con 0x7fe0e4004d40 2024-05-08T05:42:46.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.585+0000 7fe0fbfff640 1 --2- 172.21.15.45:0/1717533539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e4004d40 0x7fe0e40051a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400a830 tx=0x7fe0e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736c92c46c7eb8f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:46.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.585+0000 7fe0fa7fc640 1 -- 172.21.15.45:0/1717533539 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f400a450 con 0x7fe0e4004d40 2024-05-08T05:42:46.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.585+0000 7fe0fa7fc640 1 -- 172.21.15.45:0/1717533539 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0f402f070 con 0x7fe0e4004d40 2024-05-08T05:42:46.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.585+0000 7fe0fa7fc640 1 -- 172.21.15.45:0/1717533539 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f402bc00 con 0x7fe0e4004d40 2024-05-08T05:42:46.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.586+0000 7fe1020e1640 1 -- 172.21.15.45:0/1717533539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e4004d40 msgr2=0x7fe0e40051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.586+0000 7fe1020e1640 1 --2- 172.21.15.45:0/1717533539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0e4004d40 0x7fe0e40051a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400a830 tx=0x7fe0e4005830 comp rx=0 tx=0).stop 2024-05-08T05:42:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.586+0000 7fe0fbfff640 1 -- 172.21.15.45:0/1717533539 reap_dead start 2024-05-08T05:42:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.586+0000 7fe1020e1640 1 -- 172.21.15.45:0/1717533539 shutdown_connections 2024-05-08T05:42:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.586+0000 7fe1020e1640 1 -- 172.21.15.45:0/1717533539 >> 172.21.15.45:0/1717533539 conn(0x7fe0fc0d4fe0 msgr2=0x7fe0fc0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.586+0000 7fe1020e1640 1 -- 172.21.15.45:0/1717533539 shutdown_connections 2024-05-08T05:42:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.586+0000 7fe1020e1640 1 -- 172.21.15.45:0/1717533539 wait complete. 2024-05-08T05:42:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.587+0000 7fe1020e1640 1 Processor -- start 2024-05-08T05:42:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.587+0000 7fe1020e1640 1 -- start start 2024-05-08T05:42:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.588+0000 7fe1020e1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0fc1542d0 0x7fe0fc0b9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.588+0000 7fe1020e1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0fc1546b0 0x7fe0fc0b99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.588+0000 7fe1020e1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0fc0a0c80 0x7fe0fc0a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.588+0000 7fe1020e1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc15f760 con 0x7fe0fc0a0c80 2024-05-08T05:42:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.588+0000 7fe1020e1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc0d2370 con 0x7fe0fc1542d0 2024-05-08T05:42:46.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.588+0000 7fe1020e1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe0fc0d2800 con 0x7fe0fc1546b0 2024-05-08T05:42:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.588+0000 7fe0fbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0fc0a0c80 0x7fe0fc0a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.588+0000 7fe0fbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0fc0a0c80 0x7fe0fc0a1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41300/0 (socket says 172.21.15.45:41300) 2024-05-08T05:42:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.588+0000 7fe0fbfff640 1 -- 172.21.15.45:0/1729179876 learned_addr learned my addr 172.21.15.45:0/1729179876 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.588+0000 7fe0fb7fe640 1 --2- 172.21.15.45:0/1729179876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0fc1542d0 0x7fe0fc0b9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.588+0000 7fe0fbfff640 1 -- 172.21.15.45:0/1729179876 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0fc1546b0 msgr2=0x7fe0fc0b99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.588+0000 7fe0fbfff640 1 --2- 172.21.15.45:0/1729179876 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0fc1546b0 0x7fe0fc0b99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.588+0000 7fe0fbfff640 1 -- 172.21.15.45:0/1729179876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0fc1542d0 msgr2=0x7fe0fc0b9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.588+0000 7fe0fbfff640 1 --2- 172.21.15.45:0/1729179876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0fc1542d0 0x7fe0fc0b9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.589+0000 7fe0fbfff640 1 -- 172.21.15.45:0/1729179876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0fc0a1a90 con 0x7fe0fc0a0c80 2024-05-08T05:42:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.589+0000 7fe0fbfff640 1 --2- 172.21.15.45:0/1729179876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0fc0a0c80 0x7fe0fc0a1130 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400a830 tx=0x7fe0fc0cfbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.589+0000 7fe0e3fff640 1 -- 172.21.15.45:0/1729179876 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4039070 con 0x7fe0fc0a0c80 2024-05-08T05:42:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.589+0000 7fe0e3fff640 1 -- 172.21.15.45:0/1729179876 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0f4034470 con 0x7fe0fc0a0c80 2024-05-08T05:42:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.589+0000 7fe0e3fff640 1 -- 172.21.15.45:0/1729179876 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f400a450 con 0x7fe0fc0a0c80 2024-05-08T05:42:46.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.589+0000 7fe1020e1640 1 -- 172.21.15.45:0/1729179876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0fc0a1d70 con 0x7fe0fc0a0c80 2024-05-08T05:42:46.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.589+0000 7fe1020e1640 1 -- 172.21.15.45:0/1729179876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0fc0a2330 con 0x7fe0fc0a0c80 2024-05-08T05:42:46.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.590+0000 7fe1020e1640 1 -- 172.21.15.45:0/1729179876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0fc0693f0 con 0x7fe0fc0a0c80 2024-05-08T05:42:46.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.590+0000 7fe0e3fff640 1 -- 172.21.15.45:0/1729179876 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe0f403d030 con 0x7fe0fc0a0c80 2024-05-08T05:42:46.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.591+0000 7fe0e3fff640 1 --2- 172.21.15.45:0/1729179876 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe0e8076300 0x7fe0e80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.591+0000 7fe0e3fff640 1 -- 172.21.15.45:0/1729179876 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7fe0f40b6dc0 con 0x7fe0fc0a0c80 2024-05-08T05:42:46.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.592+0000 7fe0fb7fe640 1 --2- 172.21.15.45:0/1729179876 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe0e8076300 0x7fe0e80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.592+0000 7fe0fb7fe640 1 --2- 172.21.15.45:0/1729179876 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe0e8076300 0x7fe0e80787c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0055d0 tx=0x7fe0ec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:46.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.596+0000 7fe0e3fff640 1 -- 172.21.15.45:0/1729179876 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe0f4080640 con 0x7fe0fc0a0c80 2024-05-08T05:42:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.825+0000 7fe1020e1640 1 -- 172.21.15.45:0/1729179876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0fc08cfd0 con 0x7fe0fc0a0c80 2024-05-08T05:42:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.825+0000 7fe0e3fff640 1 -- 172.21.15.45:0/1729179876 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5172 (secure 0 0 0) 0x7fe0f407ff40 con 0x7fe0fc0a0c80 2024-05-08T05:42:46.826 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:46.826 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3295938044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3295938044},{"type":"v1","addr":"172.21.15.63:6836","nonce":3295938044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4024530715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4024530715},{"type":"v1","addr":"172.21.15.63:6837","nonce":4024530715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/97258541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":97258541},{"type":"v1","addr":"172.21.15.45:6834","nonce":97258541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:42:19.860331+0000","modified":"2024-05-08T05:42:22.891006+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4884":{"gid":4884,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/173436628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":173436628},{"type":"v1","addr":"172.21.15.45:6835","nonce":173436628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4884,"qdb_cluster":[4884]},"id":2}]} 2024-05-08T05:42:46.826 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 17 2024-05-08T05:42:46.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.827+0000 7fe1020e1640 1 -- 172.21.15.45:0/1729179876 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe0e8076300 msgr2=0x7fe0e80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.827+0000 7fe1020e1640 1 --2- 172.21.15.45:0/1729179876 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe0e8076300 0x7fe0e80787c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0055d0 tx=0x7fe0ec009040 comp rx=0 tx=0).stop 2024-05-08T05:42:46.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.827+0000 7fe1020e1640 1 -- 172.21.15.45:0/1729179876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0fc0a0c80 msgr2=0x7fe0fc0a1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.827+0000 7fe1020e1640 1 --2- 172.21.15.45:0/1729179876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0fc0a0c80 0x7fe0fc0a1130 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400a830 tx=0x7fe0fc0cfbf0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.827+0000 7fe1020e1640 1 -- 172.21.15.45:0/1729179876 shutdown_connections 2024-05-08T05:42:46.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.827+0000 7fe1020e1640 1 --2- 172.21.15.45:0/1729179876 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe0e8076300 0x7fe0e80787c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.827+0000 7fe1020e1640 1 --2- 172.21.15.45:0/1729179876 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0fc1546b0 0x7fe0fc0b99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.827+0000 7fe1020e1640 1 --2- 172.21.15.45:0/1729179876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe0fc0a0c80 0x7fe0fc0a1130 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.827+0000 7fe1020e1640 1 --2- 172.21.15.45:0/1729179876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0fc1542d0 0x7fe0fc0b9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.827+0000 7fe1020e1640 1 -- 172.21.15.45:0/1729179876 >> 172.21.15.45:0/1729179876 conn(0x7fe0fc0d4fe0 msgr2=0x7fe0fc06ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:46.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.827+0000 7fe1020e1640 1 -- 172.21.15.45:0/1729179876 shutdown_connections 2024-05-08T05:42:46.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.827+0000 7fe1020e1640 1 -- 172.21.15.45:0/1729179876 wait complete. 2024-05-08T05:42:46.839 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:42:46.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.943+0000 7fa9a450a640 1 Processor -- start 2024-05-08T05:42:46.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.943+0000 7fa9a450a640 1 -- start start 2024-05-08T05:42:46.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.944+0000 7fa9a450a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa99c0d1970 0x7fa99c0d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.944+0000 7fa9a450a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa99c0d4cc0 0x7fa99c0d50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.944+0000 7fa9a450a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa99c1630b0 con 0x7fa99c15f420 2024-05-08T05:42:46.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.944+0000 7fa9a450a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa99c0d3e70 con 0x7fa99c0d1970 2024-05-08T05:42:46.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.944+0000 7fa9a450a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa99c0d2550 con 0x7fa99c0d4cc0 2024-05-08T05:42:46.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.944+0000 7fa9a1aa5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa99c0d1970 0x7fa99c0d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.944+0000 7fa9a1aa5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa99c0d1970 0x7fa99c0d1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41306/0 (socket says 172.21.15.45:41306) 2024-05-08T05:42:46.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.944+0000 7fa9a1aa5640 1 -- 172.21.15.45:0/178317061 learned_addr learned my addr 172.21.15.45:0/178317061 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:46.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.944+0000 7fa9a2aa7640 1 --2- 172.21.15.45:0/178317061 >> v2:172.21.15.63:3300/0 conn(0x7fa99c0d4cc0 0x7fa99c0d50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.944+0000 7fa9a1aa5640 1 -- 172.21.15.45:0/178317061 >> v2:172.21.15.63:3300/0 conn(0x7fa99c0d4cc0 msgr2=0x7fa99c0d50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:46.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.944+0000 7fa9a1aa5640 1 --2- 172.21.15.45:0/178317061 >> v2:172.21.15.63:3300/0 conn(0x7fa99c0d4cc0 0x7fa99c0d50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.944+0000 7fa9a1aa5640 1 -- 172.21.15.45:0/178317061 >> v1:172.21.15.63:6789/0 conn(0x7fa99c15f420 legacy=0x7fa99c0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.944+0000 7fa9a1aa5640 1 -- 172.21.15.45:0/178317061 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99c0d2c80 con 0x7fa99c0d1970 2024-05-08T05:42:46.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.944+0000 7fa9a2aa7640 1 --2- 172.21.15.45:0/178317061 >> v2:172.21.15.63:3300/0 conn(0x7fa99c0d4cc0 0x7fa99c0d50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:46.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.945+0000 7fa9a1aa5640 1 --2- 172.21.15.45:0/178317061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c0d1970 0x7fa99c0d1e20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0d17a0 tx=0x7fa98c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a3239a5c63d633 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:46.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.945+0000 7fa9a12a4640 1 -- 172.21.15.45:0/178317061 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c037070 con 0x7fa99c0d1970 2024-05-08T05:42:46.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.945+0000 7fa9a12a4640 1 -- 172.21.15.45:0/178317061 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa98c032440 con 0x7fa99c0d1970 2024-05-08T05:42:46.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.945+0000 7fa9a12a4640 1 -- 172.21.15.45:0/178317061 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c032810 con 0x7fa99c0d1970 2024-05-08T05:42:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.946+0000 7fa9a450a640 1 -- 172.21.15.45:0/178317061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c0d1970 msgr2=0x7fa99c0d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.946+0000 7fa9a450a640 1 --2- 172.21.15.45:0/178317061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c0d1970 0x7fa99c0d1e20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0d17a0 tx=0x7fa98c02a270 comp rx=0 tx=0).stop 2024-05-08T05:42:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.946+0000 7fa9a450a640 1 -- 172.21.15.45:0/178317061 shutdown_connections 2024-05-08T05:42:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.946+0000 7fa9a450a640 1 --2- 172.21.15.45:0/178317061 >> v2:172.21.15.63:3300/0 conn(0x7fa99c0d4cc0 0x7fa99c0d50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.946+0000 7fa9a450a640 1 --2- 172.21.15.45:0/178317061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c0d1970 0x7fa99c0d1e20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.946+0000 7fa9a450a640 1 -- 172.21.15.45:0/178317061 >> 172.21.15.45:0/178317061 conn(0x7fa99c08eac0 msgr2=0x7fa99c09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.946+0000 7fa9a450a640 1 -- 172.21.15.45:0/178317061 shutdown_connections 2024-05-08T05:42:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.946+0000 7fa9a450a640 1 -- 172.21.15.45:0/178317061 wait complete. 2024-05-08T05:42:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.947+0000 7fa9a450a640 1 Processor -- start 2024-05-08T05:42:46.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.947+0000 7fa9a450a640 1 -- start start 2024-05-08T05:42:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa9a450a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa99c0d1970 0x7fa99c1e8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa9a450a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa99c0d4cc0 0x7fa99c160980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa9a450a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c0cc0e0 0x7fa99c155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa9a450a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c0d2830 con 0x7fa99c0cc0e0 2024-05-08T05:42:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa9a450a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c0d3240 con 0x7fa99c0d4cc0 2024-05-08T05:42:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa9a450a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa99c0d5620 con 0x7fa99c0d1970 2024-05-08T05:42:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa9a2aa7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c0cc0e0 0x7fa99c155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa9a2aa7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c0cc0e0 0x7fa99c155520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41310/0 (socket says 172.21.15.45:41310) 2024-05-08T05:42:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa9a2aa7640 1 -- 172.21.15.45:0/1757856576 learned_addr learned my addr 172.21.15.45:0/1757856576 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa9a1aa5640 1 --2- 172.21.15.45:0/1757856576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa99c0d4cc0 0x7fa99c160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa9a22a6640 1 --2- 172.21.15.45:0/1757856576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa99c0d1970 0x7fa99c1e8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa9a2aa7640 1 -- 172.21.15.45:0/1757856576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa99c0d1970 msgr2=0x7fa99c1e8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa9a2aa7640 1 --2- 172.21.15.45:0/1757856576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa99c0d1970 0x7fa99c1e8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa9a2aa7640 1 -- 172.21.15.45:0/1757856576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa99c0d4cc0 msgr2=0x7fa99c160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa9a2aa7640 1 --2- 172.21.15.45:0/1757856576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa99c0d4cc0 0x7fa99c160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa9a2aa7640 1 -- 172.21.15.45:0/1757856576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99c1610f0 con 0x7fa99c0cc0e0 2024-05-08T05:42:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa9a22a6640 1 --2- 172.21.15.45:0/1757856576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa99c0d1970 0x7fa99c1e8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa9a2aa7640 1 --2- 172.21.15.45:0/1757856576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c0cc0e0 0x7fa99c155520 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa99400e7e0 tx=0x7fa99400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa992ffd640 1 -- 172.21.15.45:0/1757856576 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99400f040 con 0x7fa99c0cc0e0 2024-05-08T05:42:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa992ffd640 1 -- 172.21.15.45:0/1757856576 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa994004030 con 0x7fa99c0cc0e0 2024-05-08T05:42:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa9a450a640 1 -- 172.21.15.45:0/1757856576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99c0c9b50 con 0x7fa99c0cc0e0 2024-05-08T05:42:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa992ffd640 1 -- 172.21.15.45:0/1757856576 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9940043e0 con 0x7fa99c0cc0e0 2024-05-08T05:42:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.948+0000 7fa9a450a640 1 -- 172.21.15.45:0/1757856576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa99c0ca110 con 0x7fa99c0cc0e0 2024-05-08T05:42:46.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.949+0000 7fa9a450a640 1 -- 172.21.15.45:0/1757856576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa99c0693f0 con 0x7fa99c0cc0e0 2024-05-08T05:42:46.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.950+0000 7fa992ffd640 1 -- 172.21.15.45:0/1757856576 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa99400c6d0 con 0x7fa99c0cc0e0 2024-05-08T05:42:46.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.951+0000 7fa992ffd640 1 --2- 172.21.15.45:0/1757856576 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa9800762d0 0x7fa980078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.951+0000 7fa992ffd640 1 -- 172.21.15.45:0/1757856576 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7fa994097370 con 0x7fa99c0cc0e0 2024-05-08T05:42:46.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.951+0000 7fa9a22a6640 1 --2- 172.21.15.45:0/1757856576 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa9800762d0 0x7fa980078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.951+0000 7fa9a22a6640 1 --2- 172.21.15.45:0/1757856576 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa9800762d0 0x7fa980078790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa998001da0 tx=0x7fa998009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:46.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:46.955+0000 7fa992ffd640 1 -- 172.21.15.45:0/1757856576 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa994017020 con 0x7fa99c0cc0e0 2024-05-08T05:42:47.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.148+0000 7fa9a450a640 1 -- 172.21.15.45:0/1757856576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa99c0b4c50 con 0x7fa99c0cc0e0 2024-05-08T05:42:47.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.149+0000 7fa992ffd640 1 -- 172.21.15.45:0/1757856576 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16894 (secure 0 0 0) 0x7fa994060580 con 0x7fa99c0cc0e0 2024-05-08T05:42:47.149 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:47.149 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":36,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:42:21.508281+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:42:17.918020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-05-08T05:42:18.816243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:42:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.151+0000 7fa9a450a640 1 -- 172.21.15.45:0/1757856576 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa9800762d0 msgr2=0x7fa980078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.151+0000 7fa9a450a640 1 --2- 172.21.15.45:0/1757856576 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa9800762d0 0x7fa980078790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa998001da0 tx=0x7fa998009040 comp rx=0 tx=0).stop 2024-05-08T05:42:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.151+0000 7fa9a450a640 1 -- 172.21.15.45:0/1757856576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c0cc0e0 msgr2=0x7fa99c155520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:47.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.151+0000 7fa9a450a640 1 --2- 172.21.15.45:0/1757856576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c0cc0e0 0x7fa99c155520 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa99400e7e0 tx=0x7fa99400eca0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.151+0000 7fa9a450a640 1 -- 172.21.15.45:0/1757856576 shutdown_connections 2024-05-08T05:42:47.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.151+0000 7fa9a450a640 1 --2- 172.21.15.45:0/1757856576 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa9800762d0 0x7fa980078790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.151+0000 7fa9a450a640 1 --2- 172.21.15.45:0/1757856576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa99c0cc0e0 0x7fa99c155520 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.151+0000 7fa9a450a640 1 --2- 172.21.15.45:0/1757856576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa99c0d4cc0 0x7fa99c160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.151+0000 7fa9a450a640 1 --2- 172.21.15.45:0/1757856576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa99c0d1970 0x7fa99c1e8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.151+0000 7fa9a450a640 1 -- 172.21.15.45:0/1757856576 >> 172.21.15.45:0/1757856576 conn(0x7fa99c08eac0 msgr2=0x7fa99c0b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:47.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.151+0000 7fa9a450a640 1 -- 172.21.15.45:0/1757856576 shutdown_connections 2024-05-08T05:42:47.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.151+0000 7fa9a450a640 1 -- 172.21.15.45:0/1757856576 wait complete. 2024-05-08T05:42:47.162 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T05:42:47.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.267+0000 7f455efaf640 1 Processor -- start 2024-05-08T05:42:47.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.267+0000 7f455efaf640 1 -- start start 2024-05-08T05:42:47.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.268+0000 7f455efaf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f45581629a0 0x7f45580d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:47.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.268+0000 7f455efaf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45580d2dc0 0x7f45580d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:47.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.268+0000 7f455efaf640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45580d3330 con 0x7f45580c8ed0 2024-05-08T05:42:47.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.268+0000 7f455efaf640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f45580d2170 con 0x7f45581629a0 2024-05-08T05:42:47.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.268+0000 7f455efaf640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f45580d2450 con 0x7f45580d2dc0 2024-05-08T05:42:47.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.268+0000 7f455cd4b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f45581629a0 0x7f45580d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:47.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.268+0000 7f455d54c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f45580c8ed0 0x7f45580c9360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52784/0 (socket says 172.21.15.45:52784) 2024-05-08T05:42:47.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.268+0000 7f455d54c640 1 -- 172.21.15.45:0/1705256801 learned_addr learned my addr 172.21.15.45:0/1705256801 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:47.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.268+0000 7f4557fff640 1 --2- 172.21.15.45:0/1705256801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45580d2dc0 0x7f45580d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:47.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.269+0000 7f455cd4b640 1 -- 172.21.15.45:0/1705256801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45580d2dc0 msgr2=0x7f45580d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:47.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.269+0000 7f455cd4b640 1 --2- 172.21.15.45:0/1705256801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45580d2dc0 0x7f45580d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.269+0000 7f455cd4b640 1 -- 172.21.15.45:0/1705256801 >> v1:172.21.15.45:6789/0 conn(0x7f45580c8ed0 legacy=0x7f45580c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:47.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.269+0000 7f45577fe640 1 -- 172.21.15.45:0/1705256801 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1501697965 0 0) 0x7f45580d3330 con 0x7f45580c8ed0 2024-05-08T05:42:47.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.269+0000 7f455cd4b640 1 -- 172.21.15.45:0/1705256801 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45580d2730 con 0x7f45581629a0 2024-05-08T05:42:47.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.269+0000 7f4557fff640 1 --2- 172.21.15.45:0/1705256801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45580d2dc0 0x7f45580d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:47.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.269+0000 7f455cd4b640 1 --2- 172.21.15.45:0/1705256801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45581629a0 0x7f45580d35c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f45480097a0 tx=0x7f454802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5232d5bdc9b9d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:47.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.269+0000 7f45577fe640 1 -- 172.21.15.45:0/1705256801 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4548037070 con 0x7f45581629a0 2024-05-08T05:42:47.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.269+0000 7f45577fe640 1 -- 172.21.15.45:0/1705256801 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f454802adb0 con 0x7f45581629a0 2024-05-08T05:42:47.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.269+0000 7f45577fe640 1 -- 172.21.15.45:0/1705256801 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4548032660 con 0x7f45581629a0 2024-05-08T05:42:47.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.270+0000 7f455efaf640 1 -- 172.21.15.45:0/1705256801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45581629a0 msgr2=0x7f45580d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:47.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.270+0000 7f455efaf640 1 --2- 172.21.15.45:0/1705256801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45581629a0 0x7f45580d35c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f45480097a0 tx=0x7f454802a060 comp rx=0 tx=0).stop 2024-05-08T05:42:47.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.270+0000 7f455efaf640 1 -- 172.21.15.45:0/1705256801 shutdown_connections 2024-05-08T05:42:47.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.270+0000 7f455efaf640 1 --2- 172.21.15.45:0/1705256801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45580d2dc0 0x7f45580d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.270+0000 7f455efaf640 1 --2- 172.21.15.45:0/1705256801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45581629a0 0x7f45580d35c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.270+0000 7f455efaf640 1 -- 172.21.15.45:0/1705256801 >> 172.21.15.45:0/1705256801 conn(0x7f45580d4fe0 msgr2=0x7f45580d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:47.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.270+0000 7f455efaf640 1 -- 172.21.15.45:0/1705256801 shutdown_connections 2024-05-08T05:42:47.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.270+0000 7f455efaf640 1 -- 172.21.15.45:0/1705256801 wait complete. 2024-05-08T05:42:47.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.271+0000 7f455efaf640 1 Processor -- start 2024-05-08T05:42:47.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.271+0000 7f455efaf640 1 -- start start 2024-05-08T05:42:47.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.272+0000 7f455efaf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45580c8ed0 0x7f45581523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:47.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.272+0000 7f455efaf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45580d2dc0 0x7f45580cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:47.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.272+0000 7f455efaf640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45581629a0 0x7f45580cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:47.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.272+0000 7f455efaf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f45580cf440 con 0x7f45580d2dc0 2024-05-08T05:42:47.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.272+0000 7f455efaf640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f45580cf2c0 con 0x7f45581629a0 2024-05-08T05:42:47.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.272+0000 7f455efaf640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f45580d0cb0 con 0x7f45580c8ed0 2024-05-08T05:42:47.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.272+0000 7f4557fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45580d2dc0 0x7f45580cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:47.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.272+0000 7f4557fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45580d2dc0 0x7f45580cc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41340/0 (socket says 172.21.15.45:41340) 2024-05-08T05:42:47.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.272+0000 7f4557fff640 1 -- 172.21.15.45:0/2075332821 learned_addr learned my addr 172.21.15.45:0/2075332821 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:47.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.273+0000 7f455d54c640 1 --2- 172.21.15.45:0/2075332821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45581629a0 0x7f45580cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:47.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.273+0000 7f455cd4b640 1 --2- 172.21.15.45:0/2075332821 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45580c8ed0 0x7f45581523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:47.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.273+0000 7f4557fff640 1 -- 172.21.15.45:0/2075332821 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45580c8ed0 msgr2=0x7f45581523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:47.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.273+0000 7f4557fff640 1 --2- 172.21.15.45:0/2075332821 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45580c8ed0 0x7f45581523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.273+0000 7f4557fff640 1 -- 172.21.15.45:0/2075332821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45581629a0 msgr2=0x7f45580cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:47.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.273+0000 7f4557fff640 1 --2- 172.21.15.45:0/2075332821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45581629a0 0x7f45580cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.273+0000 7f4557fff640 1 -- 172.21.15.45:0/2075332821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45581f5330 con 0x7f45580d2dc0 2024-05-08T05:42:47.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.273+0000 7f455d54c640 1 --2- 172.21.15.45:0/2075332821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45581629a0 0x7f45580cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:47.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.273+0000 7f455cd4b640 1 --2- 172.21.15.45:0/2075332821 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45580c8ed0 0x7f45581523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:47.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.273+0000 7f4557fff640 1 --2- 172.21.15.45:0/2075332821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45580d2dc0 0x7f45580cc850 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f454c00c7f0 tx=0x7f454c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:47.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.273+0000 7f45557fa640 1 -- 172.21.15.45:0/2075332821 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c015070 con 0x7f45580d2dc0 2024-05-08T05:42:47.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.273+0000 7f45557fa640 1 -- 172.21.15.45:0/2075332821 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f454c007a30 con 0x7f45580d2dc0 2024-05-08T05:42:47.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.273+0000 7f455efaf640 1 -- 172.21.15.45:0/2075332821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45581f5610 con 0x7f45580d2dc0 2024-05-08T05:42:47.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.273+0000 7f45557fa640 1 -- 172.21.15.45:0/2075332821 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c007e00 con 0x7f45580d2dc0 2024-05-08T05:42:47.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.273+0000 7f455efaf640 1 -- 172.21.15.45:0/2075332821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45581f5bd0 con 0x7f45580d2dc0 2024-05-08T05:42:47.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.274+0000 7f455efaf640 1 -- 172.21.15.45:0/2075332821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45580693f0 con 0x7f45580d2dc0 2024-05-08T05:42:47.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.275+0000 7f45557fa640 1 -- 172.21.15.45:0/2075332821 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f454c018070 con 0x7f45580d2dc0 2024-05-08T05:42:47.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.275+0000 7f45557fa640 1 --2- 172.21.15.45:0/2075332821 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f45440762d0 0x7f4544078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:47.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.275+0000 7f45557fa640 1 -- 172.21.15.45:0/2075332821 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6101+0+0 (secure 0 0 0) 0x7f454c097740 con 0x7f45580d2dc0 2024-05-08T05:42:47.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.276+0000 7f455cd4b640 1 --2- 172.21.15.45:0/2075332821 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f45440762d0 0x7f4544078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:47.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.276+0000 7f455cd4b640 1 --2- 172.21.15.45:0/2075332821 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f45440762d0 0x7f4544078790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4548034000 tx=0x7f454802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:47.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.279+0000 7f45557fa640 1 -- 172.21.15.45:0/2075332821 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f454c060f10 con 0x7f45580d2dc0 2024-05-08T05:42:47.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.512+0000 7f455efaf640 1 -- 172.21.15.45:0/2075332821 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f45581f5e10 con 0x7f45580d2dc0 2024-05-08T05:42:47.513 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-05-08T05:42:47.513+0000 7f124de12640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T05:42:47.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.514+0000 7f45557fa640 1 -- 172.21.15.45:0/2075332821 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7f454c060810 con 0x7f45580d2dc0 2024-05-08T05:42:47.514 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T05:42:47.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.517+0000 7f455efaf640 1 -- 172.21.15.45:0/2075332821 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f45440762d0 msgr2=0x7f4544078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:47.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.517+0000 7f455efaf640 1 --2- 172.21.15.45:0/2075332821 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f45440762d0 0x7f4544078790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4548034000 tx=0x7f454802e000 comp rx=0 tx=0).stop 2024-05-08T05:42:47.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.517+0000 7f455efaf640 1 -- 172.21.15.45:0/2075332821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45580d2dc0 msgr2=0x7f45580cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:47.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.517+0000 7f455efaf640 1 --2- 172.21.15.45:0/2075332821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45580d2dc0 0x7f45580cc850 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f454c00c7f0 tx=0x7f454c00cbf0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.517+0000 7f455efaf640 1 -- 172.21.15.45:0/2075332821 shutdown_connections 2024-05-08T05:42:47.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.517+0000 7f455efaf640 1 --2- 172.21.15.45:0/2075332821 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f45440762d0 0x7f4544078790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.517+0000 7f455efaf640 1 --2- 172.21.15.45:0/2075332821 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45581629a0 0x7f45580cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.517+0000 7f455efaf640 1 --2- 172.21.15.45:0/2075332821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45580d2dc0 0x7f45580cc850 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.517+0000 7f455efaf640 1 --2- 172.21.15.45:0/2075332821 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45580c8ed0 0x7f45581523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.517+0000 7f455efaf640 1 -- 172.21.15.45:0/2075332821 >> 172.21.15.45:0/2075332821 conn(0x7f45580d4fe0 msgr2=0x7f455806cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:47.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.517+0000 7f455efaf640 1 -- 172.21.15.45:0/2075332821 shutdown_connections 2024-05-08T05:42:47.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.517+0000 7f455efaf640 1 -- 172.21.15.45:0/2075332821 wait complete. 2024-05-08T05:42:47.520 INFO:tasks.ceph.mds.c.smithi045.stderr: -6574> 2024-05-08T05:42:25.463+0000 7f37c7991640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:42:47.529 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T05:42:47.553 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-05-08T05:42:47.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.636+0000 7fe74d7f7640 1 Processor -- start 2024-05-08T05:42:47.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.636+0000 7fe74d7f7640 1 -- start start 2024-05-08T05:42:47.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.637+0000 7fe74d7f7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe74806c3a0 0x7fe7480b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:47.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.637+0000 7fe74d7f7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7480aedf0 0x7fe7480b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:47.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.637+0000 7fe74d7f7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe74815d180 0x7fe74806c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:47.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.637+0000 7fe74d7f7640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe74815e360 con 0x7fe74815d180 2024-05-08T05:42:47.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.637+0000 7fe74d7f7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe74809ff30 con 0x7fe74806c3a0 2024-05-08T05:42:47.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.637+0000 7fe74d7f7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe74806ccc0 con 0x7fe7480aedf0 2024-05-08T05:42:47.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.637+0000 7fe7477fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe74815d180 0x7fe74806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:47.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.637+0000 7fe7477fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe74815d180 0x7fe74806c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41376/0 (socket says 172.21.15.45:41376) 2024-05-08T05:42:47.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.637+0000 7fe7477fe640 1 -- 172.21.15.45:0/793337417 learned_addr learned my addr 172.21.15.45:0/793337417 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:47.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.637+0000 7fe73e7fc640 1 --2- 172.21.15.45:0/793337417 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7480aedf0 0x7fe7480b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:47.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.637+0000 7fe746ffd640 1 --2- 172.21.15.45:0/793337417 >> v2:172.21.15.63:3300/0 conn(0x7fe74806c3a0 0x7fe7480b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:47.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.637+0000 7fe7477fe640 1 -- 172.21.15.45:0/793337417 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7480aedf0 msgr2=0x7fe7480b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:47.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.637+0000 7fe7477fe640 1 --2- 172.21.15.45:0/793337417 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7480aedf0 0x7fe7480b6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.637+0000 7fe7477fe640 1 -- 172.21.15.45:0/793337417 >> v2:172.21.15.63:3300/0 conn(0x7fe74806c3a0 msgr2=0x7fe7480b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:47.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.637+0000 7fe7477fe640 1 --2- 172.21.15.45:0/793337417 >> v2:172.21.15.63:3300/0 conn(0x7fe74806c3a0 0x7fe7480b13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.637+0000 7fe7477fe640 1 -- 172.21.15.45:0/793337417 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74815d750 con 0x7fe74815d180 2024-05-08T05:42:47.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.637+0000 7fe746ffd640 1 --2- 172.21.15.45:0/793337417 >> v2:172.21.15.63:3300/0 conn(0x7fe74806c3a0 0x7fe7480b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:47.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.638+0000 7fe7477fe640 1 --2- 172.21.15.45:0/793337417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe74815d180 0x7fe74806c780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe7480b1950 tx=0x7fe74002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6207522d952cb673 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:47.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.638+0000 7fe7467fc640 1 -- 172.21.15.45:0/793337417 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe740036070 con 0x7fe74815d180 2024-05-08T05:42:47.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.638+0000 7fe7467fc640 1 -- 172.21.15.45:0/793337417 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe740031600 con 0x7fe74815d180 2024-05-08T05:42:47.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.638+0000 7fe7467fc640 1 -- 172.21.15.45:0/793337417 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7400319d0 con 0x7fe74815d180 2024-05-08T05:42:47.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.639+0000 7fe74d7f7640 1 -- 172.21.15.45:0/793337417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe74815d180 msgr2=0x7fe74806c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:47.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.639+0000 7fe74d7f7640 1 --2- 172.21.15.45:0/793337417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe74815d180 0x7fe74806c780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe7480b1950 tx=0x7fe74002a6a0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.639+0000 7fe74d7f7640 1 -- 172.21.15.45:0/793337417 shutdown_connections 2024-05-08T05:42:47.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.639+0000 7fe74d7f7640 1 --2- 172.21.15.45:0/793337417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe74815d180 0x7fe74806c780 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.639+0000 7fe74d7f7640 1 --2- 172.21.15.45:0/793337417 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7480aedf0 0x7fe7480b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.639+0000 7fe74d7f7640 1 --2- 172.21.15.45:0/793337417 >> v2:172.21.15.63:3300/0 conn(0x7fe74806c3a0 0x7fe7480b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.639+0000 7fe74d7f7640 1 -- 172.21.15.45:0/793337417 >> 172.21.15.45:0/793337417 conn(0x7fe7480b3b10 msgr2=0x7fe7480b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:47.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.639+0000 7fe74d7f7640 1 -- 172.21.15.45:0/793337417 shutdown_connections 2024-05-08T05:42:47.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.639+0000 7fe74d7f7640 1 -- 172.21.15.45:0/793337417 wait complete. 2024-05-08T05:42:47.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.640+0000 7fe74d7f7640 1 Processor -- start 2024-05-08T05:42:47.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.640+0000 7fe74d7f7640 1 -- start start 2024-05-08T05:42:47.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.641+0000 7fe74d7f7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe74806c3a0 0x7fe74809d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:47.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.641+0000 7fe74d7f7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7480aedf0 0x7fe7480c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:47.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.641+0000 7fe74d7f7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe74815d180 0x7fe7480c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:47.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.641+0000 7fe74d7f7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe74815dff0 con 0x7fe74806c3a0 2024-05-08T05:42:47.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.641+0000 7fe74d7f7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe74815dd10 con 0x7fe74815d180 2024-05-08T05:42:47.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.641+0000 7fe74d7f7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe7480b4fc0 con 0x7fe7480aedf0 2024-05-08T05:42:47.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.641+0000 7fe746ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe74806c3a0 0x7fe74809d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:47.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.641+0000 7fe746ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe74806c3a0 0x7fe74809d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41386/0 (socket says 172.21.15.45:41386) 2024-05-08T05:42:47.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.641+0000 7fe746ffd640 1 -- 172.21.15.45:0/2298773423 learned_addr learned my addr 172.21.15.45:0/2298773423 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:47.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.641+0000 7fe7477fe640 1 --2- 172.21.15.45:0/2298773423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe74815d180 0x7fe7480c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:47.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.641+0000 7fe73e7fc640 1 --2- 172.21.15.45:0/2298773423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7480aedf0 0x7fe7480c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:47.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.641+0000 7fe746ffd640 1 -- 172.21.15.45:0/2298773423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7480aedf0 msgr2=0x7fe7480c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:47.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.641+0000 7fe746ffd640 1 --2- 172.21.15.45:0/2298773423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7480aedf0 0x7fe7480c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.641+0000 7fe746ffd640 1 -- 172.21.15.45:0/2298773423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe74815d180 msgr2=0x7fe7480c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:47.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.641+0000 7fe746ffd640 1 --2- 172.21.15.45:0/2298773423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe74815d180 0x7fe7480c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:47.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.641+0000 7fe746ffd640 1 -- 172.21.15.45:0/2298773423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7480c7520 con 0x7fe74806c3a0 2024-05-08T05:42:47.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.641+0000 7fe7477fe640 1 --2- 172.21.15.45:0/2298773423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe74815d180 0x7fe7480c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:47.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.641+0000 7fe746ffd640 1 --2- 172.21.15.45:0/2298773423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe74806c3a0 0x7fe74809d6d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe73800c7c0 tx=0x7fe73800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:47.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.641+0000 7fe73ffff640 1 -- 172.21.15.45:0/2298773423 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe738015070 con 0x7fe74806c3a0 2024-05-08T05:42:47.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.642+0000 7fe73ffff640 1 -- 172.21.15.45:0/2298773423 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe738007a30 con 0x7fe74806c3a0 2024-05-08T05:42:47.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.642+0000 7fe74d7f7640 1 -- 172.21.15.45:0/2298773423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7480c7800 con 0x7fe74806c3a0 2024-05-08T05:42:47.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.642+0000 7fe73ffff640 1 -- 172.21.15.45:0/2298773423 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe738007e00 con 0x7fe74806c3a0 2024-05-08T05:42:47.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.642+0000 7fe74d7f7640 1 -- 172.21.15.45:0/2298773423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7480c7dc0 con 0x7fe74806c3a0 2024-05-08T05:42:47.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.643+0000 7fe74d7f7640 1 -- 172.21.15.45:0/2298773423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7480693f0 con 0x7fe74806c3a0 2024-05-08T05:42:47.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.643+0000 7fe73ffff640 1 -- 172.21.15.45:0/2298773423 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe738018070 con 0x7fe74806c3a0 2024-05-08T05:42:47.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.644+0000 7fe73ffff640 1 --2- 172.21.15.45:0/2298773423 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe7280762d0 0x7fe728078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:47.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.644+0000 7fe73ffff640 1 -- 172.21.15.45:0/2298773423 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6187+0+0 (secure 0 0 0) 0x7fe738097660 con 0x7fe74806c3a0 2024-05-08T05:42:47.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.644+0000 7fe73e7fc640 1 --2- 172.21.15.45:0/2298773423 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe7280762d0 0x7fe728078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:47.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.644+0000 7fe73e7fc640 1 --2- 172.21.15.45:0/2298773423 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe7280762d0 0x7fe728078790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe730006fd0 tx=0x7fe730008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:47.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.648+0000 7fe73ffff640 1 -- 172.21.15.45:0/2298773423 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe738060e70 con 0x7fe74806c3a0 2024-05-08T05:42:47.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:47.876+0000 7fe74d7f7640 1 -- 172.21.15.45:0/2298773423 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe748073e20 con 0x7fe74806c3a0 2024-05-08T05:42:48.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.516+0000 7fe73ffff640 1 -- 172.21.15.45:0/2298773423 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7fe738060770 con 0x7fe74806c3a0 2024-05-08T05:42:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.519+0000 7fe74d7f7640 1 -- 172.21.15.45:0/2298773423 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe7280762d0 msgr2=0x7fe728078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.519+0000 7fe74d7f7640 1 --2- 172.21.15.45:0/2298773423 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe7280762d0 0x7fe728078790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe730006fd0 tx=0x7fe730008040 comp rx=0 tx=0).stop 2024-05-08T05:42:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.519+0000 7fe74d7f7640 1 -- 172.21.15.45:0/2298773423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe74806c3a0 msgr2=0x7fe74809d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.519+0000 7fe74d7f7640 1 --2- 172.21.15.45:0/2298773423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe74806c3a0 0x7fe74809d6d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe73800c7c0 tx=0x7fe73800cc80 comp rx=0 tx=0).stop 2024-05-08T05:42:48.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.519+0000 7fe74d7f7640 1 -- 172.21.15.45:0/2298773423 shutdown_connections 2024-05-08T05:42:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.519+0000 7fe74d7f7640 1 --2- 172.21.15.45:0/2298773423 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe7280762d0 0x7fe728078790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.519+0000 7fe74d7f7640 1 --2- 172.21.15.45:0/2298773423 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7480aedf0 0x7fe7480c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.519+0000 7fe74d7f7640 1 --2- 172.21.15.45:0/2298773423 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe74815d180 0x7fe7480c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.519+0000 7fe74d7f7640 1 --2- 172.21.15.45:0/2298773423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe74806c3a0 0x7fe74809d6d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.519+0000 7fe74d7f7640 1 -- 172.21.15.45:0/2298773423 >> 172.21.15.45:0/2298773423 conn(0x7fe7480b3b10 msgr2=0x7fe7480b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.519+0000 7fe74d7f7640 1 -- 172.21.15.45:0/2298773423 shutdown_connections 2024-05-08T05:42:48.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.519+0000 7fe74d7f7640 1 -- 172.21.15.45:0/2298773423 wait complete. 2024-05-08T05:42:48.534 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T05:42:48.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.689+0000 7fddfcd04640 1 Processor -- start 2024-05-08T05:42:48.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.689+0000 7fddfcd04640 1 -- start start 2024-05-08T05:42:48.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.690+0000 7fddfcd04640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fddf80d3670 0x7fddf80b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:48.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.690+0000 7fddfcd04640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fddf80d3a50 0x7fddf80d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:48.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.690+0000 7fddfcd04640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddf80cfc00 con 0x7fddf80d0e80 2024-05-08T05:42:48.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.690+0000 7fddfcd04640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fddf80d2a10 con 0x7fddf80d3670 2024-05-08T05:42:48.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.690+0000 7fddfcd04640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fddf80d04b0 con 0x7fddf80d3a50 2024-05-08T05:42:48.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.690+0000 7fddf659c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fddf80d3670 0x7fddf80b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:48.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.690+0000 7fddf659c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fddf80d3670 0x7fddf80b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41392/0 (socket says 172.21.15.45:41392) 2024-05-08T05:42:48.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.690+0000 7fddf659c640 1 -- 172.21.15.45:0/1352591890 learned_addr learned my addr 172.21.15.45:0/1352591890 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:48.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.691+0000 7fddf5d9b640 1 --2- 172.21.15.45:0/1352591890 >> v2:172.21.15.63:3300/0 conn(0x7fddf80d3a50 0x7fddf80d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:48.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.691+0000 7fddf659c640 1 -- 172.21.15.45:0/1352591890 >> v2:172.21.15.63:3300/0 conn(0x7fddf80d3a50 msgr2=0x7fddf80d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:48.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.691+0000 7fddf659c640 1 --2- 172.21.15.45:0/1352591890 >> v2:172.21.15.63:3300/0 conn(0x7fddf80d3a50 0x7fddf80d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:48.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.691+0000 7fddf659c640 1 -- 172.21.15.45:0/1352591890 >> v1:172.21.15.63:6789/0 conn(0x7fddf80d0e80 legacy=0x7fddf80d19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:48.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.691+0000 7fddf659c640 1 -- 172.21.15.45:0/1352591890 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf80cfea0 con 0x7fddf80d3670 2024-05-08T05:42:48.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.691+0000 7fddf659c640 1 --2- 172.21.15.45:0/1352591890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf80d3670 0x7fddf80b49c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fddf808f550 tx=0x7fdde802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe598f63da4775f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:48.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.692+0000 7fddf559a640 1 -- 172.21.15.45:0/1352591890 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8036070 con 0x7fddf80d3670 2024-05-08T05:42:48.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.692+0000 7fddf559a640 1 -- 172.21.15.45:0/1352591890 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde8031600 con 0x7fddf80d3670 2024-05-08T05:42:48.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.692+0000 7fddf559a640 1 -- 172.21.15.45:0/1352591890 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde80319d0 con 0x7fddf80d3670 2024-05-08T05:42:48.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.692+0000 7fddfcd04640 1 -- 172.21.15.45:0/1352591890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf80d3670 msgr2=0x7fddf80b49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:48.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.692+0000 7fddfcd04640 1 --2- 172.21.15.45:0/1352591890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf80d3670 0x7fddf80b49c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fddf808f550 tx=0x7fdde802a6a0 comp rx=0 tx=0).stop 2024-05-08T05:42:48.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.692+0000 7fddfcd04640 1 -- 172.21.15.45:0/1352591890 shutdown_connections 2024-05-08T05:42:48.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.692+0000 7fddfcd04640 1 --2- 172.21.15.45:0/1352591890 >> v2:172.21.15.63:3300/0 conn(0x7fddf80d3a50 0x7fddf80d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:48.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.692+0000 7fddfcd04640 1 --2- 172.21.15.45:0/1352591890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf80d3670 0x7fddf80b49c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:48.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.692+0000 7fddfcd04640 1 -- 172.21.15.45:0/1352591890 >> 172.21.15.45:0/1352591890 conn(0x7fddf809e390 msgr2=0x7fddf80d5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:48.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.692+0000 7fddfcd04640 1 -- 172.21.15.45:0/1352591890 shutdown_connections 2024-05-08T05:42:48.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.692+0000 7fddfcd04640 1 -- 172.21.15.45:0/1352591890 wait complete. 2024-05-08T05:42:48.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.694+0000 7fddfcd04640 1 Processor -- start 2024-05-08T05:42:48.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.694+0000 7fddfcd04640 1 -- start start 2024-05-08T05:42:48.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.694+0000 7fddfcd04640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddf80d0e80 0x7fddf815a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:48.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.694+0000 7fddfcd04640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf80d3670 0x7fddf80ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:48.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.694+0000 7fddfcd04640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fddf80d3a50 0x7fddf81e9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:48.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.694+0000 7fddfcd04640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fddf80d0140 con 0x7fddf80d3670 2024-05-08T05:42:48.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.694+0000 7fddfcd04640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fddf80d2730 con 0x7fddf80d3a50 2024-05-08T05:42:48.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.694+0000 7fddfcd04640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fddf80b4f00 con 0x7fddf80d0e80 2024-05-08T05:42:48.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.695+0000 7fddf5d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf80d3670 0x7fddf80ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:48.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.695+0000 7fddf5d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf80d3670 0x7fddf80ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41398/0 (socket says 172.21.15.45:41398) 2024-05-08T05:42:48.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.695+0000 7fddf5d9b640 1 -- 172.21.15.45:0/2857502511 learned_addr learned my addr 172.21.15.45:0/2857502511 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:48.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.695+0000 7fddf6d9d640 1 --2- 172.21.15.45:0/2857502511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fddf80d3a50 0x7fddf81e9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:48.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.695+0000 7fddf659c640 1 --2- 172.21.15.45:0/2857502511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddf80d0e80 0x7fddf815a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:48.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.695+0000 7fddf5d9b640 1 -- 172.21.15.45:0/2857502511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddf80d0e80 msgr2=0x7fddf815a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:48.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.695+0000 7fddf5d9b640 1 --2- 172.21.15.45:0/2857502511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddf80d0e80 0x7fddf815a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:48.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.695+0000 7fddf5d9b640 1 -- 172.21.15.45:0/2857502511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fddf80d3a50 msgr2=0x7fddf81e9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:48.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.695+0000 7fddf5d9b640 1 --2- 172.21.15.45:0/2857502511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fddf80d3a50 0x7fddf81e9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:48.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.695+0000 7fddf5d9b640 1 -- 172.21.15.45:0/2857502511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf80ccbe0 con 0x7fddf80d3670 2024-05-08T05:42:48.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.695+0000 7fddf5d9b640 1 --2- 172.21.15.45:0/2857502511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf80d3670 0x7fddf80ce7b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdde000d800 tx=0x7fdde000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:48.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.695+0000 7fddeeffd640 1 -- 172.21.15.45:0/2857502511 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde000b660 con 0x7fddf80d3670 2024-05-08T05:42:48.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.695+0000 7fddeeffd640 1 -- 172.21.15.45:0/2857502511 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde0019070 con 0x7fddf80d3670 2024-05-08T05:42:48.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.695+0000 7fddfcd04640 1 -- 172.21.15.45:0/2857502511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf80ccec0 con 0x7fddf80d3670 2024-05-08T05:42:48.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.695+0000 7fddeeffd640 1 -- 172.21.15.45:0/2857502511 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde000ba20 con 0x7fddf80d3670 2024-05-08T05:42:48.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.695+0000 7fddfcd04640 1 -- 172.21.15.45:0/2857502511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf80cd480 con 0x7fddf80d3670 2024-05-08T05:42:48.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.696+0000 7fddfcd04640 1 -- 172.21.15.45:0/2857502511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf80693f0 con 0x7fddf80d3670 2024-05-08T05:42:48.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.697+0000 7fddeeffd640 1 -- 172.21.15.45:0/2857502511 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdde000bbc0 con 0x7fddf80d3670 2024-05-08T05:42:48.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.698+0000 7fddeeffd640 1 --2- 172.21.15.45:0/2857502511 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fddd8076300 0x7fddd80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:48.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.698+0000 7fddeeffd640 1 -- 172.21.15.45:0/2857502511 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6187+0+0 (secure 0 0 0) 0x7fdde0097bf0 con 0x7fddf80d3670 2024-05-08T05:42:48.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.698+0000 7fddf659c640 1 --2- 172.21.15.45:0/2857502511 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fddd8076300 0x7fddd80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:48.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.698+0000 7fddf659c640 1 --2- 172.21.15.45:0/2857502511 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fddd8076300 0x7fddd80787c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdde8033000 tx=0x7fdde8009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:48.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.702+0000 7fddeeffd640 1 -- 172.21.15.45:0/2857502511 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdde0061370 con 0x7fddf80d3670 2024-05-08T05:42:48.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:48.896+0000 7fddfcd04640 1 -- 172.21.15.45:0/2857502511 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fddf80b5430 con 0x7fddf80d3670 2024-05-08T05:42:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.520+0000 7fddeeffd640 1 -- 172.21.15.45:0/2857502511 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) ==== 181+0+0 (secure 0 0 0) 0x7fdde0067070 con 0x7fddf80d3670 2024-05-08T05:42:49.521 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-05-08T05:42:49.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.524+0000 7fddfcd04640 1 -- 172.21.15.45:0/2857502511 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fddd8076300 msgr2=0x7fddd80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:49.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.524+0000 7fddfcd04640 1 --2- 172.21.15.45:0/2857502511 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fddd8076300 0x7fddd80787c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdde8033000 tx=0x7fdde8009000 comp rx=0 tx=0).stop 2024-05-08T05:42:49.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.524+0000 7fddfcd04640 1 -- 172.21.15.45:0/2857502511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf80d3670 msgr2=0x7fddf80ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:49.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.524+0000 7fddfcd04640 1 --2- 172.21.15.45:0/2857502511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf80d3670 0x7fddf80ce7b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdde000d800 tx=0x7fdde000dcc0 comp rx=0 tx=0).stop 2024-05-08T05:42:49.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.525+0000 7fddfcd04640 1 -- 172.21.15.45:0/2857502511 shutdown_connections 2024-05-08T05:42:49.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.525+0000 7fddfcd04640 1 --2- 172.21.15.45:0/2857502511 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fddd8076300 0x7fddd80787c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:49.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.525+0000 7fddfcd04640 1 --2- 172.21.15.45:0/2857502511 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fddf80d3a50 0x7fddf81e9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:49.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.525+0000 7fddfcd04640 1 --2- 172.21.15.45:0/2857502511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fddf80d3670 0x7fddf80ce7b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:49.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.525+0000 7fddfcd04640 1 --2- 172.21.15.45:0/2857502511 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fddf80d0e80 0x7fddf815a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:49.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.525+0000 7fddfcd04640 1 -- 172.21.15.45:0/2857502511 >> 172.21.15.45:0/2857502511 conn(0x7fddf809e390 msgr2=0x7fddf806d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:49.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.525+0000 7fddfcd04640 1 -- 172.21.15.45:0/2857502511 shutdown_connections 2024-05-08T05:42:49.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.525+0000 7fddfcd04640 1 -- 172.21.15.45:0/2857502511 wait complete. 2024-05-08T05:42:49.539 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T05:42:49.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.654+0000 7f7970cf8640 1 Processor -- start 2024-05-08T05:42:49.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.654+0000 7f7970cf8640 1 -- start start 2024-05-08T05:42:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.655+0000 7f7970cf8640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f796c0d35f0 0x7f796c0d39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.655+0000 7f7970cf8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f796c0cfbb0 0x7f796c0cfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.655+0000 7f7970cf8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f796c0d16d0 0x7f796c0d1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.655+0000 7f7970cf8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f796c0d0530 con 0x7f796c0cfbb0 2024-05-08T05:42:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.655+0000 7f7970cf8640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f796c162e60 con 0x7f796c0d35f0 2024-05-08T05:42:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.655+0000 7f7970cf8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f796c0d0770 con 0x7f796c0d16d0 2024-05-08T05:42:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.655+0000 7f7969d9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f796c0cfbb0 0x7f796c0cfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.655+0000 7f7969d9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f796c0cfbb0 0x7f796c0cfff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41404/0 (socket says 172.21.15.45:41404) 2024-05-08T05:42:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.655+0000 7f7969d9b640 1 -- 172.21.15.45:0/1391395365 learned_addr learned my addr 172.21.15.45:0/1391395365 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.655+0000 7f796ad9d640 1 --2- 172.21.15.45:0/1391395365 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f796c0d16d0 0x7f796c0d1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:49.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.655+0000 7f796a59c640 1 --2- 172.21.15.45:0/1391395365 >> v2:172.21.15.63:3300/0 conn(0x7f796c0d35f0 0x7f796c0d39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.655+0000 7f7969d9b640 1 -- 172.21.15.45:0/1391395365 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f796c0d16d0 msgr2=0x7f796c0d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.655+0000 7f7969d9b640 1 --2- 172.21.15.45:0/1391395365 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f796c0d16d0 0x7f796c0d1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.655+0000 7f7969d9b640 1 -- 172.21.15.45:0/1391395365 >> v2:172.21.15.63:3300/0 conn(0x7f796c0d35f0 msgr2=0x7f796c0d39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.655+0000 7f7969d9b640 1 --2- 172.21.15.45:0/1391395365 >> v2:172.21.15.63:3300/0 conn(0x7f796c0d35f0 0x7f796c0d39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.655+0000 7f7969d9b640 1 -- 172.21.15.45:0/1391395365 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796c0d0a50 con 0x7f796c0cfbb0 2024-05-08T05:42:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.655+0000 7f796ad9d640 1 --2- 172.21.15.45:0/1391395365 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f796c0d16d0 0x7f796c0d1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:49.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.655+0000 7f796a59c640 1 --2- 172.21.15.45:0/1391395365 >> v2:172.21.15.63:3300/0 conn(0x7f796c0d35f0 0x7f796c0d39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:49.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.656+0000 7f7969d9b640 1 --2- 172.21.15.45:0/1391395365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f796c0cfbb0 0x7f796c0cfff0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f796c0b7ba0 tx=0x7f796002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539f39b3a68ee4c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:49.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.656+0000 7f796959a640 1 -- 172.21.15.45:0/1391395365 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7960037070 con 0x7f796c0cfbb0 2024-05-08T05:42:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.656+0000 7f796959a640 1 -- 172.21.15.45:0/1391395365 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7960032440 con 0x7f796c0cfbb0 2024-05-08T05:42:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.656+0000 7f796959a640 1 -- 172.21.15.45:0/1391395365 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7960032810 con 0x7f796c0cfbb0 2024-05-08T05:42:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.657+0000 7f7970cf8640 1 -- 172.21.15.45:0/1391395365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f796c0cfbb0 msgr2=0x7f796c0cfff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.657+0000 7f7970cf8640 1 --2- 172.21.15.45:0/1391395365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f796c0cfbb0 0x7f796c0cfff0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f796c0b7ba0 tx=0x7f796002a270 comp rx=0 tx=0).stop 2024-05-08T05:42:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.657+0000 7f7970cf8640 1 -- 172.21.15.45:0/1391395365 shutdown_connections 2024-05-08T05:42:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.657+0000 7f7970cf8640 1 --2- 172.21.15.45:0/1391395365 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f796c0d16d0 0x7f796c0d1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.657+0000 7f7970cf8640 1 --2- 172.21.15.45:0/1391395365 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f796c0cfbb0 0x7f796c0cfff0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.657+0000 7f7970cf8640 1 --2- 172.21.15.45:0/1391395365 >> v2:172.21.15.63:3300/0 conn(0x7f796c0d35f0 0x7f796c0d39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.657+0000 7f7970cf8640 1 -- 172.21.15.45:0/1391395365 >> 172.21.15.45:0/1391395365 conn(0x7f796c09e3e0 msgr2=0x7f796c0d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.657+0000 7f7970cf8640 1 -- 172.21.15.45:0/1391395365 shutdown_connections 2024-05-08T05:42:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.657+0000 7f7970cf8640 1 -- 172.21.15.45:0/1391395365 wait complete. 2024-05-08T05:42:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.658+0000 7f7970cf8640 1 Processor -- start 2024-05-08T05:42:49.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.658+0000 7f7970cf8640 1 -- start start 2024-05-08T05:42:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.659+0000 7f7970cf8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f796c0cfbb0 0x7f796c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.659+0000 7f7970cf8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f796c0d16d0 0x7f796c0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:49.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.659+0000 7f7970cf8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f796c0d35f0 0x7f796c09fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.659+0000 7f7970cf8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f796c0b6230 con 0x7f796c0d35f0 2024-05-08T05:42:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.659+0000 7f7970cf8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f796c0b8000 con 0x7f796c0cfbb0 2024-05-08T05:42:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.659+0000 7f7970cf8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f796c0d3f10 con 0x7f796c0d16d0 2024-05-08T05:42:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.659+0000 7f796ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f796c0d35f0 0x7f796c09fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.659+0000 7f796ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f796c0d35f0 0x7f796c09fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41412/0 (socket says 172.21.15.45:41412) 2024-05-08T05:42:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.659+0000 7f796ad9d640 1 -- 172.21.15.45:0/3827381957 learned_addr learned my addr 172.21.15.45:0/3827381957 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.660+0000 7f796a59c640 1 --2- 172.21.15.45:0/3827381957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f796c0cfbb0 0x7f796c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.660+0000 7f7969d9b640 1 --2- 172.21.15.45:0/3827381957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f796c0d16d0 0x7f796c0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.660+0000 7f796ad9d640 1 -- 172.21.15.45:0/3827381957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f796c0d16d0 msgr2=0x7f796c0cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.660+0000 7f796ad9d640 1 --2- 172.21.15.45:0/3827381957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f796c0d16d0 0x7f796c0cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.660+0000 7f796ad9d640 1 -- 172.21.15.45:0/3827381957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f796c0cfbb0 msgr2=0x7f796c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.660+0000 7f796ad9d640 1 --2- 172.21.15.45:0/3827381957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f796c0cfbb0 0x7f796c0ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.660+0000 7f796ad9d640 1 -- 172.21.15.45:0/3827381957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796c15d8e0 con 0x7f796c0d35f0 2024-05-08T05:42:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.660+0000 7f796a59c640 1 --2- 172.21.15.45:0/3827381957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f796c0cfbb0 0x7f796c0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.660+0000 7f796ad9d640 1 --2- 172.21.15.45:0/3827381957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f796c0d35f0 0x7f796c09fb40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f796400cef0 tx=0x7f796400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.660+0000 7f795affd640 1 -- 172.21.15.45:0/3827381957 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796401a070 con 0x7f796c0d35f0 2024-05-08T05:42:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.660+0000 7f795affd640 1 -- 172.21.15.45:0/3827381957 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f796400d840 con 0x7f796c0d35f0 2024-05-08T05:42:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.660+0000 7f7970cf8640 1 -- 172.21.15.45:0/3827381957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796c15dbc0 con 0x7f796c0d35f0 2024-05-08T05:42:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.660+0000 7f795affd640 1 -- 172.21.15.45:0/3827381957 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796400dc30 con 0x7f796c0d35f0 2024-05-08T05:42:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.660+0000 7f7970cf8640 1 -- 172.21.15.45:0/3827381957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f796c15e180 con 0x7f796c0d35f0 2024-05-08T05:42:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.661+0000 7f7970cf8640 1 -- 172.21.15.45:0/3827381957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f796c0693f0 con 0x7f796c0d35f0 2024-05-08T05:42:49.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.662+0000 7f795affd640 1 -- 172.21.15.45:0/3827381957 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f796401d070 con 0x7f796c0d35f0 2024-05-08T05:42:49.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.662+0000 7f795affd640 1 --2- 172.21.15.45:0/3827381957 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f794c0762d0 0x7f794c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:49.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.662+0000 7f795affd640 1 -- 172.21.15.45:0/3827381957 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5749+0+0 (secure 0 0 0) 0x7f7964097ec0 con 0x7f796c0d35f0 2024-05-08T05:42:49.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.662+0000 7f796a59c640 1 --2- 172.21.15.45:0/3827381957 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f794c0762d0 0x7f794c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:49.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.663+0000 7f796a59c640 1 --2- 172.21.15.45:0/3827381957 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f794c0762d0 0x7f794c078790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f795c00a990 tx=0x7f795c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:49.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.667+0000 7f795affd640 1 -- 172.21.15.45:0/3827381957 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7964061750 con 0x7f796c0d35f0 2024-05-08T05:42:49.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:49.857+0000 7f7970cf8640 1 -- 172.21.15.45:0/3827381957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f796c0d2f40 con 0x7f796c0d35f0 2024-05-08T05:42:50.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.522+0000 7f795affd640 1 -- 172.21.15.45:0/3827381957 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) ==== 169+0+0 (secure 0 0 0) 0x7f7964061050 con 0x7f796c0d35f0 2024-05-08T05:42:50.523 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-05-08T05:42:50.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.526+0000 7f7970cf8640 1 -- 172.21.15.45:0/3827381957 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f794c0762d0 msgr2=0x7f794c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:50.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.526+0000 7f7970cf8640 1 --2- 172.21.15.45:0/3827381957 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f794c0762d0 0x7f794c078790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f795c00a990 tx=0x7f795c008040 comp rx=0 tx=0).stop 2024-05-08T05:42:50.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.526+0000 7f7970cf8640 1 -- 172.21.15.45:0/3827381957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f796c0d35f0 msgr2=0x7f796c09fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:50.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.526+0000 7f7970cf8640 1 --2- 172.21.15.45:0/3827381957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f796c0d35f0 0x7f796c09fb40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f796400cef0 tx=0x7f796400a710 comp rx=0 tx=0).stop 2024-05-08T05:42:50.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.527+0000 7f7970cf8640 1 -- 172.21.15.45:0/3827381957 shutdown_connections 2024-05-08T05:42:50.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.527+0000 7f7970cf8640 1 --2- 172.21.15.45:0/3827381957 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f794c0762d0 0x7f794c078790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:50.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.527+0000 7f7970cf8640 1 --2- 172.21.15.45:0/3827381957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f796c0d16d0 0x7f796c0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:50.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.527+0000 7f7970cf8640 1 --2- 172.21.15.45:0/3827381957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f796c0d35f0 0x7f796c09fb40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:50.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.527+0000 7f7970cf8640 1 --2- 172.21.15.45:0/3827381957 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f796c0cfbb0 0x7f796c0ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:50.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.527+0000 7f7970cf8640 1 -- 172.21.15.45:0/3827381957 >> 172.21.15.45:0/3827381957 conn(0x7f796c09e3e0 msgr2=0x7f796c0ba430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:50.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.527+0000 7f7970cf8640 1 -- 172.21.15.45:0/3827381957 shutdown_connections 2024-05-08T05:42:50.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.527+0000 7f7970cf8640 1 -- 172.21.15.45:0/3827381957 wait complete. 2024-05-08T05:42:50.543 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T05:42:50.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.655+0000 7ff3d0bc1640 1 Processor -- start 2024-05-08T05:42:50.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.655+0000 7ff3d0bc1640 1 -- start start 2024-05-08T05:42:50.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.656+0000 7ff3d0bc1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3cc0d3cd0 0x7ff3cc0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:50.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.656+0000 7ff3d0bc1640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff3cc0c9ee0 0x7ff3cc0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:50.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.656+0000 7ff3d0bc1640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3cc0d3330 con 0x7ff3cc1629a0 2024-05-08T05:42:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.656+0000 7ff3d0bc1640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff3cc0d2170 con 0x7ff3cc0c9ee0 2024-05-08T05:42:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.656+0000 7ff3d0bc1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff3cc0d2450 con 0x7ff3cc0d3cd0 2024-05-08T05:42:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.656+0000 7ff3ca59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff3cc1629a0 0x7ff3cc0d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36550/0 (socket says 172.21.15.45:36550) 2024-05-08T05:42:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.656+0000 7ff3ca59c640 1 -- 172.21.15.45:0/3202920082 learned_addr learned my addr 172.21.15.45:0/3202920082 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.656+0000 7ff3cad9d640 1 --2- 172.21.15.45:0/3202920082 >> v2:172.21.15.63:3300/0 conn(0x7ff3cc0c9ee0 0x7ff3cc0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.656+0000 7ff3c9d9b640 1 --2- 172.21.15.45:0/3202920082 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3cc0d3cd0 0x7ff3cc0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.656+0000 7ff3c959a640 1 -- 172.21.15.45:0/3202920082 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3648220048 0 0) 0x7ff3cc0d3330 con 0x7ff3cc1629a0 2024-05-08T05:42:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3c959a640 1 -- 172.21.15.45:0/3202920082 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3b4003520 con 0x7ff3cc1629a0 2024-05-08T05:42:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3c959a640 1 -- 172.21.15.45:0/3202920082 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 742284057 0 0) 0x7ff3b4003520 con 0x7ff3cc1629a0 2024-05-08T05:42:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3c959a640 1 -- 172.21.15.45:0/3202920082 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3cc0d3cd0 msgr2=0x7ff3cc0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3c959a640 1 --2- 172.21.15.45:0/3202920082 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3cc0d3cd0 0x7ff3cc0cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3c959a640 1 -- 172.21.15.45:0/3202920082 >> v2:172.21.15.63:3300/0 conn(0x7ff3cc0c9ee0 msgr2=0x7ff3cc0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3c959a640 1 --2- 172.21.15.45:0/3202920082 >> v2:172.21.15.63:3300/0 conn(0x7ff3cc0c9ee0 0x7ff3cc0d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3c959a640 1 -- 172.21.15.45:0/3202920082 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3cc0d2730 con 0x7ff3cc1629a0 2024-05-08T05:42:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3c959a640 1 -- 172.21.15.45:0/3202920082 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7ff3bc0016f0 con 0x7ff3cc1629a0 2024-05-08T05:42:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3c959a640 1 -- 172.21.15.45:0/3202920082 >> v1:172.21.15.45:6789/0 conn(0x7ff3cc1629a0 legacy=0x7ff3cc0d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3c959a640 1 --2- 172.21.15.45:0/3202920082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b4003a00 0x7ff3b4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3c959a640 1 --2- 172.21.15.45:0/3202920082 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3cc0d3cd0 0x7ff3b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3c959a640 1 --2- 172.21.15.45:0/3202920082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3b4004d40 0x7ff3b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3c959a640 1 -- 172.21.15.45:0/3202920082 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc0d2170 con 0x7ff3b4003a00 2024-05-08T05:42:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3c959a640 1 -- 172.21.15.45:0/3202920082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc0d2450 con 0x7ff3b4004d40 2024-05-08T05:42:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3c959a640 1 -- 172.21.15.45:0/3202920082 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff3b400baf0 con 0x7ff3cc0d3cd0 2024-05-08T05:42:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3ca59c640 1 --2- 172.21.15.45:0/3202920082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b4003a00 0x7ff3b4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3c9d9b640 1 --2- 172.21.15.45:0/3202920082 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3cc0d3cd0 0x7ff3b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3cad9d640 1 --2- 172.21.15.45:0/3202920082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3b4004d40 0x7ff3b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3ca59c640 1 -- 172.21.15.45:0/3202920082 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3cc0d3cd0 msgr2=0x7ff3b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:50.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3ca59c640 1 --2- 172.21.15.45:0/3202920082 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3cc0d3cd0 0x7ff3b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3ca59c640 1 -- 172.21.15.45:0/3202920082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3b4004d40 msgr2=0x7ff3b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3ca59c640 1 --2- 172.21.15.45:0/3202920082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3b4004d40 0x7ff3b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.657+0000 7ff3ca59c640 1 -- 172.21.15.45:0/3202920082 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b4003520 con 0x7ff3b4003a00 2024-05-08T05:42:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.658+0000 7ff3ca59c640 1 --2- 172.21.15.45:0/3202920082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b4003a00 0x7ff3b4003e20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00c290 tx=0x7ff3b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8947875f4b5f5e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.658+0000 7ff3c959a640 1 -- 172.21.15.45:0/3202920082 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc007d50 con 0x7ff3b4003a00 2024-05-08T05:42:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.658+0000 7ff3c959a640 1 -- 172.21.15.45:0/3202920082 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3bc005af0 con 0x7ff3b4003a00 2024-05-08T05:42:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.658+0000 7ff3c959a640 1 -- 172.21.15.45:0/3202920082 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc007070 con 0x7ff3b4003a00 2024-05-08T05:42:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.659+0000 7ff3d0bc1640 1 -- 172.21.15.45:0/3202920082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b4003a00 msgr2=0x7ff3b4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.659+0000 7ff3d0bc1640 1 --2- 172.21.15.45:0/3202920082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3b4003a00 0x7ff3b4003e20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00c290 tx=0x7ff3b4005830 comp rx=0 tx=0).stop 2024-05-08T05:42:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.659+0000 7ff3cad9d640 1 -- 172.21.15.45:0/3202920082 reap_dead start 2024-05-08T05:42:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.659+0000 7ff3d0bc1640 1 -- 172.21.15.45:0/3202920082 shutdown_connections 2024-05-08T05:42:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.659+0000 7ff3d0bc1640 1 -- 172.21.15.45:0/3202920082 >> 172.21.15.45:0/3202920082 conn(0x7ff3cc0d4fe0 msgr2=0x7ff3cc0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.659+0000 7ff3d0bc1640 1 -- 172.21.15.45:0/3202920082 shutdown_connections 2024-05-08T05:42:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.659+0000 7ff3d0bc1640 1 -- 172.21.15.45:0/3202920082 wait complete. 2024-05-08T05:42:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.660+0000 7ff3d0bc1640 1 Processor -- start 2024-05-08T05:42:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.660+0000 7ff3d0bc1640 1 -- start start 2024-05-08T05:42:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.661+0000 7ff3d0bc1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3b4003a00 0x7ff3cc15ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.661+0000 7ff3d0bc1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3cc0cee60 0x7ff3cc1f4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:50.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.661+0000 7ff3d0bc1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3cc1f5090 0x7ff3cc0b9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.661+0000 7ff3d0bc1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc0cf440 con 0x7ff3cc0cee60 2024-05-08T05:42:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.661+0000 7ff3d0bc1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc0cf2c0 con 0x7ff3cc1f5090 2024-05-08T05:42:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.661+0000 7ff3d0bc1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff3cc0d2dc0 con 0x7ff3b4003a00 2024-05-08T05:42:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.661+0000 7ff3c9d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3cc0cee60 0x7ff3cc1f4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.661+0000 7ff3c9d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3cc0cee60 0x7ff3cc1f4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50542/0 (socket says 172.21.15.45:50542) 2024-05-08T05:42:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.661+0000 7ff3c9d9b640 1 -- 172.21.15.45:0/3225801673 learned_addr learned my addr 172.21.15.45:0/3225801673 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.661+0000 7ff3cad9d640 1 --2- 172.21.15.45:0/3225801673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3cc1f5090 0x7ff3cc0b9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.661+0000 7ff3c9d9b640 1 -- 172.21.15.45:0/3225801673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3b4003a00 msgr2=0x7ff3cc15ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.661+0000 7ff3c9d9b640 1 --2- 172.21.15.45:0/3225801673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3b4003a00 0x7ff3cc15ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.661+0000 7ff3c9d9b640 1 -- 172.21.15.45:0/3225801673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3cc1f5090 msgr2=0x7ff3cc0b9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.661+0000 7ff3c9d9b640 1 --2- 172.21.15.45:0/3225801673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3cc1f5090 0x7ff3cc0b9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.661+0000 7ff3c9d9b640 1 -- 172.21.15.45:0/3225801673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3cc1f55b0 con 0x7ff3cc0cee60 2024-05-08T05:42:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.662+0000 7ff3c9d9b640 1 --2- 172.21.15.45:0/3225801673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3cc0cee60 0x7ff3cc1f4b50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00119a0 tx=0x7ff3c0011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:50.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.662+0000 7ff3baffd640 1 -- 172.21.15.45:0/3225801673 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c000faf0 con 0x7ff3cc0cee60 2024-05-08T05:42:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.662+0000 7ff3baffd640 1 -- 172.21.15.45:0/3225801673 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c000fc90 con 0x7ff3cc0cee60 2024-05-08T05:42:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.662+0000 7ff3d0bc1640 1 -- 172.21.15.45:0/3225801673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3cc1f5890 con 0x7ff3cc0cee60 2024-05-08T05:42:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.662+0000 7ff3baffd640 1 -- 172.21.15.45:0/3225801673 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0014440 con 0x7ff3cc0cee60 2024-05-08T05:42:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.662+0000 7ff3d0bc1640 1 -- 172.21.15.45:0/3225801673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3cc15d3c0 con 0x7ff3cc0cee60 2024-05-08T05:42:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.663+0000 7ff3d0bc1640 1 -- 172.21.15.45:0/3225801673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3cc0693f0 con 0x7ff3cc0cee60 2024-05-08T05:42:50.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.664+0000 7ff3baffd640 1 -- 172.21.15.45:0/3225801673 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff3c0012040 con 0x7ff3cc0cee60 2024-05-08T05:42:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.664+0000 7ff3baffd640 1 --2- 172.21.15.45:0/3225801673 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff3ac0762d0 0x7ff3ac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.664+0000 7ff3baffd640 1 -- 172.21.15.45:0/3225801673 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5335+0+0 (secure 0 0 0) 0x7ff3c0097aa0 con 0x7ff3cc0cee60 2024-05-08T05:42:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.664+0000 7ff3ca59c640 1 --2- 172.21.15.45:0/3225801673 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff3ac0762d0 0x7ff3ac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.665+0000 7ff3ca59c640 1 --2- 172.21.15.45:0/3225801673 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff3ac0762d0 0x7ff3ac078790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc032000 tx=0x7ff3bc009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.668+0000 7ff3baffd640 1 -- 172.21.15.45:0/3225801673 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff3c0019030 con 0x7ff3cc0cee60 2024-05-08T05:42:50.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.865+0000 7ff3d0bc1640 1 -- 172.21.15.45:0/3225801673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7ff3cc0b8110 con 0x7ff3cc0cee60 2024-05-08T05:42:50.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.868+0000 7ff3baffd640 1 -- 172.21.15.45:0/3225801673 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7ff3c0061600 con 0x7ff3cc0cee60 2024-05-08T05:42:50.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.870+0000 7ff3d0bc1640 1 -- 172.21.15.45:0/3225801673 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff3ac0762d0 msgr2=0x7ff3ac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:50.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.870+0000 7ff3d0bc1640 1 --2- 172.21.15.45:0/3225801673 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff3ac0762d0 0x7ff3ac078790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc032000 tx=0x7ff3bc009030 comp rx=0 tx=0).stop 2024-05-08T05:42:50.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.870+0000 7ff3d0bc1640 1 -- 172.21.15.45:0/3225801673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3cc0cee60 msgr2=0x7ff3cc1f4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:50.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.870+0000 7ff3d0bc1640 1 --2- 172.21.15.45:0/3225801673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3cc0cee60 0x7ff3cc1f4b50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00119a0 tx=0x7ff3c0011e60 comp rx=0 tx=0).stop 2024-05-08T05:42:50.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.870+0000 7ff3d0bc1640 1 -- 172.21.15.45:0/3225801673 shutdown_connections 2024-05-08T05:42:50.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.870+0000 7ff3d0bc1640 1 --2- 172.21.15.45:0/3225801673 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff3ac0762d0 0x7ff3ac078790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:50.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.870+0000 7ff3d0bc1640 1 --2- 172.21.15.45:0/3225801673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3cc1f5090 0x7ff3cc0b9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:50.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.870+0000 7ff3d0bc1640 1 --2- 172.21.15.45:0/3225801673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff3cc0cee60 0x7ff3cc1f4b50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:50.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.870+0000 7ff3d0bc1640 1 --2- 172.21.15.45:0/3225801673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff3b4003a00 0x7ff3cc15ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:50.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.870+0000 7ff3d0bc1640 1 -- 172.21.15.45:0/3225801673 >> 172.21.15.45:0/3225801673 conn(0x7ff3cc0d4fe0 msgr2=0x7ff3cc160a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:50.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.870+0000 7ff3d0bc1640 1 -- 172.21.15.45:0/3225801673 shutdown_connections 2024-05-08T05:42:50.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.870+0000 7ff3d0bc1640 1 -- 172.21.15.45:0/3225801673 wait complete. 2024-05-08T05:42:50.882 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-05-08T05:42:50.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.993+0000 7fd06ec72640 1 Processor -- start 2024-05-08T05:42:50.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.993+0000 7fd06ec72640 1 -- start start 2024-05-08T05:42:51.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.994+0000 7fd06ec72640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd06815bc50 0x7fd0680b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:51.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.994+0000 7fd06ec72640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0680711f0 con 0x7fd06806f540 2024-05-08T05:42:51.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.994+0000 7fd06ec72640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd06806fa40 con 0x7fd0680732c0 2024-05-08T05:42:51.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.994+0000 7fd06ec72640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd06806fd20 con 0x7fd06815bc50 2024-05-08T05:42:51.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.994+0000 7fd067fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd06806f540 0x7fd068070ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36566/0 (socket says 172.21.15.45:36566) 2024-05-08T05:42:51.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.994+0000 7fd067fff640 1 -- 172.21.15.45:0/4030831063 learned_addr learned my addr 172.21.15.45:0/4030831063 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:51.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.995+0000 7fd06ca0e640 1 --2- 172.21.15.45:0/4030831063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd06815bc50 0x7fd0680b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:51.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.995+0000 7fd0677fe640 1 -- 172.21.15.45:0/4030831063 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2191806086 0 0) 0x7fd0680711f0 con 0x7fd06806f540 2024-05-08T05:42:51.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.995+0000 7fd0677fe640 1 -- 172.21.15.45:0/4030831063 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd058003520 con 0x7fd06806f540 2024-05-08T05:42:51.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.995+0000 7fd0677fe640 1 -- 172.21.15.45:0/4030831063 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1404143794 0 0) 0x7fd06806fa40 con 0x7fd0680732c0 2024-05-08T05:42:51.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.995+0000 7fd0677fe640 1 -- 172.21.15.45:0/4030831063 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0680711f0 con 0x7fd0680732c0 2024-05-08T05:42:51.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.995+0000 7fd06ca0e640 1 -- 172.21.15.45:0/4030831063 >> v1:172.21.15.63:6789/0 conn(0x7fd0680732c0 legacy=0x7fd0680cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:51.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.995+0000 7fd06ca0e640 1 -- 172.21.15.45:0/4030831063 >> v1:172.21.15.45:6789/0 conn(0x7fd06806f540 legacy=0x7fd068070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:51.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.995+0000 7fd0677fe640 1 -- 172.21.15.45:0/4030831063 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2510079908 0 0) 0x7fd058003520 con 0x7fd06806f540 2024-05-08T05:42:51.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.995+0000 7fd06ca0e640 1 -- 172.21.15.45:0/4030831063 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd068070000 con 0x7fd06815bc50 2024-05-08T05:42:51.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.996+0000 7fd06ca0e640 1 --2- 172.21.15.45:0/4030831063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd06815bc50 0x7fd0680b5ba0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0097d0 tx=0x7fd05c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ed1c2f9bf6ab8b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:51.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.996+0000 7fd0677fe640 1 -- 172.21.15.45:0/4030831063 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c037070 con 0x7fd06815bc50 2024-05-08T05:42:51.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.996+0000 7fd0677fe640 1 -- 172.21.15.45:0/4030831063 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd05c02ac30 con 0x7fd06815bc50 2024-05-08T05:42:51.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.996+0000 7fd0677fe640 1 -- 172.21.15.45:0/4030831063 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0540016f0 con 0x7fd06815bc50 2024-05-08T05:42:51.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.997+0000 7fd06ec72640 1 -- 172.21.15.45:0/4030831063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd06815bc50 msgr2=0x7fd0680b5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:51.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.997+0000 7fd06ec72640 1 --2- 172.21.15.45:0/4030831063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd06815bc50 0x7fd0680b5ba0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd05c0097d0 tx=0x7fd05c02a1a0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.997+0000 7fd06ec72640 1 -- 172.21.15.45:0/4030831063 shutdown_connections 2024-05-08T05:42:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.997+0000 7fd06ec72640 1 --2- 172.21.15.45:0/4030831063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd06815bc50 0x7fd0680b5ba0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.997+0000 7fd06ec72640 1 -- 172.21.15.45:0/4030831063 >> 172.21.15.45:0/4030831063 conn(0x7fd0680b3c40 msgr2=0x7fd0680b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.997+0000 7fd06ec72640 1 -- 172.21.15.45:0/4030831063 shutdown_connections 2024-05-08T05:42:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.997+0000 7fd06ec72640 1 -- 172.21.15.45:0/4030831063 wait complete. 2024-05-08T05:42:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.998+0000 7fd06ec72640 1 Processor -- start 2024-05-08T05:42:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.998+0000 7fd06ec72640 1 -- start start 2024-05-08T05:42:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.999+0000 7fd06ec72640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd06806f540 0x7fd068152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.999+0000 7fd06ec72640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0680732c0 0x7fd0680cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.999+0000 7fd06ec72640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06815bc50 0x7fd0680ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.999+0000 7fd06ec72640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0680b60e0 con 0x7fd06815bc50 2024-05-08T05:42:51.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.999+0000 7fd06ec72640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd0680705c0 con 0x7fd0680732c0 2024-05-08T05:42:51.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.999+0000 7fd06ec72640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd0680b6260 con 0x7fd06806f540 2024-05-08T05:42:51.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.999+0000 7fd06d20f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06815bc50 0x7fd0680ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:51.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.999+0000 7fd06d20f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06815bc50 0x7fd0680ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50556/0 (socket says 172.21.15.45:50556) 2024-05-08T05:42:51.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:50.999+0000 7fd06d20f640 1 -- 172.21.15.45:0/1632527238 learned_addr learned my addr 172.21.15.45:0/1632527238 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.000+0000 7fd067fff640 1 --2- 172.21.15.45:0/1632527238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0680732c0 0x7fd0680cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.000+0000 7fd06ca0e640 1 --2- 172.21.15.45:0/1632527238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd06806f540 0x7fd068152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.000+0000 7fd06d20f640 1 -- 172.21.15.45:0/1632527238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd06806f540 msgr2=0x7fd068152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:51.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.000+0000 7fd06d20f640 1 --2- 172.21.15.45:0/1632527238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd06806f540 0x7fd068152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.000+0000 7fd06d20f640 1 -- 172.21.15.45:0/1632527238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0680732c0 msgr2=0x7fd0680cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.000+0000 7fd06d20f640 1 --2- 172.21.15.45:0/1632527238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0680732c0 0x7fd0680cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.000+0000 7fd06d20f640 1 -- 172.21.15.45:0/1632527238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0680ac550 con 0x7fd06815bc50 2024-05-08T05:42:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.000+0000 7fd06d20f640 1 --2- 172.21.15.45:0/1632527238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06815bc50 0x7fd0680ccdf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd06000c780 tx=0x7fd06000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:51.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.000+0000 7fd0657fa640 1 -- 172.21.15.45:0/1632527238 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060003b70 con 0x7fd06815bc50 2024-05-08T05:42:51.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.000+0000 7fd0657fa640 1 -- 172.21.15.45:0/1632527238 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd060003d10 con 0x7fd06815bc50 2024-05-08T05:42:51.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.000+0000 7fd06ec72640 1 -- 172.21.15.45:0/1632527238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0680ab4b0 con 0x7fd06815bc50 2024-05-08T05:42:51.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.000+0000 7fd0657fa640 1 -- 172.21.15.45:0/1632527238 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06000aa90 con 0x7fd06815bc50 2024-05-08T05:42:51.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.000+0000 7fd06ec72640 1 -- 172.21.15.45:0/1632527238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0680aba70 con 0x7fd06815bc50 2024-05-08T05:42:51.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.001+0000 7fd06ec72640 1 -- 172.21.15.45:0/1632527238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd068069b00 con 0x7fd06815bc50 2024-05-08T05:42:51.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.002+0000 7fd0657fa640 1 -- 172.21.15.45:0/1632527238 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd06000ac30 con 0x7fd06815bc50 2024-05-08T05:42:51.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.003+0000 7fd0657fa640 1 --2- 172.21.15.45:0/1632527238 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd04c0762d0 0x7fd04c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:51.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.003+0000 7fd0657fa640 1 -- 172.21.15.45:0/1632527238 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5335+0+0 (secure 0 0 0) 0x7fd060097230 con 0x7fd06815bc50 2024-05-08T05:42:51.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.003+0000 7fd06ca0e640 1 --2- 172.21.15.45:0/1632527238 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd04c0762d0 0x7fd04c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:51.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.003+0000 7fd06ca0e640 1 --2- 172.21.15.45:0/1632527238 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd04c0762d0 0x7fd04c078790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd05c034000 tx=0x7fd05c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:51.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.011+0000 7fd0657fa640 1 -- 172.21.15.45:0/1632527238 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd0600606c0 con 0x7fd06815bc50 2024-05-08T05:42:51.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.280+0000 7fd06ec72640 1 -- 172.21.15.45:0/1632527238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7fd06815b240 con 0x7fd06815bc50 2024-05-08T05:42:51.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.526+0000 7fd0657fa640 1 -- 172.21.15.45:0/1632527238 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v91) ==== 142+0+0 (secure 0 0 0) 0x7fd0600649c0 con 0x7fd06815bc50 2024-05-08T05:42:51.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.529+0000 7fd06ec72640 1 -- 172.21.15.45:0/1632527238 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd04c0762d0 msgr2=0x7fd04c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.529+0000 7fd06ec72640 1 --2- 172.21.15.45:0/1632527238 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd04c0762d0 0x7fd04c078790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd05c034000 tx=0x7fd05c02e000 comp rx=0 tx=0).stop 2024-05-08T05:42:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.529+0000 7fd06ec72640 1 -- 172.21.15.45:0/1632527238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06815bc50 msgr2=0x7fd0680ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.529+0000 7fd06ec72640 1 --2- 172.21.15.45:0/1632527238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06815bc50 0x7fd0680ccdf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd06000c780 tx=0x7fd06000cc40 comp rx=0 tx=0).stop 2024-05-08T05:42:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.529+0000 7fd06ec72640 1 -- 172.21.15.45:0/1632527238 shutdown_connections 2024-05-08T05:42:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.529+0000 7fd06ec72640 1 --2- 172.21.15.45:0/1632527238 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd04c0762d0 0x7fd04c078790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.529+0000 7fd06ec72640 1 --2- 172.21.15.45:0/1632527238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06815bc50 0x7fd0680ccdf0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.529+0000 7fd06ec72640 1 --2- 172.21.15.45:0/1632527238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0680732c0 0x7fd0680cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.529+0000 7fd06ec72640 1 --2- 172.21.15.45:0/1632527238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd06806f540 0x7fd068152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.529+0000 7fd06ec72640 1 -- 172.21.15.45:0/1632527238 >> 172.21.15.45:0/1632527238 conn(0x7fd0680b3c40 msgr2=0x7fd06815e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.529+0000 7fd06ec72640 1 -- 172.21.15.45:0/1632527238 shutdown_connections 2024-05-08T05:42:51.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.529+0000 7fd06ec72640 1 -- 172.21.15.45:0/1632527238 wait complete. 2024-05-08T05:42:51.541 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-08T05:42:51.541 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-05-08T05:42:51.541 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-08T05:42:51.541 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:42:51.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.665+0000 7f3fa0e62640 1 Processor -- start 2024-05-08T05:42:51.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.665+0000 7f3fa0e62640 1 -- start start 2024-05-08T05:42:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.666+0000 7f3fa0e62640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f9c1524f0 0x7f3f9c0b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.666+0000 7f3fa0e62640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3f9c06eaa0 0x7f3f9c073700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.666+0000 7f3fa0e62640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f9c0b4590 con 0x7f3f9c15bc80 2024-05-08T05:42:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.666+0000 7f3fa0e62640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3f9c0b51a0 con 0x7f3f9c06eaa0 2024-05-08T05:42:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.666+0000 7f3fa0e62640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3f9c073450 con 0x7f3f9c1524f0 2024-05-08T05:42:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.666+0000 7f3f93fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3f9c15bc80 0x7f3f9c06e390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36582/0 (socket says 172.21.15.45:36582) 2024-05-08T05:42:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.666+0000 7f3f93fff640 1 -- 172.21.15.45:0/4095434238 learned_addr learned my addr 172.21.15.45:0/4095434238 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f9ad9d640 1 --2- 172.21.15.45:0/4095434238 >> v2:172.21.15.63:3300/0 conn(0x7f3f9c06eaa0 0x7f3f9c073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f9a59c640 1 --2- 172.21.15.45:0/4095434238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f9c1524f0 0x7f3f9c0b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:51.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f99d9b640 1 -- 172.21.15.45:0/4095434238 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 729232617 0 0) 0x7f3f9c0b4590 con 0x7f3f9c15bc80 2024-05-08T05:42:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f99d9b640 1 -- 172.21.15.45:0/4095434238 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f88003520 con 0x7f3f9c15bc80 2024-05-08T05:42:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f99d9b640 1 -- 172.21.15.45:0/4095434238 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1705206772 0 0) 0x7f3f88003520 con 0x7f3f9c15bc80 2024-05-08T05:42:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f99d9b640 1 -- 172.21.15.45:0/4095434238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f9c1524f0 msgr2=0x7f3f9c0b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f99d9b640 1 --2- 172.21.15.45:0/4095434238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f9c1524f0 0x7f3f9c0b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f99d9b640 1 -- 172.21.15.45:0/4095434238 >> v2:172.21.15.63:3300/0 conn(0x7f3f9c06eaa0 msgr2=0x7f3f9c073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f99d9b640 1 --2- 172.21.15.45:0/4095434238 >> v2:172.21.15.63:3300/0 conn(0x7f3f9c06eaa0 0x7f3f9c073700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f99d9b640 1 -- 172.21.15.45:0/4095434238 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9c0b4870 con 0x7f3f9c15bc80 2024-05-08T05:42:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f99d9b640 1 -- 172.21.15.45:0/4095434238 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f3f840016f0 con 0x7f3f9c15bc80 2024-05-08T05:42:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f99d9b640 1 -- 172.21.15.45:0/4095434238 >> v1:172.21.15.45:6789/0 conn(0x7f3f9c15bc80 legacy=0x7f3f9c06e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f99d9b640 1 --2- 172.21.15.45:0/4095434238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f88003a00 0x7f3f88003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f99d9b640 1 --2- 172.21.15.45:0/4095434238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f88004800 0x7f3f88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f99d9b640 1 --2- 172.21.15.45:0/4095434238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f9c1524f0 0x7f3f880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f99d9b640 1 -- 172.21.15.45:0/4095434238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c0b51a0 con 0x7f3f88004800 2024-05-08T05:42:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f99d9b640 1 -- 172.21.15.45:0/4095434238 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c073450 con 0x7f3f88003a00 2024-05-08T05:42:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f99d9b640 1 -- 172.21.15.45:0/4095434238 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3f8800cad0 con 0x7f3f9c1524f0 2024-05-08T05:42:51.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f9a59c640 1 --2- 172.21.15.45:0/4095434238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f88003a00 0x7f3f88003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f93fff640 1 --2- 172.21.15.45:0/4095434238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f88004800 0x7f3f88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f9ad9d640 1 --2- 172.21.15.45:0/4095434238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f9c1524f0 0x7f3f880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f93fff640 1 -- 172.21.15.45:0/4095434238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f9c1524f0 msgr2=0x7f3f880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f93fff640 1 --2- 172.21.15.45:0/4095434238 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f9c1524f0 0x7f3f880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f93fff640 1 -- 172.21.15.45:0/4095434238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f88003a00 msgr2=0x7f3f88003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f93fff640 1 --2- 172.21.15.45:0/4095434238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f88003a00 0x7f3f88003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.667+0000 7f3f93fff640 1 -- 172.21.15.45:0/4095434238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f88003520 con 0x7f3f88004800 2024-05-08T05:42:51.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.668+0000 7f3f93fff640 1 --2- 172.21.15.45:0/4095434238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f88004800 0x7f3f88004c80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3f84007d10 tx=0x7f3f88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91bf2ccc7531f9a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:51.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.668+0000 7f3f99d9b640 1 -- 172.21.15.45:0/4095434238 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84007d50 con 0x7f3f88004800 2024-05-08T05:42:51.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.668+0000 7f3f99d9b640 1 -- 172.21.15.45:0/4095434238 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f84036070 con 0x7f3f88004800 2024-05-08T05:42:51.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.668+0000 7f3f99d9b640 1 -- 172.21.15.45:0/4095434238 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84005d10 con 0x7f3f88004800 2024-05-08T05:42:51.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.669+0000 7f3fa0e62640 1 -- 172.21.15.45:0/4095434238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f88004800 msgr2=0x7f3f88004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:51.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.669+0000 7f3fa0e62640 1 --2- 172.21.15.45:0/4095434238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f88004800 0x7f3f88004c80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3f84007d10 tx=0x7f3f88005830 comp rx=0 tx=0).stop 2024-05-08T05:42:51.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.669+0000 7f3f9ad9d640 1 -- 172.21.15.45:0/4095434238 reap_dead start 2024-05-08T05:42:51.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.669+0000 7f3fa0e62640 1 -- 172.21.15.45:0/4095434238 shutdown_connections 2024-05-08T05:42:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.669+0000 7f3fa0e62640 1 -- 172.21.15.45:0/4095434238 >> 172.21.15.45:0/4095434238 conn(0x7f3f9c06c3a0 msgr2=0x7f3f9c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.669+0000 7f3fa0e62640 1 -- 172.21.15.45:0/4095434238 shutdown_connections 2024-05-08T05:42:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.669+0000 7f3fa0e62640 1 -- 172.21.15.45:0/4095434238 wait complete. 2024-05-08T05:42:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.670+0000 7f3fa0e62640 1 Processor -- start 2024-05-08T05:42:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.671+0000 7f3fa0e62640 1 -- start start 2024-05-08T05:42:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.671+0000 7f3fa0e62640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f88004800 0x7f3f9c14a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.671+0000 7f3fa0e62640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f9c156160 0x7f3f9c0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.671+0000 7f3fa0e62640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f9c0b3340 0x7f3f9c15de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.671+0000 7f3fa0e62640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c1528d0 con 0x7f3f9c0b3340 2024-05-08T05:42:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.671+0000 7f3fa0e62640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c0b4e30 con 0x7f3f88004800 2024-05-08T05:42:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.671+0000 7f3fa0e62640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3f9c0b5630 con 0x7f3f9c156160 2024-05-08T05:42:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.671+0000 7f3f9ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f9c0b3340 0x7f3f9c15de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.671+0000 7f3f9ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f9c0b3340 0x7f3f9c15de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50570/0 (socket says 172.21.15.45:50570) 2024-05-08T05:42:51.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.671+0000 7f3f9ad9d640 1 -- 172.21.15.45:0/1381525926 learned_addr learned my addr 172.21.15.45:0/1381525926 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.672+0000 7f3f9a59c640 1 --2- 172.21.15.45:0/1381525926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f88004800 0x7f3f9c14a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.672+0000 7f3f93fff640 1 --2- 172.21.15.45:0/1381525926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f9c156160 0x7f3f9c0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.672+0000 7f3f9ad9d640 1 -- 172.21.15.45:0/1381525926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f9c156160 msgr2=0x7f3f9c0b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.672+0000 7f3f9ad9d640 1 --2- 172.21.15.45:0/1381525926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f9c156160 0x7f3f9c0b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.672+0000 7f3f9ad9d640 1 -- 172.21.15.45:0/1381525926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f88004800 msgr2=0x7f3f9c14a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.672+0000 7f3f9ad9d640 1 --2- 172.21.15.45:0/1381525926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f88004800 0x7f3f9c14a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.672+0000 7f3f9ad9d640 1 -- 172.21.15.45:0/1381525926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9c0ca1f0 con 0x7f3f9c0b3340 2024-05-08T05:42:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.672+0000 7f3f9a59c640 1 --2- 172.21.15.45:0/1381525926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f88004800 0x7f3f9c14a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.672+0000 7f3f93fff640 1 --2- 172.21.15.45:0/1381525926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f9c156160 0x7f3f9c0b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.672+0000 7f3f9ad9d640 1 --2- 172.21.15.45:0/1381525926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f9c0b3340 0x7f3f9c15de90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0b5fc0 tx=0x7f3f9400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.672+0000 7f3f92ffd640 1 -- 172.21.15.45:0/1381525926 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94003990 con 0x7f3f9c0b3340 2024-05-08T05:42:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.672+0000 7f3f92ffd640 1 -- 172.21.15.45:0/1381525926 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f9401c070 con 0x7f3f9c0b3340 2024-05-08T05:42:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.672+0000 7f3f92ffd640 1 -- 172.21.15.45:0/1381525926 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94003d70 con 0x7f3f9c0b3340 2024-05-08T05:42:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.672+0000 7f3fa0e62640 1 -- 172.21.15.45:0/1381525926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f9c0ca4d0 con 0x7f3f9c0b3340 2024-05-08T05:42:51.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.672+0000 7f3fa0e62640 1 -- 172.21.15.45:0/1381525926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f9c0caa90 con 0x7f3f9c0b3340 2024-05-08T05:42:51.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.673+0000 7f3fa0e62640 1 -- 172.21.15.45:0/1381525926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f9c0cac90 con 0x7f3f9c0b3340 2024-05-08T05:42:51.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.674+0000 7f3f92ffd640 1 -- 172.21.15.45:0/1381525926 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3f94004030 con 0x7f3f9c0b3340 2024-05-08T05:42:51.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.675+0000 7f3f92ffd640 1 --2- 172.21.15.45:0/1381525926 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f7c0762d0 0x7f3f7c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:51.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.675+0000 7f3f92ffd640 1 -- 172.21.15.45:0/1381525926 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5335+0+0 (secure 0 0 0) 0x7f3f94097e20 con 0x7f3f9c0b3340 2024-05-08T05:42:51.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.675+0000 7f3f9a59c640 1 --2- 172.21.15.45:0/1381525926 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f7c0762d0 0x7f3f7c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:51.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.675+0000 7f3f9a59c640 1 --2- 172.21.15.45:0/1381525926 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f7c0762d0 0x7f3f7c078790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009a60 tx=0x7f3f8c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:51.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.679+0000 7f3f92ffd640 1 -- 172.21.15.45:0/1381525926 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3f9c0cac90 con 0x7f3f9c0b3340 2024-05-08T05:42:51.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.873+0000 7f3fa0e62640 1 -- 172.21.15.45:0/1381525926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f9c0693f0 con 0x7f3f9c0b3340 2024-05-08T05:42:51.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.874+0000 7f3f92ffd640 1 -- 172.21.15.45:0/1381525926 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+13243 (secure 0 0 0) 0x7f3f94010040 con 0x7f3f9c0b3340 2024-05-08T05:42:51.874 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:51.874 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":39,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:42:50.521737+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/173436628":"2024-05-09T05:42:47.513721+0000","172.21.15.45:6833/173436628":"2024-05-09T05:42:47.513721+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:42:51.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.876+0000 7f3fa0e62640 1 -- 172.21.15.45:0/1381525926 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f7c0762d0 msgr2=0x7f3f7c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:51.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.876+0000 7f3fa0e62640 1 --2- 172.21.15.45:0/1381525926 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f7c0762d0 0x7f3f7c078790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009a60 tx=0x7f3f8c008040 comp rx=0 tx=0).stop 2024-05-08T05:42:51.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.876+0000 7f3fa0e62640 1 -- 172.21.15.45:0/1381525926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f9c0b3340 msgr2=0x7f3f9c15de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:51.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.876+0000 7f3fa0e62640 1 --2- 172.21.15.45:0/1381525926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f9c0b3340 0x7f3f9c15de90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c0b5fc0 tx=0x7f3f9400f580 comp rx=0 tx=0).stop 2024-05-08T05:42:51.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.876+0000 7f3fa0e62640 1 -- 172.21.15.45:0/1381525926 shutdown_connections 2024-05-08T05:42:51.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.876+0000 7f3fa0e62640 1 --2- 172.21.15.45:0/1381525926 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f7c0762d0 0x7f3f7c078790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.876+0000 7f3fa0e62640 1 --2- 172.21.15.45:0/1381525926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f9c156160 0x7f3f9c0b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.876+0000 7f3fa0e62640 1 --2- 172.21.15.45:0/1381525926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f9c0b3340 0x7f3f9c15de90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.876+0000 7f3fa0e62640 1 --2- 172.21.15.45:0/1381525926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f88004800 0x7f3f9c14a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.876+0000 7f3fa0e62640 1 -- 172.21.15.45:0/1381525926 >> 172.21.15.45:0/1381525926 conn(0x7f3f9c06c3a0 msgr2=0x7f3f9c0b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:51.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.876+0000 7f3fa0e62640 1 -- 172.21.15.45:0/1381525926 shutdown_connections 2024-05-08T05:42:51.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.876+0000 7f3fa0e62640 1 -- 172.21.15.45:0/1381525926 wait complete. 2024-05-08T05:42:51.887 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T05:41:18.847604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T05:42:51.888 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T05:42:51.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.994+0000 7fd73c42a640 1 Processor -- start 2024-05-08T05:42:51.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.994+0000 7fd73c42a640 1 -- start start 2024-05-08T05:42:51.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.995+0000 7fd73c42a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7340d33f0 0x7fd73415d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:51.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.995+0000 7fd73c42a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7340718c0 con 0x7fd734070b90 2024-05-08T05:42:51.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.995+0000 7fd73c42a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd734071ba0 con 0x7fd73415ccb0 2024-05-08T05:42:51.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.995+0000 7fd73c42a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd7340afba0 con 0x7fd7340d33f0 2024-05-08T05:42:51.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.996+0000 7fd73a9c7640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd734070b90 0x7fd734071020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36592/0 (socket says 172.21.15.45:36592) 2024-05-08T05:42:51.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.996+0000 7fd73a9c7640 1 -- 172.21.15.45:0/2322356205 learned_addr learned my addr 172.21.15.45:0/2322356205 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:51.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.996+0000 7fd7399c5640 1 --2- 172.21.15.45:0/2322356205 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7340d33f0 0x7fd73415d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:51.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.996+0000 7fd7391c4640 1 -- 172.21.15.45:0/2322356205 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1459504935 0 0) 0x7fd7340718c0 con 0x7fd734070b90 2024-05-08T05:42:51.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.996+0000 7fd7391c4640 1 -- 172.21.15.45:0/2322356205 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd720003520 con 0x7fd734070b90 2024-05-08T05:42:51.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.996+0000 7fd7391c4640 1 -- 172.21.15.45:0/2322356205 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3107285121 0 0) 0x7fd734071ba0 con 0x7fd73415ccb0 2024-05-08T05:42:51.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.996+0000 7fd7391c4640 1 -- 172.21.15.45:0/2322356205 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7340718c0 con 0x7fd73415ccb0 2024-05-08T05:42:51.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.996+0000 7fd7399c5640 1 -- 172.21.15.45:0/2322356205 >> v1:172.21.15.63:6789/0 conn(0x7fd73415ccb0 legacy=0x7fd7340d2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:51.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.996+0000 7fd7399c5640 1 -- 172.21.15.45:0/2322356205 >> v1:172.21.15.45:6789/0 conn(0x7fd734070b90 legacy=0x7fd734071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:51.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.996+0000 7fd7399c5640 1 -- 172.21.15.45:0/2322356205 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd734071e80 con 0x7fd7340d33f0 2024-05-08T05:42:51.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.997+0000 7fd7399c5640 1 --2- 172.21.15.45:0/2322356205 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7340d33f0 0x7fd73415d090 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd7240097d0 tx=0x7fd72402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db4ad7a8300101a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:51.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.997+0000 7fd7391c4640 1 -- 172.21.15.45:0/2322356205 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724037070 con 0x7fd7340d33f0 2024-05-08T05:42:51.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.998+0000 7fd7391c4640 1 -- 172.21.15.45:0/2322356205 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd724032440 con 0x7fd7340d33f0 2024-05-08T05:42:51.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.998+0000 7fd7391c4640 1 -- 172.21.15.45:0/2322356205 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7240327f0 con 0x7fd7340d33f0 2024-05-08T05:42:51.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.998+0000 7fd73c42a640 1 -- 172.21.15.45:0/2322356205 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7340d33f0 msgr2=0x7fd73415d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:51.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.998+0000 7fd73c42a640 1 --2- 172.21.15.45:0/2322356205 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7340d33f0 0x7fd73415d090 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd7240097d0 tx=0x7fd72402a180 comp rx=0 tx=0).stop 2024-05-08T05:42:51.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.998+0000 7fd73c42a640 1 -- 172.21.15.45:0/2322356205 shutdown_connections 2024-05-08T05:42:51.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.998+0000 7fd73c42a640 1 --2- 172.21.15.45:0/2322356205 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7340d33f0 0x7fd73415d090 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:51.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.998+0000 7fd73c42a640 1 -- 172.21.15.45:0/2322356205 >> 172.21.15.45:0/2322356205 conn(0x7fd73409ba40 msgr2=0x7fd73409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:51.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.998+0000 7fd73c42a640 1 -- 172.21.15.45:0/2322356205 shutdown_connections 2024-05-08T05:42:51.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.998+0000 7fd73c42a640 1 -- 172.21.15.45:0/2322356205 wait complete. 2024-05-08T05:42:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.999+0000 7fd73c42a640 1 Processor -- start 2024-05-08T05:42:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:51.999+0000 7fd73c42a640 1 -- start start 2024-05-08T05:42:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.000+0000 7fd73c42a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd734070b90 0x7fd73406f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.000+0000 7fd73c42a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7340d33f0 0x7fd734157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.000+0000 7fd73c42a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd73415ccb0 0x7fd7340c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.000+0000 7fd73c42a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd7340b11c0 con 0x7fd7340d33f0 2024-05-08T05:42:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.000+0000 7fd73c42a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd7340b1040 con 0x7fd734070b90 2024-05-08T05:42:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.000+0000 7fd73c42a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd7340b1340 con 0x7fd73415ccb0 2024-05-08T05:42:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.000+0000 7fd7399c5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7340d33f0 0x7fd734157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.000+0000 7fd7399c5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7340d33f0 0x7fd734157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50580/0 (socket says 172.21.15.45:50580) 2024-05-08T05:42:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.000+0000 7fd7399c5640 1 -- 172.21.15.45:0/2714083626 learned_addr learned my addr 172.21.15.45:0/2714083626 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:52.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.000+0000 7fd73a9c7640 1 --2- 172.21.15.45:0/2714083626 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd73415ccb0 0x7fd7340c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.000+0000 7fd73a1c6640 1 --2- 172.21.15.45:0/2714083626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd734070b90 0x7fd73406f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.001+0000 7fd7399c5640 1 -- 172.21.15.45:0/2714083626 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd73415ccb0 msgr2=0x7fd7340c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.001+0000 7fd7399c5640 1 --2- 172.21.15.45:0/2714083626 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd73415ccb0 0x7fd7340c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.001+0000 7fd7399c5640 1 -- 172.21.15.45:0/2714083626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd734070b90 msgr2=0x7fd73406f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.001+0000 7fd7399c5640 1 --2- 172.21.15.45:0/2714083626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd734070b90 0x7fd73406f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.001+0000 7fd7399c5640 1 -- 172.21.15.45:0/2714083626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7340c80f0 con 0x7fd7340d33f0 2024-05-08T05:42:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.001+0000 7fd7399c5640 1 --2- 172.21.15.45:0/2714083626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7340d33f0 0x7fd734157310 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd724035040 tx=0x7fd7240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.001+0000 7fd72affd640 1 -- 172.21.15.45:0/2714083626 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724037040 con 0x7fd7340d33f0 2024-05-08T05:42:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.001+0000 7fd72affd640 1 -- 172.21.15.45:0/2714083626 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd724042070 con 0x7fd7340d33f0 2024-05-08T05:42:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.001+0000 7fd73c42a640 1 -- 172.21.15.45:0/2714083626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7340c83d0 con 0x7fd7340d33f0 2024-05-08T05:42:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.001+0000 7fd72affd640 1 -- 172.21.15.45:0/2714083626 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724032810 con 0x7fd7340d33f0 2024-05-08T05:42:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.001+0000 7fd73c42a640 1 -- 172.21.15.45:0/2714083626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7340c8990 con 0x7fd7340d33f0 2024-05-08T05:42:52.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.002+0000 7fd73c42a640 1 -- 172.21.15.45:0/2714083626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd734069b00 con 0x7fd7340d33f0 2024-05-08T05:42:52.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.003+0000 7fd72affd640 1 -- 172.21.15.45:0/2714083626 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd7240329b0 con 0x7fd7340d33f0 2024-05-08T05:42:52.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.004+0000 7fd72affd640 1 --2- 172.21.15.45:0/2714083626 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7180762d0 0x7fd718078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:52.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.004+0000 7fd72affd640 1 -- 172.21.15.45:0/2714083626 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5335+0+0 (secure 0 0 0) 0x7fd7240b7bd0 con 0x7fd7340d33f0 2024-05-08T05:42:52.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.004+0000 7fd73a1c6640 1 --2- 172.21.15.45:0/2714083626 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7180762d0 0x7fd718078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:52.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.005+0000 7fd73a1c6640 1 --2- 172.21.15.45:0/2714083626 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7180762d0 0x7fd718078790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd730001810 tx=0x7fd730009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:52.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.008+0000 7fd72affd640 1 -- 172.21.15.45:0/2714083626 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd724080fc0 con 0x7fd7340d33f0 2024-05-08T05:42:52.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.200+0000 7fd73c42a640 1 -- 172.21.15.45:0/2714083626 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd734071300 con 0x7fd7340d33f0 2024-05-08T05:42:52.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.201+0000 7fd72affd640 1 -- 172.21.15.45:0/2714083626 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) ==== 93+0+10 (secure 0 0 0) 0x7fd7240852c0 con 0x7fd7340d33f0 2024-05-08T05:42:52.201 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-05-08T05:42:52.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.202+0000 7fd73c42a640 1 -- 172.21.15.45:0/2714083626 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7180762d0 msgr2=0x7fd718078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:52.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.202+0000 7fd73c42a640 1 --2- 172.21.15.45:0/2714083626 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7180762d0 0x7fd718078790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd730001810 tx=0x7fd730009040 comp rx=0 tx=0).stop 2024-05-08T05:42:52.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.202+0000 7fd73c42a640 1 -- 172.21.15.45:0/2714083626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7340d33f0 msgr2=0x7fd734157310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:52.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.202+0000 7fd73c42a640 1 --2- 172.21.15.45:0/2714083626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7340d33f0 0x7fd734157310 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd724035040 tx=0x7fd7240046c0 comp rx=0 tx=0).stop 2024-05-08T05:42:52.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.203+0000 7fd73c42a640 1 -- 172.21.15.45:0/2714083626 shutdown_connections 2024-05-08T05:42:52.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.203+0000 7fd73c42a640 1 --2- 172.21.15.45:0/2714083626 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7180762d0 0x7fd718078790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:52.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.203+0000 7fd73c42a640 1 --2- 172.21.15.45:0/2714083626 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd73415ccb0 0x7fd7340c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:52.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.203+0000 7fd73c42a640 1 --2- 172.21.15.45:0/2714083626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7340d33f0 0x7fd734157310 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:52.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.203+0000 7fd73c42a640 1 --2- 172.21.15.45:0/2714083626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd734070b90 0x7fd73406f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:52.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.203+0000 7fd73c42a640 1 -- 172.21.15.45:0/2714083626 >> 172.21.15.45:0/2714083626 conn(0x7fd73409ba40 msgr2=0x7fd7340aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:52.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.203+0000 7fd73c42a640 1 -- 172.21.15.45:0/2714083626 shutdown_connections 2024-05-08T05:42:52.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.203+0000 7fd73c42a640 1 -- 172.21.15.45:0/2714083626 wait complete. 2024-05-08T05:42:52.214 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-05-08T05:42:52.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.317+0000 7f9e84862640 1 Processor -- start 2024-05-08T05:42:52.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.317+0000 7f9e84862640 1 -- start start 2024-05-08T05:42:52.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.318+0000 7f9e84862640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9e800d39b0 0x7f9e8015e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:52.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.318+0000 7f9e84862640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e800cd9a0 0x7f9e800cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:52.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.318+0000 7f9e84862640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e80160d10 con 0x7f9e800cf8a0 2024-05-08T05:42:52.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.318+0000 7f9e84862640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9e8015d5f0 con 0x7f9e800d39b0 2024-05-08T05:42:52.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.318+0000 7f9e84862640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9e80161920 con 0x7f9e800cd9a0 2024-05-08T05:42:52.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.318+0000 7f9e7e59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9e800cf8a0 0x7f9e800cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36594/0 (socket says 172.21.15.45:36594) 2024-05-08T05:42:52.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.318+0000 7f9e7e59c640 1 -- 172.21.15.45:0/422624060 learned_addr learned my addr 172.21.15.45:0/422624060 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:52.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.318+0000 7f9e7dd9b640 1 --2- 172.21.15.45:0/422624060 >> v2:172.21.15.63:3300/0 conn(0x7f9e800d39b0 0x7f9e8015e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:52.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.318+0000 7f9e7ed9d640 1 --2- 172.21.15.45:0/422624060 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e800cd9a0 0x7f9e800cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:52.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.318+0000 7f9e7d59a640 1 -- 172.21.15.45:0/422624060 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1479997076 0 0) 0x7f9e80160d10 con 0x7f9e800cf8a0 2024-05-08T05:42:52.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7d59a640 1 -- 172.21.15.45:0/422624060 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e6c003520 con 0x7f9e800cf8a0 2024-05-08T05:42:52.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7d59a640 1 -- 172.21.15.45:0/422624060 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 610094018 0 0) 0x7f9e6c003520 con 0x7f9e800cf8a0 2024-05-08T05:42:52.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7d59a640 1 -- 172.21.15.45:0/422624060 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e800cd9a0 msgr2=0x7f9e800cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:52.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7d59a640 1 --2- 172.21.15.45:0/422624060 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e800cd9a0 0x7f9e800cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:52.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7d59a640 1 -- 172.21.15.45:0/422624060 >> v2:172.21.15.63:3300/0 conn(0x7f9e800d39b0 msgr2=0x7f9e8015e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:52.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7d59a640 1 --2- 172.21.15.45:0/422624060 >> v2:172.21.15.63:3300/0 conn(0x7f9e800d39b0 0x7f9e8015e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:52.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7d59a640 1 -- 172.21.15.45:0/422624060 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e80160ff0 con 0x7f9e800cf8a0 2024-05-08T05:42:52.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7d59a640 1 -- 172.21.15.45:0/422624060 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f9e700016f0 con 0x7f9e800cf8a0 2024-05-08T05:42:52.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7dd9b640 1 --2- 172.21.15.45:0/422624060 >> v2:172.21.15.63:3300/0 conn(0x7f9e800d39b0 0x7f9e8015e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:42:52.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7d59a640 1 -- 172.21.15.45:0/422624060 >> v1:172.21.15.45:6789/0 conn(0x7f9e800cf8a0 legacy=0x7f9e800cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:52.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7d59a640 1 --2- 172.21.15.45:0/422624060 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e6c003a00 0x7f9e6c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:52.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7d59a640 1 --2- 172.21.15.45:0/422624060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e6c004800 0x7f9e6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:52.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7d59a640 1 --2- 172.21.15.45:0/422624060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e6c0051c0 0x7f9e6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:52.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7d59a640 1 -- 172.21.15.45:0/422624060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8015d5f0 con 0x7f9e6c004800 2024-05-08T05:42:52.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7d59a640 1 -- 172.21.15.45:0/422624060 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6c00bf00 con 0x7f9e6c0051c0 2024-05-08T05:42:52.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7d59a640 1 -- 172.21.15.45:0/422624060 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9e6c00c0d0 con 0x7f9e6c003a00 2024-05-08T05:42:52.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7ed9d640 1 --2- 172.21.15.45:0/422624060 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e800cd9a0 0x7f9e800cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:42:52.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7dd9b640 1 --2- 172.21.15.45:0/422624060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e6c004800 0x7f9e6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7e59c640 1 --2- 172.21.15.45:0/422624060 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e6c003a00 0x7f9e6c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7ed9d640 1 --2- 172.21.15.45:0/422624060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e6c0051c0 0x7f9e6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7dd9b640 1 -- 172.21.15.45:0/422624060 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e6c003a00 msgr2=0x7f9e6c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7dd9b640 1 --2- 172.21.15.45:0/422624060 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e6c003a00 0x7f9e6c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7dd9b640 1 -- 172.21.15.45:0/422624060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e6c0051c0 msgr2=0x7f9e6c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7dd9b640 1 --2- 172.21.15.45:0/422624060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e6c0051c0 0x7f9e6c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.319+0000 7f9e7dd9b640 1 -- 172.21.15.45:0/422624060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6c003520 con 0x7f9e6c004800 2024-05-08T05:42:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.320+0000 7f9e7dd9b640 1 --2- 172.21.15.45:0/422624060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e6c004800 0x7f9e6c004c80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009cc0 tx=0x7f9e6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498c7a94a556be77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.320+0000 7f9e7d59a640 1 -- 172.21.15.45:0/422624060 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68036070 con 0x7f9e6c004800 2024-05-08T05:42:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.320+0000 7f9e7d59a640 1 -- 172.21.15.45:0/422624060 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e68031400 con 0x7f9e6c004800 2024-05-08T05:42:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.320+0000 7f9e7d59a640 1 -- 172.21.15.45:0/422624060 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e68009860 con 0x7f9e6c004800 2024-05-08T05:42:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.321+0000 7f9e84862640 1 -- 172.21.15.45:0/422624060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e6c004800 msgr2=0x7f9e6c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.321+0000 7f9e84862640 1 --2- 172.21.15.45:0/422624060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e6c004800 0x7f9e6c004c80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009cc0 tx=0x7f9e6c005830 comp rx=0 tx=0).stop 2024-05-08T05:42:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.321+0000 7f9e7ed9d640 1 -- 172.21.15.45:0/422624060 reap_dead start 2024-05-08T05:42:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.321+0000 7f9e84862640 1 -- 172.21.15.45:0/422624060 shutdown_connections 2024-05-08T05:42:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.321+0000 7f9e84862640 1 -- 172.21.15.45:0/422624060 >> 172.21.15.45:0/422624060 conn(0x7f9e800d2e90 msgr2=0x7f9e800d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.321+0000 7f9e84862640 1 -- 172.21.15.45:0/422624060 shutdown_connections 2024-05-08T05:42:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.321+0000 7f9e84862640 1 -- 172.21.15.45:0/422624060 wait complete. 2024-05-08T05:42:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.322+0000 7f9e84862640 1 Processor -- start 2024-05-08T05:42:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.322+0000 7f9e84862640 1 -- start start 2024-05-08T05:42:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.323+0000 7f9e84862640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e6c004800 0x7f9e800b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.323+0000 7f9e84862640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e801555e0 0x7f9e800b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.323+0000 7f9e84862640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e800b2e70 0x7f9e800b3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.323+0000 7f9e84862640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9e801615b0 con 0x7f9e6c004800 2024-05-08T05:42:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.323+0000 7f9e84862640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9e800d0700 con 0x7f9e800b2e70 2024-05-08T05:42:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.323+0000 7f9e84862640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9e800cde30 con 0x7f9e801555e0 2024-05-08T05:42:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.323+0000 7f9e7e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e6c004800 0x7f9e800b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.323+0000 7f9e7e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e6c004800 0x7f9e800b0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50588/0 (socket says 172.21.15.45:50588) 2024-05-08T05:42:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.323+0000 7f9e7e59c640 1 -- 172.21.15.45:0/3103540247 learned_addr learned my addr 172.21.15.45:0/3103540247 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.323+0000 7f9e7e59c640 1 -- 172.21.15.45:0/3103540247 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e801555e0 msgr2=0x7f9e800b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.323+0000 7f9e7e59c640 1 --2- 172.21.15.45:0/3103540247 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e801555e0 0x7f9e800b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.323+0000 7f9e7e59c640 1 -- 172.21.15.45:0/3103540247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e800b2e70 msgr2=0x7f9e800b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.323+0000 7f9e7e59c640 1 --2- 172.21.15.45:0/3103540247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e800b2e70 0x7f9e800b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.323+0000 7f9e7e59c640 1 -- 172.21.15.45:0/3103540247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e801f4890 con 0x7f9e6c004800 2024-05-08T05:42:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.324+0000 7f9e7e59c640 1 --2- 172.21.15.45:0/3103540247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e6c004800 0x7f9e800b0c70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9e70007340 tx=0x7f9e70007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.324+0000 7f9e76ffd640 1 -- 172.21.15.45:0/3103540247 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7000f7e0 con 0x7f9e6c004800 2024-05-08T05:42:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.324+0000 7f9e76ffd640 1 -- 172.21.15.45:0/3103540247 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e70018070 con 0x7f9e6c004800 2024-05-08T05:42:52.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.324+0000 7f9e84862640 1 -- 172.21.15.45:0/3103540247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e800b3bc0 con 0x7f9e6c004800 2024-05-08T05:42:52.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.324+0000 7f9e76ffd640 1 -- 172.21.15.45:0/3103540247 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7000fbe0 con 0x7f9e6c004800 2024-05-08T05:42:52.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.324+0000 7f9e84862640 1 -- 172.21.15.45:0/3103540247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e800b4180 con 0x7f9e6c004800 2024-05-08T05:42:52.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.325+0000 7f9e84862640 1 -- 172.21.15.45:0/3103540247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e800693f0 con 0x7f9e6c004800 2024-05-08T05:42:52.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.325+0000 7f9e76ffd640 1 -- 172.21.15.45:0/3103540247 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9e7001d460 con 0x7f9e6c004800 2024-05-08T05:42:52.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.326+0000 7f9e76ffd640 1 --2- 172.21.15.45:0/3103540247 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9e60076300 0x7f9e600787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:52.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.326+0000 7f9e76ffd640 1 -- 172.21.15.45:0/3103540247 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5335+0+0 (secure 0 0 0) 0x7f9e70097430 con 0x7f9e6c004800 2024-05-08T05:42:52.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.326+0000 7f9e7dd9b640 1 --2- 172.21.15.45:0/3103540247 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9e60076300 0x7f9e600787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:52.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.327+0000 7f9e7dd9b640 1 --2- 172.21.15.45:0/3103540247 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9e60076300 0x7f9e600787c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9e68033000 tx=0x7f9e6802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:52.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.330+0000 7f9e76ffd640 1 -- 172.21.15.45:0/3103540247 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9e70060f00 con 0x7f9e6c004800 2024-05-08T05:42:52.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:52.598+0000 7f9e84862640 1 -- 172.21.15.45:0/3103540247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f9e8006d140 con 0x7f9e6c004800 2024-05-08T05:42:53.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.532+0000 7f9e76ffd640 1 -- 172.21.15.45:0/3103540247 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v93) ==== 152+0+0 (secure 0 0 0) 0x7f9e70060800 con 0x7f9e6c004800 2024-05-08T05:42:53.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.535+0000 7f9e84862640 1 -- 172.21.15.45:0/3103540247 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9e60076300 msgr2=0x7f9e600787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:53.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.535+0000 7f9e84862640 1 --2- 172.21.15.45:0/3103540247 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9e60076300 0x7f9e600787c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9e68033000 tx=0x7f9e6802e000 comp rx=0 tx=0).stop 2024-05-08T05:42:53.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.535+0000 7f9e84862640 1 -- 172.21.15.45:0/3103540247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e6c004800 msgr2=0x7f9e800b0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:53.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.535+0000 7f9e84862640 1 --2- 172.21.15.45:0/3103540247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e6c004800 0x7f9e800b0c70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9e70007340 tx=0x7f9e70007800 comp rx=0 tx=0).stop 2024-05-08T05:42:53.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.535+0000 7f9e84862640 1 -- 172.21.15.45:0/3103540247 shutdown_connections 2024-05-08T05:42:53.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.535+0000 7f9e84862640 1 --2- 172.21.15.45:0/3103540247 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9e60076300 0x7f9e600787c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:53.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.535+0000 7f9e84862640 1 --2- 172.21.15.45:0/3103540247 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e801555e0 0x7f9e800b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:53.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.535+0000 7f9e84862640 1 --2- 172.21.15.45:0/3103540247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e800b2e70 0x7f9e800b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:53.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.535+0000 7f9e84862640 1 --2- 172.21.15.45:0/3103540247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e6c004800 0x7f9e800b0c70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:53.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.535+0000 7f9e84862640 1 -- 172.21.15.45:0/3103540247 >> 172.21.15.45:0/3103540247 conn(0x7f9e800d2e90 msgr2=0x7f9e8015e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:53.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.535+0000 7f9e84862640 1 -- 172.21.15.45:0/3103540247 shutdown_connections 2024-05-08T05:42:53.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.535+0000 7f9e84862640 1 -- 172.21.15.45:0/3103540247 wait complete. 2024-05-08T05:42:53.549 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T05:42:53.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.664+0000 7f0b77608640 1 Processor -- start 2024-05-08T05:42:53.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.664+0000 7f0b77608640 1 -- start start 2024-05-08T05:42:53.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.665+0000 7f0b77608640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b7015c340 0x7f0b70166970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:53.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.665+0000 7f0b77608640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b7015c920 con 0x7f0b7006e0f0 2024-05-08T05:42:53.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.665+0000 7f0b77608640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b7015cc00 con 0x7f0b70072840 2024-05-08T05:42:53.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.665+0000 7f0b77608640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7015cee0 con 0x7f0b7015c340 2024-05-08T05:42:53.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.665+0000 7f0b753a4640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0b7006e0f0 0x7f0b700d0f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36604/0 (socket says 172.21.15.45:36604) 2024-05-08T05:42:53.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.665+0000 7f0b753a4640 1 -- 172.21.15.45:0/775010098 learned_addr learned my addr 172.21.15.45:0/775010098 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:53.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b75ba5640 1 --2- 172.21.15.45:0/775010098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b7015c340 0x7f0b70166970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:53.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b67fff640 1 -- 172.21.15.45:0/775010098 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 407065417 0 0) 0x7f0b7015c920 con 0x7f0b7006e0f0 2024-05-08T05:42:53.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b67fff640 1 -- 172.21.15.45:0/775010098 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b5c003520 con 0x7f0b7006e0f0 2024-05-08T05:42:53.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b67fff640 1 -- 172.21.15.45:0/775010098 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 470780267 0 0) 0x7f0b7015cc00 con 0x7f0b70072840 2024-05-08T05:42:53.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b67fff640 1 -- 172.21.15.45:0/775010098 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b7015c920 con 0x7f0b70072840 2024-05-08T05:42:53.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b67fff640 1 -- 172.21.15.45:0/775010098 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1371637013 0 0) 0x7f0b5c003520 con 0x7f0b7006e0f0 2024-05-08T05:42:53.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b67fff640 1 -- 172.21.15.45:0/775010098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b7015c340 msgr2=0x7f0b70166970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:53.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b67fff640 1 --2- 172.21.15.45:0/775010098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b7015c340 0x7f0b70166970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:53.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b67fff640 1 -- 172.21.15.45:0/775010098 >> v1:172.21.15.63:6789/0 conn(0x7f0b70072840 legacy=0x7f0b7006e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:53.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b67fff640 1 -- 172.21.15.45:0/775010098 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7015d1c0 con 0x7f0b7006e0f0 2024-05-08T05:42:53.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b67fff640 1 -- 172.21.15.45:0/775010098 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f0b6c0016f0 con 0x7f0b7006e0f0 2024-05-08T05:42:53.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b67fff640 1 -- 172.21.15.45:0/775010098 >> v1:172.21.15.45:6789/0 conn(0x7f0b7006e0f0 legacy=0x7f0b700d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:53.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b67fff640 1 --2- 172.21.15.45:0/775010098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b5c003c60 0x7f0b5c004060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:53.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b67fff640 1 --2- 172.21.15.45:0/775010098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b5c004800 0x7f0b5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:53.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b67fff640 1 --2- 172.21.15.45:0/775010098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b7015c340 0x7f0b5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:53.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b67fff640 1 -- 172.21.15.45:0/775010098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7015cee0 con 0x7f0b5c004800 2024-05-08T05:42:53.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b67fff640 1 -- 172.21.15.45:0/775010098 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5c005540 con 0x7f0b5c003c60 2024-05-08T05:42:53.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b67fff640 1 -- 172.21.15.45:0/775010098 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0b5c00bf00 con 0x7f0b7015c340 2024-05-08T05:42:53.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b74ba3640 1 --2- 172.21.15.45:0/775010098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b5c004800 0x7f0b5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:53.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b753a4640 1 --2- 172.21.15.45:0/775010098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b5c003c60 0x7f0b5c004060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:53.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b75ba5640 1 --2- 172.21.15.45:0/775010098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b7015c340 0x7f0b5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:53.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b74ba3640 1 -- 172.21.15.45:0/775010098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b7015c340 msgr2=0x7f0b5c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:53.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b74ba3640 1 --2- 172.21.15.45:0/775010098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b7015c340 0x7f0b5c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:53.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b74ba3640 1 -- 172.21.15.45:0/775010098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b5c003c60 msgr2=0x7f0b5c004060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:53.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b74ba3640 1 --2- 172.21.15.45:0/775010098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b5c003c60 0x7f0b5c004060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:53.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.666+0000 7f0b74ba3640 1 -- 172.21.15.45:0/775010098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5c003520 con 0x7f0b5c004800 2024-05-08T05:42:53.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.667+0000 7f0b74ba3640 1 --2- 172.21.15.45:0/775010098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b5c004800 0x7f0b5c004c80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0b600018d0 tx=0x7f0b5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a735858138cc52fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:53.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.667+0000 7f0b67fff640 1 -- 172.21.15.45:0/775010098 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b60001520 con 0x7f0b5c004800 2024-05-08T05:42:53.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.667+0000 7f0b67fff640 1 -- 172.21.15.45:0/775010098 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b6002acb0 con 0x7f0b5c004800 2024-05-08T05:42:53.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.667+0000 7f0b67fff640 1 -- 172.21.15.45:0/775010098 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6002ae60 con 0x7f0b5c004800 2024-05-08T05:42:53.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.668+0000 7f0b77608640 1 -- 172.21.15.45:0/775010098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b5c004800 msgr2=0x7f0b5c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:53.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.668+0000 7f0b77608640 1 --2- 172.21.15.45:0/775010098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b5c004800 0x7f0b5c004c80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0b600018d0 tx=0x7f0b5c005830 comp rx=0 tx=0).stop 2024-05-08T05:42:53.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.668+0000 7f0b75ba5640 1 -- 172.21.15.45:0/775010098 reap_dead start 2024-05-08T05:42:53.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.668+0000 7f0b77608640 1 -- 172.21.15.45:0/775010098 shutdown_connections 2024-05-08T05:42:53.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.668+0000 7f0b77608640 1 -- 172.21.15.45:0/775010098 >> 172.21.15.45:0/775010098 conn(0x7f0b7006c3a0 msgr2=0x7f0b7006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:53.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.668+0000 7f0b77608640 1 -- 172.21.15.45:0/775010098 shutdown_connections 2024-05-08T05:42:53.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.668+0000 7f0b77608640 1 -- 172.21.15.45:0/775010098 wait complete. 2024-05-08T05:42:53.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.669+0000 7f0b77608640 1 Processor -- start 2024-05-08T05:42:53.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.669+0000 7f0b77608640 1 -- start start 2024-05-08T05:42:53.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.670+0000 7f0b77608640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b5c004800 0x7f0b701560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:53.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.670+0000 7f0b77608640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b70149410 0x7f0b7009dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:53.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.670+0000 7f0b77608640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b70149940 0x7f0b7009e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:53.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.670+0000 7f0b77608640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0b70153230 con 0x7f0b70149940 2024-05-08T05:42:53.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.670+0000 7f0b77608640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0b701530b0 con 0x7f0b5c004800 2024-05-08T05:42:53.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.670+0000 7f0b77608640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0b70164960 con 0x7f0b70149410 2024-05-08T05:42:53.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.670+0000 7f0b75ba5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b70149940 0x7f0b7009e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:53.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.670+0000 7f0b75ba5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b70149940 0x7f0b7009e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50608/0 (socket says 172.21.15.45:50608) 2024-05-08T05:42:53.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.670+0000 7f0b75ba5640 1 -- 172.21.15.45:0/2250753904 learned_addr learned my addr 172.21.15.45:0/2250753904 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:53.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.670+0000 7f0b753a4640 1 --2- 172.21.15.45:0/2250753904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b5c004800 0x7f0b701560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:53.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.670+0000 7f0b74ba3640 1 --2- 172.21.15.45:0/2250753904 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b70149410 0x7f0b7009dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:53.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.670+0000 7f0b75ba5640 1 -- 172.21.15.45:0/2250753904 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b70149410 msgr2=0x7f0b7009dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:53.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.670+0000 7f0b75ba5640 1 --2- 172.21.15.45:0/2250753904 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b70149410 0x7f0b7009dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:53.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.671+0000 7f0b75ba5640 1 -- 172.21.15.45:0/2250753904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b5c004800 msgr2=0x7f0b701560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:53.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.671+0000 7f0b75ba5640 1 --2- 172.21.15.45:0/2250753904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b5c004800 0x7f0b701560a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:53.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.671+0000 7f0b75ba5640 1 -- 172.21.15.45:0/2250753904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b700b1780 con 0x7f0b70149940 2024-05-08T05:42:53.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.671+0000 7f0b75ba5640 1 --2- 172.21.15.45:0/2250753904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b70149940 0x7f0b7009e520 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0b68011c80 tx=0x7f0b6800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:53.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.671+0000 7f0b65ffb640 1 -- 172.21.15.45:0/2250753904 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68015430 con 0x7f0b70149940 2024-05-08T05:42:53.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.671+0000 7f0b65ffb640 1 -- 172.21.15.45:0/2250753904 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b6801d070 con 0x7f0b70149940 2024-05-08T05:42:53.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.671+0000 7f0b65ffb640 1 -- 172.21.15.45:0/2250753904 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68015830 con 0x7f0b70149940 2024-05-08T05:42:53.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.671+0000 7f0b77608640 1 -- 172.21.15.45:0/2250753904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b700b1a60 con 0x7f0b70149940 2024-05-08T05:42:53.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.671+0000 7f0b77608640 1 -- 172.21.15.45:0/2250753904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b700cea60 con 0x7f0b70149940 2024-05-08T05:42:53.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.672+0000 7f0b77608640 1 -- 172.21.15.45:0/2250753904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b700693f0 con 0x7f0b70149940 2024-05-08T05:42:53.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.673+0000 7f0b65ffb640 1 -- 172.21.15.45:0/2250753904 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0b680159d0 con 0x7f0b70149940 2024-05-08T05:42:53.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.673+0000 7f0b65ffb640 1 --2- 172.21.15.45:0/2250753904 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0b540762d0 0x7f0b54078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:53.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.673+0000 7f0b65ffb640 1 -- 172.21.15.45:0/2250753904 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5335+0+0 (secure 0 0 0) 0x7f0b68097d70 con 0x7f0b70149940 2024-05-08T05:42:53.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.674+0000 7f0b753a4640 1 --2- 172.21.15.45:0/2250753904 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0b540762d0 0x7f0b54078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:53.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.674+0000 7f0b753a4640 1 --2- 172.21.15.45:0/2250753904 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0b540762d0 0x7f0b54078790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c001750 tx=0x7f0b6c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:53.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.678+0000 7f0b65ffb640 1 -- 172.21.15.45:0/2250753904 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0b68010040 con 0x7f0b70149940 2024-05-08T05:42:53.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.868+0000 7f0b77608640 1 -- 172.21.15.45:0/2250753904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0b7006cb00 con 0x7f0b70149940 2024-05-08T05:42:53.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.872+0000 7f0b65ffb640 1 -- 172.21.15.45:0/2250753904 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f0b680617a0 con 0x7f0b70149940 2024-05-08T05:42:53.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.874+0000 7f0b77608640 1 -- 172.21.15.45:0/2250753904 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0b540762d0 msgr2=0x7f0b54078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:53.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.874+0000 7f0b77608640 1 --2- 172.21.15.45:0/2250753904 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0b540762d0 0x7f0b54078790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c001750 tx=0x7f0b6c009040 comp rx=0 tx=0).stop 2024-05-08T05:42:53.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.874+0000 7f0b77608640 1 -- 172.21.15.45:0/2250753904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b70149940 msgr2=0x7f0b7009e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:53.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.874+0000 7f0b77608640 1 --2- 172.21.15.45:0/2250753904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b70149940 0x7f0b7009e520 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0b68011c80 tx=0x7f0b6800f580 comp rx=0 tx=0).stop 2024-05-08T05:42:53.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.874+0000 7f0b77608640 1 -- 172.21.15.45:0/2250753904 shutdown_connections 2024-05-08T05:42:53.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.874+0000 7f0b77608640 1 --2- 172.21.15.45:0/2250753904 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0b540762d0 0x7f0b54078790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:53.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.874+0000 7f0b77608640 1 --2- 172.21.15.45:0/2250753904 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b70149410 0x7f0b7009dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:53.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.874+0000 7f0b77608640 1 --2- 172.21.15.45:0/2250753904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b70149940 0x7f0b7009e520 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:53.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.874+0000 7f0b77608640 1 --2- 172.21.15.45:0/2250753904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b5c004800 0x7f0b701560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:53.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.874+0000 7f0b77608640 1 -- 172.21.15.45:0/2250753904 >> 172.21.15.45:0/2250753904 conn(0x7f0b7006c3a0 msgr2=0x7f0b7015ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:53.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.874+0000 7f0b77608640 1 -- 172.21.15.45:0/2250753904 shutdown_connections 2024-05-08T05:42:53.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:53.874+0000 7f0b77608640 1 -- 172.21.15.45:0/2250753904 wait complete. 2024-05-08T05:42:53.885 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T05:42:53.885 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:42:53.921 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:42:53.962 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T05:42:53.962 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:42:53.962 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:53.978 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:42:53.978 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:42:53.978 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:54.022 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:42:54.022 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:42:54.022 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:54.038 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:42:54.038 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:42:54.038 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:42:54.082 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:42:54.082 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T05:42:54.082 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:42:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.185+0000 7f085b86d640 1 Processor -- start 2024-05-08T05:42:54.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.186+0000 7f085b86d640 1 -- start start 2024-05-08T05:42:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.186+0000 7f085b86d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f08540d1970 0x7f08540d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.186+0000 7f085b86d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08540d4cc0 0x7f08540d5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:54.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.187+0000 7f085b86d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08541630b0 con 0x7f085415f420 2024-05-08T05:42:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.187+0000 7f085b86d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f08540d3e70 con 0x7f08540d1970 2024-05-08T05:42:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.187+0000 7f085b86d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f08540d5670 con 0x7f08540d4cc0 2024-05-08T05:42:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.187+0000 7f0858e08640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f08540d1970 0x7f08540d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.187+0000 7f0859609640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f085415f420 0x7f08540d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36610/0 (socket says 172.21.15.45:36610) 2024-05-08T05:42:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.187+0000 7f0859609640 1 -- 172.21.15.45:0/1261581929 learned_addr learned my addr 172.21.15.45:0/1261581929 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.187+0000 7f0859e0a640 1 --2- 172.21.15.45:0/1261581929 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08540d4cc0 0x7f08540d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.187+0000 7f0858e08640 1 -- 172.21.15.45:0/1261581929 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08540d4cc0 msgr2=0x7f08540d5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.187+0000 7f0858e08640 1 --2- 172.21.15.45:0/1261581929 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08540d4cc0 0x7f08540d5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.187+0000 7f0858e08640 1 -- 172.21.15.45:0/1261581929 >> v1:172.21.15.45:6789/0 conn(0x7f085415f420 legacy=0x7f08540d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.187+0000 7f084bfff640 1 -- 172.21.15.45:0/1261581929 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2134624711 0 0) 0x7f0850001110 con 0x7f085415f420 2024-05-08T05:42:54.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.187+0000 7f0858e08640 1 -- 172.21.15.45:0/1261581929 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08540d2550 con 0x7f08540d1970 2024-05-08T05:42:54.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.188+0000 7f0858e08640 1 --2- 172.21.15.45:0/1261581929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08540d1970 0x7f08540d1e20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f08440097d0 tx=0x7f084402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911897268676c501 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:54.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.188+0000 7f084bfff640 1 -- 172.21.15.45:0/1261581929 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844037070 con 0x7f08540d1970 2024-05-08T05:42:54.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.188+0000 7f084bfff640 1 -- 172.21.15.45:0/1261581929 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0844032440 con 0x7f08540d1970 2024-05-08T05:42:54.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.188+0000 7f084bfff640 1 -- 172.21.15.45:0/1261581929 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0844032810 con 0x7f08540d1970 2024-05-08T05:42:54.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.188+0000 7f085b86d640 1 -- 172.21.15.45:0/1261581929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08540d1970 msgr2=0x7f08540d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:54.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.188+0000 7f085b86d640 1 --2- 172.21.15.45:0/1261581929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08540d1970 0x7f08540d1e20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f08440097d0 tx=0x7f084402a1c0 comp rx=0 tx=0).stop 2024-05-08T05:42:54.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.189+0000 7f085b86d640 1 -- 172.21.15.45:0/1261581929 shutdown_connections 2024-05-08T05:42:54.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.189+0000 7f085b86d640 1 --2- 172.21.15.45:0/1261581929 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08540d4cc0 0x7f08540d5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:54.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.189+0000 7f085b86d640 1 --2- 172.21.15.45:0/1261581929 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08540d1970 0x7f08540d1e20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:54.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.189+0000 7f085b86d640 1 -- 172.21.15.45:0/1261581929 >> 172.21.15.45:0/1261581929 conn(0x7f085408eac0 msgr2=0x7f085409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:54.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.189+0000 7f085b86d640 1 -- 172.21.15.45:0/1261581929 shutdown_connections 2024-05-08T05:42:54.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.189+0000 7f085b86d640 1 -- 172.21.15.45:0/1261581929 wait complete. 2024-05-08T05:42:54.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.190+0000 7f085b86d640 1 Processor -- start 2024-05-08T05:42:54.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.190+0000 7f085b86d640 1 -- start start 2024-05-08T05:42:54.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f085b86d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08540d1970 0x7f0854158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:54.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f085b86d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08540d4cc0 0x7f08540cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:54.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f085b86d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08540c9a20 0x7f08540b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:54.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f085b86d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f08540d3240 con 0x7f08540c9a20 2024-05-08T05:42:54.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f085b86d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f08540d2f60 con 0x7f08540d4cc0 2024-05-08T05:42:54.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f085b86d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f08540d2830 con 0x7f08540d1970 2024-05-08T05:42:54.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f0859e0a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08540c9a20 0x7f08540b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:54.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f0859e0a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08540c9a20 0x7f08540b8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50626/0 (socket says 172.21.15.45:50626) 2024-05-08T05:42:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f0859e0a640 1 -- 172.21.15.45:0/3062882675 learned_addr learned my addr 172.21.15.45:0/3062882675 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f0859609640 1 --2- 172.21.15.45:0/3062882675 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08540d1970 0x7f0854158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f0858e08640 1 --2- 172.21.15.45:0/3062882675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08540d4cc0 0x7f08540cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f0859e0a640 1 -- 172.21.15.45:0/3062882675 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08540d1970 msgr2=0x7f0854158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f0859e0a640 1 --2- 172.21.15.45:0/3062882675 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08540d1970 0x7f0854158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f0859e0a640 1 -- 172.21.15.45:0/3062882675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08540d4cc0 msgr2=0x7f08540cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f0859e0a640 1 --2- 172.21.15.45:0/3062882675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08540d4cc0 0x7f08540cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f0859e0a640 1 -- 172.21.15.45:0/3062882675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08540b7e50 con 0x7f08540c9a20 2024-05-08T05:42:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f0859609640 1 --2- 172.21.15.45:0/3062882675 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08540d1970 0x7f0854158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f0859e0a640 1 --2- 172.21.15.45:0/3062882675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08540c9a20 0x7f08540b8c10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f084c007e40 tx=0x7f084c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f0849ffb640 1 -- 172.21.15.45:0/3062882675 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c014620 con 0x7f08540c9a20 2024-05-08T05:42:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f0849ffb640 1 -- 172.21.15.45:0/3062882675 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084c019070 con 0x7f08540c9a20 2024-05-08T05:42:54.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f085b86d640 1 -- 172.21.15.45:0/3062882675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08540b8130 con 0x7f08540c9a20 2024-05-08T05:42:54.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.191+0000 7f0849ffb640 1 -- 172.21.15.45:0/3062882675 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c014a20 con 0x7f08540c9a20 2024-05-08T05:42:54.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.192+0000 7f085b86d640 1 -- 172.21.15.45:0/3062882675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08540c8b00 con 0x7f08540c9a20 2024-05-08T05:42:54.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.193+0000 7f085b86d640 1 -- 172.21.15.45:0/3062882675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08540693f0 con 0x7f08540c9a20 2024-05-08T05:42:54.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.193+0000 7f0849ffb640 1 -- 172.21.15.45:0/3062882675 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f084c014bc0 con 0x7f08540c9a20 2024-05-08T05:42:54.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.194+0000 7f0849ffb640 1 --2- 172.21.15.45:0/3062882675 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0838076300 0x7f08380787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:54.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.194+0000 7f0849ffb640 1 -- 172.21.15.45:0/3062882675 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5335+0+0 (secure 0 0 0) 0x7f084c097de0 con 0x7f08540c9a20 2024-05-08T05:42:54.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.194+0000 7f0859609640 1 --2- 172.21.15.45:0/3062882675 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0838076300 0x7f08380787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:54.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.195+0000 7f0859609640 1 --2- 172.21.15.45:0/3062882675 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0838076300 0x7f08380787c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0850000f60 tx=0x7f0850009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:54.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.198+0000 7f0849ffb640 1 -- 172.21.15.45:0/3062882675 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f084c0618a0 con 0x7f08540c9a20 2024-05-08T05:42:54.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.431+0000 7f085b86d640 1 -- 172.21.15.45:0/3062882675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f085415ec40 con 0x7f08540c9a20 2024-05-08T05:42:54.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.431+0000 7f0849ffb640 1 -- 172.21.15.45:0/3062882675 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3863 (secure 0 0 0) 0x7f084c0611a0 con 0x7f08540c9a20 2024-05-08T05:42:54.432 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:54.432 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3295938044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3295938044},{"type":"v1","addr":"172.21.15.63:6836","nonce":3295938044}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4024530715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4024530715},{"type":"v1","addr":"172.21.15.63:6837","nonce":4024530715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/97258541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":97258541},{"type":"v1","addr":"172.21.15.45:6834","nonce":97258541}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5205,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1562053200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1562053200},{"type":"v1","addr":"172.21.15.45:6835","nonce":1562053200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-05-08T05:42:54.432 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 19 2024-05-08T05:42:54.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.434+0000 7f085b86d640 1 -- 172.21.15.45:0/3062882675 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0838076300 msgr2=0x7f08380787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:54.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.434+0000 7f085b86d640 1 --2- 172.21.15.45:0/3062882675 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0838076300 0x7f08380787c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0850000f60 tx=0x7f0850009040 comp rx=0 tx=0).stop 2024-05-08T05:42:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.434+0000 7f085b86d640 1 -- 172.21.15.45:0/3062882675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08540c9a20 msgr2=0x7f08540b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.434+0000 7f085b86d640 1 --2- 172.21.15.45:0/3062882675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08540c9a20 0x7f08540b8c10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f084c007e40 tx=0x7f084c00e4b0 comp rx=0 tx=0).stop 2024-05-08T05:42:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.434+0000 7f085b86d640 1 -- 172.21.15.45:0/3062882675 shutdown_connections 2024-05-08T05:42:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.434+0000 7f085b86d640 1 --2- 172.21.15.45:0/3062882675 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0838076300 0x7f08380787c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.434+0000 7f085b86d640 1 --2- 172.21.15.45:0/3062882675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08540c9a20 0x7f08540b8c10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.434+0000 7f085b86d640 1 --2- 172.21.15.45:0/3062882675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08540d4cc0 0x7f08540cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.434+0000 7f085b86d640 1 --2- 172.21.15.45:0/3062882675 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f08540d1970 0x7f0854158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.434+0000 7f085b86d640 1 -- 172.21.15.45:0/3062882675 >> 172.21.15.45:0/3062882675 conn(0x7f085408eac0 msgr2=0x7f08540cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.434+0000 7f085b86d640 1 -- 172.21.15.45:0/3062882675 shutdown_connections 2024-05-08T05:42:54.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.434+0000 7f085b86d640 1 -- 172.21.15.45:0/3062882675 wait complete. 2024-05-08T05:42:54.446 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T05:42:54.446 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T05:42:54.446 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T05:42:54.446 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:42:54.446 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T05:42:54.446 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T05:42:54.447 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T05:42:54.447 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:42:54.447 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T05:42:54.447 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T05:42:54.447 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T05:42:54.447 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:42:54.447 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T05:42:54.447 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T05:42:54.448 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T05:42:54.448 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:42:54.456 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T05:42:54.456 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T05:42:54.457 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T05:42:54.457 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T05:42:54.460 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T05:42:54.460 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T05:42:54.461 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T05:42:54.461 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T05:42:54.498 INFO:tasks.ceph.mds.a:Started 2024-05-08T05:42:54.499 DEBUG:teuthology.parallel:result is None 2024-05-08T05:42:54.499 INFO:tasks.ceph.mds.c:Started 2024-05-08T05:42:54.499 DEBUG:teuthology.parallel:result is None 2024-05-08T05:42:54.503 INFO:tasks.ceph.mds.b:Started 2024-05-08T05:42:54.503 DEBUG:teuthology.parallel:result is None 2024-05-08T05:42:54.504 INFO:tasks.ceph.mds.d:Started 2024-05-08T05:42:54.504 DEBUG:teuthology.parallel:result is None 2024-05-08T05:42:54.504 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T05:42:54.577 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-05-08T05:42:54.577 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-05-08T05:42:54.589 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-08T05:42:54.589 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-08T05:42:54.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.614+0000 7f4517146640 1 Processor -- start 2024-05-08T05:42:54.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.614+0000 7f4517146640 1 -- start start 2024-05-08T05:42:54.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.615+0000 7f4517146640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f451015f420 0x7f45100d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:54.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.615+0000 7f4517146640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45101630b0 con 0x7f45100d3ca0 2024-05-08T05:42:54.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.615+0000 7f4517146640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45100d2550 con 0x7f45100d2080 2024-05-08T05:42:54.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.615+0000 7f4517146640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f45100d2c80 con 0x7f451015f420 2024-05-08T05:42:54.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.615+0000 7f450ffff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f45100d3ca0 0x7f45100d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36626/0 (socket says 172.21.15.45:36626) 2024-05-08T05:42:54.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.615+0000 7f450ffff640 1 -- 172.21.15.45:0/1945650594 learned_addr learned my addr 172.21.15.45:0/1945650594 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:54.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.615+0000 7f4514ee2640 1 --2- 172.21.15.45:0/1945650594 >> v2:172.21.15.63:3300/0 conn(0x7f451015f420 0x7f45100d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:54.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.615+0000 7f450f7fe640 1 -- 172.21.15.45:0/1945650594 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 676848223 0 0) 0x7f44fc001110 con 0x7f45100d3ca0 2024-05-08T05:42:54.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.615+0000 7f450f7fe640 1 -- 172.21.15.45:0/1945650594 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4500003520 con 0x7f45100d3ca0 2024-05-08T05:42:54.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.615+0000 7f4514ee2640 1 -- 172.21.15.45:0/1945650594 >> v1:172.21.15.63:6789/0 conn(0x7f45100d2080 legacy=0x7f45100d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:54.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.615+0000 7f4514ee2640 1 -- 172.21.15.45:0/1945650594 >> v1:172.21.15.45:6789/0 conn(0x7f45100d3ca0 legacy=0x7f45100d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:54.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.615+0000 7f4514ee2640 1 -- 172.21.15.45:0/1945650594 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45100d2f60 con 0x7f451015f420 2024-05-08T05:42:54.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.616+0000 7f4514ee2640 1 --2- 172.21.15.45:0/1945650594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f451015f420 0x7f45100d3760 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f45040097a0 tx=0x7f450402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eccf0aace653620d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:54.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.616+0000 7f450f7fe640 1 -- 172.21.15.45:0/1945650594 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504037070 con 0x7f451015f420 2024-05-08T05:42:54.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.616+0000 7f450f7fe640 1 -- 172.21.15.45:0/1945650594 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4504032440 con 0x7f451015f420 2024-05-08T05:42:54.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.616+0000 7f450f7fe640 1 -- 172.21.15.45:0/1945650594 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45040327f0 con 0x7f451015f420 2024-05-08T05:42:54.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.617+0000 7f4517146640 1 -- 172.21.15.45:0/1945650594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f451015f420 msgr2=0x7f45100d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:54.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.617+0000 7f4517146640 1 --2- 172.21.15.45:0/1945650594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f451015f420 0x7f45100d3760 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f45040097a0 tx=0x7f450402a230 comp rx=0 tx=0).stop 2024-05-08T05:42:54.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.617+0000 7f4517146640 1 -- 172.21.15.45:0/1945650594 shutdown_connections 2024-05-08T05:42:54.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.617+0000 7f4517146640 1 --2- 172.21.15.45:0/1945650594 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f451015f420 0x7f45100d3760 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:54.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.617+0000 7f4517146640 1 -- 172.21.15.45:0/1945650594 >> 172.21.15.45:0/1945650594 conn(0x7f451008eac0 msgr2=0x7f451009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:54.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.617+0000 7f4517146640 1 -- 172.21.15.45:0/1945650594 shutdown_connections 2024-05-08T05:42:54.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.617+0000 7f4517146640 1 -- 172.21.15.45:0/1945650594 wait complete. 2024-05-08T05:42:54.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.618+0000 7f4517146640 1 Processor -- start 2024-05-08T05:42:54.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.618+0000 7f4517146640 1 -- start start 2024-05-08T05:42:54.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.619+0000 7f4517146640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45100d2080 0x7f45101585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:54.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.619+0000 7f4517146640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45100ba240 0x7f45100ba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:54.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.619+0000 7f4517146640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45100cab30 0x7f45100cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:54.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.619+0000 7f4517146640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f45100d53d0 con 0x7f45100d2080 2024-05-08T05:42:54.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.619+0000 7f4517146640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f45100d2830 con 0x7f45100cab30 2024-05-08T05:42:54.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.619+0000 7f4517146640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f45100d5550 con 0x7f45100ba240 2024-05-08T05:42:54.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.619+0000 7f4514ee2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45100d2080 0x7f45101585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:54.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.619+0000 7f4514ee2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45100d2080 0x7f45101585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50674/0 (socket says 172.21.15.45:50674) 2024-05-08T05:42:54.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.619+0000 7f4514ee2640 1 -- 172.21.15.45:0/1485010325 learned_addr learned my addr 172.21.15.45:0/1485010325 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:54.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.619+0000 7f45156e3640 1 --2- 172.21.15.45:0/1485010325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45100cab30 0x7f45100cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:54.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.619+0000 7f450ffff640 1 --2- 172.21.15.45:0/1485010325 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45100ba240 0x7f45100ba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:54.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.620+0000 7f4514ee2640 1 -- 172.21.15.45:0/1485010325 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45100ba240 msgr2=0x7f45100ba6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:54.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.620+0000 7f4514ee2640 1 --2- 172.21.15.45:0/1485010325 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45100ba240 0x7f45100ba6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:54.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.620+0000 7f4514ee2640 1 -- 172.21.15.45:0/1485010325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45100cab30 msgr2=0x7f45100cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:54.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.620+0000 7f4514ee2640 1 --2- 172.21.15.45:0/1485010325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45100cab30 0x7f45100cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:54.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.620+0000 7f4514ee2640 1 -- 172.21.15.45:0/1485010325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4510161cb0 con 0x7f45100d2080 2024-05-08T05:42:54.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.620+0000 7f450ffff640 1 --2- 172.21.15.45:0/1485010325 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45100ba240 0x7f45100ba6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:54.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.620+0000 7f4514ee2640 1 --2- 172.21.15.45:0/1485010325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45100d2080 0x7f45101585c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f45040097a0 tx=0x7f45040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:54.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.620+0000 7f450d7fa640 1 -- 172.21.15.45:0/1485010325 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504037050 con 0x7f45100d2080 2024-05-08T05:42:54.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.620+0000 7f450d7fa640 1 -- 172.21.15.45:0/1485010325 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4504038040 con 0x7f45100d2080 2024-05-08T05:42:54.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.620+0000 7f4517146640 1 -- 172.21.15.45:0/1485010325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4510160c10 con 0x7f45100d2080 2024-05-08T05:42:54.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.620+0000 7f450d7fa640 1 -- 172.21.15.45:0/1485010325 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504004470 con 0x7f45100d2080 2024-05-08T05:42:54.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.620+0000 7f4517146640 1 -- 172.21.15.45:0/1485010325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45101611d0 con 0x7f45100d2080 2024-05-08T05:42:54.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.621+0000 7f4517146640 1 -- 172.21.15.45:0/1485010325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4510068c40 con 0x7f45100d2080 2024-05-08T05:42:54.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.622+0000 7f450d7fa640 1 -- 172.21.15.45:0/1485010325 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f450403c6d0 con 0x7f45100d2080 2024-05-08T05:42:54.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.622+0000 7f450d7fa640 1 --2- 172.21.15.45:0/1485010325 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f44f40762d0 0x7f44f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:54.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.622+0000 7f450d7fa640 1 -- 172.21.15.45:0/1485010325 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5335+0+0 (secure 0 0 0) 0x7f45040b7e40 con 0x7f45100d2080 2024-05-08T05:42:54.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.622+0000 7f450ffff640 1 --2- 172.21.15.45:0/1485010325 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f44f40762d0 0x7f44f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:54.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.623+0000 7f450ffff640 1 --2- 172.21.15.45:0/1485010325 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f44f40762d0 0x7f44f4078790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f44fc007fd0 tx=0x7f44fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:54.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.626+0000 7f450d7fa640 1 -- 172.21.15.45:0/1485010325 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4510068c40 con 0x7f45100d2080 2024-05-08T05:42:54.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:54.828+0000 7f4517146640 1 -- 172.21.15.45:0/1485010325 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f45100693f0 con 0x7f45100d2080 2024-05-08T05:42:55.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.548+0000 7f450d7fa640 1 -- 172.21.15.45:0/1485010325 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) ==== 97+0+0 (secure 0 0 0) 0x7f4510161cb0 con 0x7f45100d2080 2024-05-08T05:42:55.548 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2024-05-08T05:42:55.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.551+0000 7f4517146640 1 -- 172.21.15.45:0/1485010325 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f44f40762d0 msgr2=0x7f44f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:55.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.551+0000 7f4517146640 1 --2- 172.21.15.45:0/1485010325 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f44f40762d0 0x7f44f4078790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f44fc007fd0 tx=0x7f44fc009040 comp rx=0 tx=0).stop 2024-05-08T05:42:55.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.551+0000 7f4517146640 1 -- 172.21.15.45:0/1485010325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45100d2080 msgr2=0x7f45101585c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:55.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.551+0000 7f4517146640 1 --2- 172.21.15.45:0/1485010325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45100d2080 0x7f45101585c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f45040097a0 tx=0x7f45040046c0 comp rx=0 tx=0).stop 2024-05-08T05:42:55.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.551+0000 7f4517146640 1 -- 172.21.15.45:0/1485010325 shutdown_connections 2024-05-08T05:42:55.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.551+0000 7f4517146640 1 --2- 172.21.15.45:0/1485010325 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f44f40762d0 0x7f44f4078790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:55.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.551+0000 7f4517146640 1 --2- 172.21.15.45:0/1485010325 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45100ba240 0x7f45100ba6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:55.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.551+0000 7f4517146640 1 --2- 172.21.15.45:0/1485010325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45100cab30 0x7f45100cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:55.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.551+0000 7f4517146640 1 --2- 172.21.15.45:0/1485010325 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45100d2080 0x7f45101585c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:55.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.551+0000 7f4517146640 1 -- 172.21.15.45:0/1485010325 >> 172.21.15.45:0/1485010325 conn(0x7f451008eac0 msgr2=0x7f45100d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:55.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.551+0000 7f4517146640 1 -- 172.21.15.45:0/1485010325 shutdown_connections 2024-05-08T05:42:55.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.551+0000 7f4517146640 1 -- 172.21.15.45:0/1485010325 wait complete. 2024-05-08T05:42:55.566 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T05:42:55.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.676+0000 7f5670b40640 1 Processor -- start 2024-05-08T05:42:55.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.676+0000 7f5670b40640 1 -- start start 2024-05-08T05:42:55.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.677+0000 7f5670b40640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f566c163920 0x7f566c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:55.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.677+0000 7f5670b40640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f566c163d30 con 0x7f566c070d30 2024-05-08T05:42:55.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.677+0000 7f5670b40640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f566c0d1300 con 0x7f566c0d20c0 2024-05-08T05:42:55.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.677+0000 7f5670b40640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f566c0d10d0 con 0x7f566c163920 2024-05-08T05:42:55.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.677+0000 7f566ad9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f566c163920 0x7f566c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:55.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.677+0000 7f566ad9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f566c163920 0x7f566c06d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50688/0 (socket says 172.21.15.45:50688) 2024-05-08T05:42:55.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.677+0000 7f566a59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f566c070d30 0x7f566c0703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36640/0 (socket says 172.21.15.45:36640) 2024-05-08T05:42:55.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.677+0000 7f566ad9d640 1 -- 172.21.15.45:0/320656660 learned_addr learned my addr 172.21.15.45:0/320656660 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:55.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.677+0000 7f566ad9d640 1 -- 172.21.15.45:0/320656660 >> v1:172.21.15.63:6789/0 conn(0x7f566c0d20c0 legacy=0x7f566c071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:55.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.677+0000 7f566ad9d640 1 -- 172.21.15.45:0/320656660 >> v1:172.21.15.45:6789/0 conn(0x7f566c070d30 legacy=0x7f566c0703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:55.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.677+0000 7f566ad9d640 1 -- 172.21.15.45:0/320656660 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c0d15e0 con 0x7f566c163920 2024-05-08T05:42:55.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.678+0000 7f566ad9d640 1 --2- 172.21.15.45:0/320656660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c163920 0x7f566c06d4b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5664009e50 tx=0x7f566402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f20998b2e483d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:55.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.678+0000 7f5669d9b640 1 -- 172.21.15.45:0/320656660 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664036070 con 0x7f566c163920 2024-05-08T05:42:55.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.678+0000 7f5669d9b640 1 -- 172.21.15.45:0/320656660 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5664031600 con 0x7f566c163920 2024-05-08T05:42:55.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.678+0000 7f5669d9b640 1 -- 172.21.15.45:0/320656660 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56640319d0 con 0x7f566c163920 2024-05-08T05:42:55.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.679+0000 7f5670b40640 1 -- 172.21.15.45:0/320656660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c163920 msgr2=0x7f566c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:55.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.679+0000 7f5670b40640 1 --2- 172.21.15.45:0/320656660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c163920 0x7f566c06d4b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5664009e50 tx=0x7f566402a710 comp rx=0 tx=0).stop 2024-05-08T05:42:55.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.679+0000 7f5670b40640 1 -- 172.21.15.45:0/320656660 shutdown_connections 2024-05-08T05:42:55.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.679+0000 7f5670b40640 1 --2- 172.21.15.45:0/320656660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c163920 0x7f566c06d4b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:55.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.679+0000 7f5670b40640 1 -- 172.21.15.45:0/320656660 >> 172.21.15.45:0/320656660 conn(0x7f566c0d2e90 msgr2=0x7f566c0d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:55.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.679+0000 7f5670b40640 1 -- 172.21.15.45:0/320656660 shutdown_connections 2024-05-08T05:42:55.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.679+0000 7f5670b40640 1 -- 172.21.15.45:0/320656660 wait complete. 2024-05-08T05:42:55.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.680+0000 7f5670b40640 1 Processor -- start 2024-05-08T05:42:55.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.680+0000 7f5670b40640 1 -- start start 2024-05-08T05:42:55.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.681+0000 7f5670b40640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f566c070d30 0x7f566c0b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:55.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.681+0000 7f5670b40640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f566c0d20c0 0x7f566c1f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:55.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.681+0000 7f5670b40640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c163920 0x7f566c14d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:55.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.681+0000 7f5670b40640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f566c0d1c30 con 0x7f566c163920 2024-05-08T05:42:55.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.681+0000 7f5670b40640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f566c070b80 con 0x7f566c070d30 2024-05-08T05:42:55.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.681+0000 7f5670b40640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f566c06d9f0 con 0x7f566c0d20c0 2024-05-08T05:42:55.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.681+0000 7f566ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c163920 0x7f566c14d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:55.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.681+0000 7f566ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c163920 0x7f566c14d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50700/0 (socket says 172.21.15.45:50700) 2024-05-08T05:42:55.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.681+0000 7f566ad9d640 1 -- 172.21.15.45:0/724195918 learned_addr learned my addr 172.21.15.45:0/724195918 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:55.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.681+0000 7f566a59c640 1 --2- 172.21.15.45:0/724195918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f566c070d30 0x7f566c0b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:55.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.681+0000 7f5661d9b640 1 --2- 172.21.15.45:0/724195918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f566c0d20c0 0x7f566c1f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:55.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.681+0000 7f566ad9d640 1 -- 172.21.15.45:0/724195918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f566c0d20c0 msgr2=0x7f566c1f8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:55.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.681+0000 7f566ad9d640 1 --2- 172.21.15.45:0/724195918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f566c0d20c0 0x7f566c1f8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:55.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.681+0000 7f566ad9d640 1 -- 172.21.15.45:0/724195918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f566c070d30 msgr2=0x7f566c0b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:55.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.681+0000 7f566ad9d640 1 --2- 172.21.15.45:0/724195918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f566c070d30 0x7f566c0b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:55.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.682+0000 7f566ad9d640 1 -- 172.21.15.45:0/724195918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c1f92c0 con 0x7f566c163920 2024-05-08T05:42:55.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.682+0000 7f5661d9b640 1 --2- 172.21.15.45:0/724195918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f566c0d20c0 0x7f566c1f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:55.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.682+0000 7f566a59c640 1 --2- 172.21.15.45:0/724195918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f566c070d30 0x7f566c0b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:55.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.682+0000 7f566ad9d640 1 --2- 172.21.15.45:0/724195918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c163920 0x7f566c14d260 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5664009e50 tx=0x7f5664004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:55.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.682+0000 7f56637fe640 1 -- 172.21.15.45:0/724195918 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664036050 con 0x7f566c163920 2024-05-08T05:42:55.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.682+0000 7f56637fe640 1 -- 172.21.15.45:0/724195918 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5664042070 con 0x7f566c163920 2024-05-08T05:42:55.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.682+0000 7f5670b40640 1 -- 172.21.15.45:0/724195918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566c1f95a0 con 0x7f566c163920 2024-05-08T05:42:55.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.682+0000 7f56637fe640 1 -- 172.21.15.45:0/724195918 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664009510 con 0x7f566c163920 2024-05-08T05:42:55.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.682+0000 7f5670b40640 1 -- 172.21.15.45:0/724195918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f566c14d7a0 con 0x7f566c163920 2024-05-08T05:42:55.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.683+0000 7f5670b40640 1 -- 172.21.15.45:0/724195918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f566c0693f0 con 0x7f566c163920 2024-05-08T05:42:55.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.684+0000 7f56637fe640 1 -- 172.21.15.45:0/724195918 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5664007bf0 con 0x7f566c163920 2024-05-08T05:42:55.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.684+0000 7f56637fe640 1 --2- 172.21.15.45:0/724195918 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f564c0762d0 0x7f564c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:55.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.684+0000 7f56637fe640 1 -- 172.21.15.45:0/724195918 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5249+0+0 (secure 0 0 0) 0x7f56640b8310 con 0x7f566c163920 2024-05-08T05:42:55.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.685+0000 7f566a59c640 1 --2- 172.21.15.45:0/724195918 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f564c0762d0 0x7f564c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:55.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.685+0000 7f566a59c640 1 --2- 172.21.15.45:0/724195918 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f564c0762d0 0x7f564c078790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f565c000f60 tx=0x7f565c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:55.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.688+0000 7f56637fe640 1 -- 172.21.15.45:0/724195918 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5664081ec0 con 0x7f566c163920 2024-05-08T05:42:55.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.952+0000 7f5670b40640 1 -- 172.21.15.45:0/724195918 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f566c0b64d0 con 0x7f566c163920 2024-05-08T05:42:55.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.953+0000 7f56637fe640 1 -- 172.21.15.45:0/724195918 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) ==== 80+0+5304 (secure 0 0 0) 0x7f56640b6e20 con 0x7f566c163920 2024-05-08T05:42:55.953 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:55.953 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:42:55.953 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2024-05-08T05:42:55.953 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:55.954 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2024-05-08T05:42:55.954 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmrRgXHBAAX1mgdmOD9JC4S1vJNyLSQw==", 2024-05-08T05:42:55.954 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:55.954 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:42:55.954 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:42:55.954 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:42:55.954 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:55.954 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:55.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:55.954 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:55.954 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2024-05-08T05:42:55.954 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmvn8SKBAAVkXYBbTfu98gl0AptOfJEg==", 2024-05-08T05:42:55.954 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:55.955 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:42:55.955 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:42:55.955 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:42:55.955 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:55.955 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:55.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:55.955 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:55.955 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2024-05-08T05:42:55.955 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmeiJCIxAAr3Br1fJVgR9TW2Q+4tMQBQ==", 2024-05-08T05:42:55.955 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:55.955 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:42:55.955 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:42:55.955 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:42:55.955 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:55.956 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:55.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:55.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:55.956 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2024-05-08T05:42:55.956 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmXdFILxAAxteIxUYeFdQKd/oxHLpeOQ==", 2024-05-08T05:42:55.956 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:55.956 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:42:55.956 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:42:55.956 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:42:55.956 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:55.956 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:55.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:55.956 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:55.956 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2024-05-08T05:42:55.956 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBWEDtmLdHHGRAAA1o4Z/lTGERIZhulqVtybw==", 2024-05-08T05:42:55.956 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:55.957 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:42:55.957 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:42:55.957 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:55.957 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:55.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:55.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:55.957 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2024-05-08T05:42:55.957 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBZEDtmMrRVGBAAeKBtcf5fSQMFE6IStUvokQ==", 2024-05-08T05:42:55.957 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:55.957 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:42:55.957 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:42:55.957 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:55.957 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:55.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:55.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:55.958 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2024-05-08T05:42:55.958 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBcEDtmpozQFhAAS6AF6X592Wh/QiBipcOVew==", 2024-05-08T05:42:55.958 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:55.958 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:42:55.958 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:42:55.958 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:55.958 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:55.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:55.958 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:55.958 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2024-05-08T05:42:55.958 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBfEDtm47lSFRAAt5tvpRCljxGtAmHQqcFj2A==", 2024-05-08T05:42:55.958 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:55.958 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:42:55.958 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:42:55.958 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:55.959 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:55.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:55.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:55.959 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2024-05-08T05:42:55.959 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBlEDtmDQ7mGxAA5z7rHZRf7X5g7ZiqEoO2DA==", 2024-05-08T05:42:55.959 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:55.959 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:42:55.959 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:42:55.959 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:55.959 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:55.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:55.959 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:55.959 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2024-05-08T05:42:55.959 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBoEDtmBdi0GhAAVMfg+KEJHRvaiS49PEboOg==", 2024-05-08T05:42:55.959 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:55.960 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:42:55.960 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:42:55.960 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:55.960 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:55.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:55.960 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:55.960 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2024-05-08T05:42:55.960 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBrEDtmG2daGRAANQgJFAT6gNE/vPofCv6veA==", 2024-05-08T05:42:55.960 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:55.960 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:42:55.960 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:42:55.960 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:55.960 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:55.960 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:55.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:55.961 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2024-05-08T05:42:55.961 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBuEDtm3tmDGRAARsAzm9KIzgKAs4ZtfCVdaQ==", 2024-05-08T05:42:55.961 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:55.961 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:42:55.961 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:42:55.961 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:55.961 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:55.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:55.961 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:55.961 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2024-05-08T05:42:55.961 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmDcqGMxAALgQaBYp4Zd8uMqswCXvGaw==", 2024-05-08T05:42:55.961 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:55.961 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:42:55.961 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2024-05-08T05:42:55.962 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T05:42:55.962 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:55.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:55.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:55.962 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2024-05-08T05:42:55.962 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBVEDtm6z0GARAARgRma5hPpH6X95pE5NNrQw==", 2024-05-08T05:42:55.962 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:55.962 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:42:55.962 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:42:55.962 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:42:55.962 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:42:55.962 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:55.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:55.962 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:55.962 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2024-05-08T05:42:55.962 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmAMtIORAAJf8oSwHqSzgjsTuqxtQqHQ==", 2024-05-08T05:42:55.963 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:55.963 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:42:55.963 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:42:55.963 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:42:55.963 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:42:55.963 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:55.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:55.963 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:55.963 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2024-05-08T05:42:55.963 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBVEDtm9qK1BhAAokvkJZmNxT5WER9/+EKIpA==", 2024-05-08T05:42:55.963 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:55.963 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:42:55.963 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:42:55.963 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:42:55.963 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:42:55.963 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:55.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:55.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:55.964 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2024-05-08T05:42:55.964 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmZ4Q3ABAANcqmwOmt+dNJ5wOiPHHO4Q==", 2024-05-08T05:42:55.964 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:55.964 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:42:55.964 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:42:55.964 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:42:55.964 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:55.964 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:55.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:55.964 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:55.964 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2024-05-08T05:42:55.964 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmL0XLGBAA1BSFdyiDYV9Je+igOVBvlQ==", 2024-05-08T05:42:55.964 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:55.964 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:42:55.964 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-05-08T05:42:55.965 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:55.965 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:55.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:42:55.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:55.965 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2024-05-08T05:42:55.965 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmfM5OFRAAgdJPUfqmnKU8vRrjGMNwrw==", 2024-05-08T05:42:55.965 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:42:55.965 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:42:55.965 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-05-08T05:42:55.965 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:42:55.965 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:55.965 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:55.965 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-05-08T05:42:55.965 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:42:55.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.955+0000 7f5670b40640 1 -- 172.21.15.45:0/724195918 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f564c0762d0 msgr2=0x7f564c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:55.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.955+0000 7f5670b40640 1 --2- 172.21.15.45:0/724195918 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f564c0762d0 0x7f564c078790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f565c000f60 tx=0x7f565c009040 comp rx=0 tx=0).stop 2024-05-08T05:42:55.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.955+0000 7f5670b40640 1 -- 172.21.15.45:0/724195918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c163920 msgr2=0x7f566c14d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:55.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.955+0000 7f5670b40640 1 --2- 172.21.15.45:0/724195918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c163920 0x7f566c14d260 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5664009e50 tx=0x7f5664004a40 comp rx=0 tx=0).stop 2024-05-08T05:42:55.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.955+0000 7f5670b40640 1 -- 172.21.15.45:0/724195918 shutdown_connections 2024-05-08T05:42:55.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.955+0000 7f5670b40640 1 --2- 172.21.15.45:0/724195918 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f564c0762d0 0x7f564c078790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:55.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.955+0000 7f5670b40640 1 --2- 172.21.15.45:0/724195918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f566c0d20c0 0x7f566c1f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:55.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.955+0000 7f5670b40640 1 --2- 172.21.15.45:0/724195918 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f566c163920 0x7f566c14d260 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:55.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.955+0000 7f5670b40640 1 --2- 172.21.15.45:0/724195918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f566c070d30 0x7f566c0b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:55.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.955+0000 7f5670b40640 1 -- 172.21.15.45:0/724195918 >> 172.21.15.45:0/724195918 conn(0x7f566c0d2e90 msgr2=0x7f566c15b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:55.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.955+0000 7f5670b40640 1 -- 172.21.15.45:0/724195918 shutdown_connections 2024-05-08T05:42:55.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:55.955+0000 7f5670b40640 1 -- 172.21.15.45:0/724195918 wait complete. 2024-05-08T05:42:55.968 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T05:42:56.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.071+0000 7f5452445640 1 Processor -- start 2024-05-08T05:42:56.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.072+0000 7f5452445640 1 -- start start 2024-05-08T05:42:56.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.072+0000 7f5452445640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f544c15f420 0x7f544c0d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:56.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.072+0000 7f5452445640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f544c1630b0 con 0x7f544c0d2080 2024-05-08T05:42:56.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.072+0000 7f5452445640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f544c0d2550 con 0x7f544c0d3ca0 2024-05-08T05:42:56.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.072+0000 7f5452445640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f544c0d2c80 con 0x7f544c15f420 2024-05-08T05:42:56.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.072+0000 7f544bfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f544c15f420 0x7f544c0d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:56.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.073+0000 7f54509e2640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f544c0d2080 0x7f544c0d4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36648/0 (socket says 172.21.15.45:36648) 2024-05-08T05:42:56.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.073+0000 7f54509e2640 1 -- 172.21.15.45:0/4186222928 learned_addr learned my addr 172.21.15.45:0/4186222928 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:56.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.073+0000 7f544bfff640 1 -- 172.21.15.45:0/4186222928 >> v1:172.21.15.63:6789/0 conn(0x7f544c0d3ca0 legacy=0x7f544c0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:56.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.073+0000 7f544bfff640 1 -- 172.21.15.45:0/4186222928 >> v1:172.21.15.45:6789/0 conn(0x7f544c0d2080 legacy=0x7f544c0d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:56.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.073+0000 7f544bfff640 1 -- 172.21.15.45:0/4186222928 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544c0d2f60 con 0x7f544c15f420 2024-05-08T05:42:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.074+0000 7f544bfff640 1 --2- 172.21.15.45:0/4186222928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f544c15f420 0x7f544c0d3760 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f543c0097a0 tx=0x7f543c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d186c26c86952e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.074+0000 7f544affd640 1 -- 172.21.15.45:0/4186222928 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c037070 con 0x7f544c15f420 2024-05-08T05:42:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.074+0000 7f544affd640 1 -- 172.21.15.45:0/4186222928 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f543c032440 con 0x7f544c15f420 2024-05-08T05:42:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.074+0000 7f544affd640 1 -- 172.21.15.45:0/4186222928 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c032810 con 0x7f544c15f420 2024-05-08T05:42:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.074+0000 7f5452445640 1 -- 172.21.15.45:0/4186222928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f544c15f420 msgr2=0x7f544c0d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:56.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.074+0000 7f5452445640 1 --2- 172.21.15.45:0/4186222928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f544c15f420 0x7f544c0d3760 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f543c0097a0 tx=0x7f543c02a180 comp rx=0 tx=0).stop 2024-05-08T05:42:56.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.074+0000 7f5452445640 1 -- 172.21.15.45:0/4186222928 shutdown_connections 2024-05-08T05:42:56.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.074+0000 7f5452445640 1 --2- 172.21.15.45:0/4186222928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f544c15f420 0x7f544c0d3760 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.074+0000 7f5452445640 1 -- 172.21.15.45:0/4186222928 >> 172.21.15.45:0/4186222928 conn(0x7f544c08eac0 msgr2=0x7f544c09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:56.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.074+0000 7f5452445640 1 -- 172.21.15.45:0/4186222928 shutdown_connections 2024-05-08T05:42:56.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.074+0000 7f5452445640 1 -- 172.21.15.45:0/4186222928 wait complete. 2024-05-08T05:42:56.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.076+0000 7f5452445640 1 Processor -- start 2024-05-08T05:42:56.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.076+0000 7f5452445640 1 -- start start 2024-05-08T05:42:56.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.076+0000 7f5452445640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f544c0d2080 0x7f544c1618b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:56.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.076+0000 7f5452445640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f544c15dfc0 0x7f544c0b8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:56.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.076+0000 7f5452445640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f544c15d850 0x7f544c0d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:56.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.076+0000 7f5452445640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f544c0d53d0 con 0x7f544c0d2080 2024-05-08T05:42:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.076+0000 7f5452445640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f544c0d2830 con 0x7f544c15d850 2024-05-08T05:42:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.076+0000 7f5452445640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f544c0d5550 con 0x7f544c15dfc0 2024-05-08T05:42:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.077+0000 7f544bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f544c0d2080 0x7f544c1618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.077+0000 7f544bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f544c0d2080 0x7f544c1618b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50726/0 (socket says 172.21.15.45:50726) 2024-05-08T05:42:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.077+0000 7f544bfff640 1 -- 172.21.15.45:0/4252053794 learned_addr learned my addr 172.21.15.45:0/4252053794 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.077+0000 7f54509e2640 1 --2- 172.21.15.45:0/4252053794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f544c15d850 0x7f544c0d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.077+0000 7f544b7fe640 1 --2- 172.21.15.45:0/4252053794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f544c15dfc0 0x7f544c0b8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.077+0000 7f544bfff640 1 -- 172.21.15.45:0/4252053794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f544c15dfc0 msgr2=0x7f544c0b8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.077+0000 7f544bfff640 1 --2- 172.21.15.45:0/4252053794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f544c15dfc0 0x7f544c0b8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.077+0000 7f544bfff640 1 -- 172.21.15.45:0/4252053794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f544c15d850 msgr2=0x7f544c0d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.077+0000 7f544bfff640 1 --2- 172.21.15.45:0/4252053794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f544c15d850 0x7f544c0d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.077+0000 7f544bfff640 1 -- 172.21.15.45:0/4252053794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544c0b86c0 con 0x7f544c0d2080 2024-05-08T05:42:56.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.077+0000 7f544b7fe640 1 --2- 172.21.15.45:0/4252053794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f544c15dfc0 0x7f544c0b8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:56.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.077+0000 7f544bfff640 1 --2- 172.21.15.45:0/4252053794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f544c0d2080 0x7f544c1618b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f543c0097a0 tx=0x7f543c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:56.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.077+0000 7f5448ff9640 1 -- 172.21.15.45:0/4252053794 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c041070 con 0x7f544c0d2080 2024-05-08T05:42:56.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.077+0000 7f5448ff9640 1 -- 172.21.15.45:0/4252053794 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f543c03c870 con 0x7f544c0d2080 2024-05-08T05:42:56.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.077+0000 7f5452445640 1 -- 172.21.15.45:0/4252053794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f544c161e90 con 0x7f544c0d2080 2024-05-08T05:42:56.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.077+0000 7f5448ff9640 1 -- 172.21.15.45:0/4252053794 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c037040 con 0x7f544c0d2080 2024-05-08T05:42:56.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.077+0000 7f5452445640 1 -- 172.21.15.45:0/4252053794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f544c162450 con 0x7f544c0d2080 2024-05-08T05:42:56.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.078+0000 7f5452445640 1 -- 172.21.15.45:0/4252053794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f544c0693f0 con 0x7f544c0d2080 2024-05-08T05:42:56.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.079+0000 7f5448ff9640 1 -- 172.21.15.45:0/4252053794 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f543c044070 con 0x7f544c0d2080 2024-05-08T05:42:56.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.080+0000 7f5448ff9640 1 --2- 172.21.15.45:0/4252053794 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f54380762d0 0x7f5438078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:56.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.080+0000 7f5448ff9640 1 -- 172.21.15.45:0/4252053794 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5249+0+0 (secure 0 0 0) 0x7f543c0b7fb0 con 0x7f544c0d2080 2024-05-08T05:42:56.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.080+0000 7f544b7fe640 1 --2- 172.21.15.45:0/4252053794 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f54380762d0 0x7f5438078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:56.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.081+0000 7f544b7fe640 1 --2- 172.21.15.45:0/4252053794 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f54380762d0 0x7f5438078790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f544c0ba730 tx=0x7f5440009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:56.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.084+0000 7f5448ff9640 1 -- 172.21.15.45:0/4252053794 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f543c081ad0 con 0x7f544c0d2080 2024-05-08T05:42:56.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.284+0000 7f5452445640 1 -- 172.21.15.45:0/4252053794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f544c0b4c50 con 0x7f544c0d2080 2024-05-08T05:42:56.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.284+0000 7f5448ff9640 1 -- 172.21.15.45:0/4252053794 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v43) ==== 84+0+65 (secure 0 0 0) 0x7f543c0813d0 con 0x7f544c0d2080 2024-05-08T05:42:56.285 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:56.285 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-05-08T05:42:56.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:42:56.285 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-05-08T05:42:56.285 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-05-08T05:42:56.285 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:42:56.285 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-05-08T05:42:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.286+0000 7f5452445640 1 -- 172.21.15.45:0/4252053794 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f54380762d0 msgr2=0x7f5438078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.286+0000 7f5452445640 1 --2- 172.21.15.45:0/4252053794 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f54380762d0 0x7f5438078790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f544c0ba730 tx=0x7f5440009040 comp rx=0 tx=0).stop 2024-05-08T05:42:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.286+0000 7f5452445640 1 -- 172.21.15.45:0/4252053794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f544c0d2080 msgr2=0x7f544c1618b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.286+0000 7f5452445640 1 --2- 172.21.15.45:0/4252053794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f544c0d2080 0x7f544c1618b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f543c0097a0 tx=0x7f543c0046c0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.286+0000 7f5452445640 1 -- 172.21.15.45:0/4252053794 shutdown_connections 2024-05-08T05:42:56.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.286+0000 7f5452445640 1 --2- 172.21.15.45:0/4252053794 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f54380762d0 0x7f5438078790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.286+0000 7f5452445640 1 --2- 172.21.15.45:0/4252053794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f544c15dfc0 0x7f544c0b8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.286+0000 7f5452445640 1 --2- 172.21.15.45:0/4252053794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f544c15d850 0x7f544c0d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.286+0000 7f5452445640 1 --2- 172.21.15.45:0/4252053794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f544c0d2080 0x7f544c1618b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.286+0000 7f5452445640 1 -- 172.21.15.45:0/4252053794 >> 172.21.15.45:0/4252053794 conn(0x7f544c08eac0 msgr2=0x7f544c163290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.287+0000 7f5452445640 1 -- 172.21.15.45:0/4252053794 shutdown_connections 2024-05-08T05:42:56.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.287+0000 7f5452445640 1 -- 172.21.15.45:0/4252053794 wait complete. 2024-05-08T05:42:56.297 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T05:42:56.298 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T05:42:56.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.405+0000 7f7996e54640 1 Processor -- start 2024-05-08T05:42:56.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.405+0000 7f7996e54640 1 -- start start 2024-05-08T05:42:56.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.406+0000 7f7996e54640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f79900d3920 0x7f7990070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:56.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.406+0000 7f7996e54640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f79900cef40 0x7f7990169b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:56.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.406+0000 7f7996e54640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79900d2030 con 0x7f79900cf8f0 2024-05-08T05:42:56.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.406+0000 7f7996e54640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f799015f8d0 con 0x7f79900d3920 2024-05-08T05:42:56.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.406+0000 7f7996e54640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f79900d22b0 con 0x7f79900cef40 2024-05-08T05:42:56.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.406+0000 7f798ffff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f79900d3920 0x7f7990070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:56.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.406+0000 7f798ffff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f79900d3920 0x7f7990070c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50730/0 (socket says 172.21.15.45:50730) 2024-05-08T05:42:56.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.406+0000 7f798ffff640 1 -- 172.21.15.45:0/3600006500 learned_addr learned my addr 172.21.15.45:0/3600006500 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:56.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.407+0000 7f79953f1640 1 --2- 172.21.15.45:0/3600006500 >> v2:172.21.15.63:3300/0 conn(0x7f79900cef40 0x7f7990169b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:56.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.407+0000 7f798ffff640 1 -- 172.21.15.45:0/3600006500 >> v2:172.21.15.63:3300/0 conn(0x7f79900cef40 msgr2=0x7f7990169b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:56.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.407+0000 7f798ffff640 1 --2- 172.21.15.45:0/3600006500 >> v2:172.21.15.63:3300/0 conn(0x7f79900cef40 0x7f7990169b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.407+0000 7f798ffff640 1 -- 172.21.15.45:0/3600006500 >> v1:172.21.15.63:6789/0 conn(0x7f79900cf8f0 legacy=0x7f7990070500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:56.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.407+0000 7f798ffff640 1 -- 172.21.15.45:0/3600006500 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79900d2590 con 0x7f79900d3920 2024-05-08T05:42:56.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.407+0000 7f798ffff640 1 --2- 172.21.15.45:0/3600006500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79900d3920 0x7f7990070c10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7990073c40 tx=0x7f798402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf6373d1370ce9d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:56.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.408+0000 7f798f7fe640 1 -- 172.21.15.45:0/3600006500 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984037070 con 0x7f79900d3920 2024-05-08T05:42:56.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.408+0000 7f798f7fe640 1 -- 172.21.15.45:0/3600006500 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7984032440 con 0x7f79900d3920 2024-05-08T05:42:56.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.408+0000 7f798f7fe640 1 -- 172.21.15.45:0/3600006500 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984032810 con 0x7f79900d3920 2024-05-08T05:42:56.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.408+0000 7f7996e54640 1 -- 172.21.15.45:0/3600006500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79900d3920 msgr2=0x7f7990070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:56.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.408+0000 7f7996e54640 1 --2- 172.21.15.45:0/3600006500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79900d3920 0x7f7990070c10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7990073c40 tx=0x7f798402a270 comp rx=0 tx=0).stop 2024-05-08T05:42:56.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.408+0000 7f7996e54640 1 -- 172.21.15.45:0/3600006500 shutdown_connections 2024-05-08T05:42:56.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.408+0000 7f7996e54640 1 --2- 172.21.15.45:0/3600006500 >> v2:172.21.15.63:3300/0 conn(0x7f79900cef40 0x7f7990169b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.408+0000 7f7996e54640 1 --2- 172.21.15.45:0/3600006500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79900d3920 0x7f7990070c10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.408+0000 7f7996e54640 1 -- 172.21.15.45:0/3600006500 >> 172.21.15.45:0/3600006500 conn(0x7f79900d2e00 msgr2=0x7f79900d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:56.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.408+0000 7f7996e54640 1 -- 172.21.15.45:0/3600006500 shutdown_connections 2024-05-08T05:42:56.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.408+0000 7f7996e54640 1 -- 172.21.15.45:0/3600006500 wait complete. 2024-05-08T05:42:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.410+0000 7f7996e54640 1 Processor -- start 2024-05-08T05:42:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.410+0000 7f7996e54640 1 -- start start 2024-05-08T05:42:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.410+0000 7f7996e54640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79900cef40 0x7f799015a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.410+0000 7f7996e54640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79900cf8f0 0x7f79900cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.410+0000 7f7996e54640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79900d3920 0x7f79900cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.410+0000 7f7996e54640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f79900cda10 con 0x7f79900cf8f0 2024-05-08T05:42:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.410+0000 7f7996e54640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f79900cd730 con 0x7f79900cef40 2024-05-08T05:42:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.410+0000 7f7996e54640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7990071610 con 0x7f79900d3920 2024-05-08T05:42:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.411+0000 7f798ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79900cf8f0 0x7f79900cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.411+0000 7f798ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79900cf8f0 0x7f79900cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50736/0 (socket says 172.21.15.45:50736) 2024-05-08T05:42:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.411+0000 7f798ffff640 1 -- 172.21.15.45:0/2602988660 learned_addr learned my addr 172.21.15.45:0/2602988660 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.411+0000 7f79953f1640 1 --2- 172.21.15.45:0/2602988660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79900d3920 0x7f79900cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.411+0000 7f7994bf0640 1 --2- 172.21.15.45:0/2602988660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79900cef40 0x7f799015a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.411+0000 7f798ffff640 1 -- 172.21.15.45:0/2602988660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79900d3920 msgr2=0x7f79900cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.411+0000 7f798ffff640 1 --2- 172.21.15.45:0/2602988660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79900d3920 0x7f79900cba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.411+0000 7f798ffff640 1 -- 172.21.15.45:0/2602988660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79900cef40 msgr2=0x7f799015a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.411+0000 7f798ffff640 1 --2- 172.21.15.45:0/2602988660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79900cef40 0x7f799015a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.411+0000 7f798ffff640 1 -- 172.21.15.45:0/2602988660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799015a8c0 con 0x7f79900cf8f0 2024-05-08T05:42:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.411+0000 7f7994bf0640 1 --2- 172.21.15.45:0/2602988660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79900cef40 0x7f799015a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.411+0000 7f798ffff640 1 --2- 172.21.15.45:0/2602988660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79900cf8f0 0x7f79900cb4c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f798402a2b0 tx=0x7f79840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.411+0000 7f798d7fa640 1 -- 172.21.15.45:0/2602988660 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984041020 con 0x7f79900cf8f0 2024-05-08T05:42:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.411+0000 7f798d7fa640 1 -- 172.21.15.45:0/2602988660 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79840306c0 con 0x7f79900cf8f0 2024-05-08T05:42:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.411+0000 7f7996e54640 1 -- 172.21.15.45:0/2602988660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79901f63d0 con 0x7f79900cf8f0 2024-05-08T05:42:56.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.411+0000 7f798d7fa640 1 -- 172.21.15.45:0/2602988660 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984037040 con 0x7f79900cf8f0 2024-05-08T05:42:56.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.411+0000 7f7996e54640 1 -- 172.21.15.45:0/2602988660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79900c6bf0 con 0x7f79900cf8f0 2024-05-08T05:42:56.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.412+0000 7f7996e54640 1 -- 172.21.15.45:0/2602988660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79900693f0 con 0x7f79900cf8f0 2024-05-08T05:42:56.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.413+0000 7f798d7fa640 1 -- 172.21.15.45:0/2602988660 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7984044020 con 0x7f79900cf8f0 2024-05-08T05:42:56.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.414+0000 7f798d7fa640 1 --2- 172.21.15.45:0/2602988660 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f797c0762d0 0x7f797c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:56.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.414+0000 7f798d7fa640 1 -- 172.21.15.45:0/2602988660 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5249+0+0 (secure 0 0 0) 0x7f79840b7c30 con 0x7f79900cf8f0 2024-05-08T05:42:56.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.414+0000 7f7994bf0640 1 --2- 172.21.15.45:0/2602988660 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f797c0762d0 0x7f797c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:56.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.414+0000 7f7994bf0640 1 --2- 172.21.15.45:0/2602988660 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f797c0762d0 0x7f797c078790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f798000a830 tx=0x7f7980008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:56.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.418+0000 7f798d7fa640 1 -- 172.21.15.45:0/2602988660 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f798404b030 con 0x7f79900cf8f0 2024-05-08T05:42:56.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:56.610+0000 7f7996e54640 1 -- 172.21.15.45:0/2602988660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f79900d0690 con 0x7f79900cf8f0 2024-05-08T05:42:57.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.551+0000 7f798d7fa640 1 -- 172.21.15.45:0/2602988660 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v44) ==== 138+0+0 (secure 0 0 0) 0x7f79900d2030 con 0x7f79900cf8f0 2024-05-08T05:42:57.552 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-05-08T05:42:57.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.555+0000 7f7996e54640 1 -- 172.21.15.45:0/2602988660 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f797c0762d0 msgr2=0x7f797c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:57.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.555+0000 7f7996e54640 1 --2- 172.21.15.45:0/2602988660 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f797c0762d0 0x7f797c078790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f798000a830 tx=0x7f7980008040 comp rx=0 tx=0).stop 2024-05-08T05:42:57.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.555+0000 7f7996e54640 1 -- 172.21.15.45:0/2602988660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79900cf8f0 msgr2=0x7f79900cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:57.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.555+0000 7f7996e54640 1 --2- 172.21.15.45:0/2602988660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79900cf8f0 0x7f79900cb4c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f798402a2b0 tx=0x7f79840046c0 comp rx=0 tx=0).stop 2024-05-08T05:42:57.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.556+0000 7f7996e54640 1 -- 172.21.15.45:0/2602988660 shutdown_connections 2024-05-08T05:42:57.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.556+0000 7f7996e54640 1 --2- 172.21.15.45:0/2602988660 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f797c0762d0 0x7f797c078790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:57.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.556+0000 7f7996e54640 1 --2- 172.21.15.45:0/2602988660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79900d3920 0x7f79900cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:57.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.556+0000 7f7996e54640 1 --2- 172.21.15.45:0/2602988660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79900cf8f0 0x7f79900cb4c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:57.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.556+0000 7f7996e54640 1 --2- 172.21.15.45:0/2602988660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79900cef40 0x7f799015a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:57.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.556+0000 7f7996e54640 1 -- 172.21.15.45:0/2602988660 >> 172.21.15.45:0/2602988660 conn(0x7f79900d2e00 msgr2=0x7f799006de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:57.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.556+0000 7f7996e54640 1 -- 172.21.15.45:0/2602988660 shutdown_connections 2024-05-08T05:42:57.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.556+0000 7f7996e54640 1 -- 172.21.15.45:0/2602988660 wait complete. 2024-05-08T05:42:57.573 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T05:42:57.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.690+0000 7fa7cefef640 1 Processor -- start 2024-05-08T05:42:57.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.690+0000 7fa7cefef640 1 -- start start 2024-05-08T05:42:57.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.691+0000 7fa7cefef640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa7c80b6590 0x7fa7c80d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:57.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.691+0000 7fa7cefef640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7c80d2bb0 0x7fa7c8162d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:57.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.691+0000 7fa7cefef640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7c80d3320 con 0x7fa7c8163290 2024-05-08T05:42:57.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.691+0000 7fa7cefef640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa7c80d3600 con 0x7fa7c80b6590 2024-05-08T05:42:57.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.691+0000 7fa7cefef640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c80d38e0 con 0x7fa7c80d2bb0 2024-05-08T05:42:57.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.691+0000 7fa7ccd8b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa7c80b6590 0x7fa7c80d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:57.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.691+0000 7fa7cd58c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa7c8163290 0x7fa7c815ec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36650/0 (socket says 172.21.15.45:36650) 2024-05-08T05:42:57.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.691+0000 7fa7ccd8b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa7c80b6590 0x7fa7c80d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50746/0 (socket says 172.21.15.45:50746) 2024-05-08T05:42:57.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.691+0000 7fa7cd58c640 1 -- 172.21.15.45:0/1034311872 learned_addr learned my addr 172.21.15.45:0/1034311872 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:57.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.691+0000 7fa7c7fff640 1 --2- 172.21.15.45:0/1034311872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7c80d2bb0 0x7fa7c8162d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:57.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.691+0000 7fa7ccd8b640 1 -- 172.21.15.45:0/1034311872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7c80d2bb0 msgr2=0x7fa7c8162d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:57.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.691+0000 7fa7ccd8b640 1 --2- 172.21.15.45:0/1034311872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7c80d2bb0 0x7fa7c8162d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:57.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.691+0000 7fa7ccd8b640 1 -- 172.21.15.45:0/1034311872 >> v1:172.21.15.45:6789/0 conn(0x7fa7c8163290 legacy=0x7fa7c815ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:57.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.692+0000 7fa7c77fe640 1 -- 172.21.15.45:0/1034311872 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1827473958 0 0) 0x7fa7c80d3320 con 0x7fa7c8163290 2024-05-08T05:42:57.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.692+0000 7fa7ccd8b640 1 -- 172.21.15.45:0/1034311872 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c80d3bc0 con 0x7fa7c80b6590 2024-05-08T05:42:57.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.692+0000 7fa7ccd8b640 1 --2- 172.21.15.45:0/1034311872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7c80b6590 0x7fa7c80d0bd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0097a0 tx=0x7fa7bc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca828189d2062e75 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:57.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.692+0000 7fa7c77fe640 1 -- 172.21.15.45:0/1034311872 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc037070 con 0x7fa7c80b6590 2024-05-08T05:42:57.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.692+0000 7fa7c77fe640 1 -- 172.21.15.45:0/1034311872 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7bc032440 con 0x7fa7c80b6590 2024-05-08T05:42:57.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.692+0000 7fa7c77fe640 1 -- 172.21.15.45:0/1034311872 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc032810 con 0x7fa7c80b6590 2024-05-08T05:42:57.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.693+0000 7fa7cefef640 1 -- 172.21.15.45:0/1034311872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7c80b6590 msgr2=0x7fa7c80d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:57.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.693+0000 7fa7cefef640 1 --2- 172.21.15.45:0/1034311872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7c80b6590 0x7fa7c80d0bd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0097a0 tx=0x7fa7bc02a180 comp rx=0 tx=0).stop 2024-05-08T05:42:57.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.693+0000 7fa7cefef640 1 -- 172.21.15.45:0/1034311872 shutdown_connections 2024-05-08T05:42:57.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.693+0000 7fa7cefef640 1 --2- 172.21.15.45:0/1034311872 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7c80d2bb0 0x7fa7c8162d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:57.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.693+0000 7fa7cefef640 1 --2- 172.21.15.45:0/1034311872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7c80b6590 0x7fa7c80d0bd0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:57.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.693+0000 7fa7cefef640 1 -- 172.21.15.45:0/1034311872 >> 172.21.15.45:0/1034311872 conn(0x7fa7c80d4fe0 msgr2=0x7fa7c80d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:57.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.693+0000 7fa7cefef640 1 -- 172.21.15.45:0/1034311872 shutdown_connections 2024-05-08T05:42:57.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.693+0000 7fa7cefef640 1 -- 172.21.15.45:0/1034311872 wait complete. 2024-05-08T05:42:57.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.694+0000 7fa7cefef640 1 Processor -- start 2024-05-08T05:42:57.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.694+0000 7fa7cefef640 1 -- start start 2024-05-08T05:42:57.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.695+0000 7fa7cefef640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7c80b6590 0x7fa7c80ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:57.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.695+0000 7fa7cefef640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7c80d2bb0 0x7fa7c80cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:57.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.695+0000 7fa7cefef640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7c8163290 0x7fa7c815cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:57.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.695+0000 7fa7cefef640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c815f340 con 0x7fa7c80d2bb0 2024-05-08T05:42:57.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.695+0000 7fa7cefef640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c80d3010 con 0x7fa7c80b6590 2024-05-08T05:42:57.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.695+0000 7fa7cefef640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c815f4c0 con 0x7fa7c8163290 2024-05-08T05:42:57.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.695+0000 7fa7c7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7c80d2bb0 0x7fa7c80cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:57.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.695+0000 7fa7c7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7c80d2bb0 0x7fa7c80cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50750/0 (socket says 172.21.15.45:50750) 2024-05-08T05:42:57.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.695+0000 7fa7c7fff640 1 -- 172.21.15.45:0/1153419156 learned_addr learned my addr 172.21.15.45:0/1153419156 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:57.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.696+0000 7fa7ccd8b640 1 --2- 172.21.15.45:0/1153419156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7c80b6590 0x7fa7c80ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:57.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.696+0000 7fa7c7fff640 1 -- 172.21.15.45:0/1153419156 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7c8163290 msgr2=0x7fa7c815cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:57.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.696+0000 7fa7c7fff640 1 --2- 172.21.15.45:0/1153419156 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7c8163290 0x7fa7c815cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:57.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.696+0000 7fa7c7fff640 1 -- 172.21.15.45:0/1153419156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7c80b6590 msgr2=0x7fa7c80ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:57.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.696+0000 7fa7c7fff640 1 --2- 172.21.15.45:0/1153419156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7c80b6590 0x7fa7c80ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:57.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.696+0000 7fa7c7fff640 1 -- 172.21.15.45:0/1153419156 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c81f91a0 con 0x7fa7c80d2bb0 2024-05-08T05:42:57.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.696+0000 7fa7c7fff640 1 --2- 172.21.15.45:0/1153419156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7c80d2bb0 0x7fa7c80cc0b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400d800 tx=0x7fa7b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:57.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.696+0000 7fa7c57fa640 1 -- 172.21.15.45:0/1153419156 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b400b6c0 con 0x7fa7c80d2bb0 2024-05-08T05:42:57.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.696+0000 7fa7c57fa640 1 -- 172.21.15.45:0/1153419156 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b4019070 con 0x7fa7c80d2bb0 2024-05-08T05:42:57.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.696+0000 7fa7cefef640 1 -- 172.21.15.45:0/1153419156 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c80c9ea0 con 0x7fa7c80d2bb0 2024-05-08T05:42:57.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.696+0000 7fa7c57fa640 1 -- 172.21.15.45:0/1153419156 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b400ba80 con 0x7fa7c80d2bb0 2024-05-08T05:42:57.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.696+0000 7fa7cefef640 1 -- 172.21.15.45:0/1153419156 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c80ca460 con 0x7fa7c80d2bb0 2024-05-08T05:42:57.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.697+0000 7fa7cefef640 1 -- 172.21.15.45:0/1153419156 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c80693f0 con 0x7fa7c80d2bb0 2024-05-08T05:42:57.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.698+0000 7fa7c57fa640 1 -- 172.21.15.45:0/1153419156 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa7b400bc20 con 0x7fa7c80d2bb0 2024-05-08T05:42:57.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.698+0000 7fa7c57fa640 1 --2- 172.21.15.45:0/1153419156 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa7ac0762d0 0x7fa7ac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:57.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.698+0000 7fa7c57fa640 1 -- 172.21.15.45:0/1153419156 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5619+0+0 (secure 0 0 0) 0x7fa7b4097a90 con 0x7fa7c80d2bb0 2024-05-08T05:42:57.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.698+0000 7fa7ccd8b640 1 --2- 172.21.15.45:0/1153419156 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa7ac0762d0 0x7fa7ac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:57.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.699+0000 7fa7ccd8b640 1 --2- 172.21.15.45:0/1153419156 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa7ac0762d0 0x7fa7ac078790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc034000 tx=0x7fa7bc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:57.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.703+0000 7fa7c57fa640 1 -- 172.21.15.45:0/1153419156 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa7b4061440 con 0x7fa7c80d2bb0 2024-05-08T05:42:57.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:57.893+0000 7fa7cefef640 1 -- 172.21.15.45:0/1153419156 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa7c8163070 con 0x7fa7c80d2bb0 2024-05-08T05:42:58.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.554+0000 7fa7c57fa640 1 -- 172.21.15.45:0/1153419156 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v45) ==== 170+0+0 (secure 0 0 0) 0x7fa7b4060d40 con 0x7fa7c80d2bb0 2024-05-08T05:42:58.554 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-05-08T05:42:58.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.557+0000 7fa7cefef640 1 -- 172.21.15.45:0/1153419156 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa7ac0762d0 msgr2=0x7fa7ac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:58.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.557+0000 7fa7cefef640 1 --2- 172.21.15.45:0/1153419156 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa7ac0762d0 0x7fa7ac078790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc034000 tx=0x7fa7bc02e000 comp rx=0 tx=0).stop 2024-05-08T05:42:58.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.557+0000 7fa7cefef640 1 -- 172.21.15.45:0/1153419156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7c80d2bb0 msgr2=0x7fa7c80cc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:58.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.557+0000 7fa7cefef640 1 --2- 172.21.15.45:0/1153419156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7c80d2bb0 0x7fa7c80cc0b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400d800 tx=0x7fa7b400dcc0 comp rx=0 tx=0).stop 2024-05-08T05:42:58.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.557+0000 7fa7cefef640 1 -- 172.21.15.45:0/1153419156 shutdown_connections 2024-05-08T05:42:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.557+0000 7fa7cefef640 1 --2- 172.21.15.45:0/1153419156 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa7ac0762d0 0x7fa7ac078790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.557+0000 7fa7cefef640 1 --2- 172.21.15.45:0/1153419156 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7c8163290 0x7fa7c815cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.557+0000 7fa7cefef640 1 --2- 172.21.15.45:0/1153419156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7c80d2bb0 0x7fa7c80cc0b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.557+0000 7fa7cefef640 1 --2- 172.21.15.45:0/1153419156 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7c80b6590 0x7fa7c80ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.557+0000 7fa7cefef640 1 -- 172.21.15.45:0/1153419156 >> 172.21.15.45:0/1153419156 conn(0x7fa7c80d4fe0 msgr2=0x7fa7c815dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.557+0000 7fa7cefef640 1 -- 172.21.15.45:0/1153419156 shutdown_connections 2024-05-08T05:42:58.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.557+0000 7fa7cefef640 1 -- 172.21.15.45:0/1153419156 wait complete. 2024-05-08T05:42:58.583 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T05:42:58.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.695+0000 7f027bfb6640 1 Processor -- start 2024-05-08T05:42:58.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.696+0000 7f027bfb6640 1 -- start start 2024-05-08T05:42:58.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.696+0000 7f027bfb6640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f027415f4e0 0x7f02740cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:58.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.696+0000 7f027bfb6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f02740d56b0 0x7f02740d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:58.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.696+0000 7f027bfb6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02740d0320 con 0x7f02740d3aa0 2024-05-08T05:42:58.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.696+0000 7f027bfb6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f02740d0600 con 0x7f02740d56b0 2024-05-08T05:42:58.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.696+0000 7f027bfb6640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f02740d08e0 con 0x7f027415f4e0 2024-05-08T05:42:58.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.697+0000 7f027a553640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f02740d56b0 0x7f02740d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:58.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.697+0000 7f027a553640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f02740d56b0 0x7f02740d13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50764/0 (socket says 172.21.15.45:50764) 2024-05-08T05:42:58.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.697+0000 7f027a553640 1 -- 172.21.15.45:0/271027125 learned_addr learned my addr 172.21.15.45:0/271027125 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:58.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.697+0000 7f0279551640 1 --1- 172.21.15.45:0/271027125 >> v1:172.21.15.45:6789/0 conn(0x7f02740d3aa0 0x7f02740d4fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36664/0 (socket says 172.21.15.45:36664) 2024-05-08T05:42:58.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.697+0000 7f0279d52640 1 --2- 172.21.15.45:0/271027125 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f027415f4e0 0x7f02740cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:58.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.697+0000 7f027a553640 1 -- 172.21.15.45:0/271027125 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f027415f4e0 msgr2=0x7f02740cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:58.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.697+0000 7f027a553640 1 --2- 172.21.15.45:0/271027125 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f027415f4e0 0x7f02740cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:58.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.697+0000 7f027a553640 1 -- 172.21.15.45:0/271027125 >> v1:172.21.15.45:6789/0 conn(0x7f02740d3aa0 legacy=0x7f02740d4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:58.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.697+0000 7f0278d50640 1 -- 172.21.15.45:0/271027125 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1875756822 0 0) 0x7f02740d0320 con 0x7f02740d3aa0 2024-05-08T05:42:58.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.697+0000 7f027a553640 1 -- 172.21.15.45:0/271027125 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02740d0bc0 con 0x7f02740d56b0 2024-05-08T05:42:58.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.697+0000 7f0279d52640 1 --2- 172.21.15.45:0/271027125 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f027415f4e0 0x7f02740cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:42:58.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.698+0000 7f027a553640 1 --2- 172.21.15.45:0/271027125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02740d56b0 0x7f02740d13c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0264009e50 tx=0x7f026402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec279e50899250d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:58.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.698+0000 7f0278d50640 1 -- 172.21.15.45:0/271027125 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0264036070 con 0x7f02740d56b0 2024-05-08T05:42:58.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.698+0000 7f0278d50640 1 -- 172.21.15.45:0/271027125 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0264031600 con 0x7f02740d56b0 2024-05-08T05:42:58.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.698+0000 7f0278d50640 1 -- 172.21.15.45:0/271027125 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02640319d0 con 0x7f02740d56b0 2024-05-08T05:42:58.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.698+0000 7f027bfb6640 1 -- 172.21.15.45:0/271027125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02740d56b0 msgr2=0x7f02740d13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:58.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.698+0000 7f027bfb6640 1 --2- 172.21.15.45:0/271027125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02740d56b0 0x7f02740d13c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0264009e50 tx=0x7f026402a710 comp rx=0 tx=0).stop 2024-05-08T05:42:58.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.698+0000 7f027bfb6640 1 -- 172.21.15.45:0/271027125 shutdown_connections 2024-05-08T05:42:58.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.698+0000 7f027bfb6640 1 --2- 172.21.15.45:0/271027125 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02740d56b0 0x7f02740d13c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:58.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.698+0000 7f027bfb6640 1 --2- 172.21.15.45:0/271027125 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f027415f4e0 0x7f02740cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:58.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.698+0000 7f027bfb6640 1 -- 172.21.15.45:0/271027125 >> 172.21.15.45:0/271027125 conn(0x7f027408eac0 msgr2=0x7f027409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:58.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.699+0000 7f027bfb6640 1 -- 172.21.15.45:0/271027125 shutdown_connections 2024-05-08T05:42:58.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.699+0000 7f027bfb6640 1 -- 172.21.15.45:0/271027125 wait complete. 2024-05-08T05:42:58.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.700+0000 7f027bfb6640 1 Processor -- start 2024-05-08T05:42:58.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.700+0000 7f027bfb6640 1 -- start start 2024-05-08T05:42:58.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.700+0000 7f027bfb6640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02740d3aa0 0x7f027414f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:58.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.700+0000 7f027bfb6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02740d56b0 0x7f02740b7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:58.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.700+0000 7f027bfb6640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f027415f4e0 0x7f02740b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.700+0000 7f027bfb6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f02740d3f00 con 0x7f02740d56b0 2024-05-08T05:42:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.700+0000 7f027bfb6640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f027415f8c0 con 0x7f02740d3aa0 2024-05-08T05:42:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.700+0000 7f027bfb6640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f02740d4080 con 0x7f027415f4e0 2024-05-08T05:42:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.701+0000 7f0279551640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02740d56b0 0x7f02740b7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.701+0000 7f0279551640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02740d56b0 0x7f02740b7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50772/0 (socket says 172.21.15.45:50772) 2024-05-08T05:42:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.701+0000 7f0279551640 1 -- 172.21.15.45:0/2906862271 learned_addr learned my addr 172.21.15.45:0/2906862271 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.701+0000 7f0279d52640 1 --2- 172.21.15.45:0/2906862271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02740d3aa0 0x7f027414f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.701+0000 7f027a553640 1 --2- 172.21.15.45:0/2906862271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f027415f4e0 0x7f02740b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.701+0000 7f0279551640 1 -- 172.21.15.45:0/2906862271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f027415f4e0 msgr2=0x7f02740b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.701+0000 7f0279551640 1 --2- 172.21.15.45:0/2906862271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f027415f4e0 0x7f02740b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.701+0000 7f0279551640 1 -- 172.21.15.45:0/2906862271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02740d3aa0 msgr2=0x7f027414f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.701+0000 7f0279551640 1 --2- 172.21.15.45:0/2906862271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02740d3aa0 0x7f027414f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.701+0000 7f0279551640 1 -- 172.21.15.45:0/2906862271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02740b80e0 con 0x7f02740d56b0 2024-05-08T05:42:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.701+0000 7f0279d52640 1 --2- 172.21.15.45:0/2906862271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02740d3aa0 0x7f027414f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:42:58.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.701+0000 7f0279551640 1 --2- 172.21.15.45:0/2906862271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02740d56b0 0x7f02740b7ba0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f026800c4c0 tx=0x7f026800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:58.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.701+0000 7f02727fc640 1 -- 172.21.15.45:0/2906862271 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268018070 con 0x7f02740d56b0 2024-05-08T05:42:58.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.701+0000 7f02727fc640 1 -- 172.21.15.45:0/2906862271 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0268007500 con 0x7f02740d56b0 2024-05-08T05:42:58.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.701+0000 7f027bfb6640 1 -- 172.21.15.45:0/2906862271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02740b9a30 con 0x7f02740d56b0 2024-05-08T05:42:58.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.701+0000 7f02727fc640 1 -- 172.21.15.45:0/2906862271 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268004a20 con 0x7f02740d56b0 2024-05-08T05:42:58.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.701+0000 7f027bfb6640 1 -- 172.21.15.45:0/2906862271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02740b9ff0 con 0x7f02740d56b0 2024-05-08T05:42:58.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.702+0000 7f027bfb6640 1 -- 172.21.15.45:0/2906862271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02740693f0 con 0x7f02740d56b0 2024-05-08T05:42:58.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.703+0000 7f02727fc640 1 -- 172.21.15.45:0/2906862271 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f026801b070 con 0x7f02740d56b0 2024-05-08T05:42:58.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.704+0000 7f02727fc640 1 --2- 172.21.15.45:0/2906862271 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f02600762d0 0x7f0260078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:58.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.704+0000 7f02727fc640 1 -- 172.21.15.45:0/2906862271 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6001+0+0 (secure 0 0 0) 0x7f0268096a40 con 0x7f02740d56b0 2024-05-08T05:42:58.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.704+0000 7f0279d52640 1 --2- 172.21.15.45:0/2906862271 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f02600762d0 0x7f0260078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:58.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.705+0000 7f0279d52640 1 --2- 172.21.15.45:0/2906862271 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f02600762d0 0x7f0260078790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f026c00a990 tx=0x7f026c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:58.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.708+0000 7f02727fc640 1 -- 172.21.15.45:0/2906862271 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0268060270 con 0x7f02740d56b0 2024-05-08T05:42:58.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:58.942+0000 7f027bfb6640 1 -- 172.21.15.45:0/2906862271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f027406dc70 con 0x7f02740d56b0 2024-05-08T05:42:59.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.576+0000 7f02727fc640 1 -- 172.21.15.45:0/2906862271 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v22) ==== 172+0+0 (secure 0 0 0) 0x7f026805fb70 con 0x7f02740d56b0 2024-05-08T05:42:59.577 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 7 and data pool 8 2024-05-08T05:42:59.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.591+0000 7f027bfb6640 1 -- 172.21.15.45:0/2906862271 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f02600762d0 msgr2=0x7f0260078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:59.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.591+0000 7f027bfb6640 1 --2- 172.21.15.45:0/2906862271 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f02600762d0 0x7f0260078790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f026c00a990 tx=0x7f026c008040 comp rx=0 tx=0).stop 2024-05-08T05:42:59.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.591+0000 7f027bfb6640 1 -- 172.21.15.45:0/2906862271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02740d56b0 msgr2=0x7f02740b7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:59.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.591+0000 7f027bfb6640 1 --2- 172.21.15.45:0/2906862271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02740d56b0 0x7f02740b7ba0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f026800c4c0 tx=0x7f026800c980 comp rx=0 tx=0).stop 2024-05-08T05:42:59.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.591+0000 7f027bfb6640 1 -- 172.21.15.45:0/2906862271 shutdown_connections 2024-05-08T05:42:59.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.592+0000 7f027bfb6640 1 --2- 172.21.15.45:0/2906862271 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f02600762d0 0x7f0260078790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:59.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.592+0000 7f027bfb6640 1 --2- 172.21.15.45:0/2906862271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f027415f4e0 0x7f02740b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:59.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.592+0000 7f027bfb6640 1 --2- 172.21.15.45:0/2906862271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02740d56b0 0x7f02740b7ba0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:59.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.592+0000 7f027bfb6640 1 --2- 172.21.15.45:0/2906862271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02740d3aa0 0x7f027414f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:59.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.592+0000 7f027bfb6640 1 -- 172.21.15.45:0/2906862271 >> 172.21.15.45:0/2906862271 conn(0x7f027408eac0 msgr2=0x7f02740d2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:59.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.593+0000 7f027bfb6640 1 -- 172.21.15.45:0/2906862271 shutdown_connections 2024-05-08T05:42:59.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.594+0000 7f027bfb6640 1 -- 172.21.15.45:0/2906862271 wait complete. 2024-05-08T05:42:59.613 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:42:59.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.721+0000 7f6b13ad1640 1 Processor -- start 2024-05-08T05:42:59.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.721+0000 7f6b13ad1640 1 -- start start 2024-05-08T05:42:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.722+0000 7f6b13ad1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6b0c15f380 0x7f6b0c0d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.722+0000 7f6b13ad1640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b0c0d1760 con 0x7f6b0c0d0cf0 2024-05-08T05:42:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.722+0000 7f6b13ad1640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b0c0d1a40 con 0x7f6b0c0d3cd0 2024-05-08T05:42:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.722+0000 7f6b13ad1640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f6b0c0d30f0 con 0x7f6b0c15f380 2024-05-08T05:42:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.722+0000 7f6b1186d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6b0c15f380 0x7f6b0c0d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.722+0000 7f6b1106c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6b0c0d0cf0 0x7f6b0c0d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36666/0 (socket says 172.21.15.45:36666) 2024-05-08T05:42:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.722+0000 7f6b1106c640 1 -- 172.21.15.45:0/1309507833 learned_addr learned my addr 172.21.15.45:0/1309507833 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:42:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.722+0000 7f6b1186d640 1 --2- 172.21.15.45:0/1309507833 >> v2:172.21.15.45:3300/0 conn(0x7f6b0c15f380 0x7f6b0c0d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50788/0 (socket says 172.21.15.45:50788) 2024-05-08T05:42:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.723+0000 7f6b1186d640 1 -- 172.21.15.45:0/1309507833 >> v1:172.21.15.63:6789/0 conn(0x7f6b0c0d3cd0 legacy=0x7f6b0c0b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:42:59.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.723+0000 7f6b1186d640 1 -- 172.21.15.45:0/1309507833 >> v1:172.21.15.45:6789/0 conn(0x7f6b0c0d0cf0 legacy=0x7f6b0c0d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:59.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.723+0000 7f6b1086b640 1 -- 172.21.15.45:0/1309507833 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3794314345 0 0) 0x7f6b0c0d1760 con 0x7f6b0c0d0cf0 2024-05-08T05:42:59.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.723+0000 7f6b1186d640 1 -- 172.21.15.45:0/1309507833 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b0c0d1d20 con 0x7f6b0c15f380 2024-05-08T05:42:59.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.723+0000 7f6b1186d640 1 --2- 172.21.15.45:0/1309507833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b0c15f380 0x7f6b0c0d07b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009d90 tx=0x7f6b0802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ed8154d7d85336 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:59.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.724+0000 7f6b1086b640 1 -- 172.21.15.45:0/1309507833 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08007740 con 0x7f6b0c15f380 2024-05-08T05:42:59.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.724+0000 7f6b1086b640 1 -- 172.21.15.45:0/1309507833 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b080078e0 con 0x7f6b0c15f380 2024-05-08T05:42:59.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.724+0000 7f6b1086b640 1 -- 172.21.15.45:0/1309507833 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08007cb0 con 0x7f6b0c15f380 2024-05-08T05:42:59.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.724+0000 7f6b13ad1640 1 -- 172.21.15.45:0/1309507833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b0c15f380 msgr2=0x7f6b0c0d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:59.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.724+0000 7f6b13ad1640 1 --2- 172.21.15.45:0/1309507833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b0c15f380 0x7f6b0c0d07b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009d90 tx=0x7f6b0802a660 comp rx=0 tx=0).stop 2024-05-08T05:42:59.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.724+0000 7f6b13ad1640 1 -- 172.21.15.45:0/1309507833 shutdown_connections 2024-05-08T05:42:59.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.724+0000 7f6b13ad1640 1 --2- 172.21.15.45:0/1309507833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b0c15f380 0x7f6b0c0d07b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:59.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.724+0000 7f6b13ad1640 1 -- 172.21.15.45:0/1309507833 >> 172.21.15.45:0/1309507833 conn(0x7f6b0c0d4fe0 msgr2=0x7f6b0c0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:59.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.724+0000 7f6b13ad1640 1 -- 172.21.15.45:0/1309507833 shutdown_connections 2024-05-08T05:42:59.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.724+0000 7f6b13ad1640 1 -- 172.21.15.45:0/1309507833 wait complete. 2024-05-08T05:42:59.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.725+0000 7f6b13ad1640 1 Processor -- start 2024-05-08T05:42:59.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.725+0000 7f6b13ad1640 1 -- start start 2024-05-08T05:42:59.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.726+0000 7f6b13ad1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6b0c0d0cf0 0x7f6b0c1513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:59.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.726+0000 7f6b13ad1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6b0c0d3cd0 0x7f6b0c162290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:59.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.726+0000 7f6b13ad1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b0c15f380 0x7f6b0c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:59.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.726+0000 7f6b13ad1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c15f760 con 0x7f6b0c15f380 2024-05-08T05:42:59.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.726+0000 7f6b13ad1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c0d2370 con 0x7f6b0c0d0cf0 2024-05-08T05:42:59.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.726+0000 7f6b13ad1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6b0c0d2800 con 0x7f6b0c0d3cd0 2024-05-08T05:42:59.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.726+0000 7f6b1206e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b0c15f380 0x7f6b0c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:59.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.726+0000 7f6b1206e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b0c15f380 0x7f6b0c0b8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50802/0 (socket says 172.21.15.45:50802) 2024-05-08T05:42:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.726+0000 7f6b1206e640 1 -- 172.21.15.45:0/1057442975 learned_addr learned my addr 172.21.15.45:0/1057442975 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:42:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.726+0000 7f6b1186d640 1 --2- 172.21.15.45:0/1057442975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6b0c0d0cf0 0x7f6b0c1513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.726+0000 7f6b1106c640 1 --2- 172.21.15.45:0/1057442975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6b0c0d3cd0 0x7f6b0c162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.727+0000 7f6b1206e640 1 -- 172.21.15.45:0/1057442975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6b0c0d3cd0 msgr2=0x7f6b0c162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.727+0000 7f6b1206e640 1 --2- 172.21.15.45:0/1057442975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6b0c0d3cd0 0x7f6b0c162290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.727+0000 7f6b1206e640 1 -- 172.21.15.45:0/1057442975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6b0c0d0cf0 msgr2=0x7f6b0c1513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.727+0000 7f6b1206e640 1 --2- 172.21.15.45:0/1057442975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6b0c0d0cf0 0x7f6b0c1513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.727+0000 7f6b1206e640 1 -- 172.21.15.45:0/1057442975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b0c162c90 con 0x7f6b0c15f380 2024-05-08T05:42:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.727+0000 7f6b1206e640 1 --2- 172.21.15.45:0/1057442975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b0c15f380 0x7f6b0c0b8b80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400c410 tx=0x7f6b0400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.727+0000 7f6b027fc640 1 -- 172.21.15.45:0/1057442975 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b04011790 con 0x7f6b0c15f380 2024-05-08T05:42:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.727+0000 7f6b027fc640 1 -- 172.21.15.45:0/1057442975 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b04017070 con 0x7f6b0c15f380 2024-05-08T05:42:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.727+0000 7f6b13ad1640 1 -- 172.21.15.45:0/1057442975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b0c162f70 con 0x7f6b0c15f380 2024-05-08T05:42:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.727+0000 7f6b027fc640 1 -- 172.21.15.45:0/1057442975 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b04011b70 con 0x7f6b0c15f380 2024-05-08T05:42:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.727+0000 7f6b13ad1640 1 -- 172.21.15.45:0/1057442975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b0c163530 con 0x7f6b0c15f380 2024-05-08T05:42:59.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.728+0000 7f6b13ad1640 1 -- 172.21.15.45:0/1057442975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b0c0cd230 con 0x7f6b0c15f380 2024-05-08T05:42:59.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.729+0000 7f6b027fc640 1 -- 172.21.15.45:0/1057442975 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6b04011d10 con 0x7f6b0c15f380 2024-05-08T05:42:59.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.729+0000 7f6b027fc640 1 --2- 172.21.15.45:0/1057442975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6af0076300 0x7f6af00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:59.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.729+0000 7f6b027fc640 1 -- 172.21.15.45:0/1057442975 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6101+0+0 (secure 0 0 0) 0x7f6b04097240 con 0x7f6b0c15f380 2024-05-08T05:42:59.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.729+0000 7f6b1186d640 1 --2- 172.21.15.45:0/1057442975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6af0076300 0x7f6af00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:59.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.730+0000 7f6b1186d640 1 --2- 172.21.15.45:0/1057442975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6af0076300 0x7f6af00787c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6b08034000 tx=0x7f6b0802f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:59.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.734+0000 7f6b027fc640 1 -- 172.21.15.45:0/1057442975 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6b0c0cd230 con 0x7f6b0c15f380 2024-05-08T05:42:59.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.923+0000 7f6b13ad1640 1 -- 172.21.15.45:0/1057442975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b0c0693f0 con 0x7f6b0c15f380 2024-05-08T05:42:59.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.924+0000 7f6b027fc640 1 -- 172.21.15.45:0/1057442975 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16907 (secure 0 0 0) 0x7f6b04021020 con 0x7f6b0c15f380 2024-05-08T05:42:59.926 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:42:59.926 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":46,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:42:58.943538+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:42:56.614794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T05:42:57.898381+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:42:59.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.926+0000 7f6b13ad1640 1 -- 172.21.15.45:0/1057442975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6af0076300 msgr2=0x7f6af00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:59.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.926+0000 7f6b13ad1640 1 --2- 172.21.15.45:0/1057442975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6af0076300 0x7f6af00787c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6b08034000 tx=0x7f6b0802f000 comp rx=0 tx=0).stop 2024-05-08T05:42:59.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.927+0000 7f6b13ad1640 1 -- 172.21.15.45:0/1057442975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b0c15f380 msgr2=0x7f6b0c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:59.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.927+0000 7f6b13ad1640 1 --2- 172.21.15.45:0/1057442975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b0c15f380 0x7f6b0c0b8b80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400c410 tx=0x7f6b0400c810 comp rx=0 tx=0).stop 2024-05-08T05:42:59.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.927+0000 7f6b13ad1640 1 -- 172.21.15.45:0/1057442975 shutdown_connections 2024-05-08T05:42:59.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.927+0000 7f6b13ad1640 1 --2- 172.21.15.45:0/1057442975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6af0076300 0x7f6af00787c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:59.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.927+0000 7f6b13ad1640 1 --2- 172.21.15.45:0/1057442975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6b0c0d3cd0 0x7f6b0c162290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:59.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.927+0000 7f6b13ad1640 1 --2- 172.21.15.45:0/1057442975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b0c15f380 0x7f6b0c0b8b80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:59.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.927+0000 7f6b13ad1640 1 --2- 172.21.15.45:0/1057442975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6b0c0d0cf0 0x7f6b0c1513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:59.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.927+0000 7f6b13ad1640 1 -- 172.21.15.45:0/1057442975 >> 172.21.15.45:0/1057442975 conn(0x7f6b0c0d4fe0 msgr2=0x7f6b0c0d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:59.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.927+0000 7f6b13ad1640 1 -- 172.21.15.45:0/1057442975 shutdown_connections 2024-05-08T05:42:59.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:42:59.927+0000 7f6b13ad1640 1 -- 172.21.15.45:0/1057442975 wait complete. 2024-05-08T05:42:59.939 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:43:00.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.042+0000 7f5bef511640 1 Processor -- start 2024-05-08T05:43:00.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.042+0000 7f5bef511640 1 -- start start 2024-05-08T05:43:00.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bef511640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80d1970 0x7f5be80d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:00.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bef511640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5be81630b0 con 0x7f5be815f420 2024-05-08T05:43:00.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bef511640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5be80d3e70 con 0x7f5be80d4cc0 2024-05-08T05:43:00.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bef511640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5be80d2330 con 0x7f5be80d1970 2024-05-08T05:43:00.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bed2ad640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5be815f420 0x7f5be80d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36682/0 (socket says 172.21.15.45:36682) 2024-05-08T05:43:00.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bed2ad640 1 -- 172.21.15.45:0/619532467 learned_addr learned my addr 172.21.15.45:0/619532467 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:00.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5becaac640 1 --2- 172.21.15.45:0/619532467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80d1970 0x7f5be80d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:00.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bdffff640 1 -- 172.21.15.45:0/619532467 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3336630140 0 0) 0x7f5be4001110 con 0x7f5be815f420 2024-05-08T05:43:00.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bdffff640 1 -- 172.21.15.45:0/619532467 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bd4003520 con 0x7f5be815f420 2024-05-08T05:43:00.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bdffff640 1 -- 172.21.15.45:0/619532467 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 693357365 0 0) 0x7f5bd4003520 con 0x7f5be815f420 2024-05-08T05:43:00.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bdffff640 1 -- 172.21.15.45:0/619532467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80d1970 msgr2=0x7f5be80d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:00.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bdffff640 1 --2- 172.21.15.45:0/619532467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80d1970 0x7f5be80d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:00.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bdffff640 1 -- 172.21.15.45:0/619532467 >> v1:172.21.15.63:6789/0 conn(0x7f5be80d4cc0 legacy=0x7f5be80d5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:00.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bdffff640 1 -- 172.21.15.45:0/619532467 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be80d2550 con 0x7f5be815f420 2024-05-08T05:43:00.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bdffff640 1 -- 172.21.15.45:0/619532467 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f5be40018c0 con 0x7f5be815f420 2024-05-08T05:43:00.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bdffff640 1 -- 172.21.15.45:0/619532467 >> v1:172.21.15.45:6789/0 conn(0x7f5be815f420 legacy=0x7f5be80d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:00.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bdffff640 1 --2- 172.21.15.45:0/619532467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bd40039e0 0x7f5bd4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:00.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bdffff640 1 --2- 172.21.15.45:0/619532467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80d1970 0x7f5bd4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:00.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bdffff640 1 --2- 172.21.15.45:0/619532467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bd4004d40 0x7f5bd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:00.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bdffff640 1 -- 172.21.15.45:0/619532467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5be80d2330 con 0x7f5bd40039e0 2024-05-08T05:43:00.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bdffff640 1 -- 172.21.15.45:0/619532467 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd400ba70 con 0x7f5bd4004d40 2024-05-08T05:43:00.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.043+0000 7f5bdffff640 1 -- 172.21.15.45:0/619532467 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd400bc70 con 0x7f5be80d1970 2024-05-08T05:43:00.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.044+0000 7f5bed2ad640 1 --2- 172.21.15.45:0/619532467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bd40039e0 0x7f5bd4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:00.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.044+0000 7f5bedaae640 1 --2- 172.21.15.45:0/619532467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bd4004d40 0x7f5bd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:00.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.044+0000 7f5becaac640 1 --2- 172.21.15.45:0/619532467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80d1970 0x7f5bd4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:00.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.044+0000 7f5bed2ad640 1 -- 172.21.15.45:0/619532467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80d1970 msgr2=0x7f5bd4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:00.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.044+0000 7f5bed2ad640 1 --2- 172.21.15.45:0/619532467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80d1970 0x7f5bd4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:00.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.044+0000 7f5bed2ad640 1 -- 172.21.15.45:0/619532467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bd4004d40 msgr2=0x7f5bd40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:00.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.044+0000 7f5bed2ad640 1 --2- 172.21.15.45:0/619532467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bd4004d40 0x7f5bd40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:00.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.044+0000 7f5bed2ad640 1 -- 172.21.15.45:0/619532467 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd4003520 con 0x7f5bd40039e0 2024-05-08T05:43:00.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.044+0000 7f5bedaae640 1 --2- 172.21.15.45:0/619532467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5bd4004d40 0x7f5bd40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:00.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.044+0000 7f5becaac640 1 --2- 172.21.15.45:0/619532467 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80d1970 0x7f5bd4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:00.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.045+0000 7f5bed2ad640 1 --2- 172.21.15.45:0/619532467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bd40039e0 0x7f5bd4003dc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5be400c490 tx=0x7f5bd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aece89e03857b89 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:00.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.045+0000 7f5bdffff640 1 -- 172.21.15.45:0/619532467 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4036070 con 0x7f5bd40039e0 2024-05-08T05:43:00.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.045+0000 7f5bdffff640 1 -- 172.21.15.45:0/619532467 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be4005af0 con 0x7f5bd40039e0 2024-05-08T05:43:00.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.045+0000 7f5bdffff640 1 -- 172.21.15.45:0/619532467 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be400c0e0 con 0x7f5bd40039e0 2024-05-08T05:43:00.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.045+0000 7f5bef511640 1 -- 172.21.15.45:0/619532467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bd40039e0 msgr2=0x7f5bd4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.045+0000 7f5bef511640 1 --2- 172.21.15.45:0/619532467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bd40039e0 0x7f5bd4003dc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5be400c490 tx=0x7f5bd4005830 comp rx=0 tx=0).stop 2024-05-08T05:43:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.045+0000 7f5bedaae640 1 -- 172.21.15.45:0/619532467 reap_dead start 2024-05-08T05:43:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.045+0000 7f5bef511640 1 -- 172.21.15.45:0/619532467 shutdown_connections 2024-05-08T05:43:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.045+0000 7f5bef511640 1 -- 172.21.15.45:0/619532467 >> 172.21.15.45:0/619532467 conn(0x7f5be808eac0 msgr2=0x7f5be809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.045+0000 7f5bef511640 1 -- 172.21.15.45:0/619532467 shutdown_connections 2024-05-08T05:43:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.045+0000 7f5bef511640 1 -- 172.21.15.45:0/619532467 wait complete. 2024-05-08T05:43:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.047+0000 7f5bef511640 1 Processor -- start 2024-05-08T05:43:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.047+0000 7f5bef511640 1 -- start start 2024-05-08T05:43:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.047+0000 7f5bef511640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bd40039e0 0x7f5be8153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.047+0000 7f5bef511640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be80cb3c0 0x7f5be80cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.047+0000 7f5bef511640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80b2c90 0x7f5be80b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.047+0000 7f5bef511640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5be80d3240 con 0x7f5bd40039e0 2024-05-08T05:43:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.047+0000 7f5bef511640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5be80d2f60 con 0x7f5be80cb3c0 2024-05-08T05:43:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.047+0000 7f5bef511640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5be80d2830 con 0x7f5be80b2c90 2024-05-08T05:43:00.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.047+0000 7f5bed2ad640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bd40039e0 0x7f5be8153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:00.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.047+0000 7f5bed2ad640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bd40039e0 0x7f5be8153840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50828/0 (socket says 172.21.15.45:50828) 2024-05-08T05:43:00.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.047+0000 7f5bed2ad640 1 -- 172.21.15.45:0/3874461776 learned_addr learned my addr 172.21.15.45:0/3874461776 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:00.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.048+0000 7f5bedaae640 1 --2- 172.21.15.45:0/3874461776 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80b2c90 0x7f5be80b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:00.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.048+0000 7f5becaac640 1 --2- 172.21.15.45:0/3874461776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be80cb3c0 0x7f5be80cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:00.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.048+0000 7f5bed2ad640 1 -- 172.21.15.45:0/3874461776 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80b2c90 msgr2=0x7f5be80b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:00.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.048+0000 7f5bed2ad640 1 --2- 172.21.15.45:0/3874461776 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80b2c90 0x7f5be80b3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:00.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.048+0000 7f5bed2ad640 1 -- 172.21.15.45:0/3874461776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be80cb3c0 msgr2=0x7f5be80cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:00.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.048+0000 7f5bed2ad640 1 --2- 172.21.15.45:0/3874461776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be80cb3c0 0x7f5be80cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:00.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.048+0000 7f5bed2ad640 1 -- 172.21.15.45:0/3874461776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be80ca320 con 0x7f5bd40039e0 2024-05-08T05:43:00.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.048+0000 7f5becaac640 1 --2- 172.21.15.45:0/3874461776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be80cb3c0 0x7f5be80cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:00.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.048+0000 7f5bed2ad640 1 --2- 172.21.15.45:0/3874461776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bd40039e0 0x7f5be8153840 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5be4034040 tx=0x7f5be815f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:00.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.048+0000 7f5bddffb640 1 -- 172.21.15.45:0/3874461776 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4036040 con 0x7f5bd40039e0 2024-05-08T05:43:00.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.048+0000 7f5bddffb640 1 -- 172.21.15.45:0/3874461776 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be4007070 con 0x7f5bd40039e0 2024-05-08T05:43:00.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.048+0000 7f5bef511640 1 -- 172.21.15.45:0/3874461776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be80ca600 con 0x7f5bd40039e0 2024-05-08T05:43:00.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.048+0000 7f5bddffb640 1 -- 172.21.15.45:0/3874461776 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be4007420 con 0x7f5bd40039e0 2024-05-08T05:43:00.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.048+0000 7f5bef511640 1 -- 172.21.15.45:0/3874461776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be80cabc0 con 0x7f5bd40039e0 2024-05-08T05:43:00.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.049+0000 7f5bef511640 1 -- 172.21.15.45:0/3874461776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be80693f0 con 0x7f5bd40039e0 2024-05-08T05:43:00.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.050+0000 7f5bddffb640 1 -- 172.21.15.45:0/3874461776 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5be40075c0 con 0x7f5bd40039e0 2024-05-08T05:43:00.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.050+0000 7f5bddffb640 1 --2- 172.21.15.45:0/3874461776 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5bcc076300 0x7f5bcc0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:00.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.051+0000 7f5bddffb640 1 -- 172.21.15.45:0/3874461776 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6101+0+0 (secure 0 0 0) 0x7f5be40b81b0 con 0x7f5bd40039e0 2024-05-08T05:43:00.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.051+0000 7f5becaac640 1 --2- 172.21.15.45:0/3874461776 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5bcc076300 0x7f5bcc0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:00.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.051+0000 7f5becaac640 1 --2- 172.21.15.45:0/3874461776 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5bcc076300 0x7f5bcc0787c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8006d40 tx=0x7f5bd8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:00.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.055+0000 7f5bddffb640 1 -- 172.21.15.45:0/3874461776 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5be81630b0 con 0x7f5bd40039e0 2024-05-08T05:43:00.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.251+0000 7f5bef511640 1 -- 172.21.15.45:0/3874461776 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5be80b4c50 con 0x7f5bd40039e0 2024-05-08T05:43:00.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.252+0000 7f5bddffb640 1 -- 172.21.15.45:0/3874461776 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16907 (secure 0 0 0) 0x7f5be4005ac0 con 0x7f5bd40039e0 2024-05-08T05:43:00.253 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:00.253 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":46,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:42:58.943538+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:42:56.614794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T05:42:57.898381+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:43:00.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.254+0000 7f5bef511640 1 -- 172.21.15.45:0/3874461776 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5bcc076300 msgr2=0x7f5bcc0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:00.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.254+0000 7f5bef511640 1 --2- 172.21.15.45:0/3874461776 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5bcc076300 0x7f5bcc0787c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8006d40 tx=0x7f5bd8008040 comp rx=0 tx=0).stop 2024-05-08T05:43:00.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.254+0000 7f5bef511640 1 -- 172.21.15.45:0/3874461776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bd40039e0 msgr2=0x7f5be8153840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:00.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.254+0000 7f5bef511640 1 --2- 172.21.15.45:0/3874461776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bd40039e0 0x7f5be8153840 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5be4034040 tx=0x7f5be815f810 comp rx=0 tx=0).stop 2024-05-08T05:43:00.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.254+0000 7f5bef511640 1 -- 172.21.15.45:0/3874461776 shutdown_connections 2024-05-08T05:43:00.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.254+0000 7f5bef511640 1 --2- 172.21.15.45:0/3874461776 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5bcc076300 0x7f5bcc0787c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:00.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.254+0000 7f5bef511640 1 --2- 172.21.15.45:0/3874461776 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5be80b2c90 0x7f5be80b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:00.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.254+0000 7f5bef511640 1 --2- 172.21.15.45:0/3874461776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be80cb3c0 0x7f5be80cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:00.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.254+0000 7f5bef511640 1 --2- 172.21.15.45:0/3874461776 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5bd40039e0 0x7f5be8153840 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:00.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.254+0000 7f5bef511640 1 -- 172.21.15.45:0/3874461776 >> 172.21.15.45:0/3874461776 conn(0x7f5be808eac0 msgr2=0x7f5be815deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:00.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.254+0000 7f5bef511640 1 -- 172.21.15.45:0/3874461776 shutdown_connections 2024-05-08T05:43:00.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.254+0000 7f5bef511640 1 -- 172.21.15.45:0/3874461776 wait complete. 2024-05-08T05:43:00.266 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T05:43:00.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.372+0000 7effc0846640 1 Processor -- start 2024-05-08T05:43:00.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.372+0000 7effc0846640 1 -- start start 2024-05-08T05:43:00.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.373+0000 7effc0846640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7effbc15f0c0 0x7effbc0b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:00.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.373+0000 7effc0846640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7effbc1638e0 0x7effbc0b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:00.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.373+0000 7effc0846640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effbc0b1260 con 0x7effbc06c3a0 2024-05-08T05:43:00.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.373+0000 7effc0846640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7effbc163450 con 0x7effbc1638e0 2024-05-08T05:43:00.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.373+0000 7effc0846640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7effbc1604b0 con 0x7effbc15f0c0 2024-05-08T05:43:00.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.373+0000 7effb1d9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7effbc1638e0 0x7effbc0b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:00.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.373+0000 7effb1d9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7effbc1638e0 0x7effbc0b0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50830/0 (socket says 172.21.15.45:50830) 2024-05-08T05:43:00.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.373+0000 7effb1d9b640 1 -- 172.21.15.45:0/3888085439 learned_addr learned my addr 172.21.15.45:0/3888085439 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:00.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.373+0000 7effba59c640 1 --2- 172.21.15.45:0/3888085439 >> v2:172.21.15.63:3300/0 conn(0x7effbc15f0c0 0x7effbc0b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:00.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.373+0000 7effb1d9b640 1 -- 172.21.15.45:0/3888085439 >> v2:172.21.15.63:3300/0 conn(0x7effbc15f0c0 msgr2=0x7effbc0b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:00.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.373+0000 7effb1d9b640 1 --2- 172.21.15.45:0/3888085439 >> v2:172.21.15.63:3300/0 conn(0x7effbc15f0c0 0x7effbc0b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:00.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.373+0000 7effb1d9b640 1 -- 172.21.15.45:0/3888085439 >> v1:172.21.15.63:6789/0 conn(0x7effbc06c3a0 legacy=0x7effbc073170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:00.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.373+0000 7effb1d9b640 1 -- 172.21.15.45:0/3888085439 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc0b14e0 con 0x7effbc1638e0 2024-05-08T05:43:00.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.374+0000 7effb1d9b640 1 --2- 172.21.15.45:0/3888085439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effbc1638e0 0x7effbc0b0d20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7effbc071200 tx=0x7effa402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f45176784f474cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:00.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.374+0000 7effb9d9b640 1 -- 172.21.15.45:0/3888085439 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4037070 con 0x7effbc1638e0 2024-05-08T05:43:00.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.374+0000 7effb9d9b640 1 -- 172.21.15.45:0/3888085439 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa4032440 con 0x7effbc1638e0 2024-05-08T05:43:00.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.374+0000 7effb9d9b640 1 -- 172.21.15.45:0/3888085439 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4032810 con 0x7effbc1638e0 2024-05-08T05:43:00.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.375+0000 7effc0846640 1 -- 172.21.15.45:0/3888085439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effbc1638e0 msgr2=0x7effbc0b0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:00.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.375+0000 7effc0846640 1 --2- 172.21.15.45:0/3888085439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effbc1638e0 0x7effbc0b0d20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7effbc071200 tx=0x7effa402a270 comp rx=0 tx=0).stop 2024-05-08T05:43:00.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.375+0000 7effc0846640 1 -- 172.21.15.45:0/3888085439 shutdown_connections 2024-05-08T05:43:00.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.375+0000 7effc0846640 1 --2- 172.21.15.45:0/3888085439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effbc1638e0 0x7effbc0b0d20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:00.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.375+0000 7effc0846640 1 --2- 172.21.15.45:0/3888085439 >> v2:172.21.15.63:3300/0 conn(0x7effbc15f0c0 0x7effbc0b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:00.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.375+0000 7effc0846640 1 -- 172.21.15.45:0/3888085439 >> 172.21.15.45:0/3888085439 conn(0x7effbc0b3b10 msgr2=0x7effbc0b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:00.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.375+0000 7effc0846640 1 -- 172.21.15.45:0/3888085439 shutdown_connections 2024-05-08T05:43:00.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.375+0000 7effc0846640 1 -- 172.21.15.45:0/3888085439 wait complete. 2024-05-08T05:43:00.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.376+0000 7effc0846640 1 Processor -- start 2024-05-08T05:43:00.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.376+0000 7effc0846640 1 -- start start 2024-05-08T05:43:00.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.377+0000 7effc0846640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effbc06c3a0 0x7effbc1572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:00.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.377+0000 7effc0846640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7effbc15f0c0 0x7effbc14c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:00.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.377+0000 7effc0846640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7effbc1638e0 0x7effbc158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:00.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.377+0000 7effc0846640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7effbc0b17c0 con 0x7effbc06c3a0 2024-05-08T05:43:00.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.377+0000 7effc0846640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7effbc1630e0 con 0x7effbc1638e0 2024-05-08T05:43:00.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.377+0000 7effc0846640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7effbc06c7e0 con 0x7effbc15f0c0 2024-05-08T05:43:00.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.377+0000 7effba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effbc06c3a0 0x7effbc1572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:00.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.377+0000 7effba59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effbc06c3a0 0x7effbc1572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50846/0 (socket says 172.21.15.45:50846) 2024-05-08T05:43:00.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.377+0000 7effba59c640 1 -- 172.21.15.45:0/2787587494 learned_addr learned my addr 172.21.15.45:0/2787587494 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:00.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.377+0000 7effb1d9b640 1 --2- 172.21.15.45:0/2787587494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7effbc15f0c0 0x7effbc14c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:00.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.377+0000 7effbad9d640 1 --2- 172.21.15.45:0/2787587494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7effbc1638e0 0x7effbc158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:00.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.378+0000 7effba59c640 1 -- 172.21.15.45:0/2787587494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7effbc15f0c0 msgr2=0x7effbc14c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:00.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.378+0000 7effba59c640 1 --2- 172.21.15.45:0/2787587494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7effbc15f0c0 0x7effbc14c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:00.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.378+0000 7effba59c640 1 -- 172.21.15.45:0/2787587494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7effbc1638e0 msgr2=0x7effbc158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:00.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.378+0000 7effba59c640 1 --2- 172.21.15.45:0/2787587494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7effbc1638e0 0x7effbc158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:00.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.378+0000 7effba59c640 1 -- 172.21.15.45:0/2787587494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc157bd0 con 0x7effbc06c3a0 2024-05-08T05:43:00.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.378+0000 7effb1d9b640 1 --2- 172.21.15.45:0/2787587494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7effbc15f0c0 0x7effbc14c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:00.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.378+0000 7effba59c640 1 --2- 172.21.15.45:0/2787587494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effbc06c3a0 0x7effbc1572d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7effac00e9b0 tx=0x7effac00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:00.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.378+0000 7effb37fe640 1 -- 172.21.15.45:0/2787587494 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac017070 con 0x7effbc06c3a0 2024-05-08T05:43:00.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.378+0000 7effb37fe640 1 -- 172.21.15.45:0/2787587494 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effac00c9a0 con 0x7effbc06c3a0 2024-05-08T05:43:00.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.378+0000 7effc0846640 1 -- 172.21.15.45:0/2787587494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effbc157eb0 con 0x7effbc06c3a0 2024-05-08T05:43:00.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.378+0000 7effb37fe640 1 -- 172.21.15.45:0/2787587494 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac00cd90 con 0x7effbc06c3a0 2024-05-08T05:43:00.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.378+0000 7effc0846640 1 -- 172.21.15.45:0/2787587494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effbc158750 con 0x7effbc06c3a0 2024-05-08T05:43:00.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.379+0000 7effc0846640 1 -- 172.21.15.45:0/2787587494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effbc158080 con 0x7effbc06c3a0 2024-05-08T05:43:00.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.380+0000 7effb37fe640 1 -- 172.21.15.45:0/2787587494 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7effac01a070 con 0x7effbc06c3a0 2024-05-08T05:43:00.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.380+0000 7effb37fe640 1 --2- 172.21.15.45:0/2787587494 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7eff9c076300 0x7eff9c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:00.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.380+0000 7effb37fe640 1 -- 172.21.15.45:0/2787587494 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6101+0+0 (secure 0 0 0) 0x7effac097b40 con 0x7effbc06c3a0 2024-05-08T05:43:00.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.380+0000 7effb1d9b640 1 --2- 172.21.15.45:0/2787587494 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7eff9c076300 0x7eff9c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:00.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.381+0000 7effb1d9b640 1 --2- 172.21.15.45:0/2787587494 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7eff9c076300 0x7eff9c0787c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7effa4034000 tx=0x7effa402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:00.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.385+0000 7effb37fe640 1 -- 172.21.15.45:0/2787587494 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7effac060bb0 con 0x7effbc06c3a0 2024-05-08T05:43:00.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:00.618+0000 7effc0846640 1 -- 172.21.15.45:0/2787587494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7effbc06a8f0 con 0x7effbc06c3a0 2024-05-08T05:43:01.550 INFO:tasks.ceph.mds.b.smithi063.stderr:2024-05-08T05:43:01.550+0000 7f3c6d599640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:43:01.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.586+0000 7effb37fe640 1 -- 172.21.15.45:0/2787587494 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7effac064eb0 con 0x7effbc06c3a0 2024-05-08T05:43:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.589+0000 7effc0846640 1 -- 172.21.15.45:0/2787587494 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7eff9c076300 msgr2=0x7eff9c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.589+0000 7effc0846640 1 --2- 172.21.15.45:0/2787587494 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7eff9c076300 0x7eff9c0787c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7effa4034000 tx=0x7effa402e000 comp rx=0 tx=0).stop 2024-05-08T05:43:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.589+0000 7effc0846640 1 -- 172.21.15.45:0/2787587494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effbc06c3a0 msgr2=0x7effbc1572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.589+0000 7effc0846640 1 --2- 172.21.15.45:0/2787587494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effbc06c3a0 0x7effbc1572d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7effac00e9b0 tx=0x7effac00ee70 comp rx=0 tx=0).stop 2024-05-08T05:43:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.589+0000 7effc0846640 1 -- 172.21.15.45:0/2787587494 shutdown_connections 2024-05-08T05:43:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.589+0000 7effc0846640 1 --2- 172.21.15.45:0/2787587494 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7eff9c076300 0x7eff9c0787c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.589+0000 7effc0846640 1 --2- 172.21.15.45:0/2787587494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7effbc15f0c0 0x7effbc14c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.589+0000 7effc0846640 1 --2- 172.21.15.45:0/2787587494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7effbc1638e0 0x7effbc158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.589+0000 7effc0846640 1 --2- 172.21.15.45:0/2787587494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7effbc06c3a0 0x7effbc1572d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.589+0000 7effc0846640 1 -- 172.21.15.45:0/2787587494 >> 172.21.15.45:0/2787587494 conn(0x7effbc0b3b10 msgr2=0x7effbc0d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.589+0000 7effc0846640 1 -- 172.21.15.45:0/2787587494 shutdown_connections 2024-05-08T05:43:01.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.589+0000 7effc0846640 1 -- 172.21.15.45:0/2787587494 wait complete. 2024-05-08T05:43:01.607 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T05:43:01.607 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T05:43:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.720+0000 7f03cfdf0640 1 Processor -- start 2024-05-08T05:43:01.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.720+0000 7f03cfdf0640 1 -- start start 2024-05-08T05:43:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.721+0000 7f03cfdf0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f03c80d2130 0x7f03c81639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:01.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.721+0000 7f03cfdf0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03c8160720 con 0x7f03c8071110 2024-05-08T05:43:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.721+0000 7f03cfdf0640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03c80d10d0 con 0x7f03c8070d30 2024-05-08T05:43:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.721+0000 7f03cfdf0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f03c80d1ca0 con 0x7f03c80d2130 2024-05-08T05:43:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.721+0000 7f03cd38b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f03c80d2130 0x7f03c81639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.721+0000 7f03ce38d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f03c8071110 0x7f03c806d4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37424/0 (socket says 172.21.15.45:37424) 2024-05-08T05:43:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.721+0000 7f03ce38d640 1 -- 172.21.15.45:0/2850522003 learned_addr learned my addr 172.21.15.45:0/2850522003 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.721+0000 7f03cd38b640 1 -- 172.21.15.45:0/2850522003 >> v1:172.21.15.63:6789/0 conn(0x7f03c8070d30 legacy=0x7f03c80703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.721+0000 7f03cd38b640 1 -- 172.21.15.45:0/2850522003 >> v1:172.21.15.45:6789/0 conn(0x7f03c8071110 legacy=0x7f03c806d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.721+0000 7f03ccb8a640 1 -- 172.21.15.45:0/2850522003 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1762281190 0 0) 0x7f03c8160720 con 0x7f03c8071110 2024-05-08T05:43:01.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.721+0000 7f03cd38b640 1 -- 172.21.15.45:0/2850522003 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c80d1370 con 0x7f03c80d2130 2024-05-08T05:43:01.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.722+0000 7f03cd38b640 1 --2- 172.21.15.45:0/2850522003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03c80d2130 0x7f03c81639e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f03b80097a0 tx=0x7f03b802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ecb5291721030d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:01.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.722+0000 7f03ccb8a640 1 -- 172.21.15.45:0/2850522003 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8037070 con 0x7f03c80d2130 2024-05-08T05:43:01.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.722+0000 7f03ccb8a640 1 -- 172.21.15.45:0/2850522003 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03b8032440 con 0x7f03c80d2130 2024-05-08T05:43:01.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.722+0000 7f03ccb8a640 1 -- 172.21.15.45:0/2850522003 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8032810 con 0x7f03c80d2130 2024-05-08T05:43:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.723+0000 7f03cfdf0640 1 -- 172.21.15.45:0/2850522003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03c80d2130 msgr2=0x7f03c81639e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.723+0000 7f03cfdf0640 1 --2- 172.21.15.45:0/2850522003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03c80d2130 0x7f03c81639e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f03b80097a0 tx=0x7f03b802a230 comp rx=0 tx=0).stop 2024-05-08T05:43:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.723+0000 7f03cfdf0640 1 -- 172.21.15.45:0/2850522003 shutdown_connections 2024-05-08T05:43:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.723+0000 7f03cfdf0640 1 --2- 172.21.15.45:0/2850522003 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03c80d2130 0x7f03c81639e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.723+0000 7f03cfdf0640 1 -- 172.21.15.45:0/2850522003 >> 172.21.15.45:0/2850522003 conn(0x7f03c80d2e90 msgr2=0x7f03c80d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.723+0000 7f03cfdf0640 1 -- 172.21.15.45:0/2850522003 shutdown_connections 2024-05-08T05:43:01.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.723+0000 7f03cfdf0640 1 -- 172.21.15.45:0/2850522003 wait complete. 2024-05-08T05:43:01.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.724+0000 7f03cfdf0640 1 Processor -- start 2024-05-08T05:43:01.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.724+0000 7f03cfdf0640 1 -- start start 2024-05-08T05:43:01.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.725+0000 7f03cfdf0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03c8070d30 0x7f03c8143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.725+0000 7f03cfdf0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03c8071110 0x7f03c80c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.725+0000 7f03cfdf0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03c80d2130 0x7f03c80c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.725+0000 7f03cfdf0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f03c8071540 con 0x7f03c8071110 2024-05-08T05:43:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.725+0000 7f03cfdf0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f03c80d1930 con 0x7f03c8070d30 2024-05-08T05:43:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.725+0000 7f03cfdf0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f03c80716c0 con 0x7f03c80d2130 2024-05-08T05:43:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.725+0000 7f03cd38b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03c8071110 0x7f03c80c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.725+0000 7f03cd38b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03c8071110 0x7f03c80c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47314/0 (socket says 172.21.15.45:47314) 2024-05-08T05:43:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.725+0000 7f03cd38b640 1 -- 172.21.15.45:0/2077077414 learned_addr learned my addr 172.21.15.45:0/2077077414 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.725+0000 7f03ce38d640 1 --2- 172.21.15.45:0/2077077414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03c80d2130 0x7f03c80c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.725+0000 7f03cdb8c640 1 --2- 172.21.15.45:0/2077077414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03c8070d30 0x7f03c8143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.725+0000 7f03cd38b640 1 -- 172.21.15.45:0/2077077414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03c80d2130 msgr2=0x7f03c80c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.725+0000 7f03cd38b640 1 --2- 172.21.15.45:0/2077077414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03c80d2130 0x7f03c80c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.725+0000 7f03cd38b640 1 -- 172.21.15.45:0/2077077414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03c8070d30 msgr2=0x7f03c8143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.725+0000 7f03cd38b640 1 --2- 172.21.15.45:0/2077077414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03c8070d30 0x7f03c8143da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:01.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.725+0000 7f03cd38b640 1 -- 172.21.15.45:0/2077077414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c81443b0 con 0x7f03c8071110 2024-05-08T05:43:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.725+0000 7f03ce38d640 1 --2- 172.21.15.45:0/2077077414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03c80d2130 0x7f03c80c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.725+0000 7f03cdb8c640 1 --2- 172.21.15.45:0/2077077414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03c8070d30 0x7f03c8143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.725+0000 7f03cd38b640 1 --2- 172.21.15.45:0/2077077414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03c8071110 0x7f03c80c6b10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f03b8000c00 tx=0x7f03b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.726+0000 7f03be7fc640 1 -- 172.21.15.45:0/2077077414 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8037040 con 0x7f03c8071110 2024-05-08T05:43:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.726+0000 7f03be7fc640 1 -- 172.21.15.45:0/2077077414 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03b8042070 con 0x7f03c8071110 2024-05-08T05:43:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.726+0000 7f03cfdf0640 1 -- 172.21.15.45:0/2077077414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c8144690 con 0x7f03c8071110 2024-05-08T05:43:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.726+0000 7f03be7fc640 1 -- 172.21.15.45:0/2077077414 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8030a70 con 0x7f03c8071110 2024-05-08T05:43:01.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.726+0000 7f03cfdf0640 1 -- 172.21.15.45:0/2077077414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c8144c50 con 0x7f03c8071110 2024-05-08T05:43:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.727+0000 7f03cfdf0640 1 -- 172.21.15.45:0/2077077414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03c80693f0 con 0x7f03c8071110 2024-05-08T05:43:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.727+0000 7f03be7fc640 1 -- 172.21.15.45:0/2077077414 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f03b8030c10 con 0x7f03c8071110 2024-05-08T05:43:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.728+0000 7f03be7fc640 1 --2- 172.21.15.45:0/2077077414 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f03ac076300 0x7f03ac0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.728+0000 7f03be7fc640 1 -- 172.21.15.45:0/2077077414 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7f03b80b81d0 con 0x7f03c8071110 2024-05-08T05:43:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.728+0000 7f03cdb8c640 1 --2- 172.21.15.45:0/2077077414 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f03ac076300 0x7f03ac0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:01.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.729+0000 7f03cdb8c640 1 --2- 172.21.15.45:0/2077077414 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f03ac076300 0x7f03ac0787c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f03c4007fd0 tx=0x7f03c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:01.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.732+0000 7f03be7fc640 1 -- 172.21.15.45:0/2077077414 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f03b8081990 con 0x7f03c8071110 2024-05-08T05:43:01.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:01.964+0000 7f03cfdf0640 1 -- 172.21.15.45:0/2077077414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f03c80b2100 con 0x7f03c8071110 2024-05-08T05:43:02.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.587+0000 7f03be7fc640 1 -- 172.21.15.45:0/2077077414 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f03b8081290 con 0x7f03c8071110 2024-05-08T05:43:02.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.590+0000 7f03cfdf0640 1 -- 172.21.15.45:0/2077077414 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f03ac076300 msgr2=0x7f03ac0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:02.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.590+0000 7f03cfdf0640 1 --2- 172.21.15.45:0/2077077414 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f03ac076300 0x7f03ac0787c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f03c4007fd0 tx=0x7f03c4009040 comp rx=0 tx=0).stop 2024-05-08T05:43:02.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.590+0000 7f03cfdf0640 1 -- 172.21.15.45:0/2077077414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03c8071110 msgr2=0x7f03c80c6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:02.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.590+0000 7f03cfdf0640 1 --2- 172.21.15.45:0/2077077414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03c8071110 0x7f03c80c6b10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f03b8000c00 tx=0x7f03b80046c0 comp rx=0 tx=0).stop 2024-05-08T05:43:02.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.590+0000 7f03cfdf0640 1 -- 172.21.15.45:0/2077077414 shutdown_connections 2024-05-08T05:43:02.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.590+0000 7f03cfdf0640 1 --2- 172.21.15.45:0/2077077414 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f03ac076300 0x7f03ac0787c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:02.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.590+0000 7f03cfdf0640 1 --2- 172.21.15.45:0/2077077414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03c80d2130 0x7f03c80c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:02.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.590+0000 7f03cfdf0640 1 --2- 172.21.15.45:0/2077077414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03c8071110 0x7f03c80c6b10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:02.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.590+0000 7f03cfdf0640 1 --2- 172.21.15.45:0/2077077414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03c8070d30 0x7f03c8143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:02.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.590+0000 7f03cfdf0640 1 -- 172.21.15.45:0/2077077414 >> 172.21.15.45:0/2077077414 conn(0x7f03c80d2e90 msgr2=0x7f03c806ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:02.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.590+0000 7f03cfdf0640 1 -- 172.21.15.45:0/2077077414 shutdown_connections 2024-05-08T05:43:02.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.590+0000 7f03cfdf0640 1 -- 172.21.15.45:0/2077077414 wait complete. 2024-05-08T05:43:02.605 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T05:43:02.605 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:43:02.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.712+0000 7f8e3faa9640 1 Processor -- start 2024-05-08T05:43:02.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.712+0000 7f8e3faa9640 1 -- start start 2024-05-08T05:43:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.713+0000 7f8e3faa9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8e38070d20 0x7f8e380711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.713+0000 7f8e3faa9640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e3806e110 con 0x7f8e380b2490 2024-05-08T05:43:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.713+0000 7f8e3faa9640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e3806e3f0 con 0x7f8e381629b0 2024-05-08T05:43:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.713+0000 7f8e3faa9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8e38070a40 con 0x7f8e38070d20 2024-05-08T05:43:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.713+0000 7f8e3d044640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8e380b2490 0x7f8e38070330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37428/0 (socket says 172.21.15.45:37428) 2024-05-08T05:43:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.713+0000 7f8e3d044640 1 -- 172.21.15.45:0/529046386 learned_addr learned my addr 172.21.15.45:0/529046386 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.713+0000 7f8e3e046640 1 --2- 172.21.15.45:0/529046386 >> v2:172.21.15.63:3300/0 conn(0x7f8e38070d20 0x7f8e380711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:02.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.713+0000 7f8e3c843640 1 -- 172.21.15.45:0/529046386 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1547684383 0 0) 0x7f8e3806e110 con 0x7f8e380b2490 2024-05-08T05:43:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.713+0000 7f8e3c843640 1 -- 172.21.15.45:0/529046386 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e24003520 con 0x7f8e380b2490 2024-05-08T05:43:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3c843640 1 -- 172.21.15.45:0/529046386 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3339028993 0 0) 0x7f8e24003520 con 0x7f8e380b2490 2024-05-08T05:43:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3c843640 1 -- 172.21.15.45:0/529046386 >> v2:172.21.15.63:3300/0 conn(0x7f8e38070d20 msgr2=0x7f8e380711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3c843640 1 --2- 172.21.15.45:0/529046386 >> v2:172.21.15.63:3300/0 conn(0x7f8e38070d20 0x7f8e380711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3c843640 1 -- 172.21.15.45:0/529046386 >> v1:172.21.15.63:6789/0 conn(0x7f8e381629b0 legacy=0x7f8e3806eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3c843640 1 -- 172.21.15.45:0/529046386 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3806e6d0 con 0x7f8e380b2490 2024-05-08T05:43:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3c843640 1 -- 172.21.15.45:0/529046386 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f8e280016f0 con 0x7f8e380b2490 2024-05-08T05:43:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3c843640 1 -- 172.21.15.45:0/529046386 >> v1:172.21.15.45:6789/0 conn(0x7f8e380b2490 legacy=0x7f8e38070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3c843640 1 --2- 172.21.15.45:0/529046386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e24003a50 0x7f8e24003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3c843640 1 --2- 172.21.15.45:0/529046386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e24004800 0x7f8e24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3c843640 1 --2- 172.21.15.45:0/529046386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e240052e0 0x7f8e2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3c843640 1 -- 172.21.15.45:0/529046386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8e38070a40 con 0x7f8e24003a50 2024-05-08T05:43:02.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3c843640 1 -- 172.21.15.45:0/529046386 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2400c1b0 con 0x7f8e240052e0 2024-05-08T05:43:02.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3c843640 1 -- 172.21.15.45:0/529046386 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8e2400c3b0 con 0x7f8e24004800 2024-05-08T05:43:02.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3e046640 1 --2- 172.21.15.45:0/529046386 >> v2:172.21.15.63:3300/0 conn(0x7f8e38070d20 0x7f8e380711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:02.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3d845640 1 --2- 172.21.15.45:0/529046386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e24003a50 0x7f8e24003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:02.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3d044640 1 --2- 172.21.15.45:0/529046386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e24004800 0x7f8e24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:02.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3e046640 1 --2- 172.21.15.45:0/529046386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e240052e0 0x7f8e2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:02.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3d845640 1 -- 172.21.15.45:0/529046386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e24004800 msgr2=0x7f8e24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:02.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3d845640 1 --2- 172.21.15.45:0/529046386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e24004800 0x7f8e24004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:02.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3d845640 1 -- 172.21.15.45:0/529046386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e240052e0 msgr2=0x7f8e2400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:02.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3d845640 1 --2- 172.21.15.45:0/529046386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e240052e0 0x7f8e2400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:02.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3e046640 1 -- 172.21.15.45:0/529046386 reap_dead start 2024-05-08T05:43:02.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3d845640 1 -- 172.21.15.45:0/529046386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e24003520 con 0x7f8e24003a50 2024-05-08T05:43:02.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.714+0000 7f8e3d044640 1 --2- 172.21.15.45:0/529046386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e24004800 0x7f8e24004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:02.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.715+0000 7f8e3d845640 1 --2- 172.21.15.45:0/529046386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e24003a50 0x7f8e24003e70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8e34001760 tx=0x7f8e24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9830777acb28e235 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:02.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.715+0000 7f8e3c843640 1 -- 172.21.15.45:0/529046386 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e340013b0 con 0x7f8e24003a50 2024-05-08T05:43:02.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.715+0000 7f8e3c843640 1 -- 172.21.15.45:0/529046386 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e340052d0 con 0x7f8e24003a50 2024-05-08T05:43:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.715+0000 7f8e3c843640 1 -- 172.21.15.45:0/529046386 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3402aa50 con 0x7f8e24003a50 2024-05-08T05:43:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.716+0000 7f8e3faa9640 1 -- 172.21.15.45:0/529046386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e24003a50 msgr2=0x7f8e24003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.716+0000 7f8e3faa9640 1 --2- 172.21.15.45:0/529046386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e24003a50 0x7f8e24003e70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8e34001760 tx=0x7f8e24005830 comp rx=0 tx=0).stop 2024-05-08T05:43:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.716+0000 7f8e3faa9640 1 -- 172.21.15.45:0/529046386 shutdown_connections 2024-05-08T05:43:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.716+0000 7f8e3faa9640 1 --2- 172.21.15.45:0/529046386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e24003a50 0x7f8e24003e70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.716+0000 7f8e3faa9640 1 -- 172.21.15.45:0/529046386 >> 172.21.15.45:0/529046386 conn(0x7f8e380b49d0 msgr2=0x7f8e380b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.716+0000 7f8e3faa9640 1 -- 172.21.15.45:0/529046386 shutdown_connections 2024-05-08T05:43:02.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.716+0000 7f8e3faa9640 1 -- 172.21.15.45:0/529046386 wait complete. 2024-05-08T05:43:02.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.717+0000 7f8e3faa9640 1 Processor -- start 2024-05-08T05:43:02.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.717+0000 7f8e3faa9640 1 -- start start 2024-05-08T05:43:02.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e3faa9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e24003a50 0x7f8e38154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:02.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e3faa9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e380b6c60 0x7f8e381f8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:02.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e3faa9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e381f9060 0x7f8e380c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:02.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e3faa9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8e380711d0 con 0x7f8e24003a50 2024-05-08T05:43:02.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e3faa9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3806dec0 con 0x7f8e380b6c60 2024-05-08T05:43:02.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e3faa9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8e38071350 con 0x7f8e381f9060 2024-05-08T05:43:02.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e3d845640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e24003a50 0x7f8e38154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:02.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e3d845640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e24003a50 0x7f8e38154eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47328/0 (socket says 172.21.15.45:47328) 2024-05-08T05:43:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e3d845640 1 -- 172.21.15.45:0/845238178 learned_addr learned my addr 172.21.15.45:0/845238178 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e3e046640 1 --2- 172.21.15.45:0/845238178 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e381f9060 0x7f8e380c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e3d044640 1 --2- 172.21.15.45:0/845238178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e380b6c60 0x7f8e381f8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e3d845640 1 -- 172.21.15.45:0/845238178 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e381f9060 msgr2=0x7f8e380c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e3d845640 1 --2- 172.21.15.45:0/845238178 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e381f9060 0x7f8e380c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e3d845640 1 -- 172.21.15.45:0/845238178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e380b6c60 msgr2=0x7f8e381f8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e3d845640 1 --2- 172.21.15.45:0/845238178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e380b6c60 0x7f8e381f8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e3d845640 1 -- 172.21.15.45:0/845238178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3809f490 con 0x7f8e24003a50 2024-05-08T05:43:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e3d044640 1 --2- 172.21.15.45:0/845238178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e380b6c60 0x7f8e381f8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e3d845640 1 --2- 172.21.15.45:0/845238178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e24003a50 0x7f8e38154eb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8e34005040 tx=0x7f8e34001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e2e7fc640 1 -- 172.21.15.45:0/845238178 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3403f030 con 0x7f8e24003a50 2024-05-08T05:43:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e2e7fc640 1 -- 172.21.15.45:0/845238178 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e34005480 con 0x7f8e24003a50 2024-05-08T05:43:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e3faa9640 1 -- 172.21.15.45:0/845238178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e380b70c0 con 0x7f8e24003a50 2024-05-08T05:43:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.718+0000 7f8e2e7fc640 1 -- 172.21.15.45:0/845238178 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e34037040 con 0x7f8e24003a50 2024-05-08T05:43:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.719+0000 7f8e3faa9640 1 -- 172.21.15.45:0/845238178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e381f9830 con 0x7f8e24003a50 2024-05-08T05:43:02.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.719+0000 7f8e3faa9640 1 -- 172.21.15.45:0/845238178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e381f9a70 con 0x7f8e24003a50 2024-05-08T05:43:02.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.720+0000 7f8e2e7fc640 1 -- 172.21.15.45:0/845238178 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8e34042070 con 0x7f8e24003a50 2024-05-08T05:43:02.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.721+0000 7f8e2e7fc640 1 --2- 172.21.15.45:0/845238178 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8e1c076300 0x7f8e1c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:02.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.721+0000 7f8e2e7fc640 1 -- 172.21.15.45:0/845238178 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7f8e340b70b0 con 0x7f8e24003a50 2024-05-08T05:43:02.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.721+0000 7f8e3d044640 1 --2- 172.21.15.45:0/845238178 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8e1c076300 0x7f8e1c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:02.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.721+0000 7f8e3d044640 1 --2- 172.21.15.45:0/845238178 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8e1c076300 0x7f8e1c0787c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8e28001eb0 tx=0x7f8e28001c70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:02.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.725+0000 7f8e2e7fc640 1 -- 172.21.15.45:0/845238178 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8e381f9a70 con 0x7f8e24003a50 2024-05-08T05:43:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.955+0000 7f8e3faa9640 1 -- 172.21.15.45:0/845238178 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e380693f0 con 0x7f8e24003a50 2024-05-08T05:43:02.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.956+0000 7f8e2e7fc640 1 -- 172.21.15.45:0/845238178 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5181 (secure 0 0 0) 0x7f8e34080870 con 0x7f8e24003a50 2024-05-08T05:43:02.957 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:02.958 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4262647207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4262647207},{"type":"v1","addr":"172.21.15.63:6837","nonce":4262647207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5277,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2872370835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2872370835},{"type":"v1","addr":"172.21.15.45:6834","nonce":2872370835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1909488716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1909488716},{"type":"v1","addr":"172.21.15.45:6835","nonce":1909488716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:42:58.943500+0000","modified":"2024-05-08T05:43:01.967189+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6836/4205418057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":4205418057},{"type":"v1","addr":"172.21.15.63:6836","nonce":4205418057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5292,"qdb_cluster":[5292]},"id":3}]} 2024-05-08T05:43:02.958 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-05-08T05:43:02.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.958+0000 7f8e3faa9640 1 -- 172.21.15.45:0/845238178 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8e1c076300 msgr2=0x7f8e1c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:02.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.958+0000 7f8e3faa9640 1 --2- 172.21.15.45:0/845238178 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8e1c076300 0x7f8e1c0787c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8e28001eb0 tx=0x7f8e28001c70 comp rx=0 tx=0).stop 2024-05-08T05:43:02.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.958+0000 7f8e3faa9640 1 -- 172.21.15.45:0/845238178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e24003a50 msgr2=0x7f8e38154eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:02.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.958+0000 7f8e3faa9640 1 --2- 172.21.15.45:0/845238178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e24003a50 0x7f8e38154eb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8e34005040 tx=0x7f8e34001120 comp rx=0 tx=0).stop 2024-05-08T05:43:02.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.958+0000 7f8e3faa9640 1 -- 172.21.15.45:0/845238178 shutdown_connections 2024-05-08T05:43:02.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.958+0000 7f8e3faa9640 1 --2- 172.21.15.45:0/845238178 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8e1c076300 0x7f8e1c0787c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:02.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.958+0000 7f8e3faa9640 1 --2- 172.21.15.45:0/845238178 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e381f9060 0x7f8e380c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:02.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.958+0000 7f8e3faa9640 1 --2- 172.21.15.45:0/845238178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e380b6c60 0x7f8e381f8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:02.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.958+0000 7f8e3faa9640 1 --2- 172.21.15.45:0/845238178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e24003a50 0x7f8e38154eb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:02.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.958+0000 7f8e3faa9640 1 -- 172.21.15.45:0/845238178 >> 172.21.15.45:0/845238178 conn(0x7f8e380b49d0 msgr2=0x7f8e3815ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:02.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.958+0000 7f8e3faa9640 1 -- 172.21.15.45:0/845238178 shutdown_connections 2024-05-08T05:43:02.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:02.958+0000 7f8e3faa9640 1 -- 172.21.15.45:0/845238178 wait complete. 2024-05-08T05:43:02.970 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:43:03.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.073+0000 7f19f5171640 1 Processor -- start 2024-05-08T05:43:03.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.073+0000 7f19f5171640 1 -- start start 2024-05-08T05:43:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.074+0000 7f19f5171640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19f006e0f0 0x7f19f015c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.074+0000 7f19f5171640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f19f0072840 0x7f19f00d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.074+0000 7f19f5171640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19f00d14b0 con 0x7f19f006e4d0 2024-05-08T05:43:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.074+0000 7f19f5171640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f19f015c920 con 0x7f19f0072840 2024-05-08T05:43:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.074+0000 7f19f5171640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f19f015cc00 con 0x7f19f006e0f0 2024-05-08T05:43:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.074+0000 7f19e659c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f19f0072840 0x7f19f00d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.074+0000 7f19e659c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f19f0072840 0x7f19f00d0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47336/0 (socket says 172.21.15.45:47336) 2024-05-08T05:43:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.074+0000 7f19e659c640 1 -- 172.21.15.45:0/974856117 learned_addr learned my addr 172.21.15.45:0/974856117 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.074+0000 7f19eed9d640 1 --2- 172.21.15.45:0/974856117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19f006e0f0 0x7f19f015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.074+0000 7f19e659c640 1 -- 172.21.15.45:0/974856117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19f006e0f0 msgr2=0x7f19f015c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.074+0000 7f19e659c640 1 --2- 172.21.15.45:0/974856117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19f006e0f0 0x7f19f015c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.074+0000 7f19e659c640 1 -- 172.21.15.45:0/974856117 >> v1:172.21.15.63:6789/0 conn(0x7f19f006e4d0 legacy=0x7f19f00b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:03.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.074+0000 7f19e659c640 1 -- 172.21.15.45:0/974856117 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f015cee0 con 0x7f19f0072840 2024-05-08T05:43:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.075+0000 7f19e659c640 1 --2- 172.21.15.45:0/974856117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f0072840 0x7f19f00d0f70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009870 tx=0x7f19d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc309ec3476fd1f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.075+0000 7f19ee59c640 1 -- 172.21.15.45:0/974856117 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8037070 con 0x7f19f0072840 2024-05-08T05:43:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.075+0000 7f19ee59c640 1 -- 172.21.15.45:0/974856117 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19d8032440 con 0x7f19f0072840 2024-05-08T05:43:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.075+0000 7f19ee59c640 1 -- 172.21.15.45:0/974856117 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8032810 con 0x7f19f0072840 2024-05-08T05:43:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.076+0000 7f19f5171640 1 -- 172.21.15.45:0/974856117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f0072840 msgr2=0x7f19f00d0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.076+0000 7f19f5171640 1 --2- 172.21.15.45:0/974856117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f0072840 0x7f19f00d0f70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009870 tx=0x7f19d802a270 comp rx=0 tx=0).stop 2024-05-08T05:43:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.076+0000 7f19f5171640 1 -- 172.21.15.45:0/974856117 shutdown_connections 2024-05-08T05:43:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.076+0000 7f19f5171640 1 --2- 172.21.15.45:0/974856117 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f0072840 0x7f19f00d0f70 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.076+0000 7f19f5171640 1 --2- 172.21.15.45:0/974856117 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19f006e0f0 0x7f19f015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.076+0000 7f19f5171640 1 -- 172.21.15.45:0/974856117 >> 172.21.15.45:0/974856117 conn(0x7f19f006c3a0 msgr2=0x7f19f006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.076+0000 7f19f5171640 1 -- 172.21.15.45:0/974856117 shutdown_connections 2024-05-08T05:43:03.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.076+0000 7f19f5171640 1 -- 172.21.15.45:0/974856117 wait complete. 2024-05-08T05:43:03.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.077+0000 7f19f5171640 1 Processor -- start 2024-05-08T05:43:03.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.077+0000 7f19f5171640 1 -- start start 2024-05-08T05:43:03.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.078+0000 7f19f5171640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19f006e0f0 0x7f19f01f9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.078+0000 7f19f5171640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19f006e4d0 0x7f19f00c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.078+0000 7f19f5171640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f0072840 0x7f19f00b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.078+0000 7f19f5171640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f19f00ae650 con 0x7f19f0072840 2024-05-08T05:43:03.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.078+0000 7f19f5171640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f19f006e980 con 0x7f19f006e0f0 2024-05-08T05:43:03.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.078+0000 7f19f5171640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f19f00ae7d0 con 0x7f19f006e4d0 2024-05-08T05:43:03.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.078+0000 7f19ef59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f0072840 0x7f19f00b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:03.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.078+0000 7f19ef59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f0072840 0x7f19f00b3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47348/0 (socket says 172.21.15.45:47348) 2024-05-08T05:43:03.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.078+0000 7f19ef59e640 1 -- 172.21.15.45:0/4012301261 learned_addr learned my addr 172.21.15.45:0/4012301261 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:03.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.078+0000 7f19eed9d640 1 --2- 172.21.15.45:0/4012301261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19f006e0f0 0x7f19f01f9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:03.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.078+0000 7f19e659c640 1 --2- 172.21.15.45:0/4012301261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19f006e4d0 0x7f19f00c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:03.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.078+0000 7f19ef59e640 1 -- 172.21.15.45:0/4012301261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19f006e4d0 msgr2=0x7f19f00c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:03.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.078+0000 7f19ef59e640 1 --2- 172.21.15.45:0/4012301261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19f006e4d0 0x7f19f00c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.078+0000 7f19ef59e640 1 -- 172.21.15.45:0/4012301261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19f006e0f0 msgr2=0x7f19f01f9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:03.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.078+0000 7f19ef59e640 1 --2- 172.21.15.45:0/4012301261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19f006e0f0 0x7f19f01f9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.079+0000 7f19ef59e640 1 -- 172.21.15.45:0/4012301261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f01f9780 con 0x7f19f0072840 2024-05-08T05:43:03.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.079+0000 7f19ef59e640 1 --2- 172.21.15.45:0/4012301261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f0072840 0x7f19f00b3f90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c7d0 tx=0x7f19e800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:03.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.079+0000 7f19e7fff640 1 -- 172.21.15.45:0/4012301261 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8003b70 con 0x7f19f0072840 2024-05-08T05:43:03.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.079+0000 7f19e7fff640 1 -- 172.21.15.45:0/4012301261 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e8003d10 con 0x7f19f0072840 2024-05-08T05:43:03.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.079+0000 7f19f5171640 1 -- 172.21.15.45:0/4012301261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f01581e0 con 0x7f19f0072840 2024-05-08T05:43:03.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.079+0000 7f19e7fff640 1 -- 172.21.15.45:0/4012301261 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e800aa70 con 0x7f19f0072840 2024-05-08T05:43:03.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.079+0000 7f19f5171640 1 -- 172.21.15.45:0/4012301261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f00cc960 con 0x7f19f0072840 2024-05-08T05:43:03.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.080+0000 7f19f5171640 1 -- 172.21.15.45:0/4012301261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f0070210 con 0x7f19f0072840 2024-05-08T05:43:03.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.081+0000 7f19e7fff640 1 -- 172.21.15.45:0/4012301261 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f19e800ac10 con 0x7f19f0072840 2024-05-08T05:43:03.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.081+0000 7f19e7fff640 1 --2- 172.21.15.45:0/4012301261 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f19d00762d0 0x7f19d0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.081+0000 7f19e7fff640 1 -- 172.21.15.45:0/4012301261 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7f19e80976f0 con 0x7f19f0072840 2024-05-08T05:43:03.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.081+0000 7f19eed9d640 1 --2- 172.21.15.45:0/4012301261 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f19d00762d0 0x7f19d0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:03.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.082+0000 7f19eed9d640 1 --2- 172.21.15.45:0/4012301261 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f19d00762d0 0x7f19d0078790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f19e0006fd0 tx=0x7f19e0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:03.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.085+0000 7f19e7fff640 1 -- 172.21.15.45:0/4012301261 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f19f0070210 con 0x7f19f0072840 2024-05-08T05:43:03.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.277+0000 7f19f5171640 1 -- 172.21.15.45:0/4012301261 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19f00d1030 con 0x7f19f0072840 2024-05-08T05:43:03.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.278+0000 7f19e7fff640 1 -- 172.21.15.45:0/4012301261 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16894 (secure 0 0 0) 0x7f19f01f9780 con 0x7f19f0072840 2024-05-08T05:43:03.279 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:03.280 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":47,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:43:00.576254+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:42:56.614794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T05:42:57.898381+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:43:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.280+0000 7f19f5171640 1 -- 172.21.15.45:0/4012301261 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f19d00762d0 msgr2=0x7f19d0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.280+0000 7f19f5171640 1 --2- 172.21.15.45:0/4012301261 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f19d00762d0 0x7f19d0078790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f19e0006fd0 tx=0x7f19e0008040 comp rx=0 tx=0).stop 2024-05-08T05:43:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.280+0000 7f19f5171640 1 -- 172.21.15.45:0/4012301261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f0072840 msgr2=0x7f19f00b3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.280+0000 7f19f5171640 1 --2- 172.21.15.45:0/4012301261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f0072840 0x7f19f00b3f90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c7d0 tx=0x7f19e800cc90 comp rx=0 tx=0).stop 2024-05-08T05:43:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.280+0000 7f19f5171640 1 -- 172.21.15.45:0/4012301261 shutdown_connections 2024-05-08T05:43:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.280+0000 7f19f5171640 1 --2- 172.21.15.45:0/4012301261 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f19d00762d0 0x7f19d0078790 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.280+0000 7f19f5171640 1 --2- 172.21.15.45:0/4012301261 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f19f006e4d0 0x7f19f00c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.280+0000 7f19f5171640 1 --2- 172.21.15.45:0/4012301261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f19f0072840 0x7f19f00b3f90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.280+0000 7f19f5171640 1 --2- 172.21.15.45:0/4012301261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19f006e0f0 0x7f19f01f9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.280+0000 7f19f5171640 1 -- 172.21.15.45:0/4012301261 >> 172.21.15.45:0/4012301261 conn(0x7f19f006c3a0 msgr2=0x7f19f015de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:03.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.280+0000 7f19f5171640 1 -- 172.21.15.45:0/4012301261 shutdown_connections 2024-05-08T05:43:03.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.280+0000 7f19f5171640 1 -- 172.21.15.45:0/4012301261 wait complete. 2024-05-08T05:43:03.293 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T05:43:03.293 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:43:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.396+0000 7f2a8414e640 1 Processor -- start 2024-05-08T05:43:03.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.396+0000 7f2a8414e640 1 -- start start 2024-05-08T05:43:03.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.397+0000 7f2a8414e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a7c06e0f0 0x7f2a7c0cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.397+0000 7f2a8414e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2a7c0d0360 0x7f2a7c0b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.397+0000 7f2a8414e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a7c15d2a0 con 0x7f2a7c06f060 2024-05-08T05:43:03.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.397+0000 7f2a8414e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2a7c06f440 con 0x7f2a7c0d0360 2024-05-08T05:43:03.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.397+0000 7f2a8414e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2a7c1609b0 con 0x7f2a7c06e0f0 2024-05-08T05:43:03.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.397+0000 7f2a81eea640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2a7c06f060 0x7f2a7c06fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37436/0 (socket says 172.21.15.45:37436) 2024-05-08T05:43:03.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.397+0000 7f2a81eea640 1 -- 172.21.15.45:0/1619327136 learned_addr learned my addr 172.21.15.45:0/1619327136 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:03.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.397+0000 7f2a80ee8640 1 -- 172.21.15.45:0/1619327136 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1818473432 0 0) 0x7f2a7c15d2a0 con 0x7f2a7c06f060 2024-05-08T05:43:03.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.397+0000 7f2a816e9640 1 --2- 172.21.15.45:0/1619327136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a7c06e0f0 0x7f2a7c0cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.397+0000 7f2a826eb640 1 --2- 172.21.15.45:0/1619327136 >> v2:172.21.15.63:3300/0 conn(0x7f2a7c0d0360 0x7f2a7c0b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.397+0000 7f2a80ee8640 1 -- 172.21.15.45:0/1619327136 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a68003520 con 0x7f2a7c06f060 2024-05-08T05:43:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a80ee8640 1 -- 172.21.15.45:0/1619327136 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2880410362 0 0) 0x7f2a68003520 con 0x7f2a7c06f060 2024-05-08T05:43:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a80ee8640 1 -- 172.21.15.45:0/1619327136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a7c06e0f0 msgr2=0x7f2a7c0cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a80ee8640 1 --2- 172.21.15.45:0/1619327136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a7c06e0f0 0x7f2a7c0cfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a80ee8640 1 -- 172.21.15.45:0/1619327136 >> v2:172.21.15.63:3300/0 conn(0x7f2a7c0d0360 msgr2=0x7f2a7c0b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a80ee8640 1 --2- 172.21.15.45:0/1619327136 >> v2:172.21.15.63:3300/0 conn(0x7f2a7c0d0360 0x7f2a7c0b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a80ee8640 1 -- 172.21.15.45:0/1619327136 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7c160c90 con 0x7f2a7c06f060 2024-05-08T05:43:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a80ee8640 1 -- 172.21.15.45:0/1619327136 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f2a780016f0 con 0x7f2a7c06f060 2024-05-08T05:43:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a80ee8640 1 -- 172.21.15.45:0/1619327136 >> v1:172.21.15.45:6789/0 conn(0x7f2a7c06f060 legacy=0x7f2a7c06fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a80ee8640 1 --2- 172.21.15.45:0/1619327136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a7c06e0f0 0x7f2a68003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a80ee8640 1 --2- 172.21.15.45:0/1619327136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a68003fd0 0x7f2a68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a80ee8640 1 --2- 172.21.15.45:0/1619327136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a68004d40 0x7f2a680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a80ee8640 1 -- 172.21.15.45:0/1619327136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c06f440 con 0x7f2a68003fd0 2024-05-08T05:43:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a80ee8640 1 -- 172.21.15.45:0/1619327136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c1609b0 con 0x7f2a68004d40 2024-05-08T05:43:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a80ee8640 1 -- 172.21.15.45:0/1619327136 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2a6800ba10 con 0x7f2a7c06e0f0 2024-05-08T05:43:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a816e9640 1 --2- 172.21.15.45:0/1619327136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a68003fd0 0x7f2a68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a81eea640 1 --2- 172.21.15.45:0/1619327136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a7c06e0f0 0x7f2a68003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a826eb640 1 --2- 172.21.15.45:0/1619327136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a68004d40 0x7f2a680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a816e9640 1 -- 172.21.15.45:0/1619327136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a7c06e0f0 msgr2=0x7f2a68003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a816e9640 1 --2- 172.21.15.45:0/1619327136 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a7c06e0f0 0x7f2a68003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a816e9640 1 -- 172.21.15.45:0/1619327136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a68004d40 msgr2=0x7f2a680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a816e9640 1 --2- 172.21.15.45:0/1619327136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a68004d40 0x7f2a680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.398+0000 7f2a816e9640 1 -- 172.21.15.45:0/1619327136 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a68003520 con 0x7f2a68003fd0 2024-05-08T05:43:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.399+0000 7f2a816e9640 1 --2- 172.21.15.45:0/1619327136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a68003fd0 0x7f2a68004800 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009af0 tx=0x7f2a68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a15afc3c3378e12 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.399+0000 7f2a80ee8640 1 -- 172.21.15.45:0/1619327136 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c036070 con 0x7f2a68003fd0 2024-05-08T05:43:03.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.399+0000 7f2a80ee8640 1 -- 172.21.15.45:0/1619327136 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6c031400 con 0x7f2a68003fd0 2024-05-08T05:43:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.399+0000 7f2a80ee8640 1 -- 172.21.15.45:0/1619327136 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c009690 con 0x7f2a68003fd0 2024-05-08T05:43:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.400+0000 7f2a8414e640 1 -- 172.21.15.45:0/1619327136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a68003fd0 msgr2=0x7f2a68004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.400+0000 7f2a8414e640 1 --2- 172.21.15.45:0/1619327136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a68003fd0 0x7f2a68004800 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009af0 tx=0x7f2a68005830 comp rx=0 tx=0).stop 2024-05-08T05:43:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.400+0000 7f2a826eb640 1 -- 172.21.15.45:0/1619327136 reap_dead start 2024-05-08T05:43:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.400+0000 7f2a8414e640 1 -- 172.21.15.45:0/1619327136 shutdown_connections 2024-05-08T05:43:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.400+0000 7f2a8414e640 1 -- 172.21.15.45:0/1619327136 >> 172.21.15.45:0/1619327136 conn(0x7f2a7c06c3a0 msgr2=0x7f2a7c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.400+0000 7f2a8414e640 1 -- 172.21.15.45:0/1619327136 shutdown_connections 2024-05-08T05:43:03.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.400+0000 7f2a8414e640 1 -- 172.21.15.45:0/1619327136 wait complete. 2024-05-08T05:43:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.401+0000 7f2a8414e640 1 Processor -- start 2024-05-08T05:43:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.401+0000 7f2a8414e640 1 -- start start 2024-05-08T05:43:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.402+0000 7f2a8414e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a68003fd0 0x7f2a7c153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.402+0000 7f2a8414e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a7c1434c0 0x7f2a7c151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.402+0000 7f2a8414e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a7c13fd10 0x7f2a7c140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.402+0000 7f2a8414e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c161530 con 0x7f2a7c1434c0 2024-05-08T05:43:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.402+0000 7f2a8414e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c161250 con 0x7f2a68003fd0 2024-05-08T05:43:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.402+0000 7f2a8414e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2a7c161810 con 0x7f2a7c13fd10 2024-05-08T05:43:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.402+0000 7f2a816e9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a7c1434c0 0x7f2a7c151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.402+0000 7f2a816e9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a7c1434c0 0x7f2a7c151d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47360/0 (socket says 172.21.15.45:47360) 2024-05-08T05:43:03.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.402+0000 7f2a816e9640 1 -- 172.21.15.45:0/3394817459 learned_addr learned my addr 172.21.15.45:0/3394817459 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.402+0000 7f2a826eb640 1 --2- 172.21.15.45:0/3394817459 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a7c13fd10 0x7f2a7c140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.402+0000 7f2a81eea640 1 --2- 172.21.15.45:0/3394817459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a68003fd0 0x7f2a7c153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.402+0000 7f2a816e9640 1 -- 172.21.15.45:0/3394817459 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a7c13fd10 msgr2=0x7f2a7c140190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.402+0000 7f2a816e9640 1 --2- 172.21.15.45:0/3394817459 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a7c13fd10 0x7f2a7c140190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.402+0000 7f2a816e9640 1 -- 172.21.15.45:0/3394817459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a68003fd0 msgr2=0x7f2a7c153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.402+0000 7f2a816e9640 1 --2- 172.21.15.45:0/3394817459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a68003fd0 0x7f2a7c153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.402+0000 7f2a816e9640 1 -- 172.21.15.45:0/3394817459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7c1406d0 con 0x7f2a7c1434c0 2024-05-08T05:43:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.402+0000 7f2a816e9640 1 --2- 172.21.15.45:0/3394817459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a7c1434c0 0x7f2a7c151d30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009af0 tx=0x7f2a6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.403+0000 7f2a727fc640 1 -- 172.21.15.45:0/3394817459 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c040020 con 0x7f2a7c1434c0 2024-05-08T05:43:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.403+0000 7f2a727fc640 1 -- 172.21.15.45:0/3394817459 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6c003810 con 0x7f2a7c1434c0 2024-05-08T05:43:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.403+0000 7f2a8414e640 1 -- 172.21.15.45:0/3394817459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7c154120 con 0x7f2a7c1434c0 2024-05-08T05:43:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.403+0000 7f2a727fc640 1 -- 172.21.15.45:0/3394817459 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c036040 con 0x7f2a7c1434c0 2024-05-08T05:43:03.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.403+0000 7f2a8414e640 1 -- 172.21.15.45:0/3394817459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a7c1546e0 con 0x7f2a7c1434c0 2024-05-08T05:43:03.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.404+0000 7f2a8414e640 1 -- 172.21.15.45:0/3394817459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a7c0693f0 con 0x7f2a7c1434c0 2024-05-08T05:43:03.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.404+0000 7f2a727fc640 1 -- 172.21.15.45:0/3394817459 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2a6c043020 con 0x7f2a7c1434c0 2024-05-08T05:43:03.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.405+0000 7f2a727fc640 1 --2- 172.21.15.45:0/3394817459 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2a600762d0 0x7f2a60078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.405+0000 7f2a727fc640 1 -- 172.21.15.45:0/3394817459 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7f2a6c0b7010 con 0x7f2a7c1434c0 2024-05-08T05:43:03.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.405+0000 7f2a81eea640 1 --2- 172.21.15.45:0/3394817459 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2a600762d0 0x7f2a60078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:03.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.405+0000 7f2a81eea640 1 --2- 172.21.15.45:0/3394817459 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2a600762d0 0x7f2a60078790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2a78000f60 tx=0x7f2a78000ed0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:03.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.409+0000 7f2a727fc640 1 -- 172.21.15.45:0/3394817459 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2a6c080190 con 0x7f2a7c1434c0 2024-05-08T05:43:03.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.601+0000 7f2a8414e640 1 -- 172.21.15.45:0/3394817459 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2a7c143940 con 0x7f2a600762d0 2024-05-08T05:43:03.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.608+0000 7f2a727fc640 1 -- 172.21.15.45:0/3394817459 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248351 (secure 0 0 0) 0x7f2a7c143940 con 0x7f2a600762d0 2024-05-08T05:43:03.612 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:03.613 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:43:03.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.611+0000 7f2a8414e640 1 -- 172.21.15.45:0/3394817459 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2a600762d0 msgr2=0x7f2a60078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:03.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.611+0000 7f2a8414e640 1 --2- 172.21.15.45:0/3394817459 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2a600762d0 0x7f2a60078790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2a78000f60 tx=0x7f2a78000ed0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.611+0000 7f2a8414e640 1 -- 172.21.15.45:0/3394817459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a7c1434c0 msgr2=0x7f2a7c151d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.611+0000 7f2a8414e640 1 --2- 172.21.15.45:0/3394817459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a7c1434c0 0x7f2a7c151d30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009af0 tx=0x7f2a6c004730 comp rx=0 tx=0).stop 2024-05-08T05:43:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.611+0000 7f2a8414e640 1 -- 172.21.15.45:0/3394817459 shutdown_connections 2024-05-08T05:43:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.611+0000 7f2a8414e640 1 --2- 172.21.15.45:0/3394817459 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2a600762d0 0x7f2a60078790 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.611+0000 7f2a8414e640 1 --2- 172.21.15.45:0/3394817459 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a7c13fd10 0x7f2a7c140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.611+0000 7f2a8414e640 1 --2- 172.21.15.45:0/3394817459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a7c1434c0 0x7f2a7c151d30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.611+0000 7f2a8414e640 1 --2- 172.21.15.45:0/3394817459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a68003fd0 0x7f2a7c153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.611+0000 7f2a8414e640 1 -- 172.21.15.45:0/3394817459 >> 172.21.15.45:0/3394817459 conn(0x7f2a7c06c3a0 msgr2=0x7f2a7c15d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.611+0000 7f2a8414e640 1 -- 172.21.15.45:0/3394817459 shutdown_connections 2024-05-08T05:43:03.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.611+0000 7f2a8414e640 1 -- 172.21.15.45:0/3394817459 wait complete. 2024-05-08T05:43:03.623 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-05-08T05:43:02.853692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":116,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84,"ondisk_log_size":84,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238120,"kb_used_data":5276,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736600,"statfs":{"total":773094113280,"available":772850278400,"internally_reserved":0,"allocated":5402624,"data_stored":2999561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":742,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.300007"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.567012+0000","last_change":"2024-05-08T05:42:58.567012+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.567012+0000","last_undegraded":"2024-05-08T05:42:58.567012+0000","last_fullsized":"2024-05-08T05:42:58.567012+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.590291+0000","last_change":"2024-05-08T05:42:59.586491+0000","last_active":"2024-05-08T05:42:59.590291+0000","last_peered":"2024-05-08T05:42:59.590291+0000","last_clean":"2024-05-08T05:42:59.590291+0000","last_became_active":"2024-05-08T05:42:59.586233+0000","last_became_peered":"2024-05-08T05:42:59.586233+0000","last_unstale":"2024-05-08T05:42:59.590291+0000","last_undegraded":"2024-05-08T05:42:59.590291+0000","last_fullsized":"2024-05-08T05:42:59.590291+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:32.677283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.565275+0000","last_change":"2024-05-08T05:42:58.565275+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.565275+0000","last_undegraded":"2024-05-08T05:42:58.565275+0000","last_fullsized":"2024-05-08T05:42:58.565275+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.595331+0000","last_change":"2024-05-08T05:42:59.589298+0000","last_active":"2024-05-08T05:42:59.595331+0000","last_peered":"2024-05-08T05:42:59.595331+0000","last_clean":"2024-05-08T05:42:59.595331+0000","last_became_active":"2024-05-08T05:42:59.588989+0000","last_became_peered":"2024-05-08T05:42:59.588989+0000","last_unstale":"2024-05-08T05:42:59.595331+0000","last_undegraded":"2024-05-08T05:42:59.595331+0000","last_fullsized":"2024-05-08T05:42:59.595331+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:54:26.126199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.559296+0000","last_change":"2024-05-08T05:42:58.559296+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.559296+0000","last_undegraded":"2024-05-08T05:42:58.559296+0000","last_fullsized":"2024-05-08T05:42:58.559296+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.569796+0000","last_change":"2024-05-08T05:42:58.569796+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.569796+0000","last_undegraded":"2024-05-08T05:42:58.569796+0000","last_fullsized":"2024-05-08T05:42:58.569796+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.565860+0000","last_change":"2024-05-08T05:42:58.565860+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.565860+0000","last_undegraded":"2024-05-08T05:42:58.565860+0000","last_fullsized":"2024-05-08T05:42:58.565860+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.564839+0000","last_change":"2024-05-08T05:42:58.564839+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.564839+0000","last_undegraded":"2024-05-08T05:42:58.564839+0000","last_fullsized":"2024-05-08T05:42:58.564839+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566530+0000","last_change":"2024-05-08T05:42:58.566530+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566530+0000","last_undegraded":"2024-05-08T05:42:58.566530+0000","last_fullsized":"2024-05-08T05:42:58.566530+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566857+0000","last_change":"2024-05-08T05:42:58.566857+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566857+0000","last_undegraded":"2024-05-08T05:42:58.566857+0000","last_fullsized":"2024-05-08T05:42:58.566857+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.560543+0000","last_change":"2024-05-08T05:42:58.560543+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.560543+0000","last_undegraded":"2024-05-08T05:42:58.560543+0000","last_fullsized":"2024-05-08T05:42:58.560543+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.567647+0000","last_change":"2024-05-08T05:42:58.567647+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.567647+0000","last_undegraded":"2024-05-08T05:42:58.567647+0000","last_fullsized":"2024-05-08T05:42:58.567647+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.563061+0000","last_change":"2024-05-08T05:42:58.563061+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.563061+0000","last_undegraded":"2024-05-08T05:42:58.563061+0000","last_fullsized":"2024-05-08T05:42:58.563061+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.562034+0000","last_change":"2024-05-08T05:42:58.562034+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.562034+0000","last_undegraded":"2024-05-08T05:42:58.562034+0000","last_fullsized":"2024-05-08T05:42:58.562034+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.598708+0000","last_change":"2024-05-08T05:42:59.593026+0000","last_active":"2024-05-08T05:42:59.598708+0000","last_peered":"2024-05-08T05:42:59.598708+0000","last_clean":"2024-05-08T05:42:59.598708+0000","last_became_active":"2024-05-08T05:42:59.592706+0000","last_became_peered":"2024-05-08T05:42:59.592706+0000","last_unstale":"2024-05-08T05:42:59.598708+0000","last_undegraded":"2024-05-08T05:42:59.598708+0000","last_fullsized":"2024-05-08T05:42:59.598708+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:34.093891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.561473+0000","last_change":"2024-05-08T05:42:58.561473+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.561473+0000","last_undegraded":"2024-05-08T05:42:58.561473+0000","last_fullsized":"2024-05-08T05:42:58.561473+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566253+0000","last_change":"2024-05-08T05:42:58.566253+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566253+0000","last_undegraded":"2024-05-08T05:42:58.566253+0000","last_fullsized":"2024-05-08T05:42:58.566253+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.598545+0000","last_change":"2024-05-08T05:42:59.592278+0000","last_active":"2024-05-08T05:42:59.598545+0000","last_peered":"2024-05-08T05:42:59.598545+0000","last_clean":"2024-05-08T05:42:59.598545+0000","last_became_active":"2024-05-08T05:42:59.591985+0000","last_became_peered":"2024-05-08T05:42:59.591985+0000","last_unstale":"2024-05-08T05:42:59.598545+0000","last_undegraded":"2024-05-08T05:42:59.598545+0000","last_fullsized":"2024-05-08T05:42:59.598545+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:42:12.700183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.564859+0000","last_change":"2024-05-08T05:42:58.564859+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.564859+0000","last_undegraded":"2024-05-08T05:42:58.564859+0000","last_fullsized":"2024-05-08T05:42:58.564859+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.590796+0000","last_change":"2024-05-08T05:42:59.584649+0000","last_active":"2024-05-08T05:42:59.590796+0000","last_peered":"2024-05-08T05:42:59.590796+0000","last_clean":"2024-05-08T05:42:59.590796+0000","last_became_active":"2024-05-08T05:42:59.584233+0000","last_became_peered":"2024-05-08T05:42:59.584233+0000","last_unstale":"2024-05-08T05:42:59.590796+0000","last_undegraded":"2024-05-08T05:42:59.590796+0000","last_fullsized":"2024-05-08T05:42:59.590796+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:49.405283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.595031+0000","last_change":"2024-05-08T05:42:59.591062+0000","last_active":"2024-05-08T05:42:59.595031+0000","last_peered":"2024-05-08T05:42:59.595031+0000","last_clean":"2024-05-08T05:42:59.595031+0000","last_became_active":"2024-05-08T05:42:59.590680+0000","last_became_peered":"2024-05-08T05:42:59.590680+0000","last_unstale":"2024-05-08T05:42:59.595031+0000","last_undegraded":"2024-05-08T05:42:59.595031+0000","last_fullsized":"2024-05-08T05:42:59.595031+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:05:27.975200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.563363+0000","last_change":"2024-05-08T05:42:58.563363+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.563363+0000","last_undegraded":"2024-05-08T05:42:58.563363+0000","last_fullsized":"2024-05-08T05:42:58.563363+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.560421+0000","last_change":"2024-05-08T05:42:58.560421+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.560421+0000","last_undegraded":"2024-05-08T05:42:58.560421+0000","last_fullsized":"2024-05-08T05:42:58.560421+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.561384+0000","last_change":"2024-05-08T05:42:58.561384+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.561384+0000","last_undegraded":"2024-05-08T05:42:58.561384+0000","last_fullsized":"2024-05-08T05:42:58.561384+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566986+0000","last_change":"2024-05-08T05:42:58.566986+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566986+0000","last_undegraded":"2024-05-08T05:42:58.566986+0000","last_fullsized":"2024-05-08T05:42:58.566986+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566924+0000","last_change":"2024-05-08T05:42:58.566924+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566924+0000","last_undegraded":"2024-05-08T05:42:58.566924+0000","last_fullsized":"2024-05-08T05:42:58.566924+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.565197+0000","last_change":"2024-05-08T05:42:58.565197+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.565197+0000","last_undegraded":"2024-05-08T05:42:58.565197+0000","last_fullsized":"2024-05-08T05:42:58.565197+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.589440+0000","last_change":"2024-05-08T05:42:59.582258+0000","last_active":"2024-05-08T05:42:59.589440+0000","last_peered":"2024-05-08T05:42:59.589440+0000","last_clean":"2024-05-08T05:42:59.589440+0000","last_became_active":"2024-05-08T05:42:59.581973+0000","last_became_peered":"2024-05-08T05:42:59.581973+0000","last_unstale":"2024-05-08T05:42:59.589440+0000","last_undegraded":"2024-05-08T05:42:59.589440+0000","last_fullsized":"2024-05-08T05:42:59.589440+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:45:38.272590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.562110+0000","last_change":"2024-05-08T05:42:58.562110+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.562110+0000","last_undegraded":"2024-05-08T05:42:58.562110+0000","last_fullsized":"2024-05-08T05:42:58.562110+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.590188+0000","last_change":"2024-05-08T05:42:59.583375+0000","last_active":"2024-05-08T05:42:59.590188+0000","last_peered":"2024-05-08T05:42:59.590188+0000","last_clean":"2024-05-08T05:42:59.590188+0000","last_became_active":"2024-05-08T05:42:59.583026+0000","last_became_peered":"2024-05-08T05:42:59.583026+0000","last_unstale":"2024-05-08T05:42:59.590188+0000","last_undegraded":"2024-05-08T05:42:59.590188+0000","last_fullsized":"2024-05-08T05:42:59.590188+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:09.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.562320+0000","last_change":"2024-05-08T05:42:58.562320+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.562320+0000","last_undegraded":"2024-05-08T05:42:58.562320+0000","last_fullsized":"2024-05-08T05:42:58.562320+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.595084+0000","last_change":"2024-05-08T05:42:59.586488+0000","last_active":"2024-05-08T05:42:59.595084+0000","last_peered":"2024-05-08T05:42:59.595084+0000","last_clean":"2024-05-08T05:42:59.595084+0000","last_became_active":"2024-05-08T05:42:59.585806+0000","last_became_peered":"2024-05-08T05:42:59.585806+0000","last_unstale":"2024-05-08T05:42:59.595084+0000","last_undegraded":"2024-05-08T05:42:59.595084+0000","last_fullsized":"2024-05-08T05:42:59.595084+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:42:12.700183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.598596+0000","last_change":"2024-05-08T05:42:59.593948+0000","last_active":"2024-05-08T05:42:59.598596+0000","last_peered":"2024-05-08T05:42:59.598596+0000","last_clean":"2024-05-08T05:42:59.598596+0000","last_became_active":"2024-05-08T05:42:59.593663+0000","last_became_peered":"2024-05-08T05:42:59.593663+0000","last_unstale":"2024-05-08T05:42:59.598596+0000","last_undegraded":"2024-05-08T05:42:59.598596+0000","last_fullsized":"2024-05-08T05:42:59.598596+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:47:11.811597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"46'4","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.601783+0000","last_change":"2024-05-08T05:42:58.568449+0000","last_active":"2024-05-08T05:42:59.601783+0000","last_peered":"2024-05-08T05:42:59.601783+0000","last_clean":"2024-05-08T05:42:59.601783+0000","last_became_active":"2024-05-08T05:42:58.567378+0000","last_became_peered":"2024-05-08T05:42:58.567378+0000","last_unstale":"2024-05-08T05:42:59.601783+0000","last_undegraded":"2024-05-08T05:42:59.601783+0000","last_fullsized":"2024-05-08T05:42:59.601783+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:04.957071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.591538+0000","last_change":"2024-05-08T05:42:59.583673+0000","last_active":"2024-05-08T05:42:59.591538+0000","last_peered":"2024-05-08T05:42:59.591538+0000","last_clean":"2024-05-08T05:42:59.591538+0000","last_became_active":"2024-05-08T05:42:59.583430+0000","last_became_peered":"2024-05-08T05:42:59.583430+0000","last_unstale":"2024-05-08T05:42:59.591538+0000","last_undegraded":"2024-05-08T05:42:59.591538+0000","last_fullsized":"2024-05-08T05:42:59.591538+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:32.677283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.583411+0000","last_change":"2024-05-08T05:42:58.582169+0000","last_active":"2024-05-08T05:42:58.583411+0000","last_peered":"2024-05-08T05:42:58.583411+0000","last_clean":"2024-05-08T05:42:58.583411+0000","last_became_active":"2024-05-08T05:42:58.581851+0000","last_became_peered":"2024-05-08T05:42:58.581851+0000","last_unstale":"2024-05-08T05:42:58.583411+0000","last_undegraded":"2024-05-08T05:42:58.583411+0000","last_fullsized":"2024-05-08T05:42:58.583411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:28.357714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.590873+0000","last_change":"2024-05-08T05:42:59.585789+0000","last_active":"2024-05-08T05:42:59.590873+0000","last_peered":"2024-05-08T05:42:59.590873+0000","last_clean":"2024-05-08T05:42:59.590873+0000","last_became_active":"2024-05-08T05:42:59.585486+0000","last_became_peered":"2024-05-08T05:42:59.585486+0000","last_unstale":"2024-05-08T05:42:59.590873+0000","last_undegraded":"2024-05-08T05:42:59.590873+0000","last_fullsized":"2024-05-08T05:42:59.590873+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:13.547702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.579505+0000","last_change":"2024-05-08T05:42:58.577001+0000","last_active":"2024-05-08T05:42:58.579505+0000","last_peered":"2024-05-08T05:42:58.579505+0000","last_clean":"2024-05-08T05:42:58.579505+0000","last_became_active":"2024-05-08T05:42:58.575665+0000","last_became_peered":"2024-05-08T05:42:58.575665+0000","last_unstale":"2024-05-08T05:42:58.579505+0000","last_undegraded":"2024-05-08T05:42:58.579505+0000","last_fullsized":"2024-05-08T05:42:58.579505+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:55:31.695068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.567678+0000","last_change":"2024-05-08T05:42:58.567678+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.567678+0000","last_undegraded":"2024-05-08T05:42:58.567678+0000","last_fullsized":"2024-05-08T05:42:58.567678+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.602422+0000","last_change":"2024-05-08T05:42:58.573567+0000","last_active":"2024-05-08T05:42:59.602422+0000","last_peered":"2024-05-08T05:42:59.602422+0000","last_clean":"2024-05-08T05:42:59.602422+0000","last_became_active":"2024-05-08T05:42:58.573220+0000","last_became_peered":"2024-05-08T05:42:58.573220+0000","last_unstale":"2024-05-08T05:42:59.602422+0000","last_undegraded":"2024-05-08T05:42:59.602422+0000","last_fullsized":"2024-05-08T05:42:59.602422+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:15.376533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.558663+0000","last_change":"2024-05-08T05:42:58.558663+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.558663+0000","last_undegraded":"2024-05-08T05:42:58.558663+0000","last_fullsized":"2024-05-08T05:42:58.558663+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.576887+0000","last_change":"2024-05-08T05:42:58.572908+0000","last_active":"2024-05-08T05:42:58.576887+0000","last_peered":"2024-05-08T05:42:58.576887+0000","last_clean":"2024-05-08T05:42:58.576887+0000","last_became_active":"2024-05-08T05:42:58.572613+0000","last_became_peered":"2024-05-08T05:42:58.572613+0000","last_unstale":"2024-05-08T05:42:58.576887+0000","last_undegraded":"2024-05-08T05:42:58.576887+0000","last_fullsized":"2024-05-08T05:42:58.576887+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:39.779366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.598156+0000","last_change":"2024-05-08T05:42:59.589512+0000","last_active":"2024-05-08T05:42:59.598156+0000","last_peered":"2024-05-08T05:42:59.598156+0000","last_clean":"2024-05-08T05:42:59.598156+0000","last_became_active":"2024-05-08T05:42:59.588973+0000","last_became_peered":"2024-05-08T05:42:59.588973+0000","last_unstale":"2024-05-08T05:42:59.598156+0000","last_undegraded":"2024-05-08T05:42:59.598156+0000","last_fullsized":"2024-05-08T05:42:59.598156+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:34:10.738939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.571718+0000","last_change":"2024-05-08T05:42:58.569466+0000","last_active":"2024-05-08T05:42:58.571718+0000","last_peered":"2024-05-08T05:42:58.571718+0000","last_clean":"2024-05-08T05:42:58.571718+0000","last_became_active":"2024-05-08T05:42:58.569048+0000","last_became_peered":"2024-05-08T05:42:58.569048+0000","last_unstale":"2024-05-08T05:42:58.571718+0000","last_undegraded":"2024-05-08T05:42:58.571718+0000","last_fullsized":"2024-05-08T05:42:58.571718+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:30:44.118693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.565142+0000","last_change":"2024-05-08T05:42:58.565142+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.565142+0000","last_undegraded":"2024-05-08T05:42:58.565142+0000","last_fullsized":"2024-05-08T05:42:58.565142+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.575648+0000","last_change":"2024-05-08T05:42:58.568749+0000","last_active":"2024-05-08T05:42:59.575648+0000","last_peered":"2024-05-08T05:42:59.575648+0000","last_clean":"2024-05-08T05:42:59.575648+0000","last_became_active":"2024-05-08T05:42:58.568473+0000","last_became_peered":"2024-05-08T05:42:58.568473+0000","last_unstale":"2024-05-08T05:42:59.575648+0000","last_undegraded":"2024-05-08T05:42:59.575648+0000","last_fullsized":"2024-05-08T05:42:59.575648+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:55:31.695068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.598506+0000","last_change":"2024-05-08T05:42:59.591341+0000","last_active":"2024-05-08T05:42:59.598506+0000","last_peered":"2024-05-08T05:42:59.598506+0000","last_clean":"2024-05-08T05:42:59.598506+0000","last_became_active":"2024-05-08T05:42:59.591050+0000","last_became_peered":"2024-05-08T05:42:59.591050+0000","last_unstale":"2024-05-08T05:42:59.598506+0000","last_undegraded":"2024-05-08T05:42:59.598506+0000","last_fullsized":"2024-05-08T05:42:59.598506+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:46.820598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.577480+0000","last_change":"2024-05-08T05:42:58.574070+0000","last_active":"2024-05-08T05:42:58.577480+0000","last_peered":"2024-05-08T05:42:58.577480+0000","last_clean":"2024-05-08T05:42:58.577480+0000","last_became_active":"2024-05-08T05:42:58.573769+0000","last_became_peered":"2024-05-08T05:42:58.573769+0000","last_unstale":"2024-05-08T05:42:58.577480+0000","last_undegraded":"2024-05-08T05:42:58.577480+0000","last_fullsized":"2024-05-08T05:42:58.577480+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:55:31.695068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.558457+0000","last_change":"2024-05-08T05:42:58.558457+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.558457+0000","last_undegraded":"2024-05-08T05:42:58.558457+0000","last_fullsized":"2024-05-08T05:42:58.558457+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.580407+0000","last_change":"2024-05-08T05:42:58.567761+0000","last_active":"2024-05-08T05:42:59.580407+0000","last_peered":"2024-05-08T05:42:59.580407+0000","last_clean":"2024-05-08T05:42:59.580407+0000","last_became_active":"2024-05-08T05:42:58.567490+0000","last_became_peered":"2024-05-08T05:42:58.567490+0000","last_unstale":"2024-05-08T05:42:59.580407+0000","last_undegraded":"2024-05-08T05:42:59.580407+0000","last_fullsized":"2024-05-08T05:42:59.580407+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:23:17.808492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.595122+0000","last_change":"2024-05-08T05:42:59.585553+0000","last_active":"2024-05-08T05:42:59.595122+0000","last_peered":"2024-05-08T05:42:59.595122+0000","last_clean":"2024-05-08T05:42:59.595122+0000","last_became_active":"2024-05-08T05:42:59.584903+0000","last_became_peered":"2024-05-08T05:42:59.584903+0000","last_unstale":"2024-05-08T05:42:59.595122+0000","last_undegraded":"2024-05-08T05:42:59.595122+0000","last_fullsized":"2024-05-08T05:42:59.595122+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:54:26.126199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.575724+0000","last_change":"2024-05-08T05:42:58.574888+0000","last_active":"2024-05-08T05:42:58.575724+0000","last_peered":"2024-05-08T05:42:58.575724+0000","last_clean":"2024-05-08T05:42:58.575724+0000","last_became_active":"2024-05-08T05:42:58.574585+0000","last_became_peered":"2024-05-08T05:42:58.574585+0000","last_unstale":"2024-05-08T05:42:58.575724+0000","last_undegraded":"2024-05-08T05:42:58.575724+0000","last_fullsized":"2024-05-08T05:42:58.575724+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:44:20.899641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.561535+0000","last_change":"2024-05-08T05:42:58.561535+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.561535+0000","last_undegraded":"2024-05-08T05:42:58.561535+0000","last_fullsized":"2024-05-08T05:42:58.561535+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"46'3","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.601464+0000","last_change":"2024-05-08T05:42:58.572885+0000","last_active":"2024-05-08T05:42:59.601464+0000","last_peered":"2024-05-08T05:42:59.601464+0000","last_clean":"2024-05-08T05:42:59.601464+0000","last_became_active":"2024-05-08T05:42:58.571751+0000","last_became_peered":"2024-05-08T05:42:58.571751+0000","last_unstale":"2024-05-08T05:42:59.601464+0000","last_undegraded":"2024-05-08T05:42:59.601464+0000","last_fullsized":"2024-05-08T05:42:59.601464+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:15.376533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.558448+0000","last_change":"2024-05-08T05:42:58.558448+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.558448+0000","last_undegraded":"2024-05-08T05:42:58.558448+0000","last_fullsized":"2024-05-08T05:42:58.558448+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.581919+0000","last_change":"2024-05-08T05:42:58.578675+0000","last_active":"2024-05-08T05:42:58.581919+0000","last_peered":"2024-05-08T05:42:58.581919+0000","last_clean":"2024-05-08T05:42:58.581919+0000","last_became_active":"2024-05-08T05:42:58.578130+0000","last_became_peered":"2024-05-08T05:42:58.578130+0000","last_unstale":"2024-05-08T05:42:58.581919+0000","last_undegraded":"2024-05-08T05:42:58.581919+0000","last_fullsized":"2024-05-08T05:42:58.581919+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:15.376533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.591669+0000","last_change":"2024-05-08T05:42:59.583647+0000","last_active":"2024-05-08T05:42:59.591669+0000","last_peered":"2024-05-08T05:42:59.591669+0000","last_clean":"2024-05-08T05:42:59.591669+0000","last_became_active":"2024-05-08T05:42:59.583302+0000","last_became_peered":"2024-05-08T05:42:59.583302+0000","last_unstale":"2024-05-08T05:42:59.591669+0000","last_undegraded":"2024-05-08T05:42:59.591669+0000","last_fullsized":"2024-05-08T05:42:59.591669+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:23:18.816427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.577616+0000","last_change":"2024-05-08T05:42:58.576622+0000","last_active":"2024-05-08T05:42:58.577616+0000","last_peered":"2024-05-08T05:42:58.577616+0000","last_clean":"2024-05-08T05:42:58.577616+0000","last_became_active":"2024-05-08T05:42:58.575446+0000","last_became_peered":"2024-05-08T05:42:58.575446+0000","last_unstale":"2024-05-08T05:42:58.577616+0000","last_undegraded":"2024-05-08T05:42:58.577616+0000","last_fullsized":"2024-05-08T05:42:58.577616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:46.384674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.564425+0000","last_change":"2024-05-08T05:42:58.564425+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.564425+0000","last_undegraded":"2024-05-08T05:42:58.564425+0000","last_fullsized":"2024-05-08T05:42:58.564425+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.575907+0000","last_change":"2024-05-08T05:42:58.574983+0000","last_active":"2024-05-08T05:42:59.575907+0000","last_peered":"2024-05-08T05:42:59.575907+0000","last_clean":"2024-05-08T05:42:59.575907+0000","last_became_active":"2024-05-08T05:42:58.574678+0000","last_became_peered":"2024-05-08T05:42:58.574678+0000","last_unstale":"2024-05-08T05:42:59.575907+0000","last_undegraded":"2024-05-08T05:42:59.575907+0000","last_fullsized":"2024-05-08T05:42:59.575907+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:28.357714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.565577+0000","last_change":"2024-05-08T05:42:58.565577+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.565577+0000","last_undegraded":"2024-05-08T05:42:58.565577+0000","last_fullsized":"2024-05-08T05:42:58.565577+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.577776+0000","last_change":"2024-05-08T05:42:58.572724+0000","last_active":"2024-05-08T05:42:59.577776+0000","last_peered":"2024-05-08T05:42:59.577776+0000","last_clean":"2024-05-08T05:42:59.577776+0000","last_became_active":"2024-05-08T05:42:58.572471+0000","last_became_peered":"2024-05-08T05:42:58.572471+0000","last_unstale":"2024-05-08T05:42:59.577776+0000","last_undegraded":"2024-05-08T05:42:59.577776+0000","last_fullsized":"2024-05-08T05:42:59.577776+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:28.063244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.567393+0000","last_change":"2024-05-08T05:42:58.567393+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.567393+0000","last_undegraded":"2024-05-08T05:42:58.567393+0000","last_fullsized":"2024-05-08T05:42:58.567393+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.581728+0000","last_change":"2024-05-08T05:42:58.577909+0000","last_active":"2024-05-08T05:42:58.581728+0000","last_peered":"2024-05-08T05:42:58.581728+0000","last_clean":"2024-05-08T05:42:58.581728+0000","last_became_active":"2024-05-08T05:42:58.577585+0000","last_became_peered":"2024-05-08T05:42:58.577585+0000","last_unstale":"2024-05-08T05:42:58.581728+0000","last_undegraded":"2024-05-08T05:42:58.581728+0000","last_fullsized":"2024-05-08T05:42:58.581728+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:28.063244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.564368+0000","last_change":"2024-05-08T05:42:58.564368+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.564368+0000","last_undegraded":"2024-05-08T05:42:58.564368+0000","last_fullsized":"2024-05-08T05:42:58.564368+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.580755+0000","last_change":"2024-05-08T05:42:58.579816+0000","last_active":"2024-05-08T05:42:58.580755+0000","last_peered":"2024-05-08T05:42:58.580755+0000","last_clean":"2024-05-08T05:42:58.580755+0000","last_became_active":"2024-05-08T05:42:58.579567+0000","last_became_peered":"2024-05-08T05:42:58.579567+0000","last_unstale":"2024-05-08T05:42:58.580755+0000","last_undegraded":"2024-05-08T05:42:58.580755+0000","last_fullsized":"2024-05-08T05:42:58.580755+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:32.556335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.561728+0000","last_change":"2024-05-08T05:42:58.561728+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.561728+0000","last_undegraded":"2024-05-08T05:42:58.561728+0000","last_fullsized":"2024-05-08T05:42:58.561728+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.571814+0000","last_change":"2024-05-08T05:42:58.568207+0000","last_active":"2024-05-08T05:42:58.571814+0000","last_peered":"2024-05-08T05:42:58.571814+0000","last_clean":"2024-05-08T05:42:58.571814+0000","last_became_active":"2024-05-08T05:42:58.567932+0000","last_became_peered":"2024-05-08T05:42:58.567932+0000","last_unstale":"2024-05-08T05:42:58.571814+0000","last_undegraded":"2024-05-08T05:42:58.571814+0000","last_fullsized":"2024-05-08T05:42:58.571814+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:44.502777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.558840+0000","last_change":"2024-05-08T05:42:58.558840+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.558840+0000","last_undegraded":"2024-05-08T05:42:58.558840+0000","last_fullsized":"2024-05-08T05:42:58.558840+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.599929+0000","last_change":"2024-05-08T05:42:58.573572+0000","last_active":"2024-05-08T05:42:59.599929+0000","last_peered":"2024-05-08T05:42:59.599929+0000","last_clean":"2024-05-08T05:42:59.599929+0000","last_became_active":"2024-05-08T05:42:58.573275+0000","last_became_peered":"2024-05-08T05:42:58.573275+0000","last_unstale":"2024-05-08T05:42:59.599929+0000","last_undegraded":"2024-05-08T05:42:59.599929+0000","last_fullsized":"2024-05-08T05:42:59.599929+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:23:53.968037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.562629+0000","last_change":"2024-05-08T05:42:58.562629+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.562629+0000","last_undegraded":"2024-05-08T05:42:58.562629+0000","last_fullsized":"2024-05-08T05:42:58.562629+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.576012+0000","last_change":"2024-05-08T05:42:58.579945+0000","last_active":"2024-05-08T05:42:59.576012+0000","last_peered":"2024-05-08T05:42:59.576012+0000","last_clean":"2024-05-08T05:42:59.576012+0000","last_became_active":"2024-05-08T05:42:58.579657+0000","last_became_peered":"2024-05-08T05:42:58.579657+0000","last_unstale":"2024-05-08T05:42:59.576012+0000","last_undegraded":"2024-05-08T05:42:59.576012+0000","last_fullsized":"2024-05-08T05:42:59.576012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:46.384674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.595611+0000","last_change":"2024-05-08T05:42:59.585604+0000","last_active":"2024-05-08T05:42:59.595611+0000","last_peered":"2024-05-08T05:42:59.595611+0000","last_clean":"2024-05-08T05:42:59.595611+0000","last_became_active":"2024-05-08T05:42:59.585027+0000","last_became_peered":"2024-05-08T05:42:59.585027+0000","last_unstale":"2024-05-08T05:42:59.595611+0000","last_undegraded":"2024-05-08T05:42:59.595611+0000","last_fullsized":"2024-05-08T05:42:59.595611+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:46.820598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.577900+0000","last_change":"2024-05-08T05:42:58.574363+0000","last_active":"2024-05-08T05:42:58.577900+0000","last_peered":"2024-05-08T05:42:58.577900+0000","last_clean":"2024-05-08T05:42:58.577900+0000","last_became_active":"2024-05-08T05:42:58.574101+0000","last_became_peered":"2024-05-08T05:42:58.574101+0000","last_unstale":"2024-05-08T05:42:58.577900+0000","last_undegraded":"2024-05-08T05:42:58.577900+0000","last_fullsized":"2024-05-08T05:42:58.577900+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:39.779366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.590917+0000","last_change":"2024-05-08T05:42:59.587267+0000","last_active":"2024-05-08T05:42:59.590917+0000","last_peered":"2024-05-08T05:42:59.590917+0000","last_clean":"2024-05-08T05:42:59.590917+0000","last_became_active":"2024-05-08T05:42:59.586824+0000","last_became_peered":"2024-05-08T05:42:59.586824+0000","last_unstale":"2024-05-08T05:42:59.590917+0000","last_undegraded":"2024-05-08T05:42:59.590917+0000","last_fullsized":"2024-05-08T05:42:59.590917+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:28:42.000118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.574671+0000","last_change":"2024-05-08T05:42:58.572523+0000","last_active":"2024-05-08T05:42:58.574671+0000","last_peered":"2024-05-08T05:42:58.574671+0000","last_clean":"2024-05-08T05:42:58.574671+0000","last_became_active":"2024-05-08T05:42:58.572193+0000","last_became_peered":"2024-05-08T05:42:58.572193+0000","last_unstale":"2024-05-08T05:42:58.574671+0000","last_undegraded":"2024-05-08T05:42:58.574671+0000","last_fullsized":"2024-05-08T05:42:58.574671+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:04.051382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.563623+0000","last_change":"2024-05-08T05:42:58.563623+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.563623+0000","last_undegraded":"2024-05-08T05:42:58.563623+0000","last_fullsized":"2024-05-08T05:42:58.563623+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.582841+0000","last_change":"2024-05-08T05:42:58.582112+0000","last_active":"2024-05-08T05:42:58.582841+0000","last_peered":"2024-05-08T05:42:58.582841+0000","last_clean":"2024-05-08T05:42:58.582841+0000","last_became_active":"2024-05-08T05:42:58.581779+0000","last_became_peered":"2024-05-08T05:42:58.581779+0000","last_unstale":"2024-05-08T05:42:58.582841+0000","last_undegraded":"2024-05-08T05:42:58.582841+0000","last_fullsized":"2024-05-08T05:42:58.582841+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:23:53.968037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566556+0000","last_change":"2024-05-08T05:42:58.566556+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566556+0000","last_undegraded":"2024-05-08T05:42:58.566556+0000","last_fullsized":"2024-05-08T05:42:58.566556+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.571905+0000","last_change":"2024-05-08T05:42:58.568551+0000","last_active":"2024-05-08T05:42:58.571905+0000","last_peered":"2024-05-08T05:42:58.571905+0000","last_clean":"2024-05-08T05:42:58.571905+0000","last_became_active":"2024-05-08T05:42:58.568282+0000","last_became_peered":"2024-05-08T05:42:58.568282+0000","last_unstale":"2024-05-08T05:42:58.571905+0000","last_undegraded":"2024-05-08T05:42:58.571905+0000","last_fullsized":"2024-05-08T05:42:58.571905+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:35:01.162157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.595061+0000","last_change":"2024-05-08T05:42:59.586138+0000","last_active":"2024-05-08T05:42:59.595061+0000","last_peered":"2024-05-08T05:42:59.595061+0000","last_clean":"2024-05-08T05:42:59.595061+0000","last_became_active":"2024-05-08T05:42:59.585739+0000","last_became_peered":"2024-05-08T05:42:59.585739+0000","last_unstale":"2024-05-08T05:42:59.595061+0000","last_undegraded":"2024-05-08T05:42:59.595061+0000","last_fullsized":"2024-05-08T05:42:59.595061+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:34:10.738939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.605908+0000","last_change":"2024-05-08T05:42:58.580617+0000","last_active":"2024-05-08T05:42:59.605908+0000","last_peered":"2024-05-08T05:42:59.605908+0000","last_clean":"2024-05-08T05:42:59.605908+0000","last_became_active":"2024-05-08T05:42:58.580371+0000","last_became_peered":"2024-05-08T05:42:58.580371+0000","last_unstale":"2024-05-08T05:42:59.605908+0000","last_undegraded":"2024-05-08T05:42:59.605908+0000","last_fullsized":"2024-05-08T05:42:59.605908+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:32.556335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.562862+0000","last_change":"2024-05-08T05:42:58.562862+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.562862+0000","last_undegraded":"2024-05-08T05:42:58.562862+0000","last_fullsized":"2024-05-08T05:42:58.562862+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.578285+0000","last_change":"2024-05-08T05:42:58.570248+0000","last_active":"2024-05-08T05:42:59.578285+0000","last_peered":"2024-05-08T05:42:59.578285+0000","last_clean":"2024-05-08T05:42:59.578285+0000","last_became_active":"2024-05-08T05:42:58.569966+0000","last_became_peered":"2024-05-08T05:42:58.569966+0000","last_unstale":"2024-05-08T05:42:59.578285+0000","last_undegraded":"2024-05-08T05:42:59.578285+0000","last_fullsized":"2024-05-08T05:42:59.578285+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:25.616180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.567229+0000","last_change":"2024-05-08T05:42:58.567229+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.567229+0000","last_undegraded":"2024-05-08T05:42:58.567229+0000","last_fullsized":"2024-05-08T05:42:58.567229+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.580617+0000","last_change":"2024-05-08T05:42:58.577804+0000","last_active":"2024-05-08T05:42:58.580617+0000","last_peered":"2024-05-08T05:42:58.580617+0000","last_clean":"2024-05-08T05:42:58.580617+0000","last_became_active":"2024-05-08T05:42:58.577442+0000","last_became_peered":"2024-05-08T05:42:58.577442+0000","last_unstale":"2024-05-08T05:42:58.580617+0000","last_undegraded":"2024-05-08T05:42:58.580617+0000","last_fullsized":"2024-05-08T05:42:58.580617+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:04.957071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.563979+0000","last_change":"2024-05-08T05:42:58.563979+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.563979+0000","last_undegraded":"2024-05-08T05:42:58.563979+0000","last_fullsized":"2024-05-08T05:42:58.563979+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.573849+0000","last_change":"2024-05-08T05:42:58.571644+0000","last_active":"2024-05-08T05:42:58.573849+0000","last_peered":"2024-05-08T05:42:58.573849+0000","last_clean":"2024-05-08T05:42:58.573849+0000","last_became_active":"2024-05-08T05:42:58.571348+0000","last_became_peered":"2024-05-08T05:42:58.571348+0000","last_unstale":"2024-05-08T05:42:58.573849+0000","last_undegraded":"2024-05-08T05:42:58.573849+0000","last_fullsized":"2024-05-08T05:42:58.573849+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:24:55.348642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.591367+0000","last_change":"2024-05-08T05:42:59.584308+0000","last_active":"2024-05-08T05:42:59.591367+0000","last_peered":"2024-05-08T05:42:59.591367+0000","last_clean":"2024-05-08T05:42:59.591367+0000","last_became_active":"2024-05-08T05:42:59.584026+0000","last_became_peered":"2024-05-08T05:42:59.584026+0000","last_unstale":"2024-05-08T05:42:59.591367+0000","last_undegraded":"2024-05-08T05:42:59.591367+0000","last_fullsized":"2024-05-08T05:42:59.591367+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:09.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.582620+0000","last_change":"2024-05-08T05:42:58.580460+0000","last_active":"2024-05-08T05:42:58.582620+0000","last_peered":"2024-05-08T05:42:58.582620+0000","last_clean":"2024-05-08T05:42:58.582620+0000","last_became_active":"2024-05-08T05:42:58.580085+0000","last_became_peered":"2024-05-08T05:42:58.580085+0000","last_unstale":"2024-05-08T05:42:58.582620+0000","last_undegraded":"2024-05-08T05:42:58.582620+0000","last_fullsized":"2024-05-08T05:42:58.582620+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:25.616180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.564637+0000","last_change":"2024-05-08T05:42:58.564637+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.564637+0000","last_undegraded":"2024-05-08T05:42:58.564637+0000","last_fullsized":"2024-05-08T05:42:58.564637+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.579427+0000","last_change":"2024-05-08T05:42:58.578326+0000","last_active":"2024-05-08T05:42:58.579427+0000","last_peered":"2024-05-08T05:42:58.579427+0000","last_clean":"2024-05-08T05:42:58.579427+0000","last_became_active":"2024-05-08T05:42:58.577971+0000","last_became_peered":"2024-05-08T05:42:58.577971+0000","last_unstale":"2024-05-08T05:42:58.579427+0000","last_undegraded":"2024-05-08T05:42:58.579427+0000","last_fullsized":"2024-05-08T05:42:58.579427+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:06:22.535707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566087+0000","last_change":"2024-05-08T05:42:58.566087+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566087+0000","last_undegraded":"2024-05-08T05:42:58.566087+0000","last_fullsized":"2024-05-08T05:42:58.566087+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.572277+0000","last_change":"2024-05-08T05:42:58.578763+0000","last_active":"2024-05-08T05:42:59.572277+0000","last_peered":"2024-05-08T05:42:59.572277+0000","last_clean":"2024-05-08T05:42:59.572277+0000","last_became_active":"2024-05-08T05:42:58.578430+0000","last_became_peered":"2024-05-08T05:42:58.578430+0000","last_unstale":"2024-05-08T05:42:59.572277+0000","last_undegraded":"2024-05-08T05:42:59.572277+0000","last_fullsized":"2024-05-08T05:42:59.572277+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:25.616180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.596172+0000","last_change":"2024-05-08T05:42:59.592843+0000","last_active":"2024-05-08T05:42:59.596172+0000","last_peered":"2024-05-08T05:42:59.596172+0000","last_clean":"2024-05-08T05:42:59.596172+0000","last_became_active":"2024-05-08T05:42:59.592317+0000","last_became_peered":"2024-05-08T05:42:59.592317+0000","last_unstale":"2024-05-08T05:42:59.596172+0000","last_undegraded":"2024-05-08T05:42:59.596172+0000","last_fullsized":"2024-05-08T05:42:59.596172+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:17.523257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.577643+0000","last_change":"2024-05-08T05:42:58.576321+0000","last_active":"2024-05-08T05:42:58.577643+0000","last_peered":"2024-05-08T05:42:58.577643+0000","last_clean":"2024-05-08T05:42:58.577643+0000","last_became_active":"2024-05-08T05:42:58.576057+0000","last_became_peered":"2024-05-08T05:42:58.576057+0000","last_unstale":"2024-05-08T05:42:58.577643+0000","last_undegraded":"2024-05-08T05:42:58.577643+0000","last_fullsized":"2024-05-08T05:42:58.577643+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:28.357714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":157,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.578796+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:42:59.578796+0000","last_peered":"2024-05-08T05:42:59.578796+0000","last_clean":"2024-05-08T05:42:59.578796+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:42:59.578796+0000","last_undegraded":"2024-05-08T05:42:59.578796+0000","last_fullsized":"2024-05-08T05:42:59.578796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":742,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542168,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1122304,"data_stored":818492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542169,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":235291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":228124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":516096,"data_stored":221457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":528384,"data_stored":228124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31700,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":528384,"data_stored":228124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":528384,"data_stored":228124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1110016,"data_stored":811825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:43:03.627 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:43:03.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.766+0000 7fd2d4a09640 1 Processor -- start 2024-05-08T05:43:03.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.766+0000 7fd2d4a09640 1 -- start start 2024-05-08T05:43:03.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.767+0000 7fd2d4a09640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d0070d30 0x7fd2d00d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.767+0000 7fd2d4a09640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2d0163920 0x7fd2d00703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.767+0000 7fd2d4a09640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd2d0071110 0x7fd2d006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.767+0000 7fd2d4a09640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fd2d00d10d0 con 0x7fd2d0163920 2024-05-08T05:43:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.767+0000 7fd2d4a09640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd2d0070b80 con 0x7fd2d0071110 2024-05-08T05:43:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.767+0000 7fd2d4a09640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d00d1c30 con 0x7fd2d0070d30 2024-05-08T05:43:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.767+0000 7fd2c7fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2d0163920 0x7fd2d00703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.767+0000 7fd2c7fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd2d0163920 0x7fd2d00703a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47362/0 (socket says 172.21.15.45:47362) 2024-05-08T05:43:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.767+0000 7fd2c7fff640 1 -- 172.21.15.45:0/1959268566 learned_addr learned my addr 172.21.15.45:0/1959268566 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.767+0000 7fd2c7fff640 1 -- 172.21.15.45:0/1959268566 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d0070d30 msgr2=0x7fd2d00d20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.767+0000 7fd2c7fff640 1 --2- 172.21.15.45:0/1959268566 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d0070d30 0x7fd2d00d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.767+0000 7fd2c7fff640 1 -- 172.21.15.45:0/1959268566 >> v2:172.21.15.63:3300/0 conn(0x7fd2d0071110 msgr2=0x7fd2d006d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.767+0000 7fd2c7fff640 1 --2- 172.21.15.45:0/1959268566 >> v2:172.21.15.63:3300/0 conn(0x7fd2d0071110 0x7fd2d006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.767+0000 7fd2c7fff640 1 -- 172.21.15.45:0/1959268566 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d0163d80 con 0x7fd2d0163920 2024-05-08T05:43:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.768+0000 7fd2c7fff640 1 --2- 172.21.15.45:0/1959268566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d0163920 0x7fd2d00703a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0073310 tx=0x7fd2b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ad2c17299f31b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.768+0000 7fd2cdd9b640 1 -- 172.21.15.45:0/1959268566 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8037070 con 0x7fd2d0163920 2024-05-08T05:43:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.768+0000 7fd2cdd9b640 1 -- 172.21.15.45:0/1959268566 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2b8032440 con 0x7fd2d0163920 2024-05-08T05:43:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.768+0000 7fd2cdd9b640 1 -- 172.21.15.45:0/1959268566 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8032810 con 0x7fd2d0163920 2024-05-08T05:43:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.769+0000 7fd2d4a09640 1 -- 172.21.15.45:0/1959268566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d0163920 msgr2=0x7fd2d00703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.769+0000 7fd2d4a09640 1 --2- 172.21.15.45:0/1959268566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d0163920 0x7fd2d00703a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0073310 tx=0x7fd2b802a1e0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.769+0000 7fd2d4a09640 1 -- 172.21.15.45:0/1959268566 shutdown_connections 2024-05-08T05:43:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.769+0000 7fd2d4a09640 1 --2- 172.21.15.45:0/1959268566 >> v2:172.21.15.63:3300/0 conn(0x7fd2d0071110 0x7fd2d006d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.769+0000 7fd2d4a09640 1 --2- 172.21.15.45:0/1959268566 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d0163920 0x7fd2d00703a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.769+0000 7fd2d4a09640 1 --2- 172.21.15.45:0/1959268566 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d0070d30 0x7fd2d00d20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.769+0000 7fd2d4a09640 1 -- 172.21.15.45:0/1959268566 >> 172.21.15.45:0/1959268566 conn(0x7fd2d00d2e90 msgr2=0x7fd2d00d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.769+0000 7fd2d4a09640 1 -- 172.21.15.45:0/1959268566 shutdown_connections 2024-05-08T05:43:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.769+0000 7fd2d4a09640 1 -- 172.21.15.45:0/1959268566 wait complete. 2024-05-08T05:43:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.770+0000 7fd2d4a09640 1 Processor -- start 2024-05-08T05:43:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.770+0000 7fd2d4a09640 1 -- start start 2024-05-08T05:43:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.771+0000 7fd2d4a09640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d0070d30 0x7fd2d01591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.771+0000 7fd2d4a09640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d0071110 0x7fd2d00b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.771+0000 7fd2d4a09640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d0163920 0x7fd2d00b7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.771+0000 7fd2d4a09640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d00d15e0 con 0x7fd2d0071110 2024-05-08T05:43:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.771+0000 7fd2d4a09640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d00708e0 con 0x7fd2d0163920 2024-05-08T05:43:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.771+0000 7fd2d4a09640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d00d18c0 con 0x7fd2d0070d30 2024-05-08T05:43:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.771+0000 7fd2c7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d0071110 0x7fd2d00b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.771+0000 7fd2c7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d0071110 0x7fd2d00b5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47368/0 (socket says 172.21.15.45:47368) 2024-05-08T05:43:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.771+0000 7fd2c7fff640 1 -- 172.21.15.45:0/3318718641 learned_addr learned my addr 172.21.15.45:0/3318718641 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.771+0000 7fd2ced9d640 1 --2- 172.21.15.45:0/3318718641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d0163920 0x7fd2d00b7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.771+0000 7fd2ce59c640 1 --2- 172.21.15.45:0/3318718641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d0070d30 0x7fd2d01591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.771+0000 7fd2c7fff640 1 -- 172.21.15.45:0/3318718641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d0070d30 msgr2=0x7fd2d01591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.771+0000 7fd2c7fff640 1 --2- 172.21.15.45:0/3318718641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d0070d30 0x7fd2d01591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.771+0000 7fd2c7fff640 1 -- 172.21.15.45:0/3318718641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d0163920 msgr2=0x7fd2d00b7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.771+0000 7fd2c7fff640 1 --2- 172.21.15.45:0/3318718641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d0163920 0x7fd2d00b7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.771+0000 7fd2c7fff640 1 -- 172.21.15.45:0/3318718641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d00b6220 con 0x7fd2d0071110 2024-05-08T05:43:03.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.771+0000 7fd2ce59c640 1 --2- 172.21.15.45:0/3318718641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d0070d30 0x7fd2d01591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:03.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.772+0000 7fd2c7fff640 1 --2- 172.21.15.45:0/3318718641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d0071110 0x7fd2d00b5ce0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8000c00 tx=0x7fd2b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:03.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.772+0000 7fd2c6ffd640 1 -- 172.21.15.45:0/3318718641 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8041070 con 0x7fd2d0071110 2024-05-08T05:43:03.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.772+0000 7fd2c6ffd640 1 -- 172.21.15.45:0/3318718641 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2b803c870 con 0x7fd2d0071110 2024-05-08T05:43:03.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.772+0000 7fd2d4a09640 1 -- 172.21.15.45:0/3318718641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d00b64c0 con 0x7fd2d0071110 2024-05-08T05:43:03.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.772+0000 7fd2c6ffd640 1 -- 172.21.15.45:0/3318718641 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8037040 con 0x7fd2d0071110 2024-05-08T05:43:03.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.772+0000 7fd2d4a09640 1 -- 172.21.15.45:0/3318718641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d00b6a80 con 0x7fd2d0071110 2024-05-08T05:43:03.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.773+0000 7fd2d4a09640 1 -- 172.21.15.45:0/3318718641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d00693f0 con 0x7fd2d0071110 2024-05-08T05:43:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.773+0000 7fd2c6ffd640 1 -- 172.21.15.45:0/3318718641 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd2b8044070 con 0x7fd2d0071110 2024-05-08T05:43:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.774+0000 7fd2c6ffd640 1 --2- 172.21.15.45:0/3318718641 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd2b00762d0 0x7fd2b0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.774+0000 7fd2c6ffd640 1 -- 172.21.15.45:0/3318718641 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7fd2b80b8310 con 0x7fd2d0071110 2024-05-08T05:43:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.774+0000 7fd2ce59c640 1 --2- 172.21.15.45:0/3318718641 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd2b00762d0 0x7fd2b0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.775+0000 7fd2ce59c640 1 --2- 172.21.15.45:0/3318718641 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd2b00762d0 0x7fd2b0078790 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0163340 tx=0x7fd2c0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.778+0000 7fd2c6ffd640 1 -- 172.21.15.45:0/3318718641 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd2b8081b70 con 0x7fd2d0071110 2024-05-08T05:43:03.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.971+0000 7fd2d4a09640 1 -- 172.21.15.45:0/3318718641 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd2d0159730 con 0x7fd2b00762d0 2024-05-08T05:43:03.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.978+0000 7fd2c6ffd640 1 -- 172.21.15.45:0/3318718641 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248351 (secure 0 0 0) 0x7fd2d0159730 con 0x7fd2b00762d0 2024-05-08T05:43:03.984 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:03.985 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:43:03.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.980+0000 7fd2d4a09640 1 -- 172.21.15.45:0/3318718641 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd2b00762d0 msgr2=0x7fd2b0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:03.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.980+0000 7fd2d4a09640 1 --2- 172.21.15.45:0/3318718641 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd2b00762d0 0x7fd2b0078790 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0163340 tx=0x7fd2c0008040 comp rx=0 tx=0).stop 2024-05-08T05:43:03.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.980+0000 7fd2d4a09640 1 -- 172.21.15.45:0/3318718641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d0071110 msgr2=0x7fd2d00b5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:03.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.980+0000 7fd2d4a09640 1 --2- 172.21.15.45:0/3318718641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d0071110 0x7fd2d00b5ce0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8000c00 tx=0x7fd2b80046c0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.980+0000 7fd2d4a09640 1 -- 172.21.15.45:0/3318718641 shutdown_connections 2024-05-08T05:43:03.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.980+0000 7fd2d4a09640 1 --2- 172.21.15.45:0/3318718641 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd2b00762d0 0x7fd2b0078790 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.980+0000 7fd2d4a09640 1 --2- 172.21.15.45:0/3318718641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d0163920 0x7fd2d00b7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.980+0000 7fd2d4a09640 1 --2- 172.21.15.45:0/3318718641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d0071110 0x7fd2d00b5ce0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.980+0000 7fd2d4a09640 1 --2- 172.21.15.45:0/3318718641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2d0070d30 0x7fd2d01591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.980+0000 7fd2d4a09640 1 -- 172.21.15.45:0/3318718641 >> 172.21.15.45:0/3318718641 conn(0x7fd2d00d2e90 msgr2=0x7fd2d006ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.980+0000 7fd2d4a09640 1 -- 172.21.15.45:0/3318718641 shutdown_connections 2024-05-08T05:43:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:03.980+0000 7fd2d4a09640 1 -- 172.21.15.45:0/3318718641 wait complete. 2024-05-08T05:43:03.993 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-05-08T05:43:02.853692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":116,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84,"ondisk_log_size":84,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238120,"kb_used_data":5276,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736600,"statfs":{"total":773094113280,"available":772850278400,"internally_reserved":0,"allocated":5402624,"data_stored":2999561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":742,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.300007"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.567012+0000","last_change":"2024-05-08T05:42:58.567012+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.567012+0000","last_undegraded":"2024-05-08T05:42:58.567012+0000","last_fullsized":"2024-05-08T05:42:58.567012+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.590291+0000","last_change":"2024-05-08T05:42:59.586491+0000","last_active":"2024-05-08T05:42:59.590291+0000","last_peered":"2024-05-08T05:42:59.590291+0000","last_clean":"2024-05-08T05:42:59.590291+0000","last_became_active":"2024-05-08T05:42:59.586233+0000","last_became_peered":"2024-05-08T05:42:59.586233+0000","last_unstale":"2024-05-08T05:42:59.590291+0000","last_undegraded":"2024-05-08T05:42:59.590291+0000","last_fullsized":"2024-05-08T05:42:59.590291+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:32.677283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.565275+0000","last_change":"2024-05-08T05:42:58.565275+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.565275+0000","last_undegraded":"2024-05-08T05:42:58.565275+0000","last_fullsized":"2024-05-08T05:42:58.565275+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.595331+0000","last_change":"2024-05-08T05:42:59.589298+0000","last_active":"2024-05-08T05:42:59.595331+0000","last_peered":"2024-05-08T05:42:59.595331+0000","last_clean":"2024-05-08T05:42:59.595331+0000","last_became_active":"2024-05-08T05:42:59.588989+0000","last_became_peered":"2024-05-08T05:42:59.588989+0000","last_unstale":"2024-05-08T05:42:59.595331+0000","last_undegraded":"2024-05-08T05:42:59.595331+0000","last_fullsized":"2024-05-08T05:42:59.595331+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:54:26.126199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.559296+0000","last_change":"2024-05-08T05:42:58.559296+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.559296+0000","last_undegraded":"2024-05-08T05:42:58.559296+0000","last_fullsized":"2024-05-08T05:42:58.559296+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.569796+0000","last_change":"2024-05-08T05:42:58.569796+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.569796+0000","last_undegraded":"2024-05-08T05:42:58.569796+0000","last_fullsized":"2024-05-08T05:42:58.569796+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.565860+0000","last_change":"2024-05-08T05:42:58.565860+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.565860+0000","last_undegraded":"2024-05-08T05:42:58.565860+0000","last_fullsized":"2024-05-08T05:42:58.565860+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.564839+0000","last_change":"2024-05-08T05:42:58.564839+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.564839+0000","last_undegraded":"2024-05-08T05:42:58.564839+0000","last_fullsized":"2024-05-08T05:42:58.564839+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566530+0000","last_change":"2024-05-08T05:42:58.566530+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566530+0000","last_undegraded":"2024-05-08T05:42:58.566530+0000","last_fullsized":"2024-05-08T05:42:58.566530+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566857+0000","last_change":"2024-05-08T05:42:58.566857+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566857+0000","last_undegraded":"2024-05-08T05:42:58.566857+0000","last_fullsized":"2024-05-08T05:42:58.566857+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.560543+0000","last_change":"2024-05-08T05:42:58.560543+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.560543+0000","last_undegraded":"2024-05-08T05:42:58.560543+0000","last_fullsized":"2024-05-08T05:42:58.560543+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.567647+0000","last_change":"2024-05-08T05:42:58.567647+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.567647+0000","last_undegraded":"2024-05-08T05:42:58.567647+0000","last_fullsized":"2024-05-08T05:42:58.567647+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.563061+0000","last_change":"2024-05-08T05:42:58.563061+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.563061+0000","last_undegraded":"2024-05-08T05:42:58.563061+0000","last_fullsized":"2024-05-08T05:42:58.563061+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.562034+0000","last_change":"2024-05-08T05:42:58.562034+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.562034+0000","last_undegraded":"2024-05-08T05:42:58.562034+0000","last_fullsized":"2024-05-08T05:42:58.562034+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.598708+0000","last_change":"2024-05-08T05:42:59.593026+0000","last_active":"2024-05-08T05:42:59.598708+0000","last_peered":"2024-05-08T05:42:59.598708+0000","last_clean":"2024-05-08T05:42:59.598708+0000","last_became_active":"2024-05-08T05:42:59.592706+0000","last_became_peered":"2024-05-08T05:42:59.592706+0000","last_unstale":"2024-05-08T05:42:59.598708+0000","last_undegraded":"2024-05-08T05:42:59.598708+0000","last_fullsized":"2024-05-08T05:42:59.598708+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:34.093891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.561473+0000","last_change":"2024-05-08T05:42:58.561473+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.561473+0000","last_undegraded":"2024-05-08T05:42:58.561473+0000","last_fullsized":"2024-05-08T05:42:58.561473+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566253+0000","last_change":"2024-05-08T05:42:58.566253+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566253+0000","last_undegraded":"2024-05-08T05:42:58.566253+0000","last_fullsized":"2024-05-08T05:42:58.566253+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.598545+0000","last_change":"2024-05-08T05:42:59.592278+0000","last_active":"2024-05-08T05:42:59.598545+0000","last_peered":"2024-05-08T05:42:59.598545+0000","last_clean":"2024-05-08T05:42:59.598545+0000","last_became_active":"2024-05-08T05:42:59.591985+0000","last_became_peered":"2024-05-08T05:42:59.591985+0000","last_unstale":"2024-05-08T05:42:59.598545+0000","last_undegraded":"2024-05-08T05:42:59.598545+0000","last_fullsized":"2024-05-08T05:42:59.598545+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:42:12.700183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.564859+0000","last_change":"2024-05-08T05:42:58.564859+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.564859+0000","last_undegraded":"2024-05-08T05:42:58.564859+0000","last_fullsized":"2024-05-08T05:42:58.564859+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.590796+0000","last_change":"2024-05-08T05:42:59.584649+0000","last_active":"2024-05-08T05:42:59.590796+0000","last_peered":"2024-05-08T05:42:59.590796+0000","last_clean":"2024-05-08T05:42:59.590796+0000","last_became_active":"2024-05-08T05:42:59.584233+0000","last_became_peered":"2024-05-08T05:42:59.584233+0000","last_unstale":"2024-05-08T05:42:59.590796+0000","last_undegraded":"2024-05-08T05:42:59.590796+0000","last_fullsized":"2024-05-08T05:42:59.590796+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:49.405283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.595031+0000","last_change":"2024-05-08T05:42:59.591062+0000","last_active":"2024-05-08T05:42:59.595031+0000","last_peered":"2024-05-08T05:42:59.595031+0000","last_clean":"2024-05-08T05:42:59.595031+0000","last_became_active":"2024-05-08T05:42:59.590680+0000","last_became_peered":"2024-05-08T05:42:59.590680+0000","last_unstale":"2024-05-08T05:42:59.595031+0000","last_undegraded":"2024-05-08T05:42:59.595031+0000","last_fullsized":"2024-05-08T05:42:59.595031+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:05:27.975200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.563363+0000","last_change":"2024-05-08T05:42:58.563363+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.563363+0000","last_undegraded":"2024-05-08T05:42:58.563363+0000","last_fullsized":"2024-05-08T05:42:58.563363+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.560421+0000","last_change":"2024-05-08T05:42:58.560421+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.560421+0000","last_undegraded":"2024-05-08T05:42:58.560421+0000","last_fullsized":"2024-05-08T05:42:58.560421+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.561384+0000","last_change":"2024-05-08T05:42:58.561384+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.561384+0000","last_undegraded":"2024-05-08T05:42:58.561384+0000","last_fullsized":"2024-05-08T05:42:58.561384+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566986+0000","last_change":"2024-05-08T05:42:58.566986+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566986+0000","last_undegraded":"2024-05-08T05:42:58.566986+0000","last_fullsized":"2024-05-08T05:42:58.566986+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566924+0000","last_change":"2024-05-08T05:42:58.566924+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566924+0000","last_undegraded":"2024-05-08T05:42:58.566924+0000","last_fullsized":"2024-05-08T05:42:58.566924+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.565197+0000","last_change":"2024-05-08T05:42:58.565197+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.565197+0000","last_undegraded":"2024-05-08T05:42:58.565197+0000","last_fullsized":"2024-05-08T05:42:58.565197+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.589440+0000","last_change":"2024-05-08T05:42:59.582258+0000","last_active":"2024-05-08T05:42:59.589440+0000","last_peered":"2024-05-08T05:42:59.589440+0000","last_clean":"2024-05-08T05:42:59.589440+0000","last_became_active":"2024-05-08T05:42:59.581973+0000","last_became_peered":"2024-05-08T05:42:59.581973+0000","last_unstale":"2024-05-08T05:42:59.589440+0000","last_undegraded":"2024-05-08T05:42:59.589440+0000","last_fullsized":"2024-05-08T05:42:59.589440+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:45:38.272590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.562110+0000","last_change":"2024-05-08T05:42:58.562110+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.562110+0000","last_undegraded":"2024-05-08T05:42:58.562110+0000","last_fullsized":"2024-05-08T05:42:58.562110+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.590188+0000","last_change":"2024-05-08T05:42:59.583375+0000","last_active":"2024-05-08T05:42:59.590188+0000","last_peered":"2024-05-08T05:42:59.590188+0000","last_clean":"2024-05-08T05:42:59.590188+0000","last_became_active":"2024-05-08T05:42:59.583026+0000","last_became_peered":"2024-05-08T05:42:59.583026+0000","last_unstale":"2024-05-08T05:42:59.590188+0000","last_undegraded":"2024-05-08T05:42:59.590188+0000","last_fullsized":"2024-05-08T05:42:59.590188+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:09.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.562320+0000","last_change":"2024-05-08T05:42:58.562320+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.562320+0000","last_undegraded":"2024-05-08T05:42:58.562320+0000","last_fullsized":"2024-05-08T05:42:58.562320+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.595084+0000","last_change":"2024-05-08T05:42:59.586488+0000","last_active":"2024-05-08T05:42:59.595084+0000","last_peered":"2024-05-08T05:42:59.595084+0000","last_clean":"2024-05-08T05:42:59.595084+0000","last_became_active":"2024-05-08T05:42:59.585806+0000","last_became_peered":"2024-05-08T05:42:59.585806+0000","last_unstale":"2024-05-08T05:42:59.595084+0000","last_undegraded":"2024-05-08T05:42:59.595084+0000","last_fullsized":"2024-05-08T05:42:59.595084+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:42:12.700183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.598596+0000","last_change":"2024-05-08T05:42:59.593948+0000","last_active":"2024-05-08T05:42:59.598596+0000","last_peered":"2024-05-08T05:42:59.598596+0000","last_clean":"2024-05-08T05:42:59.598596+0000","last_became_active":"2024-05-08T05:42:59.593663+0000","last_became_peered":"2024-05-08T05:42:59.593663+0000","last_unstale":"2024-05-08T05:42:59.598596+0000","last_undegraded":"2024-05-08T05:42:59.598596+0000","last_fullsized":"2024-05-08T05:42:59.598596+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:47:11.811597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"46'4","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.601783+0000","last_change":"2024-05-08T05:42:58.568449+0000","last_active":"2024-05-08T05:42:59.601783+0000","last_peered":"2024-05-08T05:42:59.601783+0000","last_clean":"2024-05-08T05:42:59.601783+0000","last_became_active":"2024-05-08T05:42:58.567378+0000","last_became_peered":"2024-05-08T05:42:58.567378+0000","last_unstale":"2024-05-08T05:42:59.601783+0000","last_undegraded":"2024-05-08T05:42:59.601783+0000","last_fullsized":"2024-05-08T05:42:59.601783+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:04.957071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.591538+0000","last_change":"2024-05-08T05:42:59.583673+0000","last_active":"2024-05-08T05:42:59.591538+0000","last_peered":"2024-05-08T05:42:59.591538+0000","last_clean":"2024-05-08T05:42:59.591538+0000","last_became_active":"2024-05-08T05:42:59.583430+0000","last_became_peered":"2024-05-08T05:42:59.583430+0000","last_unstale":"2024-05-08T05:42:59.591538+0000","last_undegraded":"2024-05-08T05:42:59.591538+0000","last_fullsized":"2024-05-08T05:42:59.591538+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:32.677283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.583411+0000","last_change":"2024-05-08T05:42:58.582169+0000","last_active":"2024-05-08T05:42:58.583411+0000","last_peered":"2024-05-08T05:42:58.583411+0000","last_clean":"2024-05-08T05:42:58.583411+0000","last_became_active":"2024-05-08T05:42:58.581851+0000","last_became_peered":"2024-05-08T05:42:58.581851+0000","last_unstale":"2024-05-08T05:42:58.583411+0000","last_undegraded":"2024-05-08T05:42:58.583411+0000","last_fullsized":"2024-05-08T05:42:58.583411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:28.357714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.590873+0000","last_change":"2024-05-08T05:42:59.585789+0000","last_active":"2024-05-08T05:42:59.590873+0000","last_peered":"2024-05-08T05:42:59.590873+0000","last_clean":"2024-05-08T05:42:59.590873+0000","last_became_active":"2024-05-08T05:42:59.585486+0000","last_became_peered":"2024-05-08T05:42:59.585486+0000","last_unstale":"2024-05-08T05:42:59.590873+0000","last_undegraded":"2024-05-08T05:42:59.590873+0000","last_fullsized":"2024-05-08T05:42:59.590873+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:13.547702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.579505+0000","last_change":"2024-05-08T05:42:58.577001+0000","last_active":"2024-05-08T05:42:58.579505+0000","last_peered":"2024-05-08T05:42:58.579505+0000","last_clean":"2024-05-08T05:42:58.579505+0000","last_became_active":"2024-05-08T05:42:58.575665+0000","last_became_peered":"2024-05-08T05:42:58.575665+0000","last_unstale":"2024-05-08T05:42:58.579505+0000","last_undegraded":"2024-05-08T05:42:58.579505+0000","last_fullsized":"2024-05-08T05:42:58.579505+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:55:31.695068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.567678+0000","last_change":"2024-05-08T05:42:58.567678+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.567678+0000","last_undegraded":"2024-05-08T05:42:58.567678+0000","last_fullsized":"2024-05-08T05:42:58.567678+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.602422+0000","last_change":"2024-05-08T05:42:58.573567+0000","last_active":"2024-05-08T05:42:59.602422+0000","last_peered":"2024-05-08T05:42:59.602422+0000","last_clean":"2024-05-08T05:42:59.602422+0000","last_became_active":"2024-05-08T05:42:58.573220+0000","last_became_peered":"2024-05-08T05:42:58.573220+0000","last_unstale":"2024-05-08T05:42:59.602422+0000","last_undegraded":"2024-05-08T05:42:59.602422+0000","last_fullsized":"2024-05-08T05:42:59.602422+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:15.376533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.558663+0000","last_change":"2024-05-08T05:42:58.558663+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.558663+0000","last_undegraded":"2024-05-08T05:42:58.558663+0000","last_fullsized":"2024-05-08T05:42:58.558663+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.576887+0000","last_change":"2024-05-08T05:42:58.572908+0000","last_active":"2024-05-08T05:42:58.576887+0000","last_peered":"2024-05-08T05:42:58.576887+0000","last_clean":"2024-05-08T05:42:58.576887+0000","last_became_active":"2024-05-08T05:42:58.572613+0000","last_became_peered":"2024-05-08T05:42:58.572613+0000","last_unstale":"2024-05-08T05:42:58.576887+0000","last_undegraded":"2024-05-08T05:42:58.576887+0000","last_fullsized":"2024-05-08T05:42:58.576887+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:39.779366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.598156+0000","last_change":"2024-05-08T05:42:59.589512+0000","last_active":"2024-05-08T05:42:59.598156+0000","last_peered":"2024-05-08T05:42:59.598156+0000","last_clean":"2024-05-08T05:42:59.598156+0000","last_became_active":"2024-05-08T05:42:59.588973+0000","last_became_peered":"2024-05-08T05:42:59.588973+0000","last_unstale":"2024-05-08T05:42:59.598156+0000","last_undegraded":"2024-05-08T05:42:59.598156+0000","last_fullsized":"2024-05-08T05:42:59.598156+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:34:10.738939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.571718+0000","last_change":"2024-05-08T05:42:58.569466+0000","last_active":"2024-05-08T05:42:58.571718+0000","last_peered":"2024-05-08T05:42:58.571718+0000","last_clean":"2024-05-08T05:42:58.571718+0000","last_became_active":"2024-05-08T05:42:58.569048+0000","last_became_peered":"2024-05-08T05:42:58.569048+0000","last_unstale":"2024-05-08T05:42:58.571718+0000","last_undegraded":"2024-05-08T05:42:58.571718+0000","last_fullsized":"2024-05-08T05:42:58.571718+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:30:44.118693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.565142+0000","last_change":"2024-05-08T05:42:58.565142+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.565142+0000","last_undegraded":"2024-05-08T05:42:58.565142+0000","last_fullsized":"2024-05-08T05:42:58.565142+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.575648+0000","last_change":"2024-05-08T05:42:58.568749+0000","last_active":"2024-05-08T05:42:59.575648+0000","last_peered":"2024-05-08T05:42:59.575648+0000","last_clean":"2024-05-08T05:42:59.575648+0000","last_became_active":"2024-05-08T05:42:58.568473+0000","last_became_peered":"2024-05-08T05:42:58.568473+0000","last_unstale":"2024-05-08T05:42:59.575648+0000","last_undegraded":"2024-05-08T05:42:59.575648+0000","last_fullsized":"2024-05-08T05:42:59.575648+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:55:31.695068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.598506+0000","last_change":"2024-05-08T05:42:59.591341+0000","last_active":"2024-05-08T05:42:59.598506+0000","last_peered":"2024-05-08T05:42:59.598506+0000","last_clean":"2024-05-08T05:42:59.598506+0000","last_became_active":"2024-05-08T05:42:59.591050+0000","last_became_peered":"2024-05-08T05:42:59.591050+0000","last_unstale":"2024-05-08T05:42:59.598506+0000","last_undegraded":"2024-05-08T05:42:59.598506+0000","last_fullsized":"2024-05-08T05:42:59.598506+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:46.820598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.577480+0000","last_change":"2024-05-08T05:42:58.574070+0000","last_active":"2024-05-08T05:42:58.577480+0000","last_peered":"2024-05-08T05:42:58.577480+0000","last_clean":"2024-05-08T05:42:58.577480+0000","last_became_active":"2024-05-08T05:42:58.573769+0000","last_became_peered":"2024-05-08T05:42:58.573769+0000","last_unstale":"2024-05-08T05:42:58.577480+0000","last_undegraded":"2024-05-08T05:42:58.577480+0000","last_fullsized":"2024-05-08T05:42:58.577480+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:55:31.695068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.558457+0000","last_change":"2024-05-08T05:42:58.558457+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.558457+0000","last_undegraded":"2024-05-08T05:42:58.558457+0000","last_fullsized":"2024-05-08T05:42:58.558457+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.580407+0000","last_change":"2024-05-08T05:42:58.567761+0000","last_active":"2024-05-08T05:42:59.580407+0000","last_peered":"2024-05-08T05:42:59.580407+0000","last_clean":"2024-05-08T05:42:59.580407+0000","last_became_active":"2024-05-08T05:42:58.567490+0000","last_became_peered":"2024-05-08T05:42:58.567490+0000","last_unstale":"2024-05-08T05:42:59.580407+0000","last_undegraded":"2024-05-08T05:42:59.580407+0000","last_fullsized":"2024-05-08T05:42:59.580407+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:23:17.808492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.595122+0000","last_change":"2024-05-08T05:42:59.585553+0000","last_active":"2024-05-08T05:42:59.595122+0000","last_peered":"2024-05-08T05:42:59.595122+0000","last_clean":"2024-05-08T05:42:59.595122+0000","last_became_active":"2024-05-08T05:42:59.584903+0000","last_became_peered":"2024-05-08T05:42:59.584903+0000","last_unstale":"2024-05-08T05:42:59.595122+0000","last_undegraded":"2024-05-08T05:42:59.595122+0000","last_fullsized":"2024-05-08T05:42:59.595122+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:54:26.126199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.575724+0000","last_change":"2024-05-08T05:42:58.574888+0000","last_active":"2024-05-08T05:42:58.575724+0000","last_peered":"2024-05-08T05:42:58.575724+0000","last_clean":"2024-05-08T05:42:58.575724+0000","last_became_active":"2024-05-08T05:42:58.574585+0000","last_became_peered":"2024-05-08T05:42:58.574585+0000","last_unstale":"2024-05-08T05:42:58.575724+0000","last_undegraded":"2024-05-08T05:42:58.575724+0000","last_fullsized":"2024-05-08T05:42:58.575724+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:44:20.899641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.561535+0000","last_change":"2024-05-08T05:42:58.561535+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.561535+0000","last_undegraded":"2024-05-08T05:42:58.561535+0000","last_fullsized":"2024-05-08T05:42:58.561535+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"46'3","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.601464+0000","last_change":"2024-05-08T05:42:58.572885+0000","last_active":"2024-05-08T05:42:59.601464+0000","last_peered":"2024-05-08T05:42:59.601464+0000","last_clean":"2024-05-08T05:42:59.601464+0000","last_became_active":"2024-05-08T05:42:58.571751+0000","last_became_peered":"2024-05-08T05:42:58.571751+0000","last_unstale":"2024-05-08T05:42:59.601464+0000","last_undegraded":"2024-05-08T05:42:59.601464+0000","last_fullsized":"2024-05-08T05:42:59.601464+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:15.376533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.558448+0000","last_change":"2024-05-08T05:42:58.558448+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.558448+0000","last_undegraded":"2024-05-08T05:42:58.558448+0000","last_fullsized":"2024-05-08T05:42:58.558448+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.581919+0000","last_change":"2024-05-08T05:42:58.578675+0000","last_active":"2024-05-08T05:42:58.581919+0000","last_peered":"2024-05-08T05:42:58.581919+0000","last_clean":"2024-05-08T05:42:58.581919+0000","last_became_active":"2024-05-08T05:42:58.578130+0000","last_became_peered":"2024-05-08T05:42:58.578130+0000","last_unstale":"2024-05-08T05:42:58.581919+0000","last_undegraded":"2024-05-08T05:42:58.581919+0000","last_fullsized":"2024-05-08T05:42:58.581919+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:15.376533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.591669+0000","last_change":"2024-05-08T05:42:59.583647+0000","last_active":"2024-05-08T05:42:59.591669+0000","last_peered":"2024-05-08T05:42:59.591669+0000","last_clean":"2024-05-08T05:42:59.591669+0000","last_became_active":"2024-05-08T05:42:59.583302+0000","last_became_peered":"2024-05-08T05:42:59.583302+0000","last_unstale":"2024-05-08T05:42:59.591669+0000","last_undegraded":"2024-05-08T05:42:59.591669+0000","last_fullsized":"2024-05-08T05:42:59.591669+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:23:18.816427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.577616+0000","last_change":"2024-05-08T05:42:58.576622+0000","last_active":"2024-05-08T05:42:58.577616+0000","last_peered":"2024-05-08T05:42:58.577616+0000","last_clean":"2024-05-08T05:42:58.577616+0000","last_became_active":"2024-05-08T05:42:58.575446+0000","last_became_peered":"2024-05-08T05:42:58.575446+0000","last_unstale":"2024-05-08T05:42:58.577616+0000","last_undegraded":"2024-05-08T05:42:58.577616+0000","last_fullsized":"2024-05-08T05:42:58.577616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:46.384674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.564425+0000","last_change":"2024-05-08T05:42:58.564425+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.564425+0000","last_undegraded":"2024-05-08T05:42:58.564425+0000","last_fullsized":"2024-05-08T05:42:58.564425+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.575907+0000","last_change":"2024-05-08T05:42:58.574983+0000","last_active":"2024-05-08T05:42:59.575907+0000","last_peered":"2024-05-08T05:42:59.575907+0000","last_clean":"2024-05-08T05:42:59.575907+0000","last_became_active":"2024-05-08T05:42:58.574678+0000","last_became_peered":"2024-05-08T05:42:58.574678+0000","last_unstale":"2024-05-08T05:42:59.575907+0000","last_undegraded":"2024-05-08T05:42:59.575907+0000","last_fullsized":"2024-05-08T05:42:59.575907+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:28.357714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.565577+0000","last_change":"2024-05-08T05:42:58.565577+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.565577+0000","last_undegraded":"2024-05-08T05:42:58.565577+0000","last_fullsized":"2024-05-08T05:42:58.565577+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.577776+0000","last_change":"2024-05-08T05:42:58.572724+0000","last_active":"2024-05-08T05:42:59.577776+0000","last_peered":"2024-05-08T05:42:59.577776+0000","last_clean":"2024-05-08T05:42:59.577776+0000","last_became_active":"2024-05-08T05:42:58.572471+0000","last_became_peered":"2024-05-08T05:42:58.572471+0000","last_unstale":"2024-05-08T05:42:59.577776+0000","last_undegraded":"2024-05-08T05:42:59.577776+0000","last_fullsized":"2024-05-08T05:42:59.577776+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:28.063244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.567393+0000","last_change":"2024-05-08T05:42:58.567393+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.567393+0000","last_undegraded":"2024-05-08T05:42:58.567393+0000","last_fullsized":"2024-05-08T05:42:58.567393+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.581728+0000","last_change":"2024-05-08T05:42:58.577909+0000","last_active":"2024-05-08T05:42:58.581728+0000","last_peered":"2024-05-08T05:42:58.581728+0000","last_clean":"2024-05-08T05:42:58.581728+0000","last_became_active":"2024-05-08T05:42:58.577585+0000","last_became_peered":"2024-05-08T05:42:58.577585+0000","last_unstale":"2024-05-08T05:42:58.581728+0000","last_undegraded":"2024-05-08T05:42:58.581728+0000","last_fullsized":"2024-05-08T05:42:58.581728+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:28.063244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.564368+0000","last_change":"2024-05-08T05:42:58.564368+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.564368+0000","last_undegraded":"2024-05-08T05:42:58.564368+0000","last_fullsized":"2024-05-08T05:42:58.564368+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.580755+0000","last_change":"2024-05-08T05:42:58.579816+0000","last_active":"2024-05-08T05:42:58.580755+0000","last_peered":"2024-05-08T05:42:58.580755+0000","last_clean":"2024-05-08T05:42:58.580755+0000","last_became_active":"2024-05-08T05:42:58.579567+0000","last_became_peered":"2024-05-08T05:42:58.579567+0000","last_unstale":"2024-05-08T05:42:58.580755+0000","last_undegraded":"2024-05-08T05:42:58.580755+0000","last_fullsized":"2024-05-08T05:42:58.580755+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:32.556335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.561728+0000","last_change":"2024-05-08T05:42:58.561728+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.561728+0000","last_undegraded":"2024-05-08T05:42:58.561728+0000","last_fullsized":"2024-05-08T05:42:58.561728+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.571814+0000","last_change":"2024-05-08T05:42:58.568207+0000","last_active":"2024-05-08T05:42:58.571814+0000","last_peered":"2024-05-08T05:42:58.571814+0000","last_clean":"2024-05-08T05:42:58.571814+0000","last_became_active":"2024-05-08T05:42:58.567932+0000","last_became_peered":"2024-05-08T05:42:58.567932+0000","last_unstale":"2024-05-08T05:42:58.571814+0000","last_undegraded":"2024-05-08T05:42:58.571814+0000","last_fullsized":"2024-05-08T05:42:58.571814+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:44.502777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.558840+0000","last_change":"2024-05-08T05:42:58.558840+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.558840+0000","last_undegraded":"2024-05-08T05:42:58.558840+0000","last_fullsized":"2024-05-08T05:42:58.558840+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.599929+0000","last_change":"2024-05-08T05:42:58.573572+0000","last_active":"2024-05-08T05:42:59.599929+0000","last_peered":"2024-05-08T05:42:59.599929+0000","last_clean":"2024-05-08T05:42:59.599929+0000","last_became_active":"2024-05-08T05:42:58.573275+0000","last_became_peered":"2024-05-08T05:42:58.573275+0000","last_unstale":"2024-05-08T05:42:59.599929+0000","last_undegraded":"2024-05-08T05:42:59.599929+0000","last_fullsized":"2024-05-08T05:42:59.599929+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:23:53.968037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.562629+0000","last_change":"2024-05-08T05:42:58.562629+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.562629+0000","last_undegraded":"2024-05-08T05:42:58.562629+0000","last_fullsized":"2024-05-08T05:42:58.562629+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.576012+0000","last_change":"2024-05-08T05:42:58.579945+0000","last_active":"2024-05-08T05:42:59.576012+0000","last_peered":"2024-05-08T05:42:59.576012+0000","last_clean":"2024-05-08T05:42:59.576012+0000","last_became_active":"2024-05-08T05:42:58.579657+0000","last_became_peered":"2024-05-08T05:42:58.579657+0000","last_unstale":"2024-05-08T05:42:59.576012+0000","last_undegraded":"2024-05-08T05:42:59.576012+0000","last_fullsized":"2024-05-08T05:42:59.576012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:46.384674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.595611+0000","last_change":"2024-05-08T05:42:59.585604+0000","last_active":"2024-05-08T05:42:59.595611+0000","last_peered":"2024-05-08T05:42:59.595611+0000","last_clean":"2024-05-08T05:42:59.595611+0000","last_became_active":"2024-05-08T05:42:59.585027+0000","last_became_peered":"2024-05-08T05:42:59.585027+0000","last_unstale":"2024-05-08T05:42:59.595611+0000","last_undegraded":"2024-05-08T05:42:59.595611+0000","last_fullsized":"2024-05-08T05:42:59.595611+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:46.820598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.577900+0000","last_change":"2024-05-08T05:42:58.574363+0000","last_active":"2024-05-08T05:42:58.577900+0000","last_peered":"2024-05-08T05:42:58.577900+0000","last_clean":"2024-05-08T05:42:58.577900+0000","last_became_active":"2024-05-08T05:42:58.574101+0000","last_became_peered":"2024-05-08T05:42:58.574101+0000","last_unstale":"2024-05-08T05:42:58.577900+0000","last_undegraded":"2024-05-08T05:42:58.577900+0000","last_fullsized":"2024-05-08T05:42:58.577900+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:39.779366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.590917+0000","last_change":"2024-05-08T05:42:59.587267+0000","last_active":"2024-05-08T05:42:59.590917+0000","last_peered":"2024-05-08T05:42:59.590917+0000","last_clean":"2024-05-08T05:42:59.590917+0000","last_became_active":"2024-05-08T05:42:59.586824+0000","last_became_peered":"2024-05-08T05:42:59.586824+0000","last_unstale":"2024-05-08T05:42:59.590917+0000","last_undegraded":"2024-05-08T05:42:59.590917+0000","last_fullsized":"2024-05-08T05:42:59.590917+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:28:42.000118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.574671+0000","last_change":"2024-05-08T05:42:58.572523+0000","last_active":"2024-05-08T05:42:58.574671+0000","last_peered":"2024-05-08T05:42:58.574671+0000","last_clean":"2024-05-08T05:42:58.574671+0000","last_became_active":"2024-05-08T05:42:58.572193+0000","last_became_peered":"2024-05-08T05:42:58.572193+0000","last_unstale":"2024-05-08T05:42:58.574671+0000","last_undegraded":"2024-05-08T05:42:58.574671+0000","last_fullsized":"2024-05-08T05:42:58.574671+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:04.051382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.563623+0000","last_change":"2024-05-08T05:42:58.563623+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.563623+0000","last_undegraded":"2024-05-08T05:42:58.563623+0000","last_fullsized":"2024-05-08T05:42:58.563623+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.582841+0000","last_change":"2024-05-08T05:42:58.582112+0000","last_active":"2024-05-08T05:42:58.582841+0000","last_peered":"2024-05-08T05:42:58.582841+0000","last_clean":"2024-05-08T05:42:58.582841+0000","last_became_active":"2024-05-08T05:42:58.581779+0000","last_became_peered":"2024-05-08T05:42:58.581779+0000","last_unstale":"2024-05-08T05:42:58.582841+0000","last_undegraded":"2024-05-08T05:42:58.582841+0000","last_fullsized":"2024-05-08T05:42:58.582841+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:23:53.968037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566556+0000","last_change":"2024-05-08T05:42:58.566556+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566556+0000","last_undegraded":"2024-05-08T05:42:58.566556+0000","last_fullsized":"2024-05-08T05:42:58.566556+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.571905+0000","last_change":"2024-05-08T05:42:58.568551+0000","last_active":"2024-05-08T05:42:58.571905+0000","last_peered":"2024-05-08T05:42:58.571905+0000","last_clean":"2024-05-08T05:42:58.571905+0000","last_became_active":"2024-05-08T05:42:58.568282+0000","last_became_peered":"2024-05-08T05:42:58.568282+0000","last_unstale":"2024-05-08T05:42:58.571905+0000","last_undegraded":"2024-05-08T05:42:58.571905+0000","last_fullsized":"2024-05-08T05:42:58.571905+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:35:01.162157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.595061+0000","last_change":"2024-05-08T05:42:59.586138+0000","last_active":"2024-05-08T05:42:59.595061+0000","last_peered":"2024-05-08T05:42:59.595061+0000","last_clean":"2024-05-08T05:42:59.595061+0000","last_became_active":"2024-05-08T05:42:59.585739+0000","last_became_peered":"2024-05-08T05:42:59.585739+0000","last_unstale":"2024-05-08T05:42:59.595061+0000","last_undegraded":"2024-05-08T05:42:59.595061+0000","last_fullsized":"2024-05-08T05:42:59.595061+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:34:10.738939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.605908+0000","last_change":"2024-05-08T05:42:58.580617+0000","last_active":"2024-05-08T05:42:59.605908+0000","last_peered":"2024-05-08T05:42:59.605908+0000","last_clean":"2024-05-08T05:42:59.605908+0000","last_became_active":"2024-05-08T05:42:58.580371+0000","last_became_peered":"2024-05-08T05:42:58.580371+0000","last_unstale":"2024-05-08T05:42:59.605908+0000","last_undegraded":"2024-05-08T05:42:59.605908+0000","last_fullsized":"2024-05-08T05:42:59.605908+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:32.556335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.562862+0000","last_change":"2024-05-08T05:42:58.562862+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.562862+0000","last_undegraded":"2024-05-08T05:42:58.562862+0000","last_fullsized":"2024-05-08T05:42:58.562862+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.578285+0000","last_change":"2024-05-08T05:42:58.570248+0000","last_active":"2024-05-08T05:42:59.578285+0000","last_peered":"2024-05-08T05:42:59.578285+0000","last_clean":"2024-05-08T05:42:59.578285+0000","last_became_active":"2024-05-08T05:42:58.569966+0000","last_became_peered":"2024-05-08T05:42:58.569966+0000","last_unstale":"2024-05-08T05:42:59.578285+0000","last_undegraded":"2024-05-08T05:42:59.578285+0000","last_fullsized":"2024-05-08T05:42:59.578285+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:25.616180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.567229+0000","last_change":"2024-05-08T05:42:58.567229+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.567229+0000","last_undegraded":"2024-05-08T05:42:58.567229+0000","last_fullsized":"2024-05-08T05:42:58.567229+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.580617+0000","last_change":"2024-05-08T05:42:58.577804+0000","last_active":"2024-05-08T05:42:58.580617+0000","last_peered":"2024-05-08T05:42:58.580617+0000","last_clean":"2024-05-08T05:42:58.580617+0000","last_became_active":"2024-05-08T05:42:58.577442+0000","last_became_peered":"2024-05-08T05:42:58.577442+0000","last_unstale":"2024-05-08T05:42:58.580617+0000","last_undegraded":"2024-05-08T05:42:58.580617+0000","last_fullsized":"2024-05-08T05:42:58.580617+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:04.957071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.563979+0000","last_change":"2024-05-08T05:42:58.563979+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.563979+0000","last_undegraded":"2024-05-08T05:42:58.563979+0000","last_fullsized":"2024-05-08T05:42:58.563979+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.573849+0000","last_change":"2024-05-08T05:42:58.571644+0000","last_active":"2024-05-08T05:42:58.573849+0000","last_peered":"2024-05-08T05:42:58.573849+0000","last_clean":"2024-05-08T05:42:58.573849+0000","last_became_active":"2024-05-08T05:42:58.571348+0000","last_became_peered":"2024-05-08T05:42:58.571348+0000","last_unstale":"2024-05-08T05:42:58.573849+0000","last_undegraded":"2024-05-08T05:42:58.573849+0000","last_fullsized":"2024-05-08T05:42:58.573849+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:24:55.348642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.591367+0000","last_change":"2024-05-08T05:42:59.584308+0000","last_active":"2024-05-08T05:42:59.591367+0000","last_peered":"2024-05-08T05:42:59.591367+0000","last_clean":"2024-05-08T05:42:59.591367+0000","last_became_active":"2024-05-08T05:42:59.584026+0000","last_became_peered":"2024-05-08T05:42:59.584026+0000","last_unstale":"2024-05-08T05:42:59.591367+0000","last_undegraded":"2024-05-08T05:42:59.591367+0000","last_fullsized":"2024-05-08T05:42:59.591367+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:09.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.582620+0000","last_change":"2024-05-08T05:42:58.580460+0000","last_active":"2024-05-08T05:42:58.582620+0000","last_peered":"2024-05-08T05:42:58.582620+0000","last_clean":"2024-05-08T05:42:58.582620+0000","last_became_active":"2024-05-08T05:42:58.580085+0000","last_became_peered":"2024-05-08T05:42:58.580085+0000","last_unstale":"2024-05-08T05:42:58.582620+0000","last_undegraded":"2024-05-08T05:42:58.582620+0000","last_fullsized":"2024-05-08T05:42:58.582620+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:25.616180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.564637+0000","last_change":"2024-05-08T05:42:58.564637+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.564637+0000","last_undegraded":"2024-05-08T05:42:58.564637+0000","last_fullsized":"2024-05-08T05:42:58.564637+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.579427+0000","last_change":"2024-05-08T05:42:58.578326+0000","last_active":"2024-05-08T05:42:58.579427+0000","last_peered":"2024-05-08T05:42:58.579427+0000","last_clean":"2024-05-08T05:42:58.579427+0000","last_became_active":"2024-05-08T05:42:58.577971+0000","last_became_peered":"2024-05-08T05:42:58.577971+0000","last_unstale":"2024-05-08T05:42:58.579427+0000","last_undegraded":"2024-05-08T05:42:58.579427+0000","last_fullsized":"2024-05-08T05:42:58.579427+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:06:22.535707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566087+0000","last_change":"2024-05-08T05:42:58.566087+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566087+0000","last_undegraded":"2024-05-08T05:42:58.566087+0000","last_fullsized":"2024-05-08T05:42:58.566087+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.572277+0000","last_change":"2024-05-08T05:42:58.578763+0000","last_active":"2024-05-08T05:42:59.572277+0000","last_peered":"2024-05-08T05:42:59.572277+0000","last_clean":"2024-05-08T05:42:59.572277+0000","last_became_active":"2024-05-08T05:42:58.578430+0000","last_became_peered":"2024-05-08T05:42:58.578430+0000","last_unstale":"2024-05-08T05:42:59.572277+0000","last_undegraded":"2024-05-08T05:42:59.572277+0000","last_fullsized":"2024-05-08T05:42:59.572277+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:25.616180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.596172+0000","last_change":"2024-05-08T05:42:59.592843+0000","last_active":"2024-05-08T05:42:59.596172+0000","last_peered":"2024-05-08T05:42:59.596172+0000","last_clean":"2024-05-08T05:42:59.596172+0000","last_became_active":"2024-05-08T05:42:59.592317+0000","last_became_peered":"2024-05-08T05:42:59.592317+0000","last_unstale":"2024-05-08T05:42:59.596172+0000","last_undegraded":"2024-05-08T05:42:59.596172+0000","last_fullsized":"2024-05-08T05:42:59.596172+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:17.523257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.577643+0000","last_change":"2024-05-08T05:42:58.576321+0000","last_active":"2024-05-08T05:42:58.577643+0000","last_peered":"2024-05-08T05:42:58.577643+0000","last_clean":"2024-05-08T05:42:58.577643+0000","last_became_active":"2024-05-08T05:42:58.576057+0000","last_became_peered":"2024-05-08T05:42:58.576057+0000","last_unstale":"2024-05-08T05:42:58.577643+0000","last_undegraded":"2024-05-08T05:42:58.577643+0000","last_fullsized":"2024-05-08T05:42:58.577643+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:28.357714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":157,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.578796+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:42:59.578796+0000","last_peered":"2024-05-08T05:42:59.578796+0000","last_clean":"2024-05-08T05:42:59.578796+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:42:59.578796+0000","last_undegraded":"2024-05-08T05:42:59.578796+0000","last_fullsized":"2024-05-08T05:42:59.578796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":742,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542168,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1122304,"data_stored":818492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542169,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":235291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":228124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":516096,"data_stored":221457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":528384,"data_stored":228124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31700,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":528384,"data_stored":228124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":528384,"data_stored":228124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1110016,"data_stored":811825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:43:03.998 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:43:04.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.136+0000 7f2b2ba62640 1 Processor -- start 2024-05-08T05:43:04.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.136+0000 7f2b2ba62640 1 -- start start 2024-05-08T05:43:04.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.137+0000 7f2b2ba62640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2b2406e0f0 0x7f2b2415c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.137+0000 7f2b2ba62640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2b24072840 0x7f2b240d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.137+0000 7f2b2ba62640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b240d14b0 con 0x7f2b2406e4d0 2024-05-08T05:43:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.137+0000 7f2b2ba62640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f2b2415c920 con 0x7f2b2406e0f0 2024-05-08T05:43:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.137+0000 7f2b2ba62640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2b2415cc00 con 0x7f2b24072840 2024-05-08T05:43:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.137+0000 7f2b297fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2b2406e0f0 0x7f2b2415c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.137+0000 7f2b297fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2b2406e0f0 0x7f2b2415c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47382/0 (socket says 172.21.15.45:47382) 2024-05-08T05:43:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.137+0000 7f2b297fe640 1 -- 172.21.15.45:0/1835232950 learned_addr learned my addr 172.21.15.45:0/1835232950 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.138+0000 7f2b28ffd640 1 --2- 172.21.15.45:0/1835232950 >> v2:172.21.15.63:3300/0 conn(0x7f2b24072840 0x7f2b240d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.138+0000 7f2b297fe640 1 -- 172.21.15.45:0/1835232950 >> v2:172.21.15.63:3300/0 conn(0x7f2b24072840 msgr2=0x7f2b240d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:04.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.138+0000 7f2b297fe640 1 --2- 172.21.15.45:0/1835232950 >> v2:172.21.15.63:3300/0 conn(0x7f2b24072840 0x7f2b240d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.138+0000 7f2b297fe640 1 -- 172.21.15.45:0/1835232950 >> v1:172.21.15.63:6789/0 conn(0x7f2b2406e4d0 legacy=0x7f2b240b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.138+0000 7f2b297fe640 1 -- 172.21.15.45:0/1835232950 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2415cee0 con 0x7f2b2406e0f0 2024-05-08T05:43:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.139+0000 7f2b297fe640 1 --2- 172.21.15.45:0/1835232950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2406e0f0 0x7f2b2415c340 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2b240727d0 tx=0x7f2b2002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645242e31804761 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.139+0000 7f2b1bfff640 1 -- 172.21.15.45:0/1835232950 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20037070 con 0x7f2b2406e0f0 2024-05-08T05:43:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.139+0000 7f2b1bfff640 1 -- 172.21.15.45:0/1835232950 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b20032440 con 0x7f2b2406e0f0 2024-05-08T05:43:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.139+0000 7f2b1bfff640 1 -- 172.21.15.45:0/1835232950 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20032810 con 0x7f2b2406e0f0 2024-05-08T05:43:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.139+0000 7f2b2ba62640 1 -- 172.21.15.45:0/1835232950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2406e0f0 msgr2=0x7f2b2415c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.139+0000 7f2b2ba62640 1 --2- 172.21.15.45:0/1835232950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2406e0f0 0x7f2b2415c340 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2b240727d0 tx=0x7f2b2002a270 comp rx=0 tx=0).stop 2024-05-08T05:43:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.139+0000 7f2b2ba62640 1 -- 172.21.15.45:0/1835232950 shutdown_connections 2024-05-08T05:43:04.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.139+0000 7f2b2ba62640 1 --2- 172.21.15.45:0/1835232950 >> v2:172.21.15.63:3300/0 conn(0x7f2b24072840 0x7f2b240d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.139+0000 7f2b2ba62640 1 --2- 172.21.15.45:0/1835232950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2406e0f0 0x7f2b2415c340 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.139+0000 7f2b2ba62640 1 -- 172.21.15.45:0/1835232950 >> 172.21.15.45:0/1835232950 conn(0x7f2b2406c3a0 msgr2=0x7f2b2406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.140+0000 7f2b2ba62640 1 -- 172.21.15.45:0/1835232950 shutdown_connections 2024-05-08T05:43:04.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.140+0000 7f2b2ba62640 1 -- 172.21.15.45:0/1835232950 wait complete. 2024-05-08T05:43:04.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.141+0000 7f2b2ba62640 1 Processor -- start 2024-05-08T05:43:04.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.141+0000 7f2b2ba62640 1 -- start start 2024-05-08T05:43:04.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b2ba62640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2406e0f0 0x7f2b2415eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:04.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b2ba62640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b2406e4d0 0x7f2b2415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:04.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b2ba62640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b24072840 0x7f2b2415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:04.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b2ba62640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2b240ae650 con 0x7f2b2406e0f0 2024-05-08T05:43:04.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b2ba62640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2406e960 con 0x7f2b24072840 2024-05-08T05:43:04.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b2ba62640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b240ae7d0 con 0x7f2b2406e4d0 2024-05-08T05:43:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b297fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2406e0f0 0x7f2b2415eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b297fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2406e0f0 0x7f2b2415eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47386/0 (socket says 172.21.15.45:47386) 2024-05-08T05:43:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b297fe640 1 -- 172.21.15.45:0/4100011847 learned_addr learned my addr 172.21.15.45:0/4100011847 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b28ffd640 1 --2- 172.21.15.45:0/4100011847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b2406e4d0 0x7f2b2415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b29fff640 1 --2- 172.21.15.45:0/4100011847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b24072840 0x7f2b2415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b297fe640 1 -- 172.21.15.45:0/4100011847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b2406e4d0 msgr2=0x7f2b2415d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b297fe640 1 --2- 172.21.15.45:0/4100011847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b2406e4d0 0x7f2b2415d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b297fe640 1 -- 172.21.15.45:0/4100011847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b24072840 msgr2=0x7f2b2415d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b297fe640 1 --2- 172.21.15.45:0/4100011847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b24072840 0x7f2b2415d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b297fe640 1 -- 172.21.15.45:0/4100011847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2415e010 con 0x7f2b2406e0f0 2024-05-08T05:43:04.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b29fff640 1 --2- 172.21.15.45:0/4100011847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b24072840 0x7f2b2415d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b297fe640 1 --2- 172.21.15.45:0/4100011847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2406e0f0 0x7f2b2415eae0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2b2002a2b0 tx=0x7f2b200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b19ffb640 1 -- 172.21.15.45:0/4100011847 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20041020 con 0x7f2b2406e0f0 2024-05-08T05:43:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b19ffb640 1 -- 172.21.15.45:0/4100011847 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b200306c0 con 0x7f2b2406e0f0 2024-05-08T05:43:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b2ba62640 1 -- 172.21.15.45:0/4100011847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b2415e2f0 con 0x7f2b2406e0f0 2024-05-08T05:43:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.142+0000 7f2b19ffb640 1 -- 172.21.15.45:0/4100011847 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20037040 con 0x7f2b2406e0f0 2024-05-08T05:43:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.143+0000 7f2b2ba62640 1 -- 172.21.15.45:0/4100011847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b241578d0 con 0x7f2b2406e0f0 2024-05-08T05:43:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.144+0000 7f2b2ba62640 1 -- 172.21.15.45:0/4100011847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b240693f0 con 0x7f2b2406e0f0 2024-05-08T05:43:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.144+0000 7f2b19ffb640 1 -- 172.21.15.45:0/4100011847 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2b20044020 con 0x7f2b2406e0f0 2024-05-08T05:43:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.145+0000 7f2b19ffb640 1 --2- 172.21.15.45:0/4100011847 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b080762d0 0x7f2b08078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:04.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.145+0000 7f2b19ffb640 1 -- 172.21.15.45:0/4100011847 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7f2b200b8010 con 0x7f2b2406e0f0 2024-05-08T05:43:04.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.145+0000 7f2b28ffd640 1 --2- 172.21.15.45:0/4100011847 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b080762d0 0x7f2b08078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:04.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.146+0000 7f2b28ffd640 1 --2- 172.21.15.45:0/4100011847 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b080762d0 0x7f2b08078790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2b14006fd0 tx=0x7f2b14008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:04.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.149+0000 7f2b19ffb640 1 -- 172.21.15.45:0/4100011847 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2b2004b030 con 0x7f2b2406e0f0 2024-05-08T05:43:04.166 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:43:04.170 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:43:04.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.343+0000 7f2b2ba62640 1 -- 172.21.15.45:0/4100011847 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2b2414b890 con 0x7f2b080762d0 2024-05-08T05:43:04.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.350+0000 7f2b19ffb640 1 -- 172.21.15.45:0/4100011847 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248351 (secure 0 0 0) 0x7f2b2414b890 con 0x7f2b080762d0 2024-05-08T05:43:04.353 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:43:04.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.352+0000 7f2b2ba62640 1 -- 172.21.15.45:0/4100011847 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b080762d0 msgr2=0x7f2b08078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:04.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.352+0000 7f2b2ba62640 1 --2- 172.21.15.45:0/4100011847 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b080762d0 0x7f2b08078790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2b14006fd0 tx=0x7f2b14008040 comp rx=0 tx=0).stop 2024-05-08T05:43:04.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.352+0000 7f2b2ba62640 1 -- 172.21.15.45:0/4100011847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2406e0f0 msgr2=0x7f2b2415eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:04.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.352+0000 7f2b2ba62640 1 --2- 172.21.15.45:0/4100011847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2406e0f0 0x7f2b2415eae0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2b2002a2b0 tx=0x7f2b200046c0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.352+0000 7f2b2ba62640 1 -- 172.21.15.45:0/4100011847 shutdown_connections 2024-05-08T05:43:04.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.352+0000 7f2b2ba62640 1 --2- 172.21.15.45:0/4100011847 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b080762d0 0x7f2b08078790 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.352+0000 7f2b2ba62640 1 --2- 172.21.15.45:0/4100011847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b2406e4d0 0x7f2b2415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.352+0000 7f2b2ba62640 1 --2- 172.21.15.45:0/4100011847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b24072840 0x7f2b2415d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.352+0000 7f2b2ba62640 1 --2- 172.21.15.45:0/4100011847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2406e0f0 0x7f2b2415eae0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.352+0000 7f2b2ba62640 1 -- 172.21.15.45:0/4100011847 >> 172.21.15.45:0/4100011847 conn(0x7f2b2406c3a0 msgr2=0x7f2b240720b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:04.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.352+0000 7f2b2ba62640 1 -- 172.21.15.45:0/4100011847 shutdown_connections 2024-05-08T05:43:04.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.352+0000 7f2b2ba62640 1 -- 172.21.15.45:0/4100011847 wait complete. 2024-05-08T05:43:04.365 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-05-08T05:43:02.853692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":116,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84,"ondisk_log_size":84,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238120,"kb_used_data":5276,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736600,"statfs":{"total":773094113280,"available":772850278400,"internally_reserved":0,"allocated":5402624,"data_stored":2999561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":742,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.300007"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.567012+0000","last_change":"2024-05-08T05:42:58.567012+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.567012+0000","last_undegraded":"2024-05-08T05:42:58.567012+0000","last_fullsized":"2024-05-08T05:42:58.567012+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.590291+0000","last_change":"2024-05-08T05:42:59.586491+0000","last_active":"2024-05-08T05:42:59.590291+0000","last_peered":"2024-05-08T05:42:59.590291+0000","last_clean":"2024-05-08T05:42:59.590291+0000","last_became_active":"2024-05-08T05:42:59.586233+0000","last_became_peered":"2024-05-08T05:42:59.586233+0000","last_unstale":"2024-05-08T05:42:59.590291+0000","last_undegraded":"2024-05-08T05:42:59.590291+0000","last_fullsized":"2024-05-08T05:42:59.590291+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:32.677283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.565275+0000","last_change":"2024-05-08T05:42:58.565275+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.565275+0000","last_undegraded":"2024-05-08T05:42:58.565275+0000","last_fullsized":"2024-05-08T05:42:58.565275+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.595331+0000","last_change":"2024-05-08T05:42:59.589298+0000","last_active":"2024-05-08T05:42:59.595331+0000","last_peered":"2024-05-08T05:42:59.595331+0000","last_clean":"2024-05-08T05:42:59.595331+0000","last_became_active":"2024-05-08T05:42:59.588989+0000","last_became_peered":"2024-05-08T05:42:59.588989+0000","last_unstale":"2024-05-08T05:42:59.595331+0000","last_undegraded":"2024-05-08T05:42:59.595331+0000","last_fullsized":"2024-05-08T05:42:59.595331+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:54:26.126199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.559296+0000","last_change":"2024-05-08T05:42:58.559296+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.559296+0000","last_undegraded":"2024-05-08T05:42:58.559296+0000","last_fullsized":"2024-05-08T05:42:58.559296+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.569796+0000","last_change":"2024-05-08T05:42:58.569796+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.569796+0000","last_undegraded":"2024-05-08T05:42:58.569796+0000","last_fullsized":"2024-05-08T05:42:58.569796+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.565860+0000","last_change":"2024-05-08T05:42:58.565860+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.565860+0000","last_undegraded":"2024-05-08T05:42:58.565860+0000","last_fullsized":"2024-05-08T05:42:58.565860+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.564839+0000","last_change":"2024-05-08T05:42:58.564839+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.564839+0000","last_undegraded":"2024-05-08T05:42:58.564839+0000","last_fullsized":"2024-05-08T05:42:58.564839+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566530+0000","last_change":"2024-05-08T05:42:58.566530+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566530+0000","last_undegraded":"2024-05-08T05:42:58.566530+0000","last_fullsized":"2024-05-08T05:42:58.566530+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566857+0000","last_change":"2024-05-08T05:42:58.566857+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566857+0000","last_undegraded":"2024-05-08T05:42:58.566857+0000","last_fullsized":"2024-05-08T05:42:58.566857+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.560543+0000","last_change":"2024-05-08T05:42:58.560543+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.560543+0000","last_undegraded":"2024-05-08T05:42:58.560543+0000","last_fullsized":"2024-05-08T05:42:58.560543+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.567647+0000","last_change":"2024-05-08T05:42:58.567647+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.567647+0000","last_undegraded":"2024-05-08T05:42:58.567647+0000","last_fullsized":"2024-05-08T05:42:58.567647+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.563061+0000","last_change":"2024-05-08T05:42:58.563061+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.563061+0000","last_undegraded":"2024-05-08T05:42:58.563061+0000","last_fullsized":"2024-05-08T05:42:58.563061+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.562034+0000","last_change":"2024-05-08T05:42:58.562034+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.562034+0000","last_undegraded":"2024-05-08T05:42:58.562034+0000","last_fullsized":"2024-05-08T05:42:58.562034+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.598708+0000","last_change":"2024-05-08T05:42:59.593026+0000","last_active":"2024-05-08T05:42:59.598708+0000","last_peered":"2024-05-08T05:42:59.598708+0000","last_clean":"2024-05-08T05:42:59.598708+0000","last_became_active":"2024-05-08T05:42:59.592706+0000","last_became_peered":"2024-05-08T05:42:59.592706+0000","last_unstale":"2024-05-08T05:42:59.598708+0000","last_undegraded":"2024-05-08T05:42:59.598708+0000","last_fullsized":"2024-05-08T05:42:59.598708+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:34.093891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.561473+0000","last_change":"2024-05-08T05:42:58.561473+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.561473+0000","last_undegraded":"2024-05-08T05:42:58.561473+0000","last_fullsized":"2024-05-08T05:42:58.561473+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566253+0000","last_change":"2024-05-08T05:42:58.566253+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566253+0000","last_undegraded":"2024-05-08T05:42:58.566253+0000","last_fullsized":"2024-05-08T05:42:58.566253+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.598545+0000","last_change":"2024-05-08T05:42:59.592278+0000","last_active":"2024-05-08T05:42:59.598545+0000","last_peered":"2024-05-08T05:42:59.598545+0000","last_clean":"2024-05-08T05:42:59.598545+0000","last_became_active":"2024-05-08T05:42:59.591985+0000","last_became_peered":"2024-05-08T05:42:59.591985+0000","last_unstale":"2024-05-08T05:42:59.598545+0000","last_undegraded":"2024-05-08T05:42:59.598545+0000","last_fullsized":"2024-05-08T05:42:59.598545+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:42:12.700183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.564859+0000","last_change":"2024-05-08T05:42:58.564859+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.564859+0000","last_undegraded":"2024-05-08T05:42:58.564859+0000","last_fullsized":"2024-05-08T05:42:58.564859+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.590796+0000","last_change":"2024-05-08T05:42:59.584649+0000","last_active":"2024-05-08T05:42:59.590796+0000","last_peered":"2024-05-08T05:42:59.590796+0000","last_clean":"2024-05-08T05:42:59.590796+0000","last_became_active":"2024-05-08T05:42:59.584233+0000","last_became_peered":"2024-05-08T05:42:59.584233+0000","last_unstale":"2024-05-08T05:42:59.590796+0000","last_undegraded":"2024-05-08T05:42:59.590796+0000","last_fullsized":"2024-05-08T05:42:59.590796+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:49.405283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.595031+0000","last_change":"2024-05-08T05:42:59.591062+0000","last_active":"2024-05-08T05:42:59.595031+0000","last_peered":"2024-05-08T05:42:59.595031+0000","last_clean":"2024-05-08T05:42:59.595031+0000","last_became_active":"2024-05-08T05:42:59.590680+0000","last_became_peered":"2024-05-08T05:42:59.590680+0000","last_unstale":"2024-05-08T05:42:59.595031+0000","last_undegraded":"2024-05-08T05:42:59.595031+0000","last_fullsized":"2024-05-08T05:42:59.595031+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:05:27.975200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.563363+0000","last_change":"2024-05-08T05:42:58.563363+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.563363+0000","last_undegraded":"2024-05-08T05:42:58.563363+0000","last_fullsized":"2024-05-08T05:42:58.563363+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.560421+0000","last_change":"2024-05-08T05:42:58.560421+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.560421+0000","last_undegraded":"2024-05-08T05:42:58.560421+0000","last_fullsized":"2024-05-08T05:42:58.560421+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.561384+0000","last_change":"2024-05-08T05:42:58.561384+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.561384+0000","last_undegraded":"2024-05-08T05:42:58.561384+0000","last_fullsized":"2024-05-08T05:42:58.561384+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566986+0000","last_change":"2024-05-08T05:42:58.566986+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566986+0000","last_undegraded":"2024-05-08T05:42:58.566986+0000","last_fullsized":"2024-05-08T05:42:58.566986+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566924+0000","last_change":"2024-05-08T05:42:58.566924+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566924+0000","last_undegraded":"2024-05-08T05:42:58.566924+0000","last_fullsized":"2024-05-08T05:42:58.566924+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.565197+0000","last_change":"2024-05-08T05:42:58.565197+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.565197+0000","last_undegraded":"2024-05-08T05:42:58.565197+0000","last_fullsized":"2024-05-08T05:42:58.565197+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.589440+0000","last_change":"2024-05-08T05:42:59.582258+0000","last_active":"2024-05-08T05:42:59.589440+0000","last_peered":"2024-05-08T05:42:59.589440+0000","last_clean":"2024-05-08T05:42:59.589440+0000","last_became_active":"2024-05-08T05:42:59.581973+0000","last_became_peered":"2024-05-08T05:42:59.581973+0000","last_unstale":"2024-05-08T05:42:59.589440+0000","last_undegraded":"2024-05-08T05:42:59.589440+0000","last_fullsized":"2024-05-08T05:42:59.589440+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:45:38.272590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.562110+0000","last_change":"2024-05-08T05:42:58.562110+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.562110+0000","last_undegraded":"2024-05-08T05:42:58.562110+0000","last_fullsized":"2024-05-08T05:42:58.562110+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.590188+0000","last_change":"2024-05-08T05:42:59.583375+0000","last_active":"2024-05-08T05:42:59.590188+0000","last_peered":"2024-05-08T05:42:59.590188+0000","last_clean":"2024-05-08T05:42:59.590188+0000","last_became_active":"2024-05-08T05:42:59.583026+0000","last_became_peered":"2024-05-08T05:42:59.583026+0000","last_unstale":"2024-05-08T05:42:59.590188+0000","last_undegraded":"2024-05-08T05:42:59.590188+0000","last_fullsized":"2024-05-08T05:42:59.590188+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:09.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.562320+0000","last_change":"2024-05-08T05:42:58.562320+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.562320+0000","last_undegraded":"2024-05-08T05:42:58.562320+0000","last_fullsized":"2024-05-08T05:42:58.562320+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.595084+0000","last_change":"2024-05-08T05:42:59.586488+0000","last_active":"2024-05-08T05:42:59.595084+0000","last_peered":"2024-05-08T05:42:59.595084+0000","last_clean":"2024-05-08T05:42:59.595084+0000","last_became_active":"2024-05-08T05:42:59.585806+0000","last_became_peered":"2024-05-08T05:42:59.585806+0000","last_unstale":"2024-05-08T05:42:59.595084+0000","last_undegraded":"2024-05-08T05:42:59.595084+0000","last_fullsized":"2024-05-08T05:42:59.595084+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:42:12.700183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.598596+0000","last_change":"2024-05-08T05:42:59.593948+0000","last_active":"2024-05-08T05:42:59.598596+0000","last_peered":"2024-05-08T05:42:59.598596+0000","last_clean":"2024-05-08T05:42:59.598596+0000","last_became_active":"2024-05-08T05:42:59.593663+0000","last_became_peered":"2024-05-08T05:42:59.593663+0000","last_unstale":"2024-05-08T05:42:59.598596+0000","last_undegraded":"2024-05-08T05:42:59.598596+0000","last_fullsized":"2024-05-08T05:42:59.598596+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:47:11.811597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"46'4","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.601783+0000","last_change":"2024-05-08T05:42:58.568449+0000","last_active":"2024-05-08T05:42:59.601783+0000","last_peered":"2024-05-08T05:42:59.601783+0000","last_clean":"2024-05-08T05:42:59.601783+0000","last_became_active":"2024-05-08T05:42:58.567378+0000","last_became_peered":"2024-05-08T05:42:58.567378+0000","last_unstale":"2024-05-08T05:42:59.601783+0000","last_undegraded":"2024-05-08T05:42:59.601783+0000","last_fullsized":"2024-05-08T05:42:59.601783+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:04.957071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.591538+0000","last_change":"2024-05-08T05:42:59.583673+0000","last_active":"2024-05-08T05:42:59.591538+0000","last_peered":"2024-05-08T05:42:59.591538+0000","last_clean":"2024-05-08T05:42:59.591538+0000","last_became_active":"2024-05-08T05:42:59.583430+0000","last_became_peered":"2024-05-08T05:42:59.583430+0000","last_unstale":"2024-05-08T05:42:59.591538+0000","last_undegraded":"2024-05-08T05:42:59.591538+0000","last_fullsized":"2024-05-08T05:42:59.591538+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:32.677283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.583411+0000","last_change":"2024-05-08T05:42:58.582169+0000","last_active":"2024-05-08T05:42:58.583411+0000","last_peered":"2024-05-08T05:42:58.583411+0000","last_clean":"2024-05-08T05:42:58.583411+0000","last_became_active":"2024-05-08T05:42:58.581851+0000","last_became_peered":"2024-05-08T05:42:58.581851+0000","last_unstale":"2024-05-08T05:42:58.583411+0000","last_undegraded":"2024-05-08T05:42:58.583411+0000","last_fullsized":"2024-05-08T05:42:58.583411+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:28.357714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.590873+0000","last_change":"2024-05-08T05:42:59.585789+0000","last_active":"2024-05-08T05:42:59.590873+0000","last_peered":"2024-05-08T05:42:59.590873+0000","last_clean":"2024-05-08T05:42:59.590873+0000","last_became_active":"2024-05-08T05:42:59.585486+0000","last_became_peered":"2024-05-08T05:42:59.585486+0000","last_unstale":"2024-05-08T05:42:59.590873+0000","last_undegraded":"2024-05-08T05:42:59.590873+0000","last_fullsized":"2024-05-08T05:42:59.590873+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:13.547702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.579505+0000","last_change":"2024-05-08T05:42:58.577001+0000","last_active":"2024-05-08T05:42:58.579505+0000","last_peered":"2024-05-08T05:42:58.579505+0000","last_clean":"2024-05-08T05:42:58.579505+0000","last_became_active":"2024-05-08T05:42:58.575665+0000","last_became_peered":"2024-05-08T05:42:58.575665+0000","last_unstale":"2024-05-08T05:42:58.579505+0000","last_undegraded":"2024-05-08T05:42:58.579505+0000","last_fullsized":"2024-05-08T05:42:58.579505+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:55:31.695068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.567678+0000","last_change":"2024-05-08T05:42:58.567678+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.567678+0000","last_undegraded":"2024-05-08T05:42:58.567678+0000","last_fullsized":"2024-05-08T05:42:58.567678+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.602422+0000","last_change":"2024-05-08T05:42:58.573567+0000","last_active":"2024-05-08T05:42:59.602422+0000","last_peered":"2024-05-08T05:42:59.602422+0000","last_clean":"2024-05-08T05:42:59.602422+0000","last_became_active":"2024-05-08T05:42:58.573220+0000","last_became_peered":"2024-05-08T05:42:58.573220+0000","last_unstale":"2024-05-08T05:42:59.602422+0000","last_undegraded":"2024-05-08T05:42:59.602422+0000","last_fullsized":"2024-05-08T05:42:59.602422+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:15.376533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.558663+0000","last_change":"2024-05-08T05:42:58.558663+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.558663+0000","last_undegraded":"2024-05-08T05:42:58.558663+0000","last_fullsized":"2024-05-08T05:42:58.558663+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.576887+0000","last_change":"2024-05-08T05:42:58.572908+0000","last_active":"2024-05-08T05:42:58.576887+0000","last_peered":"2024-05-08T05:42:58.576887+0000","last_clean":"2024-05-08T05:42:58.576887+0000","last_became_active":"2024-05-08T05:42:58.572613+0000","last_became_peered":"2024-05-08T05:42:58.572613+0000","last_unstale":"2024-05-08T05:42:58.576887+0000","last_undegraded":"2024-05-08T05:42:58.576887+0000","last_fullsized":"2024-05-08T05:42:58.576887+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:39.779366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.598156+0000","last_change":"2024-05-08T05:42:59.589512+0000","last_active":"2024-05-08T05:42:59.598156+0000","last_peered":"2024-05-08T05:42:59.598156+0000","last_clean":"2024-05-08T05:42:59.598156+0000","last_became_active":"2024-05-08T05:42:59.588973+0000","last_became_peered":"2024-05-08T05:42:59.588973+0000","last_unstale":"2024-05-08T05:42:59.598156+0000","last_undegraded":"2024-05-08T05:42:59.598156+0000","last_fullsized":"2024-05-08T05:42:59.598156+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:34:10.738939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.571718+0000","last_change":"2024-05-08T05:42:58.569466+0000","last_active":"2024-05-08T05:42:58.571718+0000","last_peered":"2024-05-08T05:42:58.571718+0000","last_clean":"2024-05-08T05:42:58.571718+0000","last_became_active":"2024-05-08T05:42:58.569048+0000","last_became_peered":"2024-05-08T05:42:58.569048+0000","last_unstale":"2024-05-08T05:42:58.571718+0000","last_undegraded":"2024-05-08T05:42:58.571718+0000","last_fullsized":"2024-05-08T05:42:58.571718+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:30:44.118693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.565142+0000","last_change":"2024-05-08T05:42:58.565142+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.565142+0000","last_undegraded":"2024-05-08T05:42:58.565142+0000","last_fullsized":"2024-05-08T05:42:58.565142+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.575648+0000","last_change":"2024-05-08T05:42:58.568749+0000","last_active":"2024-05-08T05:42:59.575648+0000","last_peered":"2024-05-08T05:42:59.575648+0000","last_clean":"2024-05-08T05:42:59.575648+0000","last_became_active":"2024-05-08T05:42:58.568473+0000","last_became_peered":"2024-05-08T05:42:58.568473+0000","last_unstale":"2024-05-08T05:42:59.575648+0000","last_undegraded":"2024-05-08T05:42:59.575648+0000","last_fullsized":"2024-05-08T05:42:59.575648+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:55:31.695068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.598506+0000","last_change":"2024-05-08T05:42:59.591341+0000","last_active":"2024-05-08T05:42:59.598506+0000","last_peered":"2024-05-08T05:42:59.598506+0000","last_clean":"2024-05-08T05:42:59.598506+0000","last_became_active":"2024-05-08T05:42:59.591050+0000","last_became_peered":"2024-05-08T05:42:59.591050+0000","last_unstale":"2024-05-08T05:42:59.598506+0000","last_undegraded":"2024-05-08T05:42:59.598506+0000","last_fullsized":"2024-05-08T05:42:59.598506+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:46.820598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.577480+0000","last_change":"2024-05-08T05:42:58.574070+0000","last_active":"2024-05-08T05:42:58.577480+0000","last_peered":"2024-05-08T05:42:58.577480+0000","last_clean":"2024-05-08T05:42:58.577480+0000","last_became_active":"2024-05-08T05:42:58.573769+0000","last_became_peered":"2024-05-08T05:42:58.573769+0000","last_unstale":"2024-05-08T05:42:58.577480+0000","last_undegraded":"2024-05-08T05:42:58.577480+0000","last_fullsized":"2024-05-08T05:42:58.577480+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:55:31.695068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.558457+0000","last_change":"2024-05-08T05:42:58.558457+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.558457+0000","last_undegraded":"2024-05-08T05:42:58.558457+0000","last_fullsized":"2024-05-08T05:42:58.558457+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.580407+0000","last_change":"2024-05-08T05:42:58.567761+0000","last_active":"2024-05-08T05:42:59.580407+0000","last_peered":"2024-05-08T05:42:59.580407+0000","last_clean":"2024-05-08T05:42:59.580407+0000","last_became_active":"2024-05-08T05:42:58.567490+0000","last_became_peered":"2024-05-08T05:42:58.567490+0000","last_unstale":"2024-05-08T05:42:59.580407+0000","last_undegraded":"2024-05-08T05:42:59.580407+0000","last_fullsized":"2024-05-08T05:42:59.580407+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:23:17.808492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.595122+0000","last_change":"2024-05-08T05:42:59.585553+0000","last_active":"2024-05-08T05:42:59.595122+0000","last_peered":"2024-05-08T05:42:59.595122+0000","last_clean":"2024-05-08T05:42:59.595122+0000","last_became_active":"2024-05-08T05:42:59.584903+0000","last_became_peered":"2024-05-08T05:42:59.584903+0000","last_unstale":"2024-05-08T05:42:59.595122+0000","last_undegraded":"2024-05-08T05:42:59.595122+0000","last_fullsized":"2024-05-08T05:42:59.595122+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:54:26.126199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.575724+0000","last_change":"2024-05-08T05:42:58.574888+0000","last_active":"2024-05-08T05:42:58.575724+0000","last_peered":"2024-05-08T05:42:58.575724+0000","last_clean":"2024-05-08T05:42:58.575724+0000","last_became_active":"2024-05-08T05:42:58.574585+0000","last_became_peered":"2024-05-08T05:42:58.574585+0000","last_unstale":"2024-05-08T05:42:58.575724+0000","last_undegraded":"2024-05-08T05:42:58.575724+0000","last_fullsized":"2024-05-08T05:42:58.575724+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:44:20.899641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.561535+0000","last_change":"2024-05-08T05:42:58.561535+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.561535+0000","last_undegraded":"2024-05-08T05:42:58.561535+0000","last_fullsized":"2024-05-08T05:42:58.561535+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"46'3","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.601464+0000","last_change":"2024-05-08T05:42:58.572885+0000","last_active":"2024-05-08T05:42:59.601464+0000","last_peered":"2024-05-08T05:42:59.601464+0000","last_clean":"2024-05-08T05:42:59.601464+0000","last_became_active":"2024-05-08T05:42:58.571751+0000","last_became_peered":"2024-05-08T05:42:58.571751+0000","last_unstale":"2024-05-08T05:42:59.601464+0000","last_undegraded":"2024-05-08T05:42:59.601464+0000","last_fullsized":"2024-05-08T05:42:59.601464+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:15.376533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.558448+0000","last_change":"2024-05-08T05:42:58.558448+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.558448+0000","last_undegraded":"2024-05-08T05:42:58.558448+0000","last_fullsized":"2024-05-08T05:42:58.558448+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.581919+0000","last_change":"2024-05-08T05:42:58.578675+0000","last_active":"2024-05-08T05:42:58.581919+0000","last_peered":"2024-05-08T05:42:58.581919+0000","last_clean":"2024-05-08T05:42:58.581919+0000","last_became_active":"2024-05-08T05:42:58.578130+0000","last_became_peered":"2024-05-08T05:42:58.578130+0000","last_unstale":"2024-05-08T05:42:58.581919+0000","last_undegraded":"2024-05-08T05:42:58.581919+0000","last_fullsized":"2024-05-08T05:42:58.581919+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:15.376533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.591669+0000","last_change":"2024-05-08T05:42:59.583647+0000","last_active":"2024-05-08T05:42:59.591669+0000","last_peered":"2024-05-08T05:42:59.591669+0000","last_clean":"2024-05-08T05:42:59.591669+0000","last_became_active":"2024-05-08T05:42:59.583302+0000","last_became_peered":"2024-05-08T05:42:59.583302+0000","last_unstale":"2024-05-08T05:42:59.591669+0000","last_undegraded":"2024-05-08T05:42:59.591669+0000","last_fullsized":"2024-05-08T05:42:59.591669+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:23:18.816427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.577616+0000","last_change":"2024-05-08T05:42:58.576622+0000","last_active":"2024-05-08T05:42:58.577616+0000","last_peered":"2024-05-08T05:42:58.577616+0000","last_clean":"2024-05-08T05:42:58.577616+0000","last_became_active":"2024-05-08T05:42:58.575446+0000","last_became_peered":"2024-05-08T05:42:58.575446+0000","last_unstale":"2024-05-08T05:42:58.577616+0000","last_undegraded":"2024-05-08T05:42:58.577616+0000","last_fullsized":"2024-05-08T05:42:58.577616+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:46.384674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.564425+0000","last_change":"2024-05-08T05:42:58.564425+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.564425+0000","last_undegraded":"2024-05-08T05:42:58.564425+0000","last_fullsized":"2024-05-08T05:42:58.564425+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.575907+0000","last_change":"2024-05-08T05:42:58.574983+0000","last_active":"2024-05-08T05:42:59.575907+0000","last_peered":"2024-05-08T05:42:59.575907+0000","last_clean":"2024-05-08T05:42:59.575907+0000","last_became_active":"2024-05-08T05:42:58.574678+0000","last_became_peered":"2024-05-08T05:42:58.574678+0000","last_unstale":"2024-05-08T05:42:59.575907+0000","last_undegraded":"2024-05-08T05:42:59.575907+0000","last_fullsized":"2024-05-08T05:42:59.575907+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:28.357714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.565577+0000","last_change":"2024-05-08T05:42:58.565577+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.565577+0000","last_undegraded":"2024-05-08T05:42:58.565577+0000","last_fullsized":"2024-05-08T05:42:58.565577+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.577776+0000","last_change":"2024-05-08T05:42:58.572724+0000","last_active":"2024-05-08T05:42:59.577776+0000","last_peered":"2024-05-08T05:42:59.577776+0000","last_clean":"2024-05-08T05:42:59.577776+0000","last_became_active":"2024-05-08T05:42:58.572471+0000","last_became_peered":"2024-05-08T05:42:58.572471+0000","last_unstale":"2024-05-08T05:42:59.577776+0000","last_undegraded":"2024-05-08T05:42:59.577776+0000","last_fullsized":"2024-05-08T05:42:59.577776+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:28.063244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.567393+0000","last_change":"2024-05-08T05:42:58.567393+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.567393+0000","last_undegraded":"2024-05-08T05:42:58.567393+0000","last_fullsized":"2024-05-08T05:42:58.567393+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.581728+0000","last_change":"2024-05-08T05:42:58.577909+0000","last_active":"2024-05-08T05:42:58.581728+0000","last_peered":"2024-05-08T05:42:58.581728+0000","last_clean":"2024-05-08T05:42:58.581728+0000","last_became_active":"2024-05-08T05:42:58.577585+0000","last_became_peered":"2024-05-08T05:42:58.577585+0000","last_unstale":"2024-05-08T05:42:58.581728+0000","last_undegraded":"2024-05-08T05:42:58.581728+0000","last_fullsized":"2024-05-08T05:42:58.581728+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:28.063244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.564368+0000","last_change":"2024-05-08T05:42:58.564368+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.564368+0000","last_undegraded":"2024-05-08T05:42:58.564368+0000","last_fullsized":"2024-05-08T05:42:58.564368+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.580755+0000","last_change":"2024-05-08T05:42:58.579816+0000","last_active":"2024-05-08T05:42:58.580755+0000","last_peered":"2024-05-08T05:42:58.580755+0000","last_clean":"2024-05-08T05:42:58.580755+0000","last_became_active":"2024-05-08T05:42:58.579567+0000","last_became_peered":"2024-05-08T05:42:58.579567+0000","last_unstale":"2024-05-08T05:42:58.580755+0000","last_undegraded":"2024-05-08T05:42:58.580755+0000","last_fullsized":"2024-05-08T05:42:58.580755+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:32.556335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.561728+0000","last_change":"2024-05-08T05:42:58.561728+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.561728+0000","last_undegraded":"2024-05-08T05:42:58.561728+0000","last_fullsized":"2024-05-08T05:42:58.561728+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.571814+0000","last_change":"2024-05-08T05:42:58.568207+0000","last_active":"2024-05-08T05:42:58.571814+0000","last_peered":"2024-05-08T05:42:58.571814+0000","last_clean":"2024-05-08T05:42:58.571814+0000","last_became_active":"2024-05-08T05:42:58.567932+0000","last_became_peered":"2024-05-08T05:42:58.567932+0000","last_unstale":"2024-05-08T05:42:58.571814+0000","last_undegraded":"2024-05-08T05:42:58.571814+0000","last_fullsized":"2024-05-08T05:42:58.571814+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:53:44.502777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.558840+0000","last_change":"2024-05-08T05:42:58.558840+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.558840+0000","last_undegraded":"2024-05-08T05:42:58.558840+0000","last_fullsized":"2024-05-08T05:42:58.558840+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.599929+0000","last_change":"2024-05-08T05:42:58.573572+0000","last_active":"2024-05-08T05:42:59.599929+0000","last_peered":"2024-05-08T05:42:59.599929+0000","last_clean":"2024-05-08T05:42:59.599929+0000","last_became_active":"2024-05-08T05:42:58.573275+0000","last_became_peered":"2024-05-08T05:42:58.573275+0000","last_unstale":"2024-05-08T05:42:59.599929+0000","last_undegraded":"2024-05-08T05:42:59.599929+0000","last_fullsized":"2024-05-08T05:42:59.599929+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:23:53.968037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.562629+0000","last_change":"2024-05-08T05:42:58.562629+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.562629+0000","last_undegraded":"2024-05-08T05:42:58.562629+0000","last_fullsized":"2024-05-08T05:42:58.562629+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.576012+0000","last_change":"2024-05-08T05:42:58.579945+0000","last_active":"2024-05-08T05:42:59.576012+0000","last_peered":"2024-05-08T05:42:59.576012+0000","last_clean":"2024-05-08T05:42:59.576012+0000","last_became_active":"2024-05-08T05:42:58.579657+0000","last_became_peered":"2024-05-08T05:42:58.579657+0000","last_unstale":"2024-05-08T05:42:59.576012+0000","last_undegraded":"2024-05-08T05:42:59.576012+0000","last_fullsized":"2024-05-08T05:42:59.576012+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:46.384674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.595611+0000","last_change":"2024-05-08T05:42:59.585604+0000","last_active":"2024-05-08T05:42:59.595611+0000","last_peered":"2024-05-08T05:42:59.595611+0000","last_clean":"2024-05-08T05:42:59.595611+0000","last_became_active":"2024-05-08T05:42:59.585027+0000","last_became_peered":"2024-05-08T05:42:59.585027+0000","last_unstale":"2024-05-08T05:42:59.595611+0000","last_undegraded":"2024-05-08T05:42:59.595611+0000","last_fullsized":"2024-05-08T05:42:59.595611+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:46.820598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.577900+0000","last_change":"2024-05-08T05:42:58.574363+0000","last_active":"2024-05-08T05:42:58.577900+0000","last_peered":"2024-05-08T05:42:58.577900+0000","last_clean":"2024-05-08T05:42:58.577900+0000","last_became_active":"2024-05-08T05:42:58.574101+0000","last_became_peered":"2024-05-08T05:42:58.574101+0000","last_unstale":"2024-05-08T05:42:58.577900+0000","last_undegraded":"2024-05-08T05:42:58.577900+0000","last_fullsized":"2024-05-08T05:42:58.577900+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:39.779366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.590917+0000","last_change":"2024-05-08T05:42:59.587267+0000","last_active":"2024-05-08T05:42:59.590917+0000","last_peered":"2024-05-08T05:42:59.590917+0000","last_clean":"2024-05-08T05:42:59.590917+0000","last_became_active":"2024-05-08T05:42:59.586824+0000","last_became_peered":"2024-05-08T05:42:59.586824+0000","last_unstale":"2024-05-08T05:42:59.590917+0000","last_undegraded":"2024-05-08T05:42:59.590917+0000","last_fullsized":"2024-05-08T05:42:59.590917+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:28:42.000118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.574671+0000","last_change":"2024-05-08T05:42:58.572523+0000","last_active":"2024-05-08T05:42:58.574671+0000","last_peered":"2024-05-08T05:42:58.574671+0000","last_clean":"2024-05-08T05:42:58.574671+0000","last_became_active":"2024-05-08T05:42:58.572193+0000","last_became_peered":"2024-05-08T05:42:58.572193+0000","last_unstale":"2024-05-08T05:42:58.574671+0000","last_undegraded":"2024-05-08T05:42:58.574671+0000","last_fullsized":"2024-05-08T05:42:58.574671+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:04.051382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.563623+0000","last_change":"2024-05-08T05:42:58.563623+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.563623+0000","last_undegraded":"2024-05-08T05:42:58.563623+0000","last_fullsized":"2024-05-08T05:42:58.563623+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.582841+0000","last_change":"2024-05-08T05:42:58.582112+0000","last_active":"2024-05-08T05:42:58.582841+0000","last_peered":"2024-05-08T05:42:58.582841+0000","last_clean":"2024-05-08T05:42:58.582841+0000","last_became_active":"2024-05-08T05:42:58.581779+0000","last_became_peered":"2024-05-08T05:42:58.581779+0000","last_unstale":"2024-05-08T05:42:58.582841+0000","last_undegraded":"2024-05-08T05:42:58.582841+0000","last_fullsized":"2024-05-08T05:42:58.582841+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:23:53.968037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566556+0000","last_change":"2024-05-08T05:42:58.566556+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566556+0000","last_undegraded":"2024-05-08T05:42:58.566556+0000","last_fullsized":"2024-05-08T05:42:58.566556+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.571905+0000","last_change":"2024-05-08T05:42:58.568551+0000","last_active":"2024-05-08T05:42:58.571905+0000","last_peered":"2024-05-08T05:42:58.571905+0000","last_clean":"2024-05-08T05:42:58.571905+0000","last_became_active":"2024-05-08T05:42:58.568282+0000","last_became_peered":"2024-05-08T05:42:58.568282+0000","last_unstale":"2024-05-08T05:42:58.571905+0000","last_undegraded":"2024-05-08T05:42:58.571905+0000","last_fullsized":"2024-05-08T05:42:58.571905+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:35:01.162157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.595061+0000","last_change":"2024-05-08T05:42:59.586138+0000","last_active":"2024-05-08T05:42:59.595061+0000","last_peered":"2024-05-08T05:42:59.595061+0000","last_clean":"2024-05-08T05:42:59.595061+0000","last_became_active":"2024-05-08T05:42:59.585739+0000","last_became_peered":"2024-05-08T05:42:59.585739+0000","last_unstale":"2024-05-08T05:42:59.595061+0000","last_undegraded":"2024-05-08T05:42:59.595061+0000","last_fullsized":"2024-05-08T05:42:59.595061+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:34:10.738939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.605908+0000","last_change":"2024-05-08T05:42:58.580617+0000","last_active":"2024-05-08T05:42:59.605908+0000","last_peered":"2024-05-08T05:42:59.605908+0000","last_clean":"2024-05-08T05:42:59.605908+0000","last_became_active":"2024-05-08T05:42:58.580371+0000","last_became_peered":"2024-05-08T05:42:58.580371+0000","last_unstale":"2024-05-08T05:42:59.605908+0000","last_undegraded":"2024-05-08T05:42:59.605908+0000","last_fullsized":"2024-05-08T05:42:59.605908+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:32.556335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.562862+0000","last_change":"2024-05-08T05:42:58.562862+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.562862+0000","last_undegraded":"2024-05-08T05:42:58.562862+0000","last_fullsized":"2024-05-08T05:42:58.562862+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.578285+0000","last_change":"2024-05-08T05:42:58.570248+0000","last_active":"2024-05-08T05:42:59.578285+0000","last_peered":"2024-05-08T05:42:59.578285+0000","last_clean":"2024-05-08T05:42:59.578285+0000","last_became_active":"2024-05-08T05:42:58.569966+0000","last_became_peered":"2024-05-08T05:42:58.569966+0000","last_unstale":"2024-05-08T05:42:59.578285+0000","last_undegraded":"2024-05-08T05:42:59.578285+0000","last_fullsized":"2024-05-08T05:42:59.578285+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:25.616180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.567229+0000","last_change":"2024-05-08T05:42:58.567229+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.567229+0000","last_undegraded":"2024-05-08T05:42:58.567229+0000","last_fullsized":"2024-05-08T05:42:58.567229+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.580617+0000","last_change":"2024-05-08T05:42:58.577804+0000","last_active":"2024-05-08T05:42:58.580617+0000","last_peered":"2024-05-08T05:42:58.580617+0000","last_clean":"2024-05-08T05:42:58.580617+0000","last_became_active":"2024-05-08T05:42:58.577442+0000","last_became_peered":"2024-05-08T05:42:58.577442+0000","last_unstale":"2024-05-08T05:42:58.580617+0000","last_undegraded":"2024-05-08T05:42:58.580617+0000","last_fullsized":"2024-05-08T05:42:58.580617+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:04.957071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.563979+0000","last_change":"2024-05-08T05:42:58.563979+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.563979+0000","last_undegraded":"2024-05-08T05:42:58.563979+0000","last_fullsized":"2024-05-08T05:42:58.563979+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.573849+0000","last_change":"2024-05-08T05:42:58.571644+0000","last_active":"2024-05-08T05:42:58.573849+0000","last_peered":"2024-05-08T05:42:58.573849+0000","last_clean":"2024-05-08T05:42:58.573849+0000","last_became_active":"2024-05-08T05:42:58.571348+0000","last_became_peered":"2024-05-08T05:42:58.571348+0000","last_unstale":"2024-05-08T05:42:58.573849+0000","last_undegraded":"2024-05-08T05:42:58.573849+0000","last_fullsized":"2024-05-08T05:42:58.573849+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:24:55.348642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.591367+0000","last_change":"2024-05-08T05:42:59.584308+0000","last_active":"2024-05-08T05:42:59.591367+0000","last_peered":"2024-05-08T05:42:59.591367+0000","last_clean":"2024-05-08T05:42:59.591367+0000","last_became_active":"2024-05-08T05:42:59.584026+0000","last_became_peered":"2024-05-08T05:42:59.584026+0000","last_unstale":"2024-05-08T05:42:59.591367+0000","last_undegraded":"2024-05-08T05:42:59.591367+0000","last_fullsized":"2024-05-08T05:42:59.591367+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:09.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.582620+0000","last_change":"2024-05-08T05:42:58.580460+0000","last_active":"2024-05-08T05:42:58.582620+0000","last_peered":"2024-05-08T05:42:58.582620+0000","last_clean":"2024-05-08T05:42:58.582620+0000","last_became_active":"2024-05-08T05:42:58.580085+0000","last_became_peered":"2024-05-08T05:42:58.580085+0000","last_unstale":"2024-05-08T05:42:58.582620+0000","last_undegraded":"2024-05-08T05:42:58.582620+0000","last_fullsized":"2024-05-08T05:42:58.582620+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:25.616180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.564637+0000","last_change":"2024-05-08T05:42:58.564637+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.564637+0000","last_undegraded":"2024-05-08T05:42:58.564637+0000","last_fullsized":"2024-05-08T05:42:58.564637+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.579427+0000","last_change":"2024-05-08T05:42:58.578326+0000","last_active":"2024-05-08T05:42:58.579427+0000","last_peered":"2024-05-08T05:42:58.579427+0000","last_clean":"2024-05-08T05:42:58.579427+0000","last_became_active":"2024-05-08T05:42:58.577971+0000","last_became_peered":"2024-05-08T05:42:58.577971+0000","last_unstale":"2024-05-08T05:42:58.579427+0000","last_undegraded":"2024-05-08T05:42:58.579427+0000","last_fullsized":"2024-05-08T05:42:58.579427+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:06:22.535707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2024-05-08T05:42:58.566087+0000","last_change":"2024-05-08T05:42:58.566087+0000","last_active":"2024-05-08T05:42:58.552624+0000","last_peered":"2024-05-08T05:42:58.552624+0000","last_clean":"2024-05-08T05:42:58.552624+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:42:58.566087+0000","last_undegraded":"2024-05-08T05:42:58.566087+0000","last_fullsized":"2024-05-08T05:42:58.566087+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.572277+0000","last_change":"2024-05-08T05:42:58.578763+0000","last_active":"2024-05-08T05:42:59.572277+0000","last_peered":"2024-05-08T05:42:59.572277+0000","last_clean":"2024-05-08T05:42:59.572277+0000","last_became_active":"2024-05-08T05:42:58.578430+0000","last_became_peered":"2024-05-08T05:42:58.578430+0000","last_unstale":"2024-05-08T05:42:59.572277+0000","last_undegraded":"2024-05-08T05:42:59.572277+0000","last_fullsized":"2024-05-08T05:42:59.572277+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:25.616180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.596172+0000","last_change":"2024-05-08T05:42:59.592843+0000","last_active":"2024-05-08T05:42:59.596172+0000","last_peered":"2024-05-08T05:42:59.596172+0000","last_clean":"2024-05-08T05:42:59.596172+0000","last_became_active":"2024-05-08T05:42:59.592317+0000","last_became_peered":"2024-05-08T05:42:59.592317+0000","last_unstale":"2024-05-08T05:42:59.596172+0000","last_undegraded":"2024-05-08T05:42:59.596172+0000","last_fullsized":"2024-05-08T05:42:59.596172+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:17.523257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-05-08T05:42:58.577643+0000","last_change":"2024-05-08T05:42:58.576321+0000","last_active":"2024-05-08T05:42:58.577643+0000","last_peered":"2024-05-08T05:42:58.577643+0000","last_clean":"2024-05-08T05:42:58.577643+0000","last_became_active":"2024-05-08T05:42:58.576057+0000","last_became_peered":"2024-05-08T05:42:58.576057+0000","last_unstale":"2024-05-08T05:42:58.577643+0000","last_undegraded":"2024-05-08T05:42:58.577643+0000","last_fullsized":"2024-05-08T05:42:58.577643+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:28.357714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":157,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.578796+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:42:59.578796+0000","last_peered":"2024-05-08T05:42:59.578796+0000","last_clean":"2024-05-08T05:42:59.578796+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:42:59.578796+0000","last_undegraded":"2024-05-08T05:42:59.578796+0000","last_fullsized":"2024-05-08T05:42:59.578796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":742,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542168,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1122304,"data_stored":818492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542169,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":235291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":228124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":516096,"data_stored":221457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":528384,"data_stored":228124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31700,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":528384,"data_stored":228124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":528384,"data_stored":228124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1110016,"data_stored":811825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:43:04.368 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3f is not active+clean 2024-05-08T05:43:04.369 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.567012+0000', 'last_change': '2024-05-08T05:42:58.567012+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.567012+0000', 'last_undegraded': '2024-05-08T05:42:58.567012+0000', 'last_fullsized': '2024-05-08T05:42:58.567012+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T05:43:04.369 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3d is not active+clean 2024-05-08T05:43:04.369 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.565275+0000', 'last_change': '2024-05-08T05:42:58.565275+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.565275+0000', 'last_undegraded': '2024-05-08T05:42:58.565275+0000', 'last_fullsized': '2024-05-08T05:42:58.565275+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T05:43:04.369 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3b is not active+clean 2024-05-08T05:43:04.369 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.559296+0000', 'last_change': '2024-05-08T05:42:58.559296+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.559296+0000', 'last_undegraded': '2024-05-08T05:42:58.559296+0000', 'last_fullsized': '2024-05-08T05:42:58.559296+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T05:43:04.369 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3a is not active+clean 2024-05-08T05:43:04.369 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.569796+0000', 'last_change': '2024-05-08T05:42:58.569796+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.569796+0000', 'last_undegraded': '2024-05-08T05:42:58.569796+0000', 'last_fullsized': '2024-05-08T05:42:58.569796+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T05:43:04.369 INFO:tasks.ceph.ceph_manager.ceph:PG 8.39 is not active+clean 2024-05-08T05:43:04.369 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.565860+0000', 'last_change': '2024-05-08T05:42:58.565860+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.565860+0000', 'last_undegraded': '2024-05-08T05:42:58.565860+0000', 'last_fullsized': '2024-05-08T05:42:58.565860+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T05:43:04.369 INFO:tasks.ceph.ceph_manager.ceph:PG 8.38 is not active+clean 2024-05-08T05:43:04.370 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.564839+0000', 'last_change': '2024-05-08T05:42:58.564839+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.564839+0000', 'last_undegraded': '2024-05-08T05:42:58.564839+0000', 'last_fullsized': '2024-05-08T05:42:58.564839+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T05:43:04.370 INFO:tasks.ceph.ceph_manager.ceph:PG 8.37 is not active+clean 2024-05-08T05:43:04.370 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.566530+0000', 'last_change': '2024-05-08T05:42:58.566530+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.566530+0000', 'last_undegraded': '2024-05-08T05:42:58.566530+0000', 'last_fullsized': '2024-05-08T05:42:58.566530+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T05:43:04.370 INFO:tasks.ceph.ceph_manager.ceph:PG 8.36 is not active+clean 2024-05-08T05:43:04.370 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.566857+0000', 'last_change': '2024-05-08T05:42:58.566857+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.566857+0000', 'last_undegraded': '2024-05-08T05:42:58.566857+0000', 'last_fullsized': '2024-05-08T05:42:58.566857+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T05:43:04.370 INFO:tasks.ceph.ceph_manager.ceph:PG 8.35 is not active+clean 2024-05-08T05:43:04.370 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.560543+0000', 'last_change': '2024-05-08T05:42:58.560543+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.560543+0000', 'last_undegraded': '2024-05-08T05:42:58.560543+0000', 'last_fullsized': '2024-05-08T05:42:58.560543+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T05:43:04.370 INFO:tasks.ceph.ceph_manager.ceph:PG 8.34 is not active+clean 2024-05-08T05:43:04.370 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.567647+0000', 'last_change': '2024-05-08T05:42:58.567647+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.567647+0000', 'last_undegraded': '2024-05-08T05:42:58.567647+0000', 'last_fullsized': '2024-05-08T05:42:58.567647+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T05:43:04.370 INFO:tasks.ceph.ceph_manager.ceph:PG 8.33 is not active+clean 2024-05-08T05:43:04.371 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.563061+0000', 'last_change': '2024-05-08T05:42:58.563061+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.563061+0000', 'last_undegraded': '2024-05-08T05:42:58.563061+0000', 'last_fullsized': '2024-05-08T05:42:58.563061+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T05:43:04.371 INFO:tasks.ceph.ceph_manager.ceph:PG 8.32 is not active+clean 2024-05-08T05:43:04.371 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.562034+0000', 'last_change': '2024-05-08T05:42:58.562034+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.562034+0000', 'last_undegraded': '2024-05-08T05:42:58.562034+0000', 'last_fullsized': '2024-05-08T05:42:58.562034+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T05:43:04.371 INFO:tasks.ceph.ceph_manager.ceph:PG 8.30 is not active+clean 2024-05-08T05:43:04.371 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.561473+0000', 'last_change': '2024-05-08T05:42:58.561473+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.561473+0000', 'last_undegraded': '2024-05-08T05:42:58.561473+0000', 'last_fullsized': '2024-05-08T05:42:58.561473+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T05:43:04.371 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2f is not active+clean 2024-05-08T05:43:04.371 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.566253+0000', 'last_change': '2024-05-08T05:42:58.566253+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.566253+0000', 'last_undegraded': '2024-05-08T05:42:58.566253+0000', 'last_fullsized': '2024-05-08T05:42:58.566253+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T05:43:04.371 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2d is not active+clean 2024-05-08T05:43:04.371 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.564859+0000', 'last_change': '2024-05-08T05:42:58.564859+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.564859+0000', 'last_undegraded': '2024-05-08T05:42:58.564859+0000', 'last_fullsized': '2024-05-08T05:42:58.564859+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T05:43:04.371 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2a is not active+clean 2024-05-08T05:43:04.372 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.563363+0000', 'last_change': '2024-05-08T05:42:58.563363+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.563363+0000', 'last_undegraded': '2024-05-08T05:42:58.563363+0000', 'last_fullsized': '2024-05-08T05:42:58.563363+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T05:43:04.372 INFO:tasks.ceph.ceph_manager.ceph:PG 8.29 is not active+clean 2024-05-08T05:43:04.372 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.560421+0000', 'last_change': '2024-05-08T05:42:58.560421+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.560421+0000', 'last_undegraded': '2024-05-08T05:42:58.560421+0000', 'last_fullsized': '2024-05-08T05:42:58.560421+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-05-08T05:43:04.372 INFO:tasks.ceph.ceph_manager.ceph:PG 8.28 is not active+clean 2024-05-08T05:43:04.372 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.561384+0000', 'last_change': '2024-05-08T05:42:58.561384+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.561384+0000', 'last_undegraded': '2024-05-08T05:42:58.561384+0000', 'last_fullsized': '2024-05-08T05:42:58.561384+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T05:43:04.372 INFO:tasks.ceph.ceph_manager.ceph:PG 8.27 is not active+clean 2024-05-08T05:43:04.372 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.566986+0000', 'last_change': '2024-05-08T05:42:58.566986+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.566986+0000', 'last_undegraded': '2024-05-08T05:42:58.566986+0000', 'last_fullsized': '2024-05-08T05:42:58.566986+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T05:43:04.372 INFO:tasks.ceph.ceph_manager.ceph:PG 8.26 is not active+clean 2024-05-08T05:43:04.372 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.566924+0000', 'last_change': '2024-05-08T05:42:58.566924+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.566924+0000', 'last_undegraded': '2024-05-08T05:42:58.566924+0000', 'last_fullsized': '2024-05-08T05:42:58.566924+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T05:43:04.372 INFO:tasks.ceph.ceph_manager.ceph:PG 8.25 is not active+clean 2024-05-08T05:43:04.372 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.565197+0000', 'last_change': '2024-05-08T05:42:58.565197+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.565197+0000', 'last_undegraded': '2024-05-08T05:42:58.565197+0000', 'last_fullsized': '2024-05-08T05:42:58.565197+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T05:43:04.373 INFO:tasks.ceph.ceph_manager.ceph:PG 8.23 is not active+clean 2024-05-08T05:43:04.373 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.562110+0000', 'last_change': '2024-05-08T05:42:58.562110+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.562110+0000', 'last_undegraded': '2024-05-08T05:42:58.562110+0000', 'last_fullsized': '2024-05-08T05:42:58.562110+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T05:43:04.373 INFO:tasks.ceph.ceph_manager.ceph:PG 8.21 is not active+clean 2024-05-08T05:43:04.373 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.562320+0000', 'last_change': '2024-05-08T05:42:58.562320+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.562320+0000', 'last_undegraded': '2024-05-08T05:42:58.562320+0000', 'last_fullsized': '2024-05-08T05:42:58.562320+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T05:43:04.373 INFO:tasks.ceph.ceph_manager.ceph:PG 8.13 is not active+clean 2024-05-08T05:43:04.373 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.567678+0000', 'last_change': '2024-05-08T05:42:58.567678+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.567678+0000', 'last_undegraded': '2024-05-08T05:42:58.567678+0000', 'last_fullsized': '2024-05-08T05:42:58.567678+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T05:43:04.373 INFO:tasks.ceph.ceph_manager.ceph:PG 8.14 is not active+clean 2024-05-08T05:43:04.373 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.558663+0000', 'last_change': '2024-05-08T05:42:58.558663+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.558663+0000', 'last_undegraded': '2024-05-08T05:42:58.558663+0000', 'last_fullsized': '2024-05-08T05:42:58.558663+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T05:43:04.373 INFO:tasks.ceph.ceph_manager.ceph:PG 8.16 is not active+clean 2024-05-08T05:43:04.373 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.565142+0000', 'last_change': '2024-05-08T05:42:58.565142+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.565142+0000', 'last_undegraded': '2024-05-08T05:42:58.565142+0000', 'last_fullsized': '2024-05-08T05:42:58.565142+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T05:43:04.373 INFO:tasks.ceph.ceph_manager.ceph:PG 8.18 is not active+clean 2024-05-08T05:43:04.374 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.558457+0000', 'last_change': '2024-05-08T05:42:58.558457+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.558457+0000', 'last_undegraded': '2024-05-08T05:42:58.558457+0000', 'last_fullsized': '2024-05-08T05:42:58.558457+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T05:43:04.374 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1a is not active+clean 2024-05-08T05:43:04.374 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.561535+0000', 'last_change': '2024-05-08T05:42:58.561535+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.561535+0000', 'last_undegraded': '2024-05-08T05:42:58.561535+0000', 'last_fullsized': '2024-05-08T05:42:58.561535+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T05:43:04.374 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1b is not active+clean 2024-05-08T05:43:04.374 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.558448+0000', 'last_change': '2024-05-08T05:42:58.558448+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.558448+0000', 'last_undegraded': '2024-05-08T05:42:58.558448+0000', 'last_fullsized': '2024-05-08T05:42:58.558448+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T05:43:04.374 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1d is not active+clean 2024-05-08T05:43:04.374 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.564425+0000', 'last_change': '2024-05-08T05:42:58.564425+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.564425+0000', 'last_undegraded': '2024-05-08T05:42:58.564425+0000', 'last_fullsized': '2024-05-08T05:42:58.564425+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T05:43:04.374 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1e is not active+clean 2024-05-08T05:43:04.374 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.565577+0000', 'last_change': '2024-05-08T05:42:58.565577+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.565577+0000', 'last_undegraded': '2024-05-08T05:42:58.565577+0000', 'last_fullsized': '2024-05-08T05:42:58.565577+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T05:43:04.375 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1f is not active+clean 2024-05-08T05:43:04.375 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.567393+0000', 'last_change': '2024-05-08T05:42:58.567393+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.567393+0000', 'last_undegraded': '2024-05-08T05:42:58.567393+0000', 'last_fullsized': '2024-05-08T05:42:58.567393+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T05:43:04.375 INFO:tasks.ceph.ceph_manager.ceph:PG 8.0 is not active+clean 2024-05-08T05:43:04.375 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.564368+0000', 'last_change': '2024-05-08T05:42:58.564368+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.564368+0000', 'last_undegraded': '2024-05-08T05:42:58.564368+0000', 'last_fullsized': '2024-05-08T05:42:58.564368+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T05:43:04.375 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1 is not active+clean 2024-05-08T05:43:04.375 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.561728+0000', 'last_change': '2024-05-08T05:42:58.561728+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.561728+0000', 'last_undegraded': '2024-05-08T05:42:58.561728+0000', 'last_fullsized': '2024-05-08T05:42:58.561728+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T05:43:04.375 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2 is not active+clean 2024-05-08T05:43:04.375 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.558840+0000', 'last_change': '2024-05-08T05:42:58.558840+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.558840+0000', 'last_undegraded': '2024-05-08T05:42:58.558840+0000', 'last_fullsized': '2024-05-08T05:42:58.558840+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-05-08T05:43:04.375 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3 is not active+clean 2024-05-08T05:43:04.375 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.562629+0000', 'last_change': '2024-05-08T05:42:58.562629+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.562629+0000', 'last_undegraded': '2024-05-08T05:42:58.562629+0000', 'last_fullsized': '2024-05-08T05:42:58.562629+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T05:43:04.375 INFO:tasks.ceph.ceph_manager.ceph:PG 8.6 is not active+clean 2024-05-08T05:43:04.375 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.563623+0000', 'last_change': '2024-05-08T05:42:58.563623+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.563623+0000', 'last_undegraded': '2024-05-08T05:42:58.563623+0000', 'last_fullsized': '2024-05-08T05:42:58.563623+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T05:43:04.376 INFO:tasks.ceph.ceph_manager.ceph:PG 8.7 is not active+clean 2024-05-08T05:43:04.376 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.566556+0000', 'last_change': '2024-05-08T05:42:58.566556+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.566556+0000', 'last_undegraded': '2024-05-08T05:42:58.566556+0000', 'last_fullsized': '2024-05-08T05:42:58.566556+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T05:43:04.376 INFO:tasks.ceph.ceph_manager.ceph:PG 8.a is not active+clean 2024-05-08T05:43:04.376 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.562862+0000', 'last_change': '2024-05-08T05:42:58.562862+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.562862+0000', 'last_undegraded': '2024-05-08T05:42:58.562862+0000', 'last_fullsized': '2024-05-08T05:42:58.562862+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-05-08T05:43:04.376 INFO:tasks.ceph.ceph_manager.ceph:PG 8.b is not active+clean 2024-05-08T05:43:04.376 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.567229+0000', 'last_change': '2024-05-08T05:42:58.567229+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.567229+0000', 'last_undegraded': '2024-05-08T05:42:58.567229+0000', 'last_fullsized': '2024-05-08T05:42:58.567229+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T05:43:04.376 INFO:tasks.ceph.ceph_manager.ceph:PG 8.c is not active+clean 2024-05-08T05:43:04.376 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.563979+0000', 'last_change': '2024-05-08T05:42:58.563979+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.563979+0000', 'last_undegraded': '2024-05-08T05:42:58.563979+0000', 'last_fullsized': '2024-05-08T05:42:58.563979+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T05:43:04.376 INFO:tasks.ceph.ceph_manager.ceph:PG 8.e is not active+clean 2024-05-08T05:43:04.376 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.564637+0000', 'last_change': '2024-05-08T05:42:58.564637+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.564637+0000', 'last_undegraded': '2024-05-08T05:42:58.564637+0000', 'last_fullsized': '2024-05-08T05:42:58.564637+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-05-08T05:43:04.377 INFO:tasks.ceph.ceph_manager.ceph:PG 8.f is not active+clean 2024-05-08T05:43:04.377 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:42:58.566087+0000', 'last_change': '2024-05-08T05:42:58.566087+0000', 'last_active': '2024-05-08T05:42:58.552624+0000', 'last_peered': '2024-05-08T05:42:58.552624+0000', 'last_clean': '2024-05-08T05:42:58.552624+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:42:58.566087+0000', 'last_undegraded': '2024-05-08T05:42:58.566087+0000', 'last_fullsized': '2024-05-08T05:42:58.566087+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'last_clean_scrub_stamp': '2024-05-08T05:42:58.552624+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-05-08T05:43:04.377 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T05:43:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.507+0000 7f3562989640 1 Processor -- start 2024-05-08T05:43:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.507+0000 7f3562989640 1 -- start start 2024-05-08T05:43:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.508+0000 7f3562989640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f355c1606d0 0x7f355c0cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.508+0000 7f3562989640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f355c0cdf00 0x7f355c0ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.508+0000 7f3562989640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f355c0d15c0 con 0x7f355c0cf580 2024-05-08T05:43:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.508+0000 7f3562989640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f355c0ce8a0 con 0x7f355c0cdf00 2024-05-08T05:43:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.508+0000 7f3562989640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f355c0d18a0 con 0x7f355c1606d0 2024-05-08T05:43:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.508+0000 7f355b7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f355c0cdf00 0x7f355c0ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.508+0000 7f3560f26640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f355c0cf580 0x7f355c0cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37452/0 (socket says 172.21.15.45:37452) 2024-05-08T05:43:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.508+0000 7f3560f26640 1 -- 172.21.15.45:0/141198583 learned_addr learned my addr 172.21.15.45:0/141198583 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.508+0000 7f355bfff640 1 --2- 172.21.15.45:0/141198583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f355c1606d0 0x7f355c0cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.508+0000 7f355b7fe640 1 -- 172.21.15.45:0/141198583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f355c1606d0 msgr2=0x7f355c0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.508+0000 7f355b7fe640 1 --2- 172.21.15.45:0/141198583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f355c1606d0 0x7f355c0cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.508+0000 7f355affd640 1 -- 172.21.15.45:0/141198583 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3181009170 0 0) 0x7f355c0d15c0 con 0x7f355c0cf580 2024-05-08T05:43:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.508+0000 7f355b7fe640 1 -- 172.21.15.45:0/141198583 >> v1:172.21.15.45:6789/0 conn(0x7f355c0cf580 legacy=0x7f355c0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.508+0000 7f355b7fe640 1 -- 172.21.15.45:0/141198583 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355c0d1b80 con 0x7f355c0cdf00 2024-05-08T05:43:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.508+0000 7f355bfff640 1 --2- 172.21.15.45:0/141198583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f355c1606d0 0x7f355c0cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.509+0000 7f355b7fe640 1 --2- 172.21.15.45:0/141198583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c0cdf00 0x7f355c0ce360 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f35500097a0 tx=0x7f355002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745de9033163e51f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.509+0000 7f355affd640 1 -- 172.21.15.45:0/141198583 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550037070 con 0x7f355c0cdf00 2024-05-08T05:43:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.509+0000 7f355affd640 1 -- 172.21.15.45:0/141198583 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f355002adb0 con 0x7f355c0cdf00 2024-05-08T05:43:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.509+0000 7f355affd640 1 -- 172.21.15.45:0/141198583 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550032660 con 0x7f355c0cdf00 2024-05-08T05:43:04.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.510+0000 7f3562989640 1 -- 172.21.15.45:0/141198583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c0cdf00 msgr2=0x7f355c0ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:04.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.510+0000 7f3562989640 1 --2- 172.21.15.45:0/141198583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c0cdf00 0x7f355c0ce360 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f35500097a0 tx=0x7f355002a060 comp rx=0 tx=0).stop 2024-05-08T05:43:04.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.510+0000 7f3562989640 1 -- 172.21.15.45:0/141198583 shutdown_connections 2024-05-08T05:43:04.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.510+0000 7f3562989640 1 --2- 172.21.15.45:0/141198583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c0cdf00 0x7f355c0ce360 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.510+0000 7f3562989640 1 --2- 172.21.15.45:0/141198583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f355c1606d0 0x7f355c0cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.510+0000 7f3562989640 1 -- 172.21.15.45:0/141198583 >> 172.21.15.45:0/141198583 conn(0x7f355c0b49d0 msgr2=0x7f355c0b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:04.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.510+0000 7f3562989640 1 -- 172.21.15.45:0/141198583 shutdown_connections 2024-05-08T05:43:04.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.510+0000 7f3562989640 1 -- 172.21.15.45:0/141198583 wait complete. 2024-05-08T05:43:04.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.511+0000 7f3562989640 1 Processor -- start 2024-05-08T05:43:04.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.511+0000 7f3562989640 1 -- start start 2024-05-08T05:43:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.512+0000 7f3562989640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f355c0cdf00 0x7f355c14f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.512+0000 7f3562989640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c0cf580 0x7f355c159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.512+0000 7f3562989640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f355c1606d0 0x7f355c0cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.512+0000 7f3562989640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f355c071a00 con 0x7f355c0cf580 2024-05-08T05:43:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.512+0000 7f3562989640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f355c0b5710 con 0x7f355c1606d0 2024-05-08T05:43:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.512+0000 7f3562989640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f355c071b80 con 0x7f355c0cdf00 2024-05-08T05:43:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.512+0000 7f355b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c0cf580 0x7f355c159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.512+0000 7f355b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c0cf580 0x7f355c159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47400/0 (socket says 172.21.15.45:47400) 2024-05-08T05:43:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.512+0000 7f355b7fe640 1 -- 172.21.15.45:0/2770224922 learned_addr learned my addr 172.21.15.45:0/2770224922 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.512+0000 7f355bfff640 1 --2- 172.21.15.45:0/2770224922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f355c0cdf00 0x7f355c14f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.512+0000 7f3560f26640 1 --2- 172.21.15.45:0/2770224922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f355c1606d0 0x7f355c0cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.512+0000 7f355b7fe640 1 -- 172.21.15.45:0/2770224922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f355c0cdf00 msgr2=0x7f355c14f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.512+0000 7f355b7fe640 1 --2- 172.21.15.45:0/2770224922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f355c0cdf00 0x7f355c14f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.512+0000 7f355b7fe640 1 -- 172.21.15.45:0/2770224922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f355c1606d0 msgr2=0x7f355c0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.512+0000 7f355b7fe640 1 --2- 172.21.15.45:0/2770224922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f355c1606d0 0x7f355c0cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.512+0000 7f355b7fe640 1 -- 172.21.15.45:0/2770224922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355c14f7e0 con 0x7f355c0cf580 2024-05-08T05:43:04.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.512+0000 7f3560f26640 1 --2- 172.21.15.45:0/2770224922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f355c1606d0 0x7f355c0cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:04.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.512+0000 7f355b7fe640 1 --2- 172.21.15.45:0/2770224922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c0cf580 0x7f355c159e20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f35500097a0 tx=0x7f35500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:04.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.513+0000 7f3558ff9640 1 -- 172.21.15.45:0/2770224922 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550037040 con 0x7f355c0cf580 2024-05-08T05:43:04.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.513+0000 7f3558ff9640 1 -- 172.21.15.45:0/2770224922 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3550004250 con 0x7f355c0cf580 2024-05-08T05:43:04.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.513+0000 7f3562989640 1 -- 172.21.15.45:0/2770224922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355c15a750 con 0x7f355c0cf580 2024-05-08T05:43:04.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.513+0000 7f3558ff9640 1 -- 172.21.15.45:0/2770224922 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550003610 con 0x7f355c0cf580 2024-05-08T05:43:04.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.513+0000 7f3562989640 1 -- 172.21.15.45:0/2770224922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f355c15ad10 con 0x7f355c0cf580 2024-05-08T05:43:04.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.514+0000 7f3562989640 1 -- 172.21.15.45:0/2770224922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f355c0693f0 con 0x7f355c0cf580 2024-05-08T05:43:04.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.514+0000 7f3558ff9640 1 -- 172.21.15.45:0/2770224922 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f355002ac70 con 0x7f355c0cf580 2024-05-08T05:43:04.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.515+0000 7f3558ff9640 1 --2- 172.21.15.45:0/2770224922 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f35480762d0 0x7f3548078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:04.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.515+0000 7f3558ff9640 1 -- 172.21.15.45:0/2770224922 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7f35500b8640 con 0x7f355c0cf580 2024-05-08T05:43:04.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.515+0000 7f355bfff640 1 --2- 172.21.15.45:0/2770224922 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f35480762d0 0x7f3548078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:04.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.515+0000 7f355bfff640 1 --2- 172.21.15.45:0/2770224922 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f35480762d0 0x7f3548078790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f354c00ab60 tx=0x7f354c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:04.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.519+0000 7f3558ff9640 1 -- 172.21.15.45:0/2770224922 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3550081d80 con 0x7f355c0cf580 2024-05-08T05:43:04.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.785+0000 7f3562989640 1 -- 172.21.15.45:0/2770224922 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f355c0b21e0 con 0x7f355c0cf580 2024-05-08T05:43:04.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.785+0000 7f3558ff9640 1 -- 172.21.15.45:0/2770224922 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1645 (secure 0 0 0) 0x7f3550081680 con 0x7f355c0cf580 2024-05-08T05:43:04.787 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:04.787 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":116,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":8,"num_up_osds":8,"osd_up_since":1715146878,"num_in_osds":8,"osd_in_since":1715146870,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":54},{"state_name":"creating+peering","count":43}],"num_pgs":97,"num_pools":3,"num_objects":9,"data_bytes":591110,"bytes_used":243834880,"bytes_avail":772850278400,"bytes_total":773094113280,"inactive_pgs_ratio":0.44329896569252014,"write_bytes_sec":579,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":26,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"b","status":"up:active","gid":5292}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-05-08T05:42:46.850710+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T05:43:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.787+0000 7f3562989640 1 -- 172.21.15.45:0/2770224922 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f35480762d0 msgr2=0x7f3548078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.787+0000 7f3562989640 1 --2- 172.21.15.45:0/2770224922 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f35480762d0 0x7f3548078790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f354c00ab60 tx=0x7f354c008040 comp rx=0 tx=0).stop 2024-05-08T05:43:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.787+0000 7f3562989640 1 -- 172.21.15.45:0/2770224922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c0cf580 msgr2=0x7f355c159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.787+0000 7f3562989640 1 --2- 172.21.15.45:0/2770224922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c0cf580 0x7f355c159e20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f35500097a0 tx=0x7f35500046c0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.787+0000 7f3562989640 1 -- 172.21.15.45:0/2770224922 shutdown_connections 2024-05-08T05:43:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.787+0000 7f3562989640 1 --2- 172.21.15.45:0/2770224922 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f35480762d0 0x7f3548078790 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.787+0000 7f3562989640 1 --2- 172.21.15.45:0/2770224922 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f355c1606d0 0x7f355c0cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.787+0000 7f3562989640 1 --2- 172.21.15.45:0/2770224922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f355c0cf580 0x7f355c159e20 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.787+0000 7f3562989640 1 --2- 172.21.15.45:0/2770224922 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f355c0cdf00 0x7f355c14f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.787+0000 7f3562989640 1 -- 172.21.15.45:0/2770224922 >> 172.21.15.45:0/2770224922 conn(0x7f355c0b49d0 msgr2=0x7f355c0b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.787+0000 7f3562989640 1 -- 172.21.15.45:0/2770224922 shutdown_connections 2024-05-08T05:43:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.787+0000 7f3562989640 1 -- 172.21.15.45:0/2770224922 wait complete. 2024-05-08T05:43:04.800 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-08T05:43:04.800 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:43:04.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.903+0000 7f3a93d58640 1 Processor -- start 2024-05-08T05:43:04.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.903+0000 7f3a93d58640 1 -- start start 2024-05-08T05:43:04.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.904+0000 7f3a93d58640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a8c1629b0 0x7f3a8c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:04.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.904+0000 7f3a93d58640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3a8c0b2490 0x7f3a8c070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:04.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.904+0000 7f3a93d58640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a8c06e110 con 0x7f3a8c070d20 2024-05-08T05:43:04.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.904+0000 7f3a93d58640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f3a8c06dec0 con 0x7f3a8c0b2490 2024-05-08T05:43:04.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.904+0000 7f3a93d58640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3a8c06e3f0 con 0x7f3a8c1629b0 2024-05-08T05:43:04.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.904+0000 7f3a912f3640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3a8c0b2490 0x7f3a8c070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:04.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.904+0000 7f3a912f3640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3a8c0b2490 0x7f3a8c070330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47410/0 (socket says 172.21.15.45:47410) 2024-05-08T05:43:04.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.904+0000 7f3a912f3640 1 -- 172.21.15.45:0/2210225830 learned_addr learned my addr 172.21.15.45:0/2210225830 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:04.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.904+0000 7f3a912f3640 1 -- 172.21.15.45:0/2210225830 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a8c1629b0 msgr2=0x7f3a8c06eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:04.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.904+0000 7f3a912f3640 1 --2- 172.21.15.45:0/2210225830 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a8c1629b0 0x7f3a8c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.904+0000 7f3a912f3640 1 -- 172.21.15.45:0/2210225830 >> v1:172.21.15.63:6789/0 conn(0x7f3a8c070d20 legacy=0x7f3a8c0afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:04.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.905+0000 7f3a912f3640 1 -- 172.21.15.45:0/2210225830 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8c06e6d0 con 0x7f3a8c0b2490 2024-05-08T05:43:04.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.905+0000 7f3a912f3640 1 --2- 172.21.15.45:0/2210225830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8c0b2490 0x7f3a8c070330 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009870 tx=0x7f3a7c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fcf8498c2d8fe49 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.905+0000 7f3a90af2640 1 -- 172.21.15.45:0/2210225830 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c037070 con 0x7f3a8c0b2490 2024-05-08T05:43:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.905+0000 7f3a90af2640 1 -- 172.21.15.45:0/2210225830 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a7c032440 con 0x7f3a8c0b2490 2024-05-08T05:43:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.905+0000 7f3a90af2640 1 -- 172.21.15.45:0/2210225830 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c032810 con 0x7f3a8c0b2490 2024-05-08T05:43:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.906+0000 7f3a93d58640 1 -- 172.21.15.45:0/2210225830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8c0b2490 msgr2=0x7f3a8c070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.906+0000 7f3a93d58640 1 --2- 172.21.15.45:0/2210225830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8c0b2490 0x7f3a8c070330 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009870 tx=0x7f3a7c02a1c0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.906+0000 7f3a93d58640 1 -- 172.21.15.45:0/2210225830 shutdown_connections 2024-05-08T05:43:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.906+0000 7f3a93d58640 1 --2- 172.21.15.45:0/2210225830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8c0b2490 0x7f3a8c070330 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.906+0000 7f3a93d58640 1 --2- 172.21.15.45:0/2210225830 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a8c1629b0 0x7f3a8c06eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.906+0000 7f3a93d58640 1 -- 172.21.15.45:0/2210225830 >> 172.21.15.45:0/2210225830 conn(0x7f3a8c0b49d0 msgr2=0x7f3a8c0b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.906+0000 7f3a93d58640 1 -- 172.21.15.45:0/2210225830 shutdown_connections 2024-05-08T05:43:04.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.906+0000 7f3a93d58640 1 -- 172.21.15.45:0/2210225830 wait complete. 2024-05-08T05:43:04.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.907+0000 7f3a93d58640 1 Processor -- start 2024-05-08T05:43:04.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.907+0000 7f3a93d58640 1 -- start start 2024-05-08T05:43:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.908+0000 7f3a93d58640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a8c070d20 0x7f3a8c15b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.908+0000 7f3a93d58640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8c0b2490 0x7f3a8c0b7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.908+0000 7f3a93d58640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a8c1629b0 0x7f3a8c0b7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.908+0000 7f3a93d58640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c070870 con 0x7f3a8c0b2490 2024-05-08T05:43:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.908+0000 7f3a93d58640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c06f410 con 0x7f3a8c1629b0 2024-05-08T05:43:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.908+0000 7f3a93d58640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3a8c0709f0 con 0x7f3a8c070d20 2024-05-08T05:43:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.908+0000 7f3a912f3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8c0b2490 0x7f3a8c0b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.908+0000 7f3a912f3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8c0b2490 0x7f3a8c0b7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47414/0 (socket says 172.21.15.45:47414) 2024-05-08T05:43:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.908+0000 7f3a912f3640 1 -- 172.21.15.45:0/3913703535 learned_addr learned my addr 172.21.15.45:0/3913703535 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:04.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.908+0000 7f3a912f3640 1 -- 172.21.15.45:0/3913703535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a8c070d20 msgr2=0x7f3a8c15b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:04.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.908+0000 7f3a912f3640 1 --2- 172.21.15.45:0/3913703535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a8c070d20 0x7f3a8c15b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.908+0000 7f3a912f3640 1 -- 172.21.15.45:0/3913703535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a8c1629b0 msgr2=0x7f3a8c0b7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:04.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.908+0000 7f3a912f3640 1 --2- 172.21.15.45:0/3913703535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a8c1629b0 0x7f3a8c0b7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:04.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.908+0000 7f3a912f3640 1 -- 172.21.15.45:0/3913703535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8c15c410 con 0x7f3a8c0b2490 2024-05-08T05:43:04.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.909+0000 7f3a912f3640 1 --2- 172.21.15.45:0/3913703535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8c0b2490 0x7f3a8c0b7780 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009870 tx=0x7f3a7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:04.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.909+0000 7f3a827fc640 1 -- 172.21.15.45:0/3913703535 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c041070 con 0x7f3a8c0b2490 2024-05-08T05:43:04.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.909+0000 7f3a827fc640 1 -- 172.21.15.45:0/3913703535 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a7c03c870 con 0x7f3a8c0b2490 2024-05-08T05:43:04.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.909+0000 7f3a93d58640 1 -- 172.21.15.45:0/3913703535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8c0b5bd0 con 0x7f3a8c0b2490 2024-05-08T05:43:04.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.909+0000 7f3a827fc640 1 -- 172.21.15.45:0/3913703535 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c037040 con 0x7f3a8c0b2490 2024-05-08T05:43:04.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.909+0000 7f3a93d58640 1 -- 172.21.15.45:0/3913703535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a8c0b6190 con 0x7f3a8c0b2490 2024-05-08T05:43:04.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.910+0000 7f3a93d58640 1 -- 172.21.15.45:0/3913703535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a8c0693f0 con 0x7f3a8c0b2490 2024-05-08T05:43:04.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.910+0000 7f3a827fc640 1 -- 172.21.15.45:0/3913703535 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3a7c044070 con 0x7f3a8c0b2490 2024-05-08T05:43:04.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.911+0000 7f3a827fc640 1 --2- 172.21.15.45:0/3913703535 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3a700762d0 0x7f3a70078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:04.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.911+0000 7f3a827fc640 1 -- 172.21.15.45:0/3913703535 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7f3a7c0b8300 con 0x7f3a8c0b2490 2024-05-08T05:43:04.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.911+0000 7f3a91af4640 1 --2- 172.21.15.45:0/3913703535 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3a700762d0 0x7f3a70078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:04.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.912+0000 7f3a91af4640 1 --2- 172.21.15.45:0/3913703535 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3a700762d0 0x7f3a70078790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009670 tx=0x7f3a88006bf0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:04.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:04.915+0000 7f3a827fc640 1 -- 172.21.15.45:0/3913703535 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3a7c081a40 con 0x7f3a8c0b2490 2024-05-08T05:43:05.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:05.108+0000 7f3a93d58640 1 -- 172.21.15.45:0/3913703535 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3a8c0b82f0 con 0x7f3a700762d0 2024-05-08T05:43:05.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:05.116+0000 7f3a827fc640 1 -- 172.21.15.45:0/3913703535 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251912 (secure 0 0 0) 0x7f3a8c0b82f0 con 0x7f3a700762d0 2024-05-08T05:43:05.119 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:05.121 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:43:05.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:05.118+0000 7f3a93d58640 1 -- 172.21.15.45:0/3913703535 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3a700762d0 msgr2=0x7f3a70078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:05.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:05.118+0000 7f3a93d58640 1 --2- 172.21.15.45:0/3913703535 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3a700762d0 0x7f3a70078790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009670 tx=0x7f3a88006bf0 comp rx=0 tx=0).stop 2024-05-08T05:43:05.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:05.118+0000 7f3a93d58640 1 -- 172.21.15.45:0/3913703535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8c0b2490 msgr2=0x7f3a8c0b7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:05.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:05.118+0000 7f3a93d58640 1 --2- 172.21.15.45:0/3913703535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8c0b2490 0x7f3a8c0b7780 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009870 tx=0x7f3a7c0046c0 comp rx=0 tx=0).stop 2024-05-08T05:43:05.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:05.118+0000 7f3a93d58640 1 -- 172.21.15.45:0/3913703535 shutdown_connections 2024-05-08T05:43:05.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:05.118+0000 7f3a93d58640 1 --2- 172.21.15.45:0/3913703535 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3a700762d0 0x7f3a70078790 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:05.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:05.118+0000 7f3a93d58640 1 --2- 172.21.15.45:0/3913703535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a8c1629b0 0x7f3a8c0b7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:05.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:05.118+0000 7f3a93d58640 1 --2- 172.21.15.45:0/3913703535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a8c0b2490 0x7f3a8c0b7780 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:05.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:05.118+0000 7f3a93d58640 1 --2- 172.21.15.45:0/3913703535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a8c070d20 0x7f3a8c15b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:05.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:05.118+0000 7f3a93d58640 1 -- 172.21.15.45:0/3913703535 >> 172.21.15.45:0/3913703535 conn(0x7f3a8c0b49d0 msgr2=0x7f3a8c161cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:05.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:05.118+0000 7f3a93d58640 1 -- 172.21.15.45:0/3913703535 shutdown_connections 2024-05-08T05:43:05.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:05.118+0000 7f3a93d58640 1 -- 172.21.15.45:0/3913703535 wait complete. 2024-05-08T05:43:05.131 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-05-08T05:43:04.854092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238344,"kb_used_data":5512,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736376,"statfs":{"total":773094113280,"available":772850049024,"internally_reserved":0,"allocated":5644288,"data_stored":3107437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.297988"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162378+0000","last_change":"2024-05-08T05:42:59.579678+0000","last_active":"2024-05-08T05:43:01.162378+0000","last_peered":"2024-05-08T05:43:01.162378+0000","last_clean":"2024-05-08T05:43:01.162378+0000","last_became_active":"2024-05-08T05:42:59.579353+0000","last_became_peered":"2024-05-08T05:42:59.579353+0000","last_unstale":"2024-05-08T05:43:01.162378+0000","last_undegraded":"2024-05-08T05:43:01.162378+0000","last_fullsized":"2024-05-08T05:43:01.162378+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:26.624115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.590291+0000","last_change":"2024-05-08T05:42:59.586491+0000","last_active":"2024-05-08T05:42:59.590291+0000","last_peered":"2024-05-08T05:42:59.590291+0000","last_clean":"2024-05-08T05:42:59.590291+0000","last_became_active":"2024-05-08T05:42:59.586233+0000","last_became_peered":"2024-05-08T05:42:59.586233+0000","last_unstale":"2024-05-08T05:42:59.590291+0000","last_undegraded":"2024-05-08T05:42:59.590291+0000","last_fullsized":"2024-05-08T05:42:59.590291+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:32.677283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585174+0000","last_change":"2024-05-08T05:42:59.590747+0000","last_active":"2024-05-08T05:43:00.585174+0000","last_peered":"2024-05-08T05:43:00.585174+0000","last_clean":"2024-05-08T05:43:00.585174+0000","last_became_active":"2024-05-08T05:42:59.590436+0000","last_became_peered":"2024-05-08T05:42:59.590436+0000","last_unstale":"2024-05-08T05:43:00.585174+0000","last_undegraded":"2024-05-08T05:43:00.585174+0000","last_fullsized":"2024-05-08T05:43:00.585174+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:41:55.376598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.595331+0000","last_change":"2024-05-08T05:42:59.589298+0000","last_active":"2024-05-08T05:42:59.595331+0000","last_peered":"2024-05-08T05:42:59.595331+0000","last_clean":"2024-05-08T05:42:59.595331+0000","last_became_active":"2024-05-08T05:42:59.588989+0000","last_became_peered":"2024-05-08T05:42:59.588989+0000","last_unstale":"2024-05-08T05:42:59.595331+0000","last_undegraded":"2024-05-08T05:42:59.595331+0000","last_fullsized":"2024-05-08T05:42:59.595331+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:54:26.126199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.886354+0000","last_change":"2024-05-08T05:42:59.589282+0000","last_active":"2024-05-08T05:43:00.886354+0000","last_peered":"2024-05-08T05:43:00.886354+0000","last_clean":"2024-05-08T05:43:00.886354+0000","last_became_active":"2024-05-08T05:42:59.589004+0000","last_became_peered":"2024-05-08T05:42:59.589004+0000","last_unstale":"2024-05-08T05:43:00.886354+0000","last_undegraded":"2024-05-08T05:43:00.886354+0000","last_fullsized":"2024-05-08T05:43:00.886354+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:05:27.975200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162111+0000","last_change":"2024-05-08T05:42:59.584415+0000","last_active":"2024-05-08T05:43:01.162111+0000","last_peered":"2024-05-08T05:43:01.162111+0000","last_clean":"2024-05-08T05:43:01.162111+0000","last_became_active":"2024-05-08T05:42:59.584129+0000","last_became_peered":"2024-05-08T05:42:59.584129+0000","last_unstale":"2024-05-08T05:43:01.162111+0000","last_undegraded":"2024-05-08T05:43:01.162111+0000","last_fullsized":"2024-05-08T05:43:01.162111+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:05.059317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.584713+0000","last_change":"2024-05-08T05:42:59.589541+0000","last_active":"2024-05-08T05:43:00.584713+0000","last_peered":"2024-05-08T05:43:00.584713+0000","last_clean":"2024-05-08T05:43:00.584713+0000","last_became_active":"2024-05-08T05:42:59.588332+0000","last_became_peered":"2024-05-08T05:42:59.588332+0000","last_unstale":"2024-05-08T05:43:00.584713+0000","last_undegraded":"2024-05-08T05:43:00.584713+0000","last_fullsized":"2024-05-08T05:43:00.584713+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:47:11.811597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582165+0000","last_change":"2024-05-08T05:42:59.591871+0000","last_active":"2024-05-08T05:43:00.582165+0000","last_peered":"2024-05-08T05:43:00.582165+0000","last_clean":"2024-05-08T05:43:00.582165+0000","last_became_active":"2024-05-08T05:42:59.591612+0000","last_became_peered":"2024-05-08T05:42:59.591612+0000","last_unstale":"2024-05-08T05:43:00.582165+0000","last_undegraded":"2024-05-08T05:43:00.582165+0000","last_fullsized":"2024-05-08T05:43:00.582165+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:38.824072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582601+0000","last_change":"2024-05-08T05:42:59.593793+0000","last_active":"2024-05-08T05:43:00.582601+0000","last_peered":"2024-05-08T05:43:00.582601+0000","last_clean":"2024-05-08T05:43:00.582601+0000","last_became_active":"2024-05-08T05:42:59.592651+0000","last_became_peered":"2024-05-08T05:42:59.592651+0000","last_unstale":"2024-05-08T05:43:00.582601+0000","last_undegraded":"2024-05-08T05:43:00.582601+0000","last_fullsized":"2024-05-08T05:43:00.582601+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:13:24.282730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162260+0000","last_change":"2024-05-08T05:42:59.584754+0000","last_active":"2024-05-08T05:43:01.162260+0000","last_peered":"2024-05-08T05:43:01.162260+0000","last_clean":"2024-05-08T05:43:01.162260+0000","last_became_active":"2024-05-08T05:42:59.584449+0000","last_became_peered":"2024-05-08T05:42:59.584449+0000","last_unstale":"2024-05-08T05:43:01.162260+0000","last_undegraded":"2024-05-08T05:43:01.162260+0000","last_fullsized":"2024-05-08T05:43:01.162260+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:29.071179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162131+0000","last_change":"2024-05-08T05:42:59.582974+0000","last_active":"2024-05-08T05:43:01.162131+0000","last_peered":"2024-05-08T05:43:01.162131+0000","last_clean":"2024-05-08T05:43:01.162131+0000","last_became_active":"2024-05-08T05:42:59.582674+0000","last_became_peered":"2024-05-08T05:42:59.582674+0000","last_unstale":"2024-05-08T05:43:01.162131+0000","last_undegraded":"2024-05-08T05:43:01.162131+0000","last_fullsized":"2024-05-08T05:43:01.162131+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:49:51.317831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582635+0000","last_change":"2024-05-08T05:42:59.596052+0000","last_active":"2024-05-08T05:43:00.582635+0000","last_peered":"2024-05-08T05:43:00.582635+0000","last_clean":"2024-05-08T05:43:00.582635+0000","last_became_active":"2024-05-08T05:42:59.595755+0000","last_became_peered":"2024-05-08T05:42:59.595755+0000","last_unstale":"2024-05-08T05:43:00.582635+0000","last_undegraded":"2024-05-08T05:43:00.582635+0000","last_fullsized":"2024-05-08T05:43:00.582635+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:29.901922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162437+0000","last_change":"2024-05-08T05:42:59.585199+0000","last_active":"2024-05-08T05:43:01.162437+0000","last_peered":"2024-05-08T05:43:01.162437+0000","last_clean":"2024-05-08T05:43:01.162437+0000","last_became_active":"2024-05-08T05:42:59.584917+0000","last_became_peered":"2024-05-08T05:42:59.584917+0000","last_unstale":"2024-05-08T05:43:01.162437+0000","last_undegraded":"2024-05-08T05:43:01.162437+0000","last_fullsized":"2024-05-08T05:43:01.162437+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:23:18.816427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.584961+0000","last_change":"2024-05-08T05:42:59.600179+0000","last_active":"2024-05-08T05:43:00.584961+0000","last_peered":"2024-05-08T05:43:00.584961+0000","last_clean":"2024-05-08T05:43:00.584961+0000","last_became_active":"2024-05-08T05:42:59.599853+0000","last_became_peered":"2024-05-08T05:42:59.599853+0000","last_unstale":"2024-05-08T05:43:00.584961+0000","last_undegraded":"2024-05-08T05:43:00.584961+0000","last_fullsized":"2024-05-08T05:43:00.584961+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:19:42.624642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.598708+0000","last_change":"2024-05-08T05:42:59.593026+0000","last_active":"2024-05-08T05:42:59.598708+0000","last_peered":"2024-05-08T05:42:59.598708+0000","last_clean":"2024-05-08T05:42:59.598708+0000","last_became_active":"2024-05-08T05:42:59.592706+0000","last_became_peered":"2024-05-08T05:42:59.592706+0000","last_unstale":"2024-05-08T05:42:59.598708+0000","last_undegraded":"2024-05-08T05:42:59.598708+0000","last_fullsized":"2024-05-08T05:42:59.598708+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:34.093891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.161882+0000","last_change":"2024-05-08T05:42:59.584032+0000","last_active":"2024-05-08T05:43:01.161882+0000","last_peered":"2024-05-08T05:43:01.161882+0000","last_clean":"2024-05-08T05:43:01.161882+0000","last_became_active":"2024-05-08T05:42:59.583732+0000","last_became_peered":"2024-05-08T05:42:59.583732+0000","last_unstale":"2024-05-08T05:43:01.161882+0000","last_undegraded":"2024-05-08T05:43:01.161882+0000","last_fullsized":"2024-05-08T05:43:01.161882+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:24:56.356577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585564+0000","last_change":"2024-05-08T05:42:59.597808+0000","last_active":"2024-05-08T05:43:00.585564+0000","last_peered":"2024-05-08T05:43:00.585564+0000","last_clean":"2024-05-08T05:43:00.585564+0000","last_became_active":"2024-05-08T05:42:59.597515+0000","last_became_peered":"2024-05-08T05:42:59.597515+0000","last_unstale":"2024-05-08T05:43:00.585564+0000","last_undegraded":"2024-05-08T05:43:00.585564+0000","last_fullsized":"2024-05-08T05:43:00.585564+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:42:14.163907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.598545+0000","last_change":"2024-05-08T05:42:59.592278+0000","last_active":"2024-05-08T05:42:59.598545+0000","last_peered":"2024-05-08T05:42:59.598545+0000","last_clean":"2024-05-08T05:42:59.598545+0000","last_became_active":"2024-05-08T05:42:59.591985+0000","last_became_peered":"2024-05-08T05:42:59.591985+0000","last_unstale":"2024-05-08T05:42:59.598545+0000","last_undegraded":"2024-05-08T05:42:59.598545+0000","last_fullsized":"2024-05-08T05:42:59.598545+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:42:12.700183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585218+0000","last_change":"2024-05-08T05:42:59.601023+0000","last_active":"2024-05-08T05:43:00.585218+0000","last_peered":"2024-05-08T05:43:00.585218+0000","last_clean":"2024-05-08T05:43:00.585218+0000","last_became_active":"2024-05-08T05:42:59.600730+0000","last_became_peered":"2024-05-08T05:42:59.600730+0000","last_unstale":"2024-05-08T05:43:00.585218+0000","last_undegraded":"2024-05-08T05:43:00.585218+0000","last_fullsized":"2024-05-08T05:43:00.585218+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:17:08.488947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162202+0000","last_change":"2024-05-08T05:42:59.584649+0000","last_active":"2024-05-08T05:43:01.162202+0000","last_peered":"2024-05-08T05:43:01.162202+0000","last_clean":"2024-05-08T05:43:01.162202+0000","last_became_active":"2024-05-08T05:42:59.584233+0000","last_became_peered":"2024-05-08T05:42:59.584233+0000","last_unstale":"2024-05-08T05:43:01.162202+0000","last_undegraded":"2024-05-08T05:43:01.162202+0000","last_fullsized":"2024-05-08T05:43:01.162202+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:02:16.483573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162694+0000","last_change":"2024-05-08T05:42:59.591062+0000","last_active":"2024-05-08T05:43:01.162694+0000","last_peered":"2024-05-08T05:43:01.162694+0000","last_clean":"2024-05-08T05:43:01.162694+0000","last_became_active":"2024-05-08T05:42:59.590680+0000","last_became_peered":"2024-05-08T05:42:59.590680+0000","last_unstale":"2024-05-08T05:43:01.162694+0000","last_undegraded":"2024-05-08T05:43:01.162694+0000","last_fullsized":"2024-05-08T05:43:01.162694+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:24:56.356577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582069+0000","last_change":"2024-05-08T05:42:59.594863+0000","last_active":"2024-05-08T05:43:00.582069+0000","last_peered":"2024-05-08T05:43:00.582069+0000","last_clean":"2024-05-08T05:43:00.582069+0000","last_became_active":"2024-05-08T05:42:59.594599+0000","last_became_peered":"2024-05-08T05:42:59.594599+0000","last_unstale":"2024-05-08T05:43:00.582069+0000","last_undegraded":"2024-05-08T05:43:00.582069+0000","last_fullsized":"2024-05-08T05:43:00.582069+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:17:08.488947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.161776+0000","last_change":"2024-05-08T05:42:59.584918+0000","last_active":"2024-05-08T05:43:01.161776+0000","last_peered":"2024-05-08T05:43:01.161776+0000","last_clean":"2024-05-08T05:43:01.161776+0000","last_became_active":"2024-05-08T05:42:59.584635+0000","last_became_peered":"2024-05-08T05:42:59.584635+0000","last_unstale":"2024-05-08T05:43:01.161776+0000","last_undegraded":"2024-05-08T05:43:01.161776+0000","last_fullsized":"2024-05-08T05:43:01.161776+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:48.297871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.584650+0000","last_change":"2024-05-08T05:42:59.591296+0000","last_active":"2024-05-08T05:43:00.584650+0000","last_peered":"2024-05-08T05:43:00.584650+0000","last_clean":"2024-05-08T05:43:00.584650+0000","last_became_active":"2024-05-08T05:42:59.590937+0000","last_became_peered":"2024-05-08T05:42:59.590937+0000","last_unstale":"2024-05-08T05:43:00.584650+0000","last_undegraded":"2024-05-08T05:43:00.584650+0000","last_fullsized":"2024-05-08T05:43:00.584650+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:34.093891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585378+0000","last_change":"2024-05-08T05:42:59.589656+0000","last_active":"2024-05-08T05:43:00.585378+0000","last_peered":"2024-05-08T05:43:00.585378+0000","last_clean":"2024-05-08T05:43:00.585378+0000","last_became_active":"2024-05-08T05:42:59.589351+0000","last_became_peered":"2024-05-08T05:42:59.589351+0000","last_unstale":"2024-05-08T05:43:00.585378+0000","last_undegraded":"2024-05-08T05:43:00.585378+0000","last_fullsized":"2024-05-08T05:43:00.585378+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:38.824072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582306+0000","last_change":"2024-05-08T05:42:59.584164+0000","last_active":"2024-05-08T05:43:00.582306+0000","last_peered":"2024-05-08T05:43:00.582306+0000","last_clean":"2024-05-08T05:43:00.582306+0000","last_became_active":"2024-05-08T05:42:59.583903+0000","last_became_peered":"2024-05-08T05:42:59.583903+0000","last_unstale":"2024-05-08T05:43:00.582306+0000","last_undegraded":"2024-05-08T05:43:00.582306+0000","last_fullsized":"2024-05-08T05:43:00.582306+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:42:14.163907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162515+0000","last_change":"2024-05-08T05:42:59.591847+0000","last_active":"2024-05-08T05:43:01.162515+0000","last_peered":"2024-05-08T05:43:01.162515+0000","last_clean":"2024-05-08T05:43:01.162515+0000","last_became_active":"2024-05-08T05:42:59.591557+0000","last_became_peered":"2024-05-08T05:42:59.591557+0000","last_unstale":"2024-05-08T05:43:01.162515+0000","last_undegraded":"2024-05-08T05:43:01.162515+0000","last_fullsized":"2024-05-08T05:43:01.162515+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:09.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162857+0000","last_change":"2024-05-08T05:42:59.582258+0000","last_active":"2024-05-08T05:43:01.162857+0000","last_peered":"2024-05-08T05:43:01.162857+0000","last_clean":"2024-05-08T05:43:01.162857+0000","last_became_active":"2024-05-08T05:42:59.581973+0000","last_became_peered":"2024-05-08T05:42:59.581973+0000","last_unstale":"2024-05-08T05:43:01.162857+0000","last_undegraded":"2024-05-08T05:43:01.162857+0000","last_fullsized":"2024-05-08T05:43:01.162857+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:05.059317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.584837+0000","last_change":"2024-05-08T05:42:59.593351+0000","last_active":"2024-05-08T05:43:00.584837+0000","last_peered":"2024-05-08T05:43:00.584837+0000","last_clean":"2024-05-08T05:43:00.584837+0000","last_became_active":"2024-05-08T05:42:59.593077+0000","last_became_peered":"2024-05-08T05:42:59.593077+0000","last_unstale":"2024-05-08T05:43:00.584837+0000","last_undegraded":"2024-05-08T05:43:00.584837+0000","last_fullsized":"2024-05-08T05:43:00.584837+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:08.244250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.590188+0000","last_change":"2024-05-08T05:42:59.583375+0000","last_active":"2024-05-08T05:42:59.590188+0000","last_peered":"2024-05-08T05:42:59.590188+0000","last_clean":"2024-05-08T05:42:59.590188+0000","last_became_active":"2024-05-08T05:42:59.583026+0000","last_became_peered":"2024-05-08T05:42:59.583026+0000","last_unstale":"2024-05-08T05:42:59.590188+0000","last_undegraded":"2024-05-08T05:42:59.590188+0000","last_fullsized":"2024-05-08T05:42:59.590188+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:09.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.584784+0000","last_change":"2024-05-08T05:42:59.589277+0000","last_active":"2024-05-08T05:43:00.584784+0000","last_peered":"2024-05-08T05:43:00.584784+0000","last_clean":"2024-05-08T05:43:00.584784+0000","last_became_active":"2024-05-08T05:42:59.588987+0000","last_became_peered":"2024-05-08T05:42:59.588987+0000","last_unstale":"2024-05-08T05:43:00.584784+0000","last_undegraded":"2024-05-08T05:43:00.584784+0000","last_fullsized":"2024-05-08T05:43:00.584784+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:54.934881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162579+0000","last_change":"2024-05-08T05:42:59.586488+0000","last_active":"2024-05-08T05:43:01.162579+0000","last_peered":"2024-05-08T05:43:01.162579+0000","last_clean":"2024-05-08T05:43:01.162579+0000","last_became_active":"2024-05-08T05:42:59.585806+0000","last_became_peered":"2024-05-08T05:42:59.585806+0000","last_unstale":"2024-05-08T05:43:01.162579+0000","last_undegraded":"2024-05-08T05:43:01.162579+0000","last_fullsized":"2024-05-08T05:43:01.162579+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:39:15.656632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.598596+0000","last_change":"2024-05-08T05:42:59.593948+0000","last_active":"2024-05-08T05:42:59.598596+0000","last_peered":"2024-05-08T05:42:59.598596+0000","last_clean":"2024-05-08T05:42:59.598596+0000","last_became_active":"2024-05-08T05:42:59.593663+0000","last_became_peered":"2024-05-08T05:42:59.593663+0000","last_unstale":"2024-05-08T05:42:59.598596+0000","last_undegraded":"2024-05-08T05:42:59.598596+0000","last_fullsized":"2024-05-08T05:42:59.598596+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:47:11.811597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"46'4","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162826+0000","last_change":"2024-05-08T05:42:58.568449+0000","last_active":"2024-05-08T05:43:01.162826+0000","last_peered":"2024-05-08T05:43:01.162826+0000","last_clean":"2024-05-08T05:43:01.162826+0000","last_became_active":"2024-05-08T05:42:58.567378+0000","last_became_peered":"2024-05-08T05:42:58.567378+0000","last_unstale":"2024-05-08T05:43:01.162826+0000","last_undegraded":"2024-05-08T05:43:01.162826+0000","last_fullsized":"2024-05-08T05:43:01.162826+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:04.957071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162401+0000","last_change":"2024-05-08T05:42:59.583673+0000","last_active":"2024-05-08T05:43:01.162401+0000","last_peered":"2024-05-08T05:43:01.162401+0000","last_clean":"2024-05-08T05:43:01.162401+0000","last_became_active":"2024-05-08T05:42:59.583430+0000","last_became_peered":"2024-05-08T05:42:59.583430+0000","last_unstale":"2024-05-08T05:43:01.162401+0000","last_undegraded":"2024-05-08T05:43:01.162401+0000","last_fullsized":"2024-05-08T05:43:01.162401+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:54.934881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162234+0000","last_change":"2024-05-08T05:42:58.582169+0000","last_active":"2024-05-08T05:43:01.162234+0000","last_peered":"2024-05-08T05:43:01.162234+0000","last_clean":"2024-05-08T05:43:01.162234+0000","last_became_active":"2024-05-08T05:42:58.581851+0000","last_became_peered":"2024-05-08T05:42:58.581851+0000","last_unstale":"2024-05-08T05:43:01.162234+0000","last_undegraded":"2024-05-08T05:43:01.162234+0000","last_fullsized":"2024-05-08T05:43:01.162234+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:32.556335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162026+0000","last_change":"2024-05-08T05:42:59.585789+0000","last_active":"2024-05-08T05:43:01.162026+0000","last_peered":"2024-05-08T05:43:01.162026+0000","last_clean":"2024-05-08T05:43:01.162026+0000","last_became_active":"2024-05-08T05:42:59.585486+0000","last_became_peered":"2024-05-08T05:42:59.585486+0000","last_unstale":"2024-05-08T05:43:01.162026+0000","last_undegraded":"2024-05-08T05:43:01.162026+0000","last_fullsized":"2024-05-08T05:43:01.162026+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:32:52.739108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162618+0000","last_change":"2024-05-08T05:42:58.577001+0000","last_active":"2024-05-08T05:43:01.162618+0000","last_peered":"2024-05-08T05:43:01.162618+0000","last_clean":"2024-05-08T05:43:01.162618+0000","last_became_active":"2024-05-08T05:42:58.575665+0000","last_became_peered":"2024-05-08T05:42:58.575665+0000","last_unstale":"2024-05-08T05:43:01.162618+0000","last_undegraded":"2024-05-08T05:43:01.162618+0000","last_fullsized":"2024-05-08T05:43:01.162618+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:46.384674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582385+0000","last_change":"2024-05-08T05:42:59.583002+0000","last_active":"2024-05-08T05:43:00.582385+0000","last_peered":"2024-05-08T05:43:00.582385+0000","last_clean":"2024-05-08T05:43:00.582385+0000","last_became_active":"2024-05-08T05:42:59.582323+0000","last_became_peered":"2024-05-08T05:42:59.582323+0000","last_unstale":"2024-05-08T05:43:00.582385+0000","last_undegraded":"2024-05-08T05:43:00.582385+0000","last_fullsized":"2024-05-08T05:43:00.582385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:15:08.074347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162613+0000","last_change":"2024-05-08T05:42:58.573567+0000","last_active":"2024-05-08T05:43:01.162613+0000","last_peered":"2024-05-08T05:43:01.162613+0000","last_clean":"2024-05-08T05:43:01.162613+0000","last_became_active":"2024-05-08T05:42:58.573220+0000","last_became_peered":"2024-05-08T05:42:58.573220+0000","last_unstale":"2024-05-08T05:43:01.162613+0000","last_undegraded":"2024-05-08T05:43:01.162613+0000","last_fullsized":"2024-05-08T05:43:01.162613+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:15.376533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.886460+0000","last_change":"2024-05-08T05:42:59.590972+0000","last_active":"2024-05-08T05:43:00.886460+0000","last_peered":"2024-05-08T05:43:00.886460+0000","last_clean":"2024-05-08T05:43:00.886460+0000","last_became_active":"2024-05-08T05:42:59.590667+0000","last_became_peered":"2024-05-08T05:42:59.590667+0000","last_unstale":"2024-05-08T05:43:00.886460+0000","last_undegraded":"2024-05-08T05:43:00.886460+0000","last_fullsized":"2024-05-08T05:43:00.886460+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:27.802918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585060+0000","last_change":"2024-05-08T05:42:58.572908+0000","last_active":"2024-05-08T05:43:00.585060+0000","last_peered":"2024-05-08T05:43:00.585060+0000","last_clean":"2024-05-08T05:43:00.585060+0000","last_became_active":"2024-05-08T05:42:58.572613+0000","last_became_peered":"2024-05-08T05:42:58.572613+0000","last_unstale":"2024-05-08T05:43:00.585060+0000","last_undegraded":"2024-05-08T05:43:00.585060+0000","last_fullsized":"2024-05-08T05:43:00.585060+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:28:40.992183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.598156+0000","last_change":"2024-05-08T05:42:59.589512+0000","last_active":"2024-05-08T05:42:59.598156+0000","last_peered":"2024-05-08T05:42:59.598156+0000","last_clean":"2024-05-08T05:42:59.598156+0000","last_became_active":"2024-05-08T05:42:59.588973+0000","last_became_peered":"2024-05-08T05:42:59.588973+0000","last_unstale":"2024-05-08T05:42:59.598156+0000","last_undegraded":"2024-05-08T05:42:59.598156+0000","last_fullsized":"2024-05-08T05:42:59.598156+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:34:10.738939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.886034+0000","last_change":"2024-05-08T05:42:58.569466+0000","last_active":"2024-05-08T05:43:00.886034+0000","last_peered":"2024-05-08T05:43:00.886034+0000","last_clean":"2024-05-08T05:43:00.886034+0000","last_became_active":"2024-05-08T05:42:58.569048+0000","last_became_peered":"2024-05-08T05:42:58.569048+0000","last_unstale":"2024-05-08T05:43:00.886034+0000","last_undegraded":"2024-05-08T05:43:00.886034+0000","last_fullsized":"2024-05-08T05:43:00.886034+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:28.357714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582595+0000","last_change":"2024-05-08T05:42:59.585222+0000","last_active":"2024-05-08T05:43:00.582595+0000","last_peered":"2024-05-08T05:43:00.582595+0000","last_clean":"2024-05-08T05:43:00.582595+0000","last_became_active":"2024-05-08T05:42:59.583480+0000","last_became_peered":"2024-05-08T05:42:59.583480+0000","last_unstale":"2024-05-08T05:43:00.582595+0000","last_undegraded":"2024-05-08T05:43:00.582595+0000","last_fullsized":"2024-05-08T05:43:00.582595+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:01:32.094856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.161928+0000","last_change":"2024-05-08T05:42:58.568749+0000","last_active":"2024-05-08T05:43:01.161928+0000","last_peered":"2024-05-08T05:43:01.161928+0000","last_clean":"2024-05-08T05:43:01.161928+0000","last_became_active":"2024-05-08T05:42:58.568473+0000","last_became_peered":"2024-05-08T05:42:58.568473+0000","last_unstale":"2024-05-08T05:43:01.161928+0000","last_undegraded":"2024-05-08T05:43:01.161928+0000","last_fullsized":"2024-05-08T05:43:01.161928+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:55:31.695068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.598506+0000","last_change":"2024-05-08T05:42:59.591341+0000","last_active":"2024-05-08T05:42:59.598506+0000","last_peered":"2024-05-08T05:42:59.598506+0000","last_clean":"2024-05-08T05:42:59.598506+0000","last_became_active":"2024-05-08T05:42:59.591050+0000","last_became_peered":"2024-05-08T05:42:59.591050+0000","last_unstale":"2024-05-08T05:42:59.598506+0000","last_undegraded":"2024-05-08T05:42:59.598506+0000","last_fullsized":"2024-05-08T05:42:59.598506+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:46.820598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.584665+0000","last_change":"2024-05-08T05:42:58.574070+0000","last_active":"2024-05-08T05:43:00.584665+0000","last_peered":"2024-05-08T05:43:00.584665+0000","last_clean":"2024-05-08T05:43:00.584665+0000","last_became_active":"2024-05-08T05:42:58.573769+0000","last_became_peered":"2024-05-08T05:42:58.573769+0000","last_unstale":"2024-05-08T05:43:00.584665+0000","last_undegraded":"2024-05-08T05:43:00.584665+0000","last_fullsized":"2024-05-08T05:43:00.584665+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:45:37.264655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.886152+0000","last_change":"2024-05-08T05:42:59.584451+0000","last_active":"2024-05-08T05:43:00.886152+0000","last_peered":"2024-05-08T05:43:00.886152+0000","last_clean":"2024-05-08T05:43:00.886152+0000","last_became_active":"2024-05-08T05:42:59.584194+0000","last_became_peered":"2024-05-08T05:42:59.584194+0000","last_unstale":"2024-05-08T05:43:00.886152+0000","last_undegraded":"2024-05-08T05:43:00.886152+0000","last_fullsized":"2024-05-08T05:43:00.886152+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:28:42.000118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.580407+0000","last_change":"2024-05-08T05:42:58.567761+0000","last_active":"2024-05-08T05:42:59.580407+0000","last_peered":"2024-05-08T05:42:59.580407+0000","last_clean":"2024-05-08T05:42:59.580407+0000","last_became_active":"2024-05-08T05:42:58.567490+0000","last_became_peered":"2024-05-08T05:42:58.567490+0000","last_unstale":"2024-05-08T05:42:59.580407+0000","last_undegraded":"2024-05-08T05:42:59.580407+0000","last_fullsized":"2024-05-08T05:42:59.580407+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:23:17.808492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162214+0000","last_change":"2024-05-08T05:42:59.585553+0000","last_active":"2024-05-08T05:43:01.162214+0000","last_peered":"2024-05-08T05:43:01.162214+0000","last_clean":"2024-05-08T05:43:01.162214+0000","last_became_active":"2024-05-08T05:42:59.584903+0000","last_became_peered":"2024-05-08T05:42:59.584903+0000","last_unstale":"2024-05-08T05:43:01.162214+0000","last_undegraded":"2024-05-08T05:43:01.162214+0000","last_fullsized":"2024-05-08T05:43:01.162214+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:34.093891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.886480+0000","last_change":"2024-05-08T05:42:58.574888+0000","last_active":"2024-05-08T05:43:00.886480+0000","last_peered":"2024-05-08T05:43:00.886480+0000","last_clean":"2024-05-08T05:43:00.886480+0000","last_became_active":"2024-05-08T05:42:58.574585+0000","last_became_peered":"2024-05-08T05:42:58.574585+0000","last_unstale":"2024-05-08T05:43:00.886480+0000","last_undegraded":"2024-05-08T05:43:00.886480+0000","last_fullsized":"2024-05-08T05:43:00.886480+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:55:31.695068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585036+0000","last_change":"2024-05-08T05:42:59.594208+0000","last_active":"2024-05-08T05:43:00.585036+0000","last_peered":"2024-05-08T05:43:00.585036+0000","last_clean":"2024-05-08T05:43:00.585036+0000","last_became_active":"2024-05-08T05:42:59.593886+0000","last_became_peered":"2024-05-08T05:42:59.593886+0000","last_unstale":"2024-05-08T05:43:00.585036+0000","last_undegraded":"2024-05-08T05:43:00.585036+0000","last_fullsized":"2024-05-08T05:43:00.585036+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:25:17.636288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"46'3","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.601464+0000","last_change":"2024-05-08T05:42:58.572885+0000","last_active":"2024-05-08T05:42:59.601464+0000","last_peered":"2024-05-08T05:42:59.601464+0000","last_clean":"2024-05-08T05:42:59.601464+0000","last_became_active":"2024-05-08T05:42:58.571751+0000","last_became_peered":"2024-05-08T05:42:58.571751+0000","last_unstale":"2024-05-08T05:42:59.601464+0000","last_undegraded":"2024-05-08T05:42:59.601464+0000","last_fullsized":"2024-05-08T05:42:59.601464+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:15.376533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.886353+0000","last_change":"2024-05-08T05:42:59.583118+0000","last_active":"2024-05-08T05:43:00.886353+0000","last_peered":"2024-05-08T05:43:00.886353+0000","last_clean":"2024-05-08T05:43:00.886353+0000","last_became_active":"2024-05-08T05:42:59.582673+0000","last_became_peered":"2024-05-08T05:42:59.582673+0000","last_unstale":"2024-05-08T05:43:00.886353+0000","last_undegraded":"2024-05-08T05:43:00.886353+0000","last_fullsized":"2024-05-08T05:43:00.886353+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:17.523257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"46'6","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582673+0000","last_change":"2024-05-08T05:42:58.578675+0000","last_active":"2024-05-08T05:43:00.582673+0000","last_peered":"2024-05-08T05:43:00.582673+0000","last_clean":"2024-05-08T05:43:00.582673+0000","last_became_active":"2024-05-08T05:42:58.578130+0000","last_became_peered":"2024-05-08T05:42:58.578130+0000","last_unstale":"2024-05-08T05:43:00.582673+0000","last_undegraded":"2024-05-08T05:43:00.582673+0000","last_fullsized":"2024-05-08T05:43:00.582673+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:31.669348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162379+0000","last_change":"2024-05-08T05:42:59.583647+0000","last_active":"2024-05-08T05:43:01.162379+0000","last_peered":"2024-05-08T05:43:01.162379+0000","last_clean":"2024-05-08T05:43:01.162379+0000","last_became_active":"2024-05-08T05:42:59.583302+0000","last_became_peered":"2024-05-08T05:42:59.583302+0000","last_unstale":"2024-05-08T05:43:01.162379+0000","last_undegraded":"2024-05-08T05:43:01.162379+0000","last_fullsized":"2024-05-08T05:43:01.162379+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:47:11.811597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585095+0000","last_change":"2024-05-08T05:42:58.576622+0000","last_active":"2024-05-08T05:43:00.585095+0000","last_peered":"2024-05-08T05:43:00.585095+0000","last_clean":"2024-05-08T05:43:00.585095+0000","last_became_active":"2024-05-08T05:42:58.575446+0000","last_became_peered":"2024-05-08T05:42:58.575446+0000","last_unstale":"2024-05-08T05:43:00.585095+0000","last_undegraded":"2024-05-08T05:43:00.585095+0000","last_fullsized":"2024-05-08T05:43:00.585095+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:05:26.967265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585284+0000","last_change":"2024-05-08T05:42:59.590992+0000","last_active":"2024-05-08T05:43:00.585284+0000","last_peered":"2024-05-08T05:43:00.585284+0000","last_clean":"2024-05-08T05:43:00.585284+0000","last_became_active":"2024-05-08T05:42:59.590614+0000","last_became_peered":"2024-05-08T05:42:59.590614+0000","last_unstale":"2024-05-08T05:43:00.585284+0000","last_undegraded":"2024-05-08T05:43:00.585284+0000","last_fullsized":"2024-05-08T05:43:00.585284+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:55.193274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162146+0000","last_change":"2024-05-08T05:42:58.574983+0000","last_active":"2024-05-08T05:43:01.162146+0000","last_peered":"2024-05-08T05:43:01.162146+0000","last_clean":"2024-05-08T05:43:01.162146+0000","last_became_active":"2024-05-08T05:42:58.574678+0000","last_became_peered":"2024-05-08T05:42:58.574678+0000","last_unstale":"2024-05-08T05:43:01.162146+0000","last_undegraded":"2024-05-08T05:43:01.162146+0000","last_fullsized":"2024-05-08T05:43:01.162146+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:28.357714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582451+0000","last_change":"2024-05-08T05:42:59.589300+0000","last_active":"2024-05-08T05:43:00.582451+0000","last_peered":"2024-05-08T05:43:00.582451+0000","last_clean":"2024-05-08T05:43:00.582451+0000","last_became_active":"2024-05-08T05:42:59.589015+0000","last_became_peered":"2024-05-08T05:42:59.589015+0000","last_unstale":"2024-05-08T05:43:00.582451+0000","last_undegraded":"2024-05-08T05:43:00.582451+0000","last_fullsized":"2024-05-08T05:43:00.582451+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:29:32.097989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.577776+0000","last_change":"2024-05-08T05:42:58.572724+0000","last_active":"2024-05-08T05:42:59.577776+0000","last_peered":"2024-05-08T05:42:59.577776+0000","last_clean":"2024-05-08T05:42:59.577776+0000","last_became_active":"2024-05-08T05:42:58.572471+0000","last_became_peered":"2024-05-08T05:42:58.572471+0000","last_unstale":"2024-05-08T05:42:59.577776+0000","last_undegraded":"2024-05-08T05:42:59.577776+0000","last_fullsized":"2024-05-08T05:42:59.577776+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:28.063244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585408+0000","last_change":"2024-05-08T05:42:59.594817+0000","last_active":"2024-05-08T05:43:00.585408+0000","last_peered":"2024-05-08T05:43:00.585408+0000","last_clean":"2024-05-08T05:43:00.585408+0000","last_became_active":"2024-05-08T05:42:59.594548+0000","last_became_peered":"2024-05-08T05:42:59.594548+0000","last_unstale":"2024-05-08T05:43:00.585408+0000","last_undegraded":"2024-05-08T05:43:00.585408+0000","last_fullsized":"2024-05-08T05:43:00.585408+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:39:37.911504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"46'8","reported_seq":22,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582259+0000","last_change":"2024-05-08T05:42:58.577909+0000","last_active":"2024-05-08T05:43:00.582259+0000","last_peered":"2024-05-08T05:43:00.582259+0000","last_clean":"2024-05-08T05:43:00.582259+0000","last_became_active":"2024-05-08T05:42:58.577585+0000","last_became_peered":"2024-05-08T05:42:58.577585+0000","last_unstale":"2024-05-08T05:43:00.582259+0000","last_undegraded":"2024-05-08T05:43:00.582259+0000","last_fullsized":"2024-05-08T05:43:00.582259+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:45.812663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582185+0000","last_change":"2024-05-08T05:42:59.581991+0000","last_active":"2024-05-08T05:43:00.582185+0000","last_peered":"2024-05-08T05:43:00.582185+0000","last_clean":"2024-05-08T05:43:00.582185+0000","last_became_active":"2024-05-08T05:42:59.581642+0000","last_became_peered":"2024-05-08T05:42:59.581642+0000","last_unstale":"2024-05-08T05:43:00.582185+0000","last_undegraded":"2024-05-08T05:43:00.582185+0000","last_fullsized":"2024-05-08T05:43:00.582185+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:39:37.911504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585430+0000","last_change":"2024-05-08T05:42:58.579816+0000","last_active":"2024-05-08T05:43:00.585430+0000","last_peered":"2024-05-08T05:43:00.585430+0000","last_clean":"2024-05-08T05:43:00.585430+0000","last_became_active":"2024-05-08T05:42:58.579567+0000","last_became_peered":"2024-05-08T05:42:58.579567+0000","last_unstale":"2024-05-08T05:43:00.585430+0000","last_undegraded":"2024-05-08T05:43:00.585430+0000","last_fullsized":"2024-05-08T05:43:00.585430+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:12.539768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.161857+0000","last_change":"2024-05-08T05:42:59.591106+0000","last_active":"2024-05-08T05:43:01.161857+0000","last_peered":"2024-05-08T05:43:01.161857+0000","last_clean":"2024-05-08T05:43:01.161857+0000","last_became_active":"2024-05-08T05:42:59.590815+0000","last_became_peered":"2024-05-08T05:42:59.590815+0000","last_unstale":"2024-05-08T05:43:01.161857+0000","last_undegraded":"2024-05-08T05:43:01.161857+0000","last_fullsized":"2024-05-08T05:43:01.161857+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:41:14.761007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162350+0000","last_change":"2024-05-08T05:42:58.568207+0000","last_active":"2024-05-08T05:43:01.162350+0000","last_peered":"2024-05-08T05:43:01.162350+0000","last_clean":"2024-05-08T05:43:01.162350+0000","last_became_active":"2024-05-08T05:42:58.567932+0000","last_became_peered":"2024-05-08T05:42:58.567932+0000","last_unstale":"2024-05-08T05:43:01.162350+0000","last_undegraded":"2024-05-08T05:43:01.162350+0000","last_fullsized":"2024-05-08T05:43:01.162350+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:33:11.854881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.886121+0000","last_change":"2024-05-08T05:42:59.591776+0000","last_active":"2024-05-08T05:43:00.886121+0000","last_peered":"2024-05-08T05:43:00.886121+0000","last_clean":"2024-05-08T05:43:00.886121+0000","last_became_active":"2024-05-08T05:42:59.591492+0000","last_became_peered":"2024-05-08T05:42:59.591492+0000","last_unstale":"2024-05-08T05:43:00.886121+0000","last_undegraded":"2024-05-08T05:43:00.886121+0000","last_fullsized":"2024-05-08T05:43:00.886121+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:13.547702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162886+0000","last_change":"2024-05-08T05:42:58.573572+0000","last_active":"2024-05-08T05:43:01.162886+0000","last_peered":"2024-05-08T05:43:01.162886+0000","last_clean":"2024-05-08T05:43:01.162886+0000","last_became_active":"2024-05-08T05:42:58.573275+0000","last_became_peered":"2024-05-08T05:42:58.573275+0000","last_unstale":"2024-05-08T05:43:01.162886+0000","last_undegraded":"2024-05-08T05:43:01.162886+0000","last_fullsized":"2024-05-08T05:43:01.162886+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:23:53.968037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162578+0000","last_change":"2024-05-08T05:42:59.575200+0000","last_active":"2024-05-08T05:43:01.162578+0000","last_peered":"2024-05-08T05:43:01.162578+0000","last_clean":"2024-05-08T05:43:01.162578+0000","last_became_active":"2024-05-08T05:42:59.574903+0000","last_became_peered":"2024-05-08T05:42:59.574903+0000","last_unstale":"2024-05-08T05:43:01.162578+0000","last_undegraded":"2024-05-08T05:43:01.162578+0000","last_fullsized":"2024-05-08T05:43:01.162578+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:54:26.126199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162882+0000","last_change":"2024-05-08T05:42:58.579945+0000","last_active":"2024-05-08T05:43:01.162882+0000","last_peered":"2024-05-08T05:43:01.162882+0000","last_clean":"2024-05-08T05:43:01.162882+0000","last_became_active":"2024-05-08T05:42:58.579657+0000","last_became_peered":"2024-05-08T05:42:58.579657+0000","last_unstale":"2024-05-08T05:43:01.162882+0000","last_undegraded":"2024-05-08T05:43:01.162882+0000","last_fullsized":"2024-05-08T05:43:01.162882+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:46.384674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162460+0000","last_change":"2024-05-08T05:42:59.585604+0000","last_active":"2024-05-08T05:43:01.162460+0000","last_peered":"2024-05-08T05:43:01.162460+0000","last_clean":"2024-05-08T05:43:01.162460+0000","last_became_active":"2024-05-08T05:42:59.585027+0000","last_became_peered":"2024-05-08T05:42:59.585027+0000","last_unstale":"2024-05-08T05:43:01.162460+0000","last_undegraded":"2024-05-08T05:43:01.162460+0000","last_fullsized":"2024-05-08T05:43:01.162460+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:08.244250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162570+0000","last_change":"2024-05-08T05:42:58.574363+0000","last_active":"2024-05-08T05:43:01.162570+0000","last_peered":"2024-05-08T05:43:01.162570+0000","last_clean":"2024-05-08T05:43:01.162570+0000","last_became_active":"2024-05-08T05:42:58.574101+0000","last_became_peered":"2024-05-08T05:42:58.574101+0000","last_unstale":"2024-05-08T05:43:01.162570+0000","last_undegraded":"2024-05-08T05:43:01.162570+0000","last_fullsized":"2024-05-08T05:43:01.162570+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:04.957071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162665+0000","last_change":"2024-05-08T05:42:59.587267+0000","last_active":"2024-05-08T05:43:01.162665+0000","last_peered":"2024-05-08T05:43:01.162665+0000","last_clean":"2024-05-08T05:43:01.162665+0000","last_became_active":"2024-05-08T05:42:59.586824+0000","last_became_peered":"2024-05-08T05:42:59.586824+0000","last_unstale":"2024-05-08T05:43:01.162665+0000","last_undegraded":"2024-05-08T05:43:01.162665+0000","last_fullsized":"2024-05-08T05:43:01.162665+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:41:14.761007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.581992+0000","last_change":"2024-05-08T05:42:58.572523+0000","last_active":"2024-05-08T05:43:00.581992+0000","last_peered":"2024-05-08T05:43:00.581992+0000","last_clean":"2024-05-08T05:43:00.581992+0000","last_became_active":"2024-05-08T05:42:58.572193+0000","last_became_peered":"2024-05-08T05:42:58.572193+0000","last_unstale":"2024-05-08T05:43:00.581992+0000","last_undegraded":"2024-05-08T05:43:00.581992+0000","last_fullsized":"2024-05-08T05:43:00.581992+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:34:09.731004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585033+0000","last_change":"2024-05-08T05:42:59.593271+0000","last_active":"2024-05-08T05:43:00.585033+0000","last_peered":"2024-05-08T05:43:00.585033+0000","last_clean":"2024-05-08T05:43:00.585033+0000","last_became_active":"2024-05-08T05:42:59.592988+0000","last_became_peered":"2024-05-08T05:42:59.592988+0000","last_unstale":"2024-05-08T05:43:00.585033+0000","last_undegraded":"2024-05-08T05:43:00.585033+0000","last_fullsized":"2024-05-08T05:43:00.585033+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:39:15.656632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.584839+0000","last_change":"2024-05-08T05:42:58.582112+0000","last_active":"2024-05-08T05:43:00.584839+0000","last_peered":"2024-05-08T05:43:00.584839+0000","last_clean":"2024-05-08T05:43:00.584839+0000","last_became_active":"2024-05-08T05:42:58.581779+0000","last_became_peered":"2024-05-08T05:42:58.581779+0000","last_unstale":"2024-05-08T05:43:00.584839+0000","last_undegraded":"2024-05-08T05:43:00.584839+0000","last_fullsized":"2024-05-08T05:43:00.584839+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:48.397348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162554+0000","last_change":"2024-05-08T05:42:59.584992+0000","last_active":"2024-05-08T05:43:01.162554+0000","last_peered":"2024-05-08T05:43:01.162554+0000","last_clean":"2024-05-08T05:43:01.162554+0000","last_became_active":"2024-05-08T05:42:59.584681+0000","last_became_peered":"2024-05-08T05:42:59.584681+0000","last_unstale":"2024-05-08T05:43:01.162554+0000","last_undegraded":"2024-05-08T05:43:01.162554+0000","last_fullsized":"2024-05-08T05:43:01.162554+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:32.677283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.161686+0000","last_change":"2024-05-08T05:42:58.568551+0000","last_active":"2024-05-08T05:43:01.161686+0000","last_peered":"2024-05-08T05:43:01.161686+0000","last_clean":"2024-05-08T05:43:01.161686+0000","last_became_active":"2024-05-08T05:42:58.568282+0000","last_became_peered":"2024-05-08T05:42:58.568282+0000","last_unstale":"2024-05-08T05:43:01.161686+0000","last_undegraded":"2024-05-08T05:43:01.161686+0000","last_fullsized":"2024-05-08T05:43:01.161686+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:34:46.120453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162305+0000","last_change":"2024-05-08T05:42:59.586138+0000","last_active":"2024-05-08T05:43:01.162305+0000","last_peered":"2024-05-08T05:43:01.162305+0000","last_clean":"2024-05-08T05:43:01.162305+0000","last_became_active":"2024-05-08T05:42:59.585739+0000","last_became_peered":"2024-05-08T05:42:59.585739+0000","last_unstale":"2024-05-08T05:43:01.162305+0000","last_undegraded":"2024-05-08T05:43:01.162305+0000","last_fullsized":"2024-05-08T05:43:01.162305+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:20:43.774846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162942+0000","last_change":"2024-05-08T05:42:58.580617+0000","last_active":"2024-05-08T05:43:01.162942+0000","last_peered":"2024-05-08T05:43:01.162942+0000","last_clean":"2024-05-08T05:43:01.162942+0000","last_became_active":"2024-05-08T05:42:58.580371+0000","last_became_peered":"2024-05-08T05:42:58.580371+0000","last_unstale":"2024-05-08T05:43:01.162942+0000","last_undegraded":"2024-05-08T05:43:01.162942+0000","last_fullsized":"2024-05-08T05:43:01.162942+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:32.556335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162155+0000","last_change":"2024-05-08T05:42:59.580372+0000","last_active":"2024-05-08T05:43:01.162155+0000","last_peered":"2024-05-08T05:43:01.162155+0000","last_clean":"2024-05-08T05:43:01.162155+0000","last_became_active":"2024-05-08T05:42:59.579909+0000","last_became_peered":"2024-05-08T05:42:59.579909+0000","last_unstale":"2024-05-08T05:43:01.162155+0000","last_undegraded":"2024-05-08T05:43:01.162155+0000","last_fullsized":"2024-05-08T05:43:01.162155+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:16.384468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.578285+0000","last_change":"2024-05-08T05:42:58.570248+0000","last_active":"2024-05-08T05:42:59.578285+0000","last_peered":"2024-05-08T05:42:59.578285+0000","last_clean":"2024-05-08T05:42:59.578285+0000","last_became_active":"2024-05-08T05:42:58.569966+0000","last_became_peered":"2024-05-08T05:42:58.569966+0000","last_unstale":"2024-05-08T05:42:59.578285+0000","last_undegraded":"2024-05-08T05:42:59.578285+0000","last_fullsized":"2024-05-08T05:42:59.578285+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:25.616180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582121+0000","last_change":"2024-05-08T05:42:59.586380+0000","last_active":"2024-05-08T05:43:00.582121+0000","last_peered":"2024-05-08T05:43:00.582121+0000","last_clean":"2024-05-08T05:43:00.582121+0000","last_became_active":"2024-05-08T05:42:59.586110+0000","last_became_peered":"2024-05-08T05:42:59.586110+0000","last_unstale":"2024-05-08T05:43:00.582121+0000","last_undegraded":"2024-05-08T05:43:00.582121+0000","last_fullsized":"2024-05-08T05:43:00.582121+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:55.193274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.584967+0000","last_change":"2024-05-08T05:42:58.577804+0000","last_active":"2024-05-08T05:43:00.584967+0000","last_peered":"2024-05-08T05:43:00.584967+0000","last_clean":"2024-05-08T05:43:00.584967+0000","last_became_active":"2024-05-08T05:42:58.577442+0000","last_became_peered":"2024-05-08T05:42:58.577442+0000","last_unstale":"2024-05-08T05:43:00.584967+0000","last_undegraded":"2024-05-08T05:43:00.584967+0000","last_fullsized":"2024-05-08T05:43:00.584967+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:16.515322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.584986+0000","last_change":"2024-05-08T05:42:59.590669+0000","last_active":"2024-05-08T05:43:00.584986+0000","last_peered":"2024-05-08T05:43:00.584986+0000","last_clean":"2024-05-08T05:43:00.584986+0000","last_became_active":"2024-05-08T05:42:59.590341+0000","last_became_peered":"2024-05-08T05:42:59.590341+0000","last_unstale":"2024-05-08T05:43:00.584986+0000","last_undegraded":"2024-05-08T05:43:00.584986+0000","last_fullsized":"2024-05-08T05:43:00.584986+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:20:43.774846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582240+0000","last_change":"2024-05-08T05:42:58.571644+0000","last_active":"2024-05-08T05:43:00.582240+0000","last_peered":"2024-05-08T05:43:00.582240+0000","last_clean":"2024-05-08T05:43:00.582240+0000","last_became_active":"2024-05-08T05:42:58.571348+0000","last_became_peered":"2024-05-08T05:42:58.571348+0000","last_unstale":"2024-05-08T05:43:00.582240+0000","last_undegraded":"2024-05-08T05:43:00.582240+0000","last_fullsized":"2024-05-08T05:43:00.582240+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:08.141740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162526+0000","last_change":"2024-05-08T05:42:59.584308+0000","last_active":"2024-05-08T05:43:01.162526+0000","last_peered":"2024-05-08T05:43:01.162526+0000","last_clean":"2024-05-08T05:43:01.162526+0000","last_became_active":"2024-05-08T05:42:59.584026+0000","last_became_peered":"2024-05-08T05:42:59.584026+0000","last_unstale":"2024-05-08T05:43:01.162526+0000","last_undegraded":"2024-05-08T05:43:01.162526+0000","last_fullsized":"2024-05-08T05:43:01.162526+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:19:42.624642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582170+0000","last_change":"2024-05-08T05:42:58.580460+0000","last_active":"2024-05-08T05:43:00.582170+0000","last_peered":"2024-05-08T05:43:00.582170+0000","last_clean":"2024-05-08T05:43:00.582170+0000","last_became_active":"2024-05-08T05:42:58.580085+0000","last_became_peered":"2024-05-08T05:42:58.580085+0000","last_unstale":"2024-05-08T05:43:00.582170+0000","last_undegraded":"2024-05-08T05:43:00.582170+0000","last_fullsized":"2024-05-08T05:43:00.582170+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:54:25.118265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585147+0000","last_change":"2024-05-08T05:42:59.597880+0000","last_active":"2024-05-08T05:43:00.585147+0000","last_peered":"2024-05-08T05:43:00.585147+0000","last_clean":"2024-05-08T05:43:00.585147+0000","last_became_active":"2024-05-08T05:42:59.597599+0000","last_became_peered":"2024-05-08T05:42:59.597599+0000","last_unstale":"2024-05-08T05:43:00.585147+0000","last_undegraded":"2024-05-08T05:43:00.585147+0000","last_fullsized":"2024-05-08T05:43:00.585147+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:01:40.198195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.886017+0000","last_change":"2024-05-08T05:42:58.578326+0000","last_active":"2024-05-08T05:43:00.886017+0000","last_peered":"2024-05-08T05:43:00.886017+0000","last_clean":"2024-05-08T05:43:00.886017+0000","last_became_active":"2024-05-08T05:42:58.577971+0000","last_became_peered":"2024-05-08T05:42:58.577971+0000","last_unstale":"2024-05-08T05:43:00.886017+0000","last_undegraded":"2024-05-08T05:43:00.886017+0000","last_fullsized":"2024-05-08T05:43:00.886017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:39.779366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582750+0000","last_change":"2024-05-08T05:42:59.581498+0000","last_active":"2024-05-08T05:43:00.582750+0000","last_peered":"2024-05-08T05:43:00.582750+0000","last_clean":"2024-05-08T05:43:00.582750+0000","last_became_active":"2024-05-08T05:42:59.581156+0000","last_became_peered":"2024-05-08T05:42:59.581156+0000","last_unstale":"2024-05-08T05:43:00.582750+0000","last_undegraded":"2024-05-08T05:43:00.582750+0000","last_fullsized":"2024-05-08T05:43:00.582750+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:12:38.601549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162684+0000","last_change":"2024-05-08T05:42:58.578763+0000","last_active":"2024-05-08T05:43:01.162684+0000","last_peered":"2024-05-08T05:43:01.162684+0000","last_clean":"2024-05-08T05:43:01.162684+0000","last_became_active":"2024-05-08T05:42:58.578430+0000","last_became_peered":"2024-05-08T05:42:58.578430+0000","last_unstale":"2024-05-08T05:43:01.162684+0000","last_undegraded":"2024-05-08T05:43:01.162684+0000","last_fullsized":"2024-05-08T05:43:01.162684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:25.616180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.161950+0000","last_change":"2024-05-08T05:42:59.592843+0000","last_active":"2024-05-08T05:43:01.161950+0000","last_peered":"2024-05-08T05:43:01.161950+0000","last_clean":"2024-05-08T05:43:01.161950+0000","last_became_active":"2024-05-08T05:42:59.592317+0000","last_became_peered":"2024-05-08T05:42:59.592317+0000","last_unstale":"2024-05-08T05:43:01.161950+0000","last_undegraded":"2024-05-08T05:43:01.161950+0000","last_fullsized":"2024-05-08T05:43:01.161950+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:27.802918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"47'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585182+0000","last_change":"2024-05-08T05:42:58.576321+0000","last_active":"2024-05-08T05:43:00.585182+0000","last_peered":"2024-05-08T05:43:00.585182+0000","last_clean":"2024-05-08T05:43:00.585182+0000","last_became_active":"2024-05-08T05:42:58.576057+0000","last_became_peered":"2024-05-08T05:42:58.576057+0000","last_unstale":"2024-05-08T05:43:00.585182+0000","last_undegraded":"2024-05-08T05:43:00.585182+0000","last_fullsized":"2024-05-08T05:43:00.585182+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:26.794984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":157,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-08T05:42:59.578796+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:42:59.578796+0000","last_peered":"2024-05-08T05:42:59.578796+0000","last_clean":"2024-05-08T05:42:59.578796+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:42:59.578796+0000","last_undegraded":"2024-05-08T05:42:59.578796+0000","last_fullsized":"2024-05-08T05:42:59.578796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542168,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1122304,"data_stored":818492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542170,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":561152,"data_stored":242635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542169,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":552960,"data_stored":241983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":242565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":569344,"data_stored":243617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":565248,"data_stored":243595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":561152,"data_stored":242143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1155072,"data_stored":832407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:43:08.136 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:43:08.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.252+0000 7fd2f3f72640 1 Processor -- start 2024-05-08T05:43:08.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.253+0000 7fd2f3f72640 1 -- start start 2024-05-08T05:43:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f3f72640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2ec1524f0 0x7fd2ec0b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f3f72640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd2ec15bc80 0x7fd2ec06e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f3f72640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2ec0b4590 con 0x7fd2ec06e8d0 2024-05-08T05:43:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f3f72640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd2ec0b51a0 con 0x7fd2ec15bc80 2024-05-08T05:43:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f3f72640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd2ec073450 con 0x7fd2ec1524f0 2024-05-08T05:43:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f250f640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd2ec06e8d0 0x7fd2ec073700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37464/0 (socket says 172.21.15.45:37464) 2024-05-08T05:43:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f250f640 1 -- 172.21.15.45:0/1420507685 learned_addr learned my addr 172.21.15.45:0/1420507685 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f150d640 1 --2- 172.21.15.45:0/1420507685 >> v2:172.21.15.63:3300/0 conn(0x7fd2ec15bc80 0x7fd2ec06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f1d0e640 1 --2- 172.21.15.45:0/1420507685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2ec1524f0 0x7fd2ec0b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f0d0c640 1 -- 172.21.15.45:0/1420507685 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2185642964 0 0) 0x7fd2ec0b4590 con 0x7fd2ec06e8d0 2024-05-08T05:43:08.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f0d0c640 1 -- 172.21.15.45:0/1420507685 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2d8003520 con 0x7fd2ec06e8d0 2024-05-08T05:43:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f0d0c640 1 -- 172.21.15.45:0/1420507685 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 804228895 0 0) 0x7fd2d8003520 con 0x7fd2ec06e8d0 2024-05-08T05:43:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f0d0c640 1 -- 172.21.15.45:0/1420507685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2ec1524f0 msgr2=0x7fd2ec0b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f0d0c640 1 --2- 172.21.15.45:0/1420507685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2ec1524f0 0x7fd2ec0b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f0d0c640 1 -- 172.21.15.45:0/1420507685 >> v2:172.21.15.63:3300/0 conn(0x7fd2ec15bc80 msgr2=0x7fd2ec06e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f0d0c640 1 --2- 172.21.15.45:0/1420507685 >> v2:172.21.15.63:3300/0 conn(0x7fd2ec15bc80 0x7fd2ec06e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f0d0c640 1 -- 172.21.15.45:0/1420507685 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ec0b4870 con 0x7fd2ec06e8d0 2024-05-08T05:43:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f0d0c640 1 -- 172.21.15.45:0/1420507685 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fd2e40016f0 con 0x7fd2ec06e8d0 2024-05-08T05:43:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f0d0c640 1 -- 172.21.15.45:0/1420507685 >> v1:172.21.15.45:6789/0 conn(0x7fd2ec06e8d0 legacy=0x7fd2ec073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f0d0c640 1 --2- 172.21.15.45:0/1420507685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2ec1524f0 0x7fd2d8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f0d0c640 1 --2- 172.21.15.45:0/1420507685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d8003fd0 0x7fd2d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f0d0c640 1 --2- 172.21.15.45:0/1420507685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d8004d40 0x7fd2d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f0d0c640 1 -- 172.21.15.45:0/1420507685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec0b51a0 con 0x7fd2d8003fd0 2024-05-08T05:43:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f0d0c640 1 -- 172.21.15.45:0/1420507685 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec073450 con 0x7fd2d8004d40 2024-05-08T05:43:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.254+0000 7fd2f0d0c640 1 -- 172.21.15.45:0/1420507685 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d800cb20 con 0x7fd2ec1524f0 2024-05-08T05:43:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.255+0000 7fd2f150d640 1 --2- 172.21.15.45:0/1420507685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d8003fd0 0x7fd2d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:08.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.255+0000 7fd2f1d0e640 1 --2- 172.21.15.45:0/1420507685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2ec1524f0 0x7fd2d8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.255+0000 7fd2f250f640 1 --2- 172.21.15.45:0/1420507685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d8004d40 0x7fd2d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.255+0000 7fd2f150d640 1 -- 172.21.15.45:0/1420507685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2ec1524f0 msgr2=0x7fd2d8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.255+0000 7fd2f150d640 1 --2- 172.21.15.45:0/1420507685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2ec1524f0 0x7fd2d8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.255+0000 7fd2f150d640 1 -- 172.21.15.45:0/1420507685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d8004d40 msgr2=0x7fd2d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.255+0000 7fd2f150d640 1 --2- 172.21.15.45:0/1420507685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d8004d40 0x7fd2d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.255+0000 7fd2f150d640 1 -- 172.21.15.45:0/1420507685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d8003520 con 0x7fd2d8003fd0 2024-05-08T05:43:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.256+0000 7fd2f150d640 1 --2- 172.21.15.45:0/1420507685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d8003fd0 0x7fd2d8004800 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009af0 tx=0x7fd2d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d181eeac0cb906d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.256+0000 7fd2f0d0c640 1 -- 172.21.15.45:0/1420507685 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0036070 con 0x7fd2d8003fd0 2024-05-08T05:43:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.256+0000 7fd2f0d0c640 1 -- 172.21.15.45:0/1420507685 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2e0031400 con 0x7fd2d8003fd0 2024-05-08T05:43:08.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.256+0000 7fd2f0d0c640 1 -- 172.21.15.45:0/1420507685 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0009690 con 0x7fd2d8003fd0 2024-05-08T05:43:08.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.256+0000 7fd2f3f72640 1 -- 172.21.15.45:0/1420507685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d8003fd0 msgr2=0x7fd2d8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:08.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.256+0000 7fd2f3f72640 1 --2- 172.21.15.45:0/1420507685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d8003fd0 0x7fd2d8004800 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009af0 tx=0x7fd2d8005830 comp rx=0 tx=0).stop 2024-05-08T05:43:08.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.256+0000 7fd2f250f640 1 -- 172.21.15.45:0/1420507685 reap_dead start 2024-05-08T05:43:08.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.256+0000 7fd2f3f72640 1 -- 172.21.15.45:0/1420507685 shutdown_connections 2024-05-08T05:43:08.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.256+0000 7fd2f3f72640 1 -- 172.21.15.45:0/1420507685 >> 172.21.15.45:0/1420507685 conn(0x7fd2ec06c3a0 msgr2=0x7fd2ec06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:08.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.256+0000 7fd2f3f72640 1 -- 172.21.15.45:0/1420507685 shutdown_connections 2024-05-08T05:43:08.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.256+0000 7fd2f3f72640 1 -- 172.21.15.45:0/1420507685 wait complete. 2024-05-08T05:43:08.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.258+0000 7fd2f3f72640 1 Processor -- start 2024-05-08T05:43:08.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.258+0000 7fd2f3f72640 1 -- start start 2024-05-08T05:43:08.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.258+0000 7fd2f3f72640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d8003fd0 0x7fd2ec14f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:08.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.258+0000 7fd2f3f72640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2ec1f9cf0 0x7fd2ec0ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.258+0000 7fd2f3f72640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2ec0cd1b0 0x7fd2ec0cd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.258+0000 7fd2f3f72640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec1528d0 con 0x7fd2ec1f9cf0 2024-05-08T05:43:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.258+0000 7fd2f3f72640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ec0b4e30 con 0x7fd2d8003fd0 2024-05-08T05:43:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.258+0000 7fd2f3f72640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd2ec0b5630 con 0x7fd2ec0cd1b0 2024-05-08T05:43:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.259+0000 7fd2f150d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2ec1f9cf0 0x7fd2ec0ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.259+0000 7fd2f150d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2ec1f9cf0 0x7fd2ec0ae640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47434/0 (socket says 172.21.15.45:47434) 2024-05-08T05:43:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.259+0000 7fd2f150d640 1 -- 172.21.15.45:0/1060288114 learned_addr learned my addr 172.21.15.45:0/1060288114 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.259+0000 7fd2f1d0e640 1 --2- 172.21.15.45:0/1060288114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d8003fd0 0x7fd2ec14f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.259+0000 7fd2f250f640 1 --2- 172.21.15.45:0/1060288114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2ec0cd1b0 0x7fd2ec0cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.259+0000 7fd2f150d640 1 -- 172.21.15.45:0/1060288114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2ec0cd1b0 msgr2=0x7fd2ec0cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.259+0000 7fd2f150d640 1 --2- 172.21.15.45:0/1060288114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2ec0cd1b0 0x7fd2ec0cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.259+0000 7fd2f150d640 1 -- 172.21.15.45:0/1060288114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d8003fd0 msgr2=0x7fd2ec14f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.259+0000 7fd2f150d640 1 --2- 172.21.15.45:0/1060288114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d8003fd0 0x7fd2ec14f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.259+0000 7fd2f150d640 1 -- 172.21.15.45:0/1060288114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ec09cce0 con 0x7fd2ec1f9cf0 2024-05-08T05:43:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.259+0000 7fd2f150d640 1 --2- 172.21.15.45:0/1060288114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2ec1f9cf0 0x7fd2ec0ae640 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009c10 tx=0x7fd2e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.259+0000 7fd2de7fc640 1 -- 172.21.15.45:0/1060288114 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0040020 con 0x7fd2ec1f9cf0 2024-05-08T05:43:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.259+0000 7fd2de7fc640 1 -- 172.21.15.45:0/1060288114 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2e0003810 con 0x7fd2ec1f9cf0 2024-05-08T05:43:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.259+0000 7fd2f3f72640 1 -- 172.21.15.45:0/1060288114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2ec09cfc0 con 0x7fd2ec1f9cf0 2024-05-08T05:43:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.259+0000 7fd2de7fc640 1 -- 172.21.15.45:0/1060288114 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0036040 con 0x7fd2ec1f9cf0 2024-05-08T05:43:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.259+0000 7fd2f3f72640 1 -- 172.21.15.45:0/1060288114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2ec09d580 con 0x7fd2ec1f9cf0 2024-05-08T05:43:08.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.261+0000 7fd2f3f72640 1 -- 172.21.15.45:0/1060288114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2ec0693f0 con 0x7fd2ec1f9cf0 2024-05-08T05:43:08.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.261+0000 7fd2de7fc640 1 -- 172.21.15.45:0/1060288114 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd2e0043020 con 0x7fd2ec1f9cf0 2024-05-08T05:43:08.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.262+0000 7fd2de7fc640 1 --2- 172.21.15.45:0/1060288114 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd2d00762d0 0x7fd2d0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:08.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.262+0000 7fd2de7fc640 1 -- 172.21.15.45:0/1060288114 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7fd2e00b70e0 con 0x7fd2ec1f9cf0 2024-05-08T05:43:08.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.262+0000 7fd2f1d0e640 1 --2- 172.21.15.45:0/1060288114 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd2d00762d0 0x7fd2d0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:08.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.262+0000 7fd2f1d0e640 1 --2- 172.21.15.45:0/1060288114 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd2d00762d0 0x7fd2d0078790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8007440 tx=0x7fd2e800b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:08.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.266+0000 7fd2de7fc640 1 -- 172.21.15.45:0/1060288114 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd2e00808b0 con 0x7fd2ec1f9cf0 2024-05-08T05:43:08.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.454+0000 7fd2f3f72640 1 -- 172.21.15.45:0/1060288114 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd2ec09ca90 con 0x7fd2d00762d0 2024-05-08T05:43:08.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.461+0000 7fd2de7fc640 1 -- 172.21.15.45:0/1060288114 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251909 (secure 0 0 0) 0x7fd2ec09ca90 con 0x7fd2d00762d0 2024-05-08T05:43:08.464 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:08.465 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:43:08.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.464+0000 7fd2f3f72640 1 -- 172.21.15.45:0/1060288114 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd2d00762d0 msgr2=0x7fd2d0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:08.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.464+0000 7fd2f3f72640 1 --2- 172.21.15.45:0/1060288114 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd2d00762d0 0x7fd2d0078790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8007440 tx=0x7fd2e800b040 comp rx=0 tx=0).stop 2024-05-08T05:43:08.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.464+0000 7fd2f3f72640 1 -- 172.21.15.45:0/1060288114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2ec1f9cf0 msgr2=0x7fd2ec0ae640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:08.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.464+0000 7fd2f3f72640 1 --2- 172.21.15.45:0/1060288114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2ec1f9cf0 0x7fd2ec0ae640 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009c10 tx=0x7fd2e0004730 comp rx=0 tx=0).stop 2024-05-08T05:43:08.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.464+0000 7fd2f3f72640 1 -- 172.21.15.45:0/1060288114 shutdown_connections 2024-05-08T05:43:08.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.464+0000 7fd2f3f72640 1 --2- 172.21.15.45:0/1060288114 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd2d00762d0 0x7fd2d0078790 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.464+0000 7fd2f3f72640 1 --2- 172.21.15.45:0/1060288114 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd2ec0cd1b0 0x7fd2ec0cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.464+0000 7fd2f3f72640 1 --2- 172.21.15.45:0/1060288114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2ec1f9cf0 0x7fd2ec0ae640 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.464+0000 7fd2f3f72640 1 --2- 172.21.15.45:0/1060288114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2d8003fd0 0x7fd2ec14f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.464+0000 7fd2f3f72640 1 -- 172.21.15.45:0/1060288114 >> 172.21.15.45:0/1060288114 conn(0x7fd2ec06c3a0 msgr2=0x7fd2ec06ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:08.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.464+0000 7fd2f3f72640 1 -- 172.21.15.45:0/1060288114 shutdown_connections 2024-05-08T05:43:08.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.464+0000 7fd2f3f72640 1 -- 172.21.15.45:0/1060288114 wait complete. 2024-05-08T05:43:08.475 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2024-05-08T05:43:06.854681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238384,"kb_used_data":5544,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736336,"statfs":{"total":773094113280,"available":772850008064,"internally_reserved":0,"allocated":5677056,"data_stored":3121420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001913"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162378+0000","last_change":"2024-05-08T05:42:59.579678+0000","last_active":"2024-05-08T05:43:01.162378+0000","last_peered":"2024-05-08T05:43:01.162378+0000","last_clean":"2024-05-08T05:43:01.162378+0000","last_became_active":"2024-05-08T05:42:59.579353+0000","last_became_peered":"2024-05-08T05:42:59.579353+0000","last_unstale":"2024-05-08T05:43:01.162378+0000","last_undegraded":"2024-05-08T05:43:01.162378+0000","last_fullsized":"2024-05-08T05:43:01.162378+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:26.624115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.588437+0000","last_change":"2024-05-08T05:42:59.586491+0000","last_active":"2024-05-08T05:43:00.588437+0000","last_peered":"2024-05-08T05:43:00.588437+0000","last_clean":"2024-05-08T05:43:00.588437+0000","last_became_active":"2024-05-08T05:42:59.586233+0000","last_became_peered":"2024-05-08T05:42:59.586233+0000","last_unstale":"2024-05-08T05:43:00.588437+0000","last_undegraded":"2024-05-08T05:43:00.588437+0000","last_fullsized":"2024-05-08T05:43:00.588437+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:39:15.656632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585174+0000","last_change":"2024-05-08T05:42:59.590747+0000","last_active":"2024-05-08T05:43:00.585174+0000","last_peered":"2024-05-08T05:43:00.585174+0000","last_clean":"2024-05-08T05:43:00.585174+0000","last_became_active":"2024-05-08T05:42:59.590436+0000","last_became_peered":"2024-05-08T05:42:59.590436+0000","last_unstale":"2024-05-08T05:43:00.585174+0000","last_undegraded":"2024-05-08T05:43:00.585174+0000","last_fullsized":"2024-05-08T05:43:00.585174+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:41:55.376598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.588282+0000","last_change":"2024-05-08T05:42:59.589298+0000","last_active":"2024-05-08T05:43:00.588282+0000","last_peered":"2024-05-08T05:43:00.588282+0000","last_clean":"2024-05-08T05:43:00.588282+0000","last_became_active":"2024-05-08T05:42:59.588989+0000","last_became_peered":"2024-05-08T05:42:59.588989+0000","last_unstale":"2024-05-08T05:43:00.588282+0000","last_undegraded":"2024-05-08T05:43:00.588282+0000","last_fullsized":"2024-05-08T05:43:00.588282+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:19:42.624642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.886354+0000","last_change":"2024-05-08T05:42:59.589282+0000","last_active":"2024-05-08T05:43:00.886354+0000","last_peered":"2024-05-08T05:43:00.886354+0000","last_clean":"2024-05-08T05:43:00.886354+0000","last_became_active":"2024-05-08T05:42:59.589004+0000","last_became_peered":"2024-05-08T05:42:59.589004+0000","last_unstale":"2024-05-08T05:43:00.886354+0000","last_undegraded":"2024-05-08T05:43:00.886354+0000","last_fullsized":"2024-05-08T05:43:00.886354+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:05:27.975200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162111+0000","last_change":"2024-05-08T05:42:59.584415+0000","last_active":"2024-05-08T05:43:01.162111+0000","last_peered":"2024-05-08T05:43:01.162111+0000","last_clean":"2024-05-08T05:43:01.162111+0000","last_became_active":"2024-05-08T05:42:59.584129+0000","last_became_peered":"2024-05-08T05:42:59.584129+0000","last_unstale":"2024-05-08T05:43:01.162111+0000","last_undegraded":"2024-05-08T05:43:01.162111+0000","last_fullsized":"2024-05-08T05:43:01.162111+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:05.059317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.584713+0000","last_change":"2024-05-08T05:42:59.589541+0000","last_active":"2024-05-08T05:43:00.584713+0000","last_peered":"2024-05-08T05:43:00.584713+0000","last_clean":"2024-05-08T05:43:00.584713+0000","last_became_active":"2024-05-08T05:42:59.588332+0000","last_became_peered":"2024-05-08T05:42:59.588332+0000","last_unstale":"2024-05-08T05:43:00.584713+0000","last_undegraded":"2024-05-08T05:43:00.584713+0000","last_fullsized":"2024-05-08T05:43:00.584713+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:47:11.811597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582165+0000","last_change":"2024-05-08T05:42:59.591871+0000","last_active":"2024-05-08T05:43:00.582165+0000","last_peered":"2024-05-08T05:43:00.582165+0000","last_clean":"2024-05-08T05:43:00.582165+0000","last_became_active":"2024-05-08T05:42:59.591612+0000","last_became_peered":"2024-05-08T05:42:59.591612+0000","last_unstale":"2024-05-08T05:43:00.582165+0000","last_undegraded":"2024-05-08T05:43:00.582165+0000","last_fullsized":"2024-05-08T05:43:00.582165+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:38.824072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582601+0000","last_change":"2024-05-08T05:42:59.593793+0000","last_active":"2024-05-08T05:43:00.582601+0000","last_peered":"2024-05-08T05:43:00.582601+0000","last_clean":"2024-05-08T05:43:00.582601+0000","last_became_active":"2024-05-08T05:42:59.592651+0000","last_became_peered":"2024-05-08T05:42:59.592651+0000","last_unstale":"2024-05-08T05:43:00.582601+0000","last_undegraded":"2024-05-08T05:43:00.582601+0000","last_fullsized":"2024-05-08T05:43:00.582601+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:13:24.282730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162260+0000","last_change":"2024-05-08T05:42:59.584754+0000","last_active":"2024-05-08T05:43:01.162260+0000","last_peered":"2024-05-08T05:43:01.162260+0000","last_clean":"2024-05-08T05:43:01.162260+0000","last_became_active":"2024-05-08T05:42:59.584449+0000","last_became_peered":"2024-05-08T05:42:59.584449+0000","last_unstale":"2024-05-08T05:43:01.162260+0000","last_undegraded":"2024-05-08T05:43:01.162260+0000","last_fullsized":"2024-05-08T05:43:01.162260+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:29.071179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162131+0000","last_change":"2024-05-08T05:42:59.582974+0000","last_active":"2024-05-08T05:43:01.162131+0000","last_peered":"2024-05-08T05:43:01.162131+0000","last_clean":"2024-05-08T05:43:01.162131+0000","last_became_active":"2024-05-08T05:42:59.582674+0000","last_became_peered":"2024-05-08T05:42:59.582674+0000","last_unstale":"2024-05-08T05:43:01.162131+0000","last_undegraded":"2024-05-08T05:43:01.162131+0000","last_fullsized":"2024-05-08T05:43:01.162131+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:49:51.317831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582635+0000","last_change":"2024-05-08T05:42:59.596052+0000","last_active":"2024-05-08T05:43:00.582635+0000","last_peered":"2024-05-08T05:43:00.582635+0000","last_clean":"2024-05-08T05:43:00.582635+0000","last_became_active":"2024-05-08T05:42:59.595755+0000","last_became_peered":"2024-05-08T05:42:59.595755+0000","last_unstale":"2024-05-08T05:43:00.582635+0000","last_undegraded":"2024-05-08T05:43:00.582635+0000","last_fullsized":"2024-05-08T05:43:00.582635+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:11:29.901922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162437+0000","last_change":"2024-05-08T05:42:59.585199+0000","last_active":"2024-05-08T05:43:01.162437+0000","last_peered":"2024-05-08T05:43:01.162437+0000","last_clean":"2024-05-08T05:43:01.162437+0000","last_became_active":"2024-05-08T05:42:59.584917+0000","last_became_peered":"2024-05-08T05:42:59.584917+0000","last_unstale":"2024-05-08T05:43:01.162437+0000","last_undegraded":"2024-05-08T05:43:01.162437+0000","last_fullsized":"2024-05-08T05:43:01.162437+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:23:18.816427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.584961+0000","last_change":"2024-05-08T05:42:59.600179+0000","last_active":"2024-05-08T05:43:00.584961+0000","last_peered":"2024-05-08T05:43:00.584961+0000","last_clean":"2024-05-08T05:43:00.584961+0000","last_became_active":"2024-05-08T05:42:59.599853+0000","last_became_peered":"2024-05-08T05:42:59.599853+0000","last_unstale":"2024-05-08T05:43:00.584961+0000","last_undegraded":"2024-05-08T05:43:00.584961+0000","last_fullsized":"2024-05-08T05:43:00.584961+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:19:42.624642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.588009+0000","last_change":"2024-05-08T05:42:59.593026+0000","last_active":"2024-05-08T05:43:00.588009+0000","last_peered":"2024-05-08T05:43:00.588009+0000","last_clean":"2024-05-08T05:43:00.588009+0000","last_became_active":"2024-05-08T05:42:59.592706+0000","last_became_peered":"2024-05-08T05:42:59.592706+0000","last_unstale":"2024-05-08T05:43:00.588009+0000","last_undegraded":"2024-05-08T05:43:00.588009+0000","last_fullsized":"2024-05-08T05:43:00.588009+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:08.244250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.161882+0000","last_change":"2024-05-08T05:42:59.584032+0000","last_active":"2024-05-08T05:43:01.161882+0000","last_peered":"2024-05-08T05:43:01.161882+0000","last_clean":"2024-05-08T05:43:01.161882+0000","last_became_active":"2024-05-08T05:42:59.583732+0000","last_became_peered":"2024-05-08T05:42:59.583732+0000","last_unstale":"2024-05-08T05:43:01.161882+0000","last_undegraded":"2024-05-08T05:43:01.161882+0000","last_fullsized":"2024-05-08T05:43:01.161882+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:24:56.356577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585564+0000","last_change":"2024-05-08T05:42:59.597808+0000","last_active":"2024-05-08T05:43:00.585564+0000","last_peered":"2024-05-08T05:43:00.585564+0000","last_clean":"2024-05-08T05:43:00.585564+0000","last_became_active":"2024-05-08T05:42:59.597515+0000","last_became_peered":"2024-05-08T05:42:59.597515+0000","last_unstale":"2024-05-08T05:43:00.585564+0000","last_undegraded":"2024-05-08T05:43:00.585564+0000","last_fullsized":"2024-05-08T05:43:00.585564+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:42:14.163907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.588625+0000","last_change":"2024-05-08T05:42:59.592278+0000","last_active":"2024-05-08T05:43:00.588625+0000","last_peered":"2024-05-08T05:43:00.588625+0000","last_clean":"2024-05-08T05:43:00.588625+0000","last_became_active":"2024-05-08T05:42:59.591985+0000","last_became_peered":"2024-05-08T05:42:59.591985+0000","last_unstale":"2024-05-08T05:43:00.588625+0000","last_undegraded":"2024-05-08T05:43:00.588625+0000","last_fullsized":"2024-05-08T05:43:00.588625+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:01:40.198195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585218+0000","last_change":"2024-05-08T05:42:59.601023+0000","last_active":"2024-05-08T05:43:00.585218+0000","last_peered":"2024-05-08T05:43:00.585218+0000","last_clean":"2024-05-08T05:43:00.585218+0000","last_became_active":"2024-05-08T05:42:59.600730+0000","last_became_peered":"2024-05-08T05:42:59.600730+0000","last_unstale":"2024-05-08T05:43:00.585218+0000","last_undegraded":"2024-05-08T05:43:00.585218+0000","last_fullsized":"2024-05-08T05:43:00.585218+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:17:08.488947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162202+0000","last_change":"2024-05-08T05:42:59.584649+0000","last_active":"2024-05-08T05:43:01.162202+0000","last_peered":"2024-05-08T05:43:01.162202+0000","last_clean":"2024-05-08T05:43:01.162202+0000","last_became_active":"2024-05-08T05:42:59.584233+0000","last_became_peered":"2024-05-08T05:42:59.584233+0000","last_unstale":"2024-05-08T05:43:01.162202+0000","last_undegraded":"2024-05-08T05:43:01.162202+0000","last_fullsized":"2024-05-08T05:43:01.162202+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:02:16.483573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162694+0000","last_change":"2024-05-08T05:42:59.591062+0000","last_active":"2024-05-08T05:43:01.162694+0000","last_peered":"2024-05-08T05:43:01.162694+0000","last_clean":"2024-05-08T05:43:01.162694+0000","last_became_active":"2024-05-08T05:42:59.590680+0000","last_became_peered":"2024-05-08T05:42:59.590680+0000","last_unstale":"2024-05-08T05:43:01.162694+0000","last_undegraded":"2024-05-08T05:43:01.162694+0000","last_fullsized":"2024-05-08T05:43:01.162694+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:24:56.356577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582069+0000","last_change":"2024-05-08T05:42:59.594863+0000","last_active":"2024-05-08T05:43:00.582069+0000","last_peered":"2024-05-08T05:43:00.582069+0000","last_clean":"2024-05-08T05:43:00.582069+0000","last_became_active":"2024-05-08T05:42:59.594599+0000","last_became_peered":"2024-05-08T05:42:59.594599+0000","last_unstale":"2024-05-08T05:43:00.582069+0000","last_undegraded":"2024-05-08T05:43:00.582069+0000","last_fullsized":"2024-05-08T05:43:00.582069+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:17:08.488947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.161776+0000","last_change":"2024-05-08T05:42:59.584918+0000","last_active":"2024-05-08T05:43:01.161776+0000","last_peered":"2024-05-08T05:43:01.161776+0000","last_clean":"2024-05-08T05:43:01.161776+0000","last_became_active":"2024-05-08T05:42:59.584635+0000","last_became_peered":"2024-05-08T05:42:59.584635+0000","last_unstale":"2024-05-08T05:43:01.161776+0000","last_undegraded":"2024-05-08T05:43:01.161776+0000","last_fullsized":"2024-05-08T05:43:01.161776+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:34:48.297871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.584650+0000","last_change":"2024-05-08T05:42:59.591296+0000","last_active":"2024-05-08T05:43:00.584650+0000","last_peered":"2024-05-08T05:43:00.584650+0000","last_clean":"2024-05-08T05:43:00.584650+0000","last_became_active":"2024-05-08T05:42:59.590937+0000","last_became_peered":"2024-05-08T05:42:59.590937+0000","last_unstale":"2024-05-08T05:43:00.584650+0000","last_undegraded":"2024-05-08T05:43:00.584650+0000","last_fullsized":"2024-05-08T05:43:00.584650+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:34.093891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585378+0000","last_change":"2024-05-08T05:42:59.589656+0000","last_active":"2024-05-08T05:43:00.585378+0000","last_peered":"2024-05-08T05:43:00.585378+0000","last_clean":"2024-05-08T05:43:00.585378+0000","last_became_active":"2024-05-08T05:42:59.589351+0000","last_became_peered":"2024-05-08T05:42:59.589351+0000","last_unstale":"2024-05-08T05:43:00.585378+0000","last_undegraded":"2024-05-08T05:43:00.585378+0000","last_fullsized":"2024-05-08T05:43:00.585378+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:38.824072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582306+0000","last_change":"2024-05-08T05:42:59.584164+0000","last_active":"2024-05-08T05:43:00.582306+0000","last_peered":"2024-05-08T05:43:00.582306+0000","last_clean":"2024-05-08T05:43:00.582306+0000","last_became_active":"2024-05-08T05:42:59.583903+0000","last_became_peered":"2024-05-08T05:42:59.583903+0000","last_unstale":"2024-05-08T05:43:00.582306+0000","last_undegraded":"2024-05-08T05:43:00.582306+0000","last_fullsized":"2024-05-08T05:43:00.582306+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:42:14.163907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162515+0000","last_change":"2024-05-08T05:42:59.591847+0000","last_active":"2024-05-08T05:43:01.162515+0000","last_peered":"2024-05-08T05:43:01.162515+0000","last_clean":"2024-05-08T05:43:01.162515+0000","last_became_active":"2024-05-08T05:42:59.591557+0000","last_became_peered":"2024-05-08T05:42:59.591557+0000","last_unstale":"2024-05-08T05:43:01.162515+0000","last_undegraded":"2024-05-08T05:43:01.162515+0000","last_fullsized":"2024-05-08T05:43:01.162515+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:09.149675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162857+0000","last_change":"2024-05-08T05:42:59.582258+0000","last_active":"2024-05-08T05:43:01.162857+0000","last_peered":"2024-05-08T05:43:01.162857+0000","last_clean":"2024-05-08T05:43:01.162857+0000","last_became_active":"2024-05-08T05:42:59.581973+0000","last_became_peered":"2024-05-08T05:42:59.581973+0000","last_unstale":"2024-05-08T05:43:01.162857+0000","last_undegraded":"2024-05-08T05:43:01.162857+0000","last_fullsized":"2024-05-08T05:43:01.162857+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:05.059317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.584837+0000","last_change":"2024-05-08T05:42:59.593351+0000","last_active":"2024-05-08T05:43:00.584837+0000","last_peered":"2024-05-08T05:43:00.584837+0000","last_clean":"2024-05-08T05:43:00.584837+0000","last_became_active":"2024-05-08T05:42:59.593077+0000","last_became_peered":"2024-05-08T05:42:59.593077+0000","last_unstale":"2024-05-08T05:43:00.584837+0000","last_undegraded":"2024-05-08T05:43:00.584837+0000","last_fullsized":"2024-05-08T05:43:00.584837+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:08.244250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.588446+0000","last_change":"2024-05-08T05:42:59.583375+0000","last_active":"2024-05-08T05:43:00.588446+0000","last_peered":"2024-05-08T05:43:00.588446+0000","last_clean":"2024-05-08T05:43:00.588446+0000","last_became_active":"2024-05-08T05:42:59.583026+0000","last_became_peered":"2024-05-08T05:42:59.583026+0000","last_unstale":"2024-05-08T05:43:00.588446+0000","last_undegraded":"2024-05-08T05:43:00.588446+0000","last_fullsized":"2024-05-08T05:43:00.588446+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:25:17.636288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.584784+0000","last_change":"2024-05-08T05:42:59.589277+0000","last_active":"2024-05-08T05:43:00.584784+0000","last_peered":"2024-05-08T05:43:00.584784+0000","last_clean":"2024-05-08T05:43:00.584784+0000","last_became_active":"2024-05-08T05:42:59.588987+0000","last_became_peered":"2024-05-08T05:42:59.588987+0000","last_unstale":"2024-05-08T05:43:00.584784+0000","last_undegraded":"2024-05-08T05:43:00.584784+0000","last_fullsized":"2024-05-08T05:43:00.584784+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:54.934881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162579+0000","last_change":"2024-05-08T05:42:59.586488+0000","last_active":"2024-05-08T05:43:01.162579+0000","last_peered":"2024-05-08T05:43:01.162579+0000","last_clean":"2024-05-08T05:43:01.162579+0000","last_became_active":"2024-05-08T05:42:59.585806+0000","last_became_peered":"2024-05-08T05:42:59.585806+0000","last_unstale":"2024-05-08T05:43:01.162579+0000","last_undegraded":"2024-05-08T05:43:01.162579+0000","last_fullsized":"2024-05-08T05:43:01.162579+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:39:15.656632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.587852+0000","last_change":"2024-05-08T05:42:59.593948+0000","last_active":"2024-05-08T05:43:00.587852+0000","last_peered":"2024-05-08T05:43:00.587852+0000","last_clean":"2024-05-08T05:43:00.587852+0000","last_became_active":"2024-05-08T05:42:59.593663+0000","last_became_peered":"2024-05-08T05:42:59.593663+0000","last_unstale":"2024-05-08T05:43:00.587852+0000","last_undegraded":"2024-05-08T05:43:00.587852+0000","last_fullsized":"2024-05-08T05:43:00.587852+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:54.934881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"46'4","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162826+0000","last_change":"2024-05-08T05:42:58.568449+0000","last_active":"2024-05-08T05:43:01.162826+0000","last_peered":"2024-05-08T05:43:01.162826+0000","last_clean":"2024-05-08T05:43:01.162826+0000","last_became_active":"2024-05-08T05:42:58.567378+0000","last_became_peered":"2024-05-08T05:42:58.567378+0000","last_unstale":"2024-05-08T05:43:01.162826+0000","last_undegraded":"2024-05-08T05:43:01.162826+0000","last_fullsized":"2024-05-08T05:43:01.162826+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:04.957071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162401+0000","last_change":"2024-05-08T05:42:59.583673+0000","last_active":"2024-05-08T05:43:01.162401+0000","last_peered":"2024-05-08T05:43:01.162401+0000","last_clean":"2024-05-08T05:43:01.162401+0000","last_became_active":"2024-05-08T05:42:59.583430+0000","last_became_peered":"2024-05-08T05:42:59.583430+0000","last_unstale":"2024-05-08T05:43:01.162401+0000","last_undegraded":"2024-05-08T05:43:01.162401+0000","last_fullsized":"2024-05-08T05:43:01.162401+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:54.934881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162234+0000","last_change":"2024-05-08T05:42:58.582169+0000","last_active":"2024-05-08T05:43:01.162234+0000","last_peered":"2024-05-08T05:43:01.162234+0000","last_clean":"2024-05-08T05:43:01.162234+0000","last_became_active":"2024-05-08T05:42:58.581851+0000","last_became_peered":"2024-05-08T05:42:58.581851+0000","last_unstale":"2024-05-08T05:43:01.162234+0000","last_undegraded":"2024-05-08T05:43:01.162234+0000","last_fullsized":"2024-05-08T05:43:01.162234+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:32.556335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162026+0000","last_change":"2024-05-08T05:42:59.585789+0000","last_active":"2024-05-08T05:43:01.162026+0000","last_peered":"2024-05-08T05:43:01.162026+0000","last_clean":"2024-05-08T05:43:01.162026+0000","last_became_active":"2024-05-08T05:42:59.585486+0000","last_became_peered":"2024-05-08T05:42:59.585486+0000","last_unstale":"2024-05-08T05:43:01.162026+0000","last_undegraded":"2024-05-08T05:43:01.162026+0000","last_fullsized":"2024-05-08T05:43:01.162026+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:32:52.739108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162618+0000","last_change":"2024-05-08T05:42:58.577001+0000","last_active":"2024-05-08T05:43:01.162618+0000","last_peered":"2024-05-08T05:43:01.162618+0000","last_clean":"2024-05-08T05:43:01.162618+0000","last_became_active":"2024-05-08T05:42:58.575665+0000","last_became_peered":"2024-05-08T05:42:58.575665+0000","last_unstale":"2024-05-08T05:43:01.162618+0000","last_undegraded":"2024-05-08T05:43:01.162618+0000","last_fullsized":"2024-05-08T05:43:01.162618+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:46.384674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582385+0000","last_change":"2024-05-08T05:42:59.583002+0000","last_active":"2024-05-08T05:43:00.582385+0000","last_peered":"2024-05-08T05:43:00.582385+0000","last_clean":"2024-05-08T05:43:00.582385+0000","last_became_active":"2024-05-08T05:42:59.582323+0000","last_became_peered":"2024-05-08T05:42:59.582323+0000","last_unstale":"2024-05-08T05:43:00.582385+0000","last_undegraded":"2024-05-08T05:43:00.582385+0000","last_fullsized":"2024-05-08T05:43:00.582385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:15:08.074347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162613+0000","last_change":"2024-05-08T05:42:58.573567+0000","last_active":"2024-05-08T05:43:01.162613+0000","last_peered":"2024-05-08T05:43:01.162613+0000","last_clean":"2024-05-08T05:43:01.162613+0000","last_became_active":"2024-05-08T05:42:58.573220+0000","last_became_peered":"2024-05-08T05:42:58.573220+0000","last_unstale":"2024-05-08T05:43:01.162613+0000","last_undegraded":"2024-05-08T05:43:01.162613+0000","last_fullsized":"2024-05-08T05:43:01.162613+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:15.376533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.886460+0000","last_change":"2024-05-08T05:42:59.590972+0000","last_active":"2024-05-08T05:43:00.886460+0000","last_peered":"2024-05-08T05:43:00.886460+0000","last_clean":"2024-05-08T05:43:00.886460+0000","last_became_active":"2024-05-08T05:42:59.590667+0000","last_became_peered":"2024-05-08T05:42:59.590667+0000","last_unstale":"2024-05-08T05:43:00.886460+0000","last_undegraded":"2024-05-08T05:43:00.886460+0000","last_fullsized":"2024-05-08T05:43:00.886460+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:27.802918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585060+0000","last_change":"2024-05-08T05:42:58.572908+0000","last_active":"2024-05-08T05:43:00.585060+0000","last_peered":"2024-05-08T05:43:00.585060+0000","last_clean":"2024-05-08T05:43:00.585060+0000","last_became_active":"2024-05-08T05:42:58.572613+0000","last_became_peered":"2024-05-08T05:42:58.572613+0000","last_unstale":"2024-05-08T05:43:00.585060+0000","last_undegraded":"2024-05-08T05:43:00.585060+0000","last_fullsized":"2024-05-08T05:43:00.585060+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:28:40.992183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.588626+0000","last_change":"2024-05-08T05:42:59.589512+0000","last_active":"2024-05-08T05:43:00.588626+0000","last_peered":"2024-05-08T05:43:00.588626+0000","last_clean":"2024-05-08T05:43:00.588626+0000","last_became_active":"2024-05-08T05:42:59.588973+0000","last_became_peered":"2024-05-08T05:42:59.588973+0000","last_unstale":"2024-05-08T05:43:00.588626+0000","last_undegraded":"2024-05-08T05:43:00.588626+0000","last_fullsized":"2024-05-08T05:43:00.588626+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:41:55.376598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.886034+0000","last_change":"2024-05-08T05:42:58.569466+0000","last_active":"2024-05-08T05:43:00.886034+0000","last_peered":"2024-05-08T05:43:00.886034+0000","last_clean":"2024-05-08T05:43:00.886034+0000","last_became_active":"2024-05-08T05:42:58.569048+0000","last_became_peered":"2024-05-08T05:42:58.569048+0000","last_unstale":"2024-05-08T05:43:00.886034+0000","last_undegraded":"2024-05-08T05:43:00.886034+0000","last_fullsized":"2024-05-08T05:43:00.886034+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:28.357714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582595+0000","last_change":"2024-05-08T05:42:59.585222+0000","last_active":"2024-05-08T05:43:00.582595+0000","last_peered":"2024-05-08T05:43:00.582595+0000","last_clean":"2024-05-08T05:43:00.582595+0000","last_became_active":"2024-05-08T05:42:59.583480+0000","last_became_peered":"2024-05-08T05:42:59.583480+0000","last_unstale":"2024-05-08T05:43:00.582595+0000","last_undegraded":"2024-05-08T05:43:00.582595+0000","last_fullsized":"2024-05-08T05:43:00.582595+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:01:32.094856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.161928+0000","last_change":"2024-05-08T05:42:58.568749+0000","last_active":"2024-05-08T05:43:01.161928+0000","last_peered":"2024-05-08T05:43:01.161928+0000","last_clean":"2024-05-08T05:43:01.161928+0000","last_became_active":"2024-05-08T05:42:58.568473+0000","last_became_peered":"2024-05-08T05:42:58.568473+0000","last_unstale":"2024-05-08T05:43:01.161928+0000","last_undegraded":"2024-05-08T05:43:01.161928+0000","last_fullsized":"2024-05-08T05:43:01.161928+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:55:31.695068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.588339+0000","last_change":"2024-05-08T05:42:59.591341+0000","last_active":"2024-05-08T05:43:00.588339+0000","last_peered":"2024-05-08T05:43:00.588339+0000","last_clean":"2024-05-08T05:43:00.588339+0000","last_became_active":"2024-05-08T05:42:59.591050+0000","last_became_peered":"2024-05-08T05:42:59.591050+0000","last_unstale":"2024-05-08T05:43:00.588339+0000","last_undegraded":"2024-05-08T05:43:00.588339+0000","last_fullsized":"2024-05-08T05:43:00.588339+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:20:43.774846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.584665+0000","last_change":"2024-05-08T05:42:58.574070+0000","last_active":"2024-05-08T05:43:00.584665+0000","last_peered":"2024-05-08T05:43:00.584665+0000","last_clean":"2024-05-08T05:43:00.584665+0000","last_became_active":"2024-05-08T05:42:58.573769+0000","last_became_peered":"2024-05-08T05:42:58.573769+0000","last_unstale":"2024-05-08T05:43:00.584665+0000","last_undegraded":"2024-05-08T05:43:00.584665+0000","last_fullsized":"2024-05-08T05:43:00.584665+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:45:37.264655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.886152+0000","last_change":"2024-05-08T05:42:59.584451+0000","last_active":"2024-05-08T05:43:00.886152+0000","last_peered":"2024-05-08T05:43:00.886152+0000","last_clean":"2024-05-08T05:43:00.886152+0000","last_became_active":"2024-05-08T05:42:59.584194+0000","last_became_peered":"2024-05-08T05:42:59.584194+0000","last_unstale":"2024-05-08T05:43:00.886152+0000","last_undegraded":"2024-05-08T05:43:00.886152+0000","last_fullsized":"2024-05-08T05:43:00.886152+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:28:42.000118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.588604+0000","last_change":"2024-05-08T05:42:58.567761+0000","last_active":"2024-05-08T05:43:00.588604+0000","last_peered":"2024-05-08T05:43:00.588604+0000","last_clean":"2024-05-08T05:43:00.588604+0000","last_became_active":"2024-05-08T05:42:58.567490+0000","last_became_peered":"2024-05-08T05:42:58.567490+0000","last_unstale":"2024-05-08T05:43:00.588604+0000","last_undegraded":"2024-05-08T05:43:00.588604+0000","last_fullsized":"2024-05-08T05:43:00.588604+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:23:17.808492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162214+0000","last_change":"2024-05-08T05:42:59.585553+0000","last_active":"2024-05-08T05:43:01.162214+0000","last_peered":"2024-05-08T05:43:01.162214+0000","last_clean":"2024-05-08T05:43:01.162214+0000","last_became_active":"2024-05-08T05:42:59.584903+0000","last_became_peered":"2024-05-08T05:42:59.584903+0000","last_unstale":"2024-05-08T05:43:01.162214+0000","last_undegraded":"2024-05-08T05:43:01.162214+0000","last_fullsized":"2024-05-08T05:43:01.162214+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:34.093891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.886480+0000","last_change":"2024-05-08T05:42:58.574888+0000","last_active":"2024-05-08T05:43:00.886480+0000","last_peered":"2024-05-08T05:43:00.886480+0000","last_clean":"2024-05-08T05:43:00.886480+0000","last_became_active":"2024-05-08T05:42:58.574585+0000","last_became_peered":"2024-05-08T05:42:58.574585+0000","last_unstale":"2024-05-08T05:43:00.886480+0000","last_undegraded":"2024-05-08T05:43:00.886480+0000","last_fullsized":"2024-05-08T05:43:00.886480+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:55:31.695068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585036+0000","last_change":"2024-05-08T05:42:59.594208+0000","last_active":"2024-05-08T05:43:00.585036+0000","last_peered":"2024-05-08T05:43:00.585036+0000","last_clean":"2024-05-08T05:43:00.585036+0000","last_became_active":"2024-05-08T05:42:59.593886+0000","last_became_peered":"2024-05-08T05:42:59.593886+0000","last_unstale":"2024-05-08T05:43:00.585036+0000","last_undegraded":"2024-05-08T05:43:00.585036+0000","last_fullsized":"2024-05-08T05:43:00.585036+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:25:17.636288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"46'3","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.588399+0000","last_change":"2024-05-08T05:42:58.572885+0000","last_active":"2024-05-08T05:43:00.588399+0000","last_peered":"2024-05-08T05:43:00.588399+0000","last_clean":"2024-05-08T05:43:00.588399+0000","last_became_active":"2024-05-08T05:42:58.571751+0000","last_became_peered":"2024-05-08T05:42:58.571751+0000","last_unstale":"2024-05-08T05:43:00.588399+0000","last_undegraded":"2024-05-08T05:43:00.588399+0000","last_fullsized":"2024-05-08T05:43:00.588399+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:15.376533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.886353+0000","last_change":"2024-05-08T05:42:59.583118+0000","last_active":"2024-05-08T05:43:00.886353+0000","last_peered":"2024-05-08T05:43:00.886353+0000","last_clean":"2024-05-08T05:43:00.886353+0000","last_became_active":"2024-05-08T05:42:59.582673+0000","last_became_peered":"2024-05-08T05:42:59.582673+0000","last_unstale":"2024-05-08T05:43:00.886353+0000","last_undegraded":"2024-05-08T05:43:00.886353+0000","last_fullsized":"2024-05-08T05:43:00.886353+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:17.523257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"46'6","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582673+0000","last_change":"2024-05-08T05:42:58.578675+0000","last_active":"2024-05-08T05:43:00.582673+0000","last_peered":"2024-05-08T05:43:00.582673+0000","last_clean":"2024-05-08T05:43:00.582673+0000","last_became_active":"2024-05-08T05:42:58.578130+0000","last_became_peered":"2024-05-08T05:42:58.578130+0000","last_unstale":"2024-05-08T05:43:00.582673+0000","last_undegraded":"2024-05-08T05:43:00.582673+0000","last_fullsized":"2024-05-08T05:43:00.582673+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:31.669348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162379+0000","last_change":"2024-05-08T05:42:59.583647+0000","last_active":"2024-05-08T05:43:01.162379+0000","last_peered":"2024-05-08T05:43:01.162379+0000","last_clean":"2024-05-08T05:43:01.162379+0000","last_became_active":"2024-05-08T05:42:59.583302+0000","last_became_peered":"2024-05-08T05:42:59.583302+0000","last_unstale":"2024-05-08T05:43:01.162379+0000","last_undegraded":"2024-05-08T05:43:01.162379+0000","last_fullsized":"2024-05-08T05:43:01.162379+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:47:11.811597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585095+0000","last_change":"2024-05-08T05:42:58.576622+0000","last_active":"2024-05-08T05:43:00.585095+0000","last_peered":"2024-05-08T05:43:00.585095+0000","last_clean":"2024-05-08T05:43:00.585095+0000","last_became_active":"2024-05-08T05:42:58.575446+0000","last_became_peered":"2024-05-08T05:42:58.575446+0000","last_unstale":"2024-05-08T05:43:00.585095+0000","last_undegraded":"2024-05-08T05:43:00.585095+0000","last_fullsized":"2024-05-08T05:43:00.585095+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:05:26.967265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585284+0000","last_change":"2024-05-08T05:42:59.590992+0000","last_active":"2024-05-08T05:43:00.585284+0000","last_peered":"2024-05-08T05:43:00.585284+0000","last_clean":"2024-05-08T05:43:00.585284+0000","last_became_active":"2024-05-08T05:42:59.590614+0000","last_became_peered":"2024-05-08T05:42:59.590614+0000","last_unstale":"2024-05-08T05:43:00.585284+0000","last_undegraded":"2024-05-08T05:43:00.585284+0000","last_fullsized":"2024-05-08T05:43:00.585284+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:55.193274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162146+0000","last_change":"2024-05-08T05:42:58.574983+0000","last_active":"2024-05-08T05:43:01.162146+0000","last_peered":"2024-05-08T05:43:01.162146+0000","last_clean":"2024-05-08T05:43:01.162146+0000","last_became_active":"2024-05-08T05:42:58.574678+0000","last_became_peered":"2024-05-08T05:42:58.574678+0000","last_unstale":"2024-05-08T05:43:01.162146+0000","last_undegraded":"2024-05-08T05:43:01.162146+0000","last_fullsized":"2024-05-08T05:43:01.162146+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:28.357714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582451+0000","last_change":"2024-05-08T05:42:59.589300+0000","last_active":"2024-05-08T05:43:00.582451+0000","last_peered":"2024-05-08T05:43:00.582451+0000","last_clean":"2024-05-08T05:43:00.582451+0000","last_became_active":"2024-05-08T05:42:59.589015+0000","last_became_peered":"2024-05-08T05:42:59.589015+0000","last_unstale":"2024-05-08T05:43:00.582451+0000","last_undegraded":"2024-05-08T05:43:00.582451+0000","last_fullsized":"2024-05-08T05:43:00.582451+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:29:32.097989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.588148+0000","last_change":"2024-05-08T05:42:58.572724+0000","last_active":"2024-05-08T05:43:00.588148+0000","last_peered":"2024-05-08T05:43:00.588148+0000","last_clean":"2024-05-08T05:43:00.588148+0000","last_became_active":"2024-05-08T05:42:58.572471+0000","last_became_peered":"2024-05-08T05:42:58.572471+0000","last_unstale":"2024-05-08T05:43:00.588148+0000","last_undegraded":"2024-05-08T05:43:00.588148+0000","last_fullsized":"2024-05-08T05:43:00.588148+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:28.063244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585408+0000","last_change":"2024-05-08T05:42:59.594817+0000","last_active":"2024-05-08T05:43:00.585408+0000","last_peered":"2024-05-08T05:43:00.585408+0000","last_clean":"2024-05-08T05:43:00.585408+0000","last_became_active":"2024-05-08T05:42:59.594548+0000","last_became_peered":"2024-05-08T05:42:59.594548+0000","last_unstale":"2024-05-08T05:43:00.585408+0000","last_undegraded":"2024-05-08T05:43:00.585408+0000","last_fullsized":"2024-05-08T05:43:00.585408+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:39:37.911504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"46'8","reported_seq":22,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582259+0000","last_change":"2024-05-08T05:42:58.577909+0000","last_active":"2024-05-08T05:43:00.582259+0000","last_peered":"2024-05-08T05:43:00.582259+0000","last_clean":"2024-05-08T05:43:00.582259+0000","last_became_active":"2024-05-08T05:42:58.577585+0000","last_became_peered":"2024-05-08T05:42:58.577585+0000","last_unstale":"2024-05-08T05:43:00.582259+0000","last_undegraded":"2024-05-08T05:43:00.582259+0000","last_fullsized":"2024-05-08T05:43:00.582259+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:45.812663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582185+0000","last_change":"2024-05-08T05:42:59.581991+0000","last_active":"2024-05-08T05:43:00.582185+0000","last_peered":"2024-05-08T05:43:00.582185+0000","last_clean":"2024-05-08T05:43:00.582185+0000","last_became_active":"2024-05-08T05:42:59.581642+0000","last_became_peered":"2024-05-08T05:42:59.581642+0000","last_unstale":"2024-05-08T05:43:00.582185+0000","last_undegraded":"2024-05-08T05:43:00.582185+0000","last_fullsized":"2024-05-08T05:43:00.582185+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:39:37.911504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585430+0000","last_change":"2024-05-08T05:42:58.579816+0000","last_active":"2024-05-08T05:43:00.585430+0000","last_peered":"2024-05-08T05:43:00.585430+0000","last_clean":"2024-05-08T05:43:00.585430+0000","last_became_active":"2024-05-08T05:42:58.579567+0000","last_became_peered":"2024-05-08T05:42:58.579567+0000","last_unstale":"2024-05-08T05:43:00.585430+0000","last_undegraded":"2024-05-08T05:43:00.585430+0000","last_fullsized":"2024-05-08T05:43:00.585430+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:12.539768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.161857+0000","last_change":"2024-05-08T05:42:59.591106+0000","last_active":"2024-05-08T05:43:01.161857+0000","last_peered":"2024-05-08T05:43:01.161857+0000","last_clean":"2024-05-08T05:43:01.161857+0000","last_became_active":"2024-05-08T05:42:59.590815+0000","last_became_peered":"2024-05-08T05:42:59.590815+0000","last_unstale":"2024-05-08T05:43:01.161857+0000","last_undegraded":"2024-05-08T05:43:01.161857+0000","last_fullsized":"2024-05-08T05:43:01.161857+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:41:14.761007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162350+0000","last_change":"2024-05-08T05:42:58.568207+0000","last_active":"2024-05-08T05:43:01.162350+0000","last_peered":"2024-05-08T05:43:01.162350+0000","last_clean":"2024-05-08T05:43:01.162350+0000","last_became_active":"2024-05-08T05:42:58.567932+0000","last_became_peered":"2024-05-08T05:42:58.567932+0000","last_unstale":"2024-05-08T05:43:01.162350+0000","last_undegraded":"2024-05-08T05:43:01.162350+0000","last_fullsized":"2024-05-08T05:43:01.162350+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:33:11.854881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.886121+0000","last_change":"2024-05-08T05:42:59.591776+0000","last_active":"2024-05-08T05:43:00.886121+0000","last_peered":"2024-05-08T05:43:00.886121+0000","last_clean":"2024-05-08T05:43:00.886121+0000","last_became_active":"2024-05-08T05:42:59.591492+0000","last_became_peered":"2024-05-08T05:42:59.591492+0000","last_unstale":"2024-05-08T05:43:00.886121+0000","last_undegraded":"2024-05-08T05:43:00.886121+0000","last_fullsized":"2024-05-08T05:43:00.886121+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:13.547702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162886+0000","last_change":"2024-05-08T05:42:58.573572+0000","last_active":"2024-05-08T05:43:01.162886+0000","last_peered":"2024-05-08T05:43:01.162886+0000","last_clean":"2024-05-08T05:43:01.162886+0000","last_became_active":"2024-05-08T05:42:58.573275+0000","last_became_peered":"2024-05-08T05:42:58.573275+0000","last_unstale":"2024-05-08T05:43:01.162886+0000","last_undegraded":"2024-05-08T05:43:01.162886+0000","last_fullsized":"2024-05-08T05:43:01.162886+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:23:53.968037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162578+0000","last_change":"2024-05-08T05:42:59.575200+0000","last_active":"2024-05-08T05:43:01.162578+0000","last_peered":"2024-05-08T05:43:01.162578+0000","last_clean":"2024-05-08T05:43:01.162578+0000","last_became_active":"2024-05-08T05:42:59.574903+0000","last_became_peered":"2024-05-08T05:42:59.574903+0000","last_unstale":"2024-05-08T05:43:01.162578+0000","last_undegraded":"2024-05-08T05:43:01.162578+0000","last_fullsized":"2024-05-08T05:43:01.162578+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:54:26.126199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162882+0000","last_change":"2024-05-08T05:42:58.579945+0000","last_active":"2024-05-08T05:43:01.162882+0000","last_peered":"2024-05-08T05:43:01.162882+0000","last_clean":"2024-05-08T05:43:01.162882+0000","last_became_active":"2024-05-08T05:42:58.579657+0000","last_became_peered":"2024-05-08T05:42:58.579657+0000","last_unstale":"2024-05-08T05:43:01.162882+0000","last_undegraded":"2024-05-08T05:43:01.162882+0000","last_fullsized":"2024-05-08T05:43:01.162882+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:46.384674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162460+0000","last_change":"2024-05-08T05:42:59.585604+0000","last_active":"2024-05-08T05:43:01.162460+0000","last_peered":"2024-05-08T05:43:01.162460+0000","last_clean":"2024-05-08T05:43:01.162460+0000","last_became_active":"2024-05-08T05:42:59.585027+0000","last_became_peered":"2024-05-08T05:42:59.585027+0000","last_unstale":"2024-05-08T05:43:01.162460+0000","last_undegraded":"2024-05-08T05:43:01.162460+0000","last_fullsized":"2024-05-08T05:43:01.162460+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:08.244250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162570+0000","last_change":"2024-05-08T05:42:58.574363+0000","last_active":"2024-05-08T05:43:01.162570+0000","last_peered":"2024-05-08T05:43:01.162570+0000","last_clean":"2024-05-08T05:43:01.162570+0000","last_became_active":"2024-05-08T05:42:58.574101+0000","last_became_peered":"2024-05-08T05:42:58.574101+0000","last_unstale":"2024-05-08T05:43:01.162570+0000","last_undegraded":"2024-05-08T05:43:01.162570+0000","last_fullsized":"2024-05-08T05:43:01.162570+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:04.957071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162665+0000","last_change":"2024-05-08T05:42:59.587267+0000","last_active":"2024-05-08T05:43:01.162665+0000","last_peered":"2024-05-08T05:43:01.162665+0000","last_clean":"2024-05-08T05:43:01.162665+0000","last_became_active":"2024-05-08T05:42:59.586824+0000","last_became_peered":"2024-05-08T05:42:59.586824+0000","last_unstale":"2024-05-08T05:43:01.162665+0000","last_undegraded":"2024-05-08T05:43:01.162665+0000","last_fullsized":"2024-05-08T05:43:01.162665+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:41:14.761007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.581992+0000","last_change":"2024-05-08T05:42:58.572523+0000","last_active":"2024-05-08T05:43:00.581992+0000","last_peered":"2024-05-08T05:43:00.581992+0000","last_clean":"2024-05-08T05:43:00.581992+0000","last_became_active":"2024-05-08T05:42:58.572193+0000","last_became_peered":"2024-05-08T05:42:58.572193+0000","last_unstale":"2024-05-08T05:43:00.581992+0000","last_undegraded":"2024-05-08T05:43:00.581992+0000","last_fullsized":"2024-05-08T05:43:00.581992+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:34:09.731004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585033+0000","last_change":"2024-05-08T05:42:59.593271+0000","last_active":"2024-05-08T05:43:00.585033+0000","last_peered":"2024-05-08T05:43:00.585033+0000","last_clean":"2024-05-08T05:43:00.585033+0000","last_became_active":"2024-05-08T05:42:59.592988+0000","last_became_peered":"2024-05-08T05:42:59.592988+0000","last_unstale":"2024-05-08T05:43:00.585033+0000","last_undegraded":"2024-05-08T05:43:00.585033+0000","last_fullsized":"2024-05-08T05:43:00.585033+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:39:15.656632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.584839+0000","last_change":"2024-05-08T05:42:58.582112+0000","last_active":"2024-05-08T05:43:00.584839+0000","last_peered":"2024-05-08T05:43:00.584839+0000","last_clean":"2024-05-08T05:43:00.584839+0000","last_became_active":"2024-05-08T05:42:58.581779+0000","last_became_peered":"2024-05-08T05:42:58.581779+0000","last_unstale":"2024-05-08T05:43:00.584839+0000","last_undegraded":"2024-05-08T05:43:00.584839+0000","last_fullsized":"2024-05-08T05:43:00.584839+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:34:48.397348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162554+0000","last_change":"2024-05-08T05:42:59.584992+0000","last_active":"2024-05-08T05:43:01.162554+0000","last_peered":"2024-05-08T05:43:01.162554+0000","last_clean":"2024-05-08T05:43:01.162554+0000","last_became_active":"2024-05-08T05:42:59.584681+0000","last_became_peered":"2024-05-08T05:42:59.584681+0000","last_unstale":"2024-05-08T05:43:01.162554+0000","last_undegraded":"2024-05-08T05:43:01.162554+0000","last_fullsized":"2024-05-08T05:43:01.162554+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:43:32.677283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.161686+0000","last_change":"2024-05-08T05:42:58.568551+0000","last_active":"2024-05-08T05:43:01.161686+0000","last_peered":"2024-05-08T05:43:01.161686+0000","last_clean":"2024-05-08T05:43:01.161686+0000","last_became_active":"2024-05-08T05:42:58.568282+0000","last_became_peered":"2024-05-08T05:42:58.568282+0000","last_unstale":"2024-05-08T05:43:01.161686+0000","last_undegraded":"2024-05-08T05:43:01.161686+0000","last_fullsized":"2024-05-08T05:43:01.161686+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:34:46.120453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162305+0000","last_change":"2024-05-08T05:42:59.586138+0000","last_active":"2024-05-08T05:43:01.162305+0000","last_peered":"2024-05-08T05:43:01.162305+0000","last_clean":"2024-05-08T05:43:01.162305+0000","last_became_active":"2024-05-08T05:42:59.585739+0000","last_became_peered":"2024-05-08T05:42:59.585739+0000","last_unstale":"2024-05-08T05:43:01.162305+0000","last_undegraded":"2024-05-08T05:43:01.162305+0000","last_fullsized":"2024-05-08T05:43:01.162305+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:20:43.774846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162942+0000","last_change":"2024-05-08T05:42:58.580617+0000","last_active":"2024-05-08T05:43:01.162942+0000","last_peered":"2024-05-08T05:43:01.162942+0000","last_clean":"2024-05-08T05:43:01.162942+0000","last_became_active":"2024-05-08T05:42:58.580371+0000","last_became_peered":"2024-05-08T05:42:58.580371+0000","last_unstale":"2024-05-08T05:43:01.162942+0000","last_undegraded":"2024-05-08T05:43:01.162942+0000","last_fullsized":"2024-05-08T05:43:01.162942+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:32.556335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162155+0000","last_change":"2024-05-08T05:42:59.580372+0000","last_active":"2024-05-08T05:43:01.162155+0000","last_peered":"2024-05-08T05:43:01.162155+0000","last_clean":"2024-05-08T05:43:01.162155+0000","last_became_active":"2024-05-08T05:42:59.579909+0000","last_became_peered":"2024-05-08T05:42:59.579909+0000","last_unstale":"2024-05-08T05:43:01.162155+0000","last_undegraded":"2024-05-08T05:43:01.162155+0000","last_fullsized":"2024-05-08T05:43:01.162155+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:16.384468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.588591+0000","last_change":"2024-05-08T05:42:58.570248+0000","last_active":"2024-05-08T05:43:00.588591+0000","last_peered":"2024-05-08T05:43:00.588591+0000","last_clean":"2024-05-08T05:43:00.588591+0000","last_became_active":"2024-05-08T05:42:58.569966+0000","last_became_peered":"2024-05-08T05:42:58.569966+0000","last_unstale":"2024-05-08T05:43:00.588591+0000","last_undegraded":"2024-05-08T05:43:00.588591+0000","last_fullsized":"2024-05-08T05:43:00.588591+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:25.616180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582121+0000","last_change":"2024-05-08T05:42:59.586380+0000","last_active":"2024-05-08T05:43:00.582121+0000","last_peered":"2024-05-08T05:43:00.582121+0000","last_clean":"2024-05-08T05:43:00.582121+0000","last_became_active":"2024-05-08T05:42:59.586110+0000","last_became_peered":"2024-05-08T05:42:59.586110+0000","last_unstale":"2024-05-08T05:43:00.582121+0000","last_undegraded":"2024-05-08T05:43:00.582121+0000","last_fullsized":"2024-05-08T05:43:00.582121+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:55.193274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.584967+0000","last_change":"2024-05-08T05:42:58.577804+0000","last_active":"2024-05-08T05:43:00.584967+0000","last_peered":"2024-05-08T05:43:00.584967+0000","last_clean":"2024-05-08T05:43:00.584967+0000","last_became_active":"2024-05-08T05:42:58.577442+0000","last_became_peered":"2024-05-08T05:42:58.577442+0000","last_unstale":"2024-05-08T05:43:00.584967+0000","last_undegraded":"2024-05-08T05:43:00.584967+0000","last_fullsized":"2024-05-08T05:43:00.584967+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:16.515322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.584986+0000","last_change":"2024-05-08T05:42:59.590669+0000","last_active":"2024-05-08T05:43:00.584986+0000","last_peered":"2024-05-08T05:43:00.584986+0000","last_clean":"2024-05-08T05:43:00.584986+0000","last_became_active":"2024-05-08T05:42:59.590341+0000","last_became_peered":"2024-05-08T05:42:59.590341+0000","last_unstale":"2024-05-08T05:43:00.584986+0000","last_undegraded":"2024-05-08T05:43:00.584986+0000","last_fullsized":"2024-05-08T05:43:00.584986+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:20:43.774846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582240+0000","last_change":"2024-05-08T05:42:58.571644+0000","last_active":"2024-05-08T05:43:00.582240+0000","last_peered":"2024-05-08T05:43:00.582240+0000","last_clean":"2024-05-08T05:43:00.582240+0000","last_became_active":"2024-05-08T05:42:58.571348+0000","last_became_peered":"2024-05-08T05:42:58.571348+0000","last_unstale":"2024-05-08T05:43:00.582240+0000","last_undegraded":"2024-05-08T05:43:00.582240+0000","last_fullsized":"2024-05-08T05:43:00.582240+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:08.141740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162526+0000","last_change":"2024-05-08T05:42:59.584308+0000","last_active":"2024-05-08T05:43:01.162526+0000","last_peered":"2024-05-08T05:43:01.162526+0000","last_clean":"2024-05-08T05:43:01.162526+0000","last_became_active":"2024-05-08T05:42:59.584026+0000","last_became_peered":"2024-05-08T05:42:59.584026+0000","last_unstale":"2024-05-08T05:43:01.162526+0000","last_undegraded":"2024-05-08T05:43:01.162526+0000","last_fullsized":"2024-05-08T05:43:01.162526+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:19:42.624642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582170+0000","last_change":"2024-05-08T05:42:58.580460+0000","last_active":"2024-05-08T05:43:00.582170+0000","last_peered":"2024-05-08T05:43:00.582170+0000","last_clean":"2024-05-08T05:43:00.582170+0000","last_became_active":"2024-05-08T05:42:58.580085+0000","last_became_peered":"2024-05-08T05:42:58.580085+0000","last_unstale":"2024-05-08T05:43:00.582170+0000","last_undegraded":"2024-05-08T05:43:00.582170+0000","last_fullsized":"2024-05-08T05:43:00.582170+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:54:25.118265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585147+0000","last_change":"2024-05-08T05:42:59.597880+0000","last_active":"2024-05-08T05:43:00.585147+0000","last_peered":"2024-05-08T05:43:00.585147+0000","last_clean":"2024-05-08T05:43:00.585147+0000","last_became_active":"2024-05-08T05:42:59.597599+0000","last_became_peered":"2024-05-08T05:42:59.597599+0000","last_unstale":"2024-05-08T05:43:00.585147+0000","last_undegraded":"2024-05-08T05:43:00.585147+0000","last_fullsized":"2024-05-08T05:43:00.585147+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:01:40.198195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.886017+0000","last_change":"2024-05-08T05:42:58.578326+0000","last_active":"2024-05-08T05:43:00.886017+0000","last_peered":"2024-05-08T05:43:00.886017+0000","last_clean":"2024-05-08T05:43:00.886017+0000","last_became_active":"2024-05-08T05:42:58.577971+0000","last_became_peered":"2024-05-08T05:42:58.577971+0000","last_unstale":"2024-05-08T05:43:00.886017+0000","last_undegraded":"2024-05-08T05:43:00.886017+0000","last_fullsized":"2024-05-08T05:43:00.886017+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:39.779366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.582750+0000","last_change":"2024-05-08T05:42:59.581498+0000","last_active":"2024-05-08T05:43:00.582750+0000","last_peered":"2024-05-08T05:43:00.582750+0000","last_clean":"2024-05-08T05:43:00.582750+0000","last_became_active":"2024-05-08T05:42:59.581156+0000","last_became_peered":"2024-05-08T05:42:59.581156+0000","last_unstale":"2024-05-08T05:43:00.582750+0000","last_undegraded":"2024-05-08T05:43:00.582750+0000","last_fullsized":"2024-05-08T05:43:00.582750+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:12:38.601549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.162684+0000","last_change":"2024-05-08T05:42:58.578763+0000","last_active":"2024-05-08T05:43:01.162684+0000","last_peered":"2024-05-08T05:43:01.162684+0000","last_clean":"2024-05-08T05:43:01.162684+0000","last_became_active":"2024-05-08T05:42:58.578430+0000","last_became_peered":"2024-05-08T05:42:58.578430+0000","last_unstale":"2024-05-08T05:43:01.162684+0000","last_undegraded":"2024-05-08T05:43:01.162684+0000","last_fullsized":"2024-05-08T05:43:01.162684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:25.616180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:01.161950+0000","last_change":"2024-05-08T05:42:59.592843+0000","last_active":"2024-05-08T05:43:01.161950+0000","last_peered":"2024-05-08T05:43:01.161950+0000","last_clean":"2024-05-08T05:43:01.161950+0000","last_became_active":"2024-05-08T05:42:59.592317+0000","last_became_peered":"2024-05-08T05:42:59.592317+0000","last_unstale":"2024-05-08T05:43:01.161950+0000","last_undegraded":"2024-05-08T05:43:01.161950+0000","last_fullsized":"2024-05-08T05:43:01.161950+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:58.552624+0000","last_clean_scrub_stamp":"2024-05-08T05:42:58.552624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:27.802918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"47'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.585182+0000","last_change":"2024-05-08T05:42:58.576321+0000","last_active":"2024-05-08T05:43:00.585182+0000","last_peered":"2024-05-08T05:43:00.585182+0000","last_clean":"2024-05-08T05:43:00.585182+0000","last_became_active":"2024-05-08T05:42:58.576057+0000","last_became_peered":"2024-05-08T05:42:58.576057+0000","last_unstale":"2024-05-08T05:43:00.585182+0000","last_undegraded":"2024-05-08T05:43:00.585182+0000","last_fullsized":"2024-05-08T05:43:00.585182+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:42:57.544689+0000","last_clean_scrub_stamp":"2024-05-08T05:42:57.544689+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:26.794984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":159,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-05-08T05:43:00.588005+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:43:00.588005+0000","last_peered":"2024-05-08T05:43:00.588005+0000","last_clean":"2024-05-08T05:43:00.588005+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:43:00.588005+0000","last_undegraded":"2024-05-08T05:43:00.588005+0000","last_fullsized":"2024-05-08T05:43:00.588005+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542169,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1155072,"data_stored":832475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542170,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":561152,"data_stored":242635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542169,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":552960,"data_stored":241983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":242565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":569344,"data_stored":243617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":565248,"data_stored":243595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":561152,"data_stored":242143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1155072,"data_stored":832407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:43:08.479 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T05:43:08.479 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T05:43:08.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.626+0000 7f76b2778640 1 Processor -- start 2024-05-08T05:43:08.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.626+0000 7f76b2778640 1 -- start start 2024-05-08T05:43:08.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.627+0000 7f76b2778640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f76ac06e0f0 0x7f76ac0cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:08.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.627+0000 7f76b2778640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76ac15d2a0 con 0x7f76ac06f060 2024-05-08T05:43:08.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.627+0000 7f76b2778640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76ac1609b0 con 0x7f76ac0d0360 2024-05-08T05:43:08.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.627+0000 7f76b2778640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f76ac0d07c0 con 0x7f76ac06e0f0 2024-05-08T05:43:08.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.627+0000 7f76ab7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f76ac06e0f0 0x7f76ac0cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:08.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.627+0000 7f76ab7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f76ac06e0f0 0x7f76ac0cfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47448/0 (socket says 172.21.15.45:47448) 2024-05-08T05:43:08.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.627+0000 7f76abfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f76ac06f060 0x7f76ac06fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37466/0 (socket says 172.21.15.45:37466) 2024-05-08T05:43:08.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.627+0000 7f76ab7fe640 1 -- 172.21.15.45:0/3921270039 learned_addr learned my addr 172.21.15.45:0/3921270039 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:08.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.628+0000 7f76ab7fe640 1 -- 172.21.15.45:0/3921270039 >> v1:172.21.15.63:6789/0 conn(0x7f76ac0d0360 legacy=0x7f76ac0b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:08.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.628+0000 7f76aaffd640 1 -- 172.21.15.45:0/3921270039 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4064568343 0 0) 0x7f76ac15d2a0 con 0x7f76ac06f060 2024-05-08T05:43:08.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.628+0000 7f76ab7fe640 1 -- 172.21.15.45:0/3921270039 >> v1:172.21.15.45:6789/0 conn(0x7f76ac06f060 legacy=0x7f76ac06fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:08.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.628+0000 7f76ab7fe640 1 -- 172.21.15.45:0/3921270039 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac160c90 con 0x7f76ac06e0f0 2024-05-08T05:43:08.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.628+0000 7f76ab7fe640 1 --2- 172.21.15.45:0/3921270039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76ac06e0f0 0x7f76ac0cfe20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009870 tx=0x7f76a002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f0024fa093135f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:08.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.628+0000 7f76aaffd640 1 -- 172.21.15.45:0/3921270039 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0037070 con 0x7f76ac06e0f0 2024-05-08T05:43:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.629+0000 7f76aaffd640 1 -- 172.21.15.45:0/3921270039 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a002ae50 con 0x7f76ac06e0f0 2024-05-08T05:43:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.629+0000 7f76aaffd640 1 -- 172.21.15.45:0/3921270039 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0032860 con 0x7f76ac06e0f0 2024-05-08T05:43:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.629+0000 7f76b2778640 1 -- 172.21.15.45:0/3921270039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76ac06e0f0 msgr2=0x7f76ac0cfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.629+0000 7f76b2778640 1 --2- 172.21.15.45:0/3921270039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76ac06e0f0 0x7f76ac0cfe20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009870 tx=0x7f76a002a300 comp rx=0 tx=0).stop 2024-05-08T05:43:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.629+0000 7f76b2778640 1 -- 172.21.15.45:0/3921270039 shutdown_connections 2024-05-08T05:43:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.629+0000 7f76b2778640 1 --2- 172.21.15.45:0/3921270039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76ac06e0f0 0x7f76ac0cfe20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.629+0000 7f76b2778640 1 -- 172.21.15.45:0/3921270039 >> 172.21.15.45:0/3921270039 conn(0x7f76ac06c3a0 msgr2=0x7f76ac06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.629+0000 7f76b2778640 1 -- 172.21.15.45:0/3921270039 shutdown_connections 2024-05-08T05:43:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.629+0000 7f76b2778640 1 -- 172.21.15.45:0/3921270039 wait complete. 2024-05-08T05:43:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.630+0000 7f76b2778640 1 Processor -- start 2024-05-08T05:43:08.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.630+0000 7f76b2778640 1 -- start start 2024-05-08T05:43:08.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.631+0000 7f76b2778640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76ac06e0f0 0x7f76ac0c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:08.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.631+0000 7f76b2778640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76ac06f060 0x7f76ac09cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:08.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.631+0000 7f76b2778640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76ac0d0360 0x7f76ac09d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:08.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.631+0000 7f76b2778640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac161250 con 0x7f76ac06f060 2024-05-08T05:43:08.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.631+0000 7f76b2778640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f76ac06f440 con 0x7f76ac0d0360 2024-05-08T05:43:08.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.631+0000 7f76b2778640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f76ac161530 con 0x7f76ac06e0f0 2024-05-08T05:43:08.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.631+0000 7f76ab7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76ac06f060 0x7f76ac09cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:08.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.631+0000 7f76ab7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76ac06f060 0x7f76ac09cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47464/0 (socket says 172.21.15.45:47464) 2024-05-08T05:43:08.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.631+0000 7f76ab7fe640 1 -- 172.21.15.45:0/4172750255 learned_addr learned my addr 172.21.15.45:0/4172750255 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:08.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.632+0000 7f76ab7fe640 1 -- 172.21.15.45:0/4172750255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76ac06e0f0 msgr2=0x7f76ac0c5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:08.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.632+0000 7f76ab7fe640 1 --2- 172.21.15.45:0/4172750255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76ac06e0f0 0x7f76ac0c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.632+0000 7f76ab7fe640 1 -- 172.21.15.45:0/4172750255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76ac0d0360 msgr2=0x7f76ac09d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:08.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.632+0000 7f76ab7fe640 1 --2- 172.21.15.45:0/4172750255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76ac0d0360 0x7f76ac09d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.632+0000 7f76ab7fe640 1 -- 172.21.15.45:0/4172750255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ac158180 con 0x7f76ac06f060 2024-05-08T05:43:08.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.632+0000 7f76ab7fe640 1 --2- 172.21.15.45:0/4172750255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76ac06f060 0x7f76ac09cbc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009870 tx=0x7f76a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:08.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.632+0000 7f76a8ff9640 1 -- 172.21.15.45:0/4172750255 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0041070 con 0x7f76ac06f060 2024-05-08T05:43:08.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.632+0000 7f76a8ff9640 1 -- 172.21.15.45:0/4172750255 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a003c870 con 0x7f76ac06f060 2024-05-08T05:43:08.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.632+0000 7f76b2778640 1 -- 172.21.15.45:0/4172750255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76ac0c66c0 con 0x7f76ac06f060 2024-05-08T05:43:08.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.632+0000 7f76a8ff9640 1 -- 172.21.15.45:0/4172750255 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0037040 con 0x7f76ac06f060 2024-05-08T05:43:08.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.632+0000 7f76b2778640 1 -- 172.21.15.45:0/4172750255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76ac0b15a0 con 0x7f76ac06f060 2024-05-08T05:43:08.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.633+0000 7f76b2778640 1 -- 172.21.15.45:0/4172750255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76ac0693f0 con 0x7f76ac06f060 2024-05-08T05:43:08.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.634+0000 7f76a8ff9640 1 -- 172.21.15.45:0/4172750255 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f76a0044070 con 0x7f76ac06f060 2024-05-08T05:43:08.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.634+0000 7f76a8ff9640 1 --2- 172.21.15.45:0/4172750255 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f76980762d0 0x7f7698078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:08.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.634+0000 7f76a8ff9640 1 -- 172.21.15.45:0/4172750255 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7f76a00b7ea0 con 0x7f76ac06f060 2024-05-08T05:43:08.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.635+0000 7f76abfff640 1 --2- 172.21.15.45:0/4172750255 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f76980762d0 0x7f7698078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:08.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.635+0000 7f76abfff640 1 --2- 172.21.15.45:0/4172750255 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f76980762d0 0x7f7698078790 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f769c000c00 tx=0x7f769c0078d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:08.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.639+0000 7f76a8ff9640 1 -- 172.21.15.45:0/4172750255 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f76ac158180 con 0x7f76ac06f060 2024-05-08T05:43:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.905+0000 7f76b2778640 1 -- 172.21.15.45:0/4172750255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f76ac0b5800 con 0x7f76ac06f060 2024-05-08T05:43:08.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.908+0000 7f76a8ff9640 1 -- 172.21.15.45:0/4172750255 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) ==== 190+0+147 (secure 0 0 0) 0x7f76a0081700 con 0x7f76ac06f060 2024-05-08T05:43:08.908 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-05-08T05:43:08.908 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBUEDtmDcqGMxAALgQaBYp4Zd8uMqswCXvGaw== 2024-05-08T05:43:08.908 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2024-05-08T05:43:08.908 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2024-05-08T05:43:08.908 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-08T05:43:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2024-05-08T05:43:08.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.910+0000 7f76b2778640 1 -- 172.21.15.45:0/4172750255 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f76980762d0 msgr2=0x7f7698078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:08.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.910+0000 7f76b2778640 1 --2- 172.21.15.45:0/4172750255 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f76980762d0 0x7f7698078790 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f769c000c00 tx=0x7f769c0078d0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.910+0000 7f76b2778640 1 -- 172.21.15.45:0/4172750255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76ac06f060 msgr2=0x7f76ac09cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:08.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.910+0000 7f76b2778640 1 --2- 172.21.15.45:0/4172750255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76ac06f060 0x7f76ac09cbc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009870 tx=0x7f76a00046c0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.910+0000 7f76b2778640 1 -- 172.21.15.45:0/4172750255 shutdown_connections 2024-05-08T05:43:08.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.910+0000 7f76b2778640 1 --2- 172.21.15.45:0/4172750255 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f76980762d0 0x7f7698078790 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.910+0000 7f76b2778640 1 --2- 172.21.15.45:0/4172750255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76ac0d0360 0x7f76ac09d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.910+0000 7f76b2778640 1 --2- 172.21.15.45:0/4172750255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76ac06f060 0x7f76ac09cbc0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.910+0000 7f76b2778640 1 --2- 172.21.15.45:0/4172750255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f76ac06e0f0 0x7f76ac0c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.910+0000 7f76b2778640 1 -- 172.21.15.45:0/4172750255 >> 172.21.15.45:0/4172750255 conn(0x7f76ac06c3a0 msgr2=0x7f76ac15ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:08.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.910+0000 7f76b2778640 1 -- 172.21.15.45:0/4172750255 shutdown_connections 2024-05-08T05:43:08.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:08.910+0000 7f76b2778640 1 -- 172.21.15.45:0/4172750255 wait complete. 2024-05-08T05:43:08.922 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:43:09.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.026+0000 7faf5556a640 1 Processor -- start 2024-05-08T05:43:09.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.026+0000 7faf5556a640 1 -- start start 2024-05-08T05:43:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.027+0000 7faf5556a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faf50070d30 0x7faf500d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.027+0000 7faf5556a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf50163920 0x7faf500703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.027+0000 7faf5556a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf50163d80 con 0x7faf50071110 2024-05-08T05:43:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.027+0000 7faf5556a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7faf500d10d0 con 0x7faf50070d30 2024-05-08T05:43:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.027+0000 7faf5556a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faf50070b80 con 0x7faf50163920 2024-05-08T05:43:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.027+0000 7faf4effd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faf50070d30 0x7faf500d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.027+0000 7faf4f7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7faf50071110 0x7faf5006d4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37472/0 (socket says 172.21.15.45:37472) 2024-05-08T05:43:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.027+0000 7faf4effd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faf50070d30 0x7faf500d20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47476/0 (socket says 172.21.15.45:47476) 2024-05-08T05:43:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.027+0000 7faf4f7fe640 1 -- 172.21.15.45:0/2402039366 learned_addr learned my addr 172.21.15.45:0/2402039366 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.027+0000 7faf4effd640 1 -- 172.21.15.45:0/2402039366 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf50163920 msgr2=0x7faf500703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.027+0000 7faf4effd640 1 --2- 172.21.15.45:0/2402039366 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf50163920 0x7faf500703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.027+0000 7faf4effd640 1 -- 172.21.15.45:0/2402039366 >> v1:172.21.15.45:6789/0 conn(0x7faf50071110 legacy=0x7faf5006d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.027+0000 7faf4effd640 1 -- 172.21.15.45:0/2402039366 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf500708e0 con 0x7faf50070d30 2024-05-08T05:43:09.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.027+0000 7faf4dffb640 1 -- 172.21.15.45:0/2402039366 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2987886735 0 0) 0x7faf50163d80 con 0x7faf50071110 2024-05-08T05:43:09.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.028+0000 7faf4effd640 1 --2- 172.21.15.45:0/2402039366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50070d30 0x7faf500d20c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7faf400097a0 tx=0x7faf4002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e6d962cbf9edb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:09.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.028+0000 7faf4dffb640 1 -- 172.21.15.45:0/2402039366 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40037070 con 0x7faf50070d30 2024-05-08T05:43:09.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.028+0000 7faf4dffb640 1 -- 172.21.15.45:0/2402039366 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf40032440 con 0x7faf50070d30 2024-05-08T05:43:09.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.028+0000 7faf4dffb640 1 -- 172.21.15.45:0/2402039366 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40032810 con 0x7faf50070d30 2024-05-08T05:43:09.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.029+0000 7faf5556a640 1 -- 172.21.15.45:0/2402039366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50070d30 msgr2=0x7faf500d20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.029+0000 7faf5556a640 1 --2- 172.21.15.45:0/2402039366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50070d30 0x7faf500d20c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7faf400097a0 tx=0x7faf4002a180 comp rx=0 tx=0).stop 2024-05-08T05:43:09.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.029+0000 7faf5556a640 1 -- 172.21.15.45:0/2402039366 shutdown_connections 2024-05-08T05:43:09.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.029+0000 7faf5556a640 1 --2- 172.21.15.45:0/2402039366 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf50163920 0x7faf500703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.029+0000 7faf5556a640 1 --2- 172.21.15.45:0/2402039366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50070d30 0x7faf500d20c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.029+0000 7faf5556a640 1 -- 172.21.15.45:0/2402039366 >> 172.21.15.45:0/2402039366 conn(0x7faf500d2e90 msgr2=0x7faf500d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:09.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.029+0000 7faf5556a640 1 -- 172.21.15.45:0/2402039366 shutdown_connections 2024-05-08T05:43:09.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.029+0000 7faf5556a640 1 -- 172.21.15.45:0/2402039366 wait complete. 2024-05-08T05:43:09.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.030+0000 7faf5556a640 1 Processor -- start 2024-05-08T05:43:09.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.030+0000 7faf5556a640 1 -- start start 2024-05-08T05:43:09.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.031+0000 7faf5556a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50070d30 0x7faf500b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.031+0000 7faf5556a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf50071110 0x7faf500b7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.031+0000 7faf5556a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf50163920 0x7faf5014a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.031+0000 7faf5556a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faf500d15e0 con 0x7faf50070d30 2024-05-08T05:43:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.031+0000 7faf5556a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faf500d1c30 con 0x7faf50071110 2024-05-08T05:43:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.031+0000 7faf5556a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faf500d18c0 con 0x7faf50163920 2024-05-08T05:43:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.031+0000 7faf4effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50070d30 0x7faf500b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.031+0000 7faf4effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50070d30 0x7faf500b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47486/0 (socket says 172.21.15.45:47486) 2024-05-08T05:43:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.031+0000 7faf4effd640 1 -- 172.21.15.45:0/1583776726 learned_addr learned my addr 172.21.15.45:0/1583776726 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.031+0000 7faf4effd640 1 -- 172.21.15.45:0/1583776726 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf50163920 msgr2=0x7faf5014a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.031+0000 7faf4effd640 1 --2- 172.21.15.45:0/1583776726 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf50163920 0x7faf5014a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.031+0000 7faf4effd640 1 -- 172.21.15.45:0/1583776726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf50071110 msgr2=0x7faf500b7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.031+0000 7faf4effd640 1 --2- 172.21.15.45:0/1583776726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf50071110 0x7faf500b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.031+0000 7faf4effd640 1 -- 172.21.15.45:0/1583776726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf500b6660 con 0x7faf50070d30 2024-05-08T05:43:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.032+0000 7faf4effd640 1 --2- 172.21.15.45:0/1583776726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50070d30 0x7faf500b5170 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7faf400097a0 tx=0x7faf400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.032+0000 7faf377fe640 1 -- 172.21.15.45:0/1583776726 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40041070 con 0x7faf50070d30 2024-05-08T05:43:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.032+0000 7faf377fe640 1 -- 172.21.15.45:0/1583776726 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf4003c890 con 0x7faf50070d30 2024-05-08T05:43:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.032+0000 7faf5556a640 1 -- 172.21.15.45:0/1583776726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf500b6940 con 0x7faf50070d30 2024-05-08T05:43:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.032+0000 7faf377fe640 1 -- 172.21.15.45:0/1583776726 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40037040 con 0x7faf50070d30 2024-05-08T05:43:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.032+0000 7faf5556a640 1 -- 172.21.15.45:0/1583776726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf500b6f00 con 0x7faf50070d30 2024-05-08T05:43:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.033+0000 7faf5556a640 1 -- 172.21.15.45:0/1583776726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf500693f0 con 0x7faf50070d30 2024-05-08T05:43:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.034+0000 7faf377fe640 1 -- 172.21.15.45:0/1583776726 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7faf40044070 con 0x7faf50070d30 2024-05-08T05:43:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.034+0000 7faf377fe640 1 --2- 172.21.15.45:0/1583776726 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faf3c0762d0 0x7faf3c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.034+0000 7faf377fe640 1 -- 172.21.15.45:0/1583776726 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7faf400b83b0 con 0x7faf50070d30 2024-05-08T05:43:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.035+0000 7faf4e7fc640 1 --2- 172.21.15.45:0/1583776726 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faf3c0762d0 0x7faf3c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.035+0000 7faf4e7fc640 1 --2- 172.21.15.45:0/1583776726 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faf3c0762d0 0x7faf3c078790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faf500b82f0 tx=0x7faf44006cb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:09.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.039+0000 7faf377fe640 1 -- 172.21.15.45:0/1583776726 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7faf40081b80 con 0x7faf50070d30 2024-05-08T05:43:09.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.276+0000 7faf5556a640 1 -- 172.21.15.45:0/1583776726 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf500d3d70 con 0x7faf50070d30 2024-05-08T05:43:09.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.277+0000 7faf377fe640 1 -- 172.21.15.45:0/1583776726 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5181 (secure 0 0 0) 0x7faf40081480 con 0x7faf50070d30 2024-05-08T05:43:09.277 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:09.277 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4262647207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4262647207},{"type":"v1","addr":"172.21.15.63:6837","nonce":4262647207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5277,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2872370835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2872370835},{"type":"v1","addr":"172.21.15.45:6834","nonce":2872370835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1909488716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1909488716},{"type":"v1","addr":"172.21.15.45:6835","nonce":1909488716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:42:58.943500+0000","modified":"2024-05-08T05:43:01.967189+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6836/4205418057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":4205418057},{"type":"v1","addr":"172.21.15.63:6836","nonce":4205418057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5292,"qdb_cluster":[5292]},"id":3}]} 2024-05-08T05:43:09.278 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-05-08T05:43:09.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.279+0000 7faf5556a640 1 -- 172.21.15.45:0/1583776726 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faf3c0762d0 msgr2=0x7faf3c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.279+0000 7faf5556a640 1 --2- 172.21.15.45:0/1583776726 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faf3c0762d0 0x7faf3c078790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faf500b82f0 tx=0x7faf44006cb0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.279+0000 7faf5556a640 1 -- 172.21.15.45:0/1583776726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50070d30 msgr2=0x7faf500b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.279+0000 7faf5556a640 1 --2- 172.21.15.45:0/1583776726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50070d30 0x7faf500b5170 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7faf400097a0 tx=0x7faf400046c0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.279+0000 7faf5556a640 1 -- 172.21.15.45:0/1583776726 shutdown_connections 2024-05-08T05:43:09.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.279+0000 7faf5556a640 1 --2- 172.21.15.45:0/1583776726 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faf3c0762d0 0x7faf3c078790 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.279+0000 7faf5556a640 1 --2- 172.21.15.45:0/1583776726 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faf50163920 0x7faf5014a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.279+0000 7faf5556a640 1 --2- 172.21.15.45:0/1583776726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faf50071110 0x7faf500b7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.279+0000 7faf5556a640 1 --2- 172.21.15.45:0/1583776726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf50070d30 0x7faf500b5170 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.279+0000 7faf5556a640 1 -- 172.21.15.45:0/1583776726 >> 172.21.15.45:0/1583776726 conn(0x7faf500d2e90 msgr2=0x7faf5006d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:09.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.279+0000 7faf5556a640 1 -- 172.21.15.45:0/1583776726 shutdown_connections 2024-05-08T05:43:09.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.279+0000 7faf5556a640 1 -- 172.21.15.45:0/1583776726 wait complete. 2024-05-08T05:43:09.692 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:42:58.943500+0000', 'modified': '2024-05-08T05:43:01.967189+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5292}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5292': {'gid': 5292, 'name': 'b', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.63:6836/4205418057', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6834', 'nonce': 4205418057}, {'type': 'v1', 'addr': '172.21.15.63:6836', 'nonce': 4205418057}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5292, 'qdb_cluster': [5292]} 2024-05-08T05:43:09.693 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T05:43:09.693 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T05:43:09.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.802+0000 7f98e76e8640 1 Processor -- start 2024-05-08T05:43:09.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.802+0000 7f98e76e8640 1 -- start start 2024-05-08T05:43:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.803+0000 7f98e76e8640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f98e01606d0 0x7f98e00cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.803+0000 7f98e76e8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98e00cdf00 0x7f98e00ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.803+0000 7f98e76e8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98e00d15c0 con 0x7f98e00cf580 2024-05-08T05:43:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.803+0000 7f98e76e8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f98e00ce8d0 con 0x7f98e00cdf00 2024-05-08T05:43:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.803+0000 7f98e76e8640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f98e00d18a0 con 0x7f98e01606d0 2024-05-08T05:43:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.803+0000 7f98e4c83640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98e00cdf00 0x7f98e00ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.803+0000 7f98e4c83640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98e00cdf00 0x7f98e00ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47496/0 (socket says 172.21.15.45:47496) 2024-05-08T05:43:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.803+0000 7f98e4c83640 1 -- 172.21.15.45:0/3520756294 learned_addr learned my addr 172.21.15.45:0/3520756294 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.803+0000 7f98e5484640 1 --2- 172.21.15.45:0/3520756294 >> v2:172.21.15.63:3300/0 conn(0x7f98e01606d0 0x7f98e00cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.803+0000 7f98e4c83640 1 -- 172.21.15.45:0/3520756294 >> v2:172.21.15.63:3300/0 conn(0x7f98e01606d0 msgr2=0x7f98e00cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.803+0000 7f98e4c83640 1 --2- 172.21.15.45:0/3520756294 >> v2:172.21.15.63:3300/0 conn(0x7f98e01606d0 0x7f98e00cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.803+0000 7f98e4c83640 1 -- 172.21.15.45:0/3520756294 >> v1:172.21.15.63:6789/0 conn(0x7f98e00cf580 legacy=0x7f98e00cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.803+0000 7f98e4c83640 1 -- 172.21.15.45:0/3520756294 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e00d1b80 con 0x7f98e00cdf00 2024-05-08T05:43:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.804+0000 7f98e4c83640 1 --2- 172.21.15.45:0/3520756294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e00cdf00 0x7f98e00ce390 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f98e00731f0 tx=0x7f98d002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2693c520c89c5f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.804+0000 7f98d7fff640 1 -- 172.21.15.45:0/3520756294 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0037070 con 0x7f98e00cdf00 2024-05-08T05:43:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.804+0000 7f98d7fff640 1 -- 172.21.15.45:0/3520756294 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d002ae00 con 0x7f98e00cdf00 2024-05-08T05:43:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.804+0000 7f98d7fff640 1 -- 172.21.15.45:0/3520756294 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0032660 con 0x7f98e00cdf00 2024-05-08T05:43:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.805+0000 7f98e76e8640 1 -- 172.21.15.45:0/3520756294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e00cdf00 msgr2=0x7f98e00ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.805+0000 7f98e76e8640 1 --2- 172.21.15.45:0/3520756294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e00cdf00 0x7f98e00ce390 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f98e00731f0 tx=0x7f98d002a0b0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.805+0000 7f98e76e8640 1 -- 172.21.15.45:0/3520756294 shutdown_connections 2024-05-08T05:43:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.805+0000 7f98e76e8640 1 --2- 172.21.15.45:0/3520756294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e00cdf00 0x7f98e00ce390 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.805+0000 7f98e76e8640 1 --2- 172.21.15.45:0/3520756294 >> v2:172.21.15.63:3300/0 conn(0x7f98e01606d0 0x7f98e00cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.805+0000 7f98e76e8640 1 -- 172.21.15.45:0/3520756294 >> 172.21.15.45:0/3520756294 conn(0x7f98e00b49d0 msgr2=0x7f98e00b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:09.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.805+0000 7f98e76e8640 1 -- 172.21.15.45:0/3520756294 shutdown_connections 2024-05-08T05:43:09.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.805+0000 7f98e76e8640 1 -- 172.21.15.45:0/3520756294 wait complete. 2024-05-08T05:43:09.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.806+0000 7f98e76e8640 1 Processor -- start 2024-05-08T05:43:09.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.806+0000 7f98e76e8640 1 -- start start 2024-05-08T05:43:09.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.807+0000 7f98e76e8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98e00cdf00 0x7f98e00cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.807+0000 7f98e76e8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98e00cf580 0x7f98e0157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.807+0000 7f98e76e8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e01606d0 0x7f98e0158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.807+0000 7f98e76e8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f98e0071a00 con 0x7f98e01606d0 2024-05-08T05:43:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.807+0000 7f98e76e8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f98e00b5710 con 0x7f98e00cdf00 2024-05-08T05:43:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.807+0000 7f98e76e8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f98e0071b80 con 0x7f98e00cf580 2024-05-08T05:43:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.807+0000 7f98e5c85640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e01606d0 0x7f98e0158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.807+0000 7f98e5c85640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e01606d0 0x7f98e0158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47504/0 (socket says 172.21.15.45:47504) 2024-05-08T05:43:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.807+0000 7f98e5c85640 1 -- 172.21.15.45:0/1259289975 learned_addr learned my addr 172.21.15.45:0/1259289975 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.807+0000 7f98e4c83640 1 --2- 172.21.15.45:0/1259289975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98e00cf580 0x7f98e0157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.807+0000 7f98e5484640 1 --2- 172.21.15.45:0/1259289975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98e00cdf00 0x7f98e00cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.807+0000 7f98e5c85640 1 -- 172.21.15.45:0/1259289975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98e00cf580 msgr2=0x7f98e0157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.807+0000 7f98e5c85640 1 --2- 172.21.15.45:0/1259289975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98e00cf580 0x7f98e0157f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.807+0000 7f98e5c85640 1 -- 172.21.15.45:0/1259289975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98e00cdf00 msgr2=0x7f98e00cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.807+0000 7f98e5c85640 1 --2- 172.21.15.45:0/1259289975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98e00cdf00 0x7f98e00cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.807+0000 7f98e5c85640 1 -- 172.21.15.45:0/1259289975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e00cade0 con 0x7f98e01606d0 2024-05-08T05:43:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.807+0000 7f98e5c85640 1 --2- 172.21.15.45:0/1259289975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e01606d0 0x7f98e0158460 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f98d800b990 tx=0x7f98d800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:09.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.807+0000 7f98d5ffb640 1 -- 172.21.15.45:0/1259289975 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8019070 con 0x7f98e01606d0 2024-05-08T05:43:09.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.807+0000 7f98d5ffb640 1 -- 172.21.15.45:0/1259289975 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d800cba0 con 0x7f98e01606d0 2024-05-08T05:43:09.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.808+0000 7f98e76e8640 1 -- 172.21.15.45:0/1259289975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e006e110 con 0x7f98e01606d0 2024-05-08T05:43:09.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.808+0000 7f98d5ffb640 1 -- 172.21.15.45:0/1259289975 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8014440 con 0x7f98e01606d0 2024-05-08T05:43:09.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.808+0000 7f98e76e8640 1 -- 172.21.15.45:0/1259289975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e006e6d0 con 0x7f98e01606d0 2024-05-08T05:43:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.809+0000 7f98d5ffb640 1 -- 172.21.15.45:0/1259289975 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f98d801c070 con 0x7f98e01606d0 2024-05-08T05:43:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.810+0000 7f98d5ffb640 1 --2- 172.21.15.45:0/1259289975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f98c40762d0 0x7f98c4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.810+0000 7f98d5ffb640 1 -- 172.21.15.45:0/1259289975 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7f98d80975b0 con 0x7f98e01606d0 2024-05-08T05:43:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.810+0000 7f98e5484640 1 --2- 172.21.15.45:0/1259289975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f98c40762d0 0x7f98c4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.811+0000 7f98ea406740 1 Processor -- start 2024-05-08T05:43:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.811+0000 7f98ea406740 1 -- start start 2024-05-08T05:43:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.811+0000 7f98e5484640 1 --2- 172.21.15.45:0/1259289975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f98c40762d0 0x7f98c4078790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f98dc006fd0 tx=0x7f98dc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.811+0000 7f98ea406740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61aaf50 0x564ec61ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.811+0000 7f98ea406740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61ab330 0x564ec61adca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.811+0000 7f98ea406740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec617dbb0 0x564ec61813c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.811+0000 7f98ea406740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564ec60e78d0 con 0x564ec617dbb0 2024-05-08T05:43:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.811+0000 7f98ea406740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564ec60a0c10 con 0x564ec61aaf50 2024-05-08T05:43:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.811+0000 7f98ea406740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564ec60e3890 con 0x564ec61ab330 2024-05-08T05:43:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.811+0000 7f98e5c85640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec617dbb0 0x564ec61813c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.812+0000 7f98e5c85640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec617dbb0 0x564ec61813c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47512/0 (socket says 172.21.15.45:47512) 2024-05-08T05:43:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.812+0000 7f98e5c85640 1 -- 172.21.15.45:0/1937650966 learned_addr learned my addr 172.21.15.45:0/1937650966 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.812+0000 7f98e4c83640 1 --2- 172.21.15.45:0/1937650966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61ab330 0x564ec61adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.812+0000 7f98e5484640 1 --2- 172.21.15.45:0/1937650966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61aaf50 0x564ec61ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.812+0000 7f98e5c85640 1 -- 172.21.15.45:0/1937650966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61ab330 msgr2=0x564ec61adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.812+0000 7f98e5c85640 1 --2- 172.21.15.45:0/1937650966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61ab330 0x564ec61adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.812+0000 7f98e5c85640 1 -- 172.21.15.45:0/1937650966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61aaf50 msgr2=0x564ec61ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.812+0000 7f98e5c85640 1 --2- 172.21.15.45:0/1937650966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61aaf50 0x564ec61ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.812+0000 7f98e5c85640 1 -- 172.21.15.45:0/1937650966 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ec60a85d0 con 0x564ec617dbb0 2024-05-08T05:43:09.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.812+0000 7f98e5484640 1 --2- 172.21.15.45:0/1937650966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61aaf50 0x564ec61ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.812+0000 7f98e5c85640 1 --2- 172.21.15.45:0/1937650966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec617dbb0 0x564ec61813c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f98d800c6a0 tx=0x7f98d805c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd8e37173168b34 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.812+0000 7f98caffd640 1 -- 172.21.15.45:0/1937650966 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8064990 con 0x564ec617dbb0 2024-05-08T05:43:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.812+0000 7f98caffd640 1 -- 172.21.15.45:0/1937650966 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d80693b0 con 0x564ec617dbb0 2024-05-08T05:43:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.812+0000 7f98caffd640 1 -- 172.21.15.45:0/1937650966 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8069120 con 0x564ec617dbb0 2024-05-08T05:43:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.813+0000 7f98ea406740 1 -- 172.21.15.45:0/1937650966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec617dbb0 msgr2=0x564ec61813c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.813+0000 7f98ea406740 1 --2- 172.21.15.45:0/1937650966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec617dbb0 0x564ec61813c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f98d800c6a0 tx=0x7f98d805c5d0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.813+0000 7f98ea406740 1 -- 172.21.15.45:0/1937650966 shutdown_connections 2024-05-08T05:43:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.813+0000 7f98ea406740 1 --2- 172.21.15.45:0/1937650966 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61ab330 0x564ec61adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.813+0000 7f98ea406740 1 --2- 172.21.15.45:0/1937650966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec617dbb0 0x564ec61813c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.813+0000 7f98ea406740 1 --2- 172.21.15.45:0/1937650966 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61aaf50 0x564ec61ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.813+0000 7f98ea406740 1 -- 172.21.15.45:0/1937650966 >> 172.21.15.45:0/1937650966 conn(0x564ec617fe20 msgr2=0x564ec61c8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.813+0000 7f98ea406740 1 -- 172.21.15.45:0/1937650966 shutdown_connections 2024-05-08T05:43:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.813+0000 7f98ea406740 1 -- 172.21.15.45:0/1937650966 wait complete. 2024-05-08T05:43:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.813+0000 7f98ea406740 1 -- start start 2024-05-08T05:43:09.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.813+0000 7f98ea406740 1 Processor -- start 2024-05-08T05:43:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.814+0000 7f98ea406740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61aaf50 0x564ec61a1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.814+0000 7f98ea406740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61ab330 0x564ec61a4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.814+0000 7f98ea406740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61a2190 0x564ec61a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.814+0000 7f98ea406740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564ec6182fd0 con 0x564ec61a2190 2024-05-08T05:43:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.814+0000 7f98ea406740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564ec60a4760 con 0x564ec61aaf50 2024-05-08T05:43:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.814+0000 7f98ea406740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564ec61832b0 con 0x564ec61ab330 2024-05-08T05:43:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.814+0000 7f98ea406740 10 client.0 _ll_register_callbacks cb 0x564ec60ddcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:43:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.814+0000 7f98e5c85640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61a2190 0x564ec61a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.814+0000 7f98e5c85640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61a2190 0x564ec61a4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47516/0 (socket says 172.21.15.45:47516) 2024-05-08T05:43:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.814+0000 7f98e5c85640 1 -- 172.21.15.45:0/3792670246 learned_addr learned my addr 172.21.15.45:0/3792670246 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.815+0000 7f98e5c85640 1 -- 172.21.15.45:0/3792670246 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61ab330 msgr2=0x564ec61a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:09.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.815+0000 7f98e4c83640 1 --2- 172.21.15.45:0/3792670246 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61ab330 0x564ec61a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.815+0000 7f98e5484640 1 --2- 172.21.15.45:0/3792670246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61aaf50 0x564ec61a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.815+0000 7f98e5c85640 1 --2- 172.21.15.45:0/3792670246 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61ab330 0x564ec61a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.815+0000 7f98e5c85640 1 -- 172.21.15.45:0/3792670246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61aaf50 msgr2=0x564ec61a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.815+0000 7f98e5c85640 1 --2- 172.21.15.45:0/3792670246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61aaf50 0x564ec61a1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.815+0000 7f98e5c85640 1 -- 172.21.15.45:0/3792670246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ec5df92d0 con 0x564ec61a2190 2024-05-08T05:43:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.815+0000 7f98e5c85640 1 --2- 172.21.15.45:0/3792670246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61a2190 0x564ec61a4c70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f98d800cfd0 tx=0x7f98d800c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97afc092c85b4aa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.815+0000 7f98bffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:43:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.815+0000 7f98bffff640 1 -- 172.21.15.45:0/3792670246 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d805cc30 con 0x564ec61a2190 2024-05-08T05:43:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.815+0000 7f98bffff640 1 -- 172.21.15.45:0/3792670246 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7f98d805f840 con 0x564ec61a2190 2024-05-08T05:43:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.815+0000 7f98ea406740 1 -- 172.21.15.45:0/3792670246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d80693b0 con 0x564ec61a2190 2024-05-08T05:43:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.815+0000 7f98bffff640 1 -- 172.21.15.45:0/3792670246 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f98d8064990 con 0x564ec61a2190 2024-05-08T05:43:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.815+0000 7f98ea406740 10 client.5454 fetch_fsmap learned FSMap version 26 2024-05-08T05:43:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.815+0000 7f98ea406740 1 -- 172.21.15.45:0/3792670246 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x564ec5ec36b0 con 0x564ec61a2190 2024-05-08T05:43:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.815+0000 7f98bffff640 1 -- 172.21.15.45:0/3792670246 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d80693b0 con 0x564ec61a2190 2024-05-08T05:43:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.815+0000 7f98bffff640 1 -- 172.21.15.45:0/3792670246 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f98d8065b20 con 0x564ec61a2190 2024-05-08T05:43:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.815+0000 7f98ea406740 10 client.5454 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T05:43:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.815+0000 7f98ea406740 10 client.5454 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:43:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.816+0000 7f98ea406740 20 client.5454 populate_metadata read hostname 'smithi045' 2024-05-08T05:43:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.816+0000 7f98ea406740 1 --2- 172.21.15.45:0/3792670246 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x564ec61bf780 0x564ec61bfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.816+0000 7f98ea406740 4 client.5454 mds_command: new command op to 5292 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:43:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.816+0000 7f98ea406740 1 -- 172.21.15.45:0/3792670246 --> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ec606b760 con 0x564ec61bf780 2024-05-08T05:43:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.816+0000 7f98e5484640 1 --2- 172.21.15.45:0/3792670246 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x564ec61bf780 0x564ec61bfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.816+0000 7f98e5484640 1 --2- 172.21.15.45:0/3792670246 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x564ec61bf780 0x564ec61bfb60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6473b0823cef079 server_cookie=f806a2bf173698c2 in_seq=0 out_seq=0 2024-05-08T05:43:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.816+0000 7f98bffff640 10 client.5454 ms_handle_connect on v2:172.21.15.63:6834/4205418057 2024-05-08T05:43:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.817+0000 7f98bffff640 1 -- 172.21.15.45:0/3792670246 <== mds.0 v2:172.21.15.63:6834/4205418057 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564ec606b760 con 0x564ec61bf780 2024-05-08T05:43:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.817+0000 7f98bffff640 10 client.5454 handle_command_reply: tid=0 2024-05-08T05:43:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.817+0000 7f98ea406740 1 client.5454 shutdown 2024-05-08T05:43:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.818+0000 7f98ea406740 1 -- 172.21.15.45:0/3792670246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61a2190 msgr2=0x564ec61a4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.818+0000 7f98ea406740 1 --2- 172.21.15.45:0/3792670246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61a2190 0x564ec61a4c70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f98d800cfd0 tx=0x7f98d800c610 comp rx=0 tx=0).stop 2024-05-08T05:43:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.818+0000 7f98ea406740 1 -- 172.21.15.45:0/3792670246 shutdown_connections 2024-05-08T05:43:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.818+0000 7f98ea406740 1 --2- 172.21.15.45:0/3792670246 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x564ec61bf780 0x564ec61bfb60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.818+0000 7f98ea406740 1 --2- 172.21.15.45:0/3792670246 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61ab330 0x564ec61a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.818+0000 7f98ea406740 1 --2- 172.21.15.45:0/3792670246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61a2190 0x564ec61a4c70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.818+0000 7f98bffff640 0 client.5454 ms_handle_reset on v2:172.21.15.63:6834/4205418057 2024-05-08T05:43:09.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.818+0000 7f98ea406740 1 --2- 172.21.15.45:0/3792670246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61aaf50 0x564ec61a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.818+0000 7f98ea406740 1 -- 172.21.15.45:0/3792670246 >> 172.21.15.45:0/3792670246 conn(0x564ec617dbb0 msgr2=0x564ec61aa340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:09.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.818+0000 7f98ea406740 1 -- 172.21.15.45:0/3792670246 shutdown_connections 2024-05-08T05:43:09.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.818+0000 7f98ea406740 1 -- 172.21.15.45:0/3792670246 wait complete. 2024-05-08T05:43:09.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.818+0000 7f98ea406740 20 client.5454 trim_cache size 0 max 16384 2024-05-08T05:43:09.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.837+0000 7f98ea406740 1 Processor -- start 2024-05-08T05:43:09.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.837+0000 7f98ea406740 1 -- start start 2024-05-08T05:43:09.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.837+0000 7f98ea406740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61a2190 0x564ec61aa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.837+0000 7f98ea406740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61aaf50 0x564ec61aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.837+0000 7f98ea406740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61ab330 0x564ec61a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.837+0000 7f98ea406740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f98e0200f90 con 0x564ec61aaf50 2024-05-08T05:43:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.837+0000 7f98ea406740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f98e00c7e70 con 0x564ec61ab330 2024-05-08T05:43:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.837+0000 7f98ea406740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f98e0200970 con 0x564ec61a2190 2024-05-08T05:43:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.837+0000 7f98e4c83640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61aaf50 0x564ec61aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.837+0000 7f98e4c83640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61aaf50 0x564ec61aa6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47518/0 (socket says 172.21.15.45:47518) 2024-05-08T05:43:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.837+0000 7f98e4c83640 1 -- 172.21.15.45:0/1973770851 learned_addr learned my addr 172.21.15.45:0/1973770851 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.838+0000 7f98e5484640 1 --2- 172.21.15.45:0/1973770851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61a2190 0x564ec61aa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.838+0000 7f98e5c85640 1 --2- 172.21.15.45:0/1973770851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61ab330 0x564ec61a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.838+0000 7f98e4c83640 1 -- 172.21.15.45:0/1973770851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61a2190 msgr2=0x564ec61aa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.838+0000 7f98e4c83640 1 --2- 172.21.15.45:0/1973770851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61a2190 0x564ec61aa190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.838+0000 7f98e4c83640 1 -- 172.21.15.45:0/1973770851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61ab330 msgr2=0x564ec61a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.838+0000 7f98e4c83640 1 --2- 172.21.15.45:0/1973770851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61ab330 0x564ec61a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.838+0000 7f98e4c83640 1 -- 172.21.15.45:0/1973770851 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ec61ae270 con 0x564ec61aaf50 2024-05-08T05:43:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.838+0000 7f98e5484640 1 --2- 172.21.15.45:0/1973770851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61a2190 0x564ec61aa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.838+0000 7f98e5c85640 1 --2- 172.21.15.45:0/1973770851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61ab330 0x564ec61a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.838+0000 7f98e4c83640 1 --2- 172.21.15.45:0/1973770851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61aaf50 0x564ec61aa6d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009960 tx=0x7f98d002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdcd9050ae484c5c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:09.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.838+0000 7f98ca7fc640 1 -- 172.21.15.45:0/1973770851 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0037040 con 0x564ec61aaf50 2024-05-08T05:43:09.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.838+0000 7f98ca7fc640 1 -- 172.21.15.45:0/1973770851 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d0032720 con 0x564ec61aaf50 2024-05-08T05:43:09.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.838+0000 7f98ca7fc640 1 -- 172.21.15.45:0/1973770851 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0032b10 con 0x564ec61aaf50 2024-05-08T05:43:09.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.839+0000 7f98ea406740 1 -- 172.21.15.45:0/1973770851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61aaf50 msgr2=0x564ec61aa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.839+0000 7f98ea406740 1 --2- 172.21.15.45:0/1973770851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61aaf50 0x564ec61aa6d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009960 tx=0x7f98d002c000 comp rx=0 tx=0).stop 2024-05-08T05:43:09.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.839+0000 7f98ea406740 1 -- 172.21.15.45:0/1973770851 shutdown_connections 2024-05-08T05:43:09.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.839+0000 7f98ea406740 1 --2- 172.21.15.45:0/1973770851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61ab330 0x564ec61a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.839+0000 7f98ea406740 1 --2- 172.21.15.45:0/1973770851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61aaf50 0x564ec61aa6d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.839+0000 7f98ea406740 1 --2- 172.21.15.45:0/1973770851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61a2190 0x564ec61aa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.839+0000 7f98ea406740 1 -- 172.21.15.45:0/1973770851 >> 172.21.15.45:0/1973770851 conn(0x564ec617dbb0 msgr2=0x564ec61bfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:09.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.839+0000 7f98ea406740 1 -- 172.21.15.45:0/1973770851 shutdown_connections 2024-05-08T05:43:09.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.839+0000 7f98ea406740 1 -- 172.21.15.45:0/1973770851 wait complete. 2024-05-08T05:43:09.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.839+0000 7f98ea406740 1 -- start start 2024-05-08T05:43:09.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.839+0000 7f98ea406740 1 Processor -- start 2024-05-08T05:43:09.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.840+0000 7f98ea406740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61a2190 0x564ec624a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.840+0000 7f98ea406740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61aaf50 0x564ec61cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.840+0000 7f98ea406740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61ab330 0x564ec61db4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.840+0000 7f98ea406740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564ec6182fd0 con 0x564ec61a2190 2024-05-08T05:43:09.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.840+0000 7f98ea406740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564ec60a4760 con 0x564ec61ab330 2024-05-08T05:43:09.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.840+0000 7f98ea406740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564ec61c43d0 con 0x564ec61aaf50 2024-05-08T05:43:09.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.840+0000 7f98ea406740 10 client.0 _ll_register_callbacks cb 0x7f98d0035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:43:09.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.840+0000 7f98e5484640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61a2190 0x564ec624a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.840+0000 7f98e5484640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61a2190 0x564ec624a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47524/0 (socket says 172.21.15.45:47524) 2024-05-08T05:43:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.840+0000 7f98e5484640 1 -- 172.21.15.45:0/2090864316 learned_addr learned my addr 172.21.15.45:0/2090864316 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98e4c83640 1 --2- 172.21.15.45:0/2090864316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61aaf50 0x564ec61cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98e5c85640 1 --2- 172.21.15.45:0/2090864316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61ab330 0x564ec61db4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98e5484640 1 -- 172.21.15.45:0/2090864316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61aaf50 msgr2=0x564ec61cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98e5484640 1 --2- 172.21.15.45:0/2090864316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61aaf50 0x564ec61cb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98e5484640 1 -- 172.21.15.45:0/2090864316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61ab330 msgr2=0x564ec61db4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98e5484640 1 --2- 172.21.15.45:0/2090864316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61ab330 0x564ec61db4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98e5484640 1 -- 172.21.15.45:0/2090864316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ec6182750 con 0x564ec61a2190 2024-05-08T05:43:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98e5c85640 1 --2- 172.21.15.45:0/2090864316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61ab330 0x564ec61db4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98e5484640 1 --2- 172.21.15.45:0/2090864316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61a2190 0x564ec624a730 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x564ec61c4ba0 tx=0x7f98dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1fb12a64d4b17b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98bffff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:43:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98bffff640 1 -- 172.21.15.45:0/2090864316 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc02d060 con 0x564ec61a2190 2024-05-08T05:43:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98bffff640 1 -- 172.21.15.45:0/2090864316 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7f98dc022db0 con 0x564ec61a2190 2024-05-08T05:43:09.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98ea406740 1 -- 172.21.15.45:0/2090864316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d0032720 con 0x564ec61a2190 2024-05-08T05:43:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98bffff640 1 -- 172.21.15.45:0/2090864316 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f98dc033050 con 0x564ec61a2190 2024-05-08T05:43:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98ea406740 10 client.5460 fetch_fsmap learned FSMap version 26 2024-05-08T05:43:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98ea406740 1 -- 172.21.15.45:0/2090864316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x564ec61acc30 con 0x564ec61a2190 2024-05-08T05:43:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98bffff640 1 -- 172.21.15.45:0/2090864316 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98d0032720 con 0x564ec61a2190 2024-05-08T05:43:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98bffff640 1 -- 172.21.15.45:0/2090864316 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f98dc036020 con 0x564ec61a2190 2024-05-08T05:43:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98ea406740 10 client.5460 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T05:43:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98ea406740 10 client.5460 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:43:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.841+0000 7f98ea406740 20 client.5460 populate_metadata read hostname 'smithi045' 2024-05-08T05:43:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.842+0000 7f98ea406740 1 --2- 172.21.15.45:0/2090864316 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x564ec6249620 0x564ec61be0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.842+0000 7f98ea406740 4 client.5460 mds_command: new command op to 5292 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:43:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.842+0000 7f98ea406740 1 -- 172.21.15.45:0/2090864316 --> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] -- command(tid 0: {"prefix": "status"}) -- 0x564ec61a3fc0 con 0x564ec6249620 2024-05-08T05:43:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.842+0000 7f98e4c83640 1 --2- 172.21.15.45:0/2090864316 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x564ec6249620 0x564ec61be0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.842+0000 7f98e4c83640 1 --2- 172.21.15.45:0/2090864316 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x564ec6249620 0x564ec61be0f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8d8c8813d47b944 server_cookie=f797be1a3828535 in_seq=0 out_seq=0 2024-05-08T05:43:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.842+0000 7f98bffff640 10 client.5460 ms_handle_connect on v2:172.21.15.63:6834/4205418057 2024-05-08T05:43:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.842+0000 7f98bffff640 1 -- 172.21.15.45:0/2090864316 <== mds.0 v2:172.21.15.63:6834/4205418057 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x564ec61a3fc0 con 0x564ec6249620 2024-05-08T05:43:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.842+0000 7f98bffff640 10 client.5460 handle_command_reply: tid=0 2024-05-08T05:43:09.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.842+0000 7f98ea406740 1 client.5460 shutdown 2024-05-08T05:43:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98ea406740 1 -- 172.21.15.45:0/2090864316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61a2190 msgr2=0x564ec624a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98ea406740 1 --2- 172.21.15.45:0/2090864316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61a2190 0x564ec624a730 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x564ec61c4ba0 tx=0x7f98dc004730 comp rx=0 tx=0).stop 2024-05-08T05:43:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98ea406740 1 -- 172.21.15.45:0/2090864316 shutdown_connections 2024-05-08T05:43:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98ea406740 1 --2- 172.21.15.45:0/2090864316 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x564ec6249620 0x564ec61be0f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98ea406740 1 --2- 172.21.15.45:0/2090864316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564ec61aaf50 0x564ec61cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98ea406740 1 --2- 172.21.15.45:0/2090864316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564ec61ab330 0x564ec61db4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98bffff640 0 client.5460 ms_handle_reset on v2:172.21.15.63:6834/4205418057 2024-05-08T05:43:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98ea406740 1 --2- 172.21.15.45:0/2090864316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564ec61a2190 0x564ec624a730 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98ea406740 1 -- 172.21.15.45:0/2090864316 >> 172.21.15.45:0/2090864316 conn(0x564ec617dbb0 msgr2=0x564ec61aa190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98ea406740 1 -- 172.21.15.45:0/2090864316 shutdown_connections 2024-05-08T05:43:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98ea406740 1 -- 172.21.15.45:0/2090864316 wait complete. 2024-05-08T05:43:09.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98ea406740 20 client.5460 trim_cache size 0 max 16384 2024-05-08T05:43:09.849 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:43:09.849 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "e9e1170a-fd27-4625-97a9-5d5f104d13a2", 2024-05-08T05:43:09.849 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-05-08T05:43:09.849 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 5292, 2024-05-08T05:43:09.849 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-05-08T05:43:09.849 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-05-08T05:43:09.849 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-05-08T05:43:09.849 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 10.265176641, 2024-05-08T05:43:09.849 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2024-05-08T05:43:09.849 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 47, 2024-05-08T05:43:09.849 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 47, 2024-05-08T05:43:09.849 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 15.251669849000001 2024-05-08T05:43:09.849 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:43:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98e76e8640 1 -- 172.21.15.45:0/1259289975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f98c40762d0 msgr2=0x7f98c4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98e76e8640 1 --2- 172.21.15.45:0/1259289975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f98c40762d0 0x7f98c4078790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f98dc006fd0 tx=0x7f98dc008040 comp rx=0 tx=0).stop 2024-05-08T05:43:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98e76e8640 1 -- 172.21.15.45:0/1259289975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e01606d0 msgr2=0x7f98e0158460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98e76e8640 1 --2- 172.21.15.45:0/1259289975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e01606d0 0x7f98e0158460 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f98d800b990 tx=0x7f98d800be50 comp rx=0 tx=0).stop 2024-05-08T05:43:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98e76e8640 1 -- 172.21.15.45:0/1259289975 shutdown_connections 2024-05-08T05:43:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98e76e8640 1 --2- 172.21.15.45:0/1259289975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f98c40762d0 0x7f98c4078790 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98e76e8640 1 --2- 172.21.15.45:0/1259289975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98e00cf580 0x7f98e0157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98e76e8640 1 --2- 172.21.15.45:0/1259289975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e01606d0 0x7f98e0158460 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98e76e8640 1 --2- 172.21.15.45:0/1259289975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98e00cdf00 0x7f98e00cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98e76e8640 1 -- 172.21.15.45:0/1259289975 >> 172.21.15.45:0/1259289975 conn(0x7f98e00b49d0 msgr2=0x7f98e00b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98e76e8640 1 -- 172.21.15.45:0/1259289975 shutdown_connections 2024-05-08T05:43:09.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.843+0000 7f98e76e8640 1 -- 172.21.15.45:0/1259289975 wait complete. 2024-05-08T05:43:09.854 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T05:43:09.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.958+0000 7f4c0ddd0640 1 Processor -- start 2024-05-08T05:43:09.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.958+0000 7f4c0ddd0640 1 -- start start 2024-05-08T05:43:09.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.959+0000 7f4c0ddd0640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4c080d1970 0x7f4c080d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.959+0000 7f4c0ddd0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c081630b0 con 0x7f4c080d4cc0 2024-05-08T05:43:09.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.959+0000 7f4c0ddd0640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c080d3e70 con 0x7f4c0815f420 2024-05-08T05:43:09.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.959+0000 7f4c0ddd0640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4c080d2550 con 0x7f4c080d1970 2024-05-08T05:43:09.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.959+0000 7f4c07fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4c080d4cc0 0x7f4c080d50e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:37474/0 (socket says 172.21.15.45:37474) 2024-05-08T05:43:09.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.959+0000 7f4c07fff640 1 -- 172.21.15.45:0/604041719 learned_addr learned my addr 172.21.15.45:0/604041719 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:09.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.959+0000 7f4c067fc640 1 -- 172.21.15.45:0/604041719 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4067161726 0 0) 0x7f4c00001490 con 0x7f4c080d4cc0 2024-05-08T05:43:09.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.959+0000 7f4c067fc640 1 -- 172.21.15.45:0/604041719 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bf0003520 con 0x7f4c080d4cc0 2024-05-08T05:43:09.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.959+0000 7f4c06ffd640 1 --2- 172.21.15.45:0/604041719 >> v2:172.21.15.63:3300/0 conn(0x7f4c080d1970 0x7f4c080d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.959+0000 7f4c067fc640 1 -- 172.21.15.45:0/604041719 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 70124595 0 0) 0x7f4bf0003520 con 0x7f4c080d4cc0 2024-05-08T05:43:09.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.959+0000 7f4c067fc640 1 -- 172.21.15.45:0/604041719 >> v2:172.21.15.63:3300/0 conn(0x7f4c080d1970 msgr2=0x7f4c080d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:09.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.959+0000 7f4c067fc640 1 --2- 172.21.15.45:0/604041719 >> v2:172.21.15.63:3300/0 conn(0x7f4c080d1970 0x7f4c080d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.959+0000 7f4c067fc640 1 -- 172.21.15.45:0/604041719 >> v1:172.21.15.63:6789/0 conn(0x7f4c0815f420 legacy=0x7f4c080d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.959+0000 7f4c067fc640 1 -- 172.21.15.45:0/604041719 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c080d2c80 con 0x7f4c080d4cc0 2024-05-08T05:43:09.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.959+0000 7f4c067fc640 1 -- 172.21.15.45:0/604041719 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f4c00001c40 con 0x7f4c080d4cc0 2024-05-08T05:43:09.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.959+0000 7f4c06ffd640 1 --2- 172.21.15.45:0/604041719 >> v2:172.21.15.63:3300/0 conn(0x7f4c080d1970 0x7f4c080d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:09.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.959+0000 7f4c067fc640 1 -- 172.21.15.45:0/604041719 >> v1:172.21.15.45:6789/0 conn(0x7f4c080d4cc0 legacy=0x7f4c080d50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.959+0000 7f4c067fc640 1 --2- 172.21.15.45:0/604041719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bf0003a50 0x7f4bf0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.960+0000 7f4c067fc640 1 --2- 172.21.15.45:0/604041719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bf0004800 0x7f4bf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.960+0000 7f4c067fc640 1 --2- 172.21.15.45:0/604041719 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4bf00052e0 0x7f4bf000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.960+0000 7f4c067fc640 1 -- 172.21.15.45:0/604041719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c080d2550 con 0x7f4bf0003a50 2024-05-08T05:43:09.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.960+0000 7f4c067fc640 1 -- 172.21.15.45:0/604041719 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf000c1b0 con 0x7f4bf0004800 2024-05-08T05:43:09.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.960+0000 7f4c067fc640 1 -- 172.21.15.45:0/604041719 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4bf000c3b0 con 0x7f4bf00052e0 2024-05-08T05:43:09.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.960+0000 7f4c077fe640 1 --2- 172.21.15.45:0/604041719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bf0003a50 0x7f4bf0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.960+0000 7f4c06ffd640 1 --2- 172.21.15.45:0/604041719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bf0004800 0x7f4bf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.960+0000 7f4c07fff640 1 --2- 172.21.15.45:0/604041719 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4bf00052e0 0x7f4bf000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.960+0000 7f4c077fe640 1 -- 172.21.15.45:0/604041719 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4bf00052e0 msgr2=0x7f4bf000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:09.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.960+0000 7f4c077fe640 1 --2- 172.21.15.45:0/604041719 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4bf00052e0 0x7f4bf000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.960+0000 7f4c077fe640 1 -- 172.21.15.45:0/604041719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bf0004800 msgr2=0x7f4bf0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:09.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.960+0000 7f4c077fe640 1 --2- 172.21.15.45:0/604041719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4bf0004800 0x7f4bf0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.960+0000 7f4c07fff640 1 -- 172.21.15.45:0/604041719 reap_dead start 2024-05-08T05:43:09.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.960+0000 7f4c077fe640 1 -- 172.21.15.45:0/604041719 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf0003520 con 0x7f4bf0003a50 2024-05-08T05:43:09.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.961+0000 7f4c077fe640 1 --2- 172.21.15.45:0/604041719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bf0003a50 0x7f4bf0003e70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8001760 tx=0x7f4bf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56416a6be85f664b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:09.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.961+0000 7f4c067fc640 1 -- 172.21.15.45:0/604041719 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf80013b0 con 0x7f4bf0003a50 2024-05-08T05:43:09.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.961+0000 7f4c067fc640 1 -- 172.21.15.45:0/604041719 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bf80052d0 con 0x7f4bf0003a50 2024-05-08T05:43:09.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.961+0000 7f4c067fc640 1 -- 172.21.15.45:0/604041719 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf802aa50 con 0x7f4bf0003a50 2024-05-08T05:43:09.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.961+0000 7f4c0ddd0640 1 -- 172.21.15.45:0/604041719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bf0003a50 msgr2=0x7f4bf0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.961+0000 7f4c0ddd0640 1 --2- 172.21.15.45:0/604041719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bf0003a50 0x7f4bf0003e70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8001760 tx=0x7f4bf0005830 comp rx=0 tx=0).stop 2024-05-08T05:43:09.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.961+0000 7f4c0ddd0640 1 -- 172.21.15.45:0/604041719 shutdown_connections 2024-05-08T05:43:09.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.961+0000 7f4c0ddd0640 1 --2- 172.21.15.45:0/604041719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bf0003a50 0x7f4bf0003e70 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.961+0000 7f4c0ddd0640 1 -- 172.21.15.45:0/604041719 >> 172.21.15.45:0/604041719 conn(0x7f4c0808eac0 msgr2=0x7f4c0809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.962+0000 7f4c0ddd0640 1 -- 172.21.15.45:0/604041719 shutdown_connections 2024-05-08T05:43:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.962+0000 7f4c0ddd0640 1 -- 172.21.15.45:0/604041719 wait complete. 2024-05-08T05:43:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.963+0000 7f4c0ddd0640 1 Processor -- start 2024-05-08T05:43:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.963+0000 7f4c0ddd0640 1 -- start start 2024-05-08T05:43:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.964+0000 7f4c0ddd0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bf0003a50 0x7f4c08154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.964+0000 7f4c0ddd0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c08155370 0x7f4c081f8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.964+0000 7f4c0ddd0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c081f9220 0x7f4c081f96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.964+0000 7f4c0ddd0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c080d2830 con 0x7f4bf0003a50 2024-05-08T05:43:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.964+0000 7f4c0ddd0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4c080d3240 con 0x7f4c08155370 2024-05-08T05:43:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.964+0000 7f4c0ddd0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4c080d1e20 con 0x7f4c081f9220 2024-05-08T05:43:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.964+0000 7f4c077fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bf0003a50 0x7f4c08154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.964+0000 7f4c077fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bf0003a50 0x7f4c08154e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:47532/0 (socket says 172.21.15.45:47532) 2024-05-08T05:43:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.964+0000 7f4c077fe640 1 -- 172.21.15.45:0/2261060190 learned_addr learned my addr 172.21.15.45:0/2261060190 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.964+0000 7f4c06ffd640 1 --2- 172.21.15.45:0/2261060190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c08155370 0x7f4c081f8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.964+0000 7f4c07fff640 1 --2- 172.21.15.45:0/2261060190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c081f9220 0x7f4c081f96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.964+0000 7f4c077fe640 1 -- 172.21.15.45:0/2261060190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c081f9220 msgr2=0x7f4c081f96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.964+0000 7f4c077fe640 1 --2- 172.21.15.45:0/2261060190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c081f9220 0x7f4c081f96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.964+0000 7f4c077fe640 1 -- 172.21.15.45:0/2261060190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c08155370 msgr2=0x7f4c081f8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:09.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.964+0000 7f4c077fe640 1 --2- 172.21.15.45:0/2261060190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c08155370 0x7f4c081f8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:09.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.964+0000 7f4c077fe640 1 -- 172.21.15.45:0/2261060190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c08161000 con 0x7f4bf0003a50 2024-05-08T05:43:09.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.964+0000 7f4c077fe640 1 --2- 172.21.15.45:0/2261060190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bf0003a50 0x7f4c08154e30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8001880 tx=0x7f4bf8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:09.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.965+0000 7f4beffff640 1 -- 172.21.15.45:0/2261060190 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf803f030 con 0x7f4bf0003a50 2024-05-08T05:43:09.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.965+0000 7f4beffff640 1 -- 172.21.15.45:0/2261060190 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bf8005480 con 0x7f4bf0003a50 2024-05-08T05:43:09.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.965+0000 7f4c0ddd0640 1 -- 172.21.15.45:0/2261060190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c080c9660 con 0x7f4bf0003a50 2024-05-08T05:43:09.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.965+0000 7f4beffff640 1 -- 172.21.15.45:0/2261060190 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8037040 con 0x7f4bf0003a50 2024-05-08T05:43:09.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.965+0000 7f4c0ddd0640 1 -- 172.21.15.45:0/2261060190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c080c9c20 con 0x7f4bf0003a50 2024-05-08T05:43:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.966+0000 7f4c0ddd0640 1 -- 172.21.15.45:0/2261060190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c08068c40 con 0x7f4bf0003a50 2024-05-08T05:43:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.966+0000 7f4beffff640 1 -- 172.21.15.45:0/2261060190 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4bf8042070 con 0x7f4bf0003a50 2024-05-08T05:43:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.967+0000 7f4beffff640 1 --2- 172.21.15.45:0/2261060190 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4bf4076300 0x7f4bf40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.967+0000 7f4beffff640 1 -- 172.21.15.45:0/2261060190 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7f4bf80b6ff0 con 0x7f4bf0003a50 2024-05-08T05:43:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.967+0000 7f4c06ffd640 1 --2- 172.21.15.45:0/2261060190 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4bf4076300 0x7f4bf40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:09.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.968+0000 7f4c06ffd640 1 --2- 172.21.15.45:0/2261060190 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4bf4076300 0x7f4bf40787c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4c080cf3b0 tx=0x7f4bfc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:09.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:09.971+0000 7f4beffff640 1 -- 172.21.15.45:0/2261060190 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4c08068c40 con 0x7f4bf0003a50 2024-05-08T05:43:10.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:10.205+0000 7f4c0ddd0640 1 -- 172.21.15.45:0/2261060190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4c080693f0 con 0x7f4bf0003a50 2024-05-08T05:43:10.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:10.206+0000 7f4beffff640 1 -- 172.21.15.45:0/2261060190 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f4bf8080740 con 0x7f4bf0003a50 2024-05-08T05:43:10.206 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T05:43:10.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:10.208+0000 7f4c0ddd0640 1 -- 172.21.15.45:0/2261060190 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4bf4076300 msgr2=0x7f4bf40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:10.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:10.208+0000 7f4c0ddd0640 1 --2- 172.21.15.45:0/2261060190 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4bf4076300 0x7f4bf40787c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4c080cf3b0 tx=0x7f4bfc008040 comp rx=0 tx=0).stop 2024-05-08T05:43:10.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:10.208+0000 7f4c0ddd0640 1 -- 172.21.15.45:0/2261060190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bf0003a50 msgr2=0x7f4c08154e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:10.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:10.208+0000 7f4c0ddd0640 1 --2- 172.21.15.45:0/2261060190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bf0003a50 0x7f4c08154e30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8001880 tx=0x7f4bf8001120 comp rx=0 tx=0).stop 2024-05-08T05:43:10.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:10.208+0000 7f4c0ddd0640 1 -- 172.21.15.45:0/2261060190 shutdown_connections 2024-05-08T05:43:10.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:10.208+0000 7f4c0ddd0640 1 --2- 172.21.15.45:0/2261060190 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4bf4076300 0x7f4bf40787c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:10.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:10.208+0000 7f4c0ddd0640 1 --2- 172.21.15.45:0/2261060190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c081f9220 0x7f4c081f96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:10.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:10.208+0000 7f4c0ddd0640 1 --2- 172.21.15.45:0/2261060190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c08155370 0x7f4c081f8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:10.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:10.208+0000 7f4c0ddd0640 1 --2- 172.21.15.45:0/2261060190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4bf0003a50 0x7f4c08154e30 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:10.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:10.208+0000 7f4c0ddd0640 1 -- 172.21.15.45:0/2261060190 >> 172.21.15.45:0/2261060190 conn(0x7f4c0808eac0 msgr2=0x7f4c0806cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:10.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:10.208+0000 7f4c0ddd0640 1 -- 172.21.15.45:0/2261060190 shutdown_connections 2024-05-08T05:43:10.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:10.208+0000 7f4c0ddd0640 1 -- 172.21.15.45:0/2261060190 wait complete. 2024-05-08T05:43:10.219 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T05:43:10.219 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:43:10.219 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-05-08T05:43:10.219 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T05:43:10.219 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:43:10.220 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T05:43:10.220 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T05:43:10.220 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:43:10.220 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:43:10.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:43:10.220 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-05-08T05:43:10.236 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:43:10.237 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:43:10.237 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:43:10.237 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:43:10.237 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-05-08T05:43:10.237 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:43:10.237 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:43:10.237 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:43:10.237 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-05-08T05:43:10.237 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:43:10.237 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:43:10.237 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-05-08T05:43:10.237 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:43:10.237 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-05-08T05:43:10.237 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-05-08T05:43:10.238 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:43:10.238 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42634sec preferred_lft 42634sec 2024-05-08T05:43:10.238 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-05-08T05:43:10.238 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:43:10.238 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:43:10.238 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-05-08T05:43:10.238 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-05-08T05:43:10.238 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:43:10.238 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T05:43:10.238 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T05:43:10.238 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:43:10.238 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c80a:aff:fe94:c77/64 scope link 2024-05-08T05:43:10.238 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:43:10.238 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:43:10.238 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 82:f3:eb:6a:57:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:43:10.239 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::80f3:ebff:fe6a:57ba/64 scope link 2024-05-08T05:43:10.239 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:43:10.239 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:43:10.239 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T05:43:10.239 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::14a2:64ff:feda:3a9c/64 scope link 2024-05-08T05:43:10.239 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:43:10.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:43:10.239 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:43:10.301 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:43:10.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:43:10.303 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-05-08T05:43:10.360 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-05-08T05:43:10.361 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:43:10.361 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:43:10.361 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:43:10.361 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:43:10.361 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:43:10.361 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:43:10.361 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:43:10.361 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:43:10.361 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:43:10.362 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:43:10.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:43:10.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:43:10.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:43:10.491 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:43:10.507 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T05:43:10.507 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T05:43:10.507 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:43:10.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:43:10.507 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-05-08T05:43:10.573 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:43:10.573 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T05:43:10.629 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:43:10.629 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:43:10.629 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:43:10.685 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T05:43:10.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:43:10.685 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:43:10.770 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:43:10.771 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:43:10.791 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:43:10.791 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:43:10.791 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-08T05:43:10.791 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@e9e1170a-fd27-4625-97a9-5d5f104d13a2.cephfs=/ 2024-05-08T05:43:10.791 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.63:3300/172.21.15.63:3301" will pass to kernel 2024-05-08T05:43:10.792 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T05:43:10.792 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:43:10.792 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:43:10.792 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:43:10.792 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:43:10.793 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:43:10.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:43:10.794 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:43:10.820 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:43:10.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:43:10.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:43:10.929 INFO:teuthology.orchestra.run.smithi045.stdout:client4922 2024-05-08T05:43:10.929 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:43:10.929 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:43:10.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:43:10.946 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-05-08T05:43:11.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:43:11.008 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-05-08T05:43:11.067 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:43:11.067 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:43:11.068 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:43:11.068 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:43:11.068 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-05-08T05:43:11.068 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:43:11.175 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627777 2024-05-08T05:43:11.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:43:11.184 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:43:11.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.328+0000 7f883f189640 1 Processor -- start 2024-05-08T05:43:11.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.328+0000 7f883f189640 1 -- start start 2024-05-08T05:43:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.329+0000 7f883f189640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f883815f420 0x7f88380d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.329+0000 7f883f189640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f88380d3ca0 0x7f88380d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.329+0000 7f883f189640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88380d1eb0 0x7f88380d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.329+0000 7f883f189640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f88381630b0 con 0x7f88380d3ca0 2024-05-08T05:43:11.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.329+0000 7f883f189640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f88380d2550 con 0x7f883815f420 2024-05-08T05:43:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.329+0000 7f883f189640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f88380d2c80 con 0x7f88380d1eb0 2024-05-08T05:43:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.329+0000 7f8837fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f88380d3ca0 0x7f88380d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.329+0000 7f8837fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f88380d3ca0 0x7f88380d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53992/0 (socket says 172.21.15.45:53992) 2024-05-08T05:43:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.329+0000 7f8837fff640 1 -- 172.21.15.45:0/1330312368 learned_addr learned my addr 172.21.15.45:0/1330312368 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.330+0000 7f883d726640 1 --2- 172.21.15.45:0/1330312368 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88380d1eb0 0x7f88380d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.330+0000 7f883cf25640 1 --2- 172.21.15.45:0/1330312368 >> v2:172.21.15.63:3300/0 conn(0x7f883815f420 0x7f88380d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.330+0000 7f8837fff640 1 -- 172.21.15.45:0/1330312368 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88380d1eb0 msgr2=0x7f88380d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.330+0000 7f8837fff640 1 --2- 172.21.15.45:0/1330312368 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88380d1eb0 0x7f88380d4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.330+0000 7f8837fff640 1 -- 172.21.15.45:0/1330312368 >> v2:172.21.15.63:3300/0 conn(0x7f883815f420 msgr2=0x7f88380d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.330+0000 7f8837fff640 1 --2- 172.21.15.45:0/1330312368 >> v2:172.21.15.63:3300/0 conn(0x7f883815f420 0x7f88380d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.330+0000 7f8837fff640 1 -- 172.21.15.45:0/1330312368 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88380d2f60 con 0x7f88380d3ca0 2024-05-08T05:43:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.330+0000 7f883d726640 1 --2- 172.21.15.45:0/1330312368 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88380d1eb0 0x7f88380d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:11.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.330+0000 7f883cf25640 1 --2- 172.21.15.45:0/1330312368 >> v2:172.21.15.63:3300/0 conn(0x7f883815f420 0x7f88380d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.330+0000 7f8837fff640 1 --2- 172.21.15.45:0/1330312368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88380d3ca0 0x7f88380d1970 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f88380d5ec0 tx=0x7f882c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff4961d058679d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.331+0000 7f88377fe640 1 -- 172.21.15.45:0/1330312368 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c037070 con 0x7f88380d3ca0 2024-05-08T05:43:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.331+0000 7f88377fe640 1 -- 172.21.15.45:0/1330312368 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f882c032440 con 0x7f88380d3ca0 2024-05-08T05:43:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.331+0000 7f88377fe640 1 -- 172.21.15.45:0/1330312368 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c032810 con 0x7f88380d3ca0 2024-05-08T05:43:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.331+0000 7f883f189640 1 -- 172.21.15.45:0/1330312368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88380d3ca0 msgr2=0x7f88380d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.331+0000 7f883f189640 1 --2- 172.21.15.45:0/1330312368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88380d3ca0 0x7f88380d1970 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f88380d5ec0 tx=0x7f882c02a1e0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.331+0000 7f883f189640 1 -- 172.21.15.45:0/1330312368 shutdown_connections 2024-05-08T05:43:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.331+0000 7f883f189640 1 --2- 172.21.15.45:0/1330312368 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88380d1eb0 0x7f88380d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.331+0000 7f883f189640 1 --2- 172.21.15.45:0/1330312368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88380d3ca0 0x7f88380d1970 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.331+0000 7f883f189640 1 --2- 172.21.15.45:0/1330312368 >> v2:172.21.15.63:3300/0 conn(0x7f883815f420 0x7f88380d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.331+0000 7f883f189640 1 -- 172.21.15.45:0/1330312368 >> 172.21.15.45:0/1330312368 conn(0x7f883808eac0 msgr2=0x7f883809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.331+0000 7f883f189640 1 -- 172.21.15.45:0/1330312368 shutdown_connections 2024-05-08T05:43:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.331+0000 7f883f189640 1 -- 172.21.15.45:0/1330312368 wait complete. 2024-05-08T05:43:11.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.332+0000 7f883f189640 1 Processor -- start 2024-05-08T05:43:11.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.332+0000 7f883f189640 1 -- start start 2024-05-08T05:43:11.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.333+0000 7f883f189640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88380d1eb0 0x7f883815e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.333+0000 7f883f189640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88380cb2b0 0x7f88380ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.333+0000 7f883f189640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88380cb730 0x7f88380cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.333+0000 7f883f189640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f88380d5200 con 0x7f88380cb2b0 2024-05-08T05:43:11.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.333+0000 7f883f189640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f88380d2830 con 0x7f88380cb730 2024-05-08T05:43:11.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.333+0000 7f883f189640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f88380d5380 con 0x7f88380d1eb0 2024-05-08T05:43:11.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.333+0000 7f8837fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88380cb2b0 0x7f88380ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.334+0000 7f8837fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88380cb2b0 0x7f88380ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53994/0 (socket says 172.21.15.45:53994) 2024-05-08T05:43:11.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.334+0000 7f8837fff640 1 -- 172.21.15.45:0/2000310092 learned_addr learned my addr 172.21.15.45:0/2000310092 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:11.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.334+0000 7f883cf25640 1 --2- 172.21.15.45:0/2000310092 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88380d1eb0 0x7f883815e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.334+0000 7f883d726640 1 --2- 172.21.15.45:0/2000310092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88380cb730 0x7f88380cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.334+0000 7f8837fff640 1 -- 172.21.15.45:0/2000310092 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88380d1eb0 msgr2=0x7f883815e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.334+0000 7f8837fff640 1 --2- 172.21.15.45:0/2000310092 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88380d1eb0 0x7f883815e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.334+0000 7f8837fff640 1 -- 172.21.15.45:0/2000310092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88380cb730 msgr2=0x7f88380cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.334+0000 7f8837fff640 1 --2- 172.21.15.45:0/2000310092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88380cb730 0x7f88380cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.334+0000 7f8837fff640 1 -- 172.21.15.45:0/2000310092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88380b7fa0 con 0x7f88380cb2b0 2024-05-08T05:43:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.334+0000 7f8837fff640 1 --2- 172.21.15.45:0/2000310092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88380cb2b0 0x7f88380ce6b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f882c02a220 tx=0x7f882c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.334+0000 7f88357fa640 1 -- 172.21.15.45:0/2000310092 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c037040 con 0x7f88380cb2b0 2024-05-08T05:43:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.334+0000 7f88357fa640 1 -- 172.21.15.45:0/2000310092 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f882c042070 con 0x7f88380cb2b0 2024-05-08T05:43:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.334+0000 7f88357fa640 1 -- 172.21.15.45:0/2000310092 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c030bb0 con 0x7f88380cb2b0 2024-05-08T05:43:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.334+0000 7f883f189640 1 -- 172.21.15.45:0/2000310092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88380b8280 con 0x7f88380cb2b0 2024-05-08T05:43:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.334+0000 7f883f189640 1 -- 172.21.15.45:0/2000310092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88380b8840 con 0x7f88380cb2b0 2024-05-08T05:43:11.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.335+0000 7f883f189640 1 -- 172.21.15.45:0/2000310092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88380693f0 con 0x7f88380cb2b0 2024-05-08T05:43:11.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.336+0000 7f88357fa640 1 -- 172.21.15.45:0/2000310092 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f882c047460 con 0x7f88380cb2b0 2024-05-08T05:43:11.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.336+0000 7f88357fa640 1 --2- 172.21.15.45:0/2000310092 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f88240762d0 0x7f8824078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.337+0000 7f88357fa640 1 -- 172.21.15.45:0/2000310092 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7f882c0b8270 con 0x7f88380cb2b0 2024-05-08T05:43:11.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.337+0000 7f883cf25640 1 --2- 172.21.15.45:0/2000310092 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f88240762d0 0x7f8824078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.337+0000 7f883cf25640 1 --2- 172.21.15.45:0/2000310092 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f88240762d0 0x7f8824078790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f88380d1ba0 tx=0x7f8830007770 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:11.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.341+0000 7f88357fa640 1 -- 172.21.15.45:0/2000310092 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f882c081480 con 0x7f88380cb2b0 2024-05-08T05:43:11.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.572+0000 7f883f189640 1 -- 172.21.15.45:0/2000310092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88380b4c50 con 0x7f88380cb2b0 2024-05-08T05:43:11.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.573+0000 7f88357fa640 1 -- 172.21.15.45:0/2000310092 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5181 (secure 0 0 0) 0x7f882c085780 con 0x7f88380cb2b0 2024-05-08T05:43:11.573 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:11.573 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4262647207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4262647207},{"type":"v1","addr":"172.21.15.63:6837","nonce":4262647207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5277,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2872370835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2872370835},{"type":"v1","addr":"172.21.15.45:6834","nonce":2872370835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1909488716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1909488716},{"type":"v1","addr":"172.21.15.45:6835","nonce":1909488716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:42:58.943500+0000","modified":"2024-05-08T05:43:01.967189+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6836/4205418057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":4205418057},{"type":"v1","addr":"172.21.15.63:6836","nonce":4205418057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5292,"qdb_cluster":[5292]},"id":3}]} 2024-05-08T05:43:11.573 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-05-08T05:43:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.575+0000 7f883f189640 1 -- 172.21.15.45:0/2000310092 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f88240762d0 msgr2=0x7f8824078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.575+0000 7f883f189640 1 --2- 172.21.15.45:0/2000310092 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f88240762d0 0x7f8824078790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f88380d1ba0 tx=0x7f8830007770 comp rx=0 tx=0).stop 2024-05-08T05:43:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.575+0000 7f883f189640 1 -- 172.21.15.45:0/2000310092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88380cb2b0 msgr2=0x7f88380ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.575+0000 7f883f189640 1 --2- 172.21.15.45:0/2000310092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88380cb2b0 0x7f88380ce6b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f882c02a220 tx=0x7f882c0046c0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.575+0000 7f883f189640 1 -- 172.21.15.45:0/2000310092 shutdown_connections 2024-05-08T05:43:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.575+0000 7f883f189640 1 --2- 172.21.15.45:0/2000310092 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f88240762d0 0x7f8824078790 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.575+0000 7f883f189640 1 --2- 172.21.15.45:0/2000310092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f88380cb730 0x7f88380cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.575+0000 7f883f189640 1 --2- 172.21.15.45:0/2000310092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88380cb2b0 0x7f88380ce6b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.575+0000 7f883f189640 1 --2- 172.21.15.45:0/2000310092 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88380d1eb0 0x7f883815e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.575+0000 7f883f189640 1 -- 172.21.15.45:0/2000310092 >> 172.21.15.45:0/2000310092 conn(0x7f883808eac0 msgr2=0x7f883806d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.575+0000 7f883f189640 1 -- 172.21.15.45:0/2000310092 shutdown_connections 2024-05-08T05:43:11.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.575+0000 7f883f189640 1 -- 172.21.15.45:0/2000310092 wait complete. 2024-05-08T05:43:11.587 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-05-08T05:43:11.726 INFO:teuthology.orchestra.run.smithi063.stdout:{"message":"","return_code":0} 2024-05-08T05:43:11.727 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-08T05:43:11.727 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-08T05:43:11.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.772+0000 7f2bb800ff40 1 Processor -- start 2024-05-08T05:43:11.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.772+0000 7f2bb800ff40 1 -- start start 2024-05-08T05:43:11.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.773+0000 7f2bb800ff40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ba00030f0 0x562605a3d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.773+0000 7f2bb800ff40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626059b1bf0 0x5626059b0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.773+0000 7f2bb800ff40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626059b16a0 0x5626059b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.773+0000 7f2bb800ff40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56260592df50 con 0x5626059b1bf0 2024-05-08T05:43:11.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.773+0000 7f2bb800ff40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562605931f90 con 0x5626059b16a0 2024-05-08T05:43:11.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.773+0000 7f2bb800ff40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x562605a3fd90 con 0x7f2ba00030f0 2024-05-08T05:43:11.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.773+0000 7f2bb4640640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626059b1bf0 0x5626059b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.773+0000 7f2bb4640640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626059b1bf0 0x5626059b0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54010/0 (socket says 172.21.15.45:54010) 2024-05-08T05:43:11.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.773+0000 7f2bb4640640 1 -- 172.21.15.45:0/1063356577 learned_addr learned my addr 172.21.15.45:0/1063356577 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:11.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.773+0000 7f2bb4e41640 1 --2- 172.21.15.45:0/1063356577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ba00030f0 0x562605a3d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.773+0000 7f2bb5642640 1 --2- 172.21.15.45:0/1063356577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626059b16a0 0x5626059b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.774+0000 7f2bb4640640 1 -- 172.21.15.45:0/1063356577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ba00030f0 msgr2=0x562605a3d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:11.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.774+0000 7f2bb4640640 1 --2- 172.21.15.45:0/1063356577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ba00030f0 0x562605a3d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.774+0000 7f2bb4640640 1 -- 172.21.15.45:0/1063356577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626059b16a0 msgr2=0x5626059b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:11.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.774+0000 7f2bb4640640 1 --2- 172.21.15.45:0/1063356577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626059b16a0 0x5626059b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.774+0000 7f2bb4640640 1 -- 172.21.15.45:0/1063356577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562605a3ffe0 con 0x5626059b1bf0 2024-05-08T05:43:11.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.774+0000 7f2bb4640640 1 --2- 172.21.15.45:0/1063356577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626059b1bf0 0x5626059b0850 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c057040 tx=0x7f2b9c04f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270de8b4e81a8eae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:11.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.774+0000 7f2bb363e640 1 -- 172.21.15.45:0/1063356577 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c0013f0 con 0x5626059b1bf0 2024-05-08T05:43:11.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.774+0000 7f2bb363e640 1 -- 172.21.15.45:0/1063356577 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b9c001590 con 0x5626059b1bf0 2024-05-08T05:43:11.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.774+0000 7f2bb363e640 1 -- 172.21.15.45:0/1063356577 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c04db50 con 0x5626059b1bf0 2024-05-08T05:43:11.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.774+0000 7f2bb800ff40 1 -- 172.21.15.45:0/1063356577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626059b1bf0 msgr2=0x5626059b0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.774+0000 7f2bb800ff40 1 --2- 172.21.15.45:0/1063356577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626059b1bf0 0x5626059b0850 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c057040 tx=0x7f2b9c04f000 comp rx=0 tx=0).stop 2024-05-08T05:43:11.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.775+0000 7f2bb800ff40 1 -- 172.21.15.45:0/1063356577 shutdown_connections 2024-05-08T05:43:11.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.775+0000 7f2bb800ff40 1 --2- 172.21.15.45:0/1063356577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5626059b16a0 0x5626059b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.775+0000 7f2bb800ff40 1 --2- 172.21.15.45:0/1063356577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626059b1bf0 0x5626059b0850 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.775+0000 7f2bb800ff40 1 --2- 172.21.15.45:0/1063356577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ba00030f0 0x562605a3d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.775+0000 7f2bb800ff40 1 -- 172.21.15.45:0/1063356577 >> 172.21.15.45:0/1063356577 conn(0x5626059857e0 msgr2=0x562605a3eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:11.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.775+0000 7f2bb800ff40 1 -- 172.21.15.45:0/1063356577 shutdown_connections 2024-05-08T05:43:11.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.775+0000 7f2bb800ff40 1 -- 172.21.15.45:0/1063356577 wait complete. 2024-05-08T05:43:11.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.775+0000 7f2bb800ff40 1 Processor -- start 2024-05-08T05:43:11.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.775+0000 7f2bb800ff40 1 -- start start 2024-05-08T05:43:11.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.776+0000 7f2bb800ff40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626059b16a0 0x5626059b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.776+0000 7f2bb800ff40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626059b1bf0 0x562605a58670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.776+0000 7f2bb800ff40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ba00030f0 0x562605a551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.776+0000 7f2bb800ff40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562605a402c0 con 0x5626059b16a0 2024-05-08T05:43:11.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.776+0000 7f2bb800ff40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562605a3db10 con 0x7f2ba00030f0 2024-05-08T05:43:11.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.776+0000 7f2bb800ff40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x562605a59a40 con 0x5626059b1bf0 2024-05-08T05:43:11.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.776+0000 7f2bb4e41640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626059b16a0 0x5626059b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.776+0000 7f2bb4e41640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626059b16a0 0x5626059b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54016/0 (socket says 172.21.15.45:54016) 2024-05-08T05:43:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.776+0000 7f2bb4e41640 1 -- 172.21.15.45:0/2821898513 learned_addr learned my addr 172.21.15.45:0/2821898513 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.776+0000 7f2bb4640640 1 --2- 172.21.15.45:0/2821898513 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626059b1bf0 0x562605a58670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.776+0000 7f2bb5642640 1 --2- 172.21.15.45:0/2821898513 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ba00030f0 0x562605a551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.776+0000 7f2bb4e41640 1 -- 172.21.15.45:0/2821898513 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626059b1bf0 msgr2=0x562605a58670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.776+0000 7f2bb4e41640 1 --2- 172.21.15.45:0/2821898513 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5626059b1bf0 0x562605a58670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.776+0000 7f2bb4e41640 1 -- 172.21.15.45:0/2821898513 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ba00030f0 msgr2=0x562605a551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.776+0000 7f2bb4e41640 1 --2- 172.21.15.45:0/2821898513 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ba00030f0 0x562605a551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.776+0000 7f2bb4e41640 1 -- 172.21.15.45:0/2821898513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562605a58bb0 con 0x5626059b16a0 2024-05-08T05:43:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.776+0000 7f2bb4e41640 1 --2- 172.21.15.45:0/2821898513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626059b16a0 0x5626059b4bb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40104b0 tx=0x7f2ba4010970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.777+0000 7f2bb263c640 1 -- 172.21.15.45:0/2821898513 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4018020 con 0x5626059b16a0 2024-05-08T05:43:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.777+0000 7f2bb263c640 1 -- 172.21.15.45:0/2821898513 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ba4006260 con 0x5626059b16a0 2024-05-08T05:43:11.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.777+0000 7f2bb800ff40 1 -- 172.21.15.45:0/2821898513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5626059b3b10 con 0x5626059b16a0 2024-05-08T05:43:11.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.777+0000 7f2bb263c640 1 -- 172.21.15.45:0/2821898513 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4014670 con 0x5626059b16a0 2024-05-08T05:43:11.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.777+0000 7f2bb800ff40 1 -- 172.21.15.45:0/2821898513 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626059b40d0 con 0x5626059b16a0 2024-05-08T05:43:11.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.778+0000 7f2bb263c640 1 -- 172.21.15.45:0/2821898513 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2ba401b020 con 0x5626059b16a0 2024-05-08T05:43:11.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.779+0000 7f2bb263c640 1 --2- 172.21.15.45:0/2821898513 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b94074cb0 0x7f2b94077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.779+0000 7f2bb263c640 1 -- 172.21.15.45:0/2821898513 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7f2ba4096870 con 0x5626059b16a0 2024-05-08T05:43:11.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.779+0000 7f2bb4640640 1 --2- 172.21.15.45:0/2821898513 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b94074cb0 0x7f2b94077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.779+0000 7f2bb800ff40 1 --2- 172.21.15.45:0/2821898513 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x562605a5e970 0x5626059cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.779+0000 7f2bb800ff40 1 -- 172.21.15.45:0/2821898513 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) -- 0x5626059cc100 con 0x562605a5e970 2024-05-08T05:43:11.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.779+0000 7f2bb5642640 1 --2- 172.21.15.45:0/2821898513 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x562605a5e970 0x5626059cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.779+0000 7f2bb4640640 1 --2- 172.21.15.45:0/2821898513 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b94074cb0 0x7f2b94077170 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c000f60 tx=0x7f2b9c051000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:11.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.779+0000 7f2bb5642640 1 --2- 172.21.15.45:0/2821898513 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x562605a5e970 0x5626059cbbc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:11.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.780+0000 7f2bb5642640 1 -- 172.21.15.45:0/2821898513 <== osd.1 v2:172.21.15.45:6824/1337179184 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f2bac009960 con 0x562605a5e970 2024-05-08T05:43:11.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.780+0000 7f2bb800ff40 1 -- 172.21.15.45:0/2821898513 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x562605a5e970 msgr2=0x5626059cbbc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.780+0000 7f2bb800ff40 1 --2- 172.21.15.45:0/2821898513 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x562605a5e970 0x5626059cbbc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.780+0000 7f2bb800ff40 1 -- 172.21.15.45:0/2821898513 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b94074cb0 msgr2=0x7f2b94077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.780+0000 7f2bb800ff40 1 --2- 172.21.15.45:0/2821898513 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b94074cb0 0x7f2b94077170 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c000f60 tx=0x7f2b9c051000 comp rx=0 tx=0).stop 2024-05-08T05:43:11.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.780+0000 7f2bb800ff40 1 -- 172.21.15.45:0/2821898513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626059b16a0 msgr2=0x5626059b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.780+0000 7f2bb800ff40 1 --2- 172.21.15.45:0/2821898513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5626059b16a0 0x5626059b4bb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40104b0 tx=0x7f2ba4010970 comp rx=0 tx=0).stop 2024-05-08T05:43:11.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.780+0000 7f2bb5642640 1 -- 172.21.15.45:0/2821898513 reap_dead start 2024-05-08T05:43:11.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.780+0000 7f2bb800ff40 1 -- 172.21.15.45:0/2821898513 shutdown_connections 2024-05-08T05:43:11.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.780+0000 7f2bb800ff40 1 -- 172.21.15.45:0/2821898513 >> 172.21.15.45:0/2821898513 conn(0x5626059857e0 msgr2=0x562605a5b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:11.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.780+0000 7f2bb800ff40 1 -- 172.21.15.45:0/2821898513 shutdown_connections 2024-05-08T05:43:11.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.780+0000 7f2bb800ff40 1 -- 172.21.15.45:0/2821898513 wait complete. 2024-05-08T05:43:11.786 INFO:teuthology.orchestra.run.smithi045.stdout:Yalphaparent_a 2024-05-08T05:43:11.786 DEBUG:teuthology.orchestra.run.smithi045:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-08T05:43:11.889 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:43:11.890 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 1099511627777, 2024-05-08T05:43:11.890 INFO:teuthology.orchestra.run.smithi045.stdout: "ancestors": [ 2024-05-08T05:43:11.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:11.890 INFO:teuthology.orchestra.run.smithi045.stdout: "dirino": 1099511627776, 2024-05-08T05:43:11.890 INFO:teuthology.orchestra.run.smithi045.stdout: "dname": "alpha", 2024-05-08T05:43:11.890 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 2 2024-05-08T05:43:11.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:43:11.890 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:11.890 INFO:teuthology.orchestra.run.smithi045.stdout: "dirino": 1, 2024-05-08T05:43:11.890 INFO:teuthology.orchestra.run.smithi045.stdout: "dname": "parent_a", 2024-05-08T05:43:11.891 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 4 2024-05-08T05:43:11.891 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:11.891 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-05-08T05:43:11.891 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 8, 2024-05-08T05:43:11.891 INFO:teuthology.orchestra.run.smithi045.stdout: "old_pools": [] 2024-05-08T05:43:11.891 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:43:11.891 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:11.901 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-05-08T05:43:11.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.947+0000 7fe7447d9f40 1 Processor -- start 2024-05-08T05:43:11.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.947+0000 7fe7447d9f40 1 -- start start 2024-05-08T05:43:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.948+0000 7fe7447d9f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6599afe80 0x55a659a3a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.948+0000 7fe7447d9f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a659a38390 0x55a659a4ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.948+0000 7fe7447d9f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a659a3b8b0 0x55a659a4f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.948+0000 7fe7447d9f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a659a56cd0 con 0x55a659a38390 2024-05-08T05:43:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.948+0000 7fe7447d9f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a659927f50 con 0x55a6599afe80 2024-05-08T05:43:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.948+0000 7fe7447d9f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55a659a39350 con 0x55a659a3b8b0 2024-05-08T05:43:11.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.948+0000 7fe741ad2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a659a38390 0x55a659a4ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.948+0000 7fe741ad2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a659a38390 0x55a659a4ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54024/0 (socket says 172.21.15.45:54024) 2024-05-08T05:43:11.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.948+0000 7fe741ad2640 1 -- 172.21.15.45:0/3836485346 learned_addr learned my addr 172.21.15.45:0/3836485346 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:11.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.948+0000 7fe742ad4640 1 --2- 172.21.15.45:0/3836485346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a659a3b8b0 0x55a659a4f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.948+0000 7fe7422d3640 1 --2- 172.21.15.45:0/3836485346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6599afe80 0x55a659a3a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.948+0000 7fe741ad2640 1 -- 172.21.15.45:0/3836485346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a659a3b8b0 msgr2=0x55a659a4f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:11.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.948+0000 7fe741ad2640 1 --2- 172.21.15.45:0/3836485346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a659a3b8b0 0x55a659a4f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.948+0000 7fe741ad2640 1 -- 172.21.15.45:0/3836485346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6599afe80 msgr2=0x55a659a3a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:11.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.948+0000 7fe741ad2640 1 --2- 172.21.15.45:0/3836485346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6599afe80 0x55a659a3a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.948+0000 7fe741ad2640 1 -- 172.21.15.45:0/3836485346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a659ae6410 con 0x55a659a38390 2024-05-08T05:43:11.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.948+0000 7fe7422d3640 1 --2- 172.21.15.45:0/3836485346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6599afe80 0x55a659a3a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:11.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.948+0000 7fe742ad4640 1 --2- 172.21.15.45:0/3836485346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a659a3b8b0 0x55a659a4f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.948+0000 7fe741ad2640 1 --2- 172.21.15.45:0/3836485346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a659a38390 0x55a659a4ec60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe72c005830 tx=0x7fe72c005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34500552121633d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.949+0000 7fe740ad0640 1 -- 172.21.15.45:0/3836485346 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c008870 con 0x55a659a38390 2024-05-08T05:43:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.949+0000 7fe740ad0640 1 -- 172.21.15.45:0/3836485346 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe72c018070 con 0x55a659a38390 2024-05-08T05:43:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.949+0000 7fe740ad0640 1 -- 172.21.15.45:0/3836485346 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c008c50 con 0x55a659a38390 2024-05-08T05:43:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.949+0000 7fe7447d9f40 1 -- 172.21.15.45:0/3836485346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a659a38390 msgr2=0x55a659a4ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.949+0000 7fe7447d9f40 1 --2- 172.21.15.45:0/3836485346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a659a38390 0x55a659a4ec60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe72c005830 tx=0x7fe72c005ac0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.949+0000 7fe7447d9f40 1 -- 172.21.15.45:0/3836485346 shutdown_connections 2024-05-08T05:43:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.949+0000 7fe7447d9f40 1 --2- 172.21.15.45:0/3836485346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a659a3b8b0 0x55a659a4f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.949+0000 7fe7447d9f40 1 --2- 172.21.15.45:0/3836485346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a659a38390 0x55a659a4ec60 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.949+0000 7fe7447d9f40 1 --2- 172.21.15.45:0/3836485346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a6599afe80 0x55a659a3a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.949+0000 7fe7447d9f40 1 -- 172.21.15.45:0/3836485346 >> 172.21.15.45:0/3836485346 conn(0x55a65997f7f0 msgr2=0x55a659a58200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.949+0000 7fe7447d9f40 1 -- 172.21.15.45:0/3836485346 shutdown_connections 2024-05-08T05:43:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.949+0000 7fe7447d9f40 1 -- 172.21.15.45:0/3836485346 wait complete. 2024-05-08T05:43:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.950+0000 7fe7447d9f40 1 Processor -- start 2024-05-08T05:43:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.950+0000 7fe7447d9f40 1 -- start start 2024-05-08T05:43:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.950+0000 7fe7447d9f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a6599afe80 0x55a659a377d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.950+0000 7fe7447d9f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a659a38390 0x55a659a4db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.950+0000 7fe7447d9f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a659a3b8b0 0x55a659a34870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.950+0000 7fe7447d9f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a6599a9880 con 0x55a659a3b8b0 2024-05-08T05:43:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.950+0000 7fe7447d9f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a659a591b0 con 0x55a659a38390 2024-05-08T05:43:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.950+0000 7fe7447d9f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55a6599c8af0 con 0x55a6599afe80 2024-05-08T05:43:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.950+0000 7fe742ad4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a659a3b8b0 0x55a659a34870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.951+0000 7fe742ad4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a659a3b8b0 0x55a659a34870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54030/0 (socket says 172.21.15.45:54030) 2024-05-08T05:43:11.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.951+0000 7fe742ad4640 1 -- 172.21.15.45:0/4162751554 learned_addr learned my addr 172.21.15.45:0/4162751554 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.951+0000 7fe7422d3640 1 --2- 172.21.15.45:0/4162751554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a6599afe80 0x55a659a377d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.951+0000 7fe741ad2640 1 --2- 172.21.15.45:0/4162751554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a659a38390 0x55a659a4db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.951+0000 7fe742ad4640 1 -- 172.21.15.45:0/4162751554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a6599afe80 msgr2=0x55a659a377d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.951+0000 7fe742ad4640 1 --2- 172.21.15.45:0/4162751554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a6599afe80 0x55a659a377d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.951+0000 7fe742ad4640 1 -- 172.21.15.45:0/4162751554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a659a38390 msgr2=0x55a659a4db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.951+0000 7fe742ad4640 1 --2- 172.21.15.45:0/4162751554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a659a38390 0x55a659a4db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.951+0000 7fe742ad4640 1 -- 172.21.15.45:0/4162751554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a659a4e0b0 con 0x55a659a3b8b0 2024-05-08T05:43:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.951+0000 7fe742ad4640 1 --2- 172.21.15.45:0/4162751554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a659a3b8b0 0x55a659a34870 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe734055040 tx=0x7fe7340033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.951+0000 7fe73b7fe640 1 -- 172.21.15.45:0/4162751554 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734053b40 con 0x55a659a3b8b0 2024-05-08T05:43:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.951+0000 7fe73b7fe640 1 -- 172.21.15.45:0/4162751554 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe73406b070 con 0x55a659a3b8b0 2024-05-08T05:43:11.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.951+0000 7fe73b7fe640 1 -- 172.21.15.45:0/4162751554 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734066400 con 0x55a659a3b8b0 2024-05-08T05:43:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.951+0000 7fe7447d9f40 1 -- 172.21.15.45:0/4162751554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a659a36170 con 0x55a659a3b8b0 2024-05-08T05:43:11.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.951+0000 7fe7447d9f40 1 -- 172.21.15.45:0/4162751554 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a659a4d090 con 0x55a659a3b8b0 2024-05-08T05:43:11.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.953+0000 7fe73b7fe640 1 -- 172.21.15.45:0/4162751554 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe734065070 con 0x55a659a3b8b0 2024-05-08T05:43:11.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.953+0000 7fe73b7fe640 1 --2- 172.21.15.45:0/4162751554 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe724074ce0 0x7fe7240771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.953+0000 7fe73b7fe640 1 -- 172.21.15.45:0/4162751554 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7fe7340da5a0 con 0x55a659a3b8b0 2024-05-08T05:43:11.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.953+0000 7fe7422d3640 1 --2- 172.21.15.45:0/4162751554 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe724074ce0 0x7fe7240771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.954+0000 7fe7447d9f40 1 --2- 172.21.15.45:0/4162751554 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55a659a34db0 0x55a659ae6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.954+0000 7fe7447d9f40 1 -- 172.21.15.45:0/4162751554 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47) -- 0x55a659afabf0 con 0x55a659a34db0 2024-05-08T05:43:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.954+0000 7fe741ad2640 1 --2- 172.21.15.45:0/4162751554 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55a659a34db0 0x55a659ae6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.954+0000 7fe7422d3640 1 --2- 172.21.15.45:0/4162751554 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe724074ce0 0x7fe7240771a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe73c003f10 tx=0x7fe73c0038b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.954+0000 7fe741ad2640 1 --2- 172.21.15.45:0/4162751554 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55a659a34db0 0x55a659ae6af0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.955+0000 7fe741ad2640 1 -- 172.21.15.45:0/4162751554 <== osd.1 v2:172.21.15.45:6824/1337179184 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v47'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe72c008770 con 0x55a659a34db0 2024-05-08T05:43:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.955+0000 7fe7447d9f40 1 -- 172.21.15.45:0/4162751554 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55a659a34db0 msgr2=0x55a659ae6af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.955+0000 7fe7447d9f40 1 --2- 172.21.15.45:0/4162751554 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55a659a34db0 0x55a659ae6af0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.955+0000 7fe7447d9f40 1 -- 172.21.15.45:0/4162751554 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe724074ce0 msgr2=0x7fe7240771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.955+0000 7fe7447d9f40 1 --2- 172.21.15.45:0/4162751554 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe724074ce0 0x7fe7240771a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe73c003f10 tx=0x7fe73c0038b0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.955+0000 7fe7447d9f40 1 -- 172.21.15.45:0/4162751554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a659a3b8b0 msgr2=0x55a659a34870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.955+0000 7fe7447d9f40 1 --2- 172.21.15.45:0/4162751554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a659a3b8b0 0x55a659a34870 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe734055040 tx=0x7fe7340033a0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.955+0000 7fe742ad4640 1 -- 172.21.15.45:0/4162751554 reap_dead start 2024-05-08T05:43:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.955+0000 7fe7447d9f40 1 -- 172.21.15.45:0/4162751554 shutdown_connections 2024-05-08T05:43:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.955+0000 7fe7447d9f40 1 -- 172.21.15.45:0/4162751554 >> 172.21.15.45:0/4162751554 conn(0x55a65997f7f0 msgr2=0x55a659a3a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.955+0000 7fe7447d9f40 1 -- 172.21.15.45:0/4162751554 shutdown_connections 2024-05-08T05:43:11.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:11.955+0000 7fe7447d9f40 1 -- 172.21.15.45:0/4162751554 wait complete. 2024-05-08T05:43:11.961 DEBUG:teuthology.orchestra.run.smithi045:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-08T05:43:11.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:43:11.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-05-08T05:43:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.071+0000 7f9cc4c93640 1 Processor -- start 2024-05-08T05:43:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.071+0000 7f9cc4c93640 1 -- start start 2024-05-08T05:43:12.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.072+0000 7f9cc4c93640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9cc00b5a50 0x7f9cc00d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.072+0000 7f9cc4c93640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cc00cfc50 0x7f9cc00d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.072+0000 7f9cc4c93640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cc0162f50 con 0x7f9cc0162b70 2024-05-08T05:43:12.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.072+0000 7f9cc4c93640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9cc00d1f60 con 0x7f9cc00b5a50 2024-05-08T05:43:12.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.072+0000 7f9cc4c93640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc00d3ee0 con 0x7f9cc00cfc50 2024-05-08T05:43:12.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.072+0000 7f9cbe59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9cc0162b70 0x7f9cc00d2f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48256/0 (socket says 172.21.15.45:48256) 2024-05-08T05:43:12.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.072+0000 7f9cbe59c640 1 -- 172.21.15.45:0/2153658115 learned_addr learned my addr 172.21.15.45:0/2153658115 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:12.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.072+0000 7f9cbed9d640 1 --2- 172.21.15.45:0/2153658115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cc00cfc50 0x7f9cc00d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.072+0000 7f9cb5d9b640 1 --2- 172.21.15.45:0/2153658115 >> v2:172.21.15.63:3300/0 conn(0x7f9cc00b5a50 0x7f9cc00d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.072+0000 7f9cbdd9b640 1 -- 172.21.15.45:0/2153658115 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3375937987 0 0) 0x7f9cb0001110 con 0x7f9cc0162b70 2024-05-08T05:43:12.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.072+0000 7f9cbdd9b640 1 -- 172.21.15.45:0/2153658115 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cac003520 con 0x7f9cc0162b70 2024-05-08T05:43:12.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.072+0000 7f9cbed9d640 1 -- 172.21.15.45:0/2153658115 >> v2:172.21.15.63:3300/0 conn(0x7f9cc00b5a50 msgr2=0x7f9cc00d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:12.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.072+0000 7f9cbed9d640 1 --2- 172.21.15.45:0/2153658115 >> v2:172.21.15.63:3300/0 conn(0x7f9cc00b5a50 0x7f9cc00d3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.072+0000 7f9cbed9d640 1 -- 172.21.15.45:0/2153658115 >> v1:172.21.15.45:6789/0 conn(0x7f9cc0162b70 legacy=0x7f9cc00d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.072+0000 7f9cbed9d640 1 -- 172.21.15.45:0/2153658115 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc00d21b0 con 0x7f9cc00cfc50 2024-05-08T05:43:12.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.073+0000 7f9cbed9d640 1 --2- 172.21.15.45:0/2153658115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cc00cfc50 0x7f9cc00d00c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009d90 tx=0x7f9cb802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d98037ea7bb098e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:12.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.073+0000 7f9cbdd9b640 1 -- 172.21.15.45:0/2153658115 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8007740 con 0x7f9cc00cfc50 2024-05-08T05:43:12.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.073+0000 7f9cbdd9b640 1 -- 172.21.15.45:0/2153658115 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb80078e0 con 0x7f9cc00cfc50 2024-05-08T05:43:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.073+0000 7f9cbdd9b640 1 -- 172.21.15.45:0/2153658115 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8007cb0 con 0x7f9cc00cfc50 2024-05-08T05:43:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.074+0000 7f9cc4c93640 1 -- 172.21.15.45:0/2153658115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cc00cfc50 msgr2=0x7f9cc00d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.074+0000 7f9cc4c93640 1 --2- 172.21.15.45:0/2153658115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cc00cfc50 0x7f9cc00d00c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009d90 tx=0x7f9cb802a660 comp rx=0 tx=0).stop 2024-05-08T05:43:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.074+0000 7f9cc4c93640 1 -- 172.21.15.45:0/2153658115 shutdown_connections 2024-05-08T05:43:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.074+0000 7f9cc4c93640 1 --2- 172.21.15.45:0/2153658115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cc00cfc50 0x7f9cc00d00c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.074+0000 7f9cc4c93640 1 --2- 172.21.15.45:0/2153658115 >> v2:172.21.15.63:3300/0 conn(0x7f9cc00b5a50 0x7f9cc00d3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.074+0000 7f9cc4c93640 1 -- 172.21.15.45:0/2153658115 >> 172.21.15.45:0/2153658115 conn(0x7f9cc00d5040 msgr2=0x7f9cc00d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.074+0000 7f9cc4c93640 1 -- 172.21.15.45:0/2153658115 shutdown_connections 2024-05-08T05:43:12.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.074+0000 7f9cc4c93640 1 -- 172.21.15.45:0/2153658115 wait complete. 2024-05-08T05:43:12.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.076+0000 7f9cc4c93640 1 Processor -- start 2024-05-08T05:43:12.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.076+0000 7f9cc4c93640 1 -- start start 2024-05-08T05:43:12.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.076+0000 7f9cc4c93640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cc00b5a50 0x7f9cc00b9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.076+0000 7f9cc4c93640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cc00cfc50 0x7f9cc014e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.076+0000 7f9cc4c93640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cc0162b70 0x7f9cc014f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.076+0000 7f9cc4c93640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc00b4170 con 0x7f9cc00b5a50 2024-05-08T05:43:12.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.076+0000 7f9cc4c93640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc00d3b90 con 0x7f9cc0162b70 2024-05-08T05:43:12.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.076+0000 7f9cc4c93640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc00b42f0 con 0x7f9cc00cfc50 2024-05-08T05:43:12.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.077+0000 7f9cbe59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cc00b5a50 0x7f9cc00b9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.077+0000 7f9cbe59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cc00b5a50 0x7f9cc00b9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54038/0 (socket says 172.21.15.45:54038) 2024-05-08T05:43:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.077+0000 7f9cbe59c640 1 -- 172.21.15.45:0/1718219960 learned_addr learned my addr 172.21.15.45:0/1718219960 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.077+0000 7f9cb5d9b640 1 --2- 172.21.15.45:0/1718219960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cc00cfc50 0x7f9cc014e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.077+0000 7f9cbed9d640 1 --2- 172.21.15.45:0/1718219960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cc0162b70 0x7f9cc014f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.077+0000 7f9cbe59c640 1 -- 172.21.15.45:0/1718219960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cc00cfc50 msgr2=0x7f9cc014e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.077+0000 7f9cbe59c640 1 --2- 172.21.15.45:0/1718219960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cc00cfc50 0x7f9cc014e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.077+0000 7f9cbe59c640 1 -- 172.21.15.45:0/1718219960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cc0162b70 msgr2=0x7f9cc014f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.077+0000 7f9cbe59c640 1 --2- 172.21.15.45:0/1718219960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cc0162b70 0x7f9cc014f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.077+0000 7f9cbe59c640 1 -- 172.21.15.45:0/1718219960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc00bad10 con 0x7f9cc00b5a50 2024-05-08T05:43:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.077+0000 7f9cbe59c640 1 --2- 172.21.15.45:0/1718219960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cc00b5a50 0x7f9cc00b9a90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000c730 tx=0x7f9cb000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.077+0000 7f9cb77fe640 1 -- 172.21.15.45:0/1718219960 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0017070 con 0x7f9cc00b5a50 2024-05-08T05:43:12.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.077+0000 7f9cb77fe640 1 -- 172.21.15.45:0/1718219960 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb0005ab0 con 0x7f9cc00b5a50 2024-05-08T05:43:12.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.077+0000 7f9cc4c93640 1 -- 172.21.15.45:0/1718219960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc00ba470 con 0x7f9cc00b5a50 2024-05-08T05:43:12.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.077+0000 7f9cb77fe640 1 -- 172.21.15.45:0/1718219960 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0005e60 con 0x7f9cc00b5a50 2024-05-08T05:43:12.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.077+0000 7f9cc4c93640 1 -- 172.21.15.45:0/1718219960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc014f060 con 0x7f9cc00b5a50 2024-05-08T05:43:12.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.079+0000 7f9cb77fe640 1 -- 172.21.15.45:0/1718219960 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9cb001a070 con 0x7f9cc00b5a50 2024-05-08T05:43:12.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.080+0000 7f9cb77fe640 1 --2- 172.21.15.45:0/1718219960 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ca00762d0 0x7f9ca0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.080+0000 7f9cb77fe640 1 -- 172.21.15.45:0/1718219960 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7f9cb0096a50 con 0x7f9cc00b5a50 2024-05-08T05:43:12.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.080+0000 7f9cb5d9b640 1 --2- 172.21.15.45:0/1718219960 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ca00762d0 0x7f9ca0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.080+0000 7f9cc79b5740 1 Processor -- start 2024-05-08T05:43:12.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.080+0000 7f9cc79b5740 1 -- start start 2024-05-08T05:43:12.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.081+0000 7f9cb5d9b640 1 --2- 172.21.15.45:0/1718219960 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ca00762d0 0x7f9ca0078790 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9cc01521e0 tx=0x7f9ca8009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:12.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.081+0000 7f9cc79b5740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56209389f220 0x562093851bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.081+0000 7f9cc79b5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562093882340 0x5620938816c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.081+0000 7f9cc79b5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093881c00 0x562093856a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.081+0000 7f9cc79b5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5620937bacf0 con 0x562093881c00 2024-05-08T05:43:12.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.081+0000 7f9cc79b5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562093773e10 con 0x562093882340 2024-05-08T05:43:12.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.081+0000 7f9cc79b5740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5620937b6cb0 con 0x56209389f220 2024-05-08T05:43:12.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.081+0000 7f9cbed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093881c00 0x562093856a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.081+0000 7f9cbed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093881c00 0x562093856a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54044/0 (socket says 172.21.15.45:54044) 2024-05-08T05:43:12.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.081+0000 7f9cbed9d640 1 -- 172.21.15.45:0/1231304707 learned_addr learned my addr 172.21.15.45:0/1231304707 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:12.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.081+0000 7f9cbe59c640 1 --2- 172.21.15.45:0/1231304707 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56209389f220 0x562093851bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.081+0000 7f9cb5d9b640 1 --2- 172.21.15.45:0/1231304707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562093882340 0x5620938816c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.082+0000 7f9cbed9d640 1 -- 172.21.15.45:0/1231304707 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56209389f220 msgr2=0x562093851bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:12.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.082+0000 7f9cbed9d640 1 --2- 172.21.15.45:0/1231304707 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56209389f220 0x562093851bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.082+0000 7f9cbed9d640 1 -- 172.21.15.45:0/1231304707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562093882340 msgr2=0x5620938816c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:12.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.082+0000 7f9cbed9d640 1 --2- 172.21.15.45:0/1231304707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562093882340 0x5620938816c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.082+0000 7f9cbed9d640 1 -- 172.21.15.45:0/1231304707 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620935947e0 con 0x562093881c00 2024-05-08T05:43:12.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.082+0000 7f9cbed9d640 1 --2- 172.21.15.45:0/1231304707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093881c00 0x562093856a10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9cb802ac50 tx=0x7f9cb802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e97b78e3af0358 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:12.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.082+0000 7f9c9ffff640 1 -- 172.21.15.45:0/1231304707 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8007740 con 0x562093881c00 2024-05-08T05:43:12.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.082+0000 7f9c9ffff640 1 -- 172.21.15.45:0/1231304707 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb80078e0 con 0x562093881c00 2024-05-08T05:43:12.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.082+0000 7f9c9ffff640 1 -- 172.21.15.45:0/1231304707 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8007c90 con 0x562093881c00 2024-05-08T05:43:12.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.083+0000 7f9cc79b5740 1 -- 172.21.15.45:0/1231304707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093881c00 msgr2=0x562093856a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.083+0000 7f9cc79b5740 1 --2- 172.21.15.45:0/1231304707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093881c00 0x562093856a10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9cb802ac50 tx=0x7f9cb802e000 comp rx=0 tx=0).stop 2024-05-08T05:43:12.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.083+0000 7f9cc79b5740 1 -- 172.21.15.45:0/1231304707 shutdown_connections 2024-05-08T05:43:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.083+0000 7f9cc79b5740 1 --2- 172.21.15.45:0/1231304707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093881c00 0x562093856a10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.083+0000 7f9cc79b5740 1 --2- 172.21.15.45:0/1231304707 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562093882340 0x5620938816c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.083+0000 7f9cc79b5740 1 --2- 172.21.15.45:0/1231304707 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56209389f220 0x562093851bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.083+0000 7f9cc79b5740 1 -- 172.21.15.45:0/1231304707 >> 172.21.15.45:0/1231304707 conn(0x562093853e20 msgr2=0x56209389c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.083+0000 7f9cc79b5740 1 -- 172.21.15.45:0/1231304707 shutdown_connections 2024-05-08T05:43:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.083+0000 7f9cc79b5740 1 -- 172.21.15.45:0/1231304707 wait complete. 2024-05-08T05:43:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.083+0000 7f9cc79b5740 1 -- start start 2024-05-08T05:43:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.084+0000 7f9cc79b5740 1 Processor -- start 2024-05-08T05:43:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.084+0000 7f9cc79b5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093882340 0x5620938724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.084+0000 7f9cc79b5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56209389f220 0x562093883240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.084+0000 7f9cc79b5740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56209387e750 0x56209387ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9cc79b5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562093855fd0 con 0x562093882340 2024-05-08T05:43:12.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9cc79b5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562093777940 con 0x56209389f220 2024-05-08T05:43:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9cc79b5740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5620938553c0 con 0x56209387e750 2024-05-08T05:43:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9cc79b5740 10 client.0 _ll_register_callbacks cb 0x56209377a440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:43:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9cbe59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093882340 0x5620938724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9cbe59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093882340 0x5620938724b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54054/0 (socket says 172.21.15.45:54054) 2024-05-08T05:43:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9cbe59c640 1 -- 172.21.15.45:0/589530377 learned_addr learned my addr 172.21.15.45:0/589530377 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9cb5d9b640 1 --2- 172.21.15.45:0/589530377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56209389f220 0x562093883240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9cbed9d640 1 --2- 172.21.15.45:0/589530377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56209387e750 0x56209387ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9cbe59c640 1 -- 172.21.15.45:0/589530377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56209387e750 msgr2=0x56209387ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9cbe59c640 1 --2- 172.21.15.45:0/589530377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56209387e750 0x56209387ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9cbe59c640 1 -- 172.21.15.45:0/589530377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56209389f220 msgr2=0x562093883240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9cbe59c640 1 --2- 172.21.15.45:0/589530377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56209389f220 0x562093883240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9cbe59c640 1 -- 172.21.15.45:0/589530377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620935f4f70 con 0x562093882340 2024-05-08T05:43:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9cbe59c640 1 --2- 172.21.15.45:0/589530377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093882340 0x5620938724b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9cb005cd30 tx=0x7f9cb005f990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8277c1b6ff41f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9c9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:43:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9c9d7fa640 1 -- 172.21.15.45:0/589530377 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0074e60 con 0x562093882340 2024-05-08T05:43:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9c9d7fa640 1 -- 172.21.15.45:0/589530377 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7f9cb007fdc0 con 0x562093882340 2024-05-08T05:43:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9cc79b5740 1 -- 172.21.15.45:0/589530377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cb80078e0 con 0x562093882340 2024-05-08T05:43:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9c9d7fa640 1 -- 172.21.15.45:0/589530377 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9cb005f5d0 con 0x562093882340 2024-05-08T05:43:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9cc79b5740 10 client.5499 fetch_fsmap learned FSMap version 26 2024-05-08T05:43:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.085+0000 7f9cc79b5740 1 -- 172.21.15.45:0/589530377 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x56209348c170 con 0x562093882340 2024-05-08T05:43:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.086+0000 7f9c9d7fa640 1 -- 172.21.15.45:0/589530377 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb80078e0 con 0x562093882340 2024-05-08T05:43:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.086+0000 7f9c9d7fa640 1 -- 172.21.15.45:0/589530377 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f9cb005b430 con 0x562093882340 2024-05-08T05:43:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.086+0000 7f9cc79b5740 10 client.5499 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T05:43:12.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.086+0000 7f9cc79b5740 10 client.5499 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-05-08T05:43:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.086+0000 7f9cc79b5740 20 client.5499 populate_metadata read hostname 'smithi045' 2024-05-08T05:43:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.086+0000 7f9cc79b5740 1 --2- 172.21.15.45:0/589530377 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x56209386fdf0 0x562093870210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.086+0000 7f9cc79b5740 4 client.5499 mds_command: new command op to 5292 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:43:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.086+0000 7f9cc79b5740 1 -- 172.21.15.45:0/589530377 --> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620937c8900 con 0x56209386fdf0 2024-05-08T05:43:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.086+0000 7f9cb5d9b640 1 --2- 172.21.15.45:0/589530377 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x56209386fdf0 0x562093870210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.086+0000 7f9cb5d9b640 1 --2- 172.21.15.45:0/589530377 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x56209386fdf0 0x562093870210 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25c44006577ec12 server_cookie=1eaf28b38075ff83 in_seq=0 out_seq=0 2024-05-08T05:43:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.086+0000 7f9c9d7fa640 10 client.5499 ms_handle_connect on v2:172.21.15.63:6834/4205418057 2024-05-08T05:43:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.087+0000 7f9c9d7fa640 1 -- 172.21.15.45:0/589530377 <== mds.0 v2:172.21.15.63:6834/4205418057 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5620937c8900 con 0x56209386fdf0 2024-05-08T05:43:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.087+0000 7f9c9d7fa640 10 client.5499 handle_command_reply: tid=0 2024-05-08T05:43:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.087+0000 7f9cc79b5740 1 client.5499 shutdown 2024-05-08T05:43:12.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.087+0000 7f9cc79b5740 1 -- 172.21.15.45:0/589530377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093882340 msgr2=0x5620938724b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.087+0000 7f9cc79b5740 1 --2- 172.21.15.45:0/589530377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093882340 0x5620938724b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9cb005cd30 tx=0x7f9cb005f990 comp rx=0 tx=0).stop 2024-05-08T05:43:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.087+0000 7f9cc79b5740 1 -- 172.21.15.45:0/589530377 shutdown_connections 2024-05-08T05:43:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.087+0000 7f9cc79b5740 1 --2- 172.21.15.45:0/589530377 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x56209386fdf0 0x562093870210 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.087+0000 7f9cc79b5740 1 --2- 172.21.15.45:0/589530377 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56209387e750 0x56209387ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.087+0000 7f9cc79b5740 1 --2- 172.21.15.45:0/589530377 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56209389f220 0x562093883240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.087+0000 7f9c9d7fa640 0 client.5499 ms_handle_reset on v2:172.21.15.63:6834/4205418057 2024-05-08T05:43:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.087+0000 7f9cc79b5740 1 --2- 172.21.15.45:0/589530377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093882340 0x5620938724b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.087+0000 7f9cc79b5740 1 -- 172.21.15.45:0/589530377 >> 172.21.15.45:0/589530377 conn(0x562093881c00 msgr2=0x5620938528f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.087+0000 7f9cc79b5740 1 -- 172.21.15.45:0/589530377 shutdown_connections 2024-05-08T05:43:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.087+0000 7f9cc79b5740 1 -- 172.21.15.45:0/589530377 wait complete. 2024-05-08T05:43:12.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.088+0000 7f9cc79b5740 20 client.5499 trim_cache size 0 max 16384 2024-05-08T05:43:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.092+0000 7f1d8f485640 1 Processor -- start 2024-05-08T05:43:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.093+0000 7f1d8f485640 1 -- start start 2024-05-08T05:43:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.093+0000 7f1d8f485640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1d8815d450 0x7f1d880ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.093+0000 7f1d8f485640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d880d3a30 0x7f1d880cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.093+0000 7f1d8f485640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1d880cfbf0 0x7f1d880d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.093+0000 7f1d8f485640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f1d8815d830 con 0x7f1d880cfbf0 2024-05-08T05:43:12.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.093+0000 7f1d8f485640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1d880d1fa0 con 0x7f1d8815d450 2024-05-08T05:43:12.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.093+0000 7f1d8f485640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1d880ced10 con 0x7f1d880d3a30 2024-05-08T05:43:12.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.094+0000 7f1d8da22640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1d880cfbf0 0x7f1d880d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.094+0000 7f1d8da22640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1d880cfbf0 0x7f1d880d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54064/0 (socket says 172.21.15.45:54064) 2024-05-08T05:43:12.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.094+0000 7f1d8da22640 1 -- 172.21.15.45:0/3429000103 learned_addr learned my addr 172.21.15.45:0/3429000103 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:12.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.094+0000 7f1d8ca20640 1 --2- 172.21.15.45:0/3429000103 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d880d3a30 0x7f1d880cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.094+0000 7f1d8d221640 1 --2- 172.21.15.45:0/3429000103 >> v2:172.21.15.63:3300/0 conn(0x7f1d8815d450 0x7f1d880ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.094+0000 7f1d8da22640 1 -- 172.21.15.45:0/3429000103 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d880d3a30 msgr2=0x7f1d880cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:12.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.094+0000 7f1d8da22640 1 --2- 172.21.15.45:0/3429000103 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d880d3a30 0x7f1d880cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.094+0000 7f1d8da22640 1 -- 172.21.15.45:0/3429000103 >> v2:172.21.15.63:3300/0 conn(0x7f1d8815d450 msgr2=0x7f1d880ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:12.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.094+0000 7f1d8da22640 1 --2- 172.21.15.45:0/3429000103 >> v2:172.21.15.63:3300/0 conn(0x7f1d8815d450 0x7f1d880ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.094+0000 7f1d8da22640 1 -- 172.21.15.45:0/3429000103 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d880b2600 con 0x7f1d880cfbf0 2024-05-08T05:43:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.095+0000 7f1d8da22640 1 --2- 172.21.15.45:0/3429000103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d880cfbf0 0x7f1d880d0cf0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1d88073100 tx=0x7f1d8002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf79a4ed189206c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.095+0000 7f1d7ffff640 1 -- 172.21.15.45:0/3429000103 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80036070 con 0x7f1d880cfbf0 2024-05-08T05:43:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.095+0000 7f1d7ffff640 1 -- 172.21.15.45:0/3429000103 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d80031600 con 0x7f1d880cfbf0 2024-05-08T05:43:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.095+0000 7f1d7ffff640 1 -- 172.21.15.45:0/3429000103 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d800319d0 con 0x7f1d880cfbf0 2024-05-08T05:43:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.096+0000 7f1d8f485640 1 -- 172.21.15.45:0/3429000103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d880cfbf0 msgr2=0x7f1d880d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.096+0000 7f1d8f485640 1 --2- 172.21.15.45:0/3429000103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d880cfbf0 0x7f1d880d0cf0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1d88073100 tx=0x7f1d8002a6a0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.096+0000 7f1d8f485640 1 -- 172.21.15.45:0/3429000103 shutdown_connections 2024-05-08T05:43:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.096+0000 7f1d8f485640 1 --2- 172.21.15.45:0/3429000103 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d880cfbf0 0x7f1d880d0cf0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.096+0000 7f1d8f485640 1 --2- 172.21.15.45:0/3429000103 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d880d3a30 0x7f1d880cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.096+0000 7f1d8f485640 1 --2- 172.21.15.45:0/3429000103 >> v2:172.21.15.63:3300/0 conn(0x7f1d8815d450 0x7f1d880ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.096+0000 7f1d8f485640 1 -- 172.21.15.45:0/3429000103 >> 172.21.15.45:0/3429000103 conn(0x7f1d880d2f10 msgr2=0x7f1d880d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.096+0000 7f1d8f485640 1 -- 172.21.15.45:0/3429000103 shutdown_connections 2024-05-08T05:43:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.096+0000 7f1d8f485640 1 -- 172.21.15.45:0/3429000103 wait complete. 2024-05-08T05:43:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.097+0000 7f1d8f485640 1 Processor -- start 2024-05-08T05:43:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.097+0000 7f1d8f485640 1 -- start start 2024-05-08T05:43:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.098+0000 7f1d8f485640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d880cfbf0 0x7f1d881f8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.098+0000 7f1d8f485640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d880d3a30 0x7f1d880c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.098+0000 7f1d8f485640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d8815d450 0x7f1d88158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.098+0000 7f1d8f485640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1d880b2bc0 con 0x7f1d880d3a30 2024-05-08T05:43:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.098+0000 7f1d8f485640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1d880b28e0 con 0x7f1d880cfbf0 2024-05-08T05:43:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.098+0000 7f1d8f485640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1d880d1230 con 0x7f1d8815d450 2024-05-08T05:43:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.098+0000 7f1d8ca20640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d880d3a30 0x7f1d880c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.098+0000 7f1d8ca20640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d880d3a30 0x7f1d880c75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54068/0 (socket says 172.21.15.45:54068) 2024-05-08T05:43:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.098+0000 7f1d8ca20640 1 -- 172.21.15.45:0/4230210253 learned_addr learned my addr 172.21.15.45:0/4230210253 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.098+0000 7f1d8d221640 1 --2- 172.21.15.45:0/4230210253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d880cfbf0 0x7f1d881f8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.098+0000 7f1d8da22640 1 --2- 172.21.15.45:0/4230210253 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d8815d450 0x7f1d88158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.098+0000 7f1d8ca20640 1 -- 172.21.15.45:0/4230210253 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d8815d450 msgr2=0x7f1d88158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.098+0000 7f1d8ca20640 1 --2- 172.21.15.45:0/4230210253 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1d8815d450 0x7f1d88158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.098+0000 7f1d8ca20640 1 -- 172.21.15.45:0/4230210253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d880cfbf0 msgr2=0x7f1d881f8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.098+0000 7f1d8ca20640 1 --2- 172.21.15.45:0/4230210253 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1d880cfbf0 0x7f1d881f8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.098+0000 7f1d8ca20640 1 -- 172.21.15.45:0/4230210253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d88158ad0 con 0x7f1d880d3a30 2024-05-08T05:43:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.098+0000 7f1d8ca20640 1 --2- 172.21.15.45:0/4230210253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d880d3a30 0x7f1d880c75f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800e8c0 tx=0x7f1d7800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.099+0000 7f1d7dffb640 1 -- 172.21.15.45:0/4230210253 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d78017070 con 0x7f1d880d3a30 2024-05-08T05:43:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.099+0000 7f1d7dffb640 1 -- 172.21.15.45:0/4230210253 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d7800c940 con 0x7f1d880d3a30 2024-05-08T05:43:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.099+0000 7f1d8f485640 1 -- 172.21.15.45:0/4230210253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d880c7d90 con 0x7f1d880d3a30 2024-05-08T05:43:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.099+0000 7f1d7dffb640 1 -- 172.21.15.45:0/4230210253 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7800cd10 con 0x7f1d880d3a30 2024-05-08T05:43:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.099+0000 7f1d8f485640 1 -- 172.21.15.45:0/4230210253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d8815f280 con 0x7f1d880d3a30 2024-05-08T05:43:12.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.099+0000 7f1d8f485640 1 -- 172.21.15.45:0/4230210253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f1d880c82b0 con 0x7f1d880d3a30 2024-05-08T05:43:12.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.100+0000 7f1d7dffb640 1 -- 172.21.15.45:0/4230210253 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1d7801a070 con 0x7f1d880d3a30 2024-05-08T05:43:12.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.101+0000 7f1d7dffb640 1 --2- 172.21.15.45:0/4230210253 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1d6c0762d0 0x7f1d6c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.101+0000 7f1d7dffb640 1 -- 172.21.15.45:0/4230210253 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7f1d78097a20 con 0x7f1d880d3a30 2024-05-08T05:43:12.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.101+0000 7f1d7dffb640 1 -- 172.21.15.45:0/4230210253 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+505 (secure 0 0 0) 0x7f1d880c82b0 con 0x7f1d880d3a30 2024-05-08T05:43:12.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.101+0000 7f1d8d221640 1 --2- 172.21.15.45:0/4230210253 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1d6c0762d0 0x7f1d6c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.101+0000 7f1d8f485640 1 -- 172.21.15.45:0/4230210253 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f1d88069b00 con 0x7f1d880d3a30 2024-05-08T05:43:12.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.101+0000 7f1d8d221640 1 --2- 172.21.15.45:0/4230210253 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1d6c0762d0 0x7f1d6c078790 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1d84009aa0 tx=0x7f1d84009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.108+0000 7f9cc79b5740 1 Processor -- start 2024-05-08T05:43:12.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.108+0000 7f9cc79b5740 1 -- start start 2024-05-08T05:43:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.109+0000 7f9cc79b5740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56209386fdf0 0x56209389c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.109+0000 7f9cc79b5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56209387e750 0x562093851700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.109+0000 7f9cc79b5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093882340 0x562093851c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.109+0000 7f9cc79b5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc0201ef0 con 0x562093882340 2024-05-08T05:43:12.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.109+0000 7f9cc79b5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc0200f60 con 0x56209387e750 2024-05-08T05:43:12.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.109+0000 7f9cc79b5740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc00cd3e0 con 0x56209386fdf0 2024-05-08T05:43:12.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.109+0000 7f9cbed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093882340 0x562093851c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.109+0000 7f9cbed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093882340 0x562093851c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54080/0 (socket says 172.21.15.45:54080) 2024-05-08T05:43:12.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.109+0000 7f9cbed9d640 1 -- 172.21.15.45:0/3848456888 learned_addr learned my addr 172.21.15.45:0/3848456888 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:12.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.109+0000 7f9cbe59c640 1 --2- 172.21.15.45:0/3848456888 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56209386fdf0 0x56209389c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.109+0000 7f9cb5d9b640 1 --2- 172.21.15.45:0/3848456888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56209387e750 0x562093851700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.109+0000 7f9cbed9d640 1 -- 172.21.15.45:0/3848456888 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56209386fdf0 msgr2=0x56209389c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:12.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.109+0000 7f9cbed9d640 1 --2- 172.21.15.45:0/3848456888 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56209386fdf0 0x56209389c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.109+0000 7f9cbed9d640 1 -- 172.21.15.45:0/3848456888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56209387e750 msgr2=0x562093851700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:12.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.109+0000 7f9cbed9d640 1 --2- 172.21.15.45:0/3848456888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56209387e750 0x562093851700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.109+0000 7f9cbed9d640 1 -- 172.21.15.45:0/3848456888 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620937b5a60 con 0x562093882340 2024-05-08T05:43:12.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.109+0000 7f9cb5d9b640 1 --2- 172.21.15.45:0/3848456888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56209387e750 0x562093851700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:12.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.109+0000 7f9cbed9d640 1 --2- 172.21.15.45:0/3848456888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093882340 0x562093851c40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9cb802a6a0 tx=0x7f9cb8030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf68706745b845f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:12.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.109+0000 7f9c9f7fe640 1 -- 172.21.15.45:0/3848456888 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8003d50 con 0x562093882340 2024-05-08T05:43:12.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.109+0000 7f9c9f7fe640 1 -- 172.21.15.45:0/3848456888 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb8004040 con 0x562093882340 2024-05-08T05:43:12.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.110+0000 7f9c9f7fe640 1 -- 172.21.15.45:0/3848456888 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8004430 con 0x562093882340 2024-05-08T05:43:12.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.110+0000 7f9cc79b5740 1 -- 172.21.15.45:0/3848456888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093882340 msgr2=0x562093851c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.110+0000 7f9cc79b5740 1 --2- 172.21.15.45:0/3848456888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093882340 0x562093851c40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9cb802a6a0 tx=0x7f9cb8030000 comp rx=0 tx=0).stop 2024-05-08T05:43:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.110+0000 7f9cc79b5740 1 -- 172.21.15.45:0/3848456888 shutdown_connections 2024-05-08T05:43:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.110+0000 7f9cc79b5740 1 --2- 172.21.15.45:0/3848456888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562093882340 0x562093851c40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.110+0000 7f9cc79b5740 1 --2- 172.21.15.45:0/3848456888 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56209387e750 0x562093851700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.110+0000 7f9cc79b5740 1 --2- 172.21.15.45:0/3848456888 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56209386fdf0 0x56209389c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.110+0000 7f9cc79b5740 1 -- 172.21.15.45:0/3848456888 >> 172.21.15.45:0/3848456888 conn(0x562093881c00 msgr2=0x5620938722c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.110+0000 7f9cc79b5740 1 -- 172.21.15.45:0/3848456888 shutdown_connections 2024-05-08T05:43:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.110+0000 7f9cc79b5740 1 -- 172.21.15.45:0/3848456888 wait complete. 2024-05-08T05:43:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.111+0000 7f9cc79b5740 1 -- start start 2024-05-08T05:43:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.111+0000 7f9cc79b5740 1 Processor -- start 2024-05-08T05:43:12.115 INFO:tasks.ceph.mds.b.smithi063.stderr:2024-05-08T05:43:12.115+0000 7f3c69591640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={4922=0-4194304@1} caps={4922=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=4922 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x557852b64c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-08T05:43:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.112+0000 7f9cc79b5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56209386fdf0 0x562093914c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.112+0000 7f9cc79b5740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56209387e750 0x562093883150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.112+0000 7f9cc79b5740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562093882340 0x562093851700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.112+0000 7f9cc79b5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc00cda30 con 0x56209386fdf0 2024-05-08T05:43:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.112+0000 7f9cc79b5740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562093857c80 con 0x562093882340 2024-05-08T05:43:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.112+0000 7f9cc79b5740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x562093855db0 con 0x56209387e750 2024-05-08T05:43:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.112+0000 7f9cc79b5740 10 client.0 _ll_register_callbacks cb 0x7f9ca80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:43:12.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.112+0000 7f9cbe59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56209386fdf0 0x562093914c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.112+0000 7f9cbe59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56209386fdf0 0x562093914c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54094/0 (socket says 172.21.15.45:54094) 2024-05-08T05:43:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.112+0000 7f9cbe59c640 1 -- 172.21.15.45:0/3840803753 learned_addr learned my addr 172.21.15.45:0/3840803753 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.112+0000 7f9cb5d9b640 1 --2- 172.21.15.45:0/3840803753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56209387e750 0x562093883150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.112+0000 7f9cbed9d640 1 --2- 172.21.15.45:0/3840803753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562093882340 0x562093851700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.112+0000 7f9cbe59c640 1 -- 172.21.15.45:0/3840803753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56209387e750 msgr2=0x562093883150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.112+0000 7f9cbe59c640 1 --2- 172.21.15.45:0/3840803753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56209387e750 0x562093883150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.112+0000 7f9cbe59c640 1 -- 172.21.15.45:0/3840803753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562093882340 msgr2=0x562093851700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.112+0000 7f9cbe59c640 1 --2- 172.21.15.45:0/3840803753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562093882340 0x562093851700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.112+0000 7f9cbe59c640 1 -- 172.21.15.45:0/3840803753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56209389c750 con 0x56209386fdf0 2024-05-08T05:43:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.112+0000 7f9cbe59c640 1 --2- 172.21.15.45:0/3840803753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56209386fdf0 0x562093914c20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0007f80 tx=0x7f9cb0094cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349d9d3d5d06a58c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.112+0000 7f9c9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:43:12.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.112+0000 7f9c9d7fa640 1 -- 172.21.15.45:0/3840803753 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb005f230 con 0x56209386fdf0 2024-05-08T05:43:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.113+0000 7f9c9d7fa640 1 -- 172.21.15.45:0/3840803753 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7f9cb0013c40 con 0x56209386fdf0 2024-05-08T05:43:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.113+0000 7f9cc79b5740 1 -- 172.21.15.45:0/3840803753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cb8004040 con 0x56209386fdf0 2024-05-08T05:43:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.113+0000 7f9c9d7fa640 1 -- 172.21.15.45:0/3840803753 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9cb006e050 con 0x56209386fdf0 2024-05-08T05:43:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.113+0000 7f9cc79b5740 10 client.5511 fetch_fsmap learned FSMap version 26 2024-05-08T05:43:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.113+0000 7f9cc79b5740 1 -- 172.21.15.45:0/3840803753 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x562093855770 con 0x56209386fdf0 2024-05-08T05:43:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.113+0000 7f9c9d7fa640 1 -- 172.21.15.45:0/3840803753 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb8004040 con 0x56209386fdf0 2024-05-08T05:43:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.113+0000 7f9c9d7fa640 1 -- 172.21.15.45:0/3840803753 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f9cb0071020 con 0x56209386fdf0 2024-05-08T05:43:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.113+0000 7f9cc79b5740 10 client.5511 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T05:43:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.113+0000 7f9cc79b5740 10 client.5511 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-05-08T05:43:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.113+0000 7f9cc79b5740 20 client.5511 populate_metadata read hostname 'smithi045' 2024-05-08T05:43:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.113+0000 7f9cc79b5740 1 --2- 172.21.15.45:0/3840803753 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x56209389f220 0x562093851c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.113+0000 7f9cc79b5740 4 client.5511 mds_command: new command op to 5292 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-08T05:43:12.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.113+0000 7f9cc79b5740 1 -- 172.21.15.45:0/3840803753 --> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x562093885840 con 0x56209389f220 2024-05-08T05:43:12.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.113+0000 7f9cb5d9b640 1 --2- 172.21.15.45:0/3840803753 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x56209389f220 0x562093851c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.114+0000 7f9cb5d9b640 1 --2- 172.21.15.45:0/3840803753 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x56209389f220 0x562093851c40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5084dc9d434db738 server_cookie=a1add6f1b1406957 in_seq=0 out_seq=0 2024-05-08T05:43:12.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.114+0000 7f9c9d7fa640 10 client.5511 ms_handle_connect on v2:172.21.15.63:6834/4205418057 2024-05-08T05:43:12.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.114+0000 7f9c9d7fa640 1 -- 172.21.15.45:0/3840803753 <== mds.0 v2:172.21.15.63:6834/4205418057 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562093885840 con 0x56209389f220 2024-05-08T05:43:12.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.114+0000 7f9c9d7fa640 10 client.5511 handle_command_reply: tid=0 2024-05-08T05:43:12.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.114+0000 7f9cc79b5740 1 client.5511 shutdown 2024-05-08T05:43:12.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.114+0000 7f9cc79b5740 1 -- 172.21.15.45:0/3840803753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56209386fdf0 msgr2=0x562093914c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.114+0000 7f9cc79b5740 1 --2- 172.21.15.45:0/3840803753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56209386fdf0 0x562093914c20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0007f80 tx=0x7f9cb0094cc0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.115+0000 7f9cc79b5740 1 -- 172.21.15.45:0/3840803753 shutdown_connections 2024-05-08T05:43:12.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.115+0000 7f9cc79b5740 1 --2- 172.21.15.45:0/3840803753 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x56209389f220 0x562093851c40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.115+0000 7f9cc79b5740 1 --2- 172.21.15.45:0/3840803753 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56209387e750 0x562093883150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.115+0000 7f9cc79b5740 1 --2- 172.21.15.45:0/3840803753 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562093882340 0x562093851700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.115+0000 7f9cc79b5740 1 --2- 172.21.15.45:0/3840803753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56209386fdf0 0x562093914c20 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.115+0000 7f9cc79b5740 1 -- 172.21.15.45:0/3840803753 >> 172.21.15.45:0/3840803753 conn(0x562093881c00 msgr2=0x5620938528f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:12.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.115+0000 7f9c9d7fa640 0 client.5511 ms_handle_reset on v2:172.21.15.63:6834/4205418057 2024-05-08T05:43:12.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.115+0000 7f9cc79b5740 1 -- 172.21.15.45:0/3840803753 shutdown_connections 2024-05-08T05:43:12.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.115+0000 7f9cc79b5740 1 -- 172.21.15.45:0/3840803753 wait complete. 2024-05-08T05:43:12.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.115+0000 7f9cc79b5740 20 client.5511 trim_cache size 0 max 16384 2024-05-08T05:43:12.123 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:43:12.123 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-05-08T05:43:12.124 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "8c95a750-4f85-466e-a42c-ba8f336e7c3a", 2024-05-08T05:43:12.124 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-05-08T05:43:12.124 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:43:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.115+0000 7f9cc4c93640 1 -- 172.21.15.45:0/1718219960 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ca00762d0 msgr2=0x7f9ca0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.115+0000 7f9cc4c93640 1 --2- 172.21.15.45:0/1718219960 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ca00762d0 0x7f9ca0078790 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9cc01521e0 tx=0x7f9ca8009190 comp rx=0 tx=0).stop 2024-05-08T05:43:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.115+0000 7f9cc4c93640 1 -- 172.21.15.45:0/1718219960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cc00b5a50 msgr2=0x7f9cc00b9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.115+0000 7f9cc4c93640 1 --2- 172.21.15.45:0/1718219960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cc00b5a50 0x7f9cc00b9a90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000c730 tx=0x7f9cb000cbf0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.115+0000 7f9cc4c93640 1 -- 172.21.15.45:0/1718219960 shutdown_connections 2024-05-08T05:43:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.115+0000 7f9cc4c93640 1 --2- 172.21.15.45:0/1718219960 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ca00762d0 0x7f9ca0078790 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.115+0000 7f9cc4c93640 1 --2- 172.21.15.45:0/1718219960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9cc00cfc50 0x7f9cc014e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.115+0000 7f9cc4c93640 1 --2- 172.21.15.45:0/1718219960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cc0162b70 0x7f9cc014f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.115+0000 7f9cc4c93640 1 --2- 172.21.15.45:0/1718219960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cc00b5a50 0x7f9cc00b9a90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.115+0000 7f9cc4c93640 1 -- 172.21.15.45:0/1718219960 >> 172.21.15.45:0/1718219960 conn(0x7f9cc00d5040 msgr2=0x7f9cc00d1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.116+0000 7f9cc4c93640 1 -- 172.21.15.45:0/1718219960 shutdown_connections 2024-05-08T05:43:12.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.116+0000 7f9cc4c93640 1 -- 172.21.15.45:0/1718219960 wait complete. 2024-05-08T05:43:12.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:43:12.127 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-05-08T05:43:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.232+0000 7fb245ef8640 1 Processor -- start 2024-05-08T05:43:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.232+0000 7fb245ef8640 1 -- start start 2024-05-08T05:43:12.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.233+0000 7fb245ef8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb240071720 0x7fb24015f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.233+0000 7fb245ef8640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb24006c3a0 0x7fb24006c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.233+0000 7fb245ef8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb240160600 con 0x7fb2400b1050 2024-05-08T05:43:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.233+0000 7fb245ef8640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb240071b80 con 0x7fb24006c3a0 2024-05-08T05:43:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.233+0000 7fb245ef8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb24006cd50 con 0x7fb240071720 2024-05-08T05:43:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.233+0000 7fb23effd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb240071720 0x7fb24015f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.233+0000 7fb23ffff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb24006c3a0 0x7fb24006c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.233+0000 7fb23f7fe640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fb2400b1050 0x7fb2400b56b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:49458/0 (socket says 172.21.15.45:49458) 2024-05-08T05:43:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.233+0000 7fb23effd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb240071720 0x7fb24015f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:57786/0 (socket says 172.21.15.45:57786) 2024-05-08T05:43:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.233+0000 7fb23f7fe640 1 -- 172.21.15.45:0/3356236054 learned_addr learned my addr 172.21.15.45:0/3356236054 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.233+0000 7fb23effd640 1 -- 172.21.15.45:0/3356236054 >> v2:172.21.15.63:3300/0 conn(0x7fb24006c3a0 msgr2=0x7fb24006c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.233+0000 7fb23effd640 1 --2- 172.21.15.45:0/3356236054 >> v2:172.21.15.63:3300/0 conn(0x7fb24006c3a0 0x7fb24006c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.233+0000 7fb23effd640 1 -- 172.21.15.45:0/3356236054 >> v1:172.21.15.63:6789/0 conn(0x7fb2400b1050 legacy=0x7fb2400b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.233+0000 7fb23effd640 1 -- 172.21.15.45:0/3356236054 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24015fd60 con 0x7fb240071720 2024-05-08T05:43:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.234+0000 7fb23effd640 1 --2- 172.21.15.45:0/3356236054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb240071720 0x7fb24015f680 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb2340097a0 tx=0x7fb23402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c23e280df47ca65 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.234+0000 7fb23e7fc640 1 -- 172.21.15.45:0/3356236054 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234037070 con 0x7fb240071720 2024-05-08T05:43:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.234+0000 7fb23e7fc640 1 -- 172.21.15.45:0/3356236054 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb234032440 con 0x7fb240071720 2024-05-08T05:43:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.234+0000 7fb23e7fc640 1 -- 172.21.15.45:0/3356236054 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234032810 con 0x7fb240071720 2024-05-08T05:43:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.235+0000 7fb245ef8640 1 -- 172.21.15.45:0/3356236054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb240071720 msgr2=0x7fb24015f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.235+0000 7fb245ef8640 1 --2- 172.21.15.45:0/3356236054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb240071720 0x7fb24015f680 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb2340097a0 tx=0x7fb23402a180 comp rx=0 tx=0).stop 2024-05-08T05:43:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.235+0000 7fb245ef8640 1 -- 172.21.15.45:0/3356236054 shutdown_connections 2024-05-08T05:43:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.235+0000 7fb245ef8640 1 --2- 172.21.15.45:0/3356236054 >> v2:172.21.15.63:3300/0 conn(0x7fb24006c3a0 0x7fb24006c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.235+0000 7fb245ef8640 1 --2- 172.21.15.45:0/3356236054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb240071720 0x7fb24015f680 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.235+0000 7fb245ef8640 1 -- 172.21.15.45:0/3356236054 >> 172.21.15.45:0/3356236054 conn(0x7fb2400b3c40 msgr2=0x7fb2400b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.235+0000 7fb245ef8640 1 -- 172.21.15.45:0/3356236054 shutdown_connections 2024-05-08T05:43:12.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.235+0000 7fb245ef8640 1 -- 172.21.15.45:0/3356236054 wait complete. 2024-05-08T05:43:12.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.236+0000 7fb245ef8640 1 Processor -- start 2024-05-08T05:43:12.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.236+0000 7fb245ef8640 1 -- start start 2024-05-08T05:43:12.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.237+0000 7fb245ef8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb24006c3a0 0x7fb2401f6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.237+0000 7fb245ef8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb240071720 0x7fb2401f8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.237+0000 7fb245ef8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2400b1050 0x7fb240151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.237+0000 7fb245ef8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb240162d10 con 0x7fb240071720 2024-05-08T05:43:12.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.237+0000 7fb245ef8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb240160320 con 0x7fb2400b1050 2024-05-08T05:43:12.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.237+0000 7fb245ef8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb2400b5dc0 con 0x7fb24006c3a0 2024-05-08T05:43:12.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.237+0000 7fb23effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb240071720 0x7fb2401f8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.237+0000 7fb23effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb240071720 0x7fb2401f8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54102/0 (socket says 172.21.15.45:54102) 2024-05-08T05:43:12.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.237+0000 7fb23effd640 1 -- 172.21.15.45:0/2983292263 learned_addr learned my addr 172.21.15.45:0/2983292263 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:12.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.237+0000 7fb23f7fe640 1 --2- 172.21.15.45:0/2983292263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb24006c3a0 0x7fb2401f6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.237+0000 7fb23ffff640 1 --2- 172.21.15.45:0/2983292263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2400b1050 0x7fb240151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.237+0000 7fb23effd640 1 -- 172.21.15.45:0/2983292263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb24006c3a0 msgr2=0x7fb2401f6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.237+0000 7fb23effd640 1 --2- 172.21.15.45:0/2983292263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb24006c3a0 0x7fb2401f6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.237+0000 7fb23effd640 1 -- 172.21.15.45:0/2983292263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2400b1050 msgr2=0x7fb240151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.237+0000 7fb23effd640 1 --2- 172.21.15.45:0/2983292263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2400b1050 0x7fb240151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.237+0000 7fb23effd640 1 -- 172.21.15.45:0/2983292263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2401520c0 con 0x7fb240071720 2024-05-08T05:43:12.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.237+0000 7fb23f7fe640 1 --2- 172.21.15.45:0/2983292263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb24006c3a0 0x7fb2401f6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:43:12.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.238+0000 7fb23effd640 1 --2- 172.21.15.45:0/2983292263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb240071720 0x7fb2401f8080 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb234035040 tx=0x7fb2340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:12.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.238+0000 7fb227fff640 1 -- 172.21.15.45:0/2983292263 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234037040 con 0x7fb240071720 2024-05-08T05:43:12.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.238+0000 7fb227fff640 1 -- 172.21.15.45:0/2983292263 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb234030500 con 0x7fb240071720 2024-05-08T05:43:12.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.238+0000 7fb245ef8640 1 -- 172.21.15.45:0/2983292263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2401568d0 con 0x7fb240071720 2024-05-08T05:43:12.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.238+0000 7fb227fff640 1 -- 172.21.15.45:0/2983292263 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234030910 con 0x7fb240071720 2024-05-08T05:43:12.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.238+0000 7fb245ef8640 1 -- 172.21.15.45:0/2983292263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb240156e90 con 0x7fb240071720 2024-05-08T05:43:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.240+0000 7fb227fff640 1 -- 172.21.15.45:0/2983292263 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb234044020 con 0x7fb240071720 2024-05-08T05:43:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.241+0000 7fb227fff640 1 --2- 172.21.15.45:0/2983292263 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb22c0762d0 0x7fb22c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.241+0000 7fb227fff640 1 -- 172.21.15.45:0/2983292263 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7fb2340b78b0 con 0x7fb240071720 2024-05-08T05:43:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.241+0000 7fb23f7fe640 1 --2- 172.21.15.45:0/2983292263 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb22c0762d0 0x7fb22c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.241+0000 7fb248c10740 1 Processor -- start 2024-05-08T05:43:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.241+0000 7fb23f7fe640 1 --2- 172.21.15.45:0/2983292263 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb22c0762d0 0x7fb22c078790 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb230000f60 tx=0x7fb230007ea0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.241+0000 7fb248c10740 1 -- start start 2024-05-08T05:43:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.242+0000 7fb248c10740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619e33cc160 0x5619e337eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.242+0000 7fb248c10740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33af280 0x5619e33ae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.242+0000 7fb248c10740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5619e33aeb40 0x5619e3383950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.242+0000 7fb248c10740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5619e334b690 con 0x5619e33af280 2024-05-08T05:43:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.242+0000 7fb248c10740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5619e32e8f30 con 0x5619e33cc160 2024-05-08T05:43:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.242+0000 7fb248c10740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5619e32a1b20 con 0x5619e33aeb40 2024-05-08T05:43:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.242+0000 7fb23effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33af280 0x5619e33ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.242+0000 7fb23effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33af280 0x5619e33ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54104/0 (socket says 172.21.15.45:54104) 2024-05-08T05:43:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.242+0000 7fb23effd640 1 -- 172.21.15.45:0/2611845032 learned_addr learned my addr 172.21.15.45:0/2611845032 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.242+0000 7fb23f7fe640 1 --2- 172.21.15.45:0/2611845032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619e33cc160 0x5619e337eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.242+0000 7fb23ffff640 1 --2- 172.21.15.45:0/2611845032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5619e33aeb40 0x5619e3383950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.243+0000 7fb23effd640 1 -- 172.21.15.45:0/2611845032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5619e33aeb40 msgr2=0x5619e3383950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.243+0000 7fb23effd640 1 --2- 172.21.15.45:0/2611845032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5619e33aeb40 0x5619e3383950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.243+0000 7fb23effd640 1 -- 172.21.15.45:0/2611845032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619e33cc160 msgr2=0x5619e337eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.243+0000 7fb23effd640 1 --2- 172.21.15.45:0/2611845032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619e33cc160 0x5619e337eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.243+0000 7fb23effd640 1 -- 172.21.15.45:0/2611845032 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619e3299e60 con 0x5619e33af280 2024-05-08T05:43:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.243+0000 7fb23effd640 1 --2- 172.21.15.45:0/2611845032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33af280 0x5619e33ae600 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb2340b5b80 tx=0x7fb234080430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4914423afdc0a79d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.243+0000 7fb2257fa640 1 -- 172.21.15.45:0/2611845032 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234084d20 con 0x5619e33af280 2024-05-08T05:43:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.243+0000 7fb2257fa640 1 -- 172.21.15.45:0/2611845032 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb234089740 con 0x5619e33af280 2024-05-08T05:43:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.243+0000 7fb2257fa640 1 -- 172.21.15.45:0/2611845032 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234080e80 con 0x5619e33af280 2024-05-08T05:43:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.244+0000 7fb248c10740 1 -- 172.21.15.45:0/2611845032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33af280 msgr2=0x5619e33ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.244+0000 7fb248c10740 1 --2- 172.21.15.45:0/2611845032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33af280 0x5619e33ae600 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb2340b5b80 tx=0x7fb234080430 comp rx=0 tx=0).stop 2024-05-08T05:43:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.244+0000 7fb248c10740 1 -- 172.21.15.45:0/2611845032 shutdown_connections 2024-05-08T05:43:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.244+0000 7fb248c10740 1 --2- 172.21.15.45:0/2611845032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5619e33aeb40 0x5619e3383950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.244+0000 7fb248c10740 1 --2- 172.21.15.45:0/2611845032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33af280 0x5619e33ae600 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.244+0000 7fb248c10740 1 --2- 172.21.15.45:0/2611845032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619e33cc160 0x5619e337eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.244+0000 7fb248c10740 1 -- 172.21.15.45:0/2611845032 >> 172.21.15.45:0/2611845032 conn(0x5619e3380d60 msgr2=0x5619e33c9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.244+0000 7fb248c10740 1 -- 172.21.15.45:0/2611845032 shutdown_connections 2024-05-08T05:43:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.244+0000 7fb248c10740 1 -- 172.21.15.45:0/2611845032 wait complete. 2024-05-08T05:43:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.244+0000 7fb248c10740 1 -- start start 2024-05-08T05:43:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.244+0000 7fb248c10740 1 Processor -- start 2024-05-08T05:43:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.245+0000 7fb248c10740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33af280 0x5619e339eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.245+0000 7fb248c10740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619e33cc160 0x5619e33ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.245+0000 7fb248c10740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5619e33b2400 0x5619e33cab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.245+0000 7fb248c10740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5619e32a55e0 con 0x5619e33af280 2024-05-08T05:43:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.245+0000 7fb248c10740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5619e32e4ef0 con 0x5619e33cc160 2024-05-08T05:43:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.245+0000 7fb248c10740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5619e334b690 con 0x5619e33b2400 2024-05-08T05:43:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.245+0000 7fb248c10740 10 client.0 _ll_register_callbacks cb 0x5619e32a0320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:43:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.245+0000 7fb23f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33af280 0x5619e339eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.245+0000 7fb23f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33af280 0x5619e339eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54106/0 (socket says 172.21.15.45:54106) 2024-05-08T05:43:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.245+0000 7fb23f7fe640 1 -- 172.21.15.45:0/1896258762 learned_addr learned my addr 172.21.15.45:0/1896258762 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.246+0000 7fb23ffff640 1 --2- 172.21.15.45:0/1896258762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5619e33b2400 0x5619e33cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.246+0000 7fb23effd640 1 --2- 172.21.15.45:0/1896258762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619e33cc160 0x5619e33ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.246+0000 7fb23f7fe640 1 -- 172.21.15.45:0/1896258762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5619e33b2400 msgr2=0x5619e33cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.246+0000 7fb23f7fe640 1 --2- 172.21.15.45:0/1896258762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5619e33b2400 0x5619e33cab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.246+0000 7fb23f7fe640 1 -- 172.21.15.45:0/1896258762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619e33cc160 msgr2=0x5619e33ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.246+0000 7fb23f7fe640 1 --2- 172.21.15.45:0/1896258762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619e33cc160 0x5619e33ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.246+0000 7fb23f7fe640 1 -- 172.21.15.45:0/1896258762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619e30c47d0 con 0x5619e33af280 2024-05-08T05:43:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.246+0000 7fb23f7fe640 1 --2- 172.21.15.45:0/1896258762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33af280 0x5619e339eda0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb230005ee0 tx=0x7fb230000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d819140e4c86a5d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.246+0000 7fb21e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:43:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.246+0000 7fb21e7fc640 1 -- 172.21.15.45:0/1896258762 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230013780 con 0x5619e33af280 2024-05-08T05:43:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.246+0000 7fb21e7fc640 1 -- 172.21.15.45:0/1896258762 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7fb230013920 con 0x5619e33af280 2024-05-08T05:43:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.246+0000 7fb248c10740 1 -- 172.21.15.45:0/1896258762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb234089740 con 0x5619e33af280 2024-05-08T05:43:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.246+0000 7fb21e7fc640 1 -- 172.21.15.45:0/1896258762 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb230013b20 con 0x5619e33af280 2024-05-08T05:43:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.246+0000 7fb248c10740 10 client.5520 fetch_fsmap learned FSMap version 26 2024-05-08T05:43:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.246+0000 7fb248c10740 1 -- 172.21.15.45:0/1896258762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5619e2ffa3f0 con 0x5619e33af280 2024-05-08T05:43:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.246+0000 7fb21e7fc640 1 -- 172.21.15.45:0/1896258762 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb234089740 con 0x5619e33af280 2024-05-08T05:43:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.247+0000 7fb21e7fc640 1 -- 172.21.15.45:0/1896258762 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7fb230013cc0 con 0x5619e33af280 2024-05-08T05:43:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.247+0000 7fb248c10740 10 client.5520 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T05:43:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.247+0000 7fb248c10740 10 client.5520 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-05-08T05:43:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.247+0000 7fb248c10740 20 client.5520 populate_metadata read hostname 'smithi045' 2024-05-08T05:43:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.247+0000 7fb248c10740 1 --2- 172.21.15.45:0/1896258762 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x5619e33d9930 0x5619e33b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.247+0000 7fb248c10740 4 client.5520 mds_command: new command op to 5292 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:43:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.247+0000 7fb248c10740 1 -- 172.21.15.45:0/1896258762 --> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619e32f6b40 con 0x5619e33d9930 2024-05-08T05:43:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.247+0000 7fb23effd640 1 --2- 172.21.15.45:0/1896258762 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x5619e33d9930 0x5619e33b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.247+0000 7fb21e7fc640 10 client.5520 ms_handle_connect on v2:172.21.15.63:6834/4205418057 2024-05-08T05:43:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.247+0000 7fb23effd640 1 --2- 172.21.15.45:0/1896258762 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x5619e33d9930 0x5619e33b08f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1aa09fc3e4589dc server_cookie=50114db27fc425d3 in_seq=0 out_seq=0 2024-05-08T05:43:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.248+0000 7fb21e7fc640 1 -- 172.21.15.45:0/1896258762 <== mds.0 v2:172.21.15.63:6834/4205418057 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5619e32f6b40 con 0x5619e33d9930 2024-05-08T05:43:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.248+0000 7fb21e7fc640 10 client.5520 handle_command_reply: tid=0 2024-05-08T05:43:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.248+0000 7fb248c10740 1 client.5520 shutdown 2024-05-08T05:43:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.248+0000 7fb248c10740 1 -- 172.21.15.45:0/1896258762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33af280 msgr2=0x5619e339eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.248+0000 7fb248c10740 1 --2- 172.21.15.45:0/1896258762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33af280 0x5619e339eda0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb230005ee0 tx=0x7fb230000ed0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.248+0000 7fb248c10740 1 -- 172.21.15.45:0/1896258762 shutdown_connections 2024-05-08T05:43:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.248+0000 7fb248c10740 1 --2- 172.21.15.45:0/1896258762 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x5619e33d9930 0x5619e33b08f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.248+0000 7fb248c10740 1 --2- 172.21.15.45:0/1896258762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5619e33b2400 0x5619e33cab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.248+0000 7fb248c10740 1 --2- 172.21.15.45:0/1896258762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619e33cc160 0x5619e33ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.248+0000 7fb248c10740 1 --2- 172.21.15.45:0/1896258762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33af280 0x5619e339eda0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.248+0000 7fb21e7fc640 0 client.5520 ms_handle_reset on v2:172.21.15.63:6834/4205418057 2024-05-08T05:43:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.248+0000 7fb248c10740 1 -- 172.21.15.45:0/1896258762 >> 172.21.15.45:0/1896258762 conn(0x5619e33aeb40 msgr2=0x5619e337e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.248+0000 7fb248c10740 1 -- 172.21.15.45:0/1896258762 shutdown_connections 2024-05-08T05:43:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.248+0000 7fb248c10740 1 -- 172.21.15.45:0/1896258762 wait complete. 2024-05-08T05:43:12.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.248+0000 7fb248c10740 20 client.5520 trim_cache size 0 max 16384 2024-05-08T05:43:12.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.266+0000 7fb248c10740 1 Processor -- start 2024-05-08T05:43:12.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.267+0000 7fb248c10740 1 -- start start 2024-05-08T05:43:12.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.267+0000 7fb248c10740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619e33af280 0x5619e33e2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.267+0000 7fb248c10740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5619e33b2400 0x5619e33abba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.267+0000 7fb248c10740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33cc160 0x5619e33ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.267+0000 7fb248c10740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb240201e80 con 0x5619e33cc160 2024-05-08T05:43:12.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.267+0000 7fb248c10740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb240200ef0 con 0x5619e33af280 2024-05-08T05:43:12.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.267+0000 7fb248c10740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb2401f5a70 con 0x5619e33b2400 2024-05-08T05:43:12.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.268+0000 7fb23ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33cc160 0x5619e33ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.268+0000 7fb23ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33cc160 0x5619e33ccc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54120/0 (socket says 172.21.15.45:54120) 2024-05-08T05:43:12.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.268+0000 7fb23ffff640 1 -- 172.21.15.45:0/2727543923 learned_addr learned my addr 172.21.15.45:0/2727543923 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:12.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.268+0000 7fb23ffff640 1 -- 172.21.15.45:0/2727543923 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5619e33b2400 msgr2=0x5619e33abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:12.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.268+0000 7fb23ffff640 1 --2- 172.21.15.45:0/2727543923 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5619e33b2400 0x5619e33abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.268+0000 7fb23ffff640 1 -- 172.21.15.45:0/2727543923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619e33af280 msgr2=0x5619e33e2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:12.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.268+0000 7fb23ffff640 1 --2- 172.21.15.45:0/2727543923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619e33af280 0x5619e33e2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.268+0000 7fb23ffff640 1 -- 172.21.15.45:0/2727543923 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619e2fba0f0 con 0x5619e33cc160 2024-05-08T05:43:12.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.268+0000 7fb23ffff640 1 --2- 172.21.15.45:0/2727543923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33cc160 0x5619e33ccc10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb238017870 tx=0x7fb238009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8f55aa0e48259 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:12.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.268+0000 7fb224ff9640 1 -- 172.21.15.45:0/2727543923 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb238003b90 con 0x5619e33cc160 2024-05-08T05:43:12.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.268+0000 7fb224ff9640 1 -- 172.21.15.45:0/2727543923 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb238003d50 con 0x5619e33cc160 2024-05-08T05:43:12.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.268+0000 7fb224ff9640 1 -- 172.21.15.45:0/2727543923 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb238015ab0 con 0x5619e33cc160 2024-05-08T05:43:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.269+0000 7fb248c10740 1 -- 172.21.15.45:0/2727543923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33cc160 msgr2=0x5619e33ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.269+0000 7fb248c10740 1 --2- 172.21.15.45:0/2727543923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33cc160 0x5619e33ccc10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb238017870 tx=0x7fb238009750 comp rx=0 tx=0).stop 2024-05-08T05:43:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.269+0000 7fb248c10740 1 -- 172.21.15.45:0/2727543923 shutdown_connections 2024-05-08T05:43:12.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.269+0000 7fb248c10740 1 --2- 172.21.15.45:0/2727543923 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5619e33b2400 0x5619e33abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.269+0000 7fb248c10740 1 --2- 172.21.15.45:0/2727543923 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33cc160 0x5619e33ccc10 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.269+0000 7fb248c10740 1 --2- 172.21.15.45:0/2727543923 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619e33af280 0x5619e33e2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.269+0000 7fb248c10740 1 -- 172.21.15.45:0/2727543923 >> 172.21.15.45:0/2727543923 conn(0x5619e33aeb40 msgr2=0x5619e33aad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.269+0000 7fb248c10740 1 -- 172.21.15.45:0/2727543923 shutdown_connections 2024-05-08T05:43:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.269+0000 7fb248c10740 1 -- 172.21.15.45:0/2727543923 wait complete. 2024-05-08T05:43:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.269+0000 7fb248c10740 1 -- start start 2024-05-08T05:43:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.270+0000 7fb248c10740 1 Processor -- start 2024-05-08T05:43:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.270+0000 7fb248c10740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619e33af280 0x5619e33c4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.270+0000 7fb248c10740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5619e33b2400 0x5619e344b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.270+0000 7fb248c10740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33cc160 0x5619e33c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.270+0000 7fb248c10740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5619e33c1830 con 0x5619e33cc160 2024-05-08T05:43:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.270+0000 7fb248c10740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb240200ac0 con 0x5619e33af280 2024-05-08T05:43:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.270+0000 7fb248c10740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5619e3385000 con 0x5619e33b2400 2024-05-08T05:43:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.270+0000 7fb248c10740 10 client.0 _ll_register_callbacks cb 0x7fb230007320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:43:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.270+0000 7fb23ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33cc160 0x5619e33c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.270+0000 7fb23ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33cc160 0x5619e33c91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54122/0 (socket says 172.21.15.45:54122) 2024-05-08T05:43:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.270+0000 7fb23ffff640 1 -- 172.21.15.45:0/870675924 learned_addr learned my addr 172.21.15.45:0/870675924 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.270+0000 7fb23f7fe640 1 --2- 172.21.15.45:0/870675924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619e33af280 0x5619e33c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.270+0000 7fb23effd640 1 --2- 172.21.15.45:0/870675924 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5619e33b2400 0x5619e344b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.271+0000 7fb23ffff640 1 -- 172.21.15.45:0/870675924 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5619e33b2400 msgr2=0x5619e344b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.271+0000 7fb23ffff640 1 --2- 172.21.15.45:0/870675924 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5619e33b2400 0x5619e344b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.271+0000 7fb23ffff640 1 -- 172.21.15.45:0/870675924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619e33af280 msgr2=0x5619e33c4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.271+0000 7fb23ffff640 1 --2- 172.21.15.45:0/870675924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619e33af280 0x5619e33c4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.271+0000 7fb23ffff640 1 -- 172.21.15.45:0/870675924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619e33e2b10 con 0x5619e33cc160 2024-05-08T05:43:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.271+0000 7fb23ffff640 1 --2- 172.21.15.45:0/870675924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33cc160 0x5619e33c91f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb23800b010 tx=0x7fb238004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec24660c62c1a979 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.271+0000 7fb21e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:43:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.271+0000 7fb21e7fc640 1 -- 172.21.15.45:0/870675924 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb238030060 con 0x5619e33cc160 2024-05-08T05:43:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.271+0000 7fb21e7fc640 1 -- 172.21.15.45:0/870675924 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 6101+0+0 (secure 0 0 0) 0x7fb238015c80 con 0x5619e33cc160 2024-05-08T05:43:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.271+0000 7fb248c10740 1 -- 172.21.15.45:0/870675924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb238003d50 con 0x5619e33cc160 2024-05-08T05:43:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.271+0000 7fb21e7fc640 1 -- 172.21.15.45:0/870675924 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb238036050 con 0x5619e33cc160 2024-05-08T05:43:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.271+0000 7fb248c10740 10 client.5526 fetch_fsmap learned FSMap version 26 2024-05-08T05:43:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.271+0000 7fb248c10740 1 -- 172.21.15.45:0/870675924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5619e344ac40 con 0x5619e33cc160 2024-05-08T05:43:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.271+0000 7fb21e7fc640 1 -- 172.21.15.45:0/870675924 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb238003d50 con 0x5619e33cc160 2024-05-08T05:43:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb21e7fc640 1 -- 172.21.15.45:0/870675924 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7fb238039020 con 0x5619e33cc160 2024-05-08T05:43:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb248c10740 10 client.5526 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T05:43:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb248c10740 10 client.5526 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-05-08T05:43:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb248c10740 20 client.5526 populate_metadata read hostname 'smithi045' 2024-05-08T05:43:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb248c10740 1 --2- 172.21.15.45:0/870675924 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x5619e33c9730 0x5619e33aad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb248c10740 4 client.5526 mds_command: new command op to 5292 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:43:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb248c10740 1 -- 172.21.15.45:0/870675924 --> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5619e32f09b0 con 0x5619e33c9730 2024-05-08T05:43:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb23f7fe640 1 --2- 172.21.15.45:0/870675924 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x5619e33c9730 0x5619e33aad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:12.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb21e7fc640 10 client.5526 ms_handle_connect on v2:172.21.15.63:6834/4205418057 2024-05-08T05:43:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb23f7fe640 1 --2- 172.21.15.45:0/870675924 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x5619e33c9730 0x5619e33aad60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fbba20eb02cf4ba server_cookie=77678320fc078d43 in_seq=0 out_seq=0 2024-05-08T05:43:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb21e7fc640 1 -- 172.21.15.45:0/870675924 <== mds.0 v2:172.21.15.63:6834/4205418057 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x5619e32f09b0 con 0x5619e33c9730 2024-05-08T05:43:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb21e7fc640 10 client.5526 handle_command_reply: tid=0 2024-05-08T05:43:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb248c10740 1 client.5526 shutdown 2024-05-08T05:43:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb248c10740 1 -- 172.21.15.45:0/870675924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33cc160 msgr2=0x5619e33c91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb248c10740 1 --2- 172.21.15.45:0/870675924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33cc160 0x5619e33c91f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb23800b010 tx=0x7fb238004ab0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb248c10740 1 -- 172.21.15.45:0/870675924 shutdown_connections 2024-05-08T05:43:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb248c10740 1 --2- 172.21.15.45:0/870675924 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x5619e33c9730 0x5619e33aad60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb248c10740 1 --2- 172.21.15.45:0/870675924 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5619e33b2400 0x5619e344b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb248c10740 1 --2- 172.21.15.45:0/870675924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5619e33cc160 0x5619e33c91f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb21e7fc640 0 client.5526 ms_handle_reset on v2:172.21.15.63:6834/4205418057 2024-05-08T05:43:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb248c10740 1 --2- 172.21.15.45:0/870675924 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5619e33af280 0x5619e33c4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.272+0000 7fb248c10740 1 -- 172.21.15.45:0/870675924 >> 172.21.15.45:0/870675924 conn(0x5619e33aeb40 msgr2=0x5619e33e3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:12.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.273+0000 7fb248c10740 1 -- 172.21.15.45:0/870675924 shutdown_connections 2024-05-08T05:43:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.273+0000 7fb248c10740 1 -- 172.21.15.45:0/870675924 wait complete. 2024-05-08T05:43:12.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.273+0000 7fb248c10740 20 client.5526 trim_cache size 0 max 16384 2024-05-08T05:43:12.279 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:43:12.279 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T05:43:12.279 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-05-08T05:43:12.279 INFO:teuthology.orchestra.run.smithi045.stdout: "8c95a750-4f85-466e-a42c-ba8f336e7c3a": { 2024-05-08T05:43:12.279 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-05-08T05:43:12.279 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "8c95a750-4f85-466e-a42c-ba8f336e7c3a", 2024-05-08T05:43:12.279 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,repair" 2024-05-08T05:43:12.279 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:12.279 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:12.279 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:43:12.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.273+0000 7fb245ef8640 1 -- 172.21.15.45:0/2983292263 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb22c0762d0 msgr2=0x7fb22c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.273+0000 7fb245ef8640 1 --2- 172.21.15.45:0/2983292263 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb22c0762d0 0x7fb22c078790 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb230000f60 tx=0x7fb230007ea0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.273+0000 7fb245ef8640 1 -- 172.21.15.45:0/2983292263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb240071720 msgr2=0x7fb2401f8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.273+0000 7fb245ef8640 1 --2- 172.21.15.45:0/2983292263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb240071720 0x7fb2401f8080 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb234035040 tx=0x7fb2340046c0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.273+0000 7fb245ef8640 1 -- 172.21.15.45:0/2983292263 shutdown_connections 2024-05-08T05:43:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.273+0000 7fb245ef8640 1 --2- 172.21.15.45:0/2983292263 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb22c0762d0 0x7fb22c078790 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.273+0000 7fb245ef8640 1 --2- 172.21.15.45:0/2983292263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2400b1050 0x7fb240151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.273+0000 7fb245ef8640 1 --2- 172.21.15.45:0/2983292263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb240071720 0x7fb2401f8080 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.273+0000 7fb245ef8640 1 --2- 172.21.15.45:0/2983292263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb24006c3a0 0x7fb2401f6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.273+0000 7fb245ef8640 1 -- 172.21.15.45:0/2983292263 >> 172.21.15.45:0/2983292263 conn(0x7fb2400b3c40 msgr2=0x7fb2400aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.273+0000 7fb245ef8640 1 -- 172.21.15.45:0/2983292263 shutdown_connections 2024-05-08T05:43:12.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.273+0000 7fb245ef8640 1 -- 172.21.15.45:0/2983292263 wait complete. 2024-05-08T05:43:12.284 INFO:tasks.cephfs.filesystem:scrub status for tag:8c95a750-4f85-466e-a42c-ba8f336e7c3a - {'path': '/', 'tag': '8c95a750-4f85-466e-a42c-ba8f336e7c3a', 'options': 'recursive,repair'} 2024-05-08T05:43:12.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:12.927+0000 7f1d7dffb640 1 -- 172.21.15.45:0/4230210253 <== mon.0 v2:172.21.15.45:3300/0 7 ==== log(6 entries from seq 1 at 2024-05-08T05:43:12.115346+0000) ==== 1078+0+0 (secure 0 0 0) 0x7f1d78095ed0 con 0x7f1d880d3a30 2024-05-08T05:43:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: cluster: 2024-05-08T05:43:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: id: e9e1170a-fd27-4625-97a9-5d5f104d13a2 2024-05-08T05:43:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: health: HEALTH_OK 2024-05-08T05:43:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: services: 2024-05-08T05:43:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-05-08T05:43:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: mgr: x(active, since 119s), standbys: y 2024-05-08T05:43:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: mds: 1/1 daemons up, 3 standby 2024-05-08T05:43:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: osd: 8 osds: 8 up (since 113s), 8 in (since 2m) 2024-05-08T05:43:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: data: 2024-05-08T05:43:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: volumes: 1/1 healthy 2024-05-08T05:43:12.928 INFO:teuthology.orchestra.run.smithi045.stdout: pools: 3 pools, 97 pgs 2024-05-08T05:43:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: objects: 24 objects, 579 KiB 2024-05-08T05:43:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: usage: 233 MiB used, 720 GiB / 720 GiB avail 2024-05-08T05:43:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: pgs: 97 active+clean 2024-05-08T05:43:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: io: 2024-05-08T05:43:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: client: 1.1 KiB/s wr, 0 op/s rd, 2 op/s wr 2024-05-08T05:43:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:12.929 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:12.929 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:43:12.115346+0000 mds.b [INF] scrub queued for path: / 2024-05-08T05:43:12.929 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:43:12.115349+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-05-08T05:43:12.929 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:43:12.115353+0000 mds.b [INF] scrub summary: active paths [/] 2024-05-08T05:43:12.929 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:43:12.116116+0000 mds.b [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-05-08T05:43:12.929 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:43:12.116126+0000 mds.b [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-05-08T05:43:12.929 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:43:12.116435+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-05-08T05:43:13.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:13.933+0000 7f1d7dffb640 1 -- 172.21.15.45:0/4230210253 <== mon.0 v2:172.21.15.45:3300/0 8 ==== log(4 entries from seq 112 at 2024-05-08T05:43:13.461453+0000) ==== 1120+0+0 (secure 0 0 0) 0x7f1d78098000 con 0x7f1d880d3a30 2024-05-08T05:43:14.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:14.938+0000 7f1d7dffb640 1 -- 172.21.15.45:0/4230210253 <== mon.0 v2:172.21.15.45:3300/0 9 ==== log(3 entries from seq 408 at 2024-05-08T05:43:13.929022+0000) ==== 709+0+0 (secure 0 0 0) 0x7f1d78060e00 con 0x7f1d880d3a30 2024-05-08T05:43:14.939 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:43:14.594328+0000 mds.b [INF] scrub summary: idle 2024-05-08T05:43:34.231 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:43:34.235 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:43:42.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:43:42.286 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-05-08T05:43:42.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.405+0000 7f8a587a5640 1 Processor -- start 2024-05-08T05:43:42.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.405+0000 7f8a587a5640 1 -- start start 2024-05-08T05:43:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.406+0000 7f8a587a5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8a50072840 0x7f8a5015c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.406+0000 7f8a587a5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8a5006e4d0 0x7f8a500aec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.406+0000 7f8a587a5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a5015c920 con 0x7f8a5006e0f0 2024-05-08T05:43:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.406+0000 7f8a587a5640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f8a5015cc00 con 0x7f8a50072840 2024-05-08T05:43:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.406+0000 7f8a587a5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8a5015cee0 con 0x7f8a5006e4d0 2024-05-08T05:43:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.406+0000 7f8a55d40640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8a50072840 0x7f8a5015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.406+0000 7f8a55d40640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8a50072840 0x7f8a5015c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58754/0 (socket says 172.21.15.45:58754) 2024-05-08T05:43:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.406+0000 7f8a55d40640 1 -- 172.21.15.45:0/3596980099 learned_addr learned my addr 172.21.15.45:0/3596980099 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.407+0000 7f8a56d42640 1 --2- 172.21.15.45:0/3596980099 >> v2:172.21.15.63:3300/0 conn(0x7f8a5006e4d0 0x7f8a500aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.407+0000 7f8a55d40640 1 -- 172.21.15.45:0/3596980099 >> v2:172.21.15.63:3300/0 conn(0x7f8a5006e4d0 msgr2=0x7f8a500aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.407+0000 7f8a55d40640 1 --2- 172.21.15.45:0/3596980099 >> v2:172.21.15.63:3300/0 conn(0x7f8a5006e4d0 0x7f8a500aec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.407+0000 7f8a55d40640 1 -- 172.21.15.45:0/3596980099 >> v1:172.21.15.63:6789/0 conn(0x7f8a5006e0f0 legacy=0x7f8a500d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:42.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.407+0000 7f8a55d40640 1 -- 172.21.15.45:0/3596980099 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5015d1c0 con 0x7f8a50072840 2024-05-08T05:43:42.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.407+0000 7f8a55d40640 1 --2- 172.21.15.45:0/3596980099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a50072840 0x7f8a5015c340 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8a50159f10 tx=0x7f8a4002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7bdc87d9a77ec9f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:42.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.408+0000 7f8a5553f640 1 -- 172.21.15.45:0/3596980099 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a40037070 con 0x7f8a50072840 2024-05-08T05:43:42.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.408+0000 7f8a5553f640 1 -- 172.21.15.45:0/3596980099 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a40032440 con 0x7f8a50072840 2024-05-08T05:43:42.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.408+0000 7f8a5553f640 1 -- 172.21.15.45:0/3596980099 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a40032810 con 0x7f8a50072840 2024-05-08T05:43:42.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.408+0000 7f8a587a5640 1 -- 172.21.15.45:0/3596980099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a50072840 msgr2=0x7f8a5015c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.408+0000 7f8a587a5640 1 --2- 172.21.15.45:0/3596980099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a50072840 0x7f8a5015c340 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8a50159f10 tx=0x7f8a4002a1e0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.408+0000 7f8a587a5640 1 -- 172.21.15.45:0/3596980099 shutdown_connections 2024-05-08T05:43:42.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.408+0000 7f8a587a5640 1 --2- 172.21.15.45:0/3596980099 >> v2:172.21.15.63:3300/0 conn(0x7f8a5006e4d0 0x7f8a500aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.408+0000 7f8a587a5640 1 --2- 172.21.15.45:0/3596980099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a50072840 0x7f8a5015c340 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.408+0000 7f8a587a5640 1 -- 172.21.15.45:0/3596980099 >> 172.21.15.45:0/3596980099 conn(0x7f8a5006c3a0 msgr2=0x7f8a5006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:42.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.408+0000 7f8a587a5640 1 -- 172.21.15.45:0/3596980099 shutdown_connections 2024-05-08T05:43:42.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.408+0000 7f8a587a5640 1 -- 172.21.15.45:0/3596980099 wait complete. 2024-05-08T05:43:42.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.409+0000 7f8a587a5640 1 Processor -- start 2024-05-08T05:43:42.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.410+0000 7f8a587a5640 1 -- start start 2024-05-08T05:43:42.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.410+0000 7f8a587a5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a5006e0f0 0x7f8a50158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.410+0000 7f8a587a5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a5006e4d0 0x7f8a500cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.410+0000 7f8a587a5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a50072840 0x7f8a50140c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.410+0000 7f8a587a5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8a500ae7d0 con 0x7f8a50072840 2024-05-08T05:43:42.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.410+0000 7f8a587a5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8a500ae650 con 0x7f8a5006e0f0 2024-05-08T05:43:42.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.410+0000 7f8a587a5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8a501530b0 con 0x7f8a5006e4d0 2024-05-08T05:43:42.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.410+0000 7f8a56d42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a50072840 0x7f8a50140c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.410+0000 7f8a56d42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a50072840 0x7f8a50140c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58760/0 (socket says 172.21.15.45:58760) 2024-05-08T05:43:42.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.410+0000 7f8a56d42640 1 -- 172.21.15.45:0/1168944855 learned_addr learned my addr 172.21.15.45:0/1168944855 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:42.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.411+0000 7f8a56541640 1 --2- 172.21.15.45:0/1168944855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a5006e0f0 0x7f8a50158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.411+0000 7f8a55d40640 1 --2- 172.21.15.45:0/1168944855 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a5006e4d0 0x7f8a500cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.411+0000 7f8a56d42640 1 -- 172.21.15.45:0/1168944855 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a5006e4d0 msgr2=0x7f8a500cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.411+0000 7f8a56d42640 1 --2- 172.21.15.45:0/1168944855 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a5006e4d0 0x7f8a500cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.411+0000 7f8a56d42640 1 -- 172.21.15.45:0/1168944855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a5006e0f0 msgr2=0x7f8a50158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.411+0000 7f8a56d42640 1 --2- 172.21.15.45:0/1168944855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a5006e0f0 0x7f8a50158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.411+0000 7f8a56d42640 1 -- 172.21.15.45:0/1168944855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5014d610 con 0x7f8a50072840 2024-05-08T05:43:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.411+0000 7f8a55d40640 1 --2- 172.21.15.45:0/1168944855 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a5006e4d0 0x7f8a500cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.411+0000 7f8a56d42640 1 --2- 172.21.15.45:0/1168944855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a50072840 0x7f8a50140c80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8a48007e40 tx=0x7f8a4800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.411+0000 7f8a46ffd640 1 -- 172.21.15.45:0/1168944855 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48014620 con 0x7f8a50072840 2024-05-08T05:43:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.411+0000 7f8a46ffd640 1 -- 172.21.15.45:0/1168944855 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a48019070 con 0x7f8a50072840 2024-05-08T05:43:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.411+0000 7f8a587a5640 1 -- 172.21.15.45:0/1168944855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5014d8f0 con 0x7f8a50072840 2024-05-08T05:43:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.411+0000 7f8a46ffd640 1 -- 172.21.15.45:0/1168944855 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48014a20 con 0x7f8a50072840 2024-05-08T05:43:42.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.411+0000 7f8a587a5640 1 -- 172.21.15.45:0/1168944855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a501591d0 con 0x7f8a50072840 2024-05-08T05:43:42.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.413+0000 7f8a46ffd640 1 -- 172.21.15.45:0/1168944855 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8a48014bc0 con 0x7f8a50072840 2024-05-08T05:43:42.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.413+0000 7f8a46ffd640 1 --2- 172.21.15.45:0/1168944855 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8a340762d0 0x7f8a34078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.414+0000 7f8a46ffd640 1 -- 172.21.15.45:0/1168944855 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6159+0+0 (secure 0 0 0) 0x7f8a48098020 con 0x7f8a50072840 2024-05-08T05:43:42.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.414+0000 7f8a56541640 1 --2- 172.21.15.45:0/1168944855 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8a340762d0 0x7f8a34078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.414+0000 7f8a5b595740 1 Processor -- start 2024-05-08T05:43:42.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.414+0000 7f8a56541640 1 --2- 172.21.15.45:0/1168944855 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8a340762d0 0x7f8a34078790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c004f60 tx=0x7f8a4c007e30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:42.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.414+0000 7f8a5b595740 1 -- start start 2024-05-08T05:43:42.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.415+0000 7f8a5b595740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b6677c160 0x559b6672eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.415+0000 7f8a5b595740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b6675f280 0x559b6675e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.415+0000 7f8a5b595740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559b6675eb40 0x559b66733950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.415+0000 7f8a5b595740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559b666fb690 con 0x559b6677c160 2024-05-08T05:43:42.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.415+0000 7f8a5b595740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559b66698f30 con 0x559b6675f280 2024-05-08T05:43:42.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.415+0000 7f8a5b595740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x559b66651b20 con 0x559b6675eb40 2024-05-08T05:43:42.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.415+0000 7f8a56541640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b6677c160 0x559b6672eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.415+0000 7f8a56541640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b6677c160 0x559b6672eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58772/0 (socket says 172.21.15.45:58772) 2024-05-08T05:43:42.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.415+0000 7f8a56541640 1 -- 172.21.15.45:0/3691482544 learned_addr learned my addr 172.21.15.45:0/3691482544 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:42.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.415+0000 7f8a55d40640 1 --2- 172.21.15.45:0/3691482544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b6675f280 0x559b6675e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.415+0000 7f8a56d42640 1 --2- 172.21.15.45:0/3691482544 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559b6675eb40 0x559b66733950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.416+0000 7f8a56541640 1 -- 172.21.15.45:0/3691482544 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559b6675eb40 msgr2=0x559b66733950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:42.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.416+0000 7f8a56541640 1 --2- 172.21.15.45:0/3691482544 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559b6675eb40 0x559b66733950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.416+0000 7f8a56541640 1 -- 172.21.15.45:0/3691482544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b6675f280 msgr2=0x559b6675e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:42.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.416+0000 7f8a56541640 1 --2- 172.21.15.45:0/3691482544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b6675f280 0x559b6675e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.416+0000 7f8a56541640 1 -- 172.21.15.45:0/3691482544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b66649e60 con 0x559b6677c160 2024-05-08T05:43:42.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.416+0000 7f8a56541640 1 --2- 172.21.15.45:0/3691482544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b6677c160 0x559b6672eaf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c003d00 tx=0x7f8a4c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e775919a1181a412 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:42.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.416+0000 7f8a33fff640 1 -- 172.21.15.45:0/3691482544 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c025070 con 0x559b6677c160 2024-05-08T05:43:42.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.416+0000 7f8a33fff640 1 -- 172.21.15.45:0/3691482544 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a4c01b050 con 0x559b6677c160 2024-05-08T05:43:42.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.416+0000 7f8a33fff640 1 -- 172.21.15.45:0/3691482544 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c0045d0 con 0x559b6677c160 2024-05-08T05:43:42.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.417+0000 7f8a5b595740 1 -- 172.21.15.45:0/3691482544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b6677c160 msgr2=0x559b6672eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.417+0000 7f8a5b595740 1 --2- 172.21.15.45:0/3691482544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b6677c160 0x559b6672eaf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c003d00 tx=0x7f8a4c012000 comp rx=0 tx=0).stop 2024-05-08T05:43:42.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.417+0000 7f8a5b595740 1 -- 172.21.15.45:0/3691482544 shutdown_connections 2024-05-08T05:43:42.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.417+0000 7f8a5b595740 1 --2- 172.21.15.45:0/3691482544 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559b6675eb40 0x559b66733950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.417+0000 7f8a5b595740 1 --2- 172.21.15.45:0/3691482544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b6675f280 0x559b6675e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.417+0000 7f8a5b595740 1 --2- 172.21.15.45:0/3691482544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b6677c160 0x559b6672eaf0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.417+0000 7f8a5b595740 1 -- 172.21.15.45:0/3691482544 >> 172.21.15.45:0/3691482544 conn(0x559b66730d60 msgr2=0x559b66779600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:42.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.417+0000 7f8a5b595740 1 -- 172.21.15.45:0/3691482544 shutdown_connections 2024-05-08T05:43:42.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.417+0000 7f8a5b595740 1 -- 172.21.15.45:0/3691482544 wait complete. 2024-05-08T05:43:42.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.417+0000 7f8a5b595740 1 -- start start 2024-05-08T05:43:42.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.418+0000 7f8a5b595740 1 Processor -- start 2024-05-08T05:43:42.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a5b595740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559b6675f280 0x559b6675f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a5b595740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b6677c160 0x559b6677ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a5b595740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b667702c0 0x559b6674b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a5b595740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559b666555e0 con 0x559b667702c0 2024-05-08T05:43:42.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a5b595740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559b66694ef0 con 0x559b6677c160 2024-05-08T05:43:42.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a5b595740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x559b66732f10 con 0x559b6675f280 2024-05-08T05:43:42.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a5b595740 10 client.0 _ll_register_callbacks cb 0x559b66650320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:43:42.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a56d42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b667702c0 0x559b6674b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a56d42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b667702c0 0x559b6674b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58774/0 (socket says 172.21.15.45:58774) 2024-05-08T05:43:42.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a56d42640 1 -- 172.21.15.45:0/1079817039 learned_addr learned my addr 172.21.15.45:0/1079817039 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:42.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a56541640 1 --2- 172.21.15.45:0/1079817039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559b6675f280 0x559b6675f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a55d40640 1 --2- 172.21.15.45:0/1079817039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b6677c160 0x559b6677ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a56d42640 1 -- 172.21.15.45:0/1079817039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559b6675f280 msgr2=0x559b6675f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:42.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a56d42640 1 --2- 172.21.15.45:0/1079817039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559b6675f280 0x559b6675f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a56d42640 1 -- 172.21.15.45:0/1079817039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b6677c160 msgr2=0x559b6677ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:42.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a56d42640 1 --2- 172.21.15.45:0/1079817039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b6677c160 0x559b6677ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a56d42640 1 -- 172.21.15.45:0/1079817039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b664747d0 con 0x559b667702c0 2024-05-08T05:43:42.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a56d42640 1 --2- 172.21.15.45:0/1079817039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b667702c0 0x559b6674b900 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8a4801d000 tx=0x7f8a48060c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca50d8d55a574e3b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:42.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a317fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:43:42.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a317fa640 1 -- 172.21.15.45:0/1079817039 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a480610c0 con 0x559b667702c0 2024-05-08T05:43:42.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a317fa640 1 -- 172.21.15.45:0/1079817039 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(49..49 src has 1..49) ==== 6159+0+0 (secure 0 0 0) 0x7f8a48060530 con 0x559b667702c0 2024-05-08T05:43:42.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a5b595740 1 -- 172.21.15.45:0/1079817039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a4c01b050 con 0x559b667702c0 2024-05-08T05:43:42.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a317fa640 1 -- 172.21.15.45:0/1079817039 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8a48096050 con 0x559b667702c0 2024-05-08T05:43:42.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.419+0000 7f8a5b595740 10 client.5538 fetch_fsmap learned FSMap version 26 2024-05-08T05:43:42.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.420+0000 7f8a5b595740 1 -- 172.21.15.45:0/1079817039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x559b663aa3f0 con 0x559b667702c0 2024-05-08T05:43:42.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.420+0000 7f8a317fa640 1 -- 172.21.15.45:0/1079817039 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a4c01b050 con 0x559b667702c0 2024-05-08T05:43:42.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.420+0000 7f8a317fa640 1 -- 172.21.15.45:0/1079817039 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f8a4805ca30 con 0x559b667702c0 2024-05-08T05:43:42.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.420+0000 7f8a5b595740 10 client.5538 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T05:43:42.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.420+0000 7f8a5b595740 10 client.5538 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-05-08T05:43:42.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.420+0000 7f8a5b595740 20 client.5538 populate_metadata read hostname 'smithi045' 2024-05-08T05:43:42.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.420+0000 7f8a5b595740 1 --2- 172.21.15.45:0/1079817039 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x559b6678ce40 0x559b6678d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.420+0000 7f8a5b595740 4 client.5538 mds_command: new command op to 5292 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:43:42.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.420+0000 7f8a5b595740 1 -- 172.21.15.45:0/1079817039 --> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b666a6b40 con 0x559b6678ce40 2024-05-08T05:43:42.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.420+0000 7f8a56541640 1 --2- 172.21.15.45:0/1079817039 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x559b6678ce40 0x559b6678d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.420+0000 7f8a56541640 1 --2- 172.21.15.45:0/1079817039 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x559b6678ce40 0x559b6678d250 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4046b495c2d9fc3e server_cookie=2b737caca82eff3b in_seq=0 out_seq=0 2024-05-08T05:43:42.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.420+0000 7f8a317fa640 10 client.5538 ms_handle_connect on v2:172.21.15.63:6834/4205418057 2024-05-08T05:43:42.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.421+0000 7f8a317fa640 1 -- 172.21.15.45:0/1079817039 <== mds.0 v2:172.21.15.63:6834/4205418057 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559b666a6b40 con 0x559b6678ce40 2024-05-08T05:43:42.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.421+0000 7f8a317fa640 10 client.5538 handle_command_reply: tid=0 2024-05-08T05:43:42.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.421+0000 7f8a5b595740 1 client.5538 shutdown 2024-05-08T05:43:42.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.421+0000 7f8a5b595740 1 -- 172.21.15.45:0/1079817039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b667702c0 msgr2=0x559b6674b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.421+0000 7f8a5b595740 1 --2- 172.21.15.45:0/1079817039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b667702c0 0x559b6674b900 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8a4801d000 tx=0x7f8a48060c40 comp rx=0 tx=0).stop 2024-05-08T05:43:42.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.421+0000 7f8a5b595740 1 -- 172.21.15.45:0/1079817039 shutdown_connections 2024-05-08T05:43:42.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.421+0000 7f8a5b595740 1 --2- 172.21.15.45:0/1079817039 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x559b6678ce40 0x559b6678d250 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.421+0000 7f8a5b595740 1 --2- 172.21.15.45:0/1079817039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b667702c0 0x559b6674b900 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.421+0000 7f8a5b595740 1 --2- 172.21.15.45:0/1079817039 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b6677c160 0x559b6677ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.421+0000 7f8a5b595740 1 --2- 172.21.15.45:0/1079817039 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559b6675f280 0x559b6675f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.421+0000 7f8a317fa640 0 client.5538 ms_handle_reset on v2:172.21.15.63:6834/4205418057 2024-05-08T05:43:42.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.421+0000 7f8a5b595740 1 -- 172.21.15.45:0/1079817039 >> 172.21.15.45:0/1079817039 conn(0x559b6675eb40 msgr2=0x559b6672e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:42.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.422+0000 7f8a5b595740 1 -- 172.21.15.45:0/1079817039 shutdown_connections 2024-05-08T05:43:42.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.422+0000 7f8a5b595740 1 -- 172.21.15.45:0/1079817039 wait complete. 2024-05-08T05:43:42.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.422+0000 7f8a5b595740 20 client.5538 trim_cache size 0 max 16384 2024-05-08T05:43:42.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.440+0000 7f8a5b595740 1 Processor -- start 2024-05-08T05:43:42.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.440+0000 7f8a5b595740 1 -- start start 2024-05-08T05:43:42.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.441+0000 7f8a5b595740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b6675f280 0x559b66759630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.441+0000 7f8a5b595740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559b667702c0 0x559b66759b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.441+0000 7f8a5b595740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b6677c160 0x559b6674b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.441+0000 7f8a5b595740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501ffc60 con 0x559b6677c160 2024-05-08T05:43:42.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.441+0000 7f8a5b595740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501fed10 con 0x559b6675f280 2024-05-08T05:43:42.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.441+0000 7f8a5b595740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8a5014d0f0 con 0x559b667702c0 2024-05-08T05:43:42.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.441+0000 7f8a56d42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b6677c160 0x559b6674b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.441+0000 7f8a56d42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b6677c160 0x559b6674b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58790/0 (socket says 172.21.15.45:58790) 2024-05-08T05:43:42.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.441+0000 7f8a56d42640 1 -- 172.21.15.45:0/4004700231 learned_addr learned my addr 172.21.15.45:0/4004700231 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:42.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.441+0000 7f8a56541640 1 --2- 172.21.15.45:0/4004700231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b6675f280 0x559b66759630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.441+0000 7f8a55d40640 1 --2- 172.21.15.45:0/4004700231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559b667702c0 0x559b66759b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.441+0000 7f8a56d42640 1 -- 172.21.15.45:0/4004700231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559b667702c0 msgr2=0x559b66759b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:42.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.441+0000 7f8a56d42640 1 --2- 172.21.15.45:0/4004700231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559b667702c0 0x559b66759b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.441+0000 7f8a56d42640 1 -- 172.21.15.45:0/4004700231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b6675f280 msgr2=0x559b66759630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:42.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.441+0000 7f8a56d42640 1 --2- 172.21.15.45:0/4004700231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b6675f280 0x559b66759630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.441+0000 7f8a56d42640 1 -- 172.21.15.45:0/4004700231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b6636a0f0 con 0x559b6677c160 2024-05-08T05:43:42.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.441+0000 7f8a56d42640 1 --2- 172.21.15.45:0/4004700231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b6677c160 0x559b6674b180 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8a4805d6b0 tx=0x7f8a48063000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa3ad66270f6b7d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:42.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.441+0000 7f8a337fe640 1 -- 172.21.15.45:0/4004700231 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a480610c0 con 0x559b6677c160 2024-05-08T05:43:42.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.441+0000 7f8a337fe640 1 -- 172.21.15.45:0/4004700231 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a48065470 con 0x559b6677c160 2024-05-08T05:43:42.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.441+0000 7f8a337fe640 1 -- 172.21.15.45:0/4004700231 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48069e90 con 0x559b6677c160 2024-05-08T05:43:42.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.442+0000 7f8a5b595740 1 -- 172.21.15.45:0/4004700231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b6677c160 msgr2=0x559b6674b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.442+0000 7f8a5b595740 1 --2- 172.21.15.45:0/4004700231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b6677c160 0x559b6674b180 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8a4805d6b0 tx=0x7f8a48063000 comp rx=0 tx=0).stop 2024-05-08T05:43:42.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.442+0000 7f8a5b595740 1 -- 172.21.15.45:0/4004700231 shutdown_connections 2024-05-08T05:43:42.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.442+0000 7f8a5b595740 1 --2- 172.21.15.45:0/4004700231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559b667702c0 0x559b66759b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.442+0000 7f8a5b595740 1 --2- 172.21.15.45:0/4004700231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b6677c160 0x559b6674b180 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.442+0000 7f8a5b595740 1 --2- 172.21.15.45:0/4004700231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b6675f280 0x559b66759630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.442+0000 7f8a5b595740 1 -- 172.21.15.45:0/4004700231 >> 172.21.15.45:0/4004700231 conn(0x559b6675eb40 msgr2=0x559b6677ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.442+0000 7f8a5b595740 1 -- 172.21.15.45:0/4004700231 shutdown_connections 2024-05-08T05:43:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.442+0000 7f8a5b595740 1 -- 172.21.15.45:0/4004700231 wait complete. 2024-05-08T05:43:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.442+0000 7f8a5b595740 1 -- start start 2024-05-08T05:43:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.443+0000 7f8a5b595740 1 Processor -- start 2024-05-08T05:43:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.443+0000 7f8a5b595740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559b6675f280 0x559b66759630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.443+0000 7f8a5b595740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b667702c0 0x559b66759b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.443+0000 7f8a5b595740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b6677c160 0x559b66761640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.443+0000 7f8a5b595740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501fe6c0 con 0x559b667702c0 2024-05-08T05:43:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.443+0000 7f8a5b595740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559b6678dd40 con 0x559b6677c160 2024-05-08T05:43:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.443+0000 7f8a5b595740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x559b6678c380 con 0x559b6675f280 2024-05-08T05:43:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.444+0000 7f8a5b595740 10 client.0 _ll_register_callbacks cb 0x7f8a48007d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:43:42.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.444+0000 7f8a55d40640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b667702c0 0x559b66759b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.444+0000 7f8a55d40640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b667702c0 0x559b66759b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58792/0 (socket says 172.21.15.45:58792) 2024-05-08T05:43:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.444+0000 7f8a55d40640 1 -- 172.21.15.45:0/584148043 learned_addr learned my addr 172.21.15.45:0/584148043 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.444+0000 7f8a56541640 1 --2- 172.21.15.45:0/584148043 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559b6675f280 0x559b66759630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.444+0000 7f8a56d42640 1 --2- 172.21.15.45:0/584148043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b6677c160 0x559b66761640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.444+0000 7f8a55d40640 1 -- 172.21.15.45:0/584148043 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559b6675f280 msgr2=0x559b66759630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.444+0000 7f8a55d40640 1 --2- 172.21.15.45:0/584148043 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559b6675f280 0x559b66759630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.444+0000 7f8a55d40640 1 -- 172.21.15.45:0/584148043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b6677c160 msgr2=0x559b66761640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.444+0000 7f8a55d40640 1 --2- 172.21.15.45:0/584148043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b6677c160 0x559b66761640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.444+0000 7f8a55d40640 1 -- 172.21.15.45:0/584148043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b6675e920 con 0x559b667702c0 2024-05-08T05:43:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.444+0000 7f8a55d40640 1 --2- 172.21.15.45:0/584148043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b667702c0 0x559b66759b70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009960 tx=0x7f8a4002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad50ff228c2d27f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.444+0000 7f8a317fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:43:42.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.444+0000 7f8a317fa640 1 -- 172.21.15.45:0/584148043 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4004a060 con 0x559b667702c0 2024-05-08T05:43:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.444+0000 7f8a317fa640 1 -- 172.21.15.45:0/584148043 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(49..49 src has 1..49) ==== 6159+0+0 (secure 0 0 0) 0x7f8a40045460 con 0x559b667702c0 2024-05-08T05:43:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.444+0000 7f8a5b595740 1 -- 172.21.15.45:0/584148043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a48065470 con 0x559b667702c0 2024-05-08T05:43:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.444+0000 7f8a317fa640 1 -- 172.21.15.45:0/584148043 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8a40050050 con 0x559b667702c0 2024-05-08T05:43:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.445+0000 7f8a5b595740 10 client.5544 fetch_fsmap learned FSMap version 26 2024-05-08T05:43:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.445+0000 7f8a5b595740 1 -- 172.21.15.45:0/584148043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x559b6678c180 con 0x559b667702c0 2024-05-08T05:43:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.445+0000 7f8a317fa640 1 -- 172.21.15.45:0/584148043 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a48065470 con 0x559b667702c0 2024-05-08T05:43:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.445+0000 7f8a317fa640 1 -- 172.21.15.45:0/584148043 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f8a40053020 con 0x559b667702c0 2024-05-08T05:43:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.445+0000 7f8a5b595740 10 client.5544 fetch_fsmap finished waiting for FSMap version 26 2024-05-08T05:43:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.445+0000 7f8a5b595740 10 client.5544 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-05-08T05:43:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.445+0000 7f8a5b595740 20 client.5544 populate_metadata read hostname 'smithi045' 2024-05-08T05:43:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.445+0000 7f8a5b595740 1 --2- 172.21.15.45:0/584148043 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x559b6674b8e0 0x559b66761b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.445+0000 7f8a5b595740 4 client.5544 mds_command: new command op to 5292 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:43:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.445+0000 7f8a5b595740 1 -- 172.21.15.45:0/584148043 --> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559b666a09b0 con 0x559b6674b8e0 2024-05-08T05:43:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.445+0000 7f8a56541640 1 --2- 172.21.15.45:0/584148043 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x559b6674b8e0 0x559b66761b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.445+0000 7f8a56541640 1 --2- 172.21.15.45:0/584148043 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x559b6674b8e0 0x559b66761b80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b8c8365dde0ee1 server_cookie=fd84b85532b4df66 in_seq=0 out_seq=0 2024-05-08T05:43:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.445+0000 7f8a317fa640 10 client.5544 ms_handle_connect on v2:172.21.15.63:6834/4205418057 2024-05-08T05:43:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.446+0000 7f8a317fa640 1 -- 172.21.15.45:0/584148043 <== mds.0 v2:172.21.15.63:6834/4205418057 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559b666a09b0 con 0x559b6674b8e0 2024-05-08T05:43:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.446+0000 7f8a317fa640 10 client.5544 handle_command_reply: tid=0 2024-05-08T05:43:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.446+0000 7f8a5b595740 1 client.5544 shutdown 2024-05-08T05:43:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.446+0000 7f8a5b595740 1 -- 172.21.15.45:0/584148043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b667702c0 msgr2=0x559b66759b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.446+0000 7f8a5b595740 1 --2- 172.21.15.45:0/584148043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b667702c0 0x559b66759b70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009960 tx=0x7f8a4002c070 comp rx=0 tx=0).stop 2024-05-08T05:43:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.446+0000 7f8a5b595740 1 -- 172.21.15.45:0/584148043 shutdown_connections 2024-05-08T05:43:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.446+0000 7f8a5b595740 1 --2- 172.21.15.45:0/584148043 >> [v2:172.21.15.63:6834/4205418057,v1:172.21.15.63:6836/4205418057] conn(0x559b6674b8e0 0x559b66761b80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.446+0000 7f8a5b595740 1 --2- 172.21.15.45:0/584148043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559b6677c160 0x559b66761640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.446+0000 7f8a5b595740 1 --2- 172.21.15.45:0/584148043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b667702c0 0x559b66759b70 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.446+0000 7f8a5b595740 1 --2- 172.21.15.45:0/584148043 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559b6675f280 0x559b66759630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.446+0000 7f8a317fa640 0 client.5544 ms_handle_reset on v2:172.21.15.63:6834/4205418057 2024-05-08T05:43:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.446+0000 7f8a5b595740 1 -- 172.21.15.45:0/584148043 >> 172.21.15.45:0/584148043 conn(0x559b6675eb40 msgr2=0x559b66771b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.446+0000 7f8a5b595740 1 -- 172.21.15.45:0/584148043 shutdown_connections 2024-05-08T05:43:42.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.446+0000 7f8a5b595740 1 -- 172.21.15.45:0/584148043 wait complete. 2024-05-08T05:43:42.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.446+0000 7f8a5b595740 20 client.5544 trim_cache size 0 max 16384 2024-05-08T05:43:42.455 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:43:42.455 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-05-08T05:43:42.455 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-05-08T05:43:42.455 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:43:42.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.447+0000 7f8a587a5640 1 -- 172.21.15.45:0/1168944855 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8a340762d0 msgr2=0x7f8a34078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.447+0000 7f8a587a5640 1 --2- 172.21.15.45:0/1168944855 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8a340762d0 0x7f8a34078790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c004f60 tx=0x7f8a4c007e30 comp rx=0 tx=0).stop 2024-05-08T05:43:42.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.447+0000 7f8a587a5640 1 -- 172.21.15.45:0/1168944855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a50072840 msgr2=0x7f8a50140c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.447+0000 7f8a587a5640 1 --2- 172.21.15.45:0/1168944855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a50072840 0x7f8a50140c80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8a48007e40 tx=0x7f8a4800e4b0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.447+0000 7f8a587a5640 1 -- 172.21.15.45:0/1168944855 shutdown_connections 2024-05-08T05:43:42.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.447+0000 7f8a587a5640 1 --2- 172.21.15.45:0/1168944855 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8a340762d0 0x7f8a34078790 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.447+0000 7f8a587a5640 1 --2- 172.21.15.45:0/1168944855 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a5006e4d0 0x7f8a500cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.447+0000 7f8a587a5640 1 --2- 172.21.15.45:0/1168944855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a50072840 0x7f8a50140c80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.447+0000 7f8a587a5640 1 --2- 172.21.15.45:0/1168944855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a5006e0f0 0x7f8a50158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.447+0000 7f8a587a5640 1 -- 172.21.15.45:0/1168944855 >> 172.21.15.45:0/1168944855 conn(0x7f8a5006c3a0 msgr2=0x7f8a50073300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:42.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.447+0000 7f8a587a5640 1 -- 172.21.15.45:0/1168944855 shutdown_connections 2024-05-08T05:43:42.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.447+0000 7f8a587a5640 1 -- 172.21.15.45:0/1168944855 wait complete. 2024-05-08T05:43:42.458 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T05:43:42.559 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:43:42.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.663+0000 7f7076f32640 1 Processor -- start 2024-05-08T05:43:42.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.663+0000 7f7076f32640 1 -- start start 2024-05-08T05:43:42.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.664+0000 7f7076f32640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70700d3a40 0x7f70700d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.664+0000 7f7076f32640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7070162bd0 0x7f70700716d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.664+0000 7f7076f32640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7070070820 con 0x7f70701627f0 2024-05-08T05:43:42.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.664+0000 7f7076f32640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7070070b30 con 0x7f7070162bd0 2024-05-08T05:43:42.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.664+0000 7f7076f32640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7070071c10 con 0x7f70700d3a40 2024-05-08T05:43:42.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.665+0000 7f706ffff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70700d3a40 0x7f70700d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.665+0000 7f70754cf640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7070162bd0 0x7f70700716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.665+0000 7f706ffff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70700d3a40 0x7f70700d0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:46012/0 (socket says 172.21.15.45:46012) 2024-05-08T05:43:42.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.665+0000 7f706ffff640 1 -- 172.21.15.45:0/4187848476 learned_addr learned my addr 172.21.15.45:0/4187848476 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:42.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.665+0000 7f7074cce640 1 --1- 172.21.15.45:0/4187848476 >> v1:172.21.15.63:6789/0 conn(0x7f70701627f0 0x7f7070070fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:57614/0 (socket says 172.21.15.45:57614) 2024-05-08T05:43:42.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.665+0000 7f706ffff640 1 -- 172.21.15.45:0/4187848476 >> v2:172.21.15.63:3300/0 conn(0x7f7070162bd0 msgr2=0x7f70700716d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:42.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.665+0000 7f706ffff640 1 --2- 172.21.15.45:0/4187848476 >> v2:172.21.15.63:3300/0 conn(0x7f7070162bd0 0x7f70700716d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.665+0000 7f706f7fe640 1 -- 172.21.15.45:0/4187848476 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3671268530 0 0) 0x7f7060001110 con 0x7f70701627f0 2024-05-08T05:43:42.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.665+0000 7f706ffff640 1 -- 172.21.15.45:0/4187848476 >> v1:172.21.15.63:6789/0 conn(0x7f70701627f0 legacy=0x7f7070070fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.665+0000 7f706ffff640 1 -- 172.21.15.45:0/4187848476 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70700d0270 con 0x7f70700d3a40 2024-05-08T05:43:42.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.666+0000 7f706ffff640 1 --2- 172.21.15.45:0/4187848476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70700d3a40 0x7f70700d0d40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7064009870 tx=0x7f706402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba135adb31196202 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:42.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.666+0000 7f706f7fe640 1 -- 172.21.15.45:0/4187848476 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7064037070 con 0x7f70700d3a40 2024-05-08T05:43:42.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.666+0000 7f706f7fe640 1 -- 172.21.15.45:0/4187848476 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7064032440 con 0x7f70700d3a40 2024-05-08T05:43:42.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.666+0000 7f706f7fe640 1 -- 172.21.15.45:0/4187848476 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7064032810 con 0x7f70700d3a40 2024-05-08T05:43:42.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.666+0000 7f7076f32640 1 -- 172.21.15.45:0/4187848476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70700d3a40 msgr2=0x7f70700d0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.666+0000 7f7076f32640 1 --2- 172.21.15.45:0/4187848476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70700d3a40 0x7f70700d0d40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7064009870 tx=0x7f706402a270 comp rx=0 tx=0).stop 2024-05-08T05:43:42.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.667+0000 7f7076f32640 1 -- 172.21.15.45:0/4187848476 shutdown_connections 2024-05-08T05:43:42.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.667+0000 7f7076f32640 1 --2- 172.21.15.45:0/4187848476 >> v2:172.21.15.63:3300/0 conn(0x7f7070162bd0 0x7f70700716d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.667+0000 7f7076f32640 1 --2- 172.21.15.45:0/4187848476 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f70700d3a40 0x7f70700d0d40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.667+0000 7f7076f32640 1 -- 172.21.15.45:0/4187848476 >> 172.21.15.45:0/4187848476 conn(0x7f70700d2e40 msgr2=0x7f70700d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:42.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.667+0000 7f7076f32640 1 -- 172.21.15.45:0/4187848476 shutdown_connections 2024-05-08T05:43:42.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.667+0000 7f7076f32640 1 -- 172.21.15.45:0/4187848476 wait complete. 2024-05-08T05:43:42.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.668+0000 7f7076f32640 1 Processor -- start 2024-05-08T05:43:42.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.668+0000 7f7076f32640 1 -- start start 2024-05-08T05:43:42.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.669+0000 7f7076f32640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70700d3a40 0x7f7070154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.669+0000 7f7076f32640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70701627f0 0x7f70700c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.669+0000 7f7076f32640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7070162bd0 0x7f7070152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.669+0000 7f7076f32640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f70700b2200 con 0x7f70700d3a40 2024-05-08T05:43:42.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.669+0000 7f7076f32640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f70700b2080 con 0x7f70701627f0 2024-05-08T05:43:42.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.669+0000 7f7076f32640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f70700b2380 con 0x7f7070162bd0 2024-05-08T05:43:42.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.669+0000 7f7074cce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70700d3a40 0x7f7070154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.669+0000 7f7074cce640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70700d3a40 0x7f7070154270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58800/0 (socket says 172.21.15.45:58800) 2024-05-08T05:43:42.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.669+0000 7f7074cce640 1 -- 172.21.15.45:0/3003993432 learned_addr learned my addr 172.21.15.45:0/3003993432 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:42.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.669+0000 7f706ffff640 1 --2- 172.21.15.45:0/3003993432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70701627f0 0x7f70700c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.669+0000 7f70754cf640 1 --2- 172.21.15.45:0/3003993432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7070162bd0 0x7f7070152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.669+0000 7f7074cce640 1 -- 172.21.15.45:0/3003993432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7070162bd0 msgr2=0x7f7070152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.669+0000 7f7074cce640 1 --2- 172.21.15.45:0/3003993432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7070162bd0 0x7f7070152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.669+0000 7f7074cce640 1 -- 172.21.15.45:0/3003993432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70701627f0 msgr2=0x7f70700c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.669+0000 7f7074cce640 1 --2- 172.21.15.45:0/3003993432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70701627f0 0x7f70700c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.669+0000 7f7074cce640 1 -- 172.21.15.45:0/3003993432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70700c96e0 con 0x7f70700d3a40 2024-05-08T05:43:42.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.670+0000 7f7074cce640 1 --2- 172.21.15.45:0/3003993432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70700d3a40 0x7f7070154270 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f706000c730 tx=0x7f706000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:42.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.670+0000 7f706d7fa640 1 -- 172.21.15.45:0/3003993432 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060018070 con 0x7f70700d3a40 2024-05-08T05:43:42.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.670+0000 7f706d7fa640 1 -- 172.21.15.45:0/3003993432 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70600070d0 con 0x7f70700d3a40 2024-05-08T05:43:42.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.670+0000 7f7076f32640 1 -- 172.21.15.45:0/3003993432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7070156090 con 0x7f70700d3a40 2024-05-08T05:43:42.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.670+0000 7f706d7fa640 1 -- 172.21.15.45:0/3003993432 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060007500 con 0x7f70700d3a40 2024-05-08T05:43:42.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.670+0000 7f7076f32640 1 -- 172.21.15.45:0/3003993432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70701537f0 con 0x7f70700d3a40 2024-05-08T05:43:42.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.671+0000 7f7076f32640 1 -- 172.21.15.45:0/3003993432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7070156260 con 0x7f70700d3a40 2024-05-08T05:43:42.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.671+0000 7f706d7fa640 1 -- 172.21.15.45:0/3003993432 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f706001b070 con 0x7f70700d3a40 2024-05-08T05:43:42.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.672+0000 7f706d7fa640 1 --2- 172.21.15.45:0/3003993432 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f705c0762d0 0x7f705c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.672+0000 7f706d7fa640 1 -- 172.21.15.45:0/3003993432 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6159+0+0 (secure 0 0 0) 0x7f70600969b0 con 0x7f70700d3a40 2024-05-08T05:43:42.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.672+0000 7f706ffff640 1 --2- 172.21.15.45:0/3003993432 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f705c0762d0 0x7f705c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.673+0000 7f706ffff640 1 --2- 172.21.15.45:0/3003993432 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f705c0762d0 0x7f705c078790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7064034000 tx=0x7f706402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:42.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.676+0000 7f706d7fa640 1 -- 172.21.15.45:0/3003993432 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7070070820 con 0x7f70700d3a40 2024-05-08T05:43:42.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.907+0000 7f7076f32640 1 -- 172.21.15.45:0/3003993432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70700693f0 con 0x7f70700d3a40 2024-05-08T05:43:42.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.908+0000 7f706d7fa640 1 -- 172.21.15.45:0/3003993432 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5181 (secure 0 0 0) 0x7f706005fb90 con 0x7f70700d3a40 2024-05-08T05:43:42.908 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:42.908 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4262647207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4262647207},{"type":"v1","addr":"172.21.15.63:6837","nonce":4262647207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5277,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2872370835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2872370835},{"type":"v1","addr":"172.21.15.45:6834","nonce":2872370835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1909488716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1909488716},{"type":"v1","addr":"172.21.15.45:6835","nonce":1909488716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:42:58.943500+0000","modified":"2024-05-08T05:43:01.967189+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6836/4205418057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":4205418057},{"type":"v1","addr":"172.21.15.63:6836","nonce":4205418057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5292,"qdb_cluster":[5292]},"id":3}]} 2024-05-08T05:43:42.909 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-05-08T05:43:42.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.910+0000 7f7076f32640 1 -- 172.21.15.45:0/3003993432 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f705c0762d0 msgr2=0x7f705c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.910+0000 7f7076f32640 1 --2- 172.21.15.45:0/3003993432 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f705c0762d0 0x7f705c078790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7064034000 tx=0x7f706402e000 comp rx=0 tx=0).stop 2024-05-08T05:43:42.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.910+0000 7f7076f32640 1 -- 172.21.15.45:0/3003993432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70700d3a40 msgr2=0x7f7070154270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.910+0000 7f7076f32640 1 --2- 172.21.15.45:0/3003993432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70700d3a40 0x7f7070154270 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f706000c730 tx=0x7f706000cbf0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.910+0000 7f7076f32640 1 -- 172.21.15.45:0/3003993432 shutdown_connections 2024-05-08T05:43:42.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.910+0000 7f7076f32640 1 --2- 172.21.15.45:0/3003993432 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f705c0762d0 0x7f705c078790 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.910+0000 7f7076f32640 1 --2- 172.21.15.45:0/3003993432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7070162bd0 0x7f7070152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.910+0000 7f7076f32640 1 --2- 172.21.15.45:0/3003993432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70701627f0 0x7f70700c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.910+0000 7f7076f32640 1 --2- 172.21.15.45:0/3003993432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70700d3a40 0x7f7070154270 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.910+0000 7f7076f32640 1 -- 172.21.15.45:0/3003993432 >> 172.21.15.45:0/3003993432 conn(0x7f70700d2e40 msgr2=0x7f70700b3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:42.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.910+0000 7f7076f32640 1 -- 172.21.15.45:0/3003993432 shutdown_connections 2024-05-08T05:43:42.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:42.910+0000 7f7076f32640 1 -- 172.21.15.45:0/3003993432 wait complete. 2024-05-08T05:43:42.921 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-05-08T05:43:43.061 INFO:teuthology.orchestra.run.smithi063.stdout:{"message":"","return_code":0} 2024-05-08T05:43:43.061 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-08T05:43:43.061 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-08T05:43:43.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.106+0000 7ff61931df40 1 Processor -- start 2024-05-08T05:43:43.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.106+0000 7ff61931df40 1 -- start start 2024-05-08T05:43:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.107+0000 7ff61931df40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a83305970 0x563a8332f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.107+0000 7ff61931df40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a833321a0 0x563a833bd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.107+0000 7ff61931df40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563a833d9320 0x563a83301ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.107+0000 7ff61931df40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563a832abf50 con 0x563a833321a0 2024-05-08T05:43:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.107+0000 7ff61931df40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563a832aff90 con 0x563a83305970 2024-05-08T05:43:43.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.107+0000 7ff61931df40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x563a83332cd0 con 0x563a833d9320 2024-05-08T05:43:43.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.107+0000 7ff615ba0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a833321a0 0x563a833bd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:43.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.107+0000 7ff615ba0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a833321a0 0x563a833bd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58826/0 (socket says 172.21.15.45:58826) 2024-05-08T05:43:43.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.107+0000 7ff615ba0640 1 -- 172.21.15.45:0/1126749973 learned_addr learned my addr 172.21.15.45:0/1126749973 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:43.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.107+0000 7ff6163a1640 1 --2- 172.21.15.45:0/1126749973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a83305970 0x563a8332f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:43.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.107+0000 7ff616ba2640 1 --2- 172.21.15.45:0/1126749973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563a833d9320 0x563a83301ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:43.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.108+0000 7ff615ba0640 1 -- 172.21.15.45:0/1126749973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563a833d9320 msgr2=0x563a83301ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:43.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.108+0000 7ff615ba0640 1 --2- 172.21.15.45:0/1126749973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563a833d9320 0x563a83301ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.108+0000 7ff615ba0640 1 -- 172.21.15.45:0/1126749973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a83305970 msgr2=0x563a8332f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:43.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.108+0000 7ff615ba0640 1 --2- 172.21.15.45:0/1126749973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a83305970 0x563a8332f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.108+0000 7ff615ba0640 1 -- 172.21.15.45:0/1126749973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a833bdc50 con 0x563a833321a0 2024-05-08T05:43:43.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.108+0000 7ff615ba0640 1 --2- 172.21.15.45:0/1126749973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a833321a0 0x563a833bd160 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff60c001e20 tx=0x7ff60c00b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee38422fa62ce83 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:43.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.108+0000 7ff614b9e640 1 -- 172.21.15.45:0/1126749973 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c00bde0 con 0x563a833321a0 2024-05-08T05:43:43.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.108+0000 7ff614b9e640 1 -- 172.21.15.45:0/1126749973 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff60c00f400 con 0x563a833321a0 2024-05-08T05:43:43.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.108+0000 7ff614b9e640 1 -- 172.21.15.45:0/1126749973 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c00f7f0 con 0x563a833321a0 2024-05-08T05:43:43.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.108+0000 7ff61931df40 1 -- 172.21.15.45:0/1126749973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a833321a0 msgr2=0x563a833bd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:43.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.108+0000 7ff61931df40 1 --2- 172.21.15.45:0/1126749973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a833321a0 0x563a833bd160 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff60c001e20 tx=0x7ff60c00b670 comp rx=0 tx=0).stop 2024-05-08T05:43:43.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.108+0000 7ff61931df40 1 -- 172.21.15.45:0/1126749973 shutdown_connections 2024-05-08T05:43:43.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.108+0000 7ff61931df40 1 --2- 172.21.15.45:0/1126749973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563a833d9320 0x563a83301ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.108+0000 7ff61931df40 1 --2- 172.21.15.45:0/1126749973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a833321a0 0x563a833bd160 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.108+0000 7ff61931df40 1 --2- 172.21.15.45:0/1126749973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a83305970 0x563a8332f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.108+0000 7ff61931df40 1 -- 172.21.15.45:0/1126749973 >> 172.21.15.45:0/1126749973 conn(0x563a833037e0 msgr2=0x563a8332d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:43.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.108+0000 7ff61931df40 1 -- 172.21.15.45:0/1126749973 shutdown_connections 2024-05-08T05:43:43.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.108+0000 7ff61931df40 1 -- 172.21.15.45:0/1126749973 wait complete. 2024-05-08T05:43:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.109+0000 7ff61931df40 1 Processor -- start 2024-05-08T05:43:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.109+0000 7ff61931df40 1 -- start start 2024-05-08T05:43:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.110+0000 7ff61931df40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563a83305970 0x563a833b97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.110+0000 7ff61931df40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a833321a0 0x563a833d8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.110+0000 7ff61931df40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a833d9320 0x563a833c1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.110+0000 7ff61931df40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563a833d9d60 con 0x563a833321a0 2024-05-08T05:43:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.110+0000 7ff61931df40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563a833d9a80 con 0x563a833d9320 2024-05-08T05:43:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.110+0000 7ff61931df40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x563a833da040 con 0x563a83305970 2024-05-08T05:43:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.110+0000 7ff615ba0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a833321a0 0x563a833d8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.110+0000 7ff615ba0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a833321a0 0x563a833d8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58828/0 (socket says 172.21.15.45:58828) 2024-05-08T05:43:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.110+0000 7ff6163a1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563a83305970 0x563a833b97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.110+0000 7ff615ba0640 1 -- 172.21.15.45:0/3348014631 learned_addr learned my addr 172.21.15.45:0/3348014631 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.110+0000 7ff616ba2640 1 --2- 172.21.15.45:0/3348014631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a833d9320 0x563a833c1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.110+0000 7ff615ba0640 1 -- 172.21.15.45:0/3348014631 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563a83305970 msgr2=0x563a833b97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.110+0000 7ff615ba0640 1 --2- 172.21.15.45:0/3348014631 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563a83305970 0x563a833b97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.110+0000 7ff615ba0640 1 -- 172.21.15.45:0/3348014631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a833d9320 msgr2=0x563a833c1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.110+0000 7ff615ba0640 1 --2- 172.21.15.45:0/3348014631 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563a833d9320 0x563a833c1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.110+0000 7ff615ba0640 1 -- 172.21.15.45:0/3348014631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a833c1120 con 0x563a833321a0 2024-05-08T05:43:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.110+0000 7ff615ba0640 1 --2- 172.21.15.45:0/3348014631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a833321a0 0x563a833d8290 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff60c000f60 tx=0x7ff60c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.111+0000 7ff6077fe640 1 -- 172.21.15.45:0/3348014631 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c020070 con 0x563a833321a0 2024-05-08T05:43:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.111+0000 7ff6077fe640 1 -- 172.21.15.45:0/3348014631 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff60c001260 con 0x563a833321a0 2024-05-08T05:43:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.111+0000 7ff61931df40 1 -- 172.21.15.45:0/3348014631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563a833c0e40 con 0x563a833321a0 2024-05-08T05:43:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.111+0000 7ff6077fe640 1 -- 172.21.15.45:0/3348014631 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c01c660 con 0x563a833321a0 2024-05-08T05:43:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.111+0000 7ff61931df40 1 -- 172.21.15.45:0/3348014631 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a833b93e0 con 0x563a833321a0 2024-05-08T05:43:43.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.112+0000 7ff6077fe640 1 -- 172.21.15.45:0/3348014631 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff60c023070 con 0x563a833321a0 2024-05-08T05:43:43.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.113+0000 7ff6077fe640 1 --2- 172.21.15.45:0/3348014631 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff5f8074cb0 0x7ff5f8077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:43.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.113+0000 7ff6077fe640 1 -- 172.21.15.45:0/3348014631 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6159+0+0 (secure 0 0 0) 0x7ff60c097a60 con 0x563a833321a0 2024-05-08T05:43:43.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.113+0000 7ff6163a1640 1 --2- 172.21.15.45:0/3348014631 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff5f8074cb0 0x7ff5f8077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:43.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.113+0000 7ff61931df40 1 --2- 172.21.15.45:0/3348014631 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x563a833d87d0 0x563a833c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:43.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.113+0000 7ff61931df40 1 -- 172.21.15.45:0/3348014631 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49) -- 0x563a8332db60 con 0x563a833d87d0 2024-05-08T05:43:43.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.113+0000 7ff616ba2640 1 --2- 172.21.15.45:0/3348014631 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x563a833d87d0 0x563a833c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:43.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.113+0000 7ff6163a1640 1 --2- 172.21.15.45:0/3348014631 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff5f8074cb0 0x7ff5f8077170 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff608057000 tx=0x7ff608052000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:43.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.113+0000 7ff616ba2640 1 --2- 172.21.15.45:0/3348014631 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x563a833d87d0 0x563a833c49b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:43.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.114+0000 7ff616ba2640 1 -- 172.21.15.45:0/3348014631 <== osd.1 v2:172.21.15.45:6824/1337179184 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7ff6100019a0 con 0x563a833d87d0 2024-05-08T05:43:43.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.114+0000 7ff61931df40 1 -- 172.21.15.45:0/3348014631 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x563a833d87d0 msgr2=0x563a833c49b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:43.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.114+0000 7ff61931df40 1 --2- 172.21.15.45:0/3348014631 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x563a833d87d0 0x563a833c49b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.114+0000 7ff61931df40 1 -- 172.21.15.45:0/3348014631 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff5f8074cb0 msgr2=0x7ff5f8077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:43.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.114+0000 7ff61931df40 1 --2- 172.21.15.45:0/3348014631 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff5f8074cb0 0x7ff5f8077170 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff608057000 tx=0x7ff608052000 comp rx=0 tx=0).stop 2024-05-08T05:43:43.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.114+0000 7ff61931df40 1 -- 172.21.15.45:0/3348014631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a833321a0 msgr2=0x563a833d8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:43.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.114+0000 7ff61931df40 1 --2- 172.21.15.45:0/3348014631 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563a833321a0 0x563a833d8290 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff60c000f60 tx=0x7ff60c000ed0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.114+0000 7ff616ba2640 1 -- 172.21.15.45:0/3348014631 reap_dead start 2024-05-08T05:43:43.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.114+0000 7ff61931df40 1 -- 172.21.15.45:0/3348014631 shutdown_connections 2024-05-08T05:43:43.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.114+0000 7ff61931df40 1 -- 172.21.15.45:0/3348014631 >> 172.21.15.45:0/3348014631 conn(0x563a833037e0 msgr2=0x563a8332f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:43.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.114+0000 7ff61931df40 1 -- 172.21.15.45:0/3348014631 shutdown_connections 2024-05-08T05:43:43.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.114+0000 7ff61931df40 1 -- 172.21.15.45:0/3348014631 wait complete. 2024-05-08T05:43:43.120 INFO:teuthology.orchestra.run.smithi045.stdout:Yalphaparent_a 2024-05-08T05:43:43.120 DEBUG:teuthology.orchestra.run.smithi045:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-08T05:43:43.168 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:43:43.168 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 1099511627777, 2024-05-08T05:43:43.168 INFO:teuthology.orchestra.run.smithi045.stdout: "ancestors": [ 2024-05-08T05:43:43.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:43.169 INFO:teuthology.orchestra.run.smithi045.stdout: "dirino": 1099511627776, 2024-05-08T05:43:43.169 INFO:teuthology.orchestra.run.smithi045.stdout: "dname": "alpha", 2024-05-08T05:43:43.169 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 4 2024-05-08T05:43:43.169 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:43:43.169 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:43.169 INFO:teuthology.orchestra.run.smithi045.stdout: "dirino": 1, 2024-05-08T05:43:43.169 INFO:teuthology.orchestra.run.smithi045.stdout: "dname": "parent_a", 2024-05-08T05:43:43.169 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 7 2024-05-08T05:43:43.169 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:43.169 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-05-08T05:43:43.169 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 8, 2024-05-08T05:43:43.169 INFO:teuthology.orchestra.run.smithi045.stdout: "old_pools": [] 2024-05-08T05:43:43.169 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:43:43.169 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:43.171 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T05:43:43.171 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:43:43.250 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:43:43.286 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T05:43:43.287 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:43:43.287 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:43:43.287 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:43:43.306 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:43:43.306 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:43:43.362 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:43:43.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.506+0000 7f1085f12640 1 Processor -- start 2024-05-08T05:43:43.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.506+0000 7f1085f12640 1 -- start start 2024-05-08T05:43:43.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.507+0000 7f1085f12640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f10801629b0 0x7f108006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:43.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.507+0000 7f1085f12640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f10800b2490 0x7f1080070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:43.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.507+0000 7f1085f12640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1080070d20 0x7f10800711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.507+0000 7f1085f12640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f108006dec0 con 0x7f10801629b0 2024-05-08T05:43:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.507+0000 7f1085f12640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f108006e110 con 0x7f10800b2490 2024-05-08T05:43:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.507+0000 7f1085f12640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f108006e3f0 con 0x7f1080070d20 2024-05-08T05:43:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.507+0000 7f107f7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f10801629b0 0x7f108006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.507+0000 7f107f7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f10801629b0 0x7f108006eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58842/0 (socket says 172.21.15.45:58842) 2024-05-08T05:43:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.507+0000 7f107f7fe640 1 -- 172.21.15.45:0/3228254964 learned_addr learned my addr 172.21.15.45:0/3228254964 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.507+0000 7f107effd640 1 --2- 172.21.15.45:0/3228254964 >> v2:172.21.15.63:3300/0 conn(0x7f10800b2490 0x7f1080070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.507+0000 7f107ffff640 1 --2- 172.21.15.45:0/3228254964 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1080070d20 0x7f10800711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.507+0000 7f107f7fe640 1 -- 172.21.15.45:0/3228254964 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1080070d20 msgr2=0x7f10800711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.507+0000 7f107f7fe640 1 --2- 172.21.15.45:0/3228254964 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1080070d20 0x7f10800711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.507+0000 7f107f7fe640 1 -- 172.21.15.45:0/3228254964 >> v2:172.21.15.63:3300/0 conn(0x7f10800b2490 msgr2=0x7f1080070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.507+0000 7f107f7fe640 1 --2- 172.21.15.45:0/3228254964 >> v2:172.21.15.63:3300/0 conn(0x7f10800b2490 0x7f1080070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.507+0000 7f107f7fe640 1 -- 172.21.15.45:0/3228254964 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108006e6d0 con 0x7f10801629b0 2024-05-08T05:43:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.507+0000 7f107ffff640 1 --2- 172.21.15.45:0/3228254964 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1080070d20 0x7f10800711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.508+0000 7f107f7fe640 1 --2- 172.21.15.45:0/3228254964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10801629b0 0x7f108006eed0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f10800b8fb0 tx=0x7f107002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb88309ae64131f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:43.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.508+0000 7f107e7fc640 1 -- 172.21.15.45:0/3228254964 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1070037070 con 0x7f10801629b0 2024-05-08T05:43:43.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.508+0000 7f107e7fc640 1 -- 172.21.15.45:0/3228254964 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1070032440 con 0x7f10801629b0 2024-05-08T05:43:43.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.508+0000 7f107e7fc640 1 -- 172.21.15.45:0/3228254964 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1070032810 con 0x7f10801629b0 2024-05-08T05:43:43.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.509+0000 7f1085f12640 1 -- 172.21.15.45:0/3228254964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10801629b0 msgr2=0x7f108006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.509+0000 7f1085f12640 1 --2- 172.21.15.45:0/3228254964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10801629b0 0x7f108006eed0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f10800b8fb0 tx=0x7f107002a1e0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.509+0000 7f1085f12640 1 -- 172.21.15.45:0/3228254964 shutdown_connections 2024-05-08T05:43:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.509+0000 7f1085f12640 1 --2- 172.21.15.45:0/3228254964 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1080070d20 0x7f10800711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.509+0000 7f1085f12640 1 --2- 172.21.15.45:0/3228254964 >> v2:172.21.15.63:3300/0 conn(0x7f10800b2490 0x7f1080070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.509+0000 7f1085f12640 1 --2- 172.21.15.45:0/3228254964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10801629b0 0x7f108006eed0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.509+0000 7f1085f12640 1 -- 172.21.15.45:0/3228254964 >> 172.21.15.45:0/3228254964 conn(0x7f10800b49d0 msgr2=0x7f10800b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.509+0000 7f1085f12640 1 -- 172.21.15.45:0/3228254964 shutdown_connections 2024-05-08T05:43:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.509+0000 7f1085f12640 1 -- 172.21.15.45:0/3228254964 wait complete. 2024-05-08T05:43:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.510+0000 7f1085f12640 1 Processor -- start 2024-05-08T05:43:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.510+0000 7f1085f12640 1 -- start start 2024-05-08T05:43:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.511+0000 7f1085f12640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1080070d20 0x7f10801f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.511+0000 7f1085f12640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10800b2490 0x7f10801592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.511+0000 7f1085f12640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10801629b0 0x7f10800cd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:43.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.511+0000 7f1085f12640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1080070870 con 0x7f10801629b0 2024-05-08T05:43:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.511+0000 7f1085f12640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f108006f410 con 0x7f1080070d20 2024-05-08T05:43:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.511+0000 7f1085f12640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f10800709f0 con 0x7f10800b2490 2024-05-08T05:43:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.511+0000 7f107ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10801629b0 0x7f10800cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.511+0000 7f107ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10801629b0 0x7f10800cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58850/0 (socket says 172.21.15.45:58850) 2024-05-08T05:43:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.511+0000 7f107ffff640 1 -- 172.21.15.45:0/238908078 learned_addr learned my addr 172.21.15.45:0/238908078 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.511+0000 7f107f7fe640 1 --2- 172.21.15.45:0/238908078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1080070d20 0x7f10801f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.511+0000 7f107effd640 1 --2- 172.21.15.45:0/238908078 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10800b2490 0x7f10801592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.511+0000 7f107ffff640 1 -- 172.21.15.45:0/238908078 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10800b2490 msgr2=0x7f10801592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.511+0000 7f107ffff640 1 --2- 172.21.15.45:0/238908078 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10800b2490 0x7f10801592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.511+0000 7f107ffff640 1 -- 172.21.15.45:0/238908078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1080070d20 msgr2=0x7f10801f8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.511+0000 7f107ffff640 1 --2- 172.21.15.45:0/238908078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1080070d20 0x7f10801f8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.511+0000 7f107ffff640 1 -- 172.21.15.45:0/238908078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10800cc910 con 0x7f10801629b0 2024-05-08T05:43:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.512+0000 7f107ffff640 1 --2- 172.21.15.45:0/238908078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10801629b0 0x7f10800cd470 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1078007de0 tx=0x7f107800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:43.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.512+0000 7f1067fff640 1 -- 172.21.15.45:0/238908078 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107801a070 con 0x7f10801629b0 2024-05-08T05:43:43.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.512+0000 7f1067fff640 1 -- 172.21.15.45:0/238908078 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1078003990 con 0x7f10801629b0 2024-05-08T05:43:43.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.512+0000 7f1085f12640 1 -- 172.21.15.45:0/238908078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10800ccbf0 con 0x7f10801629b0 2024-05-08T05:43:43.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.512+0000 7f1067fff640 1 -- 172.21.15.45:0/238908078 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078003d80 con 0x7f10801629b0 2024-05-08T05:43:43.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.512+0000 7f1085f12640 1 -- 172.21.15.45:0/238908078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10800ca390 con 0x7f10801629b0 2024-05-08T05:43:43.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.513+0000 7f1085f12640 1 -- 172.21.15.45:0/238908078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10800693f0 con 0x7f10801629b0 2024-05-08T05:43:43.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.513+0000 7f1067fff640 1 -- 172.21.15.45:0/238908078 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f107801d070 con 0x7f10801629b0 2024-05-08T05:43:43.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.514+0000 7f1067fff640 1 --2- 172.21.15.45:0/238908078 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f106c0762d0 0x7f106c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:43.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.514+0000 7f1067fff640 1 -- 172.21.15.45:0/238908078 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6159+0+0 (secure 0 0 0) 0x7f1078097ad0 con 0x7f10801629b0 2024-05-08T05:43:43.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.514+0000 7f107f7fe640 1 --2- 172.21.15.45:0/238908078 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f106c0762d0 0x7f106c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:43.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.515+0000 7f107f7fe640 1 --2- 172.21.15.45:0/238908078 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f106c0762d0 0x7f106c078790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1070034000 tx=0x7f107002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:43.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.518+0000 7f1067fff640 1 -- 172.21.15.45:0/238908078 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1078010070 con 0x7f10801629b0 2024-05-08T05:43:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.722+0000 7f1085f12640 1 -- 172.21.15.45:0/238908078 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f10800ccdc0 con 0x7f10801629b0 2024-05-08T05:43:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.722+0000 7f1067fff640 1 -- 172.21.15.45:0/238908078 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) ==== 80+0+0 (secure 0 0 0) 0x7f1078061270 con 0x7f10801629b0 2024-05-08T05:43:43.723 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-05-08T05:43:43.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.724+0000 7f1085f12640 1 -- 172.21.15.45:0/238908078 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f106c0762d0 msgr2=0x7f106c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:43.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.724+0000 7f1085f12640 1 --2- 172.21.15.45:0/238908078 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f106c0762d0 0x7f106c078790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1070034000 tx=0x7f107002e000 comp rx=0 tx=0).stop 2024-05-08T05:43:43.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.724+0000 7f1085f12640 1 -- 172.21.15.45:0/238908078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10801629b0 msgr2=0x7f10800cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:43.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.724+0000 7f1085f12640 1 --2- 172.21.15.45:0/238908078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10801629b0 0x7f10800cd470 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1078007de0 tx=0x7f107800d4a0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.724+0000 7f1085f12640 1 -- 172.21.15.45:0/238908078 shutdown_connections 2024-05-08T05:43:43.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.724+0000 7f1085f12640 1 --2- 172.21.15.45:0/238908078 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f106c0762d0 0x7f106c078790 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.724+0000 7f1085f12640 1 --2- 172.21.15.45:0/238908078 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f10800b2490 0x7f10801592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.724+0000 7f1085f12640 1 --2- 172.21.15.45:0/238908078 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10801629b0 0x7f10800cd470 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.724+0000 7f1085f12640 1 --2- 172.21.15.45:0/238908078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1080070d20 0x7f10801f8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:43.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.724+0000 7f1085f12640 1 -- 172.21.15.45:0/238908078 >> 172.21.15.45:0/238908078 conn(0x7f10800b49d0 msgr2=0x7f10800d18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:43.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.724+0000 7f1085f12640 1 -- 172.21.15.45:0/238908078 shutdown_connections 2024-05-08T05:43:43.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:43.724+0000 7f1085f12640 1 -- 172.21.15.45:0/238908078 wait complete. 2024-05-08T05:43:43.736 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T05:43:43.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:43:43.736 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T05:43:43.781 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:43:43.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:43:43.782 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T05:43:43.810 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:43:43.810 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:43:43.810 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:43:43.828 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:43:43.828 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:43:43.828 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:43:43.828 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:43:43.866 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:43:43.867 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:43:43.867 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:43:43.867 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:43:43.885 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:43:43.886 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:43:44.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.010+0000 7f6401483640 1 Processor -- start 2024-05-08T05:43:44.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.010+0000 7f6401483640 1 -- start start 2024-05-08T05:43:44.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.011+0000 7f6401483640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f63fc15f380 0x7f63fc0d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:44.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.011+0000 7f6401483640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f63fc0d3cd0 0x7f63fc0b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:44.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.011+0000 7f6401483640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63fc0d1760 con 0x7f63fc0d0cf0 2024-05-08T05:43:44.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.011+0000 7f6401483640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f63fc0d30f0 con 0x7f63fc0d3cd0 2024-05-08T05:43:44.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.011+0000 7f6401483640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f63fc0d2370 con 0x7f63fc15f380 2024-05-08T05:43:44.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.011+0000 7f63fb7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f63fc0d3cd0 0x7f63fc0b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:44.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.011+0000 7f63fb7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f63fc0d3cd0 0x7f63fc0b7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58852/0 (socket says 172.21.15.45:58852) 2024-05-08T05:43:44.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.011+0000 7f63fb7fe640 1 -- 172.21.15.45:0/884887424 learned_addr learned my addr 172.21.15.45:0/884887424 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:44.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.012+0000 7f63faffd640 1 --2- 172.21.15.45:0/884887424 >> v2:172.21.15.63:3300/0 conn(0x7f63fc15f380 0x7f63fc0d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:44.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.012+0000 7f63fb7fe640 1 -- 172.21.15.45:0/884887424 >> v2:172.21.15.63:3300/0 conn(0x7f63fc15f380 msgr2=0x7f63fc0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:44.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.012+0000 7f63fb7fe640 1 --2- 172.21.15.45:0/884887424 >> v2:172.21.15.63:3300/0 conn(0x7f63fc15f380 0x7f63fc0d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.012+0000 7f63fb7fe640 1 -- 172.21.15.45:0/884887424 >> v1:172.21.15.63:6789/0 conn(0x7f63fc0d0cf0 legacy=0x7f63fc0d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:44.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.012+0000 7f63fb7fe640 1 -- 172.21.15.45:0/884887424 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63fc0d1a40 con 0x7f63fc0d3cd0 2024-05-08T05:43:44.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.012+0000 7f63faffd640 1 --2- 172.21.15.45:0/884887424 >> v2:172.21.15.63:3300/0 conn(0x7f63fc15f380 0x7f63fc0d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:44.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.012+0000 7f63fb7fe640 1 --2- 172.21.15.45:0/884887424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63fc0d3cd0 0x7f63fc0b7e30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0b7ba0 tx=0x7f63f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07144cc16f59cdb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:44.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.012+0000 7f63f9ffb640 1 -- 172.21.15.45:0/884887424 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4037070 con 0x7f63fc0d3cd0 2024-05-08T05:43:44.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.013+0000 7f63f9ffb640 1 -- 172.21.15.45:0/884887424 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f4032440 con 0x7f63fc0d3cd0 2024-05-08T05:43:44.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.013+0000 7f63f9ffb640 1 -- 172.21.15.45:0/884887424 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4032810 con 0x7f63fc0d3cd0 2024-05-08T05:43:44.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.013+0000 7f6401483640 1 -- 172.21.15.45:0/884887424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63fc0d3cd0 msgr2=0x7f63fc0b7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:44.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.013+0000 7f6401483640 1 --2- 172.21.15.45:0/884887424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63fc0d3cd0 0x7f63fc0b7e30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0b7ba0 tx=0x7f63f402a1e0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.013+0000 7f6401483640 1 -- 172.21.15.45:0/884887424 shutdown_connections 2024-05-08T05:43:44.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.013+0000 7f6401483640 1 --2- 172.21.15.45:0/884887424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63fc0d3cd0 0x7f63fc0b7e30 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.013+0000 7f6401483640 1 --2- 172.21.15.45:0/884887424 >> v2:172.21.15.63:3300/0 conn(0x7f63fc15f380 0x7f63fc0d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.013+0000 7f6401483640 1 -- 172.21.15.45:0/884887424 >> 172.21.15.45:0/884887424 conn(0x7f63fc0d4fe0 msgr2=0x7f63fc0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:44.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.013+0000 7f6401483640 1 -- 172.21.15.45:0/884887424 shutdown_connections 2024-05-08T05:43:44.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.013+0000 7f6401483640 1 -- 172.21.15.45:0/884887424 wait complete. 2024-05-08T05:43:44.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.014+0000 7f6401483640 1 Processor -- start 2024-05-08T05:43:44.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.014+0000 7f6401483640 1 -- start start 2024-05-08T05:43:44.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.015+0000 7f6401483640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63fc0d0cf0 0x7f63fc159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:44.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.015+0000 7f6401483640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63fc0d3cd0 0x7f63fc14d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:44.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.015+0000 7f6401483640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63fc15f380 0x7f63fc15d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.015+0000 7f6401483640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc15f760 con 0x7f63fc0d0cf0 2024-05-08T05:43:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.015+0000 7f6401483640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc0d2000 con 0x7f63fc15f380 2024-05-08T05:43:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.015+0000 7f6401483640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f63fc0d2800 con 0x7f63fc0d3cd0 2024-05-08T05:43:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.015+0000 7f63faffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63fc0d0cf0 0x7f63fc159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.015+0000 7f63faffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63fc0d0cf0 0x7f63fc159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58854/0 (socket says 172.21.15.45:58854) 2024-05-08T05:43:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.015+0000 7f63faffd640 1 -- 172.21.15.45:0/3450238146 learned_addr learned my addr 172.21.15.45:0/3450238146 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.016+0000 7f63fa7fc640 1 --2- 172.21.15.45:0/3450238146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63fc0d3cd0 0x7f63fc14d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.016+0000 7f63fb7fe640 1 --2- 172.21.15.45:0/3450238146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63fc15f380 0x7f63fc15d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.016+0000 7f63faffd640 1 -- 172.21.15.45:0/3450238146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63fc0d3cd0 msgr2=0x7f63fc14d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.016+0000 7f63faffd640 1 --2- 172.21.15.45:0/3450238146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63fc0d3cd0 0x7f63fc14d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.016+0000 7f63faffd640 1 -- 172.21.15.45:0/3450238146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63fc15f380 msgr2=0x7f63fc15d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.016+0000 7f63faffd640 1 --2- 172.21.15.45:0/3450238146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63fc15f380 0x7f63fc15d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.016+0000 7f63faffd640 1 -- 172.21.15.45:0/3450238146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63fc159bc0 con 0x7f63fc0d0cf0 2024-05-08T05:43:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.016+0000 7f63fa7fc640 1 --2- 172.21.15.45:0/3450238146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63fc0d3cd0 0x7f63fc14d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.016+0000 7f63fb7fe640 1 --2- 172.21.15.45:0/3450238146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63fc15f380 0x7f63fc15d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.016+0000 7f63faffd640 1 --2- 172.21.15.45:0/3450238146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63fc0d0cf0 0x7f63fc159680 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00cf90 tx=0x7f63ec00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.016+0000 7f63f37fe640 1 -- 172.21.15.45:0/3450238146 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec01a070 con 0x7f63fc0d0cf0 2024-05-08T05:43:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.016+0000 7f63f37fe640 1 -- 172.21.15.45:0/3450238146 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63ec00d840 con 0x7f63fc0d0cf0 2024-05-08T05:43:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.016+0000 7f63f37fe640 1 -- 172.21.15.45:0/3450238146 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec00dc10 con 0x7f63fc0d0cf0 2024-05-08T05:43:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.016+0000 7f6401483640 1 -- 172.21.15.45:0/3450238146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63fc1585e0 con 0x7f63fc0d0cf0 2024-05-08T05:43:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.016+0000 7f6401483640 1 -- 172.21.15.45:0/3450238146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63fc158ba0 con 0x7f63fc0d0cf0 2024-05-08T05:43:44.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.017+0000 7f6401483640 1 -- 172.21.15.45:0/3450238146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63fc0d3af0 con 0x7f63fc0d0cf0 2024-05-08T05:43:44.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.018+0000 7f63f37fe640 1 -- 172.21.15.45:0/3450238146 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f63ec01d070 con 0x7f63fc0d0cf0 2024-05-08T05:43:44.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.018+0000 7f63f37fe640 1 --2- 172.21.15.45:0/3450238146 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f63dc0762d0 0x7f63dc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:44.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.018+0000 7f63f37fe640 1 -- 172.21.15.45:0/3450238146 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6159+0+0 (secure 0 0 0) 0x7f63ec097f60 con 0x7f63fc0d0cf0 2024-05-08T05:43:44.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.018+0000 7f63fa7fc640 1 --2- 172.21.15.45:0/3450238146 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f63dc0762d0 0x7f63dc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:44.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.019+0000 7f63fa7fc640 1 --2- 172.21.15.45:0/3450238146 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f63dc0762d0 0x7f63dc078790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0bb090 tx=0x7f63e4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:44.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.023+0000 7f63f37fe640 1 -- 172.21.15.45:0/3450238146 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f63fc0d3af0 con 0x7f63fc0d0cf0 2024-05-08T05:43:44.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.253+0000 7f6401483640 1 -- 172.21.15.45:0/3450238146 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63fc0693f0 con 0x7f63fc0d0cf0 2024-05-08T05:43:44.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.254+0000 7f63f37fe640 1 -- 172.21.15.45:0/3450238146 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5181 (secure 0 0 0) 0x7f63ec0610b0 con 0x7f63fc0d0cf0 2024-05-08T05:43:44.254 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:44.254 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4262647207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4262647207},{"type":"v1","addr":"172.21.15.63:6837","nonce":4262647207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5277,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2872370835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2872370835},{"type":"v1","addr":"172.21.15.45:6834","nonce":2872370835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1909488716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1909488716},{"type":"v1","addr":"172.21.15.45:6835","nonce":1909488716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:42:58.943500+0000","modified":"2024-05-08T05:43:01.967189+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6836/4205418057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":4205418057},{"type":"v1","addr":"172.21.15.63:6836","nonce":4205418057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5292,"qdb_cluster":[5292]},"id":3}]} 2024-05-08T05:43:44.254 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-05-08T05:43:44.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.255+0000 7f6401483640 1 -- 172.21.15.45:0/3450238146 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f63dc0762d0 msgr2=0x7f63dc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:44.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.255+0000 7f6401483640 1 --2- 172.21.15.45:0/3450238146 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f63dc0762d0 0x7f63dc078790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0bb090 tx=0x7f63e4008040 comp rx=0 tx=0).stop 2024-05-08T05:43:44.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.255+0000 7f6401483640 1 -- 172.21.15.45:0/3450238146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63fc0d0cf0 msgr2=0x7f63fc159680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:44.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.255+0000 7f6401483640 1 --2- 172.21.15.45:0/3450238146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63fc0d0cf0 0x7f63fc159680 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00cf90 tx=0x7f63ec00a7d0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.256+0000 7f6401483640 1 -- 172.21.15.45:0/3450238146 shutdown_connections 2024-05-08T05:43:44.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.256+0000 7f6401483640 1 --2- 172.21.15.45:0/3450238146 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f63dc0762d0 0x7f63dc078790 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.256+0000 7f6401483640 1 --2- 172.21.15.45:0/3450238146 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63fc0d3cd0 0x7f63fc14d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.256+0000 7f6401483640 1 --2- 172.21.15.45:0/3450238146 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63fc15f380 0x7f63fc15d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.256+0000 7f6401483640 1 --2- 172.21.15.45:0/3450238146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63fc0d0cf0 0x7f63fc159680 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.256+0000 7f6401483640 1 -- 172.21.15.45:0/3450238146 >> 172.21.15.45:0/3450238146 conn(0x7f63fc0d4fe0 msgr2=0x7f63fc15e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:44.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.256+0000 7f6401483640 1 -- 172.21.15.45:0/3450238146 shutdown_connections 2024-05-08T05:43:44.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.256+0000 7f6401483640 1 -- 172.21.15.45:0/3450238146 wait complete. 2024-05-08T05:43:44.267 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:43:44.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.370+0000 7f4b2cac4640 1 Processor -- start 2024-05-08T05:43:44.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.370+0000 7f4b2cac4640 1 -- start start 2024-05-08T05:43:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.371+0000 7f4b2cac4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4b281629b0 0x7f4b2806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.371+0000 7f4b2cac4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b280b2490 0x7f4b28070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.371+0000 7f4b2cac4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4b28070d20 0x7f4b280711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.371+0000 7f4b2cac4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4b2806dec0 con 0x7f4b281629b0 2024-05-08T05:43:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.371+0000 7f4b2cac4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4b2806e110 con 0x7f4b28070d20 2024-05-08T05:43:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.371+0000 7f4b2cac4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b2806e3f0 con 0x7f4b280b2490 2024-05-08T05:43:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.371+0000 7f4b2659c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4b281629b0 0x7f4b2806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.371+0000 7f4b2659c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4b281629b0 0x7f4b2806eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58858/0 (socket says 172.21.15.45:58858) 2024-05-08T05:43:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.371+0000 7f4b2659c640 1 -- 172.21.15.45:0/67213925 learned_addr learned my addr 172.21.15.45:0/67213925 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.371+0000 7f4b26d9d640 1 --2- 172.21.15.45:0/67213925 >> v2:172.21.15.63:3300/0 conn(0x7f4b28070d20 0x7f4b280711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.371+0000 7f4b25d9b640 1 --2- 172.21.15.45:0/67213925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b280b2490 0x7f4b28070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:44.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.371+0000 7f4b2659c640 1 -- 172.21.15.45:0/67213925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b280b2490 msgr2=0x7f4b28070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.371+0000 7f4b2659c640 1 --2- 172.21.15.45:0/67213925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b280b2490 0x7f4b28070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.371+0000 7f4b2659c640 1 -- 172.21.15.45:0/67213925 >> v2:172.21.15.63:3300/0 conn(0x7f4b28070d20 msgr2=0x7f4b280711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.371+0000 7f4b2659c640 1 --2- 172.21.15.45:0/67213925 >> v2:172.21.15.63:3300/0 conn(0x7f4b28070d20 0x7f4b280711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.371+0000 7f4b2659c640 1 -- 172.21.15.45:0/67213925 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2806e6d0 con 0x7f4b281629b0 2024-05-08T05:43:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.372+0000 7f4b2659c640 1 --2- 172.21.15.45:0/67213925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b281629b0 0x7f4b2806eed0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4b28071b40 tx=0x7f4b1802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da8ffcaadbf422b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.372+0000 7f4b2559a640 1 -- 172.21.15.45:0/67213925 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18037070 con 0x7f4b281629b0 2024-05-08T05:43:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.372+0000 7f4b2559a640 1 -- 172.21.15.45:0/67213925 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b18032440 con 0x7f4b281629b0 2024-05-08T05:43:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.372+0000 7f4b2559a640 1 -- 172.21.15.45:0/67213925 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18032810 con 0x7f4b281629b0 2024-05-08T05:43:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.373+0000 7f4b2cac4640 1 -- 172.21.15.45:0/67213925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b281629b0 msgr2=0x7f4b2806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.373+0000 7f4b2cac4640 1 --2- 172.21.15.45:0/67213925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b281629b0 0x7f4b2806eed0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4b28071b40 tx=0x7f4b1802a1e0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.373+0000 7f4b2cac4640 1 -- 172.21.15.45:0/67213925 shutdown_connections 2024-05-08T05:43:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.373+0000 7f4b2cac4640 1 --2- 172.21.15.45:0/67213925 >> v2:172.21.15.63:3300/0 conn(0x7f4b28070d20 0x7f4b280711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.373+0000 7f4b2cac4640 1 --2- 172.21.15.45:0/67213925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b280b2490 0x7f4b28070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.373+0000 7f4b2cac4640 1 --2- 172.21.15.45:0/67213925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b281629b0 0x7f4b2806eed0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.373+0000 7f4b2cac4640 1 -- 172.21.15.45:0/67213925 >> 172.21.15.45:0/67213925 conn(0x7f4b280b49d0 msgr2=0x7f4b280b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:44.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.373+0000 7f4b2cac4640 1 -- 172.21.15.45:0/67213925 shutdown_connections 2024-05-08T05:43:44.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.373+0000 7f4b2cac4640 1 -- 172.21.15.45:0/67213925 wait complete. 2024-05-08T05:43:44.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.374+0000 7f4b2cac4640 1 Processor -- start 2024-05-08T05:43:44.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.374+0000 7f4b2cac4640 1 -- start start 2024-05-08T05:43:44.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.375+0000 7f4b2cac4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b28070d20 0x7f4b28155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:44.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.375+0000 7f4b2cac4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b280b2490 0x7f4b280b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:44.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.375+0000 7f4b2cac4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b281629b0 0x7f4b280b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:44.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.375+0000 7f4b2cac4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28070870 con 0x7f4b280b2490 2024-05-08T05:43:44.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.375+0000 7f4b2cac4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2806f410 con 0x7f4b281629b0 2024-05-08T05:43:44.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.375+0000 7f4b2cac4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b280709f0 con 0x7f4b28070d20 2024-05-08T05:43:44.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.375+0000 7f4b25d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b280b2490 0x7f4b280b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:44.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.375+0000 7f4b25d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b280b2490 0x7f4b280b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58860/0 (socket says 172.21.15.45:58860) 2024-05-08T05:43:44.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.375+0000 7f4b25d9b640 1 -- 172.21.15.45:0/3554450303 learned_addr learned my addr 172.21.15.45:0/3554450303 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:44.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.376+0000 7f4b2659c640 1 --2- 172.21.15.45:0/3554450303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b28070d20 0x7f4b28155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:44.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.376+0000 7f4b26d9d640 1 --2- 172.21.15.45:0/3554450303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b281629b0 0x7f4b280b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:44.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.376+0000 7f4b25d9b640 1 -- 172.21.15.45:0/3554450303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b28070d20 msgr2=0x7f4b28155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:44.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.376+0000 7f4b25d9b640 1 --2- 172.21.15.45:0/3554450303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b28070d20 0x7f4b28155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.376+0000 7f4b25d9b640 1 -- 172.21.15.45:0/3554450303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b281629b0 msgr2=0x7f4b280b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:44.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.376+0000 7f4b25d9b640 1 --2- 172.21.15.45:0/3554450303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b281629b0 0x7f4b280b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.376+0000 7f4b25d9b640 1 -- 172.21.15.45:0/3554450303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b280ce440 con 0x7f4b280b2490 2024-05-08T05:43:44.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.376+0000 7f4b26d9d640 1 --2- 172.21.15.45:0/3554450303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b281629b0 0x7f4b280b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:44.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.376+0000 7f4b25d9b640 1 --2- 172.21.15.45:0/3554450303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b280b2490 0x7f4b280b0d00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00e9b0 tx=0x7f4b1c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:44.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.376+0000 7f4b16ffd640 1 -- 172.21.15.45:0/3554450303 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c017070 con 0x7f4b280b2490 2024-05-08T05:43:44.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.376+0000 7f4b16ffd640 1 -- 172.21.15.45:0/3554450303 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b1c00c9a0 con 0x7f4b280b2490 2024-05-08T05:43:44.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.376+0000 7f4b2cac4640 1 -- 172.21.15.45:0/3554450303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b280cd680 con 0x7f4b280b2490 2024-05-08T05:43:44.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.376+0000 7f4b16ffd640 1 -- 172.21.15.45:0/3554450303 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c00cd90 con 0x7f4b280b2490 2024-05-08T05:43:44.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.376+0000 7f4b2cac4640 1 -- 172.21.15.45:0/3554450303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b280cdc40 con 0x7f4b280b2490 2024-05-08T05:43:44.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.377+0000 7f4b2cac4640 1 -- 172.21.15.45:0/3554450303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b280693f0 con 0x7f4b280b2490 2024-05-08T05:43:44.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.378+0000 7f4b16ffd640 1 -- 172.21.15.45:0/3554450303 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4b1c01a070 con 0x7f4b280b2490 2024-05-08T05:43:44.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.378+0000 7f4b16ffd640 1 --2- 172.21.15.45:0/3554450303 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b08076300 0x7f4b080787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:44.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.378+0000 7f4b16ffd640 1 -- 172.21.15.45:0/3554450303 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6159+0+0 (secure 0 0 0) 0x7f4b1c0978e0 con 0x7f4b280b2490 2024-05-08T05:43:44.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.379+0000 7f4b2659c640 1 --2- 172.21.15.45:0/3554450303 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b08076300 0x7f4b080787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:44.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.379+0000 7f4b2659c640 1 --2- 172.21.15.45:0/3554450303 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b08076300 0x7f4b080787c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4b18034000 tx=0x7f4b1802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:44.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.383+0000 7f4b16ffd640 1 -- 172.21.15.45:0/3554450303 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4b1c060f50 con 0x7f4b280b2490 2024-05-08T05:43:44.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.610+0000 7f4b2cac4640 1 -- 172.21.15.45:0/3554450303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b2815fb40 con 0x7f4b280b2490 2024-05-08T05:43:44.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.611+0000 7f4b16ffd640 1 -- 172.21.15.45:0/3554450303 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5181 (secure 0 0 0) 0x7f4b1c060850 con 0x7f4b280b2490 2024-05-08T05:43:44.611 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:44.612 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4262647207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4262647207},{"type":"v1","addr":"172.21.15.63:6837","nonce":4262647207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5277,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2872370835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2872370835},{"type":"v1","addr":"172.21.15.45:6834","nonce":2872370835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1909488716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1909488716},{"type":"v1","addr":"172.21.15.45:6835","nonce":1909488716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:42:58.943500+0000","modified":"2024-05-08T05:43:01.967189+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6836/4205418057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":4205418057},{"type":"v1","addr":"172.21.15.63:6836","nonce":4205418057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5292,"qdb_cluster":[5292]},"id":3}]} 2024-05-08T05:43:44.612 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-05-08T05:43:44.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.613+0000 7f4b2cac4640 1 -- 172.21.15.45:0/3554450303 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b08076300 msgr2=0x7f4b080787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:44.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.613+0000 7f4b2cac4640 1 --2- 172.21.15.45:0/3554450303 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b08076300 0x7f4b080787c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4b18034000 tx=0x7f4b1802e000 comp rx=0 tx=0).stop 2024-05-08T05:43:44.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.613+0000 7f4b2cac4640 1 -- 172.21.15.45:0/3554450303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b280b2490 msgr2=0x7f4b280b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:44.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.613+0000 7f4b2cac4640 1 --2- 172.21.15.45:0/3554450303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b280b2490 0x7f4b280b0d00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00e9b0 tx=0x7f4b1c00ee70 comp rx=0 tx=0).stop 2024-05-08T05:43:44.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.613+0000 7f4b2cac4640 1 -- 172.21.15.45:0/3554450303 shutdown_connections 2024-05-08T05:43:44.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.613+0000 7f4b2cac4640 1 --2- 172.21.15.45:0/3554450303 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b08076300 0x7f4b080787c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.613+0000 7f4b2cac4640 1 --2- 172.21.15.45:0/3554450303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b281629b0 0x7f4b280b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.613+0000 7f4b2cac4640 1 --2- 172.21.15.45:0/3554450303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b280b2490 0x7f4b280b0d00 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.613+0000 7f4b2cac4640 1 --2- 172.21.15.45:0/3554450303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b28070d20 0x7f4b28155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.613+0000 7f4b2cac4640 1 -- 172.21.15.45:0/3554450303 >> 172.21.15.45:0/3554450303 conn(0x7f4b280b49d0 msgr2=0x7f4b280b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:44.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.613+0000 7f4b2cac4640 1 -- 172.21.15.45:0/3554450303 shutdown_connections 2024-05-08T05:43:44.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.613+0000 7f4b2cac4640 1 -- 172.21.15.45:0/3554450303 wait complete. 2024-05-08T05:43:44.624 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:43:44.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.727+0000 7f2b18ed1640 1 Processor -- start 2024-05-08T05:43:44.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.727+0000 7f2b18ed1640 1 -- start start 2024-05-08T05:43:44.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.728+0000 7f2b18ed1640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2b1415f420 0x7f2b140d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:44.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.728+0000 7f2b18ed1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2b140d3ca0 0x7f2b140d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:44.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.728+0000 7f2b18ed1640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b141630b0 con 0x7f2b140d1eb0 2024-05-08T05:43:44.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.728+0000 7f2b18ed1640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f2b140d2550 con 0x7f2b140d3ca0 2024-05-08T05:43:44.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.728+0000 7f2b18ed1640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2b140d2c80 con 0x7f2b1415f420 2024-05-08T05:43:44.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.729+0000 7f2b11d9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2b140d3ca0 0x7f2b140d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:44.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.729+0000 7f2b11d9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2b140d3ca0 0x7f2b140d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58866/0 (socket says 172.21.15.45:58866) 2024-05-08T05:43:44.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.729+0000 7f2b11d9b640 1 -- 172.21.15.45:0/761362421 learned_addr learned my addr 172.21.15.45:0/761362421 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:44.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.729+0000 7f2b1259c640 1 --2- 172.21.15.45:0/761362421 >> v2:172.21.15.63:3300/0 conn(0x7f2b1415f420 0x7f2b140d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:44.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.729+0000 7f2b11d9b640 1 -- 172.21.15.45:0/761362421 >> v2:172.21.15.63:3300/0 conn(0x7f2b1415f420 msgr2=0x7f2b140d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:44.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.729+0000 7f2b11d9b640 1 --2- 172.21.15.45:0/761362421 >> v2:172.21.15.63:3300/0 conn(0x7f2b1415f420 0x7f2b140d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.729+0000 7f2b11d9b640 1 -- 172.21.15.45:0/761362421 >> v1:172.21.15.63:6789/0 conn(0x7f2b140d1eb0 legacy=0x7f2b140d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:44.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.729+0000 7f2b11d9b640 1 -- 172.21.15.45:0/761362421 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b140d2f60 con 0x7f2b140d3ca0 2024-05-08T05:43:44.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.730+0000 7f2b11d9b640 1 --2- 172.21.15.45:0/761362421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b140d3ca0 0x7f2b140d1970 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2b140d5ec0 tx=0x7f2afc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f465238c667b3ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:44.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.730+0000 7f2b1159a640 1 -- 172.21.15.45:0/761362421 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc037070 con 0x7f2b140d3ca0 2024-05-08T05:43:44.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.730+0000 7f2b1159a640 1 -- 172.21.15.45:0/761362421 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2afc032440 con 0x7f2b140d3ca0 2024-05-08T05:43:44.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.730+0000 7f2b1159a640 1 -- 172.21.15.45:0/761362421 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc032810 con 0x7f2b140d3ca0 2024-05-08T05:43:44.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.730+0000 7f2b18ed1640 1 -- 172.21.15.45:0/761362421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b140d3ca0 msgr2=0x7f2b140d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:44.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.730+0000 7f2b18ed1640 1 --2- 172.21.15.45:0/761362421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b140d3ca0 0x7f2b140d1970 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2b140d5ec0 tx=0x7f2afc02a1e0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.730+0000 7f2b18ed1640 1 -- 172.21.15.45:0/761362421 shutdown_connections 2024-05-08T05:43:44.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.730+0000 7f2b18ed1640 1 --2- 172.21.15.45:0/761362421 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b140d3ca0 0x7f2b140d1970 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.730+0000 7f2b18ed1640 1 --2- 172.21.15.45:0/761362421 >> v2:172.21.15.63:3300/0 conn(0x7f2b1415f420 0x7f2b140d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.730+0000 7f2b18ed1640 1 -- 172.21.15.45:0/761362421 >> 172.21.15.45:0/761362421 conn(0x7f2b1408eac0 msgr2=0x7f2b1409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:44.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.730+0000 7f2b18ed1640 1 -- 172.21.15.45:0/761362421 shutdown_connections 2024-05-08T05:43:44.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.730+0000 7f2b18ed1640 1 -- 172.21.15.45:0/761362421 wait complete. 2024-05-08T05:43:44.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.732+0000 7f2b18ed1640 1 Processor -- start 2024-05-08T05:43:44.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.732+0000 7f2b18ed1640 1 -- start start 2024-05-08T05:43:44.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b18ed1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b140d1eb0 0x7f2b140cece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b18ed1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b1415e680 0x7f2b141e9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b18ed1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b1415e0f0 0x7f2b14161fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b18ed1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2b140d53d0 con 0x7f2b1415e0f0 2024-05-08T05:43:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b18ed1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b140d2830 con 0x7f2b1415e680 2024-05-08T05:43:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b18ed1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b140d5550 con 0x7f2b140d1eb0 2024-05-08T05:43:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b12d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b1415e0f0 0x7f2b14161fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b12d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b1415e0f0 0x7f2b14161fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58876/0 (socket says 172.21.15.45:58876) 2024-05-08T05:43:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b12d9d640 1 -- 172.21.15.45:0/1345286049 learned_addr learned my addr 172.21.15.45:0/1345286049 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b11d9b640 1 --2- 172.21.15.45:0/1345286049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b1415e680 0x7f2b141e9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b1259c640 1 --2- 172.21.15.45:0/1345286049 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b140d1eb0 0x7f2b140cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b12d9d640 1 -- 172.21.15.45:0/1345286049 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b140d1eb0 msgr2=0x7f2b140cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b12d9d640 1 --2- 172.21.15.45:0/1345286049 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b140d1eb0 0x7f2b140cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b12d9d640 1 -- 172.21.15.45:0/1345286049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b1415e680 msgr2=0x7f2b141e9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:44.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b12d9d640 1 --2- 172.21.15.45:0/1345286049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b1415e680 0x7f2b141e9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b12d9d640 1 -- 172.21.15.45:0/1345286049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b141e8b80 con 0x7f2b1415e0f0 2024-05-08T05:43:44.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b12d9d640 1 --2- 172.21.15.45:0/1345286049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b1415e0f0 0x7f2b14161fa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400c780 tx=0x7f2b0400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:44.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b0affd640 1 -- 172.21.15.45:0/1345286049 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04004b60 con 0x7f2b1415e0f0 2024-05-08T05:43:44.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b0affd640 1 -- 172.21.15.45:0/1345286049 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b04004d00 con 0x7f2b1415e0f0 2024-05-08T05:43:44.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b0affd640 1 -- 172.21.15.45:0/1345286049 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0400aa30 con 0x7f2b1415e0f0 2024-05-08T05:43:44.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.733+0000 7f2b18ed1640 1 -- 172.21.15.45:0/1345286049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b141e8e60 con 0x7f2b1415e0f0 2024-05-08T05:43:44.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.734+0000 7f2b18ed1640 1 -- 172.21.15.45:0/1345286049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b141e9420 con 0x7f2b1415e0f0 2024-05-08T05:43:44.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.735+0000 7f2b18ed1640 1 -- 172.21.15.45:0/1345286049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b140693f0 con 0x7f2b1415e0f0 2024-05-08T05:43:44.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.735+0000 7f2b0affd640 1 -- 172.21.15.45:0/1345286049 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2b0400abd0 con 0x7f2b1415e0f0 2024-05-08T05:43:44.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.736+0000 7f2b0affd640 1 --2- 172.21.15.45:0/1345286049 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2af40762d0 0x7f2af4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:44.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.736+0000 7f2b0affd640 1 -- 172.21.15.45:0/1345286049 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6159+0+0 (secure 0 0 0) 0x7f2b04097950 con 0x7f2b1415e0f0 2024-05-08T05:43:44.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.736+0000 7f2b1259c640 1 --2- 172.21.15.45:0/1345286049 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2af40762d0 0x7f2af4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:44.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.737+0000 7f2b1259c640 1 --2- 172.21.15.45:0/1345286049 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2af40762d0 0x7f2af4078790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c006fd0 tx=0x7f2b0c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:44.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.740+0000 7f2b0affd640 1 -- 172.21.15.45:0/1345286049 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2b04020070 con 0x7f2b1415e0f0 2024-05-08T05:43:44.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.932+0000 7f2b18ed1640 1 -- 172.21.15.45:0/1345286049 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b140b4c50 con 0x7f2b1415e0f0 2024-05-08T05:43:44.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.933+0000 7f2b0affd640 1 -- 172.21.15.45:0/1345286049 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16990 (secure 0 0 0) 0x7f2b04061180 con 0x7f2b1415e0f0 2024-05-08T05:43:44.934 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:44.934 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":49,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:43:14.934284+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:42:56.614794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T05:42:57.898381+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:43:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.935+0000 7f2b18ed1640 1 -- 172.21.15.45:0/1345286049 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2af40762d0 msgr2=0x7f2af4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.935+0000 7f2b18ed1640 1 --2- 172.21.15.45:0/1345286049 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2af40762d0 0x7f2af4078790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c006fd0 tx=0x7f2b0c008040 comp rx=0 tx=0).stop 2024-05-08T05:43:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.935+0000 7f2b18ed1640 1 -- 172.21.15.45:0/1345286049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b1415e0f0 msgr2=0x7f2b14161fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.935+0000 7f2b18ed1640 1 --2- 172.21.15.45:0/1345286049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b1415e0f0 0x7f2b14161fa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400c780 tx=0x7f2b0400cc40 comp rx=0 tx=0).stop 2024-05-08T05:43:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.935+0000 7f2b18ed1640 1 -- 172.21.15.45:0/1345286049 shutdown_connections 2024-05-08T05:43:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.935+0000 7f2b18ed1640 1 --2- 172.21.15.45:0/1345286049 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2af40762d0 0x7f2af4078790 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.935+0000 7f2b18ed1640 1 --2- 172.21.15.45:0/1345286049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b1415e0f0 0x7f2b14161fa0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.935+0000 7f2b18ed1640 1 --2- 172.21.15.45:0/1345286049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b1415e680 0x7f2b141e9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.935+0000 7f2b18ed1640 1 --2- 172.21.15.45:0/1345286049 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b140d1eb0 0x7f2b140cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.935+0000 7f2b18ed1640 1 -- 172.21.15.45:0/1345286049 >> 172.21.15.45:0/1345286049 conn(0x7f2b1408eac0 msgr2=0x7f2b1406d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.935+0000 7f2b18ed1640 1 -- 172.21.15.45:0/1345286049 shutdown_connections 2024-05-08T05:43:44.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:44.935+0000 7f2b18ed1640 1 -- 172.21.15.45:0/1345286049 wait complete. 2024-05-08T05:43:44.947 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T05:43:44.947 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:43:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.051+0000 7f6d03124640 1 Processor -- start 2024-05-08T05:43:45.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.051+0000 7f6d03124640 1 -- start start 2024-05-08T05:43:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.052+0000 7f6d03124640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6cfc06e0f0 0x7f6cfc15c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.052+0000 7f6d03124640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6cfc072840 0x7f6cfc0d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.052+0000 7f6d03124640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cfc06e4d0 0x7f6cfc0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.052+0000 7f6d03124640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f6cfc0d14b0 con 0x7f6cfc06e0f0 2024-05-08T05:43:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.052+0000 7f6d03124640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6cfc15c920 con 0x7f6cfc072840 2024-05-08T05:43:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.052+0000 7f6d03124640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6cfc15cc00 con 0x7f6cfc06e4d0 2024-05-08T05:43:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.052+0000 7f6d00ec0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6cfc06e0f0 0x7f6cfc15c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.052+0000 7f6d00ec0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6cfc06e0f0 0x7f6cfc15c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58884/0 (socket says 172.21.15.45:58884) 2024-05-08T05:43:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.052+0000 7f6d00ec0640 1 -- 172.21.15.45:0/233092156 learned_addr learned my addr 172.21.15.45:0/233092156 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.052+0000 7f6d016c1640 1 --2- 172.21.15.45:0/233092156 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cfc06e4d0 0x7f6cfc0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.052+0000 7f6cfbfff640 1 --2- 172.21.15.45:0/233092156 >> v2:172.21.15.63:3300/0 conn(0x7f6cfc072840 0x7f6cfc0d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.052+0000 7f6d00ec0640 1 -- 172.21.15.45:0/233092156 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cfc06e4d0 msgr2=0x7f6cfc0b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:45.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.052+0000 7f6d00ec0640 1 --2- 172.21.15.45:0/233092156 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cfc06e4d0 0x7f6cfc0b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.052+0000 7f6d00ec0640 1 -- 172.21.15.45:0/233092156 >> v2:172.21.15.63:3300/0 conn(0x7f6cfc072840 msgr2=0x7f6cfc0d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.052+0000 7f6d00ec0640 1 --2- 172.21.15.45:0/233092156 >> v2:172.21.15.63:3300/0 conn(0x7f6cfc072840 0x7f6cfc0d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.052+0000 7f6d00ec0640 1 -- 172.21.15.45:0/233092156 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cfc15cee0 con 0x7f6cfc06e0f0 2024-05-08T05:43:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.052+0000 7f6cfbfff640 1 --2- 172.21.15.45:0/233092156 >> v2:172.21.15.63:3300/0 conn(0x7f6cfc072840 0x7f6cfc0d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:45.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.052+0000 7f6d016c1640 1 --2- 172.21.15.45:0/233092156 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cfc06e4d0 0x7f6cfc0b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:45.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.053+0000 7f6d00ec0640 1 --2- 172.21.15.45:0/233092156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc06e0f0 0x7f6cfc15c340 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0727d0 tx=0x7f6cf002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1870e8d4dba4b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:45.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.053+0000 7f6cfb7fe640 1 -- 172.21.15.45:0/233092156 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0037070 con 0x7f6cfc06e0f0 2024-05-08T05:43:45.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.053+0000 7f6cfb7fe640 1 -- 172.21.15.45:0/233092156 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf0032440 con 0x7f6cfc06e0f0 2024-05-08T05:43:45.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.053+0000 7f6cfb7fe640 1 -- 172.21.15.45:0/233092156 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0032810 con 0x7f6cfc06e0f0 2024-05-08T05:43:45.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.054+0000 7f6d03124640 1 -- 172.21.15.45:0/233092156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc06e0f0 msgr2=0x7f6cfc15c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.054+0000 7f6d03124640 1 --2- 172.21.15.45:0/233092156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc06e0f0 0x7f6cfc15c340 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0727d0 tx=0x7f6cf002a1e0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.054+0000 7f6d03124640 1 -- 172.21.15.45:0/233092156 shutdown_connections 2024-05-08T05:43:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.054+0000 7f6d03124640 1 --2- 172.21.15.45:0/233092156 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cfc06e4d0 0x7f6cfc0b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.054+0000 7f6d03124640 1 --2- 172.21.15.45:0/233092156 >> v2:172.21.15.63:3300/0 conn(0x7f6cfc072840 0x7f6cfc0d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.054+0000 7f6d03124640 1 --2- 172.21.15.45:0/233092156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc06e0f0 0x7f6cfc15c340 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.054+0000 7f6d03124640 1 -- 172.21.15.45:0/233092156 >> 172.21.15.45:0/233092156 conn(0x7f6cfc06c3a0 msgr2=0x7f6cfc06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.054+0000 7f6d03124640 1 -- 172.21.15.45:0/233092156 shutdown_connections 2024-05-08T05:43:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.054+0000 7f6d03124640 1 -- 172.21.15.45:0/233092156 wait complete. 2024-05-08T05:43:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.055+0000 7f6d03124640 1 Processor -- start 2024-05-08T05:43:45.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.055+0000 7f6d03124640 1 -- start start 2024-05-08T05:43:45.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.056+0000 7f6d03124640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc06e0f0 0x7f6cfc1f2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.056+0000 7f6d03124640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cfc06e4d0 0x7f6cfc1efdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.056+0000 7f6d03124640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cfc072840 0x7f6cfc1574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.056+0000 7f6d03124640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc0ae650 con 0x7f6cfc06e0f0 2024-05-08T05:43:45.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.056+0000 7f6d03124640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6cfc06e980 con 0x7f6cfc072840 2024-05-08T05:43:45.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.056+0000 7f6d03124640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6cfc0ae7d0 con 0x7f6cfc06e4d0 2024-05-08T05:43:45.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.056+0000 7f6d00ec0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc06e0f0 0x7f6cfc1f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.056+0000 7f6d00ec0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc06e0f0 0x7f6cfc1f2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58890/0 (socket says 172.21.15.45:58890) 2024-05-08T05:43:45.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.056+0000 7f6d00ec0640 1 -- 172.21.15.45:0/2414425120 learned_addr learned my addr 172.21.15.45:0/2414425120 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:45.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.056+0000 7f6cfbfff640 1 --2- 172.21.15.45:0/2414425120 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cfc06e4d0 0x7f6cfc1efdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.056+0000 7f6d016c1640 1 --2- 172.21.15.45:0/2414425120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cfc072840 0x7f6cfc1574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.056+0000 7f6d00ec0640 1 -- 172.21.15.45:0/2414425120 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cfc06e4d0 msgr2=0x7f6cfc1efdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.056+0000 7f6d00ec0640 1 --2- 172.21.15.45:0/2414425120 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cfc06e4d0 0x7f6cfc1efdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.056+0000 7f6d00ec0640 1 -- 172.21.15.45:0/2414425120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cfc072840 msgr2=0x7f6cfc1574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.056+0000 7f6d00ec0640 1 --2- 172.21.15.45:0/2414425120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cfc072840 0x7f6cfc1574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.056+0000 7f6d00ec0640 1 -- 172.21.15.45:0/2414425120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cfc1f2fd0 con 0x7f6cfc06e0f0 2024-05-08T05:43:45.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.057+0000 7f6d00ec0640 1 --2- 172.21.15.45:0/2414425120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc06e0f0 0x7f6cfc1f2960 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6cf002a220 tx=0x7f6cf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:45.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.057+0000 7f6cf97fa640 1 -- 172.21.15.45:0/2414425120 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0041020 con 0x7f6cfc06e0f0 2024-05-08T05:43:45.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.057+0000 7f6cf97fa640 1 -- 172.21.15.45:0/2414425120 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf00306c0 con 0x7f6cfc06e0f0 2024-05-08T05:43:45.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.057+0000 7f6cf97fa640 1 -- 172.21.15.45:0/2414425120 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0037040 con 0x7f6cfc06e0f0 2024-05-08T05:43:45.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.057+0000 7f6d03124640 1 -- 172.21.15.45:0/2414425120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cfc1f32b0 con 0x7f6cfc06e0f0 2024-05-08T05:43:45.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.057+0000 7f6d03124640 1 -- 172.21.15.45:0/2414425120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cfc1f3870 con 0x7f6cfc06e0f0 2024-05-08T05:43:45.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.058+0000 7f6d03124640 1 -- 172.21.15.45:0/2414425120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cfc0693f0 con 0x7f6cfc06e0f0 2024-05-08T05:43:45.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.059+0000 7f6cf97fa640 1 -- 172.21.15.45:0/2414425120 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6cf0044020 con 0x7f6cfc06e0f0 2024-05-08T05:43:45.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.059+0000 7f6cf97fa640 1 --2- 172.21.15.45:0/2414425120 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6ce00762d0 0x7f6ce0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.059+0000 7f6cf97fa640 1 -- 172.21.15.45:0/2414425120 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6159+0+0 (secure 0 0 0) 0x7f6cf00b7ed0 con 0x7f6cfc06e0f0 2024-05-08T05:43:45.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.059+0000 7f6cfbfff640 1 --2- 172.21.15.45:0/2414425120 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6ce00762d0 0x7f6ce0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.060+0000 7f6cfbfff640 1 --2- 172.21.15.45:0/2414425120 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6ce00762d0 0x7f6ce0078790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8006fd0 tx=0x7f6ce8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:45.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.063+0000 7f6cf97fa640 1 -- 172.21.15.45:0/2414425120 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6cf004b030 con 0x7f6cfc06e0f0 2024-05-08T05:43:45.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.294+0000 7f6d03124640 1 -- 172.21.15.45:0/2414425120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cfc0afce0 con 0x7f6cfc06e0f0 2024-05-08T05:43:45.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.295+0000 7f6cf97fa640 1 -- 172.21.15.45:0/2414425120 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5181 (secure 0 0 0) 0x7f6cf0081700 con 0x7f6cfc06e0f0 2024-05-08T05:43:45.295 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:45.296 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4262647207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4262647207},{"type":"v1","addr":"172.21.15.63:6837","nonce":4262647207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5277,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2872370835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2872370835},{"type":"v1","addr":"172.21.15.45:6834","nonce":2872370835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1909488716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1909488716},{"type":"v1","addr":"172.21.15.45:6835","nonce":1909488716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:42:58.943500+0000","modified":"2024-05-08T05:43:01.967189+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6836/4205418057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":4205418057},{"type":"v1","addr":"172.21.15.63:6836","nonce":4205418057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5292,"qdb_cluster":[5292]},"id":3}]} 2024-05-08T05:43:45.296 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-05-08T05:43:45.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.297+0000 7f6d03124640 1 -- 172.21.15.45:0/2414425120 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6ce00762d0 msgr2=0x7f6ce0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.297+0000 7f6d03124640 1 --2- 172.21.15.45:0/2414425120 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6ce00762d0 0x7f6ce0078790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8006fd0 tx=0x7f6ce8008040 comp rx=0 tx=0).stop 2024-05-08T05:43:45.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.297+0000 7f6d03124640 1 -- 172.21.15.45:0/2414425120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc06e0f0 msgr2=0x7f6cfc1f2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.297+0000 7f6d03124640 1 --2- 172.21.15.45:0/2414425120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc06e0f0 0x7f6cfc1f2960 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6cf002a220 tx=0x7f6cf00046c0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.297+0000 7f6d03124640 1 -- 172.21.15.45:0/2414425120 shutdown_connections 2024-05-08T05:43:45.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.297+0000 7f6d03124640 1 --2- 172.21.15.45:0/2414425120 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6ce00762d0 0x7f6ce0078790 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.297+0000 7f6d03124640 1 --2- 172.21.15.45:0/2414425120 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6cfc06e4d0 0x7f6cfc1efdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.297+0000 7f6d03124640 1 --2- 172.21.15.45:0/2414425120 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6cfc072840 0x7f6cfc1574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.297+0000 7f6d03124640 1 --2- 172.21.15.45:0/2414425120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6cfc06e0f0 0x7f6cfc1f2960 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.297+0000 7f6d03124640 1 -- 172.21.15.45:0/2414425120 >> 172.21.15.45:0/2414425120 conn(0x7f6cfc06c3a0 msgr2=0x7f6cfc0b5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:45.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.297+0000 7f6d03124640 1 -- 172.21.15.45:0/2414425120 shutdown_connections 2024-05-08T05:43:45.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.297+0000 7f6d03124640 1 -- 172.21.15.45:0/2414425120 wait complete. 2024-05-08T05:43:45.308 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:43:45.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.412+0000 7fd576314640 1 Processor -- start 2024-05-08T05:43:45.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.412+0000 7fd576314640 1 -- start start 2024-05-08T05:43:45.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.413+0000 7fd576314640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd5701638e0 0x7fd5700b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.413+0000 7fd576314640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd57006c3a0 0x7fd57006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.413+0000 7fd576314640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5700b14e0 con 0x7fd57015f0c0 2024-05-08T05:43:45.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.413+0000 7fd576314640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fd570163450 con 0x7fd5701638e0 2024-05-08T05:43:45.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.413+0000 7fd576314640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd57006cd20 con 0x7fd57006c3a0 2024-05-08T05:43:45.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.413+0000 7fd56f7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd5701638e0 0x7fd5700b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.413+0000 7fd56f7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd5701638e0 0x7fd5700b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58902/0 (socket says 172.21.15.45:58902) 2024-05-08T05:43:45.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.413+0000 7fd56f7fe640 1 -- 172.21.15.45:0/4061708724 learned_addr learned my addr 172.21.15.45:0/4061708724 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:45.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.413+0000 7fd5748b1640 1 --2- 172.21.15.45:0/4061708724 >> v2:172.21.15.63:3300/0 conn(0x7fd57006c3a0 0x7fd57006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.414+0000 7fd56f7fe640 1 -- 172.21.15.45:0/4061708724 >> v2:172.21.15.63:3300/0 conn(0x7fd57006c3a0 msgr2=0x7fd57006c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:45.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.414+0000 7fd56f7fe640 1 --2- 172.21.15.45:0/4061708724 >> v2:172.21.15.63:3300/0 conn(0x7fd57006c3a0 0x7fd57006c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.414+0000 7fd56f7fe640 1 -- 172.21.15.45:0/4061708724 >> v1:172.21.15.63:6789/0 conn(0x7fd57015f0c0 legacy=0x7fd5700b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.414+0000 7fd56f7fe640 1 -- 172.21.15.45:0/4061708724 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5701604b0 con 0x7fd5701638e0 2024-05-08T05:43:45.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.414+0000 7fd56f7fe640 1 --2- 172.21.15.45:0/4061708724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5701638e0 0x7fd5700b6f40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd570074670 tx=0x7fd56402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d03eb75e4f3ef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:45.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.414+0000 7fd56effd640 1 -- 172.21.15.45:0/4061708724 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564037070 con 0x7fd5701638e0 2024-05-08T05:43:45.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.415+0000 7fd56effd640 1 -- 172.21.15.45:0/4061708724 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd564032440 con 0x7fd5701638e0 2024-05-08T05:43:45.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.415+0000 7fd56effd640 1 -- 172.21.15.45:0/4061708724 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564032810 con 0x7fd5701638e0 2024-05-08T05:43:45.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.415+0000 7fd576314640 1 -- 172.21.15.45:0/4061708724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5701638e0 msgr2=0x7fd5700b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.415+0000 7fd576314640 1 --2- 172.21.15.45:0/4061708724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5701638e0 0x7fd5700b6f40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd570074670 tx=0x7fd56402a270 comp rx=0 tx=0).stop 2024-05-08T05:43:45.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.415+0000 7fd576314640 1 -- 172.21.15.45:0/4061708724 shutdown_connections 2024-05-08T05:43:45.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.415+0000 7fd576314640 1 --2- 172.21.15.45:0/4061708724 >> v2:172.21.15.63:3300/0 conn(0x7fd57006c3a0 0x7fd57006c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.415+0000 7fd576314640 1 --2- 172.21.15.45:0/4061708724 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5701638e0 0x7fd5700b6f40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.415+0000 7fd576314640 1 -- 172.21.15.45:0/4061708724 >> 172.21.15.45:0/4061708724 conn(0x7fd5700b3b10 msgr2=0x7fd5700b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:45.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.415+0000 7fd576314640 1 -- 172.21.15.45:0/4061708724 shutdown_connections 2024-05-08T05:43:45.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.415+0000 7fd576314640 1 -- 172.21.15.45:0/4061708724 wait complete. 2024-05-08T05:43:45.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.416+0000 7fd576314640 1 Processor -- start 2024-05-08T05:43:45.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.417+0000 7fd576314640 1 -- start start 2024-05-08T05:43:45.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.417+0000 7fd576314640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd57006c3a0 0x7fd5700c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.417+0000 7fd576314640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd57015f0c0 0x7fd5700d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.417+0000 7fd576314640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5701638e0 0x7fd570161790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.417+0000 7fd576314640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd5700b17c0 con 0x7fd57015f0c0 2024-05-08T05:43:45.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.417+0000 7fd576314640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5701630e0 con 0x7fd57006c3a0 2024-05-08T05:43:45.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.417+0000 7fd576314640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd57015f4a0 con 0x7fd5701638e0 2024-05-08T05:43:45.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.417+0000 7fd56f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd57015f0c0 0x7fd5700d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.417+0000 7fd56f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd57015f0c0 0x7fd5700d1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58912/0 (socket says 172.21.15.45:58912) 2024-05-08T05:43:45.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.417+0000 7fd56f7fe640 1 -- 172.21.15.45:0/4232072091 learned_addr learned my addr 172.21.15.45:0/4232072091 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:45.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.418+0000 7fd5748b1640 1 --2- 172.21.15.45:0/4232072091 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5701638e0 0x7fd570161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.418+0000 7fd56ffff640 1 --2- 172.21.15.45:0/4232072091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd57006c3a0 0x7fd5700c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.418+0000 7fd56f7fe640 1 -- 172.21.15.45:0/4232072091 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5701638e0 msgr2=0x7fd570161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.418+0000 7fd56f7fe640 1 --2- 172.21.15.45:0/4232072091 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5701638e0 0x7fd570161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.418+0000 7fd56f7fe640 1 -- 172.21.15.45:0/4232072091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd57006c3a0 msgr2=0x7fd5700c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.418+0000 7fd56f7fe640 1 --2- 172.21.15.45:0/4232072091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd57006c3a0 0x7fd5700c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.418+0000 7fd56f7fe640 1 -- 172.21.15.45:0/4232072091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5701f4e10 con 0x7fd57015f0c0 2024-05-08T05:43:45.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.418+0000 7fd5748b1640 1 --2- 172.21.15.45:0/4232072091 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5701638e0 0x7fd570161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:45.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.418+0000 7fd56ffff640 1 --2- 172.21.15.45:0/4232072091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd57006c3a0 0x7fd5700c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:45.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.418+0000 7fd56f7fe640 1 --2- 172.21.15.45:0/4232072091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd57015f0c0 0x7fd5700d1ff0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd56402a2b0 tx=0x7fd5640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:45.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.418+0000 7fd56cff9640 1 -- 172.21.15.45:0/4232072091 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564037040 con 0x7fd57015f0c0 2024-05-08T05:43:45.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.418+0000 7fd56cff9640 1 -- 172.21.15.45:0/4232072091 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd564042070 con 0x7fd57015f0c0 2024-05-08T05:43:45.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.418+0000 7fd576314640 1 -- 172.21.15.45:0/4232072091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5701f5090 con 0x7fd57015f0c0 2024-05-08T05:43:45.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.418+0000 7fd56cff9640 1 -- 172.21.15.45:0/4232072091 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564030880 con 0x7fd57015f0c0 2024-05-08T05:43:45.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.418+0000 7fd576314640 1 -- 172.21.15.45:0/4232072091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5701f5930 con 0x7fd57015f0c0 2024-05-08T05:43:45.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.419+0000 7fd576314640 1 -- 172.21.15.45:0/4232072091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd57006a8f0 con 0x7fd57015f0c0 2024-05-08T05:43:45.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.420+0000 7fd56cff9640 1 -- 172.21.15.45:0/4232072091 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd564030a20 con 0x7fd57015f0c0 2024-05-08T05:43:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.420+0000 7fd56cff9640 1 --2- 172.21.15.45:0/4232072091 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd55c0762d0 0x7fd55c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.420+0000 7fd56cff9640 1 -- 172.21.15.45:0/4232072091 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6159+0+0 (secure 0 0 0) 0x7fd5640b8190 con 0x7fd57015f0c0 2024-05-08T05:43:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.421+0000 7fd56ffff640 1 --2- 172.21.15.45:0/4232072091 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd55c0762d0 0x7fd55c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.421+0000 7fd56ffff640 1 --2- 172.21.15.45:0/4232072091 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd55c0762d0 0x7fd55c078790 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd560001740 tx=0x7fd560009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:45.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.425+0000 7fd56cff9640 1 -- 172.21.15.45:0/4232072091 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd5701f4e10 con 0x7fd57015f0c0 2024-05-08T05:43:45.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.658+0000 7fd576314640 1 -- 172.21.15.45:0/4232072091 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5700693f0 con 0x7fd57015f0c0 2024-05-08T05:43:45.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.659+0000 7fd56cff9640 1 -- 172.21.15.45:0/4232072091 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5181 (secure 0 0 0) 0x7fd564081890 con 0x7fd57015f0c0 2024-05-08T05:43:45.659 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:45.659 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4262647207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4262647207},{"type":"v1","addr":"172.21.15.63:6837","nonce":4262647207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5277,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2872370835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2872370835},{"type":"v1","addr":"172.21.15.45:6834","nonce":2872370835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1909488716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1909488716},{"type":"v1","addr":"172.21.15.45:6835","nonce":1909488716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:42:58.943500+0000","modified":"2024-05-08T05:43:01.967189+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5292":{"gid":5292,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6836/4205418057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":4205418057},{"type":"v1","addr":"172.21.15.63:6836","nonce":4205418057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5292,"qdb_cluster":[5292]},"id":3}]} 2024-05-08T05:43:45.660 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-05-08T05:43:45.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.661+0000 7fd576314640 1 -- 172.21.15.45:0/4232072091 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd55c0762d0 msgr2=0x7fd55c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.661+0000 7fd576314640 1 --2- 172.21.15.45:0/4232072091 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd55c0762d0 0x7fd55c078790 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd560001740 tx=0x7fd560009040 comp rx=0 tx=0).stop 2024-05-08T05:43:45.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.661+0000 7fd576314640 1 -- 172.21.15.45:0/4232072091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd57015f0c0 msgr2=0x7fd5700d1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.661+0000 7fd576314640 1 --2- 172.21.15.45:0/4232072091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd57015f0c0 0x7fd5700d1ff0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd56402a2b0 tx=0x7fd5640046c0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.661+0000 7fd576314640 1 -- 172.21.15.45:0/4232072091 shutdown_connections 2024-05-08T05:43:45.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.661+0000 7fd576314640 1 --2- 172.21.15.45:0/4232072091 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd55c0762d0 0x7fd55c078790 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.661+0000 7fd576314640 1 --2- 172.21.15.45:0/4232072091 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd5701638e0 0x7fd570161790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.661+0000 7fd576314640 1 --2- 172.21.15.45:0/4232072091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd57015f0c0 0x7fd5700d1ff0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.661+0000 7fd576314640 1 --2- 172.21.15.45:0/4232072091 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd57006c3a0 0x7fd5700c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.661+0000 7fd576314640 1 -- 172.21.15.45:0/4232072091 >> 172.21.15.45:0/4232072091 conn(0x7fd5700b3b10 msgr2=0x7fd570160d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:45.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.661+0000 7fd576314640 1 -- 172.21.15.45:0/4232072091 shutdown_connections 2024-05-08T05:43:45.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.661+0000 7fd576314640 1 -- 172.21.15.45:0/4232072091 wait complete. 2024-05-08T05:43:45.672 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:43:45.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.776+0000 7f6613c54640 1 Processor -- start 2024-05-08T05:43:45.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.776+0000 7f6613c54640 1 -- start start 2024-05-08T05:43:45.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f6613c54640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f660c15bc80 0x7f660c06e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f6613c54640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f660c0b4590 con 0x7f660c1524f0 2024-05-08T05:43:45.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f6613c54640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f660c0b4870 con 0x7f660c06e8d0 2024-05-08T05:43:45.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f6613c54640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f660c0b51a0 con 0x7f660c15bc80 2024-05-08T05:43:45.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66119f0640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f660c1524f0 0x7f660c0b3c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58714/0 (socket says 172.21.15.45:58714) 2024-05-08T05:43:45.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66119f0640 1 -- 172.21.15.45:0/797496204 learned_addr learned my addr 172.21.15.45:0/797496204 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:45.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66111ef640 1 --2- 172.21.15.45:0/797496204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f660c15bc80 0x7f660c06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66109ee640 1 -- 172.21.15.45:0/797496204 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1014583378 0 0) 0x7f660c0b4590 con 0x7f660c1524f0 2024-05-08T05:43:45.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66109ee640 1 -- 172.21.15.45:0/797496204 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65f8003520 con 0x7f660c1524f0 2024-05-08T05:43:45.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66109ee640 1 -- 172.21.15.45:0/797496204 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 985731980 0 0) 0x7f660c0b4870 con 0x7f660c06e8d0 2024-05-08T05:43:45.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66109ee640 1 -- 172.21.15.45:0/797496204 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f660c0b4590 con 0x7f660c06e8d0 2024-05-08T05:43:45.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66109ee640 1 -- 172.21.15.45:0/797496204 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 316136108 0 0) 0x7f65f8003520 con 0x7f660c1524f0 2024-05-08T05:43:45.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66109ee640 1 -- 172.21.15.45:0/797496204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f660c15bc80 msgr2=0x7f660c06e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:45.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66109ee640 1 --2- 172.21.15.45:0/797496204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f660c15bc80 0x7f660c06e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66109ee640 1 -- 172.21.15.45:0/797496204 >> v1:172.21.15.63:6789/0 conn(0x7f660c06e8d0 legacy=0x7f660c073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66109ee640 1 -- 172.21.15.45:0/797496204 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660c0b4b50 con 0x7f660c1524f0 2024-05-08T05:43:45.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66109ee640 1 -- 172.21.15.45:0/797496204 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f66080016f0 con 0x7f660c1524f0 2024-05-08T05:43:45.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66109ee640 1 -- 172.21.15.45:0/797496204 >> v1:172.21.15.45:6789/0 conn(0x7f660c1524f0 legacy=0x7f660c0b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66109ee640 1 --2- 172.21.15.45:0/797496204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65f8003c10 0x7f65f8004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66109ee640 1 --2- 172.21.15.45:0/797496204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65f8004800 0x7f65f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66109ee640 1 --2- 172.21.15.45:0/797496204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65f80051c0 0x7f65f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66109ee640 1 -- 172.21.15.45:0/797496204 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f660c0b51a0 con 0x7f65f8003c10 2024-05-08T05:43:45.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66109ee640 1 -- 172.21.15.45:0/797496204 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65f800c090 con 0x7f65f80051c0 2024-05-08T05:43:45.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66109ee640 1 -- 172.21.15.45:0/797496204 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f65f800c290 con 0x7f65f8004800 2024-05-08T05:43:45.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.777+0000 7f66111ef640 1 --2- 172.21.15.45:0/797496204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f660c15bc80 0x7f660c06e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:43:45.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.778+0000 7f66119f0640 1 --2- 172.21.15.45:0/797496204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65f8003c10 0x7f65f8004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.778+0000 7f66121f1640 1 --2- 172.21.15.45:0/797496204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65f80051c0 0x7f65f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.778+0000 7f66111ef640 1 --2- 172.21.15.45:0/797496204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65f8004800 0x7f65f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.778+0000 7f66119f0640 1 -- 172.21.15.45:0/797496204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65f8004800 msgr2=0x7f65f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:45.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.778+0000 7f66119f0640 1 --2- 172.21.15.45:0/797496204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65f8004800 0x7f65f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.778+0000 7f66119f0640 1 -- 172.21.15.45:0/797496204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65f80051c0 msgr2=0x7f65f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:45.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.778+0000 7f66119f0640 1 --2- 172.21.15.45:0/797496204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65f80051c0 0x7f65f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.778+0000 7f66119f0640 1 -- 172.21.15.45:0/797496204 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f8003520 con 0x7f65f8003c10 2024-05-08T05:43:45.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.778+0000 7f66111ef640 1 --2- 172.21.15.45:0/797496204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65f8004800 0x7f65f8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:45.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.779+0000 7f66119f0640 1 --2- 172.21.15.45:0/797496204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65f8003c10 0x7f65f8004010 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f660800c190 tx=0x7f65f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810594382677d013 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:45.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.779+0000 7f66109ee640 1 -- 172.21.15.45:0/797496204 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608007cc0 con 0x7f65f8003c10 2024-05-08T05:43:45.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.779+0000 7f66109ee640 1 -- 172.21.15.45:0/797496204 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6608005ad0 con 0x7f65f8003c10 2024-05-08T05:43:45.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.779+0000 7f66109ee640 1 -- 172.21.15.45:0/797496204 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608007070 con 0x7f65f8003c10 2024-05-08T05:43:45.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.779+0000 7f6613c54640 1 -- 172.21.15.45:0/797496204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65f8003c10 msgr2=0x7f65f8004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.779+0000 7f6613c54640 1 --2- 172.21.15.45:0/797496204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65f8003c10 0x7f65f8004010 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f660800c190 tx=0x7f65f8005830 comp rx=0 tx=0).stop 2024-05-08T05:43:45.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.779+0000 7f66121f1640 1 -- 172.21.15.45:0/797496204 reap_dead start 2024-05-08T05:43:45.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.779+0000 7f6613c54640 1 -- 172.21.15.45:0/797496204 shutdown_connections 2024-05-08T05:43:45.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.779+0000 7f6613c54640 1 -- 172.21.15.45:0/797496204 >> 172.21.15.45:0/797496204 conn(0x7f660c06c3a0 msgr2=0x7f660c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:45.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.779+0000 7f6613c54640 1 -- 172.21.15.45:0/797496204 shutdown_connections 2024-05-08T05:43:45.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.779+0000 7f6613c54640 1 -- 172.21.15.45:0/797496204 wait complete. 2024-05-08T05:43:45.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.781+0000 7f6613c54640 1 Processor -- start 2024-05-08T05:43:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.781+0000 7f6613c54640 1 -- start start 2024-05-08T05:43:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.781+0000 7f6613c54640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65f8003c10 0x7f660c14bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.781+0000 7f6613c54640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f660c148910 0x7f660c072440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.781+0000 7f6613c54640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f660c072980 0x7f660c1f8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.781+0000 7f6613c54640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f660c0b4330 con 0x7f660c148910 2024-05-08T05:43:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.781+0000 7f6613c54640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f660c073450 con 0x7f65f8003c10 2024-05-08T05:43:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.781+0000 7f6613c54640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f660c1528d0 con 0x7f660c072980 2024-05-08T05:43:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.781+0000 7f66111ef640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f660c148910 0x7f660c072440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.782+0000 7f66111ef640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f660c148910 0x7f660c072440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58932/0 (socket says 172.21.15.45:58932) 2024-05-08T05:43:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.782+0000 7f66111ef640 1 -- 172.21.15.45:0/895781204 learned_addr learned my addr 172.21.15.45:0/895781204 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.782+0000 7f66119f0640 1 --2- 172.21.15.45:0/895781204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65f8003c10 0x7f660c14bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.782+0000 7f66121f1640 1 --2- 172.21.15.45:0/895781204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f660c072980 0x7f660c1f8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.782+0000 7f66111ef640 1 -- 172.21.15.45:0/895781204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f660c072980 msgr2=0x7f660c1f8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.782+0000 7f66111ef640 1 --2- 172.21.15.45:0/895781204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f660c072980 0x7f660c1f8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.782+0000 7f66111ef640 1 -- 172.21.15.45:0/895781204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65f8003c10 msgr2=0x7f660c14bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.782+0000 7f66111ef640 1 --2- 172.21.15.45:0/895781204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65f8003c10 0x7f660c14bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.782+0000 7f66111ef640 1 -- 172.21.15.45:0/895781204 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660c0af440 con 0x7f660c148910 2024-05-08T05:43:45.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.782+0000 7f66111ef640 1 --2- 172.21.15.45:0/895781204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f660c148910 0x7f660c072440 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0118e0 tx=0x7f65fc011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:45.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.782+0000 7f66027fc640 1 -- 172.21.15.45:0/895781204 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc00fbd0 con 0x7f660c148910 2024-05-08T05:43:45.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.782+0000 7f66027fc640 1 -- 172.21.15.45:0/895781204 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65fc009e50 con 0x7f660c148910 2024-05-08T05:43:45.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.782+0000 7f6613c54640 1 -- 172.21.15.45:0/895781204 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f660c0af720 con 0x7f660c148910 2024-05-08T05:43:45.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.782+0000 7f66027fc640 1 -- 172.21.15.45:0/895781204 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc015460 con 0x7f660c148910 2024-05-08T05:43:45.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.782+0000 7f6613c54640 1 -- 172.21.15.45:0/895781204 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f660c0afce0 con 0x7f660c148910 2024-05-08T05:43:45.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.783+0000 7f6613c54640 1 -- 172.21.15.45:0/895781204 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f660c06ff60 con 0x7f660c148910 2024-05-08T05:43:45.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.784+0000 7f66027fc640 1 -- 172.21.15.45:0/895781204 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f65fc015600 con 0x7f660c148910 2024-05-08T05:43:45.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.784+0000 7f66027fc640 1 --2- 172.21.15.45:0/895781204 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f65f00762d0 0x7f65f0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:45.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.785+0000 7f66027fc640 1 -- 172.21.15.45:0/895781204 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6159+0+0 (secure 0 0 0) 0x7f65fc098160 con 0x7f660c148910 2024-05-08T05:43:45.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.785+0000 7f66119f0640 1 --2- 172.21.15.45:0/895781204 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f65f00762d0 0x7f65f0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:45.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.785+0000 7f66119f0640 1 --2- 172.21.15.45:0/895781204 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f65f00762d0 0x7f65f0078790 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f660800c190 tx=0x7f6608034040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:45.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.789+0000 7f66027fc640 1 -- 172.21.15.45:0/895781204 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f660c06ff60 con 0x7f660c148910 2024-05-08T05:43:45.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.982+0000 7f6613c54640 1 -- 172.21.15.45:0/895781204 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f660c0693f0 con 0x7f660c148910 2024-05-08T05:43:45.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.983+0000 7f66027fc640 1 -- 172.21.15.45:0/895781204 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16990 (secure 0 0 0) 0x7f65fc019030 con 0x7f660c148910 2024-05-08T05:43:45.983 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:45.984 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":49,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:43:14.934284+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:42:56.614794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-08T05:42:57.898381+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1a","mappings":[{"from":0,"to":1}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:43:45.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.985+0000 7f6613c54640 1 -- 172.21.15.45:0/895781204 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f65f00762d0 msgr2=0x7f65f0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.985+0000 7f6613c54640 1 --2- 172.21.15.45:0/895781204 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f65f00762d0 0x7f65f0078790 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f660800c190 tx=0x7f6608034040 comp rx=0 tx=0).stop 2024-05-08T05:43:45.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.985+0000 7f6613c54640 1 -- 172.21.15.45:0/895781204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f660c148910 msgr2=0x7f660c072440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:45.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.985+0000 7f6613c54640 1 --2- 172.21.15.45:0/895781204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f660c148910 0x7f660c072440 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0118e0 tx=0x7f65fc011da0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.985+0000 7f6613c54640 1 -- 172.21.15.45:0/895781204 shutdown_connections 2024-05-08T05:43:45.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.985+0000 7f6613c54640 1 --2- 172.21.15.45:0/895781204 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f65f00762d0 0x7f65f0078790 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.985+0000 7f6613c54640 1 --2- 172.21.15.45:0/895781204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f660c072980 0x7f660c1f8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.985+0000 7f6613c54640 1 --2- 172.21.15.45:0/895781204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f660c148910 0x7f660c072440 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.985+0000 7f6613c54640 1 --2- 172.21.15.45:0/895781204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65f8003c10 0x7f660c14bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:45.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.985+0000 7f6613c54640 1 -- 172.21.15.45:0/895781204 >> 172.21.15.45:0/895781204 conn(0x7f660c06c3a0 msgr2=0x7f660c15e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:45.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.985+0000 7f6613c54640 1 -- 172.21.15.45:0/895781204 shutdown_connections 2024-05-08T05:43:45.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:45.985+0000 7f6613c54640 1 -- 172.21.15.45:0/895781204 wait complete. 2024-05-08T05:43:45.997 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T05:43:46.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.101+0000 7fc4ce8da640 1 Processor -- start 2024-05-08T05:43:46.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.101+0000 7fc4ce8da640 1 -- start start 2024-05-08T05:43:46.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.102+0000 7fc4ce8da640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4c81629a0 0x7fc4c80d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:46.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.102+0000 7fc4ce8da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4c80d2dc0 0x7fc4c80d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:46.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.102+0000 7fc4ce8da640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4c80d3330 con 0x7fc4c80c8ed0 2024-05-08T05:43:46.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.102+0000 7fc4ce8da640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc4c80d2170 con 0x7fc4c81629a0 2024-05-08T05:43:46.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.102+0000 7fc4ce8da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c80d2450 con 0x7fc4c80d2dc0 2024-05-08T05:43:46.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.102+0000 7fc4c7fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4c81629a0 0x7fc4c80d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:46.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.102+0000 7fc4cce77640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc4c80c8ed0 0x7fc4c80c9360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58728/0 (socket says 172.21.15.45:58728) 2024-05-08T05:43:46.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.102+0000 7fc4cce77640 1 -- 172.21.15.45:0/1345940144 learned_addr learned my addr 172.21.15.45:0/1345940144 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:46.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.102+0000 7fc4c7fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc4c81629a0 0x7fc4c80d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58938/0 (socket says 172.21.15.45:58938) 2024-05-08T05:43:46.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.103+0000 7fc4bffff640 1 --2- 172.21.15.45:0/1345940144 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4c80d2dc0 0x7fc4c80d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:46.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.103+0000 7fc4c7fff640 1 -- 172.21.15.45:0/1345940144 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4c80d2dc0 msgr2=0x7fc4c80d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:46.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.103+0000 7fc4c7fff640 1 --2- 172.21.15.45:0/1345940144 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4c80d2dc0 0x7fc4c80d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:46.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.103+0000 7fc4c7fff640 1 -- 172.21.15.45:0/1345940144 >> v1:172.21.15.45:6789/0 conn(0x7fc4c80c8ed0 legacy=0x7fc4c80c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:46.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.103+0000 7fc4c7fff640 1 -- 172.21.15.45:0/1345940144 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c80d2730 con 0x7fc4c81629a0 2024-05-08T05:43:46.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.103+0000 7fc4c7fff640 1 --2- 172.21.15.45:0/1345940144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c81629a0 0x7fc4c80d35c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80097a0 tx=0x7fc4b8029f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77e1ff114d15eaa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:46.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.104+0000 7fc4c77fe640 1 -- 172.21.15.45:0/1345940144 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8037070 con 0x7fc4c81629a0 2024-05-08T05:43:46.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.104+0000 7fc4c77fe640 1 -- 172.21.15.45:0/1345940144 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b802abf0 con 0x7fc4c81629a0 2024-05-08T05:43:46.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.104+0000 7fc4c77fe640 1 -- 172.21.15.45:0/1345940144 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8032440 con 0x7fc4c81629a0 2024-05-08T05:43:46.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.104+0000 7fc4ce8da640 1 -- 172.21.15.45:0/1345940144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c81629a0 msgr2=0x7fc4c80d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:46.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.104+0000 7fc4ce8da640 1 --2- 172.21.15.45:0/1345940144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c81629a0 0x7fc4c80d35c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80097a0 tx=0x7fc4b8029f60 comp rx=0 tx=0).stop 2024-05-08T05:43:46.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.104+0000 7fc4ce8da640 1 -- 172.21.15.45:0/1345940144 shutdown_connections 2024-05-08T05:43:46.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.104+0000 7fc4ce8da640 1 --2- 172.21.15.45:0/1345940144 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4c80d2dc0 0x7fc4c80d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:46.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.104+0000 7fc4ce8da640 1 --2- 172.21.15.45:0/1345940144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c81629a0 0x7fc4c80d35c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:46.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.104+0000 7fc4ce8da640 1 -- 172.21.15.45:0/1345940144 >> 172.21.15.45:0/1345940144 conn(0x7fc4c80d4fe0 msgr2=0x7fc4c80d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:46.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.104+0000 7fc4ce8da640 1 -- 172.21.15.45:0/1345940144 shutdown_connections 2024-05-08T05:43:46.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.104+0000 7fc4ce8da640 1 -- 172.21.15.45:0/1345940144 wait complete. 2024-05-08T05:43:46.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.106+0000 7fc4ce8da640 1 Processor -- start 2024-05-08T05:43:46.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.106+0000 7fc4ce8da640 1 -- start start 2024-05-08T05:43:46.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.106+0000 7fc4ce8da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4c80c8ed0 0x7fc4c80a26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:46.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.106+0000 7fc4ce8da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c80d2dc0 0x7fc4c81541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:46.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.106+0000 7fc4ce8da640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4c81629a0 0x7fc4c80c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:46.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.106+0000 7fc4ce8da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c80cf440 con 0x7fc4c80d2dc0 2024-05-08T05:43:46.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.106+0000 7fc4ce8da640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c80cf2c0 con 0x7fc4c81629a0 2024-05-08T05:43:46.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.106+0000 7fc4ce8da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c80d0cb0 con 0x7fc4c80c8ed0 2024-05-08T05:43:46.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.106+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c80d2dc0 0x7fc4c81541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:46.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.107+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c80d2dc0 0x7fc4c81541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58948/0 (socket says 172.21.15.45:58948) 2024-05-08T05:43:46.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.107+0000 7fc4bffff640 1 -- 172.21.15.45:0/919253911 learned_addr learned my addr 172.21.15.45:0/919253911 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:46.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.107+0000 7fc4cce77640 1 --2- 172.21.15.45:0/919253911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4c81629a0 0x7fc4c80c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:46.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.107+0000 7fc4bffff640 1 -- 172.21.15.45:0/919253911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4c80c8ed0 msgr2=0x7fc4c80a26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:46.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.107+0000 7fc4c7fff640 1 --2- 172.21.15.45:0/919253911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4c80c8ed0 0x7fc4c80a26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:46.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.107+0000 7fc4bffff640 1 --2- 172.21.15.45:0/919253911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4c80c8ed0 0x7fc4c80a26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:46.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.107+0000 7fc4bffff640 1 -- 172.21.15.45:0/919253911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4c81629a0 msgr2=0x7fc4c80c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:46.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.107+0000 7fc4bffff640 1 --2- 172.21.15.45:0/919253911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4c81629a0 0x7fc4c80c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:46.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.107+0000 7fc4bffff640 1 -- 172.21.15.45:0/919253911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c80a2c10 con 0x7fc4c80d2dc0 2024-05-08T05:43:46.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.107+0000 7fc4cce77640 1 --2- 172.21.15.45:0/919253911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4c81629a0 0x7fc4c80c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:46.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.107+0000 7fc4c7fff640 1 --2- 172.21.15.45:0/919253911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4c80c8ed0 0x7fc4c80a26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:46.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.107+0000 7fc4bffff640 1 --2- 172.21.15.45:0/919253911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c80d2dc0 0x7fc4c81541d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000d800 tx=0x7fc4b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:46.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.107+0000 7fc4c57fa640 1 -- 172.21.15.45:0/919253911 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b000b6c0 con 0x7fc4c80d2dc0 2024-05-08T05:43:46.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.107+0000 7fc4c57fa640 1 -- 172.21.15.45:0/919253911 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b0019070 con 0x7fc4c80d2dc0 2024-05-08T05:43:46.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.107+0000 7fc4ce8da640 1 -- 172.21.15.45:0/919253911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c80a1910 con 0x7fc4c80d2dc0 2024-05-08T05:43:46.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.107+0000 7fc4c57fa640 1 -- 172.21.15.45:0/919253911 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b000ba80 con 0x7fc4c80d2dc0 2024-05-08T05:43:46.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.107+0000 7fc4ce8da640 1 -- 172.21.15.45:0/919253911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c80a1ed0 con 0x7fc4c80d2dc0 2024-05-08T05:43:46.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.108+0000 7fc4ce8da640 1 -- 172.21.15.45:0/919253911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c80693f0 con 0x7fc4c80d2dc0 2024-05-08T05:43:46.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.109+0000 7fc4c57fa640 1 -- 172.21.15.45:0/919253911 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc4b000bc20 con 0x7fc4c80d2dc0 2024-05-08T05:43:46.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.109+0000 7fc4c57fa640 1 --2- 172.21.15.45:0/919253911 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc4a80762d0 0x7fc4a8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:46.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.109+0000 7fc4c57fa640 1 -- 172.21.15.45:0/919253911 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6159+0+0 (secure 0 0 0) 0x7fc4b0097bc0 con 0x7fc4c80d2dc0 2024-05-08T05:43:46.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.110+0000 7fc4c7fff640 1 --2- 172.21.15.45:0/919253911 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc4a80762d0 0x7fc4a8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:46.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.110+0000 7fc4c7fff640 1 --2- 172.21.15.45:0/919253911 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc4a80762d0 0x7fc4a8078790 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8034000 tx=0x7fc4b802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:46.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.114+0000 7fc4c57fa640 1 -- 172.21.15.45:0/919253911 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc4b00613f0 con 0x7fc4c80d2dc0 2024-05-08T05:43:46.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.346+0000 7fc4ce8da640 1 -- 172.21.15.45:0/919253911 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc4c80c87e0 con 0x7fc4c80d2dc0 2024-05-08T05:43:46.346 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-05-08T05:43:46.346+0000 7f124de12640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T05:43:46.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.348+0000 7fc4c57fa640 1 -- 172.21.15.45:0/919253911 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7fc4b0060cf0 con 0x7fc4c80d2dc0 2024-05-08T05:43:46.348 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T05:43:46.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.350+0000 7fc4ce8da640 1 -- 172.21.15.45:0/919253911 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc4a80762d0 msgr2=0x7fc4a8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:46.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.350+0000 7fc4ce8da640 1 --2- 172.21.15.45:0/919253911 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc4a80762d0 0x7fc4a8078790 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8034000 tx=0x7fc4b802e000 comp rx=0 tx=0).stop 2024-05-08T05:43:46.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.350+0000 7fc4ce8da640 1 -- 172.21.15.45:0/919253911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c80d2dc0 msgr2=0x7fc4c81541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:46.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.350+0000 7fc4ce8da640 1 --2- 172.21.15.45:0/919253911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c80d2dc0 0x7fc4c81541d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000d800 tx=0x7fc4b000dcc0 comp rx=0 tx=0).stop 2024-05-08T05:43:46.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.350+0000 7fc4ce8da640 1 -- 172.21.15.45:0/919253911 shutdown_connections 2024-05-08T05:43:46.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.350+0000 7fc4ce8da640 1 --2- 172.21.15.45:0/919253911 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc4a80762d0 0x7fc4a8078790 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:46.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.350+0000 7fc4ce8da640 1 --2- 172.21.15.45:0/919253911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4c81629a0 0x7fc4c80c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:46.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.350+0000 7fc4ce8da640 1 --2- 172.21.15.45:0/919253911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4c80d2dc0 0x7fc4c81541d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:46.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.350+0000 7fc4ce8da640 1 --2- 172.21.15.45:0/919253911 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4c80c8ed0 0x7fc4c80a26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:46.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.350+0000 7fc4ce8da640 1 -- 172.21.15.45:0/919253911 >> 172.21.15.45:0/919253911 conn(0x7fc4c80d4fe0 msgr2=0x7fc4c80c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:46.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.351+0000 7fc4ce8da640 1 -- 172.21.15.45:0/919253911 shutdown_connections 2024-05-08T05:43:46.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.351+0000 7fc4ce8da640 1 -- 172.21.15.45:0/919253911 wait complete. 2024-05-08T05:43:46.355 INFO:tasks.ceph.mds.b.smithi063.stderr: -6154> 2024-05-08T05:43:01.550+0000 7f3c6d599640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:43:46.358 INFO:tasks.ceph.mds.b.smithi063.stderr: -3102> 2024-05-08T05:43:12.115+0000 7f3c69591640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={4922=0-4194304@1} caps={4922=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=4922 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x557852b64c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-08T05:43:46.362 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T05:43:46.389 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-08T05:43:46.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.467+0000 7fd0776a0640 1 Processor -- start 2024-05-08T05:43:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.467+0000 7fd0776a0640 1 -- start start 2024-05-08T05:43:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.468+0000 7fd0776a0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0701629a0 0x7fd0700d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.468+0000 7fd0776a0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0700d3330 con 0x7fd0700d2dc0 2024-05-08T05:43:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.468+0000 7fd0776a0640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0700d2170 con 0x7fd0700d0cb0 2024-05-08T05:43:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.468+0000 7fd0776a0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fd0700d2450 con 0x7fd0701629a0 2024-05-08T05:43:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.468+0000 7fd07543c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd0701629a0 0x7fd0700d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.468+0000 7fd074c3b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd0700d2dc0 0x7fd0700c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58744/0 (socket says 172.21.15.45:58744) 2024-05-08T05:43:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.468+0000 7fd074c3b640 1 -- 172.21.15.45:0/359149062 learned_addr learned my addr 172.21.15.45:0/359149062 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.469+0000 7fd07543c640 1 -- 172.21.15.45:0/359149062 >> v1:172.21.15.63:6789/0 conn(0x7fd0700d0cb0 legacy=0x7fd0700c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.469+0000 7fd07543c640 1 -- 172.21.15.45:0/359149062 >> v1:172.21.15.45:6789/0 conn(0x7fd0700d2dc0 legacy=0x7fd0700c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:46.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.469+0000 7fd07543c640 1 -- 172.21.15.45:0/359149062 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0700d2730 con 0x7fd0701629a0 2024-05-08T05:43:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.469+0000 7fd07543c640 1 --2- 172.21.15.45:0/359149062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0701629a0 0x7fd0700d35c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd060009d90 tx=0x7fd06002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4668ab57ea1aeca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.470+0000 7fd06ffff640 1 -- 172.21.15.45:0/359149062 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060031600 con 0x7fd0701629a0 2024-05-08T05:43:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.470+0000 7fd06ffff640 1 -- 172.21.15.45:0/359149062 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0600317a0 con 0x7fd0701629a0 2024-05-08T05:43:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.470+0000 7fd06ffff640 1 -- 172.21.15.45:0/359149062 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060031b70 con 0x7fd0701629a0 2024-05-08T05:43:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.470+0000 7fd0776a0640 1 -- 172.21.15.45:0/359149062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0701629a0 msgr2=0x7fd0700d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.470+0000 7fd0776a0640 1 --2- 172.21.15.45:0/359149062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0701629a0 0x7fd0700d35c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd060009d90 tx=0x7fd06002a5d0 comp rx=0 tx=0).stop 2024-05-08T05:43:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.470+0000 7fd0776a0640 1 -- 172.21.15.45:0/359149062 shutdown_connections 2024-05-08T05:43:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.470+0000 7fd0776a0640 1 --2- 172.21.15.45:0/359149062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0701629a0 0x7fd0700d35c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.470+0000 7fd0776a0640 1 -- 172.21.15.45:0/359149062 >> 172.21.15.45:0/359149062 conn(0x7fd0700d4fe0 msgr2=0x7fd0700d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.470+0000 7fd0776a0640 1 -- 172.21.15.45:0/359149062 shutdown_connections 2024-05-08T05:43:46.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.470+0000 7fd0776a0640 1 -- 172.21.15.45:0/359149062 wait complete. 2024-05-08T05:43:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.471+0000 7fd0776a0640 1 Processor -- start 2024-05-08T05:43:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.472+0000 7fd0776a0640 1 -- start start 2024-05-08T05:43:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.472+0000 7fd0776a0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0700d0cb0 0x7fd07009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.472+0000 7fd0776a0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0700d2dc0 0x7fd0700a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.472+0000 7fd0776a0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0701629a0 0x7fd0700b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.472+0000 7fd0776a0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0700cf440 con 0x7fd0700d2dc0 2024-05-08T05:43:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.472+0000 7fd0776a0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd0700cf2c0 con 0x7fd0701629a0 2024-05-08T05:43:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.472+0000 7fd0776a0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd0700d3b00 con 0x7fd0700d0cb0 2024-05-08T05:43:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.472+0000 7fd074c3b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0700d2dc0 0x7fd0700a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.472+0000 7fd074c3b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0700d2dc0 0x7fd0700a10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58966/0 (socket says 172.21.15.45:58966) 2024-05-08T05:43:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.472+0000 7fd074c3b640 1 -- 172.21.15.45:0/1969405075 learned_addr learned my addr 172.21.15.45:0/1969405075 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.473+0000 7fd075c3d640 1 --2- 172.21.15.45:0/1969405075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0701629a0 0x7fd0700b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.473+0000 7fd07543c640 1 --2- 172.21.15.45:0/1969405075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0700d0cb0 0x7fd07009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.473+0000 7fd074c3b640 1 -- 172.21.15.45:0/1969405075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0700d0cb0 msgr2=0x7fd07009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:46.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.473+0000 7fd074c3b640 1 --2- 172.21.15.45:0/1969405075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0700d0cb0 0x7fd07009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:46.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.473+0000 7fd074c3b640 1 -- 172.21.15.45:0/1969405075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0701629a0 msgr2=0x7fd0700b82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:46.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.473+0000 7fd074c3b640 1 --2- 172.21.15.45:0/1969405075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0701629a0 0x7fd0700b82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:46.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.473+0000 7fd074c3b640 1 -- 172.21.15.45:0/1969405075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0700b8830 con 0x7fd0700d2dc0 2024-05-08T05:43:46.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.473+0000 7fd074c3b640 1 --2- 172.21.15.45:0/1969405075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0700d2dc0 0x7fd0700a10a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd06400c500 tx=0x7fd06400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:46.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.473+0000 7fd06dffb640 1 -- 172.21.15.45:0/1969405075 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064017070 con 0x7fd0700d2dc0 2024-05-08T05:43:46.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.473+0000 7fd06dffb640 1 -- 172.21.15.45:0/1969405075 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd064005ce0 con 0x7fd0700d2dc0 2024-05-08T05:43:46.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.473+0000 7fd06dffb640 1 -- 172.21.15.45:0/1969405075 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0640072f0 con 0x7fd0700d2dc0 2024-05-08T05:43:46.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.473+0000 7fd0776a0640 1 -- 172.21.15.45:0/1969405075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07009f2e0 con 0x7fd0700d2dc0 2024-05-08T05:43:46.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.473+0000 7fd0776a0640 1 -- 172.21.15.45:0/1969405075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0700a19b0 con 0x7fd0700d2dc0 2024-05-08T05:43:46.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.474+0000 7fd0776a0640 1 -- 172.21.15.45:0/1969405075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0700693f0 con 0x7fd0700d2dc0 2024-05-08T05:43:46.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.475+0000 7fd06dffb640 1 -- 172.21.15.45:0/1969405075 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd06401a070 con 0x7fd0700d2dc0 2024-05-08T05:43:46.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.475+0000 7fd06dffb640 1 --2- 172.21.15.45:0/1969405075 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd05c0762d0 0x7fd05c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:46.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.475+0000 7fd06dffb640 1 -- 172.21.15.45:0/1969405075 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6245+0+0 (secure 0 0 0) 0x7fd064096ab0 con 0x7fd0700d2dc0 2024-05-08T05:43:46.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.475+0000 7fd07543c640 1 --2- 172.21.15.45:0/1969405075 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd05c0762d0 0x7fd05c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:46.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.476+0000 7fd07543c640 1 --2- 172.21.15.45:0/1969405075 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd05c0762d0 0x7fd05c078790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd060034000 tx=0x7fd060033040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:46.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.480+0000 7fd06dffb640 1 -- 172.21.15.45:0/1969405075 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd06405fc30 con 0x7fd0700d2dc0 2024-05-08T05:43:46.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:46.714+0000 7fd0776a0640 1 -- 172.21.15.45:0/1969405075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd07006da10 con 0x7fd0700d2dc0 2024-05-08T05:43:47.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.353+0000 7fd06dffb640 1 -- 172.21.15.45:0/1969405075 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7fd064063f30 con 0x7fd0700d2dc0 2024-05-08T05:43:47.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.356+0000 7fd0776a0640 1 -- 172.21.15.45:0/1969405075 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd05c0762d0 msgr2=0x7fd05c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:47.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.356+0000 7fd0776a0640 1 --2- 172.21.15.45:0/1969405075 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd05c0762d0 0x7fd05c078790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd060034000 tx=0x7fd060033040 comp rx=0 tx=0).stop 2024-05-08T05:43:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.356+0000 7fd0776a0640 1 -- 172.21.15.45:0/1969405075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0700d2dc0 msgr2=0x7fd0700a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.356+0000 7fd0776a0640 1 --2- 172.21.15.45:0/1969405075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0700d2dc0 0x7fd0700a10a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd06400c500 tx=0x7fd06400c9c0 comp rx=0 tx=0).stop 2024-05-08T05:43:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.356+0000 7fd0776a0640 1 -- 172.21.15.45:0/1969405075 shutdown_connections 2024-05-08T05:43:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.356+0000 7fd0776a0640 1 --2- 172.21.15.45:0/1969405075 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd05c0762d0 0x7fd05c078790 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.356+0000 7fd0776a0640 1 --2- 172.21.15.45:0/1969405075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd0701629a0 0x7fd0700b82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.356+0000 7fd0776a0640 1 --2- 172.21.15.45:0/1969405075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0700d2dc0 0x7fd0700a10a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.356+0000 7fd0776a0640 1 --2- 172.21.15.45:0/1969405075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd0700d0cb0 0x7fd07009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.356+0000 7fd0776a0640 1 -- 172.21.15.45:0/1969405075 >> 172.21.15.45:0/1969405075 conn(0x7fd0700d4fe0 msgr2=0x7fd07006e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.356+0000 7fd0776a0640 1 -- 172.21.15.45:0/1969405075 shutdown_connections 2024-05-08T05:43:47.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.356+0000 7fd0776a0640 1 -- 172.21.15.45:0/1969405075 wait complete. 2024-05-08T05:43:47.370 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T05:43:47.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.485+0000 7f417df89640 1 Processor -- start 2024-05-08T05:43:47.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.485+0000 7f417df89640 1 -- start start 2024-05-08T05:43:47.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.486+0000 7f417df89640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f41780cf8f0 0x7f4178070500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:47.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.486+0000 7f417df89640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f41780d3920 0x7f4178070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:47.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.486+0000 7f417df89640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41780d2030 con 0x7f41780cfcd0 2024-05-08T05:43:47.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.486+0000 7f417df89640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f417815f8d0 con 0x7f41780cf8f0 2024-05-08T05:43:47.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.486+0000 7f417df89640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f41780d22b0 con 0x7f41780d3920 2024-05-08T05:43:47.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.486+0000 7f4176ffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f41780d3920 0x7f4178070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:47.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.486+0000 7f41777fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f41780cf8f0 0x7f4178070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:47.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.486+0000 7f4177fff640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f41780cfcd0 0x7f41780cdf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:57674/0 (socket says 172.21.15.45:57674) 2024-05-08T05:43:47.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.486+0000 7f4177fff640 1 -- 172.21.15.45:0/846266521 learned_addr learned my addr 172.21.15.45:0/846266521 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:47.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.487+0000 7f41767fc640 1 -- 172.21.15.45:0/846266521 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 397549548 0 0) 0x7f4170001490 con 0x7f41780cfcd0 2024-05-08T05:43:47.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.487+0000 7f41767fc640 1 -- 172.21.15.45:0/846266521 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4160003520 con 0x7f41780cfcd0 2024-05-08T05:43:47.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.487+0000 7f41777fe640 1 -- 172.21.15.45:0/846266521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f41780d3920 msgr2=0x7f4178070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:47.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.487+0000 7f41777fe640 1 --2- 172.21.15.45:0/846266521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f41780d3920 0x7f4178070a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:47.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.487+0000 7f41777fe640 1 -- 172.21.15.45:0/846266521 >> v1:172.21.15.63:6789/0 conn(0x7f41780cfcd0 legacy=0x7f41780cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:47.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.487+0000 7f41777fe640 1 -- 172.21.15.45:0/846266521 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41780d2590 con 0x7f41780cf8f0 2024-05-08T05:43:47.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.487+0000 7f41777fe640 1 --2- 172.21.15.45:0/846266521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41780cf8f0 0x7f4178070500 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f41680097a0 tx=0x7f416802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5335806c6e30c5d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:47.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.488+0000 7f41767fc640 1 -- 172.21.15.45:0/846266521 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168037070 con 0x7f41780cf8f0 2024-05-08T05:43:47.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.488+0000 7f41767fc640 1 -- 172.21.15.45:0/846266521 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4168032440 con 0x7f41780cf8f0 2024-05-08T05:43:47.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.488+0000 7f41767fc640 1 -- 172.21.15.45:0/846266521 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168032810 con 0x7f41780cf8f0 2024-05-08T05:43:47.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.488+0000 7f417df89640 1 -- 172.21.15.45:0/846266521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41780cf8f0 msgr2=0x7f4178070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:47.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.488+0000 7f417df89640 1 --2- 172.21.15.45:0/846266521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41780cf8f0 0x7f4178070500 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f41680097a0 tx=0x7f416802a210 comp rx=0 tx=0).stop 2024-05-08T05:43:47.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.488+0000 7f417df89640 1 -- 172.21.15.45:0/846266521 shutdown_connections 2024-05-08T05:43:47.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.488+0000 7f417df89640 1 --2- 172.21.15.45:0/846266521 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f41780d3920 0x7f4178070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:47.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.488+0000 7f417df89640 1 --2- 172.21.15.45:0/846266521 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41780cf8f0 0x7f4178070500 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:47.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.488+0000 7f417df89640 1 -- 172.21.15.45:0/846266521 >> 172.21.15.45:0/846266521 conn(0x7f41780d2e00 msgr2=0x7f41780d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:47.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.488+0000 7f417df89640 1 -- 172.21.15.45:0/846266521 shutdown_connections 2024-05-08T05:43:47.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.488+0000 7f417df89640 1 -- 172.21.15.45:0/846266521 wait complete. 2024-05-08T05:43:47.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.490+0000 7f417df89640 1 Processor -- start 2024-05-08T05:43:47.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.490+0000 7f417df89640 1 -- start start 2024-05-08T05:43:47.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.490+0000 7f417df89640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f41780cf8f0 0x7f41781564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:47.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.491+0000 7f417df89640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41780cfcd0 0x7f41780cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:47.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.491+0000 7f417df89640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41780d3920 0x7f41780cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:47.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.491+0000 7f417df89640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f41780cda10 con 0x7f41780d3920 2024-05-08T05:43:47.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.491+0000 7f417df89640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f41780cd730 con 0x7f41780cfcd0 2024-05-08T05:43:47.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.491+0000 7f417df89640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4178071610 con 0x7f41780cf8f0 2024-05-08T05:43:47.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.491+0000 7f4177fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41780d3920 0x7f41780cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:47.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.491+0000 7f4177fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41780d3920 0x7f41780cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58974/0 (socket says 172.21.15.45:58974) 2024-05-08T05:43:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.491+0000 7f4177fff640 1 -- 172.21.15.45:0/1566466746 learned_addr learned my addr 172.21.15.45:0/1566466746 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.491+0000 7f41777fe640 1 --2- 172.21.15.45:0/1566466746 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f41780cf8f0 0x7f41781564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.491+0000 7f4176ffd640 1 --2- 172.21.15.45:0/1566466746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41780cfcd0 0x7f41780cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.491+0000 7f4177fff640 1 -- 172.21.15.45:0/1566466746 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f41780cf8f0 msgr2=0x7f41781564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.491+0000 7f4177fff640 1 --2- 172.21.15.45:0/1566466746 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f41780cf8f0 0x7f41781564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.491+0000 7f4177fff640 1 -- 172.21.15.45:0/1566466746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41780cfcd0 msgr2=0x7f41780cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.491+0000 7f4177fff640 1 --2- 172.21.15.45:0/1566466746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41780cfcd0 0x7f41780cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.491+0000 7f4177fff640 1 -- 172.21.15.45:0/1566466746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4178159040 con 0x7f41780d3920 2024-05-08T05:43:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.491+0000 7f4177fff640 1 --2- 172.21.15.45:0/1566466746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41780d3920 0x7f41780cc600 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f417000cae0 tx=0x7f417000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.491+0000 7f415ffff640 1 -- 172.21.15.45:0/1566466746 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417000ac40 con 0x7f41780d3920 2024-05-08T05:43:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.491+0000 7f415ffff640 1 -- 172.21.15.45:0/1566466746 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f417000ade0 con 0x7f41780d3920 2024-05-08T05:43:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.491+0000 7f417df89640 1 -- 172.21.15.45:0/1566466746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4178159320 con 0x7f41780d3920 2024-05-08T05:43:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.491+0000 7f415ffff640 1 -- 172.21.15.45:0/1566466746 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170012620 con 0x7f41780d3920 2024-05-08T05:43:47.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.492+0000 7f417df89640 1 -- 172.21.15.45:0/1566466746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41780c6e60 con 0x7f41780d3920 2024-05-08T05:43:47.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.492+0000 7f417df89640 1 -- 172.21.15.45:0/1566466746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41780c7060 con 0x7f41780d3920 2024-05-08T05:43:47.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.493+0000 7f415ffff640 1 -- 172.21.15.45:0/1566466746 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f41700127c0 con 0x7f41780d3920 2024-05-08T05:43:47.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.494+0000 7f415ffff640 1 --2- 172.21.15.45:0/1566466746 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f41640762d0 0x7f4164078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:47.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.494+0000 7f415ffff640 1 -- 172.21.15.45:0/1566466746 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6245+0+0 (secure 0 0 0) 0x7f4170097d80 con 0x7f41780d3920 2024-05-08T05:43:47.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.494+0000 7f41777fe640 1 --2- 172.21.15.45:0/1566466746 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f41640762d0 0x7f4164078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:47.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.494+0000 7f41777fe640 1 --2- 172.21.15.45:0/1566466746 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f41640762d0 0x7f4164078790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4168034000 tx=0x7f416802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:47.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.498+0000 7f415ffff640 1 -- 172.21.15.45:0/1566466746 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f41780c7060 con 0x7f41780d3920 2024-05-08T05:43:47.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:47.691+0000 7f417df89640 1 -- 172.21.15.45:0/1566466746 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f41780693f0 con 0x7f41780d3920 2024-05-08T05:43:48.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.354+0000 7f415ffff640 1 -- 172.21.15.45:0/1566466746 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v51) ==== 181+0+0 (secure 0 0 0) 0x7f41780d2030 con 0x7f41780d3920 2024-05-08T05:43:48.355 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-05-08T05:43:48.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.358+0000 7f417df89640 1 -- 172.21.15.45:0/1566466746 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f41640762d0 msgr2=0x7f4164078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:48.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.358+0000 7f417df89640 1 --2- 172.21.15.45:0/1566466746 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f41640762d0 0x7f4164078790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4168034000 tx=0x7f416802e000 comp rx=0 tx=0).stop 2024-05-08T05:43:48.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.358+0000 7f417df89640 1 -- 172.21.15.45:0/1566466746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41780d3920 msgr2=0x7f41780cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:48.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.358+0000 7f417df89640 1 --2- 172.21.15.45:0/1566466746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41780d3920 0x7f41780cc600 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f417000cae0 tx=0x7f417000a060 comp rx=0 tx=0).stop 2024-05-08T05:43:48.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.358+0000 7f417df89640 1 -- 172.21.15.45:0/1566466746 shutdown_connections 2024-05-08T05:43:48.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.358+0000 7f417df89640 1 --2- 172.21.15.45:0/1566466746 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f41640762d0 0x7f4164078790 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:48.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.358+0000 7f417df89640 1 --2- 172.21.15.45:0/1566466746 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41780d3920 0x7f41780cc600 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:48.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.358+0000 7f417df89640 1 --2- 172.21.15.45:0/1566466746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41780cfcd0 0x7f41780cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:48.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.358+0000 7f417df89640 1 --2- 172.21.15.45:0/1566466746 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f41780cf8f0 0x7f41781564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:48.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.358+0000 7f417df89640 1 -- 172.21.15.45:0/1566466746 >> 172.21.15.45:0/1566466746 conn(0x7f41780d2e00 msgr2=0x7f41780b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:48.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.359+0000 7f417df89640 1 -- 172.21.15.45:0/1566466746 shutdown_connections 2024-05-08T05:43:48.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.359+0000 7f417df89640 1 -- 172.21.15.45:0/1566466746 wait complete. 2024-05-08T05:43:48.373 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T05:43:48.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.487+0000 7f77efef8640 1 Processor -- start 2024-05-08T05:43:48.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.487+0000 7f77efef8640 1 -- start start 2024-05-08T05:43:48.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.488+0000 7f77efef8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77e81629f0 0x7f77e80d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:48.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.488+0000 7f77efef8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77e80cde80 con 0x7f77e80cfef0 2024-05-08T05:43:48.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.488+0000 7f77efef8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77e80ce120 con 0x7f77e80d22a0 2024-05-08T05:43:48.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.488+0000 7f77efef8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f77e80ce400 con 0x7f77e81629f0 2024-05-08T05:43:48.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.488+0000 7f77ee495640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f77e80cfef0 0x7f77e806d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58754/0 (socket says 172.21.15.45:58754) 2024-05-08T05:43:48.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.488+0000 7f77ee495640 1 -- 172.21.15.45:0/506974598 learned_addr learned my addr 172.21.15.45:0/506974598 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.488+0000 7f77ecc92640 1 -- 172.21.15.45:0/506974598 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4200772421 0 0) 0x7f77e80cde80 con 0x7f77e80cfef0 2024-05-08T05:43:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.488+0000 7f77ecc92640 1 -- 172.21.15.45:0/506974598 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77d4003520 con 0x7f77e80cfef0 2024-05-08T05:43:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.488+0000 7f77edc94640 1 --2- 172.21.15.45:0/506974598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77e81629f0 0x7f77e80d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77ecc92640 1 -- 172.21.15.45:0/506974598 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3785359300 0 0) 0x7f77d4003520 con 0x7f77e80cfef0 2024-05-08T05:43:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77ecc92640 1 -- 172.21.15.45:0/506974598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77e81629f0 msgr2=0x7f77e80d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77ecc92640 1 --2- 172.21.15.45:0/506974598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77e81629f0 0x7f77e80d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77ecc92640 1 -- 172.21.15.45:0/506974598 >> v1:172.21.15.63:6789/0 conn(0x7f77e80d22a0 legacy=0x7f77e80d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77ecc92640 1 -- 172.21.15.45:0/506974598 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e80ce6e0 con 0x7f77e80cfef0 2024-05-08T05:43:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77ecc92640 1 -- 172.21.15.45:0/506974598 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f77e0001a70 con 0x7f77e80cfef0 2024-05-08T05:43:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77ecc92640 1 -- 172.21.15.45:0/506974598 >> v1:172.21.15.45:6789/0 conn(0x7f77e80cfef0 legacy=0x7f77e806d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77ecc92640 1 --2- 172.21.15.45:0/506974598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77e81629f0 0x7f77d4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77ecc92640 1 --2- 172.21.15.45:0/506974598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77d4004000 0x7f77d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77ecc92640 1 --2- 172.21.15.45:0/506974598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77d4004d40 0x7f77d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77ecc92640 1 -- 172.21.15.45:0/506974598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f77e80ce400 con 0x7f77d4004000 2024-05-08T05:43:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77ecc92640 1 -- 172.21.15.45:0/506974598 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f77d400bb50 con 0x7f77d4004d40 2024-05-08T05:43:48.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77ecc92640 1 -- 172.21.15.45:0/506974598 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f77d400bd50 con 0x7f77e81629f0 2024-05-08T05:43:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77ed493640 1 --2- 172.21.15.45:0/506974598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77d4004000 0x7f77d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77edc94640 1 --2- 172.21.15.45:0/506974598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77e81629f0 0x7f77d4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77ee495640 1 --2- 172.21.15.45:0/506974598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77d4004d40 0x7f77d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77ed493640 1 -- 172.21.15.45:0/506974598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77e81629f0 msgr2=0x7f77d4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77ed493640 1 --2- 172.21.15.45:0/506974598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77e81629f0 0x7f77d4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77ed493640 1 -- 172.21.15.45:0/506974598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77d4004d40 msgr2=0x7f77d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77ed493640 1 --2- 172.21.15.45:0/506974598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77d4004d40 0x7f77d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.489+0000 7f77ed493640 1 -- 172.21.15.45:0/506974598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d4003520 con 0x7f77d4004000 2024-05-08T05:43:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.490+0000 7f77ed493640 1 --2- 172.21.15.45:0/506974598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77d4004000 0x7f77d4004800 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f77d8001760 tx=0x7f77d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53688abe9b38f2b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.490+0000 7f77ecc92640 1 -- 172.21.15.45:0/506974598 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d80013b0 con 0x7f77d4004000 2024-05-08T05:43:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.490+0000 7f77ecc92640 1 -- 172.21.15.45:0/506974598 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d80052d0 con 0x7f77d4004000 2024-05-08T05:43:48.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.490+0000 7f77ecc92640 1 -- 172.21.15.45:0/506974598 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d802aaa0 con 0x7f77d4004000 2024-05-08T05:43:48.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.491+0000 7f77efef8640 1 -- 172.21.15.45:0/506974598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77d4004000 msgr2=0x7f77d4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:48.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.491+0000 7f77efef8640 1 --2- 172.21.15.45:0/506974598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77d4004000 0x7f77d4004800 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f77d8001760 tx=0x7f77d4005830 comp rx=0 tx=0).stop 2024-05-08T05:43:48.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.491+0000 7f77ee495640 1 -- 172.21.15.45:0/506974598 reap_dead start 2024-05-08T05:43:48.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.491+0000 7f77efef8640 1 -- 172.21.15.45:0/506974598 shutdown_connections 2024-05-08T05:43:48.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.491+0000 7f77efef8640 1 -- 172.21.15.45:0/506974598 >> 172.21.15.45:0/506974598 conn(0x7f77e80d2d80 msgr2=0x7f77e80d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:48.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.491+0000 7f77efef8640 1 -- 172.21.15.45:0/506974598 shutdown_connections 2024-05-08T05:43:48.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.491+0000 7f77efef8640 1 -- 172.21.15.45:0/506974598 wait complete. 2024-05-08T05:43:48.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.492+0000 7f77efef8640 1 Processor -- start 2024-05-08T05:43:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.492+0000 7f77efef8640 1 -- start start 2024-05-08T05:43:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.493+0000 7f77efef8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77d4004000 0x7f77e80b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.493+0000 7f77efef8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77e80b3df0 0x7f77e809d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.493+0000 7f77efef8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77e815a640 0x7f77e80ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.493+0000 7f77efef8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f77e80d1ef0 con 0x7f77e80b3df0 2024-05-08T05:43:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.493+0000 7f77efef8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f77e80d1d70 con 0x7f77d4004000 2024-05-08T05:43:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.493+0000 7f77efef8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f77e80d1120 con 0x7f77e815a640 2024-05-08T05:43:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.493+0000 7f77ed493640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77e80b3df0 0x7f77e809d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.493+0000 7f77ed493640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77e80b3df0 0x7f77e809d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58994/0 (socket says 172.21.15.45:58994) 2024-05-08T05:43:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.493+0000 7f77ed493640 1 -- 172.21.15.45:0/1526607742 learned_addr learned my addr 172.21.15.45:0/1526607742 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.493+0000 7f77edc94640 1 --2- 172.21.15.45:0/1526607742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77d4004000 0x7f77e80b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.493+0000 7f77ee495640 1 --2- 172.21.15.45:0/1526607742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77e815a640 0x7f77e80ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.493+0000 7f77ed493640 1 -- 172.21.15.45:0/1526607742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77e815a640 msgr2=0x7f77e80ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:48.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.493+0000 7f77ed493640 1 --2- 172.21.15.45:0/1526607742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77e815a640 0x7f77e80ccb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.493+0000 7f77ed493640 1 -- 172.21.15.45:0/1526607742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77d4004000 msgr2=0x7f77e80b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.493+0000 7f77ed493640 1 --2- 172.21.15.45:0/1526607742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77d4004000 0x7f77e80b6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.493+0000 7f77ed493640 1 -- 172.21.15.45:0/1526607742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e80b5190 con 0x7f77e80b3df0 2024-05-08T05:43:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.494+0000 7f77ed493640 1 --2- 172.21.15.45:0/1526607742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77e80b3df0 0x7f77e809d630 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f77d8001880 tx=0x7f77d8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.494+0000 7f77de7fc640 1 -- 172.21.15.45:0/1526607742 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8037040 con 0x7f77e80b3df0 2024-05-08T05:43:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.494+0000 7f77de7fc640 1 -- 172.21.15.45:0/1526607742 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d8005480 con 0x7f77e80b3df0 2024-05-08T05:43:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.494+0000 7f77efef8640 1 -- 172.21.15.45:0/1526607742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e809db70 con 0x7f77e80b3df0 2024-05-08T05:43:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.494+0000 7f77de7fc640 1 -- 172.21.15.45:0/1526607742 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d80013b0 con 0x7f77e80b3df0 2024-05-08T05:43:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.494+0000 7f77efef8640 1 -- 172.21.15.45:0/1526607742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e80b56e0 con 0x7f77e80b3df0 2024-05-08T05:43:48.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.495+0000 7f77efef8640 1 -- 172.21.15.45:0/1526607742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e80693f0 con 0x7f77e80b3df0 2024-05-08T05:43:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.496+0000 7f77de7fc640 1 -- 172.21.15.45:0/1526607742 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f77d8044020 con 0x7f77e80b3df0 2024-05-08T05:43:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.496+0000 7f77de7fc640 1 --2- 172.21.15.45:0/1526607742 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f77cc0762d0 0x7f77cc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.497+0000 7f77de7fc640 1 -- 172.21.15.45:0/1526607742 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5807+0+0 (secure 0 0 0) 0x7f77d80b6ea0 con 0x7f77e80b3df0 2024-05-08T05:43:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.497+0000 7f77edc94640 1 --2- 172.21.15.45:0/1526607742 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f77cc0762d0 0x7f77cc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:48.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.497+0000 7f77edc94640 1 --2- 172.21.15.45:0/1526607742 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f77cc0762d0 0x7f77cc078790 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009b00 tx=0x7f77e4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:48.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.501+0000 7f77de7fc640 1 -- 172.21.15.45:0/1526607742 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f77d804b020 con 0x7f77e80b3df0 2024-05-08T05:43:48.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:48.692+0000 7f77efef8640 1 -- 172.21.15.45:0/1526607742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f77e8160580 con 0x7f77e80b3df0 2024-05-08T05:43:49.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.359+0000 7f77de7fc640 1 -- 172.21.15.45:0/1526607742 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v52) ==== 169+0+0 (secure 0 0 0) 0x7f77d8080830 con 0x7f77e80b3df0 2024-05-08T05:43:49.360 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-05-08T05:43:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.362+0000 7f77efef8640 1 -- 172.21.15.45:0/1526607742 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f77cc0762d0 msgr2=0x7f77cc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.362+0000 7f77efef8640 1 --2- 172.21.15.45:0/1526607742 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f77cc0762d0 0x7f77cc078790 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009b00 tx=0x7f77e4008040 comp rx=0 tx=0).stop 2024-05-08T05:43:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.363+0000 7f77efef8640 1 -- 172.21.15.45:0/1526607742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77e80b3df0 msgr2=0x7f77e809d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:49.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.363+0000 7f77efef8640 1 --2- 172.21.15.45:0/1526607742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77e80b3df0 0x7f77e809d630 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f77d8001880 tx=0x7f77d8001120 comp rx=0 tx=0).stop 2024-05-08T05:43:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.364+0000 7f77efef8640 1 -- 172.21.15.45:0/1526607742 shutdown_connections 2024-05-08T05:43:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.364+0000 7f77efef8640 1 --2- 172.21.15.45:0/1526607742 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f77cc0762d0 0x7f77cc078790 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.364+0000 7f77efef8640 1 --2- 172.21.15.45:0/1526607742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77e815a640 0x7f77e80ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.364+0000 7f77efef8640 1 --2- 172.21.15.45:0/1526607742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77e80b3df0 0x7f77e809d630 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.364+0000 7f77efef8640 1 --2- 172.21.15.45:0/1526607742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77d4004000 0x7f77e80b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.364+0000 7f77efef8640 1 -- 172.21.15.45:0/1526607742 >> 172.21.15.45:0/1526607742 conn(0x7f77e80d2d80 msgr2=0x7f77e806d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:49.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.365+0000 7f77efef8640 1 -- 172.21.15.45:0/1526607742 shutdown_connections 2024-05-08T05:43:49.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.365+0000 7f77efef8640 1 -- 172.21.15.45:0/1526607742 wait complete. 2024-05-08T05:43:49.381 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T05:43:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.492+0000 7f3b5d16a640 1 Processor -- start 2024-05-08T05:43:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.493+0000 7f3b5d16a640 1 -- start start 2024-05-08T05:43:49.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.493+0000 7f3b5d16a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3b580d1970 0x7f3b580d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.493+0000 7f3b5d16a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b581630b0 con 0x7f3b580d4cc0 2024-05-08T05:43:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.493+0000 7f3b5d16a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b580d3e70 con 0x7f3b5815f420 2024-05-08T05:43:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.493+0000 7f3b5d16a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3b580d2550 con 0x7f3b580d1970 2024-05-08T05:43:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b5759e640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3b580d4cc0 0x7f3b580d50e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:58758/0 (socket says 172.21.15.45:58758) 2024-05-08T05:43:49.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b5759e640 1 -- 172.21.15.45:0/2435150852 learned_addr learned my addr 172.21.15.45:0/2435150852 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:49.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b5659c640 1 -- 172.21.15.45:0/2435150852 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1326121745 0 0) 0x7f3b50001490 con 0x7f3b580d4cc0 2024-05-08T05:43:49.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b4ffff640 1 --2- 172.21.15.45:0/2435150852 >> v2:172.21.15.63:3300/0 conn(0x7f3b580d1970 0x7f3b580d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:49.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b5659c640 1 -- 172.21.15.45:0/2435150852 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b44003520 con 0x7f3b580d4cc0 2024-05-08T05:43:49.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b5659c640 1 -- 172.21.15.45:0/2435150852 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1603896902 0 0) 0x7f3b44003520 con 0x7f3b580d4cc0 2024-05-08T05:43:49.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b5659c640 1 -- 172.21.15.45:0/2435150852 >> v2:172.21.15.63:3300/0 conn(0x7f3b580d1970 msgr2=0x7f3b580d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:49.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b5659c640 1 --2- 172.21.15.45:0/2435150852 >> v2:172.21.15.63:3300/0 conn(0x7f3b580d1970 0x7f3b580d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b5659c640 1 -- 172.21.15.45:0/2435150852 >> v1:172.21.15.63:6789/0 conn(0x7f3b5815f420 legacy=0x7f3b580d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:49.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b5659c640 1 -- 172.21.15.45:0/2435150852 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b580d2c80 con 0x7f3b580d4cc0 2024-05-08T05:43:49.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b5659c640 1 -- 172.21.15.45:0/2435150852 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f3b50001c40 con 0x7f3b580d4cc0 2024-05-08T05:43:49.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b5659c640 1 -- 172.21.15.45:0/2435150852 >> v1:172.21.15.45:6789/0 conn(0x7f3b580d4cc0 legacy=0x7f3b580d50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:49.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b5659c640 1 --2- 172.21.15.45:0/2435150852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b44003a50 0x7f3b44003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:49.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b5659c640 1 --2- 172.21.15.45:0/2435150852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b44004800 0x7f3b44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:49.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b5659c640 1 --2- 172.21.15.45:0/2435150852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b440051c0 0x7f3b4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:49.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b5659c640 1 -- 172.21.15.45:0/2435150852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3b580d2550 con 0x7f3b440051c0 2024-05-08T05:43:49.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b5659c640 1 -- 172.21.15.45:0/2435150852 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4400c160 con 0x7f3b44003a50 2024-05-08T05:43:49.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b5659c640 1 -- 172.21.15.45:0/2435150852 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4400c360 con 0x7f3b44004800 2024-05-08T05:43:49.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b5759e640 1 --2- 172.21.15.45:0/2435150852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b440051c0 0x7f3b4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b56d9d640 1 --2- 172.21.15.45:0/2435150852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b44003a50 0x7f3b44003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.494+0000 7f3b4ffff640 1 --2- 172.21.15.45:0/2435150852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b44004800 0x7f3b44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.495+0000 7f3b5759e640 1 -- 172.21.15.45:0/2435150852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b44004800 msgr2=0x7f3b44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.495+0000 7f3b5759e640 1 --2- 172.21.15.45:0/2435150852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b44004800 0x7f3b44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.495+0000 7f3b5759e640 1 -- 172.21.15.45:0/2435150852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b44003a50 msgr2=0x7f3b44003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.495+0000 7f3b5759e640 1 --2- 172.21.15.45:0/2435150852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b44003a50 0x7f3b44003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.495+0000 7f3b5759e640 1 -- 172.21.15.45:0/2435150852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b44003520 con 0x7f3b440051c0 2024-05-08T05:43:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.495+0000 7f3b5759e640 1 -- 172.21.15.45:0/2435150852 reap_dead start 2024-05-08T05:43:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.495+0000 7f3b5759e640 1 --2- 172.21.15.45:0/2435150852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b440051c0 0x7f3b4400b9c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3b5000aa00 tx=0x7f3b44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f0796da5a8e9f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.495+0000 7f3b5659c640 1 -- 172.21.15.45:0/2435150852 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5000a620 con 0x7f3b440051c0 2024-05-08T05:43:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.496+0000 7f3b5659c640 1 -- 172.21.15.45:0/2435150852 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b5002f070 con 0x7f3b440051c0 2024-05-08T05:43:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.496+0000 7f3b5659c640 1 -- 172.21.15.45:0/2435150852 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5002bde0 con 0x7f3b440051c0 2024-05-08T05:43:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.496+0000 7f3b5d16a640 1 -- 172.21.15.45:0/2435150852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b440051c0 msgr2=0x7f3b4400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:49.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.496+0000 7f3b5d16a640 1 --2- 172.21.15.45:0/2435150852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b440051c0 0x7f3b4400b9c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3b5000aa00 tx=0x7f3b44005830 comp rx=0 tx=0).stop 2024-05-08T05:43:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.496+0000 7f3b5d16a640 1 -- 172.21.15.45:0/2435150852 shutdown_connections 2024-05-08T05:43:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.496+0000 7f3b5d16a640 1 --2- 172.21.15.45:0/2435150852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b440051c0 0x7f3b4400b9c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.496+0000 7f3b5d16a640 1 -- 172.21.15.45:0/2435150852 >> 172.21.15.45:0/2435150852 conn(0x7f3b5808eac0 msgr2=0x7f3b5809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.496+0000 7f3b5d16a640 1 -- 172.21.15.45:0/2435150852 shutdown_connections 2024-05-08T05:43:49.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.496+0000 7f3b5d16a640 1 -- 172.21.15.45:0/2435150852 wait complete. 2024-05-08T05:43:49.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.497+0000 7f3b5d16a640 1 Processor -- start 2024-05-08T05:43:49.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.498+0000 7f3b5d16a640 1 -- start start 2024-05-08T05:43:49.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.498+0000 7f3b5d16a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b440051c0 0x7f3b5815a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:49.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.498+0000 7f3b5d16a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b580c8b00 0x7f3b580cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:49.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.498+0000 7f3b5d16a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b580b2dc0 0x7f3b580b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:49.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.498+0000 7f3b5d16a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3b580d2830 con 0x7f3b440051c0 2024-05-08T05:43:49.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.498+0000 7f3b5d16a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b580d3240 con 0x7f3b580b2dc0 2024-05-08T05:43:49.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.498+0000 7f3b5d16a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3b580d1e20 con 0x7f3b580c8b00 2024-05-08T05:43:49.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.498+0000 7f3b56d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b440051c0 0x7f3b5815a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:49.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.498+0000 7f3b56d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b440051c0 0x7f3b5815a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59014/0 (socket says 172.21.15.45:59014) 2024-05-08T05:43:49.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.498+0000 7f3b56d9d640 1 -- 172.21.15.45:0/1781836810 learned_addr learned my addr 172.21.15.45:0/1781836810 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:49.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.499+0000 7f3b5759e640 1 --2- 172.21.15.45:0/1781836810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b580b2dc0 0x7f3b580b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.499+0000 7f3b4ffff640 1 --2- 172.21.15.45:0/1781836810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b580c8b00 0x7f3b580cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.499+0000 7f3b56d9d640 1 -- 172.21.15.45:0/1781836810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b580c8b00 msgr2=0x7f3b580cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.499+0000 7f3b56d9d640 1 --2- 172.21.15.45:0/1781836810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b580c8b00 0x7f3b580cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.499+0000 7f3b56d9d640 1 -- 172.21.15.45:0/1781836810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b580b2dc0 msgr2=0x7f3b580b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.499+0000 7f3b56d9d640 1 --2- 172.21.15.45:0/1781836810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b580b2dc0 0x7f3b580b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.499+0000 7f3b56d9d640 1 -- 172.21.15.45:0/1781836810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b580c9070 con 0x7f3b440051c0 2024-05-08T05:43:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.499+0000 7f3b5759e640 1 --2- 172.21.15.45:0/1781836810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b580b2dc0 0x7f3b580b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.499+0000 7f3b4ffff640 1 --2- 172.21.15.45:0/1781836810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b580c8b00 0x7f3b580cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.499+0000 7f3b56d9d640 1 --2- 172.21.15.45:0/1781836810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b440051c0 0x7f3b5815a570 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800f850 tx=0x7f3b4800fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.499+0000 7f3b4f7fe640 1 -- 172.21.15.45:0/1781836810 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4801a070 con 0x7f3b440051c0 2024-05-08T05:43:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.499+0000 7f3b4f7fe640 1 -- 172.21.15.45:0/1781836810 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b4800fe40 con 0x7f3b440051c0 2024-05-08T05:43:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.499+0000 7f3b5d16a640 1 -- 172.21.15.45:0/1781836810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b580c9350 con 0x7f3b440051c0 2024-05-08T05:43:49.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.499+0000 7f3b4f7fe640 1 -- 172.21.15.45:0/1781836810 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b48007050 con 0x7f3b440051c0 2024-05-08T05:43:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.499+0000 7f3b5d16a640 1 -- 172.21.15.45:0/1781836810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b5815fa60 con 0x7f3b440051c0 2024-05-08T05:43:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.500+0000 7f3b5d16a640 1 -- 172.21.15.45:0/1781836810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b580cf320 con 0x7f3b440051c0 2024-05-08T05:43:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.501+0000 7f3b4f7fe640 1 -- 172.21.15.45:0/1781836810 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3b4801d070 con 0x7f3b440051c0 2024-05-08T05:43:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.501+0000 7f3b4f7fe640 1 --2- 172.21.15.45:0/1781836810 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3b38076300 0x7f3b380787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.501+0000 7f3b4f7fe640 1 -- 172.21.15.45:0/1781836810 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f3b48096b50 con 0x7f3b440051c0 2024-05-08T05:43:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.501+0000 7f3b4ffff640 1 --2- 172.21.15.45:0/1781836810 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3b38076300 0x7f3b380787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:49.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.502+0000 7f3b4ffff640 1 --2- 172.21.15.45:0/1781836810 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3b38076300 0x7f3b380787c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009980 tx=0x7f3b40008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:49.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.506+0000 7f3b4f7fe640 1 -- 172.21.15.45:0/1781836810 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3b580cf320 con 0x7f3b440051c0 2024-05-08T05:43:49.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.705+0000 7f3b5d16a640 1 -- 172.21.15.45:0/1781836810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f3b580693f0 con 0x7f3b440051c0 2024-05-08T05:43:49.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.708+0000 7f3b4f7fe640 1 -- 172.21.15.45:0/1781836810 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f3b480606b0 con 0x7f3b440051c0 2024-05-08T05:43:49.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.710+0000 7f3b5d16a640 1 -- 172.21.15.45:0/1781836810 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3b38076300 msgr2=0x7f3b380787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:49.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.710+0000 7f3b5d16a640 1 --2- 172.21.15.45:0/1781836810 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3b38076300 0x7f3b380787c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009980 tx=0x7f3b40008040 comp rx=0 tx=0).stop 2024-05-08T05:43:49.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.710+0000 7f3b5d16a640 1 -- 172.21.15.45:0/1781836810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b440051c0 msgr2=0x7f3b5815a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:49.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.710+0000 7f3b5d16a640 1 --2- 172.21.15.45:0/1781836810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b440051c0 0x7f3b5815a570 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800f850 tx=0x7f3b4800fd10 comp rx=0 tx=0).stop 2024-05-08T05:43:49.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.711+0000 7f3b5d16a640 1 -- 172.21.15.45:0/1781836810 shutdown_connections 2024-05-08T05:43:49.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.711+0000 7f3b5d16a640 1 --2- 172.21.15.45:0/1781836810 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3b38076300 0x7f3b380787c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.711+0000 7f3b5d16a640 1 --2- 172.21.15.45:0/1781836810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b580c8b00 0x7f3b580cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.711+0000 7f3b5d16a640 1 --2- 172.21.15.45:0/1781836810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b580b2dc0 0x7f3b580b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.711+0000 7f3b5d16a640 1 --2- 172.21.15.45:0/1781836810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b440051c0 0x7f3b5815a570 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.711+0000 7f3b5d16a640 1 -- 172.21.15.45:0/1781836810 >> 172.21.15.45:0/1781836810 conn(0x7f3b5808eac0 msgr2=0x7f3b5806e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:49.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.711+0000 7f3b5d16a640 1 -- 172.21.15.45:0/1781836810 shutdown_connections 2024-05-08T05:43:49.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.711+0000 7f3b5d16a640 1 -- 172.21.15.45:0/1781836810 wait complete. 2024-05-08T05:43:49.722 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-05-08T05:43:49.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.826+0000 7f6de2b04640 1 Processor -- start 2024-05-08T05:43:49.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.826+0000 7f6de2b04640 1 -- start start 2024-05-08T05:43:49.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.826+0000 7f6de2b04640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6ddc0d3cd0 0x7f6ddc0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:49.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.826+0000 7f6de2b04640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6ddc0c9ee0 0x7f6ddc0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:49.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.826+0000 7f6de2b04640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ddc0d3330 con 0x7f6ddc1629a0 2024-05-08T05:43:49.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.826+0000 7f6de2b04640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f6ddc0d2170 con 0x7f6ddc0c9ee0 2024-05-08T05:43:49.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.826+0000 7f6de2b04640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6ddc0d2450 con 0x7f6ddc0d3cd0 2024-05-08T05:43:49.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.827+0000 7f6de10a1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6ddc0c9ee0 0x7f6ddc0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:49.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.827+0000 7f6de10a1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6ddc0c9ee0 0x7f6ddc0d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59018/0 (socket says 172.21.15.45:59018) 2024-05-08T05:43:49.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.827+0000 7f6de10a1640 1 -- 172.21.15.45:0/486371080 learned_addr learned my addr 172.21.15.45:0/486371080 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:49.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.827+0000 7f6ddbfff640 1 --2- 172.21.15.45:0/486371080 >> v2:172.21.15.63:3300/0 conn(0x7f6ddc0d3cd0 0x7f6ddc0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:49.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.827+0000 7f6de10a1640 1 -- 172.21.15.45:0/486371080 >> v2:172.21.15.63:3300/0 conn(0x7f6ddc0d3cd0 msgr2=0x7f6ddc0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:49.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.827+0000 7f6de10a1640 1 --2- 172.21.15.45:0/486371080 >> v2:172.21.15.63:3300/0 conn(0x7f6ddc0d3cd0 0x7f6ddc0cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.827+0000 7f6de10a1640 1 -- 172.21.15.45:0/486371080 >> v1:172.21.15.63:6789/0 conn(0x7f6ddc1629a0 legacy=0x7f6ddc0d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:49.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.827+0000 7f6de10a1640 1 -- 172.21.15.45:0/486371080 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc0d2730 con 0x7f6ddc0c9ee0 2024-05-08T05:43:49.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.828+0000 7f6de10a1640 1 --2- 172.21.15.45:0/486371080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ddc0c9ee0 0x7f6ddc0d01e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0b7ae0 tx=0x7f6dc402a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e42c34d493f7a5f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:49.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.828+0000 7f6ddb7fe640 1 -- 172.21.15.45:0/486371080 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4037070 con 0x7f6ddc0c9ee0 2024-05-08T05:43:49.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.828+0000 7f6ddb7fe640 1 -- 172.21.15.45:0/486371080 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc4032440 con 0x7f6ddc0c9ee0 2024-05-08T05:43:49.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.828+0000 7f6ddb7fe640 1 -- 172.21.15.45:0/486371080 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4032810 con 0x7f6ddc0c9ee0 2024-05-08T05:43:49.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.828+0000 7f6de2b04640 1 -- 172.21.15.45:0/486371080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ddc0c9ee0 msgr2=0x7f6ddc0d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:49.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.828+0000 7f6de2b04640 1 --2- 172.21.15.45:0/486371080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ddc0c9ee0 0x7f6ddc0d01e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0b7ae0 tx=0x7f6dc402a150 comp rx=0 tx=0).stop 2024-05-08T05:43:49.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.828+0000 7f6de2b04640 1 -- 172.21.15.45:0/486371080 shutdown_connections 2024-05-08T05:43:49.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.828+0000 7f6de2b04640 1 --2- 172.21.15.45:0/486371080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ddc0c9ee0 0x7f6ddc0d01e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.828+0000 7f6de2b04640 1 --2- 172.21.15.45:0/486371080 >> v2:172.21.15.63:3300/0 conn(0x7f6ddc0d3cd0 0x7f6ddc0cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.828+0000 7f6de2b04640 1 -- 172.21.15.45:0/486371080 >> 172.21.15.45:0/486371080 conn(0x7f6ddc0d4fe0 msgr2=0x7f6ddc0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:49.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.828+0000 7f6de2b04640 1 -- 172.21.15.45:0/486371080 shutdown_connections 2024-05-08T05:43:49.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.828+0000 7f6de2b04640 1 -- 172.21.15.45:0/486371080 wait complete. 2024-05-08T05:43:49.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.830+0000 7f6de2b04640 1 Processor -- start 2024-05-08T05:43:49.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.830+0000 7f6de2b04640 1 -- start start 2024-05-08T05:43:49.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.830+0000 7f6de2b04640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ddc0c9ee0 0x7f6ddc0ba8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:49.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.830+0000 7f6de2b04640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ddc0d3cd0 0x7f6ddc09ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:49.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.830+0000 7f6de2b04640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ddc1629a0 0x7f6ddc1f8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:49.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.831+0000 7f6de2b04640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc0cf440 con 0x7f6ddc0d3cd0 2024-05-08T05:43:49.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.831+0000 7f6de2b04640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc0cf2c0 con 0x7f6ddc0c9ee0 2024-05-08T05:43:49.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.831+0000 7f6de2b04640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6ddc0d2dc0 con 0x7f6ddc1629a0 2024-05-08T05:43:49.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.831+0000 7f6ddbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ddc0d3cd0 0x7f6ddc09ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:49.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.831+0000 7f6ddbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ddc0d3cd0 0x7f6ddc09ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59020/0 (socket says 172.21.15.45:59020) 2024-05-08T05:43:49.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.831+0000 7f6ddbfff640 1 -- 172.21.15.45:0/2310937972 learned_addr learned my addr 172.21.15.45:0/2310937972 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:49.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.831+0000 7f6de08a0640 1 --2- 172.21.15.45:0/2310937972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ddc0c9ee0 0x7f6ddc0ba8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:49.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.831+0000 7f6de10a1640 1 --2- 172.21.15.45:0/2310937972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ddc1629a0 0x7f6ddc1f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:49.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.831+0000 7f6ddbfff640 1 -- 172.21.15.45:0/2310937972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ddc1629a0 msgr2=0x7f6ddc1f8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:49.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.831+0000 7f6ddbfff640 1 --2- 172.21.15.45:0/2310937972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ddc1629a0 0x7f6ddc1f8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.831+0000 7f6ddbfff640 1 -- 172.21.15.45:0/2310937972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ddc0c9ee0 msgr2=0x7f6ddc0ba8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:49.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.831+0000 7f6ddbfff640 1 --2- 172.21.15.45:0/2310937972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ddc0c9ee0 0x7f6ddc0ba8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:49.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.831+0000 7f6ddbfff640 1 -- 172.21.15.45:0/2310937972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc0a1980 con 0x7f6ddc0d3cd0 2024-05-08T05:43:49.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.831+0000 7f6de10a1640 1 --2- 172.21.15.45:0/2310937972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ddc1629a0 0x7f6ddc1f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.831+0000 7f6ddbfff640 1 --2- 172.21.15.45:0/2310937972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ddc0d3cd0 0x7f6ddc09ed90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc007ec0 tx=0x7f6dcc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.831+0000 7f6dd97fa640 1 -- 172.21.15.45:0/2310937972 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc014620 con 0x7f6ddc0d3cd0 2024-05-08T05:43:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.831+0000 7f6dd97fa640 1 -- 172.21.15.45:0/2310937972 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dcc019070 con 0x7f6ddc0d3cd0 2024-05-08T05:43:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.831+0000 7f6dd97fa640 1 -- 172.21.15.45:0/2310937972 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc014a20 con 0x7f6ddc0d3cd0 2024-05-08T05:43:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.831+0000 7f6de2b04640 1 -- 172.21.15.45:0/2310937972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ddc0a1c60 con 0x7f6ddc0d3cd0 2024-05-08T05:43:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.831+0000 7f6de2b04640 1 -- 172.21.15.45:0/2310937972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ddc0bae00 con 0x7f6ddc0d3cd0 2024-05-08T05:43:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.832+0000 7f6de2b04640 1 -- 172.21.15.45:0/2310937972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ddc0693f0 con 0x7f6ddc0d3cd0 2024-05-08T05:43:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.833+0000 7f6dd97fa640 1 -- 172.21.15.45:0/2310937972 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6dcc014bc0 con 0x7f6ddc0d3cd0 2024-05-08T05:43:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.833+0000 7f6dd97fa640 1 --2- 172.21.15.45:0/2310937972 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6dbc0762d0 0x7f6dbc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.834+0000 7f6dd97fa640 1 -- 172.21.15.45:0/2310937972 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f6dcc097d30 con 0x7f6ddc0d3cd0 2024-05-08T05:43:49.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.834+0000 7f6de08a0640 1 --2- 172.21.15.45:0/2310937972 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6dbc0762d0 0x7f6dbc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:49.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.834+0000 7f6de08a0640 1 --2- 172.21.15.45:0/2310937972 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6dbc0762d0 0x7f6dbc078790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0a2e30 tx=0x7f6dd4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:49.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:49.838+0000 7f6dd97fa640 1 -- 172.21.15.45:0/2310937972 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6dcc0617f0 con 0x7f6ddc0d3cd0 2024-05-08T05:43:50.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.109+0000 7f6de2b04640 1 -- 172.21.15.45:0/2310937972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f6ddc0cd100 con 0x7f6ddc0d3cd0 2024-05-08T05:43:50.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.360+0000 7f6dd97fa640 1 -- 172.21.15.45:0/2310937972 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v135) ==== 149+0+0 (secure 0 0 0) 0x7f6dcc0610f0 con 0x7f6ddc0d3cd0 2024-05-08T05:43:50.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.363+0000 7f6de2b04640 1 -- 172.21.15.45:0/2310937972 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6dbc0762d0 msgr2=0x7f6dbc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:50.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.363+0000 7f6de2b04640 1 --2- 172.21.15.45:0/2310937972 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6dbc0762d0 0x7f6dbc078790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0a2e30 tx=0x7f6dd4009040 comp rx=0 tx=0).stop 2024-05-08T05:43:50.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.363+0000 7f6de2b04640 1 -- 172.21.15.45:0/2310937972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ddc0d3cd0 msgr2=0x7f6ddc09ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:50.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.363+0000 7f6de2b04640 1 --2- 172.21.15.45:0/2310937972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ddc0d3cd0 0x7f6ddc09ed90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc007ec0 tx=0x7f6dcc00e4b0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.363+0000 7f6de2b04640 1 -- 172.21.15.45:0/2310937972 shutdown_connections 2024-05-08T05:43:50.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.363+0000 7f6de2b04640 1 --2- 172.21.15.45:0/2310937972 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6dbc0762d0 0x7f6dbc078790 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.363+0000 7f6de2b04640 1 --2- 172.21.15.45:0/2310937972 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ddc1629a0 0x7f6ddc1f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.363+0000 7f6de2b04640 1 --2- 172.21.15.45:0/2310937972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ddc0d3cd0 0x7f6ddc09ed90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.363+0000 7f6de2b04640 1 --2- 172.21.15.45:0/2310937972 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ddc0c9ee0 0x7f6ddc0ba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.363+0000 7f6de2b04640 1 -- 172.21.15.45:0/2310937972 >> 172.21.15.45:0/2310937972 conn(0x7f6ddc0d4fe0 msgr2=0x7f6ddc0d08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:50.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.363+0000 7f6de2b04640 1 -- 172.21.15.45:0/2310937972 shutdown_connections 2024-05-08T05:43:50.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.363+0000 7f6de2b04640 1 -- 172.21.15.45:0/2310937972 wait complete. 2024-05-08T05:43:50.374 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-05-08T05:43:50.375 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-08T05:43:50.375 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-08T05:43:50.375 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:43:50.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.479+0000 7f518fbc1640 1 Processor -- start 2024-05-08T05:43:50.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.479+0000 7f518fbc1640 1 -- start start 2024-05-08T05:43:50.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.480+0000 7f518fbc1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51881629a0 0x7f51880d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:50.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.480+0000 7f518fbc1640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f51880d2dc0 0x7f51880d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:50.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.480+0000 7f518fbc1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f51880c8ed0 0x7f51880c9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:50.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.480+0000 7f518fbc1640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f51880d3330 con 0x7f51880c8ed0 2024-05-08T05:43:50.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.480+0000 7f518fbc1640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f51880d2170 con 0x7f51880d2dc0 2024-05-08T05:43:50.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.480+0000 7f518fbc1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f51880d2450 con 0x7f51881629a0 2024-05-08T05:43:50.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.480+0000 7f518e15e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f51880c8ed0 0x7f51880c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:50.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.480+0000 7f518e15e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f51880c8ed0 0x7f51880c9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59026/0 (socket says 172.21.15.45:59026) 2024-05-08T05:43:50.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.480+0000 7f518e15e640 1 -- 172.21.15.45:0/2919483933 learned_addr learned my addr 172.21.15.45:0/2919483933 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:50.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.480+0000 7f518d15c640 1 --2- 172.21.15.45:0/2919483933 >> v2:172.21.15.63:3300/0 conn(0x7f51880d2dc0 0x7f51880d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:50.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.480+0000 7f518d95d640 1 --2- 172.21.15.45:0/2919483933 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51881629a0 0x7f51880d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:50.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.480+0000 7f518e15e640 1 -- 172.21.15.45:0/2919483933 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51881629a0 msgr2=0x7f51880d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:50.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.480+0000 7f518e15e640 1 --2- 172.21.15.45:0/2919483933 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51881629a0 0x7f51880d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.480+0000 7f518e15e640 1 -- 172.21.15.45:0/2919483933 >> v2:172.21.15.63:3300/0 conn(0x7f51880d2dc0 msgr2=0x7f51880d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:50.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.480+0000 7f518e15e640 1 --2- 172.21.15.45:0/2919483933 >> v2:172.21.15.63:3300/0 conn(0x7f51880d2dc0 0x7f51880d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.480+0000 7f518e15e640 1 -- 172.21.15.45:0/2919483933 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51880d2730 con 0x7f51880c8ed0 2024-05-08T05:43:50.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.480+0000 7f518d15c640 1 --2- 172.21.15.45:0/2919483933 >> v2:172.21.15.63:3300/0 conn(0x7f51880d2dc0 0x7f51880d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:50.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.481+0000 7f518e15e640 1 --2- 172.21.15.45:0/2919483933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51880c8ed0 0x7f51880c9360 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f518808da00 tx=0x7f518002a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6fb1582c0a9e63 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.481+0000 7f518c95b640 1 -- 172.21.15.45:0/2919483933 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180036070 con 0x7f51880c8ed0 2024-05-08T05:43:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.481+0000 7f518c95b640 1 -- 172.21.15.45:0/2919483933 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5180031600 con 0x7f51880c8ed0 2024-05-08T05:43:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.481+0000 7f518c95b640 1 -- 172.21.15.45:0/2919483933 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51800319d0 con 0x7f51880c8ed0 2024-05-08T05:43:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.482+0000 7f518fbc1640 1 -- 172.21.15.45:0/2919483933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51880c8ed0 msgr2=0x7f51880c9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.482+0000 7f518fbc1640 1 --2- 172.21.15.45:0/2919483933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51880c8ed0 0x7f51880c9360 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f518808da00 tx=0x7f518002a570 comp rx=0 tx=0).stop 2024-05-08T05:43:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.482+0000 7f518fbc1640 1 -- 172.21.15.45:0/2919483933 shutdown_connections 2024-05-08T05:43:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.482+0000 7f518fbc1640 1 --2- 172.21.15.45:0/2919483933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51880c8ed0 0x7f51880c9360 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.482+0000 7f518fbc1640 1 --2- 172.21.15.45:0/2919483933 >> v2:172.21.15.63:3300/0 conn(0x7f51880d2dc0 0x7f51880d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.482+0000 7f518fbc1640 1 --2- 172.21.15.45:0/2919483933 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51881629a0 0x7f51880d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.482+0000 7f518fbc1640 1 -- 172.21.15.45:0/2919483933 >> 172.21.15.45:0/2919483933 conn(0x7f51880d4fe0 msgr2=0x7f51880d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.482+0000 7f518fbc1640 1 -- 172.21.15.45:0/2919483933 shutdown_connections 2024-05-08T05:43:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.482+0000 7f518fbc1640 1 -- 172.21.15.45:0/2919483933 wait complete. 2024-05-08T05:43:50.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.483+0000 7f518fbc1640 1 Processor -- start 2024-05-08T05:43:50.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.483+0000 7f518fbc1640 1 -- start start 2024-05-08T05:43:50.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.484+0000 7f518fbc1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51880c8ed0 0x7f518815f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:50.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.484+0000 7f518fbc1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51880d2dc0 0x7f51880b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:50.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.484+0000 7f518fbc1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51881629a0 0x7f51880ce350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:50.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.484+0000 7f518fbc1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f51880cf440 con 0x7f51880c8ed0 2024-05-08T05:43:50.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.484+0000 7f518fbc1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f51880cf2c0 con 0x7f51881629a0 2024-05-08T05:43:50.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.484+0000 7f518fbc1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f51880c98a0 con 0x7f51880d2dc0 2024-05-08T05:43:50.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.484+0000 7f518d95d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51880c8ed0 0x7f518815f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.484+0000 7f518d95d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51880c8ed0 0x7f518815f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59030/0 (socket says 172.21.15.45:59030) 2024-05-08T05:43:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.484+0000 7f518d95d640 1 -- 172.21.15.45:0/1129639226 learned_addr learned my addr 172.21.15.45:0/1129639226 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.484+0000 7f518e15e640 1 --2- 172.21.15.45:0/1129639226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51881629a0 0x7f51880ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.484+0000 7f518d15c640 1 --2- 172.21.15.45:0/1129639226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51880d2dc0 0x7f51880b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.484+0000 7f518d95d640 1 -- 172.21.15.45:0/1129639226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51880d2dc0 msgr2=0x7f51880b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.484+0000 7f518d95d640 1 --2- 172.21.15.45:0/1129639226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51880d2dc0 0x7f51880b2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.484+0000 7f518d95d640 1 -- 172.21.15.45:0/1129639226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51881629a0 msgr2=0x7f51880ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.484+0000 7f518d95d640 1 --2- 172.21.15.45:0/1129639226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51881629a0 0x7f51880ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.484+0000 7f518d95d640 1 -- 172.21.15.45:0/1129639226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51880b51a0 con 0x7f51880c8ed0 2024-05-08T05:43:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.484+0000 7f518e15e640 1 --2- 172.21.15.45:0/1129639226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51881629a0 0x7f51880ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.485+0000 7f518d95d640 1 --2- 172.21.15.45:0/1129639226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51880c8ed0 0x7f518815f030 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f518400c610 tx=0x7f518400cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.485+0000 7f517e7fc640 1 -- 172.21.15.45:0/1129639226 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184015070 con 0x7f51880c8ed0 2024-05-08T05:43:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.485+0000 7f517e7fc640 1 -- 172.21.15.45:0/1129639226 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5184004220 con 0x7f51880c8ed0 2024-05-08T05:43:50.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.485+0000 7f518fbc1640 1 -- 172.21.15.45:0/1129639226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51880b5480 con 0x7f51880c8ed0 2024-05-08T05:43:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.485+0000 7f517e7fc640 1 -- 172.21.15.45:0/1129639226 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184003610 con 0x7f51880c8ed0 2024-05-08T05:43:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.485+0000 7f518fbc1640 1 -- 172.21.15.45:0/1129639226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51881f61b0 con 0x7f51880c8ed0 2024-05-08T05:43:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.486+0000 7f518fbc1640 1 -- 172.21.15.45:0/1129639226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51880693f0 con 0x7f51880c8ed0 2024-05-08T05:43:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.486+0000 7f517e7fc640 1 -- 172.21.15.45:0/1129639226 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f51840037d0 con 0x7f51880c8ed0 2024-05-08T05:43:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.487+0000 7f517e7fc640 1 --2- 172.21.15.45:0/1129639226 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f516c0762d0 0x7f516c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.487+0000 7f517e7fc640 1 -- 172.21.15.45:0/1129639226 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f5184097280 con 0x7f51880c8ed0 2024-05-08T05:43:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.487+0000 7f518d15c640 1 --2- 172.21.15.45:0/1129639226 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f516c0762d0 0x7f516c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:50.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.488+0000 7f518d15c640 1 --2- 172.21.15.45:0/1129639226 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f516c0762d0 0x7f516c078790 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f51880d4840 tx=0x7f5178008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:50.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.491+0000 7f517e7fc640 1 -- 172.21.15.45:0/1129639226 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5184060d50 con 0x7f51880c8ed0 2024-05-08T05:43:50.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.682+0000 7f518fbc1640 1 -- 172.21.15.45:0/1129639226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51880d4590 con 0x7f51880c8ed0 2024-05-08T05:43:50.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.682+0000 7f517e7fc640 1 -- 172.21.15.45:0/1129639226 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+13245 (secure 0 0 0) 0x7f5184060650 con 0x7f51880c8ed0 2024-05-08T05:43:50.683 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:50.683 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":52,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:43:49.356015+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6836/4205418057":"2024-05-09T05:43:46.347232+0000","172.21.15.63:6834/4205418057":"2024-05-09T05:43:46.347232+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:43:50.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.684+0000 7f518fbc1640 1 -- 172.21.15.45:0/1129639226 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f516c0762d0 msgr2=0x7f516c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:50.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.684+0000 7f518fbc1640 1 --2- 172.21.15.45:0/1129639226 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f516c0762d0 0x7f516c078790 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f51880d4840 tx=0x7f5178008040 comp rx=0 tx=0).stop 2024-05-08T05:43:50.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.684+0000 7f518fbc1640 1 -- 172.21.15.45:0/1129639226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51880c8ed0 msgr2=0x7f518815f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:50.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.684+0000 7f518fbc1640 1 --2- 172.21.15.45:0/1129639226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51880c8ed0 0x7f518815f030 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f518400c610 tx=0x7f518400cad0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.684+0000 7f518fbc1640 1 -- 172.21.15.45:0/1129639226 shutdown_connections 2024-05-08T05:43:50.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.684+0000 7f518fbc1640 1 --2- 172.21.15.45:0/1129639226 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f516c0762d0 0x7f516c078790 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.684+0000 7f518fbc1640 1 --2- 172.21.15.45:0/1129639226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f51880d2dc0 0x7f51880b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.684+0000 7f518fbc1640 1 --2- 172.21.15.45:0/1129639226 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f51881629a0 0x7f51880ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.684+0000 7f518fbc1640 1 --2- 172.21.15.45:0/1129639226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51880c8ed0 0x7f518815f030 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.684+0000 7f518fbc1640 1 -- 172.21.15.45:0/1129639226 >> 172.21.15.45:0/1129639226 conn(0x7f51880d4fe0 msgr2=0x7f51880cb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:50.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.684+0000 7f518fbc1640 1 -- 172.21.15.45:0/1129639226 shutdown_connections 2024-05-08T05:43:50.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.684+0000 7f518fbc1640 1 -- 172.21.15.45:0/1129639226 wait complete. 2024-05-08T05:43:50.696 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T05:41:18.847604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T05:43:50.696 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T05:43:50.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.803+0000 7f078aa75640 1 Processor -- start 2024-05-08T05:43:50.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.803+0000 7f078aa75640 1 -- start start 2024-05-08T05:43:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.803+0000 7f078aa75640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07840cfcd0 0x7f07840d4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.803+0000 7f078aa75640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f07840d54e0 0x7f07840d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:50.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.803+0000 7f078aa75640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07840d0320 con 0x7f078415f4e0 2024-05-08T05:43:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.803+0000 7f078aa75640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f07840d41b0 con 0x7f07840d54e0 2024-05-08T05:43:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.803+0000 7f078aa75640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f07840d0600 con 0x7f07840cfcd0 2024-05-08T05:43:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.804+0000 7f0789012640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f07840d54e0 0x7f07840d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.804+0000 7f0789012640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f07840d54e0 0x7f07840d13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55548/0 (socket says 172.21.15.45:55548) 2024-05-08T05:43:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.804+0000 7f0789012640 1 -- 172.21.15.45:0/1092953422 learned_addr learned my addr 172.21.15.45:0/1092953422 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.804+0000 7f0783fff640 1 --2- 172.21.15.45:0/1092953422 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07840cfcd0 0x7f07840d4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.804+0000 7f0789012640 1 -- 172.21.15.45:0/1092953422 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07840cfcd0 msgr2=0x7f07840d4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.804+0000 7f0789012640 1 --2- 172.21.15.45:0/1092953422 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07840cfcd0 0x7f07840d4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.804+0000 7f0789012640 1 -- 172.21.15.45:0/1092953422 >> v1:172.21.15.63:6789/0 conn(0x7f078415f4e0 legacy=0x7f07840d3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:50.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.804+0000 7f0789012640 1 -- 172.21.15.45:0/1092953422 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07840d08e0 con 0x7f07840d54e0 2024-05-08T05:43:50.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.805+0000 7f0789012640 1 --2- 172.21.15.45:0/1092953422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07840d54e0 0x7f07840d13c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f077c009e50 tx=0x7f077c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472e3f773dd98d02 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:50.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.805+0000 7f07837fe640 1 -- 172.21.15.45:0/1092953422 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c036070 con 0x7f07840d54e0 2024-05-08T05:43:50.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.805+0000 7f07837fe640 1 -- 172.21.15.45:0/1092953422 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f077c031600 con 0x7f07840d54e0 2024-05-08T05:43:50.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.805+0000 7f07837fe640 1 -- 172.21.15.45:0/1092953422 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c0319d0 con 0x7f07840d54e0 2024-05-08T05:43:50.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.805+0000 7f078aa75640 1 -- 172.21.15.45:0/1092953422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07840d54e0 msgr2=0x7f07840d13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:50.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.805+0000 7f078aa75640 1 --2- 172.21.15.45:0/1092953422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07840d54e0 0x7f07840d13c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f077c009e50 tx=0x7f077c02a710 comp rx=0 tx=0).stop 2024-05-08T05:43:50.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.805+0000 7f078aa75640 1 -- 172.21.15.45:0/1092953422 shutdown_connections 2024-05-08T05:43:50.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.805+0000 7f078aa75640 1 --2- 172.21.15.45:0/1092953422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07840d54e0 0x7f07840d13c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.805+0000 7f078aa75640 1 --2- 172.21.15.45:0/1092953422 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07840cfcd0 0x7f07840d4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.805+0000 7f078aa75640 1 -- 172.21.15.45:0/1092953422 >> 172.21.15.45:0/1092953422 conn(0x7f078408eac0 msgr2=0x7f078409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:50.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.805+0000 7f078aa75640 1 -- 172.21.15.45:0/1092953422 shutdown_connections 2024-05-08T05:43:50.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.805+0000 7f078aa75640 1 -- 172.21.15.45:0/1092953422 wait complete. 2024-05-08T05:43:50.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.807+0000 7f078aa75640 1 Processor -- start 2024-05-08T05:43:50.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.807+0000 7f078aa75640 1 -- start start 2024-05-08T05:43:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.807+0000 7f078aa75640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07840cfcd0 0x7f07840cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.807+0000 7f078aa75640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07840d54e0 0x7f07840ba970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.807+0000 7f078aa75640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f078415f4e0 0x7f07840ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.807+0000 7f078aa75640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f078415f8c0 con 0x7f07840cfcd0 2024-05-08T05:43:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.807+0000 7f078aa75640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f07840d0ea0 con 0x7f07840d54e0 2024-05-08T05:43:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.807+0000 7f078aa75640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f07840d1900 con 0x7f078415f4e0 2024-05-08T05:43:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.808+0000 7f0788811640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07840cfcd0 0x7f07840cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.808+0000 7f0788811640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07840cfcd0 0x7f07840cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55550/0 (socket says 172.21.15.45:55550) 2024-05-08T05:43:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.808+0000 7f0788811640 1 -- 172.21.15.45:0/564392159 learned_addr learned my addr 172.21.15.45:0/564392159 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.808+0000 7f0783fff640 1 --2- 172.21.15.45:0/564392159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07840d54e0 0x7f07840ba970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:50.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.808+0000 7f0789012640 1 --2- 172.21.15.45:0/564392159 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f078415f4e0 0x7f07840ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.808+0000 7f0788811640 1 -- 172.21.15.45:0/564392159 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f078415f4e0 msgr2=0x7f07840ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.808+0000 7f0788811640 1 --2- 172.21.15.45:0/564392159 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f078415f4e0 0x7f07840ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.808+0000 7f0788811640 1 -- 172.21.15.45:0/564392159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07840d54e0 msgr2=0x7f07840ba970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.808+0000 7f0788811640 1 --2- 172.21.15.45:0/564392159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07840d54e0 0x7f07840ba970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.808+0000 7f0788811640 1 -- 172.21.15.45:0/564392159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07840baeb0 con 0x7f07840cfcd0 2024-05-08T05:43:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.808+0000 7f0788811640 1 --2- 172.21.15.45:0/564392159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07840cfcd0 0x7f07840cc0e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f077800c3a0 tx=0x7f077800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.808+0000 7f07817fa640 1 -- 172.21.15.45:0/564392159 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0778017070 con 0x7f07840cfcd0 2024-05-08T05:43:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.808+0000 7f07817fa640 1 -- 172.21.15.45:0/564392159 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07780052d0 con 0x7f07840cfcd0 2024-05-08T05:43:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.808+0000 7f078aa75640 1 -- 172.21.15.45:0/564392159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07840cc770 con 0x7f07840cfcd0 2024-05-08T05:43:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.808+0000 7f07817fa640 1 -- 172.21.15.45:0/564392159 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0778003830 con 0x7f07840cfcd0 2024-05-08T05:43:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.808+0000 7f078aa75640 1 -- 172.21.15.45:0/564392159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07840ccd30 con 0x7f07840cfcd0 2024-05-08T05:43:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.809+0000 7f078aa75640 1 -- 172.21.15.45:0/564392159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07840693f0 con 0x7f07840cfcd0 2024-05-08T05:43:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.810+0000 7f07817fa640 1 -- 172.21.15.45:0/564392159 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f077801a070 con 0x7f07840cfcd0 2024-05-08T05:43:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.810+0000 7f07817fa640 1 --2- 172.21.15.45:0/564392159 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07680762d0 0x7f0768078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.811+0000 7f07817fa640 1 -- 172.21.15.45:0/564392159 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f07780966b0 con 0x7f07840cfcd0 2024-05-08T05:43:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.811+0000 7f0783fff640 1 --2- 172.21.15.45:0/564392159 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07680762d0 0x7f0768078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:50.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.811+0000 7f0783fff640 1 --2- 172.21.15.45:0/564392159 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07680762d0 0x7f0768078790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f07840cf7c0 tx=0x7f0770009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:50.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:50.815+0000 7f07817fa640 1 -- 172.21.15.45:0/564392159 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f077805fbc0 con 0x7f07840cfcd0 2024-05-08T05:43:51.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.007+0000 7f078aa75640 1 -- 172.21.15.45:0/564392159 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f07840d12c0 con 0x7f07840cfcd0 2024-05-08T05:43:51.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.007+0000 7f07817fa640 1 -- 172.21.15.45:0/564392159 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) ==== 93+0+10 (secure 0 0 0) 0x7f0778068020 con 0x7f07840cfcd0 2024-05-08T05:43:51.008 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-05-08T05:43:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.009+0000 7f078aa75640 1 -- 172.21.15.45:0/564392159 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07680762d0 msgr2=0x7f0768078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.009+0000 7f078aa75640 1 --2- 172.21.15.45:0/564392159 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07680762d0 0x7f0768078790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f07840cf7c0 tx=0x7f0770009190 comp rx=0 tx=0).stop 2024-05-08T05:43:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.009+0000 7f078aa75640 1 -- 172.21.15.45:0/564392159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07840cfcd0 msgr2=0x7f07840cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.009+0000 7f078aa75640 1 --2- 172.21.15.45:0/564392159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07840cfcd0 0x7f07840cc0e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f077800c3a0 tx=0x7f077800c860 comp rx=0 tx=0).stop 2024-05-08T05:43:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.009+0000 7f078aa75640 1 -- 172.21.15.45:0/564392159 shutdown_connections 2024-05-08T05:43:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.009+0000 7f078aa75640 1 --2- 172.21.15.45:0/564392159 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07680762d0 0x7f0768078790 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.009+0000 7f078aa75640 1 --2- 172.21.15.45:0/564392159 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f078415f4e0 0x7f07840ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.009+0000 7f078aa75640 1 --2- 172.21.15.45:0/564392159 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07840d54e0 0x7f07840ba970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.009+0000 7f078aa75640 1 --2- 172.21.15.45:0/564392159 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07840cfcd0 0x7f07840cc0e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.009+0000 7f078aa75640 1 -- 172.21.15.45:0/564392159 >> 172.21.15.45:0/564392159 conn(0x7f078408eac0 msgr2=0x7f07840b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:51.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.009+0000 7f078aa75640 1 -- 172.21.15.45:0/564392159 shutdown_connections 2024-05-08T05:43:51.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.009+0000 7f078aa75640 1 -- 172.21.15.45:0/564392159 wait complete. 2024-05-08T05:43:51.021 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-05-08T05:43:51.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.124+0000 7f079dddd640 1 Processor -- start 2024-05-08T05:43:51.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.124+0000 7f079dddd640 1 -- start start 2024-05-08T05:43:51.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.125+0000 7f079dddd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f079815ce30 0x7f079806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:51.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.125+0000 7f079dddd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0798161630 0x7f07980ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:51.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.125+0000 7f079dddd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07980715b0 con 0x7f079806c3a0 2024-05-08T05:43:51.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.125+0000 7f079dddd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f07980d0840 con 0x7f0798161630 2024-05-08T05:43:51.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.125+0000 7f079dddd640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0798071890 con 0x7f079815ce30 2024-05-08T05:43:51.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.125+0000 7f0796ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0798161630 0x7f07980ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:51.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.126+0000 7f0797fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f079806c3a0 0x7f07980d0cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44436/0 (socket says 172.21.15.45:44436) 2024-05-08T05:43:51.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.126+0000 7f0797fff640 1 -- 172.21.15.45:0/4040664281 learned_addr learned my addr 172.21.15.45:0/4040664281 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:51.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.126+0000 7f07977fe640 1 --2- 172.21.15.45:0/4040664281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f079815ce30 0x7f079806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:51.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.126+0000 7f0796ffd640 1 -- 172.21.15.45:0/4040664281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f079815ce30 msgr2=0x7f079806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:51.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.126+0000 7f0796ffd640 1 --2- 172.21.15.45:0/4040664281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f079815ce30 0x7f079806f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:51.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.126+0000 7f0796ffd640 1 -- 172.21.15.45:0/4040664281 >> v1:172.21.15.45:6789/0 conn(0x7f079806c3a0 legacy=0x7f07980d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:51.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.126+0000 7f07967fc640 1 -- 172.21.15.45:0/4040664281 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 536406822 0 0) 0x7f07980715b0 con 0x7f079806c3a0 2024-05-08T05:43:51.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.126+0000 7f0796ffd640 1 -- 172.21.15.45:0/4040664281 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0798071b70 con 0x7f0798161630 2024-05-08T05:43:51.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.127+0000 7f0796ffd640 1 --2- 172.21.15.45:0/4040664281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0798161630 0x7f07980ce920 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f078c0097a0 tx=0x7f078c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562884ff23c64daf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:51.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.127+0000 7f07967fc640 1 -- 172.21.15.45:0/4040664281 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c037070 con 0x7f0798161630 2024-05-08T05:43:51.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.127+0000 7f07967fc640 1 -- 172.21.15.45:0/4040664281 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f078c032440 con 0x7f0798161630 2024-05-08T05:43:51.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.127+0000 7f07967fc640 1 -- 172.21.15.45:0/4040664281 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c032810 con 0x7f0798161630 2024-05-08T05:43:51.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.127+0000 7f079dddd640 1 -- 172.21.15.45:0/4040664281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0798161630 msgr2=0x7f07980ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:51.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.127+0000 7f079dddd640 1 --2- 172.21.15.45:0/4040664281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0798161630 0x7f07980ce920 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f078c0097a0 tx=0x7f078c02a180 comp rx=0 tx=0).stop 2024-05-08T05:43:51.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.127+0000 7f079dddd640 1 -- 172.21.15.45:0/4040664281 shutdown_connections 2024-05-08T05:43:51.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.127+0000 7f079dddd640 1 --2- 172.21.15.45:0/4040664281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0798161630 0x7f07980ce920 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:51.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.127+0000 7f079dddd640 1 --2- 172.21.15.45:0/4040664281 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f079815ce30 0x7f079806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:51.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.127+0000 7f079dddd640 1 -- 172.21.15.45:0/4040664281 >> 172.21.15.45:0/4040664281 conn(0x7f07980b3c40 msgr2=0x7f07980b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:51.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.127+0000 7f079dddd640 1 -- 172.21.15.45:0/4040664281 shutdown_connections 2024-05-08T05:43:51.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.127+0000 7f079dddd640 1 -- 172.21.15.45:0/4040664281 wait complete. 2024-05-08T05:43:51.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.129+0000 7f079dddd640 1 Processor -- start 2024-05-08T05:43:51.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.129+0000 7f079dddd640 1 -- start start 2024-05-08T05:43:51.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.129+0000 7f079dddd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079806c3a0 0x7f079815c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:51.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.129+0000 7f079dddd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f079815ce30 0x7f07981f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:51.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.129+0000 7f079dddd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0798161630 0x7f0798149ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:51.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.129+0000 7f079dddd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f07980cee60 con 0x7f079806c3a0 2024-05-08T05:43:51.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.129+0000 7f079dddd640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f07980d0460 con 0x7f0798161630 2024-05-08T05:43:51.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.129+0000 7f079dddd640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f07980cefe0 con 0x7f079815ce30 2024-05-08T05:43:51.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.130+0000 7f07977fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079806c3a0 0x7f079815c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:51.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.130+0000 7f07977fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079806c3a0 0x7f079815c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55576/0 (socket says 172.21.15.45:55576) 2024-05-08T05:43:51.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.130+0000 7f07977fe640 1 -- 172.21.15.45:0/1942497718 learned_addr learned my addr 172.21.15.45:0/1942497718 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:51.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.130+0000 7f0796ffd640 1 --2- 172.21.15.45:0/1942497718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f079815ce30 0x7f07981f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:51.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.130+0000 7f0797fff640 1 --2- 172.21.15.45:0/1942497718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0798161630 0x7f0798149ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:51.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.130+0000 7f07977fe640 1 -- 172.21.15.45:0/1942497718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f079815ce30 msgr2=0x7f07981f4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:51.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.130+0000 7f07977fe640 1 --2- 172.21.15.45:0/1942497718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f079815ce30 0x7f07981f4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:51.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.130+0000 7f07977fe640 1 -- 172.21.15.45:0/1942497718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0798161630 msgr2=0x7f0798149ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:51.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.130+0000 7f07977fe640 1 --2- 172.21.15.45:0/1942497718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0798161630 0x7f0798149ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:51.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.130+0000 7f07977fe640 1 -- 172.21.15.45:0/1942497718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0798159620 con 0x7f079806c3a0 2024-05-08T05:43:51.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.130+0000 7f0797fff640 1 --2- 172.21.15.45:0/1942497718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0798161630 0x7f0798149ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:51.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.130+0000 7f07977fe640 1 --2- 172.21.15.45:0/1942497718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079806c3a0 0x7f079815c110 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f078800e960 tx=0x7f078800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:51.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.130+0000 7f077ffff640 1 -- 172.21.15.45:0/1942497718 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0788017070 con 0x7f079806c3a0 2024-05-08T05:43:51.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.130+0000 7f077ffff640 1 -- 172.21.15.45:0/1942497718 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f078800ca00 con 0x7f079806c3a0 2024-05-08T05:43:51.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.130+0000 7f079dddd640 1 -- 172.21.15.45:0/1942497718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0798159340 con 0x7f079806c3a0 2024-05-08T05:43:51.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.130+0000 7f077ffff640 1 -- 172.21.15.45:0/1942497718 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078800cdf0 con 0x7f079806c3a0 2024-05-08T05:43:51.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.130+0000 7f079dddd640 1 -- 172.21.15.45:0/1942497718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0798159cf0 con 0x7f079806c3a0 2024-05-08T05:43:51.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.131+0000 7f079dddd640 1 -- 172.21.15.45:0/1942497718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0798069b00 con 0x7f079806c3a0 2024-05-08T05:43:51.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.132+0000 7f077ffff640 1 -- 172.21.15.45:0/1942497718 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f078801a070 con 0x7f079806c3a0 2024-05-08T05:43:51.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.132+0000 7f077ffff640 1 --2- 172.21.15.45:0/1942497718 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07840762d0 0x7f0784078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:51.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.133+0000 7f077ffff640 1 -- 172.21.15.45:0/1942497718 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f0788097530 con 0x7f079806c3a0 2024-05-08T05:43:51.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.133+0000 7f0796ffd640 1 --2- 172.21.15.45:0/1942497718 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07840762d0 0x7f0784078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:51.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.133+0000 7f0796ffd640 1 --2- 172.21.15.45:0/1942497718 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07840762d0 0x7f0784078790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f078c034000 tx=0x7f078c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:51.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.137+0000 7f077ffff640 1 -- 172.21.15.45:0/1942497718 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0788060ff0 con 0x7f079806c3a0 2024-05-08T05:43:51.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:51.402+0000 7f079dddd640 1 -- 172.21.15.45:0/1942497718 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f07980b59f0 con 0x7f079806c3a0 2024-05-08T05:43:52.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.366+0000 7f077ffff640 1 -- 172.21.15.45:0/1942497718 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v137) ==== 172+0+0 (secure 0 0 0) 0x7f07880608f0 con 0x7f079806c3a0 2024-05-08T05:43:52.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.369+0000 7f079dddd640 1 -- 172.21.15.45:0/1942497718 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07840762d0 msgr2=0x7f0784078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:52.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.369+0000 7f079dddd640 1 --2- 172.21.15.45:0/1942497718 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07840762d0 0x7f0784078790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f078c034000 tx=0x7f078c02e000 comp rx=0 tx=0).stop 2024-05-08T05:43:52.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.369+0000 7f079dddd640 1 -- 172.21.15.45:0/1942497718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079806c3a0 msgr2=0x7f079815c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:52.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.369+0000 7f079dddd640 1 --2- 172.21.15.45:0/1942497718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079806c3a0 0x7f079815c110 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f078800e960 tx=0x7f078800ee20 comp rx=0 tx=0).stop 2024-05-08T05:43:52.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.369+0000 7f079dddd640 1 -- 172.21.15.45:0/1942497718 shutdown_connections 2024-05-08T05:43:52.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.369+0000 7f079dddd640 1 --2- 172.21.15.45:0/1942497718 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07840762d0 0x7f0784078790 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:52.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.369+0000 7f079dddd640 1 --2- 172.21.15.45:0/1942497718 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f079815ce30 0x7f07981f4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:52.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.369+0000 7f079dddd640 1 --2- 172.21.15.45:0/1942497718 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0798161630 0x7f0798149ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:52.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.369+0000 7f079dddd640 1 --2- 172.21.15.45:0/1942497718 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079806c3a0 0x7f079815c110 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:52.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.369+0000 7f079dddd640 1 -- 172.21.15.45:0/1942497718 >> 172.21.15.45:0/1942497718 conn(0x7f07980b3c40 msgr2=0x7f07980b1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:52.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.369+0000 7f079dddd640 1 -- 172.21.15.45:0/1942497718 shutdown_connections 2024-05-08T05:43:52.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.369+0000 7f079dddd640 1 -- 172.21.15.45:0/1942497718 wait complete. 2024-05-08T05:43:52.383 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T05:43:52.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.496+0000 7fbda9ee8640 1 Processor -- start 2024-05-08T05:43:52.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.496+0000 7fbda9ee8640 1 -- start start 2024-05-08T05:43:52.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.497+0000 7fbda9ee8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbda406df40 0x7fbda406d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:52.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.497+0000 7fbda9ee8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbda415f830 0x7fbda40d3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:52.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.497+0000 7fbda9ee8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbda40d0cb0 con 0x7fbda415f450 2024-05-08T05:43:52.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.497+0000 7fbda9ee8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fbda406d690 con 0x7fbda406df40 2024-05-08T05:43:52.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.497+0000 7fbda9ee8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbda40d0f90 con 0x7fbda415f830 2024-05-08T05:43:52.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.497+0000 7fbda2ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbda406df40 0x7fbda406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:52.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.497+0000 7fbda2ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbda406df40 0x7fbda406d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55590/0 (socket says 172.21.15.45:55590) 2024-05-08T05:43:52.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.497+0000 7fbda2ffd640 1 -- 172.21.15.45:0/3853691956 learned_addr learned my addr 172.21.15.45:0/3853691956 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:52.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.497+0000 7fbda3fff640 1 --2- 172.21.15.45:0/3853691956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbda415f830 0x7fbda40d3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:52.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.497+0000 7fbda2ffd640 1 -- 172.21.15.45:0/3853691956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbda415f830 msgr2=0x7fbda40d3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:52.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.497+0000 7fbda2ffd640 1 --2- 172.21.15.45:0/3853691956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbda415f830 0x7fbda40d3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:52.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.497+0000 7fbda2ffd640 1 -- 172.21.15.45:0/3853691956 >> v1:172.21.15.63:6789/0 conn(0x7fbda415f450 legacy=0x7fbda406d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:52.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.497+0000 7fbda2ffd640 1 -- 172.21.15.45:0/3853691956 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda40d2fe0 con 0x7fbda406df40 2024-05-08T05:43:52.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.498+0000 7fbda2ffd640 1 --2- 172.21.15.45:0/3853691956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda406df40 0x7fbda406d150 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009870 tx=0x7fbd9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e060e962f3749643 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:52.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.498+0000 7fbda27fc640 1 -- 172.21.15.45:0/3853691956 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98037070 con 0x7fbda406df40 2024-05-08T05:43:52.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.498+0000 7fbda27fc640 1 -- 172.21.15.45:0/3853691956 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd98032440 con 0x7fbda406df40 2024-05-08T05:43:52.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.498+0000 7fbda27fc640 1 -- 172.21.15.45:0/3853691956 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98032810 con 0x7fbda406df40 2024-05-08T05:43:52.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.499+0000 7fbda9ee8640 1 -- 172.21.15.45:0/3853691956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda406df40 msgr2=0x7fbda406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:52.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.499+0000 7fbda9ee8640 1 --2- 172.21.15.45:0/3853691956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda406df40 0x7fbda406d150 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009870 tx=0x7fbd9802a270 comp rx=0 tx=0).stop 2024-05-08T05:43:52.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.499+0000 7fbda9ee8640 1 -- 172.21.15.45:0/3853691956 shutdown_connections 2024-05-08T05:43:52.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.499+0000 7fbda9ee8640 1 --2- 172.21.15.45:0/3853691956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbda415f830 0x7fbda40d3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:52.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.499+0000 7fbda9ee8640 1 --2- 172.21.15.45:0/3853691956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda406df40 0x7fbda406d150 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:52.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.499+0000 7fbda9ee8640 1 -- 172.21.15.45:0/3853691956 >> 172.21.15.45:0/3853691956 conn(0x7fbda40d5040 msgr2=0x7fbda40d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:52.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.499+0000 7fbda9ee8640 1 -- 172.21.15.45:0/3853691956 shutdown_connections 2024-05-08T05:43:52.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.499+0000 7fbda9ee8640 1 -- 172.21.15.45:0/3853691956 wait complete. 2024-05-08T05:43:52.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.500+0000 7fbda9ee8640 1 Processor -- start 2024-05-08T05:43:52.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.500+0000 7fbda9ee8640 1 -- start start 2024-05-08T05:43:52.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.501+0000 7fbda9ee8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbda406df40 0x7fbda40b8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:52.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.501+0000 7fbda9ee8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbda415f450 0x7fbda415db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:52.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.501+0000 7fbda9ee8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda415f830 0x7fbda40cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:52.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.501+0000 7fbda9ee8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbda40d0880 con 0x7fbda415f830 2024-05-08T05:43:52.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.501+0000 7fbda9ee8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbda40d0700 con 0x7fbda406df40 2024-05-08T05:43:52.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.501+0000 7fbda9ee8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbda40d0a00 con 0x7fbda415f450 2024-05-08T05:43:52.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.501+0000 7fbda3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda415f830 0x7fbda40cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:52.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.501+0000 7fbda3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda415f830 0x7fbda40cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55604/0 (socket says 172.21.15.45:55604) 2024-05-08T05:43:52.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.501+0000 7fbda3fff640 1 -- 172.21.15.45:0/1545561127 learned_addr learned my addr 172.21.15.45:0/1545561127 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:52.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.501+0000 7fbda2ffd640 1 --2- 172.21.15.45:0/1545561127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbda415f450 0x7fbda415db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:52.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.501+0000 7fbda37fe640 1 --2- 172.21.15.45:0/1545561127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbda406df40 0x7fbda40b8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:52.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.501+0000 7fbda3fff640 1 -- 172.21.15.45:0/1545561127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbda415f450 msgr2=0x7fbda415db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:52.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.501+0000 7fbda3fff640 1 --2- 172.21.15.45:0/1545561127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbda415f450 0x7fbda415db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:52.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.501+0000 7fbda3fff640 1 -- 172.21.15.45:0/1545561127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbda406df40 msgr2=0x7fbda40b8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:52.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.502+0000 7fbda3fff640 1 --2- 172.21.15.45:0/1545561127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbda406df40 0x7fbda40b8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:52.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.502+0000 7fbda3fff640 1 -- 172.21.15.45:0/1545561127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda40ba410 con 0x7fbda415f830 2024-05-08T05:43:52.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.502+0000 7fbda3fff640 1 --2- 172.21.15.45:0/1545561127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda415f830 0x7fbda40cc110 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c007e30 tx=0x7fbd9c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:52.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.502+0000 7fbd8bfff640 1 -- 172.21.15.45:0/1545561127 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c014620 con 0x7fbda415f830 2024-05-08T05:43:52.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.502+0000 7fbd8bfff640 1 -- 172.21.15.45:0/1545561127 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd9c019070 con 0x7fbda415f830 2024-05-08T05:43:52.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.502+0000 7fbda9ee8640 1 -- 172.21.15.45:0/1545561127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda40ba6f0 con 0x7fbda415f830 2024-05-08T05:43:52.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.502+0000 7fbd8bfff640 1 -- 172.21.15.45:0/1545561127 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c014a00 con 0x7fbda415f830 2024-05-08T05:43:52.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.502+0000 7fbda9ee8640 1 -- 172.21.15.45:0/1545561127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda41592b0 con 0x7fbda415f830 2024-05-08T05:43:52.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.503+0000 7fbda9ee8640 1 -- 172.21.15.45:0/1545561127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbda40693f0 con 0x7fbda415f830 2024-05-08T05:43:52.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.503+0000 7fbd8bfff640 1 -- 172.21.15.45:0/1545561127 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbd9c014ba0 con 0x7fbda415f830 2024-05-08T05:43:52.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.504+0000 7fbd8bfff640 1 --2- 172.21.15.45:0/1545561127 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbd900762d0 0x7fbd90078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:52.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.504+0000 7fbd8bfff640 1 -- 172.21.15.45:0/1545561127 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7fbd9c097bf0 con 0x7fbda415f830 2024-05-08T05:43:52.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.504+0000 7fbda37fe640 1 --2- 172.21.15.45:0/1545561127 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbd900762d0 0x7fbd90078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:52.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.505+0000 7fbda37fe640 1 --2- 172.21.15.45:0/1545561127 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbd900762d0 0x7fbd90078790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbd94001720 tx=0x7fbd94009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:52.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.508+0000 7fbd8bfff640 1 -- 172.21.15.45:0/1545561127 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbd9c061070 con 0x7fbda415f830 2024-05-08T05:43:52.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.700+0000 7fbda9ee8640 1 -- 172.21.15.45:0/1545561127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fbda40008d0 con 0x7fbda415f830 2024-05-08T05:43:52.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.703+0000 7fbd8bfff640 1 -- 172.21.15.45:0/1545561127 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7fbd9c065370 con 0x7fbda415f830 2024-05-08T05:43:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.705+0000 7fbda9ee8640 1 -- 172.21.15.45:0/1545561127 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbd900762d0 msgr2=0x7fbd90078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.705+0000 7fbda9ee8640 1 --2- 172.21.15.45:0/1545561127 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbd900762d0 0x7fbd90078790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbd94001720 tx=0x7fbd94009040 comp rx=0 tx=0).stop 2024-05-08T05:43:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.705+0000 7fbda9ee8640 1 -- 172.21.15.45:0/1545561127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda415f830 msgr2=0x7fbda40cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.705+0000 7fbda9ee8640 1 --2- 172.21.15.45:0/1545561127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda415f830 0x7fbda40cc110 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c007e30 tx=0x7fbd9c00e4b0 comp rx=0 tx=0).stop 2024-05-08T05:43:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.705+0000 7fbda9ee8640 1 -- 172.21.15.45:0/1545561127 shutdown_connections 2024-05-08T05:43:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.705+0000 7fbda9ee8640 1 --2- 172.21.15.45:0/1545561127 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbd900762d0 0x7fbd90078790 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.705+0000 7fbda9ee8640 1 --2- 172.21.15.45:0/1545561127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbda415f450 0x7fbda415db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.705+0000 7fbda9ee8640 1 --2- 172.21.15.45:0/1545561127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbda415f830 0x7fbda40cc110 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.705+0000 7fbda9ee8640 1 --2- 172.21.15.45:0/1545561127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbda406df40 0x7fbda40b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.705+0000 7fbda9ee8640 1 -- 172.21.15.45:0/1545561127 >> 172.21.15.45:0/1545561127 conn(0x7fbda40d5040 msgr2=0x7fbda40b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:52.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.705+0000 7fbda9ee8640 1 -- 172.21.15.45:0/1545561127 shutdown_connections 2024-05-08T05:43:52.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:52.705+0000 7fbda9ee8640 1 -- 172.21.15.45:0/1545561127 wait complete. 2024-05-08T05:43:52.717 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T05:43:52.717 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:43:52.759 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:43:52.807 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T05:43:52.808 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:43:52.808 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:43:52.824 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:43:52.824 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:43:52.824 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:43:52.843 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:43:52.843 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:43:52.843 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:43:52.880 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:43:52.880 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:43:52.880 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:43:52.902 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:43:52.902 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T05:43:52.902 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:43:53.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.023+0000 7f82abc75640 1 Processor -- start 2024-05-08T05:43:53.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.024+0000 7f82abc75640 1 -- start start 2024-05-08T05:43:53.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.024+0000 7f82abc75640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f82a40d1970 0x7f82a40d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:53.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.024+0000 7f82abc75640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82a41630b0 con 0x7f82a40d4cc0 2024-05-08T05:43:53.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.024+0000 7f82abc75640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82a40d3e70 con 0x7f82a415f420 2024-05-08T05:43:53.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.024+0000 7f82abc75640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f82a40d2550 con 0x7f82a40d1970 2024-05-08T05:43:53.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.025+0000 7f82a9210640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f82a40d1970 0x7f82a40d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:53.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.025+0000 7f82aa212640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f82a40d4cc0 0x7f82a40d50e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44452/0 (socket says 172.21.15.45:44452) 2024-05-08T05:43:53.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.025+0000 7f82aa212640 1 -- 172.21.15.45:0/2788807255 learned_addr learned my addr 172.21.15.45:0/2788807255 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:53.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.025+0000 7f82a9210640 1 -- 172.21.15.45:0/2788807255 >> v1:172.21.15.63:6789/0 conn(0x7f82a415f420 legacy=0x7f82a40d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:53.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.025+0000 7f82a9210640 1 -- 172.21.15.45:0/2788807255 >> v1:172.21.15.45:6789/0 conn(0x7f82a40d4cc0 legacy=0x7f82a40d50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:53.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.025+0000 7f82a9210640 1 -- 172.21.15.45:0/2788807255 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a40d2c80 con 0x7f82a40d1970 2024-05-08T05:43:53.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.025+0000 7f82a8a0f640 1 -- 172.21.15.45:0/2788807255 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3740962352 0 0) 0x7f8294001490 con 0x7f82a40d4cc0 2024-05-08T05:43:53.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.026+0000 7f82a9210640 1 --2- 172.21.15.45:0/2788807255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82a40d1970 0x7f82a40d1e20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f82980097a0 tx=0x7f829802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f08cdb139ce0dc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:53.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.026+0000 7f82a8a0f640 1 -- 172.21.15.45:0/2788807255 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298037070 con 0x7f82a40d1970 2024-05-08T05:43:53.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.026+0000 7f82a8a0f640 1 -- 172.21.15.45:0/2788807255 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8298032440 con 0x7f82a40d1970 2024-05-08T05:43:53.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.026+0000 7f82a8a0f640 1 -- 172.21.15.45:0/2788807255 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298032810 con 0x7f82a40d1970 2024-05-08T05:43:53.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.026+0000 7f82abc75640 1 -- 172.21.15.45:0/2788807255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82a40d1970 msgr2=0x7f82a40d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:53.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.026+0000 7f82abc75640 1 --2- 172.21.15.45:0/2788807255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82a40d1970 0x7f82a40d1e20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f82980097a0 tx=0x7f829802a180 comp rx=0 tx=0).stop 2024-05-08T05:43:53.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.026+0000 7f82abc75640 1 -- 172.21.15.45:0/2788807255 shutdown_connections 2024-05-08T05:43:53.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.026+0000 7f82abc75640 1 --2- 172.21.15.45:0/2788807255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82a40d1970 0x7f82a40d1e20 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:53.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.026+0000 7f82abc75640 1 -- 172.21.15.45:0/2788807255 >> 172.21.15.45:0/2788807255 conn(0x7f82a408eac0 msgr2=0x7f82a409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:53.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.026+0000 7f82abc75640 1 -- 172.21.15.45:0/2788807255 shutdown_connections 2024-05-08T05:43:53.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.026+0000 7f82abc75640 1 -- 172.21.15.45:0/2788807255 wait complete. 2024-05-08T05:43:53.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.028+0000 7f82abc75640 1 Processor -- start 2024-05-08T05:43:53.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.028+0000 7f82abc75640 1 -- start start 2024-05-08T05:43:53.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.028+0000 7f82abc75640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82a40d1970 0x7f82a40ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:53.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.028+0000 7f82abc75640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82a40d4cc0 0x7f82a415d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:53.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.028+0000 7f82abc75640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a415df50 0x7f82a41f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:53.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.028+0000 7f82abc75640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82a40d2830 con 0x7f82a40d1970 2024-05-08T05:43:53.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.028+0000 7f82abc75640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82a40d3240 con 0x7f82a415df50 2024-05-08T05:43:53.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.028+0000 7f82abc75640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f82a40cfb20 con 0x7f82a40d4cc0 2024-05-08T05:43:53.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.028+0000 7f82a9a11640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82a40d1970 0x7f82a40ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:53.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.028+0000 7f82a9a11640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82a40d1970 0x7f82a40ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55610/0 (socket says 172.21.15.45:55610) 2024-05-08T05:43:53.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.028+0000 7f82a9a11640 1 -- 172.21.15.45:0/2559788244 learned_addr learned my addr 172.21.15.45:0/2559788244 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:53.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.029+0000 7f82a9210640 1 --2- 172.21.15.45:0/2559788244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82a40d4cc0 0x7f82a415d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:53.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.029+0000 7f82aa212640 1 --2- 172.21.15.45:0/2559788244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a415df50 0x7f82a41f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:53.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.029+0000 7f82a9a11640 1 -- 172.21.15.45:0/2559788244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82a40d4cc0 msgr2=0x7f82a415d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:53.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.029+0000 7f82a9a11640 1 --2- 172.21.15.45:0/2559788244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82a40d4cc0 0x7f82a415d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:53.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.029+0000 7f82a9a11640 1 -- 172.21.15.45:0/2559788244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a415df50 msgr2=0x7f82a41f53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:53.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.029+0000 7f82a9a11640 1 --2- 172.21.15.45:0/2559788244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a415df50 0x7f82a41f53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:53.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.029+0000 7f82a9a11640 1 -- 172.21.15.45:0/2559788244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a40b8f10 con 0x7f82a40d1970 2024-05-08T05:43:53.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.029+0000 7f82a9210640 1 --2- 172.21.15.45:0/2559788244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82a40d4cc0 0x7f82a415d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:53.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.029+0000 7f82a9a11640 1 --2- 172.21.15.45:0/2559788244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82a40d1970 0x7f82a40ca050 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f829c00b420 tx=0x7f829c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:53.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.029+0000 7f82a27fc640 1 -- 172.21.15.45:0/2559788244 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c011830 con 0x7f82a40d1970 2024-05-08T05:43:53.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.029+0000 7f82a27fc640 1 -- 172.21.15.45:0/2559788244 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f829c016070 con 0x7f82a40d1970 2024-05-08T05:43:53.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.029+0000 7f82abc75640 1 -- 172.21.15.45:0/2559788244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a40c95b0 con 0x7f82a40d1970 2024-05-08T05:43:53.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.029+0000 7f82a27fc640 1 -- 172.21.15.45:0/2559788244 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c011c10 con 0x7f82a40d1970 2024-05-08T05:43:53.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.029+0000 7f82abc75640 1 -- 172.21.15.45:0/2559788244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a40cb650 con 0x7f82a40d1970 2024-05-08T05:43:53.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.030+0000 7f82abc75640 1 -- 172.21.15.45:0/2559788244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a40693f0 con 0x7f82a40d1970 2024-05-08T05:43:53.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.031+0000 7f82a27fc640 1 -- 172.21.15.45:0/2559788244 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f829c01b460 con 0x7f82a40d1970 2024-05-08T05:43:53.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.031+0000 7f82a27fc640 1 --2- 172.21.15.45:0/2559788244 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8290076300 0x7f82900787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:53.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.031+0000 7f82a27fc640 1 -- 172.21.15.45:0/2559788244 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f829c096e60 con 0x7f82a40d1970 2024-05-08T05:43:53.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.032+0000 7f82a9210640 1 --2- 172.21.15.45:0/2559788244 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8290076300 0x7f82900787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:53.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.032+0000 7f82a9210640 1 --2- 172.21.15.45:0/2559788244 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8290076300 0x7f82900787c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8298034000 tx=0x7f829802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:53.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.035+0000 7f82a27fc640 1 -- 172.21.15.45:0/2559788244 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f829c020020 con 0x7f82a40d1970 2024-05-08T05:43:53.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.265+0000 7f82abc75640 1 -- 172.21.15.45:0/2559788244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82a40d2100 con 0x7f82a40d1970 2024-05-08T05:43:53.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.266+0000 7f82a27fc640 1 -- 172.21.15.45:0/2559788244 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+3869 (secure 0 0 0) 0x7f829c060930 con 0x7f82a40d1970 2024-05-08T05:43:53.266 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:53.266 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4262647207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4262647207},{"type":"v1","addr":"172.21.15.63:6837","nonce":4262647207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5277,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2872370835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2872370835},{"type":"v1","addr":"172.21.15.45:6834","nonce":2872370835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5280,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1909488716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1909488716},{"type":"v1","addr":"172.21.15.45:6835","nonce":1909488716}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5296,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2955485110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2955485110},{"type":"v1","addr":"172.21.15.63:6836","nonce":2955485110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[]} 2024-05-08T05:43:53.266 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-05-08T05:43:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.268+0000 7f82abc75640 1 -- 172.21.15.45:0/2559788244 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8290076300 msgr2=0x7f82900787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.268+0000 7f82abc75640 1 --2- 172.21.15.45:0/2559788244 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8290076300 0x7f82900787c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8298034000 tx=0x7f829802e000 comp rx=0 tx=0).stop 2024-05-08T05:43:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.268+0000 7f82abc75640 1 -- 172.21.15.45:0/2559788244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82a40d1970 msgr2=0x7f82a40ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.268+0000 7f82abc75640 1 --2- 172.21.15.45:0/2559788244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82a40d1970 0x7f82a40ca050 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f829c00b420 tx=0x7f829c00b8e0 comp rx=0 tx=0).stop 2024-05-08T05:43:53.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.268+0000 7f82abc75640 1 -- 172.21.15.45:0/2559788244 shutdown_connections 2024-05-08T05:43:53.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.268+0000 7f82abc75640 1 --2- 172.21.15.45:0/2559788244 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8290076300 0x7f82900787c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:53.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.268+0000 7f82abc75640 1 --2- 172.21.15.45:0/2559788244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82a40d4cc0 0x7f82a415d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:53.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.268+0000 7f82abc75640 1 --2- 172.21.15.45:0/2559788244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82a415df50 0x7f82a41f53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:53.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.268+0000 7f82abc75640 1 --2- 172.21.15.45:0/2559788244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82a40d1970 0x7f82a40ca050 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:53.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.268+0000 7f82abc75640 1 -- 172.21.15.45:0/2559788244 >> 172.21.15.45:0/2559788244 conn(0x7f82a408eac0 msgr2=0x7f82a406d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:53.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.268+0000 7f82abc75640 1 -- 172.21.15.45:0/2559788244 shutdown_connections 2024-05-08T05:43:53.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.268+0000 7f82abc75640 1 -- 172.21.15.45:0/2559788244 wait complete. 2024-05-08T05:43:53.279 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T05:43:53.279 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T05:43:53.279 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T05:43:53.279 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:43:53.280 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T05:43:53.280 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T05:43:53.280 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T05:43:53.280 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:43:53.280 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T05:43:53.280 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T05:43:53.280 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T05:43:53.281 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:43:53.281 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T05:43:53.281 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T05:43:53.281 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T05:43:53.281 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:43:53.289 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T05:43:53.289 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T05:43:53.290 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T05:43:53.290 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T05:43:53.293 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T05:43:53.293 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T05:43:53.294 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T05:43:53.294 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T05:43:53.332 INFO:tasks.ceph.mds.a:Started 2024-05-08T05:43:53.333 DEBUG:teuthology.parallel:result is None 2024-05-08T05:43:53.333 INFO:tasks.ceph.mds.c:Started 2024-05-08T05:43:53.333 DEBUG:teuthology.parallel:result is None 2024-05-08T05:43:53.335 INFO:tasks.ceph.mds.b:Started 2024-05-08T05:43:53.335 DEBUG:teuthology.parallel:result is None 2024-05-08T05:43:53.336 INFO:tasks.ceph.mds.d:Started 2024-05-08T05:43:53.337 DEBUG:teuthology.parallel:result is None 2024-05-08T05:43:53.337 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T05:43:53.411 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-05-08T05:43:53.411 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-05-08T05:43:53.423 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-08T05:43:53.423 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-08T05:43:53.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.447+0000 7f536a546640 1 Processor -- start 2024-05-08T05:43:53.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.447+0000 7f536a546640 1 -- start start 2024-05-08T05:43:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.448+0000 7f536a546640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f536415bc50 0x7f53640b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.448+0000 7f536a546640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f536406f540 0x7f5364070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.448+0000 7f536a546640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f536406fa40 con 0x7f5364071020 2024-05-08T05:43:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.448+0000 7f535bfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f536406f540 0x7f5364070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.448+0000 7f535bfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f536406f540 0x7f5364070ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55646/0 (socket says 172.21.15.45:55646) 2024-05-08T05:43:53.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.448+0000 7f535bfff640 1 -- 172.21.15.45:0/1060137251 learned_addr learned my addr 172.21.15.45:0/1060137251 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.448+0000 7f5368ae3640 1 --1- 172.21.15.45:0/1060137251 >> v1:172.21.15.45:6789/0 conn(0x7f5364071020 0x7f53640732c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44468/0 (socket says 172.21.15.45:44468) 2024-05-08T05:43:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.448+0000 7f536a546640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f536406fd20 con 0x7f536406f540 2024-05-08T05:43:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.448+0000 7f536a546640 1 -- 172.21.15.45:0/1060137251 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5364070000 con 0x7f536415bc50 2024-05-08T05:43:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.449+0000 7f5363fff640 1 --2- 172.21.15.45:0/1060137251 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f536415bc50 0x7f53640b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.449+0000 7f535bfff640 1 -- 172.21.15.45:0/1060137251 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f536415bc50 msgr2=0x7f53640b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.449+0000 7f535bfff640 1 --2- 172.21.15.45:0/1060137251 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f536415bc50 0x7f53640b5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.449+0000 7f535bfff640 1 -- 172.21.15.45:0/1060137251 >> v1:172.21.15.45:6789/0 conn(0x7f5364071020 legacy=0x7f53640732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.449+0000 7f53637fe640 1 -- 172.21.15.45:0/1060137251 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3881136172 0 0) 0x7f536406fa40 con 0x7f5364071020 2024-05-08T05:43:53.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.449+0000 7f535bfff640 1 -- 172.21.15.45:0/1060137251 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53640702e0 con 0x7f536406f540 2024-05-08T05:43:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.450+0000 7f535bfff640 1 --2- 172.21.15.45:0/1060137251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f536406f540 0x7f5364070ae0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f534c009870 tx=0x7f534c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1625c753bc1efac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.450+0000 7f53637fe640 1 -- 172.21.15.45:0/1060137251 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c037070 con 0x7f536406f540 2024-05-08T05:43:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.450+0000 7f53637fe640 1 -- 172.21.15.45:0/1060137251 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f534c032440 con 0x7f536406f540 2024-05-08T05:43:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.450+0000 7f53637fe640 1 -- 172.21.15.45:0/1060137251 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c032810 con 0x7f536406f540 2024-05-08T05:43:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.450+0000 7f536a546640 1 -- 172.21.15.45:0/1060137251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f536406f540 msgr2=0x7f5364070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:53.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.450+0000 7f536a546640 1 --2- 172.21.15.45:0/1060137251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f536406f540 0x7f5364070ae0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f534c009870 tx=0x7f534c02a270 comp rx=0 tx=0).stop 2024-05-08T05:43:53.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.450+0000 7f536a546640 1 -- 172.21.15.45:0/1060137251 shutdown_connections 2024-05-08T05:43:53.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.450+0000 7f536a546640 1 --2- 172.21.15.45:0/1060137251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f536406f540 0x7f5364070ae0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:53.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.450+0000 7f536a546640 1 --2- 172.21.15.45:0/1060137251 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f536415bc50 0x7f53640b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:53.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.450+0000 7f536a546640 1 -- 172.21.15.45:0/1060137251 >> 172.21.15.45:0/1060137251 conn(0x7f53640b3c40 msgr2=0x7f53640b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:53.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.450+0000 7f536a546640 1 -- 172.21.15.45:0/1060137251 shutdown_connections 2024-05-08T05:43:53.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.450+0000 7f536a546640 1 -- 172.21.15.45:0/1060137251 wait complete. 2024-05-08T05:43:53.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.452+0000 7f536a546640 1 Processor -- start 2024-05-08T05:43:53.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.452+0000 7f536a546640 1 -- start start 2024-05-08T05:43:53.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.452+0000 7f536a546640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f536406f540 0x7f536415e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:53.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.452+0000 7f536a546640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f536415bc50 0x7f5364159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:53.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.452+0000 7f536a546640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5364159e40 0x7f536415d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:53.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.453+0000 7f536a546640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f53640b6260 con 0x7f536415bc50 2024-05-08T05:43:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.453+0000 7f536a546640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f53640b60e0 con 0x7f536406f540 2024-05-08T05:43:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.453+0000 7f536a546640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f536415c030 con 0x7f5364159e40 2024-05-08T05:43:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.453+0000 7f535bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f536415bc50 0x7f5364159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.453+0000 7f535bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f536415bc50 0x7f5364159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55658/0 (socket says 172.21.15.45:55658) 2024-05-08T05:43:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.453+0000 7f535bfff640 1 -- 172.21.15.45:0/2914578748 learned_addr learned my addr 172.21.15.45:0/2914578748 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.453+0000 7f5363fff640 1 --2- 172.21.15.45:0/2914578748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f536406f540 0x7f536415e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.453+0000 7f5368ae3640 1 --2- 172.21.15.45:0/2914578748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5364159e40 0x7f536415d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.453+0000 7f535bfff640 1 -- 172.21.15.45:0/2914578748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5364159e40 msgr2=0x7f536415d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.453+0000 7f535bfff640 1 --2- 172.21.15.45:0/2914578748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5364159e40 0x7f536415d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.453+0000 7f535bfff640 1 -- 172.21.15.45:0/2914578748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f536406f540 msgr2=0x7f536415e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.453+0000 7f535bfff640 1 --2- 172.21.15.45:0/2914578748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f536406f540 0x7f536415e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.453+0000 7f535bfff640 1 -- 172.21.15.45:0/2914578748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5364150b50 con 0x7f536415bc50 2024-05-08T05:43:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.453+0000 7f535bfff640 1 --2- 172.21.15.45:0/2914578748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f536415bc50 0x7f5364159900 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f534c009870 tx=0x7f534c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.453+0000 7f53617fa640 1 -- 172.21.15.45:0/2914578748 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c041020 con 0x7f536415bc50 2024-05-08T05:43:53.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.453+0000 7f53617fa640 1 -- 172.21.15.45:0/2914578748 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f534c030640 con 0x7f536415bc50 2024-05-08T05:43:53.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.453+0000 7f536a546640 1 -- 172.21.15.45:0/2914578748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5364150e30 con 0x7f536415bc50 2024-05-08T05:43:53.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.453+0000 7f53617fa640 1 -- 172.21.15.45:0/2914578748 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c037040 con 0x7f536415bc50 2024-05-08T05:43:53.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.453+0000 7f536a546640 1 -- 172.21.15.45:0/2914578748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53641513f0 con 0x7f536415bc50 2024-05-08T05:43:53.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.454+0000 7f536a546640 1 -- 172.21.15.45:0/2914578748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5364069b00 con 0x7f536415bc50 2024-05-08T05:43:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.455+0000 7f53617fa640 1 -- 172.21.15.45:0/2914578748 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f534c044020 con 0x7f536415bc50 2024-05-08T05:43:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.456+0000 7f53617fa640 1 --2- 172.21.15.45:0/2914578748 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f53440762d0 0x7f5344078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.456+0000 7f53617fa640 1 -- 172.21.15.45:0/2914578748 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5335+0+0 (secure 0 0 0) 0x7f534c0b7b30 con 0x7f536415bc50 2024-05-08T05:43:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.456+0000 7f5363fff640 1 --2- 172.21.15.45:0/2914578748 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f53440762d0 0x7f5344078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:53.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.456+0000 7f5363fff640 1 --2- 172.21.15.45:0/2914578748 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f53440762d0 0x7f5344078790 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5354009aa0 tx=0x7f5354009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:53.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.460+0000 7f53617fa640 1 -- 172.21.15.45:0/2914578748 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f534c04b030 con 0x7f536415bc50 2024-05-08T05:43:53.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:53.662+0000 7f536a546640 1 -- 172.21.15.45:0/2914578748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f5364157250 con 0x7f536415bc50 2024-05-08T05:43:54.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.713+0000 7f53617fa640 1 -- 172.21.15.45:0/2914578748 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v57) ==== 97+0+0 (secure 0 0 0) 0x7f534c080f20 con 0x7f536415bc50 2024-05-08T05:43:54.713 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2024-05-08T05:43:54.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.716+0000 7f536a546640 1 -- 172.21.15.45:0/2914578748 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f53440762d0 msgr2=0x7f5344078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:54.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.716+0000 7f536a546640 1 --2- 172.21.15.45:0/2914578748 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f53440762d0 0x7f5344078790 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5354009aa0 tx=0x7f5354009190 comp rx=0 tx=0).stop 2024-05-08T05:43:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.716+0000 7f536a546640 1 -- 172.21.15.45:0/2914578748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f536415bc50 msgr2=0x7f5364159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.716+0000 7f536a546640 1 --2- 172.21.15.45:0/2914578748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f536415bc50 0x7f5364159900 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f534c009870 tx=0x7f534c0046c0 comp rx=0 tx=0).stop 2024-05-08T05:43:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.716+0000 7f536a546640 1 -- 172.21.15.45:0/2914578748 shutdown_connections 2024-05-08T05:43:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.716+0000 7f536a546640 1 --2- 172.21.15.45:0/2914578748 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f53440762d0 0x7f5344078790 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.716+0000 7f536a546640 1 --2- 172.21.15.45:0/2914578748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5364159e40 0x7f536415d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.716+0000 7f536a546640 1 --2- 172.21.15.45:0/2914578748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f536415bc50 0x7f5364159900 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.716+0000 7f536a546640 1 --2- 172.21.15.45:0/2914578748 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f536406f540 0x7f536415e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.716+0000 7f536a546640 1 -- 172.21.15.45:0/2914578748 >> 172.21.15.45:0/2914578748 conn(0x7f53640b3c40 msgr2=0x7f53640b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.716+0000 7f536a546640 1 -- 172.21.15.45:0/2914578748 shutdown_connections 2024-05-08T05:43:54.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.716+0000 7f536a546640 1 -- 172.21.15.45:0/2914578748 wait complete. 2024-05-08T05:43:54.730 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T05:43:54.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.840+0000 7f424b704640 1 Processor -- start 2024-05-08T05:43:54.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.840+0000 7f424b704640 1 -- start start 2024-05-08T05:43:54.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.841+0000 7f424b704640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f424415f0c0 0x7f42440b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:54.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.841+0000 7f424b704640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f42441638e0 0x7f42440b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:54.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.841+0000 7f424b704640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f424406c3a0 0x7f424406c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:54.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.841+0000 7f424b704640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4244163450 con 0x7f424415f0c0 2024-05-08T05:43:54.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.841+0000 7f424b704640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f42440b1260 con 0x7f42441638e0 2024-05-08T05:43:54.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.841+0000 7f424b704640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f42440b14e0 con 0x7f424406c3a0 2024-05-08T05:43:54.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.841+0000 7f42494a0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f424415f0c0 0x7f42440b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:54.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.841+0000 7f42494a0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f424415f0c0 0x7f42440b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55672/0 (socket says 172.21.15.45:55672) 2024-05-08T05:43:54.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.841+0000 7f42494a0640 1 -- 172.21.15.45:0/2576629282 learned_addr learned my addr 172.21.15.45:0/2576629282 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:54.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.841+0000 7f42494a0640 1 -- 172.21.15.45:0/2576629282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f424406c3a0 msgr2=0x7f424406c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:54.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.841+0000 7f4249ca1640 1 --2- 172.21.15.45:0/2576629282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f424406c3a0 0x7f424406c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:54.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.841+0000 7f4248c9f640 1 --2- 172.21.15.45:0/2576629282 >> v2:172.21.15.63:3300/0 conn(0x7f42441638e0 0x7f42440b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:54.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.841+0000 7f42494a0640 1 --2- 172.21.15.45:0/2576629282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f424406c3a0 0x7f424406c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:54.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.841+0000 7f42494a0640 1 -- 172.21.15.45:0/2576629282 >> v2:172.21.15.63:3300/0 conn(0x7f42441638e0 msgr2=0x7f42440b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:54.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.841+0000 7f42494a0640 1 --2- 172.21.15.45:0/2576629282 >> v2:172.21.15.63:3300/0 conn(0x7f42441638e0 0x7f42440b0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:54.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.841+0000 7f42494a0640 1 -- 172.21.15.45:0/2576629282 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4244162e00 con 0x7f424415f0c0 2024-05-08T05:43:54.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.842+0000 7f42494a0640 1 --2- 172.21.15.45:0/2576629282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424415f0c0 0x7f42440b6f40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f424406ea70 tx=0x7f424002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533842ccd8739238 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:54.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.842+0000 7f423bfff640 1 -- 172.21.15.45:0/2576629282 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240036070 con 0x7f424415f0c0 2024-05-08T05:43:54.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.842+0000 7f423bfff640 1 -- 172.21.15.45:0/2576629282 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4240031600 con 0x7f424415f0c0 2024-05-08T05:43:54.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.842+0000 7f423bfff640 1 -- 172.21.15.45:0/2576629282 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42400319d0 con 0x7f424415f0c0 2024-05-08T05:43:54.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.843+0000 7f424b704640 1 -- 172.21.15.45:0/2576629282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424415f0c0 msgr2=0x7f42440b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:54.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.843+0000 7f424b704640 1 --2- 172.21.15.45:0/2576629282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424415f0c0 0x7f42440b6f40 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f424406ea70 tx=0x7f424002a6a0 comp rx=0 tx=0).stop 2024-05-08T05:43:54.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.843+0000 7f424b704640 1 -- 172.21.15.45:0/2576629282 shutdown_connections 2024-05-08T05:43:54.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.843+0000 7f424b704640 1 --2- 172.21.15.45:0/2576629282 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f424406c3a0 0x7f424406c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:54.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.843+0000 7f424b704640 1 --2- 172.21.15.45:0/2576629282 >> v2:172.21.15.63:3300/0 conn(0x7f42441638e0 0x7f42440b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:54.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.843+0000 7f424b704640 1 --2- 172.21.15.45:0/2576629282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424415f0c0 0x7f42440b6f40 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:54.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.843+0000 7f424b704640 1 -- 172.21.15.45:0/2576629282 >> 172.21.15.45:0/2576629282 conn(0x7f42440b3b10 msgr2=0x7f42440b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:54.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.843+0000 7f424b704640 1 -- 172.21.15.45:0/2576629282 shutdown_connections 2024-05-08T05:43:54.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.843+0000 7f424b704640 1 -- 172.21.15.45:0/2576629282 wait complete. 2024-05-08T05:43:54.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.844+0000 7f424b704640 1 Processor -- start 2024-05-08T05:43:54.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.844+0000 7f424b704640 1 -- start start 2024-05-08T05:43:54.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.845+0000 7f424b704640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424406c3a0 0x7f424414c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:54.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.845+0000 7f424b704640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f424415f0c0 0x7f4244149490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:54.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.845+0000 7f424b704640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42441638e0 0x7f42440afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:54.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.845+0000 7f424b704640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4244073170 con 0x7f424406c3a0 2024-05-08T05:43:54.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.845+0000 7f424b704640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f42440b17c0 con 0x7f42441638e0 2024-05-08T05:43:54.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.845+0000 7f424b704640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f42440732f0 con 0x7f424415f0c0 2024-05-08T05:43:54.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.845+0000 7f42494a0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424406c3a0 0x7f424414c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:54.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.845+0000 7f42494a0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424406c3a0 0x7f424414c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55678/0 (socket says 172.21.15.45:55678) 2024-05-08T05:43:54.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.845+0000 7f42494a0640 1 -- 172.21.15.45:0/626112790 learned_addr learned my addr 172.21.15.45:0/626112790 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:54.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.845+0000 7f4249ca1640 1 --2- 172.21.15.45:0/626112790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42441638e0 0x7f42440afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:54.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.845+0000 7f4248c9f640 1 --2- 172.21.15.45:0/626112790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f424415f0c0 0x7f4244149490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:54.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.845+0000 7f42494a0640 1 -- 172.21.15.45:0/626112790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f424415f0c0 msgr2=0x7f4244149490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:54.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.845+0000 7f42494a0640 1 --2- 172.21.15.45:0/626112790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f424415f0c0 0x7f4244149490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:54.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.845+0000 7f42494a0640 1 -- 172.21.15.45:0/626112790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42441638e0 msgr2=0x7f42440afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:54.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.845+0000 7f42494a0640 1 --2- 172.21.15.45:0/626112790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42441638e0 0x7f42440afe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:54.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.845+0000 7f42494a0640 1 -- 172.21.15.45:0/626112790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424414cd80 con 0x7f424406c3a0 2024-05-08T05:43:54.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.845+0000 7f42494a0640 1 --2- 172.21.15.45:0/626112790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424406c3a0 0x7f424414c840 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4240034040 tx=0x7f4240004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:54.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.846+0000 7f4239ffb640 1 -- 172.21.15.45:0/626112790 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424003f020 con 0x7f424406c3a0 2024-05-08T05:43:54.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.846+0000 7f4239ffb640 1 -- 172.21.15.45:0/626112790 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4240037040 con 0x7f424406c3a0 2024-05-08T05:43:54.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.846+0000 7f424b704640 1 -- 172.21.15.45:0/626112790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424415a960 con 0x7f424406c3a0 2024-05-08T05:43:54.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.846+0000 7f4239ffb640 1 -- 172.21.15.45:0/626112790 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240036070 con 0x7f424406c3a0 2024-05-08T05:43:54.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.846+0000 7f424b704640 1 -- 172.21.15.45:0/626112790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f424415af20 con 0x7f424406c3a0 2024-05-08T05:43:54.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.847+0000 7f424b704640 1 -- 172.21.15.45:0/626112790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42440693f0 con 0x7f424406c3a0 2024-05-08T05:43:54.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.848+0000 7f4239ffb640 1 -- 172.21.15.45:0/626112790 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4240042020 con 0x7f424406c3a0 2024-05-08T05:43:54.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.848+0000 7f4239ffb640 1 --2- 172.21.15.45:0/626112790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f42280762d0 0x7f4228078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:54.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.848+0000 7f4239ffb640 1 -- 172.21.15.45:0/626112790 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5249+0+0 (secure 0 0 0) 0x7f42400b7a40 con 0x7f424406c3a0 2024-05-08T05:43:54.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.848+0000 7f4248c9f640 1 --2- 172.21.15.45:0/626112790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f42280762d0 0x7f4228078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:54.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.849+0000 7f4248c9f640 1 --2- 172.21.15.45:0/626112790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f42280762d0 0x7f4228078790 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4234006fd0 tx=0x7f4234008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:54.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:54.852+0000 7f4239ffb640 1 -- 172.21.15.45:0/626112790 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4240049030 con 0x7f424406c3a0 2024-05-08T05:43:55.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.116+0000 7f424b704640 1 -- 172.21.15.45:0/626112790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f42440d2360 con 0x7f424406c3a0 2024-05-08T05:43:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.116+0000 7f4239ffb640 1 -- 172.21.15.45:0/626112790 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) ==== 80+0+5304 (secure 0 0 0) 0x7f42400b7e20 con 0x7f424406c3a0 2024-05-08T05:43:55.117 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:55.117 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:43:55.117 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2024-05-08T05:43:55.117 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:55.117 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2024-05-08T05:43:55.117 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmrRgXHBAAX1mgdmOD9JC4S1vJNyLSQw==", 2024-05-08T05:43:55.117 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:43:55.117 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:43:55.117 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:43:55.118 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:43:55.118 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:43:55.118 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:43:55.118 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:55.118 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2024-05-08T05:43:55.118 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmvn8SKBAAVkXYBbTfu98gl0AptOfJEg==", 2024-05-08T05:43:55.118 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:43:55.118 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:43:55.118 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:43:55.118 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:43:55.118 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:43:55.118 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:43:55.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:55.119 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2024-05-08T05:43:55.119 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmeiJCIxAAr3Br1fJVgR9TW2Q+4tMQBQ==", 2024-05-08T05:43:55.119 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:43:55.119 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:43:55.119 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:43:55.119 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:43:55.119 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:43:55.119 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:43:55.119 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:55.119 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2024-05-08T05:43:55.119 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmXdFILxAAxteIxUYeFdQKd/oxHLpeOQ==", 2024-05-08T05:43:55.119 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:43:55.120 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:43:55.120 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:43:55.120 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:43:55.120 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:43:55.120 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:43:55.120 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:55.120 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2024-05-08T05:43:55.120 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBWEDtmLdHHGRAAA1o4Z/lTGERIZhulqVtybw==", 2024-05-08T05:43:55.120 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:43:55.120 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:43:55.120 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:43:55.120 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:43:55.120 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:43:55.121 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:55.121 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2024-05-08T05:43:55.121 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBZEDtmMrRVGBAAeKBtcf5fSQMFE6IStUvokQ==", 2024-05-08T05:43:55.121 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:43:55.121 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:43:55.121 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:43:55.121 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:43:55.121 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:43:55.121 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:55.121 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2024-05-08T05:43:55.121 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBcEDtmpozQFhAAS6AF6X592Wh/QiBipcOVew==", 2024-05-08T05:43:55.121 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:43:55.121 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:43:55.121 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:43:55.121 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:43:55.122 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:43:55.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:55.122 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2024-05-08T05:43:55.122 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBfEDtm47lSFRAAt5tvpRCljxGtAmHQqcFj2A==", 2024-05-08T05:43:55.122 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:43:55.122 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:43:55.122 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:43:55.122 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:43:55.122 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:43:55.122 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:55.122 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2024-05-08T05:43:55.123 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBlEDtmDQ7mGxAA5z7rHZRf7X5g7ZiqEoO2DA==", 2024-05-08T05:43:55.123 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:43:55.123 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:43:55.123 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:43:55.123 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:43:55.123 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:43:55.123 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:55.123 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2024-05-08T05:43:55.123 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBoEDtmBdi0GhAAVMfg+KEJHRvaiS49PEboOg==", 2024-05-08T05:43:55.123 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:43:55.123 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:43:55.123 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:43:55.123 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:43:55.123 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:43:55.124 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:55.124 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2024-05-08T05:43:55.124 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBrEDtmG2daGRAANQgJFAT6gNE/vPofCv6veA==", 2024-05-08T05:43:55.124 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:43:55.124 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:43:55.124 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:43:55.124 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:43:55.124 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:43:55.124 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:55.124 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2024-05-08T05:43:55.124 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBuEDtm3tmDGRAARsAzm9KIzgKAs4ZtfCVdaQ==", 2024-05-08T05:43:55.124 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:43:55.124 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:43:55.124 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:43:55.125 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:43:55.125 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:43:55.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:55.125 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2024-05-08T05:43:55.125 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmDcqGMxAALgQaBYp4Zd8uMqswCXvGaw==", 2024-05-08T05:43:55.125 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:43:55.125 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:43:55.125 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2024-05-08T05:43:55.125 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T05:43:55.125 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:43:55.125 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:55.125 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2024-05-08T05:43:55.125 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBVEDtm6z0GARAARgRma5hPpH6X95pE5NNrQw==", 2024-05-08T05:43:55.125 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:43:55.126 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:43:55.126 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:43:55.126 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:43:55.126 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:43:55.126 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.126 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:43:55.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:55.126 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2024-05-08T05:43:55.126 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmAMtIORAAJf8oSwHqSzgjsTuqxtQqHQ==", 2024-05-08T05:43:55.126 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:43:55.126 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:43:55.126 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:43:55.126 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:43:55.126 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:43:55.126 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:43:55.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:55.127 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2024-05-08T05:43:55.127 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBVEDtm9qK1BhAAokvkJZmNxT5WER9/+EKIpA==", 2024-05-08T05:43:55.127 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:43:55.127 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:43:55.127 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:43:55.127 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:43:55.127 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:43:55.127 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:43:55.127 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:55.127 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2024-05-08T05:43:55.127 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmZ4Q3ABAANcqmwOmt+dNJ5wOiPHHO4Q==", 2024-05-08T05:43:55.127 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:43:55.127 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:43:55.127 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:43:55.128 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:43:55.128 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:43:55.128 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:43:55.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:55.128 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2024-05-08T05:43:55.128 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmL0XLGBAA1BSFdyiDYV9Je+igOVBvlQ==", 2024-05-08T05:43:55.128 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:43:55.128 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:43:55.128 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-05-08T05:43:55.128 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:43:55.128 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:43:55.128 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:55.128 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2024-05-08T05:43:55.128 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmfM5OFRAAgdJPUfqmnKU8vRrjGMNwrw==", 2024-05-08T05:43:55.129 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:43:55.129 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:43:55.129 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-05-08T05:43:55.129 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:43:55.129 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.129 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.129 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-05-08T05:43:55.129 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:43:55.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.118+0000 7f424b704640 1 -- 172.21.15.45:0/626112790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f42280762d0 msgr2=0x7f4228078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:55.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.118+0000 7f424b704640 1 --2- 172.21.15.45:0/626112790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f42280762d0 0x7f4228078790 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4234006fd0 tx=0x7f4234008040 comp rx=0 tx=0).stop 2024-05-08T05:43:55.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.118+0000 7f424b704640 1 -- 172.21.15.45:0/626112790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424406c3a0 msgr2=0x7f424414c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:55.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.118+0000 7f424b704640 1 --2- 172.21.15.45:0/626112790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424406c3a0 0x7f424414c840 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4240034040 tx=0x7f4240004a40 comp rx=0 tx=0).stop 2024-05-08T05:43:55.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.119+0000 7f424b704640 1 -- 172.21.15.45:0/626112790 shutdown_connections 2024-05-08T05:43:55.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.119+0000 7f424b704640 1 --2- 172.21.15.45:0/626112790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f42280762d0 0x7f4228078790 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.119+0000 7f424b704640 1 --2- 172.21.15.45:0/626112790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f424415f0c0 0x7f4244149490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.119+0000 7f424b704640 1 --2- 172.21.15.45:0/626112790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42441638e0 0x7f42440afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.119+0000 7f424b704640 1 --2- 172.21.15.45:0/626112790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f424406c3a0 0x7f424414c840 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.119+0000 7f424b704640 1 -- 172.21.15.45:0/626112790 >> 172.21.15.45:0/626112790 conn(0x7f42440b3b10 msgr2=0x7f424406c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:55.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.119+0000 7f424b704640 1 -- 172.21.15.45:0/626112790 shutdown_connections 2024-05-08T05:43:55.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.119+0000 7f424b704640 1 -- 172.21.15.45:0/626112790 wait complete. 2024-05-08T05:43:55.131 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T05:43:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.235+0000 7f8e5f61d640 1 Processor -- start 2024-05-08T05:43:55.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.235+0000 7f8e5f61d640 1 -- start start 2024-05-08T05:43:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.236+0000 7f8e5f61d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e5815f380 0x7f8e580d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.236+0000 7f8e5f61d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8e580d0cf0 0x7f8e580d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.236+0000 7f8e5f61d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8e580d3b00 0x7f8e58162e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.236+0000 7f8e5f61d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f8e580d30f0 con 0x7f8e580d0cf0 2024-05-08T05:43:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.236+0000 7f8e5f61d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8e580d1150 con 0x7f8e580d3b00 2024-05-08T05:43:55.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.236+0000 7f8e5f61d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8e580d2370 con 0x7f8e5815f380 2024-05-08T05:43:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.236+0000 7f8e5cbb8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8e580d0cf0 0x7f8e580d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.236+0000 7f8e5cbb8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8e580d0cf0 0x7f8e580d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55694/0 (socket says 172.21.15.45:55694) 2024-05-08T05:43:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.236+0000 7f8e5cbb8640 1 -- 172.21.15.45:0/64262145 learned_addr learned my addr 172.21.15.45:0/64262145 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.236+0000 7f8e5dbba640 1 --2- 172.21.15.45:0/64262145 >> v2:172.21.15.63:3300/0 conn(0x7f8e580d3b00 0x7f8e58162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.236+0000 7f8e5d3b9640 1 --2- 172.21.15.45:0/64262145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e5815f380 0x7f8e580d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.236+0000 7f8e5cbb8640 1 -- 172.21.15.45:0/64262145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e5815f380 msgr2=0x7f8e580d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.236+0000 7f8e5cbb8640 1 --2- 172.21.15.45:0/64262145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e5815f380 0x7f8e580d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.236+0000 7f8e5cbb8640 1 -- 172.21.15.45:0/64262145 >> v2:172.21.15.63:3300/0 conn(0x7f8e580d3b00 msgr2=0x7f8e58162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.236+0000 7f8e5cbb8640 1 --2- 172.21.15.45:0/64262145 >> v2:172.21.15.63:3300/0 conn(0x7f8e580d3b00 0x7f8e58162e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.236+0000 7f8e5cbb8640 1 -- 172.21.15.45:0/64262145 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e580d1760 con 0x7f8e580d0cf0 2024-05-08T05:43:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.237+0000 7f8e5cbb8640 1 --2- 172.21.15.45:0/64262145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e580d0cf0 0x7f8e580d35c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8e5808f120 tx=0x7f8e4802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59faec9fe99fa925 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.237+0000 7f8e4ffff640 1 -- 172.21.15.45:0/64262145 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48037070 con 0x7f8e580d0cf0 2024-05-08T05:43:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.237+0000 7f8e4ffff640 1 -- 172.21.15.45:0/64262145 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e48032440 con 0x7f8e580d0cf0 2024-05-08T05:43:55.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.237+0000 7f8e4ffff640 1 -- 172.21.15.45:0/64262145 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48032810 con 0x7f8e580d0cf0 2024-05-08T05:43:55.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.238+0000 7f8e5f61d640 1 -- 172.21.15.45:0/64262145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e580d0cf0 msgr2=0x7f8e580d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:55.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.238+0000 7f8e5f61d640 1 --2- 172.21.15.45:0/64262145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e580d0cf0 0x7f8e580d35c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8e5808f120 tx=0x7f8e4802a1e0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.238+0000 7f8e5f61d640 1 -- 172.21.15.45:0/64262145 shutdown_connections 2024-05-08T05:43:55.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.238+0000 7f8e5f61d640 1 --2- 172.21.15.45:0/64262145 >> v2:172.21.15.63:3300/0 conn(0x7f8e580d3b00 0x7f8e58162e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.238+0000 7f8e5f61d640 1 --2- 172.21.15.45:0/64262145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e580d0cf0 0x7f8e580d35c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.238+0000 7f8e5f61d640 1 --2- 172.21.15.45:0/64262145 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e5815f380 0x7f8e580d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.238+0000 7f8e5f61d640 1 -- 172.21.15.45:0/64262145 >> 172.21.15.45:0/64262145 conn(0x7f8e580d4fe0 msgr2=0x7f8e580d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:55.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.238+0000 7f8e5f61d640 1 -- 172.21.15.45:0/64262145 shutdown_connections 2024-05-08T05:43:55.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.238+0000 7f8e5f61d640 1 -- 172.21.15.45:0/64262145 wait complete. 2024-05-08T05:43:55.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.239+0000 7f8e5f61d640 1 Processor -- start 2024-05-08T05:43:55.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.239+0000 7f8e5f61d640 1 -- start start 2024-05-08T05:43:55.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.240+0000 7f8e5f61d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e580d0cf0 0x7f8e580b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:55.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.240+0000 7f8e5f61d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e580d3b00 0x7f8e580b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:55.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.240+0000 7f8e5f61d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e5815f380 0x7f8e581514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:55.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.240+0000 7f8e5f61d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8e580d2000 con 0x7f8e580d0cf0 2024-05-08T05:43:55.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.240+0000 7f8e5f61d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8e580d1d20 con 0x7f8e580d3b00 2024-05-08T05:43:55.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.240+0000 7f8e5f61d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8e5815f760 con 0x7f8e5815f380 2024-05-08T05:43:55.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.240+0000 7f8e5d3b9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e580d0cf0 0x7f8e580b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:55.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.240+0000 7f8e5d3b9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e580d0cf0 0x7f8e580b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55704/0 (socket says 172.21.15.45:55704) 2024-05-08T05:43:55.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.240+0000 7f8e5d3b9640 1 -- 172.21.15.45:0/1835143817 learned_addr learned my addr 172.21.15.45:0/1835143817 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:55.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.240+0000 7f8e5dbba640 1 --2- 172.21.15.45:0/1835143817 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e5815f380 0x7f8e581514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:55.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.240+0000 7f8e5cbb8640 1 --2- 172.21.15.45:0/1835143817 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e580d3b00 0x7f8e580b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:55.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.240+0000 7f8e5d3b9640 1 -- 172.21.15.45:0/1835143817 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e5815f380 msgr2=0x7f8e581514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:55.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.240+0000 7f8e5d3b9640 1 --2- 172.21.15.45:0/1835143817 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e5815f380 0x7f8e581514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.240+0000 7f8e5d3b9640 1 -- 172.21.15.45:0/1835143817 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e580d3b00 msgr2=0x7f8e580b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:55.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.240+0000 7f8e5d3b9640 1 --2- 172.21.15.45:0/1835143817 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e580d3b00 0x7f8e580b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.240+0000 7f8e5d3b9640 1 -- 172.21.15.45:0/1835143817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e581519f0 con 0x7f8e580d0cf0 2024-05-08T05:43:55.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.240+0000 7f8e5d3b9640 1 --2- 172.21.15.45:0/1835143817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e580d0cf0 0x7f8e580b1bf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8e54007e40 tx=0x7f8e5400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:55.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.240+0000 7f8e4dffb640 1 -- 172.21.15.45:0/1835143817 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54014620 con 0x7f8e580d0cf0 2024-05-08T05:43:55.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.240+0000 7f8e4dffb640 1 -- 172.21.15.45:0/1835143817 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e54019070 con 0x7f8e580d0cf0 2024-05-08T05:43:55.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.240+0000 7f8e5f61d640 1 -- 172.21.15.45:0/1835143817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e580b4d90 con 0x7f8e580d0cf0 2024-05-08T05:43:55.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.241+0000 7f8e4dffb640 1 -- 172.21.15.45:0/1835143817 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54014a00 con 0x7f8e580d0cf0 2024-05-08T05:43:55.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.241+0000 7f8e5f61d640 1 -- 172.21.15.45:0/1835143817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e580b5350 con 0x7f8e580d0cf0 2024-05-08T05:43:55.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.242+0000 7f8e5f61d640 1 -- 172.21.15.45:0/1835143817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e580693f0 con 0x7f8e580d0cf0 2024-05-08T05:43:55.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.242+0000 7f8e4dffb640 1 -- 172.21.15.45:0/1835143817 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8e54014ba0 con 0x7f8e580d0cf0 2024-05-08T05:43:55.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.243+0000 7f8e4dffb640 1 --2- 172.21.15.45:0/1835143817 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8e3c0762d0 0x7f8e3c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:55.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.243+0000 7f8e4dffb640 1 -- 172.21.15.45:0/1835143817 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5249+0+0 (secure 0 0 0) 0x7f8e54097c10 con 0x7f8e580d0cf0 2024-05-08T05:43:55.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.243+0000 7f8e5cbb8640 1 --2- 172.21.15.45:0/1835143817 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8e3c0762d0 0x7f8e3c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:55.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.243+0000 7f8e5cbb8640 1 --2- 172.21.15.45:0/1835143817 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8e3c0762d0 0x7f8e3c078790 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8e48034000 tx=0x7f8e4802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:55.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.247+0000 7f8e4dffb640 1 -- 172.21.15.45:0/1835143817 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8e540617c0 con 0x7f8e580d0cf0 2024-05-08T05:43:55.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.442+0000 7f8e5f61d640 1 -- 172.21.15.45:0/1835143817 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8e5808cfd0 con 0x7f8e580d0cf0 2024-05-08T05:43:55.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.442+0000 7f8e4dffb640 1 -- 172.21.15.45:0/1835143817 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v57) ==== 84+0+65 (secure 0 0 0) 0x7f8e540610c0 con 0x7f8e580d0cf0 2024-05-08T05:43:55.443 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:55.443 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-05-08T05:43:55.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:43:55.443 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-05-08T05:43:55.443 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-05-08T05:43:55.443 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:43:55.443 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-05-08T05:43:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.444+0000 7f8e5f61d640 1 -- 172.21.15.45:0/1835143817 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8e3c0762d0 msgr2=0x7f8e3c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.444+0000 7f8e5f61d640 1 --2- 172.21.15.45:0/1835143817 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8e3c0762d0 0x7f8e3c078790 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8e48034000 tx=0x7f8e4802e000 comp rx=0 tx=0).stop 2024-05-08T05:43:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.444+0000 7f8e5f61d640 1 -- 172.21.15.45:0/1835143817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e580d0cf0 msgr2=0x7f8e580b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.444+0000 7f8e5f61d640 1 --2- 172.21.15.45:0/1835143817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e580d0cf0 0x7f8e580b1bf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8e54007e40 tx=0x7f8e5400e4b0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.444+0000 7f8e5f61d640 1 -- 172.21.15.45:0/1835143817 shutdown_connections 2024-05-08T05:43:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.444+0000 7f8e5f61d640 1 --2- 172.21.15.45:0/1835143817 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8e3c0762d0 0x7f8e3c078790 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.444+0000 7f8e5f61d640 1 --2- 172.21.15.45:0/1835143817 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8e5815f380 0x7f8e581514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.444+0000 7f8e5f61d640 1 --2- 172.21.15.45:0/1835143817 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8e580d3b00 0x7f8e580b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.444+0000 7f8e5f61d640 1 --2- 172.21.15.45:0/1835143817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e580d0cf0 0x7f8e580b1bf0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.444+0000 7f8e5f61d640 1 -- 172.21.15.45:0/1835143817 >> 172.21.15.45:0/1835143817 conn(0x7f8e580d4fe0 msgr2=0x7f8e580ba4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:55.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.444+0000 7f8e5f61d640 1 -- 172.21.15.45:0/1835143817 shutdown_connections 2024-05-08T05:43:55.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.444+0000 7f8e5f61d640 1 -- 172.21.15.45:0/1835143817 wait complete. 2024-05-08T05:43:55.455 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T05:43:55.455 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T05:43:55.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.563+0000 7f7a45340640 1 Processor -- start 2024-05-08T05:43:55.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.563+0000 7f7a45340640 1 -- start start 2024-05-08T05:43:55.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.564+0000 7f7a45340640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7a40162d70 0x7f7a40072d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:55.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.564+0000 7f7a45340640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a4006d3e0 con 0x7f7a401631d0 2024-05-08T05:43:55.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.564+0000 7f7a45340640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a40163750 con 0x7f7a4015f0d0 2024-05-08T05:43:55.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.564+0000 7f7a45340640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7a40163d10 con 0x7f7a40162d70 2024-05-08T05:43:55.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.564+0000 7f7a3f7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7a401631d0 0x7f7a400ae8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44470/0 (socket says 172.21.15.45:44470) 2024-05-08T05:43:55.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.564+0000 7f7a3f7fe640 1 -- 172.21.15.45:0/3838498806 learned_addr learned my addr 172.21.15.45:0/3838498806 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:55.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.564+0000 7f7a3e7fc640 1 -- 172.21.15.45:0/3838498806 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3030160364 0 0) 0x7f7a4006d3e0 con 0x7f7a401631d0 2024-05-08T05:43:55.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.564+0000 7f7a37fff640 1 --2- 172.21.15.45:0/3838498806 >> v2:172.21.15.63:3300/0 conn(0x7f7a40162d70 0x7f7a40072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:55.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.564+0000 7f7a3e7fc640 1 -- 172.21.15.45:0/3838498806 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a2c003520 con 0x7f7a401631d0 2024-05-08T05:43:55.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a3e7fc640 1 -- 172.21.15.45:0/3838498806 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 17471528 0 0) 0x7f7a2c003520 con 0x7f7a401631d0 2024-05-08T05:43:55.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a3e7fc640 1 -- 172.21.15.45:0/3838498806 >> v2:172.21.15.63:3300/0 conn(0x7f7a40162d70 msgr2=0x7f7a40072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:55.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a3e7fc640 1 --2- 172.21.15.45:0/3838498806 >> v2:172.21.15.63:3300/0 conn(0x7f7a40162d70 0x7f7a40072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a3e7fc640 1 -- 172.21.15.45:0/3838498806 >> v1:172.21.15.63:6789/0 conn(0x7f7a4015f0d0 legacy=0x7f7a4006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:55.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a3e7fc640 1 -- 172.21.15.45:0/3838498806 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a40163a30 con 0x7f7a401631d0 2024-05-08T05:43:55.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a3e7fc640 1 -- 172.21.15.45:0/3838498806 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f7a38001a70 con 0x7f7a401631d0 2024-05-08T05:43:55.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a3e7fc640 1 -- 172.21.15.45:0/3838498806 >> v1:172.21.15.45:6789/0 conn(0x7f7a401631d0 legacy=0x7f7a400ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:55.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a3e7fc640 1 --2- 172.21.15.45:0/3838498806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2c003a00 0x7f7a2c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:55.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a3e7fc640 1 --2- 172.21.15.45:0/3838498806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a2c004800 0x7f7a2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:55.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a3e7fc640 1 --2- 172.21.15.45:0/3838498806 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a2c0051c0 0x7f7a2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:55.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a3e7fc640 1 -- 172.21.15.45:0/3838498806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7a40163d10 con 0x7f7a2c004800 2024-05-08T05:43:55.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a3e7fc640 1 -- 172.21.15.45:0/3838498806 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c00c110 con 0x7f7a2c003a00 2024-05-08T05:43:55.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a3e7fc640 1 -- 172.21.15.45:0/3838498806 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7a2c00c310 con 0x7f7a2c0051c0 2024-05-08T05:43:55.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a37fff640 1 --2- 172.21.15.45:0/3838498806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a2c004800 0x7f7a2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:55.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a3f7fe640 1 --2- 172.21.15.45:0/3838498806 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a2c0051c0 0x7f7a2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:55.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a3effd640 1 --2- 172.21.15.45:0/3838498806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2c003a00 0x7f7a2c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:55.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a37fff640 1 -- 172.21.15.45:0/3838498806 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a2c0051c0 msgr2=0x7f7a2c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:55.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a37fff640 1 --2- 172.21.15.45:0/3838498806 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a2c0051c0 0x7f7a2c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a37fff640 1 -- 172.21.15.45:0/3838498806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2c003a00 msgr2=0x7f7a2c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:55.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a37fff640 1 --2- 172.21.15.45:0/3838498806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a2c003a00 0x7f7a2c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a37fff640 1 -- 172.21.15.45:0/3838498806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c003520 con 0x7f7a2c004800 2024-05-08T05:43:55.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.565+0000 7f7a3f7fe640 1 -- 172.21.15.45:0/3838498806 reap_dead start 2024-05-08T05:43:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.566+0000 7f7a37fff640 1 --2- 172.21.15.45:0/3838498806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a2c004800 0x7f7a2c004c80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009ac0 tx=0x7f7a2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b47c1a91ada2252 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.566+0000 7f7a3e7fc640 1 -- 172.21.15.45:0/3838498806 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28036070 con 0x7f7a2c004800 2024-05-08T05:43:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.566+0000 7f7a3e7fc640 1 -- 172.21.15.45:0/3838498806 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a28031400 con 0x7f7a2c004800 2024-05-08T05:43:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.566+0000 7f7a3e7fc640 1 -- 172.21.15.45:0/3838498806 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28009690 con 0x7f7a2c004800 2024-05-08T05:43:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.567+0000 7f7a45340640 1 -- 172.21.15.45:0/3838498806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a2c004800 msgr2=0x7f7a2c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.567+0000 7f7a45340640 1 --2- 172.21.15.45:0/3838498806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a2c004800 0x7f7a2c004c80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009ac0 tx=0x7f7a2c005830 comp rx=0 tx=0).stop 2024-05-08T05:43:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.567+0000 7f7a45340640 1 -- 172.21.15.45:0/3838498806 shutdown_connections 2024-05-08T05:43:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.567+0000 7f7a45340640 1 --2- 172.21.15.45:0/3838498806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a2c004800 0x7f7a2c004c80 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.567+0000 7f7a45340640 1 -- 172.21.15.45:0/3838498806 >> 172.21.15.45:0/3838498806 conn(0x7f7a400b3640 msgr2=0x7f7a400b3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.567+0000 7f7a45340640 1 -- 172.21.15.45:0/3838498806 shutdown_connections 2024-05-08T05:43:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.567+0000 7f7a45340640 1 -- 172.21.15.45:0/3838498806 wait complete. 2024-05-08T05:43:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.568+0000 7f7a45340640 1 Processor -- start 2024-05-08T05:43:55.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.568+0000 7f7a45340640 1 -- start start 2024-05-08T05:43:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.569+0000 7f7a45340640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a2c004800 0x7f7a400c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.569+0000 7f7a45340640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a401604d0 0x7f7a400c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.569+0000 7f7a45340640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a4006ce40 0x7f7a400c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.569+0000 7f7a45340640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4015f4b0 con 0x7f7a401604d0 2024-05-08T05:43:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.569+0000 7f7a45340640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4006c7b0 con 0x7f7a4006ce40 2024-05-08T05:43:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.569+0000 7f7a45340640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7a4015f630 con 0x7f7a2c004800 2024-05-08T05:43:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.569+0000 7f7a37fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a401604d0 0x7f7a400c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.569+0000 7f7a37fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a401604d0 0x7f7a400c64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55736/0 (socket says 172.21.15.45:55736) 2024-05-08T05:43:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.569+0000 7f7a37fff640 1 -- 172.21.15.45:0/4264078926 learned_addr learned my addr 172.21.15.45:0/4264078926 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.569+0000 7f7a3effd640 1 --2- 172.21.15.45:0/4264078926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a2c004800 0x7f7a400c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.569+0000 7f7a3f7fe640 1 --2- 172.21.15.45:0/4264078926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a4006ce40 0x7f7a400c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.569+0000 7f7a37fff640 1 -- 172.21.15.45:0/4264078926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a2c004800 msgr2=0x7f7a400c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.569+0000 7f7a37fff640 1 --2- 172.21.15.45:0/4264078926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a2c004800 0x7f7a400c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.569+0000 7f7a37fff640 1 -- 172.21.15.45:0/4264078926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a4006ce40 msgr2=0x7f7a400c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:55.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.569+0000 7f7a37fff640 1 --2- 172.21.15.45:0/4264078926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a4006ce40 0x7f7a400c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:55.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.569+0000 7f7a37fff640 1 -- 172.21.15.45:0/4264078926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a400c6d10 con 0x7f7a401604d0 2024-05-08T05:43:55.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.569+0000 7f7a3effd640 1 --2- 172.21.15.45:0/4264078926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a2c004800 0x7f7a400c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:55.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.569+0000 7f7a3f7fe640 1 --2- 172.21.15.45:0/4264078926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a4006ce40 0x7f7a400c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:55.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.570+0000 7f7a37fff640 1 --2- 172.21.15.45:0/4264078926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a401604d0 0x7f7a400c64c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7a28034040 tx=0x7f7a28004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:55.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.570+0000 7f7a377fe640 1 -- 172.21.15.45:0/4264078926 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2803f020 con 0x7f7a401604d0 2024-05-08T05:43:55.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.570+0000 7f7a377fe640 1 -- 172.21.15.45:0/4264078926 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a28037040 con 0x7f7a401604d0 2024-05-08T05:43:55.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.570+0000 7f7a45340640 1 -- 172.21.15.45:0/4264078926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a400c6ff0 con 0x7f7a401604d0 2024-05-08T05:43:55.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.570+0000 7f7a377fe640 1 -- 172.21.15.45:0/4264078926 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a28036070 con 0x7f7a401604d0 2024-05-08T05:43:55.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.570+0000 7f7a45340640 1 -- 172.21.15.45:0/4264078926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a400c7890 con 0x7f7a401604d0 2024-05-08T05:43:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.571+0000 7f7a45340640 1 -- 172.21.15.45:0/4264078926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a400693f0 con 0x7f7a401604d0 2024-05-08T05:43:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.572+0000 7f7a377fe640 1 -- 172.21.15.45:0/4264078926 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7a28042020 con 0x7f7a401604d0 2024-05-08T05:43:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.572+0000 7f7a377fe640 1 --2- 172.21.15.45:0/4264078926 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7a200762d0 0x7f7a20078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.572+0000 7f7a377fe640 1 -- 172.21.15.45:0/4264078926 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5249+0+0 (secure 0 0 0) 0x7f7a280b75a0 con 0x7f7a401604d0 2024-05-08T05:43:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.572+0000 7f7a3effd640 1 --2- 172.21.15.45:0/4264078926 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7a200762d0 0x7f7a20078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:55.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.573+0000 7f7a3effd640 1 --2- 172.21.15.45:0/4264078926 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7a200762d0 0x7f7a20078790 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7a300011a0 tx=0x7f7a30009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:55.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.576+0000 7f7a377fe640 1 -- 172.21.15.45:0/4264078926 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7a28049030 con 0x7f7a401604d0 2024-05-08T05:43:55.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:55.769+0000 7f7a45340640 1 -- 172.21.15.45:0/4264078926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7a4006d930 con 0x7f7a401604d0 2024-05-08T05:43:56.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.716+0000 7f7a377fe640 1 -- 172.21.15.45:0/4264078926 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v58) ==== 138+0+0 (secure 0 0 0) 0x7f7a28081100 con 0x7f7a401604d0 2024-05-08T05:43:56.716 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-05-08T05:43:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.719+0000 7f7a45340640 1 -- 172.21.15.45:0/4264078926 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7a200762d0 msgr2=0x7f7a20078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.719+0000 7f7a45340640 1 --2- 172.21.15.45:0/4264078926 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7a200762d0 0x7f7a20078790 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7a300011a0 tx=0x7f7a30009040 comp rx=0 tx=0).stop 2024-05-08T05:43:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.719+0000 7f7a45340640 1 -- 172.21.15.45:0/4264078926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a401604d0 msgr2=0x7f7a400c64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.719+0000 7f7a45340640 1 --2- 172.21.15.45:0/4264078926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a401604d0 0x7f7a400c64c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7a28034040 tx=0x7f7a28004730 comp rx=0 tx=0).stop 2024-05-08T05:43:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.719+0000 7f7a45340640 1 -- 172.21.15.45:0/4264078926 shutdown_connections 2024-05-08T05:43:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.719+0000 7f7a45340640 1 --2- 172.21.15.45:0/4264078926 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7a200762d0 0x7f7a20078790 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.719+0000 7f7a45340640 1 --2- 172.21.15.45:0/4264078926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7a4006ce40 0x7f7a400c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.719+0000 7f7a45340640 1 --2- 172.21.15.45:0/4264078926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a401604d0 0x7f7a400c64c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.719+0000 7f7a45340640 1 --2- 172.21.15.45:0/4264078926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7a2c004800 0x7f7a400c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.719+0000 7f7a45340640 1 -- 172.21.15.45:0/4264078926 >> 172.21.15.45:0/4264078926 conn(0x7f7a400b3640 msgr2=0x7f7a400b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.719+0000 7f7a45340640 1 -- 172.21.15.45:0/4264078926 shutdown_connections 2024-05-08T05:43:56.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.719+0000 7f7a45340640 1 -- 172.21.15.45:0/4264078926 wait complete. 2024-05-08T05:43:56.736 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T05:43:56.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.854+0000 7f16912a0640 1 Processor -- start 2024-05-08T05:43:56.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.854+0000 7f16912a0640 1 -- start start 2024-05-08T05:43:56.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.854+0000 7f16912a0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f168c15f430 0x7f168c073e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:56.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.855+0000 7f16912a0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f168c15f810 con 0x7f168c06e6f0 2024-05-08T05:43:56.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.855+0000 7f16912a0640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f168c072d80 con 0x7f168c0b5f60 2024-05-08T05:43:56.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.855+0000 7f16912a0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f168c073060 con 0x7f168c15f430 2024-05-08T05:43:56.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.855+0000 7f168b7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f168c06e6f0 0x7f168c0cefd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44482/0 (socket says 172.21.15.45:44482) 2024-05-08T05:43:56.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.855+0000 7f168b7fe640 1 -- 172.21.15.45:0/3294975520 learned_addr learned my addr 172.21.15.45:0/3294975520 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:56.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.855+0000 7f168affd640 1 --2- 172.21.15.45:0/3294975520 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f168c15f430 0x7f168c073e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:56.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.855+0000 7f1689ffb640 1 -- 172.21.15.45:0/3294975520 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3150773221 0 0) 0x7f168c15f810 con 0x7f168c06e6f0 2024-05-08T05:43:56.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.855+0000 7f1689ffb640 1 -- 172.21.15.45:0/3294975520 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1674003520 con 0x7f168c06e6f0 2024-05-08T05:43:56.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.855+0000 7f1689ffb640 1 -- 172.21.15.45:0/3294975520 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4091416458 0 0) 0x7f168c072d80 con 0x7f168c0b5f60 2024-05-08T05:43:56.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.855+0000 7f1689ffb640 1 -- 172.21.15.45:0/3294975520 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f168c15f810 con 0x7f168c0b5f60 2024-05-08T05:43:56.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.855+0000 7f1689ffb640 1 -- 172.21.15.45:0/3294975520 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2060485453 0 0) 0x7f1674003520 con 0x7f168c06e6f0 2024-05-08T05:43:56.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.855+0000 7f1689ffb640 1 -- 172.21.15.45:0/3294975520 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f168c15f430 msgr2=0x7f168c073e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:56.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.855+0000 7f1689ffb640 1 --2- 172.21.15.45:0/3294975520 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f168c15f430 0x7f168c073e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:56.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.855+0000 7f1689ffb640 1 -- 172.21.15.45:0/3294975520 >> v1:172.21.15.63:6789/0 conn(0x7f168c0b5f60 legacy=0x7f168c072480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:56.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.855+0000 7f1689ffb640 1 -- 172.21.15.45:0/3294975520 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168c073340 con 0x7f168c06e6f0 2024-05-08T05:43:56.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.855+0000 7f1689ffb640 1 -- 172.21.15.45:0/3294975520 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f1684001a70 con 0x7f168c06e6f0 2024-05-08T05:43:56.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.855+0000 7f1689ffb640 1 -- 172.21.15.45:0/3294975520 >> v1:172.21.15.45:6789/0 conn(0x7f168c06e6f0 legacy=0x7f168c0cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:56.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.855+0000 7f1689ffb640 1 --2- 172.21.15.45:0/3294975520 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f168c15f430 0x7f1674003c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:56.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.855+0000 7f1689ffb640 1 --2- 172.21.15.45:0/3294975520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16740041c0 0x7f1674004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:56.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.855+0000 7f1689ffb640 1 --2- 172.21.15.45:0/3294975520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1674004d40 0x7f16740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:56.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.856+0000 7f1689ffb640 1 -- 172.21.15.45:0/3294975520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f168c073060 con 0x7f16740041c0 2024-05-08T05:43:56.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.856+0000 7f1689ffb640 1 -- 172.21.15.45:0/3294975520 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f167400bc20 con 0x7f1674004d40 2024-05-08T05:43:56.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.856+0000 7f1689ffb640 1 -- 172.21.15.45:0/3294975520 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f167400be20 con 0x7f168c15f430 2024-05-08T05:43:56.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.856+0000 7f168a7fc640 1 --2- 172.21.15.45:0/3294975520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16740041c0 0x7f1674004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:56.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.856+0000 7f168affd640 1 --2- 172.21.15.45:0/3294975520 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f168c15f430 0x7f1674003c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:56.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.856+0000 7f168b7fe640 1 --2- 172.21.15.45:0/3294975520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1674004d40 0x7f16740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:56.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.856+0000 7f168a7fc640 1 -- 172.21.15.45:0/3294975520 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f168c15f430 msgr2=0x7f1674003c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:56.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.856+0000 7f168a7fc640 1 --2- 172.21.15.45:0/3294975520 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f168c15f430 0x7f1674003c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:56.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.856+0000 7f168a7fc640 1 -- 172.21.15.45:0/3294975520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1674004d40 msgr2=0x7f16740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:56.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.856+0000 7f168a7fc640 1 --2- 172.21.15.45:0/3294975520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1674004d40 0x7f16740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:56.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.856+0000 7f168a7fc640 1 -- 172.21.15.45:0/3294975520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1674003520 con 0x7f16740041c0 2024-05-08T05:43:56.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.857+0000 7f168a7fc640 1 --2- 172.21.15.45:0/3294975520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16740041c0 0x7f1674004800 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f16800018d0 tx=0x7f1674005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fd949000566d02 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:56.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.857+0000 7f1689ffb640 1 -- 172.21.15.45:0/3294975520 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1680001520 con 0x7f16740041c0 2024-05-08T05:43:56.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.857+0000 7f1689ffb640 1 -- 172.21.15.45:0/3294975520 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f168002acb0 con 0x7f16740041c0 2024-05-08T05:43:56.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.857+0000 7f1689ffb640 1 -- 172.21.15.45:0/3294975520 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1680005310 con 0x7f16740041c0 2024-05-08T05:43:56.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.857+0000 7f16912a0640 1 -- 172.21.15.45:0/3294975520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16740041c0 msgr2=0x7f1674004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:56.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.857+0000 7f16912a0640 1 --2- 172.21.15.45:0/3294975520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16740041c0 0x7f1674004800 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f16800018d0 tx=0x7f1674005830 comp rx=0 tx=0).stop 2024-05-08T05:43:56.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.857+0000 7f168b7fe640 1 -- 172.21.15.45:0/3294975520 reap_dead start 2024-05-08T05:43:56.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.857+0000 7f16912a0640 1 -- 172.21.15.45:0/3294975520 shutdown_connections 2024-05-08T05:43:56.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.857+0000 7f16912a0640 1 -- 172.21.15.45:0/3294975520 >> 172.21.15.45:0/3294975520 conn(0x7f168c0b2cc0 msgr2=0x7f168c0b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:56.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.857+0000 7f16912a0640 1 -- 172.21.15.45:0/3294975520 shutdown_connections 2024-05-08T05:43:56.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.857+0000 7f16912a0640 1 -- 172.21.15.45:0/3294975520 wait complete. 2024-05-08T05:43:56.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.859+0000 7f16912a0640 1 Processor -- start 2024-05-08T05:43:56.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.859+0000 7f16912a0640 1 -- start start 2024-05-08T05:43:56.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.859+0000 7f16912a0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16740041c0 0x7f168c14b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:56.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.859+0000 7f16912a0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f168c161a10 0x7f168c155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:56.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f16912a0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f168c155aa0 0x7f168c155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:56.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f16912a0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f168c073fa0 con 0x7f16740041c0 2024-05-08T05:43:56.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f16912a0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f168c073e20 con 0x7f168c161a10 2024-05-08T05:43:56.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f16912a0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f168c074120 con 0x7f168c155aa0 2024-05-08T05:43:56.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f168affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16740041c0 0x7f168c14b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:56.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f168affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16740041c0 0x7f168c14b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55746/0 (socket says 172.21.15.45:55746) 2024-05-08T05:43:56.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f168affd640 1 -- 172.21.15.45:0/2728777666 learned_addr learned my addr 172.21.15.45:0/2728777666 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:56.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f168a7fc640 1 --2- 172.21.15.45:0/2728777666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f168c161a10 0x7f168c155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:56.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f168b7fe640 1 --2- 172.21.15.45:0/2728777666 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f168c155aa0 0x7f168c155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:56.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f168affd640 1 -- 172.21.15.45:0/2728777666 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f168c155aa0 msgr2=0x7f168c155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:56.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f168affd640 1 --2- 172.21.15.45:0/2728777666 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f168c155aa0 0x7f168c155f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:56.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f168affd640 1 -- 172.21.15.45:0/2728777666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f168c161a10 msgr2=0x7f168c155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:56.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f168affd640 1 --2- 172.21.15.45:0/2728777666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f168c161a10 0x7f168c155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:56.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f168affd640 1 -- 172.21.15.45:0/2728777666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168c14c0e0 con 0x7f16740041c0 2024-05-08T05:43:56.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f168a7fc640 1 --2- 172.21.15.45:0/2728777666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f168c161a10 0x7f168c155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:56.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f168b7fe640 1 --2- 172.21.15.45:0/2728777666 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f168c155aa0 0x7f168c155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:56.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f168affd640 1 --2- 172.21.15.45:0/2728777666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16740041c0 0x7f168c14b740 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f167c0119d0 tx=0x7f167c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:56.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f16737fe640 1 -- 172.21.15.45:0/2728777666 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c00fb50 con 0x7f16740041c0 2024-05-08T05:43:56.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f16737fe640 1 -- 172.21.15.45:0/2728777666 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f167c00fcf0 con 0x7f16740041c0 2024-05-08T05:43:56.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f16912a0640 1 -- 172.21.15.45:0/2728777666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f168c15b5e0 con 0x7f16740041c0 2024-05-08T05:43:56.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f16737fe640 1 -- 172.21.15.45:0/2728777666 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c00fea0 con 0x7f16740041c0 2024-05-08T05:43:56.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.860+0000 7f16912a0640 1 -- 172.21.15.45:0/2728777666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f168c161210 con 0x7f16740041c0 2024-05-08T05:43:56.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.861+0000 7f16912a0640 1 -- 172.21.15.45:0/2728777666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f168c0693f0 con 0x7f16740041c0 2024-05-08T05:43:56.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.862+0000 7f16737fe640 1 -- 172.21.15.45:0/2728777666 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f167c012040 con 0x7f16740041c0 2024-05-08T05:43:56.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.863+0000 7f16737fe640 1 --2- 172.21.15.45:0/2728777666 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1678076300 0x7f16780787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:56.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.863+0000 7f16737fe640 1 -- 172.21.15.45:0/2728777666 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5619+0+0 (secure 0 0 0) 0x7f167c097ac0 con 0x7f16740041c0 2024-05-08T05:43:56.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.863+0000 7f168a7fc640 1 --2- 172.21.15.45:0/2728777666 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1678076300 0x7f16780787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:56.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.863+0000 7f168a7fc640 1 --2- 172.21.15.45:0/2728777666 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1678076300 0x7f16780787c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1680031000 tx=0x7f168002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:56.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:56.867+0000 7f16737fe640 1 -- 172.21.15.45:0/2728777666 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f167c019030 con 0x7f16740041c0 2024-05-08T05:43:57.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.065+0000 7f16912a0640 1 -- 172.21.15.45:0/2728777666 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f168c0b3720 con 0x7f16740041c0 2024-05-08T05:43:57.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.721+0000 7f16737fe640 1 -- 172.21.15.45:0/2728777666 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v59) ==== 170+0+0 (secure 0 0 0) 0x7f167c061470 con 0x7f16740041c0 2024-05-08T05:43:57.722 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-05-08T05:43:57.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.725+0000 7f16912a0640 1 -- 172.21.15.45:0/2728777666 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1678076300 msgr2=0x7f16780787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:57.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.725+0000 7f16912a0640 1 --2- 172.21.15.45:0/2728777666 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1678076300 0x7f16780787c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1680031000 tx=0x7f168002e000 comp rx=0 tx=0).stop 2024-05-08T05:43:57.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.725+0000 7f16912a0640 1 -- 172.21.15.45:0/2728777666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16740041c0 msgr2=0x7f168c14b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:57.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.725+0000 7f16912a0640 1 --2- 172.21.15.45:0/2728777666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16740041c0 0x7f168c14b740 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f167c0119d0 tx=0x7f167c011e90 comp rx=0 tx=0).stop 2024-05-08T05:43:57.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.725+0000 7f16912a0640 1 -- 172.21.15.45:0/2728777666 shutdown_connections 2024-05-08T05:43:57.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.725+0000 7f16912a0640 1 --2- 172.21.15.45:0/2728777666 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1678076300 0x7f16780787c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:57.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.725+0000 7f16912a0640 1 --2- 172.21.15.45:0/2728777666 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f168c155aa0 0x7f168c155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:57.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.725+0000 7f16912a0640 1 --2- 172.21.15.45:0/2728777666 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f168c161a10 0x7f168c155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:57.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.725+0000 7f16912a0640 1 --2- 172.21.15.45:0/2728777666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16740041c0 0x7f168c14b740 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:57.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.725+0000 7f16912a0640 1 -- 172.21.15.45:0/2728777666 >> 172.21.15.45:0/2728777666 conn(0x7f168c0b2cc0 msgr2=0x7f168c0cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:57.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.726+0000 7f16912a0640 1 -- 172.21.15.45:0/2728777666 shutdown_connections 2024-05-08T05:43:57.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.726+0000 7f16912a0640 1 -- 172.21.15.45:0/2728777666 wait complete. 2024-05-08T05:43:57.752 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T05:43:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.864+0000 7f2bf12a7640 1 Processor -- start 2024-05-08T05:43:57.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.864+0000 7f2bf12a7640 1 -- start start 2024-05-08T05:43:57.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.865+0000 7f2bf12a7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2bec162c90 0x7f2bec071750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.865+0000 7f2bf12a7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bec070300 con 0x7f2bec06cab0 2024-05-08T05:43:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.865+0000 7f2bf12a7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bec163120 con 0x7f2bec148760 2024-05-08T05:43:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.865+0000 7f2bf12a7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2bec0b4f40 con 0x7f2bec162c90 2024-05-08T05:43:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.866+0000 7f2beb7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2bec06cab0 0x7f2bec0b53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44488/0 (socket says 172.21.15.45:44488) 2024-05-08T05:43:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.866+0000 7f2beb7fe640 1 -- 172.21.15.45:0/3299043799 learned_addr learned my addr 172.21.15.45:0/3299043799 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.866+0000 7f2bea7fc640 1 --2- 172.21.15.45:0/3299043799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2bec162c90 0x7f2bec071750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.866+0000 7f2be9ffb640 1 -- 172.21.15.45:0/3299043799 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4052242108 0 0) 0x7f2bec070300 con 0x7f2bec06cab0 2024-05-08T05:43:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.866+0000 7f2be9ffb640 1 -- 172.21.15.45:0/3299043799 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bd8003520 con 0x7f2bec06cab0 2024-05-08T05:43:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.866+0000 7f2be9ffb640 1 -- 172.21.15.45:0/3299043799 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2540626272 0 0) 0x7f2bec163120 con 0x7f2bec148760 2024-05-08T05:43:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.866+0000 7f2be9ffb640 1 -- 172.21.15.45:0/3299043799 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bec070300 con 0x7f2bec148760 2024-05-08T05:43:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.866+0000 7f2bea7fc640 1 -- 172.21.15.45:0/3299043799 >> v1:172.21.15.63:6789/0 conn(0x7f2bec148760 legacy=0x7f2bec06c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.866+0000 7f2bea7fc640 1 -- 172.21.15.45:0/3299043799 >> v1:172.21.15.45:6789/0 conn(0x7f2bec06cab0 legacy=0x7f2bec0b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:57.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.866+0000 7f2bea7fc640 1 -- 172.21.15.45:0/3299043799 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bec163400 con 0x7f2bec162c90 2024-05-08T05:43:57.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.867+0000 7f2bea7fc640 1 --2- 172.21.15.45:0/3299043799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2bec162c90 0x7f2bec071750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40097d0 tx=0x7f2bd402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48302b3c040d83fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:57.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.867+0000 7f2be9ffb640 1 -- 172.21.15.45:0/3299043799 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4037070 con 0x7f2bec162c90 2024-05-08T05:43:57.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.867+0000 7f2be9ffb640 1 -- 172.21.15.45:0/3299043799 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd4032440 con 0x7f2bec162c90 2024-05-08T05:43:57.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.867+0000 7f2be9ffb640 1 -- 172.21.15.45:0/3299043799 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4032810 con 0x7f2bec162c90 2024-05-08T05:43:57.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.868+0000 7f2bf12a7640 1 -- 172.21.15.45:0/3299043799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2bec162c90 msgr2=0x7f2bec071750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:57.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.868+0000 7f2bf12a7640 1 --2- 172.21.15.45:0/3299043799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2bec162c90 0x7f2bec071750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40097d0 tx=0x7f2bd402a180 comp rx=0 tx=0).stop 2024-05-08T05:43:57.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.868+0000 7f2bf12a7640 1 -- 172.21.15.45:0/3299043799 shutdown_connections 2024-05-08T05:43:57.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.868+0000 7f2bf12a7640 1 --2- 172.21.15.45:0/3299043799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2bec162c90 0x7f2bec071750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:57.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.868+0000 7f2bf12a7640 1 -- 172.21.15.45:0/3299043799 >> 172.21.15.45:0/3299043799 conn(0x7f2bec0b3c40 msgr2=0x7f2bec0b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:57.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.868+0000 7f2bf12a7640 1 -- 172.21.15.45:0/3299043799 shutdown_connections 2024-05-08T05:43:57.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.868+0000 7f2bf12a7640 1 -- 172.21.15.45:0/3299043799 wait complete. 2024-05-08T05:43:57.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.869+0000 7f2bf12a7640 1 Processor -- start 2024-05-08T05:43:57.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.869+0000 7f2bf12a7640 1 -- start start 2024-05-08T05:43:57.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.870+0000 7f2bf12a7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bec06cab0 0x7f2bec145600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:57.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.870+0000 7f2bf12a7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2bec148760 0x7f2bec15e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:57.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.870+0000 7f2bf12a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bec1e9730 0x7f2bec15eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:57.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.870+0000 7f2bf12a7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec1639c0 con 0x7f2bec1e9730 2024-05-08T05:43:57.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.870+0000 7f2bf12a7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec071c90 con 0x7f2bec06cab0 2024-05-08T05:43:57.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.870+0000 7f2bf12a7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2bec148e00 con 0x7f2bec148760 2024-05-08T05:43:57.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.870+0000 7f2beb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bec1e9730 0x7f2bec15eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:57.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.870+0000 7f2beb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bec1e9730 0x7f2bec15eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55750/0 (socket says 172.21.15.45:55750) 2024-05-08T05:43:57.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.870+0000 7f2beb7fe640 1 -- 172.21.15.45:0/1940185157 learned_addr learned my addr 172.21.15.45:0/1940185157 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.870+0000 7f2beaffd640 1 --2- 172.21.15.45:0/1940185157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bec06cab0 0x7f2bec145600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.870+0000 7f2bea7fc640 1 --2- 172.21.15.45:0/1940185157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2bec148760 0x7f2bec15e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.870+0000 7f2beb7fe640 1 -- 172.21.15.45:0/1940185157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2bec148760 msgr2=0x7f2bec15e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.870+0000 7f2beb7fe640 1 --2- 172.21.15.45:0/1940185157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2bec148760 0x7f2bec15e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.870+0000 7f2beb7fe640 1 -- 172.21.15.45:0/1940185157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bec06cab0 msgr2=0x7f2bec145600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.870+0000 7f2beb7fe640 1 --2- 172.21.15.45:0/1940185157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bec06cab0 0x7f2bec145600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.870+0000 7f2beb7fe640 1 -- 172.21.15.45:0/1940185157 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bec1f5d10 con 0x7f2bec1e9730 2024-05-08T05:43:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.870+0000 7f2bea7fc640 1 --2- 172.21.15.45:0/1940185157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2bec148760 0x7f2bec15e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:43:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.870+0000 7f2beb7fe640 1 --2- 172.21.15.45:0/1940185157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bec1e9730 0x7f2bec15eaa0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00c410 tx=0x7f2bdc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.871+0000 7f2be37fe640 1 -- 172.21.15.45:0/1940185157 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc017070 con 0x7f2bec1e9730 2024-05-08T05:43:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.871+0000 7f2be37fe640 1 -- 172.21.15.45:0/1940185157 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bdc005ce0 con 0x7f2bec1e9730 2024-05-08T05:43:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.871+0000 7f2be37fe640 1 -- 172.21.15.45:0/1940185157 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0072d0 con 0x7f2bec1e9730 2024-05-08T05:43:57.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.871+0000 7f2bf12a7640 1 -- 172.21.15.45:0/1940185157 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bec1e8c50 con 0x7f2bec1e9730 2024-05-08T05:43:57.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.871+0000 7f2bf12a7640 1 -- 172.21.15.45:0/1940185157 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bec145b40 con 0x7f2bec1e9730 2024-05-08T05:43:57.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.872+0000 7f2bf12a7640 1 -- 172.21.15.45:0/1940185157 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bec071750 con 0x7f2bec1e9730 2024-05-08T05:43:57.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.872+0000 7f2be37fe640 1 -- 172.21.15.45:0/1940185157 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2bdc01a070 con 0x7f2bec1e9730 2024-05-08T05:43:57.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.873+0000 7f2be37fe640 1 --2- 172.21.15.45:0/1940185157 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2bcc0762d0 0x7f2bcc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:57.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.873+0000 7f2be37fe640 1 -- 172.21.15.45:0/1940185157 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6001+0+0 (secure 0 0 0) 0x7f2bdc0968f0 con 0x7f2bec1e9730 2024-05-08T05:43:57.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.873+0000 7f2beaffd640 1 --2- 172.21.15.45:0/1940185157 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2bcc0762d0 0x7f2bcc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:57.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.874+0000 7f2beaffd640 1 --2- 172.21.15.45:0/1940185157 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2bcc0762d0 0x7f2bcc078790 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2be4007df0 tx=0x7f2be4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:57.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:57.878+0000 7f2be37fe640 1 -- 172.21.15.45:0/1940185157 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2bec071750 con 0x7f2bec1e9730 2024-05-08T05:43:58.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.106+0000 7f2bf12a7640 1 -- 172.21.15.45:0/1940185157 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2bec1e8a40 con 0x7f2bec1e9730 2024-05-08T05:43:58.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.726+0000 7f2be37fe640 1 -- 172.21.15.45:0/1940185157 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v30) ==== 173+0+0 (secure 0 0 0) 0x7f2bdc05fb60 con 0x7f2bec1e9730 2024-05-08T05:43:58.726 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 9 and data pool 10 2024-05-08T05:43:58.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.730+0000 7f2bf12a7640 1 -- 172.21.15.45:0/1940185157 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2bcc0762d0 msgr2=0x7f2bcc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:58.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.730+0000 7f2bf12a7640 1 --2- 172.21.15.45:0/1940185157 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2bcc0762d0 0x7f2bcc078790 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2be4007df0 tx=0x7f2be4009040 comp rx=0 tx=0).stop 2024-05-08T05:43:58.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.730+0000 7f2bf12a7640 1 -- 172.21.15.45:0/1940185157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bec1e9730 msgr2=0x7f2bec15eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:58.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.730+0000 7f2bf12a7640 1 --2- 172.21.15.45:0/1940185157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bec1e9730 0x7f2bec15eaa0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00c410 tx=0x7f2bdc00c8d0 comp rx=0 tx=0).stop 2024-05-08T05:43:58.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.731+0000 7f2bf12a7640 1 -- 172.21.15.45:0/1940185157 shutdown_connections 2024-05-08T05:43:58.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.731+0000 7f2bf12a7640 1 --2- 172.21.15.45:0/1940185157 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2bcc0762d0 0x7f2bcc078790 secure :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2be4007df0 tx=0x7f2be4009040 comp rx=0 tx=0).stop 2024-05-08T05:43:58.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.731+0000 7f2bf12a7640 1 --2- 172.21.15.45:0/1940185157 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2bec148760 0x7f2bec15e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:58.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.731+0000 7f2bf12a7640 1 --2- 172.21.15.45:0/1940185157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bec1e9730 0x7f2bec15eaa0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:58.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.731+0000 7f2bf12a7640 1 --2- 172.21.15.45:0/1940185157 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2bec06cab0 0x7f2bec145600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:58.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.731+0000 7f2bf12a7640 1 -- 172.21.15.45:0/1940185157 >> 172.21.15.45:0/1940185157 conn(0x7f2bec0b3c40 msgr2=0x7f2bec06c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:58.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.731+0000 7f2bf12a7640 1 -- 172.21.15.45:0/1940185157 shutdown_connections 2024-05-08T05:43:58.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.731+0000 7f2bf12a7640 1 -- 172.21.15.45:0/1940185157 wait complete. 2024-05-08T05:43:58.760 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:43:58.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.869+0000 7fb1e2740640 1 Processor -- start 2024-05-08T05:43:58.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.869+0000 7fb1e2740640 1 -- start start 2024-05-08T05:43:58.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.869+0000 7fb1e2740640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1dc15f420 0x7fb1dc0d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:58.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.869+0000 7fb1e2740640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb1dc0d3ca0 0x7fb1dc0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:58.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.869+0000 7fb1e2740640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb1dc0d1eb0 0x7fb1dc0d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:58.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.869+0000 7fb1e2740640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb1dc0d22f0 con 0x7fb1dc0d3ca0 2024-05-08T05:43:58.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.869+0000 7fb1e2740640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb1dc1630b0 con 0x7fb1dc0d1eb0 2024-05-08T05:43:58.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.869+0000 7fb1e2740640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb1dc0d2550 con 0x7fb1dc15f420 2024-05-08T05:43:58.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.870+0000 7fb1d3fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb1dc0d3ca0 0x7fb1dc0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:58.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.870+0000 7fb1d3fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb1dc0d3ca0 0x7fb1dc0d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55756/0 (socket says 172.21.15.45:55756) 2024-05-08T05:43:58.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.870+0000 7fb1d3fff640 1 -- 172.21.15.45:0/4231600317 learned_addr learned my addr 172.21.15.45:0/4231600317 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:58.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.870+0000 7fb1e0cdd640 1 --2- 172.21.15.45:0/4231600317 >> v2:172.21.15.63:3300/0 conn(0x7fb1dc0d1eb0 0x7fb1dc0d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:58.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.870+0000 7fb1dbfff640 1 --2- 172.21.15.45:0/4231600317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1dc15f420 0x7fb1dc0d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:58.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.870+0000 7fb1d3fff640 1 -- 172.21.15.45:0/4231600317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1dc15f420 msgr2=0x7fb1dc0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:58.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.870+0000 7fb1d3fff640 1 --2- 172.21.15.45:0/4231600317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1dc15f420 0x7fb1dc0d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:58.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.870+0000 7fb1d3fff640 1 -- 172.21.15.45:0/4231600317 >> v2:172.21.15.63:3300/0 conn(0x7fb1dc0d1eb0 msgr2=0x7fb1dc0d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:58.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.870+0000 7fb1d3fff640 1 --2- 172.21.15.45:0/4231600317 >> v2:172.21.15.63:3300/0 conn(0x7fb1dc0d1eb0 0x7fb1dc0d4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:58.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.870+0000 7fb1d3fff640 1 -- 172.21.15.45:0/4231600317 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1dc0d2c80 con 0x7fb1dc0d3ca0 2024-05-08T05:43:58.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.870+0000 7fb1e0cdd640 1 --2- 172.21.15.45:0/4231600317 >> v2:172.21.15.63:3300/0 conn(0x7fb1dc0d1eb0 0x7fb1dc0d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:58.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.871+0000 7fb1d3fff640 1 --2- 172.21.15.45:0/4231600317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1dc0d3ca0 0x7fb1dc0d1970 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0d17a0 tx=0x7fb1c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b788356540b5294a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:58.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.871+0000 7fb1db7fe640 1 -- 172.21.15.45:0/4231600317 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4037070 con 0x7fb1dc0d3ca0 2024-05-08T05:43:58.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.871+0000 7fb1db7fe640 1 -- 172.21.15.45:0/4231600317 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c4032440 con 0x7fb1dc0d3ca0 2024-05-08T05:43:58.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.871+0000 7fb1db7fe640 1 -- 172.21.15.45:0/4231600317 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4032810 con 0x7fb1dc0d3ca0 2024-05-08T05:43:58.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.872+0000 7fb1e2740640 1 -- 172.21.15.45:0/4231600317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1dc0d3ca0 msgr2=0x7fb1dc0d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:58.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.872+0000 7fb1e2740640 1 --2- 172.21.15.45:0/4231600317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1dc0d3ca0 0x7fb1dc0d1970 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0d17a0 tx=0x7fb1c402a1e0 comp rx=0 tx=0).stop 2024-05-08T05:43:58.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.872+0000 7fb1e2740640 1 -- 172.21.15.45:0/4231600317 shutdown_connections 2024-05-08T05:43:58.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.872+0000 7fb1e2740640 1 --2- 172.21.15.45:0/4231600317 >> v2:172.21.15.63:3300/0 conn(0x7fb1dc0d1eb0 0x7fb1dc0d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:58.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.872+0000 7fb1e2740640 1 --2- 172.21.15.45:0/4231600317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1dc0d3ca0 0x7fb1dc0d1970 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:58.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.872+0000 7fb1e2740640 1 --2- 172.21.15.45:0/4231600317 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1dc15f420 0x7fb1dc0d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:58.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.872+0000 7fb1e2740640 1 -- 172.21.15.45:0/4231600317 >> 172.21.15.45:0/4231600317 conn(0x7fb1dc08eac0 msgr2=0x7fb1dc09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:58.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.872+0000 7fb1e2740640 1 -- 172.21.15.45:0/4231600317 shutdown_connections 2024-05-08T05:43:58.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.872+0000 7fb1e2740640 1 -- 172.21.15.45:0/4231600317 wait complete. 2024-05-08T05:43:58.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.873+0000 7fb1e2740640 1 Processor -- start 2024-05-08T05:43:58.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.873+0000 7fb1e2740640 1 -- start start 2024-05-08T05:43:58.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.874+0000 7fb1e2740640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1dc0d1eb0 0x7fb1dc1e9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:58.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.874+0000 7fb1e2740640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1dc15a500 0x7fb1dc0ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:58.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.874+0000 7fb1e2740640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1dc0b2c90 0x7fb1dc0b3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:58.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.874+0000 7fb1e2740640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc0d2830 con 0x7fb1dc0b2c90 2024-05-08T05:43:58.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.874+0000 7fb1e2740640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc0d3240 con 0x7fb1dc15a500 2024-05-08T05:43:58.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.874+0000 7fb1e2740640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb1dc0d5200 con 0x7fb1dc0d1eb0 2024-05-08T05:43:58.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.874+0000 7fb1e0cdd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1dc0b2c90 0x7fb1dc0b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:58.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.874+0000 7fb1e0cdd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1dc0b2c90 0x7fb1dc0b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55764/0 (socket says 172.21.15.45:55764) 2024-05-08T05:43:58.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.874+0000 7fb1e0cdd640 1 -- 172.21.15.45:0/3517905675 learned_addr learned my addr 172.21.15.45:0/3517905675 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:58.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.874+0000 7fb1dbfff640 1 --2- 172.21.15.45:0/3517905675 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1dc0d1eb0 0x7fb1dc1e9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:58.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.874+0000 7fb1d3fff640 1 --2- 172.21.15.45:0/3517905675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1dc15a500 0x7fb1dc0ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:58.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.874+0000 7fb1e0cdd640 1 -- 172.21.15.45:0/3517905675 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1dc0d1eb0 msgr2=0x7fb1dc1e9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:58.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.874+0000 7fb1e0cdd640 1 --2- 172.21.15.45:0/3517905675 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1dc0d1eb0 0x7fb1dc1e9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:58.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.874+0000 7fb1e0cdd640 1 -- 172.21.15.45:0/3517905675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1dc15a500 msgr2=0x7fb1dc0ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:58.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.874+0000 7fb1e0cdd640 1 --2- 172.21.15.45:0/3517905675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1dc15a500 0x7fb1dc0ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:58.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.874+0000 7fb1e0cdd640 1 -- 172.21.15.45:0/3517905675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1dc0cae80 con 0x7fb1dc0b2c90 2024-05-08T05:43:58.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.874+0000 7fb1e0cdd640 1 --2- 172.21.15.45:0/3517905675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1dc0b2c90 0x7fb1dc0b3110 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4007e40 tx=0x7fb1d400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:58.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.874+0000 7fb1d97fa640 1 -- 172.21.15.45:0/3517905675 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d401a070 con 0x7fb1dc0b2c90 2024-05-08T05:43:58.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.874+0000 7fb1d97fa640 1 -- 172.21.15.45:0/3517905675 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1d4003b90 con 0x7fb1dc0b2c90 2024-05-08T05:43:58.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.875+0000 7fb1e2740640 1 -- 172.21.15.45:0/3517905675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1dc0cb160 con 0x7fb1dc0b2c90 2024-05-08T05:43:58.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.875+0000 7fb1d97fa640 1 -- 172.21.15.45:0/3517905675 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d400de80 con 0x7fb1dc0b2c90 2024-05-08T05:43:58.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.875+0000 7fb1e2740640 1 -- 172.21.15.45:0/3517905675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1dc0cb720 con 0x7fb1dc0b2c90 2024-05-08T05:43:58.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.876+0000 7fb1e2740640 1 -- 172.21.15.45:0/3517905675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1dc0693f0 con 0x7fb1dc0b2c90 2024-05-08T05:43:58.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.876+0000 7fb1d97fa640 1 -- 172.21.15.45:0/3517905675 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb1d401d070 con 0x7fb1dc0b2c90 2024-05-08T05:43:58.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.877+0000 7fb1d97fa640 1 --2- 172.21.15.45:0/3517905675 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb1bc0762d0 0x7fb1bc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:58.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.877+0000 7fb1d97fa640 1 -- 172.21.15.45:0/3517905675 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fb1d4097a30 con 0x7fb1dc0b2c90 2024-05-08T05:43:58.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.877+0000 7fb1dbfff640 1 --2- 172.21.15.45:0/3517905675 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb1bc0762d0 0x7fb1bc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:58.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.877+0000 7fb1dbfff640 1 --2- 172.21.15.45:0/3517905675 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb1bc0762d0 0x7fb1bc078790 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009a30 tx=0x7fb1cc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:58.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:58.881+0000 7fb1d97fa640 1 -- 172.21.15.45:0/3517905675 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb1d4010070 con 0x7fb1dc0b2c90 2024-05-08T05:43:59.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.073+0000 7fb1e2740640 1 -- 172.21.15.45:0/3517905675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1dc0b4c50 con 0x7fb1dc0b2c90 2024-05-08T05:43:59.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.074+0000 7fb1d97fa640 1 -- 172.21.15.45:0/3517905675 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16849 (secure 0 0 0) 0x7fb1d40611f0 con 0x7fb1dc0b2c90 2024-05-08T05:43:59.075 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:59.075 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":60,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:43:58.107855+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:43:55.774105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-08T05:43:57.070302+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:43:59.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.077+0000 7fb1e2740640 1 -- 172.21.15.45:0/3517905675 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb1bc0762d0 msgr2=0x7fb1bc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:59.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.077+0000 7fb1e2740640 1 --2- 172.21.15.45:0/3517905675 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb1bc0762d0 0x7fb1bc078790 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009a30 tx=0x7fb1cc008040 comp rx=0 tx=0).stop 2024-05-08T05:43:59.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.077+0000 7fb1e2740640 1 -- 172.21.15.45:0/3517905675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1dc0b2c90 msgr2=0x7fb1dc0b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:59.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.077+0000 7fb1e2740640 1 --2- 172.21.15.45:0/3517905675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1dc0b2c90 0x7fb1dc0b3110 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4007e40 tx=0x7fb1d400d4a0 comp rx=0 tx=0).stop 2024-05-08T05:43:59.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.077+0000 7fb1e2740640 1 -- 172.21.15.45:0/3517905675 shutdown_connections 2024-05-08T05:43:59.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.077+0000 7fb1e2740640 1 --2- 172.21.15.45:0/3517905675 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb1bc0762d0 0x7fb1bc078790 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:59.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.077+0000 7fb1e2740640 1 --2- 172.21.15.45:0/3517905675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1dc0b2c90 0x7fb1dc0b3110 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:59.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.077+0000 7fb1e2740640 1 --2- 172.21.15.45:0/3517905675 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1dc15a500 0x7fb1dc0ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:59.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.077+0000 7fb1e2740640 1 --2- 172.21.15.45:0/3517905675 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1dc0d1eb0 0x7fb1dc1e9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:59.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.077+0000 7fb1e2740640 1 -- 172.21.15.45:0/3517905675 >> 172.21.15.45:0/3517905675 conn(0x7fb1dc08eac0 msgr2=0x7fb1dc06cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:59.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.077+0000 7fb1e2740640 1 -- 172.21.15.45:0/3517905675 shutdown_connections 2024-05-08T05:43:59.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.077+0000 7fb1e2740640 1 -- 172.21.15.45:0/3517905675 wait complete. 2024-05-08T05:43:59.088 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:43:59.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.192+0000 7fc730acf640 1 Processor -- start 2024-05-08T05:43:59.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.192+0000 7fc730acf640 1 -- start start 2024-05-08T05:43:59.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc730acf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc72c0d0ec0 0x7fc72c0d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:59.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc730acf640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc72c0d1760 con 0x7fc72c0d3b00 2024-05-08T05:43:59.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc730acf640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc72c0d1a40 con 0x7fc72c15f380 2024-05-08T05:43:59.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc730acf640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc72c0d30f0 con 0x7fc72c0d0ec0 2024-05-08T05:43:59.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc72ad9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc72c0d3b00 0x7fc72c0b7e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:44504/0 (socket says 172.21.15.45:44504) 2024-05-08T05:43:59.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc72ad9d640 1 -- 172.21.15.45:0/2013760174 learned_addr learned my addr 172.21.15.45:0/2013760174 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:59.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc723fff640 1 --2- 172.21.15.45:0/2013760174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc72c0d0ec0 0x7fc72c0d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:59.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc729d9b640 1 -- 172.21.15.45:0/2013760174 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2474560310 0 0) 0x7fc72c0d1760 con 0x7fc72c0d3b00 2024-05-08T05:43:59.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc729d9b640 1 -- 172.21.15.45:0/2013760174 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc718003520 con 0x7fc72c0d3b00 2024-05-08T05:43:59.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc729d9b640 1 -- 172.21.15.45:0/2013760174 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 752039271 0 0) 0x7fc718003520 con 0x7fc72c0d3b00 2024-05-08T05:43:59.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc729d9b640 1 -- 172.21.15.45:0/2013760174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc72c0d0ec0 msgr2=0x7fc72c0d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:59.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc729d9b640 1 --2- 172.21.15.45:0/2013760174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc72c0d0ec0 0x7fc72c0d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:59.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc729d9b640 1 -- 172.21.15.45:0/2013760174 >> v1:172.21.15.63:6789/0 conn(0x7fc72c15f380 legacy=0x7fc72c0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:59.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc729d9b640 1 -- 172.21.15.45:0/2013760174 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72c0d1d20 con 0x7fc72c0d3b00 2024-05-08T05:43:59.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc729d9b640 1 -- 172.21.15.45:0/2013760174 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fc724001a70 con 0x7fc72c0d3b00 2024-05-08T05:43:59.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc729d9b640 1 -- 172.21.15.45:0/2013760174 >> v1:172.21.15.45:6789/0 conn(0x7fc72c0d3b00 legacy=0x7fc72c0b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:59.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc729d9b640 1 --2- 172.21.15.45:0/2013760174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc718003a80 0x7fc718003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:59.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc729d9b640 1 --2- 172.21.15.45:0/2013760174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc718004800 0x7fc718004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:59.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc729d9b640 1 --2- 172.21.15.45:0/2013760174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc72c0d0ec0 0x7fc718005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:59.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc729d9b640 1 -- 172.21.15.45:0/2013760174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c0d30f0 con 0x7fc718003a80 2024-05-08T05:43:59.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc729d9b640 1 -- 172.21.15.45:0/2013760174 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc71800bb50 con 0x7fc718004800 2024-05-08T05:43:59.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.193+0000 7fc729d9b640 1 -- 172.21.15.45:0/2013760174 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc71800bd50 con 0x7fc72c0d0ec0 2024-05-08T05:43:59.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.194+0000 7fc72a59c640 1 --2- 172.21.15.45:0/2013760174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc718003a80 0x7fc718003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:59.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.194+0000 7fc72ad9d640 1 --2- 172.21.15.45:0/2013760174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc72c0d0ec0 0x7fc718005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:59.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.194+0000 7fc723fff640 1 --2- 172.21.15.45:0/2013760174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc718004800 0x7fc718004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:59.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.194+0000 7fc72a59c640 1 -- 172.21.15.45:0/2013760174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc72c0d0ec0 msgr2=0x7fc718005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:59.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.194+0000 7fc72a59c640 1 --2- 172.21.15.45:0/2013760174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc72c0d0ec0 0x7fc718005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:59.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.194+0000 7fc72a59c640 1 -- 172.21.15.45:0/2013760174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc718004800 msgr2=0x7fc718004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:59.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.194+0000 7fc72a59c640 1 --2- 172.21.15.45:0/2013760174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc718004800 0x7fc718004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:59.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.194+0000 7fc72a59c640 1 -- 172.21.15.45:0/2013760174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc718003520 con 0x7fc718003a80 2024-05-08T05:43:59.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.195+0000 7fc72a59c640 1 --2- 172.21.15.45:0/2013760174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc718003a80 0x7fc718003ea0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc71c001790 tx=0x7fc718005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d848000c5cd9d774 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:59.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.195+0000 7fc729d9b640 1 -- 172.21.15.45:0/2013760174 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0013b0 con 0x7fc718003a80 2024-05-08T05:43:59.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.195+0000 7fc729d9b640 1 -- 172.21.15.45:0/2013760174 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc71c0052d0 con 0x7fc718003a80 2024-05-08T05:43:59.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.195+0000 7fc729d9b640 1 -- 172.21.15.45:0/2013760174 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c02aae0 con 0x7fc718003a80 2024-05-08T05:43:59.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.195+0000 7fc730acf640 1 -- 172.21.15.45:0/2013760174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc718003a80 msgr2=0x7fc718003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:59.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.195+0000 7fc730acf640 1 --2- 172.21.15.45:0/2013760174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc718003a80 0x7fc718003ea0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc71c001790 tx=0x7fc718005830 comp rx=0 tx=0).stop 2024-05-08T05:43:59.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.195+0000 7fc72ad9d640 1 -- 172.21.15.45:0/2013760174 reap_dead start 2024-05-08T05:43:59.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.195+0000 7fc730acf640 1 -- 172.21.15.45:0/2013760174 shutdown_connections 2024-05-08T05:43:59.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.195+0000 7fc730acf640 1 -- 172.21.15.45:0/2013760174 >> 172.21.15.45:0/2013760174 conn(0x7fc72c0d4fe0 msgr2=0x7fc72c0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:59.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.195+0000 7fc730acf640 1 -- 172.21.15.45:0/2013760174 shutdown_connections 2024-05-08T05:43:59.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.195+0000 7fc730acf640 1 -- 172.21.15.45:0/2013760174 wait complete. 2024-05-08T05:43:59.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.197+0000 7fc730acf640 1 Processor -- start 2024-05-08T05:43:59.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.197+0000 7fc730acf640 1 -- start start 2024-05-08T05:43:59.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.197+0000 7fc730acf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc718003a80 0x7fc72c14dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:59.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc730acf640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc72c0b1ca0 0x7fc72c0b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:59.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc730acf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc72c1620a0 0x7fc72c162520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:59.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc730acf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c15f760 con 0x7fc72c1620a0 2024-05-08T05:43:59.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc730acf640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c0d2370 con 0x7fc72c0b1ca0 2024-05-08T05:43:59.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc730acf640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc72c0d2800 con 0x7fc718003a80 2024-05-08T05:43:59.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc72ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc72c1620a0 0x7fc72c162520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:59.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc72ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc72c1620a0 0x7fc72c162520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55782/0 (socket says 172.21.15.45:55782) 2024-05-08T05:43:59.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc72ad9d640 1 -- 172.21.15.45:0/3837848838 learned_addr learned my addr 172.21.15.45:0/3837848838 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:59.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc72a59c640 1 --2- 172.21.15.45:0/3837848838 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc718003a80 0x7fc72c14dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:59.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc723fff640 1 --2- 172.21.15.45:0/3837848838 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc72c0b1ca0 0x7fc72c0b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:59.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc72ad9d640 1 -- 172.21.15.45:0/3837848838 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc718003a80 msgr2=0x7fc72c14dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:59.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc72ad9d640 1 --2- 172.21.15.45:0/3837848838 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc718003a80 0x7fc72c14dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:59.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc72ad9d640 1 -- 172.21.15.45:0/3837848838 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc72c0b1ca0 msgr2=0x7fc72c0b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:59.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc72ad9d640 1 --2- 172.21.15.45:0/3837848838 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc72c0b1ca0 0x7fc72c0b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:59.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc72ad9d640 1 -- 172.21.15.45:0/3837848838 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72c14e450 con 0x7fc72c1620a0 2024-05-08T05:43:59.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc72ad9d640 1 --2- 172.21.15.45:0/3837848838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc72c1620a0 0x7fc72c162520 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc724001330 tx=0x7fc72400e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:59.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc722ffd640 1 -- 172.21.15.45:0/3837848838 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72400ec30 con 0x7fc72c1620a0 2024-05-08T05:43:59.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc722ffd640 1 -- 172.21.15.45:0/3837848838 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc724018070 con 0x7fc72c1620a0 2024-05-08T05:43:59.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc730acf640 1 -- 172.21.15.45:0/3837848838 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc72c14e140 con 0x7fc72c1620a0 2024-05-08T05:43:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc722ffd640 1 -- 172.21.15.45:0/3837848838 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724013420 con 0x7fc72c1620a0 2024-05-08T05:43:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.198+0000 7fc730acf640 1 -- 172.21.15.45:0/3837848838 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc72c14e9d0 con 0x7fc72c1620a0 2024-05-08T05:43:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.199+0000 7fc730acf640 1 -- 172.21.15.45:0/3837848838 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc72c06ed70 con 0x7fc72c1620a0 2024-05-08T05:43:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.200+0000 7fc722ffd640 1 -- 172.21.15.45:0/3837848838 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc7240135e0 con 0x7fc72c1620a0 2024-05-08T05:43:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.201+0000 7fc722ffd640 1 --2- 172.21.15.45:0/3837848838 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc70c0762d0 0x7fc70c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.201+0000 7fc722ffd640 1 -- 172.21.15.45:0/3837848838 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7fc724097b80 con 0x7fc72c1620a0 2024-05-08T05:43:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.201+0000 7fc72a59c640 1 --2- 172.21.15.45:0/3837848838 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc70c0762d0 0x7fc70c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:59.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.201+0000 7fc72a59c640 1 --2- 172.21.15.45:0/3837848838 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc70c0762d0 0x7fc70c078790 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc71c001790 tx=0x7fc71c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:59.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.205+0000 7fc722ffd640 1 -- 172.21.15.45:0/3837848838 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc72c06ed70 con 0x7fc72c1620a0 2024-05-08T05:43:59.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.399+0000 7fc730acf640 1 -- 172.21.15.45:0/3837848838 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc72c068c40 con 0x7fc72c1620a0 2024-05-08T05:43:59.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.400+0000 7fc722ffd640 1 -- 172.21.15.45:0/3837848838 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16849 (secure 0 0 0) 0x7fc72c068c40 con 0x7fc72c1620a0 2024-05-08T05:43:59.401 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:43:59.401 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":60,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:43:58.107855+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:43:55.774105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-08T05:43:57.070302+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:43:59.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.402+0000 7fc730acf640 1 -- 172.21.15.45:0/3837848838 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc70c0762d0 msgr2=0x7fc70c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:59.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.402+0000 7fc730acf640 1 --2- 172.21.15.45:0/3837848838 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc70c0762d0 0x7fc70c078790 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc71c001790 tx=0x7fc71c02e000 comp rx=0 tx=0).stop 2024-05-08T05:43:59.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.402+0000 7fc730acf640 1 -- 172.21.15.45:0/3837848838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc72c1620a0 msgr2=0x7fc72c162520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:59.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.402+0000 7fc730acf640 1 --2- 172.21.15.45:0/3837848838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc72c1620a0 0x7fc72c162520 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc724001330 tx=0x7fc72400e160 comp rx=0 tx=0).stop 2024-05-08T05:43:59.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.402+0000 7fc730acf640 1 -- 172.21.15.45:0/3837848838 shutdown_connections 2024-05-08T05:43:59.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.402+0000 7fc730acf640 1 --2- 172.21.15.45:0/3837848838 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc70c0762d0 0x7fc70c078790 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:59.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.402+0000 7fc730acf640 1 --2- 172.21.15.45:0/3837848838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc72c1620a0 0x7fc72c162520 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:59.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.402+0000 7fc730acf640 1 --2- 172.21.15.45:0/3837848838 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc72c0b1ca0 0x7fc72c0b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:59.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.402+0000 7fc730acf640 1 --2- 172.21.15.45:0/3837848838 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc718003a80 0x7fc72c14dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:59.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.402+0000 7fc730acf640 1 -- 172.21.15.45:0/3837848838 >> 172.21.15.45:0/3837848838 conn(0x7fc72c0d4fe0 msgr2=0x7fc72c0c8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:59.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.402+0000 7fc730acf640 1 -- 172.21.15.45:0/3837848838 shutdown_connections 2024-05-08T05:43:59.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.402+0000 7fc730acf640 1 -- 172.21.15.45:0/3837848838 wait complete. 2024-05-08T05:43:59.414 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T05:43:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.520+0000 7feed719d640 1 Processor -- start 2024-05-08T05:43:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.521+0000 7feed719d640 1 -- start start 2024-05-08T05:43:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.521+0000 7feed719d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feed006e0f0 0x7feed015c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.521+0000 7feed719d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7feed006e4d0 0x7feed00aec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.521+0000 7feed719d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed015c920 con 0x7feed0072840 2024-05-08T05:43:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.521+0000 7feed719d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7feed015cc00 con 0x7feed006e4d0 2024-05-08T05:43:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.521+0000 7feed719d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feed015cee0 con 0x7feed006e0f0 2024-05-08T05:43:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.522+0000 7feed573a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7feed006e4d0 0x7feed00aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.522+0000 7feed4f39640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feed006e0f0 0x7feed015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.522+0000 7feecffff640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7feed0072840 0x7feed00d0f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:32864/0 (socket says 172.21.15.45:32864) 2024-05-08T05:43:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.522+0000 7feecffff640 1 -- 172.21.15.45:0/1949419015 learned_addr learned my addr 172.21.15.45:0/1949419015 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:43:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.522+0000 7feed573a640 1 -- 172.21.15.45:0/1949419015 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feed006e0f0 msgr2=0x7feed015c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:43:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.522+0000 7feed573a640 1 --2- 172.21.15.45:0/1949419015 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feed006e0f0 0x7feed015c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.522+0000 7feed573a640 1 -- 172.21.15.45:0/1949419015 >> v1:172.21.15.63:6789/0 conn(0x7feed0072840 legacy=0x7feed00d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.522+0000 7feecf7fe640 1 -- 172.21.15.45:0/1949419015 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 887956211 0 0) 0x7feed015c920 con 0x7feed0072840 2024-05-08T05:43:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.522+0000 7feed573a640 1 -- 172.21.15.45:0/1949419015 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed015d1c0 con 0x7feed006e4d0 2024-05-08T05:43:59.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.523+0000 7feed573a640 1 --2- 172.21.15.45:0/1949419015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed006e4d0 0x7feed00aec30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feec8009d90 tx=0x7feec802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c38f2d7ccbc1fa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.523+0000 7feecf7fe640 1 -- 172.21.15.45:0/1949419015 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8007740 con 0x7feed006e4d0 2024-05-08T05:43:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.523+0000 7feecf7fe640 1 -- 172.21.15.45:0/1949419015 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec80078e0 con 0x7feed006e4d0 2024-05-08T05:43:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.523+0000 7feecf7fe640 1 -- 172.21.15.45:0/1949419015 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8007cb0 con 0x7feed006e4d0 2024-05-08T05:43:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.524+0000 7feed719d640 1 -- 172.21.15.45:0/1949419015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed006e4d0 msgr2=0x7feed00aec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.524+0000 7feed719d640 1 --2- 172.21.15.45:0/1949419015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed006e4d0 0x7feed00aec30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feec8009d90 tx=0x7feec802a660 comp rx=0 tx=0).stop 2024-05-08T05:43:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.524+0000 7feed719d640 1 -- 172.21.15.45:0/1949419015 shutdown_connections 2024-05-08T05:43:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.524+0000 7feed719d640 1 --2- 172.21.15.45:0/1949419015 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed006e4d0 0x7feed00aec30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.524+0000 7feed719d640 1 --2- 172.21.15.45:0/1949419015 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feed006e0f0 0x7feed015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.524+0000 7feed719d640 1 -- 172.21.15.45:0/1949419015 >> 172.21.15.45:0/1949419015 conn(0x7feed006c3a0 msgr2=0x7feed006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.524+0000 7feed719d640 1 -- 172.21.15.45:0/1949419015 shutdown_connections 2024-05-08T05:43:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.524+0000 7feed719d640 1 -- 172.21.15.45:0/1949419015 wait complete. 2024-05-08T05:43:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.525+0000 7feed719d640 1 Processor -- start 2024-05-08T05:43:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.525+0000 7feed719d640 1 -- start start 2024-05-08T05:43:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.526+0000 7feed719d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feed006e0f0 0x7feed01556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.526+0000 7feed719d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feed006e4d0 0x7feed00cf900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.526+0000 7feed719d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed0072840 0x7feed00ce650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.526+0000 7feed719d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feed00ae7d0 con 0x7feed006e4d0 2024-05-08T05:43:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.526+0000 7feed719d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feed00ae650 con 0x7feed0072840 2024-05-08T05:43:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.526+0000 7feed719d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feed01530b0 con 0x7feed006e0f0 2024-05-08T05:43:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.526+0000 7feecffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feed006e4d0 0x7feed00cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.526+0000 7feecffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feed006e4d0 0x7feed00cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55798/0 (socket says 172.21.15.45:55798) 2024-05-08T05:43:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.526+0000 7feecffff640 1 -- 172.21.15.45:0/3410460613 learned_addr learned my addr 172.21.15.45:0/3410460613 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:43:59.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.526+0000 7feed4f39640 1 --2- 172.21.15.45:0/3410460613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feed006e0f0 0x7feed01556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.526+0000 7feed573a640 1 --2- 172.21.15.45:0/3410460613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed0072840 0x7feed00ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.526+0000 7feecffff640 1 -- 172.21.15.45:0/3410460613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feed006e0f0 msgr2=0x7feed01556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.526+0000 7feecffff640 1 --2- 172.21.15.45:0/3410460613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feed006e0f0 0x7feed01556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.526+0000 7feecffff640 1 -- 172.21.15.45:0/3410460613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed0072840 msgr2=0x7feed00ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.526+0000 7feecffff640 1 --2- 172.21.15.45:0/3410460613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed0072840 0x7feed00ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.526+0000 7feecffff640 1 -- 172.21.15.45:0/3410460613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed0155bf0 con 0x7feed006e4d0 2024-05-08T05:43:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.526+0000 7feed573a640 1 --2- 172.21.15.45:0/3410460613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed0072840 0x7feed00ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:43:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.526+0000 7feecffff640 1 --2- 172.21.15.45:0/3410460613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feed006e4d0 0x7feed00cf900 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7feebc00c3c0 tx=0x7feebc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.527+0000 7feecd7fa640 1 -- 172.21.15.45:0/3410460613 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc0074e0 con 0x7feed006e4d0 2024-05-08T05:43:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.527+0000 7feecd7fa640 1 -- 172.21.15.45:0/3410460613 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feebc0047e0 con 0x7feed006e4d0 2024-05-08T05:43:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.527+0000 7feecd7fa640 1 -- 172.21.15.45:0/3410460613 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc004c10 con 0x7feed006e4d0 2024-05-08T05:43:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.527+0000 7feed719d640 1 -- 172.21.15.45:0/3410460613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed009e150 con 0x7feed006e4d0 2024-05-08T05:43:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.527+0000 7feed719d640 1 -- 172.21.15.45:0/3410460613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed009e710 con 0x7feed006e4d0 2024-05-08T05:43:59.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.528+0000 7feed719d640 1 -- 172.21.15.45:0/3410460613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed00693f0 con 0x7feed006e4d0 2024-05-08T05:43:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.528+0000 7feecd7fa640 1 -- 172.21.15.45:0/3410460613 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7feebc011070 con 0x7feed006e4d0 2024-05-08T05:43:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.529+0000 7feecd7fa640 1 --2- 172.21.15.45:0/3410460613 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feeb40762d0 0x7feeb4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.529+0000 7feecd7fa640 1 -- 172.21.15.45:0/3410460613 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6101+0+0 (secure 0 0 0) 0x7feebc097180 con 0x7feed006e4d0 2024-05-08T05:43:59.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.529+0000 7feed4f39640 1 --2- 172.21.15.45:0/3410460613 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feeb40762d0 0x7feeb4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:59.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.529+0000 7feed4f39640 1 --2- 172.21.15.45:0/3410460613 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feeb40762d0 0x7feeb4078790 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feec4009b40 tx=0x7feec4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:59.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.533+0000 7feecd7fa640 1 -- 172.21.15.45:0/3410460613 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7feebc0609e0 con 0x7feed006e4d0 2024-05-08T05:43:59.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:43:59.764+0000 7feed719d640 1 -- 172.21.15.45:0/3410460613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7feed0069b70 con 0x7feed006e4d0 2024-05-08T05:44:00.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.746+0000 7feecd7fa640 1 -- 172.21.15.45:0/3410460613 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) ==== 118+0+0 (secure 0 0 0) 0x7feebc0602e0 con 0x7feed006e4d0 2024-05-08T05:44:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.749+0000 7feed719d640 1 -- 172.21.15.45:0/3410460613 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feeb40762d0 msgr2=0x7feeb4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.749+0000 7feed719d640 1 --2- 172.21.15.45:0/3410460613 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feeb40762d0 0x7feeb4078790 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feec4009b40 tx=0x7feec4009190 comp rx=0 tx=0).stop 2024-05-08T05:44:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.749+0000 7feed719d640 1 -- 172.21.15.45:0/3410460613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feed006e4d0 msgr2=0x7feed00cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.749+0000 7feed719d640 1 --2- 172.21.15.45:0/3410460613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feed006e4d0 0x7feed00cf900 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7feebc00c3c0 tx=0x7feebc00c880 comp rx=0 tx=0).stop 2024-05-08T05:44:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.749+0000 7feed719d640 1 -- 172.21.15.45:0/3410460613 shutdown_connections 2024-05-08T05:44:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.749+0000 7feed719d640 1 --2- 172.21.15.45:0/3410460613 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feeb40762d0 0x7feeb4078790 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.749+0000 7feed719d640 1 --2- 172.21.15.45:0/3410460613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feed0072840 0x7feed00ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.749+0000 7feed719d640 1 --2- 172.21.15.45:0/3410460613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feed006e4d0 0x7feed00cf900 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.749+0000 7feed719d640 1 --2- 172.21.15.45:0/3410460613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feed006e0f0 0x7feed01556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.749+0000 7feed719d640 1 -- 172.21.15.45:0/3410460613 >> 172.21.15.45:0/3410460613 conn(0x7feed006c3a0 msgr2=0x7feed015d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:00.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.749+0000 7feed719d640 1 -- 172.21.15.45:0/3410460613 shutdown_connections 2024-05-08T05:44:00.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.749+0000 7feed719d640 1 -- 172.21.15.45:0/3410460613 wait complete. 2024-05-08T05:44:00.763 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T05:44:00.763 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T05:44:00.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.877+0000 7f2b04951640 1 Processor -- start 2024-05-08T05:44:00.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.877+0000 7f2b04951640 1 -- start start 2024-05-08T05:44:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.878+0000 7f2b04951640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b000b6590 0x7f2b000d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.878+0000 7f2b04951640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b000d3320 con 0x7f2b00163460 2024-05-08T05:44:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.878+0000 7f2b04951640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b000d3600 con 0x7f2b000d2bb0 2024-05-08T05:44:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.878+0000 7f2b04951640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b000d38e0 con 0x7f2b000b6590 2024-05-08T05:44:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.878+0000 7f2afed9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2b00163460 0x7f2b0015ec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53506/0 (socket says 172.21.15.45:53506) 2024-05-08T05:44:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.878+0000 7f2afed9d640 1 -- 172.21.15.45:0/1254498901 learned_addr learned my addr 172.21.15.45:0/1254498901 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:44:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.878+0000 7f2afe59c640 1 --2- 172.21.15.45:0/1254498901 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b000b6590 0x7f2b000d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.878+0000 7f2afdd9b640 1 -- 172.21.15.45:0/1254498901 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2321159684 0 0) 0x7f2b000d3320 con 0x7f2b00163460 2024-05-08T05:44:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.878+0000 7f2afdd9b640 1 -- 172.21.15.45:0/1254498901 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2aec003520 con 0x7f2b00163460 2024-05-08T05:44:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.878+0000 7f2afdd9b640 1 -- 172.21.15.45:0/1254498901 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1672197072 0 0) 0x7f2aec003520 con 0x7f2b00163460 2024-05-08T05:44:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.878+0000 7f2afdd9b640 1 -- 172.21.15.45:0/1254498901 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b000b6590 msgr2=0x7f2b000d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.878+0000 7f2afdd9b640 1 --2- 172.21.15.45:0/1254498901 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b000b6590 0x7f2b000d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.878+0000 7f2afdd9b640 1 -- 172.21.15.45:0/1254498901 >> v1:172.21.15.63:6789/0 conn(0x7f2b000d2bb0 legacy=0x7f2b00162d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.878+0000 7f2afdd9b640 1 -- 172.21.15.45:0/1254498901 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b000d3bc0 con 0x7f2b00163460 2024-05-08T05:44:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.878+0000 7f2afdd9b640 1 -- 172.21.15.45:0/1254498901 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f2af8001a70 con 0x7f2b00163460 2024-05-08T05:44:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.878+0000 7f2afdd9b640 1 -- 172.21.15.45:0/1254498901 >> v1:172.21.15.45:6789/0 conn(0x7f2b00163460 legacy=0x7f2b0015ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.878+0000 7f2afdd9b640 1 --2- 172.21.15.45:0/1254498901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2aec003a30 0x7f2aec003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.878+0000 7f2afdd9b640 1 --2- 172.21.15.45:0/1254498901 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b000b6590 0x7f2aec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.879+0000 7f2afdd9b640 1 --2- 172.21.15.45:0/1254498901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aec004d40 0x7f2aec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.879+0000 7f2afdd9b640 1 -- 172.21.15.45:0/1254498901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2b000d38e0 con 0x7f2aec004d40 2024-05-08T05:44:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.879+0000 7f2afdd9b640 1 -- 172.21.15.45:0/1254498901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec00baf0 con 0x7f2aec003a30 2024-05-08T05:44:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.879+0000 7f2afdd9b640 1 -- 172.21.15.45:0/1254498901 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2aec00bcf0 con 0x7f2b000b6590 2024-05-08T05:44:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.879+0000 7f2afed9d640 1 --2- 172.21.15.45:0/1254498901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aec004d40 0x7f2aec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.879+0000 7f2afe59c640 1 --2- 172.21.15.45:0/1254498901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2aec003a30 0x7f2aec003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.879+0000 7f2af5d9b640 1 --2- 172.21.15.45:0/1254498901 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b000b6590 0x7f2aec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.879+0000 7f2afed9d640 1 -- 172.21.15.45:0/1254498901 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b000b6590 msgr2=0x7f2aec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.879+0000 7f2afed9d640 1 --2- 172.21.15.45:0/1254498901 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b000b6590 0x7f2aec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.879+0000 7f2afed9d640 1 -- 172.21.15.45:0/1254498901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2aec003a30 msgr2=0x7f2aec003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.879+0000 7f2afed9d640 1 --2- 172.21.15.45:0/1254498901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2aec003a30 0x7f2aec003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.879+0000 7f2afed9d640 1 -- 172.21.15.45:0/1254498901 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec003520 con 0x7f2aec004d40 2024-05-08T05:44:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.880+0000 7f2afed9d640 1 --2- 172.21.15.45:0/1254498901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aec004d40 0x7f2aec0051c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2af800a830 tx=0x7f2aec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0c91ebe84e53b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.880+0000 7f2afdd9b640 1 -- 172.21.15.45:0/1254498901 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af800a450 con 0x7f2aec004d40 2024-05-08T05:44:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.880+0000 7f2afdd9b640 1 -- 172.21.15.45:0/1254498901 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af802f070 con 0x7f2aec004d40 2024-05-08T05:44:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.880+0000 7f2afdd9b640 1 -- 172.21.15.45:0/1254498901 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af802bc00 con 0x7f2aec004d40 2024-05-08T05:44:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.880+0000 7f2b04951640 1 -- 172.21.15.45:0/1254498901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aec004d40 msgr2=0x7f2aec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.880+0000 7f2b04951640 1 --2- 172.21.15.45:0/1254498901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aec004d40 0x7f2aec0051c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2af800a830 tx=0x7f2aec005830 comp rx=0 tx=0).stop 2024-05-08T05:44:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.880+0000 7f2afed9d640 1 -- 172.21.15.45:0/1254498901 reap_dead start 2024-05-08T05:44:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.880+0000 7f2b04951640 1 -- 172.21.15.45:0/1254498901 shutdown_connections 2024-05-08T05:44:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.880+0000 7f2b04951640 1 -- 172.21.15.45:0/1254498901 >> 172.21.15.45:0/1254498901 conn(0x7f2b000d4fe0 msgr2=0x7f2b000d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.880+0000 7f2b04951640 1 -- 172.21.15.45:0/1254498901 shutdown_connections 2024-05-08T05:44:00.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.880+0000 7f2b04951640 1 -- 172.21.15.45:0/1254498901 wait complete. 2024-05-08T05:44:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.882+0000 7f2b04951640 1 Processor -- start 2024-05-08T05:44:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.882+0000 7f2b04951640 1 -- start start 2024-05-08T05:44:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.883+0000 7f2b04951640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aec004d40 0x7f2b00158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.883+0000 7f2b04951640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b001e96f0 0x7f2b0015b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.883+0000 7f2b04951640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b001e9b70 0x7f2b0015bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.883+0000 7f2b04951640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2b000b9080 con 0x7f2aec004d40 2024-05-08T05:44:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.883+0000 7f2b04951640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b000d3010 con 0x7f2b001e9b70 2024-05-08T05:44:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.883+0000 7f2b04951640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b000b9200 con 0x7f2b001e96f0 2024-05-08T05:44:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.883+0000 7f2afe59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aec004d40 0x7f2b00158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.883+0000 7f2afe59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aec004d40 0x7f2b00158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36602/0 (socket says 172.21.15.45:36602) 2024-05-08T05:44:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.883+0000 7f2afe59c640 1 -- 172.21.15.45:0/1343828583 learned_addr learned my addr 172.21.15.45:0/1343828583 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.883+0000 7f2afed9d640 1 --2- 172.21.15.45:0/1343828583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b001e9b70 0x7f2b0015bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.883+0000 7f2af5d9b640 1 --2- 172.21.15.45:0/1343828583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b001e96f0 0x7f2b0015b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.883+0000 7f2afe59c640 1 -- 172.21.15.45:0/1343828583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b001e96f0 msgr2=0x7f2b0015b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.883+0000 7f2afe59c640 1 --2- 172.21.15.45:0/1343828583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b001e96f0 0x7f2b0015b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.883+0000 7f2afe59c640 1 -- 172.21.15.45:0/1343828583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b001e9b70 msgr2=0x7f2b0015bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.883+0000 7f2afe59c640 1 --2- 172.21.15.45:0/1343828583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b001e9b70 0x7f2b0015bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.883+0000 7f2afe59c640 1 -- 172.21.15.45:0/1343828583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b00159930 con 0x7f2aec004d40 2024-05-08T05:44:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.883+0000 7f2afe59c640 1 --2- 172.21.15.45:0/1343828583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aec004d40 0x7f2b00158060 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2af00119a0 tx=0x7f2af0011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.884+0000 7f2af77fe640 1 -- 172.21.15.45:0/1343828583 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af000fa90 con 0x7f2aec004d40 2024-05-08T05:44:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.884+0000 7f2af77fe640 1 -- 172.21.15.45:0/1343828583 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af000fc30 con 0x7f2aec004d40 2024-05-08T05:44:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.884+0000 7f2b04951640 1 -- 172.21.15.45:0/1343828583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b00159c10 con 0x7f2aec004d40 2024-05-08T05:44:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.884+0000 7f2af77fe640 1 -- 172.21.15.45:0/1343828583 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0014440 con 0x7f2aec004d40 2024-05-08T05:44:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.884+0000 7f2b04951640 1 -- 172.21.15.45:0/1343828583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b0015a1d0 con 0x7f2aec004d40 2024-05-08T05:44:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.885+0000 7f2b04951640 1 -- 172.21.15.45:0/1343828583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b000693f0 con 0x7f2aec004d40 2024-05-08T05:44:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.885+0000 7f2af77fe640 1 -- 172.21.15.45:0/1343828583 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2af0012040 con 0x7f2aec004d40 2024-05-08T05:44:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.886+0000 7f2af77fe640 1 --2- 172.21.15.45:0/1343828583 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2ae0076300 0x7f2ae00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.886+0000 7f2af77fe640 1 -- 172.21.15.45:0/1343828583 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f2af0097d30 con 0x7f2aec004d40 2024-05-08T05:44:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.886+0000 7f2af5d9b640 1 --2- 172.21.15.45:0/1343828583 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2ae0076300 0x7f2ae00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.887+0000 7f2af5d9b640 1 --2- 172.21.15.45:0/1343828583 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2ae0076300 0x7f2ae00787c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8000c00 tx=0x7f2ae8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:00.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:00.890+0000 7f2af77fe640 1 -- 172.21.15.45:0/1343828583 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2af0019030 con 0x7f2aec004d40 2024-05-08T05:44:01.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.119+0000 7f2b04951640 1 -- 172.21.15.45:0/1343828583 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f2b0006df40 con 0x7f2aec004d40 2024-05-08T05:44:01.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.751+0000 7f2af77fe640 1 -- 172.21.15.45:0/1343828583 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f2af0061500 con 0x7f2aec004d40 2024-05-08T05:44:01.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.754+0000 7f2b04951640 1 -- 172.21.15.45:0/1343828583 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2ae0076300 msgr2=0x7f2ae00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:01.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.754+0000 7f2b04951640 1 --2- 172.21.15.45:0/1343828583 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2ae0076300 0x7f2ae00787c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8000c00 tx=0x7f2ae8009040 comp rx=0 tx=0).stop 2024-05-08T05:44:01.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.754+0000 7f2b04951640 1 -- 172.21.15.45:0/1343828583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aec004d40 msgr2=0x7f2b00158060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:01.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.754+0000 7f2b04951640 1 --2- 172.21.15.45:0/1343828583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aec004d40 0x7f2b00158060 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2af00119a0 tx=0x7f2af0011e60 comp rx=0 tx=0).stop 2024-05-08T05:44:01.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.754+0000 7f2b04951640 1 -- 172.21.15.45:0/1343828583 shutdown_connections 2024-05-08T05:44:01.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.754+0000 7f2b04951640 1 --2- 172.21.15.45:0/1343828583 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2ae0076300 0x7f2ae00787c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:01.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.754+0000 7f2b04951640 1 --2- 172.21.15.45:0/1343828583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b001e96f0 0x7f2b0015b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:01.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.754+0000 7f2b04951640 1 --2- 172.21.15.45:0/1343828583 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b001e9b70 0x7f2b0015bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:01.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.754+0000 7f2b04951640 1 --2- 172.21.15.45:0/1343828583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aec004d40 0x7f2b00158060 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:01.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.754+0000 7f2b04951640 1 -- 172.21.15.45:0/1343828583 >> 172.21.15.45:0/1343828583 conn(0x7f2b000d4fe0 msgr2=0x7f2b0006d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:01.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.754+0000 7f2b04951640 1 -- 172.21.15.45:0/1343828583 shutdown_connections 2024-05-08T05:44:01.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.754+0000 7f2b04951640 1 -- 172.21.15.45:0/1343828583 wait complete. 2024-05-08T05:44:01.768 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T05:44:01.768 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:44:01.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.879+0000 7f3a4ffff640 1 Processor -- start 2024-05-08T05:44:01.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.879+0000 7f3a4ffff640 1 -- start start 2024-05-08T05:44:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.880+0000 7f3a4ffff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a5015f480 0x7f3a500d1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.880+0000 7f3a4ffff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3a500c8a90 0x7f3a500c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:01.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.880+0000 7f3a4ffff640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a500b6330 con 0x7f3a500d5a60 2024-05-08T05:44:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.880+0000 7f3a4ffff640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3a500d0c40 con 0x7f3a500c8a90 2024-05-08T05:44:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.880+0000 7f3a4ffff640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3a500b6610 con 0x7f3a5015f480 2024-05-08T05:44:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.880+0000 7f3a4effd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a5015f480 0x7f3a500d1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.880+0000 7f3a4f7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3a500c8a90 0x7f3a500c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.880+0000 7f3a4e7fc640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f3a500d5a60 0x7f3a500d1ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:36684/0 (socket says 172.21.15.45:36684) 2024-05-08T05:44:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.880+0000 7f3a4e7fc640 1 -- 172.21.15.45:0/1914158371 learned_addr learned my addr 172.21.15.45:0/1914158371 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:44:01.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.881+0000 7f3a4f7fe640 1 -- 172.21.15.45:0/1914158371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a5015f480 msgr2=0x7f3a500d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:01.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.881+0000 7f3a4f7fe640 1 --2- 172.21.15.45:0/1914158371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a5015f480 0x7f3a500d1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.881+0000 7f3a4f7fe640 1 -- 172.21.15.45:0/1914158371 >> v1:172.21.15.63:6789/0 conn(0x7f3a500d5a60 legacy=0x7f3a500d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.881+0000 7f3a4f7fe640 1 -- 172.21.15.45:0/1914158371 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a500d05f0 con 0x7f3a500c8a90 2024-05-08T05:44:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.881+0000 7f3a4effd640 1 --2- 172.21.15.45:0/1914158371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a5015f480 0x7f3a500d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:44:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.881+0000 7f3a4f7fe640 1 --2- 172.21.15.45:0/1914158371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a500c8a90 0x7f3a500c8f20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009d90 tx=0x7f3a4802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b8e1002e943f0d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.882+0000 7f3a4dffb640 1 -- 172.21.15.45:0/1914158371 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48007740 con 0x7f3a500c8a90 2024-05-08T05:44:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.882+0000 7f3a4dffb640 1 -- 172.21.15.45:0/1914158371 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a480078e0 con 0x7f3a500c8a90 2024-05-08T05:44:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.882+0000 7f3a4dffb640 1 -- 172.21.15.45:0/1914158371 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48007cb0 con 0x7f3a500c8a90 2024-05-08T05:44:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.882+0000 7f3a4ffff640 1 -- 172.21.15.45:0/1914158371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a500c8a90 msgr2=0x7f3a500c8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.882+0000 7f3a4ffff640 1 --2- 172.21.15.45:0/1914158371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a500c8a90 0x7f3a500c8f20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009d90 tx=0x7f3a4802a6f0 comp rx=0 tx=0).stop 2024-05-08T05:44:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.882+0000 7f3a4ffff640 1 -- 172.21.15.45:0/1914158371 shutdown_connections 2024-05-08T05:44:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.882+0000 7f3a4ffff640 1 --2- 172.21.15.45:0/1914158371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a500c8a90 0x7f3a500c8f20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.882+0000 7f3a4ffff640 1 --2- 172.21.15.45:0/1914158371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a5015f480 0x7f3a500d1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:01.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.882+0000 7f3a4ffff640 1 -- 172.21.15.45:0/1914158371 >> 172.21.15.45:0/1914158371 conn(0x7f3a500d4f40 msgr2=0x7f3a500d5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.882+0000 7f3a4ffff640 1 -- 172.21.15.45:0/1914158371 shutdown_connections 2024-05-08T05:44:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.882+0000 7f3a4ffff640 1 -- 172.21.15.45:0/1914158371 wait complete. 2024-05-08T05:44:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.884+0000 7f3a4ffff640 1 Processor -- start 2024-05-08T05:44:01.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.884+0000 7f3a4ffff640 1 -- start start 2024-05-08T05:44:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.884+0000 7f3a4ffff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a500c8a90 0x7f3a500b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.884+0000 7f3a4ffff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a500d5a60 0x7f3a50144c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.885+0000 7f3a4ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a5015f480 0x7f3a50161ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.885+0000 7f3a4ffff640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3a500d21c0 con 0x7f3a5015f480 2024-05-08T05:44:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.885+0000 7f3a4ffff640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3a500b6000 con 0x7f3a500d5a60 2024-05-08T05:44:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.885+0000 7f3a4ffff640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3a500d2340 con 0x7f3a500c8a90 2024-05-08T05:44:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.885+0000 7f3a4f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a5015f480 0x7f3a50161ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.885+0000 7f3a4f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a5015f480 0x7f3a50161ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36604/0 (socket says 172.21.15.45:36604) 2024-05-08T05:44:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.885+0000 7f3a4f7fe640 1 -- 172.21.15.45:0/2060243674 learned_addr learned my addr 172.21.15.45:0/2060243674 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.885+0000 7f3a4e7fc640 1 --2- 172.21.15.45:0/2060243674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a500d5a60 0x7f3a50144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.885+0000 7f3a4effd640 1 --2- 172.21.15.45:0/2060243674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a500c8a90 0x7f3a500b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.885+0000 7f3a4f7fe640 1 -- 172.21.15.45:0/2060243674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a500c8a90 msgr2=0x7f3a500b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.885+0000 7f3a4f7fe640 1 --2- 172.21.15.45:0/2060243674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a500c8a90 0x7f3a500b7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.885+0000 7f3a4f7fe640 1 -- 172.21.15.45:0/2060243674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a500d5a60 msgr2=0x7f3a50144c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.885+0000 7f3a4f7fe640 1 --2- 172.21.15.45:0/2060243674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a500d5a60 0x7f3a50144c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.885+0000 7f3a4f7fe640 1 -- 172.21.15.45:0/2060243674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a500cdd50 con 0x7f3a5015f480 2024-05-08T05:44:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.885+0000 7f3a4f7fe640 1 --2- 172.21.15.45:0/2060243674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a5015f480 0x7f3a50161ca0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3a4802ace0 tx=0x7f3a48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.885+0000 7f3a377fe640 1 -- 172.21.15.45:0/2060243674 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48040070 con 0x7f3a5015f480 2024-05-08T05:44:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.885+0000 7f3a377fe640 1 -- 172.21.15.45:0/2060243674 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a48031bc0 con 0x7f3a5015f480 2024-05-08T05:44:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.885+0000 7f3a4ffff640 1 -- 172.21.15.45:0/2060243674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a50145550 con 0x7f3a5015f480 2024-05-08T05:44:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.885+0000 7f3a377fe640 1 -- 172.21.15.45:0/2060243674 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48009450 con 0x7f3a5015f480 2024-05-08T05:44:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.886+0000 7f3a4ffff640 1 -- 172.21.15.45:0/2060243674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a500cdfc0 con 0x7f3a5015f480 2024-05-08T05:44:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.887+0000 7f3a4ffff640 1 -- 172.21.15.45:0/2060243674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a500ce1c0 con 0x7f3a5015f480 2024-05-08T05:44:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.887+0000 7f3a377fe640 1 -- 172.21.15.45:0/2060243674 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3a48039040 con 0x7f3a5015f480 2024-05-08T05:44:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.888+0000 7f3a377fe640 1 --2- 172.21.15.45:0/2060243674 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3a3c0762d0 0x7f3a3c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.888+0000 7f3a377fe640 1 -- 172.21.15.45:0/2060243674 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f3a480b8b70 con 0x7f3a5015f480 2024-05-08T05:44:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.888+0000 7f3a4effd640 1 --2- 172.21.15.45:0/2060243674 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3a3c0762d0 0x7f3a3c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.888+0000 7f3a4effd640 1 --2- 172.21.15.45:0/2060243674 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3a3c0762d0 0x7f3a3c078790 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009560 tx=0x7f3a40009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:01.892+0000 7f3a377fe640 1 -- 172.21.15.45:0/2060243674 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3a500ce1c0 con 0x7f3a5015f480 2024-05-08T05:44:02.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.123+0000 7f3a4ffff640 1 -- 172.21.15.45:0/2060243674 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a50050740 con 0x7f3a5015f480 2024-05-08T05:44:02.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.124+0000 7f3a377fe640 1 -- 172.21.15.45:0/2060243674 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5179 (secure 0 0 0) 0x7f3a4804c030 con 0x7f3a5015f480 2024-05-08T05:44:02.125 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:44:02.125 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/162765657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":162765657},{"type":"v1","addr":"172.21.15.63:6836","nonce":162765657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5356,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1788513331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1788513331},{"type":"v1","addr":"172.21.15.45:6834","nonce":1788513331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1173167340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1173167340},{"type":"v1","addr":"172.21.15.63:6837","nonce":1173167340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:43:58.107819+0000","modified":"2024-05-08T05:44:01.121276+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1461822016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1461822016},{"type":"v1","addr":"172.21.15.45:6835","nonce":1461822016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5697,"qdb_cluster":[5697]},"id":4}]} 2024-05-08T05:44:02.125 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 34 2024-05-08T05:44:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.126+0000 7f3a4ffff640 1 -- 172.21.15.45:0/2060243674 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3a3c0762d0 msgr2=0x7f3a3c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.126+0000 7f3a4ffff640 1 --2- 172.21.15.45:0/2060243674 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3a3c0762d0 0x7f3a3c078790 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009560 tx=0x7f3a40009190 comp rx=0 tx=0).stop 2024-05-08T05:44:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.126+0000 7f3a4ffff640 1 -- 172.21.15.45:0/2060243674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a5015f480 msgr2=0x7f3a50161ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.126+0000 7f3a4ffff640 1 --2- 172.21.15.45:0/2060243674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a5015f480 0x7f3a50161ca0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3a4802ace0 tx=0x7f3a48004a40 comp rx=0 tx=0).stop 2024-05-08T05:44:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.126+0000 7f3a4ffff640 1 -- 172.21.15.45:0/2060243674 shutdown_connections 2024-05-08T05:44:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.126+0000 7f3a4ffff640 1 --2- 172.21.15.45:0/2060243674 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3a3c0762d0 0x7f3a3c078790 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.126+0000 7f3a4ffff640 1 --2- 172.21.15.45:0/2060243674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a5015f480 0x7f3a50161ca0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.126+0000 7f3a4ffff640 1 --2- 172.21.15.45:0/2060243674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a500d5a60 0x7f3a50144c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.126+0000 7f3a4ffff640 1 --2- 172.21.15.45:0/2060243674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a500c8a90 0x7f3a500b7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.126+0000 7f3a4ffff640 1 -- 172.21.15.45:0/2060243674 >> 172.21.15.45:0/2060243674 conn(0x7f3a500d4f40 msgr2=0x7f3a500d5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.126+0000 7f3a4ffff640 1 -- 172.21.15.45:0/2060243674 shutdown_connections 2024-05-08T05:44:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.126+0000 7f3a4ffff640 1 -- 172.21.15.45:0/2060243674 wait complete. 2024-05-08T05:44:02.138 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:44:02.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.241+0000 7fe3f1283640 1 Processor -- start 2024-05-08T05:44:02.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.241+0000 7fe3f1283640 1 -- start start 2024-05-08T05:44:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.242+0000 7fe3f1283640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3ec1629a0 0x7fe3ec0d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.242+0000 7fe3f1283640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe3ec0d2dc0 0x7fe3ec0d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.242+0000 7fe3f1283640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3ec0d3330 con 0x7fe3ec0c8ed0 2024-05-08T05:44:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.242+0000 7fe3f1283640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe3ec0d2170 con 0x7fe3ec0d2dc0 2024-05-08T05:44:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.242+0000 7fe3f1283640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe3ec0d2450 con 0x7fe3ec1629a0 2024-05-08T05:44:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.242+0000 7fe3e3fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe3ec0d2dc0 0x7fe3ec0d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.242+0000 7fe3eb7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe3ec0c8ed0 0x7fe3ec0c9360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53512/0 (socket says 172.21.15.45:53512) 2024-05-08T05:44:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.242+0000 7fe3eb7fe640 1 -- 172.21.15.45:0/2550192371 learned_addr learned my addr 172.21.15.45:0/2550192371 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:44:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.242+0000 7fe3eaffd640 1 --2- 172.21.15.45:0/2550192371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3ec1629a0 0x7fe3ec0d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:02.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.242+0000 7fe3e3fff640 1 -- 172.21.15.45:0/2550192371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3ec1629a0 msgr2=0x7fe3ec0d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.242+0000 7fe3e3fff640 1 --2- 172.21.15.45:0/2550192371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3ec1629a0 0x7fe3ec0d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.242+0000 7fe3e3fff640 1 -- 172.21.15.45:0/2550192371 >> v1:172.21.15.45:6789/0 conn(0x7fe3ec0c8ed0 legacy=0x7fe3ec0c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.242+0000 7fe3e3fff640 1 -- 172.21.15.45:0/2550192371 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ec0d2730 con 0x7fe3ec0d2dc0 2024-05-08T05:44:02.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.242+0000 7fe3ea7fc640 1 -- 172.21.15.45:0/2550192371 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1633758057 0 0) 0x7fe3ec0d3330 con 0x7fe3ec0c8ed0 2024-05-08T05:44:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.243+0000 7fe3e3fff640 1 --2- 172.21.15.45:0/2550192371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3ec0d2dc0 0x7fe3ec0d3b00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40097a0 tx=0x7fe3d402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db08d522d5a1f648 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:02.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.243+0000 7fe3ea7fc640 1 -- 172.21.15.45:0/2550192371 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4037070 con 0x7fe3ec0d2dc0 2024-05-08T05:44:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.243+0000 7fe3ea7fc640 1 -- 172.21.15.45:0/2550192371 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3d402adb0 con 0x7fe3ec0d2dc0 2024-05-08T05:44:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.243+0000 7fe3ea7fc640 1 -- 172.21.15.45:0/2550192371 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4032660 con 0x7fe3ec0d2dc0 2024-05-08T05:44:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.244+0000 7fe3f1283640 1 -- 172.21.15.45:0/2550192371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3ec0d2dc0 msgr2=0x7fe3ec0d3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.244+0000 7fe3f1283640 1 --2- 172.21.15.45:0/2550192371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3ec0d2dc0 0x7fe3ec0d3b00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40097a0 tx=0x7fe3d402a060 comp rx=0 tx=0).stop 2024-05-08T05:44:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.244+0000 7fe3f1283640 1 -- 172.21.15.45:0/2550192371 shutdown_connections 2024-05-08T05:44:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.244+0000 7fe3f1283640 1 --2- 172.21.15.45:0/2550192371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3ec0d2dc0 0x7fe3ec0d3b00 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.244+0000 7fe3f1283640 1 --2- 172.21.15.45:0/2550192371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3ec1629a0 0x7fe3ec0d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.244+0000 7fe3f1283640 1 -- 172.21.15.45:0/2550192371 >> 172.21.15.45:0/2550192371 conn(0x7fe3ec0d4fe0 msgr2=0x7fe3ec0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.244+0000 7fe3f1283640 1 -- 172.21.15.45:0/2550192371 shutdown_connections 2024-05-08T05:44:02.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.244+0000 7fe3f1283640 1 -- 172.21.15.45:0/2550192371 wait complete. 2024-05-08T05:44:02.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.245+0000 7fe3f1283640 1 Processor -- start 2024-05-08T05:44:02.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.245+0000 7fe3f1283640 1 -- start start 2024-05-08T05:44:02.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.246+0000 7fe3f1283640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3ec0c8ed0 0x7fe3ec1f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.246+0000 7fe3f1283640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3ec0d2dc0 0x7fe3ec15e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.246+0000 7fe3f1283640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3ec1629a0 0x7fe3ec06e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.246+0000 7fe3f1283640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec0cf440 con 0x7fe3ec1629a0 2024-05-08T05:44:02.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.246+0000 7fe3f1283640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ec0cf2c0 con 0x7fe3ec0d2dc0 2024-05-08T05:44:02.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.246+0000 7fe3f1283640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe3ec0d0cb0 con 0x7fe3ec0c8ed0 2024-05-08T05:44:02.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.246+0000 7fe3eb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3ec1629a0 0x7fe3ec06e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:02.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.246+0000 7fe3eb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3ec1629a0 0x7fe3ec06e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36628/0 (socket says 172.21.15.45:36628) 2024-05-08T05:44:02.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.246+0000 7fe3eb7fe640 1 -- 172.21.15.45:0/1749535338 learned_addr learned my addr 172.21.15.45:0/1749535338 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:02.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.246+0000 7fe3eaffd640 1 --2- 172.21.15.45:0/1749535338 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3ec0c8ed0 0x7fe3ec1f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:02.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.246+0000 7fe3e3fff640 1 --2- 172.21.15.45:0/1749535338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3ec0d2dc0 0x7fe3ec15e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:02.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.246+0000 7fe3eb7fe640 1 -- 172.21.15.45:0/1749535338 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3ec0c8ed0 msgr2=0x7fe3ec1f4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:02.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.246+0000 7fe3eb7fe640 1 --2- 172.21.15.45:0/1749535338 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3ec0c8ed0 0x7fe3ec1f4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.246+0000 7fe3eb7fe640 1 -- 172.21.15.45:0/1749535338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3ec0d2dc0 msgr2=0x7fe3ec15e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:02.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.246+0000 7fe3eb7fe640 1 --2- 172.21.15.45:0/1749535338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3ec0d2dc0 0x7fe3ec15e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.246+0000 7fe3eb7fe640 1 -- 172.21.15.45:0/1749535338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ec06d000 con 0x7fe3ec1629a0 2024-05-08T05:44:02.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.246+0000 7fe3eb7fe640 1 --2- 172.21.15.45:0/1749535338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3ec1629a0 0x7fe3ec06e0a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400c7a0 tx=0x7fe3e400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:02.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.247+0000 7fe3e37fe640 1 -- 172.21.15.45:0/1749535338 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4018070 con 0x7fe3ec1629a0 2024-05-08T05:44:02.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.247+0000 7fe3e37fe640 1 -- 172.21.15.45:0/1749535338 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3e400a5e0 con 0x7fe3ec1629a0 2024-05-08T05:44:02.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.247+0000 7fe3f1283640 1 -- 172.21.15.45:0/1749535338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3ec06d2e0 con 0x7fe3ec1629a0 2024-05-08T05:44:02.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.247+0000 7fe3e37fe640 1 -- 172.21.15.45:0/1749535338 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e400aa50 con 0x7fe3ec1629a0 2024-05-08T05:44:02.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.247+0000 7fe3f1283640 1 -- 172.21.15.45:0/1749535338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3ec06d8a0 con 0x7fe3ec1629a0 2024-05-08T05:44:02.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.248+0000 7fe3f1283640 1 -- 172.21.15.45:0/1749535338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3ec0693f0 con 0x7fe3ec1629a0 2024-05-08T05:44:02.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.248+0000 7fe3e37fe640 1 -- 172.21.15.45:0/1749535338 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe3e400abf0 con 0x7fe3ec1629a0 2024-05-08T05:44:02.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.249+0000 7fe3e37fe640 1 --2- 172.21.15.45:0/1749535338 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe3cc0762d0 0x7fe3cc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.249+0000 7fe3e37fe640 1 -- 172.21.15.45:0/1749535338 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7fe3e4096d80 con 0x7fe3ec1629a0 2024-05-08T05:44:02.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.249+0000 7fe3eaffd640 1 --2- 172.21.15.45:0/1749535338 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe3cc0762d0 0x7fe3cc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:02.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.249+0000 7fe3eaffd640 1 --2- 172.21.15.45:0/1749535338 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe3cc0762d0 0x7fe3cc078790 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc0079a0 tx=0x7fe3dc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:02.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.253+0000 7fe3e37fe640 1 -- 172.21.15.45:0/1749535338 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe3e405fe70 con 0x7fe3ec1629a0 2024-05-08T05:44:02.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.446+0000 7fe3f1283640 1 -- 172.21.15.45:0/1749535338 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3ec0763f0 con 0x7fe3ec1629a0 2024-05-08T05:44:02.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.447+0000 7fe3e37fe640 1 -- 172.21.15.45:0/1749535338 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+16836 (secure 0 0 0) 0x7fe3e4064170 con 0x7fe3ec1629a0 2024-05-08T05:44:02.448 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:44:02.448 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":61,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:43:59.737695+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:43:55.774105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-08T05:43:57.070302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:44:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.449+0000 7fe3f1283640 1 -- 172.21.15.45:0/1749535338 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe3cc0762d0 msgr2=0x7fe3cc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.449+0000 7fe3f1283640 1 --2- 172.21.15.45:0/1749535338 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe3cc0762d0 0x7fe3cc078790 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc0079a0 tx=0x7fe3dc008040 comp rx=0 tx=0).stop 2024-05-08T05:44:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.449+0000 7fe3f1283640 1 -- 172.21.15.45:0/1749535338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3ec1629a0 msgr2=0x7fe3ec06e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.449+0000 7fe3f1283640 1 --2- 172.21.15.45:0/1749535338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3ec1629a0 0x7fe3ec06e0a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400c7a0 tx=0x7fe3e400cc60 comp rx=0 tx=0).stop 2024-05-08T05:44:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.449+0000 7fe3f1283640 1 -- 172.21.15.45:0/1749535338 shutdown_connections 2024-05-08T05:44:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.449+0000 7fe3f1283640 1 --2- 172.21.15.45:0/1749535338 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe3cc0762d0 0x7fe3cc078790 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.449+0000 7fe3f1283640 1 --2- 172.21.15.45:0/1749535338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3ec1629a0 0x7fe3ec06e0a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.449+0000 7fe3f1283640 1 --2- 172.21.15.45:0/1749535338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3ec0d2dc0 0x7fe3ec15e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.449+0000 7fe3f1283640 1 --2- 172.21.15.45:0/1749535338 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe3ec0c8ed0 0x7fe3ec1f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.449+0000 7fe3f1283640 1 -- 172.21.15.45:0/1749535338 >> 172.21.15.45:0/1749535338 conn(0x7fe3ec0d4fe0 msgr2=0x7fe3ec0d35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:02.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.449+0000 7fe3f1283640 1 -- 172.21.15.45:0/1749535338 shutdown_connections 2024-05-08T05:44:02.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.449+0000 7fe3f1283640 1 -- 172.21.15.45:0/1749535338 wait complete. 2024-05-08T05:44:02.461 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T05:44:02.461 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:44:02.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.565+0000 7f7e33ac8640 1 Processor -- start 2024-05-08T05:44:02.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.565+0000 7f7e33ac8640 1 -- start start 2024-05-08T05:44:02.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.566+0000 7f7e33ac8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7e2c072840 0x7f7e2c15c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.566+0000 7f7e33ac8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e2c06e4d0 0x7f7e2c0aec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.566+0000 7f7e33ac8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e2c15c920 con 0x7f7e2c06e0f0 2024-05-08T05:44:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.566+0000 7f7e33ac8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f7e2c15cc00 con 0x7f7e2c072840 2024-05-08T05:44:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.566+0000 7f7e33ac8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7e2c15cee0 con 0x7f7e2c06e4d0 2024-05-08T05:44:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.566+0000 7f7e31063640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7e2c072840 0x7f7e2c15c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.566+0000 7f7e31063640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7e2c072840 0x7f7e2c15c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36632/0 (socket says 172.21.15.45:36632) 2024-05-08T05:44:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.566+0000 7f7e31063640 1 -- 172.21.15.45:0/3611099314 learned_addr learned my addr 172.21.15.45:0/3611099314 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.566+0000 7f7e32065640 1 --2- 172.21.15.45:0/3611099314 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e2c06e4d0 0x7f7e2c0aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.566+0000 7f7e31063640 1 -- 172.21.15.45:0/3611099314 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e2c06e4d0 msgr2=0x7f7e2c0aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.566+0000 7f7e31063640 1 --2- 172.21.15.45:0/3611099314 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e2c06e4d0 0x7f7e2c0aec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.566+0000 7f7e31063640 1 -- 172.21.15.45:0/3611099314 >> v1:172.21.15.63:6789/0 conn(0x7f7e2c06e0f0 legacy=0x7f7e2c0d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.566+0000 7f7e31063640 1 -- 172.21.15.45:0/3611099314 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2c15d1c0 con 0x7f7e2c072840 2024-05-08T05:44:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.567+0000 7f7e31063640 1 --2- 172.21.15.45:0/3611099314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e2c072840 0x7f7e2c15c340 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009870 tx=0x7f7e1c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978bd954dea5b890 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:02.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.567+0000 7f7e30862640 1 -- 172.21.15.45:0/3611099314 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c037070 con 0x7f7e2c072840 2024-05-08T05:44:02.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.567+0000 7f7e30862640 1 -- 172.21.15.45:0/3611099314 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e1c032440 con 0x7f7e2c072840 2024-05-08T05:44:02.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.567+0000 7f7e30862640 1 -- 172.21.15.45:0/3611099314 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c032810 con 0x7f7e2c072840 2024-05-08T05:44:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.568+0000 7f7e33ac8640 1 -- 172.21.15.45:0/3611099314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e2c072840 msgr2=0x7f7e2c15c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.568+0000 7f7e33ac8640 1 --2- 172.21.15.45:0/3611099314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e2c072840 0x7f7e2c15c340 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009870 tx=0x7f7e1c02a1c0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.568+0000 7f7e33ac8640 1 -- 172.21.15.45:0/3611099314 shutdown_connections 2024-05-08T05:44:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.568+0000 7f7e33ac8640 1 --2- 172.21.15.45:0/3611099314 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e2c06e4d0 0x7f7e2c0aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.568+0000 7f7e33ac8640 1 --2- 172.21.15.45:0/3611099314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e2c072840 0x7f7e2c15c340 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.568+0000 7f7e33ac8640 1 -- 172.21.15.45:0/3611099314 >> 172.21.15.45:0/3611099314 conn(0x7f7e2c06c3a0 msgr2=0x7f7e2c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.568+0000 7f7e33ac8640 1 -- 172.21.15.45:0/3611099314 shutdown_connections 2024-05-08T05:44:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.568+0000 7f7e33ac8640 1 -- 172.21.15.45:0/3611099314 wait complete. 2024-05-08T05:44:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.569+0000 7f7e33ac8640 1 Processor -- start 2024-05-08T05:44:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.569+0000 7f7e33ac8640 1 -- start start 2024-05-08T05:44:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.570+0000 7f7e33ac8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e2c06e0f0 0x7f7e2c150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.570+0000 7f7e33ac8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e2c06e4d0 0x7f7e2c09dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.570+0000 7f7e33ac8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e2c072840 0x7f7e2c09e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.570+0000 7f7e33ac8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c0ae7d0 con 0x7f7e2c06e0f0 2024-05-08T05:44:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.570+0000 7f7e33ac8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c0ae650 con 0x7f7e2c072840 2024-05-08T05:44:02.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.570+0000 7f7e33ac8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7e2c1530b0 con 0x7f7e2c06e4d0 2024-05-08T05:44:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.570+0000 7f7e31864640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e2c06e0f0 0x7f7e2c150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.570+0000 7f7e31864640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e2c06e0f0 0x7f7e2c150800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36644/0 (socket says 172.21.15.45:36644) 2024-05-08T05:44:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.570+0000 7f7e31864640 1 -- 172.21.15.45:0/2934359234 learned_addr learned my addr 172.21.15.45:0/2934359234 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.570+0000 7f7e32065640 1 --2- 172.21.15.45:0/2934359234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e2c072840 0x7f7e2c09e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.570+0000 7f7e31063640 1 --2- 172.21.15.45:0/2934359234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e2c06e4d0 0x7f7e2c09dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.570+0000 7f7e31864640 1 -- 172.21.15.45:0/2934359234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e2c06e4d0 msgr2=0x7f7e2c09dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.570+0000 7f7e31864640 1 --2- 172.21.15.45:0/2934359234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e2c06e4d0 0x7f7e2c09dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.570+0000 7f7e31864640 1 -- 172.21.15.45:0/2934359234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e2c072840 msgr2=0x7f7e2c09e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.570+0000 7f7e31864640 1 --2- 172.21.15.45:0/2934359234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e2c072840 0x7f7e2c09e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.570+0000 7f7e31864640 1 -- 172.21.15.45:0/2934359234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2c09eb70 con 0x7f7e2c06e0f0 2024-05-08T05:44:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.571+0000 7f7e31864640 1 --2- 172.21.15.45:0/2934359234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e2c06e0f0 0x7f7e2c150800 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800c7c0 tx=0x7f7e2800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.571+0000 7f7e227fc640 1 -- 172.21.15.45:0/2934359234 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e28016020 con 0x7f7e2c06e0f0 2024-05-08T05:44:02.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.571+0000 7f7e227fc640 1 -- 172.21.15.45:0/2934359234 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e2800aa50 con 0x7f7e2c06e0f0 2024-05-08T05:44:02.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.571+0000 7f7e33ac8640 1 -- 172.21.15.45:0/2934359234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2c0ce120 con 0x7f7e2c06e0f0 2024-05-08T05:44:02.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.571+0000 7f7e227fc640 1 -- 172.21.15.45:0/2934359234 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e28012420 con 0x7f7e2c06e0f0 2024-05-08T05:44:02.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.571+0000 7f7e33ac8640 1 -- 172.21.15.45:0/2934359234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e2c0ce3c0 con 0x7f7e2c06e0f0 2024-05-08T05:44:02.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.572+0000 7f7e33ac8640 1 -- 172.21.15.45:0/2934359234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e2c0693f0 con 0x7f7e2c06e0f0 2024-05-08T05:44:02.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.572+0000 7f7e227fc640 1 -- 172.21.15.45:0/2934359234 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7e28019020 con 0x7f7e2c06e0f0 2024-05-08T05:44:02.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.573+0000 7f7e227fc640 1 --2- 172.21.15.45:0/2934359234 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7e100762d0 0x7f7e10078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.573+0000 7f7e227fc640 1 -- 172.21.15.45:0/2934359234 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f7e28097610 con 0x7f7e2c06e0f0 2024-05-08T05:44:02.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.573+0000 7f7e31063640 1 --2- 172.21.15.45:0/2934359234 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7e100762d0 0x7f7e10078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:02.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.574+0000 7f7e31063640 1 --2- 172.21.15.45:0/2934359234 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7e100762d0 0x7f7e10078790 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c034000 tx=0x7f7e1c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:02.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.577+0000 7f7e227fc640 1 -- 172.21.15.45:0/2934359234 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7e28020030 con 0x7f7e2c06e0f0 2024-05-08T05:44:02.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.764+0000 7f7e33ac8640 1 -- 172.21.15.45:0/2934359234 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e2c0cd170 con 0x7f7e100762d0 2024-05-08T05:44:02.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.771+0000 7f7e227fc640 1 -- 172.21.15.45:0/2934359234 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251938 (secure 0 0 0) 0x7f7e2c0cd170 con 0x7f7e100762d0 2024-05-08T05:44:02.774 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:44:02.775 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:44:02.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.774+0000 7f7e33ac8640 1 -- 172.21.15.45:0/2934359234 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7e100762d0 msgr2=0x7f7e10078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:02.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.774+0000 7f7e33ac8640 1 --2- 172.21.15.45:0/2934359234 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7e100762d0 0x7f7e10078790 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c034000 tx=0x7f7e1c02e000 comp rx=0 tx=0).stop 2024-05-08T05:44:02.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.774+0000 7f7e33ac8640 1 -- 172.21.15.45:0/2934359234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e2c06e0f0 msgr2=0x7f7e2c150800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:02.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.774+0000 7f7e33ac8640 1 --2- 172.21.15.45:0/2934359234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e2c06e0f0 0x7f7e2c150800 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800c7c0 tx=0x7f7e2800cc80 comp rx=0 tx=0).stop 2024-05-08T05:44:02.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.774+0000 7f7e33ac8640 1 -- 172.21.15.45:0/2934359234 shutdown_connections 2024-05-08T05:44:02.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.774+0000 7f7e33ac8640 1 --2- 172.21.15.45:0/2934359234 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7e100762d0 0x7f7e10078790 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.774+0000 7f7e33ac8640 1 --2- 172.21.15.45:0/2934359234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7e2c06e4d0 0x7f7e2c09dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.774+0000 7f7e33ac8640 1 --2- 172.21.15.45:0/2934359234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e2c072840 0x7f7e2c09e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.774+0000 7f7e33ac8640 1 --2- 172.21.15.45:0/2934359234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7e2c06e0f0 0x7f7e2c150800 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.774+0000 7f7e33ac8640 1 -- 172.21.15.45:0/2934359234 >> 172.21.15.45:0/2934359234 conn(0x7f7e2c06c3a0 msgr2=0x7f7e2c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:02.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.774+0000 7f7e33ac8640 1 -- 172.21.15.45:0/2934359234 shutdown_connections 2024-05-08T05:44:02.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.774+0000 7f7e33ac8640 1 -- 172.21.15.45:0/2934359234 wait complete. 2024-05-08T05:44:02.785 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-05-08T05:44:00.867541+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239592,"kb_used_data":6756,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735128,"statfs":{"total":773094113280,"available":772848771072,"internally_reserved":0,"allocated":6918144,"data_stored":3717221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.148417"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.761138+0000","last_change":"2024-05-08T05:43:58.753503+0000","last_active":"2024-05-08T05:43:58.761138+0000","last_peered":"2024-05-08T05:43:58.761138+0000","last_clean":"2024-05-08T05:43:58.761138+0000","last_became_active":"2024-05-08T05:43:58.753143+0000","last_became_peered":"2024-05-08T05:43:58.753143+0000","last_unstale":"2024-05-08T05:43:58.761138+0000","last_undegraded":"2024-05-08T05:43:58.761138+0000","last_fullsized":"2024-05-08T05:43:58.761138+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:53:55.951106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.759032+0000","last_change":"2024-05-08T05:43:58.750946+0000","last_active":"2024-05-08T05:43:58.759032+0000","last_peered":"2024-05-08T05:43:58.759032+0000","last_clean":"2024-05-08T05:43:58.759032+0000","last_became_active":"2024-05-08T05:43:58.750589+0000","last_became_peered":"2024-05-08T05:43:58.750589+0000","last_unstale":"2024-05-08T05:43:58.759032+0000","last_undegraded":"2024-05-08T05:43:58.759032+0000","last_fullsized":"2024-05-08T05:43:58.759032+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:57.495679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.760090+0000","last_change":"2024-05-08T05:43:58.752044+0000","last_active":"2024-05-08T05:43:58.760090+0000","last_peered":"2024-05-08T05:43:58.760090+0000","last_clean":"2024-05-08T05:43:58.760090+0000","last_became_active":"2024-05-08T05:43:58.751755+0000","last_became_peered":"2024-05-08T05:43:58.751755+0000","last_unstale":"2024-05-08T05:43:58.760090+0000","last_undegraded":"2024-05-08T05:43:58.760090+0000","last_fullsized":"2024-05-08T05:43:58.760090+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:04:07.485759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.745286+0000","last_change":"2024-05-08T05:43:58.744176+0000","last_active":"2024-05-08T05:43:58.745286+0000","last_peered":"2024-05-08T05:43:58.745286+0000","last_clean":"2024-05-08T05:43:58.745286+0000","last_became_active":"2024-05-08T05:43:58.743872+0000","last_became_peered":"2024-05-08T05:43:58.743872+0000","last_unstale":"2024-05-08T05:43:58.745286+0000","last_undegraded":"2024-05-08T05:43:58.745286+0000","last_fullsized":"2024-05-08T05:43:58.745286+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:21:42.939708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.755315+0000","last_change":"2024-05-08T05:43:58.749306+0000","last_active":"2024-05-08T05:43:58.755315+0000","last_peered":"2024-05-08T05:43:58.755315+0000","last_clean":"2024-05-08T05:43:58.755315+0000","last_became_active":"2024-05-08T05:43:58.748977+0000","last_became_peered":"2024-05-08T05:43:58.748977+0000","last_unstale":"2024-05-08T05:43:58.755315+0000","last_undegraded":"2024-05-08T05:43:58.755315+0000","last_fullsized":"2024-05-08T05:43:58.755315+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:40:37.076365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.762025+0000","last_change":"2024-05-08T05:43:58.757238+0000","last_active":"2024-05-08T05:43:58.762025+0000","last_peered":"2024-05-08T05:43:58.762025+0000","last_clean":"2024-05-08T05:43:58.762025+0000","last_became_active":"2024-05-08T05:43:58.756387+0000","last_became_peered":"2024-05-08T05:43:58.756387+0000","last_unstale":"2024-05-08T05:43:58.762025+0000","last_undegraded":"2024-05-08T05:43:58.762025+0000","last_fullsized":"2024-05-08T05:43:58.762025+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:20.614792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.758788+0000","last_change":"2024-05-08T05:43:58.757279+0000","last_active":"2024-05-08T05:43:58.758788+0000","last_peered":"2024-05-08T05:43:58.758788+0000","last_clean":"2024-05-08T05:43:58.758788+0000","last_became_active":"2024-05-08T05:43:58.756996+0000","last_became_peered":"2024-05-08T05:43:58.756996+0000","last_unstale":"2024-05-08T05:43:58.758788+0000","last_undegraded":"2024-05-08T05:43:58.758788+0000","last_fullsized":"2024-05-08T05:43:58.758788+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:03:15.648434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.755986+0000","last_change":"2024-05-08T05:43:58.754887+0000","last_active":"2024-05-08T05:43:58.755986+0000","last_peered":"2024-05-08T05:43:58.755986+0000","last_clean":"2024-05-08T05:43:58.755986+0000","last_became_active":"2024-05-08T05:43:58.754613+0000","last_became_peered":"2024-05-08T05:43:58.754613+0000","last_unstale":"2024-05-08T05:43:58.755986+0000","last_undegraded":"2024-05-08T05:43:58.755986+0000","last_fullsized":"2024-05-08T05:43:58.755986+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:28:19.201137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.757202+0000","last_change":"2024-05-08T05:43:58.751431+0000","last_active":"2024-05-08T05:43:58.757202+0000","last_peered":"2024-05-08T05:43:58.757202+0000","last_clean":"2024-05-08T05:43:58.757202+0000","last_became_active":"2024-05-08T05:43:58.751130+0000","last_became_peered":"2024-05-08T05:43:58.751130+0000","last_unstale":"2024-05-08T05:43:58.757202+0000","last_undegraded":"2024-05-08T05:43:58.757202+0000","last_fullsized":"2024-05-08T05:43:58.757202+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:24:58.636003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.754313+0000","last_change":"2024-05-08T05:43:58.746264+0000","last_active":"2024-05-08T05:43:58.754313+0000","last_peered":"2024-05-08T05:43:58.754313+0000","last_clean":"2024-05-08T05:43:58.754313+0000","last_became_active":"2024-05-08T05:43:58.745987+0000","last_became_peered":"2024-05-08T05:43:58.745987+0000","last_unstale":"2024-05-08T05:43:58.754313+0000","last_undegraded":"2024-05-08T05:43:58.754313+0000","last_fullsized":"2024-05-08T05:43:58.754313+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:17:33.940374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.759921+0000","last_change":"2024-05-08T05:43:58.752489+0000","last_active":"2024-05-08T05:43:58.759921+0000","last_peered":"2024-05-08T05:43:58.759921+0000","last_clean":"2024-05-08T05:43:58.759921+0000","last_became_active":"2024-05-08T05:43:58.752209+0000","last_became_peered":"2024-05-08T05:43:58.752209+0000","last_unstale":"2024-05-08T05:43:58.759921+0000","last_undegraded":"2024-05-08T05:43:58.759921+0000","last_fullsized":"2024-05-08T05:43:58.759921+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:18:07.653809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.756342+0000","last_change":"2024-05-08T05:43:58.754735+0000","last_active":"2024-05-08T05:43:58.756342+0000","last_peered":"2024-05-08T05:43:58.756342+0000","last_clean":"2024-05-08T05:43:58.756342+0000","last_became_active":"2024-05-08T05:43:58.754424+0000","last_became_peered":"2024-05-08T05:43:58.754424+0000","last_unstale":"2024-05-08T05:43:58.756342+0000","last_undegraded":"2024-05-08T05:43:58.756342+0000","last_fullsized":"2024-05-08T05:43:58.756342+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:46:44.645171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.751340+0000","last_change":"2024-05-08T05:43:58.745342+0000","last_active":"2024-05-08T05:43:58.751340+0000","last_peered":"2024-05-08T05:43:58.751340+0000","last_clean":"2024-05-08T05:43:58.751340+0000","last_became_active":"2024-05-08T05:43:58.744849+0000","last_became_peered":"2024-05-08T05:43:58.744849+0000","last_unstale":"2024-05-08T05:43:58.751340+0000","last_undegraded":"2024-05-08T05:43:58.751340+0000","last_fullsized":"2024-05-08T05:43:58.751340+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:04:25.561526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.762366+0000","last_change":"2024-05-08T05:43:58.753424+0000","last_active":"2024-05-08T05:43:58.762366+0000","last_peered":"2024-05-08T05:43:58.762366+0000","last_clean":"2024-05-08T05:43:58.762366+0000","last_became_active":"2024-05-08T05:43:58.751183+0000","last_became_peered":"2024-05-08T05:43:58.751183+0000","last_unstale":"2024-05-08T05:43:58.762366+0000","last_undegraded":"2024-05-08T05:43:58.762366+0000","last_fullsized":"2024-05-08T05:43:58.762366+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:55:24.366110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.762474+0000","last_change":"2024-05-08T05:43:58.757260+0000","last_active":"2024-05-08T05:43:58.762474+0000","last_peered":"2024-05-08T05:43:58.762474+0000","last_clean":"2024-05-08T05:43:58.762474+0000","last_became_active":"2024-05-08T05:43:58.754636+0000","last_became_peered":"2024-05-08T05:43:58.754636+0000","last_unstale":"2024-05-08T05:43:58.762474+0000","last_undegraded":"2024-05-08T05:43:58.762474+0000","last_fullsized":"2024-05-08T05:43:58.762474+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:43:45.044208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.760439+0000","last_change":"2024-05-08T05:43:58.759412+0000","last_active":"2024-05-08T05:43:58.760439+0000","last_peered":"2024-05-08T05:43:58.760439+0000","last_clean":"2024-05-08T05:43:58.760439+0000","last_became_active":"2024-05-08T05:43:58.758810+0000","last_became_peered":"2024-05-08T05:43:58.758810+0000","last_unstale":"2024-05-08T05:43:58.760439+0000","last_undegraded":"2024-05-08T05:43:58.760439+0000","last_fullsized":"2024-05-08T05:43:58.760439+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:42:13.925868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.757985+0000","last_change":"2024-05-08T05:43:58.751072+0000","last_active":"2024-05-08T05:43:58.757985+0000","last_peered":"2024-05-08T05:43:58.757985+0000","last_clean":"2024-05-08T05:43:58.757985+0000","last_became_active":"2024-05-08T05:43:58.750764+0000","last_became_peered":"2024-05-08T05:43:58.750764+0000","last_unstale":"2024-05-08T05:43:58.757985+0000","last_undegraded":"2024-05-08T05:43:58.757985+0000","last_fullsized":"2024-05-08T05:43:58.757985+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:42:39.962581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.745245+0000","last_change":"2024-05-08T05:43:58.743256+0000","last_active":"2024-05-08T05:43:58.745245+0000","last_peered":"2024-05-08T05:43:58.745245+0000","last_clean":"2024-05-08T05:43:58.745245+0000","last_became_active":"2024-05-08T05:43:58.742972+0000","last_became_peered":"2024-05-08T05:43:58.742972+0000","last_unstale":"2024-05-08T05:43:58.745245+0000","last_undegraded":"2024-05-08T05:43:58.745245+0000","last_fullsized":"2024-05-08T05:43:58.745245+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:07.409112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.755474+0000","last_change":"2024-05-08T05:43:58.752037+0000","last_active":"2024-05-08T05:43:58.755474+0000","last_peered":"2024-05-08T05:43:58.755474+0000","last_clean":"2024-05-08T05:43:58.755474+0000","last_became_active":"2024-05-08T05:43:58.751754+0000","last_became_peered":"2024-05-08T05:43:58.751754+0000","last_unstale":"2024-05-08T05:43:58.755474+0000","last_undegraded":"2024-05-08T05:43:58.755474+0000","last_fullsized":"2024-05-08T05:43:58.755474+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:02:31.259717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.754299+0000","last_change":"2024-05-08T05:43:58.750358+0000","last_active":"2024-05-08T05:43:58.754299+0000","last_peered":"2024-05-08T05:43:58.754299+0000","last_clean":"2024-05-08T05:43:58.754299+0000","last_became_active":"2024-05-08T05:43:58.749996+0000","last_became_peered":"2024-05-08T05:43:58.749996+0000","last_unstale":"2024-05-08T05:43:58.754299+0000","last_undegraded":"2024-05-08T05:43:58.754299+0000","last_fullsized":"2024-05-08T05:43:58.754299+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:34:19.446859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.765084+0000","last_change":"2024-05-08T05:43:58.761863+0000","last_active":"2024-05-08T05:43:58.765084+0000","last_peered":"2024-05-08T05:43:58.765084+0000","last_clean":"2024-05-08T05:43:58.765084+0000","last_became_active":"2024-05-08T05:43:58.761485+0000","last_became_peered":"2024-05-08T05:43:58.761485+0000","last_unstale":"2024-05-08T05:43:58.765084+0000","last_undegraded":"2024-05-08T05:43:58.765084+0000","last_fullsized":"2024-05-08T05:43:58.765084+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:52:49.114300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.759342+0000","last_change":"2024-05-08T05:43:58.757769+0000","last_active":"2024-05-08T05:43:58.759342+0000","last_peered":"2024-05-08T05:43:58.759342+0000","last_clean":"2024-05-08T05:43:58.759342+0000","last_became_active":"2024-05-08T05:43:58.757464+0000","last_became_peered":"2024-05-08T05:43:58.757464+0000","last_unstale":"2024-05-08T05:43:58.759342+0000","last_undegraded":"2024-05-08T05:43:58.759342+0000","last_fullsized":"2024-05-08T05:43:58.759342+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:03:49.694754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.758248+0000","last_change":"2024-05-08T05:43:58.757175+0000","last_active":"2024-05-08T05:43:58.758248+0000","last_peered":"2024-05-08T05:43:58.758248+0000","last_clean":"2024-05-08T05:43:58.758248+0000","last_became_active":"2024-05-08T05:43:58.756875+0000","last_became_peered":"2024-05-08T05:43:58.756875+0000","last_unstale":"2024-05-08T05:43:58.758248+0000","last_undegraded":"2024-05-08T05:43:58.758248+0000","last_fullsized":"2024-05-08T05:43:58.758248+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:27.264718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.762325+0000","last_change":"2024-05-08T05:43:58.752234+0000","last_active":"2024-05-08T05:43:58.762325+0000","last_peered":"2024-05-08T05:43:58.762325+0000","last_clean":"2024-05-08T05:43:58.762325+0000","last_became_active":"2024-05-08T05:43:58.751741+0000","last_became_peered":"2024-05-08T05:43:58.751741+0000","last_unstale":"2024-05-08T05:43:58.762325+0000","last_undegraded":"2024-05-08T05:43:58.762325+0000","last_fullsized":"2024-05-08T05:43:58.762325+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:04.995283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.757261+0000","last_change":"2024-05-08T05:43:58.755471+0000","last_active":"2024-05-08T05:43:58.757261+0000","last_peered":"2024-05-08T05:43:58.757261+0000","last_clean":"2024-05-08T05:43:58.757261+0000","last_became_active":"2024-05-08T05:43:58.755202+0000","last_became_peered":"2024-05-08T05:43:58.755202+0000","last_unstale":"2024-05-08T05:43:58.757261+0000","last_undegraded":"2024-05-08T05:43:58.757261+0000","last_fullsized":"2024-05-08T05:43:58.757261+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:08.842871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.761748+0000","last_change":"2024-05-08T05:43:58.756270+0000","last_active":"2024-05-08T05:43:58.761748+0000","last_peered":"2024-05-08T05:43:58.761748+0000","last_clean":"2024-05-08T05:43:58.761748+0000","last_became_active":"2024-05-08T05:43:58.755984+0000","last_became_peered":"2024-05-08T05:43:58.755984+0000","last_unstale":"2024-05-08T05:43:58.761748+0000","last_undegraded":"2024-05-08T05:43:58.761748+0000","last_fullsized":"2024-05-08T05:43:58.761748+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:15.841767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.759947+0000","last_change":"2024-05-08T05:43:58.752918+0000","last_active":"2024-05-08T05:43:58.759947+0000","last_peered":"2024-05-08T05:43:58.759947+0000","last_clean":"2024-05-08T05:43:58.759947+0000","last_became_active":"2024-05-08T05:43:58.752313+0000","last_became_peered":"2024-05-08T05:43:58.752313+0000","last_unstale":"2024-05-08T05:43:58.759947+0000","last_undegraded":"2024-05-08T05:43:58.759947+0000","last_fullsized":"2024-05-08T05:43:58.759947+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:54.358135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.761035+0000","last_change":"2024-05-08T05:43:58.752352+0000","last_active":"2024-05-08T05:43:58.761035+0000","last_peered":"2024-05-08T05:43:58.761035+0000","last_clean":"2024-05-08T05:43:58.761035+0000","last_became_active":"2024-05-08T05:43:58.751509+0000","last_became_peered":"2024-05-08T05:43:58.751509+0000","last_unstale":"2024-05-08T05:43:58.761035+0000","last_undegraded":"2024-05-08T05:43:58.761035+0000","last_fullsized":"2024-05-08T05:43:58.761035+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:23:28.230535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.757684+0000","last_change":"2024-05-08T05:43:58.747056+0000","last_active":"2024-05-08T05:43:58.757684+0000","last_peered":"2024-05-08T05:43:58.757684+0000","last_clean":"2024-05-08T05:43:58.757684+0000","last_became_active":"2024-05-08T05:43:58.746787+0000","last_became_peered":"2024-05-08T05:43:58.746787+0000","last_unstale":"2024-05-08T05:43:58.757684+0000","last_undegraded":"2024-05-08T05:43:58.757684+0000","last_fullsized":"2024-05-08T05:43:58.757684+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:55:31.453030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.756990+0000","last_change":"2024-05-08T05:43:58.755599+0000","last_active":"2024-05-08T05:43:58.756990+0000","last_peered":"2024-05-08T05:43:58.756990+0000","last_clean":"2024-05-08T05:43:58.756990+0000","last_became_active":"2024-05-08T05:43:58.755316+0000","last_became_peered":"2024-05-08T05:43:58.755316+0000","last_unstale":"2024-05-08T05:43:58.756990+0000","last_undegraded":"2024-05-08T05:43:58.756990+0000","last_fullsized":"2024-05-08T05:43:58.756990+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:54.581514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.754003+0000","last_change":"2024-05-08T05:43:58.750875+0000","last_active":"2024-05-08T05:43:58.754003+0000","last_peered":"2024-05-08T05:43:58.754003+0000","last_clean":"2024-05-08T05:43:58.754003+0000","last_became_active":"2024-05-08T05:43:58.750571+0000","last_became_peered":"2024-05-08T05:43:58.750571+0000","last_unstale":"2024-05-08T05:43:58.754003+0000","last_undegraded":"2024-05-08T05:43:58.754003+0000","last_fullsized":"2024-05-08T05:43:58.754003+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:39.363056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.755606+0000","last_change":"2024-05-08T05:43:58.754415+0000","last_active":"2024-05-08T05:43:58.755606+0000","last_peered":"2024-05-08T05:43:58.755606+0000","last_clean":"2024-05-08T05:43:58.755606+0000","last_became_active":"2024-05-08T05:43:58.754116+0000","last_became_peered":"2024-05-08T05:43:58.754116+0000","last_unstale":"2024-05-08T05:43:58.755606+0000","last_undegraded":"2024-05-08T05:43:58.755606+0000","last_fullsized":"2024-05-08T05:43:58.755606+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:55:24.366110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.764353+0000","last_change":"2024-05-08T05:43:58.758314+0000","last_active":"2024-05-08T05:43:58.764353+0000","last_peered":"2024-05-08T05:43:58.764353+0000","last_clean":"2024-05-08T05:43:58.764353+0000","last_became_active":"2024-05-08T05:43:58.758016+0000","last_became_peered":"2024-05-08T05:43:58.758016+0000","last_unstale":"2024-05-08T05:43:58.764353+0000","last_undegraded":"2024-05-08T05:43:58.764353+0000","last_fullsized":"2024-05-08T05:43:58.764353+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:13:37.766411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"60'4","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.763002+0000","last_change":"2024-05-08T05:43:57.744551+0000","last_active":"2024-05-08T05:43:58.763002+0000","last_peered":"2024-05-08T05:43:58.763002+0000","last_clean":"2024-05-08T05:43:58.763002+0000","last_became_active":"2024-05-08T05:43:57.744255+0000","last_became_peered":"2024-05-08T05:43:57.744255+0000","last_unstale":"2024-05-08T05:43:58.763002+0000","last_undegraded":"2024-05-08T05:43:58.763002+0000","last_fullsized":"2024-05-08T05:43:58.763002+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:42:04.835594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.759456+0000","last_change":"2024-05-08T05:43:58.751868+0000","last_active":"2024-05-08T05:43:58.759456+0000","last_peered":"2024-05-08T05:43:58.759456+0000","last_clean":"2024-05-08T05:43:58.759456+0000","last_became_active":"2024-05-08T05:43:58.750775+0000","last_became_peered":"2024-05-08T05:43:58.750775+0000","last_unstale":"2024-05-08T05:43:58.759456+0000","last_undegraded":"2024-05-08T05:43:58.759456+0000","last_fullsized":"2024-05-08T05:43:58.759456+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:42:54.541460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.755834+0000","last_change":"2024-05-08T05:43:57.743388+0000","last_active":"2024-05-08T05:43:58.755834+0000","last_peered":"2024-05-08T05:43:58.755834+0000","last_clean":"2024-05-08T05:43:58.755834+0000","last_became_active":"2024-05-08T05:43:57.743068+0000","last_became_peered":"2024-05-08T05:43:57.743068+0000","last_unstale":"2024-05-08T05:43:58.755834+0000","last_undegraded":"2024-05-08T05:43:58.755834+0000","last_fullsized":"2024-05-08T05:43:58.755834+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:04:06.483452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.761474+0000","last_change":"2024-05-08T05:43:58.756116+0000","last_active":"2024-05-08T05:43:58.761474+0000","last_peered":"2024-05-08T05:43:58.761474+0000","last_clean":"2024-05-08T05:43:58.761474+0000","last_became_active":"2024-05-08T05:43:58.755813+0000","last_became_peered":"2024-05-08T05:43:58.755813+0000","last_unstale":"2024-05-08T05:43:58.761474+0000","last_undegraded":"2024-05-08T05:43:58.761474+0000","last_fullsized":"2024-05-08T05:43:58.761474+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:43:45.044208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.758795+0000","last_change":"2024-05-08T05:43:57.747701+0000","last_active":"2024-05-08T05:43:58.758795+0000","last_peered":"2024-05-08T05:43:58.758795+0000","last_clean":"2024-05-08T05:43:58.758795+0000","last_became_active":"2024-05-08T05:43:57.747194+0000","last_became_peered":"2024-05-08T05:43:57.747194+0000","last_unstale":"2024-05-08T05:43:58.758795+0000","last_undegraded":"2024-05-08T05:43:58.758795+0000","last_fullsized":"2024-05-08T05:43:58.758795+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:55:30.450723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.762468+0000","last_change":"2024-05-08T05:43:58.756883+0000","last_active":"2024-05-08T05:43:58.762468+0000","last_peered":"2024-05-08T05:43:58.762468+0000","last_clean":"2024-05-08T05:43:58.762468+0000","last_became_active":"2024-05-08T05:43:58.753172+0000","last_became_peered":"2024-05-08T05:43:58.753172+0000","last_unstale":"2024-05-08T05:43:58.762468+0000","last_undegraded":"2024-05-08T05:43:58.762468+0000","last_fullsized":"2024-05-08T05:43:58.762468+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:32:38.835058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.759979+0000","last_change":"2024-05-08T05:43:57.733816+0000","last_active":"2024-05-08T05:43:58.759979+0000","last_peered":"2024-05-08T05:43:58.759979+0000","last_clean":"2024-05-08T05:43:58.759979+0000","last_became_active":"2024-05-08T05:43:57.733506+0000","last_became_peered":"2024-05-08T05:43:57.733506+0000","last_unstale":"2024-05-08T05:43:58.759979+0000","last_undegraded":"2024-05-08T05:43:58.759979+0000","last_fullsized":"2024-05-08T05:43:58.759979+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:14.839460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.758003+0000","last_change":"2024-05-08T05:43:58.755042+0000","last_active":"2024-05-08T05:43:58.758003+0000","last_peered":"2024-05-08T05:43:58.758003+0000","last_clean":"2024-05-08T05:43:58.758003+0000","last_became_active":"2024-05-08T05:43:58.754741+0000","last_became_peered":"2024-05-08T05:43:58.754741+0000","last_unstale":"2024-05-08T05:43:58.758003+0000","last_undegraded":"2024-05-08T05:43:58.758003+0000","last_fullsized":"2024-05-08T05:43:58.758003+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:31:26.967780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.746253+0000","last_change":"2024-05-08T05:43:57.735454+0000","last_active":"2024-05-08T05:43:58.746253+0000","last_peered":"2024-05-08T05:43:58.746253+0000","last_clean":"2024-05-08T05:43:58.746253+0000","last_became_active":"2024-05-08T05:43:57.735160+0000","last_became_peered":"2024-05-08T05:43:57.735160+0000","last_unstale":"2024-05-08T05:43:58.746253+0000","last_undegraded":"2024-05-08T05:43:58.746253+0000","last_fullsized":"2024-05-08T05:43:58.746253+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:24:20.210950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.757762+0000","last_change":"2024-05-08T05:43:58.751432+0000","last_active":"2024-05-08T05:43:58.757762+0000","last_peered":"2024-05-08T05:43:58.757762+0000","last_clean":"2024-05-08T05:43:58.757762+0000","last_became_active":"2024-05-08T05:43:58.751121+0000","last_became_peered":"2024-05-08T05:43:58.751121+0000","last_unstale":"2024-05-08T05:43:58.757762+0000","last_undegraded":"2024-05-08T05:43:58.757762+0000","last_fullsized":"2024-05-08T05:43:58.757762+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:41.803526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.749553+0000","last_change":"2024-05-08T05:43:57.736625+0000","last_active":"2024-05-08T05:43:58.749553+0000","last_peered":"2024-05-08T05:43:58.749553+0000","last_clean":"2024-05-08T05:43:58.749553+0000","last_became_active":"2024-05-08T05:43:57.736294+0000","last_became_peered":"2024-05-08T05:43:57.736294+0000","last_unstale":"2024-05-08T05:43:58.749553+0000","last_undegraded":"2024-05-08T05:43:58.749553+0000","last_fullsized":"2024-05-08T05:43:58.749553+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:13:36.124929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.754421+0000","last_change":"2024-05-08T05:43:58.749825+0000","last_active":"2024-05-08T05:43:58.754421+0000","last_peered":"2024-05-08T05:43:58.754421+0000","last_clean":"2024-05-08T05:43:58.754421+0000","last_became_active":"2024-05-08T05:43:58.749544+0000","last_became_peered":"2024-05-08T05:43:58.749544+0000","last_unstale":"2024-05-08T05:43:58.754421+0000","last_undegraded":"2024-05-08T05:43:58.754421+0000","last_fullsized":"2024-05-08T05:43:58.754421+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:16:07.239208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.729951+0000","last_change":"2024-05-08T05:43:57.749640+0000","last_active":"2024-05-08T05:43:58.729951+0000","last_peered":"2024-05-08T05:43:58.729951+0000","last_clean":"2024-05-08T05:43:58.729951+0000","last_became_active":"2024-05-08T05:43:57.749307+0000","last_became_peered":"2024-05-08T05:43:57.749307+0000","last_unstale":"2024-05-08T05:43:58.729951+0000","last_undegraded":"2024-05-08T05:43:58.729951+0000","last_fullsized":"2024-05-08T05:43:58.729951+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:27.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.743229+0000","last_change":"2024-05-08T05:43:58.741349+0000","last_active":"2024-05-08T05:43:58.743229+0000","last_peered":"2024-05-08T05:43:58.743229+0000","last_clean":"2024-05-08T05:43:58.743229+0000","last_became_active":"2024-05-08T05:43:58.740934+0000","last_became_peered":"2024-05-08T05:43:58.740934+0000","last_unstale":"2024-05-08T05:43:58.743229+0000","last_undegraded":"2024-05-08T05:43:58.743229+0000","last_fullsized":"2024-05-08T05:43:58.743229+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:44:39.758942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.739656+0000","last_change":"2024-05-08T05:43:57.748752+0000","last_active":"2024-05-08T05:43:58.739656+0000","last_peered":"2024-05-08T05:43:58.739656+0000","last_clean":"2024-05-08T05:43:58.739656+0000","last_became_active":"2024-05-08T05:43:57.748472+0000","last_became_peered":"2024-05-08T05:43:57.748472+0000","last_unstale":"2024-05-08T05:43:58.739656+0000","last_undegraded":"2024-05-08T05:43:58.739656+0000","last_fullsized":"2024-05-08T05:43:58.739656+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:38.360748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.758999+0000","last_change":"2024-05-08T05:43:58.750812+0000","last_active":"2024-05-08T05:43:58.758999+0000","last_peered":"2024-05-08T05:43:58.758999+0000","last_clean":"2024-05-08T05:43:58.758999+0000","last_became_active":"2024-05-08T05:43:58.750509+0000","last_became_peered":"2024-05-08T05:43:58.750509+0000","last_unstale":"2024-05-08T05:43:58.758999+0000","last_undegraded":"2024-05-08T05:43:58.758999+0000","last_fullsized":"2024-05-08T05:43:58.758999+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:43:14.229950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.738722+0000","last_change":"2024-05-08T05:43:57.746791+0000","last_active":"2024-05-08T05:43:58.738722+0000","last_peered":"2024-05-08T05:43:58.738722+0000","last_clean":"2024-05-08T05:43:58.738722+0000","last_became_active":"2024-05-08T05:43:57.746493+0000","last_became_peered":"2024-05-08T05:43:57.746493+0000","last_unstale":"2024-05-08T05:43:58.738722+0000","last_undegraded":"2024-05-08T05:43:58.738722+0000","last_fullsized":"2024-05-08T05:43:58.738722+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:14.839460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.748962+0000","last_change":"2024-05-08T05:43:58.747505+0000","last_active":"2024-05-08T05:43:58.748962+0000","last_peered":"2024-05-08T05:43:58.748962+0000","last_clean":"2024-05-08T05:43:58.748962+0000","last_became_active":"2024-05-08T05:43:58.747135+0000","last_became_peered":"2024-05-08T05:43:58.747135+0000","last_unstale":"2024-05-08T05:43:58.748962+0000","last_undegraded":"2024-05-08T05:43:58.748962+0000","last_fullsized":"2024-05-08T05:43:58.748962+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:20:05.922731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.732033+0000","last_change":"2024-05-08T05:43:57.742766+0000","last_active":"2024-05-08T05:43:58.732033+0000","last_peered":"2024-05-08T05:43:58.732033+0000","last_clean":"2024-05-08T05:43:58.732033+0000","last_became_active":"2024-05-08T05:43:57.742473+0000","last_became_peered":"2024-05-08T05:43:57.742473+0000","last_unstale":"2024-05-08T05:43:58.732033+0000","last_undegraded":"2024-05-08T05:43:58.732033+0000","last_fullsized":"2024-05-08T05:43:58.732033+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:48:09.974150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.756241+0000","last_change":"2024-05-08T05:43:58.752828+0000","last_active":"2024-05-08T05:43:58.756241+0000","last_peered":"2024-05-08T05:43:58.756241+0000","last_clean":"2024-05-08T05:43:58.756241+0000","last_became_active":"2024-05-08T05:43:58.752545+0000","last_became_peered":"2024-05-08T05:43:58.752545+0000","last_unstale":"2024-05-08T05:43:58.756241+0000","last_undegraded":"2024-05-08T05:43:58.756241+0000","last_fullsized":"2024-05-08T05:43:58.756241+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:14:23.447592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"60'3","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.761358+0000","last_change":"2024-05-08T05:43:57.747237+0000","last_active":"2024-05-08T05:43:58.761358+0000","last_peered":"2024-05-08T05:43:58.761358+0000","last_clean":"2024-05-08T05:43:58.761358+0000","last_became_active":"2024-05-08T05:43:57.746946+0000","last_became_peered":"2024-05-08T05:43:57.746946+0000","last_unstale":"2024-05-08T05:43:58.761358+0000","last_undegraded":"2024-05-08T05:43:58.761358+0000","last_fullsized":"2024-05-08T05:43:58.761358+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:07.840563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.745309+0000","last_change":"2024-05-08T05:43:58.743511+0000","last_active":"2024-05-08T05:43:58.745309+0000","last_peered":"2024-05-08T05:43:58.745309+0000","last_clean":"2024-05-08T05:43:58.745309+0000","last_became_active":"2024-05-08T05:43:58.743200+0000","last_became_peered":"2024-05-08T05:43:58.743200+0000","last_unstale":"2024-05-08T05:43:58.745309+0000","last_undegraded":"2024-05-08T05:43:58.745309+0000","last_fullsized":"2024-05-08T05:43:58.745309+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:20:41.789503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"60'6","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.765418+0000","last_change":"2024-05-08T05:43:57.741712+0000","last_active":"2024-05-08T05:43:58.765418+0000","last_peered":"2024-05-08T05:43:58.765418+0000","last_clean":"2024-05-08T05:43:58.765418+0000","last_became_active":"2024-05-08T05:43:57.741244+0000","last_became_peered":"2024-05-08T05:43:57.741244+0000","last_unstale":"2024-05-08T05:43:58.765418+0000","last_undegraded":"2024-05-08T05:43:58.765418+0000","last_fullsized":"2024-05-08T05:43:58.765418+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:52.963839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.754036+0000","last_change":"2024-05-08T05:43:58.750653+0000","last_active":"2024-05-08T05:43:58.754036+0000","last_peered":"2024-05-08T05:43:58.754036+0000","last_clean":"2024-05-08T05:43:58.754036+0000","last_became_active":"2024-05-08T05:43:58.750351+0000","last_became_peered":"2024-05-08T05:43:58.750351+0000","last_unstale":"2024-05-08T05:43:58.754036+0000","last_undegraded":"2024-05-08T05:43:58.754036+0000","last_fullsized":"2024-05-08T05:43:58.754036+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:26:16.801149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.759076+0000","last_change":"2024-05-08T05:43:57.751419+0000","last_active":"2024-05-08T05:43:58.759076+0000","last_peered":"2024-05-08T05:43:58.759076+0000","last_clean":"2024-05-08T05:43:58.759076+0000","last_became_active":"2024-05-08T05:43:57.751058+0000","last_became_peered":"2024-05-08T05:43:57.751058+0000","last_unstale":"2024-05-08T05:43:58.759076+0000","last_undegraded":"2024-05-08T05:43:58.759076+0000","last_fullsized":"2024-05-08T05:43:58.759076+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:28:18.198830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.747530+0000","last_change":"2024-05-08T05:43:58.744627+0000","last_active":"2024-05-08T05:43:58.747530+0000","last_peered":"2024-05-08T05:43:58.747530+0000","last_clean":"2024-05-08T05:43:58.747530+0000","last_became_active":"2024-05-08T05:43:58.744329+0000","last_became_peered":"2024-05-08T05:43:58.744329+0000","last_unstale":"2024-05-08T05:43:58.747530+0000","last_undegraded":"2024-05-08T05:43:58.747530+0000","last_fullsized":"2024-05-08T05:43:58.747530+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:54.914888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.738119+0000","last_change":"2024-05-08T05:43:57.740618+0000","last_active":"2024-05-08T05:43:58.738119+0000","last_peered":"2024-05-08T05:43:58.738119+0000","last_clean":"2024-05-08T05:43:58.738119+0000","last_became_active":"2024-05-08T05:43:57.740317+0000","last_became_peered":"2024-05-08T05:43:57.740317+0000","last_unstale":"2024-05-08T05:43:58.738119+0000","last_undegraded":"2024-05-08T05:43:58.738119+0000","last_fullsized":"2024-05-08T05:43:58.738119+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:53:54.948798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.754441+0000","last_change":"2024-05-08T05:43:58.751349+0000","last_active":"2024-05-08T05:43:58.754441+0000","last_peered":"2024-05-08T05:43:58.754441+0000","last_clean":"2024-05-08T05:43:58.754441+0000","last_became_active":"2024-05-08T05:43:58.751043+0000","last_became_peered":"2024-05-08T05:43:58.751043+0000","last_unstale":"2024-05-08T05:43:58.754441+0000","last_undegraded":"2024-05-08T05:43:58.754441+0000","last_fullsized":"2024-05-08T05:43:58.754441+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:30:31.262850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.741282+0000","last_change":"2024-05-08T05:43:57.748020+0000","last_active":"2024-05-08T05:43:58.741282+0000","last_peered":"2024-05-08T05:43:58.741282+0000","last_clean":"2024-05-08T05:43:58.741282+0000","last_became_active":"2024-05-08T05:43:57.746146+0000","last_became_peered":"2024-05-08T05:43:57.746146+0000","last_unstale":"2024-05-08T05:43:58.741282+0000","last_undegraded":"2024-05-08T05:43:58.741282+0000","last_fullsized":"2024-05-08T05:43:58.741282+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:52:48.111992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.761229+0000","last_change":"2024-05-08T05:43:58.753970+0000","last_active":"2024-05-08T05:43:58.761229+0000","last_peered":"2024-05-08T05:43:58.761229+0000","last_clean":"2024-05-08T05:43:58.761229+0000","last_became_active":"2024-05-08T05:43:58.753343+0000","last_became_peered":"2024-05-08T05:43:58.753343+0000","last_unstale":"2024-05-08T05:43:58.761229+0000","last_undegraded":"2024-05-08T05:43:58.761229+0000","last_fullsized":"2024-05-08T05:43:58.761229+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:07:45.223261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"60'8","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.758590+0000","last_change":"2024-05-08T05:43:57.747887+0000","last_active":"2024-05-08T05:43:58.758590+0000","last_peered":"2024-05-08T05:43:58.758590+0000","last_clean":"2024-05-08T05:43:58.758590+0000","last_became_active":"2024-05-08T05:43:57.747610+0000","last_became_peered":"2024-05-08T05:43:57.747610+0000","last_unstale":"2024-05-08T05:43:58.758590+0000","last_undegraded":"2024-05-08T05:43:58.758590+0000","last_fullsized":"2024-05-08T05:43:58.758590+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:21:53.097435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.762683+0000","last_change":"2024-05-08T05:43:58.759904+0000","last_active":"2024-05-08T05:43:58.762683+0000","last_peered":"2024-05-08T05:43:58.762683+0000","last_clean":"2024-05-08T05:43:58.762683+0000","last_became_active":"2024-05-08T05:43:58.759320+0000","last_became_peered":"2024-05-08T05:43:58.759320+0000","last_unstale":"2024-05-08T05:43:58.762683+0000","last_undegraded":"2024-05-08T05:43:58.762683+0000","last_fullsized":"2024-05-08T05:43:58.762683+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:20.614792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.762679+0000","last_change":"2024-05-08T05:43:57.739627+0000","last_active":"2024-05-08T05:43:58.762679+0000","last_peered":"2024-05-08T05:43:58.762679+0000","last_clean":"2024-05-08T05:43:58.762679+0000","last_became_active":"2024-05-08T05:43:57.739307+0000","last_became_peered":"2024-05-08T05:43:57.739307+0000","last_unstale":"2024-05-08T05:43:58.762679+0000","last_undegraded":"2024-05-08T05:43:58.762679+0000","last_fullsized":"2024-05-08T05:43:58.762679+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:27:40.487201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.755452+0000","last_change":"2024-05-08T05:43:58.746262+0000","last_active":"2024-05-08T05:43:58.755452+0000","last_peered":"2024-05-08T05:43:58.755452+0000","last_clean":"2024-05-08T05:43:58.755452+0000","last_became_active":"2024-05-08T05:43:58.745829+0000","last_became_peered":"2024-05-08T05:43:58.745829+0000","last_unstale":"2024-05-08T05:43:58.755452+0000","last_undegraded":"2024-05-08T05:43:58.755452+0000","last_fullsized":"2024-05-08T05:43:58.755452+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:42:54.541460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.760575+0000","last_change":"2024-05-08T05:43:57.745920+0000","last_active":"2024-05-08T05:43:58.760575+0000","last_peered":"2024-05-08T05:43:58.760575+0000","last_clean":"2024-05-08T05:43:58.760575+0000","last_became_active":"2024-05-08T05:43:57.745608+0000","last_became_peered":"2024-05-08T05:43:57.745608+0000","last_unstale":"2024-05-08T05:43:58.760575+0000","last_undegraded":"2024-05-08T05:43:58.760575+0000","last_fullsized":"2024-05-08T05:43:58.760575+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:46:43.642863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.754099+0000","last_change":"2024-05-08T05:43:58.750371+0000","last_active":"2024-05-08T05:43:58.754099+0000","last_peered":"2024-05-08T05:43:58.754099+0000","last_clean":"2024-05-08T05:43:58.754099+0000","last_became_active":"2024-05-08T05:43:58.750094+0000","last_became_peered":"2024-05-08T05:43:58.750094+0000","last_unstale":"2024-05-08T05:43:58.754099+0000","last_undegraded":"2024-05-08T05:43:58.754099+0000","last_fullsized":"2024-05-08T05:43:58.754099+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:14.821494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.761368+0000","last_change":"2024-05-08T05:43:57.747485+0000","last_active":"2024-05-08T05:43:58.761368+0000","last_peered":"2024-05-08T05:43:58.761368+0000","last_clean":"2024-05-08T05:43:58.761368+0000","last_became_active":"2024-05-08T05:43:57.747166+0000","last_became_peered":"2024-05-08T05:43:57.747166+0000","last_unstale":"2024-05-08T05:43:58.761368+0000","last_undegraded":"2024-05-08T05:43:58.761368+0000","last_fullsized":"2024-05-08T05:43:58.761368+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:53.579207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.754344+0000","last_change":"2024-05-08T05:43:58.749232+0000","last_active":"2024-05-08T05:43:58.754344+0000","last_peered":"2024-05-08T05:43:58.754344+0000","last_clean":"2024-05-08T05:43:58.754344+0000","last_became_active":"2024-05-08T05:43:58.748925+0000","last_became_peered":"2024-05-08T05:43:58.748925+0000","last_unstale":"2024-05-08T05:43:58.754344+0000","last_undegraded":"2024-05-08T05:43:58.754344+0000","last_fullsized":"2024-05-08T05:43:58.754344+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:37.988934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.730815+0000","last_change":"2024-05-08T05:43:57.740755+0000","last_active":"2024-05-08T05:43:58.730815+0000","last_peered":"2024-05-08T05:43:58.730815+0000","last_clean":"2024-05-08T05:43:58.730815+0000","last_became_active":"2024-05-08T05:43:57.740425+0000","last_became_peered":"2024-05-08T05:43:57.740425+0000","last_unstale":"2024-05-08T05:43:58.730815+0000","last_undegraded":"2024-05-08T05:43:58.730815+0000","last_fullsized":"2024-05-08T05:43:58.730815+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:40:34.407373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.758794+0000","last_change":"2024-05-08T05:43:58.755187+0000","last_active":"2024-05-08T05:43:58.758794+0000","last_peered":"2024-05-08T05:43:58.758794+0000","last_clean":"2024-05-08T05:43:58.758794+0000","last_became_active":"2024-05-08T05:43:58.754895+0000","last_became_peered":"2024-05-08T05:43:58.754895+0000","last_unstale":"2024-05-08T05:43:58.758794+0000","last_undegraded":"2024-05-08T05:43:58.758794+0000","last_fullsized":"2024-05-08T05:43:58.758794+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:32:38.835058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.754522+0000","last_change":"2024-05-08T05:43:57.751066+0000","last_active":"2024-05-08T05:43:58.754522+0000","last_peered":"2024-05-08T05:43:58.754522+0000","last_clean":"2024-05-08T05:43:58.754522+0000","last_became_active":"2024-05-08T05:43:57.750846+0000","last_became_peered":"2024-05-08T05:43:57.750846+0000","last_unstale":"2024-05-08T05:43:58.754522+0000","last_undegraded":"2024-05-08T05:43:58.754522+0000","last_fullsized":"2024-05-08T05:43:58.754522+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:52:48.111992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.755940+0000","last_change":"2024-05-08T05:43:58.754633+0000","last_active":"2024-05-08T05:43:58.755940+0000","last_peered":"2024-05-08T05:43:58.755940+0000","last_clean":"2024-05-08T05:43:58.755940+0000","last_became_active":"2024-05-08T05:43:58.754348+0000","last_became_peered":"2024-05-08T05:43:58.754348+0000","last_unstale":"2024-05-08T05:43:58.755940+0000","last_undegraded":"2024-05-08T05:43:58.755940+0000","last_fullsized":"2024-05-08T05:43:58.755940+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:09:12.018896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.738308+0000","last_change":"2024-05-08T05:43:57.745869+0000","last_active":"2024-05-08T05:43:58.738308+0000","last_peered":"2024-05-08T05:43:58.738308+0000","last_clean":"2024-05-08T05:43:58.738308+0000","last_became_active":"2024-05-08T05:43:57.745542+0000","last_became_peered":"2024-05-08T05:43:57.745542+0000","last_unstale":"2024-05-08T05:43:58.738308+0000","last_undegraded":"2024-05-08T05:43:58.738308+0000","last_fullsized":"2024-05-08T05:43:58.738308+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:07:44.220953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.752780+0000","last_change":"2024-05-08T05:43:58.746894+0000","last_active":"2024-05-08T05:43:58.752780+0000","last_peered":"2024-05-08T05:43:58.752780+0000","last_clean":"2024-05-08T05:43:58.752780+0000","last_became_active":"2024-05-08T05:43:58.746605+0000","last_became_peered":"2024-05-08T05:43:58.746605+0000","last_unstale":"2024-05-08T05:43:58.752780+0000","last_undegraded":"2024-05-08T05:43:58.752780+0000","last_fullsized":"2024-05-08T05:43:58.752780+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:54.358135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.739566+0000","last_change":"2024-05-08T05:43:57.745921+0000","last_active":"2024-05-08T05:43:58.739566+0000","last_peered":"2024-05-08T05:43:58.739566+0000","last_clean":"2024-05-08T05:43:58.739566+0000","last_became_active":"2024-05-08T05:43:57.745575+0000","last_became_peered":"2024-05-08T05:43:57.745575+0000","last_unstale":"2024-05-08T05:43:58.739566+0000","last_undegraded":"2024-05-08T05:43:58.739566+0000","last_fullsized":"2024-05-08T05:43:58.739566+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:17:32.938067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.748519+0000","last_change":"2024-05-08T05:43:58.746684+0000","last_active":"2024-05-08T05:43:58.748519+0000","last_peered":"2024-05-08T05:43:58.748519+0000","last_clean":"2024-05-08T05:43:58.748519+0000","last_became_active":"2024-05-08T05:43:58.746390+0000","last_became_peered":"2024-05-08T05:43:58.746390+0000","last_unstale":"2024-05-08T05:43:58.748519+0000","last_undegraded":"2024-05-08T05:43:58.748519+0000","last_fullsized":"2024-05-08T05:43:58.748519+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:04.995283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.762954+0000","last_change":"2024-05-08T05:43:57.742991+0000","last_active":"2024-05-08T05:43:58.762954+0000","last_peered":"2024-05-08T05:43:58.762954+0000","last_clean":"2024-05-08T05:43:58.762954+0000","last_became_active":"2024-05-08T05:43:57.742632+0000","last_became_peered":"2024-05-08T05:43:57.742632+0000","last_unstale":"2024-05-08T05:43:58.762954+0000","last_undegraded":"2024-05-08T05:43:58.762954+0000","last_fullsized":"2024-05-08T05:43:58.762954+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:31:51.331036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.758758+0000","last_change":"2024-05-08T05:43:58.757224+0000","last_active":"2024-05-08T05:43:58.758758+0000","last_peered":"2024-05-08T05:43:58.758758+0000","last_clean":"2024-05-08T05:43:58.758758+0000","last_became_active":"2024-05-08T05:43:58.756952+0000","last_became_peered":"2024-05-08T05:43:58.756952+0000","last_unstale":"2024-05-08T05:43:58.758758+0000","last_undegraded":"2024-05-08T05:43:58.758758+0000","last_fullsized":"2024-05-08T05:43:58.758758+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:33:51.903970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.737116+0000","last_change":"2024-05-08T05:43:57.746159+0000","last_active":"2024-05-08T05:43:58.737116+0000","last_peered":"2024-05-08T05:43:58.737116+0000","last_clean":"2024-05-08T05:43:58.737116+0000","last_became_active":"2024-05-08T05:43:57.745862+0000","last_became_peered":"2024-05-08T05:43:57.745862+0000","last_unstale":"2024-05-08T05:43:58.737116+0000","last_undegraded":"2024-05-08T05:43:58.737116+0000","last_fullsized":"2024-05-08T05:43:58.737116+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:23:27.228227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.751395+0000","last_change":"2024-05-08T05:43:58.746462+0000","last_active":"2024-05-08T05:43:58.751395+0000","last_peered":"2024-05-08T05:43:58.751395+0000","last_clean":"2024-05-08T05:43:58.751395+0000","last_became_active":"2024-05-08T05:43:58.746114+0000","last_became_peered":"2024-05-08T05:43:58.746114+0000","last_unstale":"2024-05-08T05:43:58.751395+0000","last_undegraded":"2024-05-08T05:43:58.751395+0000","last_fullsized":"2024-05-08T05:43:58.751395+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:09:37.555566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.740304+0000","last_change":"2024-05-08T05:43:57.747447+0000","last_active":"2024-05-08T05:43:58.740304+0000","last_peered":"2024-05-08T05:43:58.740304+0000","last_clean":"2024-05-08T05:43:58.740304+0000","last_became_active":"2024-05-08T05:43:57.746856+0000","last_became_peered":"2024-05-08T05:43:57.746856+0000","last_unstale":"2024-05-08T05:43:58.740304+0000","last_undegraded":"2024-05-08T05:43:58.740304+0000","last_fullsized":"2024-05-08T05:43:58.740304+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:24:57.633695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.754385+0000","last_change":"2024-05-08T05:43:58.749389+0000","last_active":"2024-05-08T05:43:58.754385+0000","last_peered":"2024-05-08T05:43:58.754385+0000","last_clean":"2024-05-08T05:43:58.754385+0000","last_became_active":"2024-05-08T05:43:58.749069+0000","last_became_peered":"2024-05-08T05:43:58.749069+0000","last_unstale":"2024-05-08T05:43:58.754385+0000","last_undegraded":"2024-05-08T05:43:58.754385+0000","last_fullsized":"2024-05-08T05:43:58.754385+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:43:13.328768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.735732+0000","last_change":"2024-05-08T05:43:57.741565+0000","last_active":"2024-05-08T05:43:58.735732+0000","last_peered":"2024-05-08T05:43:58.735732+0000","last_clean":"2024-05-08T05:43:58.735732+0000","last_became_active":"2024-05-08T05:43:57.740849+0000","last_became_peered":"2024-05-08T05:43:57.740849+0000","last_unstale":"2024-05-08T05:43:58.735732+0000","last_undegraded":"2024-05-08T05:43:58.735732+0000","last_fullsized":"2024-05-08T05:43:58.735732+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:44:45.896339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.752744+0000","last_change":"2024-05-08T05:43:58.746874+0000","last_active":"2024-05-08T05:43:58.752744+0000","last_peered":"2024-05-08T05:43:58.752744+0000","last_clean":"2024-05-08T05:43:58.752744+0000","last_became_active":"2024-05-08T05:43:58.746566+0000","last_became_peered":"2024-05-08T05:43:58.746566+0000","last_unstale":"2024-05-08T05:43:58.752744+0000","last_undegraded":"2024-05-08T05:43:58.752744+0000","last_fullsized":"2024-05-08T05:43:58.752744+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:18:07.653809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.735621+0000","last_change":"2024-05-08T05:43:57.748766+0000","last_active":"2024-05-08T05:43:58.735621+0000","last_peered":"2024-05-08T05:43:58.735621+0000","last_clean":"2024-05-08T05:43:58.735621+0000","last_became_active":"2024-05-08T05:43:57.748400+0000","last_became_peered":"2024-05-08T05:43:57.748400+0000","last_unstale":"2024-05-08T05:43:58.735621+0000","last_undegraded":"2024-05-08T05:43:58.735621+0000","last_fullsized":"2024-05-08T05:43:58.735621+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:09:11.016588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.762168+0000","last_change":"2024-05-08T05:43:58.758339+0000","last_active":"2024-05-08T05:43:58.762168+0000","last_peered":"2024-05-08T05:43:58.762168+0000","last_clean":"2024-05-08T05:43:58.762168+0000","last_became_active":"2024-05-08T05:43:58.758049+0000","last_became_peered":"2024-05-08T05:43:58.758049+0000","last_unstale":"2024-05-08T05:43:58.762168+0000","last_undegraded":"2024-05-08T05:43:58.762168+0000","last_fullsized":"2024-05-08T05:43:58.762168+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:37.988934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.730797+0000","last_change":"2024-05-08T05:43:57.737654+0000","last_active":"2024-05-08T05:43:58.730797+0000","last_peered":"2024-05-08T05:43:58.730797+0000","last_clean":"2024-05-08T05:43:58.730797+0000","last_became_active":"2024-05-08T05:43:57.737354+0000","last_became_peered":"2024-05-08T05:43:57.737354+0000","last_unstale":"2024-05-08T05:43:58.730797+0000","last_undegraded":"2024-05-08T05:43:58.730797+0000","last_fullsized":"2024-05-08T05:43:58.730797+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:26.792126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.760957+0000","last_change":"2024-05-08T05:43:58.751882+0000","last_active":"2024-05-08T05:43:58.760957+0000","last_peered":"2024-05-08T05:43:58.760957+0000","last_clean":"2024-05-08T05:43:58.760957+0000","last_became_active":"2024-05-08T05:43:58.751618+0000","last_became_peered":"2024-05-08T05:43:58.751618+0000","last_unstale":"2024-05-08T05:43:58.760957+0000","last_undegraded":"2024-05-08T05:43:58.760957+0000","last_fullsized":"2024-05-08T05:43:58.760957+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:09.681994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.730118+0000","last_change":"2024-05-08T05:43:57.749487+0000","last_active":"2024-05-08T05:43:58.730118+0000","last_peered":"2024-05-08T05:43:58.730118+0000","last_clean":"2024-05-08T05:43:58.730118+0000","last_became_active":"2024-05-08T05:43:57.749221+0000","last_became_peered":"2024-05-08T05:43:57.749221+0000","last_unstale":"2024-05-08T05:43:58.730118+0000","last_undegraded":"2024-05-08T05:43:58.730118+0000","last_fullsized":"2024-05-08T05:43:58.730118+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:41:52.162297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.762049+0000","last_change":"2024-05-08T05:43:58.760285+0000","last_active":"2024-05-08T05:43:58.762049+0000","last_peered":"2024-05-08T05:43:58.762049+0000","last_clean":"2024-05-08T05:43:58.762049+0000","last_became_active":"2024-05-08T05:43:58.760019+0000","last_became_peered":"2024-05-08T05:43:58.760019+0000","last_unstale":"2024-05-08T05:43:58.762049+0000","last_undegraded":"2024-05-08T05:43:58.762049+0000","last_fullsized":"2024-05-08T05:43:58.762049+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:25:55.521438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.735570+0000","last_change":"2024-05-08T05:43:57.748220+0000","last_active":"2024-05-08T05:43:58.735570+0000","last_peered":"2024-05-08T05:43:58.735570+0000","last_clean":"2024-05-08T05:43:58.735570+0000","last_became_active":"2024-05-08T05:43:57.747917+0000","last_became_peered":"2024-05-08T05:43:57.747917+0000","last_unstale":"2024-05-08T05:43:58.735570+0000","last_undegraded":"2024-05-08T05:43:58.735570+0000","last_fullsized":"2024-05-08T05:43:58.735570+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:13:36.764103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.764336+0000","last_change":"2024-05-08T05:43:58.756951+0000","last_active":"2024-05-08T05:43:58.764336+0000","last_peered":"2024-05-08T05:43:58.764336+0000","last_clean":"2024-05-08T05:43:58.764336+0000","last_became_active":"2024-05-08T05:43:58.756695+0000","last_became_peered":"2024-05-08T05:43:58.756695+0000","last_unstale":"2024-05-08T05:43:58.764336+0000","last_undegraded":"2024-05-08T05:43:58.764336+0000","last_fullsized":"2024-05-08T05:43:58.764336+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:12:29.066783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.729834+0000","last_change":"2024-05-08T05:43:57.734841+0000","last_active":"2024-05-08T05:43:58.729834+0000","last_peered":"2024-05-08T05:43:58.729834+0000","last_clean":"2024-05-08T05:43:58.729834+0000","last_became_active":"2024-05-08T05:43:57.734495+0000","last_became_peered":"2024-05-08T05:43:57.734495+0000","last_unstale":"2024-05-08T05:43:58.729834+0000","last_undegraded":"2024-05-08T05:43:58.729834+0000","last_fullsized":"2024-05-08T05:43:58.729834+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:19.612484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":184,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.737245+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:43:58.737245+0000","last_peered":"2024-05-08T05:43:58.737245+0000","last_clean":"2024-05-08T05:43:58.737245+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:43:58.737245+0000","last_undegraded":"2024-05-08T05:43:58.737245+0000","last_fullsized":"2024-05-08T05:43:58.737245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542180,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28320,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343520,"statfs":{"total":96636764160,"available":96607764480,"internally_reserved":0,"allocated":1331200,"data_stored":912580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542181,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":724992,"data_stored":321470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":724992,"data_stored":321458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":314269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":314269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":314269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":314269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1302528,"data_stored":904637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:44:02.789 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:44:02.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.929+0000 7f38d9924640 1 Processor -- start 2024-05-08T05:44:02.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.929+0000 7f38d9924640 1 -- start start 2024-05-08T05:44:02.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.930+0000 7f38d9924640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f38d40d3cd0 0x7f38d40cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.930+0000 7f38d9924640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f38d40c9ee0 0x7f38d40d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.930+0000 7f38d9924640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38d40d3330 con 0x7f38d41629a0 2024-05-08T05:44:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.930+0000 7f38d9924640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f38d40d2170 con 0x7f38d40c9ee0 2024-05-08T05:44:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.930+0000 7f38d9924640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f38d40d2450 con 0x7f38d40d3cd0 2024-05-08T05:44:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.930+0000 7f38d37fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f38d40c9ee0 0x7f38d40d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.930+0000 7f38d37fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f38d40c9ee0 0x7f38d40d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36648/0 (socket says 172.21.15.45:36648) 2024-05-08T05:44:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.930+0000 7f38d37fe640 1 -- 172.21.15.45:0/203864096 learned_addr learned my addr 172.21.15.45:0/203864096 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.930+0000 7f38d37fe640 1 -- 172.21.15.45:0/203864096 >> v2:172.21.15.63:3300/0 conn(0x7f38d40d3cd0 msgr2=0x7f38d40cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.930+0000 7f38d37fe640 1 --2- 172.21.15.45:0/203864096 >> v2:172.21.15.63:3300/0 conn(0x7f38d40d3cd0 0x7f38d40cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.930+0000 7f38d37fe640 1 -- 172.21.15.45:0/203864096 >> v1:172.21.15.63:6789/0 conn(0x7f38d41629a0 legacy=0x7f38d40d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:02.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.930+0000 7f38d37fe640 1 -- 172.21.15.45:0/203864096 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d40d2730 con 0x7f38d40c9ee0 2024-05-08T05:44:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.931+0000 7f38d37fe640 1 --2- 172.21.15.45:0/203864096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38d40c9ee0 0x7f38d40d01e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f38d40b7ae0 tx=0x7f38cc02a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd48c3aa332c4242 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.931+0000 7f38d27fc640 1 -- 172.21.15.45:0/203864096 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc036070 con 0x7f38d40c9ee0 2024-05-08T05:44:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.931+0000 7f38d27fc640 1 -- 172.21.15.45:0/203864096 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38cc031600 con 0x7f38d40c9ee0 2024-05-08T05:44:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.931+0000 7f38d27fc640 1 -- 172.21.15.45:0/203864096 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc0319d0 con 0x7f38d40c9ee0 2024-05-08T05:44:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.932+0000 7f38d9924640 1 -- 172.21.15.45:0/203864096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38d40c9ee0 msgr2=0x7f38d40d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.932+0000 7f38d9924640 1 --2- 172.21.15.45:0/203864096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38d40c9ee0 0x7f38d40d01e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f38d40b7ae0 tx=0x7f38cc02a570 comp rx=0 tx=0).stop 2024-05-08T05:44:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.932+0000 7f38d9924640 1 -- 172.21.15.45:0/203864096 shutdown_connections 2024-05-08T05:44:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.932+0000 7f38d9924640 1 --2- 172.21.15.45:0/203864096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38d40c9ee0 0x7f38d40d01e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.932+0000 7f38d9924640 1 --2- 172.21.15.45:0/203864096 >> v2:172.21.15.63:3300/0 conn(0x7f38d40d3cd0 0x7f38d40cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.932+0000 7f38d9924640 1 -- 172.21.15.45:0/203864096 >> 172.21.15.45:0/203864096 conn(0x7f38d40d4fe0 msgr2=0x7f38d40d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.932+0000 7f38d9924640 1 -- 172.21.15.45:0/203864096 shutdown_connections 2024-05-08T05:44:02.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.932+0000 7f38d9924640 1 -- 172.21.15.45:0/203864096 wait complete. 2024-05-08T05:44:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.933+0000 7f38d9924640 1 Processor -- start 2024-05-08T05:44:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.933+0000 7f38d9924640 1 -- start start 2024-05-08T05:44:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d9924640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38d40c9ee0 0x7f38d40b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d9924640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38d40d3cd0 0x7f38d41e7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d9924640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38d41629a0 0x7f38d40cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d9924640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f38d40cf440 con 0x7f38d41629a0 2024-05-08T05:44:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d9924640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f38d40cf2c0 con 0x7f38d40d3cd0 2024-05-08T05:44:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d9924640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f38d40d2dc0 con 0x7f38d40c9ee0 2024-05-08T05:44:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38d41629a0 0x7f38d40cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38d41629a0 0x7f38d40cd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36650/0 (socket says 172.21.15.45:36650) 2024-05-08T05:44:02.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d37fe640 1 -- 172.21.15.45:0/780835096 learned_addr learned my addr 172.21.15.45:0/780835096 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38cbfff640 1 --2- 172.21.15.45:0/780835096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38d40d3cd0 0x7f38d41e7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d2ffd640 1 --2- 172.21.15.45:0/780835096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38d40c9ee0 0x7f38d40b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d37fe640 1 -- 172.21.15.45:0/780835096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38d40c9ee0 msgr2=0x7f38d40b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d37fe640 1 --2- 172.21.15.45:0/780835096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38d40c9ee0 0x7f38d40b82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d37fe640 1 -- 172.21.15.45:0/780835096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38d40d3cd0 msgr2=0x7f38d41e7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d37fe640 1 --2- 172.21.15.45:0/780835096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38d40d3cd0 0x7f38d41e7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d37fe640 1 -- 172.21.15.45:0/780835096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d40b8970 con 0x7f38d41629a0 2024-05-08T05:44:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d2ffd640 1 --2- 172.21.15.45:0/780835096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38d40c9ee0 0x7f38d40b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:44:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d37fe640 1 --2- 172.21.15.45:0/780835096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38d41629a0 0x7f38d40cd100 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f38cc034040 tx=0x7f38cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d0ff9640 1 -- 172.21.15.45:0/780835096 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc041020 con 0x7f38d41629a0 2024-05-08T05:44:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d0ff9640 1 -- 172.21.15.45:0/780835096 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38cc007780 con 0x7f38d41629a0 2024-05-08T05:44:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d9924640 1 -- 172.21.15.45:0/780835096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d40b8c50 con 0x7f38d41629a0 2024-05-08T05:44:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.934+0000 7f38d0ff9640 1 -- 172.21.15.45:0/780835096 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc036040 con 0x7f38d41629a0 2024-05-08T05:44:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.935+0000 7f38d9924640 1 -- 172.21.15.45:0/780835096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d40b9210 con 0x7f38d41629a0 2024-05-08T05:44:02.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.935+0000 7f38d9924640 1 -- 172.21.15.45:0/780835096 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38d40693f0 con 0x7f38d41629a0 2024-05-08T05:44:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.936+0000 7f38d0ff9640 1 -- 172.21.15.45:0/780835096 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f38cc044020 con 0x7f38d41629a0 2024-05-08T05:44:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.937+0000 7f38d0ff9640 1 --2- 172.21.15.45:0/780835096 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f38b40762d0 0x7f38b4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.937+0000 7f38d0ff9640 1 -- 172.21.15.45:0/780835096 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f38cc0b84d0 con 0x7f38d41629a0 2024-05-08T05:44:02.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.937+0000 7f38d2ffd640 1 --2- 172.21.15.45:0/780835096 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f38b40762d0 0x7f38b4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:02.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.937+0000 7f38d2ffd640 1 --2- 172.21.15.45:0/780835096 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f38b40762d0 0x7f38b4078790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f38c4006fd0 tx=0x7f38c4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:02.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:02.941+0000 7f38d0ff9640 1 -- 172.21.15.45:0/780835096 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f38cc04b030 con 0x7f38d41629a0 2024-05-08T05:44:03.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.127+0000 7f38d9924640 1 -- 172.21.15.45:0/780835096 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f38d40ce890 con 0x7f38b40762d0 2024-05-08T05:44:03.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.134+0000 7f38d0ff9640 1 -- 172.21.15.45:0/780835096 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251938 (secure 0 0 0) 0x7f38d40ce890 con 0x7f38b40762d0 2024-05-08T05:44:03.137 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:44:03.138 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:44:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.137+0000 7f38d9924640 1 -- 172.21.15.45:0/780835096 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f38b40762d0 msgr2=0x7f38b4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.137+0000 7f38d9924640 1 --2- 172.21.15.45:0/780835096 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f38b40762d0 0x7f38b4078790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f38c4006fd0 tx=0x7f38c4008040 comp rx=0 tx=0).stop 2024-05-08T05:44:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.137+0000 7f38d9924640 1 -- 172.21.15.45:0/780835096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38d41629a0 msgr2=0x7f38d40cd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.137+0000 7f38d9924640 1 --2- 172.21.15.45:0/780835096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38d41629a0 0x7f38d40cd100 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f38cc034040 tx=0x7f38cc004a40 comp rx=0 tx=0).stop 2024-05-08T05:44:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.137+0000 7f38d9924640 1 -- 172.21.15.45:0/780835096 shutdown_connections 2024-05-08T05:44:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.137+0000 7f38d9924640 1 --2- 172.21.15.45:0/780835096 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f38b40762d0 0x7f38b4078790 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.137+0000 7f38d9924640 1 --2- 172.21.15.45:0/780835096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38d41629a0 0x7f38d40cd100 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.137+0000 7f38d9924640 1 --2- 172.21.15.45:0/780835096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38d40d3cd0 0x7f38d41e7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.137+0000 7f38d9924640 1 --2- 172.21.15.45:0/780835096 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38d40c9ee0 0x7f38d40b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.137+0000 7f38d9924640 1 -- 172.21.15.45:0/780835096 >> 172.21.15.45:0/780835096 conn(0x7f38d40d4fe0 msgr2=0x7f38d41612d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:03.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.137+0000 7f38d9924640 1 -- 172.21.15.45:0/780835096 shutdown_connections 2024-05-08T05:44:03.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.137+0000 7f38d9924640 1 -- 172.21.15.45:0/780835096 wait complete. 2024-05-08T05:44:03.149 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-05-08T05:44:02.868069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239592,"kb_used_data":6756,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735128,"statfs":{"total":773094113280,"available":772848771072,"internally_reserved":0,"allocated":6918144,"data_stored":3717221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001308"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.761138+0000","last_change":"2024-05-08T05:43:58.753503+0000","last_active":"2024-05-08T05:43:58.761138+0000","last_peered":"2024-05-08T05:43:58.761138+0000","last_clean":"2024-05-08T05:43:58.761138+0000","last_became_active":"2024-05-08T05:43:58.753143+0000","last_became_peered":"2024-05-08T05:43:58.753143+0000","last_unstale":"2024-05-08T05:43:58.761138+0000","last_undegraded":"2024-05-08T05:43:58.761138+0000","last_fullsized":"2024-05-08T05:43:58.761138+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:53:55.951106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.759032+0000","last_change":"2024-05-08T05:43:58.750946+0000","last_active":"2024-05-08T05:43:58.759032+0000","last_peered":"2024-05-08T05:43:58.759032+0000","last_clean":"2024-05-08T05:43:58.759032+0000","last_became_active":"2024-05-08T05:43:58.750589+0000","last_became_peered":"2024-05-08T05:43:58.750589+0000","last_unstale":"2024-05-08T05:43:58.759032+0000","last_undegraded":"2024-05-08T05:43:58.759032+0000","last_fullsized":"2024-05-08T05:43:58.759032+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:57.495679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.760090+0000","last_change":"2024-05-08T05:43:58.752044+0000","last_active":"2024-05-08T05:43:58.760090+0000","last_peered":"2024-05-08T05:43:58.760090+0000","last_clean":"2024-05-08T05:43:58.760090+0000","last_became_active":"2024-05-08T05:43:58.751755+0000","last_became_peered":"2024-05-08T05:43:58.751755+0000","last_unstale":"2024-05-08T05:43:58.760090+0000","last_undegraded":"2024-05-08T05:43:58.760090+0000","last_fullsized":"2024-05-08T05:43:58.760090+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:04:07.485759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.745286+0000","last_change":"2024-05-08T05:43:58.744176+0000","last_active":"2024-05-08T05:43:58.745286+0000","last_peered":"2024-05-08T05:43:58.745286+0000","last_clean":"2024-05-08T05:43:58.745286+0000","last_became_active":"2024-05-08T05:43:58.743872+0000","last_became_peered":"2024-05-08T05:43:58.743872+0000","last_unstale":"2024-05-08T05:43:58.745286+0000","last_undegraded":"2024-05-08T05:43:58.745286+0000","last_fullsized":"2024-05-08T05:43:58.745286+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:21:42.939708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.755315+0000","last_change":"2024-05-08T05:43:58.749306+0000","last_active":"2024-05-08T05:43:58.755315+0000","last_peered":"2024-05-08T05:43:58.755315+0000","last_clean":"2024-05-08T05:43:58.755315+0000","last_became_active":"2024-05-08T05:43:58.748977+0000","last_became_peered":"2024-05-08T05:43:58.748977+0000","last_unstale":"2024-05-08T05:43:58.755315+0000","last_undegraded":"2024-05-08T05:43:58.755315+0000","last_fullsized":"2024-05-08T05:43:58.755315+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:40:37.076365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.762025+0000","last_change":"2024-05-08T05:43:58.757238+0000","last_active":"2024-05-08T05:43:58.762025+0000","last_peered":"2024-05-08T05:43:58.762025+0000","last_clean":"2024-05-08T05:43:58.762025+0000","last_became_active":"2024-05-08T05:43:58.756387+0000","last_became_peered":"2024-05-08T05:43:58.756387+0000","last_unstale":"2024-05-08T05:43:58.762025+0000","last_undegraded":"2024-05-08T05:43:58.762025+0000","last_fullsized":"2024-05-08T05:43:58.762025+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:20.614792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.758788+0000","last_change":"2024-05-08T05:43:58.757279+0000","last_active":"2024-05-08T05:43:58.758788+0000","last_peered":"2024-05-08T05:43:58.758788+0000","last_clean":"2024-05-08T05:43:58.758788+0000","last_became_active":"2024-05-08T05:43:58.756996+0000","last_became_peered":"2024-05-08T05:43:58.756996+0000","last_unstale":"2024-05-08T05:43:58.758788+0000","last_undegraded":"2024-05-08T05:43:58.758788+0000","last_fullsized":"2024-05-08T05:43:58.758788+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:03:15.648434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.755986+0000","last_change":"2024-05-08T05:43:58.754887+0000","last_active":"2024-05-08T05:43:58.755986+0000","last_peered":"2024-05-08T05:43:58.755986+0000","last_clean":"2024-05-08T05:43:58.755986+0000","last_became_active":"2024-05-08T05:43:58.754613+0000","last_became_peered":"2024-05-08T05:43:58.754613+0000","last_unstale":"2024-05-08T05:43:58.755986+0000","last_undegraded":"2024-05-08T05:43:58.755986+0000","last_fullsized":"2024-05-08T05:43:58.755986+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:28:19.201137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.757202+0000","last_change":"2024-05-08T05:43:58.751431+0000","last_active":"2024-05-08T05:43:58.757202+0000","last_peered":"2024-05-08T05:43:58.757202+0000","last_clean":"2024-05-08T05:43:58.757202+0000","last_became_active":"2024-05-08T05:43:58.751130+0000","last_became_peered":"2024-05-08T05:43:58.751130+0000","last_unstale":"2024-05-08T05:43:58.757202+0000","last_undegraded":"2024-05-08T05:43:58.757202+0000","last_fullsized":"2024-05-08T05:43:58.757202+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:24:58.636003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.754313+0000","last_change":"2024-05-08T05:43:58.746264+0000","last_active":"2024-05-08T05:43:58.754313+0000","last_peered":"2024-05-08T05:43:58.754313+0000","last_clean":"2024-05-08T05:43:58.754313+0000","last_became_active":"2024-05-08T05:43:58.745987+0000","last_became_peered":"2024-05-08T05:43:58.745987+0000","last_unstale":"2024-05-08T05:43:58.754313+0000","last_undegraded":"2024-05-08T05:43:58.754313+0000","last_fullsized":"2024-05-08T05:43:58.754313+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:17:33.940374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.759921+0000","last_change":"2024-05-08T05:43:58.752489+0000","last_active":"2024-05-08T05:43:58.759921+0000","last_peered":"2024-05-08T05:43:58.759921+0000","last_clean":"2024-05-08T05:43:58.759921+0000","last_became_active":"2024-05-08T05:43:58.752209+0000","last_became_peered":"2024-05-08T05:43:58.752209+0000","last_unstale":"2024-05-08T05:43:58.759921+0000","last_undegraded":"2024-05-08T05:43:58.759921+0000","last_fullsized":"2024-05-08T05:43:58.759921+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:18:07.653809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.756342+0000","last_change":"2024-05-08T05:43:58.754735+0000","last_active":"2024-05-08T05:43:58.756342+0000","last_peered":"2024-05-08T05:43:58.756342+0000","last_clean":"2024-05-08T05:43:58.756342+0000","last_became_active":"2024-05-08T05:43:58.754424+0000","last_became_peered":"2024-05-08T05:43:58.754424+0000","last_unstale":"2024-05-08T05:43:58.756342+0000","last_undegraded":"2024-05-08T05:43:58.756342+0000","last_fullsized":"2024-05-08T05:43:58.756342+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:46:44.645171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.751340+0000","last_change":"2024-05-08T05:43:58.745342+0000","last_active":"2024-05-08T05:43:58.751340+0000","last_peered":"2024-05-08T05:43:58.751340+0000","last_clean":"2024-05-08T05:43:58.751340+0000","last_became_active":"2024-05-08T05:43:58.744849+0000","last_became_peered":"2024-05-08T05:43:58.744849+0000","last_unstale":"2024-05-08T05:43:58.751340+0000","last_undegraded":"2024-05-08T05:43:58.751340+0000","last_fullsized":"2024-05-08T05:43:58.751340+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:04:25.561526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.762366+0000","last_change":"2024-05-08T05:43:58.753424+0000","last_active":"2024-05-08T05:43:58.762366+0000","last_peered":"2024-05-08T05:43:58.762366+0000","last_clean":"2024-05-08T05:43:58.762366+0000","last_became_active":"2024-05-08T05:43:58.751183+0000","last_became_peered":"2024-05-08T05:43:58.751183+0000","last_unstale":"2024-05-08T05:43:58.762366+0000","last_undegraded":"2024-05-08T05:43:58.762366+0000","last_fullsized":"2024-05-08T05:43:58.762366+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:55:24.366110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.762474+0000","last_change":"2024-05-08T05:43:58.757260+0000","last_active":"2024-05-08T05:43:58.762474+0000","last_peered":"2024-05-08T05:43:58.762474+0000","last_clean":"2024-05-08T05:43:58.762474+0000","last_became_active":"2024-05-08T05:43:58.754636+0000","last_became_peered":"2024-05-08T05:43:58.754636+0000","last_unstale":"2024-05-08T05:43:58.762474+0000","last_undegraded":"2024-05-08T05:43:58.762474+0000","last_fullsized":"2024-05-08T05:43:58.762474+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:43:45.044208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.760439+0000","last_change":"2024-05-08T05:43:58.759412+0000","last_active":"2024-05-08T05:43:58.760439+0000","last_peered":"2024-05-08T05:43:58.760439+0000","last_clean":"2024-05-08T05:43:58.760439+0000","last_became_active":"2024-05-08T05:43:58.758810+0000","last_became_peered":"2024-05-08T05:43:58.758810+0000","last_unstale":"2024-05-08T05:43:58.760439+0000","last_undegraded":"2024-05-08T05:43:58.760439+0000","last_fullsized":"2024-05-08T05:43:58.760439+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:42:13.925868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.757985+0000","last_change":"2024-05-08T05:43:58.751072+0000","last_active":"2024-05-08T05:43:58.757985+0000","last_peered":"2024-05-08T05:43:58.757985+0000","last_clean":"2024-05-08T05:43:58.757985+0000","last_became_active":"2024-05-08T05:43:58.750764+0000","last_became_peered":"2024-05-08T05:43:58.750764+0000","last_unstale":"2024-05-08T05:43:58.757985+0000","last_undegraded":"2024-05-08T05:43:58.757985+0000","last_fullsized":"2024-05-08T05:43:58.757985+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:42:39.962581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.745245+0000","last_change":"2024-05-08T05:43:58.743256+0000","last_active":"2024-05-08T05:43:58.745245+0000","last_peered":"2024-05-08T05:43:58.745245+0000","last_clean":"2024-05-08T05:43:58.745245+0000","last_became_active":"2024-05-08T05:43:58.742972+0000","last_became_peered":"2024-05-08T05:43:58.742972+0000","last_unstale":"2024-05-08T05:43:58.745245+0000","last_undegraded":"2024-05-08T05:43:58.745245+0000","last_fullsized":"2024-05-08T05:43:58.745245+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:07.409112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.755474+0000","last_change":"2024-05-08T05:43:58.752037+0000","last_active":"2024-05-08T05:43:58.755474+0000","last_peered":"2024-05-08T05:43:58.755474+0000","last_clean":"2024-05-08T05:43:58.755474+0000","last_became_active":"2024-05-08T05:43:58.751754+0000","last_became_peered":"2024-05-08T05:43:58.751754+0000","last_unstale":"2024-05-08T05:43:58.755474+0000","last_undegraded":"2024-05-08T05:43:58.755474+0000","last_fullsized":"2024-05-08T05:43:58.755474+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:02:31.259717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.754299+0000","last_change":"2024-05-08T05:43:58.750358+0000","last_active":"2024-05-08T05:43:58.754299+0000","last_peered":"2024-05-08T05:43:58.754299+0000","last_clean":"2024-05-08T05:43:58.754299+0000","last_became_active":"2024-05-08T05:43:58.749996+0000","last_became_peered":"2024-05-08T05:43:58.749996+0000","last_unstale":"2024-05-08T05:43:58.754299+0000","last_undegraded":"2024-05-08T05:43:58.754299+0000","last_fullsized":"2024-05-08T05:43:58.754299+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:34:19.446859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.765084+0000","last_change":"2024-05-08T05:43:58.761863+0000","last_active":"2024-05-08T05:43:58.765084+0000","last_peered":"2024-05-08T05:43:58.765084+0000","last_clean":"2024-05-08T05:43:58.765084+0000","last_became_active":"2024-05-08T05:43:58.761485+0000","last_became_peered":"2024-05-08T05:43:58.761485+0000","last_unstale":"2024-05-08T05:43:58.765084+0000","last_undegraded":"2024-05-08T05:43:58.765084+0000","last_fullsized":"2024-05-08T05:43:58.765084+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:52:49.114300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.759342+0000","last_change":"2024-05-08T05:43:58.757769+0000","last_active":"2024-05-08T05:43:58.759342+0000","last_peered":"2024-05-08T05:43:58.759342+0000","last_clean":"2024-05-08T05:43:58.759342+0000","last_became_active":"2024-05-08T05:43:58.757464+0000","last_became_peered":"2024-05-08T05:43:58.757464+0000","last_unstale":"2024-05-08T05:43:58.759342+0000","last_undegraded":"2024-05-08T05:43:58.759342+0000","last_fullsized":"2024-05-08T05:43:58.759342+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:03:49.694754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.758248+0000","last_change":"2024-05-08T05:43:58.757175+0000","last_active":"2024-05-08T05:43:58.758248+0000","last_peered":"2024-05-08T05:43:58.758248+0000","last_clean":"2024-05-08T05:43:58.758248+0000","last_became_active":"2024-05-08T05:43:58.756875+0000","last_became_peered":"2024-05-08T05:43:58.756875+0000","last_unstale":"2024-05-08T05:43:58.758248+0000","last_undegraded":"2024-05-08T05:43:58.758248+0000","last_fullsized":"2024-05-08T05:43:58.758248+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:27.264718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.762325+0000","last_change":"2024-05-08T05:43:58.752234+0000","last_active":"2024-05-08T05:43:58.762325+0000","last_peered":"2024-05-08T05:43:58.762325+0000","last_clean":"2024-05-08T05:43:58.762325+0000","last_became_active":"2024-05-08T05:43:58.751741+0000","last_became_peered":"2024-05-08T05:43:58.751741+0000","last_unstale":"2024-05-08T05:43:58.762325+0000","last_undegraded":"2024-05-08T05:43:58.762325+0000","last_fullsized":"2024-05-08T05:43:58.762325+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:04.995283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.757261+0000","last_change":"2024-05-08T05:43:58.755471+0000","last_active":"2024-05-08T05:43:58.757261+0000","last_peered":"2024-05-08T05:43:58.757261+0000","last_clean":"2024-05-08T05:43:58.757261+0000","last_became_active":"2024-05-08T05:43:58.755202+0000","last_became_peered":"2024-05-08T05:43:58.755202+0000","last_unstale":"2024-05-08T05:43:58.757261+0000","last_undegraded":"2024-05-08T05:43:58.757261+0000","last_fullsized":"2024-05-08T05:43:58.757261+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:08.842871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.761748+0000","last_change":"2024-05-08T05:43:58.756270+0000","last_active":"2024-05-08T05:43:58.761748+0000","last_peered":"2024-05-08T05:43:58.761748+0000","last_clean":"2024-05-08T05:43:58.761748+0000","last_became_active":"2024-05-08T05:43:58.755984+0000","last_became_peered":"2024-05-08T05:43:58.755984+0000","last_unstale":"2024-05-08T05:43:58.761748+0000","last_undegraded":"2024-05-08T05:43:58.761748+0000","last_fullsized":"2024-05-08T05:43:58.761748+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:15.841767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.759947+0000","last_change":"2024-05-08T05:43:58.752918+0000","last_active":"2024-05-08T05:43:58.759947+0000","last_peered":"2024-05-08T05:43:58.759947+0000","last_clean":"2024-05-08T05:43:58.759947+0000","last_became_active":"2024-05-08T05:43:58.752313+0000","last_became_peered":"2024-05-08T05:43:58.752313+0000","last_unstale":"2024-05-08T05:43:58.759947+0000","last_undegraded":"2024-05-08T05:43:58.759947+0000","last_fullsized":"2024-05-08T05:43:58.759947+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:54.358135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.761035+0000","last_change":"2024-05-08T05:43:58.752352+0000","last_active":"2024-05-08T05:43:58.761035+0000","last_peered":"2024-05-08T05:43:58.761035+0000","last_clean":"2024-05-08T05:43:58.761035+0000","last_became_active":"2024-05-08T05:43:58.751509+0000","last_became_peered":"2024-05-08T05:43:58.751509+0000","last_unstale":"2024-05-08T05:43:58.761035+0000","last_undegraded":"2024-05-08T05:43:58.761035+0000","last_fullsized":"2024-05-08T05:43:58.761035+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:23:28.230535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.757684+0000","last_change":"2024-05-08T05:43:58.747056+0000","last_active":"2024-05-08T05:43:58.757684+0000","last_peered":"2024-05-08T05:43:58.757684+0000","last_clean":"2024-05-08T05:43:58.757684+0000","last_became_active":"2024-05-08T05:43:58.746787+0000","last_became_peered":"2024-05-08T05:43:58.746787+0000","last_unstale":"2024-05-08T05:43:58.757684+0000","last_undegraded":"2024-05-08T05:43:58.757684+0000","last_fullsized":"2024-05-08T05:43:58.757684+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:55:31.453030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.756990+0000","last_change":"2024-05-08T05:43:58.755599+0000","last_active":"2024-05-08T05:43:58.756990+0000","last_peered":"2024-05-08T05:43:58.756990+0000","last_clean":"2024-05-08T05:43:58.756990+0000","last_became_active":"2024-05-08T05:43:58.755316+0000","last_became_peered":"2024-05-08T05:43:58.755316+0000","last_unstale":"2024-05-08T05:43:58.756990+0000","last_undegraded":"2024-05-08T05:43:58.756990+0000","last_fullsized":"2024-05-08T05:43:58.756990+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:54.581514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.754003+0000","last_change":"2024-05-08T05:43:58.750875+0000","last_active":"2024-05-08T05:43:58.754003+0000","last_peered":"2024-05-08T05:43:58.754003+0000","last_clean":"2024-05-08T05:43:58.754003+0000","last_became_active":"2024-05-08T05:43:58.750571+0000","last_became_peered":"2024-05-08T05:43:58.750571+0000","last_unstale":"2024-05-08T05:43:58.754003+0000","last_undegraded":"2024-05-08T05:43:58.754003+0000","last_fullsized":"2024-05-08T05:43:58.754003+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:39.363056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.755606+0000","last_change":"2024-05-08T05:43:58.754415+0000","last_active":"2024-05-08T05:43:58.755606+0000","last_peered":"2024-05-08T05:43:58.755606+0000","last_clean":"2024-05-08T05:43:58.755606+0000","last_became_active":"2024-05-08T05:43:58.754116+0000","last_became_peered":"2024-05-08T05:43:58.754116+0000","last_unstale":"2024-05-08T05:43:58.755606+0000","last_undegraded":"2024-05-08T05:43:58.755606+0000","last_fullsized":"2024-05-08T05:43:58.755606+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:55:24.366110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.764353+0000","last_change":"2024-05-08T05:43:58.758314+0000","last_active":"2024-05-08T05:43:58.764353+0000","last_peered":"2024-05-08T05:43:58.764353+0000","last_clean":"2024-05-08T05:43:58.764353+0000","last_became_active":"2024-05-08T05:43:58.758016+0000","last_became_peered":"2024-05-08T05:43:58.758016+0000","last_unstale":"2024-05-08T05:43:58.764353+0000","last_undegraded":"2024-05-08T05:43:58.764353+0000","last_fullsized":"2024-05-08T05:43:58.764353+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:13:37.766411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"60'4","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.763002+0000","last_change":"2024-05-08T05:43:57.744551+0000","last_active":"2024-05-08T05:43:58.763002+0000","last_peered":"2024-05-08T05:43:58.763002+0000","last_clean":"2024-05-08T05:43:58.763002+0000","last_became_active":"2024-05-08T05:43:57.744255+0000","last_became_peered":"2024-05-08T05:43:57.744255+0000","last_unstale":"2024-05-08T05:43:58.763002+0000","last_undegraded":"2024-05-08T05:43:58.763002+0000","last_fullsized":"2024-05-08T05:43:58.763002+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:42:04.835594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.759456+0000","last_change":"2024-05-08T05:43:58.751868+0000","last_active":"2024-05-08T05:43:58.759456+0000","last_peered":"2024-05-08T05:43:58.759456+0000","last_clean":"2024-05-08T05:43:58.759456+0000","last_became_active":"2024-05-08T05:43:58.750775+0000","last_became_peered":"2024-05-08T05:43:58.750775+0000","last_unstale":"2024-05-08T05:43:58.759456+0000","last_undegraded":"2024-05-08T05:43:58.759456+0000","last_fullsized":"2024-05-08T05:43:58.759456+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:42:54.541460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.755834+0000","last_change":"2024-05-08T05:43:57.743388+0000","last_active":"2024-05-08T05:43:58.755834+0000","last_peered":"2024-05-08T05:43:58.755834+0000","last_clean":"2024-05-08T05:43:58.755834+0000","last_became_active":"2024-05-08T05:43:57.743068+0000","last_became_peered":"2024-05-08T05:43:57.743068+0000","last_unstale":"2024-05-08T05:43:58.755834+0000","last_undegraded":"2024-05-08T05:43:58.755834+0000","last_fullsized":"2024-05-08T05:43:58.755834+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:04:06.483452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.761474+0000","last_change":"2024-05-08T05:43:58.756116+0000","last_active":"2024-05-08T05:43:58.761474+0000","last_peered":"2024-05-08T05:43:58.761474+0000","last_clean":"2024-05-08T05:43:58.761474+0000","last_became_active":"2024-05-08T05:43:58.755813+0000","last_became_peered":"2024-05-08T05:43:58.755813+0000","last_unstale":"2024-05-08T05:43:58.761474+0000","last_undegraded":"2024-05-08T05:43:58.761474+0000","last_fullsized":"2024-05-08T05:43:58.761474+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:43:45.044208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.758795+0000","last_change":"2024-05-08T05:43:57.747701+0000","last_active":"2024-05-08T05:43:58.758795+0000","last_peered":"2024-05-08T05:43:58.758795+0000","last_clean":"2024-05-08T05:43:58.758795+0000","last_became_active":"2024-05-08T05:43:57.747194+0000","last_became_peered":"2024-05-08T05:43:57.747194+0000","last_unstale":"2024-05-08T05:43:58.758795+0000","last_undegraded":"2024-05-08T05:43:58.758795+0000","last_fullsized":"2024-05-08T05:43:58.758795+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:55:30.450723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.762468+0000","last_change":"2024-05-08T05:43:58.756883+0000","last_active":"2024-05-08T05:43:58.762468+0000","last_peered":"2024-05-08T05:43:58.762468+0000","last_clean":"2024-05-08T05:43:58.762468+0000","last_became_active":"2024-05-08T05:43:58.753172+0000","last_became_peered":"2024-05-08T05:43:58.753172+0000","last_unstale":"2024-05-08T05:43:58.762468+0000","last_undegraded":"2024-05-08T05:43:58.762468+0000","last_fullsized":"2024-05-08T05:43:58.762468+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:32:38.835058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.759979+0000","last_change":"2024-05-08T05:43:57.733816+0000","last_active":"2024-05-08T05:43:58.759979+0000","last_peered":"2024-05-08T05:43:58.759979+0000","last_clean":"2024-05-08T05:43:58.759979+0000","last_became_active":"2024-05-08T05:43:57.733506+0000","last_became_peered":"2024-05-08T05:43:57.733506+0000","last_unstale":"2024-05-08T05:43:58.759979+0000","last_undegraded":"2024-05-08T05:43:58.759979+0000","last_fullsized":"2024-05-08T05:43:58.759979+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:14.839460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.758003+0000","last_change":"2024-05-08T05:43:58.755042+0000","last_active":"2024-05-08T05:43:58.758003+0000","last_peered":"2024-05-08T05:43:58.758003+0000","last_clean":"2024-05-08T05:43:58.758003+0000","last_became_active":"2024-05-08T05:43:58.754741+0000","last_became_peered":"2024-05-08T05:43:58.754741+0000","last_unstale":"2024-05-08T05:43:58.758003+0000","last_undegraded":"2024-05-08T05:43:58.758003+0000","last_fullsized":"2024-05-08T05:43:58.758003+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:31:26.967780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.746253+0000","last_change":"2024-05-08T05:43:57.735454+0000","last_active":"2024-05-08T05:43:58.746253+0000","last_peered":"2024-05-08T05:43:58.746253+0000","last_clean":"2024-05-08T05:43:58.746253+0000","last_became_active":"2024-05-08T05:43:57.735160+0000","last_became_peered":"2024-05-08T05:43:57.735160+0000","last_unstale":"2024-05-08T05:43:58.746253+0000","last_undegraded":"2024-05-08T05:43:58.746253+0000","last_fullsized":"2024-05-08T05:43:58.746253+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:24:20.210950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.757762+0000","last_change":"2024-05-08T05:43:58.751432+0000","last_active":"2024-05-08T05:43:58.757762+0000","last_peered":"2024-05-08T05:43:58.757762+0000","last_clean":"2024-05-08T05:43:58.757762+0000","last_became_active":"2024-05-08T05:43:58.751121+0000","last_became_peered":"2024-05-08T05:43:58.751121+0000","last_unstale":"2024-05-08T05:43:58.757762+0000","last_undegraded":"2024-05-08T05:43:58.757762+0000","last_fullsized":"2024-05-08T05:43:58.757762+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:41.803526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.749553+0000","last_change":"2024-05-08T05:43:57.736625+0000","last_active":"2024-05-08T05:43:58.749553+0000","last_peered":"2024-05-08T05:43:58.749553+0000","last_clean":"2024-05-08T05:43:58.749553+0000","last_became_active":"2024-05-08T05:43:57.736294+0000","last_became_peered":"2024-05-08T05:43:57.736294+0000","last_unstale":"2024-05-08T05:43:58.749553+0000","last_undegraded":"2024-05-08T05:43:58.749553+0000","last_fullsized":"2024-05-08T05:43:58.749553+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:13:36.124929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.754421+0000","last_change":"2024-05-08T05:43:58.749825+0000","last_active":"2024-05-08T05:43:58.754421+0000","last_peered":"2024-05-08T05:43:58.754421+0000","last_clean":"2024-05-08T05:43:58.754421+0000","last_became_active":"2024-05-08T05:43:58.749544+0000","last_became_peered":"2024-05-08T05:43:58.749544+0000","last_unstale":"2024-05-08T05:43:58.754421+0000","last_undegraded":"2024-05-08T05:43:58.754421+0000","last_fullsized":"2024-05-08T05:43:58.754421+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:16:07.239208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.729951+0000","last_change":"2024-05-08T05:43:57.749640+0000","last_active":"2024-05-08T05:43:58.729951+0000","last_peered":"2024-05-08T05:43:58.729951+0000","last_clean":"2024-05-08T05:43:58.729951+0000","last_became_active":"2024-05-08T05:43:57.749307+0000","last_became_peered":"2024-05-08T05:43:57.749307+0000","last_unstale":"2024-05-08T05:43:58.729951+0000","last_undegraded":"2024-05-08T05:43:58.729951+0000","last_fullsized":"2024-05-08T05:43:58.729951+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:27.013912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.743229+0000","last_change":"2024-05-08T05:43:58.741349+0000","last_active":"2024-05-08T05:43:58.743229+0000","last_peered":"2024-05-08T05:43:58.743229+0000","last_clean":"2024-05-08T05:43:58.743229+0000","last_became_active":"2024-05-08T05:43:58.740934+0000","last_became_peered":"2024-05-08T05:43:58.740934+0000","last_unstale":"2024-05-08T05:43:58.743229+0000","last_undegraded":"2024-05-08T05:43:58.743229+0000","last_fullsized":"2024-05-08T05:43:58.743229+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:44:39.758942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.739656+0000","last_change":"2024-05-08T05:43:57.748752+0000","last_active":"2024-05-08T05:43:58.739656+0000","last_peered":"2024-05-08T05:43:58.739656+0000","last_clean":"2024-05-08T05:43:58.739656+0000","last_became_active":"2024-05-08T05:43:57.748472+0000","last_became_peered":"2024-05-08T05:43:57.748472+0000","last_unstale":"2024-05-08T05:43:58.739656+0000","last_undegraded":"2024-05-08T05:43:58.739656+0000","last_fullsized":"2024-05-08T05:43:58.739656+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:38.360748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.758999+0000","last_change":"2024-05-08T05:43:58.750812+0000","last_active":"2024-05-08T05:43:58.758999+0000","last_peered":"2024-05-08T05:43:58.758999+0000","last_clean":"2024-05-08T05:43:58.758999+0000","last_became_active":"2024-05-08T05:43:58.750509+0000","last_became_peered":"2024-05-08T05:43:58.750509+0000","last_unstale":"2024-05-08T05:43:58.758999+0000","last_undegraded":"2024-05-08T05:43:58.758999+0000","last_fullsized":"2024-05-08T05:43:58.758999+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:43:14.229950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.738722+0000","last_change":"2024-05-08T05:43:57.746791+0000","last_active":"2024-05-08T05:43:58.738722+0000","last_peered":"2024-05-08T05:43:58.738722+0000","last_clean":"2024-05-08T05:43:58.738722+0000","last_became_active":"2024-05-08T05:43:57.746493+0000","last_became_peered":"2024-05-08T05:43:57.746493+0000","last_unstale":"2024-05-08T05:43:58.738722+0000","last_undegraded":"2024-05-08T05:43:58.738722+0000","last_fullsized":"2024-05-08T05:43:58.738722+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:14.839460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.748962+0000","last_change":"2024-05-08T05:43:58.747505+0000","last_active":"2024-05-08T05:43:58.748962+0000","last_peered":"2024-05-08T05:43:58.748962+0000","last_clean":"2024-05-08T05:43:58.748962+0000","last_became_active":"2024-05-08T05:43:58.747135+0000","last_became_peered":"2024-05-08T05:43:58.747135+0000","last_unstale":"2024-05-08T05:43:58.748962+0000","last_undegraded":"2024-05-08T05:43:58.748962+0000","last_fullsized":"2024-05-08T05:43:58.748962+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:20:05.922731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.732033+0000","last_change":"2024-05-08T05:43:57.742766+0000","last_active":"2024-05-08T05:43:58.732033+0000","last_peered":"2024-05-08T05:43:58.732033+0000","last_clean":"2024-05-08T05:43:58.732033+0000","last_became_active":"2024-05-08T05:43:57.742473+0000","last_became_peered":"2024-05-08T05:43:57.742473+0000","last_unstale":"2024-05-08T05:43:58.732033+0000","last_undegraded":"2024-05-08T05:43:58.732033+0000","last_fullsized":"2024-05-08T05:43:58.732033+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:48:09.974150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.756241+0000","last_change":"2024-05-08T05:43:58.752828+0000","last_active":"2024-05-08T05:43:58.756241+0000","last_peered":"2024-05-08T05:43:58.756241+0000","last_clean":"2024-05-08T05:43:58.756241+0000","last_became_active":"2024-05-08T05:43:58.752545+0000","last_became_peered":"2024-05-08T05:43:58.752545+0000","last_unstale":"2024-05-08T05:43:58.756241+0000","last_undegraded":"2024-05-08T05:43:58.756241+0000","last_fullsized":"2024-05-08T05:43:58.756241+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:14:23.447592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"60'3","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.761358+0000","last_change":"2024-05-08T05:43:57.747237+0000","last_active":"2024-05-08T05:43:58.761358+0000","last_peered":"2024-05-08T05:43:58.761358+0000","last_clean":"2024-05-08T05:43:58.761358+0000","last_became_active":"2024-05-08T05:43:57.746946+0000","last_became_peered":"2024-05-08T05:43:57.746946+0000","last_unstale":"2024-05-08T05:43:58.761358+0000","last_undegraded":"2024-05-08T05:43:58.761358+0000","last_fullsized":"2024-05-08T05:43:58.761358+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:07.840563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.745309+0000","last_change":"2024-05-08T05:43:58.743511+0000","last_active":"2024-05-08T05:43:58.745309+0000","last_peered":"2024-05-08T05:43:58.745309+0000","last_clean":"2024-05-08T05:43:58.745309+0000","last_became_active":"2024-05-08T05:43:58.743200+0000","last_became_peered":"2024-05-08T05:43:58.743200+0000","last_unstale":"2024-05-08T05:43:58.745309+0000","last_undegraded":"2024-05-08T05:43:58.745309+0000","last_fullsized":"2024-05-08T05:43:58.745309+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:20:41.789503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"60'6","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.765418+0000","last_change":"2024-05-08T05:43:57.741712+0000","last_active":"2024-05-08T05:43:58.765418+0000","last_peered":"2024-05-08T05:43:58.765418+0000","last_clean":"2024-05-08T05:43:58.765418+0000","last_became_active":"2024-05-08T05:43:57.741244+0000","last_became_peered":"2024-05-08T05:43:57.741244+0000","last_unstale":"2024-05-08T05:43:58.765418+0000","last_undegraded":"2024-05-08T05:43:58.765418+0000","last_fullsized":"2024-05-08T05:43:58.765418+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:52.963839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.754036+0000","last_change":"2024-05-08T05:43:58.750653+0000","last_active":"2024-05-08T05:43:58.754036+0000","last_peered":"2024-05-08T05:43:58.754036+0000","last_clean":"2024-05-08T05:43:58.754036+0000","last_became_active":"2024-05-08T05:43:58.750351+0000","last_became_peered":"2024-05-08T05:43:58.750351+0000","last_unstale":"2024-05-08T05:43:58.754036+0000","last_undegraded":"2024-05-08T05:43:58.754036+0000","last_fullsized":"2024-05-08T05:43:58.754036+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:26:16.801149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.759076+0000","last_change":"2024-05-08T05:43:57.751419+0000","last_active":"2024-05-08T05:43:58.759076+0000","last_peered":"2024-05-08T05:43:58.759076+0000","last_clean":"2024-05-08T05:43:58.759076+0000","last_became_active":"2024-05-08T05:43:57.751058+0000","last_became_peered":"2024-05-08T05:43:57.751058+0000","last_unstale":"2024-05-08T05:43:58.759076+0000","last_undegraded":"2024-05-08T05:43:58.759076+0000","last_fullsized":"2024-05-08T05:43:58.759076+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:28:18.198830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.747530+0000","last_change":"2024-05-08T05:43:58.744627+0000","last_active":"2024-05-08T05:43:58.747530+0000","last_peered":"2024-05-08T05:43:58.747530+0000","last_clean":"2024-05-08T05:43:58.747530+0000","last_became_active":"2024-05-08T05:43:58.744329+0000","last_became_peered":"2024-05-08T05:43:58.744329+0000","last_unstale":"2024-05-08T05:43:58.747530+0000","last_undegraded":"2024-05-08T05:43:58.747530+0000","last_fullsized":"2024-05-08T05:43:58.747530+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:54.914888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.738119+0000","last_change":"2024-05-08T05:43:57.740618+0000","last_active":"2024-05-08T05:43:58.738119+0000","last_peered":"2024-05-08T05:43:58.738119+0000","last_clean":"2024-05-08T05:43:58.738119+0000","last_became_active":"2024-05-08T05:43:57.740317+0000","last_became_peered":"2024-05-08T05:43:57.740317+0000","last_unstale":"2024-05-08T05:43:58.738119+0000","last_undegraded":"2024-05-08T05:43:58.738119+0000","last_fullsized":"2024-05-08T05:43:58.738119+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:53:54.948798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.754441+0000","last_change":"2024-05-08T05:43:58.751349+0000","last_active":"2024-05-08T05:43:58.754441+0000","last_peered":"2024-05-08T05:43:58.754441+0000","last_clean":"2024-05-08T05:43:58.754441+0000","last_became_active":"2024-05-08T05:43:58.751043+0000","last_became_peered":"2024-05-08T05:43:58.751043+0000","last_unstale":"2024-05-08T05:43:58.754441+0000","last_undegraded":"2024-05-08T05:43:58.754441+0000","last_fullsized":"2024-05-08T05:43:58.754441+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:30:31.262850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.741282+0000","last_change":"2024-05-08T05:43:57.748020+0000","last_active":"2024-05-08T05:43:58.741282+0000","last_peered":"2024-05-08T05:43:58.741282+0000","last_clean":"2024-05-08T05:43:58.741282+0000","last_became_active":"2024-05-08T05:43:57.746146+0000","last_became_peered":"2024-05-08T05:43:57.746146+0000","last_unstale":"2024-05-08T05:43:58.741282+0000","last_undegraded":"2024-05-08T05:43:58.741282+0000","last_fullsized":"2024-05-08T05:43:58.741282+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:52:48.111992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.761229+0000","last_change":"2024-05-08T05:43:58.753970+0000","last_active":"2024-05-08T05:43:58.761229+0000","last_peered":"2024-05-08T05:43:58.761229+0000","last_clean":"2024-05-08T05:43:58.761229+0000","last_became_active":"2024-05-08T05:43:58.753343+0000","last_became_peered":"2024-05-08T05:43:58.753343+0000","last_unstale":"2024-05-08T05:43:58.761229+0000","last_undegraded":"2024-05-08T05:43:58.761229+0000","last_fullsized":"2024-05-08T05:43:58.761229+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:07:45.223261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"60'8","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.758590+0000","last_change":"2024-05-08T05:43:57.747887+0000","last_active":"2024-05-08T05:43:58.758590+0000","last_peered":"2024-05-08T05:43:58.758590+0000","last_clean":"2024-05-08T05:43:58.758590+0000","last_became_active":"2024-05-08T05:43:57.747610+0000","last_became_peered":"2024-05-08T05:43:57.747610+0000","last_unstale":"2024-05-08T05:43:58.758590+0000","last_undegraded":"2024-05-08T05:43:58.758590+0000","last_fullsized":"2024-05-08T05:43:58.758590+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:21:53.097435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.762683+0000","last_change":"2024-05-08T05:43:58.759904+0000","last_active":"2024-05-08T05:43:58.762683+0000","last_peered":"2024-05-08T05:43:58.762683+0000","last_clean":"2024-05-08T05:43:58.762683+0000","last_became_active":"2024-05-08T05:43:58.759320+0000","last_became_peered":"2024-05-08T05:43:58.759320+0000","last_unstale":"2024-05-08T05:43:58.762683+0000","last_undegraded":"2024-05-08T05:43:58.762683+0000","last_fullsized":"2024-05-08T05:43:58.762683+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:20.614792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.762679+0000","last_change":"2024-05-08T05:43:57.739627+0000","last_active":"2024-05-08T05:43:58.762679+0000","last_peered":"2024-05-08T05:43:58.762679+0000","last_clean":"2024-05-08T05:43:58.762679+0000","last_became_active":"2024-05-08T05:43:57.739307+0000","last_became_peered":"2024-05-08T05:43:57.739307+0000","last_unstale":"2024-05-08T05:43:58.762679+0000","last_undegraded":"2024-05-08T05:43:58.762679+0000","last_fullsized":"2024-05-08T05:43:58.762679+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:27:40.487201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.755452+0000","last_change":"2024-05-08T05:43:58.746262+0000","last_active":"2024-05-08T05:43:58.755452+0000","last_peered":"2024-05-08T05:43:58.755452+0000","last_clean":"2024-05-08T05:43:58.755452+0000","last_became_active":"2024-05-08T05:43:58.745829+0000","last_became_peered":"2024-05-08T05:43:58.745829+0000","last_unstale":"2024-05-08T05:43:58.755452+0000","last_undegraded":"2024-05-08T05:43:58.755452+0000","last_fullsized":"2024-05-08T05:43:58.755452+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:42:54.541460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.760575+0000","last_change":"2024-05-08T05:43:57.745920+0000","last_active":"2024-05-08T05:43:58.760575+0000","last_peered":"2024-05-08T05:43:58.760575+0000","last_clean":"2024-05-08T05:43:58.760575+0000","last_became_active":"2024-05-08T05:43:57.745608+0000","last_became_peered":"2024-05-08T05:43:57.745608+0000","last_unstale":"2024-05-08T05:43:58.760575+0000","last_undegraded":"2024-05-08T05:43:58.760575+0000","last_fullsized":"2024-05-08T05:43:58.760575+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:46:43.642863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.754099+0000","last_change":"2024-05-08T05:43:58.750371+0000","last_active":"2024-05-08T05:43:58.754099+0000","last_peered":"2024-05-08T05:43:58.754099+0000","last_clean":"2024-05-08T05:43:58.754099+0000","last_became_active":"2024-05-08T05:43:58.750094+0000","last_became_peered":"2024-05-08T05:43:58.750094+0000","last_unstale":"2024-05-08T05:43:58.754099+0000","last_undegraded":"2024-05-08T05:43:58.754099+0000","last_fullsized":"2024-05-08T05:43:58.754099+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:14.821494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.761368+0000","last_change":"2024-05-08T05:43:57.747485+0000","last_active":"2024-05-08T05:43:58.761368+0000","last_peered":"2024-05-08T05:43:58.761368+0000","last_clean":"2024-05-08T05:43:58.761368+0000","last_became_active":"2024-05-08T05:43:57.747166+0000","last_became_peered":"2024-05-08T05:43:57.747166+0000","last_unstale":"2024-05-08T05:43:58.761368+0000","last_undegraded":"2024-05-08T05:43:58.761368+0000","last_fullsized":"2024-05-08T05:43:58.761368+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:53.579207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.754344+0000","last_change":"2024-05-08T05:43:58.749232+0000","last_active":"2024-05-08T05:43:58.754344+0000","last_peered":"2024-05-08T05:43:58.754344+0000","last_clean":"2024-05-08T05:43:58.754344+0000","last_became_active":"2024-05-08T05:43:58.748925+0000","last_became_peered":"2024-05-08T05:43:58.748925+0000","last_unstale":"2024-05-08T05:43:58.754344+0000","last_undegraded":"2024-05-08T05:43:58.754344+0000","last_fullsized":"2024-05-08T05:43:58.754344+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:37.988934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.730815+0000","last_change":"2024-05-08T05:43:57.740755+0000","last_active":"2024-05-08T05:43:58.730815+0000","last_peered":"2024-05-08T05:43:58.730815+0000","last_clean":"2024-05-08T05:43:58.730815+0000","last_became_active":"2024-05-08T05:43:57.740425+0000","last_became_peered":"2024-05-08T05:43:57.740425+0000","last_unstale":"2024-05-08T05:43:58.730815+0000","last_undegraded":"2024-05-08T05:43:58.730815+0000","last_fullsized":"2024-05-08T05:43:58.730815+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:40:34.407373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.758794+0000","last_change":"2024-05-08T05:43:58.755187+0000","last_active":"2024-05-08T05:43:58.758794+0000","last_peered":"2024-05-08T05:43:58.758794+0000","last_clean":"2024-05-08T05:43:58.758794+0000","last_became_active":"2024-05-08T05:43:58.754895+0000","last_became_peered":"2024-05-08T05:43:58.754895+0000","last_unstale":"2024-05-08T05:43:58.758794+0000","last_undegraded":"2024-05-08T05:43:58.758794+0000","last_fullsized":"2024-05-08T05:43:58.758794+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:32:38.835058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.754522+0000","last_change":"2024-05-08T05:43:57.751066+0000","last_active":"2024-05-08T05:43:58.754522+0000","last_peered":"2024-05-08T05:43:58.754522+0000","last_clean":"2024-05-08T05:43:58.754522+0000","last_became_active":"2024-05-08T05:43:57.750846+0000","last_became_peered":"2024-05-08T05:43:57.750846+0000","last_unstale":"2024-05-08T05:43:58.754522+0000","last_undegraded":"2024-05-08T05:43:58.754522+0000","last_fullsized":"2024-05-08T05:43:58.754522+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:52:48.111992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.755940+0000","last_change":"2024-05-08T05:43:58.754633+0000","last_active":"2024-05-08T05:43:58.755940+0000","last_peered":"2024-05-08T05:43:58.755940+0000","last_clean":"2024-05-08T05:43:58.755940+0000","last_became_active":"2024-05-08T05:43:58.754348+0000","last_became_peered":"2024-05-08T05:43:58.754348+0000","last_unstale":"2024-05-08T05:43:58.755940+0000","last_undegraded":"2024-05-08T05:43:58.755940+0000","last_fullsized":"2024-05-08T05:43:58.755940+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:09:12.018896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.738308+0000","last_change":"2024-05-08T05:43:57.745869+0000","last_active":"2024-05-08T05:43:58.738308+0000","last_peered":"2024-05-08T05:43:58.738308+0000","last_clean":"2024-05-08T05:43:58.738308+0000","last_became_active":"2024-05-08T05:43:57.745542+0000","last_became_peered":"2024-05-08T05:43:57.745542+0000","last_unstale":"2024-05-08T05:43:58.738308+0000","last_undegraded":"2024-05-08T05:43:58.738308+0000","last_fullsized":"2024-05-08T05:43:58.738308+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:07:44.220953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.752780+0000","last_change":"2024-05-08T05:43:58.746894+0000","last_active":"2024-05-08T05:43:58.752780+0000","last_peered":"2024-05-08T05:43:58.752780+0000","last_clean":"2024-05-08T05:43:58.752780+0000","last_became_active":"2024-05-08T05:43:58.746605+0000","last_became_peered":"2024-05-08T05:43:58.746605+0000","last_unstale":"2024-05-08T05:43:58.752780+0000","last_undegraded":"2024-05-08T05:43:58.752780+0000","last_fullsized":"2024-05-08T05:43:58.752780+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:54.358135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.739566+0000","last_change":"2024-05-08T05:43:57.745921+0000","last_active":"2024-05-08T05:43:58.739566+0000","last_peered":"2024-05-08T05:43:58.739566+0000","last_clean":"2024-05-08T05:43:58.739566+0000","last_became_active":"2024-05-08T05:43:57.745575+0000","last_became_peered":"2024-05-08T05:43:57.745575+0000","last_unstale":"2024-05-08T05:43:58.739566+0000","last_undegraded":"2024-05-08T05:43:58.739566+0000","last_fullsized":"2024-05-08T05:43:58.739566+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:17:32.938067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.748519+0000","last_change":"2024-05-08T05:43:58.746684+0000","last_active":"2024-05-08T05:43:58.748519+0000","last_peered":"2024-05-08T05:43:58.748519+0000","last_clean":"2024-05-08T05:43:58.748519+0000","last_became_active":"2024-05-08T05:43:58.746390+0000","last_became_peered":"2024-05-08T05:43:58.746390+0000","last_unstale":"2024-05-08T05:43:58.748519+0000","last_undegraded":"2024-05-08T05:43:58.748519+0000","last_fullsized":"2024-05-08T05:43:58.748519+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:04.995283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"60'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.762954+0000","last_change":"2024-05-08T05:43:57.742991+0000","last_active":"2024-05-08T05:43:58.762954+0000","last_peered":"2024-05-08T05:43:58.762954+0000","last_clean":"2024-05-08T05:43:58.762954+0000","last_became_active":"2024-05-08T05:43:57.742632+0000","last_became_peered":"2024-05-08T05:43:57.742632+0000","last_unstale":"2024-05-08T05:43:58.762954+0000","last_undegraded":"2024-05-08T05:43:58.762954+0000","last_fullsized":"2024-05-08T05:43:58.762954+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:31:51.331036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.758758+0000","last_change":"2024-05-08T05:43:58.757224+0000","last_active":"2024-05-08T05:43:58.758758+0000","last_peered":"2024-05-08T05:43:58.758758+0000","last_clean":"2024-05-08T05:43:58.758758+0000","last_became_active":"2024-05-08T05:43:58.756952+0000","last_became_peered":"2024-05-08T05:43:58.756952+0000","last_unstale":"2024-05-08T05:43:58.758758+0000","last_undegraded":"2024-05-08T05:43:58.758758+0000","last_fullsized":"2024-05-08T05:43:58.758758+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:33:51.903970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.737116+0000","last_change":"2024-05-08T05:43:57.746159+0000","last_active":"2024-05-08T05:43:58.737116+0000","last_peered":"2024-05-08T05:43:58.737116+0000","last_clean":"2024-05-08T05:43:58.737116+0000","last_became_active":"2024-05-08T05:43:57.745862+0000","last_became_peered":"2024-05-08T05:43:57.745862+0000","last_unstale":"2024-05-08T05:43:58.737116+0000","last_undegraded":"2024-05-08T05:43:58.737116+0000","last_fullsized":"2024-05-08T05:43:58.737116+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:23:27.228227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.751395+0000","last_change":"2024-05-08T05:43:58.746462+0000","last_active":"2024-05-08T05:43:58.751395+0000","last_peered":"2024-05-08T05:43:58.751395+0000","last_clean":"2024-05-08T05:43:58.751395+0000","last_became_active":"2024-05-08T05:43:58.746114+0000","last_became_peered":"2024-05-08T05:43:58.746114+0000","last_unstale":"2024-05-08T05:43:58.751395+0000","last_undegraded":"2024-05-08T05:43:58.751395+0000","last_fullsized":"2024-05-08T05:43:58.751395+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:09:37.555566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.740304+0000","last_change":"2024-05-08T05:43:57.747447+0000","last_active":"2024-05-08T05:43:58.740304+0000","last_peered":"2024-05-08T05:43:58.740304+0000","last_clean":"2024-05-08T05:43:58.740304+0000","last_became_active":"2024-05-08T05:43:57.746856+0000","last_became_peered":"2024-05-08T05:43:57.746856+0000","last_unstale":"2024-05-08T05:43:58.740304+0000","last_undegraded":"2024-05-08T05:43:58.740304+0000","last_fullsized":"2024-05-08T05:43:58.740304+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:24:57.633695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.754385+0000","last_change":"2024-05-08T05:43:58.749389+0000","last_active":"2024-05-08T05:43:58.754385+0000","last_peered":"2024-05-08T05:43:58.754385+0000","last_clean":"2024-05-08T05:43:58.754385+0000","last_became_active":"2024-05-08T05:43:58.749069+0000","last_became_peered":"2024-05-08T05:43:58.749069+0000","last_unstale":"2024-05-08T05:43:58.754385+0000","last_undegraded":"2024-05-08T05:43:58.754385+0000","last_fullsized":"2024-05-08T05:43:58.754385+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:43:13.328768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.735732+0000","last_change":"2024-05-08T05:43:57.741565+0000","last_active":"2024-05-08T05:43:58.735732+0000","last_peered":"2024-05-08T05:43:58.735732+0000","last_clean":"2024-05-08T05:43:58.735732+0000","last_became_active":"2024-05-08T05:43:57.740849+0000","last_became_peered":"2024-05-08T05:43:57.740849+0000","last_unstale":"2024-05-08T05:43:58.735732+0000","last_undegraded":"2024-05-08T05:43:58.735732+0000","last_fullsized":"2024-05-08T05:43:58.735732+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:44:45.896339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.752744+0000","last_change":"2024-05-08T05:43:58.746874+0000","last_active":"2024-05-08T05:43:58.752744+0000","last_peered":"2024-05-08T05:43:58.752744+0000","last_clean":"2024-05-08T05:43:58.752744+0000","last_became_active":"2024-05-08T05:43:58.746566+0000","last_became_peered":"2024-05-08T05:43:58.746566+0000","last_unstale":"2024-05-08T05:43:58.752744+0000","last_undegraded":"2024-05-08T05:43:58.752744+0000","last_fullsized":"2024-05-08T05:43:58.752744+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:18:07.653809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.735621+0000","last_change":"2024-05-08T05:43:57.748766+0000","last_active":"2024-05-08T05:43:58.735621+0000","last_peered":"2024-05-08T05:43:58.735621+0000","last_clean":"2024-05-08T05:43:58.735621+0000","last_became_active":"2024-05-08T05:43:57.748400+0000","last_became_peered":"2024-05-08T05:43:57.748400+0000","last_unstale":"2024-05-08T05:43:58.735621+0000","last_undegraded":"2024-05-08T05:43:58.735621+0000","last_fullsized":"2024-05-08T05:43:58.735621+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:09:11.016588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.762168+0000","last_change":"2024-05-08T05:43:58.758339+0000","last_active":"2024-05-08T05:43:58.762168+0000","last_peered":"2024-05-08T05:43:58.762168+0000","last_clean":"2024-05-08T05:43:58.762168+0000","last_became_active":"2024-05-08T05:43:58.758049+0000","last_became_peered":"2024-05-08T05:43:58.758049+0000","last_unstale":"2024-05-08T05:43:58.762168+0000","last_undegraded":"2024-05-08T05:43:58.762168+0000","last_fullsized":"2024-05-08T05:43:58.762168+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:28:37.988934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.730797+0000","last_change":"2024-05-08T05:43:57.737654+0000","last_active":"2024-05-08T05:43:58.730797+0000","last_peered":"2024-05-08T05:43:58.730797+0000","last_clean":"2024-05-08T05:43:58.730797+0000","last_became_active":"2024-05-08T05:43:57.737354+0000","last_became_peered":"2024-05-08T05:43:57.737354+0000","last_unstale":"2024-05-08T05:43:58.730797+0000","last_undegraded":"2024-05-08T05:43:58.730797+0000","last_fullsized":"2024-05-08T05:43:58.730797+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:26.792126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.760957+0000","last_change":"2024-05-08T05:43:58.751882+0000","last_active":"2024-05-08T05:43:58.760957+0000","last_peered":"2024-05-08T05:43:58.760957+0000","last_clean":"2024-05-08T05:43:58.760957+0000","last_became_active":"2024-05-08T05:43:58.751618+0000","last_became_peered":"2024-05-08T05:43:58.751618+0000","last_unstale":"2024-05-08T05:43:58.760957+0000","last_undegraded":"2024-05-08T05:43:58.760957+0000","last_fullsized":"2024-05-08T05:43:58.760957+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:09.681994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.730118+0000","last_change":"2024-05-08T05:43:57.749487+0000","last_active":"2024-05-08T05:43:58.730118+0000","last_peered":"2024-05-08T05:43:58.730118+0000","last_clean":"2024-05-08T05:43:58.730118+0000","last_became_active":"2024-05-08T05:43:57.749221+0000","last_became_peered":"2024-05-08T05:43:57.749221+0000","last_unstale":"2024-05-08T05:43:58.730118+0000","last_undegraded":"2024-05-08T05:43:58.730118+0000","last_fullsized":"2024-05-08T05:43:58.730118+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:41:52.162297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.762049+0000","last_change":"2024-05-08T05:43:58.760285+0000","last_active":"2024-05-08T05:43:58.762049+0000","last_peered":"2024-05-08T05:43:58.762049+0000","last_clean":"2024-05-08T05:43:58.762049+0000","last_became_active":"2024-05-08T05:43:58.760019+0000","last_became_peered":"2024-05-08T05:43:58.760019+0000","last_unstale":"2024-05-08T05:43:58.762049+0000","last_undegraded":"2024-05-08T05:43:58.762049+0000","last_fullsized":"2024-05-08T05:43:58.762049+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:25:55.521438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.735570+0000","last_change":"2024-05-08T05:43:57.748220+0000","last_active":"2024-05-08T05:43:58.735570+0000","last_peered":"2024-05-08T05:43:58.735570+0000","last_clean":"2024-05-08T05:43:58.735570+0000","last_became_active":"2024-05-08T05:43:57.747917+0000","last_became_peered":"2024-05-08T05:43:57.747917+0000","last_unstale":"2024-05-08T05:43:58.735570+0000","last_undegraded":"2024-05-08T05:43:58.735570+0000","last_fullsized":"2024-05-08T05:43:58.735570+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:13:36.764103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.764336+0000","last_change":"2024-05-08T05:43:58.756951+0000","last_active":"2024-05-08T05:43:58.764336+0000","last_peered":"2024-05-08T05:43:58.764336+0000","last_clean":"2024-05-08T05:43:58.764336+0000","last_became_active":"2024-05-08T05:43:58.756695+0000","last_became_peered":"2024-05-08T05:43:58.756695+0000","last_unstale":"2024-05-08T05:43:58.764336+0000","last_undegraded":"2024-05-08T05:43:58.764336+0000","last_fullsized":"2024-05-08T05:43:58.764336+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:57.717485+0000","last_clean_scrub_stamp":"2024-05-08T05:43:57.717485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:12:29.066783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.729834+0000","last_change":"2024-05-08T05:43:57.734841+0000","last_active":"2024-05-08T05:43:58.729834+0000","last_peered":"2024-05-08T05:43:58.729834+0000","last_clean":"2024-05-08T05:43:58.729834+0000","last_became_active":"2024-05-08T05:43:57.734495+0000","last_became_peered":"2024-05-08T05:43:57.734495+0000","last_unstale":"2024-05-08T05:43:58.729834+0000","last_undegraded":"2024-05-08T05:43:58.729834+0000","last_fullsized":"2024-05-08T05:43:58.729834+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:43:56.715177+0000","last_clean_scrub_stamp":"2024-05-08T05:43:56.715177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:19.612484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":184,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T05:43:58.737245+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:43:58.737245+0000","last_peered":"2024-05-08T05:43:58.737245+0000","last_clean":"2024-05-08T05:43:58.737245+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:43:58.737245+0000","last_undegraded":"2024-05-08T05:43:58.737245+0000","last_fullsized":"2024-05-08T05:43:58.737245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542180,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28320,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343520,"statfs":{"total":96636764160,"available":96607764480,"internally_reserved":0,"allocated":1331200,"data_stored":912580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542181,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":724992,"data_stored":321470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":724992,"data_stored":321458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":314269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":314269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":314269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":314269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1302528,"data_stored":904637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:44:03.152 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T05:44:03.152 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T05:44:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.299+0000 7f5e8fc99640 1 Processor -- start 2024-05-08T05:44:03.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.299+0000 7f5e8fc99640 1 -- start start 2024-05-08T05:44:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.300+0000 7f5e8fc99640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5e880cfcd0 0x7f5e880d4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.300+0000 7f5e8fc99640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e880d0320 con 0x7f5e880d54e0 2024-05-08T05:44:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.300+0000 7f5e8fc99640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e880d0600 con 0x7f5e8815f4e0 2024-05-08T05:44:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.300+0000 7f5e8fc99640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f5e880d41b0 con 0x7f5e880cfcd0 2024-05-08T05:44:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.300+0000 7f5e8d234640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5e880cfcd0 0x7f5e880d4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.300+0000 7f5e8e236640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5e880d54e0 0x7f5e880d13c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53520/0 (socket says 172.21.15.45:53520) 2024-05-08T05:44:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.300+0000 7f5e8e236640 1 -- 172.21.15.45:0/1591484059 learned_addr learned my addr 172.21.15.45:0/1591484059 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:44:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.301+0000 7f5e8d234640 1 -- 172.21.15.45:0/1591484059 >> v1:172.21.15.63:6789/0 conn(0x7f5e8815f4e0 legacy=0x7f5e880d3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.301+0000 7f5e8d234640 1 -- 172.21.15.45:0/1591484059 >> v1:172.21.15.45:6789/0 conn(0x7f5e880d54e0 legacy=0x7f5e880d13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.301+0000 7f5e8ca33640 1 -- 172.21.15.45:0/1591484059 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 541081704 0 0) 0x7f5e880d0320 con 0x7f5e880d54e0 2024-05-08T05:44:03.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.301+0000 7f5e8d234640 1 -- 172.21.15.45:0/1591484059 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e880d08e0 con 0x7f5e880cfcd0 2024-05-08T05:44:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.301+0000 7f5e8d234640 1 --2- 172.21.15.45:0/1591484059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e880cfcd0 0x7f5e880d4fa0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0097a0 tx=0x7f5e7c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125daeefbf99b5de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.301+0000 7f5e8ca33640 1 -- 172.21.15.45:0/1591484059 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c037070 con 0x7f5e880cfcd0 2024-05-08T05:44:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.302+0000 7f5e8ca33640 1 -- 172.21.15.45:0/1591484059 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e7c032440 con 0x7f5e880cfcd0 2024-05-08T05:44:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.302+0000 7f5e8ca33640 1 -- 172.21.15.45:0/1591484059 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c032810 con 0x7f5e880cfcd0 2024-05-08T05:44:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.302+0000 7f5e8fc99640 1 -- 172.21.15.45:0/1591484059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e880cfcd0 msgr2=0x7f5e880d4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.302+0000 7f5e8fc99640 1 --2- 172.21.15.45:0/1591484059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e880cfcd0 0x7f5e880d4fa0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0097a0 tx=0x7f5e7c02a230 comp rx=0 tx=0).stop 2024-05-08T05:44:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.302+0000 7f5e8fc99640 1 -- 172.21.15.45:0/1591484059 shutdown_connections 2024-05-08T05:44:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.302+0000 7f5e8fc99640 1 --2- 172.21.15.45:0/1591484059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e880cfcd0 0x7f5e880d4fa0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.302+0000 7f5e8fc99640 1 -- 172.21.15.45:0/1591484059 >> 172.21.15.45:0/1591484059 conn(0x7f5e8808eac0 msgr2=0x7f5e8809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.302+0000 7f5e8fc99640 1 -- 172.21.15.45:0/1591484059 shutdown_connections 2024-05-08T05:44:03.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.302+0000 7f5e8fc99640 1 -- 172.21.15.45:0/1591484059 wait complete. 2024-05-08T05:44:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.303+0000 7f5e8fc99640 1 Processor -- start 2024-05-08T05:44:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.303+0000 7f5e8fc99640 1 -- start start 2024-05-08T05:44:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.304+0000 7f5e8fc99640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e880cfcd0 0x7f5e88158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.304+0000 7f5e8fc99640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e880d54e0 0x7f5e881f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.304+0000 7f5e8fc99640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e8815f4e0 0x7f5e880b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.304+0000 7f5e8fc99640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8815f8c0 con 0x7f5e8815f4e0 2024-05-08T05:44:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.304+0000 7f5e8fc99640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5e880d0ea0 con 0x7f5e880cfcd0 2024-05-08T05:44:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.304+0000 7f5e8fc99640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5e880d1ad0 con 0x7f5e880d54e0 2024-05-08T05:44:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.304+0000 7f5e8e236640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e8815f4e0 0x7f5e880b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.304+0000 7f5e8e236640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e8815f4e0 0x7f5e880b2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36678/0 (socket says 172.21.15.45:36678) 2024-05-08T05:44:03.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.304+0000 7f5e8e236640 1 -- 172.21.15.45:0/3463678639 learned_addr learned my addr 172.21.15.45:0/3463678639 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.304+0000 7f5e8d234640 1 --2- 172.21.15.45:0/3463678639 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e880d54e0 0x7f5e881f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.304+0000 7f5e8da35640 1 --2- 172.21.15.45:0/3463678639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e880cfcd0 0x7f5e88158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.304+0000 7f5e8e236640 1 -- 172.21.15.45:0/3463678639 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e880d54e0 msgr2=0x7f5e881f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.304+0000 7f5e8e236640 1 --2- 172.21.15.45:0/3463678639 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e880d54e0 0x7f5e881f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.304+0000 7f5e8e236640 1 -- 172.21.15.45:0/3463678639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e880cfcd0 msgr2=0x7f5e88158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.304+0000 7f5e8e236640 1 --2- 172.21.15.45:0/3463678639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e880cfcd0 0x7f5e88158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.305+0000 7f5e8e236640 1 -- 172.21.15.45:0/3463678639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e880cc330 con 0x7f5e8815f4e0 2024-05-08T05:44:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.305+0000 7f5e8d234640 1 --2- 172.21.15.45:0/3463678639 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e880d54e0 0x7f5e881f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:44:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.305+0000 7f5e8e236640 1 --2- 172.21.15.45:0/3463678639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e8815f4e0 0x7f5e880b2220 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000c410 tx=0x7f5e8000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.305+0000 7f5e867fc640 1 -- 172.21.15.45:0/3463678639 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e800074e0 con 0x7f5e8815f4e0 2024-05-08T05:44:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.305+0000 7f5e867fc640 1 -- 172.21.15.45:0/3463678639 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e800047e0 con 0x7f5e8815f4e0 2024-05-08T05:44:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.305+0000 7f5e8fc99640 1 -- 172.21.15.45:0/3463678639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e880cc610 con 0x7f5e8815f4e0 2024-05-08T05:44:03.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.305+0000 7f5e867fc640 1 -- 172.21.15.45:0/3463678639 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80004bb0 con 0x7f5e8815f4e0 2024-05-08T05:44:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.305+0000 7f5e8fc99640 1 -- 172.21.15.45:0/3463678639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e880ccbd0 con 0x7f5e8815f4e0 2024-05-08T05:44:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.306+0000 7f5e8fc99640 1 -- 172.21.15.45:0/3463678639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e880693f0 con 0x7f5e8815f4e0 2024-05-08T05:44:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.306+0000 7f5e867fc640 1 -- 172.21.15.45:0/3463678639 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5e80011070 con 0x7f5e8815f4e0 2024-05-08T05:44:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.307+0000 7f5e867fc640 1 --2- 172.21.15.45:0/3463678639 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5e740762d0 0x7f5e74078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:03.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.307+0000 7f5e867fc640 1 -- 172.21.15.45:0/3463678639 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f5e80097200 con 0x7f5e8815f4e0 2024-05-08T05:44:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.307+0000 7f5e8da35640 1 --2- 172.21.15.45:0/3463678639 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5e740762d0 0x7f5e74078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:03.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.308+0000 7f5e8da35640 1 --2- 172.21.15.45:0/3463678639 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5e740762d0 0x7f5e74078790 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5e780012e0 tx=0x7f5e78009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:03.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.311+0000 7f5e867fc640 1 -- 172.21.15.45:0/3463678639 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5e80060a60 con 0x7f5e8815f4e0 2024-05-08T05:44:03.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.576+0000 7f5e8fc99640 1 -- 172.21.15.45:0/3463678639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f5e8806ed50 con 0x7f5e8815f4e0 2024-05-08T05:44:03.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.578+0000 7f5e867fc640 1 -- 172.21.15.45:0/3463678639 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f5e80060360 con 0x7f5e8815f4e0 2024-05-08T05:44:03.578 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-05-08T05:44:03.579 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBUEDtmDcqGMxAALgQaBYp4Zd8uMqswCXvGaw== 2024-05-08T05:44:03.579 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2024-05-08T05:44:03.579 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2024-05-08T05:44:03.579 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-08T05:44:03.579 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2024-05-08T05:44:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.580+0000 7f5e8fc99640 1 -- 172.21.15.45:0/3463678639 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5e740762d0 msgr2=0x7f5e74078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.580+0000 7f5e8fc99640 1 --2- 172.21.15.45:0/3463678639 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5e740762d0 0x7f5e74078790 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5e780012e0 tx=0x7f5e78009040 comp rx=0 tx=0).stop 2024-05-08T05:44:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.580+0000 7f5e8fc99640 1 -- 172.21.15.45:0/3463678639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e8815f4e0 msgr2=0x7f5e880b2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.580+0000 7f5e8fc99640 1 --2- 172.21.15.45:0/3463678639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e8815f4e0 0x7f5e880b2220 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000c410 tx=0x7f5e8000c8d0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.581+0000 7f5e8fc99640 1 -- 172.21.15.45:0/3463678639 shutdown_connections 2024-05-08T05:44:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.581+0000 7f5e8fc99640 1 --2- 172.21.15.45:0/3463678639 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5e740762d0 0x7f5e74078790 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.581+0000 7f5e8fc99640 1 --2- 172.21.15.45:0/3463678639 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5e880d54e0 0x7f5e881f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.581+0000 7f5e8fc99640 1 --2- 172.21.15.45:0/3463678639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5e8815f4e0 0x7f5e880b2220 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.581+0000 7f5e8fc99640 1 --2- 172.21.15.45:0/3463678639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5e880cfcd0 0x7f5e88158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.581+0000 7f5e8fc99640 1 -- 172.21.15.45:0/3463678639 >> 172.21.15.45:0/3463678639 conn(0x7f5e8808eac0 msgr2=0x7f5e880d3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.581+0000 7f5e8fc99640 1 -- 172.21.15.45:0/3463678639 shutdown_connections 2024-05-08T05:44:03.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.581+0000 7f5e8fc99640 1 -- 172.21.15.45:0/3463678639 wait complete. 2024-05-08T05:44:03.592 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:44:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.695+0000 7fd569b14640 1 Processor -- start 2024-05-08T05:44:03.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.695+0000 7fd569b14640 1 -- start start 2024-05-08T05:44:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.696+0000 7fd569b14640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd56415f450 0x7fd56406d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.696+0000 7fd569b14640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd56415f830 0x7fd56406d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.696+0000 7fd569b14640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5640d0cb0 con 0x7fd5640d0700 2024-05-08T05:44:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.696+0000 7fd569b14640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fd56406d690 con 0x7fd56415f830 2024-05-08T05:44:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.696+0000 7fd569b14640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd5640d0f90 con 0x7fd56415f450 2024-05-08T05:44:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.696+0000 7fd562ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd56415f830 0x7fd56406d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:03.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.696+0000 7fd562ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd56415f830 0x7fd56406d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36690/0 (socket says 172.21.15.45:36690) 2024-05-08T05:44:03.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.696+0000 7fd562ffd640 1 -- 172.21.15.45:0/2942405679 learned_addr learned my addr 172.21.15.45:0/2942405679 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:03.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.697+0000 7fd5637fe640 1 --2- 172.21.15.45:0/2942405679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd56415f450 0x7fd56406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:03.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.697+0000 7fd562ffd640 1 -- 172.21.15.45:0/2942405679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd56415f450 msgr2=0x7fd56406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:03.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.697+0000 7fd562ffd640 1 --2- 172.21.15.45:0/2942405679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd56415f450 0x7fd56406d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.697+0000 7fd562ffd640 1 -- 172.21.15.45:0/2942405679 >> v1:172.21.15.63:6789/0 conn(0x7fd5640d0700 legacy=0x7fd5640d17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:03.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.697+0000 7fd562ffd640 1 -- 172.21.15.45:0/2942405679 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5640d2fe0 con 0x7fd56415f830 2024-05-08T05:44:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.697+0000 7fd562ffd640 1 --2- 172.21.15.45:0/2942405679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56415f830 0x7fd56406d830 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd558009870 tx=0x7fd55802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3728b7ec47e3ebbe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.698+0000 7fd5627fc640 1 -- 172.21.15.45:0/2942405679 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd558037070 con 0x7fd56415f830 2024-05-08T05:44:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.698+0000 7fd5627fc640 1 -- 172.21.15.45:0/2942405679 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd558032440 con 0x7fd56415f830 2024-05-08T05:44:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.698+0000 7fd5627fc640 1 -- 172.21.15.45:0/2942405679 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd558032810 con 0x7fd56415f830 2024-05-08T05:44:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.698+0000 7fd569b14640 1 -- 172.21.15.45:0/2942405679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56415f830 msgr2=0x7fd56406d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.698+0000 7fd569b14640 1 --2- 172.21.15.45:0/2942405679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56415f830 0x7fd56406d830 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd558009870 tx=0x7fd55802a1c0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.698+0000 7fd569b14640 1 -- 172.21.15.45:0/2942405679 shutdown_connections 2024-05-08T05:44:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.698+0000 7fd569b14640 1 --2- 172.21.15.45:0/2942405679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd56415f830 0x7fd56406d830 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.698+0000 7fd569b14640 1 --2- 172.21.15.45:0/2942405679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd56415f450 0x7fd56406d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.698+0000 7fd569b14640 1 -- 172.21.15.45:0/2942405679 >> 172.21.15.45:0/2942405679 conn(0x7fd5640d5040 msgr2=0x7fd5640d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.698+0000 7fd569b14640 1 -- 172.21.15.45:0/2942405679 shutdown_connections 2024-05-08T05:44:03.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.698+0000 7fd569b14640 1 -- 172.21.15.45:0/2942405679 wait complete. 2024-05-08T05:44:03.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.699+0000 7fd569b14640 1 Processor -- start 2024-05-08T05:44:03.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.700+0000 7fd569b14640 1 -- start start 2024-05-08T05:44:03.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.700+0000 7fd569b14640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5640d0700 0x7fd56415d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:03.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.700+0000 7fd569b14640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd56415f450 0x7fd5641605d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:03.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.700+0000 7fd569b14640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd56415f830 0x7fd5640ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:03.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.700+0000 7fd569b14640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd5640d4c50 con 0x7fd5640d0700 2024-05-08T05:44:03.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.700+0000 7fd569b14640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd5640d4ad0 con 0x7fd56415f830 2024-05-08T05:44:03.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.700+0000 7fd569b14640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd5640d4dd0 con 0x7fd56415f450 2024-05-08T05:44:03.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.701+0000 7fd5637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5640d0700 0x7fd56415d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:03.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.701+0000 7fd5637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5640d0700 0x7fd56415d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36706/0 (socket says 172.21.15.45:36706) 2024-05-08T05:44:03.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.701+0000 7fd5637fe640 1 -- 172.21.15.45:0/1664752273 learned_addr learned my addr 172.21.15.45:0/1664752273 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:03.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.701+0000 7fd562ffd640 1 --2- 172.21.15.45:0/1664752273 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd56415f450 0x7fd5641605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:03.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.701+0000 7fd563fff640 1 --2- 172.21.15.45:0/1664752273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd56415f830 0x7fd5640ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:03.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.701+0000 7fd5637fe640 1 -- 172.21.15.45:0/1664752273 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd56415f450 msgr2=0x7fd5641605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:03.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.701+0000 7fd5637fe640 1 --2- 172.21.15.45:0/1664752273 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd56415f450 0x7fd5641605d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.701+0000 7fd5637fe640 1 -- 172.21.15.45:0/1664752273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd56415f830 msgr2=0x7fd5640ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.701+0000 7fd5637fe640 1 --2- 172.21.15.45:0/1664752273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd56415f830 0x7fd5640ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.701+0000 7fd5637fe640 1 -- 172.21.15.45:0/1664752273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5640c8b70 con 0x7fd5640d0700 2024-05-08T05:44:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.701+0000 7fd563fff640 1 --2- 172.21.15.45:0/1664752273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd56415f830 0x7fd5640ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.701+0000 7fd562ffd640 1 --2- 172.21.15.45:0/1664752273 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd56415f450 0x7fd5641605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.701+0000 7fd5637fe640 1 --2- 172.21.15.45:0/1664752273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5640d0700 0x7fd56415d5f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd55400c7f0 tx=0x7fd55400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.701+0000 7fd54bfff640 1 -- 172.21.15.45:0/1664752273 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd554015070 con 0x7fd5640d0700 2024-05-08T05:44:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.701+0000 7fd54bfff640 1 -- 172.21.15.45:0/1664752273 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd554004220 con 0x7fd5640d0700 2024-05-08T05:44:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.701+0000 7fd569b14640 1 -- 172.21.15.45:0/1664752273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5640c8e50 con 0x7fd5640d0700 2024-05-08T05:44:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.701+0000 7fd54bfff640 1 -- 172.21.15.45:0/1664752273 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd554007620 con 0x7fd5640d0700 2024-05-08T05:44:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.701+0000 7fd569b14640 1 -- 172.21.15.45:0/1664752273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5640c9410 con 0x7fd5640d0700 2024-05-08T05:44:03.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.702+0000 7fd569b14640 1 -- 172.21.15.45:0/1664752273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5640693f0 con 0x7fd5640d0700 2024-05-08T05:44:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.703+0000 7fd54bfff640 1 -- 172.21.15.45:0/1664752273 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd5540079d0 con 0x7fd5640d0700 2024-05-08T05:44:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.703+0000 7fd54bfff640 1 --2- 172.21.15.45:0/1664752273 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd5500762d0 0x7fd550078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.703+0000 7fd54bfff640 1 -- 172.21.15.45:0/1664752273 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7fd554097b00 con 0x7fd5640d0700 2024-05-08T05:44:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.704+0000 7fd562ffd640 1 --2- 172.21.15.45:0/1664752273 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd5500762d0 0x7fd550078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:03.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.704+0000 7fd562ffd640 1 --2- 172.21.15.45:0/1664752273 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd5500762d0 0x7fd550078790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd558034000 tx=0x7fd55802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:03.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.707+0000 7fd54bfff640 1 -- 172.21.15.45:0/1664752273 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd5540612d0 con 0x7fd5640d0700 2024-05-08T05:44:03.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.940+0000 7fd569b14640 1 -- 172.21.15.45:0/1664752273 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5640d0340 con 0x7fd5640d0700 2024-05-08T05:44:03.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.941+0000 7fd54bfff640 1 -- 172.21.15.45:0/1664752273 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5179 (secure 0 0 0) 0x7fd5540692c0 con 0x7fd5640d0700 2024-05-08T05:44:03.941 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:44:03.942 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/162765657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":162765657},{"type":"v1","addr":"172.21.15.63:6836","nonce":162765657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5356,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1788513331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1788513331},{"type":"v1","addr":"172.21.15.45:6834","nonce":1788513331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1173167340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1173167340},{"type":"v1","addr":"172.21.15.63:6837","nonce":1173167340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:43:58.107819+0000","modified":"2024-05-08T05:44:01.121276+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1461822016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1461822016},{"type":"v1","addr":"172.21.15.45:6835","nonce":1461822016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5697,"qdb_cluster":[5697]},"id":4}]} 2024-05-08T05:44:03.942 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 34 2024-05-08T05:44:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.943+0000 7fd569b14640 1 -- 172.21.15.45:0/1664752273 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd5500762d0 msgr2=0x7fd550078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.943+0000 7fd569b14640 1 --2- 172.21.15.45:0/1664752273 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd5500762d0 0x7fd550078790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd558034000 tx=0x7fd55802e000 comp rx=0 tx=0).stop 2024-05-08T05:44:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.943+0000 7fd569b14640 1 -- 172.21.15.45:0/1664752273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5640d0700 msgr2=0x7fd56415d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.943+0000 7fd569b14640 1 --2- 172.21.15.45:0/1664752273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5640d0700 0x7fd56415d5f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd55400c7f0 tx=0x7fd55400ccb0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.943+0000 7fd569b14640 1 -- 172.21.15.45:0/1664752273 shutdown_connections 2024-05-08T05:44:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.943+0000 7fd569b14640 1 --2- 172.21.15.45:0/1664752273 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd5500762d0 0x7fd550078790 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.943+0000 7fd569b14640 1 --2- 172.21.15.45:0/1664752273 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd56415f450 0x7fd5641605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.943+0000 7fd569b14640 1 --2- 172.21.15.45:0/1664752273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd56415f830 0x7fd5640ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.943+0000 7fd569b14640 1 --2- 172.21.15.45:0/1664752273 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd5640d0700 0x7fd56415d5f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.943+0000 7fd569b14640 1 -- 172.21.15.45:0/1664752273 >> 172.21.15.45:0/1664752273 conn(0x7fd5640d5040 msgr2=0x7fd5640b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.943+0000 7fd569b14640 1 -- 172.21.15.45:0/1664752273 shutdown_connections 2024-05-08T05:44:03.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:03.943+0000 7fd569b14640 1 -- 172.21.15.45:0/1664752273 wait complete. 2024-05-08T05:44:04.295 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:44:04.298 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:44:04.357 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:43:58.107819+0000', 'modified': '2024-05-08T05:44:01.121276+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5697}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5697': {'gid': 5697, 'name': 'a', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6835/1461822016', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 1461822016}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 1461822016}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5697, 'qdb_cluster': [5697]} 2024-05-08T05:44:04.357 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T05:44:04.357 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T05:44:04.376 INFO:tasks.ceph.mds.a.smithi045.stderr:2024-05-08T05:44:04.376+0000 7f3447f21640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:44:04.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.465+0000 7f5ac9514640 1 Processor -- start 2024-05-08T05:44:04.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.465+0000 7f5ac9514640 1 -- start start 2024-05-08T05:44:04.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.466+0000 7f5ac9514640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ac41629b0 0x7f5ac406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.466+0000 7f5ac9514640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5ac40b2490 0x7f5ac4070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.466+0000 7f5ac9514640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ac406e110 con 0x7f5ac4070d20 2024-05-08T05:44:04.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.466+0000 7f5ac9514640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5ac406dec0 con 0x7f5ac40b2490 2024-05-08T05:44:04.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.466+0000 7f5ac9514640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac406e3f0 con 0x7f5ac41629b0 2024-05-08T05:44:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.466+0000 7f5ac27fc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5ac40b2490 0x7f5ac4070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.466+0000 7f5ac2ffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ac41629b0 0x7f5ac406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.466+0000 7f5ac27fc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5ac40b2490 0x7f5ac4070330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:57264/0 (socket says 172.21.15.45:57264) 2024-05-08T05:44:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.466+0000 7f5ac37fe640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f5ac4070d20 0x7f5ac40afcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:36744/0 (socket says 172.21.15.45:36744) 2024-05-08T05:44:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.466+0000 7f5ac27fc640 1 -- 172.21.15.45:0/2368615687 learned_addr learned my addr 172.21.15.45:0/2368615687 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.467+0000 7f5ac1ffb640 1 -- 172.21.15.45:0/2368615687 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2584961603 0 0) 0x7f5ac406e110 con 0x7f5ac4070d20 2024-05-08T05:44:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.467+0000 7f5ac1ffb640 1 -- 172.21.15.45:0/2368615687 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5aac003520 con 0x7f5ac4070d20 2024-05-08T05:44:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.467+0000 7f5ac27fc640 1 -- 172.21.15.45:0/2368615687 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ac41629b0 msgr2=0x7f5ac406eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.467+0000 7f5ac27fc640 1 --2- 172.21.15.45:0/2368615687 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ac41629b0 0x7f5ac406eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.467+0000 7f5ac27fc640 1 -- 172.21.15.45:0/2368615687 >> v1:172.21.15.63:6789/0 conn(0x7f5ac4070d20 legacy=0x7f5ac40afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.467+0000 7f5ac27fc640 1 -- 172.21.15.45:0/2368615687 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac406e6d0 con 0x7f5ac40b2490 2024-05-08T05:44:04.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.467+0000 7f5ac2ffd640 1 --2- 172.21.15.45:0/2368615687 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ac41629b0 0x7f5ac406eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:44:04.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.468+0000 7f5ac27fc640 1 --2- 172.21.15.45:0/2368615687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ac40b2490 0x7f5ac4070330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009870 tx=0x7f5ab802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=739cc6aba4fceee8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:04.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.468+0000 7f5ac1ffb640 1 -- 172.21.15.45:0/2368615687 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8037070 con 0x7f5ac40b2490 2024-05-08T05:44:04.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.468+0000 7f5ac1ffb640 1 -- 172.21.15.45:0/2368615687 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab8032440 con 0x7f5ac40b2490 2024-05-08T05:44:04.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.468+0000 7f5ac1ffb640 1 -- 172.21.15.45:0/2368615687 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8032810 con 0x7f5ac40b2490 2024-05-08T05:44:04.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.469+0000 7f5ac9514640 1 -- 172.21.15.45:0/2368615687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ac40b2490 msgr2=0x7f5ac4070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:04.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.469+0000 7f5ac9514640 1 --2- 172.21.15.45:0/2368615687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ac40b2490 0x7f5ac4070330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009870 tx=0x7f5ab802a270 comp rx=0 tx=0).stop 2024-05-08T05:44:04.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.469+0000 7f5ac9514640 1 -- 172.21.15.45:0/2368615687 shutdown_connections 2024-05-08T05:44:04.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.469+0000 7f5ac9514640 1 --2- 172.21.15.45:0/2368615687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ac40b2490 0x7f5ac4070330 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.469+0000 7f5ac9514640 1 --2- 172.21.15.45:0/2368615687 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ac41629b0 0x7f5ac406eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.469+0000 7f5ac9514640 1 -- 172.21.15.45:0/2368615687 >> 172.21.15.45:0/2368615687 conn(0x7f5ac40b49d0 msgr2=0x7f5ac40b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:04.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.469+0000 7f5ac9514640 1 -- 172.21.15.45:0/2368615687 shutdown_connections 2024-05-08T05:44:04.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.469+0000 7f5ac9514640 1 -- 172.21.15.45:0/2368615687 wait complete. 2024-05-08T05:44:04.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.470+0000 7f5ac9514640 1 Processor -- start 2024-05-08T05:44:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.470+0000 7f5ac9514640 1 -- start start 2024-05-08T05:44:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.471+0000 7f5ac9514640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ac4070d20 0x7f5ac41f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.471+0000 7f5ac9514640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ac40b2490 0x7f5ac40cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.471+0000 7f5ac9514640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ac41629b0 0x7f5ac41f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.471+0000 7f5ac9514640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac4070870 con 0x7f5ac4070d20 2024-05-08T05:44:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.471+0000 7f5ac9514640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac406f410 con 0x7f5ac41629b0 2024-05-08T05:44:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.471+0000 7f5ac9514640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac40709f0 con 0x7f5ac40b2490 2024-05-08T05:44:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.471+0000 7f5ac2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ac4070d20 0x7f5ac41f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.471+0000 7f5ac2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ac4070d20 0x7f5ac41f7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36720/0 (socket says 172.21.15.45:36720) 2024-05-08T05:44:04.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.471+0000 7f5ac2ffd640 1 -- 172.21.15.45:0/3909782456 learned_addr learned my addr 172.21.15.45:0/3909782456 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.471+0000 7f5ac37fe640 1 --2- 172.21.15.45:0/3909782456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ac41629b0 0x7f5ac41f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.471+0000 7f5ac27fc640 1 --2- 172.21.15.45:0/3909782456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ac40b2490 0x7f5ac40cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.471+0000 7f5ac2ffd640 1 -- 172.21.15.45:0/3909782456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ac40b2490 msgr2=0x7f5ac40cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.471+0000 7f5ac2ffd640 1 --2- 172.21.15.45:0/3909782456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ac40b2490 0x7f5ac40cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.471+0000 7f5ac2ffd640 1 -- 172.21.15.45:0/3909782456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ac41629b0 msgr2=0x7f5ac41f4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.471+0000 7f5ac2ffd640 1 --2- 172.21.15.45:0/3909782456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ac41629b0 0x7f5ac41f4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.471+0000 7f5ac2ffd640 1 -- 172.21.15.45:0/3909782456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac40ca410 con 0x7f5ac4070d20 2024-05-08T05:44:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.472+0000 7f5ac2ffd640 1 --2- 172.21.15.45:0/3909782456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ac4070d20 0x7f5ac41f7a40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400e960 tx=0x7f5ab400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.472+0000 7f5aab7fe640 1 -- 172.21.15.45:0/3909782456 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4017070 con 0x7f5ac4070d20 2024-05-08T05:44:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.472+0000 7f5aab7fe640 1 -- 172.21.15.45:0/3909782456 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab400ca00 con 0x7f5ac4070d20 2024-05-08T05:44:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.472+0000 7f5ac9514640 1 -- 172.21.15.45:0/3909782456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac40c7990 con 0x7f5ac4070d20 2024-05-08T05:44:04.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.472+0000 7f5aab7fe640 1 -- 172.21.15.45:0/3909782456 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab400cdd0 con 0x7f5ac4070d20 2024-05-08T05:44:04.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.472+0000 7f5ac9514640 1 -- 172.21.15.45:0/3909782456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac40ca6f0 con 0x7f5ac4070d20 2024-05-08T05:44:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.474+0000 7f5aab7fe640 1 -- 172.21.15.45:0/3909782456 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5ab401a070 con 0x7f5ac4070d20 2024-05-08T05:44:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.474+0000 7f5aab7fe640 1 --2- 172.21.15.45:0/3909782456 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5ab0076300 0x7f5ab00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.474+0000 7f5aab7fe640 1 -- 172.21.15.45:0/3909782456 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f5ab40979c0 con 0x7f5ac4070d20 2024-05-08T05:44:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.474+0000 7f5ac27fc640 1 --2- 172.21.15.45:0/3909782456 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5ab0076300 0x7f5ab00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.475+0000 7f5ac27fc640 1 --2- 172.21.15.45:0/3909782456 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5ab0076300 0x7f5ab00787c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8034000 tx=0x7f5ab802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.475+0000 7f5acc22f740 1 Processor -- start 2024-05-08T05:44:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.475+0000 7f5acc22f740 1 -- start start 2024-05-08T05:44:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.476+0000 7f5acc22f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e634119f50 0x55e63411c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.476+0000 7f5acc22f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e63411a330 0x55e63411cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.476+0000 7f5acc22f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6340ecbb0 0x55e6340f03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.476+0000 7f5acc22f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e6340568d0 con 0x55e6340ecbb0 2024-05-08T05:44:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.476+0000 7f5acc22f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e63400fc10 con 0x55e63411a330 2024-05-08T05:44:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.476+0000 7f5acc22f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e634052890 con 0x55e634119f50 2024-05-08T05:44:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.476+0000 7f5ac37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6340ecbb0 0x55e6340f03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.476+0000 7f5ac37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6340ecbb0 0x55e6340f03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36726/0 (socket says 172.21.15.45:36726) 2024-05-08T05:44:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.476+0000 7f5ac27fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e63411a330 0x55e63411cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.476+0000 7f5ac37fe640 1 -- 172.21.15.45:0/2581493827 learned_addr learned my addr 172.21.15.45:0/2581493827 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.476+0000 7f5ac2ffd640 1 --2- 172.21.15.45:0/2581493827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e634119f50 0x55e63411c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.476+0000 7f5ac37fe640 1 -- 172.21.15.45:0/2581493827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e634119f50 msgr2=0x55e63411c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:04.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.476+0000 7f5ac37fe640 1 --2- 172.21.15.45:0/2581493827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e634119f50 0x55e63411c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.476+0000 7f5ac37fe640 1 -- 172.21.15.45:0/2581493827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e63411a330 msgr2=0x55e63411cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.476+0000 7f5ac37fe640 1 --2- 172.21.15.45:0/2581493827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e63411a330 0x55e63411cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.476+0000 7f5ac37fe640 1 -- 172.21.15.45:0/2581493827 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6340175d0 con 0x55e6340ecbb0 2024-05-08T05:44:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.476+0000 7f5ac37fe640 1 --2- 172.21.15.45:0/2581493827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6340ecbb0 0x55e6340f03c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5abc001c50 tx=0x7f5abc002110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20845b8aa11afa30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.477+0000 7f5aa8ff9640 1 -- 172.21.15.45:0/2581493827 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc0077f0 con 0x55e6340ecbb0 2024-05-08T05:44:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.477+0000 7f5aa8ff9640 1 -- 172.21.15.45:0/2581493827 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5abc018070 con 0x55e6340ecbb0 2024-05-08T05:44:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.477+0000 7f5aa8ff9640 1 -- 172.21.15.45:0/2581493827 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc0133f0 con 0x55e6340ecbb0 2024-05-08T05:44:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.477+0000 7f5acc22f740 1 -- 172.21.15.45:0/2581493827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6340ecbb0 msgr2=0x55e6340f03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.477+0000 7f5acc22f740 1 --2- 172.21.15.45:0/2581493827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6340ecbb0 0x55e6340f03c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5abc001c50 tx=0x7f5abc002110 comp rx=0 tx=0).stop 2024-05-08T05:44:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.477+0000 7f5acc22f740 1 -- 172.21.15.45:0/2581493827 shutdown_connections 2024-05-08T05:44:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.477+0000 7f5acc22f740 1 --2- 172.21.15.45:0/2581493827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6340ecbb0 0x55e6340f03c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.477+0000 7f5acc22f740 1 --2- 172.21.15.45:0/2581493827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e63411a330 0x55e63411cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.477+0000 7f5acc22f740 1 --2- 172.21.15.45:0/2581493827 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e634119f50 0x55e63411c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.477+0000 7f5acc22f740 1 -- 172.21.15.45:0/2581493827 >> 172.21.15.45:0/2581493827 conn(0x55e6340eee20 msgr2=0x55e634137760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:04.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.477+0000 7f5acc22f740 1 -- 172.21.15.45:0/2581493827 shutdown_connections 2024-05-08T05:44:04.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.477+0000 7f5acc22f740 1 -- 172.21.15.45:0/2581493827 wait complete. 2024-05-08T05:44:04.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.478+0000 7f5acc22f740 1 -- start start 2024-05-08T05:44:04.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.478+0000 7f5acc22f740 1 Processor -- start 2024-05-08T05:44:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.479+0000 7f5acc22f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634119f50 0x55e63414ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.479+0000 7f5acc22f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e63411a330 0x55e634113210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.479+0000 7f5acc22f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e634113750 0x55e63411fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.479+0000 7f5acc22f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e6340f1fd0 con 0x55e634119f50 2024-05-08T05:44:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.479+0000 7f5acc22f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e634013760 con 0x55e634113750 2024-05-08T05:44:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.479+0000 7f5acc22f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e6340f22b0 con 0x55e63411a330 2024-05-08T05:44:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.479+0000 7f5acc22f740 10 client.0 _ll_register_callbacks cb 0x55e63404ccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:44:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.479+0000 7f5ac2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634119f50 0x55e63414ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.479+0000 7f5ac2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634119f50 0x55e63414ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36730/0 (socket says 172.21.15.45:36730) 2024-05-08T05:44:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.479+0000 7f5ac2ffd640 1 -- 172.21.15.45:0/2826839791 learned_addr learned my addr 172.21.15.45:0/2826839791 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.479+0000 7f5ac37fe640 1 --2- 172.21.15.45:0/2826839791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e634113750 0x55e63411fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.479+0000 7f5ac27fc640 1 --2- 172.21.15.45:0/2826839791 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e63411a330 0x55e634113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.479+0000 7f5ac2ffd640 1 -- 172.21.15.45:0/2826839791 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e63411a330 msgr2=0x55e634113210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.479+0000 7f5ac2ffd640 1 --2- 172.21.15.45:0/2826839791 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e63411a330 0x55e634113210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.479+0000 7f5ac2ffd640 1 -- 172.21.15.45:0/2826839791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e634113750 msgr2=0x55e63411fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.479+0000 7f5ac2ffd640 1 --2- 172.21.15.45:0/2826839791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e634113750 0x55e63411fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.479+0000 7f5ac2ffd640 1 -- 172.21.15.45:0/2826839791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e633d682d0 con 0x55e634119f50 2024-05-08T05:44:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.479+0000 7f5ac27fc640 1 --2- 172.21.15.45:0/2826839791 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e63411a330 0x55e634113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.479+0000 7f5ac2ffd640 1 --2- 172.21.15.45:0/2826839791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634119f50 0x55e63414ba70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400c520 tx=0x7f5ab4069180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55226cef39b369a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.479+0000 7f5aa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:44:04.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.480+0000 7f5aa1ffb640 1 -- 172.21.15.45:0/2826839791 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab408ee60 con 0x55e634119f50 2024-05-08T05:44:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.480+0000 7f5aa1ffb640 1 -- 172.21.15.45:0/2826839791 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f5ab405ff00 con 0x55e634119f50 2024-05-08T05:44:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.480+0000 7f5acc22f740 1 -- 172.21.15.45:0/2826839791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5abc018070 con 0x55e634119f50 2024-05-08T05:44:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.480+0000 7f5aa1ffb640 1 -- 172.21.15.45:0/2826839791 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5ab4095050 con 0x55e634119f50 2024-05-08T05:44:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.480+0000 7f5acc22f740 10 client.5817 fetch_fsmap learned FSMap version 34 2024-05-08T05:44:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.480+0000 7f5acc22f740 1 -- 172.21.15.45:0/2826839791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55e633e326b0 con 0x55e634119f50 2024-05-08T05:44:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.480+0000 7f5aa1ffb640 1 -- 172.21.15.45:0/2826839791 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5abc018070 con 0x55e634119f50 2024-05-08T05:44:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.480+0000 7f5aa1ffb640 1 -- 172.21.15.45:0/2826839791 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f5ab405c410 con 0x55e634119f50 2024-05-08T05:44:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.480+0000 7f5acc22f740 10 client.5817 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T05:44:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.480+0000 7f5acc22f740 10 client.5817 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T05:44:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.480+0000 7f5acc22f740 20 client.5817 populate_metadata read hostname 'smithi045' 2024-05-08T05:44:04.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.480+0000 7f5acc22f740 1 --2- 172.21.15.45:0/2826839791 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55e63413a860 0x55e634147f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.480+0000 7f5acc22f740 4 client.5817 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:44:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.480+0000 7f5acc22f740 1 -- 172.21.15.45:0/2826839791 --> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e633fda760 con 0x55e63413a860 2024-05-08T05:44:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.480+0000 7f5ac27fc640 1 --2- 172.21.15.45:0/2826839791 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55e63413a860 0x55e634147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.480+0000 7f5aa1ffb640 10 client.5817 ms_handle_connect on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.480+0000 7f5ac27fc640 1 --2- 172.21.15.45:0/2826839791 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55e63413a860 0x55e634147f80 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4857b4224a88c8a1 server_cookie=2db7e9595392f291 in_seq=0 out_seq=0 2024-05-08T05:44:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.481+0000 7f5aa1ffb640 1 -- 172.21.15.45:0/2826839791 <== mds.0 v2:172.21.15.45:6833/1461822016 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e633fda760 con 0x55e63413a860 2024-05-08T05:44:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.481+0000 7f5aa1ffb640 10 client.5817 handle_command_reply: tid=0 2024-05-08T05:44:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.481+0000 7f5acc22f740 1 client.5817 shutdown 2024-05-08T05:44:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.481+0000 7f5acc22f740 1 -- 172.21.15.45:0/2826839791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634119f50 msgr2=0x55e63414ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.481+0000 7f5acc22f740 1 --2- 172.21.15.45:0/2826839791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634119f50 0x55e63414ba70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400c520 tx=0x7f5ab4069180 comp rx=0 tx=0).stop 2024-05-08T05:44:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.481+0000 7f5acc22f740 1 -- 172.21.15.45:0/2826839791 shutdown_connections 2024-05-08T05:44:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.481+0000 7f5acc22f740 1 --2- 172.21.15.45:0/2826839791 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55e63413a860 0x55e634147f80 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.481+0000 7f5acc22f740 1 --2- 172.21.15.45:0/2826839791 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e63411a330 0x55e634113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.481+0000 7f5acc22f740 1 --2- 172.21.15.45:0/2826839791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e634113750 0x55e63411fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.481+0000 7f5aa1ffb640 0 client.5817 ms_handle_reset on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.481+0000 7f5acc22f740 1 --2- 172.21.15.45:0/2826839791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634119f50 0x55e63414ba70 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.481+0000 7f5acc22f740 1 -- 172.21.15.45:0/2826839791 >> 172.21.15.45:0/2826839791 conn(0x55e6340ecbb0 msgr2=0x55e6341147c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.481+0000 7f5acc22f740 1 -- 172.21.15.45:0/2826839791 shutdown_connections 2024-05-08T05:44:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.481+0000 7f5acc22f740 1 -- 172.21.15.45:0/2826839791 wait complete. 2024-05-08T05:44:04.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.482+0000 7f5acc22f740 20 client.5817 trim_cache size 0 max 16384 2024-05-08T05:44:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.500+0000 7f5acc22f740 1 Processor -- start 2024-05-08T05:44:04.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.500+0000 7f5acc22f740 1 -- start start 2024-05-08T05:44:04.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.501+0000 7f5acc22f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634113750 0x55e6340f13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.501+0000 7f5acc22f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e634119f50 0x55e6340f1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.501+0000 7f5acc22f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e63411a330 0x55e634147f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.501+0000 7f5acc22f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac4200d20 con 0x55e634113750 2024-05-08T05:44:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.501+0000 7f5acc22f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac41f8d80 con 0x55e63411a330 2024-05-08T05:44:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.501+0000 7f5acc22f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac4200730 con 0x55e634119f50 2024-05-08T05:44:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.501+0000 7f5ac2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634113750 0x55e6340f13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.501+0000 7f5ac2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634113750 0x55e6340f13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36744/0 (socket says 172.21.15.45:36744) 2024-05-08T05:44:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.501+0000 7f5ac2ffd640 1 -- 172.21.15.45:0/1729753899 learned_addr learned my addr 172.21.15.45:0/1729753899 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.501+0000 7f5ac2ffd640 1 -- 172.21.15.45:0/1729753899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e634119f50 msgr2=0x55e6340f1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.501+0000 7f5ac2ffd640 1 --2- 172.21.15.45:0/1729753899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e634119f50 0x55e6340f1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.501+0000 7f5ac2ffd640 1 -- 172.21.15.45:0/1729753899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e63411a330 msgr2=0x55e634147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.501+0000 7f5ac2ffd640 1 --2- 172.21.15.45:0/1729753899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e63411a330 0x55e634147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.501+0000 7f5ac2ffd640 1 -- 172.21.15.45:0/1729753899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e634120550 con 0x55e634113750 2024-05-08T05:44:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.502+0000 7f5ac2ffd640 1 --2- 172.21.15.45:0/1729753899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634113750 0x55e6340f13d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400e930 tx=0x7f5ab4067050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df7daa0fe85f5ee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.502+0000 7f5aa3fff640 1 -- 172.21.15.45:0/1729753899 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4021070 con 0x55e634113750 2024-05-08T05:44:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.502+0000 7f5aa3fff640 1 -- 172.21.15.45:0/1729753899 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab40605b0 con 0x55e634113750 2024-05-08T05:44:04.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.502+0000 7f5aa3fff640 1 -- 172.21.15.45:0/1729753899 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab40697c0 con 0x55e634113750 2024-05-08T05:44:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.502+0000 7f5acc22f740 1 -- 172.21.15.45:0/1729753899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634113750 msgr2=0x55e6340f13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.502+0000 7f5acc22f740 1 --2- 172.21.15.45:0/1729753899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634113750 0x55e6340f13d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400e930 tx=0x7f5ab4067050 comp rx=0 tx=0).stop 2024-05-08T05:44:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.502+0000 7f5acc22f740 1 -- 172.21.15.45:0/1729753899 shutdown_connections 2024-05-08T05:44:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.502+0000 7f5acc22f740 1 --2- 172.21.15.45:0/1729753899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e634119f50 0x55e6340f1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.502+0000 7f5acc22f740 1 --2- 172.21.15.45:0/1729753899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e63411a330 0x55e634147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.502+0000 7f5acc22f740 1 --2- 172.21.15.45:0/1729753899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634113750 0x55e6340f13d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.502+0000 7f5acc22f740 1 -- 172.21.15.45:0/1729753899 >> 172.21.15.45:0/1729753899 conn(0x55e6340ecbb0 msgr2=0x55e634112940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.502+0000 7f5acc22f740 1 -- 172.21.15.45:0/1729753899 shutdown_connections 2024-05-08T05:44:04.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.502+0000 7f5acc22f740 1 -- 172.21.15.45:0/1729753899 wait complete. 2024-05-08T05:44:04.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.503+0000 7f5acc22f740 1 -- start start 2024-05-08T05:44:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.503+0000 7f5acc22f740 1 Processor -- start 2024-05-08T05:44:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5acc22f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634113750 0x55e6341b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5acc22f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e634119f50 0x55e63412db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5acc22f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e63411a330 0x55e63412d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5acc22f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e63413a6e0 con 0x55e634113750 2024-05-08T05:44:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5acc22f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e634114240 con 0x55e634119f50 2024-05-08T05:44:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5acc22f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e63413ad00 con 0x55e63411a330 2024-05-08T05:44:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5acc22f740 10 client.0 _ll_register_callbacks cb 0x7f5ab40608c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:44:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5ac2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634113750 0x55e6341b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5ac2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634113750 0x55e6341b60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36748/0 (socket says 172.21.15.45:36748) 2024-05-08T05:44:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5ac2ffd640 1 -- 172.21.15.45:0/3995074254 learned_addr learned my addr 172.21.15.45:0/3995074254 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5ac27fc640 1 --2- 172.21.15.45:0/3995074254 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e634119f50 0x55e63412db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5ac37fe640 1 --2- 172.21.15.45:0/3995074254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e63411a330 0x55e63412d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5ac2ffd640 1 -- 172.21.15.45:0/3995074254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e63411a330 msgr2=0x55e63412d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5ac2ffd640 1 --2- 172.21.15.45:0/3995074254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e63411a330 0x55e63412d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5ac2ffd640 1 -- 172.21.15.45:0/3995074254 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e634119f50 msgr2=0x55e63412db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5ac2ffd640 1 --2- 172.21.15.45:0/3995074254 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e634119f50 0x55e63412db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5ac2ffd640 1 -- 172.21.15.45:0/3995074254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e63412e090 con 0x55e634113750 2024-05-08T05:44:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5ac37fe640 1 --2- 172.21.15.45:0/3995074254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e63411a330 0x55e63412d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5ac2ffd640 1 --2- 172.21.15.45:0/3995074254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634113750 0x55e6341b60c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5ab405cb20 tx=0x7f5ab4095b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6a406301aaa8f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5aa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:44:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5aa1ffb640 1 -- 172.21.15.45:0/3995074254 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4021050 con 0x55e634113750 2024-05-08T05:44:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5aa1ffb640 1 -- 172.21.15.45:0/3995074254 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f5ab407f070 con 0x55e634113750 2024-05-08T05:44:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5acc22f740 1 -- 172.21.15.45:0/3995074254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab40605b0 con 0x55e634113750 2024-05-08T05:44:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5aa1ffb640 1 -- 172.21.15.45:0/3995074254 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5ab4064da0 con 0x55e634113750 2024-05-08T05:44:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5acc22f740 10 client.5823 fetch_fsmap learned FSMap version 34 2024-05-08T05:44:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5acc22f740 1 -- 172.21.15.45:0/3995074254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55e63413f780 con 0x55e634113750 2024-05-08T05:44:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.504+0000 7f5aa1ffb640 1 -- 172.21.15.45:0/3995074254 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab40605b0 con 0x55e634113750 2024-05-08T05:44:04.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5aa1ffb640 1 -- 172.21.15.45:0/3995074254 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f5ab405f980 con 0x55e634113750 2024-05-08T05:44:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5acc22f740 10 client.5823 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T05:44:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5acc22f740 10 client.5823 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T05:44:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5acc22f740 20 client.5823 populate_metadata read hostname 'smithi045' 2024-05-08T05:44:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5acc22f740 1 --2- 172.21.15.45:0/3995074254 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55e6340ed750 0x55e6340edb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5acc22f740 4 client.5823 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:44:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5acc22f740 1 -- 172.21.15.45:0/3995074254 --> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] -- command(tid 0: {"prefix": "status"}) -- 0x55e6340644e0 con 0x55e6340ed750 2024-05-08T05:44:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5ac27fc640 1 --2- 172.21.15.45:0/3995074254 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55e6340ed750 0x55e6340edb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5aa1ffb640 10 client.5823 ms_handle_connect on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5ac27fc640 1 --2- 172.21.15.45:0/3995074254 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55e6340ed750 0x55e6340edb30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bc578afd3d8ba14 server_cookie=fbe2275889857d96 in_seq=0 out_seq=0 2024-05-08T05:44:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5aa1ffb640 1 -- 172.21.15.45:0/3995074254 <== mds.0 v2:172.21.15.45:6833/1461822016 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55e6340644e0 con 0x55e6340ed750 2024-05-08T05:44:04.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5aa1ffb640 10 client.5823 handle_command_reply: tid=0 2024-05-08T05:44:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5acc22f740 1 client.5823 shutdown 2024-05-08T05:44:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5acc22f740 1 -- 172.21.15.45:0/3995074254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634113750 msgr2=0x55e6341b60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5acc22f740 1 --2- 172.21.15.45:0/3995074254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634113750 0x55e6341b60c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5ab405cb20 tx=0x7f5ab4095b90 comp rx=0 tx=0).stop 2024-05-08T05:44:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5acc22f740 1 -- 172.21.15.45:0/3995074254 shutdown_connections 2024-05-08T05:44:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5acc22f740 1 --2- 172.21.15.45:0/3995074254 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55e6340ed750 0x55e6340edb30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5acc22f740 1 --2- 172.21.15.45:0/3995074254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e63411a330 0x55e63412d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5aa1ffb640 0 client.5823 ms_handle_reset on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5acc22f740 1 --2- 172.21.15.45:0/3995074254 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e634119f50 0x55e63412db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5acc22f740 1 --2- 172.21.15.45:0/3995074254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e634113750 0x55e6341b60c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5acc22f740 1 -- 172.21.15.45:0/3995074254 >> 172.21.15.45:0/3995074254 conn(0x55e6340ecbb0 msgr2=0x55e634134520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5acc22f740 1 -- 172.21.15.45:0/3995074254 shutdown_connections 2024-05-08T05:44:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.505+0000 7f5acc22f740 1 -- 172.21.15.45:0/3995074254 wait complete. 2024-05-08T05:44:04.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.506+0000 7f5acc22f740 20 client.5823 trim_cache size 0 max 16384 2024-05-08T05:44:04.512 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:44:04.512 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "e9e1170a-fd27-4625-97a9-5d5f104d13a2", 2024-05-08T05:44:04.512 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-05-08T05:44:04.512 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 5697, 2024-05-08T05:44:04.512 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-05-08T05:44:04.512 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-05-08T05:44:04.512 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-05-08T05:44:04.512 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 5.7788718799999996, 2024-05-08T05:44:04.512 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 34, 2024-05-08T05:44:04.512 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 61, 2024-05-08T05:44:04.512 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 61, 2024-05-08T05:44:04.513 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 11.091884155000001 2024-05-08T05:44:04.513 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:44:04.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.506+0000 7f5ac9514640 1 -- 172.21.15.45:0/3909782456 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5ab0076300 msgr2=0x7f5ab00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:04.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.506+0000 7f5ac9514640 1 --2- 172.21.15.45:0/3909782456 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5ab0076300 0x7f5ab00787c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8034000 tx=0x7f5ab802e000 comp rx=0 tx=0).stop 2024-05-08T05:44:04.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.506+0000 7f5ac9514640 1 -- 172.21.15.45:0/3909782456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ac4070d20 msgr2=0x7f5ac41f7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:04.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.506+0000 7f5ac9514640 1 --2- 172.21.15.45:0/3909782456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ac4070d20 0x7f5ac41f7a40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400e960 tx=0x7f5ab400ee20 comp rx=0 tx=0).stop 2024-05-08T05:44:04.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.506+0000 7f5ac9514640 1 -- 172.21.15.45:0/3909782456 shutdown_connections 2024-05-08T05:44:04.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.506+0000 7f5ac9514640 1 --2- 172.21.15.45:0/3909782456 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5ab0076300 0x7f5ab00787c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.506+0000 7f5ac9514640 1 --2- 172.21.15.45:0/3909782456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ac40b2490 0x7f5ac40cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.506+0000 7f5ac9514640 1 --2- 172.21.15.45:0/3909782456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ac41629b0 0x7f5ac41f4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.506+0000 7f5ac9514640 1 --2- 172.21.15.45:0/3909782456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ac4070d20 0x7f5ac41f7a40 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.506+0000 7f5ac9514640 1 -- 172.21.15.45:0/3909782456 >> 172.21.15.45:0/3909782456 conn(0x7f5ac40b49d0 msgr2=0x7f5ac40d1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.506+0000 7f5ac9514640 1 -- 172.21.15.45:0/3909782456 shutdown_connections 2024-05-08T05:44:04.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.506+0000 7f5ac9514640 1 -- 172.21.15.45:0/3909782456 wait complete. 2024-05-08T05:44:04.517 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T05:44:04.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.621+0000 7fc1c9d9e640 1 Processor -- start 2024-05-08T05:44:04.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.621+0000 7fc1c9d9e640 1 -- start start 2024-05-08T05:44:04.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.622+0000 7fc1c9d9e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1c4148850 0x7fc1c41600a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.622+0000 7fc1c9d9e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1c406c880 con 0x7fc1c406ef70 2024-05-08T05:44:04.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.622+0000 7fc1c9d9e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1c406cb60 con 0x7fc1c406c3a0 2024-05-08T05:44:04.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.622+0000 7fc1c9d9e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc1c41605e0 con 0x7fc1c4148850 2024-05-08T05:44:04.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.622+0000 7fc1c8d9c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc1c4148850 0x7fc1c41600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.622+0000 7fc1c959d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc1c406ef70 0x7fc1c40b4b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53532/0 (socket says 172.21.15.45:53532) 2024-05-08T05:44:04.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.622+0000 7fc1c959d640 1 -- 172.21.15.45:0/3262365280 learned_addr learned my addr 172.21.15.45:0/3262365280 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:44:04.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.623+0000 7fc1c8d9c640 1 -- 172.21.15.45:0/3262365280 >> v1:172.21.15.63:6789/0 conn(0x7fc1c406c3a0 legacy=0x7fc1c4163300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:04.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.623+0000 7fc1c8d9c640 1 -- 172.21.15.45:0/3262365280 >> v1:172.21.15.45:6789/0 conn(0x7fc1c406ef70 legacy=0x7fc1c40b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:04.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.623+0000 7fc1c37fe640 1 -- 172.21.15.45:0/3262365280 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 15092561 0 0) 0x7fc1c406c880 con 0x7fc1c406ef70 2024-05-08T05:44:04.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.623+0000 7fc1c8d9c640 1 -- 172.21.15.45:0/3262365280 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c406ce40 con 0x7fc1c4148850 2024-05-08T05:44:04.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.623+0000 7fc1c8d9c640 1 --2- 172.21.15.45:0/3262365280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1c4148850 0x7fc1c41600a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80097a0 tx=0x7fc1b802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953aecb2ca2a695c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:04.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.623+0000 7fc1c37fe640 1 -- 172.21.15.45:0/3262365280 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8037070 con 0x7fc1c4148850 2024-05-08T05:44:04.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.623+0000 7fc1c37fe640 1 -- 172.21.15.45:0/3262365280 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b8032440 con 0x7fc1c4148850 2024-05-08T05:44:04.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.623+0000 7fc1c37fe640 1 -- 172.21.15.45:0/3262365280 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8032810 con 0x7fc1c4148850 2024-05-08T05:44:04.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.624+0000 7fc1c9d9e640 1 -- 172.21.15.45:0/3262365280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1c4148850 msgr2=0x7fc1c41600a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:04.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.624+0000 7fc1c9d9e640 1 --2- 172.21.15.45:0/3262365280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1c4148850 0x7fc1c41600a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80097a0 tx=0x7fc1b802a180 comp rx=0 tx=0).stop 2024-05-08T05:44:04.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.624+0000 7fc1c9d9e640 1 -- 172.21.15.45:0/3262365280 shutdown_connections 2024-05-08T05:44:04.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.624+0000 7fc1c9d9e640 1 --2- 172.21.15.45:0/3262365280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1c4148850 0x7fc1c41600a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.624+0000 7fc1c9d9e640 1 -- 172.21.15.45:0/3262365280 >> 172.21.15.45:0/3262365280 conn(0x7fc1c40b3890 msgr2=0x7fc1c40b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:04.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.624+0000 7fc1c9d9e640 1 -- 172.21.15.45:0/3262365280 shutdown_connections 2024-05-08T05:44:04.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.624+0000 7fc1c9d9e640 1 -- 172.21.15.45:0/3262365280 wait complete. 2024-05-08T05:44:04.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.625+0000 7fc1c9d9e640 1 Processor -- start 2024-05-08T05:44:04.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.625+0000 7fc1c9d9e640 1 -- start start 2024-05-08T05:44:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.626+0000 7fc1c9d9e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1c406c3a0 0x7fc1c41617d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.626+0000 7fc1c9d9e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1c406ef70 0x7fc1c4161d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.626+0000 7fc1c9d9e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1c4148850 0x7fc1c413eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.626+0000 7fc1c9d9e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c4148c30 con 0x7fc1c4148850 2024-05-08T05:44:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.626+0000 7fc1c9d9e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c40b70d0 con 0x7fc1c406ef70 2024-05-08T05:44:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.626+0000 7fc1c9d9e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c4148db0 con 0x7fc1c406c3a0 2024-05-08T05:44:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.626+0000 7fc1c959d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1c4148850 0x7fc1c413eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.626+0000 7fc1c959d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1c4148850 0x7fc1c413eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36766/0 (socket says 172.21.15.45:36766) 2024-05-08T05:44:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.626+0000 7fc1c959d640 1 -- 172.21.15.45:0/2269194876 learned_addr learned my addr 172.21.15.45:0/2269194876 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.627+0000 7fc1c3fff640 1 --2- 172.21.15.45:0/2269194876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1c406ef70 0x7fc1c4161d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.627+0000 7fc1c8d9c640 1 --2- 172.21.15.45:0/2269194876 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1c406c3a0 0x7fc1c41617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.627+0000 7fc1c959d640 1 -- 172.21.15.45:0/2269194876 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1c406c3a0 msgr2=0x7fc1c41617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.627+0000 7fc1c959d640 1 --2- 172.21.15.45:0/2269194876 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1c406c3a0 0x7fc1c41617d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.627+0000 7fc1c959d640 1 -- 172.21.15.45:0/2269194876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1c406ef70 msgr2=0x7fc1c4161d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.627+0000 7fc1c959d640 1 --2- 172.21.15.45:0/2269194876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1c406ef70 0x7fc1c4161d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.627+0000 7fc1c959d640 1 -- 172.21.15.45:0/2269194876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c40ae9e0 con 0x7fc1c4148850 2024-05-08T05:44:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.627+0000 7fc1c3fff640 1 --2- 172.21.15.45:0/2269194876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1c406ef70 0x7fc1c4161d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.627+0000 7fc1c8d9c640 1 --2- 172.21.15.45:0/2269194876 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1c406c3a0 0x7fc1c41617d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.627+0000 7fc1c959d640 1 --2- 172.21.15.45:0/2269194876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1c4148850 0x7fc1c413eda0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00c7a0 tx=0x7fc1bc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.627+0000 7fc1c17fa640 1 -- 172.21.15.45:0/2269194876 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc018070 con 0x7fc1c4148850 2024-05-08T05:44:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.627+0000 7fc1c17fa640 1 -- 172.21.15.45:0/2269194876 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1bc00ce50 con 0x7fc1c4148850 2024-05-08T05:44:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.627+0000 7fc1c17fa640 1 -- 172.21.15.45:0/2269194876 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc00a820 con 0x7fc1c4148850 2024-05-08T05:44:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.627+0000 7fc1c9d9e640 1 -- 172.21.15.45:0/2269194876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c40aecc0 con 0x7fc1c4148850 2024-05-08T05:44:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.627+0000 7fc1c9d9e640 1 -- 172.21.15.45:0/2269194876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c40af280 con 0x7fc1c4148850 2024-05-08T05:44:04.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.628+0000 7fc1c9d9e640 1 -- 172.21.15.45:0/2269194876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c4050740 con 0x7fc1c4148850 2024-05-08T05:44:04.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.629+0000 7fc1c17fa640 1 -- 172.21.15.45:0/2269194876 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc1bc01b070 con 0x7fc1c4148850 2024-05-08T05:44:04.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.629+0000 7fc1c17fa640 1 --2- 172.21.15.45:0/2269194876 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc1a8076300 0x7fc1a80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:04.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.629+0000 7fc1c17fa640 1 -- 172.21.15.45:0/2269194876 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7fc1bc096bf0 con 0x7fc1c4148850 2024-05-08T05:44:04.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.630+0000 7fc1c8d9c640 1 --2- 172.21.15.45:0/2269194876 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc1a8076300 0x7fc1a80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:04.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.630+0000 7fc1c8d9c640 1 --2- 172.21.15.45:0/2269194876 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc1a8076300 0x7fc1a80787c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8034000 tx=0x7fc1b802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:04.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.633+0000 7fc1c17fa640 1 -- 172.21.15.45:0/2269194876 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc1bc060450 con 0x7fc1c4148850 2024-05-08T05:44:04.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.864+0000 7fc1c9d9e640 1 -- 172.21.15.45:0/2269194876 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc1c40b45f0 con 0x7fc1c4148850 2024-05-08T05:44:04.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.864+0000 7fc1c17fa640 1 -- 172.21.15.45:0/2269194876 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) ==== 53+0+73 (secure 0 0 0) 0x7fc1bc05fd50 con 0x7fc1c4148850 2024-05-08T05:44:04.865 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T05:44:04.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.866+0000 7fc1c9d9e640 1 -- 172.21.15.45:0/2269194876 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc1a8076300 msgr2=0x7fc1a80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:04.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.866+0000 7fc1c9d9e640 1 --2- 172.21.15.45:0/2269194876 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc1a8076300 0x7fc1a80787c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8034000 tx=0x7fc1b802e000 comp rx=0 tx=0).stop 2024-05-08T05:44:04.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.866+0000 7fc1c9d9e640 1 -- 172.21.15.45:0/2269194876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1c4148850 msgr2=0x7fc1c413eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:04.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.866+0000 7fc1c9d9e640 1 --2- 172.21.15.45:0/2269194876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1c4148850 0x7fc1c413eda0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00c7a0 tx=0x7fc1bc00cc60 comp rx=0 tx=0).stop 2024-05-08T05:44:04.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.866+0000 7fc1c9d9e640 1 -- 172.21.15.45:0/2269194876 shutdown_connections 2024-05-08T05:44:04.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.866+0000 7fc1c9d9e640 1 --2- 172.21.15.45:0/2269194876 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc1a8076300 0x7fc1a80787c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.866+0000 7fc1c9d9e640 1 --2- 172.21.15.45:0/2269194876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc1c4148850 0x7fc1c413eda0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.866+0000 7fc1c9d9e640 1 --2- 172.21.15.45:0/2269194876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1c406ef70 0x7fc1c4161d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.866+0000 7fc1c9d9e640 1 --2- 172.21.15.45:0/2269194876 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1c406c3a0 0x7fc1c41617d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:04.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.866+0000 7fc1c9d9e640 1 -- 172.21.15.45:0/2269194876 >> 172.21.15.45:0/2269194876 conn(0x7fc1c40b3890 msgr2=0x7fc1c40b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:04.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.866+0000 7fc1c9d9e640 1 -- 172.21.15.45:0/2269194876 shutdown_connections 2024-05-08T05:44:04.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:04.866+0000 7fc1c9d9e640 1 -- 172.21.15.45:0/2269194876 wait complete. 2024-05-08T05:44:04.878 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T05:44:04.878 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:44:04.878 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-05-08T05:44:04.878 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T05:44:04.878 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:44:04.878 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T05:44:04.878 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T05:44:04.878 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:44:04.878 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:44:04.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:44:04.878 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-05-08T05:44:04.895 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:44:04.895 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:44:04.895 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:44:04.895 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:44:04.895 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-05-08T05:44:04.895 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:44:04.896 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:44:04.896 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:44:04.896 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-05-08T05:44:04.896 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:44:04.896 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:44:04.896 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-05-08T05:44:04.896 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:44:04.896 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-05-08T05:44:04.896 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-05-08T05:44:04.896 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:44:04.896 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42580sec preferred_lft 42580sec 2024-05-08T05:44:04.896 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-05-08T05:44:04.896 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:44:04.896 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:44:04.897 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-05-08T05:44:04.897 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-05-08T05:44:04.897 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:44:04.897 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T05:44:04.897 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T05:44:04.897 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:44:04.897 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c80a:aff:fe94:c77/64 scope link 2024-05-08T05:44:04.897 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:44:04.897 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:44:04.897 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 82:f3:eb:6a:57:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:44:04.897 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::80f3:ebff:fe6a:57ba/64 scope link 2024-05-08T05:44:04.897 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:44:04.897 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:44:04.897 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T05:44:04.898 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::14a2:64ff:feda:3a9c/64 scope link 2024-05-08T05:44:04.898 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:44:04.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:44:04.898 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:44:04.959 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:44:04.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:44:04.961 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-05-08T05:44:05.018 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-05-08T05:44:05.018 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:44:05.018 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:44:05.018 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:44:05.018 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:44:05.018 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:44:05.019 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:44:05.019 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:44:05.019 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:44:05.019 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:44:05.019 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:44:05.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:44:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:44:05.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:44:05.148 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:44:05.164 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T05:44:05.164 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T05:44:05.164 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:44:05.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:44:05.165 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-05-08T05:44:05.231 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:44:05.231 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T05:44:05.287 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:44:05.287 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:44:05.287 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:44:05.343 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T05:44:05.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:44:05.343 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:44:05.430 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:44:05.430 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:44:05.438 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:44:05.439 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:44:05.439 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-08T05:44:05.439 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@e9e1170a-fd27-4625-97a9-5d5f104d13a2.cephfs=/ 2024-05-08T05:44:05.439 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.63:3300/172.21.15.63:3301" will pass to kernel 2024-05-08T05:44:05.439 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T05:44:05.439 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:44:05.439 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:44:05.439 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:44:05.440 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:44:05.441 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:44:05.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:44:05.441 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:44:05.467 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:44:05.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:44:05.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:44:05.569 INFO:teuthology.orchestra.run.smithi045.stdout:client5177 2024-05-08T05:44:05.570 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:44:05.570 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:44:05.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:44:05.589 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-05-08T05:44:05.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:44:05.646 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-05-08T05:44:05.704 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:44:05.704 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:44:05.704 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:44:05.704 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:44:05.704 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-05-08T05:44:05.705 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:44:05.814 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627777 2024-05-08T05:44:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:44:05.823 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:44:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.967+0000 7f651acb4640 1 Processor -- start 2024-05-08T05:44:05.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.967+0000 7f651acb4640 1 -- start start 2024-05-08T05:44:05.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.967+0000 7f651acb4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f651415f420 0x7f65140d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:05.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f651acb4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65140d2080 0x7f65140d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:05.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f651acb4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65141630b0 con 0x7f65140d3ca0 2024-05-08T05:44:05.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f651acb4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f65140d2550 con 0x7f651415f420 2024-05-08T05:44:05.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f651acb4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f65140d2c80 con 0x7f65140d2080 2024-05-08T05:44:05.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f6513fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f65140d3ca0 0x7f65140d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53542/0 (socket says 172.21.15.45:53542) 2024-05-08T05:44:05.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f6513fff640 1 -- 172.21.15.45:0/2323822303 learned_addr learned my addr 172.21.15.45:0/2323822303 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:44:05.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f6519251640 1 --2- 172.21.15.45:0/2323822303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65140d2080 0x7f65140d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:05.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f6518a50640 1 --2- 172.21.15.45:0/2323822303 >> v2:172.21.15.63:3300/0 conn(0x7f651415f420 0x7f65140d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:05.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f65137fe640 1 -- 172.21.15.45:0/2323822303 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 836838163 0 0) 0x7f6500001110 con 0x7f65140d3ca0 2024-05-08T05:44:05.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f65137fe640 1 -- 172.21.15.45:0/2323822303 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6504003520 con 0x7f65140d3ca0 2024-05-08T05:44:05.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f65137fe640 1 -- 172.21.15.45:0/2323822303 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3325381818 0 0) 0x7f6504003520 con 0x7f65140d3ca0 2024-05-08T05:44:05.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f65137fe640 1 -- 172.21.15.45:0/2323822303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65140d2080 msgr2=0x7f65140d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:05.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f65137fe640 1 --2- 172.21.15.45:0/2323822303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65140d2080 0x7f65140d4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:05.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f65137fe640 1 -- 172.21.15.45:0/2323822303 >> v2:172.21.15.63:3300/0 conn(0x7f651415f420 msgr2=0x7f65140d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:05.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f65137fe640 1 --2- 172.21.15.45:0/2323822303 >> v2:172.21.15.63:3300/0 conn(0x7f651415f420 0x7f65140d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f65137fe640 1 -- 172.21.15.45:0/2323822303 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65140d2f60 con 0x7f65140d3ca0 2024-05-08T05:44:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f65137fe640 1 -- 172.21.15.45:0/2323822303 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f65000018c0 con 0x7f65140d3ca0 2024-05-08T05:44:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f65137fe640 1 -- 172.21.15.45:0/2323822303 >> v1:172.21.15.45:6789/0 conn(0x7f65140d3ca0 legacy=0x7f65140d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f65137fe640 1 --2- 172.21.15.45:0/2323822303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6504003a00 0x7f6504003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f65137fe640 1 --2- 172.21.15.45:0/2323822303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6504004800 0x7f6504004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f65137fe640 1 --2- 172.21.15.45:0/2323822303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65140d2080 0x7f65040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f65137fe640 1 -- 172.21.15.45:0/2323822303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f65140d2550 con 0x7f6504004800 2024-05-08T05:44:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f65137fe640 1 -- 172.21.15.45:0/2323822303 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65140d2c80 con 0x7f6504003a00 2024-05-08T05:44:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f65137fe640 1 -- 172.21.15.45:0/2323822303 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f650400b9c0 con 0x7f65140d2080 2024-05-08T05:44:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.968+0000 7f6513fff640 1 --2- 172.21.15.45:0/2323822303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6504004800 0x7f6504004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.969+0000 7f6518a50640 1 --2- 172.21.15.45:0/2323822303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6504003a00 0x7f6504003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.969+0000 7f6519251640 1 --2- 172.21.15.45:0/2323822303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65140d2080 0x7f65040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.969+0000 7f6513fff640 1 -- 172.21.15.45:0/2323822303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65140d2080 msgr2=0x7f65040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.969+0000 7f6513fff640 1 --2- 172.21.15.45:0/2323822303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65140d2080 0x7f65040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:05.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.969+0000 7f6513fff640 1 -- 172.21.15.45:0/2323822303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6504003a00 msgr2=0x7f6504003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.969+0000 7f6513fff640 1 --2- 172.21.15.45:0/2323822303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6504003a00 0x7f6504003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.969+0000 7f6513fff640 1 -- 172.21.15.45:0/2323822303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6504003520 con 0x7f6504004800 2024-05-08T05:44:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.969+0000 7f6518a50640 1 --2- 172.21.15.45:0/2323822303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6504003a00 0x7f6504003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:44:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.969+0000 7f6513fff640 1 --2- 172.21.15.45:0/2323822303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6504004800 0x7f6504004c80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f650000c4c0 tx=0x7f6504005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd60323c37bd8db server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.970+0000 7f65137fe640 1 -- 172.21.15.45:0/2323822303 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500035070 con 0x7f6504004800 2024-05-08T05:44:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.970+0000 7f65137fe640 1 -- 172.21.15.45:0/2323822303 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65000058d0 con 0x7f6504004800 2024-05-08T05:44:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.970+0000 7f65137fe640 1 -- 172.21.15.45:0/2323822303 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650000c0e0 con 0x7f6504004800 2024-05-08T05:44:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.970+0000 7f651acb4640 1 -- 172.21.15.45:0/2323822303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6504004800 msgr2=0x7f6504004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.970+0000 7f651acb4640 1 --2- 172.21.15.45:0/2323822303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6504004800 0x7f6504004c80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f650000c4c0 tx=0x7f6504005830 comp rx=0 tx=0).stop 2024-05-08T05:44:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.970+0000 7f6519251640 1 -- 172.21.15.45:0/2323822303 reap_dead start 2024-05-08T05:44:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.970+0000 7f651acb4640 1 -- 172.21.15.45:0/2323822303 shutdown_connections 2024-05-08T05:44:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.970+0000 7f651acb4640 1 -- 172.21.15.45:0/2323822303 >> 172.21.15.45:0/2323822303 conn(0x7f651408eac0 msgr2=0x7f651409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.970+0000 7f651acb4640 1 -- 172.21.15.45:0/2323822303 shutdown_connections 2024-05-08T05:44:05.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.970+0000 7f651acb4640 1 -- 172.21.15.45:0/2323822303 wait complete. 2024-05-08T05:44:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.971+0000 7f651acb4640 1 Processor -- start 2024-05-08T05:44:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.971+0000 7f651acb4640 1 -- start start 2024-05-08T05:44:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.972+0000 7f651acb4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6504004800 0x7f65140ced80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.972+0000 7f651acb4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65140b8f80 0x7f65140b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.972+0000 7f651acb4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65140b9400 0x7f65140b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.972+0000 7f651acb4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f65140d4cc0 con 0x7f6504004800 2024-05-08T05:44:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.972+0000 7f651acb4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65140d2830 con 0x7f65140b8f80 2024-05-08T05:44:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.972+0000 7f651acb4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f65140d4e40 con 0x7f65140b9400 2024-05-08T05:44:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.972+0000 7f6518a50640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6504004800 0x7f65140ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.972+0000 7f6518a50640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6504004800 0x7f65140ced80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36772/0 (socket says 172.21.15.45:36772) 2024-05-08T05:44:05.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.972+0000 7f6518a50640 1 -- 172.21.15.45:0/3635865641 learned_addr learned my addr 172.21.15.45:0/3635865641 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.973+0000 7f6513fff640 1 --2- 172.21.15.45:0/3635865641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65140b8f80 0x7f65140b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.973+0000 7f6519251640 1 --2- 172.21.15.45:0/3635865641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65140b9400 0x7f65140b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.973+0000 7f6518a50640 1 -- 172.21.15.45:0/3635865641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65140b9400 msgr2=0x7f65140b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.973+0000 7f6518a50640 1 --2- 172.21.15.45:0/3635865641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65140b9400 0x7f65140b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.973+0000 7f6518a50640 1 -- 172.21.15.45:0/3635865641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65140b8f80 msgr2=0x7f65140b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.973+0000 7f6518a50640 1 --2- 172.21.15.45:0/3635865641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65140b8f80 0x7f65140b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.973+0000 7f6518a50640 1 -- 172.21.15.45:0/3635865641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65140b9940 con 0x7f6504004800 2024-05-08T05:44:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.973+0000 7f6519251640 1 --2- 172.21.15.45:0/3635865641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65140b9400 0x7f65140b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.973+0000 7f6513fff640 1 --2- 172.21.15.45:0/3635865641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65140b8f80 0x7f65140b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.973+0000 7f6518a50640 1 --2- 172.21.15.45:0/3635865641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6504004800 0x7f65140ced80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6508011630 tx=0x7f6508011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.973+0000 7f65117fa640 1 -- 172.21.15.45:0/3635865641 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650800f920 con 0x7f6504004800 2024-05-08T05:44:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.973+0000 7f65117fa640 1 -- 172.21.15.45:0/3635865641 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f650800fac0 con 0x7f6504004800 2024-05-08T05:44:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.973+0000 7f651acb4640 1 -- 172.21.15.45:0/3635865641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65140b9c20 con 0x7f6504004800 2024-05-08T05:44:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.973+0000 7f65117fa640 1 -- 172.21.15.45:0/3635865641 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650800fe90 con 0x7f6504004800 2024-05-08T05:44:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.973+0000 7f651acb4640 1 -- 172.21.15.45:0/3635865641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65140ba1e0 con 0x7f6504004800 2024-05-08T05:44:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.974+0000 7f651acb4640 1 -- 172.21.15.45:0/3635865641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65140693f0 con 0x7f6504004800 2024-05-08T05:44:05.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.975+0000 7f65117fa640 1 -- 172.21.15.45:0/3635865641 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6508015460 con 0x7f6504004800 2024-05-08T05:44:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.975+0000 7f65117fa640 1 --2- 172.21.15.45:0/3635865641 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f64f8076470 0x7f64f8078930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.976+0000 7f65117fa640 1 -- 172.21.15.45:0/3635865641 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f65080980e0 con 0x7f6504004800 2024-05-08T05:44:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.976+0000 7f6513fff640 1 --2- 172.21.15.45:0/3635865641 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f64f8076470 0x7f64f8078930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:05.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.976+0000 7f6513fff640 1 --2- 172.21.15.45:0/3635865641 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f64f8076470 0x7f64f8078930 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f650000c490 tx=0x7f6500009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:05.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:05.980+0000 7f65117fa640 1 -- 172.21.15.45:0/3635865641 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6508019030 con 0x7f6504004800 2024-05-08T05:44:06.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.208+0000 7f651acb4640 1 -- 172.21.15.45:0/3635865641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65140b4c50 con 0x7f6504004800 2024-05-08T05:44:06.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.209+0000 7f65117fa640 1 -- 172.21.15.45:0/3635865641 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5179 (secure 0 0 0) 0x7f65080618b0 con 0x7f6504004800 2024-05-08T05:44:06.209 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:44:06.209 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/162765657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":162765657},{"type":"v1","addr":"172.21.15.63:6836","nonce":162765657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5356,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1788513331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1788513331},{"type":"v1","addr":"172.21.15.45:6834","nonce":1788513331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1173167340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1173167340},{"type":"v1","addr":"172.21.15.63:6837","nonce":1173167340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:43:58.107819+0000","modified":"2024-05-08T05:44:01.121276+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1461822016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1461822016},{"type":"v1","addr":"172.21.15.45:6835","nonce":1461822016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5697,"qdb_cluster":[5697]},"id":4}]} 2024-05-08T05:44:06.209 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 34 2024-05-08T05:44:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.211+0000 7f651acb4640 1 -- 172.21.15.45:0/3635865641 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f64f8076470 msgr2=0x7f64f8078930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.211+0000 7f651acb4640 1 --2- 172.21.15.45:0/3635865641 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f64f8076470 0x7f64f8078930 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f650000c490 tx=0x7f6500009030 comp rx=0 tx=0).stop 2024-05-08T05:44:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.211+0000 7f651acb4640 1 -- 172.21.15.45:0/3635865641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6504004800 msgr2=0x7f65140ced80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.211+0000 7f651acb4640 1 --2- 172.21.15.45:0/3635865641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6504004800 0x7f65140ced80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6508011630 tx=0x7f6508011af0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.211+0000 7f651acb4640 1 -- 172.21.15.45:0/3635865641 shutdown_connections 2024-05-08T05:44:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.211+0000 7f651acb4640 1 --2- 172.21.15.45:0/3635865641 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f64f8076470 0x7f64f8078930 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.211+0000 7f651acb4640 1 --2- 172.21.15.45:0/3635865641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65140b9400 0x7f65140b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.211+0000 7f651acb4640 1 --2- 172.21.15.45:0/3635865641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65140b8f80 0x7f65140b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.211+0000 7f651acb4640 1 --2- 172.21.15.45:0/3635865641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6504004800 0x7f65140ced80 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.211+0000 7f651acb4640 1 -- 172.21.15.45:0/3635865641 >> 172.21.15.45:0/3635865641 conn(0x7f651408eac0 msgr2=0x7f651406e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.211+0000 7f651acb4640 1 -- 172.21.15.45:0/3635865641 shutdown_connections 2024-05-08T05:44:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.211+0000 7f651acb4640 1 -- 172.21.15.45:0/3635865641 wait complete. 2024-05-08T05:44:06.222 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-08T05:44:06.350 INFO:teuthology.orchestra.run.smithi045.stdout:{"message":"","return_code":0} 2024-05-08T05:44:06.350 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-08T05:44:06.350 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-05-08T05:44:06.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.395+0000 7fca4e622f40 1 Processor -- start 2024-05-08T05:44:06.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.395+0000 7fca4e622f40 1 -- start start 2024-05-08T05:44:06.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.396+0000 7fca4e622f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca380022f0 0x557df92dad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.396+0000 7fca4e622f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557df9226540 0x557df9264590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.396+0000 7fca4e622f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557df9264ad0 0x557df92de480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.396+0000 7fca4e622f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557df91cbf50 con 0x557df9226540 2024-05-08T05:44:06.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.396+0000 7fca4e622f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557df92facc0 con 0x557df9264ad0 2024-05-08T05:44:06.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.396+0000 7fca4e622f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x557df92fc190 con 0x7fca380022f0 2024-05-08T05:44:06.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.396+0000 7fca43fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557df9226540 0x557df9264590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.396+0000 7fca43fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557df9226540 0x557df9264590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36790/0 (socket says 172.21.15.45:36790) 2024-05-08T05:44:06.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.396+0000 7fca43fff640 1 -- 172.21.15.45:0/1817793729 learned_addr learned my addr 172.21.15.45:0/1817793729 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:06.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.396+0000 7fca4c11c640 1 --2- 172.21.15.45:0/1817793729 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca380022f0 0x557df92dad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.396+0000 7fca4c91d640 1 --2- 172.21.15.45:0/1817793729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557df9264ad0 0x557df92de480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.396+0000 7fca43fff640 1 -- 172.21.15.45:0/1817793729 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca380022f0 msgr2=0x557df92dad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.396+0000 7fca43fff640 1 --2- 172.21.15.45:0/1817793729 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca380022f0 0x557df92dad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.396+0000 7fca43fff640 1 -- 172.21.15.45:0/1817793729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557df9264ad0 msgr2=0x557df92de480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.396+0000 7fca43fff640 1 --2- 172.21.15.45:0/1817793729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557df9264ad0 0x557df92de480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.396+0000 7fca43fff640 1 -- 172.21.15.45:0/1817793729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557df92fc430 con 0x557df9226540 2024-05-08T05:44:06.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.396+0000 7fca43fff640 1 --2- 172.21.15.45:0/1817793729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557df9226540 0x557df9264590 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fca3400d6f0 tx=0x7fca3400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ceb467aa126998 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.396+0000 7fca4b11a640 1 -- 172.21.15.45:0/1817793729 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca34013070 con 0x557df9226540 2024-05-08T05:44:06.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.396+0000 7fca4b11a640 1 -- 172.21.15.45:0/1817793729 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca3400b8d0 con 0x557df9226540 2024-05-08T05:44:06.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.396+0000 7fca4b11a640 1 -- 172.21.15.45:0/1817793729 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3400bcc0 con 0x557df9226540 2024-05-08T05:44:06.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.397+0000 7fca4e622f40 1 -- 172.21.15.45:0/1817793729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557df9226540 msgr2=0x557df9264590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.397+0000 7fca4e622f40 1 --2- 172.21.15.45:0/1817793729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557df9226540 0x557df9264590 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fca3400d6f0 tx=0x7fca3400dbb0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.397+0000 7fca4e622f40 1 -- 172.21.15.45:0/1817793729 shutdown_connections 2024-05-08T05:44:06.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.397+0000 7fca4e622f40 1 --2- 172.21.15.45:0/1817793729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557df9264ad0 0x557df92de480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.397+0000 7fca4e622f40 1 --2- 172.21.15.45:0/1817793729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557df9226540 0x557df9264590 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.397+0000 7fca4e622f40 1 --2- 172.21.15.45:0/1817793729 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fca380022f0 0x557df92dad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.397+0000 7fca4e622f40 1 -- 172.21.15.45:0/1817793729 >> 172.21.15.45:0/1817793729 conn(0x557df92237e0 msgr2=0x557df92fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:06.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.397+0000 7fca4e622f40 1 -- 172.21.15.45:0/1817793729 shutdown_connections 2024-05-08T05:44:06.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.397+0000 7fca4e622f40 1 -- 172.21.15.45:0/1817793729 wait complete. 2024-05-08T05:44:06.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.398+0000 7fca4e622f40 1 Processor -- start 2024-05-08T05:44:06.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.398+0000 7fca4e622f40 1 -- start start 2024-05-08T05:44:06.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.398+0000 7fca4e622f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557df9226540 0x557df92d9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.398+0000 7fca4e622f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557df9264ad0 0x557df9266940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.398+0000 7fca4e622f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca380022f0 0x557df9268170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.398+0000 7fca4e622f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557df9252700 con 0x557df9226540 2024-05-08T05:44:06.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.398+0000 7fca4e622f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557df92f3490 con 0x7fca380022f0 2024-05-08T05:44:06.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.398+0000 7fca4e622f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x557df92df240 con 0x557df9264ad0 2024-05-08T05:44:06.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.398+0000 7fca4c11c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557df9226540 0x557df92d9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.398+0000 7fca4c11c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557df9226540 0x557df92d9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36802/0 (socket says 172.21.15.45:36802) 2024-05-08T05:44:06.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.398+0000 7fca4c11c640 1 -- 172.21.15.45:0/2402950294 learned_addr learned my addr 172.21.15.45:0/2402950294 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:06.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.399+0000 7fca43fff640 1 --2- 172.21.15.45:0/2402950294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557df9264ad0 0x557df9266940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.399+0000 7fca4c91d640 1 --2- 172.21.15.45:0/2402950294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca380022f0 0x557df9268170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.399+0000 7fca4c11c640 1 -- 172.21.15.45:0/2402950294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557df9264ad0 msgr2=0x557df9266940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.399+0000 7fca4c11c640 1 --2- 172.21.15.45:0/2402950294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557df9264ad0 0x557df9266940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.399+0000 7fca4c11c640 1 -- 172.21.15.45:0/2402950294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca380022f0 msgr2=0x557df9268170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.399+0000 7fca4c11c640 1 --2- 172.21.15.45:0/2402950294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca380022f0 0x557df9268170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.399+0000 7fca4c11c640 1 -- 172.21.15.45:0/2402950294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557df92dbbd0 con 0x557df9226540 2024-05-08T05:44:06.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.399+0000 7fca43fff640 1 --2- 172.21.15.45:0/2402950294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557df9264ad0 0x557df9266940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:44:06.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.399+0000 7fca4c91d640 1 --2- 172.21.15.45:0/2402950294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca380022f0 0x557df9268170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:44:06.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.399+0000 7fca4c11c640 1 --2- 172.21.15.45:0/2402950294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557df9226540 0x557df92d9ee0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fca3c046160 tx=0x7fca3c003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.399+0000 7fca4a118640 1 -- 172.21.15.45:0/2402950294 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c068070 con 0x557df9226540 2024-05-08T05:44:06.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.399+0000 7fca4a118640 1 -- 172.21.15.45:0/2402950294 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca3c04f860 con 0x557df9226540 2024-05-08T05:44:06.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.399+0000 7fca4e622f40 1 -- 172.21.15.45:0/2402950294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557df9252d30 con 0x557df9226540 2024-05-08T05:44:06.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.399+0000 7fca4a118640 1 -- 172.21.15.45:0/2402950294 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c04fc50 con 0x557df9226540 2024-05-08T05:44:06.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.399+0000 7fca4e622f40 1 -- 172.21.15.45:0/2402950294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557df9266e80 con 0x557df9226540 2024-05-08T05:44:06.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.401+0000 7fca4a118640 1 -- 172.21.15.45:0/2402950294 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fca3c058400 con 0x557df9226540 2024-05-08T05:44:06.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.401+0000 7fca4a118640 1 --2- 172.21.15.45:0/2402950294 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fca2c074cb0 0x7fca2c077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.401+0000 7fca4a118640 1 -- 172.21.15.45:0/2402950294 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7fca3c0d9a80 con 0x557df9226540 2024-05-08T05:44:06.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.401+0000 7fca4e622f40 1 --2- 172.21.15.45:0/2402950294 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x557df92dad60 0x557df92686b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.401+0000 7fca4e622f40 1 -- 172.21.15.45:0/2402950294 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61) -- 0x557df926b530 con 0x557df92dad60 2024-05-08T05:44:06.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.401+0000 7fca43fff640 1 --2- 172.21.15.45:0/2402950294 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fca2c074cb0 0x7fca2c077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.402+0000 7fca4c91d640 1 --2- 172.21.15.45:0/2402950294 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x557df92dad60 0x557df92686b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.402+0000 7fca4c91d640 1 --2- 172.21.15.45:0/2402950294 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x557df92dad60 0x557df92686b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.402+0000 7fca43fff640 1 --2- 172.21.15.45:0/2402950294 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fca2c074cb0 0x7fca2c077170 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x557df92f3bd0 tx=0x7fca3401b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.402+0000 7fca4c91d640 1 -- 172.21.15.45:0/2402950294 <== osd.6 v2:172.21.15.63:6800/2926759052 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) ==== 164+0+94 (crc 0 0 0) 0x7fca440117f0 con 0x557df92dad60 2024-05-08T05:44:06.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.402+0000 7fca4e622f40 1 -- 172.21.15.45:0/2402950294 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x557df92dad60 msgr2=0x557df92686b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.402+0000 7fca4e622f40 1 --2- 172.21.15.45:0/2402950294 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x557df92dad60 0x557df92686b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.402+0000 7fca4e622f40 1 -- 172.21.15.45:0/2402950294 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fca2c074cb0 msgr2=0x7fca2c077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.402+0000 7fca4e622f40 1 --2- 172.21.15.45:0/2402950294 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fca2c074cb0 0x7fca2c077170 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x557df92f3bd0 tx=0x7fca3401b040 comp rx=0 tx=0).stop 2024-05-08T05:44:06.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.402+0000 7fca4e622f40 1 -- 172.21.15.45:0/2402950294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557df9226540 msgr2=0x557df92d9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.402+0000 7fca4e622f40 1 --2- 172.21.15.45:0/2402950294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557df9226540 0x557df92d9ee0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fca3c046160 tx=0x7fca3c003020 comp rx=0 tx=0).stop 2024-05-08T05:44:06.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.402+0000 7fca4c91d640 1 -- 172.21.15.45:0/2402950294 reap_dead start 2024-05-08T05:44:06.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.402+0000 7fca4e622f40 1 -- 172.21.15.45:0/2402950294 shutdown_connections 2024-05-08T05:44:06.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.402+0000 7fca4e622f40 1 -- 172.21.15.45:0/2402950294 >> 172.21.15.45:0/2402950294 conn(0x557df92237e0 msgr2=0x557df92fb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:06.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.402+0000 7fca4e622f40 1 -- 172.21.15.45:0/2402950294 shutdown_connections 2024-05-08T05:44:06.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.402+0000 7fca4e622f40 1 -- 172.21.15.45:0/2402950294 wait complete. 2024-05-08T05:44:06.407 INFO:teuthology.orchestra.run.smithi045.stdout:Xfiledir_test 2024-05-08T05:44:06.407 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:44:06.407 DEBUG:teuthology.orchestra.run.smithi045:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-08T05:44:06.455 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:44:06.455 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 1099511627777, 2024-05-08T05:44:06.455 INFO:teuthology.orchestra.run.smithi045.stdout: "ancestors": [ 2024-05-08T05:44:06.455 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:44:06.455 INFO:teuthology.orchestra.run.smithi045.stdout: "dirino": 1099511627776, 2024-05-08T05:44:06.455 INFO:teuthology.orchestra.run.smithi045.stdout: "dname": "file", 2024-05-08T05:44:06.455 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 2 2024-05-08T05:44:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:44:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:44:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: "dirino": 1, 2024-05-08T05:44:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: "dname": "dir_test", 2024-05-08T05:44:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 4 2024-05-08T05:44:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:44:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-05-08T05:44:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: "pool": 10, 2024-05-08T05:44:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: "old_pools": [] 2024-05-08T05:44:06.456 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:44:06.456 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:44:06.457 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-05-08T05:44:06.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.544+0000 7f247ed5bf40 1 Processor -- start 2024-05-08T05:44:06.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.544+0000 7f247ed5bf40 1 -- start start 2024-05-08T05:44:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.544+0000 7f247ed5bf40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2464003ab0 0x55f8ac204920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.544+0000 7f247ed5bf40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ac15a8d0 0x55f8ac153f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.545+0000 7f247ed5bf40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ac1f8240 0x55f8ac1fb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.545+0000 7f247ed5bf40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f8ac0d1f50 con 0x7f2464003ab0 2024-05-08T05:44:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.545+0000 7f247ed5bf40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f8ac0d5f90 con 0x55f8ac15a8d0 2024-05-08T05:44:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.545+0000 7f247ed5bf40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f8ac206160 con 0x55f8ac1f8240 2024-05-08T05:44:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.545+0000 7f247c855640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2464003ab0 0x55f8ac204920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.545+0000 7f247c855640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2464003ab0 0x55f8ac204920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36824/0 (socket says 172.21.15.45:36824) 2024-05-08T05:44:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.545+0000 7f247c855640 1 -- 172.21.15.45:0/671547854 learned_addr learned my addr 172.21.15.45:0/671547854 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.545+0000 7f247c054640 1 --2- 172.21.15.45:0/671547854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ac15a8d0 0x55f8ac153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.545+0000 7f247d056640 1 --2- 172.21.15.45:0/671547854 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ac1f8240 0x55f8ac1fb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.545+0000 7f247c855640 1 -- 172.21.15.45:0/671547854 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ac1f8240 msgr2=0x55f8ac1fb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.545+0000 7f247c855640 1 --2- 172.21.15.45:0/671547854 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ac1f8240 0x55f8ac1fb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.545+0000 7f247c855640 1 -- 172.21.15.45:0/671547854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ac15a8d0 msgr2=0x55f8ac153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.545+0000 7f247c855640 1 --2- 172.21.15.45:0/671547854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ac15a8d0 0x55f8ac153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.545+0000 7f247c855640 1 -- 172.21.15.45:0/671547854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8ac204e60 con 0x7f2464003ab0 2024-05-08T05:44:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.545+0000 7f247d056640 1 --2- 172.21.15.45:0/671547854 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ac1f8240 0x55f8ac1fb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.545+0000 7f247c054640 1 --2- 172.21.15.45:0/671547854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ac15a8d0 0x55f8ac153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.545+0000 7f247c855640 1 --2- 172.21.15.45:0/671547854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2464003ab0 0x55f8ac204920 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f246c006e10 tx=0x7f246c0029f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14109ba5cc734638 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.545+0000 7f247b052640 1 -- 172.21.15.45:0/671547854 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c018070 con 0x7f2464003ab0 2024-05-08T05:44:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.545+0000 7f247b052640 1 -- 172.21.15.45:0/671547854 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f246c00f510 con 0x7f2464003ab0 2024-05-08T05:44:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.545+0000 7f247b052640 1 -- 172.21.15.45:0/671547854 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c00f8c0 con 0x7f2464003ab0 2024-05-08T05:44:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.546+0000 7f247ed5bf40 1 -- 172.21.15.45:0/671547854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2464003ab0 msgr2=0x55f8ac204920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.546+0000 7f247ed5bf40 1 --2- 172.21.15.45:0/671547854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2464003ab0 0x55f8ac204920 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f246c006e10 tx=0x7f246c0029f0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.546+0000 7f247ed5bf40 1 -- 172.21.15.45:0/671547854 shutdown_connections 2024-05-08T05:44:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.546+0000 7f247ed5bf40 1 --2- 172.21.15.45:0/671547854 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ac1f8240 0x55f8ac1fb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.546+0000 7f247ed5bf40 1 --2- 172.21.15.45:0/671547854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f8ac15a8d0 0x55f8ac153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.546+0000 7f247ed5bf40 1 --2- 172.21.15.45:0/671547854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2464003ab0 0x55f8ac204920 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.546+0000 7f247ed5bf40 1 -- 172.21.15.45:0/671547854 >> 172.21.15.45:0/671547854 conn(0x55f8ac12a070 msgr2=0x55f8ac173250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.546+0000 7f247ed5bf40 1 -- 172.21.15.45:0/671547854 shutdown_connections 2024-05-08T05:44:06.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.546+0000 7f247ed5bf40 1 -- 172.21.15.45:0/671547854 wait complete. 2024-05-08T05:44:06.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.546+0000 7f247ed5bf40 1 Processor -- start 2024-05-08T05:44:06.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.546+0000 7f247ed5bf40 1 -- start start 2024-05-08T05:44:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.547+0000 7f247ed5bf40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8ac15a8d0 0x55f8ac1e13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.547+0000 7f247ed5bf40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ac1f8240 0x55f8ac173250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.547+0000 7f247ed5bf40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2464003ab0 0x55f8ac173790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.547+0000 7f247ed5bf40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f8ac1f8790 con 0x55f8ac15a8d0 2024-05-08T05:44:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.547+0000 7f247ed5bf40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f8ac154470 con 0x7f2464003ab0 2024-05-08T05:44:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.547+0000 7f247ed5bf40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f8ac1564c0 con 0x55f8ac1f8240 2024-05-08T05:44:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.547+0000 7f247c855640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8ac15a8d0 0x55f8ac1e13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.547+0000 7f247c855640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8ac15a8d0 0x55f8ac1e13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36836/0 (socket says 172.21.15.45:36836) 2024-05-08T05:44:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.547+0000 7f247c855640 1 -- 172.21.15.45:0/788876061 learned_addr learned my addr 172.21.15.45:0/788876061 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.547+0000 7f247c054640 1 --2- 172.21.15.45:0/788876061 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ac1f8240 0x55f8ac173250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.547+0000 7f247d056640 1 --2- 172.21.15.45:0/788876061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2464003ab0 0x55f8ac173790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.547+0000 7f247c855640 1 -- 172.21.15.45:0/788876061 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ac1f8240 msgr2=0x55f8ac173250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.547+0000 7f247c855640 1 --2- 172.21.15.45:0/788876061 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f8ac1f8240 0x55f8ac173250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.547+0000 7f247c855640 1 -- 172.21.15.45:0/788876061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2464003ab0 msgr2=0x55f8ac173790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.547+0000 7f247c855640 1 --2- 172.21.15.45:0/788876061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2464003ab0 0x55f8ac173790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.547+0000 7f247c855640 1 -- 172.21.15.45:0/788876061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8ac155eb0 con 0x55f8ac15a8d0 2024-05-08T05:44:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.548+0000 7f247c855640 1 --2- 172.21.15.45:0/788876061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8ac15a8d0 0x55f8ac1e13d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f246c002a30 tx=0x7f246c001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.548+0000 7f247a050640 1 -- 172.21.15.45:0/788876061 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c018070 con 0x55f8ac15a8d0 2024-05-08T05:44:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.548+0000 7f247a050640 1 -- 172.21.15.45:0/788876061 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f246c023070 con 0x55f8ac15a8d0 2024-05-08T05:44:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.548+0000 7f247ed5bf40 1 -- 172.21.15.45:0/788876061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f8ac155bd0 con 0x55f8ac15a8d0 2024-05-08T05:44:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.548+0000 7f247a050640 1 -- 172.21.15.45:0/788876061 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c0136c0 con 0x55f8ac15a8d0 2024-05-08T05:44:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.548+0000 7f247ed5bf40 1 -- 172.21.15.45:0/788876061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8ac170da0 con 0x55f8ac15a8d0 2024-05-08T05:44:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.549+0000 7f247a050640 1 -- 172.21.15.45:0/788876061 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f246c013860 con 0x55f8ac15a8d0 2024-05-08T05:44:06.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.550+0000 7f247a050640 1 --2- 172.21.15.45:0/788876061 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2468074ce0 0x7f24680771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.550+0000 7f247a050640 1 -- 172.21.15.45:0/788876061 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f246c096ab0 con 0x55f8ac15a8d0 2024-05-08T05:44:06.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.550+0000 7f247c054640 1 --2- 172.21.15.45:0/788876061 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2468074ce0 0x7f24680771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.550+0000 7f247ed5bf40 1 --2- 172.21.15.45:0/788876061 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55f8ac290680 0x55f8ac16ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.550+0000 7f247ed5bf40 1 -- 172.21.15.45:0/788876061 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e61) -- 0x55f8ac16ef80 con 0x55f8ac290680 2024-05-08T05:44:06.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.550+0000 7f247d056640 1 --2- 172.21.15.45:0/788876061 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55f8ac290680 0x55f8ac16ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.551+0000 7f247c054640 1 --2- 172.21.15.45:0/788876061 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2468074ce0 0x7f24680771a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55f8ac205c20 tx=0x7f247000c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.551+0000 7f247d056640 1 --2- 172.21.15.45:0/788876061 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55f8ac290680 0x55f8ac16ea40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.552+0000 7f247d056640 1 -- 172.21.15.45:0/788876061 <== osd.6 v2:172.21.15.63:6800/2926759052 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v61'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f24740014f0 con 0x55f8ac290680 2024-05-08T05:44:06.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.552+0000 7f247ed5bf40 1 -- 172.21.15.45:0/788876061 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55f8ac290680 msgr2=0x55f8ac16ea40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.552+0000 7f247ed5bf40 1 --2- 172.21.15.45:0/788876061 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55f8ac290680 0x55f8ac16ea40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.552+0000 7f247ed5bf40 1 -- 172.21.15.45:0/788876061 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2468074ce0 msgr2=0x7f24680771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.552+0000 7f247ed5bf40 1 --2- 172.21.15.45:0/788876061 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2468074ce0 0x7f24680771a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55f8ac205c20 tx=0x7f247000c040 comp rx=0 tx=0).stop 2024-05-08T05:44:06.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.552+0000 7f247ed5bf40 1 -- 172.21.15.45:0/788876061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8ac15a8d0 msgr2=0x55f8ac1e13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.552+0000 7f247ed5bf40 1 --2- 172.21.15.45:0/788876061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f8ac15a8d0 0x55f8ac1e13d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f246c002a30 tx=0x7f246c001510 comp rx=0 tx=0).stop 2024-05-08T05:44:06.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.552+0000 7f247d056640 1 -- 172.21.15.45:0/788876061 reap_dead start 2024-05-08T05:44:06.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.552+0000 7f247ed5bf40 1 -- 172.21.15.45:0/788876061 shutdown_connections 2024-05-08T05:44:06.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.552+0000 7f247ed5bf40 1 -- 172.21.15.45:0/788876061 >> 172.21.15.45:0/788876061 conn(0x55f8ac12a070 msgr2=0x55f8ac159f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:06.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.552+0000 7f247ed5bf40 1 -- 172.21.15.45:0/788876061 shutdown_connections 2024-05-08T05:44:06.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.552+0000 7f247ed5bf40 1 -- 172.21.15.45:0/788876061 wait complete. 2024-05-08T05:44:06.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:44:06.555 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-05-08T05:44:06.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.662+0000 7f8357546640 1 Processor -- start 2024-05-08T05:44:06.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.663+0000 7f8357546640 1 -- start start 2024-05-08T05:44:06.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.663+0000 7f8357546640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f83500b5a50 0x7f83500d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.663+0000 7f8357546640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83500cfc50 0x7f83500d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.663+0000 7f8357546640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8350162f50 con 0x7f8350162b70 2024-05-08T05:44:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.663+0000 7f8357546640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f83500d1f60 con 0x7f83500b5a50 2024-05-08T05:44:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.663+0000 7f8357546640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f83500d3ee0 con 0x7f83500cfc50 2024-05-08T05:44:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.664+0000 7f83552e2640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8350162b70 0x7f83500d2f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:53572/0 (socket says 172.21.15.45:53572) 2024-05-08T05:44:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.664+0000 7f83552e2640 1 -- 172.21.15.45:0/2604161748 learned_addr learned my addr 172.21.15.45:0/2604161748 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:44:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.664+0000 7f8355ae3640 1 --2- 172.21.15.45:0/2604161748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83500cfc50 0x7f83500d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.664+0000 7f8354ae1640 1 --2- 172.21.15.45:0/2604161748 >> v2:172.21.15.63:3300/0 conn(0x7f83500b5a50 0x7f83500d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.664+0000 7f834ffff640 1 -- 172.21.15.45:0/2604161748 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1040300496 0 0) 0x7f8348001110 con 0x7f8350162b70 2024-05-08T05:44:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.664+0000 7f834ffff640 1 -- 172.21.15.45:0/2604161748 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8338003520 con 0x7f8350162b70 2024-05-08T05:44:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.664+0000 7f834ffff640 1 -- 172.21.15.45:0/2604161748 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3822661095 0 0) 0x7f8338003520 con 0x7f8350162b70 2024-05-08T05:44:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.664+0000 7f8355ae3640 1 -- 172.21.15.45:0/2604161748 >> v2:172.21.15.63:3300/0 conn(0x7f83500b5a50 msgr2=0x7f83500d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.664+0000 7f8355ae3640 1 --2- 172.21.15.45:0/2604161748 >> v2:172.21.15.63:3300/0 conn(0x7f83500b5a50 0x7f83500d3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.664+0000 7f8355ae3640 1 -- 172.21.15.45:0/2604161748 >> v1:172.21.15.45:6789/0 conn(0x7f8350162b70 legacy=0x7f83500d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.664+0000 7f8355ae3640 1 -- 172.21.15.45:0/2604161748 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83500d21b0 con 0x7f83500cfc50 2024-05-08T05:44:06.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.665+0000 7f8355ae3640 1 --2- 172.21.15.45:0/2604161748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83500cfc50 0x7f83500d00c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8340009d90 tx=0x7f834002a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70816549bc3d9f64 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.665+0000 7f834ffff640 1 -- 172.21.15.45:0/2604161748 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83480018c0 con 0x7f83500cfc50 2024-05-08T05:44:06.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.665+0000 7f834ffff640 1 -- 172.21.15.45:0/2604161748 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8340007540 con 0x7f83500cfc50 2024-05-08T05:44:06.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.665+0000 7f834ffff640 1 -- 172.21.15.45:0/2604161748 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83400078f0 con 0x7f83500cfc50 2024-05-08T05:44:06.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.666+0000 7f8357546640 1 -- 172.21.15.45:0/2604161748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83500cfc50 msgr2=0x7f83500d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.666+0000 7f8357546640 1 --2- 172.21.15.45:0/2604161748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83500cfc50 0x7f83500d00c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8340009d90 tx=0x7f834002a5f0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.666+0000 7f8357546640 1 -- 172.21.15.45:0/2604161748 shutdown_connections 2024-05-08T05:44:06.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.666+0000 7f8357546640 1 --2- 172.21.15.45:0/2604161748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f83500cfc50 0x7f83500d00c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.666+0000 7f8357546640 1 --2- 172.21.15.45:0/2604161748 >> v2:172.21.15.63:3300/0 conn(0x7f83500b5a50 0x7f83500d3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.666+0000 7f8357546640 1 -- 172.21.15.45:0/2604161748 >> 172.21.15.45:0/2604161748 conn(0x7f83500d5040 msgr2=0x7f83500d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.666+0000 7f8357546640 1 -- 172.21.15.45:0/2604161748 shutdown_connections 2024-05-08T05:44:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.666+0000 7f8357546640 1 -- 172.21.15.45:0/2604161748 wait complete. 2024-05-08T05:44:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.667+0000 7f8357546640 1 Processor -- start 2024-05-08T05:44:06.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.667+0000 7f8357546640 1 -- start start 2024-05-08T05:44:06.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.668+0000 7f8357546640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83500b5a50 0x7f83500c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.668+0000 7f8357546640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83500cfc50 0x7f83500ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.668+0000 7f8357546640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8350162b70 0x7f83500ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.668+0000 7f8357546640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f83500b4170 con 0x7f83500b5a50 2024-05-08T05:44:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.668+0000 7f8357546640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f83500d3b90 con 0x7f83500cfc50 2024-05-08T05:44:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.668+0000 7f8357546640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f83500b42f0 con 0x7f8350162b70 2024-05-08T05:44:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.668+0000 7f83552e2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83500b5a50 0x7f83500c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.668+0000 7f83552e2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83500b5a50 0x7f83500c9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36852/0 (socket says 172.21.15.45:36852) 2024-05-08T05:44:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.668+0000 7f83552e2640 1 -- 172.21.15.45:0/2912597018 learned_addr learned my addr 172.21.15.45:0/2912597018 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.668+0000 7f8355ae3640 1 --2- 172.21.15.45:0/2912597018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8350162b70 0x7f83500ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.668+0000 7f8354ae1640 1 --2- 172.21.15.45:0/2912597018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83500cfc50 0x7f83500ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.668+0000 7f83552e2640 1 -- 172.21.15.45:0/2912597018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8350162b70 msgr2=0x7f83500ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.668+0000 7f83552e2640 1 --2- 172.21.15.45:0/2912597018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8350162b70 0x7f83500ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.668+0000 7f83552e2640 1 -- 172.21.15.45:0/2912597018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83500cfc50 msgr2=0x7f83500ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.668+0000 7f83552e2640 1 --2- 172.21.15.45:0/2912597018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83500cfc50 0x7f83500ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.668+0000 7f83552e2640 1 -- 172.21.15.45:0/2912597018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83500b87f0 con 0x7f83500b5a50 2024-05-08T05:44:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.668+0000 7f8355ae3640 1 --2- 172.21.15.45:0/2912597018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8350162b70 0x7f83500ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:44:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.668+0000 7f83552e2640 1 --2- 172.21.15.45:0/2912597018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83500b5a50 0x7f83500c9010 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f834800c730 tx=0x7f834800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.668+0000 7f834dffb640 1 -- 172.21.15.45:0/2912597018 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348017070 con 0x7f83500b5a50 2024-05-08T05:44:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.669+0000 7f834dffb640 1 -- 172.21.15.45:0/2912597018 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8348005ad0 con 0x7f83500b5a50 2024-05-08T05:44:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.669+0000 7f834dffb640 1 -- 172.21.15.45:0/2912597018 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348007070 con 0x7f83500b5a50 2024-05-08T05:44:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.669+0000 7f8357546640 1 -- 172.21.15.45:0/2912597018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83500b8ad0 con 0x7f83500b5a50 2024-05-08T05:44:06.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.669+0000 7f8357546640 1 -- 172.21.15.45:0/2912597018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83500b9090 con 0x7f83500b5a50 2024-05-08T05:44:06.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.670+0000 7f834dffb640 1 -- 172.21.15.45:0/2912597018 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f834801a070 con 0x7f83500b5a50 2024-05-08T05:44:06.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.671+0000 7f834dffb640 1 --2- 172.21.15.45:0/2912597018 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f833c0762d0 0x7f833c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.671+0000 7f834dffb640 1 -- 172.21.15.45:0/2912597018 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f8348096bf0 con 0x7f83500b5a50 2024-05-08T05:44:06.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.671+0000 7f8354ae1640 1 --2- 172.21.15.45:0/2912597018 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f833c0762d0 0x7f833c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.671+0000 7f8354ae1640 1 --2- 172.21.15.45:0/2912597018 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f833c0762d0 0x7f833c078790 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f83500d1c70 tx=0x7f8344008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.672+0000 7f835a265740 1 Processor -- start 2024-05-08T05:44:06.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.672+0000 7f835a265740 1 -- start start 2024-05-08T05:44:06.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.672+0000 7f835a265740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354cf890 0x5583354d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.672+0000 7f835a265740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 0x5583354d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.672+0000 7f835a265740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583354a24f0 0x5583354a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.672+0000 7f835a265740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55833540bbc0 con 0x5583354cfc70 2024-05-08T05:44:06.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.672+0000 7f835a265740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5583353c4d30 con 0x5583354cf890 2024-05-08T05:44:06.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.672+0000 7f835a265740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x558335407b80 con 0x5583354a24f0 2024-05-08T05:44:06.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.673+0000 7f8354ae1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 0x5583354d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.673+0000 7f8354ae1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 0x5583354d25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36868/0 (socket says 172.21.15.45:36868) 2024-05-08T05:44:06.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.673+0000 7f8354ae1640 1 -- 172.21.15.45:0/1642297501 learned_addr learned my addr 172.21.15.45:0/1642297501 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:06.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.673+0000 7f8355ae3640 1 --2- 172.21.15.45:0/1642297501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583354a24f0 0x5583354a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.673+0000 7f83552e2640 1 --2- 172.21.15.45:0/1642297501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354cf890 0x5583354d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.673+0000 7f8354ae1640 1 -- 172.21.15.45:0/1642297501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583354a24f0 msgr2=0x5583354a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.673+0000 7f8354ae1640 1 --2- 172.21.15.45:0/1642297501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583354a24f0 0x5583354a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.673+0000 7f8354ae1640 1 -- 172.21.15.45:0/1642297501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354cf890 msgr2=0x5583354d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.673+0000 7f8354ae1640 1 --2- 172.21.15.45:0/1642297501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354cf890 0x5583354d20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.673+0000 7f8354ae1640 1 -- 172.21.15.45:0/1642297501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583353a9100 con 0x5583354cfc70 2024-05-08T05:44:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.673+0000 7f8354ae1640 1 --2- 172.21.15.45:0/1642297501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 0x5583354d25e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8344011c60 tx=0x7f8344014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c079a8aae47d56b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.673+0000 7f8336ffd640 1 -- 172.21.15.45:0/1642297501 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344011c90 con 0x5583354cfc70 2024-05-08T05:44:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.673+0000 7f8336ffd640 1 -- 172.21.15.45:0/1642297501 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8344026070 con 0x5583354cfc70 2024-05-08T05:44:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.673+0000 7f8336ffd640 1 -- 172.21.15.45:0/1642297501 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344021440 con 0x5583354cfc70 2024-05-08T05:44:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.674+0000 7f835a265740 1 -- 172.21.15.45:0/1642297501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 msgr2=0x5583354d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.674+0000 7f835a265740 1 --2- 172.21.15.45:0/1642297501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 0x5583354d25e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8344011c60 tx=0x7f8344014000 comp rx=0 tx=0).stop 2024-05-08T05:44:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.674+0000 7f835a265740 1 -- 172.21.15.45:0/1642297501 shutdown_connections 2024-05-08T05:44:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.674+0000 7f835a265740 1 --2- 172.21.15.45:0/1642297501 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583354a24f0 0x5583354a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.674+0000 7f835a265740 1 --2- 172.21.15.45:0/1642297501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 0x5583354d25e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.674+0000 7f835a265740 1 --2- 172.21.15.45:0/1642297501 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354cf890 0x5583354d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.674+0000 7f835a265740 1 -- 172.21.15.45:0/1642297501 >> 172.21.15.45:0/1642297501 conn(0x5583354a4760 msgr2=0x5583354ed0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:06.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.674+0000 7f835a265740 1 -- 172.21.15.45:0/1642297501 shutdown_connections 2024-05-08T05:44:06.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.674+0000 7f835a265740 1 -- 172.21.15.45:0/1642297501 wait complete. 2024-05-08T05:44:06.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.674+0000 7f835a265740 1 -- start start 2024-05-08T05:44:06.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.674+0000 7f835a265740 1 Processor -- start 2024-05-08T05:44:06.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.675+0000 7f835a265740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cf890 0x5583354c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.675+0000 7f835a265740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583354cfc70 0x5583354d41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.675+0000 7f835a265740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354e76d0 0x5583354d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.675+0000 7f835a265740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5583354a7910 con 0x5583354cf890 2024-05-08T05:44:06.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.675+0000 7f835a265740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5583353c8870 con 0x5583354e76d0 2024-05-08T05:44:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.675+0000 7f835a265740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5583354a7bf0 con 0x5583354cfc70 2024-05-08T05:44:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.675+0000 7f835a265740 10 client.0 _ll_register_callbacks cb 0x558335418b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:44:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.675+0000 7f83552e2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cf890 0x5583354c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.675+0000 7f83552e2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cf890 0x5583354c6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36878/0 (socket says 172.21.15.45:36878) 2024-05-08T05:44:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.675+0000 7f83552e2640 1 -- 172.21.15.45:0/2397422706 learned_addr learned my addr 172.21.15.45:0/2397422706 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.676+0000 7f8354ae1640 1 --2- 172.21.15.45:0/2397422706 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583354cfc70 0x5583354d41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.676+0000 7f8355ae3640 1 --2- 172.21.15.45:0/2397422706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354e76d0 0x5583354d4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.676+0000 7f83552e2640 1 -- 172.21.15.45:0/2397422706 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583354cfc70 msgr2=0x5583354d41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.676+0000 7f83552e2640 1 --2- 172.21.15.45:0/2397422706 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583354cfc70 0x5583354d41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.676+0000 7f83552e2640 1 -- 172.21.15.45:0/2397422706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354e76d0 msgr2=0x5583354d4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.676+0000 7f83552e2640 1 --2- 172.21.15.45:0/2397422706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354e76d0 0x5583354d4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.676+0000 7f83552e2640 1 -- 172.21.15.45:0/2397422706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583351e97b0 con 0x5583354cf890 2024-05-08T05:44:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.676+0000 7f83552e2640 1 --2- 172.21.15.45:0/2397422706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cf890 0x5583354c6b40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f834800a430 tx=0x7f83480602f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d00031ba398fbf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.676+0000 7f832bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:44:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.676+0000 7f832bfff640 1 -- 172.21.15.45:0/2397422706 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834805fd50 con 0x5583354cf890 2024-05-08T05:44:06.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.676+0000 7f832bfff640 1 -- 172.21.15.45:0/2397422706 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f834805efb0 con 0x5583354cf890 2024-05-08T05:44:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.676+0000 7f835a265740 1 -- 172.21.15.45:0/2397422706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8344026070 con 0x5583354cf890 2024-05-08T05:44:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.676+0000 7f832bfff640 1 -- 172.21.15.45:0/2397422706 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8348064100 con 0x5583354cf890 2024-05-08T05:44:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.676+0000 7f835a265740 10 client.5868 fetch_fsmap learned FSMap version 34 2024-05-08T05:44:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.676+0000 7f835a265740 1 -- 172.21.15.45:0/2397422706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x558335249f40 con 0x5583354cf890 2024-05-08T05:44:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.676+0000 7f832bfff640 1 -- 172.21.15.45:0/2397422706 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8344026070 con 0x5583354cf890 2024-05-08T05:44:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.676+0000 7f832bfff640 1 -- 172.21.15.45:0/2397422706 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f834805b6c0 con 0x5583354cf890 2024-05-08T05:44:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.676+0000 7f835a265740 10 client.5868 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T05:44:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.676+0000 7f835a265740 10 client.5868 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T05:44:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.676+0000 7f835a265740 20 client.5868 populate_metadata read hostname 'smithi045' 2024-05-08T05:44:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.677+0000 7f835a265740 1 --2- 172.21.15.45:0/2397422706 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5583354eee90 0x5583354ef270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.677+0000 7f835a265740 4 client.5868 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:44:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.677+0000 7f835a265740 1 -- 172.21.15.45:0/2397422706 --> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583354197d0 con 0x5583354eee90 2024-05-08T05:44:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.677+0000 7f8354ae1640 1 --2- 172.21.15.45:0/2397422706 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5583354eee90 0x5583354ef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.677+0000 7f8354ae1640 1 --2- 172.21.15.45:0/2397422706 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5583354eee90 0x5583354ef270 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b59d71e610e2e9e server_cookie=be0c2574a6b082f4 in_seq=0 out_seq=0 2024-05-08T05:44:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.677+0000 7f832bfff640 10 client.5868 ms_handle_connect on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.677+0000 7f832bfff640 1 -- 172.21.15.45:0/2397422706 <== mds.0 v2:172.21.15.45:6833/1461822016 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5583354197d0 con 0x5583354eee90 2024-05-08T05:44:06.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.677+0000 7f832bfff640 10 client.5868 handle_command_reply: tid=0 2024-05-08T05:44:06.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.678+0000 7f835a265740 1 client.5868 shutdown 2024-05-08T05:44:06.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.678+0000 7f835a265740 1 -- 172.21.15.45:0/2397422706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cf890 msgr2=0x5583354c6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.678+0000 7f835a265740 1 --2- 172.21.15.45:0/2397422706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cf890 0x5583354c6b40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f834800a430 tx=0x7f83480602f0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.678+0000 7f835a265740 1 -- 172.21.15.45:0/2397422706 shutdown_connections 2024-05-08T05:44:06.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.678+0000 7f835a265740 1 --2- 172.21.15.45:0/2397422706 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5583354eee90 0x5583354ef270 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.678+0000 7f835a265740 1 --2- 172.21.15.45:0/2397422706 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583354cfc70 0x5583354d41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.678+0000 7f835a265740 1 --2- 172.21.15.45:0/2397422706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354e76d0 0x5583354d4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.678+0000 7f835a265740 1 --2- 172.21.15.45:0/2397422706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cf890 0x5583354c6b40 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.678+0000 7f832bfff640 0 client.5868 ms_handle_reset on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:06.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.678+0000 7f835a265740 1 -- 172.21.15.45:0/2397422706 >> 172.21.15.45:0/2397422706 conn(0x5583354a24f0 msgr2=0x5583354a1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:06.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.678+0000 7f835a265740 1 -- 172.21.15.45:0/2397422706 shutdown_connections 2024-05-08T05:44:06.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.678+0000 7f835a265740 1 -- 172.21.15.45:0/2397422706 wait complete. 2024-05-08T05:44:06.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.678+0000 7f835a265740 20 client.5868 trim_cache size 0 max 16384 2024-05-08T05:44:06.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.696+0000 7f835a265740 1 Processor -- start 2024-05-08T05:44:06.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.696+0000 7f835a265740 1 -- start start 2024-05-08T05:44:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.697+0000 7f835a265740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354cf890 0x5583354d5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.697+0000 7f835a265740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 0x5583354d60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.697+0000 7f835a265740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583354e76d0 0x558335510b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.697+0000 7f835a265740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8350201e20 con 0x5583354cfc70 2024-05-08T05:44:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.697+0000 7f835a265740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f83500ba7d0 con 0x5583354cf890 2024-05-08T05:44:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.697+0000 7f835a265740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f83500cbb00 con 0x5583354e76d0 2024-05-08T05:44:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.697+0000 7f8354ae1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 0x5583354d60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.697+0000 7f8354ae1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 0x5583354d60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36892/0 (socket says 172.21.15.45:36892) 2024-05-08T05:44:06.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.697+0000 7f8354ae1640 1 -- 172.21.15.45:0/1869896889 learned_addr learned my addr 172.21.15.45:0/1869896889 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.697+0000 7f83552e2640 1 --2- 172.21.15.45:0/1869896889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354cf890 0x5583354d5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.697+0000 7f8355ae3640 1 --2- 172.21.15.45:0/1869896889 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583354e76d0 0x558335510b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.697+0000 7f8354ae1640 1 -- 172.21.15.45:0/1869896889 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583354e76d0 msgr2=0x558335510b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.697+0000 7f8354ae1640 1 --2- 172.21.15.45:0/1869896889 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583354e76d0 0x558335510b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.697+0000 7f8354ae1640 1 -- 172.21.15.45:0/1869896889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354cf890 msgr2=0x5583354d5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.697+0000 7f8354ae1640 1 --2- 172.21.15.45:0/1869896889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354cf890 0x5583354d5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.697+0000 7f8354ae1640 1 -- 172.21.15.45:0/1869896889 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583350dd0a0 con 0x5583354cfc70 2024-05-08T05:44:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.697+0000 7f83552e2640 1 --2- 172.21.15.45:0/1869896889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354cf890 0x5583354d5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:44:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.697+0000 7f8354ae1640 1 --2- 172.21.15.45:0/1869896889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 0x5583354d60b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8344023040 tx=0x7f8344018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2549c8978b48c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.698+0000 7f83367fc640 1 -- 172.21.15.45:0/1869896889 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834403d070 con 0x5583354cfc70 2024-05-08T05:44:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.698+0000 7f83367fc640 1 -- 172.21.15.45:0/1869896889 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8344026040 con 0x5583354cfc70 2024-05-08T05:44:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.698+0000 7f83367fc640 1 -- 172.21.15.45:0/1869896889 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344036670 con 0x5583354cfc70 2024-05-08T05:44:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.698+0000 7f835a265740 1 -- 172.21.15.45:0/1869896889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 msgr2=0x5583354d60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.698+0000 7f835a265740 1 --2- 172.21.15.45:0/1869896889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 0x5583354d60b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8344023040 tx=0x7f8344018000 comp rx=0 tx=0).stop 2024-05-08T05:44:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.698+0000 7f835a265740 1 -- 172.21.15.45:0/1869896889 shutdown_connections 2024-05-08T05:44:06.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.698+0000 7f835a265740 1 --2- 172.21.15.45:0/1869896889 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583354e76d0 0x558335510b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.698+0000 7f835a265740 1 --2- 172.21.15.45:0/1869896889 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 0x5583354d60b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.698+0000 7f835a265740 1 --2- 172.21.15.45:0/1869896889 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354cf890 0x5583354d5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.698+0000 7f835a265740 1 -- 172.21.15.45:0/1869896889 >> 172.21.15.45:0/1869896889 conn(0x5583354a24f0 msgr2=0x5583354e9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:06.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.698+0000 7f835a265740 1 -- 172.21.15.45:0/1869896889 shutdown_connections 2024-05-08T05:44:06.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.698+0000 7f835a265740 1 -- 172.21.15.45:0/1869896889 wait complete. 2024-05-08T05:44:06.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.699+0000 7f835a265740 1 -- start start 2024-05-08T05:44:06.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.699+0000 7f835a265740 1 Processor -- start 2024-05-08T05:44:06.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.700+0000 7f835a265740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354cf890 0x55833556e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.700+0000 7f835a265740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 0x5583354a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.700+0000 7f835a265740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583354e76d0 0x5583354a6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.700+0000 7f835a265740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5583354ef270 con 0x5583354cfc70 2024-05-08T05:44:06.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.700+0000 7f835a265740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f83500ba180 con 0x5583354cf890 2024-05-08T05:44:06.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.700+0000 7f835a265740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5583354d2100 con 0x5583354e76d0 2024-05-08T05:44:06.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.700+0000 7f835a265740 10 client.0 _ll_register_callbacks cb 0x7f834800cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:44:06.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.700+0000 7f8354ae1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 0x5583354a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.700+0000 7f8354ae1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 0x5583354a6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36904/0 (socket says 172.21.15.45:36904) 2024-05-08T05:44:06.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.700+0000 7f8354ae1640 1 -- 172.21.15.45:0/3663308127 learned_addr learned my addr 172.21.15.45:0/3663308127 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:06.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.700+0000 7f83552e2640 1 --2- 172.21.15.45:0/3663308127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354cf890 0x55833556e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.700+0000 7f8355ae3640 1 --2- 172.21.15.45:0/3663308127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583354e76d0 0x5583354a6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.700+0000 7f8354ae1640 1 -- 172.21.15.45:0/3663308127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583354e76d0 msgr2=0x5583354a6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.700+0000 7f8354ae1640 1 --2- 172.21.15.45:0/3663308127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583354e76d0 0x5583354a6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.700+0000 7f8354ae1640 1 -- 172.21.15.45:0/3663308127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354cf890 msgr2=0x55833556e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.700+0000 7f8354ae1640 1 --2- 172.21.15.45:0/3663308127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354cf890 0x55833556e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.700+0000 7f8354ae1640 1 -- 172.21.15.45:0/3663308127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583354ef8d0 con 0x5583354cfc70 2024-05-08T05:44:06.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.700+0000 7f8354ae1640 1 --2- 172.21.15.45:0/3663308127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 0x5583354a6a30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f834400b010 tx=0x7f83440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77108c7be2e5fb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.700+0000 7f832bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:44:06.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.700+0000 7f832bfff640 1 -- 172.21.15.45:0/3663308127 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834403d030 con 0x5583354cfc70 2024-05-08T05:44:06.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.701+0000 7f832bfff640 1 -- 172.21.15.45:0/3663308127 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7f834400f040 con 0x5583354cfc70 2024-05-08T05:44:06.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.701+0000 7f835a265740 1 -- 172.21.15.45:0/3663308127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8344026040 con 0x5583354cfc70 2024-05-08T05:44:06.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.701+0000 7f832bfff640 1 -- 172.21.15.45:0/3663308127 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8344011d10 con 0x5583354cfc70 2024-05-08T05:44:06.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.701+0000 7f835a265740 10 client.5874 fetch_fsmap learned FSMap version 34 2024-05-08T05:44:06.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.701+0000 7f835a265740 1 -- 172.21.15.45:0/3663308127 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5583354ca050 con 0x5583354cfc70 2024-05-08T05:44:06.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.701+0000 7f832bfff640 1 -- 172.21.15.45:0/3663308127 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8344026040 con 0x5583354cfc70 2024-05-08T05:44:06.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.701+0000 7f832bfff640 1 -- 172.21.15.45:0/3663308127 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f8344036450 con 0x5583354cfc70 2024-05-08T05:44:06.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.701+0000 7f835a265740 10 client.5874 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T05:44:06.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.701+0000 7f835a265740 10 client.5874 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T05:44:06.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.701+0000 7f835a265740 20 client.5874 populate_metadata read hostname 'smithi045' 2024-05-08T05:44:06.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.701+0000 7f835a265740 1 --2- 172.21.15.45:0/3663308127 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5583354ed2c0 0x5583354c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.701+0000 7f835a265740 4 client.5874 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-05-08T05:44:06.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.701+0000 7f835a265740 1 -- 172.21.15.45:0/3663308127 --> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x558335413640 con 0x5583354ed2c0 2024-05-08T05:44:06.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.701+0000 7f83552e2640 1 --2- 172.21.15.45:0/3663308127 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5583354ed2c0 0x5583354c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.701+0000 7f83552e2640 1 --2- 172.21.15.45:0/3663308127 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5583354ed2c0 0x5583354c90e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6b26592998f2308 server_cookie=ba91e1a794a6bd7b in_seq=0 out_seq=0 2024-05-08T05:44:06.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.701+0000 7f832bfff640 10 client.5874 ms_handle_connect on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:06.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.702+0000 7f832bfff640 1 -- 172.21.15.45:0/3663308127 <== mds.0 v2:172.21.15.45:6833/1461822016 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558335413640 con 0x5583354ed2c0 2024-05-08T05:44:06.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.702+0000 7f832bfff640 10 client.5874 handle_command_reply: tid=0 2024-05-08T05:44:06.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.702+0000 7f835a265740 1 client.5874 shutdown 2024-05-08T05:44:06.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.702+0000 7f835a265740 1 -- 172.21.15.45:0/3663308127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 msgr2=0x5583354a6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.702+0000 7f835a265740 1 --2- 172.21.15.45:0/3663308127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 0x5583354a6a30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f834400b010 tx=0x7f83440046c0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.702+0000 7f835a265740 1 -- 172.21.15.45:0/3663308127 shutdown_connections 2024-05-08T05:44:06.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.702+0000 7f835a265740 1 --2- 172.21.15.45:0/3663308127 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5583354ed2c0 0x5583354c90e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.702+0000 7f835a265740 1 --2- 172.21.15.45:0/3663308127 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583354e76d0 0x5583354a6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.702+0000 7f835a265740 1 --2- 172.21.15.45:0/3663308127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583354cfc70 0x5583354a6a30 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.702+0000 7f835a265740 1 --2- 172.21.15.45:0/3663308127 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583354cf890 0x55833556e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.702+0000 7f835a265740 1 -- 172.21.15.45:0/3663308127 >> 172.21.15.45:0/3663308127 conn(0x5583354a24f0 msgr2=0x5583354d5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:06.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.702+0000 7f832bfff640 0 client.5874 ms_handle_reset on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:06.708 INFO:tasks.ceph.mds.a.smithi045.stderr:2024-05-08T05:44:06.702+0000 7f3443f19640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5177=0-4194304@1} caps={5177=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5177 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x560338028b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-08T05:44:06.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.702+0000 7f835a265740 1 -- 172.21.15.45:0/3663308127 shutdown_connections 2024-05-08T05:44:06.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.702+0000 7f835a265740 1 -- 172.21.15.45:0/3663308127 wait complete. 2024-05-08T05:44:06.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.703+0000 7f835a265740 20 client.5874 trim_cache size 0 max 16384 2024-05-08T05:44:06.709 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:44:06.709 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-05-08T05:44:06.709 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "53b2e38d-5042-45be-937f-1c1d8568f686", 2024-05-08T05:44:06.709 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-05-08T05:44:06.709 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:44:06.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.703+0000 7f8357546640 1 -- 172.21.15.45:0/2912597018 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f833c0762d0 msgr2=0x7f833c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.703+0000 7f8357546640 1 --2- 172.21.15.45:0/2912597018 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f833c0762d0 0x7f833c078790 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f83500d1c70 tx=0x7f8344008040 comp rx=0 tx=0).stop 2024-05-08T05:44:06.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.703+0000 7f8357546640 1 -- 172.21.15.45:0/2912597018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83500b5a50 msgr2=0x7f83500c9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.703+0000 7f8357546640 1 --2- 172.21.15.45:0/2912597018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83500b5a50 0x7f83500c9010 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f834800c730 tx=0x7f834800cbf0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.703+0000 7f8357546640 1 -- 172.21.15.45:0/2912597018 shutdown_connections 2024-05-08T05:44:06.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.703+0000 7f8357546640 1 --2- 172.21.15.45:0/2912597018 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f833c0762d0 0x7f833c078790 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.703+0000 7f8357546640 1 --2- 172.21.15.45:0/2912597018 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8350162b70 0x7f83500ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.703+0000 7f8357546640 1 --2- 172.21.15.45:0/2912597018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83500cfc50 0x7f83500ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.703+0000 7f8357546640 1 --2- 172.21.15.45:0/2912597018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83500b5a50 0x7f83500c9010 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.703+0000 7f8357546640 1 -- 172.21.15.45:0/2912597018 >> 172.21.15.45:0/2912597018 conn(0x7f83500d5040 msgr2=0x7f83500d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:06.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.703+0000 7f8357546640 1 -- 172.21.15.45:0/2912597018 shutdown_connections 2024-05-08T05:44:06.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.703+0000 7f8357546640 1 -- 172.21.15.45:0/2912597018 wait complete. 2024-05-08T05:44:06.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:44:06.714 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-08T05:44:06.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.822+0000 7feaf3958640 1 Processor -- start 2024-05-08T05:44:06.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.823+0000 7feaf3958640 1 -- start start 2024-05-08T05:44:06.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.823+0000 7feaf3958640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feaec15f420 0x7feaec0d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.823+0000 7feaf3958640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feaec0d3ca0 0x7feaec0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.823+0000 7feaf3958640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaec1630b0 con 0x7feaec0d1eb0 2024-05-08T05:44:06.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.823+0000 7feaf3958640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7feaec0d22f0 con 0x7feaec0d3ca0 2024-05-08T05:44:06.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.823+0000 7feaf3958640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feaec0d2550 con 0x7feaec15f420 2024-05-08T05:44:06.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.823+0000 7feaf0ef3640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feaec0d3ca0 0x7feaec0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.824+0000 7feaf0ef3640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7feaec0d3ca0 0x7feaec0d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36910/0 (socket says 172.21.15.45:36910) 2024-05-08T05:44:06.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.824+0000 7feaf0ef3640 1 -- 172.21.15.45:0/232065213 learned_addr learned my addr 172.21.15.45:0/232065213 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:06.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.824+0000 7feaf16f4640 1 --2- 172.21.15.45:0/232065213 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feaec15f420 0x7feaec0d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.824+0000 7feaf0ef3640 1 -- 172.21.15.45:0/232065213 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feaec15f420 msgr2=0x7feaec0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.824+0000 7feaf0ef3640 1 --2- 172.21.15.45:0/232065213 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feaec15f420 0x7feaec0d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.824+0000 7feaf0ef3640 1 -- 172.21.15.45:0/232065213 >> v1:172.21.15.63:6789/0 conn(0x7feaec0d1eb0 legacy=0x7feaec0d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.824+0000 7feaf0ef3640 1 -- 172.21.15.45:0/232065213 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaec0d2c80 con 0x7feaec0d3ca0 2024-05-08T05:44:06.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.824+0000 7feaf0ef3640 1 --2- 172.21.15.45:0/232065213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feaec0d3ca0 0x7feaec0d1970 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7feae0009870 tx=0x7feae002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d72a8a68f7d8ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.825+0000 7feaebfff640 1 -- 172.21.15.45:0/232065213 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0037070 con 0x7feaec0d3ca0 2024-05-08T05:44:06.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.825+0000 7feaebfff640 1 -- 172.21.15.45:0/232065213 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae0032440 con 0x7feaec0d3ca0 2024-05-08T05:44:06.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.825+0000 7feaebfff640 1 -- 172.21.15.45:0/232065213 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0032810 con 0x7feaec0d3ca0 2024-05-08T05:44:06.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.825+0000 7feaf3958640 1 -- 172.21.15.45:0/232065213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feaec0d3ca0 msgr2=0x7feaec0d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.825+0000 7feaf3958640 1 --2- 172.21.15.45:0/232065213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feaec0d3ca0 0x7feaec0d1970 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7feae0009870 tx=0x7feae002a1c0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.825+0000 7feaf3958640 1 -- 172.21.15.45:0/232065213 shutdown_connections 2024-05-08T05:44:06.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.825+0000 7feaf3958640 1 --2- 172.21.15.45:0/232065213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feaec0d3ca0 0x7feaec0d1970 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.825+0000 7feaf3958640 1 --2- 172.21.15.45:0/232065213 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feaec15f420 0x7feaec0d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.825+0000 7feaf3958640 1 -- 172.21.15.45:0/232065213 >> 172.21.15.45:0/232065213 conn(0x7feaec08eac0 msgr2=0x7feaec09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:06.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.825+0000 7feaf3958640 1 -- 172.21.15.45:0/232065213 shutdown_connections 2024-05-08T05:44:06.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.825+0000 7feaf3958640 1 -- 172.21.15.45:0/232065213 wait complete. 2024-05-08T05:44:06.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.826+0000 7feaf3958640 1 Processor -- start 2024-05-08T05:44:06.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.827+0000 7feaf3958640 1 -- start start 2024-05-08T05:44:06.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.827+0000 7feaf3958640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feaec0d1eb0 0x7feaec15d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.827+0000 7feaf3958640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feaec0c8b00 0x7feaec0ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.827+0000 7feaf3958640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feaec0cb3b0 0x7feaec160de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.827+0000 7feaf3958640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feaec0d2830 con 0x7feaec0cb3b0 2024-05-08T05:44:06.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.827+0000 7feaf3958640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feaec0d3240 con 0x7feaec0c8b00 2024-05-08T05:44:06.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.827+0000 7feaf3958640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feaec0d53d0 con 0x7feaec0d1eb0 2024-05-08T05:44:06.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.827+0000 7feaf1ef5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feaec0cb3b0 0x7feaec160de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.827+0000 7feaf1ef5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feaec0cb3b0 0x7feaec160de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36916/0 (socket says 172.21.15.45:36916) 2024-05-08T05:44:06.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.828+0000 7feaf1ef5640 1 -- 172.21.15.45:0/2756335414 learned_addr learned my addr 172.21.15.45:0/2756335414 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:06.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.828+0000 7feaf16f4640 1 --2- 172.21.15.45:0/2756335414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feaec0d1eb0 0x7feaec15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.828+0000 7feaf0ef3640 1 --2- 172.21.15.45:0/2756335414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feaec0c8b00 0x7feaec0ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.828+0000 7feaf1ef5640 1 -- 172.21.15.45:0/2756335414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feaec0d1eb0 msgr2=0x7feaec15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.828+0000 7feaf1ef5640 1 --2- 172.21.15.45:0/2756335414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feaec0d1eb0 0x7feaec15d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.828+0000 7feaf1ef5640 1 -- 172.21.15.45:0/2756335414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feaec0c8b00 msgr2=0x7feaec0ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.828+0000 7feaf1ef5640 1 --2- 172.21.15.45:0/2756335414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feaec0c8b00 0x7feaec0ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.828+0000 7feaf1ef5640 1 -- 172.21.15.45:0/2756335414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaec0ca5a0 con 0x7feaec0cb3b0 2024-05-08T05:44:06.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.828+0000 7feaf1ef5640 1 --2- 172.21.15.45:0/2756335414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feaec0cb3b0 0x7feaec160de0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7feadc00c780 tx=0x7feadc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.828+0000 7feae9ffb640 1 -- 172.21.15.45:0/2756335414 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc004b60 con 0x7feaec0cb3b0 2024-05-08T05:44:06.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.828+0000 7feae9ffb640 1 -- 172.21.15.45:0/2756335414 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feadc004d00 con 0x7feaec0cb3b0 2024-05-08T05:44:06.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.828+0000 7feaf3958640 1 -- 172.21.15.45:0/2756335414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaec0c8fc0 con 0x7feaec0cb3b0 2024-05-08T05:44:06.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.828+0000 7feae9ffb640 1 -- 172.21.15.45:0/2756335414 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc00aa90 con 0x7feaec0cb3b0 2024-05-08T05:44:06.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.829+0000 7feaf3958640 1 -- 172.21.15.45:0/2756335414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaec0c9580 con 0x7feaec0cb3b0 2024-05-08T05:44:06.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.830+0000 7feae9ffb640 1 -- 172.21.15.45:0/2756335414 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7feadc00ac30 con 0x7feaec0cb3b0 2024-05-08T05:44:06.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.831+0000 7feae9ffb640 1 --2- 172.21.15.45:0/2756335414 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fead80762d0 0x7fead8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.831+0000 7feae9ffb640 1 -- 172.21.15.45:0/2756335414 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7feadc097860 con 0x7feaec0cb3b0 2024-05-08T05:44:06.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.831+0000 7feaf16f4640 1 --2- 172.21.15.45:0/2756335414 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fead80762d0 0x7fead8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.831+0000 7feaf16f4640 1 --2- 172.21.15.45:0/2756335414 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fead80762d0 0x7fead8078790 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feae40079a0 tx=0x7feae4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.831+0000 7feaf6675740 1 Processor -- start 2024-05-08T05:44:06.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.831+0000 7feaf6675740 1 -- start start 2024-05-08T05:44:06.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.832+0000 7feaf6675740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1735160 0x5606f16e7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.832+0000 7feaf6675740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1718280 0x5606f1717600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.832+0000 7feaf6675740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1717b40 0x5606f16ec950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.832+0000 7feaf6675740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5606f16b4690 con 0x5606f1735160 2024-05-08T05:44:06.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.832+0000 7feaf6675740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5606f1651f30 con 0x5606f1718280 2024-05-08T05:44:06.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.832+0000 7feaf6675740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5606f160ab20 con 0x5606f1717b40 2024-05-08T05:44:06.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.832+0000 7feaf16f4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1735160 0x5606f16e7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.832+0000 7feaf16f4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1735160 0x5606f16e7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36922/0 (socket says 172.21.15.45:36922) 2024-05-08T05:44:06.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.832+0000 7feaf16f4640 1 -- 172.21.15.45:0/3493447086 learned_addr learned my addr 172.21.15.45:0/3493447086 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:06.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.832+0000 7feaf1ef5640 1 --2- 172.21.15.45:0/3493447086 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1717b40 0x5606f16ec950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.832+0000 7feaf0ef3640 1 --2- 172.21.15.45:0/3493447086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1718280 0x5606f1717600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.832+0000 7feaf1ef5640 1 -- 172.21.15.45:0/3493447086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1718280 msgr2=0x5606f1717600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.833+0000 7feaf1ef5640 1 --2- 172.21.15.45:0/3493447086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1718280 0x5606f1717600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.833+0000 7feaf1ef5640 1 -- 172.21.15.45:0/3493447086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1735160 msgr2=0x5606f16e7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.833+0000 7feaf1ef5640 1 --2- 172.21.15.45:0/3493447086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1735160 0x5606f16e7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.833+0000 7feaf1ef5640 1 -- 172.21.15.45:0/3493447086 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606f1602e60 con 0x5606f1717b40 2024-05-08T05:44:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.833+0000 7feaf16f4640 1 --2- 172.21.15.45:0/3493447086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1735160 0x5606f16e7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:44:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.833+0000 7feaf1ef5640 1 --2- 172.21.15.45:0/3493447086 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1717b40 0x5606f16ec950 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feadc095bd0 tx=0x7feadc05c9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d38c6c0d01e54c3e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.833+0000 7fead2ffd640 1 -- 172.21.15.45:0/3493447086 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc0604f0 con 0x5606f1717b40 2024-05-08T05:44:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.833+0000 7fead2ffd640 1 -- 172.21.15.45:0/3493447086 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feadc0609e0 con 0x5606f1717b40 2024-05-08T05:44:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.833+0000 7fead2ffd640 1 -- 172.21.15.45:0/3493447086 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc064ce0 con 0x5606f1717b40 2024-05-08T05:44:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.833+0000 7feaf6675740 1 -- 172.21.15.45:0/3493447086 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1717b40 msgr2=0x5606f16ec950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.833+0000 7feaf6675740 1 --2- 172.21.15.45:0/3493447086 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1717b40 0x5606f16ec950 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feadc095bd0 tx=0x7feadc05c9d0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.834+0000 7feaf6675740 1 -- 172.21.15.45:0/3493447086 shutdown_connections 2024-05-08T05:44:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.834+0000 7feaf6675740 1 --2- 172.21.15.45:0/3493447086 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1717b40 0x5606f16ec950 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.834+0000 7feaf6675740 1 --2- 172.21.15.45:0/3493447086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1718280 0x5606f1717600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.834+0000 7feaf6675740 1 --2- 172.21.15.45:0/3493447086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1735160 0x5606f16e7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.834+0000 7feaf6675740 1 -- 172.21.15.45:0/3493447086 >> 172.21.15.45:0/3493447086 conn(0x5606f16e9d60 msgr2=0x5606f1732600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:06.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.834+0000 7feaf6675740 1 -- 172.21.15.45:0/3493447086 shutdown_connections 2024-05-08T05:44:06.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.834+0000 7feaf6675740 1 -- 172.21.15.45:0/3493447086 wait complete. 2024-05-08T05:44:06.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.834+0000 7feaf6675740 1 -- start start 2024-05-08T05:44:06.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.834+0000 7feaf6675740 1 Processor -- start 2024-05-08T05:44:06.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.835+0000 7feaf6675740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1718280 0x5606f1734140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.835+0000 7feaf6675740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1735160 0x5606f1731600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.835+0000 7feaf6675740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1719180 0x5606f1707a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.835+0000 7feaf6675740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5606f160e5e0 con 0x5606f1718280 2024-05-08T05:44:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.835+0000 7feaf6675740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5606f164def0 con 0x5606f1735160 2024-05-08T05:44:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.835+0000 7feaf6675740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5606f160ab20 con 0x5606f1719180 2024-05-08T05:44:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.835+0000 7feaf6675740 10 client.0 _ll_register_callbacks cb 0x5606f1609320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:44:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.835+0000 7feaf16f4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1718280 0x5606f1734140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.835+0000 7feaf16f4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1718280 0x5606f1734140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36938/0 (socket says 172.21.15.45:36938) 2024-05-08T05:44:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.835+0000 7feaf1ef5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1719180 0x5606f1707a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.835+0000 7feaf16f4640 1 -- 172.21.15.45:0/1549439493 learned_addr learned my addr 172.21.15.45:0/1549439493 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.835+0000 7feaf0ef3640 1 --2- 172.21.15.45:0/1549439493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1735160 0x5606f1731600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.835+0000 7feaf16f4640 1 -- 172.21.15.45:0/1549439493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1719180 msgr2=0x5606f1707a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.835+0000 7feaf16f4640 1 --2- 172.21.15.45:0/1549439493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1719180 0x5606f1707a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.835+0000 7feaf16f4640 1 -- 172.21.15.45:0/1549439493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1735160 msgr2=0x5606f1731600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.835+0000 7feaf16f4640 1 --2- 172.21.15.45:0/1549439493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1735160 0x5606f1731600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.836+0000 7feaf16f4640 1 -- 172.21.15.45:0/1549439493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606f142d7d0 con 0x5606f1718280 2024-05-08T05:44:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.836+0000 7feaf1ef5640 1 --2- 172.21.15.45:0/1549439493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1719180 0x5606f1707a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:44:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.836+0000 7feaf0ef3640 1 --2- 172.21.15.45:0/1549439493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1735160 0x5606f1731600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:44:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.836+0000 7feaf16f4640 1 --2- 172.21.15.45:0/1549439493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1718280 0x5606f1734140 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7feae4016820 tx=0x7feae40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f093bd4524ac311 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.836+0000 7feacbfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:44:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.836+0000 7feacbfff640 1 -- 172.21.15.45:0/1549439493 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae4026020 con 0x5606f1718280 2024-05-08T05:44:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.836+0000 7feacbfff640 1 -- 172.21.15.45:0/1549439493 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7feae4014930 con 0x5606f1718280 2024-05-08T05:44:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.836+0000 7feaf6675740 1 -- 172.21.15.45:0/1549439493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feadc0609e0 con 0x5606f1718280 2024-05-08T05:44:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.836+0000 7feacbfff640 1 -- 172.21.15.45:0/1549439493 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7feae4030050 con 0x5606f1718280 2024-05-08T05:44:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.836+0000 7feaf6675740 10 client.5886 fetch_fsmap learned FSMap version 34 2024-05-08T05:44:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.836+0000 7feaf6675740 1 -- 172.21.15.45:0/1549439493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5606f13633f0 con 0x5606f1718280 2024-05-08T05:44:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.836+0000 7feacbfff640 1 -- 172.21.15.45:0/1549439493 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feadc0609e0 con 0x5606f1718280 2024-05-08T05:44:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.836+0000 7feacbfff640 1 -- 172.21.15.45:0/1549439493 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7feae4033020 con 0x5606f1718280 2024-05-08T05:44:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.836+0000 7feaf6675740 10 client.5886 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T05:44:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.836+0000 7feaf6675740 10 client.5886 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T05:44:06.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.836+0000 7feaf6675740 20 client.5886 populate_metadata read hostname 'smithi045' 2024-05-08T05:44:06.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.836+0000 7feaf6675740 1 --2- 172.21.15.45:0/1549439493 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5606f1742350 0x5606f1742780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.836+0000 7feaf6675740 4 client.5886 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:44:06.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.836+0000 7feaf6675740 1 -- 172.21.15.45:0/1549439493 --> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606f165fb40 con 0x5606f1742350 2024-05-08T05:44:06.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.837+0000 7feaf0ef3640 1 --2- 172.21.15.45:0/1549439493 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5606f1742350 0x5606f1742780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.837+0000 7feaf0ef3640 1 --2- 172.21.15.45:0/1549439493 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5606f1742350 0x5606f1742780 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c48672319b7148e server_cookie=97d9261d83488454 in_seq=0 out_seq=0 2024-05-08T05:44:06.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.837+0000 7feacbfff640 10 client.5886 ms_handle_connect on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:06.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.837+0000 7feacbfff640 1 -- 172.21.15.45:0/1549439493 <== mds.0 v2:172.21.15.45:6833/1461822016 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5606f165fb40 con 0x5606f1742350 2024-05-08T05:44:06.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.837+0000 7feacbfff640 10 client.5886 handle_command_reply: tid=0 2024-05-08T05:44:06.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.837+0000 7feaf6675740 1 client.5886 shutdown 2024-05-08T05:44:06.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.838+0000 7feaf6675740 1 -- 172.21.15.45:0/1549439493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1718280 msgr2=0x5606f1734140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.838+0000 7feaf6675740 1 --2- 172.21.15.45:0/1549439493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1718280 0x5606f1734140 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7feae4016820 tx=0x7feae40046c0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.838+0000 7feaf6675740 1 -- 172.21.15.45:0/1549439493 shutdown_connections 2024-05-08T05:44:06.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.838+0000 7feaf6675740 1 --2- 172.21.15.45:0/1549439493 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5606f1742350 0x5606f1742780 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.838+0000 7feaf6675740 1 --2- 172.21.15.45:0/1549439493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1719180 0x5606f1707a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.838+0000 7feaf6675740 1 --2- 172.21.15.45:0/1549439493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1735160 0x5606f1731600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.838+0000 7feaf6675740 1 --2- 172.21.15.45:0/1549439493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1718280 0x5606f1734140 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.838+0000 7feacbfff640 0 client.5886 ms_handle_reset on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:06.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.838+0000 7feaf6675740 1 -- 172.21.15.45:0/1549439493 >> 172.21.15.45:0/1549439493 conn(0x5606f1717b40 msgr2=0x5606f16e7230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:06.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.838+0000 7feaf6675740 1 -- 172.21.15.45:0/1549439493 shutdown_connections 2024-05-08T05:44:06.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.838+0000 7feaf6675740 1 -- 172.21.15.45:0/1549439493 wait complete. 2024-05-08T05:44:06.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.838+0000 7feaf6675740 20 client.5886 trim_cache size 0 max 16384 2024-05-08T05:44:06.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.856+0000 7feaf6675740 1 Processor -- start 2024-05-08T05:44:06.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.856+0000 7feaf6675740 1 -- start start 2024-05-08T05:44:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.857+0000 7feaf6675740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1718280 0x5606f1730940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.857+0000 7feaf6675740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1719180 0x5606f1712680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.857+0000 7feaf6675740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1735160 0x5606f1740d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.857+0000 7feaf6675740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feaec201f00 con 0x5606f1718280 2024-05-08T05:44:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.857+0000 7feaf6675740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feaec200f30 con 0x5606f1719180 2024-05-08T05:44:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.857+0000 7feaf6675740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feaec15fd30 con 0x5606f1735160 2024-05-08T05:44:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.857+0000 7feaf16f4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1718280 0x5606f1730940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.857+0000 7feaf16f4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1718280 0x5606f1730940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36952/0 (socket says 172.21.15.45:36952) 2024-05-08T05:44:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.857+0000 7feaf16f4640 1 -- 172.21.15.45:0/1661982069 learned_addr learned my addr 172.21.15.45:0/1661982069 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.857+0000 7feaf1ef5640 1 --2- 172.21.15.45:0/1661982069 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1735160 0x5606f1740d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.857+0000 7feaf0ef3640 1 --2- 172.21.15.45:0/1661982069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1719180 0x5606f1712680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.857+0000 7feaf16f4640 1 -- 172.21.15.45:0/1661982069 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1735160 msgr2=0x5606f1740d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.857+0000 7feaf16f4640 1 --2- 172.21.15.45:0/1661982069 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1735160 0x5606f1740d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.857+0000 7feaf16f4640 1 -- 172.21.15.45:0/1661982069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1719180 msgr2=0x5606f1712680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.857+0000 7feaf16f4640 1 --2- 172.21.15.45:0/1661982069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1719180 0x5606f1712680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.857+0000 7feaf16f4640 1 -- 172.21.15.45:0/1661982069 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606f13230f0 con 0x5606f1718280 2024-05-08T05:44:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.857+0000 7feaf0ef3640 1 --2- 172.21.15.45:0/1661982069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1719180 0x5606f1712680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:44:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.857+0000 7feaf16f4640 1 --2- 172.21.15.45:0/1661982069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1718280 0x5606f1730940 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7feae4010830 tx=0x7feae401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f85cd4614b8cbde server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.857+0000 7fead27fc640 1 -- 172.21.15.45:0/1661982069 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae4039070 con 0x5606f1718280 2024-05-08T05:44:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.857+0000 7fead27fc640 1 -- 172.21.15.45:0/1661982069 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae401d040 con 0x5606f1718280 2024-05-08T05:44:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.858+0000 7fead27fc640 1 -- 172.21.15.45:0/1661982069 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae4026050 con 0x5606f1718280 2024-05-08T05:44:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.858+0000 7feaf6675740 1 -- 172.21.15.45:0/1661982069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1718280 msgr2=0x5606f1730940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.858+0000 7feaf6675740 1 --2- 172.21.15.45:0/1661982069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1718280 0x5606f1730940 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7feae4010830 tx=0x7feae401c000 comp rx=0 tx=0).stop 2024-05-08T05:44:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.858+0000 7feaf6675740 1 -- 172.21.15.45:0/1661982069 shutdown_connections 2024-05-08T05:44:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.858+0000 7feaf6675740 1 --2- 172.21.15.45:0/1661982069 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1735160 0x5606f1740d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.858+0000 7feaf6675740 1 --2- 172.21.15.45:0/1661982069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1719180 0x5606f1712680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.858+0000 7feaf6675740 1 --2- 172.21.15.45:0/1661982069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1718280 0x5606f1730940 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.858+0000 7feaf6675740 1 -- 172.21.15.45:0/1661982069 >> 172.21.15.45:0/1661982069 conn(0x5606f1717b40 msgr2=0x5606f1718770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:06.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.858+0000 7feaf6675740 1 -- 172.21.15.45:0/1661982069 shutdown_connections 2024-05-08T05:44:06.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.858+0000 7feaf6675740 1 -- 172.21.15.45:0/1661982069 wait complete. 2024-05-08T05:44:06.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.859+0000 7feaf6675740 1 -- start start 2024-05-08T05:44:06.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.859+0000 7feaf6675740 1 Processor -- start 2024-05-08T05:44:06.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf6675740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1718280 0x5606f17af990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf6675740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1719180 0x5606f1727f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf6675740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1735160 0x5606f1742730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf6675740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5606f1742170 con 0x5606f1735160 2024-05-08T05:44:06.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf6675740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feaec160380 con 0x5606f1719180 2024-05-08T05:44:06.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf6675740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5606f17294a0 con 0x5606f1718280 2024-05-08T05:44:06.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf6675740 10 client.0 _ll_register_callbacks cb 0x7feae4006dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:44:06.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf1ef5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1735160 0x5606f1742730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf1ef5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1735160 0x5606f1742730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36960/0 (socket says 172.21.15.45:36960) 2024-05-08T05:44:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf1ef5640 1 -- 172.21.15.45:0/3140258861 learned_addr learned my addr 172.21.15.45:0/3140258861 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf16f4640 1 --2- 172.21.15.45:0/3140258861 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1718280 0x5606f17af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf0ef3640 1 --2- 172.21.15.45:0/3140258861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1719180 0x5606f1727f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf1ef5640 1 -- 172.21.15.45:0/3140258861 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1718280 msgr2=0x5606f17af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf1ef5640 1 --2- 172.21.15.45:0/3140258861 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1718280 0x5606f17af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf1ef5640 1 -- 172.21.15.45:0/3140258861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1719180 msgr2=0x5606f1727f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf1ef5640 1 --2- 172.21.15.45:0/3140258861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1719180 0x5606f1727f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf1ef5640 1 -- 172.21.15.45:0/3140258861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606f17321f0 con 0x5606f1735160 2024-05-08T05:44:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf1ef5640 1 --2- 172.21.15.45:0/3140258861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1735160 0x5606f1742730 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7feadc0691d0 tx=0x7feadc067050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a281cadd67814c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feacbfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:44:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feacbfff640 1 -- 172.21.15.45:0/3140258861 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc069470 con 0x5606f1735160 2024-05-08T05:44:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feacbfff640 1 -- 172.21.15.45:0/3140258861 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(61..61 src has 1..61) ==== 6101+0+0 (secure 0 0 0) 0x7feadc006b50 con 0x5606f1735160 2024-05-08T05:44:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf6675740 1 -- 172.21.15.45:0/3140258861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feae401d040 con 0x5606f1735160 2024-05-08T05:44:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feacbfff640 1 -- 172.21.15.45:0/3140258861 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7feadc0604f0 con 0x5606f1735160 2024-05-08T05:44:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf6675740 10 client.5892 fetch_fsmap learned FSMap version 34 2024-05-08T05:44:06.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.860+0000 7feaf6675740 1 -- 172.21.15.45:0/3140258861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5606f1732f90 con 0x5606f1735160 2024-05-08T05:44:06.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.861+0000 7feacbfff640 1 -- 172.21.15.45:0/3140258861 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae401d040 con 0x5606f1735160 2024-05-08T05:44:06.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.861+0000 7feacbfff640 1 -- 172.21.15.45:0/3140258861 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7feadc05fc40 con 0x5606f1735160 2024-05-08T05:44:06.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.861+0000 7feaf6675740 10 client.5892 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T05:44:06.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.861+0000 7feaf6675740 10 client.5892 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T05:44:06.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.861+0000 7feaf6675740 20 client.5892 populate_metadata read hostname 'smithi045' 2024-05-08T05:44:06.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.861+0000 7feaf6675740 1 --2- 172.21.15.45:0/3140258861 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5606f16e6a30 0x5606f1742c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:06.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.861+0000 7feaf6675740 4 client.5892 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:44:06.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.861+0000 7feaf6675740 1 -- 172.21.15.45:0/3140258861 --> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5606f16599b0 con 0x5606f16e6a30 2024-05-08T05:44:06.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.861+0000 7feaf16f4640 1 --2- 172.21.15.45:0/3140258861 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5606f16e6a30 0x5606f1742c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:06.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.861+0000 7feacbfff640 10 client.5892 ms_handle_connect on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:06.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.861+0000 7feaf16f4640 1 --2- 172.21.15.45:0/3140258861 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5606f16e6a30 0x5606f1742c70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abe7ca814da349cf server_cookie=f103400cd4baf41d in_seq=0 out_seq=0 2024-05-08T05:44:06.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.861+0000 7feacbfff640 1 -- 172.21.15.45:0/3140258861 <== mds.0 v2:172.21.15.45:6833/1461822016 1 ==== command_reply(tid 0: 0 ) ==== 8+0+253 (crc 0 0 0) 0x5606f16599b0 con 0x5606f16e6a30 2024-05-08T05:44:06.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.861+0000 7feacbfff640 10 client.5892 handle_command_reply: tid=0 2024-05-08T05:44:06.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.861+0000 7feaf6675740 1 client.5892 shutdown 2024-05-08T05:44:06.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf6675740 1 -- 172.21.15.45:0/3140258861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1735160 msgr2=0x5606f1742730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf6675740 1 --2- 172.21.15.45:0/3140258861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1735160 0x5606f1742730 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7feadc0691d0 tx=0x7feadc067050 comp rx=0 tx=0).stop 2024-05-08T05:44:06.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf6675740 1 -- 172.21.15.45:0/3140258861 shutdown_connections 2024-05-08T05:44:06.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf6675740 1 --2- 172.21.15.45:0/3140258861 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5606f16e6a30 0x5606f1742c70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf6675740 1 --2- 172.21.15.45:0/3140258861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5606f1735160 0x5606f1742730 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feacbfff640 0 client.5892 ms_handle_reset on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:06.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf6675740 1 --2- 172.21.15.45:0/3140258861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5606f1719180 0x5606f1727f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf6675740 1 --2- 172.21.15.45:0/3140258861 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5606f1718280 0x5606f17af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf6675740 1 -- 172.21.15.45:0/3140258861 >> 172.21.15.45:0/3140258861 conn(0x5606f1717b40 msgr2=0x5606f1712680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:06.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf6675740 1 -- 172.21.15.45:0/3140258861 shutdown_connections 2024-05-08T05:44:06.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf6675740 1 -- 172.21.15.45:0/3140258861 wait complete. 2024-05-08T05:44:06.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf6675740 20 client.5892 trim_cache size 0 max 16384 2024-05-08T05:44:06.868 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:44:06.868 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T05:44:06.868 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-05-08T05:44:06.868 INFO:teuthology.orchestra.run.smithi045.stdout: "53b2e38d-5042-45be-937f-1c1d8568f686": { 2024-05-08T05:44:06.868 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-05-08T05:44:06.868 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "53b2e38d-5042-45be-937f-1c1d8568f686", 2024-05-08T05:44:06.868 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive" 2024-05-08T05:44:06.868 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:44:06.868 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:44:06.868 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:44:06.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf3958640 1 -- 172.21.15.45:0/2756335414 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fead80762d0 msgr2=0x7fead8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf3958640 1 --2- 172.21.15.45:0/2756335414 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fead80762d0 0x7fead8078790 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feae40079a0 tx=0x7feae4008040 comp rx=0 tx=0).stop 2024-05-08T05:44:06.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf3958640 1 -- 172.21.15.45:0/2756335414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feaec0cb3b0 msgr2=0x7feaec160de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:06.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf3958640 1 --2- 172.21.15.45:0/2756335414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feaec0cb3b0 0x7feaec160de0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7feadc00c780 tx=0x7feadc00cc40 comp rx=0 tx=0).stop 2024-05-08T05:44:06.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf3958640 1 -- 172.21.15.45:0/2756335414 shutdown_connections 2024-05-08T05:44:06.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf3958640 1 --2- 172.21.15.45:0/2756335414 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fead80762d0 0x7fead8078790 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf3958640 1 --2- 172.21.15.45:0/2756335414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feaec0cb3b0 0x7feaec160de0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf3958640 1 --2- 172.21.15.45:0/2756335414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feaec0c8b00 0x7feaec0ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf3958640 1 --2- 172.21.15.45:0/2756335414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feaec0d1eb0 0x7feaec15d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:06.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.862+0000 7feaf3958640 1 -- 172.21.15.45:0/2756335414 >> 172.21.15.45:0/2756335414 conn(0x7feaec08eac0 msgr2=0x7feaec15e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:06.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.863+0000 7feaf3958640 1 -- 172.21.15.45:0/2756335414 shutdown_connections 2024-05-08T05:44:06.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:06.863+0000 7feaf3958640 1 -- 172.21.15.45:0/2756335414 wait complete. 2024-05-08T05:44:06.874 INFO:tasks.cephfs.filesystem:scrub status for tag:53b2e38d-5042-45be-937f-1c1d8568f686 - {'path': '/', 'tag': '53b2e38d-5042-45be-937f-1c1d8568f686', 'options': 'recursive'} 2024-05-08T05:44:10.923 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-05-08T05:44:10.922+0000 7f1250617640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-08T05:44:34.423 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:44:34.427 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:44:36.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:44:36.875 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-08T05:44:36.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.994+0000 7f0842f58640 1 Processor -- start 2024-05-08T05:44:36.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.994+0000 7f0842f58640 1 -- start start 2024-05-08T05:44:36.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.995+0000 7f0842f58640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f083c06fc30 0x7f083c0af740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:36.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.995+0000 7f0842f58640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f083c0b48a0 0x7f083c0afc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:36.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.995+0000 7f0842f58640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f083c0701d0 con 0x7f083c157540 2024-05-08T05:44:36.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.995+0000 7f0842f58640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f083c0b01c0 con 0x7f083c06fc30 2024-05-08T05:44:36.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.995+0000 7f0842f58640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f083c0b3dc0 con 0x7f083c0b48a0 2024-05-08T05:44:36.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.995+0000 7f08414f5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f083c0b48a0 0x7f083c0afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:36.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.995+0000 7f0833fff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f083c06fc30 0x7f083c0af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:36.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.995+0000 7f0840cf4640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f083c157540 0x7f083c15e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:43826/0 (socket says 172.21.15.45:43826) 2024-05-08T05:44:36.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.995+0000 7f0840cf4640 1 -- 172.21.15.45:0/461762371 learned_addr learned my addr 172.21.15.45:0/461762371 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:44:36.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.996+0000 7f08337fe640 1 -- 172.21.15.45:0/461762371 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3323153228 0 0) 0x7f083c0701d0 con 0x7f083c157540 2024-05-08T05:44:36.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.996+0000 7f08337fe640 1 -- 172.21.15.45:0/461762371 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0824003520 con 0x7f083c157540 2024-05-08T05:44:36.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.996+0000 7f0833fff640 1 -- 172.21.15.45:0/461762371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f083c0b48a0 msgr2=0x7f083c0afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:36.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.996+0000 7f0833fff640 1 --2- 172.21.15.45:0/461762371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f083c0b48a0 0x7f083c0afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:36.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.996+0000 7f0833fff640 1 -- 172.21.15.45:0/461762371 >> v1:172.21.15.63:6789/0 conn(0x7f083c157540 legacy=0x7f083c15e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:36.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.996+0000 7f0833fff640 1 -- 172.21.15.45:0/461762371 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c0b40a0 con 0x7f083c06fc30 2024-05-08T05:44:36.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.996+0000 7f08414f5640 1 --2- 172.21.15.45:0/461762371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f083c0b48a0 0x7f083c0afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:44:36.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.997+0000 7f0833fff640 1 --2- 172.21.15.45:0/461762371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f083c06fc30 0x7f083c0af740 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f082c0097a0 tx=0x7f082c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=759026873f643a81 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:36.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.997+0000 7f08337fe640 1 -- 172.21.15.45:0/461762371 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c037070 con 0x7f083c06fc30 2024-05-08T05:44:36.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.997+0000 7f08337fe640 1 -- 172.21.15.45:0/461762371 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f082c032440 con 0x7f083c06fc30 2024-05-08T05:44:36.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.997+0000 7f08337fe640 1 -- 172.21.15.45:0/461762371 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c0327f0 con 0x7f083c06fc30 2024-05-08T05:44:36.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.997+0000 7f0842f58640 1 -- 172.21.15.45:0/461762371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f083c06fc30 msgr2=0x7f083c0af740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:36.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.997+0000 7f0842f58640 1 --2- 172.21.15.45:0/461762371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f083c06fc30 0x7f083c0af740 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f082c0097a0 tx=0x7f082c02a180 comp rx=0 tx=0).stop 2024-05-08T05:44:36.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.997+0000 7f0842f58640 1 -- 172.21.15.45:0/461762371 shutdown_connections 2024-05-08T05:44:36.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.997+0000 7f0842f58640 1 --2- 172.21.15.45:0/461762371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f083c0b48a0 0x7f083c0afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:36.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.997+0000 7f0842f58640 1 --2- 172.21.15.45:0/461762371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f083c06fc30 0x7f083c0af740 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:36.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.997+0000 7f0842f58640 1 -- 172.21.15.45:0/461762371 >> 172.21.15.45:0/461762371 conn(0x7f083c06c3a0 msgr2=0x7f083c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:36.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.997+0000 7f0842f58640 1 -- 172.21.15.45:0/461762371 shutdown_connections 2024-05-08T05:44:36.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.997+0000 7f0842f58640 1 -- 172.21.15.45:0/461762371 wait complete. 2024-05-08T05:44:37.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.999+0000 7f0842f58640 1 Processor -- start 2024-05-08T05:44:37.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:36.999+0000 7f0842f58640 1 -- start start 2024-05-08T05:44:37.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.000+0000 7f0842f58640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f083c06fc30 0x7f083c1f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.000+0000 7f0842f58640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f083c0b48a0 0x7f083c1557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.000+0000 7f0842f58640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f083c157540 0x7f083c15d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.000+0000 7f0842f58640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f083c157aa0 con 0x7f083c0b48a0 2024-05-08T05:44:37.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.000+0000 7f0842f58640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f083c157920 con 0x7f083c06fc30 2024-05-08T05:44:37.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.000+0000 7f0842f58640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f083c0ced20 con 0x7f083c157540 2024-05-08T05:44:37.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.000+0000 7f0833fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f083c0b48a0 0x7f083c1557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.000+0000 7f0840cf4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f083c06fc30 0x7f083c1f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.000+0000 7f0833fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f083c0b48a0 0x7f083c1557a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55442/0 (socket says 172.21.15.45:55442) 2024-05-08T05:44:37.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.000+0000 7f0833fff640 1 -- 172.21.15.45:0/3400251602 learned_addr learned my addr 172.21.15.45:0/3400251602 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:37.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.000+0000 7f08414f5640 1 --2- 172.21.15.45:0/3400251602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f083c157540 0x7f083c15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.000+0000 7f0833fff640 1 -- 172.21.15.45:0/3400251602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f083c157540 msgr2=0x7f083c15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.000+0000 7f0833fff640 1 --2- 172.21.15.45:0/3400251602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f083c157540 0x7f083c15d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.000+0000 7f0833fff640 1 -- 172.21.15.45:0/3400251602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f083c06fc30 msgr2=0x7f083c1f39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.000+0000 7f0833fff640 1 --2- 172.21.15.45:0/3400251602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f083c06fc30 0x7f083c1f39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.000+0000 7f0833fff640 1 -- 172.21.15.45:0/3400251602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c1f3f10 con 0x7f083c0b48a0 2024-05-08T05:44:37.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.000+0000 7f0833fff640 1 --2- 172.21.15.45:0/3400251602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f083c0b48a0 0x7f083c1557a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f082c0097a0 tx=0x7f082c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:37.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.001+0000 7f08317fa640 1 -- 172.21.15.45:0/3400251602 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c037050 con 0x7f083c0b48a0 2024-05-08T05:44:37.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.001+0000 7f08317fa640 1 -- 172.21.15.45:0/3400251602 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f082c038040 con 0x7f083c0b48a0 2024-05-08T05:44:37.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.001+0000 7f0842f58640 1 -- 172.21.15.45:0/3400251602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083c155ce0 con 0x7f083c0b48a0 2024-05-08T05:44:37.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.001+0000 7f08317fa640 1 -- 172.21.15.45:0/3400251602 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c004490 con 0x7f083c0b48a0 2024-05-08T05:44:37.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.001+0000 7f0842f58640 1 -- 172.21.15.45:0/3400251602 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f083c159450 con 0x7f083c0b48a0 2024-05-08T05:44:37.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.002+0000 7f08317fa640 1 -- 172.21.15.45:0/3400251602 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f082c003610 con 0x7f083c0b48a0 2024-05-08T05:44:37.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.003+0000 7f08317fa640 1 --2- 172.21.15.45:0/3400251602 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f08280762d0 0x7f0828078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.003+0000 7f08317fa640 1 -- 172.21.15.45:0/3400251602 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f082c0b80f0 con 0x7f083c0b48a0 2024-05-08T05:44:37.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.003+0000 7f0840cf4640 1 --2- 172.21.15.45:0/3400251602 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f08280762d0 0x7f0828078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.003+0000 7f0845c73740 1 Processor -- start 2024-05-08T05:44:37.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.004+0000 7f0845c73740 1 -- start start 2024-05-08T05:44:37.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.004+0000 7f0840cf4640 1 --2- 172.21.15.45:0/3400251602 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f08280762d0 0x7f0828078790 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0838005c00 tx=0x7f0838009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:37.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.004+0000 7f0845c73740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338da2450 0x55f338da6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.004+0000 7f0845c73740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d7a160 0x55f338da2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.004+0000 7f0845c73740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2d70 0x55f338da5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.004+0000 7f0845c73740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f338d42690 con 0x55f338d7a160 2024-05-08T05:44:37.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.004+0000 7f0845c73740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f338cdff30 con 0x55f338da2450 2024-05-08T05:44:37.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.004+0000 7f0845c73740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f338c98b20 con 0x55f338da2d70 2024-05-08T05:44:37.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.004+0000 7f0833fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d7a160 0x55f338da2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.004+0000 7f0833fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d7a160 0x55f338da2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55448/0 (socket says 172.21.15.45:55448) 2024-05-08T05:44:37.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.004+0000 7f0833fff640 1 -- 172.21.15.45:0/3120268990 learned_addr learned my addr 172.21.15.45:0/3120268990 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:37.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.004+0000 7f0840cf4640 1 --2- 172.21.15.45:0/3120268990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338da2450 0x55f338da6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.004+0000 7f08414f5640 1 --2- 172.21.15.45:0/3120268990 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2d70 0x55f338da5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.005+0000 7f0833fff640 1 -- 172.21.15.45:0/3120268990 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2d70 msgr2=0x55f338da5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:37.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.005+0000 7f0833fff640 1 --2- 172.21.15.45:0/3120268990 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2d70 0x55f338da5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.005+0000 7f0833fff640 1 -- 172.21.15.45:0/3120268990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338da2450 msgr2=0x55f338da6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:37.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.005+0000 7f0833fff640 1 --2- 172.21.15.45:0/3120268990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338da2450 0x55f338da6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.005+0000 7f0833fff640 1 -- 172.21.15.45:0/3120268990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f338c90e60 con 0x55f338d7a160 2024-05-08T05:44:37.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.005+0000 7f0840cf4640 1 --2- 172.21.15.45:0/3120268990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338da2450 0x55f338da6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:44:37.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.005+0000 7f0833fff640 1 --2- 172.21.15.45:0/3120268990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d7a160 0x55f338da2830 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f082c0b62d0 tx=0x7f082c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6ac199edf01f11 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:37.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.005+0000 7f08227fc640 1 -- 172.21.15.45:0/3120268990 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c081790 con 0x55f338d7a160 2024-05-08T05:44:37.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.005+0000 7f08227fc640 1 -- 172.21.15.45:0/3120268990 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f082c081090 con 0x55f338d7a160 2024-05-08T05:44:37.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.005+0000 7f08227fc640 1 -- 172.21.15.45:0/3120268990 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c085440 con 0x55f338d7a160 2024-05-08T05:44:37.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.005+0000 7f0845c73740 1 -- 172.21.15.45:0/3120268990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d7a160 msgr2=0x55f338da2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.005+0000 7f0845c73740 1 --2- 172.21.15.45:0/3120268990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d7a160 0x55f338da2830 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f082c0b62d0 tx=0x7f082c009190 comp rx=0 tx=0).stop 2024-05-08T05:44:37.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.006+0000 7f0845c73740 1 -- 172.21.15.45:0/3120268990 shutdown_connections 2024-05-08T05:44:37.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.006+0000 7f0845c73740 1 --2- 172.21.15.45:0/3120268990 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2d70 0x55f338da5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.006+0000 7f0845c73740 1 --2- 172.21.15.45:0/3120268990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d7a160 0x55f338da2830 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.006+0000 7f0845c73740 1 --2- 172.21.15.45:0/3120268990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338da2450 0x55f338da6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.006+0000 7f0845c73740 1 -- 172.21.15.45:0/3120268990 >> 172.21.15.45:0/3120268990 conn(0x55f338d77d60 msgr2=0x55f338dc0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:37.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.006+0000 7f0845c73740 1 -- 172.21.15.45:0/3120268990 shutdown_connections 2024-05-08T05:44:37.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.006+0000 7f0845c73740 1 -- 172.21.15.45:0/3120268990 wait complete. 2024-05-08T05:44:37.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.006+0000 7f0845c73740 1 -- start start 2024-05-08T05:44:37.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.006+0000 7f0845c73740 1 Processor -- start 2024-05-08T05:44:37.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.007+0000 7f0845c73740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338da2450 0x55f338dc2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.007+0000 7f0845c73740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2d70 0x55f338dc2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.007+0000 7f0845c73740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d9cdd0 0x55f338da6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.007+0000 7f0845c73740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f338c9c5e0 con 0x55f338d9cdd0 2024-05-08T05:44:37.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.007+0000 7f0845c73740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f338cdbef0 con 0x55f338da2450 2024-05-08T05:44:37.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.007+0000 7f0845c73740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f338d42690 con 0x55f338da2d70 2024-05-08T05:44:37.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.007+0000 7f0845c73740 10 client.0 _ll_register_callbacks cb 0x55f338c97320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:44:37.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.007+0000 7f08414f5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d9cdd0 0x55f338da6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.007+0000 7f08414f5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d9cdd0 0x55f338da6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55458/0 (socket says 172.21.15.45:55458) 2024-05-08T05:44:37.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.007+0000 7f08414f5640 1 -- 172.21.15.45:0/2848461279 learned_addr learned my addr 172.21.15.45:0/2848461279 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:37.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.007+0000 7f0833fff640 1 --2- 172.21.15.45:0/2848461279 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2d70 0x55f338dc2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.007+0000 7f0840cf4640 1 --2- 172.21.15.45:0/2848461279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338da2450 0x55f338dc2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.008+0000 7f08414f5640 1 -- 172.21.15.45:0/2848461279 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2d70 msgr2=0x55f338dc2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:37.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.008+0000 7f08414f5640 1 --2- 172.21.15.45:0/2848461279 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2d70 0x55f338dc2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.008+0000 7f08414f5640 1 -- 172.21.15.45:0/2848461279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338da2450 msgr2=0x55f338dc2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:37.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.008+0000 7f08414f5640 1 --2- 172.21.15.45:0/2848461279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338da2450 0x55f338dc2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.008+0000 7f08414f5640 1 -- 172.21.15.45:0/2848461279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f338abb7d0 con 0x55f338d9cdd0 2024-05-08T05:44:37.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.008+0000 7f0833fff640 1 --2- 172.21.15.45:0/2848461279 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2d70 0x55f338dc2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:44:37.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.008+0000 7f08414f5640 1 --2- 172.21.15.45:0/2848461279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d9cdd0 0x55f338da6720 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f08340146a0 tx=0x7f0834014b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b731766ac3e7c508 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:37.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.008+0000 7f081b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:44:37.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.008+0000 7f081b7fe640 1 -- 172.21.15.45:0/2848461279 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834014e70 con 0x55f338d9cdd0 2024-05-08T05:44:37.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.008+0000 7f081b7fe640 1 -- 172.21.15.45:0/2848461279 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f0834004240 con 0x55f338d9cdd0 2024-05-08T05:44:37.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.008+0000 7f0845c73740 1 -- 172.21.15.45:0/2848461279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f082c081090 con 0x55f338d9cdd0 2024-05-08T05:44:37.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.008+0000 7f081b7fe640 1 -- 172.21.15.45:0/2848461279 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0834004440 con 0x55f338d9cdd0 2024-05-08T05:44:37.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.008+0000 7f0845c73740 10 client.5901 fetch_fsmap learned FSMap version 35 2024-05-08T05:44:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.008+0000 7f0845c73740 1 -- 172.21.15.45:0/2848461279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55f3389f13f0 con 0x55f338d9cdd0 2024-05-08T05:44:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.008+0000 7f081b7fe640 1 -- 172.21.15.45:0/2848461279 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f082c081090 con 0x55f338d9cdd0 2024-05-08T05:44:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.008+0000 7f081b7fe640 1 -- 172.21.15.45:0/2848461279 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f083402a080 con 0x55f338d9cdd0 2024-05-08T05:44:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.008+0000 7f0845c73740 10 client.5901 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T05:44:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.008+0000 7f0845c73740 10 client.5901 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T05:44:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.008+0000 7f0845c73740 20 client.5901 populate_metadata read hostname 'smithi045' 2024-05-08T05:44:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.008+0000 7f0845c73740 1 --2- 172.21.15.45:0/2848461279 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55f338da4da0 0x55f338da5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.009+0000 7f0845c73740 4 client.5901 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:44:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.009+0000 7f0845c73740 1 -- 172.21.15.45:0/2848461279 --> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f338cedb40 con 0x55f338da4da0 2024-05-08T05:44:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.009+0000 7f0840cf4640 1 --2- 172.21.15.45:0/2848461279 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55f338da4da0 0x55f338da5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.009+0000 7f0840cf4640 1 --2- 172.21.15.45:0/2848461279 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55f338da4da0 0x55f338da5200 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30aa78bccd1052da server_cookie=a46599b083067457 in_seq=0 out_seq=0 2024-05-08T05:44:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.009+0000 7f081b7fe640 10 client.5901 ms_handle_connect on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.010+0000 7f081b7fe640 1 -- 172.21.15.45:0/2848461279 <== mds.0 v2:172.21.15.45:6833/1461822016 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f338cedb40 con 0x55f338da4da0 2024-05-08T05:44:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.010+0000 7f081b7fe640 10 client.5901 handle_command_reply: tid=0 2024-05-08T05:44:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.010+0000 7f0845c73740 1 client.5901 shutdown 2024-05-08T05:44:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.010+0000 7f0845c73740 1 -- 172.21.15.45:0/2848461279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d9cdd0 msgr2=0x55f338da6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.010+0000 7f0845c73740 1 --2- 172.21.15.45:0/2848461279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d9cdd0 0x55f338da6720 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f08340146a0 tx=0x7f0834014b60 comp rx=0 tx=0).stop 2024-05-08T05:44:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.010+0000 7f0845c73740 1 -- 172.21.15.45:0/2848461279 shutdown_connections 2024-05-08T05:44:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.010+0000 7f0845c73740 1 --2- 172.21.15.45:0/2848461279 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55f338da4da0 0x55f338da5200 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.010+0000 7f0845c73740 1 --2- 172.21.15.45:0/2848461279 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2d70 0x55f338dc2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.010+0000 7f0845c73740 1 --2- 172.21.15.45:0/2848461279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d9cdd0 0x55f338da6720 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.010+0000 7f0845c73740 1 --2- 172.21.15.45:0/2848461279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338da2450 0x55f338dc2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.010+0000 7f081b7fe640 0 client.5901 ms_handle_reset on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.010+0000 7f0845c73740 1 -- 172.21.15.45:0/2848461279 >> 172.21.15.45:0/2848461279 conn(0x55f338d7a160 msgr2=0x55f338d749a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.010+0000 7f0845c73740 1 -- 172.21.15.45:0/2848461279 shutdown_connections 2024-05-08T05:44:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.010+0000 7f0845c73740 1 -- 172.21.15.45:0/2848461279 wait complete. 2024-05-08T05:44:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.010+0000 7f0845c73740 20 client.5901 trim_cache size 0 max 16384 2024-05-08T05:44:37.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.029+0000 7f0845c73740 1 Processor -- start 2024-05-08T05:44:37.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.029+0000 7f0845c73740 1 -- start start 2024-05-08T05:44:37.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.029+0000 7f0845c73740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d9cdd0 0x55f338dc4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.029+0000 7f0845c73740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2450 0x55f338d749a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.029+0000 7f0845c73740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338da2d70 0x55f338d74ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.029+0000 7f0845c73740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f083c1fdb10 con 0x55f338d9cdd0 2024-05-08T05:44:37.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.029+0000 7f0845c73740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f083c073fe0 con 0x55f338da2d70 2024-05-08T05:44:37.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.029+0000 7f0845c73740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f083c0732c0 con 0x55f338da2450 2024-05-08T05:44:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.029+0000 7f0840cf4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d9cdd0 0x55f338dc4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.030+0000 7f0840cf4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d9cdd0 0x55f338dc4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55472/0 (socket says 172.21.15.45:55472) 2024-05-08T05:44:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.030+0000 7f0840cf4640 1 -- 172.21.15.45:0/4177370024 learned_addr learned my addr 172.21.15.45:0/4177370024 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.030+0000 7f0833fff640 1 --2- 172.21.15.45:0/4177370024 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2450 0x55f338d749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.030+0000 7f08414f5640 1 --2- 172.21.15.45:0/4177370024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338da2d70 0x55f338d74ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.030+0000 7f0840cf4640 1 -- 172.21.15.45:0/4177370024 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2450 msgr2=0x55f338d749a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.030+0000 7f0840cf4640 1 --2- 172.21.15.45:0/4177370024 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2450 0x55f338d749a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.030+0000 7f0840cf4640 1 -- 172.21.15.45:0/4177370024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338da2d70 msgr2=0x55f338d74ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.030+0000 7f0840cf4640 1 --2- 172.21.15.45:0/4177370024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338da2d70 0x55f338d74ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.030+0000 7f0840cf4640 1 -- 172.21.15.45:0/4177370024 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3389b10f0 con 0x55f338d9cdd0 2024-05-08T05:44:37.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.030+0000 7f0840cf4640 1 --2- 172.21.15.45:0/4177370024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d9cdd0 0x55f338dc4380 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f083c0cfc70 tx=0x7f0838012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171c0605052ea5cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:37.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.030+0000 7f0821ffb640 1 -- 172.21.15.45:0/4177370024 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838028290 con 0x55f338d9cdd0 2024-05-08T05:44:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.030+0000 7f0821ffb640 1 -- 172.21.15.45:0/4177370024 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f083801b040 con 0x55f338d9cdd0 2024-05-08T05:44:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.031+0000 7f0821ffb640 1 -- 172.21.15.45:0/4177370024 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838028630 con 0x55f338d9cdd0 2024-05-08T05:44:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.031+0000 7f0845c73740 1 -- 172.21.15.45:0/4177370024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d9cdd0 msgr2=0x55f338dc4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.031+0000 7f0845c73740 1 --2- 172.21.15.45:0/4177370024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d9cdd0 0x55f338dc4380 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f083c0cfc70 tx=0x7f0838012000 comp rx=0 tx=0).stop 2024-05-08T05:44:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.031+0000 7f0845c73740 1 -- 172.21.15.45:0/4177370024 shutdown_connections 2024-05-08T05:44:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.031+0000 7f0845c73740 1 --2- 172.21.15.45:0/4177370024 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2450 0x55f338d749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.031+0000 7f0845c73740 1 --2- 172.21.15.45:0/4177370024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338da2d70 0x55f338d74ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.031+0000 7f0845c73740 1 --2- 172.21.15.45:0/4177370024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338d9cdd0 0x55f338dc4380 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.031+0000 7f0845c73740 1 -- 172.21.15.45:0/4177370024 >> 172.21.15.45:0/4177370024 conn(0x55f338d7a160 msgr2=0x55f338da5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.031+0000 7f0845c73740 1 -- 172.21.15.45:0/4177370024 shutdown_connections 2024-05-08T05:44:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.031+0000 7f0845c73740 1 -- 172.21.15.45:0/4177370024 wait complete. 2024-05-08T05:44:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.032+0000 7f0845c73740 1 -- start start 2024-05-08T05:44:37.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.032+0000 7f0845c73740 1 Processor -- start 2024-05-08T05:44:37.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.033+0000 7f0845c73740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338d9cdd0 0x55f338d994b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.033+0000 7f0845c73740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2450 0x55f338d749a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.033+0000 7f0845c73740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338da2d70 0x55f338d74ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.033+0000 7f0845c73740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f083c073910 con 0x55f338da2d70 2024-05-08T05:44:37.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.033+0000 7f0845c73740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f338da4690 con 0x55f338d9cdd0 2024-05-08T05:44:37.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.033+0000 7f0845c73740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f338da60b0 con 0x55f338da2450 2024-05-08T05:44:37.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.033+0000 7f0845c73740 10 client.0 _ll_register_callbacks cb 0x7f08340094e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:44:37.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.033+0000 7f08414f5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338da2d70 0x55f338d74ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.033+0000 7f08414f5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338da2d70 0x55f338d74ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55486/0 (socket says 172.21.15.45:55486) 2024-05-08T05:44:37.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.033+0000 7f08414f5640 1 -- 172.21.15.45:0/1338218252 learned_addr learned my addr 172.21.15.45:0/1338218252 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:37.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.033+0000 7f0833fff640 1 --2- 172.21.15.45:0/1338218252 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2450 0x55f338d749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.033+0000 7f0840cf4640 1 --2- 172.21.15.45:0/1338218252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338d9cdd0 0x55f338d994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.033+0000 7f08414f5640 1 -- 172.21.15.45:0/1338218252 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2450 msgr2=0x55f338d749a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:37.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.033+0000 7f08414f5640 1 --2- 172.21.15.45:0/1338218252 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2450 0x55f338d749a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.033+0000 7f08414f5640 1 -- 172.21.15.45:0/1338218252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338d9cdd0 msgr2=0x55f338d994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:37.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.033+0000 7f08414f5640 1 --2- 172.21.15.45:0/1338218252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338d9cdd0 0x55f338d994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.033+0000 7f08414f5640 1 -- 172.21.15.45:0/1338218252 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f338d9a3f0 con 0x55f338da2d70 2024-05-08T05:44:37.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.033+0000 7f08414f5640 1 --2- 172.21.15.45:0/1338218252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338da2d70 0x55f338d74ee0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f08340146a0 tx=0x7f08340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4619a62e6374e1cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:37.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.033+0000 7f081b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:44:37.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.034+0000 7f081b7fe640 1 -- 172.21.15.45:0/1338218252 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834000e60 con 0x55f338da2d70 2024-05-08T05:44:37.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.034+0000 7f081b7fe640 1 -- 172.21.15.45:0/1338218252 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f0834024a40 con 0x55f338da2d70 2024-05-08T05:44:37.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.034+0000 7f0845c73740 1 -- 172.21.15.45:0/1338218252 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f083801b040 con 0x55f338da2d70 2024-05-08T05:44:37.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.034+0000 7f081b7fe640 1 -- 172.21.15.45:0/1338218252 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f08340126a0 con 0x55f338da2d70 2024-05-08T05:44:37.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.034+0000 7f0845c73740 10 client.5907 fetch_fsmap learned FSMap version 35 2024-05-08T05:44:37.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.034+0000 7f0845c73740 1 -- 172.21.15.45:0/1338218252 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55f338d9a110 con 0x55f338da2d70 2024-05-08T05:44:37.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.034+0000 7f081b7fe640 1 -- 172.21.15.45:0/1338218252 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f083801b040 con 0x55f338da2d70 2024-05-08T05:44:37.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.034+0000 7f081b7fe640 1 -- 172.21.15.45:0/1338218252 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f0834024c40 con 0x55f338da2d70 2024-05-08T05:44:37.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.034+0000 7f0845c73740 10 client.5907 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T05:44:37.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.034+0000 7f0845c73740 10 client.5907 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T05:44:37.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.034+0000 7f0845c73740 20 client.5907 populate_metadata read hostname 'smithi045' 2024-05-08T05:44:37.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.034+0000 7f0845c73740 1 --2- 172.21.15.45:0/1338218252 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55f338dc03d0 0x55f338d9d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.034+0000 7f0845c73740 4 client.5907 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:44:37.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.034+0000 7f0845c73740 1 -- 172.21.15.45:0/1338218252 --> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f338ce79b0 con 0x55f338dc03d0 2024-05-08T05:44:37.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.034+0000 7f0840cf4640 1 --2- 172.21.15.45:0/1338218252 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55f338dc03d0 0x55f338d9d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.035+0000 7f081b7fe640 10 client.5907 ms_handle_connect on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:37.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.035+0000 7f0840cf4640 1 --2- 172.21.15.45:0/1338218252 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55f338dc03d0 0x55f338d9d260 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=359fd8efcc383fae server_cookie=a9a42a848b38d300 in_seq=0 out_seq=0 2024-05-08T05:44:37.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.035+0000 7f081b7fe640 1 -- 172.21.15.45:0/1338218252 <== mds.0 v2:172.21.15.45:6833/1461822016 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f338ce79b0 con 0x55f338dc03d0 2024-05-08T05:44:37.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.035+0000 7f081b7fe640 10 client.5907 handle_command_reply: tid=0 2024-05-08T05:44:37.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.035+0000 7f0845c73740 1 client.5907 shutdown 2024-05-08T05:44:37.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.035+0000 7f0845c73740 1 -- 172.21.15.45:0/1338218252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338da2d70 msgr2=0x55f338d74ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.035+0000 7f0845c73740 1 --2- 172.21.15.45:0/1338218252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338da2d70 0x55f338d74ee0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f08340146a0 tx=0x7f08340046c0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.035+0000 7f0845c73740 1 -- 172.21.15.45:0/1338218252 shutdown_connections 2024-05-08T05:44:37.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.035+0000 7f0845c73740 1 --2- 172.21.15.45:0/1338218252 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55f338dc03d0 0x55f338d9d260 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.035+0000 7f0845c73740 1 --2- 172.21.15.45:0/1338218252 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f338da2450 0x55f338d749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.035+0000 7f0845c73740 1 --2- 172.21.15.45:0/1338218252 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f338da2d70 0x55f338d74ee0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.035+0000 7f081b7fe640 0 client.5907 ms_handle_reset on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:37.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.035+0000 7f0845c73740 1 --2- 172.21.15.45:0/1338218252 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f338d9cdd0 0x55f338d994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.035+0000 7f0845c73740 1 -- 172.21.15.45:0/1338218252 >> 172.21.15.45:0/1338218252 conn(0x55f338d7a160 msgr2=0x55f338dc3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:37.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.035+0000 7f0845c73740 1 -- 172.21.15.45:0/1338218252 shutdown_connections 2024-05-08T05:44:37.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.035+0000 7f0845c73740 1 -- 172.21.15.45:0/1338218252 wait complete. 2024-05-08T05:44:37.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.035+0000 7f0845c73740 20 client.5907 trim_cache size 0 max 16384 2024-05-08T05:44:37.041 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:44:37.041 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-05-08T05:44:37.041 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-05-08T05:44:37.041 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:44:37.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.036+0000 7f0842f58640 1 -- 172.21.15.45:0/3400251602 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f08280762d0 msgr2=0x7f0828078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.036+0000 7f0842f58640 1 --2- 172.21.15.45:0/3400251602 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f08280762d0 0x7f0828078790 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0838005c00 tx=0x7f0838009040 comp rx=0 tx=0).stop 2024-05-08T05:44:37.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.036+0000 7f0842f58640 1 -- 172.21.15.45:0/3400251602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f083c0b48a0 msgr2=0x7f083c1557a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.036+0000 7f0842f58640 1 --2- 172.21.15.45:0/3400251602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f083c0b48a0 0x7f083c1557a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f082c0097a0 tx=0x7f082c0046c0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.036+0000 7f0842f58640 1 -- 172.21.15.45:0/3400251602 shutdown_connections 2024-05-08T05:44:37.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.036+0000 7f0842f58640 1 --2- 172.21.15.45:0/3400251602 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f08280762d0 0x7f0828078790 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.036+0000 7f0842f58640 1 --2- 172.21.15.45:0/3400251602 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f083c157540 0x7f083c15d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.036+0000 7f0842f58640 1 --2- 172.21.15.45:0/3400251602 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f083c0b48a0 0x7f083c1557a0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.036+0000 7f0842f58640 1 --2- 172.21.15.45:0/3400251602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f083c06fc30 0x7f083c1f39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.036+0000 7f0842f58640 1 -- 172.21.15.45:0/3400251602 >> 172.21.15.45:0/3400251602 conn(0x7f083c06c3a0 msgr2=0x7f083c0b5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:37.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.036+0000 7f0842f58640 1 -- 172.21.15.45:0/3400251602 shutdown_connections 2024-05-08T05:44:37.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.036+0000 7f0842f58640 1 -- 172.21.15.45:0/3400251602 wait complete. 2024-05-08T05:44:37.047 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T05:44:37.047 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:44:37.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.151+0000 7f5f1af71640 1 Processor -- start 2024-05-08T05:44:37.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.151+0000 7f5f1af71640 1 -- start start 2024-05-08T05:44:37.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.151+0000 7f5f1af71640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5f14162f50 0x7f5f141633e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.151+0000 7f5f1af71640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f140cf790 0x7f5f140cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.151+0000 7f5f1af71640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f140d0500 con 0x7f5f140b21f0 2024-05-08T05:44:37.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.151+0000 7f5f1af71640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f5f1406d810 con 0x7f5f14162f50 2024-05-08T05:44:37.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.151+0000 7f5f1af71640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5f1406e0b0 con 0x7f5f140cf790 2024-05-08T05:44:37.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.152+0000 7f5f13fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5f14162f50 0x7f5f141633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.152+0000 7f5f13fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5f14162f50 0x7f5f141633e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55500/0 (socket says 172.21.15.45:55500) 2024-05-08T05:44:37.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.152+0000 7f5f13fff640 1 -- 172.21.15.45:0/1660356372 learned_addr learned my addr 172.21.15.45:0/1660356372 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:37.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.152+0000 7f5f1950e640 1 --2- 172.21.15.45:0/1660356372 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f140cf790 0x7f5f140cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.152+0000 7f5f13fff640 1 -- 172.21.15.45:0/1660356372 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f140cf790 msgr2=0x7f5f140cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:37.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.152+0000 7f5f13fff640 1 --2- 172.21.15.45:0/1660356372 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f140cf790 0x7f5f140cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.152+0000 7f5f13fff640 1 -- 172.21.15.45:0/1660356372 >> v1:172.21.15.63:6789/0 conn(0x7f5f140b21f0 legacy=0x7f5f1406e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.152+0000 7f5f13fff640 1 -- 172.21.15.45:0/1660356372 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1406da60 con 0x7f5f14162f50 2024-05-08T05:44:37.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.153+0000 7f5f13fff640 1 --2- 172.21.15.45:0/1660356372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f14162f50 0x7f5f141633e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009870 tx=0x7f5efc02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b92c7d914e742f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:37.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.153+0000 7f5f137fe640 1 -- 172.21.15.45:0/1660356372 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc037070 con 0x7f5f14162f50 2024-05-08T05:44:37.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.153+0000 7f5f137fe640 1 -- 172.21.15.45:0/1660356372 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5efc032440 con 0x7f5f14162f50 2024-05-08T05:44:37.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.153+0000 7f5f137fe640 1 -- 172.21.15.45:0/1660356372 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc032810 con 0x7f5f14162f50 2024-05-08T05:44:37.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.153+0000 7f5f1af71640 1 -- 172.21.15.45:0/1660356372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f14162f50 msgr2=0x7f5f141633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.153+0000 7f5f1af71640 1 --2- 172.21.15.45:0/1660356372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f14162f50 0x7f5f141633e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009870 tx=0x7f5efc02a130 comp rx=0 tx=0).stop 2024-05-08T05:44:37.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.153+0000 7f5f1af71640 1 -- 172.21.15.45:0/1660356372 shutdown_connections 2024-05-08T05:44:37.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.153+0000 7f5f1af71640 1 --2- 172.21.15.45:0/1660356372 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f140cf790 0x7f5f140cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.153+0000 7f5f1af71640 1 --2- 172.21.15.45:0/1660356372 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f14162f50 0x7f5f141633e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.153+0000 7f5f1af71640 1 -- 172.21.15.45:0/1660356372 >> 172.21.15.45:0/1660356372 conn(0x7f5f1409ba40 msgr2=0x7f5f1409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:37.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.153+0000 7f5f1af71640 1 -- 172.21.15.45:0/1660356372 shutdown_connections 2024-05-08T05:44:37.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.153+0000 7f5f1af71640 1 -- 172.21.15.45:0/1660356372 wait complete. 2024-05-08T05:44:37.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.155+0000 7f5f1af71640 1 Processor -- start 2024-05-08T05:44:37.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.155+0000 7f5f1af71640 1 -- start start 2024-05-08T05:44:37.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f1af71640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f140b21f0 0x7f5f14156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f1af71640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f140cf790 0x7f5f140c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f1af71640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f14162f50 0x7f5f140c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f1af71640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5f140d0140 con 0x7f5f140b21f0 2024-05-08T05:44:37.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f1af71640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5f140b25d0 con 0x7f5f140cf790 2024-05-08T05:44:37.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f1af71640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5f140d02c0 con 0x7f5f14162f50 2024-05-08T05:44:37.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f18d0d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f140b21f0 0x7f5f14156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f18d0d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f140b21f0 0x7f5f14156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55516/0 (socket says 172.21.15.45:55516) 2024-05-08T05:44:37.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f18d0d640 1 -- 172.21.15.45:0/1911270138 learned_addr learned my addr 172.21.15.45:0/1911270138 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:37.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f13fff640 1 --2- 172.21.15.45:0/1911270138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f140cf790 0x7f5f140c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f1950e640 1 --2- 172.21.15.45:0/1911270138 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f14162f50 0x7f5f140c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f18d0d640 1 -- 172.21.15.45:0/1911270138 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f14162f50 msgr2=0x7f5f140c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f18d0d640 1 --2- 172.21.15.45:0/1911270138 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f14162f50 0x7f5f140c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f18d0d640 1 -- 172.21.15.45:0/1911270138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f140cf790 msgr2=0x7f5f140c67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f18d0d640 1 --2- 172.21.15.45:0/1911270138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f140cf790 0x7f5f140c67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f18d0d640 1 -- 172.21.15.45:0/1911270138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f140b52d0 con 0x7f5f140b21f0 2024-05-08T05:44:37.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f13fff640 1 --2- 172.21.15.45:0/1911270138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f140cf790 0x7f5f140c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:37.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f1950e640 1 --2- 172.21.15.45:0/1911270138 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f14162f50 0x7f5f140c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:37.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f18d0d640 1 --2- 172.21.15.45:0/1911270138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f140b21f0 0x7f5f14156ab0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00c450 tx=0x7f5f0c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:37.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f117fa640 1 -- 172.21.15.45:0/1911270138 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c0052d0 con 0x7f5f140b21f0 2024-05-08T05:44:37.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f117fa640 1 -- 172.21.15.45:0/1911270138 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f0c005470 con 0x7f5f140b21f0 2024-05-08T05:44:37.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f1af71640 1 -- 172.21.15.45:0/1911270138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f140b55b0 con 0x7f5f140b21f0 2024-05-08T05:44:37.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f117fa640 1 -- 172.21.15.45:0/1911270138 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c012870 con 0x7f5f140b21f0 2024-05-08T05:44:37.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.156+0000 7f5f1af71640 1 -- 172.21.15.45:0/1911270138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f140b4fc0 con 0x7f5f140b21f0 2024-05-08T05:44:37.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.157+0000 7f5f1af71640 1 -- 172.21.15.45:0/1911270138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f14069b00 con 0x7f5f140b21f0 2024-05-08T05:44:37.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.158+0000 7f5f117fa640 1 -- 172.21.15.45:0/1911270138 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5f0c012a10 con 0x7f5f140b21f0 2024-05-08T05:44:37.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.159+0000 7f5f117fa640 1 --2- 172.21.15.45:0/1911270138 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5ef40762d0 0x7f5ef4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.159+0000 7f5f117fa640 1 -- 172.21.15.45:0/1911270138 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f5f0c097390 con 0x7f5f140b21f0 2024-05-08T05:44:37.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.159+0000 7f5f13fff640 1 --2- 172.21.15.45:0/1911270138 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5ef40762d0 0x7f5ef4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.159+0000 7f5f13fff640 1 --2- 172.21.15.45:0/1911270138 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5ef40762d0 0x7f5ef4078790 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5efc034000 tx=0x7f5efc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:37.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.163+0000 7f5f117fa640 1 -- 172.21.15.45:0/1911270138 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5f0c060b60 con 0x7f5f140b21f0 2024-05-08T05:44:37.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.392+0000 7f5f1af71640 1 -- 172.21.15.45:0/1911270138 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f14070a30 con 0x7f5f140b21f0 2024-05-08T05:44:37.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.393+0000 7f5f117fa640 1 -- 172.21.15.45:0/1911270138 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5179 (secure 0 0 0) 0x7f5f0c060460 con 0x7f5f140b21f0 2024-05-08T05:44:37.393 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:44:37.393 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/162765657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":162765657},{"type":"v1","addr":"172.21.15.63:6836","nonce":162765657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5356,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1788513331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1788513331},{"type":"v1","addr":"172.21.15.45:6834","nonce":1788513331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1173167340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1173167340},{"type":"v1","addr":"172.21.15.63:6837","nonce":1173167340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:43:58.107819+0000","modified":"2024-05-08T05:44:10.766944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":6,"addr":"172.21.15.45:6835/1461822016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1461822016},{"type":"v1","addr":"172.21.15.45:6835","nonce":1461822016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5697,"qdb_cluster":[5697]},"id":4}]} 2024-05-08T05:44:37.394 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 35 2024-05-08T05:44:37.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.395+0000 7f5f1af71640 1 -- 172.21.15.45:0/1911270138 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5ef40762d0 msgr2=0x7f5ef4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.395+0000 7f5f1af71640 1 --2- 172.21.15.45:0/1911270138 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5ef40762d0 0x7f5ef4078790 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5efc034000 tx=0x7f5efc02e000 comp rx=0 tx=0).stop 2024-05-08T05:44:37.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.395+0000 7f5f1af71640 1 -- 172.21.15.45:0/1911270138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f140b21f0 msgr2=0x7f5f14156ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.395+0000 7f5f1af71640 1 --2- 172.21.15.45:0/1911270138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f140b21f0 0x7f5f14156ab0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c00c450 tx=0x7f5f0c00c910 comp rx=0 tx=0).stop 2024-05-08T05:44:37.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.395+0000 7f5f1af71640 1 -- 172.21.15.45:0/1911270138 shutdown_connections 2024-05-08T05:44:37.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.395+0000 7f5f1af71640 1 --2- 172.21.15.45:0/1911270138 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5ef40762d0 0x7f5ef4078790 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.395+0000 7f5f1af71640 1 --2- 172.21.15.45:0/1911270138 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f14162f50 0x7f5f140c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.395+0000 7f5f1af71640 1 --2- 172.21.15.45:0/1911270138 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f140cf790 0x7f5f140c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.395+0000 7f5f1af71640 1 --2- 172.21.15.45:0/1911270138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f140b21f0 0x7f5f14156ab0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.395+0000 7f5f1af71640 1 -- 172.21.15.45:0/1911270138 >> 172.21.15.45:0/1911270138 conn(0x7f5f1409ba40 msgr2=0x7f5f14163380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:37.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.395+0000 7f5f1af71640 1 -- 172.21.15.45:0/1911270138 shutdown_connections 2024-05-08T05:44:37.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.395+0000 7f5f1af71640 1 -- 172.21.15.45:0/1911270138 wait complete. 2024-05-08T05:44:37.406 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-08T05:44:37.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.512+0000 7f787d8b4640 1 Processor -- start 2024-05-08T05:44:37.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.512+0000 7f787d8b4640 1 -- start start 2024-05-08T05:44:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.512+0000 7f787d8b4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f78781629b0 0x7f787806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.512+0000 7f787d8b4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7878070d20 0x7f78780711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.512+0000 7f787d8b4640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f787806e110 con 0x7f78780b2490 2024-05-08T05:44:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.513+0000 7f787d8b4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7878070a40 con 0x7f78781629b0 2024-05-08T05:44:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.513+0000 7f787d8b4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f787806dec0 con 0x7f7878070d20 2024-05-08T05:44:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.513+0000 7f78777fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7878070d20 0x7f78780711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.513+0000 7f7876ffd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f78781629b0 0x7f787806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.513+0000 7f78767fc640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f78780b2490 0x7f7878070330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:43848/0 (socket says 172.21.15.45:43848) 2024-05-08T05:44:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.513+0000 7f78767fc640 1 -- 172.21.15.45:0/3372965178 learned_addr learned my addr 172.21.15.45:0/3372965178 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:44:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.513+0000 7f78777fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7878070d20 0x7f78780711d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:55716/0 (socket says 172.21.15.45:55716) 2024-05-08T05:44:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.513+0000 7f7875ffb640 1 -- 172.21.15.45:0/3372965178 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1753645564 0 0) 0x7f787806e110 con 0x7f78780b2490 2024-05-08T05:44:37.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.513+0000 7f7875ffb640 1 -- 172.21.15.45:0/3372965178 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7860003520 con 0x7f78780b2490 2024-05-08T05:44:37.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.513+0000 7f78777fe640 1 -- 172.21.15.45:0/3372965178 >> v2:172.21.15.63:3300/0 conn(0x7f78781629b0 msgr2=0x7f787806eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:37.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.513+0000 7f78777fe640 1 --2- 172.21.15.45:0/3372965178 >> v2:172.21.15.63:3300/0 conn(0x7f78781629b0 0x7f787806eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.513+0000 7f78777fe640 1 -- 172.21.15.45:0/3372965178 >> v1:172.21.15.63:6789/0 conn(0x7f78780b2490 legacy=0x7f7878070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.513+0000 7f78777fe640 1 -- 172.21.15.45:0/3372965178 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787806e3f0 con 0x7f7878070d20 2024-05-08T05:44:37.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.513+0000 7f7876ffd640 1 --2- 172.21.15.45:0/3372965178 >> v2:172.21.15.63:3300/0 conn(0x7f78781629b0 0x7f787806eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:44:37.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.514+0000 7f78777fe640 1 --2- 172.21.15.45:0/3372965178 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7878070d20 0x7f78780711d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7870009d90 tx=0x7f787002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fb9a90dad2e70d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:37.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.514+0000 7f7875ffb640 1 -- 172.21.15.45:0/3372965178 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870007740 con 0x7f7878070d20 2024-05-08T05:44:37.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.514+0000 7f7875ffb640 1 -- 172.21.15.45:0/3372965178 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78700078e0 con 0x7f7878070d20 2024-05-08T05:44:37.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.514+0000 7f7875ffb640 1 -- 172.21.15.45:0/3372965178 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870007cb0 con 0x7f7878070d20 2024-05-08T05:44:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.515+0000 7f787d8b4640 1 -- 172.21.15.45:0/3372965178 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7878070d20 msgr2=0x7f78780711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.515+0000 7f787d8b4640 1 --2- 172.21.15.45:0/3372965178 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7878070d20 0x7f78780711d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7870009d90 tx=0x7f787002a660 comp rx=0 tx=0).stop 2024-05-08T05:44:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.515+0000 7f787d8b4640 1 -- 172.21.15.45:0/3372965178 shutdown_connections 2024-05-08T05:44:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.515+0000 7f787d8b4640 1 --2- 172.21.15.45:0/3372965178 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7878070d20 0x7f78780711d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.515+0000 7f787d8b4640 1 --2- 172.21.15.45:0/3372965178 >> v2:172.21.15.63:3300/0 conn(0x7f78781629b0 0x7f787806eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.515+0000 7f787d8b4640 1 -- 172.21.15.45:0/3372965178 >> 172.21.15.45:0/3372965178 conn(0x7f78780b49d0 msgr2=0x7f78780b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.515+0000 7f787d8b4640 1 -- 172.21.15.45:0/3372965178 shutdown_connections 2024-05-08T05:44:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.515+0000 7f787d8b4640 1 -- 172.21.15.45:0/3372965178 wait complete. 2024-05-08T05:44:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.516+0000 7f787d8b4640 1 Processor -- start 2024-05-08T05:44:37.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.516+0000 7f787d8b4640 1 -- start start 2024-05-08T05:44:37.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.517+0000 7f787d8b4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7878070d20 0x7f78781594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.517+0000 7f787d8b4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78780b2490 0x7f78780cece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.517+0000 7f787d8b4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78781629b0 0x7f78780cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.517+0000 7f787d8b4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f787806f410 con 0x7f78781629b0 2024-05-08T05:44:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.517+0000 7f787d8b4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f787806e9b0 con 0x7f78780b2490 2024-05-08T05:44:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.517+0000 7f787d8b4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f78780afcb0 con 0x7f7878070d20 2024-05-08T05:44:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.517+0000 7f78777fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78781629b0 0x7f78780cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.517+0000 7f78777fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78781629b0 0x7f78780cf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55528/0 (socket says 172.21.15.45:55528) 2024-05-08T05:44:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.517+0000 7f78777fe640 1 -- 172.21.15.45:0/1672883443 learned_addr learned my addr 172.21.15.45:0/1672883443 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.517+0000 7f7876ffd640 1 --2- 172.21.15.45:0/1672883443 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7878070d20 0x7f78781594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.517+0000 7f78767fc640 1 --2- 172.21.15.45:0/1672883443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78780b2490 0x7f78780cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.517+0000 7f78777fe640 1 -- 172.21.15.45:0/1672883443 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7878070d20 msgr2=0x7f78781594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.517+0000 7f78777fe640 1 --2- 172.21.15.45:0/1672883443 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7878070d20 0x7f78781594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.517+0000 7f78777fe640 1 -- 172.21.15.45:0/1672883443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78780b2490 msgr2=0x7f78780cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.517+0000 7f78777fe640 1 --2- 172.21.15.45:0/1672883443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78780b2490 0x7f78780cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.518+0000 7f78777fe640 1 -- 172.21.15.45:0/1672883443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78780b5d10 con 0x7f78781629b0 2024-05-08T05:44:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.518+0000 7f7876ffd640 1 --2- 172.21.15.45:0/1672883443 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7878070d20 0x7f78781594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:44:37.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.518+0000 7f78767fc640 1 --2- 172.21.15.45:0/1672883443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78780b2490 0x7f78780cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:44:37.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.518+0000 7f78777fe640 1 --2- 172.21.15.45:0/1672883443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78781629b0 0x7f78780cf9b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7870009d90 tx=0x7f7870004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:37.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.518+0000 7f785ffff640 1 -- 172.21.15.45:0/1672883443 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870040070 con 0x7f78781629b0 2024-05-08T05:44:37.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.518+0000 7f785ffff640 1 -- 172.21.15.45:0/1672883443 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7870031c40 con 0x7f78781629b0 2024-05-08T05:44:37.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.518+0000 7f785ffff640 1 -- 172.21.15.45:0/1672883443 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870009450 con 0x7f78781629b0 2024-05-08T05:44:37.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.518+0000 7f787d8b4640 1 -- 172.21.15.45:0/1672883443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78780b5ff0 con 0x7f78781629b0 2024-05-08T05:44:37.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.518+0000 7f787d8b4640 1 -- 172.21.15.45:0/1672883443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78780b65b0 con 0x7f78781629b0 2024-05-08T05:44:37.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.520+0000 7f785ffff640 1 -- 172.21.15.45:0/1672883443 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7870039040 con 0x7f78781629b0 2024-05-08T05:44:37.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.520+0000 7f785ffff640 1 --2- 172.21.15.45:0/1672883443 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f78640762d0 0x7f7864078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.520+0000 7f785ffff640 1 -- 172.21.15.45:0/1672883443 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f78700b8530 con 0x7f78781629b0 2024-05-08T05:44:37.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.520+0000 7f7876ffd640 1 --2- 172.21.15.45:0/1672883443 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f78640762d0 0x7f7864078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.521+0000 7f78805ce740 1 Processor -- start 2024-05-08T05:44:37.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.521+0000 7f78805ce740 1 -- start start 2024-05-08T05:44:37.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.521+0000 7f7876ffd640 1 --2- 172.21.15.45:0/1672883443 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f78640762d0 0x7f7864078790 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7868009af0 tx=0x7f7868008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:37.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.522+0000 7f78805ce740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcaf86d0 0x5641fcaab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.522+0000 7f78805ce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcadb7f0 0x5641fcadab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.522+0000 7f78805ce740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadb0b0 0x5641fcaafec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.522+0000 7f78805ce740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5641fc99b920 con 0x5641fcadb0b0 2024-05-08T05:44:37.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.522+0000 7f78805ce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5641fca10dd0 con 0x5641fcadb7f0 2024-05-08T05:44:37.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.522+0000 7f78805ce740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5641fc9c13e0 con 0x5641fcaf86d0 2024-05-08T05:44:37.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.522+0000 7f78777fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadb0b0 0x5641fcaafec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.522+0000 7f78777fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadb0b0 0x5641fcaafec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55544/0 (socket says 172.21.15.45:55544) 2024-05-08T05:44:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.522+0000 7f78777fe640 1 -- 172.21.15.45:0/389255829 learned_addr learned my addr 172.21.15.45:0/389255829 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.522+0000 7f7876ffd640 1 --2- 172.21.15.45:0/389255829 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcaf86d0 0x5641fcaab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.522+0000 7f78767fc640 1 --2- 172.21.15.45:0/389255829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcadb7f0 0x5641fcadab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.522+0000 7f78777fe640 1 -- 172.21.15.45:0/389255829 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcaf86d0 msgr2=0x5641fcaab2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.522+0000 7f78777fe640 1 --2- 172.21.15.45:0/389255829 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcaf86d0 0x5641fcaab2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.522+0000 7f78777fe640 1 -- 172.21.15.45:0/389255829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcadb7f0 msgr2=0x5641fcadab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.522+0000 7f78777fe640 1 --2- 172.21.15.45:0/389255829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcadb7f0 0x5641fcadab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.522+0000 7f78777fe640 1 -- 172.21.15.45:0/389255829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641fc7f27c0 con 0x5641fcadb0b0 2024-05-08T05:44:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.522+0000 7f7876ffd640 1 --2- 172.21.15.45:0/389255829 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcaf86d0 0x5641fcaab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.522+0000 7f78767fc640 1 --2- 172.21.15.45:0/389255829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcadb7f0 0x5641fcadab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.522+0000 7f78777fe640 1 --2- 172.21.15.45:0/389255829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadb0b0 0x5641fcaafec0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f78700b6810 tx=0x7f7870081110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed7ca90970b91dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.523+0000 7f785d7fa640 1 -- 172.21.15.45:0/389255829 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870049040 con 0x5641fcadb0b0 2024-05-08T05:44:37.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.523+0000 7f785d7fa640 1 -- 172.21.15.45:0/389255829 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7870081600 con 0x5641fcadb0b0 2024-05-08T05:44:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.523+0000 7f785d7fa640 1 -- 172.21.15.45:0/389255829 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78700859b0 con 0x5641fcadb0b0 2024-05-08T05:44:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.523+0000 7f78805ce740 1 -- 172.21.15.45:0/389255829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadb0b0 msgr2=0x5641fcaafec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.523+0000 7f78805ce740 1 --2- 172.21.15.45:0/389255829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadb0b0 0x5641fcaafec0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f78700b6810 tx=0x7f7870081110 comp rx=0 tx=0).stop 2024-05-08T05:44:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.523+0000 7f78805ce740 1 -- 172.21.15.45:0/389255829 shutdown_connections 2024-05-08T05:44:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.523+0000 7f78805ce740 1 --2- 172.21.15.45:0/389255829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadb0b0 0x5641fcaafec0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.523+0000 7f78805ce740 1 --2- 172.21.15.45:0/389255829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcadb7f0 0x5641fcadab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.523+0000 7f78805ce740 1 --2- 172.21.15.45:0/389255829 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcaf86d0 0x5641fcaab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.523+0000 7f78805ce740 1 -- 172.21.15.45:0/389255829 >> 172.21.15.45:0/389255829 conn(0x5641fcaad560 msgr2=0x5641fcaf5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.523+0000 7f78805ce740 1 -- 172.21.15.45:0/389255829 shutdown_connections 2024-05-08T05:44:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.523+0000 7f78805ce740 1 -- 172.21.15.45:0/389255829 wait complete. 2024-05-08T05:44:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.524+0000 7f78805ce740 1 -- start start 2024-05-08T05:44:37.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.524+0000 7f78805ce740 1 Processor -- start 2024-05-08T05:44:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.525+0000 7f78805ce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcadb7f0 0x5641fcacb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.525+0000 7f78805ce740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcaf86d0 0x5641fcaafdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.525+0000 7f78805ce740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadaa40 0x5641fcab02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.525+0000 7f78805ce740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5641fca19440 con 0x5641fcadaa40 2024-05-08T05:44:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.525+0000 7f78805ce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5641fca14120 con 0x5641fcadb7f0 2024-05-08T05:44:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.525+0000 7f78805ce740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5641fca22210 con 0x5641fcaf86d0 2024-05-08T05:44:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.525+0000 7f78805ce740 10 client.0 _ll_register_callbacks cb 0x5641fc9b9690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:44:37.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.525+0000 7f78777fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadaa40 0x5641fcab02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.525+0000 7f78777fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadaa40 0x5641fcab02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55552/0 (socket says 172.21.15.45:55552) 2024-05-08T05:44:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.525+0000 7f78777fe640 1 -- 172.21.15.45:0/2764652970 learned_addr learned my addr 172.21.15.45:0/2764652970 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.525+0000 7f78767fc640 1 --2- 172.21.15.45:0/2764652970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcaf86d0 0x5641fcaafdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.525+0000 7f7876ffd640 1 --2- 172.21.15.45:0/2764652970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcadb7f0 0x5641fcacb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.525+0000 7f78777fe640 1 -- 172.21.15.45:0/2764652970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcaf86d0 msgr2=0x5641fcaafdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.525+0000 7f78777fe640 1 --2- 172.21.15.45:0/2764652970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcaf86d0 0x5641fcaafdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.525+0000 7f78777fe640 1 -- 172.21.15.45:0/2764652970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcadb7f0 msgr2=0x5641fcacb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.525+0000 7f78777fe640 1 --2- 172.21.15.45:0/2764652970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcadb7f0 0x5641fcacb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.525+0000 7f78777fe640 1 -- 172.21.15.45:0/2764652970 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641fc852f50 con 0x5641fcadaa40 2024-05-08T05:44:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.525+0000 7f7876ffd640 1 --2- 172.21.15.45:0/2764652970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcadb7f0 0x5641fcacb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:44:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.526+0000 7f78777fe640 1 --2- 172.21.15.45:0/2764652970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadaa40 0x5641fcab02f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7870081c60 tx=0x7f7870009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0609301a727a92a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.526+0000 7f78567fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:44:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.526+0000 7f78567fc640 1 -- 172.21.15.45:0/2764652970 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870049070 con 0x5641fcadaa40 2024-05-08T05:44:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.526+0000 7f78567fc640 1 -- 172.21.15.45:0/2764652970 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f78700b80e0 con 0x5641fcadaa40 2024-05-08T05:44:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.526+0000 7f78805ce740 1 -- 172.21.15.45:0/2764652970 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7870081600 con 0x5641fcadaa40 2024-05-08T05:44:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.526+0000 7f78567fc640 1 -- 172.21.15.45:0/2764652970 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f78700859b0 con 0x5641fcadaa40 2024-05-08T05:44:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.526+0000 7f78805ce740 10 client.5922 fetch_fsmap learned FSMap version 35 2024-05-08T05:44:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.526+0000 7f78805ce740 1 -- 172.21.15.45:0/2764652970 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x5641fc6e60b0 con 0x5641fcadaa40 2024-05-08T05:44:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.526+0000 7f78567fc640 1 -- 172.21.15.45:0/2764652970 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7870081600 con 0x5641fcadaa40 2024-05-08T05:44:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.526+0000 7f78567fc640 1 -- 172.21.15.45:0/2764652970 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f787004c060 con 0x5641fcadaa40 2024-05-08T05:44:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.526+0000 7f78805ce740 10 client.5922 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T05:44:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.526+0000 7f78805ce740 10 client.5922 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T05:44:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.526+0000 7f78805ce740 20 client.5922 populate_metadata read hostname 'smithi045' 2024-05-08T05:44:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.526+0000 7f78805ce740 1 --2- 172.21.15.45:0/2764652970 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5641fcb07370 0x5641fcad8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.526+0000 7f78805ce740 4 client.5922 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:44:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.526+0000 7f78805ce740 1 -- 172.21.15.45:0/2764652970 --> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641fca22a20 con 0x5641fcb07370 2024-05-08T05:44:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.526+0000 7f7876ffd640 1 --2- 172.21.15.45:0/2764652970 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5641fcb07370 0x5641fcad8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.527+0000 7f7876ffd640 1 --2- 172.21.15.45:0/2764652970 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5641fcb07370 0x5641fcad8ae0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94885b5e362e28b8 server_cookie=707dd16166852651 in_seq=0 out_seq=0 2024-05-08T05:44:37.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.527+0000 7f78567fc640 10 client.5922 ms_handle_connect on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.527+0000 7f78567fc640 1 -- 172.21.15.45:0/2764652970 <== mds.0 v2:172.21.15.45:6833/1461822016 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5641fca22a20 con 0x5641fcb07370 2024-05-08T05:44:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.527+0000 7f78567fc640 10 client.5922 handle_command_reply: tid=0 2024-05-08T05:44:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.527+0000 7f78805ce740 1 client.5922 shutdown 2024-05-08T05:44:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.527+0000 7f78805ce740 1 -- 172.21.15.45:0/2764652970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadaa40 msgr2=0x5641fcab02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.527+0000 7f78805ce740 1 --2- 172.21.15.45:0/2764652970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadaa40 0x5641fcab02f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7870081c60 tx=0x7f7870009750 comp rx=0 tx=0).stop 2024-05-08T05:44:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.527+0000 7f78805ce740 1 -- 172.21.15.45:0/2764652970 shutdown_connections 2024-05-08T05:44:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.527+0000 7f78805ce740 1 --2- 172.21.15.45:0/2764652970 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5641fcb07370 0x5641fcad8ae0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.527+0000 7f78805ce740 1 --2- 172.21.15.45:0/2764652970 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcaf86d0 0x5641fcaafdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.527+0000 7f78805ce740 1 --2- 172.21.15.45:0/2764652970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadaa40 0x5641fcab02f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.527+0000 7f78805ce740 1 --2- 172.21.15.45:0/2764652970 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcadb7f0 0x5641fcacb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.527+0000 7f78567fc640 0 client.5922 ms_handle_reset on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.527+0000 7f78805ce740 1 -- 172.21.15.45:0/2764652970 >> 172.21.15.45:0/2764652970 conn(0x5641fcadb0b0 msgr2=0x5641fcaaaa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.527+0000 7f78805ce740 1 -- 172.21.15.45:0/2764652970 shutdown_connections 2024-05-08T05:44:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.528+0000 7f78805ce740 1 -- 172.21.15.45:0/2764652970 wait complete. 2024-05-08T05:44:37.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.528+0000 7f78805ce740 20 client.5922 trim_cache size 0 max 16384 2024-05-08T05:44:37.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.541+0000 7f78805ce740 1 Processor -- start 2024-05-08T05:44:37.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.541+0000 7f78805ce740 1 -- start start 2024-05-08T05:44:37.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.542+0000 7f78805ce740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcadaa40 0x5641fcaf9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.542+0000 7f78805ce740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadb7f0 0x5641fcaf5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.542+0000 7f78805ce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcaf86d0 0x5641fcaf5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.542+0000 7f78805ce740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7878201080 con 0x5641fcadb7f0 2024-05-08T05:44:37.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.542+0000 7f78805ce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f78780ca940 con 0x5641fcaf86d0 2024-05-08T05:44:37.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.542+0000 7f78805ce740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f78782009e0 con 0x5641fcadaa40 2024-05-08T05:44:37.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.542+0000 7f78767fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadb7f0 0x5641fcaf5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.542+0000 7f78767fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadb7f0 0x5641fcaf5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55562/0 (socket says 172.21.15.45:55562) 2024-05-08T05:44:37.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.542+0000 7f78767fc640 1 -- 172.21.15.45:0/3480370063 learned_addr learned my addr 172.21.15.45:0/3480370063 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:37.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.543+0000 7f7876ffd640 1 --2- 172.21.15.45:0/3480370063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcadaa40 0x5641fcaf9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.543+0000 7f78777fe640 1 --2- 172.21.15.45:0/3480370063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcaf86d0 0x5641fcaf5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.543+0000 7f78767fc640 1 -- 172.21.15.45:0/3480370063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcadaa40 msgr2=0x5641fcaf9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:37.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.543+0000 7f78767fc640 1 --2- 172.21.15.45:0/3480370063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcadaa40 0x5641fcaf9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.543+0000 7f78767fc640 1 -- 172.21.15.45:0/3480370063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcaf86d0 msgr2=0x5641fcaf5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:37.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.543+0000 7f78767fc640 1 --2- 172.21.15.45:0/3480370063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcaf86d0 0x5641fcaf5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.543+0000 7f78767fc640 1 -- 172.21.15.45:0/3480370063 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641fcaf6510 con 0x5641fcadb7f0 2024-05-08T05:44:37.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.543+0000 7f7876ffd640 1 --2- 172.21.15.45:0/3480370063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcadaa40 0x5641fcaf9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:37.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.543+0000 7f78777fe640 1 --2- 172.21.15.45:0/3480370063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcaf86d0 0x5641fcaf5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:37.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.543+0000 7f78767fc640 1 --2- 172.21.15.45:0/3480370063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadb7f0 0x5641fcaf5970 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f78780cc870 tx=0x7f786c001ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b372e9c9acf348c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:37.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.543+0000 7f785cff9640 1 -- 172.21.15.45:0/3480370063 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c019020 con 0x5641fcadb7f0 2024-05-08T05:44:37.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.543+0000 7f785cff9640 1 -- 172.21.15.45:0/3480370063 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f786c012b40 con 0x5641fcadb7f0 2024-05-08T05:44:37.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.543+0000 7f785cff9640 1 -- 172.21.15.45:0/3480370063 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c0058b0 con 0x5641fcadb7f0 2024-05-08T05:44:37.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.543+0000 7f78805ce740 1 -- 172.21.15.45:0/3480370063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadb7f0 msgr2=0x5641fcaf5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.543+0000 7f78805ce740 1 --2- 172.21.15.45:0/3480370063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadb7f0 0x5641fcaf5970 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f78780cc870 tx=0x7f786c001ca0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.543+0000 7f78805ce740 1 -- 172.21.15.45:0/3480370063 shutdown_connections 2024-05-08T05:44:37.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.543+0000 7f78805ce740 1 --2- 172.21.15.45:0/3480370063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcaf86d0 0x5641fcaf5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.543+0000 7f78805ce740 1 --2- 172.21.15.45:0/3480370063 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcadb7f0 0x5641fcaf5970 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.544+0000 7f78805ce740 1 --2- 172.21.15.45:0/3480370063 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcadaa40 0x5641fcaf9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.544+0000 7f78805ce740 1 -- 172.21.15.45:0/3480370063 >> 172.21.15.45:0/3480370063 conn(0x5641fcadb0b0 msgr2=0x5641fcaac030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:37.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.544+0000 7f78805ce740 1 -- 172.21.15.45:0/3480370063 shutdown_connections 2024-05-08T05:44:37.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.544+0000 7f78805ce740 1 -- 172.21.15.45:0/3480370063 wait complete. 2024-05-08T05:44:37.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.544+0000 7f78805ce740 1 -- start start 2024-05-08T05:44:37.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.544+0000 7f78805ce740 1 Processor -- start 2024-05-08T05:44:37.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.545+0000 7f78805ce740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcadaa40 0x5641fcb73c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.545+0000 7f78805ce740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcadb7f0 0x5641fcaf5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.545+0000 7f78805ce740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcaf86d0 0x5641fcaf5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.545+0000 7f78805ce740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5641fca19440 con 0x5641fcaf86d0 2024-05-08T05:44:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.545+0000 7f78805ce740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5641fc9caa20 con 0x5641fcadb7f0 2024-05-08T05:44:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.545+0000 7f78805ce740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5641fcad64b0 con 0x5641fcadaa40 2024-05-08T05:44:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.545+0000 7f78805ce740 10 client.0 _ll_register_callbacks cb 0x7f787008a3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:44:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.545+0000 7f78777fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcaf86d0 0x5641fcaf5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.545+0000 7f78777fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcaf86d0 0x5641fcaf5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55578/0 (socket says 172.21.15.45:55578) 2024-05-08T05:44:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.545+0000 7f78777fe640 1 -- 172.21.15.45:0/3171797409 learned_addr learned my addr 172.21.15.45:0/3171797409 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.545+0000 7f7876ffd640 1 --2- 172.21.15.45:0/3171797409 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcadaa40 0x5641fcb73c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.545+0000 7f78767fc640 1 --2- 172.21.15.45:0/3171797409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcadb7f0 0x5641fcaf5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.545+0000 7f78777fe640 1 -- 172.21.15.45:0/3171797409 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcadaa40 msgr2=0x5641fcb73c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.545+0000 7f78777fe640 1 --2- 172.21.15.45:0/3171797409 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcadaa40 0x5641fcb73c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.545+0000 7f78777fe640 1 -- 172.21.15.45:0/3171797409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcadb7f0 msgr2=0x5641fcaf5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:37.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.545+0000 7f78777fe640 1 --2- 172.21.15.45:0/3171797409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcadb7f0 0x5641fcaf5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.545+0000 7f78777fe640 1 -- 172.21.15.45:0/3171797409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641fcad6b80 con 0x5641fcaf86d0 2024-05-08T05:44:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.545+0000 7f78777fe640 1 --2- 172.21.15.45:0/3171797409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcaf86d0 0x5641fcaf5c50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f787007e870 tx=0x7f7870046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6435ac086ec7b79 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.545+0000 7f78567fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:44:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.546+0000 7f78567fc640 1 -- 172.21.15.45:0/3171797409 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870049070 con 0x5641fcaf86d0 2024-05-08T05:44:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.546+0000 7f78567fc640 1 -- 172.21.15.45:0/3171797409 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f78700b62e0 con 0x5641fcaf86d0 2024-05-08T05:44:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.546+0000 7f78805ce740 1 -- 172.21.15.45:0/3171797409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f786c012b40 con 0x5641fcaf86d0 2024-05-08T05:44:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.546+0000 7f78567fc640 1 -- 172.21.15.45:0/3171797409 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7870081600 con 0x5641fcaf86d0 2024-05-08T05:44:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.546+0000 7f78805ce740 10 client.5928 fetch_fsmap learned FSMap version 35 2024-05-08T05:44:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.546+0000 7f78805ce740 1 -- 172.21.15.45:0/3171797409 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x5641fcad6b80 con 0x5641fcaf86d0 2024-05-08T05:44:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.546+0000 7f78567fc640 1 -- 172.21.15.45:0/3171797409 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f786c012b40 con 0x5641fcaf86d0 2024-05-08T05:44:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.546+0000 7f78567fc640 1 -- 172.21.15.45:0/3171797409 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f7870080860 con 0x5641fcaf86d0 2024-05-08T05:44:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.546+0000 7f78805ce740 10 client.5928 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T05:44:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.546+0000 7f78805ce740 10 client.5928 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T05:44:37.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.546+0000 7f78805ce740 20 client.5928 populate_metadata read hostname 'smithi045' 2024-05-08T05:44:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.546+0000 7f78805ce740 1 --2- 172.21.15.45:0/3171797409 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5641fcaaabb0 0x5641fcaeb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.546+0000 7f78805ce740 4 client.5928 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-08T05:44:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.546+0000 7f78805ce740 1 -- 172.21.15.45:0/3171797409 --> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5641fca1c890 con 0x5641fcaaabb0 2024-05-08T05:44:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.546+0000 7f7876ffd640 1 --2- 172.21.15.45:0/3171797409 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5641fcaaabb0 0x5641fcaeb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.547+0000 7f78567fc640 10 client.5928 ms_handle_connect on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.547+0000 7f7876ffd640 1 --2- 172.21.15.45:0/3171797409 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5641fcaaabb0 0x5641fcaeb990 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4844b61f8d101455 server_cookie=26d17697683c7969 in_seq=0 out_seq=0 2024-05-08T05:44:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.547+0000 7f78567fc640 1 -- 172.21.15.45:0/3171797409 <== mds.0 v2:172.21.15.45:6833/1461822016 1 ==== command_reply(tid 0: 0 ) ==== 8+0+140 (crc 0 0 0) 0x5641fca1c890 con 0x5641fcaaabb0 2024-05-08T05:44:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.547+0000 7f78567fc640 10 client.5928 handle_command_reply: tid=0 2024-05-08T05:44:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.547+0000 7f78805ce740 1 client.5928 shutdown 2024-05-08T05:44:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.547+0000 7f78805ce740 1 -- 172.21.15.45:0/3171797409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcaf86d0 msgr2=0x5641fcaf5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.547+0000 7f78805ce740 1 --2- 172.21.15.45:0/3171797409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcaf86d0 0x5641fcaf5c50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f787007e870 tx=0x7f7870046000 comp rx=0 tx=0).stop 2024-05-08T05:44:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.547+0000 7f78805ce740 1 -- 172.21.15.45:0/3171797409 shutdown_connections 2024-05-08T05:44:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.547+0000 7f78805ce740 1 --2- 172.21.15.45:0/3171797409 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5641fcaaabb0 0x5641fcaeb990 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.547+0000 7f78805ce740 1 --2- 172.21.15.45:0/3171797409 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641fcaf86d0 0x5641fcaf5c50 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.547+0000 7f78567fc640 0 client.5928 ms_handle_reset on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:37.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.547+0000 7f78805ce740 1 --2- 172.21.15.45:0/3171797409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5641fcadb7f0 0x5641fcaf5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.547+0000 7f78805ce740 1 --2- 172.21.15.45:0/3171797409 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5641fcadaa40 0x5641fcb73c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.547+0000 7f78805ce740 1 -- 172.21.15.45:0/3171797409 >> 172.21.15.45:0/3171797409 conn(0x5641fcadb0b0 msgr2=0x5641fcad53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:37.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.547+0000 7f78805ce740 1 -- 172.21.15.45:0/3171797409 shutdown_connections 2024-05-08T05:44:37.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.547+0000 7f78805ce740 1 -- 172.21.15.45:0/3171797409 wait complete. 2024-05-08T05:44:37.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.547+0000 7f78805ce740 20 client.5928 trim_cache size 0 max 16384 2024-05-08T05:44:37.554 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:44:37.554 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-05-08T05:44:37.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:44:37.555 INFO:teuthology.orchestra.run.smithi045.stdout: "damage_type": "backtrace", 2024-05-08T05:44:37.555 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 448728150, 2024-05-08T05:44:37.555 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 1099511627777, 2024-05-08T05:44:37.555 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/dir_test/file" 2024-05-08T05:44:37.555 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:44:37.555 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-05-08T05:44:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.548+0000 7f787d8b4640 1 -- 172.21.15.45:0/1672883443 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f78640762d0 msgr2=0x7f7864078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.548+0000 7f787d8b4640 1 --2- 172.21.15.45:0/1672883443 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f78640762d0 0x7f7864078790 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7868009af0 tx=0x7f7868008040 comp rx=0 tx=0).stop 2024-05-08T05:44:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.548+0000 7f787d8b4640 1 -- 172.21.15.45:0/1672883443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78781629b0 msgr2=0x7f78780cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.548+0000 7f787d8b4640 1 --2- 172.21.15.45:0/1672883443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78781629b0 0x7f78780cf9b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7870009d90 tx=0x7f7870004a40 comp rx=0 tx=0).stop 2024-05-08T05:44:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.548+0000 7f787d8b4640 1 -- 172.21.15.45:0/1672883443 shutdown_connections 2024-05-08T05:44:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.548+0000 7f787d8b4640 1 --2- 172.21.15.45:0/1672883443 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f78640762d0 0x7f7864078790 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.548+0000 7f787d8b4640 1 --2- 172.21.15.45:0/1672883443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78781629b0 0x7f78780cf9b0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.548+0000 7f787d8b4640 1 --2- 172.21.15.45:0/1672883443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f78780b2490 0x7f78780cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.548+0000 7f787d8b4640 1 --2- 172.21.15.45:0/1672883443 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7878070d20 0x7f78781594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.548+0000 7f787d8b4640 1 -- 172.21.15.45:0/1672883443 >> 172.21.15.45:0/1672883443 conn(0x7f78780b49d0 msgr2=0x7f78780d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.548+0000 7f787d8b4640 1 -- 172.21.15.45:0/1672883443 shutdown_connections 2024-05-08T05:44:37.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.548+0000 7f787d8b4640 1 -- 172.21.15.45:0/1672883443 wait complete. 2024-05-08T05:44:37.559 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T05:44:37.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.661+0000 7f9ee70d3640 1 Processor -- start 2024-05-08T05:44:37.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.661+0000 7f9ee70d3640 1 -- start start 2024-05-08T05:44:37.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.662+0000 7f9ee70d3640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ee00d20c0 0x7f9ee0163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.662+0000 7f9ee70d3640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ee00d1300 con 0x7f9ee0071110 2024-05-08T05:44:37.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.662+0000 7f9ee70d3640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ee00d15e0 con 0x7f9ee0070d30 2024-05-08T05:44:37.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.662+0000 7f9ee70d3640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f9ee00d10d0 con 0x7f9ee00d20c0 2024-05-08T05:44:37.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.662+0000 7f9edffff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ee00d20c0 0x7f9ee0163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.662+0000 7f9ee5670640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9ee0071110 0x7f9ee006d4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46522/0 (socket says 172.21.15.45:46522) 2024-05-08T05:44:37.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.662+0000 7f9ee5670640 1 -- 172.21.15.45:0/1192762191 learned_addr learned my addr 172.21.15.45:0/1192762191 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:44:37.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.663+0000 7f9edffff640 1 -- 172.21.15.45:0/1192762191 >> v1:172.21.15.63:6789/0 conn(0x7f9ee0070d30 legacy=0x7f9ee00703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:37.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.663+0000 7f9edffff640 1 -- 172.21.15.45:0/1192762191 >> v1:172.21.15.45:6789/0 conn(0x7f9ee0071110 legacy=0x7f9ee006d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.663+0000 7f9edffff640 1 -- 172.21.15.45:0/1192762191 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee00d18c0 con 0x7f9ee00d20c0 2024-05-08T05:44:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.663+0000 7f9edffff640 1 --2- 172.21.15.45:0/1192762191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee00d20c0 0x7f9ee0163920 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8009d90 tx=0x7f9ec802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94f9d2fce76d1be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.663+0000 7f9edf7fe640 1 -- 172.21.15.45:0/1192762191 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8007740 con 0x7f9ee00d20c0 2024-05-08T05:44:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.663+0000 7f9edf7fe640 1 -- 172.21.15.45:0/1192762191 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec80078e0 con 0x7f9ee00d20c0 2024-05-08T05:44:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.664+0000 7f9edf7fe640 1 -- 172.21.15.45:0/1192762191 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8007cb0 con 0x7f9ee00d20c0 2024-05-08T05:44:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.664+0000 7f9ee70d3640 1 -- 172.21.15.45:0/1192762191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee00d20c0 msgr2=0x7f9ee0163920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.664+0000 7f9ee70d3640 1 --2- 172.21.15.45:0/1192762191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee00d20c0 0x7f9ee0163920 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8009d90 tx=0x7f9ec802a660 comp rx=0 tx=0).stop 2024-05-08T05:44:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.664+0000 7f9ee70d3640 1 -- 172.21.15.45:0/1192762191 shutdown_connections 2024-05-08T05:44:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.664+0000 7f9ee70d3640 1 --2- 172.21.15.45:0/1192762191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee00d20c0 0x7f9ee0163920 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.664+0000 7f9ee70d3640 1 -- 172.21.15.45:0/1192762191 >> 172.21.15.45:0/1192762191 conn(0x7f9ee00d2e90 msgr2=0x7f9ee00d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.664+0000 7f9ee70d3640 1 -- 172.21.15.45:0/1192762191 shutdown_connections 2024-05-08T05:44:37.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.664+0000 7f9ee70d3640 1 -- 172.21.15.45:0/1192762191 wait complete. 2024-05-08T05:44:37.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.665+0000 7f9ee70d3640 1 Processor -- start 2024-05-08T05:44:37.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.665+0000 7f9ee70d3640 1 -- start start 2024-05-08T05:44:37.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.666+0000 7f9ee70d3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ee0070d30 0x7f9ee00cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.666+0000 7f9ee70d3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee0071110 0x7f9ee015c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.666+0000 7f9ee70d3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ee00d20c0 0x7f9ee015de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.666+0000 7f9ee70d3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee00d1c30 con 0x7f9ee0071110 2024-05-08T05:44:37.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.666+0000 7f9ee70d3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee0070b80 con 0x7f9ee00d20c0 2024-05-08T05:44:37.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.666+0000 7f9ee70d3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee006dbc0 con 0x7f9ee0070d30 2024-05-08T05:44:37.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.666+0000 7f9edffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee0071110 0x7f9ee015c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.666+0000 7f9edffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee0071110 0x7f9ee015c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55586/0 (socket says 172.21.15.45:55586) 2024-05-08T05:44:37.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.666+0000 7f9edffff640 1 -- 172.21.15.45:0/2788569654 learned_addr learned my addr 172.21.15.45:0/2788569654 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:37.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.666+0000 7f9ee4e6f640 1 --2- 172.21.15.45:0/2788569654 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ee0070d30 0x7f9ee00cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.666+0000 7f9ee5670640 1 --2- 172.21.15.45:0/2788569654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ee00d20c0 0x7f9ee015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.667+0000 7f9edffff640 1 -- 172.21.15.45:0/2788569654 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ee0070d30 msgr2=0x7f9ee00cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.667+0000 7f9edffff640 1 --2- 172.21.15.45:0/2788569654 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ee0070d30 0x7f9ee00cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.667+0000 7f9edffff640 1 -- 172.21.15.45:0/2788569654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ee00d20c0 msgr2=0x7f9ee015de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.667+0000 7f9edffff640 1 --2- 172.21.15.45:0/2788569654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ee00d20c0 0x7f9ee015de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.667+0000 7f9edffff640 1 -- 172.21.15.45:0/2788569654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee015c9e0 con 0x7f9ee0071110 2024-05-08T05:44:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.667+0000 7f9edffff640 1 --2- 172.21.15.45:0/2788569654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee0071110 0x7f9ee015c4a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8009d90 tx=0x7f9ec8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.667+0000 7f9edd7fa640 1 -- 172.21.15.45:0/2788569654 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec803f070 con 0x7f9ee0071110 2024-05-08T05:44:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.667+0000 7f9edd7fa640 1 -- 172.21.15.45:0/2788569654 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec8031d00 con 0x7f9ee0071110 2024-05-08T05:44:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.667+0000 7f9ee70d3640 1 -- 172.21.15.45:0/2788569654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee00b1700 con 0x7f9ee0071110 2024-05-08T05:44:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.667+0000 7f9edd7fa640 1 -- 172.21.15.45:0/2788569654 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8044660 con 0x7f9ee0071110 2024-05-08T05:44:37.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.667+0000 7f9ee70d3640 1 -- 172.21.15.45:0/2788569654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee00b0c20 con 0x7f9ee0071110 2024-05-08T05:44:37.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.668+0000 7f9ee70d3640 1 -- 172.21.15.45:0/2788569654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ee00693f0 con 0x7f9ee0071110 2024-05-08T05:44:37.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.669+0000 7f9edd7fa640 1 -- 172.21.15.45:0/2788569654 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9ec803b410 con 0x7f9ee0071110 2024-05-08T05:44:37.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.669+0000 7f9edd7fa640 1 --2- 172.21.15.45:0/2788569654 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ec00762d0 0x7f9ec0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:37.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.669+0000 7f9edd7fa640 1 -- 172.21.15.45:0/2788569654 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f9ec80b91c0 con 0x7f9ee0071110 2024-05-08T05:44:37.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.669+0000 7f9ee4e6f640 1 --2- 172.21.15.45:0/2788569654 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ec00762d0 0x7f9ec0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:37.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.670+0000 7f9ee4e6f640 1 --2- 172.21.15.45:0/2788569654 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ec00762d0 0x7f9ec0078790 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0004ea0 tx=0x7f9ed0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:37.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.673+0000 7f9edd7fa640 1 -- 172.21.15.45:0/2788569654 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9ec80822c0 con 0x7f9ee0071110 2024-05-08T05:44:37.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.904+0000 7f9ee70d3640 1 -- 172.21.15.45:0/2788569654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9ee00d3d70 con 0x7f9ee0071110 2024-05-08T05:44:37.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.905+0000 7f9edd7fa640 1 -- 172.21.15.45:0/2788569654 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f9ec80865c0 con 0x7f9ee0071110 2024-05-08T05:44:37.906 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:44:37.906 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-08T05:44:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.907+0000 7f9ee70d3640 1 -- 172.21.15.45:0/2788569654 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ec00762d0 msgr2=0x7f9ec0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.907+0000 7f9ee70d3640 1 --2- 172.21.15.45:0/2788569654 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ec00762d0 0x7f9ec0078790 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0004ea0 tx=0x7f9ed0009040 comp rx=0 tx=0).stop 2024-05-08T05:44:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.907+0000 7f9ee70d3640 1 -- 172.21.15.45:0/2788569654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee0071110 msgr2=0x7f9ee015c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.907+0000 7f9ee70d3640 1 --2- 172.21.15.45:0/2788569654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee0071110 0x7f9ee015c4a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8009d90 tx=0x7f9ec8004a40 comp rx=0 tx=0).stop 2024-05-08T05:44:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.907+0000 7f9ee70d3640 1 -- 172.21.15.45:0/2788569654 shutdown_connections 2024-05-08T05:44:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.907+0000 7f9ee70d3640 1 --2- 172.21.15.45:0/2788569654 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ec00762d0 0x7f9ec0078790 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.907+0000 7f9ee70d3640 1 --2- 172.21.15.45:0/2788569654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ee00d20c0 0x7f9ee015de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.907+0000 7f9ee70d3640 1 --2- 172.21.15.45:0/2788569654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ee0071110 0x7f9ee015c4a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.907+0000 7f9ee70d3640 1 --2- 172.21.15.45:0/2788569654 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ee0070d30 0x7f9ee00cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.907+0000 7f9ee70d3640 1 -- 172.21.15.45:0/2788569654 >> 172.21.15.45:0/2788569654 conn(0x7f9ee00d2e90 msgr2=0x7f9ee0154360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.907+0000 7f9ee70d3640 1 -- 172.21.15.45:0/2788569654 shutdown_connections 2024-05-08T05:44:37.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:37.907+0000 7f9ee70d3640 1 -- 172.21.15.45:0/2788569654 wait complete. 2024-05-08T05:44:37.918 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-08T05:44:37.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:44:37.919 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-05-08T05:44:38.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.026+0000 7fb8c1196640 1 Processor -- start 2024-05-08T05:44:38.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.026+0000 7fb8c1196640 1 -- start start 2024-05-08T05:44:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.026+0000 7fb8c1196640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb8bc0d0940 0x7fb8bc0d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.026+0000 7fb8c1196640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8bc0b50d0 con 0x7fb8bc0d3a50 2024-05-08T05:44:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.026+0000 7fb8c1196640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8bc0cfc00 con 0x7fb8bc0d3670 2024-05-08T05:44:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.027+0000 7fb8c1196640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb8bc0d2a10 con 0x7fb8bc0d0940 2024-05-08T05:44:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.027+0000 7fb8bb59e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb8bc0d0940 0x7fb8bc0d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.027+0000 7fb8b259c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb8bc0d3a50 0x7fb8bc162920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46524/0 (socket says 172.21.15.45:46524) 2024-05-08T05:44:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.027+0000 7fb8b259c640 1 -- 172.21.15.45:0/518488114 learned_addr learned my addr 172.21.15.45:0/518488114 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:44:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.027+0000 7fb8bb59e640 1 -- 172.21.15.45:0/518488114 >> v1:172.21.15.63:6789/0 conn(0x7fb8bc0d3670 legacy=0x7fb8bc0b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.027+0000 7fb8bb59e640 1 -- 172.21.15.45:0/518488114 >> v1:172.21.15.45:6789/0 conn(0x7fb8bc0d3a50 legacy=0x7fb8bc162920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.027+0000 7fb8ba59c640 1 -- 172.21.15.45:0/518488114 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4265382392 0 0) 0x7fb8bc0b50d0 con 0x7fb8bc0d3a50 2024-05-08T05:44:38.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.027+0000 7fb8bb59e640 1 -- 172.21.15.45:0/518488114 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8bc0cfea0 con 0x7fb8bc0d0940 2024-05-08T05:44:38.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.028+0000 7fb8bb59e640 1 --2- 172.21.15.45:0/518488114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8bc0d0940 0x7fb8bc0d0dd0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009d90 tx=0x7fb8b402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297c0900f676cb24 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:38.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.028+0000 7fb8ba59c640 1 -- 172.21.15.45:0/518488114 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4007740 con 0x7fb8bc0d0940 2024-05-08T05:44:38.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.028+0000 7fb8ba59c640 1 -- 172.21.15.45:0/518488114 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b40078e0 con 0x7fb8bc0d0940 2024-05-08T05:44:38.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.028+0000 7fb8ba59c640 1 -- 172.21.15.45:0/518488114 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4007cb0 con 0x7fb8bc0d0940 2024-05-08T05:44:38.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.028+0000 7fb8c1196640 1 -- 172.21.15.45:0/518488114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8bc0d0940 msgr2=0x7fb8bc0d0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.028+0000 7fb8c1196640 1 --2- 172.21.15.45:0/518488114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8bc0d0940 0x7fb8bc0d0dd0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009d90 tx=0x7fb8b402a660 comp rx=0 tx=0).stop 2024-05-08T05:44:38.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.028+0000 7fb8c1196640 1 -- 172.21.15.45:0/518488114 shutdown_connections 2024-05-08T05:44:38.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.028+0000 7fb8c1196640 1 --2- 172.21.15.45:0/518488114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8bc0d0940 0x7fb8bc0d0dd0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.028+0000 7fb8c1196640 1 -- 172.21.15.45:0/518488114 >> 172.21.15.45:0/518488114 conn(0x7fb8bc09e390 msgr2=0x7fb8bc0d5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:38.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.028+0000 7fb8c1196640 1 -- 172.21.15.45:0/518488114 shutdown_connections 2024-05-08T05:44:38.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.028+0000 7fb8c1196640 1 -- 172.21.15.45:0/518488114 wait complete. 2024-05-08T05:44:38.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.030+0000 7fb8c1196640 1 Processor -- start 2024-05-08T05:44:38.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.030+0000 7fb8c1196640 1 -- start start 2024-05-08T05:44:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.031+0000 7fb8c1196640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8bc0d0940 0x7fb8bc1f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.031+0000 7fb8c1196640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8bc0d3670 0x7fb8bc0b9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.031+0000 7fb8c1196640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8bc0d3a50 0x7fb8bc0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.031+0000 7fb8c1196640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb8bc0d2730 con 0x7fb8bc0d3a50 2024-05-08T05:44:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.031+0000 7fb8c1196640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb8bc0d04b0 con 0x7fb8bc0d0940 2024-05-08T05:44:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.031+0000 7fb8c1196640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb8bc0d0140 con 0x7fb8bc0d3670 2024-05-08T05:44:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.031+0000 7fb8bb59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8bc0d3a50 0x7fb8bc0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.031+0000 7fb8bb59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8bc0d3a50 0x7fb8bc0cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55594/0 (socket says 172.21.15.45:55594) 2024-05-08T05:44:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.031+0000 7fb8bb59e640 1 -- 172.21.15.45:0/3602746981 learned_addr learned my addr 172.21.15.45:0/3602746981 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.031+0000 7fb8b259c640 1 --2- 172.21.15.45:0/3602746981 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8bc0d3670 0x7fb8bc0b9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.031+0000 7fb8bad9d640 1 --2- 172.21.15.45:0/3602746981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8bc0d0940 0x7fb8bc1f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.031+0000 7fb8bb59e640 1 -- 172.21.15.45:0/3602746981 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8bc0d3670 msgr2=0x7fb8bc0b9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.031+0000 7fb8bb59e640 1 --2- 172.21.15.45:0/3602746981 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8bc0d3670 0x7fb8bc0b9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.031+0000 7fb8bb59e640 1 -- 172.21.15.45:0/3602746981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8bc0d0940 msgr2=0x7fb8bc1f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.031+0000 7fb8bb59e640 1 --2- 172.21.15.45:0/3602746981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8bc0d0940 0x7fb8bc1f5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.031+0000 7fb8bb59e640 1 -- 172.21.15.45:0/3602746981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8bc0ba260 con 0x7fb8bc0d3a50 2024-05-08T05:44:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.032+0000 7fb8bb59e640 1 --2- 172.21.15.45:0/3602746981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8bc0d3a50 0x7fb8bc0cdff0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009d90 tx=0x7fb8b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.032+0000 7fb8b3fff640 1 -- 172.21.15.45:0/3602746981 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4040070 con 0x7fb8bc0d3a50 2024-05-08T05:44:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.032+0000 7fb8b3fff640 1 -- 172.21.15.45:0/3602746981 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b4031c90 con 0x7fb8bc0d3a50 2024-05-08T05:44:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.032+0000 7fb8b3fff640 1 -- 172.21.15.45:0/3602746981 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b403c640 con 0x7fb8bc0d3a50 2024-05-08T05:44:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.032+0000 7fb8c1196640 1 -- 172.21.15.45:0/3602746981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8bc1f5710 con 0x7fb8bc0d3a50 2024-05-08T05:44:38.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.032+0000 7fb8c1196640 1 -- 172.21.15.45:0/3602746981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8bc1f5cd0 con 0x7fb8bc0d3a50 2024-05-08T05:44:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.034+0000 7fb8b3fff640 1 -- 172.21.15.45:0/3602746981 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb8b4039040 con 0x7fb8bc0d3a50 2024-05-08T05:44:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.034+0000 7fb8b3fff640 1 --2- 172.21.15.45:0/3602746981 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb89c0762d0 0x7fb89c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.034+0000 7fb8b3fff640 1 -- 172.21.15.45:0/3602746981 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fb8b40b8d80 con 0x7fb8bc0d3a50 2024-05-08T05:44:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.034+0000 7fb8bad9d640 1 --2- 172.21.15.45:0/3602746981 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb89c0762d0 0x7fb89c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.035+0000 7fb8bad9d640 1 --2- 172.21.15.45:0/3602746981 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb89c0762d0 0x7fb89c078790 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac004f60 tx=0x7fb8ac007e30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.035+0000 7fb8c3f88740 1 Processor -- start 2024-05-08T05:44:38.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.035+0000 7fb8c3f88740 1 -- start start 2024-05-08T05:44:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.036+0000 7fb8c3f88740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6eaa2e80 0x55af6ea55810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.036+0000 7fb8c3f88740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6ea85fa0 0x55af6ea85320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.036+0000 7fb8c3f88740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85860 0x55af6ea5a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.036+0000 7fb8c3f88740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55af6e9bfbc0 con 0x55af6ea85860 2024-05-08T05:44:38.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.036+0000 7fb8c3f88740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55af6e978d00 con 0x55af6eaa2e80 2024-05-08T05:44:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.036+0000 7fb8c3f88740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55af6e9bbb80 con 0x55af6ea85fa0 2024-05-08T05:44:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.036+0000 7fb8bb59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85860 0x55af6ea5a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.036+0000 7fb8bb59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85860 0x55af6ea5a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55608/0 (socket says 172.21.15.45:55608) 2024-05-08T05:44:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.036+0000 7fb8bb59e640 1 -- 172.21.15.45:0/3186161656 learned_addr learned my addr 172.21.15.45:0/3186161656 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.036+0000 7fb8bad9d640 1 --2- 172.21.15.45:0/3186161656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6eaa2e80 0x55af6ea55810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.036+0000 7fb8b259c640 1 --2- 172.21.15.45:0/3186161656 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6ea85fa0 0x55af6ea85320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.036+0000 7fb8bb59e640 1 -- 172.21.15.45:0/3186161656 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6ea85fa0 msgr2=0x55af6ea85320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.036+0000 7fb8bb59e640 1 --2- 172.21.15.45:0/3186161656 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6ea85fa0 0x55af6ea85320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.036+0000 7fb8bb59e640 1 -- 172.21.15.45:0/3186161656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6eaa2e80 msgr2=0x55af6ea55810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.036+0000 7fb8bb59e640 1 --2- 172.21.15.45:0/3186161656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6eaa2e80 0x55af6ea55810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.036+0000 7fb8bb59e640 1 -- 172.21.15.45:0/3186161656 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af6e79d940 con 0x55af6ea85860 2024-05-08T05:44:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.037+0000 7fb8bb59e640 1 --2- 172.21.15.45:0/3186161656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85860 0x55af6ea5a670 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb8b407dfb0 tx=0x7fb8b4031e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b631e058303aeb29 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:38.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.037+0000 7fb8b0d99640 1 -- 172.21.15.45:0/3186161656 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4086230 con 0x55af6ea85860 2024-05-08T05:44:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.037+0000 7fb8b0d99640 1 -- 172.21.15.45:0/3186161656 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b408ac50 con 0x55af6ea85860 2024-05-08T05:44:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.037+0000 7fb8b0d99640 1 -- 172.21.15.45:0/3186161656 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b408a7c0 con 0x55af6ea85860 2024-05-08T05:44:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.037+0000 7fb8c3f88740 1 -- 172.21.15.45:0/3186161656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85860 msgr2=0x55af6ea5a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.037+0000 7fb8c3f88740 1 --2- 172.21.15.45:0/3186161656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85860 0x55af6ea5a670 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb8b407dfb0 tx=0x7fb8b4031e40 comp rx=0 tx=0).stop 2024-05-08T05:44:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.037+0000 7fb8c3f88740 1 -- 172.21.15.45:0/3186161656 shutdown_connections 2024-05-08T05:44:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.037+0000 7fb8c3f88740 1 --2- 172.21.15.45:0/3186161656 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6ea85fa0 0x55af6ea85320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.037+0000 7fb8c3f88740 1 --2- 172.21.15.45:0/3186161656 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85860 0x55af6ea5a670 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.037+0000 7fb8c3f88740 1 --2- 172.21.15.45:0/3186161656 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6eaa2e80 0x55af6ea55810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.037+0000 7fb8c3f88740 1 -- 172.21.15.45:0/3186161656 >> 172.21.15.45:0/3186161656 conn(0x55af6ea57a80 msgr2=0x55af6eaa0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.038+0000 7fb8c3f88740 1 -- 172.21.15.45:0/3186161656 shutdown_connections 2024-05-08T05:44:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.038+0000 7fb8c3f88740 1 -- 172.21.15.45:0/3186161656 wait complete. 2024-05-08T05:44:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.038+0000 7fb8c3f88740 1 -- start start 2024-05-08T05:44:38.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.038+0000 7fb8c3f88740 1 Processor -- start 2024-05-08T05:44:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.039+0000 7fb8c3f88740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 0x55af6ea9f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.039+0000 7fb8c3f88740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6eaa2e80 0x55af6ea5a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.039+0000 7fb8c3f88740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6ea760a0 0x55af6ea5a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.039+0000 7fb8c3f88740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55af6e9bfbc0 con 0x55af6ea85fa0 2024-05-08T05:44:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.039+0000 7fb8c3f88740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55af6e97c830 con 0x55af6eaa2e80 2024-05-08T05:44:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.039+0000 7fb8c3f88740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55af6ea59c30 con 0x55af6ea760a0 2024-05-08T05:44:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.039+0000 7fb8c3f88740 10 client.0 _ll_register_callbacks cb 0x55af6e9b9750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:44:38.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.039+0000 7fb8bad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 0x55af6ea9f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.039+0000 7fb8bad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 0x55af6ea9f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55612/0 (socket says 172.21.15.45:55612) 2024-05-08T05:44:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.039+0000 7fb8bad9d640 1 -- 172.21.15.45:0/3441338360 learned_addr learned my addr 172.21.15.45:0/3441338360 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.039+0000 7fb8bb59e640 1 --2- 172.21.15.45:0/3441338360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6ea760a0 0x55af6ea5a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.039+0000 7fb8b259c640 1 --2- 172.21.15.45:0/3441338360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6eaa2e80 0x55af6ea5a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.039+0000 7fb8bad9d640 1 -- 172.21.15.45:0/3441338360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6ea760a0 msgr2=0x55af6ea5a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.039+0000 7fb8bad9d640 1 --2- 172.21.15.45:0/3441338360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6ea760a0 0x55af6ea5a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.039+0000 7fb8bad9d640 1 -- 172.21.15.45:0/3441338360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6eaa2e80 msgr2=0x55af6ea5a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.039+0000 7fb8bad9d640 1 --2- 172.21.15.45:0/3441338360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6eaa2e80 0x55af6ea5a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.039+0000 7fb8bad9d640 1 -- 172.21.15.45:0/3441338360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af6e7fe0d0 con 0x55af6ea85fa0 2024-05-08T05:44:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.039+0000 7fb8b259c640 1 --2- 172.21.15.45:0/3441338360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6eaa2e80 0x55af6ea5a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:44:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.040+0000 7fb8bad9d640 1 --2- 172.21.15.45:0/3441338360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 0x55af6ea9f070 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00fe30 tx=0x7fb8ac003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ecde6db64a7c142 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.040+0000 7fb899ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:44:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.040+0000 7fb899ffb640 1 -- 172.21.15.45:0/3441338360 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac025020 con 0x55af6ea85fa0 2024-05-08T05:44:38.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.040+0000 7fb899ffb640 1 -- 172.21.15.45:0/3441338360 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fb8ac014660 con 0x55af6ea85fa0 2024-05-08T05:44:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.040+0000 7fb8c3f88740 1 -- 172.21.15.45:0/3441338360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8b408ac50 con 0x55af6ea85fa0 2024-05-08T05:44:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.040+0000 7fb899ffb640 1 -- 172.21.15.45:0/3441338360 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb8ac02f050 con 0x55af6ea85fa0 2024-05-08T05:44:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.040+0000 7fb8c3f88740 10 client.5952 fetch_fsmap learned FSMap version 35 2024-05-08T05:44:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.040+0000 7fb8c3f88740 1 -- 172.21.15.45:0/3441338360 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55af6e691230 con 0x55af6ea85fa0 2024-05-08T05:44:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.040+0000 7fb899ffb640 1 -- 172.21.15.45:0/3441338360 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b408ac50 con 0x55af6ea85fa0 2024-05-08T05:44:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.040+0000 7fb899ffb640 1 -- 172.21.15.45:0/3441338360 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7fb8ac032020 con 0x55af6ea85fa0 2024-05-08T05:44:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.040+0000 7fb8c3f88740 10 client.5952 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T05:44:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.040+0000 7fb8c3f88740 10 client.5952 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T05:44:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.040+0000 7fb8c3f88740 20 client.5952 populate_metadata read hostname 'smithi045' 2024-05-08T05:44:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.040+0000 7fb8c3f88740 1 --2- 172.21.15.45:0/3441338360 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55af6ea9ded0 0x55af6ea9e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.040+0000 7fb8c3f88740 4 client.5952 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:44:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.040+0000 7fb8c3f88740 1 -- 172.21.15.45:0/3441338360 --> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af6e9cd7d0 con 0x55af6ea9ded0 2024-05-08T05:44:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.040+0000 7fb8b259c640 1 --2- 172.21.15.45:0/3441338360 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55af6ea9ded0 0x55af6ea9e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.041+0000 7fb899ffb640 10 client.5952 ms_handle_connect on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:38.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.041+0000 7fb8b259c640 1 --2- 172.21.15.45:0/3441338360 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55af6ea9ded0 0x55af6ea9e310 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f57aae5c3cb38a2d server_cookie=f7944a1272c8b0f0 in_seq=0 out_seq=0 2024-05-08T05:44:38.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.041+0000 7fb899ffb640 1 -- 172.21.15.45:0/3441338360 <== mds.0 v2:172.21.15.45:6833/1461822016 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55af6e9cd7d0 con 0x55af6ea9ded0 2024-05-08T05:44:38.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.041+0000 7fb899ffb640 10 client.5952 handle_command_reply: tid=0 2024-05-08T05:44:38.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.041+0000 7fb8c3f88740 1 client.5952 shutdown 2024-05-08T05:44:38.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.041+0000 7fb8c3f88740 1 -- 172.21.15.45:0/3441338360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 msgr2=0x55af6ea9f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.041+0000 7fb8c3f88740 1 --2- 172.21.15.45:0/3441338360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 0x55af6ea9f070 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00fe30 tx=0x7fb8ac003760 comp rx=0 tx=0).stop 2024-05-08T05:44:38.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.041+0000 7fb8c3f88740 1 -- 172.21.15.45:0/3441338360 shutdown_connections 2024-05-08T05:44:38.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.041+0000 7fb8c3f88740 1 --2- 172.21.15.45:0/3441338360 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55af6ea9ded0 0x55af6ea9e310 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.041+0000 7fb8c3f88740 1 --2- 172.21.15.45:0/3441338360 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6ea760a0 0x55af6ea5a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.041+0000 7fb8c3f88740 1 --2- 172.21.15.45:0/3441338360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6eaa2e80 0x55af6ea5a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.041+0000 7fb899ffb640 0 client.5952 ms_handle_reset on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:38.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.041+0000 7fb8c3f88740 1 --2- 172.21.15.45:0/3441338360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 0x55af6ea9f070 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.041+0000 7fb8c3f88740 1 -- 172.21.15.45:0/3441338360 >> 172.21.15.45:0/3441338360 conn(0x55af6ea85860 msgr2=0x55af6ea54f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:38.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.042+0000 7fb8c3f88740 1 -- 172.21.15.45:0/3441338360 shutdown_connections 2024-05-08T05:44:38.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.042+0000 7fb8c3f88740 1 -- 172.21.15.45:0/3441338360 wait complete. 2024-05-08T05:44:38.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.042+0000 7fb8c3f88740 20 client.5952 trim_cache size 0 max 16384 2024-05-08T05:44:38.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.060+0000 7fb8c3f88740 1 Processor -- start 2024-05-08T05:44:38.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.060+0000 7fb8c3f88740 1 -- start start 2024-05-08T05:44:38.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.060+0000 7fb8c3f88740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6ea760a0 0x55af6ea9e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.060+0000 7fb8c3f88740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 0x55af6ea51010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.060+0000 7fb8c3f88740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6eaa2e80 0x55af6ea73330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.060+0000 7fb8c3f88740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb8bc201fc0 con 0x55af6ea85fa0 2024-05-08T05:44:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.060+0000 7fb8c3f88740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb8bc200fa0 con 0x55af6ea760a0 2024-05-08T05:44:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.060+0000 7fb8c3f88740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb8bc1f8f10 con 0x55af6eaa2e80 2024-05-08T05:44:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.061+0000 7fb8b259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 0x55af6ea51010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.061+0000 7fb8b259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 0x55af6ea51010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55624/0 (socket says 172.21.15.45:55624) 2024-05-08T05:44:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.061+0000 7fb8b259c640 1 -- 172.21.15.45:0/2052666699 learned_addr learned my addr 172.21.15.45:0/2052666699 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.061+0000 7fb8b259c640 1 -- 172.21.15.45:0/2052666699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6eaa2e80 msgr2=0x55af6ea73330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.061+0000 7fb8bb59e640 1 --2- 172.21.15.45:0/2052666699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6eaa2e80 0x55af6ea73330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.061+0000 7fb8bad9d640 1 --2- 172.21.15.45:0/2052666699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6ea760a0 0x55af6ea9e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.061+0000 7fb8b259c640 1 --2- 172.21.15.45:0/2052666699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6eaa2e80 0x55af6ea73330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.061+0000 7fb8b259c640 1 -- 172.21.15.45:0/2052666699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6ea760a0 msgr2=0x55af6ea9e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:38.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.061+0000 7fb8b259c640 1 --2- 172.21.15.45:0/2052666699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6ea760a0 0x55af6ea9e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.061+0000 7fb8b259c640 1 -- 172.21.15.45:0/2052666699 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af6ea96ed0 con 0x55af6ea85fa0 2024-05-08T05:44:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.061+0000 7fb8bad9d640 1 --2- 172.21.15.45:0/2052666699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6ea760a0 0x55af6ea9e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.061+0000 7fb8bb59e640 1 --2- 172.21.15.45:0/2052666699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6eaa2e80 0x55af6ea73330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.061+0000 7fb8b259c640 1 --2- 172.21.15.45:0/2052666699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 0x55af6ea51010 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4000f60 tx=0x7fb8a4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463ac0a45d91f0b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.061+0000 7fb89bfff640 1 -- 172.21.15.45:0/2052666699 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a401d760 con 0x55af6ea85fa0 2024-05-08T05:44:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.061+0000 7fb89bfff640 1 -- 172.21.15.45:0/2052666699 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8a401d900 con 0x55af6ea85fa0 2024-05-08T05:44:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.061+0000 7fb89bfff640 1 -- 172.21.15.45:0/2052666699 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4017670 con 0x55af6ea85fa0 2024-05-08T05:44:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.062+0000 7fb8c3f88740 1 -- 172.21.15.45:0/2052666699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 msgr2=0x55af6ea51010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.062+0000 7fb8c3f88740 1 --2- 172.21.15.45:0/2052666699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 0x55af6ea51010 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4000f60 tx=0x7fb8a4009040 comp rx=0 tx=0).stop 2024-05-08T05:44:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.062+0000 7fb8c3f88740 1 -- 172.21.15.45:0/2052666699 shutdown_connections 2024-05-08T05:44:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.062+0000 7fb8c3f88740 1 --2- 172.21.15.45:0/2052666699 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6eaa2e80 0x55af6ea73330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.062+0000 7fb8c3f88740 1 --2- 172.21.15.45:0/2052666699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 0x55af6ea51010 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.062+0000 7fb8c3f88740 1 --2- 172.21.15.45:0/2052666699 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6ea760a0 0x55af6ea9e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.062+0000 7fb8c3f88740 1 -- 172.21.15.45:0/2052666699 >> 172.21.15.45:0/2052666699 conn(0x55af6ea85860 msgr2=0x55af6eab2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:38.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.062+0000 7fb8c3f88740 1 -- 172.21.15.45:0/2052666699 shutdown_connections 2024-05-08T05:44:38.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.062+0000 7fb8c3f88740 1 -- 172.21.15.45:0/2052666699 wait complete. 2024-05-08T05:44:38.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.062+0000 7fb8c3f88740 1 -- start start 2024-05-08T05:44:38.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.062+0000 7fb8c3f88740 1 Processor -- start 2024-05-08T05:44:38.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.063+0000 7fb8c3f88740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6ea760a0 0x55af6ea818f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.063+0000 7fb8c3f88740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 0x55af6ea9bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.063+0000 7fb8c3f88740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6eaa2e80 0x55af6ea9ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.063+0000 7fb8c3f88740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb8bc200970 con 0x55af6ea85fa0 2024-05-08T05:44:38.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.063+0000 7fb8c3f88740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55af6eab32d0 con 0x55af6ea760a0 2024-05-08T05:44:38.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.063+0000 7fb8c3f88740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55af6ea55c90 con 0x55af6eaa2e80 2024-05-08T05:44:38.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.063+0000 7fb8c3f88740 10 client.0 _ll_register_callbacks cb 0x7fb8ac003940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:44:38.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.063+0000 7fb8b259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 0x55af6ea9bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.063+0000 7fb8b259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 0x55af6ea9bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55626/0 (socket says 172.21.15.45:55626) 2024-05-08T05:44:38.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.063+0000 7fb8b259c640 1 -- 172.21.15.45:0/4082059469 learned_addr learned my addr 172.21.15.45:0/4082059469 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:38.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.063+0000 7fb8bb59e640 1 --2- 172.21.15.45:0/4082059469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6eaa2e80 0x55af6ea9ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.063+0000 7fb8bad9d640 1 --2- 172.21.15.45:0/4082059469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6ea760a0 0x55af6ea818f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb8b259c640 1 -- 172.21.15.45:0/4082059469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6eaa2e80 msgr2=0x55af6ea9ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb8b259c640 1 --2- 172.21.15.45:0/4082059469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6eaa2e80 0x55af6ea9ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb8b259c640 1 -- 172.21.15.45:0/4082059469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6ea760a0 msgr2=0x55af6ea818f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb8b259c640 1 --2- 172.21.15.45:0/4082059469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6ea760a0 0x55af6ea818f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb8b259c640 1 -- 172.21.15.45:0/4082059469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af6ea81e30 con 0x55af6ea85fa0 2024-05-08T05:44:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb8b259c640 1 --2- 172.21.15.45:0/4082059469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 0x55af6ea9bb60 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4018000 tx=0x7fb8ac012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b506328e7fbf8b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb899ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:44:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb899ffb640 1 -- 172.21.15.45:0/4082059469 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4017b70 con 0x55af6ea85fa0 2024-05-08T05:44:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb899ffb640 1 -- 172.21.15.45:0/4082059469 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fb8a4017d10 con 0x55af6ea85fa0 2024-05-08T05:44:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb8c3f88740 1 -- 172.21.15.45:0/4082059469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8a401d900 con 0x55af6ea85fa0 2024-05-08T05:44:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb899ffb640 1 -- 172.21.15.45:0/4082059469 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb8a401dab0 con 0x55af6ea85fa0 2024-05-08T05:44:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb8c3f88740 10 client.5958 fetch_fsmap learned FSMap version 35 2024-05-08T05:44:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb8c3f88740 1 -- 172.21.15.45:0/4082059469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55af6ea5aaa0 con 0x55af6ea85fa0 2024-05-08T05:44:38.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb899ffb640 1 -- 172.21.15.45:0/4082059469 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8a401d900 con 0x55af6ea85fa0 2024-05-08T05:44:38.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb899ffb640 1 -- 172.21.15.45:0/4082059469 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7fb8a4030ce0 con 0x55af6ea85fa0 2024-05-08T05:44:38.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb8c3f88740 10 client.5958 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T05:44:38.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb8c3f88740 10 client.5958 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T05:44:38.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb8c3f88740 20 client.5958 populate_metadata read hostname 'smithi045' 2024-05-08T05:44:38.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb8c3f88740 1 --2- 172.21.15.45:0/4082059469 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55af6eb05320 0x55af6eaa0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb8c3f88740 4 client.5958 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-05-08T05:44:38.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.064+0000 7fb8c3f88740 1 -- 172.21.15.45:0/4082059469 --> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) -- 0x55af6e9c7640 con 0x55af6eb05320 2024-05-08T05:44:38.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.065+0000 7fb8bad9d640 1 --2- 172.21.15.45:0/4082059469 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55af6eb05320 0x55af6eaa0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.065+0000 7fb899ffb640 10 client.5958 ms_handle_connect on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:38.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.065+0000 7fb8bad9d640 1 --2- 172.21.15.45:0/4082059469 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55af6eb05320 0x55af6eaa0500 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78dead67631bdaa5 server_cookie=4cc3b940d8612521 in_seq=0 out_seq=0 2024-05-08T05:44:38.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.065+0000 7fb899ffb640 1 -- 172.21.15.45:0/4082059469 <== mds.0 v2:172.21.15.45:6833/1461822016 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55af6e9c7640 con 0x55af6eb05320 2024-05-08T05:44:38.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.065+0000 7fb899ffb640 10 client.5958 handle_command_reply: tid=0 2024-05-08T05:44:38.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.065+0000 7fb8c3f88740 1 client.5958 shutdown 2024-05-08T05:44:38.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.066+0000 7fb8c3f88740 1 -- 172.21.15.45:0/4082059469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 msgr2=0x55af6ea9bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.066+0000 7fb8c3f88740 1 --2- 172.21.15.45:0/4082059469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 0x55af6ea9bb60 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4018000 tx=0x7fb8ac012000 comp rx=0 tx=0).stop 2024-05-08T05:44:38.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.066+0000 7fb8c3f88740 1 -- 172.21.15.45:0/4082059469 shutdown_connections 2024-05-08T05:44:38.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.066+0000 7fb8c3f88740 1 --2- 172.21.15.45:0/4082059469 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55af6eb05320 0x55af6eaa0500 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.066+0000 7fb8c3f88740 1 --2- 172.21.15.45:0/4082059469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55af6eaa2e80 0x55af6ea9ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.066+0000 7fb8c3f88740 1 --2- 172.21.15.45:0/4082059469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55af6ea85fa0 0x55af6ea9bb60 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.066+0000 7fb8c3f88740 1 --2- 172.21.15.45:0/4082059469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55af6ea760a0 0x55af6ea818f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.066+0000 7fb8c3f88740 1 -- 172.21.15.45:0/4082059469 >> 172.21.15.45:0/4082059469 conn(0x55af6ea85860 msgr2=0x55af6eab2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:38.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.066+0000 7fb899ffb640 0 client.5958 ms_handle_reset on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:38.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.066+0000 7fb8c3f88740 1 -- 172.21.15.45:0/4082059469 shutdown_connections 2024-05-08T05:44:38.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.066+0000 7fb8c3f88740 1 -- 172.21.15.45:0/4082059469 wait complete. 2024-05-08T05:44:38.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.066+0000 7fb8c3f88740 20 client.5958 trim_cache size 0 max 16384 2024-05-08T05:44:38.073 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:44:38.073 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-05-08T05:44:38.073 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "30d4344c-9fc3-4a2f-ac85-063a31a4def8", 2024-05-08T05:44:38.073 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-05-08T05:44:38.073 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:44:38.073 INFO:tasks.ceph.mds.a.smithi045.stderr:2024-05-08T05:44:38.066+0000 7f3443f19640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5177=0-4194304@1} caps={5177=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5177 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x560338028b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-08T05:44:38.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.066+0000 7fb8c1196640 1 -- 172.21.15.45:0/3602746981 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb89c0762d0 msgr2=0x7fb89c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.066+0000 7fb8c1196640 1 --2- 172.21.15.45:0/3602746981 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb89c0762d0 0x7fb89c078790 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac004f60 tx=0x7fb8ac007e30 comp rx=0 tx=0).stop 2024-05-08T05:44:38.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.066+0000 7fb8c1196640 1 -- 172.21.15.45:0/3602746981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8bc0d3a50 msgr2=0x7fb8bc0cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.066+0000 7fb8c1196640 1 --2- 172.21.15.45:0/3602746981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8bc0d3a50 0x7fb8bc0cdff0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009d90 tx=0x7fb8b4004a40 comp rx=0 tx=0).stop 2024-05-08T05:44:38.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.066+0000 7fb8c1196640 1 -- 172.21.15.45:0/3602746981 shutdown_connections 2024-05-08T05:44:38.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.067+0000 7fb8c1196640 1 --2- 172.21.15.45:0/3602746981 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb89c0762d0 0x7fb89c078790 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.067+0000 7fb8c1196640 1 --2- 172.21.15.45:0/3602746981 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8bc0d3670 0x7fb8bc0b9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.067+0000 7fb8c1196640 1 --2- 172.21.15.45:0/3602746981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8bc0d3a50 0x7fb8bc0cdff0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.067+0000 7fb8c1196640 1 --2- 172.21.15.45:0/3602746981 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8bc0d0940 0x7fb8bc1f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.067+0000 7fb8c1196640 1 -- 172.21.15.45:0/3602746981 >> 172.21.15.45:0/3602746981 conn(0x7fb8bc09e390 msgr2=0x7fb8bc06d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:38.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.067+0000 7fb8c1196640 1 -- 172.21.15.45:0/3602746981 shutdown_connections 2024-05-08T05:44:38.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.067+0000 7fb8c1196640 1 -- 172.21.15.45:0/3602746981 wait complete. 2024-05-08T05:44:38.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:44:38.078 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-08T05:44:38.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.183+0000 7fa43335f640 1 Processor -- start 2024-05-08T05:44:38.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.183+0000 7fa43335f640 1 -- start start 2024-05-08T05:44:38.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.184+0000 7fa43335f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa42c06da00 0x7fa42c073800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.184+0000 7fa43335f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa42c0b65f0 con 0x7fa42c06d620 2024-05-08T05:44:38.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.184+0000 7fa43335f640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa42c0b5120 con 0x7fa42c1637b0 2024-05-08T05:44:38.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.184+0000 7fa43335f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa42c0afaa0 con 0x7fa42c06da00 2024-05-08T05:44:38.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.184+0000 7fa4310fb640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa42c06d620 0x7fa42c0af2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46536/0 (socket says 172.21.15.45:46536) 2024-05-08T05:44:38.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.184+0000 7fa4308fa640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa42c06da00 0x7fa42c073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.184+0000 7fa4310fb640 1 -- 172.21.15.45:0/2312995416 learned_addr learned my addr 172.21.15.45:0/2312995416 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:44:38.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.184+0000 7fa423fff640 1 -- 172.21.15.45:0/2312995416 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3199012040 0 0) 0x7fa42c0b65f0 con 0x7fa42c06d620 2024-05-08T05:44:38.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.184+0000 7fa423fff640 1 -- 172.21.15.45:0/2312995416 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa418003520 con 0x7fa42c06d620 2024-05-08T05:44:38.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.184+0000 7fa4308fa640 1 -- 172.21.15.45:0/2312995416 >> v1:172.21.15.63:6789/0 conn(0x7fa42c1637b0 legacy=0x7fa42c0b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.184+0000 7fa4308fa640 1 -- 172.21.15.45:0/2312995416 >> v1:172.21.15.45:6789/0 conn(0x7fa42c06d620 legacy=0x7fa42c0af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.184+0000 7fa4308fa640 1 -- 172.21.15.45:0/2312995416 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42c0b5400 con 0x7fa42c06da00 2024-05-08T05:44:38.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.185+0000 7fa4308fa640 1 --2- 172.21.15.45:0/2312995416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa42c06da00 0x7fa42c073800 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0097a0 tx=0x7fa41c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92d627aa085e1a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:38.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.185+0000 7fa423fff640 1 -- 172.21.15.45:0/2312995416 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c037070 con 0x7fa42c06da00 2024-05-08T05:44:38.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.185+0000 7fa423fff640 1 -- 172.21.15.45:0/2312995416 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa41c032440 con 0x7fa42c06da00 2024-05-08T05:44:38.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.185+0000 7fa423fff640 1 -- 172.21.15.45:0/2312995416 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c02ae60 con 0x7fa42c06da00 2024-05-08T05:44:38.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.186+0000 7fa43335f640 1 -- 172.21.15.45:0/2312995416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa42c06da00 msgr2=0x7fa42c073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.186+0000 7fa43335f640 1 --2- 172.21.15.45:0/2312995416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa42c06da00 0x7fa42c073800 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0097a0 tx=0x7fa41c02a110 comp rx=0 tx=0).stop 2024-05-08T05:44:38.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.186+0000 7fa43335f640 1 -- 172.21.15.45:0/2312995416 shutdown_connections 2024-05-08T05:44:38.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.186+0000 7fa43335f640 1 --2- 172.21.15.45:0/2312995416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa42c06da00 0x7fa42c073800 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.186+0000 7fa43335f640 1 -- 172.21.15.45:0/2312995416 >> 172.21.15.45:0/2312995416 conn(0x7fa42c0728c0 msgr2=0x7fa42c072cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:38.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.186+0000 7fa43335f640 1 -- 172.21.15.45:0/2312995416 shutdown_connections 2024-05-08T05:44:38.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.186+0000 7fa43335f640 1 -- 172.21.15.45:0/2312995416 wait complete. 2024-05-08T05:44:38.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.187+0000 7fa43335f640 1 Processor -- start 2024-05-08T05:44:38.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.187+0000 7fa43335f640 1 -- start start 2024-05-08T05:44:38.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.188+0000 7fa43335f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa42c06d620 0x7fa42c15ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.188+0000 7fa43335f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa42c06da00 0x7fa42c0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.188+0000 7fa43335f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa42c1637b0 0x7fa42c146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.188+0000 7fa43335f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c06e920 con 0x7fa42c06da00 2024-05-08T05:44:38.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.188+0000 7fa43335f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c0b5a50 con 0x7fa42c1637b0 2024-05-08T05:44:38.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.188+0000 7fa43335f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa42c06eaa0 con 0x7fa42c06d620 2024-05-08T05:44:38.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.188+0000 7fa4308fa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa42c06da00 0x7fa42c0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.188+0000 7fa4308fa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa42c06da00 0x7fa42c0cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55648/0 (socket says 172.21.15.45:55648) 2024-05-08T05:44:38.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.188+0000 7fa4308fa640 1 -- 172.21.15.45:0/2117980858 learned_addr learned my addr 172.21.15.45:0/2117980858 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:38.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.188+0000 7fa4310fb640 1 --2- 172.21.15.45:0/2117980858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa42c06d620 0x7fa42c15ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.188+0000 7fa4318fc640 1 --2- 172.21.15.45:0/2117980858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa42c1637b0 0x7fa42c146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.188+0000 7fa4308fa640 1 -- 172.21.15.45:0/2117980858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa42c06d620 msgr2=0x7fa42c15ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.188+0000 7fa4308fa640 1 --2- 172.21.15.45:0/2117980858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa42c06d620 0x7fa42c15ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.188+0000 7fa4308fa640 1 -- 172.21.15.45:0/2117980858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa42c1637b0 msgr2=0x7fa42c146930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.188+0000 7fa4308fa640 1 --2- 172.21.15.45:0/2117980858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa42c1637b0 0x7fa42c146930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.188+0000 7fa4308fa640 1 -- 172.21.15.45:0/2117980858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42c0cf3b0 con 0x7fa42c06da00 2024-05-08T05:44:38.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.189+0000 7fa4308fa640 1 --2- 172.21.15.45:0/2117980858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa42c06da00 0x7fa42c0cee40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa41c035040 tx=0x7fa41c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:38.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.189+0000 7fa421ffb640 1 -- 172.21.15.45:0/2117980858 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c037040 con 0x7fa42c06da00 2024-05-08T05:44:38.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.189+0000 7fa421ffb640 1 -- 172.21.15.45:0/2117980858 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa41c030580 con 0x7fa42c06da00 2024-05-08T05:44:38.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.189+0000 7fa43335f640 1 -- 172.21.15.45:0/2117980858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42c0cf650 con 0x7fa42c06da00 2024-05-08T05:44:38.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.189+0000 7fa421ffb640 1 -- 172.21.15.45:0/2117980858 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c030990 con 0x7fa42c06da00 2024-05-08T05:44:38.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.189+0000 7fa43335f640 1 -- 172.21.15.45:0/2117980858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa42c0cfef0 con 0x7fa42c06da00 2024-05-08T05:44:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.191+0000 7fa421ffb640 1 -- 172.21.15.45:0/2117980858 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa41c044020 con 0x7fa42c06da00 2024-05-08T05:44:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.191+0000 7fa421ffb640 1 --2- 172.21.15.45:0/2117980858 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa4100762d0 0x7fa410078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.191+0000 7fa421ffb640 1 -- 172.21.15.45:0/2117980858 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fa41c0b7b30 con 0x7fa42c06da00 2024-05-08T05:44:38.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.192+0000 7fa4310fb640 1 --2- 172.21.15.45:0/2117980858 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa4100762d0 0x7fa410078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.192+0000 7fa436077740 1 Processor -- start 2024-05-08T05:44:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.192+0000 7fa436077740 1 -- start start 2024-05-08T05:44:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.192+0000 7fa4310fb640 1 --2- 172.21.15.45:0/2117980858 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa4100762d0 0x7fa410078790 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa428005880 tx=0x7fa428009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.193+0000 7fa436077740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec13b9160 0x55cec136baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.193+0000 7fa436077740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cec139c280 0x55cec139b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.193+0000 7fa436077740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec139bb40 0x55cec1370950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.193+0000 7fa436077740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cec1338690 con 0x55cec139bb40 2024-05-08T05:44:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.193+0000 7fa436077740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cec12d5f30 con 0x55cec13b9160 2024-05-08T05:44:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.193+0000 7fa436077740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55cec128eb20 con 0x55cec139c280 2024-05-08T05:44:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.193+0000 7fa4318fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec139bb40 0x55cec1370950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.193+0000 7fa4318fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec139bb40 0x55cec1370950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55662/0 (socket says 172.21.15.45:55662) 2024-05-08T05:44:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.193+0000 7fa4318fc640 1 -- 172.21.15.45:0/3095400549 learned_addr learned my addr 172.21.15.45:0/3095400549 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.193+0000 7fa4310fb640 1 --2- 172.21.15.45:0/3095400549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec13b9160 0x55cec136baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.193+0000 7fa4308fa640 1 --2- 172.21.15.45:0/3095400549 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cec139c280 0x55cec139b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.193+0000 7fa4318fc640 1 -- 172.21.15.45:0/3095400549 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cec139c280 msgr2=0x55cec139b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.193+0000 7fa4318fc640 1 --2- 172.21.15.45:0/3095400549 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cec139c280 0x55cec139b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.193+0000 7fa4318fc640 1 -- 172.21.15.45:0/3095400549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec13b9160 msgr2=0x55cec136baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.193+0000 7fa4318fc640 1 --2- 172.21.15.45:0/3095400549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec13b9160 0x55cec136baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.193+0000 7fa4318fc640 1 -- 172.21.15.45:0/3095400549 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec1286e60 con 0x55cec139bb40 2024-05-08T05:44:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.193+0000 7fa4310fb640 1 --2- 172.21.15.45:0/3095400549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec13b9160 0x55cec136baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.193+0000 7fa4318fc640 1 --2- 172.21.15.45:0/3095400549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec139bb40 0x55cec1370950 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa424001c50 tx=0x7fa424002110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2114d0483699b20b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.194+0000 7fa416ffd640 1 -- 172.21.15.45:0/3095400549 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa424019070 con 0x55cec139bb40 2024-05-08T05:44:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.194+0000 7fa416ffd640 1 -- 172.21.15.45:0/3095400549 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa424003b80 con 0x55cec139bb40 2024-05-08T05:44:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.194+0000 7fa416ffd640 1 -- 172.21.15.45:0/3095400549 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42400e860 con 0x55cec139bb40 2024-05-08T05:44:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.194+0000 7fa436077740 1 -- 172.21.15.45:0/3095400549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec139bb40 msgr2=0x55cec1370950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.194+0000 7fa436077740 1 --2- 172.21.15.45:0/3095400549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec139bb40 0x55cec1370950 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa424001c50 tx=0x7fa424002110 comp rx=0 tx=0).stop 2024-05-08T05:44:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.194+0000 7fa436077740 1 -- 172.21.15.45:0/3095400549 shutdown_connections 2024-05-08T05:44:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.194+0000 7fa436077740 1 --2- 172.21.15.45:0/3095400549 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cec139c280 0x55cec139b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.194+0000 7fa436077740 1 --2- 172.21.15.45:0/3095400549 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec139bb40 0x55cec1370950 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.194+0000 7fa436077740 1 --2- 172.21.15.45:0/3095400549 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec13b9160 0x55cec136baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.194+0000 7fa436077740 1 -- 172.21.15.45:0/3095400549 >> 172.21.15.45:0/3095400549 conn(0x55cec136dd60 msgr2=0x55cec13b6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.194+0000 7fa436077740 1 -- 172.21.15.45:0/3095400549 shutdown_connections 2024-05-08T05:44:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.194+0000 7fa436077740 1 -- 172.21.15.45:0/3095400549 wait complete. 2024-05-08T05:44:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.195+0000 7fa436077740 1 -- start start 2024-05-08T05:44:38.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.195+0000 7fa436077740 1 Processor -- start 2024-05-08T05:44:38.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.196+0000 7fa436077740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec139c280 0x55cec139d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.196+0000 7fa436077740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cec13b9160 0x55cec138bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.196+0000 7fa436077740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec13b7c50 0x55cec139a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.196+0000 7fa436077740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cec12925e0 con 0x55cec139c280 2024-05-08T05:44:38.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.196+0000 7fa436077740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cec12d1ef0 con 0x55cec13b7c50 2024-05-08T05:44:38.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.196+0000 7fa436077740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55cec136ff10 con 0x55cec13b9160 2024-05-08T05:44:38.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.196+0000 7fa436077740 10 client.0 _ll_register_callbacks cb 0x55cec128d320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:44:38.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.196+0000 7fa4310fb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec139c280 0x55cec139d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.196+0000 7fa4310fb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec139c280 0x55cec139d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55680/0 (socket says 172.21.15.45:55680) 2024-05-08T05:44:38.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.196+0000 7fa4310fb640 1 -- 172.21.15.45:0/3899288942 learned_addr learned my addr 172.21.15.45:0/3899288942 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:38.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.196+0000 7fa4318fc640 1 --2- 172.21.15.45:0/3899288942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec13b7c50 0x55cec139a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.196+0000 7fa4308fa640 1 --2- 172.21.15.45:0/3899288942 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cec13b9160 0x55cec138bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.196+0000 7fa4310fb640 1 -- 172.21.15.45:0/3899288942 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cec13b9160 msgr2=0x55cec138bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:38.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.196+0000 7fa4310fb640 1 --2- 172.21.15.45:0/3899288942 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cec13b9160 0x55cec138bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.196+0000 7fa4310fb640 1 -- 172.21.15.45:0/3899288942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec13b7c50 msgr2=0x55cec139a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:38.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.196+0000 7fa4310fb640 1 --2- 172.21.15.45:0/3899288942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec13b7c50 0x55cec139a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.196+0000 7fa4310fb640 1 -- 172.21.15.45:0/3899288942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec10b17d0 con 0x55cec139c280 2024-05-08T05:44:38.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.196+0000 7fa4318fc640 1 --2- 172.21.15.45:0/3899288942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec13b7c50 0x55cec139a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:38.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.196+0000 7fa4310fb640 1 --2- 172.21.15.45:0/3899288942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec139c280 0x55cec139d150 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa428005f70 tx=0x7fa428000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb79636d2eff20f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:38.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.196+0000 7fa40bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:44:38.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.197+0000 7fa40bfff640 1 -- 172.21.15.45:0/3899288942 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42800eb90 con 0x55cec139c280 2024-05-08T05:44:38.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.197+0000 7fa40bfff640 1 -- 172.21.15.45:0/3899288942 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fa42800ed30 con 0x55cec139c280 2024-05-08T05:44:38.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.197+0000 7fa436077740 1 -- 172.21.15.45:0/3899288942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa424003b80 con 0x55cec139c280 2024-05-08T05:44:38.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.197+0000 7fa40bfff640 1 -- 172.21.15.45:0/3899288942 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa428022430 con 0x55cec139c280 2024-05-08T05:44:38.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.197+0000 7fa436077740 10 client.5973 fetch_fsmap learned FSMap version 35 2024-05-08T05:44:38.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.197+0000 7fa436077740 1 -- 172.21.15.45:0/3899288942 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55cec0fe73f0 con 0x55cec139c280 2024-05-08T05:44:38.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.197+0000 7fa40bfff640 1 -- 172.21.15.45:0/3899288942 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa424003b80 con 0x55cec139c280 2024-05-08T05:44:38.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.197+0000 7fa40bfff640 1 -- 172.21.15.45:0/3899288942 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7fa4280225d0 con 0x55cec139c280 2024-05-08T05:44:38.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.197+0000 7fa436077740 10 client.5973 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T05:44:38.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.197+0000 7fa436077740 10 client.5973 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T05:44:38.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.197+0000 7fa436077740 20 client.5973 populate_metadata read hostname 'smithi045' 2024-05-08T05:44:38.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.197+0000 7fa436077740 1 --2- 172.21.15.45:0/3899288942 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55cec13c61e0 0x55cec13c6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.197+0000 7fa436077740 4 client.5973 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:44:38.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.197+0000 7fa436077740 1 -- 172.21.15.45:0/3899288942 --> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cec12e3b40 con 0x55cec13c61e0 2024-05-08T05:44:38.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.197+0000 7fa4308fa640 1 --2- 172.21.15.45:0/3899288942 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55cec13c61e0 0x55cec13c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.198+0000 7fa4308fa640 1 --2- 172.21.15.45:0/3899288942 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55cec13c61e0 0x55cec13c6610 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db56a8a33f5bdb3d server_cookie=4f8bc46a03ea5cd5 in_seq=0 out_seq=0 2024-05-08T05:44:38.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.198+0000 7fa40bfff640 10 client.5973 ms_handle_connect on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:38.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.198+0000 7fa40bfff640 1 -- 172.21.15.45:0/3899288942 <== mds.0 v2:172.21.15.45:6833/1461822016 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cec12e3b40 con 0x55cec13c61e0 2024-05-08T05:44:38.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.198+0000 7fa40bfff640 10 client.5973 handle_command_reply: tid=0 2024-05-08T05:44:38.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.198+0000 7fa436077740 1 client.5973 shutdown 2024-05-08T05:44:38.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.198+0000 7fa436077740 1 -- 172.21.15.45:0/3899288942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec139c280 msgr2=0x55cec139d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.198+0000 7fa436077740 1 --2- 172.21.15.45:0/3899288942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec139c280 0x55cec139d150 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa428005f70 tx=0x7fa428000ed0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.198+0000 7fa436077740 1 -- 172.21.15.45:0/3899288942 shutdown_connections 2024-05-08T05:44:38.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.198+0000 7fa436077740 1 --2- 172.21.15.45:0/3899288942 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55cec13c61e0 0x55cec13c6610 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.198+0000 7fa436077740 1 --2- 172.21.15.45:0/3899288942 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cec13b9160 0x55cec138bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.198+0000 7fa436077740 1 --2- 172.21.15.45:0/3899288942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec13b7c50 0x55cec139a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.198+0000 7fa40bfff640 0 client.5973 ms_handle_reset on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:38.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.198+0000 7fa436077740 1 --2- 172.21.15.45:0/3899288942 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec139c280 0x55cec139d150 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.198+0000 7fa436077740 1 -- 172.21.15.45:0/3899288942 >> 172.21.15.45:0/3899288942 conn(0x55cec139bb40 msgr2=0x55cec136b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:38.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.199+0000 7fa436077740 1 -- 172.21.15.45:0/3899288942 shutdown_connections 2024-05-08T05:44:38.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.199+0000 7fa436077740 1 -- 172.21.15.45:0/3899288942 wait complete. 2024-05-08T05:44:38.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.199+0000 7fa436077740 20 client.5973 trim_cache size 0 max 16384 2024-05-08T05:44:38.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.217+0000 7fa436077740 1 Processor -- start 2024-05-08T05:44:38.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.217+0000 7fa436077740 1 -- start start 2024-05-08T05:44:38.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.217+0000 7fa436077740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec139c280 0x55cec13b9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.217+0000 7fa436077740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec13b7c50 0x55cec13b80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.217+0000 7fa436077740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cec13b9160 0x55cec13b8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.217+0000 7fa436077740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c202050 con 0x55cec13b7c50 2024-05-08T05:44:38.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.217+0000 7fa436077740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c201030 con 0x55cec139c280 2024-05-08T05:44:38.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.217+0000 7fa436077740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa42c148340 con 0x55cec13b9160 2024-05-08T05:44:38.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.218+0000 7fa4308fa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec13b7c50 0x55cec13b80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.218+0000 7fa4308fa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec13b7c50 0x55cec13b80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55682/0 (socket says 172.21.15.45:55682) 2024-05-08T05:44:38.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.218+0000 7fa4308fa640 1 -- 172.21.15.45:0/610264108 learned_addr learned my addr 172.21.15.45:0/610264108 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:38.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.218+0000 7fa4310fb640 1 --2- 172.21.15.45:0/610264108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec139c280 0x55cec13b9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.218+0000 7fa4318fc640 1 --2- 172.21.15.45:0/610264108 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cec13b9160 0x55cec13b8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.218+0000 7fa4308fa640 1 -- 172.21.15.45:0/610264108 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cec13b9160 msgr2=0x55cec13b8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:38.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.218+0000 7fa4308fa640 1 --2- 172.21.15.45:0/610264108 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cec13b9160 0x55cec13b8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.218+0000 7fa4308fa640 1 -- 172.21.15.45:0/610264108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec139c280 msgr2=0x55cec13b9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:38.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.218+0000 7fa4308fa640 1 --2- 172.21.15.45:0/610264108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec139c280 0x55cec13b9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.218+0000 7fa4308fa640 1 -- 172.21.15.45:0/610264108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec0fa70f0 con 0x55cec13b7c50 2024-05-08T05:44:38.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.218+0000 7fa4308fa640 1 --2- 172.21.15.45:0/610264108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec13b7c50 0x55cec13b80d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa41c080b90 tx=0x7fa41c0b5da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794cf47bf15be483 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:38.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.218+0000 7fa4167fc640 1 -- 172.21.15.45:0/610264108 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c0b2020 con 0x55cec13b7c50 2024-05-08T05:44:38.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.218+0000 7fa4167fc640 1 -- 172.21.15.45:0/610264108 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa41c080710 con 0x55cec13b7c50 2024-05-08T05:44:38.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.218+0000 7fa4167fc640 1 -- 172.21.15.45:0/610264108 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c081300 con 0x55cec13b7c50 2024-05-08T05:44:38.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.219+0000 7fa436077740 1 -- 172.21.15.45:0/610264108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec13b7c50 msgr2=0x55cec13b80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.219+0000 7fa436077740 1 --2- 172.21.15.45:0/610264108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec13b7c50 0x55cec13b80d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa41c080b90 tx=0x7fa41c0b5da0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.219+0000 7fa436077740 1 -- 172.21.15.45:0/610264108 shutdown_connections 2024-05-08T05:44:38.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.219+0000 7fa436077740 1 --2- 172.21.15.45:0/610264108 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cec13b9160 0x55cec13b8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.219+0000 7fa436077740 1 --2- 172.21.15.45:0/610264108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec13b7c50 0x55cec13b80d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.219+0000 7fa436077740 1 --2- 172.21.15.45:0/610264108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec139c280 0x55cec13b9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.219+0000 7fa436077740 1 -- 172.21.15.45:0/610264108 >> 172.21.15.45:0/610264108 conn(0x55cec139bb40 msgr2=0x55cec139b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:38.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.219+0000 7fa436077740 1 -- 172.21.15.45:0/610264108 shutdown_connections 2024-05-08T05:44:38.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.219+0000 7fa436077740 1 -- 172.21.15.45:0/610264108 wait complete. 2024-05-08T05:44:38.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.219+0000 7fa436077740 1 -- start start 2024-05-08T05:44:38.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.219+0000 7fa436077740 1 Processor -- start 2024-05-08T05:44:38.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.220+0000 7fa436077740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cec139c280 0x55cec1432bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.220+0000 7fa436077740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec13b7c50 0x55cec139c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.220+0000 7fa436077740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec13b9160 0x55cec13b8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.220+0000 7fa436077740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c161540 con 0x55cec13b7c50 2024-05-08T05:44:38.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.220+0000 7fa436077740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cec139cf30 con 0x55cec13b9160 2024-05-08T05:44:38.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.220+0000 7fa436077740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55cec139a070 con 0x55cec139c280 2024-05-08T05:44:38.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.220+0000 7fa436077740 10 client.0 _ll_register_callbacks cb 0x7fa428007cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:44:38.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.220+0000 7fa4308fa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec13b7c50 0x55cec139c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.220+0000 7fa4308fa640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec13b7c50 0x55cec139c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55696/0 (socket says 172.21.15.45:55696) 2024-05-08T05:44:38.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.220+0000 7fa4308fa640 1 -- 172.21.15.45:0/796566217 learned_addr learned my addr 172.21.15.45:0/796566217 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:44:38.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa4310fb640 1 --2- 172.21.15.45:0/796566217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cec139c280 0x55cec1432bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa4318fc640 1 --2- 172.21.15.45:0/796566217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec13b9160 0x55cec13b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa4308fa640 1 -- 172.21.15.45:0/796566217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cec139c280 msgr2=0x55cec1432bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:38.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa4308fa640 1 --2- 172.21.15.45:0/796566217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cec139c280 0x55cec1432bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa4308fa640 1 -- 172.21.15.45:0/796566217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec13b9160 msgr2=0x55cec13b8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:44:38.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa4308fa640 1 --2- 172.21.15.45:0/796566217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec13b9160 0x55cec13b8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa4308fa640 1 -- 172.21.15.45:0/796566217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec1433110 con 0x55cec13b7c50 2024-05-08T05:44:38.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa4318fc640 1 --2- 172.21.15.45:0/796566217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec13b9160 0x55cec13b8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:44:38.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa4308fa640 1 --2- 172.21.15.45:0/796566217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec13b7c50 0x55cec139c770 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa41c000c00 tx=0x7fa41c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1574496d353099d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:38.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa40bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:44:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa40bfff640 1 -- 172.21.15.45:0/796566217 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c0a1070 con 0x55cec13b7c50 2024-05-08T05:44:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa40bfff640 1 -- 172.21.15.45:0/796566217 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fa41c030b30 con 0x55cec13b7c50 2024-05-08T05:44:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa436077740 1 -- 172.21.15.45:0/796566217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa41c080710 con 0x55cec13b7c50 2024-05-08T05:44:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa40bfff640 1 -- 172.21.15.45:0/796566217 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa41c0b2040 con 0x55cec13b7c50 2024-05-08T05:44:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa436077740 10 client.5979 fetch_fsmap learned FSMap version 35 2024-05-08T05:44:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa436077740 1 -- 172.21.15.45:0/796566217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55cec1372d40 con 0x55cec13b7c50 2024-05-08T05:44:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa40bfff640 1 -- 172.21.15.45:0/796566217 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa41c080710 con 0x55cec13b7c50 2024-05-08T05:44:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa40bfff640 1 -- 172.21.15.45:0/796566217 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7fa41c03a070 con 0x55cec13b7c50 2024-05-08T05:44:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa436077740 10 client.5979 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T05:44:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa436077740 10 client.5979 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T05:44:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa436077740 20 client.5979 populate_metadata read hostname 'smithi045' 2024-05-08T05:44:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa436077740 1 --2- 172.21.15.45:0/796566217 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55cec13b85d0 0x55cec13ac420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa436077740 4 client.5979 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:44:38.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.221+0000 7fa436077740 1 -- 172.21.15.45:0/796566217 --> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cec12dd9b0 con 0x55cec13b85d0 2024-05-08T05:44:38.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.222+0000 7fa4310fb640 1 --2- 172.21.15.45:0/796566217 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55cec13b85d0 0x55cec13ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.222+0000 7fa40bfff640 10 client.5979 ms_handle_connect on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:38.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.222+0000 7fa4310fb640 1 --2- 172.21.15.45:0/796566217 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55cec13b85d0 0x55cec13ac420 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b6de3a5c94334c7 server_cookie=3ac1081942f22370 in_seq=0 out_seq=0 2024-05-08T05:44:38.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.222+0000 7fa40bfff640 1 -- 172.21.15.45:0/796566217 <== mds.0 v2:172.21.15.45:6833/1461822016 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55cec12dd9b0 con 0x55cec13b85d0 2024-05-08T05:44:38.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.222+0000 7fa40bfff640 10 client.5979 handle_command_reply: tid=0 2024-05-08T05:44:38.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.222+0000 7fa436077740 1 client.5979 shutdown 2024-05-08T05:44:38.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.222+0000 7fa436077740 1 -- 172.21.15.45:0/796566217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec13b7c50 msgr2=0x55cec139c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.222+0000 7fa436077740 1 --2- 172.21.15.45:0/796566217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec13b7c50 0x55cec139c770 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa41c000c00 tx=0x7fa41c008040 comp rx=0 tx=0).stop 2024-05-08T05:44:38.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.222+0000 7fa436077740 1 -- 172.21.15.45:0/796566217 shutdown_connections 2024-05-08T05:44:38.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.222+0000 7fa436077740 1 --2- 172.21.15.45:0/796566217 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55cec13b85d0 0x55cec13ac420 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.222+0000 7fa436077740 1 --2- 172.21.15.45:0/796566217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cec13b9160 0x55cec13b8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.222+0000 7fa436077740 1 --2- 172.21.15.45:0/796566217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cec13b7c50 0x55cec139c770 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.222+0000 7fa40bfff640 0 client.5979 ms_handle_reset on v2:172.21.15.45:6833/1461822016 2024-05-08T05:44:38.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.222+0000 7fa436077740 1 --2- 172.21.15.45:0/796566217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cec139c280 0x55cec1432bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.222+0000 7fa436077740 1 -- 172.21.15.45:0/796566217 >> 172.21.15.45:0/796566217 conn(0x55cec139bb40 msgr2=0x55cec139b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:38.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.222+0000 7fa436077740 1 -- 172.21.15.45:0/796566217 shutdown_connections 2024-05-08T05:44:38.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.222+0000 7fa436077740 1 -- 172.21.15.45:0/796566217 wait complete. 2024-05-08T05:44:38.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.223+0000 7fa436077740 20 client.5979 trim_cache size 0 max 16384 2024-05-08T05:44:38.230 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:44:38.230 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T05:44:38.230 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-05-08T05:44:38.230 INFO:teuthology.orchestra.run.smithi045.stdout: "30d4344c-9fc3-4a2f-ac85-063a31a4def8": { 2024-05-08T05:44:38.231 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-05-08T05:44:38.231 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "30d4344c-9fc3-4a2f-ac85-063a31a4def8", 2024-05-08T05:44:38.231 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,repair,force" 2024-05-08T05:44:38.231 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:44:38.231 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:44:38.231 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:44:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.223+0000 7fa43335f640 1 -- 172.21.15.45:0/2117980858 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa4100762d0 msgr2=0x7fa410078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.223+0000 7fa43335f640 1 --2- 172.21.15.45:0/2117980858 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa4100762d0 0x7fa410078790 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa428005880 tx=0x7fa428009040 comp rx=0 tx=0).stop 2024-05-08T05:44:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.223+0000 7fa43335f640 1 -- 172.21.15.45:0/2117980858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa42c06da00 msgr2=0x7fa42c0cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.223+0000 7fa43335f640 1 --2- 172.21.15.45:0/2117980858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa42c06da00 0x7fa42c0cee40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa41c035040 tx=0x7fa41c0046c0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.223+0000 7fa43335f640 1 -- 172.21.15.45:0/2117980858 shutdown_connections 2024-05-08T05:44:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.223+0000 7fa43335f640 1 --2- 172.21.15.45:0/2117980858 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa4100762d0 0x7fa410078790 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.223+0000 7fa43335f640 1 --2- 172.21.15.45:0/2117980858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa42c1637b0 0x7fa42c146930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.223+0000 7fa43335f640 1 --2- 172.21.15.45:0/2117980858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa42c06da00 0x7fa42c0cee40 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.223+0000 7fa43335f640 1 --2- 172.21.15.45:0/2117980858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa42c06d620 0x7fa42c15ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.223+0000 7fa43335f640 1 -- 172.21.15.45:0/2117980858 >> 172.21.15.45:0/2117980858 conn(0x7fa42c0728c0 msgr2=0x7fa42c0af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.223+0000 7fa43335f640 1 -- 172.21.15.45:0/2117980858 shutdown_connections 2024-05-08T05:44:38.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:44:38.223+0000 7fa43335f640 1 -- 172.21.15.45:0/2117980858 wait complete. 2024-05-08T05:44:38.234 INFO:tasks.cephfs.filesystem:scrub status for tag:30d4344c-9fc3-4a2f-ac85-063a31a4def8 - {'path': '/', 'tag': '30d4344c-9fc3-4a2f-ac85-063a31a4def8', 'options': 'recursive,repair,force'} 2024-05-08T05:45:04.484 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:45:04.488 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:45:08.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:08.236 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-05-08T05:45:08.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.354+0000 7f3b7630a640 1 Processor -- start 2024-05-08T05:45:08.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.354+0000 7f3b7630a640 1 -- start start 2024-05-08T05:45:08.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.355+0000 7f3b7630a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b700b21f0 0x7f3b7006e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.355+0000 7f3b7630a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3b700cfea0 0x7f3b701633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.355+0000 7f3b7630a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b700d0500 con 0x7f3b70162f50 2024-05-08T05:45:08.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.355+0000 7f3b7630a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3b701638f0 con 0x7f3b700cfea0 2024-05-08T05:45:08.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.355+0000 7f3b7630a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3b7006d810 con 0x7f3b700b21f0 2024-05-08T05:45:08.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.355+0000 7f3b6f7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3b70162f50 0x7f3b700cf790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42840/0 (socket says 172.21.15.45:42840) 2024-05-08T05:45:08.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.355+0000 7f3b6f7fe640 1 -- 172.21.15.45:0/1790458036 learned_addr learned my addr 172.21.15.45:0/1790458036 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:08.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.355+0000 7f3b6ffff640 1 --2- 172.21.15.45:0/1790458036 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b700b21f0 0x7f3b7006e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.355+0000 7f3b6effd640 1 -- 172.21.15.45:0/1790458036 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3018950481 0 0) 0x7f3b700d0500 con 0x7f3b70162f50 2024-05-08T05:45:08.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.355+0000 7f3b748a7640 1 --2- 172.21.15.45:0/1790458036 >> v2:172.21.15.63:3300/0 conn(0x7f3b700cfea0 0x7f3b701633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.355+0000 7f3b6effd640 1 -- 172.21.15.45:0/1790458036 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b58003520 con 0x7f3b70162f50 2024-05-08T05:45:08.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6effd640 1 -- 172.21.15.45:0/1790458036 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2253525688 0 0) 0x7f3b58003520 con 0x7f3b70162f50 2024-05-08T05:45:08.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6effd640 1 -- 172.21.15.45:0/1790458036 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b700b21f0 msgr2=0x7f3b7006e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6effd640 1 --2- 172.21.15.45:0/1790458036 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b700b21f0 0x7f3b7006e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6effd640 1 -- 172.21.15.45:0/1790458036 >> v2:172.21.15.63:3300/0 conn(0x7f3b700cfea0 msgr2=0x7f3b701633b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6effd640 1 --2- 172.21.15.45:0/1790458036 >> v2:172.21.15.63:3300/0 conn(0x7f3b700cfea0 0x7f3b701633b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6effd640 1 -- 172.21.15.45:0/1790458036 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7006da60 con 0x7f3b70162f50 2024-05-08T05:45:08.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6effd640 1 -- 172.21.15.45:0/1790458036 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f3b640016f0 con 0x7f3b70162f50 2024-05-08T05:45:08.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6effd640 1 -- 172.21.15.45:0/1790458036 >> v1:172.21.15.45:6789/0 conn(0x7f3b70162f50 legacy=0x7f3b700cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6effd640 1 --2- 172.21.15.45:0/1790458036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b58003a00 0x7f3b58003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6effd640 1 --2- 172.21.15.45:0/1790458036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b58004800 0x7f3b58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6effd640 1 --2- 172.21.15.45:0/1790458036 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b700b21f0 0x7f3b580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6effd640 1 -- 172.21.15.45:0/1790458036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7006d810 con 0x7f3b58004800 2024-05-08T05:45:08.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6effd640 1 -- 172.21.15.45:0/1790458036 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5800ba70 con 0x7f3b58003a00 2024-05-08T05:45:08.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6effd640 1 -- 172.21.15.45:0/1790458036 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3b5800bc70 con 0x7f3b700b21f0 2024-05-08T05:45:08.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6f7fe640 1 --2- 172.21.15.45:0/1790458036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b58004800 0x7f3b58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6ffff640 1 --2- 172.21.15.45:0/1790458036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b58003a00 0x7f3b58003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b748a7640 1 --2- 172.21.15.45:0/1790458036 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b700b21f0 0x7f3b580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6f7fe640 1 -- 172.21.15.45:0/1790458036 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b700b21f0 msgr2=0x7f3b580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6f7fe640 1 --2- 172.21.15.45:0/1790458036 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b700b21f0 0x7f3b580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6f7fe640 1 -- 172.21.15.45:0/1790458036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b58003a00 msgr2=0x7f3b58003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6f7fe640 1 --2- 172.21.15.45:0/1790458036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b58003a00 0x7f3b58003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.356+0000 7f3b6f7fe640 1 -- 172.21.15.45:0/1790458036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b58003520 con 0x7f3b58004800 2024-05-08T05:45:08.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.357+0000 7f3b6f7fe640 1 --2- 172.21.15.45:0/1790458036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b58004800 0x7f3b58004c80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3b64007fd0 tx=0x7f3b58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575407fcbe62264f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:08.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.357+0000 7f3b6effd640 1 -- 172.21.15.45:0/1790458036 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64007d50 con 0x7f3b58004800 2024-05-08T05:45:08.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.357+0000 7f3b6effd640 1 -- 172.21.15.45:0/1790458036 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b64005af0 con 0x7f3b58004800 2024-05-08T05:45:08.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.357+0000 7f3b6effd640 1 -- 172.21.15.45:0/1790458036 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64007070 con 0x7f3b58004800 2024-05-08T05:45:08.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.358+0000 7f3b7630a640 1 -- 172.21.15.45:0/1790458036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b58004800 msgr2=0x7f3b58004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.358+0000 7f3b7630a640 1 --2- 172.21.15.45:0/1790458036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b58004800 0x7f3b58004c80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3b64007fd0 tx=0x7f3b58005830 comp rx=0 tx=0).stop 2024-05-08T05:45:08.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.358+0000 7f3b748a7640 1 -- 172.21.15.45:0/1790458036 reap_dead start 2024-05-08T05:45:08.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.358+0000 7f3b7630a640 1 -- 172.21.15.45:0/1790458036 shutdown_connections 2024-05-08T05:45:08.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.358+0000 7f3b7630a640 1 -- 172.21.15.45:0/1790458036 >> 172.21.15.45:0/1790458036 conn(0x7f3b7009ba40 msgr2=0x7f3b7009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:08.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.358+0000 7f3b7630a640 1 -- 172.21.15.45:0/1790458036 shutdown_connections 2024-05-08T05:45:08.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.358+0000 7f3b7630a640 1 -- 172.21.15.45:0/1790458036 wait complete. 2024-05-08T05:45:08.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.359+0000 7f3b7630a640 1 Processor -- start 2024-05-08T05:45:08.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.359+0000 7f3b7630a640 1 -- start start 2024-05-08T05:45:08.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.360+0000 7f3b7630a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b58004800 0x7f3b700b8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.360+0000 7f3b7630a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b700c6960 0x7f3b7015a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.360+0000 7f3b7630a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b700b37a0 0x7f3b700b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.360+0000 7f3b7630a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3b700b25d0 con 0x7f3b700b37a0 2024-05-08T05:45:08.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.360+0000 7f3b7630a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7006e0b0 con 0x7f3b700c6960 2024-05-08T05:45:08.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.360+0000 7f3b7630a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3b7006e540 con 0x7f3b58004800 2024-05-08T05:45:08.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.360+0000 7f3b748a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b700b37a0 0x7f3b700b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.360+0000 7f3b748a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b700b37a0 0x7f3b700b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35332/0 (socket says 172.21.15.45:35332) 2024-05-08T05:45:08.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.360+0000 7f3b748a7640 1 -- 172.21.15.45:0/1936279565 learned_addr learned my addr 172.21.15.45:0/1936279565 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:08.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.360+0000 7f3b6ffff640 1 --2- 172.21.15.45:0/1936279565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b58004800 0x7f3b700b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.361+0000 7f3b6f7fe640 1 --2- 172.21.15.45:0/1936279565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b700c6960 0x7f3b7015a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.361+0000 7f3b748a7640 1 -- 172.21.15.45:0/1936279565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b58004800 msgr2=0x7f3b700b8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.361+0000 7f3b748a7640 1 --2- 172.21.15.45:0/1936279565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b58004800 0x7f3b700b8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.361+0000 7f3b748a7640 1 -- 172.21.15.45:0/1936279565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b700c6960 msgr2=0x7f3b7015a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.361+0000 7f3b748a7640 1 --2- 172.21.15.45:0/1936279565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b700c6960 0x7f3b7015a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.361+0000 7f3b748a7640 1 -- 172.21.15.45:0/1936279565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b700c71b0 con 0x7f3b700b37a0 2024-05-08T05:45:08.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.361+0000 7f3b6ffff640 1 --2- 172.21.15.45:0/1936279565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b58004800 0x7f3b700b8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:08.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.361+0000 7f3b748a7640 1 --2- 172.21.15.45:0/1936279565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b700b37a0 0x7f3b700b3c30 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3b700cf470 tx=0x7f3b680115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:08.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.361+0000 7f3b6cff9640 1 -- 172.21.15.45:0/1936279565 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68011e40 con 0x7f3b700b37a0 2024-05-08T05:45:08.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.361+0000 7f3b6cff9640 1 -- 172.21.15.45:0/1936279565 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b6801b070 con 0x7f3b700b37a0 2024-05-08T05:45:08.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.361+0000 7f3b7630a640 1 -- 172.21.15.45:0/1936279565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b700c7490 con 0x7f3b700b37a0 2024-05-08T05:45:08.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.361+0000 7f3b6cff9640 1 -- 172.21.15.45:0/1936279565 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68003990 con 0x7f3b700b37a0 2024-05-08T05:45:08.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.361+0000 7f3b7630a640 1 -- 172.21.15.45:0/1936279565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b700c7a50 con 0x7f3b700b37a0 2024-05-08T05:45:08.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.363+0000 7f3b6cff9640 1 -- 172.21.15.45:0/1936279565 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3b68005d10 con 0x7f3b700b37a0 2024-05-08T05:45:08.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.363+0000 7f3b6cff9640 1 --2- 172.21.15.45:0/1936279565 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3b5c0762d0 0x7f3b5c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.364+0000 7f3b6cff9640 1 -- 172.21.15.45:0/1936279565 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f3b68097e80 con 0x7f3b700b37a0 2024-05-08T05:45:08.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.364+0000 7f3b6ffff640 1 --2- 172.21.15.45:0/1936279565 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3b5c0762d0 0x7f3b5c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.364+0000 7f3b6ffff640 1 --2- 172.21.15.45:0/1936279565 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3b5c0762d0 0x7f3b5c078790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3b60009a60 tx=0x7f3b60008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:08.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.364+0000 7f3b79024740 1 Processor -- start 2024-05-08T05:45:08.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.364+0000 7f3b79024740 1 -- start start 2024-05-08T05:45:08.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.365+0000 7f3b79024740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5617509e3160 0x561750995af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.365+0000 7f3b79024740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 0x5617509c5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.365+0000 7f3b79024740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617509c5b40 0x56175099a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.365+0000 7f3b79024740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561750962690 con 0x5617509c6280 2024-05-08T05:45:08.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.365+0000 7f3b79024740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5617508fff30 con 0x5617509c5b40 2024-05-08T05:45:08.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.365+0000 7f3b79024740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5617508b8b20 con 0x5617509e3160 2024-05-08T05:45:08.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.365+0000 7f3b6f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 0x5617509c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.365+0000 7f3b6f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 0x5617509c5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35348/0 (socket says 172.21.15.45:35348) 2024-05-08T05:45:08.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.365+0000 7f3b6f7fe640 1 -- 172.21.15.45:0/4123685684 learned_addr learned my addr 172.21.15.45:0/4123685684 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:08.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.365+0000 7f3b6ffff640 1 --2- 172.21.15.45:0/4123685684 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5617509e3160 0x561750995af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.365+0000 7f3b748a7640 1 --2- 172.21.15.45:0/4123685684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617509c5b40 0x56175099a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.365+0000 7f3b6f7fe640 1 -- 172.21.15.45:0/4123685684 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5617509e3160 msgr2=0x561750995af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.365+0000 7f3b6f7fe640 1 --2- 172.21.15.45:0/4123685684 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5617509e3160 0x561750995af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.365+0000 7f3b6f7fe640 1 -- 172.21.15.45:0/4123685684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617509c5b40 msgr2=0x56175099a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.365+0000 7f3b6f7fe640 1 --2- 172.21.15.45:0/4123685684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617509c5b40 0x56175099a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.365+0000 7f3b6f7fe640 1 -- 172.21.15.45:0/4123685684 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617508b0e60 con 0x5617509c6280 2024-05-08T05:45:08.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.366+0000 7f3b6f7fe640 1 --2- 172.21.15.45:0/4123685684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 0x5617509c5600 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400c350 tx=0x7f3b64009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8e7426a4a007e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.366+0000 7f3b55ffb640 1 -- 172.21.15.45:0/4123685684 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64037040 con 0x5617509c6280 2024-05-08T05:45:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.366+0000 7f3b55ffb640 1 -- 172.21.15.45:0/4123685684 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b64042070 con 0x5617509c6280 2024-05-08T05:45:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.366+0000 7f3b55ffb640 1 -- 172.21.15.45:0/4123685684 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64004a20 con 0x5617509c6280 2024-05-08T05:45:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.366+0000 7f3b79024740 1 -- 172.21.15.45:0/4123685684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 msgr2=0x5617509c5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.366+0000 7f3b79024740 1 --2- 172.21.15.45:0/4123685684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 0x5617509c5600 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400c350 tx=0x7f3b64009030 comp rx=0 tx=0).stop 2024-05-08T05:45:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.366+0000 7f3b79024740 1 -- 172.21.15.45:0/4123685684 shutdown_connections 2024-05-08T05:45:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.367+0000 7f3b79024740 1 --2- 172.21.15.45:0/4123685684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617509c5b40 0x56175099a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.367+0000 7f3b79024740 1 --2- 172.21.15.45:0/4123685684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 0x5617509c5600 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.367+0000 7f3b79024740 1 --2- 172.21.15.45:0/4123685684 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5617509e3160 0x561750995af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.367+0000 7f3b79024740 1 -- 172.21.15.45:0/4123685684 >> 172.21.15.45:0/4123685684 conn(0x561750997d60 msgr2=0x5617509e0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.367+0000 7f3b79024740 1 -- 172.21.15.45:0/4123685684 shutdown_connections 2024-05-08T05:45:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.367+0000 7f3b79024740 1 -- 172.21.15.45:0/4123685684 wait complete. 2024-05-08T05:45:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.367+0000 7f3b79024740 1 -- start start 2024-05-08T05:45:08.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.367+0000 7f3b79024740 1 Processor -- start 2024-05-08T05:45:08.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.368+0000 7f3b79024740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617509c6280 0x5617509c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.368+0000 7f3b79024740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509e3160 0x5617509dce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.368+0000 7f3b79024740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5617509c0b10 0x5617509c3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.368+0000 7f3b79024740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5617508bc5e0 con 0x5617509e3160 2024-05-08T05:45:08.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.368+0000 7f3b79024740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5617508fbef0 con 0x5617509c6280 2024-05-08T05:45:08.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.368+0000 7f3b79024740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x561750999f10 con 0x5617509c0b10 2024-05-08T05:45:08.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.368+0000 7f3b79024740 10 client.0 _ll_register_callbacks cb 0x5617508b7320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:45:08.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.368+0000 7f3b6f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509e3160 0x5617509dce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.368+0000 7f3b6f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509e3160 0x5617509dce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35354/0 (socket says 172.21.15.45:35354) 2024-05-08T05:45:08.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.368+0000 7f3b6f7fe640 1 -- 172.21.15.45:0/3628651381 learned_addr learned my addr 172.21.15.45:0/3628651381 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:08.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.368+0000 7f3b6ffff640 1 --2- 172.21.15.45:0/3628651381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617509c6280 0x5617509c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.368+0000 7f3b748a7640 1 --2- 172.21.15.45:0/3628651381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5617509c0b10 0x5617509c3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.369+0000 7f3b6f7fe640 1 -- 172.21.15.45:0/3628651381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5617509c0b10 msgr2=0x5617509c3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.369+0000 7f3b6f7fe640 1 --2- 172.21.15.45:0/3628651381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5617509c0b10 0x5617509c3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.369+0000 7f3b6f7fe640 1 -- 172.21.15.45:0/3628651381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617509c6280 msgr2=0x5617509c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.369+0000 7f3b6f7fe640 1 --2- 172.21.15.45:0/3628651381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617509c6280 0x5617509c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.369+0000 7f3b6f7fe640 1 -- 172.21.15.45:0/3628651381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617506db7d0 con 0x5617509e3160 2024-05-08T05:45:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.369+0000 7f3b6f7fe640 1 --2- 172.21.15.45:0/3628651381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509e3160 0x5617509dce10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3b64001ca0 tx=0x7f3b6402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba6825672d6be2d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.369+0000 7f3b4effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:45:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.369+0000 7f3b4effd640 1 -- 172.21.15.45:0/3628651381 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6404a070 con 0x5617509e3160 2024-05-08T05:45:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.369+0000 7f3b4effd640 1 -- 172.21.15.45:0/3628651381 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f3b64001770 con 0x5617509e3160 2024-05-08T05:45:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.369+0000 7f3b79024740 1 -- 172.21.15.45:0/3628651381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b64042070 con 0x5617509e3160 2024-05-08T05:45:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.369+0000 7f3b4effd640 1 -- 172.21.15.45:0/3628651381 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3b640047e0 con 0x5617509e3160 2024-05-08T05:45:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.369+0000 7f3b79024740 10 client.5991 fetch_fsmap learned FSMap version 36 2024-05-08T05:45:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.369+0000 7f3b79024740 1 -- 172.21.15.45:0/3628651381 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x5617506113f0 con 0x5617509e3160 2024-05-08T05:45:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.369+0000 7f3b4effd640 1 -- 172.21.15.45:0/3628651381 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b64042070 con 0x5617509e3160 2024-05-08T05:45:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.369+0000 7f3b4effd640 1 -- 172.21.15.45:0/3628651381 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f3b64058080 con 0x5617509e3160 2024-05-08T05:45:08.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.369+0000 7f3b79024740 10 client.5991 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:45:08.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.370+0000 7f3b79024740 10 client.5991 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T05:45:08.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.370+0000 7f3b79024740 20 client.5991 populate_metadata read hostname 'smithi045' 2024-05-08T05:45:08.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.370+0000 7f3b79024740 1 --2- 172.21.15.45:0/3628651381 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5617509c2480 0x5617509c28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.370+0000 7f3b79024740 4 client.5991 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:45:08.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.370+0000 7f3b79024740 1 -- 172.21.15.45:0/3628651381 --> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56175090db40 con 0x5617509c2480 2024-05-08T05:45:08.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.370+0000 7f3b6ffff640 1 --2- 172.21.15.45:0/3628651381 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5617509c2480 0x5617509c28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.370+0000 7f3b6ffff640 1 --2- 172.21.15.45:0/3628651381 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5617509c2480 0x5617509c28a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aae211429430c132 server_cookie=266841d9f3d641b6 in_seq=0 out_seq=0 2024-05-08T05:45:08.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.370+0000 7f3b4effd640 10 client.5991 ms_handle_connect on v2:172.21.15.45:6833/1461822016 2024-05-08T05:45:08.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.371+0000 7f3b4effd640 1 -- 172.21.15.45:0/3628651381 <== mds.0 v2:172.21.15.45:6833/1461822016 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56175090db40 con 0x5617509c2480 2024-05-08T05:45:08.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.371+0000 7f3b4effd640 10 client.5991 handle_command_reply: tid=0 2024-05-08T05:45:08.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.371+0000 7f3b79024740 1 client.5991 shutdown 2024-05-08T05:45:08.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.371+0000 7f3b79024740 1 -- 172.21.15.45:0/3628651381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509e3160 msgr2=0x5617509dce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.371+0000 7f3b79024740 1 --2- 172.21.15.45:0/3628651381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509e3160 0x5617509dce10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3b64001ca0 tx=0x7f3b6402c000 comp rx=0 tx=0).stop 2024-05-08T05:45:08.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.371+0000 7f3b79024740 1 -- 172.21.15.45:0/3628651381 shutdown_connections 2024-05-08T05:45:08.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.371+0000 7f3b79024740 1 --2- 172.21.15.45:0/3628651381 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5617509c2480 0x5617509c28a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.371+0000 7f3b79024740 1 --2- 172.21.15.45:0/3628651381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5617509c0b10 0x5617509c3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.371+0000 7f3b79024740 1 --2- 172.21.15.45:0/3628651381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509e3160 0x5617509dce10 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.371+0000 7f3b79024740 1 --2- 172.21.15.45:0/3628651381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617509c6280 0x5617509c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.371+0000 7f3b79024740 1 -- 172.21.15.45:0/3628651381 >> 172.21.15.45:0/3628651381 conn(0x5617509c5b40 msgr2=0x561750995230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:08.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.371+0000 7f3b4effd640 0 client.5991 ms_handle_reset on v2:172.21.15.45:6833/1461822016 2024-05-08T05:45:08.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.371+0000 7f3b79024740 1 -- 172.21.15.45:0/3628651381 shutdown_connections 2024-05-08T05:45:08.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.371+0000 7f3b79024740 1 -- 172.21.15.45:0/3628651381 wait complete. 2024-05-08T05:45:08.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.371+0000 7f3b79024740 20 client.5991 trim_cache size 0 max 16384 2024-05-08T05:45:08.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.389+0000 7f3b79024740 1 Processor -- start 2024-05-08T05:45:08.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.389+0000 7f3b79024740 1 -- start start 2024-05-08T05:45:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.390+0000 7f3b79024740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5617509c0b10 0x5617509e17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.390+0000 7f3b79024740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 0x5617509e1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.390+0000 7f3b79024740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617509e3160 0x561750a04170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.390+0000 7f3b79024740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3b70202700 con 0x5617509c6280 2024-05-08T05:45:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.390+0000 7f3b79024740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b702016e0 con 0x5617509e3160 2024-05-08T05:45:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.390+0000 7f3b79024740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3b702009c0 con 0x5617509c0b10 2024-05-08T05:45:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.390+0000 7f3b6f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 0x5617509e1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.390+0000 7f3b6f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 0x5617509e1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35368/0 (socket says 172.21.15.45:35368) 2024-05-08T05:45:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.390+0000 7f3b6f7fe640 1 -- 172.21.15.45:0/3451309685 learned_addr learned my addr 172.21.15.45:0/3451309685 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.390+0000 7f3b6ffff640 1 --2- 172.21.15.45:0/3451309685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5617509c0b10 0x5617509e17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.390+0000 7f3b748a7640 1 --2- 172.21.15.45:0/3451309685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617509e3160 0x561750a04170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.390+0000 7f3b6f7fe640 1 -- 172.21.15.45:0/3451309685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5617509c0b10 msgr2=0x5617509e17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.390+0000 7f3b6f7fe640 1 --2- 172.21.15.45:0/3451309685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5617509c0b10 0x5617509e17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.390+0000 7f3b6f7fe640 1 -- 172.21.15.45:0/3451309685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617509e3160 msgr2=0x561750a04170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.390+0000 7f3b6f7fe640 1 --2- 172.21.15.45:0/3451309685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617509e3160 0x561750a04170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.390+0000 7f3b6f7fe640 1 -- 172.21.15.45:0/3451309685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617505d10f0 con 0x5617509c6280 2024-05-08T05:45:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.391+0000 7f3b6f7fe640 1 --2- 172.21.15.45:0/3451309685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 0x5617509e1ce0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3b64000e80 tx=0x7f3b64006070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24c60bcbf97c3c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.391+0000 7f3b557fa640 1 -- 172.21.15.45:0/3451309685 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6404a040 con 0x5617509c6280 2024-05-08T05:45:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.391+0000 7f3b557fa640 1 -- 172.21.15.45:0/3451309685 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b6405b070 con 0x5617509c6280 2024-05-08T05:45:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.391+0000 7f3b557fa640 1 -- 172.21.15.45:0/3451309685 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b640017b0 con 0x5617509c6280 2024-05-08T05:45:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.391+0000 7f3b79024740 1 -- 172.21.15.45:0/3451309685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 msgr2=0x5617509e1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.391+0000 7f3b79024740 1 --2- 172.21.15.45:0/3451309685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 0x5617509e1ce0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3b64000e80 tx=0x7f3b64006070 comp rx=0 tx=0).stop 2024-05-08T05:45:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.391+0000 7f3b79024740 1 -- 172.21.15.45:0/3451309685 shutdown_connections 2024-05-08T05:45:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.391+0000 7f3b79024740 1 --2- 172.21.15.45:0/3451309685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617509e3160 0x561750a04170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.391+0000 7f3b79024740 1 --2- 172.21.15.45:0/3451309685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 0x5617509e1ce0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.391+0000 7f3b79024740 1 --2- 172.21.15.45:0/3451309685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5617509c0b10 0x5617509e17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.391+0000 7f3b79024740 1 -- 172.21.15.45:0/3451309685 >> 172.21.15.45:0/3451309685 conn(0x5617509c5b40 msgr2=0x5617509c7190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:08.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.391+0000 7f3b79024740 1 -- 172.21.15.45:0/3451309685 shutdown_connections 2024-05-08T05:45:08.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.391+0000 7f3b79024740 1 -- 172.21.15.45:0/3451309685 wait complete. 2024-05-08T05:45:08.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.392+0000 7f3b79024740 1 -- start start 2024-05-08T05:45:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.392+0000 7f3b79024740 1 Processor -- start 2024-05-08T05:45:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.393+0000 7f3b79024740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617509c0b10 0x5617509e17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.393+0000 7f3b79024740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 0x5617509e3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.393+0000 7f3b79024740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5617509e3160 0x5617509e3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.393+0000 7f3b79024740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5617509c0920 con 0x5617509c6280 2024-05-08T05:45:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.393+0000 7f3b79024740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b70201010 con 0x5617509c0b10 2024-05-08T05:45:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.393+0000 7f3b79024740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5617509c34f0 con 0x5617509e3160 2024-05-08T05:45:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.393+0000 7f3b79024740 10 client.0 _ll_register_callbacks cb 0x7f3b6805d500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:45:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.393+0000 7f3b6f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 0x5617509e3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.393+0000 7f3b6f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 0x5617509e3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35374/0 (socket says 172.21.15.45:35374) 2024-05-08T05:45:08.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.393+0000 7f3b6f7fe640 1 -- 172.21.15.45:0/2877210204 learned_addr learned my addr 172.21.15.45:0/2877210204 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.393+0000 7f3b6f7fe640 1 -- 172.21.15.45:0/2877210204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5617509e3160 msgr2=0x5617509e3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.393+0000 7f3b748a7640 1 --2- 172.21.15.45:0/2877210204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5617509e3160 0x5617509e3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.393+0000 7f3b6ffff640 1 --2- 172.21.15.45:0/2877210204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617509c0b10 0x5617509e17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.393+0000 7f3b6f7fe640 1 --2- 172.21.15.45:0/2877210204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5617509e3160 0x5617509e3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.393+0000 7f3b6f7fe640 1 -- 172.21.15.45:0/2877210204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617509c0b10 msgr2=0x5617509e17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.393+0000 7f3b6f7fe640 1 --2- 172.21.15.45:0/2877210204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617509c0b10 0x5617509e17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.393+0000 7f3b6f7fe640 1 -- 172.21.15.45:0/2877210204 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617509bcaa0 con 0x5617509c6280 2024-05-08T05:45:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.394+0000 7f3b6f7fe640 1 --2- 172.21.15.45:0/2877210204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 0x5617509e3960 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3b64001d70 tx=0x7f3b6402d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f247b014b7f0d2f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.394+0000 7f3b4effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:45:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.394+0000 7f3b4effd640 1 -- 172.21.15.45:0/2877210204 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6404a030 con 0x5617509c6280 2024-05-08T05:45:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.394+0000 7f3b4effd640 1 -- 172.21.15.45:0/2877210204 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f3b64007160 con 0x5617509c6280 2024-05-08T05:45:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.394+0000 7f3b79024740 1 -- 172.21.15.45:0/2877210204 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b6405b070 con 0x5617509c6280 2024-05-08T05:45:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.394+0000 7f3b4effd640 1 -- 172.21.15.45:0/2877210204 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3b64069050 con 0x5617509c6280 2024-05-08T05:45:08.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.394+0000 7f3b79024740 10 client.5997 fetch_fsmap learned FSMap version 36 2024-05-08T05:45:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.394+0000 7f3b79024740 1 -- 172.21.15.45:0/2877210204 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x5617509bcaa0 con 0x5617509c6280 2024-05-08T05:45:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.394+0000 7f3b4effd640 1 -- 172.21.15.45:0/2877210204 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b6405b070 con 0x5617509c6280 2024-05-08T05:45:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.394+0000 7f3b4effd640 1 -- 172.21.15.45:0/2877210204 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f3b6406c020 con 0x5617509c6280 2024-05-08T05:45:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.394+0000 7f3b79024740 10 client.5997 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:45:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.394+0000 7f3b79024740 10 client.5997 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-05-08T05:45:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.394+0000 7f3b79024740 20 client.5997 populate_metadata read hostname 'smithi045' 2024-05-08T05:45:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.394+0000 7f3b79024740 1 --2- 172.21.15.45:0/2877210204 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5617509c67e0 0x561750a61bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.394+0000 7f3b79024740 4 client.5997 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:45:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.394+0000 7f3b79024740 1 -- 172.21.15.45:0/2877210204 --> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5617509079b0 con 0x5617509c67e0 2024-05-08T05:45:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.394+0000 7f3b6ffff640 1 --2- 172.21.15.45:0/2877210204 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5617509c67e0 0x561750a61bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.395+0000 7f3b4effd640 10 client.5997 ms_handle_connect on v2:172.21.15.45:6833/1461822016 2024-05-08T05:45:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.395+0000 7f3b6ffff640 1 --2- 172.21.15.45:0/2877210204 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5617509c67e0 0x561750a61bf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c045ae213c77e6 server_cookie=2b3144f1b28631e9 in_seq=0 out_seq=0 2024-05-08T05:45:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.395+0000 7f3b4effd640 1 -- 172.21.15.45:0/2877210204 <== mds.0 v2:172.21.15.45:6833/1461822016 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5617509079b0 con 0x5617509c67e0 2024-05-08T05:45:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.395+0000 7f3b4effd640 10 client.5997 handle_command_reply: tid=0 2024-05-08T05:45:08.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.395+0000 7f3b79024740 1 client.5997 shutdown 2024-05-08T05:45:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.395+0000 7f3b79024740 1 -- 172.21.15.45:0/2877210204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 msgr2=0x5617509e3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.395+0000 7f3b79024740 1 --2- 172.21.15.45:0/2877210204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 0x5617509e3960 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3b64001d70 tx=0x7f3b6402d000 comp rx=0 tx=0).stop 2024-05-08T05:45:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.395+0000 7f3b79024740 1 -- 172.21.15.45:0/2877210204 shutdown_connections 2024-05-08T05:45:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.395+0000 7f3b79024740 1 --2- 172.21.15.45:0/2877210204 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x5617509c67e0 0x561750a61bf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.395+0000 7f3b79024740 1 --2- 172.21.15.45:0/2877210204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5617509e3160 0x5617509e3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.395+0000 7f3b4effd640 0 client.5997 ms_handle_reset on v2:172.21.15.45:6833/1461822016 2024-05-08T05:45:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.395+0000 7f3b79024740 1 --2- 172.21.15.45:0/2877210204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5617509c6280 0x5617509e3960 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.395+0000 7f3b79024740 1 --2- 172.21.15.45:0/2877210204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5617509c0b10 0x5617509e17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.395+0000 7f3b79024740 1 -- 172.21.15.45:0/2877210204 >> 172.21.15.45:0/2877210204 conn(0x5617509c5b40 msgr2=0x561750a04390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.395+0000 7f3b79024740 1 -- 172.21.15.45:0/2877210204 shutdown_connections 2024-05-08T05:45:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.395+0000 7f3b79024740 1 -- 172.21.15.45:0/2877210204 wait complete. 2024-05-08T05:45:08.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.395+0000 7f3b79024740 20 client.5997 trim_cache size 0 max 16384 2024-05-08T05:45:08.402 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:45:08.402 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-05-08T05:45:08.402 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-05-08T05:45:08.402 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:45:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.396+0000 7f3b7630a640 1 -- 172.21.15.45:0/1936279565 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3b5c0762d0 msgr2=0x7f3b5c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.396+0000 7f3b7630a640 1 --2- 172.21.15.45:0/1936279565 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3b5c0762d0 0x7f3b5c078790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3b60009a60 tx=0x7f3b60008040 comp rx=0 tx=0).stop 2024-05-08T05:45:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.396+0000 7f3b7630a640 1 -- 172.21.15.45:0/1936279565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b700b37a0 msgr2=0x7f3b700b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.396+0000 7f3b7630a640 1 --2- 172.21.15.45:0/1936279565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b700b37a0 0x7f3b700b3c30 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3b700cf470 tx=0x7f3b680115a0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.396+0000 7f3b7630a640 1 -- 172.21.15.45:0/1936279565 shutdown_connections 2024-05-08T05:45:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.396+0000 7f3b7630a640 1 --2- 172.21.15.45:0/1936279565 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3b5c0762d0 0x7f3b5c078790 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.396+0000 7f3b7630a640 1 --2- 172.21.15.45:0/1936279565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b700b37a0 0x7f3b700b3c30 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.396+0000 7f3b7630a640 1 --2- 172.21.15.45:0/1936279565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b700c6960 0x7f3b7015a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.396+0000 7f3b7630a640 1 --2- 172.21.15.45:0/1936279565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b58004800 0x7f3b700b8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.396+0000 7f3b7630a640 1 -- 172.21.15.45:0/1936279565 >> 172.21.15.45:0/1936279565 conn(0x7f3b7009ba40 msgr2=0x7f3b7009be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:08.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.396+0000 7f3b7630a640 1 -- 172.21.15.45:0/1936279565 shutdown_connections 2024-05-08T05:45:08.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.396+0000 7f3b7630a640 1 -- 172.21.15.45:0/1936279565 wait complete. 2024-05-08T05:45:08.407 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T05:45:08.407 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:45:08.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.510+0000 7fc0cd3f0640 1 Processor -- start 2024-05-08T05:45:08.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.510+0000 7fc0cd3f0640 1 -- start start 2024-05-08T05:45:08.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.511+0000 7fc0cd3f0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0c80d0ec0 0x7fc0c80d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.511+0000 7fc0cd3f0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc0c80d3b00 0x7fc0c80b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.511+0000 7fc0cd3f0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0c80d1760 con 0x7fc0c815f380 2024-05-08T05:45:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.511+0000 7fc0cd3f0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc0c80d30f0 con 0x7fc0c80d3b00 2024-05-08T05:45:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.511+0000 7fc0cd3f0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c80d2370 con 0x7fc0c80d0ec0 2024-05-08T05:45:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.511+0000 7fc0c6ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc0c815f380 0x7fc0c80d07b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42844/0 (socket says 172.21.15.45:42844) 2024-05-08T05:45:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.511+0000 7fc0c6ffd640 1 -- 172.21.15.45:0/3722096142 learned_addr learned my addr 172.21.15.45:0/3722096142 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.511+0000 7fc0c77fe640 1 --2- 172.21.15.45:0/3722096142 >> v2:172.21.15.45:3300/0 conn(0x7fc0c80d3b00 0x7fc0c80b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.511+0000 7fc0bffff640 1 --2- 172.21.15.45:0/3722096142 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0c80d0ec0 0x7fc0c80d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.511+0000 7fc0c77fe640 1 -- 172.21.15.45:0/3722096142 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0c80d0ec0 msgr2=0x7fc0c80d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.511+0000 7fc0c77fe640 1 --2- 172.21.15.45:0/3722096142 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0c80d0ec0 0x7fc0c80d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.511+0000 7fc0c77fe640 1 -- 172.21.15.45:0/3722096142 >> v1:172.21.15.45:6789/0 conn(0x7fc0c815f380 legacy=0x7fc0c80d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.511+0000 7fc0c77fe640 1 -- 172.21.15.45:0/3722096142 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c80d1a40 con 0x7fc0c80d3b00 2024-05-08T05:45:08.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.512+0000 7fc0c77fe640 1 --2- 172.21.15.45:0/3722096142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0c80d3b00 0x7fc0c80b7e30 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009d90 tx=0x7fc0c002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5af4eb459f6374b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:08.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.512+0000 7fc0c67fc640 1 -- 172.21.15.45:0/3722096142 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0007740 con 0x7fc0c80d3b00 2024-05-08T05:45:08.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.512+0000 7fc0c67fc640 1 -- 172.21.15.45:0/3722096142 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0c00078e0 con 0x7fc0c80d3b00 2024-05-08T05:45:08.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.512+0000 7fc0c67fc640 1 -- 172.21.15.45:0/3722096142 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0007cb0 con 0x7fc0c80d3b00 2024-05-08T05:45:08.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.513+0000 7fc0cd3f0640 1 -- 172.21.15.45:0/3722096142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0c80d3b00 msgr2=0x7fc0c80b7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.513+0000 7fc0cd3f0640 1 --2- 172.21.15.45:0/3722096142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0c80d3b00 0x7fc0c80b7e30 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009d90 tx=0x7fc0c002a660 comp rx=0 tx=0).stop 2024-05-08T05:45:08.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.513+0000 7fc0cd3f0640 1 -- 172.21.15.45:0/3722096142 shutdown_connections 2024-05-08T05:45:08.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.513+0000 7fc0cd3f0640 1 --2- 172.21.15.45:0/3722096142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0c80d3b00 0x7fc0c80b7e30 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.513+0000 7fc0cd3f0640 1 --2- 172.21.15.45:0/3722096142 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0c80d0ec0 0x7fc0c80d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.513+0000 7fc0cd3f0640 1 -- 172.21.15.45:0/3722096142 >> 172.21.15.45:0/3722096142 conn(0x7fc0c80d4fe0 msgr2=0x7fc0c80d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:08.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.513+0000 7fc0cd3f0640 1 -- 172.21.15.45:0/3722096142 shutdown_connections 2024-05-08T05:45:08.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.513+0000 7fc0cd3f0640 1 -- 172.21.15.45:0/3722096142 wait complete. 2024-05-08T05:45:08.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.514+0000 7fc0cd3f0640 1 Processor -- start 2024-05-08T05:45:08.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.514+0000 7fc0cd3f0640 1 -- start start 2024-05-08T05:45:08.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.515+0000 7fc0cd3f0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0c80d0ec0 0x7fc0c815a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.515+0000 7fc0cd3f0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0c80d3b00 0x7fc0c815ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.515+0000 7fc0cd3f0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0c815f380 0x7fc0c80cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.515+0000 7fc0cd3f0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c815f760 con 0x7fc0c80d3b00 2024-05-08T05:45:08.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.515+0000 7fc0cd3f0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c80d2000 con 0x7fc0c80d0ec0 2024-05-08T05:45:08.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.515+0000 7fc0cd3f0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c80d2800 con 0x7fc0c815f380 2024-05-08T05:45:08.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.515+0000 7fc0bffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0c80d3b00 0x7fc0c815ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.515+0000 7fc0bffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0c80d3b00 0x7fc0c815ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35388/0 (socket says 172.21.15.45:35388) 2024-05-08T05:45:08.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.515+0000 7fc0bffff640 1 -- 172.21.15.45:0/2802770254 learned_addr learned my addr 172.21.15.45:0/2802770254 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:08.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.515+0000 7fc0c6ffd640 1 --2- 172.21.15.45:0/2802770254 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0c80d0ec0 0x7fc0c815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.515+0000 7fc0c77fe640 1 --2- 172.21.15.45:0/2802770254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0c815f380 0x7fc0c80cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.515+0000 7fc0bffff640 1 -- 172.21.15.45:0/2802770254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0c815f380 msgr2=0x7fc0c80cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.515+0000 7fc0bffff640 1 --2- 172.21.15.45:0/2802770254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0c815f380 0x7fc0c80cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.515+0000 7fc0bffff640 1 -- 172.21.15.45:0/2802770254 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0c80d0ec0 msgr2=0x7fc0c815a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.515+0000 7fc0bffff640 1 --2- 172.21.15.45:0/2802770254 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0c80d0ec0 0x7fc0c815a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.515+0000 7fc0bffff640 1 -- 172.21.15.45:0/2802770254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c81f5130 con 0x7fc0c80d3b00 2024-05-08T05:45:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.515+0000 7fc0bffff640 1 --2- 172.21.15.45:0/2802770254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0c80d3b00 0x7fc0c815ba70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000d800 tx=0x7fc0b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.515+0000 7fc0bf7fe640 1 -- 172.21.15.45:0/2802770254 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b000b6c0 con 0x7fc0c80d3b00 2024-05-08T05:45:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.516+0000 7fc0bf7fe640 1 -- 172.21.15.45:0/2802770254 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b0019070 con 0x7fc0c80d3b00 2024-05-08T05:45:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.516+0000 7fc0cd3f0640 1 -- 172.21.15.45:0/2802770254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c81f5410 con 0x7fc0c80d3b00 2024-05-08T05:45:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.516+0000 7fc0bf7fe640 1 -- 172.21.15.45:0/2802770254 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b000ba80 con 0x7fc0c80d3b00 2024-05-08T05:45:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.516+0000 7fc0cd3f0640 1 -- 172.21.15.45:0/2802770254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c81f5cb0 con 0x7fc0c80d3b00 2024-05-08T05:45:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.517+0000 7fc0cd3f0640 1 -- 172.21.15.45:0/2802770254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c81f5800 con 0x7fc0c80d3b00 2024-05-08T05:45:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.517+0000 7fc0bf7fe640 1 -- 172.21.15.45:0/2802770254 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc0b000bc20 con 0x7fc0c80d3b00 2024-05-08T05:45:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.518+0000 7fc0bf7fe640 1 --2- 172.21.15.45:0/2802770254 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc0a80762d0 0x7fc0a8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.518+0000 7fc0bf7fe640 1 -- 172.21.15.45:0/2802770254 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fc0b0097b90 con 0x7fc0c80d3b00 2024-05-08T05:45:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.518+0000 7fc0c6ffd640 1 --2- 172.21.15.45:0/2802770254 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc0a80762d0 0x7fc0a8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.518+0000 7fc0c6ffd640 1 --2- 172.21.15.45:0/2802770254 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc0a80762d0 0x7fc0a8078790 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8000f60 tx=0x7fc0b8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:08.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.522+0000 7fc0bf7fe640 1 -- 172.21.15.45:0/2802770254 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc0b0061360 con 0x7fc0c80d3b00 2024-05-08T05:45:08.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.751+0000 7fc0cd3f0640 1 -- 172.21.15.45:0/2802770254 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0c80693f0 con 0x7fc0c80d3b00 2024-05-08T05:45:08.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.752+0000 7fc0bf7fe640 1 -- 172.21.15.45:0/2802770254 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5180 (secure 0 0 0) 0x7fc0b0060c60 con 0x7fc0c80d3b00 2024-05-08T05:45:08.752 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:08.752 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/162765657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":162765657},{"type":"v1","addr":"172.21.15.63:6836","nonce":162765657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5356,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1788513331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1788513331},{"type":"v1","addr":"172.21.15.45:6834","nonce":1788513331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1173167340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1173167340},{"type":"v1","addr":"172.21.15.63:6837","nonce":1173167340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:43:58.107819+0000","modified":"2024-05-08T05:44:38.767928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":13,"addr":"172.21.15.45:6835/1461822016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1461822016},{"type":"v1","addr":"172.21.15.45:6835","nonce":1461822016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5697,"qdb_cluster":[5697]},"id":4}]} 2024-05-08T05:45:08.752 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 36 2024-05-08T05:45:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.754+0000 7fc0cd3f0640 1 -- 172.21.15.45:0/2802770254 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc0a80762d0 msgr2=0x7fc0a8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.754+0000 7fc0cd3f0640 1 --2- 172.21.15.45:0/2802770254 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc0a80762d0 0x7fc0a8078790 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8000f60 tx=0x7fc0b8009040 comp rx=0 tx=0).stop 2024-05-08T05:45:08.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.754+0000 7fc0cd3f0640 1 -- 172.21.15.45:0/2802770254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0c80d3b00 msgr2=0x7fc0c815ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.754+0000 7fc0cd3f0640 1 --2- 172.21.15.45:0/2802770254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0c80d3b00 0x7fc0c815ba70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000d800 tx=0x7fc0b000dcc0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.754+0000 7fc0cd3f0640 1 -- 172.21.15.45:0/2802770254 shutdown_connections 2024-05-08T05:45:08.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.754+0000 7fc0cd3f0640 1 --2- 172.21.15.45:0/2802770254 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc0a80762d0 0x7fc0a8078790 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.754+0000 7fc0cd3f0640 1 --2- 172.21.15.45:0/2802770254 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0c815f380 0x7fc0c80cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.754+0000 7fc0cd3f0640 1 --2- 172.21.15.45:0/2802770254 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0c80d3b00 0x7fc0c815ba70 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.754+0000 7fc0cd3f0640 1 --2- 172.21.15.45:0/2802770254 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0c80d0ec0 0x7fc0c815a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.754+0000 7fc0cd3f0640 1 -- 172.21.15.45:0/2802770254 >> 172.21.15.45:0/2802770254 conn(0x7fc0c80d4fe0 msgr2=0x7fc0c80b8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:08.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.754+0000 7fc0cd3f0640 1 -- 172.21.15.45:0/2802770254 shutdown_connections 2024-05-08T05:45:08.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.754+0000 7fc0cd3f0640 1 -- 172.21.15.45:0/2802770254 wait complete. 2024-05-08T05:45:08.765 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-08T05:45:08.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.871+0000 7f7df5872640 1 Processor -- start 2024-05-08T05:45:08.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.871+0000 7f7df5872640 1 -- start start 2024-05-08T05:45:08.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.872+0000 7f7df5872640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7df006e0f0 0x7f7df015c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.872+0000 7f7df5872640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7df006e4d0 0x7f7df00aec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.872+0000 7f7df5872640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7df015c920 con 0x7f7df0072840 2024-05-08T05:45:08.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.872+0000 7f7df5872640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f7df015cc00 con 0x7f7df006e0f0 2024-05-08T05:45:08.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.872+0000 7f7df5872640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7df015cee0 con 0x7f7df006e4d0 2024-05-08T05:45:08.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.872+0000 7f7deeffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7df006e0f0 0x7f7df015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.872+0000 7f7deeffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7df006e0f0 0x7f7df015c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35398/0 (socket says 172.21.15.45:35398) 2024-05-08T05:45:08.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.872+0000 7f7deeffd640 1 -- 172.21.15.45:0/2343811643 learned_addr learned my addr 172.21.15.45:0/2343811643 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:08.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.872+0000 7f7def7fe640 1 --2- 172.21.15.45:0/2343811643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7df006e4d0 0x7f7df00aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.872+0000 7f7deeffd640 1 -- 172.21.15.45:0/2343811643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7df006e4d0 msgr2=0x7f7df00aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.872+0000 7f7deeffd640 1 --2- 172.21.15.45:0/2343811643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7df006e4d0 0x7f7df00aec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.872+0000 7f7deeffd640 1 -- 172.21.15.45:0/2343811643 >> v1:172.21.15.63:6789/0 conn(0x7f7df0072840 legacy=0x7f7df00d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.872+0000 7f7deeffd640 1 -- 172.21.15.45:0/2343811643 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df015d1c0 con 0x7f7df006e0f0 2024-05-08T05:45:08.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.873+0000 7f7deeffd640 1 --2- 172.21.15.45:0/2343811643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df006e0f0 0x7f7df015c340 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009870 tx=0x7f7de002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b325b3c01406dab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:08.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.873+0000 7f7dedffb640 1 -- 172.21.15.45:0/2343811643 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0037070 con 0x7f7df006e0f0 2024-05-08T05:45:08.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.873+0000 7f7dedffb640 1 -- 172.21.15.45:0/2343811643 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de0032440 con 0x7f7df006e0f0 2024-05-08T05:45:08.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.873+0000 7f7dedffb640 1 -- 172.21.15.45:0/2343811643 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de0032810 con 0x7f7df006e0f0 2024-05-08T05:45:08.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.874+0000 7f7df5872640 1 -- 172.21.15.45:0/2343811643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df006e0f0 msgr2=0x7f7df015c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.874+0000 7f7df5872640 1 --2- 172.21.15.45:0/2343811643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df006e0f0 0x7f7df015c340 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009870 tx=0x7f7de002a270 comp rx=0 tx=0).stop 2024-05-08T05:45:08.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.874+0000 7f7df5872640 1 -- 172.21.15.45:0/2343811643 shutdown_connections 2024-05-08T05:45:08.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.874+0000 7f7df5872640 1 --2- 172.21.15.45:0/2343811643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7df006e4d0 0x7f7df00aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.874+0000 7f7df5872640 1 --2- 172.21.15.45:0/2343811643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df006e0f0 0x7f7df015c340 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.874+0000 7f7df5872640 1 -- 172.21.15.45:0/2343811643 >> 172.21.15.45:0/2343811643 conn(0x7f7df006c3a0 msgr2=0x7f7df006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:08.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.874+0000 7f7df5872640 1 -- 172.21.15.45:0/2343811643 shutdown_connections 2024-05-08T05:45:08.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.874+0000 7f7df5872640 1 -- 172.21.15.45:0/2343811643 wait complete. 2024-05-08T05:45:08.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.875+0000 7f7df5872640 1 Processor -- start 2024-05-08T05:45:08.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.875+0000 7f7df5872640 1 -- start start 2024-05-08T05:45:08.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.876+0000 7f7df5872640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7df006e0f0 0x7f7df014d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.876+0000 7f7df5872640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df006e4d0 0x7f7df00b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.876+0000 7f7df5872640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7df0072840 0x7f7df0153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.876+0000 7f7df5872640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7df00ae7d0 con 0x7f7df006e4d0 2024-05-08T05:45:08.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.876+0000 7f7df5872640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7df00ae650 con 0x7f7df0072840 2024-05-08T05:45:08.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.876+0000 7f7df5872640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7df01530b0 con 0x7f7df006e0f0 2024-05-08T05:45:08.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.876+0000 7f7dee7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df006e4d0 0x7f7df00b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.876+0000 7f7dee7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df006e4d0 0x7f7df00b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35410/0 (socket says 172.21.15.45:35410) 2024-05-08T05:45:08.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.876+0000 7f7dee7fc640 1 -- 172.21.15.45:0/252183242 learned_addr learned my addr 172.21.15.45:0/252183242 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:08.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.876+0000 7f7deeffd640 1 --2- 172.21.15.45:0/252183242 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7df006e0f0 0x7f7df014d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.876+0000 7f7def7fe640 1 --2- 172.21.15.45:0/252183242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7df0072840 0x7f7df0153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.877+0000 7f7dee7fc640 1 -- 172.21.15.45:0/252183242 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7df006e0f0 msgr2=0x7f7df014d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.877+0000 7f7dee7fc640 1 --2- 172.21.15.45:0/252183242 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7df006e0f0 0x7f7df014d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.877+0000 7f7dee7fc640 1 -- 172.21.15.45:0/252183242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7df0072840 msgr2=0x7f7df0153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.877+0000 7f7dee7fc640 1 --2- 172.21.15.45:0/252183242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7df0072840 0x7f7df0153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.877+0000 7f7dee7fc640 1 -- 172.21.15.45:0/252183242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df00b2be0 con 0x7f7df006e4d0 2024-05-08T05:45:08.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.877+0000 7f7dee7fc640 1 --2- 172.21.15.45:0/252183242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df006e4d0 0x7f7df00b26a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7de400c370 tx=0x7f7de400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:08.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.877+0000 7f7dd7fff640 1 -- 172.21.15.45:0/252183242 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4017070 con 0x7f7df006e4d0 2024-05-08T05:45:08.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.877+0000 7f7dd7fff640 1 -- 172.21.15.45:0/252183242 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de40052d0 con 0x7f7df006e4d0 2024-05-08T05:45:08.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.877+0000 7f7dd7fff640 1 -- 172.21.15.45:0/252183242 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4003830 con 0x7f7df006e4d0 2024-05-08T05:45:08.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.877+0000 7f7df5872640 1 -- 172.21.15.45:0/252183242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df0156ba0 con 0x7f7df006e4d0 2024-05-08T05:45:08.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.877+0000 7f7df5872640 1 -- 172.21.15.45:0/252183242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df0157160 con 0x7f7df006e4d0 2024-05-08T05:45:08.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.879+0000 7f7dd7fff640 1 -- 172.21.15.45:0/252183242 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7de401a070 con 0x7f7df006e4d0 2024-05-08T05:45:08.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.880+0000 7f7dd7fff640 1 --2- 172.21.15.45:0/252183242 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7ddc0762d0 0x7f7ddc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.880+0000 7f7dd7fff640 1 -- 172.21.15.45:0/252183242 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f7de4096bb0 con 0x7f7df006e4d0 2024-05-08T05:45:08.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.880+0000 7f7deeffd640 1 --2- 172.21.15.45:0/252183242 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7ddc0762d0 0x7f7ddc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.880+0000 7f7df8592740 1 Processor -- start 2024-05-08T05:45:08.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.880+0000 7f7df8592740 1 -- start start 2024-05-08T05:45:08.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.881+0000 7f7deeffd640 1 --2- 172.21.15.45:0/252183242 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7ddc0762d0 0x7f7ddc078790 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7de0034000 tx=0x7f7de002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:08.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.881+0000 7f7df8592740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cbe3d9c710 0x55cbe3dba740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.881+0000 7f7df8592740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9caf0 0x55cbe3d77ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.881+0000 7f7df8592740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cbe3d722f0 0x55cbe3d78010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.881+0000 7f7df8592740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cbe3c62920 con 0x55cbe3d9caf0 2024-05-08T05:45:08.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.881+0000 7f7df8592740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cbe3cd7dd0 con 0x55cbe3d9c710 2024-05-08T05:45:08.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.881+0000 7f7df8592740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55cbe3c883e0 con 0x55cbe3d722f0 2024-05-08T05:45:08.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.881+0000 7f7dee7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9caf0 0x55cbe3d77ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.881+0000 7f7dee7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9caf0 0x55cbe3d77ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35418/0 (socket says 172.21.15.45:35418) 2024-05-08T05:45:08.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.881+0000 7f7dee7fc640 1 -- 172.21.15.45:0/1966001255 learned_addr learned my addr 172.21.15.45:0/1966001255 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:08.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.881+0000 7f7dee7fc640 1 -- 172.21.15.45:0/1966001255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cbe3d722f0 msgr2=0x55cbe3d78010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.881+0000 7f7dee7fc640 1 --2- 172.21.15.45:0/1966001255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cbe3d722f0 0x55cbe3d78010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.882+0000 7f7dee7fc640 1 -- 172.21.15.45:0/1966001255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cbe3d9c710 msgr2=0x55cbe3dba740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.882+0000 7f7dee7fc640 1 --2- 172.21.15.45:0/1966001255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cbe3d9c710 0x55cbe3dba740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.882+0000 7f7dee7fc640 1 -- 172.21.15.45:0/1966001255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbe3ab97c0 con 0x55cbe3d9caf0 2024-05-08T05:45:08.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.882+0000 7f7dee7fc640 1 --2- 172.21.15.45:0/1966001255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9caf0 0x55cbe3d77ad0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7de4094e60 tx=0x7f7de40683c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6be4104ce77ecc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:08.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.882+0000 7f7dd57fa640 1 -- 172.21.15.45:0/1966001255 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4079070 con 0x55cbe3d9caf0 2024-05-08T05:45:08.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.882+0000 7f7dd57fa640 1 -- 172.21.15.45:0/1966001255 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de405fbf0 con 0x55cbe3d9caf0 2024-05-08T05:45:08.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.882+0000 7f7dd57fa640 1 -- 172.21.15.45:0/1966001255 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de405f050 con 0x55cbe3d9caf0 2024-05-08T05:45:08.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.882+0000 7f7df8592740 1 -- 172.21.15.45:0/1966001255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9caf0 msgr2=0x55cbe3d77ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.882+0000 7f7df8592740 1 --2- 172.21.15.45:0/1966001255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9caf0 0x55cbe3d77ad0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7de4094e60 tx=0x7f7de40683c0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.883+0000 7f7df8592740 1 -- 172.21.15.45:0/1966001255 shutdown_connections 2024-05-08T05:45:08.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.883+0000 7f7df8592740 1 --2- 172.21.15.45:0/1966001255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cbe3d722f0 0x55cbe3d78010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.883+0000 7f7df8592740 1 --2- 172.21.15.45:0/1966001255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9caf0 0x55cbe3d77ad0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.883+0000 7f7df8592740 1 --2- 172.21.15.45:0/1966001255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cbe3d9c710 0x55cbe3dba740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.883+0000 7f7df8592740 1 -- 172.21.15.45:0/1966001255 >> 172.21.15.45:0/1966001255 conn(0x55cbe3d74560 msgr2=0x55cbe3dbcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:08.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.883+0000 7f7df8592740 1 -- 172.21.15.45:0/1966001255 shutdown_connections 2024-05-08T05:45:08.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.883+0000 7f7df8592740 1 -- 172.21.15.45:0/1966001255 wait complete. 2024-05-08T05:45:08.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.883+0000 7f7df8592740 1 -- start start 2024-05-08T05:45:08.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.883+0000 7f7df8592740 1 Processor -- start 2024-05-08T05:45:08.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.884+0000 7f7df8592740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cbe3d9c710 0x55cbe3dc69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.884+0000 7f7df8592740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cbe3d9caf0 0x55cbe3da3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.884+0000 7f7df8592740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d77a40 0x55cbe3d77ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.884+0000 7f7df8592740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cbe3ce0440 con 0x55cbe3d77a40 2024-05-08T05:45:08.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.884+0000 7f7df8592740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cbe3cdb120 con 0x55cbe3d9c710 2024-05-08T05:45:08.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.884+0000 7f7df8592740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55cbe3ce9210 con 0x55cbe3d9caf0 2024-05-08T05:45:08.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.884+0000 7f7df8592740 10 client.0 _ll_register_callbacks cb 0x55cbe3c80690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:45:08.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.884+0000 7f7def7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d77a40 0x55cbe3d77ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.884+0000 7f7def7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d77a40 0x55cbe3d77ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35420/0 (socket says 172.21.15.45:35420) 2024-05-08T05:45:08.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.884+0000 7f7def7fe640 1 -- 172.21.15.45:0/973451582 learned_addr learned my addr 172.21.15.45:0/973451582 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:08.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.884+0000 7f7dee7fc640 1 --2- 172.21.15.45:0/973451582 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cbe3d9caf0 0x55cbe3da3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.884+0000 7f7deeffd640 1 --2- 172.21.15.45:0/973451582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cbe3d9c710 0x55cbe3dc69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.884+0000 7f7def7fe640 1 -- 172.21.15.45:0/973451582 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cbe3d9caf0 msgr2=0x55cbe3da3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.884+0000 7f7def7fe640 1 --2- 172.21.15.45:0/973451582 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cbe3d9caf0 0x55cbe3da3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.884+0000 7f7def7fe640 1 -- 172.21.15.45:0/973451582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cbe3d9c710 msgr2=0x55cbe3dc69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.884+0000 7f7def7fe640 1 --2- 172.21.15.45:0/973451582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cbe3d9c710 0x55cbe3dc69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.884+0000 7f7def7fe640 1 -- 172.21.15.45:0/973451582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbe3b19f50 con 0x55cbe3d77a40 2024-05-08T05:45:08.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.885+0000 7f7def7fe640 1 --2- 172.21.15.45:0/973451582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d77a40 0x55cbe3d77ec0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7de8007aa0 tx=0x7f7de8007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e2047ad3a29dbd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:08.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.885+0000 7f7dce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:45:08.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.885+0000 7f7dce7fc640 1 -- 172.21.15.45:0/973451582 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de800fc70 con 0x55cbe3d77a40 2024-05-08T05:45:08.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.885+0000 7f7dce7fc640 1 -- 172.21.15.45:0/973451582 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f7de800fe10 con 0x55cbe3d77a40 2024-05-08T05:45:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.885+0000 7f7df8592740 1 -- 172.21.15.45:0/973451582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de405fbf0 con 0x55cbe3d77a40 2024-05-08T05:45:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.885+0000 7f7dce7fc640 1 -- 172.21.15.45:0/973451582 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7de8023050 con 0x55cbe3d77a40 2024-05-08T05:45:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.885+0000 7f7df8592740 10 client.6018 fetch_fsmap learned FSMap version 36 2024-05-08T05:45:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.885+0000 7f7df8592740 1 -- 172.21.15.45:0/973451582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55cbe39ad0b0 con 0x55cbe3d77a40 2024-05-08T05:45:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.885+0000 7f7dce7fc640 1 -- 172.21.15.45:0/973451582 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de405fbf0 con 0x55cbe3d77a40 2024-05-08T05:45:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.885+0000 7f7dce7fc640 1 -- 172.21.15.45:0/973451582 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f7de8026020 con 0x55cbe3d77a40 2024-05-08T05:45:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.885+0000 7f7df8592740 10 client.6018 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:45:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.885+0000 7f7df8592740 10 client.6018 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T05:45:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.885+0000 7f7df8592740 20 client.6018 populate_metadata read hostname 'smithi045' 2024-05-08T05:45:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.885+0000 7f7df8592740 1 --2- 172.21.15.45:0/973451582 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55cbe3da14b0 0x55cbe3da1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.885+0000 7f7df8592740 4 client.6018 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:45:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.885+0000 7f7df8592740 1 -- 172.21.15.45:0/973451582 --> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbe3ce9a20 con 0x55cbe3da14b0 2024-05-08T05:45:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.885+0000 7f7deeffd640 1 --2- 172.21.15.45:0/973451582 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55cbe3da14b0 0x55cbe3da1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.886+0000 7f7deeffd640 1 --2- 172.21.15.45:0/973451582 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55cbe3da14b0 0x55cbe3da1890 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5502d4d705bce8f2 server_cookie=9112ce040eddf982 in_seq=0 out_seq=0 2024-05-08T05:45:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.886+0000 7f7dce7fc640 10 client.6018 ms_handle_connect on v2:172.21.15.45:6833/1461822016 2024-05-08T05:45:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.886+0000 7f7dce7fc640 1 -- 172.21.15.45:0/973451582 <== mds.0 v2:172.21.15.45:6833/1461822016 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cbe3ce9a20 con 0x55cbe3da14b0 2024-05-08T05:45:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.886+0000 7f7dce7fc640 10 client.6018 handle_command_reply: tid=0 2024-05-08T05:45:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.886+0000 7f7df8592740 1 client.6018 shutdown 2024-05-08T05:45:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.886+0000 7f7df8592740 1 -- 172.21.15.45:0/973451582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d77a40 msgr2=0x55cbe3d77ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.886+0000 7f7df8592740 1 --2- 172.21.15.45:0/973451582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d77a40 0x55cbe3d77ec0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7de8007aa0 tx=0x7f7de8007f60 comp rx=0 tx=0).stop 2024-05-08T05:45:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.886+0000 7f7df8592740 1 -- 172.21.15.45:0/973451582 shutdown_connections 2024-05-08T05:45:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.886+0000 7f7df8592740 1 --2- 172.21.15.45:0/973451582 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55cbe3da14b0 0x55cbe3da1890 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.886+0000 7f7df8592740 1 --2- 172.21.15.45:0/973451582 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cbe3d9caf0 0x55cbe3da3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.886+0000 7f7df8592740 1 --2- 172.21.15.45:0/973451582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d77a40 0x55cbe3d77ec0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.886+0000 7f7dce7fc640 0 client.6018 ms_handle_reset on v2:172.21.15.45:6833/1461822016 2024-05-08T05:45:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.886+0000 7f7df8592740 1 --2- 172.21.15.45:0/973451582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cbe3d9c710 0x55cbe3dc69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.887+0000 7f7df8592740 1 -- 172.21.15.45:0/973451582 >> 172.21.15.45:0/973451582 conn(0x55cbe3d722f0 msgr2=0x55cbe3d78a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.887+0000 7f7df8592740 1 -- 172.21.15.45:0/973451582 shutdown_connections 2024-05-08T05:45:08.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.887+0000 7f7df8592740 1 -- 172.21.15.45:0/973451582 wait complete. 2024-05-08T05:45:08.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.887+0000 7f7df8592740 20 client.6018 trim_cache size 0 max 16384 2024-05-08T05:45:08.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.900+0000 7f7df8592740 1 Processor -- start 2024-05-08T05:45:08.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.900+0000 7f7df8592740 1 -- start start 2024-05-08T05:45:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.901+0000 7f7df8592740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cbe3d77a40 0x55cbe3da5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.901+0000 7f7df8592740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9c710 0x55cbe3de2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.901+0000 7f7df8592740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cbe3d9caf0 0x55cbe3d789b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.901+0000 7f7df8592740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7df00b3fd0 con 0x55cbe3d9c710 2024-05-08T05:45:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.901+0000 7f7df8592740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7df01f6610 con 0x55cbe3d9caf0 2024-05-08T05:45:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.901+0000 7f7df8592740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7df0158110 con 0x55cbe3d77a40 2024-05-08T05:45:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.901+0000 7f7dee7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9c710 0x55cbe3de2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.901+0000 7f7dee7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9c710 0x55cbe3de2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35436/0 (socket says 172.21.15.45:35436) 2024-05-08T05:45:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.901+0000 7f7dee7fc640 1 -- 172.21.15.45:0/983390559 learned_addr learned my addr 172.21.15.45:0/983390559 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.902+0000 7f7dee7fc640 1 -- 172.21.15.45:0/983390559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cbe3d77a40 msgr2=0x55cbe3da5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.902+0000 7f7dee7fc640 1 --2- 172.21.15.45:0/983390559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cbe3d77a40 0x55cbe3da5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.902+0000 7f7dee7fc640 1 -- 172.21.15.45:0/983390559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cbe3d9caf0 msgr2=0x55cbe3d789b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.902+0000 7f7dee7fc640 1 --2- 172.21.15.45:0/983390559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cbe3d9caf0 0x55cbe3d789b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.902+0000 7f7dee7fc640 1 -- 172.21.15.45:0/983390559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbe3da0050 con 0x55cbe3d9c710 2024-05-08T05:45:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.902+0000 7f7dee7fc640 1 --2- 172.21.15.45:0/983390559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9c710 0x55cbe3de2d90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7de4094f10 tx=0x7f7de4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee29e46e4817d73 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.902+0000 7f7dd4ff9640 1 -- 172.21.15.45:0/983390559 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4079040 con 0x55cbe3d9c710 2024-05-08T05:45:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.902+0000 7f7dd4ff9640 1 -- 172.21.15.45:0/983390559 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de4063fa0 con 0x55cbe3d9c710 2024-05-08T05:45:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.902+0000 7f7dd4ff9640 1 -- 172.21.15.45:0/983390559 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de405f070 con 0x55cbe3d9c710 2024-05-08T05:45:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.903+0000 7f7df8592740 1 -- 172.21.15.45:0/983390559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9c710 msgr2=0x55cbe3de2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.903+0000 7f7df8592740 1 --2- 172.21.15.45:0/983390559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9c710 0x55cbe3de2d90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7de4094f10 tx=0x7f7de4014000 comp rx=0 tx=0).stop 2024-05-08T05:45:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.903+0000 7f7df8592740 1 -- 172.21.15.45:0/983390559 shutdown_connections 2024-05-08T05:45:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.903+0000 7f7df8592740 1 --2- 172.21.15.45:0/983390559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cbe3d9caf0 0x55cbe3d789b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.903+0000 7f7df8592740 1 --2- 172.21.15.45:0/983390559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9c710 0x55cbe3de2d90 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.903+0000 7f7df8592740 1 --2- 172.21.15.45:0/983390559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cbe3d77a40 0x55cbe3da5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.903+0000 7f7df8592740 1 -- 172.21.15.45:0/983390559 >> 172.21.15.45:0/983390559 conn(0x55cbe3d722f0 msgr2=0x55cbe3da1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.903+0000 7f7df8592740 1 -- 172.21.15.45:0/983390559 shutdown_connections 2024-05-08T05:45:08.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.903+0000 7f7df8592740 1 -- 172.21.15.45:0/983390559 wait complete. 2024-05-08T05:45:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.903+0000 7f7df8592740 1 -- start start 2024-05-08T05:45:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.903+0000 7f7df8592740 1 Processor -- start 2024-05-08T05:45:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.904+0000 7f7df8592740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cbe3d77a40 0x55cbe3e3f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.904+0000 7f7df8592740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cbe3d9c710 0x55cbe3d711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.904+0000 7f7df8592740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9caf0 0x55cbe3da4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.904+0000 7f7df8592740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cbe3da2350 con 0x55cbe3d9caf0 2024-05-08T05:45:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.904+0000 7f7df8592740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cbe3c91a20 con 0x55cbe3d77a40 2024-05-08T05:45:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.904+0000 7f7df8592740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55cbe3d9fe30 con 0x55cbe3d9c710 2024-05-08T05:45:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.904+0000 7f7df8592740 10 client.0 _ll_register_callbacks cb 0x7f7de80098d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:45:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.904+0000 7f7def7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9caf0 0x55cbe3da4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.904+0000 7f7def7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9caf0 0x55cbe3da4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35450/0 (socket says 172.21.15.45:35450) 2024-05-08T05:45:08.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.904+0000 7f7def7fe640 1 -- 172.21.15.45:0/759950758 learned_addr learned my addr 172.21.15.45:0/759950758 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:08.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.904+0000 7f7deeffd640 1 --2- 172.21.15.45:0/759950758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cbe3d77a40 0x55cbe3e3f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.904+0000 7f7dee7fc640 1 --2- 172.21.15.45:0/759950758 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cbe3d9c710 0x55cbe3d711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.904+0000 7f7def7fe640 1 -- 172.21.15.45:0/759950758 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cbe3d9c710 msgr2=0x55cbe3d711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.904+0000 7f7def7fe640 1 --2- 172.21.15.45:0/759950758 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cbe3d9c710 0x55cbe3d711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.904+0000 7f7def7fe640 1 -- 172.21.15.45:0/759950758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cbe3d77a40 msgr2=0x55cbe3e3f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:08.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.904+0000 7f7def7fe640 1 --2- 172.21.15.45:0/759950758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cbe3d77a40 0x55cbe3e3f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.904+0000 7f7def7fe640 1 -- 172.21.15.45:0/759950758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbe3de1ca0 con 0x55cbe3d9caf0 2024-05-08T05:45:08.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.904+0000 7f7dee7fc640 1 --2- 172.21.15.45:0/759950758 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cbe3d9c710 0x55cbe3d711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:08.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.905+0000 7f7def7fe640 1 --2- 172.21.15.45:0/759950758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9caf0 0x55cbe3da4400 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7de8016770 tx=0x7f7de8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14f7cba8dd0d430 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:08.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.905+0000 7f7dce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:45:08.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.905+0000 7f7dce7fc640 1 -- 172.21.15.45:0/759950758 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8016820 con 0x55cbe3d9caf0 2024-05-08T05:45:08.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.905+0000 7f7dce7fc640 1 -- 172.21.15.45:0/759950758 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f7de80169c0 con 0x55cbe3d9caf0 2024-05-08T05:45:08.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.905+0000 7f7df8592740 1 -- 172.21.15.45:0/759950758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de4063fa0 con 0x55cbe3d9caf0 2024-05-08T05:45:08.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.905+0000 7f7dce7fc640 1 -- 172.21.15.45:0/759950758 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7de8036050 con 0x55cbe3d9caf0 2024-05-08T05:45:08.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.905+0000 7f7df8592740 10 client.6024 fetch_fsmap learned FSMap version 36 2024-05-08T05:45:08.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.905+0000 7f7df8592740 1 -- 172.21.15.45:0/759950758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55cbe3de2540 con 0x55cbe3d9caf0 2024-05-08T05:45:08.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.905+0000 7f7dce7fc640 1 -- 172.21.15.45:0/759950758 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de4063fa0 con 0x55cbe3d9caf0 2024-05-08T05:45:08.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.905+0000 7f7dce7fc640 1 -- 172.21.15.45:0/759950758 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f7de8039020 con 0x55cbe3d9caf0 2024-05-08T05:45:08.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.905+0000 7f7df8592740 10 client.6024 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:45:08.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.905+0000 7f7df8592740 10 client.6024 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T05:45:08.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.905+0000 7f7df8592740 20 client.6024 populate_metadata read hostname 'smithi045' 2024-05-08T05:45:08.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.905+0000 7f7df8592740 1 --2- 172.21.15.45:0/759950758 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55cbe3db2fd0 0x55cbe3d9ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:08.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.905+0000 7f7df8592740 4 client.6024 mds_command: new command op to 5697 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-08T05:45:08.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.905+0000 7f7df8592740 1 -- 172.21.15.45:0/759950758 --> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55cbe3ce3890 con 0x55cbe3db2fd0 2024-05-08T05:45:08.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.905+0000 7f7deeffd640 1 --2- 172.21.15.45:0/759950758 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55cbe3db2fd0 0x55cbe3d9ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:08.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.906+0000 7f7dce7fc640 10 client.6024 ms_handle_connect on v2:172.21.15.45:6833/1461822016 2024-05-08T05:45:08.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.906+0000 7f7deeffd640 1 --2- 172.21.15.45:0/759950758 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55cbe3db2fd0 0x55cbe3d9ac40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=781dd5007ca09719 server_cookie=a05875a69dc8e74b in_seq=0 out_seq=0 2024-05-08T05:45:08.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.906+0000 7f7dce7fc640 1 -- 172.21.15.45:0/759950758 <== mds.0 v2:172.21.15.45:6833/1461822016 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cbe3ce3890 con 0x55cbe3db2fd0 2024-05-08T05:45:08.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.906+0000 7f7dce7fc640 10 client.6024 handle_command_reply: tid=0 2024-05-08T05:45:08.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.906+0000 7f7df8592740 1 client.6024 shutdown 2024-05-08T05:45:08.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.906+0000 7f7df8592740 1 -- 172.21.15.45:0/759950758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9caf0 msgr2=0x55cbe3da4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.906+0000 7f7df8592740 1 --2- 172.21.15.45:0/759950758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9caf0 0x55cbe3da4400 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7de8016770 tx=0x7f7de8004ab0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.906+0000 7f7df8592740 1 -- 172.21.15.45:0/759950758 shutdown_connections 2024-05-08T05:45:08.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.906+0000 7f7df8592740 1 --2- 172.21.15.45:0/759950758 >> [v2:172.21.15.45:6833/1461822016,v1:172.21.15.45:6835/1461822016] conn(0x55cbe3db2fd0 0x55cbe3d9ac40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.906+0000 7f7df8592740 1 --2- 172.21.15.45:0/759950758 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cbe3d9c710 0x55cbe3d711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.906+0000 7f7dce7fc640 0 client.6024 ms_handle_reset on v2:172.21.15.45:6833/1461822016 2024-05-08T05:45:08.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.906+0000 7f7df8592740 1 --2- 172.21.15.45:0/759950758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cbe3d9caf0 0x55cbe3da4400 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.906+0000 7f7df8592740 1 --2- 172.21.15.45:0/759950758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cbe3d77a40 0x55cbe3e3f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.906+0000 7f7df8592740 1 -- 172.21.15.45:0/759950758 >> 172.21.15.45:0/759950758 conn(0x55cbe3d722f0 msgr2=0x55cbe3da5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:08.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.906+0000 7f7df8592740 1 -- 172.21.15.45:0/759950758 shutdown_connections 2024-05-08T05:45:08.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.906+0000 7f7df8592740 1 -- 172.21.15.45:0/759950758 wait complete. 2024-05-08T05:45:08.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.906+0000 7f7df8592740 20 client.6024 trim_cache size 0 max 16384 2024-05-08T05:45:08.915 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:08.916 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-05-08T05:45:08.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.907+0000 7f7df5872640 1 -- 172.21.15.45:0/252183242 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7ddc0762d0 msgr2=0x7f7ddc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.907+0000 7f7df5872640 1 --2- 172.21.15.45:0/252183242 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7ddc0762d0 0x7f7ddc078790 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7de0034000 tx=0x7f7de002e000 comp rx=0 tx=0).stop 2024-05-08T05:45:08.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.907+0000 7f7df5872640 1 -- 172.21.15.45:0/252183242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df006e4d0 msgr2=0x7f7df00b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:08.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.907+0000 7f7df5872640 1 --2- 172.21.15.45:0/252183242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df006e4d0 0x7f7df00b26a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7de400c370 tx=0x7f7de400c830 comp rx=0 tx=0).stop 2024-05-08T05:45:08.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.907+0000 7f7df5872640 1 -- 172.21.15.45:0/252183242 shutdown_connections 2024-05-08T05:45:08.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.907+0000 7f7df5872640 1 --2- 172.21.15.45:0/252183242 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7ddc0762d0 0x7f7ddc078790 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.907+0000 7f7df5872640 1 --2- 172.21.15.45:0/252183242 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7df0072840 0x7f7df0153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.907+0000 7f7df5872640 1 --2- 172.21.15.45:0/252183242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7df006e4d0 0x7f7df00b26a0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.907+0000 7f7df5872640 1 --2- 172.21.15.45:0/252183242 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7df006e0f0 0x7f7df014d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:08.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.907+0000 7f7df5872640 1 -- 172.21.15.45:0/252183242 >> 172.21.15.45:0/252183242 conn(0x7f7df006c3a0 msgr2=0x7f7df015af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:08.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.907+0000 7f7df5872640 1 -- 172.21.15.45:0/252183242 shutdown_connections 2024-05-08T05:45:08.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:08.907+0000 7f7df5872640 1 -- 172.21.15.45:0/252183242 wait complete. 2024-05-08T05:45:08.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T05:45:09.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.022+0000 7f4407b51640 1 Processor -- start 2024-05-08T05:45:09.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.023+0000 7f4407b51640 1 -- start start 2024-05-08T05:45:09.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.023+0000 7f4407b51640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f440015f420 0x7f44000d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:09.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.023+0000 7f4407b51640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44000d3ca0 0x7f44000d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:09.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.023+0000 7f4407b51640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44001630b0 con 0x7f44000d1eb0 2024-05-08T05:45:09.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.023+0000 7f4407b51640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f44000d22f0 con 0x7f440015f420 2024-05-08T05:45:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.023+0000 7f4407b51640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f44000d2550 con 0x7f44000d3ca0 2024-05-08T05:45:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.024+0000 7f44058ed640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f440015f420 0x7f44000d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.024+0000 7f44058ed640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f440015f420 0x7f44000d3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35464/0 (socket says 172.21.15.45:35464) 2024-05-08T05:45:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.024+0000 7f44058ed640 1 -- 172.21.15.45:0/910625874 learned_addr learned my addr 172.21.15.45:0/910625874 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.024+0000 7f44050ec640 1 --2- 172.21.15.45:0/910625874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44000d3ca0 0x7f44000d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.024+0000 7f44058ed640 1 -- 172.21.15.45:0/910625874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44000d3ca0 msgr2=0x7f44000d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.024+0000 7f44058ed640 1 --2- 172.21.15.45:0/910625874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44000d3ca0 0x7f44000d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.024+0000 7f44058ed640 1 -- 172.21.15.45:0/910625874 >> v1:172.21.15.63:6789/0 conn(0x7f44000d1eb0 legacy=0x7f44000d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.024+0000 7f44058ed640 1 -- 172.21.15.45:0/910625874 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44000d2c80 con 0x7f440015f420 2024-05-08T05:45:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.025+0000 7f44058ed640 1 --2- 172.21.15.45:0/910625874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440015f420 0x7f44000d3760 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009870 tx=0x7f43fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2deae7567a48d4e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.025+0000 7f44048eb640 1 -- 172.21.15.45:0/910625874 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc037070 con 0x7f440015f420 2024-05-08T05:45:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.025+0000 7f44048eb640 1 -- 172.21.15.45:0/910625874 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43fc032440 con 0x7f440015f420 2024-05-08T05:45:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.025+0000 7f44048eb640 1 -- 172.21.15.45:0/910625874 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc032810 con 0x7f440015f420 2024-05-08T05:45:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.025+0000 7f4407b51640 1 -- 172.21.15.45:0/910625874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440015f420 msgr2=0x7f44000d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.025+0000 7f4407b51640 1 --2- 172.21.15.45:0/910625874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440015f420 0x7f44000d3760 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009870 tx=0x7f43fc02a270 comp rx=0 tx=0).stop 2024-05-08T05:45:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.025+0000 7f4407b51640 1 -- 172.21.15.45:0/910625874 shutdown_connections 2024-05-08T05:45:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.025+0000 7f4407b51640 1 --2- 172.21.15.45:0/910625874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44000d3ca0 0x7f44000d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.025+0000 7f4407b51640 1 --2- 172.21.15.45:0/910625874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f440015f420 0x7f44000d3760 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.025+0000 7f4407b51640 1 -- 172.21.15.45:0/910625874 >> 172.21.15.45:0/910625874 conn(0x7f440008eac0 msgr2=0x7f440009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.025+0000 7f4407b51640 1 -- 172.21.15.45:0/910625874 shutdown_connections 2024-05-08T05:45:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.025+0000 7f4407b51640 1 -- 172.21.15.45:0/910625874 wait complete. 2024-05-08T05:45:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.027+0000 7f4407b51640 1 Processor -- start 2024-05-08T05:45:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.027+0000 7f4407b51640 1 -- start start 2024-05-08T05:45:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.027+0000 7f4407b51640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44000d1eb0 0x7f4400160950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.027+0000 7f4407b51640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4400161460 0x7f44001f4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.027+0000 7f4407b51640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44000d0480 0x7f44000d0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.027+0000 7f4407b51640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f44000d2830 con 0x7f44000d1eb0 2024-05-08T05:45:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.027+0000 7f4407b51640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f44000d3240 con 0x7f4400161460 2024-05-08T05:45:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.027+0000 7f4407b51640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f44000d53d0 con 0x7f44000d0480 2024-05-08T05:45:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.028+0000 7f44058ed640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44000d1eb0 0x7f4400160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:09.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.028+0000 7f44058ed640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44000d1eb0 0x7f4400160950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35476/0 (socket says 172.21.15.45:35476) 2024-05-08T05:45:09.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.028+0000 7f44058ed640 1 -- 172.21.15.45:0/884104172 learned_addr learned my addr 172.21.15.45:0/884104172 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:09.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.028+0000 7f44060ee640 1 --2- 172.21.15.45:0/884104172 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44000d0480 0x7f44000d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:09.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.028+0000 7f44050ec640 1 --2- 172.21.15.45:0/884104172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4400161460 0x7f44001f4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:09.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.028+0000 7f44058ed640 1 -- 172.21.15.45:0/884104172 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44000d0480 msgr2=0x7f44000d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:09.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.028+0000 7f44058ed640 1 --2- 172.21.15.45:0/884104172 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44000d0480 0x7f44000d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.028+0000 7f44058ed640 1 -- 172.21.15.45:0/884104172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4400161460 msgr2=0x7f44001f4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:09.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.028+0000 7f44058ed640 1 --2- 172.21.15.45:0/884104172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4400161460 0x7f44001f4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.028+0000 7f44058ed640 1 -- 172.21.15.45:0/884104172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44001618e0 con 0x7f44000d1eb0 2024-05-08T05:45:09.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.028+0000 7f44050ec640 1 --2- 172.21.15.45:0/884104172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4400161460 0x7f44001f4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:09.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.028+0000 7f44060ee640 1 --2- 172.21.15.45:0/884104172 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44000d0480 0x7f44000d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:09.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.028+0000 7f44058ed640 1 --2- 172.21.15.45:0/884104172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44000d1eb0 0x7f4400160950 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f43fc000c00 tx=0x7f43fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:09.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.028+0000 7f43f67fc640 1 -- 172.21.15.45:0/884104172 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc037040 con 0x7f44000d1eb0 2024-05-08T05:45:09.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.028+0000 7f43f67fc640 1 -- 172.21.15.45:0/884104172 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43fc042070 con 0x7f44000d1eb0 2024-05-08T05:45:09.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.028+0000 7f43f67fc640 1 -- 172.21.15.45:0/884104172 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc030880 con 0x7f44000d1eb0 2024-05-08T05:45:09.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.028+0000 7f4407b51640 1 -- 172.21.15.45:0/884104172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44001f4e50 con 0x7f44000d1eb0 2024-05-08T05:45:09.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.028+0000 7f4407b51640 1 -- 172.21.15.45:0/884104172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4400161e90 con 0x7f44000d1eb0 2024-05-08T05:45:09.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.029+0000 7f4407b51640 1 -- 172.21.15.45:0/884104172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44000d0e40 con 0x7f44000d1eb0 2024-05-08T05:45:09.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.030+0000 7f43f67fc640 1 -- 172.21.15.45:0/884104172 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f43fc030a20 con 0x7f44000d1eb0 2024-05-08T05:45:09.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.030+0000 7f43f67fc640 1 --2- 172.21.15.45:0/884104172 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f43e40762d0 0x7f43e4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:09.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.031+0000 7f43f67fc640 1 -- 172.21.15.45:0/884104172 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f43fc0b7fe0 con 0x7f44000d1eb0 2024-05-08T05:45:09.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.031+0000 7f44050ec640 1 --2- 172.21.15.45:0/884104172 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f43e40762d0 0x7f43e4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:09.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.031+0000 7f44050ec640 1 --2- 172.21.15.45:0/884104172 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f43e40762d0 0x7f43e4078790 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f44000b2020 tx=0x7f43f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:09.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.035+0000 7f43f67fc640 1 -- 172.21.15.45:0/884104172 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f43fc0817b0 con 0x7f44000d1eb0 2024-05-08T05:45:09.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.266+0000 7f4407b51640 1 -- 172.21.15.45:0/884104172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f44000693f0 con 0x7f44000d1eb0 2024-05-08T05:45:09.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.267+0000 7f43f67fc640 1 -- 172.21.15.45:0/884104172 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f43fc0810b0 con 0x7f44000d1eb0 2024-05-08T05:45:09.267 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:09.267 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T05:45:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.269+0000 7f4407b51640 1 -- 172.21.15.45:0/884104172 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f43e40762d0 msgr2=0x7f43e4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.269+0000 7f4407b51640 1 --2- 172.21.15.45:0/884104172 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f43e40762d0 0x7f43e4078790 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f44000b2020 tx=0x7f43f0008040 comp rx=0 tx=0).stop 2024-05-08T05:45:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.269+0000 7f4407b51640 1 -- 172.21.15.45:0/884104172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44000d1eb0 msgr2=0x7f4400160950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.269+0000 7f4407b51640 1 --2- 172.21.15.45:0/884104172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44000d1eb0 0x7f4400160950 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f43fc000c00 tx=0x7f43fc0046c0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.269+0000 7f4407b51640 1 -- 172.21.15.45:0/884104172 shutdown_connections 2024-05-08T05:45:09.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.269+0000 7f4407b51640 1 --2- 172.21.15.45:0/884104172 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f43e40762d0 0x7f43e4078790 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.269+0000 7f4407b51640 1 --2- 172.21.15.45:0/884104172 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f44000d0480 0x7f44000d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.269+0000 7f4407b51640 1 --2- 172.21.15.45:0/884104172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4400161460 0x7f44001f4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.269+0000 7f4407b51640 1 --2- 172.21.15.45:0/884104172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44000d1eb0 0x7f4400160950 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.269+0000 7f4407b51640 1 -- 172.21.15.45:0/884104172 >> 172.21.15.45:0/884104172 conn(0x7f440008eac0 msgr2=0x7f440006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:09.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.269+0000 7f4407b51640 1 -- 172.21.15.45:0/884104172 shutdown_connections 2024-05-08T05:45:09.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.269+0000 7f4407b51640 1 -- 172.21.15.45:0/884104172 wait complete. 2024-05-08T05:45:09.280 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-08T05:45:09.280 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T05:45:09.280 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:45:09.325 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:45:09.368 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T05:45:09.368 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:45:09.368 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:45:09.368 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:45:09.384 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:45:09.385 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:45:09.441 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:45:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.585+0000 7f99348c8640 1 Processor -- start 2024-05-08T05:45:09.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.585+0000 7f99348c8640 1 -- start start 2024-05-08T05:45:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.586+0000 7f99348c8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f993006d620 0x7f9930073800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.586+0000 7f99348c8640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f993006da00 0x7f99300af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:09.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.586+0000 7f99348c8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99300b5ee0 0x7f993006e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.586+0000 7f99348c8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f99300afaa0 con 0x7f993006d620 2024-05-08T05:45:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.586+0000 7f99348c8640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f99300b5a50 con 0x7f993006da00 2024-05-08T05:45:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.586+0000 7f99348c8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f99300af800 con 0x7f99300b5ee0 2024-05-08T05:45:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.586+0000 7f992e59c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f993006d620 0x7f9930073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.586+0000 7f992e59c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f993006d620 0x7f9930073800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35488/0 (socket says 172.21.15.45:35488) 2024-05-08T05:45:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.586+0000 7f992e59c640 1 -- 172.21.15.45:0/267055609 learned_addr learned my addr 172.21.15.45:0/267055609 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.586+0000 7f9927fff640 1 --2- 172.21.15.45:0/267055609 >> v2:172.21.15.63:3300/0 conn(0x7f993006da00 0x7f99300af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.586+0000 7f992ed9d640 1 --2- 172.21.15.45:0/267055609 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99300b5ee0 0x7f993006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.586+0000 7f992e59c640 1 -- 172.21.15.45:0/267055609 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99300b5ee0 msgr2=0x7f993006e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.586+0000 7f992e59c640 1 --2- 172.21.15.45:0/267055609 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99300b5ee0 0x7f993006e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.586+0000 7f992e59c640 1 -- 172.21.15.45:0/267055609 >> v2:172.21.15.63:3300/0 conn(0x7f993006da00 msgr2=0x7f99300af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.586+0000 7f992e59c640 1 --2- 172.21.15.45:0/267055609 >> v2:172.21.15.63:3300/0 conn(0x7f993006da00 0x7f99300af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.586+0000 7f992e59c640 1 -- 172.21.15.45:0/267055609 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99300b5120 con 0x7f993006d620 2024-05-08T05:45:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.586+0000 7f9927fff640 1 --2- 172.21.15.45:0/267055609 >> v2:172.21.15.63:3300/0 conn(0x7f993006da00 0x7f99300af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.587+0000 7f992e59c640 1 --2- 172.21.15.45:0/267055609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f993006d620 0x7f9930073800 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f99300aebe0 tx=0x7f992002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42ee26fb36fdc74 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.587+0000 7f992dd9b640 1 -- 172.21.15.45:0/267055609 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9920037070 con 0x7f993006d620 2024-05-08T05:45:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.587+0000 7f992dd9b640 1 -- 172.21.15.45:0/267055609 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f992002ae00 con 0x7f993006d620 2024-05-08T05:45:09.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.587+0000 7f992dd9b640 1 -- 172.21.15.45:0/267055609 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9920032660 con 0x7f993006d620 2024-05-08T05:45:09.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.588+0000 7f99348c8640 1 -- 172.21.15.45:0/267055609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f993006d620 msgr2=0x7f9930073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:09.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.588+0000 7f99348c8640 1 --2- 172.21.15.45:0/267055609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f993006d620 0x7f9930073800 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f99300aebe0 tx=0x7f992002a0b0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.588+0000 7f99348c8640 1 -- 172.21.15.45:0/267055609 shutdown_connections 2024-05-08T05:45:09.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.588+0000 7f99348c8640 1 --2- 172.21.15.45:0/267055609 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99300b5ee0 0x7f993006e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.588+0000 7f99348c8640 1 --2- 172.21.15.45:0/267055609 >> v2:172.21.15.63:3300/0 conn(0x7f993006da00 0x7f99300af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.588+0000 7f99348c8640 1 --2- 172.21.15.45:0/267055609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f993006d620 0x7f9930073800 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.588+0000 7f99348c8640 1 -- 172.21.15.45:0/267055609 >> 172.21.15.45:0/267055609 conn(0x7f99300728c0 msgr2=0x7f9930072cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:09.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.588+0000 7f99348c8640 1 -- 172.21.15.45:0/267055609 shutdown_connections 2024-05-08T05:45:09.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.588+0000 7f99348c8640 1 -- 172.21.15.45:0/267055609 wait complete. 2024-05-08T05:45:09.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.589+0000 7f99348c8640 1 Processor -- start 2024-05-08T05:45:09.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.589+0000 7f99348c8640 1 -- start start 2024-05-08T05:45:09.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.590+0000 7f99348c8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f993006d620 0x7f9930150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:09.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.590+0000 7f99348c8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f993006da00 0x7f9930155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:09.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.590+0000 7f99348c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99300b5ee0 0x7f99300cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:09.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.590+0000 7f99348c8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f99300b6370 con 0x7f99300b5ee0 2024-05-08T05:45:09.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.590+0000 7f99348c8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f99300b56e0 con 0x7f993006da00 2024-05-08T05:45:09.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.590+0000 7f99348c8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f99300b64f0 con 0x7f993006d620 2024-05-08T05:45:09.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.590+0000 7f992ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99300b5ee0 0x7f99300cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:09.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.590+0000 7f992ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99300b5ee0 0x7f99300cf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35504/0 (socket says 172.21.15.45:35504) 2024-05-08T05:45:09.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.590+0000 7f992ed9d640 1 -- 172.21.15.45:0/1941242126 learned_addr learned my addr 172.21.15.45:0/1941242126 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:09.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.590+0000 7f992e59c640 1 --2- 172.21.15.45:0/1941242126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f993006d620 0x7f9930150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:09.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.590+0000 7f9927fff640 1 --2- 172.21.15.45:0/1941242126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f993006da00 0x7f9930155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:09.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.590+0000 7f992ed9d640 1 -- 172.21.15.45:0/1941242126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f993006d620 msgr2=0x7f9930150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:09.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.590+0000 7f992ed9d640 1 --2- 172.21.15.45:0/1941242126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f993006d620 0x7f9930150c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.590+0000 7f992ed9d640 1 -- 172.21.15.45:0/1941242126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f993006da00 msgr2=0x7f9930155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.590+0000 7f992ed9d640 1 --2- 172.21.15.45:0/1941242126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f993006da00 0x7f9930155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.590+0000 7f992ed9d640 1 -- 172.21.15.45:0/1941242126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9930157500 con 0x7f99300b5ee0 2024-05-08T05:45:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.591+0000 7f992ed9d640 1 --2- 172.21.15.45:0/1941242126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99300b5ee0 0x7f99300cf7f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f992800cc30 tx=0x7f992800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.591+0000 7f9926ffd640 1 -- 172.21.15.45:0/1941242126 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992800ce40 con 0x7f99300b5ee0 2024-05-08T05:45:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.591+0000 7f99348c8640 1 -- 172.21.15.45:0/1941242126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9930156460 con 0x7f99300b5ee0 2024-05-08T05:45:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.591+0000 7f9926ffd640 1 -- 172.21.15.45:0/1941242126 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9928003990 con 0x7f99300b5ee0 2024-05-08T05:45:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.591+0000 7f9926ffd640 1 -- 172.21.15.45:0/1941242126 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9928003d40 con 0x7f99300b5ee0 2024-05-08T05:45:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.591+0000 7f99348c8640 1 -- 172.21.15.45:0/1941242126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9930156d00 con 0x7f99300b5ee0 2024-05-08T05:45:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.592+0000 7f99348c8640 1 -- 172.21.15.45:0/1941242126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99300693f0 con 0x7f99300b5ee0 2024-05-08T05:45:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.593+0000 7f9926ffd640 1 -- 172.21.15.45:0/1941242126 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9928004000 con 0x7f99300b5ee0 2024-05-08T05:45:09.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.593+0000 7f9926ffd640 1 --2- 172.21.15.45:0/1941242126 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99100762d0 0x7f9910078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:09.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.593+0000 7f9926ffd640 1 -- 172.21.15.45:0/1941242126 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f9928097d00 con 0x7f99300b5ee0 2024-05-08T05:45:09.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.593+0000 7f992e59c640 1 --2- 172.21.15.45:0/1941242126 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99100762d0 0x7f9910078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:09.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.594+0000 7f992e59c640 1 --2- 172.21.15.45:0/1941242126 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99100762d0 0x7f9910078790 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9920034000 tx=0x7f992002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:09.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.598+0000 7f9926ffd640 1 -- 172.21.15.45:0/1941242126 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9928014030 con 0x7f99300b5ee0 2024-05-08T05:45:09.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.803+0000 7f99348c8640 1 -- 172.21.15.45:0/1941242126 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f993015cac0 con 0x7f99300b5ee0 2024-05-08T05:45:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.804+0000 7f9926ffd640 1 -- 172.21.15.45:0/1941242126 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v63) ==== 80+0+0 (secure 0 0 0) 0x7f99280613c0 con 0x7f99300b5ee0 2024-05-08T05:45:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-05-08T05:45:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.806+0000 7f99348c8640 1 -- 172.21.15.45:0/1941242126 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99100762d0 msgr2=0x7f9910078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.806+0000 7f99348c8640 1 --2- 172.21.15.45:0/1941242126 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99100762d0 0x7f9910078790 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9920034000 tx=0x7f992002e000 comp rx=0 tx=0).stop 2024-05-08T05:45:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.806+0000 7f99348c8640 1 -- 172.21.15.45:0/1941242126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99300b5ee0 msgr2=0x7f99300cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:09.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.806+0000 7f99348c8640 1 --2- 172.21.15.45:0/1941242126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99300b5ee0 0x7f99300cf7f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f992800cc30 tx=0x7f992800a5c0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.806+0000 7f99348c8640 1 -- 172.21.15.45:0/1941242126 shutdown_connections 2024-05-08T05:45:09.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.806+0000 7f99348c8640 1 --2- 172.21.15.45:0/1941242126 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99100762d0 0x7f9910078790 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.806+0000 7f99348c8640 1 --2- 172.21.15.45:0/1941242126 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99300b5ee0 0x7f99300cf7f0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.806+0000 7f99348c8640 1 --2- 172.21.15.45:0/1941242126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f993006da00 0x7f9930155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.806+0000 7f99348c8640 1 --2- 172.21.15.45:0/1941242126 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f993006d620 0x7f9930150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:09.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.806+0000 7f99348c8640 1 -- 172.21.15.45:0/1941242126 >> 172.21.15.45:0/1941242126 conn(0x7f99300728c0 msgr2=0x7f9930155310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:09.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.806+0000 7f99348c8640 1 -- 172.21.15.45:0/1941242126 shutdown_connections 2024-05-08T05:45:09.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:09.806+0000 7f99348c8640 1 -- 172.21.15.45:0/1941242126 wait complete. 2024-05-08T05:45:09.817 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T05:45:09.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:09.817 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T05:45:09.871 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:45:09.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:09.872 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T05:45:09.895 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:45:09.896 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:45:09.896 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:45:09.914 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:45:09.914 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:45:09.914 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:45:09.914 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:45:09.952 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:45:09.952 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:45:09.952 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:45:09.953 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:45:09.972 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:45:09.972 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:45:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.096+0000 7ff93782a640 1 Processor -- start 2024-05-08T05:45:10.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.096+0000 7ff93782a640 1 -- start start 2024-05-08T05:45:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.097+0000 7ff93782a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff9301629a0 0x7ff9300d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.097+0000 7ff93782a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff9300d0cb0 0x7ff9300d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.097+0000 7ff93782a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9300d3330 con 0x7ff9300d2dc0 2024-05-08T05:45:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.097+0000 7ff93782a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff9300d2170 con 0x7ff9300d0cb0 2024-05-08T05:45:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.097+0000 7ff93782a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff9300d2450 con 0x7ff9301629a0 2024-05-08T05:45:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.097+0000 7ff935dc7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff9300d0cb0 0x7ff9300d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.097+0000 7ff935dc7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff9300d0cb0 0x7ff9300d3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35520/0 (socket says 172.21.15.45:35520) 2024-05-08T05:45:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.097+0000 7ff935dc7640 1 -- 172.21.15.45:0/1067929530 learned_addr learned my addr 172.21.15.45:0/1067929530 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.097+0000 7ff9355c6640 1 --2- 172.21.15.45:0/1067929530 >> v2:172.21.15.63:3300/0 conn(0x7ff9301629a0 0x7ff9300d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:10.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.097+0000 7ff935dc7640 1 -- 172.21.15.45:0/1067929530 >> v2:172.21.15.63:3300/0 conn(0x7ff9301629a0 msgr2=0x7ff9300d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.097+0000 7ff935dc7640 1 --2- 172.21.15.45:0/1067929530 >> v2:172.21.15.63:3300/0 conn(0x7ff9301629a0 0x7ff9300d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.097+0000 7ff935dc7640 1 -- 172.21.15.45:0/1067929530 >> v1:172.21.15.63:6789/0 conn(0x7ff9300d2dc0 legacy=0x7ff9300c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:10.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.097+0000 7ff935dc7640 1 -- 172.21.15.45:0/1067929530 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9300d2730 con 0x7ff9300d0cb0 2024-05-08T05:45:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.098+0000 7ff935dc7640 1 --2- 172.21.15.45:0/1067929530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9300d0cb0 0x7ff9300d3b00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff9300768a0 tx=0x7ff92802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8714b8f626a5af36 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.098+0000 7ff927fff640 1 -- 172.21.15.45:0/1067929530 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff928036070 con 0x7ff9300d0cb0 2024-05-08T05:45:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.098+0000 7ff927fff640 1 -- 172.21.15.45:0/1067929530 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff928031600 con 0x7ff9300d0cb0 2024-05-08T05:45:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.098+0000 7ff927fff640 1 -- 172.21.15.45:0/1067929530 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9280319d0 con 0x7ff9300d0cb0 2024-05-08T05:45:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.099+0000 7ff93782a640 1 -- 172.21.15.45:0/1067929530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9300d0cb0 msgr2=0x7ff9300d3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:10.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.099+0000 7ff93782a640 1 --2- 172.21.15.45:0/1067929530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9300d0cb0 0x7ff9300d3b00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff9300768a0 tx=0x7ff92802a610 comp rx=0 tx=0).stop 2024-05-08T05:45:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.099+0000 7ff93782a640 1 -- 172.21.15.45:0/1067929530 shutdown_connections 2024-05-08T05:45:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.099+0000 7ff93782a640 1 --2- 172.21.15.45:0/1067929530 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9300d0cb0 0x7ff9300d3b00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.099+0000 7ff93782a640 1 --2- 172.21.15.45:0/1067929530 >> v2:172.21.15.63:3300/0 conn(0x7ff9301629a0 0x7ff9300d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.099+0000 7ff93782a640 1 -- 172.21.15.45:0/1067929530 >> 172.21.15.45:0/1067929530 conn(0x7ff9300d4fe0 msgr2=0x7ff9300d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.099+0000 7ff93782a640 1 -- 172.21.15.45:0/1067929530 shutdown_connections 2024-05-08T05:45:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.099+0000 7ff93782a640 1 -- 172.21.15.45:0/1067929530 wait complete. 2024-05-08T05:45:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.100+0000 7ff93782a640 1 Processor -- start 2024-05-08T05:45:10.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.100+0000 7ff93782a640 1 -- start start 2024-05-08T05:45:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.101+0000 7ff93782a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9300d0cb0 0x7ff9301f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.101+0000 7ff93782a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9300d2dc0 0x7ff93015b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.101+0000 7ff93782a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9301629a0 0x7ff93015bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.101+0000 7ff93782a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9300cf440 con 0x7ff9300d2dc0 2024-05-08T05:45:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.101+0000 7ff93782a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9300cf2c0 con 0x7ff9301629a0 2024-05-08T05:45:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.101+0000 7ff93782a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff9300c95e0 con 0x7ff9300d0cb0 2024-05-08T05:45:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.101+0000 7ff934dc5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9300d2dc0 0x7ff93015b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.101+0000 7ff934dc5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9300d2dc0 0x7ff93015b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35534/0 (socket says 172.21.15.45:35534) 2024-05-08T05:45:10.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.101+0000 7ff934dc5640 1 -- 172.21.15.45:0/823435046 learned_addr learned my addr 172.21.15.45:0/823435046 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.101+0000 7ff935dc7640 1 --2- 172.21.15.45:0/823435046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9301629a0 0x7ff93015bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.101+0000 7ff9355c6640 1 --2- 172.21.15.45:0/823435046 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9300d0cb0 0x7ff9301f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.101+0000 7ff934dc5640 1 -- 172.21.15.45:0/823435046 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9300d0cb0 msgr2=0x7ff9301f7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.101+0000 7ff934dc5640 1 --2- 172.21.15.45:0/823435046 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9300d0cb0 0x7ff9301f7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.101+0000 7ff934dc5640 1 -- 172.21.15.45:0/823435046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9301629a0 msgr2=0x7ff93015bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.101+0000 7ff934dc5640 1 --2- 172.21.15.45:0/823435046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9301629a0 0x7ff93015bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.101+0000 7ff934dc5640 1 -- 172.21.15.45:0/823435046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9300cab00 con 0x7ff9300d2dc0 2024-05-08T05:45:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.101+0000 7ff935dc7640 1 --2- 172.21.15.45:0/823435046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9301629a0 0x7ff93015bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.102+0000 7ff934dc5640 1 --2- 172.21.15.45:0/823435046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9300d2dc0 0x7ff93015b840 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff92000c450 tx=0x7ff92000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.102+0000 7ff925ffb640 1 -- 172.21.15.45:0/823435046 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9200052d0 con 0x7ff9300d2dc0 2024-05-08T05:45:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.102+0000 7ff925ffb640 1 -- 172.21.15.45:0/823435046 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff920005470 con 0x7ff9300d2dc0 2024-05-08T05:45:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.102+0000 7ff93782a640 1 -- 172.21.15.45:0/823435046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9300cade0 con 0x7ff9300d2dc0 2024-05-08T05:45:10.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.102+0000 7ff925ffb640 1 -- 172.21.15.45:0/823435046 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920012870 con 0x7ff9300d2dc0 2024-05-08T05:45:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.102+0000 7ff93782a640 1 -- 172.21.15.45:0/823435046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9300cb3a0 con 0x7ff9300d2dc0 2024-05-08T05:45:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.103+0000 7ff93782a640 1 -- 172.21.15.45:0/823435046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9300693f0 con 0x7ff9300d2dc0 2024-05-08T05:45:10.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.104+0000 7ff925ffb640 1 -- 172.21.15.45:0/823435046 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff920012a10 con 0x7ff9300d2dc0 2024-05-08T05:45:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.104+0000 7ff925ffb640 1 --2- 172.21.15.45:0/823435046 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9140762d0 0x7ff914078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.104+0000 7ff925ffb640 1 -- 172.21.15.45:0/823435046 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7ff920097390 con 0x7ff9300d2dc0 2024-05-08T05:45:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.105+0000 7ff9355c6640 1 --2- 172.21.15.45:0/823435046 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9140762d0 0x7ff914078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:10.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.105+0000 7ff9355c6640 1 --2- 172.21.15.45:0/823435046 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9140762d0 0x7ff914078790 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009970 tx=0x7ff92c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:10.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.109+0000 7ff925ffb640 1 -- 172.21.15.45:0/823435046 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff920060b60 con 0x7ff9300d2dc0 2024-05-08T05:45:10.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.338+0000 7ff93782a640 1 -- 172.21.15.45:0/823435046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9300c90e0 con 0x7ff9300d2dc0 2024-05-08T05:45:10.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.339+0000 7ff925ffb640 1 -- 172.21.15.45:0/823435046 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5180 (secure 0 0 0) 0x7ff920060460 con 0x7ff9300d2dc0 2024-05-08T05:45:10.339 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:10.339 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/162765657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":162765657},{"type":"v1","addr":"172.21.15.63:6836","nonce":162765657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5356,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1788513331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1788513331},{"type":"v1","addr":"172.21.15.45:6834","nonce":1788513331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1173167340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1173167340},{"type":"v1","addr":"172.21.15.63:6837","nonce":1173167340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:43:58.107819+0000","modified":"2024-05-08T05:44:38.767928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":13,"addr":"172.21.15.45:6835/1461822016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1461822016},{"type":"v1","addr":"172.21.15.45:6835","nonce":1461822016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5697,"qdb_cluster":[5697]},"id":4}]} 2024-05-08T05:45:10.340 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 36 2024-05-08T05:45:10.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.341+0000 7ff93782a640 1 -- 172.21.15.45:0/823435046 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9140762d0 msgr2=0x7ff914078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:10.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.341+0000 7ff93782a640 1 --2- 172.21.15.45:0/823435046 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9140762d0 0x7ff914078790 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009970 tx=0x7ff92c008040 comp rx=0 tx=0).stop 2024-05-08T05:45:10.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.341+0000 7ff93782a640 1 -- 172.21.15.45:0/823435046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9300d2dc0 msgr2=0x7ff93015b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:10.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.341+0000 7ff93782a640 1 --2- 172.21.15.45:0/823435046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9300d2dc0 0x7ff93015b840 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff92000c450 tx=0x7ff92000c910 comp rx=0 tx=0).stop 2024-05-08T05:45:10.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.341+0000 7ff93782a640 1 -- 172.21.15.45:0/823435046 shutdown_connections 2024-05-08T05:45:10.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.341+0000 7ff93782a640 1 --2- 172.21.15.45:0/823435046 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9140762d0 0x7ff914078790 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.341+0000 7ff93782a640 1 --2- 172.21.15.45:0/823435046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9301629a0 0x7ff93015bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.341+0000 7ff93782a640 1 --2- 172.21.15.45:0/823435046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9300d2dc0 0x7ff93015b840 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.341+0000 7ff93782a640 1 --2- 172.21.15.45:0/823435046 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9300d0cb0 0x7ff9301f7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.341+0000 7ff93782a640 1 -- 172.21.15.45:0/823435046 >> 172.21.15.45:0/823435046 conn(0x7ff9300d4fe0 msgr2=0x7ff9300b7190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:10.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.341+0000 7ff93782a640 1 -- 172.21.15.45:0/823435046 shutdown_connections 2024-05-08T05:45:10.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.341+0000 7ff93782a640 1 -- 172.21.15.45:0/823435046 wait complete. 2024-05-08T05:45:10.352 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:45:10.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.455+0000 7f69125ef640 1 Processor -- start 2024-05-08T05:45:10.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.455+0000 7f69125ef640 1 -- start start 2024-05-08T05:45:10.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.456+0000 7f69125ef640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f690c0d20c0 0x7f690c163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.456+0000 7f69125ef640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f690c0d1300 con 0x7f690c071110 2024-05-08T05:45:10.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.456+0000 7f69125ef640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f690c0d15e0 con 0x7f690c070d30 2024-05-08T05:45:10.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.456+0000 7f69125ef640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f690c0d10d0 con 0x7f690c0d20c0 2024-05-08T05:45:10.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.456+0000 7f6910b8c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f690c071110 0x7f690c06d4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:42858/0 (socket says 172.21.15.45:42858) 2024-05-08T05:45:10.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.456+0000 7f6910b8c640 1 -- 172.21.15.45:0/2320704973 learned_addr learned my addr 172.21.15.45:0/2320704973 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:10.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f690b7fe640 1 --2- 172.21.15.45:0/2320704973 >> v2:172.21.15.63:3300/0 conn(0x7f690c0d20c0 0x7f690c163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:10.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f690affd640 1 -- 172.21.15.45:0/2320704973 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4039903883 0 0) 0x7f690c0d1300 con 0x7f690c071110 2024-05-08T05:45:10.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f690affd640 1 -- 172.21.15.45:0/2320704973 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68f4003520 con 0x7f690c071110 2024-05-08T05:45:10.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f690affd640 1 -- 172.21.15.45:0/2320704973 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 996223759 0 0) 0x7f68f4003520 con 0x7f690c071110 2024-05-08T05:45:10.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f690affd640 1 -- 172.21.15.45:0/2320704973 >> v2:172.21.15.63:3300/0 conn(0x7f690c0d20c0 msgr2=0x7f690c163920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:10.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f690affd640 1 --2- 172.21.15.45:0/2320704973 >> v2:172.21.15.63:3300/0 conn(0x7f690c0d20c0 0x7f690c163920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f690affd640 1 -- 172.21.15.45:0/2320704973 >> v1:172.21.15.63:6789/0 conn(0x7f690c070d30 legacy=0x7f690c0703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:10.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f690affd640 1 -- 172.21.15.45:0/2320704973 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690c0d18c0 con 0x7f690c071110 2024-05-08T05:45:10.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f690affd640 1 -- 172.21.15.45:0/2320704973 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f6904001a70 con 0x7f690c071110 2024-05-08T05:45:10.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f690affd640 1 -- 172.21.15.45:0/2320704973 >> v1:172.21.15.45:6789/0 conn(0x7f690c071110 legacy=0x7f690c06d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:10.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f690affd640 1 --2- 172.21.15.45:0/2320704973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68f4003a00 0x7f68f4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f690affd640 1 --2- 172.21.15.45:0/2320704973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68f4004800 0x7f68f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f690affd640 1 --2- 172.21.15.45:0/2320704973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f40051c0 0x7f68f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f690affd640 1 -- 172.21.15.45:0/2320704973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f68f400c030 con 0x7f68f40051c0 2024-05-08T05:45:10.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f690affd640 1 -- 172.21.15.45:0/2320704973 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68f400c1e0 con 0x7f68f4003a00 2024-05-08T05:45:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f690affd640 1 -- 172.21.15.45:0/2320704973 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f68f400c3e0 con 0x7f68f4004800 2024-05-08T05:45:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f6910b8c640 1 --2- 172.21.15.45:0/2320704973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f40051c0 0x7f68f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f690bfff640 1 --2- 172.21.15.45:0/2320704973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68f4003a00 0x7f68f4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f690b7fe640 1 --2- 172.21.15.45:0/2320704973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68f4004800 0x7f68f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f6910b8c640 1 -- 172.21.15.45:0/2320704973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68f4004800 msgr2=0x7f68f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f6910b8c640 1 --2- 172.21.15.45:0/2320704973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68f4004800 0x7f68f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f6910b8c640 1 -- 172.21.15.45:0/2320704973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68f4003a00 msgr2=0x7f68f4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.457+0000 7f6910b8c640 1 --2- 172.21.15.45:0/2320704973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68f4003a00 0x7f68f4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.458+0000 7f6910b8c640 1 -- 172.21.15.45:0/2320704973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f4003520 con 0x7f68f40051c0 2024-05-08T05:45:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.458+0000 7f690b7fe640 1 --2- 172.21.15.45:0/2320704973 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68f4004800 0x7f68f4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.458+0000 7f6910b8c640 1 -- 172.21.15.45:0/2320704973 reap_dead start 2024-05-08T05:45:10.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.458+0000 7f6910b8c640 1 --2- 172.21.15.45:0/2320704973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f40051c0 0x7f68f400b9c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f690400a8b0 tx=0x7f68f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f567ed381e72080c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.458+0000 7f690affd640 1 -- 172.21.15.45:0/2320704973 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690400a4d0 con 0x7f68f40051c0 2024-05-08T05:45:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.458+0000 7f690affd640 1 -- 172.21.15.45:0/2320704973 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f690402f070 con 0x7f68f40051c0 2024-05-08T05:45:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.458+0000 7f690affd640 1 -- 172.21.15.45:0/2320704973 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690402bc50 con 0x7f68f40051c0 2024-05-08T05:45:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.459+0000 7f69125ef640 1 -- 172.21.15.45:0/2320704973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f40051c0 msgr2=0x7f68f400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.459+0000 7f69125ef640 1 --2- 172.21.15.45:0/2320704973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f40051c0 0x7f68f400b9c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f690400a8b0 tx=0x7f68f4005830 comp rx=0 tx=0).stop 2024-05-08T05:45:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.459+0000 7f69125ef640 1 -- 172.21.15.45:0/2320704973 shutdown_connections 2024-05-08T05:45:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.459+0000 7f69125ef640 1 --2- 172.21.15.45:0/2320704973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f40051c0 0x7f68f400b9c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.459+0000 7f69125ef640 1 -- 172.21.15.45:0/2320704973 >> 172.21.15.45:0/2320704973 conn(0x7f690c0d2e90 msgr2=0x7f690c0d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.459+0000 7f69125ef640 1 -- 172.21.15.45:0/2320704973 shutdown_connections 2024-05-08T05:45:10.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.459+0000 7f69125ef640 1 -- 172.21.15.45:0/2320704973 wait complete. 2024-05-08T05:45:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.460+0000 7f69125ef640 1 Processor -- start 2024-05-08T05:45:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.460+0000 7f69125ef640 1 -- start start 2024-05-08T05:45:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.461+0000 7f69125ef640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f40051c0 0x7f690c157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.461+0000 7f69125ef640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f690c0b7a80 0x7f690c0b7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.461+0000 7f69125ef640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f690c0b8440 0x7f690c0b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.461+0000 7f69125ef640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f690c070b80 con 0x7f68f40051c0 2024-05-08T05:45:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.461+0000 7f69125ef640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f690c0d1c30 con 0x7f690c0b8440 2024-05-08T05:45:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.461+0000 7f69125ef640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f690c163920 con 0x7f690c0b7a80 2024-05-08T05:45:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.461+0000 7f690bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f40051c0 0x7f690c157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:10.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.461+0000 7f690bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f40051c0 0x7f690c157470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35546/0 (socket says 172.21.15.45:35546) 2024-05-08T05:45:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.461+0000 7f690bfff640 1 -- 172.21.15.45:0/2812449717 learned_addr learned my addr 172.21.15.45:0/2812449717 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.462+0000 7f690bfff640 1 -- 172.21.15.45:0/2812449717 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f690c0b7a80 msgr2=0x7f690c0b7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.462+0000 7f690bfff640 1 --2- 172.21.15.45:0/2812449717 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f690c0b7a80 0x7f690c0b7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.462+0000 7f690bfff640 1 -- 172.21.15.45:0/2812449717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f690c0b8440 msgr2=0x7f690c0b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.462+0000 7f690bfff640 1 --2- 172.21.15.45:0/2812449717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f690c0b8440 0x7f690c0b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.462+0000 7f690bfff640 1 -- 172.21.15.45:0/2812449717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690c0b88c0 con 0x7f68f40051c0 2024-05-08T05:45:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.462+0000 7f690bfff640 1 --2- 172.21.15.45:0/2812449717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f40051c0 0x7f690c157470 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00f850 tx=0x7f68fc00fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.462+0000 7f6908ff9640 1 -- 172.21.15.45:0/2812449717 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc00fe40 con 0x7f68f40051c0 2024-05-08T05:45:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.462+0000 7f6908ff9640 1 -- 172.21.15.45:0/2812449717 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68fc018070 con 0x7f68f40051c0 2024-05-08T05:45:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.462+0000 7f69125ef640 1 -- 172.21.15.45:0/2812449717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690c15c880 con 0x7f68f40051c0 2024-05-08T05:45:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.462+0000 7f6908ff9640 1 -- 172.21.15.45:0/2812449717 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc00d6d0 con 0x7f68f40051c0 2024-05-08T05:45:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.462+0000 7f69125ef640 1 -- 172.21.15.45:0/2812449717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f690c15ce40 con 0x7f68f40051c0 2024-05-08T05:45:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.463+0000 7f69125ef640 1 -- 172.21.15.45:0/2812449717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f690c0693f0 con 0x7f68f40051c0 2024-05-08T05:45:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.464+0000 7f6908ff9640 1 -- 172.21.15.45:0/2812449717 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f68fc00d870 con 0x7f68f40051c0 2024-05-08T05:45:10.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.464+0000 7f6908ff9640 1 --2- 172.21.15.45:0/2812449717 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f68f80762d0 0x7f68f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.464+0000 7f6908ff9640 1 -- 172.21.15.45:0/2812449717 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f68fc097670 con 0x7f68f40051c0 2024-05-08T05:45:10.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.464+0000 7f690b7fe640 1 --2- 172.21.15.45:0/2812449717 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f68f80762d0 0x7f68f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:10.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.465+0000 7f690b7fe640 1 --2- 172.21.15.45:0/2812449717 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f68f80762d0 0x7f68f8078790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f690c1f8730 tx=0x7f6900008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:10.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.468+0000 7f6908ff9640 1 -- 172.21.15.45:0/2812449717 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f68fc060db0 con 0x7f68f40051c0 2024-05-08T05:45:10.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.702+0000 7f69125ef640 1 -- 172.21.15.45:0/2812449717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f690c0d3d70 con 0x7f68f40051c0 2024-05-08T05:45:10.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.703+0000 7f6908ff9640 1 -- 172.21.15.45:0/2812449717 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5180 (secure 0 0 0) 0x7f68fc0606b0 con 0x7f68f40051c0 2024-05-08T05:45:10.704 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:10.704 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/162765657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":162765657},{"type":"v1","addr":"172.21.15.63:6836","nonce":162765657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5356,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1788513331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1788513331},{"type":"v1","addr":"172.21.15.45:6834","nonce":1788513331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1173167340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1173167340},{"type":"v1","addr":"172.21.15.63:6837","nonce":1173167340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:43:58.107819+0000","modified":"2024-05-08T05:44:38.767928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":13,"addr":"172.21.15.45:6835/1461822016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1461822016},{"type":"v1","addr":"172.21.15.45:6835","nonce":1461822016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5697,"qdb_cluster":[5697]},"id":4}]} 2024-05-08T05:45:10.704 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 36 2024-05-08T05:45:10.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.705+0000 7f69125ef640 1 -- 172.21.15.45:0/2812449717 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f68f80762d0 msgr2=0x7f68f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:10.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.705+0000 7f69125ef640 1 --2- 172.21.15.45:0/2812449717 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f68f80762d0 0x7f68f8078790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f690c1f8730 tx=0x7f6900008040 comp rx=0 tx=0).stop 2024-05-08T05:45:10.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.705+0000 7f69125ef640 1 -- 172.21.15.45:0/2812449717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f40051c0 msgr2=0x7f690c157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:10.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.705+0000 7f69125ef640 1 --2- 172.21.15.45:0/2812449717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f40051c0 0x7f690c157470 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00f850 tx=0x7f68fc00fd10 comp rx=0 tx=0).stop 2024-05-08T05:45:10.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.705+0000 7f69125ef640 1 -- 172.21.15.45:0/2812449717 shutdown_connections 2024-05-08T05:45:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.705+0000 7f69125ef640 1 --2- 172.21.15.45:0/2812449717 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f68f80762d0 0x7f68f8078790 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.705+0000 7f69125ef640 1 --2- 172.21.15.45:0/2812449717 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f690c0b7a80 0x7f690c0b7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.705+0000 7f69125ef640 1 --2- 172.21.15.45:0/2812449717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f690c0b8440 0x7f690c0b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.705+0000 7f69125ef640 1 --2- 172.21.15.45:0/2812449717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f68f40051c0 0x7f690c157470 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.705+0000 7f69125ef640 1 -- 172.21.15.45:0/2812449717 >> 172.21.15.45:0/2812449717 conn(0x7f690c0d2e90 msgr2=0x7f690c1609f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.706+0000 7f69125ef640 1 -- 172.21.15.45:0/2812449717 shutdown_connections 2024-05-08T05:45:10.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.706+0000 7f69125ef640 1 -- 172.21.15.45:0/2812449717 wait complete. 2024-05-08T05:45:10.717 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:45:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.823+0000 7fcf4179e640 1 Processor -- start 2024-05-08T05:45:10.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.823+0000 7fcf4179e640 1 -- start start 2024-05-08T05:45:10.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.823+0000 7fcf4179e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcf3c15f380 0x7fcf3c0d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.823+0000 7fcf4179e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcf3c0d0cf0 0x7fcf3c0d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.823+0000 7fcf4179e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf3c0d3b00 0x7fcf3c162e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.823+0000 7fcf4179e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fcf3c0d30f0 con 0x7fcf3c15f380 2024-05-08T05:45:10.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.823+0000 7fcf4179e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fcf3c0d2370 con 0x7fcf3c0d0cf0 2024-05-08T05:45:10.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.823+0000 7fcf4179e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcf3c0d1760 con 0x7fcf3c0d3b00 2024-05-08T05:45:10.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.824+0000 7fcf3affd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcf3c15f380 0x7fcf3c0d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:10.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.824+0000 7fcf3affd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcf3c15f380 0x7fcf3c0d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52514/0 (socket says 172.21.15.45:52514) 2024-05-08T05:45:10.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.824+0000 7fcf3affd640 1 -- 172.21.15.45:0/3039326487 learned_addr learned my addr 172.21.15.45:0/3039326487 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:10.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.824+0000 7fcf3a7fc640 1 --2- 172.21.15.45:0/3039326487 >> v2:172.21.15.63:3300/0 conn(0x7fcf3c0d0cf0 0x7fcf3c0d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:10.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.824+0000 7fcf3b7fe640 1 --2- 172.21.15.45:0/3039326487 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf3c0d3b00 0x7fcf3c162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:10.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.824+0000 7fcf3affd640 1 -- 172.21.15.45:0/3039326487 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf3c0d3b00 msgr2=0x7fcf3c162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:10.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.824+0000 7fcf3affd640 1 --2- 172.21.15.45:0/3039326487 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf3c0d3b00 0x7fcf3c162e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.824+0000 7fcf3affd640 1 -- 172.21.15.45:0/3039326487 >> v2:172.21.15.63:3300/0 conn(0x7fcf3c0d0cf0 msgr2=0x7fcf3c0d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:10.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.824+0000 7fcf3affd640 1 --2- 172.21.15.45:0/3039326487 >> v2:172.21.15.63:3300/0 conn(0x7fcf3c0d0cf0 0x7fcf3c0d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.824+0000 7fcf3affd640 1 -- 172.21.15.45:0/3039326487 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3c0d1a40 con 0x7fcf3c15f380 2024-05-08T05:45:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.825+0000 7fcf3affd640 1 --2- 172.21.15.45:0/3039326487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf3c15f380 0x7fcf3c0d07b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0b7b00 tx=0x7fcf2c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f797f34254e4a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.825+0000 7fcf39ffb640 1 -- 172.21.15.45:0/3039326487 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c037070 con 0x7fcf3c15f380 2024-05-08T05:45:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.825+0000 7fcf39ffb640 1 -- 172.21.15.45:0/3039326487 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf2c032440 con 0x7fcf3c15f380 2024-05-08T05:45:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.825+0000 7fcf39ffb640 1 -- 172.21.15.45:0/3039326487 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c032810 con 0x7fcf3c15f380 2024-05-08T05:45:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.825+0000 7fcf4179e640 1 -- 172.21.15.45:0/3039326487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf3c15f380 msgr2=0x7fcf3c0d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.825+0000 7fcf4179e640 1 --2- 172.21.15.45:0/3039326487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf3c15f380 0x7fcf3c0d07b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0b7b00 tx=0x7fcf2c02a1e0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.825+0000 7fcf4179e640 1 -- 172.21.15.45:0/3039326487 shutdown_connections 2024-05-08T05:45:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.825+0000 7fcf4179e640 1 --2- 172.21.15.45:0/3039326487 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf3c0d3b00 0x7fcf3c162e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.825+0000 7fcf4179e640 1 --2- 172.21.15.45:0/3039326487 >> v2:172.21.15.63:3300/0 conn(0x7fcf3c0d0cf0 0x7fcf3c0d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.825+0000 7fcf4179e640 1 --2- 172.21.15.45:0/3039326487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf3c15f380 0x7fcf3c0d07b0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.825+0000 7fcf4179e640 1 -- 172.21.15.45:0/3039326487 >> 172.21.15.45:0/3039326487 conn(0x7fcf3c0d4fe0 msgr2=0x7fcf3c0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.825+0000 7fcf4179e640 1 -- 172.21.15.45:0/3039326487 shutdown_connections 2024-05-08T05:45:10.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.826+0000 7fcf4179e640 1 -- 172.21.15.45:0/3039326487 wait complete. 2024-05-08T05:45:10.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.827+0000 7fcf4179e640 1 Processor -- start 2024-05-08T05:45:10.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.827+0000 7fcf4179e640 1 -- start start 2024-05-08T05:45:10.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf4179e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf3c0d0cf0 0x7fcf3c0a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf4179e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf3c0d3b00 0x7fcf3c0a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf4179e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf3c15f380 0x7fcf3c160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf4179e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c15f760 con 0x7fcf3c15f380 2024-05-08T05:45:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf4179e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c0d2000 con 0x7fcf3c0d0cf0 2024-05-08T05:45:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf4179e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcf3c0d2800 con 0x7fcf3c0d3b00 2024-05-08T05:45:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf3b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf3c15f380 0x7fcf3c160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf3b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf3c15f380 0x7fcf3c160a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52530/0 (socket says 172.21.15.45:52530) 2024-05-08T05:45:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf3b7fe640 1 -- 172.21.15.45:0/330443887 learned_addr learned my addr 172.21.15.45:0/330443887 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf3affd640 1 --2- 172.21.15.45:0/330443887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf3c0d0cf0 0x7fcf3c0a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf3a7fc640 1 --2- 172.21.15.45:0/330443887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf3c0d3b00 0x7fcf3c0a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf3b7fe640 1 -- 172.21.15.45:0/330443887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf3c0d3b00 msgr2=0x7fcf3c0a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf3b7fe640 1 --2- 172.21.15.45:0/330443887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf3c0d3b00 0x7fcf3c0a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf3b7fe640 1 -- 172.21.15.45:0/330443887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf3c0d0cf0 msgr2=0x7fcf3c0a2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf3b7fe640 1 --2- 172.21.15.45:0/330443887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf3c0d0cf0 0x7fcf3c0a2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf3b7fe640 1 -- 172.21.15.45:0/330443887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3c0a05b0 con 0x7fcf3c15f380 2024-05-08T05:45:10.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf3b7fe640 1 --2- 172.21.15.45:0/330443887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf3c15f380 0x7fcf3c160a20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fcf34007e40 tx=0x7fcf3400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf337fe640 1 -- 172.21.15.45:0/330443887 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34014620 con 0x7fcf3c15f380 2024-05-08T05:45:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf337fe640 1 -- 172.21.15.45:0/330443887 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf34019070 con 0x7fcf3c15f380 2024-05-08T05:45:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf337fe640 1 -- 172.21.15.45:0/330443887 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34014a20 con 0x7fcf3c15f380 2024-05-08T05:45:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.828+0000 7fcf4179e640 1 -- 172.21.15.45:0/330443887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf3c0a0890 con 0x7fcf3c15f380 2024-05-08T05:45:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.829+0000 7fcf4179e640 1 -- 172.21.15.45:0/330443887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf3c0a0e50 con 0x7fcf3c15f380 2024-05-08T05:45:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.829+0000 7fcf4179e640 1 -- 172.21.15.45:0/330443887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf3c0693f0 con 0x7fcf3c15f380 2024-05-08T05:45:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.830+0000 7fcf337fe640 1 -- 172.21.15.45:0/330443887 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcf34014bc0 con 0x7fcf3c15f380 2024-05-08T05:45:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.831+0000 7fcf337fe640 1 --2- 172.21.15.45:0/330443887 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcf1c0762d0 0x7fcf1c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.831+0000 7fcf337fe640 1 -- 172.21.15.45:0/330443887 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fcf34098130 con 0x7fcf3c15f380 2024-05-08T05:45:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.831+0000 7fcf3affd640 1 --2- 172.21.15.45:0/330443887 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcf1c0762d0 0x7fcf1c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:10.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.832+0000 7fcf3affd640 1 --2- 172.21.15.45:0/330443887 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcf1c0762d0 0x7fcf1c078790 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c034000 tx=0x7fcf2c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:10.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:10.835+0000 7fcf337fe640 1 -- 172.21.15.45:0/330443887 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcf34061880 con 0x7fcf3c15f380 2024-05-08T05:45:11.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.027+0000 7fcf4179e640 1 -- 172.21.15.45:0/330443887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf3c08cfd0 con 0x7fcf3c15f380 2024-05-08T05:45:11.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.028+0000 7fcf337fe640 1 -- 172.21.15.45:0/330443887 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+17076 (secure 0 0 0) 0x7fcf34061180 con 0x7fcf3c15f380 2024-05-08T05:45:11.028 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:11.029 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":63,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:44:14.980856+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:43:55.774105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-08T05:43:57.070302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.13","mappings":[{"from":7,"to":2}]},{"pgid":"10.d","mappings":[{"from":4,"to":3}]},{"pgid":"10.13","mappings":[{"from":7,"to":5}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:45:11.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.030+0000 7fcf4179e640 1 -- 172.21.15.45:0/330443887 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcf1c0762d0 msgr2=0x7fcf1c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:11.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.030+0000 7fcf4179e640 1 --2- 172.21.15.45:0/330443887 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcf1c0762d0 0x7fcf1c078790 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c034000 tx=0x7fcf2c02e000 comp rx=0 tx=0).stop 2024-05-08T05:45:11.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.030+0000 7fcf4179e640 1 -- 172.21.15.45:0/330443887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf3c15f380 msgr2=0x7fcf3c160a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:11.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.030+0000 7fcf4179e640 1 --2- 172.21.15.45:0/330443887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf3c15f380 0x7fcf3c160a20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fcf34007e40 tx=0x7fcf3400e4b0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.030+0000 7fcf4179e640 1 -- 172.21.15.45:0/330443887 shutdown_connections 2024-05-08T05:45:11.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.030+0000 7fcf4179e640 1 --2- 172.21.15.45:0/330443887 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcf1c0762d0 0x7fcf1c078790 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.030+0000 7fcf4179e640 1 --2- 172.21.15.45:0/330443887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcf3c0d3b00 0x7fcf3c0a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.030+0000 7fcf4179e640 1 --2- 172.21.15.45:0/330443887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf3c15f380 0x7fcf3c160a20 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.030+0000 7fcf4179e640 1 --2- 172.21.15.45:0/330443887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf3c0d0cf0 0x7fcf3c0a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.030+0000 7fcf4179e640 1 -- 172.21.15.45:0/330443887 >> 172.21.15.45:0/330443887 conn(0x7fcf3c0d4fe0 msgr2=0x7fcf3c0b97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:11.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.030+0000 7fcf4179e640 1 -- 172.21.15.45:0/330443887 shutdown_connections 2024-05-08T05:45:11.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.030+0000 7fcf4179e640 1 -- 172.21.15.45:0/330443887 wait complete. 2024-05-08T05:45:11.041 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T05:45:11.042 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:45:11.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.146+0000 7f24fa22a640 1 Processor -- start 2024-05-08T05:45:11.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.146+0000 7f24fa22a640 1 -- start start 2024-05-08T05:45:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.146+0000 7f24fa22a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f24f40d1970 0x7f24f40d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.146+0000 7f24fa22a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f24f40d4cc0 0x7f24f40d5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.146+0000 7f24fa22a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24f41630b0 con 0x7f24f415f420 2024-05-08T05:45:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.147+0000 7f24fa22a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f24f40d3e70 con 0x7f24f40d1970 2024-05-08T05:45:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.147+0000 7f24fa22a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f24f40d5670 con 0x7f24f40d4cc0 2024-05-08T05:45:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.147+0000 7f24f2ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f24f40d1970 0x7f24f40d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.147+0000 7f24f37fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f24f415f420 0x7f24f40d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57572/0 (socket says 172.21.15.45:57572) 2024-05-08T05:45:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.147+0000 7f24f37fe640 1 -- 172.21.15.45:0/320358534 learned_addr learned my addr 172.21.15.45:0/320358534 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.147+0000 7f24f3fff640 1 --2- 172.21.15.45:0/320358534 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f24f40d4cc0 0x7f24f40d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:11.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.147+0000 7f24f2ffd640 1 -- 172.21.15.45:0/320358534 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f24f40d4cc0 msgr2=0x7f24f40d5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.147+0000 7f24f2ffd640 1 --2- 172.21.15.45:0/320358534 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f24f40d4cc0 0x7f24f40d5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.147+0000 7f24f2ffd640 1 -- 172.21.15.45:0/320358534 >> v1:172.21.15.45:6789/0 conn(0x7f24f415f420 legacy=0x7f24f40d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.147+0000 7f24f2ffd640 1 -- 172.21.15.45:0/320358534 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f40d2550 con 0x7f24f40d1970 2024-05-08T05:45:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.147+0000 7f24f27fc640 1 -- 172.21.15.45:0/320358534 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3966055726 0 0) 0x7f24e4001110 con 0x7f24f415f420 2024-05-08T05:45:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.148+0000 7f24f2ffd640 1 --2- 172.21.15.45:0/320358534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24f40d1970 0x7f24f40d1e20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f24e80097d0 tx=0x7f24e802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c222d9dbf397898 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.148+0000 7f24f27fc640 1 -- 172.21.15.45:0/320358534 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8037070 con 0x7f24f40d1970 2024-05-08T05:45:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.148+0000 7f24f27fc640 1 -- 172.21.15.45:0/320358534 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e8032440 con 0x7f24f40d1970 2024-05-08T05:45:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.148+0000 7f24f27fc640 1 -- 172.21.15.45:0/320358534 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8032810 con 0x7f24f40d1970 2024-05-08T05:45:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.148+0000 7f24fa22a640 1 -- 172.21.15.45:0/320358534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24f40d1970 msgr2=0x7f24f40d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.148+0000 7f24fa22a640 1 --2- 172.21.15.45:0/320358534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24f40d1970 0x7f24f40d1e20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f24e80097d0 tx=0x7f24e802a1c0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.148+0000 7f24fa22a640 1 -- 172.21.15.45:0/320358534 shutdown_connections 2024-05-08T05:45:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.148+0000 7f24fa22a640 1 --2- 172.21.15.45:0/320358534 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f24f40d4cc0 0x7f24f40d5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.149+0000 7f24fa22a640 1 --2- 172.21.15.45:0/320358534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24f40d1970 0x7f24f40d1e20 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.149+0000 7f24fa22a640 1 -- 172.21.15.45:0/320358534 >> 172.21.15.45:0/320358534 conn(0x7f24f408eac0 msgr2=0x7f24f409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:11.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.149+0000 7f24fa22a640 1 -- 172.21.15.45:0/320358534 shutdown_connections 2024-05-08T05:45:11.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.149+0000 7f24fa22a640 1 -- 172.21.15.45:0/320358534 wait complete. 2024-05-08T05:45:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.150+0000 7f24fa22a640 1 Processor -- start 2024-05-08T05:45:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.150+0000 7f24fa22a640 1 -- start start 2024-05-08T05:45:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.151+0000 7f24fa22a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24f40d1970 0x7f24f40ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.151+0000 7f24fa22a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24f40d4cc0 0x7f24f4161cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.151+0000 7f24fa22a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f24f414b850 0x7f24f41621f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.151+0000 7f24fa22a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f24f40d3240 con 0x7f24f40d4cc0 2024-05-08T05:45:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.151+0000 7f24fa22a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f24f40d2f60 con 0x7f24f40d1970 2024-05-08T05:45:11.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.151+0000 7f24fa22a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f24f40d2830 con 0x7f24f414b850 2024-05-08T05:45:11.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.151+0000 7f24f2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24f40d4cc0 0x7f24f4161cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:11.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.151+0000 7f24f2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24f40d4cc0 0x7f24f4161cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52546/0 (socket says 172.21.15.45:52546) 2024-05-08T05:45:11.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.151+0000 7f24f2ffd640 1 -- 172.21.15.45:0/3795390823 learned_addr learned my addr 172.21.15.45:0/3795390823 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:11.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.151+0000 7f24f37fe640 1 --2- 172.21.15.45:0/3795390823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24f40d1970 0x7f24f40ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:11.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.151+0000 7f24f3fff640 1 --2- 172.21.15.45:0/3795390823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f24f414b850 0x7f24f41621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:11.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.151+0000 7f24f2ffd640 1 -- 172.21.15.45:0/3795390823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f24f414b850 msgr2=0x7f24f41621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:11.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.151+0000 7f24f2ffd640 1 --2- 172.21.15.45:0/3795390823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f24f414b850 0x7f24f41621f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.151+0000 7f24f2ffd640 1 -- 172.21.15.45:0/3795390823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24f40d1970 msgr2=0x7f24f40ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:11.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.151+0000 7f24f2ffd640 1 --2- 172.21.15.45:0/3795390823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24f40d1970 0x7f24f40ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.151+0000 7f24f2ffd640 1 -- 172.21.15.45:0/3795390823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f40cee60 con 0x7f24f40d4cc0 2024-05-08T05:45:11.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.151+0000 7f24f37fe640 1 --2- 172.21.15.45:0/3795390823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24f40d1970 0x7f24f40ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:11.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.151+0000 7f24f2ffd640 1 --2- 172.21.15.45:0/3795390823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24f40d4cc0 0x7f24f4161cb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f24e8035040 tx=0x7f24e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:11.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.151+0000 7f24dbfff640 1 -- 172.21.15.45:0/3795390823 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8037050 con 0x7f24f40d4cc0 2024-05-08T05:45:11.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.152+0000 7f24dbfff640 1 -- 172.21.15.45:0/3795390823 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e8038040 con 0x7f24f40d4cc0 2024-05-08T05:45:11.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.152+0000 7f24fa22a640 1 -- 172.21.15.45:0/3795390823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f40ca010 con 0x7f24f40d4cc0 2024-05-08T05:45:11.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.152+0000 7f24dbfff640 1 -- 172.21.15.45:0/3795390823 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8004470 con 0x7f24f40d4cc0 2024-05-08T05:45:11.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.152+0000 7f24fa22a640 1 -- 172.21.15.45:0/3795390823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f40ca5d0 con 0x7f24f40d4cc0 2024-05-08T05:45:11.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.152+0000 7f24fa22a640 1 -- 172.21.15.45:0/3795390823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24f40693f0 con 0x7f24f40d4cc0 2024-05-08T05:45:11.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.153+0000 7f24dbfff640 1 -- 172.21.15.45:0/3795390823 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f24e8030900 con 0x7f24f40d4cc0 2024-05-08T05:45:11.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.154+0000 7f24dbfff640 1 --2- 172.21.15.45:0/3795390823 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f24e0076300 0x7f24e00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:11.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.154+0000 7f24dbfff640 1 -- 172.21.15.45:0/3795390823 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f24e80b82d0 con 0x7f24f40d4cc0 2024-05-08T05:45:11.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.154+0000 7f24f37fe640 1 --2- 172.21.15.45:0/3795390823 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f24e0076300 0x7f24e00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:11.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.154+0000 7f24f37fe640 1 --2- 172.21.15.45:0/3795390823 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f24e0076300 0x7f24e00787c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f24e4000f60 tx=0x7f24e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:11.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.158+0000 7f24dbfff640 1 -- 172.21.15.45:0/3795390823 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f24e8081990 con 0x7f24f40d4cc0 2024-05-08T05:45:11.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.386+0000 7f24fa22a640 1 -- 172.21.15.45:0/3795390823 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24f40d03f0 con 0x7f24f40d4cc0 2024-05-08T05:45:11.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.387+0000 7f24dbfff640 1 -- 172.21.15.45:0/3795390823 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5180 (secure 0 0 0) 0x7f24e8081290 con 0x7f24f40d4cc0 2024-05-08T05:45:11.388 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:11.388 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/162765657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":162765657},{"type":"v1","addr":"172.21.15.63:6836","nonce":162765657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5356,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1788513331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1788513331},{"type":"v1","addr":"172.21.15.45:6834","nonce":1788513331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1173167340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1173167340},{"type":"v1","addr":"172.21.15.63:6837","nonce":1173167340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:43:58.107819+0000","modified":"2024-05-08T05:44:38.767928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":13,"addr":"172.21.15.45:6835/1461822016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1461822016},{"type":"v1","addr":"172.21.15.45:6835","nonce":1461822016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5697,"qdb_cluster":[5697]},"id":4}]} 2024-05-08T05:45:11.388 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 36 2024-05-08T05:45:11.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.389+0000 7f24fa22a640 1 -- 172.21.15.45:0/3795390823 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f24e0076300 msgr2=0x7f24e00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:11.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.389+0000 7f24fa22a640 1 --2- 172.21.15.45:0/3795390823 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f24e0076300 0x7f24e00787c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f24e4000f60 tx=0x7f24e4009040 comp rx=0 tx=0).stop 2024-05-08T05:45:11.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.389+0000 7f24fa22a640 1 -- 172.21.15.45:0/3795390823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24f40d4cc0 msgr2=0x7f24f4161cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:11.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.389+0000 7f24fa22a640 1 --2- 172.21.15.45:0/3795390823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24f40d4cc0 0x7f24f4161cb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f24e8035040 tx=0x7f24e80046c0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.389+0000 7f24fa22a640 1 -- 172.21.15.45:0/3795390823 shutdown_connections 2024-05-08T05:45:11.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.389+0000 7f24fa22a640 1 --2- 172.21.15.45:0/3795390823 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f24e0076300 0x7f24e00787c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.389+0000 7f24fa22a640 1 --2- 172.21.15.45:0/3795390823 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f24f414b850 0x7f24f41621f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.389+0000 7f24fa22a640 1 --2- 172.21.15.45:0/3795390823 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f24f40d4cc0 0x7f24f4161cb0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.389+0000 7f24fa22a640 1 --2- 172.21.15.45:0/3795390823 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f24f40d1970 0x7f24f40ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.389+0000 7f24fa22a640 1 -- 172.21.15.45:0/3795390823 >> 172.21.15.45:0/3795390823 conn(0x7f24f408eac0 msgr2=0x7f24f4163290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:11.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.389+0000 7f24fa22a640 1 -- 172.21.15.45:0/3795390823 shutdown_connections 2024-05-08T05:45:11.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.389+0000 7f24fa22a640 1 -- 172.21.15.45:0/3795390823 wait complete. 2024-05-08T05:45:11.401 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:45:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.504+0000 7fb862387640 1 Processor -- start 2024-05-08T05:45:11.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.504+0000 7fb862387640 1 -- start start 2024-05-08T05:45:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.505+0000 7fb862387640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb85c06f540 0x7fb85c070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.505+0000 7fb862387640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb85c071020 0x7fb85c0732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.505+0000 7fb862387640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb85c06fa40 con 0x7fb85c15bc50 2024-05-08T05:45:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.505+0000 7fb862387640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb85c06fd20 con 0x7fb85c06f540 2024-05-08T05:45:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.505+0000 7fb862387640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb85c070000 con 0x7fb85c071020 2024-05-08T05:45:11.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.505+0000 7fb85b7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb85c06f540 0x7fb85c070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.505+0000 7fb85bfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb85c15bc50 0x7fb85c0b5ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57580/0 (socket says 172.21.15.45:57580) 2024-05-08T05:45:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.505+0000 7fb85bfff640 1 -- 172.21.15.45:0/9876185 learned_addr learned my addr 172.21.15.45:0/9876185 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.506+0000 7fb860924640 1 --2- 172.21.15.45:0/9876185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb85c071020 0x7fb85c0732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.506+0000 7fb85b7fe640 1 -- 172.21.15.45:0/9876185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb85c071020 msgr2=0x7fb85c0732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.506+0000 7fb85b7fe640 1 --2- 172.21.15.45:0/9876185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb85c071020 0x7fb85c0732c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.506+0000 7fb85b7fe640 1 -- 172.21.15.45:0/9876185 >> v1:172.21.15.45:6789/0 conn(0x7fb85c15bc50 legacy=0x7fb85c0b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.506+0000 7fb85b7fe640 1 -- 172.21.15.45:0/9876185 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb85c0702e0 con 0x7fb85c06f540 2024-05-08T05:45:11.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.506+0000 7fb85affd640 1 -- 172.21.15.45:0/9876185 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2881902829 0 0) 0x7fb85c06fa40 con 0x7fb85c15bc50 2024-05-08T05:45:11.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.506+0000 7fb85b7fe640 1 --2- 172.21.15.45:0/9876185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb85c06f540 0x7fb85c070ae0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb8500097a0 tx=0x7fb85002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead4a7a3b344e2c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.507+0000 7fb85affd640 1 -- 172.21.15.45:0/9876185 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850037070 con 0x7fb85c06f540 2024-05-08T05:45:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.507+0000 7fb85affd640 1 -- 172.21.15.45:0/9876185 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb850032440 con 0x7fb85c06f540 2024-05-08T05:45:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.507+0000 7fb85affd640 1 -- 172.21.15.45:0/9876185 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850032810 con 0x7fb85c06f540 2024-05-08T05:45:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.507+0000 7fb862387640 1 -- 172.21.15.45:0/9876185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb85c06f540 msgr2=0x7fb85c070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.507+0000 7fb862387640 1 --2- 172.21.15.45:0/9876185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb85c06f540 0x7fb85c070ae0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb8500097a0 tx=0x7fb85002a180 comp rx=0 tx=0).stop 2024-05-08T05:45:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.507+0000 7fb862387640 1 -- 172.21.15.45:0/9876185 shutdown_connections 2024-05-08T05:45:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.507+0000 7fb862387640 1 --2- 172.21.15.45:0/9876185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb85c071020 0x7fb85c0732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.507+0000 7fb862387640 1 --2- 172.21.15.45:0/9876185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb85c06f540 0x7fb85c070ae0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.507+0000 7fb862387640 1 -- 172.21.15.45:0/9876185 >> 172.21.15.45:0/9876185 conn(0x7fb85c0b3c40 msgr2=0x7fb85c0b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.507+0000 7fb862387640 1 -- 172.21.15.45:0/9876185 shutdown_connections 2024-05-08T05:45:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.507+0000 7fb862387640 1 -- 172.21.15.45:0/9876185 wait complete. 2024-05-08T05:45:11.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.508+0000 7fb862387640 1 Processor -- start 2024-05-08T05:45:11.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.509+0000 7fb862387640 1 -- start start 2024-05-08T05:45:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.509+0000 7fb862387640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb85c06f540 0x7fb85c15ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:11.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.509+0000 7fb862387640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb85c15bc50 0x7fb85c151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.509+0000 7fb862387640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb85c152000 0x7fb85c0d2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.509+0000 7fb862387640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c15c1b0 con 0x7fb85c06f540 2024-05-08T05:45:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.509+0000 7fb862387640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c15c030 con 0x7fb85c152000 2024-05-08T05:45:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.509+0000 7fb862387640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb85c073800 con 0x7fb85c15bc50 2024-05-08T05:45:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.509+0000 7fb85bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb85c06f540 0x7fb85c15ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.509+0000 7fb85bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb85c06f540 0x7fb85c15ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52576/0 (socket says 172.21.15.45:52576) 2024-05-08T05:45:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.509+0000 7fb85bfff640 1 -- 172.21.15.45:0/843809759 learned_addr learned my addr 172.21.15.45:0/843809759 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.510+0000 7fb860924640 1 --2- 172.21.15.45:0/843809759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb85c152000 0x7fb85c0d2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:11.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.510+0000 7fb85b7fe640 1 --2- 172.21.15.45:0/843809759 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb85c15bc50 0x7fb85c151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.510+0000 7fb85bfff640 1 -- 172.21.15.45:0/843809759 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb85c15bc50 msgr2=0x7fb85c151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.510+0000 7fb85bfff640 1 --2- 172.21.15.45:0/843809759 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb85c15bc50 0x7fb85c151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.510+0000 7fb85bfff640 1 -- 172.21.15.45:0/843809759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb85c152000 msgr2=0x7fb85c0d2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.510+0000 7fb85bfff640 1 --2- 172.21.15.45:0/843809759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb85c152000 0x7fb85c0d2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.510+0000 7fb85bfff640 1 -- 172.21.15.45:0/843809759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb85c1e7220 con 0x7fb85c06f540 2024-05-08T05:45:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.510+0000 7fb860924640 1 --2- 172.21.15.45:0/843809759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb85c152000 0x7fb85c0d2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.510+0000 7fb85bfff640 1 --2- 172.21.15.45:0/843809759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb85c06f540 0x7fb85c15ec30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00c3e0 tx=0x7fb84c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.510+0000 7fb858ff9640 1 -- 172.21.15.45:0/843809759 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c0074e0 con 0x7fb85c06f540 2024-05-08T05:45:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.510+0000 7fb858ff9640 1 -- 172.21.15.45:0/843809759 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb84c0047e0 con 0x7fb85c06f540 2024-05-08T05:45:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.510+0000 7fb858ff9640 1 -- 172.21.15.45:0/843809759 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84c004bd0 con 0x7fb85c06f540 2024-05-08T05:45:11.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.510+0000 7fb862387640 1 -- 172.21.15.45:0/843809759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb85c1e6c60 con 0x7fb85c06f540 2024-05-08T05:45:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.510+0000 7fb862387640 1 -- 172.21.15.45:0/843809759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb85c1e4c70 con 0x7fb85c06f540 2024-05-08T05:45:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.511+0000 7fb862387640 1 -- 172.21.15.45:0/843809759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb85c069b00 con 0x7fb85c06f540 2024-05-08T05:45:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.512+0000 7fb858ff9640 1 -- 172.21.15.45:0/843809759 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb84c011070 con 0x7fb85c06f540 2024-05-08T05:45:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.512+0000 7fb858ff9640 1 --2- 172.21.15.45:0/843809759 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb848076300 0x7fb8480787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.512+0000 7fb858ff9640 1 -- 172.21.15.45:0/843809759 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7fb84c097340 con 0x7fb85c06f540 2024-05-08T05:45:11.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.513+0000 7fb85b7fe640 1 --2- 172.21.15.45:0/843809759 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb848076300 0x7fb8480787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:11.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.513+0000 7fb85b7fe640 1 --2- 172.21.15.45:0/843809759 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb848076300 0x7fb8480787c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb850034000 tx=0x7fb85002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:11.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.517+0000 7fb858ff9640 1 -- 172.21.15.45:0/843809759 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb84c060b10 con 0x7fb85c06f540 2024-05-08T05:45:11.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.745+0000 7fb862387640 1 -- 172.21.15.45:0/843809759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb85c074690 con 0x7fb85c06f540 2024-05-08T05:45:11.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.745+0000 7fb858ff9640 1 -- 172.21.15.45:0/843809759 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5180 (secure 0 0 0) 0x7fb84c060410 con 0x7fb85c06f540 2024-05-08T05:45:11.746 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:11.746 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/162765657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":162765657},{"type":"v1","addr":"172.21.15.63:6836","nonce":162765657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5356,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1788513331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1788513331},{"type":"v1","addr":"172.21.15.45:6834","nonce":1788513331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1173167340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1173167340},{"type":"v1","addr":"172.21.15.63:6837","nonce":1173167340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:43:58.107819+0000","modified":"2024-05-08T05:44:38.767928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5697},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5697":{"gid":5697,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":13,"addr":"172.21.15.45:6835/1461822016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1461822016},{"type":"v1","addr":"172.21.15.45:6835","nonce":1461822016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5697,"qdb_cluster":[5697]},"id":4}]} 2024-05-08T05:45:11.746 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 36 2024-05-08T05:45:11.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.747+0000 7fb862387640 1 -- 172.21.15.45:0/843809759 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb848076300 msgr2=0x7fb8480787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:11.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.747+0000 7fb862387640 1 --2- 172.21.15.45:0/843809759 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb848076300 0x7fb8480787c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb850034000 tx=0x7fb85002e000 comp rx=0 tx=0).stop 2024-05-08T05:45:11.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.747+0000 7fb862387640 1 -- 172.21.15.45:0/843809759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb85c06f540 msgr2=0x7fb85c15ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:11.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.747+0000 7fb862387640 1 --2- 172.21.15.45:0/843809759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb85c06f540 0x7fb85c15ec30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00c3e0 tx=0x7fb84c00c8a0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.747+0000 7fb862387640 1 -- 172.21.15.45:0/843809759 shutdown_connections 2024-05-08T05:45:11.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.747+0000 7fb862387640 1 --2- 172.21.15.45:0/843809759 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb848076300 0x7fb8480787c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.747+0000 7fb862387640 1 --2- 172.21.15.45:0/843809759 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb85c15bc50 0x7fb85c151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.747+0000 7fb862387640 1 --2- 172.21.15.45:0/843809759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb85c152000 0x7fb85c0d2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.747+0000 7fb862387640 1 --2- 172.21.15.45:0/843809759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb85c06f540 0x7fb85c15ec30 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.747+0000 7fb862387640 1 -- 172.21.15.45:0/843809759 >> 172.21.15.45:0/843809759 conn(0x7fb85c0b3c40 msgr2=0x7fb85c159b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:11.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.747+0000 7fb862387640 1 -- 172.21.15.45:0/843809759 shutdown_connections 2024-05-08T05:45:11.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.747+0000 7fb862387640 1 -- 172.21.15.45:0/843809759 wait complete. 2024-05-08T05:45:11.759 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:45:11.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.862+0000 7f9fa095f640 1 Processor -- start 2024-05-08T05:45:11.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.862+0000 7f9fa095f640 1 -- start start 2024-05-08T05:45:11.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.863+0000 7f9fa095f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9f9c070d30 0x7f9f9c0d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:11.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.863+0000 7f9fa095f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f9c0d1300 con 0x7f9f9c163920 2024-05-08T05:45:11.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.863+0000 7f9fa095f640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f9c0d15e0 con 0x7f9f9c071110 2024-05-08T05:45:11.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.863+0000 7f9fa095f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f9f9c0d10d0 con 0x7f9f9c070d30 2024-05-08T05:45:11.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.863+0000 7f9f9a59c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9f9c070d30 0x7f9f9c0d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:11.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.863+0000 7f9f99d9b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9f9c163920 0x7f9f9c0703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57588/0 (socket says 172.21.15.45:57588) 2024-05-08T05:45:11.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.863+0000 7f9f99d9b640 1 -- 172.21.15.45:0/727018972 learned_addr learned my addr 172.21.15.45:0/727018972 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:11.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.863+0000 7f9f9a59c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9f9c070d30 0x7f9f9c0d20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52592/0 (socket says 172.21.15.45:52592) 2024-05-08T05:45:11.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.863+0000 7f9f9a59c640 1 -- 172.21.15.45:0/727018972 >> v1:172.21.15.63:6789/0 conn(0x7f9f9c071110 legacy=0x7f9f9c06d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:11.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.863+0000 7f9f9a59c640 1 -- 172.21.15.45:0/727018972 >> v1:172.21.15.45:6789/0 conn(0x7f9f9c163920 legacy=0x7f9f9c0703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:11.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.863+0000 7f9f9a59c640 1 -- 172.21.15.45:0/727018972 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9c0d18c0 con 0x7f9f9c070d30 2024-05-08T05:45:11.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.864+0000 7f9f9a59c640 1 --2- 172.21.15.45:0/727018972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f9c070d30 0x7f9f9c0d20c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0097a0 tx=0x7f9f8c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8fe9a9622b7327d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:11.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.864+0000 7f9f9959a640 1 -- 172.21.15.45:0/727018972 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c037070 con 0x7f9f9c070d30 2024-05-08T05:45:11.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.864+0000 7f9f9959a640 1 -- 172.21.15.45:0/727018972 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f8c032440 con 0x7f9f9c070d30 2024-05-08T05:45:11.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.864+0000 7f9f9959a640 1 -- 172.21.15.45:0/727018972 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c032810 con 0x7f9f9c070d30 2024-05-08T05:45:11.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.865+0000 7f9fa095f640 1 -- 172.21.15.45:0/727018972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f9c070d30 msgr2=0x7f9f9c0d20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:11.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.865+0000 7f9fa095f640 1 --2- 172.21.15.45:0/727018972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f9c070d30 0x7f9f9c0d20c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0097a0 tx=0x7f9f8c02a230 comp rx=0 tx=0).stop 2024-05-08T05:45:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.865+0000 7f9fa095f640 1 -- 172.21.15.45:0/727018972 shutdown_connections 2024-05-08T05:45:11.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.865+0000 7f9fa095f640 1 --2- 172.21.15.45:0/727018972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f9c070d30 0x7f9f9c0d20c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.865+0000 7f9fa095f640 1 -- 172.21.15.45:0/727018972 >> 172.21.15.45:0/727018972 conn(0x7f9f9c0d2e90 msgr2=0x7f9f9c0d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:11.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.865+0000 7f9fa095f640 1 -- 172.21.15.45:0/727018972 shutdown_connections 2024-05-08T05:45:11.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.865+0000 7f9fa095f640 1 -- 172.21.15.45:0/727018972 wait complete. 2024-05-08T05:45:11.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.866+0000 7f9fa095f640 1 Processor -- start 2024-05-08T05:45:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.866+0000 7f9fa095f640 1 -- start start 2024-05-08T05:45:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.867+0000 7f9fa095f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f9c070d30 0x7f9f9c0b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.867+0000 7f9fa095f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f9c071110 0x7f9f9c1f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.867+0000 7f9fa095f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f9c163920 0x7f9f9c15bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.867+0000 7f9fa095f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c070b80 con 0x7f9f9c163920 2024-05-08T05:45:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.867+0000 7f9fa095f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9c163db0 con 0x7f9f9c070d30 2024-05-08T05:45:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.867+0000 7f9fa095f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9f9c0d1c30 con 0x7f9f9c071110 2024-05-08T05:45:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.867+0000 7f9f9ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f9c163920 0x7f9f9c15bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.867+0000 7f9f9ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f9c163920 0x7f9f9c15bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52598/0 (socket says 172.21.15.45:52598) 2024-05-08T05:45:11.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.867+0000 7f9f9ad9d640 1 -- 172.21.15.45:0/1770292147 learned_addr learned my addr 172.21.15.45:0/1770292147 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:11.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.867+0000 7f9f9a59c640 1 --2- 172.21.15.45:0/1770292147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f9c070d30 0x7f9f9c0b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:11.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.867+0000 7f9f99d9b640 1 --2- 172.21.15.45:0/1770292147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f9c071110 0x7f9f9c1f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:11.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.868+0000 7f9f9ad9d640 1 -- 172.21.15.45:0/1770292147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f9c071110 msgr2=0x7f9f9c1f99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:11.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.868+0000 7f9f9ad9d640 1 --2- 172.21.15.45:0/1770292147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f9c071110 0x7f9f9c1f99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.868+0000 7f9f9ad9d640 1 -- 172.21.15.45:0/1770292147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f9c070d30 msgr2=0x7f9f9c0b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:11.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.868+0000 7f9f9ad9d640 1 --2- 172.21.15.45:0/1770292147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f9c070d30 0x7f9f9c0b68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:11.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.868+0000 7f9f9ad9d640 1 -- 172.21.15.45:0/1770292147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9c0cb7d0 con 0x7f9f9c163920 2024-05-08T05:45:11.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.868+0000 7f9f9ad9d640 1 --2- 172.21.15.45:0/1770292147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f9c163920 0x7f9f9c15bb00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400c700 tx=0x7f9f9400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:11.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.868+0000 7f9f92ffd640 1 -- 172.21.15.45:0/1770292147 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f94003b70 con 0x7f9f9c163920 2024-05-08T05:45:11.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.868+0000 7f9f92ffd640 1 -- 172.21.15.45:0/1770292147 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f94003d10 con 0x7f9f9c163920 2024-05-08T05:45:11.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.868+0000 7f9fa095f640 1 -- 172.21.15.45:0/1770292147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f9c0b7100 con 0x7f9f9c163920 2024-05-08T05:45:11.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.868+0000 7f9f92ffd640 1 -- 172.21.15.45:0/1770292147 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9400aa70 con 0x7f9f9c163920 2024-05-08T05:45:11.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.868+0000 7f9fa095f640 1 -- 172.21.15.45:0/1770292147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f9c0b76c0 con 0x7f9f9c163920 2024-05-08T05:45:11.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.869+0000 7f9fa095f640 1 -- 172.21.15.45:0/1770292147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f9c0693f0 con 0x7f9f9c163920 2024-05-08T05:45:11.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.870+0000 7f9f92ffd640 1 -- 172.21.15.45:0/1770292147 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9f9400ac10 con 0x7f9f9c163920 2024-05-08T05:45:11.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.870+0000 7f9f92ffd640 1 --2- 172.21.15.45:0/1770292147 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9f7c0762d0 0x7f9f7c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:11.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.870+0000 7f9f92ffd640 1 -- 172.21.15.45:0/1770292147 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f9f940974f0 con 0x7f9f9c163920 2024-05-08T05:45:11.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.870+0000 7f9f9a59c640 1 --2- 172.21.15.45:0/1770292147 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9f7c0762d0 0x7f9f7c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:11.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.871+0000 7f9f9a59c640 1 --2- 172.21.15.45:0/1770292147 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9f7c0762d0 0x7f9f7c078790 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c034000 tx=0x7f9f8c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:11.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:11.874+0000 7f9f92ffd640 1 -- 172.21.15.45:0/1770292147 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9f94060670 con 0x7f9f9c163920 2024-05-08T05:45:12.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.066+0000 7f9fa095f640 1 -- 172.21.15.45:0/1770292147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f9c0d3d70 con 0x7f9f9c163920 2024-05-08T05:45:12.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.068+0000 7f9f92ffd640 1 -- 172.21.15.45:0/1770292147 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+17076 (secure 0 0 0) 0x7f9f94064970 con 0x7f9f9c163920 2024-05-08T05:45:12.068 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:12.068 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":63,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:44:14.980856+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:43:55.774105+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-05-08T05:43:57.070302+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.13","mappings":[{"from":7,"to":2}]},{"pgid":"10.d","mappings":[{"from":4,"to":3}]},{"pgid":"10.13","mappings":[{"from":7,"to":5}]},{"pgid":"10.2e","mappings":[{"from":4,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:45:12.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.070+0000 7f9fa095f640 1 -- 172.21.15.45:0/1770292147 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9f7c0762d0 msgr2=0x7f9f7c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:12.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.070+0000 7f9fa095f640 1 --2- 172.21.15.45:0/1770292147 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9f7c0762d0 0x7f9f7c078790 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c034000 tx=0x7f9f8c02e000 comp rx=0 tx=0).stop 2024-05-08T05:45:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.070+0000 7f9fa095f640 1 -- 172.21.15.45:0/1770292147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f9c163920 msgr2=0x7f9f9c15bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.070+0000 7f9fa095f640 1 --2- 172.21.15.45:0/1770292147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f9c163920 0x7f9f9c15bb00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400c700 tx=0x7f9f9400cbc0 comp rx=0 tx=0).stop 2024-05-08T05:45:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.070+0000 7f9fa095f640 1 -- 172.21.15.45:0/1770292147 shutdown_connections 2024-05-08T05:45:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.070+0000 7f9fa095f640 1 --2- 172.21.15.45:0/1770292147 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9f7c0762d0 0x7f9f7c078790 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.070+0000 7f9fa095f640 1 --2- 172.21.15.45:0/1770292147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f9c071110 0x7f9f9c1f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.070+0000 7f9fa095f640 1 --2- 172.21.15.45:0/1770292147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f9c163920 0x7f9f9c15bb00 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.070+0000 7f9fa095f640 1 --2- 172.21.15.45:0/1770292147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f9c070d30 0x7f9f9c0b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.070+0000 7f9fa095f640 1 -- 172.21.15.45:0/1770292147 >> 172.21.15.45:0/1770292147 conn(0x7f9f9c0d2e90 msgr2=0x7f9f9c160e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.070+0000 7f9fa095f640 1 -- 172.21.15.45:0/1770292147 shutdown_connections 2024-05-08T05:45:12.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.070+0000 7f9fa095f640 1 -- 172.21.15.45:0/1770292147 wait complete. 2024-05-08T05:45:12.081 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T05:45:12.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.186+0000 7f2e1a9af640 1 Processor -- start 2024-05-08T05:45:12.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.186+0000 7f2e1a9af640 1 -- start start 2024-05-08T05:45:12.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.186+0000 7f2e1a9af640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2e1415ce30 0x7f2e1406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:12.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.186+0000 7f2e1a9af640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2e14161630 0x7f2e140ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:12.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.186+0000 7f2e1a9af640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e140715b0 con 0x7f2e1406c3a0 2024-05-08T05:45:12.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.187+0000 7f2e1a9af640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f2e140d0840 con 0x7f2e14161630 2024-05-08T05:45:12.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.187+0000 7f2e1a9af640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2e14071890 con 0x7f2e1415ce30 2024-05-08T05:45:12.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.187+0000 7f2e0bfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2e14161630 0x7f2e140ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:12.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.187+0000 7f2e18f4c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2e1406c3a0 0x7f2e140d0cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57590/0 (socket says 172.21.15.45:57590) 2024-05-08T05:45:12.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.187+0000 7f2e0bfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2e14161630 0x7f2e140ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52612/0 (socket says 172.21.15.45:52612) 2024-05-08T05:45:12.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.187+0000 7f2e18f4c640 1 -- 172.21.15.45:0/1473017859 learned_addr learned my addr 172.21.15.45:0/1473017859 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:12.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.187+0000 7f2e0bfff640 1 -- 172.21.15.45:0/1473017859 >> v2:172.21.15.63:3300/0 conn(0x7f2e1415ce30 msgr2=0x7f2e1406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:12.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.187+0000 7f2e0bfff640 1 --2- 172.21.15.45:0/1473017859 >> v2:172.21.15.63:3300/0 conn(0x7f2e1415ce30 0x7f2e1406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:12.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.187+0000 7f2e0bfff640 1 -- 172.21.15.45:0/1473017859 >> v1:172.21.15.45:6789/0 conn(0x7f2e1406c3a0 legacy=0x7f2e140d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:12.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.187+0000 7f2e0bfff640 1 -- 172.21.15.45:0/1473017859 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e14071b70 con 0x7f2e14161630 2024-05-08T05:45:12.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.188+0000 7f2e0bfff640 1 --2- 172.21.15.45:0/1473017859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e14161630 0x7f2e140ce920 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2e14074580 tx=0x7f2dfc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e9546c91a8a336 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:12.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.188+0000 7f2e137fe640 1 -- 172.21.15.45:0/1473017859 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc037070 con 0x7f2e14161630 2024-05-08T05:45:12.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.188+0000 7f2e137fe640 1 -- 172.21.15.45:0/1473017859 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dfc032440 con 0x7f2e14161630 2024-05-08T05:45:12.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.188+0000 7f2e137fe640 1 -- 172.21.15.45:0/1473017859 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc032810 con 0x7f2e14161630 2024-05-08T05:45:12.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.188+0000 7f2e1a9af640 1 -- 172.21.15.45:0/1473017859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e14161630 msgr2=0x7f2e140ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:12.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.188+0000 7f2e1a9af640 1 --2- 172.21.15.45:0/1473017859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e14161630 0x7f2e140ce920 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2e14074580 tx=0x7f2dfc02a1a0 comp rx=0 tx=0).stop 2024-05-08T05:45:12.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.188+0000 7f2e1a9af640 1 -- 172.21.15.45:0/1473017859 shutdown_connections 2024-05-08T05:45:12.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.188+0000 7f2e1a9af640 1 --2- 172.21.15.45:0/1473017859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e14161630 0x7f2e140ce920 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:12.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.188+0000 7f2e1a9af640 1 --2- 172.21.15.45:0/1473017859 >> v2:172.21.15.63:3300/0 conn(0x7f2e1415ce30 0x7f2e1406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:12.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.188+0000 7f2e1a9af640 1 -- 172.21.15.45:0/1473017859 >> 172.21.15.45:0/1473017859 conn(0x7f2e140b3c40 msgr2=0x7f2e140b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:12.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.189+0000 7f2e1a9af640 1 -- 172.21.15.45:0/1473017859 shutdown_connections 2024-05-08T05:45:12.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.189+0000 7f2e1a9af640 1 -- 172.21.15.45:0/1473017859 wait complete. 2024-05-08T05:45:12.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.190+0000 7f2e1a9af640 1 Processor -- start 2024-05-08T05:45:12.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.190+0000 7f2e1a9af640 1 -- start start 2024-05-08T05:45:12.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e1a9af640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e1406c3a0 0x7f2e141f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:12.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e1a9af640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e1415ce30 0x7f2e141542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:12.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e1a9af640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e14161630 0x7f2e14155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:12.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e1a9af640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e140cee60 con 0x7f2e14161630 2024-05-08T05:45:12.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e1a9af640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2e140d0460 con 0x7f2e1415ce30 2024-05-08T05:45:12.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e1a9af640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2e140cefe0 con 0x7f2e1406c3a0 2024-05-08T05:45:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e18f4c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e14161630 0x7f2e14155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e18f4c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e14161630 0x7f2e14155d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52624/0 (socket says 172.21.15.45:52624) 2024-05-08T05:45:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e18f4c640 1 -- 172.21.15.45:0/2942607759 learned_addr learned my addr 172.21.15.45:0/2942607759 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e13fff640 1 --2- 172.21.15.45:0/2942607759 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e1406c3a0 0x7f2e141f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e0bfff640 1 --2- 172.21.15.45:0/2942607759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e1415ce30 0x7f2e141542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e18f4c640 1 -- 172.21.15.45:0/2942607759 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e1406c3a0 msgr2=0x7f2e141f2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e18f4c640 1 --2- 172.21.15.45:0/2942607759 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e1406c3a0 0x7f2e141f2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e18f4c640 1 -- 172.21.15.45:0/2942607759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e1415ce30 msgr2=0x7f2e141542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e18f4c640 1 --2- 172.21.15.45:0/2942607759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e1415ce30 0x7f2e141542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e18f4c640 1 -- 172.21.15.45:0/2942607759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e14154cb0 con 0x7f2e14161630 2024-05-08T05:45:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e0bfff640 1 --2- 172.21.15.45:0/2942607759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e1415ce30 0x7f2e141542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e18f4c640 1 --2- 172.21.15.45:0/2942607759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e14161630 0x7f2e14155d50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00cb50 tx=0x7f2e0c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e117fa640 1 -- 172.21.15.45:0/2942607759 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c017070 con 0x7f2e14161630 2024-05-08T05:45:12.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e117fa640 1 -- 172.21.15.45:0/2942607759 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e0c00ad40 con 0x7f2e14161630 2024-05-08T05:45:12.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e1a9af640 1 -- 172.21.15.45:0/2942607759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e14154f90 con 0x7f2e14161630 2024-05-08T05:45:12.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e117fa640 1 -- 172.21.15.45:0/2942607759 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c012600 con 0x7f2e14161630 2024-05-08T05:45:12.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.191+0000 7f2e1a9af640 1 -- 172.21.15.45:0/2942607759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e14155550 con 0x7f2e14161630 2024-05-08T05:45:12.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.192+0000 7f2e1a9af640 1 -- 172.21.15.45:0/2942607759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e14069b00 con 0x7f2e14161630 2024-05-08T05:45:12.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.193+0000 7f2e117fa640 1 -- 172.21.15.45:0/2942607759 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2e0c01a070 con 0x7f2e14161630 2024-05-08T05:45:12.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.193+0000 7f2e117fa640 1 --2- 172.21.15.45:0/2942607759 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2df40762d0 0x7f2df4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:12.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.194+0000 7f2e117fa640 1 -- 172.21.15.45:0/2942607759 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6246+0+0 (secure 0 0 0) 0x7f2e0c097540 con 0x7f2e14161630 2024-05-08T05:45:12.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.194+0000 7f2e13fff640 1 --2- 172.21.15.45:0/2942607759 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2df40762d0 0x7f2df4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:12.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.194+0000 7f2e13fff640 1 --2- 172.21.15.45:0/2942607759 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2df40762d0 0x7f2df4078790 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009890 tx=0x7f2e04008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:12.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.198+0000 7f2e117fa640 1 -- 172.21.15.45:0/2942607759 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2e0c060620 con 0x7f2e14161630 2024-05-08T05:45:12.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.432+0000 7f2e1a9af640 1 -- 172.21.15.45:0/2942607759 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2e140b4870 con 0x7f2e14161630 2024-05-08T05:45:12.432 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-05-08T05:45:12.432+0000 7f124de12640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T05:45:12.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.433+0000 7f2e117fa640 1 -- 172.21.15.45:0/2942607759 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) ==== 161+0+0 (secure 0 0 0) 0x7f2e0c064920 con 0x7f2e14161630 2024-05-08T05:45:12.434 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T05:45:12.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.436+0000 7f2e1a9af640 1 -- 172.21.15.45:0/2942607759 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2df40762d0 msgr2=0x7f2df4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:12.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.436+0000 7f2e1a9af640 1 --2- 172.21.15.45:0/2942607759 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2df40762d0 0x7f2df4078790 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009890 tx=0x7f2e04008040 comp rx=0 tx=0).stop 2024-05-08T05:45:12.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.436+0000 7f2e1a9af640 1 -- 172.21.15.45:0/2942607759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e14161630 msgr2=0x7f2e14155d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:12.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.436+0000 7f2e1a9af640 1 --2- 172.21.15.45:0/2942607759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e14161630 0x7f2e14155d50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00cb50 tx=0x7f2e0c00a430 comp rx=0 tx=0).stop 2024-05-08T05:45:12.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.436+0000 7f2e1a9af640 1 -- 172.21.15.45:0/2942607759 shutdown_connections 2024-05-08T05:45:12.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.436+0000 7f2e1a9af640 1 --2- 172.21.15.45:0/2942607759 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2df40762d0 0x7f2df4078790 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:12.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.436+0000 7f2e1a9af640 1 --2- 172.21.15.45:0/2942607759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e14161630 0x7f2e14155d50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:12.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.436+0000 7f2e1a9af640 1 --2- 172.21.15.45:0/2942607759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e1415ce30 0x7f2e141542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:12.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.436+0000 7f2e1a9af640 1 --2- 172.21.15.45:0/2942607759 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e1406c3a0 0x7f2e141f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:12.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.437+0000 7f2e1a9af640 1 -- 172.21.15.45:0/2942607759 >> 172.21.15.45:0/2942607759 conn(0x7f2e140b3c40 msgr2=0x7f2e14157520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:12.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.437+0000 7f2e1a9af640 1 -- 172.21.15.45:0/2942607759 shutdown_connections 2024-05-08T05:45:12.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.437+0000 7f2e1a9af640 1 -- 172.21.15.45:0/2942607759 wait complete. 2024-05-08T05:45:12.440 INFO:tasks.ceph.mds.a.smithi045.stderr: -7512> 2024-05-08T05:44:04.376+0000 7f3447f21640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:45:12.443 INFO:tasks.ceph.mds.a.smithi045.stderr: -4753> 2024-05-08T05:44:06.702+0000 7f3443f19640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5177=0-4194304@1} caps={5177=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5177 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x560338028b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-08T05:45:12.445 INFO:tasks.ceph.mds.a.smithi045.stderr: -2924> 2024-05-08T05:44:38.066+0000 7f3443f19640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5177=0-4194304@1} caps={5177=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5177 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x560338028b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-08T05:45:12.449 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T05:45:12.473 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-05-08T05:45:12.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.561+0000 7faa2e719640 1 Processor -- start 2024-05-08T05:45:12.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.561+0000 7faa2e719640 1 -- start start 2024-05-08T05:45:12.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.562+0000 7faa2e719640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa280d3cd0 0x7faa280cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:12.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.562+0000 7faa2e719640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7faa280c9ee0 0x7faa280d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:12.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.562+0000 7faa2e719640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa280d3330 con 0x7faa281629a0 2024-05-08T05:45:12.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.562+0000 7faa2e719640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7faa280d2170 con 0x7faa280c9ee0 2024-05-08T05:45:12.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.562+0000 7faa2e719640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faa280d2450 con 0x7faa280d3cd0 2024-05-08T05:45:12.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.562+0000 7faa277fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa280d3cd0 0x7faa280cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:12.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.563+0000 7faa27fff640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7faa281629a0 0x7faa280d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:42116/0 (socket says 172.21.15.45:42116) 2024-05-08T05:45:12.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.563+0000 7faa277fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa280d3cd0 0x7faa280cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:46980/0 (socket says 172.21.15.45:46980) 2024-05-08T05:45:12.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.563+0000 7faa27fff640 1 -- 172.21.15.45:0/3818521630 learned_addr learned my addr 172.21.15.45:0/3818521630 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:12.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.563+0000 7faa2ccb6640 1 --2- 172.21.15.45:0/3818521630 >> v2:172.21.15.63:3300/0 conn(0x7faa280c9ee0 0x7faa280d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:12.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.563+0000 7faa277fe640 1 -- 172.21.15.45:0/3818521630 >> v2:172.21.15.63:3300/0 conn(0x7faa280c9ee0 msgr2=0x7faa280d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:12.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.563+0000 7faa277fe640 1 --2- 172.21.15.45:0/3818521630 >> v2:172.21.15.63:3300/0 conn(0x7faa280c9ee0 0x7faa280d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:12.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.563+0000 7faa277fe640 1 -- 172.21.15.45:0/3818521630 >> v1:172.21.15.63:6789/0 conn(0x7faa281629a0 legacy=0x7faa280d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:12.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.563+0000 7faa277fe640 1 -- 172.21.15.45:0/3818521630 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa280d2730 con 0x7faa280d3cd0 2024-05-08T05:45:12.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.564+0000 7faa277fe640 1 --2- 172.21.15.45:0/3818521630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa280d3cd0 0x7faa280cfca0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0097a0 tx=0x7faa1c02a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c2a644855927500 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:12.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.564+0000 7faa26ffd640 1 -- 172.21.15.45:0/3818521630 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c037070 con 0x7faa280d3cd0 2024-05-08T05:45:12.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.564+0000 7faa26ffd640 1 -- 172.21.15.45:0/3818521630 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa1c02adb0 con 0x7faa280d3cd0 2024-05-08T05:45:12.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.564+0000 7faa26ffd640 1 -- 172.21.15.45:0/3818521630 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c032660 con 0x7faa280d3cd0 2024-05-08T05:45:12.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.564+0000 7faa2e719640 1 -- 172.21.15.45:0/3818521630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa280d3cd0 msgr2=0x7faa280cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:12.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.564+0000 7faa2e719640 1 --2- 172.21.15.45:0/3818521630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa280d3cd0 0x7faa280cfca0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0097a0 tx=0x7faa1c02a060 comp rx=0 tx=0).stop 2024-05-08T05:45:12.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.564+0000 7faa2e719640 1 -- 172.21.15.45:0/3818521630 shutdown_connections 2024-05-08T05:45:12.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.564+0000 7faa2e719640 1 --2- 172.21.15.45:0/3818521630 >> v2:172.21.15.63:3300/0 conn(0x7faa280c9ee0 0x7faa280d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:12.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.564+0000 7faa2e719640 1 --2- 172.21.15.45:0/3818521630 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa280d3cd0 0x7faa280cfca0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:12.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.564+0000 7faa2e719640 1 -- 172.21.15.45:0/3818521630 >> 172.21.15.45:0/3818521630 conn(0x7faa280d4fe0 msgr2=0x7faa280d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:12.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.564+0000 7faa2e719640 1 -- 172.21.15.45:0/3818521630 shutdown_connections 2024-05-08T05:45:12.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.564+0000 7faa2e719640 1 -- 172.21.15.45:0/3818521630 wait complete. 2024-05-08T05:45:12.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.566+0000 7faa2e719640 1 Processor -- start 2024-05-08T05:45:12.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.566+0000 7faa2e719640 1 -- start start 2024-05-08T05:45:12.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.566+0000 7faa2e719640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa280c9ee0 0x7faa2814f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:12.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.566+0000 7faa2e719640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa280d3cd0 0x7faa280d0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:12.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.566+0000 7faa2e719640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa281629a0 0x7faa280d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:12.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.566+0000 7faa2e719640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faa280cf440 con 0x7faa281629a0 2024-05-08T05:45:12.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.566+0000 7faa2e719640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faa280cf2c0 con 0x7faa280d3cd0 2024-05-08T05:45:12.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.566+0000 7faa2e719640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faa280d2dc0 con 0x7faa280c9ee0 2024-05-08T05:45:12.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.567+0000 7faa2ccb6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa281629a0 0x7faa280d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:12.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.567+0000 7faa2ccb6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa281629a0 0x7faa280d0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52638/0 (socket says 172.21.15.45:52638) 2024-05-08T05:45:12.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.567+0000 7faa2ccb6640 1 -- 172.21.15.45:0/1073765188 learned_addr learned my addr 172.21.15.45:0/1073765188 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:12.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.567+0000 7faa277fe640 1 --2- 172.21.15.45:0/1073765188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa280d3cd0 0x7faa280d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:12.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.567+0000 7faa27fff640 1 --2- 172.21.15.45:0/1073765188 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa280c9ee0 0x7faa2814f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:12.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.567+0000 7faa2ccb6640 1 -- 172.21.15.45:0/1073765188 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa280c9ee0 msgr2=0x7faa2814f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:12.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.567+0000 7faa2ccb6640 1 --2- 172.21.15.45:0/1073765188 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa280c9ee0 0x7faa2814f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:12.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.567+0000 7faa2ccb6640 1 -- 172.21.15.45:0/1073765188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa280d3cd0 msgr2=0x7faa280d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:12.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.567+0000 7faa2ccb6640 1 --2- 172.21.15.45:0/1073765188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa280d3cd0 0x7faa280d0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:12.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.567+0000 7faa2ccb6640 1 -- 172.21.15.45:0/1073765188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa281f4f60 con 0x7faa281629a0 2024-05-08T05:45:12.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.567+0000 7faa27fff640 1 --2- 172.21.15.45:0/1073765188 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa280c9ee0 0x7faa2814f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:45:12.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.567+0000 7faa277fe640 1 --2- 172.21.15.45:0/1073765188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa280d3cd0 0x7faa280d0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:45:12.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.567+0000 7faa2ccb6640 1 --2- 172.21.15.45:0/1073765188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa281629a0 0x7faa280d0a50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7faa2000efc0 tx=0x7faa2000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:12.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.567+0000 7faa24ff9640 1 -- 172.21.15.45:0/1073765188 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa20018020 con 0x7faa281629a0 2024-05-08T05:45:12.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.567+0000 7faa24ff9640 1 -- 172.21.15.45:0/1073765188 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa20014480 con 0x7faa281629a0 2024-05-08T05:45:12.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.567+0000 7faa24ff9640 1 -- 172.21.15.45:0/1073765188 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa20014870 con 0x7faa281629a0 2024-05-08T05:45:12.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.567+0000 7faa2e719640 1 -- 172.21.15.45:0/1073765188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa2814f940 con 0x7faa281629a0 2024-05-08T05:45:12.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.568+0000 7faa2e719640 1 -- 172.21.15.45:0/1073765188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa281f54e0 con 0x7faa281629a0 2024-05-08T05:45:12.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.568+0000 7faa2e719640 1 -- 172.21.15.45:0/1073765188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa280693f0 con 0x7faa281629a0 2024-05-08T05:45:12.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.569+0000 7faa24ff9640 1 -- 172.21.15.45:0/1073765188 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7faa20005d10 con 0x7faa281629a0 2024-05-08T05:45:12.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.570+0000 7faa24ff9640 1 --2- 172.21.15.45:0/1073765188 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faa140762d0 0x7faa14078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:12.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.570+0000 7faa24ff9640 1 -- 172.21.15.45:0/1073765188 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6332+0+0 (secure 0 0 0) 0x7faa20098270 con 0x7faa281629a0 2024-05-08T05:45:12.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.570+0000 7faa27fff640 1 --2- 172.21.15.45:0/1073765188 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faa140762d0 0x7faa14078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:12.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.570+0000 7faa27fff640 1 --2- 172.21.15.45:0/1073765188 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faa140762d0 0x7faa14078790 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faa18000f60 tx=0x7faa18007ea0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:12.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.574+0000 7faa24ff9640 1 -- 172.21.15.45:0/1073765188 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7faa20022070 con 0x7faa281629a0 2024-05-08T05:45:12.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:12.804+0000 7faa2e719640 1 -- 172.21.15.45:0/1073765188 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7faa280d1a30 con 0x7faa281629a0 2024-05-08T05:45:13.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.438+0000 7faa24ff9640 1 -- 172.21.15.45:0/1073765188 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) ==== 104+0+0 (secure 0 0 0) 0x7faa20010040 con 0x7faa281629a0 2024-05-08T05:45:13.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.440+0000 7faa2e719640 1 -- 172.21.15.45:0/1073765188 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faa140762d0 msgr2=0x7faa14078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:13.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.440+0000 7faa2e719640 1 --2- 172.21.15.45:0/1073765188 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faa140762d0 0x7faa14078790 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faa18000f60 tx=0x7faa18007ea0 comp rx=0 tx=0).stop 2024-05-08T05:45:13.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.440+0000 7faa2e719640 1 -- 172.21.15.45:0/1073765188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa281629a0 msgr2=0x7faa280d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:13.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.440+0000 7faa2e719640 1 --2- 172.21.15.45:0/1073765188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa281629a0 0x7faa280d0a50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7faa2000efc0 tx=0x7faa2000c490 comp rx=0 tx=0).stop 2024-05-08T05:45:13.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.440+0000 7faa2e719640 1 -- 172.21.15.45:0/1073765188 shutdown_connections 2024-05-08T05:45:13.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.440+0000 7faa2e719640 1 --2- 172.21.15.45:0/1073765188 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faa140762d0 0x7faa14078790 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:13.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.440+0000 7faa2e719640 1 --2- 172.21.15.45:0/1073765188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa281629a0 0x7faa280d0a50 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:13.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.440+0000 7faa2e719640 1 --2- 172.21.15.45:0/1073765188 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa280d3cd0 0x7faa280d0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:13.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.440+0000 7faa2e719640 1 --2- 172.21.15.45:0/1073765188 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa280c9ee0 0x7faa2814f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:13.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.440+0000 7faa2e719640 1 -- 172.21.15.45:0/1073765188 >> 172.21.15.45:0/1073765188 conn(0x7faa280d4fe0 msgr2=0x7faa28160a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:13.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.440+0000 7faa2e719640 1 -- 172.21.15.45:0/1073765188 shutdown_connections 2024-05-08T05:45:13.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.441+0000 7faa2e719640 1 -- 172.21.15.45:0/1073765188 wait complete. 2024-05-08T05:45:13.454 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T05:45:13.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.605+0000 7f17e1fdd640 1 Processor -- start 2024-05-08T05:45:13.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.605+0000 7f17e1fdd640 1 -- start start 2024-05-08T05:45:13.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.606+0000 7f17e1fdd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f17dc0d3670 0x7f17dc0b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:13.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.606+0000 7f17e1fdd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17dc0cfc00 con 0x7f17dc0d1050 2024-05-08T05:45:13.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.606+0000 7f17e1fdd640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17dc0cfea0 con 0x7f17dc0d3a50 2024-05-08T05:45:13.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.606+0000 7f17e1fdd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f17dc0d2a10 con 0x7f17dc0d3670 2024-05-08T05:45:13.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.606+0000 7f17db7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f17dc0d3670 0x7f17dc0b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:13.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.606+0000 7f17dbfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f17dc0d1050 0x7f17dc162920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57606/0 (socket says 172.21.15.45:57606) 2024-05-08T05:45:13.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.606+0000 7f17dbfff640 1 -- 172.21.15.45:0/3860849702 learned_addr learned my addr 172.21.15.45:0/3860849702 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:13.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.606+0000 7f17db7fe640 1 -- 172.21.15.45:0/3860849702 >> v1:172.21.15.63:6789/0 conn(0x7f17dc0d3a50 legacy=0x7f17dc0d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:13.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.606+0000 7f17db7fe640 1 -- 172.21.15.45:0/3860849702 >> v1:172.21.15.45:6789/0 conn(0x7f17dc0d1050 legacy=0x7f17dc162920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:13.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.606+0000 7f17da7fc640 1 -- 172.21.15.45:0/3860849702 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 117580215 0 0) 0x7f17dc0cfc00 con 0x7f17dc0d1050 2024-05-08T05:45:13.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.606+0000 7f17db7fe640 1 -- 172.21.15.45:0/3860849702 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17dc0d2730 con 0x7f17dc0d3670 2024-05-08T05:45:13.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.607+0000 7f17db7fe640 1 --2- 172.21.15.45:0/3860849702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17dc0d3670 0x7f17dc0b49c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0097a0 tx=0x7f17cc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fb90301d81905c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:13.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.607+0000 7f17da7fc640 1 -- 172.21.15.45:0/3860849702 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc037070 con 0x7f17dc0d3670 2024-05-08T05:45:13.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.607+0000 7f17da7fc640 1 -- 172.21.15.45:0/3860849702 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17cc032440 con 0x7f17dc0d3670 2024-05-08T05:45:13.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.607+0000 7f17da7fc640 1 -- 172.21.15.45:0/3860849702 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc032810 con 0x7f17dc0d3670 2024-05-08T05:45:13.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.608+0000 7f17e1fdd640 1 -- 172.21.15.45:0/3860849702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17dc0d3670 msgr2=0x7f17dc0b49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:13.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.608+0000 7f17e1fdd640 1 --2- 172.21.15.45:0/3860849702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17dc0d3670 0x7f17dc0b49c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0097a0 tx=0x7f17cc02a180 comp rx=0 tx=0).stop 2024-05-08T05:45:13.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.608+0000 7f17e1fdd640 1 -- 172.21.15.45:0/3860849702 shutdown_connections 2024-05-08T05:45:13.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.608+0000 7f17e1fdd640 1 --2- 172.21.15.45:0/3860849702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17dc0d3670 0x7f17dc0b49c0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:13.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.608+0000 7f17e1fdd640 1 -- 172.21.15.45:0/3860849702 >> 172.21.15.45:0/3860849702 conn(0x7f17dc09e390 msgr2=0x7f17dc0d5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:13.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.608+0000 7f17e1fdd640 1 -- 172.21.15.45:0/3860849702 shutdown_connections 2024-05-08T05:45:13.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.608+0000 7f17e1fdd640 1 -- 172.21.15.45:0/3860849702 wait complete. 2024-05-08T05:45:13.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.609+0000 7f17e1fdd640 1 Processor -- start 2024-05-08T05:45:13.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.609+0000 7f17e1fdd640 1 -- start start 2024-05-08T05:45:13.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.610+0000 7f17e1fdd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17dc0d1050 0x7f17dc157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:13.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.610+0000 7f17e1fdd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17dc0d3670 0x7f17dc0ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:13.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.610+0000 7f17e1fdd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17dc0d3a50 0x7f17dc0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:13.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.610+0000 7f17e1fdd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc0b4f00 con 0x7f17dc0d3670 2024-05-08T05:45:13.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.610+0000 7f17e1fdd640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc0d04b0 con 0x7f17dc0d1050 2024-05-08T05:45:13.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.610+0000 7f17e1fdd640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f17dc0b5080 con 0x7f17dc0d3a50 2024-05-08T05:45:13.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.610+0000 7f17daffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17dc0d3670 0x7f17dc0ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:13.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.610+0000 7f17daffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17dc0d3670 0x7f17dc0ce530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52660/0 (socket says 172.21.15.45:52660) 2024-05-08T05:45:13.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.610+0000 7f17daffd640 1 -- 172.21.15.45:0/1009864492 learned_addr learned my addr 172.21.15.45:0/1009864492 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:13.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.610+0000 7f17db7fe640 1 --2- 172.21.15.45:0/1009864492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17dc0d1050 0x7f17dc157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:13.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.610+0000 7f17dbfff640 1 --2- 172.21.15.45:0/1009864492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17dc0d3a50 0x7f17dc0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:13.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.610+0000 7f17daffd640 1 -- 172.21.15.45:0/1009864492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17dc0d3a50 msgr2=0x7f17dc0cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:13.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.610+0000 7f17daffd640 1 --2- 172.21.15.45:0/1009864492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17dc0d3a50 0x7f17dc0cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:13.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.610+0000 7f17daffd640 1 -- 172.21.15.45:0/1009864492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17dc0d1050 msgr2=0x7f17dc157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:13.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.610+0000 7f17daffd640 1 --2- 172.21.15.45:0/1009864492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17dc0d1050 0x7f17dc157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:13.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.610+0000 7f17daffd640 1 -- 172.21.15.45:0/1009864492 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17dc0ccf10 con 0x7f17dc0d3670 2024-05-08T05:45:13.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.610+0000 7f17db7fe640 1 --2- 172.21.15.45:0/1009864492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17dc0d1050 0x7f17dc157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:13.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.611+0000 7f17daffd640 1 --2- 172.21.15.45:0/1009864492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17dc0d3670 0x7f17dc0ce530 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f17d000b420 tx=0x7f17d000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:13.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.611+0000 7f17c3fff640 1 -- 172.21.15.45:0/1009864492 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0011830 con 0x7f17dc0d3670 2024-05-08T05:45:13.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.611+0000 7f17c3fff640 1 -- 172.21.15.45:0/1009864492 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17d0016070 con 0x7f17dc0d3670 2024-05-08T05:45:13.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.611+0000 7f17e1fdd640 1 -- 172.21.15.45:0/1009864492 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17dc0cd1b0 con 0x7f17dc0d3670 2024-05-08T05:45:13.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.611+0000 7f17c3fff640 1 -- 172.21.15.45:0/1009864492 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0011c10 con 0x7f17dc0d3670 2024-05-08T05:45:13.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.611+0000 7f17e1fdd640 1 -- 172.21.15.45:0/1009864492 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17dc152e60 con 0x7f17dc0d3670 2024-05-08T05:45:13.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.612+0000 7f17e1fdd640 1 -- 172.21.15.45:0/1009864492 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17dc0693f0 con 0x7f17dc0d3670 2024-05-08T05:45:13.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.613+0000 7f17c3fff640 1 -- 172.21.15.45:0/1009864492 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f17d001b460 con 0x7f17dc0d3670 2024-05-08T05:45:13.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.613+0000 7f17c3fff640 1 --2- 172.21.15.45:0/1009864492 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f17c80762d0 0x7f17c8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:13.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.613+0000 7f17c3fff640 1 -- 172.21.15.45:0/1009864492 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6332+0+0 (secure 0 0 0) 0x7f17d0097540 con 0x7f17dc0d3670 2024-05-08T05:45:13.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.613+0000 7f17db7fe640 1 --2- 172.21.15.45:0/1009864492 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f17c80762d0 0x7f17c8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:13.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.614+0000 7f17db7fe640 1 --2- 172.21.15.45:0/1009864492 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f17c80762d0 0x7f17c8078790 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f17cc034000 tx=0x7f17cc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:13.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.618+0000 7f17c3fff640 1 -- 172.21.15.45:0/1009864492 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f17d0060c30 con 0x7f17dc0d3670 2024-05-08T05:45:13.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:13.811+0000 7f17e1fdd640 1 -- 172.21.15.45:0/1009864492 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f17dc0b4bc0 con 0x7f17dc0d3670 2024-05-08T05:45:14.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.439+0000 7f17c3fff640 1 -- 172.21.15.45:0/1009864492 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v65) ==== 181+0+0 (secure 0 0 0) 0x7f17d0060530 con 0x7f17dc0d3670 2024-05-08T05:45:14.440 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-05-08T05:45:14.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.443+0000 7f17e1fdd640 1 -- 172.21.15.45:0/1009864492 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f17c80762d0 msgr2=0x7f17c8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:14.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.443+0000 7f17e1fdd640 1 --2- 172.21.15.45:0/1009864492 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f17c80762d0 0x7f17c8078790 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f17cc034000 tx=0x7f17cc02e000 comp rx=0 tx=0).stop 2024-05-08T05:45:14.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.443+0000 7f17e1fdd640 1 -- 172.21.15.45:0/1009864492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17dc0d3670 msgr2=0x7f17dc0ce530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:14.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.443+0000 7f17e1fdd640 1 --2- 172.21.15.45:0/1009864492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17dc0d3670 0x7f17dc0ce530 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f17d000b420 tx=0x7f17d000b8e0 comp rx=0 tx=0).stop 2024-05-08T05:45:14.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.443+0000 7f17e1fdd640 1 -- 172.21.15.45:0/1009864492 shutdown_connections 2024-05-08T05:45:14.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.443+0000 7f17e1fdd640 1 --2- 172.21.15.45:0/1009864492 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f17c80762d0 0x7f17c8078790 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:14.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.443+0000 7f17e1fdd640 1 --2- 172.21.15.45:0/1009864492 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f17dc0d3a50 0x7f17dc0cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:14.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.443+0000 7f17e1fdd640 1 --2- 172.21.15.45:0/1009864492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17dc0d3670 0x7f17dc0ce530 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:14.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.443+0000 7f17e1fdd640 1 --2- 172.21.15.45:0/1009864492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17dc0d1050 0x7f17dc157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:14.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.443+0000 7f17e1fdd640 1 -- 172.21.15.45:0/1009864492 >> 172.21.15.45:0/1009864492 conn(0x7f17dc09e390 msgr2=0x7f17dc15e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:14.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.443+0000 7f17e1fdd640 1 -- 172.21.15.45:0/1009864492 shutdown_connections 2024-05-08T05:45:14.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.443+0000 7f17e1fdd640 1 -- 172.21.15.45:0/1009864492 wait complete. 2024-05-08T05:45:14.458 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T05:45:14.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.572+0000 7f5094def640 1 Processor -- start 2024-05-08T05:45:14.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.572+0000 7f5094def640 1 -- start start 2024-05-08T05:45:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.573+0000 7f5094def640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50900cfef0 0x7f50900d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.573+0000 7f5094def640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50900d1d70 0x7f509006d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.573+0000 7f5094def640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50900cde80 con 0x7f50901629f0 2024-05-08T05:45:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.573+0000 7f5094def640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f50900ce120 con 0x7f50900cfef0 2024-05-08T05:45:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.573+0000 7f5094def640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f50900ce400 con 0x7f50900d1d70 2024-05-08T05:45:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.573+0000 7f508dd9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50900cfef0 0x7f50900d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.573+0000 7f508e59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f50901629f0 0x7f50900d1120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57618/0 (socket says 172.21.15.45:57618) 2024-05-08T05:45:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.573+0000 7f508e59c640 1 -- 172.21.15.45:0/2419709226 learned_addr learned my addr 172.21.15.45:0/2419709226 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.573+0000 7f508ed9d640 1 --2- 172.21.15.45:0/2419709226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50900d1d70 0x7f509006d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:14.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.574+0000 7f508dd9b640 1 -- 172.21.15.45:0/2419709226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50900d1d70 msgr2=0x7f509006d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.574+0000 7f508dd9b640 1 --2- 172.21.15.45:0/2419709226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50900d1d70 0x7f509006d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.574+0000 7f508dd9b640 1 -- 172.21.15.45:0/2419709226 >> v1:172.21.15.45:6789/0 conn(0x7f50901629f0 legacy=0x7f50900d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.574+0000 7f508dd9b640 1 -- 172.21.15.45:0/2419709226 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50900ce6e0 con 0x7f50900cfef0 2024-05-08T05:45:14.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.574+0000 7f508ed9d640 1 --2- 172.21.15.45:0/2419709226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50900d1d70 0x7f509006d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.574+0000 7f508dd9b640 1 --2- 172.21.15.45:0/2419709226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50900cfef0 0x7f50900d1830 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f50780097a0 tx=0x7f507802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64edc2ced7963ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.574+0000 7f508d59a640 1 -- 172.21.15.45:0/2419709226 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5078037070 con 0x7f50900cfef0 2024-05-08T05:45:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.574+0000 7f508d59a640 1 -- 172.21.15.45:0/2419709226 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5078032440 con 0x7f50900cfef0 2024-05-08T05:45:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.574+0000 7f508d59a640 1 -- 172.21.15.45:0/2419709226 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5078032810 con 0x7f50900cfef0 2024-05-08T05:45:14.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.575+0000 7f5094def640 1 -- 172.21.15.45:0/2419709226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50900cfef0 msgr2=0x7f50900d1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.575+0000 7f5094def640 1 --2- 172.21.15.45:0/2419709226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50900cfef0 0x7f50900d1830 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f50780097a0 tx=0x7f507802a180 comp rx=0 tx=0).stop 2024-05-08T05:45:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.575+0000 7f5094def640 1 -- 172.21.15.45:0/2419709226 shutdown_connections 2024-05-08T05:45:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.575+0000 7f5094def640 1 --2- 172.21.15.45:0/2419709226 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50900d1d70 0x7f509006d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.575+0000 7f5094def640 1 --2- 172.21.15.45:0/2419709226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50900cfef0 0x7f50900d1830 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.575+0000 7f5094def640 1 -- 172.21.15.45:0/2419709226 >> 172.21.15.45:0/2419709226 conn(0x7f50900d2d80 msgr2=0x7f50900d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.575+0000 7f5094def640 1 -- 172.21.15.45:0/2419709226 shutdown_connections 2024-05-08T05:45:14.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.575+0000 7f5094def640 1 -- 172.21.15.45:0/2419709226 wait complete. 2024-05-08T05:45:14.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.577+0000 7f5094def640 1 Processor -- start 2024-05-08T05:45:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.577+0000 7f5094def640 1 -- start start 2024-05-08T05:45:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.577+0000 7f5094def640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50900cfef0 0x7f509009d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.577+0000 7f5094def640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50900d1d70 0x7f509009ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.577+0000 7f5094def640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50901629f0 0x7f50900a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.577+0000 7f5094def640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f50900d2420 con 0x7f50900d1d70 2024-05-08T05:45:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.577+0000 7f5094def640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50900d22a0 con 0x7f50901629f0 2024-05-08T05:45:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.577+0000 7f5094def640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f509006dc00 con 0x7f50900cfef0 2024-05-08T05:45:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.578+0000 7f508dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50900d1d70 0x7f509009ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.578+0000 7f508dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50900d1d70 0x7f509009ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52686/0 (socket says 172.21.15.45:52686) 2024-05-08T05:45:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.578+0000 7f508dd9b640 1 -- 172.21.15.45:0/1312593025 learned_addr learned my addr 172.21.15.45:0/1312593025 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.578+0000 7f508e59c640 1 --2- 172.21.15.45:0/1312593025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50900cfef0 0x7f509009d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:14.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.578+0000 7f508dd9b640 1 -- 172.21.15.45:0/1312593025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50900cfef0 msgr2=0x7f509009d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.578+0000 7f508dd9b640 1 --2- 172.21.15.45:0/1312593025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50900cfef0 0x7f509009d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.578+0000 7f508ed9d640 1 --2- 172.21.15.45:0/1312593025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50901629f0 0x7f50900a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.578+0000 7f508dd9b640 1 -- 172.21.15.45:0/1312593025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50901629f0 msgr2=0x7f50900a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.578+0000 7f508dd9b640 1 --2- 172.21.15.45:0/1312593025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50901629f0 0x7f50900a0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.578+0000 7f508dd9b640 1 -- 172.21.15.45:0/1312593025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509014d310 con 0x7f50900d1d70 2024-05-08T05:45:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.578+0000 7f508dd9b640 1 --2- 172.21.15.45:0/1312593025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50900d1d70 0x7f509009ffc0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f50780098c0 tx=0x7f50780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.578+0000 7f5086ffd640 1 -- 172.21.15.45:0/1312593025 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5078037040 con 0x7f50900d1d70 2024-05-08T05:45:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.578+0000 7f5086ffd640 1 -- 172.21.15.45:0/1312593025 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5078030770 con 0x7f50900d1d70 2024-05-08T05:45:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.578+0000 7f5086ffd640 1 -- 172.21.15.45:0/1312593025 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5078030b40 con 0x7f50900d1d70 2024-05-08T05:45:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.578+0000 7f5094def640 1 -- 172.21.15.45:0/1312593025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f509014d5f0 con 0x7f50900d1d70 2024-05-08T05:45:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.578+0000 7f5094def640 1 -- 172.21.15.45:0/1312593025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f509014dbb0 con 0x7f50900d1d70 2024-05-08T05:45:14.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.579+0000 7f5094def640 1 -- 172.21.15.45:0/1312593025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50900693f0 con 0x7f50900d1d70 2024-05-08T05:45:14.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.580+0000 7f5086ffd640 1 -- 172.21.15.45:0/1312593025 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5078044020 con 0x7f50900d1d70 2024-05-08T05:45:14.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.581+0000 7f5086ffd640 1 --2- 172.21.15.45:0/1312593025 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50700762d0 0x7f5070078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:14.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.581+0000 7f5086ffd640 1 -- 172.21.15.45:0/1312593025 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5865+0+0 (secure 0 0 0) 0x7f50780b7b60 con 0x7f50900d1d70 2024-05-08T05:45:14.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.581+0000 7f508e59c640 1 --2- 172.21.15.45:0/1312593025 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50700762d0 0x7f5070078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:14.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.581+0000 7f508e59c640 1 --2- 172.21.15.45:0/1312593025 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50700762d0 0x7f5070078790 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f50800012e0 tx=0x7f5080009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:14.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.585+0000 7f5086ffd640 1 -- 172.21.15.45:0/1312593025 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5078032e20 con 0x7f50900d1d70 2024-05-08T05:45:14.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:14.775+0000 7f5094def640 1 -- 172.21.15.45:0/1312593025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f50900d2ad0 con 0x7f50900d1d70 2024-05-08T05:45:15.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.444+0000 7f5086ffd640 1 -- 172.21.15.45:0/1312593025 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v66) ==== 169+0+0 (secure 0 0 0) 0x7f50780814b0 con 0x7f50900d1d70 2024-05-08T05:45:15.444 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-05-08T05:45:15.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.447+0000 7f5094def640 1 -- 172.21.15.45:0/1312593025 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50700762d0 msgr2=0x7f5070078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:15.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.447+0000 7f5094def640 1 --2- 172.21.15.45:0/1312593025 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50700762d0 0x7f5070078790 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f50800012e0 tx=0x7f5080009040 comp rx=0 tx=0).stop 2024-05-08T05:45:15.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.447+0000 7f5094def640 1 -- 172.21.15.45:0/1312593025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50900d1d70 msgr2=0x7f509009ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:15.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.447+0000 7f5094def640 1 --2- 172.21.15.45:0/1312593025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50900d1d70 0x7f509009ffc0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f50780098c0 tx=0x7f50780046c0 comp rx=0 tx=0).stop 2024-05-08T05:45:15.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.448+0000 7f5094def640 1 -- 172.21.15.45:0/1312593025 shutdown_connections 2024-05-08T05:45:15.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.448+0000 7f5094def640 1 --2- 172.21.15.45:0/1312593025 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50700762d0 0x7f5070078790 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:15.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.448+0000 7f5094def640 1 --2- 172.21.15.45:0/1312593025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50901629f0 0x7f50900a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:15.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.448+0000 7f5094def640 1 --2- 172.21.15.45:0/1312593025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50900d1d70 0x7f509009ffc0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:15.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.448+0000 7f5094def640 1 --2- 172.21.15.45:0/1312593025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50900cfef0 0x7f509009d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:15.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.448+0000 7f5094def640 1 -- 172.21.15.45:0/1312593025 >> 172.21.15.45:0/1312593025 conn(0x7f50900d2d80 msgr2=0x7f5090070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:15.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.448+0000 7f5094def640 1 -- 172.21.15.45:0/1312593025 shutdown_connections 2024-05-08T05:45:15.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.448+0000 7f5094def640 1 -- 172.21.15.45:0/1312593025 wait complete. 2024-05-08T05:45:15.462 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T05:45:15.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.570+0000 7f348d1af640 1 Processor -- start 2024-05-08T05:45:15.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.570+0000 7f348d1af640 1 -- start start 2024-05-08T05:45:15.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.571+0000 7f348d1af640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f34881629a0 0x7f34880d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:15.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.571+0000 7f348d1af640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f34880d2dc0 0x7f34880d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:15.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.571+0000 7f348d1af640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34880d3330 con 0x7f34880c8ed0 2024-05-08T05:45:15.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.571+0000 7f348d1af640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f34880d2170 con 0x7f34880d2dc0 2024-05-08T05:45:15.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.571+0000 7f348d1af640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f34880d2450 con 0x7f34881629a0 2024-05-08T05:45:15.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.571+0000 7f348659c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f34880d2dc0 0x7f34880d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:15.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.571+0000 7f348659c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f34880d2dc0 0x7f34880d3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52702/0 (socket says 172.21.15.45:52702) 2024-05-08T05:45:15.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.571+0000 7f348659c640 1 -- 172.21.15.45:0/1061227075 learned_addr learned my addr 172.21.15.45:0/1061227075 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:15.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.571+0000 7f3486d9d640 1 --2- 172.21.15.45:0/1061227075 >> v2:172.21.15.63:3300/0 conn(0x7f34881629a0 0x7f34880d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:15.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.572+0000 7f348659c640 1 -- 172.21.15.45:0/1061227075 >> v2:172.21.15.63:3300/0 conn(0x7f34881629a0 msgr2=0x7f34880d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:15.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.572+0000 7f348659c640 1 --2- 172.21.15.45:0/1061227075 >> v2:172.21.15.63:3300/0 conn(0x7f34881629a0 0x7f34880d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:15.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.572+0000 7f348659c640 1 -- 172.21.15.45:0/1061227075 >> v1:172.21.15.63:6789/0 conn(0x7f34880c8ed0 legacy=0x7f34880c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:15.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.572+0000 7f348659c640 1 -- 172.21.15.45:0/1061227075 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34880d2730 con 0x7f34880d2dc0 2024-05-08T05:45:15.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.572+0000 7f348659c640 1 --2- 172.21.15.45:0/1061227075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34880d2dc0 0x7f34880d3b00 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f34880768a0 tx=0x7f347002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13ff184f515a273 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:15.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.572+0000 7f3485d9b640 1 -- 172.21.15.45:0/1061227075 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470037070 con 0x7f34880d2dc0 2024-05-08T05:45:15.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.573+0000 7f3485d9b640 1 -- 172.21.15.45:0/1061227075 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3470032440 con 0x7f34880d2dc0 2024-05-08T05:45:15.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.573+0000 7f3485d9b640 1 -- 172.21.15.45:0/1061227075 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470032810 con 0x7f34880d2dc0 2024-05-08T05:45:15.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.573+0000 7f348d1af640 1 -- 172.21.15.45:0/1061227075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34880d2dc0 msgr2=0x7f34880d3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:15.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.573+0000 7f348d1af640 1 --2- 172.21.15.45:0/1061227075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34880d2dc0 0x7f34880d3b00 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f34880768a0 tx=0x7f347002a150 comp rx=0 tx=0).stop 2024-05-08T05:45:15.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.573+0000 7f348d1af640 1 -- 172.21.15.45:0/1061227075 shutdown_connections 2024-05-08T05:45:15.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.573+0000 7f348d1af640 1 --2- 172.21.15.45:0/1061227075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34880d2dc0 0x7f34880d3b00 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:15.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.573+0000 7f348d1af640 1 --2- 172.21.15.45:0/1061227075 >> v2:172.21.15.63:3300/0 conn(0x7f34881629a0 0x7f34880d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:15.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.573+0000 7f348d1af640 1 -- 172.21.15.45:0/1061227075 >> 172.21.15.45:0/1061227075 conn(0x7f34880d4fe0 msgr2=0x7f34880d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:15.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.573+0000 7f348d1af640 1 -- 172.21.15.45:0/1061227075 shutdown_connections 2024-05-08T05:45:15.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.573+0000 7f348d1af640 1 -- 172.21.15.45:0/1061227075 wait complete. 2024-05-08T05:45:15.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.575+0000 7f348d1af640 1 Processor -- start 2024-05-08T05:45:15.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.575+0000 7f348d1af640 1 -- start start 2024-05-08T05:45:15.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.575+0000 7f348d1af640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34880c8ed0 0x7f34880b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:15.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.575+0000 7f348d1af640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34880d2dc0 0x7f34881556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:15.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.575+0000 7f348d1af640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34881629a0 0x7f34880b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:15.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.575+0000 7f348d1af640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f34880cf440 con 0x7f34880c8ed0 2024-05-08T05:45:15.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.575+0000 7f348d1af640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f34880cf2c0 con 0x7f34881629a0 2024-05-08T05:45:15.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.575+0000 7f348d1af640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f34880d0cb0 con 0x7f34880d2dc0 2024-05-08T05:45:15.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.575+0000 7f3486d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34880c8ed0 0x7f34880b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:15.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.575+0000 7f3486d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34880c8ed0 0x7f34880b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52706/0 (socket says 172.21.15.45:52706) 2024-05-08T05:45:15.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.575+0000 7f3486d9d640 1 -- 172.21.15.45:0/2993239512 learned_addr learned my addr 172.21.15.45:0/2993239512 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:15.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.576+0000 7f348659c640 1 --2- 172.21.15.45:0/2993239512 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34880d2dc0 0x7f34881556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:15.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.576+0000 7f348759e640 1 --2- 172.21.15.45:0/2993239512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34881629a0 0x7f34880b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:15.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.576+0000 7f3486d9d640 1 -- 172.21.15.45:0/2993239512 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34880d2dc0 msgr2=0x7f34881556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:15.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.576+0000 7f3486d9d640 1 --2- 172.21.15.45:0/2993239512 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34880d2dc0 0x7f34881556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:15.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.576+0000 7f3486d9d640 1 -- 172.21.15.45:0/2993239512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34881629a0 msgr2=0x7f34880b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:15.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.576+0000 7f3486d9d640 1 --2- 172.21.15.45:0/2993239512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34881629a0 0x7f34880b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:15.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.576+0000 7f3486d9d640 1 -- 172.21.15.45:0/2993239512 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3488155be0 con 0x7f34880c8ed0 2024-05-08T05:45:15.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.576+0000 7f3486d9d640 1 --2- 172.21.15.45:0/2993239512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34880c8ed0 0x7f34880b1bf0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3478007e60 tx=0x7f347800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:15.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.576+0000 7f347f7fe640 1 -- 172.21.15.45:0/2993239512 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478014620 con 0x7f34880c8ed0 2024-05-08T05:45:15.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.576+0000 7f347f7fe640 1 -- 172.21.15.45:0/2993239512 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3478019070 con 0x7f34880c8ed0 2024-05-08T05:45:15.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.576+0000 7f348d1af640 1 -- 172.21.15.45:0/2993239512 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34880b5c50 con 0x7f34880c8ed0 2024-05-08T05:45:15.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.576+0000 7f347f7fe640 1 -- 172.21.15.45:0/2993239512 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478014a20 con 0x7f34880c8ed0 2024-05-08T05:45:15.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.576+0000 7f348d1af640 1 -- 172.21.15.45:0/2993239512 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34880b50d0 con 0x7f34880c8ed0 2024-05-08T05:45:15.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.577+0000 7f348d1af640 1 -- 172.21.15.45:0/2993239512 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34880693f0 con 0x7f34880c8ed0 2024-05-08T05:45:15.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.578+0000 7f347f7fe640 1 -- 172.21.15.45:0/2993239512 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3478014bc0 con 0x7f34880c8ed0 2024-05-08T05:45:15.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.578+0000 7f347f7fe640 1 --2- 172.21.15.45:0/2993239512 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f34680762d0 0x7f3468078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:15.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.579+0000 7f347f7fe640 1 -- 172.21.15.45:0/2993239512 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7f3478097d80 con 0x7f34880c8ed0 2024-05-08T05:45:15.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.579+0000 7f348659c640 1 --2- 172.21.15.45:0/2993239512 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f34680762d0 0x7f3468078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:15.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.579+0000 7f348659c640 1 --2- 172.21.15.45:0/2993239512 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f34680762d0 0x7f3468078790 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3470034000 tx=0x7f347002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:15.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.583+0000 7f347f7fe640 1 -- 172.21.15.45:0/2993239512 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3488155be0 con 0x7f34880c8ed0 2024-05-08T05:45:15.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.777+0000 7f348d1af640 1 -- 172.21.15.45:0/2993239512 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f34880b7100 con 0x7f34880c8ed0 2024-05-08T05:45:15.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.780+0000 7f347f7fe640 1 -- 172.21.15.45:0/2993239512 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f3478061850 con 0x7f34880c8ed0 2024-05-08T05:45:15.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.782+0000 7f348d1af640 1 -- 172.21.15.45:0/2993239512 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f34680762d0 msgr2=0x7f3468078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:15.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.782+0000 7f348d1af640 1 --2- 172.21.15.45:0/2993239512 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f34680762d0 0x7f3468078790 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3470034000 tx=0x7f347002e000 comp rx=0 tx=0).stop 2024-05-08T05:45:15.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.782+0000 7f348d1af640 1 -- 172.21.15.45:0/2993239512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34880c8ed0 msgr2=0x7f34880b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:15.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.782+0000 7f348d1af640 1 --2- 172.21.15.45:0/2993239512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34880c8ed0 0x7f34880b1bf0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3478007e60 tx=0x7f347800e4b0 comp rx=0 tx=0).stop 2024-05-08T05:45:15.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.782+0000 7f348d1af640 1 -- 172.21.15.45:0/2993239512 shutdown_connections 2024-05-08T05:45:15.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.782+0000 7f348d1af640 1 --2- 172.21.15.45:0/2993239512 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f34680762d0 0x7f3468078790 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:15.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.782+0000 7f348d1af640 1 --2- 172.21.15.45:0/2993239512 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f34880d2dc0 0x7f34881556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.782+0000 7f348d1af640 1 --2- 172.21.15.45:0/2993239512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34881629a0 0x7f34880b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.783+0000 7f348d1af640 1 --2- 172.21.15.45:0/2993239512 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f34880c8ed0 0x7f34880b1bf0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.783+0000 7f348d1af640 1 -- 172.21.15.45:0/2993239512 >> 172.21.15.45:0/2993239512 conn(0x7f34880d4fe0 msgr2=0x7f34881612d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.783+0000 7f348d1af640 1 -- 172.21.15.45:0/2993239512 shutdown_connections 2024-05-08T05:45:15.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.783+0000 7f348d1af640 1 -- 172.21.15.45:0/2993239512 wait complete. 2024-05-08T05:45:15.794 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-05-08T05:45:15.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.898+0000 7fc5aa5af640 1 Processor -- start 2024-05-08T05:45:15.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.898+0000 7fc5aa5af640 1 -- start start 2024-05-08T05:45:15.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.899+0000 7fc5aa5af640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5a415f420 0x7fc5a40d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:15.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.899+0000 7fc5aa5af640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc5a40d2080 0x7fc5a40d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.899+0000 7fc5aa5af640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5a41630b0 con 0x7fc5a40d3ca0 2024-05-08T05:45:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.899+0000 7fc5aa5af640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc5a40d2550 con 0x7fc5a40d2080 2024-05-08T05:45:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.899+0000 7fc5aa5af640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a40d2c80 con 0x7fc5a415f420 2024-05-08T05:45:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.899+0000 7fc5a3fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5a415f420 0x7fc5a40d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.899+0000 7fc5a8b4c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc5a40d2080 0x7fc5a40d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.899+0000 7fc59bfff640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fc5a40d3ca0 0x7fc5a40d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:42154/0 (socket says 172.21.15.45:42154) 2024-05-08T05:45:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.899+0000 7fc59bfff640 1 -- 172.21.15.45:0/451119199 learned_addr learned my addr 172.21.15.45:0/451119199 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.899+0000 7fc5a37fe640 1 -- 172.21.15.45:0/451119199 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 97778080 0 0) 0x7fc58c001110 con 0x7fc5a40d3ca0 2024-05-08T05:45:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.899+0000 7fc5a37fe640 1 -- 172.21.15.45:0/451119199 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc590003520 con 0x7fc5a40d3ca0 2024-05-08T05:45:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.899+0000 7fc5a8b4c640 1 -- 172.21.15.45:0/451119199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5a415f420 msgr2=0x7fc5a40d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.899+0000 7fc5a8b4c640 1 --2- 172.21.15.45:0/451119199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5a415f420 0x7fc5a40d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.899+0000 7fc5a8b4c640 1 -- 172.21.15.45:0/451119199 >> v1:172.21.15.63:6789/0 conn(0x7fc5a40d3ca0 legacy=0x7fc5a40d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:15.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.899+0000 7fc5a8b4c640 1 -- 172.21.15.45:0/451119199 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a40d2f60 con 0x7fc5a40d2080 2024-05-08T05:45:15.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.900+0000 7fc5a8b4c640 1 --2- 172.21.15.45:0/451119199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5a40d2080 0x7fc5a40d4cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009d90 tx=0x7fc59c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61cdac9a04c85f15 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:15.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.900+0000 7fc5a37fe640 1 -- 172.21.15.45:0/451119199 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c007740 con 0x7fc5a40d2080 2024-05-08T05:45:15.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.900+0000 7fc5a37fe640 1 -- 172.21.15.45:0/451119199 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc59c0078e0 con 0x7fc5a40d2080 2024-05-08T05:45:15.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.900+0000 7fc5a37fe640 1 -- 172.21.15.45:0/451119199 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c007c90 con 0x7fc5a40d2080 2024-05-08T05:45:15.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.901+0000 7fc5aa5af640 1 -- 172.21.15.45:0/451119199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5a40d2080 msgr2=0x7fc5a40d4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:15.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.901+0000 7fc5aa5af640 1 --2- 172.21.15.45:0/451119199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5a40d2080 0x7fc5a40d4cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009d90 tx=0x7fc59c02a660 comp rx=0 tx=0).stop 2024-05-08T05:45:15.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.901+0000 7fc5aa5af640 1 -- 172.21.15.45:0/451119199 shutdown_connections 2024-05-08T05:45:15.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.901+0000 7fc5aa5af640 1 --2- 172.21.15.45:0/451119199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5a40d2080 0x7fc5a40d4cc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:15.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.901+0000 7fc5aa5af640 1 --2- 172.21.15.45:0/451119199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5a415f420 0x7fc5a40d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:15.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.901+0000 7fc5aa5af640 1 -- 172.21.15.45:0/451119199 >> 172.21.15.45:0/451119199 conn(0x7fc5a408eac0 msgr2=0x7fc5a409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:15.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.901+0000 7fc5aa5af640 1 -- 172.21.15.45:0/451119199 shutdown_connections 2024-05-08T05:45:15.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.901+0000 7fc5aa5af640 1 -- 172.21.15.45:0/451119199 wait complete. 2024-05-08T05:45:15.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.902+0000 7fc5aa5af640 1 Processor -- start 2024-05-08T05:45:15.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.902+0000 7fc5aa5af640 1 -- start start 2024-05-08T05:45:15.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.903+0000 7fc5aa5af640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5a40d2080 0x7fc5a40c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:15.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.903+0000 7fc5aa5af640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a40ba620 0x7fc5a4155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:15.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.903+0000 7fc5aa5af640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5a40cb0f0 0x7fc5a40cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:15.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.903+0000 7fc5aa5af640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a40d5200 con 0x7fc5a40ba620 2024-05-08T05:45:15.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.903+0000 7fc5aa5af640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a40d2830 con 0x7fc5a40cb0f0 2024-05-08T05:45:15.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.903+0000 7fc5aa5af640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a40d5380 con 0x7fc5a40d2080 2024-05-08T05:45:15.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.903+0000 7fc59bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a40ba620 0x7fc5a4155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:15.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.903+0000 7fc59bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a40ba620 0x7fc5a4155060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52714/0 (socket says 172.21.15.45:52714) 2024-05-08T05:45:15.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.903+0000 7fc59bfff640 1 -- 172.21.15.45:0/2046884755 learned_addr learned my addr 172.21.15.45:0/2046884755 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:15.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.903+0000 7fc5a3fff640 1 --2- 172.21.15.45:0/2046884755 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5a40d2080 0x7fc5a40c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:15.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.903+0000 7fc5a8b4c640 1 --2- 172.21.15.45:0/2046884755 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5a40cb0f0 0x7fc5a40cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:15.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.903+0000 7fc59bfff640 1 -- 172.21.15.45:0/2046884755 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5a40d2080 msgr2=0x7fc5a40c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:15.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.903+0000 7fc59bfff640 1 --2- 172.21.15.45:0/2046884755 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5a40d2080 0x7fc5a40c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:15.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.903+0000 7fc59bfff640 1 -- 172.21.15.45:0/2046884755 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5a40cb0f0 msgr2=0x7fc5a40cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:15.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.903+0000 7fc59bfff640 1 --2- 172.21.15.45:0/2046884755 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5a40cb0f0 0x7fc5a40cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:15.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.903+0000 7fc59bfff640 1 -- 172.21.15.45:0/2046884755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a40c9b80 con 0x7fc5a40ba620 2024-05-08T05:45:15.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.904+0000 7fc59bfff640 1 --2- 172.21.15.45:0/2046884755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a40ba620 0x7fc5a4155060 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00c730 tx=0x7fc58c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:15.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.904+0000 7fc5a17fa640 1 -- 172.21.15.45:0/2046884755 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c017070 con 0x7fc5a40ba620 2024-05-08T05:45:15.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.904+0000 7fc5a17fa640 1 -- 172.21.15.45:0/2046884755 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc58c005ab0 con 0x7fc5a40ba620 2024-05-08T05:45:15.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.904+0000 7fc5aa5af640 1 -- 172.21.15.45:0/2046884755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a40ca610 con 0x7fc5a40ba620 2024-05-08T05:45:15.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.904+0000 7fc5a17fa640 1 -- 172.21.15.45:0/2046884755 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c005e60 con 0x7fc5a40ba620 2024-05-08T05:45:15.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.904+0000 7fc5aa5af640 1 -- 172.21.15.45:0/2046884755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a40cabd0 con 0x7fc5a40ba620 2024-05-08T05:45:15.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.905+0000 7fc5aa5af640 1 -- 172.21.15.45:0/2046884755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a40cadd0 con 0x7fc5a40ba620 2024-05-08T05:45:15.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.906+0000 7fc5a17fa640 1 -- 172.21.15.45:0/2046884755 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc58c01a070 con 0x7fc5a40ba620 2024-05-08T05:45:15.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.906+0000 7fc5a17fa640 1 --2- 172.21.15.45:0/2046884755 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc5840762d0 0x7fc584078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:15.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.906+0000 7fc5a17fa640 1 -- 172.21.15.45:0/2046884755 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7fc58c096640 con 0x7fc5a40ba620 2024-05-08T05:45:15.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.906+0000 7fc5a3fff640 1 --2- 172.21.15.45:0/2046884755 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc5840762d0 0x7fc584078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:15.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.907+0000 7fc5a3fff640 1 --2- 172.21.15.45:0/2046884755 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc5840762d0 0x7fc584078790 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc594009aa0 tx=0x7fc594009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:15.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:15.910+0000 7fc5a17fa640 1 -- 172.21.15.45:0/2046884755 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc5a40cadd0 con 0x7fc5a40ba620 2024-05-08T05:45:16.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.181+0000 7fc5aa5af640 1 -- 172.21.15.45:0/2046884755 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7fc5a40693f0 con 0x7fc5a40ba620 2024-05-08T05:45:16.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.445+0000 7fc5a17fa640 1 -- 172.21.15.45:0/2046884755 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v193) ==== 169+0+0 (secure 0 0 0) 0x7fc5a41630b0 con 0x7fc5a40ba620 2024-05-08T05:45:16.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.448+0000 7fc5aa5af640 1 -- 172.21.15.45:0/2046884755 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc5840762d0 msgr2=0x7fc584078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:16.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.448+0000 7fc5aa5af640 1 --2- 172.21.15.45:0/2046884755 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc5840762d0 0x7fc584078790 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc594009aa0 tx=0x7fc594009190 comp rx=0 tx=0).stop 2024-05-08T05:45:16.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.448+0000 7fc5aa5af640 1 -- 172.21.15.45:0/2046884755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a40ba620 msgr2=0x7fc5a4155060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:16.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.448+0000 7fc5aa5af640 1 --2- 172.21.15.45:0/2046884755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a40ba620 0x7fc5a4155060 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00c730 tx=0x7fc58c00cbf0 comp rx=0 tx=0).stop 2024-05-08T05:45:16.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.448+0000 7fc5aa5af640 1 -- 172.21.15.45:0/2046884755 shutdown_connections 2024-05-08T05:45:16.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.448+0000 7fc5aa5af640 1 --2- 172.21.15.45:0/2046884755 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc5840762d0 0x7fc584078790 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:16.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.448+0000 7fc5aa5af640 1 --2- 172.21.15.45:0/2046884755 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5a40cb0f0 0x7fc5a40cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:16.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.448+0000 7fc5aa5af640 1 --2- 172.21.15.45:0/2046884755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc5a40ba620 0x7fc5a4155060 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:16.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.448+0000 7fc5aa5af640 1 --2- 172.21.15.45:0/2046884755 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc5a40d2080 0x7fc5a40c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:16.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.448+0000 7fc5aa5af640 1 -- 172.21.15.45:0/2046884755 >> 172.21.15.45:0/2046884755 conn(0x7fc5a408eac0 msgr2=0x7fc5a40d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:16.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.448+0000 7fc5aa5af640 1 -- 172.21.15.45:0/2046884755 shutdown_connections 2024-05-08T05:45:16.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.448+0000 7fc5aa5af640 1 -- 172.21.15.45:0/2046884755 wait complete. 2024-05-08T05:45:16.459 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-08T05:45:16.460 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-08T05:45:16.460 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-08T05:45:16.460 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:45:16.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.564+0000 7f85851c1640 1 Processor -- start 2024-05-08T05:45:16.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.564+0000 7f85851c1640 1 -- start start 2024-05-08T05:45:16.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.565+0000 7f85851c1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8580162f50 0x7f85801633e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:16.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.565+0000 7f85851c1640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f85800cf790 0x7f85800cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:16.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.565+0000 7f85851c1640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85800d0500 con 0x7f85800b21f0 2024-05-08T05:45:16.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.565+0000 7f85851c1640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f858006d810 con 0x7f8580162f50 2024-05-08T05:45:16.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.565+0000 7f85851c1640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f858006e0b0 con 0x7f85800cf790 2024-05-08T05:45:16.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.565+0000 7f857e59c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8580162f50 0x7f85801633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:16.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.565+0000 7f857e59c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8580162f50 0x7f85801633e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52724/0 (socket says 172.21.15.45:52724) 2024-05-08T05:45:16.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.565+0000 7f857e59c640 1 -- 172.21.15.45:0/555002005 learned_addr learned my addr 172.21.15.45:0/555002005 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:16.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.566+0000 7f857f59e640 1 --2- 172.21.15.45:0/555002005 >> v2:172.21.15.63:3300/0 conn(0x7f85800cf790 0x7f85800cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:16.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.566+0000 7f857e59c640 1 -- 172.21.15.45:0/555002005 >> v2:172.21.15.63:3300/0 conn(0x7f85800cf790 msgr2=0x7f85800cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:16.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.566+0000 7f857e59c640 1 --2- 172.21.15.45:0/555002005 >> v2:172.21.15.63:3300/0 conn(0x7f85800cf790 0x7f85800cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:16.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.566+0000 7f857e59c640 1 -- 172.21.15.45:0/555002005 >> v1:172.21.15.63:6789/0 conn(0x7f85800b21f0 legacy=0x7f858006e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:16.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.566+0000 7f857e59c640 1 -- 172.21.15.45:0/555002005 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858006da60 con 0x7f8580162f50 2024-05-08T05:45:16.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.566+0000 7f857e59c640 1 --2- 172.21.15.45:0/555002005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8580162f50 0x7f85801633e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f85800cf470 tx=0x7f857402a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67309148ae63ca8c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:16.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.567+0000 7f857dd9b640 1 -- 172.21.15.45:0/555002005 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574037070 con 0x7f8580162f50 2024-05-08T05:45:16.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.567+0000 7f857dd9b640 1 -- 172.21.15.45:0/555002005 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8574032440 con 0x7f8580162f50 2024-05-08T05:45:16.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.567+0000 7f857dd9b640 1 -- 172.21.15.45:0/555002005 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574032810 con 0x7f8580162f50 2024-05-08T05:45:16.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.567+0000 7f85851c1640 1 -- 172.21.15.45:0/555002005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8580162f50 msgr2=0x7f85801633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:16.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.567+0000 7f85851c1640 1 --2- 172.21.15.45:0/555002005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8580162f50 0x7f85801633e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f85800cf470 tx=0x7f857402a150 comp rx=0 tx=0).stop 2024-05-08T05:45:16.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.567+0000 7f85851c1640 1 -- 172.21.15.45:0/555002005 shutdown_connections 2024-05-08T05:45:16.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.567+0000 7f85851c1640 1 --2- 172.21.15.45:0/555002005 >> v2:172.21.15.63:3300/0 conn(0x7f85800cf790 0x7f85800cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:16.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.567+0000 7f85851c1640 1 --2- 172.21.15.45:0/555002005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8580162f50 0x7f85801633e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:16.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.567+0000 7f85851c1640 1 -- 172.21.15.45:0/555002005 >> 172.21.15.45:0/555002005 conn(0x7f858009ba40 msgr2=0x7f858009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:16.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.567+0000 7f85851c1640 1 -- 172.21.15.45:0/555002005 shutdown_connections 2024-05-08T05:45:16.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.567+0000 7f85851c1640 1 -- 172.21.15.45:0/555002005 wait complete. 2024-05-08T05:45:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.569+0000 7f85851c1640 1 Processor -- start 2024-05-08T05:45:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.569+0000 7f85851c1640 1 -- start start 2024-05-08T05:45:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.569+0000 7f85851c1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85800b21f0 0x7f8580151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.569+0000 7f85851c1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85800cf790 0x7f85800b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.569+0000 7f85851c1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8580162f50 0x7f85801f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.569+0000 7f85851c1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f85800d0140 con 0x7f85800cf790 2024-05-08T05:45:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.569+0000 7f85851c1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f85800b25d0 con 0x7f85800b21f0 2024-05-08T05:45:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.569+0000 7f85851c1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f85800d02c0 con 0x7f8580162f50 2024-05-08T05:45:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.570+0000 7f857e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85800cf790 0x7f85800b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.570+0000 7f857e59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85800cf790 0x7f85800b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52738/0 (socket says 172.21.15.45:52738) 2024-05-08T05:45:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.570+0000 7f857e59c640 1 -- 172.21.15.45:0/13886799 learned_addr learned my addr 172.21.15.45:0/13886799 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.570+0000 7f857f59e640 1 --2- 172.21.15.45:0/13886799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8580162f50 0x7f85801f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:16.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.570+0000 7f857ed9d640 1 --2- 172.21.15.45:0/13886799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85800b21f0 0x7f8580151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:16.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.570+0000 7f857e59c640 1 -- 172.21.15.45:0/13886799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8580162f50 msgr2=0x7f85801f4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:16.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.570+0000 7f857e59c640 1 --2- 172.21.15.45:0/13886799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8580162f50 0x7f85801f4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:16.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.570+0000 7f857e59c640 1 -- 172.21.15.45:0/13886799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85800b21f0 msgr2=0x7f8580151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:16.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.570+0000 7f857e59c640 1 --2- 172.21.15.45:0/13886799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85800b21f0 0x7f8580151180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:16.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.570+0000 7f857e59c640 1 -- 172.21.15.45:0/13886799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85801f5910 con 0x7f85800cf790 2024-05-08T05:45:16.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.570+0000 7f857e59c640 1 --2- 172.21.15.45:0/13886799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85800cf790 0x7f85800b4140 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f857402a190 tx=0x7f85740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:16.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.570+0000 7f85677fe640 1 -- 172.21.15.45:0/13886799 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574041020 con 0x7f85800cf790 2024-05-08T05:45:16.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.570+0000 7f85677fe640 1 -- 172.21.15.45:0/13886799 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85740306e0 con 0x7f85800cf790 2024-05-08T05:45:16.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.570+0000 7f85851c1640 1 -- 172.21.15.45:0/13886799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85801f5bf0 con 0x7f85800cf790 2024-05-08T05:45:16.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.570+0000 7f85677fe640 1 -- 172.21.15.45:0/13886799 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574037040 con 0x7f85800cf790 2024-05-08T05:45:16.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.570+0000 7f85851c1640 1 -- 172.21.15.45:0/13886799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85800b3c20 con 0x7f85800cf790 2024-05-08T05:45:16.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.571+0000 7f85851c1640 1 -- 172.21.15.45:0/13886799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8580069b00 con 0x7f85800cf790 2024-05-08T05:45:16.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.572+0000 7f85677fe640 1 -- 172.21.15.45:0/13886799 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8574044020 con 0x7f85800cf790 2024-05-08T05:45:16.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.572+0000 7f85677fe640 1 --2- 172.21.15.45:0/13886799 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f856c076300 0x7f856c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:16.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.572+0000 7f85677fe640 1 -- 172.21.15.45:0/13886799 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7f85740b7ce0 con 0x7f85800cf790 2024-05-08T05:45:16.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.573+0000 7f857ed9d640 1 --2- 172.21.15.45:0/13886799 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f856c076300 0x7f856c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:16.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.573+0000 7f857ed9d640 1 --2- 172.21.15.45:0/13886799 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f856c076300 0x7f856c0787c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f858015bf50 tx=0x7f8570009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:16.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.577+0000 7f85677fe640 1 -- 172.21.15.45:0/13886799 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f857404b030 con 0x7f85800cf790 2024-05-08T05:45:16.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.771+0000 7f85851c1640 1 -- 172.21.15.45:0/13886799 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8580070a30 con 0x7f85800cf790 2024-05-08T05:45:16.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.772+0000 7f85677fe640 1 -- 172.21.15.45:0/13886799 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+13246 (secure 0 0 0) 0x7f85740817a0 con 0x7f85800cf790 2024-05-08T05:45:16.772 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:16.772 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":66,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:45:15.441168+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/1461822016":"2024-05-09T05:45:12.433233+0000","172.21.15.45:6833/1461822016":"2024-05-09T05:45:12.433233+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:45:16.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.774+0000 7f85851c1640 1 -- 172.21.15.45:0/13886799 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f856c076300 msgr2=0x7f856c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:16.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.774+0000 7f85851c1640 1 --2- 172.21.15.45:0/13886799 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f856c076300 0x7f856c0787c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f858015bf50 tx=0x7f8570009040 comp rx=0 tx=0).stop 2024-05-08T05:45:16.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.774+0000 7f85851c1640 1 -- 172.21.15.45:0/13886799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85800cf790 msgr2=0x7f85800b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:16.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.774+0000 7f85851c1640 1 --2- 172.21.15.45:0/13886799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85800cf790 0x7f85800b4140 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f857402a190 tx=0x7f85740046c0 comp rx=0 tx=0).stop 2024-05-08T05:45:16.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.774+0000 7f85851c1640 1 -- 172.21.15.45:0/13886799 shutdown_connections 2024-05-08T05:45:16.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.774+0000 7f85851c1640 1 --2- 172.21.15.45:0/13886799 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f856c076300 0x7f856c0787c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:16.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.774+0000 7f85851c1640 1 --2- 172.21.15.45:0/13886799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8580162f50 0x7f85801f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:16.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.774+0000 7f85851c1640 1 --2- 172.21.15.45:0/13886799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85800cf790 0x7f85800b4140 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:16.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.774+0000 7f85851c1640 1 --2- 172.21.15.45:0/13886799 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85800b21f0 0x7f8580151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:16.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.774+0000 7f85851c1640 1 -- 172.21.15.45:0/13886799 >> 172.21.15.45:0/13886799 conn(0x7f858009ba40 msgr2=0x7f858009c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:16.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.775+0000 7f85851c1640 1 -- 172.21.15.45:0/13886799 shutdown_connections 2024-05-08T05:45:16.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.775+0000 7f85851c1640 1 -- 172.21.15.45:0/13886799 wait complete. 2024-05-08T05:45:16.786 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T05:41:18.847604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T05:45:16.786 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T05:45:16.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.892+0000 7f984a409640 1 Processor -- start 2024-05-08T05:45:16.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.893+0000 7f984a409640 1 -- start start 2024-05-08T05:45:16.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.893+0000 7f984a409640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f984415f4e0 0x7f98440cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:16.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.893+0000 7f984a409640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98440d3aa0 0x7f98440d4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:16.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.893+0000 7f984a409640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98440d0320 con 0x7f98440d3f00 2024-05-08T05:45:16.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.893+0000 7f984a409640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f98440d0600 con 0x7f98440d3aa0 2024-05-08T05:45:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.893+0000 7f984a409640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f98440d08e0 con 0x7f984415f4e0 2024-05-08T05:45:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.893+0000 7f98437fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f98440d3aa0 0x7f98440d4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.893+0000 7f98489a6640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f98440d3f00 0x7f98440b50d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57620/0 (socket says 172.21.15.45:57620) 2024-05-08T05:45:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.893+0000 7f98489a6640 1 -- 172.21.15.45:0/545212703 learned_addr learned my addr 172.21.15.45:0/545212703 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.894+0000 7f9843fff640 1 --2- 172.21.15.45:0/545212703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f984415f4e0 0x7f98440cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.894+0000 7f98437fe640 1 -- 172.21.15.45:0/545212703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f984415f4e0 msgr2=0x7f98440cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.894+0000 7f98437fe640 1 --2- 172.21.15.45:0/545212703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f984415f4e0 0x7f98440cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.894+0000 7f98437fe640 1 -- 172.21.15.45:0/545212703 >> v1:172.21.15.45:6789/0 conn(0x7f98440d3f00 legacy=0x7f98440b50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.894+0000 7f9842ffd640 1 -- 172.21.15.45:0/545212703 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2039023447 0 0) 0x7f98440d0320 con 0x7f98440d3f00 2024-05-08T05:45:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.894+0000 7f98437fe640 1 -- 172.21.15.45:0/545212703 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98440d0bc0 con 0x7f98440d3aa0 2024-05-08T05:45:16.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.895+0000 7f98437fe640 1 --2- 172.21.15.45:0/545212703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98440d3aa0 0x7f98440d4fa0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f98380097a0 tx=0x7f983802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38513e73daaa0e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:16.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.895+0000 7f9842ffd640 1 -- 172.21.15.45:0/545212703 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838037070 con 0x7f98440d3aa0 2024-05-08T05:45:16.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.895+0000 7f9842ffd640 1 -- 172.21.15.45:0/545212703 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9838032440 con 0x7f98440d3aa0 2024-05-08T05:45:16.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.895+0000 7f9842ffd640 1 -- 172.21.15.45:0/545212703 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838032810 con 0x7f98440d3aa0 2024-05-08T05:45:16.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.895+0000 7f984a409640 1 -- 172.21.15.45:0/545212703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98440d3aa0 msgr2=0x7f98440d4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:16.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.895+0000 7f984a409640 1 --2- 172.21.15.45:0/545212703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98440d3aa0 0x7f98440d4fa0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f98380097a0 tx=0x7f983802a180 comp rx=0 tx=0).stop 2024-05-08T05:45:16.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.895+0000 7f984a409640 1 -- 172.21.15.45:0/545212703 shutdown_connections 2024-05-08T05:45:16.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.895+0000 7f984a409640 1 --2- 172.21.15.45:0/545212703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98440d3aa0 0x7f98440d4fa0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:16.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.895+0000 7f984a409640 1 --2- 172.21.15.45:0/545212703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f984415f4e0 0x7f98440cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:16.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.895+0000 7f984a409640 1 -- 172.21.15.45:0/545212703 >> 172.21.15.45:0/545212703 conn(0x7f984408eac0 msgr2=0x7f984409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:16.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.895+0000 7f984a409640 1 -- 172.21.15.45:0/545212703 shutdown_connections 2024-05-08T05:45:16.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.895+0000 7f984a409640 1 -- 172.21.15.45:0/545212703 wait complete. 2024-05-08T05:45:16.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.896+0000 7f984a409640 1 Processor -- start 2024-05-08T05:45:16.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.897+0000 7f984a409640 1 -- start start 2024-05-08T05:45:16.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.897+0000 7f984a409640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98440d3aa0 0x7f98440b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:16.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.897+0000 7f984a409640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98440d3f00 0x7f98440b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:16.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.897+0000 7f984a409640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f984415f4e0 0x7f9844161230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:16.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.897+0000 7f984a409640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f98440d23d0 con 0x7f984415f4e0 2024-05-08T05:45:16.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.897+0000 7f984a409640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f984415f8c0 con 0x7f98440d3aa0 2024-05-08T05:45:16.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.897+0000 7f984a409640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f98440d2550 con 0x7f98440d3f00 2024-05-08T05:45:16.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.897+0000 7f98489a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f984415f4e0 0x7f9844161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:16.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.898+0000 7f98489a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f984415f4e0 0x7f9844161230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52748/0 (socket says 172.21.15.45:52748) 2024-05-08T05:45:16.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.898+0000 7f98489a6640 1 -- 172.21.15.45:0/2871429064 learned_addr learned my addr 172.21.15.45:0/2871429064 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:16.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.898+0000 7f9843fff640 1 --2- 172.21.15.45:0/2871429064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98440d3aa0 0x7f98440b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:16.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.898+0000 7f98437fe640 1 --2- 172.21.15.45:0/2871429064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98440d3f00 0x7f98440b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:16.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.898+0000 7f98489a6640 1 -- 172.21.15.45:0/2871429064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98440d3f00 msgr2=0x7f98440b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:16.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.898+0000 7f98489a6640 1 --2- 172.21.15.45:0/2871429064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98440d3f00 0x7f98440b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:16.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.898+0000 7f98489a6640 1 -- 172.21.15.45:0/2871429064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98440d3aa0 msgr2=0x7f98440b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:16.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.898+0000 7f98489a6640 1 --2- 172.21.15.45:0/2871429064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98440d3aa0 0x7f98440b32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:16.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.898+0000 7f98489a6640 1 -- 172.21.15.45:0/2871429064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9844161f60 con 0x7f984415f4e0 2024-05-08T05:45:16.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.898+0000 7f9843fff640 1 --2- 172.21.15.45:0/2871429064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98440d3aa0 0x7f98440b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:16.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.898+0000 7f98489a6640 1 --2- 172.21.15.45:0/2871429064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f984415f4e0 0x7f9844161230 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f983c00c770 tx=0x7f983c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:16.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.898+0000 7f9840ff9640 1 -- 172.21.15.45:0/2871429064 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c018070 con 0x7f984415f4e0 2024-05-08T05:45:16.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.898+0000 7f984a409640 1 -- 172.21.15.45:0/2871429064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9844162240 con 0x7f984415f4e0 2024-05-08T05:45:16.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.898+0000 7f9840ff9640 1 -- 172.21.15.45:0/2871429064 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f983c00a5e0 con 0x7f984415f4e0 2024-05-08T05:45:16.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.898+0000 7f9840ff9640 1 -- 172.21.15.45:0/2871429064 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c00a9d0 con 0x7f984415f4e0 2024-05-08T05:45:16.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.898+0000 7f984a409640 1 -- 172.21.15.45:0/2871429064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9844162800 con 0x7f984415f4e0 2024-05-08T05:45:16.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.899+0000 7f984a409640 1 -- 172.21.15.45:0/2871429064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98440693f0 con 0x7f984415f4e0 2024-05-08T05:45:16.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.900+0000 7f9840ff9640 1 -- 172.21.15.45:0/2871429064 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f983c01b070 con 0x7f984415f4e0 2024-05-08T05:45:16.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.901+0000 7f9840ff9640 1 --2- 172.21.15.45:0/2871429064 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9830076300 0x7f98300787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:16.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.901+0000 7f9840ff9640 1 -- 172.21.15.45:0/2871429064 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7f983c096a00 con 0x7f984415f4e0 2024-05-08T05:45:16.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.901+0000 7f9843fff640 1 --2- 172.21.15.45:0/2871429064 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9830076300 0x7f98300787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:16.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.901+0000 7f9843fff640 1 --2- 172.21.15.45:0/2871429064 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9830076300 0x7f98300787c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9834006fd0 tx=0x7f9834008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:16.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:16.905+0000 7f9840ff9640 1 -- 172.21.15.45:0/2871429064 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f983c0604d0 con 0x7f984415f4e0 2024-05-08T05:45:17.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.096+0000 7f984a409640 1 -- 172.21.15.45:0/2871429064 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9844075840 con 0x7f984415f4e0 2024-05-08T05:45:17.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.097+0000 7f9840ff9640 1 -- 172.21.15.45:0/2871429064 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7f983c05fdd0 con 0x7f984415f4e0 2024-05-08T05:45:17.097 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-05-08T05:45:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.099+0000 7f984a409640 1 -- 172.21.15.45:0/2871429064 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9830076300 msgr2=0x7f98300787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.099+0000 7f984a409640 1 --2- 172.21.15.45:0/2871429064 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9830076300 0x7f98300787c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9834006fd0 tx=0x7f9834008040 comp rx=0 tx=0).stop 2024-05-08T05:45:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.099+0000 7f984a409640 1 -- 172.21.15.45:0/2871429064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f984415f4e0 msgr2=0x7f9844161230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.099+0000 7f984a409640 1 --2- 172.21.15.45:0/2871429064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f984415f4e0 0x7f9844161230 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f983c00c770 tx=0x7f983c00cc30 comp rx=0 tx=0).stop 2024-05-08T05:45:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.099+0000 7f984a409640 1 -- 172.21.15.45:0/2871429064 shutdown_connections 2024-05-08T05:45:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.099+0000 7f984a409640 1 --2- 172.21.15.45:0/2871429064 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9830076300 0x7f98300787c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.099+0000 7f984a409640 1 --2- 172.21.15.45:0/2871429064 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f98440d3f00 0x7f98440b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.099+0000 7f984a409640 1 --2- 172.21.15.45:0/2871429064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f984415f4e0 0x7f9844161230 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.099+0000 7f984a409640 1 --2- 172.21.15.45:0/2871429064 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98440d3aa0 0x7f98440b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.099+0000 7f984a409640 1 -- 172.21.15.45:0/2871429064 >> 172.21.15.45:0/2871429064 conn(0x7f984408eac0 msgr2=0x7f984406cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.099+0000 7f984a409640 1 -- 172.21.15.45:0/2871429064 shutdown_connections 2024-05-08T05:45:17.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.099+0000 7f984a409640 1 -- 172.21.15.45:0/2871429064 wait complete. 2024-05-08T05:45:17.111 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-05-08T05:45:17.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.214+0000 7f2ff5c72640 1 Processor -- start 2024-05-08T05:45:17.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.214+0000 7f2ff5c72640 1 -- start start 2024-05-08T05:45:17.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.215+0000 7f2ff5c72640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2ff0070d30 0x7f2ff00d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:17.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.215+0000 7f2ff5c72640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ff00d1300 con 0x7f2ff0071110 2024-05-08T05:45:17.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.215+0000 7f2ff5c72640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ff00d15e0 con 0x7f2ff0163920 2024-05-08T05:45:17.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.215+0000 7f2ff5c72640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2ff00d10d0 con 0x7f2ff0070d30 2024-05-08T05:45:17.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.215+0000 7f2feffff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2ff0071110 0x7f2ff006d4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57632/0 (socket says 172.21.15.45:57632) 2024-05-08T05:45:17.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.215+0000 7f2feffff640 1 -- 172.21.15.45:0/1473581217 learned_addr learned my addr 172.21.15.45:0/1473581217 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:17.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.215+0000 7f2fee7fc640 1 -- 172.21.15.45:0/1473581217 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2031247108 0 0) 0x7f2ff00d1300 con 0x7f2ff0071110 2024-05-08T05:45:17.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.215+0000 7f2fee7fc640 1 -- 172.21.15.45:0/1473581217 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fd8003520 con 0x7f2ff0071110 2024-05-08T05:45:17.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2fef7fe640 1 --2- 172.21.15.45:0/1473581217 >> v2:172.21.15.63:3300/0 conn(0x7f2ff0070d30 0x7f2ff00d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:17.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2fee7fc640 1 -- 172.21.15.45:0/1473581217 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2926263224 0 0) 0x7f2fd8003520 con 0x7f2ff0071110 2024-05-08T05:45:17.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2fee7fc640 1 -- 172.21.15.45:0/1473581217 >> v2:172.21.15.63:3300/0 conn(0x7f2ff0070d30 msgr2=0x7f2ff00d20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:17.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2fee7fc640 1 --2- 172.21.15.45:0/1473581217 >> v2:172.21.15.63:3300/0 conn(0x7f2ff0070d30 0x7f2ff00d20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:17.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2fee7fc640 1 -- 172.21.15.45:0/1473581217 >> v1:172.21.15.63:6789/0 conn(0x7f2ff0163920 legacy=0x7f2ff00703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:17.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2fee7fc640 1 -- 172.21.15.45:0/1473581217 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff00d18c0 con 0x7f2ff0071110 2024-05-08T05:45:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2fee7fc640 1 -- 172.21.15.45:0/1473581217 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f2fe8001a70 con 0x7f2ff0071110 2024-05-08T05:45:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2fee7fc640 1 -- 172.21.15.45:0/1473581217 >> v1:172.21.15.45:6789/0 conn(0x7f2ff0071110 legacy=0x7f2ff006d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2fee7fc640 1 --2- 172.21.15.45:0/1473581217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fd8003a50 0x7f2fd8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2fee7fc640 1 --2- 172.21.15.45:0/1473581217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fd8004800 0x7f2fd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2fee7fc640 1 --2- 172.21.15.45:0/1473581217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fd80052e0 0x7f2fd800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2fee7fc640 1 -- 172.21.15.45:0/1473581217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd800c180 con 0x7f2fd8003a50 2024-05-08T05:45:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2fee7fc640 1 -- 172.21.15.45:0/1473581217 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd800c330 con 0x7f2fd80052e0 2024-05-08T05:45:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2fee7fc640 1 -- 172.21.15.45:0/1473581217 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd800c530 con 0x7f2fd8004800 2024-05-08T05:45:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2fef7fe640 1 --2- 172.21.15.45:0/1473581217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fd8003a50 0x7f2fd8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2feffff640 1 --2- 172.21.15.45:0/1473581217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fd80052e0 0x7f2fd800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:17.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2feeffd640 1 --2- 172.21.15.45:0/1473581217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fd8004800 0x7f2fd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2fef7fe640 1 -- 172.21.15.45:0/1473581217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fd8004800 msgr2=0x7f2fd8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2fef7fe640 1 --2- 172.21.15.45:0/1473581217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fd8004800 0x7f2fd8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2fef7fe640 1 -- 172.21.15.45:0/1473581217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fd80052e0 msgr2=0x7f2fd800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2fef7fe640 1 --2- 172.21.15.45:0/1473581217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fd80052e0 0x7f2fd800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2fef7fe640 1 -- 172.21.15.45:0/1473581217 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd8003520 con 0x7f2fd8003a50 2024-05-08T05:45:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2feffff640 1 --2- 172.21.15.45:0/1473581217 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fd80052e0 0x7f2fd800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.216+0000 7f2feffff640 1 -- 172.21.15.45:0/1473581217 reap_dead start 2024-05-08T05:45:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.217+0000 7f2fef7fe640 1 --2- 172.21.15.45:0/1473581217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fd8003a50 0x7f2fd8003e70 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000ba00 tx=0x7f2fd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ab51db966957ee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.217+0000 7f2fee7fc640 1 -- 172.21.15.45:0/1473581217 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0030070 con 0x7f2fd8003a50 2024-05-08T05:45:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.217+0000 7f2fee7fc640 1 -- 172.21.15.45:0/1473581217 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe002adb0 con 0x7f2fd8003a50 2024-05-08T05:45:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.217+0000 7f2fee7fc640 1 -- 172.21.15.45:0/1473581217 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe000b600 con 0x7f2fd8003a50 2024-05-08T05:45:17.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.218+0000 7f2ff5c72640 1 -- 172.21.15.45:0/1473581217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fd8003a50 msgr2=0x7f2fd8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:17.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.218+0000 7f2ff5c72640 1 --2- 172.21.15.45:0/1473581217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fd8003a50 0x7f2fd8003e70 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000ba00 tx=0x7f2fd8005830 comp rx=0 tx=0).stop 2024-05-08T05:45:17.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.218+0000 7f2ff5c72640 1 -- 172.21.15.45:0/1473581217 shutdown_connections 2024-05-08T05:45:17.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.218+0000 7f2ff5c72640 1 --2- 172.21.15.45:0/1473581217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2fd8003a50 0x7f2fd8003e70 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:17.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.218+0000 7f2ff5c72640 1 -- 172.21.15.45:0/1473581217 >> 172.21.15.45:0/1473581217 conn(0x7f2ff00d2e90 msgr2=0x7f2ff00d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:17.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.218+0000 7f2ff5c72640 1 -- 172.21.15.45:0/1473581217 shutdown_connections 2024-05-08T05:45:17.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.218+0000 7f2ff5c72640 1 -- 172.21.15.45:0/1473581217 wait complete. 2024-05-08T05:45:17.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.219+0000 7f2ff5c72640 1 Processor -- start 2024-05-08T05:45:17.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.219+0000 7f2ff5c72640 1 -- start start 2024-05-08T05:45:17.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.220+0000 7f2ff5c72640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fd8003a50 0x7f2ff0157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:17.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.220+0000 7f2ff5c72640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff00c69c0 0x7f2ff00b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:17.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.220+0000 7f2ff5c72640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ff00c6e20 0x7f2ff00b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:17.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.220+0000 7f2ff5c72640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff0070b80 con 0x7f2ff00c69c0 2024-05-08T05:45:17.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.220+0000 7f2ff5c72640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff0163db0 con 0x7f2ff00c6e20 2024-05-08T05:45:17.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.220+0000 7f2ff5c72640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2ff00d1c30 con 0x7f2fd8003a50 2024-05-08T05:45:17.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.220+0000 7f2feeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff00c69c0 0x7f2ff00b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:17.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.220+0000 7f2feeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff00c69c0 0x7f2ff00b0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52758/0 (socket says 172.21.15.45:52758) 2024-05-08T05:45:17.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.220+0000 7f2feeffd640 1 -- 172.21.15.45:0/1623239688 learned_addr learned my addr 172.21.15.45:0/1623239688 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:17.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.220+0000 7f2fef7fe640 1 --2- 172.21.15.45:0/1623239688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fd8003a50 0x7f2ff0157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:17.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.220+0000 7f2feffff640 1 --2- 172.21.15.45:0/1623239688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ff00c6e20 0x7f2ff00b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:17.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.220+0000 7f2feeffd640 1 -- 172.21.15.45:0/1623239688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fd8003a50 msgr2=0x7f2ff0157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:17.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.220+0000 7f2feeffd640 1 --2- 172.21.15.45:0/1623239688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fd8003a50 0x7f2ff0157040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:17.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.220+0000 7f2feeffd640 1 -- 172.21.15.45:0/1623239688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ff00c6e20 msgr2=0x7f2ff00b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:17.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.220+0000 7f2feeffd640 1 --2- 172.21.15.45:0/1623239688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ff00c6e20 0x7f2ff00b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:17.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.220+0000 7f2feeffd640 1 -- 172.21.15.45:0/1623239688 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff0157ab0 con 0x7f2ff00c69c0 2024-05-08T05:45:17.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.221+0000 7f2feeffd640 1 --2- 172.21.15.45:0/1623239688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff00c69c0 0x7f2ff00b0390 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400e570 tx=0x7f2fe400ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:17.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.221+0000 7f2fd7fff640 1 -- 172.21.15.45:0/1623239688 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe40052e0 con 0x7f2ff00c69c0 2024-05-08T05:45:17.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.221+0000 7f2fd7fff640 1 -- 172.21.15.45:0/1623239688 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe4018070 con 0x7f2ff00c69c0 2024-05-08T05:45:17.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.221+0000 7f2fd7fff640 1 -- 172.21.15.45:0/1623239688 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe40037f0 con 0x7f2ff00c69c0 2024-05-08T05:45:17.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.221+0000 7f2ff5c72640 1 -- 172.21.15.45:0/1623239688 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff0157580 con 0x7f2ff00c69c0 2024-05-08T05:45:17.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.221+0000 7f2ff5c72640 1 -- 172.21.15.45:0/1623239688 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff00c9cb0 con 0x7f2ff00c69c0 2024-05-08T05:45:17.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.222+0000 7f2ff5c72640 1 -- 172.21.15.45:0/1623239688 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff00693f0 con 0x7f2ff00c69c0 2024-05-08T05:45:17.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.223+0000 7f2fd7fff640 1 -- 172.21.15.45:0/1623239688 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2fe4003990 con 0x7f2ff00c69c0 2024-05-08T05:45:17.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.223+0000 7f2fd7fff640 1 --2- 172.21.15.45:0/1623239688 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2fdc0762d0 0x7f2fdc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:17.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.223+0000 7f2fd7fff640 1 -- 172.21.15.45:0/1623239688 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7f2fe4097130 con 0x7f2ff00c69c0 2024-05-08T05:45:17.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.223+0000 7f2fef7fe640 1 --2- 172.21.15.45:0/1623239688 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2fdc0762d0 0x7f2fdc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:17.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.224+0000 7f2fef7fe640 1 --2- 172.21.15.45:0/1623239688 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2fdc0762d0 0x7f2fdc078790 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8004780 tx=0x7f2fe0030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:17.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.228+0000 7f2fd7fff640 1 -- 172.21.15.45:0/1623239688 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2ff0157ab0 con 0x7f2ff00c69c0 2024-05-08T05:45:17.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:17.495+0000 7f2ff5c72640 1 -- 172.21.15.45:0/1623239688 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f2ff006d140 con 0x7f2ff00c69c0 2024-05-08T05:45:18.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.451+0000 7f2fd7fff640 1 -- 172.21.15.45:0/1623239688 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v195) ==== 169+0+0 (secure 0 0 0) 0x7f2fe4060c00 con 0x7f2ff00c69c0 2024-05-08T05:45:18.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.454+0000 7f2ff5c72640 1 -- 172.21.15.45:0/1623239688 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2fdc0762d0 msgr2=0x7f2fdc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.454+0000 7f2ff5c72640 1 --2- 172.21.15.45:0/1623239688 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2fdc0762d0 0x7f2fdc078790 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8004780 tx=0x7f2fe0030000 comp rx=0 tx=0).stop 2024-05-08T05:45:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.454+0000 7f2ff5c72640 1 -- 172.21.15.45:0/1623239688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff00c69c0 msgr2=0x7f2ff00b0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.454+0000 7f2ff5c72640 1 --2- 172.21.15.45:0/1623239688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff00c69c0 0x7f2ff00b0390 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400e570 tx=0x7f2fe400ea30 comp rx=0 tx=0).stop 2024-05-08T05:45:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.454+0000 7f2ff5c72640 1 -- 172.21.15.45:0/1623239688 shutdown_connections 2024-05-08T05:45:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.454+0000 7f2ff5c72640 1 --2- 172.21.15.45:0/1623239688 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2fdc0762d0 0x7f2fdc078790 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.454+0000 7f2ff5c72640 1 --2- 172.21.15.45:0/1623239688 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ff00c6e20 0x7f2ff00b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.454+0000 7f2ff5c72640 1 --2- 172.21.15.45:0/1623239688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff00c69c0 0x7f2ff00b0390 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.454+0000 7f2ff5c72640 1 --2- 172.21.15.45:0/1623239688 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2fd8003a50 0x7f2ff0157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.454+0000 7f2ff5c72640 1 -- 172.21.15.45:0/1623239688 >> 172.21.15.45:0/1623239688 conn(0x7f2ff00d2e90 msgr2=0x7f2ff0161c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.454+0000 7f2ff5c72640 1 -- 172.21.15.45:0/1623239688 shutdown_connections 2024-05-08T05:45:18.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.454+0000 7f2ff5c72640 1 -- 172.21.15.45:0/1623239688 wait complete. 2024-05-08T05:45:18.468 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T05:45:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.582+0000 7f885b146640 1 Processor -- start 2024-05-08T05:45:18.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.582+0000 7f885b146640 1 -- start start 2024-05-08T05:45:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.582+0000 7f885b146640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f88540b2490 0x7f8854070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.582+0000 7f885b146640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8854070d20 0x7f88540711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.582+0000 7f885b146640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f885406e110 con 0x7f88541629b0 2024-05-08T05:45:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.582+0000 7f885b146640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f885406dec0 con 0x7f88540b2490 2024-05-08T05:45:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.582+0000 7f885b146640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f885406e3f0 con 0x7f8854070d20 2024-05-08T05:45:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.583+0000 7f8853fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f88540b2490 0x7f8854070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.583+0000 7f8858ee2640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f88541629b0 0x7f885406eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57640/0 (socket says 172.21.15.45:57640) 2024-05-08T05:45:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.583+0000 7f8858ee2640 1 -- 172.21.15.45:0/1658241189 learned_addr learned my addr 172.21.15.45:0/1658241189 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:18.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.583+0000 7f88596e3640 1 --2- 172.21.15.45:0/1658241189 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8854070d20 0x7f88540711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.583+0000 7f8853fff640 1 -- 172.21.15.45:0/1658241189 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8854070d20 msgr2=0x7f88540711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.583+0000 7f8853fff640 1 --2- 172.21.15.45:0/1658241189 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8854070d20 0x7f88540711d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.583+0000 7f8853fff640 1 -- 172.21.15.45:0/1658241189 >> v1:172.21.15.45:6789/0 conn(0x7f88541629b0 legacy=0x7f885406eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.583+0000 7f8853fff640 1 -- 172.21.15.45:0/1658241189 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885406e6d0 con 0x7f88540b2490 2024-05-08T05:45:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.583+0000 7f88537fe640 1 -- 172.21.15.45:0/1658241189 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 787916111 0 0) 0x7f885406e110 con 0x7f88541629b0 2024-05-08T05:45:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.584+0000 7f8853fff640 1 --2- 172.21.15.45:0/1658241189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88540b2490 0x7f8854070330 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f883c0097a0 tx=0x7f883c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c2af591604004e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.584+0000 7f88537fe640 1 -- 172.21.15.45:0/1658241189 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c037070 con 0x7f88540b2490 2024-05-08T05:45:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.584+0000 7f88537fe640 1 -- 172.21.15.45:0/1658241189 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f883c032440 con 0x7f88540b2490 2024-05-08T05:45:18.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.584+0000 7f88537fe640 1 -- 172.21.15.45:0/1658241189 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c032810 con 0x7f88540b2490 2024-05-08T05:45:18.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.584+0000 7f885b146640 1 -- 172.21.15.45:0/1658241189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88540b2490 msgr2=0x7f8854070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:18.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.584+0000 7f885b146640 1 --2- 172.21.15.45:0/1658241189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88540b2490 0x7f8854070330 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f883c0097a0 tx=0x7f883c02a180 comp rx=0 tx=0).stop 2024-05-08T05:45:18.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.584+0000 7f885b146640 1 -- 172.21.15.45:0/1658241189 shutdown_connections 2024-05-08T05:45:18.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.584+0000 7f885b146640 1 --2- 172.21.15.45:0/1658241189 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8854070d20 0x7f88540711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:18.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.584+0000 7f885b146640 1 --2- 172.21.15.45:0/1658241189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88540b2490 0x7f8854070330 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:18.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.584+0000 7f885b146640 1 -- 172.21.15.45:0/1658241189 >> 172.21.15.45:0/1658241189 conn(0x7f88540b49d0 msgr2=0x7f88540b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:18.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.584+0000 7f885b146640 1 -- 172.21.15.45:0/1658241189 shutdown_connections 2024-05-08T05:45:18.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.585+0000 7f885b146640 1 -- 172.21.15.45:0/1658241189 wait complete. 2024-05-08T05:45:18.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.586+0000 7f885b146640 1 Processor -- start 2024-05-08T05:45:18.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.586+0000 7f885b146640 1 -- start start 2024-05-08T05:45:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.587+0000 7f885b146640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8854070d20 0x7f88540b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.587+0000 7f885b146640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88540b2490 0x7f88540b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.587+0000 7f885b146640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88541629b0 0x7f88540cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.587+0000 7f885b146640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f88540709f0 con 0x7f88540b2490 2024-05-08T05:45:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.587+0000 7f885b146640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8854070870 con 0x7f8854070d20 2024-05-08T05:45:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.587+0000 7f885b146640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f88540afcb0 con 0x7f88541629b0 2024-05-08T05:45:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.587+0000 7f8853fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88540b2490 0x7f88540b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.587+0000 7f8853fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88540b2490 0x7f88540b7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52784/0 (socket says 172.21.15.45:52784) 2024-05-08T05:45:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.587+0000 7f8853fff640 1 -- 172.21.15.45:0/645590899 learned_addr learned my addr 172.21.15.45:0/645590899 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.587+0000 7f8858ee2640 1 --2- 172.21.15.45:0/645590899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8854070d20 0x7f88540b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.587+0000 7f88596e3640 1 --2- 172.21.15.45:0/645590899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88541629b0 0x7f88540cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.587+0000 7f8853fff640 1 -- 172.21.15.45:0/645590899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88541629b0 msgr2=0x7f88540cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.587+0000 7f8853fff640 1 --2- 172.21.15.45:0/645590899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88541629b0 0x7f88540cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:18.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.587+0000 7f8853fff640 1 -- 172.21.15.45:0/645590899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8854070d20 msgr2=0x7f88540b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.587+0000 7f8853fff640 1 --2- 172.21.15.45:0/645590899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8854070d20 0x7f88540b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.587+0000 7f8853fff640 1 -- 172.21.15.45:0/645590899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88540b86b0 con 0x7f88540b2490 2024-05-08T05:45:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.587+0000 7f88596e3640 1 --2- 172.21.15.45:0/645590899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88541629b0 0x7f88540cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.587+0000 7f8853fff640 1 --2- 172.21.15.45:0/645590899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88540b2490 0x7f88540b7bd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f883c035040 tx=0x7f883c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.587+0000 7f88517fa640 1 -- 172.21.15.45:0/645590899 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c037050 con 0x7f88540b2490 2024-05-08T05:45:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.588+0000 7f88517fa640 1 -- 172.21.15.45:0/645590899 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f883c038040 con 0x7f88540b2490 2024-05-08T05:45:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.588+0000 7f885b146640 1 -- 172.21.15.45:0/645590899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88540b8990 con 0x7f88540b2490 2024-05-08T05:45:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.588+0000 7f88517fa640 1 -- 172.21.15.45:0/645590899 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c004490 con 0x7f88540b2490 2024-05-08T05:45:18.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.588+0000 7f885b146640 1 -- 172.21.15.45:0/645590899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88540c7380 con 0x7f88540b2490 2024-05-08T05:45:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.589+0000 7f885b146640 1 -- 172.21.15.45:0/645590899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88540693f0 con 0x7f88540b2490 2024-05-08T05:45:18.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.589+0000 7f88517fa640 1 -- 172.21.15.45:0/645590899 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f883c030980 con 0x7f88540b2490 2024-05-08T05:45:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.590+0000 7f88517fa640 1 --2- 172.21.15.45:0/645590899 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8834076300 0x7f88340787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.590+0000 7f88517fa640 1 -- 172.21.15.45:0/645590899 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7f883c0b8030 con 0x7f88540b2490 2024-05-08T05:45:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.590+0000 7f8858ee2640 1 --2- 172.21.15.45:0/645590899 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8834076300 0x7f88340787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:18.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.590+0000 7f8858ee2640 1 --2- 172.21.15.45:0/645590899 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8834076300 0x7f88340787c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8844007fd0 tx=0x7f8844007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:18.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.594+0000 7f88517fa640 1 -- 172.21.15.45:0/645590899 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f883c081a60 con 0x7f88540b2490 2024-05-08T05:45:18.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.788+0000 7f885b146640 1 -- 172.21.15.45:0/645590899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f88540b2080 con 0x7f88540b2490 2024-05-08T05:45:18.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.792+0000 7f88517fa640 1 -- 172.21.15.45:0/645590899 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f883c081360 con 0x7f88540b2490 2024-05-08T05:45:18.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.794+0000 7f885b146640 1 -- 172.21.15.45:0/645590899 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8834076300 msgr2=0x7f88340787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:18.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.794+0000 7f885b146640 1 --2- 172.21.15.45:0/645590899 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8834076300 0x7f88340787c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8844007fd0 tx=0x7f8844007de0 comp rx=0 tx=0).stop 2024-05-08T05:45:18.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.794+0000 7f885b146640 1 -- 172.21.15.45:0/645590899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88540b2490 msgr2=0x7f88540b7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:18.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.794+0000 7f885b146640 1 --2- 172.21.15.45:0/645590899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88540b2490 0x7f88540b7bd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f883c035040 tx=0x7f883c0046c0 comp rx=0 tx=0).stop 2024-05-08T05:45:18.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.794+0000 7f885b146640 1 -- 172.21.15.45:0/645590899 shutdown_connections 2024-05-08T05:45:18.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.794+0000 7f885b146640 1 --2- 172.21.15.45:0/645590899 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8834076300 0x7f88340787c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:18.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.794+0000 7f885b146640 1 --2- 172.21.15.45:0/645590899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f88541629b0 0x7f88540cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:18.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.794+0000 7f885b146640 1 --2- 172.21.15.45:0/645590899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f88540b2490 0x7f88540b7bd0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:18.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.794+0000 7f885b146640 1 --2- 172.21.15.45:0/645590899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8854070d20 0x7f88540b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:18.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.794+0000 7f885b146640 1 -- 172.21.15.45:0/645590899 >> 172.21.15.45:0/645590899 conn(0x7f88540b49d0 msgr2=0x7f88540711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:18.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.794+0000 7f885b146640 1 -- 172.21.15.45:0/645590899 shutdown_connections 2024-05-08T05:45:18.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:18.794+0000 7f885b146640 1 -- 172.21.15.45:0/645590899 wait complete. 2024-05-08T05:45:18.805 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T05:45:18.806 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:45:18.850 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:45:18.899 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T05:45:18.899 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:45:18.899 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:45:18.917 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:45:18.918 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:45:18.918 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:45:18.938 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:45:18.939 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:45:18.939 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:45:18.978 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:45:18.978 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:45:18.978 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:45:18.997 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:45:18.997 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T05:45:18.998 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:45:19.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.121+0000 7f6aa9e64640 1 Processor -- start 2024-05-08T05:45:19.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.121+0000 7f6aa9e64640 1 -- start start 2024-05-08T05:45:19.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.122+0000 7f6aa9e64640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6aa406c3a0 0x7f6aa415d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:19.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.122+0000 7f6aa9e64640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6aa4161490 0x7f6aa415eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:19.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.122+0000 7f6aa9e64640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aa406fa90 con 0x7f6aa406f5e0 2024-05-08T05:45:19.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.122+0000 7f6aa9e64640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f6aa4160d20 con 0x7f6aa4161490 2024-05-08T05:45:19.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.122+0000 7f6aa9e64640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6aa4161000 con 0x7f6aa406c3a0 2024-05-08T05:45:19.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.122+0000 7f6aa3fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6aa4161490 0x7f6aa415eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:19.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.122+0000 7f6aa3fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6aa4161490 0x7f6aa415eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52790/0 (socket says 172.21.15.45:52790) 2024-05-08T05:45:19.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.122+0000 7f6aa3fff640 1 -- 172.21.15.45:0/175447384 learned_addr learned my addr 172.21.15.45:0/175447384 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:19.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.122+0000 7f6aa37fe640 1 --2- 172.21.15.45:0/175447384 >> v2:172.21.15.63:3300/0 conn(0x7f6aa406c3a0 0x7f6aa415d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:19.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.122+0000 7f6aa3fff640 1 -- 172.21.15.45:0/175447384 >> v2:172.21.15.63:3300/0 conn(0x7f6aa406c3a0 msgr2=0x7f6aa415d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:19.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.122+0000 7f6aa3fff640 1 --2- 172.21.15.45:0/175447384 >> v2:172.21.15.63:3300/0 conn(0x7f6aa406c3a0 0x7f6aa415d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:19.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.122+0000 7f6aa3fff640 1 -- 172.21.15.45:0/175447384 >> v1:172.21.15.63:6789/0 conn(0x7f6aa406f5e0 legacy=0x7f6aa406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:19.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.123+0000 7f6aa3fff640 1 -- 172.21.15.45:0/175447384 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa415dec0 con 0x7f6aa4161490 2024-05-08T05:45:19.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.123+0000 7f6aa3fff640 1 --2- 172.21.15.45:0/175447384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa4161490 0x7f6aa415eb00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6aa406d0d0 tx=0x7f6a9c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6a3b4b670f96d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:19.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.123+0000 7f6aa27fc640 1 -- 172.21.15.45:0/175447384 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c036070 con 0x7f6aa4161490 2024-05-08T05:45:19.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.123+0000 7f6aa27fc640 1 -- 172.21.15.45:0/175447384 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a9c031600 con 0x7f6aa4161490 2024-05-08T05:45:19.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.123+0000 7f6aa27fc640 1 -- 172.21.15.45:0/175447384 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c0319d0 con 0x7f6aa4161490 2024-05-08T05:45:19.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.124+0000 7f6aa9e64640 1 -- 172.21.15.45:0/175447384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa4161490 msgr2=0x7f6aa415eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:19.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.124+0000 7f6aa9e64640 1 --2- 172.21.15.45:0/175447384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa4161490 0x7f6aa415eb00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6aa406d0d0 tx=0x7f6a9c02a730 comp rx=0 tx=0).stop 2024-05-08T05:45:19.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.124+0000 7f6aa9e64640 1 -- 172.21.15.45:0/175447384 shutdown_connections 2024-05-08T05:45:19.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.124+0000 7f6aa9e64640 1 --2- 172.21.15.45:0/175447384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa4161490 0x7f6aa415eb00 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:19.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.124+0000 7f6aa9e64640 1 --2- 172.21.15.45:0/175447384 >> v2:172.21.15.63:3300/0 conn(0x7f6aa406c3a0 0x7f6aa415d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:19.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.124+0000 7f6aa9e64640 1 -- 172.21.15.45:0/175447384 >> 172.21.15.45:0/175447384 conn(0x7f6aa40b39a0 msgr2=0x7f6aa40b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:19.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.124+0000 7f6aa9e64640 1 -- 172.21.15.45:0/175447384 shutdown_connections 2024-05-08T05:45:19.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.124+0000 7f6aa9e64640 1 -- 172.21.15.45:0/175447384 wait complete. 2024-05-08T05:45:19.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.125+0000 7f6aa9e64640 1 Processor -- start 2024-05-08T05:45:19.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.125+0000 7f6aa9e64640 1 -- start start 2024-05-08T05:45:19.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.126+0000 7f6aa9e64640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa406c3a0 0x7f6aa413f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:19.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.126+0000 7f6aa9e64640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6aa406f5e0 0x7f6aa413c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:19.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.126+0000 7f6aa9e64640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa4161490 0x7f6aa415c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:19.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.126+0000 7f6aa9e64640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa40d1390 con 0x7f6aa406c3a0 2024-05-08T05:45:19.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.126+0000 7f6aa9e64640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa415e480 con 0x7f6aa4161490 2024-05-08T05:45:19.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.126+0000 7f6aa9e64640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa40d1510 con 0x7f6aa406f5e0 2024-05-08T05:45:19.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.126+0000 7f6aa37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa406c3a0 0x7f6aa413f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:19.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.126+0000 7f6aa37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa406c3a0 0x7f6aa413f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52796/0 (socket says 172.21.15.45:52796) 2024-05-08T05:45:19.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.126+0000 7f6aa37fe640 1 -- 172.21.15.45:0/1249757716 learned_addr learned my addr 172.21.15.45:0/1249757716 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:19.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.126+0000 7f6aa2ffd640 1 --2- 172.21.15.45:0/1249757716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6aa406f5e0 0x7f6aa413c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:19.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.126+0000 7f6aa3fff640 1 --2- 172.21.15.45:0/1249757716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa4161490 0x7f6aa415c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:19.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.127+0000 7f6aa37fe640 1 -- 172.21.15.45:0/1249757716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6aa406f5e0 msgr2=0x7f6aa413c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:19.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.127+0000 7f6aa37fe640 1 --2- 172.21.15.45:0/1249757716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6aa406f5e0 0x7f6aa413c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:19.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.127+0000 7f6aa37fe640 1 -- 172.21.15.45:0/1249757716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa4161490 msgr2=0x7f6aa415c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:19.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.127+0000 7f6aa37fe640 1 --2- 172.21.15.45:0/1249757716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa4161490 0x7f6aa415c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:19.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.127+0000 7f6aa37fe640 1 -- 172.21.15.45:0/1249757716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa413fe80 con 0x7f6aa406c3a0 2024-05-08T05:45:19.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.127+0000 7f6aa2ffd640 1 --2- 172.21.15.45:0/1249757716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6aa406f5e0 0x7f6aa413c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:19.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.127+0000 7f6aa3fff640 1 --2- 172.21.15.45:0/1249757716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa4161490 0x7f6aa415c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:19.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.127+0000 7f6aa37fe640 1 --2- 172.21.15.45:0/1249757716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa406c3a0 0x7f6aa413f940 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400d800 tx=0x7f6a9400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:19.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.127+0000 7f6a9bfff640 1 -- 172.21.15.45:0/1249757716 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9400b6c0 con 0x7f6aa406c3a0 2024-05-08T05:45:19.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.127+0000 7f6a9bfff640 1 -- 172.21.15.45:0/1249757716 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a94019070 con 0x7f6aa406c3a0 2024-05-08T05:45:19.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.127+0000 7f6aa9e64640 1 -- 172.21.15.45:0/1249757716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa4140150 con 0x7f6aa406c3a0 2024-05-08T05:45:19.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.127+0000 7f6a9bfff640 1 -- 172.21.15.45:0/1249757716 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9400ba80 con 0x7f6aa406c3a0 2024-05-08T05:45:19.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.127+0000 7f6aa9e64640 1 -- 172.21.15.45:0/1249757716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa41409f0 con 0x7f6aa406c3a0 2024-05-08T05:45:19.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.128+0000 7f6aa9e64640 1 -- 172.21.15.45:0/1249757716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aa4069b00 con 0x7f6aa406c3a0 2024-05-08T05:45:19.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.129+0000 7f6a9bfff640 1 -- 172.21.15.45:0/1249757716 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6a9400bc20 con 0x7f6aa406c3a0 2024-05-08T05:45:19.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.129+0000 7f6a9bfff640 1 --2- 172.21.15.45:0/1249757716 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6a840762d0 0x7f6a84078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:19.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.129+0000 7f6a9bfff640 1 -- 172.21.15.45:0/1249757716 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7f6a94097a20 con 0x7f6aa406c3a0 2024-05-08T05:45:19.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.129+0000 7f6aa2ffd640 1 --2- 172.21.15.45:0/1249757716 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6a840762d0 0x7f6a84078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:19.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.130+0000 7f6aa2ffd640 1 --2- 172.21.15.45:0/1249757716 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6a840762d0 0x7f6a84078790 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c007fd0 tx=0x7f6a8c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:19.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.134+0000 7f6a9bfff640 1 -- 172.21.15.45:0/1249757716 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6a94061460 con 0x7f6aa406c3a0 2024-05-08T05:45:19.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.363+0000 7f6aa9e64640 1 -- 172.21.15.45:0/1249757716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6aa40aee50 con 0x7f6aa406c3a0 2024-05-08T05:45:19.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.364+0000 7f6a9bfff640 1 -- 172.21.15.45:0/1249757716 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+3863 (secure 0 0 0) 0x7f6a94060d60 con 0x7f6aa406c3a0 2024-05-08T05:45:19.364 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:19.364 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":38,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5093,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/162765657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":162765657},{"type":"v1","addr":"172.21.15.63:6836","nonce":162765657}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5356,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1788513331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1788513331},{"type":"v1","addr":"172.21.15.45:6834","nonce":1788513331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1173167340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1173167340},{"type":"v1","addr":"172.21.15.63:6837","nonce":1173167340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6093,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/123681128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":123681128},{"type":"v1","addr":"172.21.15.45:6835","nonce":123681128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[]} 2024-05-08T05:45:19.365 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 38 2024-05-08T05:45:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.366+0000 7f6aa9e64640 1 -- 172.21.15.45:0/1249757716 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6a840762d0 msgr2=0x7f6a84078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:19.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.366+0000 7f6aa9e64640 1 --2- 172.21.15.45:0/1249757716 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6a840762d0 0x7f6a84078790 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c007fd0 tx=0x7f6a8c009040 comp rx=0 tx=0).stop 2024-05-08T05:45:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.366+0000 7f6aa9e64640 1 -- 172.21.15.45:0/1249757716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa406c3a0 msgr2=0x7f6aa413f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.366+0000 7f6aa9e64640 1 --2- 172.21.15.45:0/1249757716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa406c3a0 0x7f6aa413f940 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400d800 tx=0x7f6a9400dcc0 comp rx=0 tx=0).stop 2024-05-08T05:45:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.366+0000 7f6aa9e64640 1 -- 172.21.15.45:0/1249757716 shutdown_connections 2024-05-08T05:45:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.366+0000 7f6aa9e64640 1 --2- 172.21.15.45:0/1249757716 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6a840762d0 0x7f6a84078790 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.366+0000 7f6aa9e64640 1 --2- 172.21.15.45:0/1249757716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6aa406f5e0 0x7f6aa413c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.366+0000 7f6aa9e64640 1 --2- 172.21.15.45:0/1249757716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6aa4161490 0x7f6aa415c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.366+0000 7f6aa9e64640 1 --2- 172.21.15.45:0/1249757716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aa406c3a0 0x7f6aa413f940 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.366+0000 7f6aa9e64640 1 -- 172.21.15.45:0/1249757716 >> 172.21.15.45:0/1249757716 conn(0x7f6aa40b39a0 msgr2=0x7f6aa40ce340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.366+0000 7f6aa9e64640 1 -- 172.21.15.45:0/1249757716 shutdown_connections 2024-05-08T05:45:19.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.366+0000 7f6aa9e64640 1 -- 172.21.15.45:0/1249757716 wait complete. 2024-05-08T05:45:19.378 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T05:45:19.378 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T05:45:19.378 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T05:45:19.378 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:45:19.378 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T05:45:19.378 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T05:45:19.379 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T05:45:19.379 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:45:19.379 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T05:45:19.379 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T05:45:19.379 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T05:45:19.379 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:45:19.379 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T05:45:19.379 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T05:45:19.380 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T05:45:19.380 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:45:19.388 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T05:45:19.388 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T05:45:19.389 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T05:45:19.389 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T05:45:19.391 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T05:45:19.392 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T05:45:19.393 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T05:45:19.393 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T05:45:19.430 INFO:tasks.ceph.mds.a:Started 2024-05-08T05:45:19.431 DEBUG:teuthology.parallel:result is None 2024-05-08T05:45:19.431 INFO:tasks.ceph.mds.c:Started 2024-05-08T05:45:19.431 DEBUG:teuthology.parallel:result is None 2024-05-08T05:45:19.435 INFO:tasks.ceph.mds.b:Started 2024-05-08T05:45:19.435 DEBUG:teuthology.parallel:result is None 2024-05-08T05:45:19.437 INFO:tasks.ceph.mds.d:Started 2024-05-08T05:45:19.437 DEBUG:teuthology.parallel:result is None 2024-05-08T05:45:19.437 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T05:45:19.509 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-05-08T05:45:19.510 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-05-08T05:45:19.521 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-08T05:45:19.521 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-08T05:45:19.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.546+0000 7fc30fe28640 1 Processor -- start 2024-05-08T05:45:19.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.547+0000 7fc30fe28640 1 -- start start 2024-05-08T05:45:19.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.547+0000 7fc30fe28640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc3080d1970 0x7fc3080d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:19.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.547+0000 7fc30fe28640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3080d4cc0 0x7fc3080d5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:19.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.547+0000 7fc30fe28640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3081630b0 con 0x7fc30815f420 2024-05-08T05:45:19.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.547+0000 7fc30fe28640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc3080d3e70 con 0x7fc3080d1970 2024-05-08T05:45:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.547+0000 7fc30fe28640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc3080d5670 con 0x7fc3080d4cc0 2024-05-08T05:45:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.548+0000 7fc30dbc4640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc30815f420 0x7fc3080d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:57654/0 (socket says 172.21.15.45:57654) 2024-05-08T05:45:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.548+0000 7fc30dbc4640 1 -- 172.21.15.45:0/3951816107 learned_addr learned my addr 172.21.15.45:0/3951816107 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.548+0000 7fc30d3c3640 1 --2- 172.21.15.45:0/3951816107 >> v2:172.21.15.63:3300/0 conn(0x7fc3080d1970 0x7fc3080d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.548+0000 7fc30e3c5640 1 --2- 172.21.15.45:0/3951816107 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3080d4cc0 0x7fc3080d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.548+0000 7fc30cbc2640 1 -- 172.21.15.45:0/3951816107 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3726977625 0 0) 0x7fc304001110 con 0x7fc30815f420 2024-05-08T05:45:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.548+0000 7fc30cbc2640 1 -- 172.21.15.45:0/3951816107 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2f4003520 con 0x7fc30815f420 2024-05-08T05:45:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.548+0000 7fc30d3c3640 1 -- 172.21.15.45:0/3951816107 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3080d4cc0 msgr2=0x7fc3080d5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.548+0000 7fc30d3c3640 1 --2- 172.21.15.45:0/3951816107 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3080d4cc0 0x7fc3080d5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.548+0000 7fc30d3c3640 1 -- 172.21.15.45:0/3951816107 >> v1:172.21.15.45:6789/0 conn(0x7fc30815f420 legacy=0x7fc3080d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.548+0000 7fc30d3c3640 1 -- 172.21.15.45:0/3951816107 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3080d2550 con 0x7fc3080d1970 2024-05-08T05:45:19.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.548+0000 7fc30e3c5640 1 --2- 172.21.15.45:0/3951816107 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3080d4cc0 0x7fc3080d5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:45:19.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.549+0000 7fc30d3c3640 1 --2- 172.21.15.45:0/3951816107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3080d1970 0x7fc3080d1e20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0097d0 tx=0x7fc2fc02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9037c460b99e84de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:19.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.549+0000 7fc30cbc2640 1 -- 172.21.15.45:0/3951816107 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc037070 con 0x7fc3080d1970 2024-05-08T05:45:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.549+0000 7fc30cbc2640 1 -- 172.21.15.45:0/3951816107 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2fc032440 con 0x7fc3080d1970 2024-05-08T05:45:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.549+0000 7fc30cbc2640 1 -- 172.21.15.45:0/3951816107 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc032810 con 0x7fc3080d1970 2024-05-08T05:45:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.549+0000 7fc30fe28640 1 -- 172.21.15.45:0/3951816107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3080d1970 msgr2=0x7fc3080d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.549+0000 7fc30fe28640 1 --2- 172.21.15.45:0/3951816107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3080d1970 0x7fc3080d1e20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0097d0 tx=0x7fc2fc02a1c0 comp rx=0 tx=0).stop 2024-05-08T05:45:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.550+0000 7fc30fe28640 1 -- 172.21.15.45:0/3951816107 shutdown_connections 2024-05-08T05:45:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.550+0000 7fc30fe28640 1 --2- 172.21.15.45:0/3951816107 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3080d4cc0 0x7fc3080d5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.550+0000 7fc30fe28640 1 --2- 172.21.15.45:0/3951816107 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3080d1970 0x7fc3080d1e20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.550+0000 7fc30fe28640 1 -- 172.21.15.45:0/3951816107 >> 172.21.15.45:0/3951816107 conn(0x7fc30808eac0 msgr2=0x7fc30809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.550+0000 7fc30fe28640 1 -- 172.21.15.45:0/3951816107 shutdown_connections 2024-05-08T05:45:19.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.550+0000 7fc30fe28640 1 -- 172.21.15.45:0/3951816107 wait complete. 2024-05-08T05:45:19.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.551+0000 7fc30fe28640 1 Processor -- start 2024-05-08T05:45:19.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.551+0000 7fc30fe28640 1 -- start start 2024-05-08T05:45:19.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc30fe28640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3080d1970 0x7fc30815e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:19.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc30fe28640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3080d4cc0 0x7fc3081551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:19.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc30fe28640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc30815c960 0x7fc3080d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:19.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc30fe28640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc3080d3240 con 0x7fc3080d1970 2024-05-08T05:45:19.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc30fe28640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc3080d2f60 con 0x7fc30815c960 2024-05-08T05:45:19.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc30fe28640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc3080d2830 con 0x7fc3080d4cc0 2024-05-08T05:45:19.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc30dbc4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3080d1970 0x7fc30815e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:19.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc30dbc4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3080d1970 0x7fc30815e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52818/0 (socket says 172.21.15.45:52818) 2024-05-08T05:45:19.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc30dbc4640 1 -- 172.21.15.45:0/3116379028 learned_addr learned my addr 172.21.15.45:0/3116379028 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:19.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc30d3c3640 1 --2- 172.21.15.45:0/3116379028 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3080d4cc0 0x7fc3081551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:19.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc30e3c5640 1 --2- 172.21.15.45:0/3116379028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc30815c960 0x7fc3080d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:19.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc30dbc4640 1 -- 172.21.15.45:0/3116379028 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3080d4cc0 msgr2=0x7fc3081551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:19.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc30dbc4640 1 --2- 172.21.15.45:0/3116379028 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3080d4cc0 0x7fc3081551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:19.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc30dbc4640 1 -- 172.21.15.45:0/3116379028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc30815c960 msgr2=0x7fc3080d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:19.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc30dbc4640 1 --2- 172.21.15.45:0/3116379028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc30815c960 0x7fc3080d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:19.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc30dbc4640 1 -- 172.21.15.45:0/3116379028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3080cd220 con 0x7fc3080d1970 2024-05-08T05:45:19.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc30dbc4640 1 --2- 172.21.15.45:0/3116379028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3080d1970 0x7fc30815e780 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc30400c760 tx=0x7fc30400cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:19.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc2fa7fc640 1 -- 172.21.15.45:0/3116379028 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304017070 con 0x7fc3080d1970 2024-05-08T05:45:19.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc2fa7fc640 1 -- 172.21.15.45:0/3116379028 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc304001d30 con 0x7fc3080d1970 2024-05-08T05:45:19.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc30fe28640 1 -- 172.21.15.45:0/3116379028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30815d120 con 0x7fc3080d1970 2024-05-08T05:45:19.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc2fa7fc640 1 -- 172.21.15.45:0/3116379028 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304005cd0 con 0x7fc3080d1970 2024-05-08T05:45:19.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.552+0000 7fc30fe28640 1 -- 172.21.15.45:0/3116379028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc30815d6e0 con 0x7fc3080d1970 2024-05-08T05:45:19.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.554+0000 7fc30fe28640 1 -- 172.21.15.45:0/3116379028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3080693f0 con 0x7fc3080d1970 2024-05-08T05:45:19.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.554+0000 7fc2fa7fc640 1 -- 172.21.15.45:0/3116379028 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc30401a070 con 0x7fc3080d1970 2024-05-08T05:45:19.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.555+0000 7fc2fa7fc640 1 --2- 172.21.15.45:0/3116379028 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc2ec076300 0x7fc2ec0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:19.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.555+0000 7fc2fa7fc640 1 -- 172.21.15.45:0/3116379028 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5335+0+0 (secure 0 0 0) 0x7fc3040965c0 con 0x7fc3080d1970 2024-05-08T05:45:19.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.555+0000 7fc30d3c3640 1 --2- 172.21.15.45:0/3116379028 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc2ec076300 0x7fc2ec0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:19.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.555+0000 7fc30d3c3640 1 --2- 172.21.15.45:0/3116379028 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc2ec076300 0x7fc2ec0787c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc034000 tx=0x7fc2fc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:19.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.559+0000 7fc2fa7fc640 1 -- 172.21.15.45:0/3116379028 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc3081630b0 con 0x7fc3080d1970 2024-05-08T05:45:19.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:19.763+0000 7fc30fe28640 1 -- 172.21.15.45:0/3116379028 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc3080b4c50 con 0x7fc3080d1970 2024-05-08T05:45:20.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.805+0000 7fc2fa7fc640 1 -- 172.21.15.45:0/3116379028 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v71) ==== 97+0+0 (secure 0 0 0) 0x7fc30405f9c0 con 0x7fc3080d1970 2024-05-08T05:45:20.807 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2024-05-08T05:45:20.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.808+0000 7fc30fe28640 1 -- 172.21.15.45:0/3116379028 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc2ec076300 msgr2=0x7fc2ec0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:20.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.808+0000 7fc30fe28640 1 --2- 172.21.15.45:0/3116379028 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc2ec076300 0x7fc2ec0787c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc034000 tx=0x7fc2fc02e000 comp rx=0 tx=0).stop 2024-05-08T05:45:20.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.809+0000 7fc30fe28640 1 -- 172.21.15.45:0/3116379028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3080d1970 msgr2=0x7fc30815e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:20.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.809+0000 7fc30fe28640 1 --2- 172.21.15.45:0/3116379028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3080d1970 0x7fc30815e780 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc30400c760 tx=0x7fc30400cb60 comp rx=0 tx=0).stop 2024-05-08T05:45:20.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.809+0000 7fc30fe28640 1 -- 172.21.15.45:0/3116379028 shutdown_connections 2024-05-08T05:45:20.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.809+0000 7fc30fe28640 1 --2- 172.21.15.45:0/3116379028 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc2ec076300 0x7fc2ec0787c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:20.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.809+0000 7fc30fe28640 1 --2- 172.21.15.45:0/3116379028 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3080d4cc0 0x7fc3081551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:20.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.809+0000 7fc30fe28640 1 --2- 172.21.15.45:0/3116379028 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc30815c960 0x7fc3080d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:20.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.809+0000 7fc30fe28640 1 --2- 172.21.15.45:0/3116379028 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3080d1970 0x7fc30815e780 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:20.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.809+0000 7fc30fe28640 1 -- 172.21.15.45:0/3116379028 >> 172.21.15.45:0/3116379028 conn(0x7fc30808eac0 msgr2=0x7fc3080b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:20.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.809+0000 7fc30fe28640 1 -- 172.21.15.45:0/3116379028 shutdown_connections 2024-05-08T05:45:20.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.809+0000 7fc30fe28640 1 -- 172.21.15.45:0/3116379028 wait complete. 2024-05-08T05:45:20.824 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T05:45:20.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.932+0000 7fc915e83640 1 Processor -- start 2024-05-08T05:45:20.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.933+0000 7fc915e83640 1 -- start start 2024-05-08T05:45:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.933+0000 7fc915e83640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9100d53d0 0x7fc9100d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.933+0000 7fc915e83640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9101630b0 con 0x7fc9100d1970 2024-05-08T05:45:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.933+0000 7fc915e83640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9100d3e70 con 0x7fc91015f420 2024-05-08T05:45:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.933+0000 7fc915e83640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc9100d2550 con 0x7fc9100d53d0 2024-05-08T05:45:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.933+0000 7fc90effd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc9100d1970 0x7fc9100d4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34604/0 (socket says 172.21.15.45:34604) 2024-05-08T05:45:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.933+0000 7fc90effd640 1 -- 172.21.15.45:0/928482793 learned_addr learned my addr 172.21.15.45:0/928482793 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.934+0000 7fc90e7fc640 1 -- 172.21.15.45:0/928482793 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4288897539 0 0) 0x7fc904001110 con 0x7fc9100d1970 2024-05-08T05:45:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.934+0000 7fc90ffff640 1 --2- 172.21.15.45:0/928482793 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9100d53d0 0x7fc9100d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.934+0000 7fc90e7fc640 1 -- 172.21.15.45:0/928482793 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8f8003520 con 0x7fc9100d1970 2024-05-08T05:45:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.934+0000 7fc90e7fc640 1 -- 172.21.15.45:0/928482793 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3384156726 0 0) 0x7fc8f8003520 con 0x7fc9100d1970 2024-05-08T05:45:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.934+0000 7fc90e7fc640 1 -- 172.21.15.45:0/928482793 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9100d53d0 msgr2=0x7fc9100d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.934+0000 7fc90e7fc640 1 --2- 172.21.15.45:0/928482793 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9100d53d0 0x7fc9100d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.934+0000 7fc90e7fc640 1 -- 172.21.15.45:0/928482793 >> v1:172.21.15.63:6789/0 conn(0x7fc91015f420 legacy=0x7fc9100d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.934+0000 7fc90e7fc640 1 -- 172.21.15.45:0/928482793 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9100d2c80 con 0x7fc9100d1970 2024-05-08T05:45:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.934+0000 7fc90e7fc640 1 -- 172.21.15.45:0/928482793 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fc9040018c0 con 0x7fc9100d1970 2024-05-08T05:45:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.934+0000 7fc90e7fc640 1 -- 172.21.15.45:0/928482793 >> v1:172.21.15.45:6789/0 conn(0x7fc9100d1970 legacy=0x7fc9100d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.934+0000 7fc90e7fc640 1 --2- 172.21.15.45:0/928482793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f80039e0 0x7fc8f8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:20.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.934+0000 7fc90e7fc640 1 --2- 172.21.15.45:0/928482793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8f8004800 0x7fc8f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.934+0000 7fc90e7fc640 1 --2- 172.21.15.45:0/928482793 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9100d53d0 0x7fc8f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.934+0000 7fc90e7fc640 1 -- 172.21.15.45:0/928482793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc9100d2550 con 0x7fc8f80039e0 2024-05-08T05:45:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.934+0000 7fc90e7fc640 1 -- 172.21.15.45:0/928482793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f800ba70 con 0x7fc8f8004800 2024-05-08T05:45:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.934+0000 7fc90e7fc640 1 -- 172.21.15.45:0/928482793 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f800bc70 con 0x7fc9100d53d0 2024-05-08T05:45:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.934+0000 7fc90f7fe640 1 --2- 172.21.15.45:0/928482793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f80039e0 0x7fc8f8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.934+0000 7fc90effd640 1 --2- 172.21.15.45:0/928482793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8f8004800 0x7fc8f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.934+0000 7fc90ffff640 1 --2- 172.21.15.45:0/928482793 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9100d53d0 0x7fc8f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.935+0000 7fc90f7fe640 1 -- 172.21.15.45:0/928482793 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9100d53d0 msgr2=0x7fc8f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.935+0000 7fc90f7fe640 1 --2- 172.21.15.45:0/928482793 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9100d53d0 0x7fc8f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.935+0000 7fc90f7fe640 1 -- 172.21.15.45:0/928482793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8f8004800 msgr2=0x7fc8f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.935+0000 7fc90f7fe640 1 --2- 172.21.15.45:0/928482793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8f8004800 0x7fc8f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.935+0000 7fc90f7fe640 1 -- 172.21.15.45:0/928482793 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f8003520 con 0x7fc8f80039e0 2024-05-08T05:45:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.935+0000 7fc90ffff640 1 --2- 172.21.15.45:0/928482793 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9100d53d0 0x7fc8f80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.935+0000 7fc90effd640 1 --2- 172.21.15.45:0/928482793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8f8004800 0x7fc8f8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.935+0000 7fc90f7fe640 1 --2- 172.21.15.45:0/928482793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f80039e0 0x7fc8f8003dc0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc9000017e0 tx=0x7fc8f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3cdc86a270ce1da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:20.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.935+0000 7fc90e7fc640 1 -- 172.21.15.45:0/928482793 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900037070 con 0x7fc8f80039e0 2024-05-08T05:45:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.935+0000 7fc90e7fc640 1 -- 172.21.15.45:0/928482793 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc90002ac30 con 0x7fc8f80039e0 2024-05-08T05:45:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.935+0000 7fc90e7fc640 1 -- 172.21.15.45:0/928482793 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900001430 con 0x7fc8f80039e0 2024-05-08T05:45:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.936+0000 7fc915e83640 1 -- 172.21.15.45:0/928482793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f80039e0 msgr2=0x7fc8f8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.936+0000 7fc915e83640 1 --2- 172.21.15.45:0/928482793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8f80039e0 0x7fc8f8003dc0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc9000017e0 tx=0x7fc8f8005830 comp rx=0 tx=0).stop 2024-05-08T05:45:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.936+0000 7fc90ffff640 1 -- 172.21.15.45:0/928482793 reap_dead start 2024-05-08T05:45:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.936+0000 7fc915e83640 1 -- 172.21.15.45:0/928482793 shutdown_connections 2024-05-08T05:45:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.936+0000 7fc915e83640 1 -- 172.21.15.45:0/928482793 >> 172.21.15.45:0/928482793 conn(0x7fc91008eac0 msgr2=0x7fc91009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.936+0000 7fc915e83640 1 -- 172.21.15.45:0/928482793 shutdown_connections 2024-05-08T05:45:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.936+0000 7fc915e83640 1 -- 172.21.15.45:0/928482793 wait complete. 2024-05-08T05:45:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.937+0000 7fc915e83640 1 Processor -- start 2024-05-08T05:45:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.938+0000 7fc915e83640 1 -- start start 2024-05-08T05:45:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.938+0000 7fc915e83640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8f80039e0 0x7fc91015daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.938+0000 7fc915e83640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc91015e160 0x7fc9100ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.938+0000 7fc915e83640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9101ea490 0x7fc9101ea910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.938+0000 7fc915e83640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc9100d2830 con 0x7fc9101ea490 2024-05-08T05:45:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.938+0000 7fc915e83640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc9100d3240 con 0x7fc8f80039e0 2024-05-08T05:45:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.938+0000 7fc915e83640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc9100d1e20 con 0x7fc91015e160 2024-05-08T05:45:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.938+0000 7fc90ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9101ea490 0x7fc9101ea910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.938+0000 7fc90ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9101ea490 0x7fc9101ea910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56370/0 (socket says 172.21.15.45:56370) 2024-05-08T05:45:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.938+0000 7fc90ffff640 1 -- 172.21.15.45:0/1471822768 learned_addr learned my addr 172.21.15.45:0/1471822768 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:20.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.939+0000 7fc90f7fe640 1 --2- 172.21.15.45:0/1471822768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8f80039e0 0x7fc91015daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.939+0000 7fc90effd640 1 --2- 172.21.15.45:0/1471822768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc91015e160 0x7fc9100ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.939+0000 7fc90ffff640 1 -- 172.21.15.45:0/1471822768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc91015e160 msgr2=0x7fc9100ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.939+0000 7fc90ffff640 1 --2- 172.21.15.45:0/1471822768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc91015e160 0x7fc9100ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.939+0000 7fc90ffff640 1 -- 172.21.15.45:0/1471822768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8f80039e0 msgr2=0x7fc91015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.939+0000 7fc90ffff640 1 --2- 172.21.15.45:0/1471822768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8f80039e0 0x7fc91015daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.939+0000 7fc90ffff640 1 -- 172.21.15.45:0/1471822768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9100cefd0 con 0x7fc9101ea490 2024-05-08T05:45:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.939+0000 7fc90effd640 1 --2- 172.21.15.45:0/1471822768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc91015e160 0x7fc9100ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.939+0000 7fc90ffff640 1 --2- 172.21.15.45:0/1471822768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9101ea490 0x7fc9101ea910 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc910163dc0 tx=0x7fc908009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.939+0000 7fc8f7fff640 1 -- 172.21.15.45:0/1471822768 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908010e50 con 0x7fc9101ea490 2024-05-08T05:45:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.939+0000 7fc8f7fff640 1 -- 172.21.15.45:0/1471822768 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc908003990 con 0x7fc9101ea490 2024-05-08T05:45:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.939+0000 7fc915e83640 1 -- 172.21.15.45:0/1471822768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9100ccc90 con 0x7fc9101ea490 2024-05-08T05:45:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.939+0000 7fc8f7fff640 1 -- 172.21.15.45:0/1471822768 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908003d80 con 0x7fc9101ea490 2024-05-08T05:45:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.939+0000 7fc915e83640 1 -- 172.21.15.45:0/1471822768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9100ccf30 con 0x7fc9101ea490 2024-05-08T05:45:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.940+0000 7fc915e83640 1 -- 172.21.15.45:0/1471822768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9100693f0 con 0x7fc9101ea490 2024-05-08T05:45:20.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.941+0000 7fc8f7fff640 1 -- 172.21.15.45:0/1471822768 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc90801e020 con 0x7fc9101ea490 2024-05-08T05:45:20.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.941+0000 7fc8f7fff640 1 --2- 172.21.15.45:0/1471822768 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc8fc076300 0x7fc8fc0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:20.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.941+0000 7fc8f7fff640 1 -- 172.21.15.45:0/1471822768 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5249+0+0 (secure 0 0 0) 0x7fc9080979c0 con 0x7fc9101ea490 2024-05-08T05:45:20.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.941+0000 7fc90f7fe640 1 --2- 172.21.15.45:0/1471822768 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc8fc076300 0x7fc8fc0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:20.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.942+0000 7fc90f7fe640 1 --2- 172.21.15.45:0/1471822768 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc8fc076300 0x7fc8fc0787c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc900034000 tx=0x7fc90002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:20.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:20.946+0000 7fc8f7fff640 1 -- 172.21.15.45:0/1471822768 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc908099e20 con 0x7fc9101ea490 2024-05-08T05:45:21.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.208+0000 7fc915e83640 1 -- 172.21.15.45:0/1471822768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc9100b4c50 con 0x7fc9101ea490 2024-05-08T05:45:21.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.209+0000 7fc8f7fff640 1 -- 172.21.15.45:0/1471822768 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7fc9080614e0 con 0x7fc9101ea490 2024-05-08T05:45:21.210 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:21.210 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:45:21.210 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2024-05-08T05:45:21.210 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:45:21.210 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2024-05-08T05:45:21.210 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmrRgXHBAAX1mgdmOD9JC4S1vJNyLSQw==", 2024-05-08T05:45:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:45:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:45:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:45:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:45:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:45:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:45:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:45:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2024-05-08T05:45:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmvn8SKBAAVkXYBbTfu98gl0AptOfJEg==", 2024-05-08T05:45:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:45:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:45:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:45:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:45:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:45:21.211 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:45:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:45:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2024-05-08T05:45:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmeiJCIxAAr3Br1fJVgR9TW2Q+4tMQBQ==", 2024-05-08T05:45:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:45:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:45:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:45:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:45:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:45:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:45:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:45:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2024-05-08T05:45:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmXdFILxAAxteIxUYeFdQKd/oxHLpeOQ==", 2024-05-08T05:45:21.212 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:45:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:45:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:45:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:45:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:45:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:45:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:45:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2024-05-08T05:45:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBWEDtmLdHHGRAAA1o4Z/lTGERIZhulqVtybw==", 2024-05-08T05:45:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:45:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:45:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:45:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:45:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:45:21.213 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:45:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2024-05-08T05:45:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBZEDtmMrRVGBAAeKBtcf5fSQMFE6IStUvokQ==", 2024-05-08T05:45:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:45:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:45:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:45:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:45:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:45:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:45:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2024-05-08T05:45:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBcEDtmpozQFhAAS6AF6X592Wh/QiBipcOVew==", 2024-05-08T05:45:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:45:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:45:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:45:21.214 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:45:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:45:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:45:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2024-05-08T05:45:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBfEDtm47lSFRAAt5tvpRCljxGtAmHQqcFj2A==", 2024-05-08T05:45:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:45:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:45:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:45:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:45:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:45:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:45:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2024-05-08T05:45:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBlEDtmDQ7mGxAA5z7rHZRf7X5g7ZiqEoO2DA==", 2024-05-08T05:45:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:45:21.215 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:45:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:45:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:45:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:45:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:45:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2024-05-08T05:45:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBoEDtmBdi0GhAAVMfg+KEJHRvaiS49PEboOg==", 2024-05-08T05:45:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:45:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:45:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:45:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:45:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:45:21.216 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:45:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2024-05-08T05:45:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBrEDtmG2daGRAANQgJFAT6gNE/vPofCv6veA==", 2024-05-08T05:45:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:45:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:45:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:45:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:45:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:45:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:45:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2024-05-08T05:45:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBuEDtm3tmDGRAARsAzm9KIzgKAs4ZtfCVdaQ==", 2024-05-08T05:45:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:45:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:45:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:45:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:45:21.217 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:45:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:45:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2024-05-08T05:45:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmDcqGMxAALgQaBYp4Zd8uMqswCXvGaw==", 2024-05-08T05:45:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:45:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:45:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2024-05-08T05:45:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T05:45:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:45:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:45:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2024-05-08T05:45:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBVEDtm6z0GARAARgRma5hPpH6X95pE5NNrQw==", 2024-05-08T05:45:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:45:21.218 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:45:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:45:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:45:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:45:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:45:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:45:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2024-05-08T05:45:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmAMtIORAAJf8oSwHqSzgjsTuqxtQqHQ==", 2024-05-08T05:45:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:45:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:45:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:45:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:45:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:45:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:45:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:45:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2024-05-08T05:45:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBVEDtm9qK1BhAAokvkJZmNxT5WER9/+EKIpA==", 2024-05-08T05:45:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:45:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:45:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:45:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:45:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:45:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:45:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:45:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2024-05-08T05:45:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmZ4Q3ABAANcqmwOmt+dNJ5wOiPHHO4Q==", 2024-05-08T05:45:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:45:21.220 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:45:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:45:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:45:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:45:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:45:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:45:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2024-05-08T05:45:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmL0XLGBAA1BSFdyiDYV9Je+igOVBvlQ==", 2024-05-08T05:45:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:45:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:45:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-05-08T05:45:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:45:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:45:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:45:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2024-05-08T05:45:21.221 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmfM5OFRAAgdJPUfqmnKU8vRrjGMNwrw==", 2024-05-08T05:45:21.222 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:45:21.222 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:45:21.222 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-05-08T05:45:21.222 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:45:21.222 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.222 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.222 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-05-08T05:45:21.222 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:45:21.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.211+0000 7fc915e83640 1 -- 172.21.15.45:0/1471822768 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc8fc076300 msgr2=0x7fc8fc0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:21.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.211+0000 7fc915e83640 1 --2- 172.21.15.45:0/1471822768 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc8fc076300 0x7fc8fc0787c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc900034000 tx=0x7fc90002e000 comp rx=0 tx=0).stop 2024-05-08T05:45:21.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.211+0000 7fc915e83640 1 -- 172.21.15.45:0/1471822768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9101ea490 msgr2=0x7fc9101ea910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:21.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.211+0000 7fc915e83640 1 --2- 172.21.15.45:0/1471822768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9101ea490 0x7fc9101ea910 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc910163dc0 tx=0x7fc908009f90 comp rx=0 tx=0).stop 2024-05-08T05:45:21.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.211+0000 7fc915e83640 1 -- 172.21.15.45:0/1471822768 shutdown_connections 2024-05-08T05:45:21.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.211+0000 7fc915e83640 1 --2- 172.21.15.45:0/1471822768 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc8fc076300 0x7fc8fc0787c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.211+0000 7fc915e83640 1 --2- 172.21.15.45:0/1471822768 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc91015e160 0x7fc9100ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.211+0000 7fc915e83640 1 --2- 172.21.15.45:0/1471822768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9101ea490 0x7fc9101ea910 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.211+0000 7fc915e83640 1 --2- 172.21.15.45:0/1471822768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8f80039e0 0x7fc91015daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.211+0000 7fc915e83640 1 -- 172.21.15.45:0/1471822768 >> 172.21.15.45:0/1471822768 conn(0x7fc91008eac0 msgr2=0x7fc910163290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:21.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.211+0000 7fc915e83640 1 -- 172.21.15.45:0/1471822768 shutdown_connections 2024-05-08T05:45:21.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.211+0000 7fc915e83640 1 -- 172.21.15.45:0/1471822768 wait complete. 2024-05-08T05:45:21.225 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T05:45:21.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.328+0000 7fbdd1b33640 1 Processor -- start 2024-05-08T05:45:21.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.328+0000 7fbdd1b33640 1 -- start start 2024-05-08T05:45:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.329+0000 7fbdd1b33640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbdcc163920 0x7fbdcc06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.329+0000 7fbdd1b33640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdcc163d70 con 0x7fbdcc070d30 2024-05-08T05:45:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.329+0000 7fbdd1b33640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdcc0d1300 con 0x7fbdcc0d20c0 2024-05-08T05:45:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.329+0000 7fbdd1b33640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbdcc0d10d0 con 0x7fbdcc163920 2024-05-08T05:45:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.329+0000 7fbdcb7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbdcc070d30 0x7fbdcc0703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34620/0 (socket says 172.21.15.45:34620) 2024-05-08T05:45:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.329+0000 7fbdcb7fe640 1 -- 172.21.15.45:0/625405995 learned_addr learned my addr 172.21.15.45:0/625405995 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.329+0000 7fbdca7fc640 1 -- 172.21.15.45:0/625405995 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1350967077 0 0) 0x7fbdbc001110 con 0x7fbdcc070d30 2024-05-08T05:45:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.329+0000 7fbdcbfff640 1 --2- 172.21.15.45:0/625405995 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbdcc163920 0x7fbdcc06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.329+0000 7fbdca7fc640 1 -- 172.21.15.45:0/625405995 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdb4003520 con 0x7fbdcc070d30 2024-05-08T05:45:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.329+0000 7fbdca7fc640 1 -- 172.21.15.45:0/625405995 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3627636666 0 0) 0x7fbdb4003520 con 0x7fbdcc070d30 2024-05-08T05:45:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.329+0000 7fbdca7fc640 1 -- 172.21.15.45:0/625405995 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbdcc163920 msgr2=0x7fbdcc06d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.329+0000 7fbdca7fc640 1 --2- 172.21.15.45:0/625405995 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbdcc163920 0x7fbdcc06d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.330+0000 7fbdca7fc640 1 -- 172.21.15.45:0/625405995 >> v1:172.21.15.63:6789/0 conn(0x7fbdcc0d20c0 legacy=0x7fbdcc071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.330+0000 7fbdca7fc640 1 -- 172.21.15.45:0/625405995 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdcc0d15e0 con 0x7fbdcc070d30 2024-05-08T05:45:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.330+0000 7fbdca7fc640 1 -- 172.21.15.45:0/625405995 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fbdbc0018c0 con 0x7fbdcc070d30 2024-05-08T05:45:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.330+0000 7fbdca7fc640 1 -- 172.21.15.45:0/625405995 >> v1:172.21.15.45:6789/0 conn(0x7fbdcc070d30 legacy=0x7fbdcc0703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.330+0000 7fbdca7fc640 1 --2- 172.21.15.45:0/625405995 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbdcc163920 0x7fbdb4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.330+0000 7fbdca7fc640 1 --2- 172.21.15.45:0/625405995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdb4004000 0x7fbdb4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.330+0000 7fbdca7fc640 1 --2- 172.21.15.45:0/625405995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb4004d40 0x7fbdb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.330+0000 7fbdca7fc640 1 -- 172.21.15.45:0/625405995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb400bb50 con 0x7fbdb4004d40 2024-05-08T05:45:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.330+0000 7fbdca7fc640 1 -- 172.21.15.45:0/625405995 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb400bcd0 con 0x7fbdb4004000 2024-05-08T05:45:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.330+0000 7fbdca7fc640 1 -- 172.21.15.45:0/625405995 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbdb400bed0 con 0x7fbdcc163920 2024-05-08T05:45:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.330+0000 7fbdcbfff640 1 --2- 172.21.15.45:0/625405995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb4004d40 0x7fbdb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.330+0000 7fbdcb7fe640 1 --2- 172.21.15.45:0/625405995 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbdcc163920 0x7fbdb4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.330+0000 7fbdcaffd640 1 --2- 172.21.15.45:0/625405995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdb4004000 0x7fbdb4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.330+0000 7fbdcbfff640 1 -- 172.21.15.45:0/625405995 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbdcc163920 msgr2=0x7fbdb4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.330+0000 7fbdcbfff640 1 --2- 172.21.15.45:0/625405995 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbdcc163920 0x7fbdb4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.330+0000 7fbdcbfff640 1 -- 172.21.15.45:0/625405995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdb4004000 msgr2=0x7fbdb4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:21.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.330+0000 7fbdcbfff640 1 --2- 172.21.15.45:0/625405995 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdb4004000 0x7fbdb4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.330+0000 7fbdcbfff640 1 -- 172.21.15.45:0/625405995 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb4003520 con 0x7fbdb4004d40 2024-05-08T05:45:21.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.331+0000 7fbdcbfff640 1 --2- 172.21.15.45:0/625405995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb4004d40 0x7fbdb40051c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009fc0 tx=0x7fbdb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a264081621104c1f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:21.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.331+0000 7fbdca7fc640 1 -- 172.21.15.45:0/625405995 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4009c10 con 0x7fbdb4004d40 2024-05-08T05:45:21.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.331+0000 7fbdca7fc640 1 -- 172.21.15.45:0/625405995 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdc4036070 con 0x7fbdb4004d40 2024-05-08T05:45:21.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.331+0000 7fbdca7fc640 1 -- 172.21.15.45:0/625405995 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4031880 con 0x7fbdb4004d40 2024-05-08T05:45:21.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.332+0000 7fbdd1b33640 1 -- 172.21.15.45:0/625405995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb4004d40 msgr2=0x7fbdb40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:21.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.332+0000 7fbdd1b33640 1 --2- 172.21.15.45:0/625405995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdb4004d40 0x7fbdb40051c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009fc0 tx=0x7fbdb4005830 comp rx=0 tx=0).stop 2024-05-08T05:45:21.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.332+0000 7fbdcbfff640 1 -- 172.21.15.45:0/625405995 reap_dead start 2024-05-08T05:45:21.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.332+0000 7fbdd1b33640 1 -- 172.21.15.45:0/625405995 shutdown_connections 2024-05-08T05:45:21.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.332+0000 7fbdd1b33640 1 -- 172.21.15.45:0/625405995 >> 172.21.15.45:0/625405995 conn(0x7fbdcc0d2e90 msgr2=0x7fbdcc0d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:21.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.332+0000 7fbdd1b33640 1 -- 172.21.15.45:0/625405995 shutdown_connections 2024-05-08T05:45:21.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.332+0000 7fbdd1b33640 1 -- 172.21.15.45:0/625405995 wait complete. 2024-05-08T05:45:21.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.333+0000 7fbdd1b33640 1 Processor -- start 2024-05-08T05:45:21.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.334+0000 7fbdd1b33640 1 -- start start 2024-05-08T05:45:21.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.334+0000 7fbdd1b33640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdcc1574c0 0x7fbdcc162860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:21.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.334+0000 7fbdd1b33640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbdcc0c9380 0x7fbdcc0cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:21.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.334+0000 7fbdd1b33640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdcc0cbe40 0x7fbdcc159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:21.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.334+0000 7fbdd1b33640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc0d1c30 con 0x7fbdcc0cbe40 2024-05-08T05:45:21.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.334+0000 7fbdd1b33640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc070b80 con 0x7fbdcc1574c0 2024-05-08T05:45:21.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.334+0000 7fbdd1b33640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbdcc06d4b0 con 0x7fbdcc0c9380 2024-05-08T05:45:21.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.335+0000 7fbdcbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdcc0cbe40 0x7fbdcc159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:21.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.335+0000 7fbdcbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdcc0cbe40 0x7fbdcc159030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56382/0 (socket says 172.21.15.45:56382) 2024-05-08T05:45:21.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.335+0000 7fbdcbfff640 1 -- 172.21.15.45:0/1286503810 learned_addr learned my addr 172.21.15.45:0/1286503810 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:21.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.335+0000 7fbdcb7fe640 1 --2- 172.21.15.45:0/1286503810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdcc1574c0 0x7fbdcc162860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:21.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.335+0000 7fbdcaffd640 1 --2- 172.21.15.45:0/1286503810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbdcc0c9380 0x7fbdcc0cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:21.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.335+0000 7fbdcbfff640 1 -- 172.21.15.45:0/1286503810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbdcc0c9380 msgr2=0x7fbdcc0cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:21.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.335+0000 7fbdcbfff640 1 --2- 172.21.15.45:0/1286503810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbdcc0c9380 0x7fbdcc0cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.335+0000 7fbdcbfff640 1 -- 172.21.15.45:0/1286503810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdcc1574c0 msgr2=0x7fbdcc162860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:21.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.335+0000 7fbdcbfff640 1 --2- 172.21.15.45:0/1286503810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdcc1574c0 0x7fbdcc162860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.335+0000 7fbdcbfff640 1 -- 172.21.15.45:0/1286503810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdcc0c97e0 con 0x7fbdcc0cbe40 2024-05-08T05:45:21.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.335+0000 7fbdcbfff640 1 --2- 172.21.15.45:0/1286503810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdcc0cbe40 0x7fbdcc159030 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fbdc400e230 tx=0x7fbdc4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:21.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.335+0000 7fbdb3fff640 1 -- 172.21.15.45:0/1286503810 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4009c10 con 0x7fbdcc0cbe40 2024-05-08T05:45:21.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.335+0000 7fbdb3fff640 1 -- 172.21.15.45:0/1286503810 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdc4031dd0 con 0x7fbdcc0cbe40 2024-05-08T05:45:21.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.335+0000 7fbdd1b33640 1 -- 172.21.15.45:0/1286503810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdcc0cb2f0 con 0x7fbdcc0cbe40 2024-05-08T05:45:21.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.335+0000 7fbdb3fff640 1 -- 172.21.15.45:0/1286503810 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4003bd0 con 0x7fbdcc0cbe40 2024-05-08T05:45:21.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.335+0000 7fbdd1b33640 1 -- 172.21.15.45:0/1286503810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdcc158850 con 0x7fbdcc0cbe40 2024-05-08T05:45:21.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.336+0000 7fbdd1b33640 1 -- 172.21.15.45:0/1286503810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdcc0693f0 con 0x7fbdcc0cbe40 2024-05-08T05:45:21.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.337+0000 7fbdb3fff640 1 -- 172.21.15.45:0/1286503810 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbdc4005d10 con 0x7fbdcc0cbe40 2024-05-08T05:45:21.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.337+0000 7fbdb3fff640 1 --2- 172.21.15.45:0/1286503810 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbdb8076300 0x7fbdb80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:21.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.337+0000 7fbdb3fff640 1 -- 172.21.15.45:0/1286503810 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5249+0+0 (secure 0 0 0) 0x7fbdc40b84c0 con 0x7fbdcc0cbe40 2024-05-08T05:45:21.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.338+0000 7fbdcb7fe640 1 --2- 172.21.15.45:0/1286503810 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbdb8076300 0x7fbdb80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:21.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.338+0000 7fbdcb7fe640 1 --2- 172.21.15.45:0/1286503810 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbdb8076300 0x7fbdb80787c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc007fd0 tx=0x7fbdbc007aa0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:21.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.342+0000 7fbdb3fff640 1 -- 172.21.15.45:0/1286503810 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbdc404a060 con 0x7fbdcc0cbe40 2024-05-08T05:45:21.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.537+0000 7fbdd1b33640 1 -- 172.21.15.45:0/1286503810 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fbdcc0b15d0 con 0x7fbdcc0cbe40 2024-05-08T05:45:21.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.537+0000 7fbdb3fff640 1 -- 172.21.15.45:0/1286503810 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7fbdcc0c97e0 con 0x7fbdcc0cbe40 2024-05-08T05:45:21.538 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:21.538 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-05-08T05:45:21.539 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:45:21.539 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-05-08T05:45:21.539 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-05-08T05:45:21.539 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:21.539 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-05-08T05:45:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.539+0000 7fbdd1b33640 1 -- 172.21.15.45:0/1286503810 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbdb8076300 msgr2=0x7fbdb80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.539+0000 7fbdd1b33640 1 --2- 172.21.15.45:0/1286503810 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbdb8076300 0x7fbdb80787c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc007fd0 tx=0x7fbdbc007aa0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.539+0000 7fbdd1b33640 1 -- 172.21.15.45:0/1286503810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdcc0cbe40 msgr2=0x7fbdcc159030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.539+0000 7fbdd1b33640 1 --2- 172.21.15.45:0/1286503810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdcc0cbe40 0x7fbdcc159030 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fbdc400e230 tx=0x7fbdc4009750 comp rx=0 tx=0).stop 2024-05-08T05:45:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.539+0000 7fbdd1b33640 1 -- 172.21.15.45:0/1286503810 shutdown_connections 2024-05-08T05:45:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.539+0000 7fbdd1b33640 1 --2- 172.21.15.45:0/1286503810 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbdb8076300 0x7fbdb80787c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.539+0000 7fbdd1b33640 1 --2- 172.21.15.45:0/1286503810 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbdcc0c9380 0x7fbdcc0cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.539+0000 7fbdd1b33640 1 --2- 172.21.15.45:0/1286503810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdcc0cbe40 0x7fbdcc159030 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.539+0000 7fbdd1b33640 1 --2- 172.21.15.45:0/1286503810 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdcc1574c0 0x7fbdcc162860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.539+0000 7fbdd1b33640 1 -- 172.21.15.45:0/1286503810 >> 172.21.15.45:0/1286503810 conn(0x7fbdcc0d2e90 msgr2=0x7fbdcc160a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.539+0000 7fbdd1b33640 1 -- 172.21.15.45:0/1286503810 shutdown_connections 2024-05-08T05:45:21.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.539+0000 7fbdd1b33640 1 -- 172.21.15.45:0/1286503810 wait complete. 2024-05-08T05:45:21.552 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T05:45:21.552 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T05:45:21.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.658+0000 7f06cb724640 1 Processor -- start 2024-05-08T05:45:21.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.659+0000 7f06cb724640 1 -- start start 2024-05-08T05:45:21.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.660+0000 7f06cb724640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06c41629f0 0x7f06c40d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:21.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.660+0000 7f06cb724640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06c40d22a0 0x7f06c40d1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:21.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.660+0000 7f06cb724640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f06c40d1ba0 0x7f06c40ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:21.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.660+0000 7f06cb724640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f06c40cde80 con 0x7f06c41629f0 2024-05-08T05:45:21.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.660+0000 7f06cb724640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f06c40ce120 con 0x7f06c40d1ba0 2024-05-08T05:45:21.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.660+0000 7f06cb724640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f06c40ce400 con 0x7f06c40d22a0 2024-05-08T05:45:21.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.660+0000 7f06c94c0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06c41629f0 0x7f06c40d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:21.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.660+0000 7f06c94c0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f06c41629f0 0x7f06c40d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56384/0 (socket says 172.21.15.45:56384) 2024-05-08T05:45:21.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.660+0000 7f06c94c0640 1 -- 172.21.15.45:0/4122607516 learned_addr learned my addr 172.21.15.45:0/4122607516 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:21.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.660+0000 7f06c9cc1640 1 --2- 172.21.15.45:0/4122607516 >> v2:172.21.15.63:3300/0 conn(0x7f06c40d1ba0 0x7f06c40ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:21.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.660+0000 7f06c8cbf640 1 --2- 172.21.15.45:0/4122607516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06c40d22a0 0x7f06c40d1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:21.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.660+0000 7f06c94c0640 1 -- 172.21.15.45:0/4122607516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06c40d22a0 msgr2=0x7f06c40d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:21.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.660+0000 7f06c94c0640 1 --2- 172.21.15.45:0/4122607516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06c40d22a0 0x7f06c40d1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.660+0000 7f06c94c0640 1 -- 172.21.15.45:0/4122607516 >> v2:172.21.15.63:3300/0 conn(0x7f06c40d1ba0 msgr2=0x7f06c40ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:21.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.660+0000 7f06c94c0640 1 --2- 172.21.15.45:0/4122607516 >> v2:172.21.15.63:3300/0 conn(0x7f06c40d1ba0 0x7f06c40ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.660+0000 7f06c94c0640 1 -- 172.21.15.45:0/4122607516 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c40ce6e0 con 0x7f06c41629f0 2024-05-08T05:45:21.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.661+0000 7f06c94c0640 1 --2- 172.21.15.45:0/4122607516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06c41629f0 0x7f06c40d1120 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f06c4073050 tx=0x7f06c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b2d014062e2ff1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:21.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.661+0000 7f06bbfff640 1 -- 172.21.15.45:0/4122607516 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0037070 con 0x7f06c41629f0 2024-05-08T05:45:21.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.661+0000 7f06bbfff640 1 -- 172.21.15.45:0/4122607516 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06c0032440 con 0x7f06c41629f0 2024-05-08T05:45:21.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.661+0000 7f06bbfff640 1 -- 172.21.15.45:0/4122607516 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0032810 con 0x7f06c41629f0 2024-05-08T05:45:21.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.661+0000 7f06cb724640 1 -- 172.21.15.45:0/4122607516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06c41629f0 msgr2=0x7f06c40d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:21.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.661+0000 7f06cb724640 1 --2- 172.21.15.45:0/4122607516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06c41629f0 0x7f06c40d1120 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f06c4073050 tx=0x7f06c002a1e0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.661+0000 7f06cb724640 1 -- 172.21.15.45:0/4122607516 shutdown_connections 2024-05-08T05:45:21.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.661+0000 7f06cb724640 1 --2- 172.21.15.45:0/4122607516 >> v2:172.21.15.63:3300/0 conn(0x7f06c40d1ba0 0x7f06c40ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.661+0000 7f06cb724640 1 --2- 172.21.15.45:0/4122607516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06c40d22a0 0x7f06c40d1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.661+0000 7f06cb724640 1 --2- 172.21.15.45:0/4122607516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06c41629f0 0x7f06c40d1120 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.661+0000 7f06cb724640 1 -- 172.21.15.45:0/4122607516 >> 172.21.15.45:0/4122607516 conn(0x7f06c40d2d80 msgr2=0x7f06c40d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:21.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.662+0000 7f06cb724640 1 -- 172.21.15.45:0/4122607516 shutdown_connections 2024-05-08T05:45:21.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.662+0000 7f06cb724640 1 -- 172.21.15.45:0/4122607516 wait complete. 2024-05-08T05:45:21.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.663+0000 7f06cb724640 1 Processor -- start 2024-05-08T05:45:21.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.663+0000 7f06cb724640 1 -- start start 2024-05-08T05:45:21.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.663+0000 7f06cb724640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06c40d1ba0 0x7f06c409e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:21.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.663+0000 7f06cb724640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06c40d22a0 0x7f06c40cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:21.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06cb724640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06c41629f0 0x7f06c41f8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:21.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06cb724640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f06c40cf5a0 con 0x7f06c40d1ba0 2024-05-08T05:45:21.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06cb724640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f06c40cf420 con 0x7f06c41629f0 2024-05-08T05:45:21.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06cb724640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f06c40cf720 con 0x7f06c40d22a0 2024-05-08T05:45:21.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06c94c0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06c40d1ba0 0x7f06c409e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:21.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06c94c0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06c40d1ba0 0x7f06c409e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56394/0 (socket says 172.21.15.45:56394) 2024-05-08T05:45:21.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06c94c0640 1 -- 172.21.15.45:0/164853231 learned_addr learned my addr 172.21.15.45:0/164853231 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:21.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06c9cc1640 1 --2- 172.21.15.45:0/164853231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06c41629f0 0x7f06c41f8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:21.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06c8cbf640 1 --2- 172.21.15.45:0/164853231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06c40d22a0 0x7f06c40cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:21.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06c94c0640 1 -- 172.21.15.45:0/164853231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06c40d22a0 msgr2=0x7f06c40cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:21.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06c94c0640 1 --2- 172.21.15.45:0/164853231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06c40d22a0 0x7f06c40cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06c94c0640 1 -- 172.21.15.45:0/164853231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06c41629f0 msgr2=0x7f06c41f8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:21.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06c94c0640 1 --2- 172.21.15.45:0/164853231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06c41629f0 0x7f06c41f8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:21.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06c94c0640 1 -- 172.21.15.45:0/164853231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c409ee10 con 0x7f06c40d1ba0 2024-05-08T05:45:21.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06c9cc1640 1 --2- 172.21.15.45:0/164853231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06c41629f0 0x7f06c41f8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:21.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06c94c0640 1 --2- 172.21.15.45:0/164853231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06c40d1ba0 0x7f06c409e8d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f06c0000c00 tx=0x7f06c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:21.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06b9ffb640 1 -- 172.21.15.45:0/164853231 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0041070 con 0x7f06c40d1ba0 2024-05-08T05:45:21.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06b9ffb640 1 -- 172.21.15.45:0/164853231 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06c003c870 con 0x7f06c40d1ba0 2024-05-08T05:45:21.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06cb724640 1 -- 172.21.15.45:0/164853231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c409d830 con 0x7f06c40d1ba0 2024-05-08T05:45:21.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06b9ffb640 1 -- 172.21.15.45:0/164853231 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0037040 con 0x7f06c40d1ba0 2024-05-08T05:45:21.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.664+0000 7f06cb724640 1 -- 172.21.15.45:0/164853231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c409ddf0 con 0x7f06c40d1ba0 2024-05-08T05:45:21.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.666+0000 7f06cb724640 1 -- 172.21.15.45:0/164853231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c40693f0 con 0x7f06c40d1ba0 2024-05-08T05:45:21.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.666+0000 7f06b9ffb640 1 -- 172.21.15.45:0/164853231 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f06c0044070 con 0x7f06c40d1ba0 2024-05-08T05:45:21.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.667+0000 7f06b9ffb640 1 --2- 172.21.15.45:0/164853231 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f06a80762d0 0x7f06a8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:21.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.667+0000 7f06b9ffb640 1 -- 172.21.15.45:0/164853231 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5249+0+0 (secure 0 0 0) 0x7f06c00b7f30 con 0x7f06c40d1ba0 2024-05-08T05:45:21.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.667+0000 7f06c8cbf640 1 --2- 172.21.15.45:0/164853231 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f06a80762d0 0x7f06a8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:21.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.667+0000 7f06c8cbf640 1 --2- 172.21.15.45:0/164853231 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f06a80762d0 0x7f06a8078790 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f06c40ccdf0 tx=0x7f06b4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:21.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.671+0000 7f06b9ffb640 1 -- 172.21.15.45:0/164853231 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f06c0081a40 con 0x7f06c40d1ba0 2024-05-08T05:45:21.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:21.862+0000 7f06cb724640 1 -- 172.21.15.45:0/164853231 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f06c40d2ad0 con 0x7f06c40d1ba0 2024-05-08T05:45:22.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.809+0000 7f06b9ffb640 1 -- 172.21.15.45:0/164853231 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7f06c0081340 con 0x7f06c40d1ba0 2024-05-08T05:45:22.810 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-05-08T05:45:22.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.812+0000 7f06cb724640 1 -- 172.21.15.45:0/164853231 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f06a80762d0 msgr2=0x7f06a8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:22.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.812+0000 7f06cb724640 1 --2- 172.21.15.45:0/164853231 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f06a80762d0 0x7f06a8078790 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f06c40ccdf0 tx=0x7f06b4008040 comp rx=0 tx=0).stop 2024-05-08T05:45:22.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.812+0000 7f06cb724640 1 -- 172.21.15.45:0/164853231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06c40d1ba0 msgr2=0x7f06c409e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:22.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.812+0000 7f06cb724640 1 --2- 172.21.15.45:0/164853231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06c40d1ba0 0x7f06c409e8d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f06c0000c00 tx=0x7f06c00046c0 comp rx=0 tx=0).stop 2024-05-08T05:45:22.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.813+0000 7f06cb724640 1 -- 172.21.15.45:0/164853231 shutdown_connections 2024-05-08T05:45:22.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.813+0000 7f06cb724640 1 --2- 172.21.15.45:0/164853231 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f06a80762d0 0x7f06a8078790 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:22.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.813+0000 7f06cb724640 1 --2- 172.21.15.45:0/164853231 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f06c40d22a0 0x7f06c40cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:22.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.813+0000 7f06cb724640 1 --2- 172.21.15.45:0/164853231 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f06c41629f0 0x7f06c41f8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:22.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.813+0000 7f06cb724640 1 --2- 172.21.15.45:0/164853231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f06c40d1ba0 0x7f06c409e8d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:22.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.813+0000 7f06cb724640 1 -- 172.21.15.45:0/164853231 >> 172.21.15.45:0/164853231 conn(0x7f06c40d2d80 msgr2=0x7f06c4160c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:22.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.813+0000 7f06cb724640 1 -- 172.21.15.45:0/164853231 shutdown_connections 2024-05-08T05:45:22.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.813+0000 7f06cb724640 1 -- 172.21.15.45:0/164853231 wait complete. 2024-05-08T05:45:22.832 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T05:45:22.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.953+0000 7fee5f7ef640 1 Processor -- start 2024-05-08T05:45:22.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.954+0000 7fee5f7ef640 1 -- start start 2024-05-08T05:45:22.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.954+0000 7fee5f7ef640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee580d2130 0x7fee581639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:22.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.954+0000 7fee5f7ef640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fee58071110 0x7fee5806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:22.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.954+0000 7fee5f7ef640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee58160720 con 0x7fee58070d30 2024-05-08T05:45:22.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.954+0000 7fee5f7ef640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fee580d1ca0 con 0x7fee58071110 2024-05-08T05:45:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.954+0000 7fee5f7ef640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fee580d10d0 con 0x7fee580d2130 2024-05-08T05:45:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.955+0000 7fee5dd8c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fee58071110 0x7fee5806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.955+0000 7fee5d58b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fee58070d30 0x7fee580703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34634/0 (socket says 172.21.15.45:34634) 2024-05-08T05:45:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.955+0000 7fee5d58b640 1 -- 172.21.15.45:0/2903941841 learned_addr learned my addr 172.21.15.45:0/2903941841 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.955+0000 7fee5dd8c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fee58071110 0x7fee5806d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56402/0 (socket says 172.21.15.45:56402) 2024-05-08T05:45:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.955+0000 7fee5cd8a640 1 --2- 172.21.15.45:0/2903941841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee580d2130 0x7fee581639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.955+0000 7fee5dd8c640 1 -- 172.21.15.45:0/2903941841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee580d2130 msgr2=0x7fee581639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.955+0000 7fee5dd8c640 1 --2- 172.21.15.45:0/2903941841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee580d2130 0x7fee581639e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.955+0000 7fee5dd8c640 1 -- 172.21.15.45:0/2903941841 >> v1:172.21.15.45:6789/0 conn(0x7fee58070d30 legacy=0x7fee580703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.955+0000 7fee5dd8c640 1 -- 172.21.15.45:0/2903941841 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee580d1370 con 0x7fee58071110 2024-05-08T05:45:22.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.955+0000 7fee4ffff640 1 -- 172.21.15.45:0/2903941841 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1432901154 0 0) 0x7fee58160720 con 0x7fee58070d30 2024-05-08T05:45:22.958 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.956+0000 7fee5dd8c640 1 --2- 172.21.15.45:0/2903941841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee58071110 0x7fee5806d4b0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fee50009d90 tx=0x7fee5002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc3259f8aab2914 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:22.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.956+0000 7fee4ffff640 1 -- 172.21.15.45:0/2903941841 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee50007740 con 0x7fee58071110 2024-05-08T05:45:22.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.956+0000 7fee4ffff640 1 -- 172.21.15.45:0/2903941841 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee500078e0 con 0x7fee58071110 2024-05-08T05:45:22.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.956+0000 7fee4ffff640 1 -- 172.21.15.45:0/2903941841 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee50007cb0 con 0x7fee58071110 2024-05-08T05:45:22.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.957+0000 7fee5f7ef640 1 -- 172.21.15.45:0/2903941841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee58071110 msgr2=0x7fee5806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:22.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.957+0000 7fee5f7ef640 1 --2- 172.21.15.45:0/2903941841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee58071110 0x7fee5806d4b0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fee50009d90 tx=0x7fee5002a660 comp rx=0 tx=0).stop 2024-05-08T05:45:22.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.957+0000 7fee5f7ef640 1 -- 172.21.15.45:0/2903941841 shutdown_connections 2024-05-08T05:45:22.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.957+0000 7fee5f7ef640 1 --2- 172.21.15.45:0/2903941841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee58071110 0x7fee5806d4b0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:22.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.957+0000 7fee5f7ef640 1 --2- 172.21.15.45:0/2903941841 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee580d2130 0x7fee581639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:22.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.957+0000 7fee5f7ef640 1 -- 172.21.15.45:0/2903941841 >> 172.21.15.45:0/2903941841 conn(0x7fee580d2e90 msgr2=0x7fee580d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:22.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.957+0000 7fee5f7ef640 1 -- 172.21.15.45:0/2903941841 shutdown_connections 2024-05-08T05:45:22.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.957+0000 7fee5f7ef640 1 -- 172.21.15.45:0/2903941841 wait complete. 2024-05-08T05:45:22.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.958+0000 7fee5f7ef640 1 Processor -- start 2024-05-08T05:45:22.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.958+0000 7fee5f7ef640 1 -- start start 2024-05-08T05:45:22.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee5f7ef640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee58070d30 0x7fee580c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:22.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee5f7ef640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee58071110 0x7fee58144f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee5f7ef640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee580d2130 0x7fee5815b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee5f7ef640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fee580715a0 con 0x7fee58071110 2024-05-08T05:45:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee5f7ef640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fee580d1930 con 0x7fee58070d30 2024-05-08T05:45:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee5f7ef640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fee58071720 con 0x7fee580d2130 2024-05-08T05:45:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee5cd8a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee58071110 0x7fee58144f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee5cd8a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee58071110 0x7fee58144f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56418/0 (socket says 172.21.15.45:56418) 2024-05-08T05:45:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee5cd8a640 1 -- 172.21.15.45:0/91646790 learned_addr learned my addr 172.21.15.45:0/91646790 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee5dd8c640 1 --2- 172.21.15.45:0/91646790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee580d2130 0x7fee5815b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee5d58b640 1 --2- 172.21.15.45:0/91646790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee58070d30 0x7fee580c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee5cd8a640 1 -- 172.21.15.45:0/91646790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee580d2130 msgr2=0x7fee5815b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee5cd8a640 1 --2- 172.21.15.45:0/91646790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee580d2130 0x7fee5815b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:22.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee5cd8a640 1 -- 172.21.15.45:0/91646790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee58070d30 msgr2=0x7fee580c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:22.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee5cd8a640 1 --2- 172.21.15.45:0/91646790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee58070d30 0x7fee580c9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:22.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee5cd8a640 1 -- 172.21.15.45:0/91646790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee580cb1c0 con 0x7fee58071110 2024-05-08T05:45:22.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee5d58b640 1 --2- 172.21.15.45:0/91646790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee58070d30 0x7fee580c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:22.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee5cd8a640 1 --2- 172.21.15.45:0/91646790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee58071110 0x7fee58144f40 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fee4800d800 tx=0x7fee4800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:22.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee4dffb640 1 -- 172.21.15.45:0/91646790 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4800b6c0 con 0x7fee58071110 2024-05-08T05:45:22.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee4dffb640 1 -- 172.21.15.45:0/91646790 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee48019070 con 0x7fee58071110 2024-05-08T05:45:22.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee5f7ef640 1 -- 172.21.15.45:0/91646790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee580cb4a0 con 0x7fee58071110 2024-05-08T05:45:22.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.959+0000 7fee4dffb640 1 -- 172.21.15.45:0/91646790 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4800ba80 con 0x7fee58071110 2024-05-08T05:45:22.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.960+0000 7fee5f7ef640 1 -- 172.21.15.45:0/91646790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee580cba60 con 0x7fee58071110 2024-05-08T05:45:22.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.961+0000 7fee5f7ef640 1 -- 172.21.15.45:0/91646790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee580693f0 con 0x7fee58071110 2024-05-08T05:45:22.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.961+0000 7fee4dffb640 1 -- 172.21.15.45:0/91646790 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fee4800bc20 con 0x7fee58071110 2024-05-08T05:45:22.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.962+0000 7fee4dffb640 1 --2- 172.21.15.45:0/91646790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fee3c076300 0x7fee3c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:22.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.962+0000 7fee4dffb640 1 -- 172.21.15.45:0/91646790 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5619+0+0 (secure 0 0 0) 0x7fee48097990 con 0x7fee58071110 2024-05-08T05:45:22.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.962+0000 7fee5d58b640 1 --2- 172.21.15.45:0/91646790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fee3c076300 0x7fee3c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:22.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.963+0000 7fee5d58b640 1 --2- 172.21.15.45:0/91646790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fee3c076300 0x7fee3c0787c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fee54000f60 tx=0x7fee54007ca0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:22.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:22.967+0000 7fee4dffb640 1 -- 172.21.15.45:0/91646790 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fee48061340 con 0x7fee58071110 2024-05-08T05:45:23.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.158+0000 7fee5f7ef640 1 -- 172.21.15.45:0/91646790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fee580d3d70 con 0x7fee58071110 2024-05-08T05:45:23.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.811+0000 7fee4dffb640 1 -- 172.21.15.45:0/91646790 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) ==== 170+0+0 (secure 0 0 0) 0x7fee48060c40 con 0x7fee58071110 2024-05-08T05:45:23.813 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-05-08T05:45:23.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.815+0000 7fee5f7ef640 1 -- 172.21.15.45:0/91646790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fee3c076300 msgr2=0x7fee3c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:23.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.815+0000 7fee5f7ef640 1 --2- 172.21.15.45:0/91646790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fee3c076300 0x7fee3c0787c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fee54000f60 tx=0x7fee54007ca0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.815+0000 7fee5f7ef640 1 -- 172.21.15.45:0/91646790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee58071110 msgr2=0x7fee58144f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:23.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.815+0000 7fee5f7ef640 1 --2- 172.21.15.45:0/91646790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee58071110 0x7fee58144f40 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fee4800d800 tx=0x7fee4800dcc0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.816+0000 7fee5f7ef640 1 -- 172.21.15.45:0/91646790 shutdown_connections 2024-05-08T05:45:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.816+0000 7fee5f7ef640 1 --2- 172.21.15.45:0/91646790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fee3c076300 0x7fee3c0787c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.816+0000 7fee5f7ef640 1 --2- 172.21.15.45:0/91646790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fee580d2130 0x7fee5815b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.816+0000 7fee5f7ef640 1 --2- 172.21.15.45:0/91646790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee58071110 0x7fee58144f40 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.816+0000 7fee5f7ef640 1 --2- 172.21.15.45:0/91646790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fee58070d30 0x7fee580c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.816+0000 7fee5f7ef640 1 -- 172.21.15.45:0/91646790 >> 172.21.15.45:0/91646790 conn(0x7fee580d2e90 msgr2=0x7fee58160e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:23.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.816+0000 7fee5f7ef640 1 -- 172.21.15.45:0/91646790 shutdown_connections 2024-05-08T05:45:23.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.816+0000 7fee5f7ef640 1 -- 172.21.15.45:0/91646790 wait complete. 2024-05-08T05:45:23.847 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T05:45:23.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.958+0000 7fa8db2d1640 1 Processor -- start 2024-05-08T05:45:23.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.958+0000 7fa8db2d1640 1 -- start start 2024-05-08T05:45:23.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.959+0000 7fa8db2d1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8d415f380 0x7fa8d40d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:23.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.959+0000 7fa8db2d1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa8d40d2600 0x7fa8d40d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:23.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.959+0000 7fa8db2d1640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8d40b1c90 con 0x7fa8d40b3f30 2024-05-08T05:45:23.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.959+0000 7fa8db2d1640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa8d40d0a50 con 0x7fa8d40d2600 2024-05-08T05:45:23.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.959+0000 7fa8db2d1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa8d40d0d30 con 0x7fa8d415f380 2024-05-08T05:45:23.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.959+0000 7fa8d986e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa8d40d2600 0x7fa8d40d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:23.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.959+0000 7fa8d986e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa8d40d2600 0x7fa8d40d3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56424/0 (socket says 172.21.15.45:56424) 2024-05-08T05:45:23.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.959+0000 7fa8d986e640 1 -- 172.21.15.45:0/1733096369 learned_addr learned my addr 172.21.15.45:0/1733096369 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:23.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.960+0000 7fa8d906d640 1 --2- 172.21.15.45:0/1733096369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8d415f380 0x7fa8d40d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:23.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.960+0000 7fa8d986e640 1 -- 172.21.15.45:0/1733096369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8d415f380 msgr2=0x7fa8d40d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:23.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.960+0000 7fa8d986e640 1 --2- 172.21.15.45:0/1733096369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8d415f380 0x7fa8d40d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.960+0000 7fa8d986e640 1 -- 172.21.15.45:0/1733096369 >> v1:172.21.15.63:6789/0 conn(0x7fa8d40b3f30 legacy=0x7fa8d40b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:23.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.960+0000 7fa8d986e640 1 -- 172.21.15.45:0/1733096369 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d40b1f00 con 0x7fa8d40d2600 2024-05-08T05:45:23.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.960+0000 7fa8d986e640 1 --2- 172.21.15.45:0/1733096369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d40d2600 0x7fa8d40d3b00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009870 tx=0x7fa8cc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456ad1fdba56b994 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:23.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.961+0000 7fa8d3fff640 1 -- 172.21.15.45:0/1733096369 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc037070 con 0x7fa8d40d2600 2024-05-08T05:45:23.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.961+0000 7fa8d3fff640 1 -- 172.21.15.45:0/1733096369 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8cc032440 con 0x7fa8d40d2600 2024-05-08T05:45:23.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.961+0000 7fa8d3fff640 1 -- 172.21.15.45:0/1733096369 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc032810 con 0x7fa8d40d2600 2024-05-08T05:45:23.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.961+0000 7fa8db2d1640 1 -- 172.21.15.45:0/1733096369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d40d2600 msgr2=0x7fa8d40d3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:23.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.961+0000 7fa8db2d1640 1 --2- 172.21.15.45:0/1733096369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d40d2600 0x7fa8d40d3b00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009870 tx=0x7fa8cc02a1c0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.961+0000 7fa8db2d1640 1 -- 172.21.15.45:0/1733096369 shutdown_connections 2024-05-08T05:45:23.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.961+0000 7fa8db2d1640 1 --2- 172.21.15.45:0/1733096369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d40d2600 0x7fa8d40d3b00 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.961+0000 7fa8db2d1640 1 --2- 172.21.15.45:0/1733096369 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8d415f380 0x7fa8d40d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.961+0000 7fa8db2d1640 1 -- 172.21.15.45:0/1733096369 >> 172.21.15.45:0/1733096369 conn(0x7fa8d40d4fe0 msgr2=0x7fa8d40d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.961+0000 7fa8db2d1640 1 -- 172.21.15.45:0/1733096369 shutdown_connections 2024-05-08T05:45:23.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.961+0000 7fa8db2d1640 1 -- 172.21.15.45:0/1733096369 wait complete. 2024-05-08T05:45:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.962+0000 7fa8db2d1640 1 Processor -- start 2024-05-08T05:45:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.962+0000 7fa8db2d1640 1 -- start start 2024-05-08T05:45:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.963+0000 7fa8db2d1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d40b3f30 0x7fa8d40b9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.963+0000 7fa8db2d1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8d40d2600 0x7fa8d40b9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.963+0000 7fa8db2d1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8d415f380 0x7fa8d40ba2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.963+0000 7fa8db2d1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d40d06e0 con 0x7fa8d40b3f30 2024-05-08T05:45:23.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.963+0000 7fa8db2d1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d40d1010 con 0x7fa8d415f380 2024-05-08T05:45:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.963+0000 7fa8db2d1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa8d40b2b30 con 0x7fa8d40d2600 2024-05-08T05:45:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.963+0000 7fa8d906d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d40b3f30 0x7fa8d40b9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.963+0000 7fa8d906d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d40b3f30 0x7fa8d40b9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56432/0 (socket says 172.21.15.45:56432) 2024-05-08T05:45:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.963+0000 7fa8d906d640 1 -- 172.21.15.45:0/2204330194 learned_addr learned my addr 172.21.15.45:0/2204330194 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.964+0000 7fa8d986e640 1 --2- 172.21.15.45:0/2204330194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8d415f380 0x7fa8d40ba2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.964+0000 7fa8d886c640 1 --2- 172.21.15.45:0/2204330194 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8d40d2600 0x7fa8d40b9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.964+0000 7fa8d906d640 1 -- 172.21.15.45:0/2204330194 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8d40d2600 msgr2=0x7fa8d40b9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.964+0000 7fa8d906d640 1 --2- 172.21.15.45:0/2204330194 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8d40d2600 0x7fa8d40b9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.964+0000 7fa8d906d640 1 -- 172.21.15.45:0/2204330194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8d415f380 msgr2=0x7fa8d40ba2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.964+0000 7fa8d906d640 1 --2- 172.21.15.45:0/2204330194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8d415f380 0x7fa8d40ba2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.964+0000 7fa8d906d640 1 -- 172.21.15.45:0/2204330194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d4162870 con 0x7fa8d40b3f30 2024-05-08T05:45:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.964+0000 7fa8d906d640 1 --2- 172.21.15.45:0/2204330194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d40b3f30 0x7fa8d40b9840 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400ccd0 tx=0x7fa8c400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.964+0000 7fa8d1ffb640 1 -- 172.21.15.45:0/2204330194 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4003990 con 0x7fa8d40b3f30 2024-05-08T05:45:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.964+0000 7fa8d1ffb640 1 -- 172.21.15.45:0/2204330194 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8c4003b30 con 0x7fa8d40b3f30 2024-05-08T05:45:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.964+0000 7fa8db2d1640 1 -- 172.21.15.45:0/2204330194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d4162590 con 0x7fa8d40b3f30 2024-05-08T05:45:23.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.964+0000 7fa8d1ffb640 1 -- 172.21.15.45:0/2204330194 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4004020 con 0x7fa8d40b3f30 2024-05-08T05:45:23.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.964+0000 7fa8db2d1640 1 -- 172.21.15.45:0/2204330194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d41607e0 con 0x7fa8d40b3f30 2024-05-08T05:45:23.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.965+0000 7fa8db2d1640 1 -- 172.21.15.45:0/2204330194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8d40693f0 con 0x7fa8d40b3f30 2024-05-08T05:45:23.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.966+0000 7fa8d1ffb640 1 -- 172.21.15.45:0/2204330194 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa8c40041c0 con 0x7fa8d40b3f30 2024-05-08T05:45:23.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.966+0000 7fa8d1ffb640 1 --2- 172.21.15.45:0/2204330194 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa8c00762d0 0x7fa8c0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:23.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.967+0000 7fa8d1ffb640 1 -- 172.21.15.45:0/2204330194 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6001+0+0 (secure 0 0 0) 0x7fa8c4097b90 con 0x7fa8d40b3f30 2024-05-08T05:45:23.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.967+0000 7fa8d886c640 1 --2- 172.21.15.45:0/2204330194 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa8c00762d0 0x7fa8c0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:23.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.967+0000 7fa8d886c640 1 --2- 172.21.15.45:0/2204330194 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa8c00762d0 0x7fa8c0078790 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40cbc50 tx=0x7fa8c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:23.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:23.971+0000 7fa8d1ffb640 1 -- 172.21.15.45:0/2204330194 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa8c4014030 con 0x7fa8d40b3f30 2024-05-08T05:45:24.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.205+0000 7fa8db2d1640 1 -- 172.21.15.45:0/2204330194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa8d406d870 con 0x7fa8d40b3f30 2024-05-08T05:45:24.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.821+0000 7fa8d1ffb640 1 -- 172.21.15.45:0/2204330194 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v40) ==== 174+0+0 (secure 0 0 0) 0x7fa8c4061450 con 0x7fa8d40b3f30 2024-05-08T05:45:24.822 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 11 and data pool 12 2024-05-08T05:45:24.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.826+0000 7fa8db2d1640 1 -- 172.21.15.45:0/2204330194 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa8c00762d0 msgr2=0x7fa8c0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:24.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.826+0000 7fa8db2d1640 1 --2- 172.21.15.45:0/2204330194 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa8c00762d0 0x7fa8c0078790 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40cbc50 tx=0x7fa8c8009040 comp rx=0 tx=0).stop 2024-05-08T05:45:24.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.826+0000 7fa8db2d1640 1 -- 172.21.15.45:0/2204330194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d40b3f30 msgr2=0x7fa8d40b9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:24.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.826+0000 7fa8db2d1640 1 --2- 172.21.15.45:0/2204330194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d40b3f30 0x7fa8d40b9840 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400ccd0 tx=0x7fa8c400a5c0 comp rx=0 tx=0).stop 2024-05-08T05:45:24.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.827+0000 7fa8db2d1640 1 -- 172.21.15.45:0/2204330194 shutdown_connections 2024-05-08T05:45:24.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.827+0000 7fa8db2d1640 1 --2- 172.21.15.45:0/2204330194 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa8c00762d0 0x7fa8c0078790 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:24.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.827+0000 7fa8db2d1640 1 --2- 172.21.15.45:0/2204330194 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa8d40d2600 0x7fa8d40b9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:24.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.827+0000 7fa8db2d1640 1 --2- 172.21.15.45:0/2204330194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa8d415f380 0x7fa8d40ba2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:24.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.827+0000 7fa8db2d1640 1 --2- 172.21.15.45:0/2204330194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8d40b3f30 0x7fa8d40b9840 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:24.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.827+0000 7fa8db2d1640 1 -- 172.21.15.45:0/2204330194 >> 172.21.15.45:0/2204330194 conn(0x7fa8d40d4fe0 msgr2=0x7fa8d40d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:24.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.830+0000 7fa8db2d1640 1 -- 172.21.15.45:0/2204330194 shutdown_connections 2024-05-08T05:45:24.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.831+0000 7fa8db2d1640 1 -- 172.21.15.45:0/2204330194 wait complete. 2024-05-08T05:45:24.864 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:45:24.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.969+0000 7f7771746640 1 Processor -- start 2024-05-08T05:45:24.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.969+0000 7f7771746640 1 -- start start 2024-05-08T05:45:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.970+0000 7f7771746640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f776c15bc50 0x7f776c0b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.970+0000 7f7771746640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f776c06f540 0x7f776c070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.970+0000 7f7771746640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f776c071020 0x7f776c0732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.970+0000 7f7771746640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f776c06fa40 con 0x7f776c071020 2024-05-08T05:45:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.970+0000 7f7771746640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f776c06fd20 con 0x7f776c15bc50 2024-05-08T05:45:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.970+0000 7f7771746640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f776c070000 con 0x7f776c06f540 2024-05-08T05:45:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.970+0000 7f776b7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f776c071020 0x7f776c0732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.970+0000 7f776b7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f776c071020 0x7f776c0732c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56446/0 (socket says 172.21.15.45:56446) 2024-05-08T05:45:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.970+0000 7f776b7fe640 1 -- 172.21.15.45:0/673511364 learned_addr learned my addr 172.21.15.45:0/673511364 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.971+0000 7f776a7fc640 1 --2- 172.21.15.45:0/673511364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f776c06f540 0x7f776c070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.971+0000 7f776affd640 1 --2- 172.21.15.45:0/673511364 >> v2:172.21.15.63:3300/0 conn(0x7f776c15bc50 0x7f776c0b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:24.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.971+0000 7f776b7fe640 1 -- 172.21.15.45:0/673511364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f776c06f540 msgr2=0x7f776c070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.971+0000 7f776b7fe640 1 --2- 172.21.15.45:0/673511364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f776c06f540 0x7f776c070ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.971+0000 7f776b7fe640 1 -- 172.21.15.45:0/673511364 >> v2:172.21.15.63:3300/0 conn(0x7f776c15bc50 msgr2=0x7f776c0b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.971+0000 7f776b7fe640 1 --2- 172.21.15.45:0/673511364 >> v2:172.21.15.63:3300/0 conn(0x7f776c15bc50 0x7f776c0b5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.971+0000 7f776b7fe640 1 -- 172.21.15.45:0/673511364 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c0702e0 con 0x7f776c071020 2024-05-08T05:45:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.971+0000 7f776b7fe640 1 --2- 172.21.15.45:0/673511364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c071020 0x7f776c0732c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f776c068ee0 tx=0x7f775c02c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e19ad25b728a8c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.972+0000 7f7769ffb640 1 -- 172.21.15.45:0/673511364 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c032790 con 0x7f776c071020 2024-05-08T05:45:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.972+0000 7f7769ffb640 1 -- 172.21.15.45:0/673511364 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f775c038070 con 0x7f776c071020 2024-05-08T05:45:24.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.972+0000 7f7769ffb640 1 -- 172.21.15.45:0/673511364 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c032b50 con 0x7f776c071020 2024-05-08T05:45:24.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.972+0000 7f7771746640 1 -- 172.21.15.45:0/673511364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c071020 msgr2=0x7f776c0732c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:24.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.972+0000 7f7771746640 1 --2- 172.21.15.45:0/673511364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c071020 0x7f776c0732c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f776c068ee0 tx=0x7f775c02c890 comp rx=0 tx=0).stop 2024-05-08T05:45:24.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.972+0000 7f7771746640 1 -- 172.21.15.45:0/673511364 shutdown_connections 2024-05-08T05:45:24.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.972+0000 7f7771746640 1 --2- 172.21.15.45:0/673511364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c071020 0x7f776c0732c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:24.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.972+0000 7f7771746640 1 --2- 172.21.15.45:0/673511364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f776c06f540 0x7f776c070ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:24.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.972+0000 7f7771746640 1 --2- 172.21.15.45:0/673511364 >> v2:172.21.15.63:3300/0 conn(0x7f776c15bc50 0x7f776c0b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:24.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.972+0000 7f7771746640 1 -- 172.21.15.45:0/673511364 >> 172.21.15.45:0/673511364 conn(0x7f776c0b3c40 msgr2=0x7f776c0b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:24.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.972+0000 7f7771746640 1 -- 172.21.15.45:0/673511364 shutdown_connections 2024-05-08T05:45:24.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.972+0000 7f7771746640 1 -- 172.21.15.45:0/673511364 wait complete. 2024-05-08T05:45:24.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.974+0000 7f7771746640 1 Processor -- start 2024-05-08T05:45:24.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.974+0000 7f7771746640 1 -- start start 2024-05-08T05:45:24.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.974+0000 7f7771746640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c06f540 0x7f776c145a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:24.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.974+0000 7f7771746640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f776c15bc50 0x7f776c1552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:24.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.974+0000 7f7771746640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776c155820 0x7f776c150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:24.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.974+0000 7f7771746640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f776c0b6260 con 0x7f776c06f540 2024-05-08T05:45:24.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.974+0000 7f7771746640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f776c0b60e0 con 0x7f776c155820 2024-05-08T05:45:24.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.974+0000 7f7771746640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f776c15c030 con 0x7f776c15bc50 2024-05-08T05:45:24.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.974+0000 7f776affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c06f540 0x7f776c145a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:24.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.975+0000 7f776affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c06f540 0x7f776c145a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56448/0 (socket says 172.21.15.45:56448) 2024-05-08T05:45:24.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.975+0000 7f776affd640 1 -- 172.21.15.45:0/900245968 learned_addr learned my addr 172.21.15.45:0/900245968 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:24.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.975+0000 7f776b7fe640 1 --2- 172.21.15.45:0/900245968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776c155820 0x7f776c150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:24.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.975+0000 7f776a7fc640 1 --2- 172.21.15.45:0/900245968 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f776c15bc50 0x7f776c1552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.975+0000 7f776affd640 1 -- 172.21.15.45:0/900245968 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f776c15bc50 msgr2=0x7f776c1552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.975+0000 7f776affd640 1 --2- 172.21.15.45:0/900245968 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f776c15bc50 0x7f776c1552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.975+0000 7f776affd640 1 -- 172.21.15.45:0/900245968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776c155820 msgr2=0x7f776c150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.975+0000 7f776affd640 1 --2- 172.21.15.45:0/900245968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776c155820 0x7f776c150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.975+0000 7f776affd640 1 -- 172.21.15.45:0/900245968 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c1464b0 con 0x7f776c06f540 2024-05-08T05:45:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.975+0000 7f776affd640 1 --2- 172.21.15.45:0/900245968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c06f540 0x7f776c145a60 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7764009d10 tx=0x7f776400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.975+0000 7f77637fe640 1 -- 172.21.15.45:0/900245968 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764018020 con 0x7f776c06f540 2024-05-08T05:45:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.975+0000 7f77637fe640 1 -- 172.21.15.45:0/900245968 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7764014480 con 0x7f776c06f540 2024-05-08T05:45:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.975+0000 7f7771746640 1 -- 172.21.15.45:0/900245968 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776c155fc0 con 0x7f776c06f540 2024-05-08T05:45:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.975+0000 7f77637fe640 1 -- 172.21.15.45:0/900245968 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764014830 con 0x7f776c06f540 2024-05-08T05:45:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.975+0000 7f7771746640 1 -- 172.21.15.45:0/900245968 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f776c156580 con 0x7f776c06f540 2024-05-08T05:45:24.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.976+0000 7f7771746640 1 -- 172.21.15.45:0/900245968 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f776c0b7000 con 0x7f776c06f540 2024-05-08T05:45:24.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.977+0000 7f77637fe640 1 -- 172.21.15.45:0/900245968 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7764005d10 con 0x7f776c06f540 2024-05-08T05:45:24.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.977+0000 7f77637fe640 1 --2- 172.21.15.45:0/900245968 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f774c0762d0 0x7f774c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:24.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.978+0000 7f77637fe640 1 -- 172.21.15.45:0/900245968 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f7764098030 con 0x7f776c06f540 2024-05-08T05:45:24.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.978+0000 7f776a7fc640 1 --2- 172.21.15.45:0/900245968 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f774c0762d0 0x7f774c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:24.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.978+0000 7f776a7fc640 1 --2- 172.21.15.45:0/900245968 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f774c0762d0 0x7f774c078790 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f776c0b1770 tx=0x7f7754009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:24.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:24.982+0000 7f77637fe640 1 -- 172.21.15.45:0/900245968 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f776c0b7000 con 0x7f776c06f540 2024-05-08T05:45:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.173+0000 7f7771746640 1 -- 172.21.15.45:0/900245968 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f776c0cebe0 con 0x7f776c06f540 2024-05-08T05:45:25.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.175+0000 7f77637fe640 1 -- 172.21.15.45:0/900245968 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16910 (secure 0 0 0) 0x7f776c0cebe0 con 0x7f776c06f540 2024-05-08T05:45:25.176 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:25.176 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":74,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:45:24.208193+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:45:21.867649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-08T05:45:23.164376+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:45:25.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.176+0000 7f7771746640 1 -- 172.21.15.45:0/900245968 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f774c0762d0 msgr2=0x7f774c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:25.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.176+0000 7f7771746640 1 --2- 172.21.15.45:0/900245968 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f774c0762d0 0x7f774c078790 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f776c0b1770 tx=0x7f7754009190 comp rx=0 tx=0).stop 2024-05-08T05:45:25.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.176+0000 7f7771746640 1 -- 172.21.15.45:0/900245968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c06f540 msgr2=0x7f776c145a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:25.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.176+0000 7f7771746640 1 --2- 172.21.15.45:0/900245968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c06f540 0x7f776c145a60 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7764009d10 tx=0x7f776400c490 comp rx=0 tx=0).stop 2024-05-08T05:45:25.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.177+0000 7f7771746640 1 -- 172.21.15.45:0/900245968 shutdown_connections 2024-05-08T05:45:25.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.177+0000 7f7771746640 1 --2- 172.21.15.45:0/900245968 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f774c0762d0 0x7f774c078790 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:25.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.177+0000 7f7771746640 1 --2- 172.21.15.45:0/900245968 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f776c15bc50 0x7f776c1552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:25.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.177+0000 7f7771746640 1 --2- 172.21.15.45:0/900245968 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776c155820 0x7f776c150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:25.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.177+0000 7f7771746640 1 --2- 172.21.15.45:0/900245968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c06f540 0x7f776c145a60 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:25.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.177+0000 7f7771746640 1 -- 172.21.15.45:0/900245968 >> 172.21.15.45:0/900245968 conn(0x7f776c0b3c40 msgr2=0x7f776c0b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:25.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.177+0000 7f7771746640 1 -- 172.21.15.45:0/900245968 shutdown_connections 2024-05-08T05:45:25.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.177+0000 7f7771746640 1 -- 172.21.15.45:0/900245968 wait complete. 2024-05-08T05:45:25.189 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:45:25.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.292+0000 7f350f958640 1 Processor -- start 2024-05-08T05:45:25.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.292+0000 7f350f958640 1 -- start start 2024-05-08T05:45:25.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.293+0000 7f350f958640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35081629b0 0x7f350806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:25.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.293+0000 7f350f958640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f35080b2490 0x7f3508070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:25.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.293+0000 7f350f958640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f350806e110 con 0x7f3508070d20 2024-05-08T05:45:25.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.293+0000 7f350f958640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f350806dec0 con 0x7f35080b2490 2024-05-08T05:45:25.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.293+0000 7f350f958640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f350806e3f0 con 0x7f35081629b0 2024-05-08T05:45:25.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.293+0000 7f350d6f4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35081629b0 0x7f350806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:25.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.293+0000 7f350cef3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f35080b2490 0x7f3508070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:25.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.293+0000 7f350d6f4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35081629b0 0x7f350806eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:35320/0 (socket says 172.21.15.45:35320) 2024-05-08T05:45:25.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.293+0000 7f350d6f4640 1 -- 172.21.15.45:0/17218118 learned_addr learned my addr 172.21.15.45:0/17218118 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:25.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.293+0000 7f350def5640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f3508070d20 0x7f35080afcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:50812/0 (socket says 172.21.15.45:50812) 2024-05-08T05:45:25.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.293+0000 7f350d6f4640 1 -- 172.21.15.45:0/17218118 >> v2:172.21.15.63:3300/0 conn(0x7f35080b2490 msgr2=0x7f3508070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.293+0000 7f350d6f4640 1 --2- 172.21.15.45:0/17218118 >> v2:172.21.15.63:3300/0 conn(0x7f35080b2490 0x7f3508070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.293+0000 7f350d6f4640 1 -- 172.21.15.45:0/17218118 >> v1:172.21.15.63:6789/0 conn(0x7f3508070d20 legacy=0x7f35080afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.293+0000 7f350d6f4640 1 -- 172.21.15.45:0/17218118 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350806e6d0 con 0x7f35081629b0 2024-05-08T05:45:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.293+0000 7f350cef3640 1 --2- 172.21.15.45:0/17218118 >> v2:172.21.15.63:3300/0 conn(0x7f35080b2490 0x7f3508070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:45:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.294+0000 7f350d6f4640 1 --2- 172.21.15.45:0/17218118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35081629b0 0x7f350806eed0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3504009870 tx=0x7f350402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac3ff46f8494749b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.294+0000 7f34fffff640 1 -- 172.21.15.45:0/17218118 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504037070 con 0x7f35081629b0 2024-05-08T05:45:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.294+0000 7f34fffff640 1 -- 172.21.15.45:0/17218118 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3504032440 con 0x7f35081629b0 2024-05-08T05:45:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.294+0000 7f34fffff640 1 -- 172.21.15.45:0/17218118 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504032810 con 0x7f35081629b0 2024-05-08T05:45:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.295+0000 7f350f958640 1 -- 172.21.15.45:0/17218118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35081629b0 msgr2=0x7f350806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.295+0000 7f350f958640 1 --2- 172.21.15.45:0/17218118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35081629b0 0x7f350806eed0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3504009870 tx=0x7f350402a250 comp rx=0 tx=0).stop 2024-05-08T05:45:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.295+0000 7f350f958640 1 -- 172.21.15.45:0/17218118 shutdown_connections 2024-05-08T05:45:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.295+0000 7f350f958640 1 --2- 172.21.15.45:0/17218118 >> v2:172.21.15.63:3300/0 conn(0x7f35080b2490 0x7f3508070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.295+0000 7f350f958640 1 --2- 172.21.15.45:0/17218118 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35081629b0 0x7f350806eed0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:25.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.295+0000 7f350f958640 1 -- 172.21.15.45:0/17218118 >> 172.21.15.45:0/17218118 conn(0x7f35080b49d0 msgr2=0x7f35080b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:25.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.295+0000 7f350f958640 1 -- 172.21.15.45:0/17218118 shutdown_connections 2024-05-08T05:45:25.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.295+0000 7f350f958640 1 -- 172.21.15.45:0/17218118 wait complete. 2024-05-08T05:45:25.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.296+0000 7f350f958640 1 Processor -- start 2024-05-08T05:45:25.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.296+0000 7f350f958640 1 -- start start 2024-05-08T05:45:25.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.297+0000 7f350f958640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3508070d20 0x7f35081558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:25.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.297+0000 7f350f958640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35080b2490 0x7f35080cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:25.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.297+0000 7f350f958640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35081629b0 0x7f35080d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:25.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.297+0000 7f350f958640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3508070870 con 0x7f35080b2490 2024-05-08T05:45:25.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.297+0000 7f350f958640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f350806f410 con 0x7f35081629b0 2024-05-08T05:45:25.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.297+0000 7f350f958640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f35080709f0 con 0x7f3508070d20 2024-05-08T05:45:25.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.297+0000 7f350cef3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35080b2490 0x7f35080cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:25.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.297+0000 7f350cef3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35080b2490 0x7f35080cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56462/0 (socket says 172.21.15.45:56462) 2024-05-08T05:45:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.297+0000 7f350cef3640 1 -- 172.21.15.45:0/2287822079 learned_addr learned my addr 172.21.15.45:0/2287822079 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.297+0000 7f350def5640 1 --2- 172.21.15.45:0/2287822079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35081629b0 0x7f35080d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.297+0000 7f350d6f4640 1 --2- 172.21.15.45:0/2287822079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3508070d20 0x7f35081558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.298+0000 7f350cef3640 1 -- 172.21.15.45:0/2287822079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3508070d20 msgr2=0x7f35081558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.298+0000 7f350cef3640 1 --2- 172.21.15.45:0/2287822079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3508070d20 0x7f35081558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.298+0000 7f350cef3640 1 -- 172.21.15.45:0/2287822079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35081629b0 msgr2=0x7f35080d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.298+0000 7f350cef3640 1 --2- 172.21.15.45:0/2287822079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35081629b0 0x7f35080d1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.298+0000 7f350cef3640 1 -- 172.21.15.45:0/2287822079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35080ce4a0 con 0x7f35080b2490 2024-05-08T05:45:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.298+0000 7f350d6f4640 1 --2- 172.21.15.45:0/2287822079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3508070d20 0x7f35081558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.298+0000 7f350cef3640 1 --2- 172.21.15.45:0/2287822079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35080b2490 0x7f35080cde30 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f34f800ebe0 tx=0x7f34f800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.298+0000 7f34fdffb640 1 -- 172.21.15.45:0/2287822079 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8017070 con 0x7f35080b2490 2024-05-08T05:45:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.298+0000 7f34fdffb640 1 -- 172.21.15.45:0/2287822079 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f800cc30 con 0x7f35080b2490 2024-05-08T05:45:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.298+0000 7f34fdffb640 1 -- 172.21.15.45:0/2287822079 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8013470 con 0x7f35080b2490 2024-05-08T05:45:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.298+0000 7f350f958640 1 -- 172.21.15.45:0/2287822079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35080ce780 con 0x7f35080b2490 2024-05-08T05:45:25.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.298+0000 7f350f958640 1 -- 172.21.15.45:0/2287822079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35080ced40 con 0x7f35080b2490 2024-05-08T05:45:25.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.299+0000 7f350f958640 1 -- 172.21.15.45:0/2287822079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35080693f0 con 0x7f35080b2490 2024-05-08T05:45:25.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.300+0000 7f34fdffb640 1 -- 172.21.15.45:0/2287822079 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f34f801a070 con 0x7f35080b2490 2024-05-08T05:45:25.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.300+0000 7f34fdffb640 1 --2- 172.21.15.45:0/2287822079 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f34ec0762d0 0x7f34ec078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:25.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.300+0000 7f34fdffb640 1 -- 172.21.15.45:0/2287822079 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7f34f8097a20 con 0x7f35080b2490 2024-05-08T05:45:25.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.301+0000 7f350d6f4640 1 --2- 172.21.15.45:0/2287822079 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f34ec0762d0 0x7f34ec078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:25.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.301+0000 7f350d6f4640 1 --2- 172.21.15.45:0/2287822079 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f34ec0762d0 0x7f34ec078790 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3504034000 tx=0x7f350402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:25.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.305+0000 7f34fdffb640 1 -- 172.21.15.45:0/2287822079 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f34f8060b90 con 0x7f35080b2490 2024-05-08T05:45:25.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.496+0000 7f350f958640 1 -- 172.21.15.45:0/2287822079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35080d02f0 con 0x7f35080b2490 2024-05-08T05:45:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.497+0000 7f34fdffb640 1 -- 172.21.15.45:0/2287822079 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16910 (secure 0 0 0) 0x7f34f8064e90 con 0x7f35080b2490 2024-05-08T05:45:25.498 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:25.498 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":74,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:45:24.208193+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:45:21.867649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-08T05:45:23.164376+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:45:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.499+0000 7f350f958640 1 -- 172.21.15.45:0/2287822079 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f34ec0762d0 msgr2=0x7f34ec078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.499+0000 7f350f958640 1 --2- 172.21.15.45:0/2287822079 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f34ec0762d0 0x7f34ec078790 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3504034000 tx=0x7f350402e000 comp rx=0 tx=0).stop 2024-05-08T05:45:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.499+0000 7f350f958640 1 -- 172.21.15.45:0/2287822079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35080b2490 msgr2=0x7f35080cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.499+0000 7f350f958640 1 --2- 172.21.15.45:0/2287822079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35080b2490 0x7f35080cde30 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f34f800ebe0 tx=0x7f34f800c520 comp rx=0 tx=0).stop 2024-05-08T05:45:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.499+0000 7f350f958640 1 -- 172.21.15.45:0/2287822079 shutdown_connections 2024-05-08T05:45:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.499+0000 7f350f958640 1 --2- 172.21.15.45:0/2287822079 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f34ec0762d0 0x7f34ec078790 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.499+0000 7f350f958640 1 --2- 172.21.15.45:0/2287822079 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35081629b0 0x7f35080d1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.499+0000 7f350f958640 1 --2- 172.21.15.45:0/2287822079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35080b2490 0x7f35080cde30 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.499+0000 7f350f958640 1 --2- 172.21.15.45:0/2287822079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3508070d20 0x7f35081558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.499+0000 7f350f958640 1 -- 172.21.15.45:0/2287822079 >> 172.21.15.45:0/2287822079 conn(0x7f35080b49d0 msgr2=0x7f3508161150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.499+0000 7f350f958640 1 -- 172.21.15.45:0/2287822079 shutdown_connections 2024-05-08T05:45:25.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.499+0000 7f350f958640 1 -- 172.21.15.45:0/2287822079 wait complete. 2024-05-08T05:45:25.512 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T05:45:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.617+0000 7fc525b89640 1 Processor -- start 2024-05-08T05:45:25.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.617+0000 7fc525b89640 1 -- start start 2024-05-08T05:45:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.618+0000 7fc525b89640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc520070d30 0x7fc5200d2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.618+0000 7fc525b89640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc520071110 0x7fc52006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.618+0000 7fc525b89640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc520160720 con 0x7fc5201639e0 2024-05-08T05:45:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.618+0000 7fc525b89640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc5200d1ca0 con 0x7fc520070d30 2024-05-08T05:45:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.618+0000 7fc525b89640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc5200d10d0 con 0x7fc520071110 2024-05-08T05:45:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.618+0000 7fc51f7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc520070d30 0x7fc5200d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.618+0000 7fc51effd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc5201639e0 0x7fc5200703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34650/0 (socket says 172.21.15.45:34650) 2024-05-08T05:45:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.618+0000 7fc51effd640 1 -- 172.21.15.45:0/2613375993 learned_addr learned my addr 172.21.15.45:0/2613375993 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.618+0000 7fc51f7fe640 1 -- 172.21.15.45:0/2613375993 >> v2:172.21.15.63:3300/0 conn(0x7fc520071110 msgr2=0x7fc52006d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.618+0000 7fc51ffff640 1 --2- 172.21.15.45:0/2613375993 >> v2:172.21.15.63:3300/0 conn(0x7fc520071110 0x7fc52006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:25.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.618+0000 7fc51e7fc640 1 -- 172.21.15.45:0/2613375993 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4193614269 0 0) 0x7fc520160720 con 0x7fc5201639e0 2024-05-08T05:45:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.618+0000 7fc51f7fe640 1 --2- 172.21.15.45:0/2613375993 >> v2:172.21.15.63:3300/0 conn(0x7fc520071110 0x7fc52006d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.618+0000 7fc51f7fe640 1 -- 172.21.15.45:0/2613375993 >> v1:172.21.15.45:6789/0 conn(0x7fc5201639e0 legacy=0x7fc5200703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:25.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.618+0000 7fc51f7fe640 1 -- 172.21.15.45:0/2613375993 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5200d1370 con 0x7fc520070d30 2024-05-08T05:45:25.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.619+0000 7fc51f7fe640 1 --2- 172.21.15.45:0/2613375993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc520070d30 0x7fc5200d2130 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc5200761d0 tx=0x7fc51002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04bd739c512ad7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:25.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.619+0000 7fc51e7fc640 1 -- 172.21.15.45:0/2613375993 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510037070 con 0x7fc520070d30 2024-05-08T05:45:25.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.619+0000 7fc51e7fc640 1 -- 172.21.15.45:0/2613375993 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc510032440 con 0x7fc520070d30 2024-05-08T05:45:25.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.619+0000 7fc51e7fc640 1 -- 172.21.15.45:0/2613375993 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510032810 con 0x7fc520070d30 2024-05-08T05:45:25.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.620+0000 7fc525b89640 1 -- 172.21.15.45:0/2613375993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc520070d30 msgr2=0x7fc5200d2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:25.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.620+0000 7fc525b89640 1 --2- 172.21.15.45:0/2613375993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc520070d30 0x7fc5200d2130 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc5200761d0 tx=0x7fc51002a1a0 comp rx=0 tx=0).stop 2024-05-08T05:45:25.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.620+0000 7fc525b89640 1 -- 172.21.15.45:0/2613375993 shutdown_connections 2024-05-08T05:45:25.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.620+0000 7fc525b89640 1 --2- 172.21.15.45:0/2613375993 >> v2:172.21.15.63:3300/0 conn(0x7fc520071110 0x7fc52006d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:25.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.620+0000 7fc525b89640 1 --2- 172.21.15.45:0/2613375993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc520070d30 0x7fc5200d2130 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:25.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.620+0000 7fc525b89640 1 -- 172.21.15.45:0/2613375993 >> 172.21.15.45:0/2613375993 conn(0x7fc5200d2e90 msgr2=0x7fc5200d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:25.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.620+0000 7fc525b89640 1 -- 172.21.15.45:0/2613375993 shutdown_connections 2024-05-08T05:45:25.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.620+0000 7fc525b89640 1 -- 172.21.15.45:0/2613375993 wait complete. 2024-05-08T05:45:25.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.621+0000 7fc525b89640 1 Processor -- start 2024-05-08T05:45:25.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.621+0000 7fc525b89640 1 -- start start 2024-05-08T05:45:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.622+0000 7fc525b89640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc520070d30 0x7fc5201f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.622+0000 7fc525b89640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc520071110 0x7fc5200c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.622+0000 7fc525b89640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5201639e0 0x7fc5200c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.622+0000 7fc525b89640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc520071540 con 0x7fc520070d30 2024-05-08T05:45:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.622+0000 7fc525b89640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc5200d1930 con 0x7fc5201639e0 2024-05-08T05:45:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.622+0000 7fc525b89640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc5200716c0 con 0x7fc520071110 2024-05-08T05:45:25.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.622+0000 7fc51f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc520070d30 0x7fc5201f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.622+0000 7fc51f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc520070d30 0x7fc5201f5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56486/0 (socket says 172.21.15.45:56486) 2024-05-08T05:45:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.622+0000 7fc51f7fe640 1 -- 172.21.15.45:0/95262402 learned_addr learned my addr 172.21.15.45:0/95262402 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.622+0000 7fc51f7fe640 1 -- 172.21.15.45:0/95262402 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc520071110 msgr2=0x7fc5200c69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.622+0000 7fc51f7fe640 1 --2- 172.21.15.45:0/95262402 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc520071110 0x7fc5200c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.622+0000 7fc51f7fe640 1 -- 172.21.15.45:0/95262402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5201639e0 msgr2=0x7fc5200c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.622+0000 7fc51ffff640 1 --2- 172.21.15.45:0/95262402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5201639e0 0x7fc5200c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.622+0000 7fc51f7fe640 1 --2- 172.21.15.45:0/95262402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5201639e0 0x7fc5200c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.622+0000 7fc51f7fe640 1 -- 172.21.15.45:0/95262402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5200cbe30 con 0x7fc520070d30 2024-05-08T05:45:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.622+0000 7fc51f7fe640 1 --2- 172.21.15.45:0/95262402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc520070d30 0x7fc5201f5790 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc510000c00 tx=0x7fc5100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.623+0000 7fc507fff640 1 -- 172.21.15.45:0/95262402 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510037040 con 0x7fc520070d30 2024-05-08T05:45:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.623+0000 7fc507fff640 1 -- 172.21.15.45:0/95262402 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc510042070 con 0x7fc520070d30 2024-05-08T05:45:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.623+0000 7fc525b89640 1 -- 172.21.15.45:0/95262402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5200cc110 con 0x7fc520070d30 2024-05-08T05:45:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.623+0000 7fc507fff640 1 -- 172.21.15.45:0/95262402 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510030a70 con 0x7fc520070d30 2024-05-08T05:45:25.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.623+0000 7fc525b89640 1 -- 172.21.15.45:0/95262402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5200cc6d0 con 0x7fc520070d30 2024-05-08T05:45:25.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.624+0000 7fc525b89640 1 -- 172.21.15.45:0/95262402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc52006fc90 con 0x7fc520070d30 2024-05-08T05:45:25.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.624+0000 7fc507fff640 1 -- 172.21.15.45:0/95262402 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc510030c10 con 0x7fc520070d30 2024-05-08T05:45:25.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.625+0000 7fc507fff640 1 --2- 172.21.15.45:0/95262402 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc50c0762d0 0x7fc50c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:25.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.625+0000 7fc507fff640 1 -- 172.21.15.45:0/95262402 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6101+0+0 (secure 0 0 0) 0x7fc5100b82a0 con 0x7fc520070d30 2024-05-08T05:45:25.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.625+0000 7fc51effd640 1 --2- 172.21.15.45:0/95262402 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc50c0762d0 0x7fc50c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:25.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.626+0000 7fc51effd640 1 --2- 172.21.15.45:0/95262402 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc50c0762d0 0x7fc50c078790 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc5200d0910 tx=0x7fc5140078d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:25.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.629+0000 7fc507fff640 1 -- 172.21.15.45:0/95262402 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc52006fc90 con 0x7fc520070d30 2024-05-08T05:45:25.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:25.861+0000 7fc525b89640 1 -- 172.21.15.45:0/95262402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc520069d10 con 0x7fc520070d30 2024-05-08T05:45:26.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.837+0000 7fc507fff640 1 -- 172.21.15.45:0/95262402 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) ==== 118+0+0 (secure 0 0 0) 0x7fc520069d10 con 0x7fc520070d30 2024-05-08T05:45:26.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.840+0000 7fc525b89640 1 -- 172.21.15.45:0/95262402 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc50c0762d0 msgr2=0x7fc50c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:26.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.840+0000 7fc525b89640 1 --2- 172.21.15.45:0/95262402 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc50c0762d0 0x7fc50c078790 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc5200d0910 tx=0x7fc5140078d0 comp rx=0 tx=0).stop 2024-05-08T05:45:26.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.840+0000 7fc525b89640 1 -- 172.21.15.45:0/95262402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc520070d30 msgr2=0x7fc5201f5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:26.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.840+0000 7fc525b89640 1 --2- 172.21.15.45:0/95262402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc520070d30 0x7fc5201f5790 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc510000c00 tx=0x7fc5100046c0 comp rx=0 tx=0).stop 2024-05-08T05:45:26.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.840+0000 7fc525b89640 1 -- 172.21.15.45:0/95262402 shutdown_connections 2024-05-08T05:45:26.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.840+0000 7fc525b89640 1 --2- 172.21.15.45:0/95262402 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc50c0762d0 0x7fc50c078790 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:26.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.840+0000 7fc525b89640 1 --2- 172.21.15.45:0/95262402 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc520071110 0x7fc5200c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:26.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.840+0000 7fc525b89640 1 --2- 172.21.15.45:0/95262402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5201639e0 0x7fc5200c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:26.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.840+0000 7fc525b89640 1 --2- 172.21.15.45:0/95262402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc520070d30 0x7fc5201f5790 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:26.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.840+0000 7fc525b89640 1 -- 172.21.15.45:0/95262402 >> 172.21.15.45:0/95262402 conn(0x7fc5200d2e90 msgr2=0x7fc520162cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:26.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.841+0000 7fc525b89640 1 -- 172.21.15.45:0/95262402 shutdown_connections 2024-05-08T05:45:26.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.841+0000 7fc525b89640 1 -- 172.21.15.45:0/95262402 wait complete. 2024-05-08T05:45:26.855 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T05:45:26.856 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T05:45:26.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.967+0000 7fe0368fc640 1 Processor -- start 2024-05-08T05:45:26.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.967+0000 7fe0368fc640 1 -- start start 2024-05-08T05:45:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.968+0000 7fe0368fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe03006e0f0 0x7fe03015c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.968+0000 7fe0368fc640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe03015c920 con 0x7fe030072840 2024-05-08T05:45:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.968+0000 7fe0368fc640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe03015cc00 con 0x7fe03006e4d0 2024-05-08T05:45:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.968+0000 7fe0368fc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe03015cee0 con 0x7fe03006e0f0 2024-05-08T05:45:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.968+0000 7fe02f7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe030072840 0x7fe0300d0f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34662/0 (socket says 172.21.15.45:34662) 2024-05-08T05:45:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.968+0000 7fe02f7fe640 1 -- 172.21.15.45:0/2204992856 learned_addr learned my addr 172.21.15.45:0/2204992856 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.968+0000 7fe02effd640 1 -- 172.21.15.45:0/2204992856 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 463503823 0 0) 0x7fe03015c920 con 0x7fe030072840 2024-05-08T05:45:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.968+0000 7fe02ffff640 1 --2- 172.21.15.45:0/2204992856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe03006e0f0 0x7fe03015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:26.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.968+0000 7fe02effd640 1 -- 172.21.15.45:0/2204992856 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe018003520 con 0x7fe030072840 2024-05-08T05:45:26.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe02effd640 1 -- 172.21.15.45:0/2204992856 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3066434349 0 0) 0x7fe018003520 con 0x7fe030072840 2024-05-08T05:45:26.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe02effd640 1 -- 172.21.15.45:0/2204992856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe03006e0f0 msgr2=0x7fe03015c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:26.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe02effd640 1 --2- 172.21.15.45:0/2204992856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe03006e0f0 0x7fe03015c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:26.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe02effd640 1 -- 172.21.15.45:0/2204992856 >> v1:172.21.15.63:6789/0 conn(0x7fe03006e4d0 legacy=0x7fe0300aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:26.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe02effd640 1 -- 172.21.15.45:0/2204992856 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03015d1c0 con 0x7fe030072840 2024-05-08T05:45:26.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe02effd640 1 -- 172.21.15.45:0/2204992856 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fe0240016f0 con 0x7fe030072840 2024-05-08T05:45:26.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe02effd640 1 -- 172.21.15.45:0/2204992856 >> v1:172.21.15.45:6789/0 conn(0x7fe030072840 legacy=0x7fe0300d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:26.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe02effd640 1 --2- 172.21.15.45:0/2204992856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe018003a30 0x7fe018003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:26.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe02effd640 1 --2- 172.21.15.45:0/2204992856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe018004800 0x7fe018004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:26.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe02effd640 1 --2- 172.21.15.45:0/2204992856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe03006e0f0 0x7fe018005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:26.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe02effd640 1 -- 172.21.15.45:0/2204992856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe03015cee0 con 0x7fe018003a30 2024-05-08T05:45:26.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe02effd640 1 -- 172.21.15.45:0/2204992856 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe01800dd00 con 0x7fe018004800 2024-05-08T05:45:26.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe02effd640 1 -- 172.21.15.45:0/2204992856 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe01800df00 con 0x7fe03006e0f0 2024-05-08T05:45:26.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe02ffff640 1 --2- 172.21.15.45:0/2204992856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe018003a30 0x7fe018003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:26.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe02f7fe640 1 --2- 172.21.15.45:0/2204992856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe018004800 0x7fe018004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:26.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe034e99640 1 --2- 172.21.15.45:0/2204992856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe03006e0f0 0x7fe018005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:26.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe02ffff640 1 -- 172.21.15.45:0/2204992856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe03006e0f0 msgr2=0x7fe018005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:26.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe02ffff640 1 --2- 172.21.15.45:0/2204992856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe03006e0f0 0x7fe018005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:26.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe02ffff640 1 -- 172.21.15.45:0/2204992856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe018004800 msgr2=0x7fe018004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:26.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe02ffff640 1 --2- 172.21.15.45:0/2204992856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe018004800 0x7fe018004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:26.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe02ffff640 1 -- 172.21.15.45:0/2204992856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe018003520 con 0x7fe018003a30 2024-05-08T05:45:26.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.969+0000 7fe034e99640 1 --2- 172.21.15.45:0/2204992856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe03006e0f0 0x7fe018005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:26.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.970+0000 7fe02ffff640 1 --2- 172.21.15.45:0/2204992856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe018003a30 0x7fe018003e50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe020009ac0 tx=0x7fe018005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2818a23f3225cfa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:26.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.970+0000 7fe02effd640 1 -- 172.21.15.45:0/2204992856 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020036070 con 0x7fe018003a30 2024-05-08T05:45:26.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.970+0000 7fe02effd640 1 -- 172.21.15.45:0/2204992856 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe020031400 con 0x7fe018003a30 2024-05-08T05:45:26.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.970+0000 7fe02effd640 1 -- 172.21.15.45:0/2204992856 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020009690 con 0x7fe018003a30 2024-05-08T05:45:26.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.970+0000 7fe0368fc640 1 -- 172.21.15.45:0/2204992856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe018003a30 msgr2=0x7fe018003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:26.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.970+0000 7fe0368fc640 1 --2- 172.21.15.45:0/2204992856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe018003a30 0x7fe018003e50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe020009ac0 tx=0x7fe018005830 comp rx=0 tx=0).stop 2024-05-08T05:45:26.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.971+0000 7fe034e99640 1 -- 172.21.15.45:0/2204992856 reap_dead start 2024-05-08T05:45:26.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.971+0000 7fe0368fc640 1 -- 172.21.15.45:0/2204992856 shutdown_connections 2024-05-08T05:45:26.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.971+0000 7fe0368fc640 1 -- 172.21.15.45:0/2204992856 >> 172.21.15.45:0/2204992856 conn(0x7fe03006c3a0 msgr2=0x7fe03006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:26.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.971+0000 7fe0368fc640 1 -- 172.21.15.45:0/2204992856 shutdown_connections 2024-05-08T05:45:26.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.971+0000 7fe0368fc640 1 -- 172.21.15.45:0/2204992856 wait complete. 2024-05-08T05:45:26.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.972+0000 7fe0368fc640 1 Processor -- start 2024-05-08T05:45:26.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.972+0000 7fe0368fc640 1 -- start start 2024-05-08T05:45:26.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe0368fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe018003a30 0x7fe03009c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:26.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe0368fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0301525f0 0x7fe0301f5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:26.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe0368fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0301f5f80 0x7fe0301f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:26.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe0368fc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe0300ae7d0 con 0x7fe018003a30 2024-05-08T05:45:26.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe0368fc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe0300ae650 con 0x7fe0301525f0 2024-05-08T05:45:26.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe0368fc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe0301530b0 con 0x7fe0301f5f80 2024-05-08T05:45:26.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe02ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe018003a30 0x7fe03009c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:26.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe02ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe018003a30 0x7fe03009c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56502/0 (socket says 172.21.15.45:56502) 2024-05-08T05:45:26.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe02ffff640 1 -- 172.21.15.45:0/3950453544 learned_addr learned my addr 172.21.15.45:0/3950453544 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:26.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe02f7fe640 1 --2- 172.21.15.45:0/3950453544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0301525f0 0x7fe0301f5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:26.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe02ffff640 1 -- 172.21.15.45:0/3950453544 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0301f5f80 msgr2=0x7fe0301f64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:26.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe034e99640 1 --2- 172.21.15.45:0/3950453544 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0301f5f80 0x7fe0301f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:26.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe02ffff640 1 --2- 172.21.15.45:0/3950453544 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0301f5f80 0x7fe0301f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:26.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe02ffff640 1 -- 172.21.15.45:0/3950453544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0301525f0 msgr2=0x7fe0301f5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:26.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe02ffff640 1 --2- 172.21.15.45:0/3950453544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0301525f0 0x7fe0301f5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:26.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe02ffff640 1 -- 172.21.15.45:0/3950453544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0301e55c0 con 0x7fe018003a30 2024-05-08T05:45:26.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe02f7fe640 1 --2- 172.21.15.45:0/3950453544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0301525f0 0x7fe0301f5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:26.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe02ffff640 1 --2- 172.21.15.45:0/3950453544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe018003a30 0x7fe03009c350 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe020034040 tx=0x7fe020004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:26.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe02cff9640 1 -- 172.21.15.45:0/3950453544 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020040020 con 0x7fe018003a30 2024-05-08T05:45:26.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe02cff9640 1 -- 172.21.15.45:0/3950453544 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe020003bc0 con 0x7fe018003a30 2024-05-08T05:45:26.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe0368fc640 1 -- 172.21.15.45:0/3950453544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe03009e2e0 con 0x7fe018003a30 2024-05-08T05:45:26.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe02cff9640 1 -- 172.21.15.45:0/3950453544 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe020036040 con 0x7fe018003a30 2024-05-08T05:45:26.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.974+0000 7fe0368fc640 1 -- 172.21.15.45:0/3950453544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe03009e8a0 con 0x7fe018003a30 2024-05-08T05:45:26.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.975+0000 7fe0368fc640 1 -- 172.21.15.45:0/3950453544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe03015c550 con 0x7fe018003a30 2024-05-08T05:45:26.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.975+0000 7fe02cff9640 1 -- 172.21.15.45:0/3950453544 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe020043020 con 0x7fe018003a30 2024-05-08T05:45:26.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.976+0000 7fe02cff9640 1 --2- 172.21.15.45:0/3950453544 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe01c076300 0x7fe01c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:26.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.976+0000 7fe02cff9640 1 -- 172.21.15.45:0/3950453544 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fe0200b7880 con 0x7fe018003a30 2024-05-08T05:45:26.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.976+0000 7fe02f7fe640 1 --2- 172.21.15.45:0/3950453544 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe01c076300 0x7fe01c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:26.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.977+0000 7fe02f7fe640 1 --2- 172.21.15.45:0/3950453544 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe01c076300 0x7fe01c0787c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe0300b5a10 tx=0x7fe024009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:26.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:26.981+0000 7fe02cff9640 1 -- 172.21.15.45:0/3950453544 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe03015c550 con 0x7fe018003a30 2024-05-08T05:45:27.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.209+0000 7fe0368fc640 1 -- 172.21.15.45:0/3950453544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe0300693f0 con 0x7fe018003a30 2024-05-08T05:45:27.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.846+0000 7fe02cff9640 1 -- 172.21.15.45:0/3950453544 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7fe02004a070 con 0x7fe018003a30 2024-05-08T05:45:27.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.849+0000 7fe0368fc640 1 -- 172.21.15.45:0/3950453544 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe01c076300 msgr2=0x7fe01c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:27.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.849+0000 7fe0368fc640 1 --2- 172.21.15.45:0/3950453544 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe01c076300 0x7fe01c0787c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe0300b5a10 tx=0x7fe024009040 comp rx=0 tx=0).stop 2024-05-08T05:45:27.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.849+0000 7fe0368fc640 1 -- 172.21.15.45:0/3950453544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe018003a30 msgr2=0x7fe03009c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:27.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.849+0000 7fe0368fc640 1 --2- 172.21.15.45:0/3950453544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe018003a30 0x7fe03009c350 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe020034040 tx=0x7fe020004730 comp rx=0 tx=0).stop 2024-05-08T05:45:27.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.849+0000 7fe0368fc640 1 -- 172.21.15.45:0/3950453544 shutdown_connections 2024-05-08T05:45:27.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.849+0000 7fe0368fc640 1 --2- 172.21.15.45:0/3950453544 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe01c076300 0x7fe01c0787c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:27.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.849+0000 7fe0368fc640 1 --2- 172.21.15.45:0/3950453544 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe0301f5f80 0x7fe0301f64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:27.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.849+0000 7fe0368fc640 1 --2- 172.21.15.45:0/3950453544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe0301525f0 0x7fe0301f5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:27.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.849+0000 7fe0368fc640 1 --2- 172.21.15.45:0/3950453544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe018003a30 0x7fe03009c350 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:27.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.849+0000 7fe0368fc640 1 -- 172.21.15.45:0/3950453544 >> 172.21.15.45:0/3950453544 conn(0x7fe03006c3a0 msgr2=0x7fe03015d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:27.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.849+0000 7fe0368fc640 1 -- 172.21.15.45:0/3950453544 shutdown_connections 2024-05-08T05:45:27.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.849+0000 7fe0368fc640 1 -- 172.21.15.45:0/3950453544 wait complete. 2024-05-08T05:45:27.864 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T05:45:27.864 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:45:27.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.973+0000 7f968d66d640 1 Processor -- start 2024-05-08T05:45:27.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.973+0000 7f968d66d640 1 -- start start 2024-05-08T05:45:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.974+0000 7f968d66d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f968815bc50 0x7f96880b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.974+0000 7f968d66d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f968806f540 0x7f9688070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.974+0000 7f968d66d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f968806fa40 con 0x7f9688071020 2024-05-08T05:45:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.974+0000 7f968d66d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f968806fd20 con 0x7f968815bc50 2024-05-08T05:45:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.974+0000 7f968d66d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9688070000 con 0x7f968806f540 2024-05-08T05:45:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.975+0000 7f9686ffd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f968815bc50 0x7f96880b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.975+0000 7f96867fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f968806f540 0x7f9688070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.975+0000 7f96877fe640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f9688071020 0x7f96880732c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:50834/0 (socket says 172.21.15.45:50834) 2024-05-08T05:45:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.975+0000 7f9686ffd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f968815bc50 0x7f96880b5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:55056/0 (socket says 172.21.15.45:55056) 2024-05-08T05:45:27.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.975+0000 7f96877fe640 1 -- 172.21.15.45:0/663291586 learned_addr learned my addr 172.21.15.45:0/663291586 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:27.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.975+0000 7f96867fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f968806f540 0x7f9688070ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:35384/0 (socket says 172.21.15.45:35384) 2024-05-08T05:45:27.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.975+0000 7f96867fc640 1 -- 172.21.15.45:0/663291586 >> v2:172.21.15.63:3300/0 conn(0x7f968815bc50 msgr2=0x7f96880b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:27.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.975+0000 7f96867fc640 1 --2- 172.21.15.45:0/663291586 >> v2:172.21.15.63:3300/0 conn(0x7f968815bc50 0x7f96880b5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:27.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.975+0000 7f96867fc640 1 -- 172.21.15.45:0/663291586 >> v1:172.21.15.63:6789/0 conn(0x7f9688071020 legacy=0x7f96880732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:27.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.975+0000 7f96867fc640 1 -- 172.21.15.45:0/663291586 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96880702e0 con 0x7f968806f540 2024-05-08T05:45:27.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.975+0000 7f9686ffd640 1 --2- 172.21.15.45:0/663291586 >> v2:172.21.15.63:3300/0 conn(0x7f968815bc50 0x7f96880b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:45:27.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.976+0000 7f96867fc640 1 --2- 172.21.15.45:0/663291586 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f968806f540 0x7f9688070ae0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f967000ba40 tx=0x7f967002c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81d61d4218df8bcd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:27.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.976+0000 7f9685ffb640 1 -- 172.21.15.45:0/663291586 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670032750 con 0x7f968806f540 2024-05-08T05:45:27.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.976+0000 7f9685ffb640 1 -- 172.21.15.45:0/663291586 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9670030050 con 0x7f968806f540 2024-05-08T05:45:27.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.976+0000 7f9685ffb640 1 -- 172.21.15.45:0/663291586 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670032b10 con 0x7f968806f540 2024-05-08T05:45:27.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.976+0000 7f968d66d640 1 -- 172.21.15.45:0/663291586 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f968806f540 msgr2=0x7f9688070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:27.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.977+0000 7f968d66d640 1 --2- 172.21.15.45:0/663291586 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f968806f540 0x7f9688070ae0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f967000ba40 tx=0x7f967002c850 comp rx=0 tx=0).stop 2024-05-08T05:45:27.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.977+0000 7f968d66d640 1 -- 172.21.15.45:0/663291586 shutdown_connections 2024-05-08T05:45:27.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.977+0000 7f968d66d640 1 --2- 172.21.15.45:0/663291586 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f968806f540 0x7f9688070ae0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:27.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.977+0000 7f968d66d640 1 --2- 172.21.15.45:0/663291586 >> v2:172.21.15.63:3300/0 conn(0x7f968815bc50 0x7f96880b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:27.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.977+0000 7f968d66d640 1 -- 172.21.15.45:0/663291586 >> 172.21.15.45:0/663291586 conn(0x7f96880b3c40 msgr2=0x7f96880b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:27.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.977+0000 7f968d66d640 1 -- 172.21.15.45:0/663291586 shutdown_connections 2024-05-08T05:45:27.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.977+0000 7f968d66d640 1 -- 172.21.15.45:0/663291586 wait complete. 2024-05-08T05:45:27.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.978+0000 7f968d66d640 1 Processor -- start 2024-05-08T05:45:27.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.978+0000 7f968d66d640 1 -- start start 2024-05-08T05:45:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.979+0000 7f968d66d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f968806f540 0x7f968814f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.979+0000 7f968d66d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f968815bc50 0x7f9688150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.979+0000 7f968d66d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f968814fa90 0x7f9688152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.979+0000 7f968d66d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f96880b6260 con 0x7f968814fa90 2024-05-08T05:45:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.979+0000 7f968d66d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f96880b60e0 con 0x7f968806f540 2024-05-08T05:45:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.979+0000 7f968d66d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f968815c030 con 0x7f968815bc50 2024-05-08T05:45:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.979+0000 7f96877fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f968814fa90 0x7f9688152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.979+0000 7f9686ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f968806f540 0x7f968814f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.979+0000 7f9686ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f968806f540 0x7f968814f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:55058/0 (socket says 172.21.15.45:55058) 2024-05-08T05:45:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.979+0000 7f9686ffd640 1 -- 172.21.15.45:0/1558891465 learned_addr learned my addr 172.21.15.45:0/1558891465 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.979+0000 7f96877fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f968814fa90 0x7f9688152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56518/0 (socket says 172.21.15.45:56518) 2024-05-08T05:45:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.979+0000 7f96867fc640 1 --2- 172.21.15.45:0/1558891465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f968815bc50 0x7f9688150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.979+0000 7f9686ffd640 1 -- 172.21.15.45:0/1558891465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f968815bc50 msgr2=0x7f9688150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.979+0000 7f9686ffd640 1 --2- 172.21.15.45:0/1558891465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f968815bc50 0x7f9688150ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.979+0000 7f9686ffd640 1 -- 172.21.15.45:0/1558891465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f968814fa90 msgr2=0x7f9688152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.979+0000 7f9686ffd640 1 --2- 172.21.15.45:0/1558891465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f968814fa90 0x7f9688152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.979+0000 7f9686ffd640 1 -- 172.21.15.45:0/1558891465 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9688153310 con 0x7f968806f540 2024-05-08T05:45:27.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.979+0000 7f96867fc640 1 --2- 172.21.15.45:0/1558891465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f968815bc50 0x7f9688150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:45:27.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.979+0000 7f96877fe640 1 --2- 172.21.15.45:0/1558891465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f968814fa90 0x7f9688152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:45:27.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.979+0000 7f9686ffd640 1 --2- 172.21.15.45:0/1558891465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f968806f540 0x7f968814f1f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f967800efc0 tx=0x7f967800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:27.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.980+0000 7f967f7fe640 1 -- 172.21.15.45:0/1558891465 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678018020 con 0x7f968806f540 2024-05-08T05:45:27.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.980+0000 7f967f7fe640 1 -- 172.21.15.45:0/1558891465 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9678014680 con 0x7f968806f540 2024-05-08T05:45:27.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.980+0000 7f967f7fe640 1 -- 172.21.15.45:0/1558891465 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678014a50 con 0x7f968806f540 2024-05-08T05:45:27.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.980+0000 7f968d66d640 1 -- 172.21.15.45:0/1558891465 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96881535f0 con 0x7f968806f540 2024-05-08T05:45:27.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.980+0000 7f968d66d640 1 -- 172.21.15.45:0/1558891465 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9688153bb0 con 0x7f968806f540 2024-05-08T05:45:27.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.981+0000 7f968d66d640 1 -- 172.21.15.45:0/1558891465 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96880d02f0 con 0x7f968806f540 2024-05-08T05:45:27.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.981+0000 7f967f7fe640 1 -- 172.21.15.45:0/1558891465 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9678005d10 con 0x7f968806f540 2024-05-08T05:45:27.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.982+0000 7f967f7fe640 1 --2- 172.21.15.45:0/1558891465 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9668076300 0x7f96680787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:27.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.982+0000 7f967f7fe640 1 -- 172.21.15.45:0/1558891465 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f9678098530 con 0x7f968806f540 2024-05-08T05:45:27.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.982+0000 7f96867fc640 1 --2- 172.21.15.45:0/1558891465 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9668076300 0x7f96680787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:27.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.983+0000 7f96867fc640 1 --2- 172.21.15.45:0/1558891465 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9668076300 0x7f96680787c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9670037000 tx=0x7f9670032000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:27.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:27.988+0000 7f967f7fe640 1 -- 172.21.15.45:0/1558891465 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f96880d02f0 con 0x7f968806f540 2024-05-08T05:45:28.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.217+0000 7f968d66d640 1 -- 172.21.15.45:0/1558891465 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9688069b00 con 0x7f968806f540 2024-05-08T05:45:28.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.218+0000 7f967f7fe640 1 -- 172.21.15.45:0/1558891465 <== mon.1 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5180 (secure 0 0 0) 0x7f9678022030 con 0x7f968806f540 2024-05-08T05:45:28.221 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:28.221 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5722,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1283757699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1283757699},{"type":"v1","addr":"172.21.15.63:6837","nonce":1283757699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1963062796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1963062796},{"type":"v1","addr":"172.21.15.45:6834","nonce":1963062796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6171,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4183668301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4183668301},{"type":"v1","addr":"172.21.15.45:6835","nonce":4183668301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:45:24.208158+0000","modified":"2024-05-08T05:45:27.211503+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6180":{"gid":6180,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6836/133050464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":133050464},{"type":"v1","addr":"172.21.15.63:6836","nonce":133050464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6180,"qdb_cluster":[6180]},"id":5}]} 2024-05-08T05:45:28.222 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 44 2024-05-08T05:45:28.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.220+0000 7f968d66d640 1 -- 172.21.15.45:0/1558891465 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9668076300 msgr2=0x7f96680787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:28.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.220+0000 7f968d66d640 1 --2- 172.21.15.45:0/1558891465 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9668076300 0x7f96680787c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9670037000 tx=0x7f9670032000 comp rx=0 tx=0).stop 2024-05-08T05:45:28.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.220+0000 7f968d66d640 1 -- 172.21.15.45:0/1558891465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f968806f540 msgr2=0x7f968814f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:28.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.220+0000 7f968d66d640 1 --2- 172.21.15.45:0/1558891465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f968806f540 0x7f968814f1f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f967800efc0 tx=0x7f967800c490 comp rx=0 tx=0).stop 2024-05-08T05:45:28.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.220+0000 7f968d66d640 1 -- 172.21.15.45:0/1558891465 shutdown_connections 2024-05-08T05:45:28.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.220+0000 7f968d66d640 1 --2- 172.21.15.45:0/1558891465 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9668076300 0x7f96680787c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.220+0000 7f968d66d640 1 --2- 172.21.15.45:0/1558891465 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f968815bc50 0x7f9688150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.220+0000 7f968d66d640 1 --2- 172.21.15.45:0/1558891465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f968814fa90 0x7f9688152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.220+0000 7f968d66d640 1 --2- 172.21.15.45:0/1558891465 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f968806f540 0x7f968814f1f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.220+0000 7f968d66d640 1 -- 172.21.15.45:0/1558891465 >> 172.21.15.45:0/1558891465 conn(0x7f96880b3c40 msgr2=0x7f96880b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:28.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.220+0000 7f968d66d640 1 -- 172.21.15.45:0/1558891465 shutdown_connections 2024-05-08T05:45:28.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.220+0000 7f968d66d640 1 -- 172.21.15.45:0/1558891465 wait complete. 2024-05-08T05:45:28.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:45:28.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.340+0000 7fa143c72640 1 Processor -- start 2024-05-08T05:45:28.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.340+0000 7fa143c72640 1 -- start start 2024-05-08T05:45:28.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.341+0000 7fa143c72640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa13c070d30 0x7fa13c0d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:28.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.341+0000 7fa143c72640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa13c071110 0x7fa13c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:28.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.341+0000 7fa143c72640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa13c0d1300 con 0x7fa13c163920 2024-05-08T05:45:28.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.341+0000 7fa143c72640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa13c0d10d0 con 0x7fa13c070d30 2024-05-08T05:45:28.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.341+0000 7fa143c72640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa13c163db0 con 0x7fa13c071110 2024-05-08T05:45:28.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.341+0000 7fa14120d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa13c163920 0x7fa13c0703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34678/0 (socket says 172.21.15.45:34678) 2024-05-08T05:45:28.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.341+0000 7fa14120d640 1 -- 172.21.15.45:0/2389846936 learned_addr learned my addr 172.21.15.45:0/2389846936 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:28.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.341+0000 7fa141a0e640 1 --2- 172.21.15.45:0/2389846936 >> v2:172.21.15.63:3300/0 conn(0x7fa13c070d30 0x7fa13c0d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:28.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.341+0000 7fa14220f640 1 --2- 172.21.15.45:0/2389846936 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa13c071110 0x7fa13c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:28.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.341+0000 7fa140a0c640 1 -- 172.21.15.45:0/2389846936 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4149710509 0 0) 0x7fa13c0d1300 con 0x7fa13c163920 2024-05-08T05:45:28.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.341+0000 7fa140a0c640 1 -- 172.21.15.45:0/2389846936 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa128003520 con 0x7fa13c163920 2024-05-08T05:45:28.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.341+0000 7fa140a0c640 1 -- 172.21.15.45:0/2389846936 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2924749195 0 0) 0x7fa128003520 con 0x7fa13c163920 2024-05-08T05:45:28.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.341+0000 7fa141a0e640 1 -- 172.21.15.45:0/2389846936 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa13c071110 msgr2=0x7fa13c06d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:28.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.342+0000 7fa141a0e640 1 --2- 172.21.15.45:0/2389846936 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa13c071110 0x7fa13c06d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.342+0000 7fa141a0e640 1 -- 172.21.15.45:0/2389846936 >> v1:172.21.15.45:6789/0 conn(0x7fa13c163920 legacy=0x7fa13c0703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:28.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.342+0000 7fa141a0e640 1 -- 172.21.15.45:0/2389846936 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13c0d15e0 con 0x7fa13c070d30 2024-05-08T05:45:28.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.342+0000 7fa14220f640 1 --2- 172.21.15.45:0/2389846936 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa13c071110 0x7fa13c06d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:45:28.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.342+0000 7fa141a0e640 1 --2- 172.21.15.45:0/2389846936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa13c070d30 0x7fa13c0d20c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa1380097a0 tx=0x7fa13802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4323b2eaace102c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:28.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.343+0000 7fa140a0c640 1 -- 172.21.15.45:0/2389846936 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138037070 con 0x7fa13c070d30 2024-05-08T05:45:28.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.343+0000 7fa140a0c640 1 -- 172.21.15.45:0/2389846936 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa13802aba0 con 0x7fa13c070d30 2024-05-08T05:45:28.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.343+0000 7fa140a0c640 1 -- 172.21.15.45:0/2389846936 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c0016f0 con 0x7fa13c070d30 2024-05-08T05:45:28.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.343+0000 7fa143c72640 1 -- 172.21.15.45:0/2389846936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa13c070d30 msgr2=0x7fa13c0d20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:28.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.343+0000 7fa143c72640 1 --2- 172.21.15.45:0/2389846936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa13c070d30 0x7fa13c0d20c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa1380097a0 tx=0x7fa13802a110 comp rx=0 tx=0).stop 2024-05-08T05:45:28.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.343+0000 7fa143c72640 1 -- 172.21.15.45:0/2389846936 shutdown_connections 2024-05-08T05:45:28.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.343+0000 7fa143c72640 1 --2- 172.21.15.45:0/2389846936 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa13c071110 0x7fa13c06d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.343+0000 7fa143c72640 1 --2- 172.21.15.45:0/2389846936 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa13c070d30 0x7fa13c0d20c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.343+0000 7fa143c72640 1 -- 172.21.15.45:0/2389846936 >> 172.21.15.45:0/2389846936 conn(0x7fa13c0d2e90 msgr2=0x7fa13c0d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:28.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.343+0000 7fa143c72640 1 -- 172.21.15.45:0/2389846936 shutdown_connections 2024-05-08T05:45:28.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.343+0000 7fa143c72640 1 -- 172.21.15.45:0/2389846936 wait complete. 2024-05-08T05:45:28.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.344+0000 7fa143c72640 1 Processor -- start 2024-05-08T05:45:28.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.344+0000 7fa143c72640 1 -- start start 2024-05-08T05:45:28.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.345+0000 7fa143c72640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa13c070d30 0x7fa13c0ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:28.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.345+0000 7fa143c72640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa13c071110 0x7fa13c151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:28.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.345+0000 7fa143c72640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa13c163920 0x7fa13c0b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:28.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.345+0000 7fa143c72640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c0d1c30 con 0x7fa13c070d30 2024-05-08T05:45:28.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.345+0000 7fa143c72640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c070b80 con 0x7fa13c071110 2024-05-08T05:45:28.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.345+0000 7fa143c72640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa13c0715a0 con 0x7fa13c163920 2024-05-08T05:45:28.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.345+0000 7fa141a0e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa13c070d30 0x7fa13c0ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:28.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.345+0000 7fa141a0e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa13c070d30 0x7fa13c0ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56522/0 (socket says 172.21.15.45:56522) 2024-05-08T05:45:28.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.345+0000 7fa141a0e640 1 -- 172.21.15.45:0/3830800946 learned_addr learned my addr 172.21.15.45:0/3830800946 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:28.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.345+0000 7fa14220f640 1 --2- 172.21.15.45:0/3830800946 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa13c163920 0x7fa13c0b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:28.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.345+0000 7fa14120d640 1 --2- 172.21.15.45:0/3830800946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa13c071110 0x7fa13c151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:28.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.345+0000 7fa141a0e640 1 -- 172.21.15.45:0/3830800946 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa13c163920 msgr2=0x7fa13c0b5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:28.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.345+0000 7fa141a0e640 1 --2- 172.21.15.45:0/3830800946 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa13c163920 0x7fa13c0b5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.345+0000 7fa141a0e640 1 -- 172.21.15.45:0/3830800946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa13c071110 msgr2=0x7fa13c151650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:28.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.346+0000 7fa141a0e640 1 --2- 172.21.15.45:0/3830800946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa13c071110 0x7fa13c151650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.346+0000 7fa141a0e640 1 -- 172.21.15.45:0/3830800946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13c151b90 con 0x7fa13c070d30 2024-05-08T05:45:28.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.346+0000 7fa14120d640 1 --2- 172.21.15.45:0/3830800946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa13c071110 0x7fa13c151650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:28.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.346+0000 7fa141a0e640 1 --2- 172.21.15.45:0/3830800946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa13c070d30 0x7fa13c0ca840 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa1380098c0 tx=0x7fa1380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:28.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.346+0000 7fa1327fc640 1 -- 172.21.15.45:0/3830800946 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138037050 con 0x7fa13c070d30 2024-05-08T05:45:28.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.346+0000 7fa1327fc640 1 -- 172.21.15.45:0/3830800946 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa138043070 con 0x7fa13c070d30 2024-05-08T05:45:28.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.346+0000 7fa1327fc640 1 -- 172.21.15.45:0/3830800946 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138006e10 con 0x7fa13c070d30 2024-05-08T05:45:28.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.346+0000 7fa143c72640 1 -- 172.21.15.45:0/3830800946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13c0caed0 con 0x7fa13c070d30 2024-05-08T05:45:28.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.346+0000 7fa143c72640 1 -- 172.21.15.45:0/3830800946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa13c0cb770 con 0x7fa13c070d30 2024-05-08T05:45:28.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.347+0000 7fa143c72640 1 -- 172.21.15.45:0/3830800946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa13c06fc30 con 0x7fa13c070d30 2024-05-08T05:45:28.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.348+0000 7fa1327fc640 1 -- 172.21.15.45:0/3830800946 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa13803ca10 con 0x7fa13c070d30 2024-05-08T05:45:28.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.348+0000 7fa1327fc640 1 --2- 172.21.15.45:0/3830800946 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1200762d0 0x7fa120078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:28.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.348+0000 7fa1327fc640 1 -- 172.21.15.45:0/3830800946 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fa1380b81c0 con 0x7fa13c070d30 2024-05-08T05:45:28.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.349+0000 7fa14120d640 1 --2- 172.21.15.45:0/3830800946 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1200762d0 0x7fa120078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:28.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.349+0000 7fa14120d640 1 --2- 172.21.15.45:0/3830800946 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1200762d0 0x7fa120078790 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa13c15f0e0 tx=0x7fa12c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:28.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.353+0000 7fa1327fc640 1 -- 172.21.15.45:0/3830800946 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa13c06fc30 con 0x7fa13c070d30 2024-05-08T05:45:28.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.548+0000 7fa143c72640 1 -- 172.21.15.45:0/3830800946 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa13c0693f0 con 0x7fa13c070d30 2024-05-08T05:45:28.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.550+0000 7fa1327fc640 1 -- 172.21.15.45:0/3830800946 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+16897 (secure 0 0 0) 0x7fa138081990 con 0x7fa13c070d30 2024-05-08T05:45:28.550 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:28.550 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":75,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:45:25.833261+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:45:21.867649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-08T05:45:23.164376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:45:28.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.552+0000 7fa143c72640 1 -- 172.21.15.45:0/3830800946 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1200762d0 msgr2=0x7fa120078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:28.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.552+0000 7fa143c72640 1 --2- 172.21.15.45:0/3830800946 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1200762d0 0x7fa120078790 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa13c15f0e0 tx=0x7fa12c009040 comp rx=0 tx=0).stop 2024-05-08T05:45:28.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.552+0000 7fa143c72640 1 -- 172.21.15.45:0/3830800946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa13c070d30 msgr2=0x7fa13c0ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:28.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.552+0000 7fa143c72640 1 --2- 172.21.15.45:0/3830800946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa13c070d30 0x7fa13c0ca840 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa1380098c0 tx=0x7fa1380046c0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.552+0000 7fa143c72640 1 -- 172.21.15.45:0/3830800946 shutdown_connections 2024-05-08T05:45:28.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.552+0000 7fa143c72640 1 --2- 172.21.15.45:0/3830800946 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1200762d0 0x7fa120078790 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.552+0000 7fa143c72640 1 --2- 172.21.15.45:0/3830800946 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa13c163920 0x7fa13c0b5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.552+0000 7fa143c72640 1 --2- 172.21.15.45:0/3830800946 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa13c071110 0x7fa13c151650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.552+0000 7fa143c72640 1 --2- 172.21.15.45:0/3830800946 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa13c070d30 0x7fa13c0ca840 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.552+0000 7fa143c72640 1 -- 172.21.15.45:0/3830800946 >> 172.21.15.45:0/3830800946 conn(0x7fa13c0d2e90 msgr2=0x7fa13c0703a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:28.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.552+0000 7fa143c72640 1 -- 172.21.15.45:0/3830800946 shutdown_connections 2024-05-08T05:45:28.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.552+0000 7fa143c72640 1 -- 172.21.15.45:0/3830800946 wait complete. 2024-05-08T05:45:28.564 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T05:45:28.564 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:45:28.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.668+0000 7fe906059640 1 Processor -- start 2024-05-08T05:45:28.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.668+0000 7fe906059640 1 -- start start 2024-05-08T05:45:28.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.669+0000 7fe906059640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe90015f380 0x7fe9000d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.669+0000 7fe906059640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9000d0cf0 0x7fe9000d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.669+0000 7fe906059640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe9000d3b00 0x7fe900162e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.669+0000 7fe906059640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe9000d30f0 con 0x7fe90015f380 2024-05-08T05:45:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.669+0000 7fe906059640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe9000d1150 con 0x7fe9000d3b00 2024-05-08T05:45:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.669+0000 7fe906059640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe9000d2370 con 0x7fe9000d0cf0 2024-05-08T05:45:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.669+0000 7fe8ff7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe90015f380 0x7fe9000d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.669+0000 7fe8ff7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe90015f380 0x7fe9000d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56526/0 (socket says 172.21.15.45:56526) 2024-05-08T05:45:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.669+0000 7fe8ff7fe640 1 -- 172.21.15.45:0/880996996 learned_addr learned my addr 172.21.15.45:0/880996996 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.670+0000 7fe8f6ffd640 1 --2- 172.21.15.45:0/880996996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9000d0cf0 0x7fe9000d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.670+0000 7fe8fffff640 1 --2- 172.21.15.45:0/880996996 >> v2:172.21.15.63:3300/0 conn(0x7fe9000d3b00 0x7fe900162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.670+0000 7fe8ff7fe640 1 -- 172.21.15.45:0/880996996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9000d0cf0 msgr2=0x7fe9000d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.670+0000 7fe8ff7fe640 1 --2- 172.21.15.45:0/880996996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9000d0cf0 0x7fe9000d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.670+0000 7fe8ff7fe640 1 -- 172.21.15.45:0/880996996 >> v2:172.21.15.63:3300/0 conn(0x7fe9000d3b00 msgr2=0x7fe900162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:28.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.670+0000 7fe8ff7fe640 1 --2- 172.21.15.45:0/880996996 >> v2:172.21.15.63:3300/0 conn(0x7fe9000d3b00 0x7fe900162e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.670+0000 7fe8ff7fe640 1 -- 172.21.15.45:0/880996996 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9000d1760 con 0x7fe90015f380 2024-05-08T05:45:28.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.670+0000 7fe8f6ffd640 1 --2- 172.21.15.45:0/880996996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9000d0cf0 0x7fe9000d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:28.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.670+0000 7fe8fffff640 1 --2- 172.21.15.45:0/880996996 >> v2:172.21.15.63:3300/0 conn(0x7fe9000d3b00 0x7fe900162e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:28.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.670+0000 7fe8ff7fe640 1 --2- 172.21.15.45:0/880996996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe90015f380 0x7fe9000d07b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe90008f120 tx=0x7fe8f002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7329ed183e73d0f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:28.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.671+0000 7fe8feffd640 1 -- 172.21.15.45:0/880996996 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0037070 con 0x7fe90015f380 2024-05-08T05:45:28.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.671+0000 7fe8feffd640 1 -- 172.21.15.45:0/880996996 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f0032440 con 0x7fe90015f380 2024-05-08T05:45:28.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.671+0000 7fe8feffd640 1 -- 172.21.15.45:0/880996996 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0032810 con 0x7fe90015f380 2024-05-08T05:45:28.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.671+0000 7fe906059640 1 -- 172.21.15.45:0/880996996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe90015f380 msgr2=0x7fe9000d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:28.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.671+0000 7fe906059640 1 --2- 172.21.15.45:0/880996996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe90015f380 0x7fe9000d07b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe90008f120 tx=0x7fe8f002a1e0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.671+0000 7fe906059640 1 -- 172.21.15.45:0/880996996 shutdown_connections 2024-05-08T05:45:28.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.671+0000 7fe906059640 1 --2- 172.21.15.45:0/880996996 >> v2:172.21.15.63:3300/0 conn(0x7fe9000d3b00 0x7fe900162e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.671+0000 7fe906059640 1 --2- 172.21.15.45:0/880996996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9000d0cf0 0x7fe9000d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.671+0000 7fe906059640 1 --2- 172.21.15.45:0/880996996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe90015f380 0x7fe9000d07b0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.671+0000 7fe906059640 1 -- 172.21.15.45:0/880996996 >> 172.21.15.45:0/880996996 conn(0x7fe9000d4fe0 msgr2=0x7fe9000d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:28.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.671+0000 7fe906059640 1 -- 172.21.15.45:0/880996996 shutdown_connections 2024-05-08T05:45:28.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.671+0000 7fe906059640 1 -- 172.21.15.45:0/880996996 wait complete. 2024-05-08T05:45:28.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.672+0000 7fe906059640 1 Processor -- start 2024-05-08T05:45:28.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.673+0000 7fe906059640 1 -- start start 2024-05-08T05:45:28.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.673+0000 7fe906059640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9000d0cf0 0x7fe90009f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:28.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.673+0000 7fe906059640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9000d3b00 0x7fe90015b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:28.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.673+0000 7fe906059640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe90015f380 0x7fe9000c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:28.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.673+0000 7fe906059640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe9000d2000 con 0x7fe9000d3b00 2024-05-08T05:45:28.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.673+0000 7fe906059640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe9000d1d20 con 0x7fe9000d0cf0 2024-05-08T05:45:28.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.673+0000 7fe906059640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe90015f760 con 0x7fe90015f380 2024-05-08T05:45:28.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.673+0000 7fe8f6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9000d3b00 0x7fe90015b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:28.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.674+0000 7fe8f6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9000d3b00 0x7fe90015b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56538/0 (socket says 172.21.15.45:56538) 2024-05-08T05:45:28.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.674+0000 7fe8f6ffd640 1 -- 172.21.15.45:0/1082816576 learned_addr learned my addr 172.21.15.45:0/1082816576 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:28.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.674+0000 7fe8ff7fe640 1 --2- 172.21.15.45:0/1082816576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9000d0cf0 0x7fe90009f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:28.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.674+0000 7fe8fffff640 1 --2- 172.21.15.45:0/1082816576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe90015f380 0x7fe9000c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:28.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.674+0000 7fe8f6ffd640 1 -- 172.21.15.45:0/1082816576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe90015f380 msgr2=0x7fe9000c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:28.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.674+0000 7fe8f6ffd640 1 --2- 172.21.15.45:0/1082816576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe90015f380 0x7fe9000c9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.674+0000 7fe8f6ffd640 1 -- 172.21.15.45:0/1082816576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9000d0cf0 msgr2=0x7fe90009f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:28.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.674+0000 7fe8f6ffd640 1 --2- 172.21.15.45:0/1082816576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9000d0cf0 0x7fe90009f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.674+0000 7fe8f6ffd640 1 -- 172.21.15.45:0/1082816576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9000a2960 con 0x7fe9000d3b00 2024-05-08T05:45:28.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.674+0000 7fe8f6ffd640 1 --2- 172.21.15.45:0/1082816576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9000d3b00 0x7fe90015b790 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800c7c0 tx=0x7fe8e800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:28.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.674+0000 7fe8fcff9640 1 -- 172.21.15.45:0/1082816576 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8015070 con 0x7fe9000d3b00 2024-05-08T05:45:28.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.674+0000 7fe8fcff9640 1 -- 172.21.15.45:0/1082816576 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e8007a30 con 0x7fe9000d3b00 2024-05-08T05:45:28.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.674+0000 7fe8fcff9640 1 -- 172.21.15.45:0/1082816576 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8007e00 con 0x7fe9000d3b00 2024-05-08T05:45:28.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.674+0000 7fe906059640 1 -- 172.21.15.45:0/1082816576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9000ccb60 con 0x7fe9000d3b00 2024-05-08T05:45:28.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.674+0000 7fe906059640 1 -- 172.21.15.45:0/1082816576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9000cd400 con 0x7fe9000d3b00 2024-05-08T05:45:28.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.675+0000 7fe906059640 1 -- 172.21.15.45:0/1082816576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9000693f0 con 0x7fe9000d3b00 2024-05-08T05:45:28.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.676+0000 7fe8fcff9640 1 -- 172.21.15.45:0/1082816576 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe8e8018070 con 0x7fe9000d3b00 2024-05-08T05:45:28.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.677+0000 7fe8fcff9640 1 --2- 172.21.15.45:0/1082816576 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe8e00762d0 0x7fe8e0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:28.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.677+0000 7fe8fcff9640 1 -- 172.21.15.45:0/1082816576 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fe8e80976b0 con 0x7fe9000d3b00 2024-05-08T05:45:28.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.677+0000 7fe8ff7fe640 1 --2- 172.21.15.45:0/1082816576 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe8e00762d0 0x7fe8e0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:28.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.677+0000 7fe8ff7fe640 1 --2- 172.21.15.45:0/1082816576 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe8e00762d0 0x7fe8e0078790 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0034000 tx=0x7fe8f002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:28.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.681+0000 7fe8fcff9640 1 -- 172.21.15.45:0/1082816576 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe8e8060e70 con 0x7fe9000d3b00 2024-05-08T05:45:28.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.869+0000 7fe906059640 1 -- 172.21.15.45:0/1082816576 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe90008cfd0 con 0x7fe8e00762d0 2024-05-08T05:45:28.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.875+0000 7fe8fcff9640 1 -- 172.21.15.45:0/1082816576 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248491 (secure 0 0 0) 0x7fe90008cfd0 con 0x7fe8e00762d0 2024-05-08T05:45:28.878 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:28.880 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:45:28.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.878+0000 7fe906059640 1 -- 172.21.15.45:0/1082816576 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe8e00762d0 msgr2=0x7fe8e0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:28.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.878+0000 7fe906059640 1 --2- 172.21.15.45:0/1082816576 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe8e00762d0 0x7fe8e0078790 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0034000 tx=0x7fe8f002e000 comp rx=0 tx=0).stop 2024-05-08T05:45:28.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.878+0000 7fe906059640 1 -- 172.21.15.45:0/1082816576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9000d3b00 msgr2=0x7fe90015b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:28.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.878+0000 7fe906059640 1 --2- 172.21.15.45:0/1082816576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9000d3b00 0x7fe90015b790 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800c7c0 tx=0x7fe8e800cc80 comp rx=0 tx=0).stop 2024-05-08T05:45:28.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.879+0000 7fe906059640 1 -- 172.21.15.45:0/1082816576 shutdown_connections 2024-05-08T05:45:28.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.879+0000 7fe906059640 1 --2- 172.21.15.45:0/1082816576 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe8e00762d0 0x7fe8e0078790 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.879+0000 7fe906059640 1 --2- 172.21.15.45:0/1082816576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe90015f380 0x7fe9000c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.879+0000 7fe906059640 1 --2- 172.21.15.45:0/1082816576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9000d3b00 0x7fe90015b790 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.879+0000 7fe906059640 1 --2- 172.21.15.45:0/1082816576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9000d0cf0 0x7fe90009f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:28.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.879+0000 7fe906059640 1 -- 172.21.15.45:0/1082816576 >> 172.21.15.45:0/1082816576 conn(0x7fe9000d4fe0 msgr2=0x7fe9000d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:28.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.879+0000 7fe906059640 1 -- 172.21.15.45:0/1082816576 shutdown_connections 2024-05-08T05:45:28.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:28.879+0000 7fe906059640 1 -- 172.21.15.45:0/1082816576 wait complete. 2024-05-08T05:45:28.890 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-05-08T05:45:26.888552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592036,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":120,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89,"ondisk_log_size":89,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240868,"kb_used_data":8044,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754733852,"statfs":{"total":773094113280,"available":772847464448,"internally_reserved":0,"allocated":8237056,"data_stored":4354625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1668,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.074171"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.851710+0000","last_change":"2024-05-08T05:45:24.849892+0000","last_active":"2024-05-08T05:45:24.851710+0000","last_peered":"2024-05-08T05:45:24.851710+0000","last_clean":"2024-05-08T05:45:24.851710+0000","last_became_active":"2024-05-08T05:45:24.849483+0000","last_became_peered":"2024-05-08T05:45:24.849483+0000","last_unstale":"2024-05-08T05:45:24.851710+0000","last_undegraded":"2024-05-08T05:45:24.851710+0000","last_fullsized":"2024-05-08T05:45:24.851710+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:35:45.649423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.825664+0000","last_change":"2024-05-08T05:45:23.825664+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.825664+0000","last_undegraded":"2024-05-08T05:45:23.825664+0000","last_fullsized":"2024-05-08T05:45:23.825664+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.819878+0000","last_change":"2024-05-08T05:45:23.819878+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.819878+0000","last_undegraded":"2024-05-08T05:45:23.819878+0000","last_fullsized":"2024-05-08T05:45:23.819878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.852071+0000","last_change":"2024-05-08T05:45:24.849026+0000","last_active":"2024-05-08T05:45:24.852071+0000","last_peered":"2024-05-08T05:45:24.852071+0000","last_clean":"2024-05-08T05:45:24.852071+0000","last_became_active":"2024-05-08T05:45:24.848740+0000","last_became_peered":"2024-05-08T05:45:24.848740+0000","last_unstale":"2024-05-08T05:45:24.852071+0000","last_undegraded":"2024-05-08T05:45:24.852071+0000","last_fullsized":"2024-05-08T05:45:24.852071+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:25:33.241640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.823423+0000","last_change":"2024-05-08T05:45:23.823423+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.823423+0000","last_undegraded":"2024-05-08T05:45:23.823423+0000","last_fullsized":"2024-05-08T05:45:23.823423+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.824515+0000","last_change":"2024-05-08T05:45:23.824515+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.824515+0000","last_undegraded":"2024-05-08T05:45:23.824515+0000","last_fullsized":"2024-05-08T05:45:23.824515+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.852654+0000","last_change":"2024-05-08T05:45:24.850111+0000","last_active":"2024-05-08T05:45:24.852654+0000","last_peered":"2024-05-08T05:45:24.852654+0000","last_clean":"2024-05-08T05:45:24.852654+0000","last_became_active":"2024-05-08T05:45:24.849810+0000","last_became_peered":"2024-05-08T05:45:24.849810+0000","last_unstale":"2024-05-08T05:45:24.852654+0000","last_undegraded":"2024-05-08T05:45:24.852654+0000","last_fullsized":"2024-05-08T05:45:24.852654+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:59:57.843585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.827759+0000","last_change":"2024-05-08T05:45:23.827759+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.827759+0000","last_undegraded":"2024-05-08T05:45:23.827759+0000","last_fullsized":"2024-05-08T05:45:23.827759+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.822325+0000","last_change":"2024-05-08T05:45:23.822325+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.822325+0000","last_undegraded":"2024-05-08T05:45:23.822325+0000","last_fullsized":"2024-05-08T05:45:23.822325+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.846430+0000","last_change":"2024-05-08T05:45:24.841146+0000","last_active":"2024-05-08T05:45:24.846430+0000","last_peered":"2024-05-08T05:45:24.846430+0000","last_clean":"2024-05-08T05:45:24.846430+0000","last_became_active":"2024-05-08T05:45:24.840652+0000","last_became_peered":"2024-05-08T05:45:24.840652+0000","last_unstale":"2024-05-08T05:45:24.846430+0000","last_undegraded":"2024-05-08T05:45:24.846430+0000","last_fullsized":"2024-05-08T05:45:24.846430+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:38.823695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.825935+0000","last_change":"2024-05-08T05:45:23.825935+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.825935+0000","last_undegraded":"2024-05-08T05:45:23.825935+0000","last_fullsized":"2024-05-08T05:45:23.825935+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.847096+0000","last_change":"2024-05-08T05:45:24.840089+0000","last_active":"2024-05-08T05:45:24.847096+0000","last_peered":"2024-05-08T05:45:24.847096+0000","last_clean":"2024-05-08T05:45:24.847096+0000","last_became_active":"2024-05-08T05:45:24.839767+0000","last_became_peered":"2024-05-08T05:45:24.839767+0000","last_unstale":"2024-05-08T05:45:24.847096+0000","last_undegraded":"2024-05-08T05:45:24.847096+0000","last_fullsized":"2024-05-08T05:45:24.847096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:32:06.497004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.823045+0000","last_change":"2024-05-08T05:45:23.823045+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.823045+0000","last_undegraded":"2024-05-08T05:45:23.823045+0000","last_fullsized":"2024-05-08T05:45:23.823045+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.824045+0000","last_change":"2024-05-08T05:45:23.824045+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.824045+0000","last_undegraded":"2024-05-08T05:45:23.824045+0000","last_fullsized":"2024-05-08T05:45:23.824045+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.824072+0000","last_change":"2024-05-08T05:45:23.824072+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.824072+0000","last_undegraded":"2024-05-08T05:45:23.824072+0000","last_fullsized":"2024-05-08T05:45:23.824072+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.847010+0000","last_change":"2024-05-08T05:45:24.838511+0000","last_active":"2024-05-08T05:45:24.847010+0000","last_peered":"2024-05-08T05:45:24.847010+0000","last_clean":"2024-05-08T05:45:24.847010+0000","last_became_active":"2024-05-08T05:45:24.838217+0000","last_became_peered":"2024-05-08T05:45:24.838217+0000","last_unstale":"2024-05-08T05:45:24.847010+0000","last_undegraded":"2024-05-08T05:45:24.847010+0000","last_fullsized":"2024-05-08T05:45:24.847010+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:43:37.991934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.825063+0000","last_change":"2024-05-08T05:45:23.825063+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.825063+0000","last_undegraded":"2024-05-08T05:45:23.825063+0000","last_fullsized":"2024-05-08T05:45:23.825063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.820234+0000","last_change":"2024-05-08T05:45:23.820234+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.820234+0000","last_undegraded":"2024-05-08T05:45:23.820234+0000","last_fullsized":"2024-05-08T05:45:23.820234+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.824653+0000","last_change":"2024-05-08T05:45:23.824653+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.824653+0000","last_undegraded":"2024-05-08T05:45:23.824653+0000","last_fullsized":"2024-05-08T05:45:23.824653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.826933+0000","last_change":"2024-05-08T05:45:23.826933+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.826933+0000","last_undegraded":"2024-05-08T05:45:23.826933+0000","last_fullsized":"2024-05-08T05:45:23.826933+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.823744+0000","last_change":"2024-05-08T05:45:23.823744+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.823744+0000","last_undegraded":"2024-05-08T05:45:23.823744+0000","last_fullsized":"2024-05-08T05:45:23.823744+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.822048+0000","last_change":"2024-05-08T05:45:23.822048+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.822048+0000","last_undegraded":"2024-05-08T05:45:23.822048+0000","last_fullsized":"2024-05-08T05:45:23.822048+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.822758+0000","last_change":"2024-05-08T05:45:23.822758+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.822758+0000","last_undegraded":"2024-05-08T05:45:23.822758+0000","last_fullsized":"2024-05-08T05:45:23.822758+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.823553+0000","last_change":"2024-05-08T05:45:23.823553+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.823553+0000","last_undegraded":"2024-05-08T05:45:23.823553+0000","last_fullsized":"2024-05-08T05:45:23.823553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.821720+0000","last_change":"2024-05-08T05:45:23.821720+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.821720+0000","last_undegraded":"2024-05-08T05:45:23.821720+0000","last_fullsized":"2024-05-08T05:45:23.821720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.846405+0000","last_change":"2024-05-08T05:45:24.840913+0000","last_active":"2024-05-08T05:45:24.846405+0000","last_peered":"2024-05-08T05:45:24.846405+0000","last_clean":"2024-05-08T05:45:24.846405+0000","last_became_active":"2024-05-08T05:45:24.840291+0000","last_became_peered":"2024-05-08T05:45:24.840291+0000","last_unstale":"2024-05-08T05:45:24.846405+0000","last_undegraded":"2024-05-08T05:45:24.846405+0000","last_fullsized":"2024-05-08T05:45:24.846405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:18:01.264013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.847044+0000","last_change":"2024-05-08T05:45:24.843174+0000","last_active":"2024-05-08T05:45:24.847044+0000","last_peered":"2024-05-08T05:45:24.847044+0000","last_clean":"2024-05-08T05:45:24.847044+0000","last_became_active":"2024-05-08T05:45:24.842213+0000","last_became_peered":"2024-05-08T05:45:24.842213+0000","last_unstale":"2024-05-08T05:45:24.847044+0000","last_undegraded":"2024-05-08T05:45:24.847044+0000","last_fullsized":"2024-05-08T05:45:24.847044+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:25:54.796485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.825573+0000","last_change":"2024-05-08T05:45:23.825573+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.825573+0000","last_undegraded":"2024-05-08T05:45:23.825573+0000","last_fullsized":"2024-05-08T05:45:23.825573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.822584+0000","last_change":"2024-05-08T05:45:23.822584+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.822584+0000","last_undegraded":"2024-05-08T05:45:23.822584+0000","last_fullsized":"2024-05-08T05:45:23.822584+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.847039+0000","last_change":"2024-05-08T05:45:24.842920+0000","last_active":"2024-05-08T05:45:24.847039+0000","last_peered":"2024-05-08T05:45:24.847039+0000","last_clean":"2024-05-08T05:45:24.847039+0000","last_became_active":"2024-05-08T05:45:24.842628+0000","last_became_peered":"2024-05-08T05:45:24.842628+0000","last_unstale":"2024-05-08T05:45:24.847039+0000","last_undegraded":"2024-05-08T05:45:24.847039+0000","last_fullsized":"2024-05-08T05:45:24.847039+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:00.887389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.819540+0000","last_change":"2024-05-08T05:45:23.819540+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.819540+0000","last_undegraded":"2024-05-08T05:45:23.819540+0000","last_fullsized":"2024-05-08T05:45:23.819540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.855173+0000","last_change":"2024-05-08T05:45:24.850149+0000","last_active":"2024-05-08T05:45:24.855173+0000","last_peered":"2024-05-08T05:45:24.855173+0000","last_clean":"2024-05-08T05:45:24.855173+0000","last_became_active":"2024-05-08T05:45:24.849888+0000","last_became_peered":"2024-05-08T05:45:24.849888+0000","last_unstale":"2024-05-08T05:45:24.855173+0000","last_undegraded":"2024-05-08T05:45:24.855173+0000","last_fullsized":"2024-05-08T05:45:24.855173+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:31.609131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.819608+0000","last_change":"2024-05-08T05:45:23.819608+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.819608+0000","last_undegraded":"2024-05-08T05:45:23.819608+0000","last_fullsized":"2024-05-08T05:45:23.819608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-05-08T05:45:23.840269+0000","last_change":"2024-05-08T05:45:23.839361+0000","last_active":"2024-05-08T05:45:23.840269+0000","last_peered":"2024-05-08T05:45:23.840269+0000","last_clean":"2024-05-08T05:45:23.840269+0000","last_became_active":"2024-05-08T05:45:23.839066+0000","last_became_peered":"2024-05-08T05:45:23.839066+0000","last_unstale":"2024-05-08T05:45:23.840269+0000","last_undegraded":"2024-05-08T05:45:23.840269+0000","last_fullsized":"2024-05-08T05:45:23.840269+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:44:39.321100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.823649+0000","last_change":"2024-05-08T05:45:23.823649+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.823649+0000","last_undegraded":"2024-05-08T05:45:23.823649+0000","last_fullsized":"2024-05-08T05:45:23.823649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"74'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.852801+0000","last_change":"2024-05-08T05:45:23.839805+0000","last_active":"2024-05-08T05:45:24.852801+0000","last_peered":"2024-05-08T05:45:24.852801+0000","last_clean":"2024-05-08T05:45:24.852801+0000","last_became_active":"2024-05-08T05:45:23.839509+0000","last_became_peered":"2024-05-08T05:45:23.839509+0000","last_unstale":"2024-05-08T05:45:24.852801+0000","last_undegraded":"2024-05-08T05:45:24.852801+0000","last_fullsized":"2024-05-08T05:45:24.852801+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:01:30.445032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.822732+0000","last_change":"2024-05-08T05:45:23.822732+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.822732+0000","last_undegraded":"2024-05-08T05:45:23.822732+0000","last_fullsized":"2024-05-08T05:45:23.822732+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.853636+0000","last_change":"2024-05-08T05:45:23.832769+0000","last_active":"2024-05-08T05:45:24.853636+0000","last_peered":"2024-05-08T05:45:24.853636+0000","last_clean":"2024-05-08T05:45:24.853636+0000","last_became_active":"2024-05-08T05:45:23.832465+0000","last_became_peered":"2024-05-08T05:45:23.832465+0000","last_unstale":"2024-05-08T05:45:24.853636+0000","last_undegraded":"2024-05-08T05:45:24.853636+0000","last_fullsized":"2024-05-08T05:45:24.853636+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:05.456807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.823315+0000","last_change":"2024-05-08T05:45:23.823315+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.823315+0000","last_undegraded":"2024-05-08T05:45:23.823315+0000","last_fullsized":"2024-05-08T05:45:23.823315+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.852680+0000","last_change":"2024-05-08T05:45:23.837917+0000","last_active":"2024-05-08T05:45:24.852680+0000","last_peered":"2024-05-08T05:45:24.852680+0000","last_clean":"2024-05-08T05:45:24.852680+0000","last_became_active":"2024-05-08T05:45:23.837605+0000","last_became_peered":"2024-05-08T05:45:23.837605+0000","last_unstale":"2024-05-08T05:45:24.852680+0000","last_undegraded":"2024-05-08T05:45:24.852680+0000","last_fullsized":"2024-05-08T05:45:24.852680+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:39:02.644530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.826083+0000","last_change":"2024-05-08T05:45:23.826083+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.826083+0000","last_undegraded":"2024-05-08T05:45:23.826083+0000","last_fullsized":"2024-05-08T05:45:23.826083+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-05-08T05:45:23.836971+0000","last_change":"2024-05-08T05:45:23.833571+0000","last_active":"2024-05-08T05:45:23.836971+0000","last_peered":"2024-05-08T05:45:23.836971+0000","last_clean":"2024-05-08T05:45:23.836971+0000","last_became_active":"2024-05-08T05:45:23.833258+0000","last_became_peered":"2024-05-08T05:45:23.833258+0000","last_unstale":"2024-05-08T05:45:23.836971+0000","last_undegraded":"2024-05-08T05:45:23.836971+0000","last_fullsized":"2024-05-08T05:45:23.836971+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:00.500831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.824191+0000","last_change":"2024-05-08T05:45:23.824191+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.824191+0000","last_undegraded":"2024-05-08T05:45:23.824191+0000","last_fullsized":"2024-05-08T05:45:23.824191+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"74'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.855459+0000","last_change":"2024-05-08T05:45:23.828059+0000","last_active":"2024-05-08T05:45:24.855459+0000","last_peered":"2024-05-08T05:45:24.855459+0000","last_clean":"2024-05-08T05:45:24.855459+0000","last_became_active":"2024-05-08T05:45:23.827769+0000","last_became_peered":"2024-05-08T05:45:23.827769+0000","last_unstale":"2024-05-08T05:45:24.855459+0000","last_undegraded":"2024-05-08T05:45:24.855459+0000","last_fullsized":"2024-05-08T05:45:24.855459+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:07:35.947497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.823530+0000","last_change":"2024-05-08T05:45:23.823530+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.823530+0000","last_undegraded":"2024-05-08T05:45:23.823530+0000","last_fullsized":"2024-05-08T05:45:23.823530+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-05-08T05:45:23.837449+0000","last_change":"2024-05-08T05:45:23.836149+0000","last_active":"2024-05-08T05:45:23.837449+0000","last_peered":"2024-05-08T05:45:23.837449+0000","last_clean":"2024-05-08T05:45:23.837449+0000","last_became_active":"2024-05-08T05:45:23.835794+0000","last_became_peered":"2024-05-08T05:45:23.835794+0000","last_unstale":"2024-05-08T05:45:23.837449+0000","last_undegraded":"2024-05-08T05:45:23.837449+0000","last_fullsized":"2024-05-08T05:45:23.837449+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:00.079205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.852754+0000","last_change":"2024-05-08T05:45:24.851132+0000","last_active":"2024-05-08T05:45:24.852754+0000","last_peered":"2024-05-08T05:45:24.852754+0000","last_clean":"2024-05-08T05:45:24.852754+0000","last_became_active":"2024-05-08T05:45:24.850840+0000","last_became_peered":"2024-05-08T05:45:24.850840+0000","last_unstale":"2024-05-08T05:45:24.852754+0000","last_undegraded":"2024-05-08T05:45:24.852754+0000","last_fullsized":"2024-05-08T05:45:24.852754+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:24.831405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-05-08T05:45:23.834348+0000","last_change":"2024-05-08T05:45:23.833511+0000","last_active":"2024-05-08T05:45:23.834348+0000","last_peered":"2024-05-08T05:45:23.834348+0000","last_clean":"2024-05-08T05:45:23.834348+0000","last_became_active":"2024-05-08T05:45:23.833232+0000","last_became_peered":"2024-05-08T05:45:23.833232+0000","last_unstale":"2024-05-08T05:45:23.834348+0000","last_undegraded":"2024-05-08T05:45:23.834348+0000","last_fullsized":"2024-05-08T05:45:23.834348+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:13:54.157933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.823119+0000","last_change":"2024-05-08T05:45:23.823119+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.823119+0000","last_undegraded":"2024-05-08T05:45:23.823119+0000","last_fullsized":"2024-05-08T05:45:23.823119+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.836338+0000","last_change":"2024-05-08T05:45:23.839833+0000","last_active":"2024-05-08T05:45:24.836338+0000","last_peered":"2024-05-08T05:45:24.836338+0000","last_clean":"2024-05-08T05:45:24.836338+0000","last_became_active":"2024-05-08T05:45:23.839539+0000","last_became_peered":"2024-05-08T05:45:23.839539+0000","last_unstale":"2024-05-08T05:45:24.836338+0000","last_undegraded":"2024-05-08T05:45:24.836338+0000","last_fullsized":"2024-05-08T05:45:24.836338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:49.025014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.852198+0000","last_change":"2024-05-08T05:45:24.849376+0000","last_active":"2024-05-08T05:45:24.852198+0000","last_peered":"2024-05-08T05:45:24.852198+0000","last_clean":"2024-05-08T05:45:24.852198+0000","last_became_active":"2024-05-08T05:45:24.849085+0000","last_became_peered":"2024-05-08T05:45:24.849085+0000","last_unstale":"2024-05-08T05:45:24.852198+0000","last_undegraded":"2024-05-08T05:45:24.852198+0000","last_fullsized":"2024-05-08T05:45:24.852198+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:58:21.085324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.835791+0000","last_change":"2024-05-08T05:45:23.833756+0000","last_active":"2024-05-08T05:45:24.835791+0000","last_peered":"2024-05-08T05:45:24.835791+0000","last_clean":"2024-05-08T05:45:24.835791+0000","last_became_active":"2024-05-08T05:45:23.833496+0000","last_became_peered":"2024-05-08T05:45:23.833496+0000","last_unstale":"2024-05-08T05:45:24.835791+0000","last_undegraded":"2024-05-08T05:45:24.835791+0000","last_fullsized":"2024-05-08T05:45:24.835791+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:58:08.284451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.820754+0000","last_change":"2024-05-08T05:45:23.820754+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.820754+0000","last_undegraded":"2024-05-08T05:45:23.820754+0000","last_fullsized":"2024-05-08T05:45:23.820754+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-05-08T05:45:23.840107+0000","last_change":"2024-05-08T05:45:23.836748+0000","last_active":"2024-05-08T05:45:23.840107+0000","last_peered":"2024-05-08T05:45:23.840107+0000","last_clean":"2024-05-08T05:45:23.840107+0000","last_became_active":"2024-05-08T05:45:23.836069+0000","last_became_peered":"2024-05-08T05:45:23.836069+0000","last_unstale":"2024-05-08T05:45:23.840107+0000","last_undegraded":"2024-05-08T05:45:23.840107+0000","last_fullsized":"2024-05-08T05:45:23.840107+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:43:30.929052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.819494+0000","last_change":"2024-05-08T05:45:23.819494+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.819494+0000","last_undegraded":"2024-05-08T05:45:23.819494+0000","last_fullsized":"2024-05-08T05:45:23.819494+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"74'6","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.858199+0000","last_change":"2024-05-08T05:45:23.830570+0000","last_active":"2024-05-08T05:45:24.858199+0000","last_peered":"2024-05-08T05:45:24.858199+0000","last_clean":"2024-05-08T05:45:24.858199+0000","last_became_active":"2024-05-08T05:45:23.830284+0000","last_became_peered":"2024-05-08T05:45:23.830284+0000","last_unstale":"2024-05-08T05:45:24.858199+0000","last_undegraded":"2024-05-08T05:45:24.858199+0000","last_fullsized":"2024-05-08T05:45:24.858199+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:57:32.335429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.847044+0000","last_change":"2024-05-08T05:45:24.839688+0000","last_active":"2024-05-08T05:45:24.847044+0000","last_peered":"2024-05-08T05:45:24.847044+0000","last_clean":"2024-05-08T05:45:24.847044+0000","last_became_active":"2024-05-08T05:45:24.838944+0000","last_became_peered":"2024-05-08T05:45:24.838944+0000","last_unstale":"2024-05-08T05:45:24.847044+0000","last_undegraded":"2024-05-08T05:45:24.847044+0000","last_fullsized":"2024-05-08T05:45:24.847044+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:00:22.478483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.854365+0000","last_change":"2024-05-08T05:45:23.829939+0000","last_active":"2024-05-08T05:45:24.854365+0000","last_peered":"2024-05-08T05:45:24.854365+0000","last_clean":"2024-05-08T05:45:24.854365+0000","last_became_active":"2024-05-08T05:45:23.829637+0000","last_became_peered":"2024-05-08T05:45:23.829637+0000","last_unstale":"2024-05-08T05:45:24.854365+0000","last_undegraded":"2024-05-08T05:45:24.854365+0000","last_fullsized":"2024-05-08T05:45:24.854365+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:00.079205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.850400+0000","last_change":"2024-05-08T05:45:24.841504+0000","last_active":"2024-05-08T05:45:24.850400+0000","last_peered":"2024-05-08T05:45:24.850400+0000","last_clean":"2024-05-08T05:45:24.850400+0000","last_became_active":"2024-05-08T05:45:24.841183+0000","last_became_peered":"2024-05-08T05:45:24.841183+0000","last_unstale":"2024-05-08T05:45:24.850400+0000","last_undegraded":"2024-05-08T05:45:24.850400+0000","last_fullsized":"2024-05-08T05:45:24.850400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:40.615500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.831149+0000","last_change":"2024-05-08T05:45:23.839358+0000","last_active":"2024-05-08T05:45:24.831149+0000","last_peered":"2024-05-08T05:45:24.831149+0000","last_clean":"2024-05-08T05:45:24.831149+0000","last_became_active":"2024-05-08T05:45:23.839057+0000","last_became_peered":"2024-05-08T05:45:23.839057+0000","last_unstale":"2024-05-08T05:45:24.831149+0000","last_undegraded":"2024-05-08T05:45:24.831149+0000","last_fullsized":"2024-05-08T05:45:24.831149+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:26.832027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.851353+0000","last_change":"2024-05-08T05:45:24.844754+0000","last_active":"2024-05-08T05:45:24.851353+0000","last_peered":"2024-05-08T05:45:24.851353+0000","last_clean":"2024-05-08T05:45:24.851353+0000","last_became_active":"2024-05-08T05:45:24.844469+0000","last_became_peered":"2024-05-08T05:45:24.844469+0000","last_unstale":"2024-05-08T05:45:24.851353+0000","last_undegraded":"2024-05-08T05:45:24.851353+0000","last_fullsized":"2024-05-08T05:45:24.851353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:47:54.532670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-05-08T05:45:23.838306+0000","last_change":"2024-05-08T05:45:23.835641+0000","last_active":"2024-05-08T05:45:23.838306+0000","last_peered":"2024-05-08T05:45:23.838306+0000","last_clean":"2024-05-08T05:45:23.838306+0000","last_became_active":"2024-05-08T05:45:23.835131+0000","last_became_peered":"2024-05-08T05:45:23.835131+0000","last_unstale":"2024-05-08T05:45:23.838306+0000","last_undegraded":"2024-05-08T05:45:23.838306+0000","last_fullsized":"2024-05-08T05:45:23.838306+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:04:19.057297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.848055+0000","last_change":"2024-05-08T05:45:24.838178+0000","last_active":"2024-05-08T05:45:24.848055+0000","last_peered":"2024-05-08T05:45:24.848055+0000","last_clean":"2024-05-08T05:45:24.848055+0000","last_became_active":"2024-05-08T05:45:24.837886+0000","last_became_peered":"2024-05-08T05:45:24.837886+0000","last_unstale":"2024-05-08T05:45:24.848055+0000","last_undegraded":"2024-05-08T05:45:24.848055+0000","last_fullsized":"2024-05-08T05:45:24.848055+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:18:08.297674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-05-08T05:45:23.840620+0000","last_change":"2024-05-08T05:45:23.839351+0000","last_active":"2024-05-08T05:45:23.840620+0000","last_peered":"2024-05-08T05:45:23.840620+0000","last_clean":"2024-05-08T05:45:23.840620+0000","last_became_active":"2024-05-08T05:45:23.839050+0000","last_became_peered":"2024-05-08T05:45:23.839050+0000","last_unstale":"2024-05-08T05:45:23.840620+0000","last_undegraded":"2024-05-08T05:45:23.840620+0000","last_fullsized":"2024-05-08T05:45:23.840620+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:09:59.746816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.820192+0000","last_change":"2024-05-08T05:45:23.820192+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.820192+0000","last_undegraded":"2024-05-08T05:45:23.820192+0000","last_fullsized":"2024-05-08T05:45:23.820192+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"74'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.849740+0000","last_change":"2024-05-08T05:45:23.834931+0000","last_active":"2024-05-08T05:45:24.849740+0000","last_peered":"2024-05-08T05:45:24.849740+0000","last_clean":"2024-05-08T05:45:24.849740+0000","last_became_active":"2024-05-08T05:45:23.834636+0000","last_became_peered":"2024-05-08T05:45:23.834636+0000","last_unstale":"2024-05-08T05:45:24.849740+0000","last_undegraded":"2024-05-08T05:45:24.849740+0000","last_fullsized":"2024-05-08T05:45:24.849740+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:55:10.280907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.822136+0000","last_change":"2024-05-08T05:45:23.822136+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.822136+0000","last_undegraded":"2024-05-08T05:45:23.822136+0000","last_fullsized":"2024-05-08T05:45:23.822136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-05-08T05:45:23.837105+0000","last_change":"2024-05-08T05:45:23.832031+0000","last_active":"2024-05-08T05:45:23.837105+0000","last_peered":"2024-05-08T05:45:23.837105+0000","last_clean":"2024-05-08T05:45:23.837105+0000","last_became_active":"2024-05-08T05:45:23.831735+0000","last_became_peered":"2024-05-08T05:45:23.831735+0000","last_unstale":"2024-05-08T05:45:23.837105+0000","last_undegraded":"2024-05-08T05:45:23.837105+0000","last_fullsized":"2024-05-08T05:45:23.837105+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:14:52.885584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.822568+0000","last_change":"2024-05-08T05:45:23.822568+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.822568+0000","last_undegraded":"2024-05-08T05:45:23.822568+0000","last_fullsized":"2024-05-08T05:45:23.822568+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-05-08T05:45:23.838346+0000","last_change":"2024-05-08T05:45:23.836336+0000","last_active":"2024-05-08T05:45:23.838346+0000","last_peered":"2024-05-08T05:45:23.838346+0000","last_clean":"2024-05-08T05:45:23.838346+0000","last_became_active":"2024-05-08T05:45:23.835736+0000","last_became_peered":"2024-05-08T05:45:23.835736+0000","last_unstale":"2024-05-08T05:45:23.838346+0000","last_undegraded":"2024-05-08T05:45:23.838346+0000","last_fullsized":"2024-05-08T05:45:23.838346+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:11.989798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.820216+0000","last_change":"2024-05-08T05:45:23.820216+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.820216+0000","last_undegraded":"2024-05-08T05:45:23.820216+0000","last_fullsized":"2024-05-08T05:45:23.820216+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-05-08T05:45:23.831278+0000","last_change":"2024-05-08T05:45:23.829957+0000","last_active":"2024-05-08T05:45:23.831278+0000","last_peered":"2024-05-08T05:45:23.831278+0000","last_clean":"2024-05-08T05:45:23.831278+0000","last_became_active":"2024-05-08T05:45:23.829634+0000","last_became_peered":"2024-05-08T05:45:23.829634+0000","last_unstale":"2024-05-08T05:45:23.831278+0000","last_undegraded":"2024-05-08T05:45:23.831278+0000","last_fullsized":"2024-05-08T05:45:23.831278+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:45:58.599863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.821122+0000","last_change":"2024-05-08T05:45:23.821122+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.821122+0000","last_undegraded":"2024-05-08T05:45:23.821122+0000","last_fullsized":"2024-05-08T05:45:23.821122+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-05-08T05:45:23.834607+0000","last_change":"2024-05-08T05:45:23.832221+0000","last_active":"2024-05-08T05:45:23.834607+0000","last_peered":"2024-05-08T05:45:23.834607+0000","last_clean":"2024-05-08T05:45:23.834607+0000","last_became_active":"2024-05-08T05:45:23.831864+0000","last_became_peered":"2024-05-08T05:45:23.831864+0000","last_unstale":"2024-05-08T05:45:23.834607+0000","last_undegraded":"2024-05-08T05:45:23.834607+0000","last_fullsized":"2024-05-08T05:45:23.834607+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:42:34.004714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.823314+0000","last_change":"2024-05-08T05:45:23.823314+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.823314+0000","last_undegraded":"2024-05-08T05:45:23.823314+0000","last_fullsized":"2024-05-08T05:45:23.823314+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-05-08T05:45:23.838507+0000","last_change":"2024-05-08T05:45:23.837086+0000","last_active":"2024-05-08T05:45:23.838507+0000","last_peered":"2024-05-08T05:45:23.838507+0000","last_clean":"2024-05-08T05:45:23.838507+0000","last_became_active":"2024-05-08T05:45:23.836777+0000","last_became_peered":"2024-05-08T05:45:23.836777+0000","last_unstale":"2024-05-08T05:45:23.838507+0000","last_undegraded":"2024-05-08T05:45:23.838507+0000","last_fullsized":"2024-05-08T05:45:23.838507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:26.832027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.852022+0000","last_change":"2024-05-08T05:45:24.845436+0000","last_active":"2024-05-08T05:45:24.852022+0000","last_peered":"2024-05-08T05:45:24.852022+0000","last_clean":"2024-05-08T05:45:24.852022+0000","last_became_active":"2024-05-08T05:45:24.845164+0000","last_became_peered":"2024-05-08T05:45:24.845164+0000","last_unstale":"2024-05-08T05:45:24.852022+0000","last_undegraded":"2024-05-08T05:45:24.852022+0000","last_fullsized":"2024-05-08T05:45:24.852022+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:27.942203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-05-08T05:45:23.838423+0000","last_change":"2024-05-08T05:45:23.836394+0000","last_active":"2024-05-08T05:45:23.838423+0000","last_peered":"2024-05-08T05:45:23.838423+0000","last_clean":"2024-05-08T05:45:23.838423+0000","last_became_active":"2024-05-08T05:45:23.836107+0000","last_became_peered":"2024-05-08T05:45:23.836107+0000","last_unstale":"2024-05-08T05:45:23.838423+0000","last_undegraded":"2024-05-08T05:45:23.838423+0000","last_fullsized":"2024-05-08T05:45:23.838423+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:44:13.285394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.822950+0000","last_change":"2024-05-08T05:45:23.822950+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.822950+0000","last_undegraded":"2024-05-08T05:45:23.822950+0000","last_fullsized":"2024-05-08T05:45:23.822950+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.847440+0000","last_change":"2024-05-08T05:45:23.825726+0000","last_active":"2024-05-08T05:45:24.847440+0000","last_peered":"2024-05-08T05:45:24.847440+0000","last_clean":"2024-05-08T05:45:24.847440+0000","last_became_active":"2024-05-08T05:45:23.824860+0000","last_became_peered":"2024-05-08T05:45:23.824860+0000","last_unstale":"2024-05-08T05:45:24.847440+0000","last_undegraded":"2024-05-08T05:45:24.847440+0000","last_fullsized":"2024-05-08T05:45:24.847440+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:09:59.746816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.849548+0000","last_change":"2024-05-08T05:45:24.841858+0000","last_active":"2024-05-08T05:45:24.849548+0000","last_peered":"2024-05-08T05:45:24.849548+0000","last_clean":"2024-05-08T05:45:24.849548+0000","last_became_active":"2024-05-08T05:45:24.841555+0000","last_became_peered":"2024-05-08T05:45:24.841555+0000","last_unstale":"2024-05-08T05:45:24.849548+0000","last_undegraded":"2024-05-08T05:45:24.849548+0000","last_fullsized":"2024-05-08T05:45:24.849548+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:29:34.946299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-05-08T05:45:23.835429+0000","last_change":"2024-05-08T05:45:23.834505+0000","last_active":"2024-05-08T05:45:23.835429+0000","last_peered":"2024-05-08T05:45:23.835429+0000","last_clean":"2024-05-08T05:45:23.835429+0000","last_became_active":"2024-05-08T05:45:23.834236+0000","last_became_peered":"2024-05-08T05:45:23.834236+0000","last_unstale":"2024-05-08T05:45:23.835429+0000","last_undegraded":"2024-05-08T05:45:23.835429+0000","last_fullsized":"2024-05-08T05:45:23.835429+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:06:53.906040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.822238+0000","last_change":"2024-05-08T05:45:23.822238+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.822238+0000","last_undegraded":"2024-05-08T05:45:23.822238+0000","last_fullsized":"2024-05-08T05:45:23.822238+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.834347+0000","last_change":"2024-05-08T05:45:23.832813+0000","last_active":"2024-05-08T05:45:24.834347+0000","last_peered":"2024-05-08T05:45:24.834347+0000","last_clean":"2024-05-08T05:45:24.834347+0000","last_became_active":"2024-05-08T05:45:23.832504+0000","last_became_peered":"2024-05-08T05:45:23.832504+0000","last_unstale":"2024-05-08T05:45:24.834347+0000","last_undegraded":"2024-05-08T05:45:24.834347+0000","last_fullsized":"2024-05-08T05:45:24.834347+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:27.462868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.822118+0000","last_change":"2024-05-08T05:45:23.822118+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.822118+0000","last_undegraded":"2024-05-08T05:45:23.822118+0000","last_fullsized":"2024-05-08T05:45:23.822118+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.833309+0000","last_change":"2024-05-08T05:45:23.836946+0000","last_active":"2024-05-08T05:45:24.833309+0000","last_peered":"2024-05-08T05:45:24.833309+0000","last_clean":"2024-05-08T05:45:24.833309+0000","last_became_active":"2024-05-08T05:45:23.836645+0000","last_became_peered":"2024-05-08T05:45:23.836645+0000","last_unstale":"2024-05-08T05:45:24.833309+0000","last_undegraded":"2024-05-08T05:45:24.833309+0000","last_fullsized":"2024-05-08T05:45:24.833309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:51.671640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.849661+0000","last_change":"2024-05-08T05:45:24.840500+0000","last_active":"2024-05-08T05:45:24.849661+0000","last_peered":"2024-05-08T05:45:24.849661+0000","last_clean":"2024-05-08T05:45:24.849661+0000","last_became_active":"2024-05-08T05:45:24.840216+0000","last_became_peered":"2024-05-08T05:45:24.840216+0000","last_unstale":"2024-05-08T05:45:24.849661+0000","last_undegraded":"2024-05-08T05:45:24.849661+0000","last_fullsized":"2024-05-08T05:45:24.849661+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:59:13.415098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.832420+0000","last_change":"2024-05-08T05:45:23.838025+0000","last_active":"2024-05-08T05:45:24.832420+0000","last_peered":"2024-05-08T05:45:24.832420+0000","last_clean":"2024-05-08T05:45:24.832420+0000","last_became_active":"2024-05-08T05:45:23.837700+0000","last_became_peered":"2024-05-08T05:45:23.837700+0000","last_unstale":"2024-05-08T05:45:24.832420+0000","last_undegraded":"2024-05-08T05:45:24.832420+0000","last_fullsized":"2024-05-08T05:45:24.832420+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:06.376863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.854040+0000","last_change":"2024-05-08T05:45:24.851900+0000","last_active":"2024-05-08T05:45:24.854040+0000","last_peered":"2024-05-08T05:45:24.854040+0000","last_clean":"2024-05-08T05:45:24.854040+0000","last_became_active":"2024-05-08T05:45:24.850686+0000","last_became_peered":"2024-05-08T05:45:24.850686+0000","last_unstale":"2024-05-08T05:45:24.854040+0000","last_undegraded":"2024-05-08T05:45:24.854040+0000","last_fullsized":"2024-05-08T05:45:24.854040+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:23.389303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-05-08T05:45:23.835324+0000","last_change":"2024-05-08T05:45:23.834461+0000","last_active":"2024-05-08T05:45:23.835324+0000","last_peered":"2024-05-08T05:45:23.835324+0000","last_clean":"2024-05-08T05:45:23.835324+0000","last_became_active":"2024-05-08T05:45:23.834167+0000","last_became_peered":"2024-05-08T05:45:23.834167+0000","last_unstale":"2024-05-08T05:45:23.835324+0000","last_undegraded":"2024-05-08T05:45:23.835324+0000","last_fullsized":"2024-05-08T05:45:23.835324+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:15:02.857561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.818318+0000","last_change":"2024-05-08T05:45:23.818318+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.818318+0000","last_undegraded":"2024-05-08T05:45:23.818318+0000","last_fullsized":"2024-05-08T05:45:23.818318+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.830294+0000","last_change":"2024-05-08T05:45:23.835556+0000","last_active":"2024-05-08T05:45:24.830294+0000","last_peered":"2024-05-08T05:45:24.830294+0000","last_clean":"2024-05-08T05:45:24.830294+0000","last_became_active":"2024-05-08T05:45:23.835249+0000","last_became_peered":"2024-05-08T05:45:23.835249+0000","last_unstale":"2024-05-08T05:45:24.830294+0000","last_undegraded":"2024-05-08T05:45:24.830294+0000","last_fullsized":"2024-05-08T05:45:24.830294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:45:58.599863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2024-05-08T05:45:23.818940+0000","last_change":"2024-05-08T05:45:23.818940+0000","last_active":"2024-05-08T05:45:23.810929+0000","last_peered":"2024-05-08T05:45:23.810929+0000","last_clean":"2024-05-08T05:45:23.810929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:45:23.818940+0000","last_undegraded":"2024-05-08T05:45:23.818940+0000","last_fullsized":"2024-05-08T05:45:23.818940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-05-08T05:45:23.828529+0000","last_change":"2024-05-08T05:45:23.826960+0000","last_active":"2024-05-08T05:45:23.828529+0000","last_peered":"2024-05-08T05:45:23.828529+0000","last_clean":"2024-05-08T05:45:23.828529+0000","last_became_active":"2024-05-08T05:45:23.826022+0000","last_became_peered":"2024-05-08T05:45:23.826022+0000","last_unstale":"2024-05-08T05:45:23.828529+0000","last_undegraded":"2024-05-08T05:45:23.828529+0000","last_fullsized":"2024-05-08T05:45:23.828529+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:05:26.421641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.847011+0000","last_change":"2024-05-08T05:45:24.840526+0000","last_active":"2024-05-08T05:45:24.847011+0000","last_peered":"2024-05-08T05:45:24.847011+0000","last_clean":"2024-05-08T05:45:24.847011+0000","last_became_active":"2024-05-08T05:45:24.840230+0000","last_became_peered":"2024-05-08T05:45:24.840230+0000","last_unstale":"2024-05-08T05:45:24.847011+0000","last_undegraded":"2024-05-08T05:45:24.847011+0000","last_fullsized":"2024-05-08T05:45:24.847011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:51:34.514053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-05-08T05:45:23.840829+0000","last_change":"2024-05-08T05:45:23.840044+0000","last_active":"2024-05-08T05:45:23.840829+0000","last_peered":"2024-05-08T05:45:23.840829+0000","last_clean":"2024-05-08T05:45:23.840829+0000","last_became_active":"2024-05-08T05:45:23.839699+0000","last_became_peered":"2024-05-08T05:45:23.839699+0000","last_unstale":"2024-05-08T05:45:23.840829+0000","last_undegraded":"2024-05-08T05:45:23.840829+0000","last_fullsized":"2024-05-08T05:45:23.840829+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:51.671640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":209,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.831426+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:45:24.831426+0000","last_peered":"2024-05-08T05:45:24.831426+0000","last_clean":"2024-05-08T05:45:24.831426+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:45:24.831426+0000","last_undegraded":"2024-05-08T05:45:24.831426+0000","last_fullsized":"2024-05-08T05:45:24.831426+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":1668,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542197,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28472,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343368,"statfs":{"total":96636764160,"available":96607608832,"internally_reserved":0,"allocated":1482752,"data_stored":991271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542198,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":888832,"data_stored":400903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542197,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":888832,"data_stored":400903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":394236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":876544,"data_stored":394236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":876544,"data_stored":394236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":876544,"data_stored":394236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1470464,"data_stored":984604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:45:28.894 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:45:29.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.033+0000 7f6b3fb89640 1 Processor -- start 2024-05-08T05:45:29.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.033+0000 7f6b3fb89640 1 -- start start 2024-05-08T05:45:29.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.034+0000 7f6b3fb89640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6b38070d30 0x7f6b380d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.034+0000 7f6b3fb89640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6b38071110 0x7f6b3806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.034+0000 7f6b3fb89640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b380d1300 con 0x7f6b38163920 2024-05-08T05:45:29.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.034+0000 7f6b3fb89640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f6b380d10d0 con 0x7f6b38070d30 2024-05-08T05:45:29.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.034+0000 7f6b3fb89640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6b38163db0 con 0x7f6b38071110 2024-05-08T05:45:29.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.034+0000 7f6b3d925640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6b38070d30 0x7f6b380d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:29.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.034+0000 7f6b3d124640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6b38163920 0x7f6b380703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34680/0 (socket says 172.21.15.45:34680) 2024-05-08T05:45:29.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.034+0000 7f6b3d124640 1 -- 172.21.15.45:0/1506367264 learned_addr learned my addr 172.21.15.45:0/1506367264 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:29.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.035+0000 7f6b3e126640 1 --2- 172.21.15.45:0/1506367264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6b38071110 0x7f6b3806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:29.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.035+0000 7f6b3d925640 1 -- 172.21.15.45:0/1506367264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6b38071110 msgr2=0x7f6b3806d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:29.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.035+0000 7f6b3d925640 1 --2- 172.21.15.45:0/1506367264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6b38071110 0x7f6b3806d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.035+0000 7f6b3d925640 1 -- 172.21.15.45:0/1506367264 >> v1:172.21.15.45:6789/0 conn(0x7f6b38163920 legacy=0x7f6b380703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:29.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.035+0000 7f6b3c923640 1 -- 172.21.15.45:0/1506367264 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1966584544 0 0) 0x7f6b380d1300 con 0x7f6b38163920 2024-05-08T05:45:29.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.035+0000 7f6b3d925640 1 -- 172.21.15.45:0/1506367264 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b380d15e0 con 0x7f6b38070d30 2024-05-08T05:45:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.035+0000 7f6b3d925640 1 --2- 172.21.15.45:0/1506367264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b38070d30 0x7f6b380d20c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6b300097a0 tx=0x7f6b3002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cd03b9e42caba8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.036+0000 7f6b3c923640 1 -- 172.21.15.45:0/1506367264 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b30037070 con 0x7f6b38070d30 2024-05-08T05:45:29.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.036+0000 7f6b3c923640 1 -- 172.21.15.45:0/1506367264 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b30032440 con 0x7f6b38070d30 2024-05-08T05:45:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.036+0000 7f6b3c923640 1 -- 172.21.15.45:0/1506367264 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b30032810 con 0x7f6b38070d30 2024-05-08T05:45:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.036+0000 7f6b3fb89640 1 -- 172.21.15.45:0/1506367264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b38070d30 msgr2=0x7f6b380d20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.036+0000 7f6b3fb89640 1 --2- 172.21.15.45:0/1506367264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b38070d30 0x7f6b380d20c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6b300097a0 tx=0x7f6b3002a180 comp rx=0 tx=0).stop 2024-05-08T05:45:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.036+0000 7f6b3fb89640 1 -- 172.21.15.45:0/1506367264 shutdown_connections 2024-05-08T05:45:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.036+0000 7f6b3fb89640 1 --2- 172.21.15.45:0/1506367264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6b38071110 0x7f6b3806d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.036+0000 7f6b3fb89640 1 --2- 172.21.15.45:0/1506367264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b38070d30 0x7f6b380d20c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.036+0000 7f6b3fb89640 1 -- 172.21.15.45:0/1506367264 >> 172.21.15.45:0/1506367264 conn(0x7f6b380d2e90 msgr2=0x7f6b380d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.036+0000 7f6b3fb89640 1 -- 172.21.15.45:0/1506367264 shutdown_connections 2024-05-08T05:45:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.036+0000 7f6b3fb89640 1 -- 172.21.15.45:0/1506367264 wait complete. 2024-05-08T05:45:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.037+0000 7f6b3fb89640 1 Processor -- start 2024-05-08T05:45:29.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.038+0000 7f6b3fb89640 1 -- start start 2024-05-08T05:45:29.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.038+0000 7f6b3fb89640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6b38070d30 0x7f6b381f9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.038+0000 7f6b3fb89640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b38071110 0x7f6b380cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.038+0000 7f6b3fb89640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6b38163920 0x7f6b380b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.038+0000 7f6b3fb89640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6b380d1c30 con 0x7f6b38071110 2024-05-08T05:45:29.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.038+0000 7f6b3fb89640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6b38070b80 con 0x7f6b38163920 2024-05-08T05:45:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.038+0000 7f6b3fb89640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6b380715a0 con 0x7f6b38070d30 2024-05-08T05:45:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.038+0000 7f6b3d124640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b38071110 0x7f6b380cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.038+0000 7f6b3d124640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b38071110 0x7f6b380cac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56558/0 (socket says 172.21.15.45:56558) 2024-05-08T05:45:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.038+0000 7f6b3d124640 1 -- 172.21.15.45:0/3274248664 learned_addr learned my addr 172.21.15.45:0/3274248664 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.038+0000 7f6b3d925640 1 --2- 172.21.15.45:0/3274248664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6b38070d30 0x7f6b381f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.039+0000 7f6b3e126640 1 --2- 172.21.15.45:0/3274248664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6b38163920 0x7f6b380b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.039+0000 7f6b3d124640 1 -- 172.21.15.45:0/3274248664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6b38070d30 msgr2=0x7f6b381f9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.039+0000 7f6b3d124640 1 --2- 172.21.15.45:0/3274248664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6b38070d30 0x7f6b381f9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.039+0000 7f6b3d124640 1 -- 172.21.15.45:0/3274248664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6b38163920 msgr2=0x7f6b380b6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.039+0000 7f6b3d124640 1 --2- 172.21.15.45:0/3274248664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6b38163920 0x7f6b380b6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.039+0000 7f6b3d124640 1 -- 172.21.15.45:0/3274248664 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b381f9640 con 0x7f6b38071110 2024-05-08T05:45:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.039+0000 7f6b3d124640 1 --2- 172.21.15.45:0/3274248664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b38071110 0x7f6b380cac80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00c3a0 tx=0x7f6b2c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.039+0000 7f6b367fc640 1 -- 172.21.15.45:0/3274248664 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c0074e0 con 0x7f6b38071110 2024-05-08T05:45:29.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.039+0000 7f6b367fc640 1 -- 172.21.15.45:0/3274248664 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b2c0047e0 con 0x7f6b38071110 2024-05-08T05:45:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.039+0000 7f6b367fc640 1 -- 172.21.15.45:0/3274248664 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c004bd0 con 0x7f6b38071110 2024-05-08T05:45:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.039+0000 7f6b3fb89640 1 -- 172.21.15.45:0/3274248664 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b381f9920 con 0x7f6b38071110 2024-05-08T05:45:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.039+0000 7f6b3fb89640 1 -- 172.21.15.45:0/3274248664 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b381f9ee0 con 0x7f6b38071110 2024-05-08T05:45:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.040+0000 7f6b3fb89640 1 -- 172.21.15.45:0/3274248664 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b380693f0 con 0x7f6b38071110 2024-05-08T05:45:29.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.041+0000 7f6b367fc640 1 -- 172.21.15.45:0/3274248664 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6b2c011070 con 0x7f6b38071110 2024-05-08T05:45:29.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.041+0000 7f6b367fc640 1 --2- 172.21.15.45:0/3274248664 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6b240762d0 0x7f6b24078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.041+0000 7f6b367fc640 1 -- 172.21.15.45:0/3274248664 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f6b2c0971d0 con 0x7f6b38071110 2024-05-08T05:45:29.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.041+0000 7f6b3d925640 1 --2- 172.21.15.45:0/3274248664 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6b240762d0 0x7f6b24078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:29.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.042+0000 7f6b3d925640 1 --2- 172.21.15.45:0/3274248664 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6b240762d0 0x7f6b24078790 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6b30034000 tx=0x7f6b3002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:29.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.046+0000 7f6b367fc640 1 -- 172.21.15.45:0/3274248664 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6b2c060a30 con 0x7f6b38071110 2024-05-08T05:45:29.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.231+0000 7f6b3fb89640 1 -- 172.21.15.45:0/3274248664 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6b380d3d70 con 0x7f6b240762d0 2024-05-08T05:45:29.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.238+0000 7f6b367fc640 1 -- 172.21.15.45:0/3274248664 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252039 (secure 0 0 0) 0x7f6b380d3d70 con 0x7f6b240762d0 2024-05-08T05:45:29.240 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:29.241 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:45:29.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.241+0000 7f6b3fb89640 1 -- 172.21.15.45:0/3274248664 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6b240762d0 msgr2=0x7f6b24078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:29.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.241+0000 7f6b3fb89640 1 --2- 172.21.15.45:0/3274248664 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6b240762d0 0x7f6b24078790 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6b30034000 tx=0x7f6b3002e000 comp rx=0 tx=0).stop 2024-05-08T05:45:29.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.241+0000 7f6b3fb89640 1 -- 172.21.15.45:0/3274248664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b38071110 msgr2=0x7f6b380cac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:29.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.241+0000 7f6b3fb89640 1 --2- 172.21.15.45:0/3274248664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b38071110 0x7f6b380cac80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00c3a0 tx=0x7f6b2c00c860 comp rx=0 tx=0).stop 2024-05-08T05:45:29.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.241+0000 7f6b3fb89640 1 -- 172.21.15.45:0/3274248664 shutdown_connections 2024-05-08T05:45:29.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.241+0000 7f6b3fb89640 1 --2- 172.21.15.45:0/3274248664 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6b240762d0 0x7f6b24078790 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.241+0000 7f6b3fb89640 1 --2- 172.21.15.45:0/3274248664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6b38163920 0x7f6b380b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.241+0000 7f6b3fb89640 1 --2- 172.21.15.45:0/3274248664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b38071110 0x7f6b380cac80 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.241+0000 7f6b3fb89640 1 --2- 172.21.15.45:0/3274248664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6b38070d30 0x7f6b381f9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.241+0000 7f6b3fb89640 1 -- 172.21.15.45:0/3274248664 >> 172.21.15.45:0/3274248664 conn(0x7f6b380d2e90 msgr2=0x7f6b381609f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:29.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.241+0000 7f6b3fb89640 1 -- 172.21.15.45:0/3274248664 shutdown_connections 2024-05-08T05:45:29.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.241+0000 7f6b3fb89640 1 -- 172.21.15.45:0/3274248664 wait complete. 2024-05-08T05:45:29.252 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-05-08T05:45:28.889058+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241088,"kb_used_data":8268,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754733632,"statfs":{"total":773094113280,"available":772847239168,"internally_reserved":0,"allocated":8466432,"data_stored":4460407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001375"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.851710+0000","last_change":"2024-05-08T05:45:24.849892+0000","last_active":"2024-05-08T05:45:24.851710+0000","last_peered":"2024-05-08T05:45:24.851710+0000","last_clean":"2024-05-08T05:45:24.851710+0000","last_became_active":"2024-05-08T05:45:24.849483+0000","last_became_peered":"2024-05-08T05:45:24.849483+0000","last_unstale":"2024-05-08T05:45:24.851710+0000","last_undegraded":"2024-05-08T05:45:24.851710+0000","last_fullsized":"2024-05-08T05:45:24.851710+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:35:45.649423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385139+0000","last_change":"2024-05-08T05:45:24.846638+0000","last_active":"2024-05-08T05:45:26.385139+0000","last_peered":"2024-05-08T05:45:26.385139+0000","last_clean":"2024-05-08T05:45:26.385139+0000","last_became_active":"2024-05-08T05:45:24.846020+0000","last_became_peered":"2024-05-08T05:45:24.846020+0000","last_unstale":"2024-05-08T05:45:26.385139+0000","last_undegraded":"2024-05-08T05:45:26.385139+0000","last_fullsized":"2024-05-08T05:45:26.385139+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:31.609131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385113+0000","last_change":"2024-05-08T05:45:24.837373+0000","last_active":"2024-05-08T05:45:26.385113+0000","last_peered":"2024-05-08T05:45:26.385113+0000","last_clean":"2024-05-08T05:45:26.385113+0000","last_became_active":"2024-05-08T05:45:24.837075+0000","last_became_peered":"2024-05-08T05:45:24.837075+0000","last_unstale":"2024-05-08T05:45:26.385113+0000","last_undegraded":"2024-05-08T05:45:26.385113+0000","last_fullsized":"2024-05-08T05:45:26.385113+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:29:37.253337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.852071+0000","last_change":"2024-05-08T05:45:24.849026+0000","last_active":"2024-05-08T05:45:24.852071+0000","last_peered":"2024-05-08T05:45:24.852071+0000","last_clean":"2024-05-08T05:45:24.852071+0000","last_became_active":"2024-05-08T05:45:24.848740+0000","last_became_peered":"2024-05-08T05:45:24.848740+0000","last_unstale":"2024-05-08T05:45:24.852071+0000","last_undegraded":"2024-05-08T05:45:24.852071+0000","last_fullsized":"2024-05-08T05:45:24.852071+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:25:33.241640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.386024+0000","last_change":"2024-05-08T05:45:24.849378+0000","last_active":"2024-05-08T05:45:26.386024+0000","last_peered":"2024-05-08T05:45:26.386024+0000","last_clean":"2024-05-08T05:45:26.386024+0000","last_became_active":"2024-05-08T05:45:24.849122+0000","last_became_peered":"2024-05-08T05:45:24.849122+0000","last_unstale":"2024-05-08T05:45:26.386024+0000","last_undegraded":"2024-05-08T05:45:26.386024+0000","last_fullsized":"2024-05-08T05:45:26.386024+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:19:00.033818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:25.839412+0000","last_change":"2024-05-08T05:45:24.855500+0000","last_active":"2024-05-08T05:45:25.839412+0000","last_peered":"2024-05-08T05:45:25.839412+0000","last_clean":"2024-05-08T05:45:25.839412+0000","last_became_active":"2024-05-08T05:45:24.855175+0000","last_became_peered":"2024-05-08T05:45:24.855175+0000","last_unstale":"2024-05-08T05:45:25.839412+0000","last_undegraded":"2024-05-08T05:45:25.839412+0000","last_fullsized":"2024-05-08T05:45:25.839412+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:47:58.506113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.852654+0000","last_change":"2024-05-08T05:45:24.850111+0000","last_active":"2024-05-08T05:45:24.852654+0000","last_peered":"2024-05-08T05:45:24.852654+0000","last_clean":"2024-05-08T05:45:24.852654+0000","last_became_active":"2024-05-08T05:45:24.849810+0000","last_became_peered":"2024-05-08T05:45:24.849810+0000","last_unstale":"2024-05-08T05:45:24.852654+0000","last_undegraded":"2024-05-08T05:45:24.852654+0000","last_fullsized":"2024-05-08T05:45:24.852654+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:59:57.843585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385777+0000","last_change":"2024-05-08T05:45:24.849964+0000","last_active":"2024-05-08T05:45:26.385777+0000","last_peered":"2024-05-08T05:45:26.385777+0000","last_clean":"2024-05-08T05:45:26.385777+0000","last_became_active":"2024-05-08T05:45:24.849554+0000","last_became_peered":"2024-05-08T05:45:24.849554+0000","last_unstale":"2024-05-08T05:45:26.385777+0000","last_undegraded":"2024-05-08T05:45:26.385777+0000","last_fullsized":"2024-05-08T05:45:26.385777+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:22:20.674958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.386263+0000","last_change":"2024-05-08T05:45:24.852189+0000","last_active":"2024-05-08T05:45:26.386263+0000","last_peered":"2024-05-08T05:45:26.386263+0000","last_clean":"2024-05-08T05:45:26.386263+0000","last_became_active":"2024-05-08T05:45:24.851485+0000","last_became_peered":"2024-05-08T05:45:24.851485+0000","last_unstale":"2024-05-08T05:45:26.386263+0000","last_undegraded":"2024-05-08T05:45:26.386263+0000","last_fullsized":"2024-05-08T05:45:26.386263+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:39:03.646823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.846430+0000","last_change":"2024-05-08T05:45:24.841146+0000","last_active":"2024-05-08T05:45:24.846430+0000","last_peered":"2024-05-08T05:45:24.846430+0000","last_clean":"2024-05-08T05:45:24.846430+0000","last_became_active":"2024-05-08T05:45:24.840652+0000","last_became_peered":"2024-05-08T05:45:24.840652+0000","last_unstale":"2024-05-08T05:45:24.846430+0000","last_undegraded":"2024-05-08T05:45:24.846430+0000","last_fullsized":"2024-05-08T05:45:24.846430+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:38:38.823695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385405+0000","last_change":"2024-05-08T05:45:24.844639+0000","last_active":"2024-05-08T05:45:26.385405+0000","last_peered":"2024-05-08T05:45:26.385405+0000","last_clean":"2024-05-08T05:45:26.385405+0000","last_became_active":"2024-05-08T05:45:24.844393+0000","last_became_peered":"2024-05-08T05:45:24.844393+0000","last_unstale":"2024-05-08T05:45:26.385405+0000","last_undegraded":"2024-05-08T05:45:26.385405+0000","last_fullsized":"2024-05-08T05:45:26.385405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:47:58.506113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.847096+0000","last_change":"2024-05-08T05:45:24.840089+0000","last_active":"2024-05-08T05:45:24.847096+0000","last_peered":"2024-05-08T05:45:24.847096+0000","last_clean":"2024-05-08T05:45:24.847096+0000","last_became_active":"2024-05-08T05:45:24.839767+0000","last_became_peered":"2024-05-08T05:45:24.839767+0000","last_unstale":"2024-05-08T05:45:24.847096+0000","last_undegraded":"2024-05-08T05:45:24.847096+0000","last_fullsized":"2024-05-08T05:45:24.847096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:32:06.497004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.386098+0000","last_change":"2024-05-08T05:45:24.846489+0000","last_active":"2024-05-08T05:45:26.386098+0000","last_peered":"2024-05-08T05:45:26.386098+0000","last_clean":"2024-05-08T05:45:26.386098+0000","last_became_active":"2024-05-08T05:45:24.846191+0000","last_became_peered":"2024-05-08T05:45:24.846191+0000","last_unstale":"2024-05-08T05:45:26.386098+0000","last_undegraded":"2024-05-08T05:45:26.386098+0000","last_fullsized":"2024-05-08T05:45:26.386098+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:56:57.546474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:25.839570+0000","last_change":"2024-05-08T05:45:24.854870+0000","last_active":"2024-05-08T05:45:25.839570+0000","last_peered":"2024-05-08T05:45:25.839570+0000","last_clean":"2024-05-08T05:45:25.839570+0000","last_became_active":"2024-05-08T05:45:24.854212+0000","last_became_peered":"2024-05-08T05:45:24.854212+0000","last_unstale":"2024-05-08T05:45:25.839570+0000","last_undegraded":"2024-05-08T05:45:25.839570+0000","last_fullsized":"2024-05-08T05:45:25.839570+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:34:33.124488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385019+0000","last_change":"2024-05-08T05:45:24.832694+0000","last_active":"2024-05-08T05:45:26.385019+0000","last_peered":"2024-05-08T05:45:26.385019+0000","last_clean":"2024-05-08T05:45:26.385019+0000","last_became_active":"2024-05-08T05:45:24.832411+0000","last_became_peered":"2024-05-08T05:45:24.832411+0000","last_unstale":"2024-05-08T05:45:26.385019+0000","last_undegraded":"2024-05-08T05:45:26.385019+0000","last_fullsized":"2024-05-08T05:45:26.385019+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:00.887389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.847010+0000","last_change":"2024-05-08T05:45:24.838511+0000","last_active":"2024-05-08T05:45:24.847010+0000","last_peered":"2024-05-08T05:45:24.847010+0000","last_clean":"2024-05-08T05:45:24.847010+0000","last_became_active":"2024-05-08T05:45:24.838217+0000","last_became_peered":"2024-05-08T05:45:24.838217+0000","last_unstale":"2024-05-08T05:45:24.847010+0000","last_undegraded":"2024-05-08T05:45:24.847010+0000","last_fullsized":"2024-05-08T05:45:24.847010+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:43:37.991934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.386278+0000","last_change":"2024-05-08T05:45:24.851809+0000","last_active":"2024-05-08T05:45:26.386278+0000","last_peered":"2024-05-08T05:45:26.386278+0000","last_clean":"2024-05-08T05:45:26.386278+0000","last_became_active":"2024-05-08T05:45:24.850965+0000","last_became_peered":"2024-05-08T05:45:24.850965+0000","last_unstale":"2024-05-08T05:45:26.386278+0000","last_undegraded":"2024-05-08T05:45:26.386278+0000","last_fullsized":"2024-05-08T05:45:26.386278+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:15:31.088727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385327+0000","last_change":"2024-05-08T05:45:24.849050+0000","last_active":"2024-05-08T05:45:26.385327+0000","last_peered":"2024-05-08T05:45:26.385327+0000","last_clean":"2024-05-08T05:45:26.385327+0000","last_became_active":"2024-05-08T05:45:24.848529+0000","last_became_peered":"2024-05-08T05:45:24.848529+0000","last_unstale":"2024-05-08T05:45:26.385327+0000","last_undegraded":"2024-05-08T05:45:26.385327+0000","last_fullsized":"2024-05-08T05:45:26.385327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:06:54.908334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:25.839401+0000","last_change":"2024-05-08T05:45:24.849448+0000","last_active":"2024-05-08T05:45:25.839401+0000","last_peered":"2024-05-08T05:45:25.839401+0000","last_clean":"2024-05-08T05:45:25.839401+0000","last_became_active":"2024-05-08T05:45:24.849169+0000","last_became_peered":"2024-05-08T05:45:24.849169+0000","last_unstale":"2024-05-08T05:45:25.839401+0000","last_undegraded":"2024-05-08T05:45:25.839401+0000","last_fullsized":"2024-05-08T05:45:25.839401+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:03:32.629588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.386100+0000","last_change":"2024-05-08T05:45:24.844112+0000","last_active":"2024-05-08T05:45:26.386100+0000","last_peered":"2024-05-08T05:45:26.386100+0000","last_clean":"2024-05-08T05:45:26.386100+0000","last_became_active":"2024-05-08T05:45:24.843821+0000","last_became_peered":"2024-05-08T05:45:24.843821+0000","last_unstale":"2024-05-08T05:45:26.386100+0000","last_undegraded":"2024-05-08T05:45:26.386100+0000","last_fullsized":"2024-05-08T05:45:26.386100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:26:24.729447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.386034+0000","last_change":"2024-05-08T05:45:24.848901+0000","last_active":"2024-05-08T05:45:26.386034+0000","last_peered":"2024-05-08T05:45:26.386034+0000","last_clean":"2024-05-08T05:45:26.386034+0000","last_became_active":"2024-05-08T05:45:24.848336+0000","last_became_peered":"2024-05-08T05:45:24.848336+0000","last_unstale":"2024-05-08T05:45:26.386034+0000","last_undegraded":"2024-05-08T05:45:26.386034+0000","last_fullsized":"2024-05-08T05:45:26.386034+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:05:27.423934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385797+0000","last_change":"2024-05-08T05:45:24.848451+0000","last_active":"2024-05-08T05:45:26.385797+0000","last_peered":"2024-05-08T05:45:26.385797+0000","last_clean":"2024-05-08T05:45:26.385797+0000","last_became_active":"2024-05-08T05:45:24.848164+0000","last_became_peered":"2024-05-08T05:45:24.848164+0000","last_unstale":"2024-05-08T05:45:26.385797+0000","last_undegraded":"2024-05-08T05:45:26.385797+0000","last_fullsized":"2024-05-08T05:45:26.385797+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:57:41.935211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385252+0000","last_change":"2024-05-08T05:45:24.845458+0000","last_active":"2024-05-08T05:45:26.385252+0000","last_peered":"2024-05-08T05:45:26.385252+0000","last_clean":"2024-05-08T05:45:26.385252+0000","last_became_active":"2024-05-08T05:45:24.843862+0000","last_became_peered":"2024-05-08T05:45:24.843862+0000","last_unstale":"2024-05-08T05:45:26.385252+0000","last_undegraded":"2024-05-08T05:45:26.385252+0000","last_fullsized":"2024-05-08T05:45:26.385252+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:30.559925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.384883+0000","last_change":"2024-05-08T05:45:24.839777+0000","last_active":"2024-05-08T05:45:26.384883+0000","last_peered":"2024-05-08T05:45:26.384883+0000","last_clean":"2024-05-08T05:45:26.384883+0000","last_became_active":"2024-05-08T05:45:24.837315+0000","last_became_peered":"2024-05-08T05:45:24.837315+0000","last_unstale":"2024-05-08T05:45:26.384883+0000","last_undegraded":"2024-05-08T05:45:26.384883+0000","last_fullsized":"2024-05-08T05:45:26.384883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:51:34.514053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385288+0000","last_change":"2024-05-08T05:45:24.839488+0000","last_active":"2024-05-08T05:45:26.385288+0000","last_peered":"2024-05-08T05:45:26.385288+0000","last_clean":"2024-05-08T05:45:26.385288+0000","last_became_active":"2024-05-08T05:45:24.837275+0000","last_became_peered":"2024-05-08T05:45:24.837275+0000","last_unstale":"2024-05-08T05:45:26.385288+0000","last_undegraded":"2024-05-08T05:45:26.385288+0000","last_fullsized":"2024-05-08T05:45:26.385288+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:50:19.682079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.846405+0000","last_change":"2024-05-08T05:45:24.840913+0000","last_active":"2024-05-08T05:45:24.846405+0000","last_peered":"2024-05-08T05:45:24.846405+0000","last_clean":"2024-05-08T05:45:24.846405+0000","last_became_active":"2024-05-08T05:45:24.840291+0000","last_became_peered":"2024-05-08T05:45:24.840291+0000","last_unstale":"2024-05-08T05:45:24.846405+0000","last_undegraded":"2024-05-08T05:45:24.846405+0000","last_fullsized":"2024-05-08T05:45:24.846405+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:18:01.264013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.847044+0000","last_change":"2024-05-08T05:45:24.843174+0000","last_active":"2024-05-08T05:45:24.847044+0000","last_peered":"2024-05-08T05:45:24.847044+0000","last_clean":"2024-05-08T05:45:24.847044+0000","last_became_active":"2024-05-08T05:45:24.842213+0000","last_became_peered":"2024-05-08T05:45:24.842213+0000","last_unstale":"2024-05-08T05:45:24.847044+0000","last_undegraded":"2024-05-08T05:45:24.847044+0000","last_fullsized":"2024-05-08T05:45:24.847044+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:25:54.796485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.386256+0000","last_change":"2024-05-08T05:45:24.842670+0000","last_active":"2024-05-08T05:45:26.386256+0000","last_peered":"2024-05-08T05:45:26.386256+0000","last_clean":"2024-05-08T05:45:26.386256+0000","last_became_active":"2024-05-08T05:45:24.842414+0000","last_became_peered":"2024-05-08T05:45:24.842414+0000","last_unstale":"2024-05-08T05:45:26.386256+0000","last_undegraded":"2024-05-08T05:45:26.386256+0000","last_fullsized":"2024-05-08T05:45:26.386256+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:05:33.579203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385057+0000","last_change":"2024-05-08T05:45:24.833276+0000","last_active":"2024-05-08T05:45:26.385057+0000","last_peered":"2024-05-08T05:45:26.385057+0000","last_clean":"2024-05-08T05:45:26.385057+0000","last_became_active":"2024-05-08T05:45:24.833016+0000","last_became_peered":"2024-05-08T05:45:24.833016+0000","last_unstale":"2024-05-08T05:45:26.385057+0000","last_undegraded":"2024-05-08T05:45:26.385057+0000","last_fullsized":"2024-05-08T05:45:26.385057+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:25:54.796485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.847039+0000","last_change":"2024-05-08T05:45:24.842920+0000","last_active":"2024-05-08T05:45:24.847039+0000","last_peered":"2024-05-08T05:45:24.847039+0000","last_clean":"2024-05-08T05:45:24.847039+0000","last_became_active":"2024-05-08T05:45:24.842628+0000","last_became_peered":"2024-05-08T05:45:24.842628+0000","last_unstale":"2024-05-08T05:45:24.847039+0000","last_undegraded":"2024-05-08T05:45:24.847039+0000","last_fullsized":"2024-05-08T05:45:24.847039+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:00.887389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.384849+0000","last_change":"2024-05-08T05:45:24.836618+0000","last_active":"2024-05-08T05:45:26.384849+0000","last_peered":"2024-05-08T05:45:26.384849+0000","last_clean":"2024-05-08T05:45:26.384849+0000","last_became_active":"2024-05-08T05:45:24.836356+0000","last_became_peered":"2024-05-08T05:45:24.836356+0000","last_unstale":"2024-05-08T05:45:26.384849+0000","last_undegraded":"2024-05-08T05:45:26.384849+0000","last_fullsized":"2024-05-08T05:45:26.384849+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:24.831405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.855173+0000","last_change":"2024-05-08T05:45:24.850149+0000","last_active":"2024-05-08T05:45:24.855173+0000","last_peered":"2024-05-08T05:45:24.855173+0000","last_clean":"2024-05-08T05:45:24.855173+0000","last_became_active":"2024-05-08T05:45:24.849888+0000","last_became_peered":"2024-05-08T05:45:24.849888+0000","last_unstale":"2024-05-08T05:45:24.855173+0000","last_undegraded":"2024-05-08T05:45:24.855173+0000","last_fullsized":"2024-05-08T05:45:24.855173+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:31.609131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.384707+0000","last_change":"2024-05-08T05:45:24.850017+0000","last_active":"2024-05-08T05:45:26.384707+0000","last_peered":"2024-05-08T05:45:26.384707+0000","last_clean":"2024-05-08T05:45:26.384707+0000","last_became_active":"2024-05-08T05:45:24.849497+0000","last_became_peered":"2024-05-08T05:45:24.849497+0000","last_unstale":"2024-05-08T05:45:26.384707+0000","last_undegraded":"2024-05-08T05:45:26.384707+0000","last_fullsized":"2024-05-08T05:45:26.384707+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:59:57.843585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"74'4","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.386349+0000","last_change":"2024-05-08T05:45:23.839361+0000","last_active":"2024-05-08T05:45:26.386349+0000","last_peered":"2024-05-08T05:45:26.386349+0000","last_clean":"2024-05-08T05:45:26.386349+0000","last_became_active":"2024-05-08T05:45:23.839066+0000","last_became_peered":"2024-05-08T05:45:23.839066+0000","last_unstale":"2024-05-08T05:45:26.386349+0000","last_undegraded":"2024-05-08T05:45:26.386349+0000","last_fullsized":"2024-05-08T05:45:26.386349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:44:05.053731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385116+0000","last_change":"2024-05-08T05:45:24.837104+0000","last_active":"2024-05-08T05:45:26.385116+0000","last_peered":"2024-05-08T05:45:26.385116+0000","last_clean":"2024-05-08T05:45:26.385116+0000","last_became_active":"2024-05-08T05:45:24.836495+0000","last_became_peered":"2024-05-08T05:45:24.836495+0000","last_unstale":"2024-05-08T05:45:26.385116+0000","last_undegraded":"2024-05-08T05:45:26.385116+0000","last_fullsized":"2024-05-08T05:45:26.385116+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:35:45.649423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"74'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.852801+0000","last_change":"2024-05-08T05:45:23.839805+0000","last_active":"2024-05-08T05:45:24.852801+0000","last_peered":"2024-05-08T05:45:24.852801+0000","last_clean":"2024-05-08T05:45:24.852801+0000","last_became_active":"2024-05-08T05:45:23.839509+0000","last_became_peered":"2024-05-08T05:45:23.839509+0000","last_unstale":"2024-05-08T05:45:24.852801+0000","last_undegraded":"2024-05-08T05:45:24.852801+0000","last_fullsized":"2024-05-08T05:45:24.852801+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:01:30.445032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.386120+0000","last_change":"2024-05-08T05:45:24.848283+0000","last_active":"2024-05-08T05:45:26.386120+0000","last_peered":"2024-05-08T05:45:26.386120+0000","last_clean":"2024-05-08T05:45:26.386120+0000","last_became_active":"2024-05-08T05:45:24.847565+0000","last_became_peered":"2024-05-08T05:45:24.847565+0000","last_unstale":"2024-05-08T05:45:26.386120+0000","last_undegraded":"2024-05-08T05:45:26.386120+0000","last_fullsized":"2024-05-08T05:45:26.386120+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:45:59.602157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.853636+0000","last_change":"2024-05-08T05:45:23.832769+0000","last_active":"2024-05-08T05:45:24.853636+0000","last_peered":"2024-05-08T05:45:24.853636+0000","last_clean":"2024-05-08T05:45:24.853636+0000","last_became_active":"2024-05-08T05:45:23.832465+0000","last_became_peered":"2024-05-08T05:45:23.832465+0000","last_unstale":"2024-05-08T05:45:24.853636+0000","last_undegraded":"2024-05-08T05:45:24.853636+0000","last_fullsized":"2024-05-08T05:45:24.853636+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:05.456807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.386239+0000","last_change":"2024-05-08T05:45:24.845610+0000","last_active":"2024-05-08T05:45:26.386239+0000","last_peered":"2024-05-08T05:45:26.386239+0000","last_clean":"2024-05-08T05:45:26.386239+0000","last_became_active":"2024-05-08T05:45:24.845320+0000","last_became_peered":"2024-05-08T05:45:24.845320+0000","last_unstale":"2024-05-08T05:45:26.386239+0000","last_undegraded":"2024-05-08T05:45:26.386239+0000","last_fullsized":"2024-05-08T05:45:26.386239+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:07.379156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.852680+0000","last_change":"2024-05-08T05:45:23.837917+0000","last_active":"2024-05-08T05:45:24.852680+0000","last_peered":"2024-05-08T05:45:24.852680+0000","last_clean":"2024-05-08T05:45:24.852680+0000","last_became_active":"2024-05-08T05:45:23.837605+0000","last_became_peered":"2024-05-08T05:45:23.837605+0000","last_unstale":"2024-05-08T05:45:24.852680+0000","last_undegraded":"2024-05-08T05:45:24.852680+0000","last_fullsized":"2024-05-08T05:45:24.852680+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:39:02.644530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385058+0000","last_change":"2024-05-08T05:45:24.846580+0000","last_active":"2024-05-08T05:45:26.385058+0000","last_peered":"2024-05-08T05:45:26.385058+0000","last_clean":"2024-05-08T05:45:26.385058+0000","last_became_active":"2024-05-08T05:45:24.846278+0000","last_became_peered":"2024-05-08T05:45:24.846278+0000","last_unstale":"2024-05-08T05:45:26.385058+0000","last_undegraded":"2024-05-08T05:45:26.385058+0000","last_fullsized":"2024-05-08T05:45:26.385058+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:07:58.927739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.386441+0000","last_change":"2024-05-08T05:45:23.833571+0000","last_active":"2024-05-08T05:45:26.386441+0000","last_peered":"2024-05-08T05:45:26.386441+0000","last_clean":"2024-05-08T05:45:26.386441+0000","last_became_active":"2024-05-08T05:45:23.833258+0000","last_became_peered":"2024-05-08T05:45:23.833258+0000","last_unstale":"2024-05-08T05:45:26.386441+0000","last_undegraded":"2024-05-08T05:45:26.386441+0000","last_fullsized":"2024-05-08T05:45:26.386441+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:25:46.304408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385441+0000","last_change":"2024-05-08T05:45:24.844018+0000","last_active":"2024-05-08T05:45:26.385441+0000","last_peered":"2024-05-08T05:45:26.385441+0000","last_clean":"2024-05-08T05:45:26.385441+0000","last_became_active":"2024-05-08T05:45:24.843728+0000","last_became_peered":"2024-05-08T05:45:24.843728+0000","last_unstale":"2024-05-08T05:45:26.385441+0000","last_undegraded":"2024-05-08T05:45:26.385441+0000","last_fullsized":"2024-05-08T05:45:26.385441+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:35.776683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"74'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.855459+0000","last_change":"2024-05-08T05:45:23.828059+0000","last_active":"2024-05-08T05:45:24.855459+0000","last_peered":"2024-05-08T05:45:24.855459+0000","last_clean":"2024-05-08T05:45:24.855459+0000","last_became_active":"2024-05-08T05:45:23.827769+0000","last_became_peered":"2024-05-08T05:45:23.827769+0000","last_unstale":"2024-05-08T05:45:24.855459+0000","last_undegraded":"2024-05-08T05:45:24.855459+0000","last_fullsized":"2024-05-08T05:45:24.855459+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:07:35.947497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385699+0000","last_change":"2024-05-08T05:45:24.847356+0000","last_active":"2024-05-08T05:45:26.385699+0000","last_peered":"2024-05-08T05:45:26.385699+0000","last_clean":"2024-05-08T05:45:26.385699+0000","last_became_active":"2024-05-08T05:45:24.847013+0000","last_became_peered":"2024-05-08T05:45:24.847013+0000","last_unstale":"2024-05-08T05:45:26.385699+0000","last_undegraded":"2024-05-08T05:45:26.385699+0000","last_fullsized":"2024-05-08T05:45:26.385699+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:28:15.982342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385221+0000","last_change":"2024-05-08T05:45:23.836149+0000","last_active":"2024-05-08T05:45:26.385221+0000","last_peered":"2024-05-08T05:45:26.385221+0000","last_clean":"2024-05-08T05:45:26.385221+0000","last_became_active":"2024-05-08T05:45:23.835794+0000","last_became_peered":"2024-05-08T05:45:23.835794+0000","last_unstale":"2024-05-08T05:45:26.385221+0000","last_undegraded":"2024-05-08T05:45:26.385221+0000","last_fullsized":"2024-05-08T05:45:26.385221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:00:21.476190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.852754+0000","last_change":"2024-05-08T05:45:24.851132+0000","last_active":"2024-05-08T05:45:24.852754+0000","last_peered":"2024-05-08T05:45:24.852754+0000","last_clean":"2024-05-08T05:45:24.852754+0000","last_became_active":"2024-05-08T05:45:24.850840+0000","last_became_peered":"2024-05-08T05:45:24.850840+0000","last_unstale":"2024-05-08T05:45:24.852754+0000","last_undegraded":"2024-05-08T05:45:24.852754+0000","last_fullsized":"2024-05-08T05:45:24.852754+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:24.831405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385698+0000","last_change":"2024-05-08T05:45:23.833511+0000","last_active":"2024-05-08T05:45:26.385698+0000","last_peered":"2024-05-08T05:45:26.385698+0000","last_clean":"2024-05-08T05:45:26.385698+0000","last_became_active":"2024-05-08T05:45:23.833232+0000","last_became_peered":"2024-05-08T05:45:23.833232+0000","last_unstale":"2024-05-08T05:45:26.385698+0000","last_undegraded":"2024-05-08T05:45:26.385698+0000","last_fullsized":"2024-05-08T05:45:26.385698+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:56:06.894676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.384703+0000","last_change":"2024-05-08T05:45:24.835918+0000","last_active":"2024-05-08T05:45:26.384703+0000","last_peered":"2024-05-08T05:45:26.384703+0000","last_clean":"2024-05-08T05:45:26.384703+0000","last_became_active":"2024-05-08T05:45:24.835654+0000","last_became_peered":"2024-05-08T05:45:24.835654+0000","last_unstale":"2024-05-08T05:45:26.384703+0000","last_undegraded":"2024-05-08T05:45:26.384703+0000","last_fullsized":"2024-05-08T05:45:26.384703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:43:37.991934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.836338+0000","last_change":"2024-05-08T05:45:23.839833+0000","last_active":"2024-05-08T05:45:24.836338+0000","last_peered":"2024-05-08T05:45:24.836338+0000","last_clean":"2024-05-08T05:45:24.836338+0000","last_became_active":"2024-05-08T05:45:23.839539+0000","last_became_peered":"2024-05-08T05:45:23.839539+0000","last_unstale":"2024-05-08T05:45:24.836338+0000","last_undegraded":"2024-05-08T05:45:24.836338+0000","last_fullsized":"2024-05-08T05:45:24.836338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:49.025014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.852198+0000","last_change":"2024-05-08T05:45:24.849376+0000","last_active":"2024-05-08T05:45:24.852198+0000","last_peered":"2024-05-08T05:45:24.852198+0000","last_clean":"2024-05-08T05:45:24.852198+0000","last_became_active":"2024-05-08T05:45:24.849085+0000","last_became_peered":"2024-05-08T05:45:24.849085+0000","last_unstale":"2024-05-08T05:45:24.852198+0000","last_undegraded":"2024-05-08T05:45:24.852198+0000","last_fullsized":"2024-05-08T05:45:24.852198+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:58:21.085324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.835791+0000","last_change":"2024-05-08T05:45:23.833756+0000","last_active":"2024-05-08T05:45:24.835791+0000","last_peered":"2024-05-08T05:45:24.835791+0000","last_clean":"2024-05-08T05:45:24.835791+0000","last_became_active":"2024-05-08T05:45:23.833496+0000","last_became_peered":"2024-05-08T05:45:23.833496+0000","last_unstale":"2024-05-08T05:45:24.835791+0000","last_undegraded":"2024-05-08T05:45:24.835791+0000","last_fullsized":"2024-05-08T05:45:24.835791+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:58:08.284451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:25.839713+0000","last_change":"2024-05-08T05:45:24.839655+0000","last_active":"2024-05-08T05:45:25.839713+0000","last_peered":"2024-05-08T05:45:25.839713+0000","last_clean":"2024-05-08T05:45:25.839713+0000","last_became_active":"2024-05-08T05:45:24.839382+0000","last_became_peered":"2024-05-08T05:45:24.839382+0000","last_unstale":"2024-05-08T05:45:25.839713+0000","last_undegraded":"2024-05-08T05:45:25.839713+0000","last_fullsized":"2024-05-08T05:45:25.839713+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:25:03.727013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"74'3","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.386482+0000","last_change":"2024-05-08T05:45:23.836748+0000","last_active":"2024-05-08T05:45:26.386482+0000","last_peered":"2024-05-08T05:45:26.386482+0000","last_clean":"2024-05-08T05:45:26.386482+0000","last_became_active":"2024-05-08T05:45:23.836069+0000","last_became_peered":"2024-05-08T05:45:23.836069+0000","last_unstale":"2024-05-08T05:45:26.386482+0000","last_undegraded":"2024-05-08T05:45:26.386482+0000","last_fullsized":"2024-05-08T05:45:26.386482+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:16:34.773144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385103+0000","last_change":"2024-05-08T05:45:24.834729+0000","last_active":"2024-05-08T05:45:26.385103+0000","last_peered":"2024-05-08T05:45:26.385103+0000","last_clean":"2024-05-08T05:45:26.385103+0000","last_became_active":"2024-05-08T05:45:24.834460+0000","last_became_peered":"2024-05-08T05:45:24.834460+0000","last_unstale":"2024-05-08T05:45:26.385103+0000","last_undegraded":"2024-05-08T05:45:26.385103+0000","last_fullsized":"2024-05-08T05:45:26.385103+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:23.389303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"74'6","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.858199+0000","last_change":"2024-05-08T05:45:23.830570+0000","last_active":"2024-05-08T05:45:24.858199+0000","last_peered":"2024-05-08T05:45:24.858199+0000","last_clean":"2024-05-08T05:45:24.858199+0000","last_became_active":"2024-05-08T05:45:23.830284+0000","last_became_peered":"2024-05-08T05:45:23.830284+0000","last_unstale":"2024-05-08T05:45:24.858199+0000","last_undegraded":"2024-05-08T05:45:24.858199+0000","last_fullsized":"2024-05-08T05:45:24.858199+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:57:32.335429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.847044+0000","last_change":"2024-05-08T05:45:24.839688+0000","last_active":"2024-05-08T05:45:24.847044+0000","last_peered":"2024-05-08T05:45:24.847044+0000","last_clean":"2024-05-08T05:45:24.847044+0000","last_became_active":"2024-05-08T05:45:24.838944+0000","last_became_peered":"2024-05-08T05:45:24.838944+0000","last_unstale":"2024-05-08T05:45:24.847044+0000","last_undegraded":"2024-05-08T05:45:24.847044+0000","last_fullsized":"2024-05-08T05:45:24.847044+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:00:22.478483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.854365+0000","last_change":"2024-05-08T05:45:23.829939+0000","last_active":"2024-05-08T05:45:24.854365+0000","last_peered":"2024-05-08T05:45:24.854365+0000","last_clean":"2024-05-08T05:45:24.854365+0000","last_became_active":"2024-05-08T05:45:23.829637+0000","last_became_peered":"2024-05-08T05:45:23.829637+0000","last_unstale":"2024-05-08T05:45:24.854365+0000","last_undegraded":"2024-05-08T05:45:24.854365+0000","last_fullsized":"2024-05-08T05:45:24.854365+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:00.079205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.850400+0000","last_change":"2024-05-08T05:45:24.841504+0000","last_active":"2024-05-08T05:45:24.850400+0000","last_peered":"2024-05-08T05:45:24.850400+0000","last_clean":"2024-05-08T05:45:24.850400+0000","last_became_active":"2024-05-08T05:45:24.841183+0000","last_became_peered":"2024-05-08T05:45:24.841183+0000","last_unstale":"2024-05-08T05:45:24.850400+0000","last_undegraded":"2024-05-08T05:45:24.850400+0000","last_fullsized":"2024-05-08T05:45:24.850400+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:40.615500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.831149+0000","last_change":"2024-05-08T05:45:23.839358+0000","last_active":"2024-05-08T05:45:24.831149+0000","last_peered":"2024-05-08T05:45:24.831149+0000","last_clean":"2024-05-08T05:45:24.831149+0000","last_became_active":"2024-05-08T05:45:23.839057+0000","last_became_peered":"2024-05-08T05:45:23.839057+0000","last_unstale":"2024-05-08T05:45:24.831149+0000","last_undegraded":"2024-05-08T05:45:24.831149+0000","last_fullsized":"2024-05-08T05:45:24.831149+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:26.832027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.851353+0000","last_change":"2024-05-08T05:45:24.844754+0000","last_active":"2024-05-08T05:45:24.851353+0000","last_peered":"2024-05-08T05:45:24.851353+0000","last_clean":"2024-05-08T05:45:24.851353+0000","last_became_active":"2024-05-08T05:45:24.844469+0000","last_became_peered":"2024-05-08T05:45:24.844469+0000","last_unstale":"2024-05-08T05:45:24.851353+0000","last_undegraded":"2024-05-08T05:45:24.851353+0000","last_fullsized":"2024-05-08T05:45:24.851353+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:47:54.532670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.386252+0000","last_change":"2024-05-08T05:45:23.835641+0000","last_active":"2024-05-08T05:45:26.386252+0000","last_peered":"2024-05-08T05:45:26.386252+0000","last_clean":"2024-05-08T05:45:26.386252+0000","last_became_active":"2024-05-08T05:45:23.835131+0000","last_became_peered":"2024-05-08T05:45:23.835131+0000","last_unstale":"2024-05-08T05:45:26.386252+0000","last_undegraded":"2024-05-08T05:45:26.386252+0000","last_fullsized":"2024-05-08T05:45:26.386252+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:25:22.586829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.848055+0000","last_change":"2024-05-08T05:45:24.838178+0000","last_active":"2024-05-08T05:45:24.848055+0000","last_peered":"2024-05-08T05:45:24.848055+0000","last_clean":"2024-05-08T05:45:24.848055+0000","last_became_active":"2024-05-08T05:45:24.837886+0000","last_became_peered":"2024-05-08T05:45:24.837886+0000","last_unstale":"2024-05-08T05:45:24.848055+0000","last_undegraded":"2024-05-08T05:45:24.848055+0000","last_fullsized":"2024-05-08T05:45:24.848055+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:18:08.297674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"74'8","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.384811+0000","last_change":"2024-05-08T05:45:23.839351+0000","last_active":"2024-05-08T05:45:26.384811+0000","last_peered":"2024-05-08T05:45:26.384811+0000","last_clean":"2024-05-08T05:45:26.384811+0000","last_became_active":"2024-05-08T05:45:23.839050+0000","last_became_peered":"2024-05-08T05:45:23.839050+0000","last_unstale":"2024-05-08T05:45:26.384811+0000","last_undegraded":"2024-05-08T05:45:26.384811+0000","last_fullsized":"2024-05-08T05:45:26.384811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:58:08.284451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:25.839181+0000","last_change":"2024-05-08T05:45:24.850980+0000","last_active":"2024-05-08T05:45:25.839181+0000","last_peered":"2024-05-08T05:45:25.839181+0000","last_clean":"2024-05-08T05:45:25.839181+0000","last_became_active":"2024-05-08T05:45:24.850711+0000","last_became_peered":"2024-05-08T05:45:24.850711+0000","last_unstale":"2024-05-08T05:45:25.839181+0000","last_undegraded":"2024-05-08T05:45:25.839181+0000","last_fullsized":"2024-05-08T05:45:25.839181+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:35.776683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"74'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.849740+0000","last_change":"2024-05-08T05:45:23.834931+0000","last_active":"2024-05-08T05:45:24.849740+0000","last_peered":"2024-05-08T05:45:24.849740+0000","last_clean":"2024-05-08T05:45:24.849740+0000","last_became_active":"2024-05-08T05:45:23.834636+0000","last_became_peered":"2024-05-08T05:45:23.834636+0000","last_unstale":"2024-05-08T05:45:24.849740+0000","last_undegraded":"2024-05-08T05:45:24.849740+0000","last_fullsized":"2024-05-08T05:45:24.849740+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:55:10.280907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385852+0000","last_change":"2024-05-08T05:45:24.851580+0000","last_active":"2024-05-08T05:45:26.385852+0000","last_peered":"2024-05-08T05:45:26.385852+0000","last_clean":"2024-05-08T05:45:26.385852+0000","last_became_active":"2024-05-08T05:45:24.851313+0000","last_became_peered":"2024-05-08T05:45:24.851313+0000","last_unstale":"2024-05-08T05:45:26.385852+0000","last_undegraded":"2024-05-08T05:45:26.385852+0000","last_fullsized":"2024-05-08T05:45:26.385852+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:15.207743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385794+0000","last_change":"2024-05-08T05:45:23.832031+0000","last_active":"2024-05-08T05:45:26.385794+0000","last_peered":"2024-05-08T05:45:26.385794+0000","last_clean":"2024-05-08T05:45:26.385794+0000","last_became_active":"2024-05-08T05:45:23.831735+0000","last_became_peered":"2024-05-08T05:45:23.831735+0000","last_unstale":"2024-05-08T05:45:26.385794+0000","last_undegraded":"2024-05-08T05:45:26.385794+0000","last_fullsized":"2024-05-08T05:45:26.385794+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:46:04.850092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385720+0000","last_change":"2024-05-08T05:45:24.845513+0000","last_active":"2024-05-08T05:45:26.385720+0000","last_peered":"2024-05-08T05:45:26.385720+0000","last_clean":"2024-05-08T05:45:26.385720+0000","last_became_active":"2024-05-08T05:45:24.845224+0000","last_became_peered":"2024-05-08T05:45:24.845224+0000","last_unstale":"2024-05-08T05:45:26.385720+0000","last_undegraded":"2024-05-08T05:45:26.385720+0000","last_fullsized":"2024-05-08T05:45:26.385720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:09:34.936315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"74'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.386509+0000","last_change":"2024-05-08T05:45:23.836336+0000","last_active":"2024-05-08T05:45:26.386509+0000","last_peered":"2024-05-08T05:45:26.386509+0000","last_clean":"2024-05-08T05:45:26.386509+0000","last_became_active":"2024-05-08T05:45:23.835736+0000","last_became_peered":"2024-05-08T05:45:23.835736+0000","last_unstale":"2024-05-08T05:45:26.386509+0000","last_undegraded":"2024-05-08T05:45:26.386509+0000","last_fullsized":"2024-05-08T05:45:26.386509+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:24:53.321686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:25.839381+0000","last_change":"2024-05-08T05:45:24.851636+0000","last_active":"2024-05-08T05:45:25.839381+0000","last_peered":"2024-05-08T05:45:25.839381+0000","last_clean":"2024-05-08T05:45:25.839381+0000","last_became_active":"2024-05-08T05:45:24.851318+0000","last_became_peered":"2024-05-08T05:45:24.851318+0000","last_unstale":"2024-05-08T05:45:25.839381+0000","last_undegraded":"2024-05-08T05:45:25.839381+0000","last_fullsized":"2024-05-08T05:45:25.839381+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:50:19.682079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385066+0000","last_change":"2024-05-08T05:45:23.829957+0000","last_active":"2024-05-08T05:45:26.385066+0000","last_peered":"2024-05-08T05:45:26.385066+0000","last_clean":"2024-05-08T05:45:26.385066+0000","last_became_active":"2024-05-08T05:45:23.829634+0000","last_became_peered":"2024-05-08T05:45:23.829634+0000","last_unstale":"2024-05-08T05:45:26.385066+0000","last_undegraded":"2024-05-08T05:45:26.385066+0000","last_fullsized":"2024-05-08T05:45:26.385066+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:06.376863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385540+0000","last_change":"2024-05-08T05:45:24.843608+0000","last_active":"2024-05-08T05:45:26.385540+0000","last_peered":"2024-05-08T05:45:26.385540+0000","last_clean":"2024-05-08T05:45:26.385540+0000","last_became_active":"2024-05-08T05:45:24.843106+0000","last_became_peered":"2024-05-08T05:45:24.843106+0000","last_unstale":"2024-05-08T05:45:26.385540+0000","last_undegraded":"2024-05-08T05:45:26.385540+0000","last_fullsized":"2024-05-08T05:45:26.385540+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:25:03.727013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"74'2","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:25.839471+0000","last_change":"2024-05-08T05:45:23.832221+0000","last_active":"2024-05-08T05:45:25.839471+0000","last_peered":"2024-05-08T05:45:25.839471+0000","last_clean":"2024-05-08T05:45:25.839471+0000","last_became_active":"2024-05-08T05:45:23.831864+0000","last_became_peered":"2024-05-08T05:45:23.831864+0000","last_unstale":"2024-05-08T05:45:25.839471+0000","last_undegraded":"2024-05-08T05:45:25.839471+0000","last_fullsized":"2024-05-08T05:45:25.839471+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:27.462868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385469+0000","last_change":"2024-05-08T05:45:24.847030+0000","last_active":"2024-05-08T05:45:26.385469+0000","last_peered":"2024-05-08T05:45:26.385469+0000","last_clean":"2024-05-08T05:45:26.385469+0000","last_became_active":"2024-05-08T05:45:24.846491+0000","last_became_peered":"2024-05-08T05:45:24.846491+0000","last_unstale":"2024-05-08T05:45:26.385469+0000","last_undegraded":"2024-05-08T05:45:26.385469+0000","last_fullsized":"2024-05-08T05:45:26.385469+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:34:33.124488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:25.839223+0000","last_change":"2024-05-08T05:45:23.837086+0000","last_active":"2024-05-08T05:45:25.839223+0000","last_peered":"2024-05-08T05:45:25.839223+0000","last_clean":"2024-05-08T05:45:25.839223+0000","last_became_active":"2024-05-08T05:45:23.836777+0000","last_became_peered":"2024-05-08T05:45:23.836777+0000","last_unstale":"2024-05-08T05:45:25.839223+0000","last_undegraded":"2024-05-08T05:45:25.839223+0000","last_fullsized":"2024-05-08T05:45:25.839223+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:49.025014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.852022+0000","last_change":"2024-05-08T05:45:24.845436+0000","last_active":"2024-05-08T05:45:24.852022+0000","last_peered":"2024-05-08T05:45:24.852022+0000","last_clean":"2024-05-08T05:45:24.852022+0000","last_became_active":"2024-05-08T05:45:24.845164+0000","last_became_peered":"2024-05-08T05:45:24.845164+0000","last_unstale":"2024-05-08T05:45:24.852022+0000","last_undegraded":"2024-05-08T05:45:24.852022+0000","last_fullsized":"2024-05-08T05:45:24.852022+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:27.942203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:25.839255+0000","last_change":"2024-05-08T05:45:23.836394+0000","last_active":"2024-05-08T05:45:25.839255+0000","last_peered":"2024-05-08T05:45:25.839255+0000","last_clean":"2024-05-08T05:45:25.839255+0000","last_became_active":"2024-05-08T05:45:23.836107+0000","last_became_peered":"2024-05-08T05:45:23.836107+0000","last_unstale":"2024-05-08T05:45:25.839255+0000","last_undegraded":"2024-05-08T05:45:25.839255+0000","last_fullsized":"2024-05-08T05:45:25.839255+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:55:10.280907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385433+0000","last_change":"2024-05-08T05:45:24.853771+0000","last_active":"2024-05-08T05:45:26.385433+0000","last_peered":"2024-05-08T05:45:26.385433+0000","last_clean":"2024-05-08T05:45:26.385433+0000","last_became_active":"2024-05-08T05:45:24.853503+0000","last_became_peered":"2024-05-08T05:45:24.853503+0000","last_unstale":"2024-05-08T05:45:26.385433+0000","last_undegraded":"2024-05-08T05:45:26.385433+0000","last_fullsized":"2024-05-08T05:45:26.385433+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:41.477874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.847440+0000","last_change":"2024-05-08T05:45:23.825726+0000","last_active":"2024-05-08T05:45:24.847440+0000","last_peered":"2024-05-08T05:45:24.847440+0000","last_clean":"2024-05-08T05:45:24.847440+0000","last_became_active":"2024-05-08T05:45:23.824860+0000","last_became_peered":"2024-05-08T05:45:23.824860+0000","last_unstale":"2024-05-08T05:45:24.847440+0000","last_undegraded":"2024-05-08T05:45:24.847440+0000","last_fullsized":"2024-05-08T05:45:24.847440+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:09:59.746816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.849548+0000","last_change":"2024-05-08T05:45:24.841858+0000","last_active":"2024-05-08T05:45:24.849548+0000","last_peered":"2024-05-08T05:45:24.849548+0000","last_clean":"2024-05-08T05:45:24.849548+0000","last_became_active":"2024-05-08T05:45:24.841555+0000","last_became_peered":"2024-05-08T05:45:24.841555+0000","last_unstale":"2024-05-08T05:45:24.849548+0000","last_undegraded":"2024-05-08T05:45:24.849548+0000","last_fullsized":"2024-05-08T05:45:24.849548+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:29:34.946299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385609+0000","last_change":"2024-05-08T05:45:23.834505+0000","last_active":"2024-05-08T05:45:26.385609+0000","last_peered":"2024-05-08T05:45:26.385609+0000","last_clean":"2024-05-08T05:45:26.385609+0000","last_became_active":"2024-05-08T05:45:23.834236+0000","last_became_peered":"2024-05-08T05:45:23.834236+0000","last_unstale":"2024-05-08T05:45:26.385609+0000","last_undegraded":"2024-05-08T05:45:26.385609+0000","last_fullsized":"2024-05-08T05:45:26.385609+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:32:05.494710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.386311+0000","last_change":"2024-05-08T05:45:24.846988+0000","last_active":"2024-05-08T05:45:26.386311+0000","last_peered":"2024-05-08T05:45:26.386311+0000","last_clean":"2024-05-08T05:45:26.386311+0000","last_became_active":"2024-05-08T05:45:24.846688+0000","last_became_peered":"2024-05-08T05:45:24.846688+0000","last_unstale":"2024-05-08T05:45:26.386311+0000","last_undegraded":"2024-05-08T05:45:26.386311+0000","last_fullsized":"2024-05-08T05:45:26.386311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:57:33.337722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.834347+0000","last_change":"2024-05-08T05:45:23.832813+0000","last_active":"2024-05-08T05:45:24.834347+0000","last_peered":"2024-05-08T05:45:24.834347+0000","last_clean":"2024-05-08T05:45:24.834347+0000","last_became_active":"2024-05-08T05:45:23.832504+0000","last_became_peered":"2024-05-08T05:45:23.832504+0000","last_unstale":"2024-05-08T05:45:24.834347+0000","last_undegraded":"2024-05-08T05:45:24.834347+0000","last_fullsized":"2024-05-08T05:45:24.834347+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:27.462868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385398+0000","last_change":"2024-05-08T05:45:24.836742+0000","last_active":"2024-05-08T05:45:26.385398+0000","last_peered":"2024-05-08T05:45:26.385398+0000","last_clean":"2024-05-08T05:45:26.385398+0000","last_became_active":"2024-05-08T05:45:24.835991+0000","last_became_peered":"2024-05-08T05:45:24.835991+0000","last_unstale":"2024-05-08T05:45:26.385398+0000","last_undegraded":"2024-05-08T05:45:26.385398+0000","last_fullsized":"2024-05-08T05:45:26.385398+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:03:32.629588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.833309+0000","last_change":"2024-05-08T05:45:23.836946+0000","last_active":"2024-05-08T05:45:24.833309+0000","last_peered":"2024-05-08T05:45:24.833309+0000","last_clean":"2024-05-08T05:45:24.833309+0000","last_became_active":"2024-05-08T05:45:23.836645+0000","last_became_peered":"2024-05-08T05:45:23.836645+0000","last_unstale":"2024-05-08T05:45:24.833309+0000","last_undegraded":"2024-05-08T05:45:24.833309+0000","last_fullsized":"2024-05-08T05:45:24.833309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:51.671640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.849661+0000","last_change":"2024-05-08T05:45:24.840500+0000","last_active":"2024-05-08T05:45:24.849661+0000","last_peered":"2024-05-08T05:45:24.849661+0000","last_clean":"2024-05-08T05:45:24.849661+0000","last_became_active":"2024-05-08T05:45:24.840216+0000","last_became_peered":"2024-05-08T05:45:24.840216+0000","last_unstale":"2024-05-08T05:45:24.849661+0000","last_undegraded":"2024-05-08T05:45:24.849661+0000","last_fullsized":"2024-05-08T05:45:24.849661+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:59:13.415098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.832420+0000","last_change":"2024-05-08T05:45:23.838025+0000","last_active":"2024-05-08T05:45:24.832420+0000","last_peered":"2024-05-08T05:45:24.832420+0000","last_clean":"2024-05-08T05:45:24.832420+0000","last_became_active":"2024-05-08T05:45:23.837700+0000","last_became_peered":"2024-05-08T05:45:23.837700+0000","last_unstale":"2024-05-08T05:45:24.832420+0000","last_undegraded":"2024-05-08T05:45:24.832420+0000","last_fullsized":"2024-05-08T05:45:24.832420+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:06.376863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.854040+0000","last_change":"2024-05-08T05:45:24.851900+0000","last_active":"2024-05-08T05:45:24.854040+0000","last_peered":"2024-05-08T05:45:24.854040+0000","last_clean":"2024-05-08T05:45:24.854040+0000","last_became_active":"2024-05-08T05:45:24.850686+0000","last_became_peered":"2024-05-08T05:45:24.850686+0000","last_unstale":"2024-05-08T05:45:24.854040+0000","last_undegraded":"2024-05-08T05:45:24.854040+0000","last_fullsized":"2024-05-08T05:45:24.854040+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:23.389303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385917+0000","last_change":"2024-05-08T05:45:23.834461+0000","last_active":"2024-05-08T05:45:26.385917+0000","last_peered":"2024-05-08T05:45:26.385917+0000","last_clean":"2024-05-08T05:45:26.385917+0000","last_became_active":"2024-05-08T05:45:23.834167+0000","last_became_peered":"2024-05-08T05:45:23.834167+0000","last_unstale":"2024-05-08T05:45:26.385917+0000","last_undegraded":"2024-05-08T05:45:26.385917+0000","last_fullsized":"2024-05-08T05:45:26.385917+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:55:21.042256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385490+0000","last_change":"2024-05-08T05:45:24.839479+0000","last_active":"2024-05-08T05:45:26.385490+0000","last_peered":"2024-05-08T05:45:26.385490+0000","last_clean":"2024-05-08T05:45:26.385490+0000","last_became_active":"2024-05-08T05:45:24.839209+0000","last_became_peered":"2024-05-08T05:45:24.839209+0000","last_unstale":"2024-05-08T05:45:26.385490+0000","last_undegraded":"2024-05-08T05:45:26.385490+0000","last_fullsized":"2024-05-08T05:45:26.385490+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:10:00.749110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.830294+0000","last_change":"2024-05-08T05:45:23.835556+0000","last_active":"2024-05-08T05:45:24.830294+0000","last_peered":"2024-05-08T05:45:24.830294+0000","last_clean":"2024-05-08T05:45:24.830294+0000","last_became_active":"2024-05-08T05:45:23.835249+0000","last_became_peered":"2024-05-08T05:45:23.835249+0000","last_unstale":"2024-05-08T05:45:24.830294+0000","last_undegraded":"2024-05-08T05:45:24.830294+0000","last_fullsized":"2024-05-08T05:45:24.830294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:45:58.599863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.385362+0000","last_change":"2024-05-08T05:45:24.838107+0000","last_active":"2024-05-08T05:45:26.385362+0000","last_peered":"2024-05-08T05:45:26.385362+0000","last_clean":"2024-05-08T05:45:26.385362+0000","last_became_active":"2024-05-08T05:45:24.837792+0000","last_became_peered":"2024-05-08T05:45:24.837792+0000","last_unstale":"2024-05-08T05:45:26.385362+0000","last_undegraded":"2024-05-08T05:45:26.385362+0000","last_fullsized":"2024-05-08T05:45:26.385362+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:01.081498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:26.384810+0000","last_change":"2024-05-08T05:45:23.826960+0000","last_active":"2024-05-08T05:45:26.384810+0000","last_peered":"2024-05-08T05:45:26.384810+0000","last_clean":"2024-05-08T05:45:26.384810+0000","last_became_active":"2024-05-08T05:45:23.826022+0000","last_became_peered":"2024-05-08T05:45:23.826022+0000","last_unstale":"2024-05-08T05:45:26.384810+0000","last_undegraded":"2024-05-08T05:45:26.384810+0000","last_fullsized":"2024-05-08T05:45:26.384810+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:39:02.644530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.847011+0000","last_change":"2024-05-08T05:45:24.840526+0000","last_active":"2024-05-08T05:45:24.847011+0000","last_peered":"2024-05-08T05:45:24.847011+0000","last_clean":"2024-05-08T05:45:24.847011+0000","last_became_active":"2024-05-08T05:45:24.840230+0000","last_became_peered":"2024-05-08T05:45:24.840230+0000","last_unstale":"2024-05-08T05:45:24.847011+0000","last_undegraded":"2024-05-08T05:45:24.847011+0000","last_fullsized":"2024-05-08T05:45:24.847011+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:23.810929+0000","last_clean_scrub_stamp":"2024-05-08T05:45:23.810929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:51:34.514053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T05:45:25.839775+0000","last_change":"2024-05-08T05:45:23.840044+0000","last_active":"2024-05-08T05:45:25.839775+0000","last_peered":"2024-05-08T05:45:25.839775+0000","last_clean":"2024-05-08T05:45:25.839775+0000","last_became_active":"2024-05-08T05:45:23.839699+0000","last_became_peered":"2024-05-08T05:45:23.839699+0000","last_unstale":"2024-05-08T05:45:25.839775+0000","last_undegraded":"2024-05-08T05:45:25.839775+0000","last_fullsized":"2024-05-08T05:45:25.839775+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:22.808635+0000","last_clean_scrub_stamp":"2024-05-08T05:45:22.808635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:43:36.989641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":209,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-08T05:45:24.831426+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:45:24.831426+0000","last_peered":"2024-05-08T05:45:24.831426+0000","last_clean":"2024-05-08T05:45:24.831426+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:45:24.831426+0000","last_undegraded":"2024-05-08T05:45:24.831426+0000","last_fullsized":"2024-05-08T05:45:24.831426+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542197,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28472,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343368,"statfs":{"total":96636764160,"available":96607608832,"internally_reserved":0,"allocated":1482752,"data_stored":991271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542198,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":888832,"data_stored":400903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542197,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":888832,"data_stored":400903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":933888,"data_stored":416478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":917504,"data_stored":415344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":925696,"data_stored":415504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":921600,"data_stored":414874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1507328,"data_stored":1005130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:45:29.256 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T05:45:29.256 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T05:45:29.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.403+0000 7fa56a41c640 1 Processor -- start 2024-05-08T05:45:29.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.403+0000 7fa56a41c640 1 -- start start 2024-05-08T05:45:29.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.404+0000 7fa56a41c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa56415bc80 0x7fa56406e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.404+0000 7fa56a41c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5640b4590 con 0x7fa56406e8d0 2024-05-08T05:45:29.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.404+0000 7fa56a41c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5640b4870 con 0x7fa5641524f0 2024-05-08T05:45:29.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.404+0000 7fa56a41c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa5640b51a0 con 0x7fa56415bc80 2024-05-08T05:45:29.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.404+0000 7fa5689b9640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa56406e8d0 0x7fa564073700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34690/0 (socket says 172.21.15.45:34690) 2024-05-08T05:45:29.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.404+0000 7fa5689b9640 1 -- 172.21.15.45:0/1284013326 learned_addr learned my addr 172.21.15.45:0/1284013326 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:29.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.404+0000 7fa55bfff640 1 --2- 172.21.15.45:0/1284013326 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa56415bc80 0x7fa56406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:29.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.405+0000 7fa5637fe640 1 -- 172.21.15.45:0/1284013326 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3912099962 0 0) 0x7fa5640b4590 con 0x7fa56406e8d0 2024-05-08T05:45:29.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.405+0000 7fa5637fe640 1 -- 172.21.15.45:0/1284013326 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa550003520 con 0x7fa56406e8d0 2024-05-08T05:45:29.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.405+0000 7fa5637fe640 1 -- 172.21.15.45:0/1284013326 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3694662450 0 0) 0x7fa5640b4870 con 0x7fa5641524f0 2024-05-08T05:45:29.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.405+0000 7fa5637fe640 1 -- 172.21.15.45:0/1284013326 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5640b4590 con 0x7fa5641524f0 2024-05-08T05:45:29.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.405+0000 7fa55bfff640 1 -- 172.21.15.45:0/1284013326 >> v1:172.21.15.63:6789/0 conn(0x7fa5641524f0 legacy=0x7fa5640b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:29.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.405+0000 7fa55bfff640 1 -- 172.21.15.45:0/1284013326 >> v1:172.21.15.45:6789/0 conn(0x7fa56406e8d0 legacy=0x7fa564073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:29.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.405+0000 7fa55bfff640 1 -- 172.21.15.45:0/1284013326 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5640b4b50 con 0x7fa56415bc80 2024-05-08T05:45:29.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.406+0000 7fa55bfff640 1 --2- 172.21.15.45:0/1284013326 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa56415bc80 0x7fa56406e390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00a970 tx=0x7fa54c02b5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1628d1754b2e480 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:29.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.406+0000 7fa5637fe640 1 -- 172.21.15.45:0/1284013326 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c00b040 con 0x7fa56415bc80 2024-05-08T05:45:29.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.406+0000 7fa5637fe640 1 -- 172.21.15.45:0/1284013326 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa54c032600 con 0x7fa56415bc80 2024-05-08T05:45:29.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.406+0000 7fa5637fe640 1 -- 172.21.15.45:0/1284013326 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c0329b0 con 0x7fa56415bc80 2024-05-08T05:45:29.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.406+0000 7fa56a41c640 1 -- 172.21.15.45:0/1284013326 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa56415bc80 msgr2=0x7fa56406e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:29.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.406+0000 7fa56a41c640 1 --2- 172.21.15.45:0/1284013326 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa56415bc80 0x7fa56406e390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00a970 tx=0x7fa54c02b5d0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.406+0000 7fa56a41c640 1 -- 172.21.15.45:0/1284013326 shutdown_connections 2024-05-08T05:45:29.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.406+0000 7fa56a41c640 1 --2- 172.21.15.45:0/1284013326 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa56415bc80 0x7fa56406e390 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.407+0000 7fa56a41c640 1 -- 172.21.15.45:0/1284013326 >> 172.21.15.45:0/1284013326 conn(0x7fa56406c3a0 msgr2=0x7fa56406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:29.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.407+0000 7fa56a41c640 1 -- 172.21.15.45:0/1284013326 shutdown_connections 2024-05-08T05:45:29.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.407+0000 7fa56a41c640 1 -- 172.21.15.45:0/1284013326 wait complete. 2024-05-08T05:45:29.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.408+0000 7fa56a41c640 1 Processor -- start 2024-05-08T05:45:29.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.408+0000 7fa56a41c640 1 -- start start 2024-05-08T05:45:29.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.409+0000 7fa56a41c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa56406e8d0 0x7fa56414a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.409+0000 7fa56a41c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5641524f0 0x7fa5641f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.409+0000 7fa56a41c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa56415bc80 0x7fa5640cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.409+0000 7fa56a41c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa5640b4330 con 0x7fa5641524f0 2024-05-08T05:45:29.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.409+0000 7fa56a41c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa564073450 con 0x7fa56406e8d0 2024-05-08T05:45:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.409+0000 7fa56a41c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa5641528d0 con 0x7fa56415bc80 2024-05-08T05:45:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.409+0000 7fa55bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5641524f0 0x7fa5641f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.409+0000 7fa55bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5641524f0 0x7fa5641f5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56566/0 (socket says 172.21.15.45:56566) 2024-05-08T05:45:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.409+0000 7fa55bfff640 1 -- 172.21.15.45:0/251082994 learned_addr learned my addr 172.21.15.45:0/251082994 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.409+0000 7fa563fff640 1 --2- 172.21.15.45:0/251082994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa56406e8d0 0x7fa56414a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.409+0000 7fa5689b9640 1 --2- 172.21.15.45:0/251082994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa56415bc80 0x7fa5640cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.409+0000 7fa55bfff640 1 -- 172.21.15.45:0/251082994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa56415bc80 msgr2=0x7fa5640cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.409+0000 7fa55bfff640 1 --2- 172.21.15.45:0/251082994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa56415bc80 0x7fa5640cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.409+0000 7fa55bfff640 1 -- 172.21.15.45:0/251082994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa56406e8d0 msgr2=0x7fa56414a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.409+0000 7fa55bfff640 1 --2- 172.21.15.45:0/251082994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa56406e8d0 0x7fa56414a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.409+0000 7fa55bfff640 1 -- 172.21.15.45:0/251082994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5641f6190 con 0x7fa5641524f0 2024-05-08T05:45:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.409+0000 7fa55bfff640 1 --2- 172.21.15.45:0/251082994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5641524f0 0x7fa5641f5c50 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00aa90 tx=0x7fa54c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.409+0000 7fa5617fa640 1 -- 172.21.15.45:0/251082994 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c040070 con 0x7fa5641524f0 2024-05-08T05:45:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.409+0000 7fa5617fa640 1 -- 172.21.15.45:0/251082994 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa54c004240 con 0x7fa5641524f0 2024-05-08T05:45:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.409+0000 7fa5617fa640 1 -- 172.21.15.45:0/251082994 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c006e10 con 0x7fa5641524f0 2024-05-08T05:45:29.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.410+0000 7fa56a41c640 1 -- 172.21.15.45:0/251082994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5640cd540 con 0x7fa5641524f0 2024-05-08T05:45:29.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.410+0000 7fa56a41c640 1 -- 172.21.15.45:0/251082994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5641f6430 con 0x7fa5641524f0 2024-05-08T05:45:29.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.411+0000 7fa56a41c640 1 -- 172.21.15.45:0/251082994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa56406ccb0 con 0x7fa5641524f0 2024-05-08T05:45:29.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.411+0000 7fa5617fa640 1 -- 172.21.15.45:0/251082994 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa54c0317d0 con 0x7fa5641524f0 2024-05-08T05:45:29.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.412+0000 7fa5617fa640 1 --2- 172.21.15.45:0/251082994 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa544076300 0x7fa5440787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.412+0000 7fa5617fa640 1 -- 172.21.15.45:0/251082994 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fa54c0b7270 con 0x7fa5641524f0 2024-05-08T05:45:29.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.412+0000 7fa563fff640 1 --2- 172.21.15.45:0/251082994 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa544076300 0x7fa5440787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:29.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.413+0000 7fa563fff640 1 --2- 172.21.15.45:0/251082994 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa544076300 0x7fa5440787c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa554001810 tx=0x7fa554009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:29.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.416+0000 7fa5617fa640 1 -- 172.21.15.45:0/251082994 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa56406ccb0 con 0x7fa5641524f0 2024-05-08T05:45:29.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.683+0000 7fa56a41c640 1 -- 172.21.15.45:0/251082994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa5640693f0 con 0x7fa5641524f0 2024-05-08T05:45:29.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.686+0000 7fa5617fa640 1 -- 172.21.15.45:0/251082994 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7fa5641f6190 con 0x7fa5641524f0 2024-05-08T05:45:29.686 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-05-08T05:45:29.686 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBUEDtmDcqGMxAALgQaBYp4Zd8uMqswCXvGaw== 2024-05-08T05:45:29.686 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2024-05-08T05:45:29.686 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2024-05-08T05:45:29.686 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-08T05:45:29.687 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2024-05-08T05:45:29.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.689+0000 7fa56a41c640 1 -- 172.21.15.45:0/251082994 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa544076300 msgr2=0x7fa5440787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:29.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.689+0000 7fa56a41c640 1 --2- 172.21.15.45:0/251082994 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa544076300 0x7fa5440787c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa554001810 tx=0x7fa554009040 comp rx=0 tx=0).stop 2024-05-08T05:45:29.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.689+0000 7fa56a41c640 1 -- 172.21.15.45:0/251082994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5641524f0 msgr2=0x7fa5641f5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:29.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.689+0000 7fa56a41c640 1 --2- 172.21.15.45:0/251082994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5641524f0 0x7fa5641f5c50 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00aa90 tx=0x7fa54c004730 comp rx=0 tx=0).stop 2024-05-08T05:45:29.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.689+0000 7fa56a41c640 1 -- 172.21.15.45:0/251082994 shutdown_connections 2024-05-08T05:45:29.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.689+0000 7fa56a41c640 1 --2- 172.21.15.45:0/251082994 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa544076300 0x7fa5440787c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.689+0000 7fa56a41c640 1 --2- 172.21.15.45:0/251082994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa56415bc80 0x7fa5640cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.689+0000 7fa56a41c640 1 --2- 172.21.15.45:0/251082994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5641524f0 0x7fa5641f5c50 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.689+0000 7fa56a41c640 1 --2- 172.21.15.45:0/251082994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa56406e8d0 0x7fa56414a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.689+0000 7fa56a41c640 1 -- 172.21.15.45:0/251082994 >> 172.21.15.45:0/251082994 conn(0x7fa56406c3a0 msgr2=0x7fa564072440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:29.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.689+0000 7fa56a41c640 1 -- 172.21.15.45:0/251082994 shutdown_connections 2024-05-08T05:45:29.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.689+0000 7fa56a41c640 1 -- 172.21.15.45:0/251082994 wait complete. 2024-05-08T05:45:29.700 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:45:29.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.803+0000 7f4047351640 1 Processor -- start 2024-05-08T05:45:29.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.803+0000 7f4047351640 1 -- start start 2024-05-08T05:45:29.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.804+0000 7f4047351640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f404015f0c0 0x7f40400b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.804+0000 7f4047351640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f40401638e0 0x7f40400b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.804+0000 7f4047351640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40400b1260 con 0x7f404006c3a0 2024-05-08T05:45:29.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.804+0000 7f4047351640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4040163450 con 0x7f40401638e0 2024-05-08T05:45:29.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.804+0000 7f4047351640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f40401604b0 con 0x7f404015f0c0 2024-05-08T05:45:29.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.804+0000 7f40448ec640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f40401638e0 0x7f40400b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:29.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.804+0000 7f40458ee640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f404006c3a0 0x7f4040073170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:34704/0 (socket says 172.21.15.45:34704) 2024-05-08T05:45:29.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.804+0000 7f40458ee640 1 -- 172.21.15.45:0/3619534900 learned_addr learned my addr 172.21.15.45:0/3619534900 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:29.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.804+0000 7f40450ed640 1 --2- 172.21.15.45:0/3619534900 >> v2:172.21.15.63:3300/0 conn(0x7f404015f0c0 0x7f40400b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:29.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.805+0000 7f40448ec640 1 -- 172.21.15.45:0/3619534900 >> v2:172.21.15.63:3300/0 conn(0x7f404015f0c0 msgr2=0x7f40400b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:29.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.805+0000 7f40448ec640 1 --2- 172.21.15.45:0/3619534900 >> v2:172.21.15.63:3300/0 conn(0x7f404015f0c0 0x7f40400b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.805+0000 7f40448ec640 1 -- 172.21.15.45:0/3619534900 >> v1:172.21.15.45:6789/0 conn(0x7f404006c3a0 legacy=0x7f4040073170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:29.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.805+0000 7f4037fff640 1 -- 172.21.15.45:0/3619534900 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3947088050 0 0) 0x7f4038001490 con 0x7f404006c3a0 2024-05-08T05:45:29.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.805+0000 7f40448ec640 1 -- 172.21.15.45:0/3619534900 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40400b14e0 con 0x7f40401638e0 2024-05-08T05:45:29.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.805+0000 7f40448ec640 1 --2- 172.21.15.45:0/3619534900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40401638e0 0x7f40400b0d20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4040071200 tx=0x7f403002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ec11da19607099 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:29.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.806+0000 7f4037fff640 1 -- 172.21.15.45:0/3619534900 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030037070 con 0x7f40401638e0 2024-05-08T05:45:29.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.806+0000 7f4037fff640 1 -- 172.21.15.45:0/3619534900 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4030032440 con 0x7f40401638e0 2024-05-08T05:45:29.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.806+0000 7f4037fff640 1 -- 172.21.15.45:0/3619534900 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030032810 con 0x7f40401638e0 2024-05-08T05:45:29.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.806+0000 7f4047351640 1 -- 172.21.15.45:0/3619534900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40401638e0 msgr2=0x7f40400b0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:29.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.806+0000 7f4047351640 1 --2- 172.21.15.45:0/3619534900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40401638e0 0x7f40400b0d20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4040071200 tx=0x7f403002a1a0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.806+0000 7f4047351640 1 -- 172.21.15.45:0/3619534900 shutdown_connections 2024-05-08T05:45:29.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.806+0000 7f4047351640 1 --2- 172.21.15.45:0/3619534900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40401638e0 0x7f40400b0d20 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.806+0000 7f4047351640 1 --2- 172.21.15.45:0/3619534900 >> v2:172.21.15.63:3300/0 conn(0x7f404015f0c0 0x7f40400b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.806+0000 7f4047351640 1 -- 172.21.15.45:0/3619534900 >> 172.21.15.45:0/3619534900 conn(0x7f40400b3b10 msgr2=0x7f40400b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:29.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.806+0000 7f4047351640 1 -- 172.21.15.45:0/3619534900 shutdown_connections 2024-05-08T05:45:29.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.806+0000 7f4047351640 1 -- 172.21.15.45:0/3619534900 wait complete. 2024-05-08T05:45:29.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.807+0000 7f4047351640 1 Processor -- start 2024-05-08T05:45:29.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.807+0000 7f4047351640 1 -- start start 2024-05-08T05:45:29.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.808+0000 7f4047351640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f404006c3a0 0x7f40401499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.808+0000 7f4047351640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f404015f0c0 0x7f4040154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.808+0000 7f4047351640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40401638e0 0x7f40400d2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.808+0000 7f4047351640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f40400b17c0 con 0x7f404006c3a0 2024-05-08T05:45:29.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.808+0000 7f4047351640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f40401630e0 con 0x7f40401638e0 2024-05-08T05:45:29.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.808+0000 7f4047351640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f404006c7e0 con 0x7f404015f0c0 2024-05-08T05:45:29.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.808+0000 7f40450ed640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f404006c3a0 0x7f40401499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:29.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.808+0000 7f40450ed640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f404006c3a0 0x7f40401499e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56576/0 (socket says 172.21.15.45:56576) 2024-05-08T05:45:29.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.808+0000 7f40450ed640 1 -- 172.21.15.45:0/1700013483 learned_addr learned my addr 172.21.15.45:0/1700013483 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:29.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.809+0000 7f40458ee640 1 --2- 172.21.15.45:0/1700013483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40401638e0 0x7f40400d2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:29.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.809+0000 7f40450ed640 1 -- 172.21.15.45:0/1700013483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f404015f0c0 msgr2=0x7f4040154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:29.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.809+0000 7f40450ed640 1 --2- 172.21.15.45:0/1700013483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f404015f0c0 0x7f4040154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.809+0000 7f40450ed640 1 -- 172.21.15.45:0/1700013483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40401638e0 msgr2=0x7f40400d2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:29.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.809+0000 7f40450ed640 1 --2- 172.21.15.45:0/1700013483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40401638e0 0x7f40400d2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.809+0000 7f40450ed640 1 -- 172.21.15.45:0/1700013483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4040154c60 con 0x7f404006c3a0 2024-05-08T05:45:29.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.809+0000 7f40450ed640 1 --2- 172.21.15.45:0/1700013483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f404006c3a0 0x7f40401499e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f403c00e960 tx=0x7f403c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:29.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.809+0000 7f4035ffb640 1 -- 172.21.15.45:0/1700013483 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c017070 con 0x7f404006c3a0 2024-05-08T05:45:29.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.809+0000 7f4035ffb640 1 -- 172.21.15.45:0/1700013483 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f403c00ca00 con 0x7f404006c3a0 2024-05-08T05:45:29.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.809+0000 7f4047351640 1 -- 172.21.15.45:0/1700013483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4040154f40 con 0x7f404006c3a0 2024-05-08T05:45:29.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.809+0000 7f4035ffb640 1 -- 172.21.15.45:0/1700013483 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c00cdf0 con 0x7f404006c3a0 2024-05-08T05:45:29.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.809+0000 7f4047351640 1 -- 172.21.15.45:0/1700013483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4040155500 con 0x7f404006c3a0 2024-05-08T05:45:29.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.810+0000 7f4047351640 1 -- 172.21.15.45:0/1700013483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40400693f0 con 0x7f404006c3a0 2024-05-08T05:45:29.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.811+0000 7f4035ffb640 1 -- 172.21.15.45:0/1700013483 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f403c01a070 con 0x7f404006c3a0 2024-05-08T05:45:29.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.811+0000 7f4035ffb640 1 --2- 172.21.15.45:0/1700013483 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f40240762d0 0x7f4024078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.811+0000 7f4035ffb640 1 -- 172.21.15.45:0/1700013483 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f403c097bf0 con 0x7f404006c3a0 2024-05-08T05:45:29.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.812+0000 7f40448ec640 1 --2- 172.21.15.45:0/1700013483 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f40240762d0 0x7f4024078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:29.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.812+0000 7f40448ec640 1 --2- 172.21.15.45:0/1700013483 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f40240762d0 0x7f4024078790 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4030034000 tx=0x7f403002c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:29.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:29.815+0000 7f4035ffb640 1 -- 172.21.15.45:0/1700013483 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f403c061330 con 0x7f404006c3a0 2024-05-08T05:45:30.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.049+0000 7f4047351640 1 -- 172.21.15.45:0/1700013483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f404015dec0 con 0x7f404006c3a0 2024-05-08T05:45:30.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.049+0000 7f4035ffb640 1 -- 172.21.15.45:0/1700013483 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5180 (secure 0 0 0) 0x7f403c060c30 con 0x7f404006c3a0 2024-05-08T05:45:30.050 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:30.050 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5722,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1283757699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1283757699},{"type":"v1","addr":"172.21.15.63:6837","nonce":1283757699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1963062796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1963062796},{"type":"v1","addr":"172.21.15.45:6834","nonce":1963062796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6171,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4183668301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4183668301},{"type":"v1","addr":"172.21.15.45:6835","nonce":4183668301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:45:24.208158+0000","modified":"2024-05-08T05:45:27.211503+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6180":{"gid":6180,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6836/133050464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":133050464},{"type":"v1","addr":"172.21.15.63:6836","nonce":133050464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6180,"qdb_cluster":[6180]},"id":5}]} 2024-05-08T05:45:30.050 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 44 2024-05-08T05:45:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.051+0000 7f4047351640 1 -- 172.21.15.45:0/1700013483 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f40240762d0 msgr2=0x7f4024078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.051+0000 7f4047351640 1 --2- 172.21.15.45:0/1700013483 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f40240762d0 0x7f4024078790 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4030034000 tx=0x7f403002c000 comp rx=0 tx=0).stop 2024-05-08T05:45:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.051+0000 7f4047351640 1 -- 172.21.15.45:0/1700013483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f404006c3a0 msgr2=0x7f40401499e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.051+0000 7f4047351640 1 --2- 172.21.15.45:0/1700013483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f404006c3a0 0x7f40401499e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f403c00e960 tx=0x7f403c00ee20 comp rx=0 tx=0).stop 2024-05-08T05:45:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.051+0000 7f4047351640 1 -- 172.21.15.45:0/1700013483 shutdown_connections 2024-05-08T05:45:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.051+0000 7f4047351640 1 --2- 172.21.15.45:0/1700013483 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f40240762d0 0x7f4024078790 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.051+0000 7f4047351640 1 --2- 172.21.15.45:0/1700013483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f404015f0c0 0x7f4040154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.051+0000 7f4047351640 1 --2- 172.21.15.45:0/1700013483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40401638e0 0x7f40400d2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.051+0000 7f4047351640 1 --2- 172.21.15.45:0/1700013483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f404006c3a0 0x7f40401499e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.051+0000 7f4047351640 1 -- 172.21.15.45:0/1700013483 >> 172.21.15.45:0/1700013483 conn(0x7f40400b3b10 msgr2=0x7f4040073170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.052+0000 7f4047351640 1 -- 172.21.15.45:0/1700013483 shutdown_connections 2024-05-08T05:45:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.052+0000 7f4047351640 1 -- 172.21.15.45:0/1700013483 wait complete. 2024-05-08T05:45:30.464 INFO:tasks.ceph.mds.b.smithi063.stderr:2024-05-08T05:45:30.464+0000 7f0e4868c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:45:30.465 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:45:24.208158+0000', 'modified': '2024-05-08T05:45:27.211503+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6180}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6180': {'gid': 6180, 'name': 'b', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.63:6836/133050464', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6834', 'nonce': 133050464}, {'type': 'v1', 'addr': '172.21.15.63:6836', 'nonce': 133050464}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6180, 'qdb_cluster': [6180]} 2024-05-08T05:45:30.465 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T05:45:30.465 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T05:45:30.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.572+0000 7f02dd522640 1 Processor -- start 2024-05-08T05:45:30.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.572+0000 7f02dd522640 1 -- start start 2024-05-08T05:45:30.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.573+0000 7f02dd522640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f02d815f450 0x7f02d806d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.573+0000 7f02dd522640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02d815f830 0x7f02d806d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.573+0000 7f02dd522640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f02d80d0700 0x7f02d806dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.573+0000 7f02dd522640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f02d806d690 con 0x7f02d80d0700 2024-05-08T05:45:30.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.573+0000 7f02dd522640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f02d80d0cb0 con 0x7f02d815f450 2024-05-08T05:45:30.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.573+0000 7f02dd522640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f02d80d0f90 con 0x7f02d815f830 2024-05-08T05:45:30.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.573+0000 7f02d77fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f02d80d0700 0x7f02d806dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.573+0000 7f02d77fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f02d80d0700 0x7f02d806dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56586/0 (socket says 172.21.15.45:56586) 2024-05-08T05:45:30.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.573+0000 7f02d77fe640 1 -- 172.21.15.45:0/2076968947 learned_addr learned my addr 172.21.15.45:0/2076968947 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:30.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.573+0000 7f02d6ffd640 1 --2- 172.21.15.45:0/2076968947 >> v2:172.21.15.63:3300/0 conn(0x7f02d815f450 0x7f02d806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.573+0000 7f02d67fc640 1 --2- 172.21.15.45:0/2076968947 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02d815f830 0x7f02d806d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.573+0000 7f02d77fe640 1 -- 172.21.15.45:0/2076968947 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02d815f830 msgr2=0x7f02d806d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:30.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.573+0000 7f02d77fe640 1 --2- 172.21.15.45:0/2076968947 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02d815f830 0x7f02d806d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.573+0000 7f02d77fe640 1 -- 172.21.15.45:0/2076968947 >> v2:172.21.15.63:3300/0 conn(0x7f02d815f450 msgr2=0x7f02d806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:30.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.573+0000 7f02d77fe640 1 --2- 172.21.15.45:0/2076968947 >> v2:172.21.15.63:3300/0 conn(0x7f02d815f450 0x7f02d806d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.573+0000 7f02d77fe640 1 -- 172.21.15.45:0/2076968947 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d80d2fe0 con 0x7f02d80d0700 2024-05-08T05:45:30.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.574+0000 7f02d77fe640 1 --2- 172.21.15.45:0/2076968947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02d80d0700 0x7f02d806dd70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f02d808d6d0 tx=0x7f02d002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f3667585ca68b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:30.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.574+0000 7f02d5ffb640 1 -- 172.21.15.45:0/2076968947 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0037070 con 0x7f02d80d0700 2024-05-08T05:45:30.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.574+0000 7f02d5ffb640 1 -- 172.21.15.45:0/2076968947 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d0032440 con 0x7f02d80d0700 2024-05-08T05:45:30.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.574+0000 7f02d5ffb640 1 -- 172.21.15.45:0/2076968947 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0032810 con 0x7f02d80d0700 2024-05-08T05:45:30.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.575+0000 7f02dd522640 1 -- 172.21.15.45:0/2076968947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02d80d0700 msgr2=0x7f02d806dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:30.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.575+0000 7f02dd522640 1 --2- 172.21.15.45:0/2076968947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02d80d0700 0x7f02d806dd70 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f02d808d6d0 tx=0x7f02d002a1e0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.575+0000 7f02dd522640 1 -- 172.21.15.45:0/2076968947 shutdown_connections 2024-05-08T05:45:30.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.575+0000 7f02dd522640 1 --2- 172.21.15.45:0/2076968947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02d80d0700 0x7f02d806dd70 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.575+0000 7f02dd522640 1 --2- 172.21.15.45:0/2076968947 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02d815f830 0x7f02d806d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.575+0000 7f02dd522640 1 --2- 172.21.15.45:0/2076968947 >> v2:172.21.15.63:3300/0 conn(0x7f02d815f450 0x7f02d806d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.575+0000 7f02dd522640 1 -- 172.21.15.45:0/2076968947 >> 172.21.15.45:0/2076968947 conn(0x7f02d80d5040 msgr2=0x7f02d80d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:30.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.575+0000 7f02dd522640 1 -- 172.21.15.45:0/2076968947 shutdown_connections 2024-05-08T05:45:30.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.575+0000 7f02dd522640 1 -- 172.21.15.45:0/2076968947 wait complete. 2024-05-08T05:45:30.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.576+0000 7f02dd522640 1 Processor -- start 2024-05-08T05:45:30.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.576+0000 7f02dd522640 1 -- start start 2024-05-08T05:45:30.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.577+0000 7f02dd522640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02d80d0700 0x7f02d815ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.577+0000 7f02dd522640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02d815f450 0x7f02d80b98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.577+0000 7f02dd522640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02d815f830 0x7f02d80c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.577+0000 7f02dd522640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f02d80d1960 con 0x7f02d815f830 2024-05-08T05:45:30.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.577+0000 7f02dd522640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f02d80d17e0 con 0x7f02d815f450 2024-05-08T05:45:30.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.577+0000 7f02dd522640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f02d80d1ae0 con 0x7f02d80d0700 2024-05-08T05:45:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.577+0000 7f02d77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02d815f830 0x7f02d80c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.577+0000 7f02d77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02d815f830 0x7f02d80c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56588/0 (socket says 172.21.15.45:56588) 2024-05-08T05:45:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.577+0000 7f02d77fe640 1 -- 172.21.15.45:0/3931958101 learned_addr learned my addr 172.21.15.45:0/3931958101 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.577+0000 7f02d67fc640 1 --2- 172.21.15.45:0/3931958101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02d815f450 0x7f02d80b98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.577+0000 7f02d6ffd640 1 --2- 172.21.15.45:0/3931958101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02d80d0700 0x7f02d815ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.577+0000 7f02d77fe640 1 -- 172.21.15.45:0/3931958101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02d80d0700 msgr2=0x7f02d815ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.577+0000 7f02d77fe640 1 --2- 172.21.15.45:0/3931958101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02d80d0700 0x7f02d815ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.577+0000 7f02d77fe640 1 -- 172.21.15.45:0/3931958101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02d815f450 msgr2=0x7f02d80b98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.577+0000 7f02d77fe640 1 --2- 172.21.15.45:0/3931958101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02d815f450 0x7f02d80b98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.577+0000 7f02d77fe640 1 -- 172.21.15.45:0/3931958101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d80c8ca0 con 0x7f02d815f830 2024-05-08T05:45:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.577+0000 7f02d67fc640 1 --2- 172.21.15.45:0/3931958101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02d815f450 0x7f02d80b98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.577+0000 7f02d6ffd640 1 --2- 172.21.15.45:0/3931958101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02d80d0700 0x7f02d815ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.578+0000 7f02d77fe640 1 --2- 172.21.15.45:0/3931958101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02d815f830 0x7f02d80c9d00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f02d002a220 tx=0x7f02d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.578+0000 7f02bf7fe640 1 -- 172.21.15.45:0/3931958101 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0037040 con 0x7f02d815f830 2024-05-08T05:45:30.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.578+0000 7f02bf7fe640 1 -- 172.21.15.45:0/3931958101 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d0042070 con 0x7f02d815f830 2024-05-08T05:45:30.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.578+0000 7f02dd522640 1 -- 172.21.15.45:0/3931958101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d80c8f40 con 0x7f02d815f830 2024-05-08T05:45:30.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.578+0000 7f02bf7fe640 1 -- 172.21.15.45:0/3931958101 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0030920 con 0x7f02d815f830 2024-05-08T05:45:30.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.578+0000 7f02dd522640 1 -- 172.21.15.45:0/3931958101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d80c9500 con 0x7f02d815f830 2024-05-08T05:45:30.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.580+0000 7f02bf7fe640 1 -- 172.21.15.45:0/3931958101 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f02d0030ac0 con 0x7f02d815f830 2024-05-08T05:45:30.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.580+0000 7f02bf7fe640 1 --2- 172.21.15.45:0/3931958101 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f02c40762d0 0x7f02c4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.580+0000 7f02bf7fe640 1 -- 172.21.15.45:0/3931958101 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f02d00b7fb0 con 0x7f02d815f830 2024-05-08T05:45:30.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.581+0000 7f02d6ffd640 1 --2- 172.21.15.45:0/3931958101 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f02c40762d0 0x7f02c4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.581+0000 7f02e0234740 1 Processor -- start 2024-05-08T05:45:30.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.581+0000 7f02e0234740 1 -- start start 2024-05-08T05:45:30.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.581+0000 7f02d6ffd640 1 --2- 172.21.15.45:0/3931958101 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f02c40762d0 0x7f02c4078790 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f02d80b5610 tx=0x7f02c8003a10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:30.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.581+0000 7f02e0234740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563faebd1f50 0x563faebd4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.581+0000 7f02e0234740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd2330 0x563faebd4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.581+0000 7f02e0234740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faeba4bb0 0x563faeba83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.581+0000 7f02e0234740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563faeb0e8d0 con 0x563faeba4bb0 2024-05-08T05:45:30.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.581+0000 7f02e0234740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563faeac7c10 con 0x563faebd2330 2024-05-08T05:45:30.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.581+0000 7f02e0234740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x563faeb0a890 con 0x563faebd1f50 2024-05-08T05:45:30.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.582+0000 7f02d77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faeba4bb0 0x563faeba83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.582+0000 7f02d77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faeba4bb0 0x563faeba83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56590/0 (socket says 172.21.15.45:56590) 2024-05-08T05:45:30.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.582+0000 7f02d77fe640 1 -- 172.21.15.45:0/1753396991 learned_addr learned my addr 172.21.15.45:0/1753396991 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:30.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.582+0000 7f02d6ffd640 1 --2- 172.21.15.45:0/1753396991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563faebd1f50 0x563faebd4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.582+0000 7f02d67fc640 1 --2- 172.21.15.45:0/1753396991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd2330 0x563faebd4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.582+0000 7f02d77fe640 1 -- 172.21.15.45:0/1753396991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563faebd1f50 msgr2=0x563faebd4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:30.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.582+0000 7f02d77fe640 1 --2- 172.21.15.45:0/1753396991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563faebd1f50 0x563faebd4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.582+0000 7f02d77fe640 1 -- 172.21.15.45:0/1753396991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd2330 msgr2=0x563faebd4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:30.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.582+0000 7f02d77fe640 1 --2- 172.21.15.45:0/1753396991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd2330 0x563faebd4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.582+0000 7f02d77fe640 1 -- 172.21.15.45:0/1753396991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563faeacf5d0 con 0x563faeba4bb0 2024-05-08T05:45:30.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.582+0000 7f02d77fe640 1 --2- 172.21.15.45:0/1753396991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faeba4bb0 0x563faeba83c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f02d00b6350 tx=0x7f02d0080bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e606f9fe134fcd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:30.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.582+0000 7f02bcff9640 1 -- 172.21.15.45:0/1753396991 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0080cd0 con 0x563faeba4bb0 2024-05-08T05:45:30.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.582+0000 7f02bcff9640 1 -- 172.21.15.45:0/1753396991 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d00811c0 con 0x563faeba4bb0 2024-05-08T05:45:30.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.582+0000 7f02bcff9640 1 -- 172.21.15.45:0/1753396991 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d00854c0 con 0x563faeba4bb0 2024-05-08T05:45:30.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.583+0000 7f02e0234740 1 -- 172.21.15.45:0/1753396991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faeba4bb0 msgr2=0x563faeba83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:30.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.583+0000 7f02e0234740 1 --2- 172.21.15.45:0/1753396991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faeba4bb0 0x563faeba83c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f02d00b6350 tx=0x7f02d0080bd0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.583+0000 7f02e0234740 1 -- 172.21.15.45:0/1753396991 shutdown_connections 2024-05-08T05:45:30.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.583+0000 7f02e0234740 1 --2- 172.21.15.45:0/1753396991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faeba4bb0 0x563faeba83c0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.583+0000 7f02e0234740 1 --2- 172.21.15.45:0/1753396991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd2330 0x563faebd4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.583+0000 7f02e0234740 1 --2- 172.21.15.45:0/1753396991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563faebd1f50 0x563faebd4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.583+0000 7f02e0234740 1 -- 172.21.15.45:0/1753396991 >> 172.21.15.45:0/1753396991 conn(0x563faeba6e20 msgr2=0x563faebef760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:30.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.583+0000 7f02e0234740 1 -- 172.21.15.45:0/1753396991 shutdown_connections 2024-05-08T05:45:30.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.583+0000 7f02e0234740 1 -- 172.21.15.45:0/1753396991 wait complete. 2024-05-08T05:45:30.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.584+0000 7f02e0234740 1 -- start start 2024-05-08T05:45:30.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.584+0000 7f02e0234740 1 Processor -- start 2024-05-08T05:45:30.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02e0234740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd1f50 0x563faebca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02e0234740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563faebd2330 0x563faebcb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02e0234740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebcbd80 0x563faeba93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02e0234740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563faeac7c10 con 0x563faebcbd80 2024-05-08T05:45:30.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02e0234740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563faeacb760 con 0x563faebd1f50 2024-05-08T05:45:30.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02e0234740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x563faeb0e8d0 con 0x563faebd2330 2024-05-08T05:45:30.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02e0234740 10 client.0 _ll_register_callbacks cb 0x563faeb04cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:45:30.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02d77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebcbd80 0x563faeba93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02d77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebcbd80 0x563faeba93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56606/0 (socket says 172.21.15.45:56606) 2024-05-08T05:45:30.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02d77fe640 1 -- 172.21.15.45:0/1689478736 learned_addr learned my addr 172.21.15.45:0/1689478736 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:30.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02d6ffd640 1 --2- 172.21.15.45:0/1689478736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd1f50 0x563faebca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02d67fc640 1 --2- 172.21.15.45:0/1689478736 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563faebd2330 0x563faebcb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02d77fe640 1 -- 172.21.15.45:0/1689478736 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563faebd2330 msgr2=0x563faebcb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:30.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02d77fe640 1 --2- 172.21.15.45:0/1689478736 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563faebd2330 0x563faebcb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02d77fe640 1 -- 172.21.15.45:0/1689478736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd1f50 msgr2=0x563faebca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:30.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02d77fe640 1 --2- 172.21.15.45:0/1689478736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd1f50 0x563faebca740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02d77fe640 1 -- 172.21.15.45:0/1689478736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fae8202d0 con 0x563faebcbd80 2024-05-08T05:45:30.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02d77fe640 1 --2- 172.21.15.45:0/1689478736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebcbd80 0x563faeba93d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f02d00325c0 tx=0x7f02d0009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c769a32e330d1255 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:30.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:45:30.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02b5ffb640 1 -- 172.21.15.45:0/1689478736 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0080cd0 con 0x563faebcbd80 2024-05-08T05:45:30.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02b5ffb640 1 -- 172.21.15.45:0/1689478736 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f02d0080420 con 0x563faebcbd80 2024-05-08T05:45:30.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02e0234740 1 -- 172.21.15.45:0/1689478736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02d00811c0 con 0x563faebcbd80 2024-05-08T05:45:30.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02b5ffb640 1 -- 172.21.15.45:0/1689478736 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f02d00a1050 con 0x563faebcbd80 2024-05-08T05:45:30.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02e0234740 10 client.6297 fetch_fsmap learned FSMap version 44 2024-05-08T05:45:30.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.585+0000 7f02e0234740 1 -- 172.21.15.45:0/1689478736 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x563fae8ea6b0 con 0x563faebcbd80 2024-05-08T05:45:30.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.586+0000 7f02b5ffb640 1 -- 172.21.15.45:0/1689478736 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d00811c0 con 0x563faebcbd80 2024-05-08T05:45:30.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.586+0000 7f02b5ffb640 1 -- 172.21.15.45:0/1689478736 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f02d003a070 con 0x563faebcbd80 2024-05-08T05:45:30.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.586+0000 7f02e0234740 10 client.6297 fetch_fsmap finished waiting for FSMap version 44 2024-05-08T05:45:30.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.586+0000 7f02e0234740 10 client.6297 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:45:30.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.586+0000 7f02e0234740 20 client.6297 populate_metadata read hostname 'smithi045' 2024-05-08T05:45:30.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.586+0000 7f02e0234740 1 --2- 172.21.15.45:0/1689478736 >> [v2:172.21.15.63:6834/133050464,v1:172.21.15.63:6836/133050464] conn(0x563faebf6000 0x563faebf63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.586+0000 7f02e0234740 4 client.6297 mds_command: new command op to 6180 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:45:30.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.586+0000 7f02e0234740 1 -- 172.21.15.45:0/1689478736 --> [v2:172.21.15.63:6834/133050464,v1:172.21.15.63:6836/133050464] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563faea92760 con 0x563faebf6000 2024-05-08T05:45:30.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.586+0000 7f02d6ffd640 1 --2- 172.21.15.45:0/1689478736 >> [v2:172.21.15.63:6834/133050464,v1:172.21.15.63:6836/133050464] conn(0x563faebf6000 0x563faebf63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.586+0000 7f02b5ffb640 10 client.6297 ms_handle_connect on v2:172.21.15.63:6834/133050464 2024-05-08T05:45:30.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.586+0000 7f02d6ffd640 1 --2- 172.21.15.45:0/1689478736 >> [v2:172.21.15.63:6834/133050464,v1:172.21.15.63:6836/133050464] conn(0x563faebf6000 0x563faebf63e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae854cb0a8cb085 server_cookie=5b484ac4e129d02d in_seq=0 out_seq=0 2024-05-08T05:45:30.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.587+0000 7f02b5ffb640 1 -- 172.21.15.45:0/1689478736 <== mds.0 v2:172.21.15.63:6834/133050464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563faea92760 con 0x563faebf6000 2024-05-08T05:45:30.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.587+0000 7f02b5ffb640 10 client.6297 handle_command_reply: tid=0 2024-05-08T05:45:30.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.587+0000 7f02e0234740 1 client.6297 shutdown 2024-05-08T05:45:30.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.587+0000 7f02e0234740 1 -- 172.21.15.45:0/1689478736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebcbd80 msgr2=0x563faeba93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:30.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.587+0000 7f02e0234740 1 --2- 172.21.15.45:0/1689478736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebcbd80 0x563faeba93d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f02d00325c0 tx=0x7f02d0009240 comp rx=0 tx=0).stop 2024-05-08T05:45:30.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.587+0000 7f02e0234740 1 -- 172.21.15.45:0/1689478736 shutdown_connections 2024-05-08T05:45:30.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.587+0000 7f02e0234740 1 --2- 172.21.15.45:0/1689478736 >> [v2:172.21.15.63:6834/133050464,v1:172.21.15.63:6836/133050464] conn(0x563faebf6000 0x563faebf63e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.587+0000 7f02e0234740 1 --2- 172.21.15.45:0/1689478736 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563faebd2330 0x563faebcb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.587+0000 7f02e0234740 1 --2- 172.21.15.45:0/1689478736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebcbd80 0x563faeba93d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.587+0000 7f02b5ffb640 0 client.6297 ms_handle_reset on v2:172.21.15.63:6834/133050464 2024-05-08T05:45:30.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.587+0000 7f02e0234740 1 --2- 172.21.15.45:0/1689478736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd1f50 0x563faebca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.587+0000 7f02e0234740 1 -- 172.21.15.45:0/1689478736 >> 172.21.15.45:0/1689478736 conn(0x563faeba4bb0 msgr2=0x563faebcc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:30.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.588+0000 7f02e0234740 1 -- 172.21.15.45:0/1689478736 shutdown_connections 2024-05-08T05:45:30.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.588+0000 7f02e0234740 1 -- 172.21.15.45:0/1689478736 wait complete. 2024-05-08T05:45:30.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.588+0000 7f02e0234740 20 client.6297 trim_cache size 0 max 16384 2024-05-08T05:45:30.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.606+0000 7f02e0234740 1 Processor -- start 2024-05-08T05:45:30.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.606+0000 7f02e0234740 1 -- start start 2024-05-08T05:45:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.607+0000 7f02e0234740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563faebcbd80 0x563faeba5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.607+0000 7f02e0234740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebd1f50 0x563faebaad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.607+0000 7f02e0234740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd2330 0x563faebf3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.607+0000 7f02e0234740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f02d8200ea0 con 0x563faebd1f50 2024-05-08T05:45:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.607+0000 7f02e0234740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f02d80b88b0 con 0x563faebd2330 2024-05-08T05:45:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.607+0000 7f02e0234740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f02d8200850 con 0x563faebcbd80 2024-05-08T05:45:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.607+0000 7f02d67fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebd1f50 0x563faebaad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.607+0000 7f02d67fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebd1f50 0x563faebaad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56612/0 (socket says 172.21.15.45:56612) 2024-05-08T05:45:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.607+0000 7f02d67fc640 1 -- 172.21.15.45:0/2460429650 learned_addr learned my addr 172.21.15.45:0/2460429650 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.607+0000 7f02d67fc640 1 -- 172.21.15.45:0/2460429650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563faebcbd80 msgr2=0x563faeba5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.607+0000 7f02d67fc640 1 --2- 172.21.15.45:0/2460429650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563faebcbd80 0x563faeba5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.607+0000 7f02d67fc640 1 -- 172.21.15.45:0/2460429650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd2330 msgr2=0x563faebf3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:30.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.607+0000 7f02d77fe640 1 --2- 172.21.15.45:0/2460429650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd2330 0x563faebf3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.607+0000 7f02d67fc640 1 --2- 172.21.15.45:0/2460429650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd2330 0x563faebf3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.607+0000 7f02d67fc640 1 -- 172.21.15.45:0/2460429650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563faec13400 con 0x563faebd1f50 2024-05-08T05:45:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.607+0000 7f02d77fe640 1 --2- 172.21.15.45:0/2460429650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd2330 0x563faebf3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.608+0000 7f02d67fc640 1 --2- 172.21.15.45:0/2460429650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebd1f50 0x563faebaad70 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f02d80b4ed0 tx=0x7f02cc00bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbafa9e464cf40f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.608+0000 7f02b7fff640 1 -- 172.21.15.45:0/2460429650 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc015960 con 0x563faebd1f50 2024-05-08T05:45:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.608+0000 7f02b7fff640 1 -- 172.21.15.45:0/2460429650 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02cc015b00 con 0x563faebd1f50 2024-05-08T05:45:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.608+0000 7f02b7fff640 1 -- 172.21.15.45:0/2460429650 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc01f430 con 0x563faebd1f50 2024-05-08T05:45:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.608+0000 7f02e0234740 1 -- 172.21.15.45:0/2460429650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebd1f50 msgr2=0x563faebaad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.608+0000 7f02e0234740 1 --2- 172.21.15.45:0/2460429650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebd1f50 0x563faebaad70 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f02d80b4ed0 tx=0x7f02cc00bd90 comp rx=0 tx=0).stop 2024-05-08T05:45:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.608+0000 7f02e0234740 1 -- 172.21.15.45:0/2460429650 shutdown_connections 2024-05-08T05:45:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.608+0000 7f02e0234740 1 --2- 172.21.15.45:0/2460429650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd2330 0x563faebf3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.608+0000 7f02e0234740 1 --2- 172.21.15.45:0/2460429650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebd1f50 0x563faebaad70 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.608+0000 7f02e0234740 1 --2- 172.21.15.45:0/2460429650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563faebcbd80 0x563faeba5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.608+0000 7f02e0234740 1 -- 172.21.15.45:0/2460429650 >> 172.21.15.45:0/2460429650 conn(0x563faeba4bb0 msgr2=0x563faebca550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:30.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.608+0000 7f02e0234740 1 -- 172.21.15.45:0/2460429650 shutdown_connections 2024-05-08T05:45:30.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.608+0000 7f02e0234740 1 -- 172.21.15.45:0/2460429650 wait complete. 2024-05-08T05:45:30.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.609+0000 7f02e0234740 1 -- start start 2024-05-08T05:45:30.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.609+0000 7f02e0234740 1 Processor -- start 2024-05-08T05:45:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02e0234740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebcbd80 0x563faeba5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02e0234740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563faebd1f50 0x563faeba83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02e0234740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd2330 0x563faeba8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02e0234740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563faeac7c10 con 0x563faebcbd80 2024-05-08T05:45:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02e0234740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x563faeacb760 con 0x563faebd2330 2024-05-08T05:45:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02e0234740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x563faebf1e00 con 0x563faebd1f50 2024-05-08T05:45:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02e0234740 10 client.0 _ll_register_callbacks cb 0x7f02d0099070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:45:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02d6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebcbd80 0x563faeba5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02d6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebcbd80 0x563faeba5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56616/0 (socket says 172.21.15.45:56616) 2024-05-08T05:45:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02d6ffd640 1 -- 172.21.15.45:0/1965489115 learned_addr learned my addr 172.21.15.45:0/1965489115 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02d77fe640 1 --2- 172.21.15.45:0/1965489115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd2330 0x563faeba8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02d6ffd640 1 -- 172.21.15.45:0/1965489115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563faebd1f50 msgr2=0x563faeba83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02d67fc640 1 --2- 172.21.15.45:0/1965489115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563faebd1f50 0x563faeba83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02d6ffd640 1 --2- 172.21.15.45:0/1965489115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563faebd1f50 0x563faeba83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02d6ffd640 1 -- 172.21.15.45:0/1965489115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd2330 msgr2=0x563faeba8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02d6ffd640 1 --2- 172.21.15.45:0/1965489115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd2330 0x563faeba8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02d6ffd640 1 -- 172.21.15.45:0/1965489115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563faebf3af0 con 0x563faebcbd80 2024-05-08T05:45:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02d77fe640 1 --2- 172.21.15.45:0/1965489115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd2330 0x563faeba8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02d67fc640 1 --2- 172.21.15.45:0/1965489115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563faebd1f50 0x563faeba83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02d6ffd640 1 --2- 172.21.15.45:0/1965489115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebcbd80 0x563faeba5570 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x563faeb0e9f0 tx=0x7f02c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dedff20a059acba4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:45:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02b5ffb640 1 -- 172.21.15.45:0/1965489115 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8031060 con 0x563faebcbd80 2024-05-08T05:45:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02b5ffb640 1 -- 172.21.15.45:0/1965489115 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f02c802b2f0 con 0x563faebcbd80 2024-05-08T05:45:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.610+0000 7f02e0234740 1 -- 172.21.15.45:0/1965489115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02cc015b00 con 0x563faebcbd80 2024-05-08T05:45:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.611+0000 7f02b5ffb640 1 -- 172.21.15.45:0/1965489115 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f02c802b4f0 con 0x563faebcbd80 2024-05-08T05:45:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.611+0000 7f02e0234740 10 client.6303 fetch_fsmap learned FSMap version 44 2024-05-08T05:45:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.611+0000 7f02e0234740 1 -- 172.21.15.45:0/1965489115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) -- 0x563faebec110 con 0x563faebcbd80 2024-05-08T05:45:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.611+0000 7f02b5ffb640 1 -- 172.21.15.45:0/1965489115 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02cc015b00 con 0x563faebcbd80 2024-05-08T05:45:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.611+0000 7f02b5ffb640 1 -- 172.21.15.45:0/1965489115 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 44) ==== 3494+0+0 (secure 0 0 0) 0x7f02c802bce0 con 0x563faebcbd80 2024-05-08T05:45:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.611+0000 7f02e0234740 10 client.6303 fetch_fsmap finished waiting for FSMap version 44 2024-05-08T05:45:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.611+0000 7f02e0234740 10 client.6303 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:45:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.611+0000 7f02e0234740 20 client.6303 populate_metadata read hostname 'smithi045' 2024-05-08T05:45:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.611+0000 7f02e0234740 1 --2- 172.21.15.45:0/1965489115 >> [v2:172.21.15.63:6834/133050464,v1:172.21.15.63:6836/133050464] conn(0x563faebd07e0 0x563faebd0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.611+0000 7f02e0234740 4 client.6303 mds_command: new command op to 6180 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:45:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.611+0000 7f02e0234740 1 -- 172.21.15.45:0/1965489115 --> [v2:172.21.15.63:6834/133050464,v1:172.21.15.63:6836/133050464] -- command(tid 0: {"prefix": "status"}) -- 0x563faebabf50 con 0x563faebd07e0 2024-05-08T05:45:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.611+0000 7f02d67fc640 1 --2- 172.21.15.45:0/1965489115 >> [v2:172.21.15.63:6834/133050464,v1:172.21.15.63:6836/133050464] conn(0x563faebd07e0 0x563faebd0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.612+0000 7f02d67fc640 1 --2- 172.21.15.45:0/1965489115 >> [v2:172.21.15.63:6834/133050464,v1:172.21.15.63:6836/133050464] conn(0x563faebd07e0 0x563faebd0bc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f37083e27206d2d1 server_cookie=4bbf4b6148dfc155 in_seq=0 out_seq=0 2024-05-08T05:45:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.612+0000 7f02b5ffb640 10 client.6303 ms_handle_connect on v2:172.21.15.63:6834/133050464 2024-05-08T05:45:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.612+0000 7f02b5ffb640 1 -- 172.21.15.45:0/1965489115 <== mds.0 v2:172.21.15.63:6834/133050464 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x563faebabf50 con 0x563faebd07e0 2024-05-08T05:45:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.612+0000 7f02b5ffb640 10 client.6303 handle_command_reply: tid=0 2024-05-08T05:45:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.612+0000 7f02e0234740 1 client.6303 shutdown 2024-05-08T05:45:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.612+0000 7f02e0234740 1 -- 172.21.15.45:0/1965489115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebcbd80 msgr2=0x563faeba5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.612+0000 7f02e0234740 1 --2- 172.21.15.45:0/1965489115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebcbd80 0x563faeba5570 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x563faeb0e9f0 tx=0x7f02c8004a40 comp rx=0 tx=0).stop 2024-05-08T05:45:30.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.612+0000 7f02e0234740 1 -- 172.21.15.45:0/1965489115 shutdown_connections 2024-05-08T05:45:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.612+0000 7f02e0234740 1 --2- 172.21.15.45:0/1965489115 >> [v2:172.21.15.63:6834/133050464,v1:172.21.15.63:6836/133050464] conn(0x563faebd07e0 0x563faebd0bc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.612+0000 7f02e0234740 1 --2- 172.21.15.45:0/1965489115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x563faebd1f50 0x563faeba83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.612+0000 7f02e0234740 1 --2- 172.21.15.45:0/1965489115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x563faebd2330 0x563faeba8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.612+0000 7f02b5ffb640 0 client.6303 ms_handle_reset on v2:172.21.15.63:6834/133050464 2024-05-08T05:45:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.612+0000 7f02e0234740 1 --2- 172.21.15.45:0/1965489115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563faebcbd80 0x563faeba5570 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.612+0000 7f02e0234740 1 -- 172.21.15.45:0/1965489115 >> 172.21.15.45:0/1965489115 conn(0x563faeba4bb0 msgr2=0x563faebf63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.612+0000 7f02e0234740 1 -- 172.21.15.45:0/1965489115 shutdown_connections 2024-05-08T05:45:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.612+0000 7f02e0234740 1 -- 172.21.15.45:0/1965489115 wait complete. 2024-05-08T05:45:30.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.612+0000 7f02e0234740 20 client.6303 trim_cache size 0 max 16384 2024-05-08T05:45:30.618 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:45:30.618 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "e9e1170a-fd27-4625-97a9-5d5f104d13a2", 2024-05-08T05:45:30.619 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-05-08T05:45:30.619 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 6180, 2024-05-08T05:45:30.619 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-05-08T05:45:30.619 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-05-08T05:45:30.619 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-05-08T05:45:30.619 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 5.7892142519999998, 2024-05-08T05:45:30.619 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 44, 2024-05-08T05:45:30.619 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 75, 2024-05-08T05:45:30.619 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 75, 2024-05-08T05:45:30.619 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 11.089658225000001 2024-05-08T05:45:30.619 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:45:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.613+0000 7f02dd522640 1 -- 172.21.15.45:0/3931958101 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f02c40762d0 msgr2=0x7f02c4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.613+0000 7f02dd522640 1 --2- 172.21.15.45:0/3931958101 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f02c40762d0 0x7f02c4078790 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f02d80b5610 tx=0x7f02c8003a10 comp rx=0 tx=0).stop 2024-05-08T05:45:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.613+0000 7f02dd522640 1 -- 172.21.15.45:0/3931958101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02d815f830 msgr2=0x7f02d80c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.613+0000 7f02dd522640 1 --2- 172.21.15.45:0/3931958101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02d815f830 0x7f02d80c9d00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f02d002a220 tx=0x7f02d00046c0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.613+0000 7f02dd522640 1 -- 172.21.15.45:0/3931958101 shutdown_connections 2024-05-08T05:45:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.613+0000 7f02dd522640 1 --2- 172.21.15.45:0/3931958101 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f02c40762d0 0x7f02c4078790 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.613+0000 7f02dd522640 1 --2- 172.21.15.45:0/3931958101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02d815f830 0x7f02d80c9d00 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.613+0000 7f02dd522640 1 --2- 172.21.15.45:0/3931958101 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02d815f450 0x7f02d80b98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.613+0000 7f02dd522640 1 --2- 172.21.15.45:0/3931958101 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02d80d0700 0x7f02d815ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.613+0000 7f02dd522640 1 -- 172.21.15.45:0/3931958101 >> 172.21.15.45:0/3931958101 conn(0x7f02d80d5040 msgr2=0x7f02d806d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:30.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.613+0000 7f02dd522640 1 -- 172.21.15.45:0/3931958101 shutdown_connections 2024-05-08T05:45:30.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.613+0000 7f02dd522640 1 -- 172.21.15.45:0/3931958101 wait complete. 2024-05-08T05:45:30.624 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T05:45:30.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.727+0000 7fbf56a8c640 1 Processor -- start 2024-05-08T05:45:30.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.727+0000 7fbf56a8c640 1 -- start start 2024-05-08T05:45:30.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.728+0000 7fbf56a8c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbf50070d30 0x7fbf500d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.728+0000 7fbf56a8c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf50163920 0x7fbf500703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.728+0000 7fbf56a8c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf50163d80 con 0x7fbf50071110 2024-05-08T05:45:30.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.728+0000 7fbf56a8c640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbf500d10d0 con 0x7fbf50070d30 2024-05-08T05:45:30.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.728+0000 7fbf56a8c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf50070b80 con 0x7fbf50163920 2024-05-08T05:45:30.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.728+0000 7fbf55029640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbf50071110 0x7fbf5006d4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50546/0 (socket says 172.21.15.45:50546) 2024-05-08T05:45:30.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.728+0000 7fbf55029640 1 -- 172.21.15.45:0/4165663416 learned_addr learned my addr 172.21.15.45:0/4165663416 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:30.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.728+0000 7fbf54828640 1 --2- 172.21.15.45:0/4165663416 >> v2:172.21.15.63:3300/0 conn(0x7fbf50070d30 0x7fbf500d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.728+0000 7fbf4ffff640 1 --2- 172.21.15.45:0/4165663416 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf50163920 0x7fbf500703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.729+0000 7fbf4f7fe640 1 -- 172.21.15.45:0/4165663416 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2342020484 0 0) 0x7fbf50163d80 con 0x7fbf50071110 2024-05-08T05:45:30.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.729+0000 7fbf4f7fe640 1 -- 172.21.15.45:0/4165663416 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf3c003520 con 0x7fbf50071110 2024-05-08T05:45:30.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.729+0000 7fbf4f7fe640 1 -- 172.21.15.45:0/4165663416 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 716995826 0 0) 0x7fbf3c003520 con 0x7fbf50071110 2024-05-08T05:45:30.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.729+0000 7fbf4ffff640 1 -- 172.21.15.45:0/4165663416 >> v2:172.21.15.63:3300/0 conn(0x7fbf50070d30 msgr2=0x7fbf500d20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:30.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.729+0000 7fbf4ffff640 1 --2- 172.21.15.45:0/4165663416 >> v2:172.21.15.63:3300/0 conn(0x7fbf50070d30 0x7fbf500d20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.729+0000 7fbf4ffff640 1 -- 172.21.15.45:0/4165663416 >> v1:172.21.15.45:6789/0 conn(0x7fbf50071110 legacy=0x7fbf5006d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:30.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.729+0000 7fbf4ffff640 1 -- 172.21.15.45:0/4165663416 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf500708e0 con 0x7fbf50163920 2024-05-08T05:45:30.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.729+0000 7fbf54828640 1 --2- 172.21.15.45:0/4165663416 >> v2:172.21.15.63:3300/0 conn(0x7fbf50070d30 0x7fbf500d20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:45:30.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.730+0000 7fbf4ffff640 1 --2- 172.21.15.45:0/4165663416 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf50163920 0x7fbf500703a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbf40009d90 tx=0x7fbf4002a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a40d6fb44c133c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:30.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.730+0000 7fbf4f7fe640 1 -- 172.21.15.45:0/4165663416 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf480016f0 con 0x7fbf50163920 2024-05-08T05:45:30.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.730+0000 7fbf4f7fe640 1 -- 172.21.15.45:0/4165663416 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf40007540 con 0x7fbf50163920 2024-05-08T05:45:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.730+0000 7fbf4f7fe640 1 -- 172.21.15.45:0/4165663416 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf400078f0 con 0x7fbf50163920 2024-05-08T05:45:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.730+0000 7fbf56a8c640 1 -- 172.21.15.45:0/4165663416 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf50163920 msgr2=0x7fbf500703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.730+0000 7fbf56a8c640 1 --2- 172.21.15.45:0/4165663416 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf50163920 0x7fbf500703a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbf40009d90 tx=0x7fbf4002a5f0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.731+0000 7fbf56a8c640 1 -- 172.21.15.45:0/4165663416 shutdown_connections 2024-05-08T05:45:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.731+0000 7fbf56a8c640 1 --2- 172.21.15.45:0/4165663416 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf50163920 0x7fbf500703a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.731+0000 7fbf56a8c640 1 --2- 172.21.15.45:0/4165663416 >> v2:172.21.15.63:3300/0 conn(0x7fbf50070d30 0x7fbf500d20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.731+0000 7fbf56a8c640 1 -- 172.21.15.45:0/4165663416 >> 172.21.15.45:0/4165663416 conn(0x7fbf500d2e90 msgr2=0x7fbf500d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.731+0000 7fbf56a8c640 1 -- 172.21.15.45:0/4165663416 shutdown_connections 2024-05-08T05:45:30.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.731+0000 7fbf56a8c640 1 -- 172.21.15.45:0/4165663416 wait complete. 2024-05-08T05:45:30.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.732+0000 7fbf56a8c640 1 Processor -- start 2024-05-08T05:45:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.732+0000 7fbf56a8c640 1 -- start start 2024-05-08T05:45:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.733+0000 7fbf56a8c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf50070d30 0x7fbf500b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.733+0000 7fbf56a8c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf50071110 0x7fbf500cfff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.733+0000 7fbf56a8c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf50163920 0x7fbf500b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.733+0000 7fbf56a8c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf500d15e0 con 0x7fbf50163920 2024-05-08T05:45:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.733+0000 7fbf56a8c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf500d1c30 con 0x7fbf50070d30 2024-05-08T05:45:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.733+0000 7fbf56a8c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf500d18c0 con 0x7fbf50071110 2024-05-08T05:45:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.733+0000 7fbf55029640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf50163920 0x7fbf500b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.733+0000 7fbf55029640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf50163920 0x7fbf500b80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45322/0 (socket says 172.21.15.45:45322) 2024-05-08T05:45:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.733+0000 7fbf55029640 1 -- 172.21.15.45:0/3249266404 learned_addr learned my addr 172.21.15.45:0/3249266404 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.733+0000 7fbf54828640 1 --2- 172.21.15.45:0/3249266404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf50070d30 0x7fbf500b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.733+0000 7fbf4ffff640 1 --2- 172.21.15.45:0/3249266404 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf50071110 0x7fbf500cfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.733+0000 7fbf55029640 1 -- 172.21.15.45:0/3249266404 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf50071110 msgr2=0x7fbf500cfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.733+0000 7fbf55029640 1 --2- 172.21.15.45:0/3249266404 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf50071110 0x7fbf500cfff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.733+0000 7fbf55029640 1 -- 172.21.15.45:0/3249266404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf50070d30 msgr2=0x7fbf500b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:30.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.733+0000 7fbf55029640 1 --2- 172.21.15.45:0/3249266404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf50070d30 0x7fbf500b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.733+0000 7fbf55029640 1 -- 172.21.15.45:0/3249266404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf500b85f0 con 0x7fbf50163920 2024-05-08T05:45:30.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.733+0000 7fbf54828640 1 --2- 172.21.15.45:0/3249266404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf50070d30 0x7fbf500b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:45:30.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.733+0000 7fbf55029640 1 --2- 172.21.15.45:0/3249266404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf50163920 0x7fbf500b80b0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800c530 tx=0x7fbf4800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:30.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.734+0000 7fbf4d7fa640 1 -- 172.21.15.45:0/3249266404 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48005ce0 con 0x7fbf50163920 2024-05-08T05:45:30.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.734+0000 7fbf4d7fa640 1 -- 172.21.15.45:0/3249266404 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf48007070 con 0x7fbf50163920 2024-05-08T05:45:30.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.734+0000 7fbf56a8c640 1 -- 172.21.15.45:0/3249266404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf500b88c0 con 0x7fbf50163920 2024-05-08T05:45:30.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.734+0000 7fbf4d7fa640 1 -- 172.21.15.45:0/3249266404 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48007460 con 0x7fbf50163920 2024-05-08T05:45:30.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.734+0000 7fbf56a8c640 1 -- 172.21.15.45:0/3249266404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf500b2e60 con 0x7fbf50163920 2024-05-08T05:45:30.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.735+0000 7fbf56a8c640 1 -- 172.21.15.45:0/3249266404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf500693f0 con 0x7fbf50163920 2024-05-08T05:45:30.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.735+0000 7fbf4d7fa640 1 -- 172.21.15.45:0/3249266404 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbf48007600 con 0x7fbf50163920 2024-05-08T05:45:30.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.736+0000 7fbf4d7fa640 1 --2- 172.21.15.45:0/3249266404 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbf300762d0 0x7fbf30078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:30.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.736+0000 7fbf4d7fa640 1 -- 172.21.15.45:0/3249266404 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fbf480972a0 con 0x7fbf50163920 2024-05-08T05:45:30.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.736+0000 7fbf54828640 1 --2- 172.21.15.45:0/3249266404 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbf300762d0 0x7fbf30078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:30.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.737+0000 7fbf54828640 1 --2- 172.21.15.45:0/3249266404 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbf300762d0 0x7fbf30078790 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009560 tx=0x7fbf38009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:30.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.741+0000 7fbf4d7fa640 1 -- 172.21.15.45:0/3249266404 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbf48060420 con 0x7fbf50163920 2024-05-08T05:45:30.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.971+0000 7fbf56a8c640 1 -- 172.21.15.45:0/3249266404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbf500d3d70 con 0x7fbf50163920 2024-05-08T05:45:30.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.971+0000 7fbf4d7fa640 1 -- 172.21.15.45:0/3249266404 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) ==== 53+0+73 (secure 0 0 0) 0x7fbf48064720 con 0x7fbf50163920 2024-05-08T05:45:30.971 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T05:45:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.973+0000 7fbf56a8c640 1 -- 172.21.15.45:0/3249266404 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbf300762d0 msgr2=0x7fbf30078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.973+0000 7fbf56a8c640 1 --2- 172.21.15.45:0/3249266404 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbf300762d0 0x7fbf30078790 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009560 tx=0x7fbf38009190 comp rx=0 tx=0).stop 2024-05-08T05:45:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.973+0000 7fbf56a8c640 1 -- 172.21.15.45:0/3249266404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf50163920 msgr2=0x7fbf500b80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.973+0000 7fbf56a8c640 1 --2- 172.21.15.45:0/3249266404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf50163920 0x7fbf500b80b0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800c530 tx=0x7fbf4800c9f0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.973+0000 7fbf56a8c640 1 -- 172.21.15.45:0/3249266404 shutdown_connections 2024-05-08T05:45:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.973+0000 7fbf56a8c640 1 --2- 172.21.15.45:0/3249266404 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbf300762d0 0x7fbf30078790 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.973+0000 7fbf56a8c640 1 --2- 172.21.15.45:0/3249266404 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf50071110 0x7fbf500cfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.973+0000 7fbf56a8c640 1 --2- 172.21.15.45:0/3249266404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf50163920 0x7fbf500b80b0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.973+0000 7fbf56a8c640 1 --2- 172.21.15.45:0/3249266404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf50070d30 0x7fbf500b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.973+0000 7fbf56a8c640 1 -- 172.21.15.45:0/3249266404 >> 172.21.15.45:0/3249266404 conn(0x7fbf500d2e90 msgr2=0x7fbf50162d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.973+0000 7fbf56a8c640 1 -- 172.21.15.45:0/3249266404 shutdown_connections 2024-05-08T05:45:30.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:30.973+0000 7fbf56a8c640 1 -- 172.21.15.45:0/3249266404 wait complete. 2024-05-08T05:45:30.984 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T05:45:30.985 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:45:30.985 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-05-08T05:45:30.985 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T05:45:30.985 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:45:30.985 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T05:45:30.985 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T05:45:30.985 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:45:30.985 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:45:30.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:30.985 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-05-08T05:45:31.002 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:45:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:45:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:45:31.002 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:45:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-05-08T05:45:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:45:31.003 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:45:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:45:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-05-08T05:45:31.003 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:45:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:45:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-05-08T05:45:31.003 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:45:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-05-08T05:45:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-05-08T05:45:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:45:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42493sec preferred_lft 42493sec 2024-05-08T05:45:31.003 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-05-08T05:45:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:45:31.004 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:45:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-05-08T05:45:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-05-08T05:45:31.004 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:45:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T05:45:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T05:45:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:45:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c80a:aff:fe94:c77/64 scope link 2024-05-08T05:45:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:45:31.004 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:45:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 82:f3:eb:6a:57:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:45:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::80f3:ebff:fe6a:57ba/64 scope link 2024-05-08T05:45:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:45:31.004 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:45:31.004 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T05:45:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::14a2:64ff:feda:3a9c/64 scope link 2024-05-08T05:45:31.005 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:45:31.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:31.005 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:45:31.067 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:45:31.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:31.069 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-05-08T05:45:31.126 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-05-08T05:45:31.126 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:45:31.127 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:45:31.127 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:45:31.127 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:45:31.127 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:45:31.127 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:45:31.128 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:45:31.128 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:45:31.128 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:45:31.128 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:45:31.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:45:31.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:45:31.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:31.257 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:45:31.273 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T05:45:31.273 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T05:45:31.274 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:45:31.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:31.274 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-05-08T05:45:31.340 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:45:31.340 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T05:45:31.396 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:45:31.396 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:45:31.396 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:45:31.451 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T05:45:31.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:31.452 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:45:31.537 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:45:31.538 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:45:31.546 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:45:31.546 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:45:31.546 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-08T05:45:31.546 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@e9e1170a-fd27-4625-97a9-5d5f104d13a2.cephfs=/ 2024-05-08T05:45:31.546 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.63:3300/172.21.15.63:3301" will pass to kernel 2024-05-08T05:45:31.546 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T05:45:31.546 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:45:31.546 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:45:31.547 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:45:31.547 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:45:31.548 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:45:31.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:31.548 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:45:31.575 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:45:31.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:45:31.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:45:31.683 INFO:teuthology.orchestra.run.smithi045.stdout:client5492 2024-05-08T05:45:31.683 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:45:31.683 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:45:31.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:31.700 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-05-08T05:45:31.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:31.763 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-05-08T05:45:31.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:31.823 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-05-08T05:45:31.881 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:45:31.881 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:45:31.881 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:45:31.881 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:45:31.881 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-05-08T05:45:31.881 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:45:31.989 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627776 2024-05-08T05:45:31.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:45:31.998 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:45:31.998 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:45:32.055 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:45:32.055 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:45:32.112 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:45:32.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.256+0000 7fb948790640 1 Processor -- start 2024-05-08T05:45:32.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.256+0000 7fb948790640 1 -- start start 2024-05-08T05:45:32.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.257+0000 7fb948790640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb9400d3cd0 0x7fb9400cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:32.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.257+0000 7fb948790640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9400d3330 con 0x7fb9400c9ee0 2024-05-08T05:45:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.257+0000 7fb948790640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9400d2170 con 0x7fb9401629a0 2024-05-08T05:45:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.257+0000 7fb948790640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb9400d2450 con 0x7fb9400d3cd0 2024-05-08T05:45:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.257+0000 7fb946d2d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb9400c9ee0 0x7fb9400d01e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50554/0 (socket says 172.21.15.45:50554) 2024-05-08T05:45:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.257+0000 7fb946d2d640 1 -- 172.21.15.45:0/497413037 learned_addr learned my addr 172.21.15.45:0/497413037 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94552a640 1 -- 172.21.15.45:0/497413037 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 527953711 0 0) 0x7fb9400d3330 con 0x7fb9400c9ee0 2024-05-08T05:45:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94552a640 1 -- 172.21.15.45:0/497413037 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb928003520 con 0x7fb9400c9ee0 2024-05-08T05:45:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb945d2b640 1 --2- 172.21.15.45:0/497413037 >> v2:172.21.15.63:3300/0 conn(0x7fb9400d3cd0 0x7fb9400cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94552a640 1 -- 172.21.15.45:0/497413037 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1737366331 0 0) 0x7fb928003520 con 0x7fb9400c9ee0 2024-05-08T05:45:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94552a640 1 -- 172.21.15.45:0/497413037 >> v2:172.21.15.63:3300/0 conn(0x7fb9400d3cd0 msgr2=0x7fb9400cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94552a640 1 --2- 172.21.15.45:0/497413037 >> v2:172.21.15.63:3300/0 conn(0x7fb9400d3cd0 0x7fb9400cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94552a640 1 -- 172.21.15.45:0/497413037 >> v1:172.21.15.63:6789/0 conn(0x7fb9401629a0 legacy=0x7fb9400d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94552a640 1 -- 172.21.15.45:0/497413037 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9400d2730 con 0x7fb9400c9ee0 2024-05-08T05:45:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94552a640 1 -- 172.21.15.45:0/497413037 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fb9380016f0 con 0x7fb9400c9ee0 2024-05-08T05:45:32.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94552a640 1 -- 172.21.15.45:0/497413037 >> v1:172.21.15.45:6789/0 conn(0x7fb9400c9ee0 legacy=0x7fb9400d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94552a640 1 --2- 172.21.15.45:0/497413037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb928003a50 0x7fb928003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94552a640 1 --2- 172.21.15.45:0/497413037 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb928004800 0x7fb928004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94552a640 1 --2- 172.21.15.45:0/497413037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9280053c0 0x7fb92800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94552a640 1 -- 172.21.15.45:0/497413037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb9400d2450 con 0x7fb928003a50 2024-05-08T05:45:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94552a640 1 -- 172.21.15.45:0/497413037 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb92800c230 con 0x7fb9280053c0 2024-05-08T05:45:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94552a640 1 -- 172.21.15.45:0/497413037 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb92800c430 con 0x7fb928004800 2024-05-08T05:45:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94652c640 1 --2- 172.21.15.45:0/497413037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb928003a50 0x7fb928003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb946d2d640 1 --2- 172.21.15.45:0/497413037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9280053c0 0x7fb92800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb945d2b640 1 --2- 172.21.15.45:0/497413037 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb928004800 0x7fb928004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94652c640 1 -- 172.21.15.45:0/497413037 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb928004800 msgr2=0x7fb928004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94652c640 1 --2- 172.21.15.45:0/497413037 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb928004800 0x7fb928004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94652c640 1 -- 172.21.15.45:0/497413037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9280053c0 msgr2=0x7fb92800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94652c640 1 --2- 172.21.15.45:0/497413037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9280053c0 0x7fb92800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb94652c640 1 -- 172.21.15.45:0/497413037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb928003520 con 0x7fb928003a50 2024-05-08T05:45:32.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb946d2d640 1 --2- 172.21.15.45:0/497413037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9280053c0 0x7fb92800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:32.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.258+0000 7fb946d2d640 1 -- 172.21.15.45:0/497413037 reap_dead start 2024-05-08T05:45:32.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.259+0000 7fb94652c640 1 --2- 172.21.15.45:0/497413037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb928003a50 0x7fb928003e70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb930001e90 tx=0x7fb928005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22bb908a852f4e76 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:32.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.259+0000 7fb94552a640 1 -- 172.21.15.45:0/497413037 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb930001ab0 con 0x7fb928003a50 2024-05-08T05:45:32.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.259+0000 7fb94552a640 1 -- 172.21.15.45:0/497413037 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb93002ae40 con 0x7fb928003a50 2024-05-08T05:45:32.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.259+0000 7fb94552a640 1 -- 172.21.15.45:0/497413037 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb930003b70 con 0x7fb928003a50 2024-05-08T05:45:32.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.260+0000 7fb948790640 1 -- 172.21.15.45:0/497413037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb928003a50 msgr2=0x7fb928003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:32.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.260+0000 7fb948790640 1 --2- 172.21.15.45:0/497413037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb928003a50 0x7fb928003e70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb930001e90 tx=0x7fb928005830 comp rx=0 tx=0).stop 2024-05-08T05:45:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.260+0000 7fb948790640 1 -- 172.21.15.45:0/497413037 shutdown_connections 2024-05-08T05:45:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.260+0000 7fb948790640 1 --2- 172.21.15.45:0/497413037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb928003a50 0x7fb928003e70 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.260+0000 7fb948790640 1 -- 172.21.15.45:0/497413037 >> 172.21.15.45:0/497413037 conn(0x7fb9400d4fe0 msgr2=0x7fb9400d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.260+0000 7fb948790640 1 -- 172.21.15.45:0/497413037 shutdown_connections 2024-05-08T05:45:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.260+0000 7fb948790640 1 -- 172.21.15.45:0/497413037 wait complete. 2024-05-08T05:45:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.261+0000 7fb948790640 1 Processor -- start 2024-05-08T05:45:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.261+0000 7fb948790640 1 -- start start 2024-05-08T05:45:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.262+0000 7fb948790640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb928003a50 0x7fb940154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.262+0000 7fb948790640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9400a1900 0x7fb9400b9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.262+0000 7fb948790640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb94009ed00 0x7fb94009f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.262+0000 7fb948790640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb9400cf440 con 0x7fb94009ed00 2024-05-08T05:45:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.262+0000 7fb948790640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb9400cf2c0 con 0x7fb9400a1900 2024-05-08T05:45:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.262+0000 7fb948790640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb9400d2dc0 con 0x7fb928003a50 2024-05-08T05:45:32.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.262+0000 7fb946d2d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb94009ed00 0x7fb94009f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.262+0000 7fb946d2d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb94009ed00 0x7fb94009f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45348/0 (socket says 172.21.15.45:45348) 2024-05-08T05:45:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.262+0000 7fb946d2d640 1 -- 172.21.15.45:0/1138900263 learned_addr learned my addr 172.21.15.45:0/1138900263 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.262+0000 7fb94652c640 1 --2- 172.21.15.45:0/1138900263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb928003a50 0x7fb940154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.262+0000 7fb945d2b640 1 --2- 172.21.15.45:0/1138900263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9400a1900 0x7fb9400b9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.262+0000 7fb946d2d640 1 -- 172.21.15.45:0/1138900263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb928003a50 msgr2=0x7fb940154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.262+0000 7fb946d2d640 1 --2- 172.21.15.45:0/1138900263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb928003a50 0x7fb940154470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.262+0000 7fb946d2d640 1 -- 172.21.15.45:0/1138900263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9400a1900 msgr2=0x7fb9400b9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.262+0000 7fb946d2d640 1 --2- 172.21.15.45:0/1138900263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9400a1900 0x7fb9400b9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.262+0000 7fb946d2d640 1 -- 172.21.15.45:0/1138900263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9400a1e00 con 0x7fb94009ed00 2024-05-08T05:45:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.262+0000 7fb945d2b640 1 --2- 172.21.15.45:0/1138900263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9400a1900 0x7fb9400b9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.263+0000 7fb946d2d640 1 --2- 172.21.15.45:0/1138900263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb94009ed00 0x7fb94009f190 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb938000c00 tx=0x7fb93800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.263+0000 7fb93effd640 1 -- 172.21.15.45:0/1138900263 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938010510 con 0x7fb94009ed00 2024-05-08T05:45:32.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.263+0000 7fb93effd640 1 -- 172.21.15.45:0/1138900263 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9380106b0 con 0x7fb94009ed00 2024-05-08T05:45:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.263+0000 7fb93effd640 1 -- 172.21.15.45:0/1138900263 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938010a60 con 0x7fb94009ed00 2024-05-08T05:45:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.263+0000 7fb948790640 1 -- 172.21.15.45:0/1138900263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9400a20e0 con 0x7fb94009ed00 2024-05-08T05:45:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.263+0000 7fb948790640 1 -- 172.21.15.45:0/1138900263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9400a26a0 con 0x7fb94009ed00 2024-05-08T05:45:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.264+0000 7fb948790640 1 -- 172.21.15.45:0/1138900263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9400693f0 con 0x7fb94009ed00 2024-05-08T05:45:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.264+0000 7fb93effd640 1 -- 172.21.15.45:0/1138900263 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb938010c00 con 0x7fb94009ed00 2024-05-08T05:45:32.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.265+0000 7fb93effd640 1 --2- 172.21.15.45:0/1138900263 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb92c076300 0x7fb92c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.265+0000 7fb93effd640 1 -- 172.21.15.45:0/1138900263 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7fb938096c80 con 0x7fb94009ed00 2024-05-08T05:45:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.265+0000 7fb94652c640 1 --2- 172.21.15.45:0/1138900263 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb92c076300 0x7fb92c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:32.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.266+0000 7fb94652c640 1 --2- 172.21.15.45:0/1138900263 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb92c076300 0x7fb92c0787c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb930032000 tx=0x7fb93002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:32.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.270+0000 7fb93effd640 1 -- 172.21.15.45:0/1138900263 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb938060450 con 0x7fb94009ed00 2024-05-08T05:45:32.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.473+0000 7fb948790640 1 -- 172.21.15.45:0/1138900263 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb9400d3ab0 con 0x7fb94009ed00 2024-05-08T05:45:32.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.473+0000 7fb93effd640 1 -- 172.21.15.45:0/1138900263 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v75) ==== 80+0+0 (secure 0 0 0) 0x7fb93805fd50 con 0x7fb94009ed00 2024-05-08T05:45:32.474 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-05-08T05:45:32.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.475+0000 7fb948790640 1 -- 172.21.15.45:0/1138900263 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb92c076300 msgr2=0x7fb92c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:32.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.475+0000 7fb948790640 1 --2- 172.21.15.45:0/1138900263 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb92c076300 0x7fb92c0787c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb930032000 tx=0x7fb93002e000 comp rx=0 tx=0).stop 2024-05-08T05:45:32.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.475+0000 7fb948790640 1 -- 172.21.15.45:0/1138900263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb94009ed00 msgr2=0x7fb94009f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:32.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.475+0000 7fb948790640 1 --2- 172.21.15.45:0/1138900263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb94009ed00 0x7fb94009f190 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb938000c00 tx=0x7fb93800e960 comp rx=0 tx=0).stop 2024-05-08T05:45:32.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.475+0000 7fb948790640 1 -- 172.21.15.45:0/1138900263 shutdown_connections 2024-05-08T05:45:32.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.475+0000 7fb948790640 1 --2- 172.21.15.45:0/1138900263 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb92c076300 0x7fb92c0787c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:32.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.475+0000 7fb948790640 1 --2- 172.21.15.45:0/1138900263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb94009ed00 0x7fb94009f190 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:32.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.475+0000 7fb948790640 1 --2- 172.21.15.45:0/1138900263 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9400a1900 0x7fb9400b9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:32.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.475+0000 7fb948790640 1 --2- 172.21.15.45:0/1138900263 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb928003a50 0x7fb940154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:32.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.475+0000 7fb948790640 1 -- 172.21.15.45:0/1138900263 >> 172.21.15.45:0/1138900263 conn(0x7fb9400d4fe0 msgr2=0x7fb9400ca940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:32.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.475+0000 7fb948790640 1 -- 172.21.15.45:0/1138900263 shutdown_connections 2024-05-08T05:45:32.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.475+0000 7fb948790640 1 -- 172.21.15.45:0/1138900263 wait complete. 2024-05-08T05:45:32.486 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T05:45:32.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:32.487 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T05:45:32.557 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:45:32.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:32.557 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T05:45:32.576 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:45:32.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.720+0000 7f16ddb67640 1 Processor -- start 2024-05-08T05:45:32.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.720+0000 7f16ddb67640 1 -- start start 2024-05-08T05:45:32.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.720+0000 7f16ddb67640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16d815d160 0x7f16d81615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:32.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.720+0000 7f16ddb67640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16d80ae270 0x7f16d8143dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:32.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.720+0000 7f16ddb67640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16d80b6340 con 0x7f16d80b2ce0 2024-05-08T05:45:32.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.720+0000 7f16ddb67640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f16d80b5e10 con 0x7f16d815d160 2024-05-08T05:45:32.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.720+0000 7f16ddb67640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f16d8161b00 con 0x7f16d80ae270 2024-05-08T05:45:32.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.721+0000 7f16d77fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16d815d160 0x7f16d81615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:32.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.721+0000 7f16d7fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f16d80b2ce0 0x7f16d80b4c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50560/0 (socket says 172.21.15.45:50560) 2024-05-08T05:45:32.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.721+0000 7f16d77fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16d815d160 0x7f16d81615c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45350/0 (socket says 172.21.15.45:45350) 2024-05-08T05:45:32.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.721+0000 7f16d7fff640 1 -- 172.21.15.45:0/2008400984 learned_addr learned my addr 172.21.15.45:0/2008400984 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:32.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.721+0000 7f16d6ffd640 1 --2- 172.21.15.45:0/2008400984 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16d80ae270 0x7f16d8143dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.721+0000 7f16d77fe640 1 -- 172.21.15.45:0/2008400984 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16d80ae270 msgr2=0x7f16d8143dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.721+0000 7f16d77fe640 1 --2- 172.21.15.45:0/2008400984 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16d80ae270 0x7f16d8143dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.721+0000 7f16d77fe640 1 -- 172.21.15.45:0/2008400984 >> v1:172.21.15.45:6789/0 conn(0x7f16d80b2ce0 legacy=0x7f16d80b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.721+0000 7f16d77fe640 1 -- 172.21.15.45:0/2008400984 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d80b6060 con 0x7f16d815d160 2024-05-08T05:45:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.721+0000 7f16d67fc640 1 -- 172.21.15.45:0/2008400984 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4220255562 0 0) 0x7f16d80b6340 con 0x7f16d80b2ce0 2024-05-08T05:45:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.721+0000 7f16d6ffd640 1 --2- 172.21.15.45:0/2008400984 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16d80ae270 0x7f16d8143dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.722+0000 7f16d77fe640 1 --2- 172.21.15.45:0/2008400984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16d815d160 0x7f16d81615c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f16c80097a0 tx=0x7f16c802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ec6ffc5ba9f41a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.722+0000 7f16d67fc640 1 -- 172.21.15.45:0/2008400984 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8037070 con 0x7f16d815d160 2024-05-08T05:45:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.722+0000 7f16d67fc640 1 -- 172.21.15.45:0/2008400984 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16c802ae50 con 0x7f16d815d160 2024-05-08T05:45:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.722+0000 7f16d67fc640 1 -- 172.21.15.45:0/2008400984 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8032660 con 0x7f16d815d160 2024-05-08T05:45:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.722+0000 7f16ddb67640 1 -- 172.21.15.45:0/2008400984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16d815d160 msgr2=0x7f16d81615c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:32.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.722+0000 7f16ddb67640 1 --2- 172.21.15.45:0/2008400984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16d815d160 0x7f16d81615c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f16c80097a0 tx=0x7f16c802a100 comp rx=0 tx=0).stop 2024-05-08T05:45:32.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.722+0000 7f16ddb67640 1 -- 172.21.15.45:0/2008400984 shutdown_connections 2024-05-08T05:45:32.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.722+0000 7f16ddb67640 1 --2- 172.21.15.45:0/2008400984 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16d80ae270 0x7f16d8143dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:32.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.722+0000 7f16ddb67640 1 --2- 172.21.15.45:0/2008400984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16d815d160 0x7f16d81615c0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:32.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.722+0000 7f16ddb67640 1 -- 172.21.15.45:0/2008400984 >> 172.21.15.45:0/2008400984 conn(0x7f16d80b6570 msgr2=0x7f16d80b6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:32.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.723+0000 7f16ddb67640 1 -- 172.21.15.45:0/2008400984 shutdown_connections 2024-05-08T05:45:32.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.723+0000 7f16ddb67640 1 -- 172.21.15.45:0/2008400984 wait complete. 2024-05-08T05:45:32.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.724+0000 7f16ddb67640 1 Processor -- start 2024-05-08T05:45:32.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.724+0000 7f16ddb67640 1 -- start start 2024-05-08T05:45:32.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.725+0000 7f16ddb67640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16d80ae270 0x7f16d81f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:32.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.725+0000 7f16ddb67640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16d80b2ce0 0x7f16d815a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:32.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.725+0000 7f16ddb67640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16d815d160 0x7f16d815ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:32.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.725+0000 7f16ddb67640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16d80b4740 con 0x7f16d815d160 2024-05-08T05:45:32.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.725+0000 7f16ddb67640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f16d80b4460 con 0x7f16d80b2ce0 2024-05-08T05:45:32.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.725+0000 7f16ddb67640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f16d8144310 con 0x7f16d80ae270 2024-05-08T05:45:32.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.725+0000 7f16d7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16d815d160 0x7f16d815ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:32.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.725+0000 7f16d7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16d815d160 0x7f16d815ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45366/0 (socket says 172.21.15.45:45366) 2024-05-08T05:45:32.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.725+0000 7f16d77fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16d80ae270 0x7f16d81f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:32.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.725+0000 7f16d7fff640 1 -- 172.21.15.45:0/2364231610 learned_addr learned my addr 172.21.15.45:0/2364231610 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:32.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.725+0000 7f16d6ffd640 1 --2- 172.21.15.45:0/2364231610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16d80b2ce0 0x7f16d815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:32.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.725+0000 7f16d7fff640 1 -- 172.21.15.45:0/2364231610 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16d80ae270 msgr2=0x7f16d81f4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:32.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.725+0000 7f16d7fff640 1 --2- 172.21.15.45:0/2364231610 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16d80ae270 0x7f16d81f4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.725+0000 7f16d7fff640 1 -- 172.21.15.45:0/2364231610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16d80b2ce0 msgr2=0x7f16d815a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.725+0000 7f16d7fff640 1 --2- 172.21.15.45:0/2364231610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16d80b2ce0 0x7f16d815a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.725+0000 7f16d7fff640 1 -- 172.21.15.45:0/2364231610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d815b390 con 0x7f16d815d160 2024-05-08T05:45:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.725+0000 7f16d7fff640 1 --2- 172.21.15.45:0/2364231610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16d815d160 0x7f16d815ae50 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f16d000c7a0 tx=0x7f16d000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.725+0000 7f16bffff640 1 -- 172.21.15.45:0/2364231610 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0018070 con 0x7f16d815d160 2024-05-08T05:45:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.726+0000 7f16bffff640 1 -- 172.21.15.45:0/2364231610 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16d000a5e0 con 0x7f16d815d160 2024-05-08T05:45:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.726+0000 7f16ddb67640 1 -- 172.21.15.45:0/2364231610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d815b640 con 0x7f16d815d160 2024-05-08T05:45:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.726+0000 7f16bffff640 1 -- 172.21.15.45:0/2364231610 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d000aa10 con 0x7f16d815d160 2024-05-08T05:45:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.726+0000 7f16ddb67640 1 -- 172.21.15.45:0/2364231610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d815bee0 con 0x7f16d815d160 2024-05-08T05:45:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.726+0000 7f16ddb67640 1 -- 172.21.15.45:0/2364231610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d80aed00 con 0x7f16d815d160 2024-05-08T05:45:32.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.727+0000 7f16bffff640 1 -- 172.21.15.45:0/2364231610 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f16d001b070 con 0x7f16d815d160 2024-05-08T05:45:32.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.728+0000 7f16bffff640 1 --2- 172.21.15.45:0/2364231610 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f16c40762d0 0x7f16c4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:32.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.728+0000 7f16bffff640 1 -- 172.21.15.45:0/2364231610 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f16d0096c80 con 0x7f16d815d160 2024-05-08T05:45:32.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.728+0000 7f16d77fe640 1 --2- 172.21.15.45:0/2364231610 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f16c40762d0 0x7f16c4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:32.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.729+0000 7f16d77fe640 1 --2- 172.21.15.45:0/2364231610 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f16c40762d0 0x7f16c4078790 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f16c8034000 tx=0x7f16c802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:32.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.732+0000 7f16bffff640 1 -- 172.21.15.45:0/2364231610 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f16d80aed00 con 0x7f16d815d160 2024-05-08T05:45:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.969+0000 7f16ddb67640 1 -- 172.21.15.45:0/2364231610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16d8069b00 con 0x7f16d815d160 2024-05-08T05:45:32.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.969+0000 7f16bffff640 1 -- 172.21.15.45:0/2364231610 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+5180 (secure 0 0 0) 0x7f16d005fe00 con 0x7f16d815d160 2024-05-08T05:45:32.970 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:32.970 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5722,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1283757699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1283757699},{"type":"v1","addr":"172.21.15.63:6837","nonce":1283757699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1963062796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1963062796},{"type":"v1","addr":"172.21.15.45:6834","nonce":1963062796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6171,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4183668301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4183668301},{"type":"v1","addr":"172.21.15.45:6835","nonce":4183668301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:45:24.208158+0000","modified":"2024-05-08T05:45:27.211503+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6180},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6180":{"gid":6180,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6836/133050464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":133050464},{"type":"v1","addr":"172.21.15.63:6836","nonce":133050464}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6180,"qdb_cluster":[6180]},"id":5}]} 2024-05-08T05:45:32.970 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 44 2024-05-08T05:45:32.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.971+0000 7f16ddb67640 1 -- 172.21.15.45:0/2364231610 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f16c40762d0 msgr2=0x7f16c4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:32.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.971+0000 7f16ddb67640 1 --2- 172.21.15.45:0/2364231610 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f16c40762d0 0x7f16c4078790 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f16c8034000 tx=0x7f16c802e000 comp rx=0 tx=0).stop 2024-05-08T05:45:32.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.972+0000 7f16ddb67640 1 -- 172.21.15.45:0/2364231610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16d815d160 msgr2=0x7f16d815ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:32.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.972+0000 7f16ddb67640 1 --2- 172.21.15.45:0/2364231610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16d815d160 0x7f16d815ae50 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f16d000c7a0 tx=0x7f16d000cc60 comp rx=0 tx=0).stop 2024-05-08T05:45:32.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.972+0000 7f16ddb67640 1 -- 172.21.15.45:0/2364231610 shutdown_connections 2024-05-08T05:45:32.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.972+0000 7f16ddb67640 1 --2- 172.21.15.45:0/2364231610 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f16c40762d0 0x7f16c4078790 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:32.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.972+0000 7f16ddb67640 1 --2- 172.21.15.45:0/2364231610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16d815d160 0x7f16d815ae50 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:32.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.972+0000 7f16ddb67640 1 --2- 172.21.15.45:0/2364231610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16d80b2ce0 0x7f16d815a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:32.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.972+0000 7f16ddb67640 1 --2- 172.21.15.45:0/2364231610 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16d80ae270 0x7f16d81f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:32.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.972+0000 7f16ddb67640 1 -- 172.21.15.45:0/2364231610 >> 172.21.15.45:0/2364231610 conn(0x7f16d80b6570 msgr2=0x7f16d8145830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:32.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.972+0000 7f16ddb67640 1 -- 172.21.15.45:0/2364231610 shutdown_connections 2024-05-08T05:45:32.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:32.972+0000 7f16ddb67640 1 -- 172.21.15.45:0/2364231610 wait complete. 2024-05-08T05:45:32.983 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-05-08T05:45:33.126 INFO:teuthology.orchestra.run.smithi063.stdout:{"message":"","return_code":0} 2024-05-08T05:45:33.127 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-08T05:45:33.127 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T05:45:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.233+0000 7f5ba9cd1640 1 Processor -- start 2024-05-08T05:45:33.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.233+0000 7f5ba9cd1640 1 -- start start 2024-05-08T05:45:33.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.234+0000 7f5ba9cd1640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5ba40b2490 0x7f5ba4070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.234+0000 7f5ba9cd1640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ba406e110 con 0x7f5ba41629b0 2024-05-08T05:45:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.234+0000 7f5ba9cd1640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ba406e3f0 con 0x7f5ba4070d20 2024-05-08T05:45:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.234+0000 7f5ba9cd1640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5ba406dec0 con 0x7f5ba40b2490 2024-05-08T05:45:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.234+0000 7f5ba37fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5ba41629b0 0x7f5ba406eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50564/0 (socket says 172.21.15.45:50564) 2024-05-08T05:45:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.234+0000 7f5ba37fe640 1 -- 172.21.15.45:0/2061640124 learned_addr learned my addr 172.21.15.45:0/2061640124 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.234+0000 7f5ba2ffd640 1 --2- 172.21.15.45:0/2061640124 >> v2:172.21.15.63:3300/0 conn(0x7f5ba40b2490 0x7f5ba4070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.234+0000 7f5ba27fc640 1 -- 172.21.15.45:0/2061640124 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 90027962 0 0) 0x7f5ba406e110 con 0x7f5ba41629b0 2024-05-08T05:45:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.234+0000 7f5ba27fc640 1 -- 172.21.15.45:0/2061640124 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b90003520 con 0x7f5ba41629b0 2024-05-08T05:45:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.234+0000 7f5ba27fc640 1 -- 172.21.15.45:0/2061640124 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4169485098 0 0) 0x7f5b90003520 con 0x7f5ba41629b0 2024-05-08T05:45:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.234+0000 7f5ba2ffd640 1 -- 172.21.15.45:0/2061640124 >> v1:172.21.15.63:6789/0 conn(0x7f5ba4070d20 legacy=0x7f5ba40afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.234+0000 7f5ba2ffd640 1 -- 172.21.15.45:0/2061640124 >> v1:172.21.15.45:6789/0 conn(0x7f5ba41629b0 legacy=0x7f5ba406eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:33.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.234+0000 7f5ba2ffd640 1 -- 172.21.15.45:0/2061640124 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba406e6d0 con 0x7f5ba40b2490 2024-05-08T05:45:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.235+0000 7f5ba2ffd640 1 --2- 172.21.15.45:0/2061640124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ba40b2490 0x7f5ba4070330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0097a0 tx=0x7f5b8c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5601df1751b4e21d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.235+0000 7f5ba27fc640 1 -- 172.21.15.45:0/2061640124 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c037070 con 0x7f5ba40b2490 2024-05-08T05:45:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.235+0000 7f5ba27fc640 1 -- 172.21.15.45:0/2061640124 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b8c02ac50 con 0x7f5ba40b2490 2024-05-08T05:45:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.235+0000 7f5ba27fc640 1 -- 172.21.15.45:0/2061640124 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b940016f0 con 0x7f5ba40b2490 2024-05-08T05:45:33.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.236+0000 7f5ba9cd1640 1 -- 172.21.15.45:0/2061640124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ba40b2490 msgr2=0x7f5ba4070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.236+0000 7f5ba9cd1640 1 --2- 172.21.15.45:0/2061640124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ba40b2490 0x7f5ba4070330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0097a0 tx=0x7f5b8c02a1c0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.236+0000 7f5ba9cd1640 1 -- 172.21.15.45:0/2061640124 shutdown_connections 2024-05-08T05:45:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.236+0000 7f5ba9cd1640 1 --2- 172.21.15.45:0/2061640124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ba40b2490 0x7f5ba4070330 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.236+0000 7f5ba9cd1640 1 -- 172.21.15.45:0/2061640124 >> 172.21.15.45:0/2061640124 conn(0x7f5ba40b49d0 msgr2=0x7f5ba40b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.236+0000 7f5ba9cd1640 1 -- 172.21.15.45:0/2061640124 shutdown_connections 2024-05-08T05:45:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.236+0000 7f5ba9cd1640 1 -- 172.21.15.45:0/2061640124 wait complete. 2024-05-08T05:45:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.237+0000 7f5ba9cd1640 1 Processor -- start 2024-05-08T05:45:33.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.237+0000 7f5ba9cd1640 1 -- start start 2024-05-08T05:45:33.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.238+0000 7f5ba9cd1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ba4070d20 0x7f5ba40cd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.238+0000 7f5ba9cd1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ba40b2490 0x7f5ba41f7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.238+0000 7f5ba9cd1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ba41629b0 0x7f5ba41f7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.238+0000 7f5ba9cd1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba40709f0 con 0x7f5ba4070d20 2024-05-08T05:45:33.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.238+0000 7f5ba9cd1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba4070870 con 0x7f5ba41629b0 2024-05-08T05:45:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.238+0000 7f5ba9cd1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba40b03c0 con 0x7f5ba40b2490 2024-05-08T05:45:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.238+0000 7f5ba37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ba4070d20 0x7f5ba40cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.238+0000 7f5ba37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ba4070d20 0x7f5ba40cd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45368/0 (socket says 172.21.15.45:45368) 2024-05-08T05:45:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.238+0000 7f5ba37fe640 1 -- 172.21.15.45:0/158676750 learned_addr learned my addr 172.21.15.45:0/158676750 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.238+0000 7f5ba3fff640 1 --2- 172.21.15.45:0/158676750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ba41629b0 0x7f5ba41f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.238+0000 7f5ba2ffd640 1 --2- 172.21.15.45:0/158676750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ba40b2490 0x7f5ba41f7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.238+0000 7f5ba37fe640 1 -- 172.21.15.45:0/158676750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ba40b2490 msgr2=0x7f5ba41f7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.238+0000 7f5ba37fe640 1 --2- 172.21.15.45:0/158676750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ba40b2490 0x7f5ba41f7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.238+0000 7f5ba37fe640 1 -- 172.21.15.45:0/158676750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ba41629b0 msgr2=0x7f5ba41f7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.238+0000 7f5ba37fe640 1 --2- 172.21.15.45:0/158676750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ba41629b0 0x7f5ba41f7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.238+0000 7f5ba37fe640 1 -- 172.21.15.45:0/158676750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba40cf9b0 con 0x7f5ba4070d20 2024-05-08T05:45:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.238+0000 7f5ba2ffd640 1 --2- 172.21.15.45:0/158676750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ba40b2490 0x7f5ba41f7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:33.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.239+0000 7f5ba37fe640 1 --2- 172.21.15.45:0/158676750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ba4070d20 0x7f5ba40cd450 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400c680 tx=0x7f5b9400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:33.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.239+0000 7f5b9bfff640 1 -- 172.21.15.45:0/158676750 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b94005ad0 con 0x7f5ba4070d20 2024-05-08T05:45:33.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.239+0000 7f5b9bfff640 1 -- 172.21.15.45:0/158676750 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b94005c90 con 0x7f5ba4070d20 2024-05-08T05:45:33.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.239+0000 7f5ba9cd1640 1 -- 172.21.15.45:0/158676750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba40cfc90 con 0x7f5ba4070d20 2024-05-08T05:45:33.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.239+0000 7f5b9bfff640 1 -- 172.21.15.45:0/158676750 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b940072b0 con 0x7f5ba4070d20 2024-05-08T05:45:33.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.239+0000 7f5ba9cd1640 1 -- 172.21.15.45:0/158676750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba40d0250 con 0x7f5ba4070d20 2024-05-08T05:45:33.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.240+0000 7f5ba9cd1640 1 -- 172.21.15.45:0/158676750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ba40693f0 con 0x7f5ba4070d20 2024-05-08T05:45:33.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.241+0000 7f5b9bfff640 1 -- 172.21.15.45:0/158676750 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5b94007450 con 0x7f5ba4070d20 2024-05-08T05:45:33.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.241+0000 7f5b9bfff640 1 --2- 172.21.15.45:0/158676750 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5b840762d0 0x7f5b84078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.241+0000 7f5b9bfff640 1 -- 172.21.15.45:0/158676750 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6101+0+0 (secure 0 0 0) 0x7f5b94097400 con 0x7f5ba4070d20 2024-05-08T05:45:33.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.242+0000 7f5ba2ffd640 1 --2- 172.21.15.45:0/158676750 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5b840762d0 0x7f5b84078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.242+0000 7f5ba2ffd640 1 --2- 172.21.15.45:0/158676750 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5b840762d0 0x7f5b84078790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c034000 tx=0x7f5b8c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:33.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.246+0000 7f5b9bfff640 1 -- 172.21.15.45:0/158676750 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5b94060c60 con 0x7f5ba4070d20 2024-05-08T05:45:33.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.479+0000 7f5ba9cd1640 1 -- 172.21.15.45:0/158676750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5ba4076460 con 0x7f5ba4070d20 2024-05-08T05:45:33.480 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-05-08T05:45:33.480+0000 7f124de12640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T05:45:33.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.481+0000 7f5b9bfff640 1 -- 172.21.15.45:0/158676750 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) ==== 161+0+0 (secure 0 0 0) 0x7f5b94060560 con 0x7f5ba4070d20 2024-05-08T05:45:33.481 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T05:45:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.483+0000 7f5ba9cd1640 1 -- 172.21.15.45:0/158676750 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5b840762d0 msgr2=0x7f5b84078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.483+0000 7f5ba9cd1640 1 --2- 172.21.15.45:0/158676750 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5b840762d0 0x7f5b84078790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c034000 tx=0x7f5b8c02e000 comp rx=0 tx=0).stop 2024-05-08T05:45:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.483+0000 7f5ba9cd1640 1 -- 172.21.15.45:0/158676750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ba4070d20 msgr2=0x7f5ba40cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.483+0000 7f5ba9cd1640 1 --2- 172.21.15.45:0/158676750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ba4070d20 0x7f5ba40cd450 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5b9400c680 tx=0x7f5b9400cb40 comp rx=0 tx=0).stop 2024-05-08T05:45:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.483+0000 7f5ba9cd1640 1 -- 172.21.15.45:0/158676750 shutdown_connections 2024-05-08T05:45:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.483+0000 7f5ba9cd1640 1 --2- 172.21.15.45:0/158676750 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5b840762d0 0x7f5b84078790 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.483+0000 7f5ba9cd1640 1 --2- 172.21.15.45:0/158676750 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ba40b2490 0x7f5ba41f7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.483+0000 7f5ba9cd1640 1 --2- 172.21.15.45:0/158676750 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ba41629b0 0x7f5ba41f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.483+0000 7f5ba9cd1640 1 --2- 172.21.15.45:0/158676750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ba4070d20 0x7f5ba40cd450 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.483+0000 7f5ba9cd1640 1 -- 172.21.15.45:0/158676750 >> 172.21.15.45:0/158676750 conn(0x7f5ba40b49d0 msgr2=0x7f5ba40711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:33.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.484+0000 7f5ba9cd1640 1 -- 172.21.15.45:0/158676750 shutdown_connections 2024-05-08T05:45:33.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.484+0000 7f5ba9cd1640 1 -- 172.21.15.45:0/158676750 wait complete. 2024-05-08T05:45:33.487 INFO:tasks.ceph.mds.b.smithi063.stderr: -4463> 2024-05-08T05:45:30.464+0000 7f0e4868c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:45:33.495 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-05-08T05:45:33.522 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-08T05:45:33.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.542+0000 7faf51366f40 1 Processor -- start 2024-05-08T05:45:33.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.542+0000 7faf51366f40 1 -- start start 2024-05-08T05:45:33.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.542+0000 7faf51366f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aec55c60b0 0x55aec55ef090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.542+0000 7faf51366f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aec569b7c0 0x55aec569e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.542+0000 7faf51366f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aec569bba0 0x55aec5695c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.542+0000 7faf51366f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55aec55c6af0 con 0x55aec569bba0 2024-05-08T05:45:33.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.542+0000 7faf51366f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55aec55c6810 con 0x55aec55c60b0 2024-05-08T05:45:33.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.542+0000 7faf51366f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55aec560d670 con 0x55aec569b7c0 2024-05-08T05:45:33.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.542+0000 7faf4f661640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aec569bba0 0x55aec5695c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.542+0000 7faf4f661640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aec569bba0 0x55aec5695c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45384/0 (socket says 172.21.15.45:45384) 2024-05-08T05:45:33.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.542+0000 7faf4f661640 1 -- 172.21.15.45:0/218170543 learned_addr learned my addr 172.21.15.45:0/218170543 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:33.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.543+0000 7faf4ee60640 1 --2- 172.21.15.45:0/218170543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aec55c60b0 0x55aec55ef090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.543+0000 7faf4e65f640 1 --2- 172.21.15.45:0/218170543 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aec569b7c0 0x55aec569e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.543+0000 7faf4f661640 1 -- 172.21.15.45:0/218170543 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aec569b7c0 msgr2=0x55aec569e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:33.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.543+0000 7faf4f661640 1 --2- 172.21.15.45:0/218170543 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aec569b7c0 0x55aec569e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.543+0000 7faf4f661640 1 -- 172.21.15.45:0/218170543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aec55c60b0 msgr2=0x55aec55ef090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:33.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.543+0000 7faf4f661640 1 --2- 172.21.15.45:0/218170543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aec55c60b0 0x55aec55ef090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.543+0000 7faf4f661640 1 -- 172.21.15.45:0/218170543 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aec552d350 con 0x55aec569bba0 2024-05-08T05:45:33.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.543+0000 7faf4e65f640 1 --2- 172.21.15.45:0/218170543 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aec569b7c0 0x55aec569e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:33.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.543+0000 7faf4f661640 1 --2- 172.21.15.45:0/218170543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aec569bba0 0x55aec5695c30 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7faf48009540 tx=0x7faf48009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975d81409a2fd23 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:33.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.543+0000 7faf4d65d640 1 -- 172.21.15.45:0/218170543 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48017070 con 0x55aec569bba0 2024-05-08T05:45:33.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.543+0000 7faf4d65d640 1 -- 172.21.15.45:0/218170543 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf48006600 con 0x55aec569bba0 2024-05-08T05:45:33.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.543+0000 7faf4d65d640 1 -- 172.21.15.45:0/218170543 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf480069f0 con 0x55aec569bba0 2024-05-08T05:45:33.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.543+0000 7faf51366f40 1 -- 172.21.15.45:0/218170543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aec569bba0 msgr2=0x55aec5695c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:33.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.543+0000 7faf51366f40 1 --2- 172.21.15.45:0/218170543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aec569bba0 0x55aec5695c30 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7faf48009540 tx=0x7faf48009a00 comp rx=0 tx=0).stop 2024-05-08T05:45:33.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.544+0000 7faf51366f40 1 -- 172.21.15.45:0/218170543 shutdown_connections 2024-05-08T05:45:33.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.544+0000 7faf51366f40 1 --2- 172.21.15.45:0/218170543 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aec569b7c0 0x55aec569e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.544+0000 7faf51366f40 1 --2- 172.21.15.45:0/218170543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aec569bba0 0x55aec5695c30 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.544+0000 7faf51366f40 1 --2- 172.21.15.45:0/218170543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aec55c60b0 0x55aec55ef090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.544+0000 7faf51366f40 1 -- 172.21.15.45:0/218170543 >> 172.21.15.45:0/218170543 conn(0x55aec55f1b50 msgr2=0x55aec569bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:33.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.544+0000 7faf51366f40 1 -- 172.21.15.45:0/218170543 shutdown_connections 2024-05-08T05:45:33.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.544+0000 7faf51366f40 1 -- 172.21.15.45:0/218170543 wait complete. 2024-05-08T05:45:33.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.544+0000 7faf51366f40 1 Processor -- start 2024-05-08T05:45:33.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.544+0000 7faf51366f40 1 -- start start 2024-05-08T05:45:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.545+0000 7faf51366f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aec55c60b0 0x55aec569cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.545+0000 7faf51366f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aec569b7c0 0x55aec56974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.545+0000 7faf51366f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aec569bba0 0x55aec5697a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.545+0000 7faf51366f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55aec5693a10 con 0x55aec569bba0 2024-05-08T05:45:33.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.545+0000 7faf51366f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55aec56937c0 con 0x55aec569b7c0 2024-05-08T05:45:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.545+0000 7faf51366f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55aec569a850 con 0x55aec55c60b0 2024-05-08T05:45:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.545+0000 7faf4f661640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aec569bba0 0x55aec5697a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.545+0000 7faf4f661640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aec569bba0 0x55aec5697a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45394/0 (socket says 172.21.15.45:45394) 2024-05-08T05:45:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.545+0000 7faf4f661640 1 -- 172.21.15.45:0/1366943036 learned_addr learned my addr 172.21.15.45:0/1366943036 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.545+0000 7faf4e65f640 1 --2- 172.21.15.45:0/1366943036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aec569b7c0 0x55aec56974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.545+0000 7faf4ee60640 1 --2- 172.21.15.45:0/1366943036 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aec55c60b0 0x55aec569cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.545+0000 7faf4f661640 1 -- 172.21.15.45:0/1366943036 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aec55c60b0 msgr2=0x55aec569cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.545+0000 7faf4f661640 1 --2- 172.21.15.45:0/1366943036 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aec55c60b0 0x55aec569cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.545+0000 7faf4f661640 1 -- 172.21.15.45:0/1366943036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aec569b7c0 msgr2=0x55aec56974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:33.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.545+0000 7faf4f661640 1 --2- 172.21.15.45:0/1366943036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aec569b7c0 0x55aec56974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.545+0000 7faf4f661640 1 -- 172.21.15.45:0/1366943036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aec5698700 con 0x55aec569bba0 2024-05-08T05:45:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.545+0000 7faf4f661640 1 --2- 172.21.15.45:0/1366943036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aec569bba0 0x55aec5697a10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7faf48009fd0 tx=0x7faf48003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.546+0000 7faf37fff640 1 -- 172.21.15.45:0/1366943036 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48021020 con 0x55aec569bba0 2024-05-08T05:45:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.546+0000 7faf37fff640 1 -- 172.21.15.45:0/1366943036 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf48018040 con 0x55aec569bba0 2024-05-08T05:45:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.546+0000 7faf37fff640 1 -- 172.21.15.45:0/1366943036 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48017070 con 0x55aec569bba0 2024-05-08T05:45:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.546+0000 7faf51366f40 1 -- 172.21.15.45:0/1366943036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55aec560f5f0 con 0x55aec569bba0 2024-05-08T05:45:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.546+0000 7faf51366f40 1 -- 172.21.15.45:0/1366943036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aec5692510 con 0x55aec569bba0 2024-05-08T05:45:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.547+0000 7faf37fff640 1 -- 172.21.15.45:0/1366943036 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7faf48024020 con 0x55aec569bba0 2024-05-08T05:45:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.548+0000 7faf37fff640 1 --2- 172.21.15.45:0/1366943036 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faf3c074cb0 0x7faf3c077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.548+0000 7faf37fff640 1 -- 172.21.15.45:0/1366943036 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7faf48097290 con 0x55aec569bba0 2024-05-08T05:45:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.548+0000 7faf4ee60640 1 --2- 172.21.15.45:0/1366943036 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faf3c074cb0 0x7faf3c077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.548+0000 7faf51366f40 1 --2- 172.21.15.45:0/1366943036 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x55aec569d180 0x55aec5609be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.548+0000 7faf51366f40 1 -- 172.21.15.45:0/1366943036 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76) -- 0x55aec5692fd0 con 0x55aec569d180 2024-05-08T05:45:33.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.548+0000 7faf4e65f640 1 --2- 172.21.15.45:0/1366943036 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x55aec569d180 0x55aec5609be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.549+0000 7faf4e65f640 1 --2- 172.21.15.45:0/1366943036 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x55aec569d180 0x55aec5609be0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:33.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.549+0000 7faf4ee60640 1 --2- 172.21.15.45:0/1366943036 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faf3c074cb0 0x7faf3c077170 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faf40056000 tx=0x7faf40007030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.550+0000 7faf4e65f640 1 -- 172.21.15.45:0/1366943036 <== osd.0 v2:172.21.15.45:6800/3734454393 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v76'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7faf440076b0 con 0x55aec569d180 2024-05-08T05:45:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.550+0000 7faf51366f40 1 -- 172.21.15.45:0/1366943036 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x55aec569d180 msgr2=0x55aec5609be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.550+0000 7faf51366f40 1 --2- 172.21.15.45:0/1366943036 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x55aec569d180 0x55aec5609be0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.550+0000 7faf51366f40 1 -- 172.21.15.45:0/1366943036 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faf3c074cb0 msgr2=0x7faf3c077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:33.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.550+0000 7faf51366f40 1 --2- 172.21.15.45:0/1366943036 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7faf3c074cb0 0x7faf3c077170 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faf40056000 tx=0x7faf40007030 comp rx=0 tx=0).stop 2024-05-08T05:45:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.550+0000 7faf51366f40 1 -- 172.21.15.45:0/1366943036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aec569bba0 msgr2=0x55aec5697a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.550+0000 7faf51366f40 1 --2- 172.21.15.45:0/1366943036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aec569bba0 0x55aec5697a10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7faf48009fd0 tx=0x7faf48003020 comp rx=0 tx=0).stop 2024-05-08T05:45:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.550+0000 7faf4f661640 1 -- 172.21.15.45:0/1366943036 reap_dead start 2024-05-08T05:45:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.550+0000 7faf51366f40 1 -- 172.21.15.45:0/1366943036 shutdown_connections 2024-05-08T05:45:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.550+0000 7faf51366f40 1 -- 172.21.15.45:0/1366943036 >> 172.21.15.45:0/1366943036 conn(0x55aec55f1b50 msgr2=0x55aec569bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.550+0000 7faf51366f40 1 -- 172.21.15.45:0/1366943036 shutdown_connections 2024-05-08T05:45:33.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.550+0000 7faf51366f40 1 -- 172.21.15.45:0/1366943036 wait complete. 2024-05-08T05:45:33.554 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-08T05:45:33.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.660+0000 7f79f7445640 1 Processor -- start 2024-05-08T05:45:33.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.661+0000 7f79f7445640 1 -- start start 2024-05-08T05:45:33.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.661+0000 7f79f7445640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f79f015f380 0x7f79f00d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.661+0000 7f79f7445640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79f00b3f30 0x7f79f00d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.661+0000 7f79f7445640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79f00b1c90 con 0x7f79f00b2420 2024-05-08T05:45:33.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.661+0000 7f79f7445640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f79f00d0a50 con 0x7f79f015f380 2024-05-08T05:45:33.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.661+0000 7f79f7445640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f79f00d0d30 con 0x7f79f00b3f30 2024-05-08T05:45:33.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.661+0000 7f79f59e2640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f79f00b2420 0x7f79f00b2870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50578/0 (socket says 172.21.15.45:50578) 2024-05-08T05:45:33.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79f59e2640 1 -- 172.21.15.45:0/4068029848 learned_addr learned my addr 172.21.15.45:0/4068029848 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:33.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79f51e1640 1 --2- 172.21.15.45:0/4068029848 >> v2:172.21.15.63:3300/0 conn(0x7f79f015f380 0x7f79f00d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79f49e0640 1 --2- 172.21.15.45:0/4068029848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79f00b3f30 0x7f79f00d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79effff640 1 -- 172.21.15.45:0/4068029848 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3524266308 0 0) 0x7f79f00b1c90 con 0x7f79f00b2420 2024-05-08T05:45:33.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79effff640 1 -- 172.21.15.45:0/4068029848 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79d8003520 con 0x7f79f00b2420 2024-05-08T05:45:33.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79effff640 1 -- 172.21.15.45:0/4068029848 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3393068050 0 0) 0x7f79d8003520 con 0x7f79f00b2420 2024-05-08T05:45:33.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79effff640 1 -- 172.21.15.45:0/4068029848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79f00b3f30 msgr2=0x7f79f00d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:33.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79effff640 1 --2- 172.21.15.45:0/4068029848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79f00b3f30 0x7f79f00d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79effff640 1 -- 172.21.15.45:0/4068029848 >> v2:172.21.15.63:3300/0 conn(0x7f79f015f380 msgr2=0x7f79f00d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:33.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79effff640 1 --2- 172.21.15.45:0/4068029848 >> v2:172.21.15.63:3300/0 conn(0x7f79f015f380 0x7f79f00d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79effff640 1 -- 172.21.15.45:0/4068029848 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f00b1f00 con 0x7f79f00b2420 2024-05-08T05:45:33.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79effff640 1 -- 172.21.15.45:0/4068029848 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f79e0001a70 con 0x7f79f00b2420 2024-05-08T05:45:33.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79effff640 1 -- 172.21.15.45:0/4068029848 >> v1:172.21.15.45:6789/0 conn(0x7f79f00b2420 legacy=0x7f79f00b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:33.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79effff640 1 --2- 172.21.15.45:0/4068029848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79d8003a00 0x7f79d8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79effff640 1 --2- 172.21.15.45:0/4068029848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79f00b3f30 0x7f79d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79effff640 1 --2- 172.21.15.45:0/4068029848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79d8004d40 0x7f79d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79effff640 1 -- 172.21.15.45:0/4068029848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f79f00d0a50 con 0x7f79d8004d40 2024-05-08T05:45:33.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79effff640 1 -- 172.21.15.45:0/4068029848 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f79f00d0d30 con 0x7f79d8003a00 2024-05-08T05:45:33.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79effff640 1 -- 172.21.15.45:0/4068029848 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f79d800b9f0 con 0x7f79f00b3f30 2024-05-08T05:45:33.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79f59e2640 1 --2- 172.21.15.45:0/4068029848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79d8004d40 0x7f79d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79f51e1640 1 --2- 172.21.15.45:0/4068029848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79d8003a00 0x7f79d8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.662+0000 7f79f49e0640 1 --2- 172.21.15.45:0/4068029848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79f00b3f30 0x7f79d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.663+0000 7f79f59e2640 1 -- 172.21.15.45:0/4068029848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79f00b3f30 msgr2=0x7f79d8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:33.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.663+0000 7f79f59e2640 1 --2- 172.21.15.45:0/4068029848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79f00b3f30 0x7f79d8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.663+0000 7f79f59e2640 1 -- 172.21.15.45:0/4068029848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79d8003a00 msgr2=0x7f79d8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:33.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.663+0000 7f79f59e2640 1 --2- 172.21.15.45:0/4068029848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79d8003a00 0x7f79d8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.663+0000 7f79f59e2640 1 -- 172.21.15.45:0/4068029848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d8003520 con 0x7f79d8004d40 2024-05-08T05:45:33.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.663+0000 7f79f59e2640 1 --2- 172.21.15.45:0/4068029848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79d8004d40 0x7f79d80051c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f79e000a8a0 tx=0x7f79d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4bd4f707e6344c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:33.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.663+0000 7f79effff640 1 -- 172.21.15.45:0/4068029848 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e000a4c0 con 0x7f79d8004d40 2024-05-08T05:45:33.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.663+0000 7f79effff640 1 -- 172.21.15.45:0/4068029848 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e002f070 con 0x7f79d8004d40 2024-05-08T05:45:33.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.663+0000 7f79effff640 1 -- 172.21.15.45:0/4068029848 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e002bc80 con 0x7f79d8004d40 2024-05-08T05:45:33.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.664+0000 7f79f7445640 1 -- 172.21.15.45:0/4068029848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79d8004d40 msgr2=0x7f79d80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:33.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.664+0000 7f79f7445640 1 --2- 172.21.15.45:0/4068029848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79d8004d40 0x7f79d80051c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f79e000a8a0 tx=0x7f79d8005830 comp rx=0 tx=0).stop 2024-05-08T05:45:33.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.664+0000 7f79f59e2640 1 -- 172.21.15.45:0/4068029848 reap_dead start 2024-05-08T05:45:33.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.664+0000 7f79f7445640 1 -- 172.21.15.45:0/4068029848 shutdown_connections 2024-05-08T05:45:33.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.664+0000 7f79f7445640 1 -- 172.21.15.45:0/4068029848 >> 172.21.15.45:0/4068029848 conn(0x7f79f00d4fe0 msgr2=0x7f79f00d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:33.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.664+0000 7f79f7445640 1 -- 172.21.15.45:0/4068029848 shutdown_connections 2024-05-08T05:45:33.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.664+0000 7f79f7445640 1 -- 172.21.15.45:0/4068029848 wait complete. 2024-05-08T05:45:33.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.665+0000 7f79f7445640 1 Processor -- start 2024-05-08T05:45:33.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.665+0000 7f79f7445640 1 -- start start 2024-05-08T05:45:33.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.666+0000 7f79f7445640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79d8004d40 0x7f79f0155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.666+0000 7f79f7445640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79f0162380 0x7f79f00b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.666+0000 7f79f7445640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79f00b4bc0 0x7f79f01f8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.666+0000 7f79f7445640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f79f00d06e0 con 0x7f79d8004d40 2024-05-08T05:45:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.666+0000 7f79f7445640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f79f00d1010 con 0x7f79f00b4bc0 2024-05-08T05:45:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.666+0000 7f79f7445640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f79f00d2600 con 0x7f79f0162380 2024-05-08T05:45:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.666+0000 7f79f51e1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79d8004d40 0x7f79f0155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.666+0000 7f79f51e1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79d8004d40 0x7f79f0155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45410/0 (socket says 172.21.15.45:45410) 2024-05-08T05:45:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.666+0000 7f79f51e1640 1 -- 172.21.15.45:0/3715653692 learned_addr learned my addr 172.21.15.45:0/3715653692 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.666+0000 7f79f49e0640 1 --2- 172.21.15.45:0/3715653692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79f0162380 0x7f79f00b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.666+0000 7f79f59e2640 1 --2- 172.21.15.45:0/3715653692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79f00b4bc0 0x7f79f01f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.666+0000 7f79f51e1640 1 -- 172.21.15.45:0/3715653692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79f0162380 msgr2=0x7f79f00b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.666+0000 7f79f51e1640 1 --2- 172.21.15.45:0/3715653692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79f0162380 0x7f79f00b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.667+0000 7f79f51e1640 1 -- 172.21.15.45:0/3715653692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79f00b4bc0 msgr2=0x7f79f01f8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:33.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.667+0000 7f79f51e1640 1 --2- 172.21.15.45:0/3715653692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79f00b4bc0 0x7f79f01f8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.667+0000 7f79f51e1640 1 -- 172.21.15.45:0/3715653692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f015fd30 con 0x7f79d8004d40 2024-05-08T05:45:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.667+0000 7f79f51e1640 1 --2- 172.21.15.45:0/3715653692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79d8004d40 0x7f79f0155e10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f79e8011960 tx=0x7f79e8011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.667+0000 7f79edffb640 1 -- 172.21.15.45:0/3715653692 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e800fa90 con 0x7f79d8004d40 2024-05-08T05:45:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.667+0000 7f79edffb640 1 -- 172.21.15.45:0/3715653692 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e800fc30 con 0x7f79d8004d40 2024-05-08T05:45:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.667+0000 7f79f7445640 1 -- 172.21.15.45:0/3715653692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f0160010 con 0x7f79d8004d40 2024-05-08T05:45:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.667+0000 7f79edffb640 1 -- 172.21.15.45:0/3715653692 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e8014440 con 0x7f79d8004d40 2024-05-08T05:45:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.667+0000 7f79f7445640 1 -- 172.21.15.45:0/3715653692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f01605d0 con 0x7f79d8004d40 2024-05-08T05:45:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.668+0000 7f79f7445640 1 -- 172.21.15.45:0/3715653692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f00b4fd0 con 0x7f79d8004d40 2024-05-08T05:45:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.669+0000 7f79edffb640 1 -- 172.21.15.45:0/3715653692 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f79e8012040 con 0x7f79d8004d40 2024-05-08T05:45:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.669+0000 7f79edffb640 1 --2- 172.21.15.45:0/3715653692 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f79dc0762d0 0x7f79dc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.670+0000 7f79edffb640 1 -- 172.21.15.45:0/3715653692 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f79e8097d90 con 0x7f79d8004d40 2024-05-08T05:45:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.670+0000 7f79f49e0640 1 --2- 172.21.15.45:0/3715653692 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f79dc0762d0 0x7f79dc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:33.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.670+0000 7f79f49e0640 1 --2- 172.21.15.45:0/3715653692 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f79dc0762d0 0x7f79dc078790 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f79f0162030 tx=0x7f79e4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:33.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.674+0000 7f79edffb640 1 -- 172.21.15.45:0/3715653692 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f79f00b4fd0 con 0x7f79d8004d40 2024-05-08T05:45:33.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:33.905+0000 7f79f7445640 1 -- 172.21.15.45:0/3715653692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f79f00693f0 con 0x7f79d8004d40 2024-05-08T05:45:34.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.482+0000 7f79edffb640 1 -- 172.21.15.45:0/3715653692 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v46) ==== 162+0+0 (secure 0 0 0) 0x7f79e8019030 con 0x7f79d8004d40 2024-05-08T05:45:34.483 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-08T05:45:34.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.485+0000 7f79f7445640 1 -- 172.21.15.45:0/3715653692 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f79dc0762d0 msgr2=0x7f79dc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.485+0000 7f79f7445640 1 --2- 172.21.15.45:0/3715653692 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f79dc0762d0 0x7f79dc078790 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f79f0162030 tx=0x7f79e4008040 comp rx=0 tx=0).stop 2024-05-08T05:45:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.485+0000 7f79f7445640 1 -- 172.21.15.45:0/3715653692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79d8004d40 msgr2=0x7f79f0155e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.485+0000 7f79f7445640 1 --2- 172.21.15.45:0/3715653692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79d8004d40 0x7f79f0155e10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f79e8011960 tx=0x7f79e8011e20 comp rx=0 tx=0).stop 2024-05-08T05:45:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.485+0000 7f79f7445640 1 -- 172.21.15.45:0/3715653692 shutdown_connections 2024-05-08T05:45:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.485+0000 7f79f7445640 1 --2- 172.21.15.45:0/3715653692 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f79dc0762d0 0x7f79dc078790 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.485+0000 7f79f7445640 1 --2- 172.21.15.45:0/3715653692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f79f0162380 0x7f79f00b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.485+0000 7f79f7445640 1 --2- 172.21.15.45:0/3715653692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f79f00b4bc0 0x7f79f01f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.485+0000 7f79f7445640 1 --2- 172.21.15.45:0/3715653692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79d8004d40 0x7f79f0155e10 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.485+0000 7f79f7445640 1 -- 172.21.15.45:0/3715653692 >> 172.21.15.45:0/3715653692 conn(0x7f79f00d4fe0 msgr2=0x7f79f00b62c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.485+0000 7f79f7445640 1 -- 172.21.15.45:0/3715653692 shutdown_connections 2024-05-08T05:45:34.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.485+0000 7f79f7445640 1 -- 172.21.15.45:0/3715653692 wait complete. 2024-05-08T05:45:34.501 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:45:34.548 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:45:34.551 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:45:34.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.612+0000 7f63653e5640 1 Processor -- start 2024-05-08T05:45:34.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.612+0000 7f63653e5640 1 -- start start 2024-05-08T05:45:34.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.613+0000 7f63653e5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f636015f380 0x7f63600d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:34.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.613+0000 7f63653e5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f63600d0cf0 0x7f63600d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:34.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.613+0000 7f63653e5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f63600d3b00 0x7f6360162e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:34.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.613+0000 7f63653e5640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f63600d30f0 con 0x7f63600d0cf0 2024-05-08T05:45:34.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.613+0000 7f63653e5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f63600d1150 con 0x7f63600d3b00 2024-05-08T05:45:34.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.613+0000 7f63653e5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f63600d2370 con 0x7f636015f380 2024-05-08T05:45:34.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.613+0000 7f635e7fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f63600d0cf0 0x7f63600d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:34.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.613+0000 7f635e7fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f63600d0cf0 0x7f63600d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45420/0 (socket says 172.21.15.45:45420) 2024-05-08T05:45:34.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.613+0000 7f635e7fc640 1 -- 172.21.15.45:0/3444559248 learned_addr learned my addr 172.21.15.45:0/3444559248 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:34.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.613+0000 7f635f7fe640 1 --2- 172.21.15.45:0/3444559248 >> v2:172.21.15.63:3300/0 conn(0x7f63600d3b00 0x7f6360162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:34.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.613+0000 7f635effd640 1 --2- 172.21.15.45:0/3444559248 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f636015f380 0x7f63600d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:34.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.613+0000 7f635e7fc640 1 -- 172.21.15.45:0/3444559248 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f636015f380 msgr2=0x7f63600d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:34.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.613+0000 7f635e7fc640 1 --2- 172.21.15.45:0/3444559248 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f636015f380 0x7f63600d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:34.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.613+0000 7f635e7fc640 1 -- 172.21.15.45:0/3444559248 >> v2:172.21.15.63:3300/0 conn(0x7f63600d3b00 msgr2=0x7f6360162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:34.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.613+0000 7f635e7fc640 1 --2- 172.21.15.45:0/3444559248 >> v2:172.21.15.63:3300/0 conn(0x7f63600d3b00 0x7f6360162e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:34.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.613+0000 7f635e7fc640 1 -- 172.21.15.45:0/3444559248 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63600d1760 con 0x7f63600d0cf0 2024-05-08T05:45:34.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.613+0000 7f635f7fe640 1 --2- 172.21.15.45:0/3444559248 >> v2:172.21.15.63:3300/0 conn(0x7f63600d3b00 0x7f6360162e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:34.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.613+0000 7f635effd640 1 --2- 172.21.15.45:0/3444559248 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f636015f380 0x7f63600d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:34.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.614+0000 7f635e7fc640 1 --2- 172.21.15.45:0/3444559248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63600d0cf0 0x7f63600d35c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f636008f120 tx=0x7f635402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce74c025ec16613f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:34.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.614+0000 7f635dffb640 1 -- 172.21.15.45:0/3444559248 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354037070 con 0x7f63600d0cf0 2024-05-08T05:45:34.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.614+0000 7f635dffb640 1 -- 172.21.15.45:0/3444559248 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6354032440 con 0x7f63600d0cf0 2024-05-08T05:45:34.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.614+0000 7f635dffb640 1 -- 172.21.15.45:0/3444559248 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354032810 con 0x7f63600d0cf0 2024-05-08T05:45:34.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.615+0000 7f63653e5640 1 -- 172.21.15.45:0/3444559248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63600d0cf0 msgr2=0x7f63600d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:34.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.615+0000 7f63653e5640 1 --2- 172.21.15.45:0/3444559248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63600d0cf0 0x7f63600d35c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f636008f120 tx=0x7f635402a1e0 comp rx=0 tx=0).stop 2024-05-08T05:45:34.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.615+0000 7f63653e5640 1 -- 172.21.15.45:0/3444559248 shutdown_connections 2024-05-08T05:45:34.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.615+0000 7f63653e5640 1 --2- 172.21.15.45:0/3444559248 >> v2:172.21.15.63:3300/0 conn(0x7f63600d3b00 0x7f6360162e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:34.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.615+0000 7f63653e5640 1 --2- 172.21.15.45:0/3444559248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63600d0cf0 0x7f63600d35c0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:34.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.615+0000 7f63653e5640 1 --2- 172.21.15.45:0/3444559248 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f636015f380 0x7f63600d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:34.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.615+0000 7f63653e5640 1 -- 172.21.15.45:0/3444559248 >> 172.21.15.45:0/3444559248 conn(0x7f63600d4fe0 msgr2=0x7f63600d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:34.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.615+0000 7f63653e5640 1 -- 172.21.15.45:0/3444559248 shutdown_connections 2024-05-08T05:45:34.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.615+0000 7f63653e5640 1 -- 172.21.15.45:0/3444559248 wait complete. 2024-05-08T05:45:34.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.616+0000 7f63653e5640 1 Processor -- start 2024-05-08T05:45:34.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.616+0000 7f63653e5640 1 -- start start 2024-05-08T05:45:34.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.617+0000 7f63653e5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63600d0cf0 0x7f6360157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:34.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.617+0000 7f63653e5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63600d3b00 0x7f63600b81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:34.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.617+0000 7f63653e5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f636015f380 0x7f63600b86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:34.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.617+0000 7f63653e5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f63600d2000 con 0x7f63600d0cf0 2024-05-08T05:45:34.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.617+0000 7f63653e5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f63600d1d20 con 0x7f636015f380 2024-05-08T05:45:34.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.617+0000 7f63653e5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f636015f760 con 0x7f63600d3b00 2024-05-08T05:45:34.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.617+0000 7f635effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63600d0cf0 0x7f6360157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:34.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.617+0000 7f635effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63600d0cf0 0x7f6360157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45426/0 (socket says 172.21.15.45:45426) 2024-05-08T05:45:34.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.617+0000 7f635effd640 1 -- 172.21.15.45:0/676964605 learned_addr learned my addr 172.21.15.45:0/676964605 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:34.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.617+0000 7f635e7fc640 1 --2- 172.21.15.45:0/676964605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63600d3b00 0x7f63600b81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:34.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.617+0000 7f635f7fe640 1 --2- 172.21.15.45:0/676964605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f636015f380 0x7f63600b86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:34.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.617+0000 7f635effd640 1 -- 172.21.15.45:0/676964605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63600d3b00 msgr2=0x7f63600b81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:34.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.617+0000 7f635effd640 1 --2- 172.21.15.45:0/676964605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63600d3b00 0x7f63600b81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:34.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.617+0000 7f635effd640 1 -- 172.21.15.45:0/676964605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f636015f380 msgr2=0x7f63600b86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:34.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.617+0000 7f635effd640 1 --2- 172.21.15.45:0/676964605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f636015f380 0x7f63600b86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:34.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.618+0000 7f635effd640 1 -- 172.21.15.45:0/676964605 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63600cdb20 con 0x7f63600d0cf0 2024-05-08T05:45:34.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.618+0000 7f635effd640 1 --2- 172.21.15.45:0/676964605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63600d0cf0 0x7f6360157b80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f635000d7d0 tx=0x7f635000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:34.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.618+0000 7f63477fe640 1 -- 172.21.15.45:0/676964605 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350004030 con 0x7f63600d0cf0 2024-05-08T05:45:34.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.618+0000 7f63477fe640 1 -- 172.21.15.45:0/676964605 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63500041d0 con 0x7f63600d0cf0 2024-05-08T05:45:34.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.618+0000 7f63653e5640 1 -- 172.21.15.45:0/676964605 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63600cde00 con 0x7f63600d0cf0 2024-05-08T05:45:34.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.618+0000 7f63477fe640 1 -- 172.21.15.45:0/676964605 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63500096a0 con 0x7f63600d0cf0 2024-05-08T05:45:34.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.618+0000 7f63653e5640 1 -- 172.21.15.45:0/676964605 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63600ce3c0 con 0x7f63600d0cf0 2024-05-08T05:45:34.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.619+0000 7f63653e5640 1 -- 172.21.15.45:0/676964605 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63600d4310 con 0x7f63600d0cf0 2024-05-08T05:45:34.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.619+0000 7f63477fe640 1 -- 172.21.15.45:0/676964605 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6350009840 con 0x7f63600d0cf0 2024-05-08T05:45:34.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.620+0000 7f63477fe640 1 --2- 172.21.15.45:0/676964605 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f634c0762d0 0x7f634c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:34.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.620+0000 7f63477fe640 1 -- 172.21.15.45:0/676964605 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f6350097940 con 0x7f63600d0cf0 2024-05-08T05:45:34.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.620+0000 7f635e7fc640 1 --2- 172.21.15.45:0/676964605 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f634c0762d0 0x7f634c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:34.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.621+0000 7f635e7fc640 1 --2- 172.21.15.45:0/676964605 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f634c0762d0 0x7f634c078790 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6354034000 tx=0x7f635402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:34.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.625+0000 7f63477fe640 1 -- 172.21.15.45:0/676964605 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f63600d4310 con 0x7f63600d0cf0 2024-05-08T05:45:34.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.858+0000 7f63653e5640 1 -- 172.21.15.45:0/676964605 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63600693f0 con 0x7f63600d0cf0 2024-05-08T05:45:34.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.858+0000 7f63477fe640 1 -- 172.21.15.45:0/676964605 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5177 (secure 0 0 0) 0x7f6350060a70 con 0x7f63600d0cf0 2024-05-08T05:45:34.859 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:34.859 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5722,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1283757699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1283757699},{"type":"v1","addr":"172.21.15.63:6837","nonce":1283757699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2823770504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2823770504},{"type":"v1","addr":"172.21.15.63:6836","nonce":2823770504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1963062796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1963062796},{"type":"v1","addr":"172.21.15.45:6834","nonce":1963062796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:45:24.208158+0000","modified":"2024-05-08T05:45:34.486054+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":76,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":47,"state":"up:replay","state_seq":1,"addr":"172.21.15.45:6835/4183668301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4183668301},{"type":"v1","addr":"172.21.15.45:6835","nonce":4183668301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-05-08T05:45:34.859 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 47 2024-05-08T05:45:34.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.860+0000 7f63653e5640 1 -- 172.21.15.45:0/676964605 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f634c0762d0 msgr2=0x7f634c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:34.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.860+0000 7f63653e5640 1 --2- 172.21.15.45:0/676964605 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f634c0762d0 0x7f634c078790 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6354034000 tx=0x7f635402e000 comp rx=0 tx=0).stop 2024-05-08T05:45:34.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.860+0000 7f63653e5640 1 -- 172.21.15.45:0/676964605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63600d0cf0 msgr2=0x7f6360157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:34.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.860+0000 7f63653e5640 1 --2- 172.21.15.45:0/676964605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63600d0cf0 0x7f6360157b80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f635000d7d0 tx=0x7f635000dc90 comp rx=0 tx=0).stop 2024-05-08T05:45:34.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.860+0000 7f63653e5640 1 -- 172.21.15.45:0/676964605 shutdown_connections 2024-05-08T05:45:34.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.860+0000 7f63653e5640 1 --2- 172.21.15.45:0/676964605 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f634c0762d0 0x7f634c078790 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:34.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.860+0000 7f63653e5640 1 --2- 172.21.15.45:0/676964605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63600d3b00 0x7f63600b81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:34.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.860+0000 7f63653e5640 1 --2- 172.21.15.45:0/676964605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f636015f380 0x7f63600b86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:34.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.860+0000 7f63653e5640 1 --2- 172.21.15.45:0/676964605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63600d0cf0 0x7f6360157b80 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:34.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.860+0000 7f63653e5640 1 -- 172.21.15.45:0/676964605 >> 172.21.15.45:0/676964605 conn(0x7f63600d4fe0 msgr2=0x7f63600cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:34.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.860+0000 7f63653e5640 1 -- 172.21.15.45:0/676964605 shutdown_connections 2024-05-08T05:45:34.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:34.860+0000 7f63653e5640 1 -- 172.21.15.45:0/676964605 wait complete. 2024-05-08T05:45:35.275 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:45:24.208158+0000', 'modified': '2024-05-08T05:45:34.486054+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 76, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6171}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6171': {'gid': 6171, 'name': 'a', 'rank': 0, 'incarnation': 47, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.45:6835/4183668301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 4183668301}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 4183668301}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-08T05:45:35.275 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6171:up:replay 2024-05-08T05:45:36.276 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:45:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.394+0000 7f60a6ecc640 1 Processor -- start 2024-05-08T05:45:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.394+0000 7f60a6ecc640 1 -- start start 2024-05-08T05:45:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.394+0000 7f60a6ecc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f60a015f420 0x7f60a00d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.394+0000 7f60a6ecc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f60a00d3ca0 0x7f60a00d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.394+0000 7f60a6ecc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60a00d1eb0 0x7f60a00d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.394+0000 7f60a6ecc640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f60a01630b0 con 0x7f60a015f420 2024-05-08T05:45:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.394+0000 7f60a6ecc640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f60a00d2550 con 0x7f60a00d3ca0 2024-05-08T05:45:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.394+0000 7f60a6ecc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f60a00d2c80 con 0x7f60a00d1eb0 2024-05-08T05:45:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.395+0000 7f60a4c68640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f60a015f420 0x7f60a00d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.395+0000 7f60a4c68640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f60a015f420 0x7f60a00d3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45434/0 (socket says 172.21.15.45:45434) 2024-05-08T05:45:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.395+0000 7f60a4c68640 1 -- 172.21.15.45:0/3250369067 learned_addr learned my addr 172.21.15.45:0/3250369067 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.395+0000 7f60a5469640 1 --2- 172.21.15.45:0/3250369067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60a00d1eb0 0x7f60a00d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.395+0000 7f609ffff640 1 --2- 172.21.15.45:0/3250369067 >> v2:172.21.15.63:3300/0 conn(0x7f60a00d3ca0 0x7f60a00d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.395+0000 7f60a4c68640 1 -- 172.21.15.45:0/3250369067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60a00d1eb0 msgr2=0x7f60a00d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.395+0000 7f60a4c68640 1 --2- 172.21.15.45:0/3250369067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60a00d1eb0 0x7f60a00d4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.395+0000 7f60a4c68640 1 -- 172.21.15.45:0/3250369067 >> v2:172.21.15.63:3300/0 conn(0x7f60a00d3ca0 msgr2=0x7f60a00d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.395+0000 7f60a4c68640 1 --2- 172.21.15.45:0/3250369067 >> v2:172.21.15.63:3300/0 conn(0x7f60a00d3ca0 0x7f60a00d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.395+0000 7f60a4c68640 1 -- 172.21.15.45:0/3250369067 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a00d2f60 con 0x7f60a015f420 2024-05-08T05:45:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.395+0000 7f609ffff640 1 --2- 172.21.15.45:0/3250369067 >> v2:172.21.15.63:3300/0 conn(0x7f60a00d3ca0 0x7f60a00d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.396+0000 7f60a4c68640 1 --2- 172.21.15.45:0/3250369067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60a015f420 0x7f60a00d3760 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f60a008f380 tx=0x7f609802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946cd9b21adf673e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.396+0000 7f609f7fe640 1 -- 172.21.15.45:0/3250369067 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6098036070 con 0x7f60a015f420 2024-05-08T05:45:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.396+0000 7f609f7fe640 1 -- 172.21.15.45:0/3250369067 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6098031600 con 0x7f60a015f420 2024-05-08T05:45:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.396+0000 7f609f7fe640 1 -- 172.21.15.45:0/3250369067 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60980319d0 con 0x7f60a015f420 2024-05-08T05:45:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.396+0000 7f60a6ecc640 1 -- 172.21.15.45:0/3250369067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60a015f420 msgr2=0x7f60a00d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.396+0000 7f60a6ecc640 1 --2- 172.21.15.45:0/3250369067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60a015f420 0x7f60a00d3760 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f60a008f380 tx=0x7f609802a6a0 comp rx=0 tx=0).stop 2024-05-08T05:45:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.397+0000 7f60a6ecc640 1 -- 172.21.15.45:0/3250369067 shutdown_connections 2024-05-08T05:45:36.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.397+0000 7f60a6ecc640 1 --2- 172.21.15.45:0/3250369067 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60a00d1eb0 0x7f60a00d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.397+0000 7f60a6ecc640 1 --2- 172.21.15.45:0/3250369067 >> v2:172.21.15.63:3300/0 conn(0x7f60a00d3ca0 0x7f60a00d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.397+0000 7f60a6ecc640 1 --2- 172.21.15.45:0/3250369067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60a015f420 0x7f60a00d3760 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.397+0000 7f60a6ecc640 1 -- 172.21.15.45:0/3250369067 >> 172.21.15.45:0/3250369067 conn(0x7f60a008eac0 msgr2=0x7f60a009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.397+0000 7f60a6ecc640 1 -- 172.21.15.45:0/3250369067 shutdown_connections 2024-05-08T05:45:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.397+0000 7f60a6ecc640 1 -- 172.21.15.45:0/3250369067 wait complete. 2024-05-08T05:45:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.398+0000 7f60a6ecc640 1 Processor -- start 2024-05-08T05:45:36.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.398+0000 7f60a6ecc640 1 -- start start 2024-05-08T05:45:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.399+0000 7f60a6ecc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60a00d1eb0 0x7f60a00c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.399+0000 7f60a6ecc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60a00c8d30 0x7f60a00c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.399+0000 7f60a6ecc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60a00cad60 0x7f60a00cb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.399+0000 7f60a6ecc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f60a00d5200 con 0x7f60a00d1eb0 2024-05-08T05:45:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.399+0000 7f60a6ecc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60a00d2830 con 0x7f60a00c8d30 2024-05-08T05:45:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.399+0000 7f60a6ecc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f60a00d5380 con 0x7f60a00cad60 2024-05-08T05:45:36.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.399+0000 7f60a4c68640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60a00d1eb0 0x7f60a00c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.399+0000 7f60a4c68640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60a00d1eb0 0x7f60a00c7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45446/0 (socket says 172.21.15.45:45446) 2024-05-08T05:45:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.399+0000 7f60a4c68640 1 -- 172.21.15.45:0/2212304683 learned_addr learned my addr 172.21.15.45:0/2212304683 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.399+0000 7f609ffff640 1 --2- 172.21.15.45:0/2212304683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60a00c8d30 0x7f60a00c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.399+0000 7f60a5469640 1 --2- 172.21.15.45:0/2212304683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60a00cad60 0x7f60a00cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.399+0000 7f60a4c68640 1 -- 172.21.15.45:0/2212304683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60a00cad60 msgr2=0x7f60a00cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.399+0000 7f60a4c68640 1 --2- 172.21.15.45:0/2212304683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60a00cad60 0x7f60a00cb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.399+0000 7f60a4c68640 1 -- 172.21.15.45:0/2212304683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60a00c8d30 msgr2=0x7f60a00c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.399+0000 7f60a4c68640 1 --2- 172.21.15.45:0/2212304683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60a00c8d30 0x7f60a00c9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.399+0000 7f60a4c68640 1 -- 172.21.15.45:0/2212304683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a014b770 con 0x7f60a00d1eb0 2024-05-08T05:45:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.400+0000 7f60a4c68640 1 --2- 172.21.15.45:0/2212304683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60a00d1eb0 0x7f60a00c7a30 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6098009f40 tx=0x7f6098004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.400+0000 7f609d7fa640 1 -- 172.21.15.45:0/2212304683 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609803f020 con 0x7f60a00d1eb0 2024-05-08T05:45:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.400+0000 7f609d7fa640 1 -- 172.21.15.45:0/2212304683 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6098037040 con 0x7f60a00d1eb0 2024-05-08T05:45:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.400+0000 7f60a6ecc640 1 -- 172.21.15.45:0/2212304683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a014ba30 con 0x7f60a00d1eb0 2024-05-08T05:45:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.400+0000 7f609d7fa640 1 -- 172.21.15.45:0/2212304683 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6098036070 con 0x7f60a00d1eb0 2024-05-08T05:45:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.400+0000 7f60a6ecc640 1 -- 172.21.15.45:0/2212304683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a014bff0 con 0x7f60a00d1eb0 2024-05-08T05:45:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.401+0000 7f60a6ecc640 1 -- 172.21.15.45:0/2212304683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a00693f0 con 0x7f60a00d1eb0 2024-05-08T05:45:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.402+0000 7f609d7fa640 1 -- 172.21.15.45:0/2212304683 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6098042020 con 0x7f60a00d1eb0 2024-05-08T05:45:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.402+0000 7f609d7fa640 1 --2- 172.21.15.45:0/2212304683 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f608c076300 0x7f608c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.402+0000 7f609d7fa640 1 -- 172.21.15.45:0/2212304683 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f60980b7bb0 con 0x7f60a00d1eb0 2024-05-08T05:45:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.403+0000 7f609ffff640 1 --2- 172.21.15.45:0/2212304683 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f608c076300 0x7f608c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.403+0000 7f609ffff640 1 --2- 172.21.15.45:0/2212304683 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f608c076300 0x7f608c0787c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f60a01495b0 tx=0x7f6094008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:36.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.407+0000 7f609d7fa640 1 -- 172.21.15.45:0/2212304683 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6098049030 con 0x7f60a00d1eb0 2024-05-08T05:45:36.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.638+0000 7f60a6ecc640 1 -- 172.21.15.45:0/2212304683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60a00b4c50 con 0x7f60a00d1eb0 2024-05-08T05:45:36.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.638+0000 7f609d7fa640 1 -- 172.21.15.45:0/2212304683 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5177 (secure 0 0 0) 0x7f6098080ce0 con 0x7f60a00d1eb0 2024-05-08T05:45:36.639 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:36.639 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5722,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1283757699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1283757699},{"type":"v1","addr":"172.21.15.63:6837","nonce":1283757699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2823770504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2823770504},{"type":"v1","addr":"172.21.15.63:6836","nonce":2823770504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1963062796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1963062796},{"type":"v1","addr":"172.21.15.45:6834","nonce":1963062796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:45:24.208158+0000","modified":"2024-05-08T05:45:35.491473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":76,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":47,"state":"up:rejoin","state_seq":6,"addr":"172.21.15.45:6835/4183668301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4183668301},{"type":"v1","addr":"172.21.15.45:6835","nonce":4183668301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-05-08T05:45:36.639 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 49 2024-05-08T05:45:36.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.640+0000 7f60a6ecc640 1 -- 172.21.15.45:0/2212304683 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f608c076300 msgr2=0x7f608c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:36.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.640+0000 7f60a6ecc640 1 --2- 172.21.15.45:0/2212304683 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f608c076300 0x7f608c0787c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f60a01495b0 tx=0x7f6094008040 comp rx=0 tx=0).stop 2024-05-08T05:45:36.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.640+0000 7f60a6ecc640 1 -- 172.21.15.45:0/2212304683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60a00d1eb0 msgr2=0x7f60a00c7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:36.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.640+0000 7f60a6ecc640 1 --2- 172.21.15.45:0/2212304683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60a00d1eb0 0x7f60a00c7a30 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6098009f40 tx=0x7f6098004a40 comp rx=0 tx=0).stop 2024-05-08T05:45:36.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.640+0000 7f60a6ecc640 1 -- 172.21.15.45:0/2212304683 shutdown_connections 2024-05-08T05:45:36.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.640+0000 7f60a6ecc640 1 --2- 172.21.15.45:0/2212304683 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f608c076300 0x7f608c0787c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:36.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.640+0000 7f60a6ecc640 1 --2- 172.21.15.45:0/2212304683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f60a00cad60 0x7f60a00cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:36.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.640+0000 7f60a6ecc640 1 --2- 172.21.15.45:0/2212304683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60a00c8d30 0x7f60a00c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:36.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.640+0000 7f60a6ecc640 1 --2- 172.21.15.45:0/2212304683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60a00d1eb0 0x7f60a00c7a30 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:36.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.640+0000 7f60a6ecc640 1 -- 172.21.15.45:0/2212304683 >> 172.21.15.45:0/2212304683 conn(0x7f60a008eac0 msgr2=0x7f60a00d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:36.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.640+0000 7f60a6ecc640 1 -- 172.21.15.45:0/2212304683 shutdown_connections 2024-05-08T05:45:36.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:36.640+0000 7f60a6ecc640 1 -- 172.21.15.45:0/2212304683 wait complete. 2024-05-08T05:45:37.054 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:45:24.208158+0000', 'modified': '2024-05-08T05:45:35.491473+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 76, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6171}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6171': {'gid': 6171, 'name': 'a', 'rank': 0, 'incarnation': 47, 'state': 'up:rejoin', 'state_seq': 6, 'addr': '172.21.15.45:6835/4183668301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 4183668301}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 4183668301}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-08T05:45:37.054 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6171:up:rejoin 2024-05-08T05:45:38.054 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:45:38.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.171+0000 7f3c89262640 1 Processor -- start 2024-05-08T05:45:38.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.171+0000 7f3c89262640 1 -- start start 2024-05-08T05:45:38.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.172+0000 7f3c89262640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c840d20c0 0x7f3c84163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:38.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.172+0000 7f3c89262640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3c84071110 0x7f3c8406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:38.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.172+0000 7f3c89262640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c840d1300 con 0x7f3c84070d30 2024-05-08T05:45:38.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.172+0000 7f3c89262640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f3c840d10d0 con 0x7f3c84071110 2024-05-08T05:45:38.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.172+0000 7f3c89262640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3c84070b80 con 0x7f3c840d20c0 2024-05-08T05:45:38.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.172+0000 7f3c837fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3c84071110 0x7f3c8406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:38.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.172+0000 7f3c837fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3c84071110 0x7f3c8406d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45452/0 (socket says 172.21.15.45:45452) 2024-05-08T05:45:38.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.172+0000 7f3c837fe640 1 -- 172.21.15.45:0/2480021350 learned_addr learned my addr 172.21.15.45:0/2480021350 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:38.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.172+0000 7f3c827fc640 1 --2- 172.21.15.45:0/2480021350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c840d20c0 0x7f3c84163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:38.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.172+0000 7f3c837fe640 1 -- 172.21.15.45:0/2480021350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c840d20c0 msgr2=0x7f3c84163920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:38.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.172+0000 7f3c837fe640 1 --2- 172.21.15.45:0/2480021350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c840d20c0 0x7f3c84163920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:38.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.172+0000 7f3c837fe640 1 -- 172.21.15.45:0/2480021350 >> v1:172.21.15.63:6789/0 conn(0x7f3c84070d30 legacy=0x7f3c840703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:38.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.172+0000 7f3c837fe640 1 -- 172.21.15.45:0/2480021350 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c840d15e0 con 0x7f3c84071110 2024-05-08T05:45:38.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.173+0000 7f3c837fe640 1 --2- 172.21.15.45:0/2480021350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c84071110 0x7f3c8406d4b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009e50 tx=0x7f3c7c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e28b905ce7450cb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:38.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.173+0000 7f3c81ffb640 1 -- 172.21.15.45:0/2480021350 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c036070 con 0x7f3c84071110 2024-05-08T05:45:38.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.173+0000 7f3c81ffb640 1 -- 172.21.15.45:0/2480021350 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c7c031600 con 0x7f3c84071110 2024-05-08T05:45:38.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.173+0000 7f3c81ffb640 1 -- 172.21.15.45:0/2480021350 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c0319d0 con 0x7f3c84071110 2024-05-08T05:45:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.174+0000 7f3c89262640 1 -- 172.21.15.45:0/2480021350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c84071110 msgr2=0x7f3c8406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.174+0000 7f3c89262640 1 --2- 172.21.15.45:0/2480021350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c84071110 0x7f3c8406d4b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009e50 tx=0x7f3c7c02a710 comp rx=0 tx=0).stop 2024-05-08T05:45:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.174+0000 7f3c89262640 1 -- 172.21.15.45:0/2480021350 shutdown_connections 2024-05-08T05:45:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.174+0000 7f3c89262640 1 --2- 172.21.15.45:0/2480021350 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c84071110 0x7f3c8406d4b0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.174+0000 7f3c89262640 1 --2- 172.21.15.45:0/2480021350 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c840d20c0 0x7f3c84163920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.174+0000 7f3c89262640 1 -- 172.21.15.45:0/2480021350 >> 172.21.15.45:0/2480021350 conn(0x7f3c840d2e90 msgr2=0x7f3c840d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.174+0000 7f3c89262640 1 -- 172.21.15.45:0/2480021350 shutdown_connections 2024-05-08T05:45:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.174+0000 7f3c89262640 1 -- 172.21.15.45:0/2480021350 wait complete. 2024-05-08T05:45:38.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.175+0000 7f3c89262640 1 Processor -- start 2024-05-08T05:45:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.175+0000 7f3c89262640 1 -- start start 2024-05-08T05:45:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.176+0000 7f3c89262640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c84070d30 0x7f3c840c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.176+0000 7f3c89262640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c84071110 0x7f3c840b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.176+0000 7f3c89262640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c840d20c0 0x7f3c840b6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.176+0000 7f3c89262640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3c84071580 con 0x7f3c840d20c0 2024-05-08T05:45:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.176+0000 7f3c89262640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3c840d1c30 con 0x7f3c84071110 2024-05-08T05:45:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.176+0000 7f3c89262640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3c84071700 con 0x7f3c84070d30 2024-05-08T05:45:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.176+0000 7f3c837fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c840d20c0 0x7f3c840b6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.176+0000 7f3c837fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c840d20c0 0x7f3c840b6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45458/0 (socket says 172.21.15.45:45458) 2024-05-08T05:45:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.176+0000 7f3c837fe640 1 -- 172.21.15.45:0/1993023778 learned_addr learned my addr 172.21.15.45:0/1993023778 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:38.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.176+0000 7f3c82ffd640 1 --2- 172.21.15.45:0/1993023778 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c84070d30 0x7f3c840c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.176+0000 7f3c827fc640 1 --2- 172.21.15.45:0/1993023778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c84071110 0x7f3c840b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.177+0000 7f3c837fe640 1 -- 172.21.15.45:0/1993023778 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c84070d30 msgr2=0x7f3c840c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.177+0000 7f3c837fe640 1 --2- 172.21.15.45:0/1993023778 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c84070d30 0x7f3c840c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.177+0000 7f3c837fe640 1 -- 172.21.15.45:0/1993023778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c84071110 msgr2=0x7f3c840b6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.177+0000 7f3c837fe640 1 --2- 172.21.15.45:0/1993023778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c84071110 0x7f3c840b6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.177+0000 7f3c837fe640 1 -- 172.21.15.45:0/1993023778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c840cfc70 con 0x7f3c840d20c0 2024-05-08T05:45:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.177+0000 7f3c827fc640 1 --2- 172.21.15.45:0/1993023778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c84071110 0x7f3c840b6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.177+0000 7f3c837fe640 1 --2- 172.21.15.45:0/1993023778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c840d20c0 0x7f3c840b6860 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009e50 tx=0x7f3c7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.177+0000 7f3c6b7fe640 1 -- 172.21.15.45:0/1993023778 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c036050 con 0x7f3c840d20c0 2024-05-08T05:45:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.177+0000 7f3c6b7fe640 1 -- 172.21.15.45:0/1993023778 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c7c042070 con 0x7f3c840d20c0 2024-05-08T05:45:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.177+0000 7f3c89262640 1 -- 172.21.15.45:0/1993023778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c840cae70 con 0x7f3c840d20c0 2024-05-08T05:45:38.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.177+0000 7f3c6b7fe640 1 -- 172.21.15.45:0/1993023778 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c009510 con 0x7f3c840d20c0 2024-05-08T05:45:38.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.177+0000 7f3c89262640 1 -- 172.21.15.45:0/1993023778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c840cb430 con 0x7f3c840d20c0 2024-05-08T05:45:38.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.178+0000 7f3c89262640 1 -- 172.21.15.45:0/1993023778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c840693f0 con 0x7f3c840d20c0 2024-05-08T05:45:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.179+0000 7f3c6b7fe640 1 -- 172.21.15.45:0/1993023778 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3c7c007bf0 con 0x7f3c840d20c0 2024-05-08T05:45:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.179+0000 7f3c6b7fe640 1 --2- 172.21.15.45:0/1993023778 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3c700762d0 0x7f3c70078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.180+0000 7f3c6b7fe640 1 -- 172.21.15.45:0/1993023778 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f3c7c0b8550 con 0x7f3c840d20c0 2024-05-08T05:45:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.180+0000 7f3c82ffd640 1 --2- 172.21.15.45:0/1993023778 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3c700762d0 0x7f3c70078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:38.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.180+0000 7f3c82ffd640 1 --2- 172.21.15.45:0/1993023778 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3c700762d0 0x7f3c70078790 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3c740015e0 tx=0x7f3c74009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:38.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.184+0000 7f3c6b7fe640 1 -- 172.21.15.45:0/1993023778 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3c7c081d40 con 0x7f3c840d20c0 2024-05-08T05:45:38.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.414+0000 7f3c89262640 1 -- 172.21.15.45:0/1993023778 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c840cff50 con 0x7f3c840d20c0 2024-05-08T05:45:38.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.415+0000 7f3c6b7fe640 1 -- 172.21.15.45:0/1993023778 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5184 (secure 0 0 0) 0x7f3c7c081640 con 0x7f3c840d20c0 2024-05-08T05:45:38.781 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:45:38.781 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5722,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1283757699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1283757699},{"type":"v1","addr":"172.21.15.63:6837","nonce":1283757699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2823770504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2823770504},{"type":"v1","addr":"172.21.15.63:6836","nonce":2823770504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1963062796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1963062796},{"type":"v1","addr":"172.21.15.45:6834","nonce":1963062796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:45:24.208158+0000","modified":"2024-05-08T05:45:37.490544+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":76,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":7,"addr":"172.21.15.45:6835/4183668301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4183668301},{"type":"v1","addr":"172.21.15.45:6835","nonce":4183668301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6171,"qdb_cluster":[6171]},"id":5}]} 2024-05-08T05:45:38.784 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 50 2024-05-08T05:45:38.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.417+0000 7f3c89262640 1 -- 172.21.15.45:0/1993023778 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3c700762d0 msgr2=0x7f3c70078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:38.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.417+0000 7f3c89262640 1 --2- 172.21.15.45:0/1993023778 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3c700762d0 0x7f3c70078790 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3c740015e0 tx=0x7f3c74009040 comp rx=0 tx=0).stop 2024-05-08T05:45:38.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.417+0000 7f3c89262640 1 -- 172.21.15.45:0/1993023778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c840d20c0 msgr2=0x7f3c840b6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:38.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.417+0000 7f3c89262640 1 --2- 172.21.15.45:0/1993023778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c840d20c0 0x7f3c840b6860 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009e50 tx=0x7f3c7c004a40 comp rx=0 tx=0).stop 2024-05-08T05:45:38.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.417+0000 7f3c89262640 1 -- 172.21.15.45:0/1993023778 shutdown_connections 2024-05-08T05:45:38.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.417+0000 7f3c89262640 1 --2- 172.21.15.45:0/1993023778 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3c700762d0 0x7f3c70078790 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:38.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.417+0000 7f3c89262640 1 --2- 172.21.15.45:0/1993023778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c840d20c0 0x7f3c840b6860 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:38.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.417+0000 7f3c89262640 1 --2- 172.21.15.45:0/1993023778 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c84071110 0x7f3c840b6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:38.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.417+0000 7f3c89262640 1 --2- 172.21.15.45:0/1993023778 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3c84070d30 0x7f3c840c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:38.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.417+0000 7f3c89262640 1 -- 172.21.15.45:0/1993023778 >> 172.21.15.45:0/1993023778 conn(0x7f3c840d2e90 msgr2=0x7f3c840703a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:38.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.417+0000 7f3c89262640 1 -- 172.21.15.45:0/1993023778 shutdown_connections 2024-05-08T05:45:38.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:38.417+0000 7f3c89262640 1 -- 172.21.15.45:0/1993023778 wait complete. 2024-05-08T05:45:39.190 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:45:24.208158+0000', 'modified': '2024-05-08T05:45:37.490544+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 76, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6171}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6171': {'gid': 6171, 'name': 'a', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.45:6835/4183668301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 4183668301}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 4183668301}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6171, 'qdb_cluster': [6171]} 2024-05-08T05:45:39.190 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T05:45:39.190 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T05:45:39.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.308+0000 7f743a958640 1 Processor -- start 2024-05-08T05:45:39.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.308+0000 7f743a958640 1 -- start start 2024-05-08T05:45:39.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.309+0000 7f743a958640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f74340d1970 0x7f74340d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.309+0000 7f743a958640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74341630b0 con 0x7f743415f420 2024-05-08T05:45:39.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.309+0000 7f743a958640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74340d3e70 con 0x7f74340d4cc0 2024-05-08T05:45:39.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.309+0000 7f743a958640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f74340d2550 con 0x7f74340d1970 2024-05-08T05:45:39.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.309+0000 7f7433fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f743415f420 0x7f74340d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50586/0 (socket says 172.21.15.45:50586) 2024-05-08T05:45:39.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.309+0000 7f7433fff640 1 -- 172.21.15.45:0/652938208 learned_addr learned my addr 172.21.15.45:0/652938208 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:39.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.309+0000 7f74337fe640 1 --2- 172.21.15.45:0/652938208 >> v2:172.21.15.63:3300/0 conn(0x7f74340d1970 0x7f74340d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.309+0000 7f7432ffd640 1 -- 172.21.15.45:0/652938208 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 36853332 0 0) 0x7f7424001110 con 0x7f743415f420 2024-05-08T05:45:39.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.309+0000 7f7432ffd640 1 -- 172.21.15.45:0/652938208 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f741c003520 con 0x7f743415f420 2024-05-08T05:45:39.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.309+0000 7f7432ffd640 1 -- 172.21.15.45:0/652938208 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1308443078 0 0) 0x7f74340d3e70 con 0x7f74340d4cc0 2024-05-08T05:45:39.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.309+0000 7f7432ffd640 1 -- 172.21.15.45:0/652938208 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7424001110 con 0x7f74340d4cc0 2024-05-08T05:45:39.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.309+0000 7f7432ffd640 1 -- 172.21.15.45:0/652938208 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1873873643 0 0) 0x7f741c003520 con 0x7f743415f420 2024-05-08T05:45:39.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.309+0000 7f74337fe640 1 -- 172.21.15.45:0/652938208 >> v1:172.21.15.63:6789/0 conn(0x7f74340d4cc0 legacy=0x7f74340d50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:39.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.309+0000 7f74337fe640 1 -- 172.21.15.45:0/652938208 >> v1:172.21.15.45:6789/0 conn(0x7f743415f420 legacy=0x7f74340d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:39.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.309+0000 7f74337fe640 1 -- 172.21.15.45:0/652938208 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74340d2c80 con 0x7f74340d1970 2024-05-08T05:45:39.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.310+0000 7f74337fe640 1 --2- 172.21.15.45:0/652938208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74340d1970 0x7f74340d1e20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f74280097a0 tx=0x7f742802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad87051ac0a6c858 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:39.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.310+0000 7f7432ffd640 1 -- 172.21.15.45:0/652938208 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7428037070 con 0x7f74340d1970 2024-05-08T05:45:39.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.310+0000 7f7432ffd640 1 -- 172.21.15.45:0/652938208 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f742802ac50 con 0x7f74340d1970 2024-05-08T05:45:39.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.310+0000 7f7432ffd640 1 -- 172.21.15.45:0/652938208 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74240018c0 con 0x7f74340d1970 2024-05-08T05:45:39.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.311+0000 7f743a958640 1 -- 172.21.15.45:0/652938208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74340d1970 msgr2=0x7f74340d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:39.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.311+0000 7f743a958640 1 --2- 172.21.15.45:0/652938208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74340d1970 0x7f74340d1e20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f74280097a0 tx=0x7f742802a1c0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.311+0000 7f743a958640 1 -- 172.21.15.45:0/652938208 shutdown_connections 2024-05-08T05:45:39.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.311+0000 7f743a958640 1 --2- 172.21.15.45:0/652938208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74340d1970 0x7f74340d1e20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.311+0000 7f743a958640 1 -- 172.21.15.45:0/652938208 >> 172.21.15.45:0/652938208 conn(0x7f743408eac0 msgr2=0x7f743409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:39.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.311+0000 7f743a958640 1 -- 172.21.15.45:0/652938208 shutdown_connections 2024-05-08T05:45:39.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.311+0000 7f743a958640 1 -- 172.21.15.45:0/652938208 wait complete. 2024-05-08T05:45:39.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.312+0000 7f743a958640 1 Processor -- start 2024-05-08T05:45:39.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.312+0000 7f743a958640 1 -- start start 2024-05-08T05:45:39.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.313+0000 7f743a958640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f74340d1970 0x7f74341f91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.313+0000 7f743a958640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74340d4cc0 0x7f74340b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.313+0000 7f743a958640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74340b8100 0x7f74340ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.313+0000 7f743a958640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f74340d2830 con 0x7f74340b8100 2024-05-08T05:45:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.313+0000 7f743a958640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f74340d3240 con 0x7f74340d4cc0 2024-05-08T05:45:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.313+0000 7f743a958640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f74340cfb20 con 0x7f74340d1970 2024-05-08T05:45:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.313+0000 7f7438ef5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74340b8100 0x7f74340ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.313+0000 7f7438ef5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74340b8100 0x7f74340ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45468/0 (socket says 172.21.15.45:45468) 2024-05-08T05:45:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.313+0000 7f7438ef5640 1 -- 172.21.15.45:0/2038692038 learned_addr learned my addr 172.21.15.45:0/2038692038 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.314+0000 7f7438ef5640 1 -- 172.21.15.45:0/2038692038 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f74340d1970 msgr2=0x7f74341f91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.314+0000 7f7438ef5640 1 --2- 172.21.15.45:0/2038692038 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f74340d1970 0x7f74341f91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.314+0000 7f7438ef5640 1 -- 172.21.15.45:0/2038692038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74340d4cc0 msgr2=0x7f74340b7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:39.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.314+0000 7f74337fe640 1 --2- 172.21.15.45:0/2038692038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74340d4cc0 0x7f74340b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.314+0000 7f7438ef5640 1 --2- 172.21.15.45:0/2038692038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74340d4cc0 0x7f74340b7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.314+0000 7f7438ef5640 1 -- 172.21.15.45:0/2038692038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743415be00 con 0x7f74340b8100 2024-05-08T05:45:39.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.314+0000 7f7438ef5640 1 --2- 172.21.15.45:0/2038692038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74340b8100 0x7f74340ce6e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f742c00c7b0 tx=0x7f742c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:39.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.314+0000 7f7430ff9640 1 -- 172.21.15.45:0/2038692038 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c003b70 con 0x7f74340b8100 2024-05-08T05:45:39.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.314+0000 7f7430ff9640 1 -- 172.21.15.45:0/2038692038 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f742c003d10 con 0x7f74340b8100 2024-05-08T05:45:39.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.314+0000 7f743a958640 1 -- 172.21.15.45:0/2038692038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74341f96f0 con 0x7f74340b8100 2024-05-08T05:45:39.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.314+0000 7f7430ff9640 1 -- 172.21.15.45:0/2038692038 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c00aa90 con 0x7f74340b8100 2024-05-08T05:45:39.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.314+0000 7f743a958640 1 -- 172.21.15.45:0/2038692038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74340cdc00 con 0x7f74340b8100 2024-05-08T05:45:39.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.316+0000 7f7430ff9640 1 -- 172.21.15.45:0/2038692038 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f742c00ac30 con 0x7f74340b8100 2024-05-08T05:45:39.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.316+0000 7f7430ff9640 1 --2- 172.21.15.45:0/2038692038 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f74200762d0 0x7f7420078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.316+0000 7f7430ff9640 1 -- 172.21.15.45:0/2038692038 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f742c0977e0 con 0x7f74340b8100 2024-05-08T05:45:39.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.317+0000 7f7433fff640 1 --2- 172.21.15.45:0/2038692038 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f74200762d0 0x7f7420078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.317+0000 7f7433fff640 1 --2- 172.21.15.45:0/2038692038 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f74200762d0 0x7f7420078790 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7424001e80 tx=0x7f7424001530 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:39.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.317+0000 7f743d672740 1 Processor -- start 2024-05-08T05:45:39.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.317+0000 7f743d672740 1 -- start start 2024-05-08T05:45:39.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.318+0000 7f743d672740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc959f50 0x5575fc95c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.318+0000 7f743d672740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc95a330 0x5575fc95cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.318+0000 7f743d672740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc92cbb0 0x5575fc9303c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.318+0000 7f743d672740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5575fc8968d0 con 0x5575fc92cbb0 2024-05-08T05:45:39.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.318+0000 7f743d672740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5575fc84fc10 con 0x5575fc95a330 2024-05-08T05:45:39.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.318+0000 7f743d672740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5575fc892890 con 0x5575fc959f50 2024-05-08T05:45:39.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.318+0000 7f7438ef5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc92cbb0 0x5575fc9303c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.318+0000 7f7438ef5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc92cbb0 0x5575fc9303c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45482/0 (socket says 172.21.15.45:45482) 2024-05-08T05:45:39.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.318+0000 7f7438ef5640 1 -- 172.21.15.45:0/3817824292 learned_addr learned my addr 172.21.15.45:0/3817824292 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:39.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.318+0000 7f7433fff640 1 --2- 172.21.15.45:0/3817824292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc959f50 0x5575fc95c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.318+0000 7f74337fe640 1 --2- 172.21.15.45:0/3817824292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc95a330 0x5575fc95cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.318+0000 7f7438ef5640 1 -- 172.21.15.45:0/3817824292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc959f50 msgr2=0x5575fc95c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.318+0000 7f7438ef5640 1 --2- 172.21.15.45:0/3817824292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc959f50 0x5575fc95c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.318+0000 7f7438ef5640 1 -- 172.21.15.45:0/3817824292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc95a330 msgr2=0x5575fc95cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.318+0000 7f7438ef5640 1 --2- 172.21.15.45:0/3817824292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc95a330 0x5575fc95cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.318+0000 7f7438ef5640 1 -- 172.21.15.45:0/3817824292 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575fc8575d0 con 0x5575fc92cbb0 2024-05-08T05:45:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.318+0000 7f7433fff640 1 --2- 172.21.15.45:0/3817824292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc959f50 0x5575fc95c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.319+0000 7f7438ef5640 1 --2- 172.21.15.45:0/3817824292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc92cbb0 0x5575fc9303c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f742c05cdb0 tx=0x7f742c05ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58482f597519dd86 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.319+0000 7f7419ffb640 1 -- 172.21.15.45:0/3817824292 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c067e60 con 0x5575fc92cbb0 2024-05-08T05:45:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.319+0000 7f7419ffb640 1 -- 172.21.15.45:0/3817824292 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f742c0607c0 con 0x5575fc92cbb0 2024-05-08T05:45:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.319+0000 7f7419ffb640 1 -- 172.21.15.45:0/3817824292 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c0670d0 con 0x5575fc92cbb0 2024-05-08T05:45:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.319+0000 7f743d672740 1 -- 172.21.15.45:0/3817824292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc92cbb0 msgr2=0x5575fc9303c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:39.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.319+0000 7f743d672740 1 --2- 172.21.15.45:0/3817824292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc92cbb0 0x5575fc9303c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f742c05cdb0 tx=0x7f742c05ff30 comp rx=0 tx=0).stop 2024-05-08T05:45:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.319+0000 7f743d672740 1 -- 172.21.15.45:0/3817824292 shutdown_connections 2024-05-08T05:45:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.319+0000 7f743d672740 1 --2- 172.21.15.45:0/3817824292 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc92cbb0 0x5575fc9303c0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.319+0000 7f743d672740 1 --2- 172.21.15.45:0/3817824292 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc95a330 0x5575fc95cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.319+0000 7f743d672740 1 --2- 172.21.15.45:0/3817824292 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc959f50 0x5575fc95c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.319+0000 7f743d672740 1 -- 172.21.15.45:0/3817824292 >> 172.21.15.45:0/3817824292 conn(0x5575fc92ee20 msgr2=0x5575fc977760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.319+0000 7f743d672740 1 -- 172.21.15.45:0/3817824292 shutdown_connections 2024-05-08T05:45:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.320+0000 7f743d672740 1 -- 172.21.15.45:0/3817824292 wait complete. 2024-05-08T05:45:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.320+0000 7f743d672740 1 -- start start 2024-05-08T05:45:39.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.320+0000 7f743d672740 1 Processor -- start 2024-05-08T05:45:39.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.321+0000 7f743d672740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc959f50 0x5575fc9605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.321+0000 7f743d672740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc95a330 0x5575fc956620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.321+0000 7f743d672740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc950560 0x5575fc956b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.321+0000 7f743d672740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5575fc931fd0 con 0x5575fc95a330 2024-05-08T05:45:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.321+0000 7f743d672740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5575fc853760 con 0x5575fc950560 2024-05-08T05:45:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.321+0000 7f743d672740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5575fc9322b0 con 0x5575fc959f50 2024-05-08T05:45:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.321+0000 7f743d672740 10 client.0 _ll_register_callbacks cb 0x5575fc88ccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:45:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.321+0000 7f74337fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc95a330 0x5575fc956620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.321+0000 7f74337fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc95a330 0x5575fc956620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45492/0 (socket says 172.21.15.45:45492) 2024-05-08T05:45:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.321+0000 7f74337fe640 1 -- 172.21.15.45:0/2471752535 learned_addr learned my addr 172.21.15.45:0/2471752535 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.321+0000 7f7433fff640 1 --2- 172.21.15.45:0/2471752535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc959f50 0x5575fc9605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.321+0000 7f7438ef5640 1 --2- 172.21.15.45:0/2471752535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc950560 0x5575fc956b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.322+0000 7f74337fe640 1 -- 172.21.15.45:0/2471752535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc959f50 msgr2=0x5575fc9605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:39.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.322+0000 7f74337fe640 1 --2- 172.21.15.45:0/2471752535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc959f50 0x5575fc9605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.322+0000 7f74337fe640 1 -- 172.21.15.45:0/2471752535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc950560 msgr2=0x5575fc956b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:39.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.322+0000 7f74337fe640 1 --2- 172.21.15.45:0/2471752535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc950560 0x5575fc956b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.322+0000 7f74337fe640 1 -- 172.21.15.45:0/2471752535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575fc5a82d0 con 0x5575fc95a330 2024-05-08T05:45:39.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.322+0000 7f7438ef5640 1 --2- 172.21.15.45:0/2471752535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc950560 0x5575fc956b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:39.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.322+0000 7f7433fff640 1 --2- 172.21.15.45:0/2471752535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc959f50 0x5575fc9605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:39.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.322+0000 7f74337fe640 1 --2- 172.21.15.45:0/2471752535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc95a330 0x5575fc956620 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7428035040 tx=0x7f7428004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce0781ff65de82e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:39.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.322+0000 7f7412ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:45:39.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.322+0000 7f7412ffd640 1 -- 172.21.15.45:0/2471752535 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7428044070 con 0x5575fc95a330 2024-05-08T05:45:39.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.322+0000 7f7412ffd640 1 -- 172.21.15.45:0/2471752535 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f742803e670 con 0x5575fc95a330 2024-05-08T05:45:39.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.322+0000 7f743d672740 1 -- 172.21.15.45:0/2471752535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f742c0607c0 con 0x5575fc95a330 2024-05-08T05:45:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.322+0000 7f7412ffd640 1 -- 172.21.15.45:0/2471752535 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f742804f050 con 0x5575fc95a330 2024-05-08T05:45:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.322+0000 7f743d672740 10 client.6381 fetch_fsmap learned FSMap version 50 2024-05-08T05:45:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.322+0000 7f743d672740 1 -- 172.21.15.45:0/2471752535 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x5575fc6726b0 con 0x5575fc95a330 2024-05-08T05:45:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.322+0000 7f7412ffd640 1 -- 172.21.15.45:0/2471752535 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f742c0607c0 con 0x5575fc95a330 2024-05-08T05:45:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.322+0000 7f7412ffd640 1 -- 172.21.15.45:0/2471752535 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f7428052020 con 0x5575fc95a330 2024-05-08T05:45:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.322+0000 7f743d672740 10 client.6381 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T05:45:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.322+0000 7f743d672740 10 client.6381 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T05:45:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.322+0000 7f743d672740 20 client.6381 populate_metadata read hostname 'smithi045' 2024-05-08T05:45:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.323+0000 7f743d672740 1 --2- 172.21.15.45:0/2471752535 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x5575fc95e020 0x5575fc996660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.323+0000 7f743d672740 4 client.6381 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:45:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.323+0000 7f743d672740 1 -- 172.21.15.45:0/2471752535 --> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575fc81a760 con 0x5575fc95e020 2024-05-08T05:45:39.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.323+0000 7f7433fff640 1 --2- 172.21.15.45:0/2471752535 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x5575fc95e020 0x5575fc996660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.323+0000 7f7412ffd640 10 client.6381 ms_handle_connect on v2:172.21.15.45:6833/4183668301 2024-05-08T05:45:39.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.323+0000 7f7433fff640 1 --2- 172.21.15.45:0/2471752535 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x5575fc95e020 0x5575fc996660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84ad49eb06ecf291 server_cookie=f3ce43b9064b0ab2 in_seq=0 out_seq=0 2024-05-08T05:45:39.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.324+0000 7f7412ffd640 1 -- 172.21.15.45:0/2471752535 <== mds.0 v2:172.21.15.45:6833/4183668301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5575fc81a760 con 0x5575fc95e020 2024-05-08T05:45:39.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.324+0000 7f7412ffd640 10 client.6381 handle_command_reply: tid=0 2024-05-08T05:45:39.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.324+0000 7f743d672740 1 client.6381 shutdown 2024-05-08T05:45:39.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.324+0000 7f743d672740 1 -- 172.21.15.45:0/2471752535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc95a330 msgr2=0x5575fc956620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:39.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.324+0000 7f743d672740 1 --2- 172.21.15.45:0/2471752535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc95a330 0x5575fc956620 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7428035040 tx=0x7f7428004730 comp rx=0 tx=0).stop 2024-05-08T05:45:39.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.324+0000 7f743d672740 1 -- 172.21.15.45:0/2471752535 shutdown_connections 2024-05-08T05:45:39.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.324+0000 7f743d672740 1 --2- 172.21.15.45:0/2471752535 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x5575fc95e020 0x5575fc996660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.324+0000 7f743d672740 1 --2- 172.21.15.45:0/2471752535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc950560 0x5575fc956b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.324+0000 7f743d672740 1 --2- 172.21.15.45:0/2471752535 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc95a330 0x5575fc956620 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.324+0000 7f7412ffd640 0 client.6381 ms_handle_reset on v2:172.21.15.45:6833/4183668301 2024-05-08T05:45:39.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.324+0000 7f743d672740 1 --2- 172.21.15.45:0/2471752535 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc959f50 0x5575fc9605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.324+0000 7f743d672740 1 -- 172.21.15.45:0/2471752535 >> 172.21.15.45:0/2471752535 conn(0x5575fc92cbb0 msgr2=0x5575fc92d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:39.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.324+0000 7f743d672740 1 -- 172.21.15.45:0/2471752535 shutdown_connections 2024-05-08T05:45:39.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.324+0000 7f743d672740 1 -- 172.21.15.45:0/2471752535 wait complete. 2024-05-08T05:45:39.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.324+0000 7f743d672740 20 client.6381 trim_cache size 0 max 16384 2024-05-08T05:45:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.343+0000 7f743d672740 1 Processor -- start 2024-05-08T05:45:39.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.343+0000 7f743d672740 1 -- start start 2024-05-08T05:45:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.344+0000 7f743d672740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc950560 0x5575fc979270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.344+0000 7f743d672740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc959f50 0x5575fc9797b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.344+0000 7f743d672740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc95a330 0x5575fc958eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.344+0000 7f743d672740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7434200fb0 con 0x5575fc95a330 2024-05-08T05:45:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.344+0000 7f743d672740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f74340ba770 con 0x5575fc950560 2024-05-08T05:45:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.344+0000 7f743d672740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f74340ca930 con 0x5575fc959f50 2024-05-08T05:45:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.344+0000 7f7438ef5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc95a330 0x5575fc958eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.344+0000 7f7438ef5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc95a330 0x5575fc958eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45502/0 (socket says 172.21.15.45:45502) 2024-05-08T05:45:39.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.344+0000 7f7438ef5640 1 -- 172.21.15.45:0/35628115 learned_addr learned my addr 172.21.15.45:0/35628115 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.344+0000 7f7433fff640 1 --2- 172.21.15.45:0/35628115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc950560 0x5575fc979270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.344+0000 7f74337fe640 1 --2- 172.21.15.45:0/35628115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc959f50 0x5575fc9797b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.344+0000 7f7438ef5640 1 -- 172.21.15.45:0/35628115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc959f50 msgr2=0x5575fc9797b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.344+0000 7f7438ef5640 1 --2- 172.21.15.45:0/35628115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc959f50 0x5575fc9797b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.344+0000 7f7438ef5640 1 -- 172.21.15.45:0/35628115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc950560 msgr2=0x5575fc979270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.345+0000 7f7438ef5640 1 --2- 172.21.15.45:0/35628115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc950560 0x5575fc979270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.345+0000 7f7438ef5640 1 -- 172.21.15.45:0/35628115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575fc950370 con 0x5575fc95a330 2024-05-08T05:45:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.345+0000 7f74337fe640 1 --2- 172.21.15.45:0/35628115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc959f50 0x5575fc9797b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.345+0000 7f7438ef5640 1 --2- 172.21.15.45:0/35628115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc95a330 0x5575fc958eb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f742c060470 tx=0x7f742c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7112c1e348d254a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:39.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.345+0000 7f74197fa640 1 -- 172.21.15.45:0/35628115 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c067e60 con 0x5575fc95a330 2024-05-08T05:45:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.345+0000 7f74197fa640 1 -- 172.21.15.45:0/35628115 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f742c0607c0 con 0x5575fc95a330 2024-05-08T05:45:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.345+0000 7f74197fa640 1 -- 172.21.15.45:0/35628115 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c075dc0 con 0x5575fc95a330 2024-05-08T05:45:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.345+0000 7f743d672740 1 -- 172.21.15.45:0/35628115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc95a330 msgr2=0x5575fc958eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.345+0000 7f743d672740 1 --2- 172.21.15.45:0/35628115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc95a330 0x5575fc958eb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f742c060470 tx=0x7f742c01d000 comp rx=0 tx=0).stop 2024-05-08T05:45:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.345+0000 7f743d672740 1 -- 172.21.15.45:0/35628115 shutdown_connections 2024-05-08T05:45:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.345+0000 7f743d672740 1 --2- 172.21.15.45:0/35628115 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc959f50 0x5575fc9797b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.345+0000 7f743d672740 1 --2- 172.21.15.45:0/35628115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc95a330 0x5575fc958eb0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.345+0000 7f743d672740 1 --2- 172.21.15.45:0/35628115 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc950560 0x5575fc979270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.345+0000 7f743d672740 1 -- 172.21.15.45:0/35628115 >> 172.21.15.45:0/35628115 conn(0x5575fc92cbb0 msgr2=0x5575fc97b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:39.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.346+0000 7f743d672740 1 -- 172.21.15.45:0/35628115 shutdown_connections 2024-05-08T05:45:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.346+0000 7f743d672740 1 -- 172.21.15.45:0/35628115 wait complete. 2024-05-08T05:45:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.346+0000 7f743d672740 1 -- start start 2024-05-08T05:45:39.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.346+0000 7f743d672740 1 Processor -- start 2024-05-08T05:45:39.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.347+0000 7f743d672740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc950560 0x5575fc927f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.347+0000 7f743d672740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc959f50 0x5575fc959170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.347+0000 7f743d672740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc95a330 0x5575fc979270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.347+0000 7f743d672740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5575fc957970 con 0x5575fc950560 2024-05-08T05:45:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.347+0000 7f743d672740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5575fc9570a0 con 0x5575fc959f50 2024-05-08T05:45:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.347+0000 7f743d672740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5575fc95e830 con 0x5575fc95a330 2024-05-08T05:45:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.347+0000 7f743d672740 10 client.0 _ll_register_callbacks cb 0x7f741c0036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:45:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.347+0000 7f7433fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc950560 0x5575fc927f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.347+0000 7f7433fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc950560 0x5575fc927f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45504/0 (socket says 172.21.15.45:45504) 2024-05-08T05:45:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.347+0000 7f7433fff640 1 -- 172.21.15.45:0/2871655071 learned_addr learned my addr 172.21.15.45:0/2871655071 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.347+0000 7f7438ef5640 1 --2- 172.21.15.45:0/2871655071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc95a330 0x5575fc979270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.347+0000 7f74337fe640 1 --2- 172.21.15.45:0/2871655071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc959f50 0x5575fc959170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.347+0000 7f7433fff640 1 -- 172.21.15.45:0/2871655071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc95a330 msgr2=0x5575fc979270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.347+0000 7f7433fff640 1 --2- 172.21.15.45:0/2871655071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc95a330 0x5575fc979270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.347+0000 7f7433fff640 1 -- 172.21.15.45:0/2871655071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc959f50 msgr2=0x5575fc959170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:39.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.347+0000 7f7433fff640 1 --2- 172.21.15.45:0/2871655071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc959f50 0x5575fc959170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.347+0000 7f7433fff640 1 -- 172.21.15.45:0/2871655071 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575fc97a440 con 0x5575fc950560 2024-05-08T05:45:39.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.348+0000 7f7433fff640 1 --2- 172.21.15.45:0/2871655071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc950560 0x5575fc927f40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x5575fc957250 tx=0x7f74240012e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7071228c65639db2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:39.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.348+0000 7f7412ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:45:39.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.348+0000 7f7412ffd640 1 -- 172.21.15.45:0/2871655071 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424023e60 con 0x5575fc950560 2024-05-08T05:45:39.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.348+0000 7f7412ffd640 1 -- 172.21.15.45:0/2871655071 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f7424023840 con 0x5575fc950560 2024-05-08T05:45:39.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.348+0000 7f743d672740 1 -- 172.21.15.45:0/2871655071 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f742c0607c0 con 0x5575fc950560 2024-05-08T05:45:39.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.348+0000 7f7412ffd640 1 -- 172.21.15.45:0/2871655071 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f7424007720 con 0x5575fc950560 2024-05-08T05:45:39.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.348+0000 7f743d672740 10 client.6387 fetch_fsmap learned FSMap version 50 2024-05-08T05:45:39.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.348+0000 7f743d672740 1 -- 172.21.15.45:0/2871655071 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x5575fc9f63c0 con 0x5575fc950560 2024-05-08T05:45:39.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.348+0000 7f7412ffd640 1 -- 172.21.15.45:0/2871655071 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f742c0607c0 con 0x5575fc950560 2024-05-08T05:45:39.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.348+0000 7f7412ffd640 1 -- 172.21.15.45:0/2871655071 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f742400c730 con 0x5575fc950560 2024-05-08T05:45:39.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.348+0000 7f743d672740 10 client.6387 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T05:45:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.348+0000 7f743d672740 10 client.6387 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T05:45:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.348+0000 7f743d672740 20 client.6387 populate_metadata read hostname 'smithi045' 2024-05-08T05:45:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.348+0000 7f743d672740 1 --2- 172.21.15.45:0/2871655071 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x5575fc9f4fb0 0x5575fc9f5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.348+0000 7f743d672740 4 client.6387 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:45:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.348+0000 7f743d672740 1 -- 172.21.15.45:0/2871655071 --> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] -- command(tid 0: {"prefix": "status"}) -- 0x5575fc8a44e0 con 0x5575fc9f4fb0 2024-05-08T05:45:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.349+0000 7f74337fe640 1 --2- 172.21.15.45:0/2871655071 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x5575fc9f4fb0 0x5575fc9f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.349+0000 7f7412ffd640 10 client.6387 ms_handle_connect on v2:172.21.15.45:6833/4183668301 2024-05-08T05:45:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.349+0000 7f74337fe640 1 --2- 172.21.15.45:0/2871655071 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x5575fc9f4fb0 0x5575fc9f5390 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fdfbbc033c13016 server_cookie=d4c177638007d7fc in_seq=0 out_seq=0 2024-05-08T05:45:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.349+0000 7f7412ffd640 1 -- 172.21.15.45:0/2871655071 <== mds.0 v2:172.21.15.45:6833/4183668301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5575fc8a44e0 con 0x5575fc9f4fb0 2024-05-08T05:45:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.349+0000 7f7412ffd640 10 client.6387 handle_command_reply: tid=0 2024-05-08T05:45:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.349+0000 7f743d672740 1 client.6387 shutdown 2024-05-08T05:45:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.349+0000 7f743d672740 1 -- 172.21.15.45:0/2871655071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc950560 msgr2=0x5575fc927f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:39.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.349+0000 7f743d672740 1 --2- 172.21.15.45:0/2871655071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc950560 0x5575fc927f40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x5575fc957250 tx=0x7f74240012e0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.349+0000 7f743d672740 1 -- 172.21.15.45:0/2871655071 shutdown_connections 2024-05-08T05:45:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.349+0000 7f743d672740 1 --2- 172.21.15.45:0/2871655071 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x5575fc9f4fb0 0x5575fc9f5390 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.349+0000 7f743d672740 1 --2- 172.21.15.45:0/2871655071 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5575fc95a330 0x5575fc979270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.349+0000 7f743d672740 1 --2- 172.21.15.45:0/2871655071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5575fc959f50 0x5575fc959170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.349+0000 7f743d672740 1 --2- 172.21.15.45:0/2871655071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5575fc950560 0x5575fc927f40 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.349+0000 7f7412ffd640 0 client.6387 ms_handle_reset on v2:172.21.15.45:6833/4183668301 2024-05-08T05:45:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.349+0000 7f743d672740 1 -- 172.21.15.45:0/2871655071 >> 172.21.15.45:0/2871655071 conn(0x5575fc92cbb0 msgr2=0x5575fc978000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.349+0000 7f743d672740 1 -- 172.21.15.45:0/2871655071 shutdown_connections 2024-05-08T05:45:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.349+0000 7f743d672740 1 -- 172.21.15.45:0/2871655071 wait complete. 2024-05-08T05:45:39.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.349+0000 7f743d672740 20 client.6387 trim_cache size 0 max 16384 2024-05-08T05:45:39.359 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:45:39.359 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "e9e1170a-fd27-4625-97a9-5d5f104d13a2", 2024-05-08T05:45:39.359 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-05-08T05:45:39.359 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 6171, 2024-05-08T05:45:39.359 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-05-08T05:45:39.359 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-05-08T05:45:39.359 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-05-08T05:45:39.359 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 4.8625637069999996, 2024-05-08T05:45:39.359 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 50, 2024-05-08T05:45:39.359 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 76, 2024-05-08T05:45:39.359 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 76, 2024-05-08T05:45:39.360 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 19.838094215999998 2024-05-08T05:45:39.360 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:45:39.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.350+0000 7f743a958640 1 -- 172.21.15.45:0/2038692038 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f74200762d0 msgr2=0x7f7420078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:39.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.350+0000 7f743a958640 1 --2- 172.21.15.45:0/2038692038 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f74200762d0 0x7f7420078790 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7424001e80 tx=0x7f7424001530 comp rx=0 tx=0).stop 2024-05-08T05:45:39.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.350+0000 7f743a958640 1 -- 172.21.15.45:0/2038692038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74340b8100 msgr2=0x7f74340ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:39.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.350+0000 7f743a958640 1 --2- 172.21.15.45:0/2038692038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74340b8100 0x7f74340ce6e0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f742c00c7b0 tx=0x7f742c00cc70 comp rx=0 tx=0).stop 2024-05-08T05:45:39.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.350+0000 7f743a958640 1 -- 172.21.15.45:0/2038692038 shutdown_connections 2024-05-08T05:45:39.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.350+0000 7f743a958640 1 --2- 172.21.15.45:0/2038692038 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f74200762d0 0x7f7420078790 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.350+0000 7f743a958640 1 --2- 172.21.15.45:0/2038692038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74340b8100 0x7f74340ce6e0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.350+0000 7f743a958640 1 --2- 172.21.15.45:0/2038692038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74340d4cc0 0x7f74340b7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.350+0000 7f743a958640 1 --2- 172.21.15.45:0/2038692038 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f74340d1970 0x7f74341f91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.350+0000 7f743a958640 1 -- 172.21.15.45:0/2038692038 >> 172.21.15.45:0/2038692038 conn(0x7f743408eac0 msgr2=0x7f743415c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:39.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.350+0000 7f743a958640 1 -- 172.21.15.45:0/2038692038 shutdown_connections 2024-05-08T05:45:39.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.350+0000 7f743a958640 1 -- 172.21.15.45:0/2038692038 wait complete. 2024-05-08T05:45:39.363 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T05:45:39.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.465+0000 7f4b7861d640 1 Processor -- start 2024-05-08T05:45:39.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.465+0000 7f4b7861d640 1 -- start start 2024-05-08T05:45:39.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.466+0000 7f4b7861d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b7015f420 0x7f4b700d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.466+0000 7f4b7861d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4b700d2080 0x7f4b700d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.466+0000 7f4b7861d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b701630b0 con 0x7f4b700d3ca0 2024-05-08T05:45:39.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.466+0000 7f4b7861d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4b700d2550 con 0x7f4b700d2080 2024-05-08T05:45:39.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.466+0000 7f4b7861d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b700d2c80 con 0x7f4b7015f420 2024-05-08T05:45:39.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.467+0000 7f4b76bba640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4b700d2080 0x7f4b700d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.467+0000 7f4b75bb8640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4b700d3ca0 0x7f4b700d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50598/0 (socket says 172.21.15.45:50598) 2024-05-08T05:45:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.467+0000 7f4b75bb8640 1 -- 172.21.15.45:0/382747963 learned_addr learned my addr 172.21.15.45:0/382747963 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.467+0000 7f4b763b9640 1 --2- 172.21.15.45:0/382747963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b7015f420 0x7f4b700d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.467+0000 7f4b76bba640 1 -- 172.21.15.45:0/382747963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b7015f420 msgr2=0x7f4b700d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.467+0000 7f4b76bba640 1 --2- 172.21.15.45:0/382747963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b7015f420 0x7f4b700d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.467+0000 7f4b76bba640 1 -- 172.21.15.45:0/382747963 >> v1:172.21.15.45:6789/0 conn(0x7f4b700d3ca0 legacy=0x7f4b700d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.467+0000 7f4b76bba640 1 -- 172.21.15.45:0/382747963 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b700d2f60 con 0x7f4b700d2080 2024-05-08T05:45:39.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.467+0000 7f4b753b7640 1 -- 172.21.15.45:0/382747963 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3639952523 0 0) 0x7f4b64001110 con 0x7f4b700d3ca0 2024-05-08T05:45:39.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.468+0000 7f4b76bba640 1 --2- 172.21.15.45:0/382747963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b700d2080 0x7f4b700d4cc0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009d90 tx=0x7f4b6002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ba6542c1b344ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:39.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.468+0000 7f4b753b7640 1 -- 172.21.15.45:0/382747963 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b60007740 con 0x7f4b700d2080 2024-05-08T05:45:39.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.468+0000 7f4b753b7640 1 -- 172.21.15.45:0/382747963 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b600078e0 con 0x7f4b700d2080 2024-05-08T05:45:39.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.468+0000 7f4b753b7640 1 -- 172.21.15.45:0/382747963 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b60007cb0 con 0x7f4b700d2080 2024-05-08T05:45:39.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.468+0000 7f4b7861d640 1 -- 172.21.15.45:0/382747963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b700d2080 msgr2=0x7f4b700d4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:39.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.468+0000 7f4b7861d640 1 --2- 172.21.15.45:0/382747963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b700d2080 0x7f4b700d4cc0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009d90 tx=0x7f4b6002a660 comp rx=0 tx=0).stop 2024-05-08T05:45:39.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.468+0000 7f4b7861d640 1 -- 172.21.15.45:0/382747963 shutdown_connections 2024-05-08T05:45:39.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.468+0000 7f4b7861d640 1 --2- 172.21.15.45:0/382747963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b700d2080 0x7f4b700d4cc0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.468+0000 7f4b7861d640 1 --2- 172.21.15.45:0/382747963 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b7015f420 0x7f4b700d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.468+0000 7f4b7861d640 1 -- 172.21.15.45:0/382747963 >> 172.21.15.45:0/382747963 conn(0x7f4b7008eac0 msgr2=0x7f4b7009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:39.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.468+0000 7f4b7861d640 1 -- 172.21.15.45:0/382747963 shutdown_connections 2024-05-08T05:45:39.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.468+0000 7f4b7861d640 1 -- 172.21.15.45:0/382747963 wait complete. 2024-05-08T05:45:39.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.470+0000 7f4b7861d640 1 Processor -- start 2024-05-08T05:45:39.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.470+0000 7f4b7861d640 1 -- start start 2024-05-08T05:45:39.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.470+0000 7f4b7861d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b700d2080 0x7f4b7015a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.470+0000 7f4b7861d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b700ca250 0x7f4b700ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.470+0000 7f4b7861d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b7015c5d0 0x7f4b7015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.470+0000 7f4b7861d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b700d5200 con 0x7f4b700ca250 2024-05-08T05:45:39.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.470+0000 7f4b7861d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b700d2830 con 0x7f4b700d2080 2024-05-08T05:45:39.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.470+0000 7f4b7861d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b700d5380 con 0x7f4b7015c5d0 2024-05-08T05:45:39.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.471+0000 7f4b75bb8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b700ca250 0x7f4b700ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.471+0000 7f4b75bb8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b700ca250 0x7f4b700ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45532/0 (socket says 172.21.15.45:45532) 2024-05-08T05:45:39.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.471+0000 7f4b75bb8640 1 -- 172.21.15.45:0/2413945665 learned_addr learned my addr 172.21.15.45:0/2413945665 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:39.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.471+0000 7f4b76bba640 1 --2- 172.21.15.45:0/2413945665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b7015c5d0 0x7f4b7015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.471+0000 7f4b763b9640 1 --2- 172.21.15.45:0/2413945665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b700d2080 0x7f4b7015a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.471+0000 7f4b75bb8640 1 -- 172.21.15.45:0/2413945665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b7015c5d0 msgr2=0x7f4b7015ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:39.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.471+0000 7f4b75bb8640 1 --2- 172.21.15.45:0/2413945665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b7015c5d0 0x7f4b7015ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.471+0000 7f4b75bb8640 1 -- 172.21.15.45:0/2413945665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b700d2080 msgr2=0x7f4b7015a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:39.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.471+0000 7f4b75bb8640 1 --2- 172.21.15.45:0/2413945665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b700d2080 0x7f4b7015a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.471+0000 7f4b75bb8640 1 -- 172.21.15.45:0/2413945665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b70160700 con 0x7f4b700ca250 2024-05-08T05:45:39.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.471+0000 7f4b75bb8640 1 --2- 172.21.15.45:0/2413945665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b700ca250 0x7f4b700ca6d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400c760 tx=0x7f4b6400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:39.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.471+0000 7f4b6effd640 1 -- 172.21.15.45:0/2413945665 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64018070 con 0x7f4b700ca250 2024-05-08T05:45:39.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.471+0000 7f4b6effd640 1 -- 172.21.15.45:0/2413945665 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b640070d0 con 0x7f4b700ca250 2024-05-08T05:45:39.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.471+0000 7f4b7861d640 1 -- 172.21.15.45:0/2413945665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b7015fb80 con 0x7f4b700ca250 2024-05-08T05:45:39.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.471+0000 7f4b6effd640 1 -- 172.21.15.45:0/2413945665 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b640074c0 con 0x7f4b700ca250 2024-05-08T05:45:39.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.471+0000 7f4b7861d640 1 -- 172.21.15.45:0/2413945665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b70160140 con 0x7f4b700ca250 2024-05-08T05:45:39.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.472+0000 7f4b7861d640 1 -- 172.21.15.45:0/2413945665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b700693f0 con 0x7f4b700ca250 2024-05-08T05:45:39.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.473+0000 7f4b6effd640 1 -- 172.21.15.45:0/2413945665 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4b6401b070 con 0x7f4b700ca250 2024-05-08T05:45:39.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.473+0000 7f4b6effd640 1 --2- 172.21.15.45:0/2413945665 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b5c0762d0 0x7f4b5c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:39.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.474+0000 7f4b6effd640 1 -- 172.21.15.45:0/2413945665 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f4b64096bd0 con 0x7f4b700ca250 2024-05-08T05:45:39.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.474+0000 7f4b763b9640 1 --2- 172.21.15.45:0/2413945665 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b5c0762d0 0x7f4b5c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:39.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.474+0000 7f4b763b9640 1 --2- 172.21.15.45:0/2413945665 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b5c0762d0 0x7f4b5c078790 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009a30 tx=0x7f4b68008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:39.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.478+0000 7f4b6effd640 1 -- 172.21.15.45:0/2413945665 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4b701630b0 con 0x7f4b700ca250 2024-05-08T05:45:39.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.711+0000 7f4b7861d640 1 -- 172.21.15.45:0/2413945665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4b70075eb0 con 0x7f4b700ca250 2024-05-08T05:45:39.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.711+0000 7f4b6effd640 1 -- 172.21.15.45:0/2413945665 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) ==== 53+0+73 (secure 0 0 0) 0x7f4b64060340 con 0x7f4b700ca250 2024-05-08T05:45:39.711 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T05:45:39.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.714+0000 7f4b7861d640 1 -- 172.21.15.45:0/2413945665 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b5c0762d0 msgr2=0x7f4b5c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:39.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.714+0000 7f4b7861d640 1 --2- 172.21.15.45:0/2413945665 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b5c0762d0 0x7f4b5c078790 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009a30 tx=0x7f4b68008040 comp rx=0 tx=0).stop 2024-05-08T05:45:39.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.714+0000 7f4b7861d640 1 -- 172.21.15.45:0/2413945665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b700ca250 msgr2=0x7f4b700ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:39.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.714+0000 7f4b7861d640 1 --2- 172.21.15.45:0/2413945665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b700ca250 0x7f4b700ca6d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400c760 tx=0x7f4b6400cc20 comp rx=0 tx=0).stop 2024-05-08T05:45:39.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.714+0000 7f4b7861d640 1 -- 172.21.15.45:0/2413945665 shutdown_connections 2024-05-08T05:45:39.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.714+0000 7f4b7861d640 1 --2- 172.21.15.45:0/2413945665 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b5c0762d0 0x7f4b5c078790 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.714+0000 7f4b7861d640 1 --2- 172.21.15.45:0/2413945665 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b7015c5d0 0x7f4b7015ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.714+0000 7f4b7861d640 1 --2- 172.21.15.45:0/2413945665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b700ca250 0x7f4b700ca6d0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.714+0000 7f4b7861d640 1 --2- 172.21.15.45:0/2413945665 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b700d2080 0x7f4b7015a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:39.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.714+0000 7f4b7861d640 1 -- 172.21.15.45:0/2413945665 >> 172.21.15.45:0/2413945665 conn(0x7f4b7008eac0 msgr2=0x7f4b700b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:39.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.714+0000 7f4b7861d640 1 -- 172.21.15.45:0/2413945665 shutdown_connections 2024-05-08T05:45:39.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39.714+0000 7f4b7861d640 1 -- 172.21.15.45:0/2413945665 wait complete. 2024-05-08T05:45:39.725 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T05:45:39.725 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:45:39.725 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-05-08T05:45:39.725 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T05:45:39.725 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:45:39.725 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T05:45:39.725 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T05:45:39.725 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:45:39.725 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:45:39.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:39.726 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-05-08T05:45:39.742 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:45:39.742 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:45:39.742 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:45:39.742 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:45:39.742 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-05-08T05:45:39.742 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:45:39.742 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:45:39.742 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:45:39.742 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-05-08T05:45:39.742 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:45:39.743 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:45:39.743 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-05-08T05:45:39.743 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:45:39.743 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-05-08T05:45:39.743 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-05-08T05:45:39.743 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:45:39.743 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42485sec preferred_lft 42485sec 2024-05-08T05:45:39.743 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-05-08T05:45:39.743 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:45:39.743 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:45:39.743 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-05-08T05:45:39.743 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-05-08T05:45:39.743 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:45:39.743 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T05:45:39.744 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T05:45:39.744 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:45:39.744 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c80a:aff:fe94:c77/64 scope link 2024-05-08T05:45:39.744 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:45:39.744 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:45:39.744 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 82:f3:eb:6a:57:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:45:39.744 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::80f3:ebff:fe6a:57ba/64 scope link 2024-05-08T05:45:39.744 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:45:39.744 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:45:39.744 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T05:45:39.744 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::14a2:64ff:feda:3a9c/64 scope link 2024-05-08T05:45:39.744 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:45:39.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:39.745 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:45:39.806 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:45:39.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:39.808 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-05-08T05:45:39.865 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-05-08T05:45:39.865 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:45:39.865 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:45:39.865 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:45:39.865 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:45:39.865 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:45:39.866 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:45:39.866 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:45:39.866 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:45:39.866 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:45:39.866 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:45:39.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:45:39.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:45:39.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:39.995 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:45:40.012 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T05:45:40.012 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T05:45:40.012 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:45:40.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:40.012 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-05-08T05:45:40.078 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:45:40.078 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T05:45:40.134 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:45:40.134 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:45:40.134 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:45:40.189 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T05:45:40.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:40.190 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:45:40.274 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:45:40.275 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:45:40.284 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:45:40.284 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:45:40.284 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-08T05:45:40.284 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@e9e1170a-fd27-4625-97a9-5d5f104d13a2.cephfs=/ 2024-05-08T05:45:40.284 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.63:3300/172.21.15.63:3301" will pass to kernel 2024-05-08T05:45:40.284 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T05:45:40.284 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:45:40.284 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:45:40.285 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:45:40.285 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:45:40.286 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:45:40.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:40.286 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:45:40.311 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:45:40.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:45:40.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:45:40.410 INFO:teuthology.orchestra.run.smithi045.stdout:client5543 2024-05-08T05:45:40.410 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:45:40.410 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:45:40.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:40.427 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-05-08T05:45:40.464 INFO:tasks.ceph.mds.a.smithi045.stderr:2024-05-08T05:45:40.464+0000 7f4cf5424640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:45:40.481 INFO:teuthology.orchestra.run.smithi045.stdout:file_undamaged 2024-05-08T05:45:40.482 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:45:40.482 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:45:40.482 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:45:40.482 DEBUG:teuthology.orchestra.run.smithi045:> import json 2024-05-08T05:45:40.482 DEBUG:teuthology.orchestra.run.smithi045:> import sys 2024-05-08T05:45:40.482 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:45:40.482 DEBUG:teuthology.orchestra.run.smithi045:> try: 2024-05-08T05:45:40.483 DEBUG:teuthology.orchestra.run.smithi045:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-05-08T05:45:40.483 DEBUG:teuthology.orchestra.run.smithi045:> except OSError as e: 2024-05-08T05:45:40.483 DEBUG:teuthology.orchestra.run.smithi045:> sys.exit(e.errno) 2024-05-08T05:45:40.483 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:45:40.483 DEBUG:teuthology.orchestra.run.smithi045:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-05-08T05:45:40.483 DEBUG:teuthology.orchestra.run.smithi045:> print(json.dumps( 2024-05-08T05:45:40.483 DEBUG:teuthology.orchestra.run.smithi045:> dict([(a, getattr(s, a)) for a in attrs]), 2024-05-08T05:45:40.483 DEBUG:teuthology.orchestra.run.smithi045:> indent=2)) 2024-05-08T05:45:40.483 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:45:40.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-05-08T05:45:40.601 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-08T05:45:40.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:40.601 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-05-08T05:45:40.659 INFO:teuthology.orchestra.run.smithi045.stdout:2 2024-05-08T05:45:40.659 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:45:40.659 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:45:40.715 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:45:40.716 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:45:40.772 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:45:40.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.919+0000 7fa18c015640 1 Processor -- start 2024-05-08T05:45:40.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.919+0000 7fa18c015640 1 -- start start 2024-05-08T05:45:40.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.920+0000 7fa18c015640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18406e0f0 0x7fa18415c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:40.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.920+0000 7fa18c015640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa184072840 0x7fa1840d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:40.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.920+0000 7fa18c015640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa18406e4d0 0x7fa1840b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:40.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.920+0000 7fa18c015640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa1840d14b0 con 0x7fa184072840 2024-05-08T05:45:40.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.920+0000 7fa18c015640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa18415c920 con 0x7fa18406e4d0 2024-05-08T05:45:40.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.920+0000 7fa18c015640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa18415cc00 con 0x7fa18406e0f0 2024-05-08T05:45:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.920+0000 7fa1895b0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa184072840 0x7fa1840d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.920+0000 7fa1895b0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa184072840 0x7fa1840d0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36306/0 (socket says 172.21.15.45:36306) 2024-05-08T05:45:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.920+0000 7fa1895b0640 1 -- 172.21.15.45:0/1926151346 learned_addr learned my addr 172.21.15.45:0/1926151346 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.920+0000 7fa189db1640 1 --2- 172.21.15.45:0/1926151346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18406e0f0 0x7fa18415c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.920+0000 7fa18a5b2640 1 --2- 172.21.15.45:0/1926151346 >> v2:172.21.15.63:3300/0 conn(0x7fa18406e4d0 0x7fa1840b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.920+0000 7fa1895b0640 1 -- 172.21.15.45:0/1926151346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18406e0f0 msgr2=0x7fa18415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.920+0000 7fa1895b0640 1 --2- 172.21.15.45:0/1926151346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18406e0f0 0x7fa18415c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.920+0000 7fa1895b0640 1 -- 172.21.15.45:0/1926151346 >> v2:172.21.15.63:3300/0 conn(0x7fa18406e4d0 msgr2=0x7fa1840b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.920+0000 7fa1895b0640 1 --2- 172.21.15.45:0/1926151346 >> v2:172.21.15.63:3300/0 conn(0x7fa18406e4d0 0x7fa1840b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.920+0000 7fa1895b0640 1 -- 172.21.15.45:0/1926151346 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18415cee0 con 0x7fa184072840 2024-05-08T05:45:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.920+0000 7fa189db1640 1 --2- 172.21.15.45:0/1926151346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18406e0f0 0x7fa18415c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.920+0000 7fa18a5b2640 1 --2- 172.21.15.45:0/1926151346 >> v2:172.21.15.63:3300/0 conn(0x7fa18406e4d0 0x7fa1840b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.921+0000 7fa1895b0640 1 --2- 172.21.15.45:0/1926151346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa184072840 0x7fa1840d0f70 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa184070580 tx=0x7fa17402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d59cf2450f6e3fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:40.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.921+0000 7fa188daf640 1 -- 172.21.15.45:0/1926151346 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa174037070 con 0x7fa184072840 2024-05-08T05:45:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.921+0000 7fa188daf640 1 -- 172.21.15.45:0/1926151346 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa174032440 con 0x7fa184072840 2024-05-08T05:45:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.921+0000 7fa188daf640 1 -- 172.21.15.45:0/1926151346 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa174032810 con 0x7fa184072840 2024-05-08T05:45:40.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.922+0000 7fa18c015640 1 -- 172.21.15.45:0/1926151346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa184072840 msgr2=0x7fa1840d0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.922+0000 7fa18c015640 1 --2- 172.21.15.45:0/1926151346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa184072840 0x7fa1840d0f70 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa184070580 tx=0x7fa17402a1e0 comp rx=0 tx=0).stop 2024-05-08T05:45:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.922+0000 7fa18c015640 1 -- 172.21.15.45:0/1926151346 shutdown_connections 2024-05-08T05:45:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.922+0000 7fa18c015640 1 --2- 172.21.15.45:0/1926151346 >> v2:172.21.15.63:3300/0 conn(0x7fa18406e4d0 0x7fa1840b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.922+0000 7fa18c015640 1 --2- 172.21.15.45:0/1926151346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa184072840 0x7fa1840d0f70 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.922+0000 7fa18c015640 1 --2- 172.21.15.45:0/1926151346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18406e0f0 0x7fa18415c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.922+0000 7fa18c015640 1 -- 172.21.15.45:0/1926151346 >> 172.21.15.45:0/1926151346 conn(0x7fa18406c3a0 msgr2=0x7fa18406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.922+0000 7fa18c015640 1 -- 172.21.15.45:0/1926151346 shutdown_connections 2024-05-08T05:45:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.922+0000 7fa18c015640 1 -- 172.21.15.45:0/1926151346 wait complete. 2024-05-08T05:45:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.923+0000 7fa18c015640 1 Processor -- start 2024-05-08T05:45:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.923+0000 7fa18c015640 1 -- start start 2024-05-08T05:45:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.924+0000 7fa18c015640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa18406e0f0 0x7fa184155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.924+0000 7fa18c015640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18406e4d0 0x7fa1840b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:40.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.924+0000 7fa18c015640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa184072840 0x7fa184154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.924+0000 7fa18c015640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa1840ae650 con 0x7fa18406e0f0 2024-05-08T05:45:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.924+0000 7fa18c015640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa18406e980 con 0x7fa184072840 2024-05-08T05:45:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.924+0000 7fa18c015640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa1840ae7d0 con 0x7fa18406e4d0 2024-05-08T05:45:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.924+0000 7fa189db1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa18406e0f0 0x7fa184155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.924+0000 7fa189db1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa18406e0f0 0x7fa184155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36320/0 (socket says 172.21.15.45:36320) 2024-05-08T05:45:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.924+0000 7fa189db1640 1 -- 172.21.15.45:0/266096763 learned_addr learned my addr 172.21.15.45:0/266096763 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.924+0000 7fa1895b0640 1 --2- 172.21.15.45:0/266096763 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18406e4d0 0x7fa1840b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.924+0000 7fa18a5b2640 1 --2- 172.21.15.45:0/266096763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa184072840 0x7fa184154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.924+0000 7fa189db1640 1 -- 172.21.15.45:0/266096763 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18406e4d0 msgr2=0x7fa1840b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.924+0000 7fa189db1640 1 --2- 172.21.15.45:0/266096763 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18406e4d0 0x7fa1840b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.924+0000 7fa189db1640 1 -- 172.21.15.45:0/266096763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa184072840 msgr2=0x7fa184154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.924+0000 7fa189db1640 1 --2- 172.21.15.45:0/266096763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa184072840 0x7fa184154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.924+0000 7fa189db1640 1 -- 172.21.15.45:0/266096763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18409c2b0 con 0x7fa18406e0f0 2024-05-08T05:45:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.924+0000 7fa189db1640 1 --2- 172.21.15.45:0/266096763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa18406e0f0 0x7fa184155c10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa18000c7f0 tx=0x7fa18000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:40.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.924+0000 7fa17a7fc640 1 -- 172.21.15.45:0/266096763 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180015070 con 0x7fa18406e0f0 2024-05-08T05:45:40.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.925+0000 7fa17a7fc640 1 -- 172.21.15.45:0/266096763 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa180007a30 con 0x7fa18406e0f0 2024-05-08T05:45:40.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.925+0000 7fa18c015640 1 -- 172.21.15.45:0/266096763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa18409c590 con 0x7fa18406e0f0 2024-05-08T05:45:40.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.925+0000 7fa17a7fc640 1 -- 172.21.15.45:0/266096763 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180007e00 con 0x7fa18406e0f0 2024-05-08T05:45:40.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.925+0000 7fa18c015640 1 -- 172.21.15.45:0/266096763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa18409cb50 con 0x7fa18406e0f0 2024-05-08T05:45:40.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.925+0000 7fa18c015640 1 -- 172.21.15.45:0/266096763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1840693f0 con 0x7fa18406e0f0 2024-05-08T05:45:40.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.926+0000 7fa17a7fc640 1 -- 172.21.15.45:0/266096763 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa180018070 con 0x7fa18406e0f0 2024-05-08T05:45:40.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.927+0000 7fa17a7fc640 1 --2- 172.21.15.45:0/266096763 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1680762d0 0x7fa168078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:40.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.927+0000 7fa17a7fc640 1 -- 172.21.15.45:0/266096763 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7fa180097790 con 0x7fa18406e0f0 2024-05-08T05:45:40.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.927+0000 7fa1895b0640 1 --2- 172.21.15.45:0/266096763 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1680762d0 0x7fa168078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:40.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.927+0000 7fa1895b0640 1 --2- 172.21.15.45:0/266096763 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1680762d0 0x7fa168078790 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa174034000 tx=0x7fa17402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:40.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:40.931+0000 7fa17a7fc640 1 -- 172.21.15.45:0/266096763 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa180060f10 con 0x7fa18406e0f0 2024-05-08T05:45:41.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.135+0000 7fa18c015640 1 -- 172.21.15.45:0/266096763 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa184073df0 con 0x7fa18406e0f0 2024-05-08T05:45:41.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.136+0000 7fa17a7fc640 1 -- 172.21.15.45:0/266096763 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v76) ==== 80+0+120 (secure 0 0 0) 0x7fa180060810 con 0x7fa18406e0f0 2024-05-08T05:45:41.136 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.63:6836/133050464 2024-05-09T05:45:33.480648+0000 2024-05-08T05:45:41.136 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.63:6834/133050464 2024-05-09T05:45:33.480648+0000 2024-05-08T05:45:41.136 INFO:teuthology.orchestra.run.smithi045.stderr:listed 2 entries 2024-05-08T05:45:41.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.138+0000 7fa18c015640 1 -- 172.21.15.45:0/266096763 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1680762d0 msgr2=0x7fa168078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.138+0000 7fa18c015640 1 --2- 172.21.15.45:0/266096763 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1680762d0 0x7fa168078790 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa174034000 tx=0x7fa17402e000 comp rx=0 tx=0).stop 2024-05-08T05:45:41.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.138+0000 7fa18c015640 1 -- 172.21.15.45:0/266096763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa18406e0f0 msgr2=0x7fa184155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.138+0000 7fa18c015640 1 --2- 172.21.15.45:0/266096763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa18406e0f0 0x7fa184155c10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa18000c7f0 tx=0x7fa18000cbf0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.138+0000 7fa18c015640 1 -- 172.21.15.45:0/266096763 shutdown_connections 2024-05-08T05:45:41.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.138+0000 7fa18c015640 1 --2- 172.21.15.45:0/266096763 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1680762d0 0x7fa168078790 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.138+0000 7fa18c015640 1 --2- 172.21.15.45:0/266096763 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa18406e4d0 0x7fa1840b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.138+0000 7fa18c015640 1 --2- 172.21.15.45:0/266096763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa184072840 0x7fa184154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.138+0000 7fa18c015640 1 --2- 172.21.15.45:0/266096763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa18406e0f0 0x7fa184155c10 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.138+0000 7fa18c015640 1 -- 172.21.15.45:0/266096763 >> 172.21.15.45:0/266096763 conn(0x7fa18406c3a0 msgr2=0x7fa1840b5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:41.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.138+0000 7fa18c015640 1 -- 172.21.15.45:0/266096763 shutdown_connections 2024-05-08T05:45:41.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.138+0000 7fa18c015640 1 -- 172.21.15.45:0/266096763 wait complete. 2024-05-08T05:45:41.149 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T05:45:41.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:41.149 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T05:45:41.179 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:45:41.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:41.179 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T05:45:41.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:41.235 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-05-08T05:45:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.382+0000 7f29c1e54640 1 Processor -- start 2024-05-08T05:45:41.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.383+0000 7f29c1e54640 1 -- start start 2024-05-08T05:45:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.383+0000 7f29c1e54640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f29bc162b70 0x7f29bc0d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.383+0000 7f29c1e54640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29bc0d3480 0x7f29bc0b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.383+0000 7f29c1e54640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29bc162f50 con 0x7f29bc0d3910 2024-05-08T05:45:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.383+0000 7f29c1e54640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f29bc0d1f60 con 0x7f29bc0d3480 2024-05-08T05:45:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.383+0000 7f29c1e54640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f29bc0d3ee0 con 0x7f29bc162b70 2024-05-08T05:45:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.384+0000 7f29bbfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f29bc0d3910 0x7f29bc0b4f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38028/0 (socket says 172.21.15.45:38028) 2024-05-08T05:45:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.384+0000 7f29bbfff640 1 -- 172.21.15.45:0/3144366728 learned_addr learned my addr 172.21.15.45:0/3144366728 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:45:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.384+0000 7f29baffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29bc0d3480 0x7f29bc0b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.384+0000 7f29bb7fe640 1 --2- 172.21.15.45:0/3144366728 >> v2:172.21.15.63:3300/0 conn(0x7f29bc162b70 0x7f29bc0d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.384+0000 7f29ba7fc640 1 -- 172.21.15.45:0/3144366728 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 228006983 0 0) 0x7f29b4001490 con 0x7f29bc0d3910 2024-05-08T05:45:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.384+0000 7f29ba7fc640 1 -- 172.21.15.45:0/3144366728 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29a4003520 con 0x7f29bc0d3910 2024-05-08T05:45:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.384+0000 7f29baffd640 1 -- 172.21.15.45:0/3144366728 >> v2:172.21.15.63:3300/0 conn(0x7f29bc162b70 msgr2=0x7f29bc0d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.384+0000 7f29baffd640 1 --2- 172.21.15.45:0/3144366728 >> v2:172.21.15.63:3300/0 conn(0x7f29bc162b70 0x7f29bc0d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.384+0000 7f29baffd640 1 -- 172.21.15.45:0/3144366728 >> v1:172.21.15.45:6789/0 conn(0x7f29bc0d3910 legacy=0x7f29bc0b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.384+0000 7f29baffd640 1 -- 172.21.15.45:0/3144366728 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29bc0d21b0 con 0x7f29bc0d3480 2024-05-08T05:45:41.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.385+0000 7f29baffd640 1 --2- 172.21.15.45:0/3144366728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29bc0d3480 0x7f29bc0b4a40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f29bc08f610 tx=0x7f29b002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a32db87d90bd2e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:41.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.385+0000 7f29ba7fc640 1 -- 172.21.15.45:0/3144366728 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0037070 con 0x7f29bc0d3480 2024-05-08T05:45:41.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.385+0000 7f29ba7fc640 1 -- 172.21.15.45:0/3144366728 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29b0032440 con 0x7f29bc0d3480 2024-05-08T05:45:41.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.385+0000 7f29ba7fc640 1 -- 172.21.15.45:0/3144366728 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0032810 con 0x7f29bc0d3480 2024-05-08T05:45:41.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.385+0000 7f29c1e54640 1 -- 172.21.15.45:0/3144366728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29bc0d3480 msgr2=0x7f29bc0b4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.385+0000 7f29c1e54640 1 --2- 172.21.15.45:0/3144366728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29bc0d3480 0x7f29bc0b4a40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f29bc08f610 tx=0x7f29b002a1a0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.385+0000 7f29c1e54640 1 -- 172.21.15.45:0/3144366728 shutdown_connections 2024-05-08T05:45:41.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.385+0000 7f29c1e54640 1 --2- 172.21.15.45:0/3144366728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29bc0d3480 0x7f29bc0b4a40 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.385+0000 7f29c1e54640 1 --2- 172.21.15.45:0/3144366728 >> v2:172.21.15.63:3300/0 conn(0x7f29bc162b70 0x7f29bc0d2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.385+0000 7f29c1e54640 1 -- 172.21.15.45:0/3144366728 >> 172.21.15.45:0/3144366728 conn(0x7f29bc0d5040 msgr2=0x7f29bc0d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.385+0000 7f29c1e54640 1 -- 172.21.15.45:0/3144366728 shutdown_connections 2024-05-08T05:45:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.385+0000 7f29c1e54640 1 -- 172.21.15.45:0/3144366728 wait complete. 2024-05-08T05:45:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.387+0000 7f29c1e54640 1 Processor -- start 2024-05-08T05:45:41.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.387+0000 7f29c1e54640 1 -- start start 2024-05-08T05:45:41.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.387+0000 7f29c1e54640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29bc0d3480 0x7f29bc0cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.388+0000 7f29c1e54640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29bc0d3910 0x7f29bc1582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.388+0000 7f29c1e54640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29bc162b70 0x7f29bc0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.388+0000 7f29c1e54640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc0b42f0 con 0x7f29bc0d3480 2024-05-08T05:45:41.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.388+0000 7f29c1e54640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc0b4170 con 0x7f29bc0d3910 2024-05-08T05:45:41.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.388+0000 7f29c1e54640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f29bc0d0c60 con 0x7f29bc162b70 2024-05-08T05:45:41.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.388+0000 7f29bb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29bc0d3480 0x7f29bc0cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.388+0000 7f29bb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29bc0d3480 0x7f29bc0cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36338/0 (socket says 172.21.15.45:36338) 2024-05-08T05:45:41.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.388+0000 7f29bb7fe640 1 -- 172.21.15.45:0/688365472 learned_addr learned my addr 172.21.15.45:0/688365472 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:41.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.388+0000 7f29baffd640 1 --2- 172.21.15.45:0/688365472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29bc0d3910 0x7f29bc1582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.388+0000 7f29bb7fe640 1 -- 172.21.15.45:0/688365472 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29bc162b70 msgr2=0x7f29bc0cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.388+0000 7f29bb7fe640 1 --2- 172.21.15.45:0/688365472 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29bc162b70 0x7f29bc0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.388+0000 7f29bb7fe640 1 -- 172.21.15.45:0/688365472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29bc0d3910 msgr2=0x7f29bc1582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.388+0000 7f29bb7fe640 1 --2- 172.21.15.45:0/688365472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29bc0d3910 0x7f29bc1582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.388+0000 7f29bb7fe640 1 -- 172.21.15.45:0/688365472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29bc0c9d50 con 0x7f29bc0d3480 2024-05-08T05:45:41.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.388+0000 7f29bb7fe640 1 --2- 172.21.15.45:0/688365472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29bc0d3480 0x7f29bc0cb5d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00d5f0 tx=0x7f29ac00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:41.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.388+0000 7f29a3fff640 1 -- 172.21.15.45:0/688365472 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac0125f0 con 0x7f29bc0d3480 2024-05-08T05:45:41.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.388+0000 7f29a3fff640 1 -- 172.21.15.45:0/688365472 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29ac019070 con 0x7f29bc0d3480 2024-05-08T05:45:41.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.388+0000 7f29c1e54640 1 -- 172.21.15.45:0/688365472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29bc0ca030 con 0x7f29bc0d3480 2024-05-08T05:45:41.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.388+0000 7f29a3fff640 1 -- 172.21.15.45:0/688365472 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac0129d0 con 0x7f29bc0d3480 2024-05-08T05:45:41.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.388+0000 7f29c1e54640 1 -- 172.21.15.45:0/688365472 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29bc0ca5f0 con 0x7f29bc0d3480 2024-05-08T05:45:41.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.390+0000 7f29a3fff640 1 -- 172.21.15.45:0/688365472 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f29ac012b70 con 0x7f29bc0d3480 2024-05-08T05:45:41.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.390+0000 7f29a3fff640 1 --2- 172.21.15.45:0/688365472 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f29a80762d0 0x7f29a8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.391+0000 7f29a3fff640 1 -- 172.21.15.45:0/688365472 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f29ac097ca0 con 0x7f29bc0d3480 2024-05-08T05:45:41.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.391+0000 7f29baffd640 1 --2- 172.21.15.45:0/688365472 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f29a80762d0 0x7f29a8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.391+0000 7f29baffd640 1 --2- 172.21.15.45:0/688365472 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f29a80762d0 0x7f29a8078790 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f29b0034000 tx=0x7f29b002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.391+0000 7f29c4b6f740 1 Processor -- start 2024-05-08T05:45:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.391+0000 7f29c4b6f740 1 -- start start 2024-05-08T05:45:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.392+0000 7f29c4b6f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a36ea0 0x5647d09e9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.392+0000 7f29c4b6f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647d0a19fc0 0x5647d0a19340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.392+0000 7f29c4b6f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a19880 0x5647d09ee690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.392+0000 7f29c4b6f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5647d09531c0 con 0x5647d0a36ea0 2024-05-08T05:45:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.392+0000 7f29c4b6f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5647d090cca0 con 0x5647d0a19880 2024-05-08T05:45:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.392+0000 7f29c4b6f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5647d094f180 con 0x5647d0a19fc0 2024-05-08T05:45:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.392+0000 7f29bb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a36ea0 0x5647d09e9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.392+0000 7f29bb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a36ea0 0x5647d09e9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36342/0 (socket says 172.21.15.45:36342) 2024-05-08T05:45:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.392+0000 7f29bb7fe640 1 -- 172.21.15.45:0/1608414059 learned_addr learned my addr 172.21.15.45:0/1608414059 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.392+0000 7f29baffd640 1 --2- 172.21.15.45:0/1608414059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647d0a19fc0 0x5647d0a19340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.393+0000 7f29bbfff640 1 --2- 172.21.15.45:0/1608414059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a19880 0x5647d09ee690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.393+0000 7f29bb7fe640 1 -- 172.21.15.45:0/1608414059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647d0a19fc0 msgr2=0x5647d0a19340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.393+0000 7f29bb7fe640 1 --2- 172.21.15.45:0/1608414059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647d0a19fc0 0x5647d0a19340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.393+0000 7f29bb7fe640 1 -- 172.21.15.45:0/1608414059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a19880 msgr2=0x5647d09ee690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.393+0000 7f29bb7fe640 1 --2- 172.21.15.45:0/1608414059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a19880 0x5647d09ee690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.393+0000 7f29bb7fe640 1 -- 172.21.15.45:0/1608414059 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647d08f0db0 con 0x5647d0a36ea0 2024-05-08T05:45:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.393+0000 7f29baffd640 1 --2- 172.21.15.45:0/1608414059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647d0a19fc0 0x5647d0a19340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.393+0000 7f29bbfff640 1 --2- 172.21.15.45:0/1608414059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a19880 0x5647d09ee690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.393+0000 7f29bb7fe640 1 --2- 172.21.15.45:0/1608414059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a36ea0 0x5647d09e9830 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f29ac05d310 tx=0x7f29ac060550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a127cef17469d728 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.393+0000 7f29a17fa640 1 -- 172.21.15.45:0/1608414059 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac069440 con 0x5647d0a36ea0 2024-05-08T05:45:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.393+0000 7f29a17fa640 1 -- 172.21.15.45:0/1608414059 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29ac078070 con 0x5647d0a36ea0 2024-05-08T05:45:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.393+0000 7f29a17fa640 1 -- 172.21.15.45:0/1608414059 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac011dc0 con 0x5647d0a36ea0 2024-05-08T05:45:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.394+0000 7f29c4b6f740 1 -- 172.21.15.45:0/1608414059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a36ea0 msgr2=0x5647d09e9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.394+0000 7f29c4b6f740 1 --2- 172.21.15.45:0/1608414059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a36ea0 0x5647d09e9830 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f29ac05d310 tx=0x7f29ac060550 comp rx=0 tx=0).stop 2024-05-08T05:45:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.394+0000 7f29c4b6f740 1 -- 172.21.15.45:0/1608414059 shutdown_connections 2024-05-08T05:45:41.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.394+0000 7f29c4b6f740 1 --2- 172.21.15.45:0/1608414059 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647d0a19fc0 0x5647d0a19340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.394+0000 7f29c4b6f740 1 --2- 172.21.15.45:0/1608414059 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a19880 0x5647d09ee690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.394+0000 7f29c4b6f740 1 --2- 172.21.15.45:0/1608414059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a36ea0 0x5647d09e9830 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.394+0000 7f29c4b6f740 1 -- 172.21.15.45:0/1608414059 >> 172.21.15.45:0/1608414059 conn(0x5647d09ebaa0 msgr2=0x5647d0a34340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:41.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.394+0000 7f29c4b6f740 1 -- 172.21.15.45:0/1608414059 shutdown_connections 2024-05-08T05:45:41.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.394+0000 7f29c4b6f740 1 -- 172.21.15.45:0/1608414059 wait complete. 2024-05-08T05:45:41.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.394+0000 7f29c4b6f740 1 -- start start 2024-05-08T05:45:41.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.395+0000 7f29c4b6f740 1 Processor -- start 2024-05-08T05:45:41.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.395+0000 7f29c4b6f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647d0a19fc0 0x5647d0a2d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.395+0000 7f29c4b6f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a36ea0 0x5647d0a16a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.395+0000 7f29c4b6f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 0x5647d0a2b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.395+0000 7f29c4b6f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5647d09edc50 con 0x5647d0a1c540 2024-05-08T05:45:41.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.395+0000 7f29c4b6f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5647d08d9be0 con 0x5647d0a36ea0 2024-05-08T05:45:41.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.395+0000 7f29c4b6f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5647d09ed040 con 0x5647d0a19fc0 2024-05-08T05:45:41.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.395+0000 7f29c4b6f740 10 client.0 _ll_register_callbacks cb 0x5647d0907810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:45:41.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.395+0000 7f29bbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 0x5647d0a2b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.395+0000 7f29bbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 0x5647d0a2b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36350/0 (socket says 172.21.15.45:36350) 2024-05-08T05:45:41.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.395+0000 7f29bbfff640 1 -- 172.21.15.45:0/3640503775 learned_addr learned my addr 172.21.15.45:0/3640503775 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:41.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f29baffd640 1 --2- 172.21.15.45:0/3640503775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a36ea0 0x5647d0a16a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f29bb7fe640 1 --2- 172.21.15.45:0/3640503775 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647d0a19fc0 0x5647d0a2d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f29bbfff640 1 -- 172.21.15.45:0/3640503775 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647d0a19fc0 msgr2=0x5647d0a2d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:41.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f29bbfff640 1 --2- 172.21.15.45:0/3640503775 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647d0a19fc0 0x5647d0a2d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f29bbfff640 1 -- 172.21.15.45:0/3640503775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a36ea0 msgr2=0x5647d0a16a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:41.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f29bbfff640 1 --2- 172.21.15.45:0/3640503775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a36ea0 0x5647d0a16a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f29bbfff640 1 -- 172.21.15.45:0/3640503775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647d0730a20 con 0x5647d0a1c540 2024-05-08T05:45:41.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f29bbfff640 1 --2- 172.21.15.45:0/3640503775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 0x5647d0a2b000 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f29b400e7f0 tx=0x7f29b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80cd653fd6311284 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:41.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f299a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:45:41.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f299a7fc640 1 -- 172.21.15.45:0/3640503775 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b400c8d0 con 0x5647d0a1c540 2024-05-08T05:45:41.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f299a7fc640 1 -- 172.21.15.45:0/3640503775 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f29b400ca70 con 0x5647d0a1c540 2024-05-08T05:45:41.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f29c4b6f740 1 -- 172.21.15.45:0/3640503775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29ac078070 con 0x5647d0a1c540 2024-05-08T05:45:41.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f299a7fc640 1 -- 172.21.15.45:0/3640503775 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f29b4020050 con 0x5647d0a1c540 2024-05-08T05:45:41.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f29c4b6f740 10 client.6417 fetch_fsmap learned FSMap version 50 2024-05-08T05:45:41.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f29c4b6f740 1 -- 172.21.15.45:0/3640503775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x5647d0624310 con 0x5647d0a1c540 2024-05-08T05:45:41.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f299a7fc640 1 -- 172.21.15.45:0/3640503775 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29ac078070 con 0x5647d0a1c540 2024-05-08T05:45:41.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f299a7fc640 1 -- 172.21.15.45:0/3640503775 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f29b4023020 con 0x5647d0a1c540 2024-05-08T05:45:41.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f29c4b6f740 10 client.6417 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T05:45:41.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f29c4b6f740 10 client.6417 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-08T05:45:41.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f29c4b6f740 20 client.6417 populate_metadata read hostname 'smithi045' 2024-05-08T05:45:41.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f29c4b6f740 1 --2- 172.21.15.45:0/3640503775 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x5647d0a48860 0x5647d0a14570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f29c4b6f740 4 client.6417 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:45:41.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.396+0000 7f29c4b6f740 1 -- 172.21.15.45:0/3640503775 --> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647d09b0570 con 0x5647d0a48860 2024-05-08T05:45:41.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.397+0000 7f29bb7fe640 1 --2- 172.21.15.45:0/3640503775 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x5647d0a48860 0x5647d0a14570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.397+0000 7f29bb7fe640 1 --2- 172.21.15.45:0/3640503775 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x5647d0a48860 0x5647d0a14570 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4b3256947f0417b server_cookie=9fdcfd75cb9715a2 in_seq=0 out_seq=0 2024-05-08T05:45:41.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.397+0000 7f299a7fc640 10 client.6417 ms_handle_connect on v2:172.21.15.45:6833/4183668301 2024-05-08T05:45:41.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.397+0000 7f299a7fc640 1 -- 172.21.15.45:0/3640503775 <== mds.0 v2:172.21.15.45:6833/4183668301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5647d09b0570 con 0x5647d0a48860 2024-05-08T05:45:41.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.397+0000 7f299a7fc640 10 client.6417 handle_command_reply: tid=0 2024-05-08T05:45:41.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.397+0000 7f29c4b6f740 1 client.6417 shutdown 2024-05-08T05:45:41.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.398+0000 7f29c4b6f740 1 -- 172.21.15.45:0/3640503775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 msgr2=0x5647d0a2b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.398+0000 7f29c4b6f740 1 --2- 172.21.15.45:0/3640503775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 0x5647d0a2b000 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f29b400e7f0 tx=0x7f29b400ecb0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.398+0000 7f29c4b6f740 1 -- 172.21.15.45:0/3640503775 shutdown_connections 2024-05-08T05:45:41.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.398+0000 7f29c4b6f740 1 --2- 172.21.15.45:0/3640503775 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x5647d0a48860 0x5647d0a14570 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.398+0000 7f29c4b6f740 1 --2- 172.21.15.45:0/3640503775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 0x5647d0a2b000 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.398+0000 7f29c4b6f740 1 --2- 172.21.15.45:0/3640503775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a36ea0 0x5647d0a16a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.398+0000 7f299a7fc640 0 client.6417 ms_handle_reset on v2:172.21.15.45:6833/4183668301 2024-05-08T05:45:41.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.398+0000 7f29c4b6f740 1 --2- 172.21.15.45:0/3640503775 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647d0a19fc0 0x5647d0a2d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.398+0000 7f29c4b6f740 1 -- 172.21.15.45:0/3640503775 >> 172.21.15.45:0/3640503775 conn(0x5647d0a19880 msgr2=0x5647d0a37760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:41.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.398+0000 7f29c4b6f740 1 -- 172.21.15.45:0/3640503775 shutdown_connections 2024-05-08T05:45:41.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.398+0000 7f29c4b6f740 1 -- 172.21.15.45:0/3640503775 wait complete. 2024-05-08T05:45:41.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.398+0000 7f29c4b6f740 20 client.6417 trim_cache size 0 max 16384 2024-05-08T05:45:41.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.416+0000 7f29c4b6f740 1 Processor -- start 2024-05-08T05:45:41.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.416+0000 7f29c4b6f740 1 -- start start 2024-05-08T05:45:41.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.417+0000 7f29c4b6f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647d0a19fc0 0x5647d0a36340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.417+0000 7f29c4b6f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 0x5647d09edea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.417+0000 7f29c4b6f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a36ea0 0x5647d09ee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.417+0000 7f29c4b6f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc201fe0 con 0x5647d0a1c540 2024-05-08T05:45:41.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.417+0000 7f29c4b6f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc2010a0 con 0x5647d0a36ea0 2024-05-08T05:45:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.417+0000 7f29c4b6f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f29bc0ccfe0 con 0x5647d0a19fc0 2024-05-08T05:45:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.417+0000 7f29baffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 0x5647d09edea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.417+0000 7f29baffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 0x5647d09edea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36356/0 (socket says 172.21.15.45:36356) 2024-05-08T05:45:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.417+0000 7f29baffd640 1 -- 172.21.15.45:0/571128909 learned_addr learned my addr 172.21.15.45:0/571128909 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.417+0000 7f29bbfff640 1 --2- 172.21.15.45:0/571128909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a36ea0 0x5647d09ee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.417+0000 7f29baffd640 1 -- 172.21.15.45:0/571128909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647d0a19fc0 msgr2=0x5647d0a36340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.417+0000 7f29baffd640 1 --2- 172.21.15.45:0/571128909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647d0a19fc0 0x5647d0a36340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.417+0000 7f29baffd640 1 -- 172.21.15.45:0/571128909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a36ea0 msgr2=0x5647d09ee3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.417+0000 7f29baffd640 1 --2- 172.21.15.45:0/571128909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a36ea0 0x5647d09ee3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.417+0000 7f29baffd640 1 -- 172.21.15.45:0/571128909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647d09f0d20 con 0x5647d0a1c540 2024-05-08T05:45:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.418+0000 7f29baffd640 1 --2- 172.21.15.45:0/571128909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 0x5647d09edea0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009890 tx=0x7f29b0008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850705ef0c135815 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.418+0000 7f29a0ff9640 1 -- 172.21.15.45:0/571128909 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b004f070 con 0x5647d0a1c540 2024-05-08T05:45:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.418+0000 7f29a0ff9640 1 -- 172.21.15.45:0/571128909 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29b004a470 con 0x5647d0a1c540 2024-05-08T05:45:41.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.418+0000 7f29a0ff9640 1 -- 172.21.15.45:0/571128909 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0037040 con 0x5647d0a1c540 2024-05-08T05:45:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.418+0000 7f29c4b6f740 1 -- 172.21.15.45:0/571128909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 msgr2=0x5647d09edea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.418+0000 7f29c4b6f740 1 --2- 172.21.15.45:0/571128909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 0x5647d09edea0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009890 tx=0x7f29b0008030 comp rx=0 tx=0).stop 2024-05-08T05:45:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.418+0000 7f29c4b6f740 1 -- 172.21.15.45:0/571128909 shutdown_connections 2024-05-08T05:45:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.418+0000 7f29c4b6f740 1 --2- 172.21.15.45:0/571128909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a36ea0 0x5647d09ee3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.418+0000 7f29c4b6f740 1 --2- 172.21.15.45:0/571128909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 0x5647d09edea0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.418+0000 7f29c4b6f740 1 --2- 172.21.15.45:0/571128909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647d0a19fc0 0x5647d0a36340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.418+0000 7f29c4b6f740 1 -- 172.21.15.45:0/571128909 >> 172.21.15.45:0/571128909 conn(0x5647d0a19880 msgr2=0x5647d09ef0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.419+0000 7f29c4b6f740 1 -- 172.21.15.45:0/571128909 shutdown_connections 2024-05-08T05:45:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.419+0000 7f29c4b6f740 1 -- 172.21.15.45:0/571128909 wait complete. 2024-05-08T05:45:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.419+0000 7f29c4b6f740 1 -- start start 2024-05-08T05:45:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.419+0000 7f29c4b6f740 1 Processor -- start 2024-05-08T05:45:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.420+0000 7f29c4b6f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647d0a19fc0 0x5647d0ab0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.420+0000 7f29c4b6f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 0x5647d09e8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.420+0000 7f29c4b6f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a36ea0 0x5647d09e8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.420+0000 7f29c4b6f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc0cd630 con 0x5647d0a1c540 2024-05-08T05:45:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.420+0000 7f29c4b6f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5647d0a19de0 con 0x5647d0a36ea0 2024-05-08T05:45:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.420+0000 7f29c4b6f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5647d0a2c5f0 con 0x5647d0a19fc0 2024-05-08T05:45:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.420+0000 7f29c4b6f740 10 client.0 _ll_register_callbacks cb 0x7f29b4001a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:45:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.420+0000 7f29baffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 0x5647d09e8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.420+0000 7f29baffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 0x5647d09e8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36360/0 (socket says 172.21.15.45:36360) 2024-05-08T05:45:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.420+0000 7f29baffd640 1 -- 172.21.15.45:0/3696610289 learned_addr learned my addr 172.21.15.45:0/3696610289 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.420+0000 7f29bb7fe640 1 --2- 172.21.15.45:0/3696610289 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647d0a19fc0 0x5647d0ab0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.420+0000 7f29bbfff640 1 --2- 172.21.15.45:0/3696610289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a36ea0 0x5647d09e8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.420+0000 7f29baffd640 1 -- 172.21.15.45:0/3696610289 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647d0a19fc0 msgr2=0x5647d0ab0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.420+0000 7f29baffd640 1 --2- 172.21.15.45:0/3696610289 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647d0a19fc0 0x5647d0ab0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.420+0000 7f29baffd640 1 -- 172.21.15.45:0/3696610289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a36ea0 msgr2=0x5647d09e8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:41.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.420+0000 7f29baffd640 1 --2- 172.21.15.45:0/3696610289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a36ea0 0x5647d09e8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.420+0000 7f29baffd640 1 -- 172.21.15.45:0/3696610289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647d07911b0 con 0x5647d0a1c540 2024-05-08T05:45:41.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.421+0000 7f29baffd640 1 --2- 172.21.15.45:0/3696610289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 0x5647d09e8600 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f29b0000c00 tx=0x7f29b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5ccf6e2efd0775 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:41.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.421+0000 7f299a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:45:41.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.421+0000 7f299a7fc640 1 -- 172.21.15.45:0/3696610289 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0037070 con 0x5647d0a1c540 2024-05-08T05:45:41.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.421+0000 7f299a7fc640 1 -- 172.21.15.45:0/3696610289 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f29b004a620 con 0x5647d0a1c540 2024-05-08T05:45:41.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.421+0000 7f29c4b6f740 1 -- 172.21.15.45:0/3696610289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29b004a470 con 0x5647d0a1c540 2024-05-08T05:45:41.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.421+0000 7f299a7fc640 1 -- 172.21.15.45:0/3696610289 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f29b005d050 con 0x5647d0a1c540 2024-05-08T05:45:41.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.421+0000 7f29c4b6f740 10 client.6423 fetch_fsmap learned FSMap version 50 2024-05-08T05:45:41.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.421+0000 7f29c4b6f740 1 -- 172.21.15.45:0/3696610289 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x5647d05fd490 con 0x5647d0a1c540 2024-05-08T05:45:41.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.421+0000 7f299a7fc640 1 -- 172.21.15.45:0/3696610289 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29b004a470 con 0x5647d0a1c540 2024-05-08T05:45:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.421+0000 7f299a7fc640 1 -- 172.21.15.45:0/3696610289 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7f29b0060020 con 0x5647d0a1c540 2024-05-08T05:45:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.421+0000 7f29c4b6f740 10 client.6423 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T05:45:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.421+0000 7f29c4b6f740 10 client.6423 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-08T05:45:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.421+0000 7f29c4b6f740 20 client.6423 populate_metadata read hostname 'smithi045' 2024-05-08T05:45:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.421+0000 7f29c4b6f740 1 --2- 172.21.15.45:0/3696610289 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x5647d09e9270 0x5647d0a46750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.421+0000 7f29c4b6f740 4 client.6423 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-05-08T05:45:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.421+0000 7f29c4b6f740 1 -- 172.21.15.45:0/3696610289 --> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) -- 0x5647d08f1910 con 0x5647d09e9270 2024-05-08T05:45:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.421+0000 7f29bb7fe640 1 --2- 172.21.15.45:0/3696610289 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x5647d09e9270 0x5647d0a46750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.422+0000 7f299a7fc640 10 client.6423 ms_handle_connect on v2:172.21.15.45:6833/4183668301 2024-05-08T05:45:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.422+0000 7f29bb7fe640 1 --2- 172.21.15.45:0/3696610289 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x5647d09e9270 0x5647d0a46750 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea72113923cce818 server_cookie=b74621e65da18950 in_seq=0 out_seq=0 2024-05-08T05:45:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.422+0000 7f299a7fc640 1 -- 172.21.15.45:0/3696610289 <== mds.0 v2:172.21.15.45:6833/4183668301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5647d08f1910 con 0x5647d09e9270 2024-05-08T05:45:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.422+0000 7f299a7fc640 10 client.6423 handle_command_reply: tid=0 2024-05-08T05:45:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.422+0000 7f29c4b6f740 1 client.6423 shutdown 2024-05-08T05:45:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.422+0000 7f29c4b6f740 1 -- 172.21.15.45:0/3696610289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 msgr2=0x5647d09e8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.422+0000 7f29c4b6f740 1 --2- 172.21.15.45:0/3696610289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 0x5647d09e8600 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f29b0000c00 tx=0x7f29b0004730 comp rx=0 tx=0).stop 2024-05-08T05:45:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.422+0000 7f29c4b6f740 1 -- 172.21.15.45:0/3696610289 shutdown_connections 2024-05-08T05:45:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.422+0000 7f29c4b6f740 1 --2- 172.21.15.45:0/3696610289 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x5647d09e9270 0x5647d0a46750 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.422+0000 7f29c4b6f740 1 --2- 172.21.15.45:0/3696610289 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5647d0a36ea0 0x5647d09e8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.422+0000 7f29c4b6f740 1 --2- 172.21.15.45:0/3696610289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5647d0a1c540 0x5647d09e8600 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.422+0000 7f299a7fc640 0 client.6423 ms_handle_reset on v2:172.21.15.45:6833/4183668301 2024-05-08T05:45:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.422+0000 7f29c4b6f740 1 --2- 172.21.15.45:0/3696610289 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5647d0a19fc0 0x5647d0ab0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.422+0000 7f29c4b6f740 1 -- 172.21.15.45:0/3696610289 >> 172.21.15.45:0/3696610289 conn(0x5647d0a19880 msgr2=0x5647d0a144c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.423+0000 7f29c4b6f740 1 -- 172.21.15.45:0/3696610289 shutdown_connections 2024-05-08T05:45:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.423+0000 7f29c4b6f740 1 -- 172.21.15.45:0/3696610289 wait complete. 2024-05-08T05:45:41.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.423+0000 7f29c4b6f740 20 client.6423 trim_cache size 0 max 16384 2024-05-08T05:45:41.430 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:45:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-05-08T05:45:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "ab809561-9fd1-4d4e-972c-6b97041fa1d5", 2024-05-08T05:45:41.430 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-05-08T05:45:41.430 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:45:41.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.423+0000 7f29c1e54640 1 -- 172.21.15.45:0/688365472 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f29a80762d0 msgr2=0x7f29a8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.423+0000 7f29c1e54640 1 --2- 172.21.15.45:0/688365472 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f29a80762d0 0x7f29a8078790 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f29b0034000 tx=0x7f29b002e000 comp rx=0 tx=0).stop 2024-05-08T05:45:41.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.423+0000 7f29c1e54640 1 -- 172.21.15.45:0/688365472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29bc0d3480 msgr2=0x7f29bc0cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.423+0000 7f29c1e54640 1 --2- 172.21.15.45:0/688365472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29bc0d3480 0x7f29bc0cb5d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00d5f0 tx=0x7f29ac00dab0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.423+0000 7f29c1e54640 1 -- 172.21.15.45:0/688365472 shutdown_connections 2024-05-08T05:45:41.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.423+0000 7f29c1e54640 1 --2- 172.21.15.45:0/688365472 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f29a80762d0 0x7f29a8078790 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.423+0000 7f29c1e54640 1 --2- 172.21.15.45:0/688365472 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29bc162b70 0x7f29bc0cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.423+0000 7f29c1e54640 1 --2- 172.21.15.45:0/688365472 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29bc0d3910 0x7f29bc1582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.423+0000 7f29c1e54640 1 --2- 172.21.15.45:0/688365472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29bc0d3480 0x7f29bc0cb5d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.423+0000 7f29c1e54640 1 -- 172.21.15.45:0/688365472 >> 172.21.15.45:0/688365472 conn(0x7f29bc0d5040 msgr2=0x7f29bc0b4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:41.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.423+0000 7f29c1e54640 1 -- 172.21.15.45:0/688365472 shutdown_connections 2024-05-08T05:45:41.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.423+0000 7f29c1e54640 1 -- 172.21.15.45:0/688365472 wait complete. 2024-05-08T05:45:41.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:45:41.434 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-08T05:45:41.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.541+0000 7fdb2ab0a640 1 Processor -- start 2024-05-08T05:45:41.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.541+0000 7fdb2ab0a640 1 -- start start 2024-05-08T05:45:41.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.542+0000 7fdb2ab0a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdb24070d30 0x7fdb240d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.542+0000 7fdb2ab0a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdb24163920 0x7fdb240703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.542+0000 7fdb2ab0a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb240708e0 con 0x7fdb24071110 2024-05-08T05:45:41.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.542+0000 7fdb2ab0a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fdb240d10d0 con 0x7fdb24163920 2024-05-08T05:45:41.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.542+0000 7fdb2ab0a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fdb24163db0 con 0x7fdb24070d30 2024-05-08T05:45:41.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.542+0000 7fdb23fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdb24163920 0x7fdb240703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.542+0000 7fdb23fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fdb24163920 0x7fdb240703a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36368/0 (socket says 172.21.15.45:36368) 2024-05-08T05:45:41.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.542+0000 7fdb23fff640 1 -- 172.21.15.45:0/252796783 learned_addr learned my addr 172.21.15.45:0/252796783 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:41.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.542+0000 7fdb288a6640 1 --2- 172.21.15.45:0/252796783 >> v2:172.21.15.63:3300/0 conn(0x7fdb24070d30 0x7fdb240d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.542+0000 7fdb23fff640 1 -- 172.21.15.45:0/252796783 >> v2:172.21.15.63:3300/0 conn(0x7fdb24070d30 msgr2=0x7fdb240d20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:41.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.542+0000 7fdb23fff640 1 --2- 172.21.15.45:0/252796783 >> v2:172.21.15.63:3300/0 conn(0x7fdb24070d30 0x7fdb240d20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.542+0000 7fdb23fff640 1 -- 172.21.15.45:0/252796783 >> v1:172.21.15.63:6789/0 conn(0x7fdb24071110 legacy=0x7fdb2406d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:41.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.542+0000 7fdb23fff640 1 -- 172.21.15.45:0/252796783 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb240d1300 con 0x7fdb24163920 2024-05-08T05:45:41.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.543+0000 7fdb23fff640 1 --2- 172.21.15.45:0/252796783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb24163920 0x7fdb240703a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fdb240d3f90 tx=0x7fdb1002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2c88c1fadc833b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:41.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.543+0000 7fdb237fe640 1 -- 172.21.15.45:0/252796783 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10037070 con 0x7fdb24163920 2024-05-08T05:45:41.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.543+0000 7fdb237fe640 1 -- 172.21.15.45:0/252796783 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb10032440 con 0x7fdb24163920 2024-05-08T05:45:41.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.543+0000 7fdb237fe640 1 -- 172.21.15.45:0/252796783 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10032810 con 0x7fdb24163920 2024-05-08T05:45:41.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.544+0000 7fdb2ab0a640 1 -- 172.21.15.45:0/252796783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb24163920 msgr2=0x7fdb240703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.544+0000 7fdb2ab0a640 1 --2- 172.21.15.45:0/252796783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb24163920 0x7fdb240703a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fdb240d3f90 tx=0x7fdb1002a1e0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.544+0000 7fdb2ab0a640 1 -- 172.21.15.45:0/252796783 shutdown_connections 2024-05-08T05:45:41.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.544+0000 7fdb2ab0a640 1 --2- 172.21.15.45:0/252796783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb24163920 0x7fdb240703a0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.544+0000 7fdb2ab0a640 1 --2- 172.21.15.45:0/252796783 >> v2:172.21.15.63:3300/0 conn(0x7fdb24070d30 0x7fdb240d20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.544+0000 7fdb2ab0a640 1 -- 172.21.15.45:0/252796783 >> 172.21.15.45:0/252796783 conn(0x7fdb240d2e90 msgr2=0x7fdb240d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:41.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.544+0000 7fdb2ab0a640 1 -- 172.21.15.45:0/252796783 shutdown_connections 2024-05-08T05:45:41.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.544+0000 7fdb2ab0a640 1 -- 172.21.15.45:0/252796783 wait complete. 2024-05-08T05:45:41.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.545+0000 7fdb2ab0a640 1 Processor -- start 2024-05-08T05:45:41.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.545+0000 7fdb2ab0a640 1 -- start start 2024-05-08T05:45:41.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.546+0000 7fdb2ab0a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb24070d30 0x7fdb240b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.546+0000 7fdb2ab0a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb24071110 0x7fdb240b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.546+0000 7fdb2ab0a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb24163920 0x7fdb240b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.546+0000 7fdb2ab0a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdb240d1c30 con 0x7fdb24071110 2024-05-08T05:45:41.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.546+0000 7fdb2ab0a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24070b80 con 0x7fdb24163920 2024-05-08T05:45:41.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.546+0000 7fdb2ab0a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdb240d18c0 con 0x7fdb24070d30 2024-05-08T05:45:41.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.546+0000 7fdb23fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb24071110 0x7fdb240b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.546+0000 7fdb23fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb24071110 0x7fdb240b0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36384/0 (socket says 172.21.15.45:36384) 2024-05-08T05:45:41.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.546+0000 7fdb23fff640 1 -- 172.21.15.45:0/2389139293 learned_addr learned my addr 172.21.15.45:0/2389139293 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:41.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.546+0000 7fdb288a6640 1 --2- 172.21.15.45:0/2389139293 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb24070d30 0x7fdb240b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.546+0000 7fdb290a7640 1 --2- 172.21.15.45:0/2389139293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb24163920 0x7fdb240b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.546+0000 7fdb23fff640 1 -- 172.21.15.45:0/2389139293 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb24070d30 msgr2=0x7fdb240b70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.546+0000 7fdb23fff640 1 --2- 172.21.15.45:0/2389139293 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb24070d30 0x7fdb240b70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.546+0000 7fdb23fff640 1 -- 172.21.15.45:0/2389139293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb24163920 msgr2=0x7fdb240b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.546+0000 7fdb23fff640 1 --2- 172.21.15.45:0/2389139293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb24163920 0x7fdb240b11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.546+0000 7fdb23fff640 1 -- 172.21.15.45:0/2389139293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb240b6030 con 0x7fdb24071110 2024-05-08T05:45:41.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.546+0000 7fdb288a6640 1 --2- 172.21.15.45:0/2389139293 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb24070d30 0x7fdb240b70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:45:41.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.547+0000 7fdb23fff640 1 --2- 172.21.15.45:0/2389139293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb24071110 0x7fdb240b0c70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fdb10000c00 tx=0x7fdb100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:41.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.547+0000 7fdb217fa640 1 -- 172.21.15.45:0/2389139293 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10041020 con 0x7fdb24071110 2024-05-08T05:45:41.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.547+0000 7fdb217fa640 1 -- 172.21.15.45:0/2389139293 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb10030970 con 0x7fdb24071110 2024-05-08T05:45:41.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.547+0000 7fdb2ab0a640 1 -- 172.21.15.45:0/2389139293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb240b6310 con 0x7fdb24071110 2024-05-08T05:45:41.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.547+0000 7fdb217fa640 1 -- 172.21.15.45:0/2389139293 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10037040 con 0x7fdb24071110 2024-05-08T05:45:41.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.547+0000 7fdb2ab0a640 1 -- 172.21.15.45:0/2389139293 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb240b68d0 con 0x7fdb24071110 2024-05-08T05:45:41.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.548+0000 7fdb217fa640 1 -- 172.21.15.45:0/2389139293 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdb10044020 con 0x7fdb24071110 2024-05-08T05:45:41.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.549+0000 7fdb217fa640 1 --2- 172.21.15.45:0/2389139293 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdb080762d0 0x7fdb08078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.549+0000 7fdb217fa640 1 -- 172.21.15.45:0/2389139293 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7fdb100b8080 con 0x7fdb24071110 2024-05-08T05:45:41.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.549+0000 7fdb288a6640 1 --2- 172.21.15.45:0/2389139293 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdb080762d0 0x7fdb08078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.550+0000 7fdb288a6640 1 --2- 172.21.15.45:0/2389139293 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdb080762d0 0x7fdb08078790 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdb18006fd0 tx=0x7fdb18008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:41.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.550+0000 7fdb2d825740 1 Processor -- start 2024-05-08T05:45:41.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.550+0000 7fdb2d825740 1 -- start start 2024-05-08T05:45:41.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.551+0000 7fdb2d825740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f730160 0x56128f6e2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.551+0000 7fdb2d825740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f713280 0x56128f712600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.551+0000 7fdb2d825740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f712b40 0x56128f6e7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.551+0000 7fdb2d825740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56128f6af690 con 0x56128f713280 2024-05-08T05:45:41.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.551+0000 7fdb2d825740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56128f64cf30 con 0x56128f730160 2024-05-08T05:45:41.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.551+0000 7fdb2d825740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56128f605b20 con 0x56128f712b40 2024-05-08T05:45:41.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.551+0000 7fdb23fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f713280 0x56128f712600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.551+0000 7fdb23fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f713280 0x56128f712600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36398/0 (socket says 172.21.15.45:36398) 2024-05-08T05:45:41.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.551+0000 7fdb23fff640 1 -- 172.21.15.45:0/2228387944 learned_addr learned my addr 172.21.15.45:0/2228387944 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:41.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.551+0000 7fdb288a6640 1 --2- 172.21.15.45:0/2228387944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f730160 0x56128f6e2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.551+0000 7fdb290a7640 1 --2- 172.21.15.45:0/2228387944 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f712b40 0x56128f6e7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.551+0000 7fdb23fff640 1 -- 172.21.15.45:0/2228387944 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f712b40 msgr2=0x56128f6e7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.551+0000 7fdb23fff640 1 --2- 172.21.15.45:0/2228387944 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f712b40 0x56128f6e7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.551+0000 7fdb23fff640 1 -- 172.21.15.45:0/2228387944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f730160 msgr2=0x56128f6e2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.551+0000 7fdb23fff640 1 --2- 172.21.15.45:0/2228387944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f730160 0x56128f6e2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.551+0000 7fdb23fff640 1 -- 172.21.15.45:0/2228387944 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56128f5fde60 con 0x56128f713280 2024-05-08T05:45:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.551+0000 7fdb23fff640 1 --2- 172.21.15.45:0/2228387944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f713280 0x56128f712600 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fdb100b63d0 tx=0x7fdb10081020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569531745f704896 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.551+0000 7fdb0e7fc640 1 -- 172.21.15.45:0/2228387944 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10081890 con 0x56128f713280 2024-05-08T05:45:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.551+0000 7fdb0e7fc640 1 -- 172.21.15.45:0/2228387944 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb10081190 con 0x56128f713280 2024-05-08T05:45:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.551+0000 7fdb0e7fc640 1 -- 172.21.15.45:0/2228387944 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10085540 con 0x56128f713280 2024-05-08T05:45:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.552+0000 7fdb2d825740 1 -- 172.21.15.45:0/2228387944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f713280 msgr2=0x56128f712600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.552+0000 7fdb2d825740 1 --2- 172.21.15.45:0/2228387944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f713280 0x56128f712600 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fdb100b63d0 tx=0x7fdb10081020 comp rx=0 tx=0).stop 2024-05-08T05:45:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.552+0000 7fdb2d825740 1 -- 172.21.15.45:0/2228387944 shutdown_connections 2024-05-08T05:45:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.552+0000 7fdb2d825740 1 --2- 172.21.15.45:0/2228387944 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f712b40 0x56128f6e7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.552+0000 7fdb2d825740 1 --2- 172.21.15.45:0/2228387944 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f713280 0x56128f712600 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.552+0000 7fdb2d825740 1 --2- 172.21.15.45:0/2228387944 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f730160 0x56128f6e2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.552+0000 7fdb2d825740 1 -- 172.21.15.45:0/2228387944 >> 172.21.15.45:0/2228387944 conn(0x56128f6e4d60 msgr2=0x56128f72d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:41.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.552+0000 7fdb2d825740 1 -- 172.21.15.45:0/2228387944 shutdown_connections 2024-05-08T05:45:41.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.552+0000 7fdb2d825740 1 -- 172.21.15.45:0/2228387944 wait complete. 2024-05-08T05:45:41.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.553+0000 7fdb2d825740 1 -- start start 2024-05-08T05:45:41.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.553+0000 7fdb2d825740 1 Processor -- start 2024-05-08T05:45:41.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb2d825740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f713280 0x56128f713770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb2d825740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f730160 0x56128f711d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb2d825740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f72f110 0x56128f715840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb2d825740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56128f6095e0 con 0x56128f72f110 2024-05-08T05:45:41.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb2d825740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56128f648ef0 con 0x56128f730160 2024-05-08T05:45:41.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb2d825740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56128f6e6f10 con 0x56128f713280 2024-05-08T05:45:41.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb2d825740 10 client.0 _ll_register_callbacks cb 0x56128f604320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:45:41.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb290a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f72f110 0x56128f715840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb290a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f72f110 0x56128f715840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36412/0 (socket says 172.21.15.45:36412) 2024-05-08T05:45:41.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb290a7640 1 -- 172.21.15.45:0/2340733813 learned_addr learned my addr 172.21.15.45:0/2340733813 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:41.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb23fff640 1 --2- 172.21.15.45:0/2340733813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f730160 0x56128f711d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb288a6640 1 --2- 172.21.15.45:0/2340733813 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f713280 0x56128f713770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb290a7640 1 -- 172.21.15.45:0/2340733813 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f713280 msgr2=0x56128f713770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:41.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb290a7640 1 --2- 172.21.15.45:0/2340733813 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f713280 0x56128f713770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb290a7640 1 -- 172.21.15.45:0/2340733813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f730160 msgr2=0x56128f711d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:41.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb290a7640 1 --2- 172.21.15.45:0/2340733813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f730160 0x56128f711d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb290a7640 1 -- 172.21.15.45:0/2340733813 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56128f4287d0 con 0x56128f72f110 2024-05-08T05:45:41.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb290a7640 1 --2- 172.21.15.45:0/2340733813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f72f110 0x56128f715840 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00bc10 tx=0x7fdb1c00a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed3f5f5802479fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:41.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb037fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:45:41.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb037fe640 1 -- 172.21.15.45:0/2340733813 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c01a030 con 0x56128f72f110 2024-05-08T05:45:41.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb037fe640 1 -- 172.21.15.45:0/2340733813 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7fdb1c003970 con 0x56128f72f110 2024-05-08T05:45:41.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.554+0000 7fdb2d825740 1 -- 172.21.15.45:0/2340733813 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb10081190 con 0x56128f72f110 2024-05-08T05:45:41.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.555+0000 7fdb037fe640 1 -- 172.21.15.45:0/2340733813 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fdb1c023050 con 0x56128f72f110 2024-05-08T05:45:41.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.555+0000 7fdb2d825740 10 client.6435 fetch_fsmap learned FSMap version 50 2024-05-08T05:45:41.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.555+0000 7fdb2d825740 1 -- 172.21.15.45:0/2340733813 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x56128f35e3f0 con 0x56128f72f110 2024-05-08T05:45:41.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.555+0000 7fdb037fe640 1 -- 172.21.15.45:0/2340733813 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb10081190 con 0x56128f72f110 2024-05-08T05:45:41.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.555+0000 7fdb037fe640 1 -- 172.21.15.45:0/2340733813 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7fdb1c026020 con 0x56128f72f110 2024-05-08T05:45:41.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.555+0000 7fdb2d825740 10 client.6435 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T05:45:41.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.555+0000 7fdb2d825740 10 client.6435 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-08T05:45:41.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.555+0000 7fdb2d825740 20 client.6435 populate_metadata read hostname 'smithi045' 2024-05-08T05:45:41.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.555+0000 7fdb2d825740 1 --2- 172.21.15.45:0/2340733813 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x56128f700860 0x56128f700c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.555+0000 7fdb2d825740 4 client.6435 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:45:41.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.555+0000 7fdb2d825740 1 -- 172.21.15.45:0/2340733813 --> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56128f65ab40 con 0x56128f700860 2024-05-08T05:45:41.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.555+0000 7fdb288a6640 1 --2- 172.21.15.45:0/2340733813 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x56128f700860 0x56128f700c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.555+0000 7fdb037fe640 10 client.6435 ms_handle_connect on v2:172.21.15.45:6833/4183668301 2024-05-08T05:45:41.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.555+0000 7fdb288a6640 1 --2- 172.21.15.45:0/2340733813 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x56128f700860 0x56128f700c70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bde5c0a075fe9fdf server_cookie=325964f74c828cfd in_seq=0 out_seq=0 2024-05-08T05:45:41.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.556+0000 7fdb037fe640 1 -- 172.21.15.45:0/2340733813 <== mds.0 v2:172.21.15.45:6833/4183668301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56128f65ab40 con 0x56128f700860 2024-05-08T05:45:41.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.556+0000 7fdb037fe640 10 client.6435 handle_command_reply: tid=0 2024-05-08T05:45:41.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.556+0000 7fdb2d825740 1 client.6435 shutdown 2024-05-08T05:45:41.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.556+0000 7fdb2d825740 1 -- 172.21.15.45:0/2340733813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f72f110 msgr2=0x56128f715840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.556+0000 7fdb2d825740 1 --2- 172.21.15.45:0/2340733813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f72f110 0x56128f715840 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00bc10 tx=0x7fdb1c00a7b0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.556+0000 7fdb2d825740 1 -- 172.21.15.45:0/2340733813 shutdown_connections 2024-05-08T05:45:41.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.556+0000 7fdb2d825740 1 --2- 172.21.15.45:0/2340733813 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x56128f700860 0x56128f700c70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.556+0000 7fdb2d825740 1 --2- 172.21.15.45:0/2340733813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f72f110 0x56128f715840 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.556+0000 7fdb2d825740 1 --2- 172.21.15.45:0/2340733813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f730160 0x56128f711d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.556+0000 7fdb2d825740 1 --2- 172.21.15.45:0/2340733813 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f713280 0x56128f713770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.556+0000 7fdb2d825740 1 -- 172.21.15.45:0/2340733813 >> 172.21.15.45:0/2340733813 conn(0x56128f712b40 msgr2=0x56128f6e2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:41.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.556+0000 7fdb037fe640 0 client.6435 ms_handle_reset on v2:172.21.15.45:6833/4183668301 2024-05-08T05:45:41.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.556+0000 7fdb2d825740 1 -- 172.21.15.45:0/2340733813 shutdown_connections 2024-05-08T05:45:41.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.556+0000 7fdb2d825740 1 -- 172.21.15.45:0/2340733813 wait complete. 2024-05-08T05:45:41.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.556+0000 7fdb2d825740 20 client.6435 trim_cache size 0 max 16384 2024-05-08T05:45:41.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.574+0000 7fdb2d825740 1 Processor -- start 2024-05-08T05:45:41.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.574+0000 7fdb2d825740 1 -- start start 2024-05-08T05:45:41.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.575+0000 7fdb2d825740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f713280 0x56128f715310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.575+0000 7fdb2d825740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f72f110 0x56128f70ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.575+0000 7fdb2d825740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f730160 0x56128f70f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.575+0000 7fdb2d825740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56128f7015e0 con 0x56128f730160 2024-05-08T05:45:41.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.575+0000 7fdb2d825740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24202010 con 0x56128f72f110 2024-05-08T05:45:41.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.575+0000 7fdb2d825740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdb240ccb20 con 0x56128f713280 2024-05-08T05:45:41.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.575+0000 7fdb290a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f730160 0x56128f70f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.575+0000 7fdb290a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f730160 0x56128f70f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36422/0 (socket says 172.21.15.45:36422) 2024-05-08T05:45:41.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.575+0000 7fdb290a7640 1 -- 172.21.15.45:0/1359389611 learned_addr learned my addr 172.21.15.45:0/1359389611 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:41.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.575+0000 7fdb290a7640 1 -- 172.21.15.45:0/1359389611 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f713280 msgr2=0x56128f715310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:41.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.575+0000 7fdb288a6640 1 --2- 172.21.15.45:0/1359389611 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f713280 0x56128f715310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.575+0000 7fdb23fff640 1 --2- 172.21.15.45:0/1359389611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f72f110 0x56128f70ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.576+0000 7fdb290a7640 1 --2- 172.21.15.45:0/1359389611 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f713280 0x56128f715310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.576+0000 7fdb290a7640 1 -- 172.21.15.45:0/1359389611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f72f110 msgr2=0x56128f70ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:41.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.576+0000 7fdb290a7640 1 --2- 172.21.15.45:0/1359389611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f72f110 0x56128f70ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.576+0000 7fdb290a7640 1 -- 172.21.15.45:0/1359389611 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56128f31e0f0 con 0x56128f730160 2024-05-08T05:45:41.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.576+0000 7fdb23fff640 1 --2- 172.21.15.45:0/1359389611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f72f110 0x56128f70ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:41.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.576+0000 7fdb288a6640 1 --2- 172.21.15.45:0/1359389611 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f713280 0x56128f715310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:45:41.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.576+0000 7fdb290a7640 1 --2- 172.21.15.45:0/1359389611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f730160 0x56128f70f2a0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0094a0 tx=0x7fdb1c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f5c7be4047aeb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:41.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.576+0000 7fdb0dffb640 1 -- 172.21.15.45:0/1359389611 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c02c070 con 0x56128f730160 2024-05-08T05:45:41.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.576+0000 7fdb0dffb640 1 -- 172.21.15.45:0/1359389611 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb1c00a080 con 0x56128f730160 2024-05-08T05:45:41.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.576+0000 7fdb0dffb640 1 -- 172.21.15.45:0/1359389611 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c01a070 con 0x56128f730160 2024-05-08T05:45:41.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.576+0000 7fdb2d825740 1 -- 172.21.15.45:0/1359389611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f730160 msgr2=0x56128f70f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.576+0000 7fdb2d825740 1 --2- 172.21.15.45:0/1359389611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f730160 0x56128f70f2a0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0094a0 tx=0x7fdb1c015000 comp rx=0 tx=0).stop 2024-05-08T05:45:41.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.576+0000 7fdb2d825740 1 -- 172.21.15.45:0/1359389611 shutdown_connections 2024-05-08T05:45:41.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.576+0000 7fdb2d825740 1 --2- 172.21.15.45:0/1359389611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f730160 0x56128f70f2a0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.576+0000 7fdb2d825740 1 --2- 172.21.15.45:0/1359389611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f72f110 0x56128f70ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.576+0000 7fdb2d825740 1 --2- 172.21.15.45:0/1359389611 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f713280 0x56128f715310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.576+0000 7fdb2d825740 1 -- 172.21.15.45:0/1359389611 >> 172.21.15.45:0/1359389611 conn(0x56128f712b40 msgr2=0x56128f730a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:41.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.576+0000 7fdb2d825740 1 -- 172.21.15.45:0/1359389611 shutdown_connections 2024-05-08T05:45:41.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.576+0000 7fdb2d825740 1 -- 172.21.15.45:0/1359389611 wait complete. 2024-05-08T05:45:41.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.577+0000 7fdb2d825740 1 -- start start 2024-05-08T05:45:41.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.577+0000 7fdb2d825740 1 Processor -- start 2024-05-08T05:45:41.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.578+0000 7fdb2d825740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f713280 0x56128f730c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.578+0000 7fdb2d825740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f72f110 0x56128f7aa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.578+0000 7fdb2d825740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f730160 0x56128f7105c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.578+0000 7fdb2d825740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdb240cc4d0 con 0x56128f713280 2024-05-08T05:45:41.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.578+0000 7fdb2d825740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdb240cbe80 con 0x56128f72f110 2024-05-08T05:45:41.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.578+0000 7fdb2d825740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56128f731250 con 0x56128f730160 2024-05-08T05:45:41.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.578+0000 7fdb2d825740 10 client.0 _ll_register_callbacks cb 0x7fdb1c009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:45:41.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.578+0000 7fdb288a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f713280 0x56128f730c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.578+0000 7fdb288a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f713280 0x56128f730c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36432/0 (socket says 172.21.15.45:36432) 2024-05-08T05:45:41.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.578+0000 7fdb288a6640 1 -- 172.21.15.45:0/3643372098 learned_addr learned my addr 172.21.15.45:0/3643372098 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:45:41.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.578+0000 7fdb23fff640 1 --2- 172.21.15.45:0/3643372098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f72f110 0x56128f7aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.578+0000 7fdb290a7640 1 --2- 172.21.15.45:0/3643372098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f730160 0x56128f7105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.578+0000 7fdb288a6640 1 -- 172.21.15.45:0/3643372098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f730160 msgr2=0x56128f7105c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:41.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.578+0000 7fdb288a6640 1 --2- 172.21.15.45:0/3643372098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f730160 0x56128f7105c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.578+0000 7fdb288a6640 1 -- 172.21.15.45:0/3643372098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f72f110 msgr2=0x56128f7aa620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:45:41.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.578+0000 7fdb288a6640 1 --2- 172.21.15.45:0/3643372098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f72f110 0x56128f7aa620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.578+0000 7fdb288a6640 1 -- 172.21.15.45:0/3643372098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56128f723fa0 con 0x56128f713280 2024-05-08T05:45:41.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.579+0000 7fdb288a6640 1 --2- 172.21.15.45:0/3643372098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f713280 0x56128f730c30 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x56128f701210 tx=0x7fdb18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e7154de8b4943 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:41.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.579+0000 7fdb037fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:45:41.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.579+0000 7fdb037fe640 1 -- 172.21.15.45:0/3643372098 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1801a040 con 0x56128f713280 2024-05-08T05:45:41.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.579+0000 7fdb037fe640 1 -- 172.21.15.45:0/3643372098 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7fdb18028070 con 0x56128f713280 2024-05-08T05:45:41.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.579+0000 7fdb2d825740 1 -- 172.21.15.45:0/3643372098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb1c00a080 con 0x56128f713280 2024-05-08T05:45:41.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.579+0000 7fdb037fe640 1 -- 172.21.15.45:0/3643372098 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fdb18035050 con 0x56128f713280 2024-05-08T05:45:41.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.579+0000 7fdb2d825740 10 client.6441 fetch_fsmap learned FSMap version 50 2024-05-08T05:45:41.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.579+0000 7fdb2d825740 1 -- 172.21.15.45:0/3643372098 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x56128f723ae0 con 0x56128f713280 2024-05-08T05:45:41.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.579+0000 7fdb037fe640 1 -- 172.21.15.45:0/3643372098 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb1c00a080 con 0x56128f713280 2024-05-08T05:45:41.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.579+0000 7fdb037fe640 1 -- 172.21.15.45:0/3643372098 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 50) ==== 3494+0+0 (secure 0 0 0) 0x7fdb18038020 con 0x56128f713280 2024-05-08T05:45:41.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.579+0000 7fdb2d825740 10 client.6441 fetch_fsmap finished waiting for FSMap version 50 2024-05-08T05:45:41.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.579+0000 7fdb2d825740 10 client.6441 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-08T05:45:41.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.579+0000 7fdb2d825740 20 client.6441 populate_metadata read hostname 'smithi045' 2024-05-08T05:45:41.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.579+0000 7fdb2d825740 1 --2- 172.21.15.45:0/3643372098 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x56128f710b00 0x56128f7291d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:41.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.579+0000 7fdb2d825740 4 client.6441 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:45:41.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.579+0000 7fdb2d825740 1 -- 172.21.15.45:0/3643372098 --> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56128f6549b0 con 0x56128f710b00 2024-05-08T05:45:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.579+0000 7fdb23fff640 1 --2- 172.21.15.45:0/3643372098 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x56128f710b00 0x56128f7291d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.580+0000 7fdb23fff640 1 --2- 172.21.15.45:0/3643372098 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x56128f710b00 0x56128f7291d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e81063aef7c31dc3 server_cookie=814c5826f12f7ecc in_seq=0 out_seq=0 2024-05-08T05:45:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.580+0000 7fdb037fe640 10 client.6441 ms_handle_connect on v2:172.21.15.45:6833/4183668301 2024-05-08T05:45:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.580+0000 7fdb037fe640 1 -- 172.21.15.45:0/3643372098 <== mds.0 v2:172.21.15.45:6833/4183668301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56128f6549b0 con 0x56128f710b00 2024-05-08T05:45:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.580+0000 7fdb037fe640 10 client.6441 handle_command_reply: tid=0 2024-05-08T05:45:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.580+0000 7fdb2d825740 1 client.6441 shutdown 2024-05-08T05:45:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.580+0000 7fdb2d825740 1 -- 172.21.15.45:0/3643372098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f713280 msgr2=0x56128f730c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.580+0000 7fdb2d825740 1 --2- 172.21.15.45:0/3643372098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f713280 0x56128f730c30 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x56128f701210 tx=0x7fdb18004730 comp rx=0 tx=0).stop 2024-05-08T05:45:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.580+0000 7fdb2d825740 1 -- 172.21.15.45:0/3643372098 shutdown_connections 2024-05-08T05:45:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.580+0000 7fdb2d825740 1 --2- 172.21.15.45:0/3643372098 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x56128f710b00 0x56128f7291d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.580+0000 7fdb2d825740 1 --2- 172.21.15.45:0/3643372098 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128f730160 0x56128f7105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.580+0000 7fdb2d825740 1 --2- 172.21.15.45:0/3643372098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128f72f110 0x56128f7aa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.580+0000 7fdb037fe640 0 client.6441 ms_handle_reset on v2:172.21.15.45:6833/4183668301 2024-05-08T05:45:41.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.580+0000 7fdb2d825740 1 --2- 172.21.15.45:0/3643372098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128f713280 0x56128f730c30 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.580+0000 7fdb2d825740 1 -- 172.21.15.45:0/3643372098 >> 172.21.15.45:0/3643372098 conn(0x56128f712b40 msgr2=0x56128f713770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:41.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.580+0000 7fdb2d825740 1 -- 172.21.15.45:0/3643372098 shutdown_connections 2024-05-08T05:45:41.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.580+0000 7fdb2d825740 1 -- 172.21.15.45:0/3643372098 wait complete. 2024-05-08T05:45:41.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.580+0000 7fdb2d825740 20 client.6441 trim_cache size 0 max 16384 2024-05-08T05:45:41.587 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:45:41.587 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T05:45:41.587 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-05-08T05:45:41.587 INFO:teuthology.orchestra.run.smithi045.stdout: "ab809561-9fd1-4d4e-972c-6b97041fa1d5": { 2024-05-08T05:45:41.587 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/subdir", 2024-05-08T05:45:41.588 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "ab809561-9fd1-4d4e-972c-6b97041fa1d5", 2024-05-08T05:45:41.588 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive" 2024-05-08T05:45:41.588 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:41.588 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:45:41.588 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:45:41.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.581+0000 7fdb2ab0a640 1 -- 172.21.15.45:0/2389139293 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdb080762d0 msgr2=0x7fdb08078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.581+0000 7fdb2ab0a640 1 --2- 172.21.15.45:0/2389139293 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdb080762d0 0x7fdb08078790 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdb18006fd0 tx=0x7fdb18008040 comp rx=0 tx=0).stop 2024-05-08T05:45:41.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.581+0000 7fdb2ab0a640 1 -- 172.21.15.45:0/2389139293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb24071110 msgr2=0x7fdb240b0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:41.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.581+0000 7fdb2ab0a640 1 --2- 172.21.15.45:0/2389139293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb24071110 0x7fdb240b0c70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fdb10000c00 tx=0x7fdb100046c0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.581+0000 7fdb2ab0a640 1 -- 172.21.15.45:0/2389139293 shutdown_connections 2024-05-08T05:45:41.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.581+0000 7fdb2ab0a640 1 --2- 172.21.15.45:0/2389139293 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdb080762d0 0x7fdb08078790 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.581+0000 7fdb2ab0a640 1 --2- 172.21.15.45:0/2389139293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb24163920 0x7fdb240b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.581+0000 7fdb2ab0a640 1 --2- 172.21.15.45:0/2389139293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb24071110 0x7fdb240b0c70 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.581+0000 7fdb2ab0a640 1 --2- 172.21.15.45:0/2389139293 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdb24070d30 0x7fdb240b70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:41.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.581+0000 7fdb2ab0a640 1 -- 172.21.15.45:0/2389139293 >> 172.21.15.45:0/2389139293 conn(0x7fdb240d2e90 msgr2=0x7fdb24159cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:41.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.581+0000 7fdb2ab0a640 1 -- 172.21.15.45:0/2389139293 shutdown_connections 2024-05-08T05:45:41.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:45:41.581+0000 7fdb2ab0a640 1 -- 172.21.15.45:0/2389139293 wait complete. 2024-05-08T05:45:41.592 INFO:tasks.cephfs.filesystem:scrub status for tag:ab809561-9fd1-4d4e-972c-6b97041fa1d5 - {'path': '/subdir', 'tag': 'ab809561-9fd1-4d4e-972c-6b97041fa1d5', 'options': 'recursive'} 2024-05-08T05:45:48.948 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-05-08T05:45:48.948+0000 7f1250617640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-08T05:46:04.606 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:46:04.610 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:46:11.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:46:11.593 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-08T05:46:11.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.712+0000 7efd1ed91640 1 Processor -- start 2024-05-08T05:46:11.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.712+0000 7efd1ed91640 1 -- start start 2024-05-08T05:46:11.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.713+0000 7efd1ed91640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7efd181629b0 0x7efd1806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.713+0000 7efd1ed91640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efd18070d20 0x7efd180711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.713+0000 7efd1ed91640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd1806e110 con 0x7efd180b2490 2024-05-08T05:46:11.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.713+0000 7efd1ed91640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7efd18070a40 con 0x7efd18070d20 2024-05-08T05:46:11.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.713+0000 7efd1ed91640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7efd1806dec0 con 0x7efd181629b0 2024-05-08T05:46:11.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.713+0000 7efd1d32e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efd18070d20 0x7efd180711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.713+0000 7efd17fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7efd180b2490 0x7efd18070330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49976/0 (socket says 172.21.15.45:49976) 2024-05-08T05:46:11.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.713+0000 7efd17fff640 1 -- 172.21.15.45:0/4206212609 learned_addr learned my addr 172.21.15.45:0/4206212609 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:11.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.713+0000 7efd1cb2d640 1 --2- 172.21.15.45:0/4206212609 >> v2:172.21.15.63:3300/0 conn(0x7efd181629b0 0x7efd1806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.713+0000 7efd1d32e640 1 -- 172.21.15.45:0/4206212609 >> v2:172.21.15.63:3300/0 conn(0x7efd181629b0 msgr2=0x7efd1806eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:11.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.713+0000 7efd1d32e640 1 --2- 172.21.15.45:0/4206212609 >> v2:172.21.15.63:3300/0 conn(0x7efd181629b0 0x7efd1806eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.713+0000 7efd1d32e640 1 -- 172.21.15.45:0/4206212609 >> v1:172.21.15.45:6789/0 conn(0x7efd180b2490 legacy=0x7efd18070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:11.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.713+0000 7efd177fe640 1 -- 172.21.15.45:0/4206212609 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 798733469 0 0) 0x7efd1806e110 con 0x7efd180b2490 2024-05-08T05:46:11.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.713+0000 7efd1d32e640 1 -- 172.21.15.45:0/4206212609 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1806e3f0 con 0x7efd18070d20 2024-05-08T05:46:11.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.714+0000 7efd1d32e640 1 --2- 172.21.15.45:0/4206212609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd18070d20 0x7efd180711d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7efd18068b40 tx=0x7efd1002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0220f7a37e8b40 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:11.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.714+0000 7efd177fe640 1 -- 172.21.15.45:0/4206212609 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10007740 con 0x7efd18070d20 2024-05-08T05:46:11.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.714+0000 7efd177fe640 1 -- 172.21.15.45:0/4206212609 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd100078e0 con 0x7efd18070d20 2024-05-08T05:46:11.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.714+0000 7efd177fe640 1 -- 172.21.15.45:0/4206212609 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10007cb0 con 0x7efd18070d20 2024-05-08T05:46:11.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.715+0000 7efd1ed91640 1 -- 172.21.15.45:0/4206212609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd18070d20 msgr2=0x7efd180711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:11.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.715+0000 7efd1ed91640 1 --2- 172.21.15.45:0/4206212609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd18070d20 0x7efd180711d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7efd18068b40 tx=0x7efd1002a680 comp rx=0 tx=0).stop 2024-05-08T05:46:11.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.715+0000 7efd1ed91640 1 -- 172.21.15.45:0/4206212609 shutdown_connections 2024-05-08T05:46:11.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.715+0000 7efd1ed91640 1 --2- 172.21.15.45:0/4206212609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd18070d20 0x7efd180711d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.715+0000 7efd1ed91640 1 --2- 172.21.15.45:0/4206212609 >> v2:172.21.15.63:3300/0 conn(0x7efd181629b0 0x7efd1806eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.715+0000 7efd1ed91640 1 -- 172.21.15.45:0/4206212609 >> 172.21.15.45:0/4206212609 conn(0x7efd180b49d0 msgr2=0x7efd180b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:11.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.715+0000 7efd1ed91640 1 -- 172.21.15.45:0/4206212609 shutdown_connections 2024-05-08T05:46:11.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.715+0000 7efd1ed91640 1 -- 172.21.15.45:0/4206212609 wait complete. 2024-05-08T05:46:11.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.716+0000 7efd1ed91640 1 Processor -- start 2024-05-08T05:46:11.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.716+0000 7efd1ed91640 1 -- start start 2024-05-08T05:46:11.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.717+0000 7efd1ed91640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd18070d20 0x7efd180b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.717+0000 7efd1ed91640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd180b2490 0x7efd1814d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.717+0000 7efd1ed91640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd181629b0 0x7efd180c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.717+0000 7efd1ed91640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efd1806f410 con 0x7efd180b2490 2024-05-08T05:46:11.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.717+0000 7efd1ed91640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd1806e9b0 con 0x7efd181629b0 2024-05-08T05:46:11.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.717+0000 7efd1ed91640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efd180afcb0 con 0x7efd18070d20 2024-05-08T05:46:11.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.717+0000 7efd17fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd180b2490 0x7efd1814d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.717+0000 7efd17fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd180b2490 0x7efd1814d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39728/0 (socket says 172.21.15.45:39728) 2024-05-08T05:46:11.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.717+0000 7efd17fff640 1 -- 172.21.15.45:0/3482864208 learned_addr learned my addr 172.21.15.45:0/3482864208 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:11.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.717+0000 7efd1cb2d640 1 --2- 172.21.15.45:0/3482864208 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd18070d20 0x7efd180b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.717+0000 7efd1d32e640 1 --2- 172.21.15.45:0/3482864208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd181629b0 0x7efd180c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.717+0000 7efd17fff640 1 -- 172.21.15.45:0/3482864208 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd18070d20 msgr2=0x7efd180b6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:11.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.717+0000 7efd17fff640 1 --2- 172.21.15.45:0/3482864208 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd18070d20 0x7efd180b6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.717+0000 7efd17fff640 1 -- 172.21.15.45:0/3482864208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd181629b0 msgr2=0x7efd180c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:11.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.717+0000 7efd17fff640 1 --2- 172.21.15.45:0/3482864208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd181629b0 0x7efd180c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.717+0000 7efd17fff640 1 -- 172.21.15.45:0/3482864208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd18159a70 con 0x7efd180b2490 2024-05-08T05:46:11.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.717+0000 7efd1cb2d640 1 --2- 172.21.15.45:0/3482864208 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd18070d20 0x7efd180b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:46:11.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.717+0000 7efd1d32e640 1 --2- 172.21.15.45:0/3482864208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd181629b0 0x7efd180c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:46:11.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.718+0000 7efd17fff640 1 --2- 172.21.15.45:0/3482864208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd180b2490 0x7efd1814d1f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00c4c0 tx=0x7efd0c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:11.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.718+0000 7efd157fa640 1 -- 172.21.15.45:0/3482864208 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c018070 con 0x7efd180b2490 2024-05-08T05:46:11.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.718+0000 7efd157fa640 1 -- 172.21.15.45:0/3482864208 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd0c007500 con 0x7efd180b2490 2024-05-08T05:46:11.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.718+0000 7efd1ed91640 1 -- 172.21.15.45:0/3482864208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd180b7530 con 0x7efd180b2490 2024-05-08T05:46:11.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.718+0000 7efd157fa640 1 -- 172.21.15.45:0/3482864208 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c004a20 con 0x7efd180b2490 2024-05-08T05:46:11.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.718+0000 7efd1ed91640 1 -- 172.21.15.45:0/3482864208 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd180b6510 con 0x7efd180b2490 2024-05-08T05:46:11.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.719+0000 7efd157fa640 1 -- 172.21.15.45:0/3482864208 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efd0c01b070 con 0x7efd180b2490 2024-05-08T05:46:11.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.720+0000 7efd157fa640 1 --2- 172.21.15.45:0/3482864208 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efd040762d0 0x7efd04078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.720+0000 7efd157fa640 1 -- 172.21.15.45:0/3482864208 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7efd0c096a20 con 0x7efd180b2490 2024-05-08T05:46:11.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.720+0000 7efd1cb2d640 1 --2- 172.21.15.45:0/3482864208 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efd040762d0 0x7efd04078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.721+0000 7efd21b81740 1 Processor -- start 2024-05-08T05:46:11.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.721+0000 7efd1cb2d640 1 --2- 172.21.15.45:0/3482864208 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efd040762d0 0x7efd04078790 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efd08009a20 tx=0x7efd08008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:11.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.721+0000 7efd21b81740 1 -- start start 2024-05-08T05:46:11.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.722+0000 7efd21b81740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d41160 0x557414cf3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.722+0000 7efd21b81740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557414d24280 0x557414d23600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.722+0000 7efd21b81740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557414d23b40 0x557414cf8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.722+0000 7efd21b81740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557414cc0690 con 0x557414d41160 2024-05-08T05:46:11.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.722+0000 7efd21b81740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557414c5df30 con 0x557414d23b40 2024-05-08T05:46:11.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.722+0000 7efd21b81740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x557414c16b20 con 0x557414d24280 2024-05-08T05:46:11.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.722+0000 7efd1cb2d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d41160 0x557414cf3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.722+0000 7efd1cb2d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d41160 0x557414cf3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39740/0 (socket says 172.21.15.45:39740) 2024-05-08T05:46:11.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.722+0000 7efd1cb2d640 1 -- 172.21.15.45:0/4276892576 learned_addr learned my addr 172.21.15.45:0/4276892576 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:11.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.722+0000 7efd17fff640 1 --2- 172.21.15.45:0/4276892576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557414d24280 0x557414d23600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.722+0000 7efd1d32e640 1 --2- 172.21.15.45:0/4276892576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557414d23b40 0x557414cf8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.722+0000 7efd1cb2d640 1 -- 172.21.15.45:0/4276892576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557414d24280 msgr2=0x557414d23600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:11.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.722+0000 7efd1cb2d640 1 --2- 172.21.15.45:0/4276892576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557414d24280 0x557414d23600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.722+0000 7efd1cb2d640 1 -- 172.21.15.45:0/4276892576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557414d23b40 msgr2=0x557414cf8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:11.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.722+0000 7efd1cb2d640 1 --2- 172.21.15.45:0/4276892576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557414d23b40 0x557414cf8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.722+0000 7efd1cb2d640 1 -- 172.21.15.45:0/4276892576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557414c0ee60 con 0x557414d41160 2024-05-08T05:46:11.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.722+0000 7efd1d32e640 1 --2- 172.21.15.45:0/4276892576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557414d23b40 0x557414cf8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:11.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.722+0000 7efd17fff640 1 --2- 172.21.15.45:0/4276892576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557414d24280 0x557414d23600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:11.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.722+0000 7efd1cb2d640 1 --2- 172.21.15.45:0/4276892576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d41160 0x557414cf3af0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7efd08004290 tx=0x7efd08013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dfa3780b4bb6eb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:11.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.722+0000 7efcfe7fc640 1 -- 172.21.15.45:0/4276892576 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08027070 con 0x557414d41160 2024-05-08T05:46:11.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.723+0000 7efcfe7fc640 1 -- 172.21.15.45:0/4276892576 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd080044c0 con 0x557414d41160 2024-05-08T05:46:11.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.723+0000 7efcfe7fc640 1 -- 172.21.15.45:0/4276892576 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0800f040 con 0x557414d41160 2024-05-08T05:46:11.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.723+0000 7efd21b81740 1 -- 172.21.15.45:0/4276892576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d41160 msgr2=0x557414cf3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:11.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.723+0000 7efd21b81740 1 --2- 172.21.15.45:0/4276892576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d41160 0x557414cf3af0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7efd08004290 tx=0x7efd08013000 comp rx=0 tx=0).stop 2024-05-08T05:46:11.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.723+0000 7efd21b81740 1 -- 172.21.15.45:0/4276892576 shutdown_connections 2024-05-08T05:46:11.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.723+0000 7efd21b81740 1 --2- 172.21.15.45:0/4276892576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557414d24280 0x557414d23600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.723+0000 7efd21b81740 1 --2- 172.21.15.45:0/4276892576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557414d23b40 0x557414cf8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.723+0000 7efd21b81740 1 --2- 172.21.15.45:0/4276892576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d41160 0x557414cf3af0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.723+0000 7efd21b81740 1 -- 172.21.15.45:0/4276892576 >> 172.21.15.45:0/4276892576 conn(0x557414cf5d60 msgr2=0x557414d3e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:11.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.723+0000 7efd21b81740 1 -- 172.21.15.45:0/4276892576 shutdown_connections 2024-05-08T05:46:11.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.723+0000 7efd21b81740 1 -- 172.21.15.45:0/4276892576 wait complete. 2024-05-08T05:46:11.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.724+0000 7efd21b81740 1 -- start start 2024-05-08T05:46:11.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.724+0000 7efd21b81740 1 Processor -- start 2024-05-08T05:46:11.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.725+0000 7efd21b81740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557414d24280 0x557414d274d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.725+0000 7efd21b81740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557414d41160 0x557414d25970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.725+0000 7efd21b81740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d140b0 0x557414d3f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.725+0000 7efd21b81740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557414c1a5e0 con 0x557414d140b0 2024-05-08T05:46:11.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.725+0000 7efd21b81740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557414c59ef0 con 0x557414d41160 2024-05-08T05:46:11.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.725+0000 7efd21b81740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x557414cf7f10 con 0x557414d24280 2024-05-08T05:46:11.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.725+0000 7efd21b81740 10 client.0 _ll_register_callbacks cb 0x557414c15320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:46:11.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.725+0000 7efd1d32e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d140b0 0x557414d3f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.725+0000 7efd1d32e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d140b0 0x557414d3f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39746/0 (socket says 172.21.15.45:39746) 2024-05-08T05:46:11.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.725+0000 7efd1d32e640 1 -- 172.21.15.45:0/3849021873 learned_addr learned my addr 172.21.15.45:0/3849021873 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:11.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.725+0000 7efd17fff640 1 --2- 172.21.15.45:0/3849021873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557414d41160 0x557414d25970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.725+0000 7efd1cb2d640 1 --2- 172.21.15.45:0/3849021873 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557414d24280 0x557414d274d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.725+0000 7efd1d32e640 1 -- 172.21.15.45:0/3849021873 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557414d24280 msgr2=0x557414d274d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:11.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.725+0000 7efd1d32e640 1 --2- 172.21.15.45:0/3849021873 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557414d24280 0x557414d274d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.725+0000 7efd1d32e640 1 -- 172.21.15.45:0/3849021873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557414d41160 msgr2=0x557414d25970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:11.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.725+0000 7efd1d32e640 1 --2- 172.21.15.45:0/3849021873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557414d41160 0x557414d25970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.725+0000 7efd1d32e640 1 -- 172.21.15.45:0/3849021873 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557414a397d0 con 0x557414d140b0 2024-05-08T05:46:11.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.726+0000 7efd1d32e640 1 --2- 172.21.15.45:0/3849021873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d140b0 0x557414d3f450 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7efd10009e80 tx=0x7efd10004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c341514edd98e4e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:11.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.726+0000 7efcf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:46:11.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.726+0000 7efcf77fe640 1 -- 172.21.15.45:0/3849021873 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10009450 con 0x557414d140b0 2024-05-08T05:46:11.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.726+0000 7efcf77fe640 1 -- 172.21.15.45:0/3849021873 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7efd1003c420 con 0x557414d140b0 2024-05-08T05:46:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.726+0000 7efd21b81740 1 -- 172.21.15.45:0/3849021873 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557414c6bb40 con 0x557414d140b0 2024-05-08T05:46:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.726+0000 7efcf77fe640 1 -- 172.21.15.45:0/3849021873 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7efd1003c620 con 0x557414d140b0 2024-05-08T05:46:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.726+0000 7efd21b81740 10 client.6456 fetch_fsmap learned FSMap version 51 2024-05-08T05:46:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.726+0000 7efd21b81740 1 -- 172.21.15.45:0/3849021873 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55741496f3f0 con 0x557414d140b0 2024-05-08T05:46:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.726+0000 7efcf77fe640 1 -- 172.21.15.45:0/3849021873 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x557414c6bb40 con 0x557414d140b0 2024-05-08T05:46:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.726+0000 7efcf77fe640 1 -- 172.21.15.45:0/3849021873 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7efd1004e050 con 0x557414d140b0 2024-05-08T05:46:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.726+0000 7efd21b81740 10 client.6456 fetch_fsmap finished waiting for FSMap version 51 2024-05-08T05:46:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.726+0000 7efd21b81740 10 client.6456 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-08T05:46:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.726+0000 7efd21b81740 20 client.6456 populate_metadata read hostname 'smithi045' 2024-05-08T05:46:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.726+0000 7efd21b81740 1 --2- 172.21.15.45:0/3849021873 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x557414d4e390 0x557414d4e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.726+0000 7efd21b81740 4 client.6456 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:46:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.726+0000 7efd21b81740 1 -- 172.21.15.45:0/3849021873 --> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557414c659b0 con 0x557414d4e390 2024-05-08T05:46:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.726+0000 7efd1cb2d640 1 --2- 172.21.15.45:0/3849021873 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x557414d4e390 0x557414d4e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.727+0000 7efd1cb2d640 1 --2- 172.21.15.45:0/3849021873 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x557414d4e390 0x557414d4e770 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc58fb407a4e7720 server_cookie=ccabbf738ca0d451 in_seq=0 out_seq=0 2024-05-08T05:46:11.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.727+0000 7efcf77fe640 10 client.6456 ms_handle_connect on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.727+0000 7efcf77fe640 1 -- 172.21.15.45:0/3849021873 <== mds.0 v2:172.21.15.45:6833/4183668301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557414c659b0 con 0x557414d4e390 2024-05-08T05:46:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.727+0000 7efcf77fe640 10 client.6456 handle_command_reply: tid=0 2024-05-08T05:46:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.727+0000 7efd21b81740 1 client.6456 shutdown 2024-05-08T05:46:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.728+0000 7efd21b81740 1 -- 172.21.15.45:0/3849021873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d140b0 msgr2=0x557414d3f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.728+0000 7efd21b81740 1 --2- 172.21.15.45:0/3849021873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d140b0 0x557414d3f450 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7efd10009e80 tx=0x7efd10004ab0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.728+0000 7efd21b81740 1 -- 172.21.15.45:0/3849021873 shutdown_connections 2024-05-08T05:46:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.728+0000 7efd21b81740 1 --2- 172.21.15.45:0/3849021873 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x557414d4e390 0x557414d4e770 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.728+0000 7efd21b81740 1 --2- 172.21.15.45:0/3849021873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d140b0 0x557414d3f450 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.728+0000 7efd21b81740 1 --2- 172.21.15.45:0/3849021873 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557414d41160 0x557414d25970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.728+0000 7efd21b81740 1 --2- 172.21.15.45:0/3849021873 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557414d24280 0x557414d274d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.728+0000 7efd21b81740 1 -- 172.21.15.45:0/3849021873 >> 172.21.15.45:0/3849021873 conn(0x557414d23b40 msgr2=0x557414d41a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.728+0000 7efcf77fe640 0 client.6456 ms_handle_reset on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.728+0000 7efd21b81740 1 -- 172.21.15.45:0/3849021873 shutdown_connections 2024-05-08T05:46:11.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.728+0000 7efd21b81740 1 -- 172.21.15.45:0/3849021873 wait complete. 2024-05-08T05:46:11.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.728+0000 7efd21b81740 20 client.6456 trim_cache size 0 max 16384 2024-05-08T05:46:11.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.746+0000 7efd21b81740 1 Processor -- start 2024-05-08T05:46:11.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.746+0000 7efd21b81740 1 -- start start 2024-05-08T05:46:11.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.747+0000 7efd21b81740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d140b0 0x557414d62060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.747+0000 7efd21b81740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557414d24280 0x557414d41a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.747+0000 7efd21b81740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557414d41160 0x557414d41f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.747+0000 7efd21b81740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efd18201e30 con 0x557414d140b0 2024-05-08T05:46:11.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.747+0000 7efd21b81740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd18200ec0 con 0x557414d41160 2024-05-08T05:46:11.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.747+0000 7efd21b81740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efd180c7a20 con 0x557414d24280 2024-05-08T05:46:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.747+0000 7efd1cb2d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d140b0 0x557414d62060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.747+0000 7efd1cb2d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d140b0 0x557414d62060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39762/0 (socket says 172.21.15.45:39762) 2024-05-08T05:46:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.747+0000 7efd1cb2d640 1 -- 172.21.15.45:0/4188158621 learned_addr learned my addr 172.21.15.45:0/4188158621 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.747+0000 7efd1cb2d640 1 -- 172.21.15.45:0/4188158621 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557414d24280 msgr2=0x557414d41a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.747+0000 7efd1cb2d640 1 --2- 172.21.15.45:0/4188158621 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557414d24280 0x557414d41a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.747+0000 7efd1cb2d640 1 -- 172.21.15.45:0/4188158621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557414d41160 msgr2=0x557414d41f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.747+0000 7efd1cb2d640 1 --2- 172.21.15.45:0/4188158621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557414d41160 0x557414d41f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.747+0000 7efd1cb2d640 1 -- 172.21.15.45:0/4188158621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557414d379c0 con 0x557414d140b0 2024-05-08T05:46:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.747+0000 7efd1cb2d640 1 --2- 172.21.15.45:0/4188158621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d140b0 0x557414d62060 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7efd08004260 tx=0x7efd08015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60c1927506143d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.747+0000 7efcfdffb640 1 -- 172.21.15.45:0/4188158621 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08027040 con 0x557414d140b0 2024-05-08T05:46:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.747+0000 7efcfdffb640 1 -- 172.21.15.45:0/4188158621 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd08033070 con 0x557414d140b0 2024-05-08T05:46:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.747+0000 7efcfdffb640 1 -- 172.21.15.45:0/4188158621 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08003d50 con 0x557414d140b0 2024-05-08T05:46:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.748+0000 7efd21b81740 1 -- 172.21.15.45:0/4188158621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d140b0 msgr2=0x557414d62060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:11.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.748+0000 7efd21b81740 1 --2- 172.21.15.45:0/4188158621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d140b0 0x557414d62060 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7efd08004260 tx=0x7efd08015000 comp rx=0 tx=0).stop 2024-05-08T05:46:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.748+0000 7efd21b81740 1 -- 172.21.15.45:0/4188158621 shutdown_connections 2024-05-08T05:46:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.748+0000 7efd21b81740 1 --2- 172.21.15.45:0/4188158621 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557414d24280 0x557414d41a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.748+0000 7efd21b81740 1 --2- 172.21.15.45:0/4188158621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557414d41160 0x557414d41f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.748+0000 7efd21b81740 1 --2- 172.21.15.45:0/4188158621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414d140b0 0x557414d62060 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.748+0000 7efd21b81740 1 -- 172.21.15.45:0/4188158621 >> 172.21.15.45:0/4188158621 conn(0x557414d23b40 msgr2=0x557414d405b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.748+0000 7efd21b81740 1 -- 172.21.15.45:0/4188158621 shutdown_connections 2024-05-08T05:46:11.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.748+0000 7efd21b81740 1 -- 172.21.15.45:0/4188158621 wait complete. 2024-05-08T05:46:11.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.749+0000 7efd21b81740 1 -- start start 2024-05-08T05:46:11.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.749+0000 7efd21b81740 1 Processor -- start 2024-05-08T05:46:11.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efd21b81740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557414d24280 0x557414d3ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efd21b81740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557414d41160 0x557414d41a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efd21b81740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414dbf590 0x557414d41f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efd21b81740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efd18200920 con 0x557414dbf590 2024-05-08T05:46:11.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efd21b81740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x557414cf6900 con 0x557414d24280 2024-05-08T05:46:11.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efd21b81740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x557414d3d720 con 0x557414d41160 2024-05-08T05:46:11.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efd21b81740 10 client.0 _ll_register_callbacks cb 0x7efd08009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:46:11.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efd1d32e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414dbf590 0x557414d41f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efd1d32e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414dbf590 0x557414d41f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39778/0 (socket says 172.21.15.45:39778) 2024-05-08T05:46:11.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efd1d32e640 1 -- 172.21.15.45:0/2485384173 learned_addr learned my addr 172.21.15.45:0/2485384173 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:11.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efd1cb2d640 1 --2- 172.21.15.45:0/2485384173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557414d24280 0x557414d3ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efd17fff640 1 --2- 172.21.15.45:0/2485384173 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557414d41160 0x557414d41a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efd1d32e640 1 -- 172.21.15.45:0/2485384173 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557414d41160 msgr2=0x557414d41a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:11.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efd1d32e640 1 --2- 172.21.15.45:0/2485384173 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557414d41160 0x557414d41a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efd1d32e640 1 -- 172.21.15.45:0/2485384173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557414d24280 msgr2=0x557414d3ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:11.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efd1d32e640 1 --2- 172.21.15.45:0/2485384173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557414d24280 0x557414d3ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efd1d32e640 1 -- 172.21.15.45:0/2485384173 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55741492f0f0 con 0x557414dbf590 2024-05-08T05:46:11.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efd1d32e640 1 --2- 172.21.15.45:0/2485384173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414dbf590 0x557414d41f60 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7efd10007fe0 tx=0x7efd10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae4ad55aad03bd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:11.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efcf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:46:11.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efcf77fe640 1 -- 172.21.15.45:0/2485384173 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10009450 con 0x557414dbf590 2024-05-08T05:46:11.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efcf77fe640 1 -- 172.21.15.45:0/2485384173 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7efd1003c420 con 0x557414dbf590 2024-05-08T05:46:11.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.750+0000 7efd21b81740 1 -- 172.21.15.45:0/2485384173 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd08033070 con 0x557414dbf590 2024-05-08T05:46:11.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.751+0000 7efcf77fe640 1 -- 172.21.15.45:0/2485384173 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7efd1005c050 con 0x557414dbf590 2024-05-08T05:46:11.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.751+0000 7efd21b81740 10 client.6462 fetch_fsmap learned FSMap version 51 2024-05-08T05:46:11.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.751+0000 7efd21b81740 1 -- 172.21.15.45:0/2485384173 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x557414d13790 con 0x557414dbf590 2024-05-08T05:46:11.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.751+0000 7efcf77fe640 1 -- 172.21.15.45:0/2485384173 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd08033070 con 0x557414dbf590 2024-05-08T05:46:11.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.751+0000 7efcf77fe640 1 -- 172.21.15.45:0/2485384173 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7efd1005f020 con 0x557414dbf590 2024-05-08T05:46:11.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.751+0000 7efd21b81740 10 client.6462 fetch_fsmap finished waiting for FSMap version 51 2024-05-08T05:46:11.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.751+0000 7efd21b81740 10 client.6462 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-08T05:46:11.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.751+0000 7efd21b81740 20 client.6462 populate_metadata read hostname 'smithi045' 2024-05-08T05:46:11.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.751+0000 7efd21b81740 1 --2- 172.21.15.45:0/2485384173 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x557414cf3580 0x557414d57860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.751+0000 7efd21b81740 4 client.6462 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:46:11.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.751+0000 7efd21b81740 1 -- 172.21.15.45:0/2485384173 --> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557414be04a0 con 0x557414cf3580 2024-05-08T05:46:11.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.751+0000 7efd1cb2d640 1 --2- 172.21.15.45:0/2485384173 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x557414cf3580 0x557414d57860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.751+0000 7efcf77fe640 10 client.6462 ms_handle_connect on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:11.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.751+0000 7efd1cb2d640 1 --2- 172.21.15.45:0/2485384173 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x557414cf3580 0x557414d57860 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a962e1690ff09e4a server_cookie=c0444ad2860aefc in_seq=0 out_seq=0 2024-05-08T05:46:11.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.752+0000 7efcf77fe640 1 -- 172.21.15.45:0/2485384173 <== mds.0 v2:172.21.15.45:6833/4183668301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557414be04a0 con 0x557414cf3580 2024-05-08T05:46:11.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.752+0000 7efcf77fe640 10 client.6462 handle_command_reply: tid=0 2024-05-08T05:46:11.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.752+0000 7efd21b81740 1 client.6462 shutdown 2024-05-08T05:46:11.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.752+0000 7efd21b81740 1 -- 172.21.15.45:0/2485384173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414dbf590 msgr2=0x557414d41f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:11.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.752+0000 7efd21b81740 1 --2- 172.21.15.45:0/2485384173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414dbf590 0x557414d41f60 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7efd10007fe0 tx=0x7efd10004a40 comp rx=0 tx=0).stop 2024-05-08T05:46:11.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.752+0000 7efd21b81740 1 -- 172.21.15.45:0/2485384173 shutdown_connections 2024-05-08T05:46:11.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.752+0000 7efd21b81740 1 --2- 172.21.15.45:0/2485384173 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x557414cf3580 0x557414d57860 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.752+0000 7efd21b81740 1 --2- 172.21.15.45:0/2485384173 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x557414d41160 0x557414d41a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.752+0000 7efd21b81740 1 --2- 172.21.15.45:0/2485384173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557414dbf590 0x557414d41f60 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.752+0000 7efd21b81740 1 --2- 172.21.15.45:0/2485384173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x557414d24280 0x557414d3ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.752+0000 7efd21b81740 1 -- 172.21.15.45:0/2485384173 >> 172.21.15.45:0/2485384173 conn(0x557414d23b40 msgr2=0x557414cf6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:11.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.752+0000 7efcf77fe640 0 client.6462 ms_handle_reset on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:11.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.752+0000 7efd21b81740 1 -- 172.21.15.45:0/2485384173 shutdown_connections 2024-05-08T05:46:11.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.752+0000 7efd21b81740 1 -- 172.21.15.45:0/2485384173 wait complete. 2024-05-08T05:46:11.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.752+0000 7efd21b81740 20 client.6462 trim_cache size 0 max 16384 2024-05-08T05:46:11.758 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:46:11.758 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-05-08T05:46:11.758 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-05-08T05:46:11.758 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:46:11.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.752+0000 7efd1ed91640 1 -- 172.21.15.45:0/3482864208 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efd040762d0 msgr2=0x7efd04078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:11.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.752+0000 7efd1ed91640 1 --2- 172.21.15.45:0/3482864208 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efd040762d0 0x7efd04078790 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efd08009a20 tx=0x7efd08008040 comp rx=0 tx=0).stop 2024-05-08T05:46:11.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.752+0000 7efd1ed91640 1 -- 172.21.15.45:0/3482864208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd180b2490 msgr2=0x7efd1814d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:11.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.752+0000 7efd1ed91640 1 --2- 172.21.15.45:0/3482864208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd180b2490 0x7efd1814d1f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00c4c0 tx=0x7efd0c00c980 comp rx=0 tx=0).stop 2024-05-08T05:46:11.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.753+0000 7efd1ed91640 1 -- 172.21.15.45:0/3482864208 shutdown_connections 2024-05-08T05:46:11.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.753+0000 7efd1ed91640 1 --2- 172.21.15.45:0/3482864208 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efd040762d0 0x7efd04078790 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.753+0000 7efd1ed91640 1 --2- 172.21.15.45:0/3482864208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd181629b0 0x7efd180c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.753+0000 7efd1ed91640 1 --2- 172.21.15.45:0/3482864208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd180b2490 0x7efd1814d1f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.753+0000 7efd1ed91640 1 --2- 172.21.15.45:0/3482864208 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd18070d20 0x7efd180b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.753+0000 7efd1ed91640 1 -- 172.21.15.45:0/3482864208 >> 172.21.15.45:0/3482864208 conn(0x7efd180b49d0 msgr2=0x7efd180b3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:11.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.753+0000 7efd1ed91640 1 -- 172.21.15.45:0/3482864208 shutdown_connections 2024-05-08T05:46:11.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.753+0000 7efd1ed91640 1 -- 172.21.15.45:0/3482864208 wait complete. 2024-05-08T05:46:11.764 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T05:46:11.764 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:46:11.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.868+0000 7f8579ff2640 1 Processor -- start 2024-05-08T05:46:11.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.868+0000 7f8579ff2640 1 -- start start 2024-05-08T05:46:11.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.869+0000 7f8579ff2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85740d3cd0 0x7f85740cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.869+0000 7f8579ff2640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f85740c9ee0 0x7f85740d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.869+0000 7f8579ff2640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85740d3330 con 0x7f85741629a0 2024-05-08T05:46:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.869+0000 7f8579ff2640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f85740d2170 con 0x7f85740c9ee0 2024-05-08T05:46:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.869+0000 7f8579ff2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f85740d2450 con 0x7f85740d3cd0 2024-05-08T05:46:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.869+0000 7f8573fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f85740c9ee0 0x7f85740d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.869+0000 7f85737fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f85741629a0 0x7f85740d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49986/0 (socket says 172.21.15.45:49986) 2024-05-08T05:46:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.869+0000 7f85737fe640 1 -- 172.21.15.45:0/1004585969 learned_addr learned my addr 172.21.15.45:0/1004585969 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.869+0000 7f8572ffd640 1 --2- 172.21.15.45:0/1004585969 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85740d3cd0 0x7f85740cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.869+0000 7f8573fff640 1 -- 172.21.15.45:0/1004585969 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85740d3cd0 msgr2=0x7f85740cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.869+0000 7f8573fff640 1 --2- 172.21.15.45:0/1004585969 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85740d3cd0 0x7f85740cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.869+0000 7f8573fff640 1 -- 172.21.15.45:0/1004585969 >> v1:172.21.15.45:6789/0 conn(0x7f85741629a0 legacy=0x7f85740d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.869+0000 7f8573fff640 1 -- 172.21.15.45:0/1004585969 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85740d2730 con 0x7f85740c9ee0 2024-05-08T05:46:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.869+0000 7f85727fc640 1 -- 172.21.15.45:0/1004585969 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3190234440 0 0) 0x7f85740d3330 con 0x7f85741629a0 2024-05-08T05:46:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.870+0000 7f8573fff640 1 --2- 172.21.15.45:0/1004585969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85740c9ee0 0x7f85740d01e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f856c009d90 tx=0x7f856c02a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a733da27df3a81 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.870+0000 7f85727fc640 1 -- 172.21.15.45:0/1004585969 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c007740 con 0x7f85740c9ee0 2024-05-08T05:46:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.870+0000 7f85727fc640 1 -- 172.21.15.45:0/1004585969 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f856c0078e0 con 0x7f85740c9ee0 2024-05-08T05:46:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.870+0000 7f85727fc640 1 -- 172.21.15.45:0/1004585969 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c007cb0 con 0x7f85740c9ee0 2024-05-08T05:46:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.871+0000 7f8579ff2640 1 -- 172.21.15.45:0/1004585969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85740c9ee0 msgr2=0x7f85740d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.871+0000 7f8579ff2640 1 --2- 172.21.15.45:0/1004585969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85740c9ee0 0x7f85740d01e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f856c009d90 tx=0x7f856c02a530 comp rx=0 tx=0).stop 2024-05-08T05:46:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.871+0000 7f8579ff2640 1 -- 172.21.15.45:0/1004585969 shutdown_connections 2024-05-08T05:46:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.871+0000 7f8579ff2640 1 --2- 172.21.15.45:0/1004585969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85740c9ee0 0x7f85740d01e0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.871+0000 7f8579ff2640 1 --2- 172.21.15.45:0/1004585969 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85740d3cd0 0x7f85740cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.871+0000 7f8579ff2640 1 -- 172.21.15.45:0/1004585969 >> 172.21.15.45:0/1004585969 conn(0x7f85740d4fe0 msgr2=0x7f85740d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:11.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.871+0000 7f8579ff2640 1 -- 172.21.15.45:0/1004585969 shutdown_connections 2024-05-08T05:46:11.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.871+0000 7f8579ff2640 1 -- 172.21.15.45:0/1004585969 wait complete. 2024-05-08T05:46:11.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.872+0000 7f8579ff2640 1 Processor -- start 2024-05-08T05:46:11.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.872+0000 7f8579ff2640 1 -- start start 2024-05-08T05:46:11.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.873+0000 7f8579ff2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85740c9ee0 0x7f8574151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.873+0000 7f8579ff2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85740d3cd0 0x7f85740ba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.873+0000 7f8579ff2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85741629a0 0x7f857406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.873+0000 7f8579ff2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f85740cf440 con 0x7f85740c9ee0 2024-05-08T05:46:11.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.873+0000 7f8579ff2640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f85740cf2c0 con 0x7f85741629a0 2024-05-08T05:46:11.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.873+0000 7f8579ff2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f85740d2dc0 con 0x7f85740d3cd0 2024-05-08T05:46:11.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.873+0000 7f85737fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85740c9ee0 0x7f8574151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.873+0000 7f85737fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85740c9ee0 0x7f8574151a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39792/0 (socket says 172.21.15.45:39792) 2024-05-08T05:46:11.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.873+0000 7f85737fe640 1 -- 172.21.15.45:0/2577796747 learned_addr learned my addr 172.21.15.45:0/2577796747 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:11.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.873+0000 7f8572ffd640 1 --2- 172.21.15.45:0/2577796747 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85740d3cd0 0x7f85740ba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.873+0000 7f8573fff640 1 --2- 172.21.15.45:0/2577796747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85741629a0 0x7f857406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.873+0000 7f85737fe640 1 -- 172.21.15.45:0/2577796747 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85740d3cd0 msgr2=0x7f85740ba6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:11.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.873+0000 7f85737fe640 1 --2- 172.21.15.45:0/2577796747 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85740d3cd0 0x7f85740ba6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.873+0000 7f85737fe640 1 -- 172.21.15.45:0/2577796747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85741629a0 msgr2=0x7f857406e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:11.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.873+0000 7f85737fe640 1 --2- 172.21.15.45:0/2577796747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85741629a0 0x7f857406e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:11.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.873+0000 7f85737fe640 1 -- 172.21.15.45:0/2577796747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85740cd3c0 con 0x7f85740c9ee0 2024-05-08T05:46:11.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.873+0000 7f8572ffd640 1 --2- 172.21.15.45:0/2577796747 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85740d3cd0 0x7f85740ba6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:46:11.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.874+0000 7f85737fe640 1 --2- 172.21.15.45:0/2577796747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85740c9ee0 0x7f8574151a00 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f856400c410 tx=0x7f856400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:11.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.874+0000 7f855bfff640 1 -- 172.21.15.45:0/2577796747 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85640074e0 con 0x7f85740c9ee0 2024-05-08T05:46:11.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.874+0000 7f855bfff640 1 -- 172.21.15.45:0/2577796747 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85640047e0 con 0x7f85740c9ee0 2024-05-08T05:46:11.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.874+0000 7f8579ff2640 1 -- 172.21.15.45:0/2577796747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85740cd6a0 con 0x7f85740c9ee0 2024-05-08T05:46:11.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.874+0000 7f855bfff640 1 -- 172.21.15.45:0/2577796747 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8564004bd0 con 0x7f85740c9ee0 2024-05-08T05:46:11.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.874+0000 7f8579ff2640 1 -- 172.21.15.45:0/2577796747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85740cdc60 con 0x7f85740c9ee0 2024-05-08T05:46:11.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.875+0000 7f8579ff2640 1 -- 172.21.15.45:0/2577796747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85740cde60 con 0x7f85740c9ee0 2024-05-08T05:46:11.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.875+0000 7f855bfff640 1 -- 172.21.15.45:0/2577796747 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8564011070 con 0x7f85740c9ee0 2024-05-08T05:46:11.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.876+0000 7f855bfff640 1 --2- 172.21.15.45:0/2577796747 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f85600762d0 0x7f8560078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:11.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.876+0000 7f855bfff640 1 -- 172.21.15.45:0/2577796747 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f8564097370 con 0x7f85740c9ee0 2024-05-08T05:46:11.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.876+0000 7f8572ffd640 1 --2- 172.21.15.45:0/2577796747 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f85600762d0 0x7f8560078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:11.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.877+0000 7f8572ffd640 1 --2- 172.21.15.45:0/2577796747 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f85600762d0 0x7f8560078790 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f85741f8360 tx=0x7f8568008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:11.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:11.881+0000 7f855bfff640 1 -- 172.21.15.45:0/2577796747 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f85740cde60 con 0x7f85740c9ee0 2024-05-08T05:46:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.110+0000 7f8579ff2640 1 -- 172.21.15.45:0/2577796747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85740693f0 con 0x7f85740c9ee0 2024-05-08T05:46:12.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.111+0000 7f855bfff640 1 -- 172.21.15.45:0/2577796747 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5185 (secure 0 0 0) 0x7f8564060b80 con 0x7f85740c9ee0 2024-05-08T05:46:12.111 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:46:12.112 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5722,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1283757699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1283757699},{"type":"v1","addr":"172.21.15.63:6837","nonce":1283757699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2823770504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2823770504},{"type":"v1","addr":"172.21.15.63:6836","nonce":2823770504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1963062796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1963062796},{"type":"v1","addr":"172.21.15.45:6834","nonce":1963062796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:45:24.208158+0000","modified":"2024-05-08T05:45:48.491928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":76,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":10,"addr":"172.21.15.45:6835/4183668301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4183668301},{"type":"v1","addr":"172.21.15.45:6835","nonce":4183668301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6171,"qdb_cluster":[6171]},"id":5}]} 2024-05-08T05:46:12.112 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 51 2024-05-08T05:46:12.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.113+0000 7f8579ff2640 1 -- 172.21.15.45:0/2577796747 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f85600762d0 msgr2=0x7f8560078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.113+0000 7f8579ff2640 1 --2- 172.21.15.45:0/2577796747 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f85600762d0 0x7f8560078790 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f85741f8360 tx=0x7f8568008040 comp rx=0 tx=0).stop 2024-05-08T05:46:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.113+0000 7f8579ff2640 1 -- 172.21.15.45:0/2577796747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85740c9ee0 msgr2=0x7f8574151a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.113+0000 7f8579ff2640 1 --2- 172.21.15.45:0/2577796747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85740c9ee0 0x7f8574151a00 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f856400c410 tx=0x7f856400c8d0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.113+0000 7f8579ff2640 1 -- 172.21.15.45:0/2577796747 shutdown_connections 2024-05-08T05:46:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.113+0000 7f8579ff2640 1 --2- 172.21.15.45:0/2577796747 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f85600762d0 0x7f8560078790 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.113+0000 7f8579ff2640 1 --2- 172.21.15.45:0/2577796747 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f85740d3cd0 0x7f85740ba6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.113+0000 7f8579ff2640 1 --2- 172.21.15.45:0/2577796747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f85741629a0 0x7f857406e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.113+0000 7f8579ff2640 1 --2- 172.21.15.45:0/2577796747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f85740c9ee0 0x7f8574151a00 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.113+0000 7f8579ff2640 1 -- 172.21.15.45:0/2577796747 >> 172.21.15.45:0/2577796747 conn(0x7f85740d4fe0 msgr2=0x7f85740d0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.113+0000 7f8579ff2640 1 -- 172.21.15.45:0/2577796747 shutdown_connections 2024-05-08T05:46:12.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.113+0000 7f8579ff2640 1 -- 172.21.15.45:0/2577796747 wait complete. 2024-05-08T05:46:12.124 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-08T05:46:12.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.230+0000 7f507c740640 1 Processor -- start 2024-05-08T05:46:12.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.230+0000 7f507c740640 1 -- start start 2024-05-08T05:46:12.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.231+0000 7f507c740640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50741524f0 0x7f50740b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.231+0000 7f507c740640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f507415bc80 0x7f507406e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.231+0000 7f507c740640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f507406e8d0 0x7f5074073700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.231+0000 7f507c740640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f50740b51a0 con 0x7f50741524f0 2024-05-08T05:46:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.231+0000 7f507c740640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f50740b4590 con 0x7f507406e8d0 2024-05-08T05:46:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.231+0000 7f507c740640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f50740b4870 con 0x7f507415bc80 2024-05-08T05:46:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.231+0000 7f507a4dc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50741524f0 0x7f50740b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.231+0000 7f507a4dc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50741524f0 0x7f50740b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39804/0 (socket says 172.21.15.45:39804) 2024-05-08T05:46:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.231+0000 7f507a4dc640 1 -- 172.21.15.45:0/1537045171 learned_addr learned my addr 172.21.15.45:0/1537045171 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.231+0000 7f5079cdb640 1 --2- 172.21.15.45:0/1537045171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f507415bc80 0x7f507406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.231+0000 7f507acdd640 1 --2- 172.21.15.45:0/1537045171 >> v2:172.21.15.63:3300/0 conn(0x7f507406e8d0 0x7f5074073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.231+0000 7f507a4dc640 1 -- 172.21.15.45:0/1537045171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f507415bc80 msgr2=0x7f507406e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.231+0000 7f507a4dc640 1 --2- 172.21.15.45:0/1537045171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f507415bc80 0x7f507406e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.231+0000 7f507a4dc640 1 -- 172.21.15.45:0/1537045171 >> v2:172.21.15.63:3300/0 conn(0x7f507406e8d0 msgr2=0x7f5074073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.231+0000 7f507a4dc640 1 --2- 172.21.15.45:0/1537045171 >> v2:172.21.15.63:3300/0 conn(0x7f507406e8d0 0x7f5074073700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.231+0000 7f507a4dc640 1 -- 172.21.15.45:0/1537045171 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50740b4b50 con 0x7f50741524f0 2024-05-08T05:46:12.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.231+0000 7f507acdd640 1 --2- 172.21.15.45:0/1537045171 >> v2:172.21.15.63:3300/0 conn(0x7f507406e8d0 0x7f5074073700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:12.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.231+0000 7f5079cdb640 1 --2- 172.21.15.45:0/1537045171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f507415bc80 0x7f507406e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.232+0000 7f507a4dc640 1 --2- 172.21.15.45:0/1537045171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50741524f0 0x7f50740b3c20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f50740cce00 tx=0x7f507002c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e932310c184caf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.232+0000 7f50794da640 1 -- 172.21.15.45:0/1537045171 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507003a070 con 0x7f50741524f0 2024-05-08T05:46:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.232+0000 7f50794da640 1 -- 172.21.15.45:0/1537045171 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5070032480 con 0x7f50741524f0 2024-05-08T05:46:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.232+0000 7f50794da640 1 -- 172.21.15.45:0/1537045171 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070032850 con 0x7f50741524f0 2024-05-08T05:46:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.233+0000 7f507c740640 1 -- 172.21.15.45:0/1537045171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50741524f0 msgr2=0x7f50740b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.233+0000 7f507c740640 1 --2- 172.21.15.45:0/1537045171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50741524f0 0x7f50740b3c20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f50740cce00 tx=0x7f507002c670 comp rx=0 tx=0).stop 2024-05-08T05:46:12.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.233+0000 7f507c740640 1 -- 172.21.15.45:0/1537045171 shutdown_connections 2024-05-08T05:46:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.233+0000 7f507c740640 1 --2- 172.21.15.45:0/1537045171 >> v2:172.21.15.63:3300/0 conn(0x7f507406e8d0 0x7f5074073700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.233+0000 7f507c740640 1 --2- 172.21.15.45:0/1537045171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f507415bc80 0x7f507406e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.233+0000 7f507c740640 1 --2- 172.21.15.45:0/1537045171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50741524f0 0x7f50740b3c20 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.233+0000 7f507c740640 1 -- 172.21.15.45:0/1537045171 >> 172.21.15.45:0/1537045171 conn(0x7f507406c3a0 msgr2=0x7f507406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.233+0000 7f507c740640 1 -- 172.21.15.45:0/1537045171 shutdown_connections 2024-05-08T05:46:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.233+0000 7f507c740640 1 -- 172.21.15.45:0/1537045171 wait complete. 2024-05-08T05:46:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.234+0000 7f507c740640 1 Processor -- start 2024-05-08T05:46:12.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.234+0000 7f507c740640 1 -- start start 2024-05-08T05:46:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.235+0000 7f507c740640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f507406e8d0 0x7f507414d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.235+0000 7f507c740640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50741524f0 0x7f50740b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.235+0000 7f507c740640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f507415bc80 0x7f50740afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.235+0000 7f507c740640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f50740b5630 con 0x7f507415bc80 2024-05-08T05:46:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.235+0000 7f507c740640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50741528d0 con 0x7f507406e8d0 2024-05-08T05:46:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.235+0000 7f507c740640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f50740b2860 con 0x7f50741524f0 2024-05-08T05:46:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.235+0000 7f507acdd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f507415bc80 0x7f50740afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.235+0000 7f507acdd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f507415bc80 0x7f50740afa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39808/0 (socket says 172.21.15.45:39808) 2024-05-08T05:46:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.235+0000 7f507acdd640 1 -- 172.21.15.45:0/2470818021 learned_addr learned my addr 172.21.15.45:0/2470818021 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.235+0000 7f507a4dc640 1 --2- 172.21.15.45:0/2470818021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f507406e8d0 0x7f507414d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.235+0000 7f5079cdb640 1 --2- 172.21.15.45:0/2470818021 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50741524f0 0x7f50740b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.235+0000 7f507acdd640 1 -- 172.21.15.45:0/2470818021 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50741524f0 msgr2=0x7f50740b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.235+0000 7f507acdd640 1 --2- 172.21.15.45:0/2470818021 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50741524f0 0x7f50740b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.235+0000 7f507acdd640 1 -- 172.21.15.45:0/2470818021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f507406e8d0 msgr2=0x7f507414d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.235+0000 7f507acdd640 1 --2- 172.21.15.45:0/2470818021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f507406e8d0 0x7f507414d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.235+0000 7f507acdd640 1 -- 172.21.15.45:0/2470818021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50740b2b40 con 0x7f507415bc80 2024-05-08T05:46:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.235+0000 7f507acdd640 1 --2- 172.21.15.45:0/2470818021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f507415bc80 0x7f50740afa70 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f506c007e40 tx=0x7f506c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.235+0000 7f506affd640 1 -- 172.21.15.45:0/2470818021 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c01a070 con 0x7f507415bc80 2024-05-08T05:46:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.236+0000 7f506affd640 1 -- 172.21.15.45:0/2470818021 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f506c003b90 con 0x7f507415bc80 2024-05-08T05:46:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.236+0000 7f507c740640 1 -- 172.21.15.45:0/2470818021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50740b2e20 con 0x7f507415bc80 2024-05-08T05:46:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.236+0000 7f506affd640 1 -- 172.21.15.45:0/2470818021 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c00de80 con 0x7f507415bc80 2024-05-08T05:46:12.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.236+0000 7f507c740640 1 -- 172.21.15.45:0/2470818021 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50740b33e0 con 0x7f507415bc80 2024-05-08T05:46:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.237+0000 7f506affd640 1 -- 172.21.15.45:0/2470818021 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f506c01d070 con 0x7f507415bc80 2024-05-08T05:46:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.238+0000 7f506affd640 1 --2- 172.21.15.45:0/2470818021 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50580762d0 0x7f5058078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.238+0000 7f506affd640 1 -- 172.21.15.45:0/2470818021 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f506c097aa0 con 0x7f507415bc80 2024-05-08T05:46:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.238+0000 7f507a4dc640 1 --2- 172.21.15.45:0/2470818021 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50580762d0 0x7f5058078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.239+0000 7f507f457740 1 Processor -- start 2024-05-08T05:46:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.239+0000 7f507a4dc640 1 --2- 172.21.15.45:0/2470818021 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50580762d0 0x7f5058078790 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5070037000 tx=0x7f5070030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.239+0000 7f507f457740 1 -- start start 2024-05-08T05:46:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.240+0000 7f507f457740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 0x55dea64817c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.240+0000 7f507f457740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dea64556d0 0x55dea647dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.240+0000 7f507f457740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea647e2e0 0x55dea6480b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.240+0000 7f507f457740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dea6341920 con 0x55dea647d9c0 2024-05-08T05:46:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.240+0000 7f507f457740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dea63b6dd0 con 0x55dea647e2e0 2024-05-08T05:46:12.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.240+0000 7f507f457740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55dea63673e0 con 0x55dea64556d0 2024-05-08T05:46:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.240+0000 7f507a4dc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 0x55dea64817c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.240+0000 7f507a4dc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 0x55dea64817c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39822/0 (socket says 172.21.15.45:39822) 2024-05-08T05:46:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.240+0000 7f507a4dc640 1 -- 172.21.15.45:0/1541832498 learned_addr learned my addr 172.21.15.45:0/1541832498 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.240+0000 7f5079cdb640 1 --2- 172.21.15.45:0/1541832498 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dea64556d0 0x55dea647dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.240+0000 7f507acdd640 1 --2- 172.21.15.45:0/1541832498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea647e2e0 0x55dea6480b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.240+0000 7f507a4dc640 1 -- 172.21.15.45:0/1541832498 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dea64556d0 msgr2=0x55dea647dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.240+0000 7f507a4dc640 1 --2- 172.21.15.45:0/1541832498 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dea64556d0 0x55dea647dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.240+0000 7f507a4dc640 1 -- 172.21.15.45:0/1541832498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea647e2e0 msgr2=0x55dea6480b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.240+0000 7f507a4dc640 1 --2- 172.21.15.45:0/1541832498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea647e2e0 0x55dea6480b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.240+0000 7f507a4dc640 1 -- 172.21.15.45:0/1541832498 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dea61987c0 con 0x55dea647d9c0 2024-05-08T05:46:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.240+0000 7f507a4dc640 1 --2- 172.21.15.45:0/1541832498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 0x55dea64817c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f507000bbc0 tx=0x7f5070032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374ab84fb7eaf324 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.240+0000 7f5057fff640 1 -- 172.21.15.45:0/1541832498 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507003a040 con 0x55dea647d9c0 2024-05-08T05:46:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.240+0000 7f5057fff640 1 -- 172.21.15.45:0/1541832498 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5070032d30 con 0x55dea647d9c0 2024-05-08T05:46:12.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.240+0000 7f5057fff640 1 -- 172.21.15.45:0/1541832498 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507004a630 con 0x55dea647d9c0 2024-05-08T05:46:12.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.241+0000 7f507f457740 1 -- 172.21.15.45:0/1541832498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 msgr2=0x55dea64817c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.241+0000 7f507f457740 1 --2- 172.21.15.45:0/1541832498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 0x55dea64817c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f507000bbc0 tx=0x7f5070032000 comp rx=0 tx=0).stop 2024-05-08T05:46:12.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.241+0000 7f507f457740 1 -- 172.21.15.45:0/1541832498 shutdown_connections 2024-05-08T05:46:12.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.241+0000 7f507f457740 1 --2- 172.21.15.45:0/1541832498 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dea64556d0 0x55dea647dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.241+0000 7f507f457740 1 --2- 172.21.15.45:0/1541832498 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea647e2e0 0x55dea6480b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.241+0000 7f507f457740 1 --2- 172.21.15.45:0/1541832498 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 0x55dea64817c0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.241+0000 7f507f457740 1 -- 172.21.15.45:0/1541832498 >> 172.21.15.45:0/1541832498 conn(0x55dea6453560 msgr2=0x55dea649bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.241+0000 7f507f457740 1 -- 172.21.15.45:0/1541832498 shutdown_connections 2024-05-08T05:46:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.241+0000 7f507f457740 1 -- 172.21.15.45:0/1541832498 wait complete. 2024-05-08T05:46:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.242+0000 7f507f457740 1 -- start start 2024-05-08T05:46:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.242+0000 7f507f457740 1 Processor -- start 2024-05-08T05:46:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f507f457740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 0x55dea647fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f507f457740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dea647e2e0 0x55dea647b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f507f457740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea647ba80 0x55dea649f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f507f457740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dea63bf440 con 0x55dea647d9c0 2024-05-08T05:46:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f507f457740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dea63ba120 con 0x55dea647ba80 2024-05-08T05:46:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f507f457740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55dea63c8210 con 0x55dea647e2e0 2024-05-08T05:46:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f507f457740 10 client.0 _ll_register_callbacks cb 0x55dea635f690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:46:12.248 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f507a4dc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 0x55dea647fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f507a4dc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 0x55dea647fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39834/0 (socket says 172.21.15.45:39834) 2024-05-08T05:46:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f507a4dc640 1 -- 172.21.15.45:0/4103524026 learned_addr learned my addr 172.21.15.45:0/4103524026 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f5079cdb640 1 --2- 172.21.15.45:0/4103524026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dea647e2e0 0x55dea647b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f507acdd640 1 --2- 172.21.15.45:0/4103524026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea647ba80 0x55dea649f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f507a4dc640 1 -- 172.21.15.45:0/4103524026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dea647e2e0 msgr2=0x55dea647b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f507a4dc640 1 --2- 172.21.15.45:0/4103524026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dea647e2e0 0x55dea647b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f507a4dc640 1 -- 172.21.15.45:0/4103524026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea647ba80 msgr2=0x55dea649f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f507a4dc640 1 --2- 172.21.15.45:0/4103524026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea647ba80 0x55dea649f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f507a4dc640 1 -- 172.21.15.45:0/4103524026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dea61f8f50 con 0x55dea647d9c0 2024-05-08T05:46:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f5079cdb640 1 --2- 172.21.15.45:0/4103524026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dea647e2e0 0x55dea647b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f507a4dc640 1 --2- 172.21.15.45:0/4103524026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 0x55dea647fa90 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f507004b000 tx=0x7f507002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd564795819594d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f50557fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:46:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f50557fa640 1 -- 172.21.15.45:0/4103524026 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507005f060 con 0x55dea647d9c0 2024-05-08T05:46:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f50557fa640 1 -- 172.21.15.45:0/4103524026 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f507004acd0 con 0x55dea647d9c0 2024-05-08T05:46:12.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.243+0000 7f507f457740 1 -- 172.21.15.45:0/4103524026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5070032d30 con 0x55dea647d9c0 2024-05-08T05:46:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.244+0000 7f50557fa640 1 -- 172.21.15.45:0/4103524026 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f5070065050 con 0x55dea647d9c0 2024-05-08T05:46:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.244+0000 7f507f457740 10 client.6483 fetch_fsmap learned FSMap version 51 2024-05-08T05:46:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.244+0000 7f507f457740 1 -- 172.21.15.45:0/4103524026 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55dea608c0b0 con 0x55dea647d9c0 2024-05-08T05:46:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.244+0000 7f50557fa640 1 -- 172.21.15.45:0/4103524026 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5070032d30 con 0x55dea647d9c0 2024-05-08T05:46:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.244+0000 7f50557fa640 1 -- 172.21.15.45:0/4103524026 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f5070068020 con 0x55dea647d9c0 2024-05-08T05:46:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.244+0000 7f507f457740 10 client.6483 fetch_fsmap finished waiting for FSMap version 51 2024-05-08T05:46:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.244+0000 7f507f457740 10 client.6483 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T05:46:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.244+0000 7f507f457740 20 client.6483 populate_metadata read hostname 'smithi045' 2024-05-08T05:46:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.244+0000 7f507f457740 1 --2- 172.21.15.45:0/4103524026 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55dea64bb960 0x55dea64bdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.244+0000 7f507f457740 4 client.6483 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:46:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.244+0000 7f507f457740 1 -- 172.21.15.45:0/4103524026 --> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dea63c8a20 con 0x55dea64bb960 2024-05-08T05:46:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.244+0000 7f5079cdb640 1 --2- 172.21.15.45:0/4103524026 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55dea64bb960 0x55dea64bdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.244+0000 7f50557fa640 10 client.6483 ms_handle_connect on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.244+0000 7f5079cdb640 1 --2- 172.21.15.45:0/4103524026 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55dea64bb960 0x55dea64bdd50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=142018ecbe08d1ce server_cookie=3eddbae622bcbdea in_seq=0 out_seq=0 2024-05-08T05:46:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.245+0000 7f50557fa640 1 -- 172.21.15.45:0/4103524026 <== mds.0 v2:172.21.15.45:6833/4183668301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dea63c8a20 con 0x55dea64bb960 2024-05-08T05:46:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.245+0000 7f50557fa640 10 client.6483 handle_command_reply: tid=0 2024-05-08T05:46:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.245+0000 7f507f457740 1 client.6483 shutdown 2024-05-08T05:46:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.245+0000 7f507f457740 1 -- 172.21.15.45:0/4103524026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 msgr2=0x55dea647fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.245+0000 7f507f457740 1 --2- 172.21.15.45:0/4103524026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 0x55dea647fa90 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f507004b000 tx=0x7f507002e000 comp rx=0 tx=0).stop 2024-05-08T05:46:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.245+0000 7f507f457740 1 -- 172.21.15.45:0/4103524026 shutdown_connections 2024-05-08T05:46:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.245+0000 7f507f457740 1 --2- 172.21.15.45:0/4103524026 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55dea64bb960 0x55dea64bdd50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.245+0000 7f507f457740 1 --2- 172.21.15.45:0/4103524026 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dea647e2e0 0x55dea647b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.245+0000 7f507f457740 1 --2- 172.21.15.45:0/4103524026 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea647ba80 0x55dea649f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.245+0000 7f50557fa640 0 client.6483 ms_handle_reset on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.245+0000 7f507f457740 1 --2- 172.21.15.45:0/4103524026 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 0x55dea647fa90 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.245+0000 7f507f457740 1 -- 172.21.15.45:0/4103524026 >> 172.21.15.45:0/4103524026 conn(0x55dea64556d0 msgr2=0x55dea6456460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.245+0000 7f507f457740 1 -- 172.21.15.45:0/4103524026 shutdown_connections 2024-05-08T05:46:12.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.245+0000 7f507f457740 1 -- 172.21.15.45:0/4103524026 wait complete. 2024-05-08T05:46:12.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.245+0000 7f507f457740 20 client.6483 trim_cache size 0 max 16384 2024-05-08T05:46:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.259+0000 7f507f457740 1 Processor -- start 2024-05-08T05:46:12.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.259+0000 7f507f457740 1 -- start start 2024-05-08T05:46:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.260+0000 7f507f457740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dea647ba80 0x55dea649bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.260+0000 7f507f457740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea647d9c0 0x55dea647a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.260+0000 7f507f457740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647e2e0 0x55dea647ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.260+0000 7f507f457740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f50740cfb50 con 0x55dea647e2e0 2024-05-08T05:46:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.260+0000 7f507f457740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50740b1ec0 con 0x55dea647d9c0 2024-05-08T05:46:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.260+0000 7f507f457740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f50740cf500 con 0x55dea647ba80 2024-05-08T05:46:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.260+0000 7f507acdd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647e2e0 0x55dea647ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.260+0000 7f507acdd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647e2e0 0x55dea647ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39842/0 (socket says 172.21.15.45:39842) 2024-05-08T05:46:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.260+0000 7f507acdd640 1 -- 172.21.15.45:0/1836986534 learned_addr learned my addr 172.21.15.45:0/1836986534 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.260+0000 7f507acdd640 1 -- 172.21.15.45:0/1836986534 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dea647ba80 msgr2=0x55dea649bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.260+0000 7f507acdd640 1 --2- 172.21.15.45:0/1836986534 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dea647ba80 0x55dea649bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.260+0000 7f507acdd640 1 -- 172.21.15.45:0/1836986534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea647d9c0 msgr2=0x55dea647a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.260+0000 7f507acdd640 1 --2- 172.21.15.45:0/1836986534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea647d9c0 0x55dea647a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.260+0000 7f507acdd640 1 -- 172.21.15.45:0/1836986534 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dea647b8b0 con 0x55dea647e2e0 2024-05-08T05:46:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.260+0000 7f507acdd640 1 --2- 172.21.15.45:0/1836986534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647e2e0 0x55dea647ab30 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f506c007e10 tx=0x7f506c05d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f346ea839d2462 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.261+0000 7f50577fe640 1 -- 172.21.15.45:0/1836986534 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c064e30 con 0x55dea647e2e0 2024-05-08T05:46:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.261+0000 7f50577fe640 1 -- 172.21.15.45:0/1836986534 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f506c079020 con 0x55dea647e2e0 2024-05-08T05:46:12.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.261+0000 7f50577fe640 1 -- 172.21.15.45:0/1836986534 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c067350 con 0x55dea647e2e0 2024-05-08T05:46:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.261+0000 7f507f457740 1 -- 172.21.15.45:0/1836986534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647e2e0 msgr2=0x55dea647ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.261+0000 7f507f457740 1 --2- 172.21.15.45:0/1836986534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647e2e0 0x55dea647ab30 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f506c007e10 tx=0x7f506c05d280 comp rx=0 tx=0).stop 2024-05-08T05:46:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.261+0000 7f507f457740 1 -- 172.21.15.45:0/1836986534 shutdown_connections 2024-05-08T05:46:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.261+0000 7f507f457740 1 --2- 172.21.15.45:0/1836986534 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647e2e0 0x55dea647ab30 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.261+0000 7f507f457740 1 --2- 172.21.15.45:0/1836986534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea647d9c0 0x55dea647a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.261+0000 7f507f457740 1 --2- 172.21.15.45:0/1836986534 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dea647ba80 0x55dea649bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.261+0000 7f507f457740 1 -- 172.21.15.45:0/1836986534 >> 172.21.15.45:0/1836986534 conn(0x55dea64556d0 msgr2=0x55dea647ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.261+0000 7f507f457740 1 -- 172.21.15.45:0/1836986534 shutdown_connections 2024-05-08T05:46:12.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.261+0000 7f507f457740 1 -- 172.21.15.45:0/1836986534 wait complete. 2024-05-08T05:46:12.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.262+0000 7f507f457740 1 -- start start 2024-05-08T05:46:12.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.262+0000 7f507f457740 1 Processor -- start 2024-05-08T05:46:12.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f507f457740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dea647ba80 0x55dea651dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f507f457740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 0x55dea64b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f507f457740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea647e2e0 0x55dea64b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f507f457740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dea64b0200 con 0x55dea647d9c0 2024-05-08T05:46:12.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f507f457740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dea6370a20 con 0x55dea647e2e0 2024-05-08T05:46:12.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f507f457740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55dea649cd20 con 0x55dea647ba80 2024-05-08T05:46:12.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f507f457740 10 client.0 _ll_register_callbacks cb 0x7f506400b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:46:12.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f5079cdb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 0x55dea64b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f5079cdb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 0x55dea64b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39850/0 (socket says 172.21.15.45:39850) 2024-05-08T05:46:12.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f5079cdb640 1 -- 172.21.15.45:0/2914941546 learned_addr learned my addr 172.21.15.45:0/2914941546 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:12.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f507a4dc640 1 --2- 172.21.15.45:0/2914941546 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dea647ba80 0x55dea651dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f507acdd640 1 --2- 172.21.15.45:0/2914941546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea647e2e0 0x55dea64b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f5079cdb640 1 -- 172.21.15.45:0/2914941546 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dea647ba80 msgr2=0x55dea651dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f5079cdb640 1 --2- 172.21.15.45:0/2914941546 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dea647ba80 0x55dea651dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f5079cdb640 1 -- 172.21.15.45:0/2914941546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea647e2e0 msgr2=0x55dea64b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f5079cdb640 1 --2- 172.21.15.45:0/2914941546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea647e2e0 0x55dea64b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f5079cdb640 1 -- 172.21.15.45:0/2914941546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dea64b3020 con 0x55dea647d9c0 2024-05-08T05:46:12.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f507acdd640 1 --2- 172.21.15.45:0/2914941546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea647e2e0 0x55dea64b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:12.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f5079cdb640 1 --2- 172.21.15.45:0/2914941546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 0x55dea64b25a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5064003800 tx=0x7f506400b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6246d1e425e204f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f50557fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:46:12.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.263+0000 7f50557fa640 1 -- 172.21.15.45:0/2914941546 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506401f3f0 con 0x55dea647d9c0 2024-05-08T05:46:12.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.264+0000 7f50557fa640 1 -- 172.21.15.45:0/2914941546 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f506401f590 con 0x55dea647d9c0 2024-05-08T05:46:12.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.264+0000 7f507f457740 1 -- 172.21.15.45:0/2914941546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f506c079020 con 0x55dea647d9c0 2024-05-08T05:46:12.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.264+0000 7f50557fa640 1 -- 172.21.15.45:0/2914941546 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f506402d050 con 0x55dea647d9c0 2024-05-08T05:46:12.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.264+0000 7f507f457740 10 client.6489 fetch_fsmap learned FSMap version 51 2024-05-08T05:46:12.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.264+0000 7f507f457740 1 -- 172.21.15.45:0/2914941546 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55dea64568e0 con 0x55dea647d9c0 2024-05-08T05:46:12.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.264+0000 7f50557fa640 1 -- 172.21.15.45:0/2914941546 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f506c079020 con 0x55dea647d9c0 2024-05-08T05:46:12.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.264+0000 7f50557fa640 1 -- 172.21.15.45:0/2914941546 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f5064030020 con 0x55dea647d9c0 2024-05-08T05:46:12.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.264+0000 7f507f457740 10 client.6489 fetch_fsmap finished waiting for FSMap version 51 2024-05-08T05:46:12.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.264+0000 7f507f457740 10 client.6489 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T05:46:12.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.264+0000 7f507f457740 20 client.6489 populate_metadata read hostname 'smithi045' 2024-05-08T05:46:12.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.264+0000 7f507f457740 1 --2- 172.21.15.45:0/2914941546 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55dea64af330 0x55dea65296a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.264+0000 7f507f457740 4 client.6489 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-08T05:46:12.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.264+0000 7f507f457740 1 -- 172.21.15.45:0/2914941546 --> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55dea63c2890 con 0x55dea64af330 2024-05-08T05:46:12.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.264+0000 7f507a4dc640 1 --2- 172.21.15.45:0/2914941546 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55dea64af330 0x55dea65296a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.264+0000 7f50557fa640 10 client.6489 ms_handle_connect on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:12.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.264+0000 7f507a4dc640 1 --2- 172.21.15.45:0/2914941546 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55dea64af330 0x55dea65296a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4ebc5903243782a server_cookie=de83fa485b3ce731 in_seq=0 out_seq=0 2024-05-08T05:46:12.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.265+0000 7f50557fa640 1 -- 172.21.15.45:0/2914941546 <== mds.0 v2:172.21.15.45:6833/4183668301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+236 (crc 0 0 0) 0x55dea63c2890 con 0x55dea64af330 2024-05-08T05:46:12.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.265+0000 7f50557fa640 10 client.6489 handle_command_reply: tid=0 2024-05-08T05:46:12.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.265+0000 7f507f457740 1 client.6489 shutdown 2024-05-08T05:46:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.265+0000 7f507f457740 1 -- 172.21.15.45:0/2914941546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 msgr2=0x55dea64b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.265+0000 7f507f457740 1 --2- 172.21.15.45:0/2914941546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 0x55dea64b25a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5064003800 tx=0x7f506400b500 comp rx=0 tx=0).stop 2024-05-08T05:46:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.265+0000 7f507f457740 1 -- 172.21.15.45:0/2914941546 shutdown_connections 2024-05-08T05:46:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.265+0000 7f507f457740 1 --2- 172.21.15.45:0/2914941546 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55dea64af330 0x55dea65296a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.265+0000 7f507f457740 1 --2- 172.21.15.45:0/2914941546 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dea647e2e0 0x55dea64b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.265+0000 7f507f457740 1 --2- 172.21.15.45:0/2914941546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dea647d9c0 0x55dea64b25a0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.265+0000 7f507f457740 1 --2- 172.21.15.45:0/2914941546 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dea647ba80 0x55dea651dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.265+0000 7f50557fa640 0 client.6489 ms_handle_reset on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.265+0000 7f507f457740 1 -- 172.21.15.45:0/2914941546 >> 172.21.15.45:0/2914941546 conn(0x55dea64556d0 msgr2=0x55dea649bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.265+0000 7f507f457740 1 -- 172.21.15.45:0/2914941546 shutdown_connections 2024-05-08T05:46:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.265+0000 7f507f457740 1 -- 172.21.15.45:0/2914941546 wait complete. 2024-05-08T05:46:12.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.265+0000 7f507f457740 20 client.6489 trim_cache size 0 max 16384 2024-05-08T05:46:12.274 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:46:12.274 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-05-08T05:46:12.274 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:12.274 INFO:teuthology.orchestra.run.smithi045.stdout: "damage_type": "dentry", 2024-05-08T05:46:12.274 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 623378145, 2024-05-08T05:46:12.274 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 1099511627776, 2024-05-08T05:46:12.274 INFO:teuthology.orchestra.run.smithi045.stdout: "frag": "*", 2024-05-08T05:46:12.274 INFO:teuthology.orchestra.run.smithi045.stdout: "dname": "file_to_be_damaged", 2024-05-08T05:46:12.274 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_id": "head", 2024-05-08T05:46:12.274 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/subdir/file_to_be_damaged" 2024-05-08T05:46:12.274 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:12.274 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-05-08T05:46:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.265+0000 7f507c740640 1 -- 172.21.15.45:0/2470818021 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50580762d0 msgr2=0x7f5058078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.265+0000 7f507c740640 1 --2- 172.21.15.45:0/2470818021 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50580762d0 0x7f5058078790 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5070037000 tx=0x7f5070030000 comp rx=0 tx=0).stop 2024-05-08T05:46:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.265+0000 7f507c740640 1 -- 172.21.15.45:0/2470818021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f507415bc80 msgr2=0x7f50740afa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.265+0000 7f507c740640 1 --2- 172.21.15.45:0/2470818021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f507415bc80 0x7f50740afa70 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f506c007e40 tx=0x7f506c00d4a0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.266+0000 7f507c740640 1 -- 172.21.15.45:0/2470818021 shutdown_connections 2024-05-08T05:46:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.266+0000 7f507c740640 1 --2- 172.21.15.45:0/2470818021 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50580762d0 0x7f5058078790 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.266+0000 7f507c740640 1 --2- 172.21.15.45:0/2470818021 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50741524f0 0x7f50740b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.266+0000 7f507c740640 1 --2- 172.21.15.45:0/2470818021 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f507415bc80 0x7f50740afa70 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.266+0000 7f507c740640 1 --2- 172.21.15.45:0/2470818021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f507406e8d0 0x7f507414d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.266+0000 7f507c740640 1 -- 172.21.15.45:0/2470818021 >> 172.21.15.45:0/2470818021 conn(0x7f507406c3a0 msgr2=0x7f50740d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.266+0000 7f507c740640 1 -- 172.21.15.45:0/2470818021 shutdown_connections 2024-05-08T05:46:12.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.266+0000 7f507c740640 1 -- 172.21.15.45:0/2470818021 wait complete. 2024-05-08T05:46:12.277 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T05:46:12.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.380+0000 7f42987a4640 1 Processor -- start 2024-05-08T05:46:12.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.380+0000 7f42987a4640 1 -- start start 2024-05-08T05:46:12.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.381+0000 7f42987a4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f429006f060 0x7f429006e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.381+0000 7f42987a4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f429006fe30 0x7f42900cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.381+0000 7f42987a4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f429015d2a0 con 0x7f42900d0360 2024-05-08T05:46:12.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.381+0000 7f42987a4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f42900d07c0 con 0x7f429006fe30 2024-05-08T05:46:12.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.381+0000 7f42987a4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f429006f440 con 0x7f429006f060 2024-05-08T05:46:12.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.381+0000 7f4295d3f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f429006fe30 0x7f42900cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.381+0000 7f4296d41640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f42900d0360 0x7f42900b3f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50002/0 (socket says 172.21.15.45:50002) 2024-05-08T05:46:12.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.381+0000 7f4296d41640 1 -- 172.21.15.45:0/331648987 learned_addr learned my addr 172.21.15.45:0/331648987 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:12.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.381+0000 7f4296540640 1 --2- 172.21.15.45:0/331648987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f429006f060 0x7f429006e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.381+0000 7f4295d3f640 1 -- 172.21.15.45:0/331648987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f429006f060 msgr2=0x7f429006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.381+0000 7f4295d3f640 1 --2- 172.21.15.45:0/331648987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f429006f060 0x7f429006e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.381+0000 7f4295d3f640 1 -- 172.21.15.45:0/331648987 >> v1:172.21.15.45:6789/0 conn(0x7f42900d0360 legacy=0x7f42900b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.381+0000 7f4295d3f640 1 -- 172.21.15.45:0/331648987 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42901609b0 con 0x7f429006fe30 2024-05-08T05:46:12.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.381+0000 7f429553e640 1 -- 172.21.15.45:0/331648987 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1671279874 0 0) 0x7f429015d2a0 con 0x7f42900d0360 2024-05-08T05:46:12.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.382+0000 7f4295d3f640 1 --2- 172.21.15.45:0/331648987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f429006fe30 0x7f42900cfe20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f42800097a0 tx=0x7f428002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d7492bdcae68c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.382+0000 7f429553e640 1 -- 172.21.15.45:0/331648987 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4280037070 con 0x7f429006fe30 2024-05-08T05:46:12.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.382+0000 7f429553e640 1 -- 172.21.15.45:0/331648987 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4280032440 con 0x7f429006fe30 2024-05-08T05:46:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.382+0000 7f429553e640 1 -- 172.21.15.45:0/331648987 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4280032810 con 0x7f429006fe30 2024-05-08T05:46:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.382+0000 7f42987a4640 1 -- 172.21.15.45:0/331648987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f429006fe30 msgr2=0x7f42900cfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.382+0000 7f42987a4640 1 --2- 172.21.15.45:0/331648987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f429006fe30 0x7f42900cfe20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f42800097a0 tx=0x7f428002a180 comp rx=0 tx=0).stop 2024-05-08T05:46:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.383+0000 7f42987a4640 1 -- 172.21.15.45:0/331648987 shutdown_connections 2024-05-08T05:46:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.383+0000 7f42987a4640 1 --2- 172.21.15.45:0/331648987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f429006fe30 0x7f42900cfe20 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.383+0000 7f42987a4640 1 --2- 172.21.15.45:0/331648987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f429006f060 0x7f429006e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.383+0000 7f42987a4640 1 -- 172.21.15.45:0/331648987 >> 172.21.15.45:0/331648987 conn(0x7f429006c3a0 msgr2=0x7f429006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.383+0000 7f42987a4640 1 -- 172.21.15.45:0/331648987 shutdown_connections 2024-05-08T05:46:12.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.383+0000 7f42987a4640 1 -- 172.21.15.45:0/331648987 wait complete. 2024-05-08T05:46:12.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.384+0000 7f42987a4640 1 Processor -- start 2024-05-08T05:46:12.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.384+0000 7f42987a4640 1 -- start start 2024-05-08T05:46:12.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f42987a4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f429006f060 0x7f4290154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f42987a4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f429006fe30 0x7f429015f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f42987a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42900d0360 0x7f429015fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f42987a4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4290161250 con 0x7f42900d0360 2024-05-08T05:46:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f42987a4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4290160f70 con 0x7f429006f060 2024-05-08T05:46:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f42987a4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4290161530 con 0x7f429006fe30 2024-05-08T05:46:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f4296d41640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42900d0360 0x7f429015fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f4296d41640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42900d0360 0x7f429015fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39868/0 (socket says 172.21.15.45:39868) 2024-05-08T05:46:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f4296d41640 1 -- 172.21.15.45:0/4127863151 learned_addr learned my addr 172.21.15.45:0/4127863151 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f4295d3f640 1 --2- 172.21.15.45:0/4127863151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f429006fe30 0x7f429015f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f4296540640 1 --2- 172.21.15.45:0/4127863151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f429006f060 0x7f4290154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f4296d41640 1 -- 172.21.15.45:0/4127863151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f429006fe30 msgr2=0x7f429015f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f4296d41640 1 --2- 172.21.15.45:0/4127863151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f429006fe30 0x7f429015f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f4296d41640 1 -- 172.21.15.45:0/4127863151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f429006f060 msgr2=0x7f4290154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f4296d41640 1 --2- 172.21.15.45:0/4127863151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f429006f060 0x7f4290154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f4296d41640 1 -- 172.21.15.45:0/4127863151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42900b20d0 con 0x7f42900d0360 2024-05-08T05:46:12.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f4296d41640 1 --2- 172.21.15.45:0/4127863151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42900d0360 0x7f429015fe80 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f428800c6e0 tx=0x7f428800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f4286ffd640 1 -- 172.21.15.45:0/4127863151 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288018070 con 0x7f42900d0360 2024-05-08T05:46:12.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f4286ffd640 1 -- 172.21.15.45:0/4127863151 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4288004d60 con 0x7f42900d0360 2024-05-08T05:46:12.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f42987a4640 1 -- 172.21.15.45:0/4127863151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4290155030 con 0x7f42900d0360 2024-05-08T05:46:12.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f4286ffd640 1 -- 172.21.15.45:0/4127863151 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428800a880 con 0x7f42900d0360 2024-05-08T05:46:12.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.385+0000 7f42987a4640 1 -- 172.21.15.45:0/4127863151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f429015e710 con 0x7f42900d0360 2024-05-08T05:46:12.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.386+0000 7f42987a4640 1 -- 172.21.15.45:0/4127863151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42900693f0 con 0x7f42900d0360 2024-05-08T05:46:12.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.387+0000 7f4286ffd640 1 -- 172.21.15.45:0/4127863151 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f428801b070 con 0x7f42900d0360 2024-05-08T05:46:12.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.387+0000 7f4286ffd640 1 --2- 172.21.15.45:0/4127863151 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4274076300 0x7f42740787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.388+0000 7f4286ffd640 1 -- 172.21.15.45:0/4127863151 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f4288096be0 con 0x7f42900d0360 2024-05-08T05:46:12.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.388+0000 7f4296540640 1 --2- 172.21.15.45:0/4127863151 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4274076300 0x7f42740787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.388+0000 7f4296540640 1 --2- 172.21.15.45:0/4127863151 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4274076300 0x7f42740787c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f428c009b40 tx=0x7f428c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.392+0000 7f4286ffd640 1 -- 172.21.15.45:0/4127863151 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f428805fd10 con 0x7f42900d0360 2024-05-08T05:46:12.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.623+0000 7f42987a4640 1 -- 172.21.15.45:0/4127863151 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f429015c1f0 con 0x7f42900d0360 2024-05-08T05:46:12.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.623+0000 7f4286ffd640 1 -- 172.21.15.45:0/4127863151 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f4288064010 con 0x7f42900d0360 2024-05-08T05:46:12.623 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:46:12.623 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-08T05:46:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.625+0000 7f42987a4640 1 -- 172.21.15.45:0/4127863151 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4274076300 msgr2=0x7f42740787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.625+0000 7f42987a4640 1 --2- 172.21.15.45:0/4127863151 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4274076300 0x7f42740787c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f428c009b40 tx=0x7f428c009190 comp rx=0 tx=0).stop 2024-05-08T05:46:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.625+0000 7f42987a4640 1 -- 172.21.15.45:0/4127863151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42900d0360 msgr2=0x7f429015fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.625+0000 7f42987a4640 1 --2- 172.21.15.45:0/4127863151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42900d0360 0x7f429015fe80 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f428800c6e0 tx=0x7f428800cba0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.625+0000 7f42987a4640 1 -- 172.21.15.45:0/4127863151 shutdown_connections 2024-05-08T05:46:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.625+0000 7f42987a4640 1 --2- 172.21.15.45:0/4127863151 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4274076300 0x7f42740787c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.625+0000 7f42987a4640 1 --2- 172.21.15.45:0/4127863151 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f429006fe30 0x7f429015f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.625+0000 7f42987a4640 1 --2- 172.21.15.45:0/4127863151 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42900d0360 0x7f429015fe80 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.625+0000 7f42987a4640 1 --2- 172.21.15.45:0/4127863151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f429006f060 0x7f4290154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.625+0000 7f42987a4640 1 -- 172.21.15.45:0/4127863151 >> 172.21.15.45:0/4127863151 conn(0x7f429006c3a0 msgr2=0x7f429015c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.625+0000 7f42987a4640 1 -- 172.21.15.45:0/4127863151 shutdown_connections 2024-05-08T05:46:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.625+0000 7f42987a4640 1 -- 172.21.15.45:0/4127863151 wait complete. 2024-05-08T05:46:12.636 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-08T05:46:12.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:46:12.636 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-05-08T05:46:12.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.744+0000 7f809874e640 1 Processor -- start 2024-05-08T05:46:12.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.744+0000 7f809874e640 1 -- start start 2024-05-08T05:46:12.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.745+0000 7f809874e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f80900d3a50 0x7f8090162920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.745+0000 7f809874e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80900b50d0 con 0x7f80900d3670 2024-05-08T05:46:12.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.745+0000 7f809874e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80900cfc00 con 0x7f80900d0940 2024-05-08T05:46:12.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.745+0000 7f809874e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f80900d2a10 con 0x7f80900d3a50 2024-05-08T05:46:12.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.745+0000 7f8095ce9640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f80900d3a50 0x7f8090162920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.745+0000 7f80964ea640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f80900d3670 0x7f80900b49c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50010/0 (socket says 172.21.15.45:50010) 2024-05-08T05:46:12.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.745+0000 7f80964ea640 1 -- 172.21.15.45:0/4110367275 learned_addr learned my addr 172.21.15.45:0/4110367275 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:12.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.745+0000 7f8095ce9640 1 -- 172.21.15.45:0/4110367275 >> v1:172.21.15.63:6789/0 conn(0x7f80900d0940 legacy=0x7f80900d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.745+0000 7f8095ce9640 1 -- 172.21.15.45:0/4110367275 >> v1:172.21.15.45:6789/0 conn(0x7f80900d3670 legacy=0x7f80900b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.745+0000 7f8095ce9640 1 -- 172.21.15.45:0/4110367275 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80900cfea0 con 0x7f80900d3a50 2024-05-08T05:46:12.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.745+0000 7f80954e8640 1 -- 172.21.15.45:0/4110367275 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 388114148 0 0) 0x7f80900b50d0 con 0x7f80900d3670 2024-05-08T05:46:12.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.746+0000 7f8095ce9640 1 --2- 172.21.15.45:0/4110367275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80900d3a50 0x7f8090162920 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f80800097a0 tx=0x7f808002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cda2fc0fda69bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.746+0000 7f80954e8640 1 -- 172.21.15.45:0/4110367275 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080037070 con 0x7f80900d3a50 2024-05-08T05:46:12.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.746+0000 7f80954e8640 1 -- 172.21.15.45:0/4110367275 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8080032440 con 0x7f80900d3a50 2024-05-08T05:46:12.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.746+0000 7f80954e8640 1 -- 172.21.15.45:0/4110367275 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080032810 con 0x7f80900d3a50 2024-05-08T05:46:12.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.746+0000 7f809874e640 1 -- 172.21.15.45:0/4110367275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80900d3a50 msgr2=0x7f8090162920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.746+0000 7f809874e640 1 --2- 172.21.15.45:0/4110367275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80900d3a50 0x7f8090162920 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f80800097a0 tx=0x7f808002a230 comp rx=0 tx=0).stop 2024-05-08T05:46:12.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.747+0000 7f809874e640 1 -- 172.21.15.45:0/4110367275 shutdown_connections 2024-05-08T05:46:12.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.747+0000 7f809874e640 1 --2- 172.21.15.45:0/4110367275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80900d3a50 0x7f8090162920 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.747+0000 7f809874e640 1 -- 172.21.15.45:0/4110367275 >> 172.21.15.45:0/4110367275 conn(0x7f809009e390 msgr2=0x7f80900d5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.747+0000 7f809874e640 1 -- 172.21.15.45:0/4110367275 shutdown_connections 2024-05-08T05:46:12.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.747+0000 7f809874e640 1 -- 172.21.15.45:0/4110367275 wait complete. 2024-05-08T05:46:12.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.748+0000 7f809874e640 1 Processor -- start 2024-05-08T05:46:12.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.748+0000 7f809874e640 1 -- start start 2024-05-08T05:46:12.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.749+0000 7f809874e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80900d0940 0x7f80901576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.749+0000 7f809874e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80900d3670 0x7f8090154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.749+0000 7f809874e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80900d3a50 0x7f80901600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.749+0000 7f809874e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f80900d0140 con 0x7f80900d0940 2024-05-08T05:46:12.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.749+0000 7f809874e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f80900d04b0 con 0x7f80900d3a50 2024-05-08T05:46:12.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.749+0000 7f809874e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8090162e60 con 0x7f80900d3670 2024-05-08T05:46:12.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.749+0000 7f80964ea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80900d0940 0x7f80901576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.749+0000 7f80964ea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80900d0940 0x7f80901576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39878/0 (socket says 172.21.15.45:39878) 2024-05-08T05:46:12.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.749+0000 7f80964ea640 1 -- 172.21.15.45:0/2456451703 learned_addr learned my addr 172.21.15.45:0/2456451703 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:12.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.749+0000 7f8096ceb640 1 --2- 172.21.15.45:0/2456451703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80900d3a50 0x7f80901600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.749+0000 7f8095ce9640 1 --2- 172.21.15.45:0/2456451703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80900d3670 0x7f8090154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.749+0000 7f80964ea640 1 -- 172.21.15.45:0/2456451703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80900d3670 msgr2=0x7f8090154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.749+0000 7f80964ea640 1 --2- 172.21.15.45:0/2456451703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80900d3670 0x7f8090154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.749+0000 7f80964ea640 1 -- 172.21.15.45:0/2456451703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80900d3a50 msgr2=0x7f80901600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.749+0000 7f80964ea640 1 --2- 172.21.15.45:0/2456451703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80900d3a50 0x7f80901600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.749+0000 7f80964ea640 1 -- 172.21.15.45:0/2456451703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8090160980 con 0x7f80900d0940 2024-05-08T05:46:12.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.749+0000 7f80964ea640 1 --2- 172.21.15.45:0/2456451703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80900d0940 0x7f80901576d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f808c00c480 tx=0x7f808c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.749+0000 7f8086ffd640 1 -- 172.21.15.45:0/2456451703 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c018070 con 0x7f80900d0940 2024-05-08T05:46:12.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.749+0000 7f8086ffd640 1 -- 172.21.15.45:0/2456451703 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f808c0074e0 con 0x7f80900d0940 2024-05-08T05:46:12.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.749+0000 7f809874e640 1 -- 172.21.15.45:0/2456451703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8090156930 con 0x7f80900d0940 2024-05-08T05:46:12.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.750+0000 7f8086ffd640 1 -- 172.21.15.45:0/2456451703 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c004a20 con 0x7f80900d0940 2024-05-08T05:46:12.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.750+0000 7f809874e640 1 -- 172.21.15.45:0/2456451703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8090156ef0 con 0x7f80900d0940 2024-05-08T05:46:12.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.752+0000 7f8086ffd640 1 -- 172.21.15.45:0/2456451703 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f808c01b070 con 0x7f80900d0940 2024-05-08T05:46:12.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.752+0000 7f8086ffd640 1 --2- 172.21.15.45:0/2456451703 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f80740762d0 0x7f8074078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.752+0000 7f8086ffd640 1 -- 172.21.15.45:0/2456451703 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f808c096a70 con 0x7f80900d0940 2024-05-08T05:46:12.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.753+0000 7f8095ce9640 1 --2- 172.21.15.45:0/2456451703 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f80740762d0 0x7f8074078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.753+0000 7f809b46a740 1 Processor -- start 2024-05-08T05:46:12.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.753+0000 7f809b46a740 1 -- start start 2024-05-08T05:46:12.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.753+0000 7f8095ce9640 1 --2- 172.21.15.45:0/2456451703 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f80740762d0 0x7f8074078790 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8080034000 tx=0x7f808002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.754+0000 7f809b46a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c78e8d0 0x55e02c790a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.754+0000 7f809b46a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c7cafa0 0x55e02c78ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.754+0000 7f809b46a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02c7b9390 0x55e02c7b9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.754+0000 7f809b46a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e02c6f2ef0 con 0x55e02c7cafa0 2024-05-08T05:46:12.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.754+0000 7f809b46a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e02c6ac010 con 0x55e02c78e8d0 2024-05-08T05:46:12.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.754+0000 7f809b46a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e02c6eeeb0 con 0x55e02c7b9390 2024-05-08T05:46:12.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.754+0000 7f8095ce9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c7cafa0 0x55e02c78ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.754+0000 7f8095ce9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c7cafa0 0x55e02c78ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39884/0 (socket says 172.21.15.45:39884) 2024-05-08T05:46:12.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.754+0000 7f8095ce9640 1 -- 172.21.15.45:0/578031690 learned_addr learned my addr 172.21.15.45:0/578031690 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:12.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.754+0000 7f80964ea640 1 --2- 172.21.15.45:0/578031690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c78e8d0 0x55e02c790a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.754+0000 7f8096ceb640 1 --2- 172.21.15.45:0/578031690 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02c7b9390 0x55e02c7b9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.754+0000 7f8095ce9640 1 -- 172.21.15.45:0/578031690 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02c7b9390 msgr2=0x55e02c7b9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.754+0000 7f8095ce9640 1 --2- 172.21.15.45:0/578031690 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02c7b9390 0x55e02c7b9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.754+0000 7f8095ce9640 1 -- 172.21.15.45:0/578031690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c78e8d0 msgr2=0x55e02c790a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.754+0000 7f8095ce9640 1 --2- 172.21.15.45:0/578031690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c78e8d0 0x55e02c790a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.754+0000 7f8095ce9640 1 -- 172.21.15.45:0/578031690 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e02c4cca50 con 0x55e02c7cafa0 2024-05-08T05:46:12.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.754+0000 7f80964ea640 1 --2- 172.21.15.45:0/578031690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c78e8d0 0x55e02c790a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:46:12.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.754+0000 7f8095ce9640 1 --2- 172.21.15.45:0/578031690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c7cafa0 0x55e02c78ecb0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f808002a820 tx=0x7f808002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892e87b576e77339 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.755+0000 7f8073fff640 1 -- 172.21.15.45:0/578031690 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080042040 con 0x55e02c7cafa0 2024-05-08T05:46:12.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.755+0000 7f8073fff640 1 -- 172.21.15.45:0/578031690 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8080003fa0 con 0x55e02c7cafa0 2024-05-08T05:46:12.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.755+0000 7f8073fff640 1 -- 172.21.15.45:0/578031690 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080037040 con 0x55e02c7cafa0 2024-05-08T05:46:12.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.755+0000 7f809b46a740 1 -- 172.21.15.45:0/578031690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c7cafa0 msgr2=0x55e02c78ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.755+0000 7f809b46a740 1 --2- 172.21.15.45:0/578031690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c7cafa0 0x55e02c78ecb0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f808002a820 tx=0x7f808002c000 comp rx=0 tx=0).stop 2024-05-08T05:46:12.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.755+0000 7f809b46a740 1 -- 172.21.15.45:0/578031690 shutdown_connections 2024-05-08T05:46:12.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.755+0000 7f809b46a740 1 --2- 172.21.15.45:0/578031690 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02c7b9390 0x55e02c7b9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.755+0000 7f809b46a740 1 --2- 172.21.15.45:0/578031690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c7cafa0 0x55e02c78ecb0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.755+0000 7f809b46a740 1 --2- 172.21.15.45:0/578031690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c78e8d0 0x55e02c790a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.755+0000 7f809b46a740 1 -- 172.21.15.45:0/578031690 >> 172.21.15.45:0/578031690 conn(0x55e02c7563e0 msgr2=0x55e02c7d4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.755+0000 7f809b46a740 1 -- 172.21.15.45:0/578031690 shutdown_connections 2024-05-08T05:46:12.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.755+0000 7f809b46a740 1 -- 172.21.15.45:0/578031690 wait complete. 2024-05-08T05:46:12.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.756+0000 7f809b46a740 1 -- start start 2024-05-08T05:46:12.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.756+0000 7f809b46a740 1 Processor -- start 2024-05-08T05:46:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.757+0000 7f809b46a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c78e8d0 0x55e02c7b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.757+0000 7f809b46a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c7b9390 0x55e02c7a8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.757+0000 7f809b46a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02c7cafa0 0x55e02c7a8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.757+0000 7f809b46a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e02c6afb40 con 0x55e02c7b9390 2024-05-08T05:46:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.757+0000 7f809b46a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e02c7b6ee0 con 0x55e02c78e8d0 2024-05-08T05:46:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.757+0000 7f809b46a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e02c7cbb50 con 0x55e02c7cafa0 2024-05-08T05:46:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.757+0000 7f809b46a740 10 client.0 _ll_register_callbacks cb 0x55e02c69cb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:46:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.757+0000 7f8095ce9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c7b9390 0x55e02c7a8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.757+0000 7f8095ce9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c7b9390 0x55e02c7a8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39890/0 (socket says 172.21.15.45:39890) 2024-05-08T05:46:12.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.757+0000 7f8095ce9640 1 -- 172.21.15.45:0/1204613061 learned_addr learned my addr 172.21.15.45:0/1204613061 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.757+0000 7f8096ceb640 1 --2- 172.21.15.45:0/1204613061 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02c7cafa0 0x55e02c7a8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.757+0000 7f8095ce9640 1 -- 172.21.15.45:0/1204613061 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02c7cafa0 msgr2=0x55e02c7a8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.757+0000 7f8095ce9640 1 --2- 172.21.15.45:0/1204613061 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02c7cafa0 0x55e02c7a8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.757+0000 7f80964ea640 1 --2- 172.21.15.45:0/1204613061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c78e8d0 0x55e02c7b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.757+0000 7f8095ce9640 1 -- 172.21.15.45:0/1204613061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c78e8d0 msgr2=0x55e02c7b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.757+0000 7f8095ce9640 1 --2- 172.21.15.45:0/1204613061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c78e8d0 0x55e02c7b5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.757+0000 7f8095ce9640 1 -- 172.21.15.45:0/1204613061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e02c3c43e0 con 0x55e02c7b9390 2024-05-08T05:46:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.758+0000 7f8095ce9640 1 --2- 172.21.15.45:0/1204613061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c7b9390 0x55e02c7a8820 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f80800098c0 tx=0x7f8080004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544d18ddd204273e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.758+0000 7f80717fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:46:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.758+0000 7f80717fa640 1 -- 172.21.15.45:0/1204613061 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080037040 con 0x55e02c7b9390 2024-05-08T05:46:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.758+0000 7f80717fa640 1 -- 172.21.15.45:0/1204613061 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f808003e720 con 0x55e02c7b9390 2024-05-08T05:46:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.758+0000 7f809b46a740 1 -- 172.21.15.45:0/1204613061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e02c700b00 con 0x55e02c7b9390 2024-05-08T05:46:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.758+0000 7f80717fa640 1 -- 172.21.15.45:0/1204613061 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f8080061050 con 0x55e02c7b9390 2024-05-08T05:46:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.758+0000 7f809b46a740 10 client.6513 fetch_fsmap learned FSMap version 51 2024-05-08T05:46:12.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.758+0000 7f809b46a740 1 -- 172.21.15.45:0/1204613061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55e02c52d1e0 con 0x55e02c7b9390 2024-05-08T05:46:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.758+0000 7f80717fa640 1 -- 172.21.15.45:0/1204613061 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e02c700b00 con 0x55e02c7b9390 2024-05-08T05:46:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.758+0000 7f80717fa640 1 -- 172.21.15.45:0/1204613061 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f8080064020 con 0x55e02c7b9390 2024-05-08T05:46:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.758+0000 7f809b46a740 10 client.6513 fetch_fsmap finished waiting for FSMap version 51 2024-05-08T05:46:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.758+0000 7f809b46a740 10 client.6513 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-08T05:46:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.758+0000 7f809b46a740 20 client.6513 populate_metadata read hostname 'smithi045' 2024-05-08T05:46:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.758+0000 7f809b46a740 1 --2- 172.21.15.45:0/1204613061 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55e02c7cb3e0 0x55e02c7e4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.758+0000 7f809b46a740 4 client.6513 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:46:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.758+0000 7f809b46a740 1 -- 172.21.15.45:0/1204613061 --> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e02c6fa970 con 0x55e02c7cb3e0 2024-05-08T05:46:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.758+0000 7f80964ea640 1 --2- 172.21.15.45:0/1204613061 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55e02c7cb3e0 0x55e02c7e4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.759+0000 7f80717fa640 10 client.6513 ms_handle_connect on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.759+0000 7f80964ea640 1 --2- 172.21.15.45:0/1204613061 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55e02c7cb3e0 0x55e02c7e4710 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e09b834601247cc3 server_cookie=6f94069400295d7a in_seq=0 out_seq=0 2024-05-08T05:46:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.759+0000 7f80717fa640 1 -- 172.21.15.45:0/1204613061 <== mds.0 v2:172.21.15.45:6833/4183668301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e02c6fa970 con 0x55e02c7cb3e0 2024-05-08T05:46:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.759+0000 7f80717fa640 10 client.6513 handle_command_reply: tid=0 2024-05-08T05:46:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.759+0000 7f809b46a740 1 client.6513 shutdown 2024-05-08T05:46:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.759+0000 7f809b46a740 1 -- 172.21.15.45:0/1204613061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c7b9390 msgr2=0x55e02c7a8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.759+0000 7f809b46a740 1 --2- 172.21.15.45:0/1204613061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c7b9390 0x55e02c7a8820 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f80800098c0 tx=0x7f8080004730 comp rx=0 tx=0).stop 2024-05-08T05:46:12.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.759+0000 7f809b46a740 1 -- 172.21.15.45:0/1204613061 shutdown_connections 2024-05-08T05:46:12.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.759+0000 7f809b46a740 1 --2- 172.21.15.45:0/1204613061 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55e02c7cb3e0 0x55e02c7e4710 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.759+0000 7f809b46a740 1 --2- 172.21.15.45:0/1204613061 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02c7cafa0 0x55e02c7a8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.759+0000 7f809b46a740 1 --2- 172.21.15.45:0/1204613061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c7b9390 0x55e02c7a8820 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.759+0000 7f80717fa640 0 client.6513 ms_handle_reset on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:12.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.759+0000 7f809b46a740 1 --2- 172.21.15.45:0/1204613061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c78e8d0 0x55e02c7b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.759+0000 7f809b46a740 1 -- 172.21.15.45:0/1204613061 >> 172.21.15.45:0/1204613061 conn(0x55e02c7563e0 msgr2=0x55e02c7b72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.760+0000 7f809b46a740 1 -- 172.21.15.45:0/1204613061 shutdown_connections 2024-05-08T05:46:12.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.760+0000 7f809b46a740 1 -- 172.21.15.45:0/1204613061 wait complete. 2024-05-08T05:46:12.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.760+0000 7f809b46a740 20 client.6513 trim_cache size 0 max 16384 2024-05-08T05:46:12.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.778+0000 7f809b46a740 1 Processor -- start 2024-05-08T05:46:12.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.778+0000 7f809b46a740 1 -- start start 2024-05-08T05:46:12.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.779+0000 7f809b46a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c78e8d0 0x55e02c7a87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.779+0000 7f809b46a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c7b9390 0x55e02c847fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.779+0000 7f809b46a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02c7cafa0 0x55e02c8484e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.779+0000 7f809b46a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e02c7b6c60 con 0x55e02c78e8d0 2024-05-08T05:46:12.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.779+0000 7f809b46a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8090201c70 con 0x55e02c7b9390 2024-05-08T05:46:12.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.779+0000 7f809b46a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8090200d10 con 0x55e02c7cafa0 2024-05-08T05:46:12.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.779+0000 7f80964ea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c78e8d0 0x55e02c7a87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.779+0000 7f80964ea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c78e8d0 0x55e02c7a87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39894/0 (socket says 172.21.15.45:39894) 2024-05-08T05:46:12.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.779+0000 7f80964ea640 1 -- 172.21.15.45:0/1167411376 learned_addr learned my addr 172.21.15.45:0/1167411376 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:12.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.779+0000 7f80964ea640 1 -- 172.21.15.45:0/1167411376 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02c7cafa0 msgr2=0x55e02c8484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.779+0000 7f80964ea640 1 --2- 172.21.15.45:0/1167411376 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02c7cafa0 0x55e02c8484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.779+0000 7f80964ea640 1 -- 172.21.15.45:0/1167411376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c7b9390 msgr2=0x55e02c847fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.779+0000 7f8095ce9640 1 --2- 172.21.15.45:0/1167411376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c7b9390 0x55e02c847fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.779+0000 7f80964ea640 1 --2- 172.21.15.45:0/1167411376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c7b9390 0x55e02c847fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.779+0000 7f80964ea640 1 -- 172.21.15.45:0/1167411376 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e02c39d4a0 con 0x55e02c78e8d0 2024-05-08T05:46:12.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.779+0000 7f80964ea640 1 --2- 172.21.15.45:0/1167411376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c78e8d0 0x55e02c7a87e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f808c05cec0 tx=0x7f808c063040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3fe9e1932abf9c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.779+0000 7f80737fe640 1 -- 172.21.15.45:0/1167411376 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c068970 con 0x55e02c78e8d0 2024-05-08T05:46:12.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.779+0000 7f80737fe640 1 -- 172.21.15.45:0/1167411376 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f808c096040 con 0x55e02c78e8d0 2024-05-08T05:46:12.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.779+0000 7f80737fe640 1 -- 172.21.15.45:0/1167411376 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c0684e0 con 0x55e02c78e8d0 2024-05-08T05:46:12.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.780+0000 7f809b46a740 1 -- 172.21.15.45:0/1167411376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c78e8d0 msgr2=0x55e02c7a87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.780+0000 7f809b46a740 1 --2- 172.21.15.45:0/1167411376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c78e8d0 0x55e02c7a87e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f808c05cec0 tx=0x7f808c063040 comp rx=0 tx=0).stop 2024-05-08T05:46:12.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.780+0000 7f809b46a740 1 -- 172.21.15.45:0/1167411376 shutdown_connections 2024-05-08T05:46:12.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.780+0000 7f809b46a740 1 --2- 172.21.15.45:0/1167411376 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02c7cafa0 0x55e02c8484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.780+0000 7f809b46a740 1 --2- 172.21.15.45:0/1167411376 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c7b9390 0x55e02c847fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.780+0000 7f809b46a740 1 --2- 172.21.15.45:0/1167411376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c78e8d0 0x55e02c7a87e0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.780+0000 7f809b46a740 1 -- 172.21.15.45:0/1167411376 >> 172.21.15.45:0/1167411376 conn(0x55e02c7563e0 msgr2=0x55e02c784b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.780+0000 7f809b46a740 1 -- 172.21.15.45:0/1167411376 shutdown_connections 2024-05-08T05:46:12.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.780+0000 7f809b46a740 1 -- 172.21.15.45:0/1167411376 wait complete. 2024-05-08T05:46:12.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.781+0000 7f809b46a740 1 -- start start 2024-05-08T05:46:12.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.781+0000 7f809b46a740 1 Processor -- start 2024-05-08T05:46:12.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.781+0000 7f809b46a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c78e8d0 0x55e02c850f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f809b46a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c7b9390 0x55e02c7e6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f809b46a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02c7cafa0 0x55e02c7bcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f809b46a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8090200830 con 0x55e02c78e8d0 2024-05-08T05:46:12.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f809b46a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f80901f8dd0 con 0x55e02c7b9390 2024-05-08T05:46:12.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f809b46a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e02c7bd5d0 con 0x55e02c7cafa0 2024-05-08T05:46:12.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f809b46a740 10 client.0 _ll_register_callbacks cb 0x7f8080035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:46:12.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f80964ea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c78e8d0 0x55e02c850f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f80964ea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c78e8d0 0x55e02c850f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39898/0 (socket says 172.21.15.45:39898) 2024-05-08T05:46:12.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f80964ea640 1 -- 172.21.15.45:0/2927009792 learned_addr learned my addr 172.21.15.45:0/2927009792 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:12.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f8095ce9640 1 --2- 172.21.15.45:0/2927009792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c7b9390 0x55e02c7e6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f8096ceb640 1 --2- 172.21.15.45:0/2927009792 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02c7cafa0 0x55e02c7bcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f80964ea640 1 -- 172.21.15.45:0/2927009792 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02c7cafa0 msgr2=0x55e02c7bcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f80964ea640 1 --2- 172.21.15.45:0/2927009792 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02c7cafa0 0x55e02c7bcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f80964ea640 1 -- 172.21.15.45:0/2927009792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c7b9390 msgr2=0x55e02c7e6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f80964ea640 1 --2- 172.21.15.45:0/2927009792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c7b9390 0x55e02c7e6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f80964ea640 1 -- 172.21.15.45:0/2927009792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e02c7e74a0 con 0x55e02c78e8d0 2024-05-08T05:46:12.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f8095ce9640 1 --2- 172.21.15.45:0/2927009792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c7b9390 0x55e02c7e6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:12.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f80964ea640 1 --2- 172.21.15.45:0/2927009792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c78e8d0 0x55e02c850f30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f808c05f950 tx=0x7f808c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d145a33420a8302e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f80717fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:46:12.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f80717fa640 1 -- 172.21.15.45:0/2927009792 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c068970 con 0x55e02c78e8d0 2024-05-08T05:46:12.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f80717fa640 1 -- 172.21.15.45:0/2927009792 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f808c0684e0 con 0x55e02c78e8d0 2024-05-08T05:46:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f809b46a740 1 -- 172.21.15.45:0/2927009792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f808c096040 con 0x55e02c78e8d0 2024-05-08T05:46:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f80717fa640 1 -- 172.21.15.45:0/2927009792 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f808c068760 con 0x55e02c78e8d0 2024-05-08T05:46:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f809b46a740 10 client.6519 fetch_fsmap learned FSMap version 51 2024-05-08T05:46:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.782+0000 7f809b46a740 1 -- 172.21.15.45:0/2927009792 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55e02c7bd180 con 0x55e02c78e8d0 2024-05-08T05:46:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.783+0000 7f80717fa640 1 -- 172.21.15.45:0/2927009792 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f808c096040 con 0x55e02c78e8d0 2024-05-08T05:46:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.783+0000 7f80717fa640 1 -- 172.21.15.45:0/2927009792 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f808c004bc0 con 0x55e02c78e8d0 2024-05-08T05:46:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.783+0000 7f809b46a740 10 client.6519 fetch_fsmap finished waiting for FSMap version 51 2024-05-08T05:46:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.783+0000 7f809b46a740 10 client.6519 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-08T05:46:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.783+0000 7f809b46a740 20 client.6519 populate_metadata read hostname 'smithi045' 2024-05-08T05:46:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.783+0000 7f809b46a740 1 --2- 172.21.15.45:0/2927009792 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55e02c7d4950 0x55e02c78d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.783+0000 7f809b46a740 4 client.6519 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-05-08T05:46:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.783+0000 7f809b46a740 1 -- 172.21.15.45:0/2927009792 --> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) -- 0x55e02c789980 con 0x55e02c7d4950 2024-05-08T05:46:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.783+0000 7f8095ce9640 1 --2- 172.21.15.45:0/2927009792 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55e02c7d4950 0x55e02c78d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.783+0000 7f80717fa640 10 client.6519 ms_handle_connect on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.783+0000 7f8095ce9640 1 --2- 172.21.15.45:0/2927009792 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55e02c7d4950 0x55e02c78d090 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e360923627ffc7 server_cookie=2cb54d15b2dbbee in_seq=0 out_seq=0 2024-05-08T05:46:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.784+0000 7f80717fa640 1 -- 172.21.15.45:0/2927009792 <== mds.0 v2:172.21.15.45:6833/4183668301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e02c789980 con 0x55e02c7d4950 2024-05-08T05:46:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.784+0000 7f80717fa640 10 client.6519 handle_command_reply: tid=0 2024-05-08T05:46:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.784+0000 7f809b46a740 1 client.6519 shutdown 2024-05-08T05:46:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.784+0000 7f809b46a740 1 -- 172.21.15.45:0/2927009792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c78e8d0 msgr2=0x55e02c850f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.784+0000 7f809b46a740 1 --2- 172.21.15.45:0/2927009792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c78e8d0 0x55e02c850f30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f808c05f950 tx=0x7f808c014040 comp rx=0 tx=0).stop 2024-05-08T05:46:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.784+0000 7f809b46a740 1 -- 172.21.15.45:0/2927009792 shutdown_connections 2024-05-08T05:46:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.784+0000 7f809b46a740 1 --2- 172.21.15.45:0/2927009792 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55e02c7d4950 0x55e02c78d090 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.784+0000 7f809b46a740 1 --2- 172.21.15.45:0/2927009792 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02c7cafa0 0x55e02c7bcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.784+0000 7f809b46a740 1 --2- 172.21.15.45:0/2927009792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02c7b9390 0x55e02c7e6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.784+0000 7f809b46a740 1 --2- 172.21.15.45:0/2927009792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02c78e8d0 0x55e02c850f30 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.784+0000 7f80717fa640 0 client.6519 ms_handle_reset on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.784+0000 7f809b46a740 1 -- 172.21.15.45:0/2927009792 >> 172.21.15.45:0/2927009792 conn(0x55e02c7563e0 msgr2=0x55e02c788f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.784+0000 7f809b46a740 1 -- 172.21.15.45:0/2927009792 shutdown_connections 2024-05-08T05:46:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.784+0000 7f809b46a740 1 -- 172.21.15.45:0/2927009792 wait complete. 2024-05-08T05:46:12.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.784+0000 7f809b46a740 20 client.6519 trim_cache size 0 max 16384 2024-05-08T05:46:12.790 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:46:12.790 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-05-08T05:46:12.790 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "120d9b1e-4606-4311-9b39-c499eab18792", 2024-05-08T05:46:12.790 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-05-08T05:46:12.790 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:46:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.785+0000 7f809874e640 1 -- 172.21.15.45:0/2456451703 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f80740762d0 msgr2=0x7f8074078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.785+0000 7f809874e640 1 --2- 172.21.15.45:0/2456451703 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f80740762d0 0x7f8074078790 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8080034000 tx=0x7f808002e000 comp rx=0 tx=0).stop 2024-05-08T05:46:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.785+0000 7f809874e640 1 -- 172.21.15.45:0/2456451703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80900d0940 msgr2=0x7f80901576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.785+0000 7f809874e640 1 --2- 172.21.15.45:0/2456451703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80900d0940 0x7f80901576d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f808c00c480 tx=0x7f808c00c940 comp rx=0 tx=0).stop 2024-05-08T05:46:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.785+0000 7f809874e640 1 -- 172.21.15.45:0/2456451703 shutdown_connections 2024-05-08T05:46:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.785+0000 7f809874e640 1 --2- 172.21.15.45:0/2456451703 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f80740762d0 0x7f8074078790 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.785+0000 7f809874e640 1 --2- 172.21.15.45:0/2456451703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f80900d3670 0x7f8090154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.785+0000 7f809874e640 1 --2- 172.21.15.45:0/2456451703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f80900d3a50 0x7f80901600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.785+0000 7f809874e640 1 --2- 172.21.15.45:0/2456451703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f80900d0940 0x7f80901576d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.785+0000 7f809874e640 1 -- 172.21.15.45:0/2456451703 >> 172.21.15.45:0/2456451703 conn(0x7f809009e390 msgr2=0x7f80900b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.785+0000 7f809874e640 1 -- 172.21.15.45:0/2456451703 shutdown_connections 2024-05-08T05:46:12.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.785+0000 7f809874e640 1 -- 172.21.15.45:0/2456451703 wait complete. 2024-05-08T05:46:12.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:46:12.795 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-08T05:46:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.903+0000 7f90085f9640 1 Processor -- start 2024-05-08T05:46:12.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.903+0000 7f90085f9640 1 -- start start 2024-05-08T05:46:12.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.904+0000 7f90085f9640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f90000d0ec0 0x7f90000d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.904+0000 7f90085f9640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90000d1760 con 0x7f900015f380 2024-05-08T05:46:12.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.904+0000 7f90085f9640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90000d1a40 con 0x7f90000d3b00 2024-05-08T05:46:12.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.904+0000 7f90085f9640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f90000d30f0 con 0x7f90000d0ec0 2024-05-08T05:46:12.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.904+0000 7f9005b94640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f90000d0ec0 0x7f90000d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.904+0000 7f9006395640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f900015f380 0x7f90000d07b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50018/0 (socket says 172.21.15.45:50018) 2024-05-08T05:46:12.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.904+0000 7f9006395640 1 -- 172.21.15.45:0/2059506689 learned_addr learned my addr 172.21.15.45:0/2059506689 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:12.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.904+0000 7f9005b94640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f90000d0ec0 0x7f90000d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39906/0 (socket says 172.21.15.45:39906) 2024-05-08T05:46:12.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.904+0000 7f9005b94640 1 -- 172.21.15.45:0/2059506689 >> v1:172.21.15.63:6789/0 conn(0x7f90000d3b00 legacy=0x7f90000b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.904+0000 7f9005b94640 1 -- 172.21.15.45:0/2059506689 >> v1:172.21.15.45:6789/0 conn(0x7f900015f380 legacy=0x7f90000d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.904+0000 7f9005393640 1 -- 172.21.15.45:0/2059506689 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4038690055 0 0) 0x7f90000d1760 con 0x7f900015f380 2024-05-08T05:46:12.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.905+0000 7f9005b94640 1 -- 172.21.15.45:0/2059506689 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90000d1d20 con 0x7f90000d0ec0 2024-05-08T05:46:12.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.905+0000 7f9005b94640 1 --2- 172.21.15.45:0/2059506689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90000d0ec0 0x7f90000d35c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00097a0 tx=0x7f8ff002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bbc4e5cc0aaa6e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.905+0000 7f9005393640 1 -- 172.21.15.45:0/2059506689 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0037070 con 0x7f90000d0ec0 2024-05-08T05:46:12.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.905+0000 7f9005393640 1 -- 172.21.15.45:0/2059506689 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff0032440 con 0x7f90000d0ec0 2024-05-08T05:46:12.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.905+0000 7f9005393640 1 -- 172.21.15.45:0/2059506689 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0032810 con 0x7f90000d0ec0 2024-05-08T05:46:12.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.906+0000 7f90085f9640 1 -- 172.21.15.45:0/2059506689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90000d0ec0 msgr2=0x7f90000d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.906+0000 7f90085f9640 1 --2- 172.21.15.45:0/2059506689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90000d0ec0 0x7f90000d35c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00097a0 tx=0x7f8ff002a230 comp rx=0 tx=0).stop 2024-05-08T05:46:12.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.906+0000 7f90085f9640 1 -- 172.21.15.45:0/2059506689 shutdown_connections 2024-05-08T05:46:12.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.906+0000 7f90085f9640 1 --2- 172.21.15.45:0/2059506689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90000d0ec0 0x7f90000d35c0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.906+0000 7f90085f9640 1 -- 172.21.15.45:0/2059506689 >> 172.21.15.45:0/2059506689 conn(0x7f90000d4fe0 msgr2=0x7f90000d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.906+0000 7f90085f9640 1 -- 172.21.15.45:0/2059506689 shutdown_connections 2024-05-08T05:46:12.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.906+0000 7f90085f9640 1 -- 172.21.15.45:0/2059506689 wait complete. 2024-05-08T05:46:12.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.907+0000 7f90085f9640 1 Processor -- start 2024-05-08T05:46:12.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.907+0000 7f90085f9640 1 -- start start 2024-05-08T05:46:12.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.908+0000 7f90085f9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90000d0ec0 0x7f900015a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.908+0000 7f90085f9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90000d3b00 0x7f90000ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.908+0000 7f90085f9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f900015f380 0x7f90000caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.908+0000 7f90085f9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f90000d3f30 con 0x7f900015f380 2024-05-08T05:46:12.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.908+0000 7f90085f9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90000d2370 con 0x7f90000d3b00 2024-05-08T05:46:12.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.908+0000 7f90085f9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f900015f760 con 0x7f90000d0ec0 2024-05-08T05:46:12.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.908+0000 7f9006b96640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f900015f380 0x7f90000caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.908+0000 7f9006b96640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f900015f380 0x7f90000caf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39920/0 (socket says 172.21.15.45:39920) 2024-05-08T05:46:12.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.908+0000 7f9006b96640 1 -- 172.21.15.45:0/3803277643 learned_addr learned my addr 172.21.15.45:0/3803277643 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:12.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.908+0000 7f9005b94640 1 --2- 172.21.15.45:0/3803277643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90000d3b00 0x7f90000ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.908+0000 7f9006395640 1 --2- 172.21.15.45:0/3803277643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90000d0ec0 0x7f900015a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.908+0000 7f9006b96640 1 -- 172.21.15.45:0/3803277643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90000d0ec0 msgr2=0x7f900015a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.908+0000 7f9006b96640 1 --2- 172.21.15.45:0/3803277643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90000d0ec0 0x7f900015a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.908+0000 7f9006b96640 1 -- 172.21.15.45:0/3803277643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90000d3b00 msgr2=0x7f90000ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.908+0000 7f9006b96640 1 --2- 172.21.15.45:0/3803277643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90000d3b00 0x7f90000ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.908+0000 7f9006b96640 1 -- 172.21.15.45:0/3803277643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90000cc150 con 0x7f900015f380 2024-05-08T05:46:12.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.909+0000 7f9006b96640 1 --2- 172.21.15.45:0/3803277643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f900015f380 0x7f90000caf50 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800c7a0 tx=0x7f8ff800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.909+0000 7f8ff6ffd640 1 -- 172.21.15.45:0/3803277643 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8003b70 con 0x7f900015f380 2024-05-08T05:46:12.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.909+0000 7f8ff6ffd640 1 -- 172.21.15.45:0/3803277643 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff8003d10 con 0x7f900015f380 2024-05-08T05:46:12.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.909+0000 7f90085f9640 1 -- 172.21.15.45:0/3803277643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90000cc6d0 con 0x7f900015f380 2024-05-08T05:46:12.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.909+0000 7f8ff6ffd640 1 -- 172.21.15.45:0/3803277643 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff800aa70 con 0x7f900015f380 2024-05-08T05:46:12.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.909+0000 7f90085f9640 1 -- 172.21.15.45:0/3803277643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90000ccc90 con 0x7f900015f380 2024-05-08T05:46:12.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.911+0000 7f8ff6ffd640 1 -- 172.21.15.45:0/3803277643 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8ff800ac10 con 0x7f900015f380 2024-05-08T05:46:12.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.911+0000 7f8ff6ffd640 1 --2- 172.21.15.45:0/3803277643 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8fe40762d0 0x7f8fe4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.911+0000 7f8ff6ffd640 1 -- 172.21.15.45:0/3803277643 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f8ff8097600 con 0x7f900015f380 2024-05-08T05:46:12.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.912+0000 7f9006395640 1 --2- 172.21.15.45:0/3803277643 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8fe40762d0 0x7f8fe4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.912+0000 7f9006395640 1 --2- 172.21.15.45:0/3803277643 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8fe40762d0 0x7f8fe4078790 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc000f60 tx=0x7f8ffc007e40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.912+0000 7f900b3ea740 1 Processor -- start 2024-05-08T05:46:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.912+0000 7f900b3ea740 1 -- start start 2024-05-08T05:46:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.913+0000 7f900b3ea740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a20ebb160 0x564a20e6daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.913+0000 7f900b3ea740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20e9e280 0x564a20e9d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.913+0000 7f900b3ea740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564a20e9db40 0x564a20e72950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.913+0000 7f900b3ea740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564a20e3a690 con 0x564a20e9e280 2024-05-08T05:46:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.913+0000 7f900b3ea740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564a20dd7f30 con 0x564a20ebb160 2024-05-08T05:46:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.913+0000 7f900b3ea740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564a20d90b20 con 0x564a20e9db40 2024-05-08T05:46:12.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.913+0000 7f9005b94640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20e9e280 0x564a20e9d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.913+0000 7f9005b94640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20e9e280 0x564a20e9d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39926/0 (socket says 172.21.15.45:39926) 2024-05-08T05:46:12.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.913+0000 7f9005b94640 1 -- 172.21.15.45:0/1218570692 learned_addr learned my addr 172.21.15.45:0/1218570692 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:12.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.913+0000 7f9006b96640 1 --2- 172.21.15.45:0/1218570692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564a20e9db40 0x564a20e72950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.913+0000 7f9006395640 1 --2- 172.21.15.45:0/1218570692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a20ebb160 0x564a20e6daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.914+0000 7f9005b94640 1 -- 172.21.15.45:0/1218570692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564a20e9db40 msgr2=0x564a20e72950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.914+0000 7f9005b94640 1 --2- 172.21.15.45:0/1218570692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564a20e9db40 0x564a20e72950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.914+0000 7f9005b94640 1 -- 172.21.15.45:0/1218570692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a20ebb160 msgr2=0x564a20e6daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.914+0000 7f9005b94640 1 --2- 172.21.15.45:0/1218570692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a20ebb160 0x564a20e6daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.914+0000 7f9005b94640 1 -- 172.21.15.45:0/1218570692 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a20d88e60 con 0x564a20e9e280 2024-05-08T05:46:12.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.914+0000 7f9005b94640 1 --2- 172.21.15.45:0/1218570692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20e9e280 0x564a20e9d600 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00098c0 tx=0x7f8ff002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62cbdd7461420d1d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.914+0000 7f8fe3fff640 1 -- 172.21.15.45:0/1218570692 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0043070 con 0x564a20e9e280 2024-05-08T05:46:12.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.914+0000 7f8fe3fff640 1 -- 172.21.15.45:0/1218570692 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff0032920 con 0x564a20e9e280 2024-05-08T05:46:12.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.914+0000 7f8fe3fff640 1 -- 172.21.15.45:0/1218570692 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0037040 con 0x564a20e9e280 2024-05-08T05:46:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.915+0000 7f900b3ea740 1 -- 172.21.15.45:0/1218570692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20e9e280 msgr2=0x564a20e9d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.915+0000 7f900b3ea740 1 --2- 172.21.15.45:0/1218570692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20e9e280 0x564a20e9d600 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00098c0 tx=0x7f8ff002c000 comp rx=0 tx=0).stop 2024-05-08T05:46:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.915+0000 7f900b3ea740 1 -- 172.21.15.45:0/1218570692 shutdown_connections 2024-05-08T05:46:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.915+0000 7f900b3ea740 1 --2- 172.21.15.45:0/1218570692 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564a20e9db40 0x564a20e72950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.915+0000 7f900b3ea740 1 --2- 172.21.15.45:0/1218570692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20e9e280 0x564a20e9d600 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.915+0000 7f900b3ea740 1 --2- 172.21.15.45:0/1218570692 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a20ebb160 0x564a20e6daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.915+0000 7f900b3ea740 1 -- 172.21.15.45:0/1218570692 >> 172.21.15.45:0/1218570692 conn(0x564a20e6fd60 msgr2=0x564a20eb8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.915+0000 7f900b3ea740 1 -- 172.21.15.45:0/1218570692 shutdown_connections 2024-05-08T05:46:12.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.915+0000 7f900b3ea740 1 -- 172.21.15.45:0/1218570692 wait complete. 2024-05-08T05:46:12.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.915+0000 7f900b3ea740 1 -- start start 2024-05-08T05:46:12.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.915+0000 7f900b3ea740 1 Processor -- start 2024-05-08T05:46:12.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.916+0000 7f900b3ea740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564a20e9e280 0x564a20e9d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.916+0000 7f900b3ea740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ebb160 0x564a20ea03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.916+0000 7f900b3ea740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a20ea0910 0x564a20e9a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.916+0000 7f900b3ea740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564a20d945e0 con 0x564a20ebb160 2024-05-08T05:46:12.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.916+0000 7f900b3ea740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564a20dd3ef0 con 0x564a20ea0910 2024-05-08T05:46:12.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.916+0000 7f900b3ea740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564a20e71f10 con 0x564a20e9e280 2024-05-08T05:46:12.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.916+0000 7f900b3ea740 10 client.0 _ll_register_callbacks cb 0x564a20d8f320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:46:12.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.916+0000 7f9005b94640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ebb160 0x564a20ea03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.916+0000 7f9005b94640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ebb160 0x564a20ea03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39932/0 (socket says 172.21.15.45:39932) 2024-05-08T05:46:12.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.916+0000 7f9005b94640 1 -- 172.21.15.45:0/25780043 learned_addr learned my addr 172.21.15.45:0/25780043 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:12.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.916+0000 7f9006395640 1 --2- 172.21.15.45:0/25780043 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564a20e9e280 0x564a20e9d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.917+0000 7f9006b96640 1 --2- 172.21.15.45:0/25780043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a20ea0910 0x564a20e9a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.917+0000 7f9005b94640 1 -- 172.21.15.45:0/25780043 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564a20e9e280 msgr2=0x564a20e9d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.917+0000 7f9005b94640 1 --2- 172.21.15.45:0/25780043 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564a20e9e280 0x564a20e9d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.917+0000 7f9005b94640 1 -- 172.21.15.45:0/25780043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a20ea0910 msgr2=0x564a20e9a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.917+0000 7f9005b94640 1 --2- 172.21.15.45:0/25780043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a20ea0910 0x564a20e9a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.917+0000 7f9005b94640 1 -- 172.21.15.45:0/25780043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a20bb37d0 con 0x564a20ebb160 2024-05-08T05:46:12.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.917+0000 7f9005b94640 1 --2- 172.21.15.45:0/25780043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ebb160 0x564a20ea03d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f8ff002a820 tx=0x7f8ff0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e21274cacb32a69 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.917+0000 7f8fe17fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:46:12.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.917+0000 7f8fe17fa640 1 -- 172.21.15.45:0/25780043 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff004c020 con 0x564a20ebb160 2024-05-08T05:46:12.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.917+0000 7f8fe17fa640 1 -- 172.21.15.45:0/25780043 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f8ff0004230 con 0x564a20ebb160 2024-05-08T05:46:12.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.917+0000 7f900b3ea740 1 -- 172.21.15.45:0/25780043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ff0032920 con 0x564a20ebb160 2024-05-08T05:46:12.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.917+0000 7f8fe17fa640 1 -- 172.21.15.45:0/25780043 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f8ff0037040 con 0x564a20ebb160 2024-05-08T05:46:12.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.917+0000 7f900b3ea740 10 client.6534 fetch_fsmap learned FSMap version 51 2024-05-08T05:46:12.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.917+0000 7f900b3ea740 1 -- 172.21.15.45:0/25780043 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x564a20ae93f0 con 0x564a20ebb160 2024-05-08T05:46:12.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.917+0000 7f8fe17fa640 1 -- 172.21.15.45:0/25780043 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff0032920 con 0x564a20ebb160 2024-05-08T05:46:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.918+0000 7f8fe17fa640 1 -- 172.21.15.45:0/25780043 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f8ff0056080 con 0x564a20ebb160 2024-05-08T05:46:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.918+0000 7f900b3ea740 10 client.6534 fetch_fsmap finished waiting for FSMap version 51 2024-05-08T05:46:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.918+0000 7f900b3ea740 10 client.6534 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-08T05:46:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.918+0000 7f900b3ea740 20 client.6534 populate_metadata read hostname 'smithi045' 2024-05-08T05:46:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.918+0000 7f900b3ea740 1 --2- 172.21.15.45:0/25780043 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x564a20e9e770 0x564a20e9eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.918+0000 7f900b3ea740 4 client.6534 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:46:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.918+0000 7f900b3ea740 1 -- 172.21.15.45:0/25780043 --> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a20de5b40 con 0x564a20e9e770 2024-05-08T05:46:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.918+0000 7f9006395640 1 --2- 172.21.15.45:0/25780043 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x564a20e9e770 0x564a20e9eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.918+0000 7f8fe17fa640 10 client.6534 ms_handle_connect on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.918+0000 7f9006395640 1 --2- 172.21.15.45:0/25780043 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x564a20e9e770 0x564a20e9eb50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=185c87be64361335 server_cookie=ef36a9a49e9ffe8f in_seq=0 out_seq=0 2024-05-08T05:46:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.919+0000 7f8fe17fa640 1 -- 172.21.15.45:0/25780043 <== mds.0 v2:172.21.15.45:6833/4183668301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564a20de5b40 con 0x564a20e9e770 2024-05-08T05:46:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.919+0000 7f8fe17fa640 10 client.6534 handle_command_reply: tid=0 2024-05-08T05:46:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.919+0000 7f900b3ea740 1 client.6534 shutdown 2024-05-08T05:46:12.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.919+0000 7f900b3ea740 1 -- 172.21.15.45:0/25780043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ebb160 msgr2=0x564a20ea03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.919+0000 7f900b3ea740 1 --2- 172.21.15.45:0/25780043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ebb160 0x564a20ea03d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f8ff002a820 tx=0x7f8ff0004730 comp rx=0 tx=0).stop 2024-05-08T05:46:12.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.919+0000 7f900b3ea740 1 -- 172.21.15.45:0/25780043 shutdown_connections 2024-05-08T05:46:12.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.919+0000 7f900b3ea740 1 --2- 172.21.15.45:0/25780043 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x564a20e9e770 0x564a20e9eb50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.919+0000 7f900b3ea740 1 --2- 172.21.15.45:0/25780043 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a20ea0910 0x564a20e9a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.919+0000 7f900b3ea740 1 --2- 172.21.15.45:0/25780043 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ebb160 0x564a20ea03d0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.919+0000 7f8fe17fa640 0 client.6534 ms_handle_reset on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:12.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.919+0000 7f900b3ea740 1 --2- 172.21.15.45:0/25780043 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564a20e9e280 0x564a20e9d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.919+0000 7f900b3ea740 1 -- 172.21.15.45:0/25780043 >> 172.21.15.45:0/25780043 conn(0x564a20e9db40 msgr2=0x564a20e6d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.919+0000 7f900b3ea740 1 -- 172.21.15.45:0/25780043 shutdown_connections 2024-05-08T05:46:12.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.919+0000 7f900b3ea740 1 -- 172.21.15.45:0/25780043 wait complete. 2024-05-08T05:46:12.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.919+0000 7f900b3ea740 20 client.6534 trim_cache size 0 max 16384 2024-05-08T05:46:12.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.937+0000 7f900b3ea740 1 Processor -- start 2024-05-08T05:46:12.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.937+0000 7f900b3ea740 1 -- start start 2024-05-08T05:46:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.938+0000 7f900b3ea740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564a20e9e280 0x564a20e9f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.938+0000 7f900b3ea740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a20ea0910 0x564a20ea01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.938+0000 7f900b3ea740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ebb160 0x564a20ecad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.938+0000 7f900b3ea740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9000201ea0 con 0x564a20ebb160 2024-05-08T05:46:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.938+0000 7f900b3ea740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9000200f70 con 0x564a20ea0910 2024-05-08T05:46:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.938+0000 7f900b3ea740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f90001f98b0 con 0x564a20e9e280 2024-05-08T05:46:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.938+0000 7f9006b96640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ebb160 0x564a20ecad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.938+0000 7f9006b96640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ebb160 0x564a20ecad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39944/0 (socket says 172.21.15.45:39944) 2024-05-08T05:46:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.938+0000 7f9006b96640 1 -- 172.21.15.45:0/1973945937 learned_addr learned my addr 172.21.15.45:0/1973945937 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.938+0000 7f9006b96640 1 -- 172.21.15.45:0/1973945937 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564a20e9e280 msgr2=0x564a20e9f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.938+0000 7f9006b96640 1 --2- 172.21.15.45:0/1973945937 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564a20e9e280 0x564a20e9f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.938+0000 7f9006b96640 1 -- 172.21.15.45:0/1973945937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a20ea0910 msgr2=0x564a20ea01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.938+0000 7f9006b96640 1 --2- 172.21.15.45:0/1973945937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a20ea0910 0x564a20ea01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.938+0000 7f9006b96640 1 -- 172.21.15.45:0/1973945937 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a20aa90f0 con 0x564a20ebb160 2024-05-08T05:46:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.938+0000 7f9006b96640 1 --2- 172.21.15.45:0/1973945937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ebb160 0x564a20ecad40 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800a430 tx=0x7f8ff8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50323795e0f72c38 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.938+0000 7f8fe37fe640 1 -- 172.21.15.45:0/1973945937 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8060230 con 0x564a20ebb160 2024-05-08T05:46:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.939+0000 7f8fe37fe640 1 -- 172.21.15.45:0/1973945937 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff8060720 con 0x564a20ebb160 2024-05-08T05:46:12.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.939+0000 7f8fe37fe640 1 -- 172.21.15.45:0/1973945937 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8065dd0 con 0x564a20ebb160 2024-05-08T05:46:12.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.939+0000 7f900b3ea740 1 -- 172.21.15.45:0/1973945937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ebb160 msgr2=0x564a20ecad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.939+0000 7f900b3ea740 1 --2- 172.21.15.45:0/1973945937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ebb160 0x564a20ecad40 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800a430 tx=0x7f8ff8005040 comp rx=0 tx=0).stop 2024-05-08T05:46:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.939+0000 7f900b3ea740 1 -- 172.21.15.45:0/1973945937 shutdown_connections 2024-05-08T05:46:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.939+0000 7f900b3ea740 1 --2- 172.21.15.45:0/1973945937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ebb160 0x564a20ecad40 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.939+0000 7f900b3ea740 1 --2- 172.21.15.45:0/1973945937 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a20ea0910 0x564a20ea01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.939+0000 7f900b3ea740 1 --2- 172.21.15.45:0/1973945937 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564a20e9e280 0x564a20e9f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.939+0000 7f900b3ea740 1 -- 172.21.15.45:0/1973945937 >> 172.21.15.45:0/1973945937 conn(0x564a20e9db40 msgr2=0x564a20e9ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.939+0000 7f900b3ea740 1 -- 172.21.15.45:0/1973945937 shutdown_connections 2024-05-08T05:46:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.939+0000 7f900b3ea740 1 -- 172.21.15.45:0/1973945937 wait complete. 2024-05-08T05:46:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.940+0000 7f900b3ea740 1 -- start start 2024-05-08T05:46:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.940+0000 7f900b3ea740 1 Processor -- start 2024-05-08T05:46:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f900b3ea740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564a20e9e280 0x564a20f2fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f900b3ea740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ea0910 0x564a20e99ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f900b3ea740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a20ebb160 0x564a20e9a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f900b3ea740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564a20e8daa0 con 0x564a20ea0910 2024-05-08T05:46:12.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f900b3ea740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90001f6240 con 0x564a20ebb160 2024-05-08T05:46:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f900b3ea740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564a20ea0730 con 0x564a20e9e280 2024-05-08T05:46:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f900b3ea740 10 client.0 _ll_register_callbacks cb 0x7f8ffc007400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:46:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f9005b94640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ea0910 0x564a20e99ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f9005b94640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ea0910 0x564a20e99ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39948/0 (socket says 172.21.15.45:39948) 2024-05-08T05:46:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f9005b94640 1 -- 172.21.15.45:0/2456693917 learned_addr learned my addr 172.21.15.45:0/2456693917 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f9006395640 1 --2- 172.21.15.45:0/2456693917 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564a20e9e280 0x564a20f2fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f9006b96640 1 --2- 172.21.15.45:0/2456693917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a20ebb160 0x564a20e9a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f9005b94640 1 -- 172.21.15.45:0/2456693917 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564a20e9e280 msgr2=0x564a20f2fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f9005b94640 1 --2- 172.21.15.45:0/2456693917 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564a20e9e280 0x564a20f2fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f9005b94640 1 -- 172.21.15.45:0/2456693917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a20ebb160 msgr2=0x564a20e9a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f9005b94640 1 --2- 172.21.15.45:0/2456693917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a20ebb160 0x564a20e9a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f9005b94640 1 -- 172.21.15.45:0/2456693917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a20e6ced0 con 0x564a20ea0910 2024-05-08T05:46:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f9006395640 1 --2- 172.21.15.45:0/2456693917 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564a20e9e280 0x564a20f2fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:46:12.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f9005b94640 1 --2- 172.21.15.45:0/2456693917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ea0910 0x564a20e99ad0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f8ff002a270 tx=0x7f8ff00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d87c6e9a43dbcb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f8fe17fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:46:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f8fe17fa640 1 -- 172.21.15.45:0/2456693917 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0059060 con 0x564a20ea0910 2024-05-08T05:46:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f8fe17fa640 1 -- 172.21.15.45:0/2456693917 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f8ff0048450 con 0x564a20ea0910 2024-05-08T05:46:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.941+0000 7f900b3ea740 1 -- 172.21.15.45:0/2456693917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ff8060720 con 0x564a20ea0910 2024-05-08T05:46:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.942+0000 7f8fe17fa640 1 -- 172.21.15.45:0/2456693917 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f8ff005f050 con 0x564a20ea0910 2024-05-08T05:46:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.942+0000 7f900b3ea740 10 client.6540 fetch_fsmap learned FSMap version 51 2024-05-08T05:46:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.942+0000 7f900b3ea740 1 -- 172.21.15.45:0/2456693917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x564a20f36eb0 con 0x564a20ea0910 2024-05-08T05:46:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.942+0000 7f8fe17fa640 1 -- 172.21.15.45:0/2456693917 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff8060720 con 0x564a20ea0910 2024-05-08T05:46:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.942+0000 7f8fe17fa640 1 -- 172.21.15.45:0/2456693917 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f8ff0062020 con 0x564a20ea0910 2024-05-08T05:46:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.942+0000 7f900b3ea740 10 client.6540 fetch_fsmap finished waiting for FSMap version 51 2024-05-08T05:46:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.942+0000 7f900b3ea740 10 client.6540 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-08T05:46:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.942+0000 7f900b3ea740 20 client.6540 populate_metadata read hostname 'smithi045' 2024-05-08T05:46:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.942+0000 7f900b3ea740 1 --2- 172.21.15.45:0/2456693917 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x564a20e8e100 0x564a20f35db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.942+0000 7f900b3ea740 4 client.6540 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:46:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.942+0000 7f900b3ea740 1 -- 172.21.15.45:0/2456693917 --> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564a20ddf9b0 con 0x564a20e8e100 2024-05-08T05:46:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.942+0000 7f9006395640 1 --2- 172.21.15.45:0/2456693917 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x564a20e8e100 0x564a20f35db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.942+0000 7f8fe17fa640 10 client.6540 ms_handle_connect on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.942+0000 7f9006395640 1 --2- 172.21.15.45:0/2456693917 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x564a20e8e100 0x564a20f35db0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d94bfe38b323870 server_cookie=fc9a4b80f940ab88 in_seq=0 out_seq=0 2024-05-08T05:46:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.943+0000 7f8fe17fa640 1 -- 172.21.15.45:0/2456693917 <== mds.0 v2:172.21.15.45:6833/4183668301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x564a20ddf9b0 con 0x564a20e8e100 2024-05-08T05:46:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.943+0000 7f8fe17fa640 10 client.6540 handle_command_reply: tid=0 2024-05-08T05:46:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.943+0000 7f900b3ea740 1 client.6540 shutdown 2024-05-08T05:46:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.943+0000 7f900b3ea740 1 -- 172.21.15.45:0/2456693917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ea0910 msgr2=0x564a20e99ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.943+0000 7f900b3ea740 1 --2- 172.21.15.45:0/2456693917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ea0910 0x564a20e99ad0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f8ff002a270 tx=0x7f8ff00046c0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.943+0000 7f900b3ea740 1 -- 172.21.15.45:0/2456693917 shutdown_connections 2024-05-08T05:46:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.943+0000 7f900b3ea740 1 --2- 172.21.15.45:0/2456693917 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x564a20e8e100 0x564a20f35db0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.943+0000 7f900b3ea740 1 --2- 172.21.15.45:0/2456693917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564a20ebb160 0x564a20e9a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.943+0000 7f900b3ea740 1 --2- 172.21.15.45:0/2456693917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564a20ea0910 0x564a20e99ad0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.943+0000 7f8fe17fa640 0 client.6540 ms_handle_reset on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.943+0000 7f900b3ea740 1 --2- 172.21.15.45:0/2456693917 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564a20e9e280 0x564a20f2fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.943+0000 7f900b3ea740 1 -- 172.21.15.45:0/2456693917 >> 172.21.15.45:0/2456693917 conn(0x564a20e9db40 msgr2=0x564a20ecad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.943+0000 7f900b3ea740 1 -- 172.21.15.45:0/2456693917 shutdown_connections 2024-05-08T05:46:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.943+0000 7f900b3ea740 1 -- 172.21.15.45:0/2456693917 wait complete. 2024-05-08T05:46:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.943+0000 7f900b3ea740 20 client.6540 trim_cache size 0 max 16384 2024-05-08T05:46:12.949 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:46:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T05:46:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-05-08T05:46:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "120d9b1e-4606-4311-9b39-c499eab18792": { 2024-05-08T05:46:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/subdir", 2024-05-08T05:46:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "120d9b1e-4606-4311-9b39-c499eab18792", 2024-05-08T05:46:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,repair" 2024-05-08T05:46:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:12.950 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:12.950 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:46:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.943+0000 7f90085f9640 1 -- 172.21.15.45:0/3803277643 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8fe40762d0 msgr2=0x7f8fe4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.943+0000 7f90085f9640 1 --2- 172.21.15.45:0/3803277643 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8fe40762d0 0x7f8fe4078790 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc000f60 tx=0x7f8ffc007e40 comp rx=0 tx=0).stop 2024-05-08T05:46:12.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.943+0000 7f90085f9640 1 -- 172.21.15.45:0/3803277643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f900015f380 msgr2=0x7f90000caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:12.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.943+0000 7f90085f9640 1 --2- 172.21.15.45:0/3803277643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f900015f380 0x7f90000caf50 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800c7a0 tx=0x7f8ff800cc60 comp rx=0 tx=0).stop 2024-05-08T05:46:12.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.944+0000 7f90085f9640 1 -- 172.21.15.45:0/3803277643 shutdown_connections 2024-05-08T05:46:12.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.944+0000 7f90085f9640 1 --2- 172.21.15.45:0/3803277643 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8fe40762d0 0x7f8fe4078790 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.944+0000 7f90085f9640 1 --2- 172.21.15.45:0/3803277643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f900015f380 0x7f90000caf50 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.944+0000 7f90085f9640 1 --2- 172.21.15.45:0/3803277643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90000d3b00 0x7f90000ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.944+0000 7f90085f9640 1 --2- 172.21.15.45:0/3803277643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90000d0ec0 0x7f900015a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:12.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.944+0000 7f90085f9640 1 -- 172.21.15.45:0/3803277643 >> 172.21.15.45:0/3803277643 conn(0x7f90000d4fe0 msgr2=0x7f90000b9610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:12.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.944+0000 7f90085f9640 1 -- 172.21.15.45:0/3803277643 shutdown_connections 2024-05-08T05:46:12.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:12.944+0000 7f90085f9640 1 -- 172.21.15.45:0/3803277643 wait complete. 2024-05-08T05:46:12.954 INFO:tasks.cephfs.filesystem:scrub status for tag:120d9b1e-4606-4311-9b39-c499eab18792 - {'path': '/subdir', 'tag': '120d9b1e-4606-4311-9b39-c499eab18792', 'options': 'recursive,repair'} 2024-05-08T05:46:34.669 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:46:34.673 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:46:42.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:46:42.956 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-05-08T05:46:43.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.075+0000 7f525dbb6640 1 Processor -- start 2024-05-08T05:46:43.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.075+0000 7f525dbb6640 1 -- start start 2024-05-08T05:46:43.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.076+0000 7f525dbb6640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5258163920 0x7f525806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.076+0000 7f525dbb6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5258163d30 con 0x7f52580d20c0 2024-05-08T05:46:43.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.076+0000 7f525dbb6640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52580d1300 con 0x7f5258070d30 2024-05-08T05:46:43.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.076+0000 7f525dbb6640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f52580d10d0 con 0x7f5258163920 2024-05-08T05:46:43.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.076+0000 7f5256ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f52580d20c0 0x7f5258071110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47558/0 (socket says 172.21.15.45:47558) 2024-05-08T05:46:43.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.076+0000 7f5256ffd640 1 -- 172.21.15.45:0/4002188213 learned_addr learned my addr 172.21.15.45:0/4002188213 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:43.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.076+0000 7f5257fff640 1 --2- 172.21.15.45:0/4002188213 >> v2:172.21.15.63:3300/0 conn(0x7f5258163920 0x7f525806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.076+0000 7f52567fc640 1 -- 172.21.15.45:0/4002188213 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4079859951 0 0) 0x7f5258163d30 con 0x7f52580d20c0 2024-05-08T05:46:43.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.076+0000 7f52567fc640 1 -- 172.21.15.45:0/4002188213 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5240003520 con 0x7f52580d20c0 2024-05-08T05:46:43.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52567fc640 1 -- 172.21.15.45:0/4002188213 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2230317547 0 0) 0x7f52580d1300 con 0x7f5258070d30 2024-05-08T05:46:43.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52567fc640 1 -- 172.21.15.45:0/4002188213 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5258163d30 con 0x7f5258070d30 2024-05-08T05:46:43.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52567fc640 1 -- 172.21.15.45:0/4002188213 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3582991753 0 0) 0x7f5240003520 con 0x7f52580d20c0 2024-05-08T05:46:43.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52567fc640 1 -- 172.21.15.45:0/4002188213 >> v2:172.21.15.63:3300/0 conn(0x7f5258163920 msgr2=0x7f525806d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52567fc640 1 --2- 172.21.15.45:0/4002188213 >> v2:172.21.15.63:3300/0 conn(0x7f5258163920 0x7f525806d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52567fc640 1 -- 172.21.15.45:0/4002188213 >> v1:172.21.15.63:6789/0 conn(0x7f5258070d30 legacy=0x7f52580703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52567fc640 1 -- 172.21.15.45:0/4002188213 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52580d15e0 con 0x7f52580d20c0 2024-05-08T05:46:43.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52567fc640 1 -- 172.21.15.45:0/4002188213 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f524c0016f0 con 0x7f52580d20c0 2024-05-08T05:46:43.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52567fc640 1 -- 172.21.15.45:0/4002188213 >> v1:172.21.15.45:6789/0 conn(0x7f52580d20c0 legacy=0x7f5258071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52567fc640 1 --2- 172.21.15.45:0/4002188213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5240003c30 0x7f5240004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52567fc640 1 --2- 172.21.15.45:0/4002188213 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5240004800 0x7f5240004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52567fc640 1 --2- 172.21.15.45:0/4002188213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5240005210 0x7f524000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52567fc640 1 -- 172.21.15.45:0/4002188213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f524000c110 con 0x7f5240003c30 2024-05-08T05:46:43.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52567fc640 1 -- 172.21.15.45:0/4002188213 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f524000c2c0 con 0x7f5240005210 2024-05-08T05:46:43.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52567fc640 1 -- 172.21.15.45:0/4002188213 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f524000c4c0 con 0x7f5240004800 2024-05-08T05:46:43.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52577fe640 1 --2- 172.21.15.45:0/4002188213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5240003c30 0x7f5240004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f5256ffd640 1 --2- 172.21.15.45:0/4002188213 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5240004800 0x7f5240004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f5257fff640 1 --2- 172.21.15.45:0/4002188213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5240005210 0x7f524000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52577fe640 1 -- 172.21.15.45:0/4002188213 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5240004800 msgr2=0x7f5240004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52577fe640 1 --2- 172.21.15.45:0/4002188213 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5240004800 0x7f5240004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52577fe640 1 -- 172.21.15.45:0/4002188213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5240005210 msgr2=0x7f524000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52577fe640 1 --2- 172.21.15.45:0/4002188213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5240005210 0x7f524000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f52577fe640 1 -- 172.21.15.45:0/4002188213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5240003520 con 0x7f5240003c30 2024-05-08T05:46:43.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f5257fff640 1 --2- 172.21.15.45:0/4002188213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5240005210 0x7f524000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:43.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.077+0000 7f5257fff640 1 -- 172.21.15.45:0/4002188213 reap_dead start 2024-05-08T05:46:43.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.078+0000 7f52577fe640 1 --2- 172.21.15.45:0/4002188213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5240003c30 0x7f5240004030 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5248001900 tx=0x7f5240005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df391bf2bfd2e2b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:43.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.078+0000 7f52567fc640 1 -- 172.21.15.45:0/4002188213 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5248001520 con 0x7f5240003c30 2024-05-08T05:46:43.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.078+0000 7f52567fc640 1 -- 172.21.15.45:0/4002188213 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52480052f0 con 0x7f5240003c30 2024-05-08T05:46:43.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.078+0000 7f52567fc640 1 -- 172.21.15.45:0/4002188213 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524802acc0 con 0x7f5240003c30 2024-05-08T05:46:43.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.079+0000 7f525dbb6640 1 -- 172.21.15.45:0/4002188213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5240003c30 msgr2=0x7f5240004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.079+0000 7f525dbb6640 1 --2- 172.21.15.45:0/4002188213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5240003c30 0x7f5240004030 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5248001900 tx=0x7f5240005830 comp rx=0 tx=0).stop 2024-05-08T05:46:43.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.079+0000 7f525dbb6640 1 -- 172.21.15.45:0/4002188213 shutdown_connections 2024-05-08T05:46:43.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.079+0000 7f525dbb6640 1 --2- 172.21.15.45:0/4002188213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5240003c30 0x7f5240004030 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.079+0000 7f525dbb6640 1 -- 172.21.15.45:0/4002188213 >> 172.21.15.45:0/4002188213 conn(0x7f52580d2e90 msgr2=0x7f52580d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:43.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.079+0000 7f525dbb6640 1 -- 172.21.15.45:0/4002188213 shutdown_connections 2024-05-08T05:46:43.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.079+0000 7f525dbb6640 1 -- 172.21.15.45:0/4002188213 wait complete. 2024-05-08T05:46:43.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.080+0000 7f525dbb6640 1 Processor -- start 2024-05-08T05:46:43.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.080+0000 7f525dbb6640 1 -- start start 2024-05-08T05:46:43.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.081+0000 7f525dbb6640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5240003c30 0x7f5258157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.081+0000 7f525dbb6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52581f57e0 0x7f52580c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.081+0000 7f525dbb6640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52580c7910 0x7f52580c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.081+0000 7f525dbb6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f52580d1c30 con 0x7f52581f57e0 2024-05-08T05:46:43.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.081+0000 7f525dbb6640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5258070b80 con 0x7f52580c7910 2024-05-08T05:46:43.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.081+0000 7f525dbb6640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f52580703a0 con 0x7f5240003c30 2024-05-08T05:46:43.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.081+0000 7f5256ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52581f57e0 0x7f52580c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.081+0000 7f5256ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52581f57e0 0x7f52580c73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36504/0 (socket says 172.21.15.45:36504) 2024-05-08T05:46:43.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.081+0000 7f5256ffd640 1 -- 172.21.15.45:0/1793040270 learned_addr learned my addr 172.21.15.45:0/1793040270 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:43.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.081+0000 7f52577fe640 1 --2- 172.21.15.45:0/1793040270 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5240003c30 0x7f5258157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.081+0000 7f5257fff640 1 --2- 172.21.15.45:0/1793040270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52580c7910 0x7f52580c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.082+0000 7f5256ffd640 1 -- 172.21.15.45:0/1793040270 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5240003c30 msgr2=0x7f5258157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.082+0000 7f5256ffd640 1 --2- 172.21.15.45:0/1793040270 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5240003c30 0x7f5258157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.082+0000 7f5256ffd640 1 -- 172.21.15.45:0/1793040270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52580c7910 msgr2=0x7f52580c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.082+0000 7f5256ffd640 1 --2- 172.21.15.45:0/1793040270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52580c7910 0x7f52580c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.082+0000 7f5256ffd640 1 -- 172.21.15.45:0/1793040270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52580c8130 con 0x7f52581f57e0 2024-05-08T05:46:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.082+0000 7f5257fff640 1 --2- 172.21.15.45:0/1793040270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52580c7910 0x7f52580c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:46:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.082+0000 7f5256ffd640 1 --2- 172.21.15.45:0/1793040270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52581f57e0 0x7f52580c73d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f524c0072f0 tx=0x7f524c0077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.082+0000 7f523ffff640 1 -- 172.21.15.45:0/1793040270 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c00f9c0 con 0x7f52581f57e0 2024-05-08T05:46:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.082+0000 7f523ffff640 1 -- 172.21.15.45:0/1793040270 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f524c018070 con 0x7f52581f57e0 2024-05-08T05:46:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.082+0000 7f525dbb6640 1 -- 172.21.15.45:0/1793040270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52580c7e50 con 0x7f52581f57e0 2024-05-08T05:46:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.082+0000 7f523ffff640 1 -- 172.21.15.45:0/1793040270 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c00fdc0 con 0x7f52581f57e0 2024-05-08T05:46:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.082+0000 7f525dbb6640 1 -- 172.21.15.45:0/1793040270 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52580b2e50 con 0x7f52581f57e0 2024-05-08T05:46:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.084+0000 7f523ffff640 1 -- 172.21.15.45:0/1793040270 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f524c01d460 con 0x7f52581f57e0 2024-05-08T05:46:43.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.084+0000 7f523ffff640 1 --2- 172.21.15.45:0/1793040270 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f52440762d0 0x7f5244078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.084+0000 7f523ffff640 1 -- 172.21.15.45:0/1793040270 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f524c097830 con 0x7f52581f57e0 2024-05-08T05:46:43.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.085+0000 7f52577fe640 1 --2- 172.21.15.45:0/1793040270 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f52440762d0 0x7f5244078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.085+0000 7f52609a9740 1 Processor -- start 2024-05-08T05:46:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.085+0000 7f52609a9740 1 -- start start 2024-05-08T05:46:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.085+0000 7f52577fe640 1 --2- 172.21.15.45:0/1793040270 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f52440762d0 0x7f5244078790 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5248031000 tx=0x7f5248036040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.086+0000 7f52609a9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aa3bb22160 0x55aa3bad4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.086+0000 7f52609a9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb05280 0x55aa3bb04600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.086+0000 7f52609a9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa3bb04b40 0x55aa3bad9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.086+0000 7f52609a9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55aa3baa1690 con 0x55aa3bb05280 2024-05-08T05:46:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.086+0000 7f52609a9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55aa3ba3ef30 con 0x55aa3bb04b40 2024-05-08T05:46:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.086+0000 7f52609a9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55aa3b9f7b20 con 0x55aa3bb22160 2024-05-08T05:46:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.086+0000 7f5256ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb05280 0x55aa3bb04600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.086+0000 7f5256ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb05280 0x55aa3bb04600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36514/0 (socket says 172.21.15.45:36514) 2024-05-08T05:46:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.086+0000 7f5256ffd640 1 -- 172.21.15.45:0/2621013664 learned_addr learned my addr 172.21.15.45:0/2621013664 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.086+0000 7f5257fff640 1 --2- 172.21.15.45:0/2621013664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa3bb04b40 0x55aa3bad9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.086+0000 7f52577fe640 1 --2- 172.21.15.45:0/2621013664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aa3bb22160 0x55aa3bad4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.087+0000 7f5256ffd640 1 -- 172.21.15.45:0/2621013664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aa3bb22160 msgr2=0x55aa3bad4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.087+0000 7f5256ffd640 1 --2- 172.21.15.45:0/2621013664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aa3bb22160 0x55aa3bad4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.087+0000 7f5256ffd640 1 -- 172.21.15.45:0/2621013664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa3bb04b40 msgr2=0x55aa3bad9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.087+0000 7f5256ffd640 1 --2- 172.21.15.45:0/2621013664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa3bb04b40 0x55aa3bad9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.087+0000 7f5256ffd640 1 -- 172.21.15.45:0/2621013664 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa3b9efe60 con 0x55aa3bb05280 2024-05-08T05:46:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.087+0000 7f5256ffd640 1 --2- 172.21.15.45:0/2621013664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb05280 0x55aa3bb04600 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f524c05cea0 tx=0x7f524c0602c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c770a9ba4d84a8a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:43.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.087+0000 7f523d7fa640 1 -- 172.21.15.45:0/2621013664 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c0608b0 con 0x55aa3bb05280 2024-05-08T05:46:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.087+0000 7f523d7fa640 1 -- 172.21.15.45:0/2621013664 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f524c064c60 con 0x55aa3bb05280 2024-05-08T05:46:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.087+0000 7f523d7fa640 1 -- 172.21.15.45:0/2621013664 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c069680 con 0x55aa3bb05280 2024-05-08T05:46:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.088+0000 7f52609a9740 1 -- 172.21.15.45:0/2621013664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb05280 msgr2=0x55aa3bb04600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.088+0000 7f52609a9740 1 --2- 172.21.15.45:0/2621013664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb05280 0x55aa3bb04600 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f524c05cea0 tx=0x7f524c0602c0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.088+0000 7f52609a9740 1 -- 172.21.15.45:0/2621013664 shutdown_connections 2024-05-08T05:46:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.088+0000 7f52609a9740 1 --2- 172.21.15.45:0/2621013664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa3bb04b40 0x55aa3bad9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.088+0000 7f52609a9740 1 --2- 172.21.15.45:0/2621013664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb05280 0x55aa3bb04600 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.088+0000 7f52609a9740 1 --2- 172.21.15.45:0/2621013664 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aa3bb22160 0x55aa3bad4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.088+0000 7f52609a9740 1 -- 172.21.15.45:0/2621013664 >> 172.21.15.45:0/2621013664 conn(0x55aa3bad6d60 msgr2=0x55aa3bb1f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.088+0000 7f52609a9740 1 -- 172.21.15.45:0/2621013664 shutdown_connections 2024-05-08T05:46:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.088+0000 7f52609a9740 1 -- 172.21.15.45:0/2621013664 wait complete. 2024-05-08T05:46:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.088+0000 7f52609a9740 1 -- start start 2024-05-08T05:46:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.089+0000 7f52609a9740 1 Processor -- start 2024-05-08T05:46:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.089+0000 7f52609a9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa3bb05280 0x55aa3bb16330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.089+0000 7f52609a9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aa3bb22160 0x55aa3bb20d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.089+0000 7f52609a9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 0x55aa3baf4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.089+0000 7f52609a9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55aa3b9fb5e0 con 0x55aa3bb326b0 2024-05-08T05:46:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.089+0000 7f52609a9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55aa3ba3aef0 con 0x55aa3bb05280 2024-05-08T05:46:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.089+0000 7f52609a9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55aa3bad8f10 con 0x55aa3bb22160 2024-05-08T05:46:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.089+0000 7f52609a9740 10 client.0 _ll_register_callbacks cb 0x55aa3b9f6320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:46:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.089+0000 7f5257fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 0x55aa3baf4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.090+0000 7f5257fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 0x55aa3baf4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36530/0 (socket says 172.21.15.45:36530) 2024-05-08T05:46:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.090+0000 7f5257fff640 1 -- 172.21.15.45:0/2092979239 learned_addr learned my addr 172.21.15.45:0/2092979239 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.090+0000 7f52577fe640 1 --2- 172.21.15.45:0/2092979239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa3bb05280 0x55aa3bb16330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.090+0000 7f5256ffd640 1 --2- 172.21.15.45:0/2092979239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aa3bb22160 0x55aa3bb20d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.090+0000 7f5257fff640 1 -- 172.21.15.45:0/2092979239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aa3bb22160 msgr2=0x55aa3bb20d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.090+0000 7f5257fff640 1 --2- 172.21.15.45:0/2092979239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aa3bb22160 0x55aa3bb20d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.090+0000 7f5257fff640 1 -- 172.21.15.45:0/2092979239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa3bb05280 msgr2=0x55aa3bb16330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.090+0000 7f5257fff640 1 --2- 172.21.15.45:0/2092979239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa3bb05280 0x55aa3bb16330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.090+0000 7f5257fff640 1 -- 172.21.15.45:0/2092979239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa3b81a7d0 con 0x55aa3bb326b0 2024-05-08T05:46:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.090+0000 7f5257fff640 1 --2- 172.21.15.45:0/2092979239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 0x55aa3baf4780 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5258073310 tx=0x7f5250004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f652351b44f43b40 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.090+0000 7f52367fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:46:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.090+0000 7f52367fc640 1 -- 172.21.15.45:0/2092979239 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525001a070 con 0x55aa3bb326b0 2024-05-08T05:46:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.090+0000 7f52367fc640 1 -- 172.21.15.45:0/2092979239 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f5250007c70 con 0x55aa3bb326b0 2024-05-08T05:46:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.090+0000 7f52609a9740 1 -- 172.21.15.45:0/2092979239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f524c064c60 con 0x55aa3bb326b0 2024-05-08T05:46:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.090+0000 7f52367fc640 1 -- 172.21.15.45:0/2092979239 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f5250025050 con 0x55aa3bb326b0 2024-05-08T05:46:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.090+0000 7f52609a9740 10 client.6552 fetch_fsmap learned FSMap version 52 2024-05-08T05:46:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.090+0000 7f52609a9740 1 -- 172.21.15.45:0/2092979239 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55aa3b7503f0 con 0x55aa3bb326b0 2024-05-08T05:46:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.091+0000 7f52367fc640 1 -- 172.21.15.45:0/2092979239 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f524c064c60 con 0x55aa3bb326b0 2024-05-08T05:46:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.091+0000 7f52367fc640 1 -- 172.21.15.45:0/2092979239 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f5250028020 con 0x55aa3bb326b0 2024-05-08T05:46:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.091+0000 7f52609a9740 10 client.6552 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T05:46:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.091+0000 7f52609a9740 10 client.6552 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-08T05:46:43.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.091+0000 7f52609a9740 20 client.6552 populate_metadata read hostname 'smithi045' 2024-05-08T05:46:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.091+0000 7f52609a9740 1 --2- 172.21.15.45:0/2092979239 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55aa3bb05770 0x55aa3bb05b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.091+0000 7f52609a9740 4 client.6552 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:46:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.091+0000 7f52609a9740 1 -- 172.21.15.45:0/2092979239 --> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa3ba4cb40 con 0x55aa3bb05770 2024-05-08T05:46:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.091+0000 7f52577fe640 1 --2- 172.21.15.45:0/2092979239 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55aa3bb05770 0x55aa3bb05b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.091+0000 7f52577fe640 1 --2- 172.21.15.45:0/2092979239 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55aa3bb05770 0x55aa3bb05b80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf1f7c105b991f81 server_cookie=f58c1555296c7793 in_seq=0 out_seq=0 2024-05-08T05:46:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.091+0000 7f52367fc640 10 client.6552 ms_handle_connect on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.092+0000 7f52367fc640 1 -- 172.21.15.45:0/2092979239 <== mds.0 v2:172.21.15.45:6833/4183668301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aa3ba4cb40 con 0x55aa3bb05770 2024-05-08T05:46:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.092+0000 7f52367fc640 10 client.6552 handle_command_reply: tid=0 2024-05-08T05:46:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.092+0000 7f52609a9740 1 client.6552 shutdown 2024-05-08T05:46:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.092+0000 7f52609a9740 1 -- 172.21.15.45:0/2092979239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 msgr2=0x55aa3baf4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.092+0000 7f52609a9740 1 --2- 172.21.15.45:0/2092979239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 0x55aa3baf4780 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5258073310 tx=0x7f5250004700 comp rx=0 tx=0).stop 2024-05-08T05:46:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.092+0000 7f52609a9740 1 -- 172.21.15.45:0/2092979239 shutdown_connections 2024-05-08T05:46:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.092+0000 7f52609a9740 1 --2- 172.21.15.45:0/2092979239 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55aa3bb05770 0x55aa3bb05b80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.092+0000 7f52609a9740 1 --2- 172.21.15.45:0/2092979239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aa3bb22160 0x55aa3bb20d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.092+0000 7f52609a9740 1 --2- 172.21.15.45:0/2092979239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 0x55aa3baf4780 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.092+0000 7f52609a9740 1 --2- 172.21.15.45:0/2092979239 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa3bb05280 0x55aa3bb16330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.092+0000 7f52367fc640 0 client.6552 ms_handle_reset on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:43.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.092+0000 7f52609a9740 1 -- 172.21.15.45:0/2092979239 >> 172.21.15.45:0/2092979239 conn(0x55aa3bb04b40 msgr2=0x55aa3bad4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:43.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.092+0000 7f52609a9740 1 -- 172.21.15.45:0/2092979239 shutdown_connections 2024-05-08T05:46:43.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.092+0000 7f52609a9740 1 -- 172.21.15.45:0/2092979239 wait complete. 2024-05-08T05:46:43.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.092+0000 7f52609a9740 20 client.6552 trim_cache size 0 max 16384 2024-05-08T05:46:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.111+0000 7f52609a9740 1 Processor -- start 2024-05-08T05:46:43.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.111+0000 7f52609a9740 1 -- start start 2024-05-08T05:46:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.111+0000 7f52609a9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa3bb05280 0x55aa3bb41de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.111+0000 7f52609a9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aa3bb22160 0x55aa3bb06a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.111+0000 7f52609a9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 0x55aa3bb939a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.111+0000 7f52609a9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f52582025b0 con 0x55aa3bb326b0 2024-05-08T05:46:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.111+0000 7f52609a9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5258201610 con 0x55aa3bb05280 2024-05-08T05:46:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.111+0000 7f52609a9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f52580b8160 con 0x55aa3bb22160 2024-05-08T05:46:43.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.112+0000 7f5257fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 0x55aa3bb939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.112+0000 7f5257fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 0x55aa3bb939a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36538/0 (socket says 172.21.15.45:36538) 2024-05-08T05:46:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.112+0000 7f5257fff640 1 -- 172.21.15.45:0/3695604206 learned_addr learned my addr 172.21.15.45:0/3695604206 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.112+0000 7f5256ffd640 1 --2- 172.21.15.45:0/3695604206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aa3bb22160 0x55aa3bb06a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.112+0000 7f52577fe640 1 --2- 172.21.15.45:0/3695604206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa3bb05280 0x55aa3bb41de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.112+0000 7f5257fff640 1 -- 172.21.15.45:0/3695604206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aa3bb22160 msgr2=0x55aa3bb06a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.112+0000 7f5257fff640 1 --2- 172.21.15.45:0/3695604206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aa3bb22160 0x55aa3bb06a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.112+0000 7f5257fff640 1 -- 172.21.15.45:0/3695604206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa3bb05280 msgr2=0x55aa3bb41de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.112+0000 7f5257fff640 1 --2- 172.21.15.45:0/3695604206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa3bb05280 0x55aa3bb41de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.112+0000 7f5257fff640 1 -- 172.21.15.45:0/3695604206 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa3b7100f0 con 0x55aa3bb326b0 2024-05-08T05:46:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.112+0000 7f5257fff640 1 --2- 172.21.15.45:0/3695604206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 0x55aa3bb939a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5250007d90 tx=0x7f5250011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cc8468b6e3bc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.112+0000 7f523cff9640 1 -- 172.21.15.45:0/3695604206 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525002e070 con 0x55aa3bb326b0 2024-05-08T05:46:43.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.112+0000 7f523cff9640 1 -- 172.21.15.45:0/3695604206 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f525001f800 con 0x55aa3bb326b0 2024-05-08T05:46:43.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.112+0000 7f523cff9640 1 -- 172.21.15.45:0/3695604206 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525001a040 con 0x55aa3bb326b0 2024-05-08T05:46:43.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.113+0000 7f52609a9740 1 -- 172.21.15.45:0/3695604206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 msgr2=0x55aa3bb939a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.113+0000 7f52609a9740 1 --2- 172.21.15.45:0/3695604206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 0x55aa3bb939a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5250007d90 tx=0x7f5250011000 comp rx=0 tx=0).stop 2024-05-08T05:46:43.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.113+0000 7f52609a9740 1 -- 172.21.15.45:0/3695604206 shutdown_connections 2024-05-08T05:46:43.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.113+0000 7f52609a9740 1 --2- 172.21.15.45:0/3695604206 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aa3bb22160 0x55aa3bb06a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.113+0000 7f52609a9740 1 --2- 172.21.15.45:0/3695604206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 0x55aa3bb939a0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.113+0000 7f52609a9740 1 --2- 172.21.15.45:0/3695604206 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa3bb05280 0x55aa3bb41de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.113+0000 7f52609a9740 1 -- 172.21.15.45:0/3695604206 >> 172.21.15.45:0/3695604206 conn(0x55aa3bb04b40 msgr2=0x55aa3bb20b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:43.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.113+0000 7f52609a9740 1 -- 172.21.15.45:0/3695604206 shutdown_connections 2024-05-08T05:46:43.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.113+0000 7f52609a9740 1 -- 172.21.15.45:0/3695604206 wait complete. 2024-05-08T05:46:43.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.114+0000 7f52609a9740 1 -- start start 2024-05-08T05:46:43.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.114+0000 7f52609a9740 1 Processor -- start 2024-05-08T05:46:43.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.114+0000 7f52609a9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aa3bb05280 0x55aa3bb90ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.114+0000 7f52609a9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa3bb22160 0x55aa3bb7d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.114+0000 7f52609a9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 0x55aa3bb409c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.114+0000 7f52609a9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5258200ff0 con 0x55aa3bb326b0 2024-05-08T05:46:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.114+0000 7f52609a9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55aa3bafff00 con 0x55aa3bb22160 2024-05-08T05:46:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.114+0000 7f52609a9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55aa3bb415d0 con 0x55aa3bb05280 2024-05-08T05:46:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.114+0000 7f52609a9740 10 client.0 _ll_register_callbacks cb 0x7f5250009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:46:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.114+0000 7f5257fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 0x55aa3bb409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.114+0000 7f5257fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 0x55aa3bb409c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36542/0 (socket says 172.21.15.45:36542) 2024-05-08T05:46:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.115+0000 7f5257fff640 1 -- 172.21.15.45:0/3726114709 learned_addr learned my addr 172.21.15.45:0/3726114709 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.115+0000 7f52577fe640 1 --2- 172.21.15.45:0/3726114709 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aa3bb05280 0x55aa3bb90ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.115+0000 7f5256ffd640 1 --2- 172.21.15.45:0/3726114709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa3bb22160 0x55aa3bb7d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.115+0000 7f5257fff640 1 -- 172.21.15.45:0/3726114709 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aa3bb05280 msgr2=0x55aa3bb90ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.115+0000 7f5257fff640 1 --2- 172.21.15.45:0/3726114709 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aa3bb05280 0x55aa3bb90ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.115+0000 7f5257fff640 1 -- 172.21.15.45:0/3726114709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa3bb22160 msgr2=0x55aa3bb7d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.115+0000 7f5257fff640 1 --2- 172.21.15.45:0/3726114709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa3bb22160 0x55aa3bb7d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.115+0000 7f5257fff640 1 -- 172.21.15.45:0/3726114709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa3bb918d0 con 0x55aa3bb326b0 2024-05-08T05:46:43.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.115+0000 7f5257fff640 1 --2- 172.21.15.45:0/3726114709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 0x55aa3bb409c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5250003db0 tx=0x7f5250004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bacd715d207c295 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.115+0000 7f52367fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:46:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.115+0000 7f52367fc640 1 -- 172.21.15.45:0/3726114709 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525003a060 con 0x55aa3bb326b0 2024-05-08T05:46:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.115+0000 7f52367fc640 1 -- 172.21.15.45:0/3726114709 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f5250035450 con 0x55aa3bb326b0 2024-05-08T05:46:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.115+0000 7f52609a9740 1 -- 172.21.15.45:0/3726114709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f525001f800 con 0x55aa3bb326b0 2024-05-08T05:46:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.115+0000 7f52367fc640 1 -- 172.21.15.45:0/3726114709 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f525002e050 con 0x55aa3bb326b0 2024-05-08T05:46:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.115+0000 7f52609a9740 10 client.6558 fetch_fsmap learned FSMap version 52 2024-05-08T05:46:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.115+0000 7f52609a9740 1 -- 172.21.15.45:0/3726114709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55aa3bb7dd20 con 0x55aa3bb326b0 2024-05-08T05:46:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.115+0000 7f52367fc640 1 -- 172.21.15.45:0/3726114709 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f525001f800 con 0x55aa3bb326b0 2024-05-08T05:46:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52367fc640 1 -- 172.21.15.45:0/3726114709 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f5250028070 con 0x55aa3bb326b0 2024-05-08T05:46:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52609a9740 10 client.6558 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T05:46:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52609a9740 10 client.6558 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-05-08T05:46:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52609a9740 20 client.6558 populate_metadata read hostname 'smithi045' 2024-05-08T05:46:43.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52609a9740 1 --2- 172.21.15.45:0/3726114709 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55aa3bb40f00 0x55aa3bb00780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52609a9740 4 client.6558 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:46:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52609a9740 1 -- 172.21.15.45:0/3726114709 --> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aa3ba469b0 con 0x55aa3bb40f00 2024-05-08T05:46:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52577fe640 1 --2- 172.21.15.45:0/3726114709 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55aa3bb40f00 0x55aa3bb00780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52367fc640 10 client.6558 ms_handle_connect on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52577fe640 1 --2- 172.21.15.45:0/3726114709 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55aa3bb40f00 0x55aa3bb00780 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f096844791369c16 server_cookie=2f51d1958be6c98d in_seq=0 out_seq=0 2024-05-08T05:46:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52367fc640 1 -- 172.21.15.45:0/3726114709 <== mds.0 v2:172.21.15.45:6833/4183668301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55aa3ba469b0 con 0x55aa3bb40f00 2024-05-08T05:46:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52367fc640 10 client.6558 handle_command_reply: tid=0 2024-05-08T05:46:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52609a9740 1 client.6558 shutdown 2024-05-08T05:46:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52609a9740 1 -- 172.21.15.45:0/3726114709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 msgr2=0x55aa3bb409c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52609a9740 1 --2- 172.21.15.45:0/3726114709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 0x55aa3bb409c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5250003db0 tx=0x7f5250004ab0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52609a9740 1 -- 172.21.15.45:0/3726114709 shutdown_connections 2024-05-08T05:46:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52609a9740 1 --2- 172.21.15.45:0/3726114709 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55aa3bb40f00 0x55aa3bb00780 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52609a9740 1 --2- 172.21.15.45:0/3726114709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55aa3bb326b0 0x55aa3bb409c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52609a9740 1 --2- 172.21.15.45:0/3726114709 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55aa3bb22160 0x55aa3bb7d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52367fc640 0 client.6558 ms_handle_reset on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52609a9740 1 --2- 172.21.15.45:0/3726114709 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55aa3bb05280 0x55aa3bb90ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.116+0000 7f52609a9740 1 -- 172.21.15.45:0/3726114709 >> 172.21.15.45:0/3726114709 conn(0x55aa3bb04b40 msgr2=0x55aa3bb1e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.117+0000 7f52609a9740 1 -- 172.21.15.45:0/3726114709 shutdown_connections 2024-05-08T05:46:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.117+0000 7f52609a9740 1 -- 172.21.15.45:0/3726114709 wait complete. 2024-05-08T05:46:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.117+0000 7f52609a9740 20 client.6558 trim_cache size 0 max 16384 2024-05-08T05:46:43.124 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:46:43.124 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-05-08T05:46:43.124 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-05-08T05:46:43.124 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:46:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.117+0000 7f525dbb6640 1 -- 172.21.15.45:0/1793040270 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f52440762d0 msgr2=0x7f5244078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.117+0000 7f525dbb6640 1 --2- 172.21.15.45:0/1793040270 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f52440762d0 0x7f5244078790 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5248031000 tx=0x7f5248036040 comp rx=0 tx=0).stop 2024-05-08T05:46:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.117+0000 7f525dbb6640 1 -- 172.21.15.45:0/1793040270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52581f57e0 msgr2=0x7f52580c73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.117+0000 7f525dbb6640 1 --2- 172.21.15.45:0/1793040270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52581f57e0 0x7f52580c73d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f524c0072f0 tx=0x7f524c0077b0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.117+0000 7f525dbb6640 1 -- 172.21.15.45:0/1793040270 shutdown_connections 2024-05-08T05:46:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.117+0000 7f525dbb6640 1 --2- 172.21.15.45:0/1793040270 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f52440762d0 0x7f5244078790 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.117+0000 7f525dbb6640 1 --2- 172.21.15.45:0/1793040270 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52580c7910 0x7f52580c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.117+0000 7f525dbb6640 1 --2- 172.21.15.45:0/1793040270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52581f57e0 0x7f52580c73d0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.117+0000 7f525dbb6640 1 --2- 172.21.15.45:0/1793040270 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5240003c30 0x7f5258157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.117+0000 7f525dbb6640 1 -- 172.21.15.45:0/1793040270 >> 172.21.15.45:0/1793040270 conn(0x7f52580d2e90 msgr2=0x7f525815acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:43.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.117+0000 7f525dbb6640 1 -- 172.21.15.45:0/1793040270 shutdown_connections 2024-05-08T05:46:43.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.117+0000 7f525dbb6640 1 -- 172.21.15.45:0/1793040270 wait complete. 2024-05-08T05:46:43.128 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T05:46:43.129 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:46:43.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.232+0000 7fa5f3f24640 1 Processor -- start 2024-05-08T05:46:43.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.232+0000 7fa5f3f24640 1 -- start start 2024-05-08T05:46:43.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.233+0000 7fa5f3f24640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ec0d53d0 0x7fa5ec0d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.233+0000 7fa5f3f24640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5ec1630b0 con 0x7fa5ec0d1970 2024-05-08T05:46:43.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.233+0000 7fa5f3f24640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5ec0d3e70 con 0x7fa5ec15f420 2024-05-08T05:46:43.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.233+0000 7fa5f3f24640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa5ec0d2550 con 0x7fa5ec0d53d0 2024-05-08T05:46:43.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.233+0000 7fa5f14bf640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa5ec0d1970 0x7fa5ec0d4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47562/0 (socket says 172.21.15.45:47562) 2024-05-08T05:46:43.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.233+0000 7fa5f14bf640 1 -- 172.21.15.45:0/2885777482 learned_addr learned my addr 172.21.15.45:0/2885777482 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:43.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.234+0000 7fa5f24c1640 1 --2- 172.21.15.45:0/2885777482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ec0d53d0 0x7fa5ec0d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.234+0000 7fa5f0cbe640 1 -- 172.21.15.45:0/2885777482 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1502858921 0 0) 0x7fa5dc001110 con 0x7fa5ec0d1970 2024-05-08T05:46:43.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.234+0000 7fa5f0cbe640 1 -- 172.21.15.45:0/2885777482 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5d8003520 con 0x7fa5ec0d1970 2024-05-08T05:46:43.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.234+0000 7fa5f0cbe640 1 -- 172.21.15.45:0/2885777482 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3225278407 0 0) 0x7fa5d8003520 con 0x7fa5ec0d1970 2024-05-08T05:46:43.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.234+0000 7fa5f0cbe640 1 -- 172.21.15.45:0/2885777482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ec0d53d0 msgr2=0x7fa5ec0d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.234+0000 7fa5f0cbe640 1 --2- 172.21.15.45:0/2885777482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ec0d53d0 0x7fa5ec0d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.234+0000 7fa5f0cbe640 1 -- 172.21.15.45:0/2885777482 >> v1:172.21.15.63:6789/0 conn(0x7fa5ec15f420 legacy=0x7fa5ec0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.234+0000 7fa5f0cbe640 1 -- 172.21.15.45:0/2885777482 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ec0d2c80 con 0x7fa5ec0d1970 2024-05-08T05:46:43.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.234+0000 7fa5f0cbe640 1 -- 172.21.15.45:0/2885777482 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fa5dc0018c0 con 0x7fa5ec0d1970 2024-05-08T05:46:43.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.234+0000 7fa5f0cbe640 1 -- 172.21.15.45:0/2885777482 >> v1:172.21.15.45:6789/0 conn(0x7fa5ec0d1970 legacy=0x7fa5ec0d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.234+0000 7fa5f0cbe640 1 --2- 172.21.15.45:0/2885777482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d8003a30 0x7fa5d8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.234+0000 7fa5f0cbe640 1 --2- 172.21.15.45:0/2885777482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d8004800 0x7fa5d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.234+0000 7fa5f0cbe640 1 --2- 172.21.15.45:0/2885777482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ec0d53d0 0x7fa5d8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.234+0000 7fa5f0cbe640 1 -- 172.21.15.45:0/2885777482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec0d2550 con 0x7fa5d8003a30 2024-05-08T05:46:43.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.234+0000 7fa5f0cbe640 1 -- 172.21.15.45:0/2885777482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d800baf0 con 0x7fa5d8004800 2024-05-08T05:46:43.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.234+0000 7fa5f0cbe640 1 -- 172.21.15.45:0/2885777482 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa5d800bcf0 con 0x7fa5ec0d53d0 2024-05-08T05:46:43.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.234+0000 7fa5f1cc0640 1 --2- 172.21.15.45:0/2885777482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d8003a30 0x7fa5d8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.234+0000 7fa5f14bf640 1 --2- 172.21.15.45:0/2885777482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d8004800 0x7fa5d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.234+0000 7fa5f24c1640 1 --2- 172.21.15.45:0/2885777482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ec0d53d0 0x7fa5d8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.235+0000 7fa5f1cc0640 1 -- 172.21.15.45:0/2885777482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ec0d53d0 msgr2=0x7fa5d8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.235+0000 7fa5f1cc0640 1 --2- 172.21.15.45:0/2885777482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ec0d53d0 0x7fa5d8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.235+0000 7fa5f1cc0640 1 -- 172.21.15.45:0/2885777482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d8004800 msgr2=0x7fa5d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.235+0000 7fa5f1cc0640 1 --2- 172.21.15.45:0/2885777482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d8004800 0x7fa5d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.235+0000 7fa5f1cc0640 1 -- 172.21.15.45:0/2885777482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d8003520 con 0x7fa5d8003a30 2024-05-08T05:46:43.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.235+0000 7fa5f1cc0640 1 --2- 172.21.15.45:0/2885777482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d8003a30 0x7fa5d8003e50 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8001790 tx=0x7fa5d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12329eb3a9d80eb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:43.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.235+0000 7fa5f0cbe640 1 -- 172.21.15.45:0/2885777482 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80013b0 con 0x7fa5d8003a30 2024-05-08T05:46:43.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.235+0000 7fa5f0cbe640 1 -- 172.21.15.45:0/2885777482 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5e80052d0 con 0x7fa5d8003a30 2024-05-08T05:46:43.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.235+0000 7fa5f0cbe640 1 -- 172.21.15.45:0/2885777482 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e802aae0 con 0x7fa5d8003a30 2024-05-08T05:46:43.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.236+0000 7fa5f3f24640 1 -- 172.21.15.45:0/2885777482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d8003a30 msgr2=0x7fa5d8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.236+0000 7fa5f3f24640 1 --2- 172.21.15.45:0/2885777482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d8003a30 0x7fa5d8003e50 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8001790 tx=0x7fa5d8005830 comp rx=0 tx=0).stop 2024-05-08T05:46:43.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.236+0000 7fa5f24c1640 1 -- 172.21.15.45:0/2885777482 reap_dead start 2024-05-08T05:46:43.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.236+0000 7fa5f3f24640 1 -- 172.21.15.45:0/2885777482 shutdown_connections 2024-05-08T05:46:43.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.236+0000 7fa5f3f24640 1 -- 172.21.15.45:0/2885777482 >> 172.21.15.45:0/2885777482 conn(0x7fa5ec08eac0 msgr2=0x7fa5ec09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:43.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.236+0000 7fa5f3f24640 1 -- 172.21.15.45:0/2885777482 shutdown_connections 2024-05-08T05:46:43.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.236+0000 7fa5f3f24640 1 -- 172.21.15.45:0/2885777482 wait complete. 2024-05-08T05:46:43.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.237+0000 7fa5f3f24640 1 Processor -- start 2024-05-08T05:46:43.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.237+0000 7fa5f3f24640 1 -- start start 2024-05-08T05:46:43.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.238+0000 7fa5f3f24640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d8003a30 0x7fa5ec0cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.238+0000 7fa5f3f24640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ec0ceb10 0x7fa5ec0cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.238+0000 7fa5f3f24640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5ec0d0350 0x7fa5ec0d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.238+0000 7fa5f3f24640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec0d2830 con 0x7fa5ec0d0350 2024-05-08T05:46:43.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.238+0000 7fa5f3f24640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec0d3240 con 0x7fa5d8003a30 2024-05-08T05:46:43.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.238+0000 7fa5f3f24640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa5ec0d1e20 con 0x7fa5ec0ceb10 2024-05-08T05:46:43.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.238+0000 7fa5f24c1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5ec0d0350 0x7fa5ec0d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.238+0000 7fa5f24c1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5ec0d0350 0x7fa5ec0d07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36546/0 (socket says 172.21.15.45:36546) 2024-05-08T05:46:43.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.238+0000 7fa5f24c1640 1 -- 172.21.15.45:0/3775540729 learned_addr learned my addr 172.21.15.45:0/3775540729 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:43.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.238+0000 7fa5f1cc0640 1 --2- 172.21.15.45:0/3775540729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d8003a30 0x7fa5ec0cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.238+0000 7fa5f24c1640 1 -- 172.21.15.45:0/3775540729 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ec0ceb10 msgr2=0x7fa5ec0cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.238+0000 7fa5f24c1640 1 --2- 172.21.15.45:0/3775540729 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ec0ceb10 0x7fa5ec0cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.238+0000 7fa5f24c1640 1 -- 172.21.15.45:0/3775540729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d8003a30 msgr2=0x7fa5ec0cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.238+0000 7fa5f24c1640 1 --2- 172.21.15.45:0/3775540729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d8003a30 0x7fa5ec0cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.238+0000 7fa5f24c1640 1 -- 172.21.15.45:0/3775540729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ec0cc730 con 0x7fa5ec0d0350 2024-05-08T05:46:43.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.238+0000 7fa5f1cc0640 1 --2- 172.21.15.45:0/3775540729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d8003a30 0x7fa5ec0cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:46:43.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.239+0000 7fa5f24c1640 1 --2- 172.21.15.45:0/3775540729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5ec0d0350 0x7fa5ec0d07d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8004780 tx=0x7fa5e400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:43.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.239+0000 7fa5e27fc640 1 -- 172.21.15.45:0/3775540729 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4015430 con 0x7fa5ec0d0350 2024-05-08T05:46:43.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.239+0000 7fa5e27fc640 1 -- 172.21.15.45:0/3775540729 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5e401d070 con 0x7fa5ec0d0350 2024-05-08T05:46:43.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.239+0000 7fa5e27fc640 1 -- 172.21.15.45:0/3775540729 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4015810 con 0x7fa5ec0d0350 2024-05-08T05:46:43.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.239+0000 7fa5f3f24640 1 -- 172.21.15.45:0/3775540729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ec0cca10 con 0x7fa5ec0d0350 2024-05-08T05:46:43.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.239+0000 7fa5f3f24640 1 -- 172.21.15.45:0/3775540729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ec0ccfd0 con 0x7fa5ec0d0350 2024-05-08T05:46:43.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.240+0000 7fa5f3f24640 1 -- 172.21.15.45:0/3775540729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5ec0d0d10 con 0x7fa5ec0d0350 2024-05-08T05:46:43.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.241+0000 7fa5e27fc640 1 -- 172.21.15.45:0/3775540729 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa5e40159b0 con 0x7fa5ec0d0350 2024-05-08T05:46:43.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.241+0000 7fa5e27fc640 1 --2- 172.21.15.45:0/3775540729 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa5d00762d0 0x7fa5d0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.241+0000 7fa5e27fc640 1 -- 172.21.15.45:0/3775540729 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7fa5e4098120 con 0x7fa5ec0d0350 2024-05-08T05:46:43.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.241+0000 7fa5f1cc0640 1 --2- 172.21.15.45:0/3775540729 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa5d00762d0 0x7fa5d0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.243 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.242+0000 7fa5f1cc0640 1 --2- 172.21.15.45:0/3775540729 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa5d00762d0 0x7fa5d0078790 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8032000 tx=0x7fa5e802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:43.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.246+0000 7fa5e27fc640 1 -- 172.21.15.45:0/3775540729 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa5e4010040 con 0x7fa5ec0d0350 2024-05-08T05:46:43.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.481+0000 7fa5f3f24640 1 -- 172.21.15.45:0/3775540729 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5ec0693f0 con 0x7fa5ec0d0350 2024-05-08T05:46:43.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.481+0000 7fa5e27fc640 1 -- 172.21.15.45:0/3775540729 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5185 (secure 0 0 0) 0x7fa5e4061810 con 0x7fa5ec0d0350 2024-05-08T05:46:43.482 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:46:43.482 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5722,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1283757699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1283757699},{"type":"v1","addr":"172.21.15.63:6837","nonce":1283757699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2823770504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2823770504},{"type":"v1","addr":"172.21.15.63:6836","nonce":2823770504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1963062796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1963062796},{"type":"v1","addr":"172.21.15.45:6834","nonce":1963062796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:45:24.208158+0000","modified":"2024-05-08T05:46:16.494095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":76,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":17,"addr":"172.21.15.45:6835/4183668301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4183668301},{"type":"v1","addr":"172.21.15.45:6835","nonce":4183668301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6171,"qdb_cluster":[6171]},"id":5}]} 2024-05-08T05:46:43.482 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 52 2024-05-08T05:46:43.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.484+0000 7fa5f3f24640 1 -- 172.21.15.45:0/3775540729 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa5d00762d0 msgr2=0x7fa5d0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.484+0000 7fa5f3f24640 1 --2- 172.21.15.45:0/3775540729 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa5d00762d0 0x7fa5d0078790 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8032000 tx=0x7fa5e802e000 comp rx=0 tx=0).stop 2024-05-08T05:46:43.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.484+0000 7fa5f3f24640 1 -- 172.21.15.45:0/3775540729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5ec0d0350 msgr2=0x7fa5ec0d07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.484+0000 7fa5f3f24640 1 --2- 172.21.15.45:0/3775540729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5ec0d0350 0x7fa5ec0d07d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8004780 tx=0x7fa5e400f580 comp rx=0 tx=0).stop 2024-05-08T05:46:43.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.484+0000 7fa5f3f24640 1 -- 172.21.15.45:0/3775540729 shutdown_connections 2024-05-08T05:46:43.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.484+0000 7fa5f3f24640 1 --2- 172.21.15.45:0/3775540729 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa5d00762d0 0x7fa5d0078790 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.484+0000 7fa5f3f24640 1 --2- 172.21.15.45:0/3775540729 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa5ec0ceb10 0x7fa5ec0cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.484+0000 7fa5f3f24640 1 --2- 172.21.15.45:0/3775540729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5ec0d0350 0x7fa5ec0d07d0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.484+0000 7fa5f3f24640 1 --2- 172.21.15.45:0/3775540729 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5d8003a30 0x7fa5ec0cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.484+0000 7fa5f3f24640 1 -- 172.21.15.45:0/3775540729 >> 172.21.15.45:0/3775540729 conn(0x7fa5ec08eac0 msgr2=0x7fa5ec06cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:43.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.484+0000 7fa5f3f24640 1 -- 172.21.15.45:0/3775540729 shutdown_connections 2024-05-08T05:46:43.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.484+0000 7fa5f3f24640 1 -- 172.21.15.45:0/3775540729 wait complete. 2024-05-08T05:46:43.495 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-05-08T05:46:43.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.601+0000 7f90b6672640 1 Processor -- start 2024-05-08T05:46:43.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.601+0000 7f90b6672640 1 -- start start 2024-05-08T05:46:43.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.602+0000 7f90b6672640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f90b00b2490 0x7f90b0070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.602+0000 7f90b6672640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f90b0070d20 0x7f90b00711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.602+0000 7f90b6672640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90b006e110 con 0x7f90b01629b0 2024-05-08T05:46:43.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.602+0000 7f90b6672640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f90b006dec0 con 0x7f90b0070d20 2024-05-08T05:46:43.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.602+0000 7f90b6672640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f90b006e3f0 con 0x7f90b00b2490 2024-05-08T05:46:43.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.602+0000 7f90b4c0f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f90b0070d20 0x7f90b00711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.602+0000 7f90b4c0f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f90b0070d20 0x7f90b00711d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36560/0 (socket says 172.21.15.45:36560) 2024-05-08T05:46:43.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.602+0000 7f90b4c0f640 1 -- 172.21.15.45:0/866384152 learned_addr learned my addr 172.21.15.45:0/866384152 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:43.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.602+0000 7f90af7fe640 1 --2- 172.21.15.45:0/866384152 >> v2:172.21.15.63:3300/0 conn(0x7f90b00b2490 0x7f90b0070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.602+0000 7f90b4c0f640 1 -- 172.21.15.45:0/866384152 >> v2:172.21.15.63:3300/0 conn(0x7f90b00b2490 msgr2=0x7f90b0070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.602+0000 7f90b4c0f640 1 --2- 172.21.15.45:0/866384152 >> v2:172.21.15.63:3300/0 conn(0x7f90b00b2490 0x7f90b0070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.602+0000 7f90b4c0f640 1 -- 172.21.15.45:0/866384152 >> v1:172.21.15.63:6789/0 conn(0x7f90b01629b0 legacy=0x7f90b006eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.602+0000 7f90b4c0f640 1 -- 172.21.15.45:0/866384152 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b006e6d0 con 0x7f90b0070d20 2024-05-08T05:46:43.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.602+0000 7f90af7fe640 1 --2- 172.21.15.45:0/866384152 >> v2:172.21.15.63:3300/0 conn(0x7f90b00b2490 0x7f90b0070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:43.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.603+0000 7f90b4c0f640 1 --2- 172.21.15.45:0/866384152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b0070d20 0x7f90b00711d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f90b00b8fb0 tx=0x7f90a802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240f47c0be22e76e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:43.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.603+0000 7f90aeffd640 1 -- 172.21.15.45:0/866384152 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8036070 con 0x7f90b0070d20 2024-05-08T05:46:43.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.603+0000 7f90aeffd640 1 -- 172.21.15.45:0/866384152 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a8031600 con 0x7f90b0070d20 2024-05-08T05:46:43.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.603+0000 7f90aeffd640 1 -- 172.21.15.45:0/866384152 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a80319d0 con 0x7f90b0070d20 2024-05-08T05:46:43.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.604+0000 7f90b6672640 1 -- 172.21.15.45:0/866384152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b0070d20 msgr2=0x7f90b00711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.604+0000 7f90b6672640 1 --2- 172.21.15.45:0/866384152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b0070d20 0x7f90b00711d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f90b00b8fb0 tx=0x7f90a802a730 comp rx=0 tx=0).stop 2024-05-08T05:46:43.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.604+0000 7f90b6672640 1 -- 172.21.15.45:0/866384152 shutdown_connections 2024-05-08T05:46:43.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.604+0000 7f90b6672640 1 --2- 172.21.15.45:0/866384152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b0070d20 0x7f90b00711d0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.604+0000 7f90b6672640 1 --2- 172.21.15.45:0/866384152 >> v2:172.21.15.63:3300/0 conn(0x7f90b00b2490 0x7f90b0070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.604+0000 7f90b6672640 1 -- 172.21.15.45:0/866384152 >> 172.21.15.45:0/866384152 conn(0x7f90b00b49d0 msgr2=0x7f90b00b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:43.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.604+0000 7f90b6672640 1 -- 172.21.15.45:0/866384152 shutdown_connections 2024-05-08T05:46:43.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.604+0000 7f90b6672640 1 -- 172.21.15.45:0/866384152 wait complete. 2024-05-08T05:46:43.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.605+0000 7f90b6672640 1 Processor -- start 2024-05-08T05:46:43.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.605+0000 7f90b6672640 1 -- start start 2024-05-08T05:46:43.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.606+0000 7f90b6672640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b0070d20 0x7f90b01f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.606+0000 7f90b6672640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b00b2490 0x7f90b00cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.606+0000 7f90b6672640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b01629b0 0x7f90b00c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.606+0000 7f90b6672640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f90b00709f0 con 0x7f90b00b2490 2024-05-08T05:46:43.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.606+0000 7f90b6672640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90b0070870 con 0x7f90b01629b0 2024-05-08T05:46:43.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.606+0000 7f90b6672640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f90b00afcb0 con 0x7f90b0070d20 2024-05-08T05:46:43.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.606+0000 7f90af7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b00b2490 0x7f90b00cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.606+0000 7f90af7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b00b2490 0x7f90b00cefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36576/0 (socket says 172.21.15.45:36576) 2024-05-08T05:46:43.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.606+0000 7f90af7fe640 1 -- 172.21.15.45:0/4257495452 learned_addr learned my addr 172.21.15.45:0/4257495452 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:43.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.606+0000 7f90b4c0f640 1 --2- 172.21.15.45:0/4257495452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b01629b0 0x7f90b00c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.606+0000 7f90affff640 1 --2- 172.21.15.45:0/4257495452 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b0070d20 0x7f90b01f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.606+0000 7f90af7fe640 1 -- 172.21.15.45:0/4257495452 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b0070d20 msgr2=0x7f90b01f5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.606+0000 7f90af7fe640 1 --2- 172.21.15.45:0/4257495452 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b0070d20 0x7f90b01f5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.606+0000 7f90af7fe640 1 -- 172.21.15.45:0/4257495452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b01629b0 msgr2=0x7f90b00c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.606+0000 7f90af7fe640 1 --2- 172.21.15.45:0/4257495452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b01629b0 0x7f90b00c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.606+0000 7f90af7fe640 1 -- 172.21.15.45:0/4257495452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b00c6ad0 con 0x7f90b00b2490 2024-05-08T05:46:43.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.606+0000 7f90af7fe640 1 --2- 172.21.15.45:0/4257495452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b00b2490 0x7f90b00cefc0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f90a400e7e0 tx=0x7f90a400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:43.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.607+0000 7f90acff9640 1 -- 172.21.15.45:0/4257495452 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a400f040 con 0x7f90b00b2490 2024-05-08T05:46:43.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.607+0000 7f90acff9640 1 -- 172.21.15.45:0/4257495452 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a4004030 con 0x7f90b00b2490 2024-05-08T05:46:43.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.607+0000 7f90b6672640 1 -- 172.21.15.45:0/4257495452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b00c6db0 con 0x7f90b00b2490 2024-05-08T05:46:43.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.607+0000 7f90acff9640 1 -- 172.21.15.45:0/4257495452 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4004400 con 0x7f90b00b2490 2024-05-08T05:46:43.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.607+0000 7f90b6672640 1 -- 172.21.15.45:0/4257495452 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b00c7370 con 0x7f90b00b2490 2024-05-08T05:46:43.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.609+0000 7f90acff9640 1 -- 172.21.15.45:0/4257495452 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f90a400c6d0 con 0x7f90b00b2490 2024-05-08T05:46:43.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.610+0000 7f90acff9640 1 --2- 172.21.15.45:0/4257495452 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f909c0762d0 0x7f909c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.610+0000 7f90acff9640 1 -- 172.21.15.45:0/4257495452 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f90a40975b0 con 0x7f90b00b2490 2024-05-08T05:46:43.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.610+0000 7f90affff640 1 --2- 172.21.15.45:0/4257495452 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f909c0762d0 0x7f909c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.610+0000 7f90b938e740 1 Processor -- start 2024-05-08T05:46:43.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.610+0000 7f90b938e740 1 -- start start 2024-05-08T05:46:43.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.610+0000 7f90affff640 1 --2- 172.21.15.45:0/4257495452 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f909c0762d0 0x7f909c078790 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f90b00cd350 tx=0x7f90a0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:43.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.611+0000 7f90b938e740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bba5a796d0 0x55bba5a2c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.611+0000 7f90b938e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a5c7f0 0x55bba5a5bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.611+0000 7f90b938e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a5c0b0 0x55bba5a30ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.611+0000 7f90b938e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bba591c920 con 0x55bba5a5c0b0 2024-05-08T05:46:43.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.611+0000 7f90b938e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bba5991dd0 con 0x55bba5a5c7f0 2024-05-08T05:46:43.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.611+0000 7f90b938e740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55bba59423e0 con 0x55bba5a796d0 2024-05-08T05:46:43.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.611+0000 7f90b4c0f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a5c0b0 0x55bba5a30ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.611+0000 7f90b4c0f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a5c0b0 0x55bba5a30ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36588/0 (socket says 172.21.15.45:36588) 2024-05-08T05:46:43.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.611+0000 7f90b4c0f640 1 -- 172.21.15.45:0/2668508640 learned_addr learned my addr 172.21.15.45:0/2668508640 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:43.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.611+0000 7f90affff640 1 --2- 172.21.15.45:0/2668508640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bba5a796d0 0x55bba5a2c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.611+0000 7f90af7fe640 1 --2- 172.21.15.45:0/2668508640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a5c7f0 0x55bba5a5bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.611+0000 7f90b4c0f640 1 -- 172.21.15.45:0/2668508640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bba5a796d0 msgr2=0x55bba5a2c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.611+0000 7f90b4c0f640 1 --2- 172.21.15.45:0/2668508640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bba5a796d0 0x55bba5a2c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.611+0000 7f90b4c0f640 1 -- 172.21.15.45:0/2668508640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a5c7f0 msgr2=0x55bba5a5bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.611+0000 7f90b4c0f640 1 --2- 172.21.15.45:0/2668508640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a5c7f0 0x55bba5a5bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.611+0000 7f90b4c0f640 1 -- 172.21.15.45:0/2668508640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bba57737c0 con 0x55bba5a5c0b0 2024-05-08T05:46:43.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.611+0000 7f90af7fe640 1 --2- 172.21.15.45:0/2668508640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a5c7f0 0x55bba5a5bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:43.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.611+0000 7f90b4c0f640 1 --2- 172.21.15.45:0/2668508640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a5c0b0 0x55bba5a30ec0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f90a802ad20 tx=0x7f90a802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45794a7f265b6bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:43.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.612+0000 7f9095ffb640 1 -- 172.21.15.45:0/2668508640 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8036040 con 0x55bba5a5c0b0 2024-05-08T05:46:43.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.612+0000 7f9095ffb640 1 -- 172.21.15.45:0/2668508640 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a803d660 con 0x55bba5a5c0b0 2024-05-08T05:46:43.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.612+0000 7f9095ffb640 1 -- 172.21.15.45:0/2668508640 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a803da10 con 0x55bba5a5c0b0 2024-05-08T05:46:43.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.612+0000 7f90b938e740 1 -- 172.21.15.45:0/2668508640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a5c0b0 msgr2=0x55bba5a30ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.612+0000 7f90b938e740 1 --2- 172.21.15.45:0/2668508640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a5c0b0 0x55bba5a30ec0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f90a802ad20 tx=0x7f90a802e000 comp rx=0 tx=0).stop 2024-05-08T05:46:43.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.612+0000 7f90b938e740 1 -- 172.21.15.45:0/2668508640 shutdown_connections 2024-05-08T05:46:43.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.612+0000 7f90b938e740 1 --2- 172.21.15.45:0/2668508640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a5c0b0 0x55bba5a30ec0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.612+0000 7f90b938e740 1 --2- 172.21.15.45:0/2668508640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a5c7f0 0x55bba5a5bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.612+0000 7f90b938e740 1 --2- 172.21.15.45:0/2668508640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bba5a796d0 0x55bba5a2c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.612+0000 7f90b938e740 1 -- 172.21.15.45:0/2668508640 >> 172.21.15.45:0/2668508640 conn(0x55bba5a2e560 msgr2=0x55bba5a76b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:43.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.612+0000 7f90b938e740 1 -- 172.21.15.45:0/2668508640 shutdown_connections 2024-05-08T05:46:43.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.612+0000 7f90b938e740 1 -- 172.21.15.45:0/2668508640 wait complete. 2024-05-08T05:46:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.613+0000 7f90b938e740 1 -- start start 2024-05-08T05:46:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.613+0000 7f90b938e740 1 Processor -- start 2024-05-08T05:46:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.614+0000 7f90b938e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a5c7f0 0x55bba5a78240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.614+0000 7f90b938e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a796d0 0x55bba5a5cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.614+0000 7f90b938e740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bba5a48e30 0x55bba5a492b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.614+0000 7f90b938e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bba599a440 con 0x55bba5a5c7f0 2024-05-08T05:46:43.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.614+0000 7f90b938e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bba5995120 con 0x55bba5a796d0 2024-05-08T05:46:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.614+0000 7f90b938e740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55bba59a3210 con 0x55bba5a48e30 2024-05-08T05:46:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.614+0000 7f90b938e740 10 client.0 _ll_register_callbacks cb 0x55bba593a690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:46:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.614+0000 7f90affff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a5c7f0 0x55bba5a78240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.614+0000 7f90affff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a5c7f0 0x55bba5a78240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36598/0 (socket says 172.21.15.45:36598) 2024-05-08T05:46:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.614+0000 7f90affff640 1 -- 172.21.15.45:0/849912596 learned_addr learned my addr 172.21.15.45:0/849912596 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.614+0000 7f90af7fe640 1 --2- 172.21.15.45:0/849912596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a796d0 0x55bba5a5cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.614+0000 7f90b4c0f640 1 --2- 172.21.15.45:0/849912596 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bba5a48e30 0x55bba5a492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.614+0000 7f90affff640 1 -- 172.21.15.45:0/849912596 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bba5a48e30 msgr2=0x55bba5a492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.614+0000 7f90affff640 1 --2- 172.21.15.45:0/849912596 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bba5a48e30 0x55bba5a492b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.614+0000 7f90affff640 1 -- 172.21.15.45:0/849912596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a796d0 msgr2=0x55bba5a5cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.614+0000 7f90affff640 1 --2- 172.21.15.45:0/849912596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a796d0 0x55bba5a5cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.614+0000 7f90affff640 1 -- 172.21.15.45:0/849912596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bba57d3f50 con 0x55bba5a5c7f0 2024-05-08T05:46:43.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.615+0000 7f90affff640 1 --2- 172.21.15.45:0/849912596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a5c7f0 0x55bba5a78240 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f90a0014730 tx=0x7f90a0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b337cc335edc25 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.615+0000 7f908effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:46:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.615+0000 7f908effd640 1 -- 172.21.15.45:0/849912596 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0025020 con 0x55bba5a5c7f0 2024-05-08T05:46:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.615+0000 7f908effd640 1 -- 172.21.15.45:0/849912596 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f90a0013770 con 0x55bba5a5c7f0 2024-05-08T05:46:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.615+0000 7f90b938e740 1 -- 172.21.15.45:0/849912596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a803d660 con 0x55bba5a5c7f0 2024-05-08T05:46:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.615+0000 7f908effd640 1 -- 172.21.15.45:0/849912596 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f90a002f050 con 0x55bba5a5c7f0 2024-05-08T05:46:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.615+0000 7f90b938e740 10 client.6576 fetch_fsmap learned FSMap version 52 2024-05-08T05:46:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.615+0000 7f90b938e740 1 -- 172.21.15.45:0/849912596 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55bba56670b0 con 0x55bba5a5c7f0 2024-05-08T05:46:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.615+0000 7f908effd640 1 -- 172.21.15.45:0/849912596 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a803d660 con 0x55bba5a5c7f0 2024-05-08T05:46:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.615+0000 7f908effd640 1 -- 172.21.15.45:0/849912596 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f90a0032020 con 0x55bba5a5c7f0 2024-05-08T05:46:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.615+0000 7f90b938e740 10 client.6576 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T05:46:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.615+0000 7f90b938e740 10 client.6576 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T05:46:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.615+0000 7f90b938e740 20 client.6576 populate_metadata read hostname 'smithi045' 2024-05-08T05:46:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.615+0000 7f90b938e740 1 --2- 172.21.15.45:0/849912596 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55bba5a5d220 0x55bba5a5ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.615+0000 7f90b938e740 4 client.6576 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:46:43.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.615+0000 7f90b938e740 1 -- 172.21.15.45:0/849912596 --> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bba59a3a20 con 0x55bba5a5d220 2024-05-08T05:46:43.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.616+0000 7f90af7fe640 1 --2- 172.21.15.45:0/849912596 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55bba5a5d220 0x55bba5a5ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.616+0000 7f908effd640 10 client.6576 ms_handle_connect on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:43.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.616+0000 7f90af7fe640 1 --2- 172.21.15.45:0/849912596 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55bba5a5d220 0x55bba5a5ee20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f76be8fc3d6e5e6b server_cookie=7074b5ccbac5ee35 in_seq=0 out_seq=0 2024-05-08T05:46:43.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.616+0000 7f908effd640 1 -- 172.21.15.45:0/849912596 <== mds.0 v2:172.21.15.45:6833/4183668301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bba59a3a20 con 0x55bba5a5d220 2024-05-08T05:46:43.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.616+0000 7f908effd640 10 client.6576 handle_command_reply: tid=0 2024-05-08T05:46:43.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.616+0000 7f90b938e740 1 client.6576 shutdown 2024-05-08T05:46:43.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.617+0000 7f90b938e740 1 -- 172.21.15.45:0/849912596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a5c7f0 msgr2=0x55bba5a78240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.617+0000 7f90b938e740 1 --2- 172.21.15.45:0/849912596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a5c7f0 0x55bba5a78240 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f90a0014730 tx=0x7f90a0000ed0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.617+0000 7f90b938e740 1 -- 172.21.15.45:0/849912596 shutdown_connections 2024-05-08T05:46:43.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.617+0000 7f90b938e740 1 --2- 172.21.15.45:0/849912596 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55bba5a5d220 0x55bba5a5ee20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.617+0000 7f90b938e740 1 --2- 172.21.15.45:0/849912596 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bba5a48e30 0x55bba5a492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.617+0000 7f908effd640 0 client.6576 ms_handle_reset on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:43.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.617+0000 7f90b938e740 1 --2- 172.21.15.45:0/849912596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a796d0 0x55bba5a5cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.617+0000 7f90b938e740 1 --2- 172.21.15.45:0/849912596 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a5c7f0 0x55bba5a78240 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.617+0000 7f90b938e740 1 -- 172.21.15.45:0/849912596 >> 172.21.15.45:0/849912596 conn(0x55bba5a5c0b0 msgr2=0x55bba5a2ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:43.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.617+0000 7f90b938e740 1 -- 172.21.15.45:0/849912596 shutdown_connections 2024-05-08T05:46:43.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.617+0000 7f90b938e740 1 -- 172.21.15.45:0/849912596 wait complete. 2024-05-08T05:46:43.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.617+0000 7f90b938e740 20 client.6576 trim_cache size 0 max 16384 2024-05-08T05:46:43.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.631+0000 7f90b938e740 1 Processor -- start 2024-05-08T05:46:43.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.631+0000 7f90b938e740 1 -- start start 2024-05-08T05:46:43.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.631+0000 7f90b938e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a48e30 0x55bba5a2d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.631+0000 7f90b938e740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bba5a5c7f0 0x55bba5a86280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.631+0000 7f90b938e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a796d0 0x55bba5a867c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.631+0000 7f90b938e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f90b0201fc0 con 0x55bba5a48e30 2024-05-08T05:46:43.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.631+0000 7f90b938e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90b0201090 con 0x55bba5a796d0 2024-05-08T05:46:43.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.631+0000 7f90b938e740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f90b01f5090 con 0x55bba5a5c7f0 2024-05-08T05:46:43.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.631+0000 7f90affff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a48e30 0x55bba5a2d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.631+0000 7f90affff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a48e30 0x55bba5a2d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36602/0 (socket says 172.21.15.45:36602) 2024-05-08T05:46:43.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.631+0000 7f90affff640 1 -- 172.21.15.45:0/247664828 learned_addr learned my addr 172.21.15.45:0/247664828 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:43.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.632+0000 7f90affff640 1 -- 172.21.15.45:0/247664828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bba5a5c7f0 msgr2=0x55bba5a86280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.632+0000 7f90affff640 1 --2- 172.21.15.45:0/247664828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bba5a5c7f0 0x55bba5a86280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.632+0000 7f90affff640 1 -- 172.21.15.45:0/247664828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a796d0 msgr2=0x55bba5a867c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.632+0000 7f90affff640 1 --2- 172.21.15.45:0/247664828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a796d0 0x55bba5a867c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.632+0000 7f90affff640 1 -- 172.21.15.45:0/247664828 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bba5918290 con 0x55bba5a48e30 2024-05-08T05:46:43.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.632+0000 7f90affff640 1 --2- 172.21.15.45:0/247664828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a48e30 0x55bba5a2d030 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f90a000f620 tx=0x7f90a0021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46db28f621510863 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:43.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.632+0000 7f90957fa640 1 -- 172.21.15.45:0/247664828 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0025070 con 0x55bba5a48e30 2024-05-08T05:46:43.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.632+0000 7f90957fa640 1 -- 172.21.15.45:0/247664828 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a001b070 con 0x55bba5a48e30 2024-05-08T05:46:43.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.632+0000 7f90957fa640 1 -- 172.21.15.45:0/247664828 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0013ca0 con 0x55bba5a48e30 2024-05-08T05:46:43.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.632+0000 7f90b938e740 1 -- 172.21.15.45:0/247664828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a48e30 msgr2=0x55bba5a2d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.632+0000 7f90b938e740 1 --2- 172.21.15.45:0/247664828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a48e30 0x55bba5a2d030 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f90a000f620 tx=0x7f90a0021000 comp rx=0 tx=0).stop 2024-05-08T05:46:43.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.633+0000 7f90b938e740 1 -- 172.21.15.45:0/247664828 shutdown_connections 2024-05-08T05:46:43.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.633+0000 7f90b938e740 1 --2- 172.21.15.45:0/247664828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bba5a5c7f0 0x55bba5a86280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.633+0000 7f90b938e740 1 --2- 172.21.15.45:0/247664828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a796d0 0x55bba5a867c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.633+0000 7f90b938e740 1 --2- 172.21.15.45:0/247664828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a48e30 0x55bba5a2d030 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.633+0000 7f90b938e740 1 -- 172.21.15.45:0/247664828 >> 172.21.15.45:0/247664828 conn(0x55bba5a5c0b0 msgr2=0x55bba5a78050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:43.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.633+0000 7f90b938e740 1 -- 172.21.15.45:0/247664828 shutdown_connections 2024-05-08T05:46:43.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.633+0000 7f90b938e740 1 -- 172.21.15.45:0/247664828 wait complete. 2024-05-08T05:46:43.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.633+0000 7f90b938e740 1 -- start start 2024-05-08T05:46:43.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.633+0000 7f90b938e740 1 Processor -- start 2024-05-08T05:46:43.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.634+0000 7f90b938e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a48e30 0x55bba5af4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.634+0000 7f90b938e740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bba5a5c7f0 0x55bba5a2bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.634+0000 7f90b938e740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a796d0 0x55bba5a5ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.634+0000 7f90b938e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bba594ba20 con 0x55bba5a48e30 2024-05-08T05:46:43.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.634+0000 7f90b938e740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f90b01f56e0 con 0x55bba5a796d0 2024-05-08T05:46:43.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.634+0000 7f90b938e740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55bba5a8a910 con 0x55bba5a5c7f0 2024-05-08T05:46:43.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.634+0000 7f90b938e740 10 client.0 _ll_register_callbacks cb 0x7f90a0001400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:46:43.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.634+0000 7f90affff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a48e30 0x55bba5af4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.634+0000 7f90affff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a48e30 0x55bba5af4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36610/0 (socket says 172.21.15.45:36610) 2024-05-08T05:46:43.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.634+0000 7f90affff640 1 -- 172.21.15.45:0/3486838181 learned_addr learned my addr 172.21.15.45:0/3486838181 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.634+0000 7f90af7fe640 1 --2- 172.21.15.45:0/3486838181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bba5a5c7f0 0x55bba5a2bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.634+0000 7f90b4c0f640 1 --2- 172.21.15.45:0/3486838181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a796d0 0x55bba5a5ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.634+0000 7f90affff640 1 -- 172.21.15.45:0/3486838181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bba5a5c7f0 msgr2=0x55bba5a2bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f90affff640 1 --2- 172.21.15.45:0/3486838181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bba5a5c7f0 0x55bba5a2bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f90affff640 1 -- 172.21.15.45:0/3486838181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a796d0 msgr2=0x55bba5a5ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f90affff640 1 --2- 172.21.15.45:0/3486838181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a796d0 0x55bba5a5ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f90affff640 1 -- 172.21.15.45:0/3486838181 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bba5a9b1e0 con 0x55bba5a48e30 2024-05-08T05:46:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f90af7fe640 1 --2- 172.21.15.45:0/3486838181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bba5a5c7f0 0x55bba5a2bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f90b4c0f640 1 --2- 172.21.15.45:0/3486838181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a796d0 0x55bba5a5ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f90affff640 1 --2- 172.21.15.45:0/3486838181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a48e30 0x55bba5af4460 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f90a0001150 tx=0x7f90a0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216eee176de73d3d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f908effd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:46:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f908effd640 1 -- 172.21.15.45:0/3486838181 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0041060 con 0x55bba5a48e30 2024-05-08T05:46:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f908effd640 1 -- 172.21.15.45:0/3486838181 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f90a002a800 con 0x55bba5a48e30 2024-05-08T05:46:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f90b938e740 1 -- 172.21.15.45:0/3486838181 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a001b070 con 0x55bba5a48e30 2024-05-08T05:46:43.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f908effd640 1 -- 172.21.15.45:0/3486838181 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f90a0047050 con 0x55bba5a48e30 2024-05-08T05:46:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f90b938e740 10 client.6582 fetch_fsmap learned FSMap version 52 2024-05-08T05:46:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f90b938e740 1 -- 172.21.15.45:0/3486838181 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55bba5a5f6b0 con 0x55bba5a48e30 2024-05-08T05:46:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f908effd640 1 -- 172.21.15.45:0/3486838181 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a001b070 con 0x55bba5a48e30 2024-05-08T05:46:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f908effd640 1 -- 172.21.15.45:0/3486838181 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f90a004a020 con 0x55bba5a48e30 2024-05-08T05:46:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f90b938e740 10 client.6582 fetch_fsmap finished waiting for FSMap version 52 2024-05-08T05:46:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f90b938e740 10 client.6582 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T05:46:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f90b938e740 20 client.6582 populate_metadata read hostname 'smithi045' 2024-05-08T05:46:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f90b938e740 1 --2- 172.21.15.45:0/3486838181 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55bba5a58540 0x55bba5a479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f90b938e740 4 client.6582 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-08T05:46:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f90b938e740 1 -- 172.21.15.45:0/3486838181 --> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55bba599d890 con 0x55bba5a58540 2024-05-08T05:46:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.635+0000 7f90af7fe640 1 --2- 172.21.15.45:0/3486838181 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55bba5a58540 0x55bba5a479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.636+0000 7f908effd640 10 client.6582 ms_handle_connect on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:43.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.636+0000 7f90af7fe640 1 --2- 172.21.15.45:0/3486838181 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55bba5a58540 0x55bba5a479e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c667d3d1fb6b0cb5 server_cookie=29071788fd1895fc in_seq=0 out_seq=0 2024-05-08T05:46:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.636+0000 7f908effd640 1 -- 172.21.15.45:0/3486838181 <== mds.0 v2:172.21.15.45:6833/4183668301 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bba599d890 con 0x55bba5a58540 2024-05-08T05:46:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.636+0000 7f908effd640 10 client.6582 handle_command_reply: tid=0 2024-05-08T05:46:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.636+0000 7f90b938e740 1 client.6582 shutdown 2024-05-08T05:46:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.636+0000 7f90b938e740 1 -- 172.21.15.45:0/3486838181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a48e30 msgr2=0x55bba5af4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.636+0000 7f90b938e740 1 --2- 172.21.15.45:0/3486838181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a48e30 0x55bba5af4460 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f90a0001150 tx=0x7f90a0014000 comp rx=0 tx=0).stop 2024-05-08T05:46:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.636+0000 7f90b938e740 1 -- 172.21.15.45:0/3486838181 shutdown_connections 2024-05-08T05:46:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.636+0000 7f90b938e740 1 --2- 172.21.15.45:0/3486838181 >> [v2:172.21.15.45:6833/4183668301,v1:172.21.15.45:6835/4183668301] conn(0x55bba5a58540 0x55bba5a479e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.636+0000 7f90b938e740 1 --2- 172.21.15.45:0/3486838181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bba5a5c7f0 0x55bba5a2bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.636+0000 7f90b938e740 1 --2- 172.21.15.45:0/3486838181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bba5a796d0 0x55bba5a5ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.636+0000 7f908effd640 0 client.6582 ms_handle_reset on v2:172.21.15.45:6833/4183668301 2024-05-08T05:46:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.636+0000 7f90b938e740 1 --2- 172.21.15.45:0/3486838181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bba5a48e30 0x55bba5af4460 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.636+0000 7f90b938e740 1 -- 172.21.15.45:0/3486838181 >> 172.21.15.45:0/3486838181 conn(0x55bba5a5c0b0 msgr2=0x55bba5a492b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:43.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.636+0000 7f90b938e740 1 -- 172.21.15.45:0/3486838181 shutdown_connections 2024-05-08T05:46:43.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.636+0000 7f90b938e740 1 -- 172.21.15.45:0/3486838181 wait complete. 2024-05-08T05:46:43.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.636+0000 7f90b938e740 20 client.6582 trim_cache size 0 max 16384 2024-05-08T05:46:43.643 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:46:43.643 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-05-08T05:46:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.637+0000 7f90b6672640 1 -- 172.21.15.45:0/4257495452 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f909c0762d0 msgr2=0x7f909c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.637+0000 7f90b6672640 1 --2- 172.21.15.45:0/4257495452 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f909c0762d0 0x7f909c078790 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f90b00cd350 tx=0x7f90a0009040 comp rx=0 tx=0).stop 2024-05-08T05:46:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.637+0000 7f90b6672640 1 -- 172.21.15.45:0/4257495452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b00b2490 msgr2=0x7f90b00cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.637+0000 7f90b6672640 1 --2- 172.21.15.45:0/4257495452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b00b2490 0x7f90b00cefc0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f90a400e7e0 tx=0x7f90a400eca0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.637+0000 7f90b6672640 1 -- 172.21.15.45:0/4257495452 shutdown_connections 2024-05-08T05:46:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.637+0000 7f90b6672640 1 --2- 172.21.15.45:0/4257495452 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f909c0762d0 0x7f909c078790 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.637+0000 7f90b6672640 1 --2- 172.21.15.45:0/4257495452 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f90b01629b0 0x7f90b00c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.637+0000 7f90b6672640 1 --2- 172.21.15.45:0/4257495452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90b00b2490 0x7f90b00cefc0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.637+0000 7f90b6672640 1 --2- 172.21.15.45:0/4257495452 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f90b0070d20 0x7f90b01f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.637+0000 7f90b6672640 1 -- 172.21.15.45:0/4257495452 >> 172.21.15.45:0/4257495452 conn(0x7f90b00b49d0 msgr2=0x7f90b0161a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:43.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.637+0000 7f90b6672640 1 -- 172.21.15.45:0/4257495452 shutdown_connections 2024-05-08T05:46:43.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.637+0000 7f90b6672640 1 -- 172.21.15.45:0/4257495452 wait complete. 2024-05-08T05:46:43.648 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T05:46:43.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.750+0000 7f27b16c8640 1 Processor -- start 2024-05-08T05:46:43.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.751+0000 7f27b16c8640 1 -- start start 2024-05-08T05:46:43.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.751+0000 7f27b16c8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f27ac0d1970 0x7f27ac0d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.751+0000 7f27b16c8640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f27ac0d4cc0 0x7f27ac0d50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.751+0000 7f27b16c8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27ac1630b0 con 0x7f27ac15f420 2024-05-08T05:46:43.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.751+0000 7f27b16c8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f27ac0d3e70 con 0x7f27ac0d1970 2024-05-08T05:46:43.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.751+0000 7f27b16c8640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f27ac0d2550 con 0x7f27ac0d4cc0 2024-05-08T05:46:43.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.752+0000 7f27aa7fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f27ac0d1970 0x7f27ac0d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.752+0000 7f27aaffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f27ac15f420 0x7f27ac0d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47572/0 (socket says 172.21.15.45:47572) 2024-05-08T05:46:43.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.752+0000 7f27aaffd640 1 -- 172.21.15.45:0/1206111760 learned_addr learned my addr 172.21.15.45:0/1206111760 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:43.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.752+0000 7f27ab7fe640 1 --2- 172.21.15.45:0/1206111760 >> v2:172.21.15.63:3300/0 conn(0x7f27ac0d4cc0 0x7f27ac0d50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.752+0000 7f27aa7fc640 1 -- 172.21.15.45:0/1206111760 >> v2:172.21.15.63:3300/0 conn(0x7f27ac0d4cc0 msgr2=0x7f27ac0d50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:43.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.752+0000 7f27aa7fc640 1 --2- 172.21.15.45:0/1206111760 >> v2:172.21.15.63:3300/0 conn(0x7f27ac0d4cc0 0x7f27ac0d50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.752+0000 7f27aa7fc640 1 -- 172.21.15.45:0/1206111760 >> v1:172.21.15.45:6789/0 conn(0x7f27ac15f420 legacy=0x7f27ac0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.752+0000 7f27aa7fc640 1 -- 172.21.15.45:0/1206111760 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ac0d2c80 con 0x7f27ac0d1970 2024-05-08T05:46:43.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.753+0000 7f27aa7fc640 1 --2- 172.21.15.45:0/1206111760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27ac0d1970 0x7f27ac0d1e20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0d17a0 tx=0x7f279402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bbacb08f9e8721 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:43.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.753+0000 7f27a9ffb640 1 -- 172.21.15.45:0/1206111760 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794037070 con 0x7f27ac0d1970 2024-05-08T05:46:43.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.753+0000 7f27a9ffb640 1 -- 172.21.15.45:0/1206111760 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2794032440 con 0x7f27ac0d1970 2024-05-08T05:46:43.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.753+0000 7f27a9ffb640 1 -- 172.21.15.45:0/1206111760 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794032810 con 0x7f27ac0d1970 2024-05-08T05:46:43.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.753+0000 7f27b16c8640 1 -- 172.21.15.45:0/1206111760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27ac0d1970 msgr2=0x7f27ac0d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.753+0000 7f27b16c8640 1 --2- 172.21.15.45:0/1206111760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27ac0d1970 0x7f27ac0d1e20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0d17a0 tx=0x7f279402a1a0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.753+0000 7f27b16c8640 1 -- 172.21.15.45:0/1206111760 shutdown_connections 2024-05-08T05:46:43.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.753+0000 7f27b16c8640 1 --2- 172.21.15.45:0/1206111760 >> v2:172.21.15.63:3300/0 conn(0x7f27ac0d4cc0 0x7f27ac0d50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.753+0000 7f27b16c8640 1 --2- 172.21.15.45:0/1206111760 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27ac0d1970 0x7f27ac0d1e20 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.753+0000 7f27b16c8640 1 -- 172.21.15.45:0/1206111760 >> 172.21.15.45:0/1206111760 conn(0x7f27ac08eac0 msgr2=0x7f27ac09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:43.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.753+0000 7f27b16c8640 1 -- 172.21.15.45:0/1206111760 shutdown_connections 2024-05-08T05:46:43.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.753+0000 7f27b16c8640 1 -- 172.21.15.45:0/1206111760 wait complete. 2024-05-08T05:46:43.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.755+0000 7f27b16c8640 1 Processor -- start 2024-05-08T05:46:43.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.755+0000 7f27b16c8640 1 -- start start 2024-05-08T05:46:43.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.755+0000 7f27b16c8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27ac0d1970 0x7f27ac0ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.755+0000 7f27b16c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27ac0d4cc0 0x7f27ac151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.755+0000 7f27b16c8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27ac0cd8f0 0x7f27ac0cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.755+0000 7f27b16c8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac0d2830 con 0x7f27ac0d4cc0 2024-05-08T05:46:43.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.755+0000 7f27b16c8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f27ac0d3240 con 0x7f27ac0d1970 2024-05-08T05:46:43.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.755+0000 7f27b16c8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f27ac0d5620 con 0x7f27ac0cd8f0 2024-05-08T05:46:43.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.755+0000 7f27aa7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27ac0d4cc0 0x7f27ac151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.755+0000 7f27aa7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27ac0d4cc0 0x7f27ac151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36626/0 (socket says 172.21.15.45:36626) 2024-05-08T05:46:43.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.756+0000 7f27aa7fc640 1 -- 172.21.15.45:0/1565046006 learned_addr learned my addr 172.21.15.45:0/1565046006 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:43.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.756+0000 7f27aaffd640 1 --2- 172.21.15.45:0/1565046006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27ac0d1970 0x7f27ac0ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.756+0000 7f27ab7fe640 1 --2- 172.21.15.45:0/1565046006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27ac0cd8f0 0x7f27ac0cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.756+0000 7f27aa7fc640 1 -- 172.21.15.45:0/1565046006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27ac0cd8f0 msgr2=0x7f27ac0cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.756+0000 7f27aa7fc640 1 --2- 172.21.15.45:0/1565046006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27ac0cd8f0 0x7f27ac0cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.756+0000 7f27aa7fc640 1 -- 172.21.15.45:0/1565046006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27ac0d1970 msgr2=0x7f27ac0ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.756+0000 7f27aa7fc640 1 --2- 172.21.15.45:0/1565046006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27ac0d1970 0x7f27ac0ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.756+0000 7f27aa7fc640 1 -- 172.21.15.45:0/1565046006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ac0cc850 con 0x7f27ac0d4cc0 2024-05-08T05:46:43.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.756+0000 7f27aa7fc640 1 --2- 172.21.15.45:0/1565046006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27ac0d4cc0 0x7f27ac151500 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2794000c00 tx=0x7f27940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:43.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.756+0000 7f27a37fe640 1 -- 172.21.15.45:0/1565046006 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794037040 con 0x7f27ac0d4cc0 2024-05-08T05:46:43.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.756+0000 7f27a37fe640 1 -- 172.21.15.45:0/1565046006 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2794042070 con 0x7f27ac0d4cc0 2024-05-08T05:46:43.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.756+0000 7f27b16c8640 1 -- 172.21.15.45:0/1565046006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27ac0ccb30 con 0x7f27ac0d4cc0 2024-05-08T05:46:43.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.756+0000 7f27a37fe640 1 -- 172.21.15.45:0/1565046006 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794030bd0 con 0x7f27ac0d4cc0 2024-05-08T05:46:43.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.756+0000 7f27b16c8640 1 -- 172.21.15.45:0/1565046006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27ac0cd0f0 con 0x7f27ac0d4cc0 2024-05-08T05:46:43.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.757+0000 7f27b16c8640 1 -- 172.21.15.45:0/1565046006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27ac0693f0 con 0x7f27ac0d4cc0 2024-05-08T05:46:43.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.758+0000 7f27a37fe640 1 -- 172.21.15.45:0/1565046006 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2794047460 con 0x7f27ac0d4cc0 2024-05-08T05:46:43.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.758+0000 7f27a37fe640 1 --2- 172.21.15.45:0/1565046006 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f278c0762d0 0x7f278c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:43.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.758+0000 7f27a37fe640 1 -- 172.21.15.45:0/1565046006 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f27940b8380 con 0x7f27ac0d4cc0 2024-05-08T05:46:43.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.759+0000 7f27aaffd640 1 --2- 172.21.15.45:0/1565046006 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f278c0762d0 0x7f278c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:43.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.759+0000 7f27aaffd640 1 --2- 172.21.15.45:0/1565046006 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f278c0762d0 0x7f278c078790 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f27a4000f60 tx=0x7f27a4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:43.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.763+0000 7f27a37fe640 1 -- 172.21.15.45:0/1565046006 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2794032e20 con 0x7f27ac0d4cc0 2024-05-08T05:46:43.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.994+0000 7f27b16c8640 1 -- 172.21.15.45:0/1565046006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f27ac1638c0 con 0x7f27ac0d4cc0 2024-05-08T05:46:43.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.995+0000 7f27a37fe640 1 -- 172.21.15.45:0/1565046006 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f2794081b00 con 0x7f27ac0d4cc0 2024-05-08T05:46:43.995 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:46:43.995 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T05:46:43.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.997+0000 7f27b16c8640 1 -- 172.21.15.45:0/1565046006 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f278c0762d0 msgr2=0x7f278c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.997+0000 7f27b16c8640 1 --2- 172.21.15.45:0/1565046006 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f278c0762d0 0x7f278c078790 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f27a4000f60 tx=0x7f27a4009040 comp rx=0 tx=0).stop 2024-05-08T05:46:43.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.997+0000 7f27b16c8640 1 -- 172.21.15.45:0/1565046006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27ac0d4cc0 msgr2=0x7f27ac151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:43.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.997+0000 7f27b16c8640 1 --2- 172.21.15.45:0/1565046006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27ac0d4cc0 0x7f27ac151500 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2794000c00 tx=0x7f27940046c0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.997+0000 7f27b16c8640 1 -- 172.21.15.45:0/1565046006 shutdown_connections 2024-05-08T05:46:43.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.997+0000 7f27b16c8640 1 --2- 172.21.15.45:0/1565046006 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f278c0762d0 0x7f278c078790 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.997+0000 7f27b16c8640 1 --2- 172.21.15.45:0/1565046006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27ac0cd8f0 0x7f27ac0cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.997+0000 7f27b16c8640 1 --2- 172.21.15.45:0/1565046006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27ac0d4cc0 0x7f27ac151500 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.997+0000 7f27b16c8640 1 --2- 172.21.15.45:0/1565046006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27ac0d1970 0x7f27ac0ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:43.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.997+0000 7f27b16c8640 1 -- 172.21.15.45:0/1565046006 >> 172.21.15.45:0/1565046006 conn(0x7f27ac08eac0 msgr2=0x7f27ac0b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:43.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.997+0000 7f27b16c8640 1 -- 172.21.15.45:0/1565046006 shutdown_connections 2024-05-08T05:46:43.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:43.997+0000 7f27b16c8640 1 -- 172.21.15.45:0/1565046006 wait complete. 2024-05-08T05:46:44.008 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-08T05:46:44.008 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T05:46:44.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.112+0000 7fbf74645640 1 Processor -- start 2024-05-08T05:46:44.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.112+0000 7fbf74645640 1 -- start start 2024-05-08T05:46:44.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf74645640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0b2490 0x7fbf6c070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:44.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf74645640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbf6c070d20 0x7fbf6c0711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf74645640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf6c06e110 con 0x7fbf6c1629b0 2024-05-08T05:46:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf74645640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbf6c06dec0 con 0x7fbf6c070d20 2024-05-08T05:46:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf74645640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6c06e3f0 con 0x7fbf6c0b2490 2024-05-08T05:46:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf723e1640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbf6c1629b0 0x7fbf6c06eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47580/0 (socket says 172.21.15.45:47580) 2024-05-08T05:46:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf723e1640 1 -- 172.21.15.45:0/3710272414 learned_addr learned my addr 172.21.15.45:0/3710272414 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf713df640 1 -- 172.21.15.45:0/3710272414 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1697288777 0 0) 0x7fbf6c06e110 con 0x7fbf6c1629b0 2024-05-08T05:46:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf71be0640 1 --2- 172.21.15.45:0/3710272414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0b2490 0x7fbf6c070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf72be2640 1 --2- 172.21.15.45:0/3710272414 >> v2:172.21.15.63:3300/0 conn(0x7fbf6c070d20 0x7fbf6c0711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf713df640 1 -- 172.21.15.45:0/3710272414 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf58003520 con 0x7fbf6c1629b0 2024-05-08T05:46:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf713df640 1 -- 172.21.15.45:0/3710272414 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 302909412 0 0) 0x7fbf58003520 con 0x7fbf6c1629b0 2024-05-08T05:46:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf713df640 1 -- 172.21.15.45:0/3710272414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0b2490 msgr2=0x7fbf6c070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf713df640 1 --2- 172.21.15.45:0/3710272414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0b2490 0x7fbf6c070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf713df640 1 -- 172.21.15.45:0/3710272414 >> v2:172.21.15.63:3300/0 conn(0x7fbf6c070d20 msgr2=0x7fbf6c0711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:44.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf713df640 1 --2- 172.21.15.45:0/3710272414 >> v2:172.21.15.63:3300/0 conn(0x7fbf6c070d20 0x7fbf6c0711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:44.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf713df640 1 -- 172.21.15.45:0/3710272414 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c06e6d0 con 0x7fbf6c1629b0 2024-05-08T05:46:44.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf713df640 1 -- 172.21.15.45:0/3710272414 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fbf680016f0 con 0x7fbf6c1629b0 2024-05-08T05:46:44.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf713df640 1 -- 172.21.15.45:0/3710272414 >> v1:172.21.15.45:6789/0 conn(0x7fbf6c1629b0 legacy=0x7fbf6c06eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:44.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf713df640 1 --2- 172.21.15.45:0/3710272414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0b2490 0x7fbf58003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:44.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf713df640 1 --2- 172.21.15.45:0/3710272414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf58003fd0 0x7fbf58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:44.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf713df640 1 --2- 172.21.15.45:0/3710272414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf58004d40 0x7fbf580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:44.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf713df640 1 -- 172.21.15.45:0/3710272414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c06dec0 con 0x7fbf58003fd0 2024-05-08T05:46:44.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf713df640 1 -- 172.21.15.45:0/3710272414 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c06e3f0 con 0x7fbf58004d40 2024-05-08T05:46:44.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf713df640 1 -- 172.21.15.45:0/3710272414 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf5800ba10 con 0x7fbf6c0b2490 2024-05-08T05:46:44.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.113+0000 7fbf71be0640 1 --2- 172.21.15.45:0/3710272414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf58003fd0 0x7fbf58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:44.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.114+0000 7fbf723e1640 1 --2- 172.21.15.45:0/3710272414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0b2490 0x7fbf58003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:44.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.114+0000 7fbf72be2640 1 --2- 172.21.15.45:0/3710272414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf58004d40 0x7fbf580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:44.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.114+0000 7fbf71be0640 1 -- 172.21.15.45:0/3710272414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0b2490 msgr2=0x7fbf58003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:44.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.114+0000 7fbf71be0640 1 --2- 172.21.15.45:0/3710272414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0b2490 0x7fbf58003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:44.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.114+0000 7fbf71be0640 1 -- 172.21.15.45:0/3710272414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf58004d40 msgr2=0x7fbf580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:44.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.114+0000 7fbf71be0640 1 --2- 172.21.15.45:0/3710272414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf58004d40 0x7fbf580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:44.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.114+0000 7fbf71be0640 1 -- 172.21.15.45:0/3710272414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf58003520 con 0x7fbf58003fd0 2024-05-08T05:46:44.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.114+0000 7fbf72be2640 1 --2- 172.21.15.45:0/3710272414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf58004d40 0x7fbf580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:44.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.114+0000 7fbf723e1640 1 --2- 172.21.15.45:0/3710272414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0b2490 0x7fbf58003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:44.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.114+0000 7fbf71be0640 1 --2- 172.21.15.45:0/3710272414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf58003fd0 0x7fbf58004800 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009af0 tx=0x7fbf58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d4f5543f1ae803 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:44.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.115+0000 7fbf713df640 1 -- 172.21.15.45:0/3710272414 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c036070 con 0x7fbf58003fd0 2024-05-08T05:46:44.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.115+0000 7fbf713df640 1 -- 172.21.15.45:0/3710272414 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf5c031400 con 0x7fbf58003fd0 2024-05-08T05:46:44.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.115+0000 7fbf713df640 1 -- 172.21.15.45:0/3710272414 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c009690 con 0x7fbf58003fd0 2024-05-08T05:46:44.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.115+0000 7fbf74645640 1 -- 172.21.15.45:0/3710272414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf58003fd0 msgr2=0x7fbf58004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:44.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.115+0000 7fbf74645640 1 --2- 172.21.15.45:0/3710272414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf58003fd0 0x7fbf58004800 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009af0 tx=0x7fbf58005830 comp rx=0 tx=0).stop 2024-05-08T05:46:44.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.115+0000 7fbf72be2640 1 -- 172.21.15.45:0/3710272414 reap_dead start 2024-05-08T05:46:44.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.115+0000 7fbf74645640 1 -- 172.21.15.45:0/3710272414 shutdown_connections 2024-05-08T05:46:44.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.115+0000 7fbf74645640 1 -- 172.21.15.45:0/3710272414 >> 172.21.15.45:0/3710272414 conn(0x7fbf6c0b49d0 msgr2=0x7fbf6c0b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:44.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.115+0000 7fbf74645640 1 -- 172.21.15.45:0/3710272414 shutdown_connections 2024-05-08T05:46:44.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.115+0000 7fbf74645640 1 -- 172.21.15.45:0/3710272414 wait complete. 2024-05-08T05:46:44.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.117+0000 7fbf74645640 1 Processor -- start 2024-05-08T05:46:44.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.117+0000 7fbf74645640 1 -- start start 2024-05-08T05:46:44.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.117+0000 7fbf74645640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf58003fd0 0x7fbf6c0b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:44.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.117+0000 7fbf74645640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0b6000 0x7fbf6c0b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:44.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.117+0000 7fbf74645640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf6c14e620 0x7fbf6c14eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:44.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.117+0000 7fbf74645640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c0704b0 con 0x7fbf58003fd0 2024-05-08T05:46:44.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.117+0000 7fbf74645640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c070330 con 0x7fbf6c14e620 2024-05-08T05:46:44.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.117+0000 7fbf74645640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6c070630 con 0x7fbf6c0b6000 2024-05-08T05:46:44.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.117+0000 7fbf723e1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf58003fd0 0x7fbf6c0b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:44.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.117+0000 7fbf723e1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf58003fd0 0x7fbf6c0b5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36638/0 (socket says 172.21.15.45:36638) 2024-05-08T05:46:44.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.117+0000 7fbf723e1640 1 -- 172.21.15.45:0/2280309055 learned_addr learned my addr 172.21.15.45:0/2280309055 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:44.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.118+0000 7fbf71be0640 1 --2- 172.21.15.45:0/2280309055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0b6000 0x7fbf6c0b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.118+0000 7fbf72be2640 1 --2- 172.21.15.45:0/2280309055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf6c14e620 0x7fbf6c14eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.118+0000 7fbf723e1640 1 -- 172.21.15.45:0/2280309055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0b6000 msgr2=0x7fbf6c0b6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.118+0000 7fbf723e1640 1 --2- 172.21.15.45:0/2280309055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0b6000 0x7fbf6c0b6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.118+0000 7fbf723e1640 1 -- 172.21.15.45:0/2280309055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf6c14e620 msgr2=0x7fbf6c14eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.118+0000 7fbf723e1640 1 --2- 172.21.15.45:0/2280309055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf6c14e620 0x7fbf6c14eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.118+0000 7fbf723e1640 1 -- 172.21.15.45:0/2280309055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c14d7d0 con 0x7fbf58003fd0 2024-05-08T05:46:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.118+0000 7fbf72be2640 1 --2- 172.21.15.45:0/2280309055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf6c14e620 0x7fbf6c14eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:46:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.118+0000 7fbf723e1640 1 --2- 172.21.15.45:0/2280309055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf58003fd0 0x7fbf6c0b5ac0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fbf68001d00 tx=0x7fbf6800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.118+0000 7fbf62ffd640 1 -- 172.21.15.45:0/2280309055 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6800c870 con 0x7fbf58003fd0 2024-05-08T05:46:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.118+0000 7fbf62ffd640 1 -- 172.21.15.45:0/2280309055 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf68018070 con 0x7fbf58003fd0 2024-05-08T05:46:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.118+0000 7fbf74645640 1 -- 172.21.15.45:0/2280309055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6c0cc920 con 0x7fbf58003fd0 2024-05-08T05:46:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.118+0000 7fbf62ffd640 1 -- 172.21.15.45:0/2280309055 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6800cc10 con 0x7fbf58003fd0 2024-05-08T05:46:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.118+0000 7fbf74645640 1 -- 172.21.15.45:0/2280309055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf6c14db40 con 0x7fbf58003fd0 2024-05-08T05:46:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.119+0000 7fbf74645640 1 -- 172.21.15.45:0/2280309055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf6c0693f0 con 0x7fbf58003fd0 2024-05-08T05:46:44.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.120+0000 7fbf62ffd640 1 -- 172.21.15.45:0/2280309055 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbf6801d460 con 0x7fbf58003fd0 2024-05-08T05:46:44.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.120+0000 7fbf62ffd640 1 --2- 172.21.15.45:0/2280309055 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbf500762d0 0x7fbf50078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:44.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.121+0000 7fbf62ffd640 1 -- 172.21.15.45:0/2280309055 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7fbf68097830 con 0x7fbf58003fd0 2024-05-08T05:46:44.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.121+0000 7fbf71be0640 1 --2- 172.21.15.45:0/2280309055 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbf500762d0 0x7fbf50078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:44.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.121+0000 7fbf71be0640 1 --2- 172.21.15.45:0/2280309055 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbf500762d0 0x7fbf50078790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c033000 tx=0x7fbf5c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:44.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.125+0000 7fbf62ffd640 1 -- 172.21.15.45:0/2280309055 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbf68060f10 con 0x7fbf58003fd0 2024-05-08T05:46:44.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.355+0000 7fbf74645640 1 -- 172.21.15.45:0/2280309055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbf6c0d3d50 con 0x7fbf58003fd0 2024-05-08T05:46:44.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.356+0000 7fbf62ffd640 1 -- 172.21.15.45:0/2280309055 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) ==== 53+0+73 (secure 0 0 0) 0x7fbf68060810 con 0x7fbf58003fd0 2024-05-08T05:46:44.356 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T05:46:44.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.357+0000 7fbf74645640 1 -- 172.21.15.45:0/2280309055 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbf500762d0 msgr2=0x7fbf50078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:44.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.357+0000 7fbf74645640 1 --2- 172.21.15.45:0/2280309055 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbf500762d0 0x7fbf50078790 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c033000 tx=0x7fbf5c008030 comp rx=0 tx=0).stop 2024-05-08T05:46:44.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.357+0000 7fbf74645640 1 -- 172.21.15.45:0/2280309055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf58003fd0 msgr2=0x7fbf6c0b5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:44.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.357+0000 7fbf74645640 1 --2- 172.21.15.45:0/2280309055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf58003fd0 0x7fbf6c0b5ac0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fbf68001d00 tx=0x7fbf6800ed50 comp rx=0 tx=0).stop 2024-05-08T05:46:44.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.358+0000 7fbf74645640 1 -- 172.21.15.45:0/2280309055 shutdown_connections 2024-05-08T05:46:44.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.358+0000 7fbf74645640 1 --2- 172.21.15.45:0/2280309055 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbf500762d0 0x7fbf50078790 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:44.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.358+0000 7fbf74645640 1 --2- 172.21.15.45:0/2280309055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbf6c0b6000 0x7fbf6c0b6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:44.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.358+0000 7fbf74645640 1 --2- 172.21.15.45:0/2280309055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf6c14e620 0x7fbf6c14eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:44.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.358+0000 7fbf74645640 1 --2- 172.21.15.45:0/2280309055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf58003fd0 0x7fbf6c0b5ac0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:44.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.358+0000 7fbf74645640 1 -- 172.21.15.45:0/2280309055 >> 172.21.15.45:0/2280309055 conn(0x7fbf6c0b49d0 msgr2=0x7fbf6c0d1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:44.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.358+0000 7fbf74645640 1 -- 172.21.15.45:0/2280309055 shutdown_connections 2024-05-08T05:46:44.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44.358+0000 7fbf74645640 1 -- 172.21.15.45:0/2280309055 wait complete. 2024-05-08T05:46:44.369 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T05:46:44.369 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:46:44.369 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-05-08T05:46:44.369 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T05:46:44.369 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:46:44.369 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T05:46:44.369 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T05:46:44.369 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:46:44.369 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:46:44.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:46:44.370 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-05-08T05:46:44.389 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:46:44.389 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:46:44.389 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:46:44.389 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:46:44.390 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-05-08T05:46:44.390 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:46:44.390 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:46:44.390 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:46:44.390 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-05-08T05:46:44.390 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:46:44.390 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:46:44.390 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-05-08T05:46:44.390 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:46:44.390 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-05-08T05:46:44.390 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-05-08T05:46:44.390 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:46:44.390 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42420sec preferred_lft 42420sec 2024-05-08T05:46:44.391 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-05-08T05:46:44.391 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:46:44.391 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:46:44.391 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-05-08T05:46:44.391 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-05-08T05:46:44.391 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:46:44.391 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T05:46:44.391 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T05:46:44.391 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:46:44.391 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c80a:aff:fe94:c77/64 scope link 2024-05-08T05:46:44.391 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:46:44.391 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:46:44.391 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 82:f3:eb:6a:57:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:46:44.392 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::80f3:ebff:fe6a:57ba/64 scope link 2024-05-08T05:46:44.392 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:46:44.392 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:46:44.392 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T05:46:44.392 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::14a2:64ff:feda:3a9c/64 scope link 2024-05-08T05:46:44.392 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:46:44.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:46:44.392 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:46:44.454 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:46:44.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:46:44.456 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-05-08T05:46:44.512 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-05-08T05:46:44.512 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:46:44.512 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:46:44.513 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:46:44.513 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:46:44.513 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:46:44.513 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:46:44.513 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:46:44.513 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:46:44.513 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:46:44.513 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:46:44.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:46:44.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:46:44.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:46:44.643 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:46:44.659 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T05:46:44.659 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T05:46:44.659 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:46:44.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:46:44.659 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-05-08T05:46:44.725 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:46:44.725 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T05:46:44.782 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:46:44.783 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:46:44.783 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:46:44.839 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T05:46:44.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:46:44.839 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:46:44.924 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:46:44.924 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:46:44.932 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:46:44.932 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:46:44.932 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-08T05:46:44.932 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@e9e1170a-fd27-4625-97a9-5d5f104d13a2.cephfs=/ 2024-05-08T05:46:44.932 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.63:3300/172.21.15.63:3301" will pass to kernel 2024-05-08T05:46:44.932 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T05:46:44.933 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:46:44.933 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:46:44.933 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:46:44.933 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:46:44.934 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:46:44.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:46:44.935 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:46:44.960 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:46:45.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:46:45.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:46:45.058 INFO:teuthology.orchestra.run.smithi045.stdout:client6600 2024-05-08T05:46:45.058 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:46:45.058 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:46:45.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:46:45.075 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-05-08T05:46:45.131 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:46:45.132 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-05-08T05:46:45.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.217+0000 7f887c5d6f40 1 Processor -- start 2024-05-08T05:46:45.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.217+0000 7f887c5d6f40 1 -- start start 2024-05-08T05:46:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.218+0000 7f887c5d6f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559f3cd95f70 0x559f3cdc4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.218+0000 7f887c5d6f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3cdde9a0 0x559f3ce61950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.218+0000 7f887c5d6f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f3ce6e6e0 0x559f3ce6d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.218+0000 7f887c5d6f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559f3cd963e0 con 0x559f3cdde9a0 2024-05-08T05:46:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.218+0000 7f887c5d6f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559f3cd3ef50 con 0x559f3ce6e6e0 2024-05-08T05:46:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.218+0000 7f887c5d6f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x559f3cd98810 con 0x559f3cd95f70 2024-05-08T05:46:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.218+0000 7f88798cf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3cdde9a0 0x559f3ce61950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.218+0000 7f88798cf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3cdde9a0 0x559f3ce61950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36652/0 (socket says 172.21.15.45:36652) 2024-05-08T05:46:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.218+0000 7f88798cf640 1 -- 172.21.15.45:0/1398757945 learned_addr learned my addr 172.21.15.45:0/1398757945 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.218+0000 7f887a0d0640 1 --2- 172.21.15.45:0/1398757945 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559f3cd95f70 0x559f3cdc4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.218+0000 7f88798cf640 1 -- 172.21.15.45:0/1398757945 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559f3cd95f70 msgr2=0x559f3cdc4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.218+0000 7f88798cf640 1 --2- 172.21.15.45:0/1398757945 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559f3cd95f70 0x559f3cdc4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.218+0000 7f88798cf640 1 -- 172.21.15.45:0/1398757945 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f3ce6e6e0 msgr2=0x559f3ce6d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.218+0000 7f88798cf640 1 --2- 172.21.15.45:0/1398757945 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f3ce6e6e0 0x559f3ce6d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.218+0000 7f88798cf640 1 -- 172.21.15.45:0/1398757945 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f3ce61e90 con 0x559f3cdde9a0 2024-05-08T05:46:45.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.218+0000 7f88798cf640 1 --2- 172.21.15.45:0/1398757945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3cdde9a0 0x559f3ce61950 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f8870000c00 tx=0x7f8870051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a4bbc53bb4baba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:45.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.219+0000 7f88788cd640 1 -- 172.21.15.45:0/1398757945 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8870063070 con 0x559f3cdde9a0 2024-05-08T05:46:45.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.219+0000 7f88788cd640 1 -- 172.21.15.45:0/1398757945 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8870055b20 con 0x559f3cdde9a0 2024-05-08T05:46:45.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.219+0000 7f88788cd640 1 -- 172.21.15.45:0/1398757945 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8870068460 con 0x559f3cdde9a0 2024-05-08T05:46:45.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.219+0000 7f887c5d6f40 1 -- 172.21.15.45:0/1398757945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3cdde9a0 msgr2=0x559f3ce61950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:45.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.219+0000 7f887c5d6f40 1 --2- 172.21.15.45:0/1398757945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3cdde9a0 0x559f3ce61950 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f8870000c00 tx=0x7f8870051000 comp rx=0 tx=0).stop 2024-05-08T05:46:45.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.219+0000 7f887c5d6f40 1 -- 172.21.15.45:0/1398757945 shutdown_connections 2024-05-08T05:46:45.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.219+0000 7f887c5d6f40 1 --2- 172.21.15.45:0/1398757945 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f3ce6e6e0 0x559f3ce6d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.219+0000 7f887c5d6f40 1 --2- 172.21.15.45:0/1398757945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3cdde9a0 0x559f3ce61950 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.219+0000 7f887c5d6f40 1 --2- 172.21.15.45:0/1398757945 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559f3cd95f70 0x559f3cdc4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.219+0000 7f887c5d6f40 1 -- 172.21.15.45:0/1398757945 >> 172.21.15.45:0/1398757945 conn(0x559f3cd96b40 msgr2=0x559f3cdc3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:45.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.219+0000 7f887c5d6f40 1 -- 172.21.15.45:0/1398757945 shutdown_connections 2024-05-08T05:46:45.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.219+0000 7f887c5d6f40 1 -- 172.21.15.45:0/1398757945 wait complete. 2024-05-08T05:46:45.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.220+0000 7f887c5d6f40 1 Processor -- start 2024-05-08T05:46:45.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.220+0000 7f887c5d6f40 1 -- start start 2024-05-08T05:46:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.220+0000 7f887c5d6f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f3cd95f70 0x559f3ce5cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.220+0000 7f887c5d6f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559f3cdde9a0 0x559f3cde0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.220+0000 7f887c5d6f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3ce6e6e0 0x559f3cde0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.220+0000 7f887c5d6f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559f3ce69ff0 con 0x559f3ce6e6e0 2024-05-08T05:46:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.220+0000 7f887c5d6f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559f3ce69750 con 0x559f3cd95f70 2024-05-08T05:46:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.220+0000 7f887c5d6f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x559f3cdc5110 con 0x559f3cdde9a0 2024-05-08T05:46:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.221+0000 7f887a8d1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3ce6e6e0 0x559f3cde0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.221+0000 7f887a8d1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3ce6e6e0 0x559f3cde0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36654/0 (socket says 172.21.15.45:36654) 2024-05-08T05:46:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.221+0000 7f887a8d1640 1 -- 172.21.15.45:0/4089108798 learned_addr learned my addr 172.21.15.45:0/4089108798 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.221+0000 7f88798cf640 1 --2- 172.21.15.45:0/4089108798 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559f3cdde9a0 0x559f3cde0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.221+0000 7f887a0d0640 1 --2- 172.21.15.45:0/4089108798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f3cd95f70 0x559f3ce5cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.221+0000 7f887a8d1640 1 -- 172.21.15.45:0/4089108798 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559f3cdde9a0 msgr2=0x559f3cde0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.221+0000 7f887a8d1640 1 --2- 172.21.15.45:0/4089108798 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559f3cdde9a0 0x559f3cde0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.221+0000 7f887a8d1640 1 -- 172.21.15.45:0/4089108798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f3cd95f70 msgr2=0x559f3ce5cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.221+0000 7f887a8d1640 1 --2- 172.21.15.45:0/4089108798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559f3cd95f70 0x559f3ce5cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.221+0000 7f887a8d1640 1 -- 172.21.15.45:0/4089108798 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f3cddd680 con 0x559f3ce6e6e0 2024-05-08T05:46:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.221+0000 7f887a8d1640 1 --2- 172.21.15.45:0/4089108798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3ce6e6e0 0x559f3cde0990 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f887400bc40 tx=0x7f8874009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.221+0000 7f886b7fe640 1 -- 172.21.15.45:0/4089108798 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874008070 con 0x559f3ce6e6e0 2024-05-08T05:46:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.221+0000 7f886b7fe640 1 -- 172.21.15.45:0/4089108798 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f887400be50 con 0x559f3ce6e6e0 2024-05-08T05:46:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.221+0000 7f886b7fe640 1 -- 172.21.15.45:0/4089108798 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887400f8f0 con 0x559f3ce6e6e0 2024-05-08T05:46:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.221+0000 7f887c5d6f40 1 -- 172.21.15.45:0/4089108798 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f3ce5d790 con 0x559f3ce6e6e0 2024-05-08T05:46:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.221+0000 7f887c5d6f40 1 -- 172.21.15.45:0/4089108798 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f3cdc4a80 con 0x559f3ce6e6e0 2024-05-08T05:46:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.223+0000 7f886b7fe640 1 -- 172.21.15.45:0/4089108798 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f887400fa90 con 0x559f3ce6e6e0 2024-05-08T05:46:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.223+0000 7f886b7fe640 1 --2- 172.21.15.45:0/4089108798 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f885c074cb0 0x7f885c077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.223+0000 7f886b7fe640 1 -- 172.21.15.45:0/4089108798 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f8874096340 con 0x559f3ce6e6e0 2024-05-08T05:46:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.224+0000 7f887c5d6f40 1 --2- 172.21.15.45:0/4089108798 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x559f3ce655f0 0x559f3ce66d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.224+0000 7f887a0d0640 1 --2- 172.21.15.45:0/4089108798 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f885c074cb0 0x7f885c077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.224+0000 7f887c5d6f40 1 -- 172.21.15.45:0/4089108798 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76) -- 0x559f3ce6b530 con 0x559f3ce655f0 2024-05-08T05:46:45.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.224+0000 7f88798cf640 1 --2- 172.21.15.45:0/4089108798 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x559f3ce655f0 0x559f3ce66d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:45.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.224+0000 7f88798cf640 1 --2- 172.21.15.45:0/4089108798 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x559f3ce655f0 0x559f3ce66d40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:45.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.224+0000 7f887a0d0640 1 --2- 172.21.15.45:0/4089108798 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f885c074cb0 0x7f885c077170 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x559f3cdde2f0 tx=0x7f886c007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:45.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.226+0000 7f88798cf640 1 -- 172.21.15.45:0/4089108798 <== osd.0 v2:172.21.15.45:6800/3734454393 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v76'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8870068460 con 0x559f3ce655f0 2024-05-08T05:46:45.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.226+0000 7f887c5d6f40 1 -- 172.21.15.45:0/4089108798 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x559f3ce655f0 msgr2=0x559f3ce66d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:45.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.226+0000 7f887c5d6f40 1 --2- 172.21.15.45:0/4089108798 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x559f3ce655f0 0x559f3ce66d40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.226+0000 7f887c5d6f40 1 -- 172.21.15.45:0/4089108798 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f885c074cb0 msgr2=0x7f885c077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:45.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.226+0000 7f887c5d6f40 1 --2- 172.21.15.45:0/4089108798 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f885c074cb0 0x7f885c077170 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x559f3cdde2f0 tx=0x7f886c007040 comp rx=0 tx=0).stop 2024-05-08T05:46:45.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.226+0000 7f887c5d6f40 1 -- 172.21.15.45:0/4089108798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3ce6e6e0 msgr2=0x559f3cde0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:45.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.226+0000 7f887c5d6f40 1 --2- 172.21.15.45:0/4089108798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f3ce6e6e0 0x559f3cde0990 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f887400bc40 tx=0x7f8874009560 comp rx=0 tx=0).stop 2024-05-08T05:46:45.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.226+0000 7f887a8d1640 1 -- 172.21.15.45:0/4089108798 reap_dead start 2024-05-08T05:46:45.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.226+0000 7f887c5d6f40 1 -- 172.21.15.45:0/4089108798 shutdown_connections 2024-05-08T05:46:45.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.226+0000 7f887c5d6f40 1 -- 172.21.15.45:0/4089108798 >> 172.21.15.45:0/4089108798 conn(0x559f3cd96b40 msgr2=0x559f3ce5ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:45.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.226+0000 7f887c5d6f40 1 -- 172.21.15.45:0/4089108798 shutdown_connections 2024-05-08T05:46:45.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.226+0000 7f887c5d6f40 1 -- 172.21.15.45:0/4089108798 wait complete. 2024-05-08T05:46:45.229 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T05:46:45.229 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:46:45.270 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:46:45.321 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T05:46:45.322 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:46:45.322 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:46:45.322 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:46:45.338 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:46:45.339 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:46:45.395 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:46:45.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.543+0000 7fb9e39e4640 1 Processor -- start 2024-05-08T05:46:45.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.543+0000 7fb9e39e4640 1 -- start start 2024-05-08T05:46:45.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9e39e4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb9dc071a00 0x7fb9dc167c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:45.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9e39e4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9dc0d15c0 con 0x7fb9dc1606d0 2024-05-08T05:46:45.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9e39e4640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9dc0d18a0 con 0x7fb9dc0d0590 2024-05-08T05:46:45.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9e39e4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb9dc0d1b80 con 0x7fb9dc071a00 2024-05-08T05:46:45.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9e1780640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb9dc1606d0 0x7fb9dc0cd9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47588/0 (socket says 172.21.15.45:47588) 2024-05-08T05:46:45.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9e1780640 1 -- 172.21.15.45:0/3055789938 learned_addr learned my addr 172.21.15.45:0/3055789938 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:45.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9dbfff640 1 -- 172.21.15.45:0/3055789938 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1981090959 0 0) 0x7fb9dc0d15c0 con 0x7fb9dc1606d0 2024-05-08T05:46:45.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9dbfff640 1 -- 172.21.15.45:0/3055789938 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9c4003520 con 0x7fb9dc1606d0 2024-05-08T05:46:45.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9e1f81640 1 --2- 172.21.15.45:0/3055789938 >> v2:172.21.15.63:3300/0 conn(0x7fb9dc071a00 0x7fb9dc167c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:45.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9dbfff640 1 -- 172.21.15.45:0/3055789938 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3413664568 0 0) 0x7fb9c4003520 con 0x7fb9dc1606d0 2024-05-08T05:46:45.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9dbfff640 1 -- 172.21.15.45:0/3055789938 >> v2:172.21.15.63:3300/0 conn(0x7fb9dc071a00 msgr2=0x7fb9dc167c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:45.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9dbfff640 1 --2- 172.21.15.45:0/3055789938 >> v2:172.21.15.63:3300/0 conn(0x7fb9dc071a00 0x7fb9dc167c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9dbfff640 1 -- 172.21.15.45:0/3055789938 >> v1:172.21.15.63:6789/0 conn(0x7fb9dc0d0590 legacy=0x7fb9dc0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:45.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9dbfff640 1 -- 172.21.15.45:0/3055789938 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9dc0704d0 con 0x7fb9dc1606d0 2024-05-08T05:46:45.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9dbfff640 1 -- 172.21.15.45:0/3055789938 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fb9cc001a70 con 0x7fb9dc1606d0 2024-05-08T05:46:45.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9dbfff640 1 -- 172.21.15.45:0/3055789938 >> v1:172.21.15.45:6789/0 conn(0x7fb9dc1606d0 legacy=0x7fb9dc0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:45.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9dbfff640 1 --2- 172.21.15.45:0/3055789938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9c4003aa0 0x7fb9c4003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:45.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9dbfff640 1 --2- 172.21.15.45:0/3055789938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9c4004800 0x7fb9c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:45.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9dbfff640 1 --2- 172.21.15.45:0/3055789938 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9c40053f0 0x7fb9c400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:45.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9dbfff640 1 -- 172.21.15.45:0/3055789938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc0d1b80 con 0x7fb9c4004800 2024-05-08T05:46:45.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9dbfff640 1 -- 172.21.15.45:0/3055789938 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c400c340 con 0x7fb9c4003aa0 2024-05-08T05:46:45.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.544+0000 7fb9dbfff640 1 -- 172.21.15.45:0/3055789938 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb9c400c540 con 0x7fb9c40053f0 2024-05-08T05:46:45.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.545+0000 7fb9e0f7f640 1 --2- 172.21.15.45:0/3055789938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9c4004800 0x7fb9c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:45.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.545+0000 7fb9e1780640 1 --2- 172.21.15.45:0/3055789938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9c4003aa0 0x7fb9c4003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:45.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.545+0000 7fb9e1f81640 1 --2- 172.21.15.45:0/3055789938 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9c40053f0 0x7fb9c400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:45.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.545+0000 7fb9e0f7f640 1 -- 172.21.15.45:0/3055789938 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9c40053f0 msgr2=0x7fb9c400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:45.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.545+0000 7fb9e0f7f640 1 --2- 172.21.15.45:0/3055789938 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9c40053f0 0x7fb9c400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.545+0000 7fb9e0f7f640 1 -- 172.21.15.45:0/3055789938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9c4003aa0 msgr2=0x7fb9c4003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:45.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.545+0000 7fb9e0f7f640 1 --2- 172.21.15.45:0/3055789938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9c4003aa0 0x7fb9c4003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.545+0000 7fb9e0f7f640 1 -- 172.21.15.45:0/3055789938 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c4003520 con 0x7fb9c4004800 2024-05-08T05:46:45.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.545+0000 7fb9e1f81640 1 -- 172.21.15.45:0/3055789938 reap_dead start 2024-05-08T05:46:45.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.546+0000 7fb9e0f7f640 1 --2- 172.21.15.45:0/3055789938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9c4004800 0x7fb9c4004c80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0001760 tx=0x7fb9c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de00772be9216aed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:45.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.546+0000 7fb9dbfff640 1 -- 172.21.15.45:0/3055789938 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00013b0 con 0x7fb9c4004800 2024-05-08T05:46:45.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.546+0000 7fb9dbfff640 1 -- 172.21.15.45:0/3055789938 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9d00052d0 con 0x7fb9c4004800 2024-05-08T05:46:45.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.546+0000 7fb9dbfff640 1 -- 172.21.15.45:0/3055789938 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d002aa50 con 0x7fb9c4004800 2024-05-08T05:46:45.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.546+0000 7fb9e39e4640 1 -- 172.21.15.45:0/3055789938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9c4004800 msgr2=0x7fb9c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:45.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.546+0000 7fb9e39e4640 1 --2- 172.21.15.45:0/3055789938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9c4004800 0x7fb9c4004c80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0001760 tx=0x7fb9c4005830 comp rx=0 tx=0).stop 2024-05-08T05:46:45.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.546+0000 7fb9e39e4640 1 -- 172.21.15.45:0/3055789938 shutdown_connections 2024-05-08T05:46:45.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.546+0000 7fb9e39e4640 1 --2- 172.21.15.45:0/3055789938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9c4004800 0x7fb9c4004c80 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.546+0000 7fb9e39e4640 1 -- 172.21.15.45:0/3055789938 >> 172.21.15.45:0/3055789938 conn(0x7fb9dc0b49d0 msgr2=0x7fb9dc0b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:45.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.546+0000 7fb9e39e4640 1 -- 172.21.15.45:0/3055789938 shutdown_connections 2024-05-08T05:46:45.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.546+0000 7fb9e39e4640 1 -- 172.21.15.45:0/3055789938 wait complete. 2024-05-08T05:46:45.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.547+0000 7fb9e39e4640 1 Processor -- start 2024-05-08T05:46:45.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.548+0000 7fb9e39e4640 1 -- start start 2024-05-08T05:46:45.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.548+0000 7fb9e39e4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9c4004800 0x7fb9dc155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:45.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.548+0000 7fb9e39e4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9dc0c6680 0x7fb9dc1f91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:45.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.548+0000 7fb9e39e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9dc0b5920 0x7fb9dc0c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:45.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.548+0000 7fb9e39e4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc070b70 con 0x7fb9dc0b5920 2024-05-08T05:46:45.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.548+0000 7fb9e39e4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc0709f0 con 0x7fb9dc0c6680 2024-05-08T05:46:45.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.548+0000 7fb9e39e4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb9dc070cf0 con 0x7fb9c4004800 2024-05-08T05:46:45.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.548+0000 7fb9e1f81640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9dc0b5920 0x7fb9dc0c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:45.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.548+0000 7fb9e1f81640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9dc0b5920 0x7fb9dc0c8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36680/0 (socket says 172.21.15.45:36680) 2024-05-08T05:46:45.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.548+0000 7fb9e1f81640 1 -- 172.21.15.45:0/3522752193 learned_addr learned my addr 172.21.15.45:0/3522752193 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:45.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.549+0000 7fb9e0f7f640 1 --2- 172.21.15.45:0/3522752193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9dc0c6680 0x7fb9dc1f91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:45.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.549+0000 7fb9e1780640 1 --2- 172.21.15.45:0/3522752193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9c4004800 0x7fb9dc155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:45.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.549+0000 7fb9e1f81640 1 -- 172.21.15.45:0/3522752193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9c4004800 msgr2=0x7fb9dc155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:45.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.549+0000 7fb9e1f81640 1 --2- 172.21.15.45:0/3522752193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9c4004800 0x7fb9dc155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.549+0000 7fb9e1f81640 1 -- 172.21.15.45:0/3522752193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9dc0c6680 msgr2=0x7fb9dc1f91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:45.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.549+0000 7fb9e1f81640 1 --2- 172.21.15.45:0/3522752193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9dc0c6680 0x7fb9dc1f91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.549+0000 7fb9e1f81640 1 -- 172.21.15.45:0/3522752193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9dc0c6b00 con 0x7fb9dc0b5920 2024-05-08T05:46:45.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.549+0000 7fb9e1f81640 1 --2- 172.21.15.45:0/3522752193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9dc0b5920 0x7fb9dc0c8770 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc073660 tx=0x7fb9d400f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:45.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.549+0000 7fb9d9ffb640 1 -- 172.21.15.45:0/3522752193 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4017020 con 0x7fb9dc0b5920 2024-05-08T05:46:45.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.549+0000 7fb9d9ffb640 1 -- 172.21.15.45:0/3522752193 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9d4004440 con 0x7fb9dc0b5920 2024-05-08T05:46:45.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.549+0000 7fb9d9ffb640 1 -- 172.21.15.45:0/3522752193 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4003850 con 0x7fb9dc0b5920 2024-05-08T05:46:45.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.549+0000 7fb9e39e4640 1 -- 172.21.15.45:0/3522752193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9dc0c6d80 con 0x7fb9dc0b5920 2024-05-08T05:46:45.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.549+0000 7fb9e39e4640 1 -- 172.21.15.45:0/3522752193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9dc0c7340 con 0x7fb9dc0b5920 2024-05-08T05:46:45.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.550+0000 7fb9e39e4640 1 -- 172.21.15.45:0/3522752193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9dc0693f0 con 0x7fb9dc0b5920 2024-05-08T05:46:45.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.551+0000 7fb9d9ffb640 1 -- 172.21.15.45:0/3522752193 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb9d401a020 con 0x7fb9dc0b5920 2024-05-08T05:46:45.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.551+0000 7fb9d9ffb640 1 --2- 172.21.15.45:0/3522752193 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb9c80762d0 0x7fb9c8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:45.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.551+0000 7fb9d9ffb640 1 -- 172.21.15.45:0/3522752193 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7fb9d4097510 con 0x7fb9dc0b5920 2024-05-08T05:46:45.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.552+0000 7fb9e1780640 1 --2- 172.21.15.45:0/3522752193 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb9c80762d0 0x7fb9c8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:45.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.552+0000 7fb9e1780640 1 --2- 172.21.15.45:0/3522752193 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb9c80762d0 0x7fb9c8078790 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc001ff0 tx=0x7fb9cc006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:45.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.556+0000 7fb9d9ffb640 1 -- 172.21.15.45:0/3522752193 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb9d4021020 con 0x7fb9dc0b5920 2024-05-08T05:46:45.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.760+0000 7fb9e39e4640 1 -- 172.21.15.45:0/3522752193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb9dc0cd550 con 0x7fb9dc0b5920 2024-05-08T05:46:45.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.760+0000 7fb9d9ffb640 1 -- 172.21.15.45:0/3522752193 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v76) ==== 80+0+120 (secure 0 0 0) 0x7fb9dc0c6b00 con 0x7fb9dc0b5920 2024-05-08T05:46:45.761 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.63:6836/133050464 2024-05-09T05:45:33.480648+0000 2024-05-08T05:46:45.761 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.63:6834/133050464 2024-05-09T05:45:33.480648+0000 2024-05-08T05:46:45.761 INFO:teuthology.orchestra.run.smithi045.stderr:listed 2 entries 2024-05-08T05:46:45.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.762+0000 7fb9e39e4640 1 -- 172.21.15.45:0/3522752193 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb9c80762d0 msgr2=0x7fb9c8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:45.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.762+0000 7fb9e39e4640 1 --2- 172.21.15.45:0/3522752193 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb9c80762d0 0x7fb9c8078790 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc001ff0 tx=0x7fb9cc006040 comp rx=0 tx=0).stop 2024-05-08T05:46:45.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.762+0000 7fb9e39e4640 1 -- 172.21.15.45:0/3522752193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9dc0b5920 msgr2=0x7fb9dc0c8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:45.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.762+0000 7fb9e39e4640 1 --2- 172.21.15.45:0/3522752193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9dc0b5920 0x7fb9dc0c8770 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc073660 tx=0x7fb9d400f920 comp rx=0 tx=0).stop 2024-05-08T05:46:45.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.762+0000 7fb9e39e4640 1 -- 172.21.15.45:0/3522752193 shutdown_connections 2024-05-08T05:46:45.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.762+0000 7fb9e39e4640 1 --2- 172.21.15.45:0/3522752193 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb9c80762d0 0x7fb9c8078790 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.762+0000 7fb9e39e4640 1 --2- 172.21.15.45:0/3522752193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb9dc0b5920 0x7fb9dc0c8770 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.762+0000 7fb9e39e4640 1 --2- 172.21.15.45:0/3522752193 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9dc0c6680 0x7fb9dc1f91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.762+0000 7fb9e39e4640 1 --2- 172.21.15.45:0/3522752193 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb9c4004800 0x7fb9dc155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:45.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.763+0000 7fb9e39e4640 1 -- 172.21.15.45:0/3522752193 >> 172.21.15.45:0/3522752193 conn(0x7fb9dc0b49d0 msgr2=0x7fb9dc15f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:45.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.763+0000 7fb9e39e4640 1 -- 172.21.15.45:0/3522752193 shutdown_connections 2024-05-08T05:46:45.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:45.763+0000 7fb9e39e4640 1 -- 172.21.15.45:0/3522752193 wait complete. 2024-05-08T05:46:45.773 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T05:46:45.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:46:45.774 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T05:46:45.803 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:46:45.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:46:45.803 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T05:46:45.866 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:46:45.866 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:46:45.866 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:46:45.887 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:46:45.887 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:46:45.888 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:46:45.888 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:46:45.934 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:46:45.934 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:46:45.935 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:46:45.935 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:46:45.955 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:46:45.955 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:46:46.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.093+0000 7ff9674c5640 1 Processor -- start 2024-05-08T05:46:46.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.093+0000 7ff9674c5640 1 -- start start 2024-05-08T05:46:46.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.094+0000 7ff9674c5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff960070d30 0x7ff9600d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.094+0000 7ff9674c5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff960163920 0x7ff9600703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.094+0000 7ff9674c5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9600708e0 con 0x7ff960071110 2024-05-08T05:46:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.094+0000 7ff9674c5640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff9600d10d0 con 0x7ff960070d30 2024-05-08T05:46:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.094+0000 7ff9674c5640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff960163db0 con 0x7ff960163920 2024-05-08T05:46:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.094+0000 7ff965261640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff960070d30 0x7ff9600d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.094+0000 7ff965261640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff960070d30 0x7ff9600d20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36684/0 (socket says 172.21.15.45:36684) 2024-05-08T05:46:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.094+0000 7ff965261640 1 -- 172.21.15.45:0/2243319968 learned_addr learned my addr 172.21.15.45:0/2243319968 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.094+0000 7ff964a60640 1 --2- 172.21.15.45:0/2243319968 >> v2:172.21.15.63:3300/0 conn(0x7ff960163920 0x7ff9600703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.094+0000 7ff965261640 1 -- 172.21.15.45:0/2243319968 >> v2:172.21.15.63:3300/0 conn(0x7ff960163920 msgr2=0x7ff9600703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.094+0000 7ff965261640 1 --2- 172.21.15.45:0/2243319968 >> v2:172.21.15.63:3300/0 conn(0x7ff960163920 0x7ff9600703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.094+0000 7ff965261640 1 -- 172.21.15.45:0/2243319968 >> v1:172.21.15.63:6789/0 conn(0x7ff960071110 legacy=0x7ff96006d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.095+0000 7ff965261640 1 -- 172.21.15.45:0/2243319968 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9600d1300 con 0x7ff960070d30 2024-05-08T05:46:46.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.095+0000 7ff965261640 1 --2- 172.21.15.45:0/2243319968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff960070d30 0x7ff9600d20c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff960073710 tx=0x7ff95c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51fb8af76ad42cbc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:46.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.097+0000 7ff957fff640 1 -- 172.21.15.45:0/2243319968 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c037070 con 0x7ff960070d30 2024-05-08T05:46:46.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.097+0000 7ff957fff640 1 -- 172.21.15.45:0/2243319968 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95c032440 con 0x7ff960070d30 2024-05-08T05:46:46.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.097+0000 7ff957fff640 1 -- 172.21.15.45:0/2243319968 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c032810 con 0x7ff960070d30 2024-05-08T05:46:46.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.097+0000 7ff9674c5640 1 -- 172.21.15.45:0/2243319968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff960070d30 msgr2=0x7ff9600d20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:46.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.097+0000 7ff9674c5640 1 --2- 172.21.15.45:0/2243319968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff960070d30 0x7ff9600d20c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff960073710 tx=0x7ff95c02a270 comp rx=0 tx=0).stop 2024-05-08T05:46:46.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.097+0000 7ff9674c5640 1 -- 172.21.15.45:0/2243319968 shutdown_connections 2024-05-08T05:46:46.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.097+0000 7ff9674c5640 1 --2- 172.21.15.45:0/2243319968 >> v2:172.21.15.63:3300/0 conn(0x7ff960163920 0x7ff9600703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.097+0000 7ff9674c5640 1 --2- 172.21.15.45:0/2243319968 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff960070d30 0x7ff9600d20c0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.097+0000 7ff9674c5640 1 -- 172.21.15.45:0/2243319968 >> 172.21.15.45:0/2243319968 conn(0x7ff9600d2e90 msgr2=0x7ff9600d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:46.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.097+0000 7ff9674c5640 1 -- 172.21.15.45:0/2243319968 shutdown_connections 2024-05-08T05:46:46.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.097+0000 7ff9674c5640 1 -- 172.21.15.45:0/2243319968 wait complete. 2024-05-08T05:46:46.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.098+0000 7ff9674c5640 1 Processor -- start 2024-05-08T05:46:46.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.099+0000 7ff9674c5640 1 -- start start 2024-05-08T05:46:46.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.099+0000 7ff9674c5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff960070d30 0x7ff9600b8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.099+0000 7ff9674c5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff960071110 0x7ff9600b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.099+0000 7ff9674c5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff960163920 0x7ff9600b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.099+0000 7ff9674c5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9600d1c30 con 0x7ff960163920 2024-05-08T05:46:46.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.099+0000 7ff9674c5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff960070b80 con 0x7ff960070d30 2024-05-08T05:46:46.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.099+0000 7ff9674c5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff9600d18c0 con 0x7ff960071110 2024-05-08T05:46:46.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.099+0000 7ff965a62640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff960163920 0x7ff9600b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.099+0000 7ff965a62640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff960163920 0x7ff9600b4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36694/0 (socket says 172.21.15.45:36694) 2024-05-08T05:46:46.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.099+0000 7ff965a62640 1 -- 172.21.15.45:0/3460757912 learned_addr learned my addr 172.21.15.45:0/3460757912 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:46.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.100+0000 7ff965261640 1 --2- 172.21.15.45:0/3460757912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff960070d30 0x7ff9600b8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.100+0000 7ff964a60640 1 --2- 172.21.15.45:0/3460757912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff960071110 0x7ff9600b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.100+0000 7ff965a62640 1 -- 172.21.15.45:0/3460757912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff960071110 msgr2=0x7ff9600b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:46.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.100+0000 7ff965a62640 1 --2- 172.21.15.45:0/3460757912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff960071110 0x7ff9600b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.100+0000 7ff965a62640 1 -- 172.21.15.45:0/3460757912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff960070d30 msgr2=0x7ff9600b8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:46.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.100+0000 7ff965a62640 1 --2- 172.21.15.45:0/3460757912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff960070d30 0x7ff9600b8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.100+0000 7ff965a62640 1 -- 172.21.15.45:0/3460757912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9600b4ff0 con 0x7ff960163920 2024-05-08T05:46:46.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.100+0000 7ff964a60640 1 --2- 172.21.15.45:0/3460757912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff960071110 0x7ff9600b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:46.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.100+0000 7ff965a62640 1 --2- 172.21.15.45:0/3460757912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff960163920 0x7ff9600b4990 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff95800c6e0 tx=0x7ff95800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:46.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.100+0000 7ff955ffb640 1 -- 172.21.15.45:0/3460757912 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff958003b70 con 0x7ff960163920 2024-05-08T05:46:46.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.100+0000 7ff955ffb640 1 -- 172.21.15.45:0/3460757912 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff958003d30 con 0x7ff960163920 2024-05-08T05:46:46.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.100+0000 7ff9674c5640 1 -- 172.21.15.45:0/3460757912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9600b52d0 con 0x7ff960163920 2024-05-08T05:46:46.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.100+0000 7ff955ffb640 1 -- 172.21.15.45:0/3460757912 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff958005310 con 0x7ff960163920 2024-05-08T05:46:46.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.100+0000 7ff9674c5640 1 -- 172.21.15.45:0/3460757912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96014a950 con 0x7ff960163920 2024-05-08T05:46:46.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.102+0000 7ff9674c5640 1 -- 172.21.15.45:0/3460757912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9600693f0 con 0x7ff960163920 2024-05-08T05:46:46.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.102+0000 7ff955ffb640 1 -- 172.21.15.45:0/3460757912 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff9580054b0 con 0x7ff960163920 2024-05-08T05:46:46.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.103+0000 7ff955ffb640 1 --2- 172.21.15.45:0/3460757912 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9440762d0 0x7ff944078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.103+0000 7ff955ffb640 1 -- 172.21.15.45:0/3460757912 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7ff9580975e0 con 0x7ff960163920 2024-05-08T05:46:46.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.103+0000 7ff965261640 1 --2- 172.21.15.45:0/3460757912 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9440762d0 0x7ff944078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.104+0000 7ff965261640 1 --2- 172.21.15.45:0/3460757912 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9440762d0 0x7ff944078790 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff95c034000 tx=0x7ff95c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:46.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.107+0000 7ff955ffb640 1 -- 172.21.15.45:0/3460757912 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff958060cd0 con 0x7ff960163920 2024-05-08T05:46:46.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.336+0000 7ff9674c5640 1 -- 172.21.15.45:0/3460757912 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff96006d140 con 0x7ff960163920 2024-05-08T05:46:46.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.337+0000 7ff955ffb640 1 -- 172.21.15.45:0/3460757912 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5185 (secure 0 0 0) 0x7ff9580605d0 con 0x7ff960163920 2024-05-08T05:46:46.337 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:46:46.338 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5722,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1283757699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1283757699},{"type":"v1","addr":"172.21.15.63:6837","nonce":1283757699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2823770504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2823770504},{"type":"v1","addr":"172.21.15.63:6836","nonce":2823770504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1963062796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1963062796},{"type":"v1","addr":"172.21.15.45:6834","nonce":1963062796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:45:24.208158+0000","modified":"2024-05-08T05:46:16.494095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":76,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":17,"addr":"172.21.15.45:6835/4183668301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4183668301},{"type":"v1","addr":"172.21.15.45:6835","nonce":4183668301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6171,"qdb_cluster":[6171]},"id":5}]} 2024-05-08T05:46:46.338 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 52 2024-05-08T05:46:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.339+0000 7ff9674c5640 1 -- 172.21.15.45:0/3460757912 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9440762d0 msgr2=0x7ff944078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.339+0000 7ff9674c5640 1 --2- 172.21.15.45:0/3460757912 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9440762d0 0x7ff944078790 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff95c034000 tx=0x7ff95c02e000 comp rx=0 tx=0).stop 2024-05-08T05:46:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.339+0000 7ff9674c5640 1 -- 172.21.15.45:0/3460757912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff960163920 msgr2=0x7ff9600b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.339+0000 7ff9674c5640 1 --2- 172.21.15.45:0/3460757912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff960163920 0x7ff9600b4990 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff95800c6e0 tx=0x7ff95800cba0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.339+0000 7ff9674c5640 1 -- 172.21.15.45:0/3460757912 shutdown_connections 2024-05-08T05:46:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.339+0000 7ff9674c5640 1 --2- 172.21.15.45:0/3460757912 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff9440762d0 0x7ff944078790 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.339+0000 7ff9674c5640 1 --2- 172.21.15.45:0/3460757912 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff960071110 0x7ff9600b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.339+0000 7ff9674c5640 1 --2- 172.21.15.45:0/3460757912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff960163920 0x7ff9600b4990 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.339+0000 7ff9674c5640 1 --2- 172.21.15.45:0/3460757912 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff960070d30 0x7ff9600b8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.339+0000 7ff9674c5640 1 -- 172.21.15.45:0/3460757912 >> 172.21.15.45:0/3460757912 conn(0x7ff9600d2e90 msgr2=0x7ff9600b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.339+0000 7ff9674c5640 1 -- 172.21.15.45:0/3460757912 shutdown_connections 2024-05-08T05:46:46.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.339+0000 7ff9674c5640 1 -- 172.21.15.45:0/3460757912 wait complete. 2024-05-08T05:46:46.351 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:46:46.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.458+0000 7f1cb4de4640 1 Processor -- start 2024-05-08T05:46:46.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.458+0000 7f1cb4de4640 1 -- start start 2024-05-08T05:46:46.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.459+0000 7f1cb4de4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cb006f060 0x7f1cb006e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.459+0000 7f1cb4de4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cb015d2a0 con 0x7f1cb006fe30 2024-05-08T05:46:46.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.459+0000 7f1cb4de4640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cb01609b0 con 0x7f1cb00d0530 2024-05-08T05:46:46.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.459+0000 7f1cb4de4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb006f440 con 0x7f1cb006f060 2024-05-08T05:46:46.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.459+0000 7f1cadd9b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1cb006fe30 0x7f1cb00cfe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47602/0 (socket says 172.21.15.45:47602) 2024-05-08T05:46:46.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.459+0000 7f1cadd9b640 1 -- 172.21.15.45:0/3741670956 learned_addr learned my addr 172.21.15.45:0/3741670956 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:46.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cae59c640 1 --2- 172.21.15.45:0/3741670956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cb006f060 0x7f1cb006e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cad59a640 1 -- 172.21.15.45:0/3741670956 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3083567598 0 0) 0x7f1cb015d2a0 con 0x7f1cb006fe30 2024-05-08T05:46:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cad59a640 1 -- 172.21.15.45:0/3741670956 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c98003520 con 0x7f1cb006fe30 2024-05-08T05:46:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cad59a640 1 -- 172.21.15.45:0/3741670956 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 45653980 0 0) 0x7f1c98003520 con 0x7f1cb006fe30 2024-05-08T05:46:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cad59a640 1 -- 172.21.15.45:0/3741670956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cb006f060 msgr2=0x7f1cb006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cad59a640 1 --2- 172.21.15.45:0/3741670956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cb006f060 0x7f1cb006e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cad59a640 1 -- 172.21.15.45:0/3741670956 >> v1:172.21.15.63:6789/0 conn(0x7f1cb00d0530 legacy=0x7f1cb00b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cad59a640 1 -- 172.21.15.45:0/3741670956 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb0160c90 con 0x7f1cb006fe30 2024-05-08T05:46:46.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cad59a640 1 -- 172.21.15.45:0/3741670956 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f1ca40016f0 con 0x7f1cb006fe30 2024-05-08T05:46:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cad59a640 1 -- 172.21.15.45:0/3741670956 >> v1:172.21.15.45:6789/0 conn(0x7f1cb006fe30 legacy=0x7f1cb00cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cad59a640 1 --2- 172.21.15.45:0/3741670956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c980039e0 0x7f1c98003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cad59a640 1 --2- 172.21.15.45:0/3741670956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c98004800 0x7f1c98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cad59a640 1 --2- 172.21.15.45:0/3741670956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cb006f060 0x7f1c980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cad59a640 1 -- 172.21.15.45:0/3741670956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb006f440 con 0x7f1c980039e0 2024-05-08T05:46:46.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cad59a640 1 -- 172.21.15.45:0/3741670956 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9800ba70 con 0x7f1c98004800 2024-05-08T05:46:46.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cad59a640 1 -- 172.21.15.45:0/3741670956 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1c9800bc70 con 0x7f1cb006f060 2024-05-08T05:46:46.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cae59c640 1 --2- 172.21.15.45:0/3741670956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c980039e0 0x7f1c98003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cadd9b640 1 --2- 172.21.15.45:0/3741670956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c98004800 0x7f1c98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1caed9d640 1 --2- 172.21.15.45:0/3741670956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cb006f060 0x7f1c980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cae59c640 1 -- 172.21.15.45:0/3741670956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cb006f060 msgr2=0x7f1c980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:46.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cae59c640 1 --2- 172.21.15.45:0/3741670956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1cb006f060 0x7f1c980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cae59c640 1 -- 172.21.15.45:0/3741670956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c98004800 msgr2=0x7f1c98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:46.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cae59c640 1 --2- 172.21.15.45:0/3741670956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1c98004800 0x7f1c98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.460+0000 7f1cae59c640 1 -- 172.21.15.45:0/3741670956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c98003520 con 0x7f1c980039e0 2024-05-08T05:46:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.461+0000 7f1cae59c640 1 --2- 172.21.15.45:0/3741670956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c980039e0 0x7f1c98003dc0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009a90 tx=0x7f1c98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11fed6309b450fa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.461+0000 7f1cad59a640 1 -- 172.21.15.45:0/3741670956 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0036070 con 0x7f1c980039e0 2024-05-08T05:46:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.461+0000 7f1cad59a640 1 -- 172.21.15.45:0/3741670956 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca0031400 con 0x7f1c980039e0 2024-05-08T05:46:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.461+0000 7f1cad59a640 1 -- 172.21.15.45:0/3741670956 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0009690 con 0x7f1c980039e0 2024-05-08T05:46:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.462+0000 7f1cb4de4640 1 -- 172.21.15.45:0/3741670956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c980039e0 msgr2=0x7f1c98003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.462+0000 7f1cb4de4640 1 --2- 172.21.15.45:0/3741670956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1c980039e0 0x7f1c98003dc0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009a90 tx=0x7f1c98005830 comp rx=0 tx=0).stop 2024-05-08T05:46:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.462+0000 7f1caed9d640 1 -- 172.21.15.45:0/3741670956 reap_dead start 2024-05-08T05:46:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.462+0000 7f1cb4de4640 1 -- 172.21.15.45:0/3741670956 shutdown_connections 2024-05-08T05:46:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.462+0000 7f1cb4de4640 1 -- 172.21.15.45:0/3741670956 >> 172.21.15.45:0/3741670956 conn(0x7f1cb006c3a0 msgr2=0x7f1cb006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.462+0000 7f1cb4de4640 1 -- 172.21.15.45:0/3741670956 shutdown_connections 2024-05-08T05:46:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.462+0000 7f1cb4de4640 1 -- 172.21.15.45:0/3741670956 wait complete. 2024-05-08T05:46:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.463+0000 7f1cb4de4640 1 Processor -- start 2024-05-08T05:46:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.463+0000 7f1cb4de4640 1 -- start start 2024-05-08T05:46:46.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.464+0000 7f1cb4de4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c980039e0 0x7f1cb0154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.464+0000 7f1cb4de4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb0155180 0x7f1cb0155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.464+0000 7f1cb4de4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cb0155b40 0x7f1cb00b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.464+0000 7f1cb4de4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0161530 con 0x7f1cb0155b40 2024-05-08T05:46:46.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.464+0000 7f1cb4de4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb0161250 con 0x7f1cb0155180 2024-05-08T05:46:46.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.464+0000 7f1cb4de4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb0161810 con 0x7f1c980039e0 2024-05-08T05:46:46.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.464+0000 7f1caed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cb0155b40 0x7f1cb00b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.464+0000 7f1caed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cb0155b40 0x7f1cb00b09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36718/0 (socket says 172.21.15.45:36718) 2024-05-08T05:46:46.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.464+0000 7f1caed9d640 1 -- 172.21.15.45:0/2505024298 learned_addr learned my addr 172.21.15.45:0/2505024298 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:46.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.465+0000 7f1cadd9b640 1 --2- 172.21.15.45:0/2505024298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb0155180 0x7f1cb0155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.465+0000 7f1caed9d640 1 -- 172.21.15.45:0/2505024298 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c980039e0 msgr2=0x7f1cb0154840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:46.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.465+0000 7f1caed9d640 1 --2- 172.21.15.45:0/2505024298 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c980039e0 0x7f1cb0154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.465+0000 7f1caed9d640 1 -- 172.21.15.45:0/2505024298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb0155180 msgr2=0x7f1cb0155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:46.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.465+0000 7f1caed9d640 1 --2- 172.21.15.45:0/2505024298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb0155180 0x7f1cb0155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.465+0000 7f1caed9d640 1 -- 172.21.15.45:0/2505024298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb0155fc0 con 0x7f1cb0155b40 2024-05-08T05:46:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.465+0000 7f1caed9d640 1 --2- 172.21.15.45:0/2505024298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cb0155b40 0x7f1cb00b09b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0160910 tx=0x7f1ca800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.465+0000 7f1c9effd640 1 -- 172.21.15.45:0/2505024298 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca800ee60 con 0x7f1cb0155b40 2024-05-08T05:46:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.465+0000 7f1c9effd640 1 -- 172.21.15.45:0/2505024298 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca8018070 con 0x7f1cb0155b40 2024-05-08T05:46:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.465+0000 7f1cb4de4640 1 -- 172.21.15.45:0/2505024298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb009cb30 con 0x7f1cb0155b40 2024-05-08T05:46:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.465+0000 7f1c9effd640 1 -- 172.21.15.45:0/2505024298 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8003dc0 con 0x7f1cb0155b40 2024-05-08T05:46:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.465+0000 7f1cb4de4640 1 -- 172.21.15.45:0/2505024298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb009d0f0 con 0x7f1cb0155b40 2024-05-08T05:46:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.466+0000 7f1cb4de4640 1 -- 172.21.15.45:0/2505024298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb00693f0 con 0x7f1cb0155b40 2024-05-08T05:46:46.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.467+0000 7f1c9effd640 1 -- 172.21.15.45:0/2505024298 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1ca80050d0 con 0x7f1cb0155b40 2024-05-08T05:46:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.467+0000 7f1c9effd640 1 --2- 172.21.15.45:0/2505024298 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1c900762d0 0x7f1c90078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.467+0000 7f1c9effd640 1 -- 172.21.15.45:0/2505024298 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f1ca8097640 con 0x7f1cb0155b40 2024-05-08T05:46:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.468+0000 7f1cae59c640 1 --2- 172.21.15.45:0/2505024298 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1c900762d0 0x7f1c90078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.468+0000 7f1cae59c640 1 --2- 172.21.15.45:0/2505024298 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1c900762d0 0x7f1c90078790 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009a90 tx=0x7f1ca002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:46.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.472+0000 7f1c9effd640 1 -- 172.21.15.45:0/2505024298 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1ca8060dc0 con 0x7f1cb0155b40 2024-05-08T05:46:46.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.702+0000 7f1cb4de4640 1 -- 172.21.15.45:0/2505024298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cb006e0d0 con 0x7f1cb0155b40 2024-05-08T05:46:46.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.702+0000 7f1c9effd640 1 -- 172.21.15.45:0/2505024298 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5185 (secure 0 0 0) 0x7f1ca80606c0 con 0x7f1cb0155b40 2024-05-08T05:46:46.703 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:46:46.703 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5722,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1283757699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1283757699},{"type":"v1","addr":"172.21.15.63:6837","nonce":1283757699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2823770504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2823770504},{"type":"v1","addr":"172.21.15.63:6836","nonce":2823770504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1963062796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1963062796},{"type":"v1","addr":"172.21.15.45:6834","nonce":1963062796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:45:24.208158+0000","modified":"2024-05-08T05:46:16.494095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":76,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":17,"addr":"172.21.15.45:6835/4183668301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4183668301},{"type":"v1","addr":"172.21.15.45:6835","nonce":4183668301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6171,"qdb_cluster":[6171]},"id":5}]} 2024-05-08T05:46:46.703 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 52 2024-05-08T05:46:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.704+0000 7f1cb4de4640 1 -- 172.21.15.45:0/2505024298 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1c900762d0 msgr2=0x7f1c90078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.704+0000 7f1cb4de4640 1 --2- 172.21.15.45:0/2505024298 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1c900762d0 0x7f1c90078790 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009a90 tx=0x7f1ca002e000 comp rx=0 tx=0).stop 2024-05-08T05:46:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.704+0000 7f1cb4de4640 1 -- 172.21.15.45:0/2505024298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cb0155b40 msgr2=0x7f1cb00b09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.704+0000 7f1cb4de4640 1 --2- 172.21.15.45:0/2505024298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cb0155b40 0x7f1cb00b09b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0160910 tx=0x7f1ca800ec10 comp rx=0 tx=0).stop 2024-05-08T05:46:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.704+0000 7f1cb4de4640 1 -- 172.21.15.45:0/2505024298 shutdown_connections 2024-05-08T05:46:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.704+0000 7f1cb4de4640 1 --2- 172.21.15.45:0/2505024298 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1c900762d0 0x7f1c90078790 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.704+0000 7f1cb4de4640 1 --2- 172.21.15.45:0/2505024298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cb0155b40 0x7f1cb00b09b0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.704+0000 7f1cb4de4640 1 --2- 172.21.15.45:0/2505024298 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb0155180 0x7f1cb0155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.704+0000 7f1cb4de4640 1 --2- 172.21.15.45:0/2505024298 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1c980039e0 0x7f1cb0154840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.704+0000 7f1cb4de4640 1 -- 172.21.15.45:0/2505024298 >> 172.21.15.45:0/2505024298 conn(0x7f1cb006c3a0 msgr2=0x7f1cb00cee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:46.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.705+0000 7f1cb4de4640 1 -- 172.21.15.45:0/2505024298 shutdown_connections 2024-05-08T05:46:46.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.705+0000 7f1cb4de4640 1 -- 172.21.15.45:0/2505024298 wait complete. 2024-05-08T05:46:46.716 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:46:46.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.819+0000 7feb87275640 1 Processor -- start 2024-05-08T05:46:46.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.819+0000 7feb87275640 1 -- start start 2024-05-08T05:46:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.820+0000 7feb87275640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7feb8015f100 0x7feb800d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.820+0000 7feb87275640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb800d5bc0 0x7feb800d3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.820+0000 7feb87275640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb800d2c80 con 0x7feb80162c60 2024-05-08T05:46:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.820+0000 7feb87275640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7feb800d35b0 con 0x7feb8015f100 2024-05-08T05:46:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.820+0000 7feb87275640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feb800d2f60 con 0x7feb800d5bc0 2024-05-08T05:46:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.820+0000 7feb85812640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7feb80162c60 0x7feb800b4970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47616/0 (socket says 172.21.15.45:47616) 2024-05-08T05:46:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.820+0000 7feb85812640 1 -- 172.21.15.45:0/2468498539 learned_addr learned my addr 172.21.15.45:0/2468498539 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.820+0000 7feb84810640 1 --2- 172.21.15.45:0/2468498539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb800d5bc0 0x7feb800d3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.820+0000 7feb77fff640 1 -- 172.21.15.45:0/2468498539 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1699725642 0 0) 0x7feb800d2c80 con 0x7feb80162c60 2024-05-08T05:46:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.820+0000 7feb85011640 1 --2- 172.21.15.45:0/2468498539 >> v2:172.21.15.63:3300/0 conn(0x7feb8015f100 0x7feb800d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.820+0000 7feb77fff640 1 -- 172.21.15.45:0/2468498539 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb6c003520 con 0x7feb80162c60 2024-05-08T05:46:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.820+0000 7feb77fff640 1 -- 172.21.15.45:0/2468498539 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1387452125 0 0) 0x7feb6c003520 con 0x7feb80162c60 2024-05-08T05:46:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.820+0000 7feb77fff640 1 -- 172.21.15.45:0/2468498539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb800d5bc0 msgr2=0x7feb800d3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.820+0000 7feb77fff640 1 --2- 172.21.15.45:0/2468498539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb800d5bc0 0x7feb800d3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.820+0000 7feb77fff640 1 -- 172.21.15.45:0/2468498539 >> v2:172.21.15.63:3300/0 conn(0x7feb8015f100 msgr2=0x7feb800d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.820+0000 7feb77fff640 1 --2- 172.21.15.45:0/2468498539 >> v2:172.21.15.63:3300/0 conn(0x7feb8015f100 0x7feb800d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.820+0000 7feb77fff640 1 -- 172.21.15.45:0/2468498539 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb800d3240 con 0x7feb80162c60 2024-05-08T05:46:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.820+0000 7feb77fff640 1 -- 172.21.15.45:0/2468498539 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7feb78001a70 con 0x7feb80162c60 2024-05-08T05:46:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.820+0000 7feb77fff640 1 -- 172.21.15.45:0/2468498539 >> v1:172.21.15.45:6789/0 conn(0x7feb80162c60 legacy=0x7feb800b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.821+0000 7feb77fff640 1 --2- 172.21.15.45:0/2468498539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb800d5bc0 0x7feb6c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.821+0000 7feb77fff640 1 --2- 172.21.15.45:0/2468498539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb6c003fd0 0x7feb6c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.821+0000 7feb77fff640 1 --2- 172.21.15.45:0/2468498539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb6c004d40 0x7feb6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.821+0000 7feb77fff640 1 -- 172.21.15.45:0/2468498539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feb800d35b0 con 0x7feb6c004d40 2024-05-08T05:46:46.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.821+0000 7feb77fff640 1 -- 172.21.15.45:0/2468498539 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feb800d2f60 con 0x7feb6c003fd0 2024-05-08T05:46:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.821+0000 7feb77fff640 1 -- 172.21.15.45:0/2468498539 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feb6c00baa0 con 0x7feb800d5bc0 2024-05-08T05:46:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.821+0000 7feb85812640 1 --2- 172.21.15.45:0/2468498539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb6c004d40 0x7feb6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.821+0000 7feb84810640 1 --2- 172.21.15.45:0/2468498539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb6c003fd0 0x7feb6c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.821+0000 7feb85011640 1 --2- 172.21.15.45:0/2468498539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb800d5bc0 0x7feb6c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.821+0000 7feb85812640 1 -- 172.21.15.45:0/2468498539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb800d5bc0 msgr2=0x7feb6c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.821+0000 7feb85812640 1 --2- 172.21.15.45:0/2468498539 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb800d5bc0 0x7feb6c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.821+0000 7feb85812640 1 -- 172.21.15.45:0/2468498539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb6c003fd0 msgr2=0x7feb6c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.821+0000 7feb85812640 1 --2- 172.21.15.45:0/2468498539 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb6c003fd0 0x7feb6c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.821+0000 7feb85812640 1 -- 172.21.15.45:0/2468498539 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c003520 con 0x7feb6c004d40 2024-05-08T05:46:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.822+0000 7feb85812640 1 --2- 172.21.15.45:0/2468498539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb6c004d40 0x7feb6c0051c0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7feb7800a850 tx=0x7feb6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be442a6f616e2316 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.822+0000 7feb77fff640 1 -- 172.21.15.45:0/2468498539 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7800a4a0 con 0x7feb6c004d40 2024-05-08T05:46:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.822+0000 7feb77fff640 1 -- 172.21.15.45:0/2468498539 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb7802f070 con 0x7feb6c004d40 2024-05-08T05:46:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.822+0000 7feb77fff640 1 -- 172.21.15.45:0/2468498539 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7802bcc0 con 0x7feb6c004d40 2024-05-08T05:46:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.822+0000 7feb87275640 1 -- 172.21.15.45:0/2468498539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb6c004d40 msgr2=0x7feb6c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:46.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.822+0000 7feb87275640 1 --2- 172.21.15.45:0/2468498539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb6c004d40 0x7feb6c0051c0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7feb7800a850 tx=0x7feb6c005830 comp rx=0 tx=0).stop 2024-05-08T05:46:46.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.822+0000 7feb85812640 1 -- 172.21.15.45:0/2468498539 reap_dead start 2024-05-08T05:46:46.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.822+0000 7feb87275640 1 -- 172.21.15.45:0/2468498539 shutdown_connections 2024-05-08T05:46:46.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.822+0000 7feb87275640 1 -- 172.21.15.45:0/2468498539 >> 172.21.15.45:0/2468498539 conn(0x7feb8009e3e0 msgr2=0x7feb800d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:46.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.823+0000 7feb87275640 1 -- 172.21.15.45:0/2468498539 shutdown_connections 2024-05-08T05:46:46.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.823+0000 7feb87275640 1 -- 172.21.15.45:0/2468498539 wait complete. 2024-05-08T05:46:46.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.824+0000 7feb87275640 1 Processor -- start 2024-05-08T05:46:46.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.824+0000 7feb87275640 1 -- start start 2024-05-08T05:46:46.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.825+0000 7feb87275640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb6c004d40 0x7feb800ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.825+0000 7feb87275640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb800b3fb0 0x7feb801f8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.825+0000 7feb87275640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb800ce4c0 0x7feb800ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.825+0000 7feb87275640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feb8015f4e0 con 0x7feb800ce4c0 2024-05-08T05:46:46.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.825+0000 7feb87275640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7feb800d0d20 con 0x7feb800b3fb0 2024-05-08T05:46:46.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.825+0000 7feb87275640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7feb800b5080 con 0x7feb6c004d40 2024-05-08T05:46:46.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.825+0000 7feb85812640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb800ce4c0 0x7feb800ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.825+0000 7feb85812640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb800ce4c0 0x7feb800ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36740/0 (socket says 172.21.15.45:36740) 2024-05-08T05:46:46.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.825+0000 7feb85812640 1 -- 172.21.15.45:0/2980391948 learned_addr learned my addr 172.21.15.45:0/2980391948 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:46.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.825+0000 7feb84810640 1 --2- 172.21.15.45:0/2980391948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb800b3fb0 0x7feb801f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.825+0000 7feb85011640 1 --2- 172.21.15.45:0/2980391948 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb6c004d40 0x7feb800ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.825+0000 7feb85812640 1 -- 172.21.15.45:0/2980391948 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb6c004d40 msgr2=0x7feb800ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:46.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.825+0000 7feb85812640 1 --2- 172.21.15.45:0/2980391948 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb6c004d40 0x7feb800ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.825+0000 7feb85812640 1 -- 172.21.15.45:0/2980391948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb800b3fb0 msgr2=0x7feb801f8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:46.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.825+0000 7feb85812640 1 --2- 172.21.15.45:0/2980391948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb800b3fb0 0x7feb801f8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:46.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.825+0000 7feb85812640 1 -- 172.21.15.45:0/2980391948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb800cc850 con 0x7feb800ce4c0 2024-05-08T05:46:46.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.825+0000 7feb85812640 1 --2- 172.21.15.45:0/2980391948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb800ce4c0 0x7feb800ce940 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7feb7802b510 tx=0x7feb800cf560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:46.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.825+0000 7feb75ffb640 1 -- 172.21.15.45:0/2980391948 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb78044070 con 0x7feb800ce4c0 2024-05-08T05:46:46.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.826+0000 7feb75ffb640 1 -- 172.21.15.45:0/2980391948 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb7800a480 con 0x7feb800ce4c0 2024-05-08T05:46:46.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.826+0000 7feb87275640 1 -- 172.21.15.45:0/2980391948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb801f59a0 con 0x7feb800ce4c0 2024-05-08T05:46:46.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.826+0000 7feb75ffb640 1 -- 172.21.15.45:0/2980391948 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb78001fe0 con 0x7feb800ce4c0 2024-05-08T05:46:46.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.826+0000 7feb87275640 1 -- 172.21.15.45:0/2980391948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb801f5f60 con 0x7feb800ce4c0 2024-05-08T05:46:46.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.827+0000 7feb87275640 1 -- 172.21.15.45:0/2980391948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb800693f0 con 0x7feb800ce4c0 2024-05-08T05:46:46.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.827+0000 7feb75ffb640 1 -- 172.21.15.45:0/2980391948 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7feb78047070 con 0x7feb800ce4c0 2024-05-08T05:46:46.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.828+0000 7feb75ffb640 1 --2- 172.21.15.45:0/2980391948 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feb64076300 0x7feb640787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:46.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.828+0000 7feb75ffb640 1 -- 172.21.15.45:0/2980391948 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7feb780b77c0 con 0x7feb800ce4c0 2024-05-08T05:46:46.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.828+0000 7feb85011640 1 --2- 172.21.15.45:0/2980391948 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feb64076300 0x7feb640787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:46.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.829+0000 7feb85011640 1 --2- 172.21.15.45:0/2980391948 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feb64076300 0x7feb640787c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7feb800cbe40 tx=0x7feb7c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:46.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:46.832+0000 7feb75ffb640 1 -- 172.21.15.45:0/2980391948 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7feb78080910 con 0x7feb800ce4c0 2024-05-08T05:46:47.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.024+0000 7feb87275640 1 -- 172.21.15.45:0/2980391948 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb8015d090 con 0x7feb800ce4c0 2024-05-08T05:46:47.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.025+0000 7feb75ffb640 1 -- 172.21.15.45:0/2980391948 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+17024 (secure 0 0 0) 0x7feb78084c10 con 0x7feb800ce4c0 2024-05-08T05:46:47.025 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:46:47.025 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":76,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:45:33.480680+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:45:21.867649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-08T05:45:23.164376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6836/133050464":"2024-05-09T05:45:33.480648+0000","172.21.15.63:6834/133050464":"2024-05-09T05:45:33.480648+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:46:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.027+0000 7feb87275640 1 -- 172.21.15.45:0/2980391948 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feb64076300 msgr2=0x7feb640787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.027+0000 7feb87275640 1 --2- 172.21.15.45:0/2980391948 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feb64076300 0x7feb640787c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7feb800cbe40 tx=0x7feb7c009190 comp rx=0 tx=0).stop 2024-05-08T05:46:47.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.027+0000 7feb87275640 1 -- 172.21.15.45:0/2980391948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb800ce4c0 msgr2=0x7feb800ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:47.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.027+0000 7feb87275640 1 --2- 172.21.15.45:0/2980391948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb800ce4c0 0x7feb800ce940 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7feb7802b510 tx=0x7feb800cf560 comp rx=0 tx=0).stop 2024-05-08T05:46:47.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.027+0000 7feb87275640 1 -- 172.21.15.45:0/2980391948 shutdown_connections 2024-05-08T05:46:47.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.027+0000 7feb87275640 1 --2- 172.21.15.45:0/2980391948 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feb64076300 0x7feb640787c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.027+0000 7feb87275640 1 --2- 172.21.15.45:0/2980391948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb800ce4c0 0x7feb800ce940 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.027+0000 7feb87275640 1 --2- 172.21.15.45:0/2980391948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7feb800b3fb0 0x7feb801f8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.027+0000 7feb87275640 1 --2- 172.21.15.45:0/2980391948 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7feb6c004d40 0x7feb800ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.027+0000 7feb87275640 1 -- 172.21.15.45:0/2980391948 >> 172.21.15.45:0/2980391948 conn(0x7feb8009e3e0 msgr2=0x7feb8015e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:47.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.027+0000 7feb87275640 1 -- 172.21.15.45:0/2980391948 shutdown_connections 2024-05-08T05:46:47.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.027+0000 7feb87275640 1 -- 172.21.15.45:0/2980391948 wait complete. 2024-05-08T05:46:47.038 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T05:46:47.039 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:46:47.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.143+0000 7f322eb14640 1 Processor -- start 2024-05-08T05:46:47.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.143+0000 7f322eb14640 1 -- start start 2024-05-08T05:46:47.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.143+0000 7f322eb14640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f322806d620 0x7f3228073800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.144+0000 7f322eb14640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f322806da00 0x7f32280af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.144+0000 7f322eb14640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f32280b5ee0 0x7f322806e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.144+0000 7f322eb14640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f32280afaa0 con 0x7f322806da00 2024-05-08T05:46:47.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.144+0000 7f322eb14640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f32280b5a50 con 0x7f32280b5ee0 2024-05-08T05:46:47.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.144+0000 7f322eb14640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f32280af800 con 0x7f322806d620 2024-05-08T05:46:47.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.144+0000 7f3227fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f322806da00 0x7f32280af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.144+0000 7f3227fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f322806da00 0x7f32280af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36752/0 (socket says 172.21.15.45:36752) 2024-05-08T05:46:47.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.144+0000 7f3227fff640 1 -- 172.21.15.45:0/1209558274 learned_addr learned my addr 172.21.15.45:0/1209558274 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:47.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.144+0000 7f3227fff640 1 -- 172.21.15.45:0/1209558274 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f322806d620 msgr2=0x7f3228073800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.144+0000 7f322d0b1640 1 --2- 172.21.15.45:0/1209558274 >> v2:172.21.15.63:3300/0 conn(0x7f32280b5ee0 0x7f322806e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.144+0000 7f322c8b0640 1 --2- 172.21.15.45:0/1209558274 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f322806d620 0x7f3228073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.144+0000 7f3227fff640 1 --2- 172.21.15.45:0/1209558274 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f322806d620 0x7f3228073800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.144+0000 7f3227fff640 1 -- 172.21.15.45:0/1209558274 >> v2:172.21.15.63:3300/0 conn(0x7f32280b5ee0 msgr2=0x7f322806e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.144+0000 7f3227fff640 1 --2- 172.21.15.45:0/1209558274 >> v2:172.21.15.63:3300/0 conn(0x7f32280b5ee0 0x7f322806e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.144+0000 7f3227fff640 1 -- 172.21.15.45:0/1209558274 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32280b5120 con 0x7f322806da00 2024-05-08T05:46:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.144+0000 7f322d0b1640 1 --2- 172.21.15.45:0/1209558274 >> v2:172.21.15.63:3300/0 conn(0x7f32280b5ee0 0x7f322806e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:46:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.144+0000 7f322c8b0640 1 --2- 172.21.15.45:0/1209558274 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f322806d620 0x7f3228073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:46:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.145+0000 7f3227fff640 1 --2- 172.21.15.45:0/1209558274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f322806da00 0x7f32280af2c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f3228072850 tx=0x7f321002a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dafc37fc5d9c593 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.145+0000 7f32277fe640 1 -- 172.21.15.45:0/1209558274 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210036070 con 0x7f322806da00 2024-05-08T05:46:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.145+0000 7f32277fe640 1 -- 172.21.15.45:0/1209558274 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3210031600 con 0x7f322806da00 2024-05-08T05:46:47.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.145+0000 7f32277fe640 1 -- 172.21.15.45:0/1209558274 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32100319d0 con 0x7f322806da00 2024-05-08T05:46:47.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.145+0000 7f322eb14640 1 -- 172.21.15.45:0/1209558274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f322806da00 msgr2=0x7f32280af2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:47.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.145+0000 7f322eb14640 1 --2- 172.21.15.45:0/1209558274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f322806da00 0x7f32280af2c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f3228072850 tx=0x7f321002a570 comp rx=0 tx=0).stop 2024-05-08T05:46:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.146+0000 7f322eb14640 1 -- 172.21.15.45:0/1209558274 shutdown_connections 2024-05-08T05:46:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.146+0000 7f322eb14640 1 --2- 172.21.15.45:0/1209558274 >> v2:172.21.15.63:3300/0 conn(0x7f32280b5ee0 0x7f322806e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.146+0000 7f322eb14640 1 --2- 172.21.15.45:0/1209558274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f322806da00 0x7f32280af2c0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.146+0000 7f322eb14640 1 --2- 172.21.15.45:0/1209558274 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f322806d620 0x7f3228073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.146+0000 7f322eb14640 1 -- 172.21.15.45:0/1209558274 >> 172.21.15.45:0/1209558274 conn(0x7f32280728c0 msgr2=0x7f3228072cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.146+0000 7f322eb14640 1 -- 172.21.15.45:0/1209558274 shutdown_connections 2024-05-08T05:46:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.146+0000 7f322eb14640 1 -- 172.21.15.45:0/1209558274 wait complete. 2024-05-08T05:46:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.147+0000 7f322eb14640 1 Processor -- start 2024-05-08T05:46:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.147+0000 7f322eb14640 1 -- start start 2024-05-08T05:46:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f322eb14640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f322806d620 0x7f32281f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f322eb14640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f322806da00 0x7f322815d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f322eb14640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32280b5ee0 0x7f32281586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f322eb14640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f32280b6370 con 0x7f322806d620 2024-05-08T05:46:47.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f322eb14640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f32280b56e0 con 0x7f32280b5ee0 2024-05-08T05:46:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f322eb14640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f32280b64f0 con 0x7f322806da00 2024-05-08T05:46:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f322c8b0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f322806d620 0x7f32281f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f322c8b0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f322806d620 0x7f32281f5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36768/0 (socket says 172.21.15.45:36768) 2024-05-08T05:46:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f322c8b0640 1 -- 172.21.15.45:0/2158990842 learned_addr learned my addr 172.21.15.45:0/2158990842 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f322d0b1640 1 --2- 172.21.15.45:0/2158990842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32280b5ee0 0x7f32281586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f3227fff640 1 --2- 172.21.15.45:0/2158990842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f322806da00 0x7f322815d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f322c8b0640 1 -- 172.21.15.45:0/2158990842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f322806da00 msgr2=0x7f322815d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f322c8b0640 1 --2- 172.21.15.45:0/2158990842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f322806da00 0x7f322815d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f322c8b0640 1 -- 172.21.15.45:0/2158990842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32280b5ee0 msgr2=0x7f32281586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f322c8b0640 1 --2- 172.21.15.45:0/2158990842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32280b5ee0 0x7f32281586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f322c8b0640 1 -- 172.21.15.45:0/2158990842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3228153330 con 0x7f322806d620 2024-05-08T05:46:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f322d0b1640 1 --2- 172.21.15.45:0/2158990842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32280b5ee0 0x7f32281586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:46:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f3227fff640 1 --2- 172.21.15.45:0/2158990842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f322806da00 0x7f322815d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:46:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f322c8b0640 1 --2- 172.21.15.45:0/2158990842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f322806d620 0x7f32281f5790 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f321800c7f0 tx=0x7f321800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:47.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f32257fa640 1 -- 172.21.15.45:0/2158990842 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218015070 con 0x7f322806d620 2024-05-08T05:46:47.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f32257fa640 1 -- 172.21.15.45:0/2158990842 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3218007820 con 0x7f322806d620 2024-05-08T05:46:47.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f322eb14640 1 -- 172.21.15.45:0/2158990842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3228161170 con 0x7f322806d620 2024-05-08T05:46:47.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.148+0000 7f32257fa640 1 -- 172.21.15.45:0/2158990842 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218007c10 con 0x7f322806d620 2024-05-08T05:46:47.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.149+0000 7f322eb14640 1 -- 172.21.15.45:0/2158990842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3228161730 con 0x7f322806d620 2024-05-08T05:46:47.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.149+0000 7f322eb14640 1 -- 172.21.15.45:0/2158990842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32280693f0 con 0x7f322806d620 2024-05-08T05:46:47.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.150+0000 7f32257fa640 1 -- 172.21.15.45:0/2158990842 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3218018070 con 0x7f322806d620 2024-05-08T05:46:47.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.151+0000 7f32257fa640 1 --2- 172.21.15.45:0/2158990842 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f32080762d0 0x7f3208078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.151+0000 7f32257fa640 1 -- 172.21.15.45:0/2158990842 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f32180976c0 con 0x7f322806d620 2024-05-08T05:46:47.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.151+0000 7f3227fff640 1 --2- 172.21.15.45:0/2158990842 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f32080762d0 0x7f3208078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.152+0000 7f3227fff640 1 --2- 172.21.15.45:0/2158990842 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f32080762d0 0x7f3208078790 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3210033000 tx=0x7f3210009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:47.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.155+0000 7f32257fa640 1 -- 172.21.15.45:0/2158990842 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3218060dc0 con 0x7f322806d620 2024-05-08T05:46:47.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.384+0000 7f322eb14640 1 -- 172.21.15.45:0/2158990842 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f322815ce10 con 0x7f322806d620 2024-05-08T05:46:47.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.384+0000 7f32257fa640 1 -- 172.21.15.45:0/2158990842 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5185 (secure 0 0 0) 0x7f32180606c0 con 0x7f322806d620 2024-05-08T05:46:47.385 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:46:47.385 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5722,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1283757699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1283757699},{"type":"v1","addr":"172.21.15.63:6837","nonce":1283757699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2823770504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2823770504},{"type":"v1","addr":"172.21.15.63:6836","nonce":2823770504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1963062796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1963062796},{"type":"v1","addr":"172.21.15.45:6834","nonce":1963062796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:45:24.208158+0000","modified":"2024-05-08T05:46:16.494095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":76,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":17,"addr":"172.21.15.45:6835/4183668301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4183668301},{"type":"v1","addr":"172.21.15.45:6835","nonce":4183668301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6171,"qdb_cluster":[6171]},"id":5}]} 2024-05-08T05:46:47.386 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 52 2024-05-08T05:46:47.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.386+0000 7f322eb14640 1 -- 172.21.15.45:0/2158990842 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f32080762d0 msgr2=0x7f3208078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:47.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.386+0000 7f322eb14640 1 --2- 172.21.15.45:0/2158990842 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f32080762d0 0x7f3208078790 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3210033000 tx=0x7f3210009000 comp rx=0 tx=0).stop 2024-05-08T05:46:47.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.386+0000 7f322eb14640 1 -- 172.21.15.45:0/2158990842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f322806d620 msgr2=0x7f32281f5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:47.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.386+0000 7f322eb14640 1 --2- 172.21.15.45:0/2158990842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f322806d620 0x7f32281f5790 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f321800c7f0 tx=0x7f321800ccb0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.386+0000 7f322eb14640 1 -- 172.21.15.45:0/2158990842 shutdown_connections 2024-05-08T05:46:47.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.386+0000 7f322eb14640 1 --2- 172.21.15.45:0/2158990842 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f32080762d0 0x7f3208078790 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.386+0000 7f322eb14640 1 --2- 172.21.15.45:0/2158990842 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f322806da00 0x7f322815d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.386+0000 7f322eb14640 1 --2- 172.21.15.45:0/2158990842 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32280b5ee0 0x7f32281586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.386+0000 7f322eb14640 1 --2- 172.21.15.45:0/2158990842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f322806d620 0x7f32281f5790 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.386+0000 7f322eb14640 1 -- 172.21.15.45:0/2158990842 >> 172.21.15.45:0/2158990842 conn(0x7f32280728c0 msgr2=0x7f32280b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:47.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.387+0000 7f322eb14640 1 -- 172.21.15.45:0/2158990842 shutdown_connections 2024-05-08T05:46:47.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.387+0000 7f322eb14640 1 -- 172.21.15.45:0/2158990842 wait complete. 2024-05-08T05:46:47.398 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:46:47.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.502+0000 7fbd75124640 1 Processor -- start 2024-05-08T05:46:47.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.502+0000 7fbd75124640 1 -- start start 2024-05-08T05:46:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.503+0000 7fbd75124640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd701629a0 0x7fbd700d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.503+0000 7fbd75124640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbd700d0cb0 0x7fbd700d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.503+0000 7fbd75124640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd700d3330 con 0x7fbd700d2dc0 2024-05-08T05:46:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.503+0000 7fbd75124640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbd700d2170 con 0x7fbd700d0cb0 2024-05-08T05:46:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.503+0000 7fbd75124640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbd700d2450 con 0x7fbd701629a0 2024-05-08T05:46:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.503+0000 7fbd6e59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbd700d2dc0 0x7fbd700c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47626/0 (socket says 172.21.15.45:47626) 2024-05-08T05:46:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.503+0000 7fbd6e59c640 1 -- 172.21.15.45:0/4061246493 learned_addr learned my addr 172.21.15.45:0/4061246493 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.503+0000 7fbd6ed9d640 1 --2- 172.21.15.45:0/4061246493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd701629a0 0x7fbd700d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.503+0000 7fbd6f59e640 1 --2- 172.21.15.45:0/4061246493 >> v2:172.21.15.63:3300/0 conn(0x7fbd700d0cb0 0x7fbd700d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.503+0000 7fbd6dd9b640 1 -- 172.21.15.45:0/4061246493 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2928313635 0 0) 0x7fbd700d3330 con 0x7fbd700d2dc0 2024-05-08T05:46:47.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.503+0000 7fbd6dd9b640 1 -- 172.21.15.45:0/4061246493 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd58003520 con 0x7fbd700d2dc0 2024-05-08T05:46:47.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.504+0000 7fbd6ed9d640 1 -- 172.21.15.45:0/4061246493 >> v2:172.21.15.63:3300/0 conn(0x7fbd700d0cb0 msgr2=0x7fbd700d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:47.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.504+0000 7fbd6ed9d640 1 --2- 172.21.15.45:0/4061246493 >> v2:172.21.15.63:3300/0 conn(0x7fbd700d0cb0 0x7fbd700d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.504+0000 7fbd6ed9d640 1 -- 172.21.15.45:0/4061246493 >> v1:172.21.15.45:6789/0 conn(0x7fbd700d2dc0 legacy=0x7fbd700c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:47.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.504+0000 7fbd6ed9d640 1 -- 172.21.15.45:0/4061246493 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd700d2730 con 0x7fbd701629a0 2024-05-08T05:46:47.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.504+0000 7fbd6ed9d640 1 --2- 172.21.15.45:0/4061246493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd701629a0 0x7fbd700d35c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009d90 tx=0x7fbd6802a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70cbbab8db74af5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:47.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.504+0000 7fbd6dd9b640 1 -- 172.21.15.45:0/4061246493 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68007740 con 0x7fbd701629a0 2024-05-08T05:46:47.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.505+0000 7fbd6dd9b640 1 -- 172.21.15.45:0/4061246493 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd680078e0 con 0x7fbd701629a0 2024-05-08T05:46:47.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.505+0000 7fbd6dd9b640 1 -- 172.21.15.45:0/4061246493 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68007c90 con 0x7fbd701629a0 2024-05-08T05:46:47.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.505+0000 7fbd75124640 1 -- 172.21.15.45:0/4061246493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd701629a0 msgr2=0x7fbd700d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:47.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.505+0000 7fbd75124640 1 --2- 172.21.15.45:0/4061246493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd701629a0 0x7fbd700d35c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009d90 tx=0x7fbd6802a530 comp rx=0 tx=0).stop 2024-05-08T05:46:47.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.505+0000 7fbd75124640 1 -- 172.21.15.45:0/4061246493 shutdown_connections 2024-05-08T05:46:47.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.505+0000 7fbd75124640 1 --2- 172.21.15.45:0/4061246493 >> v2:172.21.15.63:3300/0 conn(0x7fbd700d0cb0 0x7fbd700d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.505+0000 7fbd75124640 1 --2- 172.21.15.45:0/4061246493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd701629a0 0x7fbd700d35c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.505+0000 7fbd75124640 1 -- 172.21.15.45:0/4061246493 >> 172.21.15.45:0/4061246493 conn(0x7fbd700d4fe0 msgr2=0x7fbd700d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:47.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.505+0000 7fbd75124640 1 -- 172.21.15.45:0/4061246493 shutdown_connections 2024-05-08T05:46:47.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.505+0000 7fbd75124640 1 -- 172.21.15.45:0/4061246493 wait complete. 2024-05-08T05:46:47.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.507+0000 7fbd75124640 1 Processor -- start 2024-05-08T05:46:47.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.507+0000 7fbd75124640 1 -- start start 2024-05-08T05:46:47.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.507+0000 7fbd75124640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd700d0cb0 0x7fbd700ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.507+0000 7fbd75124640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd700d2dc0 0x7fbd700cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.507+0000 7fbd75124640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd701629a0 0x7fbd7015b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.507+0000 7fbd75124640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbd700cf440 con 0x7fbd701629a0 2024-05-08T05:46:47.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.507+0000 7fbd75124640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbd700cf2c0 con 0x7fbd700d2dc0 2024-05-08T05:46:47.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.507+0000 7fbd75124640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbd700c95e0 con 0x7fbd700d0cb0 2024-05-08T05:46:47.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.507+0000 7fbd6f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd701629a0 0x7fbd7015b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.508+0000 7fbd6f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd701629a0 0x7fbd7015b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36784/0 (socket says 172.21.15.45:36784) 2024-05-08T05:46:47.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.508+0000 7fbd6f59e640 1 -- 172.21.15.45:0/3939113080 learned_addr learned my addr 172.21.15.45:0/3939113080 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:47.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.508+0000 7fbd6ed9d640 1 --2- 172.21.15.45:0/3939113080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd700d0cb0 0x7fbd700ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.508+0000 7fbd6e59c640 1 --2- 172.21.15.45:0/3939113080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd700d2dc0 0x7fbd700cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.508+0000 7fbd6f59e640 1 -- 172.21.15.45:0/3939113080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd700d0cb0 msgr2=0x7fbd700ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:47.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.508+0000 7fbd6f59e640 1 --2- 172.21.15.45:0/3939113080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd700d0cb0 0x7fbd700ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.508+0000 7fbd6f59e640 1 -- 172.21.15.45:0/3939113080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd700d2dc0 msgr2=0x7fbd700cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:47.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.508+0000 7fbd6f59e640 1 --2- 172.21.15.45:0/3939113080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd700d2dc0 0x7fbd700cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.508+0000 7fbd6f59e640 1 -- 172.21.15.45:0/3939113080 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd700ced10 con 0x7fbd701629a0 2024-05-08T05:46:47.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.508+0000 7fbd6e59c640 1 --2- 172.21.15.45:0/3939113080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd700d2dc0 0x7fbd700cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:46:47.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.508+0000 7fbd6ed9d640 1 --2- 172.21.15.45:0/3939113080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd700d0cb0 0x7fbd700ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:46:47.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.508+0000 7fbd6f59e640 1 --2- 172.21.15.45:0/3939113080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd701629a0 0x7fbd7015b900 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000e960 tx=0x7fbd6000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:47.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.508+0000 7fbd577fe640 1 -- 172.21.15.45:0/3939113080 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd60017070 con 0x7fbd701629a0 2024-05-08T05:46:47.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.508+0000 7fbd577fe640 1 -- 172.21.15.45:0/3939113080 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd6000c980 con 0x7fbd701629a0 2024-05-08T05:46:47.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.508+0000 7fbd75124640 1 -- 172.21.15.45:0/3939113080 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd700ceff0 con 0x7fbd701629a0 2024-05-08T05:46:47.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.508+0000 7fbd577fe640 1 -- 172.21.15.45:0/3939113080 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6000cd70 con 0x7fbd701629a0 2024-05-08T05:46:47.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.508+0000 7fbd75124640 1 -- 172.21.15.45:0/3939113080 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd700cc430 con 0x7fbd701629a0 2024-05-08T05:46:47.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.509+0000 7fbd75124640 1 -- 172.21.15.45:0/3939113080 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd700693f0 con 0x7fbd701629a0 2024-05-08T05:46:47.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.510+0000 7fbd577fe640 1 -- 172.21.15.45:0/3939113080 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbd6001a070 con 0x7fbd701629a0 2024-05-08T05:46:47.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.510+0000 7fbd577fe640 1 --2- 172.21.15.45:0/3939113080 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbd5c0762d0 0x7fbd5c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.510+0000 7fbd577fe640 1 -- 172.21.15.45:0/3939113080 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7fbd60097b70 con 0x7fbd701629a0 2024-05-08T05:46:47.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.511+0000 7fbd6ed9d640 1 --2- 172.21.15.45:0/3939113080 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbd5c0762d0 0x7fbd5c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.511+0000 7fbd6ed9d640 1 --2- 172.21.15.45:0/3939113080 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbd5c0762d0 0x7fbd5c078790 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbd68034000 tx=0x7fbd68033040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:47.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.515+0000 7fbd577fe640 1 -- 172.21.15.45:0/3939113080 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbd600612f0 con 0x7fbd701629a0 2024-05-08T05:46:47.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.751+0000 7fbd75124640 1 -- 172.21.15.45:0/3939113080 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd700b6b00 con 0x7fbd701629a0 2024-05-08T05:46:47.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.751+0000 7fbd577fe640 1 -- 172.21.15.45:0/3939113080 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5185 (secure 0 0 0) 0x7fbd60060bf0 con 0x7fbd701629a0 2024-05-08T05:46:47.751 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:46:47.752 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5722,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1283757699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1283757699},{"type":"v1","addr":"172.21.15.63:6837","nonce":1283757699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2823770504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2823770504},{"type":"v1","addr":"172.21.15.63:6836","nonce":2823770504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1963062796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1963062796},{"type":"v1","addr":"172.21.15.45:6834","nonce":1963062796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:45:24.208158+0000","modified":"2024-05-08T05:46:16.494095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":76,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":47,"state":"up:active","state_seq":17,"addr":"172.21.15.45:6835/4183668301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4183668301},{"type":"v1","addr":"172.21.15.45:6835","nonce":4183668301}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6171,"qdb_cluster":[6171]},"id":5}]} 2024-05-08T05:46:47.752 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 52 2024-05-08T05:46:47.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.753+0000 7fbd75124640 1 -- 172.21.15.45:0/3939113080 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbd5c0762d0 msgr2=0x7fbd5c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:47.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.753+0000 7fbd75124640 1 --2- 172.21.15.45:0/3939113080 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbd5c0762d0 0x7fbd5c078790 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbd68034000 tx=0x7fbd68033040 comp rx=0 tx=0).stop 2024-05-08T05:46:47.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.753+0000 7fbd75124640 1 -- 172.21.15.45:0/3939113080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd701629a0 msgr2=0x7fbd7015b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:47.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.753+0000 7fbd75124640 1 --2- 172.21.15.45:0/3939113080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd701629a0 0x7fbd7015b900 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000e960 tx=0x7fbd6000ee20 comp rx=0 tx=0).stop 2024-05-08T05:46:47.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.753+0000 7fbd75124640 1 -- 172.21.15.45:0/3939113080 shutdown_connections 2024-05-08T05:46:47.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.753+0000 7fbd75124640 1 --2- 172.21.15.45:0/3939113080 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbd5c0762d0 0x7fbd5c078790 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.753+0000 7fbd75124640 1 --2- 172.21.15.45:0/3939113080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd701629a0 0x7fbd7015b900 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.753+0000 7fbd75124640 1 --2- 172.21.15.45:0/3939113080 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd700d2dc0 0x7fbd700cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.753+0000 7fbd75124640 1 --2- 172.21.15.45:0/3939113080 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd700d0cb0 0x7fbd700ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.753+0000 7fbd75124640 1 -- 172.21.15.45:0/3939113080 >> 172.21.15.45:0/3939113080 conn(0x7fbd700d4fe0 msgr2=0x7fbd7006dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:47.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.753+0000 7fbd75124640 1 -- 172.21.15.45:0/3939113080 shutdown_connections 2024-05-08T05:46:47.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.753+0000 7fbd75124640 1 -- 172.21.15.45:0/3939113080 wait complete. 2024-05-08T05:46:47.764 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:46:47.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.868+0000 7fa1d7cf8640 1 Processor -- start 2024-05-08T05:46:47.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.868+0000 7fa1d7cf8640 1 -- start start 2024-05-08T05:46:47.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.869+0000 7fa1d7cf8640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa1d01629b0 0x7fa1d006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.869+0000 7fa1d7cf8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1d0070d20 0x7fa1d00711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.869+0000 7fa1d7cf8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1d006e110 con 0x7fa1d00b2490 2024-05-08T05:46:47.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.869+0000 7fa1d7cf8640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa1d0070a40 con 0x7fa1d01629b0 2024-05-08T05:46:47.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.869+0000 7fa1d7cf8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d006dec0 con 0x7fa1d0070d20 2024-05-08T05:46:47.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.869+0000 7fa1d5293640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fa1d00b2490 0x7fa1d0070330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:44768/0 (socket says 172.21.15.45:44768) 2024-05-08T05:46:47.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.869+0000 7fa1d5293640 1 -- 172.21.15.45:0/560976446 learned_addr learned my addr 172.21.15.45:0/560976446 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:47.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.869+0000 7fa1d6295640 1 --2- 172.21.15.45:0/560976446 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1d0070d20 0x7fa1d00711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.869+0000 7fa1d5a94640 1 --2- 172.21.15.45:0/560976446 >> v2:172.21.15.63:3300/0 conn(0x7fa1d01629b0 0x7fa1d006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.869+0000 7fa1d4a92640 1 -- 172.21.15.45:0/560976446 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2118845137 0 0) 0x7fa1d006e110 con 0x7fa1d00b2490 2024-05-08T05:46:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.869+0000 7fa1d4a92640 1 -- 172.21.15.45:0/560976446 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1bc003520 con 0x7fa1d00b2490 2024-05-08T05:46:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.869+0000 7fa1d5a94640 1 -- 172.21.15.45:0/560976446 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1d0070d20 msgr2=0x7fa1d00711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.869+0000 7fa1d5a94640 1 --2- 172.21.15.45:0/560976446 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1d0070d20 0x7fa1d00711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.869+0000 7fa1d5a94640 1 -- 172.21.15.45:0/560976446 >> v1:172.21.15.63:6789/0 conn(0x7fa1d00b2490 legacy=0x7fa1d0070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.869+0000 7fa1d5a94640 1 -- 172.21.15.45:0/560976446 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d006e3f0 con 0x7fa1d01629b0 2024-05-08T05:46:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.869+0000 7fa1d6295640 1 --2- 172.21.15.45:0/560976446 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1d0070d20 0x7fa1d00711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:46:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.870+0000 7fa1d5a94640 1 --2- 172.21.15.45:0/560976446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1d01629b0 0x7fa1d006eed0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0097a0 tx=0x7fa1cc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b15a6c5b8998e80f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.870+0000 7fa1d4a92640 1 -- 172.21.15.45:0/560976446 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc037070 con 0x7fa1d01629b0 2024-05-08T05:46:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.870+0000 7fa1d4a92640 1 -- 172.21.15.45:0/560976446 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1cc032440 con 0x7fa1d01629b0 2024-05-08T05:46:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.870+0000 7fa1d4a92640 1 -- 172.21.15.45:0/560976446 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc0327f0 con 0x7fa1d01629b0 2024-05-08T05:46:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.871+0000 7fa1d7cf8640 1 -- 172.21.15.45:0/560976446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1d01629b0 msgr2=0x7fa1d006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.871+0000 7fa1d7cf8640 1 --2- 172.21.15.45:0/560976446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1d01629b0 0x7fa1d006eed0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0097a0 tx=0x7fa1cc02a180 comp rx=0 tx=0).stop 2024-05-08T05:46:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.871+0000 7fa1d7cf8640 1 -- 172.21.15.45:0/560976446 shutdown_connections 2024-05-08T05:46:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.871+0000 7fa1d7cf8640 1 --2- 172.21.15.45:0/560976446 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1d0070d20 0x7fa1d00711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.871+0000 7fa1d7cf8640 1 --2- 172.21.15.45:0/560976446 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1d01629b0 0x7fa1d006eed0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.871+0000 7fa1d7cf8640 1 -- 172.21.15.45:0/560976446 >> 172.21.15.45:0/560976446 conn(0x7fa1d00b49d0 msgr2=0x7fa1d00b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.871+0000 7fa1d7cf8640 1 -- 172.21.15.45:0/560976446 shutdown_connections 2024-05-08T05:46:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.871+0000 7fa1d7cf8640 1 -- 172.21.15.45:0/560976446 wait complete. 2024-05-08T05:46:47.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.872+0000 7fa1d7cf8640 1 Processor -- start 2024-05-08T05:46:47.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.872+0000 7fa1d7cf8640 1 -- start start 2024-05-08T05:46:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.873+0000 7fa1d7cf8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1d0070d20 0x7fa1d014de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.873+0000 7fa1d7cf8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1d00b2490 0x7fa1d00cd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.873+0000 7fa1d7cf8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1d01629b0 0x7fa1d00b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.873+0000 7fa1d7cf8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d006f410 con 0x7fa1d0070d20 2024-05-08T05:46:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.873+0000 7fa1d7cf8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d006e9b0 con 0x7fa1d01629b0 2024-05-08T05:46:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.873+0000 7fa1d7cf8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d00afcb0 con 0x7fa1d00b2490 2024-05-08T05:46:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.873+0000 7fa1d5a94640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1d0070d20 0x7fa1d014de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.873+0000 7fa1d5a94640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1d0070d20 0x7fa1d014de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36796/0 (socket says 172.21.15.45:36796) 2024-05-08T05:46:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.873+0000 7fa1d5a94640 1 -- 172.21.15.45:0/908829599 learned_addr learned my addr 172.21.15.45:0/908829599 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.873+0000 7fa1d5293640 1 --2- 172.21.15.45:0/908829599 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1d00b2490 0x7fa1d00cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.873+0000 7fa1d6295640 1 --2- 172.21.15.45:0/908829599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1d01629b0 0x7fa1d00b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.873+0000 7fa1d5a94640 1 -- 172.21.15.45:0/908829599 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1d00b2490 msgr2=0x7fa1d00cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.873+0000 7fa1d5a94640 1 --2- 172.21.15.45:0/908829599 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1d00b2490 0x7fa1d00cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.873+0000 7fa1d5a94640 1 -- 172.21.15.45:0/908829599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1d01629b0 msgr2=0x7fa1d00b68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:47.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.873+0000 7fa1d5a94640 1 --2- 172.21.15.45:0/908829599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1d01629b0 0x7fa1d00b68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.873+0000 7fa1d5a94640 1 -- 172.21.15.45:0/908829599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d01f48b0 con 0x7fa1d0070d20 2024-05-08T05:46:47.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.874+0000 7fa1d5a94640 1 --2- 172.21.15.45:0/908829599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1d0070d20 0x7fa1d014de40 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0097a0 tx=0x7fa1cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:47.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.874+0000 7fa1c67fc640 1 -- 172.21.15.45:0/908829599 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc037050 con 0x7fa1d0070d20 2024-05-08T05:46:47.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.874+0000 7fa1c67fc640 1 -- 172.21.15.45:0/908829599 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1cc038040 con 0x7fa1d0070d20 2024-05-08T05:46:47.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.874+0000 7fa1d7cf8640 1 -- 172.21.15.45:0/908829599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d00b70f0 con 0x7fa1d0070d20 2024-05-08T05:46:47.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.874+0000 7fa1c67fc640 1 -- 172.21.15.45:0/908829599 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc004490 con 0x7fa1d0070d20 2024-05-08T05:46:47.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.874+0000 7fa1d7cf8640 1 -- 172.21.15.45:0/908829599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d00b76b0 con 0x7fa1d0070d20 2024-05-08T05:46:47.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.875+0000 7fa1d7cf8640 1 -- 172.21.15.45:0/908829599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d00b78f0 con 0x7fa1d0070d20 2024-05-08T05:46:47.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.876+0000 7fa1c67fc640 1 -- 172.21.15.45:0/908829599 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa1cc003610 con 0x7fa1d0070d20 2024-05-08T05:46:47.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.876+0000 7fa1c67fc640 1 --2- 172.21.15.45:0/908829599 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1b40762d0 0x7fa1b4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.877+0000 7fa1c67fc640 1 -- 172.21.15.45:0/908829599 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7fa1cc0b8050 con 0x7fa1d0070d20 2024-05-08T05:46:47.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.877+0000 7fa1d5293640 1 --2- 172.21.15.45:0/908829599 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1b40762d0 0x7fa1b4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.877+0000 7fa1d5293640 1 --2- 172.21.15.45:0/908829599 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1b40762d0 0x7fa1b4078790 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0007f50 tx=0x7fa1c0007ee0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:47.881+0000 7fa1c67fc640 1 -- 172.21.15.45:0/908829599 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa1cc0817e0 con 0x7fa1d0070d20 2024-05-08T05:46:48.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.075+0000 7fa1d7cf8640 1 -- 172.21.15.45:0/908829599 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1d00693f0 con 0x7fa1d0070d20 2024-05-08T05:46:48.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.077+0000 7fa1c67fc640 1 -- 172.21.15.45:0/908829599 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+17024 (secure 0 0 0) 0x7fa1cc0810e0 con 0x7fa1d0070d20 2024-05-08T05:46:48.077 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:46:48.077 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":76,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:45:33.480680+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:45:21.867649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-05-08T05:45:23.164376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6836/133050464":"2024-05-09T05:45:33.480648+0000","172.21.15.63:6834/133050464":"2024-05-09T05:45:33.480648+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:46:48.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.079+0000 7fa1d7cf8640 1 -- 172.21.15.45:0/908829599 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1b40762d0 msgr2=0x7fa1b4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:48.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.079+0000 7fa1d7cf8640 1 --2- 172.21.15.45:0/908829599 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1b40762d0 0x7fa1b4078790 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0007f50 tx=0x7fa1c0007ee0 comp rx=0 tx=0).stop 2024-05-08T05:46:48.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.079+0000 7fa1d7cf8640 1 -- 172.21.15.45:0/908829599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1d0070d20 msgr2=0x7fa1d014de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:48.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.079+0000 7fa1d7cf8640 1 --2- 172.21.15.45:0/908829599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1d0070d20 0x7fa1d014de40 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0097a0 tx=0x7fa1cc0046c0 comp rx=0 tx=0).stop 2024-05-08T05:46:48.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.079+0000 7fa1d7cf8640 1 -- 172.21.15.45:0/908829599 shutdown_connections 2024-05-08T05:46:48.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.079+0000 7fa1d7cf8640 1 --2- 172.21.15.45:0/908829599 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1b40762d0 0x7fa1b4078790 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:48.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.079+0000 7fa1d7cf8640 1 --2- 172.21.15.45:0/908829599 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1d00b2490 0x7fa1d00cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:48.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.079+0000 7fa1d7cf8640 1 --2- 172.21.15.45:0/908829599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1d01629b0 0x7fa1d00b68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:48.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.079+0000 7fa1d7cf8640 1 --2- 172.21.15.45:0/908829599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1d0070d20 0x7fa1d014de40 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:48.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.079+0000 7fa1d7cf8640 1 -- 172.21.15.45:0/908829599 >> 172.21.15.45:0/908829599 conn(0x7fa1d00b49d0 msgr2=0x7fa1d00b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:48.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.079+0000 7fa1d7cf8640 1 -- 172.21.15.45:0/908829599 shutdown_connections 2024-05-08T05:46:48.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.079+0000 7fa1d7cf8640 1 -- 172.21.15.45:0/908829599 wait complete. 2024-05-08T05:46:48.090 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T05:46:48.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.195+0000 7f041b21d640 1 Processor -- start 2024-05-08T05:46:48.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.196+0000 7f041b21d640 1 -- start start 2024-05-08T05:46:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.196+0000 7f041b21d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0414162f50 0x7f04141633e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.196+0000 7f041b21d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04140d0500 con 0x7f04140cf790 2024-05-08T05:46:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.196+0000 7f041b21d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f041406da60 con 0x7f04140b21f0 2024-05-08T05:46:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.196+0000 7f041b21d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f041406d810 con 0x7f0414162f50 2024-05-08T05:46:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.197+0000 7f0413fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0414162f50 0x7f04141633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.197+0000 7f04197ba640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f04140cf790 0x7f04140cfc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47636/0 (socket says 172.21.15.45:47636) 2024-05-08T05:46:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.197+0000 7f04197ba640 1 -- 172.21.15.45:0/3687003541 learned_addr learned my addr 172.21.15.45:0/3687003541 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.197+0000 7f0413fff640 1 -- 172.21.15.45:0/3687003541 >> v1:172.21.15.63:6789/0 conn(0x7f04140b21f0 legacy=0x7f041406e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.197+0000 7f0413fff640 1 -- 172.21.15.45:0/3687003541 >> v1:172.21.15.45:6789/0 conn(0x7f04140cf790 legacy=0x7f04140cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.197+0000 7f0413fff640 1 -- 172.21.15.45:0/3687003541 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041406dd40 con 0x7f0414162f50 2024-05-08T05:46:48.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.197+0000 7f04137fe640 1 -- 172.21.15.45:0/3687003541 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 388301400 0 0) 0x7f04140d0500 con 0x7f04140cf790 2024-05-08T05:46:48.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.198+0000 7f0413fff640 1 --2- 172.21.15.45:0/3687003541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0414162f50 0x7f04141633e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0404009d90 tx=0x7f040402a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334010b199105fad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:48.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.198+0000 7f04137fe640 1 -- 172.21.15.45:0/3687003541 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404007740 con 0x7f0414162f50 2024-05-08T05:46:48.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.198+0000 7f04137fe640 1 -- 172.21.15.45:0/3687003541 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04040078e0 con 0x7f0414162f50 2024-05-08T05:46:48.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.198+0000 7f04137fe640 1 -- 172.21.15.45:0/3687003541 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404007cb0 con 0x7f0414162f50 2024-05-08T05:46:48.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.198+0000 7f041b21d640 1 -- 172.21.15.45:0/3687003541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0414162f50 msgr2=0x7f04141633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:48.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.198+0000 7f041b21d640 1 --2- 172.21.15.45:0/3687003541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0414162f50 0x7f04141633e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0404009d90 tx=0x7f040402a530 comp rx=0 tx=0).stop 2024-05-08T05:46:48.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.198+0000 7f041b21d640 1 -- 172.21.15.45:0/3687003541 shutdown_connections 2024-05-08T05:46:48.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.198+0000 7f041b21d640 1 --2- 172.21.15.45:0/3687003541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0414162f50 0x7f04141633e0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:48.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.198+0000 7f041b21d640 1 -- 172.21.15.45:0/3687003541 >> 172.21.15.45:0/3687003541 conn(0x7f041409ba40 msgr2=0x7f041409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:48.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.198+0000 7f041b21d640 1 -- 172.21.15.45:0/3687003541 shutdown_connections 2024-05-08T05:46:48.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.198+0000 7f041b21d640 1 -- 172.21.15.45:0/3687003541 wait complete. 2024-05-08T05:46:48.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.199+0000 7f041b21d640 1 Processor -- start 2024-05-08T05:46:48.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.200+0000 7f041b21d640 1 -- start start 2024-05-08T05:46:48.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.200+0000 7f041b21d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04140b21f0 0x7f04140cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:48.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.200+0000 7f041b21d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04140cf790 0x7f0414151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.200+0000 7f041b21d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0414162f50 0x7f04140b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.200+0000 7f041b21d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f04140cd480 con 0x7f0414162f50 2024-05-08T05:46:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.200+0000 7f041b21d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f041406e0b0 con 0x7f04140b21f0 2024-05-08T05:46:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.200+0000 7f041b21d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f04140cd600 con 0x7f04140cf790 2024-05-08T05:46:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.201+0000 7f04197ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0414162f50 0x7f04140b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.201+0000 7f04197ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0414162f50 0x7f04140b50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36816/0 (socket says 172.21.15.45:36816) 2024-05-08T05:46:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.201+0000 7f04197ba640 1 -- 172.21.15.45:0/2572712203 learned_addr learned my addr 172.21.15.45:0/2572712203 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.201+0000 7f0413fff640 1 --2- 172.21.15.45:0/2572712203 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04140cf790 0x7f0414151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.201+0000 7f0418fb9640 1 --2- 172.21.15.45:0/2572712203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04140b21f0 0x7f04140cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.201+0000 7f04197ba640 1 -- 172.21.15.45:0/2572712203 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04140cf790 msgr2=0x7f0414151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.201+0000 7f04197ba640 1 --2- 172.21.15.45:0/2572712203 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04140cf790 0x7f0414151180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.201+0000 7f04197ba640 1 -- 172.21.15.45:0/2572712203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04140b21f0 msgr2=0x7f04140cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:48.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.201+0000 7f04197ba640 1 --2- 172.21.15.45:0/2572712203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04140b21f0 0x7f04140cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.201+0000 7f04197ba640 1 -- 172.21.15.45:0/2572712203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04140b4050 con 0x7f0414162f50 2024-05-08T05:46:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.201+0000 7f04197ba640 1 --2- 172.21.15.45:0/2572712203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0414162f50 0x7f04140b50f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f03fc00c410 tx=0x7f03fc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.201+0000 7f04117fa640 1 -- 172.21.15.45:0/2572712203 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc0074e0 con 0x7f0414162f50 2024-05-08T05:46:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.201+0000 7f04117fa640 1 -- 172.21.15.45:0/2572712203 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03fc0047e0 con 0x7f0414162f50 2024-05-08T05:46:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.201+0000 7f04117fa640 1 -- 172.21.15.45:0/2572712203 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc004bb0 con 0x7f0414162f50 2024-05-08T05:46:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.201+0000 7f041b21d640 1 -- 172.21.15.45:0/2572712203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04140b4330 con 0x7f0414162f50 2024-05-08T05:46:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.201+0000 7f041b21d640 1 -- 172.21.15.45:0/2572712203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04140b48f0 con 0x7f0414162f50 2024-05-08T05:46:48.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.202+0000 7f041b21d640 1 -- 172.21.15.45:0/2572712203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0414069b00 con 0x7f0414162f50 2024-05-08T05:46:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.203+0000 7f04117fa640 1 -- 172.21.15.45:0/2572712203 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f03fc011070 con 0x7f0414162f50 2024-05-08T05:46:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.203+0000 7f04117fa640 1 --2- 172.21.15.45:0/2572712203 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f03f40762d0 0x7f03f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.203+0000 7f04117fa640 1 -- 172.21.15.45:0/2572712203 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6187+0+0 (secure 0 0 0) 0x7f03fc097330 con 0x7f0414162f50 2024-05-08T05:46:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.204+0000 7f0418fb9640 1 --2- 172.21.15.45:0/2572712203 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f03f40762d0 0x7f03f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:48.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.204+0000 7f0418fb9640 1 --2- 172.21.15.45:0/2572712203 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f03f40762d0 0x7f03f4078790 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f040c004ea0 tx=0x7f040c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:48.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.208+0000 7f04117fa640 1 -- 172.21.15.45:0/2572712203 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f03fc060b40 con 0x7f0414162f50 2024-05-08T05:46:48.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.437+0000 7f041b21d640 1 -- 172.21.15.45:0/2572712203 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f04140cefd0 con 0x7f0414162f50 2024-05-08T05:46:48.437 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-05-08T05:46:48.437+0000 7f124de12640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T05:46:48.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.438+0000 7f04117fa640 1 -- 172.21.15.45:0/2572712203 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v53) ==== 161+0+0 (secure 0 0 0) 0x7f03fc060440 con 0x7f0414162f50 2024-05-08T05:46:48.438 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T05:46:48.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.441+0000 7f041b21d640 1 -- 172.21.15.45:0/2572712203 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f03f40762d0 msgr2=0x7f03f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:48.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.441+0000 7f041b21d640 1 --2- 172.21.15.45:0/2572712203 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f03f40762d0 0x7f03f4078790 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f040c004ea0 tx=0x7f040c009040 comp rx=0 tx=0).stop 2024-05-08T05:46:48.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.441+0000 7f041b21d640 1 -- 172.21.15.45:0/2572712203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0414162f50 msgr2=0x7f04140b50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:48.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.441+0000 7f041b21d640 1 --2- 172.21.15.45:0/2572712203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0414162f50 0x7f04140b50f0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f03fc00c410 tx=0x7f03fc00c8d0 comp rx=0 tx=0).stop 2024-05-08T05:46:48.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.442+0000 7f041b21d640 1 -- 172.21.15.45:0/2572712203 shutdown_connections 2024-05-08T05:46:48.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.442+0000 7f041b21d640 1 --2- 172.21.15.45:0/2572712203 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f03f40762d0 0x7f03f4078790 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:48.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.442+0000 7f041b21d640 1 --2- 172.21.15.45:0/2572712203 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f04140cf790 0x7f0414151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:48.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.442+0000 7f041b21d640 1 --2- 172.21.15.45:0/2572712203 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0414162f50 0x7f04140b50f0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:48.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.442+0000 7f041b21d640 1 --2- 172.21.15.45:0/2572712203 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04140b21f0 0x7f04140cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:48.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.442+0000 7f041b21d640 1 -- 172.21.15.45:0/2572712203 >> 172.21.15.45:0/2572712203 conn(0x7f041409ba40 msgr2=0x7f041409be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:48.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.442+0000 7f041b21d640 1 -- 172.21.15.45:0/2572712203 shutdown_connections 2024-05-08T05:46:48.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.442+0000 7f041b21d640 1 -- 172.21.15.45:0/2572712203 wait complete. 2024-05-08T05:46:48.444 INFO:tasks.ceph.mds.a.smithi045.stderr: -3813> 2024-05-08T05:45:40.464+0000 7f4cf5424640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:46:48.453 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T05:46:48.474 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-05-08T05:46:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.559+0000 7f17a18a7640 1 Processor -- start 2024-05-08T05:46:48.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.559+0000 7f17a18a7640 1 -- start start 2024-05-08T05:46:48.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.560+0000 7f17a18a7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f179c15bc50 0x7f179c0b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:48.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.560+0000 7f17a18a7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f179c0732c0 0x7f179c0736f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:48.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.560+0000 7f17a18a7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f179c0711f0 con 0x7f179c06f540 2024-05-08T05:46:48.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.560+0000 7f17a18a7640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f179c073c30 con 0x7f179c0732c0 2024-05-08T05:46:48.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.560+0000 7f17a18a7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f179c06fa40 con 0x7f179c15bc50 2024-05-08T05:46:48.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.560+0000 7f179b7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f179c0732c0 0x7f179c0736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:48.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.561+0000 7f179b7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f179c0732c0 0x7f179c0736f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36824/0 (socket says 172.21.15.45:36824) 2024-05-08T05:46:48.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.561+0000 7f179b7fe640 1 -- 172.21.15.45:0/420668123 learned_addr learned my addr 172.21.15.45:0/420668123 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:48.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.561+0000 7f179affd640 1 --2- 172.21.15.45:0/420668123 >> v2:172.21.15.63:3300/0 conn(0x7f179c15bc50 0x7f179c0b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:48.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.561+0000 7f179b7fe640 1 -- 172.21.15.45:0/420668123 >> v2:172.21.15.63:3300/0 conn(0x7f179c15bc50 msgr2=0x7f179c0b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:48.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.561+0000 7f179b7fe640 1 --2- 172.21.15.45:0/420668123 >> v2:172.21.15.63:3300/0 conn(0x7f179c15bc50 0x7f179c0b5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:48.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.561+0000 7f179b7fe640 1 -- 172.21.15.45:0/420668123 >> v1:172.21.15.63:6789/0 conn(0x7f179c06f540 legacy=0x7f179c070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:48.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.561+0000 7f179b7fe640 1 -- 172.21.15.45:0/420668123 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c06fd20 con 0x7f179c0732c0 2024-05-08T05:46:48.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.562+0000 7f179b7fe640 1 --2- 172.21.15.45:0/420668123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c0732c0 0x7f179c0736f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f179c068ee0 tx=0x7f178c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407823141caea882 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:48.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.562+0000 7f1799ffb640 1 -- 172.21.15.45:0/420668123 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c036070 con 0x7f179c0732c0 2024-05-08T05:46:48.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.562+0000 7f1799ffb640 1 -- 172.21.15.45:0/420668123 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f178c031600 con 0x7f179c0732c0 2024-05-08T05:46:48.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.562+0000 7f1799ffb640 1 -- 172.21.15.45:0/420668123 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c0319d0 con 0x7f179c0732c0 2024-05-08T05:46:48.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.562+0000 7f17a18a7640 1 -- 172.21.15.45:0/420668123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c0732c0 msgr2=0x7f179c0736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:48.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.562+0000 7f17a18a7640 1 --2- 172.21.15.45:0/420668123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c0732c0 0x7f179c0736f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f179c068ee0 tx=0x7f178c02a730 comp rx=0 tx=0).stop 2024-05-08T05:46:48.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.562+0000 7f17a18a7640 1 -- 172.21.15.45:0/420668123 shutdown_connections 2024-05-08T05:46:48.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.562+0000 7f17a18a7640 1 --2- 172.21.15.45:0/420668123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c0732c0 0x7f179c0736f0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:48.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.562+0000 7f17a18a7640 1 --2- 172.21.15.45:0/420668123 >> v2:172.21.15.63:3300/0 conn(0x7f179c15bc50 0x7f179c0b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:48.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.562+0000 7f17a18a7640 1 -- 172.21.15.45:0/420668123 >> 172.21.15.45:0/420668123 conn(0x7f179c0b3c40 msgr2=0x7f179c0b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:48.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.562+0000 7f17a18a7640 1 -- 172.21.15.45:0/420668123 shutdown_connections 2024-05-08T05:46:48.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.562+0000 7f17a18a7640 1 -- 172.21.15.45:0/420668123 wait complete. 2024-05-08T05:46:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.563+0000 7f17a18a7640 1 Processor -- start 2024-05-08T05:46:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.564+0000 7f17a18a7640 1 -- start start 2024-05-08T05:46:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.564+0000 7f17a18a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c06f540 0x7f179c14ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.564+0000 7f17a18a7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f179c0732c0 0x7f179c167d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.564+0000 7f17a18a7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f179c15bc50 0x7f179c150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.564+0000 7f17a18a7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f179c0705c0 con 0x7f179c06f540 2024-05-08T05:46:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.564+0000 7f17a18a7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f179c0702e0 con 0x7f179c15bc50 2024-05-08T05:46:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.564+0000 7f17a18a7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f179c0b60e0 con 0x7f179c0732c0 2024-05-08T05:46:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.564+0000 7f179affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c06f540 0x7f179c14ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:48.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.565+0000 7f179affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c06f540 0x7f179c14ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36826/0 (socket says 172.21.15.45:36826) 2024-05-08T05:46:48.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.565+0000 7f179affd640 1 -- 172.21.15.45:0/2229346493 learned_addr learned my addr 172.21.15.45:0/2229346493 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:48.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.565+0000 7f179a7fc640 1 --2- 172.21.15.45:0/2229346493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f179c0732c0 0x7f179c167d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:48.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.565+0000 7f179b7fe640 1 --2- 172.21.15.45:0/2229346493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f179c15bc50 0x7f179c150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:48.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.565+0000 7f179affd640 1 -- 172.21.15.45:0/2229346493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f179c0732c0 msgr2=0x7f179c167d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:48.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.565+0000 7f179affd640 1 --2- 172.21.15.45:0/2229346493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f179c0732c0 0x7f179c167d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:48.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.565+0000 7f179affd640 1 -- 172.21.15.45:0/2229346493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f179c15bc50 msgr2=0x7f179c150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:48.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.565+0000 7f179affd640 1 --2- 172.21.15.45:0/2229346493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f179c15bc50 0x7f179c150670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:48.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.565+0000 7f179affd640 1 -- 172.21.15.45:0/2229346493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c15dae0 con 0x7f179c06f540 2024-05-08T05:46:48.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.565+0000 7f179affd640 1 --2- 172.21.15.45:0/2229346493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c06f540 0x7f179c14ab40 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f179400d5f0 tx=0x7f179400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:48.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.565+0000 7f1793fff640 1 -- 172.21.15.45:0/2229346493 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17940125f0 con 0x7f179c06f540 2024-05-08T05:46:48.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.565+0000 7f1793fff640 1 -- 172.21.15.45:0/2229346493 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1794019070 con 0x7f179c06f540 2024-05-08T05:46:48.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.565+0000 7f1793fff640 1 -- 172.21.15.45:0/2229346493 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17940129d0 con 0x7f179c06f540 2024-05-08T05:46:48.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.565+0000 7f17a18a7640 1 -- 172.21.15.45:0/2229346493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179c14b380 con 0x7f179c06f540 2024-05-08T05:46:48.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.565+0000 7f17a18a7640 1 -- 172.21.15.45:0/2229346493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f179c168930 con 0x7f179c06f540 2024-05-08T05:46:48.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.566+0000 7f17a18a7640 1 -- 172.21.15.45:0/2229346493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f179c069b00 con 0x7f179c06f540 2024-05-08T05:46:48.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.567+0000 7f1793fff640 1 -- 172.21.15.45:0/2229346493 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1794012b70 con 0x7f179c06f540 2024-05-08T05:46:48.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.567+0000 7f1793fff640 1 --2- 172.21.15.45:0/2229346493 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f177c0762d0 0x7f177c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:48.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.567+0000 7f1793fff640 1 -- 172.21.15.45:0/2229346493 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6273+0+0 (secure 0 0 0) 0x7f1794097c80 con 0x7f179c06f540 2024-05-08T05:46:48.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.567+0000 7f179a7fc640 1 --2- 172.21.15.45:0/2229346493 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f177c0762d0 0x7f177c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:48.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.568+0000 7f179a7fc640 1 --2- 172.21.15.45:0/2229346493 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f177c0762d0 0x7f177c078790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f179c169110 tx=0x7f1784009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:48.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.572+0000 7f1793fff640 1 -- 172.21.15.45:0/2229346493 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1794061390 con 0x7f179c06f540 2024-05-08T05:46:48.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:48.805+0000 7f17a18a7640 1 -- 172.21.15.45:0/2229346493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f179c074690 con 0x7f179c06f540 2024-05-08T05:46:49.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.444+0000 7f1793fff640 1 -- 172.21.15.45:0/2229346493 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v54) ==== 104+0+0 (secure 0 0 0) 0x7f1794060c90 con 0x7f179c06f540 2024-05-08T05:46:49.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.447+0000 7f17a18a7640 1 -- 172.21.15.45:0/2229346493 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f177c0762d0 msgr2=0x7f177c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:49.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.447+0000 7f17a18a7640 1 --2- 172.21.15.45:0/2229346493 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f177c0762d0 0x7f177c078790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f179c169110 tx=0x7f1784009040 comp rx=0 tx=0).stop 2024-05-08T05:46:49.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.447+0000 7f17a18a7640 1 -- 172.21.15.45:0/2229346493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c06f540 msgr2=0x7f179c14ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:49.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.447+0000 7f17a18a7640 1 --2- 172.21.15.45:0/2229346493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c06f540 0x7f179c14ab40 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f179400d5f0 tx=0x7f179400dab0 comp rx=0 tx=0).stop 2024-05-08T05:46:49.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.447+0000 7f17a18a7640 1 -- 172.21.15.45:0/2229346493 shutdown_connections 2024-05-08T05:46:49.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.447+0000 7f17a18a7640 1 --2- 172.21.15.45:0/2229346493 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f177c0762d0 0x7f177c078790 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:49.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.447+0000 7f17a18a7640 1 --2- 172.21.15.45:0/2229346493 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f179c0732c0 0x7f179c167d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:49.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.447+0000 7f17a18a7640 1 --2- 172.21.15.45:0/2229346493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f179c15bc50 0x7f179c150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:49.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.447+0000 7f17a18a7640 1 --2- 172.21.15.45:0/2229346493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f179c06f540 0x7f179c14ab40 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:49.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.447+0000 7f17a18a7640 1 -- 172.21.15.45:0/2229346493 >> 172.21.15.45:0/2229346493 conn(0x7f179c0b3c40 msgr2=0x7f179c075840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:49.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.447+0000 7f17a18a7640 1 -- 172.21.15.45:0/2229346493 shutdown_connections 2024-05-08T05:46:49.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.447+0000 7f17a18a7640 1 -- 172.21.15.45:0/2229346493 wait complete. 2024-05-08T05:46:49.462 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T05:46:49.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.620+0000 7f3136c99640 1 Processor -- start 2024-05-08T05:46:49.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.620+0000 7f3136c99640 1 -- start start 2024-05-08T05:46:49.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.620+0000 7f3136c99640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f31300af6a0 0x7f313006ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:49.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.620+0000 7f3136c99640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31300afbe0 con 0x7f3130159770 2024-05-08T05:46:49.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.621+0000 7f3136c99640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f313015fa80 con 0x7f313006cab0 2024-05-08T05:46:49.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.621+0000 7f3136c99640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f31300b1660 con 0x7f31300af6a0 2024-05-08T05:46:49.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.621+0000 7f3135236640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f31300af6a0 0x7f313006ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:49.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.621+0000 7f3134a35640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3130159770 0x7f313006c3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47646/0 (socket says 172.21.15.45:47646) 2024-05-08T05:46:49.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.621+0000 7f3134a35640 1 -- 172.21.15.45:0/2137494721 learned_addr learned my addr 172.21.15.45:0/2137494721 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:49.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.621+0000 7f3135236640 1 -- 172.21.15.45:0/2137494721 >> v1:172.21.15.63:6789/0 conn(0x7f313006cab0 legacy=0x7f31300aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:49.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.621+0000 7f3135236640 1 -- 172.21.15.45:0/2137494721 >> v1:172.21.15.45:6789/0 conn(0x7f3130159770 legacy=0x7f313006c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:49.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.621+0000 7f3135236640 1 -- 172.21.15.45:0/2137494721 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313015fd60 con 0x7f31300af6a0 2024-05-08T05:46:49.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.622+0000 7f3135236640 1 --2- 172.21.15.45:0/2137494721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31300af6a0 0x7f313006ffe0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3128009d90 tx=0x7f312802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289d030595ef5d0c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:49.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.622+0000 7f312f7fe640 1 -- 172.21.15.45:0/2137494721 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128007740 con 0x7f31300af6a0 2024-05-08T05:46:49.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.622+0000 7f312f7fe640 1 -- 172.21.15.45:0/2137494721 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31280078e0 con 0x7f31300af6a0 2024-05-08T05:46:49.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.622+0000 7f312f7fe640 1 -- 172.21.15.45:0/2137494721 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128007cb0 con 0x7f31300af6a0 2024-05-08T05:46:49.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.622+0000 7f3136c99640 1 -- 172.21.15.45:0/2137494721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31300af6a0 msgr2=0x7f313006ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:49.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.622+0000 7f3136c99640 1 --2- 172.21.15.45:0/2137494721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31300af6a0 0x7f313006ffe0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3128009d90 tx=0x7f312802a660 comp rx=0 tx=0).stop 2024-05-08T05:46:49.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.623+0000 7f3136c99640 1 -- 172.21.15.45:0/2137494721 shutdown_connections 2024-05-08T05:46:49.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.623+0000 7f3136c99640 1 --2- 172.21.15.45:0/2137494721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31300af6a0 0x7f313006ffe0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:49.623 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.623+0000 7f3136c99640 1 -- 172.21.15.45:0/2137494721 >> 172.21.15.45:0/2137494721 conn(0x7f31300b3aa0 msgr2=0x7f31300b3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:49.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.623+0000 7f3136c99640 1 -- 172.21.15.45:0/2137494721 shutdown_connections 2024-05-08T05:46:49.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.623+0000 7f3136c99640 1 -- 172.21.15.45:0/2137494721 wait complete. 2024-05-08T05:46:49.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.624+0000 7f3136c99640 1 Processor -- start 2024-05-08T05:46:49.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.624+0000 7f3136c99640 1 -- start start 2024-05-08T05:46:49.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.625+0000 7f3136c99640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f313006cab0 0x7f3130157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:49.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.625+0000 7f3136c99640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31300af6a0 0x7f31301e8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:49.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.625+0000 7f3136c99640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3130159770 0x7f3130152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:49.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.625+0000 7f3136c99640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f313015f730 con 0x7f3130159770 2024-05-08T05:46:49.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.625+0000 7f3136c99640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f31301600d0 con 0x7f313006cab0 2024-05-08T05:46:49.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.625+0000 7f3136c99640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3130160560 con 0x7f31300af6a0 2024-05-08T05:46:49.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.625+0000 7f3135236640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3130159770 0x7f3130152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:49.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.625+0000 7f3135236640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3130159770 0x7f3130152f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36840/0 (socket says 172.21.15.45:36840) 2024-05-08T05:46:49.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.625+0000 7f3135236640 1 -- 172.21.15.45:0/37168279 learned_addr learned my addr 172.21.15.45:0/37168279 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:49.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.625+0000 7f3134a35640 1 --2- 172.21.15.45:0/37168279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f313006cab0 0x7f3130157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:49.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.625+0000 7f312ffff640 1 --2- 172.21.15.45:0/37168279 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31300af6a0 0x7f31301e8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:49.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.625+0000 7f3135236640 1 -- 172.21.15.45:0/37168279 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31300af6a0 msgr2=0x7f31301e8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:49.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.625+0000 7f3135236640 1 --2- 172.21.15.45:0/37168279 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31300af6a0 0x7f31301e8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:49.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.625+0000 7f3135236640 1 -- 172.21.15.45:0/37168279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f313006cab0 msgr2=0x7f3130157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:49.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.625+0000 7f3135236640 1 --2- 172.21.15.45:0/37168279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f313006cab0 0x7f3130157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:49.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.625+0000 7f3135236640 1 -- 172.21.15.45:0/37168279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31301e79c0 con 0x7f3130159770 2024-05-08T05:46:49.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.625+0000 7f3135236640 1 --2- 172.21.15.45:0/37168279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3130159770 0x7f3130152f10 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3128009d90 tx=0x7f3128004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:49.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.626+0000 7f312d7fa640 1 -- 172.21.15.45:0/37168279 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128040070 con 0x7f3130159770 2024-05-08T05:46:49.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.626+0000 7f312d7fa640 1 -- 172.21.15.45:0/37168279 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3128031c90 con 0x7f3130159770 2024-05-08T05:46:49.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.626+0000 7f3136c99640 1 -- 172.21.15.45:0/37168279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31301e7ca0 con 0x7f3130159770 2024-05-08T05:46:49.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.626+0000 7f312d7fa640 1 -- 172.21.15.45:0/37168279 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312803c640 con 0x7f3130159770 2024-05-08T05:46:49.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.626+0000 7f3136c99640 1 -- 172.21.15.45:0/37168279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31301e8260 con 0x7f3130159770 2024-05-08T05:46:49.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.627+0000 7f3136c99640 1 -- 172.21.15.45:0/37168279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31300b0e10 con 0x7f3130159770 2024-05-08T05:46:49.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.627+0000 7f312d7fa640 1 -- 172.21.15.45:0/37168279 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3128039040 con 0x7f3130159770 2024-05-08T05:46:49.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.628+0000 7f312d7fa640 1 --2- 172.21.15.45:0/37168279 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f31140762d0 0x7f3114078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:49.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.628+0000 7f312d7fa640 1 -- 172.21.15.45:0/37168279 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6273+0+0 (secure 0 0 0) 0x7f31280b8e10 con 0x7f3130159770 2024-05-08T05:46:49.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.628+0000 7f3134a35640 1 --2- 172.21.15.45:0/37168279 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f31140762d0 0x7f3114078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:49.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.629+0000 7f3134a35640 1 --2- 172.21.15.45:0/37168279 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f31140762d0 0x7f3114078790 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3124000f60 tx=0x7f3124009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:49.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.632+0000 7f312d7fa640 1 -- 172.21.15.45:0/37168279 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f31300b0e10 con 0x7f3130159770 2024-05-08T05:46:49.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:49.822+0000 7f3136c99640 1 -- 172.21.15.45:0/37168279 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f313015f070 con 0x7f3130159770 2024-05-08T05:46:50.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.446+0000 7f312d7fa640 1 -- 172.21.15.45:0/37168279 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v78) ==== 181+0+0 (secure 0 0 0) 0x7f31280bc030 con 0x7f3130159770 2024-05-08T05:46:50.447 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-05-08T05:46:50.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.450+0000 7f3136c99640 1 -- 172.21.15.45:0/37168279 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f31140762d0 msgr2=0x7f3114078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:50.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.450+0000 7f3136c99640 1 --2- 172.21.15.45:0/37168279 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f31140762d0 0x7f3114078790 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3124000f60 tx=0x7f3124009040 comp rx=0 tx=0).stop 2024-05-08T05:46:50.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.450+0000 7f3136c99640 1 -- 172.21.15.45:0/37168279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3130159770 msgr2=0x7f3130152f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:50.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.450+0000 7f3136c99640 1 --2- 172.21.15.45:0/37168279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3130159770 0x7f3130152f10 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3128009d90 tx=0x7f3128004a40 comp rx=0 tx=0).stop 2024-05-08T05:46:50.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.450+0000 7f3136c99640 1 -- 172.21.15.45:0/37168279 shutdown_connections 2024-05-08T05:46:50.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.450+0000 7f3136c99640 1 --2- 172.21.15.45:0/37168279 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f31140762d0 0x7f3114078790 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:50.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.451+0000 7f3136c99640 1 --2- 172.21.15.45:0/37168279 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f31300af6a0 0x7f31301e8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:50.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.451+0000 7f3136c99640 1 --2- 172.21.15.45:0/37168279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3130159770 0x7f3130152f10 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:50.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.451+0000 7f3136c99640 1 --2- 172.21.15.45:0/37168279 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f313006cab0 0x7f3130157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:50.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.451+0000 7f3136c99640 1 -- 172.21.15.45:0/37168279 >> 172.21.15.45:0/37168279 conn(0x7f31300b3aa0 msgr2=0x7f31300b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:50.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.451+0000 7f3136c99640 1 -- 172.21.15.45:0/37168279 shutdown_connections 2024-05-08T05:46:50.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.451+0000 7f3136c99640 1 -- 172.21.15.45:0/37168279 wait complete. 2024-05-08T05:46:50.465 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T05:46:50.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.580+0000 7fd794a15640 1 Processor -- start 2024-05-08T05:46:50.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.580+0000 7fd794a15640 1 -- start start 2024-05-08T05:46:50.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.580+0000 7fd794a15640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd7900d3a50 0x7fd790162920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:50.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd794a15640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7900b50d0 con 0x7fd7900d3670 2024-05-08T05:46:50.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd794a15640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7900cfc00 con 0x7fd7900d0940 2024-05-08T05:46:50.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd794a15640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd7900d2a10 con 0x7fd7900d3a50 2024-05-08T05:46:50.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd78e59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd7900d3670 0x7fd7900b49c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:47648/0 (socket says 172.21.15.45:47648) 2024-05-08T05:46:50.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd78e59c640 1 -- 172.21.15.45:0/4114184483 learned_addr learned my addr 172.21.15.45:0/4114184483 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:50.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd78dd9b640 1 -- 172.21.15.45:0/4114184483 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3216291384 0 0) 0x7fd7900b50d0 con 0x7fd7900d3670 2024-05-08T05:46:50.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd787fff640 1 --2- 172.21.15.45:0/4114184483 >> v2:172.21.15.63:3300/0 conn(0x7fd7900d3a50 0x7fd790162920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:50.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd78dd9b640 1 -- 172.21.15.45:0/4114184483 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd77c003520 con 0x7fd7900d3670 2024-05-08T05:46:50.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd78dd9b640 1 -- 172.21.15.45:0/4114184483 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4034219810 0 0) 0x7fd77c003520 con 0x7fd7900d3670 2024-05-08T05:46:50.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd78dd9b640 1 -- 172.21.15.45:0/4114184483 >> v2:172.21.15.63:3300/0 conn(0x7fd7900d3a50 msgr2=0x7fd790162920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:50.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd78dd9b640 1 --2- 172.21.15.45:0/4114184483 >> v2:172.21.15.63:3300/0 conn(0x7fd7900d3a50 0x7fd790162920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:50.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd78dd9b640 1 -- 172.21.15.45:0/4114184483 >> v1:172.21.15.63:6789/0 conn(0x7fd7900d0940 legacy=0x7fd7900d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:50.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd78dd9b640 1 -- 172.21.15.45:0/4114184483 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7900cfea0 con 0x7fd7900d3670 2024-05-08T05:46:50.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd78dd9b640 1 -- 172.21.15.45:0/4114184483 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fd7800016f0 con 0x7fd7900d3670 2024-05-08T05:46:50.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd78dd9b640 1 -- 172.21.15.45:0/4114184483 >> v1:172.21.15.45:6789/0 conn(0x7fd7900d3670 legacy=0x7fd7900b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:50.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd78dd9b640 1 --2- 172.21.15.45:0/4114184483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd77c003a50 0x7fd77c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:50.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd78dd9b640 1 --2- 172.21.15.45:0/4114184483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd77c004800 0x7fd77c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:50.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd78dd9b640 1 --2- 172.21.15.45:0/4114184483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd77c0051c0 0x7fd77c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:50.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd78dd9b640 1 -- 172.21.15.45:0/4114184483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd7900d2a10 con 0x7fd77c0051c0 2024-05-08T05:46:50.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd78dd9b640 1 -- 172.21.15.45:0/4114184483 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c00c160 con 0x7fd77c003a50 2024-05-08T05:46:50.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd78dd9b640 1 -- 172.21.15.45:0/4114184483 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd77c00c360 con 0x7fd77c004800 2024-05-08T05:46:50.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.581+0000 7fd78ed9d640 1 --2- 172.21.15.45:0/4114184483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd77c0051c0 0x7fd77c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:50.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.582+0000 7fd78e59c640 1 --2- 172.21.15.45:0/4114184483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd77c003a50 0x7fd77c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:50.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.582+0000 7fd787fff640 1 --2- 172.21.15.45:0/4114184483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd77c004800 0x7fd77c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:50.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.582+0000 7fd78ed9d640 1 -- 172.21.15.45:0/4114184483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd77c004800 msgr2=0x7fd77c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:50.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.582+0000 7fd78ed9d640 1 --2- 172.21.15.45:0/4114184483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd77c004800 0x7fd77c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:50.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.582+0000 7fd78ed9d640 1 -- 172.21.15.45:0/4114184483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd77c003a50 msgr2=0x7fd77c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:50.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.582+0000 7fd78ed9d640 1 --2- 172.21.15.45:0/4114184483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd77c003a50 0x7fd77c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:50.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.582+0000 7fd78ed9d640 1 -- 172.21.15.45:0/4114184483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77c003520 con 0x7fd77c0051c0 2024-05-08T05:46:50.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.582+0000 7fd787fff640 1 --2- 172.21.15.45:0/4114184483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd77c004800 0x7fd77c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:50.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.582+0000 7fd78e59c640 1 --2- 172.21.15.45:0/4114184483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd77c003a50 0x7fd77c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:50.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.582+0000 7fd78ed9d640 1 -- 172.21.15.45:0/4114184483 reap_dead start 2024-05-08T05:46:50.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.582+0000 7fd78ed9d640 1 --2- 172.21.15.45:0/4114184483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd77c0051c0 0x7fd77c00b9c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd788001e90 tx=0x7fd77c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d975e1022e3ac1c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:50.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.583+0000 7fd78dd9b640 1 -- 172.21.15.45:0/4114184483 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd788001ab0 con 0x7fd77c0051c0 2024-05-08T05:46:50.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.583+0000 7fd78dd9b640 1 -- 172.21.15.45:0/4114184483 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd78802ae50 con 0x7fd77c0051c0 2024-05-08T05:46:50.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.583+0000 7fd78dd9b640 1 -- 172.21.15.45:0/4114184483 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd788003bb0 con 0x7fd77c0051c0 2024-05-08T05:46:50.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.583+0000 7fd794a15640 1 -- 172.21.15.45:0/4114184483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd77c0051c0 msgr2=0x7fd77c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:50.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.583+0000 7fd794a15640 1 --2- 172.21.15.45:0/4114184483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd77c0051c0 0x7fd77c00b9c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd788001e90 tx=0x7fd77c005830 comp rx=0 tx=0).stop 2024-05-08T05:46:50.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.583+0000 7fd794a15640 1 -- 172.21.15.45:0/4114184483 shutdown_connections 2024-05-08T05:46:50.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.583+0000 7fd794a15640 1 --2- 172.21.15.45:0/4114184483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd77c0051c0 0x7fd77c00b9c0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:50.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.583+0000 7fd794a15640 1 -- 172.21.15.45:0/4114184483 >> 172.21.15.45:0/4114184483 conn(0x7fd79009e390 msgr2=0x7fd7900d5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:50.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.583+0000 7fd794a15640 1 -- 172.21.15.45:0/4114184483 shutdown_connections 2024-05-08T05:46:50.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.583+0000 7fd794a15640 1 -- 172.21.15.45:0/4114184483 wait complete. 2024-05-08T05:46:50.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.585+0000 7fd794a15640 1 Processor -- start 2024-05-08T05:46:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.585+0000 7fd794a15640 1 -- start start 2024-05-08T05:46:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.585+0000 7fd794a15640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd77c0051c0 0x7fd790155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.585+0000 7fd794a15640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7900ce3e0 0x7fd7900b9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.585+0000 7fd794a15640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7900ce860 0x7fd7900b9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.585+0000 7fd794a15640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd7900d0140 con 0x7fd7900ce3e0 2024-05-08T05:46:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.585+0000 7fd794a15640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd7900d04b0 con 0x7fd77c0051c0 2024-05-08T05:46:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.585+0000 7fd794a15640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd7900d0dd0 con 0x7fd7900ce860 2024-05-08T05:46:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.586+0000 7fd787fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7900ce3e0 0x7fd7900b9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.586+0000 7fd787fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7900ce3e0 0x7fd7900b9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36860/0 (socket says 172.21.15.45:36860) 2024-05-08T05:46:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.586+0000 7fd787fff640 1 -- 172.21.15.45:0/1722665494 learned_addr learned my addr 172.21.15.45:0/1722665494 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.586+0000 7fd78e59c640 1 --2- 172.21.15.45:0/1722665494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd77c0051c0 0x7fd790155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.586+0000 7fd78ed9d640 1 --2- 172.21.15.45:0/1722665494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7900ce860 0x7fd7900b9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:50.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.586+0000 7fd787fff640 1 -- 172.21.15.45:0/1722665494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7900ce860 msgr2=0x7fd7900b9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.586+0000 7fd787fff640 1 --2- 172.21.15.45:0/1722665494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7900ce860 0x7fd7900b9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.586+0000 7fd787fff640 1 -- 172.21.15.45:0/1722665494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd77c0051c0 msgr2=0x7fd790155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.586+0000 7fd787fff640 1 --2- 172.21.15.45:0/1722665494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd77c0051c0 0x7fd790155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.586+0000 7fd787fff640 1 -- 172.21.15.45:0/1722665494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7900b8990 con 0x7fd7900ce3e0 2024-05-08T05:46:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.586+0000 7fd787fff640 1 --2- 172.21.15.45:0/1722665494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7900ce3e0 0x7fd7900b9330 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd778011570 tx=0x7fd778011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.586+0000 7fd786ffd640 1 -- 172.21.15.45:0/1722665494 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778015870 con 0x7fd7900ce3e0 2024-05-08T05:46:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.586+0000 7fd786ffd640 1 -- 172.21.15.45:0/1722665494 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd778015a10 con 0x7fd7900ce3e0 2024-05-08T05:46:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.586+0000 7fd794a15640 1 -- 172.21.15.45:0/1722665494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7900b8c70 con 0x7fd7900ce3e0 2024-05-08T05:46:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.586+0000 7fd786ffd640 1 -- 172.21.15.45:0/1722665494 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778015dc0 con 0x7fd7900ce3e0 2024-05-08T05:46:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.586+0000 7fd794a15640 1 -- 172.21.15.45:0/1722665494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7900cc380 con 0x7fd7900ce3e0 2024-05-08T05:46:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.587+0000 7fd794a15640 1 -- 172.21.15.45:0/1722665494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7900693f0 con 0x7fd7900ce3e0 2024-05-08T05:46:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.588+0000 7fd786ffd640 1 -- 172.21.15.45:0/1722665494 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd77800f460 con 0x7fd7900ce3e0 2024-05-08T05:46:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.588+0000 7fd786ffd640 1 --2- 172.21.15.45:0/1722665494 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7700762d0 0x7fd770078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:50.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.589+0000 7fd786ffd640 1 -- 172.21.15.45:0/1722665494 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5835+0+0 (secure 0 0 0) 0x7fd778097f60 con 0x7fd7900ce3e0 2024-05-08T05:46:50.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.589+0000 7fd78e59c640 1 --2- 172.21.15.45:0/1722665494 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7700762d0 0x7fd770078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:50.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.589+0000 7fd78e59c640 1 --2- 172.21.15.45:0/1722665494 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7700762d0 0x7fd770078790 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd7900b44d0 tx=0x7fd780009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:50.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.593+0000 7fd786ffd640 1 -- 172.21.15.45:0/1722665494 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd778019030 con 0x7fd7900ce3e0 2024-05-08T05:46:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:50.783+0000 7fd794a15640 1 -- 172.21.15.45:0/1722665494 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd7900b4f40 con 0x7fd7900ce3e0 2024-05-08T05:46:51.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.451+0000 7fd786ffd640 1 -- 172.21.15.45:0/1722665494 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v79) ==== 169+0+0 (secure 0 0 0) 0x7fd778061830 con 0x7fd7900ce3e0 2024-05-08T05:46:51.452 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-05-08T05:46:51.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.456+0000 7fd794a15640 1 -- 172.21.15.45:0/1722665494 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7700762d0 msgr2=0x7fd770078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:51.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.456+0000 7fd794a15640 1 --2- 172.21.15.45:0/1722665494 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7700762d0 0x7fd770078790 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd7900b44d0 tx=0x7fd780009040 comp rx=0 tx=0).stop 2024-05-08T05:46:51.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.456+0000 7fd794a15640 1 -- 172.21.15.45:0/1722665494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7900ce3e0 msgr2=0x7fd7900b9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:51.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.456+0000 7fd794a15640 1 --2- 172.21.15.45:0/1722665494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7900ce3e0 0x7fd7900b9330 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd778011570 tx=0x7fd778011a30 comp rx=0 tx=0).stop 2024-05-08T05:46:51.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.456+0000 7fd794a15640 1 -- 172.21.15.45:0/1722665494 shutdown_connections 2024-05-08T05:46:51.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.456+0000 7fd794a15640 1 --2- 172.21.15.45:0/1722665494 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7700762d0 0x7fd770078790 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.456+0000 7fd794a15640 1 --2- 172.21.15.45:0/1722665494 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7900ce860 0x7fd7900b9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.456+0000 7fd794a15640 1 --2- 172.21.15.45:0/1722665494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7900ce3e0 0x7fd7900b9330 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.456+0000 7fd794a15640 1 --2- 172.21.15.45:0/1722665494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd77c0051c0 0x7fd790155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.456+0000 7fd794a15640 1 -- 172.21.15.45:0/1722665494 >> 172.21.15.45:0/1722665494 conn(0x7fd79009e390 msgr2=0x7fd7900cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:51.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.456+0000 7fd794a15640 1 -- 172.21.15.45:0/1722665494 shutdown_connections 2024-05-08T05:46:51.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.456+0000 7fd794a15640 1 -- 172.21.15.45:0/1722665494 wait complete. 2024-05-08T05:46:51.471 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T05:46:51.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.583+0000 7f352e346640 1 Processor -- start 2024-05-08T05:46:51.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.583+0000 7f352e346640 1 -- start start 2024-05-08T05:46:51.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.584+0000 7f352e346640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35280d3cd0 0x7f35280cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:51.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.584+0000 7f352e346640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35280c9ee0 0x7f35280d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:51.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.584+0000 7f352e346640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35280d3330 con 0x7f35281629a0 2024-05-08T05:46:51.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.584+0000 7f352e346640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f35280d2170 con 0x7f35280d3cd0 2024-05-08T05:46:51.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.584+0000 7f352e346640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f35280d2450 con 0x7f35280c9ee0 2024-05-08T05:46:51.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.584+0000 7f35277fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35280d3cd0 0x7f35280cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:51.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.584+0000 7f35277fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f35280d3cd0 0x7f35280cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51722/0 (socket says 172.21.15.45:51722) 2024-05-08T05:46:51.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.584+0000 7f35277fe640 1 -- 172.21.15.45:0/4277834326 learned_addr learned my addr 172.21.15.45:0/4277834326 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:51.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.584+0000 7f352c8e3640 1 --2- 172.21.15.45:0/4277834326 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35280c9ee0 0x7f35280d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:51.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.584+0000 7f35277fe640 1 -- 172.21.15.45:0/4277834326 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35280c9ee0 msgr2=0x7f35280d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:51.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.584+0000 7f35277fe640 1 --2- 172.21.15.45:0/4277834326 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35280c9ee0 0x7f35280d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.584+0000 7f35277fe640 1 -- 172.21.15.45:0/4277834326 >> v1:172.21.15.63:6789/0 conn(0x7f35281629a0 legacy=0x7f35280d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:51.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.584+0000 7f35277fe640 1 -- 172.21.15.45:0/4277834326 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35280d2730 con 0x7f35280d3cd0 2024-05-08T05:46:51.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.585+0000 7f35277fe640 1 --2- 172.21.15.45:0/4277834326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35280d3cd0 0x7f35280cfca0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f351c009870 tx=0x7f351c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189250eaf9b3ffc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:51.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.585+0000 7f3526ffd640 1 -- 172.21.15.45:0/4277834326 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c037070 con 0x7f35280d3cd0 2024-05-08T05:46:51.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.585+0000 7f3526ffd640 1 -- 172.21.15.45:0/4277834326 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f351c032440 con 0x7f35280d3cd0 2024-05-08T05:46:51.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.585+0000 7f3526ffd640 1 -- 172.21.15.45:0/4277834326 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c032810 con 0x7f35280d3cd0 2024-05-08T05:46:51.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.586+0000 7f352e346640 1 -- 172.21.15.45:0/4277834326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35280d3cd0 msgr2=0x7f35280cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:51.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.586+0000 7f352e346640 1 --2- 172.21.15.45:0/4277834326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35280d3cd0 0x7f35280cfca0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f351c009870 tx=0x7f351c02a130 comp rx=0 tx=0).stop 2024-05-08T05:46:51.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.586+0000 7f352e346640 1 -- 172.21.15.45:0/4277834326 shutdown_connections 2024-05-08T05:46:51.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.586+0000 7f352e346640 1 --2- 172.21.15.45:0/4277834326 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35280c9ee0 0x7f35280d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.586+0000 7f352e346640 1 --2- 172.21.15.45:0/4277834326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35280d3cd0 0x7f35280cfca0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.586+0000 7f352e346640 1 -- 172.21.15.45:0/4277834326 >> 172.21.15.45:0/4277834326 conn(0x7f35280d4fe0 msgr2=0x7f35280d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:51.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.586+0000 7f352e346640 1 -- 172.21.15.45:0/4277834326 shutdown_connections 2024-05-08T05:46:51.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.586+0000 7f352e346640 1 -- 172.21.15.45:0/4277834326 wait complete. 2024-05-08T05:46:51.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.587+0000 7f352e346640 1 Processor -- start 2024-05-08T05:46:51.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.587+0000 7f352e346640 1 -- start start 2024-05-08T05:46:51.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.589+0000 7f352e346640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35280c9ee0 0x7f35280ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:51.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.589+0000 7f352e346640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35280d3cd0 0x7f35281e7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:51.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.589+0000 7f352e346640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35281629a0 0x7f35281e81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:51.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.589+0000 7f352e346640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f35280cf440 con 0x7f35281629a0 2024-05-08T05:46:51.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.589+0000 7f352e346640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f35280cf2c0 con 0x7f35280d3cd0 2024-05-08T05:46:51.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.589+0000 7f352e346640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f35280d2dc0 con 0x7f35280c9ee0 2024-05-08T05:46:51.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.589+0000 7f352c8e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35281629a0 0x7f35281e81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:51.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.589+0000 7f352c8e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35281629a0 0x7f35281e81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51728/0 (socket says 172.21.15.45:51728) 2024-05-08T05:46:51.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.589+0000 7f352c8e3640 1 -- 172.21.15.45:0/3994583674 learned_addr learned my addr 172.21.15.45:0/3994583674 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:51.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.589+0000 7f3527fff640 1 --2- 172.21.15.45:0/3994583674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35280c9ee0 0x7f35280ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:51.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.589+0000 7f35277fe640 1 --2- 172.21.15.45:0/3994583674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35280d3cd0 0x7f35281e7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:51.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.590+0000 7f3527fff640 1 -- 172.21.15.45:0/3994583674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35280d3cd0 msgr2=0x7f35281e7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:51.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.590+0000 7f3527fff640 1 --2- 172.21.15.45:0/3994583674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35280d3cd0 0x7f35281e7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.590+0000 7f3527fff640 1 -- 172.21.15.45:0/3994583674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35281629a0 msgr2=0x7f35281e81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:51.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.590+0000 7f3527fff640 1 --2- 172.21.15.45:0/3994583674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35281629a0 0x7f35281e81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.590+0000 7f3527fff640 1 -- 172.21.15.45:0/3994583674 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35280cd050 con 0x7f35280c9ee0 2024-05-08T05:46:51.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.590+0000 7f3527fff640 1 --2- 172.21.15.45:0/3994583674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35280c9ee0 0x7f35280ccb10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f351800c3c0 tx=0x7f351800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:51.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.590+0000 7f3524ff9640 1 -- 172.21.15.45:0/3994583674 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3518017070 con 0x7f35280c9ee0 2024-05-08T05:46:51.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.590+0000 7f3524ff9640 1 -- 172.21.15.45:0/3994583674 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35180052d0 con 0x7f35280c9ee0 2024-05-08T05:46:51.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.591+0000 7f3524ff9640 1 -- 172.21.15.45:0/3994583674 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3518003830 con 0x7f35280c9ee0 2024-05-08T05:46:51.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.591+0000 7f352e346640 1 -- 172.21.15.45:0/3994583674 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35280b9590 con 0x7f35280c9ee0 2024-05-08T05:46:51.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.591+0000 7f352e346640 1 -- 172.21.15.45:0/3994583674 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f35280b9b50 con 0x7f35280c9ee0 2024-05-08T05:46:51.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.591+0000 7f352c8e3640 1 --2- 172.21.15.45:0/3994583674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35281629a0 0x7f35281e81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:51.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.592+0000 7f352e346640 1 -- 172.21.15.45:0/3994583674 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35280693f0 con 0x7f35280c9ee0 2024-05-08T05:46:51.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.593+0000 7f3524ff9640 1 -- 172.21.15.45:0/3994583674 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f351801a070 con 0x7f35280c9ee0 2024-05-08T05:46:51.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.594+0000 7f3524ff9640 1 --2- 172.21.15.45:0/3994583674 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f35140762d0 0x7f3514078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:51.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.594+0000 7f3524ff9640 1 -- 172.21.15.45:0/3994583674 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(79..79 src has 1..79) ==== 5421+0+0 (secure 0 0 0) 0x7f3518096690 con 0x7f35280c9ee0 2024-05-08T05:46:51.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.594+0000 7f35277fe640 1 --2- 172.21.15.45:0/3994583674 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f35140762d0 0x7f3514078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:51.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.595+0000 7f35277fe640 1 --2- 172.21.15.45:0/3994583674 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f35140762d0 0x7f3514078790 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f351c034000 tx=0x7f351c040040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:51.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.599+0000 7f3524ff9640 1 -- 172.21.15.45:0/3994583674 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f35180601a0 con 0x7f35280c9ee0 2024-05-08T05:46:51.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.793+0000 7f352e346640 1 -- 172.21.15.45:0/3994583674 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f35280b7100 con 0x7f35280c9ee0 2024-05-08T05:46:51.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.798+0000 7f3524ff9640 1 -- 172.21.15.45:0/3994583674 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f351805faa0 con 0x7f35280c9ee0 2024-05-08T05:46:51.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.800+0000 7f352e346640 1 -- 172.21.15.45:0/3994583674 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f35140762d0 msgr2=0x7f3514078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:51.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.800+0000 7f352e346640 1 --2- 172.21.15.45:0/3994583674 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f35140762d0 0x7f3514078790 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f351c034000 tx=0x7f351c040040 comp rx=0 tx=0).stop 2024-05-08T05:46:51.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.800+0000 7f352e346640 1 -- 172.21.15.45:0/3994583674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35280c9ee0 msgr2=0x7f35280ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:51.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.800+0000 7f352e346640 1 --2- 172.21.15.45:0/3994583674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35280c9ee0 0x7f35280ccb10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f351800c3c0 tx=0x7f351800c880 comp rx=0 tx=0).stop 2024-05-08T05:46:51.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.800+0000 7f352e346640 1 -- 172.21.15.45:0/3994583674 shutdown_connections 2024-05-08T05:46:51.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.800+0000 7f352e346640 1 --2- 172.21.15.45:0/3994583674 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f35140762d0 0x7f3514078790 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.800+0000 7f352e346640 1 --2- 172.21.15.45:0/3994583674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35281629a0 0x7f35281e81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.800+0000 7f352e346640 1 --2- 172.21.15.45:0/3994583674 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35280d3cd0 0x7f35281e7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.800+0000 7f352e346640 1 --2- 172.21.15.45:0/3994583674 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35280c9ee0 0x7f35280ccb10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.800+0000 7f352e346640 1 -- 172.21.15.45:0/3994583674 >> 172.21.15.45:0/3994583674 conn(0x7f35280d4fe0 msgr2=0x7f35280ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:51.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.800+0000 7f352e346640 1 -- 172.21.15.45:0/3994583674 shutdown_connections 2024-05-08T05:46:51.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.800+0000 7f352e346640 1 -- 172.21.15.45:0/3994583674 wait complete. 2024-05-08T05:46:51.811 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-05-08T05:46:51.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.915+0000 7fe2963c1640 1 Processor -- start 2024-05-08T05:46:51.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.915+0000 7fe2963c1640 1 -- start start 2024-05-08T05:46:51.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.916+0000 7fe2963c1640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe29006d620 0x7fe290073800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:51.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.916+0000 7fe2963c1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe29006da00 0x7fe2900af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:51.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.916+0000 7fe2963c1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe2900b5ee0 0x7fe29006e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.916+0000 7fe2963c1640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe2900afaa0 con 0x7fe29006da00 2024-05-08T05:46:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.916+0000 7fe2963c1640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe2900b5a50 con 0x7fe29006d620 2024-05-08T05:46:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.916+0000 7fe2963c1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe2900af800 con 0x7fe2900b5ee0 2024-05-08T05:46:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.916+0000 7fe28f7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe29006da00 0x7fe2900af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.916+0000 7fe28f7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe29006da00 0x7fe2900af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51742/0 (socket says 172.21.15.45:51742) 2024-05-08T05:46:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.916+0000 7fe28f7fe640 1 -- 172.21.15.45:0/1586190625 learned_addr learned my addr 172.21.15.45:0/1586190625 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.917+0000 7fe29495e640 1 --2- 172.21.15.45:0/1586190625 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe2900b5ee0 0x7fe29006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.917+0000 7fe28ffff640 1 --2- 172.21.15.45:0/1586190625 >> v2:172.21.15.63:3300/0 conn(0x7fe29006d620 0x7fe290073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.917+0000 7fe28f7fe640 1 -- 172.21.15.45:0/1586190625 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe2900b5ee0 msgr2=0x7fe29006e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.917+0000 7fe28f7fe640 1 --2- 172.21.15.45:0/1586190625 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe2900b5ee0 0x7fe29006e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.917+0000 7fe28f7fe640 1 -- 172.21.15.45:0/1586190625 >> v2:172.21.15.63:3300/0 conn(0x7fe29006d620 msgr2=0x7fe290073800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.917+0000 7fe28f7fe640 1 --2- 172.21.15.45:0/1586190625 >> v2:172.21.15.63:3300/0 conn(0x7fe29006d620 0x7fe290073800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.918 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.917+0000 7fe28f7fe640 1 -- 172.21.15.45:0/1586190625 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2900b5120 con 0x7fe29006da00 2024-05-08T05:46:51.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.917+0000 7fe28f7fe640 1 --2- 172.21.15.45:0/1586190625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe29006da00 0x7fe2900af2c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe290069030 tx=0x7fe28402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8cf77e0d6867c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:51.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.918+0000 7fe28effd640 1 -- 172.21.15.45:0/1586190625 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284037070 con 0x7fe29006da00 2024-05-08T05:46:51.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.918+0000 7fe28effd640 1 -- 172.21.15.45:0/1586190625 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe28402ae00 con 0x7fe29006da00 2024-05-08T05:46:51.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.918+0000 7fe28effd640 1 -- 172.21.15.45:0/1586190625 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284032660 con 0x7fe29006da00 2024-05-08T05:46:51.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.918+0000 7fe2963c1640 1 -- 172.21.15.45:0/1586190625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe29006da00 msgr2=0x7fe2900af2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:51.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.918+0000 7fe2963c1640 1 --2- 172.21.15.45:0/1586190625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe29006da00 0x7fe2900af2c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe290069030 tx=0x7fe28402a0b0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.918+0000 7fe2963c1640 1 -- 172.21.15.45:0/1586190625 shutdown_connections 2024-05-08T05:46:51.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.918+0000 7fe2963c1640 1 --2- 172.21.15.45:0/1586190625 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe2900b5ee0 0x7fe29006e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.918+0000 7fe2963c1640 1 --2- 172.21.15.45:0/1586190625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe29006da00 0x7fe2900af2c0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.918+0000 7fe2963c1640 1 --2- 172.21.15.45:0/1586190625 >> v2:172.21.15.63:3300/0 conn(0x7fe29006d620 0x7fe290073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.918+0000 7fe2963c1640 1 -- 172.21.15.45:0/1586190625 >> 172.21.15.45:0/1586190625 conn(0x7fe2900728c0 msgr2=0x7fe290072cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:51.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.918+0000 7fe2963c1640 1 -- 172.21.15.45:0/1586190625 shutdown_connections 2024-05-08T05:46:51.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.918+0000 7fe2963c1640 1 -- 172.21.15.45:0/1586190625 wait complete. 2024-05-08T05:46:51.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.919+0000 7fe2963c1640 1 Processor -- start 2024-05-08T05:46:51.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.920+0000 7fe2963c1640 1 -- start start 2024-05-08T05:46:51.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.920+0000 7fe2963c1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe29006d620 0x7fe2901f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:51.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.920+0000 7fe2963c1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe29006da00 0x7fe2901f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:51.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.920+0000 7fe2963c1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2900b5ee0 0x7fe290157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:51.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.920+0000 7fe2963c1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe2900b6370 con 0x7fe2900b5ee0 2024-05-08T05:46:51.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.920+0000 7fe2963c1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe2900b56e0 con 0x7fe29006da00 2024-05-08T05:46:51.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.920+0000 7fe2963c1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe2900b64f0 con 0x7fe29006d620 2024-05-08T05:46:51.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.921+0000 7fe29495e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2900b5ee0 0x7fe290157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:51.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.921+0000 7fe29495e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2900b5ee0 0x7fe290157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51750/0 (socket says 172.21.15.45:51750) 2024-05-08T05:46:51.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.921+0000 7fe29495e640 1 -- 172.21.15.45:0/2049145190 learned_addr learned my addr 172.21.15.45:0/2049145190 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:51.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.921+0000 7fe28f7fe640 1 --2- 172.21.15.45:0/2049145190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe29006da00 0x7fe2901f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:51.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.921+0000 7fe28ffff640 1 --2- 172.21.15.45:0/2049145190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe29006d620 0x7fe2901f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:51.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.921+0000 7fe29495e640 1 -- 172.21.15.45:0/2049145190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe29006d620 msgr2=0x7fe2901f90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:51.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.921+0000 7fe29495e640 1 --2- 172.21.15.45:0/2049145190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe29006d620 0x7fe2901f90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.921+0000 7fe29495e640 1 -- 172.21.15.45:0/2049145190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe29006da00 msgr2=0x7fe2901f4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:51.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.921+0000 7fe29495e640 1 --2- 172.21.15.45:0/2049145190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe29006da00 0x7fe2901f4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:51.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.921+0000 7fe29495e640 1 -- 172.21.15.45:0/2049145190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2901f9630 con 0x7fe2900b5ee0 2024-05-08T05:46:51.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.921+0000 7fe29495e640 1 --2- 172.21.15.45:0/2049145190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2900b5ee0 0x7fe290157610 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe288007e40 tx=0x7fe28800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:51.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.921+0000 7fe28cff9640 1 -- 172.21.15.45:0/2049145190 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe288014620 con 0x7fe2900b5ee0 2024-05-08T05:46:51.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.921+0000 7fe28cff9640 1 -- 172.21.15.45:0/2049145190 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe288019070 con 0x7fe2900b5ee0 2024-05-08T05:46:51.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.921+0000 7fe2963c1640 1 -- 172.21.15.45:0/2049145190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe290156210 con 0x7fe2900b5ee0 2024-05-08T05:46:51.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.921+0000 7fe28cff9640 1 -- 172.21.15.45:0/2049145190 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe288014a20 con 0x7fe2900b5ee0 2024-05-08T05:46:51.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.921+0000 7fe2963c1640 1 -- 172.21.15.45:0/2049145190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2901567d0 con 0x7fe2900b5ee0 2024-05-08T05:46:51.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.922+0000 7fe2963c1640 1 -- 172.21.15.45:0/2049145190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2900693f0 con 0x7fe2900b5ee0 2024-05-08T05:46:51.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.923+0000 7fe28cff9640 1 -- 172.21.15.45:0/2049145190 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe288014bc0 con 0x7fe2900b5ee0 2024-05-08T05:46:51.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.923+0000 7fe28cff9640 1 --2- 172.21.15.45:0/2049145190 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe27c0762d0 0x7fe27c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:51.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.923+0000 7fe28cff9640 1 -- 172.21.15.45:0/2049145190 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5421+0+0 (secure 0 0 0) 0x7fe288097ca0 con 0x7fe2900b5ee0 2024-05-08T05:46:51.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.924+0000 7fe28ffff640 1 --2- 172.21.15.45:0/2049145190 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe27c0762d0 0x7fe27c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:51.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.924+0000 7fe28ffff640 1 --2- 172.21.15.45:0/2049145190 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe27c0762d0 0x7fe27c078790 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe280003610 tx=0x7fe280008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:51.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:51.928+0000 7fe28cff9640 1 -- 172.21.15.45:0/2049145190 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe2901f9630 con 0x7fe2900b5ee0 2024-05-08T05:46:52.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.194+0000 7fe2963c1640 1 -- 172.21.15.45:0/2049145190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7fe29006f4d0 con 0x7fe2900b5ee0 2024-05-08T05:46:52.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.453+0000 7fe28cff9640 1 -- 172.21.15.45:0/2049145190 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v261) ==== 166+0+0 (secure 0 0 0) 0x7fe2880610c0 con 0x7fe2900b5ee0 2024-05-08T05:46:52.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.455+0000 7fe2963c1640 1 -- 172.21.15.45:0/2049145190 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe27c0762d0 msgr2=0x7fe27c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:52.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.455+0000 7fe2963c1640 1 --2- 172.21.15.45:0/2049145190 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe27c0762d0 0x7fe27c078790 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe280003610 tx=0x7fe280008040 comp rx=0 tx=0).stop 2024-05-08T05:46:52.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.455+0000 7fe2963c1640 1 -- 172.21.15.45:0/2049145190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2900b5ee0 msgr2=0x7fe290157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:52.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.455+0000 7fe2963c1640 1 --2- 172.21.15.45:0/2049145190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2900b5ee0 0x7fe290157610 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe288007e40 tx=0x7fe28800e4b0 comp rx=0 tx=0).stop 2024-05-08T05:46:52.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.455+0000 7fe2963c1640 1 -- 172.21.15.45:0/2049145190 shutdown_connections 2024-05-08T05:46:52.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.455+0000 7fe2963c1640 1 --2- 172.21.15.45:0/2049145190 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe27c0762d0 0x7fe27c078790 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:52.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.455+0000 7fe2963c1640 1 --2- 172.21.15.45:0/2049145190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe2900b5ee0 0x7fe290157610 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:52.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.455+0000 7fe2963c1640 1 --2- 172.21.15.45:0/2049145190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe29006da00 0x7fe2901f4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:52.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.455+0000 7fe2963c1640 1 --2- 172.21.15.45:0/2049145190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe29006d620 0x7fe2901f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:52.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.455+0000 7fe2963c1640 1 -- 172.21.15.45:0/2049145190 >> 172.21.15.45:0/2049145190 conn(0x7fe2900728c0 msgr2=0x7fe29006fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:52.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.455+0000 7fe2963c1640 1 -- 172.21.15.45:0/2049145190 shutdown_connections 2024-05-08T05:46:52.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.455+0000 7fe2963c1640 1 -- 172.21.15.45:0/2049145190 wait complete. 2024-05-08T05:46:52.467 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-08T05:46:52.467 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-05-08T05:46:52.467 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-08T05:46:52.468 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:46:52.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.572+0000 7f02acfaf640 1 Processor -- start 2024-05-08T05:46:52.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.572+0000 7f02acfaf640 1 -- start start 2024-05-08T05:46:52.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.573+0000 7f02acfaf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f02a806e0f0 0x7f02a80cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:52.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.573+0000 7f02acfaf640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f02a80d0360 0x7f02a80b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:52.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.573+0000 7f02acfaf640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02a815d2a0 con 0x7f02a806f060 2024-05-08T05:46:52.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.573+0000 7f02acfaf640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f02a80d07c0 con 0x7f02a806e0f0 2024-05-08T05:46:52.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.573+0000 7f02acfaf640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f02a806f440 con 0x7f02a80d0360 2024-05-08T05:46:52.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.573+0000 7f02a5d9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f02a806e0f0 0x7f02a80cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:52.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.573+0000 7f02a5d9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f02a806e0f0 0x7f02a80cfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51760/0 (socket says 172.21.15.45:51760) 2024-05-08T05:46:52.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.573+0000 7f02a5d9b640 1 -- 172.21.15.45:0/3837685827 learned_addr learned my addr 172.21.15.45:0/3837685827 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:52.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.573+0000 7f02a6d9d640 1 --2- 172.21.15.45:0/3837685827 >> v2:172.21.15.63:3300/0 conn(0x7f02a80d0360 0x7f02a80b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:52.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.574+0000 7f02a5d9b640 1 -- 172.21.15.45:0/3837685827 >> v2:172.21.15.63:3300/0 conn(0x7f02a80d0360 msgr2=0x7f02a80b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:52.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.574+0000 7f02a5d9b640 1 --2- 172.21.15.45:0/3837685827 >> v2:172.21.15.63:3300/0 conn(0x7f02a80d0360 0x7f02a80b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:52.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.574+0000 7f02a5d9b640 1 -- 172.21.15.45:0/3837685827 >> v1:172.21.15.63:6789/0 conn(0x7f02a806f060 legacy=0x7f02a806fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:52.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.574+0000 7f02a5d9b640 1 -- 172.21.15.45:0/3837685827 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a81609b0 con 0x7f02a806e0f0 2024-05-08T05:46:52.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.574+0000 7f02a5d9b640 1 --2- 172.21.15.45:0/3837685827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02a806e0f0 0x7f02a80cfe20 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f02a809f050 tx=0x7f029c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24a57faf486169 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:52.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.574+0000 7f02a559a640 1 -- 172.21.15.45:0/3837685827 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c037070 con 0x7f02a806e0f0 2024-05-08T05:46:52.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.575+0000 7f02a559a640 1 -- 172.21.15.45:0/3837685827 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f029c032440 con 0x7f02a806e0f0 2024-05-08T05:46:52.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.575+0000 7f02a559a640 1 -- 172.21.15.45:0/3837685827 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c032810 con 0x7f02a806e0f0 2024-05-08T05:46:52.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.575+0000 7f02acfaf640 1 -- 172.21.15.45:0/3837685827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02a806e0f0 msgr2=0x7f02a80cfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:52.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.575+0000 7f02acfaf640 1 --2- 172.21.15.45:0/3837685827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02a806e0f0 0x7f02a80cfe20 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f02a809f050 tx=0x7f029c02a270 comp rx=0 tx=0).stop 2024-05-08T05:46:52.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.575+0000 7f02acfaf640 1 -- 172.21.15.45:0/3837685827 shutdown_connections 2024-05-08T05:46:52.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.575+0000 7f02acfaf640 1 --2- 172.21.15.45:0/3837685827 >> v2:172.21.15.63:3300/0 conn(0x7f02a80d0360 0x7f02a80b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:52.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.575+0000 7f02acfaf640 1 --2- 172.21.15.45:0/3837685827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02a806e0f0 0x7f02a80cfe20 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:52.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.575+0000 7f02acfaf640 1 -- 172.21.15.45:0/3837685827 >> 172.21.15.45:0/3837685827 conn(0x7f02a806c3a0 msgr2=0x7f02a806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:52.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.575+0000 7f02acfaf640 1 -- 172.21.15.45:0/3837685827 shutdown_connections 2024-05-08T05:46:52.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.575+0000 7f02acfaf640 1 -- 172.21.15.45:0/3837685827 wait complete. 2024-05-08T05:46:52.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.576+0000 7f02acfaf640 1 Processor -- start 2024-05-08T05:46:52.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.577+0000 7f02acfaf640 1 -- start start 2024-05-08T05:46:52.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.577+0000 7f02acfaf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02a806e0f0 0x7f02a81486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:52.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.577+0000 7f02acfaf640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02a806f060 0x7f02a809ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:52.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.577+0000 7f02acfaf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02a80d0360 0x7f02a8157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:52.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.577+0000 7f02acfaf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f02a8161250 con 0x7f02a80d0360 2024-05-08T05:46:52.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.577+0000 7f02acfaf640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f02a8160f70 con 0x7f02a806f060 2024-05-08T05:46:52.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.577+0000 7f02acfaf640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f02a8161530 con 0x7f02a806e0f0 2024-05-08T05:46:52.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.577+0000 7f02a6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02a80d0360 0x7f02a8157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:52.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.577+0000 7f02a6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02a80d0360 0x7f02a8157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51768/0 (socket says 172.21.15.45:51768) 2024-05-08T05:46:52.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.577+0000 7f02a6d9d640 1 -- 172.21.15.45:0/510229926 learned_addr learned my addr 172.21.15.45:0/510229926 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:52.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.578+0000 7f02a5d9b640 1 --2- 172.21.15.45:0/510229926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02a806f060 0x7f02a809ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:52.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.578+0000 7f02a659c640 1 --2- 172.21.15.45:0/510229926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02a806e0f0 0x7f02a81486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:52.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.578+0000 7f02a6d9d640 1 -- 172.21.15.45:0/510229926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02a806e0f0 msgr2=0x7f02a81486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:52.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.578+0000 7f02a6d9d640 1 --2- 172.21.15.45:0/510229926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02a806e0f0 0x7f02a81486f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:52.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.578+0000 7f02a6d9d640 1 -- 172.21.15.45:0/510229926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02a806f060 msgr2=0x7f02a809ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:52.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.578+0000 7f02a6d9d640 1 --2- 172.21.15.45:0/510229926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02a806f060 0x7f02a809ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:52.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.578+0000 7f02a6d9d640 1 -- 172.21.15.45:0/510229926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a809d4e0 con 0x7f02a80d0360 2024-05-08T05:46:52.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.578+0000 7f02a5d9b640 1 --2- 172.21.15.45:0/510229926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02a806f060 0x7f02a809ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:52.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.578+0000 7f02a6d9d640 1 --2- 172.21.15.45:0/510229926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02a80d0360 0x7f02a8157360 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f02a000efc0 tx=0x7f02a000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:52.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.578+0000 7f0296ffd640 1 -- 172.21.15.45:0/510229926 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a0018020 con 0x7f02a80d0360 2024-05-08T05:46:52.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.578+0000 7f0296ffd640 1 -- 172.21.15.45:0/510229926 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02a0014480 con 0x7f02a80d0360 2024-05-08T05:46:52.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.578+0000 7f02acfaf640 1 -- 172.21.15.45:0/510229926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a809d7c0 con 0x7f02a80d0360 2024-05-08T05:46:52.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.578+0000 7f0296ffd640 1 -- 172.21.15.45:0/510229926 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a0014870 con 0x7f02a80d0360 2024-05-08T05:46:52.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.578+0000 7f02acfaf640 1 -- 172.21.15.45:0/510229926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a809dd80 con 0x7f02a80d0360 2024-05-08T05:46:52.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.579+0000 7f02acfaf640 1 -- 172.21.15.45:0/510229926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a8072f90 con 0x7f02a80d0360 2024-05-08T05:46:52.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.580+0000 7f0296ffd640 1 -- 172.21.15.45:0/510229926 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f02a0005d10 con 0x7f02a80d0360 2024-05-08T05:46:52.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.580+0000 7f0296ffd640 1 --2- 172.21.15.45:0/510229926 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f02880762d0 0x7f0288078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:52.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.580+0000 7f0296ffd640 1 -- 172.21.15.45:0/510229926 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5421+0+0 (secure 0 0 0) 0x7f02a0097ff0 con 0x7f02a80d0360 2024-05-08T05:46:52.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.581+0000 7f02a659c640 1 --2- 172.21.15.45:0/510229926 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f02880762d0 0x7f0288078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:52.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.581+0000 7f02a659c640 1 --2- 172.21.15.45:0/510229926 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f02880762d0 0x7f0288078790 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0298001680 tx=0x7f0298009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:52.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.585+0000 7f0296ffd640 1 -- 172.21.15.45:0/510229926 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f02a8072f90 con 0x7f02a80d0360 2024-05-08T05:46:52.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.778+0000 7f02acfaf640 1 -- 172.21.15.45:0/510229926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02a80693f0 con 0x7f02a80d0360 2024-05-08T05:46:52.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.779+0000 7f0296ffd640 1 -- 172.21.15.45:0/510229926 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+13374 (secure 0 0 0) 0x7f02a0022070 con 0x7f02a80d0360 2024-05-08T05:46:52.779 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:46:52.779 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":79,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:46:51.447513+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/4183668301":"2024-05-09T05:46:48.438425+0000","172.21.15.45:6833/4183668301":"2024-05-09T05:46:48.438425+0000","172.21.15.63:6836/133050464":"2024-05-09T05:45:33.480648+0000","172.21.15.63:6834/133050464":"2024-05-09T05:45:33.480648+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:46:52.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.781+0000 7f02acfaf640 1 -- 172.21.15.45:0/510229926 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f02880762d0 msgr2=0x7f0288078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:52.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.781+0000 7f02acfaf640 1 --2- 172.21.15.45:0/510229926 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f02880762d0 0x7f0288078790 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0298001680 tx=0x7f0298009040 comp rx=0 tx=0).stop 2024-05-08T05:46:52.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.781+0000 7f02acfaf640 1 -- 172.21.15.45:0/510229926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02a80d0360 msgr2=0x7f02a8157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:52.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.781+0000 7f02acfaf640 1 --2- 172.21.15.45:0/510229926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02a80d0360 0x7f02a8157360 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f02a000efc0 tx=0x7f02a000c490 comp rx=0 tx=0).stop 2024-05-08T05:46:52.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.781+0000 7f02acfaf640 1 -- 172.21.15.45:0/510229926 shutdown_connections 2024-05-08T05:46:52.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.781+0000 7f02acfaf640 1 --2- 172.21.15.45:0/510229926 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f02880762d0 0x7f0288078790 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:52.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.781+0000 7f02acfaf640 1 --2- 172.21.15.45:0/510229926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f02a80d0360 0x7f02a8157360 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:52.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.781+0000 7f02acfaf640 1 --2- 172.21.15.45:0/510229926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02a806f060 0x7f02a809ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:52.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.781+0000 7f02acfaf640 1 --2- 172.21.15.45:0/510229926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f02a806e0f0 0x7f02a81486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:52.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.781+0000 7f02acfaf640 1 -- 172.21.15.45:0/510229926 >> 172.21.15.45:0/510229926 conn(0x7f02a806c3a0 msgr2=0x7f02a80731b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:52.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.781+0000 7f02acfaf640 1 -- 172.21.15.45:0/510229926 shutdown_connections 2024-05-08T05:46:52.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.781+0000 7f02acfaf640 1 -- 172.21.15.45:0/510229926 wait complete. 2024-05-08T05:46:52.793 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T05:41:18.847604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T05:46:52.793 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T05:46:52.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.899+0000 7ff9b7f58640 1 Processor -- start 2024-05-08T05:46:52.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.900+0000 7ff9b7f58640 1 -- start start 2024-05-08T05:46:52.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.900+0000 7ff9b7f58640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff9b00cfcd0 0x7ff9b00d4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:52.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.900+0000 7ff9b7f58640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff9b00d54e0 0x7ff9b00d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:52.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.900+0000 7ff9b7f58640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9b00d0320 con 0x7ff9b015f4e0 2024-05-08T05:46:52.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.900+0000 7ff9b7f58640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff9b00d41b0 con 0x7ff9b00cfcd0 2024-05-08T05:46:52.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.900+0000 7ff9b7f58640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff9b00d0600 con 0x7ff9b00d54e0 2024-05-08T05:46:52.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.901+0000 7ff9b54f3640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff9b00cfcd0 0x7ff9b00d4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:52.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.901+0000 7ff9b5cf4640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff9b015f4e0 0x7ff9b00d3aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46682/0 (socket says 172.21.15.45:46682) 2024-05-08T05:46:52.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.901+0000 7ff9b5cf4640 1 -- 172.21.15.45:0/3508087624 learned_addr learned my addr 172.21.15.45:0/3508087624 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:52.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.901+0000 7ff9b64f5640 1 --2- 172.21.15.45:0/3508087624 >> v2:172.21.15.63:3300/0 conn(0x7ff9b00d54e0 0x7ff9b00d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:52.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.901+0000 7ff9b54f3640 1 -- 172.21.15.45:0/3508087624 >> v2:172.21.15.63:3300/0 conn(0x7ff9b00d54e0 msgr2=0x7ff9b00d13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:52.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.901+0000 7ff9b54f3640 1 --2- 172.21.15.45:0/3508087624 >> v2:172.21.15.63:3300/0 conn(0x7ff9b00d54e0 0x7ff9b00d13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:52.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.901+0000 7ff9b54f3640 1 -- 172.21.15.45:0/3508087624 >> v1:172.21.15.45:6789/0 conn(0x7ff9b015f4e0 legacy=0x7ff9b00d3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:52.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.901+0000 7ff9b54f3640 1 -- 172.21.15.45:0/3508087624 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b00d08e0 con 0x7ff9b00cfcd0 2024-05-08T05:46:52.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.901+0000 7ff9b4cf2640 1 -- 172.21.15.45:0/3508087624 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 162080131 0 0) 0x7ff9b00d0320 con 0x7ff9b015f4e0 2024-05-08T05:46:52.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.902+0000 7ff9b54f3640 1 --2- 172.21.15.45:0/3508087624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9b00cfcd0 0x7ff9b00d4fa0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff9b008e9d0 tx=0x7ff9a002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3bd0abcf141e6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:52.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.902+0000 7ff9b4cf2640 1 -- 172.21.15.45:0/3508087624 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0037070 con 0x7ff9b00cfcd0 2024-05-08T05:46:52.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.902+0000 7ff9b4cf2640 1 -- 172.21.15.45:0/3508087624 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9a0032440 con 0x7ff9b00cfcd0 2024-05-08T05:46:52.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.902+0000 7ff9b4cf2640 1 -- 172.21.15.45:0/3508087624 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0032810 con 0x7ff9b00cfcd0 2024-05-08T05:46:52.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.902+0000 7ff9b7f58640 1 -- 172.21.15.45:0/3508087624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9b00cfcd0 msgr2=0x7ff9b00d4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:52.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.902+0000 7ff9b7f58640 1 --2- 172.21.15.45:0/3508087624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9b00cfcd0 0x7ff9b00d4fa0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff9b008e9d0 tx=0x7ff9a002a1a0 comp rx=0 tx=0).stop 2024-05-08T05:46:52.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.902+0000 7ff9b7f58640 1 -- 172.21.15.45:0/3508087624 shutdown_connections 2024-05-08T05:46:52.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.902+0000 7ff9b7f58640 1 --2- 172.21.15.45:0/3508087624 >> v2:172.21.15.63:3300/0 conn(0x7ff9b00d54e0 0x7ff9b00d13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:52.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.902+0000 7ff9b7f58640 1 --2- 172.21.15.45:0/3508087624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9b00cfcd0 0x7ff9b00d4fa0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:52.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.902+0000 7ff9b7f58640 1 -- 172.21.15.45:0/3508087624 >> 172.21.15.45:0/3508087624 conn(0x7ff9b008eac0 msgr2=0x7ff9b009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:52.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.903+0000 7ff9b7f58640 1 -- 172.21.15.45:0/3508087624 shutdown_connections 2024-05-08T05:46:52.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.903+0000 7ff9b7f58640 1 -- 172.21.15.45:0/3508087624 wait complete. 2024-05-08T05:46:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.904+0000 7ff9b7f58640 1 Processor -- start 2024-05-08T05:46:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.904+0000 7ff9b7f58640 1 -- start start 2024-05-08T05:46:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.904+0000 7ff9b7f58640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9b00cfcd0 0x7ff9b0159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.904+0000 7ff9b7f58640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9b00d54e0 0x7ff9b00ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.904+0000 7ff9b7f58640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9b015f4e0 0x7ff9b00ce630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:52.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.904+0000 7ff9b7f58640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b015f8c0 con 0x7ff9b00cfcd0 2024-05-08T05:46:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.904+0000 7ff9b7f58640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b00d0ea0 con 0x7ff9b00d54e0 2024-05-08T05:46:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.904+0000 7ff9b7f58640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b00d1900 con 0x7ff9b015f4e0 2024-05-08T05:46:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.905+0000 7ff9b5cf4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9b00cfcd0 0x7ff9b0159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.905+0000 7ff9b5cf4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9b00cfcd0 0x7ff9b0159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51788/0 (socket says 172.21.15.45:51788) 2024-05-08T05:46:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.905+0000 7ff9b5cf4640 1 -- 172.21.15.45:0/1103859921 learned_addr learned my addr 172.21.15.45:0/1103859921 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.905+0000 7ff9b64f5640 1 --2- 172.21.15.45:0/1103859921 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9b015f4e0 0x7ff9b00ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.905+0000 7ff9b54f3640 1 --2- 172.21.15.45:0/1103859921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9b00d54e0 0x7ff9b00ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.905+0000 7ff9b5cf4640 1 -- 172.21.15.45:0/1103859921 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9b015f4e0 msgr2=0x7ff9b00ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.905+0000 7ff9b5cf4640 1 --2- 172.21.15.45:0/1103859921 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9b015f4e0 0x7ff9b00ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.905+0000 7ff9b5cf4640 1 -- 172.21.15.45:0/1103859921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9b00d54e0 msgr2=0x7ff9b00ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.905+0000 7ff9b5cf4640 1 --2- 172.21.15.45:0/1103859921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9b00d54e0 0x7ff9b00ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.905+0000 7ff9b5cf4640 1 -- 172.21.15.45:0/1103859921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b0159bc0 con 0x7ff9b00cfcd0 2024-05-08T05:46:52.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.905+0000 7ff9b5cf4640 1 --2- 172.21.15.45:0/1103859921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9b00cfcd0 0x7ff9b0159680 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00c410 tx=0x7ff9ac00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:52.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.905+0000 7ff9a67fc640 1 -- 172.21.15.45:0/1103859921 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac0074e0 con 0x7ff9b00cfcd0 2024-05-08T05:46:52.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.905+0000 7ff9a67fc640 1 -- 172.21.15.45:0/1103859921 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9ac0047e0 con 0x7ff9b00cfcd0 2024-05-08T05:46:52.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.905+0000 7ff9a67fc640 1 -- 172.21.15.45:0/1103859921 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac004bd0 con 0x7ff9b00cfcd0 2024-05-08T05:46:52.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.905+0000 7ff9b7f58640 1 -- 172.21.15.45:0/1103859921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b00b8ad0 con 0x7ff9b00cfcd0 2024-05-08T05:46:52.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.905+0000 7ff9b7f58640 1 -- 172.21.15.45:0/1103859921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b00b93e0 con 0x7ff9b00cfcd0 2024-05-08T05:46:52.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.906+0000 7ff9b7f58640 1 -- 172.21.15.45:0/1103859921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b00693f0 con 0x7ff9b00cfcd0 2024-05-08T05:46:52.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.907+0000 7ff9a67fc640 1 -- 172.21.15.45:0/1103859921 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff9ac011070 con 0x7ff9b00cfcd0 2024-05-08T05:46:52.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.907+0000 7ff9a67fc640 1 --2- 172.21.15.45:0/1103859921 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff994076300 0x7ff9940787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:52.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.907+0000 7ff9a67fc640 1 -- 172.21.15.45:0/1103859921 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5421+0+0 (secure 0 0 0) 0x7ff9ac0970e0 con 0x7ff9b00cfcd0 2024-05-08T05:46:52.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.908+0000 7ff9b54f3640 1 --2- 172.21.15.45:0/1103859921 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff994076300 0x7ff9940787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:52.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.908+0000 7ff9b54f3640 1 --2- 172.21.15.45:0/1103859921 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff994076300 0x7ff9940787c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0034000 tx=0x7ff9a002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:52.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:52.912+0000 7ff9a67fc640 1 -- 172.21.15.45:0/1103859921 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff9b0159bc0 con 0x7ff9b00cfcd0 2024-05-08T05:46:53.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.103+0000 7ff9b7f58640 1 -- 172.21.15.45:0/1103859921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff9b00d2e80 con 0x7ff9b00cfcd0 2024-05-08T05:46:53.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.103+0000 7ff9a67fc640 1 -- 172.21.15.45:0/1103859921 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7ff9ac060b60 con 0x7ff9b00cfcd0 2024-05-08T05:46:53.103 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-05-08T05:46:53.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.105+0000 7ff9b7f58640 1 -- 172.21.15.45:0/1103859921 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff994076300 msgr2=0x7ff9940787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:53.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.105+0000 7ff9b7f58640 1 --2- 172.21.15.45:0/1103859921 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff994076300 0x7ff9940787c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0034000 tx=0x7ff9a002e000 comp rx=0 tx=0).stop 2024-05-08T05:46:53.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.105+0000 7ff9b7f58640 1 -- 172.21.15.45:0/1103859921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9b00cfcd0 msgr2=0x7ff9b0159680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:53.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.105+0000 7ff9b7f58640 1 --2- 172.21.15.45:0/1103859921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9b00cfcd0 0x7ff9b0159680 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00c410 tx=0x7ff9ac00c8d0 comp rx=0 tx=0).stop 2024-05-08T05:46:53.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.105+0000 7ff9b7f58640 1 -- 172.21.15.45:0/1103859921 shutdown_connections 2024-05-08T05:46:53.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.105+0000 7ff9b7f58640 1 --2- 172.21.15.45:0/1103859921 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff994076300 0x7ff9940787c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:53.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.105+0000 7ff9b7f58640 1 --2- 172.21.15.45:0/1103859921 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff9b015f4e0 0x7ff9b00ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:53.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.105+0000 7ff9b7f58640 1 --2- 172.21.15.45:0/1103859921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9b00d54e0 0x7ff9b00ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:53.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.105+0000 7ff9b7f58640 1 --2- 172.21.15.45:0/1103859921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff9b00cfcd0 0x7ff9b0159680 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:53.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.105+0000 7ff9b7f58640 1 -- 172.21.15.45:0/1103859921 >> 172.21.15.45:0/1103859921 conn(0x7ff9b008eac0 msgr2=0x7ff9b00b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:53.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.105+0000 7ff9b7f58640 1 -- 172.21.15.45:0/1103859921 shutdown_connections 2024-05-08T05:46:53.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.105+0000 7ff9b7f58640 1 -- 172.21.15.45:0/1103859921 wait complete. 2024-05-08T05:46:53.116 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-05-08T05:46:53.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.221+0000 7f9b787f0640 1 Processor -- start 2024-05-08T05:46:53.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.221+0000 7f9b787f0640 1 -- start start 2024-05-08T05:46:53.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.222+0000 7f9b787f0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b700d20c0 0x7f9b70163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:53.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.222+0000 7f9b787f0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9b70071110 0x7f9b7006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:53.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.222+0000 7f9b787f0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b700d1300 con 0x7f9b70070d30 2024-05-08T05:46:53.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.222+0000 7f9b787f0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f9b700d10d0 con 0x7f9b70071110 2024-05-08T05:46:53.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.222+0000 7f9b787f0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9b70070b80 con 0x7f9b700d20c0 2024-05-08T05:46:53.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.222+0000 7f9b76d8d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9b70071110 0x7f9b7006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:53.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.222+0000 7f9b7658c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9b70070d30 0x7f9b700703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46690/0 (socket says 172.21.15.45:46690) 2024-05-08T05:46:53.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.222+0000 7f9b7658c640 1 -- 172.21.15.45:0/804304906 learned_addr learned my addr 172.21.15.45:0/804304906 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:53.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.222+0000 7f9b75d8b640 1 --2- 172.21.15.45:0/804304906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b700d20c0 0x7f9b70163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:53.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.222+0000 7f9b76d8d640 1 -- 172.21.15.45:0/804304906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b700d20c0 msgr2=0x7f9b70163920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:53.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.222+0000 7f9b76d8d640 1 --2- 172.21.15.45:0/804304906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b700d20c0 0x7f9b70163920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:53.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.222+0000 7f9b76d8d640 1 -- 172.21.15.45:0/804304906 >> v1:172.21.15.45:6789/0 conn(0x7f9b70070d30 legacy=0x7f9b700703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:53.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.222+0000 7f9b76d8d640 1 -- 172.21.15.45:0/804304906 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b700d15e0 con 0x7f9b70071110 2024-05-08T05:46:53.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.222+0000 7f9b7558a640 1 -- 172.21.15.45:0/804304906 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1975828465 0 0) 0x7f9b700d1300 con 0x7f9b70070d30 2024-05-08T05:46:53.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.223+0000 7f9b76d8d640 1 --2- 172.21.15.45:0/804304906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b70071110 0x7f9b7006d4b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9b68009d90 tx=0x7f9b6802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5bf3fcb47220a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:53.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.223+0000 7f9b7558a640 1 -- 172.21.15.45:0/804304906 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68007740 con 0x7f9b70071110 2024-05-08T05:46:53.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.223+0000 7f9b7558a640 1 -- 172.21.15.45:0/804304906 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b680078e0 con 0x7f9b70071110 2024-05-08T05:46:53.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.223+0000 7f9b7558a640 1 -- 172.21.15.45:0/804304906 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68007cb0 con 0x7f9b70071110 2024-05-08T05:46:53.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.224+0000 7f9b787f0640 1 -- 172.21.15.45:0/804304906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b70071110 msgr2=0x7f9b7006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:53.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.224+0000 7f9b787f0640 1 --2- 172.21.15.45:0/804304906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b70071110 0x7f9b7006d4b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9b68009d90 tx=0x7f9b6802a660 comp rx=0 tx=0).stop 2024-05-08T05:46:53.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.224+0000 7f9b787f0640 1 -- 172.21.15.45:0/804304906 shutdown_connections 2024-05-08T05:46:53.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.224+0000 7f9b787f0640 1 --2- 172.21.15.45:0/804304906 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b70071110 0x7f9b7006d4b0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:53.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.224+0000 7f9b787f0640 1 --2- 172.21.15.45:0/804304906 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b700d20c0 0x7f9b70163920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:53.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.224+0000 7f9b787f0640 1 -- 172.21.15.45:0/804304906 >> 172.21.15.45:0/804304906 conn(0x7f9b700d2e90 msgr2=0x7f9b700d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:53.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.224+0000 7f9b787f0640 1 -- 172.21.15.45:0/804304906 shutdown_connections 2024-05-08T05:46:53.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.224+0000 7f9b787f0640 1 -- 172.21.15.45:0/804304906 wait complete. 2024-05-08T05:46:53.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.225+0000 7f9b787f0640 1 Processor -- start 2024-05-08T05:46:53.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.225+0000 7f9b787f0640 1 -- start start 2024-05-08T05:46:53.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.226+0000 7f9b787f0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b70070d30 0x7f9b701f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:53.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.226+0000 7f9b787f0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b70071110 0x7f9b70157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.226+0000 7f9b787f0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b700d20c0 0x7f9b701630b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.226+0000 7f9b787f0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9b70071580 con 0x7f9b70071110 2024-05-08T05:46:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.226+0000 7f9b787f0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9b700d1c30 con 0x7f9b700d20c0 2024-05-08T05:46:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.226+0000 7f9b787f0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9b70071700 con 0x7f9b70070d30 2024-05-08T05:46:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.226+0000 7f9b75d8b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b70071110 0x7f9b70157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.226+0000 7f9b75d8b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b70071110 0x7f9b70157470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51800/0 (socket says 172.21.15.45:51800) 2024-05-08T05:46:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.226+0000 7f9b75d8b640 1 -- 172.21.15.45:0/3952933323 learned_addr learned my addr 172.21.15.45:0/3952933323 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.226+0000 7f9b7658c640 1 --2- 172.21.15.45:0/3952933323 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b70070d30 0x7f9b701f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.226+0000 7f9b76d8d640 1 --2- 172.21.15.45:0/3952933323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b700d20c0 0x7f9b701630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.226+0000 7f9b75d8b640 1 -- 172.21.15.45:0/3952933323 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b70070d30 msgr2=0x7f9b701f8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.226+0000 7f9b75d8b640 1 --2- 172.21.15.45:0/3952933323 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b70070d30 0x7f9b701f8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.226+0000 7f9b75d8b640 1 -- 172.21.15.45:0/3952933323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b700d20c0 msgr2=0x7f9b701630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:53.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.226+0000 7f9b75d8b640 1 --2- 172.21.15.45:0/3952933323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b700d20c0 0x7f9b701630b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.226+0000 7f9b75d8b640 1 -- 172.21.15.45:0/3952933323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b700cb130 con 0x7f9b70071110 2024-05-08T05:46:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.226+0000 7f9b76d8d640 1 --2- 172.21.15.45:0/3952933323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b700d20c0 0x7f9b701630b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:46:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.227+0000 7f9b75d8b640 1 --2- 172.21.15.45:0/3952933323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b70071110 0x7f9b70157470 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9b6000e9b0 tx=0x7f9b6000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.227+0000 7f9b66ffd640 1 -- 172.21.15.45:0/3952933323 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b60017070 con 0x7f9b70071110 2024-05-08T05:46:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.227+0000 7f9b66ffd640 1 -- 172.21.15.45:0/3952933323 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b6000c9a0 con 0x7f9b70071110 2024-05-08T05:46:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.227+0000 7f9b787f0640 1 -- 172.21.15.45:0/3952933323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b700cb410 con 0x7f9b70071110 2024-05-08T05:46:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.227+0000 7f9b66ffd640 1 -- 172.21.15.45:0/3952933323 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6000cd90 con 0x7f9b70071110 2024-05-08T05:46:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.227+0000 7f9b787f0640 1 -- 172.21.15.45:0/3952933323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b700cb9d0 con 0x7f9b70071110 2024-05-08T05:46:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.228+0000 7f9b787f0640 1 -- 172.21.15.45:0/3952933323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b700693f0 con 0x7f9b70071110 2024-05-08T05:46:53.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.228+0000 7f9b66ffd640 1 -- 172.21.15.45:0/3952933323 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9b6001a070 con 0x7f9b70071110 2024-05-08T05:46:53.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.229+0000 7f9b66ffd640 1 --2- 172.21.15.45:0/3952933323 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9b54076300 0x7f9b540787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:53.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.229+0000 7f9b66ffd640 1 -- 172.21.15.45:0/3952933323 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5421+0+0 (secure 0 0 0) 0x7f9b60097890 con 0x7f9b70071110 2024-05-08T05:46:53.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.229+0000 7f9b7658c640 1 --2- 172.21.15.45:0/3952933323 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9b54076300 0x7f9b540787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:53.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.230+0000 7f9b7658c640 1 --2- 172.21.15.45:0/3952933323 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9b54076300 0x7f9b540787c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c000f60 tx=0x7f9b6c007c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:53.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.233+0000 7f9b66ffd640 1 -- 172.21.15.45:0/3952933323 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9b60061310 con 0x7f9b70071110 2024-05-08T05:46:53.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:53.507+0000 7f9b787f0640 1 -- 172.21.15.45:0/3952933323 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f9b700c9c70 con 0x7f9b70071110 2024-05-08T05:46:54.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.460+0000 7f9b66ffd640 1 -- 172.21.15.45:0/3952933323 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v263) ==== 170+0+0 (secure 0 0 0) 0x7f9b60060c10 con 0x7f9b70071110 2024-05-08T05:46:54.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.463+0000 7f9b787f0640 1 -- 172.21.15.45:0/3952933323 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9b54076300 msgr2=0x7f9b540787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:54.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.463+0000 7f9b787f0640 1 --2- 172.21.15.45:0/3952933323 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9b54076300 0x7f9b540787c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c000f60 tx=0x7f9b6c007c00 comp rx=0 tx=0).stop 2024-05-08T05:46:54.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.463+0000 7f9b787f0640 1 -- 172.21.15.45:0/3952933323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b70071110 msgr2=0x7f9b70157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:54.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.463+0000 7f9b787f0640 1 --2- 172.21.15.45:0/3952933323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b70071110 0x7f9b70157470 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9b6000e9b0 tx=0x7f9b6000ee70 comp rx=0 tx=0).stop 2024-05-08T05:46:54.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.463+0000 7f9b787f0640 1 -- 172.21.15.45:0/3952933323 shutdown_connections 2024-05-08T05:46:54.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.463+0000 7f9b787f0640 1 --2- 172.21.15.45:0/3952933323 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9b54076300 0x7f9b540787c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:54.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.463+0000 7f9b787f0640 1 --2- 172.21.15.45:0/3952933323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9b700d20c0 0x7f9b701630b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:54.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.463+0000 7f9b787f0640 1 --2- 172.21.15.45:0/3952933323 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b70071110 0x7f9b70157470 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:54.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.463+0000 7f9b787f0640 1 --2- 172.21.15.45:0/3952933323 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9b70070d30 0x7f9b701f8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:54.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.463+0000 7f9b787f0640 1 -- 172.21.15.45:0/3952933323 >> 172.21.15.45:0/3952933323 conn(0x7f9b700d2e90 msgr2=0x7f9b70160a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:54.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.463+0000 7f9b787f0640 1 -- 172.21.15.45:0/3952933323 shutdown_connections 2024-05-08T05:46:54.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.463+0000 7f9b787f0640 1 -- 172.21.15.45:0/3952933323 wait complete. 2024-05-08T05:46:54.477 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T05:46:54.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.590+0000 7fef5ef46640 1 Processor -- start 2024-05-08T05:46:54.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.590+0000 7fef5ef46640 1 -- start start 2024-05-08T05:46:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.591+0000 7fef5ef46640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef580b2490 0x7fef58070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.591+0000 7fef5ef46640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef58070d20 0x7fef580711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.591+0000 7fef5ef46640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef5806e110 con 0x7fef581629b0 2024-05-08T05:46:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.591+0000 7fef5ef46640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fef5806dec0 con 0x7fef58070d20 2024-05-08T05:46:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.591+0000 7fef5ef46640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fef5806e3f0 con 0x7fef580b2490 2024-05-08T05:46:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.591+0000 7fef5d4e3640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef58070d20 0x7fef580711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.591+0000 7fef5d4e3640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef58070d20 0x7fef580711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51812/0 (socket says 172.21.15.45:51812) 2024-05-08T05:46:54.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.591+0000 7fef5d4e3640 1 -- 172.21.15.45:0/1212434469 learned_addr learned my addr 172.21.15.45:0/1212434469 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.591+0000 7fef57fff640 1 --2- 172.21.15.45:0/1212434469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef580b2490 0x7fef58070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.592+0000 7fef5d4e3640 1 -- 172.21.15.45:0/1212434469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef580b2490 msgr2=0x7fef58070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.592+0000 7fef5d4e3640 1 --2- 172.21.15.45:0/1212434469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef580b2490 0x7fef58070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.592+0000 7fef5d4e3640 1 -- 172.21.15.45:0/1212434469 >> v1:172.21.15.63:6789/0 conn(0x7fef581629b0 legacy=0x7fef5806eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:54.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.592+0000 7fef5d4e3640 1 -- 172.21.15.45:0/1212434469 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5806e6d0 con 0x7fef58070d20 2024-05-08T05:46:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.592+0000 7fef5d4e3640 1 --2- 172.21.15.45:0/1212434469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef58070d20 0x7fef580711a0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fef50009e50 tx=0x7fef5002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c7b416bee33f92 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.593+0000 7fef577fe640 1 -- 172.21.15.45:0/1212434469 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef50036070 con 0x7fef58070d20 2024-05-08T05:46:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.593+0000 7fef577fe640 1 -- 172.21.15.45:0/1212434469 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef50031600 con 0x7fef58070d20 2024-05-08T05:46:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.593+0000 7fef577fe640 1 -- 172.21.15.45:0/1212434469 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef500319d0 con 0x7fef58070d20 2024-05-08T05:46:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.593+0000 7fef5ef46640 1 -- 172.21.15.45:0/1212434469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef58070d20 msgr2=0x7fef580711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.593+0000 7fef5ef46640 1 --2- 172.21.15.45:0/1212434469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef58070d20 0x7fef580711a0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fef50009e50 tx=0x7fef5002a710 comp rx=0 tx=0).stop 2024-05-08T05:46:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.593+0000 7fef5ef46640 1 -- 172.21.15.45:0/1212434469 shutdown_connections 2024-05-08T05:46:54.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.593+0000 7fef5ef46640 1 --2- 172.21.15.45:0/1212434469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef58070d20 0x7fef580711a0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.593+0000 7fef5ef46640 1 --2- 172.21.15.45:0/1212434469 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef580b2490 0x7fef58070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.593+0000 7fef5ef46640 1 -- 172.21.15.45:0/1212434469 >> 172.21.15.45:0/1212434469 conn(0x7fef580b49d0 msgr2=0x7fef580b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.593+0000 7fef5ef46640 1 -- 172.21.15.45:0/1212434469 shutdown_connections 2024-05-08T05:46:54.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.593+0000 7fef5ef46640 1 -- 172.21.15.45:0/1212434469 wait complete. 2024-05-08T05:46:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.594+0000 7fef5ef46640 1 Processor -- start 2024-05-08T05:46:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.594+0000 7fef5ef46640 1 -- start start 2024-05-08T05:46:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.595+0000 7fef5ef46640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef58070d20 0x7fef580cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:54.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.595+0000 7fef5ef46640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef580b2490 0x7fef581566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.595+0000 7fef5ef46640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef581629b0 0x7fef580c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.595+0000 7fef5ef46640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fef580709f0 con 0x7fef580b2490 2024-05-08T05:46:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.595+0000 7fef5ef46640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef58070870 con 0x7fef581629b0 2024-05-08T05:46:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.595+0000 7fef5ef46640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fef580afcb0 con 0x7fef58070d20 2024-05-08T05:46:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.596+0000 7fef57fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef580b2490 0x7fef581566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.596+0000 7fef57fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef580b2490 0x7fef581566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51820/0 (socket says 172.21.15.45:51820) 2024-05-08T05:46:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.596+0000 7fef57fff640 1 -- 172.21.15.45:0/2218470459 learned_addr learned my addr 172.21.15.45:0/2218470459 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.596+0000 7fef5d4e3640 1 --2- 172.21.15.45:0/2218470459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef581629b0 0x7fef580c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.596+0000 7fef5cce2640 1 --2- 172.21.15.45:0/2218470459 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef58070d20 0x7fef580cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.596+0000 7fef57fff640 1 -- 172.21.15.45:0/2218470459 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef58070d20 msgr2=0x7fef580cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.596+0000 7fef57fff640 1 --2- 172.21.15.45:0/2218470459 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef58070d20 0x7fef580cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.596+0000 7fef57fff640 1 -- 172.21.15.45:0/2218470459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef581629b0 msgr2=0x7fef580c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:54.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.596+0000 7fef57fff640 1 --2- 172.21.15.45:0/2218470459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef581629b0 0x7fef580c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.596+0000 7fef57fff640 1 -- 172.21.15.45:0/2218470459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef580c7c20 con 0x7fef580b2490 2024-05-08T05:46:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.596+0000 7fef5d4e3640 1 --2- 172.21.15.45:0/2218470459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef581629b0 0x7fef580c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.596+0000 7fef57fff640 1 --2- 172.21.15.45:0/2218470459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef580b2490 0x7fef581566a0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00e9b0 tx=0x7fef4c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.596+0000 7fef557fa640 1 -- 172.21.15.45:0/2218470459 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c017070 con 0x7fef580b2490 2024-05-08T05:46:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.596+0000 7fef557fa640 1 -- 172.21.15.45:0/2218470459 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef4c00c9a0 con 0x7fef580b2490 2024-05-08T05:46:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.596+0000 7fef557fa640 1 -- 172.21.15.45:0/2218470459 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c00cd90 con 0x7fef580b2490 2024-05-08T05:46:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.596+0000 7fef5ef46640 1 -- 172.21.15.45:0/2218470459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef580b5ac0 con 0x7fef580b2490 2024-05-08T05:46:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.596+0000 7fef5ef46640 1 -- 172.21.15.45:0/2218470459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef580b6080 con 0x7fef580b2490 2024-05-08T05:46:54.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.597+0000 7fef5ef46640 1 -- 172.21.15.45:0/2218470459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef580693f0 con 0x7fef580b2490 2024-05-08T05:46:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.598+0000 7fef557fa640 1 -- 172.21.15.45:0/2218470459 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fef4c01a070 con 0x7fef580b2490 2024-05-08T05:46:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.598+0000 7fef557fa640 1 --2- 172.21.15.45:0/2218470459 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef440762d0 0x7fef44078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.599+0000 7fef557fa640 1 -- 172.21.15.45:0/2218470459 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5421+0+0 (secure 0 0 0) 0x7fef4c0978a0 con 0x7fef580b2490 2024-05-08T05:46:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.599+0000 7fef5cce2640 1 --2- 172.21.15.45:0/2218470459 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef440762d0 0x7fef44078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:54.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.599+0000 7fef5cce2640 1 --2- 172.21.15.45:0/2218470459 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef440762d0 0x7fef44078790 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fef480015e0 tx=0x7fef48009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.603+0000 7fef557fa640 1 -- 172.21.15.45:0/2218470459 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fef4c061320 con 0x7fef580b2490 2024-05-08T05:46:54.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.794+0000 7fef5ef46640 1 -- 172.21.15.45:0/2218470459 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fef580b88d0 con 0x7fef580b2490 2024-05-08T05:46:54.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.797+0000 7fef557fa640 1 -- 172.21.15.45:0/2218470459 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7fef4c060c20 con 0x7fef580b2490 2024-05-08T05:46:54.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.799+0000 7fef5ef46640 1 -- 172.21.15.45:0/2218470459 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef440762d0 msgr2=0x7fef44078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:54.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.799+0000 7fef5ef46640 1 --2- 172.21.15.45:0/2218470459 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef440762d0 0x7fef44078790 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fef480015e0 tx=0x7fef48009040 comp rx=0 tx=0).stop 2024-05-08T05:46:54.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.799+0000 7fef5ef46640 1 -- 172.21.15.45:0/2218470459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef580b2490 msgr2=0x7fef581566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:54.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.799+0000 7fef5ef46640 1 --2- 172.21.15.45:0/2218470459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef580b2490 0x7fef581566a0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00e9b0 tx=0x7fef4c00ee70 comp rx=0 tx=0).stop 2024-05-08T05:46:54.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.800+0000 7fef5ef46640 1 -- 172.21.15.45:0/2218470459 shutdown_connections 2024-05-08T05:46:54.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.800+0000 7fef5ef46640 1 --2- 172.21.15.45:0/2218470459 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef440762d0 0x7fef44078790 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:54.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.800+0000 7fef5ef46640 1 --2- 172.21.15.45:0/2218470459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef581629b0 0x7fef580c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:54.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.800+0000 7fef5ef46640 1 --2- 172.21.15.45:0/2218470459 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef580b2490 0x7fef581566a0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:54.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.800+0000 7fef5ef46640 1 --2- 172.21.15.45:0/2218470459 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef58070d20 0x7fef580cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:54.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.800+0000 7fef5ef46640 1 -- 172.21.15.45:0/2218470459 >> 172.21.15.45:0/2218470459 conn(0x7fef580b49d0 msgr2=0x7fef580d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:54.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.800+0000 7fef5ef46640 1 -- 172.21.15.45:0/2218470459 shutdown_connections 2024-05-08T05:46:54.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:54.800+0000 7fef5ef46640 1 -- 172.21.15.45:0/2218470459 wait complete. 2024-05-08T05:46:54.811 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T05:46:54.811 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:46:54.848 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:46:54.890 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T05:46:54.890 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:46:54.890 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:46:54.906 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:46:54.906 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:46:54.906 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:46:54.948 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:46:54.948 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:46:54.948 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:46:54.965 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:46:54.965 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:46:54.965 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:46:55.007 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:46:55.007 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T05:46:55.007 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:46:55.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.110+0000 7f053412c640 1 Processor -- start 2024-05-08T05:46:55.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.110+0000 7f053412c640 1 -- start start 2024-05-08T05:46:55.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.111+0000 7f053412c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f052c0733e0 0x7f052c0d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:55.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.111+0000 7f053412c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f052c0cff00 con 0x7f052c15ce90 2024-05-08T05:46:55.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.111+0000 7f053412c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f052c073950 con 0x7f052c06d3b0 2024-05-08T05:46:55.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.111+0000 7f053412c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f052c0d01a0 con 0x7f052c0733e0 2024-05-08T05:46:55.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.111+0000 7f05316c7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f052c0733e0 0x7f052c0d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:55.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.111+0000 7f0531ec8640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f052c15ce90 0x7f052c070d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46694/0 (socket says 172.21.15.45:46694) 2024-05-08T05:46:55.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.111+0000 7f05316c7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f052c0733e0 0x7f052c0d0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51828/0 (socket says 172.21.15.45:51828) 2024-05-08T05:46:55.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.111+0000 7f0531ec8640 1 -- 172.21.15.45:0/323258318 learned_addr learned my addr 172.21.15.45:0/323258318 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:55.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.112+0000 7f05316c7640 1 -- 172.21.15.45:0/323258318 >> v1:172.21.15.63:6789/0 conn(0x7f052c06d3b0 legacy=0x7f052c0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:55.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.112+0000 7f05316c7640 1 -- 172.21.15.45:0/323258318 >> v1:172.21.15.45:6789/0 conn(0x7f052c15ce90 legacy=0x7f052c070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:55.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.112+0000 7f05316c7640 1 -- 172.21.15.45:0/323258318 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052c0d0480 con 0x7f052c0733e0 2024-05-08T05:46:55.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.112+0000 7f0530ec6640 1 -- 172.21.15.45:0/323258318 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1213262383 0 0) 0x7f052c0cff00 con 0x7f052c15ce90 2024-05-08T05:46:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.112+0000 7f05316c7640 1 --2- 172.21.15.45:0/323258318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052c0733e0 0x7f052c0d0c80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f05200097a0 tx=0x7f052002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff889f4ec72af74 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.113+0000 7f0530ec6640 1 -- 172.21.15.45:0/323258318 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0520037070 con 0x7f052c0733e0 2024-05-08T05:46:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.113+0000 7f0530ec6640 1 -- 172.21.15.45:0/323258318 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0520032440 con 0x7f052c0733e0 2024-05-08T05:46:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.113+0000 7f0530ec6640 1 -- 172.21.15.45:0/323258318 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0520032810 con 0x7f052c0733e0 2024-05-08T05:46:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.113+0000 7f053412c640 1 -- 172.21.15.45:0/323258318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052c0733e0 msgr2=0x7f052c0d0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.113+0000 7f053412c640 1 --2- 172.21.15.45:0/323258318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052c0733e0 0x7f052c0d0c80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f05200097a0 tx=0x7f052002a180 comp rx=0 tx=0).stop 2024-05-08T05:46:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.113+0000 7f053412c640 1 -- 172.21.15.45:0/323258318 shutdown_connections 2024-05-08T05:46:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.113+0000 7f053412c640 1 --2- 172.21.15.45:0/323258318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052c0733e0 0x7f052c0d0c80 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.113+0000 7f053412c640 1 -- 172.21.15.45:0/323258318 >> 172.21.15.45:0/323258318 conn(0x7f052c0728c0 msgr2=0x7f052c072cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.113+0000 7f053412c640 1 -- 172.21.15.45:0/323258318 shutdown_connections 2024-05-08T05:46:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.113+0000 7f053412c640 1 -- 172.21.15.45:0/323258318 wait complete. 2024-05-08T05:46:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.115+0000 7f053412c640 1 Processor -- start 2024-05-08T05:46:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.115+0000 7f053412c640 1 -- start start 2024-05-08T05:46:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.115+0000 7f053412c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052c06d3b0 0x7f052c1430b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.115+0000 7f053412c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f052c0733e0 0x7f052c158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.115+0000 7f053412c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f052c15ce90 0x7f052c15e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.115+0000 7f053412c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f052c15d3f0 con 0x7f052c06d3b0 2024-05-08T05:46:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.115+0000 7f053412c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f052c15d270 con 0x7f052c15ce90 2024-05-08T05:46:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.115+0000 7f053412c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f052c0d11c0 con 0x7f052c0733e0 2024-05-08T05:46:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.115+0000 7f0531ec8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052c06d3b0 0x7f052c1430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.115+0000 7f0531ec8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052c06d3b0 0x7f052c1430b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51834/0 (socket says 172.21.15.45:51834) 2024-05-08T05:46:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.115+0000 7f0531ec8640 1 -- 172.21.15.45:0/2031820189 learned_addr learned my addr 172.21.15.45:0/2031820189 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:55.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.116+0000 7f0531ec8640 1 -- 172.21.15.45:0/2031820189 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f052c0733e0 msgr2=0x7f052c158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.116+0000 7f0531ec8640 1 --2- 172.21.15.45:0/2031820189 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f052c0733e0 0x7f052c158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.116+0000 7f0531ec8640 1 -- 172.21.15.45:0/2031820189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f052c15ce90 msgr2=0x7f052c15e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.116+0000 7f05326c9640 1 --2- 172.21.15.45:0/2031820189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f052c15ce90 0x7f052c15e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.116+0000 7f0531ec8640 1 --2- 172.21.15.45:0/2031820189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f052c15ce90 0x7f052c15e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.116+0000 7f0531ec8640 1 -- 172.21.15.45:0/2031820189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052c15ecc0 con 0x7f052c06d3b0 2024-05-08T05:46:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.116+0000 7f0531ec8640 1 --2- 172.21.15.45:0/2031820189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052c06d3b0 0x7f052c1430b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f052400c490 tx=0x7f052400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.116+0000 7f052a7fc640 1 -- 172.21.15.45:0/2031820189 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0524018070 con 0x7f052c06d3b0 2024-05-08T05:46:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.116+0000 7f052a7fc640 1 -- 172.21.15.45:0/2031820189 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0524007500 con 0x7f052c06d3b0 2024-05-08T05:46:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.116+0000 7f053412c640 1 -- 172.21.15.45:0/2031820189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f052c1574c0 con 0x7f052c06d3b0 2024-05-08T05:46:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.116+0000 7f052a7fc640 1 -- 172.21.15.45:0/2031820189 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0524004a20 con 0x7f052c06d3b0 2024-05-08T05:46:55.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.116+0000 7f053412c640 1 -- 172.21.15.45:0/2031820189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f052c157a80 con 0x7f052c06d3b0 2024-05-08T05:46:55.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.117+0000 7f053412c640 1 -- 172.21.15.45:0/2031820189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f052c0693f0 con 0x7f052c06d3b0 2024-05-08T05:46:55.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.118+0000 7f052a7fc640 1 -- 172.21.15.45:0/2031820189 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f052401b070 con 0x7f052c06d3b0 2024-05-08T05:46:55.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.118+0000 7f052a7fc640 1 --2- 172.21.15.45:0/2031820189 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f05180762d0 0x7f0518078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:55.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.118+0000 7f052a7fc640 1 -- 172.21.15.45:0/2031820189 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5421+0+0 (secure 0 0 0) 0x7f0524096700 con 0x7f052c06d3b0 2024-05-08T05:46:55.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.119+0000 7f05316c7640 1 --2- 172.21.15.45:0/2031820189 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f05180762d0 0x7f0518078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:55.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.119+0000 7f05316c7640 1 --2- 172.21.15.45:0/2031820189 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f05180762d0 0x7f0518078790 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0520034000 tx=0x7f052002c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:55.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.123+0000 7f052a7fc640 1 -- 172.21.15.45:0/2031820189 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0524060210 con 0x7f052c06d3b0 2024-05-08T05:46:55.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.350+0000 7f053412c640 1 -- 172.21.15.45:0/2031820189 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f052c15c010 con 0x7f052c06d3b0 2024-05-08T05:46:55.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.351+0000 7f052a7fc640 1 -- 172.21.15.45:0/2031820189 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+3869 (secure 0 0 0) 0x7f052405fb10 con 0x7f052c06d3b0 2024-05-08T05:46:55.351 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:46:55.351 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":54,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5722,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1283757699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1283757699},{"type":"v1","addr":"172.21.15.63:6837","nonce":1283757699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2823770504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2823770504},{"type":"v1","addr":"172.21.15.63:6836","nonce":2823770504}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/1963062796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1963062796},{"type":"v1","addr":"172.21.15.45:6834","nonce":1963062796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6663,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4212474495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":4212474495},{"type":"v1","addr":"172.21.15.45:6835","nonce":4212474495}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[]} 2024-05-08T05:46:55.352 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 54 2024-05-08T05:46:55.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.353+0000 7f053412c640 1 -- 172.21.15.45:0/2031820189 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f05180762d0 msgr2=0x7f0518078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:55.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.353+0000 7f053412c640 1 --2- 172.21.15.45:0/2031820189 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f05180762d0 0x7f0518078790 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0520034000 tx=0x7f052002c000 comp rx=0 tx=0).stop 2024-05-08T05:46:55.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.353+0000 7f053412c640 1 -- 172.21.15.45:0/2031820189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052c06d3b0 msgr2=0x7f052c1430b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:55.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.353+0000 7f053412c640 1 --2- 172.21.15.45:0/2031820189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052c06d3b0 0x7f052c1430b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f052400c490 tx=0x7f052400c950 comp rx=0 tx=0).stop 2024-05-08T05:46:55.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.353+0000 7f053412c640 1 -- 172.21.15.45:0/2031820189 shutdown_connections 2024-05-08T05:46:55.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.353+0000 7f053412c640 1 --2- 172.21.15.45:0/2031820189 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f05180762d0 0x7f0518078790 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:55.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.353+0000 7f053412c640 1 --2- 172.21.15.45:0/2031820189 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f052c0733e0 0x7f052c158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:55.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.353+0000 7f053412c640 1 --2- 172.21.15.45:0/2031820189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f052c15ce90 0x7f052c15e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:55.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.353+0000 7f053412c640 1 --2- 172.21.15.45:0/2031820189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f052c06d3b0 0x7f052c1430b0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:55.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.353+0000 7f053412c640 1 -- 172.21.15.45:0/2031820189 >> 172.21.15.45:0/2031820189 conn(0x7f052c0728c0 msgr2=0x7f052c0b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:55.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.353+0000 7f053412c640 1 -- 172.21.15.45:0/2031820189 shutdown_connections 2024-05-08T05:46:55.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.353+0000 7f053412c640 1 -- 172.21.15.45:0/2031820189 wait complete. 2024-05-08T05:46:55.364 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T05:46:55.364 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T05:46:55.364 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T05:46:55.364 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:46:55.365 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T05:46:55.365 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T05:46:55.365 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T05:46:55.365 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:46:55.365 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T05:46:55.365 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T05:46:55.366 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T05:46:55.366 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:46:55.366 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T05:46:55.366 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T05:46:55.366 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T05:46:55.366 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:46:55.374 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T05:46:55.374 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T05:46:55.375 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T05:46:55.375 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T05:46:55.378 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T05:46:55.378 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T05:46:55.379 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T05:46:55.379 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T05:46:55.416 INFO:tasks.ceph.mds.a:Started 2024-05-08T05:46:55.417 DEBUG:teuthology.parallel:result is None 2024-05-08T05:46:55.417 INFO:tasks.ceph.mds.c:Started 2024-05-08T05:46:55.417 DEBUG:teuthology.parallel:result is None 2024-05-08T05:46:55.424 INFO:tasks.ceph.mds.b:Started 2024-05-08T05:46:55.424 DEBUG:teuthology.parallel:result is None 2024-05-08T05:46:55.425 INFO:tasks.ceph.mds.d:Started 2024-05-08T05:46:55.425 DEBUG:teuthology.parallel:result is None 2024-05-08T05:46:55.425 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T05:46:55.496 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-05-08T05:46:55.496 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-05-08T05:46:55.511 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-08T05:46:55.511 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-08T05:46:55.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.533+0000 7f9ec8de4640 1 Processor -- start 2024-05-08T05:46:55.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.533+0000 7f9ec8de4640 1 -- start start 2024-05-08T05:46:55.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.534+0000 7f9ec8de4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ec415f2d0 0x7f9ec40b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:55.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.534+0000 7f9ec8de4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ec40d2310 0x7f9ec40d5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:55.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.534+0000 7f9ec8de4640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ec41606c0 con 0x7f9ec40d4fc0 2024-05-08T05:46:55.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.534+0000 7f9ec8de4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f9ec40d2780 con 0x7f9ec415f2d0 2024-05-08T05:46:55.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.534+0000 7f9ec8de4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec40d5960 con 0x7f9ec40d2310 2024-05-08T05:46:55.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.534+0000 7f9ec259c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ec415f2d0 0x7f9ec40b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:55.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.534+0000 7f9ec259c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9ec415f2d0 0x7f9ec40b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51864/0 (socket says 172.21.15.45:51864) 2024-05-08T05:46:55.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.534+0000 7f9ec259c640 1 -- 172.21.15.45:0/2865487986 learned_addr learned my addr 172.21.15.45:0/2865487986 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:55.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.535+0000 7f9ec259c640 1 -- 172.21.15.45:0/2865487986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ec40d2310 msgr2=0x7f9ec40d5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:55.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.535+0000 7f9ec2d9d640 1 --2- 172.21.15.45:0/2865487986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ec40d2310 0x7f9ec40d5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:55.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.535+0000 7f9ec259c640 1 --2- 172.21.15.45:0/2865487986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ec40d2310 0x7f9ec40d5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:55.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.535+0000 7f9ec259c640 1 -- 172.21.15.45:0/2865487986 >> v1:172.21.15.63:6789/0 conn(0x7f9ec40d4fc0 legacy=0x7f9ec40d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:55.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.535+0000 7f9ec259c640 1 -- 172.21.15.45:0/2865487986 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec40b4f00 con 0x7f9ec415f2d0 2024-05-08T05:46:55.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.535+0000 7f9ec2d9d640 1 --2- 172.21.15.45:0/2865487986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ec40d2310 0x7f9ec40d5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:46:55.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.535+0000 7f9ec259c640 1 --2- 172.21.15.45:0/2865487986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ec415f2d0 0x7f9ec40b5ca0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009870 tx=0x7f9eb402a1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2843de665e9ee7f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:55.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.535+0000 7f9ec159a640 1 -- 172.21.15.45:0/2865487986 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4037070 con 0x7f9ec415f2d0 2024-05-08T05:46:55.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.535+0000 7f9ec159a640 1 -- 172.21.15.45:0/2865487986 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb4032440 con 0x7f9ec415f2d0 2024-05-08T05:46:55.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.536+0000 7f9ec159a640 1 -- 172.21.15.45:0/2865487986 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4032810 con 0x7f9ec415f2d0 2024-05-08T05:46:55.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.536+0000 7f9ec8de4640 1 -- 172.21.15.45:0/2865487986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ec415f2d0 msgr2=0x7f9ec40b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:55.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.536+0000 7f9ec8de4640 1 --2- 172.21.15.45:0/2865487986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ec415f2d0 0x7f9ec40b5ca0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009870 tx=0x7f9eb402a1b0 comp rx=0 tx=0).stop 2024-05-08T05:46:55.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.536+0000 7f9ec8de4640 1 -- 172.21.15.45:0/2865487986 shutdown_connections 2024-05-08T05:46:55.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.536+0000 7f9ec8de4640 1 --2- 172.21.15.45:0/2865487986 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ec40d2310 0x7f9ec40d5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:55.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.536+0000 7f9ec8de4640 1 --2- 172.21.15.45:0/2865487986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ec415f2d0 0x7f9ec40b5ca0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:55.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.536+0000 7f9ec8de4640 1 -- 172.21.15.45:0/2865487986 >> 172.21.15.45:0/2865487986 conn(0x7f9ec40746d0 msgr2=0x7f9ec409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:55.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.536+0000 7f9ec8de4640 1 -- 172.21.15.45:0/2865487986 shutdown_connections 2024-05-08T05:46:55.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.536+0000 7f9ec8de4640 1 -- 172.21.15.45:0/2865487986 wait complete. 2024-05-08T05:46:55.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.537+0000 7f9ec8de4640 1 Processor -- start 2024-05-08T05:46:55.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.538+0000 7f9ec8de4640 1 -- start start 2024-05-08T05:46:55.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.538+0000 7f9ec8de4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ec40d2310 0x7f9ec40cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:55.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.538+0000 7f9ec8de4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ec40d4fc0 0x7f9ec4159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:55.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.538+0000 7f9ec8de4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ec415f2d0 0x7f9ec40cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:55.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.538+0000 7f9ec8de4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec40b54a0 con 0x7f9ec40d4fc0 2024-05-08T05:46:55.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.538+0000 7f9ec8de4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec40b5810 con 0x7f9ec415f2d0 2024-05-08T05:46:55.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.538+0000 7f9ec8de4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec40d2c00 con 0x7f9ec40d2310 2024-05-08T05:46:55.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.538+0000 7f9ec1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ec40d4fc0 0x7f9ec4159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:55.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.538+0000 7f9ec1d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ec40d4fc0 0x7f9ec4159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51878/0 (socket says 172.21.15.45:51878) 2024-05-08T05:46:55.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.538+0000 7f9ec1d9b640 1 -- 172.21.15.45:0/4048233406 learned_addr learned my addr 172.21.15.45:0/4048233406 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:55.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.539+0000 7f9ec2d9d640 1 --2- 172.21.15.45:0/4048233406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ec415f2d0 0x7f9ec40cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:55.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.539+0000 7f9ec259c640 1 --2- 172.21.15.45:0/4048233406 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ec40d2310 0x7f9ec40cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:55.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.539+0000 7f9ec1d9b640 1 -- 172.21.15.45:0/4048233406 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ec40d2310 msgr2=0x7f9ec40cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:55.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.539+0000 7f9ec1d9b640 1 --2- 172.21.15.45:0/4048233406 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ec40d2310 0x7f9ec40cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:55.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.539+0000 7f9ec1d9b640 1 -- 172.21.15.45:0/4048233406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ec415f2d0 msgr2=0x7f9ec40cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:55.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.539+0000 7f9ec1d9b640 1 --2- 172.21.15.45:0/4048233406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ec415f2d0 0x7f9ec40cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:55.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.539+0000 7f9ec1d9b640 1 -- 172.21.15.45:0/4048233406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec40cc870 con 0x7f9ec40d4fc0 2024-05-08T05:46:55.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.539+0000 7f9ec1d9b640 1 --2- 172.21.15.45:0/4048233406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ec40d4fc0 0x7f9ec4159640 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800c460 tx=0x7f9eb800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:55.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.539+0000 7f9eb2ffd640 1 -- 172.21.15.45:0/4048233406 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8016020 con 0x7f9ec40d4fc0 2024-05-08T05:46:55.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.539+0000 7f9eb2ffd640 1 -- 172.21.15.45:0/4048233406 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb8012870 con 0x7f9ec40d4fc0 2024-05-08T05:46:55.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.539+0000 7f9ec8de4640 1 -- 172.21.15.45:0/4048233406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec40ccb50 con 0x7f9ec40d4fc0 2024-05-08T05:46:55.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.539+0000 7f9eb2ffd640 1 -- 172.21.15.45:0/4048233406 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8012c60 con 0x7f9ec40d4fc0 2024-05-08T05:46:55.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.539+0000 7f9ec8de4640 1 -- 172.21.15.45:0/4048233406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec40cd110 con 0x7f9ec40d4fc0 2024-05-08T05:46:55.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.540+0000 7f9ec8de4640 1 -- 172.21.15.45:0/4048233406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ec4069b00 con 0x7f9ec40d4fc0 2024-05-08T05:46:55.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.541+0000 7f9eb2ffd640 1 -- 172.21.15.45:0/4048233406 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9eb8019020 con 0x7f9ec40d4fc0 2024-05-08T05:46:55.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.541+0000 7f9eb2ffd640 1 --2- 172.21.15.45:0/4048233406 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ea40762d0 0x7f9ea4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:55.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.541+0000 7f9eb2ffd640 1 -- 172.21.15.45:0/4048233406 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5421+0+0 (secure 0 0 0) 0x7f9eb8097420 con 0x7f9ec40d4fc0 2024-05-08T05:46:55.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.542+0000 7f9ec259c640 1 --2- 172.21.15.45:0/4048233406 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ea40762d0 0x7f9ea4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:55.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.542+0000 7f9ec259c640 1 --2- 172.21.15.45:0/4048233406 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ea40762d0 0x7f9ea4078790 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4034000 tx=0x7f9eb402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:55.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.546+0000 7f9eb2ffd640 1 -- 172.21.15.45:0/4048233406 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9eb8020020 con 0x7f9ec40d4fc0 2024-05-08T05:46:55.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:55.750+0000 7f9ec8de4640 1 -- 172.21.15.45:0/4048233406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f9ec40b4b60 con 0x7f9ec40d4fc0 2024-05-08T05:46:56.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.818+0000 7f9eb2ffd640 1 -- 172.21.15.45:0/4048233406 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v84) ==== 97+0+0 (secure 0 0 0) 0x7f9eb80607b0 con 0x7f9ec40d4fc0 2024-05-08T05:46:56.819 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2024-05-08T05:46:56.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.822+0000 7f9ec8de4640 1 -- 172.21.15.45:0/4048233406 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ea40762d0 msgr2=0x7f9ea4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:56.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.822+0000 7f9ec8de4640 1 --2- 172.21.15.45:0/4048233406 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ea40762d0 0x7f9ea4078790 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4034000 tx=0x7f9eb402e000 comp rx=0 tx=0).stop 2024-05-08T05:46:56.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.822+0000 7f9ec8de4640 1 -- 172.21.15.45:0/4048233406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ec40d4fc0 msgr2=0x7f9ec4159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:56.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.822+0000 7f9ec8de4640 1 --2- 172.21.15.45:0/4048233406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ec40d4fc0 0x7f9ec4159640 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800c460 tx=0x7f9eb800c920 comp rx=0 tx=0).stop 2024-05-08T05:46:56.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.822+0000 7f9ec8de4640 1 -- 172.21.15.45:0/4048233406 shutdown_connections 2024-05-08T05:46:56.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.822+0000 7f9ec8de4640 1 --2- 172.21.15.45:0/4048233406 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9ea40762d0 0x7f9ea4078790 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:56.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.822+0000 7f9ec8de4640 1 --2- 172.21.15.45:0/4048233406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ec415f2d0 0x7f9ec40cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:56.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.822+0000 7f9ec8de4640 1 --2- 172.21.15.45:0/4048233406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ec40d4fc0 0x7f9ec4159640 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:56.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.822+0000 7f9ec8de4640 1 --2- 172.21.15.45:0/4048233406 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9ec40d2310 0x7f9ec40cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:56.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.822+0000 7f9ec8de4640 1 -- 172.21.15.45:0/4048233406 >> 172.21.15.45:0/4048233406 conn(0x7f9ec40746d0 msgr2=0x7f9ec40b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:56.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.822+0000 7f9ec8de4640 1 -- 172.21.15.45:0/4048233406 shutdown_connections 2024-05-08T05:46:56.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.822+0000 7f9ec8de4640 1 -- 172.21.15.45:0/4048233406 wait complete. 2024-05-08T05:46:56.836 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T05:46:56.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.947+0000 7f8c2bd46640 1 Processor -- start 2024-05-08T05:46:56.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.948+0000 7f8c2bd46640 1 -- start start 2024-05-08T05:46:56.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.949+0000 7f8c2bd46640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8c240d1970 0x7f8c240d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:56.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.949+0000 7f8c2bd46640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c241630b0 con 0x7f8c2415f420 2024-05-08T05:46:56.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.949+0000 7f8c2bd46640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c240d3e70 con 0x7f8c240d4cc0 2024-05-08T05:46:56.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.949+0000 7f8c2bd46640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f8c240d2550 con 0x7f8c240d1970 2024-05-08T05:46:56.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.949+0000 7f8c292e1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8c240d1970 0x7f8c240d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:56.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.949+0000 7f8c29ae2640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8c2415f420 0x7f8c240d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46724/0 (socket says 172.21.15.45:46724) 2024-05-08T05:46:56.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.949+0000 7f8c29ae2640 1 -- 172.21.15.45:0/30349870 learned_addr learned my addr 172.21.15.45:0/30349870 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:56.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.949+0000 7f8c292e1640 1 -- 172.21.15.45:0/30349870 >> v1:172.21.15.63:6789/0 conn(0x7f8c240d4cc0 legacy=0x7f8c240d50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:56.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.949+0000 7f8c292e1640 1 -- 172.21.15.45:0/30349870 >> v1:172.21.15.45:6789/0 conn(0x7f8c2415f420 legacy=0x7f8c240d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:56.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.949+0000 7f8c292e1640 1 -- 172.21.15.45:0/30349870 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c240d2c80 con 0x7f8c240d1970 2024-05-08T05:46:56.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.950+0000 7f8c292e1640 1 --2- 172.21.15.45:0/30349870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c240d1970 0x7f8c240d1e20 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f8c140097a0 tx=0x7f8c1402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d2f72fab7bcfad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:56.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.950+0000 7f8c28ae0640 1 -- 172.21.15.45:0/30349870 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14037070 con 0x7f8c240d1970 2024-05-08T05:46:56.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.950+0000 7f8c28ae0640 1 -- 172.21.15.45:0/30349870 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c14032440 con 0x7f8c240d1970 2024-05-08T05:46:56.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.950+0000 7f8c28ae0640 1 -- 172.21.15.45:0/30349870 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c14032810 con 0x7f8c240d1970 2024-05-08T05:46:56.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.950+0000 7f8c2bd46640 1 -- 172.21.15.45:0/30349870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c240d1970 msgr2=0x7f8c240d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:56.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.950+0000 7f8c2bd46640 1 --2- 172.21.15.45:0/30349870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c240d1970 0x7f8c240d1e20 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f8c140097a0 tx=0x7f8c1402a180 comp rx=0 tx=0).stop 2024-05-08T05:46:56.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.951+0000 7f8c2bd46640 1 -- 172.21.15.45:0/30349870 shutdown_connections 2024-05-08T05:46:56.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.951+0000 7f8c2bd46640 1 --2- 172.21.15.45:0/30349870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c240d1970 0x7f8c240d1e20 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.951+0000 7f8c2bd46640 1 -- 172.21.15.45:0/30349870 >> 172.21.15.45:0/30349870 conn(0x7f8c2408eac0 msgr2=0x7f8c2409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.951+0000 7f8c2bd46640 1 -- 172.21.15.45:0/30349870 shutdown_connections 2024-05-08T05:46:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.951+0000 7f8c2bd46640 1 -- 172.21.15.45:0/30349870 wait complete. 2024-05-08T05:46:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.952+0000 7f8c2bd46640 1 Processor -- start 2024-05-08T05:46:56.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.952+0000 7f8c2bd46640 1 -- start start 2024-05-08T05:46:56.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.953+0000 7f8c2bd46640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c240d1970 0x7f8c24152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:56.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.953+0000 7f8c2bd46640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c240d4cc0 0x7f8c240cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.953+0000 7f8c2bd46640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c240b9670 0x7f8c240b9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.953+0000 7f8c2bd46640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8c240d2830 con 0x7f8c240b9670 2024-05-08T05:46:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.953+0000 7f8c2bd46640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8c240d3240 con 0x7f8c240d4cc0 2024-05-08T05:46:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.953+0000 7f8c2bd46640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8c240cfb20 con 0x7f8c240d1970 2024-05-08T05:46:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.953+0000 7f8c2a2e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c240b9670 0x7f8c240b9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.953+0000 7f8c2a2e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c240b9670 0x7f8c240b9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51900/0 (socket says 172.21.15.45:51900) 2024-05-08T05:46:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.953+0000 7f8c2a2e3640 1 -- 172.21.15.45:0/2293611533 learned_addr learned my addr 172.21.15.45:0/2293611533 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.953+0000 7f8c29ae2640 1 --2- 172.21.15.45:0/2293611533 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c240d1970 0x7f8c24152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.953+0000 7f8c292e1640 1 --2- 172.21.15.45:0/2293611533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c240d4cc0 0x7f8c240cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.953+0000 7f8c2a2e3640 1 -- 172.21.15.45:0/2293611533 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c240d1970 msgr2=0x7f8c24152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.953+0000 7f8c2a2e3640 1 --2- 172.21.15.45:0/2293611533 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c240d1970 0x7f8c24152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.953+0000 7f8c2a2e3640 1 -- 172.21.15.45:0/2293611533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c240d4cc0 msgr2=0x7f8c240cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:56.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.953+0000 7f8c2a2e3640 1 --2- 172.21.15.45:0/2293611533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c240d4cc0 0x7f8c240cada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.953+0000 7f8c2a2e3640 1 -- 172.21.15.45:0/2293611533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c240b87b0 con 0x7f8c240b9670 2024-05-08T05:46:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.953+0000 7f8c29ae2640 1 --2- 172.21.15.45:0/2293611533 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c240d1970 0x7f8c24152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.953+0000 7f8c2a2e3640 1 --2- 172.21.15.45:0/2293611533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c240b9670 0x7f8c240b9b20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00c7b0 tx=0x7f8c1c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.953+0000 7f8c1a7fc640 1 -- 172.21.15.45:0/2293611533 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c004b60 con 0x7f8c240b9670 2024-05-08T05:46:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.954+0000 7f8c1a7fc640 1 -- 172.21.15.45:0/2293611533 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c1c004d00 con 0x7f8c240b9670 2024-05-08T05:46:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.954+0000 7f8c2bd46640 1 -- 172.21.15.45:0/2293611533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c240b8a90 con 0x7f8c240b9670 2024-05-08T05:46:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.954+0000 7f8c1a7fc640 1 -- 172.21.15.45:0/2293611533 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c00aa90 con 0x7f8c240b9670 2024-05-08T05:46:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.954+0000 7f8c2bd46640 1 -- 172.21.15.45:0/2293611533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c240c8b00 con 0x7f8c240b9670 2024-05-08T05:46:56.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.955+0000 7f8c2bd46640 1 -- 172.21.15.45:0/2293611533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c240693f0 con 0x7f8c240b9670 2024-05-08T05:46:56.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.955+0000 7f8c1a7fc640 1 -- 172.21.15.45:0/2293611533 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8c1c00ac30 con 0x7f8c240b9670 2024-05-08T05:46:56.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.956+0000 7f8c1a7fc640 1 --2- 172.21.15.45:0/2293611533 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8c080762d0 0x7f8c08078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:56.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.956+0000 7f8c1a7fc640 1 -- 172.21.15.45:0/2293611533 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5249+0+0 (secure 0 0 0) 0x7f8c1c097910 con 0x7f8c240b9670 2024-05-08T05:46:56.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.956+0000 7f8c29ae2640 1 --2- 172.21.15.45:0/2293611533 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8c080762d0 0x7f8c08078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:56.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.957+0000 7f8c29ae2640 1 --2- 172.21.15.45:0/2293611533 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8c080762d0 0x7f8c08078790 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8c20000f60 tx=0x7f8c20009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:56.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:56.961+0000 7f8c1a7fc640 1 -- 172.21.15.45:0/2293611533 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8c1c061430 con 0x7f8c240b9670 2024-05-08T05:46:57.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.234+0000 7f8c2bd46640 1 -- 172.21.15.45:0/2293611533 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f8c240b4c50 con 0x7f8c240b9670 2024-05-08T05:46:57.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.235+0000 7f8c1a7fc640 1 -- 172.21.15.45:0/2293611533 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7f8c1c069420 con 0x7f8c240b9670 2024-05-08T05:46:57.235 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:46:57.235 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:46:57.235 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2024-05-08T05:46:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2024-05-08T05:46:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmrRgXHBAAX1mgdmOD9JC4S1vJNyLSQw==", 2024-05-08T05:46:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:46:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:46:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:46:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:46:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:46:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:46:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2024-05-08T05:46:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmvn8SKBAAVkXYBbTfu98gl0AptOfJEg==", 2024-05-08T05:46:57.236 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:46:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:46:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:46:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:46:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:46:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:46:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2024-05-08T05:46:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmeiJCIxAAr3Br1fJVgR9TW2Q+4tMQBQ==", 2024-05-08T05:46:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:46:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:46:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:46:57.237 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:46:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:46:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:46:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2024-05-08T05:46:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmXdFILxAAxteIxUYeFdQKd/oxHLpeOQ==", 2024-05-08T05:46:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:46:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:46:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:46:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:46:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:46:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:46:57.238 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2024-05-08T05:46:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBWEDtmLdHHGRAAA1o4Z/lTGERIZhulqVtybw==", 2024-05-08T05:46:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:46:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:46:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:46:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:46:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:46:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2024-05-08T05:46:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBZEDtmMrRVGBAAeKBtcf5fSQMFE6IStUvokQ==", 2024-05-08T05:46:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:46:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:46:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:46:57.239 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:46:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:46:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2024-05-08T05:46:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBcEDtmpozQFhAAS6AF6X592Wh/QiBipcOVew==", 2024-05-08T05:46:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:46:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:46:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:46:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:46:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:46:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2024-05-08T05:46:57.240 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBfEDtm47lSFRAAt5tvpRCljxGtAmHQqcFj2A==", 2024-05-08T05:46:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:46:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:46:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:46:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:46:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:46:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2024-05-08T05:46:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBlEDtmDQ7mGxAA5z7rHZRf7X5g7ZiqEoO2DA==", 2024-05-08T05:46:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:46:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:46:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:46:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:46:57.241 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:46:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2024-05-08T05:46:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBoEDtmBdi0GhAAVMfg+KEJHRvaiS49PEboOg==", 2024-05-08T05:46:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:46:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:46:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:46:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:46:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:46:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2024-05-08T05:46:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBrEDtmG2daGRAANQgJFAT6gNE/vPofCv6veA==", 2024-05-08T05:46:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:46:57.242 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:46:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:46:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:46:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:46:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2024-05-08T05:46:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBuEDtm3tmDGRAARsAzm9KIzgKAs4ZtfCVdaQ==", 2024-05-08T05:46:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:46:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:46:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:46:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:46:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:46:57.243 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2024-05-08T05:46:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmDcqGMxAALgQaBYp4Zd8uMqswCXvGaw==", 2024-05-08T05:46:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:46:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:46:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2024-05-08T05:46:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T05:46:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:46:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2024-05-08T05:46:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBVEDtm6z0GARAARgRma5hPpH6X95pE5NNrQw==", 2024-05-08T05:46:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:46:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:46:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:46:57.244 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:46:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:46:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:46:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2024-05-08T05:46:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmAMtIORAAJf8oSwHqSzgjsTuqxtQqHQ==", 2024-05-08T05:46:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:46:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:46:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:46:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:46:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:46:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:46:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:57.245 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2024-05-08T05:46:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBVEDtm9qK1BhAAokvkJZmNxT5WER9/+EKIpA==", 2024-05-08T05:46:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:46:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:46:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:46:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:46:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:46:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:46:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2024-05-08T05:46:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmZ4Q3ABAANcqmwOmt+dNJ5wOiPHHO4Q==", 2024-05-08T05:46:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:46:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:46:57.246 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:46:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:46:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:46:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:46:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2024-05-08T05:46:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmL0XLGBAA1BSFdyiDYV9Je+igOVBvlQ==", 2024-05-08T05:46:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:46:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:46:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-05-08T05:46:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:46:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:46:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:57.247 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2024-05-08T05:46:57.248 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmfM5OFRAAgdJPUfqmnKU8vRrjGMNwrw==", 2024-05-08T05:46:57.248 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:46:57.248 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:46:57.248 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-05-08T05:46:57.248 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:46:57.248 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.248 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.248 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-05-08T05:46:57.248 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:46:57.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.237+0000 7f8c2bd46640 1 -- 172.21.15.45:0/2293611533 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8c080762d0 msgr2=0x7f8c08078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:57.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.237+0000 7f8c2bd46640 1 --2- 172.21.15.45:0/2293611533 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8c080762d0 0x7f8c08078790 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8c20000f60 tx=0x7f8c20009040 comp rx=0 tx=0).stop 2024-05-08T05:46:57.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.237+0000 7f8c2bd46640 1 -- 172.21.15.45:0/2293611533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c240b9670 msgr2=0x7f8c240b9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:57.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.237+0000 7f8c2bd46640 1 --2- 172.21.15.45:0/2293611533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c240b9670 0x7f8c240b9b20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00c7b0 tx=0x7f8c1c00cc70 comp rx=0 tx=0).stop 2024-05-08T05:46:57.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.237+0000 7f8c2bd46640 1 -- 172.21.15.45:0/2293611533 shutdown_connections 2024-05-08T05:46:57.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.237+0000 7f8c2bd46640 1 --2- 172.21.15.45:0/2293611533 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8c080762d0 0x7f8c08078790 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:57.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.237+0000 7f8c2bd46640 1 --2- 172.21.15.45:0/2293611533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c240b9670 0x7f8c240b9b20 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:57.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.237+0000 7f8c2bd46640 1 --2- 172.21.15.45:0/2293611533 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c240d4cc0 0x7f8c240cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:57.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.237+0000 7f8c2bd46640 1 --2- 172.21.15.45:0/2293611533 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8c240d1970 0x7f8c24152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:57.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.237+0000 7f8c2bd46640 1 -- 172.21.15.45:0/2293611533 >> 172.21.15.45:0/2293611533 conn(0x7f8c2408eac0 msgr2=0x7f8c2406cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:57.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.237+0000 7f8c2bd46640 1 -- 172.21.15.45:0/2293611533 shutdown_connections 2024-05-08T05:46:57.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.237+0000 7f8c2bd46640 1 -- 172.21.15.45:0/2293611533 wait complete. 2024-05-08T05:46:57.251 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T05:46:57.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.355+0000 7fb7f49f7640 1 Processor -- start 2024-05-08T05:46:57.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.355+0000 7fb7f49f7640 1 -- start start 2024-05-08T05:46:57.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7f49f7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7f01629b0 0x7fb7f006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:57.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7f49f7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7f006e110 con 0x7fb7f00b2490 2024-05-08T05:46:57.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7f49f7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7f006e3f0 con 0x7fb7f0070d20 2024-05-08T05:46:57.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7f49f7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f0070a40 con 0x7fb7f01629b0 2024-05-08T05:46:57.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7edd9b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb7f00b2490 0x7fb7f0070330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46738/0 (socket says 172.21.15.45:46738) 2024-05-08T05:46:57.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7edd9b640 1 -- 172.21.15.45:0/488463622 learned_addr learned my addr 172.21.15.45:0/488463622 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:57.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7ed59a640 1 -- 172.21.15.45:0/488463622 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 984527751 0 0) 0x7fb7f006e110 con 0x7fb7f00b2490 2024-05-08T05:46:57.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7ed59a640 1 -- 172.21.15.45:0/488463622 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7d8003520 con 0x7fb7f00b2490 2024-05-08T05:46:57.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7ee59c640 1 --2- 172.21.15.45:0/488463622 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7f01629b0 0x7fb7f006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:57.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7ed59a640 1 -- 172.21.15.45:0/488463622 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1798872434 0 0) 0x7fb7d8003520 con 0x7fb7f00b2490 2024-05-08T05:46:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7ed59a640 1 -- 172.21.15.45:0/488463622 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7f01629b0 msgr2=0x7fb7f006eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7ed59a640 1 --2- 172.21.15.45:0/488463622 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7f01629b0 0x7fb7f006eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7ed59a640 1 -- 172.21.15.45:0/488463622 >> v1:172.21.15.63:6789/0 conn(0x7fb7f0070d20 legacy=0x7fb7f00afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7ed59a640 1 -- 172.21.15.45:0/488463622 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f006e6d0 con 0x7fb7f00b2490 2024-05-08T05:46:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7ed59a640 1 -- 172.21.15.45:0/488463622 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fb7e40016f0 con 0x7fb7f00b2490 2024-05-08T05:46:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7ed59a640 1 -- 172.21.15.45:0/488463622 >> v1:172.21.15.45:6789/0 conn(0x7fb7f00b2490 legacy=0x7fb7f0070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7ed59a640 1 --2- 172.21.15.45:0/488463622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7d8003a30 0x7fb7d8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7ed59a640 1 --2- 172.21.15.45:0/488463622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7d8004800 0x7fb7d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7ed59a640 1 --2- 172.21.15.45:0/488463622 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7f01629b0 0x7fb7d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7ed59a640 1 -- 172.21.15.45:0/488463622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f0070a40 con 0x7fb7d8003a30 2024-05-08T05:46:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7ed59a640 1 -- 172.21.15.45:0/488463622 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d8005520 con 0x7fb7d8004800 2024-05-08T05:46:57.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7ed59a640 1 -- 172.21.15.45:0/488463622 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb7d800bf00 con 0x7fb7f01629b0 2024-05-08T05:46:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.356+0000 7fb7ee59c640 1 --2- 172.21.15.45:0/488463622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7d8003a30 0x7fb7d8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.357+0000 7fb7edd9b640 1 --2- 172.21.15.45:0/488463622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7d8004800 0x7fb7d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.357+0000 7fb7eed9d640 1 --2- 172.21.15.45:0/488463622 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7f01629b0 0x7fb7d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.357+0000 7fb7ee59c640 1 -- 172.21.15.45:0/488463622 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7f01629b0 msgr2=0x7fb7d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.357+0000 7fb7ee59c640 1 --2- 172.21.15.45:0/488463622 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7f01629b0 0x7fb7d800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.357+0000 7fb7ee59c640 1 -- 172.21.15.45:0/488463622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7d8004800 msgr2=0x7fb7d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.357+0000 7fb7ee59c640 1 --2- 172.21.15.45:0/488463622 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7d8004800 0x7fb7d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.357+0000 7fb7ee59c640 1 -- 172.21.15.45:0/488463622 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d8003520 con 0x7fb7d8003a30 2024-05-08T05:46:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.357+0000 7fb7eed9d640 1 --2- 172.21.15.45:0/488463622 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7f01629b0 0x7fb7d800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.357+0000 7fb7ee59c640 1 --2- 172.21.15.45:0/488463622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7d8003a30 0x7fb7d8003e50 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000ba30 tx=0x7fb7d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f42d67a3d1310a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.358+0000 7fb7ed59a640 1 -- 172.21.15.45:0/488463622 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0030070 con 0x7fb7d8003a30 2024-05-08T05:46:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.358+0000 7fb7ed59a640 1 -- 172.21.15.45:0/488463622 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e002ade0 con 0x7fb7d8003a30 2024-05-08T05:46:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.358+0000 7fb7ed59a640 1 -- 172.21.15.45:0/488463622 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e000b600 con 0x7fb7d8003a30 2024-05-08T05:46:57.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.358+0000 7fb7f49f7640 1 -- 172.21.15.45:0/488463622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7d8003a30 msgr2=0x7fb7d8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.358+0000 7fb7f49f7640 1 --2- 172.21.15.45:0/488463622 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7d8003a30 0x7fb7d8003e50 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000ba30 tx=0x7fb7d8005830 comp rx=0 tx=0).stop 2024-05-08T05:46:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.358+0000 7fb7eed9d640 1 -- 172.21.15.45:0/488463622 reap_dead start 2024-05-08T05:46:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.358+0000 7fb7f49f7640 1 -- 172.21.15.45:0/488463622 shutdown_connections 2024-05-08T05:46:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.358+0000 7fb7f49f7640 1 -- 172.21.15.45:0/488463622 >> 172.21.15.45:0/488463622 conn(0x7fb7f00b49d0 msgr2=0x7fb7f00b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.358+0000 7fb7f49f7640 1 -- 172.21.15.45:0/488463622 shutdown_connections 2024-05-08T05:46:57.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.358+0000 7fb7f49f7640 1 -- 172.21.15.45:0/488463622 wait complete. 2024-05-08T05:46:57.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.360+0000 7fb7f49f7640 1 Processor -- start 2024-05-08T05:46:57.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.360+0000 7fb7f49f7640 1 -- start start 2024-05-08T05:46:57.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.360+0000 7fb7f49f7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7d8003a30 0x7fb7f01575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:57.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.360+0000 7fb7f49f7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7f00c6cb0 0x7fb7f00c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:57.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.360+0000 7fb7f49f7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7f00cd470 0x7fb7f0160850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:57.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.360+0000 7fb7f49f7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f00b03c0 con 0x7fb7f00cd470 2024-05-08T05:46:57.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.360+0000 7fb7f49f7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f006dec0 con 0x7fb7f00c6cb0 2024-05-08T05:46:57.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.360+0000 7fb7f49f7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f00b0540 con 0x7fb7d8003a30 2024-05-08T05:46:57.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.360+0000 7fb7eed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7f00cd470 0x7fb7f0160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:57.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.361+0000 7fb7eed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7f00cd470 0x7fb7f0160850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51928/0 (socket says 172.21.15.45:51928) 2024-05-08T05:46:57.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.361+0000 7fb7eed9d640 1 -- 172.21.15.45:0/1940011852 learned_addr learned my addr 172.21.15.45:0/1940011852 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:57.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.361+0000 7fb7edd9b640 1 --2- 172.21.15.45:0/1940011852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7f00c6cb0 0x7fb7f00c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.361+0000 7fb7ee59c640 1 --2- 172.21.15.45:0/1940011852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7d8003a30 0x7fb7f01575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.361+0000 7fb7eed9d640 1 -- 172.21.15.45:0/1940011852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7d8003a30 msgr2=0x7fb7f01575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.361+0000 7fb7eed9d640 1 --2- 172.21.15.45:0/1940011852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7d8003a30 0x7fb7f01575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.361+0000 7fb7eed9d640 1 -- 172.21.15.45:0/1940011852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7f00c6cb0 msgr2=0x7fb7f00c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.361+0000 7fb7eed9d640 1 --2- 172.21.15.45:0/1940011852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7f00c6cb0 0x7fb7f00c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.361+0000 7fb7eed9d640 1 -- 172.21.15.45:0/1940011852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f00c7990 con 0x7fb7f00cd470 2024-05-08T05:46:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.361+0000 7fb7edd9b640 1 --2- 172.21.15.45:0/1940011852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7f00c6cb0 0x7fb7f00c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.361+0000 7fb7eed9d640 1 --2- 172.21.15.45:0/1940011852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7f00cd470 0x7fb7f0160850 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8007fd0 tx=0x7fb7e800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.361+0000 7fb7deffd640 1 -- 172.21.15.45:0/1940011852 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8019070 con 0x7fb7f00cd470 2024-05-08T05:46:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.361+0000 7fb7deffd640 1 -- 172.21.15.45:0/1940011852 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7e8006b50 con 0x7fb7f00cd470 2024-05-08T05:46:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.361+0000 7fb7f49f7640 1 -- 172.21.15.45:0/1940011852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f00b8950 con 0x7fb7f00cd470 2024-05-08T05:46:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.361+0000 7fb7deffd640 1 -- 172.21.15.45:0/1940011852 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8003970 con 0x7fb7f00cd470 2024-05-08T05:46:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.361+0000 7fb7f49f7640 1 -- 172.21.15.45:0/1940011852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f00b6f50 con 0x7fb7f00cd470 2024-05-08T05:46:57.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.362+0000 7fb7f49f7640 1 -- 172.21.15.45:0/1940011852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f00721d0 con 0x7fb7f00cd470 2024-05-08T05:46:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.363+0000 7fb7deffd640 1 -- 172.21.15.45:0/1940011852 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb7e801c070 con 0x7fb7f00cd470 2024-05-08T05:46:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.363+0000 7fb7deffd640 1 --2- 172.21.15.45:0/1940011852 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb7d0076300 0x7fb7d00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.363+0000 7fb7deffd640 1 -- 172.21.15.45:0/1940011852 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5249+0+0 (secure 0 0 0) 0x7fb7e80966d0 con 0x7fb7f00cd470 2024-05-08T05:46:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.364+0000 7fb7ee59c640 1 --2- 172.21.15.45:0/1940011852 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb7d0076300 0x7fb7d00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.364+0000 7fb7ee59c640 1 --2- 172.21.15.45:0/1940011852 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb7d0076300 0x7fb7d00787c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000ba00 tx=0x7fb7e0030000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:57.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.367+0000 7fb7deffd640 1 -- 172.21.15.45:0/1940011852 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb7f00721d0 con 0x7fb7f00cd470 2024-05-08T05:46:57.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.561+0000 7fb7f49f7640 1 -- 172.21.15.45:0/1940011852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb7f0068c40 con 0x7fb7f00cd470 2024-05-08T05:46:57.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.561+0000 7fb7deffd640 1 -- 172.21.15.45:0/1940011852 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v84) ==== 84+0+65 (secure 0 0 0) 0x7fb7f0068c40 con 0x7fb7f00cd470 2024-05-08T05:46:57.562 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:46:57.562 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-05-08T05:46:57.562 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:46:57.562 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-05-08T05:46:57.562 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-05-08T05:46:57.562 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:46:57.562 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-05-08T05:46:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.563+0000 7fb7f49f7640 1 -- 172.21.15.45:0/1940011852 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb7d0076300 msgr2=0x7fb7d00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.563+0000 7fb7f49f7640 1 --2- 172.21.15.45:0/1940011852 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb7d0076300 0x7fb7d00787c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000ba00 tx=0x7fb7e0030000 comp rx=0 tx=0).stop 2024-05-08T05:46:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.563+0000 7fb7f49f7640 1 -- 172.21.15.45:0/1940011852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7f00cd470 msgr2=0x7fb7f0160850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.563+0000 7fb7f49f7640 1 --2- 172.21.15.45:0/1940011852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7f00cd470 0x7fb7f0160850 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8007fd0 tx=0x7fb7e800eb90 comp rx=0 tx=0).stop 2024-05-08T05:46:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.563+0000 7fb7f49f7640 1 -- 172.21.15.45:0/1940011852 shutdown_connections 2024-05-08T05:46:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.563+0000 7fb7f49f7640 1 --2- 172.21.15.45:0/1940011852 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb7d0076300 0x7fb7d00787c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.563+0000 7fb7f49f7640 1 --2- 172.21.15.45:0/1940011852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7f00cd470 0x7fb7f0160850 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.563+0000 7fb7f49f7640 1 --2- 172.21.15.45:0/1940011852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7f00c6cb0 0x7fb7f00c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.563+0000 7fb7f49f7640 1 --2- 172.21.15.45:0/1940011852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7d8003a30 0x7fb7f01575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.563+0000 7fb7f49f7640 1 -- 172.21.15.45:0/1940011852 >> 172.21.15.45:0/1940011852 conn(0x7fb7f00b49d0 msgr2=0x7fb7f0070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.563+0000 7fb7f49f7640 1 -- 172.21.15.45:0/1940011852 shutdown_connections 2024-05-08T05:46:57.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.563+0000 7fb7f49f7640 1 -- 172.21.15.45:0/1940011852 wait complete. 2024-05-08T05:46:57.574 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T05:46:57.575 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T05:46:57.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.682+0000 7fbb8ced1640 1 Processor -- start 2024-05-08T05:46:57.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.682+0000 7fbb8ced1640 1 -- start start 2024-05-08T05:46:57.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.682+0000 7fbb8ced1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbb880cf8f0 0x7fbb88070500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:57.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.683+0000 7fbb8ced1640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbb880cef40 0x7fbb88169b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:57.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.683+0000 7fbb8ced1640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb880d2030 con 0x7fbb880d3920 2024-05-08T05:46:57.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.683+0000 7fbb8ced1640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fbb8815f8d0 con 0x7fbb880cf8f0 2024-05-08T05:46:57.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.683+0000 7fbb8ced1640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbb880d22b0 con 0x7fbb880cef40 2024-05-08T05:46:57.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.683+0000 7fbb8659c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbb880cf8f0 0x7fbb88070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:57.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.683+0000 7fbb85d9b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbb880d3920 0x7fbb88070a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46740/0 (socket says 172.21.15.45:46740) 2024-05-08T05:46:57.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.683+0000 7fbb85d9b640 1 -- 172.21.15.45:0/1793963384 learned_addr learned my addr 172.21.15.45:0/1793963384 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:57.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.683+0000 7fbb86d9d640 1 --2- 172.21.15.45:0/1793963384 >> v2:172.21.15.63:3300/0 conn(0x7fbb880cef40 0x7fbb88169b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:57.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.683+0000 7fbb8659c640 1 -- 172.21.15.45:0/1793963384 >> v2:172.21.15.63:3300/0 conn(0x7fbb880cef40 msgr2=0x7fbb88169b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:57.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.683+0000 7fbb8659c640 1 --2- 172.21.15.45:0/1793963384 >> v2:172.21.15.63:3300/0 conn(0x7fbb880cef40 0x7fbb88169b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:57.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.683+0000 7fbb8659c640 1 -- 172.21.15.45:0/1793963384 >> v1:172.21.15.45:6789/0 conn(0x7fbb880d3920 legacy=0x7fbb88070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:57.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.683+0000 7fbb8659c640 1 -- 172.21.15.45:0/1793963384 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb880d2590 con 0x7fbb880cf8f0 2024-05-08T05:46:57.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.683+0000 7fbb86d9d640 1 --2- 172.21.15.45:0/1793963384 >> v2:172.21.15.63:3300/0 conn(0x7fbb880cef40 0x7fbb88169b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:57.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.684+0000 7fbb8659c640 1 --2- 172.21.15.45:0/1793963384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb880cf8f0 0x7fbb88070500 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbb88073c40 tx=0x7fbb7802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d941a8cc665edff4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:57.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.684+0000 7fbb8559a640 1 -- 172.21.15.45:0/1793963384 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78037070 con 0x7fbb880cf8f0 2024-05-08T05:46:57.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.684+0000 7fbb8559a640 1 -- 172.21.15.45:0/1793963384 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb78032440 con 0x7fbb880cf8f0 2024-05-08T05:46:57.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.684+0000 7fbb8559a640 1 -- 172.21.15.45:0/1793963384 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78032810 con 0x7fbb880cf8f0 2024-05-08T05:46:57.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.684+0000 7fbb8ced1640 1 -- 172.21.15.45:0/1793963384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb880cf8f0 msgr2=0x7fbb88070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:57.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.684+0000 7fbb8ced1640 1 --2- 172.21.15.45:0/1793963384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb880cf8f0 0x7fbb88070500 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbb88073c40 tx=0x7fbb7802a230 comp rx=0 tx=0).stop 2024-05-08T05:46:57.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.685+0000 7fbb8ced1640 1 -- 172.21.15.45:0/1793963384 shutdown_connections 2024-05-08T05:46:57.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.685+0000 7fbb8ced1640 1 --2- 172.21.15.45:0/1793963384 >> v2:172.21.15.63:3300/0 conn(0x7fbb880cef40 0x7fbb88169b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:57.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.685+0000 7fbb8ced1640 1 --2- 172.21.15.45:0/1793963384 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb880cf8f0 0x7fbb88070500 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:57.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.685+0000 7fbb8ced1640 1 -- 172.21.15.45:0/1793963384 >> 172.21.15.45:0/1793963384 conn(0x7fbb880d2e00 msgr2=0x7fbb880d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:57.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.685+0000 7fbb8ced1640 1 -- 172.21.15.45:0/1793963384 shutdown_connections 2024-05-08T05:46:57.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.685+0000 7fbb8ced1640 1 -- 172.21.15.45:0/1793963384 wait complete. 2024-05-08T05:46:57.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.686+0000 7fbb8ced1640 1 Processor -- start 2024-05-08T05:46:57.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.686+0000 7fbb8ced1640 1 -- start start 2024-05-08T05:46:57.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.687+0000 7fbb8ced1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb880cef40 0x7fbb880cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:57.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.687+0000 7fbb8ced1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb880cf8f0 0x7fbb88150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:57.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.687+0000 7fbb8ced1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb880d3920 0x7fbb881510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:57.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.687+0000 7fbb8ced1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbb880cda10 con 0x7fbb880cf8f0 2024-05-08T05:46:57.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.687+0000 7fbb8ced1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbb880cd730 con 0x7fbb880cef40 2024-05-08T05:46:57.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.687+0000 7fbb8ced1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbb88071610 con 0x7fbb880d3920 2024-05-08T05:46:57.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.687+0000 7fbb85d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb880cf8f0 0x7fbb88150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:57.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.687+0000 7fbb85d9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb880cf8f0 0x7fbb88150ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51936/0 (socket says 172.21.15.45:51936) 2024-05-08T05:46:57.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.687+0000 7fbb85d9b640 1 -- 172.21.15.45:0/3235390605 learned_addr learned my addr 172.21.15.45:0/3235390605 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:57.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.687+0000 7fbb8659c640 1 --2- 172.21.15.45:0/3235390605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb880cef40 0x7fbb880cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:57.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.687+0000 7fbb86d9d640 1 --2- 172.21.15.45:0/3235390605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb880d3920 0x7fbb881510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:57.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.687+0000 7fbb85d9b640 1 -- 172.21.15.45:0/3235390605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb880d3920 msgr2=0x7fbb881510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:57.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.687+0000 7fbb85d9b640 1 --2- 172.21.15.45:0/3235390605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb880d3920 0x7fbb881510e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:57.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.687+0000 7fbb85d9b640 1 -- 172.21.15.45:0/3235390605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb880cef40 msgr2=0x7fbb880cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:57.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.687+0000 7fbb85d9b640 1 --2- 172.21.15.45:0/3235390605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb880cef40 0x7fbb880cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:57.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.687+0000 7fbb85d9b640 1 -- 172.21.15.45:0/3235390605 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb880ccd60 con 0x7fbb880cf8f0 2024-05-08T05:46:57.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.687+0000 7fbb85d9b640 1 --2- 172.21.15.45:0/3235390605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb880cf8f0 0x7fbb88150ba0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00c4d0 tx=0x7fbb7c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:57.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.688+0000 7fbb76ffd640 1 -- 172.21.15.45:0/3235390605 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c017070 con 0x7fbb880cf8f0 2024-05-08T05:46:57.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.688+0000 7fbb76ffd640 1 -- 172.21.15.45:0/3235390605 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb7c005d00 con 0x7fbb880cf8f0 2024-05-08T05:46:57.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.688+0000 7fbb8ced1640 1 -- 172.21.15.45:0/3235390605 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb880cb5a0 con 0x7fbb880cf8f0 2024-05-08T05:46:57.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.688+0000 7fbb76ffd640 1 -- 172.21.15.45:0/3235390605 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c0072b0 con 0x7fbb880cf8f0 2024-05-08T05:46:57.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.688+0000 7fbb8ced1640 1 -- 172.21.15.45:0/3235390605 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb880cbb60 con 0x7fbb880cf8f0 2024-05-08T05:46:57.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.689+0000 7fbb8ced1640 1 -- 172.21.15.45:0/3235390605 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb880b2260 con 0x7fbb880cf8f0 2024-05-08T05:46:57.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.689+0000 7fbb76ffd640 1 -- 172.21.15.45:0/3235390605 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbb7c01a070 con 0x7fbb880cf8f0 2024-05-08T05:46:57.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.690+0000 7fbb76ffd640 1 --2- 172.21.15.45:0/3235390605 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbb680762d0 0x7fbb68078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:57.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.690+0000 7fbb76ffd640 1 -- 172.21.15.45:0/3235390605 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5249+0+0 (secure 0 0 0) 0x7fbb7c096820 con 0x7fbb880cf8f0 2024-05-08T05:46:57.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.690+0000 7fbb8659c640 1 --2- 172.21.15.45:0/3235390605 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbb680762d0 0x7fbb68078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:57.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.690+0000 7fbb8659c640 1 --2- 172.21.15.45:0/3235390605 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbb680762d0 0x7fbb68078790 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbb78034000 tx=0x7fbb7802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:57.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.694+0000 7fbb76ffd640 1 -- 172.21.15.45:0/3235390605 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbb880d2030 con 0x7fbb880cf8f0 2024-05-08T05:46:57.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:57.887+0000 7fbb8ced1640 1 -- 172.21.15.45:0/3235390605 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fbb880d3560 con 0x7fbb880cf8f0 2024-05-08T05:46:58.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.821+0000 7fbb76ffd640 1 -- 172.21.15.45:0/3235390605 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v85) ==== 138+0+0 (secure 0 0 0) 0x7fbb7c0602c0 con 0x7fbb880cf8f0 2024-05-08T05:46:58.822 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-05-08T05:46:58.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.824+0000 7fbb8ced1640 1 -- 172.21.15.45:0/3235390605 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbb680762d0 msgr2=0x7fbb68078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:58.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.824+0000 7fbb8ced1640 1 --2- 172.21.15.45:0/3235390605 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbb680762d0 0x7fbb68078790 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbb78034000 tx=0x7fbb7802e000 comp rx=0 tx=0).stop 2024-05-08T05:46:58.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.824+0000 7fbb8ced1640 1 -- 172.21.15.45:0/3235390605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb880cf8f0 msgr2=0x7fbb88150ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:58.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.824+0000 7fbb8ced1640 1 --2- 172.21.15.45:0/3235390605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb880cf8f0 0x7fbb88150ba0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00c4d0 tx=0x7fbb7c00c990 comp rx=0 tx=0).stop 2024-05-08T05:46:58.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.824+0000 7fbb8ced1640 1 -- 172.21.15.45:0/3235390605 shutdown_connections 2024-05-08T05:46:58.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.824+0000 7fbb8ced1640 1 --2- 172.21.15.45:0/3235390605 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbb680762d0 0x7fbb68078790 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:58.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.824+0000 7fbb8ced1640 1 --2- 172.21.15.45:0/3235390605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbb880d3920 0x7fbb881510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:58.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.824+0000 7fbb8ced1640 1 --2- 172.21.15.45:0/3235390605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb880cf8f0 0x7fbb88150ba0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:58.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.824+0000 7fbb8ced1640 1 --2- 172.21.15.45:0/3235390605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbb880cef40 0x7fbb880cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:58.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.824+0000 7fbb8ced1640 1 -- 172.21.15.45:0/3235390605 >> 172.21.15.45:0/3235390605 conn(0x7fbb880d2e00 msgr2=0x7fbb880afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:58.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.825+0000 7fbb8ced1640 1 -- 172.21.15.45:0/3235390605 shutdown_connections 2024-05-08T05:46:58.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.825+0000 7fbb8ced1640 1 -- 172.21.15.45:0/3235390605 wait complete. 2024-05-08T05:46:58.842 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T05:46:58.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.960+0000 7fc19e41d640 1 Processor -- start 2024-05-08T05:46:58.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.960+0000 7fc19e41d640 1 -- start start 2024-05-08T05:46:58.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.961+0000 7fc19e41d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1981639e0 0x7fc19806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:58.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.961+0000 7fc19e41d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc198160720 con 0x7fc198070d30 2024-05-08T05:46:58.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.961+0000 7fc19e41d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1980d10d0 con 0x7fc1980d2130 2024-05-08T05:46:58.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.961+0000 7fc19e41d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc1980d1ca0 con 0x7fc1981639e0 2024-05-08T05:46:58.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.961+0000 7fc197fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc198070d30 0x7fc1980703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46752/0 (socket says 172.21.15.45:46752) 2024-05-08T05:46:58.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.961+0000 7fc197fff640 1 -- 172.21.15.45:0/855322244 learned_addr learned my addr 172.21.15.45:0/855322244 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:46:58.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc19c9ba640 1 --2- 172.21.15.45:0/855322244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1981639e0 0x7fc19806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:58.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc196ffd640 1 -- 172.21.15.45:0/855322244 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3004213689 0 0) 0x7fc198160720 con 0x7fc198070d30 2024-05-08T05:46:58.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc196ffd640 1 -- 172.21.15.45:0/855322244 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc180003520 con 0x7fc198070d30 2024-05-08T05:46:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc196ffd640 1 -- 172.21.15.45:0/855322244 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2669599096 0 0) 0x7fc1980d10d0 con 0x7fc1980d2130 2024-05-08T05:46:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc196ffd640 1 -- 172.21.15.45:0/855322244 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc198160720 con 0x7fc1980d2130 2024-05-08T05:46:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc196ffd640 1 -- 172.21.15.45:0/855322244 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2334783656 0 0) 0x7fc180003520 con 0x7fc198070d30 2024-05-08T05:46:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc196ffd640 1 -- 172.21.15.45:0/855322244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1981639e0 msgr2=0x7fc19806d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc196ffd640 1 --2- 172.21.15.45:0/855322244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1981639e0 0x7fc19806d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc196ffd640 1 -- 172.21.15.45:0/855322244 >> v1:172.21.15.63:6789/0 conn(0x7fc1980d2130 legacy=0x7fc198071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc196ffd640 1 -- 172.21.15.45:0/855322244 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1980d1370 con 0x7fc198070d30 2024-05-08T05:46:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc196ffd640 1 -- 172.21.15.45:0/855322244 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fc1880016f0 con 0x7fc198070d30 2024-05-08T05:46:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc196ffd640 1 -- 172.21.15.45:0/855322244 >> v1:172.21.15.45:6789/0 conn(0x7fc198070d30 legacy=0x7fc1980703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc196ffd640 1 --2- 172.21.15.45:0/855322244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003c10 0x7fc180004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc196ffd640 1 --2- 172.21.15.45:0/855322244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc180004800 0x7fc180004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc196ffd640 1 --2- 172.21.15.45:0/855322244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1981639e0 0x7fc180005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc196ffd640 1 -- 172.21.15.45:0/855322244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc1980d1ca0 con 0x7fc180003c10 2024-05-08T05:46:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc196ffd640 1 -- 172.21.15.45:0/855322244 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc18000bb50 con 0x7fc180004800 2024-05-08T05:46:58.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc196ffd640 1 -- 172.21.15.45:0/855322244 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc18000bd20 con 0x7fc1981639e0 2024-05-08T05:46:58.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc197fff640 1 --2- 172.21.15.45:0/855322244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003c10 0x7fc180004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:58.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc1977fe640 1 --2- 172.21.15.45:0/855322244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc180004800 0x7fc180004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:58.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc19c9ba640 1 --2- 172.21.15.45:0/855322244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1981639e0 0x7fc180005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:58.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc197fff640 1 -- 172.21.15.45:0/855322244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1981639e0 msgr2=0x7fc180005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:58.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc197fff640 1 --2- 172.21.15.45:0/855322244 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1981639e0 0x7fc180005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:58.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc197fff640 1 -- 172.21.15.45:0/855322244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc180004800 msgr2=0x7fc180004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:58.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc197fff640 1 --2- 172.21.15.45:0/855322244 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc180004800 0x7fc180004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:58.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.962+0000 7fc197fff640 1 -- 172.21.15.45:0/855322244 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc180003520 con 0x7fc180003c10 2024-05-08T05:46:58.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.963+0000 7fc197fff640 1 --2- 172.21.15.45:0/855322244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003c10 0x7fc180004010 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc18800c1c0 tx=0x7fc180005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667aa87d63ae5c7e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:58.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.963+0000 7fc196ffd640 1 -- 172.21.15.45:0/855322244 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188007ce0 con 0x7fc180003c10 2024-05-08T05:46:58.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.963+0000 7fc196ffd640 1 -- 172.21.15.45:0/855322244 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc188005af0 con 0x7fc180003c10 2024-05-08T05:46:58.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.963+0000 7fc196ffd640 1 -- 172.21.15.45:0/855322244 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188007070 con 0x7fc180003c10 2024-05-08T05:46:58.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.964+0000 7fc19e41d640 1 -- 172.21.15.45:0/855322244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003c10 msgr2=0x7fc180004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:58.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.964+0000 7fc19e41d640 1 --2- 172.21.15.45:0/855322244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003c10 0x7fc180004010 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fc18800c1c0 tx=0x7fc180005830 comp rx=0 tx=0).stop 2024-05-08T05:46:58.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.964+0000 7fc19c9ba640 1 -- 172.21.15.45:0/855322244 reap_dead start 2024-05-08T05:46:58.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.964+0000 7fc19e41d640 1 -- 172.21.15.45:0/855322244 shutdown_connections 2024-05-08T05:46:58.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.964+0000 7fc19e41d640 1 -- 172.21.15.45:0/855322244 >> 172.21.15.45:0/855322244 conn(0x7fc1980d2e90 msgr2=0x7fc1980d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:58.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.964+0000 7fc19e41d640 1 -- 172.21.15.45:0/855322244 shutdown_connections 2024-05-08T05:46:58.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.964+0000 7fc19e41d640 1 -- 172.21.15.45:0/855322244 wait complete. 2024-05-08T05:46:58.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.965+0000 7fc19e41d640 1 Processor -- start 2024-05-08T05:46:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.965+0000 7fc19e41d640 1 -- start start 2024-05-08T05:46:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.966+0000 7fc19e41d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003c10 0x7fc1980c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.966+0000 7fc19e41d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1980cfd00 0x7fc1980cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.966+0000 7fc19e41d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1980cbd20 0x7fc1980cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.966+0000 7fc19e41d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc19806d4b0 con 0x7fc180003c10 2024-05-08T05:46:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.966+0000 7fc19e41d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc1980d1930 con 0x7fc1980cbd20 2024-05-08T05:46:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.966+0000 7fc19e41d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc19806d630 con 0x7fc1980cfd00 2024-05-08T05:46:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.966+0000 7fc197fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003c10 0x7fc1980c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.966+0000 7fc197fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003c10 0x7fc1980c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51946/0 (socket says 172.21.15.45:51946) 2024-05-08T05:46:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.966+0000 7fc197fff640 1 -- 172.21.15.45:0/2581588036 learned_addr learned my addr 172.21.15.45:0/2581588036 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.966+0000 7fc1977fe640 1 --2- 172.21.15.45:0/2581588036 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1980cfd00 0x7fc1980cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.966+0000 7fc19c9ba640 1 --2- 172.21.15.45:0/2581588036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1980cbd20 0x7fc1980cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:58.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.966+0000 7fc197fff640 1 -- 172.21.15.45:0/2581588036 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1980cfd00 msgr2=0x7fc1980cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.966+0000 7fc197fff640 1 --2- 172.21.15.45:0/2581588036 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1980cfd00 0x7fc1980cd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.966+0000 7fc197fff640 1 -- 172.21.15.45:0/2581588036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1980cbd20 msgr2=0x7fc1980cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.966+0000 7fc197fff640 1 --2- 172.21.15.45:0/2581588036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1980cbd20 0x7fc1980cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.966+0000 7fc197fff640 1 -- 172.21.15.45:0/2581588036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1980c95e0 con 0x7fc180003c10 2024-05-08T05:46:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.967+0000 7fc197fff640 1 --2- 172.21.15.45:0/2581588036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003c10 0x7fc1980c8d20 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc18800c2e0 tx=0x7fc1880019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.967+0000 7fc194ff9640 1 -- 172.21.15.45:0/2581588036 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188041070 con 0x7fc180003c10 2024-05-08T05:46:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.967+0000 7fc194ff9640 1 -- 172.21.15.45:0/2581588036 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1880016f0 con 0x7fc180003c10 2024-05-08T05:46:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.967+0000 7fc19e41d640 1 -- 172.21.15.45:0/2581588036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1980cafa0 con 0x7fc180003c10 2024-05-08T05:46:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.967+0000 7fc194ff9640 1 -- 172.21.15.45:0/2581588036 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188037040 con 0x7fc180003c10 2024-05-08T05:46:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.967+0000 7fc19e41d640 1 -- 172.21.15.45:0/2581588036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1980cb240 con 0x7fc180003c10 2024-05-08T05:46:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.968+0000 7fc19e41d640 1 -- 172.21.15.45:0/2581588036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1980693f0 con 0x7fc180003c10 2024-05-08T05:46:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.968+0000 7fc194ff9640 1 -- 172.21.15.45:0/2581588036 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc188044070 con 0x7fc180003c10 2024-05-08T05:46:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.969+0000 7fc194ff9640 1 --2- 172.21.15.45:0/2581588036 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc1840762d0 0x7fc184078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.969+0000 7fc194ff9640 1 -- 172.21.15.45:0/2581588036 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5619+0+0 (secure 0 0 0) 0x7fc1880b67b0 con 0x7fc180003c10 2024-05-08T05:46:58.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.969+0000 7fc1977fe640 1 --2- 172.21.15.45:0/2581588036 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc1840762d0 0x7fc184078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:58.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.970+0000 7fc1977fe640 1 --2- 172.21.15.45:0/2581588036 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc1840762d0 0x7fc184078790 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc19815edb0 tx=0x7fc18c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:58.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:58.974+0000 7fc194ff9640 1 -- 172.21.15.45:0/2581588036 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc18807fb00 con 0x7fc180003c10 2024-05-08T05:46:59.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.163+0000 7fc19e41d640 1 -- 172.21.15.45:0/2581588036 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc1980d3d70 con 0x7fc180003c10 2024-05-08T05:46:59.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.827+0000 7fc194ff9640 1 -- 172.21.15.45:0/2581588036 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v86) ==== 170+0+0 (secure 0 0 0) 0x7fc188083e00 con 0x7fc180003c10 2024-05-08T05:46:59.827 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-05-08T05:46:59.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.829+0000 7fc19e41d640 1 -- 172.21.15.45:0/2581588036 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc1840762d0 msgr2=0x7fc184078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:59.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.829+0000 7fc19e41d640 1 --2- 172.21.15.45:0/2581588036 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc1840762d0 0x7fc184078790 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc19815edb0 tx=0x7fc18c009040 comp rx=0 tx=0).stop 2024-05-08T05:46:59.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.829+0000 7fc19e41d640 1 -- 172.21.15.45:0/2581588036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003c10 msgr2=0x7fc1980c8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:59.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.829+0000 7fc19e41d640 1 --2- 172.21.15.45:0/2581588036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003c10 0x7fc1980c8d20 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc18800c2e0 tx=0x7fc1880019b0 comp rx=0 tx=0).stop 2024-05-08T05:46:59.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.830+0000 7fc19e41d640 1 -- 172.21.15.45:0/2581588036 shutdown_connections 2024-05-08T05:46:59.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.830+0000 7fc19e41d640 1 --2- 172.21.15.45:0/2581588036 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc1840762d0 0x7fc184078790 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:59.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.830+0000 7fc19e41d640 1 --2- 172.21.15.45:0/2581588036 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc1980cfd00 0x7fc1980cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:59.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.830+0000 7fc19e41d640 1 --2- 172.21.15.45:0/2581588036 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc1980cbd20 0x7fc1980cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:59.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.830+0000 7fc19e41d640 1 --2- 172.21.15.45:0/2581588036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc180003c10 0x7fc1980c8d20 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:59.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.830+0000 7fc19e41d640 1 -- 172.21.15.45:0/2581588036 >> 172.21.15.45:0/2581588036 conn(0x7fc1980d2e90 msgr2=0x7fc19815b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:59.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.830+0000 7fc19e41d640 1 -- 172.21.15.45:0/2581588036 shutdown_connections 2024-05-08T05:46:59.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.830+0000 7fc19e41d640 1 -- 172.21.15.45:0/2581588036 wait complete. 2024-05-08T05:46:59.858 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T05:46:59.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.969+0000 7fb36fada640 1 Processor -- start 2024-05-08T05:46:59.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.970+0000 7fb36fada640 1 -- start start 2024-05-08T05:46:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.970+0000 7fb36fada640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3680b21f0 0x7fb36806e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:59.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.970+0000 7fb36fada640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb368162f50 0x7fb3681633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:59.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.970+0000 7fb36fada640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3680d0500 con 0x7fb3680cf790 2024-05-08T05:46:59.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.970+0000 7fb36fada640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb3681638f0 con 0x7fb368162f50 2024-05-08T05:46:59.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.970+0000 7fb36fada640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb36806d810 con 0x7fb3680b21f0 2024-05-08T05:46:59.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.971+0000 7fb36d075640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb368162f50 0x7fb3681633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:59.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.971+0000 7fb36d075640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb368162f50 0x7fb3681633b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51954/0 (socket says 172.21.15.45:51954) 2024-05-08T05:46:59.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.971+0000 7fb36d075640 1 -- 172.21.15.45:0/138951962 learned_addr learned my addr 172.21.15.45:0/138951962 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.971+0000 7fb36d876640 1 --2- 172.21.15.45:0/138951962 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3680b21f0 0x7fb36806e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.971+0000 7fb36d075640 1 -- 172.21.15.45:0/138951962 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3680b21f0 msgr2=0x7fb36806e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.971+0000 7fb36d075640 1 --2- 172.21.15.45:0/138951962 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3680b21f0 0x7fb36806e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.971+0000 7fb36d075640 1 -- 172.21.15.45:0/138951962 >> v1:172.21.15.63:6789/0 conn(0x7fb3680cf790 legacy=0x7fb3680cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:46:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.971+0000 7fb36d075640 1 -- 172.21.15.45:0/138951962 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36806da60 con 0x7fb368162f50 2024-05-08T05:46:59.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.972+0000 7fb36d075640 1 --2- 172.21.15.45:0/138951962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb368162f50 0x7fb3681633b0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009870 tx=0x7fb35c02a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f93dfc76c427a45 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:59.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.972+0000 7fb36c874640 1 -- 172.21.15.45:0/138951962 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c037070 con 0x7fb368162f50 2024-05-08T05:46:59.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.972+0000 7fb36c874640 1 -- 172.21.15.45:0/138951962 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb35c02adf0 con 0x7fb368162f50 2024-05-08T05:46:59.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.972+0000 7fb36c874640 1 -- 172.21.15.45:0/138951962 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c032660 con 0x7fb368162f50 2024-05-08T05:46:59.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.972+0000 7fb36fada640 1 -- 172.21.15.45:0/138951962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb368162f50 msgr2=0x7fb3681633b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:59.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.972+0000 7fb36fada640 1 --2- 172.21.15.45:0/138951962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb368162f50 0x7fb3681633b0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009870 tx=0x7fb35c02a0a0 comp rx=0 tx=0).stop 2024-05-08T05:46:59.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.972+0000 7fb36fada640 1 -- 172.21.15.45:0/138951962 shutdown_connections 2024-05-08T05:46:59.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.972+0000 7fb36fada640 1 --2- 172.21.15.45:0/138951962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb368162f50 0x7fb3681633b0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:59.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.972+0000 7fb36fada640 1 --2- 172.21.15.45:0/138951962 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3680b21f0 0x7fb36806e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:59.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.972+0000 7fb36fada640 1 -- 172.21.15.45:0/138951962 >> 172.21.15.45:0/138951962 conn(0x7fb36809ba40 msgr2=0x7fb36809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:59.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.972+0000 7fb36fada640 1 -- 172.21.15.45:0/138951962 shutdown_connections 2024-05-08T05:46:59.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.973+0000 7fb36fada640 1 -- 172.21.15.45:0/138951962 wait complete. 2024-05-08T05:46:59.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.974+0000 7fb36fada640 1 Processor -- start 2024-05-08T05:46:59.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.974+0000 7fb36fada640 1 -- start start 2024-05-08T05:46:59.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.974+0000 7fb36fada640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3680b21f0 0x7fb368158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:59.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.974+0000 7fb36fada640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3680cf790 0x7fb3681e9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:59.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.974+0000 7fb36fada640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb368162f50 0x7fb3680c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:59.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.974+0000 7fb36fada640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb36806ea80 con 0x7fb3680b21f0 2024-05-08T05:46:59.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.974+0000 7fb36fada640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb36806e0b0 con 0x7fb368162f50 2024-05-08T05:46:59.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.974+0000 7fb36fada640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb3680b25d0 con 0x7fb3680cf790 2024-05-08T05:46:59.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.974+0000 7fb36d876640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3680b21f0 0x7fb368158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:59.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.975+0000 7fb36d876640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3680b21f0 0x7fb368158290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51970/0 (socket says 172.21.15.45:51970) 2024-05-08T05:46:59.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.975+0000 7fb36d876640 1 -- 172.21.15.45:0/1037256250 learned_addr learned my addr 172.21.15.45:0/1037256250 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:46:59.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.975+0000 7fb36d075640 1 --2- 172.21.15.45:0/1037256250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3680cf790 0x7fb3681e9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:59.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.975+0000 7fb36e077640 1 --2- 172.21.15.45:0/1037256250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb368162f50 0x7fb3680c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:59.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.975+0000 7fb36d876640 1 -- 172.21.15.45:0/1037256250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3680cf790 msgr2=0x7fb3681e9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:59.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.975+0000 7fb36d876640 1 --2- 172.21.15.45:0/1037256250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3680cf790 0x7fb3681e9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:59.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.975+0000 7fb36d876640 1 -- 172.21.15.45:0/1037256250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb368162f50 msgr2=0x7fb3680c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:59.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.975+0000 7fb36d876640 1 --2- 172.21.15.45:0/1037256250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb368162f50 0x7fb3680c7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:59.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.975+0000 7fb36d876640 1 -- 172.21.15.45:0/1037256250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3680b4410 con 0x7fb3680b21f0 2024-05-08T05:46:59.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.975+0000 7fb36d075640 1 --2- 172.21.15.45:0/1037256250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3680cf790 0x7fb3681e9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:59.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.975+0000 7fb36d876640 1 --2- 172.21.15.45:0/1037256250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3680b21f0 0x7fb368158290 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb36000c7f0 tx=0x7fb36000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:59.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.975+0000 7fb3667fc640 1 -- 172.21.15.45:0/1037256250 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360015070 con 0x7fb3680b21f0 2024-05-08T05:46:59.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.975+0000 7fb3667fc640 1 -- 172.21.15.45:0/1037256250 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb360004240 con 0x7fb3680b21f0 2024-05-08T05:46:59.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.975+0000 7fb3667fc640 1 -- 172.21.15.45:0/1037256250 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360007620 con 0x7fb3680b21f0 2024-05-08T05:46:59.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.975+0000 7fb36fada640 1 -- 172.21.15.45:0/1037256250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3681587d0 con 0x7fb3680b21f0 2024-05-08T05:46:59.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.975+0000 7fb36fada640 1 -- 172.21.15.45:0/1037256250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3681ea8f0 con 0x7fb3680b21f0 2024-05-08T05:46:59.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.976+0000 7fb36fada640 1 -- 172.21.15.45:0/1037256250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb368069b00 con 0x7fb3680b21f0 2024-05-08T05:46:59.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.977+0000 7fb3667fc640 1 -- 172.21.15.45:0/1037256250 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb3600079d0 con 0x7fb3680b21f0 2024-05-08T05:46:59.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.977+0000 7fb3667fc640 1 --2- 172.21.15.45:0/1037256250 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb3540762d0 0x7fb354078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:59.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.978+0000 7fb3667fc640 1 -- 172.21.15.45:0/1037256250 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6001+0+0 (secure 0 0 0) 0x7fb360097bb0 con 0x7fb3680b21f0 2024-05-08T05:46:59.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.978+0000 7fb36d075640 1 --2- 172.21.15.45:0/1037256250 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb3540762d0 0x7fb354078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:59.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.978+0000 7fb36d075640 1 --2- 172.21.15.45:0/1037256250 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb3540762d0 0x7fb354078790 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb35c034000 tx=0x7fb35c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:59.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:46:59.982+0000 7fb3667fc640 1 -- 172.21.15.45:0/1037256250 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb3600613e0 con 0x7fb3680b21f0 2024-05-08T05:47:00.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.210+0000 7fb36fada640 1 -- 172.21.15.45:0/1037256250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb368075af0 con 0x7fb3680b21f0 2024-05-08T05:47:00.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.828+0000 7fb3667fc640 1 -- 172.21.15.45:0/1037256250 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v56) ==== 174+0+0 (secure 0 0 0) 0x7fb3600693d0 con 0x7fb3680b21f0 2024-05-08T05:47:00.829 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 13 and data pool 14 2024-05-08T05:47:00.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.831+0000 7fb36fada640 1 -- 172.21.15.45:0/1037256250 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb3540762d0 msgr2=0x7fb354078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:00.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.831+0000 7fb36fada640 1 --2- 172.21.15.45:0/1037256250 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb3540762d0 0x7fb354078790 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb35c034000 tx=0x7fb35c02e000 comp rx=0 tx=0).stop 2024-05-08T05:47:00.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.831+0000 7fb36fada640 1 -- 172.21.15.45:0/1037256250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3680b21f0 msgr2=0x7fb368158290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:00.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.831+0000 7fb36fada640 1 --2- 172.21.15.45:0/1037256250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3680b21f0 0x7fb368158290 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fb36000c7f0 tx=0x7fb36000ccb0 comp rx=0 tx=0).stop 2024-05-08T05:47:00.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.831+0000 7fb36fada640 1 -- 172.21.15.45:0/1037256250 shutdown_connections 2024-05-08T05:47:00.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.831+0000 7fb36fada640 1 --2- 172.21.15.45:0/1037256250 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb3540762d0 0x7fb354078790 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:00.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.831+0000 7fb36fada640 1 --2- 172.21.15.45:0/1037256250 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb3680cf790 0x7fb3681e9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:00.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.831+0000 7fb36fada640 1 --2- 172.21.15.45:0/1037256250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb368162f50 0x7fb3680c7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:00.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.831+0000 7fb36fada640 1 --2- 172.21.15.45:0/1037256250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb3680b21f0 0x7fb368158290 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:00.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.832+0000 7fb36fada640 1 -- 172.21.15.45:0/1037256250 >> 172.21.15.45:0/1037256250 conn(0x7fb36809ba40 msgr2=0x7fb3680afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:00.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.832+0000 7fb36fada640 1 -- 172.21.15.45:0/1037256250 shutdown_connections 2024-05-08T05:47:00.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.832+0000 7fb36fada640 1 -- 172.21.15.45:0/1037256250 wait complete. 2024-05-08T05:47:00.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:47:00.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.969+0000 7fd609b46640 1 Processor -- start 2024-05-08T05:47:00.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.969+0000 7fd609b46640 1 -- start start 2024-05-08T05:47:00.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.970+0000 7fd609b46640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd6040d0ec0 0x7fd6040d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:00.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.970+0000 7fd609b46640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6040d3b00 0x7fd6040b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:00.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.970+0000 7fd609b46640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6040d1760 con 0x7fd60415f380 2024-05-08T05:47:00.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.970+0000 7fd609b46640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fd6040d30f0 con 0x7fd6040d0ec0 2024-05-08T05:47:00.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.970+0000 7fd609b46640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd6040d3f70 con 0x7fd6040d3b00 2024-05-08T05:47:00.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.970+0000 7fd602ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd6040d0ec0 0x7fd6040d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:00.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.970+0000 7fd6037fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd60415f380 0x7fd6040d07b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36790/0 (socket says 172.21.15.45:36790) 2024-05-08T05:47:00.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.970+0000 7fd6037fe640 1 -- 172.21.15.45:0/3597268349 learned_addr learned my addr 172.21.15.45:0/3597268349 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:47:00.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.970+0000 7fd603fff640 1 --2- 172.21.15.45:0/3597268349 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6040d3b00 0x7fd6040b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:00.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.970+0000 7fd602ffd640 1 -- 172.21.15.45:0/3597268349 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6040d3b00 msgr2=0x7fd6040b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:00.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.970+0000 7fd602ffd640 1 --2- 172.21.15.45:0/3597268349 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6040d3b00 0x7fd6040b7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:00.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.970+0000 7fd602ffd640 1 -- 172.21.15.45:0/3597268349 >> v1:172.21.15.45:6789/0 conn(0x7fd60415f380 legacy=0x7fd6040d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:00.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.970+0000 7fd6027fc640 1 -- 172.21.15.45:0/3597268349 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3328350963 0 0) 0x7fd6040d1760 con 0x7fd60415f380 2024-05-08T05:47:00.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.970+0000 7fd602ffd640 1 -- 172.21.15.45:0/3597268349 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6040d1a40 con 0x7fd6040d0ec0 2024-05-08T05:47:00.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.971+0000 7fd602ffd640 1 --2- 172.21.15.45:0/3597268349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6040d0ec0 0x7fd6040d35c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80097a0 tx=0x7fd5f802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbd39e7f9583902 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:00.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.971+0000 7fd6027fc640 1 -- 172.21.15.45:0/3597268349 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8037070 con 0x7fd6040d0ec0 2024-05-08T05:47:00.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.971+0000 7fd6027fc640 1 -- 172.21.15.45:0/3597268349 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5f8032440 con 0x7fd6040d0ec0 2024-05-08T05:47:00.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.971+0000 7fd6027fc640 1 -- 172.21.15.45:0/3597268349 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8032810 con 0x7fd6040d0ec0 2024-05-08T05:47:00.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.972+0000 7fd609b46640 1 -- 172.21.15.45:0/3597268349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6040d0ec0 msgr2=0x7fd6040d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.972+0000 7fd609b46640 1 --2- 172.21.15.45:0/3597268349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6040d0ec0 0x7fd6040d35c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80097a0 tx=0x7fd5f802a180 comp rx=0 tx=0).stop 2024-05-08T05:47:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.972+0000 7fd609b46640 1 -- 172.21.15.45:0/3597268349 shutdown_connections 2024-05-08T05:47:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.972+0000 7fd609b46640 1 --2- 172.21.15.45:0/3597268349 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6040d3b00 0x7fd6040b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.972+0000 7fd609b46640 1 --2- 172.21.15.45:0/3597268349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6040d0ec0 0x7fd6040d35c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.972+0000 7fd609b46640 1 -- 172.21.15.45:0/3597268349 >> 172.21.15.45:0/3597268349 conn(0x7fd6040d4fe0 msgr2=0x7fd6040d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.972+0000 7fd609b46640 1 -- 172.21.15.45:0/3597268349 shutdown_connections 2024-05-08T05:47:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.972+0000 7fd609b46640 1 -- 172.21.15.45:0/3597268349 wait complete. 2024-05-08T05:47:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.973+0000 7fd609b46640 1 Processor -- start 2024-05-08T05:47:00.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.973+0000 7fd609b46640 1 -- start start 2024-05-08T05:47:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.974+0000 7fd609b46640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6040d0ec0 0x7fd6040b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.974+0000 7fd609b46640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6040d3b00 0x7fd6040b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.974+0000 7fd609b46640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd60415f380 0x7fd6041f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.974+0000 7fd609b46640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd6040d2000 con 0x7fd60415f380 2024-05-08T05:47:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.974+0000 7fd609b46640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd6040d2370 con 0x7fd6040d0ec0 2024-05-08T05:47:00.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.974+0000 7fd609b46640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd60415f760 con 0x7fd6040d3b00 2024-05-08T05:47:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.974+0000 7fd603fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd60415f380 0x7fd6041f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.974+0000 7fd603fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd60415f380 0x7fd6041f50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43284/0 (socket says 172.21.15.45:43284) 2024-05-08T05:47:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.974+0000 7fd603fff640 1 -- 172.21.15.45:0/1671719557 learned_addr learned my addr 172.21.15.45:0/1671719557 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.974+0000 7fd602ffd640 1 --2- 172.21.15.45:0/1671719557 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6040d3b00 0x7fd6040b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.974+0000 7fd6037fe640 1 --2- 172.21.15.45:0/1671719557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6040d0ec0 0x7fd6040b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.974+0000 7fd603fff640 1 -- 172.21.15.45:0/1671719557 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6040d3b00 msgr2=0x7fd6040b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.974+0000 7fd603fff640 1 --2- 172.21.15.45:0/1671719557 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6040d3b00 0x7fd6040b2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.974+0000 7fd603fff640 1 -- 172.21.15.45:0/1671719557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6040d0ec0 msgr2=0x7fd6040b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.974+0000 7fd603fff640 1 --2- 172.21.15.45:0/1671719557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6040d0ec0 0x7fd6040b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.974+0000 7fd603fff640 1 -- 172.21.15.45:0/1671719557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6040b3280 con 0x7fd60415f380 2024-05-08T05:47:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.975+0000 7fd603fff640 1 --2- 172.21.15.45:0/1671719557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd60415f380 0x7fd6041f50e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00d800 tx=0x7fd5fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.975+0000 7fd5ebfff640 1 -- 172.21.15.45:0/1671719557 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc00b6c0 con 0x7fd60415f380 2024-05-08T05:47:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.975+0000 7fd5ebfff640 1 -- 172.21.15.45:0/1671719557 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5fc019070 con 0x7fd60415f380 2024-05-08T05:47:00.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.975+0000 7fd609b46640 1 -- 172.21.15.45:0/1671719557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6040b3560 con 0x7fd60415f380 2024-05-08T05:47:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.975+0000 7fd5ebfff640 1 -- 172.21.15.45:0/1671719557 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc00bac0 con 0x7fd60415f380 2024-05-08T05:47:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.975+0000 7fd609b46640 1 -- 172.21.15.45:0/1671719557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6040b3b20 con 0x7fd60415f380 2024-05-08T05:47:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.976+0000 7fd609b46640 1 -- 172.21.15.45:0/1671719557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6040693f0 con 0x7fd60415f380 2024-05-08T05:47:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.976+0000 7fd5ebfff640 1 -- 172.21.15.45:0/1671719557 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd5fc00bc60 con 0x7fd60415f380 2024-05-08T05:47:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.977+0000 7fd5ebfff640 1 --2- 172.21.15.45:0/1671719557 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd5f00762d0 0x7fd5f0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:00.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.977+0000 7fd5ebfff640 1 -- 172.21.15.45:0/1671719557 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fd5fc097a60 con 0x7fd60415f380 2024-05-08T05:47:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.977+0000 7fd6037fe640 1 --2- 172.21.15.45:0/1671719557 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd5f00762d0 0x7fd5f0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:00.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.978+0000 7fd6037fe640 1 --2- 172.21.15.45:0/1671719557 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd5f00762d0 0x7fd5f0078790 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd5f40012e0 tx=0x7fd5f40016f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:00.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:00.982+0000 7fd5ebfff640 1 -- 172.21.15.45:0/1671719557 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd5fc060c70 con 0x7fd60415f380 2024-05-08T05:47:01.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.174+0000 7fd609b46640 1 -- 172.21.15.45:0/1671719557 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd60408cfd0 con 0x7fd60415f380 2024-05-08T05:47:01.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.176+0000 7fd5ebfff640 1 -- 172.21.15.45:0/1671719557 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+16850 (secure 0 0 0) 0x7fd5fc064f70 con 0x7fd60415f380 2024-05-08T05:47:01.176 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:47:01.176 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":87,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:47:00.211875+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:46:57.892382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-08T05:46:59.168198+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:47:01.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.178+0000 7fd609b46640 1 -- 172.21.15.45:0/1671719557 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd5f00762d0 msgr2=0x7fd5f0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:01.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.178+0000 7fd609b46640 1 --2- 172.21.15.45:0/1671719557 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd5f00762d0 0x7fd5f0078790 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd5f40012e0 tx=0x7fd5f40016f0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.178+0000 7fd609b46640 1 -- 172.21.15.45:0/1671719557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd60415f380 msgr2=0x7fd6041f50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:01.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.178+0000 7fd609b46640 1 --2- 172.21.15.45:0/1671719557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd60415f380 0x7fd6041f50e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00d800 tx=0x7fd5fc00dcc0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.178+0000 7fd609b46640 1 -- 172.21.15.45:0/1671719557 shutdown_connections 2024-05-08T05:47:01.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.178+0000 7fd609b46640 1 --2- 172.21.15.45:0/1671719557 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd5f00762d0 0x7fd5f0078790 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.178+0000 7fd609b46640 1 --2- 172.21.15.45:0/1671719557 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd6040d3b00 0x7fd6040b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.178+0000 7fd609b46640 1 --2- 172.21.15.45:0/1671719557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd60415f380 0x7fd6041f50e0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.178+0000 7fd609b46640 1 --2- 172.21.15.45:0/1671719557 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6040d0ec0 0x7fd6040b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.178+0000 7fd609b46640 1 -- 172.21.15.45:0/1671719557 >> 172.21.15.45:0/1671719557 conn(0x7fd6040d4fe0 msgr2=0x7fd6040d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:01.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.178+0000 7fd609b46640 1 -- 172.21.15.45:0/1671719557 shutdown_connections 2024-05-08T05:47:01.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.178+0000 7fd609b46640 1 -- 172.21.15.45:0/1671719557 wait complete. 2024-05-08T05:47:01.190 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:47:01.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.303+0000 7feadb924640 1 Processor -- start 2024-05-08T05:47:01.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.303+0000 7feadb924640 1 -- start start 2024-05-08T05:47:01.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.304+0000 7feadb924640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fead40d3cd0 0x7fead40cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:01.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.304+0000 7feadb924640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fead40c9ee0 0x7fead40d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:01.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.304+0000 7feadb924640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fead40d3330 con 0x7fead41629a0 2024-05-08T05:47:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.304+0000 7feadb924640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fead40d2170 con 0x7fead40c9ee0 2024-05-08T05:47:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.304+0000 7feadb924640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fead40d2450 con 0x7fead40d3cd0 2024-05-08T05:47:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.304+0000 7fead9ec1640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fead40c9ee0 0x7fead40d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.304+0000 7fead8ebf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fead40d3cd0 0x7fead40cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.304+0000 7fead9ec1640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fead40c9ee0 0x7fead40d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:39102/0 (socket says 172.21.15.45:39102) 2024-05-08T05:47:01.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.304+0000 7fead8ebf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fead40d3cd0 0x7fead40cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:46706/0 (socket says 172.21.15.45:46706) 2024-05-08T05:47:01.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.304+0000 7fead9ec1640 1 -- 172.21.15.45:0/518624359 learned_addr learned my addr 172.21.15.45:0/518624359 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:01.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.304+0000 7fead96c0640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fead41629a0 0x7fead40d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:53246/0 (socket says 172.21.15.45:53246) 2024-05-08T05:47:01.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.305+0000 7fead9ec1640 1 -- 172.21.15.45:0/518624359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fead40d3cd0 msgr2=0x7fead40cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:01.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.305+0000 7fead9ec1640 1 --2- 172.21.15.45:0/518624359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fead40d3cd0 0x7fead40cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.305+0000 7fead9ec1640 1 -- 172.21.15.45:0/518624359 >> v1:172.21.15.63:6789/0 conn(0x7fead41629a0 legacy=0x7fead40d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:01.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.305+0000 7fead9ec1640 1 -- 172.21.15.45:0/518624359 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead40d2730 con 0x7fead40c9ee0 2024-05-08T05:47:01.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.305+0000 7fead9ec1640 1 --2- 172.21.15.45:0/518624359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fead40c9ee0 0x7fead40d01e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7feacc009870 tx=0x7feacc02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28a2a8e5fb733177 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:01.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.306+0000 7feacbfff640 1 -- 172.21.15.45:0/518624359 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc037070 con 0x7fead40c9ee0 2024-05-08T05:47:01.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.306+0000 7feacbfff640 1 -- 172.21.15.45:0/518624359 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feacc032440 con 0x7fead40c9ee0 2024-05-08T05:47:01.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.306+0000 7feacbfff640 1 -- 172.21.15.45:0/518624359 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc032810 con 0x7fead40c9ee0 2024-05-08T05:47:01.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.306+0000 7feadb924640 1 -- 172.21.15.45:0/518624359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fead40c9ee0 msgr2=0x7fead40d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.306+0000 7feadb924640 1 --2- 172.21.15.45:0/518624359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fead40c9ee0 0x7fead40d01e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7feacc009870 tx=0x7feacc02a130 comp rx=0 tx=0).stop 2024-05-08T05:47:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.306+0000 7feadb924640 1 -- 172.21.15.45:0/518624359 shutdown_connections 2024-05-08T05:47:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.306+0000 7feadb924640 1 --2- 172.21.15.45:0/518624359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fead40c9ee0 0x7fead40d01e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.306+0000 7feadb924640 1 --2- 172.21.15.45:0/518624359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fead40d3cd0 0x7fead40cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.306+0000 7feadb924640 1 -- 172.21.15.45:0/518624359 >> 172.21.15.45:0/518624359 conn(0x7fead40d4fe0 msgr2=0x7fead40d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.306+0000 7feadb924640 1 -- 172.21.15.45:0/518624359 shutdown_connections 2024-05-08T05:47:01.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.306+0000 7feadb924640 1 -- 172.21.15.45:0/518624359 wait complete. 2024-05-08T05:47:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.307+0000 7feadb924640 1 Processor -- start 2024-05-08T05:47:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.307+0000 7feadb924640 1 -- start start 2024-05-08T05:47:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.308+0000 7feadb924640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40c9ee0 0x7fead414fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.308+0000 7feadb924640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fead40d3cd0 0x7fead40a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:01.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.308+0000 7feadb924640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fead41629a0 0x7fead40a2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.308+0000 7feadb924640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fead40cf440 con 0x7fead40c9ee0 2024-05-08T05:47:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.308+0000 7feadb924640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fead40cf2c0 con 0x7fead41629a0 2024-05-08T05:47:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.308+0000 7feadb924640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fead40d2dc0 con 0x7fead40d3cd0 2024-05-08T05:47:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.308+0000 7fead96c0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40c9ee0 0x7fead414fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.308+0000 7fead96c0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40c9ee0 0x7fead414fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43300/0 (socket says 172.21.15.45:43300) 2024-05-08T05:47:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.308+0000 7fead96c0640 1 -- 172.21.15.45:0/648353386 learned_addr learned my addr 172.21.15.45:0/648353386 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.309+0000 7fead8ebf640 1 --2- 172.21.15.45:0/648353386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fead40d3cd0 0x7fead40a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.309+0000 7fead9ec1640 1 --2- 172.21.15.45:0/648353386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fead41629a0 0x7fead40a2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.309+0000 7fead96c0640 1 -- 172.21.15.45:0/648353386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fead40d3cd0 msgr2=0x7fead40a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.309+0000 7fead96c0640 1 --2- 172.21.15.45:0/648353386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fead40d3cd0 0x7fead40a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.309+0000 7fead96c0640 1 -- 172.21.15.45:0/648353386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fead41629a0 msgr2=0x7fead40a2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.309+0000 7fead96c0640 1 --2- 172.21.15.45:0/648353386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fead41629a0 0x7fead40a2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.309+0000 7fead96c0640 1 -- 172.21.15.45:0/648353386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead40ba390 con 0x7fead40c9ee0 2024-05-08T05:47:01.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.309+0000 7fead96c0640 1 --2- 172.21.15.45:0/648353386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40c9ee0 0x7fead414fb40 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fead000c7d0 tx=0x7fead000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.309+0000 7feac9ffb640 1 -- 172.21.15.45:0/648353386 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead0017070 con 0x7fead40c9ee0 2024-05-08T05:47:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.309+0000 7feac9ffb640 1 -- 172.21.15.45:0/648353386 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fead000a640 con 0x7fead40c9ee0 2024-05-08T05:47:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.309+0000 7feac9ffb640 1 -- 172.21.15.45:0/648353386 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead000a9f0 con 0x7fead40c9ee0 2024-05-08T05:47:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.309+0000 7feadb924640 1 -- 172.21.15.45:0/648353386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead4150660 con 0x7fead40c9ee0 2024-05-08T05:47:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.309+0000 7feadb924640 1 -- 172.21.15.45:0/648353386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead4150c20 con 0x7fead40c9ee0 2024-05-08T05:47:01.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.310+0000 7feadb924640 1 -- 172.21.15.45:0/648353386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fead40693f0 con 0x7fead40c9ee0 2024-05-08T05:47:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.311+0000 7feac9ffb640 1 -- 172.21.15.45:0/648353386 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fead001a070 con 0x7fead40c9ee0 2024-05-08T05:47:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.311+0000 7feac9ffb640 1 --2- 172.21.15.45:0/648353386 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feab80762d0 0x7feab8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.311+0000 7feac9ffb640 1 -- 172.21.15.45:0/648353386 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fead0096d70 con 0x7fead40c9ee0 2024-05-08T05:47:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.312+0000 7fead8ebf640 1 --2- 172.21.15.45:0/648353386 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feab80762d0 0x7feab8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:01.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.312+0000 7fead8ebf640 1 --2- 172.21.15.45:0/648353386 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feab80762d0 0x7feab8078790 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fead40bacc0 tx=0x7feac4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:01.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.316+0000 7feac9ffb640 1 -- 172.21.15.45:0/648353386 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fead005fe50 con 0x7fead40c9ee0 2024-05-08T05:47:01.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.511+0000 7feadb924640 1 -- 172.21.15.45:0/648353386 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fead40d39d0 con 0x7fead40c9ee0 2024-05-08T05:47:01.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.512+0000 7feac9ffb640 1 -- 172.21.15.45:0/648353386 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+16850 (secure 0 0 0) 0x7fead0064150 con 0x7fead40c9ee0 2024-05-08T05:47:01.512 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:47:01.513 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":87,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:47:00.211875+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:46:57.892382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-08T05:46:59.168198+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:47:01.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.514+0000 7feadb924640 1 -- 172.21.15.45:0/648353386 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feab80762d0 msgr2=0x7feab8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:01.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.514+0000 7feadb924640 1 --2- 172.21.15.45:0/648353386 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feab80762d0 0x7feab8078790 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fead40bacc0 tx=0x7feac4008040 comp rx=0 tx=0).stop 2024-05-08T05:47:01.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.514+0000 7feadb924640 1 -- 172.21.15.45:0/648353386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40c9ee0 msgr2=0x7fead414fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:01.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.514+0000 7feadb924640 1 --2- 172.21.15.45:0/648353386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40c9ee0 0x7fead414fb40 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fead000c7d0 tx=0x7fead000cc90 comp rx=0 tx=0).stop 2024-05-08T05:47:01.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.514+0000 7feadb924640 1 -- 172.21.15.45:0/648353386 shutdown_connections 2024-05-08T05:47:01.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.514+0000 7feadb924640 1 --2- 172.21.15.45:0/648353386 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7feab80762d0 0x7feab8078790 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.514+0000 7feadb924640 1 --2- 172.21.15.45:0/648353386 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fead40d3cd0 0x7fead40a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.514+0000 7feadb924640 1 --2- 172.21.15.45:0/648353386 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fead41629a0 0x7fead40a2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.514+0000 7feadb924640 1 --2- 172.21.15.45:0/648353386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fead40c9ee0 0x7fead414fb40 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.514+0000 7feadb924640 1 -- 172.21.15.45:0/648353386 >> 172.21.15.45:0/648353386 conn(0x7fead40d4fe0 msgr2=0x7fead40cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:01.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.514+0000 7feadb924640 1 -- 172.21.15.45:0/648353386 shutdown_connections 2024-05-08T05:47:01.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.514+0000 7feadb924640 1 -- 172.21.15.45:0/648353386 wait complete. 2024-05-08T05:47:01.526 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T05:47:01.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.632+0000 7fb7bc7a4640 1 Processor -- start 2024-05-08T05:47:01.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.632+0000 7fb7bc7a4640 1 -- start start 2024-05-08T05:47:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.633+0000 7fb7bc7a4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7b415f4e0 0x7fb7b40cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.633+0000 7fb7bc7a4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb7b40d3aa0 0x7fb7b40d4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.633+0000 7fb7bc7a4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb7b40d3f00 0x7fb7b40d54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.633+0000 7fb7bc7a4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb7b40d0320 con 0x7fb7b40d3f00 2024-05-08T05:47:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.633+0000 7fb7bc7a4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb7b40d0600 con 0x7fb7b40d3aa0 2024-05-08T05:47:01.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.633+0000 7fb7bc7a4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b40d08e0 con 0x7fb7b415f4e0 2024-05-08T05:47:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.634+0000 7fb7bad41640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb7b40d3f00 0x7fb7b40d54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.634+0000 7fb7bad41640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb7b40d3f00 0x7fb7b40d54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43302/0 (socket says 172.21.15.45:43302) 2024-05-08T05:47:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.634+0000 7fb7bad41640 1 -- 172.21.15.45:0/3141369242 learned_addr learned my addr 172.21.15.45:0/3141369242 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.634+0000 7fb7b9d3f640 1 --2- 172.21.15.45:0/3141369242 >> v2:172.21.15.63:3300/0 conn(0x7fb7b40d3aa0 0x7fb7b40d4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.634+0000 7fb7ba540640 1 --2- 172.21.15.45:0/3141369242 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7b415f4e0 0x7fb7b40cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.634+0000 7fb7bad41640 1 -- 172.21.15.45:0/3141369242 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7b415f4e0 msgr2=0x7fb7b40cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.634+0000 7fb7bad41640 1 --2- 172.21.15.45:0/3141369242 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7b415f4e0 0x7fb7b40cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.634+0000 7fb7bad41640 1 -- 172.21.15.45:0/3141369242 >> v2:172.21.15.63:3300/0 conn(0x7fb7b40d3aa0 msgr2=0x7fb7b40d4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.634+0000 7fb7bad41640 1 --2- 172.21.15.45:0/3141369242 >> v2:172.21.15.63:3300/0 conn(0x7fb7b40d3aa0 0x7fb7b40d4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.634+0000 7fb7bad41640 1 -- 172.21.15.45:0/3141369242 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b40d0bc0 con 0x7fb7b40d3f00 2024-05-08T05:47:01.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.634+0000 7fb7ba540640 1 --2- 172.21.15.45:0/3141369242 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7b415f4e0 0x7fb7b40cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:01.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.635+0000 7fb7bad41640 1 --2- 172.21.15.45:0/3141369242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7b40d3f00 0x7fb7b40d54e0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40d2b90 tx=0x7fb7ac02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06255c06e515f53 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:01.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.635+0000 7fb7b953e640 1 -- 172.21.15.45:0/3141369242 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac036070 con 0x7fb7b40d3f00 2024-05-08T05:47:01.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.635+0000 7fb7b953e640 1 -- 172.21.15.45:0/3141369242 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7ac031600 con 0x7fb7b40d3f00 2024-05-08T05:47:01.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.635+0000 7fb7b953e640 1 -- 172.21.15.45:0/3141369242 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac0319d0 con 0x7fb7b40d3f00 2024-05-08T05:47:01.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.635+0000 7fb7bc7a4640 1 -- 172.21.15.45:0/3141369242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7b40d3f00 msgr2=0x7fb7b40d54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:01.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.635+0000 7fb7bc7a4640 1 --2- 172.21.15.45:0/3141369242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7b40d3f00 0x7fb7b40d54e0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40d2b90 tx=0x7fb7ac02a6a0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.635+0000 7fb7bc7a4640 1 -- 172.21.15.45:0/3141369242 shutdown_connections 2024-05-08T05:47:01.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.635+0000 7fb7bc7a4640 1 --2- 172.21.15.45:0/3141369242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7b40d3f00 0x7fb7b40d54e0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.635+0000 7fb7bc7a4640 1 --2- 172.21.15.45:0/3141369242 >> v2:172.21.15.63:3300/0 conn(0x7fb7b40d3aa0 0x7fb7b40d4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.635+0000 7fb7bc7a4640 1 --2- 172.21.15.45:0/3141369242 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7b415f4e0 0x7fb7b40cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.635+0000 7fb7bc7a4640 1 -- 172.21.15.45:0/3141369242 >> 172.21.15.45:0/3141369242 conn(0x7fb7b408eac0 msgr2=0x7fb7b409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:01.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.635+0000 7fb7bc7a4640 1 -- 172.21.15.45:0/3141369242 shutdown_connections 2024-05-08T05:47:01.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.635+0000 7fb7bc7a4640 1 -- 172.21.15.45:0/3141369242 wait complete. 2024-05-08T05:47:01.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.637+0000 7fb7bc7a4640 1 Processor -- start 2024-05-08T05:47:01.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.637+0000 7fb7bc7a4640 1 -- start start 2024-05-08T05:47:01.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.637+0000 7fb7bc7a4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7b40d3aa0 0x7fb7b415fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:01.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.637+0000 7fb7bc7a4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b40d3f00 0x7fb7b41f8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:01.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.637+0000 7fb7bc7a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7b415f4e0 0x7fb7b40b8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:01.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.637+0000 7fb7bc7a4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b40b50d0 con 0x7fb7b415f4e0 2024-05-08T05:47:01.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.637+0000 7fb7bc7a4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b415f8c0 con 0x7fb7b40d3f00 2024-05-08T05:47:01.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.637+0000 7fb7bc7a4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b40b5250 con 0x7fb7b40d3aa0 2024-05-08T05:47:01.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.638+0000 7fb7bad41640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7b415f4e0 0x7fb7b40b8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.638+0000 7fb7bad41640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7b415f4e0 0x7fb7b40b8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43304/0 (socket says 172.21.15.45:43304) 2024-05-08T05:47:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.638+0000 7fb7bad41640 1 -- 172.21.15.45:0/4130295603 learned_addr learned my addr 172.21.15.45:0/4130295603 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.638+0000 7fb7b9d3f640 1 --2- 172.21.15.45:0/4130295603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b40d3f00 0x7fb7b41f8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.638+0000 7fb7ba540640 1 --2- 172.21.15.45:0/4130295603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7b40d3aa0 0x7fb7b415fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.638+0000 7fb7bad41640 1 -- 172.21.15.45:0/4130295603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7b40d3aa0 msgr2=0x7fb7b415fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.638+0000 7fb7bad41640 1 --2- 172.21.15.45:0/4130295603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7b40d3aa0 0x7fb7b415fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.638+0000 7fb7bad41640 1 -- 172.21.15.45:0/4130295603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b40d3f00 msgr2=0x7fb7b41f8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.638+0000 7fb7bad41640 1 --2- 172.21.15.45:0/4130295603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b40d3f00 0x7fb7b41f8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.638+0000 7fb7bad41640 1 -- 172.21.15.45:0/4130295603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b41f4cf0 con 0x7fb7b415f4e0 2024-05-08T05:47:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.638+0000 7fb7b9d3f640 1 --2- 172.21.15.45:0/4130295603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b40d3f00 0x7fb7b41f8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.638+0000 7fb7ba540640 1 --2- 172.21.15.45:0/4130295603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7b40d3aa0 0x7fb7b415fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.638+0000 7fb7bad41640 1 --2- 172.21.15.45:0/4130295603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7b415f4e0 0x7fb7b40b8f00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac034040 tx=0x7fb7ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.638+0000 7fb7aaffd640 1 -- 172.21.15.45:0/4130295603 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac041020 con 0x7fb7b415f4e0 2024-05-08T05:47:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.638+0000 7fb7aaffd640 1 -- 172.21.15.45:0/4130295603 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7ac0077a0 con 0x7fb7b415f4e0 2024-05-08T05:47:01.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.638+0000 7fb7aaffd640 1 -- 172.21.15.45:0/4130295603 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac036040 con 0x7fb7b415f4e0 2024-05-08T05:47:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.638+0000 7fb7bc7a4640 1 -- 172.21.15.45:0/4130295603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b41f4fd0 con 0x7fb7b415f4e0 2024-05-08T05:47:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.638+0000 7fb7bc7a4640 1 -- 172.21.15.45:0/4130295603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b41f5590 con 0x7fb7b415f4e0 2024-05-08T05:47:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.639+0000 7fb7bc7a4640 1 -- 172.21.15.45:0/4130295603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b40693f0 con 0x7fb7b415f4e0 2024-05-08T05:47:01.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.640+0000 7fb7aaffd640 1 -- 172.21.15.45:0/4130295603 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb7ac044020 con 0x7fb7b415f4e0 2024-05-08T05:47:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.640+0000 7fb7aaffd640 1 --2- 172.21.15.45:0/4130295603 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb7980762d0 0x7fb798078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.641+0000 7fb7aaffd640 1 -- 172.21.15.45:0/4130295603 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6101+0+0 (secure 0 0 0) 0x7fb7ac0b84a0 con 0x7fb7b415f4e0 2024-05-08T05:47:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.641+0000 7fb7ba540640 1 --2- 172.21.15.45:0/4130295603 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb7980762d0 0x7fb798078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:01.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.641+0000 7fb7ba540640 1 --2- 172.21.15.45:0/4130295603 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb7980762d0 0x7fb798078790 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0006fd0 tx=0x7fb7b0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:01.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.645+0000 7fb7aaffd640 1 -- 172.21.15.45:0/4130295603 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb7ac04b030 con 0x7fb7b415f4e0 2024-05-08T05:47:01.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:01.882+0000 7fb7bc7a4640 1 -- 172.21.15.45:0/4130295603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb7b41f5760 con 0x7fb7b415f4e0 2024-05-08T05:47:02.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.850+0000 7fb7aaffd640 1 -- 172.21.15.45:0/4130295603 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v59) ==== 118+0+0 (secure 0 0 0) 0x7fb7ac081bd0 con 0x7fb7b415f4e0 2024-05-08T05:47:02.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.853+0000 7fb7bc7a4640 1 -- 172.21.15.45:0/4130295603 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb7980762d0 msgr2=0x7fb798078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.853+0000 7fb7bc7a4640 1 --2- 172.21.15.45:0/4130295603 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb7980762d0 0x7fb798078790 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0006fd0 tx=0x7fb7b0008040 comp rx=0 tx=0).stop 2024-05-08T05:47:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.853+0000 7fb7bc7a4640 1 -- 172.21.15.45:0/4130295603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7b415f4e0 msgr2=0x7fb7b40b8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.853+0000 7fb7bc7a4640 1 --2- 172.21.15.45:0/4130295603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7b415f4e0 0x7fb7b40b8f00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac034040 tx=0x7fb7ac004a40 comp rx=0 tx=0).stop 2024-05-08T05:47:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.853+0000 7fb7bc7a4640 1 -- 172.21.15.45:0/4130295603 shutdown_connections 2024-05-08T05:47:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.853+0000 7fb7bc7a4640 1 --2- 172.21.15.45:0/4130295603 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb7980762d0 0x7fb798078790 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.853+0000 7fb7bc7a4640 1 --2- 172.21.15.45:0/4130295603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb7b415f4e0 0x7fb7b40b8f00 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.853+0000 7fb7bc7a4640 1 --2- 172.21.15.45:0/4130295603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb7b40d3f00 0x7fb7b41f8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.853+0000 7fb7bc7a4640 1 --2- 172.21.15.45:0/4130295603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb7b40d3aa0 0x7fb7b415fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:02.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.853+0000 7fb7bc7a4640 1 -- 172.21.15.45:0/4130295603 >> 172.21.15.45:0/4130295603 conn(0x7fb7b408eac0 msgr2=0x7fb7b40d13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:02.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.853+0000 7fb7bc7a4640 1 -- 172.21.15.45:0/4130295603 shutdown_connections 2024-05-08T05:47:02.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.853+0000 7fb7bc7a4640 1 -- 172.21.15.45:0/4130295603 wait complete. 2024-05-08T05:47:02.867 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T05:47:02.867 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T05:47:02.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.980+0000 7f47f919e640 1 Processor -- start 2024-05-08T05:47:02.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.980+0000 7f47f919e640 1 -- start start 2024-05-08T05:47:02.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.981+0000 7f47f919e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f47f4162680 0x7f47f40d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:02.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.981+0000 7f47f919e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f47f4162a60 0x7f47f40b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:02.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.981+0000 7f47f919e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47f40d3220 0x7f47f40d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:02.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.981+0000 7f47f919e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f47f40d27f0 con 0x7f47f4162a60 2024-05-08T05:47:02.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.981+0000 7f47f919e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f47f40cfd30 con 0x7f47f4162680 2024-05-08T05:47:02.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.981+0000 7f47f919e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f47f40b4460 con 0x7f47f40d3220 2024-05-08T05:47:02.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.981+0000 7f47f37fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f47f4162a60 0x7f47f40b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:02.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.981+0000 7f47f37fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f47f4162a60 0x7f47f40b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43312/0 (socket says 172.21.15.45:43312) 2024-05-08T05:47:02.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.981+0000 7f47f37fe640 1 -- 172.21.15.45:0/20433166 learned_addr learned my addr 172.21.15.45:0/20433166 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:02.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.981+0000 7f47f3fff640 1 --2- 172.21.15.45:0/20433166 >> v2:172.21.15.63:3300/0 conn(0x7f47f4162680 0x7f47f40d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:02.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.981+0000 7f47f899d640 1 --2- 172.21.15.45:0/20433166 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47f40d3220 0x7f47f40d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:02.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.982+0000 7f47f37fe640 1 -- 172.21.15.45:0/20433166 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47f40d3220 msgr2=0x7f47f40d36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:02.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.982+0000 7f47f37fe640 1 --2- 172.21.15.45:0/20433166 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47f40d3220 0x7f47f40d36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:02.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.982+0000 7f47f37fe640 1 -- 172.21.15.45:0/20433166 >> v2:172.21.15.63:3300/0 conn(0x7f47f4162680 msgr2=0x7f47f40d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:02.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.982+0000 7f47f37fe640 1 --2- 172.21.15.45:0/20433166 >> v2:172.21.15.63:3300/0 conn(0x7f47f4162680 0x7f47f40d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:02.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.982+0000 7f47f37fe640 1 -- 172.21.15.45:0/20433166 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f40d1f50 con 0x7f47f4162a60 2024-05-08T05:47:02.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.982+0000 7f47f37fe640 1 --2- 172.21.15.45:0/20433166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f4162a60 0x7f47f40b3f20 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f47f408d5f0 tx=0x7f47e802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626440dd3e2d840d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:02.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.983+0000 7f47f2ffd640 1 -- 172.21.15.45:0/20433166 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8037070 con 0x7f47f4162a60 2024-05-08T05:47:02.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.983+0000 7f47f2ffd640 1 -- 172.21.15.45:0/20433166 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47e8032440 con 0x7f47f4162a60 2024-05-08T05:47:02.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.983+0000 7f47f2ffd640 1 -- 172.21.15.45:0/20433166 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8032810 con 0x7f47f4162a60 2024-05-08T05:47:02.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.983+0000 7f47f919e640 1 -- 172.21.15.45:0/20433166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f4162a60 msgr2=0x7f47f40b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:02.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.983+0000 7f47f919e640 1 --2- 172.21.15.45:0/20433166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f4162a60 0x7f47f40b3f20 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f47f408d5f0 tx=0x7f47e802a1e0 comp rx=0 tx=0).stop 2024-05-08T05:47:02.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.983+0000 7f47f919e640 1 -- 172.21.15.45:0/20433166 shutdown_connections 2024-05-08T05:47:02.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.983+0000 7f47f919e640 1 --2- 172.21.15.45:0/20433166 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47f40d3220 0x7f47f40d36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:02.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.983+0000 7f47f919e640 1 --2- 172.21.15.45:0/20433166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f4162a60 0x7f47f40b3f20 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:02.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.983+0000 7f47f919e640 1 --2- 172.21.15.45:0/20433166 >> v2:172.21.15.63:3300/0 conn(0x7f47f4162680 0x7f47f40d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:02.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.983+0000 7f47f919e640 1 -- 172.21.15.45:0/20433166 >> 172.21.15.45:0/20433166 conn(0x7f47f409e1d0 msgr2=0x7f47f40d4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:02.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.983+0000 7f47f919e640 1 -- 172.21.15.45:0/20433166 shutdown_connections 2024-05-08T05:47:02.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.983+0000 7f47f919e640 1 -- 172.21.15.45:0/20433166 wait complete. 2024-05-08T05:47:02.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.985+0000 7f47f919e640 1 Processor -- start 2024-05-08T05:47:02.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.985+0000 7f47f919e640 1 -- start start 2024-05-08T05:47:02.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.985+0000 7f47f919e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47f40d3220 0x7f47f40b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:02.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.985+0000 7f47f919e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47f4162680 0x7f47f40caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.986+0000 7f47f919e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f4162a60 0x7f47f4156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.986+0000 7f47f919e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f47f40b1ba0 con 0x7f47f4162a60 2024-05-08T05:47:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.986+0000 7f47f919e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f47f40d2510 con 0x7f47f40d3220 2024-05-08T05:47:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.986+0000 7f47f919e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f47f40b1d20 con 0x7f47f4162680 2024-05-08T05:47:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.986+0000 7f47f899d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f4162a60 0x7f47f4156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.986+0000 7f47f899d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f4162a60 0x7f47f4156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43326/0 (socket says 172.21.15.45:43326) 2024-05-08T05:47:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.986+0000 7f47f899d640 1 -- 172.21.15.45:0/383211381 learned_addr learned my addr 172.21.15.45:0/383211381 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.986+0000 7f47f37fe640 1 --2- 172.21.15.45:0/383211381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47f4162680 0x7f47f40caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.986+0000 7f47f3fff640 1 --2- 172.21.15.45:0/383211381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47f40d3220 0x7f47f40b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.986+0000 7f47f37fe640 1 -- 172.21.15.45:0/383211381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47f40d3220 msgr2=0x7f47f40b6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.986+0000 7f47f37fe640 1 --2- 172.21.15.45:0/383211381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47f40d3220 0x7f47f40b6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.986+0000 7f47f37fe640 1 -- 172.21.15.45:0/383211381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f4162a60 msgr2=0x7f47f4156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.986+0000 7f47f37fe640 1 --2- 172.21.15.45:0/383211381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f4162a60 0x7f47f4156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:02.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.986+0000 7f47f37fe640 1 -- 172.21.15.45:0/383211381 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f40b45e0 con 0x7f47f4162680 2024-05-08T05:47:02.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.986+0000 7f47f899d640 1 --2- 172.21.15.45:0/383211381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f4162a60 0x7f47f4156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:47:02.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.986+0000 7f47f37fe640 1 --2- 172.21.15.45:0/383211381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47f4162680 0x7f47f40caeb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f47e8031000 tx=0x7f47e80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:02.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.987+0000 7f47f0ff9640 1 -- 172.21.15.45:0/383211381 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8041020 con 0x7f47f4162680 2024-05-08T05:47:02.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.987+0000 7f47f0ff9640 1 -- 172.21.15.45:0/383211381 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47e803d620 con 0x7f47f4162680 2024-05-08T05:47:02.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.987+0000 7f47f919e640 1 -- 172.21.15.45:0/383211381 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f40b48c0 con 0x7f47f4162680 2024-05-08T05:47:02.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.987+0000 7f47f0ff9640 1 -- 172.21.15.45:0/383211381 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8037040 con 0x7f47f4162680 2024-05-08T05:47:02.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.987+0000 7f47f919e640 1 -- 172.21.15.45:0/383211381 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f40b4e80 con 0x7f47f4162680 2024-05-08T05:47:02.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.988+0000 7f47f919e640 1 -- 172.21.15.45:0/383211381 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f4050740 con 0x7f47f4162680 2024-05-08T05:47:02.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.988+0000 7f47f0ff9640 1 -- 172.21.15.45:0/383211381 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f47e8044020 con 0x7f47f4162680 2024-05-08T05:47:02.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.989+0000 7f47f0ff9640 1 --2- 172.21.15.45:0/383211381 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f47e0076300 0x7f47e00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:02.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.989+0000 7f47f0ff9640 1 -- 172.21.15.45:0/383211381 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(88..88 src has 1..88) ==== 6101+0+0 (secure 0 0 0) 0x7f47e80b8130 con 0x7f47f4162680 2024-05-08T05:47:02.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.989+0000 7f47f3fff640 1 --2- 172.21.15.45:0/383211381 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f47e0076300 0x7f47e00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:02.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.990+0000 7f47f3fff640 1 --2- 172.21.15.45:0/383211381 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f47e0076300 0x7f47e00787c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009ac0 tx=0x7f47e4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:02.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:02.994+0000 7f47f0ff9640 1 -- 172.21.15.45:0/383211381 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f47e804b030 con 0x7f47f4162680 2024-05-08T05:47:03.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.227+0000 7f47f919e640 1 -- 172.21.15.45:0/383211381 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f47f409e6b0 con 0x7f47f4162680 2024-05-08T05:47:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.857+0000 7f47f0ff9640 1 -- 172.21.15.45:0/383211381 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v60) ==== 122+0+0 (secure 0 0 0) 0x7f47e8081290 con 0x7f47f4162680 2024-05-08T05:47:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.860+0000 7f47f919e640 1 -- 172.21.15.45:0/383211381 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f47e0076300 msgr2=0x7f47e00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.860+0000 7f47f919e640 1 --2- 172.21.15.45:0/383211381 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f47e0076300 0x7f47e00787c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009ac0 tx=0x7f47e4009190 comp rx=0 tx=0).stop 2024-05-08T05:47:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.860+0000 7f47f919e640 1 -- 172.21.15.45:0/383211381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47f4162680 msgr2=0x7f47f40caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.860+0000 7f47f919e640 1 --2- 172.21.15.45:0/383211381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47f4162680 0x7f47f40caeb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f47e8031000 tx=0x7f47e80046c0 comp rx=0 tx=0).stop 2024-05-08T05:47:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.860+0000 7f47f919e640 1 -- 172.21.15.45:0/383211381 shutdown_connections 2024-05-08T05:47:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.860+0000 7f47f919e640 1 --2- 172.21.15.45:0/383211381 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f47e0076300 0x7f47e00787c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.860+0000 7f47f919e640 1 --2- 172.21.15.45:0/383211381 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f47f4162680 0x7f47f40caeb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.860+0000 7f47f919e640 1 --2- 172.21.15.45:0/383211381 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47f4162a60 0x7f47f4156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.860+0000 7f47f919e640 1 --2- 172.21.15.45:0/383211381 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47f40d3220 0x7f47f40b6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.860+0000 7f47f919e640 1 -- 172.21.15.45:0/383211381 >> 172.21.15.45:0/383211381 conn(0x7f47f409e1d0 msgr2=0x7f47f40c8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.860+0000 7f47f919e640 1 -- 172.21.15.45:0/383211381 shutdown_connections 2024-05-08T05:47:03.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.860+0000 7f47f919e640 1 -- 172.21.15.45:0/383211381 wait complete. 2024-05-08T05:47:03.873 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T05:47:03.873 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:47:03.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.982+0000 7f401ddef640 1 Processor -- start 2024-05-08T05:47:03.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.982+0000 7f401ddef640 1 -- start start 2024-05-08T05:47:03.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.983+0000 7f401ddef640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f401815c340 0x7f4018166970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:03.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.983+0000 7f401ddef640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f401815c920 con 0x7f401806e0f0 2024-05-08T05:47:03.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.983+0000 7f401ddef640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f401815cc00 con 0x7f4018072840 2024-05-08T05:47:03.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.983+0000 7f401ddef640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f401815cee0 con 0x7f401815c340 2024-05-08T05:47:03.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.983+0000 7f40177fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f401806e0f0 0x7f40180d0f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36800/0 (socket says 172.21.15.45:36800) 2024-05-08T05:47:03.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.983+0000 7f40177fe640 1 -- 172.21.15.45:0/675162090 learned_addr learned my addr 172.21.15.45:0/675162090 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:47:03.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.983+0000 7f4017fff640 1 --2- 172.21.15.45:0/675162090 >> v2:172.21.15.63:3300/0 conn(0x7f401815c340 0x7f4018166970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:03.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.983+0000 7f40167fc640 1 -- 172.21.15.45:0/675162090 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1514196745 0 0) 0x7f401815c920 con 0x7f401806e0f0 2024-05-08T05:47:03.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.983+0000 7f40167fc640 1 -- 172.21.15.45:0/675162090 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4000003520 con 0x7f401806e0f0 2024-05-08T05:47:03.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.984+0000 7f40167fc640 1 -- 172.21.15.45:0/675162090 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3549841005 0 0) 0x7f4000003520 con 0x7f401806e0f0 2024-05-08T05:47:03.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.984+0000 7f4017fff640 1 -- 172.21.15.45:0/675162090 >> v1:172.21.15.63:6789/0 conn(0x7f4018072840 legacy=0x7f401806e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:03.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.984+0000 7f4017fff640 1 -- 172.21.15.45:0/675162090 >> v1:172.21.15.45:6789/0 conn(0x7f401806e0f0 legacy=0x7f40180d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:03.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.984+0000 7f4017fff640 1 -- 172.21.15.45:0/675162090 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401815d1c0 con 0x7f401815c340 2024-05-08T05:47:03.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.984+0000 7f4017fff640 1 --2- 172.21.15.45:0/675162090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f401815c340 0x7f4018166970 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4010009d90 tx=0x7f401002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2770f641b4ab5f30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:03.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.985+0000 7f40167fc640 1 -- 172.21.15.45:0/675162090 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40080016f0 con 0x7f401815c340 2024-05-08T05:47:03.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.985+0000 7f40167fc640 1 -- 172.21.15.45:0/675162090 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4010007540 con 0x7f401815c340 2024-05-08T05:47:03.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.985+0000 7f40167fc640 1 -- 172.21.15.45:0/675162090 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010007910 con 0x7f401815c340 2024-05-08T05:47:03.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.985+0000 7f401ddef640 1 -- 172.21.15.45:0/675162090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f401815c340 msgr2=0x7f4018166970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:03.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.985+0000 7f401ddef640 1 --2- 172.21.15.45:0/675162090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f401815c340 0x7f4018166970 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4010009d90 tx=0x7f401002a5f0 comp rx=0 tx=0).stop 2024-05-08T05:47:03.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.985+0000 7f401ddef640 1 -- 172.21.15.45:0/675162090 shutdown_connections 2024-05-08T05:47:03.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.985+0000 7f401ddef640 1 --2- 172.21.15.45:0/675162090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f401815c340 0x7f4018166970 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:03.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.985+0000 7f401ddef640 1 -- 172.21.15.45:0/675162090 >> 172.21.15.45:0/675162090 conn(0x7f401806c3a0 msgr2=0x7f401806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:03.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.985+0000 7f401ddef640 1 -- 172.21.15.45:0/675162090 shutdown_connections 2024-05-08T05:47:03.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.985+0000 7f401ddef640 1 -- 172.21.15.45:0/675162090 wait complete. 2024-05-08T05:47:03.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.986+0000 7f401ddef640 1 Processor -- start 2024-05-08T05:47:03.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.987+0000 7f401ddef640 1 -- start start 2024-05-08T05:47:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.987+0000 7f401ddef640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f401806e0f0 0x7f4018157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.987+0000 7f401ddef640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4018072840 0x7f4018155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.987+0000 7f401ddef640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f401815c340 0x7f4018157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.987+0000 7f401ddef640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4018153230 con 0x7f4018072840 2024-05-08T05:47:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.987+0000 7f401ddef640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f40181530b0 con 0x7f401815c340 2024-05-08T05:47:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.987+0000 7f401ddef640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4018166eb0 con 0x7f401806e0f0 2024-05-08T05:47:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.987+0000 7f4016ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4018072840 0x7f4018155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.987+0000 7f4016ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4018072840 0x7f4018155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43330/0 (socket says 172.21.15.45:43330) 2024-05-08T05:47:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.987+0000 7f4016ffd640 1 -- 172.21.15.45:0/79696089 learned_addr learned my addr 172.21.15.45:0/79696089 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.988+0000 7f4017fff640 1 --2- 172.21.15.45:0/79696089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f401815c340 0x7f4018157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.988+0000 7f40177fe640 1 --2- 172.21.15.45:0/79696089 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f401806e0f0 0x7f4018157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.988+0000 7f4016ffd640 1 -- 172.21.15.45:0/79696089 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f401806e0f0 msgr2=0x7f4018157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:03.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.988+0000 7f4016ffd640 1 --2- 172.21.15.45:0/79696089 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f401806e0f0 0x7f4018157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:03.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.988+0000 7f4016ffd640 1 -- 172.21.15.45:0/79696089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f401815c340 msgr2=0x7f4018157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:03.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.988+0000 7f4016ffd640 1 --2- 172.21.15.45:0/79696089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f401815c340 0x7f4018157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:03.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.988+0000 7f4016ffd640 1 -- 172.21.15.45:0/79696089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401814b530 con 0x7f4018072840 2024-05-08T05:47:03.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.988+0000 7f4016ffd640 1 --2- 172.21.15.45:0/79696089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4018072840 0x7f4018155750 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f400c00c4d0 tx=0x7f400c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:03.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.988+0000 7f3ffffff640 1 -- 172.21.15.45:0/79696089 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c0052d0 con 0x7f4018072840 2024-05-08T05:47:03.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.988+0000 7f3ffffff640 1 -- 172.21.15.45:0/79696089 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f400c005470 con 0x7f4018072840 2024-05-08T05:47:03.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.988+0000 7f401ddef640 1 -- 172.21.15.45:0/79696089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f401814b810 con 0x7f4018072840 2024-05-08T05:47:03.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.988+0000 7f3ffffff640 1 -- 172.21.15.45:0/79696089 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c012830 con 0x7f4018072840 2024-05-08T05:47:03.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.988+0000 7f401ddef640 1 -- 172.21.15.45:0/79696089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f401814bdd0 con 0x7f4018072840 2024-05-08T05:47:03.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.989+0000 7f401ddef640 1 -- 172.21.15.45:0/79696089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40180693f0 con 0x7f4018072840 2024-05-08T05:47:03.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.990+0000 7f3ffffff640 1 -- 172.21.15.45:0/79696089 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f400c0129d0 con 0x7f4018072840 2024-05-08T05:47:03.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.990+0000 7f3ffffff640 1 --2- 172.21.15.45:0/79696089 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f40040762d0 0x7f4004078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:03.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.991+0000 7f3ffffff640 1 -- 172.21.15.45:0/79696089 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6101+0+0 (secure 0 0 0) 0x7f400c097380 con 0x7f4018072840 2024-05-08T05:47:03.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.991+0000 7f40177fe640 1 --2- 172.21.15.45:0/79696089 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f40040762d0 0x7f4004078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:03.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.991+0000 7f40177fe640 1 --2- 172.21.15.45:0/79696089 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f40040762d0 0x7f4004078790 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4008005880 tx=0x7f4008009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:03.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:03.995+0000 7f3ffffff640 1 -- 172.21.15.45:0/79696089 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f400c060b50 con 0x7f4018072840 2024-05-08T05:47:04.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.229+0000 7f401ddef640 1 -- 172.21.15.45:0/79696089 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40180765b0 con 0x7f4018072840 2024-05-08T05:47:04.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.230+0000 7f3ffffff640 1 -- 172.21.15.45:0/79696089 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5183 (secure 0 0 0) 0x7f400c060450 con 0x7f4018072840 2024-05-08T05:47:04.230 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:47:04.230 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6744,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2854804427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2854804427},{"type":"v1","addr":"172.21.15.45:6834","nonce":2854804427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2945763952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2945763952},{"type":"v1","addr":"172.21.15.45:6835","nonce":2945763952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6759,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3194497846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":3194497846},{"type":"v1","addr":"172.21.15.63:6837","nonce":3194497846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:47:00.211837+0000","modified":"2024-05-08T05:47:03.229466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6762":{"gid":6762,"name":"d","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6836/2942326473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2942326473},{"type":"v1","addr":"172.21.15.63:6836","nonce":2942326473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6762,"qdb_cluster":[6762]},"id":6}]} 2024-05-08T05:47:04.231 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 60 2024-05-08T05:47:04.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.232+0000 7f401ddef640 1 -- 172.21.15.45:0/79696089 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f40040762d0 msgr2=0x7f4004078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.232+0000 7f401ddef640 1 --2- 172.21.15.45:0/79696089 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f40040762d0 0x7f4004078790 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4008005880 tx=0x7f4008009040 comp rx=0 tx=0).stop 2024-05-08T05:47:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.232+0000 7f401ddef640 1 -- 172.21.15.45:0/79696089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4018072840 msgr2=0x7f4018155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.232+0000 7f401ddef640 1 --2- 172.21.15.45:0/79696089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4018072840 0x7f4018155750 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f400c00c4d0 tx=0x7f400c00c990 comp rx=0 tx=0).stop 2024-05-08T05:47:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.232+0000 7f401ddef640 1 -- 172.21.15.45:0/79696089 shutdown_connections 2024-05-08T05:47:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.232+0000 7f401ddef640 1 --2- 172.21.15.45:0/79696089 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f40040762d0 0x7f4004078790 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.232+0000 7f401ddef640 1 --2- 172.21.15.45:0/79696089 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f401815c340 0x7f4018157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.232+0000 7f401ddef640 1 --2- 172.21.15.45:0/79696089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4018072840 0x7f4018155750 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.232+0000 7f401ddef640 1 --2- 172.21.15.45:0/79696089 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f401806e0f0 0x7f4018157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.232+0000 7f401ddef640 1 -- 172.21.15.45:0/79696089 >> 172.21.15.45:0/79696089 conn(0x7f401806c3a0 msgr2=0x7f401815af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.232+0000 7f401ddef640 1 -- 172.21.15.45:0/79696089 shutdown_connections 2024-05-08T05:47:04.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.232+0000 7f401ddef640 1 -- 172.21.15.45:0/79696089 wait complete. 2024-05-08T05:47:04.243 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:47:04.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.347+0000 7fa16a21c640 1 Processor -- start 2024-05-08T05:47:04.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.347+0000 7fa16a21c640 1 -- start start 2024-05-08T05:47:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.348+0000 7fa16a21c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1640d3cd0 0x7fa1640cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.348+0000 7fa16a21c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1640d3330 con 0x7fa1641629a0 2024-05-08T05:47:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.348+0000 7fa16a21c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1640d2170 con 0x7fa1640c9ee0 2024-05-08T05:47:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.348+0000 7fa16a21c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa1640d2450 con 0x7fa1640d3cd0 2024-05-08T05:47:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.348+0000 7fa162ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1640d3cd0 0x7fa1640cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.348+0000 7fa1637fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa1641629a0 0x7fa1640d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36806/0 (socket says 172.21.15.45:36806) 2024-05-08T05:47:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.348+0000 7fa1637fe640 1 -- 172.21.15.45:0/2151591874 learned_addr learned my addr 172.21.15.45:0/2151591874 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:47:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.349+0000 7fa1627fc640 1 -- 172.21.15.45:0/2151591874 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1445649879 0 0) 0x7fa1640d3330 con 0x7fa1641629a0 2024-05-08T05:47:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.349+0000 7fa1627fc640 1 -- 172.21.15.45:0/2151591874 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa14c003520 con 0x7fa1641629a0 2024-05-08T05:47:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.349+0000 7fa162ffd640 1 -- 172.21.15.45:0/2151591874 >> v1:172.21.15.63:6789/0 conn(0x7fa1640c9ee0 legacy=0x7fa1640d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.349+0000 7fa162ffd640 1 -- 172.21.15.45:0/2151591874 >> v1:172.21.15.45:6789/0 conn(0x7fa1641629a0 legacy=0x7fa1640d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:04.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.349+0000 7fa162ffd640 1 -- 172.21.15.45:0/2151591874 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1640d2730 con 0x7fa1640d3cd0 2024-05-08T05:47:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.349+0000 7fa162ffd640 1 --2- 172.21.15.45:0/2151591874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1640d3cd0 0x7fa1640cfca0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa1580097a0 tx=0x7fa15802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7695cd14ef786eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.349+0000 7fa1627fc640 1 -- 172.21.15.45:0/2151591874 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158037070 con 0x7fa1640d3cd0 2024-05-08T05:47:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.350+0000 7fa1627fc640 1 -- 172.21.15.45:0/2151591874 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa158032440 con 0x7fa1640d3cd0 2024-05-08T05:47:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.350+0000 7fa1627fc640 1 -- 172.21.15.45:0/2151591874 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15802ae60 con 0x7fa1640d3cd0 2024-05-08T05:47:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.350+0000 7fa16a21c640 1 -- 172.21.15.45:0/2151591874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1640d3cd0 msgr2=0x7fa1640cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.350+0000 7fa16a21c640 1 --2- 172.21.15.45:0/2151591874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1640d3cd0 0x7fa1640cfca0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa1580097a0 tx=0x7fa15802a110 comp rx=0 tx=0).stop 2024-05-08T05:47:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.350+0000 7fa16a21c640 1 -- 172.21.15.45:0/2151591874 shutdown_connections 2024-05-08T05:47:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.350+0000 7fa16a21c640 1 --2- 172.21.15.45:0/2151591874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1640d3cd0 0x7fa1640cfca0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.350+0000 7fa16a21c640 1 -- 172.21.15.45:0/2151591874 >> 172.21.15.45:0/2151591874 conn(0x7fa1640d4fe0 msgr2=0x7fa1640d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.350+0000 7fa16a21c640 1 -- 172.21.15.45:0/2151591874 shutdown_connections 2024-05-08T05:47:04.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.350+0000 7fa16a21c640 1 -- 172.21.15.45:0/2151591874 wait complete. 2024-05-08T05:47:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.351+0000 7fa16a21c640 1 Processor -- start 2024-05-08T05:47:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.351+0000 7fa16a21c640 1 -- start start 2024-05-08T05:47:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.352+0000 7fa16a21c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1640c9ee0 0x7fa1641587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:04.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.352+0000 7fa16a21c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1640d3cd0 0x7fa16415c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.352+0000 7fa16a21c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1641629a0 0x7fa16415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.352+0000 7fa16a21c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa1640cf440 con 0x7fa1641629a0 2024-05-08T05:47:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.352+0000 7fa16a21c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1640cf2c0 con 0x7fa1640d3cd0 2024-05-08T05:47:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.352+0000 7fa16a21c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa1640d2dc0 con 0x7fa1640c9ee0 2024-05-08T05:47:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.352+0000 7fa163fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1641629a0 0x7fa16415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.352+0000 7fa163fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1641629a0 0x7fa16415cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43348/0 (socket says 172.21.15.45:43348) 2024-05-08T05:47:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.352+0000 7fa163fff640 1 -- 172.21.15.45:0/1400918272 learned_addr learned my addr 172.21.15.45:0/1400918272 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.352+0000 7fa162ffd640 1 --2- 172.21.15.45:0/1400918272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1640d3cd0 0x7fa16415c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.352+0000 7fa1637fe640 1 --2- 172.21.15.45:0/1400918272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1640c9ee0 0x7fa1641587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.353+0000 7fa163fff640 1 -- 172.21.15.45:0/1400918272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1640c9ee0 msgr2=0x7fa1641587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.353+0000 7fa163fff640 1 --2- 172.21.15.45:0/1400918272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1640c9ee0 0x7fa1641587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.353+0000 7fa163fff640 1 -- 172.21.15.45:0/1400918272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1640d3cd0 msgr2=0x7fa16415c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:04.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.353+0000 7fa163fff640 1 --2- 172.21.15.45:0/1400918272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1640d3cd0 0x7fa16415c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.353+0000 7fa163fff640 1 -- 172.21.15.45:0/1400918272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1640cd010 con 0x7fa1641629a0 2024-05-08T05:47:04.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.353+0000 7fa163fff640 1 --2- 172.21.15.45:0/1400918272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1641629a0 0x7fa16415cdc0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00c7a0 tx=0x7fa15c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:04.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.353+0000 7fa14bfff640 1 -- 172.21.15.45:0/1400918272 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c003b70 con 0x7fa1641629a0 2024-05-08T05:47:04.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.353+0000 7fa16a21c640 1 -- 172.21.15.45:0/1400918272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1640b7e50 con 0x7fa1641629a0 2024-05-08T05:47:04.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.353+0000 7fa14bfff640 1 -- 172.21.15.45:0/1400918272 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa15c003d10 con 0x7fa1641629a0 2024-05-08T05:47:04.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.353+0000 7fa14bfff640 1 -- 172.21.15.45:0/1400918272 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00aa70 con 0x7fa1641629a0 2024-05-08T05:47:04.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.353+0000 7fa16a21c640 1 -- 172.21.15.45:0/1400918272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1640b8410 con 0x7fa1641629a0 2024-05-08T05:47:04.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.354+0000 7fa16a21c640 1 -- 172.21.15.45:0/1400918272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1640693f0 con 0x7fa1641629a0 2024-05-08T05:47:04.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.355+0000 7fa14bfff640 1 -- 172.21.15.45:0/1400918272 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa15c00ac10 con 0x7fa1641629a0 2024-05-08T05:47:04.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.355+0000 7fa14bfff640 1 --2- 172.21.15.45:0/1400918272 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1500762d0 0x7fa150078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:04.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.355+0000 7fa14bfff640 1 -- 172.21.15.45:0/1400918272 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6101+0+0 (secure 0 0 0) 0x7fa15c0976c0 con 0x7fa1641629a0 2024-05-08T05:47:04.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.356+0000 7fa1637fe640 1 --2- 172.21.15.45:0/1400918272 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1500762d0 0x7fa150078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:04.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.356+0000 7fa1637fe640 1 --2- 172.21.15.45:0/1400918272 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1500762d0 0x7fa150078790 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa154005880 tx=0x7fa154007cd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:04.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.360+0000 7fa14bfff640 1 -- 172.21.15.45:0/1400918272 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa15c060730 con 0x7fa1641629a0 2024-05-08T05:47:04.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.553+0000 7fa16a21c640 1 -- 172.21.15.45:0/1400918272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa164160e20 con 0x7fa1641629a0 2024-05-08T05:47:04.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.554+0000 7fa14bfff640 1 -- 172.21.15.45:0/1400918272 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+16837 (secure 0 0 0) 0x7fa15c064a30 con 0x7fa1641629a0 2024-05-08T05:47:04.554 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:47:04.555 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":88,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:47:01.838134+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:46:57.892382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-08T05:46:59.168198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:47:04.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.556+0000 7fa16a21c640 1 -- 172.21.15.45:0/1400918272 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1500762d0 msgr2=0x7fa150078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.556+0000 7fa16a21c640 1 --2- 172.21.15.45:0/1400918272 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1500762d0 0x7fa150078790 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa154005880 tx=0x7fa154007cd0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.556+0000 7fa16a21c640 1 -- 172.21.15.45:0/1400918272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1641629a0 msgr2=0x7fa16415cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.556+0000 7fa16a21c640 1 --2- 172.21.15.45:0/1400918272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1641629a0 0x7fa16415cdc0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00c7a0 tx=0x7fa15c00cc60 comp rx=0 tx=0).stop 2024-05-08T05:47:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.556+0000 7fa16a21c640 1 -- 172.21.15.45:0/1400918272 shutdown_connections 2024-05-08T05:47:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.556+0000 7fa16a21c640 1 --2- 172.21.15.45:0/1400918272 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1500762d0 0x7fa150078790 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.556+0000 7fa16a21c640 1 --2- 172.21.15.45:0/1400918272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1641629a0 0x7fa16415cdc0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.556+0000 7fa16a21c640 1 --2- 172.21.15.45:0/1400918272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1640d3cd0 0x7fa16415c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.556+0000 7fa16a21c640 1 --2- 172.21.15.45:0/1400918272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1640c9ee0 0x7fa1641587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.556+0000 7fa16a21c640 1 -- 172.21.15.45:0/1400918272 >> 172.21.15.45:0/1400918272 conn(0x7fa1640d4fe0 msgr2=0x7fa1640cae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.556+0000 7fa16a21c640 1 -- 172.21.15.45:0/1400918272 shutdown_connections 2024-05-08T05:47:04.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.556+0000 7fa16a21c640 1 -- 172.21.15.45:0/1400918272 wait complete. 2024-05-08T05:47:04.568 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T05:47:04.568 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:47:04.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.671+0000 7fdf836c4640 1 Processor -- start 2024-05-08T05:47:04.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.671+0000 7fdf836c4640 1 -- start start 2024-05-08T05:47:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.672+0000 7fdf836c4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf7c15f380 0x7fdf7c0d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.672+0000 7fdf836c4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdf7c0d0cf0 0x7fdf7c0d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:04.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.672+0000 7fdf836c4640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf7c0d1760 con 0x7fdf7c0d3b00 2024-05-08T05:47:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.672+0000 7fdf836c4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fdf7c0d30f0 con 0x7fdf7c0d0cf0 2024-05-08T05:47:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.672+0000 7fdf836c4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdf7c0d1150 con 0x7fdf7c15f380 2024-05-08T05:47:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.673+0000 7fdf80c5f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fdf7c0d0cf0 0x7fdf7c0d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.673+0000 7fdf81460640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf7c15f380 0x7fdf7c0d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.673+0000 7fdf81c61640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fdf7c0d3b00 0x7fdf7c162e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:53282/0 (socket says 172.21.15.45:53282) 2024-05-08T05:47:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.673+0000 7fdf81c61640 1 -- 172.21.15.45:0/3160332780 learned_addr learned my addr 172.21.15.45:0/3160332780 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:47:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.673+0000 7fdf80c5f640 1 -- 172.21.15.45:0/3160332780 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf7c15f380 msgr2=0x7fdf7c0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.673+0000 7fdf80c5f640 1 --2- 172.21.15.45:0/3160332780 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf7c15f380 0x7fdf7c0d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.673+0000 7fdf80c5f640 1 -- 172.21.15.45:0/3160332780 >> v1:172.21.15.63:6789/0 conn(0x7fdf7c0d3b00 legacy=0x7fdf7c162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.673+0000 7fdf7bfff640 1 -- 172.21.15.45:0/3160332780 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3744372138 0 0) 0x7fdf7c0d1760 con 0x7fdf7c0d3b00 2024-05-08T05:47:04.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.673+0000 7fdf80c5f640 1 -- 172.21.15.45:0/3160332780 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7c0d1a40 con 0x7fdf7c0d0cf0 2024-05-08T05:47:04.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.674+0000 7fdf80c5f640 1 --2- 172.21.15.45:0/3160332780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf7c0d0cf0 0x7fdf7c0d35c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdf700097a0 tx=0x7fdf7002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5636115e96857e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:04.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.674+0000 7fdf7bfff640 1 -- 172.21.15.45:0/3160332780 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70037070 con 0x7fdf7c0d0cf0 2024-05-08T05:47:04.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.674+0000 7fdf7bfff640 1 -- 172.21.15.45:0/3160332780 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf70032440 con 0x7fdf7c0d0cf0 2024-05-08T05:47:04.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.674+0000 7fdf7bfff640 1 -- 172.21.15.45:0/3160332780 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70032810 con 0x7fdf7c0d0cf0 2024-05-08T05:47:04.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.674+0000 7fdf836c4640 1 -- 172.21.15.45:0/3160332780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf7c0d0cf0 msgr2=0x7fdf7c0d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:04.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.674+0000 7fdf836c4640 1 --2- 172.21.15.45:0/3160332780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf7c0d0cf0 0x7fdf7c0d35c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdf700097a0 tx=0x7fdf7002a180 comp rx=0 tx=0).stop 2024-05-08T05:47:04.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.675+0000 7fdf836c4640 1 -- 172.21.15.45:0/3160332780 shutdown_connections 2024-05-08T05:47:04.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.675+0000 7fdf836c4640 1 --2- 172.21.15.45:0/3160332780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf7c0d0cf0 0x7fdf7c0d35c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.675+0000 7fdf836c4640 1 --2- 172.21.15.45:0/3160332780 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf7c15f380 0x7fdf7c0d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.675+0000 7fdf836c4640 1 -- 172.21.15.45:0/3160332780 >> 172.21.15.45:0/3160332780 conn(0x7fdf7c0d4fe0 msgr2=0x7fdf7c0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:04.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.675+0000 7fdf836c4640 1 -- 172.21.15.45:0/3160332780 shutdown_connections 2024-05-08T05:47:04.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.675+0000 7fdf836c4640 1 -- 172.21.15.45:0/3160332780 wait complete. 2024-05-08T05:47:04.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.676+0000 7fdf836c4640 1 Processor -- start 2024-05-08T05:47:04.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.676+0000 7fdf836c4640 1 -- start start 2024-05-08T05:47:04.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf836c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf7c0d0cf0 0x7fdf7c0cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf836c4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf7c0d3b00 0x7fdf7c15cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf836c4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf7c15f380 0x7fdf7c15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf836c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7c0b8e40 con 0x7fdf7c0d0cf0 2024-05-08T05:47:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf836c4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7c0d2370 con 0x7fdf7c15f380 2024-05-08T05:47:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf836c4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fdf7c0d2000 con 0x7fdf7c0d3b00 2024-05-08T05:47:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf81460640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf7c0d0cf0 0x7fdf7c0cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf81460640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf7c0d0cf0 0x7fdf7c0cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43354/0 (socket says 172.21.15.45:43354) 2024-05-08T05:47:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf81460640 1 -- 172.21.15.45:0/2499169574 learned_addr learned my addr 172.21.15.45:0/2499169574 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf80c5f640 1 --2- 172.21.15.45:0/2499169574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf7c0d3b00 0x7fdf7c15cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf81c61640 1 --2- 172.21.15.45:0/2499169574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf7c15f380 0x7fdf7c15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf81460640 1 -- 172.21.15.45:0/2499169574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf7c0d3b00 msgr2=0x7fdf7c15cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf81460640 1 --2- 172.21.15.45:0/2499169574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf7c0d3b00 0x7fdf7c15cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf81460640 1 -- 172.21.15.45:0/2499169574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf7c15f380 msgr2=0x7fdf7c15d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:04.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf81460640 1 --2- 172.21.15.45:0/2499169574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf7c15f380 0x7fdf7c15d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf81460640 1 -- 172.21.15.45:0/2499169574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7c0a0900 con 0x7fdf7c0d0cf0 2024-05-08T05:47:04.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf81460640 1 --2- 172.21.15.45:0/2499169574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf7c0d0cf0 0x7fdf7c0cdad0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fdf7400e870 tx=0x7fdf7400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:04.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf79ffb640 1 -- 172.21.15.45:0/2499169574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74017070 con 0x7fdf7c0d0cf0 2024-05-08T05:47:04.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf79ffb640 1 -- 172.21.15.45:0/2499169574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf7400c940 con 0x7fdf7c0d0cf0 2024-05-08T05:47:04.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf79ffb640 1 -- 172.21.15.45:0/2499169574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7400cd30 con 0x7fdf7c0d0cf0 2024-05-08T05:47:04.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.677+0000 7fdf836c4640 1 -- 172.21.15.45:0/2499169574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf7c0a0ba0 con 0x7fdf7c0d0cf0 2024-05-08T05:47:04.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.678+0000 7fdf836c4640 1 -- 172.21.15.45:0/2499169574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf7c09eea0 con 0x7fdf7c0d0cf0 2024-05-08T05:47:04.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.679+0000 7fdf836c4640 1 -- 172.21.15.45:0/2499169574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf7c0693f0 con 0x7fdf7c0d0cf0 2024-05-08T05:47:04.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.679+0000 7fdf79ffb640 1 -- 172.21.15.45:0/2499169574 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fdf7401a070 con 0x7fdf7c0d0cf0 2024-05-08T05:47:04.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.680+0000 7fdf79ffb640 1 --2- 172.21.15.45:0/2499169574 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdf680762d0 0x7fdf68078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:04.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.680+0000 7fdf79ffb640 1 -- 172.21.15.45:0/2499169574 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6101+0+0 (secure 0 0 0) 0x7fdf74097540 con 0x7fdf7c0d0cf0 2024-05-08T05:47:04.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.680+0000 7fdf80c5f640 1 --2- 172.21.15.45:0/2499169574 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdf680762d0 0x7fdf68078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:04.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.681+0000 7fdf80c5f640 1 --2- 172.21.15.45:0/2499169574 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdf680762d0 0x7fdf68078790 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdf70034000 tx=0x7fdf7002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:04.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.684+0000 7fdf79ffb640 1 -- 172.21.15.45:0/2499169574 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdf74060750 con 0x7fdf7c0d0cf0 2024-05-08T05:47:04.734 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:47:04.736 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:47:04.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.871+0000 7fdf836c4640 1 -- 172.21.15.45:0/2499169574 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdf7c08cfd0 con 0x7fdf680762d0 2024-05-08T05:47:04.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.878+0000 7fdf79ffb640 1 -- 172.21.15.45:0/2499169574 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242555 (secure 0 0 0) 0x7fdf7c08cfd0 con 0x7fdf680762d0 2024-05-08T05:47:04.881 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:47:04.882 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:47:04.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.881+0000 7fdf836c4640 1 -- 172.21.15.45:0/2499169574 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdf680762d0 msgr2=0x7fdf68078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:04.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.881+0000 7fdf836c4640 1 --2- 172.21.15.45:0/2499169574 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdf680762d0 0x7fdf68078790 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdf70034000 tx=0x7fdf7002e000 comp rx=0 tx=0).stop 2024-05-08T05:47:04.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.881+0000 7fdf836c4640 1 -- 172.21.15.45:0/2499169574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf7c0d0cf0 msgr2=0x7fdf7c0cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:04.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.881+0000 7fdf836c4640 1 --2- 172.21.15.45:0/2499169574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf7c0d0cf0 0x7fdf7c0cdad0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fdf7400e870 tx=0x7fdf7400ed30 comp rx=0 tx=0).stop 2024-05-08T05:47:04.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.881+0000 7fdf836c4640 1 -- 172.21.15.45:0/2499169574 shutdown_connections 2024-05-08T05:47:04.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.881+0000 7fdf836c4640 1 --2- 172.21.15.45:0/2499169574 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fdf680762d0 0x7fdf68078790 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.881+0000 7fdf836c4640 1 --2- 172.21.15.45:0/2499169574 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fdf7c0d3b00 0x7fdf7c15cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.881+0000 7fdf836c4640 1 --2- 172.21.15.45:0/2499169574 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf7c15f380 0x7fdf7c15d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.881+0000 7fdf836c4640 1 --2- 172.21.15.45:0/2499169574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdf7c0d0cf0 0x7fdf7c0cdad0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:04.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.881+0000 7fdf836c4640 1 -- 172.21.15.45:0/2499169574 >> 172.21.15.45:0/2499169574 conn(0x7fdf7c0d4fe0 msgr2=0x7fdf7c15e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:04.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.881+0000 7fdf836c4640 1 -- 172.21.15.45:0/2499169574 shutdown_connections 2024-05-08T05:47:04.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:04.881+0000 7fdf836c4640 1 -- 172.21.15.45:0/2499169574 wait complete. 2024-05-08T05:47:04.892 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-05-08T05:47:02.912945+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":98,"acting":98,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250396,"kb_used_data":9260,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754724324,"statfs":{"total":773094113280,"available":772837707776,"internally_reserved":0,"allocated":9482240,"data_stored":4946491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.087193"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2024-05-08T05:46:59.838465+0000","last_change":"2024-05-08T05:46:59.838465+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.838465+0000","last_undegraded":"2024-05-08T05:46:59.838465+0000","last_fullsized":"2024-05-08T05:46:59.838465+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2024-05-08T05:46:59.840927+0000","last_change":"2024-05-08T05:46:59.840927+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.840927+0000","last_undegraded":"2024-05-08T05:46:59.840927+0000","last_fullsized":"2024-05-08T05:46:59.840927+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2024-05-08T05:46:59.835641+0000","last_change":"2024-05-08T05:46:59.835641+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.835641+0000","last_undegraded":"2024-05-08T05:46:59.835641+0000","last_fullsized":"2024-05-08T05:46:59.835641+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2024-05-08T05:46:59.839004+0000","last_change":"2024-05-08T05:46:59.839004+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.839004+0000","last_undegraded":"2024-05-08T05:46:59.839004+0000","last_fullsized":"2024-05-08T05:46:59.839004+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2024-05-08T05:46:59.840420+0000","last_change":"2024-05-08T05:46:59.840420+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.840420+0000","last_undegraded":"2024-05-08T05:46:59.840420+0000","last_fullsized":"2024-05-08T05:46:59.840420+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2024-05-08T05:46:59.838478+0000","last_change":"2024-05-08T05:46:59.838478+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.838478+0000","last_undegraded":"2024-05-08T05:46:59.838478+0000","last_fullsized":"2024-05-08T05:46:59.838478+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T05:46:59.848437+0000","last_change":"2024-05-08T05:46:59.846402+0000","last_active":"2024-05-08T05:46:59.848437+0000","last_peered":"2024-05-08T05:46:59.848437+0000","last_clean":"2024-05-08T05:46:59.848437+0000","last_became_active":"2024-05-08T05:46:59.845575+0000","last_became_peered":"2024-05-08T05:46:59.845575+0000","last_unstale":"2024-05-08T05:46:59.848437+0000","last_undegraded":"2024-05-08T05:46:59.848437+0000","last_fullsized":"2024-05-08T05:46:59.848437+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:08:58.752524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-08T05:46:58.828559+0000","last_change":"2024-05-08T05:46:58.828559+0000","last_active":"2024-05-08T05:46:58.820463+0000","last_peered":"2024-05-08T05:46:58.820463+0000","last_clean":"2024-05-08T05:46:58.820463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:58.828559+0000","last_undegraded":"2024-05-08T05:46:58.828559+0000","last_fullsized":"2024-05-08T05:46:58.828559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2024-05-08T05:46:59.840397+0000","last_change":"2024-05-08T05:46:59.840397+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.840397+0000","last_undegraded":"2024-05-08T05:46:59.840397+0000","last_fullsized":"2024-05-08T05:46:59.840397+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T05:46:59.852877+0000","last_change":"2024-05-08T05:46:59.851832+0000","last_active":"2024-05-08T05:46:59.852877+0000","last_peered":"2024-05-08T05:46:59.852877+0000","last_clean":"2024-05-08T05:46:59.852877+0000","last_became_active":"2024-05-08T05:46:59.851566+0000","last_became_peered":"2024-05-08T05:46:59.851566+0000","last_unstale":"2024-05-08T05:46:59.852877+0000","last_undegraded":"2024-05-08T05:46:59.852877+0000","last_fullsized":"2024-05-08T05:46:59.852877+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:07.311511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T05:46:59.853750+0000","last_change":"2024-05-08T05:46:59.852748+0000","last_active":"2024-05-08T05:46:59.853750+0000","last_peered":"2024-05-08T05:46:59.853750+0000","last_clean":"2024-05-08T05:46:59.853750+0000","last_became_active":"2024-05-08T05:46:59.852436+0000","last_became_peered":"2024-05-08T05:46:59.852436+0000","last_unstale":"2024-05-08T05:46:59.853750+0000","last_undegraded":"2024-05-08T05:46:59.853750+0000","last_fullsized":"2024-05-08T05:46:59.853750+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:30.989473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-08T05:46:58.827933+0000","last_change":"2024-05-08T05:46:58.827933+0000","last_active":"2024-05-08T05:46:58.820463+0000","last_peered":"2024-05-08T05:46:58.820463+0000","last_clean":"2024-05-08T05:46:58.820463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:58.827933+0000","last_undegraded":"2024-05-08T05:46:58.827933+0000","last_fullsized":"2024-05-08T05:46:58.827933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2024-05-08T05:46:59.835075+0000","last_change":"2024-05-08T05:46:59.835075+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.835075+0000","last_undegraded":"2024-05-08T05:46:59.835075+0000","last_fullsized":"2024-05-08T05:46:59.835075+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-08T05:46:58.826209+0000","last_change":"2024-05-08T05:46:58.826209+0000","last_active":"2024-05-08T05:46:58.820463+0000","last_peered":"2024-05-08T05:46:58.820463+0000","last_clean":"2024-05-08T05:46:58.820463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:58.826209+0000","last_undegraded":"2024-05-08T05:46:58.826209+0000","last_fullsized":"2024-05-08T05:46:58.826209+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-08T05:46:58.832035+0000","last_change":"2024-05-08T05:46:58.832035+0000","last_active":"2024-05-08T05:46:58.820463+0000","last_peered":"2024-05-08T05:46:58.820463+0000","last_clean":"2024-05-08T05:46:58.820463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:58.832035+0000","last_undegraded":"2024-05-08T05:46:58.832035+0000","last_fullsized":"2024-05-08T05:46:58.832035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T05:46:59.852180+0000","last_change":"2024-05-08T05:46:59.850380+0000","last_active":"2024-05-08T05:46:59.852180+0000","last_peered":"2024-05-08T05:46:59.852180+0000","last_clean":"2024-05-08T05:46:59.852180+0000","last_became_active":"2024-05-08T05:46:59.850075+0000","last_became_peered":"2024-05-08T05:46:59.850075+0000","last_unstale":"2024-05-08T05:46:59.852180+0000","last_undegraded":"2024-05-08T05:46:59.852180+0000","last_fullsized":"2024-05-08T05:46:59.852180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:34:03.730878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T05:46:59.845638+0000","last_change":"2024-05-08T05:46:59.843987+0000","last_active":"2024-05-08T05:46:59.845638+0000","last_peered":"2024-05-08T05:46:59.845638+0000","last_clean":"2024-05-08T05:46:59.845638+0000","last_became_active":"2024-05-08T05:46:59.843622+0000","last_became_peered":"2024-05-08T05:46:59.843622+0000","last_unstale":"2024-05-08T05:46:59.845638+0000","last_undegraded":"2024-05-08T05:46:59.845638+0000","last_fullsized":"2024-05-08T05:46:59.845638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:51:38.811587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2024-05-08T05:46:59.841409+0000","last_change":"2024-05-08T05:46:59.841409+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.841409+0000","last_undegraded":"2024-05-08T05:46:59.841409+0000","last_fullsized":"2024-05-08T05:46:59.841409+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-08T05:46:58.832093+0000","last_change":"2024-05-08T05:46:58.832093+0000","last_active":"2024-05-08T05:46:58.820463+0000","last_peered":"2024-05-08T05:46:58.820463+0000","last_clean":"2024-05-08T05:46:58.820463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:58.832093+0000","last_undegraded":"2024-05-08T05:46:58.832093+0000","last_fullsized":"2024-05-08T05:46:58.832093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-08T05:46:58.828946+0000","last_change":"2024-05-08T05:46:58.828946+0000","last_active":"2024-05-08T05:46:58.820463+0000","last_peered":"2024-05-08T05:46:58.820463+0000","last_clean":"2024-05-08T05:46:58.820463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:58.828946+0000","last_undegraded":"2024-05-08T05:46:58.828946+0000","last_fullsized":"2024-05-08T05:46:58.828946+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-08T05:46:58.830619+0000","last_change":"2024-05-08T05:46:58.830619+0000","last_active":"2024-05-08T05:46:58.820463+0000","last_peered":"2024-05-08T05:46:58.820463+0000","last_clean":"2024-05-08T05:46:58.820463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:58.830619+0000","last_undegraded":"2024-05-08T05:46:58.830619+0000","last_fullsized":"2024-05-08T05:46:58.830619+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T05:46:59.851537+0000","last_change":"2024-05-08T05:46:59.850252+0000","last_active":"2024-05-08T05:46:59.851537+0000","last_peered":"2024-05-08T05:46:59.851537+0000","last_clean":"2024-05-08T05:46:59.851537+0000","last_became_active":"2024-05-08T05:46:59.849947+0000","last_became_peered":"2024-05-08T05:46:59.849947+0000","last_unstale":"2024-05-08T05:46:59.851537+0000","last_undegraded":"2024-05-08T05:46:59.851537+0000","last_fullsized":"2024-05-08T05:46:59.851537+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:20:21.059771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-08T05:46:58.830459+0000","last_change":"2024-05-08T05:46:58.830459+0000","last_active":"2024-05-08T05:46:58.820463+0000","last_peered":"2024-05-08T05:46:58.820463+0000","last_clean":"2024-05-08T05:46:58.820463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:58.830459+0000","last_undegraded":"2024-05-08T05:46:58.830459+0000","last_fullsized":"2024-05-08T05:46:58.830459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-08T05:46:58.826080+0000","last_change":"2024-05-08T05:46:58.826080+0000","last_active":"2024-05-08T05:46:58.820463+0000","last_peered":"2024-05-08T05:46:58.820463+0000","last_clean":"2024-05-08T05:46:58.820463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:58.826080+0000","last_undegraded":"2024-05-08T05:46:58.826080+0000","last_fullsized":"2024-05-08T05:46:58.826080+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-08T05:46:58.826067+0000","last_change":"2024-05-08T05:46:58.826067+0000","last_active":"2024-05-08T05:46:58.820463+0000","last_peered":"2024-05-08T05:46:58.820463+0000","last_clean":"2024-05-08T05:46:58.820463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:58.826067+0000","last_undegraded":"2024-05-08T05:46:58.826067+0000","last_fullsized":"2024-05-08T05:46:58.826067+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T05:46:59.850602+0000","last_change":"2024-05-08T05:46:59.848248+0000","last_active":"2024-05-08T05:46:59.850602+0000","last_peered":"2024-05-08T05:46:59.850602+0000","last_clean":"2024-05-08T05:46:59.850602+0000","last_became_active":"2024-05-08T05:46:59.847924+0000","last_became_peered":"2024-05-08T05:46:59.847924+0000","last_unstale":"2024-05-08T05:46:59.850602+0000","last_undegraded":"2024-05-08T05:46:59.850602+0000","last_fullsized":"2024-05-08T05:46:59.850602+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:15:43.992195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2024-05-08T05:46:59.834976+0000","last_change":"2024-05-08T05:46:59.834976+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.834976+0000","last_undegraded":"2024-05-08T05:46:59.834976+0000","last_fullsized":"2024-05-08T05:46:59.834976+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T05:46:59.851508+0000","last_change":"2024-05-08T05:46:59.847461+0000","last_active":"2024-05-08T05:46:59.851508+0000","last_peered":"2024-05-08T05:46:59.851508+0000","last_clean":"2024-05-08T05:46:59.851508+0000","last_became_active":"2024-05-08T05:46:59.847126+0000","last_became_peered":"2024-05-08T05:46:59.847126+0000","last_unstale":"2024-05-08T05:46:59.851508+0000","last_undegraded":"2024-05-08T05:46:59.851508+0000","last_fullsized":"2024-05-08T05:46:59.851508+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:41:38.444356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-08T05:46:58.828478+0000","last_change":"2024-05-08T05:46:58.828478+0000","last_active":"2024-05-08T05:46:58.820463+0000","last_peered":"2024-05-08T05:46:58.820463+0000","last_clean":"2024-05-08T05:46:58.820463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:58.828478+0000","last_undegraded":"2024-05-08T05:46:58.828478+0000","last_fullsized":"2024-05-08T05:46:58.828478+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T05:46:59.849809+0000","last_change":"2024-05-08T05:46:59.848971+0000","last_active":"2024-05-08T05:46:59.849809+0000","last_peered":"2024-05-08T05:46:59.849809+0000","last_clean":"2024-05-08T05:46:59.849809+0000","last_became_active":"2024-05-08T05:46:59.848666+0000","last_became_peered":"2024-05-08T05:46:59.848666+0000","last_unstale":"2024-05-08T05:46:59.849809+0000","last_undegraded":"2024-05-08T05:46:59.849809+0000","last_fullsized":"2024-05-08T05:46:59.849809+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:43.465819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2024-05-08T05:46:59.837876+0000","last_change":"2024-05-08T05:46:59.837876+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.837876+0000","last_undegraded":"2024-05-08T05:46:59.837876+0000","last_fullsized":"2024-05-08T05:46:59.837876+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-08T05:46:58.828132+0000","last_change":"2024-05-08T05:46:58.828132+0000","last_active":"2024-05-08T05:46:58.820463+0000","last_peered":"2024-05-08T05:46:58.820463+0000","last_clean":"2024-05-08T05:46:58.820463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:58.828132+0000","last_undegraded":"2024-05-08T05:46:58.828132+0000","last_fullsized":"2024-05-08T05:46:58.828132+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2024-05-08T05:46:59.838640+0000","last_change":"2024-05-08T05:46:59.838640+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.838640+0000","last_undegraded":"2024-05-08T05:46:59.838640+0000","last_fullsized":"2024-05-08T05:46:59.838640+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T05:46:59.845946+0000","last_change":"2024-05-08T05:46:59.844082+0000","last_active":"2024-05-08T05:46:59.845946+0000","last_peered":"2024-05-08T05:46:59.845946+0000","last_clean":"2024-05-08T05:46:59.845946+0000","last_became_active":"2024-05-08T05:46:59.843734+0000","last_became_peered":"2024-05-08T05:46:59.843734+0000","last_unstale":"2024-05-08T05:46:59.845946+0000","last_undegraded":"2024-05-08T05:46:59.845946+0000","last_fullsized":"2024-05-08T05:46:59.845946+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:56:17.117125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-08T05:46:58.828132+0000","last_change":"2024-05-08T05:46:58.828132+0000","last_active":"2024-05-08T05:46:58.820463+0000","last_peered":"2024-05-08T05:46:58.820463+0000","last_clean":"2024-05-08T05:46:58.820463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:58.828132+0000","last_undegraded":"2024-05-08T05:46:58.828132+0000","last_fullsized":"2024-05-08T05:46:58.828132+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2024-05-08T05:46:59.836417+0000","last_change":"2024-05-08T05:46:59.836417+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.836417+0000","last_undegraded":"2024-05-08T05:46:59.836417+0000","last_fullsized":"2024-05-08T05:46:59.836417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T05:46:59.852884+0000","last_change":"2024-05-08T05:46:59.851269+0000","last_active":"2024-05-08T05:46:59.852884+0000","last_peered":"2024-05-08T05:46:59.852884+0000","last_clean":"2024-05-08T05:46:59.852884+0000","last_became_active":"2024-05-08T05:46:59.850897+0000","last_became_peered":"2024-05-08T05:46:59.850897+0000","last_unstale":"2024-05-08T05:46:59.852884+0000","last_undegraded":"2024-05-08T05:46:59.852884+0000","last_fullsized":"2024-05-08T05:46:59.852884+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:20:50.410110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-08T05:46:58.827847+0000","last_change":"2024-05-08T05:46:58.827847+0000","last_active":"2024-05-08T05:46:58.820463+0000","last_peered":"2024-05-08T05:46:58.820463+0000","last_clean":"2024-05-08T05:46:58.820463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:58.827847+0000","last_undegraded":"2024-05-08T05:46:58.827847+0000","last_fullsized":"2024-05-08T05:46:58.827847+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-08T05:46:58.827843+0000","last_change":"2024-05-08T05:46:58.827843+0000","last_active":"2024-05-08T05:46:58.820463+0000","last_peered":"2024-05-08T05:46:58.820463+0000","last_clean":"2024-05-08T05:46:58.820463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:58.827843+0000","last_undegraded":"2024-05-08T05:46:58.827843+0000","last_fullsized":"2024-05-08T05:46:58.827843+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2024-05-08T05:46:59.838313+0000","last_change":"2024-05-08T05:46:59.838313+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.838313+0000","last_undegraded":"2024-05-08T05:46:59.838313+0000","last_fullsized":"2024-05-08T05:46:59.838313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T05:46:59.848086+0000","last_change":"2024-05-08T05:46:59.840480+0000","last_active":"2024-05-08T05:46:59.848086+0000","last_peered":"2024-05-08T05:46:59.848086+0000","last_clean":"2024-05-08T05:46:59.848086+0000","last_became_active":"2024-05-08T05:46:59.840201+0000","last_became_peered":"2024-05-08T05:46:59.840201+0000","last_unstale":"2024-05-08T05:46:59.848086+0000","last_undegraded":"2024-05-08T05:46:59.848086+0000","last_fullsized":"2024-05-08T05:46:59.848086+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:11:33.715098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-08T05:46:58.830476+0000","last_change":"2024-05-08T05:46:58.830476+0000","last_active":"2024-05-08T05:46:58.820463+0000","last_peered":"2024-05-08T05:46:58.820463+0000","last_clean":"2024-05-08T05:46:58.820463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:58.830476+0000","last_undegraded":"2024-05-08T05:46:58.830476+0000","last_fullsized":"2024-05-08T05:46:58.830476+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T05:46:59.848362+0000","last_change":"2024-05-08T05:46:59.846154+0000","last_active":"2024-05-08T05:46:59.848362+0000","last_peered":"2024-05-08T05:46:59.848362+0000","last_clean":"2024-05-08T05:46:59.848362+0000","last_became_active":"2024-05-08T05:46:59.845638+0000","last_became_peered":"2024-05-08T05:46:59.845638+0000","last_unstale":"2024-05-08T05:46:59.848362+0000","last_undegraded":"2024-05-08T05:46:59.848362+0000","last_fullsized":"2024-05-08T05:46:59.848362+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:36:42.649221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2024-05-08T05:46:59.835348+0000","last_change":"2024-05-08T05:46:59.835348+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.835348+0000","last_undegraded":"2024-05-08T05:46:59.835348+0000","last_fullsized":"2024-05-08T05:46:59.835348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T05:46:59.843147+0000","last_change":"2024-05-08T05:46:59.839824+0000","last_active":"2024-05-08T05:46:59.843147+0000","last_peered":"2024-05-08T05:46:59.843147+0000","last_clean":"2024-05-08T05:46:59.843147+0000","last_became_active":"2024-05-08T05:46:59.839565+0000","last_became_peered":"2024-05-08T05:46:59.839565+0000","last_unstale":"2024-05-08T05:46:59.843147+0000","last_undegraded":"2024-05-08T05:46:59.843147+0000","last_fullsized":"2024-05-08T05:46:59.843147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:35:34.561464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:46:59.822606+0000","last_change":"2024-05-08T05:46:59.822606+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.822606+0000","last_undegraded":"2024-05-08T05:46:59.822606+0000","last_fullsized":"2024-05-08T05:46:59.822606+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T05:46:59.853881+0000","last_change":"2024-05-08T05:46:59.853235+0000","last_active":"2024-05-08T05:46:59.853881+0000","last_peered":"2024-05-08T05:46:59.853881+0000","last_clean":"2024-05-08T05:46:59.853881+0000","last_became_active":"2024-05-08T05:46:59.852924+0000","last_became_peered":"2024-05-08T05:46:59.852924+0000","last_unstale":"2024-05-08T05:46:59.853881+0000","last_undegraded":"2024-05-08T05:46:59.853881+0000","last_fullsized":"2024-05-08T05:46:59.853881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:56.785804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2024-05-08T05:46:59.841810+0000","last_change":"2024-05-08T05:46:59.841810+0000","last_active":"2024-05-08T05:46:59.822606+0000","last_peered":"2024-05-08T05:46:59.822606+0000","last_clean":"2024-05-08T05:46:59.822606+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:59.841810+0000","last_undegraded":"2024-05-08T05:46:59.841810+0000","last_fullsized":"2024-05-08T05:46:59.841810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-05-08T05:46:58.826082+0000","last_change":"2024-05-08T05:46:58.826082+0000","last_active":"2024-05-08T05:46:58.820463+0000","last_peered":"2024-05-08T05:46:58.820463+0000","last_clean":"2024-05-08T05:46:58.820463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:46:58.826082+0000","last_undegraded":"2024-05-08T05:46:58.826082+0000","last_fullsized":"2024-05-08T05:46:58.826082+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":230,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T05:46:59.843912+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:46:59.843912+0000","last_peered":"2024-05-08T05:46:59.843912+0000","last_clean":"2024-05-08T05:46:59.843912+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:46:59.843912+0000","last_undegraded":"2024-05-08T05:46:59.843912+0000","last_fullsized":"2024-05-08T05:46:59.843912+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542216,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1638400,"data_stored":1064968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542217,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1044480,"data_stored":474600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542216,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1044480,"data_stored":474600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1032192,"data_stored":468391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1032192,"data_stored":468391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1032192,"data_stored":468391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1032192,"data_stored":468391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574918,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32776,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339064,"statfs":{"total":96636764160,"available":96603201536,"internally_reserved":0,"allocated":1626112,"data_stored":1058759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:47:04.897 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:47:05.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.037+0000 7fef4a4a7640 1 Processor -- start 2024-05-08T05:47:05.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.037+0000 7fef4a4a7640 1 -- start start 2024-05-08T05:47:05.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.037+0000 7fef4a4a7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fef4415f420 0x7fef440d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef4a4a7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef440d3ca0 0x7fef440d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef4a4a7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef441630b0 con 0x7fef440d1eb0 2024-05-08T05:47:05.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef4a4a7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fef440d22f0 con 0x7fef4415f420 2024-05-08T05:47:05.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef4a4a7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fef440d2550 con 0x7fef440d3ca0 2024-05-08T05:47:05.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef48a44640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fef440d1eb0 0x7fef440d4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36818/0 (socket says 172.21.15.45:36818) 2024-05-08T05:47:05.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef48a44640 1 -- 172.21.15.45:0/3093740474 learned_addr learned my addr 172.21.15.45:0/3093740474 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:47:05.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef437fe640 1 -- 172.21.15.45:0/3093740474 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2257882191 0 0) 0x7fef3c001490 con 0x7fef440d1eb0 2024-05-08T05:47:05.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef43fff640 1 --2- 172.21.15.45:0/3093740474 >> v2:172.21.15.63:3300/0 conn(0x7fef4415f420 0x7fef440d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:05.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef3bfff640 1 --2- 172.21.15.45:0/3093740474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef440d3ca0 0x7fef440d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:05.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef437fe640 1 -- 172.21.15.45:0/3093740474 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef30003520 con 0x7fef440d1eb0 2024-05-08T05:47:05.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef437fe640 1 -- 172.21.15.45:0/3093740474 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 351230277 0 0) 0x7fef30003520 con 0x7fef440d1eb0 2024-05-08T05:47:05.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef437fe640 1 -- 172.21.15.45:0/3093740474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef440d3ca0 msgr2=0x7fef440d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:05.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef437fe640 1 --2- 172.21.15.45:0/3093740474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef440d3ca0 0x7fef440d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef437fe640 1 -- 172.21.15.45:0/3093740474 >> v2:172.21.15.63:3300/0 conn(0x7fef4415f420 msgr2=0x7fef440d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:05.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef437fe640 1 --2- 172.21.15.45:0/3093740474 >> v2:172.21.15.63:3300/0 conn(0x7fef4415f420 0x7fef440d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef437fe640 1 -- 172.21.15.45:0/3093740474 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef440d2c80 con 0x7fef440d1eb0 2024-05-08T05:47:05.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef437fe640 1 -- 172.21.15.45:0/3093740474 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fef3c001c40 con 0x7fef440d1eb0 2024-05-08T05:47:05.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef437fe640 1 -- 172.21.15.45:0/3093740474 >> v1:172.21.15.45:6789/0 conn(0x7fef440d1eb0 legacy=0x7fef440d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:05.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef437fe640 1 --2- 172.21.15.45:0/3093740474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef30003a00 0x7fef30003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef437fe640 1 --2- 172.21.15.45:0/3093740474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef300043b0 0x7fef30004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef437fe640 1 --2- 172.21.15.45:0/3093740474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef30004ef0 0x7fef3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef437fe640 1 -- 172.21.15.45:0/3093740474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fef440d22f0 con 0x7fef300043b0 2024-05-08T05:47:05.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef437fe640 1 -- 172.21.15.45:0/3093740474 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef440d2550 con 0x7fef30004ef0 2024-05-08T05:47:05.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef437fe640 1 -- 172.21.15.45:0/3093740474 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fef3000bf00 con 0x7fef30003a00 2024-05-08T05:47:05.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.038+0000 7fef3bfff640 1 --2- 172.21.15.45:0/3093740474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef300043b0 0x7fef30004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:05.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.039+0000 7fef43fff640 1 --2- 172.21.15.45:0/3093740474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef30003a00 0x7fef30003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:05.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.039+0000 7fef48a44640 1 --2- 172.21.15.45:0/3093740474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef30004ef0 0x7fef3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:05.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.039+0000 7fef3bfff640 1 -- 172.21.15.45:0/3093740474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef30003a00 msgr2=0x7fef30003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:05.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.039+0000 7fef3bfff640 1 --2- 172.21.15.45:0/3093740474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef30003a00 0x7fef30003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.039+0000 7fef3bfff640 1 -- 172.21.15.45:0/3093740474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef30004ef0 msgr2=0x7fef3000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:05.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.039+0000 7fef3bfff640 1 --2- 172.21.15.45:0/3093740474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef30004ef0 0x7fef3000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.039+0000 7fef3bfff640 1 -- 172.21.15.45:0/3093740474 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef30003520 con 0x7fef300043b0 2024-05-08T05:47:05.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.039+0000 7fef3bfff640 1 --2- 172.21.15.45:0/3093740474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef300043b0 0x7fef30004880 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009ac0 tx=0x7fef30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4caa0e7f2fe787 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:05.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.040+0000 7fef437fe640 1 -- 172.21.15.45:0/3093740474 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c036070 con 0x7fef300043b0 2024-05-08T05:47:05.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.040+0000 7fef437fe640 1 -- 172.21.15.45:0/3093740474 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef2c031400 con 0x7fef300043b0 2024-05-08T05:47:05.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.040+0000 7fef437fe640 1 -- 172.21.15.45:0/3093740474 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c009690 con 0x7fef300043b0 2024-05-08T05:47:05.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.040+0000 7fef4a4a7640 1 -- 172.21.15.45:0/3093740474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef300043b0 msgr2=0x7fef30004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:05.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.040+0000 7fef4a4a7640 1 --2- 172.21.15.45:0/3093740474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef300043b0 0x7fef30004880 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009ac0 tx=0x7fef30005830 comp rx=0 tx=0).stop 2024-05-08T05:47:05.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.040+0000 7fef48a44640 1 -- 172.21.15.45:0/3093740474 reap_dead start 2024-05-08T05:47:05.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.040+0000 7fef4a4a7640 1 -- 172.21.15.45:0/3093740474 shutdown_connections 2024-05-08T05:47:05.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.040+0000 7fef4a4a7640 1 -- 172.21.15.45:0/3093740474 >> 172.21.15.45:0/3093740474 conn(0x7fef4408eac0 msgr2=0x7fef4409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:05.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.040+0000 7fef4a4a7640 1 -- 172.21.15.45:0/3093740474 shutdown_connections 2024-05-08T05:47:05.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.040+0000 7fef4a4a7640 1 -- 172.21.15.45:0/3093740474 wait complete. 2024-05-08T05:47:05.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.041+0000 7fef4a4a7640 1 Processor -- start 2024-05-08T05:47:05.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.042+0000 7fef4a4a7640 1 -- start start 2024-05-08T05:47:05.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.042+0000 7fef4a4a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef300043b0 0x7fef440cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.042+0000 7fef4a4a7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef440ce0c0 0x7fef44152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.042+0000 7fef4a4a7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef4406cf80 0x7fef4406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.042+0000 7fef4a4a7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fef440d2830 con 0x7fef300043b0 2024-05-08T05:47:05.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.042+0000 7fef4a4a7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef440d3240 con 0x7fef440ce0c0 2024-05-08T05:47:05.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.042+0000 7fef4a4a7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fef440d53d0 con 0x7fef4406cf80 2024-05-08T05:47:05.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.042+0000 7fef43fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef300043b0 0x7fef440cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:05.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.042+0000 7fef43fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef300043b0 0x7fef440cb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43360/0 (socket says 172.21.15.45:43360) 2024-05-08T05:47:05.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.042+0000 7fef43fff640 1 -- 172.21.15.45:0/3387270950 learned_addr learned my addr 172.21.15.45:0/3387270950 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:05.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.043+0000 7fef43fff640 1 -- 172.21.15.45:0/3387270950 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef4406cf80 msgr2=0x7fef4406d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:05.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.043+0000 7fef43fff640 1 --2- 172.21.15.45:0/3387270950 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef4406cf80 0x7fef4406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.043+0000 7fef3bfff640 1 --2- 172.21.15.45:0/3387270950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef440ce0c0 0x7fef44152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:05.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.043+0000 7fef43fff640 1 -- 172.21.15.45:0/3387270950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef440ce0c0 msgr2=0x7fef44152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:05.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.043+0000 7fef43fff640 1 --2- 172.21.15.45:0/3387270950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef440ce0c0 0x7fef44152480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.043+0000 7fef43fff640 1 -- 172.21.15.45:0/3387270950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef440ce630 con 0x7fef300043b0 2024-05-08T05:47:05.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.043+0000 7fef43fff640 1 --2- 172.21.15.45:0/3387270950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef300043b0 0x7fef440cb390 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fef34011940 tx=0x7fef34011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:05.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.043+0000 7fef417fa640 1 -- 172.21.15.45:0/3387270950 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3400faf0 con 0x7fef300043b0 2024-05-08T05:47:05.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.043+0000 7fef417fa640 1 -- 172.21.15.45:0/3387270950 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef3400fc90 con 0x7fef300043b0 2024-05-08T05:47:05.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.043+0000 7fef4a4a7640 1 -- 172.21.15.45:0/3387270950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef440ce910 con 0x7fef300043b0 2024-05-08T05:47:05.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.043+0000 7fef417fa640 1 -- 172.21.15.45:0/3387270950 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef34014440 con 0x7fef300043b0 2024-05-08T05:47:05.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.043+0000 7fef4a4a7640 1 -- 172.21.15.45:0/3387270950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef440ceed0 con 0x7fef300043b0 2024-05-08T05:47:05.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.044+0000 7fef4a4a7640 1 -- 172.21.15.45:0/3387270950 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef4406d940 con 0x7fef300043b0 2024-05-08T05:47:05.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.045+0000 7fef417fa640 1 -- 172.21.15.45:0/3387270950 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fef34012040 con 0x7fef300043b0 2024-05-08T05:47:05.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.045+0000 7fef417fa640 1 --2- 172.21.15.45:0/3387270950 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef240762d0 0x7fef24078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.045+0000 7fef417fa640 1 -- 172.21.15.45:0/3387270950 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6101+0+0 (secure 0 0 0) 0x7fef34097e30 con 0x7fef300043b0 2024-05-08T05:47:05.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.045+0000 7fef3bfff640 1 --2- 172.21.15.45:0/3387270950 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef240762d0 0x7fef24078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:05.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.046+0000 7fef3bfff640 1 --2- 172.21.15.45:0/3387270950 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef240762d0 0x7fef24078790 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fef2c033000 tx=0x7fef2c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:05.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.050+0000 7fef417fa640 1 -- 172.21.15.45:0/3387270950 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fef34019030 con 0x7fef300043b0 2024-05-08T05:47:05.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.238+0000 7fef4a4a7640 1 -- 172.21.15.45:0/3387270950 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef440b4c50 con 0x7fef240762d0 2024-05-08T05:47:05.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.245+0000 7fef417fa640 1 -- 172.21.15.45:0/3387270950 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252063 (secure 0 0 0) 0x7fef440b4c50 con 0x7fef240762d0 2024-05-08T05:47:05.247 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:47:05.249 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:47:05.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.248+0000 7fef4a4a7640 1 -- 172.21.15.45:0/3387270950 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef240762d0 msgr2=0x7fef24078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:05.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.248+0000 7fef4a4a7640 1 --2- 172.21.15.45:0/3387270950 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef240762d0 0x7fef24078790 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fef2c033000 tx=0x7fef2c008030 comp rx=0 tx=0).stop 2024-05-08T05:47:05.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.248+0000 7fef4a4a7640 1 -- 172.21.15.45:0/3387270950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef300043b0 msgr2=0x7fef440cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:05.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.248+0000 7fef4a4a7640 1 --2- 172.21.15.45:0/3387270950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef300043b0 0x7fef440cb390 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fef34011940 tx=0x7fef34011e00 comp rx=0 tx=0).stop 2024-05-08T05:47:05.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.248+0000 7fef4a4a7640 1 -- 172.21.15.45:0/3387270950 shutdown_connections 2024-05-08T05:47:05.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.248+0000 7fef4a4a7640 1 --2- 172.21.15.45:0/3387270950 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef240762d0 0x7fef24078790 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.248+0000 7fef4a4a7640 1 --2- 172.21.15.45:0/3387270950 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef4406cf80 0x7fef4406d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.248+0000 7fef4a4a7640 1 --2- 172.21.15.45:0/3387270950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef440ce0c0 0x7fef44152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.248+0000 7fef4a4a7640 1 --2- 172.21.15.45:0/3387270950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef300043b0 0x7fef440cb390 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.248+0000 7fef4a4a7640 1 -- 172.21.15.45:0/3387270950 >> 172.21.15.45:0/3387270950 conn(0x7fef4408eac0 msgr2=0x7fef4415dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.248+0000 7fef4a4a7640 1 -- 172.21.15.45:0/3387270950 shutdown_connections 2024-05-08T05:47:05.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.248+0000 7fef4a4a7640 1 -- 172.21.15.45:0/3387270950 wait complete. 2024-05-08T05:47:05.259 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2024-05-08T05:47:04.913283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254992,"kb_used_data":9672,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754719728,"statfs":{"total":773094113280,"available":772833001472,"internally_reserved":0,"allocated":9904128,"data_stored":5146564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001094"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.896589+0000","last_change":"2024-05-08T05:47:00.852400+0000","last_active":"2024-05-08T05:47:01.896589+0000","last_peered":"2024-05-08T05:47:01.896589+0000","last_clean":"2024-05-08T05:47:01.896589+0000","last_became_active":"2024-05-08T05:47:00.852108+0000","last_became_peered":"2024-05-08T05:47:00.852108+0000","last_unstale":"2024-05-08T05:47:01.896589+0000","last_undegraded":"2024-05-08T05:47:01.896589+0000","last_fullsized":"2024-05-08T05:47:01.896589+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:30:12.645791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.837711+0000","last_change":"2024-05-08T05:47:00.851819+0000","last_active":"2024-05-08T05:47:02.837711+0000","last_peered":"2024-05-08T05:47:02.837711+0000","last_clean":"2024-05-08T05:47:02.837711+0000","last_became_active":"2024-05-08T05:47:00.851517+0000","last_became_peered":"2024-05-08T05:47:00.851517+0000","last_unstale":"2024-05-08T05:47:02.837711+0000","last_undegraded":"2024-05-08T05:47:02.837711+0000","last_fullsized":"2024-05-08T05:47:02.837711+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:20:51.412253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.896456+0000","last_change":"2024-05-08T05:47:00.855796+0000","last_active":"2024-05-08T05:47:01.896456+0000","last_peered":"2024-05-08T05:47:01.896456+0000","last_clean":"2024-05-08T05:47:01.896456+0000","last_became_active":"2024-05-08T05:47:00.855493+0000","last_became_peered":"2024-05-08T05:47:00.855493+0000","last_unstale":"2024-05-08T05:47:01.896456+0000","last_undegraded":"2024-05-08T05:47:01.896456+0000","last_fullsized":"2024-05-08T05:47:01.896456+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:32.109827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.191214+0000","last_change":"2024-05-08T05:47:00.860370+0000","last_active":"2024-05-08T05:47:02.191214+0000","last_peered":"2024-05-08T05:47:02.191214+0000","last_clean":"2024-05-08T05:47:02.191214+0000","last_became_active":"2024-05-08T05:47:00.860088+0000","last_became_peered":"2024-05-08T05:47:00.860088+0000","last_unstale":"2024-05-08T05:47:02.191214+0000","last_undegraded":"2024-05-08T05:47:02.191214+0000","last_fullsized":"2024-05-08T05:47:02.191214+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:26:16.877245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.848095+0000","last_change":"2024-05-08T05:47:00.851909+0000","last_active":"2024-05-08T05:47:01.848095+0000","last_peered":"2024-05-08T05:47:01.848095+0000","last_clean":"2024-05-08T05:47:01.848095+0000","last_became_active":"2024-05-08T05:47:00.851504+0000","last_became_peered":"2024-05-08T05:47:00.851504+0000","last_unstale":"2024-05-08T05:47:01.848095+0000","last_undegraded":"2024-05-08T05:47:01.848095+0000","last_fullsized":"2024-05-08T05:47:01.848095+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:35:58.648967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.547699+0000","last_change":"2024-05-08T05:47:00.861965+0000","last_active":"2024-05-08T05:47:02.547699+0000","last_peered":"2024-05-08T05:47:02.547699+0000","last_clean":"2024-05-08T05:47:02.547699+0000","last_became_active":"2024-05-08T05:47:00.861669+0000","last_became_peered":"2024-05-08T05:47:00.861669+0000","last_unstale":"2024-05-08T05:47:02.547699+0000","last_undegraded":"2024-05-08T05:47:02.547699+0000","last_fullsized":"2024-05-08T05:47:02.547699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:41:09.429596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:04.123077+0000","last_change":"2024-05-08T05:47:00.847165+0000","last_active":"2024-05-08T05:47:04.123077+0000","last_peered":"2024-05-08T05:47:04.123077+0000","last_clean":"2024-05-08T05:47:04.123077+0000","last_became_active":"2024-05-08T05:47:00.846857+0000","last_became_peered":"2024-05-08T05:47:00.846857+0000","last_unstale":"2024-05-08T05:47:04.123077+0000","last_undegraded":"2024-05-08T05:47:04.123077+0000","last_fullsized":"2024-05-08T05:47:04.123077+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:15.938605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.190907+0000","last_change":"2024-05-08T05:47:00.863607+0000","last_active":"2024-05-08T05:47:02.190907+0000","last_peered":"2024-05-08T05:47:02.190907+0000","last_clean":"2024-05-08T05:47:02.190907+0000","last_became_active":"2024-05-08T05:47:00.862945+0000","last_became_peered":"2024-05-08T05:47:00.862945+0000","last_unstale":"2024-05-08T05:47:02.190907+0000","last_undegraded":"2024-05-08T05:47:02.190907+0000","last_fullsized":"2024-05-08T05:47:02.190907+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:46:31.123655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.848641+0000","last_change":"2024-05-08T05:47:00.850943+0000","last_active":"2024-05-08T05:47:01.848641+0000","last_peered":"2024-05-08T05:47:01.848641+0000","last_clean":"2024-05-08T05:47:01.848641+0000","last_became_active":"2024-05-08T05:47:00.850610+0000","last_became_peered":"2024-05-08T05:47:00.850610+0000","last_unstale":"2024-05-08T05:47:01.848641+0000","last_undegraded":"2024-05-08T05:47:01.848641+0000","last_fullsized":"2024-05-08T05:47:01.848641+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:12.748375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.844460+0000","last_change":"2024-05-08T05:47:00.852114+0000","last_active":"2024-05-08T05:47:01.844460+0000","last_peered":"2024-05-08T05:47:01.844460+0000","last_clean":"2024-05-08T05:47:01.844460+0000","last_became_active":"2024-05-08T05:47:00.851818+0000","last_became_peered":"2024-05-08T05:47:00.851818+0000","last_unstale":"2024-05-08T05:47:01.844460+0000","last_undegraded":"2024-05-08T05:47:01.844460+0000","last_fullsized":"2024-05-08T05:47:01.844460+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:59:57.097001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.848340+0000","last_change":"2024-05-08T05:47:00.854673+0000","last_active":"2024-05-08T05:47:01.848340+0000","last_peered":"2024-05-08T05:47:01.848340+0000","last_clean":"2024-05-08T05:47:01.848340+0000","last_became_active":"2024-05-08T05:47:00.854408+0000","last_became_peered":"2024-05-08T05:47:00.854408+0000","last_unstale":"2024-05-08T05:47:01.848340+0000","last_undegraded":"2024-05-08T05:47:01.848340+0000","last_fullsized":"2024-05-08T05:47:01.848340+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:16:29.634523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.843861+0000","last_change":"2024-05-08T05:47:00.845719+0000","last_active":"2024-05-08T05:47:01.843861+0000","last_peered":"2024-05-08T05:47:01.843861+0000","last_clean":"2024-05-08T05:47:01.843861+0000","last_became_active":"2024-05-08T05:47:00.845365+0000","last_became_peered":"2024-05-08T05:47:00.845365+0000","last_unstale":"2024-05-08T05:47:01.843861+0000","last_undegraded":"2024-05-08T05:47:01.843861+0000","last_fullsized":"2024-05-08T05:47:01.843861+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:19:37.275690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.191118+0000","last_change":"2024-05-08T05:47:00.863687+0000","last_active":"2024-05-08T05:47:02.191118+0000","last_peered":"2024-05-08T05:47:02.191118+0000","last_clean":"2024-05-08T05:47:02.191118+0000","last_became_active":"2024-05-08T05:47:00.863415+0000","last_became_peered":"2024-05-08T05:47:00.863415+0000","last_unstale":"2024-05-08T05:47:02.191118+0000","last_undegraded":"2024-05-08T05:47:02.191118+0000","last_fullsized":"2024-05-08T05:47:02.191118+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:08.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.847812+0000","last_change":"2024-05-08T05:47:00.864674+0000","last_active":"2024-05-08T05:47:01.847812+0000","last_peered":"2024-05-08T05:47:01.847812+0000","last_clean":"2024-05-08T05:47:01.847812+0000","last_became_active":"2024-05-08T05:47:00.864395+0000","last_became_peered":"2024-05-08T05:47:00.864395+0000","last_unstale":"2024-05-08T05:47:01.847812+0000","last_undegraded":"2024-05-08T05:47:01.847812+0000","last_fullsized":"2024-05-08T05:47:01.847812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:44.467962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.191030+0000","last_change":"2024-05-08T05:47:00.859537+0000","last_active":"2024-05-08T05:47:02.191030+0000","last_peered":"2024-05-08T05:47:02.191030+0000","last_clean":"2024-05-08T05:47:02.191030+0000","last_became_active":"2024-05-08T05:47:00.858644+0000","last_became_peered":"2024-05-08T05:47:00.858644+0000","last_unstale":"2024-05-08T05:47:02.191030+0000","last_undegraded":"2024-05-08T05:47:02.191030+0000","last_fullsized":"2024-05-08T05:47:02.191030+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:35:55.038751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.547561+0000","last_change":"2024-05-08T05:47:00.853756+0000","last_active":"2024-05-08T05:47:02.547561+0000","last_peered":"2024-05-08T05:47:02.547561+0000","last_clean":"2024-05-08T05:47:02.547561+0000","last_became_active":"2024-05-08T05:47:00.853250+0000","last_became_peered":"2024-05-08T05:47:00.853250+0000","last_unstale":"2024-05-08T05:47:02.547561+0000","last_undegraded":"2024-05-08T05:47:02.547561+0000","last_fullsized":"2024-05-08T05:47:02.547561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:01:52.374936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.548076+0000","last_change":"2024-05-08T05:47:00.848107+0000","last_active":"2024-05-08T05:47:02.548076+0000","last_peered":"2024-05-08T05:47:02.548076+0000","last_clean":"2024-05-08T05:47:02.548076+0000","last_became_active":"2024-05-08T05:47:00.847801+0000","last_became_peered":"2024-05-08T05:47:00.847801+0000","last_unstale":"2024-05-08T05:47:02.548076+0000","last_undegraded":"2024-05-08T05:47:02.548076+0000","last_fullsized":"2024-05-08T05:47:02.548076+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:33.709994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.896550+0000","last_change":"2024-05-08T05:47:00.849624+0000","last_active":"2024-05-08T05:47:01.896550+0000","last_peered":"2024-05-08T05:47:01.896550+0000","last_clean":"2024-05-08T05:47:01.896550+0000","last_became_active":"2024-05-08T05:47:00.849308+0000","last_became_peered":"2024-05-08T05:47:00.849308+0000","last_unstale":"2024-05-08T05:47:01.896550+0000","last_undegraded":"2024-05-08T05:47:01.896550+0000","last_fullsized":"2024-05-08T05:47:01.896550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:48:44.284191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.848560+0000","last_change":"2024-05-08T05:47:00.857607+0000","last_active":"2024-05-08T05:47:01.848560+0000","last_peered":"2024-05-08T05:47:01.848560+0000","last_clean":"2024-05-08T05:47:01.848560+0000","last_became_active":"2024-05-08T05:47:00.857309+0000","last_became_peered":"2024-05-08T05:47:00.857309+0000","last_unstale":"2024-05-08T05:47:01.848560+0000","last_undegraded":"2024-05-08T05:47:01.848560+0000","last_fullsized":"2024-05-08T05:47:01.848560+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:39:04.751334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.191268+0000","last_change":"2024-05-08T05:47:00.849558+0000","last_active":"2024-05-08T05:47:02.191268+0000","last_peered":"2024-05-08T05:47:02.191268+0000","last_clean":"2024-05-08T05:47:02.191268+0000","last_became_active":"2024-05-08T05:47:00.849085+0000","last_became_peered":"2024-05-08T05:47:00.849085+0000","last_unstale":"2024-05-08T05:47:02.191268+0000","last_undegraded":"2024-05-08T05:47:02.191268+0000","last_fullsized":"2024-05-08T05:47:02.191268+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:36:15.998298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.847893+0000","last_change":"2024-05-08T05:47:00.853909+0000","last_active":"2024-05-08T05:47:01.847893+0000","last_peered":"2024-05-08T05:47:01.847893+0000","last_clean":"2024-05-08T05:47:01.847893+0000","last_became_active":"2024-05-08T05:47:00.853260+0000","last_became_peered":"2024-05-08T05:47:00.853260+0000","last_unstale":"2024-05-08T05:47:01.847893+0000","last_undegraded":"2024-05-08T05:47:01.847893+0000","last_fullsized":"2024-05-08T05:47:01.847893+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:29:44.335626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.848184+0000","last_change":"2024-05-08T05:47:00.863204+0000","last_active":"2024-05-08T05:47:01.848184+0000","last_peered":"2024-05-08T05:47:01.848184+0000","last_clean":"2024-05-08T05:47:01.848184+0000","last_became_active":"2024-05-08T05:47:00.862779+0000","last_became_peered":"2024-05-08T05:47:00.862779+0000","last_unstale":"2024-05-08T05:47:01.848184+0000","last_undegraded":"2024-05-08T05:47:01.848184+0000","last_fullsized":"2024-05-08T05:47:01.848184+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:35:55.038751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.210893+0000","last_change":"2024-05-08T05:47:00.863982+0000","last_active":"2024-05-08T05:47:02.210893+0000","last_peered":"2024-05-08T05:47:02.210893+0000","last_clean":"2024-05-08T05:47:02.210893+0000","last_became_active":"2024-05-08T05:47:00.863710+0000","last_became_peered":"2024-05-08T05:47:00.863710+0000","last_unstale":"2024-05-08T05:47:02.210893+0000","last_undegraded":"2024-05-08T05:47:02.210893+0000","last_fullsized":"2024-05-08T05:47:02.210893+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:39:04.751334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.896306+0000","last_change":"2024-05-08T05:47:00.844190+0000","last_active":"2024-05-08T05:47:01.896306+0000","last_peered":"2024-05-08T05:47:01.896306+0000","last_clean":"2024-05-08T05:47:01.896306+0000","last_became_active":"2024-05-08T05:47:00.843838+0000","last_became_peered":"2024-05-08T05:47:00.843838+0000","last_unstale":"2024-05-08T05:47:01.896306+0000","last_undegraded":"2024-05-08T05:47:01.896306+0000","last_fullsized":"2024-05-08T05:47:01.896306+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:44:04.293068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.837813+0000","last_change":"2024-05-08T05:47:00.850058+0000","last_active":"2024-05-08T05:47:02.837813+0000","last_peered":"2024-05-08T05:47:02.837813+0000","last_clean":"2024-05-08T05:47:02.837813+0000","last_became_active":"2024-05-08T05:47:00.849759+0000","last_became_peered":"2024-05-08T05:47:00.849759+0000","last_unstale":"2024-05-08T05:47:02.837813+0000","last_undegraded":"2024-05-08T05:47:02.837813+0000","last_fullsized":"2024-05-08T05:47:02.837813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:08.313654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.848424+0000","last_change":"2024-05-08T05:47:00.857771+0000","last_active":"2024-05-08T05:47:01.848424+0000","last_peered":"2024-05-08T05:47:01.848424+0000","last_clean":"2024-05-08T05:47:01.848424+0000","last_became_active":"2024-05-08T05:47:00.857436+0000","last_became_peered":"2024-05-08T05:47:00.857436+0000","last_unstale":"2024-05-08T05:47:01.848424+0000","last_undegraded":"2024-05-08T05:47:01.848424+0000","last_fullsized":"2024-05-08T05:47:01.848424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:36:15.998298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.844185+0000","last_change":"2024-05-08T05:47:00.841385+0000","last_active":"2024-05-08T05:47:01.844185+0000","last_peered":"2024-05-08T05:47:01.844185+0000","last_clean":"2024-05-08T05:47:01.844185+0000","last_became_active":"2024-05-08T05:47:00.841108+0000","last_became_peered":"2024-05-08T05:47:00.841108+0000","last_unstale":"2024-05-08T05:47:01.844185+0000","last_undegraded":"2024-05-08T05:47:01.844185+0000","last_fullsized":"2024-05-08T05:47:01.844185+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:27:09.253317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.191364+0000","last_change":"2024-05-08T05:47:00.862848+0000","last_active":"2024-05-08T05:47:02.191364+0000","last_peered":"2024-05-08T05:47:02.191364+0000","last_clean":"2024-05-08T05:47:02.191364+0000","last_became_active":"2024-05-08T05:47:00.862570+0000","last_became_peered":"2024-05-08T05:47:00.862570+0000","last_unstale":"2024-05-08T05:47:02.191364+0000","last_undegraded":"2024-05-08T05:47:02.191364+0000","last_fullsized":"2024-05-08T05:47:02.191364+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:12.748375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:04.123178+0000","last_change":"2024-05-08T05:47:00.846155+0000","last_active":"2024-05-08T05:47:04.123178+0000","last_peered":"2024-05-08T05:47:04.123178+0000","last_clean":"2024-05-08T05:47:04.123178+0000","last_became_active":"2024-05-08T05:47:00.845061+0000","last_became_peered":"2024-05-08T05:47:00.845061+0000","last_unstale":"2024-05-08T05:47:04.123178+0000","last_undegraded":"2024-05-08T05:47:04.123178+0000","last_fullsized":"2024-05-08T05:47:04.123178+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:37:47.239654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.848086+0000","last_change":"2024-05-08T05:47:00.852444+0000","last_active":"2024-05-08T05:47:01.848086+0000","last_peered":"2024-05-08T05:47:01.848086+0000","last_clean":"2024-05-08T05:47:01.848086+0000","last_became_active":"2024-05-08T05:47:00.852162+0000","last_became_peered":"2024-05-08T05:47:00.852162+0000","last_unstale":"2024-05-08T05:47:01.848086+0000","last_undegraded":"2024-05-08T05:47:01.848086+0000","last_fullsized":"2024-05-08T05:47:01.848086+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:46:31.123655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.837418+0000","last_change":"2024-05-08T05:47:00.866281+0000","last_active":"2024-05-08T05:47:02.837418+0000","last_peered":"2024-05-08T05:47:02.837418+0000","last_clean":"2024-05-08T05:47:02.837418+0000","last_became_active":"2024-05-08T05:47:00.866029+0000","last_became_peered":"2024-05-08T05:47:00.866029+0000","last_unstale":"2024-05-08T05:47:02.837418+0000","last_undegraded":"2024-05-08T05:47:02.837418+0000","last_fullsized":"2024-05-08T05:47:02.837418+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:34:04.733020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.547628+0000","last_change":"2024-05-08T05:47:00.853549+0000","last_active":"2024-05-08T05:47:02.547628+0000","last_peered":"2024-05-08T05:47:02.547628+0000","last_clean":"2024-05-08T05:47:02.547628+0000","last_became_active":"2024-05-08T05:47:00.852832+0000","last_became_peered":"2024-05-08T05:47:00.852832+0000","last_unstale":"2024-05-08T05:47:02.547628+0000","last_undegraded":"2024-05-08T05:47:02.547628+0000","last_fullsized":"2024-05-08T05:47:02.547628+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:57:57.567766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.547793+0000","last_change":"2024-05-08T05:47:00.858083+0000","last_active":"2024-05-08T05:47:02.547793+0000","last_peered":"2024-05-08T05:47:02.547793+0000","last_clean":"2024-05-08T05:47:02.547793+0000","last_became_active":"2024-05-08T05:47:00.854772+0000","last_became_peered":"2024-05-08T05:47:00.854772+0000","last_unstale":"2024-05-08T05:47:02.547793+0000","last_undegraded":"2024-05-08T05:47:02.547793+0000","last_fullsized":"2024-05-08T05:47:02.547793+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:50:02.496474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"87'4","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.210974+0000","last_change":"2024-05-08T05:46:59.846402+0000","last_active":"2024-05-08T05:47:02.210974+0000","last_peered":"2024-05-08T05:47:02.210974+0000","last_clean":"2024-05-08T05:47:02.210974+0000","last_became_active":"2024-05-08T05:46:59.845575+0000","last_became_peered":"2024-05-08T05:46:59.845575+0000","last_unstale":"2024-05-08T05:47:02.210974+0000","last_undegraded":"2024-05-08T05:47:02.210974+0000","last_fullsized":"2024-05-08T05:47:02.210974+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:06:09.948583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.548008+0000","last_change":"2024-05-08T05:47:00.848961+0000","last_active":"2024-05-08T05:47:02.548008+0000","last_peered":"2024-05-08T05:47:02.548008+0000","last_clean":"2024-05-08T05:47:02.548008+0000","last_became_active":"2024-05-08T05:47:00.848613+0000","last_became_peered":"2024-05-08T05:47:00.848613+0000","last_unstale":"2024-05-08T05:47:02.548008+0000","last_undegraded":"2024-05-08T05:47:02.548008+0000","last_fullsized":"2024-05-08T05:47:02.548008+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:44:41.697582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"87'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.848495+0000","last_change":"2024-05-08T05:46:59.850241+0000","last_active":"2024-05-08T05:47:01.848495+0000","last_peered":"2024-05-08T05:47:01.848495+0000","last_clean":"2024-05-08T05:47:01.848495+0000","last_became_active":"2024-05-08T05:46:59.849925+0000","last_became_peered":"2024-05-08T05:46:59.849925+0000","last_unstale":"2024-05-08T05:47:01.848495+0000","last_undegraded":"2024-05-08T05:47:01.848495+0000","last_fullsized":"2024-05-08T05:47:01.848495+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:36:08.134022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:04.123162+0000","last_change":"2024-05-08T05:47:00.863465+0000","last_active":"2024-05-08T05:47:04.123162+0000","last_peered":"2024-05-08T05:47:04.123162+0000","last_clean":"2024-05-08T05:47:04.123162+0000","last_became_active":"2024-05-08T05:47:00.863149+0000","last_became_peered":"2024-05-08T05:47:00.863149+0000","last_unstale":"2024-05-08T05:47:04.123162+0000","last_undegraded":"2024-05-08T05:47:04.123162+0000","last_fullsized":"2024-05-08T05:47:04.123162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:39:28.929527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"87'2","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:04.123211+0000","last_change":"2024-05-08T05:46:59.851832+0000","last_active":"2024-05-08T05:47:04.123211+0000","last_peered":"2024-05-08T05:47:04.123211+0000","last_clean":"2024-05-08T05:47:04.123211+0000","last_became_active":"2024-05-08T05:46:59.851566+0000","last_became_peered":"2024-05-08T05:46:59.851566+0000","last_unstale":"2024-05-08T05:47:04.123211+0000","last_undegraded":"2024-05-08T05:47:04.123211+0000","last_fullsized":"2024-05-08T05:47:04.123211+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:02:52.230765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.548121+0000","last_change":"2024-05-08T05:47:00.844969+0000","last_active":"2024-05-08T05:47:02.548121+0000","last_peered":"2024-05-08T05:47:02.548121+0000","last_clean":"2024-05-08T05:47:02.548121+0000","last_became_active":"2024-05-08T05:47:00.844674+0000","last_became_peered":"2024-05-08T05:47:00.844674+0000","last_unstale":"2024-05-08T05:47:02.548121+0000","last_undegraded":"2024-05-08T05:47:02.548121+0000","last_fullsized":"2024-05-08T05:47:02.548121+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:33:39.662902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"87'2","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.210909+0000","last_change":"2024-05-08T05:46:59.852748+0000","last_active":"2024-05-08T05:47:02.210909+0000","last_peered":"2024-05-08T05:47:02.210909+0000","last_clean":"2024-05-08T05:47:02.210909+0000","last_became_active":"2024-05-08T05:46:59.852436+0000","last_became_peered":"2024-05-08T05:46:59.852436+0000","last_unstale":"2024-05-08T05:47:02.210909+0000","last_undegraded":"2024-05-08T05:47:02.210909+0000","last_fullsized":"2024-05-08T05:47:02.210909+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:51:38.811587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.190972+0000","last_change":"2024-05-08T05:47:00.853832+0000","last_active":"2024-05-08T05:47:02.190972+0000","last_peered":"2024-05-08T05:47:02.190972+0000","last_clean":"2024-05-08T05:47:02.190972+0000","last_became_active":"2024-05-08T05:47:00.852018+0000","last_became_peered":"2024-05-08T05:47:00.852018+0000","last_unstale":"2024-05-08T05:47:02.190972+0000","last_undegraded":"2024-05-08T05:47:02.190972+0000","last_fullsized":"2024-05-08T05:47:02.190972+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:35:58.648967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"87'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.848118+0000","last_change":"2024-05-08T05:46:59.851029+0000","last_active":"2024-05-08T05:47:01.848118+0000","last_peered":"2024-05-08T05:47:01.848118+0000","last_clean":"2024-05-08T05:47:01.848118+0000","last_became_active":"2024-05-08T05:46:59.850704+0000","last_became_peered":"2024-05-08T05:46:59.850704+0000","last_unstale":"2024-05-08T05:47:01.848118+0000","last_undegraded":"2024-05-08T05:47:01.848118+0000","last_fullsized":"2024-05-08T05:47:01.848118+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:05:07.639122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.210782+0000","last_change":"2024-05-08T05:47:00.852873+0000","last_active":"2024-05-08T05:47:02.210782+0000","last_peered":"2024-05-08T05:47:02.210782+0000","last_clean":"2024-05-08T05:47:02.210782+0000","last_became_active":"2024-05-08T05:47:00.852574+0000","last_became_peered":"2024-05-08T05:47:00.852574+0000","last_unstale":"2024-05-08T05:47:02.210782+0000","last_undegraded":"2024-05-08T05:47:02.210782+0000","last_fullsized":"2024-05-08T05:47:02.210782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:26:16.877245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"87'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.191352+0000","last_change":"2024-05-08T05:46:59.850721+0000","last_active":"2024-05-08T05:47:02.191352+0000","last_peered":"2024-05-08T05:47:02.191352+0000","last_clean":"2024-05-08T05:47:02.191352+0000","last_became_active":"2024-05-08T05:46:59.850403+0000","last_became_peered":"2024-05-08T05:46:59.850403+0000","last_unstale":"2024-05-08T05:47:02.191352+0000","last_undegraded":"2024-05-08T05:47:02.191352+0000","last_fullsized":"2024-05-08T05:47:02.191352+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:35:34.561464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.547557+0000","last_change":"2024-05-08T05:47:00.861558+0000","last_active":"2024-05-08T05:47:02.547557+0000","last_peered":"2024-05-08T05:47:02.547557+0000","last_clean":"2024-05-08T05:47:02.547557+0000","last_became_active":"2024-05-08T05:47:00.861265+0000","last_became_peered":"2024-05-08T05:47:00.861265+0000","last_unstale":"2024-05-08T05:47:02.547557+0000","last_undegraded":"2024-05-08T05:47:02.547557+0000","last_fullsized":"2024-05-08T05:47:02.547557+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:00:55.617144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.896290+0000","last_change":"2024-05-08T05:46:59.848632+0000","last_active":"2024-05-08T05:47:01.896290+0000","last_peered":"2024-05-08T05:47:01.896290+0000","last_clean":"2024-05-08T05:47:01.896290+0000","last_became_active":"2024-05-08T05:46:59.848355+0000","last_became_peered":"2024-05-08T05:46:59.848355+0000","last_unstale":"2024-05-08T05:47:01.896290+0000","last_undegraded":"2024-05-08T05:47:01.896290+0000","last_fullsized":"2024-05-08T05:47:01.896290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:07.311511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.191139+0000","last_change":"2024-05-08T05:47:00.858770+0000","last_active":"2024-05-08T05:47:02.191139+0000","last_peered":"2024-05-08T05:47:02.191139+0000","last_clean":"2024-05-08T05:47:02.191139+0000","last_became_active":"2024-05-08T05:47:00.858489+0000","last_became_peered":"2024-05-08T05:47:00.858489+0000","last_unstale":"2024-05-08T05:47:02.191139+0000","last_undegraded":"2024-05-08T05:47:02.191139+0000","last_fullsized":"2024-05-08T05:47:02.191139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:16:29.634523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:04.123214+0000","last_change":"2024-05-08T05:46:59.850380+0000","last_active":"2024-05-08T05:47:04.123214+0000","last_peered":"2024-05-08T05:47:04.123214+0000","last_clean":"2024-05-08T05:47:04.123214+0000","last_became_active":"2024-05-08T05:46:59.850075+0000","last_became_peered":"2024-05-08T05:46:59.850075+0000","last_unstale":"2024-05-08T05:47:04.123214+0000","last_undegraded":"2024-05-08T05:47:04.123214+0000","last_fullsized":"2024-05-08T05:47:04.123214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:47.851660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.848266+0000","last_change":"2024-05-08T05:47:00.861154+0000","last_active":"2024-05-08T05:47:01.848266+0000","last_peered":"2024-05-08T05:47:01.848266+0000","last_clean":"2024-05-08T05:47:01.848266+0000","last_became_active":"2024-05-08T05:47:00.860885+0000","last_became_peered":"2024-05-08T05:47:00.860885+0000","last_unstale":"2024-05-08T05:47:01.848266+0000","last_undegraded":"2024-05-08T05:47:01.848266+0000","last_fullsized":"2024-05-08T05:47:01.848266+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:10:08.200116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.837890+0000","last_change":"2024-05-08T05:46:59.843987+0000","last_active":"2024-05-08T05:47:02.837890+0000","last_peered":"2024-05-08T05:47:02.837890+0000","last_clean":"2024-05-08T05:47:02.837890+0000","last_became_active":"2024-05-08T05:46:59.843622+0000","last_became_peered":"2024-05-08T05:46:59.843622+0000","last_unstale":"2024-05-08T05:47:02.837890+0000","last_undegraded":"2024-05-08T05:47:02.837890+0000","last_fullsized":"2024-05-08T05:47:02.837890+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:35:54.036608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:04.123199+0000","last_change":"2024-05-08T05:47:00.852035+0000","last_active":"2024-05-08T05:47:04.123199+0000","last_peered":"2024-05-08T05:47:04.123199+0000","last_clean":"2024-05-08T05:47:04.123199+0000","last_became_active":"2024-05-08T05:47:00.851727+0000","last_became_peered":"2024-05-08T05:47:00.851727+0000","last_unstale":"2024-05-08T05:47:04.123199+0000","last_undegraded":"2024-05-08T05:47:04.123199+0000","last_fullsized":"2024-05-08T05:47:04.123199+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:12:57.158812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.896607+0000","last_change":"2024-05-08T05:46:59.853139+0000","last_active":"2024-05-08T05:47:01.896607+0000","last_peered":"2024-05-08T05:47:01.896607+0000","last_clean":"2024-05-08T05:47:01.896607+0000","last_became_active":"2024-05-08T05:46:59.852837+0000","last_became_peered":"2024-05-08T05:46:59.852837+0000","last_unstale":"2024-05-08T05:47:01.896607+0000","last_undegraded":"2024-05-08T05:47:01.896607+0000","last_fullsized":"2024-05-08T05:47:01.896607+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:56.785804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.896612+0000","last_change":"2024-05-08T05:47:00.845923+0000","last_active":"2024-05-08T05:47:01.896612+0000","last_peered":"2024-05-08T05:47:01.896612+0000","last_clean":"2024-05-08T05:47:01.896612+0000","last_became_active":"2024-05-08T05:47:00.845612+0000","last_became_peered":"2024-05-08T05:47:00.845612+0000","last_unstale":"2024-05-08T05:47:01.896612+0000","last_undegraded":"2024-05-08T05:47:01.896612+0000","last_fullsized":"2024-05-08T05:47:01.896612+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:15.938605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"87'3","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.848521+0000","last_change":"2024-05-08T05:46:59.849314+0000","last_active":"2024-05-08T05:47:01.848521+0000","last_peered":"2024-05-08T05:47:01.848521+0000","last_clean":"2024-05-08T05:47:01.848521+0000","last_became_active":"2024-05-08T05:46:59.848790+0000","last_became_peered":"2024-05-08T05:46:59.848790+0000","last_unstale":"2024-05-08T05:47:01.848521+0000","last_undegraded":"2024-05-08T05:47:01.848521+0000","last_fullsized":"2024-05-08T05:47:01.848521+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:33.515648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.191515+0000","last_change":"2024-05-08T05:47:00.859545+0000","last_active":"2024-05-08T05:47:02.191515+0000","last_peered":"2024-05-08T05:47:02.191515+0000","last_clean":"2024-05-08T05:47:02.191515+0000","last_became_active":"2024-05-08T05:47:00.859269+0000","last_became_peered":"2024-05-08T05:47:00.859269+0000","last_unstale":"2024-05-08T05:47:02.191515+0000","last_undegraded":"2024-05-08T05:47:02.191515+0000","last_fullsized":"2024-05-08T05:47:02.191515+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:54:24.816957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"87'6","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.547864+0000","last_change":"2024-05-08T05:46:59.840278+0000","last_active":"2024-05-08T05:47:02.547864+0000","last_peered":"2024-05-08T05:47:02.547864+0000","last_clean":"2024-05-08T05:47:02.547864+0000","last_became_active":"2024-05-08T05:46:59.839997+0000","last_became_peered":"2024-05-08T05:46:59.839997+0000","last_unstale":"2024-05-08T05:47:02.547864+0000","last_undegraded":"2024-05-08T05:47:02.547864+0000","last_fullsized":"2024-05-08T05:47:02.547864+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:47.851660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.191331+0000","last_change":"2024-05-08T05:47:00.863223+0000","last_active":"2024-05-08T05:47:02.191331+0000","last_peered":"2024-05-08T05:47:02.191331+0000","last_clean":"2024-05-08T05:47:02.191331+0000","last_became_active":"2024-05-08T05:47:00.862561+0000","last_became_peered":"2024-05-08T05:47:00.862561+0000","last_unstale":"2024-05-08T05:47:02.191331+0000","last_undegraded":"2024-05-08T05:47:02.191331+0000","last_fullsized":"2024-05-08T05:47:02.191331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:39:04.751334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"87'2","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:04.123224+0000","last_change":"2024-05-08T05:46:59.850252+0000","last_active":"2024-05-08T05:47:04.123224+0000","last_peered":"2024-05-08T05:47:04.123224+0000","last_clean":"2024-05-08T05:47:04.123224+0000","last_became_active":"2024-05-08T05:46:59.849947+0000","last_became_peered":"2024-05-08T05:46:59.849947+0000","last_unstale":"2024-05-08T05:47:04.123224+0000","last_undegraded":"2024-05-08T05:47:04.123224+0000","last_fullsized":"2024-05-08T05:47:04.123224+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:24:32.526804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.847793+0000","last_change":"2024-05-08T05:47:00.853650+0000","last_active":"2024-05-08T05:47:01.847793+0000","last_peered":"2024-05-08T05:47:01.847793+0000","last_clean":"2024-05-08T05:47:01.847793+0000","last_became_active":"2024-05-08T05:47:00.853356+0000","last_became_peered":"2024-05-08T05:47:00.853356+0000","last_unstale":"2024-05-08T05:47:01.847793+0000","last_undegraded":"2024-05-08T05:47:01.847793+0000","last_fullsized":"2024-05-08T05:47:01.847793+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:36:43.651363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.547642+0000","last_change":"2024-05-08T05:46:59.849246+0000","last_active":"2024-05-08T05:47:02.547642+0000","last_peered":"2024-05-08T05:47:02.547642+0000","last_clean":"2024-05-08T05:47:02.547642+0000","last_became_active":"2024-05-08T05:46:59.848949+0000","last_became_peered":"2024-05-08T05:46:59.848949+0000","last_unstale":"2024-05-08T05:47:02.547642+0000","last_undegraded":"2024-05-08T05:47:02.547642+0000","last_fullsized":"2024-05-08T05:47:02.547642+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:56.785804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.843771+0000","last_change":"2024-05-08T05:47:00.861174+0000","last_active":"2024-05-08T05:47:01.843771+0000","last_peered":"2024-05-08T05:47:01.843771+0000","last_clean":"2024-05-08T05:47:01.843771+0000","last_became_active":"2024-05-08T05:47:00.860833+0000","last_became_peered":"2024-05-08T05:47:00.860833+0000","last_unstale":"2024-05-08T05:47:01.843771+0000","last_undegraded":"2024-05-08T05:47:01.843771+0000","last_fullsized":"2024-05-08T05:47:01.843771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:33:42.508681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.191028+0000","last_change":"2024-05-08T05:46:59.850482+0000","last_active":"2024-05-08T05:47:02.191028+0000","last_peered":"2024-05-08T05:47:02.191028+0000","last_clean":"2024-05-08T05:47:02.191028+0000","last_became_active":"2024-05-08T05:46:59.850083+0000","last_became_peered":"2024-05-08T05:46:59.850083+0000","last_unstale":"2024-05-08T05:47:02.191028+0000","last_undegraded":"2024-05-08T05:47:02.191028+0000","last_fullsized":"2024-05-08T05:47:02.191028+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:08:58.752524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.843913+0000","last_change":"2024-05-08T05:47:00.856770+0000","last_active":"2024-05-08T05:47:01.843913+0000","last_peered":"2024-05-08T05:47:01.843913+0000","last_clean":"2024-05-08T05:47:01.843913+0000","last_became_active":"2024-05-08T05:47:00.856531+0000","last_became_peered":"2024-05-08T05:47:00.856531+0000","last_unstale":"2024-05-08T05:47:01.843913+0000","last_undegraded":"2024-05-08T05:47:01.843913+0000","last_fullsized":"2024-05-08T05:47:01.843913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:40:14.835372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"87'8","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.191048+0000","last_change":"2024-05-08T05:46:59.852055+0000","last_active":"2024-05-08T05:47:02.191048+0000","last_peered":"2024-05-08T05:47:02.191048+0000","last_clean":"2024-05-08T05:47:02.191048+0000","last_became_active":"2024-05-08T05:46:59.850503+0000","last_became_peered":"2024-05-08T05:46:59.850503+0000","last_unstale":"2024-05-08T05:47:02.191048+0000","last_undegraded":"2024-05-08T05:47:02.191048+0000","last_fullsized":"2024-05-08T05:47:02.191048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:38:56.262033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.547798+0000","last_change":"2024-05-08T05:47:00.857122+0000","last_active":"2024-05-08T05:47:02.547798+0000","last_peered":"2024-05-08T05:47:02.547798+0000","last_clean":"2024-05-08T05:47:02.547798+0000","last_became_active":"2024-05-08T05:47:00.856864+0000","last_became_peered":"2024-05-08T05:47:00.856864+0000","last_unstale":"2024-05-08T05:47:02.547798+0000","last_undegraded":"2024-05-08T05:47:02.547798+0000","last_fullsized":"2024-05-08T05:47:02.547798+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:57:36.787167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"87'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.211101+0000","last_change":"2024-05-08T05:46:59.848248+0000","last_active":"2024-05-08T05:47:02.211101+0000","last_peered":"2024-05-08T05:47:02.211101+0000","last_clean":"2024-05-08T05:47:02.211101+0000","last_became_active":"2024-05-08T05:46:59.847924+0000","last_became_peered":"2024-05-08T05:46:59.847924+0000","last_unstale":"2024-05-08T05:47:02.211101+0000","last_undegraded":"2024-05-08T05:47:02.211101+0000","last_fullsized":"2024-05-08T05:47:02.211101+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:14:34.468933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.210925+0000","last_change":"2024-05-08T05:47:00.853731+0000","last_active":"2024-05-08T05:47:02.210925+0000","last_peered":"2024-05-08T05:47:02.210925+0000","last_clean":"2024-05-08T05:47:02.210925+0000","last_became_active":"2024-05-08T05:47:00.853440+0000","last_became_peered":"2024-05-08T05:47:00.853440+0000","last_unstale":"2024-05-08T05:47:02.210925+0000","last_undegraded":"2024-05-08T05:47:02.210925+0000","last_fullsized":"2024-05-08T05:47:02.210925+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:12.748375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"87'2","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:04.123361+0000","last_change":"2024-05-08T05:46:59.847461+0000","last_active":"2024-05-08T05:47:04.123361+0000","last_peered":"2024-05-08T05:47:04.123361+0000","last_clean":"2024-05-08T05:47:04.123361+0000","last_became_active":"2024-05-08T05:46:59.847126+0000","last_became_peered":"2024-05-08T05:46:59.847126+0000","last_unstale":"2024-05-08T05:47:04.123361+0000","last_undegraded":"2024-05-08T05:47:04.123361+0000","last_fullsized":"2024-05-08T05:47:04.123361+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:22.746294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.848420+0000","last_change":"2024-05-08T05:47:00.851263+0000","last_active":"2024-05-08T05:47:01.848420+0000","last_peered":"2024-05-08T05:47:01.848420+0000","last_clean":"2024-05-08T05:47:01.848420+0000","last_became_active":"2024-05-08T05:47:00.850861+0000","last_became_peered":"2024-05-08T05:47:00.850861+0000","last_unstale":"2024-05-08T05:47:01.848420+0000","last_undegraded":"2024-05-08T05:47:01.848420+0000","last_fullsized":"2024-05-08T05:47:01.848420+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:26:16.877245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"87'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.848627+0000","last_change":"2024-05-08T05:46:59.853619+0000","last_active":"2024-05-08T05:47:01.848627+0000","last_peered":"2024-05-08T05:47:01.848627+0000","last_clean":"2024-05-08T05:47:01.848627+0000","last_became_active":"2024-05-08T05:46:59.853125+0000","last_became_peered":"2024-05-08T05:46:59.853125+0000","last_unstale":"2024-05-08T05:47:01.848627+0000","last_undegraded":"2024-05-08T05:47:01.848627+0000","last_fullsized":"2024-05-08T05:47:01.848627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:26:38.736547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.547862+0000","last_change":"2024-05-08T05:47:00.862652+0000","last_active":"2024-05-08T05:47:02.547862+0000","last_peered":"2024-05-08T05:47:02.547862+0000","last_clean":"2024-05-08T05:47:02.547862+0000","last_became_active":"2024-05-08T05:47:00.862347+0000","last_became_peered":"2024-05-08T05:47:00.862347+0000","last_unstale":"2024-05-08T05:47:02.547862+0000","last_undegraded":"2024-05-08T05:47:02.547862+0000","last_fullsized":"2024-05-08T05:47:02.547862+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:44.594673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"87'2","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.837404+0000","last_change":"2024-05-08T05:46:59.848971+0000","last_active":"2024-05-08T05:47:02.837404+0000","last_peered":"2024-05-08T05:47:02.837404+0000","last_clean":"2024-05-08T05:47:02.837404+0000","last_became_active":"2024-05-08T05:46:59.848666+0000","last_became_peered":"2024-05-08T05:46:59.848666+0000","last_unstale":"2024-05-08T05:47:02.837404+0000","last_undegraded":"2024-05-08T05:47:02.837404+0000","last_fullsized":"2024-05-08T05:47:02.837404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:35:57.646825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.837492+0000","last_change":"2024-05-08T05:47:00.852837+0000","last_active":"2024-05-08T05:47:02.837492+0000","last_peered":"2024-05-08T05:47:02.837492+0000","last_clean":"2024-05-08T05:47:02.837492+0000","last_became_active":"2024-05-08T05:47:00.852211+0000","last_became_peered":"2024-05-08T05:47:00.852211+0000","last_unstale":"2024-05-08T05:47:02.837492+0000","last_undegraded":"2024-05-08T05:47:02.837492+0000","last_fullsized":"2024-05-08T05:47:02.837492+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:41:39.446498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.843716+0000","last_change":"2024-05-08T05:46:59.848947+0000","last_active":"2024-05-08T05:47:01.843716+0000","last_peered":"2024-05-08T05:47:01.843716+0000","last_clean":"2024-05-08T05:47:01.843716+0000","last_became_active":"2024-05-08T05:46:59.848649+0000","last_became_peered":"2024-05-08T05:46:59.848649+0000","last_unstale":"2024-05-08T05:47:01.843716+0000","last_undegraded":"2024-05-08T05:47:01.843716+0000","last_fullsized":"2024-05-08T05:47:01.843716+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:38:59.840940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.837514+0000","last_change":"2024-05-08T05:47:00.864696+0000","last_active":"2024-05-08T05:47:02.837514+0000","last_peered":"2024-05-08T05:47:02.837514+0000","last_clean":"2024-05-08T05:47:02.837514+0000","last_became_active":"2024-05-08T05:47:00.864363+0000","last_became_peered":"2024-05-08T05:47:00.864363+0000","last_unstale":"2024-05-08T05:47:02.837514+0000","last_undegraded":"2024-05-08T05:47:02.837514+0000","last_fullsized":"2024-05-08T05:47:02.837514+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:20:22.061914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.837635+0000","last_change":"2024-05-08T05:46:59.844082+0000","last_active":"2024-05-08T05:47:02.837635+0000","last_peered":"2024-05-08T05:47:02.837635+0000","last_clean":"2024-05-08T05:47:02.837635+0000","last_became_active":"2024-05-08T05:46:59.843734+0000","last_became_peered":"2024-05-08T05:46:59.843734+0000","last_unstale":"2024-05-08T05:47:02.837635+0000","last_undegraded":"2024-05-08T05:47:02.837635+0000","last_fullsized":"2024-05-08T05:47:02.837635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:46:30.121512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.547795+0000","last_change":"2024-05-08T05:47:00.848222+0000","last_active":"2024-05-08T05:47:02.547795+0000","last_peered":"2024-05-08T05:47:02.547795+0000","last_clean":"2024-05-08T05:47:02.547795+0000","last_became_active":"2024-05-08T05:47:00.847949+0000","last_became_peered":"2024-05-08T05:47:00.847949+0000","last_unstale":"2024-05-08T05:47:02.547795+0000","last_undegraded":"2024-05-08T05:47:02.547795+0000","last_fullsized":"2024-05-08T05:47:02.547795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:25:22.355345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.843605+0000","last_change":"2024-05-08T05:46:59.840658+0000","last_active":"2024-05-08T05:47:01.843605+0000","last_peered":"2024-05-08T05:47:01.843605+0000","last_clean":"2024-05-08T05:47:01.843605+0000","last_became_active":"2024-05-08T05:46:59.840374+0000","last_became_peered":"2024-05-08T05:46:59.840374+0000","last_unstale":"2024-05-08T05:47:01.843605+0000","last_undegraded":"2024-05-08T05:47:01.843605+0000","last_fullsized":"2024-05-08T05:47:01.843605+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:01:32.853120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.210814+0000","last_change":"2024-05-08T05:47:00.853977+0000","last_active":"2024-05-08T05:47:02.210814+0000","last_peered":"2024-05-08T05:47:02.210814+0000","last_clean":"2024-05-08T05:47:02.210814+0000","last_became_active":"2024-05-08T05:47:00.853690+0000","last_became_peered":"2024-05-08T05:47:00.853690+0000","last_unstale":"2024-05-08T05:47:02.210814+0000","last_undegraded":"2024-05-08T05:47:02.210814+0000","last_fullsized":"2024-05-08T05:47:02.210814+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:36:15.998298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"87'2","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:04.123466+0000","last_change":"2024-05-08T05:46:59.851269+0000","last_active":"2024-05-08T05:47:04.123466+0000","last_peered":"2024-05-08T05:47:04.123466+0000","last_clean":"2024-05-08T05:47:04.123466+0000","last_became_active":"2024-05-08T05:46:59.850897+0000","last_became_peered":"2024-05-08T05:46:59.850897+0000","last_unstale":"2024-05-08T05:47:04.123466+0000","last_undegraded":"2024-05-08T05:47:04.123466+0000","last_fullsized":"2024-05-08T05:47:04.123466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:53:17.698536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.548101+0000","last_change":"2024-05-08T05:47:00.856546+0000","last_active":"2024-05-08T05:47:02.548101+0000","last_peered":"2024-05-08T05:47:02.548101+0000","last_clean":"2024-05-08T05:47:02.548101+0000","last_became_active":"2024-05-08T05:47:00.856283+0000","last_became_peered":"2024-05-08T05:47:00.856283+0000","last_unstale":"2024-05-08T05:47:02.548101+0000","last_undegraded":"2024-05-08T05:47:02.548101+0000","last_fullsized":"2024-05-08T05:47:02.548101+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:45:53.085701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"88'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.848555+0000","last_change":"2024-05-08T05:46:59.847801+0000","last_active":"2024-05-08T05:47:01.848555+0000","last_peered":"2024-05-08T05:47:01.848555+0000","last_clean":"2024-05-08T05:47:01.848555+0000","last_became_active":"2024-05-08T05:46:59.847512+0000","last_became_peered":"2024-05-08T05:46:59.847512+0000","last_unstale":"2024-05-08T05:47:01.848555+0000","last_undegraded":"2024-05-08T05:47:01.848555+0000","last_fullsized":"2024-05-08T05:47:01.848555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:26:10.786217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.547821+0000","last_change":"2024-05-08T05:47:00.845042+0000","last_active":"2024-05-08T05:47:02.547821+0000","last_peered":"2024-05-08T05:47:02.547821+0000","last_clean":"2024-05-08T05:47:02.547821+0000","last_became_active":"2024-05-08T05:47:00.844617+0000","last_became_peered":"2024-05-08T05:47:00.844617+0000","last_unstale":"2024-05-08T05:47:02.547821+0000","last_undegraded":"2024-05-08T05:47:02.547821+0000","last_fullsized":"2024-05-08T05:47:02.547821+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:44:42.120367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.848664+0000","last_change":"2024-05-08T05:46:59.849092+0000","last_active":"2024-05-08T05:47:01.848664+0000","last_peered":"2024-05-08T05:47:01.848664+0000","last_clean":"2024-05-08T05:47:01.848664+0000","last_became_active":"2024-05-08T05:46:59.848479+0000","last_became_peered":"2024-05-08T05:46:59.848479+0000","last_unstale":"2024-05-08T05:47:01.848664+0000","last_undegraded":"2024-05-08T05:47:01.848664+0000","last_fullsized":"2024-05-08T05:47:01.848664+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:00:45.072183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.837814+0000","last_change":"2024-05-08T05:47:00.854175+0000","last_active":"2024-05-08T05:47:02.837814+0000","last_peered":"2024-05-08T05:47:02.837814+0000","last_clean":"2024-05-08T05:47:02.837814+0000","last_became_active":"2024-05-08T05:47:00.853905+0000","last_became_peered":"2024-05-08T05:47:00.853905+0000","last_unstale":"2024-05-08T05:47:02.837814+0000","last_undegraded":"2024-05-08T05:47:02.837814+0000","last_fullsized":"2024-05-08T05:47:02.837814+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:35:57.787946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.210982+0000","last_change":"2024-05-08T05:46:59.840480+0000","last_active":"2024-05-08T05:47:02.210982+0000","last_peered":"2024-05-08T05:47:02.210982+0000","last_clean":"2024-05-08T05:47:02.210982+0000","last_became_active":"2024-05-08T05:46:59.840201+0000","last_became_peered":"2024-05-08T05:46:59.840201+0000","last_unstale":"2024-05-08T05:47:02.210982+0000","last_undegraded":"2024-05-08T05:47:02.210982+0000","last_fullsized":"2024-05-08T05:47:02.210982+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:56:17.117125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.843692+0000","last_change":"2024-05-08T05:47:00.850714+0000","last_active":"2024-05-08T05:47:01.843692+0000","last_peered":"2024-05-08T05:47:01.843692+0000","last_clean":"2024-05-08T05:47:01.843692+0000","last_became_active":"2024-05-08T05:47:00.850422+0000","last_became_peered":"2024-05-08T05:47:00.850422+0000","last_unstale":"2024-05-08T05:47:01.843692+0000","last_undegraded":"2024-05-08T05:47:01.843692+0000","last_fullsized":"2024-05-08T05:47:01.843692+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:01:58.490160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.547836+0000","last_change":"2024-05-08T05:46:59.851633+0000","last_active":"2024-05-08T05:47:02.547836+0000","last_peered":"2024-05-08T05:47:02.547836+0000","last_clean":"2024-05-08T05:47:02.547836+0000","last_became_active":"2024-05-08T05:46:59.851221+0000","last_became_peered":"2024-05-08T05:46:59.851221+0000","last_unstale":"2024-05-08T05:47:02.547836+0000","last_undegraded":"2024-05-08T05:47:02.547836+0000","last_fullsized":"2024-05-08T05:47:02.547836+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:02:52.230765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:01.896312+0000","last_change":"2024-05-08T05:47:00.860574+0000","last_active":"2024-05-08T05:47:01.896312+0000","last_peered":"2024-05-08T05:47:01.896312+0000","last_clean":"2024-05-08T05:47:01.896312+0000","last_became_active":"2024-05-08T05:47:00.860145+0000","last_became_peered":"2024-05-08T05:47:00.860145+0000","last_unstale":"2024-05-08T05:47:01.896312+0000","last_undegraded":"2024-05-08T05:47:01.896312+0000","last_fullsized":"2024-05-08T05:47:01.896312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:20:54.349129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.837530+0000","last_change":"2024-05-08T05:46:59.846154+0000","last_active":"2024-05-08T05:47:02.837530+0000","last_peered":"2024-05-08T05:47:02.837530+0000","last_clean":"2024-05-08T05:47:02.837530+0000","last_became_active":"2024-05-08T05:46:59.845638+0000","last_became_peered":"2024-05-08T05:46:59.845638+0000","last_unstale":"2024-05-08T05:47:02.837530+0000","last_undegraded":"2024-05-08T05:47:02.837530+0000","last_fullsized":"2024-05-08T05:47:02.837530+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:29:43.333484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.210705+0000","last_change":"2024-05-08T05:47:00.863297+0000","last_active":"2024-05-08T05:47:02.210705+0000","last_peered":"2024-05-08T05:47:02.210705+0000","last_clean":"2024-05-08T05:47:02.210705+0000","last_became_active":"2024-05-08T05:47:00.863022+0000","last_became_peered":"2024-05-08T05:47:00.863022+0000","last_unstale":"2024-05-08T05:47:02.210705+0000","last_undegraded":"2024-05-08T05:47:02.210705+0000","last_fullsized":"2024-05-08T05:47:02.210705+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:16:29.634523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.210941+0000","last_change":"2024-05-08T05:46:59.839824+0000","last_active":"2024-05-08T05:47:02.210941+0000","last_peered":"2024-05-08T05:47:02.210941+0000","last_clean":"2024-05-08T05:47:02.210941+0000","last_became_active":"2024-05-08T05:46:59.839565+0000","last_became_peered":"2024-05-08T05:46:59.839565+0000","last_unstale":"2024-05-08T05:47:02.210941+0000","last_undegraded":"2024-05-08T05:47:02.210941+0000","last_fullsized":"2024-05-08T05:47:02.210941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:20:35.012844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.547763+0000","last_change":"2024-05-08T05:47:00.853420+0000","last_active":"2024-05-08T05:47:02.547763+0000","last_peered":"2024-05-08T05:47:02.547763+0000","last_clean":"2024-05-08T05:47:02.547763+0000","last_became_active":"2024-05-08T05:47:00.853088+0000","last_became_peered":"2024-05-08T05:47:00.853088+0000","last_unstale":"2024-05-08T05:47:02.547763+0000","last_undegraded":"2024-05-08T05:47:02.547763+0000","last_fullsized":"2024-05-08T05:47:02.547763+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:50:11.792836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:04.123080+0000","last_change":"2024-05-08T05:46:59.853235+0000","last_active":"2024-05-08T05:47:04.123080+0000","last_peered":"2024-05-08T05:47:04.123080+0000","last_clean":"2024-05-08T05:47:04.123080+0000","last_became_active":"2024-05-08T05:46:59.852924+0000","last_became_peered":"2024-05-08T05:46:59.852924+0000","last_unstale":"2024-05-08T05:47:04.123080+0000","last_undegraded":"2024-05-08T05:47:04.123080+0000","last_fullsized":"2024-05-08T05:47:04.123080+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:52.162806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:04.123460+0000","last_change":"2024-05-08T05:47:00.850240+0000","last_active":"2024-05-08T05:47:04.123460+0000","last_peered":"2024-05-08T05:47:04.123460+0000","last_clean":"2024-05-08T05:47:04.123460+0000","last_became_active":"2024-05-08T05:47:00.849936+0000","last_became_peered":"2024-05-08T05:47:00.849936+0000","last_unstale":"2024-05-08T05:47:04.123460+0000","last_undegraded":"2024-05-08T05:47:04.123460+0000","last_fullsized":"2024-05-08T05:47:04.123460+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:59.822606+0000","last_clean_scrub_stamp":"2024-05-08T05:46:59.822606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:27.755868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:02.191321+0000","last_change":"2024-05-08T05:46:59.848367+0000","last_active":"2024-05-08T05:47:02.191321+0000","last_peered":"2024-05-08T05:47:02.191321+0000","last_clean":"2024-05-08T05:47:02.191321+0000","last_became_active":"2024-05-08T05:46:59.847967+0000","last_became_peered":"2024-05-08T05:46:59.847967+0000","last_unstale":"2024-05-08T05:47:02.191321+0000","last_undegraded":"2024-05-08T05:47:02.191321+0000","last_fullsized":"2024-05-08T05:47:02.191321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:46:58.820463+0000","last_clean_scrub_stamp":"2024-05-08T05:46:58.820463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:15:43.739737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":234,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:47:04.123270+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:47:04.123270+0000","last_peered":"2024-05-08T05:47:04.123270+0000","last_clean":"2024-05-08T05:47:04.123270+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:47:04.123270+0000","last_undegraded":"2024-05-08T05:47:04.123270+0000","last_fullsized":"2024-05-08T05:47:04.123270+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542217,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32824,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339016,"statfs":{"total":96636764160,"available":96603152384,"internally_reserved":0,"allocated":1683456,"data_stored":1085550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542218,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1101824,"data_stored":497342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542217,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1081344,"data_stored":495126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28092,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343748,"statfs":{"total":96636764160,"available":96607997952,"internally_reserved":0,"allocated":1089536,"data_stored":495286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574919,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1081344,"data_stored":495126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574919,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1081344,"data_stored":495126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32248,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339592,"statfs":{"total":96636764160,"available":96603742208,"internally_reserved":0,"allocated":1085440,"data_stored":495196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32840,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339000,"statfs":{"total":96636764160,"available":96603136000,"internally_reserved":0,"allocated":1699840,"data_stored":1087812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:47:05.263 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T05:47:05.263 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T05:47:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.410+0000 7f36f7ee3640 1 Processor -- start 2024-05-08T05:47:05.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.410+0000 7f36f7ee3640 1 -- start start 2024-05-08T05:47:05.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.411+0000 7f36f7ee3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f36f00d2130 0x7f36f01639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.411+0000 7f36f7ee3640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f36f0071110 0x7f36f006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.411+0000 7f36f7ee3640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36f0160720 con 0x7f36f0070d30 2024-05-08T05:47:05.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.411+0000 7f36f7ee3640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f36f00d1ca0 con 0x7f36f0071110 2024-05-08T05:47:05.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.411+0000 7f36f7ee3640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f36f00d10d0 con 0x7f36f00d2130 2024-05-08T05:47:05.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.411+0000 7f36f6480640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f36f0071110 0x7f36f006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:05.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.411+0000 7f36f6480640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f36f0071110 0x7f36f006d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43362/0 (socket says 172.21.15.45:43362) 2024-05-08T05:47:05.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.411+0000 7f36f6480640 1 -- 172.21.15.45:0/464867875 learned_addr learned my addr 172.21.15.45:0/464867875 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:05.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.411+0000 7f36f547e640 1 --2- 172.21.15.45:0/464867875 >> v2:172.21.15.63:3300/0 conn(0x7f36f00d2130 0x7f36f01639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:05.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.411+0000 7f36f6480640 1 -- 172.21.15.45:0/464867875 >> v2:172.21.15.63:3300/0 conn(0x7f36f00d2130 msgr2=0x7f36f01639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:05.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.411+0000 7f36f6480640 1 --2- 172.21.15.45:0/464867875 >> v2:172.21.15.63:3300/0 conn(0x7f36f00d2130 0x7f36f01639e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.411+0000 7f36f6480640 1 -- 172.21.15.45:0/464867875 >> v1:172.21.15.63:6789/0 conn(0x7f36f0070d30 legacy=0x7f36f00703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:05.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.411+0000 7f36f6480640 1 -- 172.21.15.45:0/464867875 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f00d1370 con 0x7f36f0071110 2024-05-08T05:47:05.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.412+0000 7f36f6480640 1 --2- 172.21.15.45:0/464867875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36f0071110 0x7f36f006d4b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f36f0071e70 tx=0x7f36e802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3436b7e497ebce68 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:05.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.412+0000 7f36f4c7d640 1 -- 172.21.15.45:0/464867875 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8036070 con 0x7f36f0071110 2024-05-08T05:47:05.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.412+0000 7f36f4c7d640 1 -- 172.21.15.45:0/464867875 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36e8031600 con 0x7f36f0071110 2024-05-08T05:47:05.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.412+0000 7f36f4c7d640 1 -- 172.21.15.45:0/464867875 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e80319d0 con 0x7f36f0071110 2024-05-08T05:47:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.413+0000 7f36f7ee3640 1 -- 172.21.15.45:0/464867875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36f0071110 msgr2=0x7f36f006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.413+0000 7f36f7ee3640 1 --2- 172.21.15.45:0/464867875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36f0071110 0x7f36f006d4b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f36f0071e70 tx=0x7f36e802a6a0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.413+0000 7f36f7ee3640 1 -- 172.21.15.45:0/464867875 shutdown_connections 2024-05-08T05:47:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.413+0000 7f36f7ee3640 1 --2- 172.21.15.45:0/464867875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36f0071110 0x7f36f006d4b0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.413+0000 7f36f7ee3640 1 --2- 172.21.15.45:0/464867875 >> v2:172.21.15.63:3300/0 conn(0x7f36f00d2130 0x7f36f01639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.413+0000 7f36f7ee3640 1 -- 172.21.15.45:0/464867875 >> 172.21.15.45:0/464867875 conn(0x7f36f00d2e90 msgr2=0x7f36f00d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.413+0000 7f36f7ee3640 1 -- 172.21.15.45:0/464867875 shutdown_connections 2024-05-08T05:47:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.413+0000 7f36f7ee3640 1 -- 172.21.15.45:0/464867875 wait complete. 2024-05-08T05:47:05.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.414+0000 7f36f7ee3640 1 Processor -- start 2024-05-08T05:47:05.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.414+0000 7f36f7ee3640 1 -- start start 2024-05-08T05:47:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.415+0000 7f36f7ee3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36f0070d30 0x7f36f01f9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.415+0000 7f36f7ee3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36f0071110 0x7f36f00ce510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.415+0000 7f36f7ee3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36f00d2130 0x7f36f00caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.415+0000 7f36f7ee3640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f36f0071540 con 0x7f36f0071110 2024-05-08T05:47:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.415+0000 7f36f7ee3640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f36f00d1930 con 0x7f36f0070d30 2024-05-08T05:47:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.415+0000 7f36f7ee3640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f36f00716c0 con 0x7f36f00d2130 2024-05-08T05:47:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.415+0000 7f36f547e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36f0071110 0x7f36f00ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.415+0000 7f36f547e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36f0071110 0x7f36f00ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43378/0 (socket says 172.21.15.45:43378) 2024-05-08T05:47:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.415+0000 7f36f547e640 1 -- 172.21.15.45:0/3628106681 learned_addr learned my addr 172.21.15.45:0/3628106681 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.415+0000 7f36f5c7f640 1 --2- 172.21.15.45:0/3628106681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36f0070d30 0x7f36f01f9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.415+0000 7f36f6480640 1 --2- 172.21.15.45:0/3628106681 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36f00d2130 0x7f36f00caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.415+0000 7f36f547e640 1 -- 172.21.15.45:0/3628106681 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36f00d2130 msgr2=0x7f36f00caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.415+0000 7f36f547e640 1 --2- 172.21.15.45:0/3628106681 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36f00d2130 0x7f36f00caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.415+0000 7f36f547e640 1 -- 172.21.15.45:0/3628106681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36f0070d30 msgr2=0x7f36f01f9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:05.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.415+0000 7f36f547e640 1 --2- 172.21.15.45:0/3628106681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36f0070d30 0x7f36f01f9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.415+0000 7f36f547e640 1 -- 172.21.15.45:0/3628106681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f00c9f60 con 0x7f36f0071110 2024-05-08T05:47:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.415+0000 7f36f5c7f640 1 --2- 172.21.15.45:0/3628106681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36f0070d30 0x7f36f01f9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:47:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.415+0000 7f36f547e640 1 --2- 172.21.15.45:0/3628106681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36f0071110 0x7f36f00ce510 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f36e000c7f0 tx=0x7f36e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.415+0000 7f36e67fc640 1 -- 172.21.15.45:0/3628106681 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0015070 con 0x7f36f0071110 2024-05-08T05:47:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.415+0000 7f36e67fc640 1 -- 172.21.15.45:0/3628106681 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36e0004240 con 0x7f36f0071110 2024-05-08T05:47:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.416+0000 7f36f7ee3640 1 -- 172.21.15.45:0/3628106681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f00cec50 con 0x7f36f0071110 2024-05-08T05:47:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.416+0000 7f36e67fc640 1 -- 172.21.15.45:0/3628106681 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0007620 con 0x7f36f0071110 2024-05-08T05:47:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.416+0000 7f36f7ee3640 1 -- 172.21.15.45:0/3628106681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f00ca200 con 0x7f36f0071110 2024-05-08T05:47:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.417+0000 7f36f7ee3640 1 -- 172.21.15.45:0/3628106681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f00693f0 con 0x7f36f0071110 2024-05-08T05:47:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.417+0000 7f36e67fc640 1 -- 172.21.15.45:0/3628106681 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f36e00079d0 con 0x7f36f0071110 2024-05-08T05:47:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.418+0000 7f36e67fc640 1 --2- 172.21.15.45:0/3628106681 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f36d40762d0 0x7f36d4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.418+0000 7f36e67fc640 1 -- 172.21.15.45:0/3628106681 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6101+0+0 (secure 0 0 0) 0x7f36e0097b20 con 0x7f36f0071110 2024-05-08T05:47:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.418+0000 7f36f5c7f640 1 --2- 172.21.15.45:0/3628106681 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f36d40762d0 0x7f36d4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.418+0000 7f36f5c7f640 1 --2- 172.21.15.45:0/3628106681 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f36d40762d0 0x7f36d4078790 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f36ec00a810 tx=0x7f36ec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:05.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.422+0000 7f36e67fc640 1 -- 172.21.15.45:0/3628106681 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f36e0061380 con 0x7f36f0071110 2024-05-08T05:47:05.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.689+0000 7f36f7ee3640 1 -- 172.21.15.45:0/3628106681 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f36f00d3d70 con 0x7f36f0071110 2024-05-08T05:47:05.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.690+0000 7f36e67fc640 1 -- 172.21.15.45:0/3628106681 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f36e0069370 con 0x7f36f0071110 2024-05-08T05:47:05.691 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-05-08T05:47:05.691 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBUEDtmDcqGMxAALgQaBYp4Zd8uMqswCXvGaw== 2024-05-08T05:47:05.691 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2024-05-08T05:47:05.691 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2024-05-08T05:47:05.691 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-08T05:47:05.691 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2024-05-08T05:47:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.693+0000 7f36f7ee3640 1 -- 172.21.15.45:0/3628106681 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f36d40762d0 msgr2=0x7f36d4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.693+0000 7f36f7ee3640 1 --2- 172.21.15.45:0/3628106681 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f36d40762d0 0x7f36d4078790 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f36ec00a810 tx=0x7f36ec008040 comp rx=0 tx=0).stop 2024-05-08T05:47:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.693+0000 7f36f7ee3640 1 -- 172.21.15.45:0/3628106681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36f0071110 msgr2=0x7f36f00ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.693+0000 7f36f7ee3640 1 --2- 172.21.15.45:0/3628106681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36f0071110 0x7f36f00ce510 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f36e000c7f0 tx=0x7f36e000ccb0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.693+0000 7f36f7ee3640 1 -- 172.21.15.45:0/3628106681 shutdown_connections 2024-05-08T05:47:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.693+0000 7f36f7ee3640 1 --2- 172.21.15.45:0/3628106681 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f36d40762d0 0x7f36d4078790 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.693+0000 7f36f7ee3640 1 --2- 172.21.15.45:0/3628106681 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36f00d2130 0x7f36f00caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.693+0000 7f36f7ee3640 1 --2- 172.21.15.45:0/3628106681 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36f0071110 0x7f36f00ce510 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.693+0000 7f36f7ee3640 1 --2- 172.21.15.45:0/3628106681 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36f0070d30 0x7f36f01f9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.693+0000 7f36f7ee3640 1 -- 172.21.15.45:0/3628106681 >> 172.21.15.45:0/3628106681 conn(0x7f36f00d2e90 msgr2=0x7f36f00b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.693+0000 7f36f7ee3640 1 -- 172.21.15.45:0/3628106681 shutdown_connections 2024-05-08T05:47:05.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.693+0000 7f36f7ee3640 1 -- 172.21.15.45:0/3628106681 wait complete. 2024-05-08T05:47:05.704 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:47:05.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.807+0000 7f9d36514640 1 Processor -- start 2024-05-08T05:47:05.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.807+0000 7f9d36514640 1 -- start start 2024-05-08T05:47:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.808+0000 7f9d36514640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9d3015f0c0 0x7f9d300b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.808+0000 7f9d36514640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9d301638e0 0x7f9d300b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.808+0000 7f9d36514640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d300b1260 con 0x7f9d3006c3a0 2024-05-08T05:47:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.808+0000 7f9d36514640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f9d30163450 con 0x7f9d3015f0c0 2024-05-08T05:47:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.808+0000 7f9d36514640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9d301604b0 con 0x7f9d301638e0 2024-05-08T05:47:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.808+0000 7f9d2ffff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9d3015f0c0 0x7f9d300b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:05.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.808+0000 7f9d34ab1640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9d3006c3a0 0x7f9d30073170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36828/0 (socket says 172.21.15.45:36828) 2024-05-08T05:47:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.808+0000 7f9d34ab1640 1 -- 172.21.15.45:0/4182598812 learned_addr learned my addr 172.21.15.45:0/4182598812 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:47:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.808+0000 7f9d2f7fe640 1 --2- 172.21.15.45:0/4182598812 >> v2:172.21.15.63:3300/0 conn(0x7f9d301638e0 0x7f9d300b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.809+0000 7f9d2ffff640 1 -- 172.21.15.45:0/4182598812 >> v2:172.21.15.63:3300/0 conn(0x7f9d301638e0 msgr2=0x7f9d300b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.809+0000 7f9d2ffff640 1 --2- 172.21.15.45:0/4182598812 >> v2:172.21.15.63:3300/0 conn(0x7f9d301638e0 0x7f9d300b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.809+0000 7f9d2ffff640 1 -- 172.21.15.45:0/4182598812 >> v1:172.21.15.45:6789/0 conn(0x7f9d3006c3a0 legacy=0x7f9d30073170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.809+0000 7f9d2ffff640 1 -- 172.21.15.45:0/4182598812 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d300b14e0 con 0x7f9d3015f0c0 2024-05-08T05:47:05.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.809+0000 7f9d2ffff640 1 --2- 172.21.15.45:0/4182598812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d3015f0c0 0x7f9d300b6f40 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f9d3006ea70 tx=0x7f9d2002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3360f80d78f75467 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.809+0000 7f9d2effd640 1 -- 172.21.15.45:0/4182598812 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20037070 con 0x7f9d3015f0c0 2024-05-08T05:47:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.809+0000 7f9d2effd640 1 -- 172.21.15.45:0/4182598812 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d20032440 con 0x7f9d3015f0c0 2024-05-08T05:47:05.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.809+0000 7f9d2effd640 1 -- 172.21.15.45:0/4182598812 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20032810 con 0x7f9d3015f0c0 2024-05-08T05:47:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.810+0000 7f9d36514640 1 -- 172.21.15.45:0/4182598812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d3015f0c0 msgr2=0x7f9d300b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.810+0000 7f9d36514640 1 --2- 172.21.15.45:0/4182598812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d3015f0c0 0x7f9d300b6f40 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f9d3006ea70 tx=0x7f9d2002a1a0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.810+0000 7f9d36514640 1 -- 172.21.15.45:0/4182598812 shutdown_connections 2024-05-08T05:47:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.810+0000 7f9d36514640 1 --2- 172.21.15.45:0/4182598812 >> v2:172.21.15.63:3300/0 conn(0x7f9d301638e0 0x7f9d300b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.810+0000 7f9d36514640 1 --2- 172.21.15.45:0/4182598812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d3015f0c0 0x7f9d300b6f40 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.810+0000 7f9d36514640 1 -- 172.21.15.45:0/4182598812 >> 172.21.15.45:0/4182598812 conn(0x7f9d300b3b10 msgr2=0x7f9d300b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.810+0000 7f9d36514640 1 -- 172.21.15.45:0/4182598812 shutdown_connections 2024-05-08T05:47:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.810+0000 7f9d36514640 1 -- 172.21.15.45:0/4182598812 wait complete. 2024-05-08T05:47:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.811+0000 7f9d36514640 1 Processor -- start 2024-05-08T05:47:05.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.811+0000 7f9d36514640 1 -- start start 2024-05-08T05:47:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.812+0000 7f9d36514640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d3006c3a0 0x7f9d300b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.812+0000 7f9d36514640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d3015f0c0 0x7f9d30156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.812+0000 7f9d36514640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d301638e0 0x7f9d30158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.812+0000 7f9d36514640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9d300b17c0 con 0x7f9d301638e0 2024-05-08T05:47:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.812+0000 7f9d36514640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9d301630e0 con 0x7f9d3006c3a0 2024-05-08T05:47:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.812+0000 7f9d36514640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9d3006c7e0 con 0x7f9d3015f0c0 2024-05-08T05:47:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.812+0000 7f9d34ab1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d301638e0 0x7f9d30158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.812+0000 7f9d34ab1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d301638e0 0x7f9d30158830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43396/0 (socket says 172.21.15.45:43396) 2024-05-08T05:47:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.812+0000 7f9d34ab1640 1 -- 172.21.15.45:0/1427187481 learned_addr learned my addr 172.21.15.45:0/1427187481 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.813+0000 7f9d2ffff640 1 --2- 172.21.15.45:0/1427187481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d3006c3a0 0x7f9d300b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.813+0000 7f9d2f7fe640 1 --2- 172.21.15.45:0/1427187481 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d3015f0c0 0x7f9d30156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.813+0000 7f9d34ab1640 1 -- 172.21.15.45:0/1427187481 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d3015f0c0 msgr2=0x7f9d30156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:05.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.813+0000 7f9d34ab1640 1 --2- 172.21.15.45:0/1427187481 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d3015f0c0 0x7f9d30156d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.813+0000 7f9d34ab1640 1 -- 172.21.15.45:0/1427187481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d3006c3a0 msgr2=0x7f9d300b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.813+0000 7f9d34ab1640 1 --2- 172.21.15.45:0/1427187481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d3006c3a0 0x7f9d300b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.813+0000 7f9d34ab1640 1 -- 172.21.15.45:0/1427187481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d30154880 con 0x7f9d301638e0 2024-05-08T05:47:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.813+0000 7f9d2ffff640 1 --2- 172.21.15.45:0/1427187481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d3006c3a0 0x7f9d300b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:47:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.813+0000 7f9d34ab1640 1 --2- 172.21.15.45:0/1427187481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d301638e0 0x7f9d30158830 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800cb80 tx=0x7f9d2800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.813+0000 7f9d2cff9640 1 -- 172.21.15.45:0/1427187481 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d28017070 con 0x7f9d301638e0 2024-05-08T05:47:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.813+0000 7f9d2cff9640 1 -- 172.21.15.45:0/1427187481 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d2800ad40 con 0x7f9d301638e0 2024-05-08T05:47:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.813+0000 7f9d2cff9640 1 -- 172.21.15.45:0/1427187481 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d28012600 con 0x7f9d301638e0 2024-05-08T05:47:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.813+0000 7f9d36514640 1 -- 172.21.15.45:0/1427187481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d30154b60 con 0x7f9d301638e0 2024-05-08T05:47:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.813+0000 7f9d36514640 1 -- 172.21.15.45:0/1427187481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d30155120 con 0x7f9d301638e0 2024-05-08T05:47:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.814+0000 7f9d36514640 1 -- 172.21.15.45:0/1427187481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d300693f0 con 0x7f9d301638e0 2024-05-08T05:47:05.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.815+0000 7f9d2cff9640 1 -- 172.21.15.45:0/1427187481 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9d2801a070 con 0x7f9d301638e0 2024-05-08T05:47:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.815+0000 7f9d2cff9640 1 --2- 172.21.15.45:0/1427187481 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9d1c0762d0 0x7f9d1c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.815+0000 7f9d2cff9640 1 -- 172.21.15.45:0/1427187481 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6101+0+0 (secure 0 0 0) 0x7f9d28097410 con 0x7f9d301638e0 2024-05-08T05:47:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.816+0000 7f9d2ffff640 1 --2- 172.21.15.45:0/1427187481 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9d1c0762d0 0x7f9d1c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:05.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.816+0000 7f9d2ffff640 1 --2- 172.21.15.45:0/1427187481 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9d1c0762d0 0x7f9d1c078790 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9d20034000 tx=0x7f9d2002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:05.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:05.820+0000 7f9d2cff9640 1 -- 172.21.15.45:0/1427187481 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9d300b1260 con 0x7f9d301638e0 2024-05-08T05:47:06.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.051+0000 7f9d36514640 1 -- 172.21.15.45:0/1427187481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d3015ecf0 con 0x7f9d301638e0 2024-05-08T05:47:06.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.052+0000 7f9d2cff9640 1 -- 172.21.15.45:0/1427187481 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5183 (secure 0 0 0) 0x7f9d28060620 con 0x7f9d301638e0 2024-05-08T05:47:06.052 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:47:06.052 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6744,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2854804427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2854804427},{"type":"v1","addr":"172.21.15.45:6834","nonce":2854804427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2945763952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2945763952},{"type":"v1","addr":"172.21.15.45:6835","nonce":2945763952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6759,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3194497846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":3194497846},{"type":"v1","addr":"172.21.15.63:6837","nonce":3194497846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:47:00.211837+0000","modified":"2024-05-08T05:47:03.229466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6762":{"gid":6762,"name":"d","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6836/2942326473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2942326473},{"type":"v1","addr":"172.21.15.63:6836","nonce":2942326473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6762,"qdb_cluster":[6762]},"id":6}]} 2024-05-08T05:47:06.052 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 60 2024-05-08T05:47:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.053+0000 7f9d36514640 1 -- 172.21.15.45:0/1427187481 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9d1c0762d0 msgr2=0x7f9d1c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.053+0000 7f9d36514640 1 --2- 172.21.15.45:0/1427187481 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9d1c0762d0 0x7f9d1c078790 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9d20034000 tx=0x7f9d2002e000 comp rx=0 tx=0).stop 2024-05-08T05:47:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.054+0000 7f9d36514640 1 -- 172.21.15.45:0/1427187481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d301638e0 msgr2=0x7f9d30158830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.054+0000 7f9d36514640 1 --2- 172.21.15.45:0/1427187481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d301638e0 0x7f9d30158830 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800cb80 tx=0x7f9d2800a430 comp rx=0 tx=0).stop 2024-05-08T05:47:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.054+0000 7f9d36514640 1 -- 172.21.15.45:0/1427187481 shutdown_connections 2024-05-08T05:47:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.054+0000 7f9d36514640 1 --2- 172.21.15.45:0/1427187481 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9d1c0762d0 0x7f9d1c078790 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.054+0000 7f9d36514640 1 --2- 172.21.15.45:0/1427187481 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9d3015f0c0 0x7f9d30156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.054+0000 7f9d36514640 1 --2- 172.21.15.45:0/1427187481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d301638e0 0x7f9d30158830 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.054+0000 7f9d36514640 1 --2- 172.21.15.45:0/1427187481 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9d3006c3a0 0x7f9d300b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.054+0000 7f9d36514640 1 -- 172.21.15.45:0/1427187481 >> 172.21.15.45:0/1427187481 conn(0x7f9d300b3b10 msgr2=0x7f9d30160910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:06.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.054+0000 7f9d36514640 1 -- 172.21.15.45:0/1427187481 shutdown_connections 2024-05-08T05:47:06.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.054+0000 7f9d36514640 1 -- 172.21.15.45:0/1427187481 wait complete. 2024-05-08T05:47:06.467 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:47:00.211837+0000', 'modified': '2024-05-08T05:47:03.229466+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6762}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6762': {'gid': 6762, 'name': 'd', 'rank': 0, 'incarnation': 56, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.63:6836/2942326473', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6834', 'nonce': 2942326473}, {'type': 'v1', 'addr': '172.21.15.63:6836', 'nonce': 2942326473}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6762, 'qdb_cluster': [6762]} 2024-05-08T05:47:06.468 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T05:47:06.468 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-08T05:47:06.475 INFO:tasks.ceph.mds.d.smithi063.stderr:2024-05-08T05:47:06.475+0000 7fb4b3348640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:47:06.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.579+0000 7f15ac4a0640 1 Processor -- start 2024-05-08T05:47:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.579+0000 7f15ac4a0640 1 -- start start 2024-05-08T05:47:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.580+0000 7f15ac4a0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15a415f380 0x7f15a40d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.580+0000 7f15ac4a0640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f15a40d0cf0 0x7f15a40d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.580+0000 7f15ac4a0640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15a40d1760 con 0x7f15a40d3b00 2024-05-08T05:47:06.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.580+0000 7f15ac4a0640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f15a40d30f0 con 0x7f15a40d0cf0 2024-05-08T05:47:06.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.580+0000 7f15ac4a0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f15a40d1150 con 0x7f15a415f380 2024-05-08T05:47:06.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.580+0000 7f15a9a3b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f15a40d0cf0 0x7f15a40d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.580+0000 7f15aa23c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15a415f380 0x7f15a40d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.580+0000 7f15aaa3d640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f15a40d3b00 0x7f15a4162e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:53298/0 (socket says 172.21.15.45:53298) 2024-05-08T05:47:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.581+0000 7f15aaa3d640 1 -- 172.21.15.45:0/2948144172 learned_addr learned my addr 172.21.15.45:0/2948144172 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:47:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.581+0000 7f15aa23c640 1 -- 172.21.15.45:0/2948144172 >> v2:172.21.15.63:3300/0 conn(0x7f15a40d0cf0 msgr2=0x7f15a40d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.581+0000 7f15aa23c640 1 --2- 172.21.15.45:0/2948144172 >> v2:172.21.15.63:3300/0 conn(0x7f15a40d0cf0 0x7f15a40d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.581+0000 7f15aa23c640 1 -- 172.21.15.45:0/2948144172 >> v1:172.21.15.63:6789/0 conn(0x7f15a40d3b00 legacy=0x7f15a4162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.581+0000 7f15aa23c640 1 -- 172.21.15.45:0/2948144172 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a40d1a40 con 0x7f15a415f380 2024-05-08T05:47:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.581+0000 7f15a9a3b640 1 --2- 172.21.15.45:0/2948144172 >> v2:172.21.15.63:3300/0 conn(0x7f15a40d0cf0 0x7f15a40d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:47:06.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.582+0000 7f15aa23c640 1 --2- 172.21.15.45:0/2948144172 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15a415f380 0x7f15a40d07b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f159c0097a0 tx=0x7f159c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2291ddc6a2d2ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.582+0000 7f15a923a640 1 -- 172.21.15.45:0/2948144172 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c037070 con 0x7f15a415f380 2024-05-08T05:47:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.582+0000 7f15a923a640 1 -- 172.21.15.45:0/2948144172 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f159c032440 con 0x7f15a415f380 2024-05-08T05:47:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.582+0000 7f15a923a640 1 -- 172.21.15.45:0/2948144172 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c032810 con 0x7f15a415f380 2024-05-08T05:47:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.582+0000 7f15ac4a0640 1 -- 172.21.15.45:0/2948144172 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15a415f380 msgr2=0x7f15a40d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.582+0000 7f15ac4a0640 1 --2- 172.21.15.45:0/2948144172 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15a415f380 0x7f15a40d07b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f159c0097a0 tx=0x7f159c02a180 comp rx=0 tx=0).stop 2024-05-08T05:47:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.583+0000 7f15ac4a0640 1 -- 172.21.15.45:0/2948144172 shutdown_connections 2024-05-08T05:47:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.583+0000 7f15ac4a0640 1 --2- 172.21.15.45:0/2948144172 >> v2:172.21.15.63:3300/0 conn(0x7f15a40d0cf0 0x7f15a40d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.583+0000 7f15ac4a0640 1 --2- 172.21.15.45:0/2948144172 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15a415f380 0x7f15a40d07b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.583+0000 7f15ac4a0640 1 -- 172.21.15.45:0/2948144172 >> 172.21.15.45:0/2948144172 conn(0x7f15a40d4fe0 msgr2=0x7f15a40d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.583+0000 7f15ac4a0640 1 -- 172.21.15.45:0/2948144172 shutdown_connections 2024-05-08T05:47:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.583+0000 7f15ac4a0640 1 -- 172.21.15.45:0/2948144172 wait complete. 2024-05-08T05:47:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.584+0000 7f15ac4a0640 1 Processor -- start 2024-05-08T05:47:06.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.584+0000 7f15ac4a0640 1 -- start start 2024-05-08T05:47:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.585+0000 7f15ac4a0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15a40d0cf0 0x7f15a40a2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.585+0000 7f15ac4a0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15a40d3b00 0x7f15a415a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.585+0000 7f15ac4a0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15a415f380 0x7f15a409fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.585+0000 7f15ac4a0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f15a40b8e40 con 0x7f15a40d3b00 2024-05-08T05:47:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.585+0000 7f15ac4a0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f15a40d2370 con 0x7f15a415f380 2024-05-08T05:47:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.585+0000 7f15ac4a0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f15a40d2000 con 0x7f15a40d0cf0 2024-05-08T05:47:06.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.585+0000 7f15a9a3b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15a40d3b00 0x7f15a415a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.585+0000 7f15a9a3b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15a40d3b00 0x7f15a415a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43406/0 (socket says 172.21.15.45:43406) 2024-05-08T05:47:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.585+0000 7f15a9a3b640 1 -- 172.21.15.45:0/1872142537 learned_addr learned my addr 172.21.15.45:0/1872142537 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.585+0000 7f15aa23c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15a40d0cf0 0x7f15a40a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.585+0000 7f15aaa3d640 1 --2- 172.21.15.45:0/1872142537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15a415f380 0x7f15a409fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.585+0000 7f15a9a3b640 1 -- 172.21.15.45:0/1872142537 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15a40d0cf0 msgr2=0x7f15a40a2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.585+0000 7f15a9a3b640 1 --2- 172.21.15.45:0/1872142537 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15a40d0cf0 0x7f15a40a2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.585+0000 7f15a9a3b640 1 -- 172.21.15.45:0/1872142537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15a415f380 msgr2=0x7f15a409fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.585+0000 7f15a9a3b640 1 --2- 172.21.15.45:0/1872142537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15a415f380 0x7f15a409fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.585+0000 7f15a9a3b640 1 -- 172.21.15.45:0/1872142537 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a40a2770 con 0x7f15a40d3b00 2024-05-08T05:47:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.585+0000 7f15aa23c640 1 --2- 172.21.15.45:0/1872142537 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15a40d0cf0 0x7f15a40a2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:47:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.585+0000 7f15a9a3b640 1 --2- 172.21.15.45:0/1872142537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15a40d3b00 0x7f15a415a5d0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f159800e990 tx=0x7f159800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.585+0000 7f15a2ffd640 1 -- 172.21.15.45:0/1872142537 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598017070 con 0x7f15a40d3b00 2024-05-08T05:47:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.586+0000 7f15a2ffd640 1 -- 172.21.15.45:0/1872142537 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f159800c9e0 con 0x7f15a40d3b00 2024-05-08T05:47:06.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.586+0000 7f15a2ffd640 1 -- 172.21.15.45:0/1872142537 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159800cdb0 con 0x7f15a40d3b00 2024-05-08T05:47:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.586+0000 7f15ac4a0640 1 -- 172.21.15.45:0/1872142537 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a40a2a50 con 0x7f15a40d3b00 2024-05-08T05:47:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.586+0000 7f15ac4a0640 1 -- 172.21.15.45:0/1872142537 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a415d260 con 0x7f15a40d3b00 2024-05-08T05:47:06.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.587+0000 7f15a2ffd640 1 -- 172.21.15.45:0/1872142537 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f159801a070 con 0x7f15a40d3b00 2024-05-08T05:47:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.588+0000 7f15a2ffd640 1 --2- 172.21.15.45:0/1872142537 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f15900762d0 0x7f1590078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.588+0000 7f15a2ffd640 1 -- 172.21.15.45:0/1872142537 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6101+0+0 (secure 0 0 0) 0x7f15980978c0 con 0x7f15a40d3b00 2024-05-08T05:47:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.588+0000 7f15aa23c640 1 --2- 172.21.15.45:0/1872142537 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f15900762d0 0x7f1590078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.589+0000 7f15af1c2740 1 Processor -- start 2024-05-08T05:47:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.589+0000 7f15aa23c640 1 --2- 172.21.15.45:0/1872142537 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f15900762d0 0x7f1590078790 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f159c034000 tx=0x7f159c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.589+0000 7f15af1c2740 1 -- start start 2024-05-08T05:47:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.589+0000 7f15af1c2740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c137ff1f50 0x55c137ff4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.590+0000 7f15af1c2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 0x55c137ff4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.590+0000 7f15af1c2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137fc4bb0 0x55c137fc83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.590+0000 7f15af1c2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c137f2e8d0 con 0x55c137ff2330 2024-05-08T05:47:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.590+0000 7f15af1c2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c137ee7c10 con 0x55c137fc4bb0 2024-05-08T05:47:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.590+0000 7f15af1c2740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55c137f2a890 con 0x55c137ff1f50 2024-05-08T05:47:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.590+0000 7f15a9a3b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 0x55c137ff4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.590+0000 7f15a9a3b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 0x55c137ff4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43422/0 (socket says 172.21.15.45:43422) 2024-05-08T05:47:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.590+0000 7f15a9a3b640 1 -- 172.21.15.45:0/4068550199 learned_addr learned my addr 172.21.15.45:0/4068550199 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.590+0000 7f15aaa3d640 1 --2- 172.21.15.45:0/4068550199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137fc4bb0 0x55c137fc83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.590+0000 7f15aa23c640 1 --2- 172.21.15.45:0/4068550199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c137ff1f50 0x55c137ff4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.590+0000 7f15a9a3b640 1 -- 172.21.15.45:0/4068550199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c137ff1f50 msgr2=0x55c137ff4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.590+0000 7f15a9a3b640 1 --2- 172.21.15.45:0/4068550199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c137ff1f50 0x55c137ff4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.590+0000 7f15a9a3b640 1 -- 172.21.15.45:0/4068550199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137fc4bb0 msgr2=0x55c137fc83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.590+0000 7f15a9a3b640 1 --2- 172.21.15.45:0/4068550199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137fc4bb0 0x55c137fc83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.590+0000 7f15a9a3b640 1 -- 172.21.15.45:0/4068550199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c137eef5d0 con 0x55c137ff2330 2024-05-08T05:47:06.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.590+0000 7f15a9a3b640 1 --2- 172.21.15.45:0/4068550199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 0x55c137ff4ca0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f159805cb50 tx=0x7f15980605e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87385c66ed95c105 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.590+0000 7f158bfff640 1 -- 172.21.15.45:0/4068550199 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598021070 con 0x55c137ff2330 2024-05-08T05:47:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.590+0000 7f158bfff640 1 -- 172.21.15.45:0/4068550199 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1598069140 con 0x55c137ff2330 2024-05-08T05:47:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.590+0000 7f158bfff640 1 -- 172.21.15.45:0/4068550199 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598064dd0 con 0x55c137ff2330 2024-05-08T05:47:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.591+0000 7f15af1c2740 1 -- 172.21.15.45:0/4068550199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 msgr2=0x55c137ff4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.591+0000 7f15af1c2740 1 --2- 172.21.15.45:0/4068550199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 0x55c137ff4ca0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f159805cb50 tx=0x7f15980605e0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.591+0000 7f15af1c2740 1 -- 172.21.15.45:0/4068550199 shutdown_connections 2024-05-08T05:47:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.591+0000 7f15af1c2740 1 --2- 172.21.15.45:0/4068550199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137fc4bb0 0x55c137fc83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.591+0000 7f15af1c2740 1 --2- 172.21.15.45:0/4068550199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 0x55c137ff4ca0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.591+0000 7f15af1c2740 1 --2- 172.21.15.45:0/4068550199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c137ff1f50 0x55c137ff4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.591+0000 7f15af1c2740 1 -- 172.21.15.45:0/4068550199 >> 172.21.15.45:0/4068550199 conn(0x55c137fc6e20 msgr2=0x55c13800f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.591+0000 7f15af1c2740 1 -- 172.21.15.45:0/4068550199 shutdown_connections 2024-05-08T05:47:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.591+0000 7f15af1c2740 1 -- 172.21.15.45:0/4068550199 wait complete. 2024-05-08T05:47:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.592+0000 7f15af1c2740 1 -- start start 2024-05-08T05:47:06.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.592+0000 7f15af1c2740 1 Processor -- start 2024-05-08T05:47:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.593+0000 7f15af1c2740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c137ff1f50 0x55c137ff5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.593+0000 7f15af1c2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137ff2330 0x55c1380230f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.593+0000 7f15af1c2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff6270 0x55c138023630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.593+0000 7f15af1c2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c137fc9fd0 con 0x55c137ff6270 2024-05-08T05:47:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.593+0000 7f15af1c2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c137eeb760 con 0x55c137ff2330 2024-05-08T05:47:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.593+0000 7f15af1c2740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55c137fca2b0 con 0x55c137ff1f50 2024-05-08T05:47:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.593+0000 7f15af1c2740 10 client.0 _ll_register_callbacks cb 0x55c137f24cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:47:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.593+0000 7f15aaa3d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff6270 0x55c138023630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.593+0000 7f15aaa3d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff6270 0x55c138023630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43424/0 (socket says 172.21.15.45:43424) 2024-05-08T05:47:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.593+0000 7f15aaa3d640 1 -- 172.21.15.45:0/2060811789 learned_addr learned my addr 172.21.15.45:0/2060811789 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.593+0000 7f15a9a3b640 1 --2- 172.21.15.45:0/2060811789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137ff2330 0x55c1380230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.593+0000 7f15aa23c640 1 --2- 172.21.15.45:0/2060811789 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c137ff1f50 0x55c137ff5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.593+0000 7f15aaa3d640 1 -- 172.21.15.45:0/2060811789 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c137ff1f50 msgr2=0x55c137ff5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.593+0000 7f15aaa3d640 1 --2- 172.21.15.45:0/2060811789 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c137ff1f50 0x55c137ff5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.593+0000 7f15aaa3d640 1 -- 172.21.15.45:0/2060811789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137ff2330 msgr2=0x55c1380230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.593+0000 7f15aaa3d640 1 --2- 172.21.15.45:0/2060811789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137ff2330 0x55c1380230f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.593+0000 7f15aaa3d640 1 -- 172.21.15.45:0/2060811789 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c137c402d0 con 0x55c137ff6270 2024-05-08T05:47:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.593+0000 7f15a9a3b640 1 --2- 172.21.15.45:0/2060811789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137ff2330 0x55c1380230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:47:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.593+0000 7f15aaa3d640 1 --2- 172.21.15.45:0/2060811789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff6270 0x55c138023630 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f1594012d10 tx=0x7f1594010610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e73ae385f99a37f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.593+0000 7f15897fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:47:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.594+0000 7f15897fa640 1 -- 172.21.15.45:0/2060811789 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594019020 con 0x55c137ff6270 2024-05-08T05:47:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.594+0000 7f15897fa640 1 -- 172.21.15.45:0/2060811789 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 6101+0+0 (secure 0 0 0) 0x7f1594010cc0 con 0x55c137ff6270 2024-05-08T05:47:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.594+0000 7f15af1c2740 1 -- 172.21.15.45:0/2060811789 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1598069140 con 0x55c137ff6270 2024-05-08T05:47:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.594+0000 7f15897fa640 1 -- 172.21.15.45:0/2060811789 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f15940070d0 con 0x55c137ff6270 2024-05-08T05:47:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.594+0000 7f15af1c2740 10 client.6876 fetch_fsmap learned FSMap version 60 2024-05-08T05:47:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.594+0000 7f15af1c2740 1 -- 172.21.15.45:0/2060811789 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x55c137d0a6b0 con 0x55c137ff6270 2024-05-08T05:47:06.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.594+0000 7f15897fa640 1 -- 172.21.15.45:0/2060811789 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1598069140 con 0x55c137ff6270 2024-05-08T05:47:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.594+0000 7f15897fa640 1 -- 172.21.15.45:0/2060811789 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 60) ==== 3494+0+0 (secure 0 0 0) 0x7f1594026020 con 0x55c137ff6270 2024-05-08T05:47:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.594+0000 7f15af1c2740 10 client.6876 fetch_fsmap finished waiting for FSMap version 60 2024-05-08T05:47:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.594+0000 7f15af1c2740 10 client.6876 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T05:47:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.595+0000 7f15af1c2740 20 client.6876 populate_metadata read hostname 'smithi045' 2024-05-08T05:47:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.595+0000 7f15af1c2740 1 --2- 172.21.15.45:0/2060811789 >> [v2:172.21.15.63:6834/2942326473,v1:172.21.15.63:6836/2942326473] conn(0x55c138017ad0 0x55c137fef480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.595+0000 7f15af1c2740 4 client.6876 mds_command: new command op to 6762 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:47:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.595+0000 7f15af1c2740 1 -- 172.21.15.45:0/2060811789 --> [v2:172.21.15.63:6834/2942326473,v1:172.21.15.63:6836/2942326473] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c137eb2760 con 0x55c138017ad0 2024-05-08T05:47:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.595+0000 7f15aa23c640 1 --2- 172.21.15.45:0/2060811789 >> [v2:172.21.15.63:6834/2942326473,v1:172.21.15.63:6836/2942326473] conn(0x55c138017ad0 0x55c137fef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.595+0000 7f15897fa640 10 client.6876 ms_handle_connect on v2:172.21.15.63:6834/2942326473 2024-05-08T05:47:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.595+0000 7f15aa23c640 1 --2- 172.21.15.45:0/2060811789 >> [v2:172.21.15.63:6834/2942326473,v1:172.21.15.63:6836/2942326473] conn(0x55c138017ad0 0x55c137fef480 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdf8e8f5e92c9ce0 server_cookie=d13891d5ecfe10f5 in_seq=0 out_seq=0 2024-05-08T05:47:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.596+0000 7f15897fa640 1 -- 172.21.15.45:0/2060811789 <== mds.0 v2:172.21.15.63:6834/2942326473 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c137eb2760 con 0x55c138017ad0 2024-05-08T05:47:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.596+0000 7f15897fa640 10 client.6876 handle_command_reply: tid=0 2024-05-08T05:47:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.596+0000 7f15af1c2740 1 client.6876 shutdown 2024-05-08T05:47:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.596+0000 7f15af1c2740 1 -- 172.21.15.45:0/2060811789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff6270 msgr2=0x55c138023630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:06.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.596+0000 7f15af1c2740 1 --2- 172.21.15.45:0/2060811789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff6270 0x55c138023630 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f1594012d10 tx=0x7f1594010610 comp rx=0 tx=0).stop 2024-05-08T05:47:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.596+0000 7f15af1c2740 1 -- 172.21.15.45:0/2060811789 shutdown_connections 2024-05-08T05:47:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.596+0000 7f15af1c2740 1 --2- 172.21.15.45:0/2060811789 >> [v2:172.21.15.63:6834/2942326473,v1:172.21.15.63:6836/2942326473] conn(0x55c138017ad0 0x55c137fef480 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.596+0000 7f15af1c2740 1 --2- 172.21.15.45:0/2060811789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff6270 0x55c138023630 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.596+0000 7f15af1c2740 1 --2- 172.21.15.45:0/2060811789 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137ff2330 0x55c1380230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.596+0000 7f15897fa640 0 client.6876 ms_handle_reset on v2:172.21.15.63:6834/2942326473 2024-05-08T05:47:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.596+0000 7f15af1c2740 1 --2- 172.21.15.45:0/2060811789 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c137ff1f50 0x55c137ff5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.596+0000 7f15af1c2740 1 -- 172.21.15.45:0/2060811789 >> 172.21.15.45:0/2060811789 conn(0x55c137fc4bb0 msgr2=0x55c137fc5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.597+0000 7f15af1c2740 1 -- 172.21.15.45:0/2060811789 shutdown_connections 2024-05-08T05:47:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.597+0000 7f15af1c2740 1 -- 172.21.15.45:0/2060811789 wait complete. 2024-05-08T05:47:06.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.597+0000 7f15af1c2740 20 client.6876 trim_cache size 0 max 16384 2024-05-08T05:47:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.615+0000 7f15af1c2740 1 Processor -- start 2024-05-08T05:47:06.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.615+0000 7f15af1c2740 1 -- start start 2024-05-08T05:47:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.616+0000 7f15af1c2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137ff1f50 0x55c137ff4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.616+0000 7f15af1c2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 0x55c137fc5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.616+0000 7f15af1c2740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c137ff6270 0x55c137fc55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.616+0000 7f15af1c2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4200e20 con 0x55c137ff2330 2024-05-08T05:47:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.616+0000 7f15af1c2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f15a40cda80 con 0x55c137ff1f50 2024-05-08T05:47:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.616+0000 7f15af1c2740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f15a42007b0 con 0x55c137ff6270 2024-05-08T05:47:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.616+0000 7f15a9a3b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 0x55c137fc5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.616+0000 7f15a9a3b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 0x55c137fc5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43440/0 (socket says 172.21.15.45:43440) 2024-05-08T05:47:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.616+0000 7f15a9a3b640 1 -- 172.21.15.45:0/2562417996 learned_addr learned my addr 172.21.15.45:0/2562417996 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.617+0000 7f15aa23c640 1 --2- 172.21.15.45:0/2562417996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137ff1f50 0x55c137ff4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.617+0000 7f15a9a3b640 1 -- 172.21.15.45:0/2562417996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c137ff6270 msgr2=0x55c137fc55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.617+0000 7f15a9a3b640 1 --2- 172.21.15.45:0/2562417996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c137ff6270 0x55c137fc55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.617+0000 7f15a9a3b640 1 -- 172.21.15.45:0/2562417996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137ff1f50 msgr2=0x55c137ff4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.617+0000 7f15a9a3b640 1 --2- 172.21.15.45:0/2562417996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137ff1f50 0x55c137ff4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.617+0000 7f15a9a3b640 1 -- 172.21.15.45:0/2562417996 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c137ff6e60 con 0x55c137ff2330 2024-05-08T05:47:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.617+0000 7f15aa23c640 1 --2- 172.21.15.45:0/2562417996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137ff1f50 0x55c137ff4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:47:06.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.617+0000 7f15a9a3b640 1 --2- 172.21.15.45:0/2562417996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 0x55c137fc5090 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f159805cb50 tx=0x7f1598097000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d88d801501b0b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.617+0000 7f158b7fe640 1 -- 172.21.15.45:0/2562417996 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598069140 con 0x55c137ff2330 2024-05-08T05:47:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.617+0000 7f158b7fe640 1 -- 172.21.15.45:0/2562417996 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f159807b070 con 0x55c137ff2330 2024-05-08T05:47:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.617+0000 7f158b7fe640 1 -- 172.21.15.45:0/2562417996 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15980672b0 con 0x55c137ff2330 2024-05-08T05:47:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.617+0000 7f15af1c2740 1 -- 172.21.15.45:0/2562417996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 msgr2=0x55c137fc5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.617+0000 7f15af1c2740 1 --2- 172.21.15.45:0/2562417996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 0x55c137fc5090 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f159805cb50 tx=0x7f1598097000 comp rx=0 tx=0).stop 2024-05-08T05:47:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.617+0000 7f15af1c2740 1 -- 172.21.15.45:0/2562417996 shutdown_connections 2024-05-08T05:47:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.617+0000 7f15af1c2740 1 --2- 172.21.15.45:0/2562417996 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c137ff6270 0x55c137fc55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.617+0000 7f15af1c2740 1 --2- 172.21.15.45:0/2562417996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 0x55c137fc5090 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.617+0000 7f15af1c2740 1 --2- 172.21.15.45:0/2562417996 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137ff1f50 0x55c137ff4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.618+0000 7f15af1c2740 1 -- 172.21.15.45:0/2562417996 >> 172.21.15.45:0/2562417996 conn(0x55c137fc4bb0 msgr2=0x55c137ff5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.618+0000 7f15af1c2740 1 -- 172.21.15.45:0/2562417996 shutdown_connections 2024-05-08T05:47:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.618+0000 7f15af1c2740 1 -- 172.21.15.45:0/2562417996 wait complete. 2024-05-08T05:47:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.618+0000 7f15af1c2740 1 -- start start 2024-05-08T05:47:06.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.618+0000 7f15af1c2740 1 Processor -- start 2024-05-08T05:47:06.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.619+0000 7f15af1c2740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137ff1f50 0x55c13808d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.619+0000 7f15af1c2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 0x55c13800f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.619+0000 7f15af1c2740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c137ff6270 0x55c13802a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.619+0000 7f15af1c2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c138017950 con 0x55c137ff2330 2024-05-08T05:47:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.619+0000 7f15af1c2740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c137ff8190 con 0x55c137ff1f50 2024-05-08T05:47:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.619+0000 7f15af1c2740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55c138010890 con 0x55c137ff6270 2024-05-08T05:47:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.619+0000 7f15af1c2740 10 client.0 _ll_register_callbacks cb 0x7f1594001e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:47:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.619+0000 7f15a9a3b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 0x55c13800f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.619+0000 7f15a9a3b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 0x55c13800f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43448/0 (socket says 172.21.15.45:43448) 2024-05-08T05:47:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.619+0000 7f15a9a3b640 1 -- 172.21.15.45:0/331298370 learned_addr learned my addr 172.21.15.45:0/331298370 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.619+0000 7f15aa23c640 1 --2- 172.21.15.45:0/331298370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137ff1f50 0x55c13808d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.619+0000 7f15aaa3d640 1 --2- 172.21.15.45:0/331298370 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c137ff6270 0x55c13802a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.619+0000 7f15a9a3b640 1 -- 172.21.15.45:0/331298370 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c137ff6270 msgr2=0x55c13802a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.619+0000 7f15a9a3b640 1 --2- 172.21.15.45:0/331298370 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c137ff6270 0x55c13802a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.620+0000 7f15a9a3b640 1 -- 172.21.15.45:0/331298370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137ff1f50 msgr2=0x55c13808d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.620+0000 7f15a9a3b640 1 --2- 172.21.15.45:0/331298370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137ff1f50 0x55c13808d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.620+0000 7f15a9a3b640 1 -- 172.21.15.45:0/331298370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c137fc4980 con 0x55c137ff2330 2024-05-08T05:47:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.620+0000 7f15aaa3d640 1 --2- 172.21.15.45:0/331298370 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c137ff6270 0x55c13802a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.620+0000 7f15a9a3b640 1 --2- 172.21.15.45:0/331298370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 0x55c13800f760 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f159805d070 tx=0x7f1598000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a68c3e65e8842ee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.620+0000 7f15897fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:47:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.620+0000 7f15897fa640 1 -- 172.21.15.45:0/331298370 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159805ff30 con 0x55c137ff2330 2024-05-08T05:47:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.620+0000 7f15897fa640 1 -- 172.21.15.45:0/331298370 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 6101+0+0 (secure 0 0 0) 0x7f159805c440 con 0x55c137ff2330 2024-05-08T05:47:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.620+0000 7f15af1c2740 1 -- 172.21.15.45:0/331298370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f159807b070 con 0x55c137ff2330 2024-05-08T05:47:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.620+0000 7f15897fa640 1 -- 172.21.15.45:0/331298370 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f15980697f0 con 0x55c137ff2330 2024-05-08T05:47:06.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.620+0000 7f15af1c2740 10 client.6882 fetch_fsmap learned FSMap version 60 2024-05-08T05:47:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.620+0000 7f15af1c2740 1 -- 172.21.15.45:0/331298370 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x55c13808d600 con 0x55c137ff2330 2024-05-08T05:47:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.620+0000 7f15897fa640 1 -- 172.21.15.45:0/331298370 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f159807b070 con 0x55c137ff2330 2024-05-08T05:47:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.620+0000 7f15897fa640 1 -- 172.21.15.45:0/331298370 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 60) ==== 3494+0+0 (secure 0 0 0) 0x7f159805c640 con 0x55c137ff2330 2024-05-08T05:47:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.620+0000 7f15af1c2740 10 client.6882 fetch_fsmap finished waiting for FSMap version 60 2024-05-08T05:47:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.620+0000 7f15af1c2740 10 client.6882 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-08T05:47:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.620+0000 7f15af1c2740 20 client.6882 populate_metadata read hostname 'smithi045' 2024-05-08T05:47:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.620+0000 7f15af1c2740 1 --2- 172.21.15.45:0/331298370 >> [v2:172.21.15.63:6834/2942326473,v1:172.21.15.63:6836/2942326473] conn(0x55c138011870 0x55c138011c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.620+0000 7f15af1c2740 4 client.6882 mds_command: new command op to 6762 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:47:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.620+0000 7f15af1c2740 1 -- 172.21.15.45:0/331298370 --> [v2:172.21.15.63:6834/2942326473,v1:172.21.15.63:6836/2942326473] -- command(tid 0: {"prefix": "status"}) -- 0x55c137f3c4e0 con 0x55c138011870 2024-05-08T05:47:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.621+0000 7f15aa23c640 1 --2- 172.21.15.45:0/331298370 >> [v2:172.21.15.63:6834/2942326473,v1:172.21.15.63:6836/2942326473] conn(0x55c138011870 0x55c138011c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.621+0000 7f15aa23c640 1 --2- 172.21.15.45:0/331298370 >> [v2:172.21.15.63:6834/2942326473,v1:172.21.15.63:6836/2942326473] conn(0x55c138011870 0x55c138011c50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6382c7f5b0221bc8 server_cookie=755fef4b05304d52 in_seq=0 out_seq=0 2024-05-08T05:47:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.621+0000 7f15897fa640 10 client.6882 ms_handle_connect on v2:172.21.15.63:6834/2942326473 2024-05-08T05:47:06.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.621+0000 7f15897fa640 1 -- 172.21.15.45:0/331298370 <== mds.0 v2:172.21.15.63:6834/2942326473 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55c137f3c4e0 con 0x55c138011870 2024-05-08T05:47:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.621+0000 7f15897fa640 10 client.6882 handle_command_reply: tid=0 2024-05-08T05:47:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.621+0000 7f15af1c2740 1 client.6882 shutdown 2024-05-08T05:47:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.621+0000 7f15af1c2740 1 -- 172.21.15.45:0/331298370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 msgr2=0x55c13800f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.621+0000 7f15af1c2740 1 --2- 172.21.15.45:0/331298370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 0x55c13800f760 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f159805d070 tx=0x7f1598000bc0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.621+0000 7f15af1c2740 1 -- 172.21.15.45:0/331298370 shutdown_connections 2024-05-08T05:47:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.621+0000 7f15af1c2740 1 --2- 172.21.15.45:0/331298370 >> [v2:172.21.15.63:6834/2942326473,v1:172.21.15.63:6836/2942326473] conn(0x55c138011870 0x55c138011c50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.621+0000 7f15af1c2740 1 --2- 172.21.15.45:0/331298370 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c137ff6270 0x55c13802a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.621+0000 7f15af1c2740 1 --2- 172.21.15.45:0/331298370 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c137ff2330 0x55c13800f760 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.621+0000 7f15897fa640 0 client.6882 ms_handle_reset on v2:172.21.15.63:6834/2942326473 2024-05-08T05:47:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.621+0000 7f15af1c2740 1 --2- 172.21.15.45:0/331298370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c137ff1f50 0x55c13808d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.621+0000 7f15af1c2740 1 -- 172.21.15.45:0/331298370 >> 172.21.15.45:0/331298370 conn(0x55c137fc4bb0 msgr2=0x55c138012a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.621+0000 7f15af1c2740 1 -- 172.21.15.45:0/331298370 shutdown_connections 2024-05-08T05:47:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.621+0000 7f15af1c2740 1 -- 172.21.15.45:0/331298370 wait complete. 2024-05-08T05:47:06.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.622+0000 7f15af1c2740 20 client.6882 trim_cache size 0 max 16384 2024-05-08T05:47:06.629 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:47:06.629 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "e9e1170a-fd27-4625-97a9-5d5f104d13a2", 2024-05-08T05:47:06.629 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-05-08T05:47:06.629 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 6762, 2024-05-08T05:47:06.629 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-05-08T05:47:06.629 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-05-08T05:47:06.629 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-05-08T05:47:06.629 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 5.7923770829999999, 2024-05-08T05:47:06.629 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 60, 2024-05-08T05:47:06.629 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 88, 2024-05-08T05:47:06.629 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 88, 2024-05-08T05:47:06.630 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 11.107711918 2024-05-08T05:47:06.630 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:47:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.622+0000 7f15ac4a0640 1 -- 172.21.15.45:0/1872142537 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f15900762d0 msgr2=0x7f1590078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:06.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.622+0000 7f15ac4a0640 1 --2- 172.21.15.45:0/1872142537 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f15900762d0 0x7f1590078790 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f159c034000 tx=0x7f159c02e000 comp rx=0 tx=0).stop 2024-05-08T05:47:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.622+0000 7f15ac4a0640 1 -- 172.21.15.45:0/1872142537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15a40d3b00 msgr2=0x7f15a415a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.622+0000 7f15ac4a0640 1 --2- 172.21.15.45:0/1872142537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15a40d3b00 0x7f15a415a5d0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f159800e990 tx=0x7f159800ee50 comp rx=0 tx=0).stop 2024-05-08T05:47:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.622+0000 7f15ac4a0640 1 -- 172.21.15.45:0/1872142537 shutdown_connections 2024-05-08T05:47:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.622+0000 7f15ac4a0640 1 --2- 172.21.15.45:0/1872142537 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f15900762d0 0x7f1590078790 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.622+0000 7f15ac4a0640 1 --2- 172.21.15.45:0/1872142537 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f15a415f380 0x7f15a409fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.622+0000 7f15ac4a0640 1 --2- 172.21.15.45:0/1872142537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15a40d3b00 0x7f15a415a5d0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.622+0000 7f15ac4a0640 1 --2- 172.21.15.45:0/1872142537 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f15a40d0cf0 0x7f15a40a2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.622+0000 7f15ac4a0640 1 -- 172.21.15.45:0/1872142537 >> 172.21.15.45:0/1872142537 conn(0x7f15a40d4fe0 msgr2=0x7f15a40d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.622+0000 7f15ac4a0640 1 -- 172.21.15.45:0/1872142537 shutdown_connections 2024-05-08T05:47:06.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.622+0000 7f15ac4a0640 1 -- 172.21.15.45:0/1872142537 wait complete. 2024-05-08T05:47:06.633 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T05:47:06.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.737+0000 7f8ccde08640 1 Processor -- start 2024-05-08T05:47:06.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.738+0000 7f8ccde08640 1 -- start start 2024-05-08T05:47:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.738+0000 7f8ccde08640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8cc815f420 0x7f8cc80d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.738+0000 7f8ccde08640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8cc80d3ca0 0x7f8cc80d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.738+0000 7f8ccde08640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8cc80d1eb0 0x7f8cc80d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.738+0000 7f8ccde08640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f8cc80d22f0 con 0x7f8cc80d1eb0 2024-05-08T05:47:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.738+0000 7f8ccde08640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8cc81630b0 con 0x7f8cc80d3ca0 2024-05-08T05:47:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.738+0000 7f8ccde08640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8cc80d2550 con 0x7f8cc815f420 2024-05-08T05:47:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.739+0000 7f8cc7fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8cc80d1eb0 0x7f8cc80d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.739+0000 7f8cc7fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8cc80d1eb0 0x7f8cc80d4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43452/0 (socket says 172.21.15.45:43452) 2024-05-08T05:47:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.739+0000 7f8cc7fff640 1 -- 172.21.15.45:0/1625611426 learned_addr learned my addr 172.21.15.45:0/1625611426 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:06.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.739+0000 7f8cc6ffd640 1 --2- 172.21.15.45:0/1625611426 >> v2:172.21.15.63:3300/0 conn(0x7f8cc80d3ca0 0x7f8cc80d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.739+0000 7f8cc77fe640 1 --2- 172.21.15.45:0/1625611426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8cc815f420 0x7f8cc80d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.739+0000 7f8cc7fff640 1 -- 172.21.15.45:0/1625611426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8cc815f420 msgr2=0x7f8cc80d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.739+0000 7f8cc7fff640 1 --2- 172.21.15.45:0/1625611426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8cc815f420 0x7f8cc80d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.739+0000 7f8cc7fff640 1 -- 172.21.15.45:0/1625611426 >> v2:172.21.15.63:3300/0 conn(0x7f8cc80d3ca0 msgr2=0x7f8cc80d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.739+0000 7f8cc7fff640 1 --2- 172.21.15.45:0/1625611426 >> v2:172.21.15.63:3300/0 conn(0x7f8cc80d3ca0 0x7f8cc80d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.739+0000 7f8cc7fff640 1 -- 172.21.15.45:0/1625611426 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc80d2c80 con 0x7f8cc80d1eb0 2024-05-08T05:47:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.740+0000 7f8cc7fff640 1 --2- 172.21.15.45:0/1625611426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc80d1eb0 0x7f8cc80d4cc0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80b7b90 tx=0x7f8cc002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f22d68f53dedc2a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.740+0000 7f8cc67fc640 1 -- 172.21.15.45:0/1625611426 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0036070 con 0x7f8cc80d1eb0 2024-05-08T05:47:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.740+0000 7f8cc67fc640 1 -- 172.21.15.45:0/1625611426 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cc0031600 con 0x7f8cc80d1eb0 2024-05-08T05:47:06.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.740+0000 7f8cc67fc640 1 -- 172.21.15.45:0/1625611426 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc00319d0 con 0x7f8cc80d1eb0 2024-05-08T05:47:06.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.740+0000 7f8ccde08640 1 -- 172.21.15.45:0/1625611426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc80d1eb0 msgr2=0x7f8cc80d4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:06.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.740+0000 7f8ccde08640 1 --2- 172.21.15.45:0/1625611426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc80d1eb0 0x7f8cc80d4cc0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80b7b90 tx=0x7f8cc002a6a0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.740+0000 7f8ccde08640 1 -- 172.21.15.45:0/1625611426 shutdown_connections 2024-05-08T05:47:06.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.740+0000 7f8ccde08640 1 --2- 172.21.15.45:0/1625611426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc80d1eb0 0x7f8cc80d4cc0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.740+0000 7f8ccde08640 1 --2- 172.21.15.45:0/1625611426 >> v2:172.21.15.63:3300/0 conn(0x7f8cc80d3ca0 0x7f8cc80d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.740+0000 7f8ccde08640 1 --2- 172.21.15.45:0/1625611426 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8cc815f420 0x7f8cc80d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.740+0000 7f8ccde08640 1 -- 172.21.15.45:0/1625611426 >> 172.21.15.45:0/1625611426 conn(0x7f8cc808eac0 msgr2=0x7f8cc809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:06.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.740+0000 7f8ccde08640 1 -- 172.21.15.45:0/1625611426 shutdown_connections 2024-05-08T05:47:06.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.740+0000 7f8ccde08640 1 -- 172.21.15.45:0/1625611426 wait complete. 2024-05-08T05:47:06.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.742+0000 7f8ccde08640 1 Processor -- start 2024-05-08T05:47:06.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.742+0000 7f8ccde08640 1 -- start start 2024-05-08T05:47:06.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.742+0000 7f8ccde08640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc80d1eb0 0x7f8cc815a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.742+0000 7f8ccde08640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cc80ce020 0x7f8cc80c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.742+0000 7f8ccde08640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8cc80cc0e0 0x7f8cc815be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.742+0000 7f8ccde08640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc80d2830 con 0x7f8cc80d1eb0 2024-05-08T05:47:06.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.742+0000 7f8ccde08640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc80d3240 con 0x7f8cc80ce020 2024-05-08T05:47:06.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.742+0000 7f8ccde08640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8cc80d5200 con 0x7f8cc80cc0e0 2024-05-08T05:47:06.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.743+0000 7f8cc77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc80d1eb0 0x7f8cc815a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.743+0000 7f8cc77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc80d1eb0 0x7f8cc815a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43464/0 (socket says 172.21.15.45:43464) 2024-05-08T05:47:06.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.743+0000 7f8cc77fe640 1 -- 172.21.15.45:0/446686444 learned_addr learned my addr 172.21.15.45:0/446686444 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:06.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.743+0000 7f8cc7fff640 1 --2- 172.21.15.45:0/446686444 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8cc80cc0e0 0x7f8cc815be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.743+0000 7f8cc6ffd640 1 --2- 172.21.15.45:0/446686444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cc80ce020 0x7f8cc80c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.743+0000 7f8cc77fe640 1 -- 172.21.15.45:0/446686444 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8cc80cc0e0 msgr2=0x7f8cc815be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:06.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.743+0000 7f8cc77fe640 1 --2- 172.21.15.45:0/446686444 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8cc80cc0e0 0x7f8cc815be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.743+0000 7f8cc77fe640 1 -- 172.21.15.45:0/446686444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cc80ce020 msgr2=0x7f8cc80c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:06.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.743+0000 7f8cc77fe640 1 --2- 172.21.15.45:0/446686444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cc80ce020 0x7f8cc80c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.743+0000 7f8cc77fe640 1 -- 172.21.15.45:0/446686444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc80ccaa0 con 0x7f8cc80d1eb0 2024-05-08T05:47:06.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.743+0000 7f8cc7fff640 1 --2- 172.21.15.45:0/446686444 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8cc80cc0e0 0x7f8cc815be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:47:06.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.743+0000 7f8cc6ffd640 1 --2- 172.21.15.45:0/446686444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cc80ce020 0x7f8cc80c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:47:06.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.743+0000 7f8cc77fe640 1 --2- 172.21.15.45:0/446686444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc80d1eb0 0x7f8cc815a590 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800d5f0 tx=0x7f8cb800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.743+0000 7f8caffff640 1 -- 172.21.15.45:0/446686444 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb80125f0 con 0x7f8cc80d1eb0 2024-05-08T05:47:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.743+0000 7f8caffff640 1 -- 172.21.15.45:0/446686444 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cb8019070 con 0x7f8cc80d1eb0 2024-05-08T05:47:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.743+0000 7f8ccde08640 1 -- 172.21.15.45:0/446686444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc80ccd80 con 0x7f8cc80d1eb0 2024-05-08T05:47:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.743+0000 7f8caffff640 1 -- 172.21.15.45:0/446686444 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb80129d0 con 0x7f8cc80d1eb0 2024-05-08T05:47:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.743+0000 7f8ccde08640 1 -- 172.21.15.45:0/446686444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc80c9990 con 0x7f8cc80d1eb0 2024-05-08T05:47:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.744+0000 7f8ccde08640 1 -- 172.21.15.45:0/446686444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cc80693f0 con 0x7f8cc80d1eb0 2024-05-08T05:47:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.745+0000 7f8caffff640 1 -- 172.21.15.45:0/446686444 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8cb8012b70 con 0x7f8cc80d1eb0 2024-05-08T05:47:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.746+0000 7f8caffff640 1 --2- 172.21.15.45:0/446686444 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8cb4076300 0x7f8cb40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.746+0000 7f8caffff640 1 -- 172.21.15.45:0/446686444 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6101+0+0 (secure 0 0 0) 0x7f8cb8097c20 con 0x7f8cc80d1eb0 2024-05-08T05:47:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.746+0000 7f8cc6ffd640 1 --2- 172.21.15.45:0/446686444 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8cb4076300 0x7f8cb40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.747+0000 7f8cc6ffd640 1 --2- 172.21.15.45:0/446686444 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8cb4076300 0x7f8cb40787c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8cc81f8d00 tx=0x7f8cbc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:06.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.750+0000 7f8caffff640 1 -- 172.21.15.45:0/446686444 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8cb8061360 con 0x7f8cc80d1eb0 2024-05-08T05:47:06.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.980+0000 7f8ccde08640 1 -- 172.21.15.45:0/446686444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8cc80c9d70 con 0x7f8cc80d1eb0 2024-05-08T05:47:06.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.980+0000 7f8caffff640 1 -- 172.21.15.45:0/446686444 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v60) ==== 53+0+73 (secure 0 0 0) 0x7f8cb8060c60 con 0x7f8cc80d1eb0 2024-05-08T05:47:06.980 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T05:47:06.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.982+0000 7f8ccde08640 1 -- 172.21.15.45:0/446686444 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8cb4076300 msgr2=0x7f8cb40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:06.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.982+0000 7f8ccde08640 1 --2- 172.21.15.45:0/446686444 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8cb4076300 0x7f8cb40787c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8cc81f8d00 tx=0x7f8cbc008040 comp rx=0 tx=0).stop 2024-05-08T05:47:06.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.982+0000 7f8ccde08640 1 -- 172.21.15.45:0/446686444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc80d1eb0 msgr2=0x7f8cc815a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:06.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.982+0000 7f8ccde08640 1 --2- 172.21.15.45:0/446686444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc80d1eb0 0x7f8cc815a590 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800d5f0 tx=0x7f8cb800dab0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.982+0000 7f8ccde08640 1 -- 172.21.15.45:0/446686444 shutdown_connections 2024-05-08T05:47:06.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.982+0000 7f8ccde08640 1 --2- 172.21.15.45:0/446686444 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8cb4076300 0x7f8cb40787c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.982+0000 7f8ccde08640 1 --2- 172.21.15.45:0/446686444 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8cc80cc0e0 0x7f8cc815be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.982+0000 7f8ccde08640 1 --2- 172.21.15.45:0/446686444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8cc80ce020 0x7f8cc80c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.982+0000 7f8ccde08640 1 --2- 172.21.15.45:0/446686444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cc80d1eb0 0x7f8cc815a590 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.982+0000 7f8ccde08640 1 -- 172.21.15.45:0/446686444 >> 172.21.15.45:0/446686444 conn(0x7f8cc808eac0 msgr2=0x7f8cc806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:06.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.982+0000 7f8ccde08640 1 -- 172.21.15.45:0/446686444 shutdown_connections 2024-05-08T05:47:06.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:06.982+0000 7f8ccde08640 1 -- 172.21.15.45:0/446686444 wait complete. 2024-05-08T05:47:06.993 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T05:47:06.993 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:47:06.994 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-05-08T05:47:06.994 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T05:47:06.994 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:47:06.994 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T05:47:06.994 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T05:47:06.994 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:47:06.994 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:47:06.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:06.994 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-05-08T05:47:07.011 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:47:07.011 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:47:07.011 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:47:07.011 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:47:07.012 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-05-08T05:47:07.012 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:47:07.012 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:47:07.012 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:47:07.012 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-05-08T05:47:07.012 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:47:07.012 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:47:07.012 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-05-08T05:47:07.012 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:47:07.012 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-05-08T05:47:07.012 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-05-08T05:47:07.012 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:47:07.012 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42397sec preferred_lft 42397sec 2024-05-08T05:47:07.012 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-05-08T05:47:07.012 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:47:07.013 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:47:07.013 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-05-08T05:47:07.013 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-05-08T05:47:07.013 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:47:07.013 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T05:47:07.013 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T05:47:07.013 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:47:07.013 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c80a:aff:fe94:c77/64 scope link 2024-05-08T05:47:07.013 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:47:07.013 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:47:07.013 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 82:f3:eb:6a:57:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:47:07.013 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::80f3:ebff:fe6a:57ba/64 scope link 2024-05-08T05:47:07.014 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:47:07.014 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:47:07.014 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T05:47:07.014 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::14a2:64ff:feda:3a9c/64 scope link 2024-05-08T05:47:07.014 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:47:07.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:07.014 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:47:07.076 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:47:07.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:07.078 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-05-08T05:47:07.134 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-05-08T05:47:07.135 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:47:07.135 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:47:07.135 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:47:07.135 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:47:07.135 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:47:07.135 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:47:07.135 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:47:07.135 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:47:07.135 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:47:07.135 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:47:07.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:47:07.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:47:07.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:07.266 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:47:07.282 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T05:47:07.282 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T05:47:07.282 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:47:07.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:07.282 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-05-08T05:47:07.348 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:47:07.348 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T05:47:07.404 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:47:07.404 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:47:07.404 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:47:07.459 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T05:47:07.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:07.460 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:47:07.545 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:47:07.545 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:47:07.553 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:47:07.553 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:47:07.553 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-08T05:47:07.553 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@e9e1170a-fd27-4625-97a9-5d5f104d13a2.cephfs=/ 2024-05-08T05:47:07.553 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.63:3300/172.21.15.63:3301" will pass to kernel 2024-05-08T05:47:07.554 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T05:47:07.554 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:47:07.554 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:47:07.554 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:47:07.554 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:47:07.555 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:47:07.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:07.555 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:47:07.581 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:47:07.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:47:07.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:47:07.687 INFO:teuthology.orchestra.run.smithi045.stdout:client6891 2024-05-08T05:47:07.688 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:47:07.688 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:47:07.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:07.704 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-05-08T05:47:07.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:07.766 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-05-08T05:47:07.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:07.824 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-05-08T05:47:07.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:07.882 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-05-08T05:47:07.944 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:47:07.944 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:47:07.944 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:47:07.945 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:47:07.945 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-05-08T05:47:07.945 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:47:08.053 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627776 2024-05-08T05:47:08.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:47:08.062 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:47:08.062 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:47:08.121 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:47:08.122 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:47:08.178 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:47:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.322+0000 7fd7ebe37640 1 Processor -- start 2024-05-08T05:47:08.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.322+0000 7fd7ebe37640 1 -- start start 2024-05-08T05:47:08.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.323+0000 7fd7ebe37640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd7e406c3a0 0x7fd7e40b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.323+0000 7fd7ebe37640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd7e415d180 0x7fd7e40b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.323+0000 7fd7ebe37640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7e409ff30 con 0x7fd7e40aedf0 2024-05-08T05:47:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.323+0000 7fd7ebe37640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fd7e415e360 con 0x7fd7e406c3a0 2024-05-08T05:47:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.323+0000 7fd7ebe37640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd7e415d750 con 0x7fd7e415d180 2024-05-08T05:47:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.323+0000 7fd7e9bd3640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd7e406c3a0 0x7fd7e40b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.323+0000 7fd7e9bd3640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd7e406c3a0 0x7fd7e40b13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43474/0 (socket says 172.21.15.45:43474) 2024-05-08T05:47:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.323+0000 7fd7e9bd3640 1 -- 172.21.15.45:0/271061797 learned_addr learned my addr 172.21.15.45:0/271061797 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.323+0000 7fd7e9bd3640 1 -- 172.21.15.45:0/271061797 >> v2:172.21.15.63:3300/0 conn(0x7fd7e415d180 msgr2=0x7fd7e40b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.323+0000 7fd7e9bd3640 1 --2- 172.21.15.45:0/271061797 >> v2:172.21.15.63:3300/0 conn(0x7fd7e415d180 0x7fd7e40b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.323+0000 7fd7e9bd3640 1 -- 172.21.15.45:0/271061797 >> v1:172.21.15.63:6789/0 conn(0x7fd7e40aedf0 legacy=0x7fd7e406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:08.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.323+0000 7fd7e9bd3640 1 -- 172.21.15.45:0/271061797 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e415da30 con 0x7fd7e406c3a0 2024-05-08T05:47:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.324+0000 7fd7e9bd3640 1 --2- 172.21.15.45:0/271061797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7e406c3a0 0x7fd7e40b13f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4073100 tx=0x7fd7e002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de88ba2d4ace8e68 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:08.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.324+0000 7fd7e8bd1640 1 -- 172.21.15.45:0/271061797 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0037070 con 0x7fd7e406c3a0 2024-05-08T05:47:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.324+0000 7fd7e8bd1640 1 -- 172.21.15.45:0/271061797 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e0032440 con 0x7fd7e406c3a0 2024-05-08T05:47:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.324+0000 7fd7e8bd1640 1 -- 172.21.15.45:0/271061797 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0032810 con 0x7fd7e406c3a0 2024-05-08T05:47:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.325+0000 7fd7ebe37640 1 -- 172.21.15.45:0/271061797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7e406c3a0 msgr2=0x7fd7e40b13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.325+0000 7fd7ebe37640 1 --2- 172.21.15.45:0/271061797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7e406c3a0 0x7fd7e40b13f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4073100 tx=0x7fd7e002a1e0 comp rx=0 tx=0).stop 2024-05-08T05:47:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.325+0000 7fd7ebe37640 1 -- 172.21.15.45:0/271061797 shutdown_connections 2024-05-08T05:47:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.325+0000 7fd7ebe37640 1 --2- 172.21.15.45:0/271061797 >> v2:172.21.15.63:3300/0 conn(0x7fd7e415d180 0x7fd7e40b6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.325+0000 7fd7ebe37640 1 --2- 172.21.15.45:0/271061797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7e406c3a0 0x7fd7e40b13f0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.325+0000 7fd7ebe37640 1 -- 172.21.15.45:0/271061797 >> 172.21.15.45:0/271061797 conn(0x7fd7e40b3b10 msgr2=0x7fd7e40b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.325+0000 7fd7ebe37640 1 -- 172.21.15.45:0/271061797 shutdown_connections 2024-05-08T05:47:08.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.325+0000 7fd7ebe37640 1 -- 172.21.15.45:0/271061797 wait complete. 2024-05-08T05:47:08.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.326+0000 7fd7ebe37640 1 Processor -- start 2024-05-08T05:47:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.326+0000 7fd7ebe37640 1 -- start start 2024-05-08T05:47:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.327+0000 7fd7ebe37640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7e406c3a0 0x7fd7e4142870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.327+0000 7fd7ebe37640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7e40aedf0 0x7fd7e41540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.327+0000 7fd7ebe37640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7e415d180 0x7fd7e41545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.327+0000 7fd7ebe37640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e40b4fc0 con 0x7fd7e406c3a0 2024-05-08T05:47:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.327+0000 7fd7ebe37640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e415dff0 con 0x7fd7e40aedf0 2024-05-08T05:47:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.327+0000 7fd7ebe37640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd7e40b5140 con 0x7fd7e415d180 2024-05-08T05:47:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.327+0000 7fd7e9bd3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7e406c3a0 0x7fd7e4142870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.327+0000 7fd7e9bd3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7e406c3a0 0x7fd7e4142870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43482/0 (socket says 172.21.15.45:43482) 2024-05-08T05:47:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.327+0000 7fd7e9bd3640 1 -- 172.21.15.45:0/1506125577 learned_addr learned my addr 172.21.15.45:0/1506125577 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.327+0000 7fd7e93d2640 1 --2- 172.21.15.45:0/1506125577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7e40aedf0 0x7fd7e41540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.327+0000 7fd7ea3d4640 1 --2- 172.21.15.45:0/1506125577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7e415d180 0x7fd7e41545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:08.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.327+0000 7fd7e9bd3640 1 -- 172.21.15.45:0/1506125577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7e415d180 msgr2=0x7fd7e41545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.327+0000 7fd7e9bd3640 1 --2- 172.21.15.45:0/1506125577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7e415d180 0x7fd7e41545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.327+0000 7fd7e9bd3640 1 -- 172.21.15.45:0/1506125577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7e40aedf0 msgr2=0x7fd7e41540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.327+0000 7fd7e9bd3640 1 --2- 172.21.15.45:0/1506125577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7e40aedf0 0x7fd7e41540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.327+0000 7fd7e9bd3640 1 -- 172.21.15.45:0/1506125577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e415c0c0 con 0x7fd7e406c3a0 2024-05-08T05:47:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.327+0000 7fd7ea3d4640 1 --2- 172.21.15.45:0/1506125577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7e415d180 0x7fd7e41545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:47:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.327+0000 7fd7e9bd3640 1 --2- 172.21.15.45:0/1506125577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7e406c3a0 0x7fd7e4142870 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0000c00 tx=0x7fd7e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.328+0000 7fd7d67fc640 1 -- 172.21.15.45:0/1506125577 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0041070 con 0x7fd7e406c3a0 2024-05-08T05:47:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.328+0000 7fd7d67fc640 1 -- 172.21.15.45:0/1506125577 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e003c870 con 0x7fd7e406c3a0 2024-05-08T05:47:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.328+0000 7fd7ebe37640 1 -- 172.21.15.45:0/1506125577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7e4154bd0 con 0x7fd7e406c3a0 2024-05-08T05:47:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.328+0000 7fd7d67fc640 1 -- 172.21.15.45:0/1506125577 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0037040 con 0x7fd7e406c3a0 2024-05-08T05:47:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.328+0000 7fd7ebe37640 1 -- 172.21.15.45:0/1506125577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7e4155190 con 0x7fd7e406c3a0 2024-05-08T05:47:08.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.329+0000 7fd7ebe37640 1 -- 172.21.15.45:0/1506125577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7e406a8f0 con 0x7fd7e406c3a0 2024-05-08T05:47:08.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.329+0000 7fd7d67fc640 1 -- 172.21.15.45:0/1506125577 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd7e0044070 con 0x7fd7e406c3a0 2024-05-08T05:47:08.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.330+0000 7fd7d67fc640 1 --2- 172.21.15.45:0/1506125577 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7c80762d0 0x7fd7c8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:08.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.330+0000 7fd7d67fc640 1 -- 172.21.15.45:0/1506125577 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6101+0+0 (secure 0 0 0) 0x7fd7e00b8310 con 0x7fd7e406c3a0 2024-05-08T05:47:08.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.330+0000 7fd7e93d2640 1 --2- 172.21.15.45:0/1506125577 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7c80762d0 0x7fd7c8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:08.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.331+0000 7fd7e93d2640 1 --2- 172.21.15.45:0/1506125577 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7c80762d0 0x7fd7c8078790 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8007fd0 tx=0x7fd7d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:08.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.334+0000 7fd7d67fc640 1 -- 172.21.15.45:0/1506125577 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd7e0081b70 con 0x7fd7e406c3a0 2024-05-08T05:47:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.539+0000 7fd7ebe37640 1 -- 172.21.15.45:0/1506125577 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd7e40693f0 con 0x7fd7e406c3a0 2024-05-08T05:47:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.539+0000 7fd7d67fc640 1 -- 172.21.15.45:0/1506125577 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v88) ==== 80+0+0 (secure 0 0 0) 0x7fd7e0081470 con 0x7fd7e406c3a0 2024-05-08T05:47:08.539 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-05-08T05:47:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.541+0000 7fd7ebe37640 1 -- 172.21.15.45:0/1506125577 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7c80762d0 msgr2=0x7fd7c8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:08.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.541+0000 7fd7ebe37640 1 --2- 172.21.15.45:0/1506125577 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7c80762d0 0x7fd7c8078790 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8007fd0 tx=0x7fd7d8009040 comp rx=0 tx=0).stop 2024-05-08T05:47:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.541+0000 7fd7ebe37640 1 -- 172.21.15.45:0/1506125577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7e406c3a0 msgr2=0x7fd7e4142870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.541+0000 7fd7ebe37640 1 --2- 172.21.15.45:0/1506125577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7e406c3a0 0x7fd7e4142870 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0000c00 tx=0x7fd7e00046c0 comp rx=0 tx=0).stop 2024-05-08T05:47:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.541+0000 7fd7ebe37640 1 -- 172.21.15.45:0/1506125577 shutdown_connections 2024-05-08T05:47:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.541+0000 7fd7ebe37640 1 --2- 172.21.15.45:0/1506125577 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7c80762d0 0x7fd7c8078790 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.541+0000 7fd7ebe37640 1 --2- 172.21.15.45:0/1506125577 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7e415d180 0x7fd7e41545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.541+0000 7fd7ebe37640 1 --2- 172.21.15.45:0/1506125577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7e40aedf0 0x7fd7e41540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.541+0000 7fd7ebe37640 1 --2- 172.21.15.45:0/1506125577 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd7e406c3a0 0x7fd7e4142870 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.541+0000 7fd7ebe37640 1 -- 172.21.15.45:0/1506125577 >> 172.21.15.45:0/1506125577 conn(0x7fd7e40b3b10 msgr2=0x7fd7e4075840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.541+0000 7fd7ebe37640 1 -- 172.21.15.45:0/1506125577 shutdown_connections 2024-05-08T05:47:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.541+0000 7fd7ebe37640 1 -- 172.21.15.45:0/1506125577 wait complete. 2024-05-08T05:47:08.552 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T05:47:08.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:08.552 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T05:47:08.607 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:47:08.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:08.607 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T05:47:08.632 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:47:08.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.777+0000 7f8ad2845640 1 Processor -- start 2024-05-08T05:47:08.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.777+0000 7f8ad2845640 1 -- start start 2024-05-08T05:47:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.777+0000 7f8ad2845640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8acc15ccb0 0x7f8acc15d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.777+0000 7f8ad2845640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8acc0d2ce0 0x7f8acc0d3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.777+0000 7f8ad2845640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8acc070b90 0x7f8acc071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.777+0000 7f8ad2845640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f8acc0afba0 con 0x7f8acc15ccb0 2024-05-08T05:47:08.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.777+0000 7f8ad2845640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8acc0718c0 con 0x7f8acc070b90 2024-05-08T05:47:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.777+0000 7f8ad2845640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8acc071ba0 con 0x7f8acc0d2ce0 2024-05-08T05:47:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.778+0000 7f8acbfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8acc15ccb0 0x7f8acc15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.778+0000 7f8acbfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8acc15ccb0 0x7f8acc15d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43492/0 (socket says 172.21.15.45:43492) 2024-05-08T05:47:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.778+0000 7f8acbfff640 1 -- 172.21.15.45:0/2095504799 learned_addr learned my addr 172.21.15.45:0/2095504799 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.778+0000 7f8ac3fff640 1 --2- 172.21.15.45:0/2095504799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8acc0d2ce0 0x7f8acc0d3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.778+0000 7f8ad0de2640 1 --2- 172.21.15.45:0/2095504799 >> v2:172.21.15.63:3300/0 conn(0x7f8acc070b90 0x7f8acc071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.778+0000 7f8acbfff640 1 -- 172.21.15.45:0/2095504799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8acc0d2ce0 msgr2=0x7f8acc0d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.778+0000 7f8acbfff640 1 --2- 172.21.15.45:0/2095504799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8acc0d2ce0 0x7f8acc0d3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.778+0000 7f8acbfff640 1 -- 172.21.15.45:0/2095504799 >> v2:172.21.15.63:3300/0 conn(0x7f8acc070b90 msgr2=0x7f8acc071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.778+0000 7f8acbfff640 1 --2- 172.21.15.45:0/2095504799 >> v2:172.21.15.63:3300/0 conn(0x7f8acc070b90 0x7f8acc071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.778+0000 7f8acbfff640 1 -- 172.21.15.45:0/2095504799 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8acc071e80 con 0x7f8acc15ccb0 2024-05-08T05:47:08.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.778+0000 7f8ac3fff640 1 --2- 172.21.15.45:0/2095504799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8acc0d2ce0 0x7f8acc0d3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:08.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.778+0000 7f8ad0de2640 1 --2- 172.21.15.45:0/2095504799 >> v2:172.21.15.63:3300/0 conn(0x7f8acc070b90 0x7f8acc071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:08.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.779+0000 7f8acbfff640 1 --2- 172.21.15.45:0/2095504799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc15ccb0 0x7f8acc15d090 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009870 tx=0x7f8abc032990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d892b467acc584c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:08.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.779+0000 7f8acb7fe640 1 -- 172.21.15.45:0/2095504799 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc03c020 con 0x7f8acc15ccb0 2024-05-08T05:47:08.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.779+0000 7f8acb7fe640 1 -- 172.21.15.45:0/2095504799 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8abc038640 con 0x7f8acc15ccb0 2024-05-08T05:47:08.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.779+0000 7f8acb7fe640 1 -- 172.21.15.45:0/2095504799 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc034040 con 0x7f8acc15ccb0 2024-05-08T05:47:08.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.780+0000 7f8ad2845640 1 -- 172.21.15.45:0/2095504799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc15ccb0 msgr2=0x7f8acc15d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:08.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.780+0000 7f8ad2845640 1 --2- 172.21.15.45:0/2095504799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc15ccb0 0x7f8acc15d090 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009870 tx=0x7f8abc032990 comp rx=0 tx=0).stop 2024-05-08T05:47:08.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.780+0000 7f8ad2845640 1 -- 172.21.15.45:0/2095504799 shutdown_connections 2024-05-08T05:47:08.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.780+0000 7f8ad2845640 1 --2- 172.21.15.45:0/2095504799 >> v2:172.21.15.63:3300/0 conn(0x7f8acc070b90 0x7f8acc071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:08.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.780+0000 7f8ad2845640 1 --2- 172.21.15.45:0/2095504799 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8acc0d2ce0 0x7f8acc0d3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:08.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.780+0000 7f8ad2845640 1 --2- 172.21.15.45:0/2095504799 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc15ccb0 0x7f8acc15d090 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:08.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.780+0000 7f8ad2845640 1 -- 172.21.15.45:0/2095504799 >> 172.21.15.45:0/2095504799 conn(0x7f8acc09ba40 msgr2=0x7f8acc09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:08.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.780+0000 7f8ad2845640 1 -- 172.21.15.45:0/2095504799 shutdown_connections 2024-05-08T05:47:08.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.780+0000 7f8ad2845640 1 -- 172.21.15.45:0/2095504799 wait complete. 2024-05-08T05:47:08.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.781+0000 7f8ad2845640 1 Processor -- start 2024-05-08T05:47:08.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.781+0000 7f8ad2845640 1 -- start start 2024-05-08T05:47:08.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.782+0000 7f8ad2845640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8acc070b90 0x7f8acc0cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:08.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.782+0000 7f8ad2845640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8acc0d2ce0 0x7f8acc14f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:08.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.782+0000 7f8ad2845640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc15ccb0 0x7f8acc06e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:08.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.782+0000 7f8ad2845640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc071560 con 0x7f8acc15ccb0 2024-05-08T05:47:08.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.782+0000 7f8ad2845640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc0d3660 con 0x7f8acc0d2ce0 2024-05-08T05:47:08.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.782+0000 7f8ad2845640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8acc0cf980 con 0x7f8acc070b90 2024-05-08T05:47:08.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.782+0000 7f8ad0de2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc15ccb0 0x7f8acc06e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:08.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.782+0000 7f8ad0de2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc15ccb0 0x7f8acc06e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43502/0 (socket says 172.21.15.45:43502) 2024-05-08T05:47:08.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.782+0000 7f8ad0de2640 1 -- 172.21.15.45:0/3329822686 learned_addr learned my addr 172.21.15.45:0/3329822686 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:08.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.782+0000 7f8ac3fff640 1 --2- 172.21.15.45:0/3329822686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8acc0d2ce0 0x7f8acc14f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:08.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.782+0000 7f8acbfff640 1 --2- 172.21.15.45:0/3329822686 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8acc070b90 0x7f8acc0cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:08.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.782+0000 7f8ad0de2640 1 -- 172.21.15.45:0/3329822686 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8acc070b90 msgr2=0x7f8acc0cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:08.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.782+0000 7f8ad0de2640 1 --2- 172.21.15.45:0/3329822686 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8acc070b90 0x7f8acc0cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:08.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.782+0000 7f8ad0de2640 1 -- 172.21.15.45:0/3329822686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8acc0d2ce0 msgr2=0x7f8acc14f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:08.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.782+0000 7f8ad0de2640 1 --2- 172.21.15.45:0/3329822686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8acc0d2ce0 0x7f8acc14f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:08.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.782+0000 7f8ad0de2640 1 -- 172.21.15.45:0/3329822686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8acc0c9f00 con 0x7f8acc15ccb0 2024-05-08T05:47:08.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.783+0000 7f8ad0de2640 1 --2- 172.21.15.45:0/3329822686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc15ccb0 0x7f8acc06e760 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4007de0 tx=0x7f8ac400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:08.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.783+0000 7f8ac97fa640 1 -- 172.21.15.45:0/3329822686 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac401a070 con 0x7f8acc15ccb0 2024-05-08T05:47:08.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.783+0000 7f8ac97fa640 1 -- 172.21.15.45:0/3329822686 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac4003990 con 0x7f8acc15ccb0 2024-05-08T05:47:08.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.783+0000 7f8ad2845640 1 -- 172.21.15.45:0/3329822686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8acc0ca1e0 con 0x7f8acc15ccb0 2024-05-08T05:47:08.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.783+0000 7f8ac97fa640 1 -- 172.21.15.45:0/3329822686 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4003d80 con 0x7f8acc15ccb0 2024-05-08T05:47:08.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.783+0000 7f8ad2845640 1 -- 172.21.15.45:0/3329822686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8acc0ca7a0 con 0x7f8acc15ccb0 2024-05-08T05:47:08.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.784+0000 7f8ad2845640 1 -- 172.21.15.45:0/3329822686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8acc069b00 con 0x7f8acc15ccb0 2024-05-08T05:47:08.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.784+0000 7f8ac97fa640 1 -- 172.21.15.45:0/3329822686 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8ac401d070 con 0x7f8acc15ccb0 2024-05-08T05:47:08.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.785+0000 7f8ac97fa640 1 --2- 172.21.15.45:0/3329822686 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8aac0762d0 0x7f8aac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:08.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.785+0000 7f8ac97fa640 1 -- 172.21.15.45:0/3329822686 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6101+0+0 (secure 0 0 0) 0x7f8ac4097a90 con 0x7f8acc15ccb0 2024-05-08T05:47:08.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.785+0000 7f8acbfff640 1 --2- 172.21.15.45:0/3329822686 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8aac0762d0 0x7f8aac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:08.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.786+0000 7f8acbfff640 1 --2- 172.21.15.45:0/3329822686 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8aac0762d0 0x7f8aac078790 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8abc03d000 tx=0x7f8abc043040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:08.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:08.789+0000 7f8ac97fa640 1 -- 172.21.15.45:0/3329822686 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8ac4010070 con 0x7f8acc15ccb0 2024-05-08T05:47:09.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.025+0000 7f8ad2845640 1 -- 172.21.15.45:0/3329822686 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8acc06e240 con 0x7f8acc15ccb0 2024-05-08T05:47:09.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.026+0000 7f8ac97fa640 1 -- 172.21.15.45:0/3329822686 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5183 (secure 0 0 0) 0x7f8ac4097020 con 0x7f8acc15ccb0 2024-05-08T05:47:09.026 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:47:09.026 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6744,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2854804427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2854804427},{"type":"v1","addr":"172.21.15.45:6834","nonce":2854804427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2945763952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2945763952},{"type":"v1","addr":"172.21.15.45:6835","nonce":2945763952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6759,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/3194497846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":3194497846},{"type":"v1","addr":"172.21.15.63:6837","nonce":3194497846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:47:00.211837+0000","modified":"2024-05-08T05:47:03.229466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6762":{"gid":6762,"name":"d","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.63:6836/2942326473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2942326473},{"type":"v1","addr":"172.21.15.63:6836","nonce":2942326473}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6762,"qdb_cluster":[6762]},"id":6}]} 2024-05-08T05:47:09.026 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 60 2024-05-08T05:47:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.027+0000 7f8ad2845640 1 -- 172.21.15.45:0/3329822686 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8aac0762d0 msgr2=0x7f8aac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.027+0000 7f8ad2845640 1 --2- 172.21.15.45:0/3329822686 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8aac0762d0 0x7f8aac078790 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8abc03d000 tx=0x7f8abc043040 comp rx=0 tx=0).stop 2024-05-08T05:47:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.027+0000 7f8ad2845640 1 -- 172.21.15.45:0/3329822686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc15ccb0 msgr2=0x7f8acc06e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.027+0000 7f8ad2845640 1 --2- 172.21.15.45:0/3329822686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc15ccb0 0x7f8acc06e760 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4007de0 tx=0x7f8ac400d4a0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.027+0000 7f8ad2845640 1 -- 172.21.15.45:0/3329822686 shutdown_connections 2024-05-08T05:47:09.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.027+0000 7f8ad2845640 1 --2- 172.21.15.45:0/3329822686 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8aac0762d0 0x7f8aac078790 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.027+0000 7f8ad2845640 1 --2- 172.21.15.45:0/3329822686 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8acc15ccb0 0x7f8acc06e760 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.027+0000 7f8ad2845640 1 --2- 172.21.15.45:0/3329822686 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8acc0d2ce0 0x7f8acc14f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.027+0000 7f8ad2845640 1 --2- 172.21.15.45:0/3329822686 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8acc070b90 0x7f8acc0cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.027+0000 7f8ad2845640 1 -- 172.21.15.45:0/3329822686 >> 172.21.15.45:0/3329822686 conn(0x7f8acc09ba40 msgr2=0x7f8acc06d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:09.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.028+0000 7f8ad2845640 1 -- 172.21.15.45:0/3329822686 shutdown_connections 2024-05-08T05:47:09.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.028+0000 7f8ad2845640 1 -- 172.21.15.45:0/3329822686 wait complete. 2024-05-08T05:47:09.039 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-05-08T05:47:09.180 INFO:teuthology.orchestra.run.smithi063.stdout:{"message":"","return_code":0} 2024-05-08T05:47:09.181 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-08T05:47:09.181 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T05:47:09.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.286+0000 7f8933c99640 1 Processor -- start 2024-05-08T05:47:09.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.287+0000 7f8933c99640 1 -- start start 2024-05-08T05:47:09.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.287+0000 7f8933c99640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f892c1638e0 0x7f892c0b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.287+0000 7f8933c99640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f892c0b14e0 con 0x7f892c15f0c0 2024-05-08T05:47:09.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.287+0000 7f8933c99640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f892c162e00 con 0x7f892c06c3a0 2024-05-08T05:47:09.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.287+0000 7f8933c99640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f892c163450 con 0x7f892c1638e0 2024-05-08T05:47:09.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8931a35640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f892c15f0c0 0x7f892c0b0d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36842/0 (socket says 172.21.15.45:36842) 2024-05-08T05:47:09.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8931a35640 1 -- 172.21.15.45:0/3608331757 learned_addr learned my addr 172.21.15.45:0/3608331757 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:47:09.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8930a33640 1 -- 172.21.15.45:0/3608331757 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4073331543 0 0) 0x7f892c0b14e0 con 0x7f892c15f0c0 2024-05-08T05:47:09.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8931234640 1 --2- 172.21.15.45:0/3608331757 >> v2:172.21.15.63:3300/0 conn(0x7f892c1638e0 0x7f892c0b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8930a33640 1 -- 172.21.15.45:0/3608331757 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8918003520 con 0x7f892c15f0c0 2024-05-08T05:47:09.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8930a33640 1 -- 172.21.15.45:0/3608331757 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3265792773 0 0) 0x7f8918003520 con 0x7f892c15f0c0 2024-05-08T05:47:09.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8930a33640 1 -- 172.21.15.45:0/3608331757 >> v2:172.21.15.63:3300/0 conn(0x7f892c1638e0 msgr2=0x7f892c0b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:09.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8930a33640 1 --2- 172.21.15.45:0/3608331757 >> v2:172.21.15.63:3300/0 conn(0x7f892c1638e0 0x7f892c0b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8930a33640 1 -- 172.21.15.45:0/3608331757 >> v1:172.21.15.63:6789/0 conn(0x7f892c06c3a0 legacy=0x7f892c15f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8930a33640 1 -- 172.21.15.45:0/3608331757 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892c1630e0 con 0x7f892c15f0c0 2024-05-08T05:47:09.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8930a33640 1 -- 172.21.15.45:0/3608331757 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f89280016f0 con 0x7f892c15f0c0 2024-05-08T05:47:09.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8930a33640 1 -- 172.21.15.45:0/3608331757 >> v1:172.21.15.45:6789/0 conn(0x7f892c15f0c0 legacy=0x7f892c0b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8930a33640 1 --2- 172.21.15.45:0/3608331757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8918003a50 0x7f8918003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8930a33640 1 --2- 172.21.15.45:0/3608331757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8918004800 0x7f8918004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8930a33640 1 --2- 172.21.15.45:0/3608331757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8918005210 0x7f891800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8930a33640 1 -- 172.21.15.45:0/3608331757 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f892c163450 con 0x7f8918005210 2024-05-08T05:47:09.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8930a33640 1 -- 172.21.15.45:0/3608331757 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f891800c140 con 0x7f8918003a50 2024-05-08T05:47:09.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8930a33640 1 -- 172.21.15.45:0/3608331757 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f891800c340 con 0x7f8918004800 2024-05-08T05:47:09.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8932236640 1 --2- 172.21.15.45:0/3608331757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8918005210 0x7f891800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8931a35640 1 --2- 172.21.15.45:0/3608331757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8918003a50 0x7f8918003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.288+0000 7f8931234640 1 --2- 172.21.15.45:0/3608331757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8918004800 0x7f8918004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.289+0000 7f8932236640 1 -- 172.21.15.45:0/3608331757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8918004800 msgr2=0x7f8918004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:09.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.289+0000 7f8932236640 1 --2- 172.21.15.45:0/3608331757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8918004800 0x7f8918004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.289+0000 7f8932236640 1 -- 172.21.15.45:0/3608331757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8918003a50 msgr2=0x7f8918003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:09.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.289+0000 7f8932236640 1 --2- 172.21.15.45:0/3608331757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8918003a50 0x7f8918003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.289+0000 7f8932236640 1 -- 172.21.15.45:0/3608331757 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8918003520 con 0x7f8918005210 2024-05-08T05:47:09.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.289+0000 7f8932236640 1 -- 172.21.15.45:0/3608331757 reap_dead start 2024-05-08T05:47:09.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.289+0000 7f8932236640 1 --2- 172.21.15.45:0/3608331757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8918005210 0x7f891800b9c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8924001e90 tx=0x7f8918005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5405a7eb2eca15b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.290+0000 7f8930a33640 1 -- 172.21.15.45:0/3608331757 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924001ab0 con 0x7f8918005210 2024-05-08T05:47:09.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.290+0000 7f8930a33640 1 -- 172.21.15.45:0/3608331757 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892402ae50 con 0x7f8918005210 2024-05-08T05:47:09.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.290+0000 7f8930a33640 1 -- 172.21.15.45:0/3608331757 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924003bb0 con 0x7f8918005210 2024-05-08T05:47:09.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.290+0000 7f8933c99640 1 -- 172.21.15.45:0/3608331757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8918005210 msgr2=0x7f891800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.290+0000 7f8933c99640 1 --2- 172.21.15.45:0/3608331757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8918005210 0x7f891800b9c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8924001e90 tx=0x7f8918005830 comp rx=0 tx=0).stop 2024-05-08T05:47:09.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.290+0000 7f8933c99640 1 -- 172.21.15.45:0/3608331757 shutdown_connections 2024-05-08T05:47:09.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.290+0000 7f8933c99640 1 --2- 172.21.15.45:0/3608331757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8918005210 0x7f891800b9c0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.290+0000 7f8933c99640 1 -- 172.21.15.45:0/3608331757 >> 172.21.15.45:0/3608331757 conn(0x7f892c0b3b10 msgr2=0x7f892c0b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:09.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.290+0000 7f8933c99640 1 -- 172.21.15.45:0/3608331757 shutdown_connections 2024-05-08T05:47:09.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.290+0000 7f8933c99640 1 -- 172.21.15.45:0/3608331757 wait complete. 2024-05-08T05:47:09.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.291+0000 7f8933c99640 1 Processor -- start 2024-05-08T05:47:09.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.291+0000 7f8933c99640 1 -- start start 2024-05-08T05:47:09.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.292+0000 7f8933c99640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8918005210 0x7f892c1498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.292+0000 7f8933c99640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f892c155ce0 0x7f892c156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.292+0000 7f8933c99640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f892c0b0720 0x7f892c0b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.292+0000 7f8933c99640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f892c0b70c0 con 0x7f892c155ce0 2024-05-08T05:47:09.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.292+0000 7f8933c99640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f892c0b6f40 con 0x7f8918005210 2024-05-08T05:47:09.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.292+0000 7f8933c99640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f892c0b7240 con 0x7f892c0b0720 2024-05-08T05:47:09.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.292+0000 7f8931234640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f892c155ce0 0x7f892c156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.292+0000 7f8931234640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f892c155ce0 0x7f892c156140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43528/0 (socket says 172.21.15.45:43528) 2024-05-08T05:47:09.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.292+0000 7f8931234640 1 -- 172.21.15.45:0/1171061682 learned_addr learned my addr 172.21.15.45:0/1171061682 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:09.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.293+0000 7f8931234640 1 -- 172.21.15.45:0/1171061682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f892c0b0720 msgr2=0x7f892c0b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.293+0000 7f8931234640 1 --2- 172.21.15.45:0/1171061682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f892c0b0720 0x7f892c0b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.293+0000 7f8931234640 1 -- 172.21.15.45:0/1171061682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8918005210 msgr2=0x7f892c1498c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.293+0000 7f8931234640 1 --2- 172.21.15.45:0/1171061682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8918005210 0x7f892c1498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.293+0000 7f8931234640 1 -- 172.21.15.45:0/1171061682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892c154340 con 0x7f892c155ce0 2024-05-08T05:47:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.293+0000 7f8931234640 1 --2- 172.21.15.45:0/1171061682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f892c155ce0 0x7f892c156140 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f891c011910 tx=0x7f891c011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.293+0000 7f89227fc640 1 -- 172.21.15.45:0/1171061682 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c00faf0 con 0x7f892c155ce0 2024-05-08T05:47:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.293+0000 7f89227fc640 1 -- 172.21.15.45:0/1171061682 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f891c00fc90 con 0x7f892c155ce0 2024-05-08T05:47:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.293+0000 7f8933c99640 1 -- 172.21.15.45:0/1171061682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f892c154620 con 0x7f892c155ce0 2024-05-08T05:47:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.293+0000 7f89227fc640 1 -- 172.21.15.45:0/1171061682 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c014440 con 0x7f892c155ce0 2024-05-08T05:47:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.293+0000 7f8933c99640 1 -- 172.21.15.45:0/1171061682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f892c154be0 con 0x7f892c155ce0 2024-05-08T05:47:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.294+0000 7f8933c99640 1 -- 172.21.15.45:0/1171061682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f892c06a8f0 con 0x7f892c155ce0 2024-05-08T05:47:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.295+0000 7f89227fc640 1 -- 172.21.15.45:0/1171061682 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f891c012040 con 0x7f892c155ce0 2024-05-08T05:47:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.295+0000 7f89227fc640 1 --2- 172.21.15.45:0/1171061682 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f89100762d0 0x7f8910078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.295+0000 7f89227fc640 1 -- 172.21.15.45:0/1171061682 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6101+0+0 (secure 0 0 0) 0x7f891c097d90 con 0x7f892c155ce0 2024-05-08T05:47:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.295+0000 7f8931a35640 1 --2- 172.21.15.45:0/1171061682 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f89100762d0 0x7f8910078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.296+0000 7f8931a35640 1 --2- 172.21.15.45:0/1171061682 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f89100762d0 0x7f8910078790 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8928007c60 tx=0x7f8928001170 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.299+0000 7f89227fc640 1 -- 172.21.15.45:0/1171061682 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f891c019030 con 0x7f892c155ce0 2024-05-08T05:47:09.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.528+0000 7f8933c99640 1 -- 172.21.15.45:0/1171061682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f892c15d9e0 con 0x7f892c155ce0 2024-05-08T05:47:09.528 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-05-08T05:47:09.528+0000 7f124de12640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T05:47:09.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.530+0000 7f89227fc640 1 -- 172.21.15.45:0/1171061682 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v61) ==== 161+0+0 (secure 0 0 0) 0x7f891c0615f0 con 0x7f892c155ce0 2024-05-08T05:47:09.530 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T05:47:09.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.533+0000 7f8933c99640 1 -- 172.21.15.45:0/1171061682 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f89100762d0 msgr2=0x7f8910078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.533+0000 7f8933c99640 1 --2- 172.21.15.45:0/1171061682 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f89100762d0 0x7f8910078790 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8928007c60 tx=0x7f8928001170 comp rx=0 tx=0).stop 2024-05-08T05:47:09.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.533+0000 7f8933c99640 1 -- 172.21.15.45:0/1171061682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f892c155ce0 msgr2=0x7f892c156140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.533+0000 7f8933c99640 1 --2- 172.21.15.45:0/1171061682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f892c155ce0 0x7f892c156140 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f891c011910 tx=0x7f891c011dd0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.533+0000 7f8933c99640 1 -- 172.21.15.45:0/1171061682 shutdown_connections 2024-05-08T05:47:09.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.533+0000 7f8933c99640 1 --2- 172.21.15.45:0/1171061682 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f89100762d0 0x7f8910078790 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.533+0000 7f8933c99640 1 --2- 172.21.15.45:0/1171061682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f892c0b0720 0x7f892c0b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.533+0000 7f8933c99640 1 --2- 172.21.15.45:0/1171061682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f892c155ce0 0x7f892c156140 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.533+0000 7f8933c99640 1 --2- 172.21.15.45:0/1171061682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8918005210 0x7f892c1498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.533+0000 7f8933c99640 1 -- 172.21.15.45:0/1171061682 >> 172.21.15.45:0/1171061682 conn(0x7f892c0b3b10 msgr2=0x7f892c06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:09.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.533+0000 7f8933c99640 1 -- 172.21.15.45:0/1171061682 shutdown_connections 2024-05-08T05:47:09.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.533+0000 7f8933c99640 1 -- 172.21.15.45:0/1171061682 wait complete. 2024-05-08T05:47:09.536 INFO:tasks.ceph.mds.d.smithi063.stderr: -4844> 2024-05-08T05:47:06.475+0000 7fb4b3348640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:47:09.544 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-05-08T05:47:09.569 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-08T05:47:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.587+0000 7f28d993cf40 1 Processor -- start 2024-05-08T05:47:09.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.588+0000 7f28d993cf40 1 -- start start 2024-05-08T05:47:09.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.588+0000 7f28d993cf40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2a1df4e70 0x55f2a1e93760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.588+0000 7f28d993cf40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2a1e7d380 0x55f2a1e9d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.588+0000 7f28d993cf40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2a1e808a0 0x55f2a1e9dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.588+0000 7f28d993cf40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f2a1d6cf50 con 0x55f2a1e808a0 2024-05-08T05:47:09.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.588+0000 7f28d993cf40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f2a1e9bcc0 con 0x55f2a1df4e70 2024-05-08T05:47:09.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.588+0000 7f28d993cf40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f2a1e0dde0 con 0x55f2a1e7d380 2024-05-08T05:47:09.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.588+0000 7f28d7c37640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2a1e808a0 0x55f2a1e9dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.588+0000 7f28d7436640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2a1df4e70 0x55f2a1e93760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.588+0000 7f28d7c37640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2a1e808a0 0x55f2a1e9dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43536/0 (socket says 172.21.15.45:43536) 2024-05-08T05:47:09.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.588+0000 7f28d7c37640 1 -- 172.21.15.45:0/19552029 learned_addr learned my addr 172.21.15.45:0/19552029 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:09.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.588+0000 7f28cffff640 1 --2- 172.21.15.45:0/19552029 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2a1e7d380 0x55f2a1e9d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.589+0000 7f28d7c37640 1 -- 172.21.15.45:0/19552029 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2a1e7d380 msgr2=0x55f2a1e9d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:09.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.589+0000 7f28d7c37640 1 --2- 172.21.15.45:0/19552029 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2a1e7d380 0x55f2a1e9d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.589+0000 7f28d7c37640 1 -- 172.21.15.45:0/19552029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2a1df4e70 msgr2=0x55f2a1e93760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:09.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.589+0000 7f28d7c37640 1 --2- 172.21.15.45:0/19552029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2a1df4e70 0x55f2a1e93760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.589+0000 7f28d7c37640 1 -- 172.21.15.45:0/19552029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2a1e7c1b0 con 0x55f2a1e808a0 2024-05-08T05:47:09.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.589+0000 7f28d7436640 1 --2- 172.21.15.45:0/19552029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2a1df4e70 0x55f2a1e93760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:47:09.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.589+0000 7f28d7c37640 1 --2- 172.21.15.45:0/19552029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2a1e808a0 0x55f2a1e9dbd0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f28d0055040 tx=0x7f28d0051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7183a88d2f99b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.589+0000 7f28d6434640 1 -- 172.21.15.45:0/19552029 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d00536e0 con 0x55f2a1e808a0 2024-05-08T05:47:09.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.589+0000 7f28d6434640 1 -- 172.21.15.45:0/19552029 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d0053880 con 0x55f2a1e808a0 2024-05-08T05:47:09.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.589+0000 7f28d6434640 1 -- 172.21.15.45:0/19552029 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0053c70 con 0x55f2a1e808a0 2024-05-08T05:47:09.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.590+0000 7f28d993cf40 1 -- 172.21.15.45:0/19552029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2a1e808a0 msgr2=0x55f2a1e9dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.590+0000 7f28d993cf40 1 --2- 172.21.15.45:0/19552029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2a1e808a0 0x55f2a1e9dbd0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f28d0055040 tx=0x7f28d0051000 comp rx=0 tx=0).stop 2024-05-08T05:47:09.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.590+0000 7f28d993cf40 1 -- 172.21.15.45:0/19552029 shutdown_connections 2024-05-08T05:47:09.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.590+0000 7f28d993cf40 1 --2- 172.21.15.45:0/19552029 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2a1e7d380 0x55f2a1e9d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.590+0000 7f28d993cf40 1 --2- 172.21.15.45:0/19552029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2a1e808a0 0x55f2a1e9dbd0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.590+0000 7f28d993cf40 1 --2- 172.21.15.45:0/19552029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2a1df4e70 0x55f2a1e93760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.590+0000 7f28d993cf40 1 -- 172.21.15.45:0/19552029 >> 172.21.15.45:0/19552029 conn(0x55f2a1dc47e0 msgr2=0x55f2a1e9cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:09.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.590+0000 7f28d993cf40 1 -- 172.21.15.45:0/19552029 shutdown_connections 2024-05-08T05:47:09.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.590+0000 7f28d993cf40 1 -- 172.21.15.45:0/19552029 wait complete. 2024-05-08T05:47:09.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.590+0000 7f28d993cf40 1 Processor -- start 2024-05-08T05:47:09.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.590+0000 7f28d993cf40 1 -- start start 2024-05-08T05:47:09.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.591+0000 7f28d993cf40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2a1df4e70 0x55f2a1dc4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.591+0000 7f28d993cf40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2a1e7d380 0x55f2a1e9a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.591+0000 7f28d993cf40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2a1e808a0 0x55f2a1f33a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.591+0000 7f28d993cf40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f2a1e0da70 con 0x55f2a1e808a0 2024-05-08T05:47:09.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.591+0000 7f28d993cf40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f2a1e94130 con 0x55f2a1df4e70 2024-05-08T05:47:09.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.591+0000 7f28d993cf40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f2a1e94550 con 0x55f2a1e7d380 2024-05-08T05:47:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.591+0000 7f28d7c37640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2a1e808a0 0x55f2a1f33a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.591+0000 7f28d7c37640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2a1e808a0 0x55f2a1f33a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43544/0 (socket says 172.21.15.45:43544) 2024-05-08T05:47:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.591+0000 7f28d7c37640 1 -- 172.21.15.45:0/2945625443 learned_addr learned my addr 172.21.15.45:0/2945625443 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.591+0000 7f28d7436640 1 --2- 172.21.15.45:0/2945625443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2a1df4e70 0x55f2a1dc4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.591+0000 7f28cffff640 1 --2- 172.21.15.45:0/2945625443 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2a1e7d380 0x55f2a1e9a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.591+0000 7f28d7c37640 1 -- 172.21.15.45:0/2945625443 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2a1e7d380 msgr2=0x55f2a1e9a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.591+0000 7f28d7c37640 1 --2- 172.21.15.45:0/2945625443 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f2a1e7d380 0x55f2a1e9a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.591+0000 7f28d7c37640 1 -- 172.21.15.45:0/2945625443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2a1df4e70 msgr2=0x55f2a1dc4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.591+0000 7f28d7c37640 1 --2- 172.21.15.45:0/2945625443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f2a1df4e70 0x55f2a1dc4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.591+0000 7f28d7c37640 1 -- 172.21.15.45:0/2945625443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2a1e0cfb0 con 0x55f2a1e808a0 2024-05-08T05:47:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.592+0000 7f28d7c37640 1 --2- 172.21.15.45:0/2945625443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2a1e808a0 0x55f2a1f33a30 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f28d0045f60 tx=0x7f28d0003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.592+0000 7f28d5432640 1 -- 172.21.15.45:0/2945625443 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0073070 con 0x55f2a1e808a0 2024-05-08T05:47:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.592+0000 7f28d5432640 1 -- 172.21.15.45:0/2945625443 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28d0053a30 con 0x55f2a1e808a0 2024-05-08T05:47:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.592+0000 7f28d5432640 1 -- 172.21.15.45:0/2945625443 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0053de0 con 0x55f2a1e808a0 2024-05-08T05:47:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.592+0000 7f28d993cf40 1 -- 172.21.15.45:0/2945625443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f2a1e0d290 con 0x55f2a1e808a0 2024-05-08T05:47:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.592+0000 7f28d993cf40 1 -- 172.21.15.45:0/2945625443 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2a1f32f30 con 0x55f2a1e808a0 2024-05-08T05:47:09.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.593+0000 7f28d5432640 1 -- 172.21.15.45:0/2945625443 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f28d0076070 con 0x55f2a1e808a0 2024-05-08T05:47:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.594+0000 7f28d5432640 1 --2- 172.21.15.45:0/2945625443 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f28b8074cb0 0x7f28b8077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.594+0000 7f28d5432640 1 -- 172.21.15.45:0/2945625443 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6187+0+0 (secure 0 0 0) 0x7f28d00dade0 con 0x55f2a1e808a0 2024-05-08T05:47:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.594+0000 7f28d7436640 1 --2- 172.21.15.45:0/2945625443 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f28b8074cb0 0x7f28b8077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.594+0000 7f28d993cf40 1 --2- 172.21.15.45:0/2945625443 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55f2a1e7b680 0x55f2a1e78f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.594+0000 7f28d993cf40 1 -- 172.21.15.45:0/2945625443 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89) -- 0x55f2a1e794c0 con 0x55f2a1e7b680 2024-05-08T05:47:09.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.594+0000 7f28cffff640 1 --2- 172.21.15.45:0/2945625443 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55f2a1e7b680 0x55f2a1e78f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.595+0000 7f28d7436640 1 --2- 172.21.15.45:0/2945625443 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f28b8074cb0 0x7f28b8077170 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x55f2a1e78b90 tx=0x7f28c8001ac0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.595+0000 7f28cffff640 1 --2- 172.21.15.45:0/2945625443 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55f2a1e7b680 0x55f2a1e78f80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.597+0000 7f28cffff640 1 -- 172.21.15.45:0/2945625443 <== osd.1 v2:172.21.15.45:6824/1337179184 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v89'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f28c0009b60 con 0x55f2a1e7b680 2024-05-08T05:47:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.598+0000 7f28d993cf40 1 -- 172.21.15.45:0/2945625443 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55f2a1e7b680 msgr2=0x55f2a1e78f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.598+0000 7f28d993cf40 1 --2- 172.21.15.45:0/2945625443 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55f2a1e7b680 0x55f2a1e78f80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.598+0000 7f28d993cf40 1 -- 172.21.15.45:0/2945625443 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f28b8074cb0 msgr2=0x7f28b8077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.598+0000 7f28d993cf40 1 --2- 172.21.15.45:0/2945625443 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f28b8074cb0 0x7f28b8077170 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x55f2a1e78b90 tx=0x7f28c8001ac0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.598+0000 7f28d993cf40 1 -- 172.21.15.45:0/2945625443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2a1e808a0 msgr2=0x55f2a1f33a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.598+0000 7f28d993cf40 1 --2- 172.21.15.45:0/2945625443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f2a1e808a0 0x55f2a1f33a30 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f28d0045f60 tx=0x7f28d0003330 comp rx=0 tx=0).stop 2024-05-08T05:47:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.598+0000 7f28d7c37640 1 -- 172.21.15.45:0/2945625443 reap_dead start 2024-05-08T05:47:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.598+0000 7f28d993cf40 1 -- 172.21.15.45:0/2945625443 shutdown_connections 2024-05-08T05:47:09.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.598+0000 7f28d993cf40 1 -- 172.21.15.45:0/2945625443 >> 172.21.15.45:0/2945625443 conn(0x55f2a1dc47e0 msgr2=0x55f2a1e7f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.598+0000 7f28d993cf40 1 -- 172.21.15.45:0/2945625443 shutdown_connections 2024-05-08T05:47:09.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.598+0000 7f28d993cf40 1 -- 172.21.15.45:0/2945625443 wait complete. 2024-05-08T05:47:09.610 DEBUG:teuthology.orchestra.run.smithi045:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --yes-i-really-really-mean-it 2024-05-08T05:47:09.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.655+0000 7f229d618280 1 -- start start 2024-05-08T05:47:09.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.655+0000 7f229d618280 1 Processor -- start 2024-05-08T05:47:09.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.656+0000 7f229d618280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56250d340c00 0x56250e112000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.656+0000 7f229d618280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250d341800 0x56250e112580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.656+0000 7f229d618280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250d341c00 0x56250e112b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.656+0000 7f229d618280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56250e0de600 con 0x56250d341c00 2024-05-08T05:47:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.656+0000 7f229d618280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56250e0de480 con 0x56250d341800 2024-05-08T05:47:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.656+0000 7f229d618280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56250e0de300 con 0x56250d340c00 2024-05-08T05:47:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.656+0000 7f229b974640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250d341c00 0x56250e112b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.656+0000 7f229b974640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250d341c00 0x56250e112b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43548/0 (socket says 172.21.15.45:43548) 2024-05-08T05:47:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.656+0000 7f229b974640 1 -- 172.21.15.45:0/343284133 learned_addr learned my addr 172.21.15.45:0/343284133 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.656+0000 7f229b173640 1 --2- 172.21.15.45:0/343284133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56250d340c00 0x56250e112000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.656+0000 7f229a972640 1 --2- 172.21.15.45:0/343284133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250d341800 0x56250e112580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.656+0000 7f229b974640 1 -- 172.21.15.45:0/343284133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56250d340c00 msgr2=0x56250e112000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.656+0000 7f229b974640 1 --2- 172.21.15.45:0/343284133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56250d340c00 0x56250e112000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.656+0000 7f229b974640 1 -- 172.21.15.45:0/343284133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250d341800 msgr2=0x56250e112580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.656+0000 7f229b974640 1 --2- 172.21.15.45:0/343284133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250d341800 0x56250e112580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.657+0000 7f229b974640 1 -- 172.21.15.45:0/343284133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56250d376780 con 0x56250d341c00 2024-05-08T05:47:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.657+0000 7f229b173640 1 --2- 172.21.15.45:0/343284133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56250d340c00 0x56250e112000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:47:09.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.657+0000 7f229b974640 1 --2- 172.21.15.45:0/343284133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250d341c00 0x56250e112b00 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x56250e1698c0 tx=0x56250e0e2360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f20f795de5087b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.657+0000 7f229896e640 1 -- 172.21.15.45:0/343284133 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56250d366680 con 0x56250d341c00 2024-05-08T05:47:09.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.657+0000 7f229896e640 1 -- 172.21.15.45:0/343284133 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56250d36ce00 con 0x56250d341c00 2024-05-08T05:47:09.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.657+0000 7f229896e640 1 -- 172.21.15.45:0/343284133 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56250d3669c0 con 0x56250d341c00 2024-05-08T05:47:09.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.657+0000 7f229d618280 1 -- 172.21.15.45:0/343284133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56250d3763c0 con 0x56250d341c00 2024-05-08T05:47:09.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.657+0000 7f229d618280 1 -- 172.21.15.45:0/343284133 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=0}) -- 0x56250d377860 con 0x56250d341c00 2024-05-08T05:47:09.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.657+0000 7f229d618280 4 waiting for MDS map... 2024-05-08T05:47:09.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.657+0000 7f229896e640 1 -- 172.21.15.45:0/343284133 <== mon.0 v2:172.21.15.45:3300/0 4 ==== osd_map(89..89 src has 1..89) ==== 6187+0+0 (secure 0 0 0) 0x56250e212600 con 0x56250d341c00 2024-05-08T05:47:09.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.657+0000 7f229896e640 1 -- 172.21.15.45:0/343284133 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 61) ==== 2885+0+0 (secure 0 0 0) 0x56250d39b180 con 0x56250d341c00 2024-05-08T05:47:09.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.657+0000 7f229d618280 4 Got MDS map 61 2024-05-08T05:47:09.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.657+0000 7f229d618280 10 main: JournalTool::main 2024-05-08T05:47:09.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.657+0000 7f229d618280 4 main: JournalTool: connecting to RADOS... 2024-05-08T05:47:09.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.658+0000 7f229d618280 1 Processor -- start 2024-05-08T05:47:09.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.658+0000 7f229d618280 1 -- start start 2024-05-08T05:47:09.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.658+0000 7f229d618280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250e215400 0x56250e113080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.658+0000 7f229d618280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e215800 0x56250e113600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.658+0000 7f229d618280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56250e215c00 0x56250e113b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.658+0000 7f229d618280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56250e0de180 con 0x56250e215800 2024-05-08T05:47:09.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.658+0000 7f229d618280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56250d3a2480 con 0x56250e215400 2024-05-08T05:47:09.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.658+0000 7f229d618280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56250d3a2900 con 0x56250e215c00 2024-05-08T05:47:09.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.659+0000 7f229b173640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e215800 0x56250e113600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.659+0000 7f229b173640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e215800 0x56250e113600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43550/0 (socket says 172.21.15.45:43550) 2024-05-08T05:47:09.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.659+0000 7f229b173640 1 -- 172.21.15.45:0/829520770 learned_addr learned my addr 172.21.15.45:0/829520770 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:09.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.659+0000 7f229a972640 1 --2- 172.21.15.45:0/829520770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250e215400 0x56250e113080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.659+0000 7f229a972640 1 --2- 172.21.15.45:0/829520770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56250e215c00 0x56250e113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.659+0000 7f229b173640 1 -- 172.21.15.45:0/829520770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56250e215c00 msgr2=0x56250e113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:09.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.659+0000 7f229b173640 1 --2- 172.21.15.45:0/829520770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56250e215c00 0x56250e113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.659+0000 7f229b173640 1 -- 172.21.15.45:0/829520770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250e215400 msgr2=0x56250e113080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:09.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.659+0000 7f229b173640 1 --2- 172.21.15.45:0/829520770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250e215400 0x56250e113080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.659+0000 7f229b173640 1 -- 172.21.15.45:0/829520770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56250d377680 con 0x56250e215800 2024-05-08T05:47:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.659+0000 7f229b173640 1 --2- 172.21.15.45:0/829520770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e215800 0x56250e113600 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x56250e169cb0 tx=0x56250e234cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac52a0c418507e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.659+0000 7f229696a640 1 -- 172.21.15.45:0/829520770 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56250d366680 con 0x56250e215800 2024-05-08T05:47:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.659+0000 7f229696a640 1 -- 172.21.15.45:0/829520770 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56250d36cc40 con 0x56250e215800 2024-05-08T05:47:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.659+0000 7f229696a640 1 -- 172.21.15.45:0/829520770 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56250d3669c0 con 0x56250e215800 2024-05-08T05:47:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.659+0000 7f229d618280 1 -- 172.21.15.45:0/829520770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e215800 msgr2=0x56250e113600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.659+0000 7f229d618280 1 --2- 172.21.15.45:0/829520770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e215800 0x56250e113600 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x56250e169cb0 tx=0x56250e234cc0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.660+0000 7f229d618280 1 -- 172.21.15.45:0/829520770 shutdown_connections 2024-05-08T05:47:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.660+0000 7f229d618280 1 --2- 172.21.15.45:0/829520770 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56250e215c00 0x56250e113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.660+0000 7f229d618280 1 --2- 172.21.15.45:0/829520770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e215800 0x56250e113600 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.660+0000 7f229d618280 1 --2- 172.21.15.45:0/829520770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250e215400 0x56250e113080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.660+0000 7f229d618280 1 -- 172.21.15.45:0/829520770 >> 172.21.15.45:0/829520770 conn(0x56250e215000 msgr2=0x56250d438800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.660+0000 7f229d618280 1 -- 172.21.15.45:0/829520770 shutdown_connections 2024-05-08T05:47:09.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.660+0000 7f229d618280 1 -- 172.21.15.45:0/829520770 wait complete. 2024-05-08T05:47:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.660+0000 7f229d618280 1 Processor -- start 2024-05-08T05:47:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.660+0000 7f229d618280 1 -- start start 2024-05-08T05:47:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f229d618280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250e215400 0x56250e113080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f229d618280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56250e215800 0x56250e113600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f229d618280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e215c00 0x56250e113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f229d618280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56250e0df380 con 0x56250e215c00 2024-05-08T05:47:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f229d618280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56250e0de180 con 0x56250e215400 2024-05-08T05:47:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f229d618280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56250e0df500 con 0x56250e215800 2024-05-08T05:47:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f229a972640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e215c00 0x56250e113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f229a972640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e215c00 0x56250e113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43552/0 (socket says 172.21.15.45:43552) 2024-05-08T05:47:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f229a972640 1 -- 172.21.15.45:0/273307743 learned_addr learned my addr 172.21.15.45:0/273307743 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:09.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f229a972640 1 --2- 172.21.15.45:0/273307743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250e215400 0x56250e113080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f229b173640 1 --2- 172.21.15.45:0/273307743 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56250e215800 0x56250e113600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f229a972640 1 -- 172.21.15.45:0/273307743 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56250e215800 msgr2=0x56250e113600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f229a972640 1 --2- 172.21.15.45:0/273307743 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56250e215800 0x56250e113600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f229a972640 1 -- 172.21.15.45:0/273307743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250e215400 msgr2=0x56250e113080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f229a972640 1 --2- 172.21.15.45:0/273307743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250e215400 0x56250e113080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f229a972640 1 -- 172.21.15.45:0/273307743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56250d376780 con 0x56250e215c00 2024-05-08T05:47:09.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f229a972640 1 --2- 172.21.15.45:0/273307743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e215c00 0x56250e113b80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x56250e231770 tx=0x56250e0e0720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f2295167640 1 -- 172.21.15.45:0/273307743 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56250d366680 con 0x56250e215c00 2024-05-08T05:47:09.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f2295167640 1 -- 172.21.15.45:0/273307743 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56250d36c8c0 con 0x56250e215c00 2024-05-08T05:47:09.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f2295167640 1 -- 172.21.15.45:0/273307743 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56250d3669c0 con 0x56250e215c00 2024-05-08T05:47:09.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f229d618280 1 -- 172.21.15.45:0/273307743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56250d3763c0 con 0x56250e215c00 2024-05-08T05:47:09.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f229d618280 1 -- 172.21.15.45:0/273307743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56250d377860 con 0x56250e215c00 2024-05-08T05:47:09.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.661+0000 7f229d618280 4 main: JournalTool: resolving pool 13 2024-05-08T05:47:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.663+0000 7f2295167640 1 -- 172.21.15.45:0/273307743 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x56250e2a0c00 con 0x56250e215c00 2024-05-08T05:47:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.663+0000 7f2295167640 1 --2- 172.21.15.45:0/273307743 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x56250e298c00 0x56250e114100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.663+0000 7f2295167640 1 -- 172.21.15.45:0/273307743 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6187+0+0 (secure 0 0 0) 0x56250e29ea00 con 0x56250e215c00 2024-05-08T05:47:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.663+0000 7f229d618280 4 main: JournalTool: creating IoCtx.. 2024-05-08T05:47:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.663+0000 7f229d618280 4 main: Executing for rank 0 2024-05-08T05:47:09.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.663+0000 7f229b173640 1 --2- 172.21.15.45:0/273307743 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x56250e298c00 0x56250e114100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.664+0000 7f229b173640 1 --2- 172.21.15.45:0/273307743 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x56250e298c00 0x56250e114100 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x56250e169cb0 tx=0x56250e234c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.664+0000 7f229d618280 1 -- start start 2024-05-08T05:47:09.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.664+0000 7f229d618280 1 Processor -- start 2024-05-08T05:47:09.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f229d618280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250e299400 0x56250e114680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f229d618280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56250e299800 0x56250e114c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f229d618280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e299c00 0x56250e115180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f229d618280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56250e0df800 con 0x56250e299c00 2024-05-08T05:47:09.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f229d618280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56250e0df980 con 0x56250e299400 2024-05-08T05:47:09.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f229d618280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56250d3a2900 con 0x56250e299800 2024-05-08T05:47:09.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f229a972640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e299c00 0x56250e115180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f229a972640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e299c00 0x56250e115180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43558/0 (socket says 172.21.15.45:43558) 2024-05-08T05:47:09.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f229a972640 1 -- 172.21.15.45:0/3547340449 learned_addr learned my addr 172.21.15.45:0/3547340449 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:09.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f229b173640 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56250e299800 0x56250e114c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f229b974640 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250e299400 0x56250e114680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f229a972640 1 -- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56250e299800 msgr2=0x56250e114c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:09.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f229a972640 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56250e299800 0x56250e114c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f229a972640 1 -- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250e299400 msgr2=0x56250e114680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:09.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f229a972640 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250e299400 0x56250e114680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f229a972640 1 -- 172.21.15.45:0/3547340449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56250d3774a0 con 0x56250e299c00 2024-05-08T05:47:09.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f229a972640 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e299c00 0x56250e115180 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x56250e2c97a0 tx=0x56250d66c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd1be22f2ee45f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f2292161640 1 -- 172.21.15.45:0/3547340449 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56250d669860 con 0x56250e299c00 2024-05-08T05:47:09.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f2292161640 1 -- 172.21.15.45:0/3547340449 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56250d36c540 con 0x56250e299c00 2024-05-08T05:47:09.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56250d376960 con 0x56250e299c00 2024-05-08T05:47:09.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=0}) -- 0x56250d377a40 con 0x56250e299c00 2024-05-08T05:47:09.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f229d618280 4 waiting for MDS map... 2024-05-08T05:47:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f2292161640 1 -- 172.21.15.45:0/3547340449 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56250d669a00 con 0x56250e299c00 2024-05-08T05:47:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.665+0000 7f2292161640 1 -- 172.21.15.45:0/3547340449 <== mon.0 v2:172.21.15.45:3300/0 4 ==== osd_map(89..89 src has 1..89) ==== 6187+0+0 (secure 0 0 0) 0x56250d73a400 con 0x56250e299c00 2024-05-08T05:47:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.666+0000 7f2292161640 1 -- 172.21.15.45:0/3547340449 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 61) ==== 2885+0+0 (secure 0 0 0) 0x56250d39b500 con 0x56250e299c00 2024-05-08T05:47:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.666+0000 7f229d618280 4 Got MDS map 61 2024-05-08T05:47:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.666+0000 7f229d618280 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x56250d73d800 0x56250e115700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.666+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56250d73dc00 con 0x56250d73d800 2024-05-08T05:47:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.666+0000 7f229b974640 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x56250d73d800 0x56250e115700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.666+0000 7f229b974640 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x56250d73d800 0x56250e115700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=943fc880fe0c5be6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.669+0000 7f229b974640 1 -- 172.21.15.45:0/3547340449 <== osd.6 v2:172.21.15.63:6800/2926759052 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x56250d44e000 con 0x56250d73d800 2024-05-08T05:47:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.669+0000 7f229d618280 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x56250d75e000 0x56250d764000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.669+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:2 13.14 13:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56250d75e400 con 0x56250d75e000 2024-05-08T05:47:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.669+0000 7f229b173640 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x56250d75e000 0x56250d764000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.669+0000 7f229b173640 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x56250d75e000 0x56250d764000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=71f3116ad22e870f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.669+0000 7f229b173640 1 -- 172.21.15.45:0/3547340449 <== osd.0 v2:172.21.15.45:6800/3734454393 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x56250d44e500 con 0x56250d75e000 2024-05-08T05:47:09.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.669+0000 7f229095e640 1 -- 172.21.15.45:0/3547340449 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:3 13.14 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e89) -- 0x56250d75fc00 con 0x56250d75e000 2024-05-08T05:47:09.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.669+0000 7f229095e640 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x56250d768000 0x56250d765b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.669+0000 7f229095e640 1 -- 172.21.15.45:0/3547340449 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:4 13.1b 13:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e89) -- 0x56250d768400 con 0x56250d768000 2024-05-08T05:47:09.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.670+0000 7f229b974640 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x56250d768000 0x56250d765b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.670+0000 7f229b173640 1 -- 172.21.15.45:0/3547340449 <== osd.0 v2:172.21.15.45:6800/3734454393 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x56250d44e500 con 0x56250d75e000 2024-05-08T05:47:09.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.670+0000 7f229b974640 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x56250d768000 0x56250d765b80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=831282405c02b87d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.670+0000 7f229b974640 1 -- 172.21.15.45:0/3547340449 <== osd.4 v2:172.21.15.63:6824/2212879151 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56250d44e000 con 0x56250d768000 2024-05-08T05:47:09.679 INFO:teuthology.orchestra.run.smithi045.stdout:old journal was 4212139~2605 2024-05-08T05:47:09.679 INFO:teuthology.orchestra.run.smithi045.stdout:new journal start will be 8388608 (4173864 bytes past old end) 2024-05-08T05:47:09.679 INFO:teuthology.orchestra.run.smithi045.stdout:writing journal head 2024-05-08T05:47:09.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.670+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:5 13.14 13:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x56250d75e800 con 0x56250d75e000 2024-05-08T05:47:09.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.672+0000 7f229b173640 1 -- 172.21.15.45:0/3547340449 <== osd.0 v2:172.21.15.45:6800/3734454393 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v89'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x56250d44e500 con 0x56250d75e000 2024-05-08T05:47:09.680 INFO:teuthology.orchestra.run.smithi045.stdout:writing EResetJournal entry 2024-05-08T05:47:09.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.672+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:6 13.1b 13:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x56250d75ec00 con 0x56250d768000 2024-05-08T05:47:09.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.672+0000 7f229d618280 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x56250d75f000 0x56250d764580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.672+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:7 13.b 13:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x56250d75f400 con 0x56250d75f000 2024-05-08T05:47:09.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.672+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:8 13.1f 13:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x56250d75f800 con 0x56250d73d800 2024-05-08T05:47:09.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.672+0000 7f229d618280 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x56250d768800 0x56250d764b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.672+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:9 13.1a 13:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x56250d768c00 con 0x56250d768800 2024-05-08T05:47:09.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.672+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:10 13.1e 13:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x56250d769000 con 0x56250d768000 2024-05-08T05:47:09.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.672+0000 7f229d618280 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x56250d769400 0x56250d765080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.672+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:11 13.1d 13:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x56250d75e400 con 0x56250d769400 2024-05-08T05:47:09.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.672+0000 7f229a972640 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x56250d768800 0x56250d764b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.672+0000 7f229b173640 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x56250d75f000 0x56250d764580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.672+0000 7f229b974640 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x56250d769400 0x56250d765080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.672+0000 7f229a972640 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x56250d768800 0x56250d764b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=c5e1f881d57f7566 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.673+0000 7f229b173640 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x56250d75f000 0x56250d764580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=421fa844d042fbb9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.673+0000 7f229b974640 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x56250d769400 0x56250d765080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=1b803c5f91f6aff7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.673+0000 7f229b974640 1 -- 172.21.15.45:0/3547340449 <== osd.6 v2:172.21.15.63:6800/2926759052 2 ==== osd_op_reply(8 200.00000004 [delete] v89'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56250d44e000 con 0x56250d73d800 2024-05-08T05:47:09.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.673+0000 7f229b974640 1 -- 172.21.15.45:0/3547340449 <== osd.4 v2:172.21.15.63:6824/2212879151 2 ==== osd_op_reply(10 200.00000006 [delete] v89'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56250d44e000 con 0x56250d768000 2024-05-08T05:47:09.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.673+0000 7f229b974640 1 -- 172.21.15.45:0/3547340449 <== osd.4 v2:172.21.15.63:6824/2212879151 3 ==== osd_op_reply(6 200.00000002 [delete] v89'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56250d44e000 con 0x56250d768000 2024-05-08T05:47:09.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.674+0000 7f229a972640 1 -- 172.21.15.45:0/3547340449 <== osd.1 v2:172.21.15.45:6824/1337179184 1 ==== osd_op_reply(9 200.00000005 [delete] v89'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56250d44e780 con 0x56250d768800 2024-05-08T05:47:09.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.676+0000 7f229b173640 1 -- 172.21.15.45:0/3547340449 <== osd.5 v2:172.21.15.63:6808/2395798255 1 ==== osd_op_reply(7 200.00000003 [delete] v89'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56250d44e500 con 0x56250d75f000 2024-05-08T05:47:09.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.676+0000 7f229095e640 1 -- 172.21.15.45:0/3547340449 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:12 13.1b 13:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x56250d75ec00 con 0x56250d768000 2024-05-08T05:47:09.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229b974640 1 -- 172.21.15.45:0/3547340449 <== osd.7 v2:172.21.15.63:6816/1163469437 1 ==== osd_op_reply(11 200.00000007 [delete] v89'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56250d44e000 con 0x56250d769400 2024-05-08T05:47:09.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229b974640 1 -- 172.21.15.45:0/3547340449 <== osd.4 v2:172.21.15.63:6824/2212879151 4 ==== osd_op_reply(12 200.00000002 [write 0~48] v89'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x56250d44e000 con 0x56250d768000 2024-05-08T05:47:09.684 INFO:teuthology.orchestra.run.smithi045.stdout:done 2024-05-08T05:47:09.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x56250d75e000 msgr2=0x56250d764000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x56250d75e000 0x56250d764000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x56250d768800 msgr2=0x56250d764b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x56250d768800 0x56250d764b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x56250d768000 msgr2=0x56250d765b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x56250d768000 0x56250d765b80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x56250d75f000 msgr2=0x56250d764580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x56250d75f000 0x56250d764580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x56250d73d800 msgr2=0x56250e115700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x56250d73d800 0x56250e115700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x56250d769400 msgr2=0x56250d765080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x56250d769400 0x56250d765080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229a972640 1 -- 172.21.15.45:0/3547340449 reap_dead start 2024-05-08T05:47:09.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e299c00 msgr2=0x56250e115180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e299c00 0x56250e115180 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x56250e2c97a0 tx=0x56250d66c180 comp rx=0 tx=0).stop 2024-05-08T05:47:09.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 shutdown_connections 2024-05-08T05:47:09.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 --2- 172.21.15.45:0/3547340449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e299c00 0x56250e115180 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 >> 172.21.15.45:0/3547340449 conn(0x56250e299000 msgr2=0x56250d438000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:09.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 shutdown_connections 2024-05-08T05:47:09.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 -- 172.21.15.45:0/3547340449 wait complete. 2024-05-08T05:47:09.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 -- 172.21.15.45:0/273307743 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x56250e298c00 msgr2=0x56250e114100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 --2- 172.21.15.45:0/273307743 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x56250e298c00 0x56250e114100 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x56250e169cb0 tx=0x56250e234c00 comp rx=0 tx=0).stop 2024-05-08T05:47:09.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 -- 172.21.15.45:0/273307743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e215c00 msgr2=0x56250e113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.677+0000 7f229d618280 1 --2- 172.21.15.45:0/273307743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e215c00 0x56250e113b80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x56250e231770 tx=0x56250e0e0720 comp rx=0 tx=0).stop 2024-05-08T05:47:09.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.678+0000 7f229d618280 1 -- 172.21.15.45:0/273307743 shutdown_connections 2024-05-08T05:47:09.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.678+0000 7f229d618280 1 --2- 172.21.15.45:0/273307743 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x56250e298c00 0x56250e114100 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.678+0000 7f229d618280 1 --2- 172.21.15.45:0/273307743 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56250e215800 0x56250e113600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.678+0000 7f229d618280 1 --2- 172.21.15.45:0/273307743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250e215c00 0x56250e113b80 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.678+0000 7f229d618280 1 --2- 172.21.15.45:0/273307743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250e215400 0x56250e113080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.678+0000 7f229d618280 1 -- 172.21.15.45:0/273307743 >> 172.21.15.45:0/273307743 conn(0x56250e215000 msgr2=0x56250d438800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:09.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.678+0000 7f229d618280 1 -- 172.21.15.45:0/273307743 shutdown_connections 2024-05-08T05:47:09.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.678+0000 7f229d618280 1 -- 172.21.15.45:0/273307743 wait complete. 2024-05-08T05:47:09.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.678+0000 7f229d618280 1 -- 172.21.15.45:0/343284133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250d341c00 msgr2=0x56250e112b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.678+0000 7f229d618280 1 --2- 172.21.15.45:0/343284133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250d341c00 0x56250e112b00 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x56250e1698c0 tx=0x56250e0e2360 comp rx=0 tx=0).stop 2024-05-08T05:47:09.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.678+0000 7f229d618280 1 -- 172.21.15.45:0/343284133 shutdown_connections 2024-05-08T05:47:09.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.678+0000 7f229d618280 1 --2- 172.21.15.45:0/343284133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56250d341c00 0x56250e112b00 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.678+0000 7f229d618280 1 --2- 172.21.15.45:0/343284133 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56250d341800 0x56250e112580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.678+0000 7f229d618280 1 --2- 172.21.15.45:0/343284133 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56250d340c00 0x56250e112000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.678+0000 7f229d618280 1 -- 172.21.15.45:0/343284133 >> 172.21.15.45:0/343284133 conn(0x56250d341000 msgr2=0x56250d439000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:09.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.678+0000 7f229d618280 1 -- 172.21.15.45:0/343284133 shutdown_connections 2024-05-08T05:47:09.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.678+0000 7f229d618280 1 -- 172.21.15.45:0/343284133 wait complete. 2024-05-08T05:47:09.691 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--yes-i-really-really-mean-it'] in time 0:00:00.080410, result: old journal was 4212139~2605 new journal start will be 8388608 (4173864 bytes past old end) writing journal head writing EResetJournal entry done 2024-05-08T05:47:09.691 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-08T05:47:09.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.797+0000 7f623c647640 1 Processor -- start 2024-05-08T05:47:09.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.797+0000 7f623c647640 1 -- start start 2024-05-08T05:47:09.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.798+0000 7f623c647640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f623406e0f0 0x7f623415c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.798+0000 7f623c647640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6234072840 0x7f62340d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.798+0000 7f623c647640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62340d14b0 con 0x7f623406e4d0 2024-05-08T05:47:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.798+0000 7f623c647640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f623415c920 con 0x7f6234072840 2024-05-08T05:47:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.798+0000 7f623c647640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f623415cc00 con 0x7f623406e0f0 2024-05-08T05:47:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.798+0000 7f6239be2640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6234072840 0x7f62340d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.798+0000 7f623abe4640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f623406e4d0 0x7f62340b4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:36870/0 (socket says 172.21.15.45:36870) 2024-05-08T05:47:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.798+0000 7f6239be2640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6234072840 0x7f62340d0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43570/0 (socket says 172.21.15.45:43570) 2024-05-08T05:47:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.798+0000 7f623abe4640 1 -- 172.21.15.45:0/1785472857 learned_addr learned my addr 172.21.15.45:0/1785472857 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:47:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.798+0000 7f623a3e3640 1 --2- 172.21.15.45:0/1785472857 >> v2:172.21.15.63:3300/0 conn(0x7f623406e0f0 0x7f623415c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.798+0000 7f6239be2640 1 -- 172.21.15.45:0/1785472857 >> v2:172.21.15.63:3300/0 conn(0x7f623406e0f0 msgr2=0x7f623415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.798+0000 7f6239be2640 1 --2- 172.21.15.45:0/1785472857 >> v2:172.21.15.63:3300/0 conn(0x7f623406e0f0 0x7f623415c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.798+0000 7f6239be2640 1 -- 172.21.15.45:0/1785472857 >> v1:172.21.15.45:6789/0 conn(0x7f623406e4d0 legacy=0x7f62340b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.798+0000 7f6239be2640 1 -- 172.21.15.45:0/1785472857 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623415cee0 con 0x7f6234072840 2024-05-08T05:47:09.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.798+0000 7f62393e1640 1 -- 172.21.15.45:0/1785472857 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1833528323 0 0) 0x7f62340d14b0 con 0x7f623406e4d0 2024-05-08T05:47:09.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.799+0000 7f6239be2640 1 --2- 172.21.15.45:0/1785472857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6234072840 0x7f62340d0f70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f62340d1cf0 tx=0x7f622802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2fb2f2845fa563 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.799+0000 7f62393e1640 1 -- 172.21.15.45:0/1785472857 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6228037070 con 0x7f6234072840 2024-05-08T05:47:09.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.799+0000 7f62393e1640 1 -- 172.21.15.45:0/1785472857 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6228032440 con 0x7f6234072840 2024-05-08T05:47:09.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.799+0000 7f62393e1640 1 -- 172.21.15.45:0/1785472857 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6228032810 con 0x7f6234072840 2024-05-08T05:47:09.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.800+0000 7f623c647640 1 -- 172.21.15.45:0/1785472857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6234072840 msgr2=0x7f62340d0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.800+0000 7f623c647640 1 --2- 172.21.15.45:0/1785472857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6234072840 0x7f62340d0f70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f62340d1cf0 tx=0x7f622802a1a0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.800+0000 7f623c647640 1 -- 172.21.15.45:0/1785472857 shutdown_connections 2024-05-08T05:47:09.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.800+0000 7f623c647640 1 --2- 172.21.15.45:0/1785472857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6234072840 0x7f62340d0f70 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.800+0000 7f623c647640 1 --2- 172.21.15.45:0/1785472857 >> v2:172.21.15.63:3300/0 conn(0x7f623406e0f0 0x7f623415c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.800+0000 7f623c647640 1 -- 172.21.15.45:0/1785472857 >> 172.21.15.45:0/1785472857 conn(0x7f623406c3a0 msgr2=0x7f623406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:09.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.800+0000 7f623c647640 1 -- 172.21.15.45:0/1785472857 shutdown_connections 2024-05-08T05:47:09.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.800+0000 7f623c647640 1 -- 172.21.15.45:0/1785472857 wait complete. 2024-05-08T05:47:09.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.801+0000 7f623c647640 1 Processor -- start 2024-05-08T05:47:09.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.801+0000 7f623c647640 1 -- start start 2024-05-08T05:47:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.802+0000 7f623c647640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f623406e0f0 0x7f6234151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.802+0000 7f623c647640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623406e4d0 0x7f62341f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.802+0000 7f623c647640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6234072840 0x7f6234072c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.802+0000 7f623c647640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f62340ae650 con 0x7f6234072840 2024-05-08T05:47:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.802+0000 7f623c647640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f623406e960 con 0x7f623406e4d0 2024-05-08T05:47:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.802+0000 7f623c647640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f62340ae7d0 con 0x7f623406e0f0 2024-05-08T05:47:09.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.802+0000 7f623abe4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6234072840 0x7f6234072c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.802+0000 7f623abe4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6234072840 0x7f6234072c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43578/0 (socket says 172.21.15.45:43578) 2024-05-08T05:47:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.802+0000 7f623abe4640 1 -- 172.21.15.45:0/4021723295 learned_addr learned my addr 172.21.15.45:0/4021723295 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.802+0000 7f623a3e3640 1 --2- 172.21.15.45:0/4021723295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f623406e0f0 0x7f6234151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.802+0000 7f6239be2640 1 --2- 172.21.15.45:0/4021723295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623406e4d0 0x7f62341f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.802+0000 7f623abe4640 1 -- 172.21.15.45:0/4021723295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f623406e0f0 msgr2=0x7f6234151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.802+0000 7f623abe4640 1 --2- 172.21.15.45:0/4021723295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f623406e0f0 0x7f6234151df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.802+0000 7f623abe4640 1 -- 172.21.15.45:0/4021723295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623406e4d0 msgr2=0x7f62341f2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.802+0000 7f623abe4640 1 --2- 172.21.15.45:0/4021723295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623406e4d0 0x7f62341f2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.802+0000 7f623abe4640 1 -- 172.21.15.45:0/4021723295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6234153150 con 0x7f6234072840 2024-05-08T05:47:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.802+0000 7f623abe4640 1 --2- 172.21.15.45:0/4021723295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6234072840 0x7f6234072c40 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f622c00c790 tx=0x7f622c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.803+0000 7f6232ffd640 1 -- 172.21.15.45:0/4021723295 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c018070 con 0x7f6234072840 2024-05-08T05:47:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.803+0000 7f6232ffd640 1 -- 172.21.15.45:0/4021723295 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f622c00a640 con 0x7f6234072840 2024-05-08T05:47:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.803+0000 7f623c647640 1 -- 172.21.15.45:0/4021723295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62341599d0 con 0x7f6234072840 2024-05-08T05:47:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.803+0000 7f6232ffd640 1 -- 172.21.15.45:0/4021723295 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c00aa50 con 0x7f6234072840 2024-05-08T05:47:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.803+0000 7f623c647640 1 -- 172.21.15.45:0/4021723295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62341536d0 con 0x7f6234072840 2024-05-08T05:47:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.804+0000 7f623c647640 1 -- 172.21.15.45:0/4021723295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f623406eae0 con 0x7f6234072840 2024-05-08T05:47:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.804+0000 7f6232ffd640 1 -- 172.21.15.45:0/4021723295 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f622c01b070 con 0x7f6234072840 2024-05-08T05:47:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.805+0000 7f6232ffd640 1 --2- 172.21.15.45:0/4021723295 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f62200762d0 0x7f6220078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.805+0000 7f6232ffd640 1 -- 172.21.15.45:0/4021723295 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6187+0+0 (secure 0 0 0) 0x7f622c096f20 con 0x7f6234072840 2024-05-08T05:47:09.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.805+0000 7f623a3e3640 1 --2- 172.21.15.45:0/4021723295 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f62200762d0 0x7f6220078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:09.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.806+0000 7f623a3e3640 1 --2- 172.21.15.45:0/4021723295 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f62200762d0 0x7f6220078790 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6224006fd0 tx=0x7f6224008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:09.809+0000 7f6232ffd640 1 -- 172.21.15.45:0/4021723295 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f623406eae0 con 0x7f6234072840 2024-05-08T05:47:10.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.045+0000 7f623c647640 1 -- 172.21.15.45:0/4021723295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f62340693f0 con 0x7f6234072840 2024-05-08T05:47:10.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.531+0000 7f6232ffd640 1 -- 172.21.15.45:0/4021723295 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v62) ==== 162+0+0 (secure 0 0 0) 0x7f622c060570 con 0x7f6234072840 2024-05-08T05:47:10.532 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-08T05:47:10.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.535+0000 7f623c647640 1 -- 172.21.15.45:0/4021723295 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f62200762d0 msgr2=0x7f6220078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:10.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.535+0000 7f623c647640 1 --2- 172.21.15.45:0/4021723295 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f62200762d0 0x7f6220078790 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6224006fd0 tx=0x7f6224008040 comp rx=0 tx=0).stop 2024-05-08T05:47:10.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.535+0000 7f623c647640 1 -- 172.21.15.45:0/4021723295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6234072840 msgr2=0x7f6234072c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:10.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.535+0000 7f623c647640 1 --2- 172.21.15.45:0/4021723295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6234072840 0x7f6234072c40 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f622c00c790 tx=0x7f622c00cc50 comp rx=0 tx=0).stop 2024-05-08T05:47:10.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.535+0000 7f623c647640 1 -- 172.21.15.45:0/4021723295 shutdown_connections 2024-05-08T05:47:10.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.535+0000 7f623c647640 1 --2- 172.21.15.45:0/4021723295 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f62200762d0 0x7f6220078790 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:10.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.535+0000 7f623c647640 1 --2- 172.21.15.45:0/4021723295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6234072840 0x7f6234072c40 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:10.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.535+0000 7f623c647640 1 --2- 172.21.15.45:0/4021723295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623406e4d0 0x7f62341f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:10.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.535+0000 7f623c647640 1 --2- 172.21.15.45:0/4021723295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f623406e0f0 0x7f6234151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:10.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.535+0000 7f623c647640 1 -- 172.21.15.45:0/4021723295 >> 172.21.15.45:0/4021723295 conn(0x7f623406c3a0 msgr2=0x7f623406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:10.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.535+0000 7f623c647640 1 -- 172.21.15.45:0/4021723295 shutdown_connections 2024-05-08T05:47:10.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.535+0000 7f623c647640 1 -- 172.21.15.45:0/4021723295 wait complete. 2024-05-08T05:47:10.549 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:47:10.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.657+0000 7fb51d859640 1 Processor -- start 2024-05-08T05:47:10.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.657+0000 7fb51d859640 1 -- start start 2024-05-08T05:47:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.658+0000 7fb51d859640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb51815f0c0 0x7fb5180b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.658+0000 7fb51d859640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5181638e0 0x7fb5180b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.658+0000 7fb51d859640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb51806c3a0 0x7fb51806c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.658+0000 7fb51d859640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb518163450 con 0x7fb51815f0c0 2024-05-08T05:47:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.658+0000 7fb51d859640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb5180b1260 con 0x7fb51806c3a0 2024-05-08T05:47:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.658+0000 7fb51d859640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb5180b14e0 con 0x7fb5181638e0 2024-05-08T05:47:10.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.658+0000 7fb516ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb51815f0c0 0x7fb5180b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.658+0000 7fb516ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb51815f0c0 0x7fb5180b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54058/0 (socket says 172.21.15.45:54058) 2024-05-08T05:47:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.658+0000 7fb516ffd640 1 -- 172.21.15.45:0/4213426017 learned_addr learned my addr 172.21.15.45:0/4213426017 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.658+0000 7fb5177fe640 1 --2- 172.21.15.45:0/4213426017 >> v2:172.21.15.63:3300/0 conn(0x7fb51806c3a0 0x7fb51806c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.658+0000 7fb50ffff640 1 --2- 172.21.15.45:0/4213426017 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5181638e0 0x7fb5180b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.658+0000 7fb516ffd640 1 -- 172.21.15.45:0/4213426017 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5181638e0 msgr2=0x7fb5180b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.658+0000 7fb516ffd640 1 --2- 172.21.15.45:0/4213426017 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5181638e0 0x7fb5180b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.658+0000 7fb516ffd640 1 -- 172.21.15.45:0/4213426017 >> v2:172.21.15.63:3300/0 conn(0x7fb51806c3a0 msgr2=0x7fb51806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.658+0000 7fb516ffd640 1 --2- 172.21.15.45:0/4213426017 >> v2:172.21.15.63:3300/0 conn(0x7fb51806c3a0 0x7fb51806c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.658+0000 7fb516ffd640 1 -- 172.21.15.45:0/4213426017 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb518162e00 con 0x7fb51815f0c0 2024-05-08T05:47:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.658+0000 7fb50ffff640 1 --2- 172.21.15.45:0/4213426017 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5181638e0 0x7fb5180b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.659+0000 7fb516ffd640 1 --2- 172.21.15.45:0/4213426017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb51815f0c0 0x7fb5180b6f40 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fb5180d1d80 tx=0x7fb50802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f39b22a0cd9a178 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.659+0000 7fb5167fc640 1 -- 172.21.15.45:0/4213426017 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508037070 con 0x7fb51815f0c0 2024-05-08T05:47:10.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.659+0000 7fb5167fc640 1 -- 172.21.15.45:0/4213426017 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb508032440 con 0x7fb51815f0c0 2024-05-08T05:47:10.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.659+0000 7fb5167fc640 1 -- 172.21.15.45:0/4213426017 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508032810 con 0x7fb51815f0c0 2024-05-08T05:47:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.660+0000 7fb51d859640 1 -- 172.21.15.45:0/4213426017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb51815f0c0 msgr2=0x7fb5180b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:10.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.660+0000 7fb51d859640 1 --2- 172.21.15.45:0/4213426017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb51815f0c0 0x7fb5180b6f40 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fb5180d1d80 tx=0x7fb50802a1e0 comp rx=0 tx=0).stop 2024-05-08T05:47:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.660+0000 7fb51d859640 1 -- 172.21.15.45:0/4213426017 shutdown_connections 2024-05-08T05:47:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.660+0000 7fb51d859640 1 --2- 172.21.15.45:0/4213426017 >> v2:172.21.15.63:3300/0 conn(0x7fb51806c3a0 0x7fb51806c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.660+0000 7fb51d859640 1 --2- 172.21.15.45:0/4213426017 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5181638e0 0x7fb5180b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.660+0000 7fb51d859640 1 --2- 172.21.15.45:0/4213426017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb51815f0c0 0x7fb5180b6f40 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.660+0000 7fb51d859640 1 -- 172.21.15.45:0/4213426017 >> 172.21.15.45:0/4213426017 conn(0x7fb5180b3b10 msgr2=0x7fb5180b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.660+0000 7fb51d859640 1 -- 172.21.15.45:0/4213426017 shutdown_connections 2024-05-08T05:47:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.660+0000 7fb51d859640 1 -- 172.21.15.45:0/4213426017 wait complete. 2024-05-08T05:47:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.661+0000 7fb51d859640 1 Processor -- start 2024-05-08T05:47:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.661+0000 7fb51d859640 1 -- start start 2024-05-08T05:47:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb51d859640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb51806c3a0 0x7fb518153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb51d859640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb51815f0c0 0x7fb5180738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb51d859640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5181638e0 0x7fb51814d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:10.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb51d859640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb518073170 con 0x7fb51815f0c0 2024-05-08T05:47:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb51d859640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb5180b17c0 con 0x7fb5181638e0 2024-05-08T05:47:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb51d859640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb5180732f0 con 0x7fb51806c3a0 2024-05-08T05:47:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb50ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb51815f0c0 0x7fb5180738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb50ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb51815f0c0 0x7fb5180738f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54070/0 (socket says 172.21.15.45:54070) 2024-05-08T05:47:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb50ffff640 1 -- 172.21.15.45:0/1832672397 learned_addr learned my addr 172.21.15.45:0/1832672397 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb516ffd640 1 --2- 172.21.15.45:0/1832672397 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb51806c3a0 0x7fb518153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb5177fe640 1 --2- 172.21.15.45:0/1832672397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5181638e0 0x7fb51814d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb50ffff640 1 -- 172.21.15.45:0/1832672397 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb51806c3a0 msgr2=0x7fb518153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb50ffff640 1 --2- 172.21.15.45:0/1832672397 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb51806c3a0 0x7fb518153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb50ffff640 1 -- 172.21.15.45:0/1832672397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5181638e0 msgr2=0x7fb51814d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb50ffff640 1 --2- 172.21.15.45:0/1832672397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5181638e0 0x7fb51814d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb50ffff640 1 -- 172.21.15.45:0/1832672397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb518151980 con 0x7fb51815f0c0 2024-05-08T05:47:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb516ffd640 1 --2- 172.21.15.45:0/1832672397 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb51806c3a0 0x7fb518153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:47:10.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb5177fe640 1 --2- 172.21.15.45:0/1832672397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5181638e0 0x7fb51814d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:47:10.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb50ffff640 1 --2- 172.21.15.45:0/1832672397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb51815f0c0 0x7fb5180738f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb50000d800 tx=0x7fb50000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:10.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb514ff9640 1 -- 172.21.15.45:0/1832672397 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500004030 con 0x7fb51815f0c0 2024-05-08T05:47:10.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb514ff9640 1 -- 172.21.15.45:0/1832672397 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5000041d0 con 0x7fb51815f0c0 2024-05-08T05:47:10.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb51d859640 1 -- 172.21.15.45:0/1832672397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb518151f00 con 0x7fb51815f0c0 2024-05-08T05:47:10.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.662+0000 7fb514ff9640 1 -- 172.21.15.45:0/1832672397 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5000096a0 con 0x7fb51815f0c0 2024-05-08T05:47:10.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.663+0000 7fb51d859640 1 -- 172.21.15.45:0/1832672397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5181524c0 con 0x7fb51815f0c0 2024-05-08T05:47:10.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.664+0000 7fb51d859640 1 -- 172.21.15.45:0/1832672397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb51806a8f0 con 0x7fb51815f0c0 2024-05-08T05:47:10.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.664+0000 7fb514ff9640 1 -- 172.21.15.45:0/1832672397 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb500009840 con 0x7fb51815f0c0 2024-05-08T05:47:10.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.665+0000 7fb514ff9640 1 --2- 172.21.15.45:0/1832672397 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb4f80762d0 0x7fb4f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:10.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.665+0000 7fb514ff9640 1 -- 172.21.15.45:0/1832672397 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6187+0+0 (secure 0 0 0) 0x7fb500097a30 con 0x7fb51815f0c0 2024-05-08T05:47:10.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.665+0000 7fb516ffd640 1 --2- 172.21.15.45:0/1832672397 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb4f80762d0 0x7fb4f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:10.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.665+0000 7fb516ffd640 1 --2- 172.21.15.45:0/1832672397 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb4f80762d0 0x7fb4f8078790 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb508034000 tx=0x7fb50802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:10.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.669+0000 7fb514ff9640 1 -- 172.21.15.45:0/1832672397 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb5000611b0 con 0x7fb51815f0c0 2024-05-08T05:47:10.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.899+0000 7fb51d859640 1 -- 172.21.15.45:0/1832672397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb51815d850 con 0x7fb51815f0c0 2024-05-08T05:47:10.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.899+0000 7fb514ff9640 1 -- 172.21.15.45:0/1832672397 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+5177 (secure 0 0 0) 0x7fb500060ab0 con 0x7fb51815f0c0 2024-05-08T05:47:10.900 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:47:10.900 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2616298287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2616298287},{"type":"v1","addr":"172.21.15.63:6836","nonce":2616298287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6744,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2854804427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2854804427},{"type":"v1","addr":"172.21.15.45:6834","nonce":2854804427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2945763952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2945763952},{"type":"v1","addr":"172.21.15.45:6835","nonce":2945763952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:47:00.211837+0000","modified":"2024-05-08T05:47:10.535776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6759},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6759":{"gid":6759,"name":"b","rank":0,"incarnation":63,"state":"up:replay","state_seq":1,"addr":"172.21.15.63:6837/3194497846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":3194497846},{"type":"v1","addr":"172.21.15.63:6837","nonce":3194497846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-08T05:47:10.900 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 63 2024-05-08T05:47:10.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.901+0000 7fb51d859640 1 -- 172.21.15.45:0/1832672397 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb4f80762d0 msgr2=0x7fb4f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:10.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.901+0000 7fb51d859640 1 --2- 172.21.15.45:0/1832672397 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb4f80762d0 0x7fb4f8078790 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb508034000 tx=0x7fb50802e000 comp rx=0 tx=0).stop 2024-05-08T05:47:10.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.901+0000 7fb51d859640 1 -- 172.21.15.45:0/1832672397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb51815f0c0 msgr2=0x7fb5180738f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:10.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.901+0000 7fb51d859640 1 --2- 172.21.15.45:0/1832672397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb51815f0c0 0x7fb5180738f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb50000d800 tx=0x7fb50000dcc0 comp rx=0 tx=0).stop 2024-05-08T05:47:10.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.901+0000 7fb51d859640 1 -- 172.21.15.45:0/1832672397 shutdown_connections 2024-05-08T05:47:10.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.901+0000 7fb51d859640 1 --2- 172.21.15.45:0/1832672397 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb4f80762d0 0x7fb4f8078790 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:10.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.901+0000 7fb51d859640 1 --2- 172.21.15.45:0/1832672397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5181638e0 0x7fb51814d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:10.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.901+0000 7fb51d859640 1 --2- 172.21.15.45:0/1832672397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb51815f0c0 0x7fb5180738f0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:10.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.901+0000 7fb51d859640 1 --2- 172.21.15.45:0/1832672397 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb51806c3a0 0x7fb518153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:10.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.901+0000 7fb51d859640 1 -- 172.21.15.45:0/1832672397 >> 172.21.15.45:0/1832672397 conn(0x7fb5180b3b10 msgr2=0x7fb51815dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:10.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.901+0000 7fb51d859640 1 -- 172.21.15.45:0/1832672397 shutdown_connections 2024-05-08T05:47:10.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:10.901+0000 7fb51d859640 1 -- 172.21.15.45:0/1832672397 wait complete. 2024-05-08T05:47:11.315 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:47:00.211837+0000', 'modified': '2024-05-08T05:47:10.535776+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 89, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6759}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6759': {'gid': 6759, 'name': 'b', 'rank': 0, 'incarnation': 63, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.63:6837/3194497846', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6835', 'nonce': 3194497846}, {'type': 'v1', 'addr': '172.21.15.63:6837', 'nonce': 3194497846}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-08T05:47:11.315 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6759:up:replay 2024-05-08T05:47:12.316 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:47:12.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.433+0000 7f547da54640 1 Processor -- start 2024-05-08T05:47:12.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.433+0000 7f547da54640 1 -- start start 2024-05-08T05:47:12.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.434+0000 7f547da54640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f547815f2d0 0x7f54780b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:12.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.434+0000 7f547da54640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f54780d4fc0 0x7f54780d5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.434+0000 7f547da54640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54781606c0 con 0x7f54780d1c00 2024-05-08T05:47:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.434+0000 7f547da54640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f54780d5960 con 0x7f54780d4fc0 2024-05-08T05:47:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.434+0000 7f547da54640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f54780b5810 con 0x7f547815f2d0 2024-05-08T05:47:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.434+0000 7f54767fc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f54780d4fc0 0x7f54780d5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.434+0000 7f5476ffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f547815f2d0 0x7f54780b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.434+0000 7f54767fc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f54780d4fc0 0x7f54780d5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:49850/0 (socket says 172.21.15.45:49850) 2024-05-08T05:47:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.434+0000 7f54767fc640 1 -- 172.21.15.45:0/766880728 learned_addr learned my addr 172.21.15.45:0/766880728 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.434+0000 7f5476ffd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f547815f2d0 0x7f54780b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:53012/0 (socket says 172.21.15.45:53012) 2024-05-08T05:47:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.434+0000 7f54777fe640 1 --1- 172.21.15.45:0/766880728 >> v1:172.21.15.63:6789/0 conn(0x7f54780d1c00 0x7f54780d2090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:60458/0 (socket says 172.21.15.45:60458) 2024-05-08T05:47:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.434+0000 7f54767fc640 1 -- 172.21.15.45:0/766880728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f547815f2d0 msgr2=0x7f54780b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.434+0000 7f54767fc640 1 --2- 172.21.15.45:0/766880728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f547815f2d0 0x7f54780b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.434+0000 7f54767fc640 1 -- 172.21.15.45:0/766880728 >> v1:172.21.15.63:6789/0 conn(0x7f54780d1c00 legacy=0x7f54780d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:12.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.434+0000 7f54767fc640 1 -- 172.21.15.45:0/766880728 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54780b4f00 con 0x7f54780d4fc0 2024-05-08T05:47:12.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.435+0000 7f54767fc640 1 --2- 172.21.15.45:0/766880728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54780d4fc0 0x7f54780d5420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5460009870 tx=0x7f546002a240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e51c73e3498a65c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:12.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.436+0000 7f5475ffb640 1 -- 172.21.15.45:0/766880728 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460037070 con 0x7f54780d4fc0 2024-05-08T05:47:12.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.436+0000 7f5475ffb640 1 -- 172.21.15.45:0/766880728 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5460032440 con 0x7f54780d4fc0 2024-05-08T05:47:12.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.436+0000 7f5475ffb640 1 -- 172.21.15.45:0/766880728 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460032810 con 0x7f54780d4fc0 2024-05-08T05:47:12.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.436+0000 7f547da54640 1 -- 172.21.15.45:0/766880728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54780d4fc0 msgr2=0x7f54780d5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:12.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.436+0000 7f547da54640 1 --2- 172.21.15.45:0/766880728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54780d4fc0 0x7f54780d5420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5460009870 tx=0x7f546002a240 comp rx=0 tx=0).stop 2024-05-08T05:47:12.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.436+0000 7f547da54640 1 -- 172.21.15.45:0/766880728 shutdown_connections 2024-05-08T05:47:12.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.436+0000 7f547da54640 1 --2- 172.21.15.45:0/766880728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54780d4fc0 0x7f54780d5420 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:12.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.436+0000 7f547da54640 1 --2- 172.21.15.45:0/766880728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f547815f2d0 0x7f54780b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:12.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.436+0000 7f547da54640 1 -- 172.21.15.45:0/766880728 >> 172.21.15.45:0/766880728 conn(0x7f54780746d0 msgr2=0x7f547809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:12.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.436+0000 7f547da54640 1 -- 172.21.15.45:0/766880728 shutdown_connections 2024-05-08T05:47:12.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.436+0000 7f547da54640 1 -- 172.21.15.45:0/766880728 wait complete. 2024-05-08T05:47:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.438+0000 7f547da54640 1 Processor -- start 2024-05-08T05:47:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.438+0000 7f547da54640 1 -- start start 2024-05-08T05:47:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.438+0000 7f547da54640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f54780d1c00 0x7f5478152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.439+0000 7f547da54640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54780d4fc0 0x7f54780ce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.439+0000 7f547da54640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f547815f2d0 0x7f54780ce950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.439+0000 7f547da54640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f54780d2c00 con 0x7f54780d4fc0 2024-05-08T05:47:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.439+0000 7f547da54640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f54780b54a0 con 0x7f547815f2d0 2024-05-08T05:47:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.439+0000 7f547da54640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f54780d2d80 con 0x7f54780d1c00 2024-05-08T05:47:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.439+0000 7f54767fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54780d4fc0 0x7f54780ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.439+0000 7f54767fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54780d4fc0 0x7f54780ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54074/0 (socket says 172.21.15.45:54074) 2024-05-08T05:47:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.439+0000 7f54767fc640 1 -- 172.21.15.45:0/2740971646 learned_addr learned my addr 172.21.15.45:0/2740971646 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.439+0000 7f5476ffd640 1 --2- 172.21.15.45:0/2740971646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f54780d1c00 0x7f5478152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:12.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.439+0000 7f54777fe640 1 --2- 172.21.15.45:0/2740971646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f547815f2d0 0x7f54780ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.439+0000 7f54767fc640 1 -- 172.21.15.45:0/2740971646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f54780d1c00 msgr2=0x7f5478152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.439+0000 7f54767fc640 1 --2- 172.21.15.45:0/2740971646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f54780d1c00 0x7f5478152410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.439+0000 7f54767fc640 1 -- 172.21.15.45:0/2740971646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f547815f2d0 msgr2=0x7f54780ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.439+0000 7f54767fc640 1 --2- 172.21.15.45:0/2740971646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f547815f2d0 0x7f54780ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.439+0000 7f54767fc640 1 -- 172.21.15.45:0/2740971646 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54780b2850 con 0x7f54780d4fc0 2024-05-08T05:47:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.439+0000 7f54777fe640 1 --2- 172.21.15.45:0/2740971646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f547815f2d0 0x7f54780ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.439+0000 7f5476ffd640 1 --2- 172.21.15.45:0/2740971646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f54780d1c00 0x7f5478152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.439+0000 7f54767fc640 1 --2- 172.21.15.45:0/2740971646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54780d4fc0 0x7f54780ce410 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5460000c00 tx=0x7f54600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.439+0000 7f546ffff640 1 -- 172.21.15.45:0/2740971646 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460037040 con 0x7f54780d4fc0 2024-05-08T05:47:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.440+0000 7f546ffff640 1 -- 172.21.15.45:0/2740971646 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5460042070 con 0x7f54780d4fc0 2024-05-08T05:47:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.440+0000 7f547da54640 1 -- 172.21.15.45:0/2740971646 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54780b2b30 con 0x7f54780d4fc0 2024-05-08T05:47:12.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.440+0000 7f546ffff640 1 -- 172.21.15.45:0/2740971646 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460030880 con 0x7f54780d4fc0 2024-05-08T05:47:12.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.440+0000 7f547da54640 1 -- 172.21.15.45:0/2740971646 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54780b30f0 con 0x7f54780d4fc0 2024-05-08T05:47:12.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.441+0000 7f547da54640 1 -- 172.21.15.45:0/2740971646 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5478069b00 con 0x7f54780d4fc0 2024-05-08T05:47:12.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.441+0000 7f546ffff640 1 -- 172.21.15.45:0/2740971646 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5460030a20 con 0x7f54780d4fc0 2024-05-08T05:47:12.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.442+0000 7f546ffff640 1 --2- 172.21.15.45:0/2740971646 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f54580762d0 0x7f5458078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:12.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.442+0000 7f546ffff640 1 -- 172.21.15.45:0/2740971646 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6187+0+0 (secure 0 0 0) 0x7f54600b8040 con 0x7f54780d4fc0 2024-05-08T05:47:12.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.443+0000 7f5476ffd640 1 --2- 172.21.15.45:0/2740971646 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f54580762d0 0x7f5458078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:12.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.443+0000 7f5476ffd640 1 --2- 172.21.15.45:0/2740971646 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f54580762d0 0x7f5458078790 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5468009fd0 tx=0x7f546800b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:12.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.447+0000 7f546ffff640 1 -- 172.21.15.45:0/2740971646 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5460081850 con 0x7f54780d4fc0 2024-05-08T05:47:12.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.680+0000 7f547da54640 1 -- 172.21.15.45:0/2740971646 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54780b4b60 con 0x7f54780d4fc0 2024-05-08T05:47:12.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.680+0000 7f546ffff640 1 -- 172.21.15.45:0/2740971646 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5177 (secure 0 0 0) 0x7f5460081150 con 0x7f54780d4fc0 2024-05-08T05:47:12.681 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:47:12.681 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2616298287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2616298287},{"type":"v1","addr":"172.21.15.63:6836","nonce":2616298287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6744,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2854804427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2854804427},{"type":"v1","addr":"172.21.15.45:6834","nonce":2854804427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2945763952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2945763952},{"type":"v1","addr":"172.21.15.45:6835","nonce":2945763952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:47:00.211837+0000","modified":"2024-05-08T05:47:11.540779+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6759},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6759":{"gid":6759,"name":"b","rank":0,"incarnation":63,"state":"up:rejoin","state_seq":6,"addr":"172.21.15.63:6837/3194497846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":3194497846},{"type":"v1","addr":"172.21.15.63:6837","nonce":3194497846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-08T05:47:12.681 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 65 2024-05-08T05:47:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.682+0000 7f547da54640 1 -- 172.21.15.45:0/2740971646 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f54580762d0 msgr2=0x7f5458078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.682+0000 7f547da54640 1 --2- 172.21.15.45:0/2740971646 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f54580762d0 0x7f5458078790 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5468009fd0 tx=0x7f546800b040 comp rx=0 tx=0).stop 2024-05-08T05:47:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.682+0000 7f547da54640 1 -- 172.21.15.45:0/2740971646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54780d4fc0 msgr2=0x7f54780ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.682+0000 7f547da54640 1 --2- 172.21.15.45:0/2740971646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54780d4fc0 0x7f54780ce410 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5460000c00 tx=0x7f54600046c0 comp rx=0 tx=0).stop 2024-05-08T05:47:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.683+0000 7f547da54640 1 -- 172.21.15.45:0/2740971646 shutdown_connections 2024-05-08T05:47:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.683+0000 7f547da54640 1 --2- 172.21.15.45:0/2740971646 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f54580762d0 0x7f5458078790 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.683+0000 7f547da54640 1 --2- 172.21.15.45:0/2740971646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f547815f2d0 0x7f54780ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.683+0000 7f547da54640 1 --2- 172.21.15.45:0/2740971646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f54780d4fc0 0x7f54780ce410 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.683+0000 7f547da54640 1 --2- 172.21.15.45:0/2740971646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f54780d1c00 0x7f5478152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.683+0000 7f547da54640 1 -- 172.21.15.45:0/2740971646 >> 172.21.15.45:0/2740971646 conn(0x7f54780746d0 msgr2=0x7f54780cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:12.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.683+0000 7f547da54640 1 -- 172.21.15.45:0/2740971646 shutdown_connections 2024-05-08T05:47:12.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:12.683+0000 7f547da54640 1 -- 172.21.15.45:0/2740971646 wait complete. 2024-05-08T05:47:13.096 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:47:00.211837+0000', 'modified': '2024-05-08T05:47:11.540779+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 89, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6759}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6759': {'gid': 6759, 'name': 'b', 'rank': 0, 'incarnation': 63, 'state': 'up:rejoin', 'state_seq': 6, 'addr': '172.21.15.63:6837/3194497846', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6835', 'nonce': 3194497846}, {'type': 'v1', 'addr': '172.21.15.63:6837', 'nonce': 3194497846}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-08T05:47:13.097 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6759:up:rejoin 2024-05-08T05:47:14.098 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:47:14.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.213+0000 7fc7bcd38640 1 Processor -- start 2024-05-08T05:47:14.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.213+0000 7fc7bcd38640 1 -- start start 2024-05-08T05:47:14.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.214+0000 7fc7bcd38640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc7b815f420 0x7fc7b80d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:14.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.214+0000 7fc7bcd38640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc7b80d2080 0x7fc7b80d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:14.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.214+0000 7fc7bcd38640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7b81630b0 con 0x7fc7b80d3ca0 2024-05-08T05:47:14.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.214+0000 7fc7bcd38640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc7b80d2550 con 0x7fc7b815f420 2024-05-08T05:47:14.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.214+0000 7fc7bcd38640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc7b80d2c80 con 0x7fc7b80d2080 2024-05-08T05:47:14.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.214+0000 7fc7b659c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc7b815f420 0x7fc7b80d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:14.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.214+0000 7fc7b659c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc7b815f420 0x7fc7b80d3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54084/0 (socket says 172.21.15.45:54084) 2024-05-08T05:47:14.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.214+0000 7fc7add9b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc7b80d3ca0 0x7fc7b80d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41262/0 (socket says 172.21.15.45:41262) 2024-05-08T05:47:14.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.214+0000 7fc7b659c640 1 -- 172.21.15.45:0/833188064 learned_addr learned my addr 172.21.15.45:0/833188064 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:14.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.214+0000 7fc7b6d9d640 1 --2- 172.21.15.45:0/833188064 >> v2:172.21.15.63:3300/0 conn(0x7fc7b80d2080 0x7fc7b80d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:14.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.214+0000 7fc7b659c640 1 -- 172.21.15.45:0/833188064 >> v2:172.21.15.63:3300/0 conn(0x7fc7b80d2080 msgr2=0x7fc7b80d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:14.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.214+0000 7fc7b659c640 1 --2- 172.21.15.45:0/833188064 >> v2:172.21.15.63:3300/0 conn(0x7fc7b80d2080 0x7fc7b80d4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:14.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.214+0000 7fc7b659c640 1 -- 172.21.15.45:0/833188064 >> v1:172.21.15.45:6789/0 conn(0x7fc7b80d3ca0 legacy=0x7fc7b80d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:14.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.214+0000 7fc7b659c640 1 -- 172.21.15.45:0/833188064 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b80d2f60 con 0x7fc7b815f420 2024-05-08T05:47:14.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.214+0000 7fc7b5d9b640 1 -- 172.21.15.45:0/833188064 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1894826993 0 0) 0x7fc7a0001110 con 0x7fc7b80d3ca0 2024-05-08T05:47:14.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.214+0000 7fc7b6d9d640 1 --2- 172.21.15.45:0/833188064 >> v2:172.21.15.63:3300/0 conn(0x7fc7b80d2080 0x7fc7b80d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:14.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.215+0000 7fc7b659c640 1 --2- 172.21.15.45:0/833188064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7b815f420 0x7fc7b80d3760 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc7b80d5e90 tx=0x7fc7a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7774668f1bdfdf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:14.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.215+0000 7fc7b5d9b640 1 -- 172.21.15.45:0/833188064 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8037070 con 0x7fc7b815f420 2024-05-08T05:47:14.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.215+0000 7fc7b5d9b640 1 -- 172.21.15.45:0/833188064 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7a8032440 con 0x7fc7b815f420 2024-05-08T05:47:14.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.215+0000 7fc7b5d9b640 1 -- 172.21.15.45:0/833188064 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8032810 con 0x7fc7b815f420 2024-05-08T05:47:14.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.216+0000 7fc7bcd38640 1 -- 172.21.15.45:0/833188064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7b815f420 msgr2=0x7fc7b80d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:14.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.216+0000 7fc7bcd38640 1 --2- 172.21.15.45:0/833188064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7b815f420 0x7fc7b80d3760 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc7b80d5e90 tx=0x7fc7a802a270 comp rx=0 tx=0).stop 2024-05-08T05:47:14.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.216+0000 7fc7bcd38640 1 -- 172.21.15.45:0/833188064 shutdown_connections 2024-05-08T05:47:14.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.216+0000 7fc7bcd38640 1 --2- 172.21.15.45:0/833188064 >> v2:172.21.15.63:3300/0 conn(0x7fc7b80d2080 0x7fc7b80d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:14.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.216+0000 7fc7bcd38640 1 --2- 172.21.15.45:0/833188064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7b815f420 0x7fc7b80d3760 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:14.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.216+0000 7fc7bcd38640 1 -- 172.21.15.45:0/833188064 >> 172.21.15.45:0/833188064 conn(0x7fc7b808eac0 msgr2=0x7fc7b809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:14.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.216+0000 7fc7bcd38640 1 -- 172.21.15.45:0/833188064 shutdown_connections 2024-05-08T05:47:14.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.216+0000 7fc7bcd38640 1 -- 172.21.15.45:0/833188064 wait complete. 2024-05-08T05:47:14.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.217+0000 7fc7bcd38640 1 Processor -- start 2024-05-08T05:47:14.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.218+0000 7fc7bcd38640 1 -- start start 2024-05-08T05:47:14.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.218+0000 7fc7bcd38640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7b80d2080 0x7fc7b8154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:14.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.218+0000 7fc7bcd38640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7b80c8b20 0x7fc7b80c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:14.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.218+0000 7fc7bcd38640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7b80ca6e0 0x7fc7b80cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:14.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.218+0000 7fc7bcd38640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b80d5200 con 0x7fc7b80ca6e0 2024-05-08T05:47:14.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.218+0000 7fc7bcd38640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b80d2830 con 0x7fc7b80d2080 2024-05-08T05:47:14.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.218+0000 7fc7bcd38640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b80d5380 con 0x7fc7b80c8b20 2024-05-08T05:47:14.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.218+0000 7fc7b6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7b80ca6e0 0x7fc7b80cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:14.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.218+0000 7fc7b6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7b80ca6e0 0x7fc7b80cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54098/0 (socket says 172.21.15.45:54098) 2024-05-08T05:47:14.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.218+0000 7fc7b6d9d640 1 -- 172.21.15.45:0/4277192035 learned_addr learned my addr 172.21.15.45:0/4277192035 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:14.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.219+0000 7fc7b659c640 1 --2- 172.21.15.45:0/4277192035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7b80d2080 0x7fc7b8154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:14.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.219+0000 7fc7add9b640 1 --2- 172.21.15.45:0/4277192035 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7b80c8b20 0x7fc7b80c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:14.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.219+0000 7fc7b6d9d640 1 -- 172.21.15.45:0/4277192035 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7b80c8b20 msgr2=0x7fc7b80c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:14.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.219+0000 7fc7b6d9d640 1 --2- 172.21.15.45:0/4277192035 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7b80c8b20 0x7fc7b80c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:14.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.219+0000 7fc7b6d9d640 1 -- 172.21.15.45:0/4277192035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7b80d2080 msgr2=0x7fc7b8154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:14.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.219+0000 7fc7b6d9d640 1 --2- 172.21.15.45:0/4277192035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7b80d2080 0x7fc7b8154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:14.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.219+0000 7fc7b6d9d640 1 -- 172.21.15.45:0/4277192035 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b8155560 con 0x7fc7b80ca6e0 2024-05-08T05:47:14.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.219+0000 7fc7add9b640 1 --2- 172.21.15.45:0/4277192035 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7b80c8b20 0x7fc7b80c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:14.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.219+0000 7fc7b6d9d640 1 --2- 172.21.15.45:0/4277192035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7b80ca6e0 0x7fc7b80cab60 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0007ef0 tx=0x7fc7b000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:14.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.219+0000 7fc7af7fe640 1 -- 172.21.15.45:0/4277192035 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b001a070 con 0x7fc7b80ca6e0 2024-05-08T05:47:14.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.219+0000 7fc7af7fe640 1 -- 172.21.15.45:0/4277192035 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b0003b90 con 0x7fc7b80ca6e0 2024-05-08T05:47:14.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.219+0000 7fc7bcd38640 1 -- 172.21.15.45:0/4277192035 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b80c9690 con 0x7fc7b80ca6e0 2024-05-08T05:47:14.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.219+0000 7fc7af7fe640 1 -- 172.21.15.45:0/4277192035 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b000de80 con 0x7fc7b80ca6e0 2024-05-08T05:47:14.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.219+0000 7fc7bcd38640 1 -- 172.21.15.45:0/4277192035 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b80cc280 con 0x7fc7b80ca6e0 2024-05-08T05:47:14.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.220+0000 7fc7bcd38640 1 -- 172.21.15.45:0/4277192035 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b80693f0 con 0x7fc7b80ca6e0 2024-05-08T05:47:14.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.221+0000 7fc7af7fe640 1 -- 172.21.15.45:0/4277192035 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc7b001d070 con 0x7fc7b80ca6e0 2024-05-08T05:47:14.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.222+0000 7fc7af7fe640 1 --2- 172.21.15.45:0/4277192035 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc7980762d0 0x7fc798078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:14.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.222+0000 7fc7af7fe640 1 -- 172.21.15.45:0/4277192035 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6187+0+0 (secure 0 0 0) 0x7fc7b0097900 con 0x7fc7b80ca6e0 2024-05-08T05:47:14.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.222+0000 7fc7b659c640 1 --2- 172.21.15.45:0/4277192035 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc7980762d0 0x7fc798078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:14.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.222+0000 7fc7b659c640 1 --2- 172.21.15.45:0/4277192035 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc7980762d0 0x7fc798078790 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8034000 tx=0x7fc7a802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:14.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.226+0000 7fc7af7fe640 1 -- 172.21.15.45:0/4277192035 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc7b0010070 con 0x7fc7b80ca6e0 2024-05-08T05:47:14.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.455+0000 7fc7bcd38640 1 -- 172.21.15.45:0/4277192035 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7b80b4c50 con 0x7fc7b80ca6e0 2024-05-08T05:47:14.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.456+0000 7fc7af7fe640 1 -- 172.21.15.45:0/4277192035 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5184 (secure 0 0 0) 0x7fc7b0060a30 con 0x7fc7b80ca6e0 2024-05-08T05:47:14.456 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:47:14.456 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2616298287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2616298287},{"type":"v1","addr":"172.21.15.63:6836","nonce":2616298287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6744,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2854804427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2854804427},{"type":"v1","addr":"172.21.15.45:6834","nonce":2854804427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2945763952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2945763952},{"type":"v1","addr":"172.21.15.45:6835","nonce":2945763952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:47:00.211837+0000","modified":"2024-05-08T05:47:13.540115+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6759},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6759":{"gid":6759,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":7,"addr":"172.21.15.63:6837/3194497846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":3194497846},{"type":"v1","addr":"172.21.15.63:6837","nonce":3194497846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6759,"qdb_cluster":[6759]},"id":6}]} 2024-05-08T05:47:14.456 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 66 2024-05-08T05:47:14.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.458+0000 7fc7bcd38640 1 -- 172.21.15.45:0/4277192035 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc7980762d0 msgr2=0x7fc798078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:14.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.458+0000 7fc7bcd38640 1 --2- 172.21.15.45:0/4277192035 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc7980762d0 0x7fc798078790 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8034000 tx=0x7fc7a802e000 comp rx=0 tx=0).stop 2024-05-08T05:47:14.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.458+0000 7fc7bcd38640 1 -- 172.21.15.45:0/4277192035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7b80ca6e0 msgr2=0x7fc7b80cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:14.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.458+0000 7fc7bcd38640 1 --2- 172.21.15.45:0/4277192035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7b80ca6e0 0x7fc7b80cab60 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0007ef0 tx=0x7fc7b000d4a0 comp rx=0 tx=0).stop 2024-05-08T05:47:14.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.458+0000 7fc7bcd38640 1 -- 172.21.15.45:0/4277192035 shutdown_connections 2024-05-08T05:47:14.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.458+0000 7fc7bcd38640 1 --2- 172.21.15.45:0/4277192035 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc7980762d0 0x7fc798078790 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:14.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.458+0000 7fc7bcd38640 1 --2- 172.21.15.45:0/4277192035 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc7b80c8b20 0x7fc7b80c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:14.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.458+0000 7fc7bcd38640 1 --2- 172.21.15.45:0/4277192035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc7b80ca6e0 0x7fc7b80cab60 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:14.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.458+0000 7fc7bcd38640 1 --2- 172.21.15.45:0/4277192035 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7b80d2080 0x7fc7b8154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:14.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.458+0000 7fc7bcd38640 1 -- 172.21.15.45:0/4277192035 >> 172.21.15.45:0/4277192035 conn(0x7fc7b808eac0 msgr2=0x7fc7b8163290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:14.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.458+0000 7fc7bcd38640 1 -- 172.21.15.45:0/4277192035 shutdown_connections 2024-05-08T05:47:14.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.458+0000 7fc7bcd38640 1 -- 172.21.15.45:0/4277192035 wait complete. 2024-05-08T05:47:14.872 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:47:00.211837+0000', 'modified': '2024-05-08T05:47:13.540115+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 89, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6759}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6759': {'gid': 6759, 'name': 'b', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.63:6837/3194497846', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.63:6835', 'nonce': 3194497846}, {'type': 'v1', 'addr': '172.21.15.63:6837', 'nonce': 3194497846}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6759, 'qdb_cluster': [6759]} 2024-05-08T05:47:14.872 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T05:47:14.872 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T05:47:14.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.981+0000 7f36a31af640 1 Processor -- start 2024-05-08T05:47:14.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.981+0000 7f36a31af640 1 -- start start 2024-05-08T05:47:14.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.982+0000 7f36a31af640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f369c0d0ec0 0x7f369c0d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:14.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.982+0000 7f36a31af640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f369c0d1760 con 0x7f369c0d3b00 2024-05-08T05:47:14.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.982+0000 7f36a31af640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f369c0d1a40 con 0x7f369c15f380 2024-05-08T05:47:14.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.982+0000 7f36a31af640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f369c0d30f0 con 0x7f369c0d0ec0 2024-05-08T05:47:14.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.982+0000 7f36a174c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f369c0d3b00 0x7f369c0b7e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41266/0 (socket says 172.21.15.45:41266) 2024-05-08T05:47:14.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.982+0000 7f36a174c640 1 -- 172.21.15.45:0/1502211079 learned_addr learned my addr 172.21.15.45:0/1502211079 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:47:14.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.982+0000 7f369bfff640 1 --2- 172.21.15.45:0/1502211079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f369c0d0ec0 0x7f369c0d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:14.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.982+0000 7f369b7fe640 1 -- 172.21.15.45:0/1502211079 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1483023415 0 0) 0x7f369c0d1760 con 0x7f369c0d3b00 2024-05-08T05:47:14.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.983+0000 7f369b7fe640 1 -- 172.21.15.45:0/1502211079 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3684003520 con 0x7f369c0d3b00 2024-05-08T05:47:14.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.983+0000 7f369bfff640 1 -- 172.21.15.45:0/1502211079 >> v1:172.21.15.63:6789/0 conn(0x7f369c15f380 legacy=0x7f369c0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:14.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.983+0000 7f369bfff640 1 -- 172.21.15.45:0/1502211079 >> v1:172.21.15.45:6789/0 conn(0x7f369c0d3b00 legacy=0x7f369c0b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:14.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.983+0000 7f369b7fe640 1 -- 172.21.15.45:0/1502211079 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2418785276 0 0) 0x7f369c0d1a40 con 0x7f369c15f380 2024-05-08T05:47:14.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.983+0000 7f369bfff640 1 -- 172.21.15.45:0/1502211079 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369c0d1d20 con 0x7f369c0d0ec0 2024-05-08T05:47:14.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.983+0000 7f369bfff640 1 --2- 172.21.15.45:0/1502211079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f369c0d0ec0 0x7f369c0d35c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f36900097d0 tx=0x7f369002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5944030a9b38e458 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:14.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.984+0000 7f369b7fe640 1 -- 172.21.15.45:0/1502211079 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690037070 con 0x7f369c0d0ec0 2024-05-08T05:47:14.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.984+0000 7f369b7fe640 1 -- 172.21.15.45:0/1502211079 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3690032440 con 0x7f369c0d0ec0 2024-05-08T05:47:14.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.984+0000 7f369b7fe640 1 -- 172.21.15.45:0/1502211079 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690032810 con 0x7f369c0d0ec0 2024-05-08T05:47:14.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.984+0000 7f36a31af640 1 -- 172.21.15.45:0/1502211079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f369c0d0ec0 msgr2=0x7f369c0d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:14.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.984+0000 7f36a31af640 1 --2- 172.21.15.45:0/1502211079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f369c0d0ec0 0x7f369c0d35c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f36900097d0 tx=0x7f369002a210 comp rx=0 tx=0).stop 2024-05-08T05:47:14.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.984+0000 7f36a31af640 1 -- 172.21.15.45:0/1502211079 shutdown_connections 2024-05-08T05:47:14.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.984+0000 7f36a31af640 1 --2- 172.21.15.45:0/1502211079 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f369c0d0ec0 0x7f369c0d35c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:14.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.984+0000 7f36a31af640 1 -- 172.21.15.45:0/1502211079 >> 172.21.15.45:0/1502211079 conn(0x7f369c0d4fe0 msgr2=0x7f369c0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:14.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.984+0000 7f36a31af640 1 -- 172.21.15.45:0/1502211079 shutdown_connections 2024-05-08T05:47:14.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.984+0000 7f36a31af640 1 -- 172.21.15.45:0/1502211079 wait complete. 2024-05-08T05:47:14.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.985+0000 7f36a31af640 1 Processor -- start 2024-05-08T05:47:14.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.986+0000 7f36a31af640 1 -- start start 2024-05-08T05:47:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.986+0000 7f36a31af640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f369c0d0ec0 0x7f369c158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.986+0000 7f36a31af640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f369c0d3b00 0x7f369c0cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.986+0000 7f36a31af640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f369c15f380 0x7f369c0cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.986+0000 7f36a31af640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f369c15f760 con 0x7f369c15f380 2024-05-08T05:47:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.986+0000 7f36a31af640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f369c0d2370 con 0x7f369c0d0ec0 2024-05-08T05:47:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.986+0000 7f36a31af640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f369c0d2800 con 0x7f369c0d3b00 2024-05-08T05:47:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.987+0000 7f36a174c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f369c15f380 0x7f369c0cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.987+0000 7f36a174c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f369c15f380 0x7f369c0cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54110/0 (socket says 172.21.15.45:54110) 2024-05-08T05:47:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.987+0000 7f36a174c640 1 -- 172.21.15.45:0/2235712952 learned_addr learned my addr 172.21.15.45:0/2235712952 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:14.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.987+0000 7f369bfff640 1 --2- 172.21.15.45:0/2235712952 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f369c0d3b00 0x7f369c0cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.987+0000 7f36a0f4b640 1 --2- 172.21.15.45:0/2235712952 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f369c0d0ec0 0x7f369c158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.987+0000 7f36a174c640 1 -- 172.21.15.45:0/2235712952 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f369c0d3b00 msgr2=0x7f369c0cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.987+0000 7f36a174c640 1 --2- 172.21.15.45:0/2235712952 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f369c0d3b00 0x7f369c0cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.987+0000 7f36a174c640 1 -- 172.21.15.45:0/2235712952 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f369c0d0ec0 msgr2=0x7f369c158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.987+0000 7f36a174c640 1 --2- 172.21.15.45:0/2235712952 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f369c0d0ec0 0x7f369c158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.987+0000 7f36a174c640 1 -- 172.21.15.45:0/2235712952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369c1f8590 con 0x7f369c15f380 2024-05-08T05:47:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.987+0000 7f36a174c640 1 --2- 172.21.15.45:0/2235712952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f369c15f380 0x7f369c0cb220 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f369400cb40 tx=0x7f369400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.987+0000 7f36997fa640 1 -- 172.21.15.45:0/2235712952 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369400aca0 con 0x7f369c15f380 2024-05-08T05:47:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.987+0000 7f36997fa640 1 -- 172.21.15.45:0/2235712952 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f369400ae40 con 0x7f369c15f380 2024-05-08T05:47:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.987+0000 7f36997fa640 1 -- 172.21.15.45:0/2235712952 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694012600 con 0x7f369c15f380 2024-05-08T05:47:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.987+0000 7f36a31af640 1 -- 172.21.15.45:0/2235712952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f369c1f8870 con 0x7f369c15f380 2024-05-08T05:47:14.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.987+0000 7f36a31af640 1 -- 172.21.15.45:0/2235712952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f369c1f8e30 con 0x7f369c15f380 2024-05-08T05:47:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.989+0000 7f36997fa640 1 -- 172.21.15.45:0/2235712952 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f36940127a0 con 0x7f369c15f380 2024-05-08T05:47:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.989+0000 7f36997fa640 1 --2- 172.21.15.45:0/2235712952 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f36880762d0 0x7f3688078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.990+0000 7f36997fa640 1 -- 172.21.15.45:0/2235712952 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7f3694097fb0 con 0x7f369c15f380 2024-05-08T05:47:14.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.990+0000 7f36a0f4b640 1 --2- 172.21.15.45:0/2235712952 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f36880762d0 0x7f3688078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:14.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.990+0000 7f36a0f4b640 1 --2- 172.21.15.45:0/2235712952 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f36880762d0 0x7f3688078790 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f368c000f60 tx=0x7f368c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:14.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.990+0000 7f36a5f9d740 1 Processor -- start 2024-05-08T05:47:14.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.991+0000 7f36a5f9d740 1 -- start start 2024-05-08T05:47:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.991+0000 7f36a5f9d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d3f50 0x5583247d6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.991+0000 7f36a5f9d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247d4330 0x5583247d6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.991+0000 7f36a5f9d740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247a6bb0 0x5583247aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:14.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.991+0000 7f36a5f9d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5583247108d0 con 0x5583247d3f50 2024-05-08T05:47:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.991+0000 7f36a5f9d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5583246c9c10 con 0x5583247d4330 2024-05-08T05:47:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.991+0000 7f36a5f9d740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55832470c890 con 0x5583247a6bb0 2024-05-08T05:47:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.991+0000 7f36a0f4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d3f50 0x5583247d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.991+0000 7f36a0f4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d3f50 0x5583247d6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54114/0 (socket says 172.21.15.45:54114) 2024-05-08T05:47:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.991+0000 7f36a0f4b640 1 -- 172.21.15.45:0/2970971202 learned_addr learned my addr 172.21.15.45:0/2970971202 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.991+0000 7f369bfff640 1 --2- 172.21.15.45:0/2970971202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247d4330 0x5583247d6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.991+0000 7f36a174c640 1 --2- 172.21.15.45:0/2970971202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247a6bb0 0x5583247aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.992+0000 7f36a0f4b640 1 -- 172.21.15.45:0/2970971202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247a6bb0 msgr2=0x5583247aa3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.992+0000 7f36a0f4b640 1 --2- 172.21.15.45:0/2970971202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247a6bb0 0x5583247aa3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.992+0000 7f36a0f4b640 1 -- 172.21.15.45:0/2970971202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247d4330 msgr2=0x5583247d6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.992+0000 7f36a0f4b640 1 --2- 172.21.15.45:0/2970971202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247d4330 0x5583247d6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.992+0000 7f36a0f4b640 1 -- 172.21.15.45:0/2970971202 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583246d15d0 con 0x5583247d3f50 2024-05-08T05:47:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.992+0000 7f369bfff640 1 --2- 172.21.15.45:0/2970971202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247d4330 0x5583247d6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:14.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.992+0000 7f36a174c640 1 --2- 172.21.15.45:0/2970971202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247a6bb0 0x5583247aa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.992+0000 7f36a0f4b640 1 --2- 172.21.15.45:0/2970971202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d3f50 0x5583247d6760 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f368c0052c0 tx=0x7f368c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a9575658aa278 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.992+0000 7f36827fc640 1 -- 172.21.15.45:0/2970971202 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c00e4b0 con 0x5583247d3f50 2024-05-08T05:47:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.992+0000 7f36827fc640 1 -- 172.21.15.45:0/2970971202 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f368c00e650 con 0x5583247d3f50 2024-05-08T05:47:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.992+0000 7f36827fc640 1 -- 172.21.15.45:0/2970971202 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c00ea20 con 0x5583247d3f50 2024-05-08T05:47:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.993+0000 7f36a5f9d740 1 -- 172.21.15.45:0/2970971202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d3f50 msgr2=0x5583247d6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.993+0000 7f36a5f9d740 1 --2- 172.21.15.45:0/2970971202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d3f50 0x5583247d6760 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f368c0052c0 tx=0x7f368c011000 comp rx=0 tx=0).stop 2024-05-08T05:47:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.993+0000 7f36a5f9d740 1 -- 172.21.15.45:0/2970971202 shutdown_connections 2024-05-08T05:47:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.993+0000 7f36a5f9d740 1 --2- 172.21.15.45:0/2970971202 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247a6bb0 0x5583247aa3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.993+0000 7f36a5f9d740 1 --2- 172.21.15.45:0/2970971202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247d4330 0x5583247d6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.993+0000 7f36a5f9d740 1 --2- 172.21.15.45:0/2970971202 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d3f50 0x5583247d6760 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.993+0000 7f36a5f9d740 1 -- 172.21.15.45:0/2970971202 >> 172.21.15.45:0/2970971202 conn(0x5583247a8e20 msgr2=0x5583247f1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.993+0000 7f36a5f9d740 1 -- 172.21.15.45:0/2970971202 shutdown_connections 2024-05-08T05:47:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.993+0000 7f36a5f9d740 1 -- 172.21.15.45:0/2970971202 wait complete. 2024-05-08T05:47:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.993+0000 7f36a5f9d740 1 -- start start 2024-05-08T05:47:14.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.993+0000 7f36a5f9d740 1 Processor -- start 2024-05-08T05:47:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.994+0000 7f36a5f9d740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247d3f50 0x5583247ebe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.994+0000 7f36a5f9d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d4330 0x5583247eb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.994+0000 7f36a5f9d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247ca140 0x5583247eb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:14.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.994+0000 7f36a5f9d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5583247abfd0 con 0x5583247d4330 2024-05-08T05:47:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.994+0000 7f36a5f9d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5583246cd760 con 0x5583247ca140 2024-05-08T05:47:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.994+0000 7f36a5f9d740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5583247ac2b0 con 0x5583247d3f50 2024-05-08T05:47:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.994+0000 7f36a5f9d740 10 client.0 _ll_register_callbacks cb 0x558324706cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:47:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.994+0000 7f369bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d4330 0x5583247eb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.994+0000 7f369bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d4330 0x5583247eb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54118/0 (socket says 172.21.15.45:54118) 2024-05-08T05:47:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.994+0000 7f369bfff640 1 -- 172.21.15.45:0/1639221358 learned_addr learned my addr 172.21.15.45:0/1639221358 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.994+0000 7f36a174c640 1 --2- 172.21.15.45:0/1639221358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247ca140 0x5583247eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.994+0000 7f36a0f4b640 1 --2- 172.21.15.45:0/1639221358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247d3f50 0x5583247ebe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f369bfff640 1 -- 172.21.15.45:0/1639221358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247d3f50 msgr2=0x5583247ebe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f369bfff640 1 --2- 172.21.15.45:0/1639221358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247d3f50 0x5583247ebe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f369bfff640 1 -- 172.21.15.45:0/1639221358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247ca140 msgr2=0x5583247eb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f369bfff640 1 --2- 172.21.15.45:0/1639221358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247ca140 0x5583247eb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f369bfff640 1 -- 172.21.15.45:0/1639221358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583244222d0 con 0x5583247d4330 2024-05-08T05:47:15.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f36a0f4b640 1 --2- 172.21.15.45:0/1639221358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247d3f50 0x5583247ebe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f369bfff640 1 --2- 172.21.15.45:0/1639221358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d4330 0x5583247eb220 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f3690009720 tx=0x7f3690004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9951c8ef2bb407b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f367b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:47:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f367b7fe640 1 -- 172.21.15.45:0/1639221358 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690043070 con 0x5583247d4330 2024-05-08T05:47:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f367b7fe640 1 -- 172.21.15.45:0/1639221358 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7f3690003a20 con 0x5583247d4330 2024-05-08T05:47:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f36a5f9d740 1 -- 172.21.15.45:0/1639221358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f368c00e650 con 0x5583247d4330 2024-05-08T05:47:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f367b7fe640 1 -- 172.21.15.45:0/1639221358 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f3690037040 con 0x5583247d4330 2024-05-08T05:47:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f36a5f9d740 10 client.6975 fetch_fsmap learned FSMap version 66 2024-05-08T05:47:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f36a5f9d740 1 -- 172.21.15.45:0/1639221358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x5583244ec6b0 con 0x5583247d4330 2024-05-08T05:47:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f367b7fe640 1 -- 172.21.15.45:0/1639221358 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f368c00e650 con 0x5583247d4330 2024-05-08T05:47:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f367b7fe640 1 -- 172.21.15.45:0/1639221358 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7f3690003c20 con 0x5583247d4330 2024-05-08T05:47:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f36a5f9d740 10 client.6975 fetch_fsmap finished waiting for FSMap version 66 2024-05-08T05:47:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f36a5f9d740 10 client.6975 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:47:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f36a5f9d740 20 client.6975 populate_metadata read hostname 'smithi045' 2024-05-08T05:47:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f36a5f9d740 1 --2- 172.21.15.45:0/1639221358 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x5583247f3ee0 0x55832480fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:15.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f36a5f9d740 4 client.6975 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:47:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.995+0000 7f36a5f9d740 1 -- 172.21.15.45:0/1639221358 --> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558324694760 con 0x5583247f3ee0 2024-05-08T05:47:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.996+0000 7f36a0f4b640 1 --2- 172.21.15.45:0/1639221358 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x5583247f3ee0 0x55832480fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.996+0000 7f36a0f4b640 1 --2- 172.21.15.45:0/1639221358 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x5583247f3ee0 0x55832480fdc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71d7307f8607005f server_cookie=ca8e88befd30a479 in_seq=0 out_seq=0 2024-05-08T05:47:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.996+0000 7f367b7fe640 10 client.6975 ms_handle_connect on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.997+0000 7f367b7fe640 1 -- 172.21.15.45:0/1639221358 <== mds.0 v2:172.21.15.63:6835/3194497846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558324694760 con 0x5583247f3ee0 2024-05-08T05:47:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.997+0000 7f367b7fe640 10 client.6975 handle_command_reply: tid=0 2024-05-08T05:47:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.997+0000 7f36a5f9d740 1 client.6975 shutdown 2024-05-08T05:47:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.997+0000 7f36a5f9d740 1 -- 172.21.15.45:0/1639221358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d4330 msgr2=0x5583247eb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.997+0000 7f36a5f9d740 1 --2- 172.21.15.45:0/1639221358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d4330 0x5583247eb220 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f3690009720 tx=0x7f3690004730 comp rx=0 tx=0).stop 2024-05-08T05:47:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.997+0000 7f36a5f9d740 1 -- 172.21.15.45:0/1639221358 shutdown_connections 2024-05-08T05:47:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.997+0000 7f36a5f9d740 1 --2- 172.21.15.45:0/1639221358 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x5583247f3ee0 0x55832480fdc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.997+0000 7f36a5f9d740 1 --2- 172.21.15.45:0/1639221358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247ca140 0x5583247eb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.997+0000 7f36a5f9d740 1 --2- 172.21.15.45:0/1639221358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d4330 0x5583247eb220 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.997+0000 7f36a5f9d740 1 --2- 172.21.15.45:0/1639221358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247d3f50 0x5583247ebe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.997+0000 7f367b7fe640 0 client.6975 ms_handle_reset on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.997+0000 7f36a5f9d740 1 -- 172.21.15.45:0/1639221358 >> 172.21.15.45:0/1639221358 conn(0x5583247a6bb0 msgr2=0x5583247a7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.997+0000 7f36a5f9d740 1 -- 172.21.15.45:0/1639221358 shutdown_connections 2024-05-08T05:47:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.997+0000 7f36a5f9d740 1 -- 172.21.15.45:0/1639221358 wait complete. 2024-05-08T05:47:15.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:14.997+0000 7f36a5f9d740 20 client.6975 trim_cache size 0 max 16384 2024-05-08T05:47:15.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.016+0000 7f36a5f9d740 1 Processor -- start 2024-05-08T05:47:15.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.016+0000 7f36a5f9d740 1 -- start start 2024-05-08T05:47:15.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.017+0000 7f36a5f9d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247ca140 0x5583247d3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.017+0000 7f36a5f9d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247d3f50 0x5583247aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.017+0000 7f36a5f9d740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247d4330 0x55832486a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.017+0000 7f36a5f9d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f369c201d90 con 0x5583247ca140 2024-05-08T05:47:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.017+0000 7f36a5f9d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f369c200df0 con 0x5583247d3f50 2024-05-08T05:47:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.017+0000 7f36a5f9d740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f369c0b2870 con 0x5583247d4330 2024-05-08T05:47:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.017+0000 7f36a0f4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247ca140 0x5583247d3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.017+0000 7f36a0f4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247ca140 0x5583247d3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54128/0 (socket says 172.21.15.45:54128) 2024-05-08T05:47:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.017+0000 7f36a0f4b640 1 -- 172.21.15.45:0/173972084 learned_addr learned my addr 172.21.15.45:0/173972084 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.017+0000 7f369bfff640 1 --2- 172.21.15.45:0/173972084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247d3f50 0x5583247aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.017+0000 7f36a174c640 1 --2- 172.21.15.45:0/173972084 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247d4330 0x55832486a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.017+0000 7f36a0f4b640 1 -- 172.21.15.45:0/173972084 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247d4330 msgr2=0x55832486a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.017+0000 7f36a0f4b640 1 --2- 172.21.15.45:0/173972084 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247d4330 0x55832486a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.017+0000 7f36a0f4b640 1 -- 172.21.15.45:0/173972084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247d3f50 msgr2=0x5583247aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:15.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.017+0000 7f36a0f4b640 1 --2- 172.21.15.45:0/173972084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247d3f50 0x5583247aa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.017+0000 7f36a0f4b640 1 -- 172.21.15.45:0/173972084 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583247d6720 con 0x5583247ca140 2024-05-08T05:47:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.017+0000 7f36a0f4b640 1 --2- 172.21.15.45:0/173972084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247ca140 0x5583247d3960 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f368c001e60 tx=0x7f368c017070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e365c4d04f250f83 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.017+0000 7f3681ffb640 1 -- 172.21.15.45:0/173972084 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c00e690 con 0x5583247ca140 2024-05-08T05:47:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.017+0000 7f3681ffb640 1 -- 172.21.15.45:0/173972084 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f368c00ea40 con 0x5583247ca140 2024-05-08T05:47:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.017+0000 7f3681ffb640 1 -- 172.21.15.45:0/173972084 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c00edf0 con 0x5583247ca140 2024-05-08T05:47:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.018+0000 7f36a5f9d740 1 -- 172.21.15.45:0/173972084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247ca140 msgr2=0x5583247d3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.018+0000 7f36a5f9d740 1 --2- 172.21.15.45:0/173972084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247ca140 0x5583247d3960 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f368c001e60 tx=0x7f368c017070 comp rx=0 tx=0).stop 2024-05-08T05:47:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.018+0000 7f36a5f9d740 1 -- 172.21.15.45:0/173972084 shutdown_connections 2024-05-08T05:47:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.018+0000 7f36a5f9d740 1 --2- 172.21.15.45:0/173972084 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247d4330 0x55832486a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.018+0000 7f36a5f9d740 1 --2- 172.21.15.45:0/173972084 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247d3f50 0x5583247aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.018+0000 7f36a5f9d740 1 --2- 172.21.15.45:0/173972084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247ca140 0x5583247d3960 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.018+0000 7f36a5f9d740 1 -- 172.21.15.45:0/173972084 >> 172.21.15.45:0/173972084 conn(0x5583247a6bb0 msgr2=0x5583247f4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.018+0000 7f36a5f9d740 1 -- 172.21.15.45:0/173972084 shutdown_connections 2024-05-08T05:47:15.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.018+0000 7f36a5f9d740 1 -- 172.21.15.45:0/173972084 wait complete. 2024-05-08T05:47:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.018+0000 7f36a5f9d740 1 -- start start 2024-05-08T05:47:15.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.019+0000 7f36a5f9d740 1 Processor -- start 2024-05-08T05:47:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.019+0000 7f36a5f9d740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247ca140 0x55832486ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.019+0000 7f36a5f9d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d3f50 0x5583247e70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.019+0000 7f36a5f9d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247d4330 0x5583247e7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.019+0000 7f36a5f9d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5583247f3d60 con 0x5583247d3f50 2024-05-08T05:47:15.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.019+0000 7f36a5f9d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5583247eaa80 con 0x5583247d4330 2024-05-08T05:47:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.019+0000 7f36a5f9d740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5583247f4320 con 0x5583247ca140 2024-05-08T05:47:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.020+0000 7f36a5f9d740 10 client.0 _ll_register_callbacks cb 0x7f3690035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:47:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.020+0000 7f369bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d3f50 0x5583247e70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.020+0000 7f369bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d3f50 0x5583247e70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54130/0 (socket says 172.21.15.45:54130) 2024-05-08T05:47:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.020+0000 7f369bfff640 1 -- 172.21.15.45:0/632485766 learned_addr learned my addr 172.21.15.45:0/632485766 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.020+0000 7f36a174c640 1 --2- 172.21.15.45:0/632485766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247d4330 0x5583247e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.020+0000 7f36a0f4b640 1 --2- 172.21.15.45:0/632485766 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247ca140 0x55832486ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.020+0000 7f369bfff640 1 -- 172.21.15.45:0/632485766 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247ca140 msgr2=0x55832486ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.020+0000 7f369bfff640 1 --2- 172.21.15.45:0/632485766 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247ca140 0x55832486ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.020+0000 7f369bfff640 1 -- 172.21.15.45:0/632485766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247d4330 msgr2=0x5583247e7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.020+0000 7f369bfff640 1 --2- 172.21.15.45:0/632485766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247d4330 0x5583247e7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.020+0000 7f369bfff640 1 -- 172.21.15.45:0/632485766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583247c9930 con 0x5583247d3f50 2024-05-08T05:47:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.020+0000 7f369bfff640 1 --2- 172.21.15.45:0/632485766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d3f50 0x5583247e70f0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f369002a250 tx=0x7f3690009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f5c4d966c68c93 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.020+0000 7f367b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:47:15.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.020+0000 7f367b7fe640 1 -- 172.21.15.45:0/632485766 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690056060 con 0x5583247d3f50 2024-05-08T05:47:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.020+0000 7f367b7fe640 1 -- 172.21.15.45:0/632485766 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7f369003d860 con 0x5583247d3f50 2024-05-08T05:47:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.020+0000 7f36a5f9d740 1 -- 172.21.15.45:0/632485766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f368c00ea40 con 0x5583247d3f50 2024-05-08T05:47:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.020+0000 7f367b7fe640 1 -- 172.21.15.45:0/632485766 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f3690043070 con 0x5583247d3f50 2024-05-08T05:47:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.020+0000 7f36a5f9d740 10 client.6981 fetch_fsmap learned FSMap version 66 2024-05-08T05:47:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.020+0000 7f36a5f9d740 1 -- 172.21.15.45:0/632485766 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x5583247ea540 con 0x5583247d3f50 2024-05-08T05:47:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.021+0000 7f367b7fe640 1 -- 172.21.15.45:0/632485766 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f368c00ea40 con 0x5583247d3f50 2024-05-08T05:47:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.021+0000 7f367b7fe640 1 -- 172.21.15.45:0/632485766 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7f3690050040 con 0x5583247d3f50 2024-05-08T05:47:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.021+0000 7f36a5f9d740 10 client.6981 fetch_fsmap finished waiting for FSMap version 66 2024-05-08T05:47:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.021+0000 7f36a5f9d740 10 client.6981 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:47:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.021+0000 7f36a5f9d740 20 client.6981 populate_metadata read hostname 'smithi045' 2024-05-08T05:47:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.021+0000 7f36a5f9d740 1 --2- 172.21.15.45:0/632485766 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x5583247f1760 0x5583247f1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.021+0000 7f36a5f9d740 4 client.6981 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:47:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.021+0000 7f36a5f9d740 1 -- 172.21.15.45:0/632485766 --> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] -- command(tid 0: {"prefix": "status"}) -- 0x55832471e4e0 con 0x5583247f1760 2024-05-08T05:47:15.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.021+0000 7f36a0f4b640 1 --2- 172.21.15.45:0/632485766 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x5583247f1760 0x5583247f1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.021+0000 7f367b7fe640 10 client.6981 ms_handle_connect on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.021+0000 7f36a0f4b640 1 --2- 172.21.15.45:0/632485766 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x5583247f1760 0x5583247f1b40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79eb06498229330c server_cookie=d3df2dfff310b5d1 in_seq=0 out_seq=0 2024-05-08T05:47:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.022+0000 7f367b7fe640 1 -- 172.21.15.45:0/632485766 <== mds.0 v2:172.21.15.63:6835/3194497846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x55832471e4e0 con 0x5583247f1760 2024-05-08T05:47:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.022+0000 7f367b7fe640 10 client.6981 handle_command_reply: tid=0 2024-05-08T05:47:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.022+0000 7f36a5f9d740 1 client.6981 shutdown 2024-05-08T05:47:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.022+0000 7f36a5f9d740 1 -- 172.21.15.45:0/632485766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d3f50 msgr2=0x5583247e70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.022+0000 7f36a5f9d740 1 --2- 172.21.15.45:0/632485766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d3f50 0x5583247e70f0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f369002a250 tx=0x7f3690009190 comp rx=0 tx=0).stop 2024-05-08T05:47:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.022+0000 7f36a5f9d740 1 -- 172.21.15.45:0/632485766 shutdown_connections 2024-05-08T05:47:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.022+0000 7f36a5f9d740 1 --2- 172.21.15.45:0/632485766 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x5583247f1760 0x5583247f1b40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.022+0000 7f36a5f9d740 1 --2- 172.21.15.45:0/632485766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5583247d4330 0x5583247e7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.022+0000 7f36a5f9d740 1 --2- 172.21.15.45:0/632485766 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5583247d3f50 0x5583247e70f0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.022+0000 7f367b7fe640 0 client.6981 ms_handle_reset on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.022+0000 7f36a5f9d740 1 --2- 172.21.15.45:0/632485766 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5583247ca140 0x55832486ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.022+0000 7f36a5f9d740 1 -- 172.21.15.45:0/632485766 >> 172.21.15.45:0/632485766 conn(0x5583247a6bb0 msgr2=0x5583247ab950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:15.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.022+0000 7f36a5f9d740 1 -- 172.21.15.45:0/632485766 shutdown_connections 2024-05-08T05:47:15.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.022+0000 7f36a5f9d740 1 -- 172.21.15.45:0/632485766 wait complete. 2024-05-08T05:47:15.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.022+0000 7f36a5f9d740 20 client.6981 trim_cache size 0 max 16384 2024-05-08T05:47:15.028 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:47:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "e9e1170a-fd27-4625-97a9-5d5f104d13a2", 2024-05-08T05:47:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-05-08T05:47:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 6759, 2024-05-08T05:47:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-05-08T05:47:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-05-08T05:47:15.028 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-05-08T05:47:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 4.48525946, 2024-05-08T05:47:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 66, 2024-05-08T05:47:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 90, 2024-05-08T05:47:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 89, 2024-05-08T05:47:15.029 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 19.508083074999998 2024-05-08T05:47:15.029 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:47:15.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.022+0000 7f36a31af640 1 -- 172.21.15.45:0/2235712952 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f36880762d0 msgr2=0x7f3688078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:15.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.022+0000 7f36a31af640 1 --2- 172.21.15.45:0/2235712952 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f36880762d0 0x7f3688078790 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f368c000f60 tx=0x7f368c009040 comp rx=0 tx=0).stop 2024-05-08T05:47:15.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.022+0000 7f36a31af640 1 -- 172.21.15.45:0/2235712952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f369c15f380 msgr2=0x7f369c0cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:15.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.022+0000 7f36a31af640 1 --2- 172.21.15.45:0/2235712952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f369c15f380 0x7f369c0cb220 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f369400cb40 tx=0x7f369400a060 comp rx=0 tx=0).stop 2024-05-08T05:47:15.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.023+0000 7f36a31af640 1 -- 172.21.15.45:0/2235712952 shutdown_connections 2024-05-08T05:47:15.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.023+0000 7f36a31af640 1 --2- 172.21.15.45:0/2235712952 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f36880762d0 0x7f3688078790 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.023+0000 7f36a31af640 1 --2- 172.21.15.45:0/2235712952 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f369c0d3b00 0x7f369c0cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.023+0000 7f36a31af640 1 --2- 172.21.15.45:0/2235712952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f369c15f380 0x7f369c0cb220 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.023+0000 7f36a31af640 1 --2- 172.21.15.45:0/2235712952 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f369c0d0ec0 0x7f369c158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.023+0000 7f36a31af640 1 -- 172.21.15.45:0/2235712952 >> 172.21.15.45:0/2235712952 conn(0x7f369c0d4fe0 msgr2=0x7f369c0ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:15.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.023+0000 7f36a31af640 1 -- 172.21.15.45:0/2235712952 shutdown_connections 2024-05-08T05:47:15.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.023+0000 7f36a31af640 1 -- 172.21.15.45:0/2235712952 wait complete. 2024-05-08T05:47:15.034 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T05:47:15.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.136+0000 7f5227733640 1 Processor -- start 2024-05-08T05:47:15.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.136+0000 7f5227733640 1 -- start start 2024-05-08T05:47:15.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.137+0000 7f5227733640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f52200d0530 0x7f52200b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:15.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.137+0000 7f5227733640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f522015d2a0 con 0x7f522006f060 2024-05-08T05:47:15.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.137+0000 7f5227733640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52201609b0 con 0x7f522006e0f0 2024-05-08T05:47:15.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.137+0000 7f5227733640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f522006f440 con 0x7f52200d0530 2024-05-08T05:47:15.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.137+0000 7f52254cf640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f522006f060 0x7f522006fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41276/0 (socket says 172.21.15.45:41276) 2024-05-08T05:47:15.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.137+0000 7f52254cf640 1 -- 172.21.15.45:0/2080011577 learned_addr learned my addr 172.21.15.45:0/2080011577 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:47:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.138+0000 7f5225cd0640 1 --2- 172.21.15.45:0/2080011577 >> v2:172.21.15.63:3300/0 conn(0x7f52200d0530 0x7f52200b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.138+0000 7f5217fff640 1 -- 172.21.15.45:0/2080011577 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1491785501 0 0) 0x7f522015d2a0 con 0x7f522006f060 2024-05-08T05:47:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.138+0000 7f5217fff640 1 -- 172.21.15.45:0/2080011577 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f520c003520 con 0x7f522006f060 2024-05-08T05:47:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.138+0000 7f5217fff640 1 -- 172.21.15.45:0/2080011577 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2485860598 0 0) 0x7f52201609b0 con 0x7f522006e0f0 2024-05-08T05:47:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.138+0000 7f5217fff640 1 -- 172.21.15.45:0/2080011577 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f522015d2a0 con 0x7f522006e0f0 2024-05-08T05:47:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.138+0000 7f5217fff640 1 -- 172.21.15.45:0/2080011577 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 641338629 0 0) 0x7f520c003520 con 0x7f522006f060 2024-05-08T05:47:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.138+0000 7f5225cd0640 1 -- 172.21.15.45:0/2080011577 >> v1:172.21.15.63:6789/0 conn(0x7f522006e0f0 legacy=0x7f52200cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.138+0000 7f5225cd0640 1 -- 172.21.15.45:0/2080011577 >> v1:172.21.15.45:6789/0 conn(0x7f522006f060 legacy=0x7f522006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:15.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.138+0000 7f5225cd0640 1 -- 172.21.15.45:0/2080011577 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5220160c90 con 0x7f52200d0530 2024-05-08T05:47:15.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.139+0000 7f5225cd0640 1 --2- 172.21.15.45:0/2080011577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52200d0530 0x7f52200b3f00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5218009d90 tx=0x7f521802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa4c15dea931e9b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:15.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.139+0000 7f5217fff640 1 -- 172.21.15.45:0/2080011577 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c0016f0 con 0x7f52200d0530 2024-05-08T05:47:15.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.139+0000 7f5217fff640 1 -- 172.21.15.45:0/2080011577 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5218007540 con 0x7f52200d0530 2024-05-08T05:47:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.139+0000 7f5217fff640 1 -- 172.21.15.45:0/2080011577 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5218007910 con 0x7f52200d0530 2024-05-08T05:47:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.139+0000 7f5227733640 1 -- 172.21.15.45:0/2080011577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52200d0530 msgr2=0x7f52200b3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.139+0000 7f5227733640 1 --2- 172.21.15.45:0/2080011577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52200d0530 0x7f52200b3f00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5218009d90 tx=0x7f521802a680 comp rx=0 tx=0).stop 2024-05-08T05:47:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.139+0000 7f5227733640 1 -- 172.21.15.45:0/2080011577 shutdown_connections 2024-05-08T05:47:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.139+0000 7f5227733640 1 --2- 172.21.15.45:0/2080011577 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52200d0530 0x7f52200b3f00 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.140+0000 7f5227733640 1 -- 172.21.15.45:0/2080011577 >> 172.21.15.45:0/2080011577 conn(0x7f522006c3a0 msgr2=0x7f522006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.140+0000 7f5227733640 1 -- 172.21.15.45:0/2080011577 shutdown_connections 2024-05-08T05:47:15.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.140+0000 7f5227733640 1 -- 172.21.15.45:0/2080011577 wait complete. 2024-05-08T05:47:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.141+0000 7f5227733640 1 Processor -- start 2024-05-08T05:47:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.141+0000 7f5227733640 1 -- start start 2024-05-08T05:47:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f5227733640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f522006e0f0 0x7f522014a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:15.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f5227733640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f522006f060 0x7f522009ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f5227733640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52200d0530 0x7f52201f2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f5227733640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5220161530 con 0x7f522006e0f0 2024-05-08T05:47:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f5227733640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5220161250 con 0x7f52200d0530 2024-05-08T05:47:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f5227733640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5220161810 con 0x7f522006f060 2024-05-08T05:47:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f52254cf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f522006e0f0 0x7f522014a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f52254cf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f522006e0f0 0x7f522014a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54144/0 (socket says 172.21.15.45:54144) 2024-05-08T05:47:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f52254cf640 1 -- 172.21.15.45:0/3262557899 learned_addr learned my addr 172.21.15.45:0/3262557899 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f5224cce640 1 --2- 172.21.15.45:0/3262557899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f522006f060 0x7f522009ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f5225cd0640 1 --2- 172.21.15.45:0/3262557899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52200d0530 0x7f52201f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f52254cf640 1 -- 172.21.15.45:0/3262557899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f522006f060 msgr2=0x7f522009ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f52254cf640 1 --2- 172.21.15.45:0/3262557899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f522006f060 0x7f522009ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f52254cf640 1 -- 172.21.15.45:0/3262557899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52200d0530 msgr2=0x7f52201f2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f52254cf640 1 --2- 172.21.15.45:0/3262557899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52200d0530 0x7f52201f2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f52254cf640 1 -- 172.21.15.45:0/3262557899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522009d430 con 0x7f522006e0f0 2024-05-08T05:47:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f52254cf640 1 --2- 172.21.15.45:0/3262557899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f522006e0f0 0x7f522014a610 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f521c00c560 tx=0x7f521c00ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f5215ffb640 1 -- 172.21.15.45:0/3262557899 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c017070 con 0x7f522006e0f0 2024-05-08T05:47:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f5215ffb640 1 -- 172.21.15.45:0/3262557899 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f521c005ce0 con 0x7f522006e0f0 2024-05-08T05:47:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f5227733640 1 -- 172.21.15.45:0/3262557899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f522009d710 con 0x7f522006e0f0 2024-05-08T05:47:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.142+0000 7f5215ffb640 1 -- 172.21.15.45:0/3262557899 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c007290 con 0x7f522006e0f0 2024-05-08T05:47:15.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.143+0000 7f5227733640 1 -- 172.21.15.45:0/3262557899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52201552e0 con 0x7f522006e0f0 2024-05-08T05:47:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.144+0000 7f5227733640 1 -- 172.21.15.45:0/3262557899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52200693f0 con 0x7f522006e0f0 2024-05-08T05:47:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.144+0000 7f5215ffb640 1 -- 172.21.15.45:0/3262557899 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f521c01a070 con 0x7f522006e0f0 2024-05-08T05:47:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.145+0000 7f5215ffb640 1 --2- 172.21.15.45:0/3262557899 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5204076300 0x7f52040787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.145+0000 7f5215ffb640 1 -- 172.21.15.45:0/3262557899 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6303+0+0 (secure 0 0 0) 0x7f521c096d40 con 0x7f522006e0f0 2024-05-08T05:47:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.145+0000 7f5224cce640 1 --2- 172.21.15.45:0/3262557899 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5204076300 0x7f52040787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:15.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.146+0000 7f5224cce640 1 --2- 172.21.15.45:0/3262557899 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5204076300 0x7f52040787c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5220158770 tx=0x7f5210009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:15.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.149+0000 7f5215ffb640 1 -- 172.21.15.45:0/3262557899 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f521c060450 con 0x7f522006e0f0 2024-05-08T05:47:15.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.377+0000 7f5227733640 1 -- 172.21.15.45:0/3262557899 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f522014ab50 con 0x7f522006e0f0 2024-05-08T05:47:15.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.378+0000 7f5215ffb640 1 -- 172.21.15.45:0/3262557899 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) ==== 53+0+73 (secure 0 0 0) 0x7f521c05fd50 con 0x7f522006e0f0 2024-05-08T05:47:15.378 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T05:47:15.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.379+0000 7f5227733640 1 -- 172.21.15.45:0/3262557899 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5204076300 msgr2=0x7f52040787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:15.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.379+0000 7f5227733640 1 --2- 172.21.15.45:0/3262557899 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5204076300 0x7f52040787c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5220158770 tx=0x7f5210009040 comp rx=0 tx=0).stop 2024-05-08T05:47:15.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.379+0000 7f5227733640 1 -- 172.21.15.45:0/3262557899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f522006e0f0 msgr2=0x7f522014a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:15.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.379+0000 7f5227733640 1 --2- 172.21.15.45:0/3262557899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f522006e0f0 0x7f522014a610 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f521c00c560 tx=0x7f521c00ca20 comp rx=0 tx=0).stop 2024-05-08T05:47:15.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.380+0000 7f5227733640 1 -- 172.21.15.45:0/3262557899 shutdown_connections 2024-05-08T05:47:15.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.380+0000 7f5227733640 1 --2- 172.21.15.45:0/3262557899 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5204076300 0x7f52040787c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.380+0000 7f5227733640 1 --2- 172.21.15.45:0/3262557899 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f522006f060 0x7f522009ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.380+0000 7f5227733640 1 --2- 172.21.15.45:0/3262557899 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52200d0530 0x7f52201f2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.380+0000 7f5227733640 1 --2- 172.21.15.45:0/3262557899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f522006e0f0 0x7f522014a610 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:15.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.380+0000 7f5227733640 1 -- 172.21.15.45:0/3262557899 >> 172.21.15.45:0/3262557899 conn(0x7f522006c3a0 msgr2=0x7f522006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:15.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.380+0000 7f5227733640 1 -- 172.21.15.45:0/3262557899 shutdown_connections 2024-05-08T05:47:15.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15.380+0000 7f5227733640 1 -- 172.21.15.45:0/3262557899 wait complete. 2024-05-08T05:47:15.391 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T05:47:15.391 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:47:15.391 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-05-08T05:47:15.391 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T05:47:15.391 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:47:15.391 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T05:47:15.391 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T05:47:15.391 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:47:15.391 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:47:15.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:15.391 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-05-08T05:47:15.408 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:47:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:47:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:47:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:47:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-05-08T05:47:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:47:15.408 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:47:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:47:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-05-08T05:47:15.408 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:47:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:47:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-05-08T05:47:15.408 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:47:15.408 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-05-08T05:47:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-05-08T05:47:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:47:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42389sec preferred_lft 42389sec 2024-05-08T05:47:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-05-08T05:47:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:47:15.409 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:47:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-05-08T05:47:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-05-08T05:47:15.409 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:47:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T05:47:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T05:47:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:47:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c80a:aff:fe94:c77/64 scope link 2024-05-08T05:47:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:47:15.409 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:47:15.409 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 82:f3:eb:6a:57:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:47:15.410 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::80f3:ebff:fe6a:57ba/64 scope link 2024-05-08T05:47:15.410 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:47:15.410 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:47:15.410 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T05:47:15.410 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::14a2:64ff:feda:3a9c/64 scope link 2024-05-08T05:47:15.410 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:47:15.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:15.410 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:47:15.472 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:47:15.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:15.474 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-05-08T05:47:15.530 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-05-08T05:47:15.530 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:47:15.530 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:47:15.530 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:47:15.530 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:47:15.531 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:47:15.531 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:47:15.531 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:47:15.531 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:47:15.531 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:47:15.531 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:47:15.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:47:15.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:47:15.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:15.662 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:47:15.681 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T05:47:15.681 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T05:47:15.681 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:47:15.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:15.681 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-05-08T05:47:15.746 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:47:15.746 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T05:47:15.803 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:47:15.803 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:47:15.803 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:47:15.859 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T05:47:15.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:15.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:47:15.945 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:47:15.945 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:47:15.954 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:47:15.954 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:47:15.954 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-08T05:47:15.954 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@e9e1170a-fd27-4625-97a9-5d5f104d13a2.cephfs=/ 2024-05-08T05:47:15.954 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.63:3300/172.21.15.63:3301" will pass to kernel 2024-05-08T05:47:15.954 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T05:47:15.954 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:47:15.955 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:47:15.955 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:47:15.955 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:47:15.956 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:47:15.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:15.956 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:47:15.982 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:47:16.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:47:16.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:47:16.081 INFO:teuthology.orchestra.run.smithi045.stdout:client5888 2024-05-08T05:47:16.081 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:47:16.081 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:47:16.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:16.098 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-05-08T05:47:16.157 INFO:tasks.ceph.mds.b.smithi063.stderr:2024-05-08T05:47:16.157+0000 7ff8b9265640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-05-08T05:47:16.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:47:16.158 INFO:teuthology.orchestra.run.smithi045.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-05-08T05:47:16.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:16.158 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-05-08T05:47:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.305+0000 7f16d390a640 1 Processor -- start 2024-05-08T05:47:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.305+0000 7f16d390a640 1 -- start start 2024-05-08T05:47:16.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.306+0000 7f16d390a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16cc0cf8f0 0x7f16cc070500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.306+0000 7f16d390a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f16cc0cef40 0x7f16cc169b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.306+0000 7f16d390a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16cc0d2030 con 0x7f16cc0d3920 2024-05-08T05:47:16.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.306+0000 7f16d390a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f16cc15f8d0 con 0x7f16cc0cf8f0 2024-05-08T05:47:16.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.306+0000 7f16d390a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f16cc0d22b0 con 0x7f16cc0cef40 2024-05-08T05:47:16.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.306+0000 7f16d16a6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16cc0cf8f0 0x7f16cc070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.306+0000 7f16d16a6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16cc0cf8f0 0x7f16cc070500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54154/0 (socket says 172.21.15.45:54154) 2024-05-08T05:47:16.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.306+0000 7f16d16a6640 1 -- 172.21.15.45:0/2301195025 learned_addr learned my addr 172.21.15.45:0/2301195025 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:16.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.307+0000 7f16d1ea7640 1 --2- 172.21.15.45:0/2301195025 >> v2:172.21.15.63:3300/0 conn(0x7f16cc0cef40 0x7f16cc169b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.307+0000 7f16d16a6640 1 -- 172.21.15.45:0/2301195025 >> v2:172.21.15.63:3300/0 conn(0x7f16cc0cef40 msgr2=0x7f16cc169b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.307+0000 7f16d16a6640 1 --2- 172.21.15.45:0/2301195025 >> v2:172.21.15.63:3300/0 conn(0x7f16cc0cef40 0x7f16cc169b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.307+0000 7f16d16a6640 1 -- 172.21.15.45:0/2301195025 >> v1:172.21.15.63:6789/0 conn(0x7f16cc0d3920 legacy=0x7f16cc070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.307+0000 7f16d16a6640 1 -- 172.21.15.45:0/2301195025 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16cc0d2590 con 0x7f16cc0cf8f0 2024-05-08T05:47:16.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.307+0000 7f16d16a6640 1 --2- 172.21.15.45:0/2301195025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16cc0cf8f0 0x7f16cc070500 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f16cc073c40 tx=0x7f16c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef58ced12184830f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:16.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.308+0000 7f16cbfff640 1 -- 172.21.15.45:0/2301195025 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4037070 con 0x7f16cc0cf8f0 2024-05-08T05:47:16.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.308+0000 7f16cbfff640 1 -- 172.21.15.45:0/2301195025 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16c4032440 con 0x7f16cc0cf8f0 2024-05-08T05:47:16.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.308+0000 7f16cbfff640 1 -- 172.21.15.45:0/2301195025 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4032810 con 0x7f16cc0cf8f0 2024-05-08T05:47:16.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.308+0000 7f16d390a640 1 -- 172.21.15.45:0/2301195025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16cc0cf8f0 msgr2=0x7f16cc070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:16.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.308+0000 7f16d390a640 1 --2- 172.21.15.45:0/2301195025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16cc0cf8f0 0x7f16cc070500 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f16cc073c40 tx=0x7f16c402a270 comp rx=0 tx=0).stop 2024-05-08T05:47:16.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.308+0000 7f16d390a640 1 -- 172.21.15.45:0/2301195025 shutdown_connections 2024-05-08T05:47:16.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.308+0000 7f16d390a640 1 --2- 172.21.15.45:0/2301195025 >> v2:172.21.15.63:3300/0 conn(0x7f16cc0cef40 0x7f16cc169b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.308+0000 7f16d390a640 1 --2- 172.21.15.45:0/2301195025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16cc0cf8f0 0x7f16cc070500 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.308+0000 7f16d390a640 1 -- 172.21.15.45:0/2301195025 >> 172.21.15.45:0/2301195025 conn(0x7f16cc0d2e00 msgr2=0x7f16cc0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:16.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.308+0000 7f16d390a640 1 -- 172.21.15.45:0/2301195025 shutdown_connections 2024-05-08T05:47:16.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.308+0000 7f16d390a640 1 -- 172.21.15.45:0/2301195025 wait complete. 2024-05-08T05:47:16.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.309+0000 7f16d390a640 1 Processor -- start 2024-05-08T05:47:16.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.309+0000 7f16d390a640 1 -- start start 2024-05-08T05:47:16.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.310+0000 7f16d390a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16cc0cef40 0x7f16cc151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.310+0000 7f16d390a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16cc0cf8f0 0x7f16cc1522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.310+0000 7f16d390a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16cc0d3920 0x7f16cc0cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.310+0000 7f16d390a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc0cda10 con 0x7f16cc0d3920 2024-05-08T05:47:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.310+0000 7f16d390a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc0cd730 con 0x7f16cc0cf8f0 2024-05-08T05:47:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.310+0000 7f16d390a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f16cc071610 con 0x7f16cc0cef40 2024-05-08T05:47:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.310+0000 7f16d1ea7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16cc0d3920 0x7f16cc0cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.310+0000 7f16d1ea7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16cc0d3920 0x7f16cc0cb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54156/0 (socket says 172.21.15.45:54156) 2024-05-08T05:47:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.310+0000 7f16d1ea7640 1 -- 172.21.15.45:0/1691190412 learned_addr learned my addr 172.21.15.45:0/1691190412 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.311+0000 7f16d0ea5640 1 --2- 172.21.15.45:0/1691190412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16cc0cf8f0 0x7f16cc1522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.311+0000 7f16d16a6640 1 --2- 172.21.15.45:0/1691190412 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16cc0cef40 0x7f16cc151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.311+0000 7f16d1ea7640 1 -- 172.21.15.45:0/1691190412 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16cc0cef40 msgr2=0x7f16cc151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.311+0000 7f16d1ea7640 1 --2- 172.21.15.45:0/1691190412 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16cc0cef40 0x7f16cc151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.311+0000 7f16d1ea7640 1 -- 172.21.15.45:0/1691190412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16cc0cf8f0 msgr2=0x7f16cc1522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.311+0000 7f16d1ea7640 1 --2- 172.21.15.45:0/1691190412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16cc0cf8f0 0x7f16cc1522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.311+0000 7f16d1ea7640 1 -- 172.21.15.45:0/1691190412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16cc152d00 con 0x7f16cc0d3920 2024-05-08T05:47:16.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.311+0000 7f16d16a6640 1 --2- 172.21.15.45:0/1691190412 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16cc0cef40 0x7f16cc151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:16.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.311+0000 7f16d1ea7640 1 --2- 172.21.15.45:0/1691190412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16cc0d3920 0x7f16cc0cb460 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f16bc007e60 tx=0x7f16bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:16.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.311+0000 7f16c9ffb640 1 -- 172.21.15.45:0/1691190412 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc014420 con 0x7f16cc0d3920 2024-05-08T05:47:16.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.311+0000 7f16c9ffb640 1 -- 172.21.15.45:0/1691190412 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16bc019070 con 0x7f16cc0d3920 2024-05-08T05:47:16.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.311+0000 7f16d390a640 1 -- 172.21.15.45:0/1691190412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16cc152fe0 con 0x7f16cc0d3920 2024-05-08T05:47:16.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.311+0000 7f16c9ffb640 1 -- 172.21.15.45:0/1691190412 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc014820 con 0x7f16cc0d3920 2024-05-08T05:47:16.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.311+0000 7f16d390a640 1 -- 172.21.15.45:0/1691190412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16cc1535a0 con 0x7f16cc0d3920 2024-05-08T05:47:16.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.313+0000 7f16c9ffb640 1 -- 172.21.15.45:0/1691190412 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f16bc0149c0 con 0x7f16cc0d3920 2024-05-08T05:47:16.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.313+0000 7f16c9ffb640 1 --2- 172.21.15.45:0/1691190412 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f16b80762d0 0x7f16b8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.313+0000 7f16c9ffb640 1 -- 172.21.15.45:0/1691190412 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f16bc098050 con 0x7f16cc0d3920 2024-05-08T05:47:16.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.314+0000 7f16d16a6640 1 --2- 172.21.15.45:0/1691190412 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f16b80762d0 0x7f16b8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.314+0000 7f16d16a6640 1 --2- 172.21.15.45:0/1691190412 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f16b80762d0 0x7f16b8078790 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f16c4034000 tx=0x7f16c402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:16.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.314+0000 7f16d661d740 1 Processor -- start 2024-05-08T05:47:16.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.314+0000 7f16d661d740 1 -- start start 2024-05-08T05:47:16.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.315+0000 7f16d661d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d538932890 0x55d5389350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.315+0000 7f16d661d740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d538932c70 0x55d5389355e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.315+0000 7f16d661d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5389054f0 0x55d538908d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.315+0000 7f16d661d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d53886ebc0 con 0x55d5389054f0 2024-05-08T05:47:16.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.315+0000 7f16d661d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d538827d30 con 0x55d538932890 2024-05-08T05:47:16.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.315+0000 7f16d661d740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d53886ab80 con 0x55d538932c70 2024-05-08T05:47:16.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.315+0000 7f16d1ea7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5389054f0 0x55d538908d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.315+0000 7f16d1ea7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5389054f0 0x55d538908d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54162/0 (socket says 172.21.15.45:54162) 2024-05-08T05:47:16.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.315+0000 7f16d1ea7640 1 -- 172.21.15.45:0/3210459733 learned_addr learned my addr 172.21.15.45:0/3210459733 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:16.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.315+0000 7f16d0ea5640 1 --2- 172.21.15.45:0/3210459733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d538932c70 0x55d5389355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.315+0000 7f16d16a6640 1 --2- 172.21.15.45:0/3210459733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d538932890 0x55d5389350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.315+0000 7f16d1ea7640 1 -- 172.21.15.45:0/3210459733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d538932c70 msgr2=0x55d5389355e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.315+0000 7f16d1ea7640 1 --2- 172.21.15.45:0/3210459733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d538932c70 0x55d5389355e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.315+0000 7f16d1ea7640 1 -- 172.21.15.45:0/3210459733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d538932890 msgr2=0x55d5389350a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.315+0000 7f16d1ea7640 1 --2- 172.21.15.45:0/3210459733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d538932890 0x55d5389350a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.315+0000 7f16d1ea7640 1 -- 172.21.15.45:0/3210459733 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d53880c100 con 0x55d5389054f0 2024-05-08T05:47:16.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.316+0000 7f16d1ea7640 1 --2- 172.21.15.45:0/3210459733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5389054f0 0x55d538908d00 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00c040 tx=0x7f16bc060800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba91e70497b38f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:16.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.316+0000 7f16b2ffd640 1 -- 172.21.15.45:0/3210459733 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc07a020 con 0x55d5389054f0 2024-05-08T05:47:16.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.316+0000 7f16b2ffd640 1 -- 172.21.15.45:0/3210459733 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16bc060fd0 con 0x55d5389054f0 2024-05-08T05:47:16.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.316+0000 7f16b2ffd640 1 -- 172.21.15.45:0/3210459733 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc069ba0 con 0x55d5389054f0 2024-05-08T05:47:16.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.316+0000 7f16d661d740 1 -- 172.21.15.45:0/3210459733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5389054f0 msgr2=0x55d538908d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:16.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.316+0000 7f16d661d740 1 --2- 172.21.15.45:0/3210459733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5389054f0 0x55d538908d00 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00c040 tx=0x7f16bc060800 comp rx=0 tx=0).stop 2024-05-08T05:47:16.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.316+0000 7f16d661d740 1 -- 172.21.15.45:0/3210459733 shutdown_connections 2024-05-08T05:47:16.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.316+0000 7f16d661d740 1 --2- 172.21.15.45:0/3210459733 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d538932c70 0x55d5389355e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.316+0000 7f16d661d740 1 --2- 172.21.15.45:0/3210459733 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5389054f0 0x55d538908d00 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.316+0000 7f16d661d740 1 --2- 172.21.15.45:0/3210459733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d538932890 0x55d5389350a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.316+0000 7f16d661d740 1 -- 172.21.15.45:0/3210459733 >> 172.21.15.45:0/3210459733 conn(0x55d538907760 msgr2=0x55d5389500a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:16.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.316+0000 7f16d661d740 1 -- 172.21.15.45:0/3210459733 shutdown_connections 2024-05-08T05:47:16.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.316+0000 7f16d661d740 1 -- 172.21.15.45:0/3210459733 wait complete. 2024-05-08T05:47:16.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.317+0000 7f16d661d740 1 -- start start 2024-05-08T05:47:16.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.317+0000 7f16d661d740 1 Processor -- start 2024-05-08T05:47:16.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.318+0000 7f16d661d740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d538932890 0x55d538928e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.318+0000 7f16d661d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d538932c70 0x55d53892a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.318+0000 7f16d661d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5389294f0 0x55d538952480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.318+0000 7f16d661d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d53890a910 con 0x55d538932c70 2024-05-08T05:47:16.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.318+0000 7f16d661d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d53882b870 con 0x55d5389294f0 2024-05-08T05:47:16.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.318+0000 7f16d661d740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d53890abf0 con 0x55d538932890 2024-05-08T05:47:16.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.318+0000 7f16d0ea5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d538932c70 0x55d53892a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.318+0000 7f16d661d740 10 client.0 _ll_register_callbacks cb 0x55d53887bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:47:16.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.318+0000 7f16d0ea5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d538932c70 0x55d53892a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54164/0 (socket says 172.21.15.45:54164) 2024-05-08T05:47:16.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.318+0000 7f16d0ea5640 1 -- 172.21.15.45:0/655792887 learned_addr learned my addr 172.21.15.45:0/655792887 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:16.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.318+0000 7f16d1ea7640 1 --2- 172.21.15.45:0/655792887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5389294f0 0x55d538952480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.318+0000 7f16d16a6640 1 --2- 172.21.15.45:0/655792887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d538932890 0x55d538928e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.318+0000 7f16d0ea5640 1 -- 172.21.15.45:0/655792887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d538932890 msgr2=0x55d538928e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.318+0000 7f16d0ea5640 1 --2- 172.21.15.45:0/655792887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d538932890 0x55d538928e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.318+0000 7f16d0ea5640 1 -- 172.21.15.45:0/655792887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5389294f0 msgr2=0x55d538952480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.318+0000 7f16d0ea5640 1 --2- 172.21.15.45:0/655792887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5389294f0 0x55d538952480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.318+0000 7f16d0ea5640 1 -- 172.21.15.45:0/655792887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d53864c7b0 con 0x55d538932c70 2024-05-08T05:47:16.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.319+0000 7f16d0ea5640 1 --2- 172.21.15.45:0/655792887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d538932c70 0x55d53892a410 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f16c0006fd0 tx=0x7f16c0010680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8724119086b7ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:16.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.319+0000 7f16abfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:47:16.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.319+0000 7f16abfff640 1 -- 172.21.15.45:0/655792887 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c0018070 con 0x55d538932c70 2024-05-08T05:47:16.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.319+0000 7f16abfff640 1 -- 172.21.15.45:0/655792887 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f16c0014800 con 0x55d538932c70 2024-05-08T05:47:16.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.319+0000 7f16d661d740 1 -- 172.21.15.45:0/655792887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16bc060fd0 con 0x55d538932c70 2024-05-08T05:47:16.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.319+0000 7f16abfff640 1 -- 172.21.15.45:0/655792887 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f16c0023050 con 0x55d538932c70 2024-05-08T05:47:16.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.319+0000 7f16d661d740 10 client.6999 fetch_fsmap learned FSMap version 66 2024-05-08T05:47:16.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.319+0000 7f16d661d740 1 -- 172.21.15.45:0/655792887 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55d5386acf40 con 0x55d538932c70 2024-05-08T05:47:16.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.319+0000 7f16abfff640 1 -- 172.21.15.45:0/655792887 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16bc060fd0 con 0x55d538932c70 2024-05-08T05:47:16.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.319+0000 7f16abfff640 1 -- 172.21.15.45:0/655792887 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7f16c0026020 con 0x55d538932c70 2024-05-08T05:47:16.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.319+0000 7f16d661d740 10 client.6999 fetch_fsmap finished waiting for FSMap version 66 2024-05-08T05:47:16.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.319+0000 7f16d661d740 10 client.6999 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-08T05:47:16.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.319+0000 7f16d661d740 20 client.6999 populate_metadata read hostname 'smithi045' 2024-05-08T05:47:16.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.319+0000 7f16d661d740 1 --2- 172.21.15.45:0/655792887 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55d53892cca0 0x55d53892ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.319+0000 7f16d661d740 4 client.6999 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:47:16.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.319+0000 7f16d661d740 1 -- 172.21.15.45:0/655792887 --> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d53887c7d0 con 0x55d53892cca0 2024-05-08T05:47:16.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.320+0000 7f16d16a6640 1 --2- 172.21.15.45:0/655792887 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55d53892cca0 0x55d53892ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.320+0000 7f16d16a6640 1 --2- 172.21.15.45:0/655792887 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55d53892cca0 0x55d53892ea90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60f37f13206b202f server_cookie=25be7a5e61ae232f in_seq=0 out_seq=0 2024-05-08T05:47:16.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.320+0000 7f16abfff640 10 client.6999 ms_handle_connect on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:16.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.321+0000 7f16abfff640 1 -- 172.21.15.45:0/655792887 <== mds.0 v2:172.21.15.63:6835/3194497846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d53887c7d0 con 0x55d53892cca0 2024-05-08T05:47:16.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.321+0000 7f16abfff640 10 client.6999 handle_command_reply: tid=0 2024-05-08T05:47:16.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.321+0000 7f16d661d740 1 client.6999 shutdown 2024-05-08T05:47:16.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.321+0000 7f16d661d740 1 -- 172.21.15.45:0/655792887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d538932c70 msgr2=0x55d53892a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:16.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.321+0000 7f16d661d740 1 --2- 172.21.15.45:0/655792887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d538932c70 0x55d53892a410 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f16c0006fd0 tx=0x7f16c0010680 comp rx=0 tx=0).stop 2024-05-08T05:47:16.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.321+0000 7f16d661d740 1 -- 172.21.15.45:0/655792887 shutdown_connections 2024-05-08T05:47:16.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.321+0000 7f16d661d740 1 --2- 172.21.15.45:0/655792887 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55d53892cca0 0x55d53892ea90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.321+0000 7f16d661d740 1 --2- 172.21.15.45:0/655792887 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5389294f0 0x55d538952480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.321+0000 7f16d661d740 1 --2- 172.21.15.45:0/655792887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d538932c70 0x55d53892a410 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.321+0000 7f16abfff640 0 client.6999 ms_handle_reset on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:16.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.321+0000 7f16d661d740 1 --2- 172.21.15.45:0/655792887 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d538932890 0x55d538928e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.321+0000 7f16d661d740 1 -- 172.21.15.45:0/655792887 >> 172.21.15.45:0/655792887 conn(0x55d5389054f0 msgr2=0x55d538904c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:16.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.321+0000 7f16d661d740 1 -- 172.21.15.45:0/655792887 shutdown_connections 2024-05-08T05:47:16.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.321+0000 7f16d661d740 1 -- 172.21.15.45:0/655792887 wait complete. 2024-05-08T05:47:16.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.321+0000 7f16d661d740 20 client.6999 trim_cache size 0 max 16384 2024-05-08T05:47:16.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.339+0000 7f16d661d740 1 Processor -- start 2024-05-08T05:47:16.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.339+0000 7f16d661d740 1 -- start start 2024-05-08T05:47:16.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.340+0000 7f16d661d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5389294f0 0x55d53892c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.340+0000 7f16d661d740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d538932890 0x55d538905e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.340+0000 7f16d661d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d538932c70 0x55d538952650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.340+0000 7f16d661d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc2022c0 con 0x55d538932c70 2024-05-08T05:47:16.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.340+0000 7f16d661d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc2012a0 con 0x55d5389294f0 2024-05-08T05:47:16.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.340+0000 7f16d661d740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f16cc200600 con 0x55d538932890 2024-05-08T05:47:16.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.340+0000 7f16d1ea7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d538932c70 0x55d538952650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.340+0000 7f16d1ea7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d538932c70 0x55d538952650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54178/0 (socket says 172.21.15.45:54178) 2024-05-08T05:47:16.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.340+0000 7f16d1ea7640 1 -- 172.21.15.45:0/255105598 learned_addr learned my addr 172.21.15.45:0/255105598 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.340+0000 7f16d16a6640 1 --2- 172.21.15.45:0/255105598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5389294f0 0x55d53892c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.340+0000 7f16d0ea5640 1 --2- 172.21.15.45:0/255105598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d538932890 0x55d538905e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.340+0000 7f16d1ea7640 1 -- 172.21.15.45:0/255105598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d538932890 msgr2=0x55d538905e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.340+0000 7f16d1ea7640 1 --2- 172.21.15.45:0/255105598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d538932890 0x55d538905e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.340+0000 7f16d1ea7640 1 -- 172.21.15.45:0/255105598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5389294f0 msgr2=0x55d53892c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.341+0000 7f16d1ea7640 1 --2- 172.21.15.45:0/255105598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5389294f0 0x55d53892c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.341+0000 7f16d1ea7640 1 -- 172.21.15.45:0/255105598 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5385400a0 con 0x55d538932c70 2024-05-08T05:47:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.341+0000 7f16d1ea7640 1 --2- 172.21.15.45:0/255105598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d538932c70 0x55d538952650 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f16bc060f60 tx=0x7f16bc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd5481e398f3604 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.341+0000 7f16b27fc640 1 -- 172.21.15.45:0/255105598 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc07e070 con 0x55d538932c70 2024-05-08T05:47:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.341+0000 7f16b27fc640 1 -- 172.21.15.45:0/255105598 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16bc060fd0 con 0x55d538932c70 2024-05-08T05:47:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.341+0000 7f16b27fc640 1 -- 172.21.15.45:0/255105598 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc07a040 con 0x55d538932c70 2024-05-08T05:47:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.341+0000 7f16d661d740 1 -- 172.21.15.45:0/255105598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d538932c70 msgr2=0x55d538952650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.341+0000 7f16d661d740 1 --2- 172.21.15.45:0/255105598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d538932c70 0x55d538952650 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f16bc060f60 tx=0x7f16bc01f000 comp rx=0 tx=0).stop 2024-05-08T05:47:16.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.341+0000 7f16d661d740 1 -- 172.21.15.45:0/255105598 shutdown_connections 2024-05-08T05:47:16.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.341+0000 7f16d661d740 1 --2- 172.21.15.45:0/255105598 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d538932890 0x55d538905e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.341+0000 7f16d661d740 1 --2- 172.21.15.45:0/255105598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d538932c70 0x55d538952650 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.341+0000 7f16d661d740 1 --2- 172.21.15.45:0/255105598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d5389294f0 0x55d53892c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.341+0000 7f16d661d740 1 -- 172.21.15.45:0/255105598 >> 172.21.15.45:0/255105598 conn(0x55d5389054f0 msgr2=0x55d538904c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:16.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.341+0000 7f16d661d740 1 -- 172.21.15.45:0/255105598 shutdown_connections 2024-05-08T05:47:16.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.341+0000 7f16d661d740 1 -- 172.21.15.45:0/255105598 wait complete. 2024-05-08T05:47:16.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.342+0000 7f16d661d740 1 -- start start 2024-05-08T05:47:16.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.342+0000 7f16d661d740 1 Processor -- start 2024-05-08T05:47:16.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.343+0000 7f16d661d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5389294f0 0x55d53890c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.343+0000 7f16d661d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d538932890 0x55d538973e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.343+0000 7f16d661d740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d538932c70 0x55d538962a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.343+0000 7f16d661d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d538929da0 con 0x55d5389294f0 2024-05-08T05:47:16.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.343+0000 7f16d661d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc200c50 con 0x55d538932890 2024-05-08T05:47:16.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.343+0000 7f16d661d740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d538904ab0 con 0x55d538932c70 2024-05-08T05:47:16.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.343+0000 7f16d661d740 10 client.0 _ll_register_callbacks cb 0x7f16c00092f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:47:16.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.343+0000 7f16d16a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5389294f0 0x55d53890c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.343+0000 7f16d16a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5389294f0 0x55d53890c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54184/0 (socket says 172.21.15.45:54184) 2024-05-08T05:47:16.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.343+0000 7f16d16a6640 1 -- 172.21.15.45:0/3753108716 learned_addr learned my addr 172.21.15.45:0/3753108716 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:16.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.343+0000 7f16d0ea5640 1 --2- 172.21.15.45:0/3753108716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d538932890 0x55d538973e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.343+0000 7f16d1ea7640 1 --2- 172.21.15.45:0/3753108716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d538932c70 0x55d538962a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.343+0000 7f16d16a6640 1 -- 172.21.15.45:0/3753108716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d538932c70 msgr2=0x55d538962a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.343+0000 7f16d16a6640 1 --2- 172.21.15.45:0/3753108716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d538932c70 0x55d538962a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.343+0000 7f16d16a6640 1 -- 172.21.15.45:0/3753108716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d538932890 msgr2=0x55d538973e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.343+0000 7f16d16a6640 1 --2- 172.21.15.45:0/3753108716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d538932890 0x55d538973e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.343+0000 7f16d16a6640 1 -- 172.21.15.45:0/3753108716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5389743d0 con 0x55d5389294f0 2024-05-08T05:47:16.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.343+0000 7f16d16a6640 1 --2- 172.21.15.45:0/3753108716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5389294f0 0x55d53890c380 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f16c40329d0 tx=0x7f16c402c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f0cb8b51c5ab9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:16.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.343+0000 7f16abfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:47:16.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.344+0000 7f16abfff640 1 -- 172.21.15.45:0/3753108716 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4052070 con 0x55d5389294f0 2024-05-08T05:47:16.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.344+0000 7f16abfff640 1 -- 172.21.15.45:0/3753108716 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f16c403e820 con 0x55d5389294f0 2024-05-08T05:47:16.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.344+0000 7f16d661d740 1 -- 172.21.15.45:0/3753108716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16bc060fd0 con 0x55d5389294f0 2024-05-08T05:47:16.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.344+0000 7f16abfff640 1 -- 172.21.15.45:0/3753108716 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f16c405d050 con 0x55d5389294f0 2024-05-08T05:47:16.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.344+0000 7f16d661d740 10 client.7005 fetch_fsmap learned FSMap version 66 2024-05-08T05:47:16.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.344+0000 7f16d661d740 1 -- 172.21.15.45:0/3753108716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55d538927930 con 0x55d5389294f0 2024-05-08T05:47:16.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.344+0000 7f16abfff640 1 -- 172.21.15.45:0/3753108716 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16bc060fd0 con 0x55d5389294f0 2024-05-08T05:47:16.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.344+0000 7f16abfff640 1 -- 172.21.15.45:0/3753108716 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7f16c4060020 con 0x55d5389294f0 2024-05-08T05:47:16.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.344+0000 7f16d661d740 10 client.7005 fetch_fsmap finished waiting for FSMap version 66 2024-05-08T05:47:16.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.344+0000 7f16d661d740 10 client.7005 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-08T05:47:16.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.344+0000 7f16d661d740 20 client.7005 populate_metadata read hostname 'smithi045' 2024-05-08T05:47:16.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.344+0000 7f16d661d740 1 --2- 172.21.15.45:0/3753108716 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55d538904250 0x55d538962f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.344+0000 7f16d661d740 4 client.7005 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-05-08T05:47:16.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.344+0000 7f16d661d740 1 -- 172.21.15.45:0/3753108716 --> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) -- 0x55d53890af60 con 0x55d538904250 2024-05-08T05:47:16.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.344+0000 7f16d0ea5640 1 --2- 172.21.15.45:0/3753108716 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55d538904250 0x55d538962f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.345+0000 7f16abfff640 10 client.7005 ms_handle_connect on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:16.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.345+0000 7f16d0ea5640 1 --2- 172.21.15.45:0/3753108716 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55d538904250 0x55d538962f50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=184fc97c5b2b4e14 server_cookie=ef1b93923a80259a in_seq=0 out_seq=0 2024-05-08T05:47:16.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.345+0000 7f16abfff640 1 -- 172.21.15.45:0/3753108716 <== mds.0 v2:172.21.15.63:6835/3194497846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d53890af60 con 0x55d538904250 2024-05-08T05:47:16.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.345+0000 7f16abfff640 10 client.7005 handle_command_reply: tid=0 2024-05-08T05:47:16.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.345+0000 7f16d661d740 1 client.7005 shutdown 2024-05-08T05:47:16.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.345+0000 7f16d661d740 1 -- 172.21.15.45:0/3753108716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5389294f0 msgr2=0x55d53890c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:16.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.345+0000 7f16d661d740 1 --2- 172.21.15.45:0/3753108716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5389294f0 0x55d53890c380 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f16c40329d0 tx=0x7f16c402c070 comp rx=0 tx=0).stop 2024-05-08T05:47:16.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.345+0000 7f16d661d740 1 -- 172.21.15.45:0/3753108716 shutdown_connections 2024-05-08T05:47:16.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.345+0000 7f16d661d740 1 --2- 172.21.15.45:0/3753108716 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55d538904250 0x55d538962f50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.345+0000 7f16d661d740 1 --2- 172.21.15.45:0/3753108716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d538932c70 0x55d538962a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.345+0000 7f16d661d740 1 --2- 172.21.15.45:0/3753108716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d538932890 0x55d538973e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.345+0000 7f16abfff640 0 client.7005 ms_handle_reset on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:16.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.345+0000 7f16d661d740 1 --2- 172.21.15.45:0/3753108716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d5389294f0 0x55d53890c380 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.345+0000 7f16d661d740 1 -- 172.21.15.45:0/3753108716 >> 172.21.15.45:0/3753108716 conn(0x55d5389054f0 msgr2=0x55d538905c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:16.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.345+0000 7f16d661d740 1 -- 172.21.15.45:0/3753108716 shutdown_connections 2024-05-08T05:47:16.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.345+0000 7f16d661d740 1 -- 172.21.15.45:0/3753108716 wait complete. 2024-05-08T05:47:16.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.346+0000 7f16d661d740 20 client.7005 trim_cache size 0 max 16384 2024-05-08T05:47:16.353 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:47:16.353 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-05-08T05:47:16.353 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "c354641c-b3b9-45f0-a5e8-f4ef5e484d4c", 2024-05-08T05:47:16.353 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-05-08T05:47:16.353 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:47:16.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.346+0000 7f16d390a640 1 -- 172.21.15.45:0/1691190412 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f16b80762d0 msgr2=0x7f16b8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:16.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.346+0000 7f16d390a640 1 --2- 172.21.15.45:0/1691190412 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f16b80762d0 0x7f16b8078790 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f16c4034000 tx=0x7f16c402e000 comp rx=0 tx=0).stop 2024-05-08T05:47:16.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.346+0000 7f16d390a640 1 -- 172.21.15.45:0/1691190412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16cc0d3920 msgr2=0x7f16cc0cb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:16.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.346+0000 7f16d390a640 1 --2- 172.21.15.45:0/1691190412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16cc0d3920 0x7f16cc0cb460 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f16bc007e60 tx=0x7f16bc00e4b0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.346+0000 7f16d390a640 1 -- 172.21.15.45:0/1691190412 shutdown_connections 2024-05-08T05:47:16.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.346+0000 7f16d390a640 1 --2- 172.21.15.45:0/1691190412 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f16b80762d0 0x7f16b8078790 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.346+0000 7f16d390a640 1 --2- 172.21.15.45:0/1691190412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16cc0d3920 0x7f16cc0cb460 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.346+0000 7f16d390a640 1 --2- 172.21.15.45:0/1691190412 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16cc0cf8f0 0x7f16cc1522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.346+0000 7f16d390a640 1 --2- 172.21.15.45:0/1691190412 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16cc0cef40 0x7f16cc151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.346+0000 7f16d390a640 1 -- 172.21.15.45:0/1691190412 >> 172.21.15.45:0/1691190412 conn(0x7f16cc0d2e00 msgr2=0x7f16cc0702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:16.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.346+0000 7f16d390a640 1 -- 172.21.15.45:0/1691190412 shutdown_connections 2024-05-08T05:47:16.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.346+0000 7f16d390a640 1 -- 172.21.15.45:0/1691190412 wait complete. 2024-05-08T05:47:16.355 INFO:tasks.ceph.mds.b.smithi063.stderr:2024-05-08T05:47:16.345+0000 7ff8b9265640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-05-08T05:47:07.822465+0000 1=1+0) n(v0 rc2024-05-08T05:47:07.942464+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x561e6ad95b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-08T05:47:07.822465+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-08T05:47:07.942464+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-05-08T05:47:16.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:16.359 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-08T05:47:16.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.464+0000 7f496e3e4640 1 Processor -- start 2024-05-08T05:47:16.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.464+0000 7f496e3e4640 1 -- start start 2024-05-08T05:47:16.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.465+0000 7f496e3e4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f49680b2490 0x7f4968070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.465+0000 7f496e3e4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f496806e110 con 0x7f4968070d20 2024-05-08T05:47:16.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.465+0000 7f496e3e4640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f496806e3f0 con 0x7f49681629b0 2024-05-08T05:47:16.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.465+0000 7f496e3e4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f496806dec0 con 0x7f49680b2490 2024-05-08T05:47:16.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.465+0000 7f496c981640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4968070d20 0x7f49680afcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41282/0 (socket says 172.21.15.45:41282) 2024-05-08T05:47:16.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.465+0000 7f496c981640 1 -- 172.21.15.45:0/2519911258 learned_addr learned my addr 172.21.15.45:0/2519911258 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:47:16.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.465+0000 7f49677fe640 1 --2- 172.21.15.45:0/2519911258 >> v2:172.21.15.63:3300/0 conn(0x7f49680b2490 0x7f4968070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.465+0000 7f4966ffd640 1 -- 172.21.15.45:0/2519911258 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2865812950 0 0) 0x7f496806e110 con 0x7f4968070d20 2024-05-08T05:47:16.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f4966ffd640 1 -- 172.21.15.45:0/2519911258 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4950003520 con 0x7f4968070d20 2024-05-08T05:47:16.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f4966ffd640 1 -- 172.21.15.45:0/2519911258 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3679234538 0 0) 0x7f4950003520 con 0x7f4968070d20 2024-05-08T05:47:16.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f4966ffd640 1 -- 172.21.15.45:0/2519911258 >> v2:172.21.15.63:3300/0 conn(0x7f49680b2490 msgr2=0x7f4968070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f4966ffd640 1 --2- 172.21.15.45:0/2519911258 >> v2:172.21.15.63:3300/0 conn(0x7f49680b2490 0x7f4968070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f4966ffd640 1 -- 172.21.15.45:0/2519911258 >> v1:172.21.15.63:6789/0 conn(0x7f49681629b0 legacy=0x7f496806eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:16.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f4966ffd640 1 -- 172.21.15.45:0/2519911258 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496806e6d0 con 0x7f4968070d20 2024-05-08T05:47:16.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f4966ffd640 1 -- 172.21.15.45:0/2519911258 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f4960001a70 con 0x7f4968070d20 2024-05-08T05:47:16.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f4966ffd640 1 -- 172.21.15.45:0/2519911258 >> v1:172.21.15.45:6789/0 conn(0x7f4968070d20 legacy=0x7f49680afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:16.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f4966ffd640 1 --2- 172.21.15.45:0/2519911258 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4950003a50 0x7f4950003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f4966ffd640 1 --2- 172.21.15.45:0/2519911258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4950004800 0x7f4950004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f4966ffd640 1 --2- 172.21.15.45:0/2519911258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49500051c0 0x7f495000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f4966ffd640 1 -- 172.21.15.45:0/2519911258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f496806dec0 con 0x7f49500051c0 2024-05-08T05:47:16.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f4966ffd640 1 -- 172.21.15.45:0/2519911258 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f495000c160 con 0x7f4950004800 2024-05-08T05:47:16.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f4966ffd640 1 -- 172.21.15.45:0/2519911258 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f495000c360 con 0x7f4950003a50 2024-05-08T05:47:16.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f496c981640 1 --2- 172.21.15.45:0/2519911258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49500051c0 0x7f495000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f4967fff640 1 --2- 172.21.15.45:0/2519911258 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4950003a50 0x7f4950003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f49677fe640 1 --2- 172.21.15.45:0/2519911258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4950004800 0x7f4950004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f496c981640 1 -- 172.21.15.45:0/2519911258 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4950003a50 msgr2=0x7f4950003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f496c981640 1 --2- 172.21.15.45:0/2519911258 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4950003a50 0x7f4950003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f496c981640 1 -- 172.21.15.45:0/2519911258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4950004800 msgr2=0x7f4950004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f496c981640 1 --2- 172.21.15.45:0/2519911258 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4950004800 0x7f4950004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.466+0000 7f496c981640 1 -- 172.21.15.45:0/2519911258 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4950003520 con 0x7f49500051c0 2024-05-08T05:47:16.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.467+0000 7f496c981640 1 -- 172.21.15.45:0/2519911258 reap_dead start 2024-05-08T05:47:16.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.467+0000 7f496c981640 1 --2- 172.21.15.45:0/2519911258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49500051c0 0x7f495000b9c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f496000a830 tx=0x7f4950005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a07b75e99110b3a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:16.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.467+0000 7f4966ffd640 1 -- 172.21.15.45:0/2519911258 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496000a480 con 0x7f49500051c0 2024-05-08T05:47:16.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.467+0000 7f4966ffd640 1 -- 172.21.15.45:0/2519911258 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f496002f070 con 0x7f49500051c0 2024-05-08T05:47:16.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.467+0000 7f4966ffd640 1 -- 172.21.15.45:0/2519911258 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496002bc40 con 0x7f49500051c0 2024-05-08T05:47:16.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.468+0000 7f496e3e4640 1 -- 172.21.15.45:0/2519911258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49500051c0 msgr2=0x7f495000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:16.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.468+0000 7f496e3e4640 1 --2- 172.21.15.45:0/2519911258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49500051c0 0x7f495000b9c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f496000a830 tx=0x7f4950005830 comp rx=0 tx=0).stop 2024-05-08T05:47:16.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.468+0000 7f496e3e4640 1 -- 172.21.15.45:0/2519911258 shutdown_connections 2024-05-08T05:47:16.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.468+0000 7f496e3e4640 1 --2- 172.21.15.45:0/2519911258 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49500051c0 0x7f495000b9c0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.468+0000 7f496e3e4640 1 -- 172.21.15.45:0/2519911258 >> 172.21.15.45:0/2519911258 conn(0x7f49680b49d0 msgr2=0x7f49680b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:16.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.468+0000 7f496e3e4640 1 -- 172.21.15.45:0/2519911258 shutdown_connections 2024-05-08T05:47:16.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.468+0000 7f496e3e4640 1 -- 172.21.15.45:0/2519911258 wait complete. 2024-05-08T05:47:16.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.469+0000 7f496e3e4640 1 Processor -- start 2024-05-08T05:47:16.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.469+0000 7f496e3e4640 1 -- start start 2024-05-08T05:47:16.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.470+0000 7f496e3e4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49500051c0 0x7f49681f5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.470+0000 7f496e3e4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49681f2100 0x7f49680b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.470+0000 7f496e3e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49681f2580 0x7f49680b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.470+0000 7f496e3e4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4968071350 con 0x7f49681f2580 2024-05-08T05:47:16.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.470+0000 7f496e3e4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f49680711d0 con 0x7f49681f2100 2024-05-08T05:47:16.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.470+0000 7f496e3e4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f49680714d0 con 0x7f49500051c0 2024-05-08T05:47:16.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.470+0000 7f496c981640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49681f2580 0x7f49680b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.470+0000 7f496c981640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49681f2580 0x7f49680b7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54202/0 (socket says 172.21.15.45:54202) 2024-05-08T05:47:16.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.470+0000 7f496c981640 1 -- 172.21.15.45:0/961918660 learned_addr learned my addr 172.21.15.45:0/961918660 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:16.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.470+0000 7f4967fff640 1 --2- 172.21.15.45:0/961918660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49500051c0 0x7f49681f5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.470+0000 7f49677fe640 1 --2- 172.21.15.45:0/961918660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49681f2100 0x7f49680b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.470+0000 7f496c981640 1 -- 172.21.15.45:0/961918660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49500051c0 msgr2=0x7f49681f5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:16.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.470+0000 7f496c981640 1 --2- 172.21.15.45:0/961918660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49500051c0 0x7f49681f5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.470+0000 7f496c981640 1 -- 172.21.15.45:0/961918660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49681f2100 msgr2=0x7f49680b7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:16.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.470+0000 7f496c981640 1 --2- 172.21.15.45:0/961918660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49681f2100 0x7f49680b7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.470+0000 7f496c981640 1 -- 172.21.15.45:0/961918660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49680c6a60 con 0x7f49681f2580 2024-05-08T05:47:16.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.470+0000 7f4967fff640 1 --2- 172.21.15.45:0/961918660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49500051c0 0x7f49681f5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:47:16.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.470+0000 7f49677fe640 1 --2- 172.21.15.45:0/961918660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49681f2100 0x7f49680b7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:47:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.471+0000 7f496c981640 1 --2- 172.21.15.45:0/961918660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49681f2580 0x7f49680b7790 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f496000a950 tx=0x7f4960001d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.471+0000 7f4964ff9640 1 -- 172.21.15.45:0/961918660 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496000a480 con 0x7f49681f2580 2024-05-08T05:47:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.471+0000 7f4964ff9640 1 -- 172.21.15.45:0/961918660 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4960004b60 con 0x7f49681f2580 2024-05-08T05:47:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.471+0000 7f496e3e4640 1 -- 172.21.15.45:0/961918660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49680c6d40 con 0x7f49681f2580 2024-05-08T05:47:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.471+0000 7f4964ff9640 1 -- 172.21.15.45:0/961918660 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960001ff0 con 0x7f49681f2580 2024-05-08T05:47:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.471+0000 7f496e3e4640 1 -- 172.21.15.45:0/961918660 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49680b5dd0 con 0x7f49681f2580 2024-05-08T05:47:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.472+0000 7f4964ff9640 1 -- 172.21.15.45:0/961918660 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4960034470 con 0x7f49681f2580 2024-05-08T05:47:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.473+0000 7f4964ff9640 1 --2- 172.21.15.45:0/961918660 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f49540762d0 0x7f4954078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.473+0000 7f4967fff640 1 --2- 172.21.15.45:0/961918660 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f49540762d0 0x7f4954078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.473+0000 7f4964ff9640 1 -- 172.21.15.45:0/961918660 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f49600b7920 con 0x7f49681f2580 2024-05-08T05:47:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.474+0000 7f4967fff640 1 --2- 172.21.15.45:0/961918660 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f49540762d0 0x7f4954078790 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4958005600 tx=0x7f4958009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.474+0000 7f49711d9740 1 Processor -- start 2024-05-08T05:47:16.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.474+0000 7f49711d9740 1 -- start start 2024-05-08T05:47:16.477 INFO:tasks.ceph.mds.b.smithi063.stderr:2024-05-08T05:47:16.475+0000 7ff8bd26d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:47:16.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.475+0000 7f49711d9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df65678160 0x55df6562aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.475+0000 7f49711d9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df6565b280 0x55df6565a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.475+0000 7f49711d9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55df6565ab40 0x55df6562f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.475+0000 7f49711d9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55df655f7690 con 0x55df65678160 2024-05-08T05:47:16.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.475+0000 7f49711d9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55df65594f30 con 0x55df6565b280 2024-05-08T05:47:16.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.475+0000 7f49711d9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55df6554db20 con 0x55df6565ab40 2024-05-08T05:47:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.475+0000 7f4967fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df65678160 0x55df6562aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.475+0000 7f4967fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df65678160 0x55df6562aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54216/0 (socket says 172.21.15.45:54216) 2024-05-08T05:47:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.475+0000 7f4967fff640 1 -- 172.21.15.45:0/662032624 learned_addr learned my addr 172.21.15.45:0/662032624 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.475+0000 7f496c981640 1 --2- 172.21.15.45:0/662032624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55df6565ab40 0x55df6562f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.475+0000 7f49677fe640 1 --2- 172.21.15.45:0/662032624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df6565b280 0x55df6565a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.475+0000 7f4967fff640 1 -- 172.21.15.45:0/662032624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55df6565ab40 msgr2=0x55df6562f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.475+0000 7f4967fff640 1 --2- 172.21.15.45:0/662032624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55df6565ab40 0x55df6562f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.475+0000 7f4967fff640 1 -- 172.21.15.45:0/662032624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df6565b280 msgr2=0x55df6565a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.475+0000 7f4967fff640 1 --2- 172.21.15.45:0/662032624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df6565b280 0x55df6565a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.475+0000 7f4967fff640 1 -- 172.21.15.45:0/662032624 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df65545e60 con 0x55df65678160 2024-05-08T05:47:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.475+0000 7f49677fe640 1 --2- 172.21.15.45:0/662032624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df6565b280 0x55df6565a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.475+0000 7f496c981640 1 --2- 172.21.15.45:0/662032624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55df6565ab40 0x55df6562f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.476+0000 7f4967fff640 1 --2- 172.21.15.45:0/662032624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df65678160 0x55df6562aaf0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f49580016a0 tx=0x7f4958013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7580d53ee4995b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:16.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.476+0000 7f494dffb640 1 -- 172.21.15.45:0/662032624 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958027070 con 0x55df65678160 2024-05-08T05:47:16.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.476+0000 7f494dffb640 1 -- 172.21.15.45:0/662032624 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4958014840 con 0x55df65678160 2024-05-08T05:47:16.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.476+0000 7f494dffb640 1 -- 172.21.15.45:0/662032624 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958014bf0 con 0x55df65678160 2024-05-08T05:47:16.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.476+0000 7f49711d9740 1 -- 172.21.15.45:0/662032624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df65678160 msgr2=0x55df6562aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:16.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.476+0000 7f49711d9740 1 --2- 172.21.15.45:0/662032624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df65678160 0x55df6562aaf0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f49580016a0 tx=0x7f4958013000 comp rx=0 tx=0).stop 2024-05-08T05:47:16.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.476+0000 7f49711d9740 1 -- 172.21.15.45:0/662032624 shutdown_connections 2024-05-08T05:47:16.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.476+0000 7f49711d9740 1 --2- 172.21.15.45:0/662032624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55df6565ab40 0x55df6562f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.476+0000 7f49711d9740 1 --2- 172.21.15.45:0/662032624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df6565b280 0x55df6565a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.476+0000 7f49711d9740 1 --2- 172.21.15.45:0/662032624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df65678160 0x55df6562aaf0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.476+0000 7f49711d9740 1 -- 172.21.15.45:0/662032624 >> 172.21.15.45:0/662032624 conn(0x55df6562cd60 msgr2=0x55df65675600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:16.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.476+0000 7f49711d9740 1 -- 172.21.15.45:0/662032624 shutdown_connections 2024-05-08T05:47:16.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.476+0000 7f49711d9740 1 -- 172.21.15.45:0/662032624 wait complete. 2024-05-08T05:47:16.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.477+0000 7f49711d9740 1 -- start start 2024-05-08T05:47:16.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.477+0000 7f49711d9740 1 Processor -- start 2024-05-08T05:47:16.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.478+0000 7f49711d9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565b280 0x55df6565de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.478+0000 7f49711d9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df65678160 0x55df65686b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.478+0000 7f49711d9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55df65688f80 0x55df65655f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.478+0000 7f49711d9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55df655515e0 con 0x55df6565b280 2024-05-08T05:47:16.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.478+0000 7f49711d9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55df65590ef0 con 0x55df65678160 2024-05-08T05:47:16.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.478+0000 7f49711d9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55df6562ef10 con 0x55df65688f80 2024-05-08T05:47:16.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.478+0000 7f49711d9740 10 client.0 _ll_register_callbacks cb 0x55df6554c320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:47:16.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.478+0000 7f4967fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565b280 0x55df6565de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.478+0000 7f4967fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565b280 0x55df6565de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54226/0 (socket says 172.21.15.45:54226) 2024-05-08T05:47:16.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.478+0000 7f4967fff640 1 -- 172.21.15.45:0/1798596991 learned_addr learned my addr 172.21.15.45:0/1798596991 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:16.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.478+0000 7f49677fe640 1 --2- 172.21.15.45:0/1798596991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df65678160 0x55df65686b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.478+0000 7f496c981640 1 --2- 172.21.15.45:0/1798596991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55df65688f80 0x55df65655f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.478+0000 7f4967fff640 1 -- 172.21.15.45:0/1798596991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55df65688f80 msgr2=0x55df65655f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.478+0000 7f4967fff640 1 --2- 172.21.15.45:0/1798596991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55df65688f80 0x55df65655f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.478+0000 7f4967fff640 1 -- 172.21.15.45:0/1798596991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df65678160 msgr2=0x55df65686b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.478+0000 7f4967fff640 1 --2- 172.21.15.45:0/1798596991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df65678160 0x55df65686b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.478+0000 7f4967fff640 1 -- 172.21.15.45:0/1798596991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df653707d0 con 0x55df6565b280 2024-05-08T05:47:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.479+0000 7f4967fff640 1 --2- 172.21.15.45:0/1798596991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565b280 0x55df6565de70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4958003fd0 tx=0x7f4958001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f202d71d6c2d8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.479+0000 7f4946ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:47:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.479+0000 7f4946ffd640 1 -- 172.21.15.45:0/1798596991 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958027070 con 0x55df6565b280 2024-05-08T05:47:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.479+0000 7f49711d9740 1 -- 172.21.15.45:0/1798596991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4958014840 con 0x55df6565b280 2024-05-08T05:47:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.479+0000 7f4946ffd640 1 -- 172.21.15.45:0/1798596991 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f4958007220 con 0x55df6565b280 2024-05-08T05:47:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.479+0000 7f4946ffd640 1 -- 172.21.15.45:0/1798596991 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f4958037050 con 0x55df6565b280 2024-05-08T05:47:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.479+0000 7f49711d9740 10 client.7017 fetch_fsmap learned FSMap version 66 2024-05-08T05:47:16.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.479+0000 7f49711d9740 1 -- 172.21.15.45:0/1798596991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55df652a63f0 con 0x55df6565b280 2024-05-08T05:47:16.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.479+0000 7f4946ffd640 1 -- 172.21.15.45:0/1798596991 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4958014840 con 0x55df6565b280 2024-05-08T05:47:16.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.479+0000 7f4946ffd640 1 -- 172.21.15.45:0/1798596991 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7f495803a020 con 0x55df6565b280 2024-05-08T05:47:16.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.479+0000 7f49711d9740 10 client.7017 fetch_fsmap finished waiting for FSMap version 66 2024-05-08T05:47:16.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.479+0000 7f49711d9740 10 client.7017 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-08T05:47:16.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.479+0000 7f49711d9740 20 client.7017 populate_metadata read hostname 'smithi045' 2024-05-08T05:47:16.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.479+0000 7f49711d9740 1 --2- 172.21.15.45:0/1798596991 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55df65686520 0x55df65676ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.479+0000 7f49711d9740 4 client.7017 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:47:16.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.479+0000 7f49711d9740 1 -- 172.21.15.45:0/1798596991 --> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df655a2b40 con 0x55df65686520 2024-05-08T05:47:16.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.480+0000 7f49677fe640 1 --2- 172.21.15.45:0/1798596991 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55df65686520 0x55df65676ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.480+0000 7f49677fe640 1 --2- 172.21.15.45:0/1798596991 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55df65686520 0x55df65676ed0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef14fe2fda27d140 server_cookie=7b82306a526a4538 in_seq=0 out_seq=0 2024-05-08T05:47:16.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.480+0000 7f4946ffd640 10 client.7017 ms_handle_connect on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:16.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.480+0000 7f4946ffd640 1 -- 172.21.15.45:0/1798596991 <== mds.0 v2:172.21.15.63:6835/3194497846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55df655a2b40 con 0x55df65686520 2024-05-08T05:47:16.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.481+0000 7f4946ffd640 10 client.7017 handle_command_reply: tid=0 2024-05-08T05:47:16.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.481+0000 7f49711d9740 1 client.7017 shutdown 2024-05-08T05:47:16.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.481+0000 7f49711d9740 1 -- 172.21.15.45:0/1798596991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565b280 msgr2=0x55df6565de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:16.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.481+0000 7f49711d9740 1 --2- 172.21.15.45:0/1798596991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565b280 0x55df6565de70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4958003fd0 tx=0x7f4958001120 comp rx=0 tx=0).stop 2024-05-08T05:47:16.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.481+0000 7f49711d9740 1 -- 172.21.15.45:0/1798596991 shutdown_connections 2024-05-08T05:47:16.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.481+0000 7f49711d9740 1 --2- 172.21.15.45:0/1798596991 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55df65686520 0x55df65676ed0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.481+0000 7f49711d9740 1 --2- 172.21.15.45:0/1798596991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55df65688f80 0x55df65655f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.481+0000 7f49711d9740 1 --2- 172.21.15.45:0/1798596991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df65678160 0x55df65686b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.481+0000 7f4946ffd640 0 client.7017 ms_handle_reset on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:16.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.481+0000 7f49711d9740 1 --2- 172.21.15.45:0/1798596991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565b280 0x55df6565de70 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.481+0000 7f49711d9740 1 -- 172.21.15.45:0/1798596991 >> 172.21.15.45:0/1798596991 conn(0x55df6565ab40 msgr2=0x55df6562e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:16.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.481+0000 7f49711d9740 1 -- 172.21.15.45:0/1798596991 shutdown_connections 2024-05-08T05:47:16.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.481+0000 7f49711d9740 1 -- 172.21.15.45:0/1798596991 wait complete. 2024-05-08T05:47:16.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.481+0000 7f49711d9740 20 client.7017 trim_cache size 0 max 16384 2024-05-08T05:47:16.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.499+0000 7f49711d9740 1 Processor -- start 2024-05-08T05:47:16.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.499+0000 7f49711d9740 1 -- start start 2024-05-08T05:47:16.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.500+0000 7f49711d9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565ab40 0x55df6565d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.500+0000 7f49711d9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df6565b280 0x55df656544e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.500+0000 7f49711d9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55df65678160 0x55df65654a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.500+0000 7f49711d9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f49682028f0 con 0x55df6565ab40 2024-05-08T05:47:16.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.500+0000 7f49711d9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f49682018d0 con 0x55df6565b280 2024-05-08T05:47:16.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.500+0000 7f49711d9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f49680c98c0 con 0x55df65678160 2024-05-08T05:47:16.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.500+0000 7f4967fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565ab40 0x55df6565d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.500+0000 7f4967fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565ab40 0x55df6565d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54236/0 (socket says 172.21.15.45:54236) 2024-05-08T05:47:16.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.500+0000 7f4967fff640 1 -- 172.21.15.45:0/130358897 learned_addr learned my addr 172.21.15.45:0/130358897 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:16.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.500+0000 7f4967fff640 1 -- 172.21.15.45:0/130358897 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55df65678160 msgr2=0x55df65654a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.500+0000 7f4967fff640 1 --2- 172.21.15.45:0/130358897 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55df65678160 0x55df65654a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.500+0000 7f4967fff640 1 -- 172.21.15.45:0/130358897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df6565b280 msgr2=0x55df656544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.500+0000 7f4967fff640 1 --2- 172.21.15.45:0/130358897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df6565b280 0x55df656544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.500+0000 7f4967fff640 1 -- 172.21.15.45:0/130358897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df652660f0 con 0x55df6565ab40 2024-05-08T05:47:16.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.500+0000 7f4967fff640 1 --2- 172.21.15.45:0/130358897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565ab40 0x55df6565d970 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f4958005340 tx=0x7f495802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1e8762541bf11d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:16.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.500+0000 7f494d7fa640 1 -- 172.21.15.45:0/130358897 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958014840 con 0x55df6565ab40 2024-05-08T05:47:16.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.500+0000 7f494d7fa640 1 -- 172.21.15.45:0/130358897 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4958028040 con 0x55df6565ab40 2024-05-08T05:47:16.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.500+0000 7f494d7fa640 1 -- 172.21.15.45:0/130358897 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49580218f0 con 0x55df6565ab40 2024-05-08T05:47:16.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.501+0000 7f49711d9740 1 -- 172.21.15.45:0/130358897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565ab40 msgr2=0x55df6565d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:16.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.501+0000 7f49711d9740 1 --2- 172.21.15.45:0/130358897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565ab40 0x55df6565d970 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f4958005340 tx=0x7f495802c000 comp rx=0 tx=0).stop 2024-05-08T05:47:16.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.501+0000 7f49711d9740 1 -- 172.21.15.45:0/130358897 shutdown_connections 2024-05-08T05:47:16.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.501+0000 7f49711d9740 1 --2- 172.21.15.45:0/130358897 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55df65678160 0x55df65654a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.501+0000 7f49711d9740 1 --2- 172.21.15.45:0/130358897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df6565b280 0x55df656544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.501+0000 7f49711d9740 1 --2- 172.21.15.45:0/130358897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565ab40 0x55df6565d970 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.501+0000 7f49711d9740 1 -- 172.21.15.45:0/130358897 >> 172.21.15.45:0/130358897 conn(0x55df65686d20 msgr2=0x55df65655f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:16.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.501+0000 7f49711d9740 1 -- 172.21.15.45:0/130358897 shutdown_connections 2024-05-08T05:47:16.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.501+0000 7f49711d9740 1 -- 172.21.15.45:0/130358897 wait complete. 2024-05-08T05:47:16.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.501+0000 7f49711d9740 1 -- start start 2024-05-08T05:47:16.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.502+0000 7f49711d9740 1 Processor -- start 2024-05-08T05:47:16.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.502+0000 7f49711d9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565ab40 0x55df65654ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.502+0000 7f49711d9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df6565b280 0x55df656d7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.502+0000 7f49711d9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55df65678160 0x55df656d89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.502+0000 7f49711d9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4968201280 con 0x55df6565ab40 2024-05-08T05:47:16.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.502+0000 7f49711d9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55df656577b0 con 0x55df6565b280 2024-05-08T05:47:16.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.502+0000 7f49711d9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55df65698960 con 0x55df65678160 2024-05-08T05:47:16.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.502+0000 7f49711d9740 10 client.0 _ll_register_callbacks cb 0x7f4958025070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:47:16.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.503+0000 7f4967fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565ab40 0x55df65654ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.503+0000 7f4967fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565ab40 0x55df65654ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54246/0 (socket says 172.21.15.45:54246) 2024-05-08T05:47:16.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.503+0000 7f4967fff640 1 -- 172.21.15.45:0/1793239516 learned_addr learned my addr 172.21.15.45:0/1793239516 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:16.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.503+0000 7f49677fe640 1 --2- 172.21.15.45:0/1793239516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df6565b280 0x55df656d7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.503+0000 7f496c981640 1 --2- 172.21.15.45:0/1793239516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55df65678160 0x55df656d89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.503+0000 7f4967fff640 1 -- 172.21.15.45:0/1793239516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55df65678160 msgr2=0x55df656d89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.503+0000 7f4967fff640 1 --2- 172.21.15.45:0/1793239516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55df65678160 0x55df656d89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.503+0000 7f4967fff640 1 -- 172.21.15.45:0/1793239516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df6565b280 msgr2=0x55df656d7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:16.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.503+0000 7f4967fff640 1 --2- 172.21.15.45:0/1793239516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df6565b280 0x55df656d7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.503+0000 7f4967fff640 1 -- 172.21.15.45:0/1793239516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df65698bb0 con 0x55df6565ab40 2024-05-08T05:47:16.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.503+0000 7f4967fff640 1 --2- 172.21.15.45:0/1793239516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565ab40 0x55df65654ef0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f495802f040 tx=0x7f495800c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61c1369220fb30d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:16.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.503+0000 7f4946ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:47:16.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.503+0000 7f4946ffd640 1 -- 172.21.15.45:0/1793239516 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958014850 con 0x55df6565ab40 2024-05-08T05:47:16.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.503+0000 7f4946ffd640 1 -- 172.21.15.45:0/1793239516 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f49580218b0 con 0x55df6565ab40 2024-05-08T05:47:16.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.503+0000 7f49711d9740 1 -- 172.21.15.45:0/1793239516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4958028040 con 0x55df6565ab40 2024-05-08T05:47:16.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.503+0000 7f4946ffd640 1 -- 172.21.15.45:0/1793239516 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f4958051050 con 0x55df6565ab40 2024-05-08T05:47:16.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.503+0000 7f49711d9740 10 client.7023 fetch_fsmap learned FSMap version 66 2024-05-08T05:47:16.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.503+0000 7f49711d9740 1 -- 172.21.15.45:0/1793239516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55df65698bb0 con 0x55df6565ab40 2024-05-08T05:47:16.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.504+0000 7f4946ffd640 1 -- 172.21.15.45:0/1793239516 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4958028040 con 0x55df6565ab40 2024-05-08T05:47:16.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.504+0000 7f4946ffd640 1 -- 172.21.15.45:0/1793239516 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 66) ==== 3494+0+0 (secure 0 0 0) 0x7f4958054020 con 0x55df6565ab40 2024-05-08T05:47:16.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.504+0000 7f49711d9740 10 client.7023 fetch_fsmap finished waiting for FSMap version 66 2024-05-08T05:47:16.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.504+0000 7f49711d9740 10 client.7023 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-08T05:47:16.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.504+0000 7f49711d9740 20 client.7023 populate_metadata read hostname 'smithi045' 2024-05-08T05:47:16.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.504+0000 7f49711d9740 1 --2- 172.21.15.45:0/1793239516 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55df65688f80 0x55df65656ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:16.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.504+0000 7f49711d9740 4 client.7023 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:47:16.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.504+0000 7f49711d9740 1 -- 172.21.15.45:0/1793239516 --> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df6559c9b0 con 0x55df65688f80 2024-05-08T05:47:16.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.504+0000 7f49677fe640 1 --2- 172.21.15.45:0/1793239516 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55df65688f80 0x55df65656ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:16.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.504+0000 7f49677fe640 1 --2- 172.21.15.45:0/1793239516 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55df65688f80 0x55df65656ba0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c7a8e08121bd73c server_cookie=8522279c7626dcd8 in_seq=0 out_seq=0 2024-05-08T05:47:16.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.504+0000 7f4946ffd640 10 client.7023 ms_handle_connect on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:16.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.504+0000 7f4946ffd640 1 -- 172.21.15.45:0/1793239516 <== mds.0 v2:172.21.15.63:6835/3194497846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+256 (crc 0 0 0) 0x55df6559c9b0 con 0x55df65688f80 2024-05-08T05:47:16.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.504+0000 7f4946ffd640 10 client.7023 handle_command_reply: tid=0 2024-05-08T05:47:16.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.504+0000 7f49711d9740 1 client.7023 shutdown 2024-05-08T05:47:16.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f49711d9740 1 -- 172.21.15.45:0/1793239516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565ab40 msgr2=0x55df65654ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:16.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f49711d9740 1 --2- 172.21.15.45:0/1793239516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565ab40 0x55df65654ef0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f495802f040 tx=0x7f495800c000 comp rx=0 tx=0).stop 2024-05-08T05:47:16.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f49711d9740 1 -- 172.21.15.45:0/1793239516 shutdown_connections 2024-05-08T05:47:16.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f49711d9740 1 --2- 172.21.15.45:0/1793239516 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55df65688f80 0x55df65656ba0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f49711d9740 1 --2- 172.21.15.45:0/1793239516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55df65678160 0x55df656d89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f49711d9740 1 --2- 172.21.15.45:0/1793239516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55df6565b280 0x55df656d7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f4946ffd640 0 client.7023 ms_handle_reset on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:16.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f49711d9740 1 --2- 172.21.15.45:0/1793239516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df6565ab40 0x55df65654ef0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f49711d9740 1 -- 172.21.15.45:0/1793239516 >> 172.21.15.45:0/1793239516 conn(0x55df65686d20 msgr2=0x55df656754b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:16.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f49711d9740 1 -- 172.21.15.45:0/1793239516 shutdown_connections 2024-05-08T05:47:16.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f49711d9740 1 -- 172.21.15.45:0/1793239516 wait complete. 2024-05-08T05:47:16.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f49711d9740 20 client.7023 trim_cache size 0 max 16384 2024-05-08T05:47:16.511 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:47:16.511 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T05:47:16.511 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-05-08T05:47:16.511 INFO:teuthology.orchestra.run.smithi045.stdout: "c354641c-b3b9-45f0-a5e8-f4ef5e484d4c": { 2024-05-08T05:47:16.512 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/dir", 2024-05-08T05:47:16.512 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "c354641c-b3b9-45f0-a5e8-f4ef5e484d4c", 2024-05-08T05:47:16.512 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive" 2024-05-08T05:47:16.512 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:47:16.512 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:47:16.512 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:47:16.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f496e3e4640 1 -- 172.21.15.45:0/961918660 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f49540762d0 msgr2=0x7f4954078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:16.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f496e3e4640 1 --2- 172.21.15.45:0/961918660 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f49540762d0 0x7f4954078790 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4958005600 tx=0x7f4958009040 comp rx=0 tx=0).stop 2024-05-08T05:47:16.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f496e3e4640 1 -- 172.21.15.45:0/961918660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49681f2580 msgr2=0x7f49680b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:16.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f496e3e4640 1 --2- 172.21.15.45:0/961918660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49681f2580 0x7f49680b7790 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f496000a950 tx=0x7f4960001d70 comp rx=0 tx=0).stop 2024-05-08T05:47:16.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f496e3e4640 1 -- 172.21.15.45:0/961918660 shutdown_connections 2024-05-08T05:47:16.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f496e3e4640 1 --2- 172.21.15.45:0/961918660 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f49540762d0 0x7f4954078790 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f496e3e4640 1 --2- 172.21.15.45:0/961918660 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49681f2580 0x7f49680b7790 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f496e3e4640 1 --2- 172.21.15.45:0/961918660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49681f2100 0x7f49680b7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f496e3e4640 1 --2- 172.21.15.45:0/961918660 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49500051c0 0x7f49681f5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:16.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f496e3e4640 1 -- 172.21.15.45:0/961918660 >> 172.21.15.45:0/961918660 conn(0x7f49680b49d0 msgr2=0x7f496815fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:16.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.505+0000 7f496e3e4640 1 -- 172.21.15.45:0/961918660 shutdown_connections 2024-05-08T05:47:16.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:16.506+0000 7f496e3e4640 1 -- 172.21.15.45:0/961918660 wait complete. 2024-05-08T05:47:16.516 INFO:tasks.cephfs.filesystem:scrub status for tag:c354641c-b3b9-45f0-a5e8-f4ef5e484d4c - {'path': '/dir', 'tag': 'c354641c-b3b9-45f0-a5e8-f4ef5e484d4c', 'options': 'recursive'} 2024-05-08T05:47:20.971 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-05-08T05:47:20.971+0000 7f1250617640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-05-08T05:47:34.799 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:47:34.802 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:47:46.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:46.517 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-08T05:47:46.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.635+0000 7f0d49459640 1 Processor -- start 2024-05-08T05:47:46.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.635+0000 7f0d49459640 1 -- start start 2024-05-08T05:47:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.636+0000 7f0d49459640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0d4415f420 0x7f0d440d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.636+0000 7f0d49459640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d441630b0 con 0x7f0d440d2080 2024-05-08T05:47:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.636+0000 7f0d49459640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d440d2550 con 0x7f0d440d3ca0 2024-05-08T05:47:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.636+0000 7f0d49459640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0d440d2c80 con 0x7f0d4415f420 2024-05-08T05:47:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.636+0000 7f0d437fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0d440d2080 0x7f0d440d4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41034/0 (socket says 172.21.15.45:41034) 2024-05-08T05:47:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.636+0000 7f0d437fe640 1 -- 172.21.15.45:0/1670181744 learned_addr learned my addr 172.21.15.45:0/1670181744 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:47:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.636+0000 7f0d42ffd640 1 --2- 172.21.15.45:0/1670181744 >> v2:172.21.15.63:3300/0 conn(0x7f0d4415f420 0x7f0d440d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.637+0000 7f0d41ffb640 1 -- 172.21.15.45:0/1670181744 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3027602972 0 0) 0x7f0d3c001490 con 0x7f0d440d2080 2024-05-08T05:47:46.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.637+0000 7f0d41ffb640 1 -- 172.21.15.45:0/1670181744 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d2c003520 con 0x7f0d440d2080 2024-05-08T05:47:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.637+0000 7f0d41ffb640 1 -- 172.21.15.45:0/1670181744 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1758353683 0 0) 0x7f0d2c003520 con 0x7f0d440d2080 2024-05-08T05:47:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.637+0000 7f0d42ffd640 1 -- 172.21.15.45:0/1670181744 >> v1:172.21.15.63:6789/0 conn(0x7f0d440d3ca0 legacy=0x7f0d440d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.637+0000 7f0d42ffd640 1 -- 172.21.15.45:0/1670181744 >> v1:172.21.15.45:6789/0 conn(0x7f0d440d2080 legacy=0x7f0d440d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:46.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.637+0000 7f0d42ffd640 1 -- 172.21.15.45:0/1670181744 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d440d2f60 con 0x7f0d4415f420 2024-05-08T05:47:46.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.638+0000 7f0d42ffd640 1 --2- 172.21.15.45:0/1670181744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d4415f420 0x7f0d440d3760 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0d340097a0 tx=0x7f0d3402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18fa417747d76e51 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:46.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.638+0000 7f0d41ffb640 1 -- 172.21.15.45:0/1670181744 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34037070 con 0x7f0d4415f420 2024-05-08T05:47:46.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.638+0000 7f0d41ffb640 1 -- 172.21.15.45:0/1670181744 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d3402ac50 con 0x7f0d4415f420 2024-05-08T05:47:46.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.638+0000 7f0d41ffb640 1 -- 172.21.15.45:0/1670181744 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c001c40 con 0x7f0d4415f420 2024-05-08T05:47:46.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.639+0000 7f0d49459640 1 -- 172.21.15.45:0/1670181744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d4415f420 msgr2=0x7f0d440d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.639+0000 7f0d49459640 1 --2- 172.21.15.45:0/1670181744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d4415f420 0x7f0d440d3760 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0d340097a0 tx=0x7f0d3402a1c0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.639+0000 7f0d49459640 1 -- 172.21.15.45:0/1670181744 shutdown_connections 2024-05-08T05:47:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.639+0000 7f0d49459640 1 --2- 172.21.15.45:0/1670181744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d4415f420 0x7f0d440d3760 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.639+0000 7f0d49459640 1 -- 172.21.15.45:0/1670181744 >> 172.21.15.45:0/1670181744 conn(0x7f0d4408eac0 msgr2=0x7f0d4409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.639+0000 7f0d49459640 1 -- 172.21.15.45:0/1670181744 shutdown_connections 2024-05-08T05:47:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.639+0000 7f0d49459640 1 -- 172.21.15.45:0/1670181744 wait complete. 2024-05-08T05:47:46.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.640+0000 7f0d49459640 1 Processor -- start 2024-05-08T05:47:46.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.640+0000 7f0d49459640 1 -- start start 2024-05-08T05:47:46.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.641+0000 7f0d49459640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d440d2080 0x7f0d440ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.641+0000 7f0d49459640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d440ce630 0x7f0d440b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.641+0000 7f0d49459640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d440b2180 0x7f0d440b2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.641+0000 7f0d49459640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0d440d53d0 con 0x7f0d440d2080 2024-05-08T05:47:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.641+0000 7f0d49459640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0d440d2830 con 0x7f0d440b2180 2024-05-08T05:47:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.641+0000 7f0d49459640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0d440d5550 con 0x7f0d440ce630 2024-05-08T05:47:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.641+0000 7f0d42ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d440d2080 0x7f0d440ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.641+0000 7f0d42ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d440d2080 0x7f0d440ce0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36386/0 (socket says 172.21.15.45:36386) 2024-05-08T05:47:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.641+0000 7f0d42ffd640 1 -- 172.21.15.45:0/1139273175 learned_addr learned my addr 172.21.15.45:0/1139273175 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.641+0000 7f0d437fe640 1 --2- 172.21.15.45:0/1139273175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d440b2180 0x7f0d440b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.641+0000 7f0d427fc640 1 --2- 172.21.15.45:0/1139273175 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d440ce630 0x7f0d440b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.641+0000 7f0d42ffd640 1 -- 172.21.15.45:0/1139273175 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d440ce630 msgr2=0x7f0d440b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.641+0000 7f0d42ffd640 1 --2- 172.21.15.45:0/1139273175 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d440ce630 0x7f0d440b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.641+0000 7f0d42ffd640 1 -- 172.21.15.45:0/1139273175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d440b2180 msgr2=0x7f0d440b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.641+0000 7f0d42ffd640 1 --2- 172.21.15.45:0/1139273175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d440b2180 0x7f0d440b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.641+0000 7f0d42ffd640 1 -- 172.21.15.45:0/1139273175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d440c8ea0 con 0x7f0d440d2080 2024-05-08T05:47:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.642+0000 7f0d42ffd640 1 --2- 172.21.15.45:0/1139273175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d440d2080 0x7f0d440ce0f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0d340097a0 tx=0x7f0d340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:46.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.642+0000 7f0d2b7fe640 1 -- 172.21.15.45:0/1139273175 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34041070 con 0x7f0d440d2080 2024-05-08T05:47:46.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.642+0000 7f0d2b7fe640 1 -- 172.21.15.45:0/1139273175 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d3403c460 con 0x7f0d440d2080 2024-05-08T05:47:46.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.642+0000 7f0d49459640 1 -- 172.21.15.45:0/1139273175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d440c9180 con 0x7f0d440d2080 2024-05-08T05:47:46.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.642+0000 7f0d2b7fe640 1 -- 172.21.15.45:0/1139273175 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34037040 con 0x7f0d440d2080 2024-05-08T05:47:46.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.642+0000 7f0d49459640 1 -- 172.21.15.45:0/1139273175 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d440c9740 con 0x7f0d440d2080 2024-05-08T05:47:46.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.643+0000 7f0d2b7fe640 1 -- 172.21.15.45:0/1139273175 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0d34044070 con 0x7f0d440d2080 2024-05-08T05:47:46.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.644+0000 7f0d2b7fe640 1 --2- 172.21.15.45:0/1139273175 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0d30076300 0x7f0d300787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.644+0000 7f0d2b7fe640 1 -- 172.21.15.45:0/1139273175 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f0d340b79e0 con 0x7f0d440d2080 2024-05-08T05:47:46.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.644+0000 7f0d427fc640 1 --2- 172.21.15.45:0/1139273175 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0d30076300 0x7f0d300787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.645+0000 7f0d427fc640 1 --2- 172.21.15.45:0/1139273175 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0d30076300 0x7f0d300787c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0d38000f60 tx=0x7f0d38009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:46.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.645+0000 7f0d4c173740 1 Processor -- start 2024-05-08T05:47:46.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.645+0000 7f0d4c173740 1 -- start start 2024-05-08T05:47:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.646+0000 7f0d4c173740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe2b160 0x562ebbdddaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.646+0000 7f0d4c173740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe0e280 0x562ebbe0d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.646+0000 7f0d4c173740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0db40 0x562ebbde2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.646+0000 7f0d4c173740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562ebbdaa690 con 0x562ebbe0db40 2024-05-08T05:47:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.646+0000 7f0d4c173740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562ebbd47f30 con 0x562ebbe2b160 2024-05-08T05:47:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.646+0000 7f0d4c173740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x562ebbd00b20 con 0x562ebbe0e280 2024-05-08T05:47:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.646+0000 7f0d437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0db40 0x562ebbde2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.646+0000 7f0d437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0db40 0x562ebbde2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36400/0 (socket says 172.21.15.45:36400) 2024-05-08T05:47:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.646+0000 7f0d437fe640 1 -- 172.21.15.45:0/4277211055 learned_addr learned my addr 172.21.15.45:0/4277211055 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.646+0000 7f0d42ffd640 1 --2- 172.21.15.45:0/4277211055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe2b160 0x562ebbdddaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.646+0000 7f0d427fc640 1 --2- 172.21.15.45:0/4277211055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe0e280 0x562ebbe0d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.646+0000 7f0d437fe640 1 -- 172.21.15.45:0/4277211055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe0e280 msgr2=0x562ebbe0d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:46.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.646+0000 7f0d437fe640 1 --2- 172.21.15.45:0/4277211055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe0e280 0x562ebbe0d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.646+0000 7f0d437fe640 1 -- 172.21.15.45:0/4277211055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe2b160 msgr2=0x562ebbdddaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.646+0000 7f0d437fe640 1 --2- 172.21.15.45:0/4277211055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe2b160 0x562ebbdddaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.646+0000 7f0d437fe640 1 -- 172.21.15.45:0/4277211055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ebbcf8e60 con 0x562ebbe0db40 2024-05-08T05:47:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.646+0000 7f0d42ffd640 1 --2- 172.21.15.45:0/4277211055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe2b160 0x562ebbdddaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.646+0000 7f0d427fc640 1 --2- 172.21.15.45:0/4277211055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe0e280 0x562ebbe0d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.647+0000 7f0d437fe640 1 --2- 172.21.15.45:0/4277211055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0db40 0x562ebbde2950 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00f390 tx=0x7f0d3c001b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a97b255501cb0bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.647+0000 7f0d28ff9640 1 -- 172.21.15.45:0/4277211055 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c0072e0 con 0x562ebbe0db40 2024-05-08T05:47:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.647+0000 7f0d28ff9640 1 -- 172.21.15.45:0/4277211055 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d3c018070 con 0x562ebbe0db40 2024-05-08T05:47:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.647+0000 7f0d28ff9640 1 -- 172.21.15.45:0/4277211055 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c007680 con 0x562ebbe0db40 2024-05-08T05:47:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.647+0000 7f0d4c173740 1 -- 172.21.15.45:0/4277211055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0db40 msgr2=0x562ebbde2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.647+0000 7f0d4c173740 1 --2- 172.21.15.45:0/4277211055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0db40 0x562ebbde2950 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00f390 tx=0x7f0d3c001b60 comp rx=0 tx=0).stop 2024-05-08T05:47:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.647+0000 7f0d4c173740 1 -- 172.21.15.45:0/4277211055 shutdown_connections 2024-05-08T05:47:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.647+0000 7f0d4c173740 1 --2- 172.21.15.45:0/4277211055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe0e280 0x562ebbe0d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.647+0000 7f0d4c173740 1 --2- 172.21.15.45:0/4277211055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0db40 0x562ebbde2950 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.647+0000 7f0d4c173740 1 --2- 172.21.15.45:0/4277211055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe2b160 0x562ebbdddaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.647+0000 7f0d4c173740 1 -- 172.21.15.45:0/4277211055 >> 172.21.15.45:0/4277211055 conn(0x562ebbddfd60 msgr2=0x562ebbe28600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.648+0000 7f0d4c173740 1 -- 172.21.15.45:0/4277211055 shutdown_connections 2024-05-08T05:47:46.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.648+0000 7f0d4c173740 1 -- 172.21.15.45:0/4277211055 wait complete. 2024-05-08T05:47:46.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.648+0000 7f0d4c173740 1 -- start start 2024-05-08T05:47:46.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.648+0000 7f0d4c173740 1 Processor -- start 2024-05-08T05:47:46.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.649+0000 7f0d4c173740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0e280 0x562ebbdfdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.649+0000 7f0d4c173740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe2b160 0x562ebbe20550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.649+0000 7f0d4c173740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe08960 0x562ebbe09c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.649+0000 7f0d4c173740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562ebbd045e0 con 0x562ebbe0e280 2024-05-08T05:47:46.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.649+0000 7f0d4c173740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x562ebbd43ef0 con 0x562ebbe2b160 2024-05-08T05:47:46.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.649+0000 7f0d4c173740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x562ebbde1f10 con 0x562ebbe08960 2024-05-08T05:47:46.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.649+0000 7f0d4c173740 10 client.0 _ll_register_callbacks cb 0x562ebbcff320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:47:46.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.649+0000 7f0d42ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0e280 0x562ebbdfdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.649+0000 7f0d42ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0e280 0x562ebbdfdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36402/0 (socket says 172.21.15.45:36402) 2024-05-08T05:47:46.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.649+0000 7f0d42ffd640 1 -- 172.21.15.45:0/2763836587 learned_addr learned my addr 172.21.15.45:0/2763836587 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:46.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.649+0000 7f0d42ffd640 1 -- 172.21.15.45:0/2763836587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe08960 msgr2=0x562ebbe09c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:46.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.649+0000 7f0d437fe640 1 --2- 172.21.15.45:0/2763836587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe08960 0x562ebbe09c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.649+0000 7f0d427fc640 1 --2- 172.21.15.45:0/2763836587 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe2b160 0x562ebbe20550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.649+0000 7f0d42ffd640 1 --2- 172.21.15.45:0/2763836587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe08960 0x562ebbe09c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.655 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.649+0000 7f0d42ffd640 1 -- 172.21.15.45:0/2763836587 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe2b160 msgr2=0x562ebbe20550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:46.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.649+0000 7f0d42ffd640 1 --2- 172.21.15.45:0/2763836587 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe2b160 0x562ebbe20550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.649+0000 7f0d42ffd640 1 -- 172.21.15.45:0/2763836587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ebbb237d0 con 0x562ebbe0e280 2024-05-08T05:47:46.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.650+0000 7f0d42ffd640 1 --2- 172.21.15.45:0/2763836587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0e280 0x562ebbdfdea0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f0d340b5cc0 tx=0x7f0d34089640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ecfa545099a72fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:46.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.650+0000 7f0d21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:47:46.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.650+0000 7f0d21ffb640 1 -- 172.21.15.45:0/2763836587 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34080900 con 0x562ebbe0e280 2024-05-08T05:47:46.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.650+0000 7f0d21ffb640 1 -- 172.21.15.45:0/2763836587 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f0d3407c4a0 con 0x562ebbe0e280 2024-05-08T05:47:46.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.650+0000 7f0d4c173740 1 -- 172.21.15.45:0/2763836587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d3c018070 con 0x562ebbe0e280 2024-05-08T05:47:46.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.650+0000 7f0d21ffb640 1 -- 172.21.15.45:0/2763836587 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f0d340802a0 con 0x562ebbe0e280 2024-05-08T05:47:46.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.650+0000 7f0d4c173740 10 client.7035 fetch_fsmap learned FSMap version 67 2024-05-08T05:47:46.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.650+0000 7f0d4c173740 1 -- 172.21.15.45:0/2763836587 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x562ebba593f0 con 0x562ebbe0e280 2024-05-08T05:47:46.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.650+0000 7f0d21ffb640 1 -- 172.21.15.45:0/2763836587 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d3c018070 con 0x562ebbe0e280 2024-05-08T05:47:46.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.650+0000 7f0d21ffb640 1 -- 172.21.15.45:0/2763836587 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f0d3407c6a0 con 0x562ebbe0e280 2024-05-08T05:47:46.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.650+0000 7f0d4c173740 10 client.7035 fetch_fsmap finished waiting for FSMap version 67 2024-05-08T05:47:46.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.650+0000 7f0d4c173740 10 client.7035 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-08T05:47:46.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.650+0000 7f0d4c173740 20 client.7035 populate_metadata read hostname 'smithi045' 2024-05-08T05:47:46.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.650+0000 7f0d4c173740 1 --2- 172.21.15.45:0/2763836587 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x562ebbe0e770 0x562ebbe0eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.650+0000 7f0d4c173740 4 client.7035 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:47:46.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.650+0000 7f0d4c173740 1 -- 172.21.15.45:0/2763836587 --> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ebbd55b40 con 0x562ebbe0e770 2024-05-08T05:47:46.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.651+0000 7f0d427fc640 1 --2- 172.21.15.45:0/2763836587 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x562ebbe0e770 0x562ebbe0eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.651+0000 7f0d427fc640 1 --2- 172.21.15.45:0/2763836587 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x562ebbe0e770 0x562ebbe0eb90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d2daa75ad6ebc3a server_cookie=48bbef4d31c967d in_seq=0 out_seq=0 2024-05-08T05:47:46.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.651+0000 7f0d21ffb640 10 client.7035 ms_handle_connect on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:46.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.652+0000 7f0d21ffb640 1 -- 172.21.15.45:0/2763836587 <== mds.0 v2:172.21.15.63:6835/3194497846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562ebbd55b40 con 0x562ebbe0e770 2024-05-08T05:47:46.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.652+0000 7f0d21ffb640 10 client.7035 handle_command_reply: tid=0 2024-05-08T05:47:46.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.652+0000 7f0d4c173740 1 client.7035 shutdown 2024-05-08T05:47:46.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.652+0000 7f0d4c173740 1 -- 172.21.15.45:0/2763836587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0e280 msgr2=0x562ebbdfdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:46.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.652+0000 7f0d4c173740 1 --2- 172.21.15.45:0/2763836587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0e280 0x562ebbdfdea0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f0d340b5cc0 tx=0x7f0d34089640 comp rx=0 tx=0).stop 2024-05-08T05:47:46.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.652+0000 7f0d4c173740 1 -- 172.21.15.45:0/2763836587 shutdown_connections 2024-05-08T05:47:46.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.652+0000 7f0d4c173740 1 --2- 172.21.15.45:0/2763836587 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x562ebbe0e770 0x562ebbe0eb90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.652+0000 7f0d4c173740 1 --2- 172.21.15.45:0/2763836587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe08960 0x562ebbe09c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.652+0000 7f0d4c173740 1 --2- 172.21.15.45:0/2763836587 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe2b160 0x562ebbe20550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.652+0000 7f0d21ffb640 0 client.7035 ms_handle_reset on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:46.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.652+0000 7f0d4c173740 1 --2- 172.21.15.45:0/2763836587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0e280 0x562ebbdfdea0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.652+0000 7f0d4c173740 1 -- 172.21.15.45:0/2763836587 >> 172.21.15.45:0/2763836587 conn(0x562ebbe0db40 msgr2=0x562ebbddd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:46.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.652+0000 7f0d4c173740 1 -- 172.21.15.45:0/2763836587 shutdown_connections 2024-05-08T05:47:46.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.652+0000 7f0d4c173740 1 -- 172.21.15.45:0/2763836587 wait complete. 2024-05-08T05:47:46.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.652+0000 7f0d4c173740 20 client.7035 trim_cache size 0 max 16384 2024-05-08T05:47:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.670+0000 7f0d4c173740 1 Processor -- start 2024-05-08T05:47:46.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.671+0000 7f0d4c173740 1 -- start start 2024-05-08T05:47:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.671+0000 7f0d4c173740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe08960 0x562ebbdfdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.671+0000 7f0d4c173740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0e280 0x562ebbde3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.671+0000 7f0d4c173740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe2b160 0x562ebbde3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.671+0000 7f0d4c173740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44202250 con 0x562ebbe0e280 2024-05-08T05:47:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.671+0000 7f0d4c173740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44201270 con 0x562ebbe08960 2024-05-08T05:47:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.671+0000 7f0d4c173740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0d440caf60 con 0x562ebbe2b160 2024-05-08T05:47:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.671+0000 7f0d427fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0e280 0x562ebbde3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.671+0000 7f0d427fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0e280 0x562ebbde3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36404/0 (socket says 172.21.15.45:36404) 2024-05-08T05:47:46.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.671+0000 7f0d427fc640 1 -- 172.21.15.45:0/3179298328 learned_addr learned my addr 172.21.15.45:0/3179298328 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.671+0000 7f0d437fe640 1 --2- 172.21.15.45:0/3179298328 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe2b160 0x562ebbde3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.671+0000 7f0d42ffd640 1 --2- 172.21.15.45:0/3179298328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe08960 0x562ebbdfdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.672+0000 7f0d427fc640 1 -- 172.21.15.45:0/3179298328 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe2b160 msgr2=0x562ebbde3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.672+0000 7f0d427fc640 1 --2- 172.21.15.45:0/3179298328 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe2b160 0x562ebbde3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.672+0000 7f0d427fc640 1 -- 172.21.15.45:0/3179298328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe08960 msgr2=0x562ebbdfdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.672+0000 7f0d427fc640 1 --2- 172.21.15.45:0/3179298328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe08960 0x562ebbdfdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.672+0000 7f0d427fc640 1 -- 172.21.15.45:0/3179298328 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ebba190f0 con 0x562ebbe0e280 2024-05-08T05:47:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.672+0000 7f0d427fc640 1 --2- 172.21.15.45:0/3179298328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0e280 0x562ebbde3310 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x562ebbe29700 tx=0x7f0d38012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908a0fafedf1f8ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.672+0000 7f0d23fff640 1 -- 172.21.15.45:0/3179298328 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3800fe80 con 0x562ebbe0e280 2024-05-08T05:47:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.672+0000 7f0d23fff640 1 -- 172.21.15.45:0/3179298328 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d3802c070 con 0x562ebbe0e280 2024-05-08T05:47:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.672+0000 7f0d23fff640 1 -- 172.21.15.45:0/3179298328 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38015b30 con 0x562ebbe0e280 2024-05-08T05:47:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.672+0000 7f0d4c173740 1 -- 172.21.15.45:0/3179298328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0e280 msgr2=0x562ebbde3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.672+0000 7f0d4c173740 1 --2- 172.21.15.45:0/3179298328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0e280 0x562ebbde3310 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x562ebbe29700 tx=0x7f0d38012000 comp rx=0 tx=0).stop 2024-05-08T05:47:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.672+0000 7f0d4c173740 1 -- 172.21.15.45:0/3179298328 shutdown_connections 2024-05-08T05:47:46.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.672+0000 7f0d4c173740 1 --2- 172.21.15.45:0/3179298328 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe2b160 0x562ebbde3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.672+0000 7f0d4c173740 1 --2- 172.21.15.45:0/3179298328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe0e280 0x562ebbde3310 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.672+0000 7f0d4c173740 1 --2- 172.21.15.45:0/3179298328 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe08960 0x562ebbdfdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.672+0000 7f0d4c173740 1 -- 172.21.15.45:0/3179298328 >> 172.21.15.45:0/3179298328 conn(0x562ebbe0db40 msgr2=0x562ebbe20f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.673+0000 7f0d4c173740 1 -- 172.21.15.45:0/3179298328 shutdown_connections 2024-05-08T05:47:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.673+0000 7f0d4c173740 1 -- 172.21.15.45:0/3179298328 wait complete. 2024-05-08T05:47:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.673+0000 7f0d4c173740 1 -- start start 2024-05-08T05:47:46.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.673+0000 7f0d4c173740 1 Processor -- start 2024-05-08T05:47:46.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.674+0000 7f0d4c173740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe08960 0x562ebbe0fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.674+0000 7f0d4c173740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe0e280 0x562ebbe9f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.674+0000 7f0d4c173740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe2b160 0x562ebbea5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.674+0000 7f0d4c173740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562ebbe2cad0 con 0x562ebbe2b160 2024-05-08T05:47:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.674+0000 7f0d4c173740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44200c50 con 0x562ebbe08960 2024-05-08T05:47:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.674+0000 7f0d4c173740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x562ebbdd85d0 con 0x562ebbe0e280 2024-05-08T05:47:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.674+0000 7f0d4c173740 10 client.0 _ll_register_callbacks cb 0x7f0d34009110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:47:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.674+0000 7f0d437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe2b160 0x562ebbea5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.674+0000 7f0d437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe2b160 0x562ebbea5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36414/0 (socket says 172.21.15.45:36414) 2024-05-08T05:47:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.674+0000 7f0d437fe640 1 -- 172.21.15.45:0/4185313565 learned_addr learned my addr 172.21.15.45:0/4185313565 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.674+0000 7f0d42ffd640 1 --2- 172.21.15.45:0/4185313565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe08960 0x562ebbe0fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.674+0000 7f0d427fc640 1 --2- 172.21.15.45:0/4185313565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe0e280 0x562ebbe9f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.674+0000 7f0d437fe640 1 -- 172.21.15.45:0/4185313565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe0e280 msgr2=0x562ebbe9f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.674+0000 7f0d437fe640 1 --2- 172.21.15.45:0/4185313565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe0e280 0x562ebbe9f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.674+0000 7f0d437fe640 1 -- 172.21.15.45:0/4185313565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe08960 msgr2=0x562ebbe0fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.674+0000 7f0d437fe640 1 --2- 172.21.15.45:0/4185313565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe08960 0x562ebbe0fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.674+0000 7f0d437fe640 1 -- 172.21.15.45:0/4185313565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ebbe27a80 con 0x562ebbe2b160 2024-05-08T05:47:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.674+0000 7f0d437fe640 1 --2- 172.21.15.45:0/4185313565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe2b160 0x562ebbea5ee0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c015040 tx=0x7f0d3c005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844e05322b39b20 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.674+0000 7f0d21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:47:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.675+0000 7f0d21ffb640 1 -- 172.21.15.45:0/4185313565 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c007a60 con 0x562ebbe2b160 2024-05-08T05:47:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.675+0000 7f0d21ffb640 1 -- 172.21.15.45:0/4185313565 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f0d3c029d40 con 0x562ebbe2b160 2024-05-08T05:47:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.675+0000 7f0d4c173740 1 -- 172.21.15.45:0/4185313565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d3802c070 con 0x562ebbe2b160 2024-05-08T05:47:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.675+0000 7f0d21ffb640 1 -- 172.21.15.45:0/4185313565 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f0d3c030050 con 0x562ebbe2b160 2024-05-08T05:47:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.675+0000 7f0d4c173740 10 client.7041 fetch_fsmap learned FSMap version 67 2024-05-08T05:47:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.675+0000 7f0d4c173740 1 -- 172.21.15.45:0/4185313565 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x562ebbe10130 con 0x562ebbe2b160 2024-05-08T05:47:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.675+0000 7f0d21ffb640 1 -- 172.21.15.45:0/4185313565 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d3802c070 con 0x562ebbe2b160 2024-05-08T05:47:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.675+0000 7f0d21ffb640 1 -- 172.21.15.45:0/4185313565 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f0d3c033020 con 0x562ebbe2b160 2024-05-08T05:47:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.675+0000 7f0d4c173740 10 client.7041 fetch_fsmap finished waiting for FSMap version 67 2024-05-08T05:47:46.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.675+0000 7f0d4c173740 10 client.7041 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-08T05:47:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.675+0000 7f0d4c173740 20 client.7041 populate_metadata read hostname 'smithi045' 2024-05-08T05:47:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.675+0000 7f0d4c173740 1 --2- 172.21.15.45:0/4185313565 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x562ebbe0f470 0x562ebbe20320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.675+0000 7f0d4c173740 4 client.7041 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:47:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.675+0000 7f0d4c173740 1 -- 172.21.15.45:0/4185313565 --> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562ebbd4f9b0 con 0x562ebbe0f470 2024-05-08T05:47:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.675+0000 7f0d42ffd640 1 --2- 172.21.15.45:0/4185313565 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x562ebbe0f470 0x562ebbe20320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.676+0000 7f0d21ffb640 10 client.7041 ms_handle_connect on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.676+0000 7f0d42ffd640 1 --2- 172.21.15.45:0/4185313565 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x562ebbe0f470 0x562ebbe20320 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f0ef9bfa447856 server_cookie=a353cece190fe645 in_seq=0 out_seq=0 2024-05-08T05:47:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.676+0000 7f0d21ffb640 1 -- 172.21.15.45:0/4185313565 <== mds.0 v2:172.21.15.63:6835/3194497846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562ebbd4f9b0 con 0x562ebbe0f470 2024-05-08T05:47:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.676+0000 7f0d21ffb640 10 client.7041 handle_command_reply: tid=0 2024-05-08T05:47:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.676+0000 7f0d4c173740 1 client.7041 shutdown 2024-05-08T05:47:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.676+0000 7f0d4c173740 1 -- 172.21.15.45:0/4185313565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe2b160 msgr2=0x562ebbea5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.676+0000 7f0d4c173740 1 --2- 172.21.15.45:0/4185313565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe2b160 0x562ebbea5ee0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c015040 tx=0x7f0d3c005b70 comp rx=0 tx=0).stop 2024-05-08T05:47:46.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.676+0000 7f0d4c173740 1 -- 172.21.15.45:0/4185313565 shutdown_connections 2024-05-08T05:47:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.676+0000 7f0d4c173740 1 --2- 172.21.15.45:0/4185313565 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x562ebbe0f470 0x562ebbe20320 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.676+0000 7f0d4c173740 1 --2- 172.21.15.45:0/4185313565 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x562ebbe0e280 0x562ebbe9f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.676+0000 7f0d4c173740 1 --2- 172.21.15.45:0/4185313565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562ebbe2b160 0x562ebbea5ee0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.676+0000 7f0d21ffb640 0 client.7041 ms_handle_reset on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.676+0000 7f0d4c173740 1 --2- 172.21.15.45:0/4185313565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x562ebbe08960 0x562ebbe0fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.676+0000 7f0d4c173740 1 -- 172.21.15.45:0/4185313565 >> 172.21.15.45:0/4185313565 conn(0x562ebbe0db40 msgr2=0x562ebbdd9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.676+0000 7f0d4c173740 1 -- 172.21.15.45:0/4185313565 shutdown_connections 2024-05-08T05:47:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.676+0000 7f0d4c173740 1 -- 172.21.15.45:0/4185313565 wait complete. 2024-05-08T05:47:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.676+0000 7f0d4c173740 20 client.7041 trim_cache size 0 max 16384 2024-05-08T05:47:46.683 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:47:46.683 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-05-08T05:47:46.683 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-05-08T05:47:46.684 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:47:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.677+0000 7f0d49459640 1 -- 172.21.15.45:0/1139273175 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0d30076300 msgr2=0x7f0d300787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:46.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.677+0000 7f0d49459640 1 --2- 172.21.15.45:0/1139273175 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0d30076300 0x7f0d300787c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0d38000f60 tx=0x7f0d38009040 comp rx=0 tx=0).stop 2024-05-08T05:47:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.677+0000 7f0d49459640 1 -- 172.21.15.45:0/1139273175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d440d2080 msgr2=0x7f0d440ce0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.677+0000 7f0d49459640 1 --2- 172.21.15.45:0/1139273175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d440d2080 0x7f0d440ce0f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0d340097a0 tx=0x7f0d340046c0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.677+0000 7f0d49459640 1 -- 172.21.15.45:0/1139273175 shutdown_connections 2024-05-08T05:47:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.677+0000 7f0d49459640 1 --2- 172.21.15.45:0/1139273175 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0d30076300 0x7f0d300787c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.677+0000 7f0d49459640 1 --2- 172.21.15.45:0/1139273175 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d440ce630 0x7f0d440b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.677+0000 7f0d49459640 1 --2- 172.21.15.45:0/1139273175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d440b2180 0x7f0d440b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.677+0000 7f0d49459640 1 --2- 172.21.15.45:0/1139273175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d440d2080 0x7f0d440ce0f0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.677+0000 7f0d49459640 1 -- 172.21.15.45:0/1139273175 >> 172.21.15.45:0/1139273175 conn(0x7f0d4408eac0 msgr2=0x7f0d4415e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.677+0000 7f0d49459640 1 -- 172.21.15.45:0/1139273175 shutdown_connections 2024-05-08T05:47:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.677+0000 7f0d49459640 1 -- 172.21.15.45:0/1139273175 wait complete. 2024-05-08T05:47:46.689 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T05:47:46.689 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:47:46.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.792+0000 7f8ba58c8640 1 Processor -- start 2024-05-08T05:47:46.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.792+0000 7f8ba58c8640 1 -- start start 2024-05-08T05:47:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.793+0000 7f8ba58c8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ba006f570 0x7f8ba00cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.793+0000 7f8ba58c8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ba00ce600 con 0x7f8ba006f190 2024-05-08T05:47:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.793+0000 7f8ba58c8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ba0073180 con 0x7f8ba006f9d0 2024-05-08T05:47:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.793+0000 7f8ba58c8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f8ba00b09c0 con 0x7f8ba006f570 2024-05-08T05:47:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.793+0000 7f8b9e7fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ba006f570 0x7f8ba00cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.793+0000 7f8b9effd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8ba006f190 0x7f8ba0073f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41050/0 (socket says 172.21.15.45:41050) 2024-05-08T05:47:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.793+0000 7f8b9e7fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8ba006f570 0x7f8ba00cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36420/0 (socket says 172.21.15.45:36420) 2024-05-08T05:47:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.793+0000 7f8b9effd640 1 -- 172.21.15.45:0/1936892574 learned_addr learned my addr 172.21.15.45:0/1936892574 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:47:46.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.794+0000 7f8b9e7fc640 1 -- 172.21.15.45:0/1936892574 >> v1:172.21.15.63:6789/0 conn(0x7f8ba006f9d0 legacy=0x7f8ba00adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:46.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.794+0000 7f8b9e7fc640 1 -- 172.21.15.45:0/1936892574 >> v1:172.21.15.45:6789/0 conn(0x7f8ba006f190 legacy=0x7f8ba0073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:46.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.794+0000 7f8b9e7fc640 1 -- 172.21.15.45:0/1936892574 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba0073420 con 0x7f8ba006f570 2024-05-08T05:47:46.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.794+0000 7f8b9dffb640 1 -- 172.21.15.45:0/1936892574 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 480678097 0 0) 0x7f8ba00ce600 con 0x7f8ba006f190 2024-05-08T05:47:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.794+0000 7f8b9e7fc640 1 --2- 172.21.15.45:0/1936892574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba006f570 0x7f8ba00cda20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f8b880097a0 tx=0x7f8b8802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25938aacdc9d0534 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.795+0000 7f8b9dffb640 1 -- 172.21.15.45:0/1936892574 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88037070 con 0x7f8ba006f570 2024-05-08T05:47:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.795+0000 7f8b9dffb640 1 -- 172.21.15.45:0/1936892574 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b88032440 con 0x7f8ba006f570 2024-05-08T05:47:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.795+0000 7f8b9dffb640 1 -- 172.21.15.45:0/1936892574 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88032810 con 0x7f8ba006f570 2024-05-08T05:47:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.795+0000 7f8ba58c8640 1 -- 172.21.15.45:0/1936892574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba006f570 msgr2=0x7f8ba00cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.795+0000 7f8ba58c8640 1 --2- 172.21.15.45:0/1936892574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba006f570 0x7f8ba00cda20 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f8b880097a0 tx=0x7f8b8802a1a0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.795+0000 7f8ba58c8640 1 -- 172.21.15.45:0/1936892574 shutdown_connections 2024-05-08T05:47:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.795+0000 7f8ba58c8640 1 --2- 172.21.15.45:0/1936892574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba006f570 0x7f8ba00cda20 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.795+0000 7f8ba58c8640 1 -- 172.21.15.45:0/1936892574 >> 172.21.15.45:0/1936892574 conn(0x7f8ba00b2cc0 msgr2=0x7f8ba00b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.795+0000 7f8ba58c8640 1 -- 172.21.15.45:0/1936892574 shutdown_connections 2024-05-08T05:47:46.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.795+0000 7f8ba58c8640 1 -- 172.21.15.45:0/1936892574 wait complete. 2024-05-08T05:47:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.796+0000 7f8ba58c8640 1 Processor -- start 2024-05-08T05:47:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.797+0000 7f8ba58c8640 1 -- start start 2024-05-08T05:47:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.797+0000 7f8ba58c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba006f190 0x7f8ba01f9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.797+0000 7f8ba58c8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ba006f570 0x7f8ba0157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.797+0000 7f8ba58c8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ba00cbc60 0x7f8ba015d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.797+0000 7f8ba58c8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba00cdf60 con 0x7f8ba006f190 2024-05-08T05:47:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.797+0000 7f8ba58c8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba00ce0e0 con 0x7f8ba006f570 2024-05-08T05:47:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.797+0000 7f8ba58c8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba00ce260 con 0x7f8ba00cbc60 2024-05-08T05:47:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.797+0000 7f8b9effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba006f190 0x7f8ba01f9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.797+0000 7f8b9effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba006f190 0x7f8ba01f9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36424/0 (socket says 172.21.15.45:36424) 2024-05-08T05:47:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.797+0000 7f8b9effd640 1 -- 172.21.15.45:0/4081948318 learned_addr learned my addr 172.21.15.45:0/4081948318 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:46.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.798+0000 7f8b9e7fc640 1 --2- 172.21.15.45:0/4081948318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ba006f570 0x7f8ba0157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.798+0000 7f8b9f7fe640 1 --2- 172.21.15.45:0/4081948318 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ba00cbc60 0x7f8ba015d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.798+0000 7f8b9effd640 1 -- 172.21.15.45:0/4081948318 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ba00cbc60 msgr2=0x7f8ba015d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.798+0000 7f8b9effd640 1 --2- 172.21.15.45:0/4081948318 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ba00cbc60 0x7f8ba015d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.798+0000 7f8b9effd640 1 -- 172.21.15.45:0/4081948318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ba006f570 msgr2=0x7f8ba0157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.798+0000 7f8b9effd640 1 --2- 172.21.15.45:0/4081948318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ba006f570 0x7f8ba0157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.798+0000 7f8b9effd640 1 -- 172.21.15.45:0/4081948318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba01f9a70 con 0x7f8ba006f190 2024-05-08T05:47:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.798+0000 7f8b9e7fc640 1 --2- 172.21.15.45:0/4081948318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ba006f570 0x7f8ba0157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.798+0000 7f8b9effd640 1 --2- 172.21.15.45:0/4081948318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba006f190 0x7f8ba01f9530 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800c760 tx=0x7f8b9800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.798+0000 7f8b97fff640 1 -- 172.21.15.45:0/4081948318 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98018070 con 0x7f8ba006f190 2024-05-08T05:47:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.798+0000 7f8b97fff640 1 -- 172.21.15.45:0/4081948318 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b9800a640 con 0x7f8ba006f190 2024-05-08T05:47:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.798+0000 7f8ba58c8640 1 -- 172.21.15.45:0/4081948318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba00cabc0 con 0x7f8ba006f190 2024-05-08T05:47:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.798+0000 7f8b97fff640 1 -- 172.21.15.45:0/4081948318 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9800aa70 con 0x7f8ba006f190 2024-05-08T05:47:46.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.798+0000 7f8ba58c8640 1 -- 172.21.15.45:0/4081948318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba00cb180 con 0x7f8ba006f190 2024-05-08T05:47:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.799+0000 7f8ba58c8640 1 -- 172.21.15.45:0/4081948318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba00693f0 con 0x7f8ba006f190 2024-05-08T05:47:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.800+0000 7f8b97fff640 1 -- 172.21.15.45:0/4081948318 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8b9801b070 con 0x7f8ba006f190 2024-05-08T05:47:46.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.800+0000 7f8b97fff640 1 --2- 172.21.15.45:0/4081948318 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8b800762d0 0x7f8b80078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:46.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.800+0000 7f8b97fff640 1 -- 172.21.15.45:0/4081948318 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f8b98096e20 con 0x7f8ba006f190 2024-05-08T05:47:46.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.801+0000 7f8b9e7fc640 1 --2- 172.21.15.45:0/4081948318 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8b800762d0 0x7f8b80078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:46.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.801+0000 7f8b9e7fc640 1 --2- 172.21.15.45:0/4081948318 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8b800762d0 0x7f8b80078790 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8b88034000 tx=0x7f8b8802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:46.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:46.805+0000 7f8b97fff640 1 -- 172.21.15.45:0/4081948318 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8ba01f9a70 con 0x7f8ba006f190 2024-05-08T05:47:47.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.040+0000 7f8ba58c8640 1 -- 172.21.15.45:0/4081948318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ba00ce930 con 0x7f8ba006f190 2024-05-08T05:47:47.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.040+0000 7f8b97fff640 1 -- 172.21.15.45:0/4081948318 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5184 (secure 0 0 0) 0x7f8b98060520 con 0x7f8ba006f190 2024-05-08T05:47:47.042 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:47:47.042 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2616298287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2616298287},{"type":"v1","addr":"172.21.15.63:6836","nonce":2616298287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6744,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2854804427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2854804427},{"type":"v1","addr":"172.21.15.45:6834","nonce":2854804427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2945763952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2945763952},{"type":"v1","addr":"172.21.15.45:6835","nonce":2945763952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:47:00.211837+0000","modified":"2024-05-08T05:47:20.541349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6759},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6759":{"gid":6759,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":9,"addr":"172.21.15.63:6837/3194497846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":3194497846},{"type":"v1","addr":"172.21.15.63:6837","nonce":3194497846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6759,"qdb_cluster":[6759]},"id":6}]} 2024-05-08T05:47:47.042 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 67 2024-05-08T05:47:47.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.042+0000 7f8ba58c8640 1 -- 172.21.15.45:0/4081948318 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8b800762d0 msgr2=0x7f8b80078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.042+0000 7f8ba58c8640 1 --2- 172.21.15.45:0/4081948318 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8b800762d0 0x7f8b80078790 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8b88034000 tx=0x7f8b8802e000 comp rx=0 tx=0).stop 2024-05-08T05:47:47.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.042+0000 7f8ba58c8640 1 -- 172.21.15.45:0/4081948318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba006f190 msgr2=0x7f8ba01f9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.042+0000 7f8ba58c8640 1 --2- 172.21.15.45:0/4081948318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba006f190 0x7f8ba01f9530 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800c760 tx=0x7f8b9800cc20 comp rx=0 tx=0).stop 2024-05-08T05:47:47.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.042+0000 7f8ba58c8640 1 -- 172.21.15.45:0/4081948318 shutdown_connections 2024-05-08T05:47:47.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.042+0000 7f8ba58c8640 1 --2- 172.21.15.45:0/4081948318 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8b800762d0 0x7f8b80078790 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.042+0000 7f8ba58c8640 1 --2- 172.21.15.45:0/4081948318 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8ba00cbc60 0x7f8ba015d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.042+0000 7f8ba58c8640 1 --2- 172.21.15.45:0/4081948318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8ba006f570 0x7f8ba0157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.042+0000 7f8ba58c8640 1 --2- 172.21.15.45:0/4081948318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ba006f190 0x7f8ba01f9530 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.042+0000 7f8ba58c8640 1 -- 172.21.15.45:0/4081948318 >> 172.21.15.45:0/4081948318 conn(0x7f8ba00b2cc0 msgr2=0x7f8ba00d08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.042+0000 7f8ba58c8640 1 -- 172.21.15.45:0/4081948318 shutdown_connections 2024-05-08T05:47:47.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.042+0000 7f8ba58c8640 1 -- 172.21.15.45:0/4081948318 wait complete. 2024-05-08T05:47:47.054 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2024-05-08T05:47:47.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.159+0000 7fcafafe4640 1 Processor -- start 2024-05-08T05:47:47.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.159+0000 7fcafafe4640 1 -- start start 2024-05-08T05:47:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.160+0000 7fcafafe4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcaf415f380 0x7fcaf40d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.160+0000 7fcafafe4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcaf40d0cf0 0x7fcaf40d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.160+0000 7fcafafe4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcaf40d3b00 0x7fcaf4162e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.160+0000 7fcafafe4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fcaf40d30f0 con 0x7fcaf40d3b00 2024-05-08T05:47:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.160+0000 7fcafafe4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fcaf40d1150 con 0x7fcaf40d0cf0 2024-05-08T05:47:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.160+0000 7fcafafe4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf40d2370 con 0x7fcaf415f380 2024-05-08T05:47:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.160+0000 7fcaf9581640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcaf40d3b00 0x7fcaf4162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.160+0000 7fcaf9581640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcaf40d3b00 0x7fcaf4162e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36430/0 (socket says 172.21.15.45:36430) 2024-05-08T05:47:47.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.160+0000 7fcaf9581640 1 -- 172.21.15.45:0/1754468008 learned_addr learned my addr 172.21.15.45:0/1754468008 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.160+0000 7fcaf8d80640 1 --2- 172.21.15.45:0/1754468008 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcaf415f380 0x7fcaf40d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.160+0000 7fcaf3fff640 1 --2- 172.21.15.45:0/1754468008 >> v2:172.21.15.63:3300/0 conn(0x7fcaf40d0cf0 0x7fcaf40d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.160+0000 7fcaf9581640 1 -- 172.21.15.45:0/1754468008 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcaf415f380 msgr2=0x7fcaf40d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.160+0000 7fcaf9581640 1 --2- 172.21.15.45:0/1754468008 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcaf415f380 0x7fcaf40d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.160+0000 7fcaf9581640 1 -- 172.21.15.45:0/1754468008 >> v2:172.21.15.63:3300/0 conn(0x7fcaf40d0cf0 msgr2=0x7fcaf40d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.160+0000 7fcaf9581640 1 --2- 172.21.15.45:0/1754468008 >> v2:172.21.15.63:3300/0 conn(0x7fcaf40d0cf0 0x7fcaf40d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.160+0000 7fcaf9581640 1 -- 172.21.15.45:0/1754468008 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf40d1760 con 0x7fcaf40d3b00 2024-05-08T05:47:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.161+0000 7fcaf9581640 1 --2- 172.21.15.45:0/1754468008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaf40d3b00 0x7fcaf4162e90 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fcaf408f8b0 tx=0x7fcaec02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6366ffc0f0a71d59 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.161+0000 7fcaf37fe640 1 -- 172.21.15.45:0/1754468008 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec036070 con 0x7fcaf40d3b00 2024-05-08T05:47:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.161+0000 7fcaf37fe640 1 -- 172.21.15.45:0/1754468008 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaec031600 con 0x7fcaf40d3b00 2024-05-08T05:47:47.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.161+0000 7fcaf37fe640 1 -- 172.21.15.45:0/1754468008 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec0319d0 con 0x7fcaf40d3b00 2024-05-08T05:47:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.162+0000 7fcafafe4640 1 -- 172.21.15.45:0/1754468008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaf40d3b00 msgr2=0x7fcaf4162e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.162+0000 7fcafafe4640 1 --2- 172.21.15.45:0/1754468008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaf40d3b00 0x7fcaf4162e90 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fcaf408f8b0 tx=0x7fcaec02a6a0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.162+0000 7fcafafe4640 1 -- 172.21.15.45:0/1754468008 shutdown_connections 2024-05-08T05:47:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.162+0000 7fcafafe4640 1 --2- 172.21.15.45:0/1754468008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaf40d3b00 0x7fcaf4162e90 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.162+0000 7fcafafe4640 1 --2- 172.21.15.45:0/1754468008 >> v2:172.21.15.63:3300/0 conn(0x7fcaf40d0cf0 0x7fcaf40d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.162+0000 7fcafafe4640 1 --2- 172.21.15.45:0/1754468008 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcaf415f380 0x7fcaf40d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.162+0000 7fcafafe4640 1 -- 172.21.15.45:0/1754468008 >> 172.21.15.45:0/1754468008 conn(0x7fcaf40d4fe0 msgr2=0x7fcaf40d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.162+0000 7fcafafe4640 1 -- 172.21.15.45:0/1754468008 shutdown_connections 2024-05-08T05:47:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.162+0000 7fcafafe4640 1 -- 172.21.15.45:0/1754468008 wait complete. 2024-05-08T05:47:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.163+0000 7fcafafe4640 1 Processor -- start 2024-05-08T05:47:47.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.163+0000 7fcafafe4640 1 -- start start 2024-05-08T05:47:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcafafe4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaf40d0cf0 0x7fcaf40a2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcafafe4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcaf40d3b00 0x7fcaf415cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcafafe4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcaf415f380 0x7fcaf40ba360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcafafe4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf40d2000 con 0x7fcaf40d0cf0 2024-05-08T05:47:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcafafe4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf40d1d20 con 0x7fcaf40d3b00 2024-05-08T05:47:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcafafe4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf415f760 con 0x7fcaf415f380 2024-05-08T05:47:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcaf8d80640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaf40d0cf0 0x7fcaf40a2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcaf8d80640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaf40d0cf0 0x7fcaf40a2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36446/0 (socket says 172.21.15.45:36446) 2024-05-08T05:47:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcaf8d80640 1 -- 172.21.15.45:0/2574622804 learned_addr learned my addr 172.21.15.45:0/2574622804 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcaf9581640 1 --2- 172.21.15.45:0/2574622804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcaf415f380 0x7fcaf40ba360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcaf3fff640 1 --2- 172.21.15.45:0/2574622804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcaf40d3b00 0x7fcaf415cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcaf8d80640 1 -- 172.21.15.45:0/2574622804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcaf415f380 msgr2=0x7fcaf40ba360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcaf8d80640 1 --2- 172.21.15.45:0/2574622804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcaf415f380 0x7fcaf40ba360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcaf8d80640 1 -- 172.21.15.45:0/2574622804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcaf40d3b00 msgr2=0x7fcaf415cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcaf8d80640 1 --2- 172.21.15.45:0/2574622804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcaf40d3b00 0x7fcaf415cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcaf8d80640 1 -- 172.21.15.45:0/2574622804 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf41f8a80 con 0x7fcaf40d0cf0 2024-05-08T05:47:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcaf9581640 1 --2- 172.21.15.45:0/2574622804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcaf415f380 0x7fcaf40ba360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcaf3fff640 1 --2- 172.21.15.45:0/2574622804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcaf40d3b00 0x7fcaf415cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcaf8d80640 1 --2- 172.21.15.45:0/2574622804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaf40d0cf0 0x7fcaf40a2750 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fcae800c7f0 tx=0x7fcae800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcaf17fa640 1 -- 172.21.15.45:0/2574622804 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8015070 con 0x7fcaf40d0cf0 2024-05-08T05:47:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcaf17fa640 1 -- 172.21.15.45:0/2574622804 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae8004220 con 0x7fcaf40d0cf0 2024-05-08T05:47:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.164+0000 7fcafafe4640 1 -- 172.21.15.45:0/2574622804 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf41f8d60 con 0x7fcaf40d0cf0 2024-05-08T05:47:47.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.165+0000 7fcaf17fa640 1 -- 172.21.15.45:0/2574622804 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8007620 con 0x7fcaf40d0cf0 2024-05-08T05:47:47.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.165+0000 7fcafafe4640 1 -- 172.21.15.45:0/2574622804 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf41f9320 con 0x7fcaf40d0cf0 2024-05-08T05:47:47.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.166+0000 7fcaf17fa640 1 -- 172.21.15.45:0/2574622804 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcae80079d0 con 0x7fcaf40d0cf0 2024-05-08T05:47:47.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.167+0000 7fcaf17fa640 1 --2- 172.21.15.45:0/2574622804 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcad80762d0 0x7fcad8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.167+0000 7fcaf17fa640 1 -- 172.21.15.45:0/2574622804 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7fcae8097c60 con 0x7fcaf40d0cf0 2024-05-08T05:47:47.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.167+0000 7fcaf3fff640 1 --2- 172.21.15.45:0/2574622804 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcad80762d0 0x7fcad8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.167+0000 7fcafdddb740 1 Processor -- start 2024-05-08T05:47:47.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.167+0000 7fcafdddb740 1 -- start start 2024-05-08T05:47:47.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.167+0000 7fcaf3fff640 1 --2- 172.21.15.45:0/2574622804 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcad80762d0 0x7fcad8078790 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcaf40a1da0 tx=0x7fcae0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.168+0000 7fcafdddb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06426d0 0x55dbe05f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.168+0000 7fcafdddb740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe06257f0 0x55dbe0624b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.168+0000 7fcafdddb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06250b0 0x55dbe05f9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.168+0000 7fcafdddb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dbe04e5920 con 0x55dbe06250b0 2024-05-08T05:47:47.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.168+0000 7fcafdddb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dbe055add0 con 0x55dbe06426d0 2024-05-08T05:47:47.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.168+0000 7fcafdddb740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55dbe050b3e0 con 0x55dbe06257f0 2024-05-08T05:47:47.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.168+0000 7fcaf9581640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06250b0 0x55dbe05f9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.168+0000 7fcaf9581640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06250b0 0x55dbe05f9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36448/0 (socket says 172.21.15.45:36448) 2024-05-08T05:47:47.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.168+0000 7fcaf9581640 1 -- 172.21.15.45:0/401557677 learned_addr learned my addr 172.21.15.45:0/401557677 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.168+0000 7fcaf8d80640 1 --2- 172.21.15.45:0/401557677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06426d0 0x55dbe05f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.168+0000 7fcaf3fff640 1 --2- 172.21.15.45:0/401557677 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe06257f0 0x55dbe0624b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.169+0000 7fcaf9581640 1 -- 172.21.15.45:0/401557677 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe06257f0 msgr2=0x55dbe0624b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.169+0000 7fcaf9581640 1 --2- 172.21.15.45:0/401557677 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe06257f0 0x55dbe0624b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.169+0000 7fcaf9581640 1 -- 172.21.15.45:0/401557677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06426d0 msgr2=0x55dbe05f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.169+0000 7fcaf9581640 1 --2- 172.21.15.45:0/401557677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06426d0 0x55dbe05f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.169+0000 7fcaf9581640 1 -- 172.21.15.45:0/401557677 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbe033c7c0 con 0x55dbe06250b0 2024-05-08T05:47:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.169+0000 7fcaf9581640 1 --2- 172.21.15.45:0/401557677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06250b0 0x55dbe05f9ec0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fcaec02ac90 tx=0x7fcaec02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e19b05738aad4db server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.169+0000 7fcade7fc640 1 -- 172.21.15.45:0/401557677 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec036040 con 0x55dbe06250b0 2024-05-08T05:47:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.169+0000 7fcade7fc640 1 -- 172.21.15.45:0/401557677 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaec031680 con 0x55dbe06250b0 2024-05-08T05:47:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.169+0000 7fcade7fc640 1 -- 172.21.15.45:0/401557677 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec031a30 con 0x55dbe06250b0 2024-05-08T05:47:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.169+0000 7fcafdddb740 1 -- 172.21.15.45:0/401557677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06250b0 msgr2=0x55dbe05f9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.169+0000 7fcafdddb740 1 --2- 172.21.15.45:0/401557677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06250b0 0x55dbe05f9ec0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fcaec02ac90 tx=0x7fcaec02e000 comp rx=0 tx=0).stop 2024-05-08T05:47:47.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.170+0000 7fcafdddb740 1 -- 172.21.15.45:0/401557677 shutdown_connections 2024-05-08T05:47:47.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.170+0000 7fcafdddb740 1 --2- 172.21.15.45:0/401557677 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe06257f0 0x55dbe0624b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.170+0000 7fcafdddb740 1 --2- 172.21.15.45:0/401557677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06250b0 0x55dbe05f9ec0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.170+0000 7fcafdddb740 1 --2- 172.21.15.45:0/401557677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06426d0 0x55dbe05f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.170+0000 7fcafdddb740 1 -- 172.21.15.45:0/401557677 >> 172.21.15.45:0/401557677 conn(0x55dbe05f7560 msgr2=0x55dbe063fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.170+0000 7fcafdddb740 1 -- 172.21.15.45:0/401557677 shutdown_connections 2024-05-08T05:47:47.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.170+0000 7fcafdddb740 1 -- 172.21.15.45:0/401557677 wait complete. 2024-05-08T05:47:47.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.170+0000 7fcafdddb740 1 -- start start 2024-05-08T05:47:47.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.170+0000 7fcafdddb740 1 Processor -- start 2024-05-08T05:47:47.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.171+0000 7fcafdddb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06257f0 0x55dbe0624ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.171+0000 7fcafdddb740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe06426d0 0x55dbe06278d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.171+0000 7fcafdddb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe0627e10 0x55dbe063fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.171+0000 7fcafdddb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dbe0563440 con 0x55dbe0627e10 2024-05-08T05:47:47.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.171+0000 7fcafdddb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dbe055e120 con 0x55dbe06257f0 2024-05-08T05:47:47.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.171+0000 7fcafdddb740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55dbe056c210 con 0x55dbe06426d0 2024-05-08T05:47:47.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.171+0000 7fcafdddb740 10 client.0 _ll_register_callbacks cb 0x55dbe0503690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:47:47.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.171+0000 7fcaf9581640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe0627e10 0x55dbe063fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.171+0000 7fcaf9581640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe0627e10 0x55dbe063fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36456/0 (socket says 172.21.15.45:36456) 2024-05-08T05:47:47.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.171+0000 7fcaf9581640 1 -- 172.21.15.45:0/2906449183 learned_addr learned my addr 172.21.15.45:0/2906449183 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:47.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.171+0000 7fcaf9581640 1 -- 172.21.15.45:0/2906449183 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe06426d0 msgr2=0x55dbe06278d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.171+0000 7fcaf3fff640 1 --2- 172.21.15.45:0/2906449183 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe06426d0 0x55dbe06278d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.171+0000 7fcaf8d80640 1 --2- 172.21.15.45:0/2906449183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06257f0 0x55dbe0624ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcaf9581640 1 --2- 172.21.15.45:0/2906449183 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe06426d0 0x55dbe06278d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcaf9581640 1 -- 172.21.15.45:0/2906449183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06257f0 msgr2=0x55dbe0624ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcaf9581640 1 --2- 172.21.15.45:0/2906449183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06257f0 0x55dbe0624ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcaf9581640 1 -- 172.21.15.45:0/2906449183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbe039cf50 con 0x55dbe0627e10 2024-05-08T05:47:47.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcaf3fff640 1 --2- 172.21.15.45:0/2906449183 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe06426d0 0x55dbe06278d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:47:47.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcaf8d80640 1 --2- 172.21.15.45:0/2906449183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06257f0 0x55dbe0624ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:47:47.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcaf9581640 1 --2- 172.21.15.45:0/2906449183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe0627e10 0x55dbe063fb20 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0093e0 tx=0x7fcaec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536d5ccb141a0fda server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcad37fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:47:47.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcad37fe640 1 -- 172.21.15.45:0/2906449183 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec04a070 con 0x55dbe0627e10 2024-05-08T05:47:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcad37fe640 1 -- 172.21.15.45:0/2906449183 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7fcaec031a40 con 0x55dbe0627e10 2024-05-08T05:47:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcafdddb740 1 -- 172.21.15.45:0/2906449183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaec031680 con 0x55dbe0627e10 2024-05-08T05:47:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcad37fe640 1 -- 172.21.15.45:0/2906449183 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fcaec055050 con 0x55dbe0627e10 2024-05-08T05:47:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcafdddb740 10 client.7062 fetch_fsmap learned FSMap version 67 2024-05-08T05:47:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcafdddb740 1 -- 172.21.15.45:0/2906449183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55dbe02300b0 con 0x55dbe0627e10 2024-05-08T05:47:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcad37fe640 1 -- 172.21.15.45:0/2906449183 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaec031680 con 0x55dbe0627e10 2024-05-08T05:47:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcad37fe640 1 -- 172.21.15.45:0/2906449183 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fcaec058020 con 0x55dbe0627e10 2024-05-08T05:47:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcafdddb740 10 client.7062 fetch_fsmap finished waiting for FSMap version 67 2024-05-08T05:47:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcafdddb740 10 client.7062 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:47:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcafdddb740 20 client.7062 populate_metadata read hostname 'smithi045' 2024-05-08T05:47:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcafdddb740 1 --2- 172.21.15.45:0/2906449183 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55dbe0622ae0 0x55dbe0622ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcafdddb740 4 client.7062 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:47:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.172+0000 7fcafdddb740 1 -- 172.21.15.45:0/2906449183 --> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbe056ca20 con 0x55dbe0622ae0 2024-05-08T05:47:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.173+0000 7fcaf8d80640 1 --2- 172.21.15.45:0/2906449183 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55dbe0622ae0 0x55dbe0622ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.173+0000 7fcad37fe640 10 client.7062 ms_handle_connect on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.173+0000 7fcaf8d80640 1 --2- 172.21.15.45:0/2906449183 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55dbe0622ae0 0x55dbe0622ec0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=776db5342972016d server_cookie=5486a17cd38f026f in_seq=0 out_seq=0 2024-05-08T05:47:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.174+0000 7fcad37fe640 1 -- 172.21.15.45:0/2906449183 <== mds.0 v2:172.21.15.63:6835/3194497846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dbe056ca20 con 0x55dbe0622ae0 2024-05-08T05:47:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.174+0000 7fcad37fe640 10 client.7062 handle_command_reply: tid=0 2024-05-08T05:47:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.174+0000 7fcafdddb740 1 client.7062 shutdown 2024-05-08T05:47:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.174+0000 7fcafdddb740 1 -- 172.21.15.45:0/2906449183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe0627e10 msgr2=0x55dbe063fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.174+0000 7fcafdddb740 1 --2- 172.21.15.45:0/2906449183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe0627e10 0x55dbe063fb20 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0093e0 tx=0x7fcaec004ab0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.174+0000 7fcafdddb740 1 -- 172.21.15.45:0/2906449183 shutdown_connections 2024-05-08T05:47:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.174+0000 7fcafdddb740 1 --2- 172.21.15.45:0/2906449183 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55dbe0622ae0 0x55dbe0622ec0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.174+0000 7fcafdddb740 1 --2- 172.21.15.45:0/2906449183 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe06426d0 0x55dbe06278d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.174+0000 7fcafdddb740 1 --2- 172.21.15.45:0/2906449183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe0627e10 0x55dbe063fb20 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.174+0000 7fcad37fe640 0 client.7062 ms_handle_reset on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.174+0000 7fcafdddb740 1 --2- 172.21.15.45:0/2906449183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06257f0 0x55dbe0624ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.174+0000 7fcafdddb740 1 -- 172.21.15.45:0/2906449183 >> 172.21.15.45:0/2906449183 conn(0x55dbe06250b0 msgr2=0x55dbe05f4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.174+0000 7fcafdddb740 1 -- 172.21.15.45:0/2906449183 shutdown_connections 2024-05-08T05:47:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.174+0000 7fcafdddb740 1 -- 172.21.15.45:0/2906449183 wait complete. 2024-05-08T05:47:47.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.174+0000 7fcafdddb740 20 client.7062 trim_cache size 0 max 16384 2024-05-08T05:47:47.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.188+0000 7fcafdddb740 1 Processor -- start 2024-05-08T05:47:47.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.188+0000 7fcafdddb740 1 -- start start 2024-05-08T05:47:47.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.188+0000 7fcafdddb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06257f0 0x55dbe05f8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.188+0000 7fcafdddb740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe0627e10 0x55dbe05f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.188+0000 7fcafdddb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06426d0 0x55dbe0622ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.188+0000 7fcafdddb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf4200ff0 con 0x55dbe06257f0 2024-05-08T05:47:47.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.188+0000 7fcafdddb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf40a06f0 con 0x55dbe06426d0 2024-05-08T05:47:47.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.188+0000 7fcafdddb740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf40a0d40 con 0x55dbe0627e10 2024-05-08T05:47:47.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.189+0000 7fcaf8d80640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06257f0 0x55dbe05f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.189+0000 7fcaf8d80640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06257f0 0x55dbe05f8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36458/0 (socket says 172.21.15.45:36458) 2024-05-08T05:47:47.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.189+0000 7fcaf8d80640 1 -- 172.21.15.45:0/64695645 learned_addr learned my addr 172.21.15.45:0/64695645 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:47.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.189+0000 7fcaf3fff640 1 --2- 172.21.15.45:0/64695645 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe0627e10 0x55dbe05f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.189+0000 7fcaf9581640 1 --2- 172.21.15.45:0/64695645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06426d0 0x55dbe0622ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.189+0000 7fcaf8d80640 1 -- 172.21.15.45:0/64695645 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe0627e10 msgr2=0x55dbe05f8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.189+0000 7fcaf8d80640 1 --2- 172.21.15.45:0/64695645 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe0627e10 0x55dbe05f8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.189+0000 7fcaf8d80640 1 -- 172.21.15.45:0/64695645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06426d0 msgr2=0x55dbe0622ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.189+0000 7fcaf8d80640 1 --2- 172.21.15.45:0/64695645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06426d0 0x55dbe0622ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.189+0000 7fcaf8d80640 1 -- 172.21.15.45:0/64695645 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbe05efd40 con 0x55dbe06257f0 2024-05-08T05:47:47.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.189+0000 7fcaf9581640 1 --2- 172.21.15.45:0/64695645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06426d0 0x55dbe0622ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:47:47.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.189+0000 7fcaf8d80640 1 --2- 172.21.15.45:0/64695645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06257f0 0x55dbe05f8870 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fcae80608d0 tx=0x7fcae8063040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9776cddd93b2416f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.189+0000 7fcaddffb640 1 -- 172.21.15.45:0/64695645 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae801e070 con 0x55dbe06257f0 2024-05-08T05:47:47.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.189+0000 7fcaddffb640 1 -- 172.21.15.45:0/64695645 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae8094070 con 0x55dbe06257f0 2024-05-08T05:47:47.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.189+0000 7fcaddffb640 1 -- 172.21.15.45:0/64695645 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae80612d0 con 0x55dbe06257f0 2024-05-08T05:47:47.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.190+0000 7fcafdddb740 1 -- 172.21.15.45:0/64695645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06257f0 msgr2=0x55dbe05f8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.190+0000 7fcafdddb740 1 --2- 172.21.15.45:0/64695645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06257f0 0x55dbe05f8870 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fcae80608d0 tx=0x7fcae8063040 comp rx=0 tx=0).stop 2024-05-08T05:47:47.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.190+0000 7fcafdddb740 1 -- 172.21.15.45:0/64695645 shutdown_connections 2024-05-08T05:47:47.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.190+0000 7fcafdddb740 1 --2- 172.21.15.45:0/64695645 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe0627e10 0x55dbe05f8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.190+0000 7fcafdddb740 1 --2- 172.21.15.45:0/64695645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06426d0 0x55dbe0622ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.190+0000 7fcafdddb740 1 --2- 172.21.15.45:0/64695645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06257f0 0x55dbe05f8870 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.190+0000 7fcafdddb740 1 -- 172.21.15.45:0/64695645 >> 172.21.15.45:0/64695645 conn(0x55dbe06250b0 msgr2=0x55dbe05f7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.190+0000 7fcafdddb740 1 -- 172.21.15.45:0/64695645 shutdown_connections 2024-05-08T05:47:47.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.190+0000 7fcafdddb740 1 -- 172.21.15.45:0/64695645 wait complete. 2024-05-08T05:47:47.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.190+0000 7fcafdddb740 1 -- start start 2024-05-08T05:47:47.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.190+0000 7fcafdddb740 1 Processor -- start 2024-05-08T05:47:47.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.191+0000 7fcafdddb740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06257f0 0x55dbe0621610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.191+0000 7fcafdddb740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe0627e10 0x55dbe0621b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.191+0000 7fcafdddb740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06426d0 0x55dbe06629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.191+0000 7fcafdddb740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dbe06285e0 con 0x55dbe06426d0 2024-05-08T05:47:47.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.191+0000 7fcafdddb740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dbe0514a20 con 0x55dbe06257f0 2024-05-08T05:47:47.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.191+0000 7fcafdddb740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55dbe0625e00 con 0x55dbe0627e10 2024-05-08T05:47:47.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.191+0000 7fcafdddb740 10 client.0 _ll_register_callbacks cb 0x7fcaec009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:47:47.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.191+0000 7fcaf9581640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06426d0 0x55dbe06629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.191+0000 7fcaf9581640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06426d0 0x55dbe06629d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36466/0 (socket says 172.21.15.45:36466) 2024-05-08T05:47:47.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.191+0000 7fcaf9581640 1 -- 172.21.15.45:0/4143000977 learned_addr learned my addr 172.21.15.45:0/4143000977 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:47.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.191+0000 7fcaf3fff640 1 --2- 172.21.15.45:0/4143000977 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe0627e10 0x55dbe0621b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.191+0000 7fcaf8d80640 1 --2- 172.21.15.45:0/4143000977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06257f0 0x55dbe0621610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.192+0000 7fcaf9581640 1 -- 172.21.15.45:0/4143000977 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe0627e10 msgr2=0x55dbe0621b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.192+0000 7fcaf9581640 1 --2- 172.21.15.45:0/4143000977 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe0627e10 0x55dbe0621b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.192+0000 7fcaf9581640 1 -- 172.21.15.45:0/4143000977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06257f0 msgr2=0x55dbe0621610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.192+0000 7fcaf9581640 1 --2- 172.21.15.45:0/4143000977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06257f0 0x55dbe0621610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.192+0000 7fcaf9581640 1 -- 172.21.15.45:0/4143000977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbe0663aa0 con 0x55dbe06426d0 2024-05-08T05:47:47.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.192+0000 7fcaf9581640 1 --2- 172.21.15.45:0/4143000977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06426d0 0x55dbe06629d0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fcaec031800 tx=0x7fcaec009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c90ae3cab3a33c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.192+0000 7fcad37fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:47:47.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.192+0000 7fcad37fe640 1 -- 172.21.15.45:0/4143000977 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec05e060 con 0x55dbe06426d0 2024-05-08T05:47:47.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.192+0000 7fcad37fe640 1 -- 172.21.15.45:0/4143000977 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7fcaec03c860 con 0x55dbe06426d0 2024-05-08T05:47:47.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.192+0000 7fcafdddb740 1 -- 172.21.15.45:0/4143000977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcae8094070 con 0x55dbe06426d0 2024-05-08T05:47:47.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.192+0000 7fcad37fe640 1 -- 172.21.15.45:0/4143000977 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fcaec064050 con 0x55dbe06426d0 2024-05-08T05:47:47.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.192+0000 7fcafdddb740 10 client.7068 fetch_fsmap learned FSMap version 67 2024-05-08T05:47:47.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.192+0000 7fcafdddb740 1 -- 172.21.15.45:0/4143000977 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55dbe065a0a0 con 0x55dbe06426d0 2024-05-08T05:47:47.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.192+0000 7fcad37fe640 1 -- 172.21.15.45:0/4143000977 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae8094070 con 0x55dbe06426d0 2024-05-08T05:47:47.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.192+0000 7fcad37fe640 1 -- 172.21.15.45:0/4143000977 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7fcaec067020 con 0x55dbe06426d0 2024-05-08T05:47:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.192+0000 7fcafdddb740 10 client.7068 fetch_fsmap finished waiting for FSMap version 67 2024-05-08T05:47:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.193+0000 7fcafdddb740 10 client.7068 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:47:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.193+0000 7fcafdddb740 20 client.7068 populate_metadata read hostname 'smithi045' 2024-05-08T05:47:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.193+0000 7fcafdddb740 1 --2- 172.21.15.45:0/4143000977 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55dbe0663300 0x55dbe0635500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.193+0000 7fcafdddb740 4 client.7068 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-08T05:47:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.193+0000 7fcafdddb740 1 -- 172.21.15.45:0/4143000977 --> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55dbe0566890 con 0x55dbe0663300 2024-05-08T05:47:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.193+0000 7fcaf8d80640 1 --2- 172.21.15.45:0/4143000977 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55dbe0663300 0x55dbe0635500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.193+0000 7fcad37fe640 10 client.7068 ms_handle_connect on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.193+0000 7fcaf8d80640 1 --2- 172.21.15.45:0/4143000977 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55dbe0663300 0x55dbe0635500 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2a9f95e9fc03545 server_cookie=6c477c6261521349 in_seq=0 out_seq=0 2024-05-08T05:47:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.193+0000 7fcad37fe640 1 -- 172.21.15.45:0/4143000977 <== mds.0 v2:172.21.15.63:6835/3194497846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+420 (crc 0 0 0) 0x55dbe0566890 con 0x55dbe0663300 2024-05-08T05:47:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.193+0000 7fcad37fe640 10 client.7068 handle_command_reply: tid=0 2024-05-08T05:47:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.193+0000 7fcafdddb740 1 client.7068 shutdown 2024-05-08T05:47:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafdddb740 1 -- 172.21.15.45:0/4143000977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06426d0 msgr2=0x55dbe06629d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafdddb740 1 --2- 172.21.15.45:0/4143000977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06426d0 0x55dbe06629d0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fcaec031800 tx=0x7fcaec009000 comp rx=0 tx=0).stop 2024-05-08T05:47:47.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafdddb740 1 -- 172.21.15.45:0/4143000977 shutdown_connections 2024-05-08T05:47:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafdddb740 1 --2- 172.21.15.45:0/4143000977 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55dbe0663300 0x55dbe0635500 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafdddb740 1 --2- 172.21.15.45:0/4143000977 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dbe0627e10 0x55dbe0621b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafdddb740 1 --2- 172.21.15.45:0/4143000977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dbe06426d0 0x55dbe06629d0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcad37fe640 0 client.7068 ms_handle_reset on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafdddb740 1 --2- 172.21.15.45:0/4143000977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dbe06257f0 0x55dbe0621610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafdddb740 1 -- 172.21.15.45:0/4143000977 >> 172.21.15.45:0/4143000977 conn(0x55dbe06250b0 msgr2=0x55dbe0622ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafdddb740 1 -- 172.21.15.45:0/4143000977 shutdown_connections 2024-05-08T05:47:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafdddb740 1 -- 172.21.15.45:0/4143000977 wait complete. 2024-05-08T05:47:47.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafdddb740 20 client.7068 trim_cache size 0 max 16384 2024-05-08T05:47:47.201 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:47:47.201 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-05-08T05:47:47.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:47:47.202 INFO:teuthology.orchestra.run.smithi045.stdout: "damage_type": "dir_frag", 2024-05-08T05:47:47.202 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 2682458941, 2024-05-08T05:47:47.202 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 1099511627776, 2024-05-08T05:47:47.202 INFO:teuthology.orchestra.run.smithi045.stdout: "frag": "*", 2024-05-08T05:47:47.202 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/dir" 2024-05-08T05:47:47.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:47:47.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:47:47.202 INFO:teuthology.orchestra.run.smithi045.stdout: "damage_type": "backtrace", 2024-05-08T05:47:47.202 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 3259427672, 2024-05-08T05:47:47.202 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 1099511627777, 2024-05-08T05:47:47.202 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/testdir/hardlink" 2024-05-08T05:47:47.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:47:47.202 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:47:47.202 INFO:teuthology.orchestra.run.smithi045.stdout: "damage_type": "backtrace", 2024-05-08T05:47:47.203 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 3974101289, 2024-05-08T05:47:47.203 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 1099511627776, 2024-05-08T05:47:47.203 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/dir" 2024-05-08T05:47:47.203 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:47:47.203 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-05-08T05:47:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafafe4640 1 -- 172.21.15.45:0/2574622804 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcad80762d0 msgr2=0x7fcad8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafafe4640 1 --2- 172.21.15.45:0/2574622804 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcad80762d0 0x7fcad8078790 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcaf40a1da0 tx=0x7fcae0008040 comp rx=0 tx=0).stop 2024-05-08T05:47:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafafe4640 1 -- 172.21.15.45:0/2574622804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaf40d0cf0 msgr2=0x7fcaf40a2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafafe4640 1 --2- 172.21.15.45:0/2574622804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaf40d0cf0 0x7fcaf40a2750 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fcae800c7f0 tx=0x7fcae800ccb0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafafe4640 1 -- 172.21.15.45:0/2574622804 shutdown_connections 2024-05-08T05:47:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafafe4640 1 --2- 172.21.15.45:0/2574622804 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcad80762d0 0x7fcad8078790 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafafe4640 1 --2- 172.21.15.45:0/2574622804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcaf415f380 0x7fcaf40ba360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafafe4640 1 --2- 172.21.15.45:0/2574622804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcaf40d3b00 0x7fcaf415cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafafe4640 1 --2- 172.21.15.45:0/2574622804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcaf40d0cf0 0x7fcaf40a2750 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafafe4640 1 -- 172.21.15.45:0/2574622804 >> 172.21.15.45:0/2574622804 conn(0x7fcaf40d4fe0 msgr2=0x7fcaf40cb920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafafe4640 1 -- 172.21.15.45:0/2574622804 shutdown_connections 2024-05-08T05:47:47.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.194+0000 7fcafafe4640 1 -- 172.21.15.45:0/2574622804 wait complete. 2024-05-08T05:47:47.206 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T05:47:47.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.308+0000 7fced38da640 1 Processor -- start 2024-05-08T05:47:47.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.308+0000 7fced38da640 1 -- start start 2024-05-08T05:47:47.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.309+0000 7fced38da640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcecc0d1970 0x7fcecc0d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.309+0000 7fced38da640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcecc0d4cc0 0x7fcecc0d50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.309+0000 7fced38da640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcecc1630b0 con 0x7fcecc15f420 2024-05-08T05:47:47.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.309+0000 7fced38da640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fcecc0d3e70 con 0x7fcecc0d1970 2024-05-08T05:47:47.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.309+0000 7fced38da640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fcecc0d2550 con 0x7fcecc0d4cc0 2024-05-08T05:47:47.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.309+0000 7fced0e75640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcecc0d1970 0x7fcecc0d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.309+0000 7fced0e75640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcecc0d1970 0x7fcecc0d1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36478/0 (socket says 172.21.15.45:36478) 2024-05-08T05:47:47.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.309+0000 7fced0e75640 1 -- 172.21.15.45:0/3398345081 learned_addr learned my addr 172.21.15.45:0/3398345081 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:47.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.309+0000 7fced1676640 1 --1- 172.21.15.45:0/3398345081 >> v1:172.21.15.45:6789/0 conn(0x7fcecc15f420 0x7fcecc0d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41064/0 (socket says 172.21.15.45:41064) 2024-05-08T05:47:47.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.309+0000 7fced1e77640 1 --2- 172.21.15.45:0/3398345081 >> v2:172.21.15.63:3300/0 conn(0x7fcecc0d4cc0 0x7fcecc0d50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.309+0000 7fced0e75640 1 -- 172.21.15.45:0/3398345081 >> v2:172.21.15.63:3300/0 conn(0x7fcecc0d4cc0 msgr2=0x7fcecc0d50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.309+0000 7fced0e75640 1 --2- 172.21.15.45:0/3398345081 >> v2:172.21.15.63:3300/0 conn(0x7fcecc0d4cc0 0x7fcecc0d50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.309+0000 7fced0e75640 1 -- 172.21.15.45:0/3398345081 >> v1:172.21.15.45:6789/0 conn(0x7fcecc15f420 legacy=0x7fcecc0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.309+0000 7fced0e75640 1 -- 172.21.15.45:0/3398345081 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcecc0d2c80 con 0x7fcecc0d1970 2024-05-08T05:47:47.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.310+0000 7fced0e75640 1 --2- 172.21.15.45:0/3398345081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcecc0d1970 0x7fcecc0d1e20 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0d17a0 tx=0x7fcebc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af63398079271b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.310+0000 7fcec3fff640 1 -- 172.21.15.45:0/3398345081 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc037070 con 0x7fcecc0d1970 2024-05-08T05:47:47.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.310+0000 7fcec3fff640 1 -- 172.21.15.45:0/3398345081 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcebc032440 con 0x7fcecc0d1970 2024-05-08T05:47:47.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.310+0000 7fcec3fff640 1 -- 172.21.15.45:0/3398345081 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc032810 con 0x7fcecc0d1970 2024-05-08T05:47:47.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.310+0000 7fced38da640 1 -- 172.21.15.45:0/3398345081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcecc0d1970 msgr2=0x7fcecc0d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.310+0000 7fced38da640 1 --2- 172.21.15.45:0/3398345081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcecc0d1970 0x7fcecc0d1e20 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0d17a0 tx=0x7fcebc02a270 comp rx=0 tx=0).stop 2024-05-08T05:47:47.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.311+0000 7fced38da640 1 -- 172.21.15.45:0/3398345081 shutdown_connections 2024-05-08T05:47:47.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.311+0000 7fced38da640 1 --2- 172.21.15.45:0/3398345081 >> v2:172.21.15.63:3300/0 conn(0x7fcecc0d4cc0 0x7fcecc0d50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.311+0000 7fced38da640 1 --2- 172.21.15.45:0/3398345081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcecc0d1970 0x7fcecc0d1e20 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.311+0000 7fced38da640 1 -- 172.21.15.45:0/3398345081 >> 172.21.15.45:0/3398345081 conn(0x7fcecc08eac0 msgr2=0x7fcecc09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.311+0000 7fced38da640 1 -- 172.21.15.45:0/3398345081 shutdown_connections 2024-05-08T05:47:47.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.311+0000 7fced38da640 1 -- 172.21.15.45:0/3398345081 wait complete. 2024-05-08T05:47:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.312+0000 7fced38da640 1 Processor -- start 2024-05-08T05:47:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.312+0000 7fced38da640 1 -- start start 2024-05-08T05:47:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fced38da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcecc0d1970 0x7fcecc0ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fced38da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcecc0d4cc0 0x7fcecc151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fced38da640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcecc0cd8f0 0x7fcecc0cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fced38da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc0d2830 con 0x7fcecc0d1970 2024-05-08T05:47:47.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fced38da640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc0d3240 con 0x7fcecc0cd8f0 2024-05-08T05:47:47.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fced38da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcecc0d5620 con 0x7fcecc0d4cc0 2024-05-08T05:47:47.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fced1676640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcecc0d1970 0x7fcecc0ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fced1676640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcecc0d1970 0x7fcecc0ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36482/0 (socket says 172.21.15.45:36482) 2024-05-08T05:47:47.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fced1676640 1 -- 172.21.15.45:0/602233403 learned_addr learned my addr 172.21.15.45:0/602233403 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:47.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fced1e77640 1 --2- 172.21.15.45:0/602233403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcecc0cd8f0 0x7fcecc0cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fced0e75640 1 --2- 172.21.15.45:0/602233403 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcecc0d4cc0 0x7fcecc151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fced1676640 1 -- 172.21.15.45:0/602233403 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcecc0d4cc0 msgr2=0x7fcecc151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fced1676640 1 --2- 172.21.15.45:0/602233403 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcecc0d4cc0 0x7fcecc151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fced1676640 1 -- 172.21.15.45:0/602233403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcecc0cd8f0 msgr2=0x7fcecc0cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fced1676640 1 --2- 172.21.15.45:0/602233403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcecc0cd8f0 0x7fcecc0cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fced1676640 1 -- 172.21.15.45:0/602233403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcecc0cc850 con 0x7fcecc0d1970 2024-05-08T05:47:47.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fced1e77640 1 --2- 172.21.15.45:0/602233403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcecc0cd8f0 0x7fcecc0cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:47.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fced1676640 1 --2- 172.21.15.45:0/602233403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcecc0d1970 0x7fcecc0ca7c0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fcec800c7a0 tx=0x7fcec800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fcec1ffb640 1 -- 172.21.15.45:0/602233403 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8017070 con 0x7fcecc0d1970 2024-05-08T05:47:47.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fcec1ffb640 1 -- 172.21.15.45:0/602233403 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec800a640 con 0x7fcecc0d1970 2024-05-08T05:47:47.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fced38da640 1 -- 172.21.15.45:0/602233403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcecc0ccb30 con 0x7fcecc0d1970 2024-05-08T05:47:47.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.313+0000 7fcec1ffb640 1 -- 172.21.15.45:0/602233403 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec800a9f0 con 0x7fcecc0d1970 2024-05-08T05:47:47.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.314+0000 7fced38da640 1 -- 172.21.15.45:0/602233403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcecc0cd0f0 con 0x7fcecc0d1970 2024-05-08T05:47:47.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.315+0000 7fced38da640 1 -- 172.21.15.45:0/602233403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcecc0693f0 con 0x7fcecc0d1970 2024-05-08T05:47:47.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.315+0000 7fcec1ffb640 1 -- 172.21.15.45:0/602233403 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcec801a070 con 0x7fcecc0d1970 2024-05-08T05:47:47.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.315+0000 7fcec1ffb640 1 --2- 172.21.15.45:0/602233403 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fceb00762d0 0x7fceb0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.316+0000 7fcec1ffb640 1 -- 172.21.15.45:0/602233403 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7fcec8096d90 con 0x7fcecc0d1970 2024-05-08T05:47:47.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.316+0000 7fced0e75640 1 --2- 172.21.15.45:0/602233403 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fceb00762d0 0x7fceb0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.316+0000 7fced0e75640 1 --2- 172.21.15.45:0/602233403 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fceb00762d0 0x7fceb0078790 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fcebc034000 tx=0x7fcebc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.320+0000 7fcec1ffb640 1 -- 172.21.15.45:0/602233403 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcecc1630b0 con 0x7fcecc0d1970 2024-05-08T05:47:47.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.551+0000 7fced38da640 1 -- 172.21.15.45:0/602233403 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fcecc1638c0 con 0x7fcecc0d1970 2024-05-08T05:47:47.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.552+0000 7fcec1ffb640 1 -- 172.21.15.45:0/602233403 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7fcec805fe50 con 0x7fcecc0d1970 2024-05-08T05:47:47.553 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:47:47.553 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-05-08T05:47:47.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.554+0000 7fced38da640 1 -- 172.21.15.45:0/602233403 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fceb00762d0 msgr2=0x7fceb0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.554+0000 7fced38da640 1 --2- 172.21.15.45:0/602233403 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fceb00762d0 0x7fceb0078790 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fcebc034000 tx=0x7fcebc02e000 comp rx=0 tx=0).stop 2024-05-08T05:47:47.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.554+0000 7fced38da640 1 -- 172.21.15.45:0/602233403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcecc0d1970 msgr2=0x7fcecc0ca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.554+0000 7fced38da640 1 --2- 172.21.15.45:0/602233403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcecc0d1970 0x7fcecc0ca7c0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fcec800c7a0 tx=0x7fcec800cc60 comp rx=0 tx=0).stop 2024-05-08T05:47:47.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.554+0000 7fced38da640 1 -- 172.21.15.45:0/602233403 shutdown_connections 2024-05-08T05:47:47.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.554+0000 7fced38da640 1 --2- 172.21.15.45:0/602233403 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fceb00762d0 0x7fceb0078790 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.554+0000 7fced38da640 1 --2- 172.21.15.45:0/602233403 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcecc0d4cc0 0x7fcecc151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.554+0000 7fced38da640 1 --2- 172.21.15.45:0/602233403 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcecc0cd8f0 0x7fcecc0cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.554+0000 7fced38da640 1 --2- 172.21.15.45:0/602233403 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcecc0d1970 0x7fcecc0ca7c0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.554+0000 7fced38da640 1 -- 172.21.15.45:0/602233403 >> 172.21.15.45:0/602233403 conn(0x7fcecc08eac0 msgr2=0x7fcecc0b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.554+0000 7fced38da640 1 -- 172.21.15.45:0/602233403 shutdown_connections 2024-05-08T05:47:47.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.554+0000 7fced38da640 1 -- 172.21.15.45:0/602233403 wait complete. 2024-05-08T05:47:47.566 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-08T05:47:47.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:47.566 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-05-08T05:47:47.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.674+0000 7f3b206d1640 1 Processor -- start 2024-05-08T05:47:47.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.674+0000 7f3b206d1640 1 -- start start 2024-05-08T05:47:47.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.675+0000 7f3b206d1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b18159920 0x7f3b180ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.675+0000 7f3b206d1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3b180aeb90 0x7f3b180d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.675+0000 7f3b206d1640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b180afa00 con 0x7f3b180d0df0 2024-05-08T05:47:47.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.675+0000 7f3b206d1640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f3b180d1260 con 0x7f3b180aeb90 2024-05-08T05:47:47.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.675+0000 7f3b206d1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3b180af160 con 0x7f3b18159920 2024-05-08T05:47:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.675+0000 7f3b1dc6c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3b180aeb90 0x7f3b180d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.675+0000 7f3b1dc6c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3b180aeb90 0x7f3b180d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36488/0 (socket says 172.21.15.45:36488) 2024-05-08T05:47:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.675+0000 7f3b1dc6c640 1 -- 172.21.15.45:0/2305323729 learned_addr learned my addr 172.21.15.45:0/2305323729 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.675+0000 7f3b1ec6e640 1 --1- 172.21.15.45:0/2305323729 >> v1:172.21.15.45:6789/0 conn(0x7f3b180d0df0 0x7f3b180b4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41068/0 (socket says 172.21.15.45:41068) 2024-05-08T05:47:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.675+0000 7f3b1e46d640 1 --2- 172.21.15.45:0/2305323729 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b18159920 0x7f3b180ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.675+0000 7f3b1dc6c640 1 -- 172.21.15.45:0/2305323729 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b18159920 msgr2=0x7f3b180ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.675+0000 7f3b1dc6c640 1 --2- 172.21.15.45:0/2305323729 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b18159920 0x7f3b180ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.675+0000 7f3b1dc6c640 1 -- 172.21.15.45:0/2305323729 >> v1:172.21.15.45:6789/0 conn(0x7f3b180d0df0 legacy=0x7f3b180b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.675+0000 7f3b1dc6c640 1 -- 172.21.15.45:0/2305323729 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b180af440 con 0x7f3b180aeb90 2024-05-08T05:47:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.675+0000 7f3b1d46b640 1 -- 172.21.15.45:0/2305323729 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1911146937 0 0) 0x7f3b180afa00 con 0x7f3b180d0df0 2024-05-08T05:47:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.675+0000 7f3b1e46d640 1 --2- 172.21.15.45:0/2305323729 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b18159920 0x7f3b180ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.675+0000 7f3b1dc6c640 1 --2- 172.21.15.45:0/2305323729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b180aeb90 0x7f3b180d08b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009870 tx=0x7f3b0802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc318fa642db9bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.675+0000 7f3b1d46b640 1 -- 172.21.15.45:0/2305323729 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b08037070 con 0x7f3b180aeb90 2024-05-08T05:47:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.675+0000 7f3b1d46b640 1 -- 172.21.15.45:0/2305323729 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b08032440 con 0x7f3b180aeb90 2024-05-08T05:47:47.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.675+0000 7f3b1d46b640 1 -- 172.21.15.45:0/2305323729 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b08032810 con 0x7f3b180aeb90 2024-05-08T05:47:47.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.676+0000 7f3b206d1640 1 -- 172.21.15.45:0/2305323729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b180aeb90 msgr2=0x7f3b180d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.676+0000 7f3b206d1640 1 --2- 172.21.15.45:0/2305323729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b180aeb90 0x7f3b180d08b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009870 tx=0x7f3b0802a1c0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.676+0000 7f3b206d1640 1 -- 172.21.15.45:0/2305323729 shutdown_connections 2024-05-08T05:47:47.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.676+0000 7f3b206d1640 1 --2- 172.21.15.45:0/2305323729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b180aeb90 0x7f3b180d08b0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.676+0000 7f3b206d1640 1 --2- 172.21.15.45:0/2305323729 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b18159920 0x7f3b180ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.676+0000 7f3b206d1640 1 -- 172.21.15.45:0/2305323729 >> 172.21.15.45:0/2305323729 conn(0x7f3b1806c3a0 msgr2=0x7f3b1806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.676+0000 7f3b206d1640 1 -- 172.21.15.45:0/2305323729 shutdown_connections 2024-05-08T05:47:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.676+0000 7f3b206d1640 1 -- 172.21.15.45:0/2305323729 wait complete. 2024-05-08T05:47:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.678+0000 7f3b206d1640 1 Processor -- start 2024-05-08T05:47:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.678+0000 7f3b206d1640 1 -- start start 2024-05-08T05:47:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b206d1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b180aeb90 0x7f3b1809d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b206d1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b180d0df0 0x7f3b180b5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b206d1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b18159920 0x7f3b1809bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b206d1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3b18159d00 con 0x7f3b18159920 2024-05-08T05:47:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b206d1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1806e190 con 0x7f3b180d0df0 2024-05-08T05:47:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b206d1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3b1815c5e0 con 0x7f3b180aeb90 2024-05-08T05:47:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b1ec6e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b18159920 0x7f3b1809bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b1ec6e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b18159920 0x7f3b1809bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36496/0 (socket says 172.21.15.45:36496) 2024-05-08T05:47:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b1ec6e640 1 -- 172.21.15.45:0/1606499229 learned_addr learned my addr 172.21.15.45:0/1606499229 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b1ec6e640 1 -- 172.21.15.45:0/1606499229 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b180aeb90 msgr2=0x7f3b1809d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b1ec6e640 1 --2- 172.21.15.45:0/1606499229 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b180aeb90 0x7f3b1809d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b1dc6c640 1 --2- 172.21.15.45:0/1606499229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b180d0df0 0x7f3b180b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b1ec6e640 1 -- 172.21.15.45:0/1606499229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b180d0df0 msgr2=0x7f3b180b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b1ec6e640 1 --2- 172.21.15.45:0/1606499229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b180d0df0 0x7f3b180b5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b1ec6e640 1 -- 172.21.15.45:0/1606499229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b180b5b90 con 0x7f3b18159920 2024-05-08T05:47:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b1dc6c640 1 --2- 172.21.15.45:0/1606499229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b180d0df0 0x7f3b180b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:47:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b1ec6e640 1 --2- 172.21.15.45:0/1606499229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b18159920 0x7f3b1809bf70 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000c790 tx=0x7f3b1000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b0effd640 1 -- 172.21.15.45:0/1606499229 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10018070 con 0x7f3b18159920 2024-05-08T05:47:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b0effd640 1 -- 172.21.15.45:0/1606499229 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b1000a640 con 0x7f3b18159920 2024-05-08T05:47:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b206d1640 1 -- 172.21.15.45:0/1606499229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b1809e0d0 con 0x7f3b18159920 2024-05-08T05:47:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b0effd640 1 -- 172.21.15.45:0/1606499229 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1000aa10 con 0x7f3b18159920 2024-05-08T05:47:47.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.679+0000 7f3b206d1640 1 -- 172.21.15.45:0/1606499229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b1809e690 con 0x7f3b18159920 2024-05-08T05:47:47.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.680+0000 7f3b0effd640 1 -- 172.21.15.45:0/1606499229 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3b1001b070 con 0x7f3b18159920 2024-05-08T05:47:47.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.681+0000 7f3b0effd640 1 --2- 172.21.15.45:0/1606499229 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3afc0762d0 0x7f3afc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.681+0000 7f3b0effd640 1 -- 172.21.15.45:0/1606499229 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f3b10096e10 con 0x7f3b18159920 2024-05-08T05:47:47.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.681+0000 7f3b1e46d640 1 --2- 172.21.15.45:0/1606499229 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3afc0762d0 0x7f3afc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.682+0000 7f3b233ef740 1 Processor -- start 2024-05-08T05:47:47.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.682+0000 7f3b233ef740 1 -- start start 2024-05-08T05:47:47.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.682+0000 7f3b1e46d640 1 --2- 172.21.15.45:0/1606499229 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3afc0762d0 0x7f3afc078790 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3b14006e80 tx=0x7f3b14008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.682+0000 7f3b233ef740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c2fd765220 0x55c2fd717bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.682+0000 7f3b233ef740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c2fd748340 0x55c2fd7476c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.682+0000 7f3b233ef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd747c00 0x55c2fd71ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.682+0000 7f3b233ef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c2fd680cf0 con 0x55c2fd747c00 2024-05-08T05:47:47.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.682+0000 7f3b233ef740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c2fd639e10 con 0x55c2fd748340 2024-05-08T05:47:47.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.683+0000 7f3b233ef740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55c2fd67ccb0 con 0x55c2fd765220 2024-05-08T05:47:47.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.683+0000 7f3b1ec6e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd747c00 0x55c2fd71ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.683+0000 7f3b1ec6e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd747c00 0x55c2fd71ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36510/0 (socket says 172.21.15.45:36510) 2024-05-08T05:47:47.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.683+0000 7f3b1ec6e640 1 -- 172.21.15.45:0/361433002 learned_addr learned my addr 172.21.15.45:0/361433002 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:47.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.683+0000 7f3b1e46d640 1 --2- 172.21.15.45:0/361433002 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c2fd765220 0x55c2fd717bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.683+0000 7f3b1ec6e640 1 -- 172.21.15.45:0/361433002 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c2fd765220 msgr2=0x55c2fd717bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.683+0000 7f3b1ec6e640 1 --2- 172.21.15.45:0/361433002 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c2fd765220 0x55c2fd717bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.683+0000 7f3b1dc6c640 1 --2- 172.21.15.45:0/361433002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c2fd748340 0x55c2fd7476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.683+0000 7f3b1ec6e640 1 -- 172.21.15.45:0/361433002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c2fd748340 msgr2=0x55c2fd7476c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.683+0000 7f3b1ec6e640 1 --2- 172.21.15.45:0/361433002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c2fd748340 0x55c2fd7476c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.683+0000 7f3b1ec6e640 1 -- 172.21.15.45:0/361433002 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2fd45a7e0 con 0x55c2fd747c00 2024-05-08T05:47:47.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.683+0000 7f3b1ec6e640 1 --2- 172.21.15.45:0/361433002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd747c00 0x55c2fd71ca10 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3b10095090 tx=0x7f3b1005f590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591fb278fb7ee3fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.683+0000 7f3afbfff640 1 -- 172.21.15.45:0/361433002 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b10079380 con 0x55c2fd747c00 2024-05-08T05:47:47.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.683+0000 7f3afbfff640 1 -- 172.21.15.45:0/361433002 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b1005fe20 con 0x55c2fd747c00 2024-05-08T05:47:47.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.683+0000 7f3afbfff640 1 -- 172.21.15.45:0/361433002 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1000adf0 con 0x55c2fd747c00 2024-05-08T05:47:47.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.684+0000 7f3b233ef740 1 -- 172.21.15.45:0/361433002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd747c00 msgr2=0x55c2fd71ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.684+0000 7f3b233ef740 1 --2- 172.21.15.45:0/361433002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd747c00 0x55c2fd71ca10 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3b10095090 tx=0x7f3b1005f590 comp rx=0 tx=0).stop 2024-05-08T05:47:47.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.684+0000 7f3b233ef740 1 -- 172.21.15.45:0/361433002 shutdown_connections 2024-05-08T05:47:47.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.684+0000 7f3b233ef740 1 --2- 172.21.15.45:0/361433002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd747c00 0x55c2fd71ca10 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.684+0000 7f3b233ef740 1 --2- 172.21.15.45:0/361433002 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c2fd748340 0x55c2fd7476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.684+0000 7f3b233ef740 1 --2- 172.21.15.45:0/361433002 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c2fd765220 0x55c2fd717bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.684+0000 7f3b233ef740 1 -- 172.21.15.45:0/361433002 >> 172.21.15.45:0/361433002 conn(0x55c2fd719e20 msgr2=0x55c2fd7626c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.684+0000 7f3b233ef740 1 -- 172.21.15.45:0/361433002 shutdown_connections 2024-05-08T05:47:47.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.684+0000 7f3b233ef740 1 -- 172.21.15.45:0/361433002 wait complete. 2024-05-08T05:47:47.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.685+0000 7f3b233ef740 1 -- start start 2024-05-08T05:47:47.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.685+0000 7f3b233ef740 1 Processor -- start 2024-05-08T05:47:47.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.685+0000 7f3b233ef740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c2fd748340 0x55c2fd742cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.685+0000 7f3b233ef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd765220 0x55c2fd75a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.685+0000 7f3b233ef740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c2fd763710 0x55c2fd763b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.685+0000 7f3b233ef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c2fd680cf0 con 0x55c2fd765220 2024-05-08T05:47:47.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.685+0000 7f3b233ef740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c2fd63d940 con 0x55c2fd748340 2024-05-08T05:47:47.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.685+0000 7f3b233ef740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55c2fd71bfd0 con 0x55c2fd763710 2024-05-08T05:47:47.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.686+0000 7f3b233ef740 10 client.0 _ll_register_callbacks cb 0x55c2fd640440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:47:47.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.686+0000 7f3b1dc6c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd765220 0x55c2fd75a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.686+0000 7f3b1dc6c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd765220 0x55c2fd75a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36526/0 (socket says 172.21.15.45:36526) 2024-05-08T05:47:47.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.686+0000 7f3b1dc6c640 1 -- 172.21.15.45:0/3840825204 learned_addr learned my addr 172.21.15.45:0/3840825204 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:47.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.686+0000 7f3b1dc6c640 1 -- 172.21.15.45:0/3840825204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c2fd763710 msgr2=0x55c2fd763b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.686+0000 7f3b1dc6c640 1 --2- 172.21.15.45:0/3840825204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c2fd763710 0x55c2fd763b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.686+0000 7f3b1dc6c640 1 -- 172.21.15.45:0/3840825204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c2fd748340 msgr2=0x55c2fd742cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.686+0000 7f3b1dc6c640 1 --2- 172.21.15.45:0/3840825204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c2fd748340 0x55c2fd742cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.686+0000 7f3b1dc6c640 1 -- 172.21.15.45:0/3840825204 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2fd4baf70 con 0x55c2fd765220 2024-05-08T05:47:47.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.686+0000 7f3b1dc6c640 1 --2- 172.21.15.45:0/3840825204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd765220 0x55c2fd75a080 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f3b0802a200 tx=0x7f3b08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ade512b67fb704 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.686+0000 7f3af97fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:47:47.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.686+0000 7f3af97fa640 1 -- 172.21.15.45:0/3840825204 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b08037040 con 0x55c2fd765220 2024-05-08T05:47:47.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.686+0000 7f3af97fa640 1 -- 172.21.15.45:0/3840825204 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f3b0803e660 con 0x55c2fd765220 2024-05-08T05:47:47.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.686+0000 7f3b233ef740 1 -- 172.21.15.45:0/3840825204 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b1005fe20 con 0x55c2fd765220 2024-05-08T05:47:47.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.686+0000 7f3af97fa640 1 -- 172.21.15.45:0/3840825204 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f3b0804c050 con 0x55c2fd765220 2024-05-08T05:47:47.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.686+0000 7f3b233ef740 10 client.7092 fetch_fsmap learned FSMap version 67 2024-05-08T05:47:47.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.686+0000 7f3b233ef740 1 -- 172.21.15.45:0/3840825204 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55c2fd352170 con 0x55c2fd765220 2024-05-08T05:47:47.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.687+0000 7f3af97fa640 1 -- 172.21.15.45:0/3840825204 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b1005fe20 con 0x55c2fd765220 2024-05-08T05:47:47.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.687+0000 7f3af97fa640 1 -- 172.21.15.45:0/3840825204 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f3b0804f020 con 0x55c2fd765220 2024-05-08T05:47:47.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.687+0000 7f3b233ef740 10 client.7092 fetch_fsmap finished waiting for FSMap version 67 2024-05-08T05:47:47.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.687+0000 7f3b233ef740 10 client.7092 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-08T05:47:47.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.687+0000 7f3b233ef740 20 client.7092 populate_metadata read hostname 'smithi045' 2024-05-08T05:47:47.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.687+0000 7f3b233ef740 1 --2- 172.21.15.45:0/3840825204 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55c2fd748d00 0x55c2fd749160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.687+0000 7f3b233ef740 4 client.7092 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:47:47.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.687+0000 7f3b233ef740 1 -- 172.21.15.45:0/3840825204 --> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2fd68e900 con 0x55c2fd748d00 2024-05-08T05:47:47.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.687+0000 7f3b1e46d640 1 --2- 172.21.15.45:0/3840825204 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55c2fd748d00 0x55c2fd749160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.687+0000 7f3b1e46d640 1 --2- 172.21.15.45:0/3840825204 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55c2fd748d00 0x55c2fd749160 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b407dabc0f57074 server_cookie=b33e66f43ebba53a in_seq=0 out_seq=0 2024-05-08T05:47:47.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.687+0000 7f3af97fa640 10 client.7092 ms_handle_connect on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:47.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.688+0000 7f3af97fa640 1 -- 172.21.15.45:0/3840825204 <== mds.0 v2:172.21.15.63:6835/3194497846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c2fd68e900 con 0x55c2fd748d00 2024-05-08T05:47:47.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.688+0000 7f3af97fa640 10 client.7092 handle_command_reply: tid=0 2024-05-08T05:47:47.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.688+0000 7f3b233ef740 1 client.7092 shutdown 2024-05-08T05:47:47.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.688+0000 7f3b233ef740 1 -- 172.21.15.45:0/3840825204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd765220 msgr2=0x55c2fd75a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.688+0000 7f3b233ef740 1 --2- 172.21.15.45:0/3840825204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd765220 0x55c2fd75a080 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f3b0802a200 tx=0x7f3b08004730 comp rx=0 tx=0).stop 2024-05-08T05:47:47.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.689+0000 7f3b233ef740 1 -- 172.21.15.45:0/3840825204 shutdown_connections 2024-05-08T05:47:47.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.689+0000 7f3b233ef740 1 --2- 172.21.15.45:0/3840825204 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55c2fd748d00 0x55c2fd749160 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.689+0000 7f3b233ef740 1 --2- 172.21.15.45:0/3840825204 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c2fd763710 0x55c2fd763b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.689+0000 7f3b233ef740 1 --2- 172.21.15.45:0/3840825204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd765220 0x55c2fd75a080 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.689+0000 7f3af97fa640 0 client.7092 ms_handle_reset on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:47.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.689+0000 7f3b233ef740 1 --2- 172.21.15.45:0/3840825204 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c2fd748340 0x55c2fd742cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.689+0000 7f3b233ef740 1 -- 172.21.15.45:0/3840825204 >> 172.21.15.45:0/3840825204 conn(0x55c2fd747c00 msgr2=0x55c2fd7188f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.689+0000 7f3b233ef740 1 -- 172.21.15.45:0/3840825204 shutdown_connections 2024-05-08T05:47:47.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.689+0000 7f3b233ef740 1 -- 172.21.15.45:0/3840825204 wait complete. 2024-05-08T05:47:47.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.689+0000 7f3b233ef740 20 client.7092 trim_cache size 0 max 16384 2024-05-08T05:47:47.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.707+0000 7f3b233ef740 1 Processor -- start 2024-05-08T05:47:47.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.707+0000 7f3b233ef740 1 -- start start 2024-05-08T05:47:47.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.708+0000 7f3b233ef740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c2fd748340 0x55c2fd786270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.708+0000 7f3b233ef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd748d00 0x55c2fd744d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.708+0000 7f3b233ef740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c2fd763710 0x55c2fd717c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.708+0000 7f3b233ef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3b18201f70 con 0x55c2fd748d00 2024-05-08T05:47:47.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.708+0000 7f3b233ef740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b18200f50 con 0x55c2fd748340 2024-05-08T05:47:47.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.708+0000 7f3b233ef740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3b181f9060 con 0x55c2fd763710 2024-05-08T05:47:47.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.708+0000 7f3b1dc6c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd748d00 0x55c2fd744d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.708+0000 7f3b1dc6c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd748d00 0x55c2fd744d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36542/0 (socket says 172.21.15.45:36542) 2024-05-08T05:47:47.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.708+0000 7f3b1dc6c640 1 -- 172.21.15.45:0/1066028617 learned_addr learned my addr 172.21.15.45:0/1066028617 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:47.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.708+0000 7f3b1ec6e640 1 --2- 172.21.15.45:0/1066028617 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c2fd763710 0x55c2fd717c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.708+0000 7f3b1e46d640 1 --2- 172.21.15.45:0/1066028617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c2fd748340 0x55c2fd786270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.708+0000 7f3b1dc6c640 1 -- 172.21.15.45:0/1066028617 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c2fd763710 msgr2=0x55c2fd717c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.708+0000 7f3b1dc6c640 1 --2- 172.21.15.45:0/1066028617 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c2fd763710 0x55c2fd717c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.708+0000 7f3b1dc6c640 1 -- 172.21.15.45:0/1066028617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c2fd748340 msgr2=0x55c2fd786270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.708+0000 7f3b1dc6c640 1 --2- 172.21.15.45:0/1066028617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c2fd748340 0x55c2fd786270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.708+0000 7f3b1dc6c640 1 -- 172.21.15.45:0/1066028617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2fd71af60 con 0x55c2fd748d00 2024-05-08T05:47:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.708+0000 7f3b1dc6c640 1 --2- 172.21.15.45:0/1066028617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd748d00 0x55c2fd744d30 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009770 tx=0x7f3b0803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870a470a4b54fcf3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.708+0000 7f3afb7fe640 1 -- 172.21.15.45:0/1066028617 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b08037040 con 0x55c2fd748d00 2024-05-08T05:47:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.708+0000 7f3afb7fe640 1 -- 172.21.15.45:0/1066028617 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b0803e680 con 0x55c2fd748d00 2024-05-08T05:47:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.708+0000 7f3afb7fe640 1 -- 172.21.15.45:0/1066028617 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0803ec60 con 0x55c2fd748d00 2024-05-08T05:47:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.709+0000 7f3b233ef740 1 -- 172.21.15.45:0/1066028617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd748d00 msgr2=0x55c2fd744d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.709+0000 7f3b233ef740 1 --2- 172.21.15.45:0/1066028617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd748d00 0x55c2fd744d30 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009770 tx=0x7f3b0803b000 comp rx=0 tx=0).stop 2024-05-08T05:47:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.709+0000 7f3b233ef740 1 -- 172.21.15.45:0/1066028617 shutdown_connections 2024-05-08T05:47:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.709+0000 7f3b233ef740 1 --2- 172.21.15.45:0/1066028617 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c2fd763710 0x55c2fd717c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.709+0000 7f3b233ef740 1 --2- 172.21.15.45:0/1066028617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd748d00 0x55c2fd744d30 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.709+0000 7f3b233ef740 1 --2- 172.21.15.45:0/1066028617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c2fd748340 0x55c2fd786270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.709+0000 7f3b233ef740 1 -- 172.21.15.45:0/1066028617 >> 172.21.15.45:0/1066028617 conn(0x55c2fd747c00 msgr2=0x55c2fd75abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.709+0000 7f3b233ef740 1 -- 172.21.15.45:0/1066028617 shutdown_connections 2024-05-08T05:47:47.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.709+0000 7f3b233ef740 1 -- 172.21.15.45:0/1066028617 wait complete. 2024-05-08T05:47:47.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.710+0000 7f3b233ef740 1 -- start start 2024-05-08T05:47:47.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.710+0000 7f3b233ef740 1 Processor -- start 2024-05-08T05:47:47.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.710+0000 7f3b233ef740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c2fd748340 0x55c2fd762d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b233ef740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c2fd748d00 0x55c2fd77b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b233ef740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd763710 0x55c2fd770de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b233ef740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c2fd775540 con 0x55c2fd763710 2024-05-08T05:47:47.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b233ef740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3b18200aa0 con 0x55c2fd748340 2024-05-08T05:47:47.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b233ef740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55c2fd759c80 con 0x55c2fd748d00 2024-05-08T05:47:47.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b233ef740 10 client.0 _ll_register_callbacks cb 0x7f3b08035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:47:47.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b1ec6e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd763710 0x55c2fd770de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b1ec6e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd763710 0x55c2fd770de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36546/0 (socket says 172.21.15.45:36546) 2024-05-08T05:47:47.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b1ec6e640 1 -- 172.21.15.45:0/1802563558 learned_addr learned my addr 172.21.15.45:0/1802563558 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:47.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b1e46d640 1 --2- 172.21.15.45:0/1802563558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c2fd748340 0x55c2fd762d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b1dc6c640 1 --2- 172.21.15.45:0/1802563558 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c2fd748d00 0x55c2fd77b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b1ec6e640 1 -- 172.21.15.45:0/1802563558 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c2fd748d00 msgr2=0x55c2fd77b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b1ec6e640 1 --2- 172.21.15.45:0/1802563558 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c2fd748d00 0x55c2fd77b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b1ec6e640 1 -- 172.21.15.45:0/1802563558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c2fd748340 msgr2=0x55c2fd762d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b1ec6e640 1 --2- 172.21.15.45:0/1802563558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c2fd748340 0x55c2fd762d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b1ec6e640 1 -- 172.21.15.45:0/1802563558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2fd7632d0 con 0x55c2fd763710 2024-05-08T05:47:47.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b1e46d640 1 --2- 172.21.15.45:0/1802563558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c2fd748340 0x55c2fd762d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:47.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b1ec6e640 1 --2- 172.21.15.45:0/1802563558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd763710 0x55c2fd770de0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f3b1005d090 tx=0x7f3b10094fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5bea304f68e450 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3af97fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:47:47.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3af97fa640 1 -- 172.21.15.45:0/1802563558 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1005fe20 con 0x55c2fd763710 2024-05-08T05:47:47.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3af97fa640 1 -- 172.21.15.45:0/1802563558 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f3b10066400 con 0x55c2fd763710 2024-05-08T05:47:47.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b233ef740 1 -- 172.21.15.45:0/1802563558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b0803e680 con 0x55c2fd763710 2024-05-08T05:47:47.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3af97fa640 1 -- 172.21.15.45:0/1802563558 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f3b100641d0 con 0x55c2fd763710 2024-05-08T05:47:47.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b233ef740 10 client.7098 fetch_fsmap learned FSMap version 67 2024-05-08T05:47:47.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.711+0000 7f3b233ef740 1 -- 172.21.15.45:0/1802563558 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55c2fd762590 con 0x55c2fd763710 2024-05-08T05:47:47.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.712+0000 7f3af97fa640 1 -- 172.21.15.45:0/1802563558 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b0803e680 con 0x55c2fd763710 2024-05-08T05:47:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.712+0000 7f3af97fa640 1 -- 172.21.15.45:0/1802563558 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f3b1005f080 con 0x55c2fd763710 2024-05-08T05:47:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.712+0000 7f3b233ef740 10 client.7098 fetch_fsmap finished waiting for FSMap version 67 2024-05-08T05:47:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.712+0000 7f3b233ef740 10 client.7098 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-08T05:47:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.712+0000 7f3b233ef740 20 client.7098 populate_metadata read hostname 'smithi045' 2024-05-08T05:47:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.712+0000 7f3b233ef740 1 --2- 172.21.15.45:0/1802563558 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55c2fd765220 0x55c2fd7493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.712+0000 7f3b233ef740 4 client.7098 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-05-08T05:47:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.712+0000 7f3b233ef740 1 -- 172.21.15.45:0/1802563558 --> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x55c2fd688770 con 0x55c2fd765220 2024-05-08T05:47:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.712+0000 7f3b1e46d640 1 --2- 172.21.15.45:0/1802563558 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55c2fd765220 0x55c2fd7493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.712+0000 7f3af97fa640 10 client.7098 ms_handle_connect on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.712+0000 7f3b1e46d640 1 --2- 172.21.15.45:0/1802563558 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55c2fd765220 0x55c2fd7493c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f6d4dd7c46ecccc server_cookie=7027c11f4352ace5 in_seq=0 out_seq=0 2024-05-08T05:47:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.713+0000 7f3af97fa640 1 -- 172.21.15.45:0/1802563558 <== mds.0 v2:172.21.15.63:6835/3194497846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c2fd688770 con 0x55c2fd765220 2024-05-08T05:47:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.713+0000 7f3af97fa640 10 client.7098 handle_command_reply: tid=0 2024-05-08T05:47:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.713+0000 7f3b233ef740 1 client.7098 shutdown 2024-05-08T05:47:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.713+0000 7f3b233ef740 1 -- 172.21.15.45:0/1802563558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd763710 msgr2=0x55c2fd770de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.713+0000 7f3b233ef740 1 --2- 172.21.15.45:0/1802563558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd763710 0x55c2fd770de0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f3b1005d090 tx=0x7f3b10094fc0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.713+0000 7f3b233ef740 1 -- 172.21.15.45:0/1802563558 shutdown_connections 2024-05-08T05:47:47.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.713+0000 7f3b233ef740 1 --2- 172.21.15.45:0/1802563558 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55c2fd765220 0x55c2fd7493c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.713+0000 7f3b233ef740 1 --2- 172.21.15.45:0/1802563558 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c2fd748d00 0x55c2fd77b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.713+0000 7f3b233ef740 1 --2- 172.21.15.45:0/1802563558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2fd763710 0x55c2fd770de0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.713+0000 7f3b233ef740 1 --2- 172.21.15.45:0/1802563558 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c2fd748340 0x55c2fd762d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.713+0000 7f3af97fa640 0 client.7098 ms_handle_reset on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:47.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.713+0000 7f3b233ef740 1 -- 172.21.15.45:0/1802563558 >> 172.21.15.45:0/1802563558 conn(0x55c2fd747c00 msgr2=0x55c2fd7188f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.713+0000 7f3b233ef740 1 -- 172.21.15.45:0/1802563558 shutdown_connections 2024-05-08T05:47:47.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.713+0000 7f3b233ef740 1 -- 172.21.15.45:0/1802563558 wait complete. 2024-05-08T05:47:47.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.713+0000 7f3b233ef740 20 client.7098 trim_cache size 0 max 16384 2024-05-08T05:47:47.721 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:47:47.721 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-05-08T05:47:47.721 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "c0a4ed8f-dc3b-4518-bdf7-d16a432cb9d3", 2024-05-08T05:47:47.721 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-05-08T05:47:47.721 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:47:47.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.714+0000 7f3b206d1640 1 -- 172.21.15.45:0/1606499229 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3afc0762d0 msgr2=0x7f3afc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.714+0000 7f3b206d1640 1 --2- 172.21.15.45:0/1606499229 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3afc0762d0 0x7f3afc078790 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3b14006e80 tx=0x7f3b14008040 comp rx=0 tx=0).stop 2024-05-08T05:47:47.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.714+0000 7f3b206d1640 1 -- 172.21.15.45:0/1606499229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b18159920 msgr2=0x7f3b1809bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.714+0000 7f3b206d1640 1 --2- 172.21.15.45:0/1606499229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b18159920 0x7f3b1809bf70 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000c790 tx=0x7f3b1000cc50 comp rx=0 tx=0).stop 2024-05-08T05:47:47.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.714+0000 7f3b206d1640 1 -- 172.21.15.45:0/1606499229 shutdown_connections 2024-05-08T05:47:47.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.714+0000 7f3b206d1640 1 --2- 172.21.15.45:0/1606499229 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3afc0762d0 0x7f3afc078790 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.714+0000 7f3b206d1640 1 --2- 172.21.15.45:0/1606499229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b18159920 0x7f3b1809bf70 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.714+0000 7f3b206d1640 1 --2- 172.21.15.45:0/1606499229 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3b180d0df0 0x7f3b180b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.714+0000 7f3b206d1640 1 --2- 172.21.15.45:0/1606499229 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3b180aeb90 0x7f3b1809d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.714+0000 7f3b206d1640 1 -- 172.21.15.45:0/1606499229 >> 172.21.15.45:0/1606499229 conn(0x7f3b1806c3a0 msgr2=0x7f3b18158a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.714+0000 7f3b206d1640 1 -- 172.21.15.45:0/1606499229 shutdown_connections 2024-05-08T05:47:47.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.714+0000 7f3b206d1640 1 -- 172.21.15.45:0/1606499229 wait complete. 2024-05-08T05:47:47.723 INFO:tasks.ceph.mds.b.smithi063.stderr:2024-05-08T05:47:47.714+0000 7ff8b9265640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-05-08T05:47:07.822465+0000 1=1+0)->f(v1 m2024-05-08T05:47:07.822465+0000) n(v0 rc2024-05-08T05:47:07.942464+0000 2=1+1)->n(v0 rc2024-05-08T05:47:07.942464+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x561e6ad95b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-08T05:47:07.822465+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-08T05:47:07.942464+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-05-08T05:47:47.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:47:47.726 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-08T05:47:47.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.831+0000 7f966f62c640 1 Processor -- start 2024-05-08T05:47:47.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.832+0000 7f966f62c640 1 -- start start 2024-05-08T05:47:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.832+0000 7f966f62c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96680d1970 0x7f96680d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.832+0000 7f966f62c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96681630b0 con 0x7f966815f420 2024-05-08T05:47:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.832+0000 7f966f62c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96680d3e70 con 0x7f96680d4cc0 2024-05-08T05:47:47.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.832+0000 7f966f62c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f96680d2330 con 0x7f96680d1970 2024-05-08T05:47:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.833+0000 7f966d3c8640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f966815f420 0x7f96680d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:41084/0 (socket says 172.21.15.45:41084) 2024-05-08T05:47:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.833+0000 7f966d3c8640 1 -- 172.21.15.45:0/1336664993 learned_addr learned my addr 172.21.15.45:0/1336664993 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:47:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.833+0000 7f966cbc7640 1 --2- 172.21.15.45:0/1336664993 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96680d1970 0x7f96680d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.833+0000 7f9667fff640 1 -- 172.21.15.45:0/1336664993 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2513839657 0 0) 0x7f9660001110 con 0x7f966815f420 2024-05-08T05:47:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.833+0000 7f9667fff640 1 -- 172.21.15.45:0/1336664993 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9650003520 con 0x7f966815f420 2024-05-08T05:47:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.833+0000 7f9667fff640 1 -- 172.21.15.45:0/1336664993 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2138632682 0 0) 0x7f96680d3e70 con 0x7f96680d4cc0 2024-05-08T05:47:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.833+0000 7f9667fff640 1 -- 172.21.15.45:0/1336664993 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9660001110 con 0x7f96680d4cc0 2024-05-08T05:47:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.833+0000 7f966cbc7640 1 -- 172.21.15.45:0/1336664993 >> v1:172.21.15.63:6789/0 conn(0x7f96680d4cc0 legacy=0x7f96680d5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.833+0000 7f966cbc7640 1 -- 172.21.15.45:0/1336664993 >> v1:172.21.15.45:6789/0 conn(0x7f966815f420 legacy=0x7f96680d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.833+0000 7f966cbc7640 1 -- 172.21.15.45:0/1336664993 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96680d2550 con 0x7f96680d1970 2024-05-08T05:47:47.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.834+0000 7f966cbc7640 1 --2- 172.21.15.45:0/1336664993 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96680d1970 0x7f96680d1df0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f965c0097d0 tx=0x7f965c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce84b9ca9d73b85c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.834+0000 7f9667fff640 1 -- 172.21.15.45:0/1336664993 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c037070 con 0x7f96680d1970 2024-05-08T05:47:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.834+0000 7f9667fff640 1 -- 172.21.15.45:0/1336664993 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f965c032440 con 0x7f96680d1970 2024-05-08T05:47:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.834+0000 7f9667fff640 1 -- 172.21.15.45:0/1336664993 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c0327f0 con 0x7f96680d1970 2024-05-08T05:47:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.835+0000 7f966f62c640 1 -- 172.21.15.45:0/1336664993 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96680d1970 msgr2=0x7f96680d1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.835+0000 7f966f62c640 1 --2- 172.21.15.45:0/1336664993 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96680d1970 0x7f96680d1df0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f965c0097d0 tx=0x7f965c02a210 comp rx=0 tx=0).stop 2024-05-08T05:47:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.835+0000 7f966f62c640 1 -- 172.21.15.45:0/1336664993 shutdown_connections 2024-05-08T05:47:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.835+0000 7f966f62c640 1 --2- 172.21.15.45:0/1336664993 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96680d1970 0x7f96680d1df0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.835+0000 7f966f62c640 1 -- 172.21.15.45:0/1336664993 >> 172.21.15.45:0/1336664993 conn(0x7f966808eac0 msgr2=0x7f966809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.835+0000 7f966f62c640 1 -- 172.21.15.45:0/1336664993 shutdown_connections 2024-05-08T05:47:47.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.835+0000 7f966f62c640 1 -- 172.21.15.45:0/1336664993 wait complete. 2024-05-08T05:47:47.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.836+0000 7f966f62c640 1 Processor -- start 2024-05-08T05:47:47.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.836+0000 7f966f62c640 1 -- start start 2024-05-08T05:47:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.837+0000 7f966f62c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96680d1970 0x7f96681f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.837+0000 7f966f62c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96680d4cc0 0x7f96681f5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.837+0000 7f966f62c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96680c96a0 0x7f96680c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.837+0000 7f966f62c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f96680d3240 con 0x7f96680d4cc0 2024-05-08T05:47:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.837+0000 7f966f62c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f96680d2f60 con 0x7f96680c96a0 2024-05-08T05:47:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.837+0000 7f966f62c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f96680d2830 con 0x7f96680d1970 2024-05-08T05:47:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.837+0000 7f966cbc7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96680d4cc0 0x7f96681f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.837+0000 7f966cbc7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96680d4cc0 0x7f96681f5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36560/0 (socket says 172.21.15.45:36560) 2024-05-08T05:47:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.837+0000 7f966cbc7640 1 -- 172.21.15.45:0/145738555 learned_addr learned my addr 172.21.15.45:0/145738555 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.837+0000 7f966d3c8640 1 --2- 172.21.15.45:0/145738555 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96680d1970 0x7f96681f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.837+0000 7f966dbc9640 1 --2- 172.21.15.45:0/145738555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96680c96a0 0x7f96680c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.837+0000 7f966cbc7640 1 -- 172.21.15.45:0/145738555 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96680d1970 msgr2=0x7f96681f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.837+0000 7f966cbc7640 1 --2- 172.21.15.45:0/145738555 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96680d1970 0x7f96681f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.837+0000 7f966cbc7640 1 -- 172.21.15.45:0/145738555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96680c96a0 msgr2=0x7f96680c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.837+0000 7f966cbc7640 1 --2- 172.21.15.45:0/145738555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96680c96a0 0x7f96680c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.837+0000 7f966cbc7640 1 -- 172.21.15.45:0/145738555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96681f5d80 con 0x7f96680d4cc0 2024-05-08T05:47:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.837+0000 7f966dbc9640 1 --2- 172.21.15.45:0/145738555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96680c96a0 0x7f96680c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:47:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.837+0000 7f966cbc7640 1 --2- 172.21.15.45:0/145738555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96680d4cc0 0x7f96681f5310 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f965c035040 tx=0x7f965c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.838+0000 7f9665ffb640 1 -- 172.21.15.45:0/145738555 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c041020 con 0x7f96680d4cc0 2024-05-08T05:47:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.838+0000 7f9665ffb640 1 -- 172.21.15.45:0/145738555 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f965c03d6e0 con 0x7f96680d4cc0 2024-05-08T05:47:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.838+0000 7f966f62c640 1 -- 172.21.15.45:0/145738555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f966815aa50 con 0x7f96680d4cc0 2024-05-08T05:47:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.838+0000 7f9665ffb640 1 -- 172.21.15.45:0/145738555 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c037040 con 0x7f96680d4cc0 2024-05-08T05:47:47.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.838+0000 7f966f62c640 1 -- 172.21.15.45:0/145738555 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96680c8bc0 con 0x7f96680d4cc0 2024-05-08T05:47:47.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.840+0000 7f9665ffb640 1 -- 172.21.15.45:0/145738555 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f965c044020 con 0x7f96680d4cc0 2024-05-08T05:47:47.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.840+0000 7f9665ffb640 1 --2- 172.21.15.45:0/145738555 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f96540762d0 0x7f9654078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.840+0000 7f9665ffb640 1 -- 172.21.15.45:0/145738555 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f965c0b8050 con 0x7f96680d4cc0 2024-05-08T05:47:47.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.841+0000 7f966d3c8640 1 --2- 172.21.15.45:0/145738555 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f96540762d0 0x7f9654078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.841+0000 7f967241f740 1 Processor -- start 2024-05-08T05:47:47.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.841+0000 7f967241f740 1 -- start start 2024-05-08T05:47:47.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.841+0000 7f966d3c8640 1 --2- 172.21.15.45:0/145738555 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f96540762d0 0x7f9654078790 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9660007aa0 tx=0x7f9660009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f967241f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc9191f160 0x55bc918d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f967241f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 0x55bc91901600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f967241f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc91901b40 0x55bc918d6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f967241f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bc9189e690 con 0x55bc91901b40 2024-05-08T05:47:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f967241f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bc9183bf30 con 0x55bc91902280 2024-05-08T05:47:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f967241f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55bc917f4b20 con 0x55bc9191f160 2024-05-08T05:47:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f966dbc9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc91901b40 0x55bc918d6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f966d3c8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc9191f160 0x55bc918d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f966dbc9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc91901b40 0x55bc918d6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36564/0 (socket says 172.21.15.45:36564) 2024-05-08T05:47:47.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f966dbc9640 1 -- 172.21.15.45:0/2404811604 learned_addr learned my addr 172.21.15.45:0/2404811604 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f966cbc7640 1 --2- 172.21.15.45:0/2404811604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 0x55bc91901600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f966dbc9640 1 -- 172.21.15.45:0/2404811604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc9191f160 msgr2=0x55bc918d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f966dbc9640 1 --2- 172.21.15.45:0/2404811604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc9191f160 0x55bc918d1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f966dbc9640 1 -- 172.21.15.45:0/2404811604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 msgr2=0x55bc91901600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f966dbc9640 1 --2- 172.21.15.45:0/2404811604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 0x55bc91901600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f966dbc9640 1 -- 172.21.15.45:0/2404811604 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc917ece60 con 0x55bc91901b40 2024-05-08T05:47:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f966dbc9640 1 --2- 172.21.15.45:0/2404811604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc91901b40 0x55bc918d6950 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f9658001ee0 tx=0x7f965800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62136524465f1a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f964effd640 1 -- 172.21.15.45:0/2404811604 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658016020 con 0x55bc91901b40 2024-05-08T05:47:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f964effd640 1 -- 172.21.15.45:0/2404811604 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96580123f0 con 0x55bc91901b40 2024-05-08T05:47:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f964effd640 1 -- 172.21.15.45:0/2404811604 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96580127a0 con 0x55bc91901b40 2024-05-08T05:47:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f967241f740 1 -- 172.21.15.45:0/2404811604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc91901b40 msgr2=0x55bc918d6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f967241f740 1 --2- 172.21.15.45:0/2404811604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc91901b40 0x55bc918d6950 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f9658001ee0 tx=0x7f965800d520 comp rx=0 tx=0).stop 2024-05-08T05:47:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f967241f740 1 -- 172.21.15.45:0/2404811604 shutdown_connections 2024-05-08T05:47:47.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f967241f740 1 --2- 172.21.15.45:0/2404811604 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc91901b40 0x55bc918d6950 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f967241f740 1 --2- 172.21.15.45:0/2404811604 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 0x55bc91901600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f967241f740 1 --2- 172.21.15.45:0/2404811604 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc9191f160 0x55bc918d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f967241f740 1 -- 172.21.15.45:0/2404811604 >> 172.21.15.45:0/2404811604 conn(0x55bc918d3d60 msgr2=0x55bc9191c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f967241f740 1 -- 172.21.15.45:0/2404811604 shutdown_connections 2024-05-08T05:47:47.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.843+0000 7f967241f740 1 -- 172.21.15.45:0/2404811604 wait complete. 2024-05-08T05:47:47.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.844+0000 7f967241f740 1 -- start start 2024-05-08T05:47:47.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.844+0000 7f967241f740 1 Processor -- start 2024-05-08T05:47:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.845+0000 7f967241f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 0x55bc9192d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.845+0000 7f967241f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc9191f160 0x55bc9191d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.845+0000 7f967241f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc91903530 0x55bc9191de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.845+0000 7f967241f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bc917f85e0 con 0x55bc9191f160 2024-05-08T05:47:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.845+0000 7f967241f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bc91837ef0 con 0x55bc91902280 2024-05-08T05:47:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.845+0000 7f967241f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55bc918d5f10 con 0x55bc91903530 2024-05-08T05:47:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.845+0000 7f967241f740 10 client.0 _ll_register_callbacks cb 0x55bc917f3320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:47:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.845+0000 7f966cbc7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc9191f160 0x55bc9191d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.845+0000 7f966cbc7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc9191f160 0x55bc9191d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36578/0 (socket says 172.21.15.45:36578) 2024-05-08T05:47:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.845+0000 7f966cbc7640 1 -- 172.21.15.45:0/3075486715 learned_addr learned my addr 172.21.15.45:0/3075486715 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.845+0000 7f966d3c8640 1 --2- 172.21.15.45:0/3075486715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 0x55bc9192d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.845+0000 7f966dbc9640 1 --2- 172.21.15.45:0/3075486715 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc91903530 0x55bc9191de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.845+0000 7f966cbc7640 1 -- 172.21.15.45:0/3075486715 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc91903530 msgr2=0x55bc9191de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.845+0000 7f966cbc7640 1 --2- 172.21.15.45:0/3075486715 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc91903530 0x55bc9191de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.845+0000 7f966cbc7640 1 -- 172.21.15.45:0/3075486715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 msgr2=0x55bc9192d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.845+0000 7f966cbc7640 1 --2- 172.21.15.45:0/3075486715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 0x55bc9192d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.845+0000 7f966cbc7640 1 -- 172.21.15.45:0/3075486715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc916177d0 con 0x55bc9191f160 2024-05-08T05:47:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.845+0000 7f966cbc7640 1 --2- 172.21.15.45:0/3075486715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc9191f160 0x55bc9191d930 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f965c02a250 tx=0x7f965c0815f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8653599d7b2baf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.845+0000 7f9647fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:47:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.846+0000 7f9647fff640 1 -- 172.21.15.45:0/3075486715 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c080fd0 con 0x55bc9191f160 2024-05-08T05:47:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.846+0000 7f9647fff640 1 -- 172.21.15.45:0/3075486715 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f965c07c940 con 0x55bc9191f160 2024-05-08T05:47:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.846+0000 7f967241f740 1 -- 172.21.15.45:0/3075486715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96580123f0 con 0x55bc9191f160 2024-05-08T05:47:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.846+0000 7f9647fff640 1 -- 172.21.15.45:0/3075486715 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f965c085380 con 0x55bc9191f160 2024-05-08T05:47:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.846+0000 7f967241f740 10 client.7110 fetch_fsmap learned FSMap version 67 2024-05-08T05:47:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.846+0000 7f967241f740 1 -- 172.21.15.45:0/3075486715 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55bc9154d3f0 con 0x55bc9191f160 2024-05-08T05:47:47.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.846+0000 7f9647fff640 1 -- 172.21.15.45:0/3075486715 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96580123f0 con 0x55bc9191f160 2024-05-08T05:47:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.846+0000 7f9647fff640 1 -- 172.21.15.45:0/3075486715 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f965c03a030 con 0x55bc9191f160 2024-05-08T05:47:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.846+0000 7f967241f740 10 client.7110 fetch_fsmap finished waiting for FSMap version 67 2024-05-08T05:47:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.846+0000 7f967241f740 10 client.7110 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-08T05:47:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.846+0000 7f967241f740 20 client.7110 populate_metadata read hostname 'smithi045' 2024-05-08T05:47:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.846+0000 7f967241f740 1 --2- 172.21.15.45:0/3075486715 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55bc918fcff0 0x55bc91913790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.846+0000 7f967241f740 4 client.7110 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:47:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.846+0000 7f967241f740 1 -- 172.21.15.45:0/3075486715 --> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc91849b40 con 0x55bc918fcff0 2024-05-08T05:47:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.846+0000 7f966d3c8640 1 --2- 172.21.15.45:0/3075486715 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55bc918fcff0 0x55bc91913790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.847+0000 7f9647fff640 10 client.7110 ms_handle_connect on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.847+0000 7f966d3c8640 1 --2- 172.21.15.45:0/3075486715 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55bc918fcff0 0x55bc91913790 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=810855dd105b38c8 server_cookie=459c8c723b203cf4 in_seq=0 out_seq=0 2024-05-08T05:47:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.848+0000 7f9647fff640 1 -- 172.21.15.45:0/3075486715 <== mds.0 v2:172.21.15.63:6835/3194497846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bc91849b40 con 0x55bc918fcff0 2024-05-08T05:47:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.848+0000 7f9647fff640 10 client.7110 handle_command_reply: tid=0 2024-05-08T05:47:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.848+0000 7f967241f740 1 client.7110 shutdown 2024-05-08T05:47:47.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.848+0000 7f967241f740 1 -- 172.21.15.45:0/3075486715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc9191f160 msgr2=0x55bc9191d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.848+0000 7f967241f740 1 --2- 172.21.15.45:0/3075486715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc9191f160 0x55bc9191d930 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f965c02a250 tx=0x7f965c0815f0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.848+0000 7f967241f740 1 -- 172.21.15.45:0/3075486715 shutdown_connections 2024-05-08T05:47:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.848+0000 7f967241f740 1 --2- 172.21.15.45:0/3075486715 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55bc918fcff0 0x55bc91913790 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.848+0000 7f967241f740 1 --2- 172.21.15.45:0/3075486715 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc91903530 0x55bc9191de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.848+0000 7f967241f740 1 --2- 172.21.15.45:0/3075486715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc9191f160 0x55bc9191d930 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.848+0000 7f967241f740 1 --2- 172.21.15.45:0/3075486715 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 0x55bc9192d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.848+0000 7f9647fff640 0 client.7110 ms_handle_reset on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.848+0000 7f967241f740 1 -- 172.21.15.45:0/3075486715 >> 172.21.15.45:0/3075486715 conn(0x55bc91901b40 msgr2=0x55bc918d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.848+0000 7f967241f740 1 -- 172.21.15.45:0/3075486715 shutdown_connections 2024-05-08T05:47:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.848+0000 7f967241f740 1 -- 172.21.15.45:0/3075486715 wait complete. 2024-05-08T05:47:47.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.848+0000 7f967241f740 20 client.7110 trim_cache size 0 max 16384 2024-05-08T05:47:47.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.867+0000 7f967241f740 1 Processor -- start 2024-05-08T05:47:47.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.867+0000 7f967241f740 1 -- start start 2024-05-08T05:47:47.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.867+0000 7f967241f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 0x55bc919910e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.867+0000 7f967241f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc91903530 0x55bc91915160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.867+0000 7f967241f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc9191f160 0x55bc919156a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.867+0000 7f967241f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9668202100 con 0x55bc9191f160 2024-05-08T05:47:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.867+0000 7f967241f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9668201100 con 0x55bc91902280 2024-05-08T05:47:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.867+0000 7f967241f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f96681611d0 con 0x55bc91903530 2024-05-08T05:47:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.867+0000 7f966dbc9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc9191f160 0x55bc919156a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.868+0000 7f966dbc9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc9191f160 0x55bc919156a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36592/0 (socket says 172.21.15.45:36592) 2024-05-08T05:47:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.868+0000 7f966dbc9640 1 -- 172.21.15.45:0/4269063933 learned_addr learned my addr 172.21.15.45:0/4269063933 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.868+0000 7f966cbc7640 1 --2- 172.21.15.45:0/4269063933 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc91903530 0x55bc91915160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.868+0000 7f966d3c8640 1 --2- 172.21.15.45:0/4269063933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 0x55bc919910e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.868+0000 7f966dbc9640 1 -- 172.21.15.45:0/4269063933 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc91903530 msgr2=0x55bc91915160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.868+0000 7f966dbc9640 1 --2- 172.21.15.45:0/4269063933 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc91903530 0x55bc91915160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.868+0000 7f966dbc9640 1 -- 172.21.15.45:0/4269063933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 msgr2=0x55bc919910e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.868+0000 7f966dbc9640 1 --2- 172.21.15.45:0/4269063933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 0x55bc919910e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.868+0000 7f966dbc9640 1 -- 172.21.15.45:0/4269063933 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc918d4ea0 con 0x55bc9191f160 2024-05-08T05:47:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.868+0000 7f966d3c8640 1 --2- 172.21.15.45:0/4269063933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 0x55bc919910e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:47:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.868+0000 7f966cbc7640 1 --2- 172.21.15.45:0/4269063933 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc91903530 0x55bc91915160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:47:47.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.868+0000 7f966dbc9640 1 --2- 172.21.15.45:0/4269063933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc9191f160 0x55bc919156a0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f965800da20 tx=0x7f9658012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f23d2e395476c91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.868+0000 7f964e7fc640 1 -- 172.21.15.45:0/4269063933 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658016040 con 0x55bc9191f160 2024-05-08T05:47:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.868+0000 7f964e7fc640 1 -- 172.21.15.45:0/4269063933 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f965800de40 con 0x55bc9191f160 2024-05-08T05:47:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.868+0000 7f964e7fc640 1 -- 172.21.15.45:0/4269063933 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658003b70 con 0x55bc9191f160 2024-05-08T05:47:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.869+0000 7f967241f740 1 -- 172.21.15.45:0/4269063933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc9191f160 msgr2=0x55bc919156a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.869+0000 7f967241f740 1 --2- 172.21.15.45:0/4269063933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc9191f160 0x55bc919156a0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f965800da20 tx=0x7f9658012000 comp rx=0 tx=0).stop 2024-05-08T05:47:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.869+0000 7f967241f740 1 -- 172.21.15.45:0/4269063933 shutdown_connections 2024-05-08T05:47:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.869+0000 7f967241f740 1 --2- 172.21.15.45:0/4269063933 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc91903530 0x55bc91915160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.869+0000 7f967241f740 1 --2- 172.21.15.45:0/4269063933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc9191f160 0x55bc919156a0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.869+0000 7f967241f740 1 --2- 172.21.15.45:0/4269063933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 0x55bc919910e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.869+0000 7f967241f740 1 -- 172.21.15.45:0/4269063933 >> 172.21.15.45:0/4269063933 conn(0x55bc91901b40 msgr2=0x55bc919408c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.869+0000 7f967241f740 1 -- 172.21.15.45:0/4269063933 shutdown_connections 2024-05-08T05:47:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.869+0000 7f967241f740 1 -- 172.21.15.45:0/4269063933 wait complete. 2024-05-08T05:47:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.869+0000 7f967241f740 1 -- start start 2024-05-08T05:47:47.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.869+0000 7f967241f740 1 Processor -- start 2024-05-08T05:47:47.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.870+0000 7f967241f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 0x55bc91920130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.870+0000 7f967241f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc91903530 0x55bc9199a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.870+0000 7f967241f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc9191f160 0x55bc918d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.870+0000 7f967241f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bc9191e780 con 0x55bc91903530 2024-05-08T05:47:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.870+0000 7f967241f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9668200ae0 con 0x55bc91902280 2024-05-08T05:47:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.870+0000 7f967241f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55bc91914730 con 0x55bc9191f160 2024-05-08T05:47:47.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.870+0000 7f967241f740 10 client.0 _ll_register_callbacks cb 0x7f965c07cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:47:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.870+0000 7f966cbc7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc91903530 0x55bc9199a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.870+0000 7f966cbc7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc91903530 0x55bc9199a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36608/0 (socket says 172.21.15.45:36608) 2024-05-08T05:47:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.871+0000 7f966cbc7640 1 -- 172.21.15.45:0/1166597848 learned_addr learned my addr 172.21.15.45:0/1166597848 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:47:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.871+0000 7f966dbc9640 1 --2- 172.21.15.45:0/1166597848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc9191f160 0x55bc918d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.871+0000 7f966d3c8640 1 --2- 172.21.15.45:0/1166597848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 0x55bc91920130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.871+0000 7f966cbc7640 1 -- 172.21.15.45:0/1166597848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc9191f160 msgr2=0x55bc918d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.871+0000 7f966cbc7640 1 --2- 172.21.15.45:0/1166597848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc9191f160 0x55bc918d09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.871+0000 7f966cbc7640 1 -- 172.21.15.45:0/1166597848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 msgr2=0x55bc91920130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:47:47.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.871+0000 7f966cbc7640 1 --2- 172.21.15.45:0/1166597848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 0x55bc91920130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.871+0000 7f966cbc7640 1 -- 172.21.15.45:0/1166597848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc9150d0f0 con 0x55bc91903530 2024-05-08T05:47:47.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.871+0000 7f966cbc7640 1 --2- 172.21.15.45:0/1166597848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc91903530 0x55bc9199a3f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f965c07d5c0 tx=0x7f965c0b61f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d70489437d1608 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:47.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.871+0000 7f9647fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:47:47.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.871+0000 7f9647fff640 1 -- 172.21.15.45:0/1166597848 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965c080920 con 0x55bc91903530 2024-05-08T05:47:47.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.871+0000 7f9647fff640 1 -- 172.21.15.45:0/1166597848 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f965c0b0dc0 con 0x55bc91903530 2024-05-08T05:47:47.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.871+0000 7f967241f740 1 -- 172.21.15.45:0/1166597848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f965800de40 con 0x55bc91903530 2024-05-08T05:47:47.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.871+0000 7f9647fff640 1 -- 172.21.15.45:0/1166597848 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f965c080050 con 0x55bc91903530 2024-05-08T05:47:47.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.871+0000 7f967241f740 10 client.7116 fetch_fsmap learned FSMap version 67 2024-05-08T05:47:47.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.871+0000 7f967241f740 1 -- 172.21.15.45:0/1166597848 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55bc91915710 con 0x55bc91903530 2024-05-08T05:47:47.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.871+0000 7f9647fff640 1 -- 172.21.15.45:0/1166597848 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f965800de40 con 0x55bc91903530 2024-05-08T05:47:47.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.871+0000 7f9647fff640 1 -- 172.21.15.45:0/1166597848 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f965c09a380 con 0x55bc91903530 2024-05-08T05:47:47.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.871+0000 7f967241f740 10 client.7116 fetch_fsmap finished waiting for FSMap version 67 2024-05-08T05:47:47.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.872+0000 7f967241f740 10 client.7116 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-08T05:47:47.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.872+0000 7f967241f740 20 client.7116 populate_metadata read hostname 'smithi045' 2024-05-08T05:47:47.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.872+0000 7f967241f740 1 --2- 172.21.15.45:0/1166597848 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55bc918d7310 0x55bc918fbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:47.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.872+0000 7f967241f740 4 client.7116 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:47:47.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.872+0000 7f967241f740 1 -- 172.21.15.45:0/1166597848 --> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc918439b0 con 0x55bc918d7310 2024-05-08T05:47:47.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.872+0000 7f966d3c8640 1 --2- 172.21.15.45:0/1166597848 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55bc918d7310 0x55bc918fbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:47.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.872+0000 7f966d3c8640 1 --2- 172.21.15.45:0/1166597848 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55bc918d7310 0x55bc918fbd40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92562349e0f7730a server_cookie=cb964b764212936d in_seq=0 out_seq=0 2024-05-08T05:47:47.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.872+0000 7f9647fff640 10 client.7116 ms_handle_connect on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:47.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.872+0000 7f9647fff640 1 -- 172.21.15.45:0/1166597848 <== mds.0 v2:172.21.15.63:6835/3194497846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x55bc918439b0 con 0x55bc918d7310 2024-05-08T05:47:47.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.872+0000 7f9647fff640 10 client.7116 handle_command_reply: tid=0 2024-05-08T05:47:47.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.872+0000 7f967241f740 1 client.7116 shutdown 2024-05-08T05:47:47.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.872+0000 7f967241f740 1 -- 172.21.15.45:0/1166597848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc91903530 msgr2=0x55bc9199a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.872+0000 7f967241f740 1 --2- 172.21.15.45:0/1166597848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc91903530 0x55bc9199a3f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f965c07d5c0 tx=0x7f965c0b61f0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.872+0000 7f967241f740 1 -- 172.21.15.45:0/1166597848 shutdown_connections 2024-05-08T05:47:47.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.872+0000 7f967241f740 1 --2- 172.21.15.45:0/1166597848 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55bc918d7310 0x55bc918fbd40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.872+0000 7f967241f740 1 --2- 172.21.15.45:0/1166597848 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bc9191f160 0x55bc918d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.872+0000 7f967241f740 1 --2- 172.21.15.45:0/1166597848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc91903530 0x55bc9199a3f0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.872+0000 7f9647fff640 0 client.7116 ms_handle_reset on v2:172.21.15.63:6835/3194497846 2024-05-08T05:47:47.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.872+0000 7f967241f740 1 --2- 172.21.15.45:0/1166597848 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bc91902280 0x55bc91920130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.872+0000 7f967241f740 1 -- 172.21.15.45:0/1166597848 >> 172.21.15.45:0/1166597848 conn(0x55bc91901b40 msgr2=0x55bc9191fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.873+0000 7f967241f740 1 -- 172.21.15.45:0/1166597848 shutdown_connections 2024-05-08T05:47:47.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.873+0000 7f967241f740 1 -- 172.21.15.45:0/1166597848 wait complete. 2024-05-08T05:47:47.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.873+0000 7f967241f740 20 client.7116 trim_cache size 0 max 16384 2024-05-08T05:47:47.887 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:47:47.887 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T05:47:47.888 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-05-08T05:47:47.888 INFO:teuthology.orchestra.run.smithi045.stdout: "c0a4ed8f-dc3b-4518-bdf7-d16a432cb9d3": { 2024-05-08T05:47:47.888 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/dir", 2024-05-08T05:47:47.888 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "c0a4ed8f-dc3b-4518-bdf7-d16a432cb9d3", 2024-05-08T05:47:47.888 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,repair" 2024-05-08T05:47:47.889 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:47:47.889 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:47:47.889 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:47:47.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.873+0000 7f966f62c640 1 -- 172.21.15.45:0/145738555 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f96540762d0 msgr2=0x7f9654078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.873+0000 7f966f62c640 1 --2- 172.21.15.45:0/145738555 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f96540762d0 0x7f9654078790 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9660007aa0 tx=0x7f9660009040 comp rx=0 tx=0).stop 2024-05-08T05:47:47.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.873+0000 7f966f62c640 1 -- 172.21.15.45:0/145738555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96680d4cc0 msgr2=0x7f96681f5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:47.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.873+0000 7f966f62c640 1 --2- 172.21.15.45:0/145738555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96680d4cc0 0x7f96681f5310 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f965c035040 tx=0x7f965c0046c0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.873+0000 7f966f62c640 1 -- 172.21.15.45:0/145738555 shutdown_connections 2024-05-08T05:47:47.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.873+0000 7f966f62c640 1 --2- 172.21.15.45:0/145738555 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f96540762d0 0x7f9654078790 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.873+0000 7f966f62c640 1 --2- 172.21.15.45:0/145738555 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96680c96a0 0x7f96680c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.873+0000 7f966f62c640 1 --2- 172.21.15.45:0/145738555 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96680d4cc0 0x7f96681f5310 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.873+0000 7f966f62c640 1 --2- 172.21.15.45:0/145738555 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96680d1970 0x7f96681f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:47.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.873+0000 7f966f62c640 1 -- 172.21.15.45:0/145738555 >> 172.21.15.45:0/145738555 conn(0x7f966808eac0 msgr2=0x7f96680cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:47.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.873+0000 7f966f62c640 1 -- 172.21.15.45:0/145738555 shutdown_connections 2024-05-08T05:47:47.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:47:47.873+0000 7f966f62c640 1 -- 172.21.15.45:0/145738555 wait complete. 2024-05-08T05:47:47.893 INFO:tasks.cephfs.filesystem:scrub status for tag:c0a4ed8f-dc3b-4518-bdf7-d16a432cb9d3 - {'path': '/dir', 'tag': 'c0a4ed8f-dc3b-4518-bdf7-d16a432cb9d3', 'options': 'recursive,repair'} 2024-05-08T05:48:04.864 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:48:04.867 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:48:17.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:48:17.894 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-08T05:48:18.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.013+0000 7f2229cfc640 1 Processor -- start 2024-05-08T05:48:18.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.013+0000 7f2229cfc640 1 -- start start 2024-05-08T05:48:18.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.014+0000 7f2229cfc640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f222406d620 0x7f2224073800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.014+0000 7f2229cfc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f222406da00 0x7f22240af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.014+0000 7f2229cfc640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22240af800 con 0x7f22240b5ee0 2024-05-08T05:48:18.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.014+0000 7f2229cfc640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f22240afaa0 con 0x7f222406d620 2024-05-08T05:48:18.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.014+0000 7f2229cfc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f22240b5a50 con 0x7f222406da00 2024-05-08T05:48:18.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.014+0000 7f2223fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f22240b5ee0 0x7f222406e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45658/0 (socket says 172.21.15.45:45658) 2024-05-08T05:48:18.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.014+0000 7f2223fff640 1 -- 172.21.15.45:0/2935329738 learned_addr learned my addr 172.21.15.45:0/2935329738 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:48:18.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f2222ffd640 1 --2- 172.21.15.45:0/2935329738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f222406da00 0x7f22240af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f22237fe640 1 --2- 172.21.15.45:0/2935329738 >> v2:172.21.15.63:3300/0 conn(0x7f222406d620 0x7f2224073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f22227fc640 1 -- 172.21.15.45:0/2935329738 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1860980657 0 0) 0x7f22240af800 con 0x7f22240b5ee0 2024-05-08T05:48:18.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f22227fc640 1 -- 172.21.15.45:0/2935329738 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f220c003520 con 0x7f22240b5ee0 2024-05-08T05:48:18.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f22227fc640 1 -- 172.21.15.45:0/2935329738 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1691781719 0 0) 0x7f220c003520 con 0x7f22240b5ee0 2024-05-08T05:48:18.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f22227fc640 1 -- 172.21.15.45:0/2935329738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f222406da00 msgr2=0x7f22240af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f22227fc640 1 --2- 172.21.15.45:0/2935329738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f222406da00 0x7f22240af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f22227fc640 1 -- 172.21.15.45:0/2935329738 >> v2:172.21.15.63:3300/0 conn(0x7f222406d620 msgr2=0x7f2224073800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f22227fc640 1 --2- 172.21.15.45:0/2935329738 >> v2:172.21.15.63:3300/0 conn(0x7f222406d620 0x7f2224073800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f22227fc640 1 -- 172.21.15.45:0/2935329738 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22240b5120 con 0x7f22240b5ee0 2024-05-08T05:48:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f22227fc640 1 -- 172.21.15.45:0/2935329738 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f221c001a70 con 0x7f22240b5ee0 2024-05-08T05:48:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f22227fc640 1 -- 172.21.15.45:0/2935329738 >> v1:172.21.15.45:6789/0 conn(0x7f22240b5ee0 legacy=0x7f222406e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f22227fc640 1 --2- 172.21.15.45:0/2935329738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f222406da00 0x7f220c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f22227fc640 1 --2- 172.21.15.45:0/2935329738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f220c003fd0 0x7f220c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f22227fc640 1 --2- 172.21.15.45:0/2935329738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f220c004d40 0x7f220c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f22227fc640 1 -- 172.21.15.45:0/2935329738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f22240afaa0 con 0x7f220c004d40 2024-05-08T05:48:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f22227fc640 1 -- 172.21.15.45:0/2935329738 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f22240b5a50 con 0x7f220c003fd0 2024-05-08T05:48:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f22227fc640 1 -- 172.21.15.45:0/2935329738 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f220c00bb50 con 0x7f222406da00 2024-05-08T05:48:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f2223fff640 1 --2- 172.21.15.45:0/2935329738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f220c004d40 0x7f220c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f22237fe640 1 --2- 172.21.15.45:0/2935329738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f222406da00 0x7f220c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.015+0000 7f2222ffd640 1 --2- 172.21.15.45:0/2935329738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f220c003fd0 0x7f220c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.016+0000 7f2223fff640 1 -- 172.21.15.45:0/2935329738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f222406da00 msgr2=0x7f220c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.016+0000 7f2223fff640 1 --2- 172.21.15.45:0/2935329738 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f222406da00 0x7f220c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.016+0000 7f2223fff640 1 -- 172.21.15.45:0/2935329738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f220c003fd0 msgr2=0x7f220c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.016+0000 7f2223fff640 1 --2- 172.21.15.45:0/2935329738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f220c003fd0 0x7f220c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.016+0000 7f2223fff640 1 -- 172.21.15.45:0/2935329738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220c003520 con 0x7f220c004d40 2024-05-08T05:48:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.016+0000 7f2223fff640 1 --2- 172.21.15.45:0/2935329738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f220c004d40 0x7f220c0051c0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f221c00a8a0 tx=0x7f220c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537910b7948e17c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.017+0000 7f22227fc640 1 -- 172.21.15.45:0/2935329738 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c00a4c0 con 0x7f220c004d40 2024-05-08T05:48:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.017+0000 7f22227fc640 1 -- 172.21.15.45:0/2935329738 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221c02f070 con 0x7f220c004d40 2024-05-08T05:48:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.017+0000 7f22227fc640 1 -- 172.21.15.45:0/2935329738 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c02bca0 con 0x7f220c004d40 2024-05-08T05:48:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.017+0000 7f2229cfc640 1 -- 172.21.15.45:0/2935329738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f220c004d40 msgr2=0x7f220c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.017+0000 7f2229cfc640 1 --2- 172.21.15.45:0/2935329738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f220c004d40 0x7f220c0051c0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f221c00a8a0 tx=0x7f220c005830 comp rx=0 tx=0).stop 2024-05-08T05:48:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.017+0000 7f2223fff640 1 -- 172.21.15.45:0/2935329738 reap_dead start 2024-05-08T05:48:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.017+0000 7f2229cfc640 1 -- 172.21.15.45:0/2935329738 shutdown_connections 2024-05-08T05:48:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.017+0000 7f2229cfc640 1 -- 172.21.15.45:0/2935329738 >> 172.21.15.45:0/2935329738 conn(0x7f22240728c0 msgr2=0x7f2224072cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.017+0000 7f2229cfc640 1 -- 172.21.15.45:0/2935329738 shutdown_connections 2024-05-08T05:48:18.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.017+0000 7f2229cfc640 1 -- 172.21.15.45:0/2935329738 wait complete. 2024-05-08T05:48:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.019+0000 7f2229cfc640 1 Processor -- start 2024-05-08T05:48:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.019+0000 7f2229cfc640 1 -- start start 2024-05-08T05:48:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.019+0000 7f2229cfc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22241f4ab0 0x7f222415f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.019+0000 7f2229cfc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22241f4e90 0x7f222415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f2229cfc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f222415d2e0 0x7f222415d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f2229cfc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f222406f030 con 0x7f22241f4ab0 2024-05-08T05:48:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f2229cfc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f22240b56e0 con 0x7f22241f4e90 2024-05-08T05:48:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f2229cfc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f222406f1b0 con 0x7f222415d2e0 2024-05-08T05:48:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f22237fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22241f4ab0 0x7f222415f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f22237fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22241f4ab0 0x7f222415f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35076/0 (socket says 172.21.15.45:35076) 2024-05-08T05:48:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f22237fe640 1 -- 172.21.15.45:0/302613128 learned_addr learned my addr 172.21.15.45:0/302613128 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f2223fff640 1 --2- 172.21.15.45:0/302613128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f222415d2e0 0x7f222415d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f2222ffd640 1 --2- 172.21.15.45:0/302613128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22241f4e90 0x7f222415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f22237fe640 1 -- 172.21.15.45:0/302613128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f222415d2e0 msgr2=0x7f222415d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f22237fe640 1 --2- 172.21.15.45:0/302613128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f222415d2e0 0x7f222415d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f22237fe640 1 -- 172.21.15.45:0/302613128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22241f4e90 msgr2=0x7f222415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f22237fe640 1 --2- 172.21.15.45:0/302613128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22241f4e90 0x7f222415ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f22237fe640 1 -- 172.21.15.45:0/302613128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2224160160 con 0x7f22241f4ab0 2024-05-08T05:48:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f2222ffd640 1 --2- 172.21.15.45:0/302613128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22241f4e90 0x7f222415ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f22237fe640 1 --2- 172.21.15.45:0/302613128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22241f4ab0 0x7f222415f5c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2214011aa0 tx=0x7f2214011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f220bfff640 1 -- 172.21.15.45:0/302613128 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221400fb80 con 0x7f22241f4ab0 2024-05-08T05:48:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f220bfff640 1 -- 172.21.15.45:0/302613128 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221400fd20 con 0x7f22241f4ab0 2024-05-08T05:48:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f2229cfc640 1 -- 172.21.15.45:0/302613128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f222415fba0 con 0x7f22241f4ab0 2024-05-08T05:48:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f220bfff640 1 -- 172.21.15.45:0/302613128 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214014660 con 0x7f22241f4ab0 2024-05-08T05:48:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.020+0000 7f2229cfc640 1 -- 172.21.15.45:0/302613128 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f222415a810 con 0x7f22241f4ab0 2024-05-08T05:48:18.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.022+0000 7f220bfff640 1 -- 172.21.15.45:0/302613128 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2214012040 con 0x7f22241f4ab0 2024-05-08T05:48:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.023+0000 7f220bfff640 1 --2- 172.21.15.45:0/302613128 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2210076300 0x7f22100787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.023+0000 7f220bfff640 1 -- 172.21.15.45:0/302613128 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f2214097f00 con 0x7f22241f4ab0 2024-05-08T05:48:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.023+0000 7f2222ffd640 1 --2- 172.21.15.45:0/302613128 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2210076300 0x7f22100787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.024+0000 7f222ca11740 1 Processor -- start 2024-05-08T05:48:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.024+0000 7f222ca11740 1 -- start start 2024-05-08T05:48:18.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.024+0000 7f2222ffd640 1 --2- 172.21.15.45:0/302613128 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2210076300 0x7f22100787c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f22241f5270 tx=0x7f2218008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.025+0000 7f222ca11740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a43160 0x55b3929f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.025+0000 7f222ca11740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a26280 0x55b392a25600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.025+0000 7f222ca11740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a25b40 0x55b3929fa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.025+0000 7f222ca11740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b3929c2690 con 0x55b392a25b40 2024-05-08T05:48:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.025+0000 7f222ca11740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b39295ff30 con 0x55b392a26280 2024-05-08T05:48:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.025+0000 7f222ca11740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55b392918b20 con 0x55b392a43160 2024-05-08T05:48:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.025+0000 7f2223fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a25b40 0x55b3929fa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.025+0000 7f2223fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a25b40 0x55b3929fa950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35082/0 (socket says 172.21.15.45:35082) 2024-05-08T05:48:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.025+0000 7f2223fff640 1 -- 172.21.15.45:0/2843660437 learned_addr learned my addr 172.21.15.45:0/2843660437 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.025+0000 7f22237fe640 1 --2- 172.21.15.45:0/2843660437 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a43160 0x55b3929f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.025+0000 7f2222ffd640 1 --2- 172.21.15.45:0/2843660437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a26280 0x55b392a25600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.025+0000 7f2223fff640 1 -- 172.21.15.45:0/2843660437 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a43160 msgr2=0x55b3929f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.025+0000 7f2223fff640 1 --2- 172.21.15.45:0/2843660437 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a43160 0x55b3929f5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.025+0000 7f2223fff640 1 -- 172.21.15.45:0/2843660437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a26280 msgr2=0x55b392a25600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.025+0000 7f2223fff640 1 --2- 172.21.15.45:0/2843660437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a26280 0x55b392a25600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.025+0000 7f2223fff640 1 -- 172.21.15.45:0/2843660437 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b392910e60 con 0x55b392a25b40 2024-05-08T05:48:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.025+0000 7f2223fff640 1 --2- 172.21.15.45:0/2843660437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a25b40 0x55b3929fa950 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f221c00a990 tx=0x7f221c007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f72fbeb4e27acab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.025+0000 7f22097fa640 1 -- 172.21.15.45:0/2843660437 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c001a70 con 0x55b392a25b40 2024-05-08T05:48:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.025+0000 7f22097fa640 1 -- 172.21.15.45:0/2843660437 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221c02f040 con 0x55b392a25b40 2024-05-08T05:48:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.025+0000 7f22097fa640 1 -- 172.21.15.45:0/2843660437 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c004d60 con 0x55b392a25b40 2024-05-08T05:48:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.026+0000 7f222ca11740 1 -- 172.21.15.45:0/2843660437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a25b40 msgr2=0x55b3929fa950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.026+0000 7f222ca11740 1 --2- 172.21.15.45:0/2843660437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a25b40 0x55b3929fa950 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f221c00a990 tx=0x7f221c007000 comp rx=0 tx=0).stop 2024-05-08T05:48:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.026+0000 7f222ca11740 1 -- 172.21.15.45:0/2843660437 shutdown_connections 2024-05-08T05:48:18.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.026+0000 7f222ca11740 1 --2- 172.21.15.45:0/2843660437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a25b40 0x55b3929fa950 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.026+0000 7f222ca11740 1 --2- 172.21.15.45:0/2843660437 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a26280 0x55b392a25600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.026+0000 7f222ca11740 1 --2- 172.21.15.45:0/2843660437 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a43160 0x55b3929f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.026+0000 7f222ca11740 1 -- 172.21.15.45:0/2843660437 >> 172.21.15.45:0/2843660437 conn(0x55b3929f7d60 msgr2=0x55b392a40600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.026+0000 7f222ca11740 1 -- 172.21.15.45:0/2843660437 shutdown_connections 2024-05-08T05:48:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.026+0000 7f222ca11740 1 -- 172.21.15.45:0/2843660437 wait complete. 2024-05-08T05:48:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.027+0000 7f222ca11740 1 -- start start 2024-05-08T05:48:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.027+0000 7f222ca11740 1 Processor -- start 2024-05-08T05:48:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f222ca11740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a26280 0x55b392a52fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f222ca11740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a43160 0x55b392a51280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f222ca11740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a517c0 0x55b392a26770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f222ca11740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b39291c5e0 con 0x55b392a26280 2024-05-08T05:48:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f222ca11740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b39295bef0 con 0x55b392a43160 2024-05-08T05:48:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f222ca11740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55b3929f9f10 con 0x55b392a517c0 2024-05-08T05:48:18.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f222ca11740 10 client.0 _ll_register_callbacks cb 0x55b392917320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:48:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f22237fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a26280 0x55b392a52fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f22237fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a26280 0x55b392a52fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35090/0 (socket says 172.21.15.45:35090) 2024-05-08T05:48:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f22237fe640 1 -- 172.21.15.45:0/3008614812 learned_addr learned my addr 172.21.15.45:0/3008614812 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f2222ffd640 1 --2- 172.21.15.45:0/3008614812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a43160 0x55b392a51280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f2223fff640 1 --2- 172.21.15.45:0/3008614812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a517c0 0x55b392a26770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f22237fe640 1 -- 172.21.15.45:0/3008614812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a517c0 msgr2=0x55b392a26770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f22237fe640 1 --2- 172.21.15.45:0/3008614812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a517c0 0x55b392a26770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f22237fe640 1 -- 172.21.15.45:0/3008614812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a43160 msgr2=0x55b392a51280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f22237fe640 1 --2- 172.21.15.45:0/3008614812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a43160 0x55b392a51280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f22237fe640 1 -- 172.21.15.45:0/3008614812 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39273b7d0 con 0x55b392a26280 2024-05-08T05:48:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f2222ffd640 1 --2- 172.21.15.45:0/3008614812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a43160 0x55b392a51280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f2223fff640 1 --2- 172.21.15.45:0/3008614812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a517c0 0x55b392a26770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:18.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f22237fe640 1 --2- 172.21.15.45:0/3008614812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a26280 0x55b392a52fe0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f2214060c00 tx=0x7f22140614d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9fb8c5dbb8e1d42 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f22027fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:48:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f22027fc640 1 -- 172.21.15.45:0/3008614812 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214060f00 con 0x55b392a26280 2024-05-08T05:48:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.028+0000 7f22027fc640 1 -- 172.21.15.45:0/3008614812 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f2214092d20 con 0x55b392a26280 2024-05-08T05:48:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.029+0000 7f222ca11740 1 -- 172.21.15.45:0/3008614812 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f221c02f040 con 0x55b392a26280 2024-05-08T05:48:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.029+0000 7f22027fc640 1 -- 172.21.15.45:0/3008614812 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f22140652b0 con 0x55b392a26280 2024-05-08T05:48:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.029+0000 7f222ca11740 10 client.7128 fetch_fsmap learned FSMap version 67 2024-05-08T05:48:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.029+0000 7f222ca11740 1 -- 172.21.15.45:0/3008614812 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55b3926713f0 con 0x55b392a26280 2024-05-08T05:48:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.029+0000 7f22027fc640 1 -- 172.21.15.45:0/3008614812 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221c02f040 con 0x55b392a26280 2024-05-08T05:48:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.029+0000 7f22027fc640 1 -- 172.21.15.45:0/3008614812 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f2214025030 con 0x55b392a26280 2024-05-08T05:48:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.029+0000 7f222ca11740 10 client.7128 fetch_fsmap finished waiting for FSMap version 67 2024-05-08T05:48:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.029+0000 7f222ca11740 10 client.7128 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-08T05:48:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.029+0000 7f222ca11740 20 client.7128 populate_metadata read hostname 'smithi045' 2024-05-08T05:48:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.029+0000 7f222ca11740 1 --2- 172.21.15.45:0/3008614812 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55b392a26d50 0x55b392a43a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.029+0000 7f222ca11740 4 client.7128 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:48:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.029+0000 7f222ca11740 1 -- 172.21.15.45:0/3008614812 --> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b39296db40 con 0x55b392a26d50 2024-05-08T05:48:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.029+0000 7f2222ffd640 1 --2- 172.21.15.45:0/3008614812 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55b392a26d50 0x55b392a43a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.030+0000 7f22027fc640 10 client.7128 ms_handle_connect on v2:172.21.15.63:6835/3194497846 2024-05-08T05:48:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.030+0000 7f2222ffd640 1 --2- 172.21.15.45:0/3008614812 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55b392a26d50 0x55b392a43a20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ebe8c4884de59b4 server_cookie=7bde7ba7a7987dde in_seq=0 out_seq=0 2024-05-08T05:48:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.030+0000 7f22027fc640 1 -- 172.21.15.45:0/3008614812 <== mds.0 v2:172.21.15.63:6835/3194497846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b39296db40 con 0x55b392a26d50 2024-05-08T05:48:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.030+0000 7f22027fc640 10 client.7128 handle_command_reply: tid=0 2024-05-08T05:48:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.031+0000 7f222ca11740 1 client.7128 shutdown 2024-05-08T05:48:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.031+0000 7f222ca11740 1 -- 172.21.15.45:0/3008614812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a26280 msgr2=0x55b392a52fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.031+0000 7f222ca11740 1 --2- 172.21.15.45:0/3008614812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a26280 0x55b392a52fe0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f2214060c00 tx=0x7f22140614d0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.031+0000 7f222ca11740 1 -- 172.21.15.45:0/3008614812 shutdown_connections 2024-05-08T05:48:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.031+0000 7f222ca11740 1 --2- 172.21.15.45:0/3008614812 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55b392a26d50 0x55b392a43a20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.031+0000 7f222ca11740 1 --2- 172.21.15.45:0/3008614812 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a517c0 0x55b392a26770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.031+0000 7f222ca11740 1 --2- 172.21.15.45:0/3008614812 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a43160 0x55b392a51280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.031+0000 7f222ca11740 1 --2- 172.21.15.45:0/3008614812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a26280 0x55b392a52fe0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.031+0000 7f22027fc640 0 client.7128 ms_handle_reset on v2:172.21.15.63:6835/3194497846 2024-05-08T05:48:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.031+0000 7f222ca11740 1 -- 172.21.15.45:0/3008614812 >> 172.21.15.45:0/3008614812 conn(0x55b392a25b40 msgr2=0x55b392a40e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.031+0000 7f222ca11740 1 -- 172.21.15.45:0/3008614812 shutdown_connections 2024-05-08T05:48:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.031+0000 7f222ca11740 1 -- 172.21.15.45:0/3008614812 wait complete. 2024-05-08T05:48:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.031+0000 7f222ca11740 20 client.7128 trim_cache size 0 max 16384 2024-05-08T05:48:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.049+0000 7f222ca11740 1 Processor -- start 2024-05-08T05:48:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.049+0000 7f222ca11740 1 -- start start 2024-05-08T05:48:18.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.050+0000 7f222ca11740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a26280 0x55b3929f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.050+0000 7f222ca11740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a43160 0x55b392a41730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.050+0000 7f222ca11740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a517c0 0x55b392a1e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.050+0000 7f222ca11740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2224202cc0 con 0x55b392a517c0 2024-05-08T05:48:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.050+0000 7f222ca11740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2224201ce0 con 0x55b392a43160 2024-05-08T05:48:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.050+0000 7f222ca11740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f222415c150 con 0x55b392a26280 2024-05-08T05:48:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.050+0000 7f2223fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a517c0 0x55b392a1e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.050+0000 7f2223fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a517c0 0x55b392a1e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35096/0 (socket says 172.21.15.45:35096) 2024-05-08T05:48:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.050+0000 7f2223fff640 1 -- 172.21.15.45:0/2074755371 learned_addr learned my addr 172.21.15.45:0/2074755371 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.051+0000 7f2222ffd640 1 --2- 172.21.15.45:0/2074755371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a43160 0x55b392a41730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.051+0000 7f2223fff640 1 -- 172.21.15.45:0/2074755371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a26280 msgr2=0x55b3929f4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.051+0000 7f22237fe640 1 --2- 172.21.15.45:0/2074755371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a26280 0x55b3929f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.051+0000 7f2223fff640 1 --2- 172.21.15.45:0/2074755371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a26280 0x55b3929f4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.051+0000 7f2223fff640 1 -- 172.21.15.45:0/2074755371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a43160 msgr2=0x55b392a41730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.051+0000 7f2223fff640 1 --2- 172.21.15.45:0/2074755371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a43160 0x55b392a41730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.051+0000 7f2223fff640 1 -- 172.21.15.45:0/2074755371 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b392a43d90 con 0x55b392a517c0 2024-05-08T05:48:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.051+0000 7f2222ffd640 1 --2- 172.21.15.45:0/2074755371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a43160 0x55b392a41730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:48:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.051+0000 7f2223fff640 1 --2- 172.21.15.45:0/2074755371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a517c0 0x55b392a1e4e0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f221c00a870 tx=0x7f221c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5378d664f76559 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.051+0000 7f2208ff9640 1 -- 172.21.15.45:0/2074755371 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c0011c0 con 0x55b392a517c0 2024-05-08T05:48:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.051+0000 7f2208ff9640 1 -- 172.21.15.45:0/2074755371 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221c046070 con 0x55b392a517c0 2024-05-08T05:48:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.051+0000 7f2208ff9640 1 -- 172.21.15.45:0/2074755371 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c001830 con 0x55b392a517c0 2024-05-08T05:48:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.052+0000 7f222ca11740 1 -- 172.21.15.45:0/2074755371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a517c0 msgr2=0x55b392a1e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.052+0000 7f222ca11740 1 --2- 172.21.15.45:0/2074755371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a517c0 0x55b392a1e4e0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f221c00a870 tx=0x7f221c00a060 comp rx=0 tx=0).stop 2024-05-08T05:48:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.052+0000 7f222ca11740 1 -- 172.21.15.45:0/2074755371 shutdown_connections 2024-05-08T05:48:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.052+0000 7f222ca11740 1 --2- 172.21.15.45:0/2074755371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a517c0 0x55b392a1e4e0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.052+0000 7f222ca11740 1 --2- 172.21.15.45:0/2074755371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a43160 0x55b392a41730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.052+0000 7f222ca11740 1 --2- 172.21.15.45:0/2074755371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a26280 0x55b3929f4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.052+0000 7f222ca11740 1 -- 172.21.15.45:0/2074755371 >> 172.21.15.45:0/2074755371 conn(0x55b392a25b40 msgr2=0x55b3929f9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.052+0000 7f222ca11740 1 -- 172.21.15.45:0/2074755371 shutdown_connections 2024-05-08T05:48:18.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.052+0000 7f222ca11740 1 -- 172.21.15.45:0/2074755371 wait complete. 2024-05-08T05:48:18.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.052+0000 7f222ca11740 1 -- start start 2024-05-08T05:48:18.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.053+0000 7f222ca11740 1 Processor -- start 2024-05-08T05:48:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.053+0000 7f222ca11740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a26280 0x55b392abd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.053+0000 7f222ca11740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a43160 0x55b3929f9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.053+0000 7f222ca11740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a517c0 0x55b392a3e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.053+0000 7f222ca11740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2224201610 con 0x55b392a43160 2024-05-08T05:48:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.053+0000 7f222ca11740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55b392a399e0 con 0x55b392a26280 2024-05-08T05:48:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.053+0000 7f222ca11740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55b392a42490 con 0x55b392a517c0 2024-05-08T05:48:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.053+0000 7f222ca11740 10 client.0 _ll_register_callbacks cb 0x7f22140119f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:48:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.053+0000 7f2222ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a43160 0x55b3929f9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.053+0000 7f2222ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a43160 0x55b3929f9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35112/0 (socket says 172.21.15.45:35112) 2024-05-08T05:48:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.053+0000 7f2222ffd640 1 -- 172.21.15.45:0/100284835 learned_addr learned my addr 172.21.15.45:0/100284835 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.054+0000 7f22237fe640 1 --2- 172.21.15.45:0/100284835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a26280 0x55b392abd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.054+0000 7f2223fff640 1 --2- 172.21.15.45:0/100284835 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a517c0 0x55b392a3e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.054+0000 7f2222ffd640 1 -- 172.21.15.45:0/100284835 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a517c0 msgr2=0x55b392a3e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.054+0000 7f2222ffd640 1 --2- 172.21.15.45:0/100284835 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a517c0 0x55b392a3e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.054+0000 7f2222ffd640 1 -- 172.21.15.45:0/100284835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a26280 msgr2=0x55b392abd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.054+0000 7f2222ffd640 1 --2- 172.21.15.45:0/100284835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a26280 0x55b392abd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.054+0000 7f2222ffd640 1 -- 172.21.15.45:0/100284835 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3926310f0 con 0x55b392a43160 2024-05-08T05:48:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.054+0000 7f2223fff640 1 --2- 172.21.15.45:0/100284835 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a517c0 0x55b392a3e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.054+0000 7f2222ffd640 1 --2- 172.21.15.45:0/100284835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a43160 0x55b3929f9a10 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2218009fd0 tx=0x7f2218004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6495fe91d4db2b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.054+0000 7f22027fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:48:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.054+0000 7f22027fc640 1 -- 172.21.15.45:0/100284835 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221802d4e0 con 0x55b392a43160 2024-05-08T05:48:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.054+0000 7f22027fc640 1 -- 172.21.15.45:0/100284835 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f221802d680 con 0x55b392a43160 2024-05-08T05:48:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.054+0000 7f222ca11740 1 -- 172.21.15.45:0/100284835 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f221c046070 con 0x55b392a43160 2024-05-08T05:48:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.054+0000 7f22027fc640 1 -- 172.21.15.45:0/100284835 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f221803d050 con 0x55b392a43160 2024-05-08T05:48:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.054+0000 7f222ca11740 10 client.7134 fetch_fsmap learned FSMap version 67 2024-05-08T05:48:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.054+0000 7f222ca11740 1 -- 172.21.15.45:0/100284835 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55b392a42140 con 0x55b392a43160 2024-05-08T05:48:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.054+0000 7f22027fc640 1 -- 172.21.15.45:0/100284835 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221c046070 con 0x55b392a43160 2024-05-08T05:48:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.055+0000 7f22027fc640 1 -- 172.21.15.45:0/100284835 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f2218040020 con 0x55b392a43160 2024-05-08T05:48:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.055+0000 7f222ca11740 10 client.7134 fetch_fsmap finished waiting for FSMap version 67 2024-05-08T05:48:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.055+0000 7f222ca11740 10 client.7134 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-05-08T05:48:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.055+0000 7f222ca11740 20 client.7134 populate_metadata read hostname 'smithi045' 2024-05-08T05:48:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.055+0000 7f222ca11740 1 --2- 172.21.15.45:0/100284835 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55b392a3e880 0x55b392a373e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.055+0000 7f222ca11740 4 client.7134 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:48:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.055+0000 7f222ca11740 1 -- 172.21.15.45:0/100284835 --> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b3929679b0 con 0x55b392a3e880 2024-05-08T05:48:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.055+0000 7f22237fe640 1 --2- 172.21.15.45:0/100284835 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55b392a3e880 0x55b392a373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.055+0000 7f22237fe640 1 --2- 172.21.15.45:0/100284835 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55b392a3e880 0x55b392a373e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=901eae68dc2084ba server_cookie=af277d9b56b3a24e in_seq=0 out_seq=0 2024-05-08T05:48:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.055+0000 7f22027fc640 10 client.7134 ms_handle_connect on v2:172.21.15.63:6835/3194497846 2024-05-08T05:48:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.055+0000 7f22027fc640 1 -- 172.21.15.45:0/100284835 <== mds.0 v2:172.21.15.63:6835/3194497846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b3929679b0 con 0x55b392a3e880 2024-05-08T05:48:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.055+0000 7f22027fc640 10 client.7134 handle_command_reply: tid=0 2024-05-08T05:48:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.055+0000 7f222ca11740 1 client.7134 shutdown 2024-05-08T05:48:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f222ca11740 1 -- 172.21.15.45:0/100284835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a43160 msgr2=0x55b3929f9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f222ca11740 1 --2- 172.21.15.45:0/100284835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a43160 0x55b3929f9a10 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2218009fd0 tx=0x7f2218004730 comp rx=0 tx=0).stop 2024-05-08T05:48:18.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f222ca11740 1 -- 172.21.15.45:0/100284835 shutdown_connections 2024-05-08T05:48:18.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f222ca11740 1 --2- 172.21.15.45:0/100284835 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x55b392a3e880 0x55b392a373e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f222ca11740 1 --2- 172.21.15.45:0/100284835 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55b392a517c0 0x55b392a3e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f222ca11740 1 --2- 172.21.15.45:0/100284835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b392a43160 0x55b3929f9a10 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f22027fc640 0 client.7134 ms_handle_reset on v2:172.21.15.63:6835/3194497846 2024-05-08T05:48:18.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f222ca11740 1 --2- 172.21.15.45:0/100284835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55b392a26280 0x55b392abd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f222ca11740 1 -- 172.21.15.45:0/100284835 >> 172.21.15.45:0/100284835 conn(0x55b392a25b40 msgr2=0x55b392a1e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:18.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f222ca11740 1 -- 172.21.15.45:0/100284835 shutdown_connections 2024-05-08T05:48:18.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f222ca11740 1 -- 172.21.15.45:0/100284835 wait complete. 2024-05-08T05:48:18.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f222ca11740 20 client.7134 trim_cache size 0 max 16384 2024-05-08T05:48:18.063 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:48:18.063 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-05-08T05:48:18.063 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-05-08T05:48:18.063 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:48:18.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f2229cfc640 1 -- 172.21.15.45:0/302613128 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2210076300 msgr2=0x7f22100787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f2229cfc640 1 --2- 172.21.15.45:0/302613128 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2210076300 0x7f22100787c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f22241f5270 tx=0x7f2218008040 comp rx=0 tx=0).stop 2024-05-08T05:48:18.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f2229cfc640 1 -- 172.21.15.45:0/302613128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22241f4ab0 msgr2=0x7f222415f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f2229cfc640 1 --2- 172.21.15.45:0/302613128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22241f4ab0 0x7f222415f5c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2214011aa0 tx=0x7f2214011f60 comp rx=0 tx=0).stop 2024-05-08T05:48:18.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f2229cfc640 1 -- 172.21.15.45:0/302613128 shutdown_connections 2024-05-08T05:48:18.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f2229cfc640 1 --2- 172.21.15.45:0/302613128 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2210076300 0x7f22100787c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f2229cfc640 1 --2- 172.21.15.45:0/302613128 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f222415d2e0 0x7f222415d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f2229cfc640 1 --2- 172.21.15.45:0/302613128 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f22241f4e90 0x7f222415ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f2229cfc640 1 --2- 172.21.15.45:0/302613128 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22241f4ab0 0x7f222415f5c0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f2229cfc640 1 -- 172.21.15.45:0/302613128 >> 172.21.15.45:0/302613128 conn(0x7f22240728c0 msgr2=0x7f2224073540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:18.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.056+0000 7f2229cfc640 1 -- 172.21.15.45:0/302613128 shutdown_connections 2024-05-08T05:48:18.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.057+0000 7f2229cfc640 1 -- 172.21.15.45:0/302613128 wait complete. 2024-05-08T05:48:18.068 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T05:48:18.068 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:48:18.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.171+0000 7fad0a1af640 1 Processor -- start 2024-05-08T05:48:18.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.171+0000 7fad0a1af640 1 -- start start 2024-05-08T05:48:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.172+0000 7fad0a1af640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fad041629a0 0x7fad040d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.172+0000 7fad0a1af640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fad040d2dc0 0x7fad040d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.172+0000 7fad0a1af640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad040c8ed0 0x7fad040c9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.172+0000 7fad0a1af640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fad040d3330 con 0x7fad041629a0 2024-05-08T05:48:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.172+0000 7fad0a1af640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fad040d2170 con 0x7fad040d2dc0 2024-05-08T05:48:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.172+0000 7fad0a1af640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fad040d2450 con 0x7fad040c8ed0 2024-05-08T05:48:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.172+0000 7fad037fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fad041629a0 0x7fad040d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.172+0000 7fad037fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fad041629a0 0x7fad040d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35122/0 (socket says 172.21.15.45:35122) 2024-05-08T05:48:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.172+0000 7fad037fe640 1 -- 172.21.15.45:0/713051591 learned_addr learned my addr 172.21.15.45:0/713051591 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.173+0000 7fad03fff640 1 --2- 172.21.15.45:0/713051591 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad040c8ed0 0x7fad040c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.173+0000 7fad037fe640 1 -- 172.21.15.45:0/713051591 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad040c8ed0 msgr2=0x7fad040c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.173+0000 7fad02ffd640 1 --2- 172.21.15.45:0/713051591 >> v2:172.21.15.63:3300/0 conn(0x7fad040d2dc0 0x7fad040d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.173+0000 7fad037fe640 1 --2- 172.21.15.45:0/713051591 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad040c8ed0 0x7fad040c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.173+0000 7fad037fe640 1 -- 172.21.15.45:0/713051591 >> v2:172.21.15.63:3300/0 conn(0x7fad040d2dc0 msgr2=0x7fad040d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.173+0000 7fad037fe640 1 --2- 172.21.15.45:0/713051591 >> v2:172.21.15.63:3300/0 conn(0x7fad040d2dc0 0x7fad040d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.173+0000 7fad037fe640 1 -- 172.21.15.45:0/713051591 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad040d2730 con 0x7fad041629a0 2024-05-08T05:48:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.173+0000 7fad03fff640 1 --2- 172.21.15.45:0/713051591 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad040c8ed0 0x7fad040c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:48:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.173+0000 7fad02ffd640 1 --2- 172.21.15.45:0/713051591 >> v2:172.21.15.63:3300/0 conn(0x7fad040d2dc0 0x7fad040d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:48:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.173+0000 7fad037fe640 1 --2- 172.21.15.45:0/713051591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad041629a0 0x7fad040d35c0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fad040b7ae0 tx=0x7facf402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cadfa4031596528 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.174+0000 7fad027fc640 1 -- 172.21.15.45:0/713051591 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4037070 con 0x7fad041629a0 2024-05-08T05:48:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.174+0000 7fad027fc640 1 -- 172.21.15.45:0/713051591 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facf402ae00 con 0x7fad041629a0 2024-05-08T05:48:18.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.174+0000 7fad027fc640 1 -- 172.21.15.45:0/713051591 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf4032660 con 0x7fad041629a0 2024-05-08T05:48:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.174+0000 7fad0a1af640 1 -- 172.21.15.45:0/713051591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad041629a0 msgr2=0x7fad040d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.174+0000 7fad0a1af640 1 --2- 172.21.15.45:0/713051591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad041629a0 0x7fad040d35c0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fad040b7ae0 tx=0x7facf402a0b0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.174+0000 7fad0a1af640 1 -- 172.21.15.45:0/713051591 shutdown_connections 2024-05-08T05:48:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.174+0000 7fad0a1af640 1 --2- 172.21.15.45:0/713051591 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad040c8ed0 0x7fad040c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.174+0000 7fad0a1af640 1 --2- 172.21.15.45:0/713051591 >> v2:172.21.15.63:3300/0 conn(0x7fad040d2dc0 0x7fad040d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.174+0000 7fad0a1af640 1 --2- 172.21.15.45:0/713051591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad041629a0 0x7fad040d35c0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.174+0000 7fad0a1af640 1 -- 172.21.15.45:0/713051591 >> 172.21.15.45:0/713051591 conn(0x7fad040d4fe0 msgr2=0x7fad040d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.174+0000 7fad0a1af640 1 -- 172.21.15.45:0/713051591 shutdown_connections 2024-05-08T05:48:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.174+0000 7fad0a1af640 1 -- 172.21.15.45:0/713051591 wait complete. 2024-05-08T05:48:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.175+0000 7fad0a1af640 1 Processor -- start 2024-05-08T05:48:18.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.176+0000 7fad0a1af640 1 -- start start 2024-05-08T05:48:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.176+0000 7fad0a1af640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad040c8ed0 0x7fad0415e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.176+0000 7fad0a1af640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad040d2dc0 0x7fad040a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.176+0000 7fad0a1af640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad041629a0 0x7fad0415f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.176+0000 7fad0a1af640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fad040cf440 con 0x7fad041629a0 2024-05-08T05:48:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.176+0000 7fad0a1af640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fad040cf2c0 con 0x7fad040d2dc0 2024-05-08T05:48:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.176+0000 7fad0a1af640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fad040c98a0 con 0x7fad040c8ed0 2024-05-08T05:48:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.177+0000 7fad03fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad041629a0 0x7fad0415f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.177+0000 7fad03fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad041629a0 0x7fad0415f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35132/0 (socket says 172.21.15.45:35132) 2024-05-08T05:48:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.177+0000 7fad03fff640 1 -- 172.21.15.45:0/1343474624 learned_addr learned my addr 172.21.15.45:0/1343474624 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.177+0000 7fad02ffd640 1 --2- 172.21.15.45:0/1343474624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad040d2dc0 0x7fad040a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.177+0000 7fad037fe640 1 --2- 172.21.15.45:0/1343474624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad040c8ed0 0x7fad0415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.177+0000 7fad03fff640 1 -- 172.21.15.45:0/1343474624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad040c8ed0 msgr2=0x7fad0415e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.177+0000 7fad03fff640 1 --2- 172.21.15.45:0/1343474624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad040c8ed0 0x7fad0415e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.177+0000 7fad03fff640 1 -- 172.21.15.45:0/1343474624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad040d2dc0 msgr2=0x7fad040a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.177+0000 7fad03fff640 1 --2- 172.21.15.45:0/1343474624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad040d2dc0 0x7fad040a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.177+0000 7fad03fff640 1 -- 172.21.15.45:0/1343474624 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad040a2460 con 0x7fad041629a0 2024-05-08T05:48:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.177+0000 7fad03fff640 1 --2- 172.21.15.45:0/1343474624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad041629a0 0x7fad0415f3f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7facfc00cef0 tx=0x7facfc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.177+0000 7facebfff640 1 -- 172.21.15.45:0/1343474624 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc01a070 con 0x7fad041629a0 2024-05-08T05:48:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.177+0000 7facebfff640 1 -- 172.21.15.45:0/1343474624 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facfc00d840 con 0x7fad041629a0 2024-05-08T05:48:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.177+0000 7fad0a1af640 1 -- 172.21.15.45:0/1343474624 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad040a2740 con 0x7fad041629a0 2024-05-08T05:48:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.177+0000 7facebfff640 1 -- 172.21.15.45:0/1343474624 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc00dc30 con 0x7fad041629a0 2024-05-08T05:48:18.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.177+0000 7fad0a1af640 1 -- 172.21.15.45:0/1343474624 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad040b9940 con 0x7fad041629a0 2024-05-08T05:48:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.178+0000 7fad0a1af640 1 -- 172.21.15.45:0/1343474624 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad040c9660 con 0x7fad041629a0 2024-05-08T05:48:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.179+0000 7facebfff640 1 -- 172.21.15.45:0/1343474624 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7facfc01d070 con 0x7fad041629a0 2024-05-08T05:48:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.179+0000 7facebfff640 1 --2- 172.21.15.45:0/1343474624 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7facf00762d0 0x7facf0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.180+0000 7facebfff640 1 -- 172.21.15.45:0/1343474624 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7facfc097fe0 con 0x7fad041629a0 2024-05-08T05:48:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.180+0000 7fad037fe640 1 --2- 172.21.15.45:0/1343474624 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7facf00762d0 0x7facf0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.180+0000 7fad037fe640 1 --2- 172.21.15.45:0/1343474624 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7facf00762d0 0x7facf0078790 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7facf4034000 tx=0x7facf402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:18.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.184+0000 7facebfff640 1 -- 172.21.15.45:0/1343474624 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fad040c9660 con 0x7fad041629a0 2024-05-08T05:48:18.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.413+0000 7fad0a1af640 1 -- 172.21.15.45:0/1343474624 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad040693f0 con 0x7fad041629a0 2024-05-08T05:48:18.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.413+0000 7facebfff640 1 -- 172.21.15.45:0/1343474624 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5184 (secure 0 0 0) 0x7facfc0610b0 con 0x7fad041629a0 2024-05-08T05:48:18.414 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:48:18.414 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2616298287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2616298287},{"type":"v1","addr":"172.21.15.63:6836","nonce":2616298287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6744,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2854804427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2854804427},{"type":"v1","addr":"172.21.15.45:6834","nonce":2854804427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2945763952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2945763952},{"type":"v1","addr":"172.21.15.45:6835","nonce":2945763952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:47:00.211837+0000","modified":"2024-05-08T05:47:20.541349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6759},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6759":{"gid":6759,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":9,"addr":"172.21.15.63:6837/3194497846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":3194497846},{"type":"v1","addr":"172.21.15.63:6837","nonce":3194497846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6759,"qdb_cluster":[6759]},"id":6}]} 2024-05-08T05:48:18.414 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 67 2024-05-08T05:48:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.416+0000 7fad0a1af640 1 -- 172.21.15.45:0/1343474624 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7facf00762d0 msgr2=0x7facf0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.416+0000 7fad0a1af640 1 --2- 172.21.15.45:0/1343474624 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7facf00762d0 0x7facf0078790 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7facf4034000 tx=0x7facf402e000 comp rx=0 tx=0).stop 2024-05-08T05:48:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.416+0000 7fad0a1af640 1 -- 172.21.15.45:0/1343474624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad041629a0 msgr2=0x7fad0415f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.416+0000 7fad0a1af640 1 --2- 172.21.15.45:0/1343474624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad041629a0 0x7fad0415f3f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7facfc00cef0 tx=0x7facfc00a710 comp rx=0 tx=0).stop 2024-05-08T05:48:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.416+0000 7fad0a1af640 1 -- 172.21.15.45:0/1343474624 shutdown_connections 2024-05-08T05:48:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.416+0000 7fad0a1af640 1 --2- 172.21.15.45:0/1343474624 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7facf00762d0 0x7facf0078790 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.416+0000 7fad0a1af640 1 --2- 172.21.15.45:0/1343474624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad041629a0 0x7fad0415f3f0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.416+0000 7fad0a1af640 1 --2- 172.21.15.45:0/1343474624 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fad040d2dc0 0x7fad040a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.416+0000 7fad0a1af640 1 --2- 172.21.15.45:0/1343474624 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fad040c8ed0 0x7fad0415e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.416+0000 7fad0a1af640 1 -- 172.21.15.45:0/1343474624 >> 172.21.15.45:0/1343474624 conn(0x7fad040d4fe0 msgr2=0x7fad040d35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.416+0000 7fad0a1af640 1 -- 172.21.15.45:0/1343474624 shutdown_connections 2024-05-08T05:48:18.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.416+0000 7fad0a1af640 1 -- 172.21.15.45:0/1343474624 wait complete. 2024-05-08T05:48:18.427 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2024-05-08T05:48:18.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.532+0000 7f1bc1d89640 1 Processor -- start 2024-05-08T05:48:18.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.533+0000 7f1bc1d89640 1 -- start start 2024-05-08T05:48:18.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.533+0000 7f1bc1d89640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1bbc0cfea0 0x7f1bbc1633e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.533+0000 7f1bc1d89640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bbc0d0500 con 0x7f1bbc0b21f0 2024-05-08T05:48:18.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.533+0000 7f1bc1d89640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bbc06da60 con 0x7f1bbc162f50 2024-05-08T05:48:18.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.533+0000 7f1bc1d89640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f1bbc06d810 con 0x7f1bbc0cfea0 2024-05-08T05:48:18.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.533+0000 7f1bbbfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1bbc0cfea0 0x7f1bbc1633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.533+0000 7f1bbb7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1bbc0b21f0 0x7f1bbc06e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:45670/0 (socket says 172.21.15.45:45670) 2024-05-08T05:48:18.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.533+0000 7f1bbbfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1bbc0cfea0 0x7f1bbc1633e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35148/0 (socket says 172.21.15.45:35148) 2024-05-08T05:48:18.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.533+0000 7f1bbb7fe640 1 -- 172.21.15.45:0/222766706 learned_addr learned my addr 172.21.15.45:0/222766706 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:48:18.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.534+0000 7f1bbbfff640 1 -- 172.21.15.45:0/222766706 >> v1:172.21.15.63:6789/0 conn(0x7f1bbc162f50 legacy=0x7f1bbc0cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.534+0000 7f1bbbfff640 1 -- 172.21.15.45:0/222766706 >> v1:172.21.15.45:6789/0 conn(0x7f1bbc0b21f0 legacy=0x7f1bbc06e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.534+0000 7f1bba7fc640 1 -- 172.21.15.45:0/222766706 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1109412266 0 0) 0x7f1bbc0d0500 con 0x7f1bbc0b21f0 2024-05-08T05:48:18.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.534+0000 7f1bbbfff640 1 -- 172.21.15.45:0/222766706 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bbc06dd40 con 0x7f1bbc0cfea0 2024-05-08T05:48:18.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.535+0000 7f1bbbfff640 1 --2- 172.21.15.45:0/222766706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bbc0cfea0 0x7f1bbc1633e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009d90 tx=0x7f1bb402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9c010d83a4f5ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:18.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.535+0000 7f1bba7fc640 1 -- 172.21.15.45:0/222766706 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4031600 con 0x7f1bbc0cfea0 2024-05-08T05:48:18.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.535+0000 7f1bba7fc640 1 -- 172.21.15.45:0/222766706 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb40317a0 con 0x7f1bbc0cfea0 2024-05-08T05:48:18.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.535+0000 7f1bba7fc640 1 -- 172.21.15.45:0/222766706 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4031b70 con 0x7f1bbc0cfea0 2024-05-08T05:48:18.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.535+0000 7f1bc1d89640 1 -- 172.21.15.45:0/222766706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bbc0cfea0 msgr2=0x7f1bbc1633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.535+0000 7f1bc1d89640 1 --2- 172.21.15.45:0/222766706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bbc0cfea0 0x7f1bbc1633e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009d90 tx=0x7f1bb402a5d0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.535+0000 7f1bc1d89640 1 -- 172.21.15.45:0/222766706 shutdown_connections 2024-05-08T05:48:18.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.535+0000 7f1bc1d89640 1 --2- 172.21.15.45:0/222766706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bbc0cfea0 0x7f1bbc1633e0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.535+0000 7f1bc1d89640 1 -- 172.21.15.45:0/222766706 >> 172.21.15.45:0/222766706 conn(0x7f1bbc09ba40 msgr2=0x7f1bbc09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:18.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.535+0000 7f1bc1d89640 1 -- 172.21.15.45:0/222766706 shutdown_connections 2024-05-08T05:48:18.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.535+0000 7f1bc1d89640 1 -- 172.21.15.45:0/222766706 wait complete. 2024-05-08T05:48:18.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.536+0000 7f1bc1d89640 1 Processor -- start 2024-05-08T05:48:18.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.537+0000 7f1bc1d89640 1 -- start start 2024-05-08T05:48:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.537+0000 7f1bc1d89640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bbc0b21f0 0x7f1bbc0c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.537+0000 7f1bc1d89640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bbc0cfea0 0x7f1bbc0cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.537+0000 7f1bc1d89640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bbc162f50 0x7f1bbc0cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.537+0000 7f1bc1d89640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc0cd480 con 0x7f1bbc0b21f0 2024-05-08T05:48:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.537+0000 7f1bc1d89640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc06e0b0 con 0x7f1bbc162f50 2024-05-08T05:48:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.537+0000 7f1bc1d89640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1bbc0cd600 con 0x7f1bbc0cfea0 2024-05-08T05:48:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.537+0000 7f1bbb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bbc0b21f0 0x7f1bbc0c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.538+0000 7f1bbb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bbc0b21f0 0x7f1bbc0c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35162/0 (socket says 172.21.15.45:35162) 2024-05-08T05:48:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.538+0000 7f1bbb7fe640 1 -- 172.21.15.45:0/4044634340 learned_addr learned my addr 172.21.15.45:0/4044634340 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.538+0000 7f1bbbfff640 1 --2- 172.21.15.45:0/4044634340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bbc162f50 0x7f1bbc0cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.538+0000 7f1bbaffd640 1 --2- 172.21.15.45:0/4044634340 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bbc0cfea0 0x7f1bbc0cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.538+0000 7f1bbb7fe640 1 -- 172.21.15.45:0/4044634340 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bbc0cfea0 msgr2=0x7f1bbc0cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.538+0000 7f1bbb7fe640 1 --2- 172.21.15.45:0/4044634340 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bbc0cfea0 0x7f1bbc0cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.538+0000 7f1bbb7fe640 1 -- 172.21.15.45:0/4044634340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bbc162f50 msgr2=0x7f1bbc0cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.538+0000 7f1bbb7fe640 1 --2- 172.21.15.45:0/4044634340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bbc162f50 0x7f1bbc0cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.538+0000 7f1bbb7fe640 1 -- 172.21.15.45:0/4044634340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bbc0c9c10 con 0x7f1bbc0b21f0 2024-05-08T05:48:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.538+0000 7f1bbb7fe640 1 --2- 172.21.15.45:0/4044634340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bbc0b21f0 0x7f1bbc0c8450 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00c480 tx=0x7f1bac00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.538+0000 7f1ba3fff640 1 -- 172.21.15.45:0/4044634340 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac018070 con 0x7f1bbc0b21f0 2024-05-08T05:48:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.538+0000 7f1ba3fff640 1 -- 172.21.15.45:0/4044634340 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bac0074e0 con 0x7f1bbc0b21f0 2024-05-08T05:48:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.538+0000 7f1bc1d89640 1 -- 172.21.15.45:0/4044634340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bbc0c8b70 con 0x7f1bbc0b21f0 2024-05-08T05:48:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.538+0000 7f1ba3fff640 1 -- 172.21.15.45:0/4044634340 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac004a20 con 0x7f1bbc0b21f0 2024-05-08T05:48:18.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.538+0000 7f1bc1d89640 1 -- 172.21.15.45:0/4044634340 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bbc0c9130 con 0x7f1bbc0b21f0 2024-05-08T05:48:18.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.540+0000 7f1ba3fff640 1 -- 172.21.15.45:0/4044634340 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1bac01b070 con 0x7f1bbc0b21f0 2024-05-08T05:48:18.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.540+0000 7f1ba3fff640 1 --2- 172.21.15.45:0/4044634340 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1ba80762d0 0x7f1ba8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.540+0000 7f1ba3fff640 1 -- 172.21.15.45:0/4044634340 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f1bac096b90 con 0x7f1bbc0b21f0 2024-05-08T05:48:18.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.541+0000 7f1bbaffd640 1 --2- 172.21.15.45:0/4044634340 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1ba80762d0 0x7f1ba8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.541+0000 7f1bbaffd640 1 --2- 172.21.15.45:0/4044634340 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1ba80762d0 0x7f1ba8078790 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0d38f0 tx=0x7f1bb0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:18.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.541+0000 7f1bc4b79740 1 Processor -- start 2024-05-08T05:48:18.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.541+0000 7f1bc4b79740 1 -- start start 2024-05-08T05:48:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.542+0000 7f1bc4b79740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1de6d0 0x561acd1912f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.542+0000 7f1bc4b79740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1c17f0 0x561acd1c0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.542+0000 7f1bc4b79740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1c10b0 0x561acd195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.542+0000 7f1bc4b79740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561acd081920 con 0x561acd1c10b0 2024-05-08T05:48:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.542+0000 7f1bc4b79740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561acd0f6dd0 con 0x561acd1de6d0 2024-05-08T05:48:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.542+0000 7f1bc4b79740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x561acd0a73e0 con 0x561acd1c17f0 2024-05-08T05:48:18.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.542+0000 7f1bbbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1c10b0 0x561acd195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.542+0000 7f1bbb7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1de6d0 0x561acd1912f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.542+0000 7f1bbbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1c10b0 0x561acd195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35172/0 (socket says 172.21.15.45:35172) 2024-05-08T05:48:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.542+0000 7f1bbbfff640 1 -- 172.21.15.45:0/2827304395 learned_addr learned my addr 172.21.15.45:0/2827304395 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.542+0000 7f1bbaffd640 1 --2- 172.21.15.45:0/2827304395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1c17f0 0x561acd1c0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.543+0000 7f1bbbfff640 1 -- 172.21.15.45:0/2827304395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1c17f0 msgr2=0x561acd1c0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.543+0000 7f1bbbfff640 1 --2- 172.21.15.45:0/2827304395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1c17f0 0x561acd1c0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.543+0000 7f1bbbfff640 1 -- 172.21.15.45:0/2827304395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1de6d0 msgr2=0x561acd1912f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.543+0000 7f1bbbfff640 1 --2- 172.21.15.45:0/2827304395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1de6d0 0x561acd1912f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.543+0000 7f1bbbfff640 1 -- 172.21.15.45:0/2827304395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561acced87c0 con 0x561acd1c10b0 2024-05-08T05:48:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.543+0000 7f1bbbfff640 1 --2- 172.21.15.45:0/2827304395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1c10b0 0x561acd195ec0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009e80 tx=0x7f1bb402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913628f0b09d4783 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.543+0000 7f1ba17fa640 1 -- 172.21.15.45:0/2827304395 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4031a80 con 0x561acd1c10b0 2024-05-08T05:48:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.543+0000 7f1ba17fa640 1 -- 172.21.15.45:0/2827304395 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb4041070 con 0x561acd1c10b0 2024-05-08T05:48:18.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.543+0000 7f1ba17fa640 1 -- 172.21.15.45:0/2827304395 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4031e20 con 0x561acd1c10b0 2024-05-08T05:48:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.544+0000 7f1bc4b79740 1 -- 172.21.15.45:0/2827304395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1c10b0 msgr2=0x561acd195ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.544+0000 7f1bc4b79740 1 --2- 172.21.15.45:0/2827304395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1c10b0 0x561acd195ec0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009e80 tx=0x7f1bb402e000 comp rx=0 tx=0).stop 2024-05-08T05:48:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.544+0000 7f1bc4b79740 1 -- 172.21.15.45:0/2827304395 shutdown_connections 2024-05-08T05:48:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.544+0000 7f1bc4b79740 1 --2- 172.21.15.45:0/2827304395 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1c17f0 0x561acd1c0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.544+0000 7f1bc4b79740 1 --2- 172.21.15.45:0/2827304395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1c10b0 0x561acd195ec0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.544+0000 7f1bc4b79740 1 --2- 172.21.15.45:0/2827304395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1de6d0 0x561acd1912f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.544+0000 7f1bc4b79740 1 -- 172.21.15.45:0/2827304395 >> 172.21.15.45:0/2827304395 conn(0x561acd193560 msgr2=0x561acd1dbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.544+0000 7f1bc4b79740 1 -- 172.21.15.45:0/2827304395 shutdown_connections 2024-05-08T05:48:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.544+0000 7f1bc4b79740 1 -- 172.21.15.45:0/2827304395 wait complete. 2024-05-08T05:48:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.544+0000 7f1bc4b79740 1 -- start start 2024-05-08T05:48:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.544+0000 7f1bc4b79740 1 Processor -- start 2024-05-08T05:48:18.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.545+0000 7f1bc4b79740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1c17f0 0x561acd1bbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.545+0000 7f1bc4b79740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1de6d0 0x561acd1ebbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.545+0000 7f1bc4b79740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1ec0f0 0x561acd1c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.545+0000 7f1bc4b79740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561acd0ff440 con 0x561acd1c17f0 2024-05-08T05:48:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.545+0000 7f1bc4b79740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561acd0fa120 con 0x561acd1ec0f0 2024-05-08T05:48:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.545+0000 7f1bc4b79740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x561acd108210 con 0x561acd1de6d0 2024-05-08T05:48:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.545+0000 7f1bc4b79740 10 client.0 _ll_register_callbacks cb 0x561acd09f690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:48:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.545+0000 7f1bbb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1c17f0 0x561acd1bbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.545+0000 7f1bbb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1c17f0 0x561acd1bbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35188/0 (socket says 172.21.15.45:35188) 2024-05-08T05:48:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.545+0000 7f1bbb7fe640 1 -- 172.21.15.45:0/81972113 learned_addr learned my addr 172.21.15.45:0/81972113 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1bbbfff640 1 --2- 172.21.15.45:0/81972113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1ec0f0 0x561acd1c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1bbaffd640 1 --2- 172.21.15.45:0/81972113 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1de6d0 0x561acd1ebbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1bbb7fe640 1 -- 172.21.15.45:0/81972113 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1de6d0 msgr2=0x561acd1ebbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1bbb7fe640 1 --2- 172.21.15.45:0/81972113 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1de6d0 0x561acd1ebbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1bbb7fe640 1 -- 172.21.15.45:0/81972113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1ec0f0 msgr2=0x561acd1c3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1bbb7fe640 1 --2- 172.21.15.45:0/81972113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1ec0f0 0x561acd1c3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1bbb7fe640 1 -- 172.21.15.45:0/81972113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561accf38f50 con 0x561acd1c17f0 2024-05-08T05:48:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1bbb7fe640 1 --2- 172.21.15.45:0/81972113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1c17f0 0x561acd1bbcc0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1bac05cec0 tx=0x7f1bac05f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50db8c23b82e064c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1b9a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:48:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1b9a7fc640 1 -- 172.21.15.45:0/81972113 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac068a20 con 0x561acd1c17f0 2024-05-08T05:48:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1b9a7fc640 1 -- 172.21.15.45:0/81972113 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f1bac004dd0 con 0x561acd1c17f0 2024-05-08T05:48:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1bc4b79740 1 -- 172.21.15.45:0/81972113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bb4041070 con 0x561acd1c17f0 2024-05-08T05:48:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1b9a7fc640 1 -- 172.21.15.45:0/81972113 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f1bac05eeb0 con 0x561acd1c17f0 2024-05-08T05:48:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1bc4b79740 10 client.7155 fetch_fsmap learned FSMap version 67 2024-05-08T05:48:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1bc4b79740 1 -- 172.21.15.45:0/81972113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x561accdcc0b0 con 0x561acd1c17f0 2024-05-08T05:48:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1b9a7fc640 1 -- 172.21.15.45:0/81972113 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb4041070 con 0x561acd1c17f0 2024-05-08T05:48:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1b9a7fc640 1 -- 172.21.15.45:0/81972113 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f1bac07a030 con 0x561acd1c17f0 2024-05-08T05:48:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1bc4b79740 10 client.7155 fetch_fsmap finished waiting for FSMap version 67 2024-05-08T05:48:18.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1bc4b79740 10 client.7155 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:48:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1bc4b79740 20 client.7155 populate_metadata read hostname 'smithi045' 2024-05-08T05:48:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1bc4b79740 1 --2- 172.21.15.45:0/81972113 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x561acd1bf090 0x561acd1c1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1bc4b79740 4 client.7155 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:48:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.546+0000 7f1bc4b79740 1 -- 172.21.15.45:0/81972113 --> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561acd108a20 con 0x561acd1bf090 2024-05-08T05:48:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.547+0000 7f1bbaffd640 1 --2- 172.21.15.45:0/81972113 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x561acd1bf090 0x561acd1c1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.547+0000 7f1b9a7fc640 10 client.7155 ms_handle_connect on v2:172.21.15.63:6835/3194497846 2024-05-08T05:48:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.547+0000 7f1bbaffd640 1 --2- 172.21.15.45:0/81972113 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x561acd1bf090 0x561acd1c1d90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a7965d8a63793aa server_cookie=e23e1b60adfa0599 in_seq=0 out_seq=0 2024-05-08T05:48:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.548+0000 7f1b9a7fc640 1 -- 172.21.15.45:0/81972113 <== mds.0 v2:172.21.15.63:6835/3194497846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561acd108a20 con 0x561acd1bf090 2024-05-08T05:48:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.548+0000 7f1b9a7fc640 10 client.7155 handle_command_reply: tid=0 2024-05-08T05:48:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.548+0000 7f1bc4b79740 1 client.7155 shutdown 2024-05-08T05:48:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.548+0000 7f1bc4b79740 1 -- 172.21.15.45:0/81972113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1c17f0 msgr2=0x561acd1bbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.548+0000 7f1bc4b79740 1 --2- 172.21.15.45:0/81972113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1c17f0 0x561acd1bbcc0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1bac05cec0 tx=0x7f1bac05f890 comp rx=0 tx=0).stop 2024-05-08T05:48:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.548+0000 7f1bc4b79740 1 -- 172.21.15.45:0/81972113 shutdown_connections 2024-05-08T05:48:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.548+0000 7f1bc4b79740 1 --2- 172.21.15.45:0/81972113 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x561acd1bf090 0x561acd1c1d90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.548+0000 7f1bc4b79740 1 --2- 172.21.15.45:0/81972113 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1de6d0 0x561acd1ebbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.548+0000 7f1b9a7fc640 0 client.7155 ms_handle_reset on v2:172.21.15.63:6835/3194497846 2024-05-08T05:48:18.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.548+0000 7f1bc4b79740 1 --2- 172.21.15.45:0/81972113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1ec0f0 0x561acd1c3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.548+0000 7f1bc4b79740 1 --2- 172.21.15.45:0/81972113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1c17f0 0x561acd1bbcc0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.548+0000 7f1bc4b79740 1 -- 172.21.15.45:0/81972113 >> 172.21.15.45:0/81972113 conn(0x561acd1c10b0 msgr2=0x561acd190a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:18.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.548+0000 7f1bc4b79740 1 -- 172.21.15.45:0/81972113 shutdown_connections 2024-05-08T05:48:18.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.548+0000 7f1bc4b79740 1 -- 172.21.15.45:0/81972113 wait complete. 2024-05-08T05:48:18.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.548+0000 7f1bc4b79740 20 client.7155 trim_cache size 0 max 16384 2024-05-08T05:48:18.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.562+0000 7f1bc4b79740 1 Processor -- start 2024-05-08T05:48:18.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.562+0000 7f1bc4b79740 1 -- start start 2024-05-08T05:48:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.563+0000 7f1bc4b79740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1c17f0 0x561acd1ff8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.563+0000 7f1bc4b79740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1de6d0 0x561acd1bbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.563+0000 7f1bc4b79740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1bc200 0x561acd1dabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.563+0000 7f1bc4b79740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc201050 con 0x561acd1bc200 2024-05-08T05:48:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.563+0000 7f1bc4b79740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc2003c0 con 0x561acd1c17f0 2024-05-08T05:48:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.563+0000 7f1bc4b79740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1bbc200a10 con 0x561acd1de6d0 2024-05-08T05:48:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.563+0000 7f1bbbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1bc200 0x561acd1dabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.563+0000 7f1bbbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1bc200 0x561acd1dabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35196/0 (socket says 172.21.15.45:35196) 2024-05-08T05:48:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.563+0000 7f1bbbfff640 1 -- 172.21.15.45:0/2788119795 learned_addr learned my addr 172.21.15.45:0/2788119795 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.563+0000 7f1bbaffd640 1 --2- 172.21.15.45:0/2788119795 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1de6d0 0x561acd1bbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.563+0000 7f1bbb7fe640 1 --2- 172.21.15.45:0/2788119795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1c17f0 0x561acd1ff8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.563+0000 7f1bbbfff640 1 -- 172.21.15.45:0/2788119795 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1de6d0 msgr2=0x561acd1bbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.563+0000 7f1bbbfff640 1 --2- 172.21.15.45:0/2788119795 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1de6d0 0x561acd1bbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.563+0000 7f1bbbfff640 1 -- 172.21.15.45:0/2788119795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1c17f0 msgr2=0x561acd1ff8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.563+0000 7f1bbbfff640 1 --2- 172.21.15.45:0/2788119795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1c17f0 0x561acd1ff8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.563+0000 7f1bbbfff640 1 -- 172.21.15.45:0/2788119795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561acd1f5a90 con 0x561acd1bc200 2024-05-08T05:48:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.563+0000 7f1bbbfff640 1 --2- 172.21.15.45:0/2788119795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1bc200 0x561acd1dabc0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f1bb40316d0 tx=0x7f1bb4030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b3664bdd37e2b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.564+0000 7f1ba0ff9640 1 -- 172.21.15.45:0/2788119795 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4003990 con 0x561acd1bc200 2024-05-08T05:48:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.564+0000 7f1ba0ff9640 1 -- 172.21.15.45:0/2788119795 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb4041070 con 0x561acd1bc200 2024-05-08T05:48:18.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.564+0000 7f1ba0ff9640 1 -- 172.21.15.45:0/2788119795 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4003d70 con 0x561acd1bc200 2024-05-08T05:48:18.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.564+0000 7f1bc4b79740 1 -- 172.21.15.45:0/2788119795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1bc200 msgr2=0x561acd1dabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.564+0000 7f1bc4b79740 1 --2- 172.21.15.45:0/2788119795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1bc200 0x561acd1dabc0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f1bb40316d0 tx=0x7f1bb4030000 comp rx=0 tx=0).stop 2024-05-08T05:48:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.564+0000 7f1bc4b79740 1 -- 172.21.15.45:0/2788119795 shutdown_connections 2024-05-08T05:48:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.564+0000 7f1bc4b79740 1 --2- 172.21.15.45:0/2788119795 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1de6d0 0x561acd1bbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.564+0000 7f1bc4b79740 1 --2- 172.21.15.45:0/2788119795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1bc200 0x561acd1dabc0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.564+0000 7f1bc4b79740 1 --2- 172.21.15.45:0/2788119795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1c17f0 0x561acd1ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.564+0000 7f1bc4b79740 1 -- 172.21.15.45:0/2788119795 >> 172.21.15.45:0/2788119795 conn(0x561acd1c10b0 msgr2=0x561acd1c0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.564+0000 7f1bc4b79740 1 -- 172.21.15.45:0/2788119795 shutdown_connections 2024-05-08T05:48:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.564+0000 7f1bc4b79740 1 -- 172.21.15.45:0/2788119795 wait complete. 2024-05-08T05:48:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.565+0000 7f1bc4b79740 1 -- start start 2024-05-08T05:48:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.565+0000 7f1bc4b79740 1 Processor -- start 2024-05-08T05:48:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1bc4b79740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1bc200 0x561acd1ba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1bc4b79740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1c17f0 0x561acd1eb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1bc4b79740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1de6d0 0x561acd25c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1bc4b79740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561acd1bef10 con 0x561acd1de6d0 2024-05-08T05:48:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1bc4b79740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561acd0b0a20 con 0x561acd1bc200 2024-05-08T05:48:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1bc4b79740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x561acd1dbb20 con 0x561acd1c17f0 2024-05-08T05:48:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1bc4b79740 10 client.0 _ll_register_callbacks cb 0x7f1bac05fab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:48:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1bbbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1de6d0 0x561acd25c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1bbbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1de6d0 0x561acd25c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35200/0 (socket says 172.21.15.45:35200) 2024-05-08T05:48:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1bbbfff640 1 -- 172.21.15.45:0/1061277147 learned_addr learned my addr 172.21.15.45:0/1061277147 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1bbaffd640 1 --2- 172.21.15.45:0/1061277147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1c17f0 0x561acd1eb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1bbb7fe640 1 --2- 172.21.15.45:0/1061277147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1bc200 0x561acd1ba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1bbbfff640 1 -- 172.21.15.45:0/1061277147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1c17f0 msgr2=0x561acd1eb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1bbbfff640 1 --2- 172.21.15.45:0/1061277147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1c17f0 0x561acd1eb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1bbbfff640 1 -- 172.21.15.45:0/1061277147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1bc200 msgr2=0x561acd1ba340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1bbbfff640 1 --2- 172.21.15.45:0/1061277147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1bc200 0x561acd1ba340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1bbbfff640 1 -- 172.21.15.45:0/1061277147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561acd18bd40 con 0x561acd1de6d0 2024-05-08T05:48:18.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1bbaffd640 1 --2- 172.21.15.45:0/1061277147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1c17f0 0x561acd1eb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1bbb7fe640 1 --2- 172.21.15.45:0/1061277147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1bc200 0x561acd1ba340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1bbbfff640 1 --2- 172.21.15.45:0/1061277147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1de6d0 0x561acd25c660 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f1bb404c040 tx=0x7f1bb4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d20f08ec69f15de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.566+0000 7f1b9a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:48:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.567+0000 7f1b9a7fc640 1 -- 172.21.15.45:0/1061277147 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4053070 con 0x561acd1de6d0 2024-05-08T05:48:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.567+0000 7f1b9a7fc640 1 -- 172.21.15.45:0/1061277147 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f1bb4031c90 con 0x561acd1de6d0 2024-05-08T05:48:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.567+0000 7f1bc4b79740 1 -- 172.21.15.45:0/1061277147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bb4041070 con 0x561acd1de6d0 2024-05-08T05:48:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.567+0000 7f1b9a7fc640 1 -- 172.21.15.45:0/1061277147 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f1bb405e050 con 0x561acd1de6d0 2024-05-08T05:48:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.567+0000 7f1bc4b79740 10 client.7161 fetch_fsmap learned FSMap version 67 2024-05-08T05:48:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.567+0000 7f1bc4b79740 1 -- 172.21.15.45:0/1061277147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x561acd1c1dd0 con 0x561acd1de6d0 2024-05-08T05:48:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.567+0000 7f1b9a7fc640 1 -- 172.21.15.45:0/1061277147 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb4041070 con 0x561acd1de6d0 2024-05-08T05:48:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.567+0000 7f1b9a7fc640 1 -- 172.21.15.45:0/1061277147 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f1bb4004030 con 0x561acd1de6d0 2024-05-08T05:48:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.567+0000 7f1bc4b79740 10 client.7161 fetch_fsmap finished waiting for FSMap version 67 2024-05-08T05:48:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.567+0000 7f1bc4b79740 10 client.7161 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:48:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.567+0000 7f1bc4b79740 20 client.7161 populate_metadata read hostname 'smithi045' 2024-05-08T05:48:18.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.567+0000 7f1bc4b79740 1 --2- 172.21.15.45:0/1061277147 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x561acd1f4a80 0x561acd1f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.567+0000 7f1bc4b79740 4 client.7161 mds_command: new command op to 6759 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-08T05:48:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.567+0000 7f1bc4b79740 1 -- 172.21.15.45:0/1061277147 --> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x561acd102890 con 0x561acd1f4a80 2024-05-08T05:48:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.567+0000 7f1bbb7fe640 1 --2- 172.21.15.45:0/1061277147 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x561acd1f4a80 0x561acd1f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.568+0000 7f1b9a7fc640 10 client.7161 ms_handle_connect on v2:172.21.15.63:6835/3194497846 2024-05-08T05:48:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.568+0000 7f1bbb7fe640 1 --2- 172.21.15.45:0/1061277147 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x561acd1f4a80 0x561acd1f4e60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c82a2df1f1b77bc3 server_cookie=d82eb27562d524b4 in_seq=0 out_seq=0 2024-05-08T05:48:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.568+0000 7f1b9a7fc640 1 -- 172.21.15.45:0/1061277147 <== mds.0 v2:172.21.15.63:6835/3194497846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+144 (crc 0 0 0) 0x561acd102890 con 0x561acd1f4a80 2024-05-08T05:48:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.568+0000 7f1b9a7fc640 10 client.7161 handle_command_reply: tid=0 2024-05-08T05:48:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.568+0000 7f1bc4b79740 1 client.7161 shutdown 2024-05-08T05:48:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.568+0000 7f1bc4b79740 1 -- 172.21.15.45:0/1061277147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1de6d0 msgr2=0x561acd25c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.568+0000 7f1bc4b79740 1 --2- 172.21.15.45:0/1061277147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1de6d0 0x561acd25c660 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f1bb404c040 tx=0x7f1bb4004ab0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.568+0000 7f1bc4b79740 1 -- 172.21.15.45:0/1061277147 shutdown_connections 2024-05-08T05:48:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.568+0000 7f1bc4b79740 1 --2- 172.21.15.45:0/1061277147 >> [v2:172.21.15.63:6835/3194497846,v1:172.21.15.63:6837/3194497846] conn(0x561acd1f4a80 0x561acd1f4e60 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.568+0000 7f1bc4b79740 1 --2- 172.21.15.45:0/1061277147 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561acd1c17f0 0x561acd1eb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.568+0000 7f1bc4b79740 1 --2- 172.21.15.45:0/1061277147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561acd1de6d0 0x561acd25c660 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.568+0000 7f1b9a7fc640 0 client.7161 ms_handle_reset on v2:172.21.15.63:6835/3194497846 2024-05-08T05:48:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.568+0000 7f1bc4b79740 1 --2- 172.21.15.45:0/1061277147 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561acd1bc200 0x561acd1ba340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.568+0000 7f1bc4b79740 1 -- 172.21.15.45:0/1061277147 >> 172.21.15.45:0/1061277147 conn(0x561acd1c10b0 msgr2=0x561acd1901a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.568+0000 7f1bc4b79740 1 -- 172.21.15.45:0/1061277147 shutdown_connections 2024-05-08T05:48:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.568+0000 7f1bc4b79740 1 -- 172.21.15.45:0/1061277147 wait complete. 2024-05-08T05:48:18.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.568+0000 7f1bc4b79740 20 client.7161 trim_cache size 0 max 16384 2024-05-08T05:48:18.574 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:48:18.574 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-05-08T05:48:18.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:48:18.574 INFO:teuthology.orchestra.run.smithi045.stdout: "damage_type": "backtrace", 2024-05-08T05:48:18.574 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 3259427672, 2024-05-08T05:48:18.574 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 1099511627777, 2024-05-08T05:48:18.575 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/testdir/hardlink" 2024-05-08T05:48:18.575 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:48:18.575 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-05-08T05:48:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.569+0000 7f1bc1d89640 1 -- 172.21.15.45:0/4044634340 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1ba80762d0 msgr2=0x7f1ba8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.569+0000 7f1bc1d89640 1 --2- 172.21.15.45:0/4044634340 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1ba80762d0 0x7f1ba8078790 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0d38f0 tx=0x7f1bb0009040 comp rx=0 tx=0).stop 2024-05-08T05:48:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.569+0000 7f1bc1d89640 1 -- 172.21.15.45:0/4044634340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bbc0b21f0 msgr2=0x7f1bbc0c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.569+0000 7f1bc1d89640 1 --2- 172.21.15.45:0/4044634340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bbc0b21f0 0x7f1bbc0c8450 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00c480 tx=0x7f1bac00c940 comp rx=0 tx=0).stop 2024-05-08T05:48:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.569+0000 7f1bc1d89640 1 -- 172.21.15.45:0/4044634340 shutdown_connections 2024-05-08T05:48:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.569+0000 7f1bc1d89640 1 --2- 172.21.15.45:0/4044634340 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1ba80762d0 0x7f1ba8078790 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.569+0000 7f1bc1d89640 1 --2- 172.21.15.45:0/4044634340 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1bbc0cfea0 0x7f1bbc0cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.569+0000 7f1bc1d89640 1 --2- 172.21.15.45:0/4044634340 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bbc162f50 0x7f1bbc0cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.569+0000 7f1bc1d89640 1 --2- 172.21.15.45:0/4044634340 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1bbc0b21f0 0x7f1bbc0c8450 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.569+0000 7f1bc1d89640 1 -- 172.21.15.45:0/4044634340 >> 172.21.15.45:0/4044634340 conn(0x7f1bbc09ba40 msgr2=0x7f1bbc1603c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.569+0000 7f1bc1d89640 1 -- 172.21.15.45:0/4044634340 shutdown_connections 2024-05-08T05:48:18.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.569+0000 7f1bc1d89640 1 -- 172.21.15.45:0/4044634340 wait complete. 2024-05-08T05:48:18.579 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:48:18.580 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:48:18.596 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:48:18.596 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:48:18.653 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:48:18.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.797+0000 7f91b548c640 1 Processor -- start 2024-05-08T05:48:18.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.797+0000 7f91b548c640 1 -- start start 2024-05-08T05:48:18.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.798+0000 7f91b548c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f91b0157540 0x7f91b015e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.798+0000 7f91b548c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f91b006fc30 0x7f91b00af740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.798+0000 7f91b548c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91b015e550 con 0x7f91b00b48a0 2024-05-08T05:48:18.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.798+0000 7f91b548c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f91b00701d0 con 0x7f91b0157540 2024-05-08T05:48:18.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.798+0000 7f91b548c640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f91b00b3dc0 con 0x7f91b006fc30 2024-05-08T05:48:18.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.798+0000 7f91aeffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f91b0157540 0x7f91b015e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.798+0000 7f91aeffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f91b0157540 0x7f91b015e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35208/0 (socket says 172.21.15.45:35208) 2024-05-08T05:48:18.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.798+0000 7f91aeffd640 1 -- 172.21.15.45:0/3181071568 learned_addr learned my addr 172.21.15.45:0/3181071568 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:18.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.798+0000 7f91ae7fc640 1 --2- 172.21.15.45:0/3181071568 >> v2:172.21.15.63:3300/0 conn(0x7f91b006fc30 0x7f91b00af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.799+0000 7f91aeffd640 1 -- 172.21.15.45:0/3181071568 >> v2:172.21.15.63:3300/0 conn(0x7f91b006fc30 msgr2=0x7f91b00af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:18.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.799+0000 7f91aeffd640 1 --2- 172.21.15.45:0/3181071568 >> v2:172.21.15.63:3300/0 conn(0x7f91b006fc30 0x7f91b00af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.799+0000 7f91aeffd640 1 -- 172.21.15.45:0/3181071568 >> v1:172.21.15.63:6789/0 conn(0x7f91b00b48a0 legacy=0x7f91b00ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.799+0000 7f91aeffd640 1 -- 172.21.15.45:0/3181071568 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b00b40a0 con 0x7f91b0157540 2024-05-08T05:48:18.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.799+0000 7f91aeffd640 1 --2- 172.21.15.45:0/3181071568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b0157540 0x7f91b015e010 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f91b0160ac0 tx=0x7f91a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687e3085474fc8df server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:18.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.799+0000 7f91adffb640 1 -- 172.21.15.45:0/3181071568 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a0037070 con 0x7f91b0157540 2024-05-08T05:48:18.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.800+0000 7f91adffb640 1 -- 172.21.15.45:0/3181071568 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a0032440 con 0x7f91b0157540 2024-05-08T05:48:18.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.800+0000 7f91adffb640 1 -- 172.21.15.45:0/3181071568 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a0032810 con 0x7f91b0157540 2024-05-08T05:48:18.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.800+0000 7f91b548c640 1 -- 172.21.15.45:0/3181071568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b0157540 msgr2=0x7f91b015e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.800+0000 7f91b548c640 1 --2- 172.21.15.45:0/3181071568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b0157540 0x7f91b015e010 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f91b0160ac0 tx=0x7f91a002a270 comp rx=0 tx=0).stop 2024-05-08T05:48:18.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.800+0000 7f91b548c640 1 -- 172.21.15.45:0/3181071568 shutdown_connections 2024-05-08T05:48:18.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.800+0000 7f91b548c640 1 --2- 172.21.15.45:0/3181071568 >> v2:172.21.15.63:3300/0 conn(0x7f91b006fc30 0x7f91b00af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.800+0000 7f91b548c640 1 --2- 172.21.15.45:0/3181071568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b0157540 0x7f91b015e010 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.800+0000 7f91b548c640 1 -- 172.21.15.45:0/3181071568 >> 172.21.15.45:0/3181071568 conn(0x7f91b006c3a0 msgr2=0x7f91b006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:18.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.800+0000 7f91b548c640 1 -- 172.21.15.45:0/3181071568 shutdown_connections 2024-05-08T05:48:18.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.800+0000 7f91b548c640 1 -- 172.21.15.45:0/3181071568 wait complete. 2024-05-08T05:48:18.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.802+0000 7f91b548c640 1 Processor -- start 2024-05-08T05:48:18.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.802+0000 7f91b548c640 1 -- start start 2024-05-08T05:48:18.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91b548c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b006fc30 0x7f91b01f2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91b548c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91b00b48a0 0x7f91b014a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91b548c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91b0157540 0x7f91b00b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91b548c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f91b0157aa0 con 0x7f91b006fc30 2024-05-08T05:48:18.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91b548c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f91b0157920 con 0x7f91b00b48a0 2024-05-08T05:48:18.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91b548c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f91b00afc80 con 0x7f91b0157540 2024-05-08T05:48:18.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91aeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b006fc30 0x7f91b01f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91aeffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b006fc30 0x7f91b01f2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35214/0 (socket says 172.21.15.45:35214) 2024-05-08T05:48:18.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91aeffd640 1 -- 172.21.15.45:0/2637588667 learned_addr learned my addr 172.21.15.45:0/2637588667 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:18.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91ae7fc640 1 --2- 172.21.15.45:0/2637588667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91b00b48a0 0x7f91b014a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91af7fe640 1 --2- 172.21.15.45:0/2637588667 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91b0157540 0x7f91b00b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91aeffd640 1 -- 172.21.15.45:0/2637588667 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91b0157540 msgr2=0x7f91b00b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91aeffd640 1 --2- 172.21.15.45:0/2637588667 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91b0157540 0x7f91b00b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91aeffd640 1 -- 172.21.15.45:0/2637588667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91b00b48a0 msgr2=0x7f91b014a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:18.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91aeffd640 1 --2- 172.21.15.45:0/2637588667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91b00b48a0 0x7f91b014a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:18.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91aeffd640 1 -- 172.21.15.45:0/2637588667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b014b380 con 0x7f91b006fc30 2024-05-08T05:48:18.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91af7fe640 1 --2- 172.21.15.45:0/2637588667 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91b0157540 0x7f91b00b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:18.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91aeffd640 1 --2- 172.21.15.45:0/2637588667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b006fc30 0x7f91b01f2260 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f91a002a2b0 tx=0x7f91a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:18.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91977fe640 1 -- 172.21.15.45:0/2637588667 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a0037040 con 0x7f91b006fc30 2024-05-08T05:48:18.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91977fe640 1 -- 172.21.15.45:0/2637588667 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a0042070 con 0x7f91b006fc30 2024-05-08T05:48:18.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91b548c640 1 -- 172.21.15.45:0/2637588667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b014b660 con 0x7f91b006fc30 2024-05-08T05:48:18.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91977fe640 1 -- 172.21.15.45:0/2637588667 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a0030880 con 0x7f91b006fc30 2024-05-08T05:48:18.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.803+0000 7f91b548c640 1 -- 172.21.15.45:0/2637588667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b014bc20 con 0x7f91b006fc30 2024-05-08T05:48:18.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.805+0000 7f91b548c640 1 -- 172.21.15.45:0/2637588667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b00693f0 con 0x7f91b006fc30 2024-05-08T05:48:18.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.805+0000 7f91977fe640 1 -- 172.21.15.45:0/2637588667 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f91a0030a20 con 0x7f91b006fc30 2024-05-08T05:48:18.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.806+0000 7f91977fe640 1 --2- 172.21.15.45:0/2637588667 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f919c0762d0 0x7f919c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:18.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.806+0000 7f91977fe640 1 -- 172.21.15.45:0/2637588667 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f91a00b80d0 con 0x7f91b006fc30 2024-05-08T05:48:18.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.806+0000 7f91ae7fc640 1 --2- 172.21.15.45:0/2637588667 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f919c0762d0 0x7f919c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:18.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.807+0000 7f91ae7fc640 1 --2- 172.21.15.45:0/2637588667 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f919c0762d0 0x7f919c078790 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f91b006f820 tx=0x7f91a4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:18.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:18.810+0000 7f91977fe640 1 -- 172.21.15.45:0/2637588667 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f91a00817e0 con 0x7f91b006fc30 2024-05-08T05:48:19.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.011+0000 7f91b548c640 1 -- 172.21.15.45:0/2637588667 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f91b015fe40 con 0x7f91b006fc30 2024-05-08T05:48:19.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.011+0000 7f91977fe640 1 -- 172.21.15.45:0/2637588667 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v91) ==== 80+0+122 (secure 0 0 0) 0x7f91a00810e0 con 0x7f91b006fc30 2024-05-08T05:48:19.012 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.63:6836/2942326473 2024-05-09T05:47:09.529461+0000 2024-05-08T05:48:19.012 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.63:6834/2942326473 2024-05-09T05:47:09.529461+0000 2024-05-08T05:48:19.012 INFO:teuthology.orchestra.run.smithi045.stderr:listed 2 entries 2024-05-08T05:48:19.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.014+0000 7f91b548c640 1 -- 172.21.15.45:0/2637588667 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f919c0762d0 msgr2=0x7f919c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:19.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.014+0000 7f91b548c640 1 --2- 172.21.15.45:0/2637588667 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f919c0762d0 0x7f919c078790 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f91b006f820 tx=0x7f91a4009190 comp rx=0 tx=0).stop 2024-05-08T05:48:19.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.014+0000 7f91b548c640 1 -- 172.21.15.45:0/2637588667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b006fc30 msgr2=0x7f91b01f2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:19.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.014+0000 7f91b548c640 1 --2- 172.21.15.45:0/2637588667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b006fc30 0x7f91b01f2260 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f91a002a2b0 tx=0x7f91a00046c0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.014+0000 7f91b548c640 1 -- 172.21.15.45:0/2637588667 shutdown_connections 2024-05-08T05:48:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.014+0000 7f91b548c640 1 --2- 172.21.15.45:0/2637588667 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f919c0762d0 0x7f919c078790 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.014+0000 7f91b548c640 1 --2- 172.21.15.45:0/2637588667 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f91b0157540 0x7f91b00b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.014+0000 7f91b548c640 1 --2- 172.21.15.45:0/2637588667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f91b00b48a0 0x7f91b014a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.014+0000 7f91b548c640 1 --2- 172.21.15.45:0/2637588667 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f91b006fc30 0x7f91b01f2260 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.014+0000 7f91b548c640 1 -- 172.21.15.45:0/2637588667 >> 172.21.15.45:0/2637588667 conn(0x7f91b006c3a0 msgr2=0x7f91b01566e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.014+0000 7f91b548c640 1 -- 172.21.15.45:0/2637588667 shutdown_connections 2024-05-08T05:48:19.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.014+0000 7f91b548c640 1 -- 172.21.15.45:0/2637588667 wait complete. 2024-05-08T05:48:19.025 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T05:48:19.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:48:19.025 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T05:48:19.054 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:48:19.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:48:19.054 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T05:48:19.112 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:48:19.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.255+0000 7f8b0a2e8640 1 Processor -- start 2024-05-08T05:48:19.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.255+0000 7f8b0a2e8640 1 -- start start 2024-05-08T05:48:19.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.256+0000 7f8b0a2e8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b04070d30 0x7f8b040d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:19.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.256+0000 7f8b0a2e8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8b04163920 0x7f8b040703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:19.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.256+0000 7f8b0a2e8640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8b04071110 0x7f8b0406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:19.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.256+0000 7f8b0a2e8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f8b040d10d0 con 0x7f8b04163920 2024-05-08T05:48:19.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.256+0000 7f8b0a2e8640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8b04070b80 con 0x7f8b04071110 2024-05-08T05:48:19.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.256+0000 7f8b0a2e8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8b040d1c30 con 0x7f8b04070d30 2024-05-08T05:48:19.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.256+0000 7f8b037fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8b04163920 0x7f8b040703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:19.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.256+0000 7f8b037fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8b04163920 0x7f8b040703a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35230/0 (socket says 172.21.15.45:35230) 2024-05-08T05:48:19.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.256+0000 7f8b037fe640 1 -- 172.21.15.45:0/3839906554 learned_addr learned my addr 172.21.15.45:0/3839906554 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:19.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.256+0000 7f8b03fff640 1 --2- 172.21.15.45:0/3839906554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b04070d30 0x7f8b040d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:19.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.256+0000 7f8b08885640 1 --2- 172.21.15.45:0/3839906554 >> v2:172.21.15.63:3300/0 conn(0x7f8b04071110 0x7f8b0406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:19.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.257+0000 7f8b037fe640 1 -- 172.21.15.45:0/3839906554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b04070d30 msgr2=0x7f8b040d20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:19.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.257+0000 7f8b037fe640 1 --2- 172.21.15.45:0/3839906554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b04070d30 0x7f8b040d20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.257+0000 7f8b037fe640 1 -- 172.21.15.45:0/3839906554 >> v2:172.21.15.63:3300/0 conn(0x7f8b04071110 msgr2=0x7f8b0406d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:19.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.257+0000 7f8b037fe640 1 --2- 172.21.15.45:0/3839906554 >> v2:172.21.15.63:3300/0 conn(0x7f8b04071110 0x7f8b0406d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.257+0000 7f8b037fe640 1 -- 172.21.15.45:0/3839906554 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b04163d80 con 0x7f8b04163920 2024-05-08T05:48:19.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.257+0000 7f8b037fe640 1 --2- 172.21.15.45:0/3839906554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b04163920 0x7f8b040703a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f8b04073310 tx=0x7f8af802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f405e46a0d9a1115 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:19.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.258+0000 7f8b02ffd640 1 -- 172.21.15.45:0/3839906554 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af8037070 con 0x7f8b04163920 2024-05-08T05:48:19.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.258+0000 7f8b02ffd640 1 -- 172.21.15.45:0/3839906554 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af8032440 con 0x7f8b04163920 2024-05-08T05:48:19.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.258+0000 7f8b02ffd640 1 -- 172.21.15.45:0/3839906554 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af8032810 con 0x7f8b04163920 2024-05-08T05:48:19.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.258+0000 7f8b0a2e8640 1 -- 172.21.15.45:0/3839906554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b04163920 msgr2=0x7f8b040703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:19.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.258+0000 7f8b0a2e8640 1 --2- 172.21.15.45:0/3839906554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b04163920 0x7f8b040703a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f8b04073310 tx=0x7f8af802a1e0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.258+0000 7f8b0a2e8640 1 -- 172.21.15.45:0/3839906554 shutdown_connections 2024-05-08T05:48:19.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.258+0000 7f8b0a2e8640 1 --2- 172.21.15.45:0/3839906554 >> v2:172.21.15.63:3300/0 conn(0x7f8b04071110 0x7f8b0406d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.258+0000 7f8b0a2e8640 1 --2- 172.21.15.45:0/3839906554 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b04163920 0x7f8b040703a0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.258+0000 7f8b0a2e8640 1 --2- 172.21.15.45:0/3839906554 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b04070d30 0x7f8b040d20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.258+0000 7f8b0a2e8640 1 -- 172.21.15.45:0/3839906554 >> 172.21.15.45:0/3839906554 conn(0x7f8b040d2e90 msgr2=0x7f8b040d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:19.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.258+0000 7f8b0a2e8640 1 -- 172.21.15.45:0/3839906554 shutdown_connections 2024-05-08T05:48:19.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.258+0000 7f8b0a2e8640 1 -- 172.21.15.45:0/3839906554 wait complete. 2024-05-08T05:48:19.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.259+0000 7f8b0a2e8640 1 Processor -- start 2024-05-08T05:48:19.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.260+0000 7f8b0a2e8640 1 -- start start 2024-05-08T05:48:19.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.260+0000 7f8b0a2e8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b04070d30 0x7f8b040cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:19.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.260+0000 7f8b0a2e8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b04071110 0x7f8b040c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:19.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.260+0000 7f8b0a2e8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b04163920 0x7f8b040b7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:19.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.260+0000 7f8b0a2e8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8b040d15e0 con 0x7f8b04070d30 2024-05-08T05:48:19.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.260+0000 7f8b0a2e8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8b040708e0 con 0x7f8b04163920 2024-05-08T05:48:19.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.260+0000 7f8b0a2e8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8b040d18c0 con 0x7f8b04071110 2024-05-08T05:48:19.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.260+0000 7f8b03fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b04070d30 0x7f8b040cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:19.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.260+0000 7f8b03fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b04070d30 0x7f8b040cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35234/0 (socket says 172.21.15.45:35234) 2024-05-08T05:48:19.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.260+0000 7f8b03fff640 1 -- 172.21.15.45:0/1861279683 learned_addr learned my addr 172.21.15.45:0/1861279683 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:19.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.261+0000 7f8b08885640 1 --2- 172.21.15.45:0/1861279683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b04163920 0x7f8b040b7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:19.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.261+0000 7f8b037fe640 1 --2- 172.21.15.45:0/1861279683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b04071110 0x7f8b040c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:19.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.261+0000 7f8b03fff640 1 -- 172.21.15.45:0/1861279683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b04071110 msgr2=0x7f8b040c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:19.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.261+0000 7f8b03fff640 1 --2- 172.21.15.45:0/1861279683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b04071110 0x7f8b040c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.261+0000 7f8b03fff640 1 -- 172.21.15.45:0/1861279683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b04163920 msgr2=0x7f8b040b7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:19.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.261+0000 7f8b03fff640 1 --2- 172.21.15.45:0/1861279683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b04163920 0x7f8b040b7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.261+0000 7f8b03fff640 1 -- 172.21.15.45:0/1861279683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b040b6b80 con 0x7f8b04070d30 2024-05-08T05:48:19.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.261+0000 7f8b08885640 1 --2- 172.21.15.45:0/1861279683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b04163920 0x7f8b040b7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:19.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.261+0000 7f8b03fff640 1 --2- 172.21.15.45:0/1861279683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b04070d30 0x7f8b040cc670 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f8af400e8a0 tx=0x7f8af400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:19.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.261+0000 7f8b00ff9640 1 -- 172.21.15.45:0/1861279683 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4017070 con 0x7f8b04070d30 2024-05-08T05:48:19.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.261+0000 7f8b00ff9640 1 -- 172.21.15.45:0/1861279683 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af400ee90 con 0x7f8b04070d30 2024-05-08T05:48:19.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.261+0000 7f8b0a2e8640 1 -- 172.21.15.45:0/1861279683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b040b6e60 con 0x7f8b04070d30 2024-05-08T05:48:19.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.261+0000 7f8b00ff9640 1 -- 172.21.15.45:0/1861279683 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af400cb80 con 0x7f8b04070d30 2024-05-08T05:48:19.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.261+0000 7f8b0a2e8640 1 -- 172.21.15.45:0/1861279683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b040c6ef0 con 0x7f8b04070d30 2024-05-08T05:48:19.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.262+0000 7f8b0a2e8640 1 -- 172.21.15.45:0/1861279683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b040693f0 con 0x7f8b04070d30 2024-05-08T05:48:19.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.263+0000 7f8b00ff9640 1 -- 172.21.15.45:0/1861279683 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8af401a070 con 0x7f8b04070d30 2024-05-08T05:48:19.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.263+0000 7f8b00ff9640 1 --2- 172.21.15.45:0/1861279683 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8af0076300 0x7f8af00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:19.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.264+0000 7f8b00ff9640 1 -- 172.21.15.45:0/1861279683 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f8af4097960 con 0x7f8b04070d30 2024-05-08T05:48:19.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.264+0000 7f8b037fe640 1 --2- 172.21.15.45:0/1861279683 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8af0076300 0x7f8af00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:19.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.264+0000 7f8b037fe640 1 --2- 172.21.15.45:0/1861279683 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8af0076300 0x7f8af00787c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8af8034000 tx=0x7f8af802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:19.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.268+0000 7f8b00ff9640 1 -- 172.21.15.45:0/1861279683 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8af4061100 con 0x7f8b04070d30 2024-05-08T05:48:19.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.496+0000 7f8b0a2e8640 1 -- 172.21.15.45:0/1861279683 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b040d3d70 con 0x7f8b04070d30 2024-05-08T05:48:19.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.497+0000 7f8b00ff9640 1 -- 172.21.15.45:0/1861279683 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5184 (secure 0 0 0) 0x7f8af4060a00 con 0x7f8b04070d30 2024-05-08T05:48:19.497 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:48:19.498 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2616298287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2616298287},{"type":"v1","addr":"172.21.15.63:6836","nonce":2616298287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6744,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2854804427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2854804427},{"type":"v1","addr":"172.21.15.45:6834","nonce":2854804427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2945763952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2945763952},{"type":"v1","addr":"172.21.15.45:6835","nonce":2945763952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:47:00.211837+0000","modified":"2024-05-08T05:47:20.541349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6759},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6759":{"gid":6759,"name":"b","rank":0,"incarnation":63,"state":"up:active","state_seq":9,"addr":"172.21.15.63:6837/3194497846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":3194497846},{"type":"v1","addr":"172.21.15.63:6837","nonce":3194497846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6759,"qdb_cluster":[6759]},"id":6}]} 2024-05-08T05:48:19.498 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 67 2024-05-08T05:48:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.499+0000 7f8b0a2e8640 1 -- 172.21.15.45:0/1861279683 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8af0076300 msgr2=0x7f8af00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.499+0000 7f8b0a2e8640 1 --2- 172.21.15.45:0/1861279683 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8af0076300 0x7f8af00787c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8af8034000 tx=0x7f8af802e000 comp rx=0 tx=0).stop 2024-05-08T05:48:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.499+0000 7f8b0a2e8640 1 -- 172.21.15.45:0/1861279683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b04070d30 msgr2=0x7f8b040cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.499+0000 7f8b0a2e8640 1 --2- 172.21.15.45:0/1861279683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b04070d30 0x7f8b040cc670 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f8af400e8a0 tx=0x7f8af400ed60 comp rx=0 tx=0).stop 2024-05-08T05:48:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.499+0000 7f8b0a2e8640 1 -- 172.21.15.45:0/1861279683 shutdown_connections 2024-05-08T05:48:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.499+0000 7f8b0a2e8640 1 --2- 172.21.15.45:0/1861279683 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8af0076300 0x7f8af00787c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.499+0000 7f8b0a2e8640 1 --2- 172.21.15.45:0/1861279683 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b04071110 0x7f8b040c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.499+0000 7f8b0a2e8640 1 --2- 172.21.15.45:0/1861279683 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b04163920 0x7f8b040b7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.499+0000 7f8b0a2e8640 1 --2- 172.21.15.45:0/1861279683 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b04070d30 0x7f8b040cc670 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.499+0000 7f8b0a2e8640 1 -- 172.21.15.45:0/1861279683 >> 172.21.15.45:0/1861279683 conn(0x7f8b040d2e90 msgr2=0x7f8b04154460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.499+0000 7f8b0a2e8640 1 -- 172.21.15.45:0/1861279683 shutdown_connections 2024-05-08T05:48:19.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.499+0000 7f8b0a2e8640 1 -- 172.21.15.45:0/1861279683 wait complete. 2024-05-08T05:48:19.510 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-05-08T05:48:19.658 INFO:teuthology.orchestra.run.smithi063.stdout:{"message":"","return_code":0} 2024-05-08T05:48:19.658 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-08T05:48:19.658 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T05:48:19.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.764+0000 7f2b30f89640 1 Processor -- start 2024-05-08T05:48:19.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.764+0000 7f2b30f89640 1 -- start start 2024-05-08T05:48:19.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.764+0000 7f2b30f89640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b2c1629a0 0x7f2b2c0d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:19.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.764+0000 7f2b30f89640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2b2c0d2dc0 0x7f2b2c0d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:19.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.764+0000 7f2b30f89640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2b2c0c8ed0 0x7f2b2c0c9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:19.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.764+0000 7f2b30f89640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f2b2c0d3330 con 0x7f2b2c0c8ed0 2024-05-08T05:48:19.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.764+0000 7f2b30f89640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2b2c0d2170 con 0x7f2b2c0d2dc0 2024-05-08T05:48:19.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.764+0000 7f2b30f89640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b2c0d2450 con 0x7f2b2c1629a0 2024-05-08T05:48:19.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.765+0000 7f2b2ad9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2b2c0c8ed0 0x7f2b2c0c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:19.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.765+0000 7f2b2ad9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2b2c0c8ed0 0x7f2b2c0c9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35240/0 (socket says 172.21.15.45:35240) 2024-05-08T05:48:19.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.765+0000 7f2b2ad9d640 1 -- 172.21.15.45:0/2867697729 learned_addr learned my addr 172.21.15.45:0/2867697729 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:19.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.765+0000 7f2b2a59c640 1 --2- 172.21.15.45:0/2867697729 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b2c1629a0 0x7f2b2c0d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:19.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.765+0000 7f2b29d9b640 1 --2- 172.21.15.45:0/2867697729 >> v2:172.21.15.63:3300/0 conn(0x7f2b2c0d2dc0 0x7f2b2c0d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:19.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.765+0000 7f2b2ad9d640 1 -- 172.21.15.45:0/2867697729 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b2c1629a0 msgr2=0x7f2b2c0d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:19.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.765+0000 7f2b2ad9d640 1 --2- 172.21.15.45:0/2867697729 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b2c1629a0 0x7f2b2c0d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.765+0000 7f2b2ad9d640 1 -- 172.21.15.45:0/2867697729 >> v2:172.21.15.63:3300/0 conn(0x7f2b2c0d2dc0 msgr2=0x7f2b2c0d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:19.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.765+0000 7f2b2ad9d640 1 --2- 172.21.15.45:0/2867697729 >> v2:172.21.15.63:3300/0 conn(0x7f2b2c0d2dc0 0x7f2b2c0d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.765+0000 7f2b2ad9d640 1 -- 172.21.15.45:0/2867697729 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2c0d2730 con 0x7f2b2c0c8ed0 2024-05-08T05:48:19.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.765+0000 7f2b29d9b640 1 --2- 172.21.15.45:0/2867697729 >> v2:172.21.15.63:3300/0 conn(0x7f2b2c0d2dc0 0x7f2b2c0d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:19.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.766+0000 7f2b2ad9d640 1 --2- 172.21.15.45:0/2867697729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2c0c8ed0 0x7f2b2c0c9360 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c08da00 tx=0x7f2b1c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f296de93d385ddc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:19.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.766+0000 7f2b2959a640 1 -- 172.21.15.45:0/2867697729 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c037070 con 0x7f2b2c0c8ed0 2024-05-08T05:48:19.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.766+0000 7f2b2959a640 1 -- 172.21.15.45:0/2867697729 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1c02ae00 con 0x7f2b2c0c8ed0 2024-05-08T05:48:19.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.766+0000 7f2b2959a640 1 -- 172.21.15.45:0/2867697729 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c032660 con 0x7f2b2c0c8ed0 2024-05-08T05:48:19.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.766+0000 7f2b30f89640 1 -- 172.21.15.45:0/2867697729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2c0c8ed0 msgr2=0x7f2b2c0c9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:19.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.766+0000 7f2b30f89640 1 --2- 172.21.15.45:0/2867697729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2c0c8ed0 0x7f2b2c0c9360 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c08da00 tx=0x7f2b1c02a0b0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.766+0000 7f2b30f89640 1 -- 172.21.15.45:0/2867697729 shutdown_connections 2024-05-08T05:48:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.766+0000 7f2b30f89640 1 --2- 172.21.15.45:0/2867697729 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2c0c8ed0 0x7f2b2c0c9360 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.766+0000 7f2b30f89640 1 --2- 172.21.15.45:0/2867697729 >> v2:172.21.15.63:3300/0 conn(0x7f2b2c0d2dc0 0x7f2b2c0d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.766+0000 7f2b30f89640 1 --2- 172.21.15.45:0/2867697729 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b2c1629a0 0x7f2b2c0d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.766+0000 7f2b30f89640 1 -- 172.21.15.45:0/2867697729 >> 172.21.15.45:0/2867697729 conn(0x7f2b2c0d4fe0 msgr2=0x7f2b2c0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.767+0000 7f2b30f89640 1 -- 172.21.15.45:0/2867697729 shutdown_connections 2024-05-08T05:48:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.767+0000 7f2b30f89640 1 -- 172.21.15.45:0/2867697729 wait complete. 2024-05-08T05:48:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.768+0000 7f2b30f89640 1 Processor -- start 2024-05-08T05:48:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.768+0000 7f2b30f89640 1 -- start start 2024-05-08T05:48:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b30f89640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b2c0c8ed0 0x7f2b2c0b8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b30f89640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b2c0d2dc0 0x7f2b2c0ba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b30f89640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2c1629a0 0x7f2b2c15fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b30f89640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c0cf440 con 0x7f2b2c1629a0 2024-05-08T05:48:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b30f89640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c0cf2c0 con 0x7f2b2c0c8ed0 2024-05-08T05:48:19.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b30f89640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b2c0c98a0 con 0x7f2b2c0d2dc0 2024-05-08T05:48:19.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b2ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2c1629a0 0x7f2b2c15fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:19.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b2ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2c1629a0 0x7f2b2c15fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35246/0 (socket says 172.21.15.45:35246) 2024-05-08T05:48:19.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b2ad9d640 1 -- 172.21.15.45:0/2078333473 learned_addr learned my addr 172.21.15.45:0/2078333473 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:19.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b2a59c640 1 --2- 172.21.15.45:0/2078333473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b2c0c8ed0 0x7f2b2c0b8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:19.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b29d9b640 1 --2- 172.21.15.45:0/2078333473 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b2c0d2dc0 0x7f2b2c0ba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:19.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b2ad9d640 1 -- 172.21.15.45:0/2078333473 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b2c0d2dc0 msgr2=0x7f2b2c0ba4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:19.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b2ad9d640 1 --2- 172.21.15.45:0/2078333473 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b2c0d2dc0 0x7f2b2c0ba4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b2ad9d640 1 -- 172.21.15.45:0/2078333473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b2c0c8ed0 msgr2=0x7f2b2c0b8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:19.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b2ad9d640 1 --2- 172.21.15.45:0/2078333473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b2c0c8ed0 0x7f2b2c0b8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:19.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b2ad9d640 1 -- 172.21.15.45:0/2078333473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2c0b7f50 con 0x7f2b2c1629a0 2024-05-08T05:48:19.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b2ad9d640 1 --2- 172.21.15.45:0/2078333473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2c1629a0 0x7f2b2c15fa10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c0090b0 tx=0x7f2b1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:19.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b22ffd640 1 -- 172.21.15.45:0/2078333473 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c042070 con 0x7f2b2c1629a0 2024-05-08T05:48:19.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b22ffd640 1 -- 172.21.15.45:0/2078333473 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1c03e670 con 0x7f2b2c1629a0 2024-05-08T05:48:19.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b30f89640 1 -- 172.21.15.45:0/2078333473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b2c0b8510 con 0x7f2b2c1629a0 2024-05-08T05:48:19.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b22ffd640 1 -- 172.21.15.45:0/2078333473 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c037040 con 0x7f2b2c1629a0 2024-05-08T05:48:19.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.769+0000 7f2b30f89640 1 -- 172.21.15.45:0/2078333473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b2c0b8ad0 con 0x7f2b2c1629a0 2024-05-08T05:48:19.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.771+0000 7f2b30f89640 1 -- 172.21.15.45:0/2078333473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b2c0693f0 con 0x7f2b2c1629a0 2024-05-08T05:48:19.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.771+0000 7f2b22ffd640 1 -- 172.21.15.45:0/2078333473 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2b1c03a040 con 0x7f2b2c1629a0 2024-05-08T05:48:19.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.771+0000 7f2b22ffd640 1 --2- 172.21.15.45:0/2078333473 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b0c076300 0x7f2b0c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:19.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.772+0000 7f2b22ffd640 1 -- 172.21.15.45:0/2078333473 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6303+0+0 (secure 0 0 0) 0x7f2b1c02ae00 con 0x7f2b2c1629a0 2024-05-08T05:48:19.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.772+0000 7f2b2a59c640 1 --2- 172.21.15.45:0/2078333473 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b0c076300 0x7f2b0c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:19.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.772+0000 7f2b2a59c640 1 --2- 172.21.15.45:0/2078333473 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b0c076300 0x7f2b0c0787c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2b240099c0 tx=0x7f2b24003990 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:19.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:19.776+0000 7f2b22ffd640 1 -- 172.21.15.45:0/2078333473 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2b1c081080 con 0x7f2b2c1629a0 2024-05-08T05:48:20.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.007+0000 7f2b30f89640 1 -- 172.21.15.45:0/2078333473 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2b2c0b78f0 con 0x7f2b2c1629a0 2024-05-08T05:48:20.008 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-05-08T05:48:20.008+0000 7f124de12640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T05:48:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.009+0000 7f2b22ffd640 1 -- 172.21.15.45:0/2078333473 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) ==== 161+0+0 (secure 0 0 0) 0x7f2b1c089070 con 0x7f2b2c1629a0 2024-05-08T05:48:20.009 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T05:48:20.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.011+0000 7f2b30f89640 1 -- 172.21.15.45:0/2078333473 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b0c076300 msgr2=0x7f2b0c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.011+0000 7f2b30f89640 1 --2- 172.21.15.45:0/2078333473 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b0c076300 0x7f2b0c0787c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2b240099c0 tx=0x7f2b24003990 comp rx=0 tx=0).stop 2024-05-08T05:48:20.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.011+0000 7f2b30f89640 1 -- 172.21.15.45:0/2078333473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2c1629a0 msgr2=0x7f2b2c15fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.011+0000 7f2b30f89640 1 --2- 172.21.15.45:0/2078333473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2c1629a0 0x7f2b2c15fa10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c0090b0 tx=0x7f2b1c0046c0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.012+0000 7f2b30f89640 1 -- 172.21.15.45:0/2078333473 shutdown_connections 2024-05-08T05:48:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.012+0000 7f2b30f89640 1 --2- 172.21.15.45:0/2078333473 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b0c076300 0x7f2b0c0787c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.012+0000 7f2b30f89640 1 --2- 172.21.15.45:0/2078333473 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b2c0d2dc0 0x7f2b2c0ba4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.012+0000 7f2b30f89640 1 --2- 172.21.15.45:0/2078333473 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b2c1629a0 0x7f2b2c15fa10 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.012+0000 7f2b30f89640 1 --2- 172.21.15.45:0/2078333473 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b2c0c8ed0 0x7f2b2c0b8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.012+0000 7f2b30f89640 1 -- 172.21.15.45:0/2078333473 >> 172.21.15.45:0/2078333473 conn(0x7f2b2c0d4fe0 msgr2=0x7f2b2c0d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.012+0000 7f2b30f89640 1 -- 172.21.15.45:0/2078333473 shutdown_connections 2024-05-08T05:48:20.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.012+0000 7f2b30f89640 1 -- 172.21.15.45:0/2078333473 wait complete. 2024-05-08T05:48:20.016 INFO:tasks.ceph.mds.b.smithi063.stderr: -4165> 2024-05-08T05:47:16.157+0000 7ff8b9265640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-05-08T05:48:20.016 INFO:tasks.ceph.mds.b.smithi063.stderr: -3999> 2024-05-08T05:47:16.345+0000 7ff8b9265640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-05-08T05:47:07.822465+0000 1=1+0) n(v0 rc2024-05-08T05:47:07.942464+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x561e6ad95b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-08T05:47:07.822465+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-08T05:47:07.942464+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-05-08T05:48:20.017 INFO:tasks.ceph.mds.b.smithi063.stderr: -3986> 2024-05-08T05:47:16.475+0000 7ff8bd26d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:48:20.018 INFO:tasks.ceph.mds.b.smithi063.stderr: -2148> 2024-05-08T05:47:47.714+0000 7ff8b9265640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-05-08T05:47:07.822465+0000 1=1+0)->f(v1 m2024-05-08T05:47:07.822465+0000) n(v0 rc2024-05-08T05:47:07.942464+0000 2=1+1)->n(v0 rc2024-05-08T05:47:07.942464+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x561e6ad95b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-05-08T05:47:07.822465+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-05-08T05:47:07.942464+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-05-08T05:48:20.023 DEBUG:teuthology.orchestra.run.smithi045:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-05-08T05:48:20.046 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-08T05:48:20.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.064+0000 7f857fe0f740 1 -- start start 2024-05-08T05:48:20.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.064+0000 7f857fe0f740 1 Processor -- start 2024-05-08T05:48:20.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857fe0f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a666870c00 0x55a66764e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857fe0f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a666871800 0x55a66764e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857fe0f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a666871c00 0x55a66764eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857fe0f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a667668180 con 0x55a666871c00 2024-05-08T05:48:20.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857fe0f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a667668300 con 0x55a666870c00 2024-05-08T05:48:20.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857fe0f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55a667668480 con 0x55a666871800 2024-05-08T05:48:20.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857e32a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a666871c00 0x55a66764eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857e32a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a666871c00 0x55a66764eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35268/0 (socket says 172.21.15.45:35268) 2024-05-08T05:48:20.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857e32a640 1 -- 172.21.15.45:0/2175807276 learned_addr learned my addr 172.21.15.45:0/2175807276 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:20.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857db29640 1 --2- 172.21.15.45:0/2175807276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a666870c00 0x55a66764e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857d328640 1 --2- 172.21.15.45:0/2175807276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a666871800 0x55a66764e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857e32a640 1 -- 172.21.15.45:0/2175807276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a666871800 msgr2=0x55a66764e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:20.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857e32a640 1 --2- 172.21.15.45:0/2175807276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a666871800 0x55a66764e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857e32a640 1 -- 172.21.15.45:0/2175807276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a666870c00 msgr2=0x55a66764e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:20.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857e32a640 1 --2- 172.21.15.45:0/2175807276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a666870c00 0x55a66764e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857e32a640 1 -- 172.21.15.45:0/2175807276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a66689d680 con 0x55a666871c00 2024-05-08T05:48:20.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857d328640 1 --2- 172.21.15.45:0/2175807276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a666871800 0x55a66764e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:20.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857e32a640 1 --2- 172.21.15.45:0/2175807276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a666871c00 0x55a66764eb00 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x55a667699d40 tx=0x55a6669410e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4c54867351a008 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857b324640 1 -- 172.21.15.45:0/2175807276 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a666896680 con 0x55a666871c00 2024-05-08T05:48:20.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857b324640 1 -- 172.21.15.45:0/2175807276 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a6668b8e00 con 0x55a666871c00 2024-05-08T05:48:20.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857b324640 1 -- 172.21.15.45:0/2175807276 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a666896b60 con 0x55a666871c00 2024-05-08T05:48:20.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857fe0f740 1 -- 172.21.15.45:0/2175807276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a66689d4a0 con 0x55a666871c00 2024-05-08T05:48:20.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857fe0f740 1 -- 172.21.15.45:0/2175807276 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55a66689da40 con 0x55a666871c00 2024-05-08T05:48:20.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.065+0000 7f857fe0f740 4 waiting for MDS map... 2024-05-08T05:48:20.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.066+0000 7f857b324640 1 -- 172.21.15.45:0/2175807276 <== mon.0 v2:172.21.15.45:3300/0 4 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x55a667756a00 con 0x55a666871c00 2024-05-08T05:48:20.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.066+0000 7f857b324640 1 -- 172.21.15.45:0/2175807276 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 68) ==== 2885+0+0 (secure 0 0 0) 0x55a6668c9180 con 0x55a666871c00 2024-05-08T05:48:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.066+0000 7f857fe0f740 4 Got MDS map 68 2024-05-08T05:48:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.066+0000 7f857fe0f740 4 datascan.main: Using metadata pool output 2024-05-08T05:48:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.066+0000 7f857fe0f740 4 datascan.main: connecting to RADOS... 2024-05-08T05:48:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.066+0000 7f857fe0f740 1 Processor -- start 2024-05-08T05:48:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.066+0000 7f857fe0f740 1 -- start start 2024-05-08T05:48:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.067+0000 7f857fe0f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a667759400 0x55a66764f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.067+0000 7f857fe0f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a667759800 0x55a66764f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.067+0000 7f857fe0f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a667759c00 0x55a66764fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.067+0000 7f857fe0f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a667668600 con 0x55a667759400 2024-05-08T05:48:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.067+0000 7f857fe0f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a667668780 con 0x55a667759800 2024-05-08T05:48:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.067+0000 7f857fe0f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55a667668900 con 0x55a667759c00 2024-05-08T05:48:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.067+0000 7f857d328640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a667759400 0x55a66764f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.067+0000 7f857d328640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a667759400 0x55a66764f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35276/0 (socket says 172.21.15.45:35276) 2024-05-08T05:48:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.067+0000 7f857d328640 1 -- 172.21.15.45:0/1035511804 learned_addr learned my addr 172.21.15.45:0/1035511804 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.067+0000 7f857db29640 1 --2- 172.21.15.45:0/1035511804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a667759800 0x55a66764f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.067+0000 7f857d328640 1 --2- 172.21.15.45:0/1035511804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a667759c00 0x55a66764fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.067+0000 7f857d328640 1 -- 172.21.15.45:0/1035511804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a667759c00 msgr2=0x55a66764fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.067+0000 7f857d328640 1 --2- 172.21.15.45:0/1035511804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a667759c00 0x55a66764fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.067+0000 7f857d328640 1 -- 172.21.15.45:0/1035511804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a667759800 msgr2=0x55a66764f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.067+0000 7f857d328640 1 --2- 172.21.15.45:0/1035511804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a667759800 0x55a66764f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.067+0000 7f857d328640 1 -- 172.21.15.45:0/1035511804 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a66689d860 con 0x55a667759400 2024-05-08T05:48:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.067+0000 7f857d328640 1 --2- 172.21.15.45:0/1035511804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a667759400 0x55a66764f080 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x55a667744c00 tx=0x55a666940f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7eecf805f3a8d19 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.068+0000 7f8579320640 1 -- 172.21.15.45:0/1035511804 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a6668969c0 con 0x55a667759400 2024-05-08T05:48:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.068+0000 7f8579320640 1 -- 172.21.15.45:0/1035511804 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a6668b8c40 con 0x55a667759400 2024-05-08T05:48:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.068+0000 7f8579320640 1 -- 172.21.15.45:0/1035511804 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a666896680 con 0x55a667759400 2024-05-08T05:48:20.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.068+0000 7f857fe0f740 1 -- 172.21.15.45:0/1035511804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a667759400 msgr2=0x55a66764f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.068+0000 7f857fe0f740 1 --2- 172.21.15.45:0/1035511804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a667759400 0x55a66764f080 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x55a667744c00 tx=0x55a666940f00 comp rx=0 tx=0).stop 2024-05-08T05:48:20.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.068+0000 7f857fe0f740 1 -- 172.21.15.45:0/1035511804 shutdown_connections 2024-05-08T05:48:20.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.068+0000 7f857fe0f740 1 --2- 172.21.15.45:0/1035511804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a667759c00 0x55a66764fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.068+0000 7f857fe0f740 1 --2- 172.21.15.45:0/1035511804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a667759800 0x55a66764f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.068+0000 7f857fe0f740 1 --2- 172.21.15.45:0/1035511804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a667759400 0x55a66764f080 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.068+0000 7f857fe0f740 1 -- 172.21.15.45:0/1035511804 >> 172.21.15.45:0/1035511804 conn(0x55a667759000 msgr2=0x55a6675ed800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:20.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.068+0000 7f857fe0f740 1 -- 172.21.15.45:0/1035511804 shutdown_connections 2024-05-08T05:48:20.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.068+0000 7f857fe0f740 1 -- 172.21.15.45:0/1035511804 wait complete. 2024-05-08T05:48:20.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.069+0000 7f857fe0f740 1 Processor -- start 2024-05-08T05:48:20.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.069+0000 7f857fe0f740 1 -- start start 2024-05-08T05:48:20.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.069+0000 7f857fe0f740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a667759400 0x55a66764f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.069+0000 7f857fe0f740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a667759800 0x55a66764f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.069+0000 7f857fe0f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a667759c00 0x55a66764fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.069+0000 7f857fe0f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a667669980 con 0x55a667759c00 2024-05-08T05:48:20.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.069+0000 7f857fe0f740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55a667669800 con 0x55a667759800 2024-05-08T05:48:20.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.069+0000 7f857fe0f740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55a667669680 con 0x55a667759400 2024-05-08T05:48:20.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.069+0000 7f857d328640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a667759c00 0x55a66764fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.069+0000 7f857d328640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a667759c00 0x55a66764fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35286/0 (socket says 172.21.15.45:35286) 2024-05-08T05:48:20.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.069+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 learned_addr learned my addr 172.21.15.45:0/3168206066 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:20.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.069+0000 7f857db29640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a667759800 0x55a66764f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.069+0000 7f857d328640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a667759400 0x55a66764f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.070+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a667759400 msgr2=0x55a66764f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.070+0000 7f857d328640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a667759400 0x55a66764f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.070+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a667759800 msgr2=0x55a66764f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.070+0000 7f857d328640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a667759800 0x55a66764f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.070+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a66689d680 con 0x55a667759c00 2024-05-08T05:48:20.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.070+0000 7f857d328640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a667759c00 0x55a66764fb80 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x55a667745ec0 tx=0x55a666940ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.070+0000 7f8577b1d640 1 -- 172.21.15.45:0/3168206066 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a666896b60 con 0x55a667759c00 2024-05-08T05:48:20.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.070+0000 7f8577b1d640 1 -- 172.21.15.45:0/3168206066 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a6668b8a80 con 0x55a667759c00 2024-05-08T05:48:20.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.070+0000 7f857fe0f740 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a66689d4a0 con 0x55a667759c00 2024-05-08T05:48:20.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.070+0000 7f8577b1d640 1 -- 172.21.15.45:0/3168206066 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a6668964e0 con 0x55a667759c00 2024-05-08T05:48:20.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.070+0000 7f857fe0f740 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a66689d2c0 con 0x55a667759c00 2024-05-08T05:48:20.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.070+0000 7f857fe0f740 4 datascan.init: resolving metadata pool 13 2024-05-08T05:48:20.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.071+0000 7f8577b1d640 1 -- 172.21.15.45:0/3168206066 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x55a66774cc00 con 0x55a667759c00 2024-05-08T05:48:20.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.072+0000 7f8577b1d640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x55a666c20000 0x55a667650100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.072+0000 7f8577b1d640 1 -- 172.21.15.45:0/3168206066 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x55a667680600 con 0x55a667759c00 2024-05-08T05:48:20.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.072+0000 7f857fe0f740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-08T05:48:20.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.072+0000 7f857fe0f740 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-05-08T05:48:20.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.072+0000 7f857fe0f740 4 datascan.main: opening data pool 'cephfs_data' 2024-05-08T05:48:20.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.072+0000 7f857fe0f740 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x55a666c20400 0x55a667650680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.072+0000 7f857fe0f740 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:1 14.0 14:00000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c20800 con 0x55a666c20400 2024-05-08T05:48:20.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.072+0000 7f857db29640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x55a666c20000 0x55a667650100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.072+0000 7f857d328640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x55a666c20400 0x55a667650680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.072+0000 7f857d328640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x55a666c20400 0x55a667650680 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.073+0000 7f857db29640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x55a666c20000 0x55a667650100 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55a667619200 tx=0x55a666ba8c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.076+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.2 v2:172.21.15.45:6808/785834340 1 ==== osd_op_reply(1 [pgnls start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602000 con 0x55a666c20400 2024-05-08T05:48:20.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.076+0000 7f8578b1f640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x55a666c2a000 0x55a666c30000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.076+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:2 14.20 14:04000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c2a400 con 0x55a666c2a000 2024-05-08T05:48:20.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.076+0000 7f857e32a640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x55a666c2a000 0x55a666c30000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.076+0000 7f857e32a640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x55a666c2a000 0x55a666c30000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.078+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.0 v2:172.21.15.45:6800/3734454393 1 ==== osd_op_reply(2 [pgnls start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c2a000 2024-05-08T05:48:20.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.078+0000 7f8579320640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55a666c2ac00 0x55a666c30580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.078+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:3 14.10 14:08000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c2b000 con 0x55a666c2ac00 2024-05-08T05:48:20.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.078+0000 7f857db29640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55a666c2ac00 0x55a666c30580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.078+0000 7f857db29640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55a666c2ac00 0x55a666c30580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.080+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.1 v2:172.21.15.45:6824/1337179184 1 ==== osd_op_reply(3 [pgnls start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2ac00 2024-05-08T05:48:20.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.080+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:4 14.30 14:0c000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c2a800 con 0x55a666c2a000 2024-05-08T05:48:20.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.081+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.0 v2:172.21.15.45:6800/3734454393 2 ==== osd_op_reply(4 [pgnls start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c2a000 2024-05-08T05:48:20.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.081+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:5 14.8 14:10000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c2b400 con 0x55a666c2a000 2024-05-08T05:48:20.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.081+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.0 v2:172.21.15.45:6800/3734454393 3 ==== osd_op_reply(5 [pgnls start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c2a000 2024-05-08T05:48:20.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.081+0000 7f8578b1f640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x55a666c2b800 0x55a666c30b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.081+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:6 14.28 14:14000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c2bc00 con 0x55a666c2b800 2024-05-08T05:48:20.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.081+0000 7f857d328640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x55a666c2b800 0x55a666c30b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.081+0000 7f857d328640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x55a666c2b800 0x55a666c30b00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.082+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.3 v2:172.21.15.45:6816/1259225446 1 ==== osd_op_reply(6 [pgnls start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602000 con 0x55a666c2b800 2024-05-08T05:48:20.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.082+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:7 14.18 14:18000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3e000 con 0x55a666c2ac00 2024-05-08T05:48:20.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.082+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.1 v2:172.21.15.45:6824/1337179184 2 ==== osd_op_reply(7 [pgnls start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2ac00 2024-05-08T05:48:20.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.082+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:8 14.38 14:1c000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3ec00 con 0x55a666c2ac00 2024-05-08T05:48:20.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.082+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.1 v2:172.21.15.45:6824/1337179184 3 ==== osd_op_reply(8 [pgnls start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2ac00 2024-05-08T05:48:20.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.082+0000 7f8579320640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55a666c3e400 0x55a666c31600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.082+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:9 14.4 14:20000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3e800 con 0x55a666c3e400 2024-05-08T05:48:20.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.082+0000 7f857e32a640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55a666c3e400 0x55a666c31600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.083+0000 7f857e32a640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55a666c3e400 0x55a666c31600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.085+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.6 v2:172.21.15.63:6800/2926759052 1 ==== osd_op_reply(9 [pgnls start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c3e400 2024-05-08T05:48:20.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.085+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:10 14.24 14:24000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3f000 con 0x55a666c2ac00 2024-05-08T05:48:20.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.085+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.1 v2:172.21.15.45:6824/1337179184 4 ==== osd_op_reply(10 [pgnls start_epoch 92 out=49b] v88'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2ac00 2024-05-08T05:48:20.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.085+0000 7f8579320640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x55a666c2b000 0x55a666c31b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.085+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:11 14.14 14:28000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c2a400 con 0x55a666c2b000 2024-05-08T05:48:20.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.085+0000 7f857db29640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x55a666c2b000 0x55a666c31b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.086+0000 7f857db29640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x55a666c2b000 0x55a666c31b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.088+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.4 v2:172.21.15.63:6824/2212879151 1 ==== osd_op_reply(11 [pgnls start_epoch 92 out=49b] v88'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2b000 2024-05-08T05:48:20.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.088+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:12 14.34 14:2c000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3f400 con 0x55a666c20400 2024-05-08T05:48:20.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.088+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.2 v2:172.21.15.45:6808/785834340 2 ==== osd_op_reply(12 [pgnls start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602000 con 0x55a666c20400 2024-05-08T05:48:20.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.088+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:13 14.c 14:30000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c2a800 con 0x55a666c2a000 2024-05-08T05:48:20.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.088+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.0 v2:172.21.15.45:6800/3734454393 4 ==== osd_op_reply(13 [pgnls start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c2a000 2024-05-08T05:48:20.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.088+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:14 14.2c 14:34000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3f800 con 0x55a666c2ac00 2024-05-08T05:48:20.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.088+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.1 v2:172.21.15.45:6824/1337179184 5 ==== osd_op_reply(14 [pgnls start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2ac00 2024-05-08T05:48:20.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.088+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:15 14.1c 14:38000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a667758800 con 0x55a666c2a000 2024-05-08T05:48:20.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.089+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.0 v2:172.21.15.45:6800/3734454393 5 ==== osd_op_reply(15 [pgnls start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c2a000 2024-05-08T05:48:20.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.089+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:16 14.3c 14:3c000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3e000 con 0x55a666c20400 2024-05-08T05:48:20.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.089+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.2 v2:172.21.15.45:6808/785834340 3 ==== osd_op_reply(16 [pgnls start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602000 con 0x55a666c20400 2024-05-08T05:48:20.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.089+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:17 14.2 14:40000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3f400 con 0x55a666c3e400 2024-05-08T05:48:20.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.089+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.6 v2:172.21.15.63:6800/2926759052 2 ==== osd_op_reply(17 [pgnls start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c3e400 2024-05-08T05:48:20.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.089+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:18 14.22 14:44000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3ec00 con 0x55a666c2b000 2024-05-08T05:48:20.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.089+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.4 v2:172.21.15.63:6824/2212879151 2 ==== osd_op_reply(18 [pgnls start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2b000 2024-05-08T05:48:20.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.089+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:19 14.12 14:48000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3e000 con 0x55a666c20400 2024-05-08T05:48:20.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.090+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.2 v2:172.21.15.45:6808/785834340 4 ==== osd_op_reply(19 [pgnls start_epoch 92 out=49b] v88'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602000 con 0x55a666c20400 2024-05-08T05:48:20.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.090+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:20 14.32 14:4c000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3f000 con 0x55a666c2b000 2024-05-08T05:48:20.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.090+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.4 v2:172.21.15.63:6824/2212879151 3 ==== osd_op_reply(20 [pgnls start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2b000 2024-05-08T05:48:20.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.090+0000 7f8579320640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55a666c2bc00 0x55a666c32100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.090+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:21 14.a 14:50000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c20800 con 0x55a666c2bc00 2024-05-08T05:48:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.090+0000 7f857d328640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55a666c2bc00 0x55a666c32100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.090+0000 7f857d328640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55a666c2bc00 0x55a666c32100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.092+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.5 v2:172.21.15.63:6808/2395798255 1 ==== osd_op_reply(21 [pgnls start_epoch 92 out=49b] v88'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602000 con 0x55a666c2bc00 2024-05-08T05:48:20.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.092+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:22 14.2a 14:54000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3e800 con 0x55a666c2b000 2024-05-08T05:48:20.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.093+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.4 v2:172.21.15.63:6824/2212879151 4 ==== osd_op_reply(22 [pgnls start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2b000 2024-05-08T05:48:20.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.093+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:23 14.1a 14:58000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a667657400 con 0x55a666c2a000 2024-05-08T05:48:20.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.093+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.0 v2:172.21.15.45:6800/3734454393 6 ==== osd_op_reply(23 [pgnls start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c2a000 2024-05-08T05:48:20.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.093+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:24 14.3a 14:5c000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c2b400 con 0x55a666c2a000 2024-05-08T05:48:20.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.093+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.0 v2:172.21.15.45:6800/3734454393 7 ==== osd_op_reply(24 [pgnls start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c2a000 2024-05-08T05:48:20.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.093+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:25 14.6 14:60000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3f800 con 0x55a666c2a000 2024-05-08T05:48:20.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.093+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.0 v2:172.21.15.45:6800/3734454393 8 ==== osd_op_reply(25 [pgnls start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c2a000 2024-05-08T05:48:20.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.093+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:26 14.26 14:64000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3f400 con 0x55a666c2b000 2024-05-08T05:48:20.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.094+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.4 v2:172.21.15.63:6824/2212879151 5 ==== osd_op_reply(26 [pgnls start_epoch 92 out=49b] v88'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2b000 2024-05-08T05:48:20.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.094+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:27 14.16 14:68000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3ec00 con 0x55a666c2b800 2024-05-08T05:48:20.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.094+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.3 v2:172.21.15.45:6816/1259225446 2 ==== osd_op_reply(27 [pgnls start_epoch 92 out=49b] v88'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602000 con 0x55a666c2b800 2024-05-08T05:48:20.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.094+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:28 14.36 14:6c000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3f800 con 0x55a666c20400 2024-05-08T05:48:20.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.094+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.2 v2:172.21.15.45:6808/785834340 5 ==== osd_op_reply(28 [pgnls start_epoch 92 out=49b] v88'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602000 con 0x55a666c20400 2024-05-08T05:48:20.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.094+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:29 14.e 14:70000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3f000 con 0x55a666c2b000 2024-05-08T05:48:20.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.095+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.4 v2:172.21.15.63:6824/2212879151 6 ==== osd_op_reply(29 [pgnls start_epoch 92 out=49b] v88'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2b000 2024-05-08T05:48:20.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.095+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:30 14.2e 14:74000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c2a800 con 0x55a666c2b800 2024-05-08T05:48:20.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.095+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.3 v2:172.21.15.45:6816/1259225446 3 ==== osd_op_reply(30 [pgnls start_epoch 92 out=49b] v88'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602000 con 0x55a666c2b800 2024-05-08T05:48:20.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.095+0000 7f8579320640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x55a666c3e800 0x55a666c32680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.095+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:31 14.1e 14:78000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3fc00 con 0x55a666c3e800 2024-05-08T05:48:20.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.095+0000 7f857e32a640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x55a666c3e800 0x55a666c32680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.095+0000 7f857e32a640 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x55a666c3e800 0x55a666c32680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.096+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.7 v2:172.21.15.63:6816/1163469437 1 ==== osd_op_reply(31 [pgnls start_epoch 92 out=49b] v88'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c3e800 2024-05-08T05:48:20.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.096+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:32 14.3e 14:7c000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c2b400 con 0x55a666c2bc00 2024-05-08T05:48:20.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.096+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.5 v2:172.21.15.63:6808/2395798255 2 ==== osd_op_reply(32 [pgnls start_epoch 92 out=49b] v88'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602000 con 0x55a666c2bc00 2024-05-08T05:48:20.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.096+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:33 14.1 14:80000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c2a400 con 0x55a666c2b800 2024-05-08T05:48:20.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.096+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.3 v2:172.21.15.45:6816/1259225446 4 ==== osd_op_reply(33 [pgnls start_epoch 92 out=49b] v88'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602000 con 0x55a666c2b800 2024-05-08T05:48:20.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.096+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:34 14.21 14:84000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a667657400 con 0x55a666c2bc00 2024-05-08T05:48:20.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.097+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.5 v2:172.21.15.63:6808/2395798255 3 ==== osd_op_reply(34 [pgnls start_epoch 92 out=49b] v88'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602000 con 0x55a666c2bc00 2024-05-08T05:48:20.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.097+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:35 14.11 14:88000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c2a800 con 0x55a666c2b000 2024-05-08T05:48:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.097+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.4 v2:172.21.15.63:6824/2212879151 7 ==== osd_op_reply(35 [pgnls start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2b000 2024-05-08T05:48:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.097+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:36 14.31 14:8c000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a667758800 con 0x55a666c2ac00 2024-05-08T05:48:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.097+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.1 v2:172.21.15.45:6824/1337179184 6 ==== osd_op_reply(36 [pgnls start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2ac00 2024-05-08T05:48:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.098+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:37 14.9 14:90000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c2b400 con 0x55a666c2bc00 2024-05-08T05:48:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.098+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.5 v2:172.21.15.63:6808/2395798255 4 ==== osd_op_reply(37 [pgnls start_epoch 92 out=49b] v88'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602000 con 0x55a666c2bc00 2024-05-08T05:48:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.098+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:38 14.29 14:94000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c56000 con 0x55a666c3e400 2024-05-08T05:48:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.098+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.6 v2:172.21.15.63:6800/2926759052 3 ==== osd_op_reply(38 [pgnls start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c3e400 2024-05-08T05:48:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.098+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:39 14.19 14:98000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3e000 con 0x55a666c3e400 2024-05-08T05:48:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.098+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.6 v2:172.21.15.63:6800/2926759052 4 ==== osd_op_reply(39 [pgnls start_epoch 92 out=49b] v88'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c3e400 2024-05-08T05:48:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.098+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:40 14.39 14:9c000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c56400 con 0x55a666c3e800 2024-05-08T05:48:20.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.099+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.7 v2:172.21.15.63:6816/1163469437 2 ==== osd_op_reply(40 [pgnls start_epoch 92 out=49b] v88'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c3e800 2024-05-08T05:48:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.099+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:41 14.5 14:a0000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3ec00 con 0x55a666c2a000 2024-05-08T05:48:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.099+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.0 v2:172.21.15.45:6800/3734454393 9 ==== osd_op_reply(41 [pgnls start_epoch 92 out=49b] v88'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c2a000 2024-05-08T05:48:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.099+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:42 14.25 14:a4000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c56800 con 0x55a666c20400 2024-05-08T05:48:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.099+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.2 v2:172.21.15.45:6808/785834340 6 ==== osd_op_reply(42 [pgnls start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602000 con 0x55a666c20400 2024-05-08T05:48:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.099+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:43 14.15 14:a8000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3f800 con 0x55a666c3e800 2024-05-08T05:48:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.099+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.7 v2:172.21.15.63:6816/1163469437 3 ==== osd_op_reply(43 [pgnls start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c3e800 2024-05-08T05:48:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.099+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:44 14.35 14:ac000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c56c00 con 0x55a666c2b000 2024-05-08T05:48:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.100+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.4 v2:172.21.15.63:6824/2212879151 8 ==== osd_op_reply(44 [pgnls start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2b000 2024-05-08T05:48:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.100+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:45 14.d 14:b0000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c20800 con 0x55a666c3e400 2024-05-08T05:48:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.100+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.6 v2:172.21.15.63:6800/2926759052 5 ==== osd_op_reply(45 [pgnls start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c3e400 2024-05-08T05:48:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.100+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:46 14.2d 14:b4000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c57000 con 0x55a666c2b000 2024-05-08T05:48:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.100+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.4 v2:172.21.15.63:6824/2212879151 9 ==== osd_op_reply(46 [pgnls start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2b000 2024-05-08T05:48:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.100+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:47 14.1d 14:b8000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3f400 con 0x55a666c2a000 2024-05-08T05:48:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.100+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.0 v2:172.21.15.45:6800/3734454393 10 ==== osd_op_reply(47 [pgnls start_epoch 92 out=49b] v88'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c2a000 2024-05-08T05:48:20.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.100+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:48 14.3d 14:bc000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c57400 con 0x55a666c2b800 2024-05-08T05:48:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.101+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.3 v2:172.21.15.45:6816/1259225446 5 ==== osd_op_reply(48 [pgnls start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602000 con 0x55a666c2b800 2024-05-08T05:48:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.101+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:49 14.3 14:c0000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3f000 con 0x55a666c2a000 2024-05-08T05:48:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.101+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.0 v2:172.21.15.45:6800/3734454393 11 ==== osd_op_reply(49 [pgnls start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c2a000 2024-05-08T05:48:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.101+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:50 14.23 14:c4000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c57800 con 0x55a666c3e800 2024-05-08T05:48:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.101+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.7 v2:172.21.15.63:6816/1163469437 4 ==== osd_op_reply(50 [pgnls start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c3e800 2024-05-08T05:48:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.101+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:51 14.13 14:c8000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3fc00 con 0x55a666c20400 2024-05-08T05:48:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.101+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.2 v2:172.21.15.45:6808/785834340 7 ==== osd_op_reply(51 [pgnls start_epoch 92 out=49b] v88'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602000 con 0x55a666c20400 2024-05-08T05:48:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.101+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:52 14.33 14:cc000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c56c00 con 0x55a666c2ac00 2024-05-08T05:48:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.102+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.1 v2:172.21.15.45:6824/1337179184 7 ==== osd_op_reply(52 [pgnls start_epoch 92 out=49b] v88'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2ac00 2024-05-08T05:48:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.102+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:53 14.b 14:d0000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3e000 con 0x55a666c2a000 2024-05-08T05:48:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.102+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.0 v2:172.21.15.45:6800/3734454393 12 ==== osd_op_reply(53 [pgnls start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c2a000 2024-05-08T05:48:20.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.102+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:54 14.2b 14:d4000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c57000 con 0x55a666c2b000 2024-05-08T05:48:20.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.102+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.4 v2:172.21.15.63:6824/2212879151 10 ==== osd_op_reply(54 [pgnls start_epoch 92 out=49b] v88'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2b000 2024-05-08T05:48:20.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.102+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:55 14.1b 14:d8000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3ec00 con 0x55a666c2ac00 2024-05-08T05:48:20.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.103+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.1 v2:172.21.15.45:6824/1337179184 8 ==== osd_op_reply(55 [pgnls start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2ac00 2024-05-08T05:48:20.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.103+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:56 14.3b 14:dc000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c56000 con 0x55a666c2b000 2024-05-08T05:48:20.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.103+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.4 v2:172.21.15.63:6824/2212879151 11 ==== osd_op_reply(56 [pgnls start_epoch 92 out=49b] v88'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2b000 2024-05-08T05:48:20.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.103+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:57 14.7 14:e0000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3f800 con 0x55a666c2bc00 2024-05-08T05:48:20.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.103+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.5 v2:172.21.15.63:6808/2395798255 5 ==== osd_op_reply(57 [pgnls start_epoch 92 out=81b] v88'6 uv6 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55a667602000 con 0x55a666c2bc00 2024-05-08T05:48:20.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.103+0000 7f857fe0f740 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:58 14.7 14:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x55a666c20c00 con 0x55a666c2bc00 2024-05-08T05:48:20.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.103+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.5 v2:172.21.15.63:6808/2395798255 6 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55a667602000 con 0x55a666c2bc00 2024-05-08T05:48:20.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.103+0000 7f857fe0f740 10 datascan.operator(): handling object 1099511627777.0 2024-05-08T05:48:20.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.103+0000 7f857fe0f740 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:59 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x55a666c21000 con 0x55a666c2bc00 2024-05-08T05:48:20.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.105+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.5 v2:172.21.15.63:6808/2395798255 7 ==== osd_op_reply(59 10000000001.00000000 [call] v92'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55a667602000 con 0x55a666c2bc00 2024-05-08T05:48:20.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.105+0000 7f857fe0f740 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:60 14.27 14:e4000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c21400 con 0x55a666c2bc00 2024-05-08T05:48:20.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.105+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.5 v2:172.21.15.63:6808/2395798255 8 ==== osd_op_reply(60 [pgnls start_epoch 92 out=49b] v88'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602000 con 0x55a666c2bc00 2024-05-08T05:48:20.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.105+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:61 14.17 14:e8000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a667758800 con 0x55a666c2ac00 2024-05-08T05:48:20.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.106+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.1 v2:172.21.15.45:6824/1337179184 9 ==== osd_op_reply(61 [pgnls start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2ac00 2024-05-08T05:48:20.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.106+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:62 14.37 14:ec000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c56400 con 0x55a666c2b000 2024-05-08T05:48:20.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.106+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 <== osd.4 v2:172.21.15.63:6824/2212879151 12 ==== osd_op_reply(62 [pgnls start_epoch 92 out=49b] v88'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602780 con 0x55a666c2b000 2024-05-08T05:48:20.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.106+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:63 14.f 14:f0000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c2a800 con 0x55a666c3e800 2024-05-08T05:48:20.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.106+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.7 v2:172.21.15.63:6816/1163469437 5 ==== osd_op_reply(63 [pgnls start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c3e800 2024-05-08T05:48:20.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.106+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:64 14.2f 14:f4000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c57c00 con 0x55a666c2a000 2024-05-08T05:48:20.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.106+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.0 v2:172.21.15.45:6800/3734454393 13 ==== osd_op_reply(64 [pgnls start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c2a000 2024-05-08T05:48:20.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.106+0000 7f8579320640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:65 14.1f 14:f8000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c3fc00 con 0x55a666c2a000 2024-05-08T05:48:20.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857e32a640 1 -- 172.21.15.45:0/3168206066 <== osd.0 v2:172.21.15.45:6800/3734454393 14 ==== osd_op_reply(65 [pgnls start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602500 con 0x55a666c2a000 2024-05-08T05:48:20.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f8578b1f640 1 -- 172.21.15.45:0/3168206066 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:66 14.3f 14:fc000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x55a666c20800 con 0x55a666c2b800 2024-05-08T05:48:20.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857d328640 1 -- 172.21.15.45:0/3168206066 <== osd.3 v2:172.21.15.45:6816/1259225446 6 ==== osd_op_reply(66 [pgnls start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55a667602000 con 0x55a666c2b800 2024-05-08T05:48:20.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 -- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x55a666c2a000 msgr2=0x55a666c30000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x55a666c2a000 0x55a666c30000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 -- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55a666c2ac00 msgr2=0x55a666c30580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55a666c2ac00 0x55a666c30580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 -- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x55a666c20400 msgr2=0x55a667650680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x55a666c20400 0x55a667650680 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 -- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x55a666c2b800 msgr2=0x55a666c30b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x55a666c2b800 0x55a666c30b00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 -- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x55a666c2b000 msgr2=0x55a666c31b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x55a666c2b000 0x55a666c31b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 reap_dead start 2024-05-08T05:48:20.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 -- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55a666c2bc00 msgr2=0x55a666c32100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55a666c2bc00 0x55a666c32100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857db29640 1 -- 172.21.15.45:0/3168206066 reap_dead start 2024-05-08T05:48:20.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 -- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55a666c3e400 msgr2=0x55a666c31600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55a666c3e400 0x55a666c31600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 -- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x55a666c3e800 msgr2=0x55a666c32680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x55a666c3e800 0x55a666c32680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 -- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x55a666c20000 msgr2=0x55a667650100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x55a666c20000 0x55a667650100 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55a667619200 tx=0x55a666ba8c00 comp rx=0 tx=0).stop 2024-05-08T05:48:20.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 -- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a667759c00 msgr2=0x55a66764fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a667759c00 0x55a66764fb80 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x55a667745ec0 tx=0x55a666940ea0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 -- 172.21.15.45:0/3168206066 shutdown_connections 2024-05-08T05:48:20.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x55a666c3e800 0x55a666c32680 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55a666c3e400 0x55a666c31600 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x55a666c20000 0x55a667650100 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 --2- 172.21.15.45:0/3168206066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a667759c00 0x55a66764fb80 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 -- 172.21.15.45:0/3168206066 >> 172.21.15.45:0/3168206066 conn(0x55a667759000 msgr2=0x55a6675ed800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:20.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.107+0000 7f857fe0f740 1 -- 172.21.15.45:0/3168206066 shutdown_connections 2024-05-08T05:48:20.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.108+0000 7f857fe0f740 1 -- 172.21.15.45:0/3168206066 wait complete. 2024-05-08T05:48:20.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.108+0000 7f857fe0f740 1 -- 172.21.15.45:0/2175807276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a666871c00 msgr2=0x55a66764eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.108+0000 7f857fe0f740 1 --2- 172.21.15.45:0/2175807276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a666871c00 0x55a66764eb00 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x55a667699d40 tx=0x55a6669410e0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.108+0000 7f857fe0f740 1 -- 172.21.15.45:0/2175807276 shutdown_connections 2024-05-08T05:48:20.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.108+0000 7f857fe0f740 1 --2- 172.21.15.45:0/2175807276 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55a666871800 0x55a66764e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.108+0000 7f857fe0f740 1 --2- 172.21.15.45:0/2175807276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a666871c00 0x55a66764eb00 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.108+0000 7f857fe0f740 1 --2- 172.21.15.45:0/2175807276 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55a666870c00 0x55a66764e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.108+0000 7f857fe0f740 1 -- 172.21.15.45:0/2175807276 >> 172.21.15.45:0/2175807276 conn(0x55a666871000 msgr2=0x55a6675ed000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:20.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.108+0000 7f857fe0f740 1 -- 172.21.15.45:0/2175807276 shutdown_connections 2024-05-08T05:48:20.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.108+0000 7f857fe0f740 1 -- 172.21.15.45:0/2175807276 wait complete. 2024-05-08T05:48:20.115 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.092509, result: 2024-05-08T05:48:20.116 DEBUG:teuthology.orchestra.run.smithi045:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-05-08T05:48:20.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.196+0000 7fde1501d740 1 -- start start 2024-05-08T05:48:20.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.196+0000 7fde1501d740 1 Processor -- start 2024-05-08T05:48:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.197+0000 7fde1501d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b3c544c00 0x564b3d320000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.197+0000 7fde1501d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b3c545800 0x564b3d320580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.197+0000 7fde1501d740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564b3c545c00 0x564b3d320b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.197+0000 7fde1501d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564b3d2c2180 con 0x564b3c544c00 2024-05-08T05:48:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.197+0000 7fde1501d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564b3d2c2300 con 0x564b3c545800 2024-05-08T05:48:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.197+0000 7fde1501d740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564b3d2c2480 con 0x564b3c545c00 2024-05-08T05:48:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.197+0000 7fde12d0e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b3c544c00 0x564b3d320000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.197+0000 7fde12d0e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b3c544c00 0x564b3d320000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35298/0 (socket says 172.21.15.45:35298) 2024-05-08T05:48:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.197+0000 7fde12d0e640 1 -- 172.21.15.45:0/4172759371 learned_addr learned my addr 172.21.15.45:0/4172759371 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.197+0000 7fde12d0e640 1 -- 172.21.15.45:0/4172759371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564b3c545c00 msgr2=0x564b3d320b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.197+0000 7fde12d0e640 1 --2- 172.21.15.45:0/4172759371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564b3c545c00 0x564b3d320b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.198+0000 7fde12d0e640 1 -- 172.21.15.45:0/4172759371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b3c545800 msgr2=0x564b3d320580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:20.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.198+0000 7fde1250d640 1 --2- 172.21.15.45:0/4172759371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b3c545800 0x564b3d320580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.198+0000 7fde12d0e640 1 --2- 172.21.15.45:0/4172759371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b3c545800 0x564b3d320580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.198+0000 7fde12d0e640 1 -- 172.21.15.45:0/4172759371 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b3c571680 con 0x564b3c544c00 2024-05-08T05:48:20.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.198+0000 7fde1250d640 1 --2- 172.21.15.45:0/4172759371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b3c545800 0x564b3d320580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:48:20.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.198+0000 7fde12d0e640 1 --2- 172.21.15.45:0/4172759371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b3c544c00 0x564b3d320000 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x564b3d36bbf0 tx=0x564b3c614a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314beef01b38973d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.198+0000 7fde10509640 1 -- 172.21.15.45:0/4172759371 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564b3c56a680 con 0x564b3c544c00 2024-05-08T05:48:20.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.198+0000 7fde10509640 1 -- 172.21.15.45:0/4172759371 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564b3c58c1c0 con 0x564b3c544c00 2024-05-08T05:48:20.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.198+0000 7fde10509640 1 -- 172.21.15.45:0/4172759371 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564b3c56ab60 con 0x564b3c544c00 2024-05-08T05:48:20.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.198+0000 7fde1501d740 1 -- 172.21.15.45:0/4172759371 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b3c571860 con 0x564b3c544c00 2024-05-08T05:48:20.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.198+0000 7fde1501d740 1 -- 172.21.15.45:0/4172759371 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=0}) -- 0x564b3c5712c0 con 0x564b3c544c00 2024-05-08T05:48:20.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.198+0000 7fde1501d740 4 waiting for MDS map... 2024-05-08T05:48:20.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.198+0000 7fde10509640 1 -- 172.21.15.45:0/4172759371 <== mon.0 v2:172.21.15.45:3300/0 4 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x564b3d41a600 con 0x564b3c544c00 2024-05-08T05:48:20.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.198+0000 7fde10509640 1 -- 172.21.15.45:0/4172759371 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 68) ==== 2885+0+0 (secure 0 0 0) 0x564b3c59d180 con 0x564b3c544c00 2024-05-08T05:48:20.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.198+0000 7fde1501d740 4 Got MDS map 68 2024-05-08T05:48:20.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.198+0000 7fde1501d740 4 datascan.main: Using metadata pool output 2024-05-08T05:48:20.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.198+0000 7fde1501d740 4 datascan.main: connecting to RADOS... 2024-05-08T05:48:20.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.199+0000 7fde1501d740 1 Processor -- start 2024-05-08T05:48:20.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.199+0000 7fde1501d740 1 -- start start 2024-05-08T05:48:20.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.199+0000 7fde1501d740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564b3d421400 0x564b3d321080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.199+0000 7fde1501d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b3d421800 0x564b3d321600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.199+0000 7fde1501d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b3d421c00 0x564b3d321b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.199+0000 7fde1501d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564b3d2c2600 con 0x564b3d421800 2024-05-08T05:48:20.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.199+0000 7fde1501d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564b3d2c2780 con 0x564b3d421c00 2024-05-08T05:48:20.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.199+0000 7fde1501d740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564b3d2c2900 con 0x564b3d421400 2024-05-08T05:48:20.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.200+0000 7fde12d0e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b3d421800 0x564b3d321600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.200+0000 7fde1350f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564b3d421400 0x564b3d321080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.200+0000 7fde12d0e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b3d421800 0x564b3d321600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35308/0 (socket says 172.21.15.45:35308) 2024-05-08T05:48:20.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.200+0000 7fde12d0e640 1 -- 172.21.15.45:0/2028036110 learned_addr learned my addr 172.21.15.45:0/2028036110 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.200+0000 7fde1250d640 1 --2- 172.21.15.45:0/2028036110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b3d421c00 0x564b3d321b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.200+0000 7fde12d0e640 1 -- 172.21.15.45:0/2028036110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564b3d421400 msgr2=0x564b3d321080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.200+0000 7fde12d0e640 1 --2- 172.21.15.45:0/2028036110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564b3d421400 0x564b3d321080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.200+0000 7fde12d0e640 1 -- 172.21.15.45:0/2028036110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b3d421c00 msgr2=0x564b3d321b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.200+0000 7fde12d0e640 1 --2- 172.21.15.45:0/2028036110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b3d421c00 0x564b3d321b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.200+0000 7fde12d0e640 1 -- 172.21.15.45:0/2028036110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b3c5714a0 con 0x564b3d421800 2024-05-08T05:48:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.200+0000 7fde12d0e640 1 --2- 172.21.15.45:0/2028036110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b3d421800 0x564b3d321600 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x564b3d43ad80 tx=0x564b3d2ea720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3d2656de4f5fa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.200+0000 7fde0e505640 1 -- 172.21.15.45:0/2028036110 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564b3c56a820 con 0x564b3d421800 2024-05-08T05:48:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.200+0000 7fde0e505640 1 -- 172.21.15.45:0/2028036110 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564b3c58ce00 con 0x564b3d421800 2024-05-08T05:48:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.200+0000 7fde0e505640 1 -- 172.21.15.45:0/2028036110 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564b3c56a680 con 0x564b3d421800 2024-05-08T05:48:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.201+0000 7fde1501d740 1 -- 172.21.15.45:0/2028036110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b3d421800 msgr2=0x564b3d321600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.201+0000 7fde1501d740 1 --2- 172.21.15.45:0/2028036110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b3d421800 0x564b3d321600 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x564b3d43ad80 tx=0x564b3d2ea720 comp rx=0 tx=0).stop 2024-05-08T05:48:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.201+0000 7fde1501d740 1 -- 172.21.15.45:0/2028036110 shutdown_connections 2024-05-08T05:48:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.201+0000 7fde1501d740 1 --2- 172.21.15.45:0/2028036110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b3d421c00 0x564b3d321b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.201+0000 7fde1501d740 1 --2- 172.21.15.45:0/2028036110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b3d421800 0x564b3d321600 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.201+0000 7fde1501d740 1 --2- 172.21.15.45:0/2028036110 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564b3d421400 0x564b3d321080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.201+0000 7fde1501d740 1 -- 172.21.15.45:0/2028036110 >> 172.21.15.45:0/2028036110 conn(0x564b3d421000 msgr2=0x564b3c637000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:20.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.201+0000 7fde1501d740 1 -- 172.21.15.45:0/2028036110 shutdown_connections 2024-05-08T05:48:20.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.201+0000 7fde1501d740 1 -- 172.21.15.45:0/2028036110 wait complete. 2024-05-08T05:48:20.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.201+0000 7fde1501d740 1 Processor -- start 2024-05-08T05:48:20.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.201+0000 7fde1501d740 1 -- start start 2024-05-08T05:48:20.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.202+0000 7fde1501d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b3d421400 0x564b3d321080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.202+0000 7fde1501d740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564b3d421800 0x564b3d321600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.202+0000 7fde1501d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b3d421c00 0x564b3d321b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.202+0000 7fde1501d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564b3d2c3b00 con 0x564b3d421c00 2024-05-08T05:48:20.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.202+0000 7fde1501d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564b3d2c3980 con 0x564b3d421400 2024-05-08T05:48:20.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.202+0000 7fde1501d740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564b3d2c2600 con 0x564b3d421800 2024-05-08T05:48:20.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.202+0000 7fde1250d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b3d421c00 0x564b3d321b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.202+0000 7fde1250d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b3d421c00 0x564b3d321b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35320/0 (socket says 172.21.15.45:35320) 2024-05-08T05:48:20.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.202+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 learned_addr learned my addr 172.21.15.45:0/3739278721 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:20.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.202+0000 7fde12d0e640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564b3d421800 0x564b3d321600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.202+0000 7fde1350f640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b3d421400 0x564b3d321080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.202+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564b3d421800 msgr2=0x564b3d321600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.202+0000 7fde1250d640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564b3d421800 0x564b3d321600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.202+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b3d421400 msgr2=0x564b3d321080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.202+0000 7fde1250d640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b3d421400 0x564b3d321080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.202+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b3c571680 con 0x564b3d421c00 2024-05-08T05:48:20.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.202+0000 7fde1250d640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b3d421c00 0x564b3d321b80 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x564b3d43baa0 tx=0x564b3d2ebf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.203+0000 7fde0cd02640 1 -- 172.21.15.45:0/3739278721 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564b3c56a9c0 con 0x564b3d421c00 2024-05-08T05:48:20.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.203+0000 7fde0cd02640 1 -- 172.21.15.45:0/3739278721 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564b3c58ca80 con 0x564b3d421c00 2024-05-08T05:48:20.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.203+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564b3c571860 con 0x564b3d421c00 2024-05-08T05:48:20.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.203+0000 7fde0cd02640 1 -- 172.21.15.45:0/3739278721 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564b3c56a820 con 0x564b3d421c00 2024-05-08T05:48:20.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.203+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b3c5710e0 con 0x564b3d421c00 2024-05-08T05:48:20.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.203+0000 7fde1501d740 4 datascan.init: resolving metadata pool 13 2024-05-08T05:48:20.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.204+0000 7fde0cd02640 1 -- 172.21.15.45:0/3739278721 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x564b3d494c00 con 0x564b3d421c00 2024-05-08T05:48:20.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.204+0000 7fde0cd02640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x564b3d34b000 0x564b3d322100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.204+0000 7fde0cd02640 1 -- 172.21.15.45:0/3739278721 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x564b3d350600 con 0x564b3d421c00 2024-05-08T05:48:20.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.204+0000 7fde1501d740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-08T05:48:20.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.204+0000 7fde1501d740 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-05-08T05:48:20.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.204+0000 7fde1501d740 4 datascan.main: opening data pool 'cephfs_data' 2024-05-08T05:48:20.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.204+0000 7fde1350f640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x564b3d34b000 0x564b3d322100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.204+0000 7fde1501d740 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x564b3d34b400 0x564b3d322680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.204+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x564b3d34b800 con 0x564b3d34b400 2024-05-08T05:48:20.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.205+0000 7fde12d0e640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x564b3d34b400 0x564b3d322680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.205+0000 7fde1350f640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x564b3d34b000 0x564b3d322100 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x564b3d36bb60 tx=0x564b3c864fc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.205+0000 7fde12d0e640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x564b3d34b400 0x564b3d322680 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.205+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv25 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.205+0000 7fde1501d740 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x564b3d34bc00 0x564b3d322c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.205+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:2 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x564b3d34ac00 con 0x564b3d34bc00 2024-05-08T05:48:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.205+0000 7fde1350f640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x564b3d34bc00 0x564b3d322c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.206+0000 7fde1350f640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x564b3d34bc00 0x564b3d322c00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.206+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.6 v2:172.21.15.63:6800/2926759052 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3d34bc00 2024-05-08T05:48:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.206+0000 7fde1501d740 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x564b3d34a800 0x564b3d323180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.206+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:3 14.0 14:00000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3d34a400 con 0x564b3d34a800 2024-05-08T05:48:20.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.206+0000 7fde12d0e640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x564b3d34a800 0x564b3d323180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.206+0000 7fde12d0e640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x564b3d34a800 0x564b3d323180 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.207+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.2 v2:172.21.15.45:6808/785834340 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34a800 2024-05-08T05:48:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.207+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:4 14.20 14:04000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c858c00 con 0x564b3d34b400 2024-05-08T05:48:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.207+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.207+0000 7fde0e505640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x564b3c859000 0x564b3c8f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.207+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:5 14.10 14:08000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859400 con 0x564b3c859000 2024-05-08T05:48:20.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.207+0000 7fde1250d640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x564b3c859000 0x564b3c8f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.207+0000 7fde1250d640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x564b3c859000 0x564b3c8f8000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.208+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.208+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:6 14.30 14:0c000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859c00 con 0x564b3d34b400 2024-05-08T05:48:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.208+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.208+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:7 14.8 14:10000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859800 con 0x564b3d34b400 2024-05-08T05:48:20.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.208+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 4 ==== osd_op_reply(7 [pgnls-filter start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.208+0000 7fde0dd04640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x564b3c900000 0x564b3c8f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.208+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:8 14.28 14:14000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c900400 con 0x564b3c900000 2024-05-08T05:48:20.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.208+0000 7fde1350f640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x564b3c900000 0x564b3c8f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.209+0000 7fde1350f640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x564b3c900000 0x564b3c8f8580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.209+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.3 v2:172.21.15.45:6816/1259225446 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3c900000 2024-05-08T05:48:20.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.209+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:9 14.18 14:18000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c900c00 con 0x564b3c859000 2024-05-08T05:48:20.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.209+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.209+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:10 14.38 14:1c000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c900800 con 0x564b3c859000 2024-05-08T05:48:20.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.209+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.209+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:11 14.4 14:20000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c901000 con 0x564b3d34bc00 2024-05-08T05:48:20.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.210+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.6 v2:172.21.15.63:6800/2926759052 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3d34bc00 2024-05-08T05:48:20.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.210+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:12 14.24 14:24000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c900400 con 0x564b3c859000 2024-05-08T05:48:20.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.210+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 92 out=49b] v88'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.210+0000 7fde0e505640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x564b3c901400 0x564b3c8f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.210+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:13 14.14 14:28000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c900c00 con 0x564b3c901400 2024-05-08T05:48:20.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.210+0000 7fde12d0e640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x564b3c901400 0x564b3c8f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.210+0000 7fde12d0e640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x564b3c901400 0x564b3c8f8b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.211+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 92 out=49b] v88'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.211+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:14 14.34 14:2c000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c901000 con 0x564b3d34a800 2024-05-08T05:48:20.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.211+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.2 v2:172.21.15.45:6808/785834340 2 ==== osd_op_reply(14 [pgnls-filter start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34a800 2024-05-08T05:48:20.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.211+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:15 14.c 14:30000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c900800 con 0x564b3d34b400 2024-05-08T05:48:20.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.211+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 5 ==== osd_op_reply(15 [pgnls-filter start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.211+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:16 14.2c 14:34000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3d34ac00 con 0x564b3c859000 2024-05-08T05:48:20.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.211+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 5 ==== osd_op_reply(16 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.212+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:17 14.1c 14:38000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859400 con 0x564b3d34b400 2024-05-08T05:48:20.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.212+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.212+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:18 14.3c 14:3c000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c858800 con 0x564b3d34a800 2024-05-08T05:48:20.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.212+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.2 v2:172.21.15.45:6808/785834340 3 ==== osd_op_reply(18 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34a800 2024-05-08T05:48:20.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.212+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:19 14.2 14:40000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3d32d400 con 0x564b3d34bc00 2024-05-08T05:48:20.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.212+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.6 v2:172.21.15.63:6800/2926759052 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3d34bc00 2024-05-08T05:48:20.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.212+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:20 14.22 14:44000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c900c00 con 0x564b3c901400 2024-05-08T05:48:20.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.213+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.213+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:21 14.12 14:48000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c901000 con 0x564b3d34a800 2024-05-08T05:48:20.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.213+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.2 v2:172.21.15.45:6808/785834340 4 ==== osd_op_reply(21 [pgnls-filter start_epoch 92 out=49b] v88'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34a800 2024-05-08T05:48:20.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.213+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:22 14.32 14:4c000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c858c00 con 0x564b3c901400 2024-05-08T05:48:20.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.213+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 3 ==== osd_op_reply(22 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.213+0000 7fde0e505640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x564b3c900800 0x564b3c8f9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.213+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:23 14.a 14:50000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c900400 con 0x564b3c900800 2024-05-08T05:48:20.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.213+0000 7fde1250d640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x564b3c900800 0x564b3c8f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.214+0000 7fde1250d640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x564b3c900800 0x564b3c8f9080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.214+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.5 v2:172.21.15.63:6808/2395798255 1 ==== osd_op_reply(23 [pgnls-filter start_epoch 92 out=49b] v88'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c900800 2024-05-08T05:48:20.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.214+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:24 14.2a 14:54000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859c00 con 0x564b3c901400 2024-05-08T05:48:20.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.214+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.214+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:25 14.1a 14:58000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c901800 con 0x564b3d34b400 2024-05-08T05:48:20.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.215+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.215+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:26 14.3a 14:5c000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859800 con 0x564b3d34b400 2024-05-08T05:48:20.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.215+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 8 ==== osd_op_reply(26 [pgnls-filter start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.215+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:27 14.6 14:60000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c901c00 con 0x564b3d34b400 2024-05-08T05:48:20.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.215+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 9 ==== osd_op_reply(27 [pgnls-filter start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.215+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:28 14.26 14:64000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3d34a400 con 0x564b3c901400 2024-05-08T05:48:20.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.215+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 5 ==== osd_op_reply(28 [pgnls-filter start_epoch 92 out=49b] v88'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.215+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:29 14.16 14:68000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859400 con 0x564b3c900000 2024-05-08T05:48:20.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.216+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.3 v2:172.21.15.45:6816/1259225446 2 ==== osd_op_reply(29 [pgnls-filter start_epoch 92 out=49b] v88'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3c900000 2024-05-08T05:48:20.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.216+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:30 14.36 14:6c000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859c00 con 0x564b3d34a800 2024-05-08T05:48:20.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.216+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.2 v2:172.21.15.45:6808/785834340 5 ==== osd_op_reply(30 [pgnls-filter start_epoch 92 out=49b] v88'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34a800 2024-05-08T05:48:20.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.216+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:31 14.e 14:70000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c858800 con 0x564b3c901400 2024-05-08T05:48:20.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.216+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 6 ==== osd_op_reply(31 [pgnls-filter start_epoch 92 out=49b] v88'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.216+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:32 14.2e 14:74000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859800 con 0x564b3c900000 2024-05-08T05:48:20.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.217+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.3 v2:172.21.15.45:6816/1259225446 3 ==== osd_op_reply(32 [pgnls-filter start_epoch 92 out=49b] v88'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3c900000 2024-05-08T05:48:20.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.217+0000 7fde0e505640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x564b3c858c00 0x564b3c8f9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.217+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:33 14.1e 14:78000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c900c00 con 0x564b3c858c00 2024-05-08T05:48:20.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.217+0000 7fde1350f640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x564b3c858c00 0x564b3c8f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.217+0000 7fde1350f640 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x564b3c858c00 0x564b3c8f9600 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.217+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.7 v2:172.21.15.63:6816/1163469437 1 ==== osd_op_reply(33 [pgnls-filter start_epoch 92 out=49b] v88'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3c858c00 2024-05-08T05:48:20.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.217+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:34 14.3e 14:7c000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91c000 con 0x564b3c900800 2024-05-08T05:48:20.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.218+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.5 v2:172.21.15.63:6808/2395798255 2 ==== osd_op_reply(34 [pgnls-filter start_epoch 92 out=49b] v88'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c900800 2024-05-08T05:48:20.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.218+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:35 14.1 14:80000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c901000 con 0x564b3c900000 2024-05-08T05:48:20.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.218+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.3 v2:172.21.15.45:6816/1259225446 4 ==== osd_op_reply(35 [pgnls-filter start_epoch 92 out=49b] v88'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3c900000 2024-05-08T05:48:20.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.218+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:36 14.21 14:84000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91c400 con 0x564b3c900800 2024-05-08T05:48:20.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.218+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.5 v2:172.21.15.63:6808/2395798255 3 ==== osd_op_reply(36 [pgnls-filter start_epoch 92 out=49b] v88'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c900800 2024-05-08T05:48:20.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.218+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:37 14.11 14:88000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c901800 con 0x564b3c901400 2024-05-08T05:48:20.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.218+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.219+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:38 14.31 14:8c000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91c800 con 0x564b3c859000 2024-05-08T05:48:20.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.219+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 6 ==== osd_op_reply(38 [pgnls-filter start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.219+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:39 14.9 14:90000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91dc00 con 0x564b3c900800 2024-05-08T05:48:20.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.219+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.5 v2:172.21.15.63:6808/2395798255 4 ==== osd_op_reply(39 [pgnls-filter start_epoch 92 out=49b] v88'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c900800 2024-05-08T05:48:20.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.219+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:40 14.29 14:94000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91cc00 con 0x564b3d34bc00 2024-05-08T05:48:20.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.219+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.6 v2:172.21.15.63:6800/2926759052 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3d34bc00 2024-05-08T05:48:20.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.219+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:41 14.19 14:98000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c900c00 con 0x564b3d34bc00 2024-05-08T05:48:20.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.220+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.6 v2:172.21.15.63:6800/2926759052 5 ==== osd_op_reply(41 [pgnls-filter start_epoch 92 out=49b] v88'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3d34bc00 2024-05-08T05:48:20.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.220+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:42 14.39 14:9c000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91d000 con 0x564b3c858c00 2024-05-08T05:48:20.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.220+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.7 v2:172.21.15.63:6816/1163469437 2 ==== osd_op_reply(42 [pgnls-filter start_epoch 92 out=49b] v88'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3c858c00 2024-05-08T05:48:20.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.220+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:43 14.5 14:a0000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c901000 con 0x564b3d34b400 2024-05-08T05:48:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.220+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 10 ==== osd_op_reply(43 [pgnls-filter start_epoch 92 out=49b] v88'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.220+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:44 14.25 14:a4000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91d400 con 0x564b3d34a800 2024-05-08T05:48:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.220+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.2 v2:172.21.15.45:6808/785834340 6 ==== osd_op_reply(44 [pgnls-filter start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34a800 2024-05-08T05:48:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.220+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:45 14.15 14:a8000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859400 con 0x564b3c858c00 2024-05-08T05:48:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.221+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.7 v2:172.21.15.63:6816/1163469437 3 ==== osd_op_reply(45 [pgnls-filter start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3c858c00 2024-05-08T05:48:20.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.221+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:46 14.35 14:ac000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91d800 con 0x564b3c901400 2024-05-08T05:48:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.221+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 8 ==== osd_op_reply(46 [pgnls-filter start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.221+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:47 14.d 14:b0000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859800 con 0x564b3d34bc00 2024-05-08T05:48:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.221+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.6 v2:172.21.15.63:6800/2926759052 6 ==== osd_op_reply(47 [pgnls-filter start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3d34bc00 2024-05-08T05:48:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.221+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:48 14.2d 14:b4000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91c000 con 0x564b3c901400 2024-05-08T05:48:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.221+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.221+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:49 14.1d 14:b8000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3d32d400 con 0x564b3d34b400 2024-05-08T05:48:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.222+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 11 ==== osd_op_reply(49 [pgnls-filter start_epoch 92 out=49b] v88'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.222+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:50 14.3d 14:bc000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91c400 con 0x564b3c900000 2024-05-08T05:48:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.222+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.3 v2:172.21.15.45:6816/1259225446 5 ==== osd_op_reply(50 [pgnls-filter start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3c900000 2024-05-08T05:48:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.222+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:51 14.3 14:c0000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c900c00 con 0x564b3d34b400 2024-05-08T05:48:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.222+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 12 ==== osd_op_reply(51 [pgnls-filter start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.222+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:52 14.23 14:c4000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91c800 con 0x564b3c858c00 2024-05-08T05:48:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.223+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.7 v2:172.21.15.63:6816/1163469437 4 ==== osd_op_reply(52 [pgnls-filter start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3c858c00 2024-05-08T05:48:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.223+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:53 14.13 14:c8000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c901c00 con 0x564b3d34a800 2024-05-08T05:48:20.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.223+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.2 v2:172.21.15.45:6808/785834340 7 ==== osd_op_reply(53 [pgnls-filter start_epoch 92 out=49b] v88'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34a800 2024-05-08T05:48:20.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.223+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:54 14.33 14:cc000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91dc00 con 0x564b3c859000 2024-05-08T05:48:20.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.223+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 7 ==== osd_op_reply(54 [pgnls-filter start_epoch 92 out=49b] v88'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.223+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:55 14.b 14:d0000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c901800 con 0x564b3d34b400 2024-05-08T05:48:20.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.223+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 13 ==== osd_op_reply(55 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.223+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:56 14.2b 14:d4000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c900400 con 0x564b3c901400 2024-05-08T05:48:20.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.224+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 10 ==== osd_op_reply(56 [pgnls-filter start_epoch 92 out=49b] v88'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.224+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:57 14.1b 14:d8000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859400 con 0x564b3c859000 2024-05-08T05:48:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.224+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 8 ==== osd_op_reply(57 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.224+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:58 14.3b 14:dc000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3d34ac00 con 0x564b3c901400 2024-05-08T05:48:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.224+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 11 ==== osd_op_reply(58 [pgnls-filter start_epoch 92 out=49b] v88'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.224+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:59 14.7 14:e0000000::::head [pgnls-filter start_epoch 92 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859800 con 0x564b3c900800 2024-05-08T05:48:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.225+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.5 v2:172.21.15.63:6808/2395798255 5 ==== osd_op_reply(59 [pgnls-filter start_epoch 92 out=81b] v92'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c900800 2024-05-08T05:48:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.225+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:60 14.0 14:00000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3d34a000 con 0x564b3d34a800 2024-05-08T05:48:20.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.225+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.2 v2:172.21.15.45:6808/785834340 8 ==== osd_op_reply(60 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34a800 2024-05-08T05:48:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.225+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:61 14.20 14:04000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859c00 con 0x564b3d34b400 2024-05-08T05:48:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.225+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 14 ==== osd_op_reply(61 [pgnls-filter start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.225+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:62 14.10 14:08000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91cc00 con 0x564b3c859000 2024-05-08T05:48:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.225+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 9 ==== osd_op_reply(62 [pgnls-filter start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.225+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:63 14.30 14:0c000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c858800 con 0x564b3d34b400 2024-05-08T05:48:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.226+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 15 ==== osd_op_reply(63 [pgnls-filter start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.226+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:64 14.8 14:10000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91d000 con 0x564b3d34b400 2024-05-08T05:48:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.226+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 16 ==== osd_op_reply(64 [pgnls-filter start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.226+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:65 14.28 14:14000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3d34a400 con 0x564b3c900000 2024-05-08T05:48:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.226+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.3 v2:172.21.15.45:6816/1259225446 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3c900000 2024-05-08T05:48:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.226+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:66 14.18 14:18000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91c400 con 0x564b3c859000 2024-05-08T05:48:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.226+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 10 ==== osd_op_reply(66 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.284 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.226+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:67 14.38 14:1c000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859c00 con 0x564b3c859000 2024-05-08T05:48:20.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.226+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 11 ==== osd_op_reply(67 [pgnls-filter start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.226+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:68 14.4 14:20000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91d400 con 0x564b3d34bc00 2024-05-08T05:48:20.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.227+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.6 v2:172.21.15.63:6800/2926759052 7 ==== osd_op_reply(68 [pgnls-filter start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3d34bc00 2024-05-08T05:48:20.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.227+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:69 14.24 14:24000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c858800 con 0x564b3c859000 2024-05-08T05:48:20.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.227+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 12 ==== osd_op_reply(69 [pgnls-filter start_epoch 92 out=49b] v88'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.227+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:70 14.14 14:28000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91d800 con 0x564b3c901400 2024-05-08T05:48:20.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.228+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 12 ==== osd_op_reply(70 [pgnls-filter start_epoch 92 out=49b] v88'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.228+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:71 14.34 14:2c000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3d34ac00 con 0x564b3d34a800 2024-05-08T05:48:20.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.228+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.2 v2:172.21.15.45:6808/785834340 9 ==== osd_op_reply(71 [pgnls-filter start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34a800 2024-05-08T05:48:20.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.228+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:72 14.c 14:30000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91c000 con 0x564b3d34b400 2024-05-08T05:48:20.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.228+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 17 ==== osd_op_reply(72 [pgnls-filter start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.228+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:73 14.2c 14:34000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3d34a000 con 0x564b3c859000 2024-05-08T05:48:20.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.228+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 13 ==== osd_op_reply(73 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.228+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:74 14.1c 14:38000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c901000 con 0x564b3d34b400 2024-05-08T05:48:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.228+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 18 ==== osd_op_reply(74 [pgnls-filter start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.228+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:75 14.3c 14:3c000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c900c00 con 0x564b3d34a800 2024-05-08T05:48:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.229+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.2 v2:172.21.15.45:6808/785834340 10 ==== osd_op_reply(75 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34a800 2024-05-08T05:48:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.229+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:76 14.2 14:40000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91dc00 con 0x564b3d34bc00 2024-05-08T05:48:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.229+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.6 v2:172.21.15.63:6800/2926759052 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3d34bc00 2024-05-08T05:48:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.229+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:77 14.22 14:44000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c901c00 con 0x564b3c901400 2024-05-08T05:48:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.230+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 13 ==== osd_op_reply(77 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.230+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:78 14.12 14:48000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91cc00 con 0x564b3d34a800 2024-05-08T05:48:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.230+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.2 v2:172.21.15.45:6808/785834340 11 ==== osd_op_reply(78 [pgnls-filter start_epoch 92 out=49b] v88'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34a800 2024-05-08T05:48:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.230+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:79 14.32 14:4c000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c901800 con 0x564b3c901400 2024-05-08T05:48:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.230+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.230+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:80 14.a 14:50000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91c400 con 0x564b3c900800 2024-05-08T05:48:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.231+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.5 v2:172.21.15.63:6808/2395798255 6 ==== osd_op_reply(80 [pgnls-filter start_epoch 92 out=49b] v88'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c900800 2024-05-08T05:48:20.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.231+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:81 14.2a 14:54000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c900400 con 0x564b3c901400 2024-05-08T05:48:20.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.231+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 15 ==== osd_op_reply(81 [pgnls-filter start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.231+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:82 14.1a 14:58000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91d000 con 0x564b3d34b400 2024-05-08T05:48:20.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.231+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 19 ==== osd_op_reply(82 [pgnls-filter start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.231+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:83 14.3a 14:5c000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3d32d400 con 0x564b3d34b400 2024-05-08T05:48:20.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.231+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 20 ==== osd_op_reply(83 [pgnls-filter start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.231+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:84 14.6 14:60000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859400 con 0x564b3d34b400 2024-05-08T05:48:20.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.231+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 21 ==== osd_op_reply(84 [pgnls-filter start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.232+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:85 14.26 14:64000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91d800 con 0x564b3c901400 2024-05-08T05:48:20.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.232+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 16 ==== osd_op_reply(85 [pgnls-filter start_epoch 92 out=49b] v88'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.232+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:86 14.16 14:68000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859800 con 0x564b3c900000 2024-05-08T05:48:20.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.232+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.3 v2:172.21.15.45:6816/1259225446 7 ==== osd_op_reply(86 [pgnls-filter start_epoch 92 out=49b] v88'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3c900000 2024-05-08T05:48:20.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.232+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:87 14.36 14:6c000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91c000 con 0x564b3d34a800 2024-05-08T05:48:20.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.232+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.2 v2:172.21.15.45:6808/785834340 12 ==== osd_op_reply(87 [pgnls-filter start_epoch 92 out=49b] v88'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34a800 2024-05-08T05:48:20.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.232+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:88 14.e 14:70000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859c00 con 0x564b3c901400 2024-05-08T05:48:20.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.233+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 17 ==== osd_op_reply(88 [pgnls-filter start_epoch 92 out=49b] v88'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.233+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:89 14.2e 14:74000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91cc00 con 0x564b3c900000 2024-05-08T05:48:20.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.233+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.3 v2:172.21.15.45:6816/1259225446 8 ==== osd_op_reply(89 [pgnls-filter start_epoch 92 out=49b] v88'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3c900000 2024-05-08T05:48:20.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.233+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:90 14.1e 14:78000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c858800 con 0x564b3c858c00 2024-05-08T05:48:20.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.233+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.7 v2:172.21.15.63:6816/1163469437 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 92 out=49b] v88'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3c858c00 2024-05-08T05:48:20.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.233+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:91 14.3e 14:7c000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91d000 con 0x564b3c900800 2024-05-08T05:48:20.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.233+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.5 v2:172.21.15.63:6808/2395798255 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 92 out=49b] v88'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c900800 2024-05-08T05:48:20.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.234+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:92 14.1 14:80000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3d32d400 con 0x564b3c900000 2024-05-08T05:48:20.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.234+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.3 v2:172.21.15.45:6816/1259225446 9 ==== osd_op_reply(92 [pgnls-filter start_epoch 92 out=49b] v88'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3c900000 2024-05-08T05:48:20.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.234+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:93 14.21 14:84000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c901000 con 0x564b3c900800 2024-05-08T05:48:20.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.235+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.5 v2:172.21.15.63:6808/2395798255 8 ==== osd_op_reply(93 [pgnls-filter start_epoch 92 out=49b] v88'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c900800 2024-05-08T05:48:20.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.235+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:94 14.11 14:88000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859800 con 0x564b3c901400 2024-05-08T05:48:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.235+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 18 ==== osd_op_reply(94 [pgnls-filter start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.235+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:95 14.31 14:8c000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c900c00 con 0x564b3c859000 2024-05-08T05:48:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.235+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 14 ==== osd_op_reply(95 [pgnls-filter start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.235+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:96 14.9 14:90000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c858800 con 0x564b3c900800 2024-05-08T05:48:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.235+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.5 v2:172.21.15.63:6808/2395798255 9 ==== osd_op_reply(96 [pgnls-filter start_epoch 92 out=49b] v88'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c900800 2024-05-08T05:48:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.235+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:97 14.29 14:94000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c901c00 con 0x564b3d34bc00 2024-05-08T05:48:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.236+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.6 v2:172.21.15.63:6800/2926759052 9 ==== osd_op_reply(97 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3d34bc00 2024-05-08T05:48:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.236+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:98 14.19 14:98000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91c800 con 0x564b3d34bc00 2024-05-08T05:48:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.236+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.6 v2:172.21.15.63:6800/2926759052 10 ==== osd_op_reply(98 [pgnls-filter start_epoch 92 out=49b] v88'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3d34bc00 2024-05-08T05:48:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.236+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:99 14.39 14:9c000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c901800 con 0x564b3c858c00 2024-05-08T05:48:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.236+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.7 v2:172.21.15.63:6816/1163469437 6 ==== osd_op_reply(99 [pgnls-filter start_epoch 92 out=49b] v88'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3c858c00 2024-05-08T05:48:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.236+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:100 14.5 14:a0000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91d400 con 0x564b3d34b400 2024-05-08T05:48:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.237+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 22 ==== osd_op_reply(100 [pgnls-filter start_epoch 92 out=49b] v88'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.237+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:101 14.25 14:a4000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c900400 con 0x564b3d34a800 2024-05-08T05:48:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.237+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.2 v2:172.21.15.45:6808/785834340 13 ==== osd_op_reply(101 [pgnls-filter start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34a800 2024-05-08T05:48:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.237+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:102 14.15 14:a8000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91dc00 con 0x564b3c858c00 2024-05-08T05:48:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.237+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.7 v2:172.21.15.63:6816/1163469437 7 ==== osd_op_reply(102 [pgnls-filter start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3c858c00 2024-05-08T05:48:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.237+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:103 14.35 14:ac000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3d34ac00 con 0x564b3c901400 2024-05-08T05:48:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.237+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 19 ==== osd_op_reply(103 [pgnls-filter start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.237+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:104 14.d 14:b0000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c91cc00 con 0x564b3d34bc00 2024-05-08T05:48:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.238+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.6 v2:172.21.15.63:6800/2926759052 11 ==== osd_op_reply(104 [pgnls-filter start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3d34bc00 2024-05-08T05:48:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.238+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:105 14.2d 14:b4000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c92e000 con 0x564b3c901400 2024-05-08T05:48:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.238+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 20 ==== osd_op_reply(105 [pgnls-filter start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.238+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:106 14.1d 14:b8000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3d34a400 con 0x564b3d34b400 2024-05-08T05:48:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.238+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 23 ==== osd_op_reply(106 [pgnls-filter start_epoch 92 out=49b] v88'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.238+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:107 14.3d 14:bc000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c92e400 con 0x564b3c900000 2024-05-08T05:48:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.238+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.3 v2:172.21.15.45:6816/1259225446 10 ==== osd_op_reply(107 [pgnls-filter start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3c900000 2024-05-08T05:48:20.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.238+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:108 14.3 14:c0000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c930000 con 0x564b3d34b400 2024-05-08T05:48:20.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.239+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 24 ==== osd_op_reply(108 [pgnls-filter start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.239+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:109 14.23 14:c4000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c92e800 con 0x564b3c858c00 2024-05-08T05:48:20.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.239+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.7 v2:172.21.15.63:6816/1163469437 8 ==== osd_op_reply(109 [pgnls-filter start_epoch 92 out=49b] v88'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3c858c00 2024-05-08T05:48:20.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.239+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:110 14.13 14:c8000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c930400 con 0x564b3d34a800 2024-05-08T05:48:20.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.239+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.2 v2:172.21.15.45:6808/785834340 14 ==== osd_op_reply(110 [pgnls-filter start_epoch 92 out=49b] v88'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34a800 2024-05-08T05:48:20.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.239+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:111 14.33 14:cc000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c92ec00 con 0x564b3c859000 2024-05-08T05:48:20.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.239+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 15 ==== osd_op_reply(111 [pgnls-filter start_epoch 92 out=49b] v88'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.239+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:112 14.b 14:d0000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c930800 con 0x564b3d34b400 2024-05-08T05:48:20.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.240+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 25 ==== osd_op_reply(112 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.240+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:113 14.2b 14:d4000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c92f000 con 0x564b3c901400 2024-05-08T05:48:20.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.240+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 21 ==== osd_op_reply(113 [pgnls-filter start_epoch 92 out=49b] v88'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.240+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:114 14.1b 14:d8000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c930c00 con 0x564b3c859000 2024-05-08T05:48:20.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.240+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 16 ==== osd_op_reply(114 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.240+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:115 14.3b 14:dc000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c92f400 con 0x564b3c901400 2024-05-08T05:48:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.240+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 22 ==== osd_op_reply(115 [pgnls-filter start_epoch 92 out=49b] v88'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.240+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:116 14.7 14:e0000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c931000 con 0x564b3c900800 2024-05-08T05:48:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.241+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.5 v2:172.21.15.63:6808/2395798255 10 ==== osd_op_reply(116 [pgnls-filter start_epoch 92 out=81b] v92'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c900800 2024-05-08T05:48:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.241+0000 7fde1501d740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-05-08T05:48:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.241+0000 7fde1501d740 10 datascan.operator(): handling object 10000000001.0 2024-05-08T05:48:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.241+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:117 14.7 14:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x564b3d420800 con 0x564b3c900800 2024-05-08T05:48:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.241+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.5 v2:172.21.15.63:6808/2395798255 11 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) ==== 416+0+157 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c900800 2024-05-08T05:48:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.241+0000 7fde1501d740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-05-08T05:48:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.241+0000 7fde1501d740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-05-08T05:48:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.241+0000 7fde1501d740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-05-08T05:48:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.241+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:118 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x564b3d34a000 con 0x564b3c859000 2024-05-08T05:48:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.242+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 17 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) ==== 164+0+59 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.242+0000 7fde1501d740 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-05-08T05:48:20.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.242+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x564b3d34ac00 con 0x564b3d34bc00 2024-05-08T05:48:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.242+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.6 v2:172.21.15.63:6800/2926759052 12 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=581b] v0'0 uv9 ondisk = 0) ==== 154+0+581 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3d34bc00 2024-05-08T05:48:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.242+0000 7fde1501d740 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-05-08T05:48:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.242+0000 7fde1501d740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-05-08T05:48:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.242+0000 7fde1501d740 20 datascan.get_frag_of: frag is * 2024-05-08T05:48:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.242+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:120 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x564b3d34a400 con 0x564b3c859000 2024-05-08T05:48:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.242+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 18 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.242+0000 7fde1501d740 10 datascan.find_or_create_dirfrag: read_version = 17 2024-05-08T05:48:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.242+0000 7fde1501d740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-05-08T05:48:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.242+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:121 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x564b3c858800 con 0x564b3c859000 2024-05-08T05:48:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.243+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 19 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.243+0000 7fde1501d740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-08T05:48:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.243+0000 7fde1501d740 20 datascan.read_dentry: file_head not found in result 2024-05-08T05:48:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.243+0000 7fde1501d740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-05-08T05:48:20.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.243+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:122 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859400 con 0x564b3c859000 2024-05-08T05:48:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.244+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 20 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v92'18 uv18 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.244+0000 7fde1501d740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-08T05:48:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.244+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:123 14.27 14:e4000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c859c00 con 0x564b3c900800 2024-05-08T05:48:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.244+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.5 v2:172.21.15.63:6808/2395798255 12 ==== osd_op_reply(123 [pgnls-filter start_epoch 92 out=49b] v88'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c900800 2024-05-08T05:48:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.244+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:124 14.17 14:e8000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c931400 con 0x564b3c859000 2024-05-08T05:48:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.245+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 <== osd.1 v2:172.21.15.45:6824/1337179184 21 ==== osd_op_reply(124 [pgnls-filter start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6780 con 0x564b3c859000 2024-05-08T05:48:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.245+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:125 14.37 14:ec000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c92f800 con 0x564b3c901400 2024-05-08T05:48:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.245+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.4 v2:172.21.15.63:6824/2212879151 23 ==== osd_op_reply(125 [pgnls-filter start_epoch 92 out=49b] v88'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3c901400 2024-05-08T05:48:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.245+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:126 14.f 14:f0000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c931800 con 0x564b3c858c00 2024-05-08T05:48:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.245+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.7 v2:172.21.15.63:6816/1163469437 9 ==== osd_op_reply(126 [pgnls-filter start_epoch 92 out=49b] v88'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3c858c00 2024-05-08T05:48:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.245+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:127 14.2f 14:f4000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c92fc00 con 0x564b3d34b400 2024-05-08T05:48:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 26 ==== osd_op_reply(127 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde0dd04640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:128 14.1f 14:f8000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c931c00 con 0x564b3d34b400 2024-05-08T05:48:20.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde12d0e640 1 -- 172.21.15.45:0/3739278721 <== osd.0 v2:172.21.15.45:6800/3734454393 27 ==== osd_op_reply(128 [pgnls-filter start_epoch 92 out=49b] v88'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6000 con 0x564b3d34b400 2024-05-08T05:48:20.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde0e505640 1 -- 172.21.15.45:0/3739278721 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:129 14.3f 14:fc000000::::head [pgnls-filter start_epoch 92 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x564b3c931000 con 0x564b3c900000 2024-05-08T05:48:20.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1350f640 1 -- 172.21.15.45:0/3739278721 <== osd.3 v2:172.21.15.45:6816/1259225446 11 ==== osd_op_reply(129 [pgnls-filter start_epoch 92 out=49b] v88'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x564b3d2d6500 con 0x564b3c900000 2024-05-08T05:48:20.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x564b3d34b400 msgr2=0x564b3d322680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x564b3d34b400 0x564b3d322680 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x564b3c859000 msgr2=0x564b3c8f8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x564b3c859000 0x564b3c8f8000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x564b3d34a800 msgr2=0x564b3d323180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x564b3d34a800 0x564b3d323180 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x564b3c900000 msgr2=0x564b3c8f8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x564b3c900000 0x564b3c8f8580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x564b3c901400 msgr2=0x564b3c8f8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x564b3c901400 0x564b3c8f8b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x564b3c900800 msgr2=0x564b3c8f9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x564b3c900800 0x564b3c8f9080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x564b3d34bc00 msgr2=0x564b3d322c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x564b3d34bc00 0x564b3d322c00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x564b3c858c00 msgr2=0x564b3c8f9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x564b3c858c00 0x564b3c8f9600 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 reap_dead start 2024-05-08T05:48:20.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x564b3d34b000 msgr2=0x564b3d322100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x564b3d34b000 0x564b3d322100 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x564b3d36bb60 tx=0x564b3c864fc0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 reap_dead start 2024-05-08T05:48:20.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b3d421c00 msgr2=0x564b3d321b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 --2- 172.21.15.45:0/3739278721 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b3d421c00 0x564b3d321b80 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x564b3d43baa0 tx=0x564b3d2ebf20 comp rx=0 tx=0).stop 2024-05-08T05:48:20.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1250d640 1 -- 172.21.15.45:0/3739278721 reap_dead start 2024-05-08T05:48:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 shutdown_connections 2024-05-08T05:48:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 >> 172.21.15.45:0/3739278721 conn(0x564b3d421000 msgr2=0x564b3c637000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.246+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 shutdown_connections 2024-05-08T05:48:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.247+0000 7fde1501d740 1 -- 172.21.15.45:0/3739278721 wait complete. 2024-05-08T05:48:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.247+0000 7fde1501d740 1 -- 172.21.15.45:0/4172759371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b3c544c00 msgr2=0x564b3d320000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.247+0000 7fde1501d740 1 --2- 172.21.15.45:0/4172759371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b3c544c00 0x564b3d320000 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x564b3d36bbf0 tx=0x564b3c614a80 comp rx=0 tx=0).stop 2024-05-08T05:48:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.247+0000 7fde1501d740 1 -- 172.21.15.45:0/4172759371 shutdown_connections 2024-05-08T05:48:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.247+0000 7fde1501d740 1 --2- 172.21.15.45:0/4172759371 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564b3c545c00 0x564b3d320b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.247+0000 7fde1501d740 1 --2- 172.21.15.45:0/4172759371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564b3c545800 0x564b3d320580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.247+0000 7fde1501d740 1 --2- 172.21.15.45:0/4172759371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564b3c544c00 0x564b3d320000 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.247+0000 7fde1501d740 1 -- 172.21.15.45:0/4172759371 >> 172.21.15.45:0/4172759371 conn(0x564b3c545000 msgr2=0x564b3c636800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.247+0000 7fde1501d740 1 -- 172.21.15.45:0/4172759371 shutdown_connections 2024-05-08T05:48:20.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.247+0000 7fde1501d740 1 -- 172.21.15.45:0/4172759371 wait complete. 2024-05-08T05:48:20.298 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.182215, result: 2024-05-08T05:48:20.298 DEBUG:teuthology.orchestra.run.smithi045:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-05-08T05:48:20.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.343+0000 7f558f41a740 1 -- start start 2024-05-08T05:48:20.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.343+0000 7f558f41a740 1 Processor -- start 2024-05-08T05:48:20.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.343+0000 7f558f41a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558d04146c00 0x558d04f24000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.343+0000 7f558f41a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d04147800 0x558d04f24580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.343+0000 7f558f41a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d04147c00 0x558d04f24b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.343+0000 7f558f41a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558d04f28180 con 0x558d04147800 2024-05-08T05:48:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.343+0000 7f558f41a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558d04f28300 con 0x558d04147c00 2024-05-08T05:48:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.343+0000 7f558f41a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x558d04f28480 con 0x558d04146c00 2024-05-08T05:48:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.343+0000 7f558c838640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d04147800 0x558d04f24580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.343+0000 7f558c838640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d04147800 0x558d04f24580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35328/0 (socket says 172.21.15.45:35328) 2024-05-08T05:48:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.343+0000 7f558c838640 1 -- 172.21.15.45:0/3738411627 learned_addr learned my addr 172.21.15.45:0/3738411627 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.344+0000 7f558d039640 1 --2- 172.21.15.45:0/3738411627 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558d04146c00 0x558d04f24000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.344+0000 7f558d83a640 1 --2- 172.21.15.45:0/3738411627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d04147c00 0x558d04f24b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.344+0000 7f558c838640 1 -- 172.21.15.45:0/3738411627 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558d04146c00 msgr2=0x558d04f24000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.344+0000 7f558c838640 1 --2- 172.21.15.45:0/3738411627 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558d04146c00 0x558d04f24000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.344+0000 7f558c838640 1 -- 172.21.15.45:0/3738411627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d04147c00 msgr2=0x558d04f24b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.344+0000 7f558c838640 1 --2- 172.21.15.45:0/3738411627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d04147c00 0x558d04f24b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.344+0000 7f558c838640 1 -- 172.21.15.45:0/3738411627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d04173680 con 0x558d04147800 2024-05-08T05:48:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.344+0000 7f558d039640 1 --2- 172.21.15.45:0/3738411627 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558d04146c00 0x558d04f24000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:48:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.344+0000 7f558d83a640 1 --2- 172.21.15.45:0/3738411627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d04147c00 0x558d04f24b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:48:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.344+0000 7f558c838640 1 --2- 172.21.15.45:0/3738411627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d04147800 0x558d04f24580 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x558d04f7bc80 tx=0x558d04eec600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d457f9206883f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.344+0000 7f558a834640 1 -- 172.21.15.45:0/3738411627 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558d0416c680 con 0x558d04147800 2024-05-08T05:48:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.344+0000 7f558a834640 1 -- 172.21.15.45:0/3738411627 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x558d0418e1c0 con 0x558d04147800 2024-05-08T05:48:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.344+0000 7f558a834640 1 -- 172.21.15.45:0/3738411627 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558d0416c9c0 con 0x558d04147800 2024-05-08T05:48:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.344+0000 7f558f41a740 1 -- 172.21.15.45:0/3738411627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d04173a40 con 0x558d04147800 2024-05-08T05:48:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.344+0000 7f558f41a740 1 -- 172.21.15.45:0/3738411627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=0}) -- 0x558d041732c0 con 0x558d04147800 2024-05-08T05:48:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.344+0000 7f558f41a740 4 waiting for MDS map... 2024-05-08T05:48:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.344+0000 7f558a834640 1 -- 172.21.15.45:0/3738411627 <== mon.0 v2:172.21.15.45:3300/0 4 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x558d04f4a800 con 0x558d04147800 2024-05-08T05:48:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.344+0000 7f558a834640 1 -- 172.21.15.45:0/3738411627 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 68) ==== 2885+0+0 (secure 0 0 0) 0x558d0419f180 con 0x558d04147800 2024-05-08T05:48:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.344+0000 7f558f41a740 4 Got MDS map 68 2024-05-08T05:48:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.345+0000 7f558f41a740 4 datascan.main: Using metadata pool output 2024-05-08T05:48:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.345+0000 7f558f41a740 4 datascan.main: connecting to RADOS... 2024-05-08T05:48:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.345+0000 7f558f41a740 1 Processor -- start 2024-05-08T05:48:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.345+0000 7f558f41a740 1 -- start start 2024-05-08T05:48:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.346+0000 7f558f41a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558d05084000 0x558d04f25080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.346+0000 7f558f41a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d05084400 0x558d04f25600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.346+0000 7f558f41a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d05084800 0x558d04f25b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.346+0000 7f558f41a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558d04f28600 con 0x558d05084400 2024-05-08T05:48:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.346+0000 7f558f41a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558d04f28780 con 0x558d05084800 2024-05-08T05:48:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.346+0000 7f558f41a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x558d04f28900 con 0x558d05084000 2024-05-08T05:48:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.346+0000 7f558d039640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d05084400 0x558d04f25600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.346+0000 7f558d039640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d05084400 0x558d04f25600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35334/0 (socket says 172.21.15.45:35334) 2024-05-08T05:48:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.346+0000 7f558d039640 1 -- 172.21.15.45:0/2074218375 learned_addr learned my addr 172.21.15.45:0/2074218375 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.346+0000 7f558d83a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558d05084000 0x558d04f25080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.346+0000 7f558c838640 1 --2- 172.21.15.45:0/2074218375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d05084800 0x558d04f25b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.346+0000 7f558d039640 1 -- 172.21.15.45:0/2074218375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558d05084000 msgr2=0x558d04f25080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.346+0000 7f558d039640 1 --2- 172.21.15.45:0/2074218375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558d05084000 0x558d04f25080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.346+0000 7f558d039640 1 -- 172.21.15.45:0/2074218375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d05084800 msgr2=0x558d04f25b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.346+0000 7f558d039640 1 --2- 172.21.15.45:0/2074218375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d05084800 0x558d04f25b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.346+0000 7f558d039640 1 -- 172.21.15.45:0/2074218375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d041734a0 con 0x558d05084400 2024-05-08T05:48:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.346+0000 7f558d039640 1 --2- 172.21.15.45:0/2074218375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d05084400 0x558d04f25600 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x558d04f63140 tx=0x558d050398c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6110fef7c16719b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.346+0000 7f5588830640 1 -- 172.21.15.45:0/2074218375 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558d0416c680 con 0x558d05084400 2024-05-08T05:48:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.346+0000 7f5588830640 1 -- 172.21.15.45:0/2074218375 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x558d0418ec40 con 0x558d05084400 2024-05-08T05:48:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.346+0000 7f5588830640 1 -- 172.21.15.45:0/2074218375 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558d0416c9c0 con 0x558d05084400 2024-05-08T05:48:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.347+0000 7f558f41a740 1 -- 172.21.15.45:0/2074218375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d05084400 msgr2=0x558d04f25600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.347+0000 7f558f41a740 1 --2- 172.21.15.45:0/2074218375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d05084400 0x558d04f25600 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x558d04f63140 tx=0x558d050398c0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.347+0000 7f558f41a740 1 -- 172.21.15.45:0/2074218375 shutdown_connections 2024-05-08T05:48:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.347+0000 7f558f41a740 1 --2- 172.21.15.45:0/2074218375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d05084800 0x558d04f25b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.347+0000 7f558f41a740 1 --2- 172.21.15.45:0/2074218375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d05084400 0x558d04f25600 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.347+0000 7f558f41a740 1 --2- 172.21.15.45:0/2074218375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558d05084000 0x558d04f25080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.347+0000 7f558f41a740 1 -- 172.21.15.45:0/2074218375 >> 172.21.15.45:0/2074218375 conn(0x558d04f2dc00 msgr2=0x558d04ec5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.347+0000 7f558f41a740 1 -- 172.21.15.45:0/2074218375 shutdown_connections 2024-05-08T05:48:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.347+0000 7f558f41a740 1 -- 172.21.15.45:0/2074218375 wait complete. 2024-05-08T05:48:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.347+0000 7f558f41a740 1 Processor -- start 2024-05-08T05:48:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.347+0000 7f558f41a740 1 -- start start 2024-05-08T05:48:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.348+0000 7f558f41a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558d05084000 0x558d04f25080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.348+0000 7f558f41a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d05084400 0x558d04f25600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.348+0000 7f558f41a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d05084800 0x558d04f25b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.348+0000 7f558f41a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558d04f29c80 con 0x558d05084800 2024-05-08T05:48:20.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.348+0000 7f558f41a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558d04f29b00 con 0x558d05084400 2024-05-08T05:48:20.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.348+0000 7f558f41a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x558d04f29980 con 0x558d05084000 2024-05-08T05:48:20.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.348+0000 7f558c838640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d05084800 0x558d04f25b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.348+0000 7f558c838640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d05084800 0x558d04f25b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35336/0 (socket says 172.21.15.45:35336) 2024-05-08T05:48:20.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.348+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 learned_addr learned my addr 172.21.15.45:0/2648007777 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:20.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.348+0000 7f558d039640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d05084400 0x558d04f25600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.348+0000 7f558d83a640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558d05084000 0x558d04f25080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.348+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558d05084000 msgr2=0x558d04f25080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.348+0000 7f558c838640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558d05084000 0x558d04f25080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.348+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d05084400 msgr2=0x558d04f25600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.348+0000 7f558c838640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d05084400 0x558d04f25600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.349+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d041730e0 con 0x558d05084800 2024-05-08T05:48:20.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.349+0000 7f558c838640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d05084800 0x558d04f25b80 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x558d04f62840 tx=0x558d04eec720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.349+0000 7f558702d640 1 -- 172.21.15.45:0/2648007777 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558d0416c680 con 0x558d05084800 2024-05-08T05:48:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.349+0000 7f558702d640 1 -- 172.21.15.45:0/2648007777 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x558d0418ee00 con 0x558d05084800 2024-05-08T05:48:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.349+0000 7f558702d640 1 -- 172.21.15.45:0/2648007777 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558d0416c9c0 con 0x558d05084800 2024-05-08T05:48:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.349+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558d041723c0 con 0x558d05084800 2024-05-08T05:48:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.349+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d04172b40 con 0x558d05084800 2024-05-08T05:48:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.349+0000 7f558f41a740 4 datascan.init: resolving metadata pool 13 2024-05-08T05:48:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.350+0000 7f558702d640 1 -- 172.21.15.45:0/2648007777 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x558d05094c00 con 0x558d05084800 2024-05-08T05:48:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.350+0000 7f558702d640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x558d050a7000 0x558d04f26100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.351+0000 7f558702d640 1 -- 172.21.15.45:0/2648007777 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x558d04ee5e00 con 0x558d05084800 2024-05-08T05:48:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.351+0000 7f558f41a740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-08T05:48:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.351+0000 7f558f41a740 4 datascan.main: resolving metadata pool 13 2024-05-08T05:48:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.351+0000 7f558d83a640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x558d050a7000 0x558d04f26100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.351+0000 7f558f41a740 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x558d05084c00 0x558d04f26680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.351+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d05085000 con 0x558d05084c00 2024-05-08T05:48:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.351+0000 7f558d039640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x558d05084c00 0x558d04f26680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.351+0000 7f558d83a640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x558d050a7000 0x558d04f26100 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x558d04eef080 tx=0x558d04471020 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.351+0000 7f558d039640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x558d05084c00 0x558d04f26680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.351+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 1 ==== osd_op_reply(1 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.351+0000 7f558d039640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x558d05085c00 0x558d044f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.351+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d05085400 con 0x558d05085c00 2024-05-08T05:48:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.352+0000 7f558d83a640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x558d05085c00 0x558d044f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.352+0000 7f558d83a640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x558d05085c00 0x558d044f6000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.352+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 1 ==== osd_op_reply(2 [pgnls start_epoch 92 out=177b] v92'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.352+0000 7f558d83a640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x558d05085400 0x558d044f6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.352+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d050a7c00 con 0x558d05085400 2024-05-08T05:48:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.352+0000 7f558d039640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x558d05085400 0x558d044f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.353+0000 7f558d039640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x558d05085400 0x558d044f6580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.353+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.2 v2:172.21.15.45:6808/785834340 1 ==== osd_op_reply(3 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8000 con 0x558d05085400 2024-05-08T05:48:20.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.353+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d050a7c00 con 0x558d05084c00 2024-05-08T05:48:20.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.353+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 2 ==== osd_op_reply(4 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.353+0000 7f558d039640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x558d050a7c00 0x558d044f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.353+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d05085800 con 0x558d050a7c00 2024-05-08T05:48:20.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.353+0000 7f558c838640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x558d050a7c00 0x558d044f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.354+0000 7f558c838640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x558d050a7c00 0x558d044f6b00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.354+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.6 v2:172.21.15.63:6800/2926759052 1 ==== osd_op_reply(5 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8780 con 0x558d050a7c00 2024-05-08T05:48:20.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.354+0000 7f558c838640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x558d05085800 0x558d044f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.354+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d050a6c00 con 0x558d05085800 2024-05-08T05:48:20.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.354+0000 7f558d83a640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x558d05085800 0x558d044f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.354+0000 7f558d83a640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x558d05085800 0x558d044f7600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.355+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.0 v2:172.21.15.45:6800/3734454393 1 ==== osd_op_reply(6 [pgnls start_epoch 92 out=149b] v91'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085800 2024-05-08T05:48:20.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.355+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d050a6c00 con 0x558d05085c00 2024-05-08T05:48:20.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.355+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 2 ==== osd_op_reply(7 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.355+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d050a6c00 con 0x558d050a7c00 2024-05-08T05:48:20.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.355+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.6 v2:172.21.15.63:6800/2926759052 2 ==== osd_op_reply(8 [pgnls start_epoch 92 out=73b] v91'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x558d04ed8780 con 0x558d050a7c00 2024-05-08T05:48:20.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.355+0000 7f558c838640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x558d050a6c00 0x558d044f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.355+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514000 con 0x558d050a6c00 2024-05-08T05:48:20.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.355+0000 7f558d039640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x558d050a6c00 0x558d044f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.356+0000 7f558d039640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x558d050a6c00 0x558d044f7b80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.356+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.5 v2:172.21.15.63:6808/2395798255 1 ==== osd_op_reply(9 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8000 con 0x558d050a6c00 2024-05-08T05:48:20.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.356+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514000 con 0x558d05085800 2024-05-08T05:48:20.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.356+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.0 v2:172.21.15.45:6800/3734454393 2 ==== osd_op_reply(10 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085800 2024-05-08T05:48:20.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.356+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514000 con 0x558d05085400 2024-05-08T05:48:20.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.357+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.2 v2:172.21.15.45:6808/785834340 2 ==== osd_op_reply(11 [pgnls start_epoch 92 out=81b] v91'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x558d04ed8000 con 0x558d05085400 2024-05-08T05:48:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.357+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514000 con 0x558d05085c00 2024-05-08T05:48:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.357+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 3 ==== osd_op_reply(12 [pgnls start_epoch 92 out=49b] v89'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.357+0000 7f558d83a640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x558d04514000 0x558d044f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.357+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515400 con 0x558d04514000 2024-05-08T05:48:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.357+0000 7f558c838640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x558d04514000 0x558d044f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.357+0000 7f558c838640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x558d04514000 0x558d044f8680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.358+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.4 v2:172.21.15.63:6824/2212879151 1 ==== osd_op_reply(13 [pgnls start_epoch 92 out=77b] v91'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x558d04ed8780 con 0x558d04514000 2024-05-08T05:48:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.358+0000 7f558c838640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x558d04515400 0x558d044f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.358+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514400 con 0x558d04515400 2024-05-08T05:48:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.358+0000 7f558d83a640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x558d04515400 0x558d044f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.358+0000 7f558d83a640 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x558d04515400 0x558d044f8100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.358+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.3 v2:172.21.15.45:6816/1259225446 1 ==== osd_op_reply(14 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8500 con 0x558d04515400 2024-05-08T05:48:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.358+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514400 con 0x558d05084c00 2024-05-08T05:48:20.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.359+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 3 ==== osd_op_reply(15 [pgnls start_epoch 92 out=73b] v91'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.359+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514400 con 0x558d04514000 2024-05-08T05:48:20.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.359+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.4 v2:172.21.15.63:6824/2212879151 2 ==== osd_op_reply(16 [pgnls start_epoch 92 out=49b] v89'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8780 con 0x558d04514000 2024-05-08T05:48:20.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.359+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514400 con 0x558d050a7c00 2024-05-08T05:48:20.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.359+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.6 v2:172.21.15.63:6800/2926759052 3 ==== osd_op_reply(17 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8780 con 0x558d050a7c00 2024-05-08T05:48:20.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.359+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514400 con 0x558d05085c00 2024-05-08T05:48:20.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.359+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 4 ==== osd_op_reply(18 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.359+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514400 con 0x558d050a6c00 2024-05-08T05:48:20.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.360+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.5 v2:172.21.15.63:6808/2395798255 2 ==== osd_op_reply(19 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8000 con 0x558d050a6c00 2024-05-08T05:48:20.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.360+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514400 con 0x558d04515400 2024-05-08T05:48:20.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.360+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.3 v2:172.21.15.45:6816/1259225446 2 ==== osd_op_reply(20 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8500 con 0x558d04515400 2024-05-08T05:48:20.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.360+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514400 con 0x558d04514000 2024-05-08T05:48:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.360+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.4 v2:172.21.15.63:6824/2212879151 3 ==== osd_op_reply(21 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8780 con 0x558d04514000 2024-05-08T05:48:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.360+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514400 con 0x558d04514000 2024-05-08T05:48:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.360+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.4 v2:172.21.15.63:6824/2212879151 4 ==== osd_op_reply(22 [pgnls start_epoch 92 out=97b] v91'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x558d04ed8780 con 0x558d04514000 2024-05-08T05:48:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.360+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514400 con 0x558d04514000 2024-05-08T05:48:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.361+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.4 v2:172.21.15.63:6824/2212879151 5 ==== osd_op_reply(23 [pgnls start_epoch 92 out=74b] v87'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x558d04ed8780 con 0x558d04514000 2024-05-08T05:48:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.361+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514400 con 0x558d05084c00 2024-05-08T05:48:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.361+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 4 ==== osd_op_reply(24 [pgnls start_epoch 92 out=73b] v91'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.361+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514400 con 0x558d05085800 2024-05-08T05:48:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.361+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.0 v2:172.21.15.45:6800/3734454393 3 ==== osd_op_reply(25 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085800 2024-05-08T05:48:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.361+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514400 con 0x558d05084c00 2024-05-08T05:48:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.361+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 5 ==== osd_op_reply(26 [pgnls start_epoch 92 out=98b] v91'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.362+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514400 con 0x558d050a6c00 2024-05-08T05:48:20.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.362+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.5 v2:172.21.15.63:6808/2395798255 3 ==== osd_op_reply(27 [pgnls start_epoch 92 out=76b] v91'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x558d04ed8000 con 0x558d050a6c00 2024-05-08T05:48:20.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.362+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514400 con 0x558d04514000 2024-05-08T05:48:20.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.362+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.4 v2:172.21.15.63:6824/2212879151 6 ==== osd_op_reply(28 [pgnls start_epoch 92 out=73b] v91'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x558d04ed8780 con 0x558d04514000 2024-05-08T05:48:20.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.362+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514400 con 0x558d05084c00 2024-05-08T05:48:20.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.363+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 6 ==== osd_op_reply(29 [pgnls start_epoch 92 out=73b] v91'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.363+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514400 con 0x558d050a6c00 2024-05-08T05:48:20.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.363+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.5 v2:172.21.15.63:6808/2395798255 4 ==== osd_op_reply(30 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8000 con 0x558d050a6c00 2024-05-08T05:48:20.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.363+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514400 con 0x558d050a7c00 2024-05-08T05:48:20.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.363+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.6 v2:172.21.15.63:6800/2926759052 4 ==== osd_op_reply(31 [pgnls start_epoch 92 out=73b] v87'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x558d04ed8780 con 0x558d050a7c00 2024-05-08T05:48:20.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.363+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04514400 con 0x558d050a7c00 2024-05-08T05:48:20.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.363+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.6 v2:172.21.15.63:6800/2926759052 5 ==== osd_op_reply(32 [pgnls start_epoch 92 out=101b] v91'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x558d04ed8780 con 0x558d050a7c00 2024-05-08T05:48:20.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.363+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-05-08T05:48:20.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.364+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451c000 con 0x558d05085c00 2024-05-08T05:48:20.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.364+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.364+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-05-08T05:48:20.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.364+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:34 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451c400 con 0x558d05085c00 2024-05-08T05:48:20.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.364+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.364+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-05-08T05:48:20.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.364+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:35 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451c800 con 0x558d05085c00 2024-05-08T05:48:20.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.365+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.365+0000 7f558f41a740 20 datascan.scan_links: injected ino detected 2024-05-08T05:48:20.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.365+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-05-08T05:48:20.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.365+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:36 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451cc00 con 0x558d05085c00 2024-05-08T05:48:20.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.365+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.365+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-05-08T05:48:20.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.365+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:37 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451d000 con 0x558d05085c00 2024-05-08T05:48:20.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.365+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.365+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-05-08T05:48:20.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.365+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-08T05:48:20.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.365+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-05-08T05:48:20.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.365+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-08T05:48:20.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.365+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-05-08T05:48:20.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.365+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-08T05:48:20.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.365+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-05-08T05:48:20.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.365+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:38 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451d400 con 0x558d05085800 2024-05-08T05:48:20.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.365+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.0 v2:172.21.15.45:6800/3734454393 4 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085800 2024-05-08T05:48:20.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.365+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-05-08T05:48:20.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.365+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:39 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451c000 con 0x558d050a7c00 2024-05-08T05:48:20.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.366+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.6 v2:172.21.15.63:6800/2926759052 6 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x558d04ed8780 con 0x558d050a7c00 2024-05-08T05:48:20.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.366+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-05-08T05:48:20.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.366+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:40 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451c400 con 0x558d05085400 2024-05-08T05:48:20.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.366+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.2 v2:172.21.15.45:6808/785834340 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x558d04ed8000 con 0x558d05085400 2024-05-08T05:48:20.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.366+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-05-08T05:48:20.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.366+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-08T05:48:20.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.366+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-05-08T05:48:20.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.366+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:41 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451c800 con 0x558d05084c00 2024-05-08T05:48:20.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.366+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 7 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.366+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-05-08T05:48:20.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.367+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:42 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451d800 con 0x558d04514000 2024-05-08T05:48:20.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.367+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.4 v2:172.21.15.63:6824/2212879151 7 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x558d04ed8780 con 0x558d04514000 2024-05-08T05:48:20.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.367+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-05-08T05:48:20.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.367+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-08T05:48:20.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.367+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-05-08T05:48:20.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.367+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-08T05:48:20.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.367+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-05-08T05:48:20.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.367+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:43 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451dc00 con 0x558d05084c00 2024-05-08T05:48:20.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.367+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 8 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.367+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-05-08T05:48:20.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.367+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-08T05:48:20.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.367+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-05-08T05:48:20.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.367+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:44 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d04515c00 con 0x558d05084c00 2024-05-08T05:48:20.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.367+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 9 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.367+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-05-08T05:48:20.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.367+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-08T05:48:20.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.367+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 200.00000002 2024-05-08T05:48:20.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.367+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-05-08T05:48:20.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.367+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-05-08T05:48:20.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.367+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:45 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d05084c00 2024-05-08T05:48:20.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.368+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 10 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.368+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-05-08T05:48:20.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.368+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-08T05:48:20.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.368+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-05-08T05:48:20.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.368+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-08T05:48:20.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.368+0000 7f558f41a740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-05-08T05:48:20.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.368+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:46 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d04515c00 con 0x558d050a7c00 2024-05-08T05:48:20.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.368+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.6 v2:172.21.15.63:6800/2926759052 7 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) ==== 154+0+1163 (crc 0 0 0) 0x558d04ed8780 con 0x558d050a7c00 2024-05-08T05:48:20.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.368+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:47 13.0 13:00000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d05084c00 2024-05-08T05:48:20.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.369+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 11 ==== osd_op_reply(47 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.369+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:48 13.10 13:08000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d05085c00 2024-05-08T05:48:20.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.369+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 10 ==== osd_op_reply(48 [pgnls start_epoch 92 out=177b] v92'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.369+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:49 13.8 13:10000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d05085400 2024-05-08T05:48:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.369+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.2 v2:172.21.15.45:6808/785834340 4 ==== osd_op_reply(49 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8000 con 0x558d05085400 2024-05-08T05:48:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.369+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:50 13.18 13:18000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d05084c00 2024-05-08T05:48:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.369+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 12 ==== osd_op_reply(50 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.369+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:51 13.4 13:20000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d050a7c00 2024-05-08T05:48:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.370+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.6 v2:172.21.15.63:6800/2926759052 8 ==== osd_op_reply(51 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8780 con 0x558d050a7c00 2024-05-08T05:48:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.370+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:52 13.14 13:28000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d05085800 2024-05-08T05:48:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.370+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.0 v2:172.21.15.45:6800/3734454393 5 ==== osd_op_reply(52 [pgnls start_epoch 92 out=149b] v91'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085800 2024-05-08T05:48:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.370+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:53 13.c 13:30000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d05085c00 2024-05-08T05:48:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.370+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 11 ==== osd_op_reply(53 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.370+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:54 13.1c 13:38000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d050a7c00 2024-05-08T05:48:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.370+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.6 v2:172.21.15.63:6800/2926759052 9 ==== osd_op_reply(54 [pgnls start_epoch 92 out=73b] v91'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x558d04ed8780 con 0x558d050a7c00 2024-05-08T05:48:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.370+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:55 13.2 13:40000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d050a6c00 2024-05-08T05:48:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.371+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.5 v2:172.21.15.63:6808/2395798255 5 ==== osd_op_reply(55 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8000 con 0x558d050a6c00 2024-05-08T05:48:20.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.371+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:56 13.12 13:48000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d05085800 2024-05-08T05:48:20.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.371+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.0 v2:172.21.15.45:6800/3734454393 6 ==== osd_op_reply(56 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085800 2024-05-08T05:48:20.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.371+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:57 13.a 13:50000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d05085400 2024-05-08T05:48:20.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.371+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.2 v2:172.21.15.45:6808/785834340 5 ==== osd_op_reply(57 [pgnls start_epoch 92 out=81b] v91'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x558d04ed8000 con 0x558d05085400 2024-05-08T05:48:20.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.371+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:58 13.1a 13:58000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d05085c00 2024-05-08T05:48:20.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.371+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 12 ==== osd_op_reply(58 [pgnls start_epoch 92 out=49b] v89'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.371+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:59 13.6 13:60000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d04514000 2024-05-08T05:48:20.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.372+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.4 v2:172.21.15.63:6824/2212879151 8 ==== osd_op_reply(59 [pgnls start_epoch 92 out=77b] v91'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x558d04ed8780 con 0x558d04514000 2024-05-08T05:48:20.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.372+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:60 13.16 13:68000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d04515400 2024-05-08T05:48:20.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.372+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.3 v2:172.21.15.45:6816/1259225446 3 ==== osd_op_reply(60 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8500 con 0x558d04515400 2024-05-08T05:48:20.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.372+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:61 13.e 13:70000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d05084c00 2024-05-08T05:48:20.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.372+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 13 ==== osd_op_reply(61 [pgnls start_epoch 92 out=73b] v91'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.372+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:62 13.1e 13:78000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d04514000 2024-05-08T05:48:20.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.373+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.4 v2:172.21.15.63:6824/2212879151 9 ==== osd_op_reply(62 [pgnls start_epoch 92 out=49b] v89'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8780 con 0x558d04514000 2024-05-08T05:48:20.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.373+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:63 13.1 13:80000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d050a7c00 2024-05-08T05:48:20.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.373+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.6 v2:172.21.15.63:6800/2926759052 10 ==== osd_op_reply(63 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8780 con 0x558d050a7c00 2024-05-08T05:48:20.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.373+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:64 13.11 13:88000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d05085c00 2024-05-08T05:48:20.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.373+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 13 ==== osd_op_reply(64 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.373+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:65 13.9 13:90000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d050a6c00 2024-05-08T05:48:20.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.373+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.5 v2:172.21.15.63:6808/2395798255 6 ==== osd_op_reply(65 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8000 con 0x558d050a6c00 2024-05-08T05:48:20.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.373+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:66 13.19 13:98000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d04515400 2024-05-08T05:48:20.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.374+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.3 v2:172.21.15.45:6816/1259225446 4 ==== osd_op_reply(66 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8500 con 0x558d04515400 2024-05-08T05:48:20.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.374+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:67 13.5 13:a0000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d04514000 2024-05-08T05:48:20.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.374+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.4 v2:172.21.15.63:6824/2212879151 10 ==== osd_op_reply(67 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8780 con 0x558d04514000 2024-05-08T05:48:20.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.374+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:68 13.15 13:a8000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d04514000 2024-05-08T05:48:20.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.374+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.4 v2:172.21.15.63:6824/2212879151 11 ==== osd_op_reply(68 [pgnls start_epoch 92 out=97b] v91'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x558d04ed8780 con 0x558d04514000 2024-05-08T05:48:20.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.374+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:69 13.d 13:b0000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d04514000 2024-05-08T05:48:20.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.374+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.4 v2:172.21.15.63:6824/2212879151 12 ==== osd_op_reply(69 [pgnls start_epoch 92 out=74b] v87'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x558d04ed8780 con 0x558d04514000 2024-05-08T05:48:20.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.374+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:70 13.1d 13:b8000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d05084c00 2024-05-08T05:48:20.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.375+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 14 ==== osd_op_reply(70 [pgnls start_epoch 92 out=73b] v91'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.375+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:71 13.3 13:c0000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d05085800 2024-05-08T05:48:20.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.375+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.0 v2:172.21.15.45:6800/3734454393 7 ==== osd_op_reply(71 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085800 2024-05-08T05:48:20.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.375+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:72 13.13 13:c8000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d05084c00 2024-05-08T05:48:20.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.375+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 15 ==== osd_op_reply(72 [pgnls start_epoch 92 out=98b] v91'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.375+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:73 13.b 13:d0000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d050a6c00 2024-05-08T05:48:20.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.375+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.5 v2:172.21.15.63:6808/2395798255 7 ==== osd_op_reply(73 [pgnls start_epoch 92 out=76b] v91'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x558d04ed8000 con 0x558d050a6c00 2024-05-08T05:48:20.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.375+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:74 13.1b 13:d8000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d04514000 2024-05-08T05:48:20.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.376+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.4 v2:172.21.15.63:6824/2212879151 13 ==== osd_op_reply(74 [pgnls start_epoch 92 out=73b] v91'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x558d04ed8780 con 0x558d04514000 2024-05-08T05:48:20.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.376+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:75 13.7 13:e0000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d05084c00 2024-05-08T05:48:20.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.376+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 16 ==== osd_op_reply(75 [pgnls start_epoch 92 out=73b] v91'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.376+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:76 13.17 13:e8000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d050a6c00 2024-05-08T05:48:20.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.376+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.5 v2:172.21.15.63:6808/2395798255 8 ==== osd_op_reply(76 [pgnls start_epoch 92 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558d04ed8000 con 0x558d050a6c00 2024-05-08T05:48:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.376+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:77 13.f 13:f0000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d050a7c00 2024-05-08T05:48:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.377+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.6 v2:172.21.15.63:6800/2926759052 11 ==== osd_op_reply(77 [pgnls start_epoch 92 out=73b] v87'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x558d04ed8780 con 0x558d050a7c00 2024-05-08T05:48:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.377+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:78 13.1f 13:f8000000::::head [pgnls start_epoch 92 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e92) -- 0x558d04515800 con 0x558d050a7c00 2024-05-08T05:48:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.377+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.6 v2:172.21.15.63:6800/2926759052 12 ==== osd_op_reply(78 [pgnls start_epoch 92 out=101b] v91'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x558d04ed8780 con 0x558d050a7c00 2024-05-08T05:48:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.377+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-05-08T05:48:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.377+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:79 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451c400 con 0x558d05085c00 2024-05-08T05:48:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.377+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 14 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.377+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-05-08T05:48:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.377+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:80 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451c800 con 0x558d05085c00 2024-05-08T05:48:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.378+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 15 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.378+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-05-08T05:48:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.378+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:81 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451dc00 con 0x558d05085c00 2024-05-08T05:48:20.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.378+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 16 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.378+0000 7f558f41a740 20 datascan.scan_links: injected ino detected 2024-05-08T05:48:20.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.378+0000 7f558f41a740 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-05-08T05:48:20.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.378+0000 7f558f41a740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-05-08T05:48:20.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.378+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-05-08T05:48:20.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.378+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:82 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d05085000 con 0x558d05085c00 2024-05-08T05:48:20.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.378+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 17 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.378+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-05-08T05:48:20.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.378+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:83 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d04515000 con 0x558d05085c00 2024-05-08T05:48:20.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.379+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 18 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.379+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-05-08T05:48:20.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.379+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-08T05:48:20.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.379+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-05-08T05:48:20.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.379+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-08T05:48:20.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.379+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-05-08T05:48:20.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.379+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-08T05:48:20.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.379+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-05-08T05:48:20.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.379+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:84 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d04514c00 con 0x558d05085800 2024-05-08T05:48:20.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.379+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.0 v2:172.21.15.45:6800/3734454393 8 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085800 2024-05-08T05:48:20.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.379+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-05-08T05:48:20.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.379+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:85 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d04514800 con 0x558d050a7c00 2024-05-08T05:48:20.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.379+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.6 v2:172.21.15.63:6800/2926759052 13 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x558d04ed8780 con 0x558d050a7c00 2024-05-08T05:48:20.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.379+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-05-08T05:48:20.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.379+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:86 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d04514400 con 0x558d05085400 2024-05-08T05:48:20.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.380+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.2 v2:172.21.15.45:6808/785834340 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x558d04ed8000 con 0x558d05085400 2024-05-08T05:48:20.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.380+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-05-08T05:48:20.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.380+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-08T05:48:20.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.380+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-05-08T05:48:20.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.380+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:87 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d04515c00 con 0x558d05084c00 2024-05-08T05:48:20.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.380+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 17 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.380+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-05-08T05:48:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.380+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:88 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451c000 con 0x558d04514000 2024-05-08T05:48:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.380+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.4 v2:172.21.15.63:6824/2212879151 14 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x558d04ed8780 con 0x558d04514000 2024-05-08T05:48:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.380+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-05-08T05:48:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.380+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-08T05:48:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.380+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-05-08T05:48:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.380+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-08T05:48:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.380+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-05-08T05:48:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.380+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451c400 con 0x558d05084c00 2024-05-08T05:48:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.381+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 18 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.381+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-05-08T05:48:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.381+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-08T05:48:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.381+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-05-08T05:48:20.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.381+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:90 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451c800 con 0x558d05084c00 2024-05-08T05:48:20.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.381+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 19 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.381+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-05-08T05:48:20.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.381+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-08T05:48:20.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.381+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 200.00000002 2024-05-08T05:48:20.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.381+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-05-08T05:48:20.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.381+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-05-08T05:48:20.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.381+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:91 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451cc00 con 0x558d05084c00 2024-05-08T05:48:20.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.381+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 20 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.381+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-05-08T05:48:20.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.381+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-08T05:48:20.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.381+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-05-08T05:48:20.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.381+0000 7f558f41a740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-08T05:48:20.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.381+0000 7f558f41a740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-05-08T05:48:20.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.381+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:92 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451d000 con 0x558d050a7c00 2024-05-08T05:48:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.382+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.6 v2:172.21.15.63:6800/2926759052 14 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) ==== 154+0+1163 (crc 0 0 0) 0x558d04ed8780 con 0x558d050a7c00 2024-05-08T05:48:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.382+0000 7f558f41a740 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-05-08T05:48:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.382+0000 7f558f41a740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-05-08T05:48:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.382+0000 7f558f41a740 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-05-08T05:48:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.382+0000 7f558f41a740 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-05-08T05:48:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.382+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:93 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451d400 con 0x558d05085c00 2024-05-08T05:48:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.382+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 19 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.382+0000 7f558f41a740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-08T05:48:20.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.382+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:94 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x558d0451d800 con 0x558d05085c00 2024-05-08T05:48:20.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.383+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 20 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v92'19 uv19 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.383+0000 7f558f41a740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-08T05:48:20.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.383+0000 7f558f41a740 10 datascan.scan_links: processing 1 injected_inos 2024-05-08T05:48:20.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.383+0000 7f558f41a740 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-05-08T05:48:20.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.383+0000 7f558f41a740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-05-08T05:48:20.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.383+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:95 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d0451dc00 con 0x558d05085c00 2024-05-08T05:48:20.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.383+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 21 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.383+0000 7f558f41a740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-05-08T05:48:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.383+0000 7f558f41a740 20 datascan.scan_links: first is now 2 2024-05-08T05:48:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.383+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:96 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x558d04526000 con 0x558d05085c00 2024-05-08T05:48:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.384+0000 7f558d83a640 1 -- 172.21.15.45:0/2648007777 <== osd.1 v2:172.21.15.45:6824/1337179184 22 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v92'20 uv20 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x558d04ed8500 con 0x558d05085c00 2024-05-08T05:48:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.384+0000 7f558f41a740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-05-08T05:48:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.384+0000 7f558f41a740 10 datascan.scan_links: updating inotable 2024-05-08T05:48:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.384+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:97 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d04526400 con 0x558d05084c00 2024-05-08T05:48:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.385+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 21 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.385+0000 7f558f41a740 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-05-08T05:48:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.385+0000 7f558f41a740 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-05-08T05:48:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.385+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:98 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) -- 0x558d04526800 con 0x558d05084c00 2024-05-08T05:48:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.386+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 <== osd.7 v2:172.21.15.63:6816/1163469437 22 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v92'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x558d04ed8000 con 0x558d05084c00 2024-05-08T05:48:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.386+0000 7f558f41a740 10 datascan.scan_links: updating snaptable 2024-05-08T05:48:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.386+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:99 13.d 13:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) -- 0x558d04526c00 con 0x558d04514000 2024-05-08T05:48:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558c838640 1 -- 172.21.15.45:0/2648007777 <== osd.4 v2:172.21.15.63:6824/2212879151 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x558d04ed8780 con 0x558d04514000 2024-05-08T05:48:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x558d05085800 msgr2=0x558d044f7600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x558d05085800 0x558d044f7600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x558d05085c00 msgr2=0x558d044f6000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x558d05085c00 0x558d044f6000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x558d05085400 msgr2=0x558d044f6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x558d05085400 0x558d044f6580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x558d04515400 msgr2=0x558d044f8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x558d04515400 0x558d044f8100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 reap_dead start 2024-05-08T05:48:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x558d04514000 msgr2=0x558d044f8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x558d04514000 0x558d044f8680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x558d050a6c00 msgr2=0x558d044f7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x558d050a6c00 0x558d044f7b80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x558d050a7c00 msgr2=0x558d044f6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x558d050a7c00 0x558d044f6b00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x558d05084c00 msgr2=0x558d04f26680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x558d05084c00 0x558d04f26680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558d039640 1 -- 172.21.15.45:0/2648007777 reap_dead start 2024-05-08T05:48:20.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x558d050a7000 msgr2=0x558d04f26100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x558d050a7000 0x558d04f26100 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x558d04eef080 tx=0x558d04471020 comp rx=0 tx=0).stop 2024-05-08T05:48:20.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d05084800 msgr2=0x558d04f25b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d05084800 0x558d04f25b80 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x558d04f62840 tx=0x558d04eec720 comp rx=0 tx=0).stop 2024-05-08T05:48:20.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 shutdown_connections 2024-05-08T05:48:20.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x558d050a7000 0x558d04f26100 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 --2- 172.21.15.45:0/2648007777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d05084800 0x558d04f25b80 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 >> 172.21.15.45:0/2648007777 conn(0x558d04f2dc00 msgr2=0x558d04ec5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:20.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 shutdown_connections 2024-05-08T05:48:20.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 -- 172.21.15.45:0/2648007777 wait complete. 2024-05-08T05:48:20.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 -- 172.21.15.45:0/3738411627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d04147800 msgr2=0x558d04f24580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 --2- 172.21.15.45:0/3738411627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d04147800 0x558d04f24580 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x558d04f7bc80 tx=0x558d04eec600 comp rx=0 tx=0).stop 2024-05-08T05:48:20.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 -- 172.21.15.45:0/3738411627 shutdown_connections 2024-05-08T05:48:20.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 --2- 172.21.15.45:0/3738411627 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558d04147c00 0x558d04f24b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 --2- 172.21.15.45:0/3738411627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558d04147800 0x558d04f24580 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 --2- 172.21.15.45:0/3738411627 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558d04146c00 0x558d04f24000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 -- 172.21.15.45:0/3738411627 >> 172.21.15.45:0/3738411627 conn(0x558d04147000 msgr2=0x558d04ec4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:20.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 -- 172.21.15.45:0/3738411627 shutdown_connections 2024-05-08T05:48:20.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.387+0000 7f558f41a740 1 -- 172.21.15.45:0/3738411627 wait complete. 2024-05-08T05:48:20.412 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.113606, result: 2024-05-08T05:48:20.412 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-08T05:48:20.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.555+0000 7f36bb77f640 1 Processor -- start 2024-05-08T05:48:20.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.555+0000 7f36bb77f640 1 -- start start 2024-05-08T05:48:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.556+0000 7f36bb77f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f36b4070d30 0x7f36b40d2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.556+0000 7f36bb77f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f36b41639e0 0x7f36b40703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.556+0000 7f36bb77f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36b4071110 0x7f36b406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.556+0000 7f36bb77f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f36b40d1ca0 con 0x7f36b41639e0 2024-05-08T05:48:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.556+0000 7f36bb77f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f36b40d10d0 con 0x7f36b4070d30 2024-05-08T05:48:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.556+0000 7f36bb77f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f36b40d1370 con 0x7f36b4071110 2024-05-08T05:48:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.556+0000 7f36b8d1a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f36b41639e0 0x7f36b40703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.556+0000 7f36b8d1a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f36b41639e0 0x7f36b40703a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35350/0 (socket says 172.21.15.45:35350) 2024-05-08T05:48:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.556+0000 7f36b8d1a640 1 -- 172.21.15.45:0/715230954 learned_addr learned my addr 172.21.15.45:0/715230954 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.556+0000 7f36b951b640 1 --2- 172.21.15.45:0/715230954 >> v2:172.21.15.63:3300/0 conn(0x7f36b4070d30 0x7f36b40d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.556+0000 7f36b9d1c640 1 --2- 172.21.15.45:0/715230954 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36b4071110 0x7f36b406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.556+0000 7f36b8d1a640 1 -- 172.21.15.45:0/715230954 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36b4071110 msgr2=0x7f36b406d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.556+0000 7f36b8d1a640 1 --2- 172.21.15.45:0/715230954 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36b4071110 0x7f36b406d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.556+0000 7f36b8d1a640 1 -- 172.21.15.45:0/715230954 >> v2:172.21.15.63:3300/0 conn(0x7f36b4070d30 msgr2=0x7f36b40d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:20.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.556+0000 7f36b8d1a640 1 --2- 172.21.15.45:0/715230954 >> v2:172.21.15.63:3300/0 conn(0x7f36b4070d30 0x7f36b40d2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.556+0000 7f36b8d1a640 1 -- 172.21.15.45:0/715230954 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b40d1650 con 0x7f36b41639e0 2024-05-08T05:48:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.557+0000 7f36b8d1a640 1 --2- 172.21.15.45:0/715230954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36b41639e0 0x7f36b40703a0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f36b40701d0 tx=0x7f36a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fe179ab98a8bb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.557+0000 7f36abfff640 1 -- 172.21.15.45:0/715230954 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4037070 con 0x7f36b41639e0 2024-05-08T05:48:20.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.557+0000 7f36abfff640 1 -- 172.21.15.45:0/715230954 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36a4032440 con 0x7f36b41639e0 2024-05-08T05:48:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.557+0000 7f36abfff640 1 -- 172.21.15.45:0/715230954 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4032810 con 0x7f36b41639e0 2024-05-08T05:48:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.558+0000 7f36bb77f640 1 -- 172.21.15.45:0/715230954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36b41639e0 msgr2=0x7f36b40703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.558+0000 7f36bb77f640 1 --2- 172.21.15.45:0/715230954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36b41639e0 0x7f36b40703a0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f36b40701d0 tx=0x7f36a402a1e0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.558+0000 7f36bb77f640 1 -- 172.21.15.45:0/715230954 shutdown_connections 2024-05-08T05:48:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.558+0000 7f36bb77f640 1 --2- 172.21.15.45:0/715230954 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36b4071110 0x7f36b406d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.558+0000 7f36bb77f640 1 --2- 172.21.15.45:0/715230954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36b41639e0 0x7f36b40703a0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.558+0000 7f36bb77f640 1 --2- 172.21.15.45:0/715230954 >> v2:172.21.15.63:3300/0 conn(0x7f36b4070d30 0x7f36b40d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.558+0000 7f36bb77f640 1 -- 172.21.15.45:0/715230954 >> 172.21.15.45:0/715230954 conn(0x7f36b40d2e90 msgr2=0x7f36b40d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.558+0000 7f36bb77f640 1 -- 172.21.15.45:0/715230954 shutdown_connections 2024-05-08T05:48:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.558+0000 7f36bb77f640 1 -- 172.21.15.45:0/715230954 wait complete. 2024-05-08T05:48:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.559+0000 7f36bb77f640 1 Processor -- start 2024-05-08T05:48:20.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.559+0000 7f36bb77f640 1 -- start start 2024-05-08T05:48:20.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.560+0000 7f36bb77f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36b4070d30 0x7f36b4151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.560+0000 7f36bb77f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36b4071110 0x7f36b40cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.560+0000 7f36bb77f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36b41639e0 0x7f36b40ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.560+0000 7f36bb77f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f36b4071580 con 0x7f36b4070d30 2024-05-08T05:48:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.560+0000 7f36bb77f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f36b40708e0 con 0x7f36b41639e0 2024-05-08T05:48:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.560+0000 7f36bb77f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f36b4071700 con 0x7f36b4071110 2024-05-08T05:48:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.560+0000 7f36b951b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36b4070d30 0x7f36b4151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.560+0000 7f36b951b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36b4070d30 0x7f36b4151c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35364/0 (socket says 172.21.15.45:35364) 2024-05-08T05:48:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.560+0000 7f36b951b640 1 -- 172.21.15.45:0/1423791066 learned_addr learned my addr 172.21.15.45:0/1423791066 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.560+0000 7f36b9d1c640 1 --2- 172.21.15.45:0/1423791066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36b41639e0 0x7f36b40ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.560+0000 7f36b8d1a640 1 --2- 172.21.15.45:0/1423791066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36b4071110 0x7f36b40cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.560+0000 7f36b951b640 1 -- 172.21.15.45:0/1423791066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36b4071110 msgr2=0x7f36b40cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.560+0000 7f36b951b640 1 --2- 172.21.15.45:0/1423791066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36b4071110 0x7f36b40cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.560+0000 7f36b951b640 1 -- 172.21.15.45:0/1423791066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36b41639e0 msgr2=0x7f36b40ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:20.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.560+0000 7f36b951b640 1 --2- 172.21.15.45:0/1423791066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36b41639e0 0x7f36b40ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:20.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.560+0000 7f36b951b640 1 -- 172.21.15.45:0/1423791066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b40c7e20 con 0x7f36b4070d30 2024-05-08T05:48:20.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.561+0000 7f36b951b640 1 --2- 172.21.15.45:0/1423791066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36b4070d30 0x7f36b4151c80 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f36b000ef00 tx=0x7f36b000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.561+0000 7f36a9ffb640 1 -- 172.21.15.45:0/1423791066 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0018020 con 0x7f36b4070d30 2024-05-08T05:48:20.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.561+0000 7f36a9ffb640 1 -- 172.21.15.45:0/1423791066 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36b000bd90 con 0x7f36b4070d30 2024-05-08T05:48:20.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.561+0000 7f36bb77f640 1 -- 172.21.15.45:0/1423791066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b40c8100 con 0x7f36b4070d30 2024-05-08T05:48:20.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.561+0000 7f36a9ffb640 1 -- 172.21.15.45:0/1423791066 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0014620 con 0x7f36b4070d30 2024-05-08T05:48:20.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.561+0000 7f36bb77f640 1 -- 172.21.15.45:0/1423791066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b40c9f60 con 0x7f36b4070d30 2024-05-08T05:48:20.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.562+0000 7f36bb77f640 1 -- 172.21.15.45:0/1423791066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36b40693f0 con 0x7f36b4070d30 2024-05-08T05:48:20.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.563+0000 7f36a9ffb640 1 -- 172.21.15.45:0/1423791066 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f36b0005d10 con 0x7f36b4070d30 2024-05-08T05:48:20.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.563+0000 7f36a9ffb640 1 --2- 172.21.15.45:0/1423791066 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f36980762d0 0x7f3698078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:20.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.563+0000 7f36a9ffb640 1 -- 172.21.15.45:0/1423791066 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7f36b0097f20 con 0x7f36b4070d30 2024-05-08T05:48:20.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.563+0000 7f36b8d1a640 1 --2- 172.21.15.45:0/1423791066 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f36980762d0 0x7f3698078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:20.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.564+0000 7f36b8d1a640 1 --2- 172.21.15.45:0/1423791066 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f36980762d0 0x7f3698078790 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f36a4034000 tx=0x7f36a402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:20.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.568+0000 7f36a9ffb640 1 -- 172.21.15.45:0/1423791066 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f36b0022070 con 0x7f36b4070d30 2024-05-08T05:48:20.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:20.798+0000 7f36bb77f640 1 -- 172.21.15.45:0/1423791066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f36b40b2100 con 0x7f36b4070d30 2024-05-08T05:48:21.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.010+0000 7f36a9ffb640 1 -- 172.21.15.45:0/1423791066 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v69) ==== 162+0+0 (secure 0 0 0) 0x7f36b0061010 con 0x7f36b4070d30 2024-05-08T05:48:21.011 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-08T05:48:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.013+0000 7f36bb77f640 1 -- 172.21.15.45:0/1423791066 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f36980762d0 msgr2=0x7f3698078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.013+0000 7f36bb77f640 1 --2- 172.21.15.45:0/1423791066 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f36980762d0 0x7f3698078790 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f36a4034000 tx=0x7f36a402e000 comp rx=0 tx=0).stop 2024-05-08T05:48:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.013+0000 7f36bb77f640 1 -- 172.21.15.45:0/1423791066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36b4070d30 msgr2=0x7f36b4151c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.013+0000 7f36bb77f640 1 --2- 172.21.15.45:0/1423791066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36b4070d30 0x7f36b4151c80 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f36b000ef00 tx=0x7f36b000b480 comp rx=0 tx=0).stop 2024-05-08T05:48:21.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.013+0000 7f36bb77f640 1 -- 172.21.15.45:0/1423791066 shutdown_connections 2024-05-08T05:48:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.013+0000 7f36bb77f640 1 --2- 172.21.15.45:0/1423791066 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f36980762d0 0x7f3698078790 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.013+0000 7f36bb77f640 1 --2- 172.21.15.45:0/1423791066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f36b4071110 0x7f36b40cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.013+0000 7f36bb77f640 1 --2- 172.21.15.45:0/1423791066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f36b41639e0 0x7f36b40ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.013+0000 7f36bb77f640 1 --2- 172.21.15.45:0/1423791066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f36b4070d30 0x7f36b4151c80 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.013+0000 7f36bb77f640 1 -- 172.21.15.45:0/1423791066 >> 172.21.15.45:0/1423791066 conn(0x7f36b40d2e90 msgr2=0x7f36b40ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.013+0000 7f36bb77f640 1 -- 172.21.15.45:0/1423791066 shutdown_connections 2024-05-08T05:48:21.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.013+0000 7f36bb77f640 1 -- 172.21.15.45:0/1423791066 wait complete. 2024-05-08T05:48:21.026 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:48:21.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.130+0000 7ff13a122640 1 Processor -- start 2024-05-08T05:48:21.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.130+0000 7ff13a122640 1 -- start start 2024-05-08T05:48:21.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.131+0000 7ff13a122640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff13406e0f0 0x7ff13415c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:21.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.131+0000 7ff13a122640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff13406e4d0 0x7ff1340aec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:21.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.131+0000 7ff13a122640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff13415c920 con 0x7ff134072840 2024-05-08T05:48:21.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.131+0000 7ff13a122640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff13415cc00 con 0x7ff13406e0f0 2024-05-08T05:48:21.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.131+0000 7ff13a122640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff13415cee0 con 0x7ff13406e4d0 2024-05-08T05:48:21.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.131+0000 7ff1337fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff13406e0f0 0x7ff13415c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:21.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.131+0000 7ff1337fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff13406e0f0 0x7ff13415c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43290/0 (socket says 172.21.15.45:43290) 2024-05-08T05:48:21.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.131+0000 7ff1337fe640 1 -- 172.21.15.45:0/3147498971 learned_addr learned my addr 172.21.15.45:0/3147498971 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:21.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.132+0000 7ff133fff640 1 --2- 172.21.15.45:0/3147498971 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff13406e4d0 0x7ff1340aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:21.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.132+0000 7ff1337fe640 1 -- 172.21.15.45:0/3147498971 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff13406e4d0 msgr2=0x7ff1340aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:21.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.132+0000 7ff1337fe640 1 --2- 172.21.15.45:0/3147498971 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff13406e4d0 0x7ff1340aec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:21.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.132+0000 7ff1337fe640 1 -- 172.21.15.45:0/3147498971 >> v1:172.21.15.63:6789/0 conn(0x7ff134072840 legacy=0x7ff1340d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:21.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.132+0000 7ff1337fe640 1 -- 172.21.15.45:0/3147498971 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13415d1c0 con 0x7ff13406e0f0 2024-05-08T05:48:21.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.132+0000 7ff1337fe640 1 --2- 172.21.15.45:0/3147498971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff13406e0f0 0x7ff13415c340 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7ff124009870 tx=0x7ff12402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57cd5f00f8fc59f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:21.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.133+0000 7ff1327fc640 1 -- 172.21.15.45:0/3147498971 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124037070 con 0x7ff13406e0f0 2024-05-08T05:48:21.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.133+0000 7ff1327fc640 1 -- 172.21.15.45:0/3147498971 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff124032440 con 0x7ff13406e0f0 2024-05-08T05:48:21.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.133+0000 7ff1327fc640 1 -- 172.21.15.45:0/3147498971 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124032810 con 0x7ff13406e0f0 2024-05-08T05:48:21.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.133+0000 7ff13a122640 1 -- 172.21.15.45:0/3147498971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff13406e0f0 msgr2=0x7ff13415c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:21.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.133+0000 7ff13a122640 1 --2- 172.21.15.45:0/3147498971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff13406e0f0 0x7ff13415c340 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7ff124009870 tx=0x7ff12402a1c0 comp rx=0 tx=0).stop 2024-05-08T05:48:21.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.133+0000 7ff13a122640 1 -- 172.21.15.45:0/3147498971 shutdown_connections 2024-05-08T05:48:21.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.133+0000 7ff13a122640 1 --2- 172.21.15.45:0/3147498971 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff13406e4d0 0x7ff1340aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:21.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.133+0000 7ff13a122640 1 --2- 172.21.15.45:0/3147498971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff13406e0f0 0x7ff13415c340 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:21.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.133+0000 7ff13a122640 1 -- 172.21.15.45:0/3147498971 >> 172.21.15.45:0/3147498971 conn(0x7ff13406c3a0 msgr2=0x7ff13406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:21.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.133+0000 7ff13a122640 1 -- 172.21.15.45:0/3147498971 shutdown_connections 2024-05-08T05:48:21.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.133+0000 7ff13a122640 1 -- 172.21.15.45:0/3147498971 wait complete. 2024-05-08T05:48:21.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.134+0000 7ff13a122640 1 Processor -- start 2024-05-08T05:48:21.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.135+0000 7ff13a122640 1 -- start start 2024-05-08T05:48:21.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.135+0000 7ff13a122640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff13406e0f0 0x7ff134157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:21.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.135+0000 7ff13a122640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff13406e4d0 0x7ff1341526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:21.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.135+0000 7ff13a122640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff134072840 0x7ff13414a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:21.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.135+0000 7ff13a122640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff1340ae7d0 con 0x7ff134072840 2024-05-08T05:48:21.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.135+0000 7ff13a122640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff1340ae650 con 0x7ff13406e0f0 2024-05-08T05:48:21.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.135+0000 7ff13a122640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff1341530b0 con 0x7ff13406e4d0 2024-05-08T05:48:21.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.135+0000 7ff133fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff134072840 0x7ff13414a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:21.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.136+0000 7ff133fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff134072840 0x7ff13414a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43302/0 (socket says 172.21.15.45:43302) 2024-05-08T05:48:21.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.136+0000 7ff133fff640 1 -- 172.21.15.45:0/2294735011 learned_addr learned my addr 172.21.15.45:0/2294735011 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:21.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.136+0000 7ff1337fe640 1 --2- 172.21.15.45:0/2294735011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff13406e0f0 0x7ff134157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:21.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.136+0000 7ff132ffd640 1 --2- 172.21.15.45:0/2294735011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff13406e4d0 0x7ff1341526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:21.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.136+0000 7ff133fff640 1 -- 172.21.15.45:0/2294735011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff13406e4d0 msgr2=0x7ff1341526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:21.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.136+0000 7ff133fff640 1 --2- 172.21.15.45:0/2294735011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff13406e4d0 0x7ff1341526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:21.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.136+0000 7ff133fff640 1 -- 172.21.15.45:0/2294735011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff13406e0f0 msgr2=0x7ff134157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:21.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.136+0000 7ff133fff640 1 --2- 172.21.15.45:0/2294735011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff13406e0f0 0x7ff134157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:21.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.136+0000 7ff133fff640 1 -- 172.21.15.45:0/2294735011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff134159b40 con 0x7ff134072840 2024-05-08T05:48:21.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.136+0000 7ff132ffd640 1 --2- 172.21.15.45:0/2294735011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff13406e4d0 0x7ff1341526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:48:21.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.136+0000 7ff133fff640 1 --2- 172.21.15.45:0/2294735011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff134072840 0x7ff13414a020 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00cd70 tx=0x7ff12c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:21.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.136+0000 7ff11bfff640 1 -- 172.21.15.45:0/2294735011 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c003b90 con 0x7ff134072840 2024-05-08T05:48:21.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.136+0000 7ff11bfff640 1 -- 172.21.15.45:0/2294735011 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff12c003d30 con 0x7ff134072840 2024-05-08T05:48:21.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.136+0000 7ff13a122640 1 -- 172.21.15.45:0/2294735011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1341f6650 con 0x7ff134072840 2024-05-08T05:48:21.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.136+0000 7ff11bfff640 1 -- 172.21.15.45:0/2294735011 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c004240 con 0x7ff134072840 2024-05-08T05:48:21.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.136+0000 7ff13a122640 1 -- 172.21.15.45:0/2294735011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1341f6c10 con 0x7ff134072840 2024-05-08T05:48:21.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.137+0000 7ff13a122640 1 -- 172.21.15.45:0/2294735011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1341f6e10 con 0x7ff134072840 2024-05-08T05:48:21.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.138+0000 7ff11bfff640 1 -- 172.21.15.45:0/2294735011 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff12c0043e0 con 0x7ff134072840 2024-05-08T05:48:21.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.138+0000 7ff11bfff640 1 --2- 172.21.15.45:0/2294735011 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff1200762d0 0x7ff120078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:21.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.139+0000 7ff11bfff640 1 -- 172.21.15.45:0/2294735011 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7ff12c098310 con 0x7ff134072840 2024-05-08T05:48:21.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.139+0000 7ff1337fe640 1 --2- 172.21.15.45:0/2294735011 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff1200762d0 0x7ff120078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:21.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.139+0000 7ff1337fe640 1 --2- 172.21.15.45:0/2294735011 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff1200762d0 0x7ff120078790 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff124034000 tx=0x7ff12402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:21.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.143+0000 7ff11bfff640 1 -- 172.21.15.45:0/2294735011 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff1341f6e10 con 0x7ff134072840 2024-05-08T05:48:21.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.375+0000 7ff13a122640 1 -- 172.21.15.45:0/2294735011 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1340693f0 con 0x7ff134072840 2024-05-08T05:48:21.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.375+0000 7ff11bfff640 1 -- 172.21.15.45:0/2294735011 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5177 (secure 0 0 0) 0x7ff12c061400 con 0x7ff134072840 2024-05-08T05:48:21.376 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:48:21.376 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":70,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2616298287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2616298287},{"type":"v1","addr":"172.21.15.63:6836","nonce":2616298287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6472,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1347752311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1347752311},{"type":"v1","addr":"172.21.15.63:6837","nonce":1347752311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6744,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2854804427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2854804427},{"type":"v1","addr":"172.21.15.45:6834","nonce":2854804427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:47:00.211837+0000","modified":"2024-05-08T05:48:21.013971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6747},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6747":{"gid":6747,"name":"c","rank":0,"incarnation":70,"state":"up:replay","state_seq":1,"addr":"172.21.15.45:6835/2945763952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2945763952},{"type":"v1","addr":"172.21.15.45:6835","nonce":2945763952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-08T05:48:21.376 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 70 2024-05-08T05:48:21.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.378+0000 7ff13a122640 1 -- 172.21.15.45:0/2294735011 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff1200762d0 msgr2=0x7ff120078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:21.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.378+0000 7ff13a122640 1 --2- 172.21.15.45:0/2294735011 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff1200762d0 0x7ff120078790 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff124034000 tx=0x7ff12402e000 comp rx=0 tx=0).stop 2024-05-08T05:48:21.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.378+0000 7ff13a122640 1 -- 172.21.15.45:0/2294735011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff134072840 msgr2=0x7ff13414a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:21.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.378+0000 7ff13a122640 1 --2- 172.21.15.45:0/2294735011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff134072840 0x7ff13414a020 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00cd70 tx=0x7ff12c00a660 comp rx=0 tx=0).stop 2024-05-08T05:48:21.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.378+0000 7ff13a122640 1 -- 172.21.15.45:0/2294735011 shutdown_connections 2024-05-08T05:48:21.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.378+0000 7ff13a122640 1 --2- 172.21.15.45:0/2294735011 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff1200762d0 0x7ff120078790 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:21.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.378+0000 7ff13a122640 1 --2- 172.21.15.45:0/2294735011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff13406e4d0 0x7ff1341526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:21.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.378+0000 7ff13a122640 1 --2- 172.21.15.45:0/2294735011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff134072840 0x7ff13414a020 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:21.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.378+0000 7ff13a122640 1 --2- 172.21.15.45:0/2294735011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff13406e0f0 0x7ff134157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:21.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.378+0000 7ff13a122640 1 -- 172.21.15.45:0/2294735011 >> 172.21.15.45:0/2294735011 conn(0x7ff13406c3a0 msgr2=0x7ff1340aec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:21.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.378+0000 7ff13a122640 1 -- 172.21.15.45:0/2294735011 shutdown_connections 2024-05-08T05:48:21.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:21.378+0000 7ff13a122640 1 -- 172.21.15.45:0/2294735011 wait complete. 2024-05-08T05:48:21.791 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:47:00.211837+0000', 'modified': '2024-05-08T05:48:21.013971+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 92, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6747}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6747': {'gid': 6747, 'name': 'c', 'rank': 0, 'incarnation': 70, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.45:6835/2945763952', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 2945763952}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 2945763952}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-08T05:48:21.792 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6747:up:replay 2024-05-08T05:48:22.793 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:48:22.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.909+0000 7f6bebf19640 1 Processor -- start 2024-05-08T05:48:22.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.910+0000 7f6bebf19640 1 -- start start 2024-05-08T05:48:22.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.910+0000 7f6bebf19640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6be40d53d0 0x7f6be40d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:22.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.910+0000 7f6bebf19640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6be41630b0 con 0x7f6be40d1970 2024-05-08T05:48:22.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.910+0000 7f6bebf19640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6be40d3e70 con 0x7f6be415f420 2024-05-08T05:48:22.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.910+0000 7f6bebf19640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6be40d2550 con 0x7f6be40d53d0 2024-05-08T05:48:22.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6be94b4640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6be40d1970 0x7f6be40d4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48558/0 (socket says 172.21.15.45:48558) 2024-05-08T05:48:22.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6be94b4640 1 -- 172.21.15.45:0/3161635640 learned_addr learned my addr 172.21.15.45:0/3161635640 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:48:22.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6bea4b6640 1 --2- 172.21.15.45:0/3161635640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6be40d53d0 0x7f6be40d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:22.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6be8cb3640 1 -- 172.21.15.45:0/3161635640 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1103171850 0 0) 0x7f6bd4001110 con 0x7f6be40d1970 2024-05-08T05:48:22.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6be8cb3640 1 -- 172.21.15.45:0/3161635640 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6bd0003520 con 0x7f6be40d1970 2024-05-08T05:48:22.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6be8cb3640 1 -- 172.21.15.45:0/3161635640 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4121915206 0 0) 0x7f6be40d3e70 con 0x7f6be415f420 2024-05-08T05:48:22.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6be8cb3640 1 -- 172.21.15.45:0/3161635640 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6bd4001110 con 0x7f6be415f420 2024-05-08T05:48:22.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6be8cb3640 1 -- 172.21.15.45:0/3161635640 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 602239403 0 0) 0x7f6bd0003520 con 0x7f6be40d1970 2024-05-08T05:48:22.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6be8cb3640 1 -- 172.21.15.45:0/3161635640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6be40d53d0 msgr2=0x7f6be40d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:22.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6be8cb3640 1 --2- 172.21.15.45:0/3161635640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6be40d53d0 0x7f6be40d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:22.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6be8cb3640 1 -- 172.21.15.45:0/3161635640 >> v1:172.21.15.63:6789/0 conn(0x7f6be415f420 legacy=0x7f6be40d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:22.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6be8cb3640 1 -- 172.21.15.45:0/3161635640 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be40d2c80 con 0x7f6be40d1970 2024-05-08T05:48:22.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6be8cb3640 1 -- 172.21.15.45:0/3161635640 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f6bd40018c0 con 0x7f6be40d1970 2024-05-08T05:48:22.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6be8cb3640 1 -- 172.21.15.45:0/3161635640 >> v1:172.21.15.45:6789/0 conn(0x7f6be40d1970 legacy=0x7f6be40d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:22.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6be8cb3640 1 --2- 172.21.15.45:0/3161635640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bd0003c10 0x7f6bd0004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:22.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6be8cb3640 1 --2- 172.21.15.45:0/3161635640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bd0004800 0x7f6bd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:22.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6be8cb3640 1 --2- 172.21.15.45:0/3161635640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6bd00051c0 0x7f6bd000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:22.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6be8cb3640 1 -- 172.21.15.45:0/3161635640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6be40d2550 con 0x7f6bd0003c10 2024-05-08T05:48:22.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6be8cb3640 1 -- 172.21.15.45:0/3161635640 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd000c090 con 0x7f6bd0004800 2024-05-08T05:48:22.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6be8cb3640 1 -- 172.21.15.45:0/3161635640 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6bd000c290 con 0x7f6bd00051c0 2024-05-08T05:48:22.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6bea4b6640 1 --2- 172.21.15.45:0/3161635640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6be40d53d0 0x7f6be40d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:48:22.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.911+0000 7f6be9cb5640 1 --2- 172.21.15.45:0/3161635640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bd0003c10 0x7f6bd0004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:22.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.912+0000 7f6be94b4640 1 --2- 172.21.15.45:0/3161635640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bd0004800 0x7f6bd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:22.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.912+0000 7f6bea4b6640 1 --2- 172.21.15.45:0/3161635640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6bd00051c0 0x7f6bd000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:22.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.912+0000 7f6be9cb5640 1 -- 172.21.15.45:0/3161635640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6bd00051c0 msgr2=0x7f6bd000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:22.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.912+0000 7f6be9cb5640 1 --2- 172.21.15.45:0/3161635640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6bd00051c0 0x7f6bd000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:22.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.912+0000 7f6be9cb5640 1 -- 172.21.15.45:0/3161635640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bd0004800 msgr2=0x7f6bd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:22.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.912+0000 7f6be9cb5640 1 --2- 172.21.15.45:0/3161635640 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6bd0004800 0x7f6bd0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:22.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.912+0000 7f6be9cb5640 1 -- 172.21.15.45:0/3161635640 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd0003520 con 0x7f6bd0003c10 2024-05-08T05:48:22.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.912+0000 7f6bea4b6640 1 --2- 172.21.15.45:0/3161635640 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6bd00051c0 0x7f6bd000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:22.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.913+0000 7f6be9cb5640 1 --2- 172.21.15.45:0/3161635640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bd0003c10 0x7f6bd0004010 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6be0002000 tx=0x7f6bd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f250bbb15b65e080 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:22.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.913+0000 7f6be8cb3640 1 -- 172.21.15.45:0/3161635640 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be0001c20 con 0x7f6bd0003c10 2024-05-08T05:48:22.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.913+0000 7f6be8cb3640 1 -- 172.21.15.45:0/3161635640 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6be0006d70 con 0x7f6bd0003c10 2024-05-08T05:48:22.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.913+0000 7f6be8cb3640 1 -- 172.21.15.45:0/3161635640 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6be0003bb0 con 0x7f6bd0003c10 2024-05-08T05:48:22.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.913+0000 7f6bebf19640 1 -- 172.21.15.45:0/3161635640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bd0003c10 msgr2=0x7f6bd0004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:22.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.913+0000 7f6bebf19640 1 --2- 172.21.15.45:0/3161635640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6bd0003c10 0x7f6bd0004010 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6be0002000 tx=0x7f6bd0005830 comp rx=0 tx=0).stop 2024-05-08T05:48:22.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.913+0000 7f6bea4b6640 1 -- 172.21.15.45:0/3161635640 reap_dead start 2024-05-08T05:48:22.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.913+0000 7f6bebf19640 1 -- 172.21.15.45:0/3161635640 shutdown_connections 2024-05-08T05:48:22.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.913+0000 7f6bebf19640 1 -- 172.21.15.45:0/3161635640 >> 172.21.15.45:0/3161635640 conn(0x7f6be408eac0 msgr2=0x7f6be409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:22.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.913+0000 7f6bebf19640 1 -- 172.21.15.45:0/3161635640 shutdown_connections 2024-05-08T05:48:22.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.913+0000 7f6bebf19640 1 -- 172.21.15.45:0/3161635640 wait complete. 2024-05-08T05:48:22.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.915+0000 7f6bebf19640 1 Processor -- start 2024-05-08T05:48:22.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.915+0000 7f6bebf19640 1 -- start start 2024-05-08T05:48:22.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.915+0000 7f6bebf19640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6bd0003c10 0x7f6be40b9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:22.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.915+0000 7f6bebf19640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6be41f9480 0x7f6be40ba280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:22.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.915+0000 7f6bebf19640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6be40ba7c0 0x7f6be40c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:22.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.915+0000 7f6bebf19640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6be40d2830 con 0x7f6be40ba7c0 2024-05-08T05:48:22.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.915+0000 7f6bebf19640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6be40d3240 con 0x7f6be41f9480 2024-05-08T05:48:22.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.915+0000 7f6bebf19640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6be40d1e20 con 0x7f6bd0003c10 2024-05-08T05:48:22.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.916+0000 7f6bea4b6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6be40ba7c0 0x7f6be40c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:22.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.916+0000 7f6bea4b6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6be40ba7c0 0x7f6be40c9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43320/0 (socket says 172.21.15.45:43320) 2024-05-08T05:48:22.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.916+0000 7f6bea4b6640 1 -- 172.21.15.45:0/1926904685 learned_addr learned my addr 172.21.15.45:0/1926904685 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:22.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.916+0000 7f6be94b4640 1 --2- 172.21.15.45:0/1926904685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6be41f9480 0x7f6be40ba280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:22.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.916+0000 7f6be9cb5640 1 --2- 172.21.15.45:0/1926904685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6bd0003c10 0x7f6be40b9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:22.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.916+0000 7f6bea4b6640 1 -- 172.21.15.45:0/1926904685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6bd0003c10 msgr2=0x7f6be40b9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:22.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.916+0000 7f6bea4b6640 1 --2- 172.21.15.45:0/1926904685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6bd0003c10 0x7f6be40b9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:22.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.916+0000 7f6bea4b6640 1 -- 172.21.15.45:0/1926904685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6be41f9480 msgr2=0x7f6be40ba280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.916+0000 7f6bea4b6640 1 --2- 172.21.15.45:0/1926904685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6be41f9480 0x7f6be40ba280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.916+0000 7f6bea4b6640 1 -- 172.21.15.45:0/1926904685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be40b8290 con 0x7f6be40ba7c0 2024-05-08T05:48:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.916+0000 7f6bea4b6640 1 --2- 172.21.15.45:0/1926904685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6be40ba7c0 0x7f6be40c9b50 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6be40b6a30 tx=0x7f6bdc011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.916+0000 7f6bda7fc640 1 -- 172.21.15.45:0/1926904685 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc004440 con 0x7f6be40ba7c0 2024-05-08T05:48:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.916+0000 7f6bda7fc640 1 -- 172.21.15.45:0/1926904685 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bdc01c070 con 0x7f6be40ba7c0 2024-05-08T05:48:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.916+0000 7f6bebf19640 1 -- 172.21.15.45:0/1926904685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be40b8570 con 0x7f6be40ba7c0 2024-05-08T05:48:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.916+0000 7f6bda7fc640 1 -- 172.21.15.45:0/1926904685 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc003830 con 0x7f6be40ba7c0 2024-05-08T05:48:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.916+0000 7f6bebf19640 1 -- 172.21.15.45:0/1926904685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be40b8b30 con 0x7f6be40ba7c0 2024-05-08T05:48:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.917+0000 7f6bebf19640 1 -- 172.21.15.45:0/1926904685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be40693f0 con 0x7f6be40ba7c0 2024-05-08T05:48:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.918+0000 7f6bda7fc640 1 -- 172.21.15.45:0/1926904685 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6bdc0039d0 con 0x7f6be40ba7c0 2024-05-08T05:48:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.918+0000 7f6bda7fc640 1 --2- 172.21.15.45:0/1926904685 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6bc80762d0 0x7f6bc8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.919+0000 7f6bda7fc640 1 -- 172.21.15.45:0/1926904685 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7f6bdc097990 con 0x7f6be40ba7c0 2024-05-08T05:48:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.919+0000 7f6be9cb5640 1 --2- 172.21.15.45:0/1926904685 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6bc80762d0 0x7f6bc8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:22.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.919+0000 7f6be9cb5640 1 --2- 172.21.15.45:0/1926904685 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6bc80762d0 0x7f6bc8078790 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6be0002000 tx=0x7f6be0009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:22.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:22.923+0000 7f6bda7fc640 1 -- 172.21.15.45:0/1926904685 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6bdc026020 con 0x7f6be40ba7c0 2024-05-08T05:48:23.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:23.160+0000 7f6bebf19640 1 -- 172.21.15.45:0/1926904685 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6be40b4c50 con 0x7f6be40ba7c0 2024-05-08T05:48:23.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:23.160+0000 7f6bda7fc640 1 -- 172.21.15.45:0/1926904685 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5178 (secure 0 0 0) 0x7f6bdc061040 con 0x7f6be40ba7c0 2024-05-08T05:48:23.161 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:48:23.161 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2616298287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2616298287},{"type":"v1","addr":"172.21.15.63:6836","nonce":2616298287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6472,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1347752311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1347752311},{"type":"v1","addr":"172.21.15.63:6837","nonce":1347752311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6744,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2854804427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2854804427},{"type":"v1","addr":"172.21.15.45:6834","nonce":2854804427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:47:00.211837+0000","modified":"2024-05-08T05:48:22.019369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6747},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6747":{"gid":6747,"name":"c","rank":0,"incarnation":70,"state":"up:rejoin","state_seq":24,"addr":"172.21.15.45:6835/2945763952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2945763952},{"type":"v1","addr":"172.21.15.45:6835","nonce":2945763952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-05-08T05:48:23.161 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 72 2024-05-08T05:48:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:23.162+0000 7f6bebf19640 1 -- 172.21.15.45:0/1926904685 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6bc80762d0 msgr2=0x7f6bc8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:23.162+0000 7f6bebf19640 1 --2- 172.21.15.45:0/1926904685 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6bc80762d0 0x7f6bc8078790 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6be0002000 tx=0x7f6be0009030 comp rx=0 tx=0).stop 2024-05-08T05:48:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:23.162+0000 7f6bebf19640 1 -- 172.21.15.45:0/1926904685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6be40ba7c0 msgr2=0x7f6be40c9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:23.162+0000 7f6bebf19640 1 --2- 172.21.15.45:0/1926904685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6be40ba7c0 0x7f6be40c9b50 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6be40b6a30 tx=0x7f6bdc011a80 comp rx=0 tx=0).stop 2024-05-08T05:48:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:23.162+0000 7f6bebf19640 1 -- 172.21.15.45:0/1926904685 shutdown_connections 2024-05-08T05:48:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:23.162+0000 7f6bebf19640 1 --2- 172.21.15.45:0/1926904685 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6bc80762d0 0x7f6bc8078790 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:23.162+0000 7f6bebf19640 1 --2- 172.21.15.45:0/1926904685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6be40ba7c0 0x7f6be40c9b50 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:23.162+0000 7f6bebf19640 1 --2- 172.21.15.45:0/1926904685 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6be41f9480 0x7f6be40ba280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:23.162+0000 7f6bebf19640 1 --2- 172.21.15.45:0/1926904685 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6bd0003c10 0x7f6be40b9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:23.162+0000 7f6bebf19640 1 -- 172.21.15.45:0/1926904685 >> 172.21.15.45:0/1926904685 conn(0x7f6be408eac0 msgr2=0x7f6be40cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:23.162+0000 7f6bebf19640 1 -- 172.21.15.45:0/1926904685 shutdown_connections 2024-05-08T05:48:23.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:23.162+0000 7f6bebf19640 1 -- 172.21.15.45:0/1926904685 wait complete. 2024-05-08T05:48:23.576 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:47:00.211837+0000', 'modified': '2024-05-08T05:48:22.019369+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 92, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6747}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6747': {'gid': 6747, 'name': 'c', 'rank': 0, 'incarnation': 70, 'state': 'up:rejoin', 'state_seq': 24, 'addr': '172.21.15.45:6835/2945763952', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 2945763952}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 2945763952}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-08T05:48:23.576 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6747:up:rejoin 2024-05-08T05:48:24.577 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:48:24.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.693+0000 7ff0c7659640 1 Processor -- start 2024-05-08T05:48:24.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.694+0000 7ff0c7659640 1 -- start start 2024-05-08T05:48:24.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.694+0000 7ff0c7659640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff0c015f420 0x7ff0c00d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:24.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.694+0000 7ff0c7659640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0c00d2080 0x7ff0c00d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:24.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.694+0000 7ff0c7659640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0c01630b0 con 0x7ff0c00d3ca0 2024-05-08T05:48:24.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.694+0000 7ff0c7659640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff0c00d2550 con 0x7ff0c015f420 2024-05-08T05:48:24.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.694+0000 7ff0c7659640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff0c00d2c80 con 0x7ff0c00d2080 2024-05-08T05:48:24.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.695+0000 7ff0c53f5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff0c015f420 0x7ff0c00d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:24.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.695+0000 7ff0c53f5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff0c015f420 0x7ff0c00d3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43332/0 (socket says 172.21.15.45:43332) 2024-05-08T05:48:24.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.695+0000 7ff0c53f5640 1 -- 172.21.15.45:0/4020379394 learned_addr learned my addr 172.21.15.45:0/4020379394 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:24.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.695+0000 7ff0c5bf6640 1 --2- 172.21.15.45:0/4020379394 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0c00d2080 0x7ff0c00d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:24.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.695+0000 7ff0c53f5640 1 -- 172.21.15.45:0/4020379394 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0c00d2080 msgr2=0x7ff0c00d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:24.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.695+0000 7ff0c53f5640 1 --2- 172.21.15.45:0/4020379394 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0c00d2080 0x7ff0c00d4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:24.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.695+0000 7ff0c53f5640 1 -- 172.21.15.45:0/4020379394 >> v1:172.21.15.63:6789/0 conn(0x7ff0c00d3ca0 legacy=0x7ff0c00d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:24.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.695+0000 7ff0c53f5640 1 -- 172.21.15.45:0/4020379394 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c00d2f60 con 0x7ff0c015f420 2024-05-08T05:48:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.696+0000 7ff0c53f5640 1 --2- 172.21.15.45:0/4020379394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0c015f420 0x7ff0c00d3760 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009e50 tx=0x7ff0b002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61106f3377a8e98c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.696+0000 7ff0bffff640 1 -- 172.21.15.45:0/4020379394 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0036070 con 0x7ff0c015f420 2024-05-08T05:48:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.696+0000 7ff0bffff640 1 -- 172.21.15.45:0/4020379394 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0b0031600 con 0x7ff0c015f420 2024-05-08T05:48:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.696+0000 7ff0bffff640 1 -- 172.21.15.45:0/4020379394 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b00319d0 con 0x7ff0c015f420 2024-05-08T05:48:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.696+0000 7ff0c7659640 1 -- 172.21.15.45:0/4020379394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0c015f420 msgr2=0x7ff0c00d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.696+0000 7ff0c7659640 1 --2- 172.21.15.45:0/4020379394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0c015f420 0x7ff0c00d3760 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009e50 tx=0x7ff0b002a710 comp rx=0 tx=0).stop 2024-05-08T05:48:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.697+0000 7ff0c7659640 1 -- 172.21.15.45:0/4020379394 shutdown_connections 2024-05-08T05:48:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.697+0000 7ff0c7659640 1 --2- 172.21.15.45:0/4020379394 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0c00d2080 0x7ff0c00d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:24.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.697+0000 7ff0c7659640 1 --2- 172.21.15.45:0/4020379394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0c015f420 0x7ff0c00d3760 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:24.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.697+0000 7ff0c7659640 1 -- 172.21.15.45:0/4020379394 >> 172.21.15.45:0/4020379394 conn(0x7ff0c008eac0 msgr2=0x7ff0c009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:24.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.697+0000 7ff0c7659640 1 -- 172.21.15.45:0/4020379394 shutdown_connections 2024-05-08T05:48:24.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.697+0000 7ff0c7659640 1 -- 172.21.15.45:0/4020379394 wait complete. 2024-05-08T05:48:24.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.698+0000 7ff0c7659640 1 Processor -- start 2024-05-08T05:48:24.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.698+0000 7ff0c7659640 1 -- start start 2024-05-08T05:48:24.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.699+0000 7ff0c7659640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0c00d2080 0x7ff0c00cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:24.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.699+0000 7ff0c7659640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0c015dde0 0x7ff0c01ea880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:24.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.699+0000 7ff0c7659640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0c015d6c0 0x7ff0c006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:24.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.699+0000 7ff0c7659640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c00d5200 con 0x7ff0c015d6c0 2024-05-08T05:48:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.699+0000 7ff0c7659640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c00d2830 con 0x7ff0c015dde0 2024-05-08T05:48:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.699+0000 7ff0c7659640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff0c00d5380 con 0x7ff0c00d2080 2024-05-08T05:48:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.699+0000 7ff0c5bf6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0c015d6c0 0x7ff0c006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.699+0000 7ff0c5bf6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0c015d6c0 0x7ff0c006e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43346/0 (socket says 172.21.15.45:43346) 2024-05-08T05:48:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.699+0000 7ff0c5bf6640 1 -- 172.21.15.45:0/2753887010 learned_addr learned my addr 172.21.15.45:0/2753887010 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.699+0000 7ff0c53f5640 1 --2- 172.21.15.45:0/2753887010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0c00d2080 0x7ff0c00cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.699+0000 7ff0c4bf4640 1 --2- 172.21.15.45:0/2753887010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0c015dde0 0x7ff0c01ea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.699+0000 7ff0c5bf6640 1 -- 172.21.15.45:0/2753887010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0c00d2080 msgr2=0x7ff0c00cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.699+0000 7ff0c5bf6640 1 --2- 172.21.15.45:0/2753887010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0c00d2080 0x7ff0c00cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:24.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.699+0000 7ff0c5bf6640 1 -- 172.21.15.45:0/2753887010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0c015dde0 msgr2=0x7ff0c01ea880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:24.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.699+0000 7ff0c5bf6640 1 --2- 172.21.15.45:0/2753887010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0c015dde0 0x7ff0c01ea880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:24.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.699+0000 7ff0c5bf6640 1 -- 172.21.15.45:0/2753887010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c01e97e0 con 0x7ff0c015d6c0 2024-05-08T05:48:24.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.699+0000 7ff0c4bf4640 1 --2- 172.21.15.45:0/2753887010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0c015dde0 0x7ff0c01ea880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:48:24.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.699+0000 7ff0c53f5640 1 --2- 172.21.15.45:0/2753887010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0c00d2080 0x7ff0c00cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:48:24.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.699+0000 7ff0c5bf6640 1 --2- 172.21.15.45:0/2753887010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0c015d6c0 0x7ff0c006e090 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800d830 tx=0x7ff0b800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:24.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.700+0000 7ff0bdffb640 1 -- 172.21.15.45:0/2753887010 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b800b6c0 con 0x7ff0c015d6c0 2024-05-08T05:48:24.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.700+0000 7ff0bdffb640 1 -- 172.21.15.45:0/2753887010 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0b8019070 con 0x7ff0c015d6c0 2024-05-08T05:48:24.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.700+0000 7ff0bdffb640 1 -- 172.21.15.45:0/2753887010 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b800bac0 con 0x7ff0c015d6c0 2024-05-08T05:48:24.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.700+0000 7ff0c7659640 1 -- 172.21.15.45:0/2753887010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0c01e9ac0 con 0x7ff0c015d6c0 2024-05-08T05:48:24.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.700+0000 7ff0c7659640 1 -- 172.21.15.45:0/2753887010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0c01ea080 con 0x7ff0c015d6c0 2024-05-08T05:48:24.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.701+0000 7ff0c7659640 1 -- 172.21.15.45:0/2753887010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0c00693f0 con 0x7ff0c015d6c0 2024-05-08T05:48:24.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.701+0000 7ff0bdffb640 1 -- 172.21.15.45:0/2753887010 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff0b801e460 con 0x7ff0c015d6c0 2024-05-08T05:48:24.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.702+0000 7ff0bdffb640 1 --2- 172.21.15.45:0/2753887010 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff0ac076300 0x7ff0ac0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:24.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.702+0000 7ff0bdffb640 1 -- 172.21.15.45:0/2753887010 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7ff0b8097ec0 con 0x7ff0c015d6c0 2024-05-08T05:48:24.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.702+0000 7ff0c53f5640 1 --2- 172.21.15.45:0/2753887010 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff0ac076300 0x7ff0ac0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:24.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.702+0000 7ff0c53f5640 1 --2- 172.21.15.45:0/2753887010 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff0ac076300 0x7ff0ac0787c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0033000 tx=0x7ff0b0009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:24.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.706+0000 7ff0bdffb640 1 -- 172.21.15.45:0/2753887010 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff0b8061570 con 0x7ff0c015d6c0 2024-05-08T05:48:24.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.935+0000 7ff0c7659640 1 -- 172.21.15.45:0/2753887010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0c00b4c50 con 0x7ff0c015d6c0 2024-05-08T05:48:24.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.936+0000 7ff0bdffb640 1 -- 172.21.15.45:0/2753887010 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5185 (secure 0 0 0) 0x7ff0b8060e70 con 0x7ff0c015d6c0 2024-05-08T05:48:24.936 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:48:24.936 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2616298287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2616298287},{"type":"v1","addr":"172.21.15.63:6836","nonce":2616298287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6472,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1347752311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1347752311},{"type":"v1","addr":"172.21.15.63:6837","nonce":1347752311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6744,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2854804427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2854804427},{"type":"v1","addr":"172.21.15.45:6834","nonce":2854804427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:47:00.211837+0000","modified":"2024-05-08T05:48:24.018244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6747},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6747":{"gid":6747,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":25,"addr":"172.21.15.45:6835/2945763952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2945763952},{"type":"v1","addr":"172.21.15.45:6835","nonce":2945763952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6747,"qdb_cluster":[6747]},"id":6}]} 2024-05-08T05:48:24.936 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 73 2024-05-08T05:48:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.938+0000 7ff0c7659640 1 -- 172.21.15.45:0/2753887010 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff0ac076300 msgr2=0x7ff0ac0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.938+0000 7ff0c7659640 1 --2- 172.21.15.45:0/2753887010 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff0ac076300 0x7ff0ac0787c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0033000 tx=0x7ff0b0009000 comp rx=0 tx=0).stop 2024-05-08T05:48:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.938+0000 7ff0c7659640 1 -- 172.21.15.45:0/2753887010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0c015d6c0 msgr2=0x7ff0c006e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.938+0000 7ff0c7659640 1 --2- 172.21.15.45:0/2753887010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0c015d6c0 0x7ff0c006e090 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800d830 tx=0x7ff0b800dcf0 comp rx=0 tx=0).stop 2024-05-08T05:48:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.938+0000 7ff0c7659640 1 -- 172.21.15.45:0/2753887010 shutdown_connections 2024-05-08T05:48:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.938+0000 7ff0c7659640 1 --2- 172.21.15.45:0/2753887010 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff0ac076300 0x7ff0ac0787c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.938+0000 7ff0c7659640 1 --2- 172.21.15.45:0/2753887010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0c015d6c0 0x7ff0c006e090 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:24.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.938+0000 7ff0c7659640 1 --2- 172.21.15.45:0/2753887010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0c015dde0 0x7ff0c01ea880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.938+0000 7ff0c7659640 1 --2- 172.21.15.45:0/2753887010 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff0c00d2080 0x7ff0c00cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.938+0000 7ff0c7659640 1 -- 172.21.15.45:0/2753887010 >> 172.21.15.45:0/2753887010 conn(0x7ff0c008eac0 msgr2=0x7ff0c006cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.938+0000 7ff0c7659640 1 -- 172.21.15.45:0/2753887010 shutdown_connections 2024-05-08T05:48:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:24.938+0000 7ff0c7659640 1 -- 172.21.15.45:0/2753887010 wait complete. 2024-05-08T05:48:25.351 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:47:00.211837+0000', 'modified': '2024-05-08T05:48:24.018244+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 92, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6747}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6747': {'gid': 6747, 'name': 'c', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.45:6835/2945763952', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6833', 'nonce': 2945763952}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 2945763952}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6747, 'qdb_cluster': [6747]} 2024-05-08T05:48:25.351 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T05:48:25.351 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T05:48:25.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.460+0000 7f5bad446640 1 Processor -- start 2024-05-08T05:48:25.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.460+0000 7f5bad446640 1 -- start start 2024-05-08T05:48:25.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.460+0000 7f5bad446640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ba8070d20 0x7f5ba80711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.460+0000 7f5bad446640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ba806e110 con 0x7f5ba80b2490 2024-05-08T05:48:25.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.460+0000 7f5bad446640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ba806e3f0 con 0x7f5ba81629b0 2024-05-08T05:48:25.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.460+0000 7f5bad446640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba8070a40 con 0x7f5ba8070d20 2024-05-08T05:48:25.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.460+0000 7f5ba67fc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5ba80b2490 0x7f5ba8070330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48562/0 (socket says 172.21.15.45:48562) 2024-05-08T05:48:25.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.461+0000 7f5ba67fc640 1 -- 172.21.15.45:0/649346440 learned_addr learned my addr 172.21.15.45:0/649346440 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:48:25.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.461+0000 7f5ba77fe640 1 --2- 172.21.15.45:0/649346440 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ba8070d20 0x7f5ba80711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.461+0000 7f5ba5ffb640 1 -- 172.21.15.45:0/649346440 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4217241614 0 0) 0x7f5ba806e110 con 0x7f5ba80b2490 2024-05-08T05:48:25.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.461+0000 7f5ba5ffb640 1 -- 172.21.15.45:0/649346440 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b94003520 con 0x7f5ba80b2490 2024-05-08T05:48:25.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.461+0000 7f5ba5ffb640 1 -- 172.21.15.45:0/649346440 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 944491811 0 0) 0x7f5b94003520 con 0x7f5ba80b2490 2024-05-08T05:48:25.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.461+0000 7f5ba5ffb640 1 -- 172.21.15.45:0/649346440 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ba8070d20 msgr2=0x7f5ba80711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:25.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.461+0000 7f5ba5ffb640 1 --2- 172.21.15.45:0/649346440 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ba8070d20 0x7f5ba80711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.461+0000 7f5ba5ffb640 1 -- 172.21.15.45:0/649346440 >> v1:172.21.15.63:6789/0 conn(0x7f5ba81629b0 legacy=0x7f5ba806eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:25.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.461+0000 7f5ba5ffb640 1 -- 172.21.15.45:0/649346440 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba806e6d0 con 0x7f5ba80b2490 2024-05-08T05:48:25.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.461+0000 7f5ba5ffb640 1 -- 172.21.15.45:0/649346440 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f5b900016f0 con 0x7f5ba80b2490 2024-05-08T05:48:25.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.461+0000 7f5ba5ffb640 1 -- 172.21.15.45:0/649346440 >> v1:172.21.15.45:6789/0 conn(0x7f5ba80b2490 legacy=0x7f5ba8070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:25.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.461+0000 7f5ba5ffb640 1 --2- 172.21.15.45:0/649346440 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ba8070d20 0x7f5b94003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.461+0000 7f5ba5ffb640 1 --2- 172.21.15.45:0/649346440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b94004000 0x7f5b94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.461+0000 7f5ba5ffb640 1 --2- 172.21.15.45:0/649346440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b94004d40 0x7f5b940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.461+0000 7f5ba5ffb640 1 -- 172.21.15.45:0/649346440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba8070a40 con 0x7f5b94004000 2024-05-08T05:48:25.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.461+0000 7f5ba5ffb640 1 -- 172.21.15.45:0/649346440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9400bb50 con 0x7f5b94004d40 2024-05-08T05:48:25.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.461+0000 7f5ba5ffb640 1 -- 172.21.15.45:0/649346440 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5b9400bd50 con 0x7f5ba8070d20 2024-05-08T05:48:25.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.461+0000 7f5ba67fc640 1 --2- 172.21.15.45:0/649346440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b94004000 0x7f5b94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.461+0000 7f5ba6ffd640 1 --2- 172.21.15.45:0/649346440 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ba8070d20 0x7f5b94003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.461+0000 7f5ba77fe640 1 --2- 172.21.15.45:0/649346440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b94004d40 0x7f5b940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.462+0000 7f5ba67fc640 1 -- 172.21.15.45:0/649346440 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ba8070d20 msgr2=0x7f5b94003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:25.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.462+0000 7f5ba67fc640 1 --2- 172.21.15.45:0/649346440 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ba8070d20 0x7f5b94003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.462+0000 7f5ba67fc640 1 -- 172.21.15.45:0/649346440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b94004d40 msgr2=0x7f5b940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:25.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.462+0000 7f5ba67fc640 1 --2- 172.21.15.45:0/649346440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b94004d40 0x7f5b940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.462+0000 7f5ba67fc640 1 -- 172.21.15.45:0/649346440 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b94003520 con 0x7f5b94004000 2024-05-08T05:48:25.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.462+0000 7f5ba77fe640 1 --2- 172.21.15.45:0/649346440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5b94004d40 0x7f5b940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:25.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.462+0000 7f5ba67fc640 1 --2- 172.21.15.45:0/649346440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b94004000 0x7f5b94004800 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000c2f0 tx=0x7f5b94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ad1af82ed969cb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:25.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.462+0000 7f5ba5ffb640 1 -- 172.21.15.45:0/649346440 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b90007d90 con 0x7f5b94004000 2024-05-08T05:48:25.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.463+0000 7f5ba5ffb640 1 -- 172.21.15.45:0/649346440 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b90005af0 con 0x7f5b94004000 2024-05-08T05:48:25.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.463+0000 7f5ba5ffb640 1 -- 172.21.15.45:0/649346440 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b90007070 con 0x7f5b94004000 2024-05-08T05:48:25.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.463+0000 7f5bad446640 1 -- 172.21.15.45:0/649346440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b94004000 msgr2=0x7f5b94004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:25.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.463+0000 7f5bad446640 1 --2- 172.21.15.45:0/649346440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b94004000 0x7f5b94004800 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000c2f0 tx=0x7f5b94005830 comp rx=0 tx=0).stop 2024-05-08T05:48:25.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.463+0000 7f5ba77fe640 1 -- 172.21.15.45:0/649346440 reap_dead start 2024-05-08T05:48:25.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.463+0000 7f5bad446640 1 -- 172.21.15.45:0/649346440 shutdown_connections 2024-05-08T05:48:25.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.463+0000 7f5bad446640 1 -- 172.21.15.45:0/649346440 >> 172.21.15.45:0/649346440 conn(0x7f5ba80b49d0 msgr2=0x7f5ba80b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:25.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.463+0000 7f5bad446640 1 -- 172.21.15.45:0/649346440 shutdown_connections 2024-05-08T05:48:25.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.463+0000 7f5bad446640 1 -- 172.21.15.45:0/649346440 wait complete. 2024-05-08T05:48:25.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.464+0000 7f5bad446640 1 Processor -- start 2024-05-08T05:48:25.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.465+0000 7f5bad446640 1 -- start start 2024-05-08T05:48:25.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.465+0000 7f5bad446640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b94004000 0x7f5ba814f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.465+0000 7f5bad446640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ba80c76d0 0x7f5ba80b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.465+0000 7f5bad446640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ba80c7b50 0x7f5ba80b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.465+0000 7f5bad446640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba80711d0 con 0x7f5b94004000 2024-05-08T05:48:25.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.465+0000 7f5bad446640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba806dec0 con 0x7f5ba80c7b50 2024-05-08T05:48:25.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.465+0000 7f5bad446640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba8071350 con 0x7f5ba80c76d0 2024-05-08T05:48:25.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.465+0000 7f5ba6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b94004000 0x7f5ba814f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.465+0000 7f5ba6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b94004000 0x7f5ba814f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43362/0 (socket says 172.21.15.45:43362) 2024-05-08T05:48:25.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.465+0000 7f5ba6ffd640 1 -- 172.21.15.45:0/1076291255 learned_addr learned my addr 172.21.15.45:0/1076291255 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:25.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.466+0000 7f5ba6ffd640 1 -- 172.21.15.45:0/1076291255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ba80c76d0 msgr2=0x7f5ba80b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:25.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.466+0000 7f5ba6ffd640 1 --2- 172.21.15.45:0/1076291255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ba80c76d0 0x7f5ba80b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.466+0000 7f5ba6ffd640 1 -- 172.21.15.45:0/1076291255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ba80c7b50 msgr2=0x7f5ba80b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:25.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.466+0000 7f5ba6ffd640 1 --2- 172.21.15.45:0/1076291255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ba80c7b50 0x7f5ba80b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.466+0000 7f5ba6ffd640 1 -- 172.21.15.45:0/1076291255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba80c6930 con 0x7f5b94004000 2024-05-08T05:48:25.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.466+0000 7f5ba6ffd640 1 --2- 172.21.15.45:0/1076291255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b94004000 0x7f5ba814f2f0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000fb90 tx=0x7f5ba000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:25.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.466+0000 7f5b9f7fe640 1 -- 172.21.15.45:0/1076291255 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba000fe90 con 0x7f5b94004000 2024-05-08T05:48:25.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.466+0000 7f5b9f7fe640 1 -- 172.21.15.45:0/1076291255 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba0018070 con 0x7f5b94004000 2024-05-08T05:48:25.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.466+0000 7f5bad446640 1 -- 172.21.15.45:0/1076291255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba814f830 con 0x7f5b94004000 2024-05-08T05:48:25.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.466+0000 7f5b9f7fe640 1 -- 172.21.15.45:0/1076291255 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba000ddf0 con 0x7f5b94004000 2024-05-08T05:48:25.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.466+0000 7f5bad446640 1 -- 172.21.15.45:0/1076291255 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba80c6bf0 con 0x7f5b94004000 2024-05-08T05:48:25.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.468+0000 7f5b9f7fe640 1 -- 172.21.15.45:0/1076291255 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5ba0003970 con 0x7f5b94004000 2024-05-08T05:48:25.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.469+0000 7f5b9f7fe640 1 --2- 172.21.15.45:0/1076291255 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5b880764a0 0x7f5b88078960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.469+0000 7f5b9f7fe640 1 -- 172.21.15.45:0/1076291255 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7f5ba0097a10 con 0x7f5b94004000 2024-05-08T05:48:25.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.469+0000 7f5ba67fc640 1 --2- 172.21.15.45:0/1076291255 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5b880764a0 0x7f5b88078960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.469+0000 7f5ba67fc640 1 --2- 172.21.15.45:0/1076291255 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5b880764a0 0x7f5b88078960 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000c2f0 tx=0x7f5b90007000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:25.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.469+0000 7f5bb0233740 1 Processor -- start 2024-05-08T05:48:25.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.470+0000 7f5bb0233740 1 -- start start 2024-05-08T05:48:25.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.470+0000 7f5bb0233740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d18f50 0x558cc4d1b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.470+0000 7f5bb0233740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d19330 0x558cc4d1bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.470+0000 7f5bb0233740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4cebbb0 0x558cc4cef3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.470+0000 7f5bb0233740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558cc4c558d0 con 0x558cc4cebbb0 2024-05-08T05:48:25.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.470+0000 7f5bb0233740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558cc4c0ec10 con 0x558cc4d19330 2024-05-08T05:48:25.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.470+0000 7f5bb0233740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x558cc4c51890 con 0x558cc4d18f50 2024-05-08T05:48:25.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.470+0000 7f5ba77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4cebbb0 0x558cc4cef3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.470+0000 7f5ba77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4cebbb0 0x558cc4cef3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43364/0 (socket says 172.21.15.45:43364) 2024-05-08T05:48:25.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.470+0000 7f5ba77fe640 1 -- 172.21.15.45:0/726044197 learned_addr learned my addr 172.21.15.45:0/726044197 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:25.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.471+0000 7f5ba67fc640 1 --2- 172.21.15.45:0/726044197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d19330 0x558cc4d1bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.471+0000 7f5ba6ffd640 1 --2- 172.21.15.45:0/726044197 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d18f50 0x558cc4d1b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.471+0000 7f5ba77fe640 1 -- 172.21.15.45:0/726044197 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d18f50 msgr2=0x558cc4d1b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.471+0000 7f5ba77fe640 1 --2- 172.21.15.45:0/726044197 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d18f50 0x558cc4d1b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.471+0000 7f5ba77fe640 1 -- 172.21.15.45:0/726044197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d19330 msgr2=0x558cc4d1bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.471+0000 7f5ba77fe640 1 --2- 172.21.15.45:0/726044197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d19330 0x558cc4d1bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.471+0000 7f5ba77fe640 1 -- 172.21.15.45:0/726044197 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cc4c165d0 con 0x558cc4cebbb0 2024-05-08T05:48:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.471+0000 7f5ba67fc640 1 --2- 172.21.15.45:0/726044197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d19330 0x558cc4d1bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.471+0000 7f5ba77fe640 1 --2- 172.21.15.45:0/726044197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4cebbb0 0x558cc4cef3c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80702c0 tx=0x7f5b98009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4292159c019621 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.471+0000 7f5b9cff9640 1 -- 172.21.15.45:0/726044197 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b98019020 con 0x558cc4cebbb0 2024-05-08T05:48:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.471+0000 7f5b9cff9640 1 -- 172.21.15.45:0/726044197 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b98004230 con 0x558cc4cebbb0 2024-05-08T05:48:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.471+0000 7f5b9cff9640 1 -- 172.21.15.45:0/726044197 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b98011730 con 0x558cc4cebbb0 2024-05-08T05:48:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.472+0000 7f5bb0233740 1 -- 172.21.15.45:0/726044197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4cebbb0 msgr2=0x558cc4cef3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.472+0000 7f5bb0233740 1 --2- 172.21.15.45:0/726044197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4cebbb0 0x558cc4cef3c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80702c0 tx=0x7f5b98009cb0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.472+0000 7f5bb0233740 1 -- 172.21.15.45:0/726044197 shutdown_connections 2024-05-08T05:48:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.472+0000 7f5bb0233740 1 --2- 172.21.15.45:0/726044197 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4cebbb0 0x558cc4cef3c0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.472+0000 7f5bb0233740 1 --2- 172.21.15.45:0/726044197 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d19330 0x558cc4d1bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.472+0000 7f5bb0233740 1 --2- 172.21.15.45:0/726044197 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d18f50 0x558cc4d1b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.472+0000 7f5bb0233740 1 -- 172.21.15.45:0/726044197 >> 172.21.15.45:0/726044197 conn(0x558cc4cede20 msgr2=0x558cc4d36760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.472+0000 7f5bb0233740 1 -- 172.21.15.45:0/726044197 shutdown_connections 2024-05-08T05:48:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.472+0000 7f5bb0233740 1 -- 172.21.15.45:0/726044197 wait complete. 2024-05-08T05:48:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.472+0000 7f5bb0233740 1 -- start start 2024-05-08T05:48:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.472+0000 7f5bb0233740 1 Processor -- start 2024-05-08T05:48:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.473+0000 7f5bb0233740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d18f50 0x558cc4d47740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.473+0000 7f5bb0233740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d19330 0x558cc4d1e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.473+0000 7f5bb0233740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d388e0 0x558cc4d1c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.473+0000 7f5bb0233740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558cc4cf0fd0 con 0x558cc4d388e0 2024-05-08T05:48:25.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.473+0000 7f5bb0233740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558cc4c12760 con 0x558cc4d19330 2024-05-08T05:48:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.473+0000 7f5bb0233740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x558cc4cf12b0 con 0x558cc4d18f50 2024-05-08T05:48:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.473+0000 7f5bb0233740 10 client.0 _ll_register_callbacks cb 0x558cc4c4bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:48:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.473+0000 7f5ba77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d388e0 0x558cc4d1c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.473+0000 7f5ba77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d388e0 0x558cc4d1c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43378/0 (socket says 172.21.15.45:43378) 2024-05-08T05:48:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.473+0000 7f5ba77fe640 1 -- 172.21.15.45:0/194047990 learned_addr learned my addr 172.21.15.45:0/194047990 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5ba6ffd640 1 --2- 172.21.15.45:0/194047990 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d18f50 0x558cc4d47740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5ba67fc640 1 --2- 172.21.15.45:0/194047990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d19330 0x558cc4d1e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5ba77fe640 1 -- 172.21.15.45:0/194047990 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d18f50 msgr2=0x558cc4d47740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5ba77fe640 1 --2- 172.21.15.45:0/194047990 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d18f50 0x558cc4d47740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5ba77fe640 1 -- 172.21.15.45:0/194047990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d19330 msgr2=0x558cc4d1e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5ba77fe640 1 --2- 172.21.15.45:0/194047990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d19330 0x558cc4d1e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5ba77fe640 1 -- 172.21.15.45:0/194047990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cc49672d0 con 0x558cc4d388e0 2024-05-08T05:48:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5ba77fe640 1 --2- 172.21.15.45:0/194047990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d388e0 0x558cc4d1c290 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8073290 tx=0x7f5b98009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46bf35564bbdd79 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5b85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:48:25.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5b85ffb640 1 -- 172.21.15.45:0/194047990 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9802c060 con 0x558cc4d388e0 2024-05-08T05:48:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5b85ffb640 1 -- 172.21.15.45:0/194047990 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7f5b98015460 con 0x558cc4d388e0 2024-05-08T05:48:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5bb0233740 1 -- 172.21.15.45:0/194047990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b98004230 con 0x558cc4d388e0 2024-05-08T05:48:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5b85ffb640 1 -- 172.21.15.45:0/194047990 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f5b98032050 con 0x558cc4d388e0 2024-05-08T05:48:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5bb0233740 10 client.7263 fetch_fsmap learned FSMap version 73 2024-05-08T05:48:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5bb0233740 1 -- 172.21.15.45:0/194047990 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x558cc4a316b0 con 0x558cc4d388e0 2024-05-08T05:48:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5b85ffb640 1 -- 172.21.15.45:0/194047990 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b98004230 con 0x558cc4d388e0 2024-05-08T05:48:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5b85ffb640 1 -- 172.21.15.45:0/194047990 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7f5b98035020 con 0x558cc4d388e0 2024-05-08T05:48:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5bb0233740 10 client.7263 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T05:48:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5bb0233740 10 client.7263 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T05:48:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5bb0233740 20 client.7263 populate_metadata read hostname 'smithi045' 2024-05-08T05:48:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.474+0000 7f5bb0233740 1 --2- 172.21.15.45:0/194047990 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x558cc4d01450 0x558cc4d55440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.475+0000 7f5bb0233740 4 client.7263 mds_command: new command op to 6747 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:48:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.475+0000 7f5bb0233740 1 -- 172.21.15.45:0/194047990 --> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cc4bd9760 con 0x558cc4d01450 2024-05-08T05:48:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.475+0000 7f5ba6ffd640 1 --2- 172.21.15.45:0/194047990 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x558cc4d01450 0x558cc4d55440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.475+0000 7f5ba6ffd640 1 --2- 172.21.15.45:0/194047990 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x558cc4d01450 0x558cc4d55440 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ea1ca8b656bb8f6 server_cookie=889c82e5943b9e83 in_seq=0 out_seq=0 2024-05-08T05:48:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.475+0000 7f5b85ffb640 10 client.7263 ms_handle_connect on v2:172.21.15.45:6833/2945763952 2024-05-08T05:48:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.476+0000 7f5b85ffb640 1 -- 172.21.15.45:0/194047990 <== mds.0 v2:172.21.15.45:6833/2945763952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558cc4bd9760 con 0x558cc4d01450 2024-05-08T05:48:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.476+0000 7f5b85ffb640 10 client.7263 handle_command_reply: tid=0 2024-05-08T05:48:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.476+0000 7f5bb0233740 1 client.7263 shutdown 2024-05-08T05:48:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.476+0000 7f5bb0233740 1 -- 172.21.15.45:0/194047990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d388e0 msgr2=0x558cc4d1c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.476+0000 7f5bb0233740 1 --2- 172.21.15.45:0/194047990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d388e0 0x558cc4d1c290 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8073290 tx=0x7f5b98009190 comp rx=0 tx=0).stop 2024-05-08T05:48:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.476+0000 7f5bb0233740 1 -- 172.21.15.45:0/194047990 shutdown_connections 2024-05-08T05:48:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.476+0000 7f5bb0233740 1 --2- 172.21.15.45:0/194047990 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x558cc4d01450 0x558cc4d55440 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.476+0000 7f5bb0233740 1 --2- 172.21.15.45:0/194047990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d388e0 0x558cc4d1c290 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.476+0000 7f5bb0233740 1 --2- 172.21.15.45:0/194047990 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d19330 0x558cc4d1e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.476+0000 7f5bb0233740 1 --2- 172.21.15.45:0/194047990 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d18f50 0x558cc4d47740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.476+0000 7f5b85ffb640 0 client.7263 ms_handle_reset on v2:172.21.15.45:6833/2945763952 2024-05-08T05:48:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.476+0000 7f5bb0233740 1 -- 172.21.15.45:0/194047990 >> 172.21.15.45:0/194047990 conn(0x558cc4cebbb0 msgr2=0x558cc4d1f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:25.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.476+0000 7f5bb0233740 1 -- 172.21.15.45:0/194047990 shutdown_connections 2024-05-08T05:48:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.476+0000 7f5bb0233740 1 -- 172.21.15.45:0/194047990 wait complete. 2024-05-08T05:48:25.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.476+0000 7f5bb0233740 20 client.7263 trim_cache size 0 max 16384 2024-05-08T05:48:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.495+0000 7f5bb0233740 1 Processor -- start 2024-05-08T05:48:25.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.495+0000 7f5bb0233740 1 -- start start 2024-05-08T05:48:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.495+0000 7f5bb0233740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d18f50 0x558cc4d36630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.496+0000 7f5bb0233740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d19330 0x558cc4cec770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.496+0000 7f5bb0233740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d388e0 0x558cc4cf1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.496+0000 7f5bb0233740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba8201a90 con 0x558cc4d388e0 2024-05-08T05:48:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.496+0000 7f5bb0233740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba8200dc0 con 0x558cc4d18f50 2024-05-08T05:48:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.496+0000 7f5bb0233740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba82013e0 con 0x558cc4d19330 2024-05-08T05:48:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.496+0000 7f5ba77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d388e0 0x558cc4cf1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.496+0000 7f5ba77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d388e0 0x558cc4cf1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43382/0 (socket says 172.21.15.45:43382) 2024-05-08T05:48:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.496+0000 7f5ba77fe640 1 -- 172.21.15.45:0/2587289781 learned_addr learned my addr 172.21.15.45:0/2587289781 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.496+0000 7f5ba6ffd640 1 --2- 172.21.15.45:0/2587289781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d18f50 0x558cc4d36630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.496+0000 7f5ba67fc640 1 --2- 172.21.15.45:0/2587289781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d19330 0x558cc4cec770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.496+0000 7f5ba77fe640 1 -- 172.21.15.45:0/2587289781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d19330 msgr2=0x558cc4cec770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.496+0000 7f5ba77fe640 1 --2- 172.21.15.45:0/2587289781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d19330 0x558cc4cec770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.496+0000 7f5ba77fe640 1 -- 172.21.15.45:0/2587289781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d18f50 msgr2=0x558cc4d36630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.496+0000 7f5ba77fe640 1 --2- 172.21.15.45:0/2587289781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d18f50 0x558cc4d36630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.496+0000 7f5ba77fe640 1 -- 172.21.15.45:0/2587289781 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cc4d30de0 con 0x558cc4d388e0 2024-05-08T05:48:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.496+0000 7f5ba67fc640 1 --2- 172.21.15.45:0/2587289781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d19330 0x558cc4cec770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:48:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.496+0000 7f5ba77fe640 1 --2- 172.21.15.45:0/2587289781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d388e0 0x558cc4cf1250 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5b98008010 tx=0x7f5b98023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a9b98dff04c95 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.496+0000 7f5b87fff640 1 -- 172.21.15.45:0/2587289781 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9803b070 con 0x558cc4d388e0 2024-05-08T05:48:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.496+0000 7f5b87fff640 1 -- 172.21.15.45:0/2587289781 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b98015870 con 0x558cc4d388e0 2024-05-08T05:48:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.496+0000 7f5b87fff640 1 -- 172.21.15.45:0/2587289781 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b98019040 con 0x558cc4d388e0 2024-05-08T05:48:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.497+0000 7f5bb0233740 1 -- 172.21.15.45:0/2587289781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d388e0 msgr2=0x558cc4cf1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.497+0000 7f5bb0233740 1 --2- 172.21.15.45:0/2587289781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d388e0 0x558cc4cf1250 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5b98008010 tx=0x7f5b98023000 comp rx=0 tx=0).stop 2024-05-08T05:48:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.497+0000 7f5bb0233740 1 -- 172.21.15.45:0/2587289781 shutdown_connections 2024-05-08T05:48:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.497+0000 7f5bb0233740 1 --2- 172.21.15.45:0/2587289781 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d19330 0x558cc4cec770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.497+0000 7f5bb0233740 1 --2- 172.21.15.45:0/2587289781 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d388e0 0x558cc4cf1250 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.497+0000 7f5bb0233740 1 --2- 172.21.15.45:0/2587289781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d18f50 0x558cc4d36630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.497+0000 7f5bb0233740 1 -- 172.21.15.45:0/2587289781 >> 172.21.15.45:0/2587289781 conn(0x558cc4cebbb0 msgr2=0x558cc4d31260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.497+0000 7f5bb0233740 1 -- 172.21.15.45:0/2587289781 shutdown_connections 2024-05-08T05:48:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.497+0000 7f5bb0233740 1 -- 172.21.15.45:0/2587289781 wait complete. 2024-05-08T05:48:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.497+0000 7f5bb0233740 1 -- start start 2024-05-08T05:48:25.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.497+0000 7f5bb0233740 1 Processor -- start 2024-05-08T05:48:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.498+0000 7f5bb0233740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d18f50 0x558cc4d97220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.498+0000 7f5bb0233740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d19330 0x558cc4cef3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.498+0000 7f5bb0233740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d388e0 0x558cc4ceac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.498+0000 7f5bb0233740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558cc4d38d40 con 0x558cc4d18f50 2024-05-08T05:48:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.498+0000 7f5bb0233740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558cc4d1f1d0 con 0x558cc4d388e0 2024-05-08T05:48:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.498+0000 7f5bb0233740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x558cc4d13b50 con 0x558cc4d19330 2024-05-08T05:48:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.498+0000 7f5bb0233740 10 client.0 _ll_register_callbacks cb 0x7f5b98009770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:48:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.498+0000 7f5ba6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d18f50 0x558cc4d97220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.498+0000 7f5ba6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d18f50 0x558cc4d97220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43384/0 (socket says 172.21.15.45:43384) 2024-05-08T05:48:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.498+0000 7f5ba6ffd640 1 -- 172.21.15.45:0/2208405054 learned_addr learned my addr 172.21.15.45:0/2208405054 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5ba77fe640 1 --2- 172.21.15.45:0/2208405054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d388e0 0x558cc4ceac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5ba67fc640 1 --2- 172.21.15.45:0/2208405054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d19330 0x558cc4cef3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5ba6ffd640 1 -- 172.21.15.45:0/2208405054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d19330 msgr2=0x558cc4cef3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5ba6ffd640 1 --2- 172.21.15.45:0/2208405054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d19330 0x558cc4cef3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5ba6ffd640 1 -- 172.21.15.45:0/2208405054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d388e0 msgr2=0x558cc4ceac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5ba6ffd640 1 --2- 172.21.15.45:0/2208405054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d388e0 0x558cc4ceac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5ba6ffd640 1 -- 172.21.15.45:0/2208405054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cc4db8240 con 0x558cc4d18f50 2024-05-08T05:48:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5ba77fe640 1 --2- 172.21.15.45:0/2208405054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d388e0 0x558cc4ceac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5ba6ffd640 1 --2- 172.21.15.45:0/2208405054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d18f50 0x558cc4d97220 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000da20 tx=0x7f5ba00604e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa3424e9021b512 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5b85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:48:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5b85ffb640 1 -- 172.21.15.45:0/2208405054 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba00609b0 con 0x558cc4d18f50 2024-05-08T05:48:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5b85ffb640 1 -- 172.21.15.45:0/2208405054 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7f5ba0003c60 con 0x558cc4d18f50 2024-05-08T05:48:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5bb0233740 1 -- 172.21.15.45:0/2208405054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b98015870 con 0x558cc4d18f50 2024-05-08T05:48:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5b85ffb640 1 -- 172.21.15.45:0/2208405054 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f5ba0082050 con 0x558cc4d18f50 2024-05-08T05:48:25.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5bb0233740 10 client.7269 fetch_fsmap learned FSMap version 73 2024-05-08T05:48:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5bb0233740 1 -- 172.21.15.45:0/2208405054 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x558cc4cef900 con 0x558cc4d18f50 2024-05-08T05:48:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5b85ffb640 1 -- 172.21.15.45:0/2208405054 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b98015870 con 0x558cc4d18f50 2024-05-08T05:48:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5b85ffb640 1 -- 172.21.15.45:0/2208405054 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7f5ba0061ce0 con 0x558cc4d18f50 2024-05-08T05:48:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5bb0233740 10 client.7269 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T05:48:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5bb0233740 10 client.7269 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T05:48:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5bb0233740 20 client.7269 populate_metadata read hostname 'smithi045' 2024-05-08T05:48:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5bb0233740 1 --2- 172.21.15.45:0/2208405054 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x558cc4d18770 0x558cc4d350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5bb0233740 4 client.7269 mds_command: new command op to 6747 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:48:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.499+0000 7f5bb0233740 1 -- 172.21.15.45:0/2208405054 --> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] -- command(tid 0: {"prefix": "status"}) -- 0x558cc4c634e0 con 0x558cc4d18770 2024-05-08T05:48:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.500+0000 7f5ba67fc640 1 --2- 172.21.15.45:0/2208405054 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x558cc4d18770 0x558cc4d350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.500+0000 7f5ba67fc640 1 --2- 172.21.15.45:0/2208405054 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x558cc4d18770 0x558cc4d350a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab6f51b104fded34 server_cookie=9067ede8eccb6005 in_seq=0 out_seq=0 2024-05-08T05:48:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.500+0000 7f5b85ffb640 10 client.7269 ms_handle_connect on v2:172.21.15.45:6833/2945763952 2024-05-08T05:48:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.500+0000 7f5b85ffb640 1 -- 172.21.15.45:0/2208405054 <== mds.0 v2:172.21.15.45:6833/2945763952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x558cc4c634e0 con 0x558cc4d18770 2024-05-08T05:48:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.500+0000 7f5b85ffb640 10 client.7269 handle_command_reply: tid=0 2024-05-08T05:48:25.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.500+0000 7f5bb0233740 1 client.7269 shutdown 2024-05-08T05:48:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.500+0000 7f5bb0233740 1 -- 172.21.15.45:0/2208405054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d18f50 msgr2=0x558cc4d97220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.500+0000 7f5bb0233740 1 --2- 172.21.15.45:0/2208405054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d18f50 0x558cc4d97220 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000da20 tx=0x7f5ba00604e0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.500+0000 7f5bb0233740 1 -- 172.21.15.45:0/2208405054 shutdown_connections 2024-05-08T05:48:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.500+0000 7f5bb0233740 1 --2- 172.21.15.45:0/2208405054 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x558cc4d18770 0x558cc4d350a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.500+0000 7f5bb0233740 1 --2- 172.21.15.45:0/2208405054 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x558cc4d19330 0x558cc4cef3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.500+0000 7f5bb0233740 1 --2- 172.21.15.45:0/2208405054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558cc4d388e0 0x558cc4ceac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.500+0000 7f5b85ffb640 0 client.7269 ms_handle_reset on v2:172.21.15.45:6833/2945763952 2024-05-08T05:48:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.500+0000 7f5bb0233740 1 --2- 172.21.15.45:0/2208405054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558cc4d18f50 0x558cc4d97220 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.500+0000 7f5bb0233740 1 -- 172.21.15.45:0/2208405054 >> 172.21.15.45:0/2208405054 conn(0x558cc4cebbb0 msgr2=0x558cc4d31260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.500+0000 7f5bb0233740 1 -- 172.21.15.45:0/2208405054 shutdown_connections 2024-05-08T05:48:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.500+0000 7f5bb0233740 1 -- 172.21.15.45:0/2208405054 wait complete. 2024-05-08T05:48:25.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.500+0000 7f5bb0233740 20 client.7269 trim_cache size 0 max 16384 2024-05-08T05:48:25.507 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:48:25.508 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "e9e1170a-fd27-4625-97a9-5d5f104d13a2", 2024-05-08T05:48:25.508 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-05-08T05:48:25.508 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 6747, 2024-05-08T05:48:25.508 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-05-08T05:48:25.508 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-05-08T05:48:25.508 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-05-08T05:48:25.508 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 4.4855993209999996, 2024-05-08T05:48:25.508 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 73, 2024-05-08T05:48:25.508 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 92, 2024-05-08T05:48:25.508 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 92, 2024-05-08T05:48:25.508 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 90.002627343 2024-05-08T05:48:25.508 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:48:25.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.501+0000 7f5bad446640 1 -- 172.21.15.45:0/1076291255 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5b880764a0 msgr2=0x7f5b88078960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:25.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.501+0000 7f5bad446640 1 --2- 172.21.15.45:0/1076291255 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5b880764a0 0x7f5b88078960 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000c2f0 tx=0x7f5b90007000 comp rx=0 tx=0).stop 2024-05-08T05:48:25.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.501+0000 7f5bad446640 1 -- 172.21.15.45:0/1076291255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b94004000 msgr2=0x7f5ba814f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:25.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.501+0000 7f5bad446640 1 --2- 172.21.15.45:0/1076291255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b94004000 0x7f5ba814f2f0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000fb90 tx=0x7f5ba000d520 comp rx=0 tx=0).stop 2024-05-08T05:48:25.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.501+0000 7f5bad446640 1 -- 172.21.15.45:0/1076291255 shutdown_connections 2024-05-08T05:48:25.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.501+0000 7f5bad446640 1 --2- 172.21.15.45:0/1076291255 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5b880764a0 0x7f5b88078960 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.501+0000 7f5bad446640 1 --2- 172.21.15.45:0/1076291255 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ba80c76d0 0x7f5ba80b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.501+0000 7f5bad446640 1 --2- 172.21.15.45:0/1076291255 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ba80c7b50 0x7f5ba80b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.501+0000 7f5bad446640 1 --2- 172.21.15.45:0/1076291255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b94004000 0x7f5ba814f2f0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.501+0000 7f5bad446640 1 -- 172.21.15.45:0/1076291255 >> 172.21.15.45:0/1076291255 conn(0x7f5ba80b49d0 msgr2=0x7f5ba8160220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:25.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.501+0000 7f5bad446640 1 -- 172.21.15.45:0/1076291255 shutdown_connections 2024-05-08T05:48:25.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.501+0000 7f5bad446640 1 -- 172.21.15.45:0/1076291255 wait complete. 2024-05-08T05:48:25.512 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T05:48:25.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.615+0000 7ff76bce1640 1 Processor -- start 2024-05-08T05:48:25.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.615+0000 7ff76bce1640 1 -- start start 2024-05-08T05:48:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.616+0000 7ff76bce1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7640d3cd0 0x7ff7640cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.616+0000 7ff76bce1640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff7640c9ee0 0x7ff7640d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.616+0000 7ff76bce1640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7640d3330 con 0x7ff7641629a0 2024-05-08T05:48:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.616+0000 7ff76bce1640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff7640d2170 con 0x7ff7640c9ee0 2024-05-08T05:48:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.616+0000 7ff76bce1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff7640d2450 con 0x7ff7640d3cd0 2024-05-08T05:48:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.616+0000 7ff769a7d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff7641629a0 0x7ff7640d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48576/0 (socket says 172.21.15.45:48576) 2024-05-08T05:48:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.616+0000 7ff769a7d640 1 -- 172.21.15.45:0/705255404 learned_addr learned my addr 172.21.15.45:0/705255404 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:48:25.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.616+0000 7ff76a27e640 1 --2- 172.21.15.45:0/705255404 >> v2:172.21.15.63:3300/0 conn(0x7ff7640c9ee0 0x7ff7640d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.616+0000 7ff76927c640 1 --2- 172.21.15.45:0/705255404 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7640d3cd0 0x7ff7640cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.616+0000 7ff768a7b640 1 -- 172.21.15.45:0/705255404 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 521280129 0 0) 0x7ff7640d3330 con 0x7ff7641629a0 2024-05-08T05:48:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.616+0000 7ff768a7b640 1 -- 172.21.15.45:0/705255404 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff750003520 con 0x7ff7641629a0 2024-05-08T05:48:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.616+0000 7ff76a27e640 1 -- 172.21.15.45:0/705255404 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7640d3cd0 msgr2=0x7ff7640cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.616+0000 7ff76a27e640 1 --2- 172.21.15.45:0/705255404 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7640d3cd0 0x7ff7640cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.616+0000 7ff76a27e640 1 -- 172.21.15.45:0/705255404 >> v1:172.21.15.45:6789/0 conn(0x7ff7641629a0 legacy=0x7ff7640d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:25.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.616+0000 7ff76a27e640 1 -- 172.21.15.45:0/705255404 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7640d2730 con 0x7ff7640c9ee0 2024-05-08T05:48:25.619 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.617+0000 7ff76927c640 1 --2- 172.21.15.45:0/705255404 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7640d3cd0 0x7ff7640cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:48:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.617+0000 7ff76a27e640 1 --2- 172.21.15.45:0/705255404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7640c9ee0 0x7ff7640d01e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009d90 tx=0x7ff75c02a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2189c0f92f91d8ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:25.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.617+0000 7ff768a7b640 1 -- 172.21.15.45:0/705255404 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c007740 con 0x7ff7640c9ee0 2024-05-08T05:48:25.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.617+0000 7ff768a7b640 1 -- 172.21.15.45:0/705255404 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff75c0078e0 con 0x7ff7640c9ee0 2024-05-08T05:48:25.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.617+0000 7ff768a7b640 1 -- 172.21.15.45:0/705255404 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c007c90 con 0x7ff7640c9ee0 2024-05-08T05:48:25.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.618+0000 7ff76bce1640 1 -- 172.21.15.45:0/705255404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7640c9ee0 msgr2=0x7ff7640d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:25.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.618+0000 7ff76bce1640 1 --2- 172.21.15.45:0/705255404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7640c9ee0 0x7ff7640d01e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009d90 tx=0x7ff75c02a530 comp rx=0 tx=0).stop 2024-05-08T05:48:25.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.618+0000 7ff76bce1640 1 -- 172.21.15.45:0/705255404 shutdown_connections 2024-05-08T05:48:25.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.618+0000 7ff76bce1640 1 --2- 172.21.15.45:0/705255404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7640c9ee0 0x7ff7640d01e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.618+0000 7ff76bce1640 1 --2- 172.21.15.45:0/705255404 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7640d3cd0 0x7ff7640cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.618+0000 7ff76bce1640 1 -- 172.21.15.45:0/705255404 >> 172.21.15.45:0/705255404 conn(0x7ff7640d4fe0 msgr2=0x7ff7640d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:25.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.618+0000 7ff76bce1640 1 -- 172.21.15.45:0/705255404 shutdown_connections 2024-05-08T05:48:25.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.618+0000 7ff76bce1640 1 -- 172.21.15.45:0/705255404 wait complete. 2024-05-08T05:48:25.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.619+0000 7ff76bce1640 1 Processor -- start 2024-05-08T05:48:25.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.619+0000 7ff76bce1640 1 -- start start 2024-05-08T05:48:25.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.620+0000 7ff76bce1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7640c9ee0 0x7ff764157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.620+0000 7ff76bce1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7640d3cd0 0x7ff7640d0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.620+0000 7ff76bce1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7641629a0 0x7ff7640d0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.620+0000 7ff76bce1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff7640cf440 con 0x7ff7641629a0 2024-05-08T05:48:25.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.620+0000 7ff76bce1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff7640cf2c0 con 0x7ff7640d3cd0 2024-05-08T05:48:25.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.620+0000 7ff76bce1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff7640d2dc0 con 0x7ff7640c9ee0 2024-05-08T05:48:25.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.620+0000 7ff76a27e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7641629a0 0x7ff7640d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.620+0000 7ff76a27e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7641629a0 0x7ff7640d0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43398/0 (socket says 172.21.15.45:43398) 2024-05-08T05:48:25.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.620+0000 7ff76a27e640 1 -- 172.21.15.45:0/3566501347 learned_addr learned my addr 172.21.15.45:0/3566501347 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:25.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.620+0000 7ff76927c640 1 --2- 172.21.15.45:0/3566501347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7640d3cd0 0x7ff7640d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.620+0000 7ff769a7d640 1 --2- 172.21.15.45:0/3566501347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7640c9ee0 0x7ff764157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.620+0000 7ff76a27e640 1 -- 172.21.15.45:0/3566501347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7640c9ee0 msgr2=0x7ff764157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:25.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.620+0000 7ff76a27e640 1 --2- 172.21.15.45:0/3566501347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7640c9ee0 0x7ff764157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.620+0000 7ff76a27e640 1 -- 172.21.15.45:0/3566501347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7640d3cd0 msgr2=0x7ff7640d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:25.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.620+0000 7ff76a27e640 1 --2- 172.21.15.45:0/3566501347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7640d3cd0 0x7ff7640d0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.620+0000 7ff76a27e640 1 -- 172.21.15.45:0/3566501347 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76415c6a0 con 0x7ff7641629a0 2024-05-08T05:48:25.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.620+0000 7ff769a7d640 1 --2- 172.21.15.45:0/3566501347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7640c9ee0 0x7ff764157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:25.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.621+0000 7ff76a27e640 1 --2- 172.21.15.45:0/3566501347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7641629a0 0x7ff7640d0860 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009eb0 tx=0x7ff75c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:25.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.621+0000 7ff75a7fc640 1 -- 172.21.15.45:0/3566501347 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c009450 con 0x7ff7641629a0 2024-05-08T05:48:25.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.621+0000 7ff75a7fc640 1 -- 172.21.15.45:0/3566501347 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff75c004240 con 0x7ff7641629a0 2024-05-08T05:48:25.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.621+0000 7ff76bce1640 1 -- 172.21.15.45:0/3566501347 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7640b8df0 con 0x7ff7641629a0 2024-05-08T05:48:25.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.621+0000 7ff75a7fc640 1 -- 172.21.15.45:0/3566501347 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c004630 con 0x7ff7641629a0 2024-05-08T05:48:25.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.621+0000 7ff76bce1640 1 -- 172.21.15.45:0/3566501347 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7640b93b0 con 0x7ff7641629a0 2024-05-08T05:48:25.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.622+0000 7ff76bce1640 1 -- 172.21.15.45:0/3566501347 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff764068c40 con 0x7ff7641629a0 2024-05-08T05:48:25.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.623+0000 7ff75a7fc640 1 -- 172.21.15.45:0/3566501347 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff75c039040 con 0x7ff7641629a0 2024-05-08T05:48:25.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.623+0000 7ff75a7fc640 1 --2- 172.21.15.45:0/3566501347 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff7480762d0 0x7ff748078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:25.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.623+0000 7ff75a7fc640 1 -- 172.21.15.45:0/3566501347 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7ff75c0b8740 con 0x7ff7641629a0 2024-05-08T05:48:25.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.623+0000 7ff769a7d640 1 --2- 172.21.15.45:0/3566501347 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff7480762d0 0x7ff748078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:25.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.624+0000 7ff769a7d640 1 --2- 172.21.15.45:0/3566501347 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff7480762d0 0x7ff748078790 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff760007fd0 tx=0x7ff760009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:25.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.628+0000 7ff75a7fc640 1 -- 172.21.15.45:0/3566501347 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff764068c40 con 0x7ff7641629a0 2024-05-08T05:48:25.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.860+0000 7ff76bce1640 1 -- 172.21.15.45:0/3566501347 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff7640693f0 con 0x7ff7641629a0 2024-05-08T05:48:25.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.861+0000 7ff75a7fc640 1 -- 172.21.15.45:0/3566501347 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v73) ==== 53+0+73 (secure 0 0 0) 0x7ff75c040030 con 0x7ff7641629a0 2024-05-08T05:48:25.861 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T05:48:25.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.863+0000 7ff76bce1640 1 -- 172.21.15.45:0/3566501347 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff7480762d0 msgr2=0x7ff748078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:25.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.863+0000 7ff76bce1640 1 --2- 172.21.15.45:0/3566501347 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff7480762d0 0x7ff748078790 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff760007fd0 tx=0x7ff760009040 comp rx=0 tx=0).stop 2024-05-08T05:48:25.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.863+0000 7ff76bce1640 1 -- 172.21.15.45:0/3566501347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7641629a0 msgr2=0x7ff7640d0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:25.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.863+0000 7ff76bce1640 1 --2- 172.21.15.45:0/3566501347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7641629a0 0x7ff7640d0860 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009eb0 tx=0x7ff75c004a40 comp rx=0 tx=0).stop 2024-05-08T05:48:25.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.863+0000 7ff76bce1640 1 -- 172.21.15.45:0/3566501347 shutdown_connections 2024-05-08T05:48:25.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.863+0000 7ff76bce1640 1 --2- 172.21.15.45:0/3566501347 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff7480762d0 0x7ff748078790 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.863+0000 7ff76bce1640 1 --2- 172.21.15.45:0/3566501347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff7641629a0 0x7ff7640d0860 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.863+0000 7ff76bce1640 1 --2- 172.21.15.45:0/3566501347 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7640d3cd0 0x7ff7640d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.863+0000 7ff76bce1640 1 --2- 172.21.15.45:0/3566501347 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff7640c9ee0 0x7ff764157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.863+0000 7ff76bce1640 1 -- 172.21.15.45:0/3566501347 >> 172.21.15.45:0/3566501347 conn(0x7ff7640d4fe0 msgr2=0x7ff7640d35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:25.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.863+0000 7ff76bce1640 1 -- 172.21.15.45:0/3566501347 shutdown_connections 2024-05-08T05:48:25.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:25.863+0000 7ff76bce1640 1 -- 172.21.15.45:0/3566501347 wait complete. 2024-05-08T05:48:25.874 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T05:48:25.874 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:48:25.874 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-05-08T05:48:25.874 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T05:48:25.874 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:48:25.874 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T05:48:25.874 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T05:48:25.874 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:48:25.874 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:48:25.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:48:25.875 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-05-08T05:48:25.891 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:48:25.892 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:48:25.892 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:48:25.892 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:48:25.892 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-05-08T05:48:25.892 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:48:25.892 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:48:25.892 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:48:25.892 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-05-08T05:48:25.892 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:48:25.892 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:48:25.892 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-05-08T05:48:25.892 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:48:25.892 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-05-08T05:48:25.893 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-05-08T05:48:25.893 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:48:25.893 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42319sec preferred_lft 42319sec 2024-05-08T05:48:25.893 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-05-08T05:48:25.893 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:48:25.893 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:48:25.893 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-05-08T05:48:25.893 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-05-08T05:48:25.893 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:48:25.893 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T05:48:25.893 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T05:48:25.893 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:48:25.893 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c80a:aff:fe94:c77/64 scope link 2024-05-08T05:48:25.893 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:48:25.893 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:48:25.894 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 82:f3:eb:6a:57:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:48:25.894 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::80f3:ebff:fe6a:57ba/64 scope link 2024-05-08T05:48:25.894 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:48:25.894 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:48:25.894 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T05:48:25.894 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::14a2:64ff:feda:3a9c/64 scope link 2024-05-08T05:48:25.894 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:48:25.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:48:25.894 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:48:25.956 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:48:25.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:48:25.958 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-05-08T05:48:26.015 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-05-08T05:48:26.015 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:48:26.015 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:48:26.015 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:48:26.015 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:48:26.015 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:48:26.016 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:48:26.016 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:48:26.016 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:48:26.016 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:48:26.016 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:48:26.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:48:26.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:48:26.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:48:26.145 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:48:26.161 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T05:48:26.161 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T05:48:26.161 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:48:26.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:48:26.162 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-05-08T05:48:26.228 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:48:26.228 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T05:48:26.283 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:48:26.284 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:48:26.284 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:48:26.340 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T05:48:26.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:48:26.340 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:48:26.425 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:48:26.425 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:48:26.439 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:48:26.439 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:48:26.439 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-08T05:48:26.439 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@e9e1170a-fd27-4625-97a9-5d5f104d13a2.cephfs=/ 2024-05-08T05:48:26.439 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.63:3300/172.21.15.63:3301" will pass to kernel 2024-05-08T05:48:26.439 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T05:48:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:48:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:48:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:48:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:48:26.441 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:48:26.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:48:26.441 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:48:26.467 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:48:26.475 INFO:tasks.ceph.mds.c.smithi045.stderr:2024-05-08T05:48:26.476+0000 7fdbea21d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:48:26.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:48:26.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:48:26.533 INFO:teuthology.orchestra.run.smithi045.stdout:client6538 2024-05-08T05:48:26.533 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:48:26.533 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:48:26.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:48:26.549 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-05-08T05:48:26.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.697+0000 7f4b45d4e640 1 Processor -- start 2024-05-08T05:48:26.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.697+0000 7f4b45d4e640 1 -- start start 2024-05-08T05:48:26.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.698+0000 7f4b45d4e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4b40159670 0x7f4b400b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.698+0000 7f4b45d4e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b400af7e0 con 0x7f4b4006c3a0 2024-05-08T05:48:26.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.698+0000 7f4b45d4e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b400afac0 con 0x7f4b40163660 2024-05-08T05:48:26.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.698+0000 7f4b45d4e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4b400afda0 con 0x7f4b40159670 2024-05-08T05:48:26.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.698+0000 7f4b3ffff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f4b4006c3a0 0x7f4b4015f6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48588/0 (socket says 172.21.15.45:48588) 2024-05-08T05:48:26.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.698+0000 7f4b3ffff640 1 -- 172.21.15.45:0/4214850482 learned_addr learned my addr 172.21.15.45:0/4214850482 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:48:26.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.698+0000 7f4b3f7fe640 1 --2- 172.21.15.45:0/4214850482 >> v2:172.21.15.63:3300/0 conn(0x7f4b40159670 0x7f4b400b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.698+0000 7f4b3e7fc640 1 -- 172.21.15.45:0/4214850482 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1457928925 0 0) 0x7f4b400af7e0 con 0x7f4b4006c3a0 2024-05-08T05:48:26.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.698+0000 7f4b3e7fc640 1 -- 172.21.15.45:0/4214850482 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b28003520 con 0x7f4b4006c3a0 2024-05-08T05:48:26.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.698+0000 7f4b3e7fc640 1 -- 172.21.15.45:0/4214850482 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2320909908 0 0) 0x7f4b400afac0 con 0x7f4b40163660 2024-05-08T05:48:26.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.698+0000 7f4b3e7fc640 1 -- 172.21.15.45:0/4214850482 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b400af7e0 con 0x7f4b40163660 2024-05-08T05:48:26.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3e7fc640 1 -- 172.21.15.45:0/4214850482 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1410680441 0 0) 0x7f4b28003520 con 0x7f4b4006c3a0 2024-05-08T05:48:26.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3e7fc640 1 -- 172.21.15.45:0/4214850482 >> v2:172.21.15.63:3300/0 conn(0x7f4b40159670 msgr2=0x7f4b400b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3e7fc640 1 --2- 172.21.15.45:0/4214850482 >> v2:172.21.15.63:3300/0 conn(0x7f4b40159670 0x7f4b400b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3e7fc640 1 -- 172.21.15.45:0/4214850482 >> v1:172.21.15.63:6789/0 conn(0x7f4b40163660 legacy=0x7f4b400aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3e7fc640 1 -- 172.21.15.45:0/4214850482 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b40162e60 con 0x7f4b4006c3a0 2024-05-08T05:48:26.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3e7fc640 1 -- 172.21.15.45:0/4214850482 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f4b38001a70 con 0x7f4b4006c3a0 2024-05-08T05:48:26.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3e7fc640 1 -- 172.21.15.45:0/4214850482 >> v1:172.21.15.45:6789/0 conn(0x7f4b4006c3a0 legacy=0x7f4b4015f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3e7fc640 1 --2- 172.21.15.45:0/4214850482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b28003c30 0x7f4b28004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3e7fc640 1 --2- 172.21.15.45:0/4214850482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b28004800 0x7f4b28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3e7fc640 1 --2- 172.21.15.45:0/4214850482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b28005210 0x7f4b2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3e7fc640 1 -- 172.21.15.45:0/4214850482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b400afda0 con 0x7f4b28004800 2024-05-08T05:48:26.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3e7fc640 1 -- 172.21.15.45:0/4214850482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2800c140 con 0x7f4b28003c30 2024-05-08T05:48:26.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3e7fc640 1 -- 172.21.15.45:0/4214850482 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b2800c340 con 0x7f4b28005210 2024-05-08T05:48:26.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3effd640 1 --2- 172.21.15.45:0/4214850482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b28004800 0x7f4b28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3f7fe640 1 --2- 172.21.15.45:0/4214850482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b28003c30 0x7f4b28004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3ffff640 1 --2- 172.21.15.45:0/4214850482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b28005210 0x7f4b2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3effd640 1 -- 172.21.15.45:0/4214850482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b28005210 msgr2=0x7f4b2800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3effd640 1 --2- 172.21.15.45:0/4214850482 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b28005210 0x7f4b2800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3effd640 1 -- 172.21.15.45:0/4214850482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b28003c30 msgr2=0x7f4b28004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3effd640 1 --2- 172.21.15.45:0/4214850482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b28003c30 0x7f4b28004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3effd640 1 -- 172.21.15.45:0/4214850482 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b28003520 con 0x7f4b28004800 2024-05-08T05:48:26.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.699+0000 7f4b3ffff640 1 -- 172.21.15.45:0/4214850482 reap_dead start 2024-05-08T05:48:26.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.700+0000 7f4b3effd640 1 --2- 172.21.15.45:0/4214850482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b28004800 0x7f4b28004c80 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f4b34001900 tx=0x7f4b28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121a2ed8218a66a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:26.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.700+0000 7f4b3e7fc640 1 -- 172.21.15.45:0/4214850482 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34001520 con 0x7f4b28004800 2024-05-08T05:48:26.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.700+0000 7f4b3e7fc640 1 -- 172.21.15.45:0/4214850482 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b3402acd0 con 0x7f4b28004800 2024-05-08T05:48:26.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.700+0000 7f4b3e7fc640 1 -- 172.21.15.45:0/4214850482 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34005310 con 0x7f4b28004800 2024-05-08T05:48:26.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.700+0000 7f4b45d4e640 1 -- 172.21.15.45:0/4214850482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b28004800 msgr2=0x7f4b28004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.700+0000 7f4b45d4e640 1 --2- 172.21.15.45:0/4214850482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b28004800 0x7f4b28004c80 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f4b34001900 tx=0x7f4b28005830 comp rx=0 tx=0).stop 2024-05-08T05:48:26.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.700+0000 7f4b45d4e640 1 -- 172.21.15.45:0/4214850482 shutdown_connections 2024-05-08T05:48:26.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.700+0000 7f4b45d4e640 1 --2- 172.21.15.45:0/4214850482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b28004800 0x7f4b28004c80 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.700+0000 7f4b45d4e640 1 -- 172.21.15.45:0/4214850482 >> 172.21.15.45:0/4214850482 conn(0x7f4b400b3aa0 msgr2=0x7f4b400b3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:26.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.700+0000 7f4b45d4e640 1 -- 172.21.15.45:0/4214850482 shutdown_connections 2024-05-08T05:48:26.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.700+0000 7f4b45d4e640 1 -- 172.21.15.45:0/4214850482 wait complete. 2024-05-08T05:48:26.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.702+0000 7f4b45d4e640 1 Processor -- start 2024-05-08T05:48:26.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.702+0000 7f4b45d4e640 1 -- start start 2024-05-08T05:48:26.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.702+0000 7f4b45d4e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b28004800 0x7f4b40153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b45d4e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b40157a40 0x7f4b401f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b45d4e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b40157ea0 0x7f4b401483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b45d4e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b40159a50 con 0x7f4b40157a40 2024-05-08T05:48:26.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b45d4e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b40163b10 con 0x7f4b28004800 2024-05-08T05:48:26.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b45d4e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b40159bd0 con 0x7f4b40157ea0 2024-05-08T05:48:26.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b3effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b40157a40 0x7f4b401f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b3effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b40157a40 0x7f4b401f85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43406/0 (socket says 172.21.15.45:43406) 2024-05-08T05:48:26.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b3effd640 1 -- 172.21.15.45:0/930031804 learned_addr learned my addr 172.21.15.45:0/930031804 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:26.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b3f7fe640 1 --2- 172.21.15.45:0/930031804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b28004800 0x7f4b40153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b3ffff640 1 --2- 172.21.15.45:0/930031804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b40157ea0 0x7f4b401483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b3effd640 1 -- 172.21.15.45:0/930031804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b40157ea0 msgr2=0x7f4b401483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b3effd640 1 --2- 172.21.15.45:0/930031804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b40157ea0 0x7f4b401483f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b3effd640 1 -- 172.21.15.45:0/930031804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b28004800 msgr2=0x7f4b40153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b3effd640 1 --2- 172.21.15.45:0/930031804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b28004800 0x7f4b40153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b3effd640 1 -- 172.21.15.45:0/930031804 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b40153200 con 0x7f4b40157a40 2024-05-08T05:48:26.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b3effd640 1 --2- 172.21.15.45:0/930031804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b40157a40 0x7f4b401f85b0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f4b34001900 tx=0x7f4b340012f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:26.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b27fff640 1 -- 172.21.15.45:0/930031804 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34042070 con 0x7f4b40157a40 2024-05-08T05:48:26.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b27fff640 1 -- 172.21.15.45:0/930031804 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b340075e0 con 0x7f4b40157a40 2024-05-08T05:48:26.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b45d4e640 1 -- 172.21.15.45:0/930031804 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b401f9050 con 0x7f4b40157a40 2024-05-08T05:48:26.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b27fff640 1 -- 172.21.15.45:0/930031804 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34007a10 con 0x7f4b40157a40 2024-05-08T05:48:26.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.703+0000 7f4b45d4e640 1 -- 172.21.15.45:0/930031804 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b401534a0 con 0x7f4b40157a40 2024-05-08T05:48:26.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.705+0000 7f4b27fff640 1 -- 172.21.15.45:0/930031804 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4b34045070 con 0x7f4b40157a40 2024-05-08T05:48:26.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.705+0000 7f4b27fff640 1 --2- 172.21.15.45:0/930031804 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b2c076300 0x7f4b2c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.706+0000 7f4b27fff640 1 -- 172.21.15.45:0/930031804 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7f4b340b6ab0 con 0x7f4b40157a40 2024-05-08T05:48:26.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.706+0000 7f4b3f7fe640 1 --2- 172.21.15.45:0/930031804 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b2c076300 0x7f4b2c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.706+0000 7f4b48a6a740 1 Processor -- start 2024-05-08T05:48:26.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.706+0000 7f4b3f7fe640 1 --2- 172.21.15.45:0/930031804 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b2c076300 0x7f4b2c0787c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009b60 tx=0x7f4b30009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:26.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.706+0000 7f4b48a6a740 1 -- start start 2024-05-08T05:48:26.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.707+0000 7f4b48a6a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4d3220 0x55bb3f485bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.707+0000 7f4b48a6a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4b6340 0x55bb3f4b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.707+0000 7f4b48a6a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4b5c00 0x55bb3f48aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.707+0000 7f4b48a6a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bb3f3eecf0 con 0x55bb3f4b5c00 2024-05-08T05:48:26.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.707+0000 7f4b48a6a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bb3f3a7e10 con 0x55bb3f4d3220 2024-05-08T05:48:26.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.707+0000 7f4b48a6a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55bb3f3eacb0 con 0x55bb3f4b6340 2024-05-08T05:48:26.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.707+0000 7f4b3ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4b5c00 0x55bb3f48aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.707+0000 7f4b3ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4b5c00 0x55bb3f48aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43408/0 (socket says 172.21.15.45:43408) 2024-05-08T05:48:26.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.707+0000 7f4b3ffff640 1 -- 172.21.15.45:0/1023684652 learned_addr learned my addr 172.21.15.45:0/1023684652 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:26.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.707+0000 7f4b3f7fe640 1 --2- 172.21.15.45:0/1023684652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4d3220 0x55bb3f485bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.707+0000 7f4b3effd640 1 --2- 172.21.15.45:0/1023684652 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4b6340 0x55bb3f4b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.708+0000 7f4b3ffff640 1 -- 172.21.15.45:0/1023684652 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4b6340 msgr2=0x55bb3f4b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.708+0000 7f4b3ffff640 1 --2- 172.21.15.45:0/1023684652 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4b6340 0x55bb3f4b56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.708+0000 7f4b3ffff640 1 -- 172.21.15.45:0/1023684652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4d3220 msgr2=0x55bb3f485bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.708+0000 7f4b3ffff640 1 --2- 172.21.15.45:0/1023684652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4d3220 0x55bb3f485bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.708+0000 7f4b3ffff640 1 -- 172.21.15.45:0/1023684652 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb3f1c87e0 con 0x55bb3f4b5c00 2024-05-08T05:48:26.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.708+0000 7f4b3ffff640 1 --2- 172.21.15.45:0/1023684652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4b5c00 0x55bb3f48aa10 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4b400760d0 tx=0x7f4b38005f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36093c7ed8a13470 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:26.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.708+0000 7f4b257fa640 1 -- 172.21.15.45:0/1023684652 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3801a070 con 0x55bb3f4b5c00 2024-05-08T05:48:26.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.708+0000 7f4b257fa640 1 -- 172.21.15.45:0/1023684652 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b38007a60 con 0x55bb3f4b5c00 2024-05-08T05:48:26.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.708+0000 7f4b257fa640 1 -- 172.21.15.45:0/1023684652 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38015690 con 0x55bb3f4b5c00 2024-05-08T05:48:26.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.709+0000 7f4b48a6a740 1 -- 172.21.15.45:0/1023684652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4b5c00 msgr2=0x55bb3f48aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.709+0000 7f4b48a6a740 1 --2- 172.21.15.45:0/1023684652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4b5c00 0x55bb3f48aa10 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4b400760d0 tx=0x7f4b38005f40 comp rx=0 tx=0).stop 2024-05-08T05:48:26.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.709+0000 7f4b48a6a740 1 -- 172.21.15.45:0/1023684652 shutdown_connections 2024-05-08T05:48:26.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.709+0000 7f4b48a6a740 1 --2- 172.21.15.45:0/1023684652 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4b6340 0x55bb3f4b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.709+0000 7f4b48a6a740 1 --2- 172.21.15.45:0/1023684652 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4b5c00 0x55bb3f48aa10 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.709+0000 7f4b48a6a740 1 --2- 172.21.15.45:0/1023684652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4d3220 0x55bb3f485bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.709+0000 7f4b48a6a740 1 -- 172.21.15.45:0/1023684652 >> 172.21.15.45:0/1023684652 conn(0x55bb3f487e20 msgr2=0x55bb3f4d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:26.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.709+0000 7f4b48a6a740 1 -- 172.21.15.45:0/1023684652 shutdown_connections 2024-05-08T05:48:26.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.709+0000 7f4b48a6a740 1 -- 172.21.15.45:0/1023684652 wait complete. 2024-05-08T05:48:26.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.709+0000 7f4b48a6a740 1 -- start start 2024-05-08T05:48:26.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.709+0000 7f4b48a6a740 1 Processor -- start 2024-05-08T05:48:26.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.710+0000 7f4b48a6a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4b6340 0x55bb3f4d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.710+0000 7f4b48a6a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4d3220 0x55bb3f4d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.710+0000 7f4b48a6a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4a30f0 0x55bb3f48aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.710+0000 7f4b48a6a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bb3f489fd0 con 0x55bb3f4d3220 2024-05-08T05:48:26.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.710+0000 7f4b48a6a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55bb3f3ab940 con 0x55bb3f4a30f0 2024-05-08T05:48:26.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.710+0000 7f4b48a6a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55bb3f4893c0 con 0x55bb3f4b6340 2024-05-08T05:48:26.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.710+0000 7f4b3effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4d3220 0x55bb3f4d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.710+0000 7f4b48a6a740 10 client.0 _ll_register_callbacks cb 0x55bb3f3ae440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:48:26.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.710+0000 7f4b3effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4d3220 0x55bb3f4d0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43414/0 (socket says 172.21.15.45:43414) 2024-05-08T05:48:26.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.710+0000 7f4b3effd640 1 -- 172.21.15.45:0/3269638032 learned_addr learned my addr 172.21.15.45:0/3269638032 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:26.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.710+0000 7f4b3ffff640 1 --2- 172.21.15.45:0/3269638032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4a30f0 0x55bb3f48aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.710+0000 7f4b3f7fe640 1 --2- 172.21.15.45:0/3269638032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4b6340 0x55bb3f4d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b3effd640 1 -- 172.21.15.45:0/3269638032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4b6340 msgr2=0x55bb3f4d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b3effd640 1 --2- 172.21.15.45:0/3269638032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4b6340 0x55bb3f4d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b3effd640 1 -- 172.21.15.45:0/3269638032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4a30f0 msgr2=0x55bb3f48aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b3effd640 1 --2- 172.21.15.45:0/3269638032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4a30f0 0x55bb3f48aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b3effd640 1 -- 172.21.15.45:0/3269638032 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb3f228f70 con 0x55bb3f4d3220 2024-05-08T05:48:26.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b3effd640 1 --2- 172.21.15.45:0/3269638032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4d3220 0x55bb3f4d0dd0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f4b3407f7a0 tx=0x7f4b3407f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f15deea8d6e7f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:26.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b1e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:48:26.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b1e7fc640 1 -- 172.21.15.45:0/3269638032 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b340b2060 con 0x55bb3f4d3220 2024-05-08T05:48:26.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b1e7fc640 1 -- 172.21.15.45:0/3269638032 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7f4b3407ed00 con 0x55bb3f4d3220 2024-05-08T05:48:26.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b48a6a740 1 -- 172.21.15.45:0/3269638032 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b38007a60 con 0x55bb3f4d3220 2024-05-08T05:48:26.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b1e7fc640 1 -- 172.21.15.45:0/3269638032 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f4b3409b030 con 0x55bb3f4d3220 2024-05-08T05:48:26.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b48a6a740 10 client.7287 fetch_fsmap learned FSMap version 73 2024-05-08T05:48:26.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b48a6a740 1 -- 172.21.15.45:0/3269638032 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55bb3f0c0170 con 0x55bb3f4d3220 2024-05-08T05:48:26.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b1e7fc640 1 -- 172.21.15.45:0/3269638032 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b38007a60 con 0x55bb3f4d3220 2024-05-08T05:48:26.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b1e7fc640 1 -- 172.21.15.45:0/3269638032 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7f4b3407b410 con 0x55bb3f4d3220 2024-05-08T05:48:26.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b48a6a740 10 client.7287 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T05:48:26.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b48a6a740 10 client.7287 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-08T05:48:26.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b48a6a740 20 client.7287 populate_metadata read hostname 'smithi045' 2024-05-08T05:48:26.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b48a6a740 1 --2- 172.21.15.45:0/3269638032 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x55bb3f4b6830 0x55bb3f4b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b48a6a740 4 client.7287 mds_command: new command op to 6747 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:48:26.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.711+0000 7f4b48a6a740 1 -- 172.21.15.45:0/3269638032 --> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb3f3fc900 con 0x55bb3f4b6830 2024-05-08T05:48:26.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.712+0000 7f4b3f7fe640 1 --2- 172.21.15.45:0/3269638032 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x55bb3f4b6830 0x55bb3f4b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.712+0000 7f4b3f7fe640 1 --2- 172.21.15.45:0/3269638032 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x55bb3f4b6830 0x55bb3f4b6c40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8681648918054cd server_cookie=de2eec133c61ac88 in_seq=0 out_seq=0 2024-05-08T05:48:26.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.712+0000 7f4b1e7fc640 10 client.7287 ms_handle_connect on v2:172.21.15.45:6833/2945763952 2024-05-08T05:48:26.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.712+0000 7f4b1e7fc640 1 -- 172.21.15.45:0/3269638032 <== mds.0 v2:172.21.15.45:6833/2945763952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bb3f3fc900 con 0x55bb3f4b6830 2024-05-08T05:48:26.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.712+0000 7f4b1e7fc640 10 client.7287 handle_command_reply: tid=0 2024-05-08T05:48:26.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.712+0000 7f4b48a6a740 1 client.7287 shutdown 2024-05-08T05:48:26.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.713+0000 7f4b48a6a740 1 -- 172.21.15.45:0/3269638032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4d3220 msgr2=0x55bb3f4d0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.713+0000 7f4b48a6a740 1 --2- 172.21.15.45:0/3269638032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4d3220 0x55bb3f4d0dd0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f4b3407f7a0 tx=0x7f4b3407f4b0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.713+0000 7f4b48a6a740 1 -- 172.21.15.45:0/3269638032 shutdown_connections 2024-05-08T05:48:26.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.713+0000 7f4b48a6a740 1 --2- 172.21.15.45:0/3269638032 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x55bb3f4b6830 0x55bb3f4b6c40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.713+0000 7f4b48a6a740 1 --2- 172.21.15.45:0/3269638032 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4a30f0 0x55bb3f48aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.713+0000 7f4b48a6a740 1 --2- 172.21.15.45:0/3269638032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4d3220 0x55bb3f4d0dd0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.713+0000 7f4b48a6a740 1 --2- 172.21.15.45:0/3269638032 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4b6340 0x55bb3f4d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.713+0000 7f4b1e7fc640 0 client.7287 ms_handle_reset on v2:172.21.15.45:6833/2945763952 2024-05-08T05:48:26.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.713+0000 7f4b48a6a740 1 -- 172.21.15.45:0/3269638032 >> 172.21.15.45:0/3269638032 conn(0x55bb3f4b5c00 msgr2=0x55bb3f4852f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:26.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.713+0000 7f4b48a6a740 1 -- 172.21.15.45:0/3269638032 shutdown_connections 2024-05-08T05:48:26.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.713+0000 7f4b48a6a740 1 -- 172.21.15.45:0/3269638032 wait complete. 2024-05-08T05:48:26.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.713+0000 7f4b48a6a740 20 client.7287 trim_cache size 0 max 16384 2024-05-08T05:48:26.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.731+0000 7f4b48a6a740 1 Processor -- start 2024-05-08T05:48:26.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.731+0000 7f4b48a6a740 1 -- start start 2024-05-08T05:48:26.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.732+0000 7f4b48a6a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4a30f0 0x55bb3f4b2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.732+0000 7f4b48a6a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4b6340 0x55bb3f486980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.732+0000 7f4b48a6a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4d3220 0x55bb3f4b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.732+0000 7f4b48a6a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b40202840 con 0x55bb3f4d3220 2024-05-08T05:48:26.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.732+0000 7f4b48a6a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b40201820 con 0x55bb3f4a30f0 2024-05-08T05:48:26.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.732+0000 7f4b48a6a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b40200bb0 con 0x55bb3f4b6340 2024-05-08T05:48:26.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.732+0000 7f4b3ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4d3220 0x55bb3f4b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.732+0000 7f4b3ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4d3220 0x55bb3f4b6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43428/0 (socket says 172.21.15.45:43428) 2024-05-08T05:48:26.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.732+0000 7f4b3ffff640 1 -- 172.21.15.45:0/2928477055 learned_addr learned my addr 172.21.15.45:0/2928477055 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:26.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.732+0000 7f4b3f7fe640 1 --2- 172.21.15.45:0/2928477055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4a30f0 0x55bb3f4b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.732+0000 7f4b3effd640 1 --2- 172.21.15.45:0/2928477055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4b6340 0x55bb3f486980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.732+0000 7f4b3ffff640 1 -- 172.21.15.45:0/2928477055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4b6340 msgr2=0x55bb3f486980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.732+0000 7f4b3ffff640 1 --2- 172.21.15.45:0/2928477055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4b6340 0x55bb3f486980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.732+0000 7f4b3ffff640 1 -- 172.21.15.45:0/2928477055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4a30f0 msgr2=0x55bb3f4b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.732+0000 7f4b3ffff640 1 --2- 172.21.15.45:0/2928477055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4a30f0 0x55bb3f4b2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.732+0000 7f4b3ffff640 1 -- 172.21.15.45:0/2928477055 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb3f4cde30 con 0x55bb3f4d3220 2024-05-08T05:48:26.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.732+0000 7f4b3f7fe640 1 --2- 172.21.15.45:0/2928477055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4a30f0 0x55bb3f4b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:26.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.732+0000 7f4b3ffff640 1 --2- 172.21.15.45:0/2928477055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4d3220 0x55bb3f4b6c40 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f4b400b6bb0 tx=0x7f4b3800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9920ff25a61c9b9a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:26.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.732+0000 7f4b24ff9640 1 -- 172.21.15.45:0/2928477055 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38024070 con 0x55bb3f4d3220 2024-05-08T05:48:26.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.733+0000 7f4b24ff9640 1 -- 172.21.15.45:0/2928477055 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b3801b040 con 0x55bb3f4d3220 2024-05-08T05:48:26.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.733+0000 7f4b24ff9640 1 -- 172.21.15.45:0/2928477055 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3801a070 con 0x55bb3f4d3220 2024-05-08T05:48:26.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.733+0000 7f4b48a6a740 1 -- 172.21.15.45:0/2928477055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4d3220 msgr2=0x55bb3f4b6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.733+0000 7f4b48a6a740 1 --2- 172.21.15.45:0/2928477055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4d3220 0x55bb3f4b6c40 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f4b400b6bb0 tx=0x7f4b3800e000 comp rx=0 tx=0).stop 2024-05-08T05:48:26.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.733+0000 7f4b48a6a740 1 -- 172.21.15.45:0/2928477055 shutdown_connections 2024-05-08T05:48:26.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.733+0000 7f4b48a6a740 1 --2- 172.21.15.45:0/2928477055 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4b6340 0x55bb3f486980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.733+0000 7f4b48a6a740 1 --2- 172.21.15.45:0/2928477055 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4d3220 0x55bb3f4b6c40 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.733+0000 7f4b48a6a740 1 --2- 172.21.15.45:0/2928477055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4a30f0 0x55bb3f4b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.733+0000 7f4b48a6a740 1 -- 172.21.15.45:0/2928477055 >> 172.21.15.45:0/2928477055 conn(0x55bb3f4b5c00 msgr2=0x55bb3f4d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:26.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.733+0000 7f4b48a6a740 1 -- 172.21.15.45:0/2928477055 shutdown_connections 2024-05-08T05:48:26.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.733+0000 7f4b48a6a740 1 -- 172.21.15.45:0/2928477055 wait complete. 2024-05-08T05:48:26.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.734+0000 7f4b48a6a740 1 -- start start 2024-05-08T05:48:26.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.734+0000 7f4b48a6a740 1 Processor -- start 2024-05-08T05:48:26.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.734+0000 7f4b48a6a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4a30f0 0x55bb3f552860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b48a6a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4b6340 0x55bb3f4b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b48a6a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4d3220 0x55bb3f4b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b48a6a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bb3f4b7810 con 0x55bb3f4a30f0 2024-05-08T05:48:26.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b48a6a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b402011d0 con 0x55bb3f4b6340 2024-05-08T05:48:26.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b48a6a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55bb3f4d2b60 con 0x55bb3f4d3220 2024-05-08T05:48:26.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b48a6a740 10 client.0 _ll_register_callbacks cb 0x7f4b3407b960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:48:26.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b3f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4a30f0 0x55bb3f552860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b3f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4a30f0 0x55bb3f552860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43430/0 (socket says 172.21.15.45:43430) 2024-05-08T05:48:26.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b3f7fe640 1 -- 172.21.15.45:0/895975436 learned_addr learned my addr 172.21.15.45:0/895975436 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:26.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b3effd640 1 --2- 172.21.15.45:0/895975436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4b6340 0x55bb3f4b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b3ffff640 1 --2- 172.21.15.45:0/895975436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4d3220 0x55bb3f4b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b3f7fe640 1 -- 172.21.15.45:0/895975436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4d3220 msgr2=0x55bb3f4b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b3f7fe640 1 --2- 172.21.15.45:0/895975436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4d3220 0x55bb3f4b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b3f7fe640 1 -- 172.21.15.45:0/895975436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4b6340 msgr2=0x55bb3f4b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b3f7fe640 1 --2- 172.21.15.45:0/895975436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4b6340 0x55bb3f4b0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b3f7fe640 1 -- 172.21.15.45:0/895975436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb3f551cf0 con 0x55bb3f4a30f0 2024-05-08T05:48:26.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b3ffff640 1 --2- 172.21.15.45:0/895975436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4d3220 0x55bb3f4b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:26.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b3f7fe640 1 --2- 172.21.15.45:0/895975436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4a30f0 0x55bb3f552860 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x55bb3f4e4270 tx=0x7f4b30017070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ad37fe69ba7469 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:26.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b1e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:48:26.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b1e7fc640 1 -- 172.21.15.45:0/895975436 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3001b070 con 0x55bb3f4a30f0 2024-05-08T05:48:26.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b1e7fc640 1 -- 172.21.15.45:0/895975436 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7f4b3002e590 con 0x55bb3f4a30f0 2024-05-08T05:48:26.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b48a6a740 1 -- 172.21.15.45:0/895975436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b3801b040 con 0x55bb3f4a30f0 2024-05-08T05:48:26.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b1e7fc640 1 -- 172.21.15.45:0/895975436 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f4b3003e050 con 0x55bb3f4a30f0 2024-05-08T05:48:26.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b48a6a740 10 client.7293 fetch_fsmap learned FSMap version 73 2024-05-08T05:48:26.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.735+0000 7f4b48a6a740 1 -- 172.21.15.45:0/895975436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55bb3f4df960 con 0x55bb3f4a30f0 2024-05-08T05:48:26.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.736+0000 7f4b1e7fc640 1 -- 172.21.15.45:0/895975436 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b3801b040 con 0x55bb3f4a30f0 2024-05-08T05:48:26.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.736+0000 7f4b1e7fc640 1 -- 172.21.15.45:0/895975436 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7f4b30041020 con 0x55bb3f4a30f0 2024-05-08T05:48:26.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.736+0000 7f4b48a6a740 10 client.7293 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T05:48:26.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.736+0000 7f4b48a6a740 10 client.7293 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-08T05:48:26.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.736+0000 7f4b48a6a740 20 client.7293 populate_metadata read hostname 'smithi045' 2024-05-08T05:48:26.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.736+0000 7f4b48a6a740 1 --2- 172.21.15.45:0/895975436 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x55bb3f4b82f0 0x55bb3f54e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.736+0000 7f4b48a6a740 4 client.7293 mds_command: new command op to 6747 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-05-08T05:48:26.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.736+0000 7f4b48a6a740 1 -- 172.21.15.45:0/895975436 --> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x55bb3f3f6770 con 0x55bb3f4b82f0 2024-05-08T05:48:26.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.736+0000 7f4b3effd640 1 --2- 172.21.15.45:0/895975436 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x55bb3f4b82f0 0x55bb3f54e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.736+0000 7f4b1e7fc640 10 client.7293 ms_handle_connect on v2:172.21.15.45:6833/2945763952 2024-05-08T05:48:26.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.736+0000 7f4b3effd640 1 --2- 172.21.15.45:0/895975436 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x55bb3f4b82f0 0x55bb3f54e700 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76e3c19ce9eb1094 server_cookie=79cc1af1a6cb6522 in_seq=0 out_seq=0 2024-05-08T05:48:26.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.737+0000 7f4b1e7fc640 1 -- 172.21.15.45:0/895975436 <== mds.0 v2:172.21.15.45:6833/2945763952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bb3f3f6770 con 0x55bb3f4b82f0 2024-05-08T05:48:26.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.737+0000 7f4b1e7fc640 10 client.7293 handle_command_reply: tid=0 2024-05-08T05:48:26.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.737+0000 7f4b48a6a740 1 client.7293 shutdown 2024-05-08T05:48:26.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.737+0000 7f4b48a6a740 1 -- 172.21.15.45:0/895975436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4a30f0 msgr2=0x55bb3f552860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.737+0000 7f4b48a6a740 1 --2- 172.21.15.45:0/895975436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4a30f0 0x55bb3f552860 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x55bb3f4e4270 tx=0x7f4b30017070 comp rx=0 tx=0).stop 2024-05-08T05:48:26.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.737+0000 7f4b48a6a740 1 -- 172.21.15.45:0/895975436 shutdown_connections 2024-05-08T05:48:26.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.737+0000 7f4b48a6a740 1 --2- 172.21.15.45:0/895975436 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x55bb3f4b82f0 0x55bb3f54e700 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.737+0000 7f4b48a6a740 1 --2- 172.21.15.45:0/895975436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55bb3f4d3220 0x55bb3f4b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.737+0000 7f4b48a6a740 1 --2- 172.21.15.45:0/895975436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55bb3f4b6340 0x55bb3f4b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.737+0000 7f4b48a6a740 1 --2- 172.21.15.45:0/895975436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bb3f4a30f0 0x55bb3f552860 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.737+0000 7f4b1e7fc640 0 client.7293 ms_handle_reset on v2:172.21.15.45:6833/2945763952 2024-05-08T05:48:26.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.737+0000 7f4b48a6a740 1 -- 172.21.15.45:0/895975436 >> 172.21.15.45:0/895975436 conn(0x55bb3f4b5c00 msgr2=0x55bb3f4b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:26.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.737+0000 7f4b48a6a740 1 -- 172.21.15.45:0/895975436 shutdown_connections 2024-05-08T05:48:26.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.737+0000 7f4b48a6a740 1 -- 172.21.15.45:0/895975436 wait complete. 2024-05-08T05:48:26.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.737+0000 7f4b48a6a740 20 client.7293 trim_cache size 0 max 16384 2024-05-08T05:48:26.743 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:48:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-05-08T05:48:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "f25f8a41-59a5-4238-ac0c-9f02c13b554d", 2024-05-08T05:48:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-05-08T05:48:26.744 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:48:26.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.738+0000 7f4b45d4e640 1 -- 172.21.15.45:0/930031804 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b2c076300 msgr2=0x7f4b2c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.738+0000 7f4b45d4e640 1 --2- 172.21.15.45:0/930031804 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b2c076300 0x7f4b2c0787c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009b60 tx=0x7f4b30009190 comp rx=0 tx=0).stop 2024-05-08T05:48:26.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.738+0000 7f4b45d4e640 1 -- 172.21.15.45:0/930031804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b40157a40 msgr2=0x7f4b401f85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.738+0000 7f4b45d4e640 1 --2- 172.21.15.45:0/930031804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b40157a40 0x7f4b401f85b0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f4b34001900 tx=0x7f4b340012f0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.738+0000 7f4b45d4e640 1 -- 172.21.15.45:0/930031804 shutdown_connections 2024-05-08T05:48:26.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.738+0000 7f4b45d4e640 1 --2- 172.21.15.45:0/930031804 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4b2c076300 0x7f4b2c0787c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.738+0000 7f4b45d4e640 1 --2- 172.21.15.45:0/930031804 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b40157ea0 0x7f4b401483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.738+0000 7f4b45d4e640 1 --2- 172.21.15.45:0/930031804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b40157a40 0x7f4b401f85b0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.738+0000 7f4b45d4e640 1 --2- 172.21.15.45:0/930031804 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b28004800 0x7f4b40153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.738+0000 7f4b45d4e640 1 -- 172.21.15.45:0/930031804 >> 172.21.15.45:0/930031804 conn(0x7f4b400b3aa0 msgr2=0x7f4b400b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:26.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.738+0000 7f4b45d4e640 1 -- 172.21.15.45:0/930031804 shutdown_connections 2024-05-08T05:48:26.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.738+0000 7f4b45d4e640 1 -- 172.21.15.45:0/930031804 wait complete. 2024-05-08T05:48:26.745 INFO:tasks.ceph.mds.c.smithi045.stderr:2024-05-08T05:48:26.738+0000 7fdbe6215640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x5607b28f8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-08T05:48:26.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:48:26.749 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-08T05:48:26.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.854+0000 7fa7ea971640 1 Processor -- start 2024-05-08T05:48:26.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.854+0000 7fa7ea971640 1 -- start start 2024-05-08T05:48:26.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.855+0000 7fa7ea971640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa7e40d0ec0 0x7fa7e40d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.855+0000 7fa7ea971640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7e40d1760 con 0x7fa7e40d3b00 2024-05-08T05:48:26.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.855+0000 7fa7ea971640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7e40d1a40 con 0x7fa7e415f380 2024-05-08T05:48:26.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.855+0000 7fa7ea971640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e40d30f0 con 0x7fa7e40d0ec0 2024-05-08T05:48:26.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.855+0000 7fa7e8f0e640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa7e40d3b00 0x7fa7e40b7e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48602/0 (socket says 172.21.15.45:48602) 2024-05-08T05:48:26.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.855+0000 7fa7e8f0e640 1 -- 172.21.15.45:0/1728618991 learned_addr learned my addr 172.21.15.45:0/1728618991 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:48:26.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e2ffd640 1 -- 172.21.15.45:0/1728618991 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4270498892 0 0) 0x7fa7e40d1760 con 0x7fa7e40d3b00 2024-05-08T05:48:26.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e37fe640 1 --2- 172.21.15.45:0/1728618991 >> v2:172.21.15.63:3300/0 conn(0x7fa7e40d0ec0 0x7fa7e40d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e2ffd640 1 -- 172.21.15.45:0/1728618991 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7cc003520 con 0x7fa7e40d3b00 2024-05-08T05:48:26.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e2ffd640 1 -- 172.21.15.45:0/1728618991 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2835689989 0 0) 0x7fa7cc003520 con 0x7fa7e40d3b00 2024-05-08T05:48:26.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e2ffd640 1 -- 172.21.15.45:0/1728618991 >> v2:172.21.15.63:3300/0 conn(0x7fa7e40d0ec0 msgr2=0x7fa7e40d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e2ffd640 1 --2- 172.21.15.45:0/1728618991 >> v2:172.21.15.63:3300/0 conn(0x7fa7e40d0ec0 0x7fa7e40d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e2ffd640 1 -- 172.21.15.45:0/1728618991 >> v1:172.21.15.63:6789/0 conn(0x7fa7e415f380 legacy=0x7fa7e40d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e2ffd640 1 -- 172.21.15.45:0/1728618991 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e40d1d20 con 0x7fa7e40d3b00 2024-05-08T05:48:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e2ffd640 1 -- 172.21.15.45:0/1728618991 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fa7dc001a70 con 0x7fa7e40d3b00 2024-05-08T05:48:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e2ffd640 1 -- 172.21.15.45:0/1728618991 >> v1:172.21.15.45:6789/0 conn(0x7fa7e40d3b00 legacy=0x7fa7e40b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e2ffd640 1 --2- 172.21.15.45:0/1728618991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7cc003a50 0x7fa7cc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e2ffd640 1 --2- 172.21.15.45:0/1728618991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7cc004800 0x7fa7cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e2ffd640 1 --2- 172.21.15.45:0/1728618991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7cc0051c0 0x7fa7cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e2ffd640 1 -- 172.21.15.45:0/1728618991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e40d30f0 con 0x7fa7cc003a50 2024-05-08T05:48:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e2ffd640 1 -- 172.21.15.45:0/1728618991 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc00c160 con 0x7fa7cc004800 2024-05-08T05:48:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e2ffd640 1 -- 172.21.15.45:0/1728618991 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa7cc00c360 con 0x7fa7cc0051c0 2024-05-08T05:48:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e3fff640 1 --2- 172.21.15.45:0/1728618991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7cc003a50 0x7fa7cc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e37fe640 1 --2- 172.21.15.45:0/1728618991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7cc004800 0x7fa7cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e8f0e640 1 --2- 172.21.15.45:0/1728618991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7cc0051c0 0x7fa7cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e3fff640 1 -- 172.21.15.45:0/1728618991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7cc0051c0 msgr2=0x7fa7cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e3fff640 1 --2- 172.21.15.45:0/1728618991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7cc0051c0 0x7fa7cc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.856+0000 7fa7e3fff640 1 -- 172.21.15.45:0/1728618991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7cc004800 msgr2=0x7fa7cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.857+0000 7fa7e3fff640 1 --2- 172.21.15.45:0/1728618991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7cc004800 0x7fa7cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.857+0000 7fa7e3fff640 1 -- 172.21.15.45:0/1728618991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7cc003520 con 0x7fa7cc003a50 2024-05-08T05:48:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.857+0000 7fa7e8f0e640 1 --2- 172.21.15.45:0/1728618991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7cc0051c0 0x7fa7cc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.857+0000 7fa7e8f0e640 1 -- 172.21.15.45:0/1728618991 reap_dead start 2024-05-08T05:48:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.857+0000 7fa7e3fff640 1 --2- 172.21.15.45:0/1728618991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7cc003a50 0x7fa7cc003e70 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4001790 tx=0x7fa7cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1db8d3a02d4f65 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.857+0000 7fa7e2ffd640 1 -- 172.21.15.45:0/1728618991 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40013b0 con 0x7fa7cc003a50 2024-05-08T05:48:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.857+0000 7fa7e2ffd640 1 -- 172.21.15.45:0/1728618991 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d40052d0 con 0x7fa7cc003a50 2024-05-08T05:48:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.857+0000 7fa7e2ffd640 1 -- 172.21.15.45:0/1728618991 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d402aac0 con 0x7fa7cc003a50 2024-05-08T05:48:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.858+0000 7fa7ea971640 1 -- 172.21.15.45:0/1728618991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7cc003a50 msgr2=0x7fa7cc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.858+0000 7fa7ea971640 1 --2- 172.21.15.45:0/1728618991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7cc003a50 0x7fa7cc003e70 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4001790 tx=0x7fa7cc005830 comp rx=0 tx=0).stop 2024-05-08T05:48:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.858+0000 7fa7ea971640 1 -- 172.21.15.45:0/1728618991 shutdown_connections 2024-05-08T05:48:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.858+0000 7fa7ea971640 1 --2- 172.21.15.45:0/1728618991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7cc003a50 0x7fa7cc003e70 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.858+0000 7fa7ea971640 1 -- 172.21.15.45:0/1728618991 >> 172.21.15.45:0/1728618991 conn(0x7fa7e40d4fe0 msgr2=0x7fa7e40d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.858+0000 7fa7ea971640 1 -- 172.21.15.45:0/1728618991 shutdown_connections 2024-05-08T05:48:26.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.858+0000 7fa7ea971640 1 -- 172.21.15.45:0/1728618991 wait complete. 2024-05-08T05:48:26.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.859+0000 7fa7ea971640 1 Processor -- start 2024-05-08T05:48:26.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.859+0000 7fa7ea971640 1 -- start start 2024-05-08T05:48:26.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.860+0000 7fa7ea971640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7cc003a50 0x7fa7e415e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.860+0000 7fa7ea971640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e415feb0 0x7fa7e40cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.860+0000 7fa7ea971640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7e40cde50 0x7fa7e41f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.860+0000 7fa7ea971640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e40d3f30 con 0x7fa7e415feb0 2024-05-08T05:48:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.860+0000 7fa7ea971640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e40d2370 con 0x7fa7e40cde50 2024-05-08T05:48:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.860+0000 7fa7ea971640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e415f760 con 0x7fa7cc003a50 2024-05-08T05:48:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.860+0000 7fa7e37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e415feb0 0x7fa7e40cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.860+0000 7fa7e37fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e415feb0 0x7fa7e40cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43450/0 (socket says 172.21.15.45:43450) 2024-05-08T05:48:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.860+0000 7fa7e37fe640 1 -- 172.21.15.45:0/2390957424 learned_addr learned my addr 172.21.15.45:0/2390957424 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.860+0000 7fa7e8f0e640 1 --2- 172.21.15.45:0/2390957424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7e40cde50 0x7fa7e41f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.860+0000 7fa7e3fff640 1 --2- 172.21.15.45:0/2390957424 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7cc003a50 0x7fa7e415e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.861+0000 7fa7e37fe640 1 -- 172.21.15.45:0/2390957424 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7cc003a50 msgr2=0x7fa7e415e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.861+0000 7fa7e37fe640 1 --2- 172.21.15.45:0/2390957424 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7cc003a50 0x7fa7e415e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.861+0000 7fa7e37fe640 1 -- 172.21.15.45:0/2390957424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7e40cde50 msgr2=0x7fa7e41f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.861+0000 7fa7e37fe640 1 --2- 172.21.15.45:0/2390957424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7e40cde50 0x7fa7e41f85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.861+0000 7fa7e37fe640 1 -- 172.21.15.45:0/2390957424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e409f0e0 con 0x7fa7e415feb0 2024-05-08T05:48:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.861+0000 7fa7e8f0e640 1 --2- 172.21.15.45:0/2390957424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7e40cde50 0x7fa7e41f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.861+0000 7fa7e3fff640 1 --2- 172.21.15.45:0/2390957424 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7cc003a50 0x7fa7e415e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.861+0000 7fa7e37fe640 1 --2- 172.21.15.45:0/2390957424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e415feb0 0x7fa7e40cd910 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8011ad0 tx=0x7fa7d8011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.861+0000 7fa7e0ff9640 1 -- 172.21.15.45:0/2390957424 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d800fb50 con 0x7fa7e415feb0 2024-05-08T05:48:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.861+0000 7fa7e0ff9640 1 -- 172.21.15.45:0/2390957424 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d800fcf0 con 0x7fa7e415feb0 2024-05-08T05:48:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.861+0000 7fa7ea971640 1 -- 172.21.15.45:0/2390957424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e409f3c0 con 0x7fa7e415feb0 2024-05-08T05:48:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.861+0000 7fa7e0ff9640 1 -- 172.21.15.45:0/2390957424 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d800fea0 con 0x7fa7e415feb0 2024-05-08T05:48:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.861+0000 7fa7ea971640 1 -- 172.21.15.45:0/2390957424 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e409f980 con 0x7fa7e415feb0 2024-05-08T05:48:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.863+0000 7fa7e0ff9640 1 -- 172.21.15.45:0/2390957424 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa7d8012040 con 0x7fa7e415feb0 2024-05-08T05:48:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.863+0000 7fa7e0ff9640 1 --2- 172.21.15.45:0/2390957424 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa7d0076300 0x7fa7d00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.863+0000 7fa7e0ff9640 1 -- 172.21.15.45:0/2390957424 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7fa7d8097ed0 con 0x7fa7e415feb0 2024-05-08T05:48:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.863+0000 7fa7e3fff640 1 --2- 172.21.15.45:0/2390957424 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa7d0076300 0x7fa7d00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.864+0000 7fa7ed763740 1 Processor -- start 2024-05-08T05:48:26.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.864+0000 7fa7ed763740 1 -- start start 2024-05-08T05:48:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.864+0000 7fa7e3fff640 1 --2- 172.21.15.45:0/2390957424 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa7d0076300 0x7fa7d00787c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4032000 tx=0x7fa7d402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.864+0000 7fa7ed763740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c5f160 0x55dd60c11af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.864+0000 7fa7ed763740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c42280 0x55dd60c41600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.865+0000 7fa7ed763740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c41b40 0x55dd60c16950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.865+0000 7fa7ed763740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dd60bde690 con 0x55dd60c41b40 2024-05-08T05:48:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.865+0000 7fa7ed763740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dd60b7bf30 con 0x55dd60c42280 2024-05-08T05:48:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.865+0000 7fa7ed763740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55dd60b34b20 con 0x55dd60c5f160 2024-05-08T05:48:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.865+0000 7fa7e8f0e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c41b40 0x55dd60c16950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.865+0000 7fa7e8f0e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c41b40 0x55dd60c16950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43458/0 (socket says 172.21.15.45:43458) 2024-05-08T05:48:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.865+0000 7fa7e8f0e640 1 -- 172.21.15.45:0/869350174 learned_addr learned my addr 172.21.15.45:0/869350174 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.865+0000 7fa7e3fff640 1 --2- 172.21.15.45:0/869350174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c5f160 0x55dd60c11af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.865+0000 7fa7e37fe640 1 --2- 172.21.15.45:0/869350174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c42280 0x55dd60c41600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.865+0000 7fa7e8f0e640 1 -- 172.21.15.45:0/869350174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c5f160 msgr2=0x55dd60c11af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.865+0000 7fa7e8f0e640 1 --2- 172.21.15.45:0/869350174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c5f160 0x55dd60c11af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.865+0000 7fa7e8f0e640 1 -- 172.21.15.45:0/869350174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c42280 msgr2=0x55dd60c41600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.865+0000 7fa7e8f0e640 1 --2- 172.21.15.45:0/869350174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c42280 0x55dd60c41600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.865+0000 7fa7e8f0e640 1 -- 172.21.15.45:0/869350174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd60b2ce60 con 0x55dd60c41b40 2024-05-08T05:48:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.865+0000 7fa7e37fe640 1 --2- 172.21.15.45:0/869350174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c42280 0x55dd60c41600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.865+0000 7fa7e8f0e640 1 --2- 172.21.15.45:0/869350174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c41b40 0x55dd60c16950 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc001a50 tx=0x7fa7dc005f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6835ec1db7fcb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.865+0000 7fa7c9ffb640 1 -- 172.21.15.45:0/869350174 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc01a070 con 0x55dd60c41b40 2024-05-08T05:48:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.865+0000 7fa7c9ffb640 1 -- 172.21.15.45:0/869350174 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7dc007720 con 0x55dd60c41b40 2024-05-08T05:48:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.865+0000 7fa7c9ffb640 1 -- 172.21.15.45:0/869350174 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc014400 con 0x55dd60c41b40 2024-05-08T05:48:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.866+0000 7fa7ed763740 1 -- 172.21.15.45:0/869350174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c41b40 msgr2=0x55dd60c16950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.866+0000 7fa7ed763740 1 --2- 172.21.15.45:0/869350174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c41b40 0x55dd60c16950 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc001a50 tx=0x7fa7dc005f40 comp rx=0 tx=0).stop 2024-05-08T05:48:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.866+0000 7fa7ed763740 1 -- 172.21.15.45:0/869350174 shutdown_connections 2024-05-08T05:48:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.866+0000 7fa7ed763740 1 --2- 172.21.15.45:0/869350174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c41b40 0x55dd60c16950 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.866+0000 7fa7ed763740 1 --2- 172.21.15.45:0/869350174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c42280 0x55dd60c41600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.866+0000 7fa7ed763740 1 --2- 172.21.15.45:0/869350174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c5f160 0x55dd60c11af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.866+0000 7fa7ed763740 1 -- 172.21.15.45:0/869350174 >> 172.21.15.45:0/869350174 conn(0x55dd60c13d60 msgr2=0x55dd60c5c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:26.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.866+0000 7fa7ed763740 1 -- 172.21.15.45:0/869350174 shutdown_connections 2024-05-08T05:48:26.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.866+0000 7fa7ed763740 1 -- 172.21.15.45:0/869350174 wait complete. 2024-05-08T05:48:26.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.866+0000 7fa7ed763740 1 -- start start 2024-05-08T05:48:26.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.867+0000 7fa7ed763740 1 Processor -- start 2024-05-08T05:48:26.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.867+0000 7fa7ed763740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c42280 0x55dd60c31ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.867+0000 7fa7ed763740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c5f160 0x55dd60c54800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.867+0000 7fa7ed763740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c3cf90 0x55dd60c44ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.867+0000 7fa7ed763740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dd60b385e0 con 0x55dd60c42280 2024-05-08T05:48:26.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.867+0000 7fa7ed763740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dd60b77ef0 con 0x55dd60c5f160 2024-05-08T05:48:26.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.867+0000 7fa7ed763740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55dd60c15f10 con 0x55dd60c3cf90 2024-05-08T05:48:26.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.867+0000 7fa7ed763740 10 client.0 _ll_register_callbacks cb 0x55dd60b33320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:48:26.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.868+0000 7fa7e3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c42280 0x55dd60c31ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.868+0000 7fa7e3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c42280 0x55dd60c31ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43464/0 (socket says 172.21.15.45:43464) 2024-05-08T05:48:26.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.868+0000 7fa7e3fff640 1 -- 172.21.15.45:0/1471732856 learned_addr learned my addr 172.21.15.45:0/1471732856 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:26.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.868+0000 7fa7e37fe640 1 --2- 172.21.15.45:0/1471732856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c5f160 0x55dd60c54800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.868+0000 7fa7e8f0e640 1 --2- 172.21.15.45:0/1471732856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c3cf90 0x55dd60c44ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.868+0000 7fa7e3fff640 1 -- 172.21.15.45:0/1471732856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c3cf90 msgr2=0x55dd60c44ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.868+0000 7fa7e3fff640 1 --2- 172.21.15.45:0/1471732856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c3cf90 0x55dd60c44ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.868+0000 7fa7e3fff640 1 -- 172.21.15.45:0/1471732856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c5f160 msgr2=0x55dd60c54800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.868+0000 7fa7e3fff640 1 --2- 172.21.15.45:0/1471732856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c5f160 0x55dd60c54800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.868+0000 7fa7e3fff640 1 -- 172.21.15.45:0/1471732856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd609577d0 con 0x55dd60c42280 2024-05-08T05:48:26.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.868+0000 7fa7e3fff640 1 --2- 172.21.15.45:0/1471732856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c42280 0x55dd60c31ea0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fa7d402a570 tx=0x7fa7d4001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea198af9bd839ae2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:26.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.868+0000 7fa7c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:48:26.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.868+0000 7fa7c2ffd640 1 -- 172.21.15.45:0/1471732856 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d404c070 con 0x55dd60c42280 2024-05-08T05:48:26.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.868+0000 7fa7c2ffd640 1 -- 172.21.15.45:0/1471732856 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7fa7d4007590 con 0x55dd60c42280 2024-05-08T05:48:26.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.868+0000 7fa7ed763740 1 -- 172.21.15.45:0/1471732856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7dc007720 con 0x55dd60c42280 2024-05-08T05:48:26.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.868+0000 7fa7c2ffd640 1 -- 172.21.15.45:0/1471732856 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fa7d4057050 con 0x55dd60c42280 2024-05-08T05:48:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.868+0000 7fa7ed763740 10 client.7305 fetch_fsmap learned FSMap version 73 2024-05-08T05:48:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.868+0000 7fa7ed763740 1 -- 172.21.15.45:0/1471732856 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55dd6088d3f0 con 0x55dd60c42280 2024-05-08T05:48:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.869+0000 7fa7c2ffd640 1 -- 172.21.15.45:0/1471732856 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7dc007720 con 0x55dd60c42280 2024-05-08T05:48:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.869+0000 7fa7c2ffd640 1 -- 172.21.15.45:0/1471732856 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7fa7d405a020 con 0x55dd60c42280 2024-05-08T05:48:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.869+0000 7fa7ed763740 10 client.7305 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T05:48:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.869+0000 7fa7ed763740 10 client.7305 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-08T05:48:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.869+0000 7fa7ed763740 20 client.7305 populate_metadata read hostname 'smithi045' 2024-05-08T05:48:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.869+0000 7fa7ed763740 1 --2- 172.21.15.45:0/1471732856 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x55dd60c6c6f0 0x55dd60c5d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.869+0000 7fa7ed763740 4 client.7305 mds_command: new command op to 6747 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:48:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.869+0000 7fa7ed763740 1 -- 172.21.15.45:0/1471732856 --> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd60b89b40 con 0x55dd60c6c6f0 2024-05-08T05:48:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.869+0000 7fa7e37fe640 1 --2- 172.21.15.45:0/1471732856 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x55dd60c6c6f0 0x55dd60c5d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.869+0000 7fa7c2ffd640 10 client.7305 ms_handle_connect on v2:172.21.15.45:6833/2945763952 2024-05-08T05:48:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.869+0000 7fa7e37fe640 1 --2- 172.21.15.45:0/1471732856 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x55dd60c6c6f0 0x55dd60c5d780 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=158302ca424cc5c1 server_cookie=30ee79bead3c620e in_seq=0 out_seq=0 2024-05-08T05:48:26.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.870+0000 7fa7c2ffd640 1 -- 172.21.15.45:0/1471732856 <== mds.0 v2:172.21.15.45:6833/2945763952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dd60b89b40 con 0x55dd60c6c6f0 2024-05-08T05:48:26.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.870+0000 7fa7c2ffd640 10 client.7305 handle_command_reply: tid=0 2024-05-08T05:48:26.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.870+0000 7fa7ed763740 1 client.7305 shutdown 2024-05-08T05:48:26.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.870+0000 7fa7ed763740 1 -- 172.21.15.45:0/1471732856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c42280 msgr2=0x55dd60c31ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.870+0000 7fa7ed763740 1 --2- 172.21.15.45:0/1471732856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c42280 0x55dd60c31ea0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fa7d402a570 tx=0x7fa7d4001190 comp rx=0 tx=0).stop 2024-05-08T05:48:26.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.870+0000 7fa7ed763740 1 -- 172.21.15.45:0/1471732856 shutdown_connections 2024-05-08T05:48:26.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.870+0000 7fa7ed763740 1 --2- 172.21.15.45:0/1471732856 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x55dd60c6c6f0 0x55dd60c5d780 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.870+0000 7fa7ed763740 1 --2- 172.21.15.45:0/1471732856 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c3cf90 0x55dd60c44ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.870+0000 7fa7ed763740 1 --2- 172.21.15.45:0/1471732856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c5f160 0x55dd60c54800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.870+0000 7fa7c2ffd640 0 client.7305 ms_handle_reset on v2:172.21.15.45:6833/2945763952 2024-05-08T05:48:26.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.870+0000 7fa7ed763740 1 --2- 172.21.15.45:0/1471732856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c42280 0x55dd60c31ea0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.870+0000 7fa7ed763740 1 -- 172.21.15.45:0/1471732856 >> 172.21.15.45:0/1471732856 conn(0x55dd60c41b40 msgr2=0x55dd60c11230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:26.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.870+0000 7fa7ed763740 1 -- 172.21.15.45:0/1471732856 shutdown_connections 2024-05-08T05:48:26.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.870+0000 7fa7ed763740 1 -- 172.21.15.45:0/1471732856 wait complete. 2024-05-08T05:48:26.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.870+0000 7fa7ed763740 20 client.7305 trim_cache size 0 max 16384 2024-05-08T05:48:26.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.889+0000 7fa7ed763740 1 Processor -- start 2024-05-08T05:48:26.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.889+0000 7fa7ed763740 1 -- start start 2024-05-08T05:48:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.889+0000 7fa7ed763740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c3cf90 0x55dd60c54f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.889+0000 7fa7ed763740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c42280 0x55dd60c5fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.889+0000 7fa7ed763740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c5f160 0x55dd60c5ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.889+0000 7fa7ed763740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e4202a40 con 0x55dd60c3cf90 2024-05-08T05:48:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.889+0000 7fa7ed763740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e4201a20 con 0x55dd60c42280 2024-05-08T05:48:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.889+0000 7fa7ed763740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e4200d80 con 0x55dd60c5f160 2024-05-08T05:48:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.889+0000 7fa7e3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c3cf90 0x55dd60c54f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.889+0000 7fa7e3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c3cf90 0x55dd60c54f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43466/0 (socket says 172.21.15.45:43466) 2024-05-08T05:48:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.889+0000 7fa7e3fff640 1 -- 172.21.15.45:0/567841785 learned_addr learned my addr 172.21.15.45:0/567841785 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.890+0000 7fa7e37fe640 1 --2- 172.21.15.45:0/567841785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c42280 0x55dd60c5fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.890+0000 7fa7e8f0e640 1 --2- 172.21.15.45:0/567841785 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c5f160 0x55dd60c5ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.890+0000 7fa7e3fff640 1 -- 172.21.15.45:0/567841785 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c5f160 msgr2=0x55dd60c5ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.890+0000 7fa7e3fff640 1 --2- 172.21.15.45:0/567841785 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c5f160 0x55dd60c5ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.890+0000 7fa7e3fff640 1 -- 172.21.15.45:0/567841785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c42280 msgr2=0x55dd60c5fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.890+0000 7fa7e3fff640 1 --2- 172.21.15.45:0/567841785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c42280 0x55dd60c5fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.890+0000 7fa7e3fff640 1 -- 172.21.15.45:0/567841785 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd6084d0f0 con 0x55dd60c3cf90 2024-05-08T05:48:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.890+0000 7fa7e3fff640 1 --2- 172.21.15.45:0/567841785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c3cf90 0x55dd60c54f10 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fa7d402afc0 tx=0x7fa7d4048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca41ed59338686ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.890+0000 7fa7c97fa640 1 -- 172.21.15.45:0/567841785 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d404c070 con 0x55dd60c3cf90 2024-05-08T05:48:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.890+0000 7fa7c97fa640 1 -- 172.21.15.45:0/567841785 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d4060070 con 0x55dd60c3cf90 2024-05-08T05:48:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.890+0000 7fa7c97fa640 1 -- 172.21.15.45:0/567841785 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40035f0 con 0x55dd60c3cf90 2024-05-08T05:48:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.891+0000 7fa7ed763740 1 -- 172.21.15.45:0/567841785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c3cf90 msgr2=0x55dd60c54f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.891+0000 7fa7ed763740 1 --2- 172.21.15.45:0/567841785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c3cf90 0x55dd60c54f10 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fa7d402afc0 tx=0x7fa7d4048000 comp rx=0 tx=0).stop 2024-05-08T05:48:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.891+0000 7fa7ed763740 1 -- 172.21.15.45:0/567841785 shutdown_connections 2024-05-08T05:48:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.891+0000 7fa7ed763740 1 --2- 172.21.15.45:0/567841785 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c5f160 0x55dd60c5ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.891+0000 7fa7ed763740 1 --2- 172.21.15.45:0/567841785 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c42280 0x55dd60c5fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.891+0000 7fa7ed763740 1 --2- 172.21.15.45:0/567841785 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c3cf90 0x55dd60c54f10 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.891+0000 7fa7ed763740 1 -- 172.21.15.45:0/567841785 >> 172.21.15.45:0/567841785 conn(0x55dd60c41b40 msgr2=0x55dd60c12830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.891+0000 7fa7ed763740 1 -- 172.21.15.45:0/567841785 shutdown_connections 2024-05-08T05:48:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.891+0000 7fa7ed763740 1 -- 172.21.15.45:0/567841785 wait complete. 2024-05-08T05:48:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.891+0000 7fa7ed763740 1 -- start start 2024-05-08T05:48:26.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.891+0000 7fa7ed763740 1 Processor -- start 2024-05-08T05:48:26.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.892+0000 7fa7ed763740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c3cf90 0x55dd60c109a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.892+0000 7fa7ed763740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c42280 0x55dd60cdccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.892+0000 7fa7ed763740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c5f160 0x55dd60c3d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.892+0000 7fa7ed763740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e42013d0 con 0x55dd60c3cf90 2024-05-08T05:48:26.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.892+0000 7fa7ed763740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dd60c459d0 con 0x55dd60c5f160 2024-05-08T05:48:26.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.892+0000 7fa7ed763740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55dd60cdd890 con 0x55dd60c42280 2024-05-08T05:48:26.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.892+0000 7fa7ed763740 10 client.0 _ll_register_callbacks cb 0x7fa7cc004680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:48:26.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.892+0000 7fa7e3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c3cf90 0x55dd60c109a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.892+0000 7fa7e3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c3cf90 0x55dd60c109a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43480/0 (socket says 172.21.15.45:43480) 2024-05-08T05:48:26.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.892+0000 7fa7e3fff640 1 -- 172.21.15.45:0/3385825711 learned_addr learned my addr 172.21.15.45:0/3385825711 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:26.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.892+0000 7fa7e8f0e640 1 --2- 172.21.15.45:0/3385825711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c5f160 0x55dd60c3d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.892+0000 7fa7e37fe640 1 --2- 172.21.15.45:0/3385825711 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c42280 0x55dd60cdccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7e3fff640 1 -- 172.21.15.45:0/3385825711 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c42280 msgr2=0x55dd60cdccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7e3fff640 1 --2- 172.21.15.45:0/3385825711 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c42280 0x55dd60cdccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7e3fff640 1 -- 172.21.15.45:0/3385825711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c5f160 msgr2=0x55dd60c3d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:26.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7e3fff640 1 --2- 172.21.15.45:0/3385825711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c5f160 0x55dd60c3d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7e3fff640 1 -- 172.21.15.45:0/3385825711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd60c0d310 con 0x55dd60c3cf90 2024-05-08T05:48:26.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7e8f0e640 1 --2- 172.21.15.45:0/3385825711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c5f160 0x55dd60c3d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:26.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7e37fe640 1 --2- 172.21.15.45:0/3385825711 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c42280 0x55dd60cdccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:26.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7e3fff640 1 --2- 172.21.15.45:0/3385825711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c3cf90 0x55dd60c109a0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fa7d402a570 tx=0x7fa7d403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1260677aa4fc7f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:26.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:48:26.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7c2ffd640 1 -- 172.21.15.45:0/3385825711 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d404c040 con 0x55dd60c3cf90 2024-05-08T05:48:26.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7c2ffd640 1 -- 172.21.15.45:0/3385825711 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7fa7d4052b90 con 0x55dd60c3cf90 2024-05-08T05:48:26.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7ed763740 1 -- 172.21.15.45:0/3385825711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7d4060070 con 0x55dd60c3cf90 2024-05-08T05:48:26.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7c2ffd640 1 -- 172.21.15.45:0/3385825711 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fa7d4071050 con 0x55dd60c3cf90 2024-05-08T05:48:26.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7ed763740 10 client.7311 fetch_fsmap learned FSMap version 73 2024-05-08T05:48:26.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7ed763740 1 -- 172.21.15.45:0/3385825711 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55dd60cdd3a0 con 0x55dd60c3cf90 2024-05-08T05:48:26.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7c2ffd640 1 -- 172.21.15.45:0/3385825711 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d4060070 con 0x55dd60c3cf90 2024-05-08T05:48:26.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7c2ffd640 1 -- 172.21.15.45:0/3385825711 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7fa7d4074020 con 0x55dd60c3cf90 2024-05-08T05:48:26.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7ed763740 10 client.7311 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T05:48:26.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7ed763740 10 client.7311 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-08T05:48:26.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7ed763740 20 client.7311 populate_metadata read hostname 'smithi045' 2024-05-08T05:48:26.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7ed763740 1 --2- 172.21.15.45:0/3385825711 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x55dd60c3c710 0x55dd60c3de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:26.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7ed763740 4 client.7311 mds_command: new command op to 6747 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:48:26.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.893+0000 7fa7ed763740 1 -- 172.21.15.45:0/3385825711 --> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dd60b839b0 con 0x55dd60c3c710 2024-05-08T05:48:26.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.894+0000 7fa7e37fe640 1 --2- 172.21.15.45:0/3385825711 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x55dd60c3c710 0x55dd60c3de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:26.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.894+0000 7fa7c2ffd640 10 client.7311 ms_handle_connect on v2:172.21.15.45:6833/2945763952 2024-05-08T05:48:26.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.894+0000 7fa7e37fe640 1 --2- 172.21.15.45:0/3385825711 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x55dd60c3c710 0x55dd60c3de80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d77c898107c608 server_cookie=b0cf6fa34219b251 in_seq=0 out_seq=0 2024-05-08T05:48:26.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.894+0000 7fa7c2ffd640 1 -- 172.21.15.45:0/3385825711 <== mds.0 v2:172.21.15.45:6833/2945763952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x55dd60b839b0 con 0x55dd60c3c710 2024-05-08T05:48:26.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.894+0000 7fa7c2ffd640 10 client.7311 handle_command_reply: tid=0 2024-05-08T05:48:26.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.894+0000 7fa7ed763740 1 client.7311 shutdown 2024-05-08T05:48:26.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.894+0000 7fa7ed763740 1 -- 172.21.15.45:0/3385825711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c3cf90 msgr2=0x55dd60c109a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.894+0000 7fa7ed763740 1 --2- 172.21.15.45:0/3385825711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c3cf90 0x55dd60c109a0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fa7d402a570 tx=0x7fa7d403c000 comp rx=0 tx=0).stop 2024-05-08T05:48:26.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.894+0000 7fa7ed763740 1 -- 172.21.15.45:0/3385825711 shutdown_connections 2024-05-08T05:48:26.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.894+0000 7fa7ed763740 1 --2- 172.21.15.45:0/3385825711 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x55dd60c3c710 0x55dd60c3de80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.894+0000 7fa7ed763740 1 --2- 172.21.15.45:0/3385825711 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dd60c42280 0x55dd60cdccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.894+0000 7fa7ed763740 1 --2- 172.21.15.45:0/3385825711 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dd60c5f160 0x55dd60c3d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.894+0000 7fa7ed763740 1 --2- 172.21.15.45:0/3385825711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dd60c3cf90 0x55dd60c109a0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.894+0000 7fa7c2ffd640 0 client.7311 ms_handle_reset on v2:172.21.15.45:6833/2945763952 2024-05-08T05:48:26.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.894+0000 7fa7ed763740 1 -- 172.21.15.45:0/3385825711 >> 172.21.15.45:0/3385825711 conn(0x55dd60c41b40 msgr2=0x55dd60c5fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:26.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.894+0000 7fa7ed763740 1 -- 172.21.15.45:0/3385825711 shutdown_connections 2024-05-08T05:48:26.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.894+0000 7fa7ed763740 1 -- 172.21.15.45:0/3385825711 wait complete. 2024-05-08T05:48:26.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.895+0000 7fa7ed763740 20 client.7311 trim_cache size 0 max 16384 2024-05-08T05:48:26.902 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:48:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T05:48:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-05-08T05:48:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "f25f8a41-59a5-4238-ac0c-9f02c13b554d": { 2024-05-08T05:48:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/dir", 2024-05-08T05:48:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "f25f8a41-59a5-4238-ac0c-9f02c13b554d", 2024-05-08T05:48:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,repair" 2024-05-08T05:48:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:48:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:48:26.904 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:48:26.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.895+0000 7fa7ea971640 1 -- 172.21.15.45:0/2390957424 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa7d0076300 msgr2=0x7fa7d00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.895+0000 7fa7ea971640 1 --2- 172.21.15.45:0/2390957424 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa7d0076300 0x7fa7d00787c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4032000 tx=0x7fa7d402e000 comp rx=0 tx=0).stop 2024-05-08T05:48:26.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.895+0000 7fa7ea971640 1 -- 172.21.15.45:0/2390957424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e415feb0 msgr2=0x7fa7e40cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:26.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.895+0000 7fa7ea971640 1 --2- 172.21.15.45:0/2390957424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e415feb0 0x7fa7e40cd910 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8011ad0 tx=0x7fa7d8011f90 comp rx=0 tx=0).stop 2024-05-08T05:48:26.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.895+0000 7fa7ea971640 1 -- 172.21.15.45:0/2390957424 shutdown_connections 2024-05-08T05:48:26.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.895+0000 7fa7ea971640 1 --2- 172.21.15.45:0/2390957424 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa7d0076300 0x7fa7d00787c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.895+0000 7fa7ea971640 1 --2- 172.21.15.45:0/2390957424 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa7e40cde50 0x7fa7e41f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.895+0000 7fa7ea971640 1 --2- 172.21.15.45:0/2390957424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7e415feb0 0x7fa7e40cd910 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.895+0000 7fa7ea971640 1 --2- 172.21.15.45:0/2390957424 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa7cc003a50 0x7fa7e415e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:26.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.895+0000 7fa7ea971640 1 -- 172.21.15.45:0/2390957424 >> 172.21.15.45:0/2390957424 conn(0x7fa7e40d4fe0 msgr2=0x7fa7e40d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:26.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.895+0000 7fa7ea971640 1 -- 172.21.15.45:0/2390957424 shutdown_connections 2024-05-08T05:48:26.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:26.895+0000 7fa7ea971640 1 -- 172.21.15.45:0/2390957424 wait complete. 2024-05-08T05:48:26.909 INFO:tasks.cephfs.filesystem:scrub status for tag:f25f8a41-59a5-4238-ac0c-9f02c13b554d - {'path': '/dir', 'tag': 'f25f8a41-59a5-4238-ac0c-9f02c13b554d', 'options': 'recursive,repair'} 2024-05-08T05:48:34.937 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:48:34.940 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:48:56.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:48:56.910 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-05-08T05:48:57.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.031+0000 7f4d92015640 1 Processor -- start 2024-05-08T05:48:57.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.031+0000 7f4d92015640 1 -- start start 2024-05-08T05:48:57.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.031+0000 7f4d92015640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4d8c157540 0x7f4d8c15e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.031+0000 7f4d92015640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4d8c0b48a0 0x7f4d8c0ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.032+0000 7f4d92015640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d8c15e550 con 0x7f4d8c06fc30 2024-05-08T05:48:57.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.032+0000 7f4d92015640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4d8c0701d0 con 0x7f4d8c157540 2024-05-08T05:48:57.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.032+0000 7f4d92015640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4d8c0b3dc0 con 0x7f4d8c0b48a0 2024-05-08T05:48:57.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.032+0000 7f4d8bfff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4d8c0b48a0 0x7f4d8c0ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.032+0000 7f4d8b7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4d8c157540 0x7f4d8c15e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.032+0000 7f4d8bfff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4d8c0b48a0 0x7f4d8c0ced20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:35274/0 (socket says 172.21.15.45:35274) 2024-05-08T05:48:57.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.032+0000 7f4d8bfff640 1 -- 172.21.15.45:0/558735221 learned_addr learned my addr 172.21.15.45:0/558735221 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:57.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.032+0000 7f4d8affd640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f4d8c06fc30 0x7f4d8c0af740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:34290/0 (socket says 172.21.15.45:34290) 2024-05-08T05:48:57.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.032+0000 7f4d8a7fc640 1 -- 172.21.15.45:0/558735221 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1092578956 0 0) 0x7f4d80001110 con 0x7f4d8c06fc30 2024-05-08T05:48:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.032+0000 7f4d8a7fc640 1 -- 172.21.15.45:0/558735221 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d74003520 con 0x7f4d8c06fc30 2024-05-08T05:48:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.032+0000 7f4d8bfff640 1 -- 172.21.15.45:0/558735221 >> v2:172.21.15.63:3300/0 conn(0x7f4d8c157540 msgr2=0x7f4d8c15e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.032+0000 7f4d8bfff640 1 --2- 172.21.15.45:0/558735221 >> v2:172.21.15.63:3300/0 conn(0x7f4d8c157540 0x7f4d8c15e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.032+0000 7f4d8bfff640 1 -- 172.21.15.45:0/558735221 >> v1:172.21.15.63:6789/0 conn(0x7f4d8c06fc30 legacy=0x7f4d8c0af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.032+0000 7f4d8bfff640 1 -- 172.21.15.45:0/558735221 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8c0b40a0 con 0x7f4d8c0b48a0 2024-05-08T05:48:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.032+0000 7f4d8b7fe640 1 --2- 172.21.15.45:0/558735221 >> v2:172.21.15.63:3300/0 conn(0x7f4d8c157540 0x7f4d8c15e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:48:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.033+0000 7f4d8bfff640 1 --2- 172.21.15.45:0/558735221 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4d8c0b48a0 0x7f4d8c0ced20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4d84009e50 tx=0x7f4d8402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b818e25c73098e44 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.033+0000 7f4d8a7fc640 1 -- 172.21.15.45:0/558735221 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84036070 con 0x7f4d8c0b48a0 2024-05-08T05:48:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.033+0000 7f4d8a7fc640 1 -- 172.21.15.45:0/558735221 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d84031600 con 0x7f4d8c0b48a0 2024-05-08T05:48:57.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.033+0000 7f4d8a7fc640 1 -- 172.21.15.45:0/558735221 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d840319d0 con 0x7f4d8c0b48a0 2024-05-08T05:48:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.034+0000 7f4d92015640 1 -- 172.21.15.45:0/558735221 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4d8c0b48a0 msgr2=0x7f4d8c0ced20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.034+0000 7f4d92015640 1 --2- 172.21.15.45:0/558735221 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4d8c0b48a0 0x7f4d8c0ced20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4d84009e50 tx=0x7f4d8402a710 comp rx=0 tx=0).stop 2024-05-08T05:48:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.034+0000 7f4d92015640 1 -- 172.21.15.45:0/558735221 shutdown_connections 2024-05-08T05:48:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.034+0000 7f4d92015640 1 --2- 172.21.15.45:0/558735221 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4d8c0b48a0 0x7f4d8c0ced20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.034+0000 7f4d92015640 1 --2- 172.21.15.45:0/558735221 >> v2:172.21.15.63:3300/0 conn(0x7f4d8c157540 0x7f4d8c15e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.034+0000 7f4d92015640 1 -- 172.21.15.45:0/558735221 >> 172.21.15.45:0/558735221 conn(0x7f4d8c06c3a0 msgr2=0x7f4d8c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.034+0000 7f4d92015640 1 -- 172.21.15.45:0/558735221 shutdown_connections 2024-05-08T05:48:57.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.034+0000 7f4d92015640 1 -- 172.21.15.45:0/558735221 wait complete. 2024-05-08T05:48:57.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.035+0000 7f4d92015640 1 Processor -- start 2024-05-08T05:48:57.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.036+0000 7f4d92015640 1 -- start start 2024-05-08T05:48:57.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.036+0000 7f4d92015640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d8c06fc30 0x7f4d8c14e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.036+0000 7f4d92015640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4d8c0b48a0 0x7f4d8c152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.036+0000 7f4d92015640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d8c157540 0x7f4d8c153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.036+0000 7f4d92015640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c157aa0 con 0x7f4d8c06fc30 2024-05-08T05:48:57.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.036+0000 7f4d92015640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c157920 con 0x7f4d8c157540 2024-05-08T05:48:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.036+0000 7f4d92015640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4d8c0afe50 con 0x7f4d8c0b48a0 2024-05-08T05:48:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.036+0000 7f4d8b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d8c06fc30 0x7f4d8c14e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.036+0000 7f4d8b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d8c06fc30 0x7f4d8c14e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60048/0 (socket says 172.21.15.45:60048) 2024-05-08T05:48:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.036+0000 7f4d8b7fe640 1 -- 172.21.15.45:0/374107658 learned_addr learned my addr 172.21.15.45:0/374107658 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.037+0000 7f4d8affd640 1 --2- 172.21.15.45:0/374107658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4d8c0b48a0 0x7f4d8c152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.037+0000 7f4d8bfff640 1 --2- 172.21.15.45:0/374107658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d8c157540 0x7f4d8c153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.037+0000 7f4d8b7fe640 1 -- 172.21.15.45:0/374107658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4d8c0b48a0 msgr2=0x7f4d8c152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.037+0000 7f4d8b7fe640 1 --2- 172.21.15.45:0/374107658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4d8c0b48a0 0x7f4d8c152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.037+0000 7f4d8b7fe640 1 -- 172.21.15.45:0/374107658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d8c157540 msgr2=0x7f4d8c153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.037+0000 7f4d8b7fe640 1 --2- 172.21.15.45:0/374107658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d8c157540 0x7f4d8c153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.037+0000 7f4d8b7fe640 1 -- 172.21.15.45:0/374107658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8c1f9250 con 0x7f4d8c06fc30 2024-05-08T05:48:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.037+0000 7f4d8b7fe640 1 --2- 172.21.15.45:0/374107658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d8c06fc30 0x7f4d8c14e7e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00e960 tx=0x7f4d7c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.037+0000 7f4d73fff640 1 -- 172.21.15.45:0/374107658 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c017070 con 0x7f4d8c06fc30 2024-05-08T05:48:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.037+0000 7f4d73fff640 1 -- 172.21.15.45:0/374107658 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7c00ca00 con 0x7f4d8c06fc30 2024-05-08T05:48:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.037+0000 7f4d92015640 1 -- 172.21.15.45:0/374107658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d8c1f8490 con 0x7f4d8c06fc30 2024-05-08T05:48:57.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.037+0000 7f4d73fff640 1 -- 172.21.15.45:0/374107658 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c00cdd0 con 0x7f4d8c06fc30 2024-05-08T05:48:57.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.037+0000 7f4d92015640 1 -- 172.21.15.45:0/374107658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d8c1f8a50 con 0x7f4d8c06fc30 2024-05-08T05:48:57.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.041+0000 7f4d73fff640 1 -- 172.21.15.45:0/374107658 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4d7c01a070 con 0x7f4d8c06fc30 2024-05-08T05:48:57.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.041+0000 7f4d73fff640 1 --2- 172.21.15.45:0/374107658 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4d780762d0 0x7f4d78078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.041+0000 7f4d73fff640 1 -- 172.21.15.45:0/374107658 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7f4d7c097630 con 0x7f4d8c06fc30 2024-05-08T05:48:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.041+0000 7f4d8affd640 1 --2- 172.21.15.45:0/374107658 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4d780762d0 0x7f4d78078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.041+0000 7f4d94e0a740 1 Processor -- start 2024-05-08T05:48:57.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.041+0000 7f4d94e0a740 1 -- start start 2024-05-08T05:48:57.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.041+0000 7f4d8affd640 1 --2- 172.21.15.45:0/374107658 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4d780762d0 0x7f4d78078790 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c15acc0 tx=0x7f4d800047e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.042+0000 7f4d94e0a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a8c160 0x556282a3eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.042+0000 7f4d94e0a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556282a6f280 0x556282a6e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.042+0000 7f4d94e0a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556282a6eb40 0x556282a43950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.042+0000 7f4d94e0a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556282a0b690 con 0x556282a8c160 2024-05-08T05:48:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.042+0000 7f4d94e0a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5562829a8f30 con 0x556282a6f280 2024-05-08T05:48:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.042+0000 7f4d94e0a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x556282961b20 con 0x556282a6eb40 2024-05-08T05:48:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.042+0000 7f4d8b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a8c160 0x556282a3eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.042+0000 7f4d8b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a8c160 0x556282a3eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60050/0 (socket says 172.21.15.45:60050) 2024-05-08T05:48:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.042+0000 7f4d8affd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556282a6f280 0x556282a6e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.042+0000 7f4d8b7fe640 1 -- 172.21.15.45:0/3870133571 learned_addr learned my addr 172.21.15.45:0/3870133571 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:57.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.042+0000 7f4d8bfff640 1 --2- 172.21.15.45:0/3870133571 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556282a6eb40 0x556282a43950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.042+0000 7f4d8b7fe640 1 -- 172.21.15.45:0/3870133571 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556282a6eb40 msgr2=0x556282a43950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.042+0000 7f4d8b7fe640 1 --2- 172.21.15.45:0/3870133571 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556282a6eb40 0x556282a43950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.042+0000 7f4d8b7fe640 1 -- 172.21.15.45:0/3870133571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556282a6f280 msgr2=0x556282a6e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.042+0000 7f4d8b7fe640 1 --2- 172.21.15.45:0/3870133571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556282a6f280 0x556282a6e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.042+0000 7f4d8b7fe640 1 -- 172.21.15.45:0/3870133571 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556282959e60 con 0x556282a8c160 2024-05-08T05:48:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.042+0000 7f4d8b7fe640 1 --2- 172.21.15.45:0/3870133571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a8c160 0x556282a3eaf0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c05cd10 tx=0x7f4d7c05c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627e8541b6c4b6b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.042+0000 7f4d717fa640 1 -- 172.21.15.45:0/3870133571 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c021070 con 0x556282a8c160 2024-05-08T05:48:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.042+0000 7f4d717fa640 1 -- 172.21.15.45:0/3870133571 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7c064a20 con 0x556282a8c160 2024-05-08T05:48:57.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.042+0000 7f4d717fa640 1 -- 172.21.15.45:0/3870133571 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c069440 con 0x556282a8c160 2024-05-08T05:48:57.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.043+0000 7f4d94e0a740 1 -- 172.21.15.45:0/3870133571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a8c160 msgr2=0x556282a3eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.043+0000 7f4d94e0a740 1 --2- 172.21.15.45:0/3870133571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a8c160 0x556282a3eaf0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c05cd10 tx=0x7f4d7c05c6f0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.043+0000 7f4d94e0a740 1 -- 172.21.15.45:0/3870133571 shutdown_connections 2024-05-08T05:48:57.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.043+0000 7f4d94e0a740 1 --2- 172.21.15.45:0/3870133571 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556282a6eb40 0x556282a43950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.043+0000 7f4d94e0a740 1 --2- 172.21.15.45:0/3870133571 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556282a6f280 0x556282a6e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.043+0000 7f4d94e0a740 1 --2- 172.21.15.45:0/3870133571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a8c160 0x556282a3eaf0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.043+0000 7f4d94e0a740 1 -- 172.21.15.45:0/3870133571 >> 172.21.15.45:0/3870133571 conn(0x556282a40d60 msgr2=0x556282a89600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:57.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.043+0000 7f4d94e0a740 1 -- 172.21.15.45:0/3870133571 shutdown_connections 2024-05-08T05:48:57.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.043+0000 7f4d94e0a740 1 -- 172.21.15.45:0/3870133571 wait complete. 2024-05-08T05:48:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.044+0000 7f4d94e0a740 1 -- start start 2024-05-08T05:48:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.044+0000 7f4d94e0a740 1 Processor -- start 2024-05-08T05:48:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d94e0a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a6f280 0x556282a816c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d94e0a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556282a8c160 0x556282a8ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d94e0a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556282a5c010 0x556282a8cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d94e0a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5562829655e0 con 0x556282a6f280 2024-05-08T05:48:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d94e0a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5562829a4ef0 con 0x556282a8c160 2024-05-08T05:48:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d94e0a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x556282a42f10 con 0x556282a5c010 2024-05-08T05:48:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d8b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a6f280 0x556282a816c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d8b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a6f280 0x556282a816c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60054/0 (socket says 172.21.15.45:60054) 2024-05-08T05:48:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d94e0a740 10 client.0 _ll_register_callbacks cb 0x556282960320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:48:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d8b7fe640 1 -- 172.21.15.45:0/1114951337 learned_addr learned my addr 172.21.15.45:0/1114951337 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d8affd640 1 --2- 172.21.15.45:0/1114951337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556282a8c160 0x556282a8ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d8bfff640 1 --2- 172.21.15.45:0/1114951337 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556282a5c010 0x556282a8cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d8b7fe640 1 -- 172.21.15.45:0/1114951337 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556282a5c010 msgr2=0x556282a8cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d8b7fe640 1 --2- 172.21.15.45:0/1114951337 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556282a5c010 0x556282a8cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d8b7fe640 1 -- 172.21.15.45:0/1114951337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556282a8c160 msgr2=0x556282a8ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d8b7fe640 1 --2- 172.21.15.45:0/1114951337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556282a8c160 0x556282a8ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d8b7fe640 1 -- 172.21.15.45:0/1114951337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562827847d0 con 0x556282a6f280 2024-05-08T05:48:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d8bfff640 1 --2- 172.21.15.45:0/1114951337 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556282a5c010 0x556282a8cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d8b7fe640 1 --2- 172.21.15.45:0/1114951337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a6f280 0x556282a816c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c095990 tx=0x7f4d7c097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c2fabea8aeba18 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d6a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:48:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.045+0000 7f4d6a7fc640 1 -- 172.21.15.45:0/1114951337 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c095370 con 0x556282a6f280 2024-05-08T05:48:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.046+0000 7f4d6a7fc640 1 -- 172.21.15.45:0/1114951337 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7f4d7c095510 con 0x556282a6f280 2024-05-08T05:48:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.046+0000 7f4d94e0a740 1 -- 172.21.15.45:0/1114951337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d7c064a20 con 0x556282a6f280 2024-05-08T05:48:57.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.046+0000 7f4d6a7fc640 1 -- 172.21.15.45:0/1114951337 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f4d7c05f980 con 0x556282a6f280 2024-05-08T05:48:57.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.046+0000 7f4d94e0a740 10 client.7320 fetch_fsmap learned FSMap version 73 2024-05-08T05:48:57.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.046+0000 7f4d94e0a740 1 -- 172.21.15.45:0/1114951337 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x5562826ba3f0 con 0x556282a6f280 2024-05-08T05:48:57.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.046+0000 7f4d6a7fc640 1 -- 172.21.15.45:0/1114951337 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7c064a20 con 0x556282a6f280 2024-05-08T05:48:57.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.046+0000 7f4d6a7fc640 1 -- 172.21.15.45:0/1114951337 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7f4d7c024060 con 0x556282a6f280 2024-05-08T05:48:57.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.046+0000 7f4d94e0a740 10 client.7320 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T05:48:57.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.046+0000 7f4d94e0a740 10 client.7320 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-08T05:48:57.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.046+0000 7f4d94e0a740 20 client.7320 populate_metadata read hostname 'smithi045' 2024-05-08T05:48:57.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.046+0000 7f4d94e0a740 1 --2- 172.21.15.45:0/1114951337 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x556282a71300 0x556282a71700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.046+0000 7f4d94e0a740 4 client.7320 mds_command: new command op to 6747 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:48:57.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.046+0000 7f4d94e0a740 1 -- 172.21.15.45:0/1114951337 --> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562829b6b40 con 0x556282a71300 2024-05-08T05:48:57.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.046+0000 7f4d8affd640 1 --2- 172.21.15.45:0/1114951337 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x556282a71300 0x556282a71700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.046+0000 7f4d8affd640 1 --2- 172.21.15.45:0/1114951337 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x556282a71300 0x556282a71700 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98af50c5d9051365 server_cookie=652ffa0b4cd39837 in_seq=0 out_seq=0 2024-05-08T05:48:57.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.046+0000 7f4d6a7fc640 10 client.7320 ms_handle_connect on v2:172.21.15.45:6833/2945763952 2024-05-08T05:48:57.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.047+0000 7f4d6a7fc640 1 -- 172.21.15.45:0/1114951337 <== mds.0 v2:172.21.15.45:6833/2945763952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5562829b6b40 con 0x556282a71300 2024-05-08T05:48:57.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.047+0000 7f4d6a7fc640 10 client.7320 handle_command_reply: tid=0 2024-05-08T05:48:57.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.047+0000 7f4d94e0a740 1 client.7320 shutdown 2024-05-08T05:48:57.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.047+0000 7f4d94e0a740 1 -- 172.21.15.45:0/1114951337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a6f280 msgr2=0x556282a816c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.047+0000 7f4d94e0a740 1 --2- 172.21.15.45:0/1114951337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a6f280 0x556282a816c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c095990 tx=0x7f4d7c097040 comp rx=0 tx=0).stop 2024-05-08T05:48:57.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.047+0000 7f4d94e0a740 1 -- 172.21.15.45:0/1114951337 shutdown_connections 2024-05-08T05:48:57.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.047+0000 7f4d94e0a740 1 --2- 172.21.15.45:0/1114951337 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x556282a71300 0x556282a71700 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.047+0000 7f4d94e0a740 1 --2- 172.21.15.45:0/1114951337 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556282a5c010 0x556282a8cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.047+0000 7f4d94e0a740 1 --2- 172.21.15.45:0/1114951337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556282a8c160 0x556282a8ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.047+0000 7f4d94e0a740 1 --2- 172.21.15.45:0/1114951337 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a6f280 0x556282a816c0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.047+0000 7f4d6a7fc640 0 client.7320 ms_handle_reset on v2:172.21.15.45:6833/2945763952 2024-05-08T05:48:57.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.047+0000 7f4d94e0a740 1 -- 172.21.15.45:0/1114951337 >> 172.21.15.45:0/1114951337 conn(0x556282a6eb40 msgr2=0x556282a3e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:57.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.047+0000 7f4d94e0a740 1 -- 172.21.15.45:0/1114951337 shutdown_connections 2024-05-08T05:48:57.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.047+0000 7f4d94e0a740 1 -- 172.21.15.45:0/1114951337 wait complete. 2024-05-08T05:48:57.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.047+0000 7f4d94e0a740 20 client.7320 trim_cache size 0 max 16384 2024-05-08T05:48:57.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.066+0000 7f4d94e0a740 1 Processor -- start 2024-05-08T05:48:57.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.066+0000 7f4d94e0a740 1 -- start start 2024-05-08T05:48:57.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.067+0000 7f4d94e0a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a5c010 0x556282a5b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.067+0000 7f4d94e0a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556282a6f280 0x556282a89400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.067+0000 7f4d94e0a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556282a8c160 0x556282a89940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.067+0000 7f4d94e0a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c202310 con 0x556282a5c010 2024-05-08T05:48:57.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.067+0000 7f4d94e0a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c201330 con 0x556282a6f280 2024-05-08T05:48:57.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.067+0000 7f4d94e0a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4d8c2006f0 con 0x556282a8c160 2024-05-08T05:48:57.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.067+0000 7f4d8b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a5c010 0x556282a5b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.067+0000 7f4d8b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a5c010 0x556282a5b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60062/0 (socket says 172.21.15.45:60062) 2024-05-08T05:48:57.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.067+0000 7f4d8b7fe640 1 -- 172.21.15.45:0/3729149190 learned_addr learned my addr 172.21.15.45:0/3729149190 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:57.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.067+0000 7f4d8b7fe640 1 -- 172.21.15.45:0/3729149190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556282a8c160 msgr2=0x556282a89940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.067+0000 7f4d8b7fe640 1 --2- 172.21.15.45:0/3729149190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556282a8c160 0x556282a89940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.067+0000 7f4d8b7fe640 1 -- 172.21.15.45:0/3729149190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556282a6f280 msgr2=0x556282a89400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.067+0000 7f4d8b7fe640 1 --2- 172.21.15.45:0/3729149190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556282a6f280 0x556282a89400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.067+0000 7f4d8b7fe640 1 -- 172.21.15.45:0/3729149190 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55628267a0f0 con 0x556282a5c010 2024-05-08T05:48:57.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.067+0000 7f4d8b7fe640 1 --2- 172.21.15.45:0/3729149190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a5c010 0x556282a5b690 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c05ccc0 tx=0x7f4d7c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f758b4c0e426d2a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.067+0000 7f4d70ff9640 1 -- 172.21.15.45:0/3729149190 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c06e070 con 0x556282a5c010 2024-05-08T05:48:57.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.067+0000 7f4d70ff9640 1 -- 172.21.15.45:0/3729149190 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7c092070 con 0x556282a5c010 2024-05-08T05:48:57.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.067+0000 7f4d70ff9640 1 -- 172.21.15.45:0/3729149190 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c021040 con 0x556282a5c010 2024-05-08T05:48:57.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.068+0000 7f4d94e0a740 1 -- 172.21.15.45:0/3729149190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a5c010 msgr2=0x556282a5b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.068+0000 7f4d94e0a740 1 --2- 172.21.15.45:0/3729149190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a5c010 0x556282a5b690 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c05ccc0 tx=0x7f4d7c01e000 comp rx=0 tx=0).stop 2024-05-08T05:48:57.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.068+0000 7f4d94e0a740 1 -- 172.21.15.45:0/3729149190 shutdown_connections 2024-05-08T05:48:57.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.068+0000 7f4d94e0a740 1 --2- 172.21.15.45:0/3729149190 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556282a8c160 0x556282a89940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.068+0000 7f4d94e0a740 1 --2- 172.21.15.45:0/3729149190 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556282a6f280 0x556282a89400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.068+0000 7f4d94e0a740 1 --2- 172.21.15.45:0/3729149190 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a5c010 0x556282a5b690 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.068+0000 7f4d94e0a740 1 -- 172.21.15.45:0/3729149190 >> 172.21.15.45:0/3729149190 conn(0x556282a6eb40 msgr2=0x556282a80fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:57.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.068+0000 7f4d94e0a740 1 -- 172.21.15.45:0/3729149190 shutdown_connections 2024-05-08T05:48:57.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.068+0000 7f4d94e0a740 1 -- 172.21.15.45:0/3729149190 wait complete. 2024-05-08T05:48:57.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.069+0000 7f4d94e0a740 1 -- start start 2024-05-08T05:48:57.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.069+0000 7f4d94e0a740 1 Processor -- start 2024-05-08T05:48:57.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.070+0000 7f4d94e0a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556282a5c010 0x556282b09d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.070+0000 7f4d94e0a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a6f280 0x556282a3e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.070+0000 7f4d94e0a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556282a8c160 0x556282a3e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.070+0000 7f4d94e0a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x556282a6e8a0 con 0x556282a6f280 2024-05-08T05:48:57.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.070+0000 7f4d94e0a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c200d10 con 0x556282a5c010 2024-05-08T05:48:57.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.070+0000 7f4d94e0a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x556282a83000 con 0x556282a8c160 2024-05-08T05:48:57.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.070+0000 7f4d94e0a740 10 client.0 _ll_register_callbacks cb 0x7f4d80004970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:48:57.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.070+0000 7f4d8affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a6f280 0x556282a3e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.070+0000 7f4d8affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a6f280 0x556282a3e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60072/0 (socket says 172.21.15.45:60072) 2024-05-08T05:48:57.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.070+0000 7f4d8b7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556282a5c010 0x556282b09d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.070+0000 7f4d8affd640 1 -- 172.21.15.45:0/1909579728 learned_addr learned my addr 172.21.15.45:0/1909579728 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:57.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.070+0000 7f4d8bfff640 1 --2- 172.21.15.45:0/1909579728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556282a8c160 0x556282a3e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.070+0000 7f4d8affd640 1 -- 172.21.15.45:0/1909579728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556282a8c160 msgr2=0x556282a3e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.070+0000 7f4d8affd640 1 --2- 172.21.15.45:0/1909579728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556282a8c160 0x556282a3e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.070+0000 7f4d8affd640 1 -- 172.21.15.45:0/1909579728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556282a5c010 msgr2=0x556282b09d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.070+0000 7f4d8affd640 1 --2- 172.21.15.45:0/1909579728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556282a5c010 0x556282b09d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.070+0000 7f4d8affd640 1 -- 172.21.15.45:0/1909579728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556282a3ed10 con 0x556282a6f280 2024-05-08T05:48:57.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.071+0000 7f4d8affd640 1 --2- 172.21.15.45:0/1909579728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a6f280 0x556282a3e100 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x556282a81cc0 tx=0x7f4d80000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ea3b423e0ba2ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.071+0000 7f4d6a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:48:57.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.071+0000 7f4d6a7fc640 1 -- 172.21.15.45:0/1909579728 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d800075d0 con 0x556282a6f280 2024-05-08T05:48:57.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.071+0000 7f4d6a7fc640 1 -- 172.21.15.45:0/1909579728 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7f4d80024110 con 0x556282a6f280 2024-05-08T05:48:57.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.071+0000 7f4d94e0a740 1 -- 172.21.15.45:0/1909579728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d7c092070 con 0x556282a6f280 2024-05-08T05:48:57.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.072+0000 7f4d6a7fc640 1 -- 172.21.15.45:0/1909579728 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f4d800291d0 con 0x556282a6f280 2024-05-08T05:48:57.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.072+0000 7f4d94e0a740 10 client.7326 fetch_fsmap learned FSMap version 73 2024-05-08T05:48:57.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.072+0000 7f4d94e0a740 1 -- 172.21.15.45:0/1909579728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x556282a6b0b0 con 0x556282a6f280 2024-05-08T05:48:57.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.072+0000 7f4d6a7fc640 1 -- 172.21.15.45:0/1909579728 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7c092070 con 0x556282a6f280 2024-05-08T05:48:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.072+0000 7f4d6a7fc640 1 -- 172.21.15.45:0/1909579728 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7f4d80029ce0 con 0x556282a6f280 2024-05-08T05:48:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.072+0000 7f4d94e0a740 10 client.7326 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T05:48:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.072+0000 7f4d94e0a740 10 client.7326 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-05-08T05:48:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.072+0000 7f4d94e0a740 20 client.7326 populate_metadata read hostname 'smithi045' 2024-05-08T05:48:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.072+0000 7f4d94e0a740 1 --2- 172.21.15.45:0/1909579728 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x556282a8b9d0 0x556282a9b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.072+0000 7f4d94e0a740 4 client.7326 mds_command: new command op to 6747 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:48:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.072+0000 7f4d94e0a740 1 -- 172.21.15.45:0/1909579728 --> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5562829b09b0 con 0x556282a8b9d0 2024-05-08T05:48:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.072+0000 7f4d8b7fe640 1 --2- 172.21.15.45:0/1909579728 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x556282a8b9d0 0x556282a9b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.072+0000 7f4d6a7fc640 10 client.7326 ms_handle_connect on v2:172.21.15.45:6833/2945763952 2024-05-08T05:48:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.072+0000 7f4d8b7fe640 1 --2- 172.21.15.45:0/1909579728 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x556282a8b9d0 0x556282a9b550 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b00dea88830c66bc server_cookie=5dc7b634cdf9311d in_seq=0 out_seq=0 2024-05-08T05:48:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.073+0000 7f4d6a7fc640 1 -- 172.21.15.45:0/1909579728 <== mds.0 v2:172.21.15.45:6833/2945763952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5562829b09b0 con 0x556282a8b9d0 2024-05-08T05:48:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.073+0000 7f4d6a7fc640 10 client.7326 handle_command_reply: tid=0 2024-05-08T05:48:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.073+0000 7f4d94e0a740 1 client.7326 shutdown 2024-05-08T05:48:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.073+0000 7f4d94e0a740 1 -- 172.21.15.45:0/1909579728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a6f280 msgr2=0x556282a3e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.073+0000 7f4d94e0a740 1 --2- 172.21.15.45:0/1909579728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a6f280 0x556282a3e100 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x556282a81cc0 tx=0x7f4d80000fe0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.073+0000 7f4d94e0a740 1 -- 172.21.15.45:0/1909579728 shutdown_connections 2024-05-08T05:48:57.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.073+0000 7f4d94e0a740 1 --2- 172.21.15.45:0/1909579728 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x556282a8b9d0 0x556282a9b550 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.073+0000 7f4d94e0a740 1 --2- 172.21.15.45:0/1909579728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x556282a8c160 0x556282a3e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.073+0000 7f4d94e0a740 1 --2- 172.21.15.45:0/1909579728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x556282a6f280 0x556282a3e100 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.073+0000 7f4d6a7fc640 0 client.7326 ms_handle_reset on v2:172.21.15.45:6833/2945763952 2024-05-08T05:48:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.073+0000 7f4d94e0a740 1 --2- 172.21.15.45:0/1909579728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x556282a5c010 0x556282b09d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.073+0000 7f4d94e0a740 1 -- 172.21.15.45:0/1909579728 >> 172.21.15.45:0/1909579728 conn(0x556282a6eb40 msgr2=0x556282a89120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.073+0000 7f4d94e0a740 1 -- 172.21.15.45:0/1909579728 shutdown_connections 2024-05-08T05:48:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.073+0000 7f4d94e0a740 1 -- 172.21.15.45:0/1909579728 wait complete. 2024-05-08T05:48:57.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.073+0000 7f4d94e0a740 20 client.7326 trim_cache size 0 max 16384 2024-05-08T05:48:57.078 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:48:57.078 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-05-08T05:48:57.079 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-05-08T05:48:57.079 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:48:57.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.073+0000 7f4d92015640 1 -- 172.21.15.45:0/374107658 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4d780762d0 msgr2=0x7f4d78078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.073+0000 7f4d92015640 1 --2- 172.21.15.45:0/374107658 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4d780762d0 0x7f4d78078790 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c15acc0 tx=0x7f4d800047e0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.074+0000 7f4d92015640 1 -- 172.21.15.45:0/374107658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d8c06fc30 msgr2=0x7f4d8c14e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.074+0000 7f4d92015640 1 --2- 172.21.15.45:0/374107658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d8c06fc30 0x7f4d8c14e7e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00e960 tx=0x7f4d7c00ee20 comp rx=0 tx=0).stop 2024-05-08T05:48:57.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.074+0000 7f4d92015640 1 -- 172.21.15.45:0/374107658 shutdown_connections 2024-05-08T05:48:57.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.074+0000 7f4d92015640 1 --2- 172.21.15.45:0/374107658 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4d780762d0 0x7f4d78078790 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.074+0000 7f4d92015640 1 --2- 172.21.15.45:0/374107658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4d8c0b48a0 0x7f4d8c152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.074+0000 7f4d92015640 1 --2- 172.21.15.45:0/374107658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4d8c157540 0x7f4d8c153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.074+0000 7f4d92015640 1 --2- 172.21.15.45:0/374107658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d8c06fc30 0x7f4d8c14e7e0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.074+0000 7f4d92015640 1 -- 172.21.15.45:0/374107658 >> 172.21.15.45:0/374107658 conn(0x7f4d8c06c3a0 msgr2=0x7f4d8c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:57.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.074+0000 7f4d92015640 1 -- 172.21.15.45:0/374107658 shutdown_connections 2024-05-08T05:48:57.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.074+0000 7f4d92015640 1 -- 172.21.15.45:0/374107658 wait complete. 2024-05-08T05:48:57.086 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T05:48:57.086 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:48:57.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.189+0000 7fe87e1a4640 1 Processor -- start 2024-05-08T05:48:57.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.189+0000 7fe87e1a4640 1 -- start start 2024-05-08T05:48:57.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.190+0000 7fe87e1a4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe8780d0590 0x7fe8780ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.190+0000 7fe87e1a4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8780ce610 0x7fe8780709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.190+0000 7fe87e1a4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8780d15c0 con 0x7fe8781606d0 2024-05-08T05:48:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.190+0000 7fe87e1a4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe8780d18a0 con 0x7fe8780d0590 2024-05-08T05:48:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.190+0000 7fe87e1a4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe8780d1b80 con 0x7fe8780ce610 2024-05-08T05:48:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.190+0000 7fe8777fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe8781606d0 0x7fe8780cd9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38772/0 (socket says 172.21.15.45:38772) 2024-05-08T05:48:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.190+0000 7fe8777fe640 1 -- 172.21.15.45:0/1405710401 learned_addr learned my addr 172.21.15.45:0/1405710401 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:48:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.190+0000 7fe876ffd640 1 --2- 172.21.15.45:0/1405710401 >> v2:172.21.15.63:3300/0 conn(0x7fe8780d0590 0x7fe8780ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.190+0000 7fe877fff640 1 --2- 172.21.15.45:0/1405710401 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8780ce610 0x7fe8780709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.190+0000 7fe8767fc640 1 -- 172.21.15.45:0/1405710401 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3663827258 0 0) 0x7fe8780d15c0 con 0x7fe8781606d0 2024-05-08T05:48:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.190+0000 7fe8767fc640 1 -- 172.21.15.45:0/1405710401 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe860003520 con 0x7fe8781606d0 2024-05-08T05:48:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe8767fc640 1 -- 172.21.15.45:0/1405710401 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4082824282 0 0) 0x7fe860003520 con 0x7fe8781606d0 2024-05-08T05:48:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe8767fc640 1 -- 172.21.15.45:0/1405710401 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8780ce610 msgr2=0x7fe8780709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe8767fc640 1 --2- 172.21.15.45:0/1405710401 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8780ce610 0x7fe8780709f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe8767fc640 1 -- 172.21.15.45:0/1405710401 >> v2:172.21.15.63:3300/0 conn(0x7fe8780d0590 msgr2=0x7fe8780ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe8767fc640 1 --2- 172.21.15.45:0/1405710401 >> v2:172.21.15.63:3300/0 conn(0x7fe8780d0590 0x7fe8780ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe8767fc640 1 -- 172.21.15.45:0/1405710401 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8780704d0 con 0x7fe8781606d0 2024-05-08T05:48:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe8767fc640 1 -- 172.21.15.45:0/1405710401 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fe8680016f0 con 0x7fe8781606d0 2024-05-08T05:48:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe8767fc640 1 -- 172.21.15.45:0/1405710401 >> v1:172.21.15.45:6789/0 conn(0x7fe8781606d0 legacy=0x7fe8780cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe8767fc640 1 --2- 172.21.15.45:0/1405710401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe860003a50 0x7fe860003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe8767fc640 1 --2- 172.21.15.45:0/1405710401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860004800 0x7fe860004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe8767fc640 1 --2- 172.21.15.45:0/1405710401 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8780ce610 0x7fe8600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe8767fc640 1 -- 172.21.15.45:0/1405710401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe8780d18a0 con 0x7fe860004800 2024-05-08T05:48:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe8767fc640 1 -- 172.21.15.45:0/1405710401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8780d1b80 con 0x7fe860003a50 2024-05-08T05:48:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe8767fc640 1 -- 172.21.15.45:0/1405710401 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe86000bb40 con 0x7fe8780ce610 2024-05-08T05:48:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe876ffd640 1 --2- 172.21.15.45:0/1405710401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860004800 0x7fe860004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe8777fe640 1 --2- 172.21.15.45:0/1405710401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe860003a50 0x7fe860003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe877fff640 1 --2- 172.21.15.45:0/1405710401 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8780ce610 0x7fe8600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe876ffd640 1 -- 172.21.15.45:0/1405710401 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8780ce610 msgr2=0x7fe8600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe876ffd640 1 --2- 172.21.15.45:0/1405710401 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8780ce610 0x7fe8600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe876ffd640 1 -- 172.21.15.45:0/1405710401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe860003a50 msgr2=0x7fe860003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe876ffd640 1 --2- 172.21.15.45:0/1405710401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe860003a50 0x7fe860003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.191+0000 7fe876ffd640 1 -- 172.21.15.45:0/1405710401 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe860003520 con 0x7fe860004800 2024-05-08T05:48:57.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.192+0000 7fe876ffd640 1 --2- 172.21.15.45:0/1405710401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860004800 0x7fe860004c80 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009a90 tx=0x7fe860005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ccb3480aeb5e8e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.192+0000 7fe8767fc640 1 -- 172.21.15.45:0/1405710401 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c036070 con 0x7fe860004800 2024-05-08T05:48:57.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.192+0000 7fe8767fc640 1 -- 172.21.15.45:0/1405710401 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe86c031400 con 0x7fe860004800 2024-05-08T05:48:57.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.192+0000 7fe8767fc640 1 -- 172.21.15.45:0/1405710401 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c009690 con 0x7fe860004800 2024-05-08T05:48:57.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.193+0000 7fe87e1a4640 1 -- 172.21.15.45:0/1405710401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860004800 msgr2=0x7fe860004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.193+0000 7fe87e1a4640 1 --2- 172.21.15.45:0/1405710401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860004800 0x7fe860004c80 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009a90 tx=0x7fe860005830 comp rx=0 tx=0).stop 2024-05-08T05:48:57.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.193+0000 7fe877fff640 1 -- 172.21.15.45:0/1405710401 reap_dead start 2024-05-08T05:48:57.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.193+0000 7fe87e1a4640 1 -- 172.21.15.45:0/1405710401 shutdown_connections 2024-05-08T05:48:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.193+0000 7fe87e1a4640 1 -- 172.21.15.45:0/1405710401 >> 172.21.15.45:0/1405710401 conn(0x7fe8780b49d0 msgr2=0x7fe8780b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.193+0000 7fe87e1a4640 1 -- 172.21.15.45:0/1405710401 shutdown_connections 2024-05-08T05:48:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.193+0000 7fe87e1a4640 1 -- 172.21.15.45:0/1405710401 wait complete. 2024-05-08T05:48:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.194+0000 7fe87e1a4640 1 Processor -- start 2024-05-08T05:48:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.194+0000 7fe87e1a4640 1 -- start start 2024-05-08T05:48:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.195+0000 7fe87e1a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860004800 0x7fe87806dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.195+0000 7fe87e1a4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8781f4fb0 0x7fe8780b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.195+0000 7fe87e1a4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8781f8c40 0x7fe8781f90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.195+0000 7fe87e1a4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe878070b70 con 0x7fe860004800 2024-05-08T05:48:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.195+0000 7fe87e1a4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe8780709f0 con 0x7fe8781f4fb0 2024-05-08T05:48:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.195+0000 7fe87e1a4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe878070cf0 con 0x7fe8781f8c40 2024-05-08T05:48:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.195+0000 7fe8777fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860004800 0x7fe87806dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.195+0000 7fe8777fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860004800 0x7fe87806dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60088/0 (socket says 172.21.15.45:60088) 2024-05-08T05:48:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.195+0000 7fe8777fe640 1 -- 172.21.15.45:0/2038614207 learned_addr learned my addr 172.21.15.45:0/2038614207 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:57.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.195+0000 7fe8777fe640 1 -- 172.21.15.45:0/2038614207 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8781f8c40 msgr2=0x7fe8781f90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.195+0000 7fe8777fe640 1 --2- 172.21.15.45:0/2038614207 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8781f8c40 0x7fe8781f90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.195+0000 7fe8777fe640 1 -- 172.21.15.45:0/2038614207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8781f4fb0 msgr2=0x7fe8780b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.195+0000 7fe8777fe640 1 --2- 172.21.15.45:0/2038614207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8781f4fb0 0x7fe8780b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.195+0000 7fe8777fe640 1 -- 172.21.15.45:0/2038614207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8781f5720 con 0x7fe860004800 2024-05-08T05:48:57.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.195+0000 7fe8777fe640 1 --2- 172.21.15.45:0/2038614207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860004800 0x7fe87806dec0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe868004f40 tx=0x7fe8680074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.196+0000 7fe85ffff640 1 -- 172.21.15.45:0/2038614207 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86800f940 con 0x7fe860004800 2024-05-08T05:48:57.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.196+0000 7fe85ffff640 1 -- 172.21.15.45:0/2038614207 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe868018070 con 0x7fe860004800 2024-05-08T05:48:57.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.196+0000 7fe87e1a4640 1 -- 172.21.15.45:0/2038614207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8781f5a00 con 0x7fe860004800 2024-05-08T05:48:57.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.196+0000 7fe85ffff640 1 -- 172.21.15.45:0/2038614207 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86800fd40 con 0x7fe860004800 2024-05-08T05:48:57.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.196+0000 7fe87e1a4640 1 -- 172.21.15.45:0/2038614207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8781f8720 con 0x7fe860004800 2024-05-08T05:48:57.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.197+0000 7fe87e1a4640 1 -- 172.21.15.45:0/2038614207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8780693f0 con 0x7fe860004800 2024-05-08T05:48:57.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.197+0000 7fe85ffff640 1 -- 172.21.15.45:0/2038614207 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe86801d460 con 0x7fe860004800 2024-05-08T05:48:57.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.198+0000 7fe85ffff640 1 --2- 172.21.15.45:0/2038614207 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe864076300 0x7fe8640787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.198+0000 7fe85ffff640 1 -- 172.21.15.45:0/2038614207 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7fe868097720 con 0x7fe860004800 2024-05-08T05:48:57.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.198+0000 7fe876ffd640 1 --2- 172.21.15.45:0/2038614207 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe864076300 0x7fe8640787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.198+0000 7fe876ffd640 1 --2- 172.21.15.45:0/2038614207 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe864076300 0x7fe8640787c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe86c033000 tx=0x7fe86c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.202+0000 7fe85ffff640 1 -- 172.21.15.45:0/2038614207 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe868060e60 con 0x7fe860004800 2024-05-08T05:48:57.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.433+0000 7fe87e1a4640 1 -- 172.21.15.45:0/2038614207 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8780b8c00 con 0x7fe860004800 2024-05-08T05:48:57.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.433+0000 7fe85ffff640 1 -- 172.21.15.45:0/2038614207 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5185 (secure 0 0 0) 0x7fe868060760 con 0x7fe860004800 2024-05-08T05:48:57.435 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:48:57.435 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2616298287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2616298287},{"type":"v1","addr":"172.21.15.63:6836","nonce":2616298287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6472,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1347752311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1347752311},{"type":"v1","addr":"172.21.15.63:6837","nonce":1347752311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6744,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2854804427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2854804427},{"type":"v1","addr":"172.21.15.45:6834","nonce":2854804427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:47:00.211837+0000","modified":"2024-05-08T05:48:24.018244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6747},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6747":{"gid":6747,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":25,"addr":"172.21.15.45:6835/2945763952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2945763952},{"type":"v1","addr":"172.21.15.45:6835","nonce":2945763952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6747,"qdb_cluster":[6747]},"id":6}]} 2024-05-08T05:48:57.435 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 73 2024-05-08T05:48:57.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.435+0000 7fe87e1a4640 1 -- 172.21.15.45:0/2038614207 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe864076300 msgr2=0x7fe8640787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.435+0000 7fe87e1a4640 1 --2- 172.21.15.45:0/2038614207 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe864076300 0x7fe8640787c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe86c033000 tx=0x7fe86c02e000 comp rx=0 tx=0).stop 2024-05-08T05:48:57.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.435+0000 7fe87e1a4640 1 -- 172.21.15.45:0/2038614207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860004800 msgr2=0x7fe87806dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.435+0000 7fe87e1a4640 1 --2- 172.21.15.45:0/2038614207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860004800 0x7fe87806dec0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe868004f40 tx=0x7fe8680074f0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.435+0000 7fe87e1a4640 1 -- 172.21.15.45:0/2038614207 shutdown_connections 2024-05-08T05:48:57.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.435+0000 7fe87e1a4640 1 --2- 172.21.15.45:0/2038614207 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe864076300 0x7fe8640787c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.435+0000 7fe87e1a4640 1 --2- 172.21.15.45:0/2038614207 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe8781f8c40 0x7fe8781f90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.435+0000 7fe87e1a4640 1 --2- 172.21.15.45:0/2038614207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8781f4fb0 0x7fe8780b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.435+0000 7fe87e1a4640 1 --2- 172.21.15.45:0/2038614207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe860004800 0x7fe87806dec0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.435+0000 7fe87e1a4640 1 -- 172.21.15.45:0/2038614207 >> 172.21.15.45:0/2038614207 conn(0x7fe8780b49d0 msgr2=0x7fe8780b3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:57.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.435+0000 7fe87e1a4640 1 -- 172.21.15.45:0/2038614207 shutdown_connections 2024-05-08T05:48:57.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.435+0000 7fe87e1a4640 1 -- 172.21.15.45:0/2038614207 wait complete. 2024-05-08T05:48:57.447 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-05-08T05:48:57.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.552+0000 7f631aff0640 1 Processor -- start 2024-05-08T05:48:57.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.552+0000 7f631aff0640 1 -- start start 2024-05-08T05:48:57.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.553+0000 7f631aff0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63141629a0 0x7f63140d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.553+0000 7f631aff0640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f63140d2dc0 0x7f63140d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.553+0000 7f631aff0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f63140c8ed0 0x7f63140c9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.553+0000 7f631aff0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f63140d3330 con 0x7f63140c8ed0 2024-05-08T05:48:57.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.553+0000 7f631aff0640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f63140d2170 con 0x7f63140d2dc0 2024-05-08T05:48:57.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.553+0000 7f631aff0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f63140d2450 con 0x7f63141629a0 2024-05-08T05:48:57.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.553+0000 7f631958d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f63140c8ed0 0x7f63140c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.553+0000 7f631958d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f63140c8ed0 0x7f63140c9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60092/0 (socket says 172.21.15.45:60092) 2024-05-08T05:48:57.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.553+0000 7f631958d640 1 -- 172.21.15.45:0/2395070091 learned_addr learned my addr 172.21.15.45:0/2395070091 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:57.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.553+0000 7f6318d8c640 1 --2- 172.21.15.45:0/2395070091 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63141629a0 0x7f63140d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.553+0000 7f630bfff640 1 --2- 172.21.15.45:0/2395070091 >> v2:172.21.15.63:3300/0 conn(0x7f63140d2dc0 0x7f63140d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.553+0000 7f631958d640 1 -- 172.21.15.45:0/2395070091 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63141629a0 msgr2=0x7f63140d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.553+0000 7f631958d640 1 --2- 172.21.15.45:0/2395070091 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63141629a0 0x7f63140d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.553+0000 7f631958d640 1 -- 172.21.15.45:0/2395070091 >> v2:172.21.15.63:3300/0 conn(0x7f63140d2dc0 msgr2=0x7f63140d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.553+0000 7f631958d640 1 --2- 172.21.15.45:0/2395070091 >> v2:172.21.15.63:3300/0 conn(0x7f63140d2dc0 0x7f63140d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.553+0000 7f631958d640 1 -- 172.21.15.45:0/2395070091 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63140d2730 con 0x7f63140c8ed0 2024-05-08T05:48:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.554+0000 7f631958d640 1 --2- 172.21.15.45:0/2395070091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63140c8ed0 0x7f63140c9360 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f631408da00 tx=0x7f630c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c97ac39c0fb90e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.554+0000 7f630b7fe640 1 -- 172.21.15.45:0/2395070091 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c037070 con 0x7f63140c8ed0 2024-05-08T05:48:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.554+0000 7f630b7fe640 1 -- 172.21.15.45:0/2395070091 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f630c02ae00 con 0x7f63140c8ed0 2024-05-08T05:48:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.554+0000 7f630b7fe640 1 -- 172.21.15.45:0/2395070091 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c032660 con 0x7f63140c8ed0 2024-05-08T05:48:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.555+0000 7f631aff0640 1 -- 172.21.15.45:0/2395070091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63140c8ed0 msgr2=0x7f63140c9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.555+0000 7f631aff0640 1 --2- 172.21.15.45:0/2395070091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63140c8ed0 0x7f63140c9360 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f631408da00 tx=0x7f630c02a0b0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.555+0000 7f631aff0640 1 -- 172.21.15.45:0/2395070091 shutdown_connections 2024-05-08T05:48:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.555+0000 7f631aff0640 1 --2- 172.21.15.45:0/2395070091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63140c8ed0 0x7f63140c9360 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.555+0000 7f631aff0640 1 --2- 172.21.15.45:0/2395070091 >> v2:172.21.15.63:3300/0 conn(0x7f63140d2dc0 0x7f63140d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.555+0000 7f631aff0640 1 --2- 172.21.15.45:0/2395070091 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63141629a0 0x7f63140d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.555+0000 7f631aff0640 1 -- 172.21.15.45:0/2395070091 >> 172.21.15.45:0/2395070091 conn(0x7f63140d4fe0 msgr2=0x7f63140d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:57.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.555+0000 7f631aff0640 1 -- 172.21.15.45:0/2395070091 shutdown_connections 2024-05-08T05:48:57.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.555+0000 7f631aff0640 1 -- 172.21.15.45:0/2395070091 wait complete. 2024-05-08T05:48:57.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.556+0000 7f631aff0640 1 Processor -- start 2024-05-08T05:48:57.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.556+0000 7f631aff0640 1 -- start start 2024-05-08T05:48:57.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.557+0000 7f631aff0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63140c8ed0 0x7f63140a27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.557+0000 7f631aff0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63140d2dc0 0x7f63140ce970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.557+0000 7f631aff0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63141629a0 0x7f63140cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.557+0000 7f631aff0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f63140cf440 con 0x7f63140c8ed0 2024-05-08T05:48:57.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.557+0000 7f631aff0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f63140cf2c0 con 0x7f63140d2dc0 2024-05-08T05:48:57.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.557+0000 7f631aff0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f63140c98a0 con 0x7f63141629a0 2024-05-08T05:48:57.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.557+0000 7f6318d8c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63140c8ed0 0x7f63140a27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.557+0000 7f6318d8c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63140c8ed0 0x7f63140a27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60104/0 (socket says 172.21.15.45:60104) 2024-05-08T05:48:57.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.557+0000 7f6318d8c640 1 -- 172.21.15.45:0/549993366 learned_addr learned my addr 172.21.15.45:0/549993366 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.557+0000 7f630bfff640 1 --2- 172.21.15.45:0/549993366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63140d2dc0 0x7f63140ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.557+0000 7f631958d640 1 --2- 172.21.15.45:0/549993366 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63141629a0 0x7f63140cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.557+0000 7f6318d8c640 1 -- 172.21.15.45:0/549993366 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63141629a0 msgr2=0x7f63140cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.557+0000 7f6318d8c640 1 --2- 172.21.15.45:0/549993366 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63141629a0 0x7f63140cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.557+0000 7f6318d8c640 1 -- 172.21.15.45:0/549993366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63140d2dc0 msgr2=0x7f63140ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.557+0000 7f6318d8c640 1 --2- 172.21.15.45:0/549993366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63140d2dc0 0x7f63140ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.557+0000 7f6318d8c640 1 -- 172.21.15.45:0/549993366 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63140ccc20 con 0x7f63140c8ed0 2024-05-08T05:48:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.557+0000 7f630bfff640 1 --2- 172.21.15.45:0/549993366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63140d2dc0 0x7f63140ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.557+0000 7f6318d8c640 1 --2- 172.21.15.45:0/549993366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63140c8ed0 0x7f63140a27d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f631000efc0 tx=0x7f631000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.558+0000 7f63097fa640 1 -- 172.21.15.45:0/549993366 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6310018020 con 0x7f63140c8ed0 2024-05-08T05:48:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.558+0000 7f63097fa640 1 -- 172.21.15.45:0/549993366 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6310014480 con 0x7f63140c8ed0 2024-05-08T05:48:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.558+0000 7f631aff0640 1 -- 172.21.15.45:0/549993366 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63140ccf00 con 0x7f63140c8ed0 2024-05-08T05:48:57.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.558+0000 7f63097fa640 1 -- 172.21.15.45:0/549993366 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6310014850 con 0x7f63140c8ed0 2024-05-08T05:48:57.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.558+0000 7f631aff0640 1 -- 172.21.15.45:0/549993366 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63140a2d10 con 0x7f63140c8ed0 2024-05-08T05:48:57.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.560+0000 7f63097fa640 1 -- 172.21.15.45:0/549993366 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6310005d10 con 0x7f63140c8ed0 2024-05-08T05:48:57.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.560+0000 7f63097fa640 1 --2- 172.21.15.45:0/549993366 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f63000762d0 0x7f6300078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.561+0000 7f63097fa640 1 -- 172.21.15.45:0/549993366 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7f6310098390 con 0x7f63140c8ed0 2024-05-08T05:48:57.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.561+0000 7f630bfff640 1 --2- 172.21.15.45:0/549993366 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f63000762d0 0x7f6300078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.561+0000 7f631dde3740 1 Processor -- start 2024-05-08T05:48:57.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.561+0000 7f631dde3740 1 -- start start 2024-05-08T05:48:57.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.561+0000 7f630bfff640 1 --2- 172.21.15.45:0/549993366 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f63000762d0 0x7f6300078790 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f631409fb50 tx=0x7f6304008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.562+0000 7f631dde3740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b5c6d0 0x560412b0f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.562+0000 7f631dde3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b3f7f0 0x560412b3eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.562+0000 7f631dde3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b3f0b0 0x560412b13ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.562+0000 7f631dde3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5604129ff920 con 0x560412b3f0b0 2024-05-08T05:48:57.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.562+0000 7f631dde3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560412a74dd0 con 0x560412b3f7f0 2024-05-08T05:48:57.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.562+0000 7f631dde3740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x560412a253e0 con 0x560412b5c6d0 2024-05-08T05:48:57.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.562+0000 7f631958d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b3f0b0 0x560412b13ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.562+0000 7f631958d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b3f0b0 0x560412b13ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60114/0 (socket says 172.21.15.45:60114) 2024-05-08T05:48:57.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.562+0000 7f631958d640 1 -- 172.21.15.45:0/4187050375 learned_addr learned my addr 172.21.15.45:0/4187050375 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:57.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.562+0000 7f6318d8c640 1 --2- 172.21.15.45:0/4187050375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b5c6d0 0x560412b0f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.562+0000 7f630bfff640 1 --2- 172.21.15.45:0/4187050375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b3f7f0 0x560412b3eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.562+0000 7f631958d640 1 -- 172.21.15.45:0/4187050375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b5c6d0 msgr2=0x560412b0f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.562+0000 7f631958d640 1 --2- 172.21.15.45:0/4187050375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b5c6d0 0x560412b0f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.562+0000 7f631958d640 1 -- 172.21.15.45:0/4187050375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b3f7f0 msgr2=0x560412b3eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.562+0000 7f631958d640 1 --2- 172.21.15.45:0/4187050375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b3f7f0 0x560412b3eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.562+0000 7f631958d640 1 -- 172.21.15.45:0/4187050375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604128567c0 con 0x560412b3f0b0 2024-05-08T05:48:57.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.562+0000 7f631958d640 1 --2- 172.21.15.45:0/4187050375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b3f0b0 0x560412b13ec0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f630c009960 tx=0x7f630c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136136bed91de5be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.563+0000 7f62fa7fc640 1 -- 172.21.15.45:0/4187050375 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c043070 con 0x560412b3f0b0 2024-05-08T05:48:57.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.563+0000 7f62fa7fc640 1 -- 172.21.15.45:0/4187050375 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f630c03c640 con 0x560412b3f0b0 2024-05-08T05:48:57.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.563+0000 7f62fa7fc640 1 -- 172.21.15.45:0/4187050375 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c037040 con 0x560412b3f0b0 2024-05-08T05:48:57.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.563+0000 7f631dde3740 1 -- 172.21.15.45:0/4187050375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b3f0b0 msgr2=0x560412b13ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.563+0000 7f631dde3740 1 --2- 172.21.15.45:0/4187050375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b3f0b0 0x560412b13ec0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f630c009960 tx=0x7f630c02c000 comp rx=0 tx=0).stop 2024-05-08T05:48:57.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.563+0000 7f631dde3740 1 -- 172.21.15.45:0/4187050375 shutdown_connections 2024-05-08T05:48:57.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.563+0000 7f631dde3740 1 --2- 172.21.15.45:0/4187050375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b3f0b0 0x560412b13ec0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.563+0000 7f631dde3740 1 --2- 172.21.15.45:0/4187050375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b3f7f0 0x560412b3eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.563+0000 7f631dde3740 1 --2- 172.21.15.45:0/4187050375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b5c6d0 0x560412b0f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.563+0000 7f631dde3740 1 -- 172.21.15.45:0/4187050375 >> 172.21.15.45:0/4187050375 conn(0x560412b11560 msgr2=0x560412b59b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:57.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.563+0000 7f631dde3740 1 -- 172.21.15.45:0/4187050375 shutdown_connections 2024-05-08T05:48:57.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.563+0000 7f631dde3740 1 -- 172.21.15.45:0/4187050375 wait complete. 2024-05-08T05:48:57.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.564+0000 7f631dde3740 1 -- start start 2024-05-08T05:48:57.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.564+0000 7f631dde3740 1 Processor -- start 2024-05-08T05:48:57.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.565+0000 7f631dde3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b3f7f0 0x560412b50c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.565+0000 7f631dde3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 0x560412b59f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.565+0000 7f631dde3740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b42610 0x560412b2cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.565+0000 7f631dde3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560412a7d440 con 0x560412b5c6d0 2024-05-08T05:48:57.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.565+0000 7f631dde3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560412a78120 con 0x560412b3f7f0 2024-05-08T05:48:57.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.565+0000 7f631dde3740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x560412a86210 con 0x560412b42610 2024-05-08T05:48:57.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.565+0000 7f631dde3740 10 client.0 _ll_register_callbacks cb 0x560412a1d690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:48:57.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.565+0000 7f630bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 0x560412b59f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.565+0000 7f630bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 0x560412b59f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60124/0 (socket says 172.21.15.45:60124) 2024-05-08T05:48:57.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.565+0000 7f630bfff640 1 -- 172.21.15.45:0/1248645428 learned_addr learned my addr 172.21.15.45:0/1248645428 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:57.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.565+0000 7f6318d8c640 1 --2- 172.21.15.45:0/1248645428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b3f7f0 0x560412b50c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.565+0000 7f631958d640 1 --2- 172.21.15.45:0/1248645428 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b42610 0x560412b2cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.565+0000 7f630bfff640 1 -- 172.21.15.45:0/1248645428 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b42610 msgr2=0x560412b2cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.565+0000 7f630bfff640 1 --2- 172.21.15.45:0/1248645428 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b42610 0x560412b2cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.565+0000 7f630bfff640 1 -- 172.21.15.45:0/1248645428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b3f7f0 msgr2=0x560412b50c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.565+0000 7f630bfff640 1 --2- 172.21.15.45:0/1248645428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b3f7f0 0x560412b50c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.565+0000 7f630bfff640 1 -- 172.21.15.45:0/1248645428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604128b6f50 con 0x560412b5c6d0 2024-05-08T05:48:57.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.565+0000 7f630bfff640 1 --2- 172.21.15.45:0/1248645428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 0x560412b59f70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f63040157a0 tx=0x7f63040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c67b5f7c8545ef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.565+0000 7f62f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:48:57.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.566+0000 7f62f37fe640 1 -- 172.21.15.45:0/1248645428 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6304015de0 con 0x560412b5c6d0 2024-05-08T05:48:57.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.566+0000 7f62f37fe640 1 -- 172.21.15.45:0/1248645428 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7f6304014960 con 0x560412b5c6d0 2024-05-08T05:48:57.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.566+0000 7f631dde3740 1 -- 172.21.15.45:0/1248645428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f630c03c640 con 0x560412b5c6d0 2024-05-08T05:48:57.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.566+0000 7f62f37fe640 1 -- 172.21.15.45:0/1248645428 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f6304032050 con 0x560412b5c6d0 2024-05-08T05:48:57.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.566+0000 7f631dde3740 10 client.7344 fetch_fsmap learned FSMap version 73 2024-05-08T05:48:57.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.566+0000 7f631dde3740 1 -- 172.21.15.45:0/1248645428 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x56041274a0b0 con 0x560412b5c6d0 2024-05-08T05:48:57.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.566+0000 7f62f37fe640 1 -- 172.21.15.45:0/1248645428 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f630c03c640 con 0x560412b5c6d0 2024-05-08T05:48:57.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.566+0000 7f62f37fe640 1 -- 172.21.15.45:0/1248645428 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7f6304035020 con 0x560412b5c6d0 2024-05-08T05:48:57.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.566+0000 7f631dde3740 10 client.7344 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T05:48:57.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.566+0000 7f631dde3740 10 client.7344 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T05:48:57.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.566+0000 7f631dde3740 20 client.7344 populate_metadata read hostname 'smithi045' 2024-05-08T05:48:57.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.566+0000 7f631dde3740 1 --2- 172.21.15.45:0/1248645428 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x560412b6d190 0x560412b6d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.566+0000 7f631dde3740 4 client.7344 mds_command: new command op to 6747 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:48:57.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.566+0000 7f631dde3740 1 -- 172.21.15.45:0/1248645428 --> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560412a86a20 con 0x560412b6d190 2024-05-08T05:48:57.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.566+0000 7f6318d8c640 1 --2- 172.21.15.45:0/1248645428 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x560412b6d190 0x560412b6d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.566+0000 7f6318d8c640 1 --2- 172.21.15.45:0/1248645428 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x560412b6d190 0x560412b6d570 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60d80144871816e2 server_cookie=54e3c37016da2c0c in_seq=0 out_seq=0 2024-05-08T05:48:57.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.566+0000 7f62f37fe640 10 client.7344 ms_handle_connect on v2:172.21.15.45:6833/2945763952 2024-05-08T05:48:57.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.567+0000 7f62f37fe640 1 -- 172.21.15.45:0/1248645428 <== mds.0 v2:172.21.15.45:6833/2945763952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560412a86a20 con 0x560412b6d190 2024-05-08T05:48:57.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.567+0000 7f62f37fe640 10 client.7344 handle_command_reply: tid=0 2024-05-08T05:48:57.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.567+0000 7f631dde3740 1 client.7344 shutdown 2024-05-08T05:48:57.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.567+0000 7f631dde3740 1 -- 172.21.15.45:0/1248645428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 msgr2=0x560412b59f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.567+0000 7f631dde3740 1 --2- 172.21.15.45:0/1248645428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 0x560412b59f70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f63040157a0 tx=0x7f63040046c0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.567+0000 7f631dde3740 1 -- 172.21.15.45:0/1248645428 shutdown_connections 2024-05-08T05:48:57.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.567+0000 7f631dde3740 1 --2- 172.21.15.45:0/1248645428 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x560412b6d190 0x560412b6d570 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.567+0000 7f631dde3740 1 --2- 172.21.15.45:0/1248645428 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b42610 0x560412b2cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.567+0000 7f62f37fe640 0 client.7344 ms_handle_reset on v2:172.21.15.45:6833/2945763952 2024-05-08T05:48:57.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.567+0000 7f631dde3740 1 --2- 172.21.15.45:0/1248645428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 0x560412b59f70 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.567+0000 7f631dde3740 1 --2- 172.21.15.45:0/1248645428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b3f7f0 0x560412b50c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.567+0000 7f631dde3740 1 -- 172.21.15.45:0/1248645428 >> 172.21.15.45:0/1248645428 conn(0x560412b3f0b0 msgr2=0x560412b0ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:57.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.567+0000 7f631dde3740 1 -- 172.21.15.45:0/1248645428 shutdown_connections 2024-05-08T05:48:57.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.567+0000 7f631dde3740 1 -- 172.21.15.45:0/1248645428 wait complete. 2024-05-08T05:48:57.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.568+0000 7f631dde3740 20 client.7344 trim_cache size 0 max 16384 2024-05-08T05:48:57.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.581+0000 7f631dde3740 1 Processor -- start 2024-05-08T05:48:57.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.581+0000 7f631dde3740 1 -- start start 2024-05-08T05:48:57.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.582+0000 7f631dde3740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b3f7f0 0x560412b7dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.582+0000 7f631dde3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b42610 0x560412b5be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.582+0000 7f631dde3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 0x560412b14b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.582+0000 7f631dde3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f63142011c0 con 0x560412b5c6d0 2024-05-08T05:48:57.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.582+0000 7f631dde3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f63141f9190 con 0x560412b42610 2024-05-08T05:48:57.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.582+0000 7f631dde3740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6314200b10 con 0x560412b3f7f0 2024-05-08T05:48:57.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.582+0000 7f631958d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 0x560412b14b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.582+0000 7f631958d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 0x560412b14b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60138/0 (socket says 172.21.15.45:60138) 2024-05-08T05:48:57.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.582+0000 7f631958d640 1 -- 172.21.15.45:0/3429993814 learned_addr learned my addr 172.21.15.45:0/3429993814 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:57.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.582+0000 7f6318d8c640 1 --2- 172.21.15.45:0/3429993814 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b3f7f0 0x560412b7dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.582+0000 7f630bfff640 1 --2- 172.21.15.45:0/3429993814 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b42610 0x560412b5be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.583+0000 7f631958d640 1 -- 172.21.15.45:0/3429993814 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b3f7f0 msgr2=0x560412b7dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.583+0000 7f631958d640 1 --2- 172.21.15.45:0/3429993814 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b3f7f0 0x560412b7dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.583+0000 7f631958d640 1 -- 172.21.15.45:0/3429993814 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b42610 msgr2=0x560412b5be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.583+0000 7f631958d640 1 --2- 172.21.15.45:0/3429993814 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b42610 0x560412b5be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.583+0000 7f631958d640 1 -- 172.21.15.45:0/3429993814 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560412b10360 con 0x560412b5c6d0 2024-05-08T05:48:57.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.583+0000 7f6318d8c640 1 --2- 172.21.15.45:0/3429993814 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b3f7f0 0x560412b7dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:48:57.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.583+0000 7f631958d640 1 --2- 172.21.15.45:0/3429993814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 0x560412b14b80 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f630c042000 tx=0x7f630c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9dd87855669c5d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.583+0000 7f62f9ffb640 1 -- 172.21.15.45:0/3429993814 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c04e070 con 0x560412b5c6d0 2024-05-08T05:48:57.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.583+0000 7f62f9ffb640 1 -- 172.21.15.45:0/3429993814 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f630c0043f0 con 0x560412b5c6d0 2024-05-08T05:48:57.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.583+0000 7f62f9ffb640 1 -- 172.21.15.45:0/3429993814 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c037040 con 0x560412b5c6d0 2024-05-08T05:48:57.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.583+0000 7f631dde3740 1 -- 172.21.15.45:0/3429993814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 msgr2=0x560412b14b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.583+0000 7f631dde3740 1 --2- 172.21.15.45:0/3429993814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 0x560412b14b80 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f630c042000 tx=0x7f630c008030 comp rx=0 tx=0).stop 2024-05-08T05:48:57.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.583+0000 7f631dde3740 1 -- 172.21.15.45:0/3429993814 shutdown_connections 2024-05-08T05:48:57.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.583+0000 7f631dde3740 1 --2- 172.21.15.45:0/3429993814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 0x560412b14b80 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.583+0000 7f631dde3740 1 --2- 172.21.15.45:0/3429993814 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b42610 0x560412b5be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.584+0000 7f631dde3740 1 --2- 172.21.15.45:0/3429993814 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b3f7f0 0x560412b7dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.584+0000 7f631dde3740 1 -- 172.21.15.45:0/3429993814 >> 172.21.15.45:0/3429993814 conn(0x560412b3f0b0 msgr2=0x560412b2ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:57.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.584+0000 7f631dde3740 1 -- 172.21.15.45:0/3429993814 shutdown_connections 2024-05-08T05:48:57.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.584+0000 7f631dde3740 1 -- 172.21.15.45:0/3429993814 wait complete. 2024-05-08T05:48:57.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.584+0000 7f631dde3740 1 -- start start 2024-05-08T05:48:57.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.584+0000 7f631dde3740 1 Processor -- start 2024-05-08T05:48:57.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.585+0000 7f631dde3740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b3f7f0 0x560412bdb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.585+0000 7f631dde3740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b42610 0x560412b5aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.585+0000 7f631dde3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 0x560412b5af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.585+0000 7f631dde3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560412b2d3b0 con 0x560412b5c6d0 2024-05-08T05:48:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.585+0000 7f631dde3740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560412a2ea20 con 0x560412b42610 2024-05-08T05:48:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.585+0000 7f631dde3740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x560412b42a50 con 0x560412b3f7f0 2024-05-08T05:48:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.585+0000 7f631dde3740 10 client.0 _ll_register_callbacks cb 0x7f6304011a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:48:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.585+0000 7f631958d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 0x560412b5af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.585+0000 7f631958d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 0x560412b5af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60142/0 (socket says 172.21.15.45:60142) 2024-05-08T05:48:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.585+0000 7f631958d640 1 -- 172.21.15.45:0/3527848185 learned_addr learned my addr 172.21.15.45:0/3527848185 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.585+0000 7f6318d8c640 1 --2- 172.21.15.45:0/3527848185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b3f7f0 0x560412bdb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.585+0000 7f630bfff640 1 --2- 172.21.15.45:0/3527848185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b42610 0x560412b5aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.585+0000 7f631958d640 1 -- 172.21.15.45:0/3527848185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b3f7f0 msgr2=0x560412bdb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.585+0000 7f631958d640 1 --2- 172.21.15.45:0/3527848185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b3f7f0 0x560412bdb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.585+0000 7f631958d640 1 -- 172.21.15.45:0/3527848185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b42610 msgr2=0x560412b5aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.585+0000 7f631958d640 1 --2- 172.21.15.45:0/3527848185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b42610 0x560412b5aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.585+0000 7f631958d640 1 -- 172.21.15.45:0/3527848185 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560412b52f80 con 0x560412b5c6d0 2024-05-08T05:48:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.585+0000 7f6318d8c640 1 --2- 172.21.15.45:0/3527848185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b3f7f0 0x560412bdb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.586+0000 7f630bfff640 1 --2- 172.21.15.45:0/3527848185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b42610 0x560412b5aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.586+0000 7f631958d640 1 --2- 172.21.15.45:0/3527848185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 0x560412b5af50 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f630c02a0f0 tx=0x7f630c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22fa45e6ae1dcf4e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.586+0000 7f62f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:48:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.586+0000 7f62f37fe640 1 -- 172.21.15.45:0/3527848185 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c057060 con 0x560412b5c6d0 2024-05-08T05:48:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.586+0000 7f62f37fe640 1 -- 172.21.15.45:0/3527848185 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7f630c006430 con 0x560412b5c6d0 2024-05-08T05:48:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.586+0000 7f631dde3740 1 -- 172.21.15.45:0/3527848185 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f630c0043f0 con 0x560412b5c6d0 2024-05-08T05:48:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.586+0000 7f62f37fe640 1 -- 172.21.15.45:0/3527848185 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f630c05d050 con 0x560412b5c6d0 2024-05-08T05:48:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.586+0000 7f631dde3740 10 client.7350 fetch_fsmap learned FSMap version 73 2024-05-08T05:48:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.586+0000 7f631dde3740 1 -- 172.21.15.45:0/3527848185 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x560412b58f50 con 0x560412b5c6d0 2024-05-08T05:48:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.586+0000 7f62f37fe640 1 -- 172.21.15.45:0/3527848185 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f630c0043f0 con 0x560412b5c6d0 2024-05-08T05:48:57.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.586+0000 7f62f37fe640 1 -- 172.21.15.45:0/3527848185 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 73) ==== 3494+0+0 (secure 0 0 0) 0x7f630c060020 con 0x560412b5c6d0 2024-05-08T05:48:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.586+0000 7f631dde3740 10 client.7350 fetch_fsmap finished waiting for FSMap version 73 2024-05-08T05:48:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.586+0000 7f631dde3740 10 client.7350 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T05:48:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.586+0000 7f631dde3740 20 client.7350 populate_metadata read hostname 'smithi045' 2024-05-08T05:48:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.586+0000 7f631dde3740 1 --2- 172.21.15.45:0/3527848185 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x560412b5d3f0 0x560412b5d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.586+0000 7f631dde3740 4 client.7350 mds_command: new command op to 6747 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-05-08T05:48:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.586+0000 7f631dde3740 1 -- 172.21.15.45:0/3527848185 --> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x560412a80890 con 0x560412b5d3f0 2024-05-08T05:48:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.586+0000 7f6318d8c640 1 --2- 172.21.15.45:0/3527848185 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x560412b5d3f0 0x560412b5d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.587+0000 7f6318d8c640 1 --2- 172.21.15.45:0/3527848185 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x560412b5d3f0 0x560412b5d7d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cae2c1b6aeafc2da server_cookie=7303e677be11c4e9 in_seq=0 out_seq=0 2024-05-08T05:48:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.587+0000 7f62f37fe640 10 client.7350 ms_handle_connect on v2:172.21.15.45:6833/2945763952 2024-05-08T05:48:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.587+0000 7f62f37fe640 1 -- 172.21.15.45:0/3527848185 <== mds.0 v2:172.21.15.45:6833/2945763952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560412a80890 con 0x560412b5d3f0 2024-05-08T05:48:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.587+0000 7f62f37fe640 10 client.7350 handle_command_reply: tid=0 2024-05-08T05:48:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.587+0000 7f631dde3740 1 client.7350 shutdown 2024-05-08T05:48:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.587+0000 7f631dde3740 1 -- 172.21.15.45:0/3527848185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 msgr2=0x560412b5af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.587+0000 7f631dde3740 1 --2- 172.21.15.45:0/3527848185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 0x560412b5af50 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f630c02a0f0 tx=0x7f630c004730 comp rx=0 tx=0).stop 2024-05-08T05:48:57.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.587+0000 7f631dde3740 1 -- 172.21.15.45:0/3527848185 shutdown_connections 2024-05-08T05:48:57.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.587+0000 7f631dde3740 1 --2- 172.21.15.45:0/3527848185 >> [v2:172.21.15.45:6833/2945763952,v1:172.21.15.45:6835/2945763952] conn(0x560412b5d3f0 0x560412b5d7d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.587+0000 7f631dde3740 1 --2- 172.21.15.45:0/3527848185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560412b5c6d0 0x560412b5af50 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.587+0000 7f631dde3740 1 --2- 172.21.15.45:0/3527848185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560412b42610 0x560412b5aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.587+0000 7f631dde3740 1 --2- 172.21.15.45:0/3527848185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560412b3f7f0 0x560412bdb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.587+0000 7f631dde3740 1 -- 172.21.15.45:0/3527848185 >> 172.21.15.45:0/3527848185 conn(0x560412b3f0b0 msgr2=0x560412b7dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:57.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.587+0000 7f62f37fe640 0 client.7350 ms_handle_reset on v2:172.21.15.45:6833/2945763952 2024-05-08T05:48:57.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.587+0000 7f631dde3740 1 -- 172.21.15.45:0/3527848185 shutdown_connections 2024-05-08T05:48:57.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.587+0000 7f631dde3740 1 -- 172.21.15.45:0/3527848185 wait complete. 2024-05-08T05:48:57.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.587+0000 7f631dde3740 20 client.7350 trim_cache size 0 max 16384 2024-05-08T05:48:57.595 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:48:57.595 INFO:teuthology.orchestra.run.smithi045.stdout:[] 2024-05-08T05:48:57.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.588+0000 7f631aff0640 1 -- 172.21.15.45:0/549993366 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f63000762d0 msgr2=0x7f6300078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.588+0000 7f631aff0640 1 --2- 172.21.15.45:0/549993366 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f63000762d0 0x7f6300078790 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f631409fb50 tx=0x7f6304008040 comp rx=0 tx=0).stop 2024-05-08T05:48:57.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.588+0000 7f631aff0640 1 -- 172.21.15.45:0/549993366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63140c8ed0 msgr2=0x7f63140a27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.588+0000 7f631aff0640 1 --2- 172.21.15.45:0/549993366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63140c8ed0 0x7f63140a27d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f631000efc0 tx=0x7f631000c490 comp rx=0 tx=0).stop 2024-05-08T05:48:57.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.588+0000 7f631aff0640 1 -- 172.21.15.45:0/549993366 shutdown_connections 2024-05-08T05:48:57.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.588+0000 7f631aff0640 1 --2- 172.21.15.45:0/549993366 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f63000762d0 0x7f6300078790 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.588+0000 7f631aff0640 1 --2- 172.21.15.45:0/549993366 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f63141629a0 0x7f63140cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.588+0000 7f631aff0640 1 --2- 172.21.15.45:0/549993366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f63140d2dc0 0x7f63140ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.588+0000 7f631aff0640 1 --2- 172.21.15.45:0/549993366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f63140c8ed0 0x7f63140a27d0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.588+0000 7f631aff0640 1 -- 172.21.15.45:0/549993366 >> 172.21.15.45:0/549993366 conn(0x7f63140d4fe0 msgr2=0x7f6314161c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:57.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.588+0000 7f631aff0640 1 -- 172.21.15.45:0/549993366 shutdown_connections 2024-05-08T05:48:57.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.588+0000 7f631aff0640 1 -- 172.21.15.45:0/549993366 wait complete. 2024-05-08T05:48:57.600 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T05:48:57.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.703+0000 7f0d0ea59640 1 Processor -- start 2024-05-08T05:48:57.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.703+0000 7f0d0ea59640 1 -- start start 2024-05-08T05:48:57.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.703+0000 7f0d0ea59640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0d081629b0 0x7f0d0806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.703+0000 7f0d0ea59640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d0806e110 con 0x7f0d080b2490 2024-05-08T05:48:57.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.703+0000 7f0d0ea59640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d0806e3f0 con 0x7f0d08070d20 2024-05-08T05:48:57.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.703+0000 7f0d0ea59640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f0d08070a40 con 0x7f0d081629b0 2024-05-08T05:48:57.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.703+0000 7f0d07fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0d081629b0 0x7f0d0806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.703+0000 7f0d07fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0d081629b0 0x7f0d0806eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60150/0 (socket says 172.21.15.45:60150) 2024-05-08T05:48:57.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.703+0000 7f0d077fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0d080b2490 0x7f0d08070330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38786/0 (socket says 172.21.15.45:38786) 2024-05-08T05:48:57.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.703+0000 7f0d07fff640 1 -- 172.21.15.45:0/2973654330 learned_addr learned my addr 172.21.15.45:0/2973654330 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:57.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.703+0000 7f0d07fff640 1 -- 172.21.15.45:0/2973654330 >> v1:172.21.15.63:6789/0 conn(0x7f0d08070d20 legacy=0x7f0d080afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:57.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.703+0000 7f0d07fff640 1 -- 172.21.15.45:0/2973654330 >> v1:172.21.15.45:6789/0 conn(0x7f0d080b2490 legacy=0x7f0d08070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.703+0000 7f0d07fff640 1 -- 172.21.15.45:0/2973654330 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0806e6d0 con 0x7f0d081629b0 2024-05-08T05:48:57.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.704+0000 7f0d07fff640 1 --2- 172.21.15.45:0/2973654330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d081629b0 0x7f0d0806eed0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009870 tx=0x7f0cf802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68d8a10f294087b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.704+0000 7f0d06ffd640 1 -- 172.21.15.45:0/2973654330 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8037070 con 0x7f0d081629b0 2024-05-08T05:48:57.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.704+0000 7f0d06ffd640 1 -- 172.21.15.45:0/2973654330 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf8032440 con 0x7f0d081629b0 2024-05-08T05:48:57.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.704+0000 7f0d06ffd640 1 -- 172.21.15.45:0/2973654330 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8032810 con 0x7f0d081629b0 2024-05-08T05:48:57.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.705+0000 7f0d0ea59640 1 -- 172.21.15.45:0/2973654330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d081629b0 msgr2=0x7f0d0806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.705+0000 7f0d0ea59640 1 --2- 172.21.15.45:0/2973654330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d081629b0 0x7f0d0806eed0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009870 tx=0x7f0cf802a250 comp rx=0 tx=0).stop 2024-05-08T05:48:57.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.705+0000 7f0d0ea59640 1 -- 172.21.15.45:0/2973654330 shutdown_connections 2024-05-08T05:48:57.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.705+0000 7f0d0ea59640 1 --2- 172.21.15.45:0/2973654330 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d081629b0 0x7f0d0806eed0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.705+0000 7f0d0ea59640 1 -- 172.21.15.45:0/2973654330 >> 172.21.15.45:0/2973654330 conn(0x7f0d080b49d0 msgr2=0x7f0d080b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:57.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.705+0000 7f0d0ea59640 1 -- 172.21.15.45:0/2973654330 shutdown_connections 2024-05-08T05:48:57.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.705+0000 7f0d0ea59640 1 -- 172.21.15.45:0/2973654330 wait complete. 2024-05-08T05:48:57.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.706+0000 7f0d0ea59640 1 Processor -- start 2024-05-08T05:48:57.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.706+0000 7f0d0ea59640 1 -- start start 2024-05-08T05:48:57.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.707+0000 7f0d0ea59640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d08070d20 0x7f0d080b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.707+0000 7f0d0ea59640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d080b2490 0x7f0d080c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.707+0000 7f0d0ea59640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d081629b0 0x7f0d0814f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.707+0000 7f0d0ea59640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0806f410 con 0x7f0d08070d20 2024-05-08T05:48:57.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.707+0000 7f0d0ea59640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0806dec0 con 0x7f0d080b2490 2024-05-08T05:48:57.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.707+0000 7f0d0ea59640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0d080b03c0 con 0x7f0d081629b0 2024-05-08T05:48:57.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.707+0000 7f0d07fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d08070d20 0x7f0d080b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.707+0000 7f0d07fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d08070d20 0x7f0d080b6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60158/0 (socket says 172.21.15.45:60158) 2024-05-08T05:48:57.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.707+0000 7f0d07fff640 1 -- 172.21.15.45:0/3827300824 learned_addr learned my addr 172.21.15.45:0/3827300824 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:57.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.707+0000 7f0d077fe640 1 --2- 172.21.15.45:0/3827300824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d080b2490 0x7f0d080c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.707+0000 7f0d0cff6640 1 --2- 172.21.15.45:0/3827300824 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d081629b0 0x7f0d0814f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.707+0000 7f0d07fff640 1 -- 172.21.15.45:0/3827300824 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d081629b0 msgr2=0x7f0d0814f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.707+0000 7f0d07fff640 1 --2- 172.21.15.45:0/3827300824 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d081629b0 0x7f0d0814f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.707+0000 7f0d07fff640 1 -- 172.21.15.45:0/3827300824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d080b2490 msgr2=0x7f0d080c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.707+0000 7f0d07fff640 1 --2- 172.21.15.45:0/3827300824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d080b2490 0x7f0d080c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.707+0000 7f0d07fff640 1 -- 172.21.15.45:0/3827300824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d080b7710 con 0x7f0d08070d20 2024-05-08T05:48:57.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.707+0000 7f0d077fe640 1 --2- 172.21.15.45:0/3827300824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d080b2490 0x7f0d080c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:48:57.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.708+0000 7f0d07fff640 1 --2- 172.21.15.45:0/3827300824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d08070d20 0x7f0d080b6ec0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8000c00 tx=0x7f0cf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.708+0000 7f0d04ff9640 1 -- 172.21.15.45:0/3827300824 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8037040 con 0x7f0d08070d20 2024-05-08T05:48:57.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.708+0000 7f0d04ff9640 1 -- 172.21.15.45:0/3827300824 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf8042070 con 0x7f0d08070d20 2024-05-08T05:48:57.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.708+0000 7f0d0ea59640 1 -- 172.21.15.45:0/3827300824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d080b79f0 con 0x7f0d08070d20 2024-05-08T05:48:57.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.708+0000 7f0d04ff9640 1 -- 172.21.15.45:0/3827300824 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8030bd0 con 0x7f0d08070d20 2024-05-08T05:48:57.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.708+0000 7f0d0ea59640 1 -- 172.21.15.45:0/3827300824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d080b7fb0 con 0x7f0d08070d20 2024-05-08T05:48:57.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.709+0000 7f0d0ea59640 1 -- 172.21.15.45:0/3827300824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d080693f0 con 0x7f0d08070d20 2024-05-08T05:48:57.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.710+0000 7f0d04ff9640 1 -- 172.21.15.45:0/3827300824 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0cf8047460 con 0x7f0d08070d20 2024-05-08T05:48:57.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.710+0000 7f0d04ff9640 1 --2- 172.21.15.45:0/3827300824 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0ce80762d0 0x7f0ce8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.710+0000 7f0d04ff9640 1 -- 172.21.15.45:0/3827300824 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7f0cf80b84d0 con 0x7f0d08070d20 2024-05-08T05:48:57.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.710+0000 7f0d077fe640 1 --2- 172.21.15.45:0/3827300824 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0ce80762d0 0x7f0ce8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.711+0000 7f0d077fe640 1 --2- 172.21.15.45:0/3827300824 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0ce80762d0 0x7f0ce8078790 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0d080c98c0 tx=0x7f0cf0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.715+0000 7f0d04ff9640 1 -- 172.21.15.45:0/3827300824 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0cf8081b80 con 0x7f0d08070d20 2024-05-08T05:48:57.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.948+0000 7f0d0ea59640 1 -- 172.21.15.45:0/3827300824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0d08070560 con 0x7f0d08070d20 2024-05-08T05:48:57.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.948+0000 7f0d04ff9640 1 -- 172.21.15.45:0/3827300824 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f0cf8081480 con 0x7f0d08070d20 2024-05-08T05:48:57.948 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:48:57.948 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T05:48:57.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.950+0000 7f0d0ea59640 1 -- 172.21.15.45:0/3827300824 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0ce80762d0 msgr2=0x7f0ce8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.950+0000 7f0d0ea59640 1 --2- 172.21.15.45:0/3827300824 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0ce80762d0 0x7f0ce8078790 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0d080c98c0 tx=0x7f0cf0009040 comp rx=0 tx=0).stop 2024-05-08T05:48:57.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.950+0000 7f0d0ea59640 1 -- 172.21.15.45:0/3827300824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d08070d20 msgr2=0x7f0d080b6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.950+0000 7f0d0ea59640 1 --2- 172.21.15.45:0/3827300824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d08070d20 0x7f0d080b6ec0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8000c00 tx=0x7f0cf80046c0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.950+0000 7f0d0ea59640 1 -- 172.21.15.45:0/3827300824 shutdown_connections 2024-05-08T05:48:57.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.950+0000 7f0d0ea59640 1 --2- 172.21.15.45:0/3827300824 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0ce80762d0 0x7f0ce8078790 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.950+0000 7f0d0ea59640 1 --2- 172.21.15.45:0/3827300824 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d081629b0 0x7f0d0814f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.950+0000 7f0d0ea59640 1 --2- 172.21.15.45:0/3827300824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d080b2490 0x7f0d080c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.950+0000 7f0d0ea59640 1 --2- 172.21.15.45:0/3827300824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d08070d20 0x7f0d080b6ec0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.950+0000 7f0d0ea59640 1 -- 172.21.15.45:0/3827300824 >> 172.21.15.45:0/3827300824 conn(0x7f0d080b49d0 msgr2=0x7f0d080b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:57.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.950+0000 7f0d0ea59640 1 -- 172.21.15.45:0/3827300824 shutdown_connections 2024-05-08T05:48:57.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:57.950+0000 7f0d0ea59640 1 -- 172.21.15.45:0/3827300824 wait complete. 2024-05-08T05:48:57.962 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-05-08T05:48:57.962 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T05:48:57.962 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:48:58.010 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:48:58.054 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T05:48:58.054 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:48:58.055 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:48:58.055 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:48:58.075 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:48:58.075 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:48:58.132 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:48:58.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.276+0000 7f6e73132640 1 Processor -- start 2024-05-08T05:48:58.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.276+0000 7f6e73132640 1 -- start start 2024-05-08T05:48:58.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.277+0000 7f6e73132640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6e6c15f380 0x7f6e6c0d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:58.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.277+0000 7f6e73132640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e6c0d0cf0 0x7f6e6c0d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:58.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.277+0000 7f6e73132640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e6c0d1760 con 0x7f6e6c0d3b00 2024-05-08T05:48:58.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.277+0000 7f6e73132640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6e6c0d30f0 con 0x7f6e6c15f380 2024-05-08T05:48:58.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.277+0000 7f6e73132640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6e6c0d1150 con 0x7f6e6c0d0cf0 2024-05-08T05:48:58.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.278+0000 7f6e6bfff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e6c0d0cf0 0x7f6e6c0d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:58.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.278+0000 7f6e70ece640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6e6c15f380 0x7f6e6c0d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:58.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.278+0000 7f6e716cf640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f6e6c0d3b00 0x7f6e6c162e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:34322/0 (socket says 172.21.15.45:34322) 2024-05-08T05:48:58.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.278+0000 7f6e70ece640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6e6c15f380 0x7f6e6c0d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:47288/0 (socket says 172.21.15.45:47288) 2024-05-08T05:48:58.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.278+0000 7f6e6bfff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e6c0d0cf0 0x7f6e6c0d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:35438/0 (socket says 172.21.15.45:35438) 2024-05-08T05:48:58.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.278+0000 7f6e716cf640 1 -- 172.21.15.45:0/3339915976 learned_addr learned my addr 172.21.15.45:0/3339915976 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:48:58.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.278+0000 7f6e6bfff640 1 -- 172.21.15.45:0/3339915976 >> v2:172.21.15.63:3300/0 conn(0x7f6e6c15f380 msgr2=0x7f6e6c0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:58.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.278+0000 7f6e6bfff640 1 --2- 172.21.15.45:0/3339915976 >> v2:172.21.15.63:3300/0 conn(0x7f6e6c15f380 0x7f6e6c0d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:58.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.278+0000 7f6e6bfff640 1 -- 172.21.15.45:0/3339915976 >> v1:172.21.15.63:6789/0 conn(0x7f6e6c0d3b00 legacy=0x7f6e6c162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:58.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.278+0000 7f6e6bfff640 1 -- 172.21.15.45:0/3339915976 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6c0d1a40 con 0x7f6e6c0d0cf0 2024-05-08T05:48:58.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.278+0000 7f6e70ece640 1 --2- 172.21.15.45:0/3339915976 >> v2:172.21.15.63:3300/0 conn(0x7f6e6c15f380 0x7f6e6c0d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:48:58.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.279+0000 7f6e6bfff640 1 --2- 172.21.15.45:0/3339915976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e6c0d0cf0 0x7f6e6c0d35c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6e600097a0 tx=0x7f6e6002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1505b4de7f347b08 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:58.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.279+0000 7f6e6b7fe640 1 -- 172.21.15.45:0/3339915976 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e60037070 con 0x7f6e6c0d0cf0 2024-05-08T05:48:58.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.279+0000 7f6e6b7fe640 1 -- 172.21.15.45:0/3339915976 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e60032440 con 0x7f6e6c0d0cf0 2024-05-08T05:48:58.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.279+0000 7f6e6b7fe640 1 -- 172.21.15.45:0/3339915976 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e60032810 con 0x7f6e6c0d0cf0 2024-05-08T05:48:58.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.279+0000 7f6e73132640 1 -- 172.21.15.45:0/3339915976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e6c0d0cf0 msgr2=0x7f6e6c0d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:58.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.280+0000 7f6e73132640 1 --2- 172.21.15.45:0/3339915976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e6c0d0cf0 0x7f6e6c0d35c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6e600097a0 tx=0x7f6e6002a180 comp rx=0 tx=0).stop 2024-05-08T05:48:58.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.280+0000 7f6e73132640 1 -- 172.21.15.45:0/3339915976 shutdown_connections 2024-05-08T05:48:58.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.280+0000 7f6e73132640 1 --2- 172.21.15.45:0/3339915976 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e6c0d0cf0 0x7f6e6c0d35c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:58.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.280+0000 7f6e73132640 1 --2- 172.21.15.45:0/3339915976 >> v2:172.21.15.63:3300/0 conn(0x7f6e6c15f380 0x7f6e6c0d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:58.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.280+0000 7f6e73132640 1 -- 172.21.15.45:0/3339915976 >> 172.21.15.45:0/3339915976 conn(0x7f6e6c0d4fe0 msgr2=0x7f6e6c0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:58.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.280+0000 7f6e73132640 1 -- 172.21.15.45:0/3339915976 shutdown_connections 2024-05-08T05:48:58.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.280+0000 7f6e73132640 1 -- 172.21.15.45:0/3339915976 wait complete. 2024-05-08T05:48:58.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.281+0000 7f6e73132640 1 Processor -- start 2024-05-08T05:48:58.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.281+0000 7f6e73132640 1 -- start start 2024-05-08T05:48:58.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.282+0000 7f6e73132640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c0d0cf0 0x7f6e6c163790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:58.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.282+0000 7f6e73132640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e6c0d3b00 0x7f6e6c0cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:58.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.282+0000 7f6e73132640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e6c15f380 0x7f6e6c15c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:58.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.282+0000 7f6e73132640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c0b8e40 con 0x7f6e6c15f380 2024-05-08T05:48:58.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.282+0000 7f6e73132640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c0d2370 con 0x7f6e6c0d0cf0 2024-05-08T05:48:58.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.282+0000 7f6e73132640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6e6c0d2000 con 0x7f6e6c0d3b00 2024-05-08T05:48:58.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.282+0000 7f6e716cf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e6c15f380 0x7f6e6c15c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:58.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.282+0000 7f6e716cf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e6c15f380 0x7f6e6c15c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60164/0 (socket says 172.21.15.45:60164) 2024-05-08T05:48:58.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.282+0000 7f6e716cf640 1 -- 172.21.15.45:0/1799935582 learned_addr learned my addr 172.21.15.45:0/1799935582 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:58.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.282+0000 7f6e70ece640 1 --2- 172.21.15.45:0/1799935582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c0d0cf0 0x7f6e6c163790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:58.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.282+0000 7f6e6bfff640 1 --2- 172.21.15.45:0/1799935582 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e6c0d3b00 0x7f6e6c0cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:58.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.282+0000 7f6e716cf640 1 -- 172.21.15.45:0/1799935582 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e6c0d3b00 msgr2=0x7f6e6c0cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:58.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.282+0000 7f6e716cf640 1 --2- 172.21.15.45:0/1799935582 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e6c0d3b00 0x7f6e6c0cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:58.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.282+0000 7f6e716cf640 1 -- 172.21.15.45:0/1799935582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c0d0cf0 msgr2=0x7f6e6c163790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:58.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.282+0000 7f6e716cf640 1 --2- 172.21.15.45:0/1799935582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c0d0cf0 0x7f6e6c163790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:58.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.282+0000 7f6e716cf640 1 -- 172.21.15.45:0/1799935582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6c1f5af0 con 0x7f6e6c15f380 2024-05-08T05:48:58.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.282+0000 7f6e716cf640 1 --2- 172.21.15.45:0/1799935582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e6c15f380 0x7f6e6c15c710 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00c450 tx=0x7f6e5c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:58.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.283+0000 7f6e697fa640 1 -- 172.21.15.45:0/1799935582 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c017070 con 0x7f6e6c15f380 2024-05-08T05:48:58.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.283+0000 7f6e697fa640 1 -- 172.21.15.45:0/1799935582 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e5c005ce0 con 0x7f6e6c15f380 2024-05-08T05:48:58.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.283+0000 7f6e73132640 1 -- 172.21.15.45:0/1799935582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6c1f5dd0 con 0x7f6e6c15f380 2024-05-08T05:48:58.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.283+0000 7f6e697fa640 1 -- 172.21.15.45:0/1799935582 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c007270 con 0x7f6e6c15f380 2024-05-08T05:48:58.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.283+0000 7f6e73132640 1 -- 172.21.15.45:0/1799935582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e6c09efe0 con 0x7f6e6c15f380 2024-05-08T05:48:58.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.284+0000 7f6e73132640 1 -- 172.21.15.45:0/1799935582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6c0693f0 con 0x7f6e6c15f380 2024-05-08T05:48:58.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.284+0000 7f6e697fa640 1 -- 172.21.15.45:0/1799935582 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6e5c01a070 con 0x7f6e6c15f380 2024-05-08T05:48:58.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.285+0000 7f6e697fa640 1 --2- 172.21.15.45:0/1799935582 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6e580762d0 0x7f6e58078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:58.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.285+0000 7f6e697fa640 1 -- 172.21.15.45:0/1799935582 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7f6e5c096b30 con 0x7f6e6c15f380 2024-05-08T05:48:58.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.285+0000 7f6e70ece640 1 --2- 172.21.15.45:0/1799935582 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6e580762d0 0x7f6e58078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:58.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.286+0000 7f6e70ece640 1 --2- 172.21.15.45:0/1799935582 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6e580762d0 0x7f6e58078790 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0b65b0 tx=0x7f6e6400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:58.289 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.289+0000 7f6e697fa640 1 -- 172.21.15.45:0/1799935582 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6e5c05fb80 con 0x7f6e6c15f380 2024-05-08T05:48:58.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.492+0000 7f6e73132640 1 -- 172.21.15.45:0/1799935582 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6e6c08cfd0 con 0x7f6e6c15f380 2024-05-08T05:48:58.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.493+0000 7f6e697fa640 1 -- 172.21.15.45:0/1799935582 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v92) ==== 80+0+244 (secure 0 0 0) 0x7f6e5c063e80 con 0x7f6e6c15f380 2024-05-08T05:48:58.493 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.63:6837/3194497846 2024-05-09T05:48:20.008781+0000 2024-05-08T05:48:58.493 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.63:6835/3194497846 2024-05-09T05:48:20.008781+0000 2024-05-08T05:48:58.493 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.63:6836/2942326473 2024-05-09T05:47:09.529461+0000 2024-05-08T05:48:58.494 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.63:6834/2942326473 2024-05-09T05:47:09.529461+0000 2024-05-08T05:48:58.494 INFO:teuthology.orchestra.run.smithi045.stderr:listed 4 entries 2024-05-08T05:48:58.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.495+0000 7f6e73132640 1 -- 172.21.15.45:0/1799935582 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6e580762d0 msgr2=0x7f6e58078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:58.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.495+0000 7f6e73132640 1 --2- 172.21.15.45:0/1799935582 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6e580762d0 0x7f6e58078790 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0b65b0 tx=0x7f6e6400b040 comp rx=0 tx=0).stop 2024-05-08T05:48:58.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.495+0000 7f6e73132640 1 -- 172.21.15.45:0/1799935582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e6c15f380 msgr2=0x7f6e6c15c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:58.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.495+0000 7f6e73132640 1 --2- 172.21.15.45:0/1799935582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e6c15f380 0x7f6e6c15c710 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00c450 tx=0x7f6e5c00c910 comp rx=0 tx=0).stop 2024-05-08T05:48:58.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.495+0000 7f6e73132640 1 -- 172.21.15.45:0/1799935582 shutdown_connections 2024-05-08T05:48:58.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.495+0000 7f6e73132640 1 --2- 172.21.15.45:0/1799935582 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6e580762d0 0x7f6e58078790 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:58.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.495+0000 7f6e73132640 1 --2- 172.21.15.45:0/1799935582 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e6c0d3b00 0x7f6e6c0cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:58.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.495+0000 7f6e73132640 1 --2- 172.21.15.45:0/1799935582 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e6c15f380 0x7f6e6c15c710 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:58.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.495+0000 7f6e73132640 1 --2- 172.21.15.45:0/1799935582 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c0d0cf0 0x7f6e6c163790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:58.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.495+0000 7f6e73132640 1 -- 172.21.15.45:0/1799935582 >> 172.21.15.45:0/1799935582 conn(0x7f6e6c0d4fe0 msgr2=0x7f6e6c0b8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:58.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.495+0000 7f6e73132640 1 -- 172.21.15.45:0/1799935582 shutdown_connections 2024-05-08T05:48:58.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.495+0000 7f6e73132640 1 -- 172.21.15.45:0/1799935582 wait complete. 2024-05-08T05:48:58.506 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T05:48:58.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:48:58.506 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T05:48:58.536 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:48:58.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:48:58.536 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T05:48:58.592 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:48:58.592 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:48:58.593 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:48:58.613 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:48:58.613 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:48:58.613 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:48:58.613 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:48:58.649 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:48:58.650 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:48:58.650 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:48:58.650 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:48:58.673 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:48:58.673 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:48:58.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.793+0000 7f7966408640 1 Processor -- start 2024-05-08T05:48:58.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.793+0000 7f7966408640 1 -- start start 2024-05-08T05:48:58.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.794+0000 7f7966408640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f796006c3a0 0x7f79600b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:58.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.794+0000 7f7966408640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f79600aedf0 0x7f79600b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.794+0000 7f7966408640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f796009ff30 con 0x7f796015d180 2024-05-08T05:48:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.794+0000 7f7966408640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f796015e360 con 0x7f79600aedf0 2024-05-08T05:48:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.794+0000 7f7966408640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f796015d750 con 0x7f796006c3a0 2024-05-08T05:48:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.794+0000 7f795f7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f79600aedf0 0x7f79600b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.794+0000 7f79649a5640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f796015d180 0x7f796006c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38794/0 (socket says 172.21.15.45:38794) 2024-05-08T05:48:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.794+0000 7f79649a5640 1 -- 172.21.15.45:0/3197059963 learned_addr learned my addr 172.21.15.45:0/3197059963 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:48:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.794+0000 7f795f7fe640 1 -- 172.21.15.45:0/3197059963 >> v2:172.21.15.63:3300/0 conn(0x7f796006c3a0 msgr2=0x7f79600b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.794+0000 7f795ffff640 1 --2- 172.21.15.45:0/3197059963 >> v2:172.21.15.63:3300/0 conn(0x7f796006c3a0 0x7f79600b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.794+0000 7f795f7fe640 1 --2- 172.21.15.45:0/3197059963 >> v2:172.21.15.63:3300/0 conn(0x7f796006c3a0 0x7f79600b13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.794+0000 7f795f7fe640 1 -- 172.21.15.45:0/3197059963 >> v1:172.21.15.45:6789/0 conn(0x7f796015d180 legacy=0x7f796006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.794+0000 7f795effd640 1 -- 172.21.15.45:0/3197059963 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 821641642 0 0) 0x7f796009ff30 con 0x7f796015d180 2024-05-08T05:48:58.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.794+0000 7f795f7fe640 1 -- 172.21.15.45:0/3197059963 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796015da30 con 0x7f79600aedf0 2024-05-08T05:48:58.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.795+0000 7f795f7fe640 1 --2- 172.21.15.45:0/3197059963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79600aedf0 0x7f79600b6130 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f79600b1950 tx=0x7f795402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9bed478f7fd468 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:58.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.795+0000 7f795effd640 1 -- 172.21.15.45:0/3197059963 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954037070 con 0x7f79600aedf0 2024-05-08T05:48:58.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.795+0000 7f795effd640 1 -- 172.21.15.45:0/3197059963 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7954032440 con 0x7f79600aedf0 2024-05-08T05:48:58.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.795+0000 7f795effd640 1 -- 172.21.15.45:0/3197059963 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954032810 con 0x7f79600aedf0 2024-05-08T05:48:58.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.796+0000 7f7966408640 1 -- 172.21.15.45:0/3197059963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79600aedf0 msgr2=0x7f79600b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:58.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.796+0000 7f7966408640 1 --2- 172.21.15.45:0/3197059963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79600aedf0 0x7f79600b6130 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f79600b1950 tx=0x7f795402a1a0 comp rx=0 tx=0).stop 2024-05-08T05:48:58.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.796+0000 7f7966408640 1 -- 172.21.15.45:0/3197059963 shutdown_connections 2024-05-08T05:48:58.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.796+0000 7f7966408640 1 --2- 172.21.15.45:0/3197059963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79600aedf0 0x7f79600b6130 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:58.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.796+0000 7f7966408640 1 --2- 172.21.15.45:0/3197059963 >> v2:172.21.15.63:3300/0 conn(0x7f796006c3a0 0x7f79600b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:58.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.796+0000 7f7966408640 1 -- 172.21.15.45:0/3197059963 >> 172.21.15.45:0/3197059963 conn(0x7f79600b3b10 msgr2=0x7f79600b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:58.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.796+0000 7f7966408640 1 -- 172.21.15.45:0/3197059963 shutdown_connections 2024-05-08T05:48:58.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.796+0000 7f7966408640 1 -- 172.21.15.45:0/3197059963 wait complete. 2024-05-08T05:48:58.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.797+0000 7f7966408640 1 Processor -- start 2024-05-08T05:48:58.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.797+0000 7f7966408640 1 -- start start 2024-05-08T05:48:58.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.798+0000 7f7966408640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f796006c3a0 0x7f79601533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:58.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.798+0000 7f7966408640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79600aedf0 0x7f796006f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:58.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.798+0000 7f7966408640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f796015d180 0x7f796006f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:58.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.798+0000 7f7966408640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f79600b4fc0 con 0x7f79600aedf0 2024-05-08T05:48:58.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.798+0000 7f7966408640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f796015dff0 con 0x7f796015d180 2024-05-08T05:48:58.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.798+0000 7f7966408640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f79600b5140 con 0x7f796006c3a0 2024-05-08T05:48:58.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.798+0000 7f795f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79600aedf0 0x7f796006f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:58.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.798+0000 7f795f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79600aedf0 0x7f796006f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60186/0 (socket says 172.21.15.45:60186) 2024-05-08T05:48:58.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.798+0000 7f795f7fe640 1 -- 172.21.15.45:0/327792006 learned_addr learned my addr 172.21.15.45:0/327792006 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:58.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.798+0000 7f79649a5640 1 --2- 172.21.15.45:0/327792006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f796015d180 0x7f796006f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:58.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.798+0000 7f795ffff640 1 --2- 172.21.15.45:0/327792006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f796006c3a0 0x7f79601533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:58.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.798+0000 7f795f7fe640 1 -- 172.21.15.45:0/327792006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f796006c3a0 msgr2=0x7f79601533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:58.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.798+0000 7f795f7fe640 1 --2- 172.21.15.45:0/327792006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f796006c3a0 0x7f79601533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:58.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.798+0000 7f795f7fe640 1 -- 172.21.15.45:0/327792006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f796015d180 msgr2=0x7f796006f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:58.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.798+0000 7f795f7fe640 1 --2- 172.21.15.45:0/327792006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f796015d180 0x7f796006f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:58.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.798+0000 7f795f7fe640 1 -- 172.21.15.45:0/327792006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79600d0730 con 0x7f79600aedf0 2024-05-08T05:48:58.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.799+0000 7f795f7fe640 1 --2- 172.21.15.45:0/327792006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79600aedf0 0x7f796006f010 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f7954000c00 tx=0x7f79540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:58.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.799+0000 7f795cff9640 1 -- 172.21.15.45:0/327792006 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954037040 con 0x7f79600aedf0 2024-05-08T05:48:58.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.799+0000 7f795cff9640 1 -- 172.21.15.45:0/327792006 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7954042070 con 0x7f79600aedf0 2024-05-08T05:48:58.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.799+0000 7f795cff9640 1 -- 172.21.15.45:0/327792006 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954030a50 con 0x7f79600aedf0 2024-05-08T05:48:58.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.799+0000 7f7966408640 1 -- 172.21.15.45:0/327792006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7960150190 con 0x7f79600aedf0 2024-05-08T05:48:58.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.799+0000 7f7966408640 1 -- 172.21.15.45:0/327792006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7960151520 con 0x7f79600aedf0 2024-05-08T05:48:58.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.800+0000 7f7966408640 1 -- 172.21.15.45:0/327792006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f796006a8f0 con 0x7f79600aedf0 2024-05-08T05:48:58.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.801+0000 7f795cff9640 1 -- 172.21.15.45:0/327792006 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7954030bf0 con 0x7f79600aedf0 2024-05-08T05:48:58.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.801+0000 7f795cff9640 1 --2- 172.21.15.45:0/327792006 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f794c076300 0x7f794c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:58.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.801+0000 7f795cff9640 1 -- 172.21.15.45:0/327792006 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7f79540b82f0 con 0x7f79600aedf0 2024-05-08T05:48:58.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.801+0000 7f795ffff640 1 --2- 172.21.15.45:0/327792006 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f794c076300 0x7f794c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:58.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.802+0000 7f795ffff640 1 --2- 172.21.15.45:0/327792006 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f794c076300 0x7f794c0787c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7950006fd0 tx=0x7f7950008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:58.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:58.805+0000 7f795cff9640 1 -- 172.21.15.45:0/327792006 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7960150190 con 0x7f79600aedf0 2024-05-08T05:48:59.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.035+0000 7f7966408640 1 -- 172.21.15.45:0/327792006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79600b6920 con 0x7f79600aedf0 2024-05-08T05:48:59.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.036+0000 7f795cff9640 1 -- 172.21.15.45:0/327792006 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5185 (secure 0 0 0) 0x7f7954081990 con 0x7f79600aedf0 2024-05-08T05:48:59.036 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:48:59.037 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2616298287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2616298287},{"type":"v1","addr":"172.21.15.63:6836","nonce":2616298287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6472,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1347752311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1347752311},{"type":"v1","addr":"172.21.15.63:6837","nonce":1347752311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6744,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2854804427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2854804427},{"type":"v1","addr":"172.21.15.45:6834","nonce":2854804427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:47:00.211837+0000","modified":"2024-05-08T05:48:24.018244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6747},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6747":{"gid":6747,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":25,"addr":"172.21.15.45:6835/2945763952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2945763952},{"type":"v1","addr":"172.21.15.45:6835","nonce":2945763952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6747,"qdb_cluster":[6747]},"id":6}]} 2024-05-08T05:48:59.037 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 73 2024-05-08T05:48:59.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.038+0000 7f7966408640 1 -- 172.21.15.45:0/327792006 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f794c076300 msgr2=0x7f794c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:59.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.038+0000 7f7966408640 1 --2- 172.21.15.45:0/327792006 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f794c076300 0x7f794c0787c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7950006fd0 tx=0x7f7950008040 comp rx=0 tx=0).stop 2024-05-08T05:48:59.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.038+0000 7f7966408640 1 -- 172.21.15.45:0/327792006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79600aedf0 msgr2=0x7f796006f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:59.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.038+0000 7f7966408640 1 --2- 172.21.15.45:0/327792006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79600aedf0 0x7f796006f010 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f7954000c00 tx=0x7f79540046c0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.038+0000 7f7966408640 1 -- 172.21.15.45:0/327792006 shutdown_connections 2024-05-08T05:48:59.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.038+0000 7f7966408640 1 --2- 172.21.15.45:0/327792006 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f794c076300 0x7f794c0787c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.038+0000 7f7966408640 1 --2- 172.21.15.45:0/327792006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f796015d180 0x7f796006f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.038+0000 7f7966408640 1 --2- 172.21.15.45:0/327792006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f79600aedf0 0x7f796006f010 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.038+0000 7f7966408640 1 --2- 172.21.15.45:0/327792006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f796006c3a0 0x7f79601533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.038+0000 7f7966408640 1 -- 172.21.15.45:0/327792006 >> 172.21.15.45:0/327792006 conn(0x7f79600b3b10 msgr2=0x7f796015f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:59.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.038+0000 7f7966408640 1 -- 172.21.15.45:0/327792006 shutdown_connections 2024-05-08T05:48:59.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.038+0000 7f7966408640 1 -- 172.21.15.45:0/327792006 wait complete. 2024-05-08T05:48:59.050 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:48:59.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.154+0000 7fafc7e93640 1 Processor -- start 2024-05-08T05:48:59.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.154+0000 7fafc7e93640 1 -- start start 2024-05-08T05:48:59.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.155+0000 7fafc7e93640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fafc01629a0 0x7fafc00d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:59.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.155+0000 7fafc7e93640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fafc00d0cb0 0x7fafc00d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:59.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.155+0000 7fafc7e93640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafc00d3330 con 0x7fafc00d2dc0 2024-05-08T05:48:59.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.155+0000 7fafc7e93640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fafc00d2170 con 0x7fafc01629a0 2024-05-08T05:48:59.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.155+0000 7fafc7e93640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fafc00d2450 con 0x7fafc00d0cb0 2024-05-08T05:48:59.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.155+0000 7fafc5c2f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fafc01629a0 0x7fafc00d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:59.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.155+0000 7fafc5c2f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fafc01629a0 0x7fafc00d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60202/0 (socket says 172.21.15.45:60202) 2024-05-08T05:48:59.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.155+0000 7fafc5c2f640 1 -- 172.21.15.45:0/3562646106 learned_addr learned my addr 172.21.15.45:0/3562646106 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:59.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.155+0000 7fafc6430640 1 --2- 172.21.15.45:0/3562646106 >> v2:172.21.15.63:3300/0 conn(0x7fafc00d0cb0 0x7fafc00d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:59.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.155+0000 7fafc5c2f640 1 -- 172.21.15.45:0/3562646106 >> v2:172.21.15.63:3300/0 conn(0x7fafc00d0cb0 msgr2=0x7fafc00d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:59.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.156+0000 7fafc5c2f640 1 --2- 172.21.15.45:0/3562646106 >> v2:172.21.15.63:3300/0 conn(0x7fafc00d0cb0 0x7fafc00d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.156+0000 7fafc5c2f640 1 -- 172.21.15.45:0/3562646106 >> v1:172.21.15.63:6789/0 conn(0x7fafc00d2dc0 legacy=0x7fafc00c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:59.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.156+0000 7fafc5c2f640 1 -- 172.21.15.45:0/3562646106 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc00d2730 con 0x7fafc01629a0 2024-05-08T05:48:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.156+0000 7fafc5c2f640 1 --2- 172.21.15.45:0/3562646106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafc01629a0 0x7fafc00d35c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fafc008e380 tx=0x7fafbc02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f056b53e57d3283b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.156+0000 7fafc4c2d640 1 -- 172.21.15.45:0/3562646106 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc037070 con 0x7fafc01629a0 2024-05-08T05:48:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.156+0000 7fafc4c2d640 1 -- 172.21.15.45:0/3562646106 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafbc032440 con 0x7fafc01629a0 2024-05-08T05:48:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.156+0000 7fafc4c2d640 1 -- 172.21.15.45:0/3562646106 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc032810 con 0x7fafc01629a0 2024-05-08T05:48:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.157+0000 7fafc7e93640 1 -- 172.21.15.45:0/3562646106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafc01629a0 msgr2=0x7fafc00d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.157+0000 7fafc7e93640 1 --2- 172.21.15.45:0/3562646106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafc01629a0 0x7fafc00d35c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fafc008e380 tx=0x7fafbc02a150 comp rx=0 tx=0).stop 2024-05-08T05:48:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.157+0000 7fafc7e93640 1 -- 172.21.15.45:0/3562646106 shutdown_connections 2024-05-08T05:48:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.157+0000 7fafc7e93640 1 --2- 172.21.15.45:0/3562646106 >> v2:172.21.15.63:3300/0 conn(0x7fafc00d0cb0 0x7fafc00d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.157+0000 7fafc7e93640 1 --2- 172.21.15.45:0/3562646106 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafc01629a0 0x7fafc00d35c0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.157+0000 7fafc7e93640 1 -- 172.21.15.45:0/3562646106 >> 172.21.15.45:0/3562646106 conn(0x7fafc00d4fe0 msgr2=0x7fafc00d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.157+0000 7fafc7e93640 1 -- 172.21.15.45:0/3562646106 shutdown_connections 2024-05-08T05:48:59.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.157+0000 7fafc7e93640 1 -- 172.21.15.45:0/3562646106 wait complete. 2024-05-08T05:48:59.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.158+0000 7fafc7e93640 1 Processor -- start 2024-05-08T05:48:59.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.158+0000 7fafc7e93640 1 -- start start 2024-05-08T05:48:59.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.159+0000 7fafc7e93640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafc00d0cb0 0x7fafc0154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:59.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.159+0000 7fafc7e93640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafc00d2dc0 0x7fafc00a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:59.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.159+0000 7fafc7e93640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fafc01629a0 0x7fafc00a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:59.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.159+0000 7fafc7e93640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fafc00cf440 con 0x7fafc00d0cb0 2024-05-08T05:48:59.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.159+0000 7fafc7e93640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fafc00cf2c0 con 0x7fafc00d2dc0 2024-05-08T05:48:59.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.159+0000 7fafc7e93640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fafc00c95e0 con 0x7fafc01629a0 2024-05-08T05:48:59.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.159+0000 7fafc5c2f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafc00d0cb0 0x7fafc0154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:59.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.159+0000 7fafc5c2f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafc00d0cb0 0x7fafc0154340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60208/0 (socket says 172.21.15.45:60208) 2024-05-08T05:48:59.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.159+0000 7fafc5c2f640 1 -- 172.21.15.45:0/529497302 learned_addr learned my addr 172.21.15.45:0/529497302 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:59.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.159+0000 7fafc5c2f640 1 -- 172.21.15.45:0/529497302 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fafc01629a0 msgr2=0x7fafc00a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:59.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.159+0000 7fafc5c2f640 1 --2- 172.21.15.45:0/529497302 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fafc01629a0 0x7fafc00a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.159+0000 7fafc5c2f640 1 -- 172.21.15.45:0/529497302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafc00d2dc0 msgr2=0x7fafc00a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:59.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.159+0000 7fafc5c2f640 1 --2- 172.21.15.45:0/529497302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafc00d2dc0 0x7fafc00a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.159+0000 7fafc5c2f640 1 -- 172.21.15.45:0/529497302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc00a1980 con 0x7fafc00d0cb0 2024-05-08T05:48:59.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.160+0000 7fafc5c2f640 1 --2- 172.21.15.45:0/529497302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafc00d0cb0 0x7fafc0154340 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fafbc02a190 tx=0x7fafbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:59.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.160+0000 7fafb67fc640 1 -- 172.21.15.45:0/529497302 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc041020 con 0x7fafc00d0cb0 2024-05-08T05:48:59.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.160+0000 7fafb67fc640 1 -- 172.21.15.45:0/529497302 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafbc0306c0 con 0x7fafc00d0cb0 2024-05-08T05:48:59.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.160+0000 7fafc7e93640 1 -- 172.21.15.45:0/529497302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc00a1c60 con 0x7fafc00d0cb0 2024-05-08T05:48:59.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.160+0000 7fafb67fc640 1 -- 172.21.15.45:0/529497302 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc037040 con 0x7fafc00d0cb0 2024-05-08T05:48:59.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.160+0000 7fafc7e93640 1 -- 172.21.15.45:0/529497302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc00a2220 con 0x7fafc00d0cb0 2024-05-08T05:48:59.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.161+0000 7fafc7e93640 1 -- 172.21.15.45:0/529497302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc00693f0 con 0x7fafc00d0cb0 2024-05-08T05:48:59.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.161+0000 7fafb67fc640 1 -- 172.21.15.45:0/529497302 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fafbc044020 con 0x7fafc00d0cb0 2024-05-08T05:48:59.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.162+0000 7fafb67fc640 1 --2- 172.21.15.45:0/529497302 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fafa4076300 0x7fafa40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:59.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.162+0000 7fafb67fc640 1 -- 172.21.15.45:0/529497302 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7fafbc0b8030 con 0x7fafc00d0cb0 2024-05-08T05:48:59.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.162+0000 7fafc542e640 1 --2- 172.21.15.45:0/529497302 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fafa4076300 0x7fafa40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:59.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.163+0000 7fafc542e640 1 --2- 172.21.15.45:0/529497302 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fafa4076300 0x7fafa40787c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fafb0000f60 tx=0x7fafb00079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:59.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.166+0000 7fafb67fc640 1 -- 172.21.15.45:0/529497302 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fafbc04b030 con 0x7fafc00d0cb0 2024-05-08T05:48:59.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.394+0000 7fafc7e93640 1 -- 172.21.15.45:0/529497302 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafc00c9c70 con 0x7fafc00d0cb0 2024-05-08T05:48:59.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.395+0000 7fafb67fc640 1 -- 172.21.15.45:0/529497302 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5185 (secure 0 0 0) 0x7fafbc081770 con 0x7fafc00d0cb0 2024-05-08T05:48:59.395 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:48:59.395 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2616298287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2616298287},{"type":"v1","addr":"172.21.15.63:6836","nonce":2616298287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6472,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1347752311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1347752311},{"type":"v1","addr":"172.21.15.63:6837","nonce":1347752311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6744,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2854804427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2854804427},{"type":"v1","addr":"172.21.15.45:6834","nonce":2854804427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:47:00.211837+0000","modified":"2024-05-08T05:48:24.018244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6747},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6747":{"gid":6747,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":25,"addr":"172.21.15.45:6835/2945763952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2945763952},{"type":"v1","addr":"172.21.15.45:6835","nonce":2945763952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6747,"qdb_cluster":[6747]},"id":6}]} 2024-05-08T05:48:59.396 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 73 2024-05-08T05:48:59.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.397+0000 7fafc7e93640 1 -- 172.21.15.45:0/529497302 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fafa4076300 msgr2=0x7fafa40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:59.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.397+0000 7fafc7e93640 1 --2- 172.21.15.45:0/529497302 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fafa4076300 0x7fafa40787c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fafb0000f60 tx=0x7fafb00079b0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.397+0000 7fafc7e93640 1 -- 172.21.15.45:0/529497302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafc00d0cb0 msgr2=0x7fafc0154340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:59.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.397+0000 7fafc7e93640 1 --2- 172.21.15.45:0/529497302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafc00d0cb0 0x7fafc0154340 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fafbc02a190 tx=0x7fafbc0046c0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.397+0000 7fafc7e93640 1 -- 172.21.15.45:0/529497302 shutdown_connections 2024-05-08T05:48:59.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.397+0000 7fafc7e93640 1 --2- 172.21.15.45:0/529497302 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fafa4076300 0x7fafa40787c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.397+0000 7fafc7e93640 1 --2- 172.21.15.45:0/529497302 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fafc01629a0 0x7fafc00a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.397+0000 7fafc7e93640 1 --2- 172.21.15.45:0/529497302 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafc00d2dc0 0x7fafc00a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.397+0000 7fafc7e93640 1 --2- 172.21.15.45:0/529497302 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fafc00d0cb0 0x7fafc0154340 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.397+0000 7fafc7e93640 1 -- 172.21.15.45:0/529497302 >> 172.21.15.45:0/529497302 conn(0x7fafc00d4fe0 msgr2=0x7fafc00b6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:59.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.397+0000 7fafc7e93640 1 -- 172.21.15.45:0/529497302 shutdown_connections 2024-05-08T05:48:59.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.397+0000 7fafc7e93640 1 -- 172.21.15.45:0/529497302 wait complete. 2024-05-08T05:48:59.408 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:48:59.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.512+0000 7ff4d7099640 1 Processor -- start 2024-05-08T05:48:59.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.512+0000 7ff4d7099640 1 -- start start 2024-05-08T05:48:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4d7099640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff4d00b2490 0x7ff4d0070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4d7099640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4d006e110 con 0x7ff4d01629b0 2024-05-08T05:48:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4d7099640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4d006e3f0 con 0x7ff4d0070d20 2024-05-08T05:48:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4d7099640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff4d006dec0 con 0x7ff4d00b2490 2024-05-08T05:48:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4d4e35640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff4d01629b0 0x7ff4d006eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38802/0 (socket says 172.21.15.45:38802) 2024-05-08T05:48:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4d4e35640 1 -- 172.21.15.45:0/1240852643 learned_addr learned my addr 172.21.15.45:0/1240852643 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:48:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4cffff640 1 --2- 172.21.15.45:0/1240852643 >> v2:172.21.15.63:3300/0 conn(0x7ff4d00b2490 0x7ff4d0070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4cf7fe640 1 -- 172.21.15.45:0/1240852643 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1958727503 0 0) 0x7ff4d006e110 con 0x7ff4d01629b0 2024-05-08T05:48:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4cf7fe640 1 -- 172.21.15.45:0/1240852643 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4c0003520 con 0x7ff4d01629b0 2024-05-08T05:48:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4cf7fe640 1 -- 172.21.15.45:0/1240852643 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3141970225 0 0) 0x7ff4d006e3f0 con 0x7ff4d0070d20 2024-05-08T05:48:59.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4cf7fe640 1 -- 172.21.15.45:0/1240852643 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4d006e110 con 0x7ff4d0070d20 2024-05-08T05:48:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4cf7fe640 1 -- 172.21.15.45:0/1240852643 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3848838152 0 0) 0x7ff4c0003520 con 0x7ff4d01629b0 2024-05-08T05:48:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4cf7fe640 1 -- 172.21.15.45:0/1240852643 >> v2:172.21.15.63:3300/0 conn(0x7ff4d00b2490 msgr2=0x7ff4d0070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4cf7fe640 1 --2- 172.21.15.45:0/1240852643 >> v2:172.21.15.63:3300/0 conn(0x7ff4d00b2490 0x7ff4d0070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4cf7fe640 1 -- 172.21.15.45:0/1240852643 >> v1:172.21.15.63:6789/0 conn(0x7ff4d0070d20 legacy=0x7ff4d00afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4cf7fe640 1 -- 172.21.15.45:0/1240852643 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d006e6d0 con 0x7ff4d01629b0 2024-05-08T05:48:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4cf7fe640 1 -- 172.21.15.45:0/1240852643 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7ff4c40016f0 con 0x7ff4d01629b0 2024-05-08T05:48:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4cf7fe640 1 -- 172.21.15.45:0/1240852643 >> v1:172.21.15.45:6789/0 conn(0x7ff4d01629b0 legacy=0x7ff4d006eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4cf7fe640 1 --2- 172.21.15.45:0/1240852643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4c0003c30 0x7ff4c0004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4cf7fe640 1 --2- 172.21.15.45:0/1240852643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4c0004800 0x7ff4c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4cf7fe640 1 --2- 172.21.15.45:0/1240852643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c0005210 0x7ff4c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4cf7fe640 1 -- 172.21.15.45:0/1240852643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d006dec0 con 0x7ff4c0005210 2024-05-08T05:48:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4cf7fe640 1 -- 172.21.15.45:0/1240852643 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c000c140 con 0x7ff4c0004800 2024-05-08T05:48:59.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4cf7fe640 1 -- 172.21.15.45:0/1240852643 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c000c340 con 0x7ff4c0003c30 2024-05-08T05:48:59.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.513+0000 7ff4d5636640 1 --2- 172.21.15.45:0/1240852643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c0005210 0x7ff4c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:59.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.514+0000 7ff4cffff640 1 --2- 172.21.15.45:0/1240852643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4c0004800 0x7ff4c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:59.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.514+0000 7ff4d4e35640 1 --2- 172.21.15.45:0/1240852643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4c0003c30 0x7ff4c0004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:59.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.514+0000 7ff4d5636640 1 -- 172.21.15.45:0/1240852643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4c0003c30 msgr2=0x7ff4c0004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:59.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.514+0000 7ff4d5636640 1 --2- 172.21.15.45:0/1240852643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4c0003c30 0x7ff4c0004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.514+0000 7ff4d5636640 1 -- 172.21.15.45:0/1240852643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4c0004800 msgr2=0x7ff4c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:59.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.514+0000 7ff4d5636640 1 --2- 172.21.15.45:0/1240852643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4c0004800 0x7ff4c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.514+0000 7ff4d5636640 1 -- 172.21.15.45:0/1240852643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c0003520 con 0x7ff4c0005210 2024-05-08T05:48:59.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.514+0000 7ff4d5636640 1 -- 172.21.15.45:0/1240852643 reap_dead start 2024-05-08T05:48:59.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.514+0000 7ff4d5636640 1 --2- 172.21.15.45:0/1240852643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c0005210 0x7ff4c000b9c0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8002000 tx=0x7ff4c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8887a982e334aa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:59.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.515+0000 7ff4cf7fe640 1 -- 172.21.15.45:0/1240852643 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8001c20 con 0x7ff4c0005210 2024-05-08T05:48:59.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.515+0000 7ff4cf7fe640 1 -- 172.21.15.45:0/1240852643 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c8003b70 con 0x7ff4c0005210 2024-05-08T05:48:59.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.515+0000 7ff4cf7fe640 1 -- 172.21.15.45:0/1240852643 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c80050d0 con 0x7ff4c0005210 2024-05-08T05:48:59.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.515+0000 7ff4d7099640 1 -- 172.21.15.45:0/1240852643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c0005210 msgr2=0x7ff4c000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.515+0000 7ff4d7099640 1 --2- 172.21.15.45:0/1240852643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c0005210 0x7ff4c000b9c0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8002000 tx=0x7ff4c0005830 comp rx=0 tx=0).stop 2024-05-08T05:48:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.515+0000 7ff4d7099640 1 -- 172.21.15.45:0/1240852643 shutdown_connections 2024-05-08T05:48:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.515+0000 7ff4d7099640 1 --2- 172.21.15.45:0/1240852643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c0005210 0x7ff4c000b9c0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.515+0000 7ff4d7099640 1 -- 172.21.15.45:0/1240852643 >> 172.21.15.45:0/1240852643 conn(0x7ff4d00b49d0 msgr2=0x7ff4d00b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.515+0000 7ff4d7099640 1 -- 172.21.15.45:0/1240852643 shutdown_connections 2024-05-08T05:48:59.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.515+0000 7ff4d7099640 1 -- 172.21.15.45:0/1240852643 wait complete. 2024-05-08T05:48:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.516+0000 7ff4d7099640 1 Processor -- start 2024-05-08T05:48:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.516+0000 7ff4d7099640 1 -- start start 2024-05-08T05:48:59.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.517+0000 7ff4d7099640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c0005210 0x7ff4d00b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.517+0000 7ff4d7099640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d00b5ac0 0x7ff4d00b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.517+0000 7ff4d7099640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4d015bb70 0x7ff4d015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.517+0000 7ff4d7099640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d0071350 con 0x7ff4c0005210 2024-05-08T05:48:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.517+0000 7ff4d7099640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d00711d0 con 0x7ff4d00b5ac0 2024-05-08T05:48:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.517+0000 7ff4d7099640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff4d00714d0 con 0x7ff4d015bb70 2024-05-08T05:48:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.517+0000 7ff4d4e35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c0005210 0x7ff4d00b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.517+0000 7ff4d4e35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c0005210 0x7ff4d00b70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60216/0 (socket says 172.21.15.45:60216) 2024-05-08T05:48:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.517+0000 7ff4d4e35640 1 -- 172.21.15.45:0/1142673974 learned_addr learned my addr 172.21.15.45:0/1142673974 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.518+0000 7ff4cffff640 1 --2- 172.21.15.45:0/1142673974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d00b5ac0 0x7ff4d00b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.518+0000 7ff4d4e35640 1 -- 172.21.15.45:0/1142673974 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4d015bb70 msgr2=0x7ff4d015bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.518+0000 7ff4d5636640 1 --2- 172.21.15.45:0/1142673974 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4d015bb70 0x7ff4d015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.518+0000 7ff4d4e35640 1 --2- 172.21.15.45:0/1142673974 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4d015bb70 0x7ff4d015bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.518+0000 7ff4d4e35640 1 -- 172.21.15.45:0/1142673974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d00b5ac0 msgr2=0x7ff4d00b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:59.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.518+0000 7ff4d4e35640 1 --2- 172.21.15.45:0/1142673974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d00b5ac0 0x7ff4d00b0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.518+0000 7ff4d4e35640 1 -- 172.21.15.45:0/1142673974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d015c530 con 0x7ff4c0005210 2024-05-08T05:48:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.518+0000 7ff4cffff640 1 --2- 172.21.15.45:0/1142673974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d00b5ac0 0x7ff4d00b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:48:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.518+0000 7ff4d5636640 1 --2- 172.21.15.45:0/1142673974 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4d015bb70 0x7ff4d015bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:48:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.518+0000 7ff4d4e35640 1 --2- 172.21.15.45:0/1142673974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c0005210 0x7ff4d00b70c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ff4c40072c0 tx=0x7ff4c4007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.518+0000 7ff4cd7fa640 1 -- 172.21.15.45:0/1142673974 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c40016b0 con 0x7ff4c0005210 2024-05-08T05:48:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.518+0000 7ff4cd7fa640 1 -- 172.21.15.45:0/1142673974 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c4018070 con 0x7ff4c0005210 2024-05-08T05:48:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.518+0000 7ff4d7099640 1 -- 172.21.15.45:0/1142673974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d00b7910 con 0x7ff4c0005210 2024-05-08T05:48:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.518+0000 7ff4cd7fa640 1 -- 172.21.15.45:0/1142673974 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c400fbe0 con 0x7ff4c0005210 2024-05-08T05:48:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.518+0000 7ff4d7099640 1 -- 172.21.15.45:0/1142673974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d00b7ed0 con 0x7ff4c0005210 2024-05-08T05:48:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.519+0000 7ff4d7099640 1 -- 172.21.15.45:0/1142673974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d00693f0 con 0x7ff4c0005210 2024-05-08T05:48:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.520+0000 7ff4cd7fa640 1 -- 172.21.15.45:0/1142673974 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff4c401d460 con 0x7ff4c0005210 2024-05-08T05:48:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.520+0000 7ff4cd7fa640 1 --2- 172.21.15.45:0/1142673974 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff4b40762d0 0x7ff4b4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.520+0000 7ff4cd7fa640 1 -- 172.21.15.45:0/1142673974 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7ff4c4097830 con 0x7ff4c0005210 2024-05-08T05:48:59.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.521+0000 7ff4cffff640 1 --2- 172.21.15.45:0/1142673974 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff4b40762d0 0x7ff4b4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:59.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.521+0000 7ff4cffff640 1 --2- 172.21.15.45:0/1142673974 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff4b40762d0 0x7ff4b4078790 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0093d0 tx=0x7ff4bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:59.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.525+0000 7ff4cd7fa640 1 -- 172.21.15.45:0/1142673974 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff4c4060f70 con 0x7ff4c0005210 2024-05-08T05:48:59.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.716+0000 7ff4d7099640 1 -- 172.21.15.45:0/1142673974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4d00b44d0 con 0x7ff4c0005210 2024-05-08T05:48:59.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.717+0000 7ff4cd7fa640 1 -- 172.21.15.45:0/1142673974 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+17289 (secure 0 0 0) 0x7ff4c4060870 con 0x7ff4c0005210 2024-05-08T05:48:59.718 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:48:59.718 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":92,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:48:20.008805+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:46:57.892382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-08T05:46:59.168198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":1,"to":7}]},{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.3a","mappings":[{"from":1,"to":5}]},{"pgid":"14.3c","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6837/3194497846":"2024-05-09T05:48:20.008781+0000","172.21.15.63:6835/3194497846":"2024-05-09T05:48:20.008781+0000","172.21.15.63:6836/2942326473":"2024-05-09T05:47:09.529461+0000","172.21.15.63:6834/2942326473":"2024-05-09T05:47:09.529461+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:48:59.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.719+0000 7ff4d7099640 1 -- 172.21.15.45:0/1142673974 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff4b40762d0 msgr2=0x7ff4b4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:59.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.720+0000 7ff4d7099640 1 --2- 172.21.15.45:0/1142673974 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff4b40762d0 0x7ff4b4078790 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0093d0 tx=0x7ff4bc008040 comp rx=0 tx=0).stop 2024-05-08T05:48:59.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.720+0000 7ff4d7099640 1 -- 172.21.15.45:0/1142673974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c0005210 msgr2=0x7ff4d00b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:59.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.720+0000 7ff4d7099640 1 --2- 172.21.15.45:0/1142673974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c0005210 0x7ff4d00b70c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ff4c40072c0 tx=0x7ff4c4007780 comp rx=0 tx=0).stop 2024-05-08T05:48:59.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.720+0000 7ff4d7099640 1 -- 172.21.15.45:0/1142673974 shutdown_connections 2024-05-08T05:48:59.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.720+0000 7ff4d7099640 1 --2- 172.21.15.45:0/1142673974 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff4b40762d0 0x7ff4b4078790 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.720+0000 7ff4d7099640 1 --2- 172.21.15.45:0/1142673974 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4d015bb70 0x7ff4d015bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.720+0000 7ff4d7099640 1 --2- 172.21.15.45:0/1142673974 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4d00b5ac0 0x7ff4d00b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.720+0000 7ff4d7099640 1 --2- 172.21.15.45:0/1142673974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4c0005210 0x7ff4d00b70c0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.720+0000 7ff4d7099640 1 -- 172.21.15.45:0/1142673974 >> 172.21.15.45:0/1142673974 conn(0x7ff4d00b49d0 msgr2=0x7ff4d00d1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:59.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.720+0000 7ff4d7099640 1 -- 172.21.15.45:0/1142673974 shutdown_connections 2024-05-08T05:48:59.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.720+0000 7ff4d7099640 1 -- 172.21.15.45:0/1142673974 wait complete. 2024-05-08T05:48:59.732 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T05:48:59.732 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:48:59.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.835+0000 7fa1e835f640 1 Processor -- start 2024-05-08T05:48:59.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.835+0000 7fa1e835f640 1 -- start start 2024-05-08T05:48:59.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.836+0000 7fa1e835f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1e015f420 0x7fa1e00d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:59.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.836+0000 7fa1e835f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa1e00d2080 0x7fa1e00d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:59.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.836+0000 7fa1e835f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1e01630b0 con 0x7fa1e00d3ca0 2024-05-08T05:48:59.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.836+0000 7fa1e835f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa1e00d2550 con 0x7fa1e015f420 2024-05-08T05:48:59.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.836+0000 7fa1e835f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa1e00d2c80 con 0x7fa1e00d2080 2024-05-08T05:48:59.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.836+0000 7fa1e60fb640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1e015f420 0x7fa1e00d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:59.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.836+0000 7fa1e58fa640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa1e00d3ca0 0x7fa1e00d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38808/0 (socket says 172.21.15.45:38808) 2024-05-08T05:48:59.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.836+0000 7fa1e58fa640 1 -- 172.21.15.45:0/1626878704 learned_addr learned my addr 172.21.15.45:0/1626878704 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:48:59.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.837+0000 7fa1e68fc640 1 --2- 172.21.15.45:0/1626878704 >> v2:172.21.15.63:3300/0 conn(0x7fa1e00d2080 0x7fa1e00d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:59.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.837+0000 7fa1e60fb640 1 -- 172.21.15.45:0/1626878704 >> v2:172.21.15.63:3300/0 conn(0x7fa1e00d2080 msgr2=0x7fa1e00d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:48:59.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.837+0000 7fa1e60fb640 1 --2- 172.21.15.45:0/1626878704 >> v2:172.21.15.63:3300/0 conn(0x7fa1e00d2080 0x7fa1e00d4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.837+0000 7fa1e60fb640 1 -- 172.21.15.45:0/1626878704 >> v1:172.21.15.45:6789/0 conn(0x7fa1e00d3ca0 legacy=0x7fa1e00d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:59.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.837+0000 7fa1e60fb640 1 -- 172.21.15.45:0/1626878704 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e00d2f60 con 0x7fa1e015f420 2024-05-08T05:48:59.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.837+0000 7fa1e50f9640 1 -- 172.21.15.45:0/1626878704 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 320778730 0 0) 0x7fa1d4001110 con 0x7fa1e00d3ca0 2024-05-08T05:48:59.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.837+0000 7fa1e68fc640 1 --2- 172.21.15.45:0/1626878704 >> v2:172.21.15.63:3300/0 conn(0x7fa1e00d2080 0x7fa1e00d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:59.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.838+0000 7fa1e60fb640 1 --2- 172.21.15.45:0/1626878704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e015f420 0x7fa1e00d3760 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00d5e90 tx=0x7fa1d802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45eb47a89baf3e86 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:59.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.838+0000 7fa1e50f9640 1 -- 172.21.15.45:0/1626878704 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8037070 con 0x7fa1e015f420 2024-05-08T05:48:59.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.838+0000 7fa1e50f9640 1 -- 172.21.15.45:0/1626878704 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d8032440 con 0x7fa1e015f420 2024-05-08T05:48:59.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.838+0000 7fa1e50f9640 1 -- 172.21.15.45:0/1626878704 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8032810 con 0x7fa1e015f420 2024-05-08T05:48:59.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.838+0000 7fa1e835f640 1 -- 172.21.15.45:0/1626878704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e015f420 msgr2=0x7fa1e00d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:59.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.838+0000 7fa1e835f640 1 --2- 172.21.15.45:0/1626878704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e015f420 0x7fa1e00d3760 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00d5e90 tx=0x7fa1d802a1a0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.838+0000 7fa1e835f640 1 -- 172.21.15.45:0/1626878704 shutdown_connections 2024-05-08T05:48:59.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.838+0000 7fa1e835f640 1 --2- 172.21.15.45:0/1626878704 >> v2:172.21.15.63:3300/0 conn(0x7fa1e00d2080 0x7fa1e00d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.838+0000 7fa1e835f640 1 --2- 172.21.15.45:0/1626878704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e015f420 0x7fa1e00d3760 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.838+0000 7fa1e835f640 1 -- 172.21.15.45:0/1626878704 >> 172.21.15.45:0/1626878704 conn(0x7fa1e008eac0 msgr2=0x7fa1e009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:59.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.838+0000 7fa1e835f640 1 -- 172.21.15.45:0/1626878704 shutdown_connections 2024-05-08T05:48:59.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.838+0000 7fa1e835f640 1 -- 172.21.15.45:0/1626878704 wait complete. 2024-05-08T05:48:59.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.840+0000 7fa1e835f640 1 Processor -- start 2024-05-08T05:48:59.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.840+0000 7fa1e835f640 1 -- start start 2024-05-08T05:48:59.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.840+0000 7fa1e835f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00d2080 0x7fa1e01624a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:59.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.840+0000 7fa1e835f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1e015f880 0x7fa1e01f96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:59.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.840+0000 7fa1e835f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e00b5ed0 0x7fa1e014ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:59.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.840+0000 7fa1e835f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e00d5200 con 0x7fa1e00b5ed0 2024-05-08T05:48:59.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.840+0000 7fa1e835f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e00d2830 con 0x7fa1e00d2080 2024-05-08T05:48:59.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.840+0000 7fa1e835f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e00d5380 con 0x7fa1e015f880 2024-05-08T05:48:59.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.840+0000 7fa1e68fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e00b5ed0 0x7fa1e014ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:59.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.840+0000 7fa1e68fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e00b5ed0 0x7fa1e014ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60240/0 (socket says 172.21.15.45:60240) 2024-05-08T05:48:59.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.840+0000 7fa1e68fc640 1 -- 172.21.15.45:0/293063794 learned_addr learned my addr 172.21.15.45:0/293063794 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:48:59.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.841+0000 7fa1e60fb640 1 --2- 172.21.15.45:0/293063794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00d2080 0x7fa1e01624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:59.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.841+0000 7fa1e68fc640 1 -- 172.21.15.45:0/293063794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1e015f880 msgr2=0x7fa1e01f96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:59.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.841+0000 7fa1e68fc640 1 --2- 172.21.15.45:0/293063794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1e015f880 0x7fa1e01f96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.841+0000 7fa1e68fc640 1 -- 172.21.15.45:0/293063794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00d2080 msgr2=0x7fa1e01624a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:59.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.841+0000 7fa1e68fc640 1 --2- 172.21.15.45:0/293063794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00d2080 0x7fa1e01624a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:59.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.841+0000 7fa1e68fc640 1 -- 172.21.15.45:0/293063794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e015fd00 con 0x7fa1e00b5ed0 2024-05-08T05:48:59.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.841+0000 7fa1e68fc640 1 --2- 172.21.15.45:0/293063794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e00b5ed0 0x7fa1e014ddc0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0007ec0 tx=0x7fa1d000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:59.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.841+0000 7fa1deffd640 1 -- 172.21.15.45:0/293063794 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d001a070 con 0x7fa1e00b5ed0 2024-05-08T05:48:59.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.841+0000 7fa1deffd640 1 -- 172.21.15.45:0/293063794 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d0003b90 con 0x7fa1e00b5ed0 2024-05-08T05:48:59.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.841+0000 7fa1e835f640 1 -- 172.21.15.45:0/293063794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e01616e0 con 0x7fa1e00b5ed0 2024-05-08T05:48:59.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.841+0000 7fa1deffd640 1 -- 172.21.15.45:0/293063794 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0004030 con 0x7fa1e00b5ed0 2024-05-08T05:48:59.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.841+0000 7fa1e835f640 1 -- 172.21.15.45:0/293063794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e0161ca0 con 0x7fa1e00b5ed0 2024-05-08T05:48:59.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.842+0000 7fa1e835f640 1 -- 172.21.15.45:0/293063794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e00693f0 con 0x7fa1e00b5ed0 2024-05-08T05:48:59.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.843+0000 7fa1deffd640 1 -- 172.21.15.45:0/293063794 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa1d001d070 con 0x7fa1e00b5ed0 2024-05-08T05:48:59.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.844+0000 7fa1deffd640 1 --2- 172.21.15.45:0/293063794 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1cc0762d0 0x7fa1cc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:59.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.844+0000 7fa1deffd640 1 -- 172.21.15.45:0/293063794 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7fa1d0097b40 con 0x7fa1e00b5ed0 2024-05-08T05:48:59.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.844+0000 7fa1e60fb640 1 --2- 172.21.15.45:0/293063794 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1cc0762d0 0x7fa1cc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:59.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.844+0000 7fa1e60fb640 1 --2- 172.21.15.45:0/293063794 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1cc0762d0 0x7fa1cc078790 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8034000 tx=0x7fa1d802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:59.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:48:59.848+0000 7fa1deffd640 1 -- 172.21.15.45:0/293063794 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa1d0010070 con 0x7fa1e00b5ed0 2024-05-08T05:49:00.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.079+0000 7fa1e835f640 1 -- 172.21.15.45:0/293063794 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1e00babf0 con 0x7fa1e00b5ed0 2024-05-08T05:49:00.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.080+0000 7fa1deffd640 1 -- 172.21.15.45:0/293063794 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5185 (secure 0 0 0) 0x7fa1d0061280 con 0x7fa1e00b5ed0 2024-05-08T05:49:00.080 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:00.081 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2616298287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2616298287},{"type":"v1","addr":"172.21.15.63:6836","nonce":2616298287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6472,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1347752311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1347752311},{"type":"v1","addr":"172.21.15.63:6837","nonce":1347752311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6744,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2854804427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2854804427},{"type":"v1","addr":"172.21.15.45:6834","nonce":2854804427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:47:00.211837+0000","modified":"2024-05-08T05:48:24.018244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6747},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6747":{"gid":6747,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":25,"addr":"172.21.15.45:6835/2945763952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2945763952},{"type":"v1","addr":"172.21.15.45:6835","nonce":2945763952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6747,"qdb_cluster":[6747]},"id":6}]} 2024-05-08T05:49:00.081 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 73 2024-05-08T05:49:00.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.082+0000 7fa1e835f640 1 -- 172.21.15.45:0/293063794 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1cc0762d0 msgr2=0x7fa1cc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.082+0000 7fa1e835f640 1 --2- 172.21.15.45:0/293063794 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1cc0762d0 0x7fa1cc078790 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8034000 tx=0x7fa1d802e000 comp rx=0 tx=0).stop 2024-05-08T05:49:00.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.082+0000 7fa1e835f640 1 -- 172.21.15.45:0/293063794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e00b5ed0 msgr2=0x7fa1e014ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.082+0000 7fa1e835f640 1 --2- 172.21.15.45:0/293063794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e00b5ed0 0x7fa1e014ddc0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0007ec0 tx=0x7fa1d000d4a0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.082+0000 7fa1e835f640 1 -- 172.21.15.45:0/293063794 shutdown_connections 2024-05-08T05:49:00.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.082+0000 7fa1e835f640 1 --2- 172.21.15.45:0/293063794 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1cc0762d0 0x7fa1cc078790 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.082+0000 7fa1e835f640 1 --2- 172.21.15.45:0/293063794 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1e015f880 0x7fa1e01f96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.082+0000 7fa1e835f640 1 --2- 172.21.15.45:0/293063794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1e00b5ed0 0x7fa1e014ddc0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.082+0000 7fa1e835f640 1 --2- 172.21.15.45:0/293063794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00d2080 0x7fa1e01624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.082+0000 7fa1e835f640 1 -- 172.21.15.45:0/293063794 >> 172.21.15.45:0/293063794 conn(0x7fa1e008eac0 msgr2=0x7fa1e006cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:00.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.082+0000 7fa1e835f640 1 -- 172.21.15.45:0/293063794 shutdown_connections 2024-05-08T05:49:00.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.082+0000 7fa1e835f640 1 -- 172.21.15.45:0/293063794 wait complete. 2024-05-08T05:49:00.093 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:49:00.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.197+0000 7f5c0d30a640 1 Processor -- start 2024-05-08T05:49:00.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.197+0000 7f5c0d30a640 1 -- start start 2024-05-08T05:49:00.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.198+0000 7f5c0d30a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5c0815f0c0 0x7f5c080b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.198+0000 7f5c0d30a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c0806c3a0 0x7f5c0806c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.198+0000 7f5c0d30a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c080b14e0 con 0x7f5c081638e0 2024-05-08T05:49:00.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.198+0000 7f5c0d30a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f5c08163450 con 0x7f5c0815f0c0 2024-05-08T05:49:00.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.198+0000 7f5c0d30a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5c081604b0 con 0x7f5c0806c3a0 2024-05-08T05:49:00.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.198+0000 7f5c06ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5c0815f0c0 0x7f5c080b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.198+0000 7f5c067fc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5c081638e0 0x7f5c080b0d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38820/0 (socket says 172.21.15.45:38820) 2024-05-08T05:49:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.198+0000 7f5c067fc640 1 -- 172.21.15.45:0/1336340548 learned_addr learned my addr 172.21.15.45:0/1336340548 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.198+0000 7f5c077fe640 1 --2- 172.21.15.45:0/1336340548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c0806c3a0 0x7f5c0806c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.198+0000 7f5c06ffd640 1 -- 172.21.15.45:0/1336340548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c0806c3a0 msgr2=0x7f5c0806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.198+0000 7f5c06ffd640 1 --2- 172.21.15.45:0/1336340548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c0806c3a0 0x7f5c0806c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.198+0000 7f5c06ffd640 1 -- 172.21.15.45:0/1336340548 >> v1:172.21.15.45:6789/0 conn(0x7f5c081638e0 legacy=0x7f5c080b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.198+0000 7f5c05ffb640 1 -- 172.21.15.45:0/1336340548 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2322467082 0 0) 0x7f5c080b14e0 con 0x7f5c081638e0 2024-05-08T05:49:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.198+0000 7f5c06ffd640 1 -- 172.21.15.45:0/1336340548 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c08162e00 con 0x7f5c0815f0c0 2024-05-08T05:49:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.199+0000 7f5c06ffd640 1 --2- 172.21.15.45:0/1336340548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c0815f0c0 0x7f5c080b6f40 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009d90 tx=0x7f5c0002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca59ce02727b070 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.199+0000 7f5c05ffb640 1 -- 172.21.15.45:0/1336340548 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00007740 con 0x7f5c0815f0c0 2024-05-08T05:49:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.199+0000 7f5c05ffb640 1 -- 172.21.15.45:0/1336340548 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c000078e0 con 0x7f5c0815f0c0 2024-05-08T05:49:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.199+0000 7f5c05ffb640 1 -- 172.21.15.45:0/1336340548 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00007cb0 con 0x7f5c0815f0c0 2024-05-08T05:49:00.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.200+0000 7f5c0d30a640 1 -- 172.21.15.45:0/1336340548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c0815f0c0 msgr2=0x7f5c080b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.200+0000 7f5c0d30a640 1 --2- 172.21.15.45:0/1336340548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c0815f0c0 0x7f5c080b6f40 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009d90 tx=0x7f5c0002a660 comp rx=0 tx=0).stop 2024-05-08T05:49:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.200+0000 7f5c0d30a640 1 -- 172.21.15.45:0/1336340548 shutdown_connections 2024-05-08T05:49:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.200+0000 7f5c0d30a640 1 --2- 172.21.15.45:0/1336340548 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c0806c3a0 0x7f5c0806c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.200+0000 7f5c0d30a640 1 --2- 172.21.15.45:0/1336340548 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c0815f0c0 0x7f5c080b6f40 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.200+0000 7f5c0d30a640 1 -- 172.21.15.45:0/1336340548 >> 172.21.15.45:0/1336340548 conn(0x7f5c080b3b10 msgr2=0x7f5c080b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.200+0000 7f5c0d30a640 1 -- 172.21.15.45:0/1336340548 shutdown_connections 2024-05-08T05:49:00.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.200+0000 7f5c0d30a640 1 -- 172.21.15.45:0/1336340548 wait complete. 2024-05-08T05:49:00.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.201+0000 7f5c0d30a640 1 Processor -- start 2024-05-08T05:49:00.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.201+0000 7f5c0d30a640 1 -- start start 2024-05-08T05:49:00.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.202+0000 7f5c0d30a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c0806c3a0 0x7f5c0815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.202+0000 7f5c0d30a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c0815f0c0 0x7f5c08153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.202+0000 7f5c0d30a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c081638e0 0x7f5c08153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.202+0000 7f5c0d30a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0815f4a0 con 0x7f5c081638e0 2024-05-08T05:49:00.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.202+0000 7f5c0d30a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5c080b17c0 con 0x7f5c0806c3a0 2024-05-08T05:49:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.202+0000 7f5c0d30a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5c0815f620 con 0x7f5c0815f0c0 2024-05-08T05:49:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.202+0000 7f5c077fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c081638e0 0x7f5c08153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.202+0000 7f5c077fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c081638e0 0x7f5c08153a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60258/0 (socket says 172.21.15.45:60258) 2024-05-08T05:49:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.202+0000 7f5c077fe640 1 -- 172.21.15.45:0/3389205819 learned_addr learned my addr 172.21.15.45:0/3389205819 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.202+0000 7f5c067fc640 1 --2- 172.21.15.45:0/3389205819 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c0815f0c0 0x7f5c08153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.202+0000 7f5c06ffd640 1 --2- 172.21.15.45:0/3389205819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c0806c3a0 0x7f5c0815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.202+0000 7f5c077fe640 1 -- 172.21.15.45:0/3389205819 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c0815f0c0 msgr2=0x7f5c08153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.202+0000 7f5c077fe640 1 --2- 172.21.15.45:0/3389205819 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c0815f0c0 0x7f5c08153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.202+0000 7f5c077fe640 1 -- 172.21.15.45:0/3389205819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c0806c3a0 msgr2=0x7f5c0815c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.202+0000 7f5c077fe640 1 --2- 172.21.15.45:0/3389205819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c0806c3a0 0x7f5c0815c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.202+0000 7f5c077fe640 1 -- 172.21.15.45:0/3389205819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c081f4b50 con 0x7f5c081638e0 2024-05-08T05:49:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.203+0000 7f5c077fe640 1 --2- 172.21.15.45:0/3389205819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c081638e0 0x7f5c08153a70 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800d800 tx=0x7f5bf800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:00.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.203+0000 7f5bff7fe640 1 -- 172.21.15.45:0/3389205819 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf800b6c0 con 0x7f5c081638e0 2024-05-08T05:49:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.203+0000 7f5bff7fe640 1 -- 172.21.15.45:0/3389205819 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf8019070 con 0x7f5c081638e0 2024-05-08T05:49:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.203+0000 7f5bff7fe640 1 -- 172.21.15.45:0/3389205819 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf800bac0 con 0x7f5c081638e0 2024-05-08T05:49:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.203+0000 7f5c0d30a640 1 -- 172.21.15.45:0/3389205819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c081f4e30 con 0x7f5c081638e0 2024-05-08T05:49:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.203+0000 7f5c0d30a640 1 -- 172.21.15.45:0/3389205819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c081f53f0 con 0x7f5c081638e0 2024-05-08T05:49:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.204+0000 7f5c0d30a640 1 -- 172.21.15.45:0/3389205819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c080693f0 con 0x7f5c081638e0 2024-05-08T05:49:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.204+0000 7f5bff7fe640 1 -- 172.21.15.45:0/3389205819 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5bf800bc60 con 0x7f5c081638e0 2024-05-08T05:49:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.205+0000 7f5bff7fe640 1 --2- 172.21.15.45:0/3389205819 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5be80762d0 0x7f5be8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.205+0000 7f5bff7fe640 1 -- 172.21.15.45:0/3389205819 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7f5bf8097c10 con 0x7f5c081638e0 2024-05-08T05:49:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.205+0000 7f5c06ffd640 1 --2- 172.21.15.45:0/3389205819 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5be80762d0 0x7f5be8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:00.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.206+0000 7f5c06ffd640 1 --2- 172.21.15.45:0/3389205819 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5be80762d0 0x7f5be8078790 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5c00034000 tx=0x7f5c0002f000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:00.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.210+0000 7f5bff7fe640 1 -- 172.21.15.45:0/3389205819 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5bf8060c70 con 0x7f5c081638e0 2024-05-08T05:49:00.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.437+0000 7f5c0d30a640 1 -- 172.21.15.45:0/3389205819 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c081542c0 con 0x7f5c081638e0 2024-05-08T05:49:00.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.437+0000 7f5bff7fe640 1 -- 172.21.15.45:0/3389205819 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5185 (secure 0 0 0) 0x7f5bf8064f70 con 0x7f5c081638e0 2024-05-08T05:49:00.438 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:00.438 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2616298287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2616298287},{"type":"v1","addr":"172.21.15.63:6836","nonce":2616298287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6472,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1347752311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1347752311},{"type":"v1","addr":"172.21.15.63:6837","nonce":1347752311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6744,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2854804427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2854804427},{"type":"v1","addr":"172.21.15.45:6834","nonce":2854804427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:47:00.211837+0000","modified":"2024-05-08T05:48:24.018244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6747},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6747":{"gid":6747,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":25,"addr":"172.21.15.45:6835/2945763952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":2945763952},{"type":"v1","addr":"172.21.15.45:6835","nonce":2945763952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6747,"qdb_cluster":[6747]},"id":6}]} 2024-05-08T05:49:00.438 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 73 2024-05-08T05:49:00.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.439+0000 7f5c0d30a640 1 -- 172.21.15.45:0/3389205819 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5be80762d0 msgr2=0x7f5be8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.439+0000 7f5c0d30a640 1 --2- 172.21.15.45:0/3389205819 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5be80762d0 0x7f5be8078790 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5c00034000 tx=0x7f5c0002f000 comp rx=0 tx=0).stop 2024-05-08T05:49:00.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.439+0000 7f5c0d30a640 1 -- 172.21.15.45:0/3389205819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c081638e0 msgr2=0x7f5c08153a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.439+0000 7f5c0d30a640 1 --2- 172.21.15.45:0/3389205819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c081638e0 0x7f5c08153a70 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800d800 tx=0x7f5bf800dcc0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.439+0000 7f5c0d30a640 1 -- 172.21.15.45:0/3389205819 shutdown_connections 2024-05-08T05:49:00.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.439+0000 7f5c0d30a640 1 --2- 172.21.15.45:0/3389205819 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5be80762d0 0x7f5be8078790 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.439+0000 7f5c0d30a640 1 --2- 172.21.15.45:0/3389205819 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c0815f0c0 0x7f5c08153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.439+0000 7f5c0d30a640 1 --2- 172.21.15.45:0/3389205819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c081638e0 0x7f5c08153a70 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.439+0000 7f5c0d30a640 1 --2- 172.21.15.45:0/3389205819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c0806c3a0 0x7f5c0815c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.439+0000 7f5c0d30a640 1 -- 172.21.15.45:0/3389205819 >> 172.21.15.45:0/3389205819 conn(0x7f5c080b3b10 msgr2=0x7f5c08073d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:00.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.439+0000 7f5c0d30a640 1 -- 172.21.15.45:0/3389205819 shutdown_connections 2024-05-08T05:49:00.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.440+0000 7f5c0d30a640 1 -- 172.21.15.45:0/3389205819 wait complete. 2024-05-08T05:49:00.451 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:49:00.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.554+0000 7f2a1613f640 1 Processor -- start 2024-05-08T05:49:00.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.555+0000 7f2a1613f640 1 -- start start 2024-05-08T05:49:00.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.555+0000 7f2a1613f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2a100d1970 0x7f2a100d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.555+0000 7f2a1613f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2a100d4cc0 0x7f2a100d50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.555+0000 7f2a1613f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a101630b0 con 0x7f2a1015f420 2024-05-08T05:49:00.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.555+0000 7f2a1613f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f2a100d3e70 con 0x7f2a100d4cc0 2024-05-08T05:49:00.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.555+0000 7f2a1613f640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2a100d2550 con 0x7f2a100d1970 2024-05-08T05:49:00.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.555+0000 7f2a0ffff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2a100d4cc0 0x7f2a100d50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:00.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.555+0000 7f2a0f7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2a1015f420 0x7f2a100d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38826/0 (socket says 172.21.15.45:38826) 2024-05-08T05:49:00.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.556+0000 7f2a0f7fe640 1 -- 172.21.15.45:0/2659764543 learned_addr learned my addr 172.21.15.45:0/2659764543 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:00.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.556+0000 7f2a0effd640 1 --2- 172.21.15.45:0/2659764543 >> v2:172.21.15.63:3300/0 conn(0x7f2a100d1970 0x7f2a100d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:00.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.556+0000 7f2a0ffff640 1 -- 172.21.15.45:0/2659764543 >> v2:172.21.15.63:3300/0 conn(0x7f2a100d1970 msgr2=0x7f2a100d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:00.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.556+0000 7f2a0ffff640 1 --2- 172.21.15.45:0/2659764543 >> v2:172.21.15.63:3300/0 conn(0x7f2a100d1970 0x7f2a100d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.556+0000 7f2a0ffff640 1 -- 172.21.15.45:0/2659764543 >> v1:172.21.15.45:6789/0 conn(0x7f2a1015f420 legacy=0x7f2a100d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.556+0000 7f2a0ffff640 1 -- 172.21.15.45:0/2659764543 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a100d2c80 con 0x7f2a100d4cc0 2024-05-08T05:49:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.557+0000 7f2a0ffff640 1 --2- 172.21.15.45:0/2659764543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a100d4cc0 0x7f2a100d50e0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f2a1008f380 tx=0x7f2a0802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29e25600edab8c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.557+0000 7f2a0e7fc640 1 -- 172.21.15.45:0/2659764543 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08037070 con 0x7f2a100d4cc0 2024-05-08T05:49:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.557+0000 7f2a0e7fc640 1 -- 172.21.15.45:0/2659764543 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a08032440 con 0x7f2a100d4cc0 2024-05-08T05:49:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.557+0000 7f2a0e7fc640 1 -- 172.21.15.45:0/2659764543 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08032810 con 0x7f2a100d4cc0 2024-05-08T05:49:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.557+0000 7f2a1613f640 1 -- 172.21.15.45:0/2659764543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a100d4cc0 msgr2=0x7f2a100d50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.557+0000 7f2a1613f640 1 --2- 172.21.15.45:0/2659764543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a100d4cc0 0x7f2a100d50e0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f2a1008f380 tx=0x7f2a0802a1a0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.557+0000 7f2a1613f640 1 -- 172.21.15.45:0/2659764543 shutdown_connections 2024-05-08T05:49:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.557+0000 7f2a1613f640 1 --2- 172.21.15.45:0/2659764543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a100d4cc0 0x7f2a100d50e0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.557+0000 7f2a1613f640 1 --2- 172.21.15.45:0/2659764543 >> v2:172.21.15.63:3300/0 conn(0x7f2a100d1970 0x7f2a100d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.557+0000 7f2a1613f640 1 -- 172.21.15.45:0/2659764543 >> 172.21.15.45:0/2659764543 conn(0x7f2a1008eac0 msgr2=0x7f2a1009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:00.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.557+0000 7f2a1613f640 1 -- 172.21.15.45:0/2659764543 shutdown_connections 2024-05-08T05:49:00.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.557+0000 7f2a1613f640 1 -- 172.21.15.45:0/2659764543 wait complete. 2024-05-08T05:49:00.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.558+0000 7f2a1613f640 1 Processor -- start 2024-05-08T05:49:00.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.559+0000 7f2a1613f640 1 -- start start 2024-05-08T05:49:00.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.559+0000 7f2a1613f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a100d1970 0x7f2a100ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.559+0000 7f2a1613f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a100d4cc0 0x7f2a10154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.559+0000 7f2a1613f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a100c9b40 0x7f2a100c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.559+0000 7f2a1613f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2a100d2830 con 0x7f2a100d1970 2024-05-08T05:49:00.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.559+0000 7f2a1613f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2a100d3240 con 0x7f2a100d4cc0 2024-05-08T05:49:00.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.559+0000 7f2a1613f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2a100d5620 con 0x7f2a100c9b40 2024-05-08T05:49:00.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.560+0000 7f2a0f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a100d1970 0x7f2a100ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:00.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.560+0000 7f2a0f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a100d1970 0x7f2a100ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60272/0 (socket says 172.21.15.45:60272) 2024-05-08T05:49:00.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.560+0000 7f2a0f7fe640 1 -- 172.21.15.45:0/830550380 learned_addr learned my addr 172.21.15.45:0/830550380 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:00.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.560+0000 7f2a0effd640 1 --2- 172.21.15.45:0/830550380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a100d4cc0 0x7f2a10154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:00.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.560+0000 7f2a0ffff640 1 --2- 172.21.15.45:0/830550380 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a100c9b40 0x7f2a100c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:00.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.560+0000 7f2a0f7fe640 1 -- 172.21.15.45:0/830550380 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a100c9b40 msgr2=0x7f2a100c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.560+0000 7f2a0f7fe640 1 --2- 172.21.15.45:0/830550380 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a100c9b40 0x7f2a100c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.560+0000 7f2a0f7fe640 1 -- 172.21.15.45:0/830550380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a100d4cc0 msgr2=0x7f2a10154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.560+0000 7f2a0f7fe640 1 --2- 172.21.15.45:0/830550380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a100d4cc0 0x7f2a10154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.560+0000 7f2a0f7fe640 1 -- 172.21.15.45:0/830550380 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a10160e00 con 0x7f2a100d1970 2024-05-08T05:49:00.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.560+0000 7f2a0effd640 1 --2- 172.21.15.45:0/830550380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a100d4cc0 0x7f2a10154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:00.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.560+0000 7f2a0f7fe640 1 --2- 172.21.15.45:0/830550380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a100d1970 0x7f2a100ca930 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000cae0 tx=0x7f2a0000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:00.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.560+0000 7f29f7fff640 1 -- 172.21.15.45:0/830550380 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00017070 con 0x7f2a100d1970 2024-05-08T05:49:00.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.560+0000 7f29f7fff640 1 -- 172.21.15.45:0/830550380 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a0000ac60 con 0x7f2a100d1970 2024-05-08T05:49:00.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.560+0000 7f2a1613f640 1 -- 172.21.15.45:0/830550380 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a101610e0 con 0x7f2a100d1970 2024-05-08T05:49:00.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.560+0000 7f29f7fff640 1 -- 172.21.15.45:0/830550380 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00012640 con 0x7f2a100d1970 2024-05-08T05:49:00.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.560+0000 7f2a1613f640 1 -- 172.21.15.45:0/830550380 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a100ca530 con 0x7f2a100d1970 2024-05-08T05:49:00.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.561+0000 7f2a1613f640 1 -- 172.21.15.45:0/830550380 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a100d50e0 con 0x7f2a100d1970 2024-05-08T05:49:00.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.562+0000 7f29f7fff640 1 -- 172.21.15.45:0/830550380 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2a0001a070 con 0x7f2a100d1970 2024-05-08T05:49:00.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.562+0000 7f29f7fff640 1 --2- 172.21.15.45:0/830550380 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f29fc0762d0 0x7f29fc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.562+0000 7f29f7fff640 1 -- 172.21.15.45:0/830550380 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7f2a000976c0 con 0x7f2a100d1970 2024-05-08T05:49:00.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.563+0000 7f2a0effd640 1 --2- 172.21.15.45:0/830550380 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f29fc0762d0 0x7f29fc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:00.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.563+0000 7f2a0effd640 1 --2- 172.21.15.45:0/830550380 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f29fc0762d0 0x7f29fc078790 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2a100cf0e0 tx=0x7f2a04008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:00.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.567+0000 7f29f7fff640 1 -- 172.21.15.45:0/830550380 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2a100d50e0 con 0x7f2a100d1970 2024-05-08T05:49:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.756+0000 7f2a1613f640 1 -- 172.21.15.45:0/830550380 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a101638c0 con 0x7f2a100d1970 2024-05-08T05:49:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.757+0000 7f29f7fff640 1 -- 172.21.15.45:0/830550380 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+17289 (secure 0 0 0) 0x7f2a101630b0 con 0x7f2a100d1970 2024-05-08T05:49:00.758 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:00.758 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":92,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:48:20.008805+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:46:57.892382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-05-08T05:46:59.168198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":1,"to":7}]},{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.3a","mappings":[{"from":1,"to":5}]},{"pgid":"14.3c","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6837/3194497846":"2024-05-09T05:48:20.008781+0000","172.21.15.63:6835/3194497846":"2024-05-09T05:48:20.008781+0000","172.21.15.63:6836/2942326473":"2024-05-09T05:47:09.529461+0000","172.21.15.63:6834/2942326473":"2024-05-09T05:47:09.529461+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:49:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.760+0000 7f2a1613f640 1 -- 172.21.15.45:0/830550380 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f29fc0762d0 msgr2=0x7f29fc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.760+0000 7f2a1613f640 1 --2- 172.21.15.45:0/830550380 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f29fc0762d0 0x7f29fc078790 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2a100cf0e0 tx=0x7f2a04008040 comp rx=0 tx=0).stop 2024-05-08T05:49:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.760+0000 7f2a1613f640 1 -- 172.21.15.45:0/830550380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a100d1970 msgr2=0x7f2a100ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.760+0000 7f2a1613f640 1 --2- 172.21.15.45:0/830550380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a100d1970 0x7f2a100ca930 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000cae0 tx=0x7f2a0000a060 comp rx=0 tx=0).stop 2024-05-08T05:49:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.760+0000 7f2a1613f640 1 -- 172.21.15.45:0/830550380 shutdown_connections 2024-05-08T05:49:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.760+0000 7f2a1613f640 1 --2- 172.21.15.45:0/830550380 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f29fc0762d0 0x7f29fc078790 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.760+0000 7f2a1613f640 1 --2- 172.21.15.45:0/830550380 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2a100c9b40 0x7f2a100c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.760+0000 7f2a1613f640 1 --2- 172.21.15.45:0/830550380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2a100d4cc0 0x7f2a10154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.760+0000 7f2a1613f640 1 --2- 172.21.15.45:0/830550380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2a100d1970 0x7f2a100ca930 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.760+0000 7f2a1613f640 1 -- 172.21.15.45:0/830550380 >> 172.21.15.45:0/830550380 conn(0x7f2a1008eac0 msgr2=0x7f2a1006d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.760+0000 7f2a1613f640 1 -- 172.21.15.45:0/830550380 shutdown_connections 2024-05-08T05:49:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.760+0000 7f2a1613f640 1 -- 172.21.15.45:0/830550380 wait complete. 2024-05-08T05:49:00.772 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T05:49:00.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.879+0000 7f9351a74640 1 Processor -- start 2024-05-08T05:49:00.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.879+0000 7f9351a74640 1 -- start start 2024-05-08T05:49:00.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.880+0000 7f9351a74640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f934c0d1970 0x7f934c0d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.880+0000 7f9351a74640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c0d4cc0 0x7f934c0d5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.880+0000 7f9351a74640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f934c1630b0 con 0x7f934c15f420 2024-05-08T05:49:00.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.880+0000 7f9351a74640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f934c0d3e70 con 0x7f934c0d1970 2024-05-08T05:49:00.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.880+0000 7f9351a74640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f934c0d5670 con 0x7f934c0d4cc0 2024-05-08T05:49:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.880+0000 7f934affd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f934c0d1970 0x7f934c0d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.880+0000 7f934b7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f934c15f420 0x7f934c0d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51116/0 (socket says 172.21.15.45:51116) 2024-05-08T05:49:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.880+0000 7f934b7fe640 1 -- 172.21.15.45:0/4276119945 learned_addr learned my addr 172.21.15.45:0/4276119945 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.880+0000 7f934bfff640 1 --2- 172.21.15.45:0/4276119945 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c0d4cc0 0x7f934c0d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.880+0000 7f934affd640 1 -- 172.21.15.45:0/4276119945 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c0d4cc0 msgr2=0x7f934c0d5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.880+0000 7f934affd640 1 --2- 172.21.15.45:0/4276119945 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c0d4cc0 0x7f934c0d5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.880+0000 7f934affd640 1 -- 172.21.15.45:0/4276119945 >> v1:172.21.15.45:6789/0 conn(0x7f934c15f420 legacy=0x7f934c0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.880+0000 7f934affd640 1 -- 172.21.15.45:0/4276119945 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934c0d2550 con 0x7f934c0d1970 2024-05-08T05:49:00.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.880+0000 7f934a7fc640 1 -- 172.21.15.45:0/4276119945 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3566707534 0 0) 0x7f933c001110 con 0x7f934c15f420 2024-05-08T05:49:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.881+0000 7f934affd640 1 --2- 172.21.15.45:0/4276119945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c0d1970 0x7f934c0d1e20 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f93400097d0 tx=0x7f934002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b8eed891b28c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.881+0000 7f934a7fc640 1 -- 172.21.15.45:0/4276119945 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340037070 con 0x7f934c0d1970 2024-05-08T05:49:00.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.881+0000 7f934a7fc640 1 -- 172.21.15.45:0/4276119945 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9340032440 con 0x7f934c0d1970 2024-05-08T05:49:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.881+0000 7f934a7fc640 1 -- 172.21.15.45:0/4276119945 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340032810 con 0x7f934c0d1970 2024-05-08T05:49:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.882+0000 7f9351a74640 1 -- 172.21.15.45:0/4276119945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c0d1970 msgr2=0x7f934c0d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.882+0000 7f9351a74640 1 --2- 172.21.15.45:0/4276119945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c0d1970 0x7f934c0d1e20 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f93400097d0 tx=0x7f934002a1c0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.882+0000 7f9351a74640 1 -- 172.21.15.45:0/4276119945 shutdown_connections 2024-05-08T05:49:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.882+0000 7f9351a74640 1 --2- 172.21.15.45:0/4276119945 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c0d4cc0 0x7f934c0d5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.882+0000 7f9351a74640 1 --2- 172.21.15.45:0/4276119945 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c0d1970 0x7f934c0d1e20 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.882+0000 7f9351a74640 1 -- 172.21.15.45:0/4276119945 >> 172.21.15.45:0/4276119945 conn(0x7f934c08eac0 msgr2=0x7f934c09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.882+0000 7f9351a74640 1 -- 172.21.15.45:0/4276119945 shutdown_connections 2024-05-08T05:49:00.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.882+0000 7f9351a74640 1 -- 172.21.15.45:0/4276119945 wait complete. 2024-05-08T05:49:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.883+0000 7f9351a74640 1 Processor -- start 2024-05-08T05:49:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.883+0000 7f9351a74640 1 -- start start 2024-05-08T05:49:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.884+0000 7f9351a74640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c0d1970 0x7f934c15d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.884+0000 7f9351a74640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c0d4cc0 0x7f934c153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.884+0000 7f9351a74640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f934c0b7c00 0x7f934c0cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.884+0000 7f9351a74640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f934c0d3240 con 0x7f934c0d1970 2024-05-08T05:49:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.884+0000 7f9351a74640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f934c0d2f60 con 0x7f934c0b7c00 2024-05-08T05:49:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.884+0000 7f9351a74640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f934c0d2830 con 0x7f934c0d4cc0 2024-05-08T05:49:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.884+0000 7f934b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c0d1970 0x7f934c15d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:00.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.884+0000 7f934b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c0d1970 0x7f934c15d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59554/0 (socket says 172.21.15.45:59554) 2024-05-08T05:49:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.884+0000 7f934b7fe640 1 -- 172.21.15.45:0/3118689434 learned_addr learned my addr 172.21.15.45:0/3118689434 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.884+0000 7f934b7fe640 1 -- 172.21.15.45:0/3118689434 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c0d4cc0 msgr2=0x7f934c153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.884+0000 7f934b7fe640 1 --2- 172.21.15.45:0/3118689434 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c0d4cc0 0x7f934c153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.884+0000 7f934b7fe640 1 -- 172.21.15.45:0/3118689434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f934c0b7c00 msgr2=0x7f934c0cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.884+0000 7f934b7fe640 1 --2- 172.21.15.45:0/3118689434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f934c0b7c00 0x7f934c0cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.884+0000 7f934b7fe640 1 -- 172.21.15.45:0/3118689434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934c153ad0 con 0x7f934c0d1970 2024-05-08T05:49:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.884+0000 7f934b7fe640 1 --2- 172.21.15.45:0/3118689434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c0d1970 0x7f934c15d660 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f933c00c400 tx=0x7f933c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.884+0000 7f9333fff640 1 -- 172.21.15.45:0/3118689434 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c017070 con 0x7f934c0d1970 2024-05-08T05:49:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.884+0000 7f9333fff640 1 -- 172.21.15.45:0/3118689434 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f933c005ce0 con 0x7f934c0d1970 2024-05-08T05:49:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.884+0000 7f9333fff640 1 -- 172.21.15.45:0/3118689434 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c007310 con 0x7f934c0d1970 2024-05-08T05:49:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.884+0000 7f9351a74640 1 -- 172.21.15.45:0/3118689434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f934c0ca280 con 0x7f934c0d1970 2024-05-08T05:49:00.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.885+0000 7f9351a74640 1 -- 172.21.15.45:0/3118689434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f934c0ca840 con 0x7f934c0d1970 2024-05-08T05:49:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.886+0000 7f9351a74640 1 -- 172.21.15.45:0/3118689434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f934c0693f0 con 0x7f934c0d1970 2024-05-08T05:49:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.886+0000 7f9333fff640 1 -- 172.21.15.45:0/3118689434 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f933c01a070 con 0x7f934c0d1970 2024-05-08T05:49:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.887+0000 7f9333fff640 1 --2- 172.21.15.45:0/3118689434 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9338076300 0x7f93380787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.887+0000 7f9333fff640 1 -- 172.21.15.45:0/3118689434 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6389+0+0 (secure 0 0 0) 0x7f933c096bb0 con 0x7f934c0d1970 2024-05-08T05:49:00.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.887+0000 7f934affd640 1 --2- 172.21.15.45:0/3118689434 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9338076300 0x7f93380787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:00.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.888+0000 7f934affd640 1 --2- 172.21.15.45:0/3118689434 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9338076300 0x7f93380787c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9340034000 tx=0x7f934002c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:00.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:00.891+0000 7f9333fff640 1 -- 172.21.15.45:0/3118689434 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f934c1630b0 con 0x7f934c0d1970 2024-05-08T05:49:01.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.122+0000 7f9351a74640 1 -- 172.21.15.45:0/3118689434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f934c0b4c50 con 0x7f934c0d1970 2024-05-08T05:49:01.122 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-05-08T05:49:01.123+0000 7f124de12640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T05:49:01.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.124+0000 7f9333fff640 1 -- 172.21.15.45:0/3118689434 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) ==== 161+0+0 (secure 0 0 0) 0x7f933c0602f0 con 0x7f934c0d1970 2024-05-08T05:49:01.124 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T05:49:01.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.126+0000 7f9351a74640 1 -- 172.21.15.45:0/3118689434 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9338076300 msgr2=0x7f93380787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.126+0000 7f9351a74640 1 --2- 172.21.15.45:0/3118689434 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9338076300 0x7f93380787c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9340034000 tx=0x7f934002c000 comp rx=0 tx=0).stop 2024-05-08T05:49:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.126+0000 7f9351a74640 1 -- 172.21.15.45:0/3118689434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c0d1970 msgr2=0x7f934c15d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.126+0000 7f9351a74640 1 --2- 172.21.15.45:0/3118689434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c0d1970 0x7f934c15d660 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f933c00c400 tx=0x7f933c00c8c0 comp rx=0 tx=0).stop 2024-05-08T05:49:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.126+0000 7f9351a74640 1 -- 172.21.15.45:0/3118689434 shutdown_connections 2024-05-08T05:49:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.127+0000 7f9351a74640 1 --2- 172.21.15.45:0/3118689434 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9338076300 0x7f93380787c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.127+0000 7f9351a74640 1 --2- 172.21.15.45:0/3118689434 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f934c0d4cc0 0x7f934c153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.127+0000 7f9351a74640 1 --2- 172.21.15.45:0/3118689434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f934c0b7c00 0x7f934c0cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.127+0000 7f9351a74640 1 --2- 172.21.15.45:0/3118689434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f934c0d1970 0x7f934c15d660 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:01.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.127+0000 7f9351a74640 1 -- 172.21.15.45:0/3118689434 >> 172.21.15.45:0/3118689434 conn(0x7f934c08eac0 msgr2=0x7f934c15e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.127+0000 7f9351a74640 1 -- 172.21.15.45:0/3118689434 shutdown_connections 2024-05-08T05:49:01.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.127+0000 7f9351a74640 1 -- 172.21.15.45:0/3118689434 wait complete. 2024-05-08T05:49:01.129 INFO:tasks.ceph.mds.c.smithi045.stderr: -2180> 2024-05-08T05:48:26.476+0000 7fdbea21d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:49:01.130 INFO:tasks.ceph.mds.c.smithi045.stderr: -1827> 2024-05-08T05:48:26.738+0000 7fdbe6215640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x5607b28f8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-05-08T05:49:01.139 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T05:49:01.158 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-05-08T05:49:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.246+0000 7efee1987640 1 Processor -- start 2024-05-08T05:49:01.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.247+0000 7efee1987640 1 -- start start 2024-05-08T05:49:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.247+0000 7efee1987640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efedc0b21f0 0x7efedc06e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:01.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.247+0000 7efee1987640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7efedc0cfea0 0x7efedc1633e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.247+0000 7efee1987640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efedc0d0500 con 0x7efedc162f50 2024-05-08T05:49:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.247+0000 7efee1987640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7efedc06d810 con 0x7efedc0b21f0 2024-05-08T05:49:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.247+0000 7efee1987640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7efedc06e0b0 con 0x7efedc0cfea0 2024-05-08T05:49:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.247+0000 7efedaffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efedc0b21f0 0x7efedc06e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.248+0000 7efedaffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efedc0b21f0 0x7efedc06e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59586/0 (socket says 172.21.15.45:59586) 2024-05-08T05:49:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.248+0000 7efedaffd640 1 -- 172.21.15.45:0/69955677 learned_addr learned my addr 172.21.15.45:0/69955677 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.248+0000 7efedb7fe640 1 --2- 172.21.15.45:0/69955677 >> v2:172.21.15.63:3300/0 conn(0x7efedc0cfea0 0x7efedc1633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.248+0000 7efedaffd640 1 -- 172.21.15.45:0/69955677 >> v2:172.21.15.63:3300/0 conn(0x7efedc0cfea0 msgr2=0x7efedc1633e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.248+0000 7efedaffd640 1 --2- 172.21.15.45:0/69955677 >> v2:172.21.15.63:3300/0 conn(0x7efedc0cfea0 0x7efedc1633e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.248+0000 7efedaffd640 1 -- 172.21.15.45:0/69955677 >> v1:172.21.15.63:6789/0 conn(0x7efedc162f50 legacy=0x7efedc0cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.248+0000 7efedaffd640 1 -- 172.21.15.45:0/69955677 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efedc06da60 con 0x7efedc0b21f0 2024-05-08T05:49:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.249+0000 7efedaffd640 1 --2- 172.21.15.45:0/69955677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efedc0b21f0 0x7efedc06e540 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7efedc0cf470 tx=0x7efecc02a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a916b0407cc2aec7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:01.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.249+0000 7efed9ffb640 1 -- 172.21.15.45:0/69955677 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc036070 con 0x7efedc0b21f0 2024-05-08T05:49:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.249+0000 7efed9ffb640 1 -- 172.21.15.45:0/69955677 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efecc031600 con 0x7efedc0b21f0 2024-05-08T05:49:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.249+0000 7efed9ffb640 1 -- 172.21.15.45:0/69955677 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc0319d0 con 0x7efedc0b21f0 2024-05-08T05:49:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.249+0000 7efee1987640 1 -- 172.21.15.45:0/69955677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efedc0b21f0 msgr2=0x7efedc06e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.249+0000 7efee1987640 1 --2- 172.21.15.45:0/69955677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efedc0b21f0 0x7efedc06e540 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7efedc0cf470 tx=0x7efecc02a570 comp rx=0 tx=0).stop 2024-05-08T05:49:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.249+0000 7efee1987640 1 -- 172.21.15.45:0/69955677 shutdown_connections 2024-05-08T05:49:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.249+0000 7efee1987640 1 --2- 172.21.15.45:0/69955677 >> v2:172.21.15.63:3300/0 conn(0x7efedc0cfea0 0x7efedc1633e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.249+0000 7efee1987640 1 --2- 172.21.15.45:0/69955677 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efedc0b21f0 0x7efedc06e540 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.249+0000 7efee1987640 1 -- 172.21.15.45:0/69955677 >> 172.21.15.45:0/69955677 conn(0x7efedc09ba40 msgr2=0x7efedc09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.249+0000 7efee1987640 1 -- 172.21.15.45:0/69955677 shutdown_connections 2024-05-08T05:49:01.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.249+0000 7efee1987640 1 -- 172.21.15.45:0/69955677 wait complete. 2024-05-08T05:49:01.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.251+0000 7efee1987640 1 Processor -- start 2024-05-08T05:49:01.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.251+0000 7efee1987640 1 -- start start 2024-05-08T05:49:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.251+0000 7efee1987640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efedc0b21f0 0x7efedc1f8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.251+0000 7efee1987640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efedc0cfea0 0x7efedc0b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.251+0000 7efee1987640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efedc162f50 0x7efedc0b7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.251+0000 7efee1987640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efedc0b25d0 con 0x7efedc0b21f0 2024-05-08T05:49:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.251+0000 7efee1987640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efedc06ea80 con 0x7efedc0cfea0 2024-05-08T05:49:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.251+0000 7efee1987640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efedc0cd480 con 0x7efedc162f50 2024-05-08T05:49:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.252+0000 7efedaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efedc0b21f0 0x7efedc1f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:01.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.252+0000 7efedaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efedc0b21f0 0x7efedc1f8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59588/0 (socket says 172.21.15.45:59588) 2024-05-08T05:49:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.252+0000 7efedaffd640 1 -- 172.21.15.45:0/2797963867 learned_addr learned my addr 172.21.15.45:0/2797963867 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.252+0000 7efeda7fc640 1 --2- 172.21.15.45:0/2797963867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efedc0cfea0 0x7efedc0b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.252+0000 7efedb7fe640 1 --2- 172.21.15.45:0/2797963867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efedc162f50 0x7efedc0b7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.252+0000 7efedaffd640 1 -- 172.21.15.45:0/2797963867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efedc162f50 msgr2=0x7efedc0b7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.252+0000 7efedaffd640 1 --2- 172.21.15.45:0/2797963867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efedc162f50 0x7efedc0b7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.252+0000 7efedaffd640 1 -- 172.21.15.45:0/2797963867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efedc0cfea0 msgr2=0x7efedc0b6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.252+0000 7efedaffd640 1 --2- 172.21.15.45:0/2797963867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efedc0cfea0 0x7efedc0b6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.252+0000 7efedaffd640 1 -- 172.21.15.45:0/2797963867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efedc0b2a00 con 0x7efedc0b21f0 2024-05-08T05:49:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.252+0000 7efedaffd640 1 --2- 172.21.15.45:0/2797963867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efedc0b21f0 0x7efedc1f8300 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7efecc009f40 tx=0x7efecc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.252+0000 7efed3fff640 1 -- 172.21.15.45:0/2797963867 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc041030 con 0x7efedc0b21f0 2024-05-08T05:49:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.252+0000 7efed3fff640 1 -- 172.21.15.45:0/2797963867 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efecc031400 con 0x7efedc0b21f0 2024-05-08T05:49:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.252+0000 7efee1987640 1 -- 172.21.15.45:0/2797963867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efedc0b7af0 con 0x7efedc0b21f0 2024-05-08T05:49:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.252+0000 7efed3fff640 1 -- 172.21.15.45:0/2797963867 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc036040 con 0x7efedc0b21f0 2024-05-08T05:49:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.252+0000 7efee1987640 1 -- 172.21.15.45:0/2797963867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efedc0b80b0 con 0x7efedc0b21f0 2024-05-08T05:49:01.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.253+0000 7efee1987640 1 -- 172.21.15.45:0/2797963867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efedc069b00 con 0x7efedc0b21f0 2024-05-08T05:49:01.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.254+0000 7efed3fff640 1 -- 172.21.15.45:0/2797963867 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efecc044070 con 0x7efedc0b21f0 2024-05-08T05:49:01.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.254+0000 7efed3fff640 1 --2- 172.21.15.45:0/2797963867 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efebc0762d0 0x7efebc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:01.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.255+0000 7efed3fff640 1 -- 172.21.15.45:0/2797963867 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6475+0+0 (secure 0 0 0) 0x7efecc0b8450 con 0x7efedc0b21f0 2024-05-08T05:49:01.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.255+0000 7efeda7fc640 1 --2- 172.21.15.45:0/2797963867 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efebc0762d0 0x7efebc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:01.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.255+0000 7efeda7fc640 1 --2- 172.21.15.45:0/2797963867 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efebc0762d0 0x7efebc078790 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efec4006fd0 tx=0x7efec4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:01.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.259+0000 7efed3fff640 1 -- 172.21.15.45:0/2797963867 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efecc04b070 con 0x7efedc0b21f0 2024-05-08T05:49:01.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:01.491+0000 7efee1987640 1 -- 172.21.15.45:0/2797963867 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7efedc070a30 con 0x7efedc0b21f0 2024-05-08T05:49:02.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.125+0000 7efed3fff640 1 -- 172.21.15.45:0/2797963867 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v75) ==== 104+0+0 (secure 0 0 0) 0x7efecc081980 con 0x7efedc0b21f0 2024-05-08T05:49:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.128+0000 7efee1987640 1 -- 172.21.15.45:0/2797963867 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efebc0762d0 msgr2=0x7efebc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.128+0000 7efee1987640 1 --2- 172.21.15.45:0/2797963867 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efebc0762d0 0x7efebc078790 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efec4006fd0 tx=0x7efec4008040 comp rx=0 tx=0).stop 2024-05-08T05:49:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.128+0000 7efee1987640 1 -- 172.21.15.45:0/2797963867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efedc0b21f0 msgr2=0x7efedc1f8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.128+0000 7efee1987640 1 --2- 172.21.15.45:0/2797963867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efedc0b21f0 0x7efedc1f8300 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7efecc009f40 tx=0x7efecc004a40 comp rx=0 tx=0).stop 2024-05-08T05:49:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.128+0000 7efee1987640 1 -- 172.21.15.45:0/2797963867 shutdown_connections 2024-05-08T05:49:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.128+0000 7efee1987640 1 --2- 172.21.15.45:0/2797963867 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efebc0762d0 0x7efebc078790 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.128+0000 7efee1987640 1 --2- 172.21.15.45:0/2797963867 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efedc162f50 0x7efedc0b7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.128+0000 7efee1987640 1 --2- 172.21.15.45:0/2797963867 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efedc0cfea0 0x7efedc0b6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.128+0000 7efee1987640 1 --2- 172.21.15.45:0/2797963867 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efedc0b21f0 0x7efedc1f8300 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.128+0000 7efee1987640 1 -- 172.21.15.45:0/2797963867 >> 172.21.15.45:0/2797963867 conn(0x7efedc09ba40 msgr2=0x7efedc160b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.128+0000 7efee1987640 1 -- 172.21.15.45:0/2797963867 shutdown_connections 2024-05-08T05:49:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.128+0000 7efee1987640 1 -- 172.21.15.45:0/2797963867 wait complete. 2024-05-08T05:49:02.142 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T05:49:02.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.298+0000 7f1b5d693640 1 Processor -- start 2024-05-08T05:49:02.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.298+0000 7f1b5d693640 1 -- start start 2024-05-08T05:49:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.299+0000 7f1b5d693640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1b580cfff0 0x7f1b580b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.299+0000 7f1b5d693640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b58162e60 con 0x7f1b580d35f0 2024-05-08T05:49:02.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.299+0000 7f1b5d693640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b580d04a0 con 0x7f1b580cfbb0 2024-05-08T05:49:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.299+0000 7f1b5d693640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f1b580d0770 con 0x7f1b580cfff0 2024-05-08T05:49:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.299+0000 7f1b577fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f1b580cfff0 0x7f1b580b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.299+0000 7f1b56ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1b580d35f0 0x7f1b580d39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51134/0 (socket says 172.21.15.45:51134) 2024-05-08T05:49:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.299+0000 7f1b56ffd640 1 -- 172.21.15.45:0/476484083 learned_addr learned my addr 172.21.15.45:0/476484083 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.299+0000 7f1b577fe640 1 -- 172.21.15.45:0/476484083 >> v1:172.21.15.63:6789/0 conn(0x7f1b580cfbb0 legacy=0x7f1b580b8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.299+0000 7f1b577fe640 1 -- 172.21.15.45:0/476484083 >> v1:172.21.15.45:6789/0 conn(0x7f1b580d35f0 legacy=0x7f1b580d39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:02.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.299+0000 7f1b577fe640 1 -- 172.21.15.45:0/476484083 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b580d0a50 con 0x7f1b580cfff0 2024-05-08T05:49:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.300+0000 7f1b577fe640 1 --2- 172.21.15.45:0/476484083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b580cfff0 0x7f1b580b8c30 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f1b500097a0 tx=0x7f1b5002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8dce25cf7dca40 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.300+0000 7f1b55ffb640 1 -- 172.21.15.45:0/476484083 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50037070 con 0x7f1b580cfff0 2024-05-08T05:49:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.300+0000 7f1b55ffb640 1 -- 172.21.15.45:0/476484083 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b50032640 con 0x7f1b580cfff0 2024-05-08T05:49:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.300+0000 7f1b55ffb640 1 -- 172.21.15.45:0/476484083 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50032a10 con 0x7f1b580cfff0 2024-05-08T05:49:02.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.301+0000 7f1b5d693640 1 -- 172.21.15.45:0/476484083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b580cfff0 msgr2=0x7f1b580b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.301+0000 7f1b5d693640 1 --2- 172.21.15.45:0/476484083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b580cfff0 0x7f1b580b8c30 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f1b500097a0 tx=0x7f1b5002a2c0 comp rx=0 tx=0).stop 2024-05-08T05:49:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.301+0000 7f1b5d693640 1 -- 172.21.15.45:0/476484083 shutdown_connections 2024-05-08T05:49:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.301+0000 7f1b5d693640 1 --2- 172.21.15.45:0/476484083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b580cfff0 0x7f1b580b8c30 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.301+0000 7f1b5d693640 1 -- 172.21.15.45:0/476484083 >> 172.21.15.45:0/476484083 conn(0x7f1b5809e3e0 msgr2=0x7f1b580d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.301+0000 7f1b5d693640 1 -- 172.21.15.45:0/476484083 shutdown_connections 2024-05-08T05:49:02.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.301+0000 7f1b5d693640 1 -- 172.21.15.45:0/476484083 wait complete. 2024-05-08T05:49:02.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.302+0000 7f1b5d693640 1 Processor -- start 2024-05-08T05:49:02.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.302+0000 7f1b5d693640 1 -- start start 2024-05-08T05:49:02.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.303+0000 7f1b5d693640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b580cfbb0 0x7f1b58158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.303+0000 7f1b5d693640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b580cfff0 0x7f1b58154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.303+0000 7f1b5d693640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b580d35f0 0x7f1b5815f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.303+0000 7f1b5d693640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1b580b6230 con 0x7f1b580cfbb0 2024-05-08T05:49:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.303+0000 7f1b5d693640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1b580b8000 con 0x7f1b580cfff0 2024-05-08T05:49:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.303+0000 7f1b5d693640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1b580b9170 con 0x7f1b580d35f0 2024-05-08T05:49:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.303+0000 7f1b56ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b580cfbb0 0x7f1b58158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.303+0000 7f1b56ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b580cfbb0 0x7f1b58158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59600/0 (socket says 172.21.15.45:59600) 2024-05-08T05:49:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.303+0000 7f1b56ffd640 1 -- 172.21.15.45:0/1025521041 learned_addr learned my addr 172.21.15.45:0/1025521041 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.303+0000 7f1b567fc640 1 --2- 172.21.15.45:0/1025521041 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b580cfff0 0x7f1b58154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.303+0000 7f1b577fe640 1 --2- 172.21.15.45:0/1025521041 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b580d35f0 0x7f1b5815f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.303+0000 7f1b56ffd640 1 -- 172.21.15.45:0/1025521041 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b580d35f0 msgr2=0x7f1b5815f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.303+0000 7f1b56ffd640 1 --2- 172.21.15.45:0/1025521041 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b580d35f0 0x7f1b5815f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.303+0000 7f1b56ffd640 1 -- 172.21.15.45:0/1025521041 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b580cfff0 msgr2=0x7f1b58154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:02.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.303+0000 7f1b56ffd640 1 --2- 172.21.15.45:0/1025521041 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b580cfff0 0x7f1b58154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.303+0000 7f1b56ffd640 1 -- 172.21.15.45:0/1025521041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5815e5b0 con 0x7f1b580cfbb0 2024-05-08T05:49:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.303+0000 7f1b567fc640 1 --2- 172.21.15.45:0/1025521041 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b580cfff0 0x7f1b58154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:49:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.303+0000 7f1b577fe640 1 --2- 172.21.15.45:0/1025521041 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b580d35f0 0x7f1b5815f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:49:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.304+0000 7f1b56ffd640 1 --2- 172.21.15.45:0/1025521041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b580cfbb0 0x7f1b58158a70 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800c400 tx=0x7f1b4800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.304+0000 7f1b3f7fe640 1 -- 172.21.15.45:0/1025521041 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48017070 con 0x7f1b580cfbb0 2024-05-08T05:49:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.304+0000 7f1b3f7fe640 1 -- 172.21.15.45:0/1025521041 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b48005ce0 con 0x7f1b580cfbb0 2024-05-08T05:49:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.304+0000 7f1b3f7fe640 1 -- 172.21.15.45:0/1025521041 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b480072d0 con 0x7f1b580cfbb0 2024-05-08T05:49:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.304+0000 7f1b5d693640 1 -- 172.21.15.45:0/1025521041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5815e890 con 0x7f1b580cfbb0 2024-05-08T05:49:02.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.304+0000 7f1b5d693640 1 -- 172.21.15.45:0/1025521041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b5815ee50 con 0x7f1b580cfbb0 2024-05-08T05:49:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.305+0000 7f1b5d693640 1 -- 172.21.15.45:0/1025521041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b580693f0 con 0x7f1b580cfbb0 2024-05-08T05:49:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.306+0000 7f1b3f7fe640 1 -- 172.21.15.45:0/1025521041 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1b4801a070 con 0x7f1b580cfbb0 2024-05-08T05:49:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.306+0000 7f1b3f7fe640 1 --2- 172.21.15.45:0/1025521041 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1b440762d0 0x7f1b44078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.306+0000 7f1b3f7fe640 1 -- 172.21.15.45:0/1025521041 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6475+0+0 (secure 0 0 0) 0x7f1b48096b60 con 0x7f1b580cfbb0 2024-05-08T05:49:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.306+0000 7f1b567fc640 1 --2- 172.21.15.45:0/1025521041 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1b440762d0 0x7f1b44078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.307+0000 7f1b567fc640 1 --2- 172.21.15.45:0/1025521041 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1b440762d0 0x7f1b44078790 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1b58159270 tx=0x7f1b4c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.310+0000 7f1b3f7fe640 1 -- 172.21.15.45:0/1025521041 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1b4805fc00 con 0x7f1b580cfbb0 2024-05-08T05:49:02.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:02.502+0000 7f1b5d693640 1 -- 172.21.15.45:0/1025521041 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1b580d33d0 con 0x7f1b580cfbb0 2024-05-08T05:49:03.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.132+0000 7f1b3f7fe640 1 -- 172.21.15.45:0/1025521041 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) ==== 181+0+0 (secure 0 0 0) 0x7f1b480960d0 con 0x7f1b580cfbb0 2024-05-08T05:49:03.132 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-05-08T05:49:03.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.135+0000 7f1b5d693640 1 -- 172.21.15.45:0/1025521041 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1b440762d0 msgr2=0x7f1b44078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:03.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.135+0000 7f1b5d693640 1 --2- 172.21.15.45:0/1025521041 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1b440762d0 0x7f1b44078790 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1b58159270 tx=0x7f1b4c009040 comp rx=0 tx=0).stop 2024-05-08T05:49:03.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.135+0000 7f1b5d693640 1 -- 172.21.15.45:0/1025521041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b580cfbb0 msgr2=0x7f1b58158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:03.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.135+0000 7f1b5d693640 1 --2- 172.21.15.45:0/1025521041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b580cfbb0 0x7f1b58158a70 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800c400 tx=0x7f1b4800c8c0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.136+0000 7f1b5d693640 1 -- 172.21.15.45:0/1025521041 shutdown_connections 2024-05-08T05:49:03.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.136+0000 7f1b5d693640 1 --2- 172.21.15.45:0/1025521041 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1b440762d0 0x7f1b44078790 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.136+0000 7f1b5d693640 1 --2- 172.21.15.45:0/1025521041 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1b580d35f0 0x7f1b5815f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.136+0000 7f1b5d693640 1 --2- 172.21.15.45:0/1025521041 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b580cfff0 0x7f1b58154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.136+0000 7f1b5d693640 1 --2- 172.21.15.45:0/1025521041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b580cfbb0 0x7f1b58158a70 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.136+0000 7f1b5d693640 1 -- 172.21.15.45:0/1025521041 >> 172.21.15.45:0/1025521041 conn(0x7f1b5809e3e0 msgr2=0x7f1b58161460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:03.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.136+0000 7f1b5d693640 1 -- 172.21.15.45:0/1025521041 shutdown_connections 2024-05-08T05:49:03.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.136+0000 7f1b5d693640 1 -- 172.21.15.45:0/1025521041 wait complete. 2024-05-08T05:49:03.150 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T05:49:03.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.262+0000 7f8677e07640 1 Processor -- start 2024-05-08T05:49:03.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.262+0000 7f8677e07640 1 -- start start 2024-05-08T05:49:03.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.263+0000 7f8677e07640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f86700d3670 0x7f86700b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.263+0000 7f8677e07640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86700d1050 0x7f8670162920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.263+0000 7f8677e07640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86700cfc00 con 0x7f86700d3a50 2024-05-08T05:49:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.263+0000 7f8677e07640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f86700d2a10 con 0x7f86700d3670 2024-05-08T05:49:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.263+0000 7f8677e07640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f86700d04b0 con 0x7f86700d1050 2024-05-08T05:49:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.263+0000 7f8675ba3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f86700d3670 0x7f86700b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.263+0000 7f86763a4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86700d1050 0x7f8670162920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.263+0000 7f86753a2640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f86700d3a50 0x7f86700d0940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:39644/0 (socket says 172.21.15.45:39644) 2024-05-08T05:49:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.263+0000 7f86753a2640 1 -- 172.21.15.45:0/1362099878 learned_addr learned my addr 172.21.15.45:0/1362099878 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.264+0000 7f86763a4640 1 -- 172.21.15.45:0/1362099878 >> v2:172.21.15.63:3300/0 conn(0x7f86700d3670 msgr2=0x7f86700b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.264+0000 7f86763a4640 1 --2- 172.21.15.45:0/1362099878 >> v2:172.21.15.63:3300/0 conn(0x7f86700d3670 0x7f86700b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.264+0000 7f86763a4640 1 -- 172.21.15.45:0/1362099878 >> v1:172.21.15.63:6789/0 conn(0x7f86700d3a50 legacy=0x7f86700d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:03.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.264+0000 7f86763a4640 1 -- 172.21.15.45:0/1362099878 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86700cfea0 con 0x7f86700d1050 2024-05-08T05:49:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.264+0000 7f86763a4640 1 --2- 172.21.15.45:0/1362099878 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86700d1050 0x7f8670162920 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f86700d0630 tx=0x7f866c02a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2aaa493ef8b51846 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.265+0000 7f8674ba1640 1 -- 172.21.15.45:0/1362099878 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c02ae60 con 0x7f86700d1050 2024-05-08T05:49:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.265+0000 7f8674ba1640 1 -- 172.21.15.45:0/1362099878 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f866c007740 con 0x7f86700d1050 2024-05-08T05:49:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.265+0000 7f8674ba1640 1 -- 172.21.15.45:0/1362099878 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c007b10 con 0x7f86700d1050 2024-05-08T05:49:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.265+0000 7f8677e07640 1 -- 172.21.15.45:0/1362099878 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86700d1050 msgr2=0x7f8670162920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.265+0000 7f8677e07640 1 --2- 172.21.15.45:0/1362099878 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86700d1050 0x7f8670162920 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f86700d0630 tx=0x7f866c02a640 comp rx=0 tx=0).stop 2024-05-08T05:49:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.265+0000 7f8677e07640 1 -- 172.21.15.45:0/1362099878 shutdown_connections 2024-05-08T05:49:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.265+0000 7f8677e07640 1 --2- 172.21.15.45:0/1362099878 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86700d1050 0x7f8670162920 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.265+0000 7f8677e07640 1 --2- 172.21.15.45:0/1362099878 >> v2:172.21.15.63:3300/0 conn(0x7f86700d3670 0x7f86700b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.265+0000 7f8677e07640 1 -- 172.21.15.45:0/1362099878 >> 172.21.15.45:0/1362099878 conn(0x7f867009e390 msgr2=0x7f86700d5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.265+0000 7f8677e07640 1 -- 172.21.15.45:0/1362099878 shutdown_connections 2024-05-08T05:49:03.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.265+0000 7f8677e07640 1 -- 172.21.15.45:0/1362099878 wait complete. 2024-05-08T05:49:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.266+0000 7f8677e07640 1 Processor -- start 2024-05-08T05:49:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.266+0000 7f8677e07640 1 -- start start 2024-05-08T05:49:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.267+0000 7f8677e07640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86700d1050 0x7f86700ba1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.267+0000 7f8677e07640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86700d3670 0x7f86700b9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.267+0000 7f8677e07640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86700d3a50 0x7f86700b9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.267+0000 7f8677e07640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8670162e60 con 0x7f86700d1050 2024-05-08T05:49:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.267+0000 7f8677e07640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86700d0140 con 0x7f86700d3a50 2024-05-08T05:49:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.267+0000 7f8677e07640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f86700b4f00 con 0x7f86700d3670 2024-05-08T05:49:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.267+0000 7f8675ba3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86700d1050 0x7f86700ba1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:03.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.267+0000 7f8675ba3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86700d1050 0x7f86700ba1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59614/0 (socket says 172.21.15.45:59614) 2024-05-08T05:49:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.267+0000 7f8675ba3640 1 -- 172.21.15.45:0/762649331 learned_addr learned my addr 172.21.15.45:0/762649331 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.268+0000 7f86763a4640 1 --2- 172.21.15.45:0/762649331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86700d3a50 0x7f86700b9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.268+0000 7f86753a2640 1 --2- 172.21.15.45:0/762649331 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86700d3670 0x7f86700b9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.268+0000 7f8675ba3640 1 -- 172.21.15.45:0/762649331 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86700d3670 msgr2=0x7f86700b9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.268+0000 7f8675ba3640 1 --2- 172.21.15.45:0/762649331 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86700d3670 0x7f86700b9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.268+0000 7f8675ba3640 1 -- 172.21.15.45:0/762649331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86700d3a50 msgr2=0x7f86700b9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.268+0000 7f8675ba3640 1 --2- 172.21.15.45:0/762649331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86700d3a50 0x7f86700b9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.268+0000 7f8675ba3640 1 -- 172.21.15.45:0/762649331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86700b7ec0 con 0x7f86700d1050 2024-05-08T05:49:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.268+0000 7f86763a4640 1 --2- 172.21.15.45:0/762649331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86700d3a50 0x7f86700b9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:49:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.268+0000 7f86753a2640 1 --2- 172.21.15.45:0/762649331 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86700d3670 0x7f86700b9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:49:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.268+0000 7f8675ba3640 1 --2- 172.21.15.45:0/762649331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86700d1050 0x7f86700ba1c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f866800e9b0 tx=0x7f866800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:03.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.268+0000 7f86667fc640 1 -- 172.21.15.45:0/762649331 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668017070 con 0x7f86700d1050 2024-05-08T05:49:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.268+0000 7f86667fc640 1 -- 172.21.15.45:0/762649331 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f866800c9a0 con 0x7f86700d1050 2024-05-08T05:49:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.268+0000 7f86667fc640 1 -- 172.21.15.45:0/762649331 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866800cd90 con 0x7f86700d1050 2024-05-08T05:49:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.268+0000 7f8677e07640 1 -- 172.21.15.45:0/762649331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86700ce380 con 0x7f86700d1050 2024-05-08T05:49:03.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.268+0000 7f8677e07640 1 -- 172.21.15.45:0/762649331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86700ce940 con 0x7f86700d1050 2024-05-08T05:49:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.269+0000 7f8677e07640 1 -- 172.21.15.45:0/762649331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86700693f0 con 0x7f86700d1050 2024-05-08T05:49:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.270+0000 7f86667fc640 1 -- 172.21.15.45:0/762649331 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f866801a070 con 0x7f86700d1050 2024-05-08T05:49:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.270+0000 7f86667fc640 1 --2- 172.21.15.45:0/762649331 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f86540762d0 0x7f8654078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.270+0000 7f86667fc640 1 -- 172.21.15.45:0/762649331 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6037+0+0 (secure 0 0 0) 0x7f8668097720 con 0x7f86700d1050 2024-05-08T05:49:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.271+0000 7f86753a2640 1 --2- 172.21.15.45:0/762649331 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f86540762d0 0x7f8654078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:03.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.271+0000 7f86753a2640 1 --2- 172.21.15.45:0/762649331 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f86540762d0 0x7f8654078790 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f86700c84f0 tx=0x7f8660009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:03.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.275+0000 7f86667fc640 1 -- 172.21.15.45:0/762649331 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8668060f40 con 0x7f86700d1050 2024-05-08T05:49:03.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:03.465+0000 7f8677e07640 1 -- 172.21.15.45:0/762649331 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f86700b9fa0 con 0x7f86700d1050 2024-05-08T05:49:04.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.133+0000 7f86667fc640 1 -- 172.21.15.45:0/762649331 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) ==== 169+0+0 (secure 0 0 0) 0x7f8668060840 con 0x7f86700d1050 2024-05-08T05:49:04.134 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-05-08T05:49:04.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.137+0000 7f8677e07640 1 -- 172.21.15.45:0/762649331 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f86540762d0 msgr2=0x7f8654078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:04.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.137+0000 7f8677e07640 1 --2- 172.21.15.45:0/762649331 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f86540762d0 0x7f8654078790 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f86700c84f0 tx=0x7f8660009040 comp rx=0 tx=0).stop 2024-05-08T05:49:04.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.137+0000 7f8677e07640 1 -- 172.21.15.45:0/762649331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86700d1050 msgr2=0x7f86700ba1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:04.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.137+0000 7f8677e07640 1 --2- 172.21.15.45:0/762649331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86700d1050 0x7f86700ba1c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f866800e9b0 tx=0x7f866800ee70 comp rx=0 tx=0).stop 2024-05-08T05:49:04.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.138+0000 7f8677e07640 1 -- 172.21.15.45:0/762649331 shutdown_connections 2024-05-08T05:49:04.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.138+0000 7f8677e07640 1 --2- 172.21.15.45:0/762649331 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f86540762d0 0x7f8654078790 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.138+0000 7f8677e07640 1 --2- 172.21.15.45:0/762649331 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86700d3670 0x7f86700b9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.138+0000 7f8677e07640 1 --2- 172.21.15.45:0/762649331 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86700d3a50 0x7f86700b9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.138+0000 7f8677e07640 1 --2- 172.21.15.45:0/762649331 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86700d1050 0x7f86700ba1c0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.138+0000 7f8677e07640 1 -- 172.21.15.45:0/762649331 >> 172.21.15.45:0/762649331 conn(0x7f867009e390 msgr2=0x7f867006e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:04.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.138+0000 7f8677e07640 1 -- 172.21.15.45:0/762649331 shutdown_connections 2024-05-08T05:49:04.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.138+0000 7f8677e07640 1 -- 172.21.15.45:0/762649331 wait complete. 2024-05-08T05:49:04.154 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T05:49:04.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.264+0000 7f4c987af640 1 Processor -- start 2024-05-08T05:49:04.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.264+0000 7f4c987af640 1 -- start start 2024-05-08T05:49:04.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.265+0000 7f4c987af640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4c9015f420 0x7f4c900d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:04.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.265+0000 7f4c987af640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c900d3ca0 0x7f4c900d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:04.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.265+0000 7f4c987af640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4c900d1eb0 0x7f4c900d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:04.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.265+0000 7f4c987af640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4c900d22f0 con 0x7f4c900d1eb0 2024-05-08T05:49:04.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.265+0000 7f4c987af640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4c901630b0 con 0x7f4c9015f420 2024-05-08T05:49:04.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.265+0000 7f4c987af640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4c900d2550 con 0x7f4c900d3ca0 2024-05-08T05:49:04.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.266+0000 7f4c96d4c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4c900d1eb0 0x7f4c900d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:04.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.266+0000 7f4c96d4c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4c900d1eb0 0x7f4c900d4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59620/0 (socket says 172.21.15.45:59620) 2024-05-08T05:49:04.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.266+0000 7f4c96d4c640 1 -- 172.21.15.45:0/791353447 learned_addr learned my addr 172.21.15.45:0/791353447 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:04.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.266+0000 7f4c96d4c640 1 -- 172.21.15.45:0/791353447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c900d3ca0 msgr2=0x7f4c900d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:04.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.266+0000 7f4c96d4c640 1 --2- 172.21.15.45:0/791353447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c900d3ca0 0x7f4c900d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.266+0000 7f4c96d4c640 1 -- 172.21.15.45:0/791353447 >> v2:172.21.15.63:3300/0 conn(0x7f4c9015f420 msgr2=0x7f4c900d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:04.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.266+0000 7f4c9654b640 1 --2- 172.21.15.45:0/791353447 >> v2:172.21.15.63:3300/0 conn(0x7f4c9015f420 0x7f4c900d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:04.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.266+0000 7f4c96d4c640 1 --2- 172.21.15.45:0/791353447 >> v2:172.21.15.63:3300/0 conn(0x7f4c9015f420 0x7f4c900d3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.266+0000 7f4c96d4c640 1 -- 172.21.15.45:0/791353447 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c900d2c80 con 0x7f4c900d1eb0 2024-05-08T05:49:04.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.267+0000 7f4c96d4c640 1 --2- 172.21.15.45:0/791353447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c900d1eb0 0x7f4c900d4cc0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f4c900d5ec0 tx=0x7f4c8002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e7382e26993332 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:04.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.267+0000 7f4c95549640 1 -- 172.21.15.45:0/791353447 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80036070 con 0x7f4c900d1eb0 2024-05-08T05:49:04.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.267+0000 7f4c95549640 1 -- 172.21.15.45:0/791353447 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c80031600 con 0x7f4c900d1eb0 2024-05-08T05:49:04.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.267+0000 7f4c95549640 1 -- 172.21.15.45:0/791353447 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c800319d0 con 0x7f4c900d1eb0 2024-05-08T05:49:04.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.267+0000 7f4c987af640 1 -- 172.21.15.45:0/791353447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c900d1eb0 msgr2=0x7f4c900d4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:04.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.267+0000 7f4c987af640 1 --2- 172.21.15.45:0/791353447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c900d1eb0 0x7f4c900d4cc0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f4c900d5ec0 tx=0x7f4c8002a6a0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.267+0000 7f4c987af640 1 -- 172.21.15.45:0/791353447 shutdown_connections 2024-05-08T05:49:04.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.267+0000 7f4c987af640 1 --2- 172.21.15.45:0/791353447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c900d1eb0 0x7f4c900d4cc0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.267+0000 7f4c987af640 1 --2- 172.21.15.45:0/791353447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c900d3ca0 0x7f4c900d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.267+0000 7f4c987af640 1 --2- 172.21.15.45:0/791353447 >> v2:172.21.15.63:3300/0 conn(0x7f4c9015f420 0x7f4c900d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.267+0000 7f4c987af640 1 -- 172.21.15.45:0/791353447 >> 172.21.15.45:0/791353447 conn(0x7f4c9008eac0 msgr2=0x7f4c9009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:04.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.267+0000 7f4c987af640 1 -- 172.21.15.45:0/791353447 shutdown_connections 2024-05-08T05:49:04.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.267+0000 7f4c987af640 1 -- 172.21.15.45:0/791353447 wait complete. 2024-05-08T05:49:04.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.269+0000 7f4c987af640 1 Processor -- start 2024-05-08T05:49:04.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.269+0000 7f4c987af640 1 -- start start 2024-05-08T05:49:04.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.269+0000 7f4c987af640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c900d1eb0 0x7f4c9015cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:04.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.269+0000 7f4c987af640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c9015b530 0x7f4c900b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:04.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.269+0000 7f4c987af640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c900b8170 0x7f4c900c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:04.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.269+0000 7f4c987af640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c900d2830 con 0x7f4c9015b530 2024-05-08T05:49:04.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.269+0000 7f4c987af640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4c900d3240 con 0x7f4c900b8170 2024-05-08T05:49:04.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.269+0000 7f4c987af640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4c900d5200 con 0x7f4c900d1eb0 2024-05-08T05:49:04.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.269+0000 7f4c95d4a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c9015b530 0x7f4c900b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:04.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.269+0000 7f4c95d4a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c9015b530 0x7f4c900b7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59636/0 (socket says 172.21.15.45:59636) 2024-05-08T05:49:04.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.269+0000 7f4c95d4a640 1 -- 172.21.15.45:0/1577395308 learned_addr learned my addr 172.21.15.45:0/1577395308 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:04.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.270+0000 7f4c96d4c640 1 --2- 172.21.15.45:0/1577395308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c900b8170 0x7f4c900c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:04.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.270+0000 7f4c9654b640 1 --2- 172.21.15.45:0/1577395308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c900d1eb0 0x7f4c9015cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:04.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.270+0000 7f4c95d4a640 1 -- 172.21.15.45:0/1577395308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c900d1eb0 msgr2=0x7f4c9015cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:04.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.270+0000 7f4c95d4a640 1 --2- 172.21.15.45:0/1577395308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c900d1eb0 0x7f4c9015cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.270+0000 7f4c95d4a640 1 -- 172.21.15.45:0/1577395308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c900b8170 msgr2=0x7f4c900c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:04.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.270+0000 7f4c95d4a640 1 --2- 172.21.15.45:0/1577395308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c900b8170 0x7f4c900c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.270+0000 7f4c95d4a640 1 -- 172.21.15.45:0/1577395308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9015ba30 con 0x7f4c9015b530 2024-05-08T05:49:04.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.270+0000 7f4c96d4c640 1 --2- 172.21.15.45:0/1577395308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c900b8170 0x7f4c900c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:49:04.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.270+0000 7f4c95d4a640 1 --2- 172.21.15.45:0/1577395308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c9015b530 0x7f4c900b7c30 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400b4a0 tx=0x7f4c8400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:04.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.270+0000 7f4c8effd640 1 -- 172.21.15.45:0/1577395308 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84014070 con 0x7f4c9015b530 2024-05-08T05:49:04.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.270+0000 7f4c8effd640 1 -- 172.21.15.45:0/1577395308 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c84010610 con 0x7f4c9015b530 2024-05-08T05:49:04.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.270+0000 7f4c987af640 1 -- 172.21.15.45:0/1577395308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c9015bd10 con 0x7f4c9015b530 2024-05-08T05:49:04.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.270+0000 7f4c8effd640 1 -- 172.21.15.45:0/1577395308 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84010a00 con 0x7f4c9015b530 2024-05-08T05:49:04.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.270+0000 7f4c987af640 1 -- 172.21.15.45:0/1577395308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c9015c2d0 con 0x7f4c9015b530 2024-05-08T05:49:04.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.271+0000 7f4c987af640 1 -- 172.21.15.45:0/1577395308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c900693f0 con 0x7f4c9015b530 2024-05-08T05:49:04.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.272+0000 7f4c8effd640 1 -- 172.21.15.45:0/1577395308 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4c84017070 con 0x7f4c9015b530 2024-05-08T05:49:04.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.272+0000 7f4c8effd640 1 --2- 172.21.15.45:0/1577395308 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4c7c076300 0x7f4c7c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:04.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.272+0000 7f4c8effd640 1 -- 172.21.15.45:0/1577395308 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5507+0+0 (secure 0 0 0) 0x7f4c84096f40 con 0x7f4c9015b530 2024-05-08T05:49:04.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.273+0000 7f4c9654b640 1 --2- 172.21.15.45:0/1577395308 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4c7c076300 0x7f4c7c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:04.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.273+0000 7f4c9654b640 1 --2- 172.21.15.45:0/1577395308 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4c7c076300 0x7f4c7c0787c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4c88006fd0 tx=0x7f4c88008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:04.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.277+0000 7f4c8effd640 1 -- 172.21.15.45:0/1577395308 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4c84060960 con 0x7f4c9015b530 2024-05-08T05:49:04.466 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.466+0000 7f4c987af640 1 -- 172.21.15.45:0/1577395308 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f4c900b4c50 con 0x7f4c9015b530 2024-05-08T05:49:04.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.469+0000 7f4c8effd640 1 -- 172.21.15.45:0/1577395308 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f4c84060260 con 0x7f4c9015b530 2024-05-08T05:49:04.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.471+0000 7f4c987af640 1 -- 172.21.15.45:0/1577395308 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4c7c076300 msgr2=0x7f4c7c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:04.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.471+0000 7f4c987af640 1 --2- 172.21.15.45:0/1577395308 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4c7c076300 0x7f4c7c0787c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4c88006fd0 tx=0x7f4c88008040 comp rx=0 tx=0).stop 2024-05-08T05:49:04.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.471+0000 7f4c987af640 1 -- 172.21.15.45:0/1577395308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c9015b530 msgr2=0x7f4c900b7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:04.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.471+0000 7f4c987af640 1 --2- 172.21.15.45:0/1577395308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c9015b530 0x7f4c900b7c30 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400b4a0 tx=0x7f4c8400b960 comp rx=0 tx=0).stop 2024-05-08T05:49:04.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.471+0000 7f4c987af640 1 -- 172.21.15.45:0/1577395308 shutdown_connections 2024-05-08T05:49:04.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.471+0000 7f4c987af640 1 --2- 172.21.15.45:0/1577395308 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4c7c076300 0x7f4c7c0787c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.471+0000 7f4c987af640 1 --2- 172.21.15.45:0/1577395308 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c900b8170 0x7f4c900c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.471+0000 7f4c987af640 1 --2- 172.21.15.45:0/1577395308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c9015b530 0x7f4c900b7c30 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.471+0000 7f4c987af640 1 --2- 172.21.15.45:0/1577395308 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c900d1eb0 0x7f4c9015cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.471+0000 7f4c987af640 1 -- 172.21.15.45:0/1577395308 >> 172.21.15.45:0/1577395308 conn(0x7f4c9008eac0 msgr2=0x7f4c9006e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:04.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.471+0000 7f4c987af640 1 -- 172.21.15.45:0/1577395308 shutdown_connections 2024-05-08T05:49:04.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.471+0000 7f4c987af640 1 -- 172.21.15.45:0/1577395308 wait complete. 2024-05-08T05:49:04.482 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-05-08T05:49:04.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.586+0000 7f7fc1b89640 1 Processor -- start 2024-05-08T05:49:04.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.586+0000 7f7fc1b89640 1 -- start start 2024-05-08T05:49:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.587+0000 7f7fc1b89640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7fbc15ce90 0x7f7fbc070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.587+0000 7f7fc1b89640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fbc0733e0 0x7f7fbc0d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.587+0000 7f7fc1b89640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7fbc06d3b0 0x7f7fbc06d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.587+0000 7f7fc1b89640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f7fbc06dd60 con 0x7f7fbc06d3b0 2024-05-08T05:49:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.587+0000 7f7fc1b89640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7fbc071280 con 0x7f7fbc15ce90 2024-05-08T05:49:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.587+0000 7f7fc1b89640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7fbc0cff00 con 0x7f7fbc0733e0 2024-05-08T05:49:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.587+0000 7f7fbbfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7fbc06d3b0 0x7f7fbc06d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:04.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.587+0000 7f7fbbfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7fbc06d3b0 0x7f7fbc06d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59642/0 (socket says 172.21.15.45:59642) 2024-05-08T05:49:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.587+0000 7f7fbbfff640 1 -- 172.21.15.45:0/1208493356 learned_addr learned my addr 172.21.15.45:0/1208493356 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.588+0000 7f7fbb7fe640 1 --2- 172.21.15.45:0/1208493356 >> v2:172.21.15.63:3300/0 conn(0x7f7fbc15ce90 0x7f7fbc070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.588+0000 7f7fbaffd640 1 --2- 172.21.15.45:0/1208493356 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fbc0733e0 0x7f7fbc0d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.588+0000 7f7fbbfff640 1 -- 172.21.15.45:0/1208493356 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fbc0733e0 msgr2=0x7f7fbc0d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.588+0000 7f7fbbfff640 1 --2- 172.21.15.45:0/1208493356 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fbc0733e0 0x7f7fbc0d0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.588+0000 7f7fbbfff640 1 -- 172.21.15.45:0/1208493356 >> v2:172.21.15.63:3300/0 conn(0x7f7fbc15ce90 msgr2=0x7f7fbc070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.588+0000 7f7fbbfff640 1 --2- 172.21.15.45:0/1208493356 >> v2:172.21.15.63:3300/0 conn(0x7f7fbc15ce90 0x7f7fbc070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.588+0000 7f7fbbfff640 1 -- 172.21.15.45:0/1208493356 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc073950 con 0x7f7fbc06d3b0 2024-05-08T05:49:04.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.588+0000 7f7fbbfff640 1 --2- 172.21.15.45:0/1208493356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fbc06d3b0 0x7f7fbc06d820 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0742d0 tx=0x7f7fb402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c396d58c38e3b795 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:04.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.589+0000 7f7fba7fc640 1 -- 172.21.15.45:0/1208493356 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4036070 con 0x7f7fbc06d3b0 2024-05-08T05:49:04.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.589+0000 7f7fba7fc640 1 -- 172.21.15.45:0/1208493356 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb4031600 con 0x7f7fbc06d3b0 2024-05-08T05:49:04.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.589+0000 7f7fba7fc640 1 -- 172.21.15.45:0/1208493356 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb40319d0 con 0x7f7fbc06d3b0 2024-05-08T05:49:04.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.589+0000 7f7fc1b89640 1 -- 172.21.15.45:0/1208493356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fbc06d3b0 msgr2=0x7f7fbc06d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:04.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.589+0000 7f7fc1b89640 1 --2- 172.21.15.45:0/1208493356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fbc06d3b0 0x7f7fbc06d820 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0742d0 tx=0x7f7fb402a6a0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.589+0000 7f7fc1b89640 1 -- 172.21.15.45:0/1208493356 shutdown_connections 2024-05-08T05:49:04.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.589+0000 7f7fc1b89640 1 --2- 172.21.15.45:0/1208493356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fbc06d3b0 0x7f7fbc06d820 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.589+0000 7f7fc1b89640 1 --2- 172.21.15.45:0/1208493356 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fbc0733e0 0x7f7fbc0d0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.589+0000 7f7fc1b89640 1 --2- 172.21.15.45:0/1208493356 >> v2:172.21.15.63:3300/0 conn(0x7f7fbc15ce90 0x7f7fbc070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.589+0000 7f7fc1b89640 1 -- 172.21.15.45:0/1208493356 >> 172.21.15.45:0/1208493356 conn(0x7f7fbc0728c0 msgr2=0x7f7fbc072cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:04.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.589+0000 7f7fc1b89640 1 -- 172.21.15.45:0/1208493356 shutdown_connections 2024-05-08T05:49:04.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.589+0000 7f7fc1b89640 1 -- 172.21.15.45:0/1208493356 wait complete. 2024-05-08T05:49:04.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.590+0000 7f7fc1b89640 1 Processor -- start 2024-05-08T05:49:04.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.591+0000 7f7fc1b89640 1 -- start start 2024-05-08T05:49:04.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.591+0000 7f7fc1b89640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fbc06d3b0 0x7f7fbc15d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:04.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.591+0000 7f7fc1b89640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fbc0733e0 0x7f7fbc150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:04.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.591+0000 7f7fc1b89640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fbc15ce90 0x7f7fbc0af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:04.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.591+0000 7f7fc1b89640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc0d0760 con 0x7f7fbc15ce90 2024-05-08T05:49:04.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.591+0000 7f7fc1b89640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc0d0480 con 0x7f7fbc06d3b0 2024-05-08T05:49:04.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.591+0000 7f7fc1b89640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7fbc15d270 con 0x7f7fbc0733e0 2024-05-08T05:49:04.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.591+0000 7f7fbbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fbc15ce90 0x7f7fbc0af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:04.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.591+0000 7f7fbbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fbc15ce90 0x7f7fbc0af7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59644/0 (socket says 172.21.15.45:59644) 2024-05-08T05:49:04.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.591+0000 7f7fbbfff640 1 -- 172.21.15.45:0/781943956 learned_addr learned my addr 172.21.15.45:0/781943956 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:04.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.592+0000 7f7fbaffd640 1 --2- 172.21.15.45:0/781943956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fbc0733e0 0x7f7fbc150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:04.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.592+0000 7f7fbb7fe640 1 --2- 172.21.15.45:0/781943956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fbc06d3b0 0x7f7fbc15d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:04.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.592+0000 7f7fbbfff640 1 -- 172.21.15.45:0/781943956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fbc0733e0 msgr2=0x7f7fbc150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:04.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.592+0000 7f7fbbfff640 1 --2- 172.21.15.45:0/781943956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fbc0733e0 0x7f7fbc150310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.592+0000 7f7fbbfff640 1 -- 172.21.15.45:0/781943956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fbc06d3b0 msgr2=0x7f7fbc15d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:04.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.592+0000 7f7fbbfff640 1 --2- 172.21.15.45:0/781943956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fbc06d3b0 0x7f7fbc15d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:04.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.592+0000 7f7fbbfff640 1 -- 172.21.15.45:0/781943956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc06ef60 con 0x7f7fbc15ce90 2024-05-08T05:49:04.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.592+0000 7f7fbaffd640 1 --2- 172.21.15.45:0/781943956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fbc0733e0 0x7f7fbc150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:04.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.592+0000 7f7fbb7fe640 1 --2- 172.21.15.45:0/781943956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fbc06d3b0 0x7f7fbc15d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:04.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.592+0000 7f7fbbfff640 1 --2- 172.21.15.45:0/781943956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fbc15ce90 0x7f7fbc0af7a0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009f40 tx=0x7f7fb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:04.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.592+0000 7f7fa3fff640 1 -- 172.21.15.45:0/781943956 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4036050 con 0x7f7fbc15ce90 2024-05-08T05:49:04.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.592+0000 7f7fa3fff640 1 -- 172.21.15.45:0/781943956 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb4042070 con 0x7f7fbc15ce90 2024-05-08T05:49:04.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.592+0000 7f7fc1b89640 1 -- 172.21.15.45:0/781943956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fbc0b00d0 con 0x7f7fbc15ce90 2024-05-08T05:49:04.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.592+0000 7f7fa3fff640 1 -- 172.21.15.45:0/781943956 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4009510 con 0x7f7fbc15ce90 2024-05-08T05:49:04.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.592+0000 7f7fc1b89640 1 -- 172.21.15.45:0/781943956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fbc06f4e0 con 0x7f7fbc15ce90 2024-05-08T05:49:04.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.593+0000 7f7fc1b89640 1 -- 172.21.15.45:0/781943956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fbc0693f0 con 0x7f7fbc15ce90 2024-05-08T05:49:04.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.594+0000 7f7fa3fff640 1 -- 172.21.15.45:0/781943956 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7fb4007bd0 con 0x7f7fbc15ce90 2024-05-08T05:49:04.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.594+0000 7f7fa3fff640 1 --2- 172.21.15.45:0/781943956 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7fa80762d0 0x7f7fa8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:04.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.595+0000 7f7fa3fff640 1 -- 172.21.15.45:0/781943956 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5507+0+0 (secure 0 0 0) 0x7f7fb40b84b0 con 0x7f7fbc15ce90 2024-05-08T05:49:04.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.595+0000 7f7fbb7fe640 1 --2- 172.21.15.45:0/781943956 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7fa80762d0 0x7f7fa8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:04.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.595+0000 7f7fbb7fe640 1 --2- 172.21.15.45:0/781943956 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7fa80762d0 0x7f7fa8078790 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00b970 tx=0x7f7fac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:04.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.599+0000 7f7fa3fff640 1 -- 172.21.15.45:0/781943956 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7fb4081e30 con 0x7f7fbc15ce90 2024-05-08T05:49:04.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:04.863+0000 7f7fc1b89640 1 -- 172.21.15.45:0/781943956 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f7fbc0b0ea0 con 0x7f7fbc15ce90 2024-05-08T05:49:05.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:49:05.005 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:49:05.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.142+0000 7f7fa3fff640 1 -- 172.21.15.45:0/781943956 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v348) ==== 167+0+0 (secure 0 0 0) 0x7f7fb4081730 con 0x7f7fbc15ce90 2024-05-08T05:49:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.144+0000 7f7fc1b89640 1 -- 172.21.15.45:0/781943956 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7fa80762d0 msgr2=0x7f7fa8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:05.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.144+0000 7f7fc1b89640 1 --2- 172.21.15.45:0/781943956 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7fa80762d0 0x7f7fa8078790 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00b970 tx=0x7f7fac008040 comp rx=0 tx=0).stop 2024-05-08T05:49:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.144+0000 7f7fc1b89640 1 -- 172.21.15.45:0/781943956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fbc15ce90 msgr2=0x7f7fbc0af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.144+0000 7f7fc1b89640 1 --2- 172.21.15.45:0/781943956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fbc15ce90 0x7f7fbc0af7a0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009f40 tx=0x7f7fb4004a40 comp rx=0 tx=0).stop 2024-05-08T05:49:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.144+0000 7f7fc1b89640 1 -- 172.21.15.45:0/781943956 shutdown_connections 2024-05-08T05:49:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.144+0000 7f7fc1b89640 1 --2- 172.21.15.45:0/781943956 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7fa80762d0 0x7f7fa8078790 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.144+0000 7f7fc1b89640 1 --2- 172.21.15.45:0/781943956 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7fbc0733e0 0x7f7fbc150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.144+0000 7f7fc1b89640 1 --2- 172.21.15.45:0/781943956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fbc15ce90 0x7f7fbc0af7a0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.144+0000 7f7fc1b89640 1 --2- 172.21.15.45:0/781943956 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fbc06d3b0 0x7f7fbc15d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.144+0000 7f7fc1b89640 1 -- 172.21.15.45:0/781943956 >> 172.21.15.45:0/781943956 conn(0x7f7fbc0728c0 msgr2=0x7f7fbc0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.144+0000 7f7fc1b89640 1 -- 172.21.15.45:0/781943956 shutdown_connections 2024-05-08T05:49:05.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.144+0000 7f7fc1b89640 1 -- 172.21.15.45:0/781943956 wait complete. 2024-05-08T05:49:05.155 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-05-08T05:49:05.156 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-08T05:49:05.156 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:49:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.260+0000 7f03b50a7640 1 Processor -- start 2024-05-08T05:49:05.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.260+0000 7f03b50a7640 1 -- start start 2024-05-08T05:49:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.261+0000 7f03b50a7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f03b01629a0 0x7f03b00d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:05.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.261+0000 7f03b50a7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f03b00d2dc0 0x7f03b00d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.261+0000 7f03b50a7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03b00d3330 con 0x7f03b00c8ed0 2024-05-08T05:49:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.261+0000 7f03b50a7640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f03b00d2170 con 0x7f03b01629a0 2024-05-08T05:49:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.261+0000 7f03b50a7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f03b00d2450 con 0x7f03b00d2dc0 2024-05-08T05:49:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.261+0000 7f03aed9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f03b01629a0 0x7f03b00d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.261+0000 7f03aed9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f03b01629a0 0x7f03b00d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59656/0 (socket says 172.21.15.45:59656) 2024-05-08T05:49:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.261+0000 7f03aed9d640 1 -- 172.21.15.45:0/1225365894 learned_addr learned my addr 172.21.15.45:0/1225365894 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.261+0000 7f03a7fff640 1 --2- 172.21.15.45:0/1225365894 >> v2:172.21.15.63:3300/0 conn(0x7f03b00d2dc0 0x7f03b00d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.261+0000 7f03aed9d640 1 -- 172.21.15.45:0/1225365894 >> v2:172.21.15.63:3300/0 conn(0x7f03b00d2dc0 msgr2=0x7f03b00d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.261+0000 7f03aed9d640 1 --2- 172.21.15.45:0/1225365894 >> v2:172.21.15.63:3300/0 conn(0x7f03b00d2dc0 0x7f03b00d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.261+0000 7f03aed9d640 1 -- 172.21.15.45:0/1225365894 >> v1:172.21.15.63:6789/0 conn(0x7f03b00c8ed0 legacy=0x7f03b00c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:05.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.261+0000 7f03aed9d640 1 -- 172.21.15.45:0/1225365894 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b00d2730 con 0x7f03b01629a0 2024-05-08T05:49:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.262+0000 7f03aed9d640 1 --2- 172.21.15.45:0/1225365894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03b01629a0 0x7f03b00d35c0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f03b00b7ae0 tx=0x7f03a002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c377709c9f29fbba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.262+0000 7f03ae59c640 1 -- 172.21.15.45:0/1225365894 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a0037070 con 0x7f03b01629a0 2024-05-08T05:49:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.262+0000 7f03ae59c640 1 -- 172.21.15.45:0/1225365894 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a0032440 con 0x7f03b01629a0 2024-05-08T05:49:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.262+0000 7f03ae59c640 1 -- 172.21.15.45:0/1225365894 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a0032810 con 0x7f03b01629a0 2024-05-08T05:49:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.263+0000 7f03b50a7640 1 -- 172.21.15.45:0/1225365894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03b01629a0 msgr2=0x7f03b00d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.263+0000 7f03b50a7640 1 --2- 172.21.15.45:0/1225365894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03b01629a0 0x7f03b00d35c0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f03b00b7ae0 tx=0x7f03a002a150 comp rx=0 tx=0).stop 2024-05-08T05:49:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.263+0000 7f03b50a7640 1 -- 172.21.15.45:0/1225365894 shutdown_connections 2024-05-08T05:49:05.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.263+0000 7f03b50a7640 1 --2- 172.21.15.45:0/1225365894 >> v2:172.21.15.63:3300/0 conn(0x7f03b00d2dc0 0x7f03b00d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.263+0000 7f03b50a7640 1 --2- 172.21.15.45:0/1225365894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03b01629a0 0x7f03b00d35c0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.263+0000 7f03b50a7640 1 -- 172.21.15.45:0/1225365894 >> 172.21.15.45:0/1225365894 conn(0x7f03b00d4fe0 msgr2=0x7f03b00d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:05.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.263+0000 7f03b50a7640 1 -- 172.21.15.45:0/1225365894 shutdown_connections 2024-05-08T05:49:05.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.263+0000 7f03b50a7640 1 -- 172.21.15.45:0/1225365894 wait complete. 2024-05-08T05:49:05.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.264+0000 7f03b50a7640 1 Processor -- start 2024-05-08T05:49:05.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.264+0000 7f03b50a7640 1 -- start start 2024-05-08T05:49:05.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.265+0000 7f03b50a7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03b00c8ed0 0x7f03b00b8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:05.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.265+0000 7f03b50a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03b00d2dc0 0x7f03b01f4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:05.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.265+0000 7f03b50a7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03b01629a0 0x7f03b00cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:05.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.265+0000 7f03b50a7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f03b00cf440 con 0x7f03b00d2dc0 2024-05-08T05:49:05.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.265+0000 7f03b50a7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f03b00cf2c0 con 0x7f03b00c8ed0 2024-05-08T05:49:05.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.265+0000 7f03b50a7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f03b00d0cb0 con 0x7f03b01629a0 2024-05-08T05:49:05.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.265+0000 7f03a7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03b00d2dc0 0x7f03b01f4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.265+0000 7f03a7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03b00d2dc0 0x7f03b01f4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59666/0 (socket says 172.21.15.45:59666) 2024-05-08T05:49:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.265+0000 7f03a7fff640 1 -- 172.21.15.45:0/783076321 learned_addr learned my addr 172.21.15.45:0/783076321 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.265+0000 7f03aed9d640 1 --2- 172.21.15.45:0/783076321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03b00c8ed0 0x7f03b00b8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.265+0000 7f03af59e640 1 --2- 172.21.15.45:0/783076321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03b01629a0 0x7f03b00cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.265+0000 7f03a7fff640 1 -- 172.21.15.45:0/783076321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03b01629a0 msgr2=0x7f03b00cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.265+0000 7f03a7fff640 1 --2- 172.21.15.45:0/783076321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03b01629a0 0x7f03b00cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.265+0000 7f03a7fff640 1 -- 172.21.15.45:0/783076321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03b00c8ed0 msgr2=0x7f03b00b8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.265+0000 7f03a7fff640 1 --2- 172.21.15.45:0/783076321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03b00c8ed0 0x7f03b00b8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.265+0000 7f03a7fff640 1 -- 172.21.15.45:0/783076321 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b00b8e60 con 0x7f03b00d2dc0 2024-05-08T05:49:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.265+0000 7f03a7fff640 1 --2- 172.21.15.45:0/783076321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03b00d2dc0 0x7f03b01f4420 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f039800e9e0 tx=0x7f039800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.266+0000 7f03a77fe640 1 -- 172.21.15.45:0/783076321 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398017070 con 0x7f03b00d2dc0 2024-05-08T05:49:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.266+0000 7f03a77fe640 1 -- 172.21.15.45:0/783076321 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f039800c9d0 con 0x7f03b00d2dc0 2024-05-08T05:49:05.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.266+0000 7f03a77fe640 1 -- 172.21.15.45:0/783076321 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039800cdc0 con 0x7f03b00d2dc0 2024-05-08T05:49:05.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.266+0000 7f03b50a7640 1 -- 172.21.15.45:0/783076321 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b00b9140 con 0x7f03b00d2dc0 2024-05-08T05:49:05.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.266+0000 7f03b50a7640 1 -- 172.21.15.45:0/783076321 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b01f4d00 con 0x7f03b00d2dc0 2024-05-08T05:49:05.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.267+0000 7f03b50a7640 1 -- 172.21.15.45:0/783076321 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b00693f0 con 0x7f03b00d2dc0 2024-05-08T05:49:05.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.267+0000 7f03a77fe640 1 -- 172.21.15.45:0/783076321 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f039801a070 con 0x7f03b00d2dc0 2024-05-08T05:49:05.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.268+0000 7f03a77fe640 1 --2- 172.21.15.45:0/783076321 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f03900762d0 0x7f0390078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:05.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.268+0000 7f03a77fe640 1 -- 172.21.15.45:0/783076321 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5507+0+0 (secure 0 0 0) 0x7f0398097840 con 0x7f03b00d2dc0 2024-05-08T05:49:05.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.268+0000 7f03aed9d640 1 --2- 172.21.15.45:0/783076321 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f03900762d0 0x7f0390078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:05.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.268+0000 7f03aed9d640 1 --2- 172.21.15.45:0/783076321 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f03900762d0 0x7f0390078790 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f03a0034000 tx=0x7f03a002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:05.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.272+0000 7f03a77fe640 1 -- 172.21.15.45:0/783076321 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0398060c10 con 0x7f03b00d2dc0 2024-05-08T05:49:05.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.466+0000 7f03b50a7640 1 -- 172.21.15.45:0/783076321 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03b00b8b50 con 0x7f03b00d2dc0 2024-05-08T05:49:05.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.469+0000 7f03a77fe640 1 -- 172.21.15.45:0/783076321 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+13506 (secure 0 0 0) 0x7f0398064f10 con 0x7f03b00d2dc0 2024-05-08T05:49:05.470 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:05.473 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":95,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:49:04.133192+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/2945763952":"2024-05-09T05:49:01.123702+0000","172.21.15.45:6833/2945763952":"2024-05-09T05:49:01.123702+0000","172.21.15.63:6837/3194497846":"2024-05-09T05:48:20.008781+0000","172.21.15.63:6835/3194497846":"2024-05-09T05:48:20.008781+0000","172.21.15.63:6836/2942326473":"2024-05-09T05:47:09.529461+0000","172.21.15.63:6834/2942326473":"2024-05-09T05:47:09.529461+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:49:05.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.472+0000 7f03b50a7640 1 -- 172.21.15.45:0/783076321 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f03900762d0 msgr2=0x7f0390078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:05.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.472+0000 7f03b50a7640 1 --2- 172.21.15.45:0/783076321 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f03900762d0 0x7f0390078790 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f03a0034000 tx=0x7f03a002e000 comp rx=0 tx=0).stop 2024-05-08T05:49:05.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.472+0000 7f03b50a7640 1 -- 172.21.15.45:0/783076321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03b00d2dc0 msgr2=0x7f03b01f4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:05.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.472+0000 7f03b50a7640 1 --2- 172.21.15.45:0/783076321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03b00d2dc0 0x7f03b01f4420 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f039800e9e0 tx=0x7f039800eea0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.473+0000 7f03b50a7640 1 -- 172.21.15.45:0/783076321 shutdown_connections 2024-05-08T05:49:05.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.473+0000 7f03b50a7640 1 --2- 172.21.15.45:0/783076321 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f03900762d0 0x7f0390078790 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.473+0000 7f03b50a7640 1 --2- 172.21.15.45:0/783076321 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03b01629a0 0x7f03b00cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.473+0000 7f03b50a7640 1 --2- 172.21.15.45:0/783076321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03b00d2dc0 0x7f03b01f4420 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.473+0000 7f03b50a7640 1 --2- 172.21.15.45:0/783076321 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03b00c8ed0 0x7f03b00b8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.473+0000 7f03b50a7640 1 -- 172.21.15.45:0/783076321 >> 172.21.15.45:0/783076321 conn(0x7f03b00d4fe0 msgr2=0x7f03b0160a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:05.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.473+0000 7f03b50a7640 1 -- 172.21.15.45:0/783076321 shutdown_connections 2024-05-08T05:49:05.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.473+0000 7f03b50a7640 1 -- 172.21.15.45:0/783076321 wait complete. 2024-05-08T05:49:05.484 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T05:41:18.847604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T05:49:05.484 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T05:49:05.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.591+0000 7ff8068a2640 1 Processor -- start 2024-05-08T05:49:05.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.591+0000 7ff8068a2640 1 -- start start 2024-05-08T05:49:05.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.592+0000 7ff8068a2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8000d3cd0 0x7ff8000d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:05.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.592+0000 7ff8068a2640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff8000b3f30 0x7ff8000d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:05.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.592+0000 7ff8068a2640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8000b1c90 con 0x7ff80015f380 2024-05-08T05:49:05.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.592+0000 7ff8068a2640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff8000d0a50 con 0x7ff8000b3f30 2024-05-08T05:49:05.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.592+0000 7ff8068a2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff8000d0d30 con 0x7ff8000d3cd0 2024-05-08T05:49:05.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.592+0000 7ff7fffff640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7ff80015f380 0x7ff8000d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:39650/0 (socket says 172.21.15.45:39650) 2024-05-08T05:49:05.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.592+0000 7ff804e3f640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff8000b3f30 0x7ff8000d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:05.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.592+0000 7ff7ff7fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8000d3cd0 0x7ff8000d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:05.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.592+0000 7ff7fffff640 1 -- 172.21.15.45:0/3132710957 learned_addr learned my addr 172.21.15.45:0/3132710957 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:05.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.592+0000 7ff7feffd640 1 -- 172.21.15.45:0/3132710957 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 830662731 0 0) 0x7ff8000b1c90 con 0x7ff80015f380 2024-05-08T05:49:05.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.593+0000 7ff7ff7fe640 1 -- 172.21.15.45:0/3132710957 >> v2:172.21.15.63:3300/0 conn(0x7ff8000b3f30 msgr2=0x7ff8000d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:05.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.593+0000 7ff7ff7fe640 1 --2- 172.21.15.45:0/3132710957 >> v2:172.21.15.63:3300/0 conn(0x7ff8000b3f30 0x7ff8000d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.593+0000 7ff7ff7fe640 1 -- 172.21.15.45:0/3132710957 >> v1:172.21.15.63:6789/0 conn(0x7ff80015f380 legacy=0x7ff8000d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.593+0000 7ff7ff7fe640 1 -- 172.21.15.45:0/3132710957 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8000b1f00 con 0x7ff8000d3cd0 2024-05-08T05:49:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.593+0000 7ff7ff7fe640 1 --2- 172.21.15.45:0/3132710957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8000d3cd0 0x7ff8000d15f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40097a0 tx=0x7ff7f402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=300ef7640cec6380 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.593+0000 7ff7feffd640 1 -- 172.21.15.45:0/3132710957 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4037070 con 0x7ff8000d3cd0 2024-05-08T05:49:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.593+0000 7ff7feffd640 1 -- 172.21.15.45:0/3132710957 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7f4032440 con 0x7ff8000d3cd0 2024-05-08T05:49:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.594+0000 7ff7feffd640 1 -- 172.21.15.45:0/3132710957 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4032810 con 0x7ff8000d3cd0 2024-05-08T05:49:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.594+0000 7ff8068a2640 1 -- 172.21.15.45:0/3132710957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8000d3cd0 msgr2=0x7ff8000d15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.594+0000 7ff8068a2640 1 --2- 172.21.15.45:0/3132710957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8000d3cd0 0x7ff8000d15f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40097a0 tx=0x7ff7f402a180 comp rx=0 tx=0).stop 2024-05-08T05:49:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.594+0000 7ff8068a2640 1 -- 172.21.15.45:0/3132710957 shutdown_connections 2024-05-08T05:49:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.594+0000 7ff8068a2640 1 --2- 172.21.15.45:0/3132710957 >> v2:172.21.15.63:3300/0 conn(0x7ff8000b3f30 0x7ff8000d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.594+0000 7ff8068a2640 1 --2- 172.21.15.45:0/3132710957 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8000d3cd0 0x7ff8000d15f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.594+0000 7ff8068a2640 1 -- 172.21.15.45:0/3132710957 >> 172.21.15.45:0/3132710957 conn(0x7ff8000d4fe0 msgr2=0x7ff8000d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.594+0000 7ff8068a2640 1 -- 172.21.15.45:0/3132710957 shutdown_connections 2024-05-08T05:49:05.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.594+0000 7ff8068a2640 1 -- 172.21.15.45:0/3132710957 wait complete. 2024-05-08T05:49:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.595+0000 7ff8068a2640 1 Processor -- start 2024-05-08T05:49:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.595+0000 7ff8068a2640 1 -- start start 2024-05-08T05:49:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.596+0000 7ff8068a2640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8000b3f30 0x7ff8000cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.596+0000 7ff8068a2640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8000d3cd0 0x7ff80015e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.596+0000 7ff8068a2640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff80015f380 0x7ff8000ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.596+0000 7ff8068a2640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff8000d06e0 con 0x7ff80015f380 2024-05-08T05:49:05.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.596+0000 7ff8068a2640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff8000d1010 con 0x7ff8000b3f30 2024-05-08T05:49:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.596+0000 7ff8068a2640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff8000b3430 con 0x7ff8000d3cd0 2024-05-08T05:49:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.596+0000 7ff804e3f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff80015f380 0x7ff8000ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.596+0000 7ff804e3f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff80015f380 0x7ff8000ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59680/0 (socket says 172.21.15.45:59680) 2024-05-08T05:49:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.596+0000 7ff804e3f640 1 -- 172.21.15.45:0/2401842682 learned_addr learned my addr 172.21.15.45:0/2401842682 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.596+0000 7ff7fffff640 1 --2- 172.21.15.45:0/2401842682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8000b3f30 0x7ff8000cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.597+0000 7ff7ff7fe640 1 --2- 172.21.15.45:0/2401842682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8000d3cd0 0x7ff80015e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.597+0000 7ff804e3f640 1 -- 172.21.15.45:0/2401842682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8000d3cd0 msgr2=0x7ff80015e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.597+0000 7ff804e3f640 1 --2- 172.21.15.45:0/2401842682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8000d3cd0 0x7ff80015e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.597+0000 7ff804e3f640 1 -- 172.21.15.45:0/2401842682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8000b3f30 msgr2=0x7ff8000cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.597+0000 7ff804e3f640 1 --2- 172.21.15.45:0/2401842682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8000b3f30 0x7ff8000cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.597+0000 7ff804e3f640 1 -- 172.21.15.45:0/2401842682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8000cdc40 con 0x7ff80015f380 2024-05-08T05:49:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.597+0000 7ff7ff7fe640 1 --2- 172.21.15.45:0/2401842682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8000d3cd0 0x7ff80015e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:49:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.597+0000 7ff804e3f640 1 --2- 172.21.15.45:0/2401842682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff80015f380 0x7ff8000ccc50 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800efc0 tx=0x7ff7f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:05.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.597+0000 7ff7fcff9640 1 -- 172.21.15.45:0/2401842682 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8018020 con 0x7ff80015f380 2024-05-08T05:49:05.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.597+0000 7ff7fcff9640 1 -- 172.21.15.45:0/2401842682 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7f8014480 con 0x7ff80015f380 2024-05-08T05:49:05.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.597+0000 7ff8068a2640 1 -- 172.21.15.45:0/2401842682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8000cd3a0 con 0x7ff80015f380 2024-05-08T05:49:05.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.597+0000 7ff7fcff9640 1 -- 172.21.15.45:0/2401842682 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8014870 con 0x7ff80015f380 2024-05-08T05:49:05.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.597+0000 7ff8068a2640 1 -- 172.21.15.45:0/2401842682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8000cd960 con 0x7ff80015f380 2024-05-08T05:49:05.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.598+0000 7ff8068a2640 1 -- 172.21.15.45:0/2401842682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8000693f0 con 0x7ff80015f380 2024-05-08T05:49:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.599+0000 7ff7fcff9640 1 -- 172.21.15.45:0/2401842682 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff7f8005d10 con 0x7ff80015f380 2024-05-08T05:49:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.599+0000 7ff7fcff9640 1 --2- 172.21.15.45:0/2401842682 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff7ec0762d0 0x7ff7ec078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.599+0000 7ff7fcff9640 1 -- 172.21.15.45:0/2401842682 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5507+0+0 (secure 0 0 0) 0x7ff7f8098130 con 0x7ff80015f380 2024-05-08T05:49:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.599+0000 7ff7fffff640 1 --2- 172.21.15.45:0/2401842682 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff7ec0762d0 0x7ff7ec078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:05.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.600+0000 7ff7fffff640 1 --2- 172.21.15.45:0/2401842682 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff7ec0762d0 0x7ff7ec078790 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0000f60 tx=0x7ff7f0007c00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:05.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.603+0000 7ff7fcff9640 1 -- 172.21.15.45:0/2401842682 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff7f8022070 con 0x7ff80015f380 2024-05-08T05:49:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.793+0000 7ff8068a2640 1 -- 172.21.15.45:0/2401842682 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff8000763f0 con 0x7ff80015f380 2024-05-08T05:49:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.793+0000 7ff7fcff9640 1 -- 172.21.15.45:0/2401842682 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7ff7f8010040 con 0x7ff80015f380 2024-05-08T05:49:05.793 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-05-08T05:49:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.795+0000 7ff8068a2640 1 -- 172.21.15.45:0/2401842682 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff7ec0762d0 msgr2=0x7ff7ec078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.795+0000 7ff8068a2640 1 --2- 172.21.15.45:0/2401842682 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff7ec0762d0 0x7ff7ec078790 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0000f60 tx=0x7ff7f0007c00 comp rx=0 tx=0).stop 2024-05-08T05:49:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.795+0000 7ff8068a2640 1 -- 172.21.15.45:0/2401842682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff80015f380 msgr2=0x7ff8000ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.795+0000 7ff8068a2640 1 --2- 172.21.15.45:0/2401842682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff80015f380 0x7ff8000ccc50 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800efc0 tx=0x7ff7f800c490 comp rx=0 tx=0).stop 2024-05-08T05:49:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.795+0000 7ff8068a2640 1 -- 172.21.15.45:0/2401842682 shutdown_connections 2024-05-08T05:49:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.795+0000 7ff8068a2640 1 --2- 172.21.15.45:0/2401842682 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff7ec0762d0 0x7ff7ec078790 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.795+0000 7ff8068a2640 1 --2- 172.21.15.45:0/2401842682 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff8000d3cd0 0x7ff80015e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.795+0000 7ff8068a2640 1 --2- 172.21.15.45:0/2401842682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff80015f380 0x7ff8000ccc50 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.795+0000 7ff8068a2640 1 --2- 172.21.15.45:0/2401842682 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8000b3f30 0x7ff8000cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.795+0000 7ff8068a2640 1 -- 172.21.15.45:0/2401842682 >> 172.21.15.45:0/2401842682 conn(0x7ff8000d4fe0 msgr2=0x7ff8000d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.795+0000 7ff8068a2640 1 -- 172.21.15.45:0/2401842682 shutdown_connections 2024-05-08T05:49:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.795+0000 7ff8068a2640 1 -- 172.21.15.45:0/2401842682 wait complete. 2024-05-08T05:49:05.807 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-05-08T05:49:05.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.913+0000 7f60a002a640 1 Processor -- start 2024-05-08T05:49:05.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.914+0000 7f60a002a640 1 -- start start 2024-05-08T05:49:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.914+0000 7f60a002a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f60981627f0 0x7f60980d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.914+0000 7f60a002a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6098070820 con 0x7f60980d3a40 2024-05-08T05:49:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.914+0000 7f60a002a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60980d0270 con 0x7f6098162bd0 2024-05-08T05:49:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.914+0000 7f60a002a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f6098070b30 con 0x7f60981627f0 2024-05-08T05:49:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.915+0000 7f609ddc6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f60981627f0 0x7f60980d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.915+0000 7f609d5c5640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f60980d3a40 0x7f6098070fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51136/0 (socket says 172.21.15.45:51136) 2024-05-08T05:49:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.915+0000 7f609d5c5640 1 -- 172.21.15.45:0/2992835140 learned_addr learned my addr 172.21.15.45:0/2992835140 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.915+0000 7f609ddc6640 1 -- 172.21.15.45:0/2992835140 >> v1:172.21.15.63:6789/0 conn(0x7f6098162bd0 legacy=0x7f60980716d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.915+0000 7f609ddc6640 1 -- 172.21.15.45:0/2992835140 >> v1:172.21.15.45:6789/0 conn(0x7f60980d3a40 legacy=0x7f6098070fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:05.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.915+0000 7f609ddc6640 1 -- 172.21.15.45:0/2992835140 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60980d0550 con 0x7f60981627f0 2024-05-08T05:49:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.916+0000 7f609ddc6640 1 --2- 172.21.15.45:0/2992835140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60981627f0 0x7f60980d0d40 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f60940097a0 tx=0x7f609402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d003505bd32279b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.916+0000 7f609cdc4640 1 -- 172.21.15.45:0/2992835140 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094037070 con 0x7f60981627f0 2024-05-08T05:49:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.916+0000 7f609cdc4640 1 -- 172.21.15.45:0/2992835140 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6094032440 con 0x7f60981627f0 2024-05-08T05:49:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.916+0000 7f609cdc4640 1 -- 172.21.15.45:0/2992835140 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094032810 con 0x7f60981627f0 2024-05-08T05:49:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.916+0000 7f60a002a640 1 -- 172.21.15.45:0/2992835140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60981627f0 msgr2=0x7f60980d0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.916+0000 7f60a002a640 1 --2- 172.21.15.45:0/2992835140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60981627f0 0x7f60980d0d40 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f60940097a0 tx=0x7f609402a180 comp rx=0 tx=0).stop 2024-05-08T05:49:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.916+0000 7f60a002a640 1 -- 172.21.15.45:0/2992835140 shutdown_connections 2024-05-08T05:49:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.916+0000 7f60a002a640 1 --2- 172.21.15.45:0/2992835140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60981627f0 0x7f60980d0d40 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.916+0000 7f60a002a640 1 -- 172.21.15.45:0/2992835140 >> 172.21.15.45:0/2992835140 conn(0x7f60980d2e40 msgr2=0x7f60980d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.917+0000 7f60a002a640 1 -- 172.21.15.45:0/2992835140 shutdown_connections 2024-05-08T05:49:05.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.917+0000 7f60a002a640 1 -- 172.21.15.45:0/2992835140 wait complete. 2024-05-08T05:49:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.918+0000 7f60a002a640 1 Processor -- start 2024-05-08T05:49:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.918+0000 7f60a002a640 1 -- start start 2024-05-08T05:49:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.918+0000 7f60a002a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60980d3a40 0x7f6098160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.918+0000 7f60a002a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60981627f0 0x7f609815a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.918+0000 7f60a002a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6098162bd0 0x7f60981499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f60a002a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f60981630a0 con 0x7f60981627f0 2024-05-08T05:49:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f60a002a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f60980b2290 con 0x7f60980d3a40 2024-05-08T05:49:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f60a002a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6098163220 con 0x7f6098162bd0 2024-05-08T05:49:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f609d5c5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60981627f0 0x7f609815a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f609d5c5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60981627f0 0x7f609815a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59692/0 (socket says 172.21.15.45:59692) 2024-05-08T05:49:05.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f609d5c5640 1 -- 172.21.15.45:0/3385194761 learned_addr learned my addr 172.21.15.45:0/3385194761 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f609ddc6640 1 --2- 172.21.15.45:0/3385194761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60980d3a40 0x7f6098160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f609e5c7640 1 --2- 172.21.15.45:0/3385194761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6098162bd0 0x7f60981499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f609d5c5640 1 -- 172.21.15.45:0/3385194761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6098162bd0 msgr2=0x7f60981499e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f609d5c5640 1 --2- 172.21.15.45:0/3385194761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6098162bd0 0x7f60981499e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f609d5c5640 1 -- 172.21.15.45:0/3385194761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60980d3a40 msgr2=0x7f6098160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f609d5c5640 1 --2- 172.21.15.45:0/3385194761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60980d3a40 0x7f6098160880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f609d5c5640 1 -- 172.21.15.45:0/3385194761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609815a8e0 con 0x7f60981627f0 2024-05-08T05:49:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f609ddc6640 1 --2- 172.21.15.45:0/3385194761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60980d3a40 0x7f6098160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:49:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f609e5c7640 1 --2- 172.21.15.45:0/3385194761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6098162bd0 0x7f60981499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:49:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f609d5c5640 1 --2- 172.21.15.45:0/3385194761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60981627f0 0x7f609815a3a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f608800c500 tx=0x7f608800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f608e7fc640 1 -- 172.21.15.45:0/3385194761 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088017070 con 0x7f60981627f0 2024-05-08T05:49:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f608e7fc640 1 -- 172.21.15.45:0/3385194761 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6088005d00 con 0x7f60981627f0 2024-05-08T05:49:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f60a002a640 1 -- 172.21.15.45:0/3385194761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6098149f20 con 0x7f60981627f0 2024-05-08T05:49:05.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f608e7fc640 1 -- 172.21.15.45:0/3385194761 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088007330 con 0x7f60981627f0 2024-05-08T05:49:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.919+0000 7f60a002a640 1 -- 172.21.15.45:0/3385194761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6098161190 con 0x7f60981627f0 2024-05-08T05:49:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.920+0000 7f60a002a640 1 -- 172.21.15.45:0/3385194761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6098161c40 con 0x7f60981627f0 2024-05-08T05:49:05.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.921+0000 7f608e7fc640 1 -- 172.21.15.45:0/3385194761 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f608801a070 con 0x7f60981627f0 2024-05-08T05:49:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.921+0000 7f608e7fc640 1 --2- 172.21.15.45:0/3385194761 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f607c0762d0 0x7f607c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.922+0000 7f608e7fc640 1 -- 172.21.15.45:0/3385194761 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5507+0+0 (secure 0 0 0) 0x7f60880968e0 con 0x7f60981627f0 2024-05-08T05:49:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.922+0000 7f609ddc6640 1 --2- 172.21.15.45:0/3385194761 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f607c0762d0 0x7f607c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:05.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.922+0000 7f609ddc6640 1 --2- 172.21.15.45:0/3385194761 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f607c0762d0 0x7f607c078790 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6094034000 tx=0x7f609402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:05.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:05.926+0000 7f608e7fc640 1 -- 172.21.15.45:0/3385194761 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6098070820 con 0x7f60981627f0 2024-05-08T05:49:06.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:06.195+0000 7f60a002a640 1 -- 172.21.15.45:0/3385194761 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f60980693f0 con 0x7f60981627f0 2024-05-08T05:49:07.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.148+0000 7f608e7fc640 1 -- 172.21.15.45:0/3385194761 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v350) ==== 149+0+0 (secure 0 0 0) 0x7f6088060300 con 0x7f60981627f0 2024-05-08T05:49:07.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.151+0000 7f60a002a640 1 -- 172.21.15.45:0/3385194761 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f607c0762d0 msgr2=0x7f607c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:07.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.151+0000 7f60a002a640 1 --2- 172.21.15.45:0/3385194761 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f607c0762d0 0x7f607c078790 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6094034000 tx=0x7f609402e000 comp rx=0 tx=0).stop 2024-05-08T05:49:07.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.152+0000 7f60a002a640 1 -- 172.21.15.45:0/3385194761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60981627f0 msgr2=0x7f609815a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:07.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.152+0000 7f60a002a640 1 --2- 172.21.15.45:0/3385194761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60981627f0 0x7f609815a3a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f608800c500 tx=0x7f608800c9c0 comp rx=0 tx=0).stop 2024-05-08T05:49:07.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.152+0000 7f60a002a640 1 -- 172.21.15.45:0/3385194761 shutdown_connections 2024-05-08T05:49:07.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.152+0000 7f60a002a640 1 --2- 172.21.15.45:0/3385194761 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f607c0762d0 0x7f607c078790 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:07.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.152+0000 7f60a002a640 1 --2- 172.21.15.45:0/3385194761 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6098162bd0 0x7f60981499e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:07.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.152+0000 7f60a002a640 1 --2- 172.21.15.45:0/3385194761 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60981627f0 0x7f609815a3a0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:07.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.152+0000 7f60a002a640 1 --2- 172.21.15.45:0/3385194761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60980d3a40 0x7f6098160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:07.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.152+0000 7f60a002a640 1 -- 172.21.15.45:0/3385194761 >> 172.21.15.45:0/3385194761 conn(0x7f60980d2e40 msgr2=0x7f60980b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:07.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.152+0000 7f60a002a640 1 -- 172.21.15.45:0/3385194761 shutdown_connections 2024-05-08T05:49:07.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.152+0000 7f60a002a640 1 -- 172.21.15.45:0/3385194761 wait complete. 2024-05-08T05:49:07.165 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T05:49:07.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.278+0000 7f14cb820640 1 Processor -- start 2024-05-08T05:49:07.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.278+0000 7f14cb820640 1 -- start start 2024-05-08T05:49:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.279+0000 7f14cb820640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c40b5ed0 0x7f14c40b7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.279+0000 7f14cb820640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f14c40d18e0 0x7f14c40d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.279+0000 7f14cb820640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14c40d07f0 con 0x7f14c4162910 2024-05-08T05:49:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.279+0000 7f14cb820640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f14c40d2be0 con 0x7f14c40d18e0 2024-05-08T05:49:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.279+0000 7f14cb820640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f14c4162cf0 con 0x7f14c40b5ed0 2024-05-08T05:49:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.279+0000 7f14c9dbd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f14c40d18e0 0x7f14c40d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.279+0000 7f14c9dbd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f14c40d18e0 0x7f14c40d1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59706/0 (socket says 172.21.15.45:59706) 2024-05-08T05:49:07.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.279+0000 7f14c9dbd640 1 -- 172.21.15.45:0/2611848309 learned_addr learned my addr 172.21.15.45:0/2611848309 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.280+0000 7f14c8dbb640 1 --2- 172.21.15.45:0/2611848309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c40b5ed0 0x7f14c40b7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.280+0000 7f14c9dbd640 1 -- 172.21.15.45:0/2611848309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c40b5ed0 msgr2=0x7f14c40b7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.280+0000 7f14c9dbd640 1 --2- 172.21.15.45:0/2611848309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c40b5ed0 0x7f14c40b7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.280+0000 7f14c9dbd640 1 -- 172.21.15.45:0/2611848309 >> v1:172.21.15.63:6789/0 conn(0x7f14c4162910 legacy=0x7f14c40d5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:07.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.280+0000 7f14c9dbd640 1 -- 172.21.15.45:0/2611848309 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c40d0ad0 con 0x7f14c40d18e0 2024-05-08T05:49:07.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.280+0000 7f14c9dbd640 1 --2- 172.21.15.45:0/2611848309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14c40d18e0 0x7f14c40d1d70 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009870 tx=0x7f14bc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8b317c38901a08 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:07.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.281+0000 7f14bbfff640 1 -- 172.21.15.45:0/2611848309 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc037070 con 0x7f14c40d18e0 2024-05-08T05:49:07.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.281+0000 7f14bbfff640 1 -- 172.21.15.45:0/2611848309 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14bc032440 con 0x7f14c40d18e0 2024-05-08T05:49:07.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.281+0000 7f14bbfff640 1 -- 172.21.15.45:0/2611848309 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc032810 con 0x7f14c40d18e0 2024-05-08T05:49:07.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.281+0000 7f14cb820640 1 -- 172.21.15.45:0/2611848309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14c40d18e0 msgr2=0x7f14c40d1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:07.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.281+0000 7f14cb820640 1 --2- 172.21.15.45:0/2611848309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14c40d18e0 0x7f14c40d1d70 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009870 tx=0x7f14bc02a1c0 comp rx=0 tx=0).stop 2024-05-08T05:49:07.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.281+0000 7f14cb820640 1 -- 172.21.15.45:0/2611848309 shutdown_connections 2024-05-08T05:49:07.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.281+0000 7f14cb820640 1 --2- 172.21.15.45:0/2611848309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14c40d18e0 0x7f14c40d1d70 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:07.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.281+0000 7f14cb820640 1 --2- 172.21.15.45:0/2611848309 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c40b5ed0 0x7f14c40b7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:07.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.281+0000 7f14cb820640 1 -- 172.21.15.45:0/2611848309 >> 172.21.15.45:0/2611848309 conn(0x7f14c409e3e0 msgr2=0x7f14c40d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:07.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.281+0000 7f14cb820640 1 -- 172.21.15.45:0/2611848309 shutdown_connections 2024-05-08T05:49:07.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.281+0000 7f14cb820640 1 -- 172.21.15.45:0/2611848309 wait complete. 2024-05-08T05:49:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.282+0000 7f14cb820640 1 Processor -- start 2024-05-08T05:49:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.283+0000 7f14cb820640 1 -- start start 2024-05-08T05:49:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.283+0000 7f14cb820640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14c40b5ed0 0x7f14c40ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.283+0000 7f14cb820640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14c40d18e0 0x7f14c414d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.283+0000 7f14cb820640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c4162910 0x7f14c40ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.283+0000 7f14cb820640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14c40d3350 con 0x7f14c40d18e0 2024-05-08T05:49:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.283+0000 7f14cb820640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f14c40d2e30 con 0x7f14c40b5ed0 2024-05-08T05:49:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.283+0000 7f14cb820640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f14c40d34d0 con 0x7f14c4162910 2024-05-08T05:49:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.283+0000 7f14c8dbb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14c40d18e0 0x7f14c414d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.283+0000 7f14c8dbb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14c40d18e0 0x7f14c414d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59710/0 (socket says 172.21.15.45:59710) 2024-05-08T05:49:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.283+0000 7f14c8dbb640 1 -- 172.21.15.45:0/850843878 learned_addr learned my addr 172.21.15.45:0/850843878 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:07.285 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.284+0000 7f14c8dbb640 1 -- 172.21.15.45:0/850843878 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c4162910 msgr2=0x7f14c40ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.284+0000 7f14c8dbb640 1 --2- 172.21.15.45:0/850843878 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c4162910 0x7f14c40ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.284+0000 7f14c8dbb640 1 -- 172.21.15.45:0/850843878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14c40b5ed0 msgr2=0x7f14c40ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.284+0000 7f14c8dbb640 1 --2- 172.21.15.45:0/850843878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14c40b5ed0 0x7f14c40ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.284+0000 7f14c8dbb640 1 -- 172.21.15.45:0/850843878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c40cad40 con 0x7f14c40d18e0 2024-05-08T05:49:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.284+0000 7f14c8dbb640 1 --2- 172.21.15.45:0/850843878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14c40d18e0 0x7f14c414d4a0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f14b400c820 tx=0x7f14b400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.284+0000 7f14b9ffb640 1 -- 172.21.15.45:0/850843878 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4015070 con 0x7f14c40d18e0 2024-05-08T05:49:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.284+0000 7f14b9ffb640 1 -- 172.21.15.45:0/850843878 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b4004240 con 0x7f14c40d18e0 2024-05-08T05:49:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.284+0000 7f14cb820640 1 -- 172.21.15.45:0/850843878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c40cafc0 con 0x7f14c40d18e0 2024-05-08T05:49:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.284+0000 7f14b9ffb640 1 -- 172.21.15.45:0/850843878 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b400ce10 con 0x7f14c40d18e0 2024-05-08T05:49:07.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.284+0000 7f14cb820640 1 -- 172.21.15.45:0/850843878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c40cb580 con 0x7f14c40d18e0 2024-05-08T05:49:07.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.285+0000 7f14cb820640 1 -- 172.21.15.45:0/850843878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c40693f0 con 0x7f14c40d18e0 2024-05-08T05:49:07.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.286+0000 7f14b9ffb640 1 -- 172.21.15.45:0/850843878 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f14b40077d0 con 0x7f14c40d18e0 2024-05-08T05:49:07.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.286+0000 7f14b9ffb640 1 --2- 172.21.15.45:0/850843878 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f14a80762d0 0x7f14a8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:07.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.287+0000 7f14b9ffb640 1 -- 172.21.15.45:0/850843878 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5507+0+0 (secure 0 0 0) 0x7f14b4097400 con 0x7f14c40d18e0 2024-05-08T05:49:07.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.287+0000 7f14c95bc640 1 --2- 172.21.15.45:0/850843878 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f14a80762d0 0x7f14a8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:07.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.287+0000 7f14c95bc640 1 --2- 172.21.15.45:0/850843878 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f14a80762d0 0x7f14a8078790 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f14c000ab90 tx=0x7f14c0009230 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:07.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.291+0000 7f14b9ffb640 1 -- 172.21.15.45:0/850843878 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f14c40cad40 con 0x7f14c40d18e0 2024-05-08T05:49:07.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.486+0000 7f14cb820640 1 -- 172.21.15.45:0/850843878 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f14c40cf4d0 con 0x7f14c40d18e0 2024-05-08T05:49:07.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.489+0000 7f14b9ffb640 1 -- 172.21.15.45:0/850843878 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f14b4060d80 con 0x7f14c40d18e0 2024-05-08T05:49:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.491+0000 7f14cb820640 1 -- 172.21.15.45:0/850843878 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f14a80762d0 msgr2=0x7f14a8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.491+0000 7f14cb820640 1 --2- 172.21.15.45:0/850843878 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f14a80762d0 0x7f14a8078790 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f14c000ab90 tx=0x7f14c0009230 comp rx=0 tx=0).stop 2024-05-08T05:49:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.491+0000 7f14cb820640 1 -- 172.21.15.45:0/850843878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14c40d18e0 msgr2=0x7f14c414d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.491+0000 7f14cb820640 1 --2- 172.21.15.45:0/850843878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14c40d18e0 0x7f14c414d4a0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f14b400c820 tx=0x7f14b400cce0 comp rx=0 tx=0).stop 2024-05-08T05:49:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.491+0000 7f14cb820640 1 -- 172.21.15.45:0/850843878 shutdown_connections 2024-05-08T05:49:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.491+0000 7f14cb820640 1 --2- 172.21.15.45:0/850843878 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f14a80762d0 0x7f14a8078790 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.491+0000 7f14cb820640 1 --2- 172.21.15.45:0/850843878 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14c4162910 0x7f14c40ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:07.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.491+0000 7f14cb820640 1 --2- 172.21.15.45:0/850843878 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14c40d18e0 0x7f14c414d4a0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.491+0000 7f14cb820640 1 --2- 172.21.15.45:0/850843878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14c40b5ed0 0x7f14c40ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.491+0000 7f14cb820640 1 -- 172.21.15.45:0/850843878 >> 172.21.15.45:0/850843878 conn(0x7f14c409e3e0 msgr2=0x7f14c41604c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.491+0000 7f14cb820640 1 -- 172.21.15.45:0/850843878 shutdown_connections 2024-05-08T05:49:07.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.491+0000 7f14cb820640 1 -- 172.21.15.45:0/850843878 wait complete. 2024-05-08T05:49:07.503 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T05:49:07.503 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:49:07.557 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:49:07.615 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T05:49:07.615 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:49:07.615 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:49:07.631 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:49:07.632 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:49:07.632 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:49:07.651 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:49:07.651 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:49:07.651 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:49:07.688 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:49:07.688 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:49:07.688 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:49:07.709 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:49:07.709 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T05:49:07.709 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:49:07.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.831+0000 7f6194e7e640 1 Processor -- start 2024-05-08T05:49:07.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.831+0000 7f6194e7e640 1 -- start start 2024-05-08T05:49:07.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.832+0000 7f6194e7e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f61900aedf0 0x7f61900b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:07.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.832+0000 7f6194e7e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f619009ff30 con 0x7f619006c3a0 2024-05-08T05:49:07.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.832+0000 7f6194e7e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f619015d750 con 0x7f619015d180 2024-05-08T05:49:07.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.832+0000 7f6194e7e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f619015e360 con 0x7f61900aedf0 2024-05-08T05:49:07.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.832+0000 7f618dd9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f61900aedf0 0x7f61900b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:07.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.832+0000 7f618e59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f619006c3a0 0x7f619006c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:51138/0 (socket says 172.21.15.45:51138) 2024-05-08T05:49:07.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.832+0000 7f618e59c640 1 -- 172.21.15.45:0/322705348 learned_addr learned my addr 172.21.15.45:0/322705348 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:07.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.832+0000 7f618dd9b640 1 -- 172.21.15.45:0/322705348 >> v1:172.21.15.63:6789/0 conn(0x7f619015d180 legacy=0x7f61900b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:07.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.832+0000 7f618dd9b640 1 -- 172.21.15.45:0/322705348 >> v1:172.21.15.45:6789/0 conn(0x7f619006c3a0 legacy=0x7f619006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:07.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.832+0000 7f618dd9b640 1 -- 172.21.15.45:0/322705348 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619015da30 con 0x7f61900aedf0 2024-05-08T05:49:07.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.833+0000 7f618dd9b640 1 --2- 172.21.15.45:0/322705348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61900aedf0 0x7f61900b13f0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f61780097a0 tx=0x7f617802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206b964bed02b8e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:07.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.833+0000 7f618d59a640 1 -- 172.21.15.45:0/322705348 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178037070 con 0x7f61900aedf0 2024-05-08T05:49:07.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.833+0000 7f618d59a640 1 -- 172.21.15.45:0/322705348 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f617802ad10 con 0x7f61900aedf0 2024-05-08T05:49:07.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.833+0000 7f618d59a640 1 -- 172.21.15.45:0/322705348 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178032660 con 0x7f61900aedf0 2024-05-08T05:49:07.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.834+0000 7f6194e7e640 1 -- 172.21.15.45:0/322705348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61900aedf0 msgr2=0x7f61900b13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:07.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.834+0000 7f6194e7e640 1 --2- 172.21.15.45:0/322705348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61900aedf0 0x7f61900b13f0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f61780097a0 tx=0x7f617802a080 comp rx=0 tx=0).stop 2024-05-08T05:49:07.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.834+0000 7f6194e7e640 1 -- 172.21.15.45:0/322705348 shutdown_connections 2024-05-08T05:49:07.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.834+0000 7f6194e7e640 1 --2- 172.21.15.45:0/322705348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61900aedf0 0x7f61900b13f0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:07.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.834+0000 7f6194e7e640 1 -- 172.21.15.45:0/322705348 >> 172.21.15.45:0/322705348 conn(0x7f61900b3b10 msgr2=0x7f61900b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:07.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.834+0000 7f6194e7e640 1 -- 172.21.15.45:0/322705348 shutdown_connections 2024-05-08T05:49:07.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.834+0000 7f6194e7e640 1 -- 172.21.15.45:0/322705348 wait complete. 2024-05-08T05:49:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.835+0000 7f6194e7e640 1 Processor -- start 2024-05-08T05:49:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.835+0000 7f6194e7e640 1 -- start start 2024-05-08T05:49:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.836+0000 7f6194e7e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f619006c3a0 0x7f619009e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.836+0000 7f6194e7e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61900aedf0 0x7f619009ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.836+0000 7f6194e7e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f619015d180 0x7f619014b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:07.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.836+0000 7f6194e7e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f61900b6130 con 0x7f61900aedf0 2024-05-08T05:49:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.836+0000 7f6194e7e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f619015dff0 con 0x7f619006c3a0 2024-05-08T05:49:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.836+0000 7f6194e7e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f61900b62b0 con 0x7f619015d180 2024-05-08T05:49:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.836+0000 7f618dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61900aedf0 0x7f619009ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.836+0000 7f618dd9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61900aedf0 0x7f619009ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59734/0 (socket says 172.21.15.45:59734) 2024-05-08T05:49:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.836+0000 7f618dd9b640 1 -- 172.21.15.45:0/3493378516 learned_addr learned my addr 172.21.15.45:0/3493378516 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.836+0000 7f618e59c640 1 --2- 172.21.15.45:0/3493378516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f619006c3a0 0x7f619009e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.836+0000 7f618ed9d640 1 --2- 172.21.15.45:0/3493378516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f619015d180 0x7f619014b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.836+0000 7f618dd9b640 1 -- 172.21.15.45:0/3493378516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f619015d180 msgr2=0x7f619014b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.836+0000 7f618dd9b640 1 --2- 172.21.15.45:0/3493378516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f619015d180 0x7f619014b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.836+0000 7f618dd9b640 1 -- 172.21.15.45:0/3493378516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f619006c3a0 msgr2=0x7f619009e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.836+0000 7f618dd9b640 1 --2- 172.21.15.45:0/3493378516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f619006c3a0 0x7f619009e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:07.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.836+0000 7f618dd9b640 1 -- 172.21.15.45:0/3493378516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619009bf80 con 0x7f61900aedf0 2024-05-08T05:49:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.836+0000 7f618dd9b640 1 --2- 172.21.15.45:0/3493378516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61900aedf0 0x7f619009ba40 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f6178000c00 tx=0x7f61780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.837+0000 7f6186ffd640 1 -- 172.21.15.45:0/3493378516 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178037040 con 0x7f61900aedf0 2024-05-08T05:49:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.837+0000 7f6186ffd640 1 -- 172.21.15.45:0/3493378516 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6178004030 con 0x7f61900aedf0 2024-05-08T05:49:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.837+0000 7f6194e7e640 1 -- 172.21.15.45:0/3493378516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f619009ed50 con 0x7f61900aedf0 2024-05-08T05:49:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.837+0000 7f6186ffd640 1 -- 172.21.15.45:0/3493378516 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178004420 con 0x7f61900aedf0 2024-05-08T05:49:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.837+0000 7f6194e7e640 1 -- 172.21.15.45:0/3493378516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f619009f310 con 0x7f61900aedf0 2024-05-08T05:49:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.838+0000 7f6194e7e640 1 -- 172.21.15.45:0/3493378516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f619006a8f0 con 0x7f61900aedf0 2024-05-08T05:49:07.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.838+0000 7f6186ffd640 1 -- 172.21.15.45:0/3493378516 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f617803a070 con 0x7f61900aedf0 2024-05-08T05:49:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.839+0000 7f6186ffd640 1 --2- 172.21.15.45:0/3493378516 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f61700762d0 0x7f6170078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.839+0000 7f6186ffd640 1 -- 172.21.15.45:0/3493378516 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5507+0+0 (secure 0 0 0) 0x7f61780b7a30 con 0x7f61900aedf0 2024-05-08T05:49:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.839+0000 7f618e59c640 1 --2- 172.21.15.45:0/3493378516 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f61700762d0 0x7f6170078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:07.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.839+0000 7f618e59c640 1 --2- 172.21.15.45:0/3493378516 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f61700762d0 0x7f6170078790 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6180000f60 tx=0x7f6180007ca0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:07.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:07.843+0000 7f6186ffd640 1 -- 172.21.15.45:0/3493378516 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6178041020 con 0x7f61900aedf0 2024-05-08T05:49:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:08.073+0000 7f6194e7e640 1 -- 172.21.15.45:0/3493378516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f619006a8f0 con 0x7f61900aedf0 2024-05-08T05:49:08.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:08.073+0000 7f6186ffd640 1 -- 172.21.15.45:0/3493378516 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+3869 (secure 0 0 0) 0x7f61780814e0 con 0x7f61900aedf0 2024-05-08T05:49:08.074 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:08.074 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":75,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/2616298287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":2616298287},{"type":"v1","addr":"172.21.15.63:6836","nonce":2616298287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6472,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/1347752311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":1347752311},{"type":"v1","addr":"172.21.15.63:6837","nonce":1347752311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69},{"gid":6744,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6834/2854804427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2854804427},{"type":"v1","addr":"172.21.15.45:6834","nonce":2854804427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":7425,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/1879451169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6833","nonce":1879451169},{"type":"v1","addr":"172.21.15.45:6835","nonce":1879451169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[]} 2024-05-08T05:49:08.074 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 75 2024-05-08T05:49:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:08.075+0000 7f6194e7e640 1 -- 172.21.15.45:0/3493378516 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f61700762d0 msgr2=0x7f6170078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:08.075+0000 7f6194e7e640 1 --2- 172.21.15.45:0/3493378516 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f61700762d0 0x7f6170078790 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6180000f60 tx=0x7f6180007ca0 comp rx=0 tx=0).stop 2024-05-08T05:49:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:08.075+0000 7f6194e7e640 1 -- 172.21.15.45:0/3493378516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61900aedf0 msgr2=0x7f619009ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:08.075+0000 7f6194e7e640 1 --2- 172.21.15.45:0/3493378516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61900aedf0 0x7f619009ba40 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f6178000c00 tx=0x7f61780046c0 comp rx=0 tx=0).stop 2024-05-08T05:49:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:08.075+0000 7f6194e7e640 1 -- 172.21.15.45:0/3493378516 shutdown_connections 2024-05-08T05:49:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:08.075+0000 7f6194e7e640 1 --2- 172.21.15.45:0/3493378516 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f61700762d0 0x7f6170078790 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:08.076+0000 7f6194e7e640 1 --2- 172.21.15.45:0/3493378516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f619015d180 0x7f619014b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:08.076+0000 7f6194e7e640 1 --2- 172.21.15.45:0/3493378516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61900aedf0 0x7f619009ba40 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:08.076+0000 7f6194e7e640 1 --2- 172.21.15.45:0/3493378516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f619006c3a0 0x7f619009e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:08.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:08.076+0000 7f6194e7e640 1 -- 172.21.15.45:0/3493378516 >> 172.21.15.45:0/3493378516 conn(0x7f61900b3b10 msgr2=0x7f619015f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:08.076+0000 7f6194e7e640 1 -- 172.21.15.45:0/3493378516 shutdown_connections 2024-05-08T05:49:08.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:08.076+0000 7f6194e7e640 1 -- 172.21.15.45:0/3493378516 wait complete. 2024-05-08T05:49:08.087 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T05:49:08.087 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T05:49:08.087 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T05:49:08.087 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:49:08.088 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T05:49:08.088 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T05:49:08.088 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T05:49:08.088 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:49:08.088 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T05:49:08.088 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T05:49:08.088 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T05:49:08.088 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:49:08.089 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T05:49:08.089 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T05:49:08.089 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T05:49:08.089 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:49:08.098 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T05:49:08.098 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T05:49:08.101 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T05:49:08.101 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T05:49:08.102 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T05:49:08.102 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T05:49:08.141 INFO:tasks.ceph.mds.c:Started 2024-05-08T05:49:08.141 DEBUG:teuthology.parallel:result is None 2024-05-08T05:49:08.143 INFO:tasks.ceph.mds.b:Started 2024-05-08T05:49:08.144 DEBUG:teuthology.parallel:result is None 2024-05-08T05:49:08.144 INFO:tasks.ceph.mds.d:Started 2024-05-08T05:49:08.144 DEBUG:teuthology.parallel:result is None 2024-05-08T05:49:08.216 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-05-08T05:49:08.231 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-08T05:49:08.231 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-08T05:49:11.252 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-05-08T05:49:14.098 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T05:49:14.099 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T05:49:14.102 INFO:tasks.ceph.mds.a:Started 2024-05-08T05:49:14.102 DEBUG:teuthology.parallel:result is None 2024-05-08T05:49:14.102 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T05:49:14.191 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-05-08T05:49:14.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.221+0000 7fbd7f26d640 1 Processor -- start 2024-05-08T05:49:14.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.221+0000 7fbd7f26d640 1 -- start start 2024-05-08T05:49:14.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.222+0000 7fbd7f26d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbd7806e0f0 0x7fbd780cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:14.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.222+0000 7fbd7f26d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd7815d2a0 con 0x7fbd7806f060 2024-05-08T05:49:14.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.222+0000 7fbd7f26d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd781609b0 con 0x7fbd780d0360 2024-05-08T05:49:14.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.222+0000 7fbd7f26d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fbd780d07c0 con 0x7fbd7806e0f0 2024-05-08T05:49:14.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.222+0000 7fbd7c808640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbd7806e0f0 0x7fbd780cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:14.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.222+0000 7fbd7d009640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fbd7806f060 0x7fbd7806fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35460/0 (socket says 172.21.15.45:35460) 2024-05-08T05:49:14.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.222+0000 7fbd7c808640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbd7806e0f0 0x7fbd780cfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43100/0 (socket says 172.21.15.45:43100) 2024-05-08T05:49:14.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.222+0000 7fbd7d009640 1 -- 172.21.15.45:0/21058661 learned_addr learned my addr 172.21.15.45:0/21058661 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:14.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.222+0000 7fbd7c808640 1 -- 172.21.15.45:0/21058661 >> v1:172.21.15.63:6789/0 conn(0x7fbd780d0360 legacy=0x7fbd780b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:14.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.222+0000 7fbd7c808640 1 -- 172.21.15.45:0/21058661 >> v1:172.21.15.45:6789/0 conn(0x7fbd7806f060 legacy=0x7fbd7806fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:14.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.222+0000 7fbd7c808640 1 -- 172.21.15.45:0/21058661 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd78160c90 con 0x7fbd7806e0f0 2024-05-08T05:49:14.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.223+0000 7fbd7c808640 1 --2- 172.21.15.45:0/21058661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd7806e0f0 0x7fbd780cfe20 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fbd680097a0 tx=0x7fbd6802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47f1dc9b900c752 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:14.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.223+0000 7fbd6ffff640 1 -- 172.21.15.45:0/21058661 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68037070 con 0x7fbd7806e0f0 2024-05-08T05:49:14.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.223+0000 7fbd6ffff640 1 -- 172.21.15.45:0/21058661 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd68032440 con 0x7fbd7806e0f0 2024-05-08T05:49:14.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.223+0000 7fbd6ffff640 1 -- 172.21.15.45:0/21058661 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68032810 con 0x7fbd7806e0f0 2024-05-08T05:49:14.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.224+0000 7fbd7f26d640 1 -- 172.21.15.45:0/21058661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd7806e0f0 msgr2=0x7fbd780cfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:14.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.224+0000 7fbd7f26d640 1 --2- 172.21.15.45:0/21058661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd7806e0f0 0x7fbd780cfe20 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fbd680097a0 tx=0x7fbd6802a230 comp rx=0 tx=0).stop 2024-05-08T05:49:14.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.224+0000 7fbd7f26d640 1 -- 172.21.15.45:0/21058661 shutdown_connections 2024-05-08T05:49:14.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.224+0000 7fbd7f26d640 1 --2- 172.21.15.45:0/21058661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd7806e0f0 0x7fbd780cfe20 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:14.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.224+0000 7fbd7f26d640 1 -- 172.21.15.45:0/21058661 >> 172.21.15.45:0/21058661 conn(0x7fbd7806c3a0 msgr2=0x7fbd7806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:14.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.224+0000 7fbd7f26d640 1 -- 172.21.15.45:0/21058661 shutdown_connections 2024-05-08T05:49:14.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.224+0000 7fbd7f26d640 1 -- 172.21.15.45:0/21058661 wait complete. 2024-05-08T05:49:14.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.225+0000 7fbd7f26d640 1 Processor -- start 2024-05-08T05:49:14.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.225+0000 7fbd7f26d640 1 -- start start 2024-05-08T05:49:14.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.226+0000 7fbd7f26d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd7806e0f0 0x7fbd781554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:14.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.226+0000 7fbd7f26d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd7806f060 0x7fbd780b0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:14.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.226+0000 7fbd7f26d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd780d0360 0x7fbd780b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:14.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.226+0000 7fbd7f26d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbd78161250 con 0x7fbd7806e0f0 2024-05-08T05:49:14.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.226+0000 7fbd7f26d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7806f440 con 0x7fbd780d0360 2024-05-08T05:49:14.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.226+0000 7fbd7f26d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbd78161530 con 0x7fbd7806f060 2024-05-08T05:49:14.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.226+0000 7fbd7d009640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd7806e0f0 0x7fbd781554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:14.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.226+0000 7fbd7d009640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd7806e0f0 0x7fbd781554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43114/0 (socket says 172.21.15.45:43114) 2024-05-08T05:49:14.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.226+0000 7fbd7d009640 1 -- 172.21.15.45:0/3761159706 learned_addr learned my addr 172.21.15.45:0/3761159706 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:14.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.226+0000 7fbd7d80a640 1 --2- 172.21.15.45:0/3761159706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd780d0360 0x7fbd780b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:14.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.226+0000 7fbd7c808640 1 --2- 172.21.15.45:0/3761159706 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd7806f060 0x7fbd780b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:14.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.226+0000 7fbd7d009640 1 -- 172.21.15.45:0/3761159706 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd7806f060 msgr2=0x7fbd780b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:14.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.226+0000 7fbd7d009640 1 --2- 172.21.15.45:0/3761159706 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd7806f060 0x7fbd780b0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:14.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.226+0000 7fbd7d009640 1 -- 172.21.15.45:0/3761159706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd780d0360 msgr2=0x7fbd780b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:14.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.226+0000 7fbd7d009640 1 --2- 172.21.15.45:0/3761159706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd780d0360 0x7fbd780b0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:14.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.226+0000 7fbd7d009640 1 -- 172.21.15.45:0/3761159706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd78155d00 con 0x7fbd7806e0f0 2024-05-08T05:49:14.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.227+0000 7fbd7d009640 1 --2- 172.21.15.45:0/3761159706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd7806e0f0 0x7fbd781554b0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fbd7400c500 tx=0x7fbd7400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:14.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.227+0000 7fbd6dffb640 1 -- 172.21.15.45:0/3761159706 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd74017070 con 0x7fbd7806e0f0 2024-05-08T05:49:14.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.227+0000 7fbd6dffb640 1 -- 172.21.15.45:0/3761159706 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd74005ce0 con 0x7fbd7806e0f0 2024-05-08T05:49:14.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.227+0000 7fbd7f26d640 1 -- 172.21.15.45:0/3761159706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7809e100 con 0x7fbd7806e0f0 2024-05-08T05:49:14.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.227+0000 7fbd6dffb640 1 -- 172.21.15.45:0/3761159706 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd740072f0 con 0x7fbd7806e0f0 2024-05-08T05:49:14.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.227+0000 7fbd7f26d640 1 -- 172.21.15.45:0/3761159706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd78156280 con 0x7fbd7806e0f0 2024-05-08T05:49:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.228+0000 7fbd7f26d640 1 -- 172.21.15.45:0/3761159706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd780693f0 con 0x7fbd7806e0f0 2024-05-08T05:49:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.229+0000 7fbd6dffb640 1 -- 172.21.15.45:0/3761159706 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbd7401a070 con 0x7fbd7806e0f0 2024-05-08T05:49:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.229+0000 7fbd6dffb640 1 --2- 172.21.15.45:0/3761159706 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbd5c0762d0 0x7fbd5c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.229+0000 7fbd6dffb640 1 -- 172.21.15.45:0/3761159706 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5507+0+0 (secure 0 0 0) 0x7fbd74096800 con 0x7fbd7806e0f0 2024-05-08T05:49:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.230+0000 7fbd7c808640 1 --2- 172.21.15.45:0/3761159706 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbd5c0762d0 0x7fbd5c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:14.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.230+0000 7fbd7c808640 1 --2- 172.21.15.45:0/3761159706 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbd5c0762d0 0x7fbd5c078790 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbd68034000 tx=0x7fbd6802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:14.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.234+0000 7fbd6dffb640 1 -- 172.21.15.45:0/3761159706 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbd7405fc60 con 0x7fbd7806e0f0 2024-05-08T05:49:14.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:14.437+0000 7fbd7f26d640 1 -- 172.21.15.45:0/3761159706 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fbd7809e500 con 0x7fbd7806e0f0 2024-05-08T05:49:16.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.176+0000 7fbd6dffb640 1 -- 172.21.15.45:0/3761159706 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v100) ==== 97+0+0 (secure 0 0 0) 0x7fbd7809e500 con 0x7fbd7806e0f0 2024-05-08T05:49:16.176 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2024-05-08T05:49:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.179+0000 7fbd7f26d640 1 -- 172.21.15.45:0/3761159706 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbd5c0762d0 msgr2=0x7fbd5c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:16.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.179+0000 7fbd7f26d640 1 --2- 172.21.15.45:0/3761159706 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbd5c0762d0 0x7fbd5c078790 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbd68034000 tx=0x7fbd6802e000 comp rx=0 tx=0).stop 2024-05-08T05:49:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.179+0000 7fbd7f26d640 1 -- 172.21.15.45:0/3761159706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd7806e0f0 msgr2=0x7fbd781554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.179+0000 7fbd7f26d640 1 --2- 172.21.15.45:0/3761159706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd7806e0f0 0x7fbd781554b0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fbd7400c500 tx=0x7fbd7400c9c0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.179+0000 7fbd7f26d640 1 -- 172.21.15.45:0/3761159706 shutdown_connections 2024-05-08T05:49:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.179+0000 7fbd7f26d640 1 --2- 172.21.15.45:0/3761159706 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbd5c0762d0 0x7fbd5c078790 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.179+0000 7fbd7f26d640 1 --2- 172.21.15.45:0/3761159706 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbd7806f060 0x7fbd780b0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.179+0000 7fbd7f26d640 1 --2- 172.21.15.45:0/3761159706 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd780d0360 0x7fbd780b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.179+0000 7fbd7f26d640 1 --2- 172.21.15.45:0/3761159706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd7806e0f0 0x7fbd781554b0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.179+0000 7fbd7f26d640 1 -- 172.21.15.45:0/3761159706 >> 172.21.15.45:0/3761159706 conn(0x7fbd7806c3a0 msgr2=0x7fbd7815f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.179+0000 7fbd7f26d640 1 -- 172.21.15.45:0/3761159706 shutdown_connections 2024-05-08T05:49:16.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.179+0000 7fbd7f26d640 1 -- 172.21.15.45:0/3761159706 wait complete. 2024-05-08T05:49:16.193 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T05:49:16.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.303+0000 7f74ccdd4640 1 Processor -- start 2024-05-08T05:49:16.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.303+0000 7f74ccdd4640 1 -- start start 2024-05-08T05:49:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.304+0000 7f74ccdd4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f74c8070d30 0x7f74c80d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.304+0000 7f74ccdd4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f74c8071110 0x7f74c806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.304+0000 7f74ccdd4640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74c80d1300 con 0x7f74c8163920 2024-05-08T05:49:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.304+0000 7f74ccdd4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f74c80d10d0 con 0x7f74c8071110 2024-05-08T05:49:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.304+0000 7f74ccdd4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f74c8163db0 con 0x7f74c8070d30 2024-05-08T05:49:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.304+0000 7f74c6d9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f74c8071110 0x7f74c806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.304+0000 7f74c6d9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f74c8071110 0x7f74c806d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43128/0 (socket says 172.21.15.45:43128) 2024-05-08T05:49:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.304+0000 7f74c6d9d640 1 -- 172.21.15.45:0/3164795280 learned_addr learned my addr 172.21.15.45:0/3164795280 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.304+0000 7f74c659c640 1 --2- 172.21.15.45:0/3164795280 >> v2:172.21.15.63:3300/0 conn(0x7f74c8070d30 0x7f74c80d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.305+0000 7f74c6d9d640 1 -- 172.21.15.45:0/3164795280 >> v2:172.21.15.63:3300/0 conn(0x7f74c8070d30 msgr2=0x7f74c80d20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.305+0000 7f74c6d9d640 1 --2- 172.21.15.45:0/3164795280 >> v2:172.21.15.63:3300/0 conn(0x7f74c8070d30 0x7f74c80d20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.305+0000 7f74c6d9d640 1 -- 172.21.15.45:0/3164795280 >> v1:172.21.15.63:6789/0 conn(0x7f74c8163920 legacy=0x7f74c80703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:16.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.305+0000 7f74c6d9d640 1 -- 172.21.15.45:0/3164795280 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c80d15e0 con 0x7f74c8071110 2024-05-08T05:49:16.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.305+0000 7f74c6d9d640 1 --2- 172.21.15.45:0/3164795280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74c8071110 0x7f74c806d4b0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f74c80d3f90 tx=0x7f74c002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3aed654483aabf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:16.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.306+0000 7f74c559a640 1 -- 172.21.15.45:0/3164795280 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0036070 con 0x7f74c8071110 2024-05-08T05:49:16.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.306+0000 7f74c559a640 1 -- 172.21.15.45:0/3164795280 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74c0031600 con 0x7f74c8071110 2024-05-08T05:49:16.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.306+0000 7f74c559a640 1 -- 172.21.15.45:0/3164795280 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c00319d0 con 0x7f74c8071110 2024-05-08T05:49:16.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.306+0000 7f74ccdd4640 1 -- 172.21.15.45:0/3164795280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74c8071110 msgr2=0x7f74c806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:16.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.306+0000 7f74ccdd4640 1 --2- 172.21.15.45:0/3164795280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74c8071110 0x7f74c806d4b0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f74c80d3f90 tx=0x7f74c002a730 comp rx=0 tx=0).stop 2024-05-08T05:49:16.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.306+0000 7f74ccdd4640 1 -- 172.21.15.45:0/3164795280 shutdown_connections 2024-05-08T05:49:16.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.306+0000 7f74ccdd4640 1 --2- 172.21.15.45:0/3164795280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74c8071110 0x7f74c806d4b0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.306+0000 7f74ccdd4640 1 --2- 172.21.15.45:0/3164795280 >> v2:172.21.15.63:3300/0 conn(0x7f74c8070d30 0x7f74c80d20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.306+0000 7f74ccdd4640 1 -- 172.21.15.45:0/3164795280 >> 172.21.15.45:0/3164795280 conn(0x7f74c80d2e90 msgr2=0x7f74c80d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:16.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.306+0000 7f74ccdd4640 1 -- 172.21.15.45:0/3164795280 shutdown_connections 2024-05-08T05:49:16.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.306+0000 7f74ccdd4640 1 -- 172.21.15.45:0/3164795280 wait complete. 2024-05-08T05:49:16.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.308+0000 7f74ccdd4640 1 Processor -- start 2024-05-08T05:49:16.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.308+0000 7f74ccdd4640 1 -- start start 2024-05-08T05:49:16.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.308+0000 7f74ccdd4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74c8070d30 0x7f74c81545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:16.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.308+0000 7f74ccdd4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f74c8071110 0x7f74c80cf630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:16.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.308+0000 7f74ccdd4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74c8163920 0x7f74c80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:16.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.309+0000 7f74ccdd4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f74c80d1c30 con 0x7f74c8163920 2024-05-08T05:49:16.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.309+0000 7f74ccdd4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f74c8070b80 con 0x7f74c8070d30 2024-05-08T05:49:16.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.309+0000 7f74ccdd4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f74c8071560 con 0x7f74c8071110 2024-05-08T05:49:16.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.309+0000 7f74c6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74c8163920 0x7f74c80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:16.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.309+0000 7f74c6d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74c8163920 0x7f74c80cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43130/0 (socket says 172.21.15.45:43130) 2024-05-08T05:49:16.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.309+0000 7f74c6d9d640 1 -- 172.21.15.45:0/1438099310 learned_addr learned my addr 172.21.15.45:0/1438099310 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:16.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.309+0000 7f74c659c640 1 --2- 172.21.15.45:0/1438099310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74c8070d30 0x7f74c81545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:16.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.309+0000 7f74c5d9b640 1 --2- 172.21.15.45:0/1438099310 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f74c8071110 0x7f74c80cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:16.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.309+0000 7f74c6d9d640 1 -- 172.21.15.45:0/1438099310 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f74c8071110 msgr2=0x7f74c80cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:16.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.309+0000 7f74c6d9d640 1 --2- 172.21.15.45:0/1438099310 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f74c8071110 0x7f74c80cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.309+0000 7f74c6d9d640 1 -- 172.21.15.45:0/1438099310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74c8070d30 msgr2=0x7f74c81545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:16.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.309+0000 7f74c6d9d640 1 --2- 172.21.15.45:0/1438099310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74c8070d30 0x7f74c81545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.309+0000 7f74c6d9d640 1 -- 172.21.15.45:0/1438099310 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c80c9140 con 0x7f74c8163920 2024-05-08T05:49:16.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.309+0000 7f74c6d9d640 1 --2- 172.21.15.45:0/1438099310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74c8163920 0x7f74c80cfb70 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009f40 tx=0x7f74c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:16.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.309+0000 7f74b6ffd640 1 -- 172.21.15.45:0/1438099310 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c003f020 con 0x7f74c8163920 2024-05-08T05:49:16.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.309+0000 7f74b6ffd640 1 -- 172.21.15.45:0/1438099310 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74c0037040 con 0x7f74c8163920 2024-05-08T05:49:16.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.309+0000 7f74b6ffd640 1 -- 172.21.15.45:0/1438099310 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0036070 con 0x7f74c8163920 2024-05-08T05:49:16.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.309+0000 7f74ccdd4640 1 -- 172.21.15.45:0/1438099310 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c80c9420 con 0x7f74c8163920 2024-05-08T05:49:16.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.310+0000 7f74ccdd4640 1 -- 172.21.15.45:0/1438099310 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c815c110 con 0x7f74c8163920 2024-05-08T05:49:16.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.311+0000 7f74ccdd4640 1 -- 172.21.15.45:0/1438099310 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74c8068c40 con 0x7f74c8163920 2024-05-08T05:49:16.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.311+0000 7f74b6ffd640 1 -- 172.21.15.45:0/1438099310 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f74c0042020 con 0x7f74c8163920 2024-05-08T05:49:16.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.312+0000 7f74b6ffd640 1 --2- 172.21.15.45:0/1438099310 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f74a80762d0 0x7f74a8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:16.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.312+0000 7f74b6ffd640 1 -- 172.21.15.45:0/1438099310 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5249+0+0 (secure 0 0 0) 0x7f74c00b7ba0 con 0x7f74c8163920 2024-05-08T05:49:16.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.312+0000 7f74c659c640 1 --2- 172.21.15.45:0/1438099310 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f74a80762d0 0x7f74a8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:16.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.313+0000 7f74c659c640 1 --2- 172.21.15.45:0/1438099310 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f74a80762d0 0x7f74a8078790 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f74b8006fd0 tx=0x7f74b8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:16.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.316+0000 7f74b6ffd640 1 -- 172.21.15.45:0/1438099310 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f74c8068c40 con 0x7f74c8163920 2024-05-08T05:49:16.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.581+0000 7f74ccdd4640 1 -- 172.21.15.45:0/1438099310 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f74c80693f0 con 0x7f74c8163920 2024-05-08T05:49:16.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.582+0000 7f74b6ffd640 1 -- 172.21.15.45:0/1438099310 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f74c0049020 con 0x7f74c8163920 2024-05-08T05:49:16.582 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:16.582 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:49:16.582 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2024-05-08T05:49:16.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:49:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2024-05-08T05:49:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmrRgXHBAAX1mgdmOD9JC4S1vJNyLSQw==", 2024-05-08T05:49:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:49:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:49:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:49:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:49:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:49:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:49:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:49:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2024-05-08T05:49:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmvn8SKBAAVkXYBbTfu98gl0AptOfJEg==", 2024-05-08T05:49:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:49:16.583 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:49:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:49:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:49:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:49:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:49:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:49:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2024-05-08T05:49:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmeiJCIxAAr3Br1fJVgR9TW2Q+4tMQBQ==", 2024-05-08T05:49:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:49:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:49:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:49:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:49:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:49:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:49:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:49:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2024-05-08T05:49:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmXdFILxAAxteIxUYeFdQKd/oxHLpeOQ==", 2024-05-08T05:49:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:49:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:49:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:49:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:49:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:49:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:49:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:49:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2024-05-08T05:49:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBWEDtmLdHHGRAAA1o4Z/lTGERIZhulqVtybw==", 2024-05-08T05:49:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:49:16.585 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:49:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:49:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:49:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:49:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:49:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2024-05-08T05:49:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBZEDtmMrRVGBAAeKBtcf5fSQMFE6IStUvokQ==", 2024-05-08T05:49:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:49:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:49:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:49:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:49:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:49:16.586 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:49:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2024-05-08T05:49:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBcEDtmpozQFhAAS6AF6X592Wh/QiBipcOVew==", 2024-05-08T05:49:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:49:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:49:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:49:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:49:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:49:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:49:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2024-05-08T05:49:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBfEDtm47lSFRAAt5tvpRCljxGtAmHQqcFj2A==", 2024-05-08T05:49:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:49:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:49:16.587 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:49:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:49:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:49:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:49:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2024-05-08T05:49:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBlEDtmDQ7mGxAA5z7rHZRf7X5g7ZiqEoO2DA==", 2024-05-08T05:49:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:49:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:49:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:49:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:49:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:49:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:49:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2024-05-08T05:49:16.588 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBoEDtmBdi0GhAAVMfg+KEJHRvaiS49PEboOg==", 2024-05-08T05:49:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:49:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:49:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:49:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:49:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:49:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:49:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2024-05-08T05:49:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBrEDtmG2daGRAANQgJFAT6gNE/vPofCv6veA==", 2024-05-08T05:49:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:49:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:49:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:49:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:49:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:49:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:49:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2024-05-08T05:49:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBuEDtm3tmDGRAARsAzm9KIzgKAs4ZtfCVdaQ==", 2024-05-08T05:49:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:49:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:49:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:49:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:49:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:49:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:49:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2024-05-08T05:49:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmDcqGMxAALgQaBYp4Zd8uMqswCXvGaw==", 2024-05-08T05:49:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:49:16.590 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:49:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2024-05-08T05:49:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T05:49:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:49:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:49:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2024-05-08T05:49:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBVEDtm6z0GARAARgRma5hPpH6X95pE5NNrQw==", 2024-05-08T05:49:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:49:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:49:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:49:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:49:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:49:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:49:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:49:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2024-05-08T05:49:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmAMtIORAAJf8oSwHqSzgjsTuqxtQqHQ==", 2024-05-08T05:49:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:49:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:49:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:49:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:49:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:49:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:49:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:49:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2024-05-08T05:49:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBVEDtm9qK1BhAAokvkJZmNxT5WER9/+EKIpA==", 2024-05-08T05:49:16.592 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:49:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:49:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:49:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:49:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:49:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:49:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:49:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2024-05-08T05:49:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmZ4Q3ABAANcqmwOmt+dNJ5wOiPHHO4Q==", 2024-05-08T05:49:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:49:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:49:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:49:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:49:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:49:16.593 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:49:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:49:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2024-05-08T05:49:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmL0XLGBAA1BSFdyiDYV9Je+igOVBvlQ==", 2024-05-08T05:49:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:49:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:49:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-05-08T05:49:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:49:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:49:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:49:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2024-05-08T05:49:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmfM5OFRAAgdJPUfqmnKU8vRrjGMNwrw==", 2024-05-08T05:49:16.594 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:49:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:49:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-05-08T05:49:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:49:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.595 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-05-08T05:49:16.595 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:49:16.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.584+0000 7f74ccdd4640 1 -- 172.21.15.45:0/1438099310 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f74a80762d0 msgr2=0x7f74a8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:16.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.584+0000 7f74ccdd4640 1 --2- 172.21.15.45:0/1438099310 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f74a80762d0 0x7f74a8078790 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f74b8006fd0 tx=0x7f74b8008040 comp rx=0 tx=0).stop 2024-05-08T05:49:16.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.584+0000 7f74ccdd4640 1 -- 172.21.15.45:0/1438099310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74c8163920 msgr2=0x7f74c80cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:16.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.584+0000 7f74ccdd4640 1 --2- 172.21.15.45:0/1438099310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74c8163920 0x7f74c80cfb70 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009f40 tx=0x7f74c0004a40 comp rx=0 tx=0).stop 2024-05-08T05:49:16.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.584+0000 7f74ccdd4640 1 -- 172.21.15.45:0/1438099310 shutdown_connections 2024-05-08T05:49:16.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.584+0000 7f74ccdd4640 1 --2- 172.21.15.45:0/1438099310 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f74a80762d0 0x7f74a8078790 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.584+0000 7f74ccdd4640 1 --2- 172.21.15.45:0/1438099310 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f74c8071110 0x7f74c80cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.584+0000 7f74ccdd4640 1 --2- 172.21.15.45:0/1438099310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74c8163920 0x7f74c80cfb70 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.584+0000 7f74ccdd4640 1 --2- 172.21.15.45:0/1438099310 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74c8070d30 0x7f74c81545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.584+0000 7f74ccdd4640 1 -- 172.21.15.45:0/1438099310 >> 172.21.15.45:0/1438099310 conn(0x7f74c80d2e90 msgr2=0x7f74c80b4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:16.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.584+0000 7f74ccdd4640 1 -- 172.21.15.45:0/1438099310 shutdown_connections 2024-05-08T05:49:16.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.584+0000 7f74ccdd4640 1 -- 172.21.15.45:0/1438099310 wait complete. 2024-05-08T05:49:16.598 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T05:49:16.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.701+0000 7f58d3009640 1 Processor -- start 2024-05-08T05:49:16.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.701+0000 7f58d3009640 1 -- start start 2024-05-08T05:49:16.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.702+0000 7f58d3009640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc1629b0 0x7f58cc06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:16.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.702+0000 7f58d3009640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f58cc0b2490 0x7f58cc070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.702+0000 7f58d3009640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58cc06e110 con 0x7f58cc070d20 2024-05-08T05:49:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.702+0000 7f58d3009640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f58cc06dec0 con 0x7f58cc0b2490 2024-05-08T05:49:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.702+0000 7f58d3009640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f58cc06e3f0 con 0x7f58cc1629b0 2024-05-08T05:49:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.702+0000 7f58d15a6640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f58cc070d20 0x7f58cc0afcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35472/0 (socket says 172.21.15.45:35472) 2024-05-08T05:49:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.702+0000 7f58d15a6640 1 -- 172.21.15.45:0/986010474 learned_addr learned my addr 172.21.15.45:0/986010474 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.703+0000 7f58cbfff640 1 --2- 172.21.15.45:0/986010474 >> v2:172.21.15.63:3300/0 conn(0x7f58cc0b2490 0x7f58cc070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.703+0000 7f58d0da5640 1 --2- 172.21.15.45:0/986010474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc1629b0 0x7f58cc06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.703+0000 7f58cb7fe640 1 -- 172.21.15.45:0/986010474 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2769316445 0 0) 0x7f58cc06e110 con 0x7f58cc070d20 2024-05-08T05:49:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.703+0000 7f58cb7fe640 1 -- 172.21.15.45:0/986010474 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58b4003520 con 0x7f58cc070d20 2024-05-08T05:49:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.703+0000 7f58d0da5640 1 -- 172.21.15.45:0/986010474 >> v2:172.21.15.63:3300/0 conn(0x7f58cc0b2490 msgr2=0x7f58cc070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.703+0000 7f58d0da5640 1 --2- 172.21.15.45:0/986010474 >> v2:172.21.15.63:3300/0 conn(0x7f58cc0b2490 0x7f58cc070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.703+0000 7f58d0da5640 1 -- 172.21.15.45:0/986010474 >> v1:172.21.15.45:6789/0 conn(0x7f58cc070d20 legacy=0x7f58cc0afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:16.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.703+0000 7f58d0da5640 1 -- 172.21.15.45:0/986010474 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc06e6d0 con 0x7f58cc1629b0 2024-05-08T05:49:16.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.704+0000 7f58d0da5640 1 --2- 172.21.15.45:0/986010474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc1629b0 0x7f58cc06eed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0097a0 tx=0x7f58bc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2c989568f8c5f35 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.704+0000 7f58cb7fe640 1 -- 172.21.15.45:0/986010474 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc037070 con 0x7f58cc1629b0 2024-05-08T05:49:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.704+0000 7f58cb7fe640 1 -- 172.21.15.45:0/986010474 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58bc032440 con 0x7f58cc1629b0 2024-05-08T05:49:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.704+0000 7f58cb7fe640 1 -- 172.21.15.45:0/986010474 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc032810 con 0x7f58cc1629b0 2024-05-08T05:49:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.704+0000 7f58d3009640 1 -- 172.21.15.45:0/986010474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc1629b0 msgr2=0x7f58cc06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.704+0000 7f58d3009640 1 --2- 172.21.15.45:0/986010474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc1629b0 0x7f58cc06eed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0097a0 tx=0x7f58bc02a180 comp rx=0 tx=0).stop 2024-05-08T05:49:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.704+0000 7f58d3009640 1 -- 172.21.15.45:0/986010474 shutdown_connections 2024-05-08T05:49:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.704+0000 7f58d3009640 1 --2- 172.21.15.45:0/986010474 >> v2:172.21.15.63:3300/0 conn(0x7f58cc0b2490 0x7f58cc070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.704+0000 7f58d3009640 1 --2- 172.21.15.45:0/986010474 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc1629b0 0x7f58cc06eed0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.705+0000 7f58d3009640 1 -- 172.21.15.45:0/986010474 >> 172.21.15.45:0/986010474 conn(0x7f58cc0b49d0 msgr2=0x7f58cc0b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.705+0000 7f58d3009640 1 -- 172.21.15.45:0/986010474 shutdown_connections 2024-05-08T05:49:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.705+0000 7f58d3009640 1 -- 172.21.15.45:0/986010474 wait complete. 2024-05-08T05:49:16.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.706+0000 7f58d3009640 1 Processor -- start 2024-05-08T05:49:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.706+0000 7f58d3009640 1 -- start start 2024-05-08T05:49:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58d3009640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58cc070d20 0x7f58cc157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58d3009640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58cc0b2490 0x7f58cc0cd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58d3009640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc1629b0 0x7f58cc0cef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58d3009640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc070870 con 0x7f58cc0b2490 2024-05-08T05:49:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58d3009640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc06f410 con 0x7f58cc070d20 2024-05-08T05:49:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58d3009640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f58cc0709f0 con 0x7f58cc1629b0 2024-05-08T05:49:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58cbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58cc0b2490 0x7f58cc0cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58cbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58cc0b2490 0x7f58cc0cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43134/0 (socket says 172.21.15.45:43134) 2024-05-08T05:49:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58cbfff640 1 -- 172.21.15.45:0/4219313893 learned_addr learned my addr 172.21.15.45:0/4219313893 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58d0da5640 1 --2- 172.21.15.45:0/4219313893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58cc070d20 0x7f58cc157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58d15a6640 1 --2- 172.21.15.45:0/4219313893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc1629b0 0x7f58cc0cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58cbfff640 1 -- 172.21.15.45:0/4219313893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc1629b0 msgr2=0x7f58cc0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58cbfff640 1 --2- 172.21.15.45:0/4219313893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc1629b0 0x7f58cc0cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58cbfff640 1 -- 172.21.15.45:0/4219313893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58cc070d20 msgr2=0x7f58cc157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58cbfff640 1 --2- 172.21.15.45:0/4219313893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58cc070d20 0x7f58cc157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58cbfff640 1 -- 172.21.15.45:0/4219313893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc1577d0 con 0x7f58cc0b2490 2024-05-08T05:49:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58cbfff640 1 --2- 172.21.15.45:0/4219313893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58cc0b2490 0x7f58cc0cd470 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f58c000e9b0 tx=0x7f58c000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58c97fa640 1 -- 172.21.15.45:0/4219313893 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c0017070 con 0x7f58cc0b2490 2024-05-08T05:49:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58c97fa640 1 -- 172.21.15.45:0/4219313893 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c000c9a0 con 0x7f58cc0b2490 2024-05-08T05:49:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58d3009640 1 -- 172.21.15.45:0/4219313893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58cc0b11c0 con 0x7f58cc0b2490 2024-05-08T05:49:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.707+0000 7f58c97fa640 1 -- 172.21.15.45:0/4219313893 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c000cd90 con 0x7f58cc0b2490 2024-05-08T05:49:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.708+0000 7f58d3009640 1 -- 172.21.15.45:0/4219313893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58cc0c6c10 con 0x7f58cc0b2490 2024-05-08T05:49:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.709+0000 7f58d3009640 1 -- 172.21.15.45:0/4219313893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58cc0693f0 con 0x7f58cc0b2490 2024-05-08T05:49:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.709+0000 7f58c97fa640 1 -- 172.21.15.45:0/4219313893 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f58c001a070 con 0x7f58cc0b2490 2024-05-08T05:49:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.710+0000 7f58c97fa640 1 --2- 172.21.15.45:0/4219313893 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f58b80762d0 0x7f58b8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.710+0000 7f58c97fa640 1 -- 172.21.15.45:0/4219313893 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5249+0+0 (secure 0 0 0) 0x7f58c00973e0 con 0x7f58cc0b2490 2024-05-08T05:49:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.710+0000 7f58d0da5640 1 --2- 172.21.15.45:0/4219313893 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f58b80762d0 0x7f58b8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:16.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.711+0000 7f58d0da5640 1 --2- 172.21.15.45:0/4219313893 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f58b80762d0 0x7f58b8078790 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f58bc034000 tx=0x7f58bc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:16.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.714+0000 7f58c97fa640 1 -- 172.21.15.45:0/4219313893 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f58c00608a0 con 0x7f58cc0b2490 2024-05-08T05:49:16.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.907+0000 7f58d3009640 1 -- 172.21.15.45:0/4219313893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f58cc0c69c0 con 0x7f58cc0b2490 2024-05-08T05:49:16.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.908+0000 7f58c97fa640 1 -- 172.21.15.45:0/4219313893 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v100) ==== 84+0+65 (secure 0 0 0) 0x7f58c0064ba0 con 0x7f58cc0b2490 2024-05-08T05:49:16.908 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:16.908 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-05-08T05:49:16.908 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:49:16.908 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-05-08T05:49:16.908 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-05-08T05:49:16.908 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:16.908 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-05-08T05:49:16.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.910+0000 7f58d3009640 1 -- 172.21.15.45:0/4219313893 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f58b80762d0 msgr2=0x7f58b8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:16.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.910+0000 7f58d3009640 1 --2- 172.21.15.45:0/4219313893 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f58b80762d0 0x7f58b8078790 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f58bc034000 tx=0x7f58bc02e000 comp rx=0 tx=0).stop 2024-05-08T05:49:16.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.910+0000 7f58d3009640 1 -- 172.21.15.45:0/4219313893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58cc0b2490 msgr2=0x7f58cc0cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:16.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.910+0000 7f58d3009640 1 --2- 172.21.15.45:0/4219313893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58cc0b2490 0x7f58cc0cd470 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f58c000e9b0 tx=0x7f58c000ee70 comp rx=0 tx=0).stop 2024-05-08T05:49:16.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.910+0000 7f58d3009640 1 -- 172.21.15.45:0/4219313893 shutdown_connections 2024-05-08T05:49:16.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.910+0000 7f58d3009640 1 --2- 172.21.15.45:0/4219313893 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f58b80762d0 0x7f58b8078790 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.910+0000 7f58d3009640 1 --2- 172.21.15.45:0/4219313893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f58cc1629b0 0x7f58cc0cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.910+0000 7f58d3009640 1 --2- 172.21.15.45:0/4219313893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f58cc0b2490 0x7f58cc0cd470 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.910+0000 7f58d3009640 1 --2- 172.21.15.45:0/4219313893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58cc070d20 0x7f58cc157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:16.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.910+0000 7f58d3009640 1 -- 172.21.15.45:0/4219313893 >> 172.21.15.45:0/4219313893 conn(0x7f58cc0b49d0 msgr2=0x7f58cc0d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:16.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.910+0000 7f58d3009640 1 -- 172.21.15.45:0/4219313893 shutdown_connections 2024-05-08T05:49:16.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:16.910+0000 7f58d3009640 1 -- 172.21.15.45:0/4219313893 wait complete. 2024-05-08T05:49:16.921 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T05:49:16.921 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T05:49:17.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.028+0000 7f84f766d640 1 Processor -- start 2024-05-08T05:49:17.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.029+0000 7f84f766d640 1 -- start start 2024-05-08T05:49:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.029+0000 7f84f766d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f84f0162b70 0x7f84f00d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.029+0000 7f84f766d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84f0162f50 con 0x7f84f00d3910 2024-05-08T05:49:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.029+0000 7f84f766d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84f00d3ee0 con 0x7f84f00d3480 2024-05-08T05:49:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.029+0000 7f84f766d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f84f00d1f60 con 0x7f84f0162b70 2024-05-08T05:49:17.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84f5c0a640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f84f00d3910 0x7f84f00b5150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35480/0 (socket says 172.21.15.45:35480) 2024-05-08T05:49:17.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84f5c0a640 1 -- 172.21.15.45:0/2903693152 learned_addr learned my addr 172.21.15.45:0/2903693152 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:17.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84f5409640 1 --2- 172.21.15.45:0/2903693152 >> v2:172.21.15.63:3300/0 conn(0x7f84f0162b70 0x7f84f00d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:17.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84effff640 1 -- 172.21.15.45:0/2903693152 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2320931468 0 0) 0x7f84e0001490 con 0x7f84f00d3910 2024-05-08T05:49:17.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84effff640 1 -- 172.21.15.45:0/2903693152 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84d8003520 con 0x7f84f00d3910 2024-05-08T05:49:17.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84effff640 1 -- 172.21.15.45:0/2903693152 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1463960109 0 0) 0x7f84d8003520 con 0x7f84f00d3910 2024-05-08T05:49:17.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84effff640 1 -- 172.21.15.45:0/2903693152 >> v2:172.21.15.63:3300/0 conn(0x7f84f0162b70 msgr2=0x7f84f00d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:17.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84effff640 1 --2- 172.21.15.45:0/2903693152 >> v2:172.21.15.63:3300/0 conn(0x7f84f0162b70 0x7f84f00d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:17.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84effff640 1 -- 172.21.15.45:0/2903693152 >> v1:172.21.15.63:6789/0 conn(0x7f84f00d3480 legacy=0x7f84f00b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:17.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84effff640 1 -- 172.21.15.45:0/2903693152 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f00d21b0 con 0x7f84f00d3910 2024-05-08T05:49:17.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84effff640 1 -- 172.21.15.45:0/2903693152 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f84e0001c40 con 0x7f84f00d3910 2024-05-08T05:49:17.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84effff640 1 -- 172.21.15.45:0/2903693152 >> v1:172.21.15.45:6789/0 conn(0x7f84f00d3910 legacy=0x7f84f00b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:17.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84effff640 1 --2- 172.21.15.45:0/2903693152 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84d8003a00 0x7f84d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:17.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84effff640 1 --2- 172.21.15.45:0/2903693152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84d8004800 0x7f84d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:17.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84effff640 1 --2- 172.21.15.45:0/2903693152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84d80051c0 0x7f84d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:17.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84effff640 1 -- 172.21.15.45:0/2903693152 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f84f00d1f60 con 0x7f84d8004800 2024-05-08T05:49:17.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84effff640 1 -- 172.21.15.45:0/2903693152 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f84d800c060 con 0x7f84d80051c0 2024-05-08T05:49:17.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84effff640 1 -- 172.21.15.45:0/2903693152 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f84d800c260 con 0x7f84d8003a00 2024-05-08T05:49:17.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84f4c08640 1 --2- 172.21.15.45:0/2903693152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84d8004800 0x7f84d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:17.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84f5409640 1 --2- 172.21.15.45:0/2903693152 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84d8003a00 0x7f84d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:17.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.030+0000 7f84f5c0a640 1 --2- 172.21.15.45:0/2903693152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84d80051c0 0x7f84d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:17.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.031+0000 7f84f4c08640 1 -- 172.21.15.45:0/2903693152 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84d8003a00 msgr2=0x7f84d8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:17.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.031+0000 7f84f4c08640 1 --2- 172.21.15.45:0/2903693152 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84d8003a00 0x7f84d8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:17.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.031+0000 7f84f4c08640 1 -- 172.21.15.45:0/2903693152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84d80051c0 msgr2=0x7f84d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:17.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.031+0000 7f84f4c08640 1 --2- 172.21.15.45:0/2903693152 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84d80051c0 0x7f84d800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:17.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.031+0000 7f84f4c08640 1 -- 172.21.15.45:0/2903693152 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d8003520 con 0x7f84d8004800 2024-05-08T05:49:17.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.031+0000 7f84f5c0a640 1 -- 172.21.15.45:0/2903693152 reap_dead start 2024-05-08T05:49:17.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.031+0000 7f84f4c08640 1 --2- 172.21.15.45:0/2903693152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84d8004800 0x7f84d8004c80 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f84e4001810 tx=0x7f84d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847b5f81ed8b335e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:17.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.031+0000 7f84effff640 1 -- 172.21.15.45:0/2903693152 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4037070 con 0x7f84d8004800 2024-05-08T05:49:17.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.031+0000 7f84effff640 1 -- 172.21.15.45:0/2903693152 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e402ac30 con 0x7f84d8004800 2024-05-08T05:49:17.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.031+0000 7f84effff640 1 -- 172.21.15.45:0/2903693152 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4001430 con 0x7f84d8004800 2024-05-08T05:49:17.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.032+0000 7f84f766d640 1 -- 172.21.15.45:0/2903693152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84d8004800 msgr2=0x7f84d8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:17.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.032+0000 7f84f766d640 1 --2- 172.21.15.45:0/2903693152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84d8004800 0x7f84d8004c80 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f84e4001810 tx=0x7f84d8005830 comp rx=0 tx=0).stop 2024-05-08T05:49:17.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.032+0000 7f84f766d640 1 -- 172.21.15.45:0/2903693152 shutdown_connections 2024-05-08T05:49:17.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.032+0000 7f84f766d640 1 --2- 172.21.15.45:0/2903693152 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84d8004800 0x7f84d8004c80 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:17.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.032+0000 7f84f766d640 1 -- 172.21.15.45:0/2903693152 >> 172.21.15.45:0/2903693152 conn(0x7f84f00d5040 msgr2=0x7f84f00d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:17.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.032+0000 7f84f766d640 1 -- 172.21.15.45:0/2903693152 shutdown_connections 2024-05-08T05:49:17.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.032+0000 7f84f766d640 1 -- 172.21.15.45:0/2903693152 wait complete. 2024-05-08T05:49:17.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.033+0000 7f84f766d640 1 Processor -- start 2024-05-08T05:49:17.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.033+0000 7f84f766d640 1 -- start start 2024-05-08T05:49:17.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.034+0000 7f84f766d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84d8004800 0x7f84f0151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:17.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.034+0000 7f84f766d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84f00b8d70 0x7f84f006cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:17.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.034+0000 7f84f766d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84f006d480 0x7f84f00b9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:17.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.034+0000 7f84f766d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f84f00b42f0 con 0x7f84f006d480 2024-05-08T05:49:17.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.034+0000 7f84f766d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f84f00b4170 con 0x7f84d8004800 2024-05-08T05:49:17.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.034+0000 7f84f766d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f84f00d2f40 con 0x7f84f00b8d70 2024-05-08T05:49:17.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.034+0000 7f84f5c0a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84f006d480 0x7f84f00b9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:17.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.034+0000 7f84f5c0a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84f006d480 0x7f84f00b9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43148/0 (socket says 172.21.15.45:43148) 2024-05-08T05:49:17.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.034+0000 7f84f5c0a640 1 -- 172.21.15.45:0/1251524354 learned_addr learned my addr 172.21.15.45:0/1251524354 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:17.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.035+0000 7f84f5c0a640 1 -- 172.21.15.45:0/1251524354 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84f00b8d70 msgr2=0x7f84f006cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:17.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.035+0000 7f84f5409640 1 --2- 172.21.15.45:0/1251524354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84d8004800 0x7f84f0151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:17.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.035+0000 7f84f5c0a640 1 --2- 172.21.15.45:0/1251524354 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84f00b8d70 0x7f84f006cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:17.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.035+0000 7f84f5c0a640 1 -- 172.21.15.45:0/1251524354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84d8004800 msgr2=0x7f84f0151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:17.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.035+0000 7f84f5c0a640 1 --2- 172.21.15.45:0/1251524354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84d8004800 0x7f84f0151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:17.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.035+0000 7f84f5c0a640 1 -- 172.21.15.45:0/1251524354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f00b7cd0 con 0x7f84f006d480 2024-05-08T05:49:17.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.035+0000 7f84f5c0a640 1 --2- 172.21.15.45:0/1251524354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84f006d480 0x7f84f00b9ab0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f84f006f400 tx=0x7f84e000af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:17.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.035+0000 7f84edffb640 1 -- 172.21.15.45:0/1251524354 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0007800 con 0x7f84f006d480 2024-05-08T05:49:17.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.035+0000 7f84edffb640 1 -- 172.21.15.45:0/1251524354 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e0018070 con 0x7f84f006d480 2024-05-08T05:49:17.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.035+0000 7f84f766d640 1 -- 172.21.15.45:0/1251524354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f00b7fb0 con 0x7f84f006d480 2024-05-08T05:49:17.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.035+0000 7f84edffb640 1 -- 172.21.15.45:0/1251524354 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e00133f0 con 0x7f84f006d480 2024-05-08T05:49:17.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.035+0000 7f84f766d640 1 -- 172.21.15.45:0/1251524354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f00b8570 con 0x7f84f006d480 2024-05-08T05:49:17.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.036+0000 7f84f766d640 1 -- 172.21.15.45:0/1251524354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f00693f0 con 0x7f84f006d480 2024-05-08T05:49:17.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.037+0000 7f84edffb640 1 -- 172.21.15.45:0/1251524354 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f84e0013590 con 0x7f84f006d480 2024-05-08T05:49:17.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.037+0000 7f84edffb640 1 --2- 172.21.15.45:0/1251524354 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f84dc0762d0 0x7f84dc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:17.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.037+0000 7f84edffb640 1 -- 172.21.15.45:0/1251524354 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5249+0+0 (secure 0 0 0) 0x7f84e0097980 con 0x7f84f006d480 2024-05-08T05:49:17.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.038+0000 7f84f5409640 1 --2- 172.21.15.45:0/1251524354 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f84dc0762d0 0x7f84dc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:17.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.038+0000 7f84f5409640 1 --2- 172.21.15.45:0/1251524354 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f84dc0762d0 0x7f84dc078790 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f84e8009a00 tx=0x7f84e8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:17.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.041+0000 7f84edffb640 1 -- 172.21.15.45:0/1251524354 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f84f0162f50 con 0x7f84f006d480 2024-05-08T05:49:17.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:17.236+0000 7f84f766d640 1 -- 172.21.15.45:0/1251524354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f84f00b5bc0 con 0x7f84f006d480 2024-05-08T05:49:17.965 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-05-08T05:49:18.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.195+0000 7f84edffb640 1 -- 172.21.15.45:0/1251524354 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v101) ==== 138+0+0 (secure 0 0 0) 0x7f84e0061530 con 0x7f84f006d480 2024-05-08T05:49:18.195 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-05-08T05:49:18.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.198+0000 7f84f766d640 1 -- 172.21.15.45:0/1251524354 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f84dc0762d0 msgr2=0x7f84dc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:18.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.198+0000 7f84f766d640 1 --2- 172.21.15.45:0/1251524354 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f84dc0762d0 0x7f84dc078790 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f84e8009a00 tx=0x7f84e8008040 comp rx=0 tx=0).stop 2024-05-08T05:49:18.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.198+0000 7f84f766d640 1 -- 172.21.15.45:0/1251524354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84f006d480 msgr2=0x7f84f00b9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:18.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.198+0000 7f84f766d640 1 --2- 172.21.15.45:0/1251524354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84f006d480 0x7f84f00b9ab0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f84f006f400 tx=0x7f84e000af60 comp rx=0 tx=0).stop 2024-05-08T05:49:18.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.199+0000 7f84f766d640 1 -- 172.21.15.45:0/1251524354 shutdown_connections 2024-05-08T05:49:18.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.199+0000 7f84f766d640 1 --2- 172.21.15.45:0/1251524354 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f84dc0762d0 0x7f84dc078790 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:18.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.199+0000 7f84f766d640 1 --2- 172.21.15.45:0/1251524354 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f84f00b8d70 0x7f84f006cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:18.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.199+0000 7f84f766d640 1 --2- 172.21.15.45:0/1251524354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f84f006d480 0x7f84f00b9ab0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:18.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.199+0000 7f84f766d640 1 --2- 172.21.15.45:0/1251524354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84d8004800 0x7f84f0151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:18.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.199+0000 7f84f766d640 1 -- 172.21.15.45:0/1251524354 >> 172.21.15.45:0/1251524354 conn(0x7f84f00d5040 msgr2=0x7f84f015e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:18.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.199+0000 7f84f766d640 1 -- 172.21.15.45:0/1251524354 shutdown_connections 2024-05-08T05:49:18.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.199+0000 7f84f766d640 1 -- 172.21.15.45:0/1251524354 wait complete. 2024-05-08T05:49:18.216 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T05:49:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.333+0000 7fb14745c640 1 Processor -- start 2024-05-08T05:49:18.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.333+0000 7fb14745c640 1 -- start start 2024-05-08T05:49:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.334+0000 7fb14745c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb140163460 0x7fb1400d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.334+0000 7fb14745c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1400d2bb0 0x7fb14015ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.334+0000 7fb14745c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1400d3320 con 0x7fb1400b6590 2024-05-08T05:49:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.334+0000 7fb14745c640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fb1400d3600 con 0x7fb140163460 2024-05-08T05:49:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.334+0000 7fb14745c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb1400d38e0 con 0x7fb1400d2bb0 2024-05-08T05:49:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.334+0000 7fb1459f9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1400d2bb0 0x7fb14015ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:18.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.334+0000 7fb1449f7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fb140163460 0x7fb1400d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.334+0000 7fb1459f9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1400d2bb0 0x7fb14015ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:39968/0 (socket says 172.21.15.45:39968) 2024-05-08T05:49:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.334+0000 7fb1451f8640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fb1400b6590 0x7fb140162d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:36386/0 (socket says 172.21.15.45:36386) 2024-05-08T05:49:18.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.334+0000 7fb1459f9640 1 -- 172.21.15.45:0/2089089785 learned_addr learned my addr 172.21.15.45:0/2089089785 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.335+0000 7fb1459f9640 1 -- 172.21.15.45:0/2089089785 >> v2:172.21.15.63:3300/0 conn(0x7fb140163460 msgr2=0x7fb1400d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.335+0000 7fb1459f9640 1 --2- 172.21.15.45:0/2089089785 >> v2:172.21.15.63:3300/0 conn(0x7fb140163460 0x7fb1400d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.335+0000 7fb1459f9640 1 -- 172.21.15.45:0/2089089785 >> v1:172.21.15.63:6789/0 conn(0x7fb1400b6590 legacy=0x7fb140162d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.335+0000 7fb1459f9640 1 -- 172.21.15.45:0/2089089785 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1400d3bc0 con 0x7fb1400d2bb0 2024-05-08T05:49:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.335+0000 7fb1449f7640 1 --2- 172.21.15.45:0/2089089785 >> v2:172.21.15.63:3300/0 conn(0x7fb140163460 0x7fb1400d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:49:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.335+0000 7fb1459f9640 1 --2- 172.21.15.45:0/2089089785 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1400d2bb0 0x7fb14015ec30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb138009e50 tx=0x7fb13802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=172e26ecd142e0f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.336+0000 7fb137fff640 1 -- 172.21.15.45:0/2089089785 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb138036070 con 0x7fb1400d2bb0 2024-05-08T05:49:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.336+0000 7fb137fff640 1 -- 172.21.15.45:0/2089089785 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb138031600 con 0x7fb1400d2bb0 2024-05-08T05:49:18.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.336+0000 7fb137fff640 1 -- 172.21.15.45:0/2089089785 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1380319d0 con 0x7fb1400d2bb0 2024-05-08T05:49:18.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.336+0000 7fb14745c640 1 -- 172.21.15.45:0/2089089785 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1400d2bb0 msgr2=0x7fb14015ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:18.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.336+0000 7fb14745c640 1 --2- 172.21.15.45:0/2089089785 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1400d2bb0 0x7fb14015ec30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb138009e50 tx=0x7fb13802a710 comp rx=0 tx=0).stop 2024-05-08T05:49:18.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.336+0000 7fb14745c640 1 -- 172.21.15.45:0/2089089785 shutdown_connections 2024-05-08T05:49:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.336+0000 7fb14745c640 1 --2- 172.21.15.45:0/2089089785 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb1400d2bb0 0x7fb14015ec30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.336+0000 7fb14745c640 1 --2- 172.21.15.45:0/2089089785 >> v2:172.21.15.63:3300/0 conn(0x7fb140163460 0x7fb1400d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.336+0000 7fb14745c640 1 -- 172.21.15.45:0/2089089785 >> 172.21.15.45:0/2089089785 conn(0x7fb1400d4fe0 msgr2=0x7fb1400d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.336+0000 7fb14745c640 1 -- 172.21.15.45:0/2089089785 shutdown_connections 2024-05-08T05:49:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.336+0000 7fb14745c640 1 -- 172.21.15.45:0/2089089785 wait complete. 2024-05-08T05:49:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.338+0000 7fb14745c640 1 Processor -- start 2024-05-08T05:49:18.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.338+0000 7fb14745c640 1 -- start start 2024-05-08T05:49:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.338+0000 7fb14745c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1400b6590 0x7fb14015fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.338+0000 7fb14745c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1400d2bb0 0x7fb14014b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.338+0000 7fb14745c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb140163460 0x7fb140162130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.338+0000 7fb14745c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb14015f170 con 0x7fb1400d2bb0 2024-05-08T05:49:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.338+0000 7fb14745c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb1400d3040 con 0x7fb1400b6590 2024-05-08T05:49:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.338+0000 7fb14745c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb14015f2f0 con 0x7fb140163460 2024-05-08T05:49:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.338+0000 7fb1449f7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1400d2bb0 0x7fb14014b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.339+0000 7fb1449f7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1400d2bb0 0x7fb14014b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43154/0 (socket says 172.21.15.45:43154) 2024-05-08T05:49:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.339+0000 7fb1449f7640 1 -- 172.21.15.45:0/3340723831 learned_addr learned my addr 172.21.15.45:0/3340723831 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.339+0000 7fb1459f9640 1 --2- 172.21.15.45:0/3340723831 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb140163460 0x7fb140162130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.339+0000 7fb1451f8640 1 --2- 172.21.15.45:0/3340723831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1400b6590 0x7fb14015fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.339+0000 7fb1449f7640 1 -- 172.21.15.45:0/3340723831 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb140163460 msgr2=0x7fb140162130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:18.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.339+0000 7fb1449f7640 1 --2- 172.21.15.45:0/3340723831 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb140163460 0x7fb140162130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.339+0000 7fb1449f7640 1 -- 172.21.15.45:0/3340723831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1400b6590 msgr2=0x7fb14015fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.339+0000 7fb1449f7640 1 --2- 172.21.15.45:0/3340723831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1400b6590 0x7fb14015fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.339+0000 7fb1449f7640 1 -- 172.21.15.45:0/3340723831 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14014b920 con 0x7fb1400d2bb0 2024-05-08T05:49:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.339+0000 7fb1449f7640 1 --2- 172.21.15.45:0/3340723831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1400d2bb0 0x7fb14014b3e0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fb13000e990 tx=0x7fb13000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.339+0000 7fb135ffb640 1 -- 172.21.15.45:0/3340723831 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb130017070 con 0x7fb1400d2bb0 2024-05-08T05:49:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.339+0000 7fb135ffb640 1 -- 172.21.15.45:0/3340723831 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb13000c9e0 con 0x7fb1400d2bb0 2024-05-08T05:49:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.339+0000 7fb135ffb640 1 -- 172.21.15.45:0/3340723831 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13000cdb0 con 0x7fb1400d2bb0 2024-05-08T05:49:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.339+0000 7fb14745c640 1 -- 172.21.15.45:0/3340723831 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb140162960 con 0x7fb1400d2bb0 2024-05-08T05:49:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.340+0000 7fb14745c640 1 -- 172.21.15.45:0/3340723831 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1400ca0d0 con 0x7fb1400d2bb0 2024-05-08T05:49:18.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.340+0000 7fb14745c640 1 -- 172.21.15.45:0/3340723831 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1400693f0 con 0x7fb1400d2bb0 2024-05-08T05:49:18.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.341+0000 7fb135ffb640 1 -- 172.21.15.45:0/3340723831 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb13001a070 con 0x7fb1400d2bb0 2024-05-08T05:49:18.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.342+0000 7fb135ffb640 1 --2- 172.21.15.45:0/3340723831 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb1240762d0 0x7fb124078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:18.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.342+0000 7fb135ffb640 1 -- 172.21.15.45:0/3340723831 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5619+0+0 (secure 0 0 0) 0x7fb130097490 con 0x7fb1400d2bb0 2024-05-08T05:49:18.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.342+0000 7fb1451f8640 1 --2- 172.21.15.45:0/3340723831 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb1240762d0 0x7fb124078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:18.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.343+0000 7fb1451f8640 1 --2- 172.21.15.45:0/3340723831 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb1240762d0 0x7fb124078790 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb13c000f60 tx=0x7fb13c007c80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:18.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.346+0000 7fb135ffb640 1 -- 172.21.15.45:0/3340723831 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb14014b920 con 0x7fb1400d2bb0 2024-05-08T05:49:18.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:18.538+0000 7fb14745c640 1 -- 172.21.15.45:0/3340723831 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb14006da10 con 0x7fb1400d2bb0 2024-05-08T05:49:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.201+0000 7fb135ffb640 1 -- 172.21.15.45:0/3340723831 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v102) ==== 170+0+0 (secure 0 0 0) 0x7fb130060e40 con 0x7fb1400d2bb0 2024-05-08T05:49:19.201 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-05-08T05:49:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.204+0000 7fb14745c640 1 -- 172.21.15.45:0/3340723831 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb1240762d0 msgr2=0x7fb124078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.204+0000 7fb14745c640 1 --2- 172.21.15.45:0/3340723831 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb1240762d0 0x7fb124078790 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb13c000f60 tx=0x7fb13c007c80 comp rx=0 tx=0).stop 2024-05-08T05:49:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.204+0000 7fb14745c640 1 -- 172.21.15.45:0/3340723831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1400d2bb0 msgr2=0x7fb14014b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.204+0000 7fb14745c640 1 --2- 172.21.15.45:0/3340723831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1400d2bb0 0x7fb14014b3e0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fb13000e990 tx=0x7fb13000ee50 comp rx=0 tx=0).stop 2024-05-08T05:49:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.204+0000 7fb14745c640 1 -- 172.21.15.45:0/3340723831 shutdown_connections 2024-05-08T05:49:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.204+0000 7fb14745c640 1 --2- 172.21.15.45:0/3340723831 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb1240762d0 0x7fb124078790 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.204+0000 7fb14745c640 1 --2- 172.21.15.45:0/3340723831 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb140163460 0x7fb140162130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.204+0000 7fb14745c640 1 --2- 172.21.15.45:0/3340723831 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1400d2bb0 0x7fb14014b3e0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.204+0000 7fb14745c640 1 --2- 172.21.15.45:0/3340723831 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb1400b6590 0x7fb14015fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.204+0000 7fb14745c640 1 -- 172.21.15.45:0/3340723831 >> 172.21.15.45:0/3340723831 conn(0x7fb1400d4fe0 msgr2=0x7fb14006cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.204+0000 7fb14745c640 1 -- 172.21.15.45:0/3340723831 shutdown_connections 2024-05-08T05:49:19.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.204+0000 7fb14745c640 1 -- 172.21.15.45:0/3340723831 wait complete. 2024-05-08T05:49:19.231 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T05:49:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.343+0000 7fd25f2a7640 1 Processor -- start 2024-05-08T05:49:19.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.343+0000 7fd25f2a7640 1 -- start start 2024-05-08T05:49:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.344+0000 7fd25f2a7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd2580b9080 0x7fd2580d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.344+0000 7fd25f2a7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2580d3320 con 0x7fd258163460 2024-05-08T05:49:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.344+0000 7fd25f2a7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2580d3600 con 0x7fd2580b6590 2024-05-08T05:49:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.344+0000 7fd25f2a7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd2580d38e0 con 0x7fd2580b9080 2024-05-08T05:49:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.344+0000 7fd25c842640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd258163460 0x7fd25815ec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35496/0 (socket says 172.21.15.45:35496) 2024-05-08T05:49:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.344+0000 7fd25c842640 1 -- 172.21.15.45:0/511125578 learned_addr learned my addr 172.21.15.45:0/511125578 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd24ffff640 1 -- 172.21.15.45:0/511125578 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4227078202 0 0) 0x7fd2580d3320 con 0x7fd258163460 2024-05-08T05:49:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd24ffff640 1 -- 172.21.15.45:0/511125578 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd244003520 con 0x7fd258163460 2024-05-08T05:49:19.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd25d844640 1 --2- 172.21.15.45:0/511125578 >> v2:172.21.15.63:3300/0 conn(0x7fd2580b9080 0x7fd2580d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd24ffff640 1 -- 172.21.15.45:0/511125578 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3290292782 0 0) 0x7fd244003520 con 0x7fd258163460 2024-05-08T05:49:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd24ffff640 1 -- 172.21.15.45:0/511125578 >> v2:172.21.15.63:3300/0 conn(0x7fd2580b9080 msgr2=0x7fd2580d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd24ffff640 1 --2- 172.21.15.45:0/511125578 >> v2:172.21.15.63:3300/0 conn(0x7fd2580b9080 0x7fd2580d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd24ffff640 1 -- 172.21.15.45:0/511125578 >> v1:172.21.15.63:6789/0 conn(0x7fd2580b6590 legacy=0x7fd258162d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd24ffff640 1 -- 172.21.15.45:0/511125578 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2580d3bc0 con 0x7fd258163460 2024-05-08T05:49:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd24ffff640 1 -- 172.21.15.45:0/511125578 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fd2480016f0 con 0x7fd258163460 2024-05-08T05:49:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd24ffff640 1 -- 172.21.15.45:0/511125578 >> v1:172.21.15.45:6789/0 conn(0x7fd258163460 legacy=0x7fd25815ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd24ffff640 1 --2- 172.21.15.45:0/511125578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd244003a50 0x7fd244003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd24ffff640 1 --2- 172.21.15.45:0/511125578 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd244004800 0x7fd244004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd24ffff640 1 --2- 172.21.15.45:0/511125578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2440052e0 0x7fd24400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd24ffff640 1 -- 172.21.15.45:0/511125578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2580d38e0 con 0x7fd244003a50 2024-05-08T05:49:19.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd24ffff640 1 -- 172.21.15.45:0/511125578 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd24400c1b0 con 0x7fd2440052e0 2024-05-08T05:49:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd24ffff640 1 -- 172.21.15.45:0/511125578 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd24400c3b0 con 0x7fd244004800 2024-05-08T05:49:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd25d043640 1 --2- 172.21.15.45:0/511125578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd244003a50 0x7fd244003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd25d844640 1 --2- 172.21.15.45:0/511125578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2440052e0 0x7fd24400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd25c842640 1 --2- 172.21.15.45:0/511125578 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd244004800 0x7fd244004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd25d043640 1 -- 172.21.15.45:0/511125578 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd244004800 msgr2=0x7fd244004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd25d043640 1 --2- 172.21.15.45:0/511125578 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd244004800 0x7fd244004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd25d043640 1 -- 172.21.15.45:0/511125578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2440052e0 msgr2=0x7fd24400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd25d043640 1 --2- 172.21.15.45:0/511125578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2440052e0 0x7fd24400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd25d043640 1 -- 172.21.15.45:0/511125578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd244003520 con 0x7fd244003a50 2024-05-08T05:49:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd25d844640 1 --2- 172.21.15.45:0/511125578 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd2440052e0 0x7fd24400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.345+0000 7fd25d844640 1 -- 172.21.15.45:0/511125578 reap_dead start 2024-05-08T05:49:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.346+0000 7fd25d043640 1 --2- 172.21.15.45:0/511125578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd244003a50 0x7fd244003e70 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fd254001760 tx=0x7fd244005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8137b526df6eaa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.346+0000 7fd24ffff640 1 -- 172.21.15.45:0/511125578 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2540013b0 con 0x7fd244003a50 2024-05-08T05:49:19.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.346+0000 7fd24ffff640 1 -- 172.21.15.45:0/511125578 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2540052d0 con 0x7fd244003a50 2024-05-08T05:49:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.346+0000 7fd24ffff640 1 -- 172.21.15.45:0/511125578 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25402aa50 con 0x7fd244003a50 2024-05-08T05:49:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.347+0000 7fd25f2a7640 1 -- 172.21.15.45:0/511125578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd244003a50 msgr2=0x7fd244003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.347+0000 7fd25f2a7640 1 --2- 172.21.15.45:0/511125578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd244003a50 0x7fd244003e70 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fd254001760 tx=0x7fd244005830 comp rx=0 tx=0).stop 2024-05-08T05:49:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.347+0000 7fd25f2a7640 1 -- 172.21.15.45:0/511125578 shutdown_connections 2024-05-08T05:49:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.347+0000 7fd25f2a7640 1 --2- 172.21.15.45:0/511125578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd244003a50 0x7fd244003e70 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.347+0000 7fd25f2a7640 1 -- 172.21.15.45:0/511125578 >> 172.21.15.45:0/511125578 conn(0x7fd2580d4fe0 msgr2=0x7fd2580d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.347+0000 7fd25f2a7640 1 -- 172.21.15.45:0/511125578 shutdown_connections 2024-05-08T05:49:19.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.347+0000 7fd25f2a7640 1 -- 172.21.15.45:0/511125578 wait complete. 2024-05-08T05:49:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.348+0000 7fd25f2a7640 1 Processor -- start 2024-05-08T05:49:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.348+0000 7fd25f2a7640 1 -- start start 2024-05-08T05:49:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.349+0000 7fd25f2a7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd244003a50 0x7fd2581f9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.349+0000 7fd25f2a7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd25814dff0 0x7fd2581f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.349+0000 7fd25f2a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2581592d0 0x7fd258159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.349+0000 7fd25f2a7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2580d0d50 con 0x7fd2581592d0 2024-05-08T05:49:19.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.349+0000 7fd25f2a7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd2580d0bd0 con 0x7fd25814dff0 2024-05-08T05:49:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.349+0000 7fd25f2a7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd2580d0ed0 con 0x7fd244003a50 2024-05-08T05:49:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.349+0000 7fd25d844640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2581592d0 0x7fd258159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.349+0000 7fd25d844640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2581592d0 0x7fd258159750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43174/0 (socket says 172.21.15.45:43174) 2024-05-08T05:49:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.349+0000 7fd25d844640 1 -- 172.21.15.45:0/3249530725 learned_addr learned my addr 172.21.15.45:0/3249530725 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.349+0000 7fd25c842640 1 --2- 172.21.15.45:0/3249530725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd25814dff0 0x7fd2581f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.349+0000 7fd25d043640 1 --2- 172.21.15.45:0/3249530725 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd244003a50 0x7fd2581f9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.349+0000 7fd25d844640 1 -- 172.21.15.45:0/3249530725 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd244003a50 msgr2=0x7fd2581f9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.349+0000 7fd25d844640 1 --2- 172.21.15.45:0/3249530725 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd244003a50 0x7fd2581f9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.349+0000 7fd25d844640 1 -- 172.21.15.45:0/3249530725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd25814dff0 msgr2=0x7fd2581f4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.349+0000 7fd25d844640 1 --2- 172.21.15.45:0/3249530725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd25814dff0 0x7fd2581f4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.349+0000 7fd25d844640 1 -- 172.21.15.45:0/3249530725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2581f9890 con 0x7fd2581592d0 2024-05-08T05:49:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.350+0000 7fd25d844640 1 --2- 172.21.15.45:0/3249530725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2581592d0 0x7fd258159750 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fd25808ed10 tx=0x7fd250004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.350+0000 7fd24dffb640 1 -- 172.21.15.45:0/3249530725 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250007c80 con 0x7fd2581592d0 2024-05-08T05:49:19.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.350+0000 7fd24dffb640 1 -- 172.21.15.45:0/3249530725 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd250019070 con 0x7fd2581592d0 2024-05-08T05:49:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.350+0000 7fd25f2a7640 1 -- 172.21.15.45:0/3249530725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2580cd730 con 0x7fd2581592d0 2024-05-08T05:49:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.350+0000 7fd24dffb640 1 -- 172.21.15.45:0/3249530725 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250014650 con 0x7fd2581592d0 2024-05-08T05:49:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.350+0000 7fd25f2a7640 1 -- 172.21.15.45:0/3249530725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2580cdcf0 con 0x7fd2581592d0 2024-05-08T05:49:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.351+0000 7fd25f2a7640 1 -- 172.21.15.45:0/3249530725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2580693f0 con 0x7fd2581592d0 2024-05-08T05:49:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.352+0000 7fd24dffb640 1 -- 172.21.15.45:0/3249530725 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd2500147f0 con 0x7fd2581592d0 2024-05-08T05:49:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.352+0000 7fd24dffb640 1 --2- 172.21.15.45:0/3249530725 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd23c0762d0 0x7fd23c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.352+0000 7fd24dffb640 1 -- 172.21.15.45:0/3249530725 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6001+0+0 (secure 0 0 0) 0x7fd250097ad0 con 0x7fd2581592d0 2024-05-08T05:49:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.352+0000 7fd25d043640 1 --2- 172.21.15.45:0/3249530725 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd23c0762d0 0x7fd23c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:19.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.353+0000 7fd25d043640 1 --2- 172.21.15.45:0/3249530725 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd23c0762d0 0x7fd23c078790 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd254032000 tx=0x7fd25402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:19.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.357+0000 7fd24dffb640 1 -- 172.21.15.45:0/3249530725 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd250060cb0 con 0x7fd2581592d0 2024-05-08T05:49:19.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:19.586+0000 7fd25f2a7640 1 -- 172.21.15.45:0/3249530725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd2580ba650 con 0x7fd2581592d0 2024-05-08T05:49:20.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.202+0000 7fd24dffb640 1 -- 172.21.15.45:0/3249530725 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v79) ==== 174+0+0 (secure 0 0 0) 0x7fd250064fb0 con 0x7fd2581592d0 2024-05-08T05:49:20.203 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 15 and data pool 16 2024-05-08T05:49:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.207+0000 7fd25f2a7640 1 -- 172.21.15.45:0/3249530725 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd23c0762d0 msgr2=0x7fd23c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.207+0000 7fd25f2a7640 1 --2- 172.21.15.45:0/3249530725 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd23c0762d0 0x7fd23c078790 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd254032000 tx=0x7fd25402e000 comp rx=0 tx=0).stop 2024-05-08T05:49:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.207+0000 7fd25f2a7640 1 -- 172.21.15.45:0/3249530725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2581592d0 msgr2=0x7fd258159750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.207+0000 7fd25f2a7640 1 --2- 172.21.15.45:0/3249530725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2581592d0 0x7fd258159750 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fd25808ed10 tx=0x7fd250004840 comp rx=0 tx=0).stop 2024-05-08T05:49:20.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.207+0000 7fd25f2a7640 1 -- 172.21.15.45:0/3249530725 shutdown_connections 2024-05-08T05:49:20.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.207+0000 7fd25f2a7640 1 --2- 172.21.15.45:0/3249530725 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd23c0762d0 0x7fd23c078790 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.207+0000 7fd25f2a7640 1 --2- 172.21.15.45:0/3249530725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2581592d0 0x7fd258159750 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.207+0000 7fd25f2a7640 1 --2- 172.21.15.45:0/3249530725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd25814dff0 0x7fd2581f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.207+0000 7fd25f2a7640 1 --2- 172.21.15.45:0/3249530725 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd244003a50 0x7fd2581f9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.207+0000 7fd25f2a7640 1 -- 172.21.15.45:0/3249530725 >> 172.21.15.45:0/3249530725 conn(0x7fd2580d4fe0 msgr2=0x7fd25806d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:20.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.209+0000 7fd25f2a7640 1 -- 172.21.15.45:0/3249530725 shutdown_connections 2024-05-08T05:49:20.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.209+0000 7fd25f2a7640 1 -- 172.21.15.45:0/3249530725 wait complete. 2024-05-08T05:49:20.234 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:49:20.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.345+0000 7f1914485640 1 Processor -- start 2024-05-08T05:49:20.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.345+0000 7f1914485640 1 -- start start 2024-05-08T05:49:20.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.346+0000 7f1914485640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f190c15ce90 0x7f190c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:20.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.346+0000 7f1914485640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f190c06d3b0 0x7f190c06d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:20.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.346+0000 7f1914485640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f190c071280 con 0x7f190c0733e0 2024-05-08T05:49:20.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.346+0000 7f1914485640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f190c06dd60 con 0x7f190c15ce90 2024-05-08T05:49:20.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.346+0000 7f1914485640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f190c0d1390 con 0x7f190c06d3b0 2024-05-08T05:49:20.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.346+0000 7f1912221640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f190c15ce90 0x7f190c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:20.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.346+0000 7f1912221640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f190c15ce90 0x7f190c070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43182/0 (socket says 172.21.15.45:43182) 2024-05-08T05:49:20.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.346+0000 7f1912221640 1 -- 172.21.15.45:0/4291009171 learned_addr learned my addr 172.21.15.45:0/4291009171 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:20.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.347+0000 7f1912221640 1 -- 172.21.15.45:0/4291009171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f190c06d3b0 msgr2=0x7f190c06d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:20.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.347+0000 7f1912a22640 1 --2- 172.21.15.45:0/4291009171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f190c06d3b0 0x7f190c06d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:20.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.347+0000 7f1912221640 1 --2- 172.21.15.45:0/4291009171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f190c06d3b0 0x7f190c06d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.347+0000 7f1912221640 1 -- 172.21.15.45:0/4291009171 >> v1:172.21.15.63:6789/0 conn(0x7f190c0733e0 legacy=0x7f190c0d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:20.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.347+0000 7f1912221640 1 -- 172.21.15.45:0/4291009171 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190c0cff00 con 0x7f190c15ce90 2024-05-08T05:49:20.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.347+0000 7f1912221640 1 --2- 172.21.15.45:0/4291009171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f190c15ce90 0x7f190c070d40 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f1904009870 tx=0x7f190402a1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5286a3e840ccb5f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:20.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.347+0000 7f191121f640 1 -- 172.21.15.45:0/4291009171 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1904037070 con 0x7f190c15ce90 2024-05-08T05:49:20.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.348+0000 7f191121f640 1 -- 172.21.15.45:0/4291009171 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1904032440 con 0x7f190c15ce90 2024-05-08T05:49:20.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.348+0000 7f191121f640 1 -- 172.21.15.45:0/4291009171 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1904032810 con 0x7f190c15ce90 2024-05-08T05:49:20.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.348+0000 7f1914485640 1 -- 172.21.15.45:0/4291009171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f190c15ce90 msgr2=0x7f190c070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:20.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.348+0000 7f1914485640 1 --2- 172.21.15.45:0/4291009171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f190c15ce90 0x7f190c070d40 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f1904009870 tx=0x7f190402a1b0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.348+0000 7f1914485640 1 -- 172.21.15.45:0/4291009171 shutdown_connections 2024-05-08T05:49:20.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.348+0000 7f1914485640 1 --2- 172.21.15.45:0/4291009171 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f190c06d3b0 0x7f190c06d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.348+0000 7f1914485640 1 --2- 172.21.15.45:0/4291009171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f190c15ce90 0x7f190c070d40 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.348+0000 7f1914485640 1 -- 172.21.15.45:0/4291009171 >> 172.21.15.45:0/4291009171 conn(0x7f190c0728c0 msgr2=0x7f190c072cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:20.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.348+0000 7f1914485640 1 -- 172.21.15.45:0/4291009171 shutdown_connections 2024-05-08T05:49:20.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.348+0000 7f1914485640 1 -- 172.21.15.45:0/4291009171 wait complete. 2024-05-08T05:49:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.349+0000 7f1914485640 1 Processor -- start 2024-05-08T05:49:20.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.349+0000 7f1914485640 1 -- start start 2024-05-08T05:49:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.350+0000 7f1914485640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f190c06d3b0 0x7f190c149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.350+0000 7f1914485640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f190c0733e0 0x7f190c1582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.350+0000 7f1914485640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f190c15ce90 0x7f190c06fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.350+0000 7f1914485640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f190c0d0480 con 0x7f190c0733e0 2024-05-08T05:49:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.350+0000 7f1914485640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f190c0d01a0 con 0x7f190c15ce90 2024-05-08T05:49:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.350+0000 7f1914485640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f190c0d0760 con 0x7f190c06d3b0 2024-05-08T05:49:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.350+0000 7f1911a20640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f190c0733e0 0x7f190c1582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.350+0000 7f1911a20640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f190c0733e0 0x7f190c1582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43198/0 (socket says 172.21.15.45:43198) 2024-05-08T05:49:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.350+0000 7f1911a20640 1 -- 172.21.15.45:0/1897600060 learned_addr learned my addr 172.21.15.45:0/1897600060 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.350+0000 7f1912221640 1 --2- 172.21.15.45:0/1897600060 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f190c06d3b0 0x7f190c149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.350+0000 7f1912a22640 1 --2- 172.21.15.45:0/1897600060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f190c15ce90 0x7f190c06fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.351+0000 7f1911a20640 1 -- 172.21.15.45:0/1897600060 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f190c06d3b0 msgr2=0x7f190c149440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:20.354 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.351+0000 7f1911a20640 1 --2- 172.21.15.45:0/1897600060 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f190c06d3b0 0x7f190c149440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.351+0000 7f1911a20640 1 -- 172.21.15.45:0/1897600060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f190c15ce90 msgr2=0x7f190c06fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.351+0000 7f1911a20640 1 --2- 172.21.15.45:0/1897600060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f190c15ce90 0x7f190c06fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.351+0000 7f1911a20640 1 -- 172.21.15.45:0/1897600060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190c15b120 con 0x7f190c0733e0 2024-05-08T05:49:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.351+0000 7f1912a22640 1 --2- 172.21.15.45:0/1897600060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f190c15ce90 0x7f190c06fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:49:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.351+0000 7f1912221640 1 --2- 172.21.15.45:0/1897600060 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f190c06d3b0 0x7f190c149440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:49:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.351+0000 7f1911a20640 1 --2- 172.21.15.45:0/1897600060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f190c0733e0 0x7f190c1582f0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f190000c460 tx=0x7f190000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.351+0000 7f190affd640 1 -- 172.21.15.45:0/1897600060 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1900016020 con 0x7f190c0733e0 2024-05-08T05:49:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.351+0000 7f190affd640 1 -- 172.21.15.45:0/1897600060 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1900012870 con 0x7f190c0733e0 2024-05-08T05:49:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.351+0000 7f1914485640 1 -- 172.21.15.45:0/1897600060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190c15b400 con 0x7f190c0733e0 2024-05-08T05:49:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.351+0000 7f190affd640 1 -- 172.21.15.45:0/1897600060 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1900012c60 con 0x7f190c0733e0 2024-05-08T05:49:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.351+0000 7f1914485640 1 -- 172.21.15.45:0/1897600060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f190c15b9c0 con 0x7f190c0733e0 2024-05-08T05:49:20.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.352+0000 7f1914485640 1 -- 172.21.15.45:0/1897600060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f190c0693f0 con 0x7f190c0733e0 2024-05-08T05:49:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.353+0000 7f190affd640 1 -- 172.21.15.45:0/1897600060 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1900019020 con 0x7f190c0733e0 2024-05-08T05:49:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.353+0000 7f190affd640 1 --2- 172.21.15.45:0/1897600060 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f18f80762d0 0x7f18f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.353+0000 7f190affd640 1 -- 172.21.15.45:0/1897600060 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7f19000974c0 con 0x7f190c0733e0 2024-05-08T05:49:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.353+0000 7f1912221640 1 --2- 172.21.15.45:0/1897600060 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f18f80762d0 0x7f18f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:20.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.354+0000 7f1912221640 1 --2- 172.21.15.45:0/1897600060 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f18f80762d0 0x7f18f8078790 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1904034000 tx=0x7f190402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:20.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.358+0000 7f190affd640 1 -- 172.21.15.45:0/1897600060 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1900020030 con 0x7f190c0733e0 2024-05-08T05:49:20.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.552+0000 7f1914485640 1 -- 172.21.15.45:0/1897600060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f190c0b5770 con 0x7f190c0733e0 2024-05-08T05:49:20.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.553+0000 7f190affd640 1 -- 172.21.15.45:0/1897600060 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16849 (secure 0 0 0) 0x7f1900060c90 con 0x7f190c0733e0 2024-05-08T05:49:20.553 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:20.553 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":103,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:49:19.587362+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:49:17.241406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-08T05:49:18.542707+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:49:20.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.555+0000 7f1914485640 1 -- 172.21.15.45:0/1897600060 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f18f80762d0 msgr2=0x7f18f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:20.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.555+0000 7f1914485640 1 --2- 172.21.15.45:0/1897600060 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f18f80762d0 0x7f18f8078790 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1904034000 tx=0x7f190402e000 comp rx=0 tx=0).stop 2024-05-08T05:49:20.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.555+0000 7f1914485640 1 -- 172.21.15.45:0/1897600060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f190c0733e0 msgr2=0x7f190c1582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:20.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.555+0000 7f1914485640 1 --2- 172.21.15.45:0/1897600060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f190c0733e0 0x7f190c1582f0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f190000c460 tx=0x7f190000c920 comp rx=0 tx=0).stop 2024-05-08T05:49:20.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.555+0000 7f1914485640 1 -- 172.21.15.45:0/1897600060 shutdown_connections 2024-05-08T05:49:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.555+0000 7f1914485640 1 --2- 172.21.15.45:0/1897600060 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f18f80762d0 0x7f18f8078790 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.555+0000 7f1914485640 1 --2- 172.21.15.45:0/1897600060 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f190c15ce90 0x7f190c06fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.555+0000 7f1914485640 1 --2- 172.21.15.45:0/1897600060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f190c0733e0 0x7f190c1582f0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.555+0000 7f1914485640 1 --2- 172.21.15.45:0/1897600060 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f190c06d3b0 0x7f190c149440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.555+0000 7f1914485640 1 -- 172.21.15.45:0/1897600060 >> 172.21.15.45:0/1897600060 conn(0x7f190c0728c0 msgr2=0x7f190c0b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.555+0000 7f1914485640 1 -- 172.21.15.45:0/1897600060 shutdown_connections 2024-05-08T05:49:20.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.555+0000 7f1914485640 1 -- 172.21.15.45:0/1897600060 wait complete. 2024-05-08T05:49:20.566 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:49:20.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.670+0000 7f5ece3a4640 1 Processor -- start 2024-05-08T05:49:20.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.671+0000 7f5ece3a4640 1 -- start start 2024-05-08T05:49:20.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.671+0000 7f5ece3a4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ec81525f0 0x7f5ec806e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:20.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.671+0000 7f5ece3a4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ec81529d0 con 0x7f5ec806e8d0 2024-05-08T05:49:20.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.671+0000 7f5ece3a4640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ec80b4ae0 con 0x7f5ec80b4450 2024-05-08T05:49:20.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.671+0000 7f5ece3a4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec80b48c0 con 0x7f5ec81525f0 2024-05-08T05:49:20.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.672+0000 7f5ebffff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5ec806e8d0 0x7f5ec80b3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49838/0 (socket says 172.21.15.45:49838) 2024-05-08T05:49:20.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.672+0000 7f5ebffff640 1 -- 172.21.15.45:0/612287809 learned_addr learned my addr 172.21.15.45:0/612287809 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:20.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.672+0000 7f5ec7fff640 1 --2- 172.21.15.45:0/612287809 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ec81525f0 0x7f5ec806e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:20.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.672+0000 7f5ec77fe640 1 -- 172.21.15.45:0/612287809 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2676854005 0 0) 0x7f5ec81529d0 con 0x7f5ec806e8d0 2024-05-08T05:49:20.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.672+0000 7f5ec77fe640 1 -- 172.21.15.45:0/612287809 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5eb4003520 con 0x7f5ec806e8d0 2024-05-08T05:49:20.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.672+0000 7f5ec77fe640 1 -- 172.21.15.45:0/612287809 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1358862668 0 0) 0x7f5ec80b4ae0 con 0x7f5ec80b4450 2024-05-08T05:49:20.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.672+0000 7f5ec77fe640 1 -- 172.21.15.45:0/612287809 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ec81529d0 con 0x7f5ec80b4450 2024-05-08T05:49:20.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.672+0000 7f5ec77fe640 1 -- 172.21.15.45:0/612287809 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1005789581 0 0) 0x7f5eb4003520 con 0x7f5ec806e8d0 2024-05-08T05:49:20.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.672+0000 7f5ec7fff640 1 -- 172.21.15.45:0/612287809 >> v1:172.21.15.63:6789/0 conn(0x7f5ec80b4450 legacy=0x7f5ec815cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:20.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.672+0000 7f5ec7fff640 1 -- 172.21.15.45:0/612287809 >> v1:172.21.15.45:6789/0 conn(0x7f5ec806e8d0 legacy=0x7f5ec80b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:20.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.672+0000 7f5ec7fff640 1 -- 172.21.15.45:0/612287809 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec80b4dc0 con 0x7f5ec81525f0 2024-05-08T05:49:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.673+0000 7f5ec7fff640 1 --2- 172.21.15.45:0/612287809 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ec81525f0 0x7f5ec806e390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80097d0 tx=0x7f5eb802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3edf1117523abb15 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.673+0000 7f5ec77fe640 1 -- 172.21.15.45:0/612287809 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8037070 con 0x7f5ec81525f0 2024-05-08T05:49:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.673+0000 7f5ec77fe640 1 -- 172.21.15.45:0/612287809 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb802ab90 con 0x7f5ec81525f0 2024-05-08T05:49:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.673+0000 7f5ec77fe640 1 -- 172.21.15.45:0/612287809 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb00016f0 con 0x7f5ec81525f0 2024-05-08T05:49:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.674+0000 7f5ece3a4640 1 -- 172.21.15.45:0/612287809 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ec81525f0 msgr2=0x7f5ec806e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.674+0000 7f5ece3a4640 1 --2- 172.21.15.45:0/612287809 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ec81525f0 0x7f5ec806e390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80097d0 tx=0x7f5eb802a100 comp rx=0 tx=0).stop 2024-05-08T05:49:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.674+0000 7f5ece3a4640 1 -- 172.21.15.45:0/612287809 shutdown_connections 2024-05-08T05:49:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.674+0000 7f5ece3a4640 1 --2- 172.21.15.45:0/612287809 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ec81525f0 0x7f5ec806e390 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.674+0000 7f5ece3a4640 1 -- 172.21.15.45:0/612287809 >> 172.21.15.45:0/612287809 conn(0x7f5ec806c3a0 msgr2=0x7f5ec806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.674+0000 7f5ece3a4640 1 -- 172.21.15.45:0/612287809 shutdown_connections 2024-05-08T05:49:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.674+0000 7f5ece3a4640 1 -- 172.21.15.45:0/612287809 wait complete. 2024-05-08T05:49:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.675+0000 7f5ece3a4640 1 Processor -- start 2024-05-08T05:49:20.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.675+0000 7f5ece3a4640 1 -- start start 2024-05-08T05:49:20.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.676+0000 7f5ece3a4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec806e8d0 0x7f5ec815f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:20.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.676+0000 7f5ece3a4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ec80b4450 0x7f5ec809bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:20.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.676+0000 7f5ece3a4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ec81525f0 0x7f5ec81f8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:20.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.676+0000 7f5ece3a4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec80b5380 con 0x7f5ec806e8d0 2024-05-08T05:49:20.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.676+0000 7f5ece3a4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec806ed10 con 0x7f5ec80b4450 2024-05-08T05:49:20.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.676+0000 7f5ece3a4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec80d0bf0 con 0x7f5ec81525f0 2024-05-08T05:49:20.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.676+0000 7f5ec7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec806e8d0 0x7f5ec815f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:20.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.676+0000 7f5ec7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec806e8d0 0x7f5ec815f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45266/0 (socket says 172.21.15.45:45266) 2024-05-08T05:49:20.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.676+0000 7f5ec7fff640 1 -- 172.21.15.45:0/944743868 learned_addr learned my addr 172.21.15.45:0/944743868 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:20.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.676+0000 7f5ecc941640 1 --2- 172.21.15.45:0/944743868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ec81525f0 0x7f5ec81f8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:20.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.676+0000 7f5ebffff640 1 --2- 172.21.15.45:0/944743868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ec80b4450 0x7f5ec809bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:20.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.676+0000 7f5ec7fff640 1 -- 172.21.15.45:0/944743868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ec81525f0 msgr2=0x7f5ec81f8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:20.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.676+0000 7f5ec7fff640 1 --2- 172.21.15.45:0/944743868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ec81525f0 0x7f5ec81f8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.676+0000 7f5ec7fff640 1 -- 172.21.15.45:0/944743868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ec80b4450 msgr2=0x7f5ec809bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:20.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.676+0000 7f5ec7fff640 1 --2- 172.21.15.45:0/944743868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ec80b4450 0x7f5ec809bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.676+0000 7f5ec7fff640 1 -- 172.21.15.45:0/944743868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec809e6a0 con 0x7f5ec806e8d0 2024-05-08T05:49:20.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.676+0000 7f5ec7fff640 1 --2- 172.21.15.45:0/944743868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec806e8d0 0x7f5ec815f980 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80097d0 tx=0x7f5eb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:20.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.676+0000 7f5ec57fa640 1 -- 172.21.15.45:0/944743868 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8037050 con 0x7f5ec806e8d0 2024-05-08T05:49:20.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.677+0000 7f5ec57fa640 1 -- 172.21.15.45:0/944743868 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb8038040 con 0x7f5ec806e8d0 2024-05-08T05:49:20.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.677+0000 7f5ec57fa640 1 -- 172.21.15.45:0/944743868 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8006e10 con 0x7f5ec806e8d0 2024-05-08T05:49:20.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.677+0000 7f5ece3a4640 1 -- 172.21.15.45:0/944743868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec809e980 con 0x7f5ec806e8d0 2024-05-08T05:49:20.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.677+0000 7f5ece3a4640 1 -- 172.21.15.45:0/944743868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec8072730 con 0x7f5ec806e8d0 2024-05-08T05:49:20.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.678+0000 7f5ece3a4640 1 -- 172.21.15.45:0/944743868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec80693f0 con 0x7f5ec806e8d0 2024-05-08T05:49:20.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.678+0000 7f5ec57fa640 1 -- 172.21.15.45:0/944743868 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5eb8045020 con 0x7f5ec806e8d0 2024-05-08T05:49:20.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.679+0000 7f5ec57fa640 1 --2- 172.21.15.45:0/944743868 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5ea80762d0 0x7f5ea8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:20.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.679+0000 7f5ec57fa640 1 -- 172.21.15.45:0/944743868 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7f5eb80b7840 con 0x7f5ec806e8d0 2024-05-08T05:49:20.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.679+0000 7f5ebffff640 1 --2- 172.21.15.45:0/944743868 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5ea80762d0 0x7f5ea8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:20.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.680+0000 7f5ebffff640 1 --2- 172.21.15.45:0/944743868 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5ea80762d0 0x7f5ea8078790 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8074430 tx=0x7f5eb0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:20.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.683+0000 7f5ec57fa640 1 -- 172.21.15.45:0/944743868 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5eb804c020 con 0x7f5ec806e8d0 2024-05-08T05:49:20.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.876+0000 7f5ece3a4640 1 -- 172.21.15.45:0/944743868 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ec80b3a90 con 0x7f5ec806e8d0 2024-05-08T05:49:20.877 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.877+0000 7f5ec57fa640 1 -- 172.21.15.45:0/944743868 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16849 (secure 0 0 0) 0x7f5eb8081000 con 0x7f5ec806e8d0 2024-05-08T05:49:20.878 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:20.878 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":103,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:49:19.587362+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:49:17.241406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-08T05:49:18.542707+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:49:20.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.879+0000 7f5ece3a4640 1 -- 172.21.15.45:0/944743868 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5ea80762d0 msgr2=0x7f5ea8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.879+0000 7f5ece3a4640 1 --2- 172.21.15.45:0/944743868 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5ea80762d0 0x7f5ea8078790 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8074430 tx=0x7f5eb0009040 comp rx=0 tx=0).stop 2024-05-08T05:49:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.879+0000 7f5ece3a4640 1 -- 172.21.15.45:0/944743868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec806e8d0 msgr2=0x7f5ec815f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.879+0000 7f5ece3a4640 1 --2- 172.21.15.45:0/944743868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec806e8d0 0x7f5ec815f980 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80097d0 tx=0x7f5eb80046c0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.879+0000 7f5ece3a4640 1 -- 172.21.15.45:0/944743868 shutdown_connections 2024-05-08T05:49:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.879+0000 7f5ece3a4640 1 --2- 172.21.15.45:0/944743868 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5ea80762d0 0x7f5ea8078790 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.879+0000 7f5ece3a4640 1 --2- 172.21.15.45:0/944743868 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5ec81525f0 0x7f5ec81f8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.879+0000 7f5ece3a4640 1 --2- 172.21.15.45:0/944743868 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5ec80b4450 0x7f5ec809bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.879+0000 7f5ece3a4640 1 --2- 172.21.15.45:0/944743868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ec806e8d0 0x7f5ec815f980 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.879+0000 7f5ece3a4640 1 -- 172.21.15.45:0/944743868 >> 172.21.15.45:0/944743868 conn(0x7f5ec806c3a0 msgr2=0x7f5ec80ae860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.879+0000 7f5ece3a4640 1 -- 172.21.15.45:0/944743868 shutdown_connections 2024-05-08T05:49:20.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.879+0000 7f5ece3a4640 1 -- 172.21.15.45:0/944743868 wait complete. 2024-05-08T05:49:20.891 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T05:49:20.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.998+0000 7f8693c99640 1 Processor -- start 2024-05-08T05:49:20.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.998+0000 7f8693c99640 1 -- start start 2024-05-08T05:49:21.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.999+0000 7f8693c99640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f868c06da00 0x7f868c073800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:21.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.999+0000 7f8693c99640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f868c1637b0 0x7f868c0b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.999+0000 7f8693c99640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f868c0b5120 con 0x7f868c06d620 2024-05-08T05:49:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.999+0000 7f8693c99640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f868c0afaa0 con 0x7f868c06da00 2024-05-08T05:49:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.999+0000 7f8693c99640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f868c0b5a50 con 0x7f868c1637b0 2024-05-08T05:49:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.999+0000 7f8691234640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f868c06da00 0x7f868c073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.999+0000 7f8691234640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f868c06da00 0x7f868c073800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45272/0 (socket says 172.21.15.45:45272) 2024-05-08T05:49:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.999+0000 7f8691234640 1 -- 172.21.15.45:0/3841757286 learned_addr learned my addr 172.21.15.45:0/3841757286 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.999+0000 7f8692236640 1 --2- 172.21.15.45:0/3841757286 >> v2:172.21.15.63:3300/0 conn(0x7f868c1637b0 0x7f868c0b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.999+0000 7f8691234640 1 -- 172.21.15.45:0/3841757286 >> v2:172.21.15.63:3300/0 conn(0x7f868c1637b0 msgr2=0x7f868c0b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.999+0000 7f8691234640 1 --2- 172.21.15.45:0/3841757286 >> v2:172.21.15.63:3300/0 conn(0x7f868c1637b0 0x7f868c0b5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.999+0000 7f8691234640 1 -- 172.21.15.45:0/3841757286 >> v1:172.21.15.63:6789/0 conn(0x7f868c06d620 legacy=0x7f868c0af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:20.999+0000 7f8691234640 1 -- 172.21.15.45:0/3841757286 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868c0b5400 con 0x7f868c06da00 2024-05-08T05:49:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.000+0000 7f8691234640 1 --2- 172.21.15.45:0/3841757286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f868c06da00 0x7f868c073800 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f868c06e780 tx=0x7f867c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3076119e702361 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.000+0000 7f8690a33640 1 -- 172.21.15.45:0/3841757286 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c037070 con 0x7f868c06da00 2024-05-08T05:49:21.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.000+0000 7f8690a33640 1 -- 172.21.15.45:0/3841757286 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f867c02ae00 con 0x7f868c06da00 2024-05-08T05:49:21.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.000+0000 7f8690a33640 1 -- 172.21.15.45:0/3841757286 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c032660 con 0x7f868c06da00 2024-05-08T05:49:21.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.001+0000 7f8693c99640 1 -- 172.21.15.45:0/3841757286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f868c06da00 msgr2=0x7f868c073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:21.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.001+0000 7f8693c99640 1 --2- 172.21.15.45:0/3841757286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f868c06da00 0x7f868c073800 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f868c06e780 tx=0x7f867c02a0b0 comp rx=0 tx=0).stop 2024-05-08T05:49:21.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.001+0000 7f8693c99640 1 -- 172.21.15.45:0/3841757286 shutdown_connections 2024-05-08T05:49:21.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.001+0000 7f8693c99640 1 --2- 172.21.15.45:0/3841757286 >> v2:172.21.15.63:3300/0 conn(0x7f868c1637b0 0x7f868c0b5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:21.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.001+0000 7f8693c99640 1 --2- 172.21.15.45:0/3841757286 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f868c06da00 0x7f868c073800 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:21.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.001+0000 7f8693c99640 1 -- 172.21.15.45:0/3841757286 >> 172.21.15.45:0/3841757286 conn(0x7f868c0728c0 msgr2=0x7f868c072cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:21.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.001+0000 7f8693c99640 1 -- 172.21.15.45:0/3841757286 shutdown_connections 2024-05-08T05:49:21.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.001+0000 7f8693c99640 1 -- 172.21.15.45:0/3841757286 wait complete. 2024-05-08T05:49:21.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.002+0000 7f8693c99640 1 Processor -- start 2024-05-08T05:49:21.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.002+0000 7f8693c99640 1 -- start start 2024-05-08T05:49:21.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.003+0000 7f8693c99640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f868c06d620 0x7f868c0cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:21.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.003+0000 7f8693c99640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f868c06da00 0x7f868c1570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:21.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.003+0000 7f8693c99640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f868c1637b0 0x7f868c1490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:21.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.003+0000 7f8693c99640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f868c0b65a0 con 0x7f868c1637b0 2024-05-08T05:49:21.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.003+0000 7f8693c99640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f868c0b6420 con 0x7f868c06d620 2024-05-08T05:49:21.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.003+0000 7f8693c99640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f868c06e920 con 0x7f868c06da00 2024-05-08T05:49:21.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.003+0000 7f8692236640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f868c1637b0 0x7f868c1490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:21.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.003+0000 7f8692236640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f868c1637b0 0x7f868c1490a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45278/0 (socket says 172.21.15.45:45278) 2024-05-08T05:49:21.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.003+0000 7f8692236640 1 -- 172.21.15.45:0/702427291 learned_addr learned my addr 172.21.15.45:0/702427291 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:21.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.003+0000 7f8691a35640 1 --2- 172.21.15.45:0/702427291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f868c06d620 0x7f868c0cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:21.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.003+0000 7f8692236640 1 -- 172.21.15.45:0/702427291 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f868c06da00 msgr2=0x7f868c1570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:21.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.003+0000 7f8691234640 1 --2- 172.21.15.45:0/702427291 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f868c06da00 0x7f868c1570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:21.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.003+0000 7f8692236640 1 --2- 172.21.15.45:0/702427291 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f868c06da00 0x7f868c1570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:21.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.003+0000 7f8692236640 1 -- 172.21.15.45:0/702427291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f868c06d620 msgr2=0x7f868c0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:21.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.003+0000 7f8692236640 1 --2- 172.21.15.45:0/702427291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f868c06d620 0x7f868c0cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:21.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.003+0000 7f8692236640 1 -- 172.21.15.45:0/702427291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868c0ce4b0 con 0x7f868c1637b0 2024-05-08T05:49:21.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.003+0000 7f8691a35640 1 --2- 172.21.15.45:0/702427291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f868c06d620 0x7f868c0cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:49:21.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.003+0000 7f8691234640 1 --2- 172.21.15.45:0/702427291 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f868c06da00 0x7f868c1570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:49:21.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.004+0000 7f8692236640 1 --2- 172.21.15.45:0/702427291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f868c1637b0 0x7f868c1490a0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f8684007e60 tx=0x7f868400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:21.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.004+0000 7f86827fc640 1 -- 172.21.15.45:0/702427291 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684014420 con 0x7f868c1637b0 2024-05-08T05:49:21.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.004+0000 7f86827fc640 1 -- 172.21.15.45:0/702427291 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8684019070 con 0x7f868c1637b0 2024-05-08T05:49:21.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.004+0000 7f8693c99640 1 -- 172.21.15.45:0/702427291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868c0ce790 con 0x7f868c1637b0 2024-05-08T05:49:21.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.004+0000 7f86827fc640 1 -- 172.21.15.45:0/702427291 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684014820 con 0x7f868c1637b0 2024-05-08T05:49:21.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.004+0000 7f8693c99640 1 -- 172.21.15.45:0/702427291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f868c0ced50 con 0x7f868c1637b0 2024-05-08T05:49:21.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.005+0000 7f8693c99640 1 -- 172.21.15.45:0/702427291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f868c0693f0 con 0x7f868c1637b0 2024-05-08T05:49:21.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.006+0000 7f86827fc640 1 -- 172.21.15.45:0/702427291 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f86840149c0 con 0x7f868c1637b0 2024-05-08T05:49:21.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.006+0000 7f86827fc640 1 --2- 172.21.15.45:0/702427291 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f86700762d0 0x7f8670078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:21.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.006+0000 7f86827fc640 1 -- 172.21.15.45:0/702427291 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7f8684097f70 con 0x7f868c1637b0 2024-05-08T05:49:21.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.006+0000 7f8691a35640 1 --2- 172.21.15.45:0/702427291 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f86700762d0 0x7f8670078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:21.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.007+0000 7f8691a35640 1 --2- 172.21.15.45:0/702427291 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f86700762d0 0x7f8670078790 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8688003eb0 tx=0x7f8688009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:21.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.011+0000 7f86827fc640 1 -- 172.21.15.45:0/702427291 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8684061740 con 0x7f868c1637b0 2024-05-08T05:49:21.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:21.244+0000 7f8693c99640 1 -- 172.21.15.45:0/702427291 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f868c0b16f0 con 0x7f868c1637b0 2024-05-08T05:49:22.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.224+0000 7f86827fc640 1 -- 172.21.15.45:0/702427291 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v82) ==== 118+0+0 (secure 0 0 0) 0x7f8684061040 con 0x7f868c1637b0 2024-05-08T05:49:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.227+0000 7f8693c99640 1 -- 172.21.15.45:0/702427291 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f86700762d0 msgr2=0x7f8670078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.227+0000 7f8693c99640 1 --2- 172.21.15.45:0/702427291 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f86700762d0 0x7f8670078790 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8688003eb0 tx=0x7f8688009040 comp rx=0 tx=0).stop 2024-05-08T05:49:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.227+0000 7f8693c99640 1 -- 172.21.15.45:0/702427291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f868c1637b0 msgr2=0x7f868c1490a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.227+0000 7f8693c99640 1 --2- 172.21.15.45:0/702427291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f868c1637b0 0x7f868c1490a0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f8684007e60 tx=0x7f868400e4b0 comp rx=0 tx=0).stop 2024-05-08T05:49:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.227+0000 7f8693c99640 1 -- 172.21.15.45:0/702427291 shutdown_connections 2024-05-08T05:49:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.227+0000 7f8693c99640 1 --2- 172.21.15.45:0/702427291 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f86700762d0 0x7f8670078790 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:22.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.227+0000 7f8693c99640 1 --2- 172.21.15.45:0/702427291 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f868c06da00 0x7f868c1570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.227+0000 7f8693c99640 1 --2- 172.21.15.45:0/702427291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f868c1637b0 0x7f868c1490a0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.227+0000 7f8693c99640 1 --2- 172.21.15.45:0/702427291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f868c06d620 0x7f868c0cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.227+0000 7f8693c99640 1 -- 172.21.15.45:0/702427291 >> 172.21.15.45:0/702427291 conn(0x7f868c0728c0 msgr2=0x7f868c06c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.227+0000 7f8693c99640 1 -- 172.21.15.45:0/702427291 shutdown_connections 2024-05-08T05:49:22.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.227+0000 7f8693c99640 1 -- 172.21.15.45:0/702427291 wait complete. 2024-05-08T05:49:22.243 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T05:49:22.243 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T05:49:22.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.357+0000 7f45c162a640 1 Processor -- start 2024-05-08T05:49:22.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.357+0000 7f45c162a640 1 -- start start 2024-05-08T05:49:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.357+0000 7f45c162a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f45bc0b5ed0 0x7f45bc0b7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:22.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.357+0000 7f45c162a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45bc0d18e0 0x7f45bc0d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.357+0000 7f45c162a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45bc0d07f0 con 0x7f45bc162910 2024-05-08T05:49:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.357+0000 7f45c162a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f45bc0d2be0 con 0x7f45bc0b5ed0 2024-05-08T05:49:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.357+0000 7f45c162a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f45bc162cf0 con 0x7f45bc0d18e0 2024-05-08T05:49:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.358+0000 7f45baffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f45bc162910 0x7f45bc0d5970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49844/0 (socket says 172.21.15.45:49844) 2024-05-08T05:49:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.358+0000 7f45baffd640 1 -- 172.21.15.45:0/12953869 learned_addr learned my addr 172.21.15.45:0/12953869 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.358+0000 7f45ba7fc640 1 --2- 172.21.15.45:0/12953869 >> v2:172.21.15.45:3300/0 conn(0x7f45bc0b5ed0 0x7f45bc0b7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.358+0000 7f45bb7fe640 1 --2- 172.21.15.45:0/12953869 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45bc0d18e0 0x7f45bc0d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.358+0000 7f45b9ffb640 1 -- 172.21.15.45:0/12953869 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3193546381 0 0) 0x7f45bc0d07f0 con 0x7f45bc162910 2024-05-08T05:49:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.358+0000 7f45b9ffb640 1 -- 172.21.15.45:0/12953869 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45a8003520 con 0x7f45bc162910 2024-05-08T05:49:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.358+0000 7f45ba7fc640 1 -- 172.21.15.45:0/12953869 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45bc0d18e0 msgr2=0x7f45bc0d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.358+0000 7f45ba7fc640 1 --2- 172.21.15.45:0/12953869 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45bc0d18e0 0x7f45bc0d1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.358+0000 7f45ba7fc640 1 -- 172.21.15.45:0/12953869 >> v1:172.21.15.45:6789/0 conn(0x7f45bc162910 legacy=0x7f45bc0d5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.358+0000 7f45ba7fc640 1 -- 172.21.15.45:0/12953869 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc0d0ad0 con 0x7f45bc0b5ed0 2024-05-08T05:49:22.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.359+0000 7f45ba7fc640 1 --2- 172.21.15.45:0/12953869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45bc0b5ed0 0x7f45bc0b7e60 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f45a40097d0 tx=0x7f45a402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f2114576d25669 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.359+0000 7f45b9ffb640 1 -- 172.21.15.45:0/12953869 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4037070 con 0x7f45bc0b5ed0 2024-05-08T05:49:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.359+0000 7f45b9ffb640 1 -- 172.21.15.45:0/12953869 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a4032440 con 0x7f45bc0b5ed0 2024-05-08T05:49:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.359+0000 7f45b9ffb640 1 -- 172.21.15.45:0/12953869 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4032810 con 0x7f45bc0b5ed0 2024-05-08T05:49:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.359+0000 7f45c162a640 1 -- 172.21.15.45:0/12953869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45bc0b5ed0 msgr2=0x7f45bc0b7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.359+0000 7f45c162a640 1 --2- 172.21.15.45:0/12953869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45bc0b5ed0 0x7f45bc0b7e60 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f45a40097d0 tx=0x7f45a402a1c0 comp rx=0 tx=0).stop 2024-05-08T05:49:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.359+0000 7f45c162a640 1 -- 172.21.15.45:0/12953869 shutdown_connections 2024-05-08T05:49:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.359+0000 7f45c162a640 1 --2- 172.21.15.45:0/12953869 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45bc0d18e0 0x7f45bc0d1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.359+0000 7f45c162a640 1 --2- 172.21.15.45:0/12953869 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45bc0b5ed0 0x7f45bc0b7e60 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.359+0000 7f45c162a640 1 -- 172.21.15.45:0/12953869 >> 172.21.15.45:0/12953869 conn(0x7f45bc09e3e0 msgr2=0x7f45bc0d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.359+0000 7f45c162a640 1 -- 172.21.15.45:0/12953869 shutdown_connections 2024-05-08T05:49:22.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.359+0000 7f45c162a640 1 -- 172.21.15.45:0/12953869 wait complete. 2024-05-08T05:49:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.361+0000 7f45c162a640 1 Processor -- start 2024-05-08T05:49:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.361+0000 7f45c162a640 1 -- start start 2024-05-08T05:49:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.362+0000 7f45c162a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45bc0b5ed0 0x7f45bc1594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.362+0000 7f45c162a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45bc0d18e0 0x7f45bc14da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.362+0000 7f45c162a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45bc162910 0x7f45bc15d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.362+0000 7f45c162a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc0d3350 con 0x7f45bc0d18e0 2024-05-08T05:49:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.362+0000 7f45c162a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f45bc0d2e30 con 0x7f45bc0b5ed0 2024-05-08T05:49:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.362+0000 7f45c162a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f45bc0d34d0 con 0x7f45bc162910 2024-05-08T05:49:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.362+0000 7f45ba7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45bc0d18e0 0x7f45bc14da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:22.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.362+0000 7f45ba7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45bc0d18e0 0x7f45bc14da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45294/0 (socket says 172.21.15.45:45294) 2024-05-08T05:49:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.362+0000 7f45ba7fc640 1 -- 172.21.15.45:0/1773360480 learned_addr learned my addr 172.21.15.45:0/1773360480 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.362+0000 7f45baffd640 1 --2- 172.21.15.45:0/1773360480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45bc0b5ed0 0x7f45bc1594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.362+0000 7f45bb7fe640 1 --2- 172.21.15.45:0/1773360480 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45bc162910 0x7f45bc15d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.362+0000 7f45ba7fc640 1 -- 172.21.15.45:0/1773360480 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45bc162910 msgr2=0x7f45bc15d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.362+0000 7f45ba7fc640 1 --2- 172.21.15.45:0/1773360480 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45bc162910 0x7f45bc15d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.362+0000 7f45ba7fc640 1 -- 172.21.15.45:0/1773360480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45bc0b5ed0 msgr2=0x7f45bc1594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.362+0000 7f45ba7fc640 1 --2- 172.21.15.45:0/1773360480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45bc0b5ed0 0x7f45bc1594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.362+0000 7f45ba7fc640 1 -- 172.21.15.45:0/1773360480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45bc15b260 con 0x7f45bc0d18e0 2024-05-08T05:49:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.362+0000 7f45ba7fc640 1 --2- 172.21.15.45:0/1773360480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45bc0d18e0 0x7f45bc14da70 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f45a40098f0 tx=0x7f45a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.362+0000 7f45b37fe640 1 -- 172.21.15.45:0/1773360480 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4037040 con 0x7f45bc0d18e0 2024-05-08T05:49:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.363+0000 7f45b37fe640 1 -- 172.21.15.45:0/1773360480 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a4042070 con 0x7f45bc0d18e0 2024-05-08T05:49:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.363+0000 7f45b37fe640 1 -- 172.21.15.45:0/1773360480 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a403b930 con 0x7f45bc0d18e0 2024-05-08T05:49:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.363+0000 7f45c162a640 1 -- 172.21.15.45:0/1773360480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45bc15b540 con 0x7f45bc0d18e0 2024-05-08T05:49:22.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.363+0000 7f45c162a640 1 -- 172.21.15.45:0/1773360480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45bc15bb00 con 0x7f45bc0d18e0 2024-05-08T05:49:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.364+0000 7f45c162a640 1 -- 172.21.15.45:0/1773360480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45bc0693f0 con 0x7f45bc0d18e0 2024-05-08T05:49:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.364+0000 7f45b37fe640 1 -- 172.21.15.45:0/1773360480 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f45a403bad0 con 0x7f45bc0d18e0 2024-05-08T05:49:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.365+0000 7f45b37fe640 1 --2- 172.21.15.45:0/1773360480 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f459c0762d0 0x7f459c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.365+0000 7f45b37fe640 1 -- 172.21.15.45:0/1773360480 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f45a40b8030 con 0x7f45bc0d18e0 2024-05-08T05:49:22.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.365+0000 7f45baffd640 1 --2- 172.21.15.45:0/1773360480 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f459c0762d0 0x7f459c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:22.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.365+0000 7f45baffd640 1 --2- 172.21.15.45:0/1773360480 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f459c0762d0 0x7f459c078790 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f45b4005880 tx=0x7f45b4006aa0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:22.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.369+0000 7f45b37fe640 1 -- 172.21.15.45:0/1773360480 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f45a4081110 con 0x7f45bc0d18e0 2024-05-08T05:49:22.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:22.597+0000 7f45c162a640 1 -- 172.21.15.45:0/1773360480 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f45bc0d2690 con 0x7f45bc0d18e0 2024-05-08T05:49:23.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.230+0000 7f45b37fe640 1 -- 172.21.15.45:0/1773360480 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v83) ==== 122+0+0 (secure 0 0 0) 0x7f45a4085410 con 0x7f45bc0d18e0 2024-05-08T05:49:23.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.234+0000 7f45c162a640 1 -- 172.21.15.45:0/1773360480 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f459c0762d0 msgr2=0x7f459c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:23.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.234+0000 7f45c162a640 1 --2- 172.21.15.45:0/1773360480 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f459c0762d0 0x7f459c078790 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f45b4005880 tx=0x7f45b4006aa0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.234+0000 7f45c162a640 1 -- 172.21.15.45:0/1773360480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45bc0d18e0 msgr2=0x7f45bc14da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:23.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.234+0000 7f45c162a640 1 --2- 172.21.15.45:0/1773360480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45bc0d18e0 0x7f45bc14da70 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f45a40098f0 tx=0x7f45a40046c0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.234+0000 7f45c162a640 1 -- 172.21.15.45:0/1773360480 shutdown_connections 2024-05-08T05:49:23.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.234+0000 7f45c162a640 1 --2- 172.21.15.45:0/1773360480 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f459c0762d0 0x7f459c078790 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.234+0000 7f45c162a640 1 --2- 172.21.15.45:0/1773360480 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45bc162910 0x7f45bc15d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.234+0000 7f45c162a640 1 --2- 172.21.15.45:0/1773360480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45bc0d18e0 0x7f45bc14da70 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.234+0000 7f45c162a640 1 --2- 172.21.15.45:0/1773360480 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45bc0b5ed0 0x7f45bc1594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.234+0000 7f45c162a640 1 -- 172.21.15.45:0/1773360480 >> 172.21.15.45:0/1773360480 conn(0x7f45bc09e3e0 msgr2=0x7f45bc162eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:23.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.234+0000 7f45c162a640 1 -- 172.21.15.45:0/1773360480 shutdown_connections 2024-05-08T05:49:23.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.234+0000 7f45c162a640 1 -- 172.21.15.45:0/1773360480 wait complete. 2024-05-08T05:49:23.247 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T05:49:23.248 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:49:23.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.358+0000 7f9332625640 1 Processor -- start 2024-05-08T05:49:23.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.358+0000 7f9332625640 1 -- start start 2024-05-08T05:49:23.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.359+0000 7f9332625640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f932c0d0ec0 0x7f932c0d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.359+0000 7f9332625640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f932c0d3b00 0x7f932c0b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.359+0000 7f9332625640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f932c0d1760 con 0x7f932c15f380 2024-05-08T05:49:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.359+0000 7f9332625640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f932c0d30f0 con 0x7f932c0d0ec0 2024-05-08T05:49:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.359+0000 7f9332625640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f932c0d2370 con 0x7f932c0d3b00 2024-05-08T05:49:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.359+0000 7f932b7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f932c0d0ec0 0x7f932c0d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.359+0000 7f932b7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f932c0d0ec0 0x7f932c0d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45308/0 (socket says 172.21.15.45:45308) 2024-05-08T05:49:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.359+0000 7f932b7fe640 1 -- 172.21.15.45:0/2288402868 learned_addr learned my addr 172.21.15.45:0/2288402868 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.360+0000 7f932b7fe640 1 -- 172.21.15.45:0/2288402868 >> v2:172.21.15.63:3300/0 conn(0x7f932c0d3b00 msgr2=0x7f932c0b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.360+0000 7f932b7fe640 1 --2- 172.21.15.45:0/2288402868 >> v2:172.21.15.63:3300/0 conn(0x7f932c0d3b00 0x7f932c0b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.360+0000 7f932b7fe640 1 -- 172.21.15.45:0/2288402868 >> v1:172.21.15.63:6789/0 conn(0x7f932c15f380 legacy=0x7f932c0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:23.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.360+0000 7f932b7fe640 1 -- 172.21.15.45:0/2288402868 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c0d1a40 con 0x7f932c0d0ec0 2024-05-08T05:49:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.360+0000 7f932b7fe640 1 --2- 172.21.15.45:0/2288402868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932c0d0ec0 0x7f932c0d35c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f932c163f60 tx=0x7f932002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a408604bb3cc008 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.360+0000 7f932affd640 1 -- 172.21.15.45:0/2288402868 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320037070 con 0x7f932c0d0ec0 2024-05-08T05:49:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.361+0000 7f932affd640 1 -- 172.21.15.45:0/2288402868 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9320032440 con 0x7f932c0d0ec0 2024-05-08T05:49:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.361+0000 7f932affd640 1 -- 172.21.15.45:0/2288402868 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320032810 con 0x7f932c0d0ec0 2024-05-08T05:49:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.361+0000 7f9332625640 1 -- 172.21.15.45:0/2288402868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932c0d0ec0 msgr2=0x7f932c0d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.361+0000 7f9332625640 1 --2- 172.21.15.45:0/2288402868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932c0d0ec0 0x7f932c0d35c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f932c163f60 tx=0x7f932002a1e0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.361+0000 7f9332625640 1 -- 172.21.15.45:0/2288402868 shutdown_connections 2024-05-08T05:49:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.361+0000 7f9332625640 1 --2- 172.21.15.45:0/2288402868 >> v2:172.21.15.63:3300/0 conn(0x7f932c0d3b00 0x7f932c0b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.361+0000 7f9332625640 1 --2- 172.21.15.45:0/2288402868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932c0d0ec0 0x7f932c0d35c0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.361+0000 7f9332625640 1 -- 172.21.15.45:0/2288402868 >> 172.21.15.45:0/2288402868 conn(0x7f932c0d4fe0 msgr2=0x7f932c0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.361+0000 7f9332625640 1 -- 172.21.15.45:0/2288402868 shutdown_connections 2024-05-08T05:49:23.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.361+0000 7f9332625640 1 -- 172.21.15.45:0/2288402868 wait complete. 2024-05-08T05:49:23.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.362+0000 7f9332625640 1 Processor -- start 2024-05-08T05:49:23.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.363+0000 7f9332625640 1 -- start start 2024-05-08T05:49:23.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.363+0000 7f9332625640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932c0d0ec0 0x7f932c0a2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:23.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.363+0000 7f9332625640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f932c0d3b00 0x7f932c1f6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:23.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.363+0000 7f9332625640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f932c15f380 0x7f932c0cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:23.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.363+0000 7f9332625640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f932c0d2000 con 0x7f932c0d0ec0 2024-05-08T05:49:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.363+0000 7f9332625640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f932c0d3f30 con 0x7f932c0d3b00 2024-05-08T05:49:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.363+0000 7f9332625640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f932c15f760 con 0x7f932c15f380 2024-05-08T05:49:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.363+0000 7f932bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932c0d0ec0 0x7f932c0a2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.364+0000 7f932bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932c0d0ec0 0x7f932c0a2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45320/0 (socket says 172.21.15.45:45320) 2024-05-08T05:49:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.364+0000 7f932bfff640 1 -- 172.21.15.45:0/2309660649 learned_addr learned my addr 172.21.15.45:0/2309660649 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.364+0000 7f932b7fe640 1 --2- 172.21.15.45:0/2309660649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f932c0d3b00 0x7f932c1f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.364+0000 7f9330bc2640 1 --2- 172.21.15.45:0/2309660649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f932c15f380 0x7f932c0cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.364+0000 7f932bfff640 1 -- 172.21.15.45:0/2309660649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f932c15f380 msgr2=0x7f932c0cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.364+0000 7f932bfff640 1 --2- 172.21.15.45:0/2309660649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f932c15f380 0x7f932c0cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.364+0000 7f932bfff640 1 -- 172.21.15.45:0/2309660649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f932c0d3b00 msgr2=0x7f932c1f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.364+0000 7f932bfff640 1 --2- 172.21.15.45:0/2309660649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f932c0d3b00 0x7f932c1f6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.364+0000 7f932bfff640 1 -- 172.21.15.45:0/2309660649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c09fe60 con 0x7f932c0d0ec0 2024-05-08T05:49:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.364+0000 7f932bfff640 1 --2- 172.21.15.45:0/2309660649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932c0d0ec0 0x7f932c0a2770 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f931c00c410 tx=0x7f931c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:23.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.364+0000 7f9328ff9640 1 -- 172.21.15.45:0/2309660649 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c017070 con 0x7f932c0d0ec0 2024-05-08T05:49:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.364+0000 7f9328ff9640 1 -- 172.21.15.45:0/2309660649 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f931c0035f0 con 0x7f932c0d0ec0 2024-05-08T05:49:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.364+0000 7f9328ff9640 1 -- 172.21.15.45:0/2309660649 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c003a20 con 0x7f932c0d0ec0 2024-05-08T05:49:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.364+0000 7f9332625640 1 -- 172.21.15.45:0/2309660649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932c0a00d0 con 0x7f932c0d0ec0 2024-05-08T05:49:23.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.364+0000 7f9332625640 1 -- 172.21.15.45:0/2309660649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f932c0a0690 con 0x7f932c0d0ec0 2024-05-08T05:49:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.366+0000 7f9332625640 1 -- 172.21.15.45:0/2309660649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f932c0693f0 con 0x7f932c0d0ec0 2024-05-08T05:49:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.366+0000 7f9328ff9640 1 -- 172.21.15.45:0/2309660649 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f931c003bc0 con 0x7f932c0d0ec0 2024-05-08T05:49:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.367+0000 7f9328ff9640 1 --2- 172.21.15.45:0/2309660649 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f93180762d0 0x7f9318078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.367+0000 7f9328ff9640 1 -- 172.21.15.45:0/2309660649 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f931c096ab0 con 0x7f932c0d0ec0 2024-05-08T05:49:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.367+0000 7f932b7fe640 1 --2- 172.21.15.45:0/2309660649 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f93180762d0 0x7f9318078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:23.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.367+0000 7f932b7fe640 1 --2- 172.21.15.45:0/2309660649 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f93180762d0 0x7f9318078790 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9320034000 tx=0x7f932002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:23.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.371+0000 7f9328ff9640 1 -- 172.21.15.45:0/2309660649 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f931c060280 con 0x7f932c0d0ec0 2024-05-08T05:49:23.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.599+0000 7f9332625640 1 -- 172.21.15.45:0/2309660649 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f932c0d3760 con 0x7f932c0d0ec0 2024-05-08T05:49:23.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.599+0000 7f9328ff9640 1 -- 172.21.15.45:0/2309660649 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5180 (secure 0 0 0) 0x7f931c0010c0 con 0x7f932c0d0ec0 2024-05-08T05:49:23.600 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:23.600 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3692881261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3692881261},{"type":"v1","addr":"172.21.15.63:6836","nonce":3692881261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7497,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2916399818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2916399818},{"type":"v1","addr":"172.21.15.45:6833","nonce":2916399818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7515,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2978250004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":2978250004},{"type":"v1","addr":"172.21.15.63:6837","nonce":2978250004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:49:19.587326+0000","modified":"2024-05-08T05:49:22.599177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7524":{"gid":7524,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/878092981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":878092981},{"type":"v1","addr":"172.21.15.45:6835","nonce":878092981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7524,"qdb_cluster":[7524]},"id":7}]} 2024-05-08T05:49:23.600 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 83 2024-05-08T05:49:23.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.601+0000 7f9332625640 1 -- 172.21.15.45:0/2309660649 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f93180762d0 msgr2=0x7f9318078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:23.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.601+0000 7f9332625640 1 --2- 172.21.15.45:0/2309660649 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f93180762d0 0x7f9318078790 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9320034000 tx=0x7f932002e000 comp rx=0 tx=0).stop 2024-05-08T05:49:23.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.601+0000 7f9332625640 1 -- 172.21.15.45:0/2309660649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932c0d0ec0 msgr2=0x7f932c0a2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:23.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.601+0000 7f9332625640 1 --2- 172.21.15.45:0/2309660649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932c0d0ec0 0x7f932c0a2770 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f931c00c410 tx=0x7f931c00c8d0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.601+0000 7f9332625640 1 -- 172.21.15.45:0/2309660649 shutdown_connections 2024-05-08T05:49:23.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.601+0000 7f9332625640 1 --2- 172.21.15.45:0/2309660649 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f93180762d0 0x7f9318078790 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.601+0000 7f9332625640 1 --2- 172.21.15.45:0/2309660649 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f932c15f380 0x7f932c0cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.601+0000 7f9332625640 1 --2- 172.21.15.45:0/2309660649 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f932c0d3b00 0x7f932c1f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.601+0000 7f9332625640 1 --2- 172.21.15.45:0/2309660649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f932c0d0ec0 0x7f932c0a2770 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.601+0000 7f9332625640 1 -- 172.21.15.45:0/2309660649 >> 172.21.15.45:0/2309660649 conn(0x7f932c0d4fe0 msgr2=0x7f932c0b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:23.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.602+0000 7f9332625640 1 -- 172.21.15.45:0/2309660649 shutdown_connections 2024-05-08T05:49:23.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.602+0000 7f9332625640 1 -- 172.21.15.45:0/2309660649 wait complete. 2024-05-08T05:49:23.613 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:49:23.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.716+0000 7f82a0ea7640 1 Processor -- start 2024-05-08T05:49:23.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.716+0000 7f82a0ea7640 1 -- start start 2024-05-08T05:49:23.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.717+0000 7f82a0ea7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f829c15ccb0 0x7f829c15d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:23.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.717+0000 7f82a0ea7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f829c0d2ce0 0x7f829c0d3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:23.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.717+0000 7f82a0ea7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f829c0718c0 con 0x7f829c070b90 2024-05-08T05:49:23.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.717+0000 7f82a0ea7640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f829c0afba0 con 0x7f829c15ccb0 2024-05-08T05:49:23.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.717+0000 7f82a0ea7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f829c071ba0 con 0x7f829c0d2ce0 2024-05-08T05:49:23.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.718+0000 7f829a59c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f829c15ccb0 0x7f829c15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:23.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.718+0000 7f829a59c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f829c15ccb0 0x7f829c15d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45328/0 (socket says 172.21.15.45:45328) 2024-05-08T05:49:23.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.718+0000 7f829a59c640 1 -- 172.21.15.45:0/4150680069 learned_addr learned my addr 172.21.15.45:0/4150680069 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:23.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.718+0000 7f8299d9b640 1 --2- 172.21.15.45:0/4150680069 >> v2:172.21.15.63:3300/0 conn(0x7f829c0d2ce0 0x7f829c0d3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:23.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.718+0000 7f829a59c640 1 -- 172.21.15.45:0/4150680069 >> v2:172.21.15.63:3300/0 conn(0x7f829c0d2ce0 msgr2=0x7f829c0d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:23.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.718+0000 7f829a59c640 1 --2- 172.21.15.45:0/4150680069 >> v2:172.21.15.63:3300/0 conn(0x7f829c0d2ce0 0x7f829c0d3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.718+0000 7f829a59c640 1 -- 172.21.15.45:0/4150680069 >> v1:172.21.15.63:6789/0 conn(0x7f829c070b90 legacy=0x7f829c071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:23.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.718+0000 7f829a59c640 1 -- 172.21.15.45:0/4150680069 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829c071e80 con 0x7f829c15ccb0 2024-05-08T05:49:23.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.718+0000 7f8299d9b640 1 --2- 172.21.15.45:0/4150680069 >> v2:172.21.15.63:3300/0 conn(0x7f829c0d2ce0 0x7f829c0d3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:49:23.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.719+0000 7f829a59c640 1 --2- 172.21.15.45:0/4150680069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829c15ccb0 0x7f829c15d090 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f829c0b20c0 tx=0x7f829402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c379bd1bcd0f64e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:23.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.719+0000 7f829959a640 1 -- 172.21.15.45:0/4150680069 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294037070 con 0x7f829c15ccb0 2024-05-08T05:49:23.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.719+0000 7f829959a640 1 -- 172.21.15.45:0/4150680069 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8294032440 con 0x7f829c15ccb0 2024-05-08T05:49:23.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.719+0000 7f829959a640 1 -- 172.21.15.45:0/4150680069 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294032810 con 0x7f829c15ccb0 2024-05-08T05:49:23.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.719+0000 7f82a0ea7640 1 -- 172.21.15.45:0/4150680069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829c15ccb0 msgr2=0x7f829c15d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:23.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.719+0000 7f82a0ea7640 1 --2- 172.21.15.45:0/4150680069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829c15ccb0 0x7f829c15d090 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f829c0b20c0 tx=0x7f829402a140 comp rx=0 tx=0).stop 2024-05-08T05:49:23.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.719+0000 7f82a0ea7640 1 -- 172.21.15.45:0/4150680069 shutdown_connections 2024-05-08T05:49:23.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.719+0000 7f82a0ea7640 1 --2- 172.21.15.45:0/4150680069 >> v2:172.21.15.63:3300/0 conn(0x7f829c0d2ce0 0x7f829c0d3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.719+0000 7f82a0ea7640 1 --2- 172.21.15.45:0/4150680069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829c15ccb0 0x7f829c15d090 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.719+0000 7f82a0ea7640 1 -- 172.21.15.45:0/4150680069 >> 172.21.15.45:0/4150680069 conn(0x7f829c09ba40 msgr2=0x7f829c09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:23.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.719+0000 7f82a0ea7640 1 -- 172.21.15.45:0/4150680069 shutdown_connections 2024-05-08T05:49:23.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.719+0000 7f82a0ea7640 1 -- 172.21.15.45:0/4150680069 wait complete. 2024-05-08T05:49:23.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.721+0000 7f82a0ea7640 1 Processor -- start 2024-05-08T05:49:23.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.721+0000 7f82a0ea7640 1 -- start start 2024-05-08T05:49:23.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f82a0ea7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f829c070b90 0x7f829c15e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:23.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f82a0ea7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f829c0d2ce0 0x7f829c0cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:23.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f82a0ea7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829c15ccb0 0x7f829c0cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:23.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f82a0ea7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f829c0cf980 con 0x7f829c15ccb0 2024-05-08T05:49:23.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f82a0ea7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f829c0d3660 con 0x7f829c0d2ce0 2024-05-08T05:49:23.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f82a0ea7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f829c0cfb00 con 0x7f829c070b90 2024-05-08T05:49:23.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f829ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829c15ccb0 0x7f829c0cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:23.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f829ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829c15ccb0 0x7f829c0cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45340/0 (socket says 172.21.15.45:45340) 2024-05-08T05:49:23.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f829ad9d640 1 -- 172.21.15.45:0/3384456762 learned_addr learned my addr 172.21.15.45:0/3384456762 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:23.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f829a59c640 1 --2- 172.21.15.45:0/3384456762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f829c070b90 0x7f829c15e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:23.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f8299d9b640 1 --2- 172.21.15.45:0/3384456762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f829c0d2ce0 0x7f829c0cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:23.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f829ad9d640 1 -- 172.21.15.45:0/3384456762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f829c070b90 msgr2=0x7f829c15e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:23.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f829ad9d640 1 --2- 172.21.15.45:0/3384456762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f829c070b90 0x7f829c15e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f829ad9d640 1 -- 172.21.15.45:0/3384456762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f829c0d2ce0 msgr2=0x7f829c0cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:23.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f829ad9d640 1 --2- 172.21.15.45:0/3384456762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f829c0d2ce0 0x7f829c0cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f829ad9d640 1 -- 172.21.15.45:0/3384456762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829c15e9e0 con 0x7f829c15ccb0 2024-05-08T05:49:23.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f829ad9d640 1 --2- 172.21.15.45:0/3384456762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829c15ccb0 0x7f829c0cc0a0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f828c00b970 tx=0x7f828c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:23.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f8292ffd640 1 -- 172.21.15.45:0/3384456762 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c019070 con 0x7f829c15ccb0 2024-05-08T05:49:23.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f8292ffd640 1 -- 172.21.15.45:0/3384456762 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f828c00cb40 con 0x7f829c15ccb0 2024-05-08T05:49:23.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f8292ffd640 1 -- 172.21.15.45:0/3384456762 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c014440 con 0x7f829c15ccb0 2024-05-08T05:49:23.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f82a0ea7640 1 -- 172.21.15.45:0/3384456762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829c15ecc0 con 0x7f829c15ccb0 2024-05-08T05:49:23.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.722+0000 7f82a0ea7640 1 -- 172.21.15.45:0/3384456762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f829c15f280 con 0x7f829c15ccb0 2024-05-08T05:49:23.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.723+0000 7f82a0ea7640 1 -- 172.21.15.45:0/3384456762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f829c069b00 con 0x7f829c15ccb0 2024-05-08T05:49:23.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.724+0000 7f8292ffd640 1 -- 172.21.15.45:0/3384456762 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f828c01c070 con 0x7f829c15ccb0 2024-05-08T05:49:23.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.725+0000 7f8292ffd640 1 --2- 172.21.15.45:0/3384456762 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f827c0762d0 0x7f827c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:23.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.725+0000 7f8292ffd640 1 -- 172.21.15.45:0/3384456762 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f828c097390 con 0x7f829c15ccb0 2024-05-08T05:49:23.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.725+0000 7f829a59c640 1 --2- 172.21.15.45:0/3384456762 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f827c0762d0 0x7f827c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:23.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.725+0000 7f829a59c640 1 --2- 172.21.15.45:0/3384456762 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f827c0762d0 0x7f827c078790 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8294034000 tx=0x7f829402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:23.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.729+0000 7f8292ffd640 1 -- 172.21.15.45:0/3384456762 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f828c060510 con 0x7f829c15ccb0 2024-05-08T05:49:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.922+0000 7f82a0ea7640 1 -- 172.21.15.45:0/3384456762 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f829c070380 con 0x7f829c15ccb0 2024-05-08T05:49:23.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.924+0000 7f8292ffd640 1 -- 172.21.15.45:0/3384456762 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16836 (secure 0 0 0) 0x7f828c064810 con 0x7f829c15ccb0 2024-05-08T05:49:23.924 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:23.924 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":104,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:49:21.213946+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:49:17.241406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-08T05:49:18.542707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:49:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.926+0000 7f82a0ea7640 1 -- 172.21.15.45:0/3384456762 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f827c0762d0 msgr2=0x7f827c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.926+0000 7f82a0ea7640 1 --2- 172.21.15.45:0/3384456762 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f827c0762d0 0x7f827c078790 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8294034000 tx=0x7f829402e000 comp rx=0 tx=0).stop 2024-05-08T05:49:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.926+0000 7f82a0ea7640 1 -- 172.21.15.45:0/3384456762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829c15ccb0 msgr2=0x7f829c0cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.926+0000 7f82a0ea7640 1 --2- 172.21.15.45:0/3384456762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829c15ccb0 0x7f829c0cc0a0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f828c00b970 tx=0x7f828c00be30 comp rx=0 tx=0).stop 2024-05-08T05:49:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.926+0000 7f82a0ea7640 1 -- 172.21.15.45:0/3384456762 shutdown_connections 2024-05-08T05:49:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.926+0000 7f82a0ea7640 1 --2- 172.21.15.45:0/3384456762 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f827c0762d0 0x7f827c078790 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.926+0000 7f82a0ea7640 1 --2- 172.21.15.45:0/3384456762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f829c15ccb0 0x7f829c0cc0a0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.926+0000 7f82a0ea7640 1 --2- 172.21.15.45:0/3384456762 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f829c0d2ce0 0x7f829c0cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.926+0000 7f82a0ea7640 1 --2- 172.21.15.45:0/3384456762 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f829c070b90 0x7f829c15e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.926+0000 7f82a0ea7640 1 -- 172.21.15.45:0/3384456762 >> 172.21.15.45:0/3384456762 conn(0x7f829c09ba40 msgr2=0x7f829c0b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.926+0000 7f82a0ea7640 1 -- 172.21.15.45:0/3384456762 shutdown_connections 2024-05-08T05:49:23.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:23.926+0000 7f82a0ea7640 1 -- 172.21.15.45:0/3384456762 wait complete. 2024-05-08T05:49:23.937 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T05:49:23.937 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:49:24.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.041+0000 7f2cf72bd640 1 Processor -- start 2024-05-08T05:49:24.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.041+0000 7f2cf72bd640 1 -- start start 2024-05-08T05:49:24.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.042+0000 7f2cf72bd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2cf01629a0 0x7f2cf00d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:24.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.042+0000 7f2cf72bd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cf00d2dc0 0x7f2cf00d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:24.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.042+0000 7f2cf72bd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cf00d3330 con 0x7f2cf00c8ed0 2024-05-08T05:49:24.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.042+0000 7f2cf72bd640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2cf00d2170 con 0x7f2cf01629a0 2024-05-08T05:49:24.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.042+0000 7f2cf72bd640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2cf00d2450 con 0x7f2cf00d2dc0 2024-05-08T05:49:24.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.042+0000 7f2cf585a640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2cf00c8ed0 0x7f2cf00c9360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49848/0 (socket says 172.21.15.45:49848) 2024-05-08T05:49:24.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.042+0000 7f2cf585a640 1 -- 172.21.15.45:0/3488864077 learned_addr learned my addr 172.21.15.45:0/3488864077 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:24.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.042+0000 7f2cf5059640 1 --2- 172.21.15.45:0/3488864077 >> v2:172.21.15.63:3300/0 conn(0x7f2cf01629a0 0x7f2cf00d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:24.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.042+0000 7f2cf4858640 1 --2- 172.21.15.45:0/3488864077 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cf00d2dc0 0x7f2cf00d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:24.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.043+0000 7f2ce7fff640 1 -- 172.21.15.45:0/3488864077 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2867402825 0 0) 0x7f2cf00d3330 con 0x7f2cf00c8ed0 2024-05-08T05:49:24.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.043+0000 7f2ce7fff640 1 -- 172.21.15.45:0/3488864077 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2cdc003520 con 0x7f2cf00c8ed0 2024-05-08T05:49:24.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.043+0000 7f2cf5059640 1 -- 172.21.15.45:0/3488864077 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cf00d2dc0 msgr2=0x7f2cf00d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:24.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.043+0000 7f2cf5059640 1 --2- 172.21.15.45:0/3488864077 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cf00d2dc0 0x7f2cf00d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.043+0000 7f2cf5059640 1 -- 172.21.15.45:0/3488864077 >> v1:172.21.15.45:6789/0 conn(0x7f2cf00c8ed0 legacy=0x7f2cf00c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:24.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.043+0000 7f2cf5059640 1 -- 172.21.15.45:0/3488864077 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf00d2730 con 0x7f2cf01629a0 2024-05-08T05:49:24.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.043+0000 7f2cf5059640 1 --2- 172.21.15.45:0/3488864077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf01629a0 0x7f2cf00d35c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2cec0097a0 tx=0x7f2cec02a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cde29830f76474d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:24.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.044+0000 7f2ce7fff640 1 -- 172.21.15.45:0/3488864077 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec037070 con 0x7f2cf01629a0 2024-05-08T05:49:24.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.044+0000 7f2ce7fff640 1 -- 172.21.15.45:0/3488864077 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cec02adb0 con 0x7f2cf01629a0 2024-05-08T05:49:24.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.044+0000 7f2ce7fff640 1 -- 172.21.15.45:0/3488864077 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec032660 con 0x7f2cf01629a0 2024-05-08T05:49:24.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.044+0000 7f2cf72bd640 1 -- 172.21.15.45:0/3488864077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf01629a0 msgr2=0x7f2cf00d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:24.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.044+0000 7f2cf72bd640 1 --2- 172.21.15.45:0/3488864077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf01629a0 0x7f2cf00d35c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2cec0097a0 tx=0x7f2cec02a060 comp rx=0 tx=0).stop 2024-05-08T05:49:24.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.044+0000 7f2cf72bd640 1 -- 172.21.15.45:0/3488864077 shutdown_connections 2024-05-08T05:49:24.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.044+0000 7f2cf72bd640 1 --2- 172.21.15.45:0/3488864077 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cf00d2dc0 0x7f2cf00d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.044+0000 7f2cf72bd640 1 --2- 172.21.15.45:0/3488864077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf01629a0 0x7f2cf00d35c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.044+0000 7f2cf72bd640 1 -- 172.21.15.45:0/3488864077 >> 172.21.15.45:0/3488864077 conn(0x7f2cf00d4fe0 msgr2=0x7f2cf00d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:24.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.044+0000 7f2cf72bd640 1 -- 172.21.15.45:0/3488864077 shutdown_connections 2024-05-08T05:49:24.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.044+0000 7f2cf72bd640 1 -- 172.21.15.45:0/3488864077 wait complete. 2024-05-08T05:49:24.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.045+0000 7f2cf72bd640 1 Processor -- start 2024-05-08T05:49:24.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.046+0000 7f2cf72bd640 1 -- start start 2024-05-08T05:49:24.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.046+0000 7f2cf72bd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cf00c8ed0 0x7f2cf01f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:24.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.046+0000 7f2cf72bd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cf00d2dc0 0x7f2cf014e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:24.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.046+0000 7f2cf72bd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf01629a0 0x7f2cf015c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:24.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.046+0000 7f2cf72bd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf00cf440 con 0x7f2cf00c8ed0 2024-05-08T05:49:24.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.046+0000 7f2cf72bd640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf00cf2c0 con 0x7f2cf01629a0 2024-05-08T05:49:24.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.046+0000 7f2cf72bd640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2cf00d0cb0 con 0x7f2cf00d2dc0 2024-05-08T05:49:24.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.046+0000 7f2cf5059640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cf00c8ed0 0x7f2cf01f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:24.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.046+0000 7f2cf5059640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cf00c8ed0 0x7f2cf01f4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45352/0 (socket says 172.21.15.45:45352) 2024-05-08T05:49:24.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.046+0000 7f2cf5059640 1 -- 172.21.15.45:0/897192275 learned_addr learned my addr 172.21.15.45:0/897192275 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:24.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.047+0000 7f2cf585a640 1 --2- 172.21.15.45:0/897192275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf01629a0 0x7f2cf015c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:24.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.047+0000 7f2cf4858640 1 --2- 172.21.15.45:0/897192275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cf00d2dc0 0x7f2cf014e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:24.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.047+0000 7f2cf5059640 1 -- 172.21.15.45:0/897192275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cf00d2dc0 msgr2=0x7f2cf014e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:24.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.047+0000 7f2cf5059640 1 --2- 172.21.15.45:0/897192275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cf00d2dc0 0x7f2cf014e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.047+0000 7f2cf5059640 1 -- 172.21.15.45:0/897192275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf01629a0 msgr2=0x7f2cf015c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:24.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.047+0000 7f2cf5059640 1 --2- 172.21.15.45:0/897192275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf01629a0 0x7f2cf015c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.047+0000 7f2cf5059640 1 -- 172.21.15.45:0/897192275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf014eb10 con 0x7f2cf00c8ed0 2024-05-08T05:49:24.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.047+0000 7f2cf5059640 1 --2- 172.21.15.45:0/897192275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cf00c8ed0 0x7f2cf01f4490 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2cec035040 tx=0x7f2cec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:24.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.047+0000 7f2ce5ffb640 1 -- 172.21.15.45:0/897192275 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec041070 con 0x7f2cf00c8ed0 2024-05-08T05:49:24.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.047+0000 7f2ce5ffb640 1 -- 172.21.15.45:0/897192275 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cec03c460 con 0x7f2cf00c8ed0 2024-05-08T05:49:24.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.047+0000 7f2ce5ffb640 1 -- 172.21.15.45:0/897192275 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec037040 con 0x7f2cf00c8ed0 2024-05-08T05:49:24.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.047+0000 7f2cf72bd640 1 -- 172.21.15.45:0/897192275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cf014edf0 con 0x7f2cf00c8ed0 2024-05-08T05:49:24.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.047+0000 7f2cf72bd640 1 -- 172.21.15.45:0/897192275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cf01522f0 con 0x7f2cf00c8ed0 2024-05-08T05:49:24.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.048+0000 7f2cf72bd640 1 -- 172.21.15.45:0/897192275 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cf014bfe0 con 0x7f2cf00c8ed0 2024-05-08T05:49:24.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.049+0000 7f2ce5ffb640 1 -- 172.21.15.45:0/897192275 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2cec03a070 con 0x7f2cf00c8ed0 2024-05-08T05:49:24.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.049+0000 7f2ce5ffb640 1 --2- 172.21.15.45:0/897192275 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2cd40762d0 0x7f2cd4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:24.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.050+0000 7f2ce5ffb640 1 -- 172.21.15.45:0/897192275 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f2cec0b78b0 con 0x7f2cf00c8ed0 2024-05-08T05:49:24.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.050+0000 7f2cf4858640 1 --2- 172.21.15.45:0/897192275 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2cd40762d0 0x7f2cd4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:24.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.050+0000 7f2cf4858640 1 --2- 172.21.15.45:0/897192275 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2cd40762d0 0x7f2cd4078790 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2cf01611a0 tx=0x7f2ce0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:24.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.054+0000 7f2ce5ffb640 1 -- 172.21.15.45:0/897192275 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2cec080a20 con 0x7f2cf00c8ed0 2024-05-08T05:49:24.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.246+0000 7f2cf72bd640 1 -- 172.21.15.45:0/897192275 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2cf006e4c0 con 0x7f2cd40762d0 2024-05-08T05:49:24.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.252+0000 7f2ce5ffb640 1 -- 172.21.15.45:0/897192275 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242520 (secure 0 0 0) 0x7f2cf006e4c0 con 0x7f2cd40762d0 2024-05-08T05:49:24.255 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:24.256 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:49:24.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.255+0000 7f2cf72bd640 1 -- 172.21.15.45:0/897192275 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2cd40762d0 msgr2=0x7f2cd4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:24.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.255+0000 7f2cf72bd640 1 --2- 172.21.15.45:0/897192275 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2cd40762d0 0x7f2cd4078790 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2cf01611a0 tx=0x7f2ce0009190 comp rx=0 tx=0).stop 2024-05-08T05:49:24.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.255+0000 7f2cf72bd640 1 -- 172.21.15.45:0/897192275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cf00c8ed0 msgr2=0x7f2cf01f4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:24.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.255+0000 7f2cf72bd640 1 --2- 172.21.15.45:0/897192275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cf00c8ed0 0x7f2cf01f4490 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2cec035040 tx=0x7f2cec0046c0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.255+0000 7f2cf72bd640 1 -- 172.21.15.45:0/897192275 shutdown_connections 2024-05-08T05:49:24.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.255+0000 7f2cf72bd640 1 --2- 172.21.15.45:0/897192275 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2cd40762d0 0x7f2cd4078790 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.255+0000 7f2cf72bd640 1 --2- 172.21.15.45:0/897192275 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2cf00d2dc0 0x7f2cf014e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.255+0000 7f2cf72bd640 1 --2- 172.21.15.45:0/897192275 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2cf01629a0 0x7f2cf015c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.255+0000 7f2cf72bd640 1 --2- 172.21.15.45:0/897192275 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2cf00c8ed0 0x7f2cf01f4490 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.255+0000 7f2cf72bd640 1 -- 172.21.15.45:0/897192275 >> 172.21.15.45:0/897192275 conn(0x7f2cf00d4fe0 msgr2=0x7f2cf01612d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:24.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.255+0000 7f2cf72bd640 1 -- 172.21.15.45:0/897192275 shutdown_connections 2024-05-08T05:49:24.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.255+0000 7f2cf72bd640 1 -- 172.21.15.45:0/897192275 wait complete. 2024-05-08T05:49:24.266 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-05-08T05:49:22.948120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":108,"acting":108,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260404,"kb_used_data":10932,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714316,"statfs":{"total":773094113280,"available":772827459584,"internally_reserved":0,"allocated":11194368,"data_stored":5794375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.748919"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.210877+0000","last_change":"2024-05-08T05:49:19.210877+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.210877+0000","last_undegraded":"2024-05-08T05:49:19.210877+0000","last_fullsized":"2024-05-08T05:49:19.210877+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.209934+0000","last_change":"2024-05-08T05:49:19.209934+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.209934+0000","last_undegraded":"2024-05-08T05:49:19.209934+0000","last_fullsized":"2024-05-08T05:49:19.209934+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.211413+0000","last_change":"2024-05-08T05:49:19.211413+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.211413+0000","last_undegraded":"2024-05-08T05:49:19.211413+0000","last_fullsized":"2024-05-08T05:49:19.211413+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.210766+0000","last_change":"2024-05-08T05:49:19.210766+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.210766+0000","last_undegraded":"2024-05-08T05:49:19.210766+0000","last_fullsized":"2024-05-08T05:49:19.210766+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.211164+0000","last_change":"2024-05-08T05:49:19.211164+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.211164+0000","last_undegraded":"2024-05-08T05:49:19.211164+0000","last_fullsized":"2024-05-08T05:49:19.211164+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.212601+0000","last_change":"2024-05-08T05:49:19.212601+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.212601+0000","last_undegraded":"2024-05-08T05:49:19.212601+0000","last_fullsized":"2024-05-08T05:49:19.212601+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.211903+0000","last_change":"2024-05-08T05:49:19.211903+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.211903+0000","last_undegraded":"2024-05-08T05:49:19.211903+0000","last_fullsized":"2024-05-08T05:49:19.211903+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.214901+0000","last_change":"2024-05-08T05:49:19.214901+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.214901+0000","last_undegraded":"2024-05-08T05:49:19.214901+0000","last_fullsized":"2024-05-08T05:49:19.214901+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.209892+0000","last_change":"2024-05-08T05:49:19.209892+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.209892+0000","last_undegraded":"2024-05-08T05:49:19.209892+0000","last_fullsized":"2024-05-08T05:49:19.209892+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.209907+0000","last_change":"2024-05-08T05:49:19.209907+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.209907+0000","last_undegraded":"2024-05-08T05:49:19.209907+0000","last_fullsized":"2024-05-08T05:49:19.209907+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.211263+0000","last_change":"2024-05-08T05:49:19.211263+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.211263+0000","last_undegraded":"2024-05-08T05:49:19.211263+0000","last_fullsized":"2024-05-08T05:49:19.211263+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.210963+0000","last_change":"2024-05-08T05:49:19.210963+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.210963+0000","last_undegraded":"2024-05-08T05:49:19.210963+0000","last_fullsized":"2024-05-08T05:49:19.210963+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.200329+0000","last_change":"2024-05-08T05:49:18.200329+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.200329+0000","last_undegraded":"2024-05-08T05:49:18.200329+0000","last_fullsized":"2024-05-08T05:49:18.200329+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.206429+0000","last_change":"2024-05-08T05:49:18.206429+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.206429+0000","last_undegraded":"2024-05-08T05:49:18.206429+0000","last_fullsized":"2024-05-08T05:49:18.206429+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.205185+0000","last_change":"2024-05-08T05:49:18.205185+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.205185+0000","last_undegraded":"2024-05-08T05:49:18.205185+0000","last_fullsized":"2024-05-08T05:49:18.205185+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.209205+0000","last_change":"2024-05-08T05:49:19.209205+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.209205+0000","last_undegraded":"2024-05-08T05:49:19.209205+0000","last_fullsized":"2024-05-08T05:49:19.209205+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.203113+0000","last_change":"2024-05-08T05:49:18.203113+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.203113+0000","last_undegraded":"2024-05-08T05:49:18.203113+0000","last_fullsized":"2024-05-08T05:49:18.203113+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.230993+0000","last_change":"2024-05-08T05:49:19.224342+0000","last_active":"2024-05-08T05:49:19.230993+0000","last_peered":"2024-05-08T05:49:19.230993+0000","last_clean":"2024-05-08T05:49:19.230993+0000","last_became_active":"2024-05-08T05:49:19.224020+0000","last_became_peered":"2024-05-08T05:49:19.224020+0000","last_unstale":"2024-05-08T05:49:19.230993+0000","last_undegraded":"2024-05-08T05:49:19.230993+0000","last_fullsized":"2024-05-08T05:49:19.230993+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:00.780658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.223380+0000","last_change":"2024-05-08T05:49:19.219998+0000","last_active":"2024-05-08T05:49:19.223380+0000","last_peered":"2024-05-08T05:49:19.223380+0000","last_clean":"2024-05-08T05:49:19.223380+0000","last_became_active":"2024-05-08T05:49:19.219620+0000","last_became_peered":"2024-05-08T05:49:19.219620+0000","last_unstale":"2024-05-08T05:49:19.223380+0000","last_undegraded":"2024-05-08T05:49:19.223380+0000","last_fullsized":"2024-05-08T05:49:19.223380+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:53:45.139823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.209372+0000","last_change":"2024-05-08T05:49:19.209372+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.209372+0000","last_undegraded":"2024-05-08T05:49:19.209372+0000","last_fullsized":"2024-05-08T05:49:19.209372+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.205273+0000","last_change":"2024-05-08T05:49:18.205273+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.205273+0000","last_undegraded":"2024-05-08T05:49:18.205273+0000","last_fullsized":"2024-05-08T05:49:18.205273+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.200331+0000","last_change":"2024-05-08T05:49:18.200331+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.200331+0000","last_undegraded":"2024-05-08T05:49:18.200331+0000","last_fullsized":"2024-05-08T05:49:18.200331+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.205560+0000","last_change":"2024-05-08T05:49:18.205560+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.205560+0000","last_undegraded":"2024-05-08T05:49:18.205560+0000","last_fullsized":"2024-05-08T05:49:18.205560+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.204671+0000","last_change":"2024-05-08T05:49:18.204671+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.204671+0000","last_undegraded":"2024-05-08T05:49:18.204671+0000","last_fullsized":"2024-05-08T05:49:18.204671+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.228309+0000","last_change":"2024-05-08T05:49:19.220761+0000","last_active":"2024-05-08T05:49:19.228309+0000","last_peered":"2024-05-08T05:49:19.228309+0000","last_clean":"2024-05-08T05:49:19.228309+0000","last_became_active":"2024-05-08T05:49:19.220244+0000","last_became_peered":"2024-05-08T05:49:19.220244+0000","last_unstale":"2024-05-08T05:49:19.228309+0000","last_undegraded":"2024-05-08T05:49:19.228309+0000","last_fullsized":"2024-05-08T05:49:19.228309+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:02:04.763302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.228333+0000","last_change":"2024-05-08T05:49:19.220558+0000","last_active":"2024-05-08T05:49:19.228333+0000","last_peered":"2024-05-08T05:49:19.228333+0000","last_clean":"2024-05-08T05:49:19.228333+0000","last_became_active":"2024-05-08T05:49:19.220251+0000","last_became_peered":"2024-05-08T05:49:19.220251+0000","last_unstale":"2024-05-08T05:49:19.228333+0000","last_undegraded":"2024-05-08T05:49:19.228333+0000","last_fullsized":"2024-05-08T05:49:19.228333+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:41:58.204309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.210499+0000","last_change":"2024-05-08T05:49:19.210499+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.210499+0000","last_undegraded":"2024-05-08T05:49:19.210499+0000","last_fullsized":"2024-05-08T05:49:19.210499+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.202415+0000","last_change":"2024-05-08T05:49:18.202415+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.202415+0000","last_undegraded":"2024-05-08T05:49:18.202415+0000","last_fullsized":"2024-05-08T05:49:18.202415+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.210499+0000","last_change":"2024-05-08T05:49:19.210499+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.210499+0000","last_undegraded":"2024-05-08T05:49:19.210499+0000","last_fullsized":"2024-05-08T05:49:19.210499+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.200372+0000","last_change":"2024-05-08T05:49:18.200372+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.200372+0000","last_undegraded":"2024-05-08T05:49:18.200372+0000","last_fullsized":"2024-05-08T05:49:18.200372+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.205198+0000","last_change":"2024-05-08T05:49:18.205198+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.205198+0000","last_undegraded":"2024-05-08T05:49:18.205198+0000","last_fullsized":"2024-05-08T05:49:18.205198+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.205172+0000","last_change":"2024-05-08T05:49:18.205172+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.205172+0000","last_undegraded":"2024-05-08T05:49:18.205172+0000","last_fullsized":"2024-05-08T05:49:18.205172+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.223455+0000","last_change":"2024-05-08T05:49:19.219459+0000","last_active":"2024-05-08T05:49:19.223455+0000","last_peered":"2024-05-08T05:49:19.223455+0000","last_clean":"2024-05-08T05:49:19.223455+0000","last_became_active":"2024-05-08T05:49:19.219157+0000","last_became_peered":"2024-05-08T05:49:19.219157+0000","last_unstale":"2024-05-08T05:49:19.223455+0000","last_undegraded":"2024-05-08T05:49:19.223455+0000","last_fullsized":"2024-05-08T05:49:19.223455+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:31.735227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.209452+0000","last_change":"2024-05-08T05:49:19.209452+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.209452+0000","last_undegraded":"2024-05-08T05:49:19.209452+0000","last_fullsized":"2024-05-08T05:49:19.209452+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.228331+0000","last_change":"2024-05-08T05:49:19.221812+0000","last_active":"2024-05-08T05:49:19.228331+0000","last_peered":"2024-05-08T05:49:19.228331+0000","last_clean":"2024-05-08T05:49:19.228331+0000","last_became_active":"2024-05-08T05:49:19.221501+0000","last_became_peered":"2024-05-08T05:49:19.221501+0000","last_unstale":"2024-05-08T05:49:19.228331+0000","last_undegraded":"2024-05-08T05:49:19.228331+0000","last_fullsized":"2024-05-08T05:49:19.228331+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:03.552401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.228399+0000","last_change":"2024-05-08T05:49:19.222187+0000","last_active":"2024-05-08T05:49:19.228399+0000","last_peered":"2024-05-08T05:49:19.228399+0000","last_clean":"2024-05-08T05:49:19.228399+0000","last_became_active":"2024-05-08T05:49:19.221863+0000","last_became_peered":"2024-05-08T05:49:19.221863+0000","last_unstale":"2024-05-08T05:49:19.228399+0000","last_undegraded":"2024-05-08T05:49:19.228399+0000","last_fullsized":"2024-05-08T05:49:19.228399+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:06:33.415067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.205592+0000","last_change":"2024-05-08T05:49:18.205592+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.205592+0000","last_undegraded":"2024-05-08T05:49:18.205592+0000","last_fullsized":"2024-05-08T05:49:18.205592+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.206288+0000","last_change":"2024-05-08T05:49:18.206288+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.206288+0000","last_undegraded":"2024-05-08T05:49:18.206288+0000","last_fullsized":"2024-05-08T05:49:18.206288+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.223584+0000","last_change":"2024-05-08T05:49:19.218899+0000","last_active":"2024-05-08T05:49:19.223584+0000","last_peered":"2024-05-08T05:49:19.223584+0000","last_clean":"2024-05-08T05:49:19.223584+0000","last_became_active":"2024-05-08T05:49:19.218594+0000","last_became_peered":"2024-05-08T05:49:19.218594+0000","last_unstale":"2024-05-08T05:49:19.223584+0000","last_undegraded":"2024-05-08T05:49:19.223584+0000","last_fullsized":"2024-05-08T05:49:19.223584+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:04:04.861967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.206002+0000","last_change":"2024-05-08T05:49:18.206002+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.206002+0000","last_undegraded":"2024-05-08T05:49:18.206002+0000","last_fullsized":"2024-05-08T05:49:18.206002+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.202375+0000","last_change":"2024-05-08T05:49:18.202375+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.202375+0000","last_undegraded":"2024-05-08T05:49:18.202375+0000","last_fullsized":"2024-05-08T05:49:18.202375+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.204232+0000","last_change":"2024-05-08T05:49:18.204232+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.204232+0000","last_undegraded":"2024-05-08T05:49:18.204232+0000","last_fullsized":"2024-05-08T05:49:18.204232+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.228557+0000","last_change":"2024-05-08T05:49:19.222418+0000","last_active":"2024-05-08T05:49:19.228557+0000","last_peered":"2024-05-08T05:49:19.228557+0000","last_clean":"2024-05-08T05:49:19.228557+0000","last_became_active":"2024-05-08T05:49:19.222125+0000","last_became_peered":"2024-05-08T05:49:19.222125+0000","last_unstale":"2024-05-08T05:49:19.228557+0000","last_undegraded":"2024-05-08T05:49:19.228557+0000","last_fullsized":"2024-05-08T05:49:19.228557+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:19:50.613280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.209679+0000","last_change":"2024-05-08T05:49:19.209679+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.209679+0000","last_undegraded":"2024-05-08T05:49:19.209679+0000","last_fullsized":"2024-05-08T05:49:19.209679+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.202626+0000","last_change":"2024-05-08T05:49:18.202626+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.202626+0000","last_undegraded":"2024-05-08T05:49:18.202626+0000","last_fullsized":"2024-05-08T05:49:18.202626+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.205720+0000","last_change":"2024-05-08T05:49:18.205720+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.205720+0000","last_undegraded":"2024-05-08T05:49:18.205720+0000","last_fullsized":"2024-05-08T05:49:18.205720+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.205244+0000","last_change":"2024-05-08T05:49:18.205244+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.205244+0000","last_undegraded":"2024-05-08T05:49:18.205244+0000","last_fullsized":"2024-05-08T05:49:18.205244+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.213641+0000","last_change":"2024-05-08T05:49:19.213641+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.213641+0000","last_undegraded":"2024-05-08T05:49:19.213641+0000","last_fullsized":"2024-05-08T05:49:19.213641+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.223605+0000","last_change":"2024-05-08T05:49:19.219966+0000","last_active":"2024-05-08T05:49:19.223605+0000","last_peered":"2024-05-08T05:49:19.223605+0000","last_clean":"2024-05-08T05:49:19.223605+0000","last_became_active":"2024-05-08T05:49:19.219555+0000","last_became_peered":"2024-05-08T05:49:19.219555+0000","last_unstale":"2024-05-08T05:49:19.223605+0000","last_undegraded":"2024-05-08T05:49:19.223605+0000","last_fullsized":"2024-05-08T05:49:19.223605+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:37:05.033005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.211624+0000","last_change":"2024-05-08T05:49:19.211624+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.211624+0000","last_undegraded":"2024-05-08T05:49:19.211624+0000","last_fullsized":"2024-05-08T05:49:19.211624+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.205321+0000","last_change":"2024-05-08T05:49:18.205321+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.205321+0000","last_undegraded":"2024-05-08T05:49:18.205321+0000","last_fullsized":"2024-05-08T05:49:18.205321+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.209777+0000","last_change":"2024-05-08T05:49:19.209777+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.209777+0000","last_undegraded":"2024-05-08T05:49:19.209777+0000","last_fullsized":"2024-05-08T05:49:19.209777+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.205802+0000","last_change":"2024-05-08T05:49:18.205802+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.205802+0000","last_undegraded":"2024-05-08T05:49:18.205802+0000","last_fullsized":"2024-05-08T05:49:18.205802+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":260,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.215190+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:49:19.215190+0000","last_peered":"2024-05-08T05:49:19.215190+0000","last_clean":"2024-05-08T05:49:19.215190+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:49:19.215190+0000","last_undegraded":"2024-05-08T05:49:19.215190+0000","last_fullsized":"2024-05-08T05:49:19.215190+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":42,"acting":42,"num_store_stats":3},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542244,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33000,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":1855488,"data_stored":1172792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542245,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1261568,"data_stored":582424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542244,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1261568,"data_stored":582424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1236992,"data_stored":569548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574946,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1236992,"data_stored":569548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574946,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1249280,"data_stored":575757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1249280,"data_stored":575757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574946,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32988,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338852,"statfs":{"total":96636764160,"available":96602984448,"internally_reserved":0,"allocated":1843200,"data_stored":1166125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:49:24.271 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:49:24.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.412+0000 7fa01dea0640 1 Processor -- start 2024-05-08T05:49:24.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.412+0000 7fa01dea0640 1 -- start start 2024-05-08T05:49:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.412+0000 7fa01dea0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0181629a0 0x7fa0180d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.412+0000 7fa01dea0640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa0180d2dc0 0x7fa0180d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.413+0000 7fa01dea0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa0180c8ed0 0x7fa0180c9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.413+0000 7fa01dea0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa0180d3330 con 0x7fa0180c8ed0 2024-05-08T05:49:24.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.413+0000 7fa01dea0640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa0180d2170 con 0x7fa0180d2dc0 2024-05-08T05:49:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.413+0000 7fa01dea0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa0180d2450 con 0x7fa0181629a0 2024-05-08T05:49:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.413+0000 7fa017fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa0180c8ed0 0x7fa0180c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.413+0000 7fa017fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa0180c8ed0 0x7fa0180c9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45362/0 (socket says 172.21.15.45:45362) 2024-05-08T05:49:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.413+0000 7fa017fff640 1 -- 172.21.15.45:0/2908336363 learned_addr learned my addr 172.21.15.45:0/2908336363 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.413+0000 7fa017fff640 1 -- 172.21.15.45:0/2908336363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0181629a0 msgr2=0x7fa0180d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.413+0000 7fa017fff640 1 --2- 172.21.15.45:0/2908336363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0181629a0 0x7fa0180d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.413+0000 7fa017fff640 1 -- 172.21.15.45:0/2908336363 >> v2:172.21.15.63:3300/0 conn(0x7fa0180d2dc0 msgr2=0x7fa0180d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.413+0000 7fa016ffd640 1 --2- 172.21.15.45:0/2908336363 >> v2:172.21.15.63:3300/0 conn(0x7fa0180d2dc0 0x7fa0180d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.413+0000 7fa017fff640 1 --2- 172.21.15.45:0/2908336363 >> v2:172.21.15.63:3300/0 conn(0x7fa0180d2dc0 0x7fa0180d3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.413+0000 7fa017fff640 1 -- 172.21.15.45:0/2908336363 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0180d2730 con 0x7fa0180c8ed0 2024-05-08T05:49:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.414+0000 7fa017fff640 1 --2- 172.21.15.45:0/2908336363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0180c8ed0 0x7fa0180c9360 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fa01808da00 tx=0x7fa01002a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51aa412fbacf40b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.414+0000 7fa0167fc640 1 -- 172.21.15.45:0/2908336363 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa010036070 con 0x7fa0180c8ed0 2024-05-08T05:49:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.414+0000 7fa0167fc640 1 -- 172.21.15.45:0/2908336363 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa010031600 con 0x7fa0180c8ed0 2024-05-08T05:49:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.414+0000 7fa0167fc640 1 -- 172.21.15.45:0/2908336363 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0100319d0 con 0x7fa0180c8ed0 2024-05-08T05:49:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.414+0000 7fa01dea0640 1 -- 172.21.15.45:0/2908336363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0180c8ed0 msgr2=0x7fa0180c9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.414+0000 7fa01dea0640 1 --2- 172.21.15.45:0/2908336363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0180c8ed0 0x7fa0180c9360 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fa01808da00 tx=0x7fa01002a570 comp rx=0 tx=0).stop 2024-05-08T05:49:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.414+0000 7fa01dea0640 1 -- 172.21.15.45:0/2908336363 shutdown_connections 2024-05-08T05:49:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.414+0000 7fa01dea0640 1 --2- 172.21.15.45:0/2908336363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0180c8ed0 0x7fa0180c9360 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.414+0000 7fa01dea0640 1 --2- 172.21.15.45:0/2908336363 >> v2:172.21.15.63:3300/0 conn(0x7fa0180d2dc0 0x7fa0180d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.414+0000 7fa01dea0640 1 --2- 172.21.15.45:0/2908336363 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0181629a0 0x7fa0180d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.414+0000 7fa01dea0640 1 -- 172.21.15.45:0/2908336363 >> 172.21.15.45:0/2908336363 conn(0x7fa0180d4fe0 msgr2=0x7fa0180d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.415+0000 7fa01dea0640 1 -- 172.21.15.45:0/2908336363 shutdown_connections 2024-05-08T05:49:24.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.415+0000 7fa01dea0640 1 -- 172.21.15.45:0/2908336363 wait complete. 2024-05-08T05:49:24.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.416+0000 7fa01dea0640 1 Processor -- start 2024-05-08T05:49:24.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.416+0000 7fa01dea0640 1 -- start start 2024-05-08T05:49:24.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7fa01dea0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0180c8ed0 0x7fa018155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:24.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7fa01dea0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0180d2dc0 0x7fa0180a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:24.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7fa01dea0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0181629a0 0x7fa0180c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:24.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7fa01dea0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0180cf440 con 0x7fa0180c8ed0 2024-05-08T05:49:24.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7fa01dea0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa0180cf2c0 con 0x7fa0181629a0 2024-05-08T05:49:24.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7fa01dea0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa0180c98a0 con 0x7fa0180d2dc0 2024-05-08T05:49:24.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7fa0177fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0180c8ed0 0x7fa018155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:24.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7fa0177fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0180c8ed0 0x7fa018155480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45378/0 (socket says 172.21.15.45:45378) 2024-05-08T05:49:24.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7fa0177fe640 1 -- 172.21.15.45:0/605732725 learned_addr learned my addr 172.21.15.45:0/605732725 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:24.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7fa016ffd640 1 --2- 172.21.15.45:0/605732725 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0180d2dc0 0x7fa0180a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:24.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7fa017fff640 1 --2- 172.21.15.45:0/605732725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0181629a0 0x7fa0180c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:24.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7fa0177fe640 1 -- 172.21.15.45:0/605732725 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0180d2dc0 msgr2=0x7fa0180a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:24.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7fa0177fe640 1 --2- 172.21.15.45:0/605732725 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0180d2dc0 0x7fa0180a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7fa0177fe640 1 -- 172.21.15.45:0/605732725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0181629a0 msgr2=0x7fa0180c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:24.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7fa0177fe640 1 --2- 172.21.15.45:0/605732725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0181629a0 0x7fa0180c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7fa0177fe640 1 -- 172.21.15.45:0/605732725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0180a1e60 con 0x7fa0180c8ed0 2024-05-08T05:49:24.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7fa016ffd640 1 --2- 172.21.15.45:0/605732725 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0180d2dc0 0x7fa0180a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:24.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7fa0177fe640 1 --2- 172.21.15.45:0/605732725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0180c8ed0 0x7fa018155480 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fa00800b410 tx=0x7fa00800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:24.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7f9ffffff640 1 -- 172.21.15.45:0/605732725 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008015020 con 0x7fa0180c8ed0 2024-05-08T05:49:24.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7f9ffffff640 1 -- 172.21.15.45:0/605732725 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa008011850 con 0x7fa0180c8ed0 2024-05-08T05:49:24.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7fa01dea0640 1 -- 172.21.15.45:0/605732725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0180a2140 con 0x7fa0180c8ed0 2024-05-08T05:49:24.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.417+0000 7f9ffffff640 1 -- 172.21.15.45:0/605732725 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008011c40 con 0x7fa0180c8ed0 2024-05-08T05:49:24.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.418+0000 7fa01dea0640 1 -- 172.21.15.45:0/605732725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0180a2700 con 0x7fa0180c8ed0 2024-05-08T05:49:24.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.419+0000 7fa01dea0640 1 -- 172.21.15.45:0/605732725 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0180693f0 con 0x7fa0180c8ed0 2024-05-08T05:49:24.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.419+0000 7f9ffffff640 1 -- 172.21.15.45:0/605732725 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa008018020 con 0x7fa0180c8ed0 2024-05-08T05:49:24.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.420+0000 7f9ffffff640 1 --2- 172.21.15.45:0/605732725 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa0040762d0 0x7fa004078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:24.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.420+0000 7f9ffffff640 1 -- 172.21.15.45:0/605732725 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fa008097730 con 0x7fa0180c8ed0 2024-05-08T05:49:24.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.420+0000 7fa016ffd640 1 --2- 172.21.15.45:0/605732725 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa0040762d0 0x7fa004078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:24.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.421+0000 7fa016ffd640 1 --2- 172.21.15.45:0/605732725 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa0040762d0 0x7fa004078790 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa0180cf8a0 tx=0x7fa00c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:24.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.424+0000 7f9ffffff640 1 -- 172.21.15.45:0/605732725 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa00801f020 con 0x7fa0180c8ed0 2024-05-08T05:49:24.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.615+0000 7fa01dea0640 1 -- 172.21.15.45:0/605732725 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa0180c8740 con 0x7fa0040762d0 2024-05-08T05:49:24.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.621+0000 7f9ffffff640 1 -- 172.21.15.45:0/605732725 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242520 (secure 0 0 0) 0x7fa0180c8740 con 0x7fa0040762d0 2024-05-08T05:49:24.623 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:24.625 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:49:24.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.624+0000 7fa01dea0640 1 -- 172.21.15.45:0/605732725 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa0040762d0 msgr2=0x7fa004078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:24.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.624+0000 7fa01dea0640 1 --2- 172.21.15.45:0/605732725 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa0040762d0 0x7fa004078790 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa0180cf8a0 tx=0x7fa00c008040 comp rx=0 tx=0).stop 2024-05-08T05:49:24.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.624+0000 7fa01dea0640 1 -- 172.21.15.45:0/605732725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0180c8ed0 msgr2=0x7fa018155480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:24.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.624+0000 7fa01dea0640 1 --2- 172.21.15.45:0/605732725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0180c8ed0 0x7fa018155480 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fa00800b410 tx=0x7fa00800b8d0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.624+0000 7fa01dea0640 1 -- 172.21.15.45:0/605732725 shutdown_connections 2024-05-08T05:49:24.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.624+0000 7fa01dea0640 1 --2- 172.21.15.45:0/605732725 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa0040762d0 0x7fa004078790 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.624+0000 7fa01dea0640 1 --2- 172.21.15.45:0/605732725 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa0180d2dc0 0x7fa0180a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.624+0000 7fa01dea0640 1 --2- 172.21.15.45:0/605732725 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0181629a0 0x7fa0180c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.624+0000 7fa01dea0640 1 --2- 172.21.15.45:0/605732725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0180c8ed0 0x7fa018155480 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.624+0000 7fa01dea0640 1 -- 172.21.15.45:0/605732725 >> 172.21.15.45:0/605732725 conn(0x7fa0180d4fe0 msgr2=0x7fa01806d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:24.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.624+0000 7fa01dea0640 1 -- 172.21.15.45:0/605732725 shutdown_connections 2024-05-08T05:49:24.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.624+0000 7fa01dea0640 1 -- 172.21.15.45:0/605732725 wait complete. 2024-05-08T05:49:24.635 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-05-08T05:49:22.948120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":108,"acting":108,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260404,"kb_used_data":10932,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714316,"statfs":{"total":773094113280,"available":772827459584,"internally_reserved":0,"allocated":11194368,"data_stored":5794375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.748919"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.210877+0000","last_change":"2024-05-08T05:49:19.210877+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.210877+0000","last_undegraded":"2024-05-08T05:49:19.210877+0000","last_fullsized":"2024-05-08T05:49:19.210877+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.209934+0000","last_change":"2024-05-08T05:49:19.209934+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.209934+0000","last_undegraded":"2024-05-08T05:49:19.209934+0000","last_fullsized":"2024-05-08T05:49:19.209934+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.211413+0000","last_change":"2024-05-08T05:49:19.211413+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.211413+0000","last_undegraded":"2024-05-08T05:49:19.211413+0000","last_fullsized":"2024-05-08T05:49:19.211413+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.210766+0000","last_change":"2024-05-08T05:49:19.210766+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.210766+0000","last_undegraded":"2024-05-08T05:49:19.210766+0000","last_fullsized":"2024-05-08T05:49:19.210766+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.211164+0000","last_change":"2024-05-08T05:49:19.211164+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.211164+0000","last_undegraded":"2024-05-08T05:49:19.211164+0000","last_fullsized":"2024-05-08T05:49:19.211164+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.212601+0000","last_change":"2024-05-08T05:49:19.212601+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.212601+0000","last_undegraded":"2024-05-08T05:49:19.212601+0000","last_fullsized":"2024-05-08T05:49:19.212601+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.211903+0000","last_change":"2024-05-08T05:49:19.211903+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.211903+0000","last_undegraded":"2024-05-08T05:49:19.211903+0000","last_fullsized":"2024-05-08T05:49:19.211903+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.214901+0000","last_change":"2024-05-08T05:49:19.214901+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.214901+0000","last_undegraded":"2024-05-08T05:49:19.214901+0000","last_fullsized":"2024-05-08T05:49:19.214901+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.209892+0000","last_change":"2024-05-08T05:49:19.209892+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.209892+0000","last_undegraded":"2024-05-08T05:49:19.209892+0000","last_fullsized":"2024-05-08T05:49:19.209892+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.209907+0000","last_change":"2024-05-08T05:49:19.209907+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.209907+0000","last_undegraded":"2024-05-08T05:49:19.209907+0000","last_fullsized":"2024-05-08T05:49:19.209907+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.211263+0000","last_change":"2024-05-08T05:49:19.211263+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.211263+0000","last_undegraded":"2024-05-08T05:49:19.211263+0000","last_fullsized":"2024-05-08T05:49:19.211263+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.210963+0000","last_change":"2024-05-08T05:49:19.210963+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.210963+0000","last_undegraded":"2024-05-08T05:49:19.210963+0000","last_fullsized":"2024-05-08T05:49:19.210963+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.200329+0000","last_change":"2024-05-08T05:49:18.200329+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.200329+0000","last_undegraded":"2024-05-08T05:49:18.200329+0000","last_fullsized":"2024-05-08T05:49:18.200329+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.206429+0000","last_change":"2024-05-08T05:49:18.206429+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.206429+0000","last_undegraded":"2024-05-08T05:49:18.206429+0000","last_fullsized":"2024-05-08T05:49:18.206429+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.205185+0000","last_change":"2024-05-08T05:49:18.205185+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.205185+0000","last_undegraded":"2024-05-08T05:49:18.205185+0000","last_fullsized":"2024-05-08T05:49:18.205185+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.209205+0000","last_change":"2024-05-08T05:49:19.209205+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.209205+0000","last_undegraded":"2024-05-08T05:49:19.209205+0000","last_fullsized":"2024-05-08T05:49:19.209205+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.203113+0000","last_change":"2024-05-08T05:49:18.203113+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.203113+0000","last_undegraded":"2024-05-08T05:49:18.203113+0000","last_fullsized":"2024-05-08T05:49:18.203113+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.230993+0000","last_change":"2024-05-08T05:49:19.224342+0000","last_active":"2024-05-08T05:49:19.230993+0000","last_peered":"2024-05-08T05:49:19.230993+0000","last_clean":"2024-05-08T05:49:19.230993+0000","last_became_active":"2024-05-08T05:49:19.224020+0000","last_became_peered":"2024-05-08T05:49:19.224020+0000","last_unstale":"2024-05-08T05:49:19.230993+0000","last_undegraded":"2024-05-08T05:49:19.230993+0000","last_fullsized":"2024-05-08T05:49:19.230993+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:00.780658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.223380+0000","last_change":"2024-05-08T05:49:19.219998+0000","last_active":"2024-05-08T05:49:19.223380+0000","last_peered":"2024-05-08T05:49:19.223380+0000","last_clean":"2024-05-08T05:49:19.223380+0000","last_became_active":"2024-05-08T05:49:19.219620+0000","last_became_peered":"2024-05-08T05:49:19.219620+0000","last_unstale":"2024-05-08T05:49:19.223380+0000","last_undegraded":"2024-05-08T05:49:19.223380+0000","last_fullsized":"2024-05-08T05:49:19.223380+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:53:45.139823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.209372+0000","last_change":"2024-05-08T05:49:19.209372+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.209372+0000","last_undegraded":"2024-05-08T05:49:19.209372+0000","last_fullsized":"2024-05-08T05:49:19.209372+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.205273+0000","last_change":"2024-05-08T05:49:18.205273+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.205273+0000","last_undegraded":"2024-05-08T05:49:18.205273+0000","last_fullsized":"2024-05-08T05:49:18.205273+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.200331+0000","last_change":"2024-05-08T05:49:18.200331+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.200331+0000","last_undegraded":"2024-05-08T05:49:18.200331+0000","last_fullsized":"2024-05-08T05:49:18.200331+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.205560+0000","last_change":"2024-05-08T05:49:18.205560+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.205560+0000","last_undegraded":"2024-05-08T05:49:18.205560+0000","last_fullsized":"2024-05-08T05:49:18.205560+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.204671+0000","last_change":"2024-05-08T05:49:18.204671+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.204671+0000","last_undegraded":"2024-05-08T05:49:18.204671+0000","last_fullsized":"2024-05-08T05:49:18.204671+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.228309+0000","last_change":"2024-05-08T05:49:19.220761+0000","last_active":"2024-05-08T05:49:19.228309+0000","last_peered":"2024-05-08T05:49:19.228309+0000","last_clean":"2024-05-08T05:49:19.228309+0000","last_became_active":"2024-05-08T05:49:19.220244+0000","last_became_peered":"2024-05-08T05:49:19.220244+0000","last_unstale":"2024-05-08T05:49:19.228309+0000","last_undegraded":"2024-05-08T05:49:19.228309+0000","last_fullsized":"2024-05-08T05:49:19.228309+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:02:04.763302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.228333+0000","last_change":"2024-05-08T05:49:19.220558+0000","last_active":"2024-05-08T05:49:19.228333+0000","last_peered":"2024-05-08T05:49:19.228333+0000","last_clean":"2024-05-08T05:49:19.228333+0000","last_became_active":"2024-05-08T05:49:19.220251+0000","last_became_peered":"2024-05-08T05:49:19.220251+0000","last_unstale":"2024-05-08T05:49:19.228333+0000","last_undegraded":"2024-05-08T05:49:19.228333+0000","last_fullsized":"2024-05-08T05:49:19.228333+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:41:58.204309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.210499+0000","last_change":"2024-05-08T05:49:19.210499+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.210499+0000","last_undegraded":"2024-05-08T05:49:19.210499+0000","last_fullsized":"2024-05-08T05:49:19.210499+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.202415+0000","last_change":"2024-05-08T05:49:18.202415+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.202415+0000","last_undegraded":"2024-05-08T05:49:18.202415+0000","last_fullsized":"2024-05-08T05:49:18.202415+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.210499+0000","last_change":"2024-05-08T05:49:19.210499+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.210499+0000","last_undegraded":"2024-05-08T05:49:19.210499+0000","last_fullsized":"2024-05-08T05:49:19.210499+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.200372+0000","last_change":"2024-05-08T05:49:18.200372+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.200372+0000","last_undegraded":"2024-05-08T05:49:18.200372+0000","last_fullsized":"2024-05-08T05:49:18.200372+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.205198+0000","last_change":"2024-05-08T05:49:18.205198+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.205198+0000","last_undegraded":"2024-05-08T05:49:18.205198+0000","last_fullsized":"2024-05-08T05:49:18.205198+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.205172+0000","last_change":"2024-05-08T05:49:18.205172+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.205172+0000","last_undegraded":"2024-05-08T05:49:18.205172+0000","last_fullsized":"2024-05-08T05:49:18.205172+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.223455+0000","last_change":"2024-05-08T05:49:19.219459+0000","last_active":"2024-05-08T05:49:19.223455+0000","last_peered":"2024-05-08T05:49:19.223455+0000","last_clean":"2024-05-08T05:49:19.223455+0000","last_became_active":"2024-05-08T05:49:19.219157+0000","last_became_peered":"2024-05-08T05:49:19.219157+0000","last_unstale":"2024-05-08T05:49:19.223455+0000","last_undegraded":"2024-05-08T05:49:19.223455+0000","last_fullsized":"2024-05-08T05:49:19.223455+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:31.735227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.209452+0000","last_change":"2024-05-08T05:49:19.209452+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.209452+0000","last_undegraded":"2024-05-08T05:49:19.209452+0000","last_fullsized":"2024-05-08T05:49:19.209452+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.228331+0000","last_change":"2024-05-08T05:49:19.221812+0000","last_active":"2024-05-08T05:49:19.228331+0000","last_peered":"2024-05-08T05:49:19.228331+0000","last_clean":"2024-05-08T05:49:19.228331+0000","last_became_active":"2024-05-08T05:49:19.221501+0000","last_became_peered":"2024-05-08T05:49:19.221501+0000","last_unstale":"2024-05-08T05:49:19.228331+0000","last_undegraded":"2024-05-08T05:49:19.228331+0000","last_fullsized":"2024-05-08T05:49:19.228331+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:03.552401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.228399+0000","last_change":"2024-05-08T05:49:19.222187+0000","last_active":"2024-05-08T05:49:19.228399+0000","last_peered":"2024-05-08T05:49:19.228399+0000","last_clean":"2024-05-08T05:49:19.228399+0000","last_became_active":"2024-05-08T05:49:19.221863+0000","last_became_peered":"2024-05-08T05:49:19.221863+0000","last_unstale":"2024-05-08T05:49:19.228399+0000","last_undegraded":"2024-05-08T05:49:19.228399+0000","last_fullsized":"2024-05-08T05:49:19.228399+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:06:33.415067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.205592+0000","last_change":"2024-05-08T05:49:18.205592+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.205592+0000","last_undegraded":"2024-05-08T05:49:18.205592+0000","last_fullsized":"2024-05-08T05:49:18.205592+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.206288+0000","last_change":"2024-05-08T05:49:18.206288+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.206288+0000","last_undegraded":"2024-05-08T05:49:18.206288+0000","last_fullsized":"2024-05-08T05:49:18.206288+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.223584+0000","last_change":"2024-05-08T05:49:19.218899+0000","last_active":"2024-05-08T05:49:19.223584+0000","last_peered":"2024-05-08T05:49:19.223584+0000","last_clean":"2024-05-08T05:49:19.223584+0000","last_became_active":"2024-05-08T05:49:19.218594+0000","last_became_peered":"2024-05-08T05:49:19.218594+0000","last_unstale":"2024-05-08T05:49:19.223584+0000","last_undegraded":"2024-05-08T05:49:19.223584+0000","last_fullsized":"2024-05-08T05:49:19.223584+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:04:04.861967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.206002+0000","last_change":"2024-05-08T05:49:18.206002+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.206002+0000","last_undegraded":"2024-05-08T05:49:18.206002+0000","last_fullsized":"2024-05-08T05:49:18.206002+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.202375+0000","last_change":"2024-05-08T05:49:18.202375+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.202375+0000","last_undegraded":"2024-05-08T05:49:18.202375+0000","last_fullsized":"2024-05-08T05:49:18.202375+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.204232+0000","last_change":"2024-05-08T05:49:18.204232+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.204232+0000","last_undegraded":"2024-05-08T05:49:18.204232+0000","last_fullsized":"2024-05-08T05:49:18.204232+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.228557+0000","last_change":"2024-05-08T05:49:19.222418+0000","last_active":"2024-05-08T05:49:19.228557+0000","last_peered":"2024-05-08T05:49:19.228557+0000","last_clean":"2024-05-08T05:49:19.228557+0000","last_became_active":"2024-05-08T05:49:19.222125+0000","last_became_peered":"2024-05-08T05:49:19.222125+0000","last_unstale":"2024-05-08T05:49:19.228557+0000","last_undegraded":"2024-05-08T05:49:19.228557+0000","last_fullsized":"2024-05-08T05:49:19.228557+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:19:50.613280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.209679+0000","last_change":"2024-05-08T05:49:19.209679+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.209679+0000","last_undegraded":"2024-05-08T05:49:19.209679+0000","last_fullsized":"2024-05-08T05:49:19.209679+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.202626+0000","last_change":"2024-05-08T05:49:18.202626+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.202626+0000","last_undegraded":"2024-05-08T05:49:18.202626+0000","last_fullsized":"2024-05-08T05:49:18.202626+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.205720+0000","last_change":"2024-05-08T05:49:18.205720+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.205720+0000","last_undegraded":"2024-05-08T05:49:18.205720+0000","last_fullsized":"2024-05-08T05:49:18.205720+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-05-08T05:49:19.196621+0000","last_change":"2024-05-08T05:49:19.196621+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.196621+0000","last_undegraded":"2024-05-08T05:49:19.196621+0000","last_fullsized":"2024-05-08T05:49:19.196621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.205244+0000","last_change":"2024-05-08T05:49:18.205244+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.205244+0000","last_undegraded":"2024-05-08T05:49:18.205244+0000","last_fullsized":"2024-05-08T05:49:18.205244+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.213641+0000","last_change":"2024-05-08T05:49:19.213641+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.213641+0000","last_undegraded":"2024-05-08T05:49:19.213641+0000","last_fullsized":"2024-05-08T05:49:19.213641+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.223605+0000","last_change":"2024-05-08T05:49:19.219966+0000","last_active":"2024-05-08T05:49:19.223605+0000","last_peered":"2024-05-08T05:49:19.223605+0000","last_clean":"2024-05-08T05:49:19.223605+0000","last_became_active":"2024-05-08T05:49:19.219555+0000","last_became_peered":"2024-05-08T05:49:19.219555+0000","last_unstale":"2024-05-08T05:49:19.223605+0000","last_undegraded":"2024-05-08T05:49:19.223605+0000","last_fullsized":"2024-05-08T05:49:19.223605+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:37:05.033005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.211624+0000","last_change":"2024-05-08T05:49:19.211624+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.211624+0000","last_undegraded":"2024-05-08T05:49:19.211624+0000","last_fullsized":"2024-05-08T05:49:19.211624+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.205321+0000","last_change":"2024-05-08T05:49:18.205321+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.205321+0000","last_undegraded":"2024-05-08T05:49:18.205321+0000","last_fullsized":"2024-05-08T05:49:18.205321+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-05-08T05:49:19.209777+0000","last_change":"2024-05-08T05:49:19.209777+0000","last_active":"2024-05-08T05:49:19.196621+0000","last_peered":"2024-05-08T05:49:19.196621+0000","last_clean":"2024-05-08T05:49:19.196621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:19.209777+0000","last_undegraded":"2024-05-08T05:49:19.209777+0000","last_fullsized":"2024-05-08T05:49:19.209777+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-05-08T05:49:18.205802+0000","last_change":"2024-05-08T05:49:18.205802+0000","last_active":"2024-05-08T05:49:18.188589+0000","last_peered":"2024-05-08T05:49:18.188589+0000","last_clean":"2024-05-08T05:49:18.188589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:49:18.205802+0000","last_undegraded":"2024-05-08T05:49:18.205802+0000","last_fullsized":"2024-05-08T05:49:18.205802+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":260,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T05:49:19.215190+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:49:19.215190+0000","last_peered":"2024-05-08T05:49:19.215190+0000","last_clean":"2024-05-08T05:49:19.215190+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:49:19.215190+0000","last_undegraded":"2024-05-08T05:49:19.215190+0000","last_fullsized":"2024-05-08T05:49:19.215190+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":42,"acting":42,"num_store_stats":3},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542244,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33000,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":1855488,"data_stored":1172792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542245,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1261568,"data_stored":582424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542244,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1261568,"data_stored":582424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1236992,"data_stored":569548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574946,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1236992,"data_stored":569548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574946,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1249280,"data_stored":575757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1249280,"data_stored":575757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574946,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32988,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338852,"statfs":{"total":96636764160,"available":96602984448,"internally_reserved":0,"allocated":1843200,"data_stored":1166125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:49:24.639 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:49:24.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.779+0000 7fe7f5a54640 1 Processor -- start 2024-05-08T05:49:24.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.779+0000 7fe7f5a54640 1 -- start start 2024-05-08T05:49:24.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.780+0000 7fe7f5a54640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe7f00d0590 0x7fe7f00ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:24.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.780+0000 7fe7f5a54640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe7f00ce610 0x7fe7f00709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:24.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.780+0000 7fe7f5a54640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7f00d15c0 con 0x7fe7f01606d0 2024-05-08T05:49:24.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.780+0000 7fe7f5a54640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f00d18a0 con 0x7fe7f00d0590 2024-05-08T05:49:24.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.780+0000 7fe7f5a54640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f00d1b80 con 0x7fe7f00ce610 2024-05-08T05:49:24.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.780+0000 7fe7ee7fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe7f00d0590 0x7fe7f00ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:24.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.780+0000 7fe7ee7fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe7f00d0590 0x7fe7f00ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45386/0 (socket says 172.21.15.45:45386) 2024-05-08T05:49:24.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.780+0000 7fe7eeffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe7f01606d0 0x7fe7f00cd9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49856/0 (socket says 172.21.15.45:49856) 2024-05-08T05:49:24.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.780+0000 7fe7ee7fc640 1 -- 172.21.15.45:0/3146249002 learned_addr learned my addr 172.21.15.45:0/3146249002 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:24.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.780+0000 7fe7ef7fe640 1 --2- 172.21.15.45:0/3146249002 >> v2:172.21.15.63:3300/0 conn(0x7fe7f00ce610 0x7fe7f00709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:24.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.780+0000 7fe7ee7fc640 1 -- 172.21.15.45:0/3146249002 >> v2:172.21.15.63:3300/0 conn(0x7fe7f00ce610 msgr2=0x7fe7f00709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:24.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.780+0000 7fe7ee7fc640 1 --2- 172.21.15.45:0/3146249002 >> v2:172.21.15.63:3300/0 conn(0x7fe7f00ce610 0x7fe7f00709f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.780+0000 7fe7ee7fc640 1 -- 172.21.15.45:0/3146249002 >> v1:172.21.15.45:6789/0 conn(0x7fe7f01606d0 legacy=0x7fe7f00cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:24.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.780+0000 7fe7edffb640 1 -- 172.21.15.45:0/3146249002 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1831517924 0 0) 0x7fe7f00d15c0 con 0x7fe7f01606d0 2024-05-08T05:49:24.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.781+0000 7fe7ee7fc640 1 -- 172.21.15.45:0/3146249002 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f00704d0 con 0x7fe7f00d0590 2024-05-08T05:49:24.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.781+0000 7fe7ee7fc640 1 --2- 172.21.15.45:0/3146249002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7f00d0590 0x7fe7f00ce0d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00b8e20 tx=0x7fe7e402a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d82fab46e293260 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:24.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.781+0000 7fe7edffb640 1 -- 172.21.15.45:0/3146249002 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4037070 con 0x7fe7f00d0590 2024-05-08T05:49:24.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.781+0000 7fe7edffb640 1 -- 172.21.15.45:0/3146249002 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e4032440 con 0x7fe7f00d0590 2024-05-08T05:49:24.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.781+0000 7fe7edffb640 1 -- 172.21.15.45:0/3146249002 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4032810 con 0x7fe7f00d0590 2024-05-08T05:49:24.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.782+0000 7fe7f5a54640 1 -- 172.21.15.45:0/3146249002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7f00d0590 msgr2=0x7fe7f00ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:24.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.782+0000 7fe7f5a54640 1 --2- 172.21.15.45:0/3146249002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7f00d0590 0x7fe7f00ce0d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00b8e20 tx=0x7fe7e402a150 comp rx=0 tx=0).stop 2024-05-08T05:49:24.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.782+0000 7fe7f5a54640 1 -- 172.21.15.45:0/3146249002 shutdown_connections 2024-05-08T05:49:24.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.782+0000 7fe7f5a54640 1 --2- 172.21.15.45:0/3146249002 >> v2:172.21.15.63:3300/0 conn(0x7fe7f00ce610 0x7fe7f00709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.782+0000 7fe7f5a54640 1 --2- 172.21.15.45:0/3146249002 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7f00d0590 0x7fe7f00ce0d0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.782+0000 7fe7f5a54640 1 -- 172.21.15.45:0/3146249002 >> 172.21.15.45:0/3146249002 conn(0x7fe7f00b49d0 msgr2=0x7fe7f00b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:24.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.782+0000 7fe7f5a54640 1 -- 172.21.15.45:0/3146249002 shutdown_connections 2024-05-08T05:49:24.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.782+0000 7fe7f5a54640 1 -- 172.21.15.45:0/3146249002 wait complete. 2024-05-08T05:49:24.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.783+0000 7fe7f5a54640 1 Processor -- start 2024-05-08T05:49:24.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.783+0000 7fe7f5a54640 1 -- start start 2024-05-08T05:49:24.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.784+0000 7fe7f5a54640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7f00ce610 0x7fe7f015b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:24.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.784+0000 7fe7f5a54640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7f00d0590 0x7fe7f015cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:24.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.784+0000 7fe7f5a54640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7f01606d0 0x7fe7f00c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:24.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.784+0000 7fe7f5a54640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f00710b0 con 0x7fe7f00d0590 2024-05-08T05:49:24.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.784+0000 7fe7f5a54640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f0070f30 con 0x7fe7f01606d0 2024-05-08T05:49:24.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.784+0000 7fe7f5a54640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f0071230 con 0x7fe7f00ce610 2024-05-08T05:49:24.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.784+0000 7fe7ee7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7f00d0590 0x7fe7f015cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:24.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.784+0000 7fe7ee7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7f00d0590 0x7fe7f015cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45396/0 (socket says 172.21.15.45:45396) 2024-05-08T05:49:24.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.784+0000 7fe7ee7fc640 1 -- 172.21.15.45:0/1230023851 learned_addr learned my addr 172.21.15.45:0/1230023851 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:24.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.784+0000 7fe7ee7fc640 1 -- 172.21.15.45:0/1230023851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7f00ce610 msgr2=0x7fe7f015b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:24.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.784+0000 7fe7ee7fc640 1 --2- 172.21.15.45:0/1230023851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7f00ce610 0x7fe7f015b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.784+0000 7fe7ee7fc640 1 -- 172.21.15.45:0/1230023851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7f01606d0 msgr2=0x7fe7f00c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:24.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.784+0000 7fe7ee7fc640 1 --2- 172.21.15.45:0/1230023851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7f01606d0 0x7fe7f00c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.784+0000 7fe7ee7fc640 1 -- 172.21.15.45:0/1230023851 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f00c7ee0 con 0x7fe7f00d0590 2024-05-08T05:49:24.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.785+0000 7fe7ee7fc640 1 --2- 172.21.15.45:0/1230023851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7f00d0590 0x7fe7f015cfb0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4000c00 tx=0x7fe7e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:24.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.785+0000 7fe7d7fff640 1 -- 172.21.15.45:0/1230023851 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4037040 con 0x7fe7f00d0590 2024-05-08T05:49:24.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.785+0000 7fe7d7fff640 1 -- 172.21.15.45:0/1230023851 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e4004230 con 0x7fe7f00d0590 2024-05-08T05:49:24.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.785+0000 7fe7f5a54640 1 -- 172.21.15.45:0/1230023851 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f00c8ab0 con 0x7fe7f00d0590 2024-05-08T05:49:24.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.785+0000 7fe7d7fff640 1 -- 172.21.15.45:0/1230023851 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4003610 con 0x7fe7f00d0590 2024-05-08T05:49:24.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.785+0000 7fe7f5a54640 1 -- 172.21.15.45:0/1230023851 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f015ba10 con 0x7fe7f00d0590 2024-05-08T05:49:24.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.786+0000 7fe7f5a54640 1 -- 172.21.15.45:0/1230023851 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f00693f0 con 0x7fe7f00d0590 2024-05-08T05:49:24.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.786+0000 7fe7d7fff640 1 -- 172.21.15.45:0/1230023851 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe7e40037b0 con 0x7fe7f00d0590 2024-05-08T05:49:24.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.787+0000 7fe7d7fff640 1 --2- 172.21.15.45:0/1230023851 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe7dc076300 0x7fe7dc0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:24.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.787+0000 7fe7d7fff640 1 -- 172.21.15.45:0/1230023851 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fe7e40b8580 con 0x7fe7f00d0590 2024-05-08T05:49:24.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.787+0000 7fe7eeffd640 1 --2- 172.21.15.45:0/1230023851 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe7dc076300 0x7fe7dc0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:24.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.788+0000 7fe7eeffd640 1 --2- 172.21.15.45:0/1230023851 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe7dc076300 0x7fe7dc0787c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0007fd0 tx=0x7fe7e0007960 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:24.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.791+0000 7fe7d7fff640 1 -- 172.21.15.45:0/1230023851 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe7e4081de0 con 0x7fe7f00d0590 2024-05-08T05:49:24.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.979+0000 7fe7f5a54640 1 -- 172.21.15.45:0/1230023851 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe7f00cfee0 con 0x7fe7dc076300 2024-05-08T05:49:24.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.986+0000 7fe7d7fff640 1 -- 172.21.15.45:0/1230023851 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252475 (secure 0 0 0) 0x7fe7f00cfee0 con 0x7fe7dc076300 2024-05-08T05:49:24.989 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:24.990 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:49:24.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.989+0000 7fe7f5a54640 1 -- 172.21.15.45:0/1230023851 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe7dc076300 msgr2=0x7fe7dc0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:24.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.989+0000 7fe7f5a54640 1 --2- 172.21.15.45:0/1230023851 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe7dc076300 0x7fe7dc0787c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0007fd0 tx=0x7fe7e0007960 comp rx=0 tx=0).stop 2024-05-08T05:49:24.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.989+0000 7fe7f5a54640 1 -- 172.21.15.45:0/1230023851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7f00d0590 msgr2=0x7fe7f015cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:24.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.989+0000 7fe7f5a54640 1 --2- 172.21.15.45:0/1230023851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7f00d0590 0x7fe7f015cfb0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4000c00 tx=0x7fe7e40046c0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.989+0000 7fe7f5a54640 1 -- 172.21.15.45:0/1230023851 shutdown_connections 2024-05-08T05:49:24.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.989+0000 7fe7f5a54640 1 --2- 172.21.15.45:0/1230023851 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe7dc076300 0x7fe7dc0787c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.989+0000 7fe7f5a54640 1 --2- 172.21.15.45:0/1230023851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7f01606d0 0x7fe7f00c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.989+0000 7fe7f5a54640 1 --2- 172.21.15.45:0/1230023851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7f00d0590 0x7fe7f015cfb0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.989+0000 7fe7f5a54640 1 --2- 172.21.15.45:0/1230023851 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe7f00ce610 0x7fe7f015b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:24.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.989+0000 7fe7f5a54640 1 -- 172.21.15.45:0/1230023851 >> 172.21.15.45:0/1230023851 conn(0x7fe7f00b49d0 msgr2=0x7fe7f00cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:24.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.989+0000 7fe7f5a54640 1 -- 172.21.15.45:0/1230023851 shutdown_connections 2024-05-08T05:49:24.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:24.989+0000 7fe7f5a54640 1 -- 172.21.15.45:0/1230023851 wait complete. 2024-05-08T05:49:25.000 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-05-08T05:49:24.948475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260724,"kb_used_data":11272,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713996,"statfs":{"total":773094113280,"available":772827131904,"internally_reserved":0,"allocated":11542528,"data_stored":5955820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001390"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220540+0000","last_change":"2024-05-08T05:49:20.238531+0000","last_active":"2024-05-08T05:49:21.220540+0000","last_peered":"2024-05-08T05:49:21.220540+0000","last_clean":"2024-05-08T05:49:21.220540+0000","last_became_active":"2024-05-08T05:49:20.238249+0000","last_became_peered":"2024-05-08T05:49:20.238249+0000","last_unstale":"2024-05-08T05:49:21.220540+0000","last_undegraded":"2024-05-08T05:49:21.220540+0000","last_fullsized":"2024-05-08T05:49:21.220540+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:00:42.502385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741649+0000","last_change":"2024-05-08T05:49:20.230490+0000","last_active":"2024-05-08T05:49:21.741649+0000","last_peered":"2024-05-08T05:49:21.741649+0000","last_clean":"2024-05-08T05:49:21.741649+0000","last_became_active":"2024-05-08T05:49:20.230163+0000","last_became_peered":"2024-05-08T05:49:20.230163+0000","last_unstale":"2024-05-08T05:49:21.741649+0000","last_undegraded":"2024-05-08T05:49:21.741649+0000","last_fullsized":"2024-05-08T05:49:21.741649+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:01.788691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220788+0000","last_change":"2024-05-08T05:49:20.228822+0000","last_active":"2024-05-08T05:49:21.220788+0000","last_peered":"2024-05-08T05:49:21.220788+0000","last_clean":"2024-05-08T05:49:21.220788+0000","last_became_active":"2024-05-08T05:49:20.228543+0000","last_became_peered":"2024-05-08T05:49:20.228543+0000","last_unstale":"2024-05-08T05:49:21.220788+0000","last_undegraded":"2024-05-08T05:49:21.220788+0000","last_fullsized":"2024-05-08T05:49:21.220788+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:15.407492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740921+0000","last_change":"2024-05-08T05:49:20.227003+0000","last_active":"2024-05-08T05:49:21.740921+0000","last_peered":"2024-05-08T05:49:21.740921+0000","last_clean":"2024-05-08T05:49:21.740921+0000","last_became_active":"2024-05-08T05:49:20.226673+0000","last_became_peered":"2024-05-08T05:49:20.226673+0000","last_unstale":"2024-05-08T05:49:21.740921+0000","last_undegraded":"2024-05-08T05:49:21.740921+0000","last_fullsized":"2024-05-08T05:49:21.740921+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:58:10.209690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740362+0000","last_change":"2024-05-08T05:49:20.228592+0000","last_active":"2024-05-08T05:49:21.740362+0000","last_peered":"2024-05-08T05:49:21.740362+0000","last_clean":"2024-05-08T05:49:21.740362+0000","last_became_active":"2024-05-08T05:49:20.228303+0000","last_became_peered":"2024-05-08T05:49:20.228303+0000","last_unstale":"2024-05-08T05:49:21.740362+0000","last_undegraded":"2024-05-08T05:49:21.740362+0000","last_fullsized":"2024-05-08T05:49:21.740362+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:42.929693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740876+0000","last_change":"2024-05-08T05:49:20.226897+0000","last_active":"2024-05-08T05:49:21.740876+0000","last_peered":"2024-05-08T05:49:21.740876+0000","last_clean":"2024-05-08T05:49:21.740876+0000","last_became_active":"2024-05-08T05:49:20.226591+0000","last_became_peered":"2024-05-08T05:49:20.226591+0000","last_unstale":"2024-05-08T05:49:21.740876+0000","last_undegraded":"2024-05-08T05:49:21.740876+0000","last_fullsized":"2024-05-08T05:49:21.740876+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:22.685457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740345+0000","last_change":"2024-05-08T05:49:20.229697+0000","last_active":"2024-05-08T05:49:21.740345+0000","last_peered":"2024-05-08T05:49:21.740345+0000","last_clean":"2024-05-08T05:49:21.740345+0000","last_became_active":"2024-05-08T05:49:20.229413+0000","last_became_peered":"2024-05-08T05:49:20.229413+0000","last_unstale":"2024-05-08T05:49:21.740345+0000","last_undegraded":"2024-05-08T05:49:21.740345+0000","last_fullsized":"2024-05-08T05:49:21.740345+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:45.603541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712046+0000","last_change":"2024-05-08T05:49:20.231788+0000","last_active":"2024-05-08T05:49:21.712046+0000","last_peered":"2024-05-08T05:49:21.712046+0000","last_clean":"2024-05-08T05:49:21.712046+0000","last_became_active":"2024-05-08T05:49:20.231323+0000","last_became_peered":"2024-05-08T05:49:20.231323+0000","last_unstale":"2024-05-08T05:49:21.712046+0000","last_undegraded":"2024-05-08T05:49:21.712046+0000","last_fullsized":"2024-05-08T05:49:21.712046+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:40:06.613670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741679+0000","last_change":"2024-05-08T05:49:20.230080+0000","last_active":"2024-05-08T05:49:21.741679+0000","last_peered":"2024-05-08T05:49:21.741679+0000","last_clean":"2024-05-08T05:49:21.741679+0000","last_became_active":"2024-05-08T05:49:20.228395+0000","last_became_peered":"2024-05-08T05:49:20.228395+0000","last_unstale":"2024-05-08T05:49:21.741679+0000","last_undegraded":"2024-05-08T05:49:21.741679+0000","last_fullsized":"2024-05-08T05:49:21.741679+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:00:59.034430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740259+0000","last_change":"2024-05-08T05:49:20.235915+0000","last_active":"2024-05-08T05:49:21.740259+0000","last_peered":"2024-05-08T05:49:21.740259+0000","last_clean":"2024-05-08T05:49:21.740259+0000","last_became_active":"2024-05-08T05:49:20.235612+0000","last_became_peered":"2024-05-08T05:49:20.235612+0000","last_unstale":"2024-05-08T05:49:21.740259+0000","last_undegraded":"2024-05-08T05:49:21.740259+0000","last_fullsized":"2024-05-08T05:49:21.740259+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:33.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740474+0000","last_change":"2024-05-08T05:49:20.236606+0000","last_active":"2024-05-08T05:49:21.740474+0000","last_peered":"2024-05-08T05:49:21.740474+0000","last_clean":"2024-05-08T05:49:21.740474+0000","last_became_active":"2024-05-08T05:49:20.236336+0000","last_became_peered":"2024-05-08T05:49:20.236336+0000","last_unstale":"2024-05-08T05:49:21.740474+0000","last_undegraded":"2024-05-08T05:49:21.740474+0000","last_fullsized":"2024-05-08T05:49:21.740474+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:58.466547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222198+0000","last_change":"2024-05-08T05:49:20.225246+0000","last_active":"2024-05-08T05:49:21.222198+0000","last_peered":"2024-05-08T05:49:21.222198+0000","last_clean":"2024-05-08T05:49:21.222198+0000","last_became_active":"2024-05-08T05:49:20.224935+0000","last_became_peered":"2024-05-08T05:49:20.224935+0000","last_unstale":"2024-05-08T05:49:21.222198+0000","last_undegraded":"2024-05-08T05:49:21.222198+0000","last_fullsized":"2024-05-08T05:49:21.222198+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:45.603541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740114+0000","last_change":"2024-05-08T05:49:20.233325+0000","last_active":"2024-05-08T05:49:21.740114+0000","last_peered":"2024-05-08T05:49:21.740114+0000","last_clean":"2024-05-08T05:49:21.740114+0000","last_became_active":"2024-05-08T05:49:20.233018+0000","last_became_peered":"2024-05-08T05:49:20.233018+0000","last_unstale":"2024-05-08T05:49:21.740114+0000","last_undegraded":"2024-05-08T05:49:21.740114+0000","last_fullsized":"2024-05-08T05:49:21.740114+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:43:11.551671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740998+0000","last_change":"2024-05-08T05:49:20.224796+0000","last_active":"2024-05-08T05:49:21.740998+0000","last_peered":"2024-05-08T05:49:21.740998+0000","last_clean":"2024-05-08T05:49:21.740998+0000","last_became_active":"2024-05-08T05:49:20.224480+0000","last_became_peered":"2024-05-08T05:49:20.224480+0000","last_unstale":"2024-05-08T05:49:21.740998+0000","last_undegraded":"2024-05-08T05:49:21.740998+0000","last_fullsized":"2024-05-08T05:49:21.740998+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:58:10.209690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740735+0000","last_change":"2024-05-08T05:49:20.229339+0000","last_active":"2024-05-08T05:49:21.740735+0000","last_peered":"2024-05-08T05:49:21.740735+0000","last_clean":"2024-05-08T05:49:21.740735+0000","last_became_active":"2024-05-08T05:49:20.229066+0000","last_became_peered":"2024-05-08T05:49:20.229066+0000","last_unstale":"2024-05-08T05:49:21.740735+0000","last_undegraded":"2024-05-08T05:49:21.740735+0000","last_fullsized":"2024-05-08T05:49:21.740735+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:54.490928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740316+0000","last_change":"2024-05-08T05:49:20.228402+0000","last_active":"2024-05-08T05:49:21.740316+0000","last_peered":"2024-05-08T05:49:21.740316+0000","last_clean":"2024-05-08T05:49:21.740316+0000","last_became_active":"2024-05-08T05:49:20.228132+0000","last_became_peered":"2024-05-08T05:49:20.228132+0000","last_unstale":"2024-05-08T05:49:21.740316+0000","last_undegraded":"2024-05-08T05:49:21.740316+0000","last_fullsized":"2024-05-08T05:49:21.740316+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:55:30.372059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220628+0000","last_change":"2024-05-08T05:49:20.228209+0000","last_active":"2024-05-08T05:49:21.220628+0000","last_peered":"2024-05-08T05:49:21.220628+0000","last_clean":"2024-05-08T05:49:21.220628+0000","last_became_active":"2024-05-08T05:49:20.227869+0000","last_became_peered":"2024-05-08T05:49:20.227869+0000","last_unstale":"2024-05-08T05:49:21.220628+0000","last_undegraded":"2024-05-08T05:49:21.220628+0000","last_fullsized":"2024-05-08T05:49:21.220628+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:24.394443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741707+0000","last_change":"2024-05-08T05:49:20.218731+0000","last_active":"2024-05-08T05:49:21.741707+0000","last_peered":"2024-05-08T05:49:21.741707+0000","last_clean":"2024-05-08T05:49:21.741707+0000","last_became_active":"2024-05-08T05:49:20.218388+0000","last_became_peered":"2024-05-08T05:49:20.218388+0000","last_unstale":"2024-05-08T05:49:21.741707+0000","last_undegraded":"2024-05-08T05:49:21.741707+0000","last_fullsized":"2024-05-08T05:49:21.741707+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:00:44.400710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220462+0000","last_change":"2024-05-08T05:49:20.241520+0000","last_active":"2024-05-08T05:49:21.220462+0000","last_peered":"2024-05-08T05:49:21.220462+0000","last_clean":"2024-05-08T05:49:21.220462+0000","last_became_active":"2024-05-08T05:49:20.241194+0000","last_became_peered":"2024-05-08T05:49:20.241194+0000","last_unstale":"2024-05-08T05:49:21.220462+0000","last_undegraded":"2024-05-08T05:49:21.220462+0000","last_fullsized":"2024-05-08T05:49:21.220462+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:44:17.867536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222371+0000","last_change":"2024-05-08T05:49:20.222796+0000","last_active":"2024-05-08T05:49:21.222371+0000","last_peered":"2024-05-08T05:49:21.222371+0000","last_clean":"2024-05-08T05:49:21.222371+0000","last_became_active":"2024-05-08T05:49:20.222415+0000","last_became_peered":"2024-05-08T05:49:20.222415+0000","last_unstale":"2024-05-08T05:49:21.222371+0000","last_undegraded":"2024-05-08T05:49:21.222371+0000","last_fullsized":"2024-05-08T05:49:21.222371+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:20.999287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712297+0000","last_change":"2024-05-08T05:49:20.236618+0000","last_active":"2024-05-08T05:49:21.712297+0000","last_peered":"2024-05-08T05:49:21.712297+0000","last_clean":"2024-05-08T05:49:21.712297+0000","last_became_active":"2024-05-08T05:49:20.236323+0000","last_became_peered":"2024-05-08T05:49:20.236323+0000","last_unstale":"2024-05-08T05:49:21.712297+0000","last_undegraded":"2024-05-08T05:49:21.712297+0000","last_fullsized":"2024-05-08T05:49:21.712297+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:43:28.803611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740244+0000","last_change":"2024-05-08T05:49:20.228986+0000","last_active":"2024-05-08T05:49:21.740244+0000","last_peered":"2024-05-08T05:49:21.740244+0000","last_clean":"2024-05-08T05:49:21.740244+0000","last_became_active":"2024-05-08T05:49:20.228723+0000","last_became_peered":"2024-05-08T05:49:20.228723+0000","last_unstale":"2024-05-08T05:49:21.740244+0000","last_undegraded":"2024-05-08T05:49:21.740244+0000","last_fullsized":"2024-05-08T05:49:21.740244+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:17:48.859578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220270+0000","last_change":"2024-05-08T05:49:20.225628+0000","last_active":"2024-05-08T05:49:21.220270+0000","last_peered":"2024-05-08T05:49:21.220270+0000","last_clean":"2024-05-08T05:49:21.220270+0000","last_became_active":"2024-05-08T05:49:20.225065+0000","last_became_peered":"2024-05-08T05:49:20.225065+0000","last_unstale":"2024-05-08T05:49:21.220270+0000","last_undegraded":"2024-05-08T05:49:21.220270+0000","last_fullsized":"2024-05-08T05:49:21.220270+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:00:44.636681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740709+0000","last_change":"2024-05-08T05:49:20.239478+0000","last_active":"2024-05-08T05:49:21.740709+0000","last_peered":"2024-05-08T05:49:21.740709+0000","last_clean":"2024-05-08T05:49:21.740709+0000","last_became_active":"2024-05-08T05:49:20.239214+0000","last_became_peered":"2024-05-08T05:49:20.239214+0000","last_unstale":"2024-05-08T05:49:21.740709+0000","last_undegraded":"2024-05-08T05:49:21.740709+0000","last_fullsized":"2024-05-08T05:49:21.740709+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:45.603541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222394+0000","last_change":"2024-05-08T05:49:20.229377+0000","last_active":"2024-05-08T05:49:21.222394+0000","last_peered":"2024-05-08T05:49:21.222394+0000","last_clean":"2024-05-08T05:49:21.222394+0000","last_became_active":"2024-05-08T05:49:20.229049+0000","last_became_peered":"2024-05-08T05:49:20.229049+0000","last_unstale":"2024-05-08T05:49:21.222394+0000","last_undegraded":"2024-05-08T05:49:21.222394+0000","last_fullsized":"2024-05-08T05:49:21.222394+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:26:27.901303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222386+0000","last_change":"2024-05-08T05:49:20.231741+0000","last_active":"2024-05-08T05:49:21.222386+0000","last_peered":"2024-05-08T05:49:21.222386+0000","last_clean":"2024-05-08T05:49:21.222386+0000","last_became_active":"2024-05-08T05:49:20.231459+0000","last_became_peered":"2024-05-08T05:49:20.231459+0000","last_unstale":"2024-05-08T05:49:21.222386+0000","last_undegraded":"2024-05-08T05:49:21.222386+0000","last_fullsized":"2024-05-08T05:49:21.222386+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:22.685457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740538+0000","last_change":"2024-05-08T05:49:20.228283+0000","last_active":"2024-05-08T05:49:21.740538+0000","last_peered":"2024-05-08T05:49:21.740538+0000","last_clean":"2024-05-08T05:49:21.740538+0000","last_became_active":"2024-05-08T05:49:20.227936+0000","last_became_peered":"2024-05-08T05:49:20.227936+0000","last_unstale":"2024-05-08T05:49:21.740538+0000","last_undegraded":"2024-05-08T05:49:21.740538+0000","last_fullsized":"2024-05-08T05:49:21.740538+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:45.720876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741377+0000","last_change":"2024-05-08T05:49:20.231471+0000","last_active":"2024-05-08T05:49:21.741377+0000","last_peered":"2024-05-08T05:49:21.741377+0000","last_clean":"2024-05-08T05:49:21.741377+0000","last_became_active":"2024-05-08T05:49:20.230910+0000","last_became_peered":"2024-05-08T05:49:20.230910+0000","last_unstale":"2024-05-08T05:49:21.741377+0000","last_undegraded":"2024-05-08T05:49:21.741377+0000","last_fullsized":"2024-05-08T05:49:21.741377+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:02:05.771334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712332+0000","last_change":"2024-05-08T05:49:20.230069+0000","last_active":"2024-05-08T05:49:21.712332+0000","last_peered":"2024-05-08T05:49:21.712332+0000","last_clean":"2024-05-08T05:49:21.712332+0000","last_became_active":"2024-05-08T05:49:20.229773+0000","last_became_peered":"2024-05-08T05:49:20.229773+0000","last_unstale":"2024-05-08T05:49:21.712332+0000","last_undegraded":"2024-05-08T05:49:21.712332+0000","last_fullsized":"2024-05-08T05:49:21.712332+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:52:31.166852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.221696+0000","last_change":"2024-05-08T05:49:20.228744+0000","last_active":"2024-05-08T05:49:21.221696+0000","last_peered":"2024-05-08T05:49:21.221696+0000","last_clean":"2024-05-08T05:49:21.221696+0000","last_became_active":"2024-05-08T05:49:20.228426+0000","last_became_peered":"2024-05-08T05:49:20.228426+0000","last_unstale":"2024-05-08T05:49:21.221696+0000","last_undegraded":"2024-05-08T05:49:21.221696+0000","last_fullsized":"2024-05-08T05:49:21.221696+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:58.466547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740518+0000","last_change":"2024-05-08T05:49:20.239053+0000","last_active":"2024-05-08T05:49:21.740518+0000","last_peered":"2024-05-08T05:49:21.740518+0000","last_clean":"2024-05-08T05:49:21.740518+0000","last_became_active":"2024-05-08T05:49:20.238726+0000","last_became_peered":"2024-05-08T05:49:20.238726+0000","last_unstale":"2024-05-08T05:49:21.740518+0000","last_undegraded":"2024-05-08T05:49:21.740518+0000","last_fullsized":"2024-05-08T05:49:21.740518+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:20.999287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712233+0000","last_change":"2024-05-08T05:49:20.235736+0000","last_active":"2024-05-08T05:49:21.712233+0000","last_peered":"2024-05-08T05:49:21.712233+0000","last_clean":"2024-05-08T05:49:21.712233+0000","last_became_active":"2024-05-08T05:49:20.235467+0000","last_became_peered":"2024-05-08T05:49:20.235467+0000","last_unstale":"2024-05-08T05:49:21.712233+0000","last_undegraded":"2024-05-08T05:49:21.712233+0000","last_fullsized":"2024-05-08T05:49:21.712233+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:04:11.748952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220519+0000","last_change":"2024-05-08T05:49:20.226786+0000","last_active":"2024-05-08T05:49:21.220519+0000","last_peered":"2024-05-08T05:49:21.220519+0000","last_clean":"2024-05-08T05:49:21.220519+0000","last_became_active":"2024-05-08T05:49:20.223580+0000","last_became_peered":"2024-05-08T05:49:20.223580+0000","last_unstale":"2024-05-08T05:49:21.220519+0000","last_undegraded":"2024-05-08T05:49:21.220519+0000","last_fullsized":"2024-05-08T05:49:21.220519+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:32:33.705132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"103'4","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740566+0000","last_change":"2024-05-08T05:49:19.226689+0000","last_active":"2024-05-08T05:49:21.740566+0000","last_peered":"2024-05-08T05:49:21.740566+0000","last_clean":"2024-05-08T05:49:21.740566+0000","last_became_active":"2024-05-08T05:49:19.226431+0000","last_became_peered":"2024-05-08T05:49:19.226431+0000","last_unstale":"2024-05-08T05:49:21.740566+0000","last_undegraded":"2024-05-08T05:49:21.740566+0000","last_fullsized":"2024-05-08T05:49:21.740566+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:03.552401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712265+0000","last_change":"2024-05-08T05:49:20.235706+0000","last_active":"2024-05-08T05:49:21.712265+0000","last_peered":"2024-05-08T05:49:21.712265+0000","last_clean":"2024-05-08T05:49:21.712265+0000","last_became_active":"2024-05-08T05:49:20.235443+0000","last_became_peered":"2024-05-08T05:49:20.235443+0000","last_unstale":"2024-05-08T05:49:21.712265+0000","last_undegraded":"2024-05-08T05:49:21.712265+0000","last_fullsized":"2024-05-08T05:49:21.712265+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:03:14.991159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712275+0000","last_change":"2024-05-08T05:49:19.225382+0000","last_active":"2024-05-08T05:49:21.712275+0000","last_peered":"2024-05-08T05:49:21.712275+0000","last_clean":"2024-05-08T05:49:21.712275+0000","last_became_active":"2024-05-08T05:49:19.225112+0000","last_became_peered":"2024-05-08T05:49:19.225112+0000","last_unstale":"2024-05-08T05:49:21.712275+0000","last_undegraded":"2024-05-08T05:49:21.712275+0000","last_fullsized":"2024-05-08T05:49:21.712275+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:38:16.153929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222069+0000","last_change":"2024-05-08T05:49:20.220318+0000","last_active":"2024-05-08T05:49:21.222069+0000","last_peered":"2024-05-08T05:49:21.222069+0000","last_clean":"2024-05-08T05:49:21.222069+0000","last_became_active":"2024-05-08T05:49:20.217950+0000","last_became_peered":"2024-05-08T05:49:20.217950+0000","last_unstale":"2024-05-08T05:49:21.222069+0000","last_undegraded":"2024-05-08T05:49:21.222069+0000","last_fullsized":"2024-05-08T05:49:21.222069+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:45.720876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222167+0000","last_change":"2024-05-08T05:49:19.217546+0000","last_active":"2024-05-08T05:49:21.222167+0000","last_peered":"2024-05-08T05:49:21.222167+0000","last_clean":"2024-05-08T05:49:21.222167+0000","last_became_active":"2024-05-08T05:49:19.217253+0000","last_became_peered":"2024-05-08T05:49:19.217253+0000","last_unstale":"2024-05-08T05:49:21.222167+0000","last_undegraded":"2024-05-08T05:49:21.222167+0000","last_fullsized":"2024-05-08T05:49:21.222167+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:06:33.415067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740912+0000","last_change":"2024-05-08T05:49:20.231417+0000","last_active":"2024-05-08T05:49:21.740912+0000","last_peered":"2024-05-08T05:49:21.740912+0000","last_clean":"2024-05-08T05:49:21.740912+0000","last_became_active":"2024-05-08T05:49:20.227160+0000","last_became_peered":"2024-05-08T05:49:20.227160+0000","last_unstale":"2024-05-08T05:49:21.740912+0000","last_undegraded":"2024-05-08T05:49:21.740912+0000","last_fullsized":"2024-05-08T05:49:21.740912+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:22.685457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740857+0000","last_change":"2024-05-08T05:49:19.223093+0000","last_active":"2024-05-08T05:49:21.740857+0000","last_peered":"2024-05-08T05:49:21.740857+0000","last_clean":"2024-05-08T05:49:21.740857+0000","last_became_active":"2024-05-08T05:49:19.222791+0000","last_became_peered":"2024-05-08T05:49:19.222791+0000","last_unstale":"2024-05-08T05:49:21.740857+0000","last_undegraded":"2024-05-08T05:49:21.740857+0000","last_fullsized":"2024-05-08T05:49:21.740857+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:39:02.674771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741089+0000","last_change":"2024-05-08T05:49:20.229318+0000","last_active":"2024-05-08T05:49:21.741089+0000","last_peered":"2024-05-08T05:49:21.741089+0000","last_clean":"2024-05-08T05:49:21.741089+0000","last_became_active":"2024-05-08T05:49:20.229024+0000","last_became_peered":"2024-05-08T05:49:20.229024+0000","last_unstale":"2024-05-08T05:49:21.741089+0000","last_undegraded":"2024-05-08T05:49:21.741089+0000","last_fullsized":"2024-05-08T05:49:21.741089+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:59:57.323293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741131+0000","last_change":"2024-05-08T05:49:19.224342+0000","last_active":"2024-05-08T05:49:21.741131+0000","last_peered":"2024-05-08T05:49:21.741131+0000","last_clean":"2024-05-08T05:49:21.741131+0000","last_became_active":"2024-05-08T05:49:19.224020+0000","last_became_peered":"2024-05-08T05:49:19.224020+0000","last_unstale":"2024-05-08T05:49:21.741131+0000","last_undegraded":"2024-05-08T05:49:21.741131+0000","last_fullsized":"2024-05-08T05:49:21.741131+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:00:58.026398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220824+0000","last_change":"2024-05-08T05:49:20.224834+0000","last_active":"2024-05-08T05:49:21.220824+0000","last_peered":"2024-05-08T05:49:21.220824+0000","last_clean":"2024-05-08T05:49:21.220824+0000","last_became_active":"2024-05-08T05:49:20.224174+0000","last_became_peered":"2024-05-08T05:49:20.224174+0000","last_unstale":"2024-05-08T05:49:21.220824+0000","last_undegraded":"2024-05-08T05:49:21.220824+0000","last_fullsized":"2024-05-08T05:49:21.220824+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:49:23.007222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740317+0000","last_change":"2024-05-08T05:49:19.219998+0000","last_active":"2024-05-08T05:49:21.740317+0000","last_peered":"2024-05-08T05:49:21.740317+0000","last_clean":"2024-05-08T05:49:21.740317+0000","last_became_active":"2024-05-08T05:49:19.219620+0000","last_became_peered":"2024-05-08T05:49:19.219620+0000","last_unstale":"2024-05-08T05:49:21.740317+0000","last_undegraded":"2024-05-08T05:49:21.740317+0000","last_fullsized":"2024-05-08T05:49:21.740317+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:40:47.136337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741866+0000","last_change":"2024-05-08T05:49:20.236304+0000","last_active":"2024-05-08T05:49:21.741866+0000","last_peered":"2024-05-08T05:49:21.741866+0000","last_clean":"2024-05-08T05:49:21.741866+0000","last_became_active":"2024-05-08T05:49:20.235963+0000","last_became_peered":"2024-05-08T05:49:20.235963+0000","last_unstale":"2024-05-08T05:49:21.741866+0000","last_undegraded":"2024-05-08T05:49:21.741866+0000","last_fullsized":"2024-05-08T05:49:21.741866+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:55:24.707275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.221673+0000","last_change":"2024-05-08T05:49:19.217561+0000","last_active":"2024-05-08T05:49:21.221673+0000","last_peered":"2024-05-08T05:49:21.221673+0000","last_clean":"2024-05-08T05:49:21.221673+0000","last_became_active":"2024-05-08T05:49:19.217233+0000","last_became_peered":"2024-05-08T05:49:19.217233+0000","last_unstale":"2024-05-08T05:49:21.221673+0000","last_undegraded":"2024-05-08T05:49:21.221673+0000","last_fullsized":"2024-05-08T05:49:21.221673+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:39:02.674771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740504+0000","last_change":"2024-05-08T05:49:20.239988+0000","last_active":"2024-05-08T05:49:21.740504+0000","last_peered":"2024-05-08T05:49:21.740504+0000","last_clean":"2024-05-08T05:49:21.740504+0000","last_became_active":"2024-05-08T05:49:20.239698+0000","last_became_peered":"2024-05-08T05:49:20.239698+0000","last_unstale":"2024-05-08T05:49:21.740504+0000","last_undegraded":"2024-05-08T05:49:21.740504+0000","last_fullsized":"2024-05-08T05:49:21.740504+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:55:30.372059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.739885+0000","last_change":"2024-05-08T05:49:19.219609+0000","last_active":"2024-05-08T05:49:21.739885+0000","last_peered":"2024-05-08T05:49:21.739885+0000","last_clean":"2024-05-08T05:49:21.739885+0000","last_became_active":"2024-05-08T05:49:19.219336+0000","last_became_peered":"2024-05-08T05:49:19.219336+0000","last_unstale":"2024-05-08T05:49:21.739885+0000","last_undegraded":"2024-05-08T05:49:21.739885+0000","last_fullsized":"2024-05-08T05:49:21.739885+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:26:26.893270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220290+0000","last_change":"2024-05-08T05:49:20.241346+0000","last_active":"2024-05-08T05:49:21.220290+0000","last_peered":"2024-05-08T05:49:21.220290+0000","last_clean":"2024-05-08T05:49:21.220290+0000","last_became_active":"2024-05-08T05:49:20.241001+0000","last_became_peered":"2024-05-08T05:49:20.241001+0000","last_unstale":"2024-05-08T05:49:21.220290+0000","last_undegraded":"2024-05-08T05:49:21.220290+0000","last_fullsized":"2024-05-08T05:49:21.220290+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:17:51.356349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222388+0000","last_change":"2024-05-08T05:49:19.217501+0000","last_active":"2024-05-08T05:49:21.222388+0000","last_peered":"2024-05-08T05:49:21.222388+0000","last_clean":"2024-05-08T05:49:21.222388+0000","last_became_active":"2024-05-08T05:49:19.217160+0000","last_became_peered":"2024-05-08T05:49:19.217160+0000","last_unstale":"2024-05-08T05:49:21.222388+0000","last_undegraded":"2024-05-08T05:49:21.222388+0000","last_fullsized":"2024-05-08T05:49:21.222388+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:40:47.136337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740750+0000","last_change":"2024-05-08T05:49:20.225909+0000","last_active":"2024-05-08T05:49:21.740750+0000","last_peered":"2024-05-08T05:49:21.740750+0000","last_clean":"2024-05-08T05:49:21.740750+0000","last_became_active":"2024-05-08T05:49:20.225617+0000","last_became_peered":"2024-05-08T05:49:20.225617+0000","last_unstale":"2024-05-08T05:49:21.740750+0000","last_undegraded":"2024-05-08T05:49:21.740750+0000","last_fullsized":"2024-05-08T05:49:21.740750+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:20.999287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220357+0000","last_change":"2024-05-08T05:49:19.211783+0000","last_active":"2024-05-08T05:49:21.220357+0000","last_peered":"2024-05-08T05:49:21.220357+0000","last_clean":"2024-05-08T05:49:21.220357+0000","last_became_active":"2024-05-08T05:49:19.211495+0000","last_became_peered":"2024-05-08T05:49:19.211495+0000","last_unstale":"2024-05-08T05:49:21.220357+0000","last_undegraded":"2024-05-08T05:49:21.220357+0000","last_fullsized":"2024-05-08T05:49:21.220357+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:33:36.204034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712197+0000","last_change":"2024-05-08T05:49:20.232202+0000","last_active":"2024-05-08T05:49:21.712197+0000","last_peered":"2024-05-08T05:49:21.712197+0000","last_clean":"2024-05-08T05:49:21.712197+0000","last_became_active":"2024-05-08T05:49:20.231919+0000","last_became_peered":"2024-05-08T05:49:20.231919+0000","last_unstale":"2024-05-08T05:49:21.712197+0000","last_undegraded":"2024-05-08T05:49:21.712197+0000","last_fullsized":"2024-05-08T05:49:21.712197+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:30:47.129884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"103'3","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740334+0000","last_change":"2024-05-08T05:49:19.220761+0000","last_active":"2024-05-08T05:49:21.740334+0000","last_peered":"2024-05-08T05:49:21.740334+0000","last_clean":"2024-05-08T05:49:21.740334+0000","last_became_active":"2024-05-08T05:49:19.220244+0000","last_became_peered":"2024-05-08T05:49:19.220244+0000","last_unstale":"2024-05-08T05:49:21.740334+0000","last_undegraded":"2024-05-08T05:49:21.740334+0000","last_fullsized":"2024-05-08T05:49:21.740334+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:00:43.392678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740671+0000","last_change":"2024-05-08T05:49:20.235723+0000","last_active":"2024-05-08T05:49:21.740671+0000","last_peered":"2024-05-08T05:49:21.740671+0000","last_clean":"2024-05-08T05:49:21.740671+0000","last_became_active":"2024-05-08T05:49:20.235051+0000","last_became_peered":"2024-05-08T05:49:20.235051+0000","last_unstale":"2024-05-08T05:49:21.740671+0000","last_undegraded":"2024-05-08T05:49:21.740671+0000","last_fullsized":"2024-05-08T05:49:21.740671+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:45.720876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"103'6","reported_seq":20,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741184+0000","last_change":"2024-05-08T05:49:19.220558+0000","last_active":"2024-05-08T05:49:21.741184+0000","last_peered":"2024-05-08T05:49:21.741184+0000","last_clean":"2024-05-08T05:49:21.741184+0000","last_became_active":"2024-05-08T05:49:19.220251+0000","last_became_peered":"2024-05-08T05:49:19.220251+0000","last_unstale":"2024-05-08T05:49:21.741184+0000","last_undegraded":"2024-05-08T05:49:21.741184+0000","last_fullsized":"2024-05-08T05:49:21.741184+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:06:33.415067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740567+0000","last_change":"2024-05-08T05:49:20.229666+0000","last_active":"2024-05-08T05:49:21.740567+0000","last_peered":"2024-05-08T05:49:21.740567+0000","last_clean":"2024-05-08T05:49:21.740567+0000","last_became_active":"2024-05-08T05:49:20.227506+0000","last_became_peered":"2024-05-08T05:49:20.227506+0000","last_unstale":"2024-05-08T05:49:21.740567+0000","last_undegraded":"2024-05-08T05:49:21.740567+0000","last_fullsized":"2024-05-08T05:49:21.740567+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:45.603541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740730+0000","last_change":"2024-05-08T05:49:19.225759+0000","last_active":"2024-05-08T05:49:21.740730+0000","last_peered":"2024-05-08T05:49:21.740730+0000","last_clean":"2024-05-08T05:49:21.740730+0000","last_became_active":"2024-05-08T05:49:19.225054+0000","last_became_peered":"2024-05-08T05:49:19.225054+0000","last_unstale":"2024-05-08T05:49:21.740730+0000","last_undegraded":"2024-05-08T05:49:21.740730+0000","last_fullsized":"2024-05-08T05:49:21.740730+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:06:33.415067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741384+0000","last_change":"2024-05-08T05:49:20.225107+0000","last_active":"2024-05-08T05:49:21.741384+0000","last_peered":"2024-05-08T05:49:21.741384+0000","last_clean":"2024-05-08T05:49:21.741384+0000","last_became_active":"2024-05-08T05:49:20.224832+0000","last_became_peered":"2024-05-08T05:49:20.224832+0000","last_unstale":"2024-05-08T05:49:21.741384+0000","last_undegraded":"2024-05-08T05:49:21.741384+0000","last_fullsized":"2024-05-08T05:49:21.741384+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:04.560434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740682+0000","last_change":"2024-05-08T05:49:19.220845+0000","last_active":"2024-05-08T05:49:21.740682+0000","last_peered":"2024-05-08T05:49:21.740682+0000","last_clean":"2024-05-08T05:49:21.740682+0000","last_became_active":"2024-05-08T05:49:19.220519+0000","last_became_peered":"2024-05-08T05:49:19.220519+0000","last_unstale":"2024-05-08T05:49:21.740682+0000","last_undegraded":"2024-05-08T05:49:21.740682+0000","last_fullsized":"2024-05-08T05:49:21.740682+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:02:04.763302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712390+0000","last_change":"2024-05-08T05:49:20.234140+0000","last_active":"2024-05-08T05:49:21.712390+0000","last_peered":"2024-05-08T05:49:21.712390+0000","last_clean":"2024-05-08T05:49:21.712390+0000","last_became_active":"2024-05-08T05:49:20.233639+0000","last_became_peered":"2024-05-08T05:49:20.233639+0000","last_unstale":"2024-05-08T05:49:21.712390+0000","last_undegraded":"2024-05-08T05:49:21.712390+0000","last_fullsized":"2024-05-08T05:49:21.712390+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:52:21.870490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712126+0000","last_change":"2024-05-08T05:49:19.226502+0000","last_active":"2024-05-08T05:49:21.712126+0000","last_peered":"2024-05-08T05:49:21.712126+0000","last_clean":"2024-05-08T05:49:21.712126+0000","last_became_active":"2024-05-08T05:49:19.226217+0000","last_became_peered":"2024-05-08T05:49:19.226217+0000","last_unstale":"2024-05-08T05:49:21.712126+0000","last_undegraded":"2024-05-08T05:49:21.712126+0000","last_fullsized":"2024-05-08T05:49:21.712126+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:26:51.894930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220622+0000","last_change":"2024-05-08T05:49:20.228556+0000","last_active":"2024-05-08T05:49:21.220622+0000","last_peered":"2024-05-08T05:49:21.220622+0000","last_clean":"2024-05-08T05:49:21.220622+0000","last_became_active":"2024-05-08T05:49:20.228242+0000","last_became_peered":"2024-05-08T05:49:20.228242+0000","last_unstale":"2024-05-08T05:49:21.220622+0000","last_undegraded":"2024-05-08T05:49:21.220622+0000","last_fullsized":"2024-05-08T05:49:21.220622+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:44.880570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"103'8","reported_seq":22,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.221517+0000","last_change":"2024-05-08T05:49:19.221739+0000","last_active":"2024-05-08T05:49:21.221517+0000","last_peered":"2024-05-08T05:49:21.221517+0000","last_clean":"2024-05-08T05:49:21.221517+0000","last_became_active":"2024-05-08T05:49:19.221466+0000","last_became_peered":"2024-05-08T05:49:19.221466+0000","last_unstale":"2024-05-08T05:49:21.221517+0000","last_undegraded":"2024-05-08T05:49:21.221517+0000","last_fullsized":"2024-05-08T05:49:21.221517+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:01.127846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712529+0000","last_change":"2024-05-08T05:49:20.231557+0000","last_active":"2024-05-08T05:49:21.712529+0000","last_peered":"2024-05-08T05:49:21.712529+0000","last_clean":"2024-05-08T05:49:21.712529+0000","last_became_active":"2024-05-08T05:49:20.231028+0000","last_became_peered":"2024-05-08T05:49:20.231028+0000","last_unstale":"2024-05-08T05:49:21.712529+0000","last_undegraded":"2024-05-08T05:49:21.712529+0000","last_fullsized":"2024-05-08T05:49:21.712529+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:59:56.161183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741817+0000","last_change":"2024-05-08T05:49:19.219459+0000","last_active":"2024-05-08T05:49:21.741817+0000","last_peered":"2024-05-08T05:49:21.741817+0000","last_clean":"2024-05-08T05:49:21.741817+0000","last_became_active":"2024-05-08T05:49:19.219157+0000","last_became_peered":"2024-05-08T05:49:19.219157+0000","last_unstale":"2024-05-08T05:49:21.741817+0000","last_undegraded":"2024-05-08T05:49:21.741817+0000","last_fullsized":"2024-05-08T05:49:21.741817+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:57.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741175+0000","last_change":"2024-05-08T05:49:20.238812+0000","last_active":"2024-05-08T05:49:21.741175+0000","last_peered":"2024-05-08T05:49:21.741175+0000","last_clean":"2024-05-08T05:49:21.741175+0000","last_became_active":"2024-05-08T05:49:20.238456+0000","last_became_peered":"2024-05-08T05:49:20.238456+0000","last_unstale":"2024-05-08T05:49:21.741175+0000","last_undegraded":"2024-05-08T05:49:21.741175+0000","last_fullsized":"2024-05-08T05:49:21.741175+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:26:27.901303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741280+0000","last_change":"2024-05-08T05:49:19.221812+0000","last_active":"2024-05-08T05:49:21.741280+0000","last_peered":"2024-05-08T05:49:21.741280+0000","last_clean":"2024-05-08T05:49:21.741280+0000","last_became_active":"2024-05-08T05:49:19.221501+0000","last_became_peered":"2024-05-08T05:49:19.221501+0000","last_unstale":"2024-05-08T05:49:21.741280+0000","last_undegraded":"2024-05-08T05:49:21.741280+0000","last_fullsized":"2024-05-08T05:49:21.741280+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:55:23.699243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740514+0000","last_change":"2024-05-08T05:49:20.235973+0000","last_active":"2024-05-08T05:49:21.740514+0000","last_peered":"2024-05-08T05:49:21.740514+0000","last_clean":"2024-05-08T05:49:21.740514+0000","last_became_active":"2024-05-08T05:49:20.235442+0000","last_became_peered":"2024-05-08T05:49:20.235442+0000","last_unstale":"2024-05-08T05:49:21.740514+0000","last_undegraded":"2024-05-08T05:49:21.740514+0000","last_fullsized":"2024-05-08T05:49:21.740514+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:58.466547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740680+0000","last_change":"2024-05-08T05:49:19.222187+0000","last_active":"2024-05-08T05:49:21.740680+0000","last_peered":"2024-05-08T05:49:21.740680+0000","last_clean":"2024-05-08T05:49:21.740680+0000","last_became_active":"2024-05-08T05:49:19.221863+0000","last_became_peered":"2024-05-08T05:49:19.221863+0000","last_unstale":"2024-05-08T05:49:21.740680+0000","last_undegraded":"2024-05-08T05:49:21.740680+0000","last_fullsized":"2024-05-08T05:49:21.740680+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:57.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220349+0000","last_change":"2024-05-08T05:49:20.237821+0000","last_active":"2024-05-08T05:49:21.220349+0000","last_peered":"2024-05-08T05:49:21.220349+0000","last_clean":"2024-05-08T05:49:21.220349+0000","last_became_active":"2024-05-08T05:49:20.237417+0000","last_became_peered":"2024-05-08T05:49:20.237417+0000","last_unstale":"2024-05-08T05:49:21.220349+0000","last_undegraded":"2024-05-08T05:49:21.220349+0000","last_fullsized":"2024-05-08T05:49:21.220349+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:06:50.147335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712469+0000","last_change":"2024-05-08T05:49:19.222219+0000","last_active":"2024-05-08T05:49:21.712469+0000","last_peered":"2024-05-08T05:49:21.712469+0000","last_clean":"2024-05-08T05:49:21.712469+0000","last_became_active":"2024-05-08T05:49:19.221938+0000","last_became_peered":"2024-05-08T05:49:19.221938+0000","last_unstale":"2024-05-08T05:49:21.712469+0000","last_undegraded":"2024-05-08T05:49:21.712469+0000","last_fullsized":"2024-05-08T05:49:21.712469+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:11.598891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220747+0000","last_change":"2024-05-08T05:49:20.225075+0000","last_active":"2024-05-08T05:49:21.220747+0000","last_peered":"2024-05-08T05:49:21.220747+0000","last_clean":"2024-05-08T05:49:21.220747+0000","last_became_active":"2024-05-08T05:49:20.224537+0000","last_became_peered":"2024-05-08T05:49:20.224537+0000","last_unstale":"2024-05-08T05:49:21.220747+0000","last_undegraded":"2024-05-08T05:49:21.220747+0000","last_fullsized":"2024-05-08T05:49:21.220747+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:10:17.796692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.711959+0000","last_change":"2024-05-08T05:49:19.226655+0000","last_active":"2024-05-08T05:49:21.711959+0000","last_peered":"2024-05-08T05:49:21.711959+0000","last_clean":"2024-05-08T05:49:21.711959+0000","last_became_active":"2024-05-08T05:49:19.226385+0000","last_became_peered":"2024-05-08T05:49:19.226385+0000","last_unstale":"2024-05-08T05:49:21.711959+0000","last_undegraded":"2024-05-08T05:49:21.711959+0000","last_fullsized":"2024-05-08T05:49:21.711959+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:09.778236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740421+0000","last_change":"2024-05-08T05:49:20.235440+0000","last_active":"2024-05-08T05:49:21.740421+0000","last_peered":"2024-05-08T05:49:21.740421+0000","last_clean":"2024-05-08T05:49:21.740421+0000","last_became_active":"2024-05-08T05:49:20.235109+0000","last_became_peered":"2024-05-08T05:49:20.235109+0000","last_unstale":"2024-05-08T05:49:21.740421+0000","last_undegraded":"2024-05-08T05:49:21.740421+0000","last_fullsized":"2024-05-08T05:49:21.740421+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:08:25.723528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740617+0000","last_change":"2024-05-08T05:49:19.218899+0000","last_active":"2024-05-08T05:49:21.740617+0000","last_peered":"2024-05-08T05:49:21.740617+0000","last_clean":"2024-05-08T05:49:21.740617+0000","last_became_active":"2024-05-08T05:49:19.218594+0000","last_became_peered":"2024-05-08T05:49:19.218594+0000","last_unstale":"2024-05-08T05:49:21.740617+0000","last_undegraded":"2024-05-08T05:49:21.740617+0000","last_fullsized":"2024-05-08T05:49:21.740617+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:01.127846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712522+0000","last_change":"2024-05-08T05:49:20.232503+0000","last_active":"2024-05-08T05:49:21.712522+0000","last_peered":"2024-05-08T05:49:21.712522+0000","last_clean":"2024-05-08T05:49:21.712522+0000","last_became_active":"2024-05-08T05:49:20.232220+0000","last_became_peered":"2024-05-08T05:49:20.232220+0000","last_unstale":"2024-05-08T05:49:21.712522+0000","last_undegraded":"2024-05-08T05:49:21.712522+0000","last_fullsized":"2024-05-08T05:49:21.712522+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:27:41.729361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712472+0000","last_change":"2024-05-08T05:49:19.225879+0000","last_active":"2024-05-08T05:49:21.712472+0000","last_peered":"2024-05-08T05:49:21.712472+0000","last_clean":"2024-05-08T05:49:21.712472+0000","last_became_active":"2024-05-08T05:49:19.225603+0000","last_became_peered":"2024-05-08T05:49:19.225603+0000","last_unstale":"2024-05-08T05:49:21.712472+0000","last_undegraded":"2024-05-08T05:49:21.712472+0000","last_fullsized":"2024-05-08T05:49:21.712472+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:36:23.099004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220321+0000","last_change":"2024-05-08T05:49:20.227026+0000","last_active":"2024-05-08T05:49:21.220321+0000","last_peered":"2024-05-08T05:49:21.220321+0000","last_clean":"2024-05-08T05:49:21.220321+0000","last_became_active":"2024-05-08T05:49:20.226533+0000","last_became_peered":"2024-05-08T05:49:20.226533+0000","last_unstale":"2024-05-08T05:49:21.220321+0000","last_undegraded":"2024-05-08T05:49:21.220321+0000","last_fullsized":"2024-05-08T05:49:21.220321+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:56:54.954385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740487+0000","last_change":"2024-05-08T05:49:19.225433+0000","last_active":"2024-05-08T05:49:21.740487+0000","last_peered":"2024-05-08T05:49:21.740487+0000","last_clean":"2024-05-08T05:49:21.740487+0000","last_became_active":"2024-05-08T05:49:19.225126+0000","last_became_peered":"2024-05-08T05:49:19.225126+0000","last_unstale":"2024-05-08T05:49:21.740487+0000","last_undegraded":"2024-05-08T05:49:21.740487+0000","last_fullsized":"2024-05-08T05:49:21.740487+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:01.127846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.221654+0000","last_change":"2024-05-08T05:49:20.222133+0000","last_active":"2024-05-08T05:49:21.221654+0000","last_peered":"2024-05-08T05:49:21.221654+0000","last_clean":"2024-05-08T05:49:21.221654+0000","last_became_active":"2024-05-08T05:49:20.221812+0000","last_became_peered":"2024-05-08T05:49:20.221812+0000","last_unstale":"2024-05-08T05:49:21.221654+0000","last_undegraded":"2024-05-08T05:49:21.221654+0000","last_fullsized":"2024-05-08T05:49:21.221654+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:42.929693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"104'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220391+0000","last_change":"2024-05-08T05:49:19.221744+0000","last_active":"2024-05-08T05:49:21.220391+0000","last_peered":"2024-05-08T05:49:21.220391+0000","last_clean":"2024-05-08T05:49:21.220391+0000","last_became_active":"2024-05-08T05:49:19.221395+0000","last_became_peered":"2024-05-08T05:49:19.221395+0000","last_unstale":"2024-05-08T05:49:21.220391+0000","last_undegraded":"2024-05-08T05:49:21.220391+0000","last_fullsized":"2024-05-08T05:49:21.220391+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:01:13.200266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740671+0000","last_change":"2024-05-08T05:49:20.231747+0000","last_active":"2024-05-08T05:49:21.740671+0000","last_peered":"2024-05-08T05:49:21.740671+0000","last_clean":"2024-05-08T05:49:21.740671+0000","last_became_active":"2024-05-08T05:49:20.231483+0000","last_became_peered":"2024-05-08T05:49:20.231483+0000","last_unstale":"2024-05-08T05:49:21.740671+0000","last_undegraded":"2024-05-08T05:49:21.740671+0000","last_fullsized":"2024-05-08T05:49:21.740671+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:54.490928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741491+0000","last_change":"2024-05-08T05:49:19.222418+0000","last_active":"2024-05-08T05:49:21.741491+0000","last_peered":"2024-05-08T05:49:21.741491+0000","last_clean":"2024-05-08T05:49:21.741491+0000","last_became_active":"2024-05-08T05:49:19.222125+0000","last_became_peered":"2024-05-08T05:49:19.222125+0000","last_unstale":"2024-05-08T05:49:21.741491+0000","last_undegraded":"2024-05-08T05:49:21.741491+0000","last_fullsized":"2024-05-08T05:49:21.741491+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:40:47.136337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740889+0000","last_change":"2024-05-08T05:49:20.230762+0000","last_active":"2024-05-08T05:49:21.740889+0000","last_peered":"2024-05-08T05:49:21.740889+0000","last_clean":"2024-05-08T05:49:21.740889+0000","last_became_active":"2024-05-08T05:49:20.228489+0000","last_became_peered":"2024-05-08T05:49:20.228489+0000","last_unstale":"2024-05-08T05:49:21.740889+0000","last_undegraded":"2024-05-08T05:49:21.740889+0000","last_fullsized":"2024-05-08T05:49:21.740889+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:20.999287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740940+0000","last_change":"2024-05-08T05:49:19.221630+0000","last_active":"2024-05-08T05:49:21.740940+0000","last_peered":"2024-05-08T05:49:21.740940+0000","last_clean":"2024-05-08T05:49:21.740940+0000","last_became_active":"2024-05-08T05:49:19.221299+0000","last_became_peered":"2024-05-08T05:49:19.221299+0000","last_unstale":"2024-05-08T05:49:21.740940+0000","last_undegraded":"2024-05-08T05:49:21.740940+0000","last_fullsized":"2024-05-08T05:49:21.740940+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:53:45.139823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712268+0000","last_change":"2024-05-08T05:49:20.235059+0000","last_active":"2024-05-08T05:49:21.712268+0000","last_peered":"2024-05-08T05:49:21.712268+0000","last_clean":"2024-05-08T05:49:21.712268+0000","last_became_active":"2024-05-08T05:49:20.234561+0000","last_became_peered":"2024-05-08T05:49:20.234561+0000","last_unstale":"2024-05-08T05:49:21.712268+0000","last_undegraded":"2024-05-08T05:49:21.712268+0000","last_fullsized":"2024-05-08T05:49:21.712268+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:00:16.941782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.221905+0000","last_change":"2024-05-08T05:49:19.215532+0000","last_active":"2024-05-08T05:49:21.221905+0000","last_peered":"2024-05-08T05:49:21.221905+0000","last_clean":"2024-05-08T05:49:21.221905+0000","last_became_active":"2024-05-08T05:49:19.215217+0000","last_became_peered":"2024-05-08T05:49:19.215217+0000","last_unstale":"2024-05-08T05:49:21.221905+0000","last_undegraded":"2024-05-08T05:49:21.221905+0000","last_fullsized":"2024-05-08T05:49:21.221905+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:53:45.139823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712742+0000","last_change":"2024-05-08T05:49:20.232929+0000","last_active":"2024-05-08T05:49:21.712742+0000","last_peered":"2024-05-08T05:49:21.712742+0000","last_clean":"2024-05-08T05:49:21.712742+0000","last_became_active":"2024-05-08T05:49:20.232635+0000","last_became_peered":"2024-05-08T05:49:20.232635+0000","last_unstale":"2024-05-08T05:49:21.712742+0000","last_undegraded":"2024-05-08T05:49:21.712742+0000","last_fullsized":"2024-05-08T05:49:21.712742+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:47:01.494382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712087+0000","last_change":"2024-05-08T05:49:19.222351+0000","last_active":"2024-05-08T05:49:21.712087+0000","last_peered":"2024-05-08T05:49:21.712087+0000","last_clean":"2024-05-08T05:49:21.712087+0000","last_became_active":"2024-05-08T05:49:19.222092+0000","last_became_peered":"2024-05-08T05:49:19.222092+0000","last_unstale":"2024-05-08T05:49:21.712087+0000","last_undegraded":"2024-05-08T05:49:21.712087+0000","last_fullsized":"2024-05-08T05:49:21.712087+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:22:40.427897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741491+0000","last_change":"2024-05-08T05:49:20.235238+0000","last_active":"2024-05-08T05:49:21.741491+0000","last_peered":"2024-05-08T05:49:21.741491+0000","last_clean":"2024-05-08T05:49:21.741491+0000","last_became_active":"2024-05-08T05:49:20.234933+0000","last_became_peered":"2024-05-08T05:49:20.234933+0000","last_unstale":"2024-05-08T05:49:21.741491+0000","last_undegraded":"2024-05-08T05:49:21.741491+0000","last_fullsized":"2024-05-08T05:49:21.741491+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:59:57.323293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740680+0000","last_change":"2024-05-08T05:49:19.219966+0000","last_active":"2024-05-08T05:49:21.740680+0000","last_peered":"2024-05-08T05:49:21.740680+0000","last_clean":"2024-05-08T05:49:21.740680+0000","last_became_active":"2024-05-08T05:49:19.219555+0000","last_became_peered":"2024-05-08T05:49:19.219555+0000","last_unstale":"2024-05-08T05:49:21.740680+0000","last_undegraded":"2024-05-08T05:49:21.740680+0000","last_fullsized":"2024-05-08T05:49:21.740680+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:53:45.139823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741173+0000","last_change":"2024-05-08T05:49:20.230394+0000","last_active":"2024-05-08T05:49:21.741173+0000","last_peered":"2024-05-08T05:49:21.741173+0000","last_clean":"2024-05-08T05:49:21.741173+0000","last_became_active":"2024-05-08T05:49:20.229931+0000","last_became_peered":"2024-05-08T05:49:20.229931+0000","last_unstale":"2024-05-08T05:49:21.741173+0000","last_undegraded":"2024-05-08T05:49:21.741173+0000","last_fullsized":"2024-05-08T05:49:21.741173+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:26:27.901303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712584+0000","last_change":"2024-05-08T05:49:19.224009+0000","last_active":"2024-05-08T05:49:21.712584+0000","last_peered":"2024-05-08T05:49:21.712584+0000","last_clean":"2024-05-08T05:49:21.712584+0000","last_became_active":"2024-05-08T05:49:19.223694+0000","last_became_peered":"2024-05-08T05:49:19.223694+0000","last_unstale":"2024-05-08T05:49:21.712584+0000","last_undegraded":"2024-05-08T05:49:21.712584+0000","last_fullsized":"2024-05-08T05:49:21.712584+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:36:26.679637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740331+0000","last_change":"2024-05-08T05:49:20.224133+0000","last_active":"2024-05-08T05:49:21.740331+0000","last_peered":"2024-05-08T05:49:21.740331+0000","last_clean":"2024-05-08T05:49:21.740331+0000","last_became_active":"2024-05-08T05:49:20.223808+0000","last_became_peered":"2024-05-08T05:49:20.223808+0000","last_unstale":"2024-05-08T05:49:21.740331+0000","last_undegraded":"2024-05-08T05:49:21.740331+0000","last_fullsized":"2024-05-08T05:49:21.740331+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:42.929693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712441+0000","last_change":"2024-05-08T05:49:19.227947+0000","last_active":"2024-05-08T05:49:21.712441+0000","last_peered":"2024-05-08T05:49:21.712441+0000","last_clean":"2024-05-08T05:49:21.712441+0000","last_became_active":"2024-05-08T05:49:19.227650+0000","last_became_peered":"2024-05-08T05:49:19.227650+0000","last_unstale":"2024-05-08T05:49:21.712441+0000","last_undegraded":"2024-05-08T05:49:21.712441+0000","last_fullsized":"2024-05-08T05:49:21.712441+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:07.219786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":264,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740434+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:49:21.740434+0000","last_peered":"2024-05-08T05:49:21.740434+0000","last_clean":"2024-05-08T05:49:21.740434+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:49:21.740434+0000","last_undegraded":"2024-05-08T05:49:21.740434+0000","last_fullsized":"2024-05-08T05:49:21.740434+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542245,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1888256,"data_stored":1186811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542246,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1306624,"data_stored":598499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542245,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":596283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574948,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1294336,"data_stored":596375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574947,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":596865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574947,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":596317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574947,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":596317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574947,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1896448,"data_stored":1188353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:49:25.004 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T05:49:25.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.145+0000 7f777217f640 1 Processor -- start 2024-05-08T05:49:25.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.145+0000 7f777217f640 1 -- start start 2024-05-08T05:49:25.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.146+0000 7f777217f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f776c15f420 0x7f776c0d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:25.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.146+0000 7f777217f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f776c0d2080 0x7f776c0d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:25.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.146+0000 7f777217f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f776c1630b0 con 0x7f776c0d3ca0 2024-05-08T05:49:25.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.146+0000 7f777217f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f776c0d2550 con 0x7f776c0d2080 2024-05-08T05:49:25.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.146+0000 7f777217f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f776c0d2c80 con 0x7f776c15f420 2024-05-08T05:49:25.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.146+0000 7f776bfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f776c0d2080 0x7f776c0d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:25.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.146+0000 7f776bfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f776c0d2080 0x7f776c0d4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45410/0 (socket says 172.21.15.45:45410) 2024-05-08T05:49:25.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.146+0000 7f776affd640 1 --1- 172.21.15.45:0/308992935 >> v1:172.21.15.45:6789/0 conn(0x7f776c0d3ca0 0x7f776c0d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49858/0 (socket says 172.21.15.45:49858) 2024-05-08T05:49:25.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.146+0000 7f776bfff640 1 -- 172.21.15.45:0/308992935 learned_addr learned my addr 172.21.15.45:0/308992935 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:25.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.147+0000 7f776b7fe640 1 --2- 172.21.15.45:0/308992935 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f776c15f420 0x7f776c0d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:25.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.147+0000 7f776bfff640 1 -- 172.21.15.45:0/308992935 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f776c15f420 msgr2=0x7f776c0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:25.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.147+0000 7f776bfff640 1 --2- 172.21.15.45:0/308992935 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f776c15f420 0x7f776c0d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:25.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.147+0000 7f776bfff640 1 -- 172.21.15.45:0/308992935 >> v1:172.21.15.45:6789/0 conn(0x7f776c0d3ca0 legacy=0x7f776c0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:25.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.147+0000 7f776bfff640 1 -- 172.21.15.45:0/308992935 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c0d2f60 con 0x7f776c0d2080 2024-05-08T05:49:25.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.147+0000 7f776bfff640 1 --2- 172.21.15.45:0/308992935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c0d2080 0x7f776c0d4cc0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f7764009e50 tx=0x7f776402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08cb52309608ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:25.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.148+0000 7f776a7fc640 1 -- 172.21.15.45:0/308992935 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764036070 con 0x7f776c0d2080 2024-05-08T05:49:25.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.148+0000 7f776a7fc640 1 -- 172.21.15.45:0/308992935 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7764031600 con 0x7f776c0d2080 2024-05-08T05:49:25.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.148+0000 7f776a7fc640 1 -- 172.21.15.45:0/308992935 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77640319d0 con 0x7f776c0d2080 2024-05-08T05:49:25.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.148+0000 7f777217f640 1 -- 172.21.15.45:0/308992935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c0d2080 msgr2=0x7f776c0d4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:25.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.148+0000 7f777217f640 1 --2- 172.21.15.45:0/308992935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c0d2080 0x7f776c0d4cc0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f7764009e50 tx=0x7f776402a710 comp rx=0 tx=0).stop 2024-05-08T05:49:25.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.148+0000 7f777217f640 1 -- 172.21.15.45:0/308992935 shutdown_connections 2024-05-08T05:49:25.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.148+0000 7f777217f640 1 --2- 172.21.15.45:0/308992935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c0d2080 0x7f776c0d4cc0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:25.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.148+0000 7f777217f640 1 --2- 172.21.15.45:0/308992935 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f776c15f420 0x7f776c0d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:25.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.148+0000 7f777217f640 1 -- 172.21.15.45:0/308992935 >> 172.21.15.45:0/308992935 conn(0x7f776c08eac0 msgr2=0x7f776c09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:25.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.148+0000 7f777217f640 1 -- 172.21.15.45:0/308992935 shutdown_connections 2024-05-08T05:49:25.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.148+0000 7f777217f640 1 -- 172.21.15.45:0/308992935 wait complete. 2024-05-08T05:49:25.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.149+0000 7f777217f640 1 Processor -- start 2024-05-08T05:49:25.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.150+0000 7f777217f640 1 -- start start 2024-05-08T05:49:25.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.150+0000 7f777217f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f776c0d2080 0x7f776c1f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.150+0000 7f777217f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c0cc0e0 0x7f776c0c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.150+0000 7f777217f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776c0cc560 0x7f776c0ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.150+0000 7f777217f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f776c0d5200 con 0x7f776c0cc0e0 2024-05-08T05:49:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.150+0000 7f777217f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f776c0d2830 con 0x7f776c0cc560 2024-05-08T05:49:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.150+0000 7f777217f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f776c0d5380 con 0x7f776c0d2080 2024-05-08T05:49:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.150+0000 7f776affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c0cc0e0 0x7f776c0c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.151+0000 7f776affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c0cc0e0 0x7f776c0c9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45426/0 (socket says 172.21.15.45:45426) 2024-05-08T05:49:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.151+0000 7f776affd640 1 -- 172.21.15.45:0/3019825994 learned_addr learned my addr 172.21.15.45:0/3019825994 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.151+0000 7f776b7fe640 1 --2- 172.21.15.45:0/3019825994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f776c0d2080 0x7f776c1f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.151+0000 7f776bfff640 1 --2- 172.21.15.45:0/3019825994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776c0cc560 0x7f776c0ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.151+0000 7f776affd640 1 -- 172.21.15.45:0/3019825994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f776c0d2080 msgr2=0x7f776c1f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.151+0000 7f776affd640 1 --2- 172.21.15.45:0/3019825994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f776c0d2080 0x7f776c1f4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.151+0000 7f776affd640 1 -- 172.21.15.45:0/3019825994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776c0cc560 msgr2=0x7f776c0ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.151+0000 7f776affd640 1 --2- 172.21.15.45:0/3019825994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776c0cc560 0x7f776c0ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.151+0000 7f776affd640 1 -- 172.21.15.45:0/3019825994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c15fc40 con 0x7f776c0cc0e0 2024-05-08T05:49:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.151+0000 7f776affd640 1 --2- 172.21.15.45:0/3019825994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c0cc0e0 0x7f776c0c9b50 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f776000c500 tx=0x7f776000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.151+0000 7f7753fff640 1 -- 172.21.15.45:0/3019825994 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760018070 con 0x7f776c0cc0e0 2024-05-08T05:49:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.151+0000 7f7753fff640 1 -- 172.21.15.45:0/3019825994 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7760007500 con 0x7f776c0cc0e0 2024-05-08T05:49:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.151+0000 7f777217f640 1 -- 172.21.15.45:0/3019825994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776c161bb0 con 0x7f776c0cc0e0 2024-05-08T05:49:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.151+0000 7f7753fff640 1 -- 172.21.15.45:0/3019825994 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77600049e0 con 0x7f776c0cc0e0 2024-05-08T05:49:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.151+0000 7f777217f640 1 -- 172.21.15.45:0/3019825994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f776c0ca9a0 con 0x7f776c0cc0e0 2024-05-08T05:49:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.152+0000 7f777217f640 1 -- 172.21.15.45:0/3019825994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f776c0693f0 con 0x7f776c0cc0e0 2024-05-08T05:49:25.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.153+0000 7f7753fff640 1 -- 172.21.15.45:0/3019825994 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f776001b070 con 0x7f776c0cc0e0 2024-05-08T05:49:25.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.153+0000 7f7753fff640 1 --2- 172.21.15.45:0/3019825994 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f77580762d0 0x7f7758078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:25.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.154+0000 7f7753fff640 1 -- 172.21.15.45:0/3019825994 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f7760096a50 con 0x7f776c0cc0e0 2024-05-08T05:49:25.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.154+0000 7f776b7fe640 1 --2- 172.21.15.45:0/3019825994 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f77580762d0 0x7f7758078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:25.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.154+0000 7f776b7fe640 1 --2- 172.21.15.45:0/3019825994 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f77580762d0 0x7f7758078790 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f775c0079a0 tx=0x7f775c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:25.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.158+0000 7f7753fff640 1 -- 172.21.15.45:0/3019825994 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f776c1630b0 con 0x7f776c0cc0e0 2024-05-08T05:49:25.186 INFO:tasks.ceph.mds.a.smithi045.stderr:2024-05-08T05:49:25.187+0000 7fdc0f8e4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:49:25.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.426+0000 7f777217f640 1 -- 172.21.15.45:0/3019825994 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f776c0b4c50 con 0x7f776c0cc0e0 2024-05-08T05:49:25.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.426+0000 7f7753fff640 1 -- 172.21.15.45:0/3019825994 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1305 (secure 0 0 0) 0x7f776005fc60 con 0x7f776c0cc0e0 2024-05-08T05:49:25.427 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:25.427 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":497,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1715146878,"num_in_osds":8,"osd_in_since":1715146870,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":266981376,"bytes_avail":772827131904,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":83,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7524}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-05-08T05:48:42.938675+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T05:49:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.428+0000 7f777217f640 1 -- 172.21.15.45:0/3019825994 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f77580762d0 msgr2=0x7f7758078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.428+0000 7f777217f640 1 --2- 172.21.15.45:0/3019825994 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f77580762d0 0x7f7758078790 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f775c0079a0 tx=0x7f775c008040 comp rx=0 tx=0).stop 2024-05-08T05:49:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.428+0000 7f777217f640 1 -- 172.21.15.45:0/3019825994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c0cc0e0 msgr2=0x7f776c0c9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.428+0000 7f777217f640 1 --2- 172.21.15.45:0/3019825994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c0cc0e0 0x7f776c0c9b50 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f776000c500 tx=0x7f776000c9c0 comp rx=0 tx=0).stop 2024-05-08T05:49:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.428+0000 7f777217f640 1 -- 172.21.15.45:0/3019825994 shutdown_connections 2024-05-08T05:49:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.428+0000 7f777217f640 1 --2- 172.21.15.45:0/3019825994 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f77580762d0 0x7f7758078790 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.428+0000 7f777217f640 1 --2- 172.21.15.45:0/3019825994 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f776c0cc560 0x7f776c0ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.428+0000 7f777217f640 1 --2- 172.21.15.45:0/3019825994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f776c0cc0e0 0x7f776c0c9b50 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.428+0000 7f777217f640 1 --2- 172.21.15.45:0/3019825994 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f776c0d2080 0x7f776c1f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.428+0000 7f777217f640 1 -- 172.21.15.45:0/3019825994 >> 172.21.15.45:0/3019825994 conn(0x7f776c08eac0 msgr2=0x7f776c0b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.428+0000 7f777217f640 1 -- 172.21.15.45:0/3019825994 shutdown_connections 2024-05-08T05:49:25.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.428+0000 7f777217f640 1 -- 172.21.15.45:0/3019825994 wait complete. 2024-05-08T05:49:25.440 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-08T05:49:25.440 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:49:25.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.543+0000 7f82eff9d640 1 Processor -- start 2024-05-08T05:49:25.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.543+0000 7f82eff9d640 1 -- start start 2024-05-08T05:49:25.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.544+0000 7f82eff9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f82e815c340 0x7f82e8166970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:25.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.544+0000 7f82eff9d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e815c920 con 0x7f82e8072840 2024-05-08T05:49:25.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.544+0000 7f82eff9d640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e815cc00 con 0x7f82e806e0f0 2024-05-08T05:49:25.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.544+0000 7f82eff9d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f82e815cee0 con 0x7f82e815c340 2024-05-08T05:49:25.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.544+0000 7f82ed538640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f82e8072840 0x7f82e806e4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49874/0 (socket says 172.21.15.45:49874) 2024-05-08T05:49:25.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.544+0000 7f82ed538640 1 -- 172.21.15.45:0/924317216 learned_addr learned my addr 172.21.15.45:0/924317216 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:25.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82ecd37640 1 -- 172.21.15.45:0/924317216 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 396422744 0 0) 0x7f82e815c920 con 0x7f82e8072840 2024-05-08T05:49:25.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82ecd37640 1 -- 172.21.15.45:0/924317216 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82d4003520 con 0x7f82e8072840 2024-05-08T05:49:25.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82ee53a640 1 --2- 172.21.15.45:0/924317216 >> v2:172.21.15.63:3300/0 conn(0x7f82e815c340 0x7f82e8166970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:25.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82ecd37640 1 -- 172.21.15.45:0/924317216 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 341739454 0 0) 0x7f82d4003520 con 0x7f82e8072840 2024-05-08T05:49:25.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82ecd37640 1 -- 172.21.15.45:0/924317216 >> v2:172.21.15.63:3300/0 conn(0x7f82e815c340 msgr2=0x7f82e8166970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:25.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82ecd37640 1 --2- 172.21.15.45:0/924317216 >> v2:172.21.15.63:3300/0 conn(0x7f82e815c340 0x7f82e8166970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:25.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82ecd37640 1 -- 172.21.15.45:0/924317216 >> v1:172.21.15.63:6789/0 conn(0x7f82e806e0f0 legacy=0x7f82e80d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:25.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82ecd37640 1 -- 172.21.15.45:0/924317216 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e815d1c0 con 0x7f82e8072840 2024-05-08T05:49:25.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82ecd37640 1 -- 172.21.15.45:0/924317216 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f82d80016f0 con 0x7f82e8072840 2024-05-08T05:49:25.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82ecd37640 1 -- 172.21.15.45:0/924317216 >> v1:172.21.15.45:6789/0 conn(0x7f82e8072840 legacy=0x7f82e806e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:25.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82ecd37640 1 --2- 172.21.15.45:0/924317216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82d4003a50 0x7f82d4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:25.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82ecd37640 1 --2- 172.21.15.45:0/924317216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82d4004800 0x7f82d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:25.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82ecd37640 1 --2- 172.21.15.45:0/924317216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82d40052e0 0x7f82d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:25.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82ecd37640 1 -- 172.21.15.45:0/924317216 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82e815cee0 con 0x7f82d4003a50 2024-05-08T05:49:25.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82ecd37640 1 -- 172.21.15.45:0/924317216 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82d400c1b0 con 0x7f82d4004800 2024-05-08T05:49:25.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82ecd37640 1 -- 172.21.15.45:0/924317216 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f82d400c3b0 con 0x7f82d40052e0 2024-05-08T05:49:25.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82edd39640 1 --2- 172.21.15.45:0/924317216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82d4003a50 0x7f82d4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:25.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82ee53a640 1 --2- 172.21.15.45:0/924317216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82d40052e0 0x7f82d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:25.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82ed538640 1 --2- 172.21.15.45:0/924317216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82d4004800 0x7f82d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:25.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82edd39640 1 -- 172.21.15.45:0/924317216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82d40052e0 msgr2=0x7f82d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:25.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82edd39640 1 --2- 172.21.15.45:0/924317216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82d40052e0 0x7f82d400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:25.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82edd39640 1 -- 172.21.15.45:0/924317216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82d4004800 msgr2=0x7f82d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:25.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82edd39640 1 --2- 172.21.15.45:0/924317216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82d4004800 0x7f82d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:25.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82edd39640 1 -- 172.21.15.45:0/924317216 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d4003520 con 0x7f82d4003a50 2024-05-08T05:49:25.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.545+0000 7f82ee53a640 1 -- 172.21.15.45:0/924317216 reap_dead start 2024-05-08T05:49:25.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.546+0000 7f82edd39640 1 --2- 172.21.15.45:0/924317216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82d4003a50 0x7f82d4003e70 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f82e4001760 tx=0x7f82d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d7fdb37f44db7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:25.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.546+0000 7f82ecd37640 1 -- 172.21.15.45:0/924317216 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e40013b0 con 0x7f82d4003a50 2024-05-08T05:49:25.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.546+0000 7f82ecd37640 1 -- 172.21.15.45:0/924317216 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82e40052d0 con 0x7f82d4003a50 2024-05-08T05:49:25.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.546+0000 7f82ecd37640 1 -- 172.21.15.45:0/924317216 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e402aa40 con 0x7f82d4003a50 2024-05-08T05:49:25.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.547+0000 7f82eff9d640 1 -- 172.21.15.45:0/924317216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82d4003a50 msgr2=0x7f82d4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:25.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.547+0000 7f82eff9d640 1 --2- 172.21.15.45:0/924317216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82d4003a50 0x7f82d4003e70 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f82e4001760 tx=0x7f82d4005830 comp rx=0 tx=0).stop 2024-05-08T05:49:25.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.547+0000 7f82eff9d640 1 -- 172.21.15.45:0/924317216 shutdown_connections 2024-05-08T05:49:25.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.547+0000 7f82eff9d640 1 --2- 172.21.15.45:0/924317216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82d4003a50 0x7f82d4003e70 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:25.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.547+0000 7f82eff9d640 1 -- 172.21.15.45:0/924317216 >> 172.21.15.45:0/924317216 conn(0x7f82e806c3a0 msgr2=0x7f82e806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:25.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.547+0000 7f82eff9d640 1 -- 172.21.15.45:0/924317216 shutdown_connections 2024-05-08T05:49:25.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.547+0000 7f82eff9d640 1 -- 172.21.15.45:0/924317216 wait complete. 2024-05-08T05:49:25.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.548+0000 7f82eff9d640 1 Processor -- start 2024-05-08T05:49:25.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.548+0000 7f82eff9d640 1 -- start start 2024-05-08T05:49:25.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.549+0000 7f82eff9d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82d4003a50 0x7f82e815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:25.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.549+0000 7f82eff9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82e809e370 0x7f82e81e8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:25.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.549+0000 7f82eff9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e8143200 0x7f82e8143680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:25.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.549+0000 7f82eff9d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f82e8153230 con 0x7f82e8143200 2024-05-08T05:49:25.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.549+0000 7f82eff9d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f82e81530b0 con 0x7f82e809e370 2024-05-08T05:49:25.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.549+0000 7f82eff9d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f82e80d0f70 con 0x7f82d4003a50 2024-05-08T05:49:25.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.549+0000 7f82ee53a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e8143200 0x7f82e8143680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:25.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.549+0000 7f82ee53a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e8143200 0x7f82e8143680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45440/0 (socket says 172.21.15.45:45440) 2024-05-08T05:49:25.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.549+0000 7f82ee53a640 1 -- 172.21.15.45:0/633261743 learned_addr learned my addr 172.21.15.45:0/633261743 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:25.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.549+0000 7f82edd39640 1 --2- 172.21.15.45:0/633261743 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82d4003a50 0x7f82e815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:25.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.549+0000 7f82ed538640 1 --2- 172.21.15.45:0/633261743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82e809e370 0x7f82e81e8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:25.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.549+0000 7f82ee53a640 1 -- 172.21.15.45:0/633261743 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82d4003a50 msgr2=0x7f82e815d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:25.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.549+0000 7f82ee53a640 1 --2- 172.21.15.45:0/633261743 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82d4003a50 0x7f82e815d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:25.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.549+0000 7f82ee53a640 1 -- 172.21.15.45:0/633261743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82e809e370 msgr2=0x7f82e81e8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:25.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.549+0000 7f82ee53a640 1 --2- 172.21.15.45:0/633261743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82e809e370 0x7f82e81e8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:25.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.549+0000 7f82ee53a640 1 -- 172.21.15.45:0/633261743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e815dad0 con 0x7f82e8143200 2024-05-08T05:49:25.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.550+0000 7f82ee53a640 1 --2- 172.21.15.45:0/633261743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e8143200 0x7f82e8143680 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f82e80b01d0 tx=0x7f82e000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:25.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.550+0000 7f82de7fc640 1 -- 172.21.15.45:0/633261743 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e000fe10 con 0x7f82e8143200 2024-05-08T05:49:25.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.550+0000 7f82de7fc640 1 -- 172.21.15.45:0/633261743 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82e001d070 con 0x7f82e8143200 2024-05-08T05:49:25.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.550+0000 7f82de7fc640 1 -- 172.21.15.45:0/633261743 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0015670 con 0x7f82e8143200 2024-05-08T05:49:25.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.550+0000 7f82eff9d640 1 -- 172.21.15.45:0/633261743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e815ddb0 con 0x7f82e8143200 2024-05-08T05:49:25.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.550+0000 7f82eff9d640 1 -- 172.21.15.45:0/633261743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e815e370 con 0x7f82e8143200 2024-05-08T05:49:25.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.551+0000 7f82eff9d640 1 -- 172.21.15.45:0/633261743 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82e80693f0 con 0x7f82e8143200 2024-05-08T05:49:25.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.552+0000 7f82de7fc640 1 -- 172.21.15.45:0/633261743 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f82e0015810 con 0x7f82e8143200 2024-05-08T05:49:25.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.552+0000 7f82de7fc640 1 --2- 172.21.15.45:0/633261743 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f82cc076300 0x7f82cc0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:25.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.552+0000 7f82de7fc640 1 -- 172.21.15.45:0/633261743 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f82e00981d0 con 0x7f82e8143200 2024-05-08T05:49:25.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.552+0000 7f82edd39640 1 --2- 172.21.15.45:0/633261743 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f82cc076300 0x7f82cc0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:25.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.553+0000 7f82edd39640 1 --2- 172.21.15.45:0/633261743 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f82cc076300 0x7f82cc0787c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f82e4032000 tx=0x7f82e402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:25.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.556+0000 7f82de7fc640 1 -- 172.21.15.45:0/633261743 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f82e0010040 con 0x7f82e8143200 2024-05-08T05:49:25.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.743+0000 7f82eff9d640 1 -- 172.21.15.45:0/633261743 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f82e80765b0 con 0x7f82cc076300 2024-05-08T05:49:25.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.750+0000 7f82de7fc640 1 -- 172.21.15.45:0/633261743 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252475 (secure 0 0 0) 0x7f82e80765b0 con 0x7f82cc076300 2024-05-08T05:49:25.752 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:25.754 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:49:25.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.753+0000 7f82eff9d640 1 -- 172.21.15.45:0/633261743 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f82cc076300 msgr2=0x7f82cc0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:25.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.753+0000 7f82eff9d640 1 --2- 172.21.15.45:0/633261743 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f82cc076300 0x7f82cc0787c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f82e4032000 tx=0x7f82e402e000 comp rx=0 tx=0).stop 2024-05-08T05:49:25.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.753+0000 7f82eff9d640 1 -- 172.21.15.45:0/633261743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e8143200 msgr2=0x7f82e8143680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:25.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.753+0000 7f82eff9d640 1 --2- 172.21.15.45:0/633261743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e8143200 0x7f82e8143680 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f82e80b01d0 tx=0x7f82e000f580 comp rx=0 tx=0).stop 2024-05-08T05:49:25.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.753+0000 7f82eff9d640 1 -- 172.21.15.45:0/633261743 shutdown_connections 2024-05-08T05:49:25.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.753+0000 7f82eff9d640 1 --2- 172.21.15.45:0/633261743 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f82cc076300 0x7f82cc0787c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:25.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.753+0000 7f82eff9d640 1 --2- 172.21.15.45:0/633261743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f82e8143200 0x7f82e8143680 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:25.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.753+0000 7f82eff9d640 1 --2- 172.21.15.45:0/633261743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f82e809e370 0x7f82e81e8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:25.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.753+0000 7f82eff9d640 1 --2- 172.21.15.45:0/633261743 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f82d4003a50 0x7f82e815d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:25.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.753+0000 7f82eff9d640 1 -- 172.21.15.45:0/633261743 >> 172.21.15.45:0/633261743 conn(0x7f82e806c3a0 msgr2=0x7f82e806e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:25.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.754+0000 7f82eff9d640 1 -- 172.21.15.45:0/633261743 shutdown_connections 2024-05-08T05:49:25.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:25.754+0000 7f82eff9d640 1 -- 172.21.15.45:0/633261743 wait complete. 2024-05-08T05:49:25.765 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-05-08T05:49:24.948475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260724,"kb_used_data":11272,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713996,"statfs":{"total":773094113280,"available":772827131904,"internally_reserved":0,"allocated":11542528,"data_stored":5955820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001390"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220540+0000","last_change":"2024-05-08T05:49:20.238531+0000","last_active":"2024-05-08T05:49:21.220540+0000","last_peered":"2024-05-08T05:49:21.220540+0000","last_clean":"2024-05-08T05:49:21.220540+0000","last_became_active":"2024-05-08T05:49:20.238249+0000","last_became_peered":"2024-05-08T05:49:20.238249+0000","last_unstale":"2024-05-08T05:49:21.220540+0000","last_undegraded":"2024-05-08T05:49:21.220540+0000","last_fullsized":"2024-05-08T05:49:21.220540+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:00:42.502385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741649+0000","last_change":"2024-05-08T05:49:20.230490+0000","last_active":"2024-05-08T05:49:21.741649+0000","last_peered":"2024-05-08T05:49:21.741649+0000","last_clean":"2024-05-08T05:49:21.741649+0000","last_became_active":"2024-05-08T05:49:20.230163+0000","last_became_peered":"2024-05-08T05:49:20.230163+0000","last_unstale":"2024-05-08T05:49:21.741649+0000","last_undegraded":"2024-05-08T05:49:21.741649+0000","last_fullsized":"2024-05-08T05:49:21.741649+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:01.788691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220788+0000","last_change":"2024-05-08T05:49:20.228822+0000","last_active":"2024-05-08T05:49:21.220788+0000","last_peered":"2024-05-08T05:49:21.220788+0000","last_clean":"2024-05-08T05:49:21.220788+0000","last_became_active":"2024-05-08T05:49:20.228543+0000","last_became_peered":"2024-05-08T05:49:20.228543+0000","last_unstale":"2024-05-08T05:49:21.220788+0000","last_undegraded":"2024-05-08T05:49:21.220788+0000","last_fullsized":"2024-05-08T05:49:21.220788+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:15.407492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740921+0000","last_change":"2024-05-08T05:49:20.227003+0000","last_active":"2024-05-08T05:49:21.740921+0000","last_peered":"2024-05-08T05:49:21.740921+0000","last_clean":"2024-05-08T05:49:21.740921+0000","last_became_active":"2024-05-08T05:49:20.226673+0000","last_became_peered":"2024-05-08T05:49:20.226673+0000","last_unstale":"2024-05-08T05:49:21.740921+0000","last_undegraded":"2024-05-08T05:49:21.740921+0000","last_fullsized":"2024-05-08T05:49:21.740921+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:58:10.209690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740362+0000","last_change":"2024-05-08T05:49:20.228592+0000","last_active":"2024-05-08T05:49:21.740362+0000","last_peered":"2024-05-08T05:49:21.740362+0000","last_clean":"2024-05-08T05:49:21.740362+0000","last_became_active":"2024-05-08T05:49:20.228303+0000","last_became_peered":"2024-05-08T05:49:20.228303+0000","last_unstale":"2024-05-08T05:49:21.740362+0000","last_undegraded":"2024-05-08T05:49:21.740362+0000","last_fullsized":"2024-05-08T05:49:21.740362+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:42.929693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740876+0000","last_change":"2024-05-08T05:49:20.226897+0000","last_active":"2024-05-08T05:49:21.740876+0000","last_peered":"2024-05-08T05:49:21.740876+0000","last_clean":"2024-05-08T05:49:21.740876+0000","last_became_active":"2024-05-08T05:49:20.226591+0000","last_became_peered":"2024-05-08T05:49:20.226591+0000","last_unstale":"2024-05-08T05:49:21.740876+0000","last_undegraded":"2024-05-08T05:49:21.740876+0000","last_fullsized":"2024-05-08T05:49:21.740876+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:22.685457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740345+0000","last_change":"2024-05-08T05:49:20.229697+0000","last_active":"2024-05-08T05:49:21.740345+0000","last_peered":"2024-05-08T05:49:21.740345+0000","last_clean":"2024-05-08T05:49:21.740345+0000","last_became_active":"2024-05-08T05:49:20.229413+0000","last_became_peered":"2024-05-08T05:49:20.229413+0000","last_unstale":"2024-05-08T05:49:21.740345+0000","last_undegraded":"2024-05-08T05:49:21.740345+0000","last_fullsized":"2024-05-08T05:49:21.740345+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:45.603541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712046+0000","last_change":"2024-05-08T05:49:20.231788+0000","last_active":"2024-05-08T05:49:21.712046+0000","last_peered":"2024-05-08T05:49:21.712046+0000","last_clean":"2024-05-08T05:49:21.712046+0000","last_became_active":"2024-05-08T05:49:20.231323+0000","last_became_peered":"2024-05-08T05:49:20.231323+0000","last_unstale":"2024-05-08T05:49:21.712046+0000","last_undegraded":"2024-05-08T05:49:21.712046+0000","last_fullsized":"2024-05-08T05:49:21.712046+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:40:06.613670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741679+0000","last_change":"2024-05-08T05:49:20.230080+0000","last_active":"2024-05-08T05:49:21.741679+0000","last_peered":"2024-05-08T05:49:21.741679+0000","last_clean":"2024-05-08T05:49:21.741679+0000","last_became_active":"2024-05-08T05:49:20.228395+0000","last_became_peered":"2024-05-08T05:49:20.228395+0000","last_unstale":"2024-05-08T05:49:21.741679+0000","last_undegraded":"2024-05-08T05:49:21.741679+0000","last_fullsized":"2024-05-08T05:49:21.741679+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:00:59.034430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740259+0000","last_change":"2024-05-08T05:49:20.235915+0000","last_active":"2024-05-08T05:49:21.740259+0000","last_peered":"2024-05-08T05:49:21.740259+0000","last_clean":"2024-05-08T05:49:21.740259+0000","last_became_active":"2024-05-08T05:49:20.235612+0000","last_became_peered":"2024-05-08T05:49:20.235612+0000","last_unstale":"2024-05-08T05:49:21.740259+0000","last_undegraded":"2024-05-08T05:49:21.740259+0000","last_fullsized":"2024-05-08T05:49:21.740259+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:33.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740474+0000","last_change":"2024-05-08T05:49:20.236606+0000","last_active":"2024-05-08T05:49:21.740474+0000","last_peered":"2024-05-08T05:49:21.740474+0000","last_clean":"2024-05-08T05:49:21.740474+0000","last_became_active":"2024-05-08T05:49:20.236336+0000","last_became_peered":"2024-05-08T05:49:20.236336+0000","last_unstale":"2024-05-08T05:49:21.740474+0000","last_undegraded":"2024-05-08T05:49:21.740474+0000","last_fullsized":"2024-05-08T05:49:21.740474+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:58.466547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222198+0000","last_change":"2024-05-08T05:49:20.225246+0000","last_active":"2024-05-08T05:49:21.222198+0000","last_peered":"2024-05-08T05:49:21.222198+0000","last_clean":"2024-05-08T05:49:21.222198+0000","last_became_active":"2024-05-08T05:49:20.224935+0000","last_became_peered":"2024-05-08T05:49:20.224935+0000","last_unstale":"2024-05-08T05:49:21.222198+0000","last_undegraded":"2024-05-08T05:49:21.222198+0000","last_fullsized":"2024-05-08T05:49:21.222198+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:45.603541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740114+0000","last_change":"2024-05-08T05:49:20.233325+0000","last_active":"2024-05-08T05:49:21.740114+0000","last_peered":"2024-05-08T05:49:21.740114+0000","last_clean":"2024-05-08T05:49:21.740114+0000","last_became_active":"2024-05-08T05:49:20.233018+0000","last_became_peered":"2024-05-08T05:49:20.233018+0000","last_unstale":"2024-05-08T05:49:21.740114+0000","last_undegraded":"2024-05-08T05:49:21.740114+0000","last_fullsized":"2024-05-08T05:49:21.740114+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:43:11.551671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740998+0000","last_change":"2024-05-08T05:49:20.224796+0000","last_active":"2024-05-08T05:49:21.740998+0000","last_peered":"2024-05-08T05:49:21.740998+0000","last_clean":"2024-05-08T05:49:21.740998+0000","last_became_active":"2024-05-08T05:49:20.224480+0000","last_became_peered":"2024-05-08T05:49:20.224480+0000","last_unstale":"2024-05-08T05:49:21.740998+0000","last_undegraded":"2024-05-08T05:49:21.740998+0000","last_fullsized":"2024-05-08T05:49:21.740998+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:58:10.209690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740735+0000","last_change":"2024-05-08T05:49:20.229339+0000","last_active":"2024-05-08T05:49:21.740735+0000","last_peered":"2024-05-08T05:49:21.740735+0000","last_clean":"2024-05-08T05:49:21.740735+0000","last_became_active":"2024-05-08T05:49:20.229066+0000","last_became_peered":"2024-05-08T05:49:20.229066+0000","last_unstale":"2024-05-08T05:49:21.740735+0000","last_undegraded":"2024-05-08T05:49:21.740735+0000","last_fullsized":"2024-05-08T05:49:21.740735+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:54.490928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740316+0000","last_change":"2024-05-08T05:49:20.228402+0000","last_active":"2024-05-08T05:49:21.740316+0000","last_peered":"2024-05-08T05:49:21.740316+0000","last_clean":"2024-05-08T05:49:21.740316+0000","last_became_active":"2024-05-08T05:49:20.228132+0000","last_became_peered":"2024-05-08T05:49:20.228132+0000","last_unstale":"2024-05-08T05:49:21.740316+0000","last_undegraded":"2024-05-08T05:49:21.740316+0000","last_fullsized":"2024-05-08T05:49:21.740316+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:55:30.372059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220628+0000","last_change":"2024-05-08T05:49:20.228209+0000","last_active":"2024-05-08T05:49:21.220628+0000","last_peered":"2024-05-08T05:49:21.220628+0000","last_clean":"2024-05-08T05:49:21.220628+0000","last_became_active":"2024-05-08T05:49:20.227869+0000","last_became_peered":"2024-05-08T05:49:20.227869+0000","last_unstale":"2024-05-08T05:49:21.220628+0000","last_undegraded":"2024-05-08T05:49:21.220628+0000","last_fullsized":"2024-05-08T05:49:21.220628+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:24.394443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741707+0000","last_change":"2024-05-08T05:49:20.218731+0000","last_active":"2024-05-08T05:49:21.741707+0000","last_peered":"2024-05-08T05:49:21.741707+0000","last_clean":"2024-05-08T05:49:21.741707+0000","last_became_active":"2024-05-08T05:49:20.218388+0000","last_became_peered":"2024-05-08T05:49:20.218388+0000","last_unstale":"2024-05-08T05:49:21.741707+0000","last_undegraded":"2024-05-08T05:49:21.741707+0000","last_fullsized":"2024-05-08T05:49:21.741707+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:00:44.400710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220462+0000","last_change":"2024-05-08T05:49:20.241520+0000","last_active":"2024-05-08T05:49:21.220462+0000","last_peered":"2024-05-08T05:49:21.220462+0000","last_clean":"2024-05-08T05:49:21.220462+0000","last_became_active":"2024-05-08T05:49:20.241194+0000","last_became_peered":"2024-05-08T05:49:20.241194+0000","last_unstale":"2024-05-08T05:49:21.220462+0000","last_undegraded":"2024-05-08T05:49:21.220462+0000","last_fullsized":"2024-05-08T05:49:21.220462+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:44:17.867536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222371+0000","last_change":"2024-05-08T05:49:20.222796+0000","last_active":"2024-05-08T05:49:21.222371+0000","last_peered":"2024-05-08T05:49:21.222371+0000","last_clean":"2024-05-08T05:49:21.222371+0000","last_became_active":"2024-05-08T05:49:20.222415+0000","last_became_peered":"2024-05-08T05:49:20.222415+0000","last_unstale":"2024-05-08T05:49:21.222371+0000","last_undegraded":"2024-05-08T05:49:21.222371+0000","last_fullsized":"2024-05-08T05:49:21.222371+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:20.999287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712297+0000","last_change":"2024-05-08T05:49:20.236618+0000","last_active":"2024-05-08T05:49:21.712297+0000","last_peered":"2024-05-08T05:49:21.712297+0000","last_clean":"2024-05-08T05:49:21.712297+0000","last_became_active":"2024-05-08T05:49:20.236323+0000","last_became_peered":"2024-05-08T05:49:20.236323+0000","last_unstale":"2024-05-08T05:49:21.712297+0000","last_undegraded":"2024-05-08T05:49:21.712297+0000","last_fullsized":"2024-05-08T05:49:21.712297+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:43:28.803611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740244+0000","last_change":"2024-05-08T05:49:20.228986+0000","last_active":"2024-05-08T05:49:21.740244+0000","last_peered":"2024-05-08T05:49:21.740244+0000","last_clean":"2024-05-08T05:49:21.740244+0000","last_became_active":"2024-05-08T05:49:20.228723+0000","last_became_peered":"2024-05-08T05:49:20.228723+0000","last_unstale":"2024-05-08T05:49:21.740244+0000","last_undegraded":"2024-05-08T05:49:21.740244+0000","last_fullsized":"2024-05-08T05:49:21.740244+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:17:48.859578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220270+0000","last_change":"2024-05-08T05:49:20.225628+0000","last_active":"2024-05-08T05:49:21.220270+0000","last_peered":"2024-05-08T05:49:21.220270+0000","last_clean":"2024-05-08T05:49:21.220270+0000","last_became_active":"2024-05-08T05:49:20.225065+0000","last_became_peered":"2024-05-08T05:49:20.225065+0000","last_unstale":"2024-05-08T05:49:21.220270+0000","last_undegraded":"2024-05-08T05:49:21.220270+0000","last_fullsized":"2024-05-08T05:49:21.220270+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:00:44.636681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740709+0000","last_change":"2024-05-08T05:49:20.239478+0000","last_active":"2024-05-08T05:49:21.740709+0000","last_peered":"2024-05-08T05:49:21.740709+0000","last_clean":"2024-05-08T05:49:21.740709+0000","last_became_active":"2024-05-08T05:49:20.239214+0000","last_became_peered":"2024-05-08T05:49:20.239214+0000","last_unstale":"2024-05-08T05:49:21.740709+0000","last_undegraded":"2024-05-08T05:49:21.740709+0000","last_fullsized":"2024-05-08T05:49:21.740709+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:45.603541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222394+0000","last_change":"2024-05-08T05:49:20.229377+0000","last_active":"2024-05-08T05:49:21.222394+0000","last_peered":"2024-05-08T05:49:21.222394+0000","last_clean":"2024-05-08T05:49:21.222394+0000","last_became_active":"2024-05-08T05:49:20.229049+0000","last_became_peered":"2024-05-08T05:49:20.229049+0000","last_unstale":"2024-05-08T05:49:21.222394+0000","last_undegraded":"2024-05-08T05:49:21.222394+0000","last_fullsized":"2024-05-08T05:49:21.222394+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:26:27.901303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222386+0000","last_change":"2024-05-08T05:49:20.231741+0000","last_active":"2024-05-08T05:49:21.222386+0000","last_peered":"2024-05-08T05:49:21.222386+0000","last_clean":"2024-05-08T05:49:21.222386+0000","last_became_active":"2024-05-08T05:49:20.231459+0000","last_became_peered":"2024-05-08T05:49:20.231459+0000","last_unstale":"2024-05-08T05:49:21.222386+0000","last_undegraded":"2024-05-08T05:49:21.222386+0000","last_fullsized":"2024-05-08T05:49:21.222386+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:22.685457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740538+0000","last_change":"2024-05-08T05:49:20.228283+0000","last_active":"2024-05-08T05:49:21.740538+0000","last_peered":"2024-05-08T05:49:21.740538+0000","last_clean":"2024-05-08T05:49:21.740538+0000","last_became_active":"2024-05-08T05:49:20.227936+0000","last_became_peered":"2024-05-08T05:49:20.227936+0000","last_unstale":"2024-05-08T05:49:21.740538+0000","last_undegraded":"2024-05-08T05:49:21.740538+0000","last_fullsized":"2024-05-08T05:49:21.740538+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:45.720876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741377+0000","last_change":"2024-05-08T05:49:20.231471+0000","last_active":"2024-05-08T05:49:21.741377+0000","last_peered":"2024-05-08T05:49:21.741377+0000","last_clean":"2024-05-08T05:49:21.741377+0000","last_became_active":"2024-05-08T05:49:20.230910+0000","last_became_peered":"2024-05-08T05:49:20.230910+0000","last_unstale":"2024-05-08T05:49:21.741377+0000","last_undegraded":"2024-05-08T05:49:21.741377+0000","last_fullsized":"2024-05-08T05:49:21.741377+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:02:05.771334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712332+0000","last_change":"2024-05-08T05:49:20.230069+0000","last_active":"2024-05-08T05:49:21.712332+0000","last_peered":"2024-05-08T05:49:21.712332+0000","last_clean":"2024-05-08T05:49:21.712332+0000","last_became_active":"2024-05-08T05:49:20.229773+0000","last_became_peered":"2024-05-08T05:49:20.229773+0000","last_unstale":"2024-05-08T05:49:21.712332+0000","last_undegraded":"2024-05-08T05:49:21.712332+0000","last_fullsized":"2024-05-08T05:49:21.712332+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:52:31.166852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.221696+0000","last_change":"2024-05-08T05:49:20.228744+0000","last_active":"2024-05-08T05:49:21.221696+0000","last_peered":"2024-05-08T05:49:21.221696+0000","last_clean":"2024-05-08T05:49:21.221696+0000","last_became_active":"2024-05-08T05:49:20.228426+0000","last_became_peered":"2024-05-08T05:49:20.228426+0000","last_unstale":"2024-05-08T05:49:21.221696+0000","last_undegraded":"2024-05-08T05:49:21.221696+0000","last_fullsized":"2024-05-08T05:49:21.221696+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:58.466547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740518+0000","last_change":"2024-05-08T05:49:20.239053+0000","last_active":"2024-05-08T05:49:21.740518+0000","last_peered":"2024-05-08T05:49:21.740518+0000","last_clean":"2024-05-08T05:49:21.740518+0000","last_became_active":"2024-05-08T05:49:20.238726+0000","last_became_peered":"2024-05-08T05:49:20.238726+0000","last_unstale":"2024-05-08T05:49:21.740518+0000","last_undegraded":"2024-05-08T05:49:21.740518+0000","last_fullsized":"2024-05-08T05:49:21.740518+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:20.999287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712233+0000","last_change":"2024-05-08T05:49:20.235736+0000","last_active":"2024-05-08T05:49:21.712233+0000","last_peered":"2024-05-08T05:49:21.712233+0000","last_clean":"2024-05-08T05:49:21.712233+0000","last_became_active":"2024-05-08T05:49:20.235467+0000","last_became_peered":"2024-05-08T05:49:20.235467+0000","last_unstale":"2024-05-08T05:49:21.712233+0000","last_undegraded":"2024-05-08T05:49:21.712233+0000","last_fullsized":"2024-05-08T05:49:21.712233+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:04:11.748952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220519+0000","last_change":"2024-05-08T05:49:20.226786+0000","last_active":"2024-05-08T05:49:21.220519+0000","last_peered":"2024-05-08T05:49:21.220519+0000","last_clean":"2024-05-08T05:49:21.220519+0000","last_became_active":"2024-05-08T05:49:20.223580+0000","last_became_peered":"2024-05-08T05:49:20.223580+0000","last_unstale":"2024-05-08T05:49:21.220519+0000","last_undegraded":"2024-05-08T05:49:21.220519+0000","last_fullsized":"2024-05-08T05:49:21.220519+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:32:33.705132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"103'4","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740566+0000","last_change":"2024-05-08T05:49:19.226689+0000","last_active":"2024-05-08T05:49:21.740566+0000","last_peered":"2024-05-08T05:49:21.740566+0000","last_clean":"2024-05-08T05:49:21.740566+0000","last_became_active":"2024-05-08T05:49:19.226431+0000","last_became_peered":"2024-05-08T05:49:19.226431+0000","last_unstale":"2024-05-08T05:49:21.740566+0000","last_undegraded":"2024-05-08T05:49:21.740566+0000","last_fullsized":"2024-05-08T05:49:21.740566+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:03.552401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712265+0000","last_change":"2024-05-08T05:49:20.235706+0000","last_active":"2024-05-08T05:49:21.712265+0000","last_peered":"2024-05-08T05:49:21.712265+0000","last_clean":"2024-05-08T05:49:21.712265+0000","last_became_active":"2024-05-08T05:49:20.235443+0000","last_became_peered":"2024-05-08T05:49:20.235443+0000","last_unstale":"2024-05-08T05:49:21.712265+0000","last_undegraded":"2024-05-08T05:49:21.712265+0000","last_fullsized":"2024-05-08T05:49:21.712265+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:03:14.991159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712275+0000","last_change":"2024-05-08T05:49:19.225382+0000","last_active":"2024-05-08T05:49:21.712275+0000","last_peered":"2024-05-08T05:49:21.712275+0000","last_clean":"2024-05-08T05:49:21.712275+0000","last_became_active":"2024-05-08T05:49:19.225112+0000","last_became_peered":"2024-05-08T05:49:19.225112+0000","last_unstale":"2024-05-08T05:49:21.712275+0000","last_undegraded":"2024-05-08T05:49:21.712275+0000","last_fullsized":"2024-05-08T05:49:21.712275+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:38:16.153929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222069+0000","last_change":"2024-05-08T05:49:20.220318+0000","last_active":"2024-05-08T05:49:21.222069+0000","last_peered":"2024-05-08T05:49:21.222069+0000","last_clean":"2024-05-08T05:49:21.222069+0000","last_became_active":"2024-05-08T05:49:20.217950+0000","last_became_peered":"2024-05-08T05:49:20.217950+0000","last_unstale":"2024-05-08T05:49:21.222069+0000","last_undegraded":"2024-05-08T05:49:21.222069+0000","last_fullsized":"2024-05-08T05:49:21.222069+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:45.720876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222167+0000","last_change":"2024-05-08T05:49:19.217546+0000","last_active":"2024-05-08T05:49:21.222167+0000","last_peered":"2024-05-08T05:49:21.222167+0000","last_clean":"2024-05-08T05:49:21.222167+0000","last_became_active":"2024-05-08T05:49:19.217253+0000","last_became_peered":"2024-05-08T05:49:19.217253+0000","last_unstale":"2024-05-08T05:49:21.222167+0000","last_undegraded":"2024-05-08T05:49:21.222167+0000","last_fullsized":"2024-05-08T05:49:21.222167+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:06:33.415067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740912+0000","last_change":"2024-05-08T05:49:20.231417+0000","last_active":"2024-05-08T05:49:21.740912+0000","last_peered":"2024-05-08T05:49:21.740912+0000","last_clean":"2024-05-08T05:49:21.740912+0000","last_became_active":"2024-05-08T05:49:20.227160+0000","last_became_peered":"2024-05-08T05:49:20.227160+0000","last_unstale":"2024-05-08T05:49:21.740912+0000","last_undegraded":"2024-05-08T05:49:21.740912+0000","last_fullsized":"2024-05-08T05:49:21.740912+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:22.685457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740857+0000","last_change":"2024-05-08T05:49:19.223093+0000","last_active":"2024-05-08T05:49:21.740857+0000","last_peered":"2024-05-08T05:49:21.740857+0000","last_clean":"2024-05-08T05:49:21.740857+0000","last_became_active":"2024-05-08T05:49:19.222791+0000","last_became_peered":"2024-05-08T05:49:19.222791+0000","last_unstale":"2024-05-08T05:49:21.740857+0000","last_undegraded":"2024-05-08T05:49:21.740857+0000","last_fullsized":"2024-05-08T05:49:21.740857+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:39:02.674771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741089+0000","last_change":"2024-05-08T05:49:20.229318+0000","last_active":"2024-05-08T05:49:21.741089+0000","last_peered":"2024-05-08T05:49:21.741089+0000","last_clean":"2024-05-08T05:49:21.741089+0000","last_became_active":"2024-05-08T05:49:20.229024+0000","last_became_peered":"2024-05-08T05:49:20.229024+0000","last_unstale":"2024-05-08T05:49:21.741089+0000","last_undegraded":"2024-05-08T05:49:21.741089+0000","last_fullsized":"2024-05-08T05:49:21.741089+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:59:57.323293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741131+0000","last_change":"2024-05-08T05:49:19.224342+0000","last_active":"2024-05-08T05:49:21.741131+0000","last_peered":"2024-05-08T05:49:21.741131+0000","last_clean":"2024-05-08T05:49:21.741131+0000","last_became_active":"2024-05-08T05:49:19.224020+0000","last_became_peered":"2024-05-08T05:49:19.224020+0000","last_unstale":"2024-05-08T05:49:21.741131+0000","last_undegraded":"2024-05-08T05:49:21.741131+0000","last_fullsized":"2024-05-08T05:49:21.741131+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:00:58.026398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220824+0000","last_change":"2024-05-08T05:49:20.224834+0000","last_active":"2024-05-08T05:49:21.220824+0000","last_peered":"2024-05-08T05:49:21.220824+0000","last_clean":"2024-05-08T05:49:21.220824+0000","last_became_active":"2024-05-08T05:49:20.224174+0000","last_became_peered":"2024-05-08T05:49:20.224174+0000","last_unstale":"2024-05-08T05:49:21.220824+0000","last_undegraded":"2024-05-08T05:49:21.220824+0000","last_fullsized":"2024-05-08T05:49:21.220824+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:49:23.007222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740317+0000","last_change":"2024-05-08T05:49:19.219998+0000","last_active":"2024-05-08T05:49:21.740317+0000","last_peered":"2024-05-08T05:49:21.740317+0000","last_clean":"2024-05-08T05:49:21.740317+0000","last_became_active":"2024-05-08T05:49:19.219620+0000","last_became_peered":"2024-05-08T05:49:19.219620+0000","last_unstale":"2024-05-08T05:49:21.740317+0000","last_undegraded":"2024-05-08T05:49:21.740317+0000","last_fullsized":"2024-05-08T05:49:21.740317+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:40:47.136337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741866+0000","last_change":"2024-05-08T05:49:20.236304+0000","last_active":"2024-05-08T05:49:21.741866+0000","last_peered":"2024-05-08T05:49:21.741866+0000","last_clean":"2024-05-08T05:49:21.741866+0000","last_became_active":"2024-05-08T05:49:20.235963+0000","last_became_peered":"2024-05-08T05:49:20.235963+0000","last_unstale":"2024-05-08T05:49:21.741866+0000","last_undegraded":"2024-05-08T05:49:21.741866+0000","last_fullsized":"2024-05-08T05:49:21.741866+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:55:24.707275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.221673+0000","last_change":"2024-05-08T05:49:19.217561+0000","last_active":"2024-05-08T05:49:21.221673+0000","last_peered":"2024-05-08T05:49:21.221673+0000","last_clean":"2024-05-08T05:49:21.221673+0000","last_became_active":"2024-05-08T05:49:19.217233+0000","last_became_peered":"2024-05-08T05:49:19.217233+0000","last_unstale":"2024-05-08T05:49:21.221673+0000","last_undegraded":"2024-05-08T05:49:21.221673+0000","last_fullsized":"2024-05-08T05:49:21.221673+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:39:02.674771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740504+0000","last_change":"2024-05-08T05:49:20.239988+0000","last_active":"2024-05-08T05:49:21.740504+0000","last_peered":"2024-05-08T05:49:21.740504+0000","last_clean":"2024-05-08T05:49:21.740504+0000","last_became_active":"2024-05-08T05:49:20.239698+0000","last_became_peered":"2024-05-08T05:49:20.239698+0000","last_unstale":"2024-05-08T05:49:21.740504+0000","last_undegraded":"2024-05-08T05:49:21.740504+0000","last_fullsized":"2024-05-08T05:49:21.740504+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:55:30.372059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.739885+0000","last_change":"2024-05-08T05:49:19.219609+0000","last_active":"2024-05-08T05:49:21.739885+0000","last_peered":"2024-05-08T05:49:21.739885+0000","last_clean":"2024-05-08T05:49:21.739885+0000","last_became_active":"2024-05-08T05:49:19.219336+0000","last_became_peered":"2024-05-08T05:49:19.219336+0000","last_unstale":"2024-05-08T05:49:21.739885+0000","last_undegraded":"2024-05-08T05:49:21.739885+0000","last_fullsized":"2024-05-08T05:49:21.739885+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:26:26.893270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220290+0000","last_change":"2024-05-08T05:49:20.241346+0000","last_active":"2024-05-08T05:49:21.220290+0000","last_peered":"2024-05-08T05:49:21.220290+0000","last_clean":"2024-05-08T05:49:21.220290+0000","last_became_active":"2024-05-08T05:49:20.241001+0000","last_became_peered":"2024-05-08T05:49:20.241001+0000","last_unstale":"2024-05-08T05:49:21.220290+0000","last_undegraded":"2024-05-08T05:49:21.220290+0000","last_fullsized":"2024-05-08T05:49:21.220290+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:17:51.356349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222388+0000","last_change":"2024-05-08T05:49:19.217501+0000","last_active":"2024-05-08T05:49:21.222388+0000","last_peered":"2024-05-08T05:49:21.222388+0000","last_clean":"2024-05-08T05:49:21.222388+0000","last_became_active":"2024-05-08T05:49:19.217160+0000","last_became_peered":"2024-05-08T05:49:19.217160+0000","last_unstale":"2024-05-08T05:49:21.222388+0000","last_undegraded":"2024-05-08T05:49:21.222388+0000","last_fullsized":"2024-05-08T05:49:21.222388+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:40:47.136337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740750+0000","last_change":"2024-05-08T05:49:20.225909+0000","last_active":"2024-05-08T05:49:21.740750+0000","last_peered":"2024-05-08T05:49:21.740750+0000","last_clean":"2024-05-08T05:49:21.740750+0000","last_became_active":"2024-05-08T05:49:20.225617+0000","last_became_peered":"2024-05-08T05:49:20.225617+0000","last_unstale":"2024-05-08T05:49:21.740750+0000","last_undegraded":"2024-05-08T05:49:21.740750+0000","last_fullsized":"2024-05-08T05:49:21.740750+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:20.999287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220357+0000","last_change":"2024-05-08T05:49:19.211783+0000","last_active":"2024-05-08T05:49:21.220357+0000","last_peered":"2024-05-08T05:49:21.220357+0000","last_clean":"2024-05-08T05:49:21.220357+0000","last_became_active":"2024-05-08T05:49:19.211495+0000","last_became_peered":"2024-05-08T05:49:19.211495+0000","last_unstale":"2024-05-08T05:49:21.220357+0000","last_undegraded":"2024-05-08T05:49:21.220357+0000","last_fullsized":"2024-05-08T05:49:21.220357+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:33:36.204034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712197+0000","last_change":"2024-05-08T05:49:20.232202+0000","last_active":"2024-05-08T05:49:21.712197+0000","last_peered":"2024-05-08T05:49:21.712197+0000","last_clean":"2024-05-08T05:49:21.712197+0000","last_became_active":"2024-05-08T05:49:20.231919+0000","last_became_peered":"2024-05-08T05:49:20.231919+0000","last_unstale":"2024-05-08T05:49:21.712197+0000","last_undegraded":"2024-05-08T05:49:21.712197+0000","last_fullsized":"2024-05-08T05:49:21.712197+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:30:47.129884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"103'3","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740334+0000","last_change":"2024-05-08T05:49:19.220761+0000","last_active":"2024-05-08T05:49:21.740334+0000","last_peered":"2024-05-08T05:49:21.740334+0000","last_clean":"2024-05-08T05:49:21.740334+0000","last_became_active":"2024-05-08T05:49:19.220244+0000","last_became_peered":"2024-05-08T05:49:19.220244+0000","last_unstale":"2024-05-08T05:49:21.740334+0000","last_undegraded":"2024-05-08T05:49:21.740334+0000","last_fullsized":"2024-05-08T05:49:21.740334+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:00:43.392678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740671+0000","last_change":"2024-05-08T05:49:20.235723+0000","last_active":"2024-05-08T05:49:21.740671+0000","last_peered":"2024-05-08T05:49:21.740671+0000","last_clean":"2024-05-08T05:49:21.740671+0000","last_became_active":"2024-05-08T05:49:20.235051+0000","last_became_peered":"2024-05-08T05:49:20.235051+0000","last_unstale":"2024-05-08T05:49:21.740671+0000","last_undegraded":"2024-05-08T05:49:21.740671+0000","last_fullsized":"2024-05-08T05:49:21.740671+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:45.720876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"103'6","reported_seq":20,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741184+0000","last_change":"2024-05-08T05:49:19.220558+0000","last_active":"2024-05-08T05:49:21.741184+0000","last_peered":"2024-05-08T05:49:21.741184+0000","last_clean":"2024-05-08T05:49:21.741184+0000","last_became_active":"2024-05-08T05:49:19.220251+0000","last_became_peered":"2024-05-08T05:49:19.220251+0000","last_unstale":"2024-05-08T05:49:21.741184+0000","last_undegraded":"2024-05-08T05:49:21.741184+0000","last_fullsized":"2024-05-08T05:49:21.741184+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:06:33.415067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740567+0000","last_change":"2024-05-08T05:49:20.229666+0000","last_active":"2024-05-08T05:49:21.740567+0000","last_peered":"2024-05-08T05:49:21.740567+0000","last_clean":"2024-05-08T05:49:21.740567+0000","last_became_active":"2024-05-08T05:49:20.227506+0000","last_became_peered":"2024-05-08T05:49:20.227506+0000","last_unstale":"2024-05-08T05:49:21.740567+0000","last_undegraded":"2024-05-08T05:49:21.740567+0000","last_fullsized":"2024-05-08T05:49:21.740567+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:45.603541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740730+0000","last_change":"2024-05-08T05:49:19.225759+0000","last_active":"2024-05-08T05:49:21.740730+0000","last_peered":"2024-05-08T05:49:21.740730+0000","last_clean":"2024-05-08T05:49:21.740730+0000","last_became_active":"2024-05-08T05:49:19.225054+0000","last_became_peered":"2024-05-08T05:49:19.225054+0000","last_unstale":"2024-05-08T05:49:21.740730+0000","last_undegraded":"2024-05-08T05:49:21.740730+0000","last_fullsized":"2024-05-08T05:49:21.740730+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:06:33.415067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741384+0000","last_change":"2024-05-08T05:49:20.225107+0000","last_active":"2024-05-08T05:49:21.741384+0000","last_peered":"2024-05-08T05:49:21.741384+0000","last_clean":"2024-05-08T05:49:21.741384+0000","last_became_active":"2024-05-08T05:49:20.224832+0000","last_became_peered":"2024-05-08T05:49:20.224832+0000","last_unstale":"2024-05-08T05:49:21.741384+0000","last_undegraded":"2024-05-08T05:49:21.741384+0000","last_fullsized":"2024-05-08T05:49:21.741384+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:04.560434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740682+0000","last_change":"2024-05-08T05:49:19.220845+0000","last_active":"2024-05-08T05:49:21.740682+0000","last_peered":"2024-05-08T05:49:21.740682+0000","last_clean":"2024-05-08T05:49:21.740682+0000","last_became_active":"2024-05-08T05:49:19.220519+0000","last_became_peered":"2024-05-08T05:49:19.220519+0000","last_unstale":"2024-05-08T05:49:21.740682+0000","last_undegraded":"2024-05-08T05:49:21.740682+0000","last_fullsized":"2024-05-08T05:49:21.740682+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:02:04.763302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712390+0000","last_change":"2024-05-08T05:49:20.234140+0000","last_active":"2024-05-08T05:49:21.712390+0000","last_peered":"2024-05-08T05:49:21.712390+0000","last_clean":"2024-05-08T05:49:21.712390+0000","last_became_active":"2024-05-08T05:49:20.233639+0000","last_became_peered":"2024-05-08T05:49:20.233639+0000","last_unstale":"2024-05-08T05:49:21.712390+0000","last_undegraded":"2024-05-08T05:49:21.712390+0000","last_fullsized":"2024-05-08T05:49:21.712390+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:52:21.870490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712126+0000","last_change":"2024-05-08T05:49:19.226502+0000","last_active":"2024-05-08T05:49:21.712126+0000","last_peered":"2024-05-08T05:49:21.712126+0000","last_clean":"2024-05-08T05:49:21.712126+0000","last_became_active":"2024-05-08T05:49:19.226217+0000","last_became_peered":"2024-05-08T05:49:19.226217+0000","last_unstale":"2024-05-08T05:49:21.712126+0000","last_undegraded":"2024-05-08T05:49:21.712126+0000","last_fullsized":"2024-05-08T05:49:21.712126+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:26:51.894930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220622+0000","last_change":"2024-05-08T05:49:20.228556+0000","last_active":"2024-05-08T05:49:21.220622+0000","last_peered":"2024-05-08T05:49:21.220622+0000","last_clean":"2024-05-08T05:49:21.220622+0000","last_became_active":"2024-05-08T05:49:20.228242+0000","last_became_peered":"2024-05-08T05:49:20.228242+0000","last_unstale":"2024-05-08T05:49:21.220622+0000","last_undegraded":"2024-05-08T05:49:21.220622+0000","last_fullsized":"2024-05-08T05:49:21.220622+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:44.880570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"103'8","reported_seq":22,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.221517+0000","last_change":"2024-05-08T05:49:19.221739+0000","last_active":"2024-05-08T05:49:21.221517+0000","last_peered":"2024-05-08T05:49:21.221517+0000","last_clean":"2024-05-08T05:49:21.221517+0000","last_became_active":"2024-05-08T05:49:19.221466+0000","last_became_peered":"2024-05-08T05:49:19.221466+0000","last_unstale":"2024-05-08T05:49:21.221517+0000","last_undegraded":"2024-05-08T05:49:21.221517+0000","last_fullsized":"2024-05-08T05:49:21.221517+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:01.127846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712529+0000","last_change":"2024-05-08T05:49:20.231557+0000","last_active":"2024-05-08T05:49:21.712529+0000","last_peered":"2024-05-08T05:49:21.712529+0000","last_clean":"2024-05-08T05:49:21.712529+0000","last_became_active":"2024-05-08T05:49:20.231028+0000","last_became_peered":"2024-05-08T05:49:20.231028+0000","last_unstale":"2024-05-08T05:49:21.712529+0000","last_undegraded":"2024-05-08T05:49:21.712529+0000","last_fullsized":"2024-05-08T05:49:21.712529+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:59:56.161183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741817+0000","last_change":"2024-05-08T05:49:19.219459+0000","last_active":"2024-05-08T05:49:21.741817+0000","last_peered":"2024-05-08T05:49:21.741817+0000","last_clean":"2024-05-08T05:49:21.741817+0000","last_became_active":"2024-05-08T05:49:19.219157+0000","last_became_peered":"2024-05-08T05:49:19.219157+0000","last_unstale":"2024-05-08T05:49:21.741817+0000","last_undegraded":"2024-05-08T05:49:21.741817+0000","last_fullsized":"2024-05-08T05:49:21.741817+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:57.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741175+0000","last_change":"2024-05-08T05:49:20.238812+0000","last_active":"2024-05-08T05:49:21.741175+0000","last_peered":"2024-05-08T05:49:21.741175+0000","last_clean":"2024-05-08T05:49:21.741175+0000","last_became_active":"2024-05-08T05:49:20.238456+0000","last_became_peered":"2024-05-08T05:49:20.238456+0000","last_unstale":"2024-05-08T05:49:21.741175+0000","last_undegraded":"2024-05-08T05:49:21.741175+0000","last_fullsized":"2024-05-08T05:49:21.741175+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:26:27.901303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741280+0000","last_change":"2024-05-08T05:49:19.221812+0000","last_active":"2024-05-08T05:49:21.741280+0000","last_peered":"2024-05-08T05:49:21.741280+0000","last_clean":"2024-05-08T05:49:21.741280+0000","last_became_active":"2024-05-08T05:49:19.221501+0000","last_became_peered":"2024-05-08T05:49:19.221501+0000","last_unstale":"2024-05-08T05:49:21.741280+0000","last_undegraded":"2024-05-08T05:49:21.741280+0000","last_fullsized":"2024-05-08T05:49:21.741280+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:55:23.699243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740514+0000","last_change":"2024-05-08T05:49:20.235973+0000","last_active":"2024-05-08T05:49:21.740514+0000","last_peered":"2024-05-08T05:49:21.740514+0000","last_clean":"2024-05-08T05:49:21.740514+0000","last_became_active":"2024-05-08T05:49:20.235442+0000","last_became_peered":"2024-05-08T05:49:20.235442+0000","last_unstale":"2024-05-08T05:49:21.740514+0000","last_undegraded":"2024-05-08T05:49:21.740514+0000","last_fullsized":"2024-05-08T05:49:21.740514+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:58.466547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740680+0000","last_change":"2024-05-08T05:49:19.222187+0000","last_active":"2024-05-08T05:49:21.740680+0000","last_peered":"2024-05-08T05:49:21.740680+0000","last_clean":"2024-05-08T05:49:21.740680+0000","last_became_active":"2024-05-08T05:49:19.221863+0000","last_became_peered":"2024-05-08T05:49:19.221863+0000","last_unstale":"2024-05-08T05:49:21.740680+0000","last_undegraded":"2024-05-08T05:49:21.740680+0000","last_fullsized":"2024-05-08T05:49:21.740680+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:57.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220349+0000","last_change":"2024-05-08T05:49:20.237821+0000","last_active":"2024-05-08T05:49:21.220349+0000","last_peered":"2024-05-08T05:49:21.220349+0000","last_clean":"2024-05-08T05:49:21.220349+0000","last_became_active":"2024-05-08T05:49:20.237417+0000","last_became_peered":"2024-05-08T05:49:20.237417+0000","last_unstale":"2024-05-08T05:49:21.220349+0000","last_undegraded":"2024-05-08T05:49:21.220349+0000","last_fullsized":"2024-05-08T05:49:21.220349+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:06:50.147335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712469+0000","last_change":"2024-05-08T05:49:19.222219+0000","last_active":"2024-05-08T05:49:21.712469+0000","last_peered":"2024-05-08T05:49:21.712469+0000","last_clean":"2024-05-08T05:49:21.712469+0000","last_became_active":"2024-05-08T05:49:19.221938+0000","last_became_peered":"2024-05-08T05:49:19.221938+0000","last_unstale":"2024-05-08T05:49:21.712469+0000","last_undegraded":"2024-05-08T05:49:21.712469+0000","last_fullsized":"2024-05-08T05:49:21.712469+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:11.598891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220747+0000","last_change":"2024-05-08T05:49:20.225075+0000","last_active":"2024-05-08T05:49:21.220747+0000","last_peered":"2024-05-08T05:49:21.220747+0000","last_clean":"2024-05-08T05:49:21.220747+0000","last_became_active":"2024-05-08T05:49:20.224537+0000","last_became_peered":"2024-05-08T05:49:20.224537+0000","last_unstale":"2024-05-08T05:49:21.220747+0000","last_undegraded":"2024-05-08T05:49:21.220747+0000","last_fullsized":"2024-05-08T05:49:21.220747+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:10:17.796692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.711959+0000","last_change":"2024-05-08T05:49:19.226655+0000","last_active":"2024-05-08T05:49:21.711959+0000","last_peered":"2024-05-08T05:49:21.711959+0000","last_clean":"2024-05-08T05:49:21.711959+0000","last_became_active":"2024-05-08T05:49:19.226385+0000","last_became_peered":"2024-05-08T05:49:19.226385+0000","last_unstale":"2024-05-08T05:49:21.711959+0000","last_undegraded":"2024-05-08T05:49:21.711959+0000","last_fullsized":"2024-05-08T05:49:21.711959+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:09.778236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740421+0000","last_change":"2024-05-08T05:49:20.235440+0000","last_active":"2024-05-08T05:49:21.740421+0000","last_peered":"2024-05-08T05:49:21.740421+0000","last_clean":"2024-05-08T05:49:21.740421+0000","last_became_active":"2024-05-08T05:49:20.235109+0000","last_became_peered":"2024-05-08T05:49:20.235109+0000","last_unstale":"2024-05-08T05:49:21.740421+0000","last_undegraded":"2024-05-08T05:49:21.740421+0000","last_fullsized":"2024-05-08T05:49:21.740421+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:08:25.723528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740617+0000","last_change":"2024-05-08T05:49:19.218899+0000","last_active":"2024-05-08T05:49:21.740617+0000","last_peered":"2024-05-08T05:49:21.740617+0000","last_clean":"2024-05-08T05:49:21.740617+0000","last_became_active":"2024-05-08T05:49:19.218594+0000","last_became_peered":"2024-05-08T05:49:19.218594+0000","last_unstale":"2024-05-08T05:49:21.740617+0000","last_undegraded":"2024-05-08T05:49:21.740617+0000","last_fullsized":"2024-05-08T05:49:21.740617+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:01.127846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712522+0000","last_change":"2024-05-08T05:49:20.232503+0000","last_active":"2024-05-08T05:49:21.712522+0000","last_peered":"2024-05-08T05:49:21.712522+0000","last_clean":"2024-05-08T05:49:21.712522+0000","last_became_active":"2024-05-08T05:49:20.232220+0000","last_became_peered":"2024-05-08T05:49:20.232220+0000","last_unstale":"2024-05-08T05:49:21.712522+0000","last_undegraded":"2024-05-08T05:49:21.712522+0000","last_fullsized":"2024-05-08T05:49:21.712522+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:27:41.729361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712472+0000","last_change":"2024-05-08T05:49:19.225879+0000","last_active":"2024-05-08T05:49:21.712472+0000","last_peered":"2024-05-08T05:49:21.712472+0000","last_clean":"2024-05-08T05:49:21.712472+0000","last_became_active":"2024-05-08T05:49:19.225603+0000","last_became_peered":"2024-05-08T05:49:19.225603+0000","last_unstale":"2024-05-08T05:49:21.712472+0000","last_undegraded":"2024-05-08T05:49:21.712472+0000","last_fullsized":"2024-05-08T05:49:21.712472+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:36:23.099004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220321+0000","last_change":"2024-05-08T05:49:20.227026+0000","last_active":"2024-05-08T05:49:21.220321+0000","last_peered":"2024-05-08T05:49:21.220321+0000","last_clean":"2024-05-08T05:49:21.220321+0000","last_became_active":"2024-05-08T05:49:20.226533+0000","last_became_peered":"2024-05-08T05:49:20.226533+0000","last_unstale":"2024-05-08T05:49:21.220321+0000","last_undegraded":"2024-05-08T05:49:21.220321+0000","last_fullsized":"2024-05-08T05:49:21.220321+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:56:54.954385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740487+0000","last_change":"2024-05-08T05:49:19.225433+0000","last_active":"2024-05-08T05:49:21.740487+0000","last_peered":"2024-05-08T05:49:21.740487+0000","last_clean":"2024-05-08T05:49:21.740487+0000","last_became_active":"2024-05-08T05:49:19.225126+0000","last_became_peered":"2024-05-08T05:49:19.225126+0000","last_unstale":"2024-05-08T05:49:21.740487+0000","last_undegraded":"2024-05-08T05:49:21.740487+0000","last_fullsized":"2024-05-08T05:49:21.740487+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:01.127846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.221654+0000","last_change":"2024-05-08T05:49:20.222133+0000","last_active":"2024-05-08T05:49:21.221654+0000","last_peered":"2024-05-08T05:49:21.221654+0000","last_clean":"2024-05-08T05:49:21.221654+0000","last_became_active":"2024-05-08T05:49:20.221812+0000","last_became_peered":"2024-05-08T05:49:20.221812+0000","last_unstale":"2024-05-08T05:49:21.221654+0000","last_undegraded":"2024-05-08T05:49:21.221654+0000","last_fullsized":"2024-05-08T05:49:21.221654+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:42.929693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"104'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220391+0000","last_change":"2024-05-08T05:49:19.221744+0000","last_active":"2024-05-08T05:49:21.220391+0000","last_peered":"2024-05-08T05:49:21.220391+0000","last_clean":"2024-05-08T05:49:21.220391+0000","last_became_active":"2024-05-08T05:49:19.221395+0000","last_became_peered":"2024-05-08T05:49:19.221395+0000","last_unstale":"2024-05-08T05:49:21.220391+0000","last_undegraded":"2024-05-08T05:49:21.220391+0000","last_fullsized":"2024-05-08T05:49:21.220391+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:01:13.200266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740671+0000","last_change":"2024-05-08T05:49:20.231747+0000","last_active":"2024-05-08T05:49:21.740671+0000","last_peered":"2024-05-08T05:49:21.740671+0000","last_clean":"2024-05-08T05:49:21.740671+0000","last_became_active":"2024-05-08T05:49:20.231483+0000","last_became_peered":"2024-05-08T05:49:20.231483+0000","last_unstale":"2024-05-08T05:49:21.740671+0000","last_undegraded":"2024-05-08T05:49:21.740671+0000","last_fullsized":"2024-05-08T05:49:21.740671+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:54.490928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741491+0000","last_change":"2024-05-08T05:49:19.222418+0000","last_active":"2024-05-08T05:49:21.741491+0000","last_peered":"2024-05-08T05:49:21.741491+0000","last_clean":"2024-05-08T05:49:21.741491+0000","last_became_active":"2024-05-08T05:49:19.222125+0000","last_became_peered":"2024-05-08T05:49:19.222125+0000","last_unstale":"2024-05-08T05:49:21.741491+0000","last_undegraded":"2024-05-08T05:49:21.741491+0000","last_fullsized":"2024-05-08T05:49:21.741491+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:40:47.136337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740889+0000","last_change":"2024-05-08T05:49:20.230762+0000","last_active":"2024-05-08T05:49:21.740889+0000","last_peered":"2024-05-08T05:49:21.740889+0000","last_clean":"2024-05-08T05:49:21.740889+0000","last_became_active":"2024-05-08T05:49:20.228489+0000","last_became_peered":"2024-05-08T05:49:20.228489+0000","last_unstale":"2024-05-08T05:49:21.740889+0000","last_undegraded":"2024-05-08T05:49:21.740889+0000","last_fullsized":"2024-05-08T05:49:21.740889+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:20.999287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740940+0000","last_change":"2024-05-08T05:49:19.221630+0000","last_active":"2024-05-08T05:49:21.740940+0000","last_peered":"2024-05-08T05:49:21.740940+0000","last_clean":"2024-05-08T05:49:21.740940+0000","last_became_active":"2024-05-08T05:49:19.221299+0000","last_became_peered":"2024-05-08T05:49:19.221299+0000","last_unstale":"2024-05-08T05:49:21.740940+0000","last_undegraded":"2024-05-08T05:49:21.740940+0000","last_fullsized":"2024-05-08T05:49:21.740940+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:53:45.139823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712268+0000","last_change":"2024-05-08T05:49:20.235059+0000","last_active":"2024-05-08T05:49:21.712268+0000","last_peered":"2024-05-08T05:49:21.712268+0000","last_clean":"2024-05-08T05:49:21.712268+0000","last_became_active":"2024-05-08T05:49:20.234561+0000","last_became_peered":"2024-05-08T05:49:20.234561+0000","last_unstale":"2024-05-08T05:49:21.712268+0000","last_undegraded":"2024-05-08T05:49:21.712268+0000","last_fullsized":"2024-05-08T05:49:21.712268+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:00:16.941782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.221905+0000","last_change":"2024-05-08T05:49:19.215532+0000","last_active":"2024-05-08T05:49:21.221905+0000","last_peered":"2024-05-08T05:49:21.221905+0000","last_clean":"2024-05-08T05:49:21.221905+0000","last_became_active":"2024-05-08T05:49:19.215217+0000","last_became_peered":"2024-05-08T05:49:19.215217+0000","last_unstale":"2024-05-08T05:49:21.221905+0000","last_undegraded":"2024-05-08T05:49:21.221905+0000","last_fullsized":"2024-05-08T05:49:21.221905+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:53:45.139823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712742+0000","last_change":"2024-05-08T05:49:20.232929+0000","last_active":"2024-05-08T05:49:21.712742+0000","last_peered":"2024-05-08T05:49:21.712742+0000","last_clean":"2024-05-08T05:49:21.712742+0000","last_became_active":"2024-05-08T05:49:20.232635+0000","last_became_peered":"2024-05-08T05:49:20.232635+0000","last_unstale":"2024-05-08T05:49:21.712742+0000","last_undegraded":"2024-05-08T05:49:21.712742+0000","last_fullsized":"2024-05-08T05:49:21.712742+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:47:01.494382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712087+0000","last_change":"2024-05-08T05:49:19.222351+0000","last_active":"2024-05-08T05:49:21.712087+0000","last_peered":"2024-05-08T05:49:21.712087+0000","last_clean":"2024-05-08T05:49:21.712087+0000","last_became_active":"2024-05-08T05:49:19.222092+0000","last_became_peered":"2024-05-08T05:49:19.222092+0000","last_unstale":"2024-05-08T05:49:21.712087+0000","last_undegraded":"2024-05-08T05:49:21.712087+0000","last_fullsized":"2024-05-08T05:49:21.712087+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:22:40.427897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741491+0000","last_change":"2024-05-08T05:49:20.235238+0000","last_active":"2024-05-08T05:49:21.741491+0000","last_peered":"2024-05-08T05:49:21.741491+0000","last_clean":"2024-05-08T05:49:21.741491+0000","last_became_active":"2024-05-08T05:49:20.234933+0000","last_became_peered":"2024-05-08T05:49:20.234933+0000","last_unstale":"2024-05-08T05:49:21.741491+0000","last_undegraded":"2024-05-08T05:49:21.741491+0000","last_fullsized":"2024-05-08T05:49:21.741491+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:59:57.323293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740680+0000","last_change":"2024-05-08T05:49:19.219966+0000","last_active":"2024-05-08T05:49:21.740680+0000","last_peered":"2024-05-08T05:49:21.740680+0000","last_clean":"2024-05-08T05:49:21.740680+0000","last_became_active":"2024-05-08T05:49:19.219555+0000","last_became_peered":"2024-05-08T05:49:19.219555+0000","last_unstale":"2024-05-08T05:49:21.740680+0000","last_undegraded":"2024-05-08T05:49:21.740680+0000","last_fullsized":"2024-05-08T05:49:21.740680+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:53:45.139823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741173+0000","last_change":"2024-05-08T05:49:20.230394+0000","last_active":"2024-05-08T05:49:21.741173+0000","last_peered":"2024-05-08T05:49:21.741173+0000","last_clean":"2024-05-08T05:49:21.741173+0000","last_became_active":"2024-05-08T05:49:20.229931+0000","last_became_peered":"2024-05-08T05:49:20.229931+0000","last_unstale":"2024-05-08T05:49:21.741173+0000","last_undegraded":"2024-05-08T05:49:21.741173+0000","last_fullsized":"2024-05-08T05:49:21.741173+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:26:27.901303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712584+0000","last_change":"2024-05-08T05:49:19.224009+0000","last_active":"2024-05-08T05:49:21.712584+0000","last_peered":"2024-05-08T05:49:21.712584+0000","last_clean":"2024-05-08T05:49:21.712584+0000","last_became_active":"2024-05-08T05:49:19.223694+0000","last_became_peered":"2024-05-08T05:49:19.223694+0000","last_unstale":"2024-05-08T05:49:21.712584+0000","last_undegraded":"2024-05-08T05:49:21.712584+0000","last_fullsized":"2024-05-08T05:49:21.712584+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:36:26.679637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740331+0000","last_change":"2024-05-08T05:49:20.224133+0000","last_active":"2024-05-08T05:49:21.740331+0000","last_peered":"2024-05-08T05:49:21.740331+0000","last_clean":"2024-05-08T05:49:21.740331+0000","last_became_active":"2024-05-08T05:49:20.223808+0000","last_became_peered":"2024-05-08T05:49:20.223808+0000","last_unstale":"2024-05-08T05:49:21.740331+0000","last_undegraded":"2024-05-08T05:49:21.740331+0000","last_fullsized":"2024-05-08T05:49:21.740331+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:42.929693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712441+0000","last_change":"2024-05-08T05:49:19.227947+0000","last_active":"2024-05-08T05:49:21.712441+0000","last_peered":"2024-05-08T05:49:21.712441+0000","last_clean":"2024-05-08T05:49:21.712441+0000","last_became_active":"2024-05-08T05:49:19.227650+0000","last_became_peered":"2024-05-08T05:49:19.227650+0000","last_unstale":"2024-05-08T05:49:21.712441+0000","last_undegraded":"2024-05-08T05:49:21.712441+0000","last_fullsized":"2024-05-08T05:49:21.712441+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:07.219786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":264,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740434+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:49:21.740434+0000","last_peered":"2024-05-08T05:49:21.740434+0000","last_clean":"2024-05-08T05:49:21.740434+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:49:21.740434+0000","last_undegraded":"2024-05-08T05:49:21.740434+0000","last_fullsized":"2024-05-08T05:49:21.740434+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542245,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1888256,"data_stored":1186811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542246,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1306624,"data_stored":598499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542245,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":596283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574948,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1294336,"data_stored":596375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574947,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":596865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574947,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":596317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574947,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":596317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574947,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1896448,"data_stored":1188353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:49:28.770 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:49:28.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.887+0000 7f6c2a951640 1 Processor -- start 2024-05-08T05:49:28.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.887+0000 7f6c2a951640 1 -- start start 2024-05-08T05:49:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.888+0000 7f6c2a951640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6c240b2490 0x7f6c24070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.888+0000 7f6c2a951640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c24070d20 0x7f6c240711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.888+0000 7f6c2a951640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c2406e110 con 0x7f6c241629b0 2024-05-08T05:49:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.888+0000 7f6c2a951640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6c2406dec0 con 0x7f6c240b2490 2024-05-08T05:49:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.888+0000 7f6c2a951640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2406e3f0 con 0x7f6c24070d20 2024-05-08T05:49:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.888+0000 7f6c23fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6c241629b0 0x7f6c2406eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49888/0 (socket says 172.21.15.45:49888) 2024-05-08T05:49:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.888+0000 7f6c23fff640 1 -- 172.21.15.45:0/4286271623 learned_addr learned my addr 172.21.15.45:0/4286271623 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.888+0000 7f6c1bfff640 1 --2- 172.21.15.45:0/4286271623 >> v2:172.21.15.63:3300/0 conn(0x7f6c240b2490 0x7f6c24070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.888+0000 7f6c28eee640 1 --2- 172.21.15.45:0/4286271623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c24070d20 0x7f6c240711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.888+0000 7f6c237fe640 1 -- 172.21.15.45:0/4286271623 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3810852314 0 0) 0x7f6c2406e110 con 0x7f6c241629b0 2024-05-08T05:49:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.889+0000 7f6c237fe640 1 -- 172.21.15.45:0/4286271623 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c10003520 con 0x7f6c241629b0 2024-05-08T05:49:28.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.889+0000 7f6c237fe640 1 -- 172.21.15.45:0/4286271623 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2646027172 0 0) 0x7f6c10003520 con 0x7f6c241629b0 2024-05-08T05:49:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.889+0000 7f6c237fe640 1 -- 172.21.15.45:0/4286271623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c24070d20 msgr2=0x7f6c240711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.889+0000 7f6c237fe640 1 --2- 172.21.15.45:0/4286271623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c24070d20 0x7f6c240711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.889+0000 7f6c237fe640 1 -- 172.21.15.45:0/4286271623 >> v2:172.21.15.63:3300/0 conn(0x7f6c240b2490 msgr2=0x7f6c24070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.889+0000 7f6c237fe640 1 --2- 172.21.15.45:0/4286271623 >> v2:172.21.15.63:3300/0 conn(0x7f6c240b2490 0x7f6c24070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.889+0000 7f6c237fe640 1 -- 172.21.15.45:0/4286271623 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2406e6d0 con 0x7f6c241629b0 2024-05-08T05:49:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.889+0000 7f6c237fe640 1 -- 172.21.15.45:0/4286271623 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f6c140016f0 con 0x7f6c241629b0 2024-05-08T05:49:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.889+0000 7f6c237fe640 1 -- 172.21.15.45:0/4286271623 >> v1:172.21.15.45:6789/0 conn(0x7f6c241629b0 legacy=0x7f6c2406eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.889+0000 7f6c237fe640 1 --2- 172.21.15.45:0/4286271623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c10003a00 0x7f6c10003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.889+0000 7f6c237fe640 1 --2- 172.21.15.45:0/4286271623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c10004800 0x7f6c10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.889+0000 7f6c237fe640 1 --2- 172.21.15.45:0/4286271623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c100051c0 0x7f6c1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.889+0000 7f6c237fe640 1 -- 172.21.15.45:0/4286271623 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2406dec0 con 0x7f6c100051c0 2024-05-08T05:49:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.889+0000 7f6c237fe640 1 -- 172.21.15.45:0/4286271623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2406e3f0 con 0x7f6c10003a00 2024-05-08T05:49:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.889+0000 7f6c237fe640 1 -- 172.21.15.45:0/4286271623 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6c1000bf50 con 0x7f6c10004800 2024-05-08T05:49:28.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.889+0000 7f6c28eee640 1 --2- 172.21.15.45:0/4286271623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c24070d20 0x7f6c240711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:49:28.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.889+0000 7f6c1bfff640 1 --2- 172.21.15.45:0/4286271623 >> v2:172.21.15.63:3300/0 conn(0x7f6c240b2490 0x7f6c24070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:49:28.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.889+0000 7f6c23fff640 1 --2- 172.21.15.45:0/4286271623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c10003a00 0x7f6c10003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:28.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.889+0000 7f6c28eee640 1 --2- 172.21.15.45:0/4286271623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c100051c0 0x7f6c1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:28.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.889+0000 7f6c1bfff640 1 --2- 172.21.15.45:0/4286271623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c10004800 0x7f6c10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:28.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.890+0000 7f6c23fff640 1 -- 172.21.15.45:0/4286271623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c10004800 msgr2=0x7f6c10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:28.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.890+0000 7f6c23fff640 1 --2- 172.21.15.45:0/4286271623 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c10004800 0x7f6c10004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:28.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.890+0000 7f6c23fff640 1 -- 172.21.15.45:0/4286271623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c100051c0 msgr2=0x7f6c1000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:28.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.890+0000 7f6c23fff640 1 --2- 172.21.15.45:0/4286271623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c100051c0 0x7f6c1000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:28.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.890+0000 7f6c23fff640 1 -- 172.21.15.45:0/4286271623 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c10003520 con 0x7f6c10003a00 2024-05-08T05:49:28.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.890+0000 7f6c28eee640 1 --2- 172.21.15.45:0/4286271623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c100051c0 0x7f6c1000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:49:28.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.890+0000 7f6c23fff640 1 --2- 172.21.15.45:0/4286271623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c10003a00 0x7f6c10003e20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400c2c0 tx=0x7f6c10005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdd2ecc56e24e534 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:28.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.891+0000 7f6c237fe640 1 -- 172.21.15.45:0/4286271623 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14007d50 con 0x7f6c10003a00 2024-05-08T05:49:28.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.891+0000 7f6c237fe640 1 -- 172.21.15.45:0/4286271623 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c14005ad0 con 0x7f6c10003a00 2024-05-08T05:49:28.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.891+0000 7f6c237fe640 1 -- 172.21.15.45:0/4286271623 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c14007070 con 0x7f6c10003a00 2024-05-08T05:49:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.891+0000 7f6c2a951640 1 -- 172.21.15.45:0/4286271623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c10003a00 msgr2=0x7f6c10003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.891+0000 7f6c2a951640 1 --2- 172.21.15.45:0/4286271623 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c10003a00 0x7f6c10003e20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400c2c0 tx=0x7f6c10005830 comp rx=0 tx=0).stop 2024-05-08T05:49:28.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.891+0000 7f6c28eee640 1 -- 172.21.15.45:0/4286271623 reap_dead start 2024-05-08T05:49:28.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.891+0000 7f6c2a951640 1 -- 172.21.15.45:0/4286271623 shutdown_connections 2024-05-08T05:49:28.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.891+0000 7f6c2a951640 1 -- 172.21.15.45:0/4286271623 >> 172.21.15.45:0/4286271623 conn(0x7f6c240b49d0 msgr2=0x7f6c240b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:28.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.891+0000 7f6c2a951640 1 -- 172.21.15.45:0/4286271623 shutdown_connections 2024-05-08T05:49:28.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.891+0000 7f6c2a951640 1 -- 172.21.15.45:0/4286271623 wait complete. 2024-05-08T05:49:28.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.893+0000 7f6c2a951640 1 Processor -- start 2024-05-08T05:49:28.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.893+0000 7f6c2a951640 1 -- start start 2024-05-08T05:49:28.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c2a951640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c10003a00 0x7f6c240ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:28.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c2a951640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c24154ee0 0x7f6c24154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:28.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c2a951640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c240b71c0 0x7f6c240b7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:28.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c2a951640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6c240709f0 con 0x7f6c240b71c0 2024-05-08T05:49:28.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c2a951640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6c24070870 con 0x7f6c10003a00 2024-05-08T05:49:28.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c2a951640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6c240711d0 con 0x7f6c24154ee0 2024-05-08T05:49:28.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c28eee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c240b71c0 0x7f6c240b7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c28eee640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c240b71c0 0x7f6c240b7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45466/0 (socket says 172.21.15.45:45466) 2024-05-08T05:49:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c28eee640 1 -- 172.21.15.45:0/1385296885 learned_addr learned my addr 172.21.15.45:0/1385296885 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c23fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c10003a00 0x7f6c240ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c1bfff640 1 --2- 172.21.15.45:0/1385296885 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c24154ee0 0x7f6c24154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c28eee640 1 -- 172.21.15.45:0/1385296885 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c24154ee0 msgr2=0x7f6c24154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c28eee640 1 --2- 172.21.15.45:0/1385296885 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c24154ee0 0x7f6c24154710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c28eee640 1 -- 172.21.15.45:0/1385296885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c10003a00 msgr2=0x7f6c240ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c28eee640 1 --2- 172.21.15.45:0/1385296885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c10003a00 0x7f6c240ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c28eee640 1 -- 172.21.15.45:0/1385296885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c241f9260 con 0x7f6c240b71c0 2024-05-08T05:49:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c28eee640 1 --2- 172.21.15.45:0/1385296885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c240b71c0 0x7f6c240b7640 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f6c24073290 tx=0x7f6c1c00f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c217fa640 1 -- 172.21.15.45:0/1385296885 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c009d40 con 0x7f6c240b71c0 2024-05-08T05:49:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c217fa640 1 -- 172.21.15.45:0/1385296885 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c1c01c070 con 0x7f6c240b71c0 2024-05-08T05:49:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c2a951640 1 -- 172.21.15.45:0/1385296885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c241f84a0 con 0x7f6c240b71c0 2024-05-08T05:49:28.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c217fa640 1 -- 172.21.15.45:0/1385296885 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c003db0 con 0x7f6c240b71c0 2024-05-08T05:49:28.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.894+0000 7f6c2a951640 1 -- 172.21.15.45:0/1385296885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c241f8a60 con 0x7f6c240b71c0 2024-05-08T05:49:28.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.895+0000 7f6c2a951640 1 -- 172.21.15.45:0/1385296885 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c240693f0 con 0x7f6c240b71c0 2024-05-08T05:49:28.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.896+0000 7f6c217fa640 1 -- 172.21.15.45:0/1385296885 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6c1c004030 con 0x7f6c240b71c0 2024-05-08T05:49:28.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.897+0000 7f6c217fa640 1 --2- 172.21.15.45:0/1385296885 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6c040762d0 0x7f6c04078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:28.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.897+0000 7f6c217fa640 1 -- 172.21.15.45:0/1385296885 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f6c1c098120 con 0x7f6c240b71c0 2024-05-08T05:49:28.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.897+0000 7f6c23fff640 1 --2- 172.21.15.45:0/1385296885 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6c040762d0 0x7f6c04078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:28.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.897+0000 7f6c23fff640 1 --2- 172.21.15.45:0/1385296885 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6c040762d0 0x7f6c04078790 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6c14032000 tx=0x7f6c14009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:28.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:28.901+0000 7f6c217fa640 1 -- 172.21.15.45:0/1385296885 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6c1c010040 con 0x7f6c240b71c0 2024-05-08T05:49:29.090 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.090+0000 7f6c2a951640 1 -- 172.21.15.45:0/1385296885 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6c240cd270 con 0x7f6c040762d0 2024-05-08T05:49:29.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.096+0000 7f6c217fa640 1 -- 172.21.15.45:0/1385296885 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252415 (secure 0 0 0) 0x7f6c240cd270 con 0x7f6c040762d0 2024-05-08T05:49:29.099 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:29.100 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:49:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.099+0000 7f6c2a951640 1 -- 172.21.15.45:0/1385296885 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6c040762d0 msgr2=0x7f6c04078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:29.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.099+0000 7f6c2a951640 1 --2- 172.21.15.45:0/1385296885 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6c040762d0 0x7f6c04078790 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6c14032000 tx=0x7f6c14009030 comp rx=0 tx=0).stop 2024-05-08T05:49:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.099+0000 7f6c2a951640 1 -- 172.21.15.45:0/1385296885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c240b71c0 msgr2=0x7f6c240b7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.099+0000 7f6c2a951640 1 --2- 172.21.15.45:0/1385296885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c240b71c0 0x7f6c240b7640 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f6c24073290 tx=0x7f6c1c00f640 comp rx=0 tx=0).stop 2024-05-08T05:49:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.099+0000 7f6c2a951640 1 -- 172.21.15.45:0/1385296885 shutdown_connections 2024-05-08T05:49:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.099+0000 7f6c2a951640 1 --2- 172.21.15.45:0/1385296885 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6c040762d0 0x7f6c04078790 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.099+0000 7f6c2a951640 1 --2- 172.21.15.45:0/1385296885 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6c24154ee0 0x7f6c24154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.099+0000 7f6c2a951640 1 --2- 172.21.15.45:0/1385296885 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c240b71c0 0x7f6c240b7640 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.099+0000 7f6c2a951640 1 --2- 172.21.15.45:0/1385296885 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c10003a00 0x7f6c240ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.099+0000 7f6c2a951640 1 -- 172.21.15.45:0/1385296885 >> 172.21.15.45:0/1385296885 conn(0x7f6c240b49d0 msgr2=0x7f6c240d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.099+0000 7f6c2a951640 1 -- 172.21.15.45:0/1385296885 shutdown_connections 2024-05-08T05:49:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.099+0000 7f6c2a951640 1 -- 172.21.15.45:0/1385296885 wait complete. 2024-05-08T05:49:29.111 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-05-08T05:49:28.949421+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260724,"kb_used_data":11272,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713996,"statfs":{"total":773094113280,"available":772827131904,"internally_reserved":0,"allocated":11542528,"data_stored":5955820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.741751"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220540+0000","last_change":"2024-05-08T05:49:20.238531+0000","last_active":"2024-05-08T05:49:21.220540+0000","last_peered":"2024-05-08T05:49:21.220540+0000","last_clean":"2024-05-08T05:49:21.220540+0000","last_became_active":"2024-05-08T05:49:20.238249+0000","last_became_peered":"2024-05-08T05:49:20.238249+0000","last_unstale":"2024-05-08T05:49:21.220540+0000","last_undegraded":"2024-05-08T05:49:21.220540+0000","last_fullsized":"2024-05-08T05:49:21.220540+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:00:42.502385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741649+0000","last_change":"2024-05-08T05:49:20.230490+0000","last_active":"2024-05-08T05:49:21.741649+0000","last_peered":"2024-05-08T05:49:21.741649+0000","last_clean":"2024-05-08T05:49:21.741649+0000","last_became_active":"2024-05-08T05:49:20.230163+0000","last_became_peered":"2024-05-08T05:49:20.230163+0000","last_unstale":"2024-05-08T05:49:21.741649+0000","last_undegraded":"2024-05-08T05:49:21.741649+0000","last_fullsized":"2024-05-08T05:49:21.741649+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:01.788691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220788+0000","last_change":"2024-05-08T05:49:20.228822+0000","last_active":"2024-05-08T05:49:21.220788+0000","last_peered":"2024-05-08T05:49:21.220788+0000","last_clean":"2024-05-08T05:49:21.220788+0000","last_became_active":"2024-05-08T05:49:20.228543+0000","last_became_peered":"2024-05-08T05:49:20.228543+0000","last_unstale":"2024-05-08T05:49:21.220788+0000","last_undegraded":"2024-05-08T05:49:21.220788+0000","last_fullsized":"2024-05-08T05:49:21.220788+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:15.407492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740921+0000","last_change":"2024-05-08T05:49:20.227003+0000","last_active":"2024-05-08T05:49:21.740921+0000","last_peered":"2024-05-08T05:49:21.740921+0000","last_clean":"2024-05-08T05:49:21.740921+0000","last_became_active":"2024-05-08T05:49:20.226673+0000","last_became_peered":"2024-05-08T05:49:20.226673+0000","last_unstale":"2024-05-08T05:49:21.740921+0000","last_undegraded":"2024-05-08T05:49:21.740921+0000","last_fullsized":"2024-05-08T05:49:21.740921+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:58:10.209690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740362+0000","last_change":"2024-05-08T05:49:20.228592+0000","last_active":"2024-05-08T05:49:21.740362+0000","last_peered":"2024-05-08T05:49:21.740362+0000","last_clean":"2024-05-08T05:49:21.740362+0000","last_became_active":"2024-05-08T05:49:20.228303+0000","last_became_peered":"2024-05-08T05:49:20.228303+0000","last_unstale":"2024-05-08T05:49:21.740362+0000","last_undegraded":"2024-05-08T05:49:21.740362+0000","last_fullsized":"2024-05-08T05:49:21.740362+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:42.929693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740876+0000","last_change":"2024-05-08T05:49:20.226897+0000","last_active":"2024-05-08T05:49:21.740876+0000","last_peered":"2024-05-08T05:49:21.740876+0000","last_clean":"2024-05-08T05:49:21.740876+0000","last_became_active":"2024-05-08T05:49:20.226591+0000","last_became_peered":"2024-05-08T05:49:20.226591+0000","last_unstale":"2024-05-08T05:49:21.740876+0000","last_undegraded":"2024-05-08T05:49:21.740876+0000","last_fullsized":"2024-05-08T05:49:21.740876+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:22.685457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740345+0000","last_change":"2024-05-08T05:49:20.229697+0000","last_active":"2024-05-08T05:49:21.740345+0000","last_peered":"2024-05-08T05:49:21.740345+0000","last_clean":"2024-05-08T05:49:21.740345+0000","last_became_active":"2024-05-08T05:49:20.229413+0000","last_became_peered":"2024-05-08T05:49:20.229413+0000","last_unstale":"2024-05-08T05:49:21.740345+0000","last_undegraded":"2024-05-08T05:49:21.740345+0000","last_fullsized":"2024-05-08T05:49:21.740345+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:45.603541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712046+0000","last_change":"2024-05-08T05:49:20.231788+0000","last_active":"2024-05-08T05:49:21.712046+0000","last_peered":"2024-05-08T05:49:21.712046+0000","last_clean":"2024-05-08T05:49:21.712046+0000","last_became_active":"2024-05-08T05:49:20.231323+0000","last_became_peered":"2024-05-08T05:49:20.231323+0000","last_unstale":"2024-05-08T05:49:21.712046+0000","last_undegraded":"2024-05-08T05:49:21.712046+0000","last_fullsized":"2024-05-08T05:49:21.712046+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:40:06.613670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741679+0000","last_change":"2024-05-08T05:49:20.230080+0000","last_active":"2024-05-08T05:49:21.741679+0000","last_peered":"2024-05-08T05:49:21.741679+0000","last_clean":"2024-05-08T05:49:21.741679+0000","last_became_active":"2024-05-08T05:49:20.228395+0000","last_became_peered":"2024-05-08T05:49:20.228395+0000","last_unstale":"2024-05-08T05:49:21.741679+0000","last_undegraded":"2024-05-08T05:49:21.741679+0000","last_fullsized":"2024-05-08T05:49:21.741679+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:00:59.034430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740259+0000","last_change":"2024-05-08T05:49:20.235915+0000","last_active":"2024-05-08T05:49:21.740259+0000","last_peered":"2024-05-08T05:49:21.740259+0000","last_clean":"2024-05-08T05:49:21.740259+0000","last_became_active":"2024-05-08T05:49:20.235612+0000","last_became_peered":"2024-05-08T05:49:20.235612+0000","last_unstale":"2024-05-08T05:49:21.740259+0000","last_undegraded":"2024-05-08T05:49:21.740259+0000","last_fullsized":"2024-05-08T05:49:21.740259+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:33.750665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740474+0000","last_change":"2024-05-08T05:49:20.236606+0000","last_active":"2024-05-08T05:49:21.740474+0000","last_peered":"2024-05-08T05:49:21.740474+0000","last_clean":"2024-05-08T05:49:21.740474+0000","last_became_active":"2024-05-08T05:49:20.236336+0000","last_became_peered":"2024-05-08T05:49:20.236336+0000","last_unstale":"2024-05-08T05:49:21.740474+0000","last_undegraded":"2024-05-08T05:49:21.740474+0000","last_fullsized":"2024-05-08T05:49:21.740474+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:58.466547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222198+0000","last_change":"2024-05-08T05:49:20.225246+0000","last_active":"2024-05-08T05:49:21.222198+0000","last_peered":"2024-05-08T05:49:21.222198+0000","last_clean":"2024-05-08T05:49:21.222198+0000","last_became_active":"2024-05-08T05:49:20.224935+0000","last_became_peered":"2024-05-08T05:49:20.224935+0000","last_unstale":"2024-05-08T05:49:21.222198+0000","last_undegraded":"2024-05-08T05:49:21.222198+0000","last_fullsized":"2024-05-08T05:49:21.222198+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:45.603541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740114+0000","last_change":"2024-05-08T05:49:20.233325+0000","last_active":"2024-05-08T05:49:21.740114+0000","last_peered":"2024-05-08T05:49:21.740114+0000","last_clean":"2024-05-08T05:49:21.740114+0000","last_became_active":"2024-05-08T05:49:20.233018+0000","last_became_peered":"2024-05-08T05:49:20.233018+0000","last_unstale":"2024-05-08T05:49:21.740114+0000","last_undegraded":"2024-05-08T05:49:21.740114+0000","last_fullsized":"2024-05-08T05:49:21.740114+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:43:11.551671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740998+0000","last_change":"2024-05-08T05:49:20.224796+0000","last_active":"2024-05-08T05:49:21.740998+0000","last_peered":"2024-05-08T05:49:21.740998+0000","last_clean":"2024-05-08T05:49:21.740998+0000","last_became_active":"2024-05-08T05:49:20.224480+0000","last_became_peered":"2024-05-08T05:49:20.224480+0000","last_unstale":"2024-05-08T05:49:21.740998+0000","last_undegraded":"2024-05-08T05:49:21.740998+0000","last_fullsized":"2024-05-08T05:49:21.740998+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:58:10.209690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740735+0000","last_change":"2024-05-08T05:49:20.229339+0000","last_active":"2024-05-08T05:49:21.740735+0000","last_peered":"2024-05-08T05:49:21.740735+0000","last_clean":"2024-05-08T05:49:21.740735+0000","last_became_active":"2024-05-08T05:49:20.229066+0000","last_became_peered":"2024-05-08T05:49:20.229066+0000","last_unstale":"2024-05-08T05:49:21.740735+0000","last_undegraded":"2024-05-08T05:49:21.740735+0000","last_fullsized":"2024-05-08T05:49:21.740735+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:54.490928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740316+0000","last_change":"2024-05-08T05:49:20.228402+0000","last_active":"2024-05-08T05:49:21.740316+0000","last_peered":"2024-05-08T05:49:21.740316+0000","last_clean":"2024-05-08T05:49:21.740316+0000","last_became_active":"2024-05-08T05:49:20.228132+0000","last_became_peered":"2024-05-08T05:49:20.228132+0000","last_unstale":"2024-05-08T05:49:21.740316+0000","last_undegraded":"2024-05-08T05:49:21.740316+0000","last_fullsized":"2024-05-08T05:49:21.740316+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:55:30.372059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220628+0000","last_change":"2024-05-08T05:49:20.228209+0000","last_active":"2024-05-08T05:49:21.220628+0000","last_peered":"2024-05-08T05:49:21.220628+0000","last_clean":"2024-05-08T05:49:21.220628+0000","last_became_active":"2024-05-08T05:49:20.227869+0000","last_became_peered":"2024-05-08T05:49:20.227869+0000","last_unstale":"2024-05-08T05:49:21.220628+0000","last_undegraded":"2024-05-08T05:49:21.220628+0000","last_fullsized":"2024-05-08T05:49:21.220628+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:24.394443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741707+0000","last_change":"2024-05-08T05:49:20.218731+0000","last_active":"2024-05-08T05:49:21.741707+0000","last_peered":"2024-05-08T05:49:21.741707+0000","last_clean":"2024-05-08T05:49:21.741707+0000","last_became_active":"2024-05-08T05:49:20.218388+0000","last_became_peered":"2024-05-08T05:49:20.218388+0000","last_unstale":"2024-05-08T05:49:21.741707+0000","last_undegraded":"2024-05-08T05:49:21.741707+0000","last_fullsized":"2024-05-08T05:49:21.741707+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:00:44.400710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220462+0000","last_change":"2024-05-08T05:49:20.241520+0000","last_active":"2024-05-08T05:49:21.220462+0000","last_peered":"2024-05-08T05:49:21.220462+0000","last_clean":"2024-05-08T05:49:21.220462+0000","last_became_active":"2024-05-08T05:49:20.241194+0000","last_became_peered":"2024-05-08T05:49:20.241194+0000","last_unstale":"2024-05-08T05:49:21.220462+0000","last_undegraded":"2024-05-08T05:49:21.220462+0000","last_fullsized":"2024-05-08T05:49:21.220462+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:44:17.867536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222371+0000","last_change":"2024-05-08T05:49:20.222796+0000","last_active":"2024-05-08T05:49:21.222371+0000","last_peered":"2024-05-08T05:49:21.222371+0000","last_clean":"2024-05-08T05:49:21.222371+0000","last_became_active":"2024-05-08T05:49:20.222415+0000","last_became_peered":"2024-05-08T05:49:20.222415+0000","last_unstale":"2024-05-08T05:49:21.222371+0000","last_undegraded":"2024-05-08T05:49:21.222371+0000","last_fullsized":"2024-05-08T05:49:21.222371+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:20.999287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712297+0000","last_change":"2024-05-08T05:49:20.236618+0000","last_active":"2024-05-08T05:49:21.712297+0000","last_peered":"2024-05-08T05:49:21.712297+0000","last_clean":"2024-05-08T05:49:21.712297+0000","last_became_active":"2024-05-08T05:49:20.236323+0000","last_became_peered":"2024-05-08T05:49:20.236323+0000","last_unstale":"2024-05-08T05:49:21.712297+0000","last_undegraded":"2024-05-08T05:49:21.712297+0000","last_fullsized":"2024-05-08T05:49:21.712297+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:43:28.803611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740244+0000","last_change":"2024-05-08T05:49:20.228986+0000","last_active":"2024-05-08T05:49:21.740244+0000","last_peered":"2024-05-08T05:49:21.740244+0000","last_clean":"2024-05-08T05:49:21.740244+0000","last_became_active":"2024-05-08T05:49:20.228723+0000","last_became_peered":"2024-05-08T05:49:20.228723+0000","last_unstale":"2024-05-08T05:49:21.740244+0000","last_undegraded":"2024-05-08T05:49:21.740244+0000","last_fullsized":"2024-05-08T05:49:21.740244+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:17:48.859578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220270+0000","last_change":"2024-05-08T05:49:20.225628+0000","last_active":"2024-05-08T05:49:21.220270+0000","last_peered":"2024-05-08T05:49:21.220270+0000","last_clean":"2024-05-08T05:49:21.220270+0000","last_became_active":"2024-05-08T05:49:20.225065+0000","last_became_peered":"2024-05-08T05:49:20.225065+0000","last_unstale":"2024-05-08T05:49:21.220270+0000","last_undegraded":"2024-05-08T05:49:21.220270+0000","last_fullsized":"2024-05-08T05:49:21.220270+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:00:44.636681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740709+0000","last_change":"2024-05-08T05:49:20.239478+0000","last_active":"2024-05-08T05:49:21.740709+0000","last_peered":"2024-05-08T05:49:21.740709+0000","last_clean":"2024-05-08T05:49:21.740709+0000","last_became_active":"2024-05-08T05:49:20.239214+0000","last_became_peered":"2024-05-08T05:49:20.239214+0000","last_unstale":"2024-05-08T05:49:21.740709+0000","last_undegraded":"2024-05-08T05:49:21.740709+0000","last_fullsized":"2024-05-08T05:49:21.740709+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:45.603541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222394+0000","last_change":"2024-05-08T05:49:20.229377+0000","last_active":"2024-05-08T05:49:21.222394+0000","last_peered":"2024-05-08T05:49:21.222394+0000","last_clean":"2024-05-08T05:49:21.222394+0000","last_became_active":"2024-05-08T05:49:20.229049+0000","last_became_peered":"2024-05-08T05:49:20.229049+0000","last_unstale":"2024-05-08T05:49:21.222394+0000","last_undegraded":"2024-05-08T05:49:21.222394+0000","last_fullsized":"2024-05-08T05:49:21.222394+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:26:27.901303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222386+0000","last_change":"2024-05-08T05:49:20.231741+0000","last_active":"2024-05-08T05:49:21.222386+0000","last_peered":"2024-05-08T05:49:21.222386+0000","last_clean":"2024-05-08T05:49:21.222386+0000","last_became_active":"2024-05-08T05:49:20.231459+0000","last_became_peered":"2024-05-08T05:49:20.231459+0000","last_unstale":"2024-05-08T05:49:21.222386+0000","last_undegraded":"2024-05-08T05:49:21.222386+0000","last_fullsized":"2024-05-08T05:49:21.222386+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:22.685457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740538+0000","last_change":"2024-05-08T05:49:20.228283+0000","last_active":"2024-05-08T05:49:21.740538+0000","last_peered":"2024-05-08T05:49:21.740538+0000","last_clean":"2024-05-08T05:49:21.740538+0000","last_became_active":"2024-05-08T05:49:20.227936+0000","last_became_peered":"2024-05-08T05:49:20.227936+0000","last_unstale":"2024-05-08T05:49:21.740538+0000","last_undegraded":"2024-05-08T05:49:21.740538+0000","last_fullsized":"2024-05-08T05:49:21.740538+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:45.720876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741377+0000","last_change":"2024-05-08T05:49:20.231471+0000","last_active":"2024-05-08T05:49:21.741377+0000","last_peered":"2024-05-08T05:49:21.741377+0000","last_clean":"2024-05-08T05:49:21.741377+0000","last_became_active":"2024-05-08T05:49:20.230910+0000","last_became_peered":"2024-05-08T05:49:20.230910+0000","last_unstale":"2024-05-08T05:49:21.741377+0000","last_undegraded":"2024-05-08T05:49:21.741377+0000","last_fullsized":"2024-05-08T05:49:21.741377+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:02:05.771334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712332+0000","last_change":"2024-05-08T05:49:20.230069+0000","last_active":"2024-05-08T05:49:21.712332+0000","last_peered":"2024-05-08T05:49:21.712332+0000","last_clean":"2024-05-08T05:49:21.712332+0000","last_became_active":"2024-05-08T05:49:20.229773+0000","last_became_peered":"2024-05-08T05:49:20.229773+0000","last_unstale":"2024-05-08T05:49:21.712332+0000","last_undegraded":"2024-05-08T05:49:21.712332+0000","last_fullsized":"2024-05-08T05:49:21.712332+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:52:31.166852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.221696+0000","last_change":"2024-05-08T05:49:20.228744+0000","last_active":"2024-05-08T05:49:21.221696+0000","last_peered":"2024-05-08T05:49:21.221696+0000","last_clean":"2024-05-08T05:49:21.221696+0000","last_became_active":"2024-05-08T05:49:20.228426+0000","last_became_peered":"2024-05-08T05:49:20.228426+0000","last_unstale":"2024-05-08T05:49:21.221696+0000","last_undegraded":"2024-05-08T05:49:21.221696+0000","last_fullsized":"2024-05-08T05:49:21.221696+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:58.466547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740518+0000","last_change":"2024-05-08T05:49:20.239053+0000","last_active":"2024-05-08T05:49:21.740518+0000","last_peered":"2024-05-08T05:49:21.740518+0000","last_clean":"2024-05-08T05:49:21.740518+0000","last_became_active":"2024-05-08T05:49:20.238726+0000","last_became_peered":"2024-05-08T05:49:20.238726+0000","last_unstale":"2024-05-08T05:49:21.740518+0000","last_undegraded":"2024-05-08T05:49:21.740518+0000","last_fullsized":"2024-05-08T05:49:21.740518+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:20.999287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712233+0000","last_change":"2024-05-08T05:49:20.235736+0000","last_active":"2024-05-08T05:49:21.712233+0000","last_peered":"2024-05-08T05:49:21.712233+0000","last_clean":"2024-05-08T05:49:21.712233+0000","last_became_active":"2024-05-08T05:49:20.235467+0000","last_became_peered":"2024-05-08T05:49:20.235467+0000","last_unstale":"2024-05-08T05:49:21.712233+0000","last_undegraded":"2024-05-08T05:49:21.712233+0000","last_fullsized":"2024-05-08T05:49:21.712233+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:04:11.748952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220519+0000","last_change":"2024-05-08T05:49:20.226786+0000","last_active":"2024-05-08T05:49:21.220519+0000","last_peered":"2024-05-08T05:49:21.220519+0000","last_clean":"2024-05-08T05:49:21.220519+0000","last_became_active":"2024-05-08T05:49:20.223580+0000","last_became_peered":"2024-05-08T05:49:20.223580+0000","last_unstale":"2024-05-08T05:49:21.220519+0000","last_undegraded":"2024-05-08T05:49:21.220519+0000","last_fullsized":"2024-05-08T05:49:21.220519+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:32:33.705132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"103'4","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740566+0000","last_change":"2024-05-08T05:49:19.226689+0000","last_active":"2024-05-08T05:49:21.740566+0000","last_peered":"2024-05-08T05:49:21.740566+0000","last_clean":"2024-05-08T05:49:21.740566+0000","last_became_active":"2024-05-08T05:49:19.226431+0000","last_became_peered":"2024-05-08T05:49:19.226431+0000","last_unstale":"2024-05-08T05:49:21.740566+0000","last_undegraded":"2024-05-08T05:49:21.740566+0000","last_fullsized":"2024-05-08T05:49:21.740566+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:03.552401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712265+0000","last_change":"2024-05-08T05:49:20.235706+0000","last_active":"2024-05-08T05:49:21.712265+0000","last_peered":"2024-05-08T05:49:21.712265+0000","last_clean":"2024-05-08T05:49:21.712265+0000","last_became_active":"2024-05-08T05:49:20.235443+0000","last_became_peered":"2024-05-08T05:49:20.235443+0000","last_unstale":"2024-05-08T05:49:21.712265+0000","last_undegraded":"2024-05-08T05:49:21.712265+0000","last_fullsized":"2024-05-08T05:49:21.712265+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:03:14.991159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712275+0000","last_change":"2024-05-08T05:49:19.225382+0000","last_active":"2024-05-08T05:49:21.712275+0000","last_peered":"2024-05-08T05:49:21.712275+0000","last_clean":"2024-05-08T05:49:21.712275+0000","last_became_active":"2024-05-08T05:49:19.225112+0000","last_became_peered":"2024-05-08T05:49:19.225112+0000","last_unstale":"2024-05-08T05:49:21.712275+0000","last_undegraded":"2024-05-08T05:49:21.712275+0000","last_fullsized":"2024-05-08T05:49:21.712275+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:38:16.153929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222069+0000","last_change":"2024-05-08T05:49:20.220318+0000","last_active":"2024-05-08T05:49:21.222069+0000","last_peered":"2024-05-08T05:49:21.222069+0000","last_clean":"2024-05-08T05:49:21.222069+0000","last_became_active":"2024-05-08T05:49:20.217950+0000","last_became_peered":"2024-05-08T05:49:20.217950+0000","last_unstale":"2024-05-08T05:49:21.222069+0000","last_undegraded":"2024-05-08T05:49:21.222069+0000","last_fullsized":"2024-05-08T05:49:21.222069+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:45.720876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222167+0000","last_change":"2024-05-08T05:49:19.217546+0000","last_active":"2024-05-08T05:49:21.222167+0000","last_peered":"2024-05-08T05:49:21.222167+0000","last_clean":"2024-05-08T05:49:21.222167+0000","last_became_active":"2024-05-08T05:49:19.217253+0000","last_became_peered":"2024-05-08T05:49:19.217253+0000","last_unstale":"2024-05-08T05:49:21.222167+0000","last_undegraded":"2024-05-08T05:49:21.222167+0000","last_fullsized":"2024-05-08T05:49:21.222167+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:06:33.415067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740912+0000","last_change":"2024-05-08T05:49:20.231417+0000","last_active":"2024-05-08T05:49:21.740912+0000","last_peered":"2024-05-08T05:49:21.740912+0000","last_clean":"2024-05-08T05:49:21.740912+0000","last_became_active":"2024-05-08T05:49:20.227160+0000","last_became_peered":"2024-05-08T05:49:20.227160+0000","last_unstale":"2024-05-08T05:49:21.740912+0000","last_undegraded":"2024-05-08T05:49:21.740912+0000","last_fullsized":"2024-05-08T05:49:21.740912+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:22.685457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740857+0000","last_change":"2024-05-08T05:49:19.223093+0000","last_active":"2024-05-08T05:49:21.740857+0000","last_peered":"2024-05-08T05:49:21.740857+0000","last_clean":"2024-05-08T05:49:21.740857+0000","last_became_active":"2024-05-08T05:49:19.222791+0000","last_became_peered":"2024-05-08T05:49:19.222791+0000","last_unstale":"2024-05-08T05:49:21.740857+0000","last_undegraded":"2024-05-08T05:49:21.740857+0000","last_fullsized":"2024-05-08T05:49:21.740857+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:39:02.674771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741089+0000","last_change":"2024-05-08T05:49:20.229318+0000","last_active":"2024-05-08T05:49:21.741089+0000","last_peered":"2024-05-08T05:49:21.741089+0000","last_clean":"2024-05-08T05:49:21.741089+0000","last_became_active":"2024-05-08T05:49:20.229024+0000","last_became_peered":"2024-05-08T05:49:20.229024+0000","last_unstale":"2024-05-08T05:49:21.741089+0000","last_undegraded":"2024-05-08T05:49:21.741089+0000","last_fullsized":"2024-05-08T05:49:21.741089+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:59:57.323293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741131+0000","last_change":"2024-05-08T05:49:19.224342+0000","last_active":"2024-05-08T05:49:21.741131+0000","last_peered":"2024-05-08T05:49:21.741131+0000","last_clean":"2024-05-08T05:49:21.741131+0000","last_became_active":"2024-05-08T05:49:19.224020+0000","last_became_peered":"2024-05-08T05:49:19.224020+0000","last_unstale":"2024-05-08T05:49:21.741131+0000","last_undegraded":"2024-05-08T05:49:21.741131+0000","last_fullsized":"2024-05-08T05:49:21.741131+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:00:58.026398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220824+0000","last_change":"2024-05-08T05:49:20.224834+0000","last_active":"2024-05-08T05:49:21.220824+0000","last_peered":"2024-05-08T05:49:21.220824+0000","last_clean":"2024-05-08T05:49:21.220824+0000","last_became_active":"2024-05-08T05:49:20.224174+0000","last_became_peered":"2024-05-08T05:49:20.224174+0000","last_unstale":"2024-05-08T05:49:21.220824+0000","last_undegraded":"2024-05-08T05:49:21.220824+0000","last_fullsized":"2024-05-08T05:49:21.220824+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:49:23.007222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740317+0000","last_change":"2024-05-08T05:49:19.219998+0000","last_active":"2024-05-08T05:49:21.740317+0000","last_peered":"2024-05-08T05:49:21.740317+0000","last_clean":"2024-05-08T05:49:21.740317+0000","last_became_active":"2024-05-08T05:49:19.219620+0000","last_became_peered":"2024-05-08T05:49:19.219620+0000","last_unstale":"2024-05-08T05:49:21.740317+0000","last_undegraded":"2024-05-08T05:49:21.740317+0000","last_fullsized":"2024-05-08T05:49:21.740317+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:40:47.136337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741866+0000","last_change":"2024-05-08T05:49:20.236304+0000","last_active":"2024-05-08T05:49:21.741866+0000","last_peered":"2024-05-08T05:49:21.741866+0000","last_clean":"2024-05-08T05:49:21.741866+0000","last_became_active":"2024-05-08T05:49:20.235963+0000","last_became_peered":"2024-05-08T05:49:20.235963+0000","last_unstale":"2024-05-08T05:49:21.741866+0000","last_undegraded":"2024-05-08T05:49:21.741866+0000","last_fullsized":"2024-05-08T05:49:21.741866+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:55:24.707275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.221673+0000","last_change":"2024-05-08T05:49:19.217561+0000","last_active":"2024-05-08T05:49:21.221673+0000","last_peered":"2024-05-08T05:49:21.221673+0000","last_clean":"2024-05-08T05:49:21.221673+0000","last_became_active":"2024-05-08T05:49:19.217233+0000","last_became_peered":"2024-05-08T05:49:19.217233+0000","last_unstale":"2024-05-08T05:49:21.221673+0000","last_undegraded":"2024-05-08T05:49:21.221673+0000","last_fullsized":"2024-05-08T05:49:21.221673+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:39:02.674771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740504+0000","last_change":"2024-05-08T05:49:20.239988+0000","last_active":"2024-05-08T05:49:21.740504+0000","last_peered":"2024-05-08T05:49:21.740504+0000","last_clean":"2024-05-08T05:49:21.740504+0000","last_became_active":"2024-05-08T05:49:20.239698+0000","last_became_peered":"2024-05-08T05:49:20.239698+0000","last_unstale":"2024-05-08T05:49:21.740504+0000","last_undegraded":"2024-05-08T05:49:21.740504+0000","last_fullsized":"2024-05-08T05:49:21.740504+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:55:30.372059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.739885+0000","last_change":"2024-05-08T05:49:19.219609+0000","last_active":"2024-05-08T05:49:21.739885+0000","last_peered":"2024-05-08T05:49:21.739885+0000","last_clean":"2024-05-08T05:49:21.739885+0000","last_became_active":"2024-05-08T05:49:19.219336+0000","last_became_peered":"2024-05-08T05:49:19.219336+0000","last_unstale":"2024-05-08T05:49:21.739885+0000","last_undegraded":"2024-05-08T05:49:21.739885+0000","last_fullsized":"2024-05-08T05:49:21.739885+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:26:26.893270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220290+0000","last_change":"2024-05-08T05:49:20.241346+0000","last_active":"2024-05-08T05:49:21.220290+0000","last_peered":"2024-05-08T05:49:21.220290+0000","last_clean":"2024-05-08T05:49:21.220290+0000","last_became_active":"2024-05-08T05:49:20.241001+0000","last_became_peered":"2024-05-08T05:49:20.241001+0000","last_unstale":"2024-05-08T05:49:21.220290+0000","last_undegraded":"2024-05-08T05:49:21.220290+0000","last_fullsized":"2024-05-08T05:49:21.220290+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:17:51.356349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.222388+0000","last_change":"2024-05-08T05:49:19.217501+0000","last_active":"2024-05-08T05:49:21.222388+0000","last_peered":"2024-05-08T05:49:21.222388+0000","last_clean":"2024-05-08T05:49:21.222388+0000","last_became_active":"2024-05-08T05:49:19.217160+0000","last_became_peered":"2024-05-08T05:49:19.217160+0000","last_unstale":"2024-05-08T05:49:21.222388+0000","last_undegraded":"2024-05-08T05:49:21.222388+0000","last_fullsized":"2024-05-08T05:49:21.222388+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:40:47.136337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740750+0000","last_change":"2024-05-08T05:49:20.225909+0000","last_active":"2024-05-08T05:49:21.740750+0000","last_peered":"2024-05-08T05:49:21.740750+0000","last_clean":"2024-05-08T05:49:21.740750+0000","last_became_active":"2024-05-08T05:49:20.225617+0000","last_became_peered":"2024-05-08T05:49:20.225617+0000","last_unstale":"2024-05-08T05:49:21.740750+0000","last_undegraded":"2024-05-08T05:49:21.740750+0000","last_fullsized":"2024-05-08T05:49:21.740750+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:20.999287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220357+0000","last_change":"2024-05-08T05:49:19.211783+0000","last_active":"2024-05-08T05:49:21.220357+0000","last_peered":"2024-05-08T05:49:21.220357+0000","last_clean":"2024-05-08T05:49:21.220357+0000","last_became_active":"2024-05-08T05:49:19.211495+0000","last_became_peered":"2024-05-08T05:49:19.211495+0000","last_unstale":"2024-05-08T05:49:21.220357+0000","last_undegraded":"2024-05-08T05:49:21.220357+0000","last_fullsized":"2024-05-08T05:49:21.220357+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:33:36.204034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712197+0000","last_change":"2024-05-08T05:49:20.232202+0000","last_active":"2024-05-08T05:49:21.712197+0000","last_peered":"2024-05-08T05:49:21.712197+0000","last_clean":"2024-05-08T05:49:21.712197+0000","last_became_active":"2024-05-08T05:49:20.231919+0000","last_became_peered":"2024-05-08T05:49:20.231919+0000","last_unstale":"2024-05-08T05:49:21.712197+0000","last_undegraded":"2024-05-08T05:49:21.712197+0000","last_fullsized":"2024-05-08T05:49:21.712197+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:30:47.129884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"103'3","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740334+0000","last_change":"2024-05-08T05:49:19.220761+0000","last_active":"2024-05-08T05:49:21.740334+0000","last_peered":"2024-05-08T05:49:21.740334+0000","last_clean":"2024-05-08T05:49:21.740334+0000","last_became_active":"2024-05-08T05:49:19.220244+0000","last_became_peered":"2024-05-08T05:49:19.220244+0000","last_unstale":"2024-05-08T05:49:21.740334+0000","last_undegraded":"2024-05-08T05:49:21.740334+0000","last_fullsized":"2024-05-08T05:49:21.740334+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:00:43.392678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740671+0000","last_change":"2024-05-08T05:49:20.235723+0000","last_active":"2024-05-08T05:49:21.740671+0000","last_peered":"2024-05-08T05:49:21.740671+0000","last_clean":"2024-05-08T05:49:21.740671+0000","last_became_active":"2024-05-08T05:49:20.235051+0000","last_became_peered":"2024-05-08T05:49:20.235051+0000","last_unstale":"2024-05-08T05:49:21.740671+0000","last_undegraded":"2024-05-08T05:49:21.740671+0000","last_fullsized":"2024-05-08T05:49:21.740671+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:45.720876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"103'6","reported_seq":20,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741184+0000","last_change":"2024-05-08T05:49:19.220558+0000","last_active":"2024-05-08T05:49:21.741184+0000","last_peered":"2024-05-08T05:49:21.741184+0000","last_clean":"2024-05-08T05:49:21.741184+0000","last_became_active":"2024-05-08T05:49:19.220251+0000","last_became_peered":"2024-05-08T05:49:19.220251+0000","last_unstale":"2024-05-08T05:49:21.741184+0000","last_undegraded":"2024-05-08T05:49:21.741184+0000","last_fullsized":"2024-05-08T05:49:21.741184+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:06:33.415067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740567+0000","last_change":"2024-05-08T05:49:20.229666+0000","last_active":"2024-05-08T05:49:21.740567+0000","last_peered":"2024-05-08T05:49:21.740567+0000","last_clean":"2024-05-08T05:49:21.740567+0000","last_became_active":"2024-05-08T05:49:20.227506+0000","last_became_peered":"2024-05-08T05:49:20.227506+0000","last_unstale":"2024-05-08T05:49:21.740567+0000","last_undegraded":"2024-05-08T05:49:21.740567+0000","last_fullsized":"2024-05-08T05:49:21.740567+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:28:45.603541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740730+0000","last_change":"2024-05-08T05:49:19.225759+0000","last_active":"2024-05-08T05:49:21.740730+0000","last_peered":"2024-05-08T05:49:21.740730+0000","last_clean":"2024-05-08T05:49:21.740730+0000","last_became_active":"2024-05-08T05:49:19.225054+0000","last_became_peered":"2024-05-08T05:49:19.225054+0000","last_unstale":"2024-05-08T05:49:21.740730+0000","last_undegraded":"2024-05-08T05:49:21.740730+0000","last_fullsized":"2024-05-08T05:49:21.740730+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:06:33.415067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741384+0000","last_change":"2024-05-08T05:49:20.225107+0000","last_active":"2024-05-08T05:49:21.741384+0000","last_peered":"2024-05-08T05:49:21.741384+0000","last_clean":"2024-05-08T05:49:21.741384+0000","last_became_active":"2024-05-08T05:49:20.224832+0000","last_became_peered":"2024-05-08T05:49:20.224832+0000","last_unstale":"2024-05-08T05:49:21.741384+0000","last_undegraded":"2024-05-08T05:49:21.741384+0000","last_fullsized":"2024-05-08T05:49:21.741384+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:04.560434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740682+0000","last_change":"2024-05-08T05:49:19.220845+0000","last_active":"2024-05-08T05:49:21.740682+0000","last_peered":"2024-05-08T05:49:21.740682+0000","last_clean":"2024-05-08T05:49:21.740682+0000","last_became_active":"2024-05-08T05:49:19.220519+0000","last_became_peered":"2024-05-08T05:49:19.220519+0000","last_unstale":"2024-05-08T05:49:21.740682+0000","last_undegraded":"2024-05-08T05:49:21.740682+0000","last_fullsized":"2024-05-08T05:49:21.740682+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:02:04.763302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712390+0000","last_change":"2024-05-08T05:49:20.234140+0000","last_active":"2024-05-08T05:49:21.712390+0000","last_peered":"2024-05-08T05:49:21.712390+0000","last_clean":"2024-05-08T05:49:21.712390+0000","last_became_active":"2024-05-08T05:49:20.233639+0000","last_became_peered":"2024-05-08T05:49:20.233639+0000","last_unstale":"2024-05-08T05:49:21.712390+0000","last_undegraded":"2024-05-08T05:49:21.712390+0000","last_fullsized":"2024-05-08T05:49:21.712390+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:52:21.870490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712126+0000","last_change":"2024-05-08T05:49:19.226502+0000","last_active":"2024-05-08T05:49:21.712126+0000","last_peered":"2024-05-08T05:49:21.712126+0000","last_clean":"2024-05-08T05:49:21.712126+0000","last_became_active":"2024-05-08T05:49:19.226217+0000","last_became_peered":"2024-05-08T05:49:19.226217+0000","last_unstale":"2024-05-08T05:49:21.712126+0000","last_undegraded":"2024-05-08T05:49:21.712126+0000","last_fullsized":"2024-05-08T05:49:21.712126+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:26:51.894930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220622+0000","last_change":"2024-05-08T05:49:20.228556+0000","last_active":"2024-05-08T05:49:21.220622+0000","last_peered":"2024-05-08T05:49:21.220622+0000","last_clean":"2024-05-08T05:49:21.220622+0000","last_became_active":"2024-05-08T05:49:20.228242+0000","last_became_peered":"2024-05-08T05:49:20.228242+0000","last_unstale":"2024-05-08T05:49:21.220622+0000","last_undegraded":"2024-05-08T05:49:21.220622+0000","last_fullsized":"2024-05-08T05:49:21.220622+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:44.880570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"103'8","reported_seq":22,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.221517+0000","last_change":"2024-05-08T05:49:19.221739+0000","last_active":"2024-05-08T05:49:21.221517+0000","last_peered":"2024-05-08T05:49:21.221517+0000","last_clean":"2024-05-08T05:49:21.221517+0000","last_became_active":"2024-05-08T05:49:19.221466+0000","last_became_peered":"2024-05-08T05:49:19.221466+0000","last_unstale":"2024-05-08T05:49:21.221517+0000","last_undegraded":"2024-05-08T05:49:21.221517+0000","last_fullsized":"2024-05-08T05:49:21.221517+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:01.127846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712529+0000","last_change":"2024-05-08T05:49:20.231557+0000","last_active":"2024-05-08T05:49:21.712529+0000","last_peered":"2024-05-08T05:49:21.712529+0000","last_clean":"2024-05-08T05:49:21.712529+0000","last_became_active":"2024-05-08T05:49:20.231028+0000","last_became_peered":"2024-05-08T05:49:20.231028+0000","last_unstale":"2024-05-08T05:49:21.712529+0000","last_undegraded":"2024-05-08T05:49:21.712529+0000","last_fullsized":"2024-05-08T05:49:21.712529+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:59:56.161183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741817+0000","last_change":"2024-05-08T05:49:19.219459+0000","last_active":"2024-05-08T05:49:21.741817+0000","last_peered":"2024-05-08T05:49:21.741817+0000","last_clean":"2024-05-08T05:49:21.741817+0000","last_became_active":"2024-05-08T05:49:19.219157+0000","last_became_peered":"2024-05-08T05:49:19.219157+0000","last_unstale":"2024-05-08T05:49:21.741817+0000","last_undegraded":"2024-05-08T05:49:21.741817+0000","last_fullsized":"2024-05-08T05:49:21.741817+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:57.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741175+0000","last_change":"2024-05-08T05:49:20.238812+0000","last_active":"2024-05-08T05:49:21.741175+0000","last_peered":"2024-05-08T05:49:21.741175+0000","last_clean":"2024-05-08T05:49:21.741175+0000","last_became_active":"2024-05-08T05:49:20.238456+0000","last_became_peered":"2024-05-08T05:49:20.238456+0000","last_unstale":"2024-05-08T05:49:21.741175+0000","last_undegraded":"2024-05-08T05:49:21.741175+0000","last_fullsized":"2024-05-08T05:49:21.741175+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:26:27.901303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741280+0000","last_change":"2024-05-08T05:49:19.221812+0000","last_active":"2024-05-08T05:49:21.741280+0000","last_peered":"2024-05-08T05:49:21.741280+0000","last_clean":"2024-05-08T05:49:21.741280+0000","last_became_active":"2024-05-08T05:49:19.221501+0000","last_became_peered":"2024-05-08T05:49:19.221501+0000","last_unstale":"2024-05-08T05:49:21.741280+0000","last_undegraded":"2024-05-08T05:49:21.741280+0000","last_fullsized":"2024-05-08T05:49:21.741280+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:55:23.699243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740514+0000","last_change":"2024-05-08T05:49:20.235973+0000","last_active":"2024-05-08T05:49:21.740514+0000","last_peered":"2024-05-08T05:49:21.740514+0000","last_clean":"2024-05-08T05:49:21.740514+0000","last_became_active":"2024-05-08T05:49:20.235442+0000","last_became_peered":"2024-05-08T05:49:20.235442+0000","last_unstale":"2024-05-08T05:49:21.740514+0000","last_undegraded":"2024-05-08T05:49:21.740514+0000","last_fullsized":"2024-05-08T05:49:21.740514+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:58.466547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740680+0000","last_change":"2024-05-08T05:49:19.222187+0000","last_active":"2024-05-08T05:49:21.740680+0000","last_peered":"2024-05-08T05:49:21.740680+0000","last_clean":"2024-05-08T05:49:21.740680+0000","last_became_active":"2024-05-08T05:49:19.221863+0000","last_became_peered":"2024-05-08T05:49:19.221863+0000","last_unstale":"2024-05-08T05:49:21.740680+0000","last_undegraded":"2024-05-08T05:49:21.740680+0000","last_fullsized":"2024-05-08T05:49:21.740680+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:57.962989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220349+0000","last_change":"2024-05-08T05:49:20.237821+0000","last_active":"2024-05-08T05:49:21.220349+0000","last_peered":"2024-05-08T05:49:21.220349+0000","last_clean":"2024-05-08T05:49:21.220349+0000","last_became_active":"2024-05-08T05:49:20.237417+0000","last_became_peered":"2024-05-08T05:49:20.237417+0000","last_unstale":"2024-05-08T05:49:21.220349+0000","last_undegraded":"2024-05-08T05:49:21.220349+0000","last_fullsized":"2024-05-08T05:49:21.220349+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:06:50.147335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712469+0000","last_change":"2024-05-08T05:49:19.222219+0000","last_active":"2024-05-08T05:49:21.712469+0000","last_peered":"2024-05-08T05:49:21.712469+0000","last_clean":"2024-05-08T05:49:21.712469+0000","last_became_active":"2024-05-08T05:49:19.221938+0000","last_became_peered":"2024-05-08T05:49:19.221938+0000","last_unstale":"2024-05-08T05:49:21.712469+0000","last_undegraded":"2024-05-08T05:49:21.712469+0000","last_fullsized":"2024-05-08T05:49:21.712469+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:11.598891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220747+0000","last_change":"2024-05-08T05:49:20.225075+0000","last_active":"2024-05-08T05:49:21.220747+0000","last_peered":"2024-05-08T05:49:21.220747+0000","last_clean":"2024-05-08T05:49:21.220747+0000","last_became_active":"2024-05-08T05:49:20.224537+0000","last_became_peered":"2024-05-08T05:49:20.224537+0000","last_unstale":"2024-05-08T05:49:21.220747+0000","last_undegraded":"2024-05-08T05:49:21.220747+0000","last_fullsized":"2024-05-08T05:49:21.220747+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:10:17.796692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.711959+0000","last_change":"2024-05-08T05:49:19.226655+0000","last_active":"2024-05-08T05:49:21.711959+0000","last_peered":"2024-05-08T05:49:21.711959+0000","last_clean":"2024-05-08T05:49:21.711959+0000","last_became_active":"2024-05-08T05:49:19.226385+0000","last_became_peered":"2024-05-08T05:49:19.226385+0000","last_unstale":"2024-05-08T05:49:21.711959+0000","last_undegraded":"2024-05-08T05:49:21.711959+0000","last_fullsized":"2024-05-08T05:49:21.711959+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:23:09.778236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740421+0000","last_change":"2024-05-08T05:49:20.235440+0000","last_active":"2024-05-08T05:49:21.740421+0000","last_peered":"2024-05-08T05:49:21.740421+0000","last_clean":"2024-05-08T05:49:21.740421+0000","last_became_active":"2024-05-08T05:49:20.235109+0000","last_became_peered":"2024-05-08T05:49:20.235109+0000","last_unstale":"2024-05-08T05:49:21.740421+0000","last_undegraded":"2024-05-08T05:49:21.740421+0000","last_fullsized":"2024-05-08T05:49:21.740421+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:08:25.723528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740617+0000","last_change":"2024-05-08T05:49:19.218899+0000","last_active":"2024-05-08T05:49:21.740617+0000","last_peered":"2024-05-08T05:49:21.740617+0000","last_clean":"2024-05-08T05:49:21.740617+0000","last_became_active":"2024-05-08T05:49:19.218594+0000","last_became_peered":"2024-05-08T05:49:19.218594+0000","last_unstale":"2024-05-08T05:49:21.740617+0000","last_undegraded":"2024-05-08T05:49:21.740617+0000","last_fullsized":"2024-05-08T05:49:21.740617+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:01.127846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712522+0000","last_change":"2024-05-08T05:49:20.232503+0000","last_active":"2024-05-08T05:49:21.712522+0000","last_peered":"2024-05-08T05:49:21.712522+0000","last_clean":"2024-05-08T05:49:21.712522+0000","last_became_active":"2024-05-08T05:49:20.232220+0000","last_became_peered":"2024-05-08T05:49:20.232220+0000","last_unstale":"2024-05-08T05:49:21.712522+0000","last_undegraded":"2024-05-08T05:49:21.712522+0000","last_fullsized":"2024-05-08T05:49:21.712522+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:27:41.729361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712472+0000","last_change":"2024-05-08T05:49:19.225879+0000","last_active":"2024-05-08T05:49:21.712472+0000","last_peered":"2024-05-08T05:49:21.712472+0000","last_clean":"2024-05-08T05:49:21.712472+0000","last_became_active":"2024-05-08T05:49:19.225603+0000","last_became_peered":"2024-05-08T05:49:19.225603+0000","last_unstale":"2024-05-08T05:49:21.712472+0000","last_undegraded":"2024-05-08T05:49:21.712472+0000","last_fullsized":"2024-05-08T05:49:21.712472+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:36:23.099004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220321+0000","last_change":"2024-05-08T05:49:20.227026+0000","last_active":"2024-05-08T05:49:21.220321+0000","last_peered":"2024-05-08T05:49:21.220321+0000","last_clean":"2024-05-08T05:49:21.220321+0000","last_became_active":"2024-05-08T05:49:20.226533+0000","last_became_peered":"2024-05-08T05:49:20.226533+0000","last_unstale":"2024-05-08T05:49:21.220321+0000","last_undegraded":"2024-05-08T05:49:21.220321+0000","last_fullsized":"2024-05-08T05:49:21.220321+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:56:54.954385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740487+0000","last_change":"2024-05-08T05:49:19.225433+0000","last_active":"2024-05-08T05:49:21.740487+0000","last_peered":"2024-05-08T05:49:21.740487+0000","last_clean":"2024-05-08T05:49:21.740487+0000","last_became_active":"2024-05-08T05:49:19.225126+0000","last_became_peered":"2024-05-08T05:49:19.225126+0000","last_unstale":"2024-05-08T05:49:21.740487+0000","last_undegraded":"2024-05-08T05:49:21.740487+0000","last_fullsized":"2024-05-08T05:49:21.740487+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:01.127846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.221654+0000","last_change":"2024-05-08T05:49:20.222133+0000","last_active":"2024-05-08T05:49:21.221654+0000","last_peered":"2024-05-08T05:49:21.221654+0000","last_clean":"2024-05-08T05:49:21.221654+0000","last_became_active":"2024-05-08T05:49:20.221812+0000","last_became_peered":"2024-05-08T05:49:20.221812+0000","last_unstale":"2024-05-08T05:49:21.221654+0000","last_undegraded":"2024-05-08T05:49:21.221654+0000","last_fullsized":"2024-05-08T05:49:21.221654+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:42.929693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"104'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.220391+0000","last_change":"2024-05-08T05:49:19.221744+0000","last_active":"2024-05-08T05:49:21.220391+0000","last_peered":"2024-05-08T05:49:21.220391+0000","last_clean":"2024-05-08T05:49:21.220391+0000","last_became_active":"2024-05-08T05:49:19.221395+0000","last_became_peered":"2024-05-08T05:49:19.221395+0000","last_unstale":"2024-05-08T05:49:21.220391+0000","last_undegraded":"2024-05-08T05:49:21.220391+0000","last_fullsized":"2024-05-08T05:49:21.220391+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:01:13.200266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740671+0000","last_change":"2024-05-08T05:49:20.231747+0000","last_active":"2024-05-08T05:49:21.740671+0000","last_peered":"2024-05-08T05:49:21.740671+0000","last_clean":"2024-05-08T05:49:21.740671+0000","last_became_active":"2024-05-08T05:49:20.231483+0000","last_became_peered":"2024-05-08T05:49:20.231483+0000","last_unstale":"2024-05-08T05:49:21.740671+0000","last_undegraded":"2024-05-08T05:49:21.740671+0000","last_fullsized":"2024-05-08T05:49:21.740671+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:54.490928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741491+0000","last_change":"2024-05-08T05:49:19.222418+0000","last_active":"2024-05-08T05:49:21.741491+0000","last_peered":"2024-05-08T05:49:21.741491+0000","last_clean":"2024-05-08T05:49:21.741491+0000","last_became_active":"2024-05-08T05:49:19.222125+0000","last_became_peered":"2024-05-08T05:49:19.222125+0000","last_unstale":"2024-05-08T05:49:21.741491+0000","last_undegraded":"2024-05-08T05:49:21.741491+0000","last_fullsized":"2024-05-08T05:49:21.741491+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:40:47.136337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740889+0000","last_change":"2024-05-08T05:49:20.230762+0000","last_active":"2024-05-08T05:49:21.740889+0000","last_peered":"2024-05-08T05:49:21.740889+0000","last_clean":"2024-05-08T05:49:21.740889+0000","last_became_active":"2024-05-08T05:49:20.228489+0000","last_became_peered":"2024-05-08T05:49:20.228489+0000","last_unstale":"2024-05-08T05:49:21.740889+0000","last_undegraded":"2024-05-08T05:49:21.740889+0000","last_fullsized":"2024-05-08T05:49:21.740889+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:20.999287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740940+0000","last_change":"2024-05-08T05:49:19.221630+0000","last_active":"2024-05-08T05:49:21.740940+0000","last_peered":"2024-05-08T05:49:21.740940+0000","last_clean":"2024-05-08T05:49:21.740940+0000","last_became_active":"2024-05-08T05:49:19.221299+0000","last_became_peered":"2024-05-08T05:49:19.221299+0000","last_unstale":"2024-05-08T05:49:21.740940+0000","last_undegraded":"2024-05-08T05:49:21.740940+0000","last_fullsized":"2024-05-08T05:49:21.740940+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:53:45.139823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712268+0000","last_change":"2024-05-08T05:49:20.235059+0000","last_active":"2024-05-08T05:49:21.712268+0000","last_peered":"2024-05-08T05:49:21.712268+0000","last_clean":"2024-05-08T05:49:21.712268+0000","last_became_active":"2024-05-08T05:49:20.234561+0000","last_became_peered":"2024-05-08T05:49:20.234561+0000","last_unstale":"2024-05-08T05:49:21.712268+0000","last_undegraded":"2024-05-08T05:49:21.712268+0000","last_fullsized":"2024-05-08T05:49:21.712268+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:00:16.941782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.221905+0000","last_change":"2024-05-08T05:49:19.215532+0000","last_active":"2024-05-08T05:49:21.221905+0000","last_peered":"2024-05-08T05:49:21.221905+0000","last_clean":"2024-05-08T05:49:21.221905+0000","last_became_active":"2024-05-08T05:49:19.215217+0000","last_became_peered":"2024-05-08T05:49:19.215217+0000","last_unstale":"2024-05-08T05:49:21.221905+0000","last_undegraded":"2024-05-08T05:49:21.221905+0000","last_fullsized":"2024-05-08T05:49:21.221905+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:53:45.139823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712742+0000","last_change":"2024-05-08T05:49:20.232929+0000","last_active":"2024-05-08T05:49:21.712742+0000","last_peered":"2024-05-08T05:49:21.712742+0000","last_clean":"2024-05-08T05:49:21.712742+0000","last_became_active":"2024-05-08T05:49:20.232635+0000","last_became_peered":"2024-05-08T05:49:20.232635+0000","last_unstale":"2024-05-08T05:49:21.712742+0000","last_undegraded":"2024-05-08T05:49:21.712742+0000","last_fullsized":"2024-05-08T05:49:21.712742+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:47:01.494382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712087+0000","last_change":"2024-05-08T05:49:19.222351+0000","last_active":"2024-05-08T05:49:21.712087+0000","last_peered":"2024-05-08T05:49:21.712087+0000","last_clean":"2024-05-08T05:49:21.712087+0000","last_became_active":"2024-05-08T05:49:19.222092+0000","last_became_peered":"2024-05-08T05:49:19.222092+0000","last_unstale":"2024-05-08T05:49:21.712087+0000","last_undegraded":"2024-05-08T05:49:21.712087+0000","last_fullsized":"2024-05-08T05:49:21.712087+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:22:40.427897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741491+0000","last_change":"2024-05-08T05:49:20.235238+0000","last_active":"2024-05-08T05:49:21.741491+0000","last_peered":"2024-05-08T05:49:21.741491+0000","last_clean":"2024-05-08T05:49:21.741491+0000","last_became_active":"2024-05-08T05:49:20.234933+0000","last_became_peered":"2024-05-08T05:49:20.234933+0000","last_unstale":"2024-05-08T05:49:21.741491+0000","last_undegraded":"2024-05-08T05:49:21.741491+0000","last_fullsized":"2024-05-08T05:49:21.741491+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:59:57.323293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740680+0000","last_change":"2024-05-08T05:49:19.219966+0000","last_active":"2024-05-08T05:49:21.740680+0000","last_peered":"2024-05-08T05:49:21.740680+0000","last_clean":"2024-05-08T05:49:21.740680+0000","last_became_active":"2024-05-08T05:49:19.219555+0000","last_became_peered":"2024-05-08T05:49:19.219555+0000","last_unstale":"2024-05-08T05:49:21.740680+0000","last_undegraded":"2024-05-08T05:49:21.740680+0000","last_fullsized":"2024-05-08T05:49:21.740680+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:53:45.139823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.741173+0000","last_change":"2024-05-08T05:49:20.230394+0000","last_active":"2024-05-08T05:49:21.741173+0000","last_peered":"2024-05-08T05:49:21.741173+0000","last_clean":"2024-05-08T05:49:21.741173+0000","last_became_active":"2024-05-08T05:49:20.229931+0000","last_became_peered":"2024-05-08T05:49:20.229931+0000","last_unstale":"2024-05-08T05:49:21.741173+0000","last_undegraded":"2024-05-08T05:49:21.741173+0000","last_fullsized":"2024-05-08T05:49:21.741173+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:26:27.901303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712584+0000","last_change":"2024-05-08T05:49:19.224009+0000","last_active":"2024-05-08T05:49:21.712584+0000","last_peered":"2024-05-08T05:49:21.712584+0000","last_clean":"2024-05-08T05:49:21.712584+0000","last_became_active":"2024-05-08T05:49:19.223694+0000","last_became_peered":"2024-05-08T05:49:19.223694+0000","last_unstale":"2024-05-08T05:49:21.712584+0000","last_undegraded":"2024-05-08T05:49:21.712584+0000","last_fullsized":"2024-05-08T05:49:21.712584+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:36:26.679637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740331+0000","last_change":"2024-05-08T05:49:20.224133+0000","last_active":"2024-05-08T05:49:21.740331+0000","last_peered":"2024-05-08T05:49:21.740331+0000","last_clean":"2024-05-08T05:49:21.740331+0000","last_became_active":"2024-05-08T05:49:20.223808+0000","last_became_peered":"2024-05-08T05:49:20.223808+0000","last_unstale":"2024-05-08T05:49:21.740331+0000","last_undegraded":"2024-05-08T05:49:21.740331+0000","last_fullsized":"2024-05-08T05:49:21.740331+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:19.196621+0000","last_clean_scrub_stamp":"2024-05-08T05:49:19.196621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:42.929693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.712441+0000","last_change":"2024-05-08T05:49:19.227947+0000","last_active":"2024-05-08T05:49:21.712441+0000","last_peered":"2024-05-08T05:49:21.712441+0000","last_clean":"2024-05-08T05:49:21.712441+0000","last_became_active":"2024-05-08T05:49:19.227650+0000","last_became_peered":"2024-05-08T05:49:19.227650+0000","last_unstale":"2024-05-08T05:49:21.712441+0000","last_undegraded":"2024-05-08T05:49:21.712441+0000","last_fullsized":"2024-05-08T05:49:21.712441+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:18.188589+0000","last_clean_scrub_stamp":"2024-05-08T05:49:18.188589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:15:07.219786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":264,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-05-08T05:49:21.740434+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:49:21.740434+0000","last_peered":"2024-05-08T05:49:21.740434+0000","last_clean":"2024-05-08T05:49:21.740434+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:49:21.740434+0000","last_undegraded":"2024-05-08T05:49:21.740434+0000","last_fullsized":"2024-05-08T05:49:21.740434+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542245,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1888256,"data_stored":1186811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542246,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1306624,"data_stored":598499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542245,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":596283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1294336,"data_stored":596375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574948,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":596865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574948,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":596317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":596317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574948,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1896448,"data_stored":1188353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:49:29.114 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T05:49:29.115 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T05:49:29.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.262+0000 7fe959c46640 1 Processor -- start 2024-05-08T05:49:29.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.262+0000 7fe959c46640 1 -- start start 2024-05-08T05:49:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.263+0000 7fe959c46640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9540b3f30 0x7fe9540d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.263+0000 7fe959c46640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9540b1c90 con 0x7fe9540d3cd0 2024-05-08T05:49:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.263+0000 7fe959c46640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9540d0a50 con 0x7fe95415f380 2024-05-08T05:49:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.263+0000 7fe959c46640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe9540d0d30 con 0x7fe9540b3f30 2024-05-08T05:49:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.263+0000 7fe94affd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe9540d3cd0 0x7fe9540d15f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49894/0 (socket says 172.21.15.45:49894) 2024-05-08T05:49:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.263+0000 7fe94affd640 1 -- 172.21.15.45:0/3634132383 learned_addr learned my addr 172.21.15.45:0/3634132383 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.263+0000 7fe953fff640 1 --2- 172.21.15.45:0/3634132383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9540b3f30 0x7fe9540d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.263+0000 7fe952ffd640 1 -- 172.21.15.45:0/3634132383 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1460140148 0 0) 0x7fe9540b1c90 con 0x7fe9540d3cd0 2024-05-08T05:49:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.263+0000 7fe952ffd640 1 -- 172.21.15.45:0/3634132383 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe940003520 con 0x7fe9540d3cd0 2024-05-08T05:49:29.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe952ffd640 1 -- 172.21.15.45:0/3634132383 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2586831820 0 0) 0x7fe940003520 con 0x7fe9540d3cd0 2024-05-08T05:49:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe952ffd640 1 -- 172.21.15.45:0/3634132383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9540b3f30 msgr2=0x7fe9540d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe952ffd640 1 --2- 172.21.15.45:0/3634132383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9540b3f30 0x7fe9540d1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe952ffd640 1 -- 172.21.15.45:0/3634132383 >> v1:172.21.15.63:6789/0 conn(0x7fe95415f380 legacy=0x7fe9540d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe952ffd640 1 -- 172.21.15.45:0/3634132383 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9540b1f00 con 0x7fe9540d3cd0 2024-05-08T05:49:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe952ffd640 1 -- 172.21.15.45:0/3634132383 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fe93c0016f0 con 0x7fe9540d3cd0 2024-05-08T05:49:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe952ffd640 1 -- 172.21.15.45:0/3634132383 >> v1:172.21.15.45:6789/0 conn(0x7fe9540d3cd0 legacy=0x7fe9540d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe952ffd640 1 --2- 172.21.15.45:0/3634132383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9400039e0 0x7fe940003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe952ffd640 1 --2- 172.21.15.45:0/3634132383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe940004800 0x7fe940004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe952ffd640 1 --2- 172.21.15.45:0/3634132383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9540b3f30 0x7fe9400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe952ffd640 1 -- 172.21.15.45:0/3634132383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe9540d0d30 con 0x7fe940004800 2024-05-08T05:49:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe952ffd640 1 -- 172.21.15.45:0/3634132383 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe94000ba70 con 0x7fe9400039e0 2024-05-08T05:49:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe952ffd640 1 -- 172.21.15.45:0/3634132383 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe94000bc70 con 0x7fe9540b3f30 2024-05-08T05:49:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe94affd640 1 --2- 172.21.15.45:0/3634132383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe940004800 0x7fe940004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe9537fe640 1 --2- 172.21.15.45:0/3634132383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9400039e0 0x7fe940003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe953fff640 1 --2- 172.21.15.45:0/3634132383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9540b3f30 0x7fe9400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:29.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe94affd640 1 -- 172.21.15.45:0/3634132383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9540b3f30 msgr2=0x7fe9400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe94affd640 1 --2- 172.21.15.45:0/3634132383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe9540b3f30 0x7fe9400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe94affd640 1 -- 172.21.15.45:0/3634132383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9400039e0 msgr2=0x7fe940003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe94affd640 1 --2- 172.21.15.45:0/3634132383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9400039e0 0x7fe940003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe94affd640 1 -- 172.21.15.45:0/3634132383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe940003520 con 0x7fe940004800 2024-05-08T05:49:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.264+0000 7fe9537fe640 1 --2- 172.21.15.45:0/3634132383 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe9400039e0 0x7fe940003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.265+0000 7fe94affd640 1 --2- 172.21.15.45:0/3634132383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe940004800 0x7fe940004c80 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00c2f0 tx=0x7fe940005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ca47f7d74fd384 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.265+0000 7fe952ffd640 1 -- 172.21.15.45:0/3634132383 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c007d90 con 0x7fe940004800 2024-05-08T05:49:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.265+0000 7fe952ffd640 1 -- 172.21.15.45:0/3634132383 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe93c005ad0 con 0x7fe940004800 2024-05-08T05:49:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.265+0000 7fe952ffd640 1 -- 172.21.15.45:0/3634132383 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c007070 con 0x7fe940004800 2024-05-08T05:49:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.266+0000 7fe959c46640 1 -- 172.21.15.45:0/3634132383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe940004800 msgr2=0x7fe940004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.266+0000 7fe959c46640 1 --2- 172.21.15.45:0/3634132383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe940004800 0x7fe940004c80 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00c2f0 tx=0x7fe940005830 comp rx=0 tx=0).stop 2024-05-08T05:49:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.266+0000 7fe953fff640 1 -- 172.21.15.45:0/3634132383 reap_dead start 2024-05-08T05:49:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.266+0000 7fe959c46640 1 -- 172.21.15.45:0/3634132383 shutdown_connections 2024-05-08T05:49:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.266+0000 7fe959c46640 1 -- 172.21.15.45:0/3634132383 >> 172.21.15.45:0/3634132383 conn(0x7fe9540d4fe0 msgr2=0x7fe9540d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.266+0000 7fe959c46640 1 -- 172.21.15.45:0/3634132383 shutdown_connections 2024-05-08T05:49:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.266+0000 7fe959c46640 1 -- 172.21.15.45:0/3634132383 wait complete. 2024-05-08T05:49:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.267+0000 7fe959c46640 1 Processor -- start 2024-05-08T05:49:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.267+0000 7fe959c46640 1 -- start start 2024-05-08T05:49:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.268+0000 7fe959c46640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe940004800 0x7fe9541f2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.268+0000 7fe959c46640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9540ca600 0x7fe9540caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.268+0000 7fe959c46640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe95415d4e0 0x7fe95415d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.268+0000 7fe959c46640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe9540d06e0 con 0x7fe9540ca600 2024-05-08T05:49:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.268+0000 7fe959c46640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe9540d1010 con 0x7fe940004800 2024-05-08T05:49:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.268+0000 7fe959c46640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe9540b3430 con 0x7fe95415d4e0 2024-05-08T05:49:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.268+0000 7fe94affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9540ca600 0x7fe9540caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.268+0000 7fe94affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9540ca600 0x7fe9540caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45486/0 (socket says 172.21.15.45:45486) 2024-05-08T05:49:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.268+0000 7fe94affd640 1 -- 172.21.15.45:0/473678358 learned_addr learned my addr 172.21.15.45:0/473678358 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.268+0000 7fe9537fe640 1 --2- 172.21.15.45:0/473678358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe940004800 0x7fe9541f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.268+0000 7fe953fff640 1 --2- 172.21.15.45:0/473678358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe95415d4e0 0x7fe95415d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.268+0000 7fe94affd640 1 -- 172.21.15.45:0/473678358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe95415d4e0 msgr2=0x7fe95415d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.268+0000 7fe94affd640 1 --2- 172.21.15.45:0/473678358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe95415d4e0 0x7fe95415d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.268+0000 7fe94affd640 1 -- 172.21.15.45:0/473678358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe940004800 msgr2=0x7fe9541f2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.268+0000 7fe94affd640 1 --2- 172.21.15.45:0/473678358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe940004800 0x7fe9541f2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.268+0000 7fe94affd640 1 -- 172.21.15.45:0/473678358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9541f4fb0 con 0x7fe9540ca600 2024-05-08T05:49:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.268+0000 7fe9537fe640 1 --2- 172.21.15.45:0/473678358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe940004800 0x7fe9541f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.269+0000 7fe94affd640 1 --2- 172.21.15.45:0/473678358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9540ca600 0x7fe9540caa80 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00c410 tx=0x7fe9540d0c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.269+0000 7fe950ff9640 1 -- 172.21.15.45:0/473678358 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c040020 con 0x7fe9540ca600 2024-05-08T05:49:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.269+0000 7fe950ff9640 1 -- 172.21.15.45:0/473678358 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe93c0014b0 con 0x7fe9540ca600 2024-05-08T05:49:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.269+0000 7fe959c46640 1 -- 172.21.15.45:0/473678358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9541f5290 con 0x7fe9540ca600 2024-05-08T05:49:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.269+0000 7fe950ff9640 1 -- 172.21.15.45:0/473678358 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c037040 con 0x7fe9540ca600 2024-05-08T05:49:29.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.269+0000 7fe959c46640 1 -- 172.21.15.45:0/473678358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9541f5850 con 0x7fe9540ca600 2024-05-08T05:49:29.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.270+0000 7fe959c46640 1 -- 172.21.15.45:0/473678358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe95415dea0 con 0x7fe9540ca600 2024-05-08T05:49:29.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.270+0000 7fe950ff9640 1 -- 172.21.15.45:0/473678358 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe93c043020 con 0x7fe9540ca600 2024-05-08T05:49:29.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.271+0000 7fe950ff9640 1 --2- 172.21.15.45:0/473678358 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe9340762c0 0x7fe934078780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:29.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.271+0000 7fe950ff9640 1 -- 172.21.15.45:0/473678358 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fe93c0b6a50 con 0x7fe9540ca600 2024-05-08T05:49:29.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.271+0000 7fe9537fe640 1 --2- 172.21.15.45:0/473678358 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe9340762c0 0x7fe934078780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:29.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.272+0000 7fe9537fe640 1 --2- 172.21.15.45:0/473678358 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe9340762c0 0x7fe934078780 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe9540c8430 tx=0x7fe944009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:29.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.276+0000 7fe950ff9640 1 -- 172.21.15.45:0/473678358 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe93c080220 con 0x7fe9540ca600 2024-05-08T05:49:29.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.546+0000 7fe959c46640 1 -- 172.21.15.45:0/473678358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe9540693f0 con 0x7fe9540ca600 2024-05-08T05:49:29.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.548+0000 7fe950ff9640 1 -- 172.21.15.45:0/473678358 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7fe93c07fb20 con 0x7fe9540ca600 2024-05-08T05:49:29.549 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-05-08T05:49:29.549 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBUEDtmDcqGMxAALgQaBYp4Zd8uMqswCXvGaw== 2024-05-08T05:49:29.549 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2024-05-08T05:49:29.549 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2024-05-08T05:49:29.549 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-08T05:49:29.549 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2024-05-08T05:49:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.551+0000 7fe959c46640 1 -- 172.21.15.45:0/473678358 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe9340762c0 msgr2=0x7fe934078780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.551+0000 7fe959c46640 1 --2- 172.21.15.45:0/473678358 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe9340762c0 0x7fe934078780 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe9540c8430 tx=0x7fe944009040 comp rx=0 tx=0).stop 2024-05-08T05:49:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.551+0000 7fe959c46640 1 -- 172.21.15.45:0/473678358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9540ca600 msgr2=0x7fe9540caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.551+0000 7fe959c46640 1 --2- 172.21.15.45:0/473678358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9540ca600 0x7fe9540caa80 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00c410 tx=0x7fe9540d0c20 comp rx=0 tx=0).stop 2024-05-08T05:49:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.551+0000 7fe959c46640 1 -- 172.21.15.45:0/473678358 shutdown_connections 2024-05-08T05:49:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.551+0000 7fe959c46640 1 --2- 172.21.15.45:0/473678358 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe9340762c0 0x7fe934078780 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.551+0000 7fe959c46640 1 --2- 172.21.15.45:0/473678358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe95415d4e0 0x7fe95415d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.551+0000 7fe959c46640 1 --2- 172.21.15.45:0/473678358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9540ca600 0x7fe9540caa80 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.551+0000 7fe959c46640 1 --2- 172.21.15.45:0/473678358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe940004800 0x7fe9541f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.551+0000 7fe959c46640 1 -- 172.21.15.45:0/473678358 >> 172.21.15.45:0/473678358 conn(0x7fe9540d4fe0 msgr2=0x7fe95415bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.551+0000 7fe959c46640 1 -- 172.21.15.45:0/473678358 shutdown_connections 2024-05-08T05:49:29.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.551+0000 7fe959c46640 1 -- 172.21.15.45:0/473678358 wait complete. 2024-05-08T05:49:29.562 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:49:29.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.666+0000 7efd25f14640 1 Processor -- start 2024-05-08T05:49:29.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.666+0000 7efd25f14640 1 -- start start 2024-05-08T05:49:29.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.667+0000 7efd25f14640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efd201629a0 0x7efd200d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:29.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.667+0000 7efd25f14640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7efd200d2dc0 0x7efd200d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:29.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.667+0000 7efd25f14640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd200d3330 con 0x7efd200c8ed0 2024-05-08T05:49:29.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.667+0000 7efd25f14640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7efd200d2170 con 0x7efd201629a0 2024-05-08T05:49:29.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.667+0000 7efd25f14640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7efd200d2450 con 0x7efd200d2dc0 2024-05-08T05:49:29.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.667+0000 7efd1f7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efd201629a0 0x7efd200d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:29.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.667+0000 7efd1ffff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7efd200c8ed0 0x7efd200c9340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49906/0 (socket says 172.21.15.45:49906) 2024-05-08T05:49:29.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.667+0000 7efd1ffff640 1 -- 172.21.15.45:0/631377284 learned_addr learned my addr 172.21.15.45:0/631377284 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:29.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.667+0000 7efd1effd640 1 --2- 172.21.15.45:0/631377284 >> v2:172.21.15.63:3300/0 conn(0x7efd200d2dc0 0x7efd200d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.667+0000 7efd1e7fc640 1 -- 172.21.15.45:0/631377284 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 199928456 0 0) 0x7efd200d3330 con 0x7efd200c8ed0 2024-05-08T05:49:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.667+0000 7efd1e7fc640 1 -- 172.21.15.45:0/631377284 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd08003520 con 0x7efd200c8ed0 2024-05-08T05:49:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.667+0000 7efd1f7fe640 1 -- 172.21.15.45:0/631377284 >> v2:172.21.15.63:3300/0 conn(0x7efd200d2dc0 msgr2=0x7efd200d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.667+0000 7efd1f7fe640 1 --2- 172.21.15.45:0/631377284 >> v2:172.21.15.63:3300/0 conn(0x7efd200d2dc0 0x7efd200d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.667+0000 7efd1f7fe640 1 -- 172.21.15.45:0/631377284 >> v1:172.21.15.45:6789/0 conn(0x7efd200c8ed0 legacy=0x7efd200c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.667+0000 7efd1f7fe640 1 -- 172.21.15.45:0/631377284 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd200d2730 con 0x7efd201629a0 2024-05-08T05:49:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.668+0000 7efd1f7fe640 1 --2- 172.21.15.45:0/631377284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd201629a0 0x7efd200d35c0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7efd200b7ae0 tx=0x7efd1002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e97dd708e6bc8b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.668+0000 7efd1e7fc640 1 -- 172.21.15.45:0/631377284 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10037070 con 0x7efd201629a0 2024-05-08T05:49:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.668+0000 7efd1e7fc640 1 -- 172.21.15.45:0/631377284 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd1002add0 con 0x7efd201629a0 2024-05-08T05:49:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.668+0000 7efd1e7fc640 1 -- 172.21.15.45:0/631377284 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10032640 con 0x7efd201629a0 2024-05-08T05:49:29.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.669+0000 7efd25f14640 1 -- 172.21.15.45:0/631377284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd201629a0 msgr2=0x7efd200d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:29.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.669+0000 7efd25f14640 1 --2- 172.21.15.45:0/631377284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd201629a0 0x7efd200d35c0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7efd200b7ae0 tx=0x7efd1002a080 comp rx=0 tx=0).stop 2024-05-08T05:49:29.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.669+0000 7efd25f14640 1 -- 172.21.15.45:0/631377284 shutdown_connections 2024-05-08T05:49:29.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.669+0000 7efd25f14640 1 --2- 172.21.15.45:0/631377284 >> v2:172.21.15.63:3300/0 conn(0x7efd200d2dc0 0x7efd200d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.669+0000 7efd25f14640 1 --2- 172.21.15.45:0/631377284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd201629a0 0x7efd200d35c0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.669+0000 7efd25f14640 1 -- 172.21.15.45:0/631377284 >> 172.21.15.45:0/631377284 conn(0x7efd200d4fe0 msgr2=0x7efd200d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:29.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.669+0000 7efd25f14640 1 -- 172.21.15.45:0/631377284 shutdown_connections 2024-05-08T05:49:29.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.669+0000 7efd25f14640 1 -- 172.21.15.45:0/631377284 wait complete. 2024-05-08T05:49:29.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.670+0000 7efd25f14640 1 Processor -- start 2024-05-08T05:49:29.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.670+0000 7efd25f14640 1 -- start start 2024-05-08T05:49:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.671+0000 7efd25f14640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd200c8ed0 0x7efd2015b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.671+0000 7efd25f14640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd200d2dc0 0x7efd200b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.671+0000 7efd25f14640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd201629a0 0x7efd200b8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.671+0000 7efd25f14640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efd200cf440 con 0x7efd200d2dc0 2024-05-08T05:49:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.671+0000 7efd25f14640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efd200cf2c0 con 0x7efd201629a0 2024-05-08T05:49:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.671+0000 7efd25f14640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efd200d0cb0 con 0x7efd200c8ed0 2024-05-08T05:49:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.671+0000 7efd1effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd200d2dc0 0x7efd200b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.671+0000 7efd1effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd200d2dc0 0x7efd200b7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45490/0 (socket says 172.21.15.45:45490) 2024-05-08T05:49:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.671+0000 7efd1effd640 1 -- 172.21.15.45:0/698830676 learned_addr learned my addr 172.21.15.45:0/698830676 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.671+0000 7efd1f7fe640 1 --2- 172.21.15.45:0/698830676 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd200c8ed0 0x7efd2015b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.671+0000 7efd1ffff640 1 --2- 172.21.15.45:0/698830676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd201629a0 0x7efd200b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.671+0000 7efd1effd640 1 -- 172.21.15.45:0/698830676 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd200c8ed0 msgr2=0x7efd2015b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.671+0000 7efd1effd640 1 --2- 172.21.15.45:0/698830676 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd200c8ed0 0x7efd2015b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.671+0000 7efd1effd640 1 -- 172.21.15.45:0/698830676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd201629a0 msgr2=0x7efd200b8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:29.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.671+0000 7efd1effd640 1 --2- 172.21.15.45:0/698830676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd201629a0 0x7efd200b8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.671+0000 7efd1effd640 1 -- 172.21.15.45:0/698830676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd201f4c40 con 0x7efd200d2dc0 2024-05-08T05:49:29.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.671+0000 7efd1effd640 1 --2- 172.21.15.45:0/698830676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd200d2dc0 0x7efd200b7c00 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7efd1400e9e0 tx=0x7efd1400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:29.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.672+0000 7efd07fff640 1 -- 172.21.15.45:0/698830676 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14017070 con 0x7efd200d2dc0 2024-05-08T05:49:29.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.672+0000 7efd07fff640 1 -- 172.21.15.45:0/698830676 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd1400c9d0 con 0x7efd200d2dc0 2024-05-08T05:49:29.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.672+0000 7efd25f14640 1 -- 172.21.15.45:0/698830676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd200cc0b0 con 0x7efd200d2dc0 2024-05-08T05:49:29.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.672+0000 7efd07fff640 1 -- 172.21.15.45:0/698830676 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1400cdc0 con 0x7efd200d2dc0 2024-05-08T05:49:29.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.672+0000 7efd25f14640 1 -- 172.21.15.45:0/698830676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd200cc650 con 0x7efd200d2dc0 2024-05-08T05:49:29.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.673+0000 7efd25f14640 1 -- 172.21.15.45:0/698830676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd200693f0 con 0x7efd200d2dc0 2024-05-08T05:49:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.673+0000 7efd07fff640 1 -- 172.21.15.45:0/698830676 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efd1401a070 con 0x7efd200d2dc0 2024-05-08T05:49:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.674+0000 7efd07fff640 1 --2- 172.21.15.45:0/698830676 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efd0c0762d0 0x7efd0c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.674+0000 7efd07fff640 1 -- 172.21.15.45:0/698830676 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7efd14097730 con 0x7efd200d2dc0 2024-05-08T05:49:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.674+0000 7efd1f7fe640 1 --2- 172.21.15.45:0/698830676 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efd0c0762d0 0x7efd0c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.675+0000 7efd1f7fe640 1 --2- 172.21.15.45:0/698830676 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efd0c0762d0 0x7efd0c078790 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7efd10034000 tx=0x7efd1002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:29.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.679+0000 7efd07fff640 1 -- 172.21.15.45:0/698830676 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efd140608c0 con 0x7efd200d2dc0 2024-05-08T05:49:29.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.909+0000 7efd25f14640 1 -- 172.21.15.45:0/698830676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd200d1e20 con 0x7efd200d2dc0 2024-05-08T05:49:29.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.910+0000 7efd07fff640 1 -- 172.21.15.45:0/698830676 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5180 (secure 0 0 0) 0x7efd14064bc0 con 0x7efd200d2dc0 2024-05-08T05:49:29.910 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:29.910 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3692881261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3692881261},{"type":"v1","addr":"172.21.15.63:6836","nonce":3692881261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7497,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2916399818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2916399818},{"type":"v1","addr":"172.21.15.45:6833","nonce":2916399818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7515,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2978250004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":2978250004},{"type":"v1","addr":"172.21.15.63:6837","nonce":2978250004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:49:19.587326+0000","modified":"2024-05-08T05:49:22.599177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7524":{"gid":7524,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/878092981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":878092981},{"type":"v1","addr":"172.21.15.45:6835","nonce":878092981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7524,"qdb_cluster":[7524]},"id":7}]} 2024-05-08T05:49:29.910 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 83 2024-05-08T05:49:29.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.912+0000 7efd25f14640 1 -- 172.21.15.45:0/698830676 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efd0c0762d0 msgr2=0x7efd0c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:29.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.912+0000 7efd25f14640 1 --2- 172.21.15.45:0/698830676 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efd0c0762d0 0x7efd0c078790 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7efd10034000 tx=0x7efd1002e000 comp rx=0 tx=0).stop 2024-05-08T05:49:29.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.912+0000 7efd25f14640 1 -- 172.21.15.45:0/698830676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd200d2dc0 msgr2=0x7efd200b7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:29.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.912+0000 7efd25f14640 1 --2- 172.21.15.45:0/698830676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd200d2dc0 0x7efd200b7c00 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7efd1400e9e0 tx=0x7efd1400eea0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.912+0000 7efd25f14640 1 -- 172.21.15.45:0/698830676 shutdown_connections 2024-05-08T05:49:29.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.912+0000 7efd25f14640 1 --2- 172.21.15.45:0/698830676 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efd0c0762d0 0x7efd0c078790 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.912+0000 7efd25f14640 1 --2- 172.21.15.45:0/698830676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd201629a0 0x7efd200b8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.912+0000 7efd25f14640 1 --2- 172.21.15.45:0/698830676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd200d2dc0 0x7efd200b7c00 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.912+0000 7efd25f14640 1 --2- 172.21.15.45:0/698830676 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efd200c8ed0 0x7efd2015b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:29.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.912+0000 7efd25f14640 1 -- 172.21.15.45:0/698830676 >> 172.21.15.45:0/698830676 conn(0x7efd200d4fe0 msgr2=0x7efd201612d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:29.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.912+0000 7efd25f14640 1 -- 172.21.15.45:0/698830676 shutdown_connections 2024-05-08T05:49:29.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:29.912+0000 7efd25f14640 1 -- 172.21.15.45:0/698830676 wait complete. 2024-05-08T05:49:30.326 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:49:19.587326+0000', 'modified': '2024-05-08T05:49:22.599177+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7524}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7524': {'gid': 7524, 'name': 'a', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6835/878092981', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 878092981}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 878092981}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7524, 'qdb_cluster': [7524]} 2024-05-08T05:49:30.326 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T05:49:30.326 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T05:49:30.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.435+0000 7f4b0cb71640 1 Processor -- start 2024-05-08T05:49:30.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.435+0000 7f4b0cb71640 1 -- start start 2024-05-08T05:49:30.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.436+0000 7f4b0cb71640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4b081629b0 0x7f4b0806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.436+0000 7f4b0cb71640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4b080b2490 0x7f4b08070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.436+0000 7f4b0cb71640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b0806e110 con 0x7f4b08070d20 2024-05-08T05:49:30.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.436+0000 7f4b0cb71640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4b0806dec0 con 0x7f4b080b2490 2024-05-08T05:49:30.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.436+0000 7f4b0cb71640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4b0806e3f0 con 0x7f4b081629b0 2024-05-08T05:49:30.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.436+0000 7f4b05d9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4b080b2490 0x7f4b08070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.436+0000 7f4b05d9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4b080b2490 0x7f4b08070330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45502/0 (socket says 172.21.15.45:45502) 2024-05-08T05:49:30.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.436+0000 7f4b05d9b640 1 -- 172.21.15.45:0/1476232082 learned_addr learned my addr 172.21.15.45:0/1476232082 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:30.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.436+0000 7f4b0659c640 1 --2- 172.21.15.45:0/1476232082 >> v2:172.21.15.63:3300/0 conn(0x7f4b081629b0 0x7f4b0806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.436+0000 7f4b05d9b640 1 -- 172.21.15.45:0/1476232082 >> v2:172.21.15.63:3300/0 conn(0x7f4b081629b0 msgr2=0x7f4b0806eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:30.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.436+0000 7f4b05d9b640 1 --2- 172.21.15.45:0/1476232082 >> v2:172.21.15.63:3300/0 conn(0x7f4b081629b0 0x7f4b0806eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.436+0000 7f4b05d9b640 1 -- 172.21.15.45:0/1476232082 >> v1:172.21.15.63:6789/0 conn(0x7f4b08070d20 legacy=0x7f4b080afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:30.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.436+0000 7f4b05d9b640 1 -- 172.21.15.45:0/1476232082 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0806e6d0 con 0x7f4b080b2490 2024-05-08T05:49:30.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.437+0000 7f4b05d9b640 1 --2- 172.21.15.45:0/1476232082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b080b2490 0x7f4b08070330 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f4b08068b40 tx=0x7f4af002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b54eae91ac32ac0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:30.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.437+0000 7f4b0559a640 1 -- 172.21.15.45:0/1476232082 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af0037070 con 0x7f4b080b2490 2024-05-08T05:49:30.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.437+0000 7f4b0559a640 1 -- 172.21.15.45:0/1476232082 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4af0032440 con 0x7f4b080b2490 2024-05-08T05:49:30.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.437+0000 7f4b0559a640 1 -- 172.21.15.45:0/1476232082 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af0032810 con 0x7f4b080b2490 2024-05-08T05:49:30.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.438+0000 7f4b0cb71640 1 -- 172.21.15.45:0/1476232082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b080b2490 msgr2=0x7f4b08070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:30.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.438+0000 7f4b0cb71640 1 --2- 172.21.15.45:0/1476232082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b080b2490 0x7f4b08070330 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f4b08068b40 tx=0x7f4af002a270 comp rx=0 tx=0).stop 2024-05-08T05:49:30.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.438+0000 7f4b0cb71640 1 -- 172.21.15.45:0/1476232082 shutdown_connections 2024-05-08T05:49:30.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.438+0000 7f4b0cb71640 1 --2- 172.21.15.45:0/1476232082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b080b2490 0x7f4b08070330 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.438+0000 7f4b0cb71640 1 --2- 172.21.15.45:0/1476232082 >> v2:172.21.15.63:3300/0 conn(0x7f4b081629b0 0x7f4b0806eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.438+0000 7f4b0cb71640 1 -- 172.21.15.45:0/1476232082 >> 172.21.15.45:0/1476232082 conn(0x7f4b080b49d0 msgr2=0x7f4b080b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:30.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.438+0000 7f4b0cb71640 1 -- 172.21.15.45:0/1476232082 shutdown_connections 2024-05-08T05:49:30.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.438+0000 7f4b0cb71640 1 -- 172.21.15.45:0/1476232082 wait complete. 2024-05-08T05:49:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.439+0000 7f4b0cb71640 1 Processor -- start 2024-05-08T05:49:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.439+0000 7f4b0cb71640 1 -- start start 2024-05-08T05:49:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.440+0000 7f4b0cb71640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b08070d20 0x7f4b080cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.440+0000 7f4b0cb71640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b080b2490 0x7f4b081f8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.440+0000 7f4b0cb71640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b081629b0 0x7f4b081f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.440+0000 7f4b0cb71640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b08070870 con 0x7f4b08070d20 2024-05-08T05:49:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.440+0000 7f4b0cb71640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0806f410 con 0x7f4b081629b0 2024-05-08T05:49:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.440+0000 7f4b0cb71640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b080709f0 con 0x7f4b080b2490 2024-05-08T05:49:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.440+0000 7f4b0659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b08070d20 0x7f4b080cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.440+0000 7f4b0659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b08070d20 0x7f4b080cf6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45508/0 (socket says 172.21.15.45:45508) 2024-05-08T05:49:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.440+0000 7f4b0659c640 1 -- 172.21.15.45:0/1379964775 learned_addr learned my addr 172.21.15.45:0/1379964775 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.441+0000 7f4b06d9d640 1 --2- 172.21.15.45:0/1379964775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b081629b0 0x7f4b081f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.441+0000 7f4b0659c640 1 -- 172.21.15.45:0/1379964775 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b080b2490 msgr2=0x7f4b081f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.441+0000 7f4b0659c640 1 --2- 172.21.15.45:0/1379964775 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b080b2490 0x7f4b081f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.441+0000 7f4b0659c640 1 -- 172.21.15.45:0/1379964775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b081629b0 msgr2=0x7f4b081f8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.441+0000 7f4b0659c640 1 --2- 172.21.15.45:0/1379964775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b081629b0 0x7f4b081f8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.441+0000 7f4b0659c640 1 -- 172.21.15.45:0/1379964775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b080c77d0 con 0x7f4b08070d20 2024-05-08T05:49:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.441+0000 7f4b06d9d640 1 --2- 172.21.15.45:0/1379964775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b081629b0 0x7f4b081f8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:49:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.441+0000 7f4b0659c640 1 --2- 172.21.15.45:0/1379964775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b08070d20 0x7f4b080cf6b0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f4af800d620 tx=0x7f4af800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.441+0000 7f4afeffd640 1 -- 172.21.15.45:0/1379964775 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af800b620 con 0x7f4b08070d20 2024-05-08T05:49:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.441+0000 7f4afeffd640 1 -- 172.21.15.45:0/1379964775 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4af8019070 con 0x7f4b08070d20 2024-05-08T05:49:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.441+0000 7f4afeffd640 1 -- 172.21.15.45:0/1379964775 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af800ba00 con 0x7f4b08070d20 2024-05-08T05:49:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.441+0000 7f4b0cb71640 1 -- 172.21.15.45:0/1379964775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b080c7ab0 con 0x7f4b08070d20 2024-05-08T05:49:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.441+0000 7f4b0cb71640 1 -- 172.21.15.45:0/1379964775 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b080c8070 con 0x7f4b08070d20 2024-05-08T05:49:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.443+0000 7f4afeffd640 1 -- 172.21.15.45:0/1379964775 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4af800bba0 con 0x7f4b08070d20 2024-05-08T05:49:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.443+0000 7f4afeffd640 1 --2- 172.21.15.45:0/1379964775 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4ae80762d0 0x7f4ae8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.443+0000 7f4afeffd640 1 -- 172.21.15.45:0/1379964775 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f4af8097c30 con 0x7f4b08070d20 2024-05-08T05:49:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.443+0000 7f4b05d9b640 1 --2- 172.21.15.45:0/1379964775 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4ae80762d0 0x7f4ae8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.444+0000 7f4b0f966740 1 Processor -- start 2024-05-08T05:49:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.444+0000 7f4b0f966740 1 -- start start 2024-05-08T05:49:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.444+0000 7f4b05d9b640 1 --2- 172.21.15.45:0/1379964775 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4ae80762d0 0x7f4ae8078790 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4af0034000 tx=0x7f4af002c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.445+0000 7f4b0f966740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16a4f50 0x5651a16a7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.445+0000 7f4b0f966740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16a5330 0x5651a16a7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.445+0000 7f4b0f966740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a1677bb0 0x5651a167b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.445+0000 7f4b0f966740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5651a15e18d0 con 0x5651a1677bb0 2024-05-08T05:49:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.445+0000 7f4b0f966740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5651a159ac10 con 0x5651a16a4f50 2024-05-08T05:49:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.445+0000 7f4b0f966740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5651a15dd890 con 0x5651a16a5330 2024-05-08T05:49:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.445+0000 7f4b06d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a1677bb0 0x5651a167b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.445+0000 7f4b06d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a1677bb0 0x5651a167b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45510/0 (socket says 172.21.15.45:45510) 2024-05-08T05:49:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.445+0000 7f4b06d9d640 1 -- 172.21.15.45:0/1392487343 learned_addr learned my addr 172.21.15.45:0/1392487343 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.445+0000 7f4b0659c640 1 --2- 172.21.15.45:0/1392487343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16a4f50 0x5651a16a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.445+0000 7f4b05d9b640 1 --2- 172.21.15.45:0/1392487343 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16a5330 0x5651a16a7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.445+0000 7f4b06d9d640 1 -- 172.21.15.45:0/1392487343 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16a5330 msgr2=0x5651a16a7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.445+0000 7f4b06d9d640 1 --2- 172.21.15.45:0/1392487343 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16a5330 0x5651a16a7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.445+0000 7f4b06d9d640 1 -- 172.21.15.45:0/1392487343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16a4f50 msgr2=0x5651a16a7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.445+0000 7f4b06d9d640 1 --2- 172.21.15.45:0/1392487343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16a4f50 0x5651a16a7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.445+0000 7f4b06d9d640 1 -- 172.21.15.45:0/1392487343 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a15a25d0 con 0x5651a1677bb0 2024-05-08T05:49:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.445+0000 7f4b06d9d640 1 --2- 172.21.15.45:0/1392487343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a1677bb0 0x5651a167b3c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f4b00001e90 tx=0x7f4b0000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b53cea188f71f56 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.445+0000 7f4ae7fff640 1 -- 172.21.15.45:0/1392487343 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00018070 con 0x5651a1677bb0 2024-05-08T05:49:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.445+0000 7f4ae7fff640 1 -- 172.21.15.45:0/1392487343 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b000020a0 con 0x5651a1677bb0 2024-05-08T05:49:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.445+0000 7f4ae7fff640 1 -- 172.21.15.45:0/1392487343 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00003b90 con 0x5651a1677bb0 2024-05-08T05:49:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.446+0000 7f4b0f966740 1 -- 172.21.15.45:0/1392487343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a1677bb0 msgr2=0x5651a167b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.446+0000 7f4b0f966740 1 --2- 172.21.15.45:0/1392487343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a1677bb0 0x5651a167b3c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f4b00001e90 tx=0x7f4b0000d520 comp rx=0 tx=0).stop 2024-05-08T05:49:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.446+0000 7f4b0f966740 1 -- 172.21.15.45:0/1392487343 shutdown_connections 2024-05-08T05:49:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.446+0000 7f4b0f966740 1 --2- 172.21.15.45:0/1392487343 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16a5330 0x5651a16a7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.446+0000 7f4b0f966740 1 --2- 172.21.15.45:0/1392487343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a1677bb0 0x5651a167b3c0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.446+0000 7f4b0f966740 1 --2- 172.21.15.45:0/1392487343 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16a4f50 0x5651a16a7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.446+0000 7f4b0f966740 1 -- 172.21.15.45:0/1392487343 >> 172.21.15.45:0/1392487343 conn(0x5651a1679e20 msgr2=0x5651a16c2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.446+0000 7f4b0f966740 1 -- 172.21.15.45:0/1392487343 shutdown_connections 2024-05-08T05:49:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.446+0000 7f4b0f966740 1 -- 172.21.15.45:0/1392487343 wait complete. 2024-05-08T05:49:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.447+0000 7f4b0f966740 1 -- start start 2024-05-08T05:49:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.447+0000 7f4b0f966740 1 Processor -- start 2024-05-08T05:49:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.448+0000 7f4b0f966740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 0x5651a169c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.448+0000 7f4b0f966740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16a5330 0x5651a16c5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.448+0000 7f4b0f966740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16c5ed0 0x5651a167c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.448+0000 7f4b0f966740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5651a167cfd0 con 0x5651a16a4f50 2024-05-08T05:49:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.448+0000 7f4b0f966740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5651a159e760 con 0x5651a16c5ed0 2024-05-08T05:49:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.448+0000 7f4b0f966740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5651a167d2b0 con 0x5651a16a5330 2024-05-08T05:49:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.448+0000 7f4b0f966740 10 client.0 _ll_register_callbacks cb 0x5651a15d7cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:49:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.448+0000 7f4b0659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 0x5651a169c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.448+0000 7f4b0659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 0x5651a169c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45520/0 (socket says 172.21.15.45:45520) 2024-05-08T05:49:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.448+0000 7f4b0659c640 1 -- 172.21.15.45:0/2096673934 learned_addr learned my addr 172.21.15.45:0/2096673934 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.448+0000 7f4b0659c640 1 -- 172.21.15.45:0/2096673934 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16a5330 msgr2=0x5651a16c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.448+0000 7f4b0659c640 1 --2- 172.21.15.45:0/2096673934 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16a5330 0x5651a16c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.448+0000 7f4b0659c640 1 -- 172.21.15.45:0/2096673934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16c5ed0 msgr2=0x5651a167c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.448+0000 7f4b0659c640 1 --2- 172.21.15.45:0/2096673934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16c5ed0 0x5651a167c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.448+0000 7f4b0659c640 1 -- 172.21.15.45:0/2096673934 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a12f32d0 con 0x5651a16a4f50 2024-05-08T05:49:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.448+0000 7f4b0659c640 1 --2- 172.21.15.45:0/2096673934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 0x5651a169c100 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f4af801d000 tx=0x7f4af8069410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd950fe81477989 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.448+0000 7f4ae57fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:49:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.449+0000 7f4ae57fa640 1 -- 172.21.15.45:0/2096673934 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8090070 con 0x5651a16a4f50 2024-05-08T05:49:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.449+0000 7f4ae57fa640 1 -- 172.21.15.45:0/2096673934 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f4af8011ba0 con 0x5651a16a4f50 2024-05-08T05:49:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.449+0000 7f4b0f966740 1 -- 172.21.15.45:0/2096673934 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5651a1565760 con 0x5651a16a4f50 2024-05-08T05:49:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.449+0000 7f4ae57fa640 1 -- 172.21.15.45:0/2096673934 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f4af8080050 con 0x5651a16a4f50 2024-05-08T05:49:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.449+0000 7f4b0f966740 10 client.7668 fetch_fsmap learned FSMap version 83 2024-05-08T05:49:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.449+0000 7f4b0f966740 1 -- 172.21.15.45:0/2096673934 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x5651a13bd6b0 con 0x5651a16a4f50 2024-05-08T05:49:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.449+0000 7f4ae57fa640 1 -- 172.21.15.45:0/2096673934 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5651a1565760 con 0x5651a16a4f50 2024-05-08T05:49:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.449+0000 7f4ae57fa640 1 -- 172.21.15.45:0/2096673934 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7f4af8013070 con 0x5651a16a4f50 2024-05-08T05:49:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.449+0000 7f4b0f966740 10 client.7668 fetch_fsmap finished waiting for FSMap version 83 2024-05-08T05:49:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.449+0000 7f4b0f966740 10 client.7668 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T05:49:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.449+0000 7f4b0f966740 20 client.7668 populate_metadata read hostname 'smithi045' 2024-05-08T05:49:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.449+0000 7f4b0f966740 1 --2- 172.21.15.45:0/2096673934 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x5651a16c3d60 0x5651a169e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.449+0000 7f4b0f966740 4 client.7668 mds_command: new command op to 7524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:49:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.449+0000 7f4b0f966740 1 -- 172.21.15.45:0/2096673934 --> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651a15ef4e0 con 0x5651a16c3d60 2024-05-08T05:49:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.449+0000 7f4b05d9b640 1 --2- 172.21.15.45:0/2096673934 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x5651a16c3d60 0x5651a169e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.449+0000 7f4ae57fa640 10 client.7668 ms_handle_connect on v2:172.21.15.45:6834/878092981 2024-05-08T05:49:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.449+0000 7f4b05d9b640 1 --2- 172.21.15.45:0/2096673934 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x5651a16c3d60 0x5651a169e9e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b591f8982e7de8b server_cookie=75350d4d8726415f in_seq=0 out_seq=0 2024-05-08T05:49:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.450+0000 7f4ae57fa640 1 -- 172.21.15.45:0/2096673934 <== mds.0 v2:172.21.15.45:6834/878092981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5651a15ef4e0 con 0x5651a16c3d60 2024-05-08T05:49:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.450+0000 7f4ae57fa640 10 client.7668 handle_command_reply: tid=0 2024-05-08T05:49:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.450+0000 7f4b0f966740 1 client.7668 shutdown 2024-05-08T05:49:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.450+0000 7f4b0f966740 1 -- 172.21.15.45:0/2096673934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 msgr2=0x5651a169c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.450+0000 7f4b0f966740 1 --2- 172.21.15.45:0/2096673934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 0x5651a169c100 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f4af801d000 tx=0x7f4af8069410 comp rx=0 tx=0).stop 2024-05-08T05:49:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.450+0000 7f4b0f966740 1 -- 172.21.15.45:0/2096673934 shutdown_connections 2024-05-08T05:49:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.450+0000 7f4b0f966740 1 --2- 172.21.15.45:0/2096673934 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x5651a16c3d60 0x5651a169e9e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.450+0000 7f4b0f966740 1 --2- 172.21.15.45:0/2096673934 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16a5330 0x5651a16c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.450+0000 7f4b0f966740 1 --2- 172.21.15.45:0/2096673934 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16c5ed0 0x5651a167c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.450+0000 7f4ae57fa640 0 client.7668 ms_handle_reset on v2:172.21.15.45:6834/878092981 2024-05-08T05:49:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.450+0000 7f4b0f966740 1 --2- 172.21.15.45:0/2096673934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 0x5651a169c100 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.450+0000 7f4b0f966740 1 -- 172.21.15.45:0/2096673934 >> 172.21.15.45:0/2096673934 conn(0x5651a1677bb0 msgr2=0x5651a16a35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.451+0000 7f4b0f966740 1 -- 172.21.15.45:0/2096673934 shutdown_connections 2024-05-08T05:49:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.451+0000 7f4b0f966740 1 -- 172.21.15.45:0/2096673934 wait complete. 2024-05-08T05:49:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.451+0000 7f4b0f966740 20 client.7668 trim_cache size 0 max 16384 2024-05-08T05:49:30.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.469+0000 7f4b0f966740 1 Processor -- start 2024-05-08T05:49:30.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.469+0000 7f4b0f966740 1 -- start start 2024-05-08T05:49:30.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.470+0000 7f4b0f966740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 0x5651a169b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.470+0000 7f4b0f966740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16a5330 0x5651a16a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.470+0000 7f4b0f966740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16c5ed0 0x5651a16a4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.470+0000 7f4b0f966740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b08200f70 con 0x5651a16a4f50 2024-05-08T05:49:30.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.470+0000 7f4b0f966740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4b080ce310 con 0x5651a16a5330 2024-05-08T05:49:30.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.470+0000 7f4b0f966740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4b080ce960 con 0x5651a16c5ed0 2024-05-08T05:49:30.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.470+0000 7f4b0659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 0x5651a169b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.470+0000 7f4b0659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 0x5651a169b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45524/0 (socket says 172.21.15.45:45524) 2024-05-08T05:49:30.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.470+0000 7f4b0659c640 1 -- 172.21.15.45:0/2370645861 learned_addr learned my addr 172.21.15.45:0/2370645861 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:30.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.470+0000 7f4b06d9d640 1 --2- 172.21.15.45:0/2370645861 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16c5ed0 0x5651a16a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.470+0000 7f4b05d9b640 1 --2- 172.21.15.45:0/2370645861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16a5330 0x5651a16a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.470+0000 7f4b0659c640 1 -- 172.21.15.45:0/2370645861 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16c5ed0 msgr2=0x5651a16a4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:30.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.470+0000 7f4b0659c640 1 --2- 172.21.15.45:0/2370645861 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16c5ed0 0x5651a16a4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.470+0000 7f4b0659c640 1 -- 172.21.15.45:0/2370645861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16a5330 msgr2=0x5651a16a2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:30.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.470+0000 7f4b0659c640 1 --2- 172.21.15.45:0/2370645861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16a5330 0x5651a16a2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.470+0000 7f4b0659c640 1 -- 172.21.15.45:0/2370645861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a1678130 con 0x5651a16a4f50 2024-05-08T05:49:30.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.470+0000 7f4b06d9d640 1 --2- 172.21.15.45:0/2370645861 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16c5ed0 0x5651a16a4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:30.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.470+0000 7f4b05d9b640 1 --2- 172.21.15.45:0/2370645861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16a5330 0x5651a16a2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:30.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.471+0000 7f4b0659c640 1 --2- 172.21.15.45:0/2370645861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 0x5651a169b620 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f4af805df70 tx=0x7f4af8078000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e01b97d8e6c7f73 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:30.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.471+0000 7f4ae77fe640 1 -- 172.21.15.45:0/2370645861 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af806d030 con 0x5651a16a4f50 2024-05-08T05:49:30.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.471+0000 7f4ae77fe640 1 -- 172.21.15.45:0/2370645861 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4af8069ad0 con 0x5651a16a4f50 2024-05-08T05:49:30.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.471+0000 7f4ae77fe640 1 -- 172.21.15.45:0/2370645861 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8090050 con 0x5651a16a4f50 2024-05-08T05:49:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.471+0000 7f4b0f966740 1 -- 172.21.15.45:0/2370645861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 msgr2=0x5651a169b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.471+0000 7f4b0f966740 1 --2- 172.21.15.45:0/2370645861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 0x5651a169b620 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f4af805df70 tx=0x7f4af8078000 comp rx=0 tx=0).stop 2024-05-08T05:49:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.471+0000 7f4b0f966740 1 -- 172.21.15.45:0/2370645861 shutdown_connections 2024-05-08T05:49:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.471+0000 7f4b0f966740 1 --2- 172.21.15.45:0/2370645861 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16c5ed0 0x5651a16a4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.471+0000 7f4b0f966740 1 --2- 172.21.15.45:0/2370645861 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16a5330 0x5651a16a2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.471+0000 7f4b0f966740 1 --2- 172.21.15.45:0/2370645861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 0x5651a169b620 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.471+0000 7f4b0f966740 1 -- 172.21.15.45:0/2370645861 >> 172.21.15.45:0/2370645861 conn(0x5651a1677bb0 msgr2=0x5651a16aa280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.471+0000 7f4b0f966740 1 -- 172.21.15.45:0/2370645861 shutdown_connections 2024-05-08T05:49:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.471+0000 7f4b0f966740 1 -- 172.21.15.45:0/2370645861 wait complete. 2024-05-08T05:49:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.472+0000 7f4b0f966740 1 -- start start 2024-05-08T05:49:30.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.472+0000 7f4b0f966740 1 Processor -- start 2024-05-08T05:49:30.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4b0f966740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 0x5651a173ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4b0f966740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16a5330 0x5651a167ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4b0f966740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16c5ed0 0x5651a167bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4b0f966740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5651a169f430 con 0x5651a16a4f50 2024-05-08T05:49:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4b0f966740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5651a169db60 con 0x5651a16c5ed0 2024-05-08T05:49:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4b0f966740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5651a16c41e0 con 0x5651a16a5330 2024-05-08T05:49:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4b0f966740 10 client.0 _ll_register_callbacks cb 0x7f4af8060ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:49:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4b0659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 0x5651a173ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4b0659c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 0x5651a173ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45540/0 (socket says 172.21.15.45:45540) 2024-05-08T05:49:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4b0659c640 1 -- 172.21.15.45:0/894384921 learned_addr learned my addr 172.21.15.45:0/894384921 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4b06d9d640 1 --2- 172.21.15.45:0/894384921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16c5ed0 0x5651a167bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4b05d9b640 1 --2- 172.21.15.45:0/894384921 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16a5330 0x5651a167ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4b0659c640 1 -- 172.21.15.45:0/894384921 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16a5330 msgr2=0x5651a167ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4b0659c640 1 --2- 172.21.15.45:0/894384921 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16a5330 0x5651a167ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4b0659c640 1 -- 172.21.15.45:0/894384921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16c5ed0 msgr2=0x5651a167bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:30.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4b0659c640 1 --2- 172.21.15.45:0/894384921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16c5ed0 0x5651a167bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4b0659c640 1 -- 172.21.15.45:0/894384921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a167d5f0 con 0x5651a16a4f50 2024-05-08T05:49:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4b05d9b640 1 --2- 172.21.15.45:0/894384921 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16a5330 0x5651a167ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4b06d9d640 1 --2- 172.21.15.45:0/894384921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16c5ed0 0x5651a167bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4b0659c640 1 --2- 172.21.15.45:0/894384921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 0x5651a173ff30 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f4af801d000 tx=0x7f4af8095ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e5554ff8f6060b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.473+0000 7f4ae57fa640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:49:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.474+0000 7f4ae57fa640 1 -- 172.21.15.45:0/894384921 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af806d030 con 0x5651a16a4f50 2024-05-08T05:49:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.474+0000 7f4ae57fa640 1 -- 172.21.15.45:0/894384921 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f4af805ff60 con 0x5651a16a4f50 2024-05-08T05:49:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.474+0000 7f4b0f966740 1 -- 172.21.15.45:0/894384921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4af8069ad0 con 0x5651a16a4f50 2024-05-08T05:49:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.474+0000 7f4ae57fa640 1 -- 172.21.15.45:0/894384921 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f4af8090070 con 0x5651a16a4f50 2024-05-08T05:49:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.474+0000 7f4b0f966740 10 client.7674 fetch_fsmap learned FSMap version 83 2024-05-08T05:49:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.474+0000 7f4b0f966740 1 -- 172.21.15.45:0/894384921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x5651a17404d0 con 0x5651a16a4f50 2024-05-08T05:49:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.474+0000 7f4ae57fa640 1 -- 172.21.15.45:0/894384921 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4af8069ad0 con 0x5651a16a4f50 2024-05-08T05:49:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.474+0000 7f4ae57fa640 1 -- 172.21.15.45:0/894384921 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7f4af80672b0 con 0x5651a16a4f50 2024-05-08T05:49:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.474+0000 7f4b0f966740 10 client.7674 fetch_fsmap finished waiting for FSMap version 83 2024-05-08T05:49:30.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.474+0000 7f4b0f966740 10 client.7674 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T05:49:30.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.474+0000 7f4b0f966740 20 client.7674 populate_metadata read hostname 'smithi045' 2024-05-08T05:49:30.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.474+0000 7f4b0f966740 1 --2- 172.21.15.45:0/894384921 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x5651a1678980 0x5651a16aad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.474+0000 7f4b0f966740 4 client.7674 mds_command: new command op to 7524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:49:30.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.474+0000 7f4b0f966740 1 -- 172.21.15.45:0/894384921 --> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] -- command(tid 0: {"prefix": "status"}) -- 0x5651a15e9350 con 0x5651a1678980 2024-05-08T05:49:30.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.474+0000 7f4b05d9b640 1 --2- 172.21.15.45:0/894384921 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x5651a1678980 0x5651a16aad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.475+0000 7f4ae57fa640 10 client.7674 ms_handle_connect on v2:172.21.15.45:6834/878092981 2024-05-08T05:49:30.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.475+0000 7f4b05d9b640 1 --2- 172.21.15.45:0/894384921 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x5651a1678980 0x5651a16aad80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=717be3ab33520d14 server_cookie=1baa02eafea376f5 in_seq=0 out_seq=0 2024-05-08T05:49:30.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.475+0000 7f4ae57fa640 1 -- 172.21.15.45:0/894384921 <== mds.0 v2:172.21.15.45:6834/878092981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x5651a15e9350 con 0x5651a1678980 2024-05-08T05:49:30.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.475+0000 7f4ae57fa640 10 client.7674 handle_command_reply: tid=0 2024-05-08T05:49:30.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.475+0000 7f4b0f966740 1 client.7674 shutdown 2024-05-08T05:49:30.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.475+0000 7f4b0f966740 1 -- 172.21.15.45:0/894384921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 msgr2=0x5651a173ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:30.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.475+0000 7f4b0f966740 1 --2- 172.21.15.45:0/894384921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 0x5651a173ff30 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f4af801d000 tx=0x7f4af8095ea0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.475+0000 7f4b0f966740 1 -- 172.21.15.45:0/894384921 shutdown_connections 2024-05-08T05:49:30.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.475+0000 7f4b0f966740 1 --2- 172.21.15.45:0/894384921 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x5651a1678980 0x5651a16aad80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.475+0000 7f4b0f966740 1 --2- 172.21.15.45:0/894384921 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5651a16a5330 0x5651a167ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.475+0000 7f4b0f966740 1 --2- 172.21.15.45:0/894384921 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5651a16c5ed0 0x5651a167bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.475+0000 7f4ae57fa640 0 client.7674 ms_handle_reset on v2:172.21.15.45:6834/878092981 2024-05-08T05:49:30.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.475+0000 7f4b0f966740 1 --2- 172.21.15.45:0/894384921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5651a16a4f50 0x5651a173ff30 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.475+0000 7f4b0f966740 1 -- 172.21.15.45:0/894384921 >> 172.21.15.45:0/894384921 conn(0x5651a1677bb0 msgr2=0x5651a16aa280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:30.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.475+0000 7f4b0f966740 1 -- 172.21.15.45:0/894384921 shutdown_connections 2024-05-08T05:49:30.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.475+0000 7f4b0f966740 1 -- 172.21.15.45:0/894384921 wait complete. 2024-05-08T05:49:30.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.475+0000 7f4b0f966740 20 client.7674 trim_cache size 0 max 16384 2024-05-08T05:49:30.482 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:49:30.483 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "e9e1170a-fd27-4625-97a9-5d5f104d13a2", 2024-05-08T05:49:30.483 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-05-08T05:49:30.483 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 7524, 2024-05-08T05:49:30.483 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-05-08T05:49:30.483 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-05-08T05:49:30.483 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-05-08T05:49:30.483 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 10.271707446000001, 2024-05-08T05:49:30.483 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 83, 2024-05-08T05:49:30.483 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 104, 2024-05-08T05:49:30.483 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 104, 2024-05-08T05:49:30.483 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 16.281505987999999 2024-05-08T05:49:30.484 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:49:30.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.476+0000 7f4b0cb71640 1 -- 172.21.15.45:0/1379964775 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4ae80762d0 msgr2=0x7f4ae8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:30.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.476+0000 7f4b0cb71640 1 --2- 172.21.15.45:0/1379964775 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4ae80762d0 0x7f4ae8078790 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4af0034000 tx=0x7f4af002c000 comp rx=0 tx=0).stop 2024-05-08T05:49:30.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.476+0000 7f4b0cb71640 1 -- 172.21.15.45:0/1379964775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b08070d20 msgr2=0x7f4b080cf6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:30.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.476+0000 7f4b0cb71640 1 --2- 172.21.15.45:0/1379964775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b08070d20 0x7f4b080cf6b0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f4af800d620 tx=0x7f4af800dae0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.476+0000 7f4b0cb71640 1 -- 172.21.15.45:0/1379964775 shutdown_connections 2024-05-08T05:49:30.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.476+0000 7f4b0cb71640 1 --2- 172.21.15.45:0/1379964775 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4ae80762d0 0x7f4ae8078790 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.476+0000 7f4b0cb71640 1 --2- 172.21.15.45:0/1379964775 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4b080b2490 0x7f4b081f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.476+0000 7f4b0cb71640 1 --2- 172.21.15.45:0/1379964775 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4b081629b0 0x7f4b081f8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.476+0000 7f4b0cb71640 1 --2- 172.21.15.45:0/1379964775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b08070d20 0x7f4b080cf6b0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.476+0000 7f4b0cb71640 1 -- 172.21.15.45:0/1379964775 >> 172.21.15.45:0/1379964775 conn(0x7f4b080b49d0 msgr2=0x7f4b0815fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:30.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.476+0000 7f4b0cb71640 1 -- 172.21.15.45:0/1379964775 shutdown_connections 2024-05-08T05:49:30.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.476+0000 7f4b0cb71640 1 -- 172.21.15.45:0/1379964775 wait complete. 2024-05-08T05:49:30.487 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T05:49:30.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.590+0000 7f013330a640 1 Processor -- start 2024-05-08T05:49:30.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.590+0000 7f013330a640 1 -- start start 2024-05-08T05:49:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.591+0000 7f013330a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f012c163920 0x7f012c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.591+0000 7f013330a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f012c163d70 con 0x7f012c070d30 2024-05-08T05:49:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.591+0000 7f013330a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f012c0d1300 con 0x7f012c0d20c0 2024-05-08T05:49:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.591+0000 7f013330a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f012c0d10d0 con 0x7f012c163920 2024-05-08T05:49:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.591+0000 7f01310a6640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f012c070d30 0x7f012c0703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49908/0 (socket says 172.21.15.45:49908) 2024-05-08T05:49:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.591+0000 7f01310a6640 1 -- 172.21.15.45:0/2090694583 learned_addr learned my addr 172.21.15.45:0/2090694583 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.592+0000 7f01318a7640 1 --2- 172.21.15.45:0/2090694583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f012c163920 0x7f012c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.592+0000 7f0123fff640 1 -- 172.21.15.45:0/2090694583 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3316212080 0 0) 0x7f0128001110 con 0x7f012c070d30 2024-05-08T05:49:30.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.592+0000 7f0123fff640 1 -- 172.21.15.45:0/2090694583 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0118003520 con 0x7f012c070d30 2024-05-08T05:49:30.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.592+0000 7f0123fff640 1 -- 172.21.15.45:0/2090694583 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2035329071 0 0) 0x7f012c0d1300 con 0x7f012c0d20c0 2024-05-08T05:49:30.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.592+0000 7f0123fff640 1 -- 172.21.15.45:0/2090694583 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0128001110 con 0x7f012c0d20c0 2024-05-08T05:49:30.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.592+0000 7f01318a7640 1 -- 172.21.15.45:0/2090694583 >> v1:172.21.15.63:6789/0 conn(0x7f012c0d20c0 legacy=0x7f012c071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:30.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.592+0000 7f01318a7640 1 -- 172.21.15.45:0/2090694583 >> v1:172.21.15.45:6789/0 conn(0x7f012c070d30 legacy=0x7f012c0703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:30.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.592+0000 7f01318a7640 1 -- 172.21.15.45:0/2090694583 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012c0d15e0 con 0x7f012c163920 2024-05-08T05:49:30.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.593+0000 7f01318a7640 1 --2- 172.21.15.45:0/2090694583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f012c163920 0x7f012c06d4b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0124009dc0 tx=0x7f012402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=189f5039a41b39ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:30.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.593+0000 7f0123fff640 1 -- 172.21.15.45:0/2090694583 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124007740 con 0x7f012c163920 2024-05-08T05:49:30.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.593+0000 7f0123fff640 1 -- 172.21.15.45:0/2090694583 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01240078e0 con 0x7f012c163920 2024-05-08T05:49:30.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.593+0000 7f0123fff640 1 -- 172.21.15.45:0/2090694583 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124007cb0 con 0x7f012c163920 2024-05-08T05:49:30.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.594+0000 7f013330a640 1 -- 172.21.15.45:0/2090694583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f012c163920 msgr2=0x7f012c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:30.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.594+0000 7f013330a640 1 --2- 172.21.15.45:0/2090694583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f012c163920 0x7f012c06d4b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0124009dc0 tx=0x7f012402a6d0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.594+0000 7f013330a640 1 -- 172.21.15.45:0/2090694583 shutdown_connections 2024-05-08T05:49:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.594+0000 7f013330a640 1 --2- 172.21.15.45:0/2090694583 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f012c163920 0x7f012c06d4b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.594+0000 7f013330a640 1 -- 172.21.15.45:0/2090694583 >> 172.21.15.45:0/2090694583 conn(0x7f012c0d2e90 msgr2=0x7f012c0d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.594+0000 7f013330a640 1 -- 172.21.15.45:0/2090694583 shutdown_connections 2024-05-08T05:49:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.594+0000 7f013330a640 1 -- 172.21.15.45:0/2090694583 wait complete. 2024-05-08T05:49:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.595+0000 7f013330a640 1 Processor -- start 2024-05-08T05:49:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.595+0000 7f013330a640 1 -- start start 2024-05-08T05:49:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.596+0000 7f013330a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f012c070d30 0x7f012c0cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.596+0000 7f013330a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f012c0d20c0 0x7f012c0b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.596+0000 7f013330a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f012c163920 0x7f012c0b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.596+0000 7f013330a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f012c0d1c30 con 0x7f012c163920 2024-05-08T05:49:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.596+0000 7f013330a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f012c070b80 con 0x7f012c0d20c0 2024-05-08T05:49:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.596+0000 7f013330a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f012c06d9f0 con 0x7f012c070d30 2024-05-08T05:49:30.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.596+0000 7f01318a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f012c163920 0x7f012c0b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.596+0000 7f01318a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f012c163920 0x7f012c0b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45554/0 (socket says 172.21.15.45:45554) 2024-05-08T05:49:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.596+0000 7f01318a7640 1 -- 172.21.15.45:0/1047789475 learned_addr learned my addr 172.21.15.45:0/1047789475 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.596+0000 7f01308a5640 1 --2- 172.21.15.45:0/1047789475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f012c0d20c0 0x7f012c0b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.596+0000 7f01310a6640 1 --2- 172.21.15.45:0/1047789475 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f012c070d30 0x7f012c0cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.596+0000 7f01318a7640 1 -- 172.21.15.45:0/1047789475 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f012c070d30 msgr2=0x7f012c0cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.596+0000 7f01318a7640 1 --2- 172.21.15.45:0/1047789475 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f012c070d30 0x7f012c0cf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.596+0000 7f01318a7640 1 -- 172.21.15.45:0/1047789475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f012c0d20c0 msgr2=0x7f012c0b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.596+0000 7f01318a7640 1 --2- 172.21.15.45:0/1047789475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f012c0d20c0 0x7f012c0b0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.596+0000 7f01318a7640 1 -- 172.21.15.45:0/1047789475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012c0cd610 con 0x7f012c163920 2024-05-08T05:49:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.596+0000 7f01318a7640 1 --2- 172.21.15.45:0/1047789475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f012c163920 0x7f012c0b1450 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f0124009ee0 tx=0x7f0124004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.596+0000 7f0121ffb640 1 -- 172.21.15.45:0/1047789475 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012403f070 con 0x7f012c163920 2024-05-08T05:49:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.596+0000 7f0121ffb640 1 -- 172.21.15.45:0/1047789475 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0124031bc0 con 0x7f012c163920 2024-05-08T05:49:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.597+0000 7f013330a640 1 -- 172.21.15.45:0/1047789475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012c0cd8f0 con 0x7f012c163920 2024-05-08T05:49:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.597+0000 7f0121ffb640 1 -- 172.21.15.45:0/1047789475 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124009450 con 0x7f012c163920 2024-05-08T05:49:30.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.597+0000 7f013330a640 1 -- 172.21.15.45:0/1047789475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f012c0cdeb0 con 0x7f012c163920 2024-05-08T05:49:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.598+0000 7f013330a640 1 -- 172.21.15.45:0/1047789475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f012c0693f0 con 0x7f012c163920 2024-05-08T05:49:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.598+0000 7f0121ffb640 1 -- 172.21.15.45:0/1047789475 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f012403b410 con 0x7f012c163920 2024-05-08T05:49:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.599+0000 7f0121ffb640 1 --2- 172.21.15.45:0/1047789475 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f01100762d0 0x7f0110078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.599+0000 7f0121ffb640 1 -- 172.21.15.45:0/1047789475 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f01240b8a50 con 0x7f012c163920 2024-05-08T05:49:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.599+0000 7f01310a6640 1 --2- 172.21.15.45:0/1047789475 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f01100762d0 0x7f0110078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.600+0000 7f01310a6640 1 --2- 172.21.15.45:0/1047789475 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f01100762d0 0x7f0110078790 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0128005880 tx=0x7f0128009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:30.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.603+0000 7f0121ffb640 1 -- 172.21.15.45:0/1047789475 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0124081ab0 con 0x7f012c163920 2024-05-08T05:49:30.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.833+0000 7f013330a640 1 -- 172.21.15.45:0/1047789475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f012c06d140 con 0x7f012c163920 2024-05-08T05:49:30.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.833+0000 7f0121ffb640 1 -- 172.21.15.45:0/1047789475 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v83) ==== 53+0+73 (secure 0 0 0) 0x7f0124085db0 con 0x7f012c163920 2024-05-08T05:49:30.834 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T05:49:30.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.835+0000 7f013330a640 1 -- 172.21.15.45:0/1047789475 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f01100762d0 msgr2=0x7f0110078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:30.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.835+0000 7f013330a640 1 --2- 172.21.15.45:0/1047789475 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f01100762d0 0x7f0110078790 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0128005880 tx=0x7f0128009040 comp rx=0 tx=0).stop 2024-05-08T05:49:30.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.835+0000 7f013330a640 1 -- 172.21.15.45:0/1047789475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f012c163920 msgr2=0x7f012c0b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:30.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.835+0000 7f013330a640 1 --2- 172.21.15.45:0/1047789475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f012c163920 0x7f012c0b1450 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f0124009ee0 tx=0x7f0124004a40 comp rx=0 tx=0).stop 2024-05-08T05:49:30.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.836+0000 7f013330a640 1 -- 172.21.15.45:0/1047789475 shutdown_connections 2024-05-08T05:49:30.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.836+0000 7f013330a640 1 --2- 172.21.15.45:0/1047789475 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f01100762d0 0x7f0110078790 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.836+0000 7f013330a640 1 --2- 172.21.15.45:0/1047789475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f012c163920 0x7f012c0b1450 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.836+0000 7f013330a640 1 --2- 172.21.15.45:0/1047789475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f012c0d20c0 0x7f012c0b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.836+0000 7f013330a640 1 --2- 172.21.15.45:0/1047789475 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f012c070d30 0x7f012c0cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.836+0000 7f013330a640 1 -- 172.21.15.45:0/1047789475 >> 172.21.15.45:0/1047789475 conn(0x7f012c0d2e90 msgr2=0x7f012c0b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.836+0000 7f013330a640 1 -- 172.21.15.45:0/1047789475 shutdown_connections 2024-05-08T05:49:30.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:30.836+0000 7f013330a640 1 -- 172.21.15.45:0/1047789475 wait complete. 2024-05-08T05:49:30.849 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T05:49:30.850 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:49:30.850 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-05-08T05:49:30.850 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T05:49:30.850 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:49:30.850 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T05:49:30.850 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T05:49:30.850 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:49:30.850 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:49:30.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:30.850 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-05-08T05:49:30.868 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:49:30.868 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:49:30.868 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:49:30.869 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:49:30.869 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-05-08T05:49:30.869 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:49:30.869 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:49:30.869 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:49:30.869 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-05-08T05:49:30.870 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:49:30.870 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:49:30.870 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-05-08T05:49:30.870 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:49:30.870 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-05-08T05:49:30.871 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-05-08T05:49:30.871 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:49:30.871 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42254sec preferred_lft 42254sec 2024-05-08T05:49:30.871 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-05-08T05:49:30.871 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:49:30.871 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:49:30.872 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-05-08T05:49:30.872 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-05-08T05:49:30.872 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:49:30.872 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T05:49:30.872 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T05:49:30.872 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:49:30.873 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c80a:aff:fe94:c77/64 scope link 2024-05-08T05:49:30.873 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:49:30.873 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:49:30.873 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 82:f3:eb:6a:57:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:49:30.873 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::80f3:ebff:fe6a:57ba/64 scope link 2024-05-08T05:49:30.873 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:49:30.874 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:49:30.874 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T05:49:30.874 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::14a2:64ff:feda:3a9c/64 scope link 2024-05-08T05:49:30.874 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:49:30.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:30.875 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:49:30.933 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:49:30.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:30.936 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-05-08T05:49:30.993 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-05-08T05:49:30.993 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:49:30.993 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:49:30.993 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:49:30.993 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:49:30.994 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:49:30.994 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:49:30.994 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:49:30.994 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:49:30.994 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:49:30.994 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:49:31.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:49:31.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:49:31.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:31.123 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:49:31.139 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T05:49:31.139 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T05:49:31.139 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:49:31.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:31.139 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-05-08T05:49:31.205 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:49:31.205 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T05:49:31.261 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:49:31.262 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:49:31.262 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:49:31.317 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T05:49:31.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:31.318 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:49:31.402 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:49:31.402 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:49:31.410 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:49:31.410 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:49:31.410 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-08T05:49:31.411 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@e9e1170a-fd27-4625-97a9-5d5f104d13a2.cephfs=/ 2024-05-08T05:49:31.411 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.63:3300/172.21.15.63:3301" will pass to kernel 2024-05-08T05:49:31.411 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T05:49:31.411 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:49:31.411 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:49:31.411 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:49:31.411 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:49:31.413 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:49:31.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:31.413 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:49:31.439 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:49:31.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:49:31.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:49:31.542 INFO:teuthology.orchestra.run.smithi045.stdout:client6275 2024-05-08T05:49:31.542 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:49:31.542 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:49:31.559 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:49:31.559 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:49:31.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:31.615 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-08T05:49:31.710 INFO:teuthology.orchestra.run.smithi045.stderr:6+0 records in 2024-05-08T05:49:31.710 INFO:teuthology.orchestra.run.smithi045.stderr:6+0 records out 2024-05-08T05:49:31.710 INFO:teuthology.orchestra.run.smithi045.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0342213 s, 184 MB/s 2024-05-08T05:49:31.710 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:49:31.710 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:49:31.727 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:49:31.727 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:49:31.783 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:49:31.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.928+0000 7f3da081c640 1 Processor -- start 2024-05-08T05:49:31.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.928+0000 7f3da081c640 1 -- start start 2024-05-08T05:49:31.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.929+0000 7f3da081c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d9c0d53d0 0x7f3d9c0d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:31.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.929+0000 7f3da081c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d9c1630b0 con 0x7f3d9c0d1970 2024-05-08T05:49:31.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.929+0000 7f3da081c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d9c0d3e70 con 0x7f3d9c15f420 2024-05-08T05:49:31.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.929+0000 7f3da081c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f3d9c0d2550 con 0x7f3d9c0d53d0 2024-05-08T05:49:31.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.929+0000 7f3d9ad9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d9c0d53d0 0x7f3d9c0d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:31.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.930+0000 7f3d99d9b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3d9c0d1970 0x7f3d9c0d4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59940/0 (socket says 172.21.15.45:59940) 2024-05-08T05:49:31.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.930+0000 7f3d99d9b640 1 -- 172.21.15.45:0/2297883590 learned_addr learned my addr 172.21.15.45:0/2297883590 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:31.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.930+0000 7f3d9ad9d640 1 -- 172.21.15.45:0/2297883590 >> v1:172.21.15.63:6789/0 conn(0x7f3d9c15f420 legacy=0x7f3d9c0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:31.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.930+0000 7f3d9ad9d640 1 -- 172.21.15.45:0/2297883590 >> v1:172.21.15.45:6789/0 conn(0x7f3d9c0d1970 legacy=0x7f3d9c0d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:31.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.930+0000 7f3d9ad9d640 1 -- 172.21.15.45:0/2297883590 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9c0d2c80 con 0x7f3d9c0d53d0 2024-05-08T05:49:31.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.930+0000 7f3d9959a640 1 -- 172.21.15.45:0/2297883590 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1023285866 0 0) 0x7f3d84001110 con 0x7f3d9c0d1970 2024-05-08T05:49:31.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.931+0000 7f3d9ad9d640 1 --2- 172.21.15.45:0/2297883590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d9c0d53d0 0x7f3d9c0d1e20 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f3d940097a0 tx=0x7f3d9402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f476000d8c26b94b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:31.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.931+0000 7f3d9959a640 1 -- 172.21.15.45:0/2297883590 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94037070 con 0x7f3d9c0d53d0 2024-05-08T05:49:31.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.931+0000 7f3d9959a640 1 -- 172.21.15.45:0/2297883590 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d94032440 con 0x7f3d9c0d53d0 2024-05-08T05:49:31.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.931+0000 7f3d9959a640 1 -- 172.21.15.45:0/2297883590 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94032810 con 0x7f3d9c0d53d0 2024-05-08T05:49:31.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.931+0000 7f3da081c640 1 -- 172.21.15.45:0/2297883590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d9c0d53d0 msgr2=0x7f3d9c0d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:31.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.931+0000 7f3da081c640 1 --2- 172.21.15.45:0/2297883590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d9c0d53d0 0x7f3d9c0d1e20 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f3d940097a0 tx=0x7f3d9402a180 comp rx=0 tx=0).stop 2024-05-08T05:49:31.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.931+0000 7f3da081c640 1 -- 172.21.15.45:0/2297883590 shutdown_connections 2024-05-08T05:49:31.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.931+0000 7f3da081c640 1 --2- 172.21.15.45:0/2297883590 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d9c0d53d0 0x7f3d9c0d1e20 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:31.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.931+0000 7f3da081c640 1 -- 172.21.15.45:0/2297883590 >> 172.21.15.45:0/2297883590 conn(0x7f3d9c08eac0 msgr2=0x7f3d9c09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:31.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.931+0000 7f3da081c640 1 -- 172.21.15.45:0/2297883590 shutdown_connections 2024-05-08T05:49:31.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.931+0000 7f3da081c640 1 -- 172.21.15.45:0/2297883590 wait complete. 2024-05-08T05:49:31.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.933+0000 7f3da081c640 1 Processor -- start 2024-05-08T05:49:31.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.933+0000 7f3da081c640 1 -- start start 2024-05-08T05:49:31.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.934+0000 7f3da081c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d9c0d1970 0x7f3d9c14bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:31.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.934+0000 7f3da081c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d9c0d53d0 0x7f3d9c14d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:31.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.934+0000 7f3da081c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d9c06dd70 0x7f3d9c06e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:31.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.934+0000 7f3da081c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c0d2830 con 0x7f3d9c06dd70 2024-05-08T05:49:31.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.934+0000 7f3da081c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c0d3240 con 0x7f3d9c0d53d0 2024-05-08T05:49:31.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.934+0000 7f3da081c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3d9c0cfb20 con 0x7f3d9c0d1970 2024-05-08T05:49:31.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.934+0000 7f3d9ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d9c06dd70 0x7f3d9c06e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:31.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.934+0000 7f3d9ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d9c06dd70 0x7f3d9c06e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37658/0 (socket says 172.21.15.45:37658) 2024-05-08T05:49:31.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.934+0000 7f3d9ad9d640 1 -- 172.21.15.45:0/1884242939 learned_addr learned my addr 172.21.15.45:0/1884242939 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:31.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.934+0000 7f3d99d9b640 1 --2- 172.21.15.45:0/1884242939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d9c0d53d0 0x7f3d9c14d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:31.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.934+0000 7f3d9a59c640 1 --2- 172.21.15.45:0/1884242939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d9c0d1970 0x7f3d9c14bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:31.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.934+0000 7f3d9ad9d640 1 -- 172.21.15.45:0/1884242939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d9c0d1970 msgr2=0x7f3d9c14bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:31.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.934+0000 7f3d9ad9d640 1 --2- 172.21.15.45:0/1884242939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d9c0d1970 0x7f3d9c14bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:31.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.934+0000 7f3d9ad9d640 1 -- 172.21.15.45:0/1884242939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d9c0d53d0 msgr2=0x7f3d9c14d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:31.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.934+0000 7f3d9ad9d640 1 --2- 172.21.15.45:0/1884242939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d9c0d53d0 0x7f3d9c14d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:31.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.934+0000 7f3d9ad9d640 1 -- 172.21.15.45:0/1884242939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9c14c600 con 0x7f3d9c06dd70 2024-05-08T05:49:31.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.934+0000 7f3d9ad9d640 1 --2- 172.21.15.45:0/1884242939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d9c06dd70 0x7f3d9c06e200 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f3d940097a0 tx=0x7f3d940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:31.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.935+0000 7f3d92ffd640 1 -- 172.21.15.45:0/1884242939 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94037040 con 0x7f3d9c06dd70 2024-05-08T05:49:31.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.935+0000 7f3d92ffd640 1 -- 172.21.15.45:0/1884242939 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d94004230 con 0x7f3d9c06dd70 2024-05-08T05:49:31.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.935+0000 7f3da081c640 1 -- 172.21.15.45:0/1884242939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d9c14c8e0 con 0x7f3d9c06dd70 2024-05-08T05:49:31.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.935+0000 7f3d92ffd640 1 -- 172.21.15.45:0/1884242939 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94032e20 con 0x7f3d9c06dd70 2024-05-08T05:49:31.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.935+0000 7f3da081c640 1 -- 172.21.15.45:0/1884242939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d9c14cea0 con 0x7f3d9c06dd70 2024-05-08T05:49:31.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.936+0000 7f3da081c640 1 -- 172.21.15.45:0/1884242939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d9c0d3760 con 0x7f3d9c06dd70 2024-05-08T05:49:31.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.936+0000 7f3d92ffd640 1 -- 172.21.15.45:0/1884242939 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3d94030770 con 0x7f3d9c06dd70 2024-05-08T05:49:31.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.937+0000 7f3d92ffd640 1 --2- 172.21.15.45:0/1884242939 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3d7c0762d0 0x7f3d7c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:31.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.937+0000 7f3d92ffd640 1 -- 172.21.15.45:0/1884242939 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f3d940b8920 con 0x7f3d9c06dd70 2024-05-08T05:49:31.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.937+0000 7f3d9a59c640 1 --2- 172.21.15.45:0/1884242939 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3d7c0762d0 0x7f3d7c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:31.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.937+0000 7f3d9a59c640 1 --2- 172.21.15.45:0/1884242939 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3d7c0762d0 0x7f3d7c078790 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c00b450 tx=0x7f3d8c00d040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:31.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:31.941+0000 7f3d92ffd640 1 -- 172.21.15.45:0/1884242939 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3d9c0d3760 con 0x7f3d9c06dd70 2024-05-08T05:49:32.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.144+0000 7f3da081c640 1 -- 172.21.15.45:0/1884242939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3d9c0693f0 con 0x7f3d9c06dd70 2024-05-08T05:49:32.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.145+0000 7f3d92ffd640 1 -- 172.21.15.45:0/1884242939 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v104) ==== 80+0+0 (secure 0 0 0) 0x7f3d94081a10 con 0x7f3d9c06dd70 2024-05-08T05:49:32.145 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-05-08T05:49:32.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.147+0000 7f3da081c640 1 -- 172.21.15.45:0/1884242939 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3d7c0762d0 msgr2=0x7f3d7c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:32.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.147+0000 7f3da081c640 1 --2- 172.21.15.45:0/1884242939 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3d7c0762d0 0x7f3d7c078790 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c00b450 tx=0x7f3d8c00d040 comp rx=0 tx=0).stop 2024-05-08T05:49:32.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.147+0000 7f3da081c640 1 -- 172.21.15.45:0/1884242939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d9c06dd70 msgr2=0x7f3d9c06e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:32.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.147+0000 7f3da081c640 1 --2- 172.21.15.45:0/1884242939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d9c06dd70 0x7f3d9c06e200 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f3d940097a0 tx=0x7f3d940046c0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.147+0000 7f3da081c640 1 -- 172.21.15.45:0/1884242939 shutdown_connections 2024-05-08T05:49:32.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.147+0000 7f3da081c640 1 --2- 172.21.15.45:0/1884242939 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3d7c0762d0 0x7f3d7c078790 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.147+0000 7f3da081c640 1 --2- 172.21.15.45:0/1884242939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d9c06dd70 0x7f3d9c06e200 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.147+0000 7f3da081c640 1 --2- 172.21.15.45:0/1884242939 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d9c0d53d0 0x7f3d9c14d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.147+0000 7f3da081c640 1 --2- 172.21.15.45:0/1884242939 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d9c0d1970 0x7f3d9c14bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.147+0000 7f3da081c640 1 -- 172.21.15.45:0/1884242939 >> 172.21.15.45:0/1884242939 conn(0x7f3d9c08eac0 msgr2=0x7f3d9c06cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:32.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.147+0000 7f3da081c640 1 -- 172.21.15.45:0/1884242939 shutdown_connections 2024-05-08T05:49:32.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.147+0000 7f3da081c640 1 -- 172.21.15.45:0/1884242939 wait complete. 2024-05-08T05:49:32.157 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T05:49:32.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:32.158 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T05:49:32.190 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:49:32.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:32.190 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T05:49:32.248 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:49:32.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.391+0000 7fbe21064640 1 Processor -- start 2024-05-08T05:49:32.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.391+0000 7fbe21064640 1 -- start start 2024-05-08T05:49:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.392+0000 7fbe21064640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbe1c06d620 0x7fbe1c073800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.392+0000 7fbe21064640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe1c06da00 0x7fbe1c0af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.392+0000 7fbe21064640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbe1c0b5ee0 0x7fbe1c06e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.392+0000 7fbe21064640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fbe1c0afaa0 con 0x7fbe1c0b5ee0 2024-05-08T05:49:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.392+0000 7fbe21064640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbe1c0b5a50 con 0x7fbe1c06d620 2024-05-08T05:49:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.392+0000 7fbe21064640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbe1c0af800 con 0x7fbe1c06da00 2024-05-08T05:49:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.392+0000 7fbe1b59e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbe1c0b5ee0 0x7fbe1c06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.392+0000 7fbe1b59e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbe1c0b5ee0 0x7fbe1c06e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37670/0 (socket says 172.21.15.45:37670) 2024-05-08T05:49:32.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.392+0000 7fbe1b59e640 1 -- 172.21.15.45:0/698592821 learned_addr learned my addr 172.21.15.45:0/698592821 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.392+0000 7fbe1ad9d640 1 --2- 172.21.15.45:0/698592821 >> v2:172.21.15.63:3300/0 conn(0x7fbe1c06d620 0x7fbe1c073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.393+0000 7fbe1b59e640 1 -- 172.21.15.45:0/698592821 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe1c06da00 msgr2=0x7fbe1c0af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.393+0000 7fbe1b59e640 1 --2- 172.21.15.45:0/698592821 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe1c06da00 0x7fbe1c0af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.393+0000 7fbe1b59e640 1 -- 172.21.15.45:0/698592821 >> v2:172.21.15.63:3300/0 conn(0x7fbe1c06d620 msgr2=0x7fbe1c073800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.393+0000 7fbe1b59e640 1 --2- 172.21.15.45:0/698592821 >> v2:172.21.15.63:3300/0 conn(0x7fbe1c06d620 0x7fbe1c073800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.393+0000 7fbe1b59e640 1 -- 172.21.15.45:0/698592821 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1c0b5120 con 0x7fbe1c0b5ee0 2024-05-08T05:49:32.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.393+0000 7fbe1b59e640 1 --2- 172.21.15.45:0/698592821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe1c0b5ee0 0x7fbe1c06e920 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c069030 tx=0x7fbe1402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab54c213e5de5b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:32.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.393+0000 7fbe19d9b640 1 -- 172.21.15.45:0/698592821 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14037070 con 0x7fbe1c0b5ee0 2024-05-08T05:49:32.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.394+0000 7fbe19d9b640 1 -- 172.21.15.45:0/698592821 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe1402ae00 con 0x7fbe1c0b5ee0 2024-05-08T05:49:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.394+0000 7fbe19d9b640 1 -- 172.21.15.45:0/698592821 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14032660 con 0x7fbe1c0b5ee0 2024-05-08T05:49:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.394+0000 7fbe21064640 1 -- 172.21.15.45:0/698592821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe1c0b5ee0 msgr2=0x7fbe1c06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.394+0000 7fbe21064640 1 --2- 172.21.15.45:0/698592821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe1c0b5ee0 0x7fbe1c06e920 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c069030 tx=0x7fbe1402a0b0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.394+0000 7fbe21064640 1 -- 172.21.15.45:0/698592821 shutdown_connections 2024-05-08T05:49:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.394+0000 7fbe21064640 1 --2- 172.21.15.45:0/698592821 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe1c0b5ee0 0x7fbe1c06e920 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.394+0000 7fbe21064640 1 --2- 172.21.15.45:0/698592821 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe1c06da00 0x7fbe1c0af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.394+0000 7fbe21064640 1 --2- 172.21.15.45:0/698592821 >> v2:172.21.15.63:3300/0 conn(0x7fbe1c06d620 0x7fbe1c073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.394+0000 7fbe21064640 1 -- 172.21.15.45:0/698592821 >> 172.21.15.45:0/698592821 conn(0x7fbe1c0728c0 msgr2=0x7fbe1c072cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.394+0000 7fbe21064640 1 -- 172.21.15.45:0/698592821 shutdown_connections 2024-05-08T05:49:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.394+0000 7fbe21064640 1 -- 172.21.15.45:0/698592821 wait complete. 2024-05-08T05:49:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.396+0000 7fbe21064640 1 Processor -- start 2024-05-08T05:49:32.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.396+0000 7fbe21064640 1 -- start start 2024-05-08T05:49:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.396+0000 7fbe21064640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe1c06d620 0x7fbe1c1e99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.396+0000 7fbe21064640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe1c06da00 0x7fbe1c141c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.396+0000 7fbe21064640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe1c0b5ee0 0x7fbe1c142140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.396+0000 7fbe21064640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c0b6370 con 0x7fbe1c0b5ee0 2024-05-08T05:49:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.396+0000 7fbe21064640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c0b56e0 con 0x7fbe1c06d620 2024-05-08T05:49:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.396+0000 7fbe21064640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbe1c0b64f0 con 0x7fbe1c06da00 2024-05-08T05:49:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.396+0000 7fbe1b59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe1c0b5ee0 0x7fbe1c142140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.397+0000 7fbe1b59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe1c0b5ee0 0x7fbe1c142140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37676/0 (socket says 172.21.15.45:37676) 2024-05-08T05:49:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.397+0000 7fbe1b59e640 1 -- 172.21.15.45:0/1657799374 learned_addr learned my addr 172.21.15.45:0/1657799374 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.397+0000 7fbe1ad9d640 1 --2- 172.21.15.45:0/1657799374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe1c06d620 0x7fbe1c1e99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:32.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.397+0000 7fbe1a59c640 1 --2- 172.21.15.45:0/1657799374 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe1c06da00 0x7fbe1c141c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.397+0000 7fbe1b59e640 1 -- 172.21.15.45:0/1657799374 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe1c06da00 msgr2=0x7fbe1c141c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.397+0000 7fbe1b59e640 1 --2- 172.21.15.45:0/1657799374 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe1c06da00 0x7fbe1c141c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.397+0000 7fbe1b59e640 1 -- 172.21.15.45:0/1657799374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe1c06d620 msgr2=0x7fbe1c1e99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.397+0000 7fbe1b59e640 1 --2- 172.21.15.45:0/1657799374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe1c06d620 0x7fbe1c1e99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.397+0000 7fbe1b59e640 1 -- 172.21.15.45:0/1657799374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1c1e7ea0 con 0x7fbe1c0b5ee0 2024-05-08T05:49:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.397+0000 7fbe1ad9d640 1 --2- 172.21.15.45:0/1657799374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe1c06d620 0x7fbe1c1e99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.397+0000 7fbe1a59c640 1 --2- 172.21.15.45:0/1657799374 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe1c06da00 0x7fbe1c141c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.397+0000 7fbe1b59e640 1 --2- 172.21.15.45:0/1657799374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe1c0b5ee0 0x7fbe1c142140 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fbe1402a0f0 tx=0x7fbe140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.397+0000 7fbe137fe640 1 -- 172.21.15.45:0/1657799374 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14041070 con 0x7fbe1c0b5ee0 2024-05-08T05:49:32.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.397+0000 7fbe137fe640 1 -- 172.21.15.45:0/1657799374 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe1403c670 con 0x7fbe1c0b5ee0 2024-05-08T05:49:32.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.397+0000 7fbe21064640 1 -- 172.21.15.45:0/1657799374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1c1e8180 con 0x7fbe1c0b5ee0 2024-05-08T05:49:32.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.397+0000 7fbe137fe640 1 -- 172.21.15.45:0/1657799374 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14037040 con 0x7fbe1c0b5ee0 2024-05-08T05:49:32.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.397+0000 7fbe21064640 1 -- 172.21.15.45:0/1657799374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe1c1e8740 con 0x7fbe1c0b5ee0 2024-05-08T05:49:32.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.398+0000 7fbe21064640 1 -- 172.21.15.45:0/1657799374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe1c13f890 con 0x7fbe1c0b5ee0 2024-05-08T05:49:32.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.399+0000 7fbe137fe640 1 -- 172.21.15.45:0/1657799374 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbe1403a070 con 0x7fbe1c0b5ee0 2024-05-08T05:49:32.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.400+0000 7fbe137fe640 1 --2- 172.21.15.45:0/1657799374 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbdfc0762d0 0x7fbdfc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:32.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.400+0000 7fbe137fe640 1 -- 172.21.15.45:0/1657799374 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fbe140b7c60 con 0x7fbe1c0b5ee0 2024-05-08T05:49:32.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.400+0000 7fbe1ad9d640 1 --2- 172.21.15.45:0/1657799374 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbdfc0762d0 0x7fbdfc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:32.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.400+0000 7fbe1ad9d640 1 --2- 172.21.15.45:0/1657799374 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbdfc0762d0 0x7fbdfc078790 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c007fc0 tx=0x7fbe0c0075b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:32.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.404+0000 7fbe137fe640 1 -- 172.21.15.45:0/1657799374 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbe1c13f890 con 0x7fbe1c0b5ee0 2024-05-08T05:49:32.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.634+0000 7fbe21064640 1 -- 172.21.15.45:0/1657799374 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe1c0693f0 con 0x7fbe1c0b5ee0 2024-05-08T05:49:32.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.634+0000 7fbe137fe640 1 -- 172.21.15.45:0/1657799374 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5180 (secure 0 0 0) 0x7fbe140814c0 con 0x7fbe1c0b5ee0 2024-05-08T05:49:32.635 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:32.635 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3692881261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3692881261},{"type":"v1","addr":"172.21.15.63:6836","nonce":3692881261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7497,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2916399818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2916399818},{"type":"v1","addr":"172.21.15.45:6833","nonce":2916399818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7515,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2978250004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":2978250004},{"type":"v1","addr":"172.21.15.63:6837","nonce":2978250004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:49:19.587326+0000","modified":"2024-05-08T05:49:22.599177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7524":{"gid":7524,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/878092981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":878092981},{"type":"v1","addr":"172.21.15.45:6835","nonce":878092981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7524,"qdb_cluster":[7524]},"id":7}]} 2024-05-08T05:49:32.635 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 83 2024-05-08T05:49:32.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.637+0000 7fbe21064640 1 -- 172.21.15.45:0/1657799374 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbdfc0762d0 msgr2=0x7fbdfc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:32.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.637+0000 7fbe21064640 1 --2- 172.21.15.45:0/1657799374 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbdfc0762d0 0x7fbdfc078790 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c007fc0 tx=0x7fbe0c0075b0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.637+0000 7fbe21064640 1 -- 172.21.15.45:0/1657799374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe1c0b5ee0 msgr2=0x7fbe1c142140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:32.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.637+0000 7fbe21064640 1 --2- 172.21.15.45:0/1657799374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe1c0b5ee0 0x7fbe1c142140 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fbe1402a0f0 tx=0x7fbe140046c0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.637+0000 7fbe21064640 1 -- 172.21.15.45:0/1657799374 shutdown_connections 2024-05-08T05:49:32.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.637+0000 7fbe21064640 1 --2- 172.21.15.45:0/1657799374 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbdfc0762d0 0x7fbdfc078790 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.637+0000 7fbe21064640 1 --2- 172.21.15.45:0/1657799374 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe1c06da00 0x7fbe1c141c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.637+0000 7fbe21064640 1 --2- 172.21.15.45:0/1657799374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe1c0b5ee0 0x7fbe1c142140 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.637+0000 7fbe21064640 1 --2- 172.21.15.45:0/1657799374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe1c06d620 0x7fbe1c1e99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.637+0000 7fbe21064640 1 -- 172.21.15.45:0/1657799374 >> 172.21.15.45:0/1657799374 conn(0x7fbe1c0728c0 msgr2=0x7fbe1c155530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:32.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.637+0000 7fbe21064640 1 -- 172.21.15.45:0/1657799374 shutdown_connections 2024-05-08T05:49:32.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.637+0000 7fbe21064640 1 -- 172.21.15.45:0/1657799374 wait complete. 2024-05-08T05:49:32.648 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-08T05:49:32.775 INFO:teuthology.orchestra.run.smithi045.stdout:{"message":"","return_code":0} 2024-05-08T05:49:32.775 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-08T05:49:32.776 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:49:32.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.879+0000 7f7d218c5640 1 Processor -- start 2024-05-08T05:49:32.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.879+0000 7f7d218c5640 1 -- start start 2024-05-08T05:49:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.880+0000 7f7d218c5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d1c15ccb0 0x7f7d1c15d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.880+0000 7f7d218c5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d1c070b90 0x7f7d1c071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.880+0000 7f7d218c5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d1c0718c0 con 0x7f7d1c0d2ce0 2024-05-08T05:49:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.880+0000 7f7d218c5640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f7d1c0afba0 con 0x7f7d1c15ccb0 2024-05-08T05:49:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.880+0000 7f7d218c5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7d1c071ba0 con 0x7f7d1c070b90 2024-05-08T05:49:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.880+0000 7f7d1affd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d1c15ccb0 0x7f7d1c15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.880+0000 7f7d1affd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d1c15ccb0 0x7f7d1c15d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37692/0 (socket says 172.21.15.45:37692) 2024-05-08T05:49:32.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.880+0000 7f7d1affd640 1 -- 172.21.15.45:0/2758345342 learned_addr learned my addr 172.21.15.45:0/2758345342 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.881+0000 7f7d1affd640 1 -- 172.21.15.45:0/2758345342 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d1c070b90 msgr2=0x7f7d1c071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.881+0000 7f7d1affd640 1 --2- 172.21.15.45:0/2758345342 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d1c070b90 0x7f7d1c071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.881+0000 7f7d1affd640 1 -- 172.21.15.45:0/2758345342 >> v1:172.21.15.63:6789/0 conn(0x7f7d1c0d2ce0 legacy=0x7f7d1c0d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:32.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.881+0000 7f7d1affd640 1 -- 172.21.15.45:0/2758345342 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1c071e80 con 0x7f7d1c15ccb0 2024-05-08T05:49:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.881+0000 7f7d1affd640 1 --2- 172.21.15.45:0/2758345342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1c15ccb0 0x7f7d1c15d090 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009e50 tx=0x7f7d0c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3539299da2d259eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.882+0000 7f7d19ffb640 1 -- 172.21.15.45:0/2758345342 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c036070 con 0x7f7d1c15ccb0 2024-05-08T05:49:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.882+0000 7f7d19ffb640 1 -- 172.21.15.45:0/2758345342 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d0c031600 con 0x7f7d1c15ccb0 2024-05-08T05:49:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.882+0000 7f7d19ffb640 1 -- 172.21.15.45:0/2758345342 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c0319d0 con 0x7f7d1c15ccb0 2024-05-08T05:49:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.882+0000 7f7d218c5640 1 -- 172.21.15.45:0/2758345342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1c15ccb0 msgr2=0x7f7d1c15d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.882+0000 7f7d218c5640 1 --2- 172.21.15.45:0/2758345342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1c15ccb0 0x7f7d1c15d090 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009e50 tx=0x7f7d0c02a680 comp rx=0 tx=0).stop 2024-05-08T05:49:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.882+0000 7f7d218c5640 1 -- 172.21.15.45:0/2758345342 shutdown_connections 2024-05-08T05:49:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.882+0000 7f7d218c5640 1 --2- 172.21.15.45:0/2758345342 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d1c070b90 0x7f7d1c071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.882+0000 7f7d218c5640 1 --2- 172.21.15.45:0/2758345342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1c15ccb0 0x7f7d1c15d090 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.882+0000 7f7d218c5640 1 -- 172.21.15.45:0/2758345342 >> 172.21.15.45:0/2758345342 conn(0x7f7d1c09ba40 msgr2=0x7f7d1c09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.882+0000 7f7d218c5640 1 -- 172.21.15.45:0/2758345342 shutdown_connections 2024-05-08T05:49:32.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.882+0000 7f7d218c5640 1 -- 172.21.15.45:0/2758345342 wait complete. 2024-05-08T05:49:32.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.883+0000 7f7d218c5640 1 Processor -- start 2024-05-08T05:49:32.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.884+0000 7f7d218c5640 1 -- start start 2024-05-08T05:49:32.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.884+0000 7f7d218c5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d1c070b90 0x7f7d1c15f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:32.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.884+0000 7f7d218c5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d1c0d2ce0 0x7f7d1c150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:32.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.884+0000 7f7d218c5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1c15ccb0 0x7f7d1c0c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:32.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.884+0000 7f7d218c5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c0cf980 con 0x7f7d1c15ccb0 2024-05-08T05:49:32.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.884+0000 7f7d218c5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c071560 con 0x7f7d1c0d2ce0 2024-05-08T05:49:32.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.884+0000 7f7d218c5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7d1c0cfb00 con 0x7f7d1c070b90 2024-05-08T05:49:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.884+0000 7f7d1b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1c15ccb0 0x7f7d1c0c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.885+0000 7f7d1b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1c15ccb0 0x7f7d1c0c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37702/0 (socket says 172.21.15.45:37702) 2024-05-08T05:49:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.885+0000 7f7d1b7fe640 1 -- 172.21.15.45:0/1289055307 learned_addr learned my addr 172.21.15.45:0/1289055307 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.885+0000 7f7d1affd640 1 --2- 172.21.15.45:0/1289055307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d1c070b90 0x7f7d1c15f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.885+0000 7f7d1a7fc640 1 --2- 172.21.15.45:0/1289055307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d1c0d2ce0 0x7f7d1c150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.885+0000 7f7d1b7fe640 1 -- 172.21.15.45:0/1289055307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d1c070b90 msgr2=0x7f7d1c15f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.885+0000 7f7d1b7fe640 1 --2- 172.21.15.45:0/1289055307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d1c070b90 0x7f7d1c15f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.885+0000 7f7d1b7fe640 1 -- 172.21.15.45:0/1289055307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d1c0d2ce0 msgr2=0x7f7d1c150170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.885+0000 7f7d1b7fe640 1 --2- 172.21.15.45:0/1289055307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d1c0d2ce0 0x7f7d1c150170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.885+0000 7f7d1b7fe640 1 -- 172.21.15.45:0/1289055307 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1c0b5f80 con 0x7f7d1c15ccb0 2024-05-08T05:49:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.885+0000 7f7d1b7fe640 1 --2- 172.21.15.45:0/1289055307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1c15ccb0 0x7f7d1c0c8730 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400b410 tx=0x7f7d1400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.885+0000 7f7d03fff640 1 -- 172.21.15.45:0/1289055307 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14015020 con 0x7f7d1c15ccb0 2024-05-08T05:49:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.885+0000 7f7d03fff640 1 -- 172.21.15.45:0/1289055307 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d14011850 con 0x7f7d1c15ccb0 2024-05-08T05:49:32.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.885+0000 7f7d218c5640 1 -- 172.21.15.45:0/1289055307 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1c15d090 con 0x7f7d1c15ccb0 2024-05-08T05:49:32.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.885+0000 7f7d03fff640 1 -- 172.21.15.45:0/1289055307 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14011c40 con 0x7f7d1c15ccb0 2024-05-08T05:49:32.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.885+0000 7f7d218c5640 1 -- 172.21.15.45:0/1289055307 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d1c15d650 con 0x7f7d1c15ccb0 2024-05-08T05:49:32.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.886+0000 7f7d218c5640 1 -- 172.21.15.45:0/1289055307 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d1c069b00 con 0x7f7d1c15ccb0 2024-05-08T05:49:32.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.887+0000 7f7d03fff640 1 -- 172.21.15.45:0/1289055307 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7d14018020 con 0x7f7d1c15ccb0 2024-05-08T05:49:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.887+0000 7f7d03fff640 1 --2- 172.21.15.45:0/1289055307 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7d080762d0 0x7f7d08078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.887+0000 7f7d03fff640 1 -- 172.21.15.45:0/1289055307 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f7d14097570 con 0x7f7d1c15ccb0 2024-05-08T05:49:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.888+0000 7f7d1affd640 1 --2- 172.21.15.45:0/1289055307 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7d080762d0 0x7f7d08078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.888+0000 7f7d1affd640 1 --2- 172.21.15.45:0/1289055307 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7d080762d0 0x7f7d08078790 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c033000 tx=0x7f7d0c009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:32.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:32.892+0000 7f7d03fff640 1 -- 172.21.15.45:0/1289055307 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7d1401f020 con 0x7f7d1c15ccb0 2024-05-08T05:49:33.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.121+0000 7f7d218c5640 1 -- 172.21.15.45:0/1289055307 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d1c070380 con 0x7f7d1c15ccb0 2024-05-08T05:49:33.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.122+0000 7f7d03fff640 1 -- 172.21.15.45:0/1289055307 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5180 (secure 0 0 0) 0x7f7d140606e0 con 0x7f7d1c15ccb0 2024-05-08T05:49:33.122 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:33.122 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3692881261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3692881261},{"type":"v1","addr":"172.21.15.63:6836","nonce":3692881261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7497,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2916399818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2916399818},{"type":"v1","addr":"172.21.15.45:6833","nonce":2916399818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7515,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2978250004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":2978250004},{"type":"v1","addr":"172.21.15.63:6837","nonce":2978250004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:49:19.587326+0000","modified":"2024-05-08T05:49:22.599177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7524":{"gid":7524,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/878092981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":878092981},{"type":"v1","addr":"172.21.15.45:6835","nonce":878092981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7524,"qdb_cluster":[7524]},"id":7}]} 2024-05-08T05:49:33.123 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 83 2024-05-08T05:49:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.124+0000 7f7d218c5640 1 -- 172.21.15.45:0/1289055307 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7d080762d0 msgr2=0x7f7d08078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.124+0000 7f7d218c5640 1 --2- 172.21.15.45:0/1289055307 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7d080762d0 0x7f7d08078790 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c033000 tx=0x7f7d0c009000 comp rx=0 tx=0).stop 2024-05-08T05:49:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.124+0000 7f7d218c5640 1 -- 172.21.15.45:0/1289055307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1c15ccb0 msgr2=0x7f7d1c0c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.124+0000 7f7d218c5640 1 --2- 172.21.15.45:0/1289055307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1c15ccb0 0x7f7d1c0c8730 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400b410 tx=0x7f7d1400b8d0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.124+0000 7f7d218c5640 1 -- 172.21.15.45:0/1289055307 shutdown_connections 2024-05-08T05:49:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.124+0000 7f7d218c5640 1 --2- 172.21.15.45:0/1289055307 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7d080762d0 0x7f7d08078790 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.124+0000 7f7d218c5640 1 --2- 172.21.15.45:0/1289055307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d1c15ccb0 0x7f7d1c0c8730 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.124+0000 7f7d218c5640 1 --2- 172.21.15.45:0/1289055307 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d1c0d2ce0 0x7f7d1c150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.124+0000 7f7d218c5640 1 --2- 172.21.15.45:0/1289055307 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d1c070b90 0x7f7d1c15f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.124+0000 7f7d218c5640 1 -- 172.21.15.45:0/1289055307 >> 172.21.15.45:0/1289055307 conn(0x7f7d1c09ba40 msgr2=0x7f7d1c0ceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.124+0000 7f7d218c5640 1 -- 172.21.15.45:0/1289055307 shutdown_connections 2024-05-08T05:49:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.124+0000 7f7d218c5640 1 -- 172.21.15.45:0/1289055307 wait complete. 2024-05-08T05:49:33.135 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-05-08T05:49:33.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.179+0000 7f27a9a22f40 1 Processor -- start 2024-05-08T05:49:33.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.179+0000 7f27a9a22f40 1 -- start start 2024-05-08T05:49:33.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.180+0000 7f27a9a22f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d014ff9a20 0x55d0150d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:33.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.180+0000 7f27a9a22f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0150444d0 0x55d0150c9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:33.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.180+0000 7f27a9a22f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0150d26d0 0x55d0150c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:33.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.180+0000 7f27a9a22f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d014ffc800 con 0x55d014ff9a20 2024-05-08T05:49:33.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.180+0000 7f27a9a22f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d014fa2f50 con 0x55d0150444d0 2024-05-08T05:49:33.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.180+0000 7f27a9a22f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d014ffcae0 con 0x55d0150d26d0 2024-05-08T05:49:33.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.180+0000 7f27a751c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d014ff9a20 0x55d0150d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:33.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.180+0000 7f27a751c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d014ff9a20 0x55d0150d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37712/0 (socket says 172.21.15.45:37712) 2024-05-08T05:49:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.180+0000 7f27a751c640 1 -- 172.21.15.45:0/4242724318 learned_addr learned my addr 172.21.15.45:0/4242724318 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.180+0000 7f27a6d1b640 1 --2- 172.21.15.45:0/4242724318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0150444d0 0x55d0150c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.180+0000 7f27a7d1d640 1 --2- 172.21.15.45:0/4242724318 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0150d26d0 0x55d0150c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.180+0000 7f27a751c640 1 -- 172.21.15.45:0/4242724318 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0150d26d0 msgr2=0x55d0150c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.180+0000 7f27a751c640 1 --2- 172.21.15.45:0/4242724318 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0150d26d0 0x55d0150c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.180+0000 7f27a751c640 1 -- 172.21.15.45:0/4242724318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0150444d0 msgr2=0x55d0150c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.180+0000 7f27a751c640 1 --2- 172.21.15.45:0/4242724318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0150444d0 0x55d0150c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.180+0000 7f27a751c640 1 -- 172.21.15.45:0/4242724318 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d015028a70 con 0x55d014ff9a20 2024-05-08T05:49:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.180+0000 7f27a6d1b640 1 --2- 172.21.15.45:0/4242724318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0150444d0 0x55d0150c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.180+0000 7f27a751c640 1 --2- 172.21.15.45:0/4242724318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d014ff9a20 0x55d0150d0f40 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f2798001e70 tx=0x7f279800b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29bfab0da5f00886 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.181+0000 7f27a5d19640 1 -- 172.21.15.45:0/4242724318 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798012020 con 0x55d014ff9a20 2024-05-08T05:49:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.181+0000 7f27a5d19640 1 -- 172.21.15.45:0/4242724318 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f279800e670 con 0x55d014ff9a20 2024-05-08T05:49:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.181+0000 7f27a5d19640 1 -- 172.21.15.45:0/4242724318 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279800eaa0 con 0x55d014ff9a20 2024-05-08T05:49:33.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.181+0000 7f27a9a22f40 1 -- 172.21.15.45:0/4242724318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d014ff9a20 msgr2=0x55d0150d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:33.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.181+0000 7f27a9a22f40 1 --2- 172.21.15.45:0/4242724318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d014ff9a20 0x55d0150d0f40 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f2798001e70 tx=0x7f279800b780 comp rx=0 tx=0).stop 2024-05-08T05:49:33.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.181+0000 7f27a9a22f40 1 -- 172.21.15.45:0/4242724318 shutdown_connections 2024-05-08T05:49:33.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.181+0000 7f27a9a22f40 1 --2- 172.21.15.45:0/4242724318 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0150d26d0 0x55d0150c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.181+0000 7f27a9a22f40 1 --2- 172.21.15.45:0/4242724318 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d0150444d0 0x55d0150c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.181+0000 7f27a9a22f40 1 --2- 172.21.15.45:0/4242724318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d014ff9a20 0x55d0150d0f40 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.181+0000 7f27a9a22f40 1 -- 172.21.15.45:0/4242724318 >> 172.21.15.45:0/4242724318 conn(0x55d014ffab30 msgr2=0x55d0150cde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:33.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.181+0000 7f27a9a22f40 1 -- 172.21.15.45:0/4242724318 shutdown_connections 2024-05-08T05:49:33.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.181+0000 7f27a9a22f40 1 -- 172.21.15.45:0/4242724318 wait complete. 2024-05-08T05:49:33.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.182+0000 7f27a9a22f40 1 Processor -- start 2024-05-08T05:49:33.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.182+0000 7f27a9a22f40 1 -- start start 2024-05-08T05:49:33.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.182+0000 7f27a9a22f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d014ff9a20 0x55d0150c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:33.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.182+0000 7f27a9a22f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0150444d0 0x55d01516d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:33.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.182+0000 7f27a9a22f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0150d26d0 0x55d0150c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:33.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.182+0000 7f27a9a22f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d015025eb0 con 0x55d0150444d0 2024-05-08T05:49:33.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.182+0000 7f27a9a22f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55d01502a870 con 0x55d014ff9a20 2024-05-08T05:49:33.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.182+0000 7f27a9a22f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55d01502aac0 con 0x55d0150d26d0 2024-05-08T05:49:33.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.183+0000 7f27a6d1b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0150444d0 0x55d01516d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.183+0000 7f27a6d1b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0150444d0 0x55d01516d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37724/0 (socket says 172.21.15.45:37724) 2024-05-08T05:49:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.183+0000 7f27a6d1b640 1 -- 172.21.15.45:0/4242212367 learned_addr learned my addr 172.21.15.45:0/4242212367 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.183+0000 7f27a7d1d640 1 --2- 172.21.15.45:0/4242212367 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0150d26d0 0x55d0150c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.183+0000 7f27a751c640 1 --2- 172.21.15.45:0/4242212367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d014ff9a20 0x55d0150c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.183+0000 7f27a6d1b640 1 -- 172.21.15.45:0/4242212367 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0150d26d0 msgr2=0x55d0150c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.183+0000 7f27a6d1b640 1 --2- 172.21.15.45:0/4242212367 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55d0150d26d0 0x55d0150c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.183+0000 7f27a6d1b640 1 -- 172.21.15.45:0/4242212367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d014ff9a20 msgr2=0x55d0150c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.183+0000 7f27a6d1b640 1 --2- 172.21.15.45:0/4242212367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55d014ff9a20 0x55d0150c5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.183+0000 7f27a6d1b640 1 -- 172.21.15.45:0/4242212367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d015027560 con 0x55d0150444d0 2024-05-08T05:49:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.183+0000 7f27a6d1b640 1 --2- 172.21.15.45:0/4242212367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0150444d0 0x55d01516d3f0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f279000a4f0 tx=0x7f2790002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.183+0000 7f27a4d17640 1 -- 172.21.15.45:0/4242212367 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790063030 con 0x55d0150444d0 2024-05-08T05:49:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.183+0000 7f27a4d17640 1 -- 172.21.15.45:0/4242212367 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2790054930 con 0x55d0150444d0 2024-05-08T05:49:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.183+0000 7f27a9a22f40 1 -- 172.21.15.45:0/4242212367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d0150ca570 con 0x55d0150444d0 2024-05-08T05:49:33.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.183+0000 7f27a4d17640 1 -- 172.21.15.45:0/4242212367 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790059040 con 0x55d0150444d0 2024-05-08T05:49:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.183+0000 7f27a9a22f40 1 -- 172.21.15.45:0/4242212367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0150cab30 con 0x55d0150444d0 2024-05-08T05:49:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.185+0000 7f27a4d17640 1 -- 172.21.15.45:0/4242212367 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2790066030 con 0x55d0150444d0 2024-05-08T05:49:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.185+0000 7f27a4d17640 1 --2- 172.21.15.45:0/4242212367 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2788074cb0 0x7f2788077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.185+0000 7f27a4d17640 1 -- 172.21.15.45:0/4242212367 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f27900d9b40 con 0x55d0150444d0 2024-05-08T05:49:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.186+0000 7f27a751c640 1 --2- 172.21.15.45:0/4242212367 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2788074cb0 0x7f2788077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.186+0000 7f27a9a22f40 1 --2- 172.21.15.45:0/4242212367 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55d0150cb050 0x55d0150cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.186+0000 7f27a9a22f40 1 -- 172.21.15.45:0/4242212367 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e104) -- 0x55d0150268a0 con 0x55d0150cb050 2024-05-08T05:49:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.186+0000 7f27a7d1d640 1 --2- 172.21.15.45:0/4242212367 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55d0150cb050 0x55d0150cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.186+0000 7f27a7d1d640 1 --2- 172.21.15.45:0/4242212367 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55d0150cb050 0x55d0150cb430 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.186+0000 7f27a751c640 1 --2- 172.21.15.45:0/4242212367 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2788074cb0 0x7f2788077170 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2798001e70 tx=0x7f2798009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:33.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.186+0000 7f27a7d1d640 1 -- 172.21.15.45:0/4242212367 <== osd.1 v2:172.21.15.45:6824/1337179184 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7f27a000e960 con 0x55d0150cb050 2024-05-08T05:49:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.186+0000 7f27a9a22f40 1 -- 172.21.15.45:0/4242212367 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55d0150cb050 msgr2=0x55d0150cb430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.186+0000 7f27a9a22f40 1 --2- 172.21.15.45:0/4242212367 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55d0150cb050 0x55d0150cb430 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.186+0000 7f27a9a22f40 1 -- 172.21.15.45:0/4242212367 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2788074cb0 msgr2=0x7f2788077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.186+0000 7f27a9a22f40 1 --2- 172.21.15.45:0/4242212367 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2788074cb0 0x7f2788077170 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2798001e70 tx=0x7f2798009000 comp rx=0 tx=0).stop 2024-05-08T05:49:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.186+0000 7f27a9a22f40 1 -- 172.21.15.45:0/4242212367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0150444d0 msgr2=0x55d01516d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.186+0000 7f27a9a22f40 1 --2- 172.21.15.45:0/4242212367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d0150444d0 0x55d01516d3f0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f279000a4f0 tx=0x7f2790002fb0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.186+0000 7f27a7d1d640 1 -- 172.21.15.45:0/4242212367 reap_dead start 2024-05-08T05:49:33.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.187+0000 7f27a9a22f40 1 -- 172.21.15.45:0/4242212367 shutdown_connections 2024-05-08T05:49:33.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.187+0000 7f27a9a22f40 1 -- 172.21.15.45:0/4242212367 >> 172.21.15.45:0/4242212367 conn(0x55d014ffab30 msgr2=0x55d015166270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:33.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.187+0000 7f27a9a22f40 1 -- 172.21.15.45:0/4242212367 shutdown_connections 2024-05-08T05:49:33.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.187+0000 7f27a9a22f40 1 -- 172.21.15.45:0/4242212367 wait complete. 2024-05-08T05:49:33.192 INFO:teuthology.orchestra.run.smithi045.stdout:����� 2024-05-08T05:49:33.192 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T05:49:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.296+0000 7f3f726fc640 1 Processor -- start 2024-05-08T05:49:33.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.296+0000 7f3f726fc640 1 -- start start 2024-05-08T05:49:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.297+0000 7f3f726fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f6c163920 0x7f3f6c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.297+0000 7f3f726fc640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f6c163d70 con 0x7f3f6c070d30 2024-05-08T05:49:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.297+0000 7f3f726fc640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f6c0d1300 con 0x7f3f6c0d20c0 2024-05-08T05:49:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.297+0000 7f3f726fc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3f6c0d10d0 con 0x7f3f6c163920 2024-05-08T05:49:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.297+0000 7f3f6bfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3f6c070d30 0x7f3f6c0703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59944/0 (socket says 172.21.15.45:59944) 2024-05-08T05:49:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.297+0000 7f3f6bfff640 1 -- 172.21.15.45:0/2378694185 learned_addr learned my addr 172.21.15.45:0/2378694185 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.297+0000 7f3f70c99640 1 --2- 172.21.15.45:0/2378694185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f6c163920 0x7f3f6c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:33.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.297+0000 7f3f6affd640 1 -- 172.21.15.45:0/2378694185 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3877857340 0 0) 0x7f3f5c001110 con 0x7f3f6c070d30 2024-05-08T05:49:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.297+0000 7f3f6affd640 1 -- 172.21.15.45:0/2378694185 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f54003520 con 0x7f3f6c070d30 2024-05-08T05:49:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f6affd640 1 -- 172.21.15.45:0/2378694185 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2757290431 0 0) 0x7f3f6c0d1300 con 0x7f3f6c0d20c0 2024-05-08T05:49:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f6affd640 1 -- 172.21.15.45:0/2378694185 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f5c001110 con 0x7f3f6c0d20c0 2024-05-08T05:49:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f6affd640 1 -- 172.21.15.45:0/2378694185 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3922857745 0 0) 0x7f3f54003520 con 0x7f3f6c070d30 2024-05-08T05:49:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f6affd640 1 -- 172.21.15.45:0/2378694185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f6c163920 msgr2=0x7f3f6c06d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f6affd640 1 --2- 172.21.15.45:0/2378694185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f6c163920 0x7f3f6c06d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f6affd640 1 -- 172.21.15.45:0/2378694185 >> v1:172.21.15.63:6789/0 conn(0x7f3f6c0d20c0 legacy=0x7f3f6c071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f6affd640 1 -- 172.21.15.45:0/2378694185 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6c0d15e0 con 0x7f3f6c070d30 2024-05-08T05:49:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f6affd640 1 -- 172.21.15.45:0/2378694185 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f3f5c0018c0 con 0x7f3f6c070d30 2024-05-08T05:49:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f6affd640 1 -- 172.21.15.45:0/2378694185 >> v1:172.21.15.45:6789/0 conn(0x7f3f6c070d30 legacy=0x7f3f6c0703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f6affd640 1 --2- 172.21.15.45:0/2378694185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f6c163920 0x7f3f54003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f6affd640 1 --2- 172.21.15.45:0/2378694185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f54004210 0x7f3f54004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f6affd640 1 --2- 172.21.15.45:0/2378694185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f54004d90 0x7f3f54005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:33.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f6affd640 1 -- 172.21.15.45:0/2378694185 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5400bba0 con 0x7f3f54004d90 2024-05-08T05:49:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f6affd640 1 -- 172.21.15.45:0/2378694185 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5400bd50 con 0x7f3f54004210 2024-05-08T05:49:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f6affd640 1 -- 172.21.15.45:0/2378694185 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3f5400bf50 con 0x7f3f6c163920 2024-05-08T05:49:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f70c99640 1 --2- 172.21.15.45:0/2378694185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f54004d90 0x7f3f54005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f6b7fe640 1 --2- 172.21.15.45:0/2378694185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f54004210 0x7f3f54004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f6bfff640 1 --2- 172.21.15.45:0/2378694185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f6c163920 0x7f3f54003c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f70c99640 1 -- 172.21.15.45:0/2378694185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f6c163920 msgr2=0x7f3f54003c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f70c99640 1 --2- 172.21.15.45:0/2378694185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f6c163920 0x7f3f54003c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f70c99640 1 -- 172.21.15.45:0/2378694185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f54004210 msgr2=0x7f3f54004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f70c99640 1 --2- 172.21.15.45:0/2378694185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f54004210 0x7f3f54004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.298+0000 7f3f70c99640 1 -- 172.21.15.45:0/2378694185 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f54003520 con 0x7f3f54004d90 2024-05-08T05:49:33.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.299+0000 7f3f70c99640 1 --2- 172.21.15.45:0/2378694185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f54004d90 0x7f3f54005210 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3f64009fc0 tx=0x7f3f54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730b54658d781417 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:33.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.299+0000 7f3f6affd640 1 -- 172.21.15.45:0/2378694185 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64009c10 con 0x7f3f54004d90 2024-05-08T05:49:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.299+0000 7f3f6affd640 1 -- 172.21.15.45:0/2378694185 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f64036070 con 0x7f3f54004d90 2024-05-08T05:49:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.299+0000 7f3f6affd640 1 -- 172.21.15.45:0/2378694185 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64031880 con 0x7f3f54004d90 2024-05-08T05:49:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.300+0000 7f3f726fc640 1 -- 172.21.15.45:0/2378694185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f54004d90 msgr2=0x7f3f54005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.300+0000 7f3f726fc640 1 --2- 172.21.15.45:0/2378694185 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f54004d90 0x7f3f54005210 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3f64009fc0 tx=0x7f3f54005830 comp rx=0 tx=0).stop 2024-05-08T05:49:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.300+0000 7f3f70c99640 1 -- 172.21.15.45:0/2378694185 reap_dead start 2024-05-08T05:49:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.300+0000 7f3f726fc640 1 -- 172.21.15.45:0/2378694185 shutdown_connections 2024-05-08T05:49:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.300+0000 7f3f726fc640 1 -- 172.21.15.45:0/2378694185 >> 172.21.15.45:0/2378694185 conn(0x7f3f6c0d2e90 msgr2=0x7f3f6c0d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.300+0000 7f3f726fc640 1 -- 172.21.15.45:0/2378694185 shutdown_connections 2024-05-08T05:49:33.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.300+0000 7f3f726fc640 1 -- 172.21.15.45:0/2378694185 wait complete. 2024-05-08T05:49:33.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.301+0000 7f3f726fc640 1 Processor -- start 2024-05-08T05:49:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.301+0000 7f3f726fc640 1 -- start start 2024-05-08T05:49:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f726fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f6c1573c0 0x7f3f6c0b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f726fc640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f6c0b3490 0x7f3f6c0cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f726fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f6c0cbf90 0x7f3f6c1f92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f726fc640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6c0d1c30 con 0x7f3f6c1573c0 2024-05-08T05:49:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f726fc640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6c070b80 con 0x7f3f6c0cbf90 2024-05-08T05:49:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f726fc640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3f6c06d4b0 con 0x7f3f6c0b3490 2024-05-08T05:49:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f6bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f6c1573c0 0x7f3f6c0b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f6bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f6c1573c0 0x7f3f6c0b2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37746/0 (socket says 172.21.15.45:37746) 2024-05-08T05:49:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f6bfff640 1 -- 172.21.15.45:0/1287057414 learned_addr learned my addr 172.21.15.45:0/1287057414 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f6b7fe640 1 --2- 172.21.15.45:0/1287057414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f6c0b3490 0x7f3f6c0cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f70c99640 1 --2- 172.21.15.45:0/1287057414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f6c0cbf90 0x7f3f6c1f92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f6bfff640 1 -- 172.21.15.45:0/1287057414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f6c0b3490 msgr2=0x7f3f6c0cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:33.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f6bfff640 1 --2- 172.21.15.45:0/1287057414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f6c0b3490 0x7f3f6c0cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f6bfff640 1 -- 172.21.15.45:0/1287057414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f6c0cbf90 msgr2=0x7f3f6c1f92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f6bfff640 1 --2- 172.21.15.45:0/1287057414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f6c0cbf90 0x7f3f6c1f92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f6bfff640 1 -- 172.21.15.45:0/1287057414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6c0cc7d0 con 0x7f3f6c1573c0 2024-05-08T05:49:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f6b7fe640 1 --2- 172.21.15.45:0/1287057414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f6c0b3490 0x7f3f6c0cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f70c99640 1 --2- 172.21.15.45:0/1287057414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f6c0cbf90 0x7f3f6c1f92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f6bfff640 1 --2- 172.21.15.45:0/1287057414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f6c1573c0 0x7f3f6c0b2f50 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c001a30 tx=0x7f3f5c007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f68ff9640 1 -- 172.21.15.45:0/1287057414 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c00fa80 con 0x7f3f6c1573c0 2024-05-08T05:49:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f68ff9640 1 -- 172.21.15.45:0/1287057414 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f5c018070 con 0x7f3f6c1573c0 2024-05-08T05:49:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f68ff9640 1 -- 172.21.15.45:0/1287057414 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5c00fe80 con 0x7f3f6c1573c0 2024-05-08T05:49:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.302+0000 7f3f726fc640 1 -- 172.21.15.45:0/1287057414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f6c0ce4d0 con 0x7f3f6c1573c0 2024-05-08T05:49:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.303+0000 7f3f726fc640 1 -- 172.21.15.45:0/1287057414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f6c0cea90 con 0x7f3f6c1573c0 2024-05-08T05:49:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.304+0000 7f3f726fc640 1 -- 172.21.15.45:0/1287057414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f6c0cec90 con 0x7f3f6c1573c0 2024-05-08T05:49:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.304+0000 7f3f68ff9640 1 -- 172.21.15.45:0/1287057414 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3f5c01d460 con 0x7f3f6c1573c0 2024-05-08T05:49:33.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.305+0000 7f3f68ff9640 1 --2- 172.21.15.45:0/1287057414 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f580762d0 0x7f3f58078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:33.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.305+0000 7f3f68ff9640 1 -- 172.21.15.45:0/1287057414 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f3f5c0978f0 con 0x7f3f6c1573c0 2024-05-08T05:49:33.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.305+0000 7f3f6b7fe640 1 --2- 172.21.15.45:0/1287057414 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f580762d0 0x7f3f58078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:33.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.305+0000 7f3f6b7fe640 1 --2- 172.21.15.45:0/1287057414 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f580762d0 0x7f3f58078790 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0cd0d0 tx=0x7f3f60009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:33.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.309+0000 7f3f68ff9640 1 -- 172.21.15.45:0/1287057414 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3f6c0cec90 con 0x7f3f6c1573c0 2024-05-08T05:49:33.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.538+0000 7f3f726fc640 1 -- 172.21.15.45:0/1287057414 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3f6c0693f0 con 0x7f3f6c1573c0 2024-05-08T05:49:33.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.539+0000 7f3f68ff9640 1 -- 172.21.15.45:0/1287057414 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v83) ==== 53+0+73 (secure 0 0 0) 0x7f3f6c163d70 con 0x7f3f6c1573c0 2024-05-08T05:49:33.539 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T05:49:33.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.540+0000 7f3f726fc640 1 -- 172.21.15.45:0/1287057414 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f580762d0 msgr2=0x7f3f58078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:33.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.540+0000 7f3f726fc640 1 --2- 172.21.15.45:0/1287057414 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f580762d0 0x7f3f58078790 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0cd0d0 tx=0x7f3f60009040 comp rx=0 tx=0).stop 2024-05-08T05:49:33.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.541+0000 7f3f726fc640 1 -- 172.21.15.45:0/1287057414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f6c1573c0 msgr2=0x7f3f6c0b2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:33.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.541+0000 7f3f726fc640 1 --2- 172.21.15.45:0/1287057414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f6c1573c0 0x7f3f6c0b2f50 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c001a30 tx=0x7f3f5c007720 comp rx=0 tx=0).stop 2024-05-08T05:49:33.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.541+0000 7f3f726fc640 1 -- 172.21.15.45:0/1287057414 shutdown_connections 2024-05-08T05:49:33.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.541+0000 7f3f726fc640 1 --2- 172.21.15.45:0/1287057414 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f580762d0 0x7f3f58078790 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.541+0000 7f3f726fc640 1 --2- 172.21.15.45:0/1287057414 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f6c0b3490 0x7f3f6c0cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.541+0000 7f3f726fc640 1 --2- 172.21.15.45:0/1287057414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f6c0cbf90 0x7f3f6c1f92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.541+0000 7f3f726fc640 1 --2- 172.21.15.45:0/1287057414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f6c1573c0 0x7f3f6c0b2f50 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.541+0000 7f3f726fc640 1 -- 172.21.15.45:0/1287057414 >> 172.21.15.45:0/1287057414 conn(0x7f3f6c0d2e90 msgr2=0x7f3f6c06e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:33.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.541+0000 7f3f726fc640 1 -- 172.21.15.45:0/1287057414 shutdown_connections 2024-05-08T05:49:33.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33.541+0000 7f3f726fc640 1 -- 172.21.15.45:0/1287057414 wait complete. 2024-05-08T05:49:33.552 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T05:49:33.552 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:49:33.552 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-05-08T05:49:33.552 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T05:49:33.552 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:49:33.552 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T05:49:33.552 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T05:49:33.552 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:49:33.552 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:49:33.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:33.553 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-05-08T05:49:33.569 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:49:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:49:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:49:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:49:33.569 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-05-08T05:49:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:49:33.570 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:49:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:49:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-05-08T05:49:33.570 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:49:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:49:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-05-08T05:49:33.570 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:49:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-05-08T05:49:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-05-08T05:49:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:49:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42251sec preferred_lft 42251sec 2024-05-08T05:49:33.570 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-05-08T05:49:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:49:33.571 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:49:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-05-08T05:49:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-05-08T05:49:33.571 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:49:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T05:49:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T05:49:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:49:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c80a:aff:fe94:c77/64 scope link 2024-05-08T05:49:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:49:33.571 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:49:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 82:f3:eb:6a:57:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:49:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::80f3:ebff:fe6a:57ba/64 scope link 2024-05-08T05:49:33.571 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:49:33.571 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:49:33.572 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T05:49:33.572 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::14a2:64ff:feda:3a9c/64 scope link 2024-05-08T05:49:33.572 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:49:33.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:33.572 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:49:33.635 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:49:33.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:33.637 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-05-08T05:49:33.693 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-05-08T05:49:33.694 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:49:33.694 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:49:33.694 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:49:33.694 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:49:33.694 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:49:33.694 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:49:33.694 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:49:33.694 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:49:33.694 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:49:33.695 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:49:33.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:49:33.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:49:33.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:33.824 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:49:33.840 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T05:49:33.840 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T05:49:33.840 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:49:33.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:33.841 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-05-08T05:49:33.907 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:49:33.907 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T05:49:33.963 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:49:33.963 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:49:33.964 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:49:34.019 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T05:49:34.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:34.020 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:49:34.104 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:49:34.104 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:49:34.112 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:49:34.112 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:49:34.112 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-08T05:49:34.112 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@e9e1170a-fd27-4625-97a9-5d5f104d13a2.cephfs=/ 2024-05-08T05:49:34.112 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.63:3300/172.21.15.63:3301" will pass to kernel 2024-05-08T05:49:34.114 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T05:49:34.114 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:49:34.114 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:49:34.114 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:49:34.114 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:49:34.114 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:49:34.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:34.114 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:49:34.146 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:49:34.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:49:34.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:49:34.245 INFO:teuthology.orchestra.run.smithi045.stdout:client6862 2024-05-08T05:49:34.245 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:49:34.245 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:49:34.263 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:49:34.263 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:49:34.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:34.320 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-08T05:49:34.412 INFO:teuthology.orchestra.run.smithi045.stderr:6+0 records in 2024-05-08T05:49:34.412 INFO:teuthology.orchestra.run.smithi045.stderr:6+0 records out 2024-05-08T05:49:34.413 INFO:teuthology.orchestra.run.smithi045.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0334639 s, 188 MB/s 2024-05-08T05:49:34.413 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:49:34.413 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:49:34.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:34.429 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-05-08T05:49:34.517 INFO:teuthology.orchestra.run.smithi045.stderr:6+0 records in 2024-05-08T05:49:34.517 INFO:teuthology.orchestra.run.smithi045.stderr:6+0 records out 2024-05-08T05:49:34.517 INFO:teuthology.orchestra.run.smithi045.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0308082 s, 204 MB/s 2024-05-08T05:49:34.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:34.518 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-08T05:49:34.535 INFO:teuthology.orchestra.run.smithi045.stdout:./ 2024-05-08T05:49:34.536 INFO:teuthology.orchestra.run.smithi045.stdout:./file2_sixmegs 2024-05-08T05:49:34.536 INFO:teuthology.orchestra.run.smithi045.stdout:./file3_sixmegs 2024-05-08T05:49:34.536 INFO:teuthology.orchestra.run.smithi045.stdout:./file1_sixmegs 2024-05-08T05:49:34.536 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:49:34.536 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:49:34.536 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:49:34.536 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:49:34.536 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-08T05:49:34.536 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:49:34.644 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:49:34.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:49:34.653 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:49:34.653 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:49:34.653 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:49:34.653 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:49:34.653 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-05-08T05:49:34.653 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:49:34.760 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627777 2024-05-08T05:49:34.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:49:34.770 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:49:34.770 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:49:34.770 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:49:34.770 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:49:34.770 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-05-08T05:49:34.770 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:49:34.878 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627778 2024-05-08T05:49:34.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:49:34.888 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:49:34.888 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:49:34.888 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:49:34.888 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:49:34.888 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-05-08T05:49:34.888 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:49:34.996 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627776 2024-05-08T05:49:34.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:49:35.006 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:49:35.006 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:49:35.062 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:49:35.063 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:49:35.065 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:49:35.106 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:49:35.120 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:49:35.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.301+0000 7f8b2af2b640 1 Processor -- start 2024-05-08T05:49:35.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.301+0000 7f8b2af2b640 1 -- start start 2024-05-08T05:49:35.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.302+0000 7f8b2af2b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8b240d3cd0 0x7f8b240cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:35.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.302+0000 7f8b2af2b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8b240c9ee0 0x7f8b240d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:35.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.302+0000 7f8b2af2b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b240d3330 con 0x7f8b241629a0 2024-05-08T05:49:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.302+0000 7f8b2af2b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f8b240d2170 con 0x7f8b240d3cd0 2024-05-08T05:49:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.302+0000 7f8b2af2b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8b240d2450 con 0x7f8b240c9ee0 2024-05-08T05:49:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.302+0000 7f8b23fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8b240d3cd0 0x7f8b240cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.302+0000 7f8b28cc7640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8b241629a0 0x7f8b240d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59958/0 (socket says 172.21.15.45:59958) 2024-05-08T05:49:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.302+0000 7f8b23fff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f8b240d3cd0 0x7f8b240cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37750/0 (socket says 172.21.15.45:37750) 2024-05-08T05:49:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.302+0000 7f8b28cc7640 1 -- 172.21.15.45:0/687057385 learned_addr learned my addr 172.21.15.45:0/687057385 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.302+0000 7f8b23fff640 1 -- 172.21.15.45:0/687057385 >> v2:172.21.15.63:3300/0 conn(0x7f8b240c9ee0 msgr2=0x7f8b240d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.302+0000 7f8b23fff640 1 --2- 172.21.15.45:0/687057385 >> v2:172.21.15.63:3300/0 conn(0x7f8b240c9ee0 0x7f8b240d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.302+0000 7f8b23fff640 1 -- 172.21.15.45:0/687057385 >> v1:172.21.15.45:6789/0 conn(0x7f8b241629a0 legacy=0x7f8b240d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:35.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.302+0000 7f8b23fff640 1 -- 172.21.15.45:0/687057385 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b240d2730 con 0x7f8b240d3cd0 2024-05-08T05:49:35.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.303+0000 7f8b23fff640 1 --2- 172.21.15.45:0/687057385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b240d3cd0 0x7f8b240cfca0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8b2408e4c0 tx=0x7f8b1802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9a2f98deedc054 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:35.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.303+0000 7f8b237fe640 1 -- 172.21.15.45:0/687057385 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b18037070 con 0x7f8b240d3cd0 2024-05-08T05:49:35.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.303+0000 7f8b237fe640 1 -- 172.21.15.45:0/687057385 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b1802add0 con 0x7f8b240d3cd0 2024-05-08T05:49:35.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.303+0000 7f8b237fe640 1 -- 172.21.15.45:0/687057385 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b18032660 con 0x7f8b240d3cd0 2024-05-08T05:49:35.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.304+0000 7f8b2af2b640 1 -- 172.21.15.45:0/687057385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b240d3cd0 msgr2=0x7f8b240cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:35.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.304+0000 7f8b2af2b640 1 --2- 172.21.15.45:0/687057385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b240d3cd0 0x7f8b240cfca0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8b2408e4c0 tx=0x7f8b1802a080 comp rx=0 tx=0).stop 2024-05-08T05:49:35.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.304+0000 7f8b2af2b640 1 -- 172.21.15.45:0/687057385 shutdown_connections 2024-05-08T05:49:35.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.304+0000 7f8b2af2b640 1 --2- 172.21.15.45:0/687057385 >> v2:172.21.15.63:3300/0 conn(0x7f8b240c9ee0 0x7f8b240d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:35.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.304+0000 7f8b2af2b640 1 --2- 172.21.15.45:0/687057385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b240d3cd0 0x7f8b240cfca0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:35.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.304+0000 7f8b2af2b640 1 -- 172.21.15.45:0/687057385 >> 172.21.15.45:0/687057385 conn(0x7f8b240d4fe0 msgr2=0x7f8b240d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:35.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.304+0000 7f8b2af2b640 1 -- 172.21.15.45:0/687057385 shutdown_connections 2024-05-08T05:49:35.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.304+0000 7f8b2af2b640 1 -- 172.21.15.45:0/687057385 wait complete. 2024-05-08T05:49:35.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.305+0000 7f8b2af2b640 1 Processor -- start 2024-05-08T05:49:35.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.305+0000 7f8b2af2b640 1 -- start start 2024-05-08T05:49:35.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.306+0000 7f8b2af2b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b240c9ee0 0x7f8b240cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:35.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.306+0000 7f8b2af2b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b240d3cd0 0x7f8b2415f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:35.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.306+0000 7f8b2af2b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b241629a0 0x7f8b241e99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:35.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.306+0000 7f8b2af2b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8b240cf440 con 0x7f8b241629a0 2024-05-08T05:49:35.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.306+0000 7f8b2af2b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8b240cf2c0 con 0x7f8b240c9ee0 2024-05-08T05:49:35.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.306+0000 7f8b2af2b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8b240d2dc0 con 0x7f8b240d3cd0 2024-05-08T05:49:35.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.306+0000 7f8b294c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b241629a0 0x7f8b241e99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:35.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.306+0000 7f8b294c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b241629a0 0x7f8b241e99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37766/0 (socket says 172.21.15.45:37766) 2024-05-08T05:49:35.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.306+0000 7f8b294c8640 1 -- 172.21.15.45:0/484837517 learned_addr learned my addr 172.21.15.45:0/484837517 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:35.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.306+0000 7f8b23fff640 1 --2- 172.21.15.45:0/484837517 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b240d3cd0 0x7f8b2415f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:35.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.306+0000 7f8b28cc7640 1 --2- 172.21.15.45:0/484837517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b240c9ee0 0x7f8b240cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:35.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.306+0000 7f8b294c8640 1 -- 172.21.15.45:0/484837517 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b240d3cd0 msgr2=0x7f8b2415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:35.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.306+0000 7f8b294c8640 1 --2- 172.21.15.45:0/484837517 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b240d3cd0 0x7f8b2415f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:35.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.306+0000 7f8b294c8640 1 -- 172.21.15.45:0/484837517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b240c9ee0 msgr2=0x7f8b240cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:35.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.306+0000 7f8b294c8640 1 --2- 172.21.15.45:0/484837517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b240c9ee0 0x7f8b240cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:35.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.306+0000 7f8b294c8640 1 -- 172.21.15.45:0/484837517 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b241ea9e0 con 0x7f8b241629a0 2024-05-08T05:49:35.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.307+0000 7f8b294c8640 1 --2- 172.21.15.45:0/484837517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b241629a0 0x7f8b241e99b0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400b450 tx=0x7f8b1400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:35.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.307+0000 7f8b217fa640 1 -- 172.21.15.45:0/484837517 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14015020 con 0x7f8b241629a0 2024-05-08T05:49:35.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.307+0000 7f8b217fa640 1 -- 172.21.15.45:0/484837517 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b14011890 con 0x7f8b241629a0 2024-05-08T05:49:35.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.307+0000 7f8b2af2b640 1 -- 172.21.15.45:0/484837517 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b240ccc40 con 0x7f8b241629a0 2024-05-08T05:49:35.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.307+0000 7f8b217fa640 1 -- 172.21.15.45:0/484837517 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14011c60 con 0x7f8b241629a0 2024-05-08T05:49:35.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.307+0000 7f8b2af2b640 1 -- 172.21.15.45:0/484837517 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b240cd200 con 0x7f8b241629a0 2024-05-08T05:49:35.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.308+0000 7f8b2af2b640 1 -- 172.21.15.45:0/484837517 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b240693f0 con 0x7f8b241629a0 2024-05-08T05:49:35.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.309+0000 7f8b217fa640 1 -- 172.21.15.45:0/484837517 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8b14018020 con 0x7f8b241629a0 2024-05-08T05:49:35.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.309+0000 7f8b217fa640 1 --2- 172.21.15.45:0/484837517 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8b100762d0 0x7f8b10078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:35.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.309+0000 7f8b217fa640 1 -- 172.21.15.45:0/484837517 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f8b140978e0 con 0x7f8b241629a0 2024-05-08T05:49:35.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.310+0000 7f8b28cc7640 1 --2- 172.21.15.45:0/484837517 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8b100762d0 0x7f8b10078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:35.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.310+0000 7f8b28cc7640 1 --2- 172.21.15.45:0/484837517 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8b100762d0 0x7f8b10078790 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0012e0 tx=0x7f8b1c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:35.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.314+0000 7f8b217fa640 1 -- 172.21.15.45:0/484837517 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8b1401f020 con 0x7f8b241629a0 2024-05-08T05:49:35.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.517+0000 7f8b2af2b640 1 -- 172.21.15.45:0/484837517 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8b240cd440 con 0x7f8b241629a0 2024-05-08T05:49:35.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.518+0000 7f8b217fa640 1 -- 172.21.15.45:0/484837517 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v104) ==== 80+0+0 (secure 0 0 0) 0x7f8b1400d040 con 0x7f8b241629a0 2024-05-08T05:49:35.518 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-05-08T05:49:35.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.519+0000 7f8b2af2b640 1 -- 172.21.15.45:0/484837517 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8b100762d0 msgr2=0x7f8b10078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:35.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.519+0000 7f8b2af2b640 1 --2- 172.21.15.45:0/484837517 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8b100762d0 0x7f8b10078790 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c0012e0 tx=0x7f8b1c009040 comp rx=0 tx=0).stop 2024-05-08T05:49:35.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.519+0000 7f8b2af2b640 1 -- 172.21.15.45:0/484837517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b241629a0 msgr2=0x7f8b241e99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:35.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.519+0000 7f8b2af2b640 1 --2- 172.21.15.45:0/484837517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b241629a0 0x7f8b241e99b0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400b450 tx=0x7f8b1400b910 comp rx=0 tx=0).stop 2024-05-08T05:49:35.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.520+0000 7f8b2af2b640 1 -- 172.21.15.45:0/484837517 shutdown_connections 2024-05-08T05:49:35.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.520+0000 7f8b2af2b640 1 --2- 172.21.15.45:0/484837517 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8b100762d0 0x7f8b10078790 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:35.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.520+0000 7f8b2af2b640 1 --2- 172.21.15.45:0/484837517 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8b240d3cd0 0x7f8b2415f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:35.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.520+0000 7f8b2af2b640 1 --2- 172.21.15.45:0/484837517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b241629a0 0x7f8b241e99b0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:35.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.520+0000 7f8b2af2b640 1 --2- 172.21.15.45:0/484837517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b240c9ee0 0x7f8b240cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:35.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.520+0000 7f8b2af2b640 1 -- 172.21.15.45:0/484837517 >> 172.21.15.45:0/484837517 conn(0x7f8b240d4fe0 msgr2=0x7f8b240caaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:35.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.520+0000 7f8b2af2b640 1 -- 172.21.15.45:0/484837517 shutdown_connections 2024-05-08T05:49:35.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.520+0000 7f8b2af2b640 1 -- 172.21.15.45:0/484837517 wait complete. 2024-05-08T05:49:35.531 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T05:49:35.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:35.531 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T05:49:35.579 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:49:35.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:35.579 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T05:49:35.608 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:49:35.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.752+0000 7fe37e23e640 1 Processor -- start 2024-05-08T05:49:35.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.752+0000 7fe37e23e640 1 -- start start 2024-05-08T05:49:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.753+0000 7fe37e23e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe3780d53d0 0x7fe3780d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.753+0000 7fe37e23e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3781630b0 con 0x7fe3780d1970 2024-05-08T05:49:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.753+0000 7fe37e23e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3780d3e70 con 0x7fe37815f420 2024-05-08T05:49:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.753+0000 7fe37e23e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe3780d2550 con 0x7fe3780d53d0 2024-05-08T05:49:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.753+0000 7fe376ffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe3780d1970 0x7fe3780d4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59962/0 (socket says 172.21.15.45:59962) 2024-05-08T05:49:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.753+0000 7fe376ffd640 1 -- 172.21.15.45:0/898885457 learned_addr learned my addr 172.21.15.45:0/898885457 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.753+0000 7fe377fff640 1 --2- 172.21.15.45:0/898885457 >> v2:172.21.15.63:3300/0 conn(0x7fe3780d53d0 0x7fe3780d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.753+0000 7fe3767fc640 1 -- 172.21.15.45:0/898885457 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 779420550 0 0) 0x7fe360001110 con 0x7fe3780d1970 2024-05-08T05:49:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.753+0000 7fe3767fc640 1 -- 172.21.15.45:0/898885457 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe364003520 con 0x7fe3780d1970 2024-05-08T05:49:35.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.753+0000 7fe3767fc640 1 -- 172.21.15.45:0/898885457 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 659285289 0 0) 0x7fe364003520 con 0x7fe3780d1970 2024-05-08T05:49:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.753+0000 7fe3767fc640 1 -- 172.21.15.45:0/898885457 >> v2:172.21.15.63:3300/0 conn(0x7fe3780d53d0 msgr2=0x7fe3780d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.753+0000 7fe3767fc640 1 --2- 172.21.15.45:0/898885457 >> v2:172.21.15.63:3300/0 conn(0x7fe3780d53d0 0x7fe3780d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.753+0000 7fe3767fc640 1 -- 172.21.15.45:0/898885457 >> v1:172.21.15.63:6789/0 conn(0x7fe37815f420 legacy=0x7fe3780d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.754+0000 7fe3767fc640 1 -- 172.21.15.45:0/898885457 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3780d2c80 con 0x7fe3780d1970 2024-05-08T05:49:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.754+0000 7fe3767fc640 1 -- 172.21.15.45:0/898885457 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fe3600018c0 con 0x7fe3780d1970 2024-05-08T05:49:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.754+0000 7fe3767fc640 1 -- 172.21.15.45:0/898885457 >> v1:172.21.15.45:6789/0 conn(0x7fe3780d1970 legacy=0x7fe3780d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.754+0000 7fe3767fc640 1 --2- 172.21.15.45:0/898885457 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe364003a50 0x7fe364003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.754+0000 7fe3767fc640 1 --2- 172.21.15.45:0/898885457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe364004800 0x7fe364004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.754+0000 7fe3767fc640 1 --2- 172.21.15.45:0/898885457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3640051c0 0x7fe36400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.754+0000 7fe3767fc640 1 -- 172.21.15.45:0/898885457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe3780d2550 con 0x7fe364004800 2024-05-08T05:49:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.754+0000 7fe3767fc640 1 -- 172.21.15.45:0/898885457 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe36400c160 con 0x7fe3640051c0 2024-05-08T05:49:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.754+0000 7fe3767fc640 1 -- 172.21.15.45:0/898885457 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe36400c360 con 0x7fe364003a50 2024-05-08T05:49:35.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.754+0000 7fe377fff640 1 --2- 172.21.15.45:0/898885457 >> v2:172.21.15.63:3300/0 conn(0x7fe3780d53d0 0x7fe3780d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:49:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.754+0000 7fe376ffd640 1 --2- 172.21.15.45:0/898885457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe364004800 0x7fe364004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.754+0000 7fe3777fe640 1 --2- 172.21.15.45:0/898885457 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe364003a50 0x7fe364003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.754+0000 7fe377fff640 1 --2- 172.21.15.45:0/898885457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3640051c0 0x7fe36400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.754+0000 7fe376ffd640 1 -- 172.21.15.45:0/898885457 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe364003a50 msgr2=0x7fe364003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.754+0000 7fe376ffd640 1 --2- 172.21.15.45:0/898885457 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe364003a50 0x7fe364003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.754+0000 7fe376ffd640 1 -- 172.21.15.45:0/898885457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3640051c0 msgr2=0x7fe36400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.754+0000 7fe376ffd640 1 --2- 172.21.15.45:0/898885457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3640051c0 0x7fe36400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.754+0000 7fe377fff640 1 -- 172.21.15.45:0/898885457 reap_dead start 2024-05-08T05:49:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.754+0000 7fe376ffd640 1 -- 172.21.15.45:0/898885457 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe364003520 con 0x7fe364004800 2024-05-08T05:49:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.755+0000 7fe376ffd640 1 --2- 172.21.15.45:0/898885457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe364004800 0x7fe364004c80 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fe36000c4c0 tx=0x7fe364005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f76773cb759e35 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.755+0000 7fe3767fc640 1 -- 172.21.15.45:0/898885457 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360035070 con 0x7fe364004800 2024-05-08T05:49:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.755+0000 7fe3767fc640 1 -- 172.21.15.45:0/898885457 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe360005ad0 con 0x7fe364004800 2024-05-08T05:49:35.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.755+0000 7fe3767fc640 1 -- 172.21.15.45:0/898885457 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36000c0e0 con 0x7fe364004800 2024-05-08T05:49:35.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.756+0000 7fe37e23e640 1 -- 172.21.15.45:0/898885457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe364004800 msgr2=0x7fe364004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:35.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.756+0000 7fe37e23e640 1 --2- 172.21.15.45:0/898885457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe364004800 0x7fe364004c80 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fe36000c4c0 tx=0x7fe364005830 comp rx=0 tx=0).stop 2024-05-08T05:49:35.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.756+0000 7fe37e23e640 1 -- 172.21.15.45:0/898885457 shutdown_connections 2024-05-08T05:49:35.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.756+0000 7fe37e23e640 1 --2- 172.21.15.45:0/898885457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe364004800 0x7fe364004c80 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:35.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.756+0000 7fe37e23e640 1 -- 172.21.15.45:0/898885457 >> 172.21.15.45:0/898885457 conn(0x7fe37808eac0 msgr2=0x7fe37809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:35.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.756+0000 7fe37e23e640 1 -- 172.21.15.45:0/898885457 shutdown_connections 2024-05-08T05:49:35.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.756+0000 7fe37e23e640 1 -- 172.21.15.45:0/898885457 wait complete. 2024-05-08T05:49:35.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.757+0000 7fe37e23e640 1 Processor -- start 2024-05-08T05:49:35.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.757+0000 7fe37e23e640 1 -- start start 2024-05-08T05:49:35.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.758+0000 7fe37e23e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe364004800 0x7fe3780cea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:35.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.758+0000 7fe37e23e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3780b7c30 0x7fe378152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:35.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.758+0000 7fe37e23e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37806e270 0x7fe37806e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:35.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.758+0000 7fe37e23e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe3780d2830 con 0x7fe37806e270 2024-05-08T05:49:35.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.758+0000 7fe37e23e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe3780d3240 con 0x7fe3780b7c30 2024-05-08T05:49:35.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.758+0000 7fe37e23e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe3780d1e20 con 0x7fe364004800 2024-05-08T05:49:35.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.758+0000 7fe377fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37806e270 0x7fe37806e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:35.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.758+0000 7fe377fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37806e270 0x7fe37806e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37778/0 (socket says 172.21.15.45:37778) 2024-05-08T05:49:35.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.758+0000 7fe377fff640 1 -- 172.21.15.45:0/1138264044 learned_addr learned my addr 172.21.15.45:0/1138264044 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:35.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.758+0000 7fe376ffd640 1 --2- 172.21.15.45:0/1138264044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3780b7c30 0x7fe378152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:35.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.758+0000 7fe3777fe640 1 --2- 172.21.15.45:0/1138264044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe364004800 0x7fe3780cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:35.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.758+0000 7fe377fff640 1 -- 172.21.15.45:0/1138264044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe364004800 msgr2=0x7fe3780cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:35.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.758+0000 7fe377fff640 1 --2- 172.21.15.45:0/1138264044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe364004800 0x7fe3780cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:35.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.758+0000 7fe377fff640 1 -- 172.21.15.45:0/1138264044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3780b7c30 msgr2=0x7fe378152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:35.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.758+0000 7fe377fff640 1 --2- 172.21.15.45:0/1138264044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3780b7c30 0x7fe378152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:35.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.758+0000 7fe377fff640 1 -- 172.21.15.45:0/1138264044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3780b8160 con 0x7fe37806e270 2024-05-08T05:49:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.758+0000 7fe377fff640 1 --2- 172.21.15.45:0/1138264044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37806e270 0x7fe37806e6f0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe37808f5c0 tx=0x7fe37000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.759+0000 7fe36ffff640 1 -- 172.21.15.45:0/1138264044 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37000fdc0 con 0x7fe37806e270 2024-05-08T05:49:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.759+0000 7fe36ffff640 1 -- 172.21.15.45:0/1138264044 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe370003990 con 0x7fe37806e270 2024-05-08T05:49:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.759+0000 7fe37e23e640 1 -- 172.21.15.45:0/1138264044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3780b8440 con 0x7fe37806e270 2024-05-08T05:49:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.759+0000 7fe36ffff640 1 -- 172.21.15.45:0/1138264044 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe370003d80 con 0x7fe37806e270 2024-05-08T05:49:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.759+0000 7fe37e23e640 1 -- 172.21.15.45:0/1138264044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3780b8a00 con 0x7fe37806e270 2024-05-08T05:49:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.760+0000 7fe37e23e640 1 -- 172.21.15.45:0/1138264044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe37806ec30 con 0x7fe37806e270 2024-05-08T05:49:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.760+0000 7fe36ffff640 1 -- 172.21.15.45:0/1138264044 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe37001f020 con 0x7fe37806e270 2024-05-08T05:49:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.761+0000 7fe36ffff640 1 --2- 172.21.15.45:0/1138264044 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe3580762d0 0x7fe358078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.761+0000 7fe36ffff640 1 -- 172.21.15.45:0/1138264044 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fe370097cf0 con 0x7fe37806e270 2024-05-08T05:49:35.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.761+0000 7fe3777fe640 1 --2- 172.21.15.45:0/1138264044 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe3580762d0 0x7fe358078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:35.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.761+0000 7fe3777fe640 1 --2- 172.21.15.45:0/1138264044 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe3580762d0 0x7fe358078790 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe3680017a0 tx=0x7fe368009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:35.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.765+0000 7fe36ffff640 1 -- 172.21.15.45:0/1138264044 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe37009c070 con 0x7fe37806e270 2024-05-08T05:49:35.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:35.999+0000 7fe37e23e640 1 -- 172.21.15.45:0/1138264044 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3780693f0 con 0x7fe37806e270 2024-05-08T05:49:36.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.000+0000 7fe36ffff640 1 -- 172.21.15.45:0/1138264044 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5180 (secure 0 0 0) 0x7fe370026020 con 0x7fe37806e270 2024-05-08T05:49:36.000 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:36.000 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3692881261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3692881261},{"type":"v1","addr":"172.21.15.63:6836","nonce":3692881261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7497,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2916399818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2916399818},{"type":"v1","addr":"172.21.15.45:6833","nonce":2916399818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7515,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2978250004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":2978250004},{"type":"v1","addr":"172.21.15.63:6837","nonce":2978250004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:49:19.587326+0000","modified":"2024-05-08T05:49:22.599177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7524":{"gid":7524,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/878092981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":878092981},{"type":"v1","addr":"172.21.15.45:6835","nonce":878092981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7524,"qdb_cluster":[7524]},"id":7}]} 2024-05-08T05:49:36.001 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 83 2024-05-08T05:49:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.002+0000 7fe37e23e640 1 -- 172.21.15.45:0/1138264044 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe3580762d0 msgr2=0x7fe358078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.002+0000 7fe37e23e640 1 --2- 172.21.15.45:0/1138264044 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe3580762d0 0x7fe358078790 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe3680017a0 tx=0x7fe368009040 comp rx=0 tx=0).stop 2024-05-08T05:49:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.002+0000 7fe37e23e640 1 -- 172.21.15.45:0/1138264044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37806e270 msgr2=0x7fe37806e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.002+0000 7fe37e23e640 1 --2- 172.21.15.45:0/1138264044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37806e270 0x7fe37806e6f0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe37808f5c0 tx=0x7fe37000f580 comp rx=0 tx=0).stop 2024-05-08T05:49:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.002+0000 7fe37e23e640 1 -- 172.21.15.45:0/1138264044 shutdown_connections 2024-05-08T05:49:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.002+0000 7fe37e23e640 1 --2- 172.21.15.45:0/1138264044 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe3580762d0 0x7fe358078790 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.002+0000 7fe37e23e640 1 --2- 172.21.15.45:0/1138264044 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37806e270 0x7fe37806e6f0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.002+0000 7fe37e23e640 1 --2- 172.21.15.45:0/1138264044 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3780b7c30 0x7fe378152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.002+0000 7fe37e23e640 1 --2- 172.21.15.45:0/1138264044 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe364004800 0x7fe3780cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.002+0000 7fe37e23e640 1 -- 172.21.15.45:0/1138264044 >> 172.21.15.45:0/1138264044 conn(0x7fe37808eac0 msgr2=0x7fe37806cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.003+0000 7fe37e23e640 1 -- 172.21.15.45:0/1138264044 shutdown_connections 2024-05-08T05:49:36.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.003+0000 7fe37e23e640 1 -- 172.21.15.45:0/1138264044 wait complete. 2024-05-08T05:49:36.014 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-05-08T05:49:36.142 INFO:teuthology.orchestra.run.smithi045.stdout:{"message":"","return_code":0} 2024-05-08T05:49:36.142 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-08T05:49:36.143 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:49:36.143 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:49:36.159 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:49:36.159 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:49:36.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:36.159 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T05:49:36.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:49:36.215 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-08T05:49:36.215 DEBUG:teuthology.orchestra.run.smithi045:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-08T05:49:36.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:36.258 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-05-08T05:49:36.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.366+0000 7f082201c640 1 Processor -- start 2024-05-08T05:49:36.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.366+0000 7f082201c640 1 -- start start 2024-05-08T05:49:36.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.367+0000 7f082201c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f081c162b70 0x7f081c0d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.367+0000 7f082201c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f081c162f50 con 0x7f081c0d3910 2024-05-08T05:49:36.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.367+0000 7f082201c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f081c0d3ee0 con 0x7f081c0d3480 2024-05-08T05:49:36.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.367+0000 7f082201c640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f081c0d1f60 con 0x7f081c162b70 2024-05-08T05:49:36.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.367+0000 7f081bfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f081c0d3910 0x7f081c0b5150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59978/0 (socket says 172.21.15.45:59978) 2024-05-08T05:49:36.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.367+0000 7f081bfff640 1 -- 172.21.15.45:0/1986526429 learned_addr learned my addr 172.21.15.45:0/1986526429 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:36.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.367+0000 7f081a7fc640 1 -- 172.21.15.45:0/1986526429 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1304862454 0 0) 0x7f0814001110 con 0x7f081c0d3910 2024-05-08T05:49:36.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.367+0000 7f081a7fc640 1 -- 172.21.15.45:0/1986526429 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0804003520 con 0x7f081c0d3910 2024-05-08T05:49:36.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.367+0000 7f081b7fe640 1 --2- 172.21.15.45:0/1986526429 >> v2:172.21.15.63:3300/0 conn(0x7f081c162b70 0x7f081c0d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081a7fc640 1 -- 172.21.15.45:0/1986526429 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3041158121 0 0) 0x7f0804003520 con 0x7f081c0d3910 2024-05-08T05:49:36.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081a7fc640 1 -- 172.21.15.45:0/1986526429 >> v2:172.21.15.63:3300/0 conn(0x7f081c162b70 msgr2=0x7f081c0d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081a7fc640 1 --2- 172.21.15.45:0/1986526429 >> v2:172.21.15.63:3300/0 conn(0x7f081c162b70 0x7f081c0d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081a7fc640 1 -- 172.21.15.45:0/1986526429 >> v1:172.21.15.63:6789/0 conn(0x7f081c0d3480 legacy=0x7f081c0b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081a7fc640 1 -- 172.21.15.45:0/1986526429 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081c0d21b0 con 0x7f081c0d3910 2024-05-08T05:49:36.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081a7fc640 1 -- 172.21.15.45:0/1986526429 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f08140018c0 con 0x7f081c0d3910 2024-05-08T05:49:36.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081a7fc640 1 -- 172.21.15.45:0/1986526429 >> v1:172.21.15.45:6789/0 conn(0x7f081c0d3910 legacy=0x7f081c0b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081a7fc640 1 --2- 172.21.15.45:0/1986526429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0804003a50 0x7f0804003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081a7fc640 1 --2- 172.21.15.45:0/1986526429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0804004800 0x7f0804004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081a7fc640 1 --2- 172.21.15.45:0/1986526429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08040051c0 0x7f080400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081a7fc640 1 -- 172.21.15.45:0/1986526429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f081c0d1f60 con 0x7f08040051c0 2024-05-08T05:49:36.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081a7fc640 1 -- 172.21.15.45:0/1986526429 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f080400c160 con 0x7f0804004800 2024-05-08T05:49:36.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081a7fc640 1 -- 172.21.15.45:0/1986526429 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f080400c360 con 0x7f0804003a50 2024-05-08T05:49:36.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081bfff640 1 --2- 172.21.15.45:0/1986526429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08040051c0 0x7f080400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081b7fe640 1 --2- 172.21.15.45:0/1986526429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0804003a50 0x7f0804003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081affd640 1 --2- 172.21.15.45:0/1986526429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0804004800 0x7f0804004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081bfff640 1 -- 172.21.15.45:0/1986526429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0804003a50 msgr2=0x7f0804003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081bfff640 1 --2- 172.21.15.45:0/1986526429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0804003a50 0x7f0804003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081bfff640 1 -- 172.21.15.45:0/1986526429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0804004800 msgr2=0x7f0804004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081bfff640 1 --2- 172.21.15.45:0/1986526429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0804004800 0x7f0804004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.368+0000 7f081bfff640 1 -- 172.21.15.45:0/1986526429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0804003520 con 0x7f08040051c0 2024-05-08T05:49:36.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.369+0000 7f081bfff640 1 -- 172.21.15.45:0/1986526429 reap_dead start 2024-05-08T05:49:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.369+0000 7f081bfff640 1 --2- 172.21.15.45:0/1986526429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08040051c0 0x7f080400b9c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f081400c4c0 tx=0x7f0804005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b29e2a649171525 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.369+0000 7f081a7fc640 1 -- 172.21.15.45:0/1986526429 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081400c0e0 con 0x7f08040051c0 2024-05-08T05:49:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.369+0000 7f081a7fc640 1 -- 172.21.15.45:0/1986526429 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08140058d0 con 0x7f08040051c0 2024-05-08T05:49:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.369+0000 7f081a7fc640 1 -- 172.21.15.45:0/1986526429 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814005cc0 con 0x7f08040051c0 2024-05-08T05:49:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.370+0000 7f082201c640 1 -- 172.21.15.45:0/1986526429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08040051c0 msgr2=0x7f080400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.370+0000 7f082201c640 1 --2- 172.21.15.45:0/1986526429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08040051c0 0x7f080400b9c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f081400c4c0 tx=0x7f0804005830 comp rx=0 tx=0).stop 2024-05-08T05:49:36.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.370+0000 7f082201c640 1 -- 172.21.15.45:0/1986526429 shutdown_connections 2024-05-08T05:49:36.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.370+0000 7f082201c640 1 --2- 172.21.15.45:0/1986526429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08040051c0 0x7f080400b9c0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.370+0000 7f082201c640 1 -- 172.21.15.45:0/1986526429 >> 172.21.15.45:0/1986526429 conn(0x7f081c0d5040 msgr2=0x7f081c0d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:36.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.370+0000 7f082201c640 1 -- 172.21.15.45:0/1986526429 shutdown_connections 2024-05-08T05:49:36.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.370+0000 7f082201c640 1 -- 172.21.15.45:0/1986526429 wait complete. 2024-05-08T05:49:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.371+0000 7f082201c640 1 Processor -- start 2024-05-08T05:49:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.371+0000 7f082201c640 1 -- start start 2024-05-08T05:49:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.372+0000 7f082201c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08040051c0 0x7f081c151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.372+0000 7f082201c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f081c0cf220 0x7f081c1e9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.372+0000 7f082201c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f081c1e9760 0x7f081c1f8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.372+0000 7f082201c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f081c0b42f0 con 0x7f081c0cf220 2024-05-08T05:49:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.372+0000 7f082201c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f081c0b4170 con 0x7f08040051c0 2024-05-08T05:49:36.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.372+0000 7f082201c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f081c0d2f40 con 0x7f081c1e9760 2024-05-08T05:49:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.372+0000 7f081affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f081c0cf220 0x7f081c1e9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.372+0000 7f081affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f081c0cf220 0x7f081c1e9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37792/0 (socket says 172.21.15.45:37792) 2024-05-08T05:49:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.372+0000 7f081affd640 1 -- 172.21.15.45:0/4128071641 learned_addr learned my addr 172.21.15.45:0/4128071641 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.372+0000 7f081bfff640 1 --2- 172.21.15.45:0/4128071641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f081c1e9760 0x7f081c1f8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.372+0000 7f081b7fe640 1 --2- 172.21.15.45:0/4128071641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08040051c0 0x7f081c151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.373+0000 7f081affd640 1 -- 172.21.15.45:0/4128071641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f081c1e9760 msgr2=0x7f081c1f8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.373+0000 7f081affd640 1 --2- 172.21.15.45:0/4128071641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f081c1e9760 0x7f081c1f8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.373+0000 7f081affd640 1 -- 172.21.15.45:0/4128071641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08040051c0 msgr2=0x7f081c151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.373+0000 7f081affd640 1 --2- 172.21.15.45:0/4128071641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08040051c0 0x7f081c151df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.373+0000 7f081affd640 1 -- 172.21.15.45:0/4128071641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081c0ce1e0 con 0x7f081c0cf220 2024-05-08T05:49:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.373+0000 7f081affd640 1 --2- 172.21.15.45:0/4128071641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f081c0cf220 0x7f081c1e9220 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f081000f820 tx=0x7f081000fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.373+0000 7f0803fff640 1 -- 172.21.15.45:0/4128071641 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081001a070 con 0x7f081c0cf220 2024-05-08T05:49:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.373+0000 7f0803fff640 1 -- 172.21.15.45:0/4128071641 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0810005110 con 0x7f081c0cf220 2024-05-08T05:49:36.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.373+0000 7f082201c640 1 -- 172.21.15.45:0/4128071641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081c0ce460 con 0x7f081c0cf220 2024-05-08T05:49:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.373+0000 7f0803fff640 1 -- 172.21.15.45:0/4128071641 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810007050 con 0x7f081c0cf220 2024-05-08T05:49:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.373+0000 7f082201c640 1 -- 172.21.15.45:0/4128071641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f081c0cea20 con 0x7f081c0cf220 2024-05-08T05:49:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.375+0000 7f0803fff640 1 -- 172.21.15.45:0/4128071641 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f081001d070 con 0x7f081c0cf220 2024-05-08T05:49:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.376+0000 7f0803fff640 1 --2- 172.21.15.45:0/4128071641 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0808076300 0x7f08080787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.376+0000 7f0803fff640 1 -- 172.21.15.45:0/4128071641 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f0810096f00 con 0x7f081c0cf220 2024-05-08T05:49:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.376+0000 7f081b7fe640 1 --2- 172.21.15.45:0/4128071641 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0808076300 0x7f08080787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.376+0000 7f081b7fe640 1 --2- 172.21.15.45:0/4128071641 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0808076300 0x7f08080787c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f080c014450 tx=0x7f080c00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.376+0000 7f0824e1a740 1 Processor -- start 2024-05-08T05:49:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.377+0000 7f0824e1a740 1 -- start start 2024-05-08T05:49:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.377+0000 7f0824e1a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a860220 0x55cd2a812bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.377+0000 7f0824e1a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a843340 0x55cd2a8426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.377+0000 7f0824e1a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cd2a842c00 0x55cd2a817a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.377+0000 7f0824e1a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cd2a77bcf0 con 0x55cd2a843340 2024-05-08T05:49:36.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.377+0000 7f0824e1a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cd2a734e10 con 0x55cd2a860220 2024-05-08T05:49:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.377+0000 7f0824e1a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55cd2a777cb0 con 0x55cd2a842c00 2024-05-08T05:49:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.378+0000 7f081affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a843340 0x55cd2a8426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.378+0000 7f081b7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a860220 0x55cd2a812bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.378+0000 7f081affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a843340 0x55cd2a8426c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37794/0 (socket says 172.21.15.45:37794) 2024-05-08T05:49:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.378+0000 7f081affd640 1 -- 172.21.15.45:0/25007349 learned_addr learned my addr 172.21.15.45:0/25007349 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.378+0000 7f081bfff640 1 --2- 172.21.15.45:0/25007349 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cd2a842c00 0x55cd2a817a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.378+0000 7f081affd640 1 -- 172.21.15.45:0/25007349 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cd2a842c00 msgr2=0x55cd2a817a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.378+0000 7f081affd640 1 --2- 172.21.15.45:0/25007349 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cd2a842c00 0x55cd2a817a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.378+0000 7f081affd640 1 -- 172.21.15.45:0/25007349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a860220 msgr2=0x55cd2a812bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.378+0000 7f081affd640 1 --2- 172.21.15.45:0/25007349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a860220 0x55cd2a812bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.378+0000 7f081affd640 1 -- 172.21.15.45:0/25007349 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd2a5557e0 con 0x55cd2a843340 2024-05-08T05:49:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.378+0000 7f081affd640 1 --2- 172.21.15.45:0/25007349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a843340 0x55cd2a8426c0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f0810095240 tx=0x7f081005fb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca64765be99eaab0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.378+0000 7f08017fa640 1 -- 172.21.15.45:0/25007349 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810064380 con 0x55cd2a843340 2024-05-08T05:49:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.378+0000 7f08017fa640 1 -- 172.21.15.45:0/25007349 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0810068da0 con 0x55cd2a843340 2024-05-08T05:49:36.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.378+0000 7f08017fa640 1 -- 172.21.15.45:0/25007349 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081005f4e0 con 0x55cd2a843340 2024-05-08T05:49:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.379+0000 7f0824e1a740 1 -- 172.21.15.45:0/25007349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a843340 msgr2=0x55cd2a8426c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.379+0000 7f0824e1a740 1 --2- 172.21.15.45:0/25007349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a843340 0x55cd2a8426c0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f0810095240 tx=0x7f081005fb90 comp rx=0 tx=0).stop 2024-05-08T05:49:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.379+0000 7f0824e1a740 1 -- 172.21.15.45:0/25007349 shutdown_connections 2024-05-08T05:49:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.379+0000 7f0824e1a740 1 --2- 172.21.15.45:0/25007349 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cd2a842c00 0x55cd2a817a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.379+0000 7f0824e1a740 1 --2- 172.21.15.45:0/25007349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a843340 0x55cd2a8426c0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.379+0000 7f0824e1a740 1 --2- 172.21.15.45:0/25007349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a860220 0x55cd2a812bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.379+0000 7f0824e1a740 1 -- 172.21.15.45:0/25007349 >> 172.21.15.45:0/25007349 conn(0x55cd2a814e20 msgr2=0x55cd2a85d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.379+0000 7f0824e1a740 1 -- 172.21.15.45:0/25007349 shutdown_connections 2024-05-08T05:49:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.379+0000 7f0824e1a740 1 -- 172.21.15.45:0/25007349 wait complete. 2024-05-08T05:49:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.380+0000 7f0824e1a740 1 -- start start 2024-05-08T05:49:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.380+0000 7f0824e1a740 1 Processor -- start 2024-05-08T05:49:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.380+0000 7f0824e1a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cd2a843340 0x55cd2a830080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.380+0000 7f0824e1a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a860220 0x55cd2a85ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.380+0000 7f0824e1a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a843830 0x55cd2a85d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.380+0000 7f0824e1a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cd2a816fd0 con 0x55cd2a860220 2024-05-08T05:49:36.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.380+0000 7f0824e1a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cd2a738940 con 0x55cd2a843830 2024-05-08T05:49:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.380+0000 7f0824e1a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55cd2a8163c0 con 0x55cd2a843340 2024-05-08T05:49:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.380+0000 7f0824e1a740 10 client.0 _ll_register_callbacks cb 0x55cd2a73b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:49:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.381+0000 7f081affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a860220 0x55cd2a85ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.381+0000 7f081affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a860220 0x55cd2a85ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37808/0 (socket says 172.21.15.45:37808) 2024-05-08T05:49:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.381+0000 7f081affd640 1 -- 172.21.15.45:0/425074516 learned_addr learned my addr 172.21.15.45:0/425074516 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.381+0000 7f081b7fe640 1 --2- 172.21.15.45:0/425074516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cd2a843340 0x55cd2a830080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.381+0000 7f081bfff640 1 --2- 172.21.15.45:0/425074516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a843830 0x55cd2a85d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.381+0000 7f081affd640 1 -- 172.21.15.45:0/425074516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cd2a843340 msgr2=0x55cd2a830080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.381+0000 7f081affd640 1 --2- 172.21.15.45:0/425074516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cd2a843340 0x55cd2a830080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.381+0000 7f081affd640 1 -- 172.21.15.45:0/425074516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a843830 msgr2=0x55cd2a85d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.381+0000 7f081affd640 1 --2- 172.21.15.45:0/425074516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a843830 0x55cd2a85d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.381+0000 7f081affd640 1 -- 172.21.15.45:0/425074516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd2a5b5f70 con 0x55cd2a860220 2024-05-08T05:49:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.381+0000 7f081affd640 1 --2- 172.21.15.45:0/425074516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a860220 0x55cd2a85ea20 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f0810060240 tx=0x7f0810001e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e1e64ad428c522 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.381+0000 7f07fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:49:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.381+0000 7f07fa7fc640 1 -- 172.21.15.45:0/425074516 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810064380 con 0x55cd2a860220 2024-05-08T05:49:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.381+0000 7f07fa7fc640 1 -- 172.21.15.45:0/425074516 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f081005ed50 con 0x55cd2a860220 2024-05-08T05:49:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.381+0000 7f0824e1a740 1 -- 172.21.15.45:0/425074516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0810068da0 con 0x55cd2a860220 2024-05-08T05:49:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.381+0000 7f07fa7fc640 1 -- 172.21.15.45:0/425074516 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f081005ef50 con 0x55cd2a860220 2024-05-08T05:49:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.381+0000 7f0824e1a740 10 client.7746 fetch_fsmap learned FSMap version 83 2024-05-08T05:49:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.381+0000 7f0824e1a740 1 -- 172.21.15.45:0/425074516 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x55cd2a44d170 con 0x55cd2a860220 2024-05-08T05:49:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.382+0000 7f07fa7fc640 1 -- 172.21.15.45:0/425074516 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0810068da0 con 0x55cd2a860220 2024-05-08T05:49:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.382+0000 7f07fa7fc640 1 -- 172.21.15.45:0/425074516 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7f0810079120 con 0x55cd2a860220 2024-05-08T05:49:36.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.382+0000 7f0824e1a740 10 client.7746 fetch_fsmap finished waiting for FSMap version 83 2024-05-08T05:49:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.382+0000 7f0824e1a740 10 client.7746 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-08T05:49:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.382+0000 7f0824e1a740 20 client.7746 populate_metadata read hostname 'smithi045' 2024-05-08T05:49:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.382+0000 7f0824e1a740 1 --2- 172.21.15.45:0/425074516 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55cd2a85e220 0x55cd2a832940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.382+0000 7f0824e1a740 4 client.7746 mds_command: new command op to 7524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:49:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.382+0000 7f0824e1a740 1 -- 172.21.15.45:0/425074516 --> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd2a789900 con 0x55cd2a85e220 2024-05-08T05:49:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.382+0000 7f081b7fe640 1 --2- 172.21.15.45:0/425074516 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55cd2a85e220 0x55cd2a832940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.382+0000 7f07fa7fc640 10 client.7746 ms_handle_connect on v2:172.21.15.45:6834/878092981 2024-05-08T05:49:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.382+0000 7f081b7fe640 1 --2- 172.21.15.45:0/425074516 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55cd2a85e220 0x55cd2a832940 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2233100703f6dd67 server_cookie=82c0f8f00b2d170a in_seq=0 out_seq=0 2024-05-08T05:49:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.383+0000 7f07fa7fc640 1 -- 172.21.15.45:0/425074516 <== mds.0 v2:172.21.15.45:6834/878092981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cd2a789900 con 0x55cd2a85e220 2024-05-08T05:49:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.383+0000 7f07fa7fc640 10 client.7746 handle_command_reply: tid=0 2024-05-08T05:49:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.383+0000 7f0824e1a740 1 client.7746 shutdown 2024-05-08T05:49:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.383+0000 7f0824e1a740 1 -- 172.21.15.45:0/425074516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a860220 msgr2=0x55cd2a85ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.383+0000 7f0824e1a740 1 --2- 172.21.15.45:0/425074516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a860220 0x55cd2a85ea20 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f0810060240 tx=0x7f0810001e40 comp rx=0 tx=0).stop 2024-05-08T05:49:36.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.383+0000 7f0824e1a740 1 -- 172.21.15.45:0/425074516 shutdown_connections 2024-05-08T05:49:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.383+0000 7f0824e1a740 1 --2- 172.21.15.45:0/425074516 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55cd2a85e220 0x55cd2a832940 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.383+0000 7f0824e1a740 1 --2- 172.21.15.45:0/425074516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a843830 0x55cd2a85d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.383+0000 7f07fa7fc640 0 client.7746 ms_handle_reset on v2:172.21.15.45:6834/878092981 2024-05-08T05:49:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.383+0000 7f0824e1a740 1 --2- 172.21.15.45:0/425074516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a860220 0x55cd2a85ea20 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.383+0000 7f0824e1a740 1 --2- 172.21.15.45:0/425074516 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cd2a843340 0x55cd2a830080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.383+0000 7f0824e1a740 1 -- 172.21.15.45:0/425074516 >> 172.21.15.45:0/425074516 conn(0x55cd2a842c00 msgr2=0x55cd2a8122f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.383+0000 7f0824e1a740 1 -- 172.21.15.45:0/425074516 shutdown_connections 2024-05-08T05:49:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.383+0000 7f0824e1a740 1 -- 172.21.15.45:0/425074516 wait complete. 2024-05-08T05:49:36.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.383+0000 7f0824e1a740 20 client.7746 trim_cache size 0 max 16384 2024-05-08T05:49:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.386+0000 7f9f20b46640 1 Processor -- start 2024-05-08T05:49:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.386+0000 7f9f20b46640 1 -- start start 2024-05-08T05:49:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.387+0000 7f9f20b46640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9f1c1599c0 0x7f9f1c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.387+0000 7f9f20b46640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f1c06c440 con 0x7f9f1c163440 2024-05-08T05:49:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.387+0000 7f9f20b46640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f1c0b08e0 con 0x7f9f1c068520 2024-05-08T05:49:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.387+0000 7f9f20b46640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9f1c0b06b0 con 0x7f9f1c1599c0 2024-05-08T05:49:36.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.387+0000 7f9f1ad9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9f1c163440 0x7f9f1c06cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59992/0 (socket says 172.21.15.45:59992) 2024-05-08T05:49:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.387+0000 7f9f1ad9d640 1 -- 172.21.15.45:0/3949123456 learned_addr learned my addr 172.21.15.45:0/3949123456 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f1a59c640 1 --2- 172.21.15.45:0/3949123456 >> v2:172.21.15.63:3300/0 conn(0x7f9f1c1599c0 0x7f9f1c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f19d9b640 1 -- 172.21.15.45:0/3949123456 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4036091990 0 0) 0x7f9f1c06c440 con 0x7f9f1c163440 2024-05-08T05:49:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f19d9b640 1 -- 172.21.15.45:0/3949123456 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f08003520 con 0x7f9f1c163440 2024-05-08T05:49:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f19d9b640 1 -- 172.21.15.45:0/3949123456 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1270065123 0 0) 0x7f9f08003520 con 0x7f9f1c163440 2024-05-08T05:49:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f19d9b640 1 -- 172.21.15.45:0/3949123456 >> v2:172.21.15.63:3300/0 conn(0x7f9f1c1599c0 msgr2=0x7f9f1c0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f19d9b640 1 --2- 172.21.15.45:0/3949123456 >> v2:172.21.15.63:3300/0 conn(0x7f9f1c1599c0 0x7f9f1c0cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f19d9b640 1 -- 172.21.15.45:0/3949123456 >> v1:172.21.15.63:6789/0 conn(0x7f9f1c068520 legacy=0x7f9f1c162d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f19d9b640 1 -- 172.21.15.45:0/3949123456 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1c0b0bc0 con 0x7f9f1c163440 2024-05-08T05:49:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f19d9b640 1 -- 172.21.15.45:0/3949123456 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f9f14001a70 con 0x7f9f1c163440 2024-05-08T05:49:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f19d9b640 1 -- 172.21.15.45:0/3949123456 >> v1:172.21.15.45:6789/0 conn(0x7f9f1c163440 legacy=0x7f9f1c06cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f19d9b640 1 --2- 172.21.15.45:0/3949123456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f08003a00 0x7f9f08003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f19d9b640 1 --2- 172.21.15.45:0/3949123456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f08004800 0x7f9f08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f19d9b640 1 --2- 172.21.15.45:0/3949123456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f080051c0 0x7f9f0800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f19d9b640 1 -- 172.21.15.45:0/3949123456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0800c0e0 con 0x7f9f080051c0 2024-05-08T05:49:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f19d9b640 1 -- 172.21.15.45:0/3949123456 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0800c290 con 0x7f9f08004800 2024-05-08T05:49:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f19d9b640 1 -- 172.21.15.45:0/3949123456 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9f0800c490 con 0x7f9f08003a00 2024-05-08T05:49:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f1ad9d640 1 --2- 172.21.15.45:0/3949123456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f080051c0 0x7f9f0800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f1a59c640 1 --2- 172.21.15.45:0/3949123456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f08003a00 0x7f9f08003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f11d9b640 1 --2- 172.21.15.45:0/3949123456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f08004800 0x7f9f08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f1ad9d640 1 -- 172.21.15.45:0/3949123456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f08003a00 msgr2=0x7f9f08003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f1ad9d640 1 --2- 172.21.15.45:0/3949123456 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f08003a00 0x7f9f08003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f1ad9d640 1 -- 172.21.15.45:0/3949123456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f08004800 msgr2=0x7f9f08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f1ad9d640 1 --2- 172.21.15.45:0/3949123456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f08004800 0x7f9f08004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f1ad9d640 1 -- 172.21.15.45:0/3949123456 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f08003520 con 0x7f9f080051c0 2024-05-08T05:49:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.388+0000 7f9f11d9b640 1 --2- 172.21.15.45:0/3949123456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f08004800 0x7f9f08004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.389+0000 7f9f1ad9d640 1 -- 172.21.15.45:0/3949123456 reap_dead start 2024-05-08T05:49:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.389+0000 7f9f1ad9d640 1 --2- 172.21.15.45:0/3949123456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f080051c0 0x7f9f0800b9c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400a7c0 tx=0x7f9f08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f172b650fe77f037 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.389+0000 7f9f19d9b640 1 -- 172.21.15.45:0/3949123456 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1400a410 con 0x7f9f080051c0 2024-05-08T05:49:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.389+0000 7f9f19d9b640 1 -- 172.21.15.45:0/3949123456 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f1402f070 con 0x7f9f080051c0 2024-05-08T05:49:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.389+0000 7f9f19d9b640 1 -- 172.21.15.45:0/3949123456 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1402bb90 con 0x7f9f080051c0 2024-05-08T05:49:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.390+0000 7f9f20b46640 1 -- 172.21.15.45:0/3949123456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f080051c0 msgr2=0x7f9f0800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.390+0000 7f9f20b46640 1 --2- 172.21.15.45:0/3949123456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f080051c0 0x7f9f0800b9c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400a7c0 tx=0x7f9f08005830 comp rx=0 tx=0).stop 2024-05-08T05:49:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.390+0000 7f9f20b46640 1 -- 172.21.15.45:0/3949123456 shutdown_connections 2024-05-08T05:49:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.390+0000 7f9f20b46640 1 --2- 172.21.15.45:0/3949123456 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f080051c0 0x7f9f0800b9c0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.390+0000 7f9f20b46640 1 -- 172.21.15.45:0/3949123456 >> 172.21.15.45:0/3949123456 conn(0x7f9f1c0d1490 msgr2=0x7f9f1c0d1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:36.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.390+0000 7f9f20b46640 1 -- 172.21.15.45:0/3949123456 shutdown_connections 2024-05-08T05:49:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.390+0000 7f9f20b46640 1 -- 172.21.15.45:0/3949123456 wait complete. 2024-05-08T05:49:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.392+0000 7f9f20b46640 1 Processor -- start 2024-05-08T05:49:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.392+0000 7f9f20b46640 1 -- start start 2024-05-08T05:49:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.392+0000 7f9f20b46640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f080051c0 0x7f9f1c157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.392+0000 7f9f20b46640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f1c15cb20 0x7f9f1c15a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.392+0000 7f9f20b46640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f1c15cfa0 0x7f9f1c15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.392+0000 7f9f20b46640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c06c6b0 con 0x7f9f1c15cfa0 2024-05-08T05:49:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.392+0000 7f9f20b46640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c0b6430 con 0x7f9f1c15cb20 2024-05-08T05:49:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.392+0000 7f9f20b46640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9f1c0b0ea0 con 0x7f9f080051c0 2024-05-08T05:49:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.392+0000 7f9f1ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f1c15cfa0 0x7f9f1c15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.393+0000 7f9f1ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f1c15cfa0 0x7f9f1c15ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37822/0 (socket says 172.21.15.45:37822) 2024-05-08T05:49:36.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.393+0000 7f9f1ad9d640 1 -- 172.21.15.45:0/1723978194 learned_addr learned my addr 172.21.15.45:0/1723978194 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:36.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.393+0000 7f9f11d9b640 1 --2- 172.21.15.45:0/1723978194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f1c15cb20 0x7f9f1c15a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.393+0000 7f9f1a59c640 1 --2- 172.21.15.45:0/1723978194 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f080051c0 0x7f9f1c157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.393+0000 7f9f1ad9d640 1 -- 172.21.15.45:0/1723978194 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f080051c0 msgr2=0x7f9f1c157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.393+0000 7f9f1ad9d640 1 --2- 172.21.15.45:0/1723978194 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f080051c0 0x7f9f1c157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.393+0000 7f9f1ad9d640 1 -- 172.21.15.45:0/1723978194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f1c15cb20 msgr2=0x7f9f1c15a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.393+0000 7f9f1ad9d640 1 --2- 172.21.15.45:0/1723978194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f1c15cb20 0x7f9f1c15a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.393+0000 7f9f1ad9d640 1 -- 172.21.15.45:0/1723978194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1c160ff0 con 0x7f9f1c15cfa0 2024-05-08T05:49:36.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.393+0000 7f9f11d9b640 1 --2- 172.21.15.45:0/1723978194 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f1c15cb20 0x7f9f1c15a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:36.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.393+0000 7f9f1ad9d640 1 --2- 172.21.15.45:0/1723978194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f1c15cfa0 0x7f9f1c15ae90 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f9f14000f80 tx=0x7f9f14001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:36.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.393+0000 7f9f137fe640 1 -- 172.21.15.45:0/1723978194 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14004d60 con 0x7f9f1c15cfa0 2024-05-08T05:49:36.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.393+0000 7f9f20b46640 1 -- 172.21.15.45:0/1723978194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1c06f6c0 con 0x7f9f1c15cfa0 2024-05-08T05:49:36.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.393+0000 7f9f137fe640 1 -- 172.21.15.45:0/1723978194 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f1402b940 con 0x7f9f1c15cfa0 2024-05-08T05:49:36.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.393+0000 7f9f20b46640 1 -- 172.21.15.45:0/1723978194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f1c06fc80 con 0x7f9f1c15cfa0 2024-05-08T05:49:36.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.393+0000 7f9f137fe640 1 -- 172.21.15.45:0/1723978194 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1402bd30 con 0x7f9f1c15cfa0 2024-05-08T05:49:36.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.394+0000 7f9f20b46640 1 -- 172.21.15.45:0/1723978194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f9f1c0693f0 con 0x7f9f1c15cfa0 2024-05-08T05:49:36.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.395+0000 7f9f137fe640 1 -- 172.21.15.45:0/1723978194 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9f14034470 con 0x7f9f1c15cfa0 2024-05-08T05:49:36.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.396+0000 7f9f137fe640 1 --2- 172.21.15.45:0/1723978194 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9efc076300 0x7f9efc0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.396+0000 7f9f137fe640 1 -- 172.21.15.45:0/1723978194 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f9f140b7910 con 0x7f9f1c15cfa0 2024-05-08T05:49:36.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.396+0000 7f9f1a59c640 1 --2- 172.21.15.45:0/1723978194 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9efc076300 0x7f9efc0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.396+0000 7f9f137fe640 1 -- 172.21.15.45:0/1723978194 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+511 (secure 0 0 0) 0x7f9f14047020 con 0x7f9f1c15cfa0 2024-05-08T05:49:36.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.396+0000 7f9f20b46640 1 -- 172.21.15.45:0/1723978194 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f9f1c0681e0 con 0x7f9f1c15cfa0 2024-05-08T05:49:36.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.396+0000 7f9f1a59c640 1 --2- 172.21.15.45:0/1723978194 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9efc076300 0x7f9efc0787c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c009a80 tx=0x7f9f0c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:36.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.403+0000 7f0824e1a740 1 Processor -- start 2024-05-08T05:49:36.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.403+0000 7f0824e1a740 1 -- start start 2024-05-08T05:49:36.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.404+0000 7f0824e1a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a843340 0x55cd2a830ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.404+0000 7f0824e1a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cd2a843830 0x55cd2a860cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.404+0000 7f0824e1a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a860220 0x55cd2a841690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.404+0000 7f0824e1a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f081c202560 con 0x55cd2a860220 2024-05-08T05:49:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.404+0000 7f0824e1a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f081c201580 con 0x55cd2a843340 2024-05-08T05:49:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.404+0000 7f0824e1a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f081c2008e0 con 0x55cd2a843830 2024-05-08T05:49:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.404+0000 7f081bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a860220 0x55cd2a841690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.404+0000 7f081bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a860220 0x55cd2a841690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37826/0 (socket says 172.21.15.45:37826) 2024-05-08T05:49:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.404+0000 7f081bfff640 1 -- 172.21.15.45:0/1757994341 learned_addr learned my addr 172.21.15.45:0/1757994341 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.404+0000 7f081b7fe640 1 --2- 172.21.15.45:0/1757994341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a843340 0x55cd2a830ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.404+0000 7f081affd640 1 --2- 172.21.15.45:0/1757994341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cd2a843830 0x55cd2a860cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.404+0000 7f081bfff640 1 -- 172.21.15.45:0/1757994341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cd2a843830 msgr2=0x55cd2a860cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.404+0000 7f081bfff640 1 --2- 172.21.15.45:0/1757994341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cd2a843830 0x55cd2a860cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.404+0000 7f081bfff640 1 -- 172.21.15.45:0/1757994341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a843340 msgr2=0x55cd2a830ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.404+0000 7f081bfff640 1 --2- 172.21.15.45:0/1757994341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a843340 0x55cd2a830ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.404+0000 7f081bfff640 1 -- 172.21.15.45:0/1757994341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd2a831010 con 0x55cd2a860220 2024-05-08T05:49:36.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.404+0000 7f081b7fe640 1 --2- 172.21.15.45:0/1757994341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a843340 0x55cd2a830ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:49:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.405+0000 7f081bfff640 1 --2- 172.21.15.45:0/1757994341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a860220 0x55cd2a841690 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f081402afb0 tx=0x7f0814007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad1c4f2f86c9a96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.405+0000 7f0800ff9640 1 -- 172.21.15.45:0/1757994341 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814001740 con 0x55cd2a860220 2024-05-08T05:49:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.405+0000 7f0800ff9640 1 -- 172.21.15.45:0/1757994341 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08140047e0 con 0x55cd2a860220 2024-05-08T05:49:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.405+0000 7f0800ff9640 1 -- 172.21.15.45:0/1757994341 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814004bb0 con 0x55cd2a860220 2024-05-08T05:49:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.405+0000 7f0824e1a740 1 -- 172.21.15.45:0/1757994341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a860220 msgr2=0x55cd2a841690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.405+0000 7f0824e1a740 1 --2- 172.21.15.45:0/1757994341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a860220 0x55cd2a841690 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f081402afb0 tx=0x7f0814007000 comp rx=0 tx=0).stop 2024-05-08T05:49:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.406+0000 7f0824e1a740 1 -- 172.21.15.45:0/1757994341 shutdown_connections 2024-05-08T05:49:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.406+0000 7f0824e1a740 1 --2- 172.21.15.45:0/1757994341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cd2a843830 0x55cd2a860cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.406+0000 7f0824e1a740 1 --2- 172.21.15.45:0/1757994341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a860220 0x55cd2a841690 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.406+0000 7f0824e1a740 1 --2- 172.21.15.45:0/1757994341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a843340 0x55cd2a830ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.406+0000 7f0824e1a740 1 -- 172.21.15.45:0/1757994341 >> 172.21.15.45:0/1757994341 conn(0x55cd2a842c00 msgr2=0x55cd2a8800e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.406+0000 7f0824e1a740 1 -- 172.21.15.45:0/1757994341 shutdown_connections 2024-05-08T05:49:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.406+0000 7f0824e1a740 1 -- 172.21.15.45:0/1757994341 wait complete. 2024-05-08T05:49:36.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.406+0000 7f0824e1a740 1 -- start start 2024-05-08T05:49:36.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.406+0000 7f0824e1a740 1 Processor -- start 2024-05-08T05:49:36.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.407+0000 7f0824e1a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a843340 0x55cd2a83d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.407+0000 7f0824e1a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cd2a843830 0x55cd2a8405b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.407+0000 7f0824e1a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a860220 0x55cd2a816480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.407+0000 7f0824e1a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cd2a85f2b0 con 0x55cd2a843340 2024-05-08T05:49:36.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.407+0000 7f0824e1a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f081c200f30 con 0x55cd2a860220 2024-05-08T05:49:36.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.407+0000 7f0824e1a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55cd2a8138f0 con 0x55cd2a843830 2024-05-08T05:49:36.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.407+0000 7f0824e1a740 10 client.0 _ll_register_callbacks cb 0x7f0810079070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:49:36.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.407+0000 7f081b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a843340 0x55cd2a83d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.407+0000 7f081b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a843340 0x55cd2a83d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37838/0 (socket says 172.21.15.45:37838) 2024-05-08T05:49:36.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.407+0000 7f081b7fe640 1 -- 172.21.15.45:0/389841613 learned_addr learned my addr 172.21.15.45:0/389841613 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:36.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.407+0000 7f081affd640 1 --2- 172.21.15.45:0/389841613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cd2a843830 0x55cd2a8405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.407+0000 7f081bfff640 1 --2- 172.21.15.45:0/389841613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a860220 0x55cd2a816480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.407+0000 7f081b7fe640 1 -- 172.21.15.45:0/389841613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cd2a843830 msgr2=0x55cd2a8405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.407+0000 7f081b7fe640 1 --2- 172.21.15.45:0/389841613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cd2a843830 0x55cd2a8405b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.407+0000 7f081b7fe640 1 -- 172.21.15.45:0/389841613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a860220 msgr2=0x55cd2a816480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.407+0000 7f081b7fe640 1 --2- 172.21.15.45:0/389841613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a860220 0x55cd2a816480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.407+0000 7f081b7fe640 1 -- 172.21.15.45:0/389841613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd2a8530f0 con 0x55cd2a843340 2024-05-08T05:49:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.407+0000 7f081bfff640 1 --2- 172.21.15.45:0/389841613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a860220 0x55cd2a816480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:49:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.408+0000 7f081b7fe640 1 --2- 172.21.15.45:0/389841613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a843340 0x55cd2a83d690 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f080c016b90 tx=0x7f080c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb385695729a54f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.408+0000 7f07fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:49:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.408+0000 7f07fa7fc640 1 -- 172.21.15.45:0/389841613 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c01a040 con 0x55cd2a843340 2024-05-08T05:49:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.408+0000 7f07fa7fc640 1 -- 172.21.15.45:0/389841613 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f080c02e440 con 0x55cd2a843340 2024-05-08T05:49:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.408+0000 7f0824e1a740 1 -- 172.21.15.45:0/389841613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08140047e0 con 0x55cd2a843340 2024-05-08T05:49:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.408+0000 7f07fa7fc640 1 -- 172.21.15.45:0/389841613 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f080c03c050 con 0x55cd2a843340 2024-05-08T05:49:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.408+0000 7f0824e1a740 10 client.7758 fetch_fsmap learned FSMap version 83 2024-05-08T05:49:36.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.408+0000 7f0824e1a740 1 -- 172.21.15.45:0/389841613 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x55cd2a83f980 con 0x55cd2a843340 2024-05-08T05:49:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.408+0000 7f07fa7fc640 1 -- 172.21.15.45:0/389841613 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08140047e0 con 0x55cd2a843340 2024-05-08T05:49:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.408+0000 7f07fa7fc640 1 -- 172.21.15.45:0/389841613 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7f080c03f020 con 0x55cd2a843340 2024-05-08T05:49:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.408+0000 7f0824e1a740 10 client.7758 fetch_fsmap finished waiting for FSMap version 83 2024-05-08T05:49:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.408+0000 7f0824e1a740 10 client.7758 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-08T05:49:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.408+0000 7f0824e1a740 20 client.7758 populate_metadata read hostname 'smithi045' 2024-05-08T05:49:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.408+0000 7f0824e1a740 1 --2- 172.21.15.45:0/389841613 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55cd2a840af0 0x55cd2a8169c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.408+0000 7f0824e1a740 4 client.7758 mds_command: new command op to 7524 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-08T05:49:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.408+0000 7f0824e1a740 1 -- 172.21.15.45:0/389841613 --> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55cd2a783770 con 0x55cd2a840af0 2024-05-08T05:49:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.408+0000 7f081affd640 1 --2- 172.21.15.45:0/389841613 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55cd2a840af0 0x55cd2a8169c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.409+0000 7f07fa7fc640 10 client.7758 ms_handle_connect on v2:172.21.15.45:6834/878092981 2024-05-08T05:49:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.409+0000 7f081affd640 1 --2- 172.21.15.45:0/389841613 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55cd2a840af0 0x55cd2a8169c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a795cf5b9a6c7c0a server_cookie=f23949b5554af990 in_seq=0 out_seq=0 2024-05-08T05:49:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.409+0000 7f07fa7fc640 1 -- 172.21.15.45:0/389841613 <== mds.0 v2:172.21.15.45:6834/878092981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cd2a783770 con 0x55cd2a840af0 2024-05-08T05:49:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.409+0000 7f07fa7fc640 10 client.7758 handle_command_reply: tid=0 2024-05-08T05:49:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.409+0000 7f0824e1a740 1 client.7758 shutdown 2024-05-08T05:49:36.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.409+0000 7f0824e1a740 1 -- 172.21.15.45:0/389841613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a843340 msgr2=0x55cd2a83d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.409+0000 7f0824e1a740 1 --2- 172.21.15.45:0/389841613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a843340 0x55cd2a83d690 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f080c016b90 tx=0x7f080c004ab0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.409+0000 7f0824e1a740 1 -- 172.21.15.45:0/389841613 shutdown_connections 2024-05-08T05:49:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.409+0000 7f0824e1a740 1 --2- 172.21.15.45:0/389841613 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55cd2a840af0 0x55cd2a8169c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.409+0000 7f0824e1a740 1 --2- 172.21.15.45:0/389841613 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cd2a843830 0x55cd2a8405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.409+0000 7f07fa7fc640 0 client.7758 ms_handle_reset on v2:172.21.15.45:6834/878092981 2024-05-08T05:49:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.409+0000 7f0824e1a740 1 --2- 172.21.15.45:0/389841613 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cd2a860220 0x55cd2a816480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.409+0000 7f0824e1a740 1 --2- 172.21.15.45:0/389841613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cd2a843340 0x55cd2a83d690 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.409+0000 7f0824e1a740 1 -- 172.21.15.45:0/389841613 >> 172.21.15.45:0/389841613 conn(0x55cd2a842c00 msgr2=0x55cd2a86d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.410+0000 7f0824e1a740 1 -- 172.21.15.45:0/389841613 shutdown_connections 2024-05-08T05:49:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.410+0000 7f0824e1a740 1 -- 172.21.15.45:0/389841613 wait complete. 2024-05-08T05:49:36.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.410+0000 7f0824e1a740 20 client.7758 trim_cache size 0 max 16384 2024-05-08T05:49:36.421 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:49:36.421 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-05-08T05:49:36.421 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "f3388a8c-bd7b-41d7-ba2e-af000ca2e492", 2024-05-08T05:49:36.421 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-05-08T05:49:36.421 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:49:36.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.410+0000 7f082201c640 1 -- 172.21.15.45:0/4128071641 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0808076300 msgr2=0x7f08080787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.410+0000 7f082201c640 1 --2- 172.21.15.45:0/4128071641 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0808076300 0x7f08080787c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f080c014450 tx=0x7f080c00b040 comp rx=0 tx=0).stop 2024-05-08T05:49:36.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.410+0000 7f082201c640 1 -- 172.21.15.45:0/4128071641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f081c0cf220 msgr2=0x7f081c1e9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.410+0000 7f082201c640 1 --2- 172.21.15.45:0/4128071641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f081c0cf220 0x7f081c1e9220 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f081000f820 tx=0x7f081000fce0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.410+0000 7f082201c640 1 -- 172.21.15.45:0/4128071641 shutdown_connections 2024-05-08T05:49:36.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.410+0000 7f082201c640 1 --2- 172.21.15.45:0/4128071641 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0808076300 0x7f08080787c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.410+0000 7f082201c640 1 --2- 172.21.15.45:0/4128071641 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f081c1e9760 0x7f081c1f8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.410+0000 7f082201c640 1 --2- 172.21.15.45:0/4128071641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f081c0cf220 0x7f081c1e9220 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.410+0000 7f082201c640 1 --2- 172.21.15.45:0/4128071641 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f08040051c0 0x7f081c151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.410+0000 7f082201c640 1 -- 172.21.15.45:0/4128071641 >> 172.21.15.45:0/4128071641 conn(0x7f081c0d5040 msgr2=0x7f081c161d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:36.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.411+0000 7f082201c640 1 -- 172.21.15.45:0/4128071641 shutdown_connections 2024-05-08T05:49:36.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.411+0000 7f082201c640 1 -- 172.21.15.45:0/4128071641 wait complete. 2024-05-08T05:49:36.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:49:36.424 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-08T05:49:36.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.529+0000 7fc36b37f640 1 Processor -- start 2024-05-08T05:49:36.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.530+0000 7fc36b37f640 1 -- start start 2024-05-08T05:49:36.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc36b37f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3640d1970 0x7fc3640d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc36b37f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3641630b0 con 0x7fc36415f420 2024-05-08T05:49:36.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc36b37f640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3640d3e70 con 0x7fc3640d4cc0 2024-05-08T05:49:36.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc36b37f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc3640d2330 con 0x7fc3640d1970 2024-05-08T05:49:36.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc36911b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc36415f420 0x7fc3640d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60008/0 (socket says 172.21.15.45:60008) 2024-05-08T05:49:36.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc36911b640 1 -- 172.21.15.45:0/447961400 learned_addr learned my addr 172.21.15.45:0/447961400 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:49:36.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc363fff640 1 -- 172.21.15.45:0/447961400 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3997154207 0 0) 0x7fc35c001490 con 0x7fc36415f420 2024-05-08T05:49:36.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc363fff640 1 -- 172.21.15.45:0/447961400 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc34c003520 con 0x7fc36415f420 2024-05-08T05:49:36.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc36891a640 1 --2- 172.21.15.45:0/447961400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3640d1970 0x7fc3640d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc363fff640 1 -- 172.21.15.45:0/447961400 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1296472842 0 0) 0x7fc34c003520 con 0x7fc36415f420 2024-05-08T05:49:36.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc363fff640 1 -- 172.21.15.45:0/447961400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3640d1970 msgr2=0x7fc3640d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc363fff640 1 --2- 172.21.15.45:0/447961400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3640d1970 0x7fc3640d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc363fff640 1 -- 172.21.15.45:0/447961400 >> v1:172.21.15.63:6789/0 conn(0x7fc3640d4cc0 legacy=0x7fc3640d5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc363fff640 1 -- 172.21.15.45:0/447961400 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3640d2550 con 0x7fc36415f420 2024-05-08T05:49:36.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc363fff640 1 -- 172.21.15.45:0/447961400 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fc35c001c40 con 0x7fc36415f420 2024-05-08T05:49:36.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc363fff640 1 -- 172.21.15.45:0/447961400 >> v1:172.21.15.45:6789/0 conn(0x7fc36415f420 legacy=0x7fc3640d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc363fff640 1 --2- 172.21.15.45:0/447961400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc34c003a30 0x7fc34c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc363fff640 1 --2- 172.21.15.45:0/447961400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3640d1970 0x7fc34c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc363fff640 1 --2- 172.21.15.45:0/447961400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc34c004d40 0x7fc34c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc363fff640 1 -- 172.21.15.45:0/447961400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc3640d2330 con 0x7fc34c004d40 2024-05-08T05:49:36.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc363fff640 1 -- 172.21.15.45:0/447961400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c00baf0 con 0x7fc34c003a30 2024-05-08T05:49:36.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc363fff640 1 -- 172.21.15.45:0/447961400 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc34c00bcf0 con 0x7fc3640d1970 2024-05-08T05:49:36.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc36911b640 1 --2- 172.21.15.45:0/447961400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc34c003a30 0x7fc34c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.531+0000 7fc36991c640 1 --2- 172.21.15.45:0/447961400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc34c004d40 0x7fc34c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.532+0000 7fc36891a640 1 --2- 172.21.15.45:0/447961400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3640d1970 0x7fc34c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.532+0000 7fc36991c640 1 -- 172.21.15.45:0/447961400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3640d1970 msgr2=0x7fc34c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.532+0000 7fc36991c640 1 --2- 172.21.15.45:0/447961400 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc3640d1970 0x7fc34c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.532+0000 7fc36991c640 1 -- 172.21.15.45:0/447961400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc34c003a30 msgr2=0x7fc34c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.532+0000 7fc36991c640 1 --2- 172.21.15.45:0/447961400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc34c003a30 0x7fc34c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.532+0000 7fc36991c640 1 -- 172.21.15.45:0/447961400 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc34c003520 con 0x7fc34c004d40 2024-05-08T05:49:36.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.533+0000 7fc36991c640 1 --2- 172.21.15.45:0/447961400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc34c004d40 0x7fc34c0051c0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fc354001790 tx=0x7fc34c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3af6837f30dbbc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:36.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.533+0000 7fc363fff640 1 -- 172.21.15.45:0/447961400 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3540013b0 con 0x7fc34c004d40 2024-05-08T05:49:36.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.533+0000 7fc363fff640 1 -- 172.21.15.45:0/447961400 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3540052d0 con 0x7fc34c004d40 2024-05-08T05:49:36.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.533+0000 7fc363fff640 1 -- 172.21.15.45:0/447961400 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35402aae0 con 0x7fc34c004d40 2024-05-08T05:49:36.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.533+0000 7fc36b37f640 1 -- 172.21.15.45:0/447961400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc34c004d40 msgr2=0x7fc34c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.533+0000 7fc36b37f640 1 --2- 172.21.15.45:0/447961400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc34c004d40 0x7fc34c0051c0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fc354001790 tx=0x7fc34c005830 comp rx=0 tx=0).stop 2024-05-08T05:49:36.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.533+0000 7fc36991c640 1 -- 172.21.15.45:0/447961400 reap_dead start 2024-05-08T05:49:36.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.533+0000 7fc36b37f640 1 -- 172.21.15.45:0/447961400 shutdown_connections 2024-05-08T05:49:36.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.533+0000 7fc36b37f640 1 -- 172.21.15.45:0/447961400 >> 172.21.15.45:0/447961400 conn(0x7fc36408eac0 msgr2=0x7fc36409e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:36.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.533+0000 7fc36b37f640 1 -- 172.21.15.45:0/447961400 shutdown_connections 2024-05-08T05:49:36.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.533+0000 7fc36b37f640 1 -- 172.21.15.45:0/447961400 wait complete. 2024-05-08T05:49:36.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.535+0000 7fc36b37f640 1 Processor -- start 2024-05-08T05:49:36.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.535+0000 7fc36b37f640 1 -- start start 2024-05-08T05:49:36.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.535+0000 7fc36b37f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc34c004d40 0x7fc3641f8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.535+0000 7fc36b37f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3640c9530 0x7fc3640b96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.535+0000 7fc36b37f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3640cad10 0x7fc3640b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.535+0000 7fc36b37f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc3640d3240 con 0x7fc3640cad10 2024-05-08T05:49:36.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.535+0000 7fc36b37f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc3640d2f60 con 0x7fc3640c9530 2024-05-08T05:49:36.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.535+0000 7fc36b37f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc3640d2830 con 0x7fc34c004d40 2024-05-08T05:49:36.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.535+0000 7fc36991c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3640cad10 0x7fc3640b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.536+0000 7fc36991c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3640cad10 0x7fc3640b9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37848/0 (socket says 172.21.15.45:37848) 2024-05-08T05:49:36.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.536+0000 7fc36991c640 1 -- 172.21.15.45:0/3185318005 learned_addr learned my addr 172.21.15.45:0/3185318005 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:36.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.536+0000 7fc36911b640 1 --2- 172.21.15.45:0/3185318005 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc34c004d40 0x7fc3641f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.536+0000 7fc36891a640 1 --2- 172.21.15.45:0/3185318005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3640c9530 0x7fc3640b96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.536+0000 7fc36991c640 1 -- 172.21.15.45:0/3185318005 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc34c004d40 msgr2=0x7fc3641f8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.536+0000 7fc36991c640 1 --2- 172.21.15.45:0/3185318005 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc34c004d40 0x7fc3641f8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.536+0000 7fc36991c640 1 -- 172.21.15.45:0/3185318005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3640c9530 msgr2=0x7fc3640b96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.536+0000 7fc36991c640 1 --2- 172.21.15.45:0/3185318005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3640c9530 0x7fc3640b96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.536+0000 7fc36991c640 1 -- 172.21.15.45:0/3185318005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3640c9aa0 con 0x7fc3640cad10 2024-05-08T05:49:36.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.536+0000 7fc36991c640 1 --2- 172.21.15.45:0/3185318005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3640cad10 0x7fc3640b9bf0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fc3540018b0 tx=0x7fc354001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:36.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.536+0000 7fc361ffb640 1 -- 172.21.15.45:0/3185318005 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354037040 con 0x7fc3640cad10 2024-05-08T05:49:36.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.536+0000 7fc361ffb640 1 -- 172.21.15.45:0/3185318005 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc354040070 con 0x7fc3640cad10 2024-05-08T05:49:36.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.536+0000 7fc36b37f640 1 -- 172.21.15.45:0/3185318005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3640c9d80 con 0x7fc3640cad10 2024-05-08T05:49:36.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.536+0000 7fc361ffb640 1 -- 172.21.15.45:0/3185318005 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354007a70 con 0x7fc3640cad10 2024-05-08T05:49:36.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.536+0000 7fc36b37f640 1 -- 172.21.15.45:0/3185318005 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3640ca340 con 0x7fc3640cad10 2024-05-08T05:49:36.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.538+0000 7fc361ffb640 1 -- 172.21.15.45:0/3185318005 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc3540052d0 con 0x7fc3640cad10 2024-05-08T05:49:36.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.538+0000 7fc361ffb640 1 --2- 172.21.15.45:0/3185318005 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc350076300 0x7fc3500787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.539+0000 7fc361ffb640 1 -- 172.21.15.45:0/3185318005 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fc3540b6ea0 con 0x7fc3640cad10 2024-05-08T05:49:36.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.539+0000 7fc36911b640 1 --2- 172.21.15.45:0/3185318005 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc350076300 0x7fc3500787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.539+0000 7fc36911b640 1 --2- 172.21.15.45:0/3185318005 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc350076300 0x7fc3500787c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc35c001d60 tx=0x7fc35c006040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:36.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.540+0000 7fc36e171740 1 Processor -- start 2024-05-08T05:49:36.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.540+0000 7fc36e171740 1 -- start start 2024-05-08T05:49:36.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.540+0000 7fc36e171740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e8426e1160 0x55e842693af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.540+0000 7fc36e171740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426c4280 0x55e8426c3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.540+0000 7fc36e171740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8426c3b40 0x55e842698950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.540+0000 7fc36e171740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e842660690 con 0x55e8426c4280 2024-05-08T05:49:36.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.540+0000 7fc36e171740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e8425fdf30 con 0x55e8426c3b40 2024-05-08T05:49:36.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.540+0000 7fc36e171740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e8425b6b20 con 0x55e8426e1160 2024-05-08T05:49:36.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.540+0000 7fc36891a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426c4280 0x55e8426c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.540+0000 7fc36891a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426c4280 0x55e8426c3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37858/0 (socket says 172.21.15.45:37858) 2024-05-08T05:49:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.541+0000 7fc36891a640 1 -- 172.21.15.45:0/1096397272 learned_addr learned my addr 172.21.15.45:0/1096397272 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.541+0000 7fc36911b640 1 --2- 172.21.15.45:0/1096397272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e8426e1160 0x55e842693af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.541+0000 7fc36891a640 1 -- 172.21.15.45:0/1096397272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e8426e1160 msgr2=0x55e842693af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.541+0000 7fc36891a640 1 --2- 172.21.15.45:0/1096397272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e8426e1160 0x55e842693af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.541+0000 7fc36991c640 1 --2- 172.21.15.45:0/1096397272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8426c3b40 0x55e842698950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.541+0000 7fc36891a640 1 -- 172.21.15.45:0/1096397272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8426c3b40 msgr2=0x55e842698950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.541+0000 7fc36891a640 1 --2- 172.21.15.45:0/1096397272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8426c3b40 0x55e842698950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.541+0000 7fc36891a640 1 -- 172.21.15.45:0/1096397272 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8425aee60 con 0x55e8426c4280 2024-05-08T05:49:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.541+0000 7fc36891a640 1 --2- 172.21.15.45:0/1096397272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426c4280 0x55e8426c3600 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fc358009cf0 tx=0x7fc358014780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a72e1a8eeeb146 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.541+0000 7fc34affd640 1 -- 172.21.15.45:0/1096397272 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358015420 con 0x55e8426c4280 2024-05-08T05:49:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.541+0000 7fc34affd640 1 -- 172.21.15.45:0/1096397272 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc358019070 con 0x55e8426c4280 2024-05-08T05:49:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.541+0000 7fc34affd640 1 -- 172.21.15.45:0/1096397272 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358015800 con 0x55e8426c4280 2024-05-08T05:49:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.542+0000 7fc36e171740 1 -- 172.21.15.45:0/1096397272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426c4280 msgr2=0x55e8426c3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.542+0000 7fc36e171740 1 --2- 172.21.15.45:0/1096397272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426c4280 0x55e8426c3600 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fc358009cf0 tx=0x7fc358014780 comp rx=0 tx=0).stop 2024-05-08T05:49:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.542+0000 7fc36e171740 1 -- 172.21.15.45:0/1096397272 shutdown_connections 2024-05-08T05:49:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.542+0000 7fc36e171740 1 --2- 172.21.15.45:0/1096397272 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8426c3b40 0x55e842698950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.542+0000 7fc36e171740 1 --2- 172.21.15.45:0/1096397272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426c4280 0x55e8426c3600 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.542+0000 7fc36e171740 1 --2- 172.21.15.45:0/1096397272 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e8426e1160 0x55e842693af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.542+0000 7fc36e171740 1 -- 172.21.15.45:0/1096397272 >> 172.21.15.45:0/1096397272 conn(0x55e842695d60 msgr2=0x55e8426de600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.542+0000 7fc36e171740 1 -- 172.21.15.45:0/1096397272 shutdown_connections 2024-05-08T05:49:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.542+0000 7fc36e171740 1 -- 172.21.15.45:0/1096397272 wait complete. 2024-05-08T05:49:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.542+0000 7fc36e171740 1 -- start start 2024-05-08T05:49:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.542+0000 7fc36e171740 1 Processor -- start 2024-05-08T05:49:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.543+0000 7fc36e171740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8426c4280 0x55e8426b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.543+0000 7fc36e171740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426e1160 0x55e8426dd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.543+0000 7fc36e171740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e8426dc1b0 0x55e8426d5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.543+0000 7fc36e171740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e8425ba5e0 con 0x55e8426e1160 2024-05-08T05:49:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.543+0000 7fc36e171740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e8425f9ef0 con 0x55e8426c4280 2024-05-08T05:49:36.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.543+0000 7fc36e171740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e842697f10 con 0x55e8426dc1b0 2024-05-08T05:49:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.543+0000 7fc36e171740 10 client.0 _ll_register_callbacks cb 0x55e8425b5320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:49:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.543+0000 7fc36891a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426e1160 0x55e8426dd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.543+0000 7fc36891a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426e1160 0x55e8426dd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37866/0 (socket says 172.21.15.45:37866) 2024-05-08T05:49:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.543+0000 7fc36891a640 1 -- 172.21.15.45:0/1399422621 learned_addr learned my addr 172.21.15.45:0/1399422621 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.544+0000 7fc36911b640 1 --2- 172.21.15.45:0/1399422621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8426c4280 0x55e8426b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.544+0000 7fc36991c640 1 --2- 172.21.15.45:0/1399422621 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e8426dc1b0 0x55e8426d5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.544+0000 7fc36891a640 1 -- 172.21.15.45:0/1399422621 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e8426dc1b0 msgr2=0x55e8426d5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.544+0000 7fc36891a640 1 --2- 172.21.15.45:0/1399422621 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e8426dc1b0 0x55e8426d5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.544+0000 7fc36891a640 1 -- 172.21.15.45:0/1399422621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8426c4280 msgr2=0x55e8426b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.544+0000 7fc36891a640 1 --2- 172.21.15.45:0/1399422621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8426c4280 0x55e8426b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.544+0000 7fc36891a640 1 -- 172.21.15.45:0/1399422621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8423d97d0 con 0x55e8426e1160 2024-05-08T05:49:36.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.544+0000 7fc36891a640 1 --2- 172.21.15.45:0/1399422621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426e1160 0x55e8426dd9c0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fc35801f040 tx=0x7fc3580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526d02cb9bf24c78 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.544+0000 7fc343fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:49:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.544+0000 7fc343fff640 1 -- 172.21.15.45:0/1399422621 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35802c020 con 0x55e8426e1160 2024-05-08T05:49:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.544+0000 7fc343fff640 1 -- 172.21.15.45:0/1399422621 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fc3580066c0 con 0x55e8426e1160 2024-05-08T05:49:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.544+0000 7fc36e171740 1 -- 172.21.15.45:0/1399422621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc358019070 con 0x55e8426e1160 2024-05-08T05:49:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.544+0000 7fc343fff640 1 -- 172.21.15.45:0/1399422621 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fc358036050 con 0x55e8426e1160 2024-05-08T05:49:36.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.544+0000 7fc36e171740 10 client.7770 fetch_fsmap learned FSMap version 83 2024-05-08T05:49:36.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.544+0000 7fc36e171740 1 -- 172.21.15.45:0/1399422621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x55e84230f3f0 con 0x55e8426e1160 2024-05-08T05:49:36.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.544+0000 7fc343fff640 1 -- 172.21.15.45:0/1399422621 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc358019070 con 0x55e8426e1160 2024-05-08T05:49:36.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.544+0000 7fc343fff640 1 -- 172.21.15.45:0/1399422621 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7fc358039020 con 0x55e8426e1160 2024-05-08T05:49:36.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.544+0000 7fc36e171740 10 client.7770 fetch_fsmap finished waiting for FSMap version 83 2024-05-08T05:49:36.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.545+0000 7fc36e171740 10 client.7770 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-08T05:49:36.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.545+0000 7fc36e171740 20 client.7770 populate_metadata read hostname 'smithi045' 2024-05-08T05:49:36.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.545+0000 7fc36e171740 1 --2- 172.21.15.45:0/1399422621 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55e8426d62e0 0x55e842698310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.545+0000 7fc36e171740 4 client.7770 mds_command: new command op to 7524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:49:36.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.545+0000 7fc36e171740 1 -- 172.21.15.45:0/1399422621 --> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e84260bb40 con 0x55e8426d62e0 2024-05-08T05:49:36.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.545+0000 7fc36911b640 1 --2- 172.21.15.45:0/1399422621 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55e8426d62e0 0x55e842698310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.545+0000 7fc343fff640 10 client.7770 ms_handle_connect on v2:172.21.15.45:6834/878092981 2024-05-08T05:49:36.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.545+0000 7fc36911b640 1 --2- 172.21.15.45:0/1399422621 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55e8426d62e0 0x55e842698310 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6c48296c3a481d9 server_cookie=bea3a24ed75e2893 in_seq=0 out_seq=0 2024-05-08T05:49:36.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.546+0000 7fc343fff640 1 -- 172.21.15.45:0/1399422621 <== mds.0 v2:172.21.15.45:6834/878092981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e84260bb40 con 0x55e8426d62e0 2024-05-08T05:49:36.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.546+0000 7fc343fff640 10 client.7770 handle_command_reply: tid=0 2024-05-08T05:49:36.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.546+0000 7fc36e171740 1 client.7770 shutdown 2024-05-08T05:49:36.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.546+0000 7fc36e171740 1 -- 172.21.15.45:0/1399422621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426e1160 msgr2=0x55e8426dd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.546+0000 7fc36e171740 1 --2- 172.21.15.45:0/1399422621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426e1160 0x55e8426dd9c0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fc35801f040 tx=0x7fc3580046c0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.546+0000 7fc36e171740 1 -- 172.21.15.45:0/1399422621 shutdown_connections 2024-05-08T05:49:36.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.546+0000 7fc36e171740 1 --2- 172.21.15.45:0/1399422621 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55e8426d62e0 0x55e842698310 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.546+0000 7fc36e171740 1 --2- 172.21.15.45:0/1399422621 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e8426dc1b0 0x55e8426d5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.546+0000 7fc36e171740 1 --2- 172.21.15.45:0/1399422621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426e1160 0x55e8426dd9c0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.546+0000 7fc343fff640 0 client.7770 ms_handle_reset on v2:172.21.15.45:6834/878092981 2024-05-08T05:49:36.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.546+0000 7fc36e171740 1 --2- 172.21.15.45:0/1399422621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8426c4280 0x55e8426b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.546+0000 7fc36e171740 1 -- 172.21.15.45:0/1399422621 >> 172.21.15.45:0/1399422621 conn(0x55e8426c3b40 msgr2=0x55e842693230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:36.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.546+0000 7fc36e171740 1 -- 172.21.15.45:0/1399422621 shutdown_connections 2024-05-08T05:49:36.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.546+0000 7fc36e171740 1 -- 172.21.15.45:0/1399422621 wait complete. 2024-05-08T05:49:36.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.546+0000 7fc36e171740 20 client.7770 trim_cache size 0 max 16384 2024-05-08T05:49:36.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.564+0000 7fc36e171740 1 Processor -- start 2024-05-08T05:49:36.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.564+0000 7fc36e171740 1 -- start start 2024-05-08T05:49:36.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.565+0000 7fc36e171740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e8426c4280 0x55e842699700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.565+0000 7fc36e171740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8426dc1b0 0x55e8426bdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.565+0000 7fc36e171740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426e1160 0x55e8426be2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.565+0000 7fc36e171740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc3642029a0 con 0x55e8426e1160 2024-05-08T05:49:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.565+0000 7fc36e171740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc3642019e0 con 0x55e8426dc1b0 2024-05-08T05:49:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.565+0000 7fc36e171740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc364200d40 con 0x55e8426c4280 2024-05-08T05:49:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.565+0000 7fc36991c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426e1160 0x55e8426be2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.565+0000 7fc36991c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426e1160 0x55e8426be2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37878/0 (socket says 172.21.15.45:37878) 2024-05-08T05:49:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.565+0000 7fc36991c640 1 -- 172.21.15.45:0/2499596668 learned_addr learned my addr 172.21.15.45:0/2499596668 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.565+0000 7fc36991c640 1 -- 172.21.15.45:0/2499596668 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e8426c4280 msgr2=0x55e842699700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.565+0000 7fc36991c640 1 --2- 172.21.15.45:0/2499596668 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e8426c4280 0x55e842699700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.565+0000 7fc36991c640 1 -- 172.21.15.45:0/2499596668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8426dc1b0 msgr2=0x55e8426bdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.565+0000 7fc36991c640 1 --2- 172.21.15.45:0/2499596668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8426dc1b0 0x55e8426bdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.565+0000 7fc36991c640 1 -- 172.21.15.45:0/2499596668 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8422cf0f0 con 0x55e8426e1160 2024-05-08T05:49:36.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.565+0000 7fc36991c640 1 --2- 172.21.15.45:0/2499596668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426e1160 0x55e8426be2e0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fc35407c560 tx=0x7fc354007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f980fc4f21f1ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:36.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.565+0000 7fc34a7fc640 1 -- 172.21.15.45:0/2499596668 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354080670 con 0x55e8426e1160 2024-05-08T05:49:36.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.565+0000 7fc34a7fc640 1 -- 172.21.15.45:0/2499596668 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc35407ff70 con 0x55e8426e1160 2024-05-08T05:49:36.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.565+0000 7fc34a7fc640 1 -- 172.21.15.45:0/2499596668 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354084320 con 0x55e8426e1160 2024-05-08T05:49:36.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.566+0000 7fc36e171740 1 -- 172.21.15.45:0/2499596668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426e1160 msgr2=0x55e8426be2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.566+0000 7fc36e171740 1 --2- 172.21.15.45:0/2499596668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426e1160 0x55e8426be2e0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fc35407c560 tx=0x7fc354007cc0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.566+0000 7fc36e171740 1 -- 172.21.15.45:0/2499596668 shutdown_connections 2024-05-08T05:49:36.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.566+0000 7fc36e171740 1 --2- 172.21.15.45:0/2499596668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426e1160 0x55e8426be2e0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.566+0000 7fc36e171740 1 --2- 172.21.15.45:0/2499596668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8426dc1b0 0x55e8426bdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.566+0000 7fc36e171740 1 --2- 172.21.15.45:0/2499596668 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e8426c4280 0x55e842699700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.566+0000 7fc36e171740 1 -- 172.21.15.45:0/2499596668 >> 172.21.15.45:0/2499596668 conn(0x55e8426c3b40 msgr2=0x55e8426da8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.566+0000 7fc36e171740 1 -- 172.21.15.45:0/2499596668 shutdown_connections 2024-05-08T05:49:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.566+0000 7fc36e171740 1 -- 172.21.15.45:0/2499596668 wait complete. 2024-05-08T05:49:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.567+0000 7fc36e171740 1 -- start start 2024-05-08T05:49:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.567+0000 7fc36e171740 1 Processor -- start 2024-05-08T05:49:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.567+0000 7fc36e171740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426c4280 0x55e8426bdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc36e171740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e8426dc1b0 0x55e8426be2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc36e171740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8426e1160 0x55e8426bf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc36e171740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc364201390 con 0x55e8426c4280 2024-05-08T05:49:36.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc36e171740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e84260bb40 con 0x55e8426e1160 2024-05-08T05:49:36.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc36e171740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e84275d170 con 0x55e8426dc1b0 2024-05-08T05:49:36.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc36e171740 10 client.0 _ll_register_callbacks cb 0x7fc358009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:49:36.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc36911b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426c4280 0x55e8426bdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc36911b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426c4280 0x55e8426bdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37884/0 (socket says 172.21.15.45:37884) 2024-05-08T05:49:36.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc36911b640 1 -- 172.21.15.45:0/746662184 learned_addr learned my addr 172.21.15.45:0/746662184 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:49:36.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc36991c640 1 --2- 172.21.15.45:0/746662184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8426e1160 0x55e8426bf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc36891a640 1 --2- 172.21.15.45:0/746662184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e8426dc1b0 0x55e8426be2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc36911b640 1 -- 172.21.15.45:0/746662184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e8426dc1b0 msgr2=0x55e8426be2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc36911b640 1 --2- 172.21.15.45:0/746662184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e8426dc1b0 0x55e8426be2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc36911b640 1 -- 172.21.15.45:0/746662184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8426e1160 msgr2=0x55e8426bf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:49:36.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc36911b640 1 --2- 172.21.15.45:0/746662184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8426e1160 0x55e8426bf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc36911b640 1 -- 172.21.15.45:0/746662184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e842702240 con 0x55e8426c4280 2024-05-08T05:49:36.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc36911b640 1 --2- 172.21.15.45:0/746662184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426c4280 0x55e8426bdda0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00af50 tx=0x7fc35c00a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ee3a14a097058f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:36.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc343fff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:49:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc343fff640 1 -- 172.21.15.45:0/746662184 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35c02f060 con 0x55e8426c4280 2024-05-08T05:49:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc343fff640 1 -- 172.21.15.45:0/746662184 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fc35c010040 con 0x55e8426c4280 2024-05-08T05:49:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc36e171740 1 -- 172.21.15.45:0/746662184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc35407ff70 con 0x55e8426c4280 2024-05-08T05:49:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc343fff640 1 -- 172.21.15.45:0/746662184 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fc35c035050 con 0x55e8426c4280 2024-05-08T05:49:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.568+0000 7fc36e171740 10 client.7776 fetch_fsmap learned FSMap version 83 2024-05-08T05:49:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.569+0000 7fc36e171740 1 -- 172.21.15.45:0/746662184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x55e8426c08b0 con 0x55e8426c4280 2024-05-08T05:49:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.569+0000 7fc343fff640 1 -- 172.21.15.45:0/746662184 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc35407ff70 con 0x55e8426c4280 2024-05-08T05:49:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.569+0000 7fc343fff640 1 -- 172.21.15.45:0/746662184 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7fc35c038020 con 0x55e8426c4280 2024-05-08T05:49:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.569+0000 7fc36e171740 10 client.7776 fetch_fsmap finished waiting for FSMap version 83 2024-05-08T05:49:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.569+0000 7fc36e171740 10 client.7776 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-08T05:49:36.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.569+0000 7fc36e171740 20 client.7776 populate_metadata read hostname 'smithi045' 2024-05-08T05:49:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.569+0000 7fc36e171740 1 --2- 172.21.15.45:0/746662184 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55e8426de180 0x55e8426bfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.569+0000 7fc36e171740 4 client.7776 mds_command: new command op to 7524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:49:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.569+0000 7fc36e171740 1 -- 172.21.15.45:0/746662184 --> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e8426059b0 con 0x55e8426de180 2024-05-08T05:49:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.569+0000 7fc36891a640 1 --2- 172.21.15.45:0/746662184 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55e8426de180 0x55e8426bfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.569+0000 7fc343fff640 10 client.7776 ms_handle_connect on v2:172.21.15.45:6834/878092981 2024-05-08T05:49:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.569+0000 7fc36891a640 1 --2- 172.21.15.45:0/746662184 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55e8426de180 0x55e8426bfef0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b77fb72f65b3821 server_cookie=325a57b2c900b6f in_seq=0 out_seq=0 2024-05-08T05:49:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.569+0000 7fc343fff640 1 -- 172.21.15.45:0/746662184 <== mds.0 v2:172.21.15.45:6834/878092981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55e8426059b0 con 0x55e8426de180 2024-05-08T05:49:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.569+0000 7fc343fff640 10 client.7776 handle_command_reply: tid=0 2024-05-08T05:49:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.569+0000 7fc36e171740 1 client.7776 shutdown 2024-05-08T05:49:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36e171740 1 -- 172.21.15.45:0/746662184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426c4280 msgr2=0x55e8426bdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36e171740 1 --2- 172.21.15.45:0/746662184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426c4280 0x55e8426bdda0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00af50 tx=0x7fc35c00a1e0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36e171740 1 -- 172.21.15.45:0/746662184 shutdown_connections 2024-05-08T05:49:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36e171740 1 --2- 172.21.15.45:0/746662184 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55e8426de180 0x55e8426bfef0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36e171740 1 --2- 172.21.15.45:0/746662184 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e8426dc1b0 0x55e8426be2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36e171740 1 --2- 172.21.15.45:0/746662184 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e8426e1160 0x55e8426bf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36e171740 1 --2- 172.21.15.45:0/746662184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e8426c4280 0x55e8426bdda0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36e171740 1 -- 172.21.15.45:0/746662184 >> 172.21.15.45:0/746662184 conn(0x55e8426c3b40 msgr2=0x55e842693d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:36.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc343fff640 0 client.7776 ms_handle_reset on v2:172.21.15.45:6834/878092981 2024-05-08T05:49:36.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36e171740 1 -- 172.21.15.45:0/746662184 shutdown_connections 2024-05-08T05:49:36.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36e171740 1 -- 172.21.15.45:0/746662184 wait complete. 2024-05-08T05:49:36.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36e171740 20 client.7776 trim_cache size 0 max 16384 2024-05-08T05:49:36.585 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:49:36.585 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T05:49:36.585 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-05-08T05:49:36.586 INFO:teuthology.orchestra.run.smithi045.stdout: "f3388a8c-bd7b-41d7-ba2e-af000ca2e492": { 2024-05-08T05:49:36.586 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-05-08T05:49:36.586 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "f3388a8c-bd7b-41d7-ba2e-af000ca2e492", 2024-05-08T05:49:36.586 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,repair" 2024-05-08T05:49:36.586 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:36.586 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:49:36.586 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:49:36.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36b37f640 1 -- 172.21.15.45:0/3185318005 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc350076300 msgr2=0x7fc3500787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36b37f640 1 --2- 172.21.15.45:0/3185318005 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc350076300 0x7fc3500787c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc35c001d60 tx=0x7fc35c006040 comp rx=0 tx=0).stop 2024-05-08T05:49:36.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36b37f640 1 -- 172.21.15.45:0/3185318005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3640cad10 msgr2=0x7fc3640b9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:36.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36b37f640 1 --2- 172.21.15.45:0/3185318005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3640cad10 0x7fc3640b9bf0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fc3540018b0 tx=0x7fc354001120 comp rx=0 tx=0).stop 2024-05-08T05:49:36.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36b37f640 1 -- 172.21.15.45:0/3185318005 shutdown_connections 2024-05-08T05:49:36.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36b37f640 1 --2- 172.21.15.45:0/3185318005 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc350076300 0x7fc3500787c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36b37f640 1 --2- 172.21.15.45:0/3185318005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3640cad10 0x7fc3640b9bf0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36b37f640 1 --2- 172.21.15.45:0/3185318005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3640c9530 0x7fc3640b96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36b37f640 1 --2- 172.21.15.45:0/3185318005 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc34c004d40 0x7fc3641f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:36.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.570+0000 7fc36b37f640 1 -- 172.21.15.45:0/3185318005 >> 172.21.15.45:0/3185318005 conn(0x7fc36408eac0 msgr2=0x7fc3640b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:36.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.571+0000 7fc36b37f640 1 -- 172.21.15.45:0/3185318005 shutdown_connections 2024-05-08T05:49:36.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:36.571+0000 7fc36b37f640 1 -- 172.21.15.45:0/3185318005 wait complete. 2024-05-08T05:49:36.589 INFO:tasks.cephfs.filesystem:scrub status for tag:f3388a8c-bd7b-41d7-ba2e-af000ca2e492 - {'path': '/', 'tag': 'f3388a8c-bd7b-41d7-ba2e-af000ca2e492', 'options': 'recursive,repair'} 2024-05-08T05:49:37.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:37.273+0000 7f9f137fe640 1 -- 172.21.15.45:0/1723978194 <== mon.0 v2:172.21.15.45:3300/0 7 ==== log(4 entries from seq 1 at 2024-05-08T05:49:36.410256+0000) ==== 695+0+0 (secure 0 0 0) 0x7f9f14085e00 con 0x7f9f1c15cfa0 2024-05-08T05:49:37.274 INFO:teuthology.orchestra.run.smithi045.stdout: cluster: 2024-05-08T05:49:37.274 INFO:teuthology.orchestra.run.smithi045.stdout: id: e9e1170a-fd27-4625-97a9-5d5f104d13a2 2024-05-08T05:49:37.275 INFO:teuthology.orchestra.run.smithi045.stdout: health: HEALTH_OK 2024-05-08T05:49:37.275 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:37.275 INFO:teuthology.orchestra.run.smithi045.stdout: services: 2024-05-08T05:49:37.275 INFO:teuthology.orchestra.run.smithi045.stdout: mon: 3 daemons, quorum a,b,c (age 8m) 2024-05-08T05:49:37.275 INFO:teuthology.orchestra.run.smithi045.stdout: mgr: x(active, since 8m), standbys: y 2024-05-08T05:49:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: mds: 1/1 daemons up, 3 standby 2024-05-08T05:49:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: osd: 8 osds: 8 up (since 8m), 8 in (since 8m) 2024-05-08T05:49:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: data: 2024-05-08T05:49:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: volumes: 1/1 healthy 2024-05-08T05:49:37.276 INFO:teuthology.orchestra.run.smithi045.stdout: pools: 3 pools, 97 pgs 2024-05-08T05:49:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: objects: 27 objects, 11 MiB 2024-05-08T05:49:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: usage: 269 MiB used, 720 GiB / 720 GiB avail 2024-05-08T05:49:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: pgs: 97 active+clean 2024-05-08T05:49:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: io: 2024-05-08T05:49:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: client: 85 B/s rd, 857 KiB/s wr, 0 op/s rd, 6 op/s wr 2024-05-08T05:49:37.277 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:37.278 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:49:37.278 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:49:36.410256+0000 mds.a [INF] scrub queued for path: / 2024-05-08T05:49:37.278 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:49:36.410260+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-05-08T05:49:37.278 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:49:36.410265+0000 mds.a [INF] scrub summary: active paths [/] 2024-05-08T05:49:37.278 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:49:36.411318+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-05-08T05:49:39.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:49:39.281+0000 7f9f137fe640 1 -- 172.21.15.45:0/1723978194 <== mon.0 v2:172.21.15.45:3300/0 8 ==== log(1 entries from seq 5 at 2024-05-08T05:49:39.196961+0000) ==== 189+0+0 (secure 0 0 0) 0x7f9f140b8000 con 0x7f9f1c15cfa0 2024-05-08T05:49:39.282 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:49:39.196961+0000 mds.a [INF] scrub summary: idle 2024-05-08T05:50:00.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:00.311+0000 7f9f137fe640 1 -- 172.21.15.45:0/1723978194 <== mon.0 v2:172.21.15.45:3300/0 9 ==== log(1 entries from seq 912 at 2024-05-08T05:50:00.000185+0000) ==== 187+0+0 (secure 0 0 0) 0x7f9f140b5d20 con 0x7f9f1c15cfa0 2024-05-08T05:50:00.312 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:50:00.000185+0000 mon.a [INF] overall HEALTH_OK 2024-05-08T05:50:05.173 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:50:05.177 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:50:06.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:50:06.591 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-08T05:50:06.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.709+0000 7f9a0916a640 1 Processor -- start 2024-05-08T05:50:06.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.709+0000 7f9a0916a640 1 -- start start 2024-05-08T05:50:06.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.710+0000 7f9a0916a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9a0406e0f0 0x7f9a0415c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.710+0000 7f9a0916a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9a04072840 0x7f9a040d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.710+0000 7f9a0916a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9a0406e4d0 0x7f9a040b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.710+0000 7f9a0916a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f9a040d14b0 con 0x7f9a0406e0f0 2024-05-08T05:50:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.710+0000 7f9a0916a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9a0415c920 con 0x7f9a04072840 2024-05-08T05:50:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.710+0000 7f9a0916a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9a0415cc00 con 0x7f9a0406e4d0 2024-05-08T05:50:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.710+0000 7f9a02d9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9a0406e0f0 0x7f9a0415c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.710+0000 7f9a02d9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9a0406e0f0 0x7f9a0415c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50404/0 (socket says 172.21.15.45:50404) 2024-05-08T05:50:06.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.710+0000 7f9a02d9d640 1 -- 172.21.15.45:0/535749607 learned_addr learned my addr 172.21.15.45:0/535749607 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:06.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.711+0000 7f9a0259c640 1 --2- 172.21.15.45:0/535749607 >> v2:172.21.15.63:3300/0 conn(0x7f9a04072840 0x7f9a040d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.711+0000 7f9a0359e640 1 --2- 172.21.15.45:0/535749607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9a0406e4d0 0x7f9a040b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.711+0000 7f9a02d9d640 1 -- 172.21.15.45:0/535749607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9a0406e4d0 msgr2=0x7f9a040b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:06.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.711+0000 7f9a02d9d640 1 --2- 172.21.15.45:0/535749607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9a0406e4d0 0x7f9a040b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.711+0000 7f9a02d9d640 1 -- 172.21.15.45:0/535749607 >> v2:172.21.15.63:3300/0 conn(0x7f9a04072840 msgr2=0x7f9a040d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:06.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.711+0000 7f9a02d9d640 1 --2- 172.21.15.45:0/535749607 >> v2:172.21.15.63:3300/0 conn(0x7f9a04072840 0x7f9a040d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.711+0000 7f9a02d9d640 1 -- 172.21.15.45:0/535749607 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0415cee0 con 0x7f9a0406e0f0 2024-05-08T05:50:06.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.711+0000 7f9a0359e640 1 --2- 172.21.15.45:0/535749607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9a0406e4d0 0x7f9a040b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:06.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.711+0000 7f9a02d9d640 1 --2- 172.21.15.45:0/535749607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a0406e0f0 0x7f9a0415c340 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f9a040727d0 tx=0x7f99f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80fd47a4e51eb7f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:06.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.713+0000 7f9a01d9b640 1 -- 172.21.15.45:0/535749607 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4037070 con 0x7f9a0406e0f0 2024-05-08T05:50:06.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.713+0000 7f9a01d9b640 1 -- 172.21.15.45:0/535749607 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f4032440 con 0x7f9a0406e0f0 2024-05-08T05:50:06.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.713+0000 7f9a01d9b640 1 -- 172.21.15.45:0/535749607 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4032810 con 0x7f9a0406e0f0 2024-05-08T05:50:06.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.713+0000 7f9a0916a640 1 -- 172.21.15.45:0/535749607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a0406e0f0 msgr2=0x7f9a0415c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:06.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.713+0000 7f9a0916a640 1 --2- 172.21.15.45:0/535749607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a0406e0f0 0x7f9a0415c340 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f9a040727d0 tx=0x7f99f402a1e0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.713+0000 7f9a0916a640 1 -- 172.21.15.45:0/535749607 shutdown_connections 2024-05-08T05:50:06.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.713+0000 7f9a0916a640 1 --2- 172.21.15.45:0/535749607 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9a0406e4d0 0x7f9a040b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.713+0000 7f9a0916a640 1 --2- 172.21.15.45:0/535749607 >> v2:172.21.15.63:3300/0 conn(0x7f9a04072840 0x7f9a040d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.713+0000 7f9a0916a640 1 --2- 172.21.15.45:0/535749607 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a0406e0f0 0x7f9a0415c340 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.713+0000 7f9a0916a640 1 -- 172.21.15.45:0/535749607 >> 172.21.15.45:0/535749607 conn(0x7f9a0406c3a0 msgr2=0x7f9a0406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:06.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.713+0000 7f9a0916a640 1 -- 172.21.15.45:0/535749607 shutdown_connections 2024-05-08T05:50:06.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.713+0000 7f9a0916a640 1 -- 172.21.15.45:0/535749607 wait complete. 2024-05-08T05:50:06.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.715+0000 7f9a0916a640 1 Processor -- start 2024-05-08T05:50:06.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.715+0000 7f9a0916a640 1 -- start start 2024-05-08T05:50:06.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f9a0916a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9a0406e0f0 0x7f9a04151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f9a0916a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a0406e4d0 0x7f9a04154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f9a0916a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a04072840 0x7f9a040b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f9a0916a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9a040ae650 con 0x7f9a04072840 2024-05-08T05:50:06.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f9a0916a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0406e980 con 0x7f9a0406e4d0 2024-05-08T05:50:06.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f9a0916a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9a040ae7d0 con 0x7f9a0406e0f0 2024-05-08T05:50:06.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f9a0359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a04072840 0x7f9a040b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f9a0359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a04072840 0x7f9a040b27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50414/0 (socket says 172.21.15.45:50414) 2024-05-08T05:50:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f9a0359e640 1 -- 172.21.15.45:0/1936787131 learned_addr learned my addr 172.21.15.45:0/1936787131 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f9a0259c640 1 --2- 172.21.15.45:0/1936787131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a0406e4d0 0x7f9a04154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f9a02d9d640 1 --2- 172.21.15.45:0/1936787131 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9a0406e0f0 0x7f9a04151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f9a0359e640 1 -- 172.21.15.45:0/1936787131 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9a0406e0f0 msgr2=0x7f9a04151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f9a0359e640 1 --2- 172.21.15.45:0/1936787131 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9a0406e0f0 0x7f9a04151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f9a0359e640 1 -- 172.21.15.45:0/1936787131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a0406e4d0 msgr2=0x7f9a04154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f9a0359e640 1 --2- 172.21.15.45:0/1936787131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a0406e4d0 0x7f9a04154e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f9a0359e640 1 -- 172.21.15.45:0/1936787131 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a040cdb00 con 0x7f9a04072840 2024-05-08T05:50:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f9a0359e640 1 --2- 172.21.15.45:0/1936787131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a04072840 0x7f9a040b27f0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00cf20 tx=0x7f99fc00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f99eb7fe640 1 -- 172.21.15.45:0/1936787131 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc01a070 con 0x7f9a04072840 2024-05-08T05:50:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f99eb7fe640 1 -- 172.21.15.45:0/1936787131 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99fc00d840 con 0x7f9a04072840 2024-05-08T05:50:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f9a0916a640 1 -- 172.21.15.45:0/1936787131 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a040cdde0 con 0x7f9a04072840 2024-05-08T05:50:06.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f99eb7fe640 1 -- 172.21.15.45:0/1936787131 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc00dc30 con 0x7f9a04072840 2024-05-08T05:50:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.716+0000 7f9a0916a640 1 -- 172.21.15.45:0/1936787131 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a040ce3a0 con 0x7f9a04072840 2024-05-08T05:50:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.718+0000 7f99eb7fe640 1 -- 172.21.15.45:0/1936787131 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f99fc01d070 con 0x7f9a04072840 2024-05-08T05:50:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.718+0000 7f99eb7fe640 1 --2- 172.21.15.45:0/1936787131 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99f00762d0 0x7f99f0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.718+0000 7f99eb7fe640 1 -- 172.21.15.45:0/1936787131 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f99fc097fc0 con 0x7f9a04072840 2024-05-08T05:50:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.718+0000 7f9a02d9d640 1 --2- 172.21.15.45:0/1936787131 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99f00762d0 0x7f99f0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.719+0000 7f9a02d9d640 1 --2- 172.21.15.45:0/1936787131 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99f00762d0 0x7f99f0078790 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f99f4034000 tx=0x7f99f402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.719+0000 7f9a0be83740 1 Processor -- start 2024-05-08T05:50:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.719+0000 7f9a0be83740 1 -- start start 2024-05-08T05:50:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0be83740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65de170 0x55c3c65b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0be83740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65de550 0x55c3c65e0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0be83740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65e0f50 0x55c3c65b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0be83740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c3c6579690 con 0x55c3c65de550 2024-05-08T05:50:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0be83740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c3c6516f30 con 0x55c3c65e0f50 2024-05-08T05:50:06.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0be83740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55c3c64cfb20 con 0x55c3c65de170 2024-05-08T05:50:06.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65de550 0x55c3c65e0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65de550 0x55c3c65e0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50430/0 (socket says 172.21.15.45:50430) 2024-05-08T05:50:06.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0259c640 1 -- 172.21.15.45:0/602136192 learned_addr learned my addr 172.21.15.45:0/602136192 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:06.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a02d9d640 1 --2- 172.21.15.45:0/602136192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65de170 0x55c3c65b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0359e640 1 --2- 172.21.15.45:0/602136192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65e0f50 0x55c3c65b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0259c640 1 -- 172.21.15.45:0/602136192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65de170 msgr2=0x55c3c65b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:06.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0259c640 1 --2- 172.21.15.45:0/602136192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65de170 0x55c3c65b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0259c640 1 -- 172.21.15.45:0/602136192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65e0f50 msgr2=0x55c3c65b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:06.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0259c640 1 --2- 172.21.15.45:0/602136192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65e0f50 0x55c3c65b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0259c640 1 -- 172.21.15.45:0/602136192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3c64c7e60 con 0x55c3c65de550 2024-05-08T05:50:06.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0259c640 1 --2- 172.21.15.45:0/602136192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65de550 0x55c3c65e0a10 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f99f8007850 tx=0x7f99f8007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcb9b082130eeb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:06.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f99e8ff9640 1 -- 172.21.15.45:0/602136192 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8018020 con 0x55c3c65de550 2024-05-08T05:50:06.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f99e8ff9640 1 -- 172.21.15.45:0/602136192 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f8004300 con 0x55c3c65de550 2024-05-08T05:50:06.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f99e8ff9640 1 -- 172.21.15.45:0/602136192 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8010a30 con 0x55c3c65de550 2024-05-08T05:50:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0be83740 1 -- 172.21.15.45:0/602136192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65de550 msgr2=0x55c3c65e0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0be83740 1 --2- 172.21.15.45:0/602136192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65de550 0x55c3c65e0a10 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f99f8007850 tx=0x7f99f8007d10 comp rx=0 tx=0).stop 2024-05-08T05:50:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0be83740 1 -- 172.21.15.45:0/602136192 shutdown_connections 2024-05-08T05:50:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0be83740 1 --2- 172.21.15.45:0/602136192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65e0f50 0x55c3c65b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0be83740 1 --2- 172.21.15.45:0/602136192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65de550 0x55c3c65e0a10 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0be83740 1 --2- 172.21.15.45:0/602136192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65de170 0x55c3c65b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0be83740 1 -- 172.21.15.45:0/602136192 >> 172.21.15.45:0/602136192 conn(0x55c3c65b3170 msgr2=0x55c3c65fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0be83740 1 -- 172.21.15.45:0/602136192 shutdown_connections 2024-05-08T05:50:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.721+0000 7f9a0be83740 1 -- 172.21.15.45:0/602136192 wait complete. 2024-05-08T05:50:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.722+0000 7f9a0be83740 1 -- start start 2024-05-08T05:50:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.722+0000 7f9a0be83740 1 Processor -- start 2024-05-08T05:50:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.723+0000 7f9a0be83740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65de550 0x55c3c65e4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.723+0000 7f9a0be83740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65e0f50 0x55c3c65f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.723+0000 7f9a0be83740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65f8e80 0x55c3c65fa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.723+0000 7f9a0be83740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c3c64d35e0 con 0x55c3c65e0f50 2024-05-08T05:50:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.723+0000 7f9a0be83740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55c3c6512ef0 con 0x55c3c65f8e80 2024-05-08T05:50:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.723+0000 7f9a0be83740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55c3c65b5810 con 0x55c3c65de550 2024-05-08T05:50:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.723+0000 7f9a0be83740 10 client.0 _ll_register_callbacks cb 0x55c3c64ce320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:50:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.723+0000 7f9a0259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65e0f50 0x55c3c65f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.723+0000 7f9a0259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65e0f50 0x55c3c65f7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50446/0 (socket says 172.21.15.45:50446) 2024-05-08T05:50:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.723+0000 7f9a0259c640 1 -- 172.21.15.45:0/3400376676 learned_addr learned my addr 172.21.15.45:0/3400376676 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.723+0000 7f9a0359e640 1 --2- 172.21.15.45:0/3400376676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65f8e80 0x55c3c65fa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.723+0000 7f9a02d9d640 1 --2- 172.21.15.45:0/3400376676 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65de550 0x55c3c65e4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.723+0000 7f9a0259c640 1 -- 172.21.15.45:0/3400376676 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65de550 msgr2=0x55c3c65e4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.723+0000 7f9a0259c640 1 --2- 172.21.15.45:0/3400376676 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65de550 0x55c3c65e4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.723+0000 7f9a0259c640 1 -- 172.21.15.45:0/3400376676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65f8e80 msgr2=0x55c3c65fa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.723+0000 7f9a0259c640 1 --2- 172.21.15.45:0/3400376676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65f8e80 0x55c3c65fa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.723+0000 7f9a0259c640 1 -- 172.21.15.45:0/3400376676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3c62f27d0 con 0x55c3c65e0f50 2024-05-08T05:50:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.723+0000 7f9a0259c640 1 --2- 172.21.15.45:0/3400376676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65e0f50 0x55c3c65f7ad0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f99f8016000 tx=0x7f99f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33482aea40d31f30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.723+0000 7f99e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:50:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.724+0000 7f99e1ffb640 1 -- 172.21.15.45:0/3400376676 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8018040 con 0x55c3c65e0f50 2024-05-08T05:50:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.724+0000 7f99e1ffb640 1 -- 172.21.15.45:0/3400376676 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f99f8003610 con 0x55c3c65e0f50 2024-05-08T05:50:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.724+0000 7f9a0be83740 1 -- 172.21.15.45:0/3400376676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99f8004300 con 0x55c3c65e0f50 2024-05-08T05:50:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.724+0000 7f99e1ffb640 1 -- 172.21.15.45:0/3400376676 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f99f802f050 con 0x55c3c65e0f50 2024-05-08T05:50:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.724+0000 7f9a0be83740 10 client.7788 fetch_fsmap learned FSMap version 83 2024-05-08T05:50:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.724+0000 7f9a0be83740 1 -- 172.21.15.45:0/3400376676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x55c3c62283f0 con 0x55c3c65e0f50 2024-05-08T05:50:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.724+0000 7f99e1ffb640 1 -- 172.21.15.45:0/3400376676 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f8004300 con 0x55c3c65e0f50 2024-05-08T05:50:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.724+0000 7f99e1ffb640 1 -- 172.21.15.45:0/3400376676 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7f99f8032020 con 0x55c3c65e0f50 2024-05-08T05:50:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.724+0000 7f9a0be83740 10 client.7788 fetch_fsmap finished waiting for FSMap version 83 2024-05-08T05:50:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.724+0000 7f9a0be83740 10 client.7788 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-08T05:50:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.724+0000 7f9a0be83740 20 client.7788 populate_metadata read hostname 'smithi045' 2024-05-08T05:50:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.724+0000 7f9a0be83740 1 --2- 172.21.15.45:0/3400376676 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55c3c65fadb0 0x55c3c65dc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.724+0000 7f9a0be83740 4 client.7788 mds_command: new command op to 7524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:50:06.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.724+0000 7f9a0be83740 1 -- 172.21.15.45:0/3400376676 --> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3c6524b40 con 0x55c3c65fadb0 2024-05-08T05:50:06.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.724+0000 7f9a02d9d640 1 --2- 172.21.15.45:0/3400376676 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55c3c65fadb0 0x55c3c65dc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.726+0000 7f99e1ffb640 10 client.7788 ms_handle_connect on v2:172.21.15.45:6834/878092981 2024-05-08T05:50:06.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.726+0000 7f9a02d9d640 1 --2- 172.21.15.45:0/3400376676 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55c3c65fadb0 0x55c3c65dc150 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37568f1eca42f806 server_cookie=13c17fddba5de617 in_seq=0 out_seq=0 2024-05-08T05:50:06.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.726+0000 7f99e1ffb640 1 -- 172.21.15.45:0/3400376676 <== mds.0 v2:172.21.15.45:6834/878092981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c3c6524b40 con 0x55c3c65fadb0 2024-05-08T05:50:06.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.726+0000 7f99e1ffb640 10 client.7788 handle_command_reply: tid=0 2024-05-08T05:50:06.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.726+0000 7f9a0be83740 1 client.7788 shutdown 2024-05-08T05:50:06.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.726+0000 7f9a0be83740 1 -- 172.21.15.45:0/3400376676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65e0f50 msgr2=0x55c3c65f7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:06.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.726+0000 7f9a0be83740 1 --2- 172.21.15.45:0/3400376676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65e0f50 0x55c3c65f7ad0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f99f8016000 tx=0x7f99f8004730 comp rx=0 tx=0).stop 2024-05-08T05:50:06.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.726+0000 7f9a0be83740 1 -- 172.21.15.45:0/3400376676 shutdown_connections 2024-05-08T05:50:06.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.726+0000 7f9a0be83740 1 --2- 172.21.15.45:0/3400376676 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55c3c65fadb0 0x55c3c65dc150 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.726+0000 7f9a0be83740 1 --2- 172.21.15.45:0/3400376676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65f8e80 0x55c3c65fa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.726+0000 7f9a0be83740 1 --2- 172.21.15.45:0/3400376676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65e0f50 0x55c3c65f7ad0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.726+0000 7f9a0be83740 1 --2- 172.21.15.45:0/3400376676 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65de550 0x55c3c65e4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.726+0000 7f99e1ffb640 0 client.7788 ms_handle_reset on v2:172.21.15.45:6834/878092981 2024-05-08T05:50:06.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.726+0000 7f9a0be83740 1 -- 172.21.15.45:0/3400376676 >> 172.21.15.45:0/3400376676 conn(0x55c3c65de170 msgr2=0x55c3c65fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:06.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.726+0000 7f9a0be83740 1 -- 172.21.15.45:0/3400376676 shutdown_connections 2024-05-08T05:50:06.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.726+0000 7f9a0be83740 1 -- 172.21.15.45:0/3400376676 wait complete. 2024-05-08T05:50:06.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.726+0000 7f9a0be83740 20 client.7788 trim_cache size 0 max 16384 2024-05-08T05:50:06.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.744+0000 7f9a0be83740 1 Processor -- start 2024-05-08T05:50:06.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.744+0000 7f9a0be83740 1 -- start start 2024-05-08T05:50:06.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.744+0000 7f9a0be83740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65de550 0x55c3c65e26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f9a0be83740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65e0f50 0x55c3c65d6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f9a0be83740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65f8e80 0x55c3c65fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f9a0be83740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9a041fd850 con 0x55c3c65e0f50 2024-05-08T05:50:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f9a0be83740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9a041fc8a0 con 0x55c3c65de550 2024-05-08T05:50:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f9a0be83740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9a04157fa0 con 0x55c3c65f8e80 2024-05-08T05:50:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f9a0259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65e0f50 0x55c3c65d6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f9a0259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65e0f50 0x55c3c65d6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50460/0 (socket says 172.21.15.45:50460) 2024-05-08T05:50:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f9a0259c640 1 -- 172.21.15.45:0/3359134418 learned_addr learned my addr 172.21.15.45:0/3359134418 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f9a02d9d640 1 --2- 172.21.15.45:0/3359134418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65de550 0x55c3c65e26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f9a0359e640 1 --2- 172.21.15.45:0/3359134418 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65f8e80 0x55c3c65fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f9a0259c640 1 -- 172.21.15.45:0/3359134418 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65f8e80 msgr2=0x55c3c65fcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f9a0259c640 1 --2- 172.21.15.45:0/3359134418 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65f8e80 0x55c3c65fcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f9a0259c640 1 -- 172.21.15.45:0/3359134418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65de550 msgr2=0x55c3c65e26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:06.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f9a0259c640 1 --2- 172.21.15.45:0/3359134418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65de550 0x55c3c65e26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f9a0259c640 1 -- 172.21.15.45:0/3359134418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3c61e80f0 con 0x55c3c65e0f50 2024-05-08T05:50:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f9a0359e640 1 --2- 172.21.15.45:0/3359134418 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65f8e80 0x55c3c65fcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f9a02d9d640 1 --2- 172.21.15.45:0/3359134418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65de550 0x55c3c65e26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f9a0259c640 1 --2- 172.21.15.45:0/3359134418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65e0f50 0x55c3c65d6aa0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f99f8007850 tx=0x7f99f8020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ad9b6806b579e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f99e3fff640 1 -- 172.21.15.45:0/3359134418 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8003a20 con 0x55c3c65e0f50 2024-05-08T05:50:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f99e3fff640 1 -- 172.21.15.45:0/3359134418 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f8022070 con 0x55c3c65e0f50 2024-05-08T05:50:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.745+0000 7f99e3fff640 1 -- 172.21.15.45:0/3359134418 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8010b40 con 0x55c3c65e0f50 2024-05-08T05:50:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.746+0000 7f9a0be83740 1 -- 172.21.15.45:0/3359134418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65e0f50 msgr2=0x55c3c65d6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.746+0000 7f9a0be83740 1 --2- 172.21.15.45:0/3359134418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65e0f50 0x55c3c65d6aa0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f99f8007850 tx=0x7f99f8020070 comp rx=0 tx=0).stop 2024-05-08T05:50:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.746+0000 7f9a0be83740 1 -- 172.21.15.45:0/3359134418 shutdown_connections 2024-05-08T05:50:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.746+0000 7f9a0be83740 1 --2- 172.21.15.45:0/3359134418 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65f8e80 0x55c3c65fcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.746+0000 7f9a0be83740 1 --2- 172.21.15.45:0/3359134418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65e0f50 0x55c3c65d6aa0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.746+0000 7f9a0be83740 1 --2- 172.21.15.45:0/3359134418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65de550 0x55c3c65e26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.746+0000 7f9a0be83740 1 -- 172.21.15.45:0/3359134418 >> 172.21.15.45:0/3359134418 conn(0x55c3c65de170 msgr2=0x55c3c65f53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:06.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.746+0000 7f9a0be83740 1 -- 172.21.15.45:0/3359134418 shutdown_connections 2024-05-08T05:50:06.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.746+0000 7f9a0be83740 1 -- 172.21.15.45:0/3359134418 wait complete. 2024-05-08T05:50:06.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.746+0000 7f9a0be83740 1 -- start start 2024-05-08T05:50:06.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.747+0000 7f9a0be83740 1 Processor -- start 2024-05-08T05:50:06.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.747+0000 7f9a0be83740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65de550 0x55c3c65ff860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.747+0000 7f9a0be83740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65e0f50 0x55c3c665f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.747+0000 7f9a0be83740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65f8e80 0x55c3c65e4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.747+0000 7f9a0be83740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c3c65a84f0 con 0x55c3c65f8e80 2024-05-08T05:50:06.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.747+0000 7f9a0be83740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9a041fc2b0 con 0x55c3c65de550 2024-05-08T05:50:06.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.747+0000 7f9a0be83740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55c3c6620390 con 0x55c3c65e0f50 2024-05-08T05:50:06.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.747+0000 7f9a0be83740 10 client.0 _ll_register_callbacks cb 0x7f99f4035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:50:06.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65f8e80 0x55c3c65e4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0359e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65f8e80 0x55c3c65e4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50472/0 (socket says 172.21.15.45:50472) 2024-05-08T05:50:06.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0359e640 1 -- 172.21.15.45:0/1402869894 learned_addr learned my addr 172.21.15.45:0/1402869894 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:06.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a02d9d640 1 --2- 172.21.15.45:0/1402869894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65de550 0x55c3c65ff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0259c640 1 --2- 172.21.15.45:0/1402869894 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65e0f50 0x55c3c665f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0359e640 1 -- 172.21.15.45:0/1402869894 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65e0f50 msgr2=0x55c3c665f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:06.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0359e640 1 --2- 172.21.15.45:0/1402869894 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65e0f50 0x55c3c665f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0359e640 1 -- 172.21.15.45:0/1402869894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65de550 msgr2=0x55c3c65ff860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:06.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0359e640 1 --2- 172.21.15.45:0/1402869894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65de550 0x55c3c65ff860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0359e640 1 -- 172.21.15.45:0/1402869894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3c6679480 con 0x55c3c65f8e80 2024-05-08T05:50:06.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a02d9d640 1 --2- 172.21.15.45:0/1402869894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65de550 0x55c3c65ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:06.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0359e640 1 --2- 172.21.15.45:0/1402869894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65f8e80 0x55c3c65e4400 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00cef0 tx=0x7f99fc061200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7f95c36485bd91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:06.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f99e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:50:06.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f99e1ffb640 1 -- 172.21.15.45:0/1402869894 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc069bd0 con 0x55c3c65f8e80 2024-05-08T05:50:06.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f99e1ffb640 1 -- 172.21.15.45:0/1402869894 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f99fc064c70 con 0x55c3c65f8e80 2024-05-08T05:50:06.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0be83740 1 -- 172.21.15.45:0/1402869894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99f8022070 con 0x55c3c65f8e80 2024-05-08T05:50:06.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f99e1ffb640 1 -- 172.21.15.45:0/1402869894 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f99fc0603a0 con 0x55c3c65f8e80 2024-05-08T05:50:06.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0be83740 10 client.7794 fetch_fsmap learned FSMap version 83 2024-05-08T05:50:06.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0be83740 1 -- 172.21.15.45:0/1402869894 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x55c3c65da420 con 0x55c3c65f8e80 2024-05-08T05:50:06.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f99e1ffb640 1 -- 172.21.15.45:0/1402869894 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f8022070 con 0x55c3c65f8e80 2024-05-08T05:50:06.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f99e1ffb640 1 -- 172.21.15.45:0/1402869894 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7f99fc060540 con 0x55c3c65f8e80 2024-05-08T05:50:06.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0be83740 10 client.7794 fetch_fsmap finished waiting for FSMap version 83 2024-05-08T05:50:06.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0be83740 10 client.7794 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-08T05:50:06.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0be83740 20 client.7794 populate_metadata read hostname 'smithi045' 2024-05-08T05:50:06.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0be83740 1 --2- 172.21.15.45:0/1402869894 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55c3c65e4940 0x55c3c65d9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0be83740 4 client.7794 mds_command: new command op to 7524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:50:06.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0be83740 1 -- 172.21.15.45:0/1402869894 --> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c3c651e9b0 con 0x55c3c65e4940 2024-05-08T05:50:06.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a02d9d640 1 --2- 172.21.15.45:0/1402869894 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55c3c65e4940 0x55c3c65d9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f99e1ffb640 10 client.7794 ms_handle_connect on v2:172.21.15.45:6834/878092981 2024-05-08T05:50:06.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a02d9d640 1 --2- 172.21.15.45:0/1402869894 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55c3c65e4940 0x55c3c65d9450 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a3925eeb2f05bc4 server_cookie=f8c7f4c93c13b012 in_seq=0 out_seq=0 2024-05-08T05:50:06.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f99e1ffb640 1 -- 172.21.15.45:0/1402869894 <== mds.0 v2:172.21.15.45:6834/878092981 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c3c651e9b0 con 0x55c3c65e4940 2024-05-08T05:50:06.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f99e1ffb640 10 client.7794 handle_command_reply: tid=0 2024-05-08T05:50:06.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0be83740 1 client.7794 shutdown 2024-05-08T05:50:06.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0be83740 1 -- 172.21.15.45:0/1402869894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65f8e80 msgr2=0x55c3c65e4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:06.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0be83740 1 --2- 172.21.15.45:0/1402869894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65f8e80 0x55c3c65e4400 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00cef0 tx=0x7f99fc061200 comp rx=0 tx=0).stop 2024-05-08T05:50:06.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0be83740 1 -- 172.21.15.45:0/1402869894 shutdown_connections 2024-05-08T05:50:06.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0be83740 1 --2- 172.21.15.45:0/1402869894 >> [v2:172.21.15.45:6834/878092981,v1:172.21.15.45:6835/878092981] conn(0x55c3c65e4940 0x55c3c65d9450 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0be83740 1 --2- 172.21.15.45:0/1402869894 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55c3c65e0f50 0x55c3c665f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0be83740 1 --2- 172.21.15.45:0/1402869894 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c3c65f8e80 0x55c3c65e4400 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0be83740 1 --2- 172.21.15.45:0/1402869894 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55c3c65de550 0x55c3c65ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f99e1ffb640 0 client.7794 ms_handle_reset on v2:172.21.15.45:6834/878092981 2024-05-08T05:50:06.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0be83740 1 -- 172.21.15.45:0/1402869894 >> 172.21.15.45:0/1402869894 conn(0x55c3c65de170 msgr2=0x55c3c65b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:06.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0be83740 1 -- 172.21.15.45:0/1402869894 shutdown_connections 2024-05-08T05:50:06.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.749+0000 7f9a0be83740 1 -- 172.21.15.45:0/1402869894 wait complete. 2024-05-08T05:50:06.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.751+0000 7f9a0be83740 20 client.7794 trim_cache size 0 max 16384 2024-05-08T05:50:06.758 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:50:06.758 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-05-08T05:50:06.758 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-05-08T05:50:06.758 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:50:06.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.751+0000 7f9a0916a640 1 -- 172.21.15.45:0/1936787131 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99f00762d0 msgr2=0x7f99f0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:06.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.751+0000 7f9a0916a640 1 --2- 172.21.15.45:0/1936787131 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99f00762d0 0x7f99f0078790 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f99f4034000 tx=0x7f99f402e000 comp rx=0 tx=0).stop 2024-05-08T05:50:06.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.751+0000 7f9a0916a640 1 -- 172.21.15.45:0/1936787131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a04072840 msgr2=0x7f9a040b27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:06.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.751+0000 7f9a0916a640 1 --2- 172.21.15.45:0/1936787131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a04072840 0x7f9a040b27f0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00cf20 tx=0x7f99fc00a7d0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.751+0000 7f9a0916a640 1 -- 172.21.15.45:0/1936787131 shutdown_connections 2024-05-08T05:50:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.751+0000 7f9a0916a640 1 --2- 172.21.15.45:0/1936787131 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99f00762d0 0x7f99f0078790 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.751+0000 7f9a0916a640 1 --2- 172.21.15.45:0/1936787131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a04072840 0x7f9a040b27f0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.751+0000 7f9a0916a640 1 --2- 172.21.15.45:0/1936787131 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9a0406e4d0 0x7f9a04154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.751+0000 7f9a0916a640 1 --2- 172.21.15.45:0/1936787131 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9a0406e0f0 0x7f9a04151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.751+0000 7f9a0916a640 1 -- 172.21.15.45:0/1936787131 >> 172.21.15.45:0/1936787131 conn(0x7f9a0406c3a0 msgr2=0x7f9a040afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.751+0000 7f9a0916a640 1 -- 172.21.15.45:0/1936787131 shutdown_connections 2024-05-08T05:50:06.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.751+0000 7f9a0916a640 1 -- 172.21.15.45:0/1936787131 wait complete. 2024-05-08T05:50:06.762 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T05:50:06.863 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T05:50:06.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.969+0000 7fbcfdd90640 1 Processor -- start 2024-05-08T05:50:06.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.969+0000 7fbcfdd90640 1 -- start start 2024-05-08T05:50:06.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.969+0000 7fbcfdd90640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbcf815f0c0 0x7fbcf80b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.969+0000 7fbcfdd90640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbcf806c3a0 0x7fbcf806c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.969+0000 7fbcfdd90640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcf80b14e0 con 0x7fbcf81638e0 2024-05-08T05:50:06.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.969+0000 7fbcfdd90640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbcf8163450 con 0x7fbcf815f0c0 2024-05-08T05:50:06.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.969+0000 7fbcfdd90640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf81604b0 con 0x7fbcf806c3a0 2024-05-08T05:50:06.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.970+0000 7fbcf77fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbcf815f0c0 0x7fbcf80b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.970+0000 7fbcf77fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbcf815f0c0 0x7fbcf80b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:58166/0 (socket says 172.21.15.45:58166) 2024-05-08T05:50:06.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.970+0000 7fbcf7fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbcf806c3a0 0x7fbcf806c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.970+0000 7fbcf77fe640 1 -- 172.21.15.45:0/2167166948 learned_addr learned my addr 172.21.15.45:0/2167166948 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:06.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.970+0000 7fbcf6ffd640 1 --1- 172.21.15.45:0/2167166948 >> v1:172.21.15.63:6789/0 conn(0x7fbcf81638e0 0x7fbcf80b0d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:56258/0 (socket says 172.21.15.45:56258) 2024-05-08T05:50:06.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.970+0000 7fbcf77fe640 1 -- 172.21.15.45:0/2167166948 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbcf806c3a0 msgr2=0x7fbcf806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:06.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.970+0000 7fbcf77fe640 1 --2- 172.21.15.45:0/2167166948 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbcf806c3a0 0x7fbcf806c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.970+0000 7fbcf77fe640 1 -- 172.21.15.45:0/2167166948 >> v1:172.21.15.63:6789/0 conn(0x7fbcf81638e0 legacy=0x7fbcf80b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:06.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.970+0000 7fbcf77fe640 1 -- 172.21.15.45:0/2167166948 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf8162e00 con 0x7fbcf815f0c0 2024-05-08T05:50:06.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.971+0000 7fbcf77fe640 1 --2- 172.21.15.45:0/2167166948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcf815f0c0 0x7fbcf80b6f40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009870 tx=0x7fbce802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73e3920b7385d8f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:06.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.971+0000 7fbcf67fc640 1 -- 172.21.15.45:0/2167166948 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8037070 con 0x7fbcf815f0c0 2024-05-08T05:50:06.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.971+0000 7fbcf67fc640 1 -- 172.21.15.45:0/2167166948 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbce8032440 con 0x7fbcf815f0c0 2024-05-08T05:50:06.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.971+0000 7fbcf67fc640 1 -- 172.21.15.45:0/2167166948 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8032810 con 0x7fbcf815f0c0 2024-05-08T05:50:06.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.971+0000 7fbcfdd90640 1 -- 172.21.15.45:0/2167166948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcf815f0c0 msgr2=0x7fbcf80b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:06.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.971+0000 7fbcfdd90640 1 --2- 172.21.15.45:0/2167166948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcf815f0c0 0x7fbcf80b6f40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009870 tx=0x7fbce802a270 comp rx=0 tx=0).stop 2024-05-08T05:50:06.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.972+0000 7fbcfdd90640 1 -- 172.21.15.45:0/2167166948 shutdown_connections 2024-05-08T05:50:06.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.972+0000 7fbcfdd90640 1 --2- 172.21.15.45:0/2167166948 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbcf806c3a0 0x7fbcf806c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.972+0000 7fbcfdd90640 1 --2- 172.21.15.45:0/2167166948 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcf815f0c0 0x7fbcf80b6f40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.972+0000 7fbcfdd90640 1 -- 172.21.15.45:0/2167166948 >> 172.21.15.45:0/2167166948 conn(0x7fbcf80b3b10 msgr2=0x7fbcf80b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:06.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.972+0000 7fbcfdd90640 1 -- 172.21.15.45:0/2167166948 shutdown_connections 2024-05-08T05:50:06.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.972+0000 7fbcfdd90640 1 -- 172.21.15.45:0/2167166948 wait complete. 2024-05-08T05:50:06.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.973+0000 7fbcfdd90640 1 Processor -- start 2024-05-08T05:50:06.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.973+0000 7fbcfdd90640 1 -- start start 2024-05-08T05:50:06.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.974+0000 7fbcfdd90640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbcf806c3a0 0x7fbcf8156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.974+0000 7fbcfdd90640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcf815f0c0 0x7fbcf815bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.974+0000 7fbcfdd90640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcf81638e0 0x7fbcf815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.974+0000 7fbcfdd90640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf815f4a0 con 0x7fbcf815f0c0 2024-05-08T05:50:06.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.974+0000 7fbcfdd90640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf80b17c0 con 0x7fbcf81638e0 2024-05-08T05:50:06.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.974+0000 7fbcfdd90640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf815f620 con 0x7fbcf806c3a0 2024-05-08T05:50:06.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.974+0000 7fbcf6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcf815f0c0 0x7fbcf815bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.974+0000 7fbcf77fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbcf806c3a0 0x7fbcf8156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.974+0000 7fbcf77fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbcf806c3a0 0x7fbcf8156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:33008/0 (socket says 172.21.15.45:33008) 2024-05-08T05:50:06.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.974+0000 7fbcf77fe640 1 -- 172.21.15.45:0/3184194650 learned_addr learned my addr 172.21.15.45:0/3184194650 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:06.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.974+0000 7fbcf7fff640 1 --2- 172.21.15.45:0/3184194650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcf81638e0 0x7fbcf815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.974+0000 7fbcf77fe640 1 -- 172.21.15.45:0/3184194650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcf81638e0 msgr2=0x7fbcf815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:06.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.974+0000 7fbcf77fe640 1 --2- 172.21.15.45:0/3184194650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcf81638e0 0x7fbcf815c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.974+0000 7fbcf77fe640 1 -- 172.21.15.45:0/3184194650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcf815f0c0 msgr2=0x7fbcf815bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:06.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.974+0000 7fbcf77fe640 1 --2- 172.21.15.45:0/3184194650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcf815f0c0 0x7fbcf815bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.974+0000 7fbcf77fe640 1 -- 172.21.15.45:0/3184194650 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf81f5860 con 0x7fbcf806c3a0 2024-05-08T05:50:06.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.974+0000 7fbcf6ffd640 1 --2- 172.21.15.45:0/3184194650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcf815f0c0 0x7fbcf815bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:50:06.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.974+0000 7fbcf77fe640 1 --2- 172.21.15.45:0/3184194650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbcf806c3a0 0x7fbcf8156c00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbce8000c00 tx=0x7fbce80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:06.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.975+0000 7fbcdffff640 1 -- 172.21.15.45:0/3184194650 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8037040 con 0x7fbcf806c3a0 2024-05-08T05:50:06.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.975+0000 7fbcdffff640 1 -- 172.21.15.45:0/3184194650 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbce8042070 con 0x7fbcf806c3a0 2024-05-08T05:50:06.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.975+0000 7fbcdffff640 1 -- 172.21.15.45:0/3184194650 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8030860 con 0x7fbcf806c3a0 2024-05-08T05:50:06.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.975+0000 7fbcfdd90640 1 -- 172.21.15.45:0/3184194650 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf815aa20 con 0x7fbcf806c3a0 2024-05-08T05:50:06.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.975+0000 7fbcfdd90640 1 -- 172.21.15.45:0/3184194650 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf815afe0 con 0x7fbcf806c3a0 2024-05-08T05:50:06.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.976+0000 7fbcfdd90640 1 -- 172.21.15.45:0/3184194650 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf80693f0 con 0x7fbcf806c3a0 2024-05-08T05:50:06.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.977+0000 7fbcdffff640 1 -- 172.21.15.45:0/3184194650 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbce8030a00 con 0x7fbcf806c3a0 2024-05-08T05:50:06.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.977+0000 7fbcdffff640 1 --2- 172.21.15.45:0/3184194650 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbce40762d0 0x7fbce4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:06.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.977+0000 7fbcdffff640 1 -- 172.21.15.45:0/3184194650 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fbce80b8de0 con 0x7fbcf806c3a0 2024-05-08T05:50:06.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.977+0000 7fbcf6ffd640 1 --2- 172.21.15.45:0/3184194650 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbce40762d0 0x7fbce4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:06.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.978+0000 7fbcf6ffd640 1 --2- 172.21.15.45:0/3184194650 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbce40762d0 0x7fbce4078790 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8162890 tx=0x7fbcec000bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:06.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:06.982+0000 7fbcdffff640 1 -- 172.21.15.45:0/3184194650 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbce8081fe0 con 0x7fbcf806c3a0 2024-05-08T05:50:07.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.214+0000 7fbcfdd90640 1 -- 172.21.15.45:0/3184194650 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbcf815b7e0 con 0x7fbcf806c3a0 2024-05-08T05:50:07.216 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-05-08T05:50:07.216+0000 7f124de12640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T05:50:07.223 INFO:tasks.ceph.mds.a.smithi045.stderr: -7068> 2024-05-08T05:49:25.187+0000 7fdc0f8e4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:50:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.223+0000 7fbcdffff640 1 -- 172.21.15.45:0/3184194650 <== mon.2 v2:172.21.15.63:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) ==== 161+0+0 (secure 0 0 0) 0x7fbce80862e0 con 0x7fbcf806c3a0 2024-05-08T05:50:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T05:50:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.226+0000 7fbcfdd90640 1 -- 172.21.15.45:0/3184194650 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbce40762d0 msgr2=0x7fbce4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.226+0000 7fbcfdd90640 1 --2- 172.21.15.45:0/3184194650 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbce40762d0 0x7fbce4078790 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8162890 tx=0x7fbcec000bc0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.226+0000 7fbcfdd90640 1 -- 172.21.15.45:0/3184194650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbcf806c3a0 msgr2=0x7fbcf8156c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.226+0000 7fbcfdd90640 1 --2- 172.21.15.45:0/3184194650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbcf806c3a0 0x7fbcf8156c00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbce8000c00 tx=0x7fbce80046c0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.226+0000 7fbcfdd90640 1 -- 172.21.15.45:0/3184194650 shutdown_connections 2024-05-08T05:50:07.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.226+0000 7fbcfdd90640 1 --2- 172.21.15.45:0/3184194650 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbce40762d0 0x7fbce4078790 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.226+0000 7fbcfdd90640 1 --2- 172.21.15.45:0/3184194650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcf81638e0 0x7fbcf815c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.226+0000 7fbcfdd90640 1 --2- 172.21.15.45:0/3184194650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbcf815f0c0 0x7fbcf815bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.226+0000 7fbcfdd90640 1 --2- 172.21.15.45:0/3184194650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbcf806c3a0 0x7fbcf8156c00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.226+0000 7fbcfdd90640 1 -- 172.21.15.45:0/3184194650 >> 172.21.15.45:0/3184194650 conn(0x7fbcf80b3b10 msgr2=0x7fbcf8160cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:07.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.226+0000 7fbcfdd90640 1 -- 172.21.15.45:0/3184194650 shutdown_connections 2024-05-08T05:50:07.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.226+0000 7fbcfdd90640 1 -- 172.21.15.45:0/3184194650 wait complete. 2024-05-08T05:50:07.238 DEBUG:teuthology.orchestra.run.smithi045:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-05-08T05:50:07.256 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-05-08T05:50:07.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.283+0000 7f749a920280 1 -- start start 2024-05-08T05:50:07.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.283+0000 7f749a920280 1 Processor -- start 2024-05-08T05:50:07.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f749a920280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec8a18c00 0x560ec97f0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f749a920280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ec8a19800 0x560ec97f0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f749a920280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560ec8a19c00 0x560ec97f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f749a920280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560ec97b6180 con 0x560ec8a18c00 2024-05-08T05:50:07.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f749a920280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560ec8a7a900 con 0x560ec8a19800 2024-05-08T05:50:07.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f749a920280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x560ec8a7a480 con 0x560ec8a19c00 2024-05-08T05:50:07.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f7498654640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec8a18c00 0x560ec97f0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f7498654640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec8a18c00 0x560ec97f0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50516/0 (socket says 172.21.15.45:50516) 2024-05-08T05:50:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f7498654640 1 -- 172.21.15.45:0/3189630192 learned_addr learned my addr 172.21.15.45:0/3189630192 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f7497e53640 1 --2- 172.21.15.45:0/3189630192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ec8a19800 0x560ec97f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f7498e55640 1 --2- 172.21.15.45:0/3189630192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560ec8a19c00 0x560ec97f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f7498654640 1 -- 172.21.15.45:0/3189630192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560ec8a19c00 msgr2=0x560ec97f0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f7498654640 1 --2- 172.21.15.45:0/3189630192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560ec8a19c00 0x560ec97f0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f7498654640 1 -- 172.21.15.45:0/3189630192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ec8a19800 msgr2=0x560ec97f0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f7498654640 1 --2- 172.21.15.45:0/3189630192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ec8a19800 0x560ec97f0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f7498654640 1 -- 172.21.15.45:0/3189630192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ec8a4e780 con 0x560ec8a18c00 2024-05-08T05:50:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f7498654640 1 --2- 172.21.15.45:0/3189630192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec8a18c00 0x560ec97f0000 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x560ec984b7d0 tx=0x560ec97baa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf834d222c9cb60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f7495e4f640 1 -- 172.21.15.45:0/3189630192 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560ec8a3e680 con 0x560ec8a18c00 2024-05-08T05:50:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f7495e4f640 1 -- 172.21.15.45:0/3189630192 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x560ec8a44700 con 0x560ec8a18c00 2024-05-08T05:50:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f7495e4f640 1 -- 172.21.15.45:0/3189630192 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560ec8a3eb60 con 0x560ec8a18c00 2024-05-08T05:50:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f749a920280 1 -- 172.21.15.45:0/3189630192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ec8a4f4a0 con 0x560ec8a18c00 2024-05-08T05:50:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f749a920280 1 -- 172.21.15.45:0/3189630192 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=0}) -- 0x560ec8a4e000 con 0x560ec8a18c00 2024-05-08T05:50:07.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.284+0000 7f749a920280 4 waiting for MDS map... 2024-05-08T05:50:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.285+0000 7f7495e4f640 1 -- 172.21.15.45:0/3189630192 <== mon.0 v2:172.21.15.45:3300/0 4 ==== osd_map(105..105 src has 1..105) ==== 6187+0+0 (secure 0 0 0) 0x560ec98fc600 con 0x560ec8a18c00 2024-05-08T05:50:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.285+0000 7f7495e4f640 1 -- 172.21.15.45:0/3189630192 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 84) ==== 2885+0+0 (secure 0 0 0) 0x560ec8a73180 con 0x560ec8a18c00 2024-05-08T05:50:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.285+0000 7f749a920280 4 Got MDS map 84 2024-05-08T05:50:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.285+0000 7f749a920280 10 main: JournalTool::main 2024-05-08T05:50:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.285+0000 7f749a920280 4 main: JournalTool: connecting to RADOS... 2024-05-08T05:50:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.285+0000 7f749a920280 1 Processor -- start 2024-05-08T05:50:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.285+0000 7f749a920280 1 -- start start 2024-05-08T05:50:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.286+0000 7f749a920280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec9805400 0x560ec97f1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.286+0000 7f749a920280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ec9805800 0x560ec97f1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.286+0000 7f749a920280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560ec9805c00 0x560ec97f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.286+0000 7f749a920280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560ec8a7a180 con 0x560ec9805400 2024-05-08T05:50:07.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.286+0000 7f749a920280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560ec97b6d80 con 0x560ec9805800 2024-05-08T05:50:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.286+0000 7f749a920280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x560ec97b6f00 con 0x560ec9805c00 2024-05-08T05:50:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.286+0000 7f7498e55640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec9805400 0x560ec97f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.286+0000 7f7498e55640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec9805400 0x560ec97f1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50530/0 (socket says 172.21.15.45:50530) 2024-05-08T05:50:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.286+0000 7f7498e55640 1 -- 172.21.15.45:0/1904269111 learned_addr learned my addr 172.21.15.45:0/1904269111 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.286+0000 7f7498654640 1 --2- 172.21.15.45:0/1904269111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ec9805800 0x560ec97f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.286+0000 7f7497e53640 1 --2- 172.21.15.45:0/1904269111 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560ec9805c00 0x560ec97f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.286+0000 7f7498e55640 1 -- 172.21.15.45:0/1904269111 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560ec9805c00 msgr2=0x560ec97f1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.286+0000 7f7498e55640 1 --2- 172.21.15.45:0/1904269111 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560ec9805c00 0x560ec97f1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.286+0000 7f7498e55640 1 -- 172.21.15.45:0/1904269111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ec9805800 msgr2=0x560ec97f1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.286+0000 7f7498e55640 1 --2- 172.21.15.45:0/1904269111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ec9805800 0x560ec97f1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.286+0000 7f7498e55640 1 -- 172.21.15.45:0/1904269111 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ec8a4e3c0 con 0x560ec9805400 2024-05-08T05:50:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.286+0000 7f7498654640 1 --2- 172.21.15.45:0/1904269111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ec9805800 0x560ec97f1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:50:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.287+0000 7f7498e55640 1 --2- 172.21.15.45:0/1904269111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec9805400 0x560ec97f1080 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x560ec9914660 tx=0x560ec9917800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd615a17af417473 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.287+0000 7f7493e4b640 1 -- 172.21.15.45:0/1904269111 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560ec8a3e680 con 0x560ec9805400 2024-05-08T05:50:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.287+0000 7f7493e4b640 1 -- 172.21.15.45:0/1904269111 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x560ec8a44fc0 con 0x560ec9805400 2024-05-08T05:50:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.287+0000 7f7493e4b640 1 -- 172.21.15.45:0/1904269111 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560ec8a3eb60 con 0x560ec9805400 2024-05-08T05:50:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.287+0000 7f749a920280 1 -- 172.21.15.45:0/1904269111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec9805400 msgr2=0x560ec97f1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.287+0000 7f749a920280 1 --2- 172.21.15.45:0/1904269111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec9805400 0x560ec97f1080 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x560ec9914660 tx=0x560ec9917800 comp rx=0 tx=0).stop 2024-05-08T05:50:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.287+0000 7f749a920280 1 -- 172.21.15.45:0/1904269111 shutdown_connections 2024-05-08T05:50:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.287+0000 7f749a920280 1 --2- 172.21.15.45:0/1904269111 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560ec9805c00 0x560ec97f1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.287+0000 7f749a920280 1 --2- 172.21.15.45:0/1904269111 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ec9805800 0x560ec97f1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.287+0000 7f749a920280 1 --2- 172.21.15.45:0/1904269111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec9805400 0x560ec97f1080 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.287+0000 7f749a920280 1 -- 172.21.15.45:0/1904269111 >> 172.21.15.45:0/1904269111 conn(0x560ec9805000 msgr2=0x560ec8b13000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.287+0000 7f749a920280 1 -- 172.21.15.45:0/1904269111 shutdown_connections 2024-05-08T05:50:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.287+0000 7f749a920280 1 -- 172.21.15.45:0/1904269111 wait complete. 2024-05-08T05:50:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.288+0000 7f749a920280 1 Processor -- start 2024-05-08T05:50:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.288+0000 7f749a920280 1 -- start start 2024-05-08T05:50:07.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.288+0000 7f749a920280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec9805400 0x560ec97f1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.288+0000 7f749a920280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ec9805800 0x560ec97f1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.288+0000 7f749a920280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560ec9805c00 0x560ec97f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.288+0000 7f749a920280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560ec97b7680 con 0x560ec9805400 2024-05-08T05:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.288+0000 7f749a920280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x560ec97b7200 con 0x560ec9805800 2024-05-08T05:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.288+0000 7f749a920280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x560ec97b7080 con 0x560ec9805c00 2024-05-08T05:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.288+0000 7f7498e55640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec9805400 0x560ec97f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.288+0000 7f7498e55640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec9805400 0x560ec97f1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50534/0 (socket says 172.21.15.45:50534) 2024-05-08T05:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.288+0000 7f7498e55640 1 -- 172.21.15.45:0/1400146129 learned_addr learned my addr 172.21.15.45:0/1400146129 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.289+0000 7f7498654640 1 --2- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ec9805800 0x560ec97f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.289+0000 7f7497e53640 1 --2- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560ec9805c00 0x560ec97f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.289+0000 7f7498e55640 1 -- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560ec9805c00 msgr2=0x560ec97f1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.289+0000 7f7498e55640 1 --2- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560ec9805c00 0x560ec97f1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.289+0000 7f7498e55640 1 -- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ec9805800 msgr2=0x560ec97f1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.289+0000 7f7498e55640 1 --2- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ec9805800 0x560ec97f1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.289+0000 7f7498e55640 1 -- 172.21.15.45:0/1400146129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ec8a4e780 con 0x560ec9805400 2024-05-08T05:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.289+0000 7f7498e55640 1 --2- 172.21.15.45:0/1400146129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec9805400 0x560ec97f1080 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x560ec9915920 tx=0x560ec97b88a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.289+0000 7f7492648640 1 -- 172.21.15.45:0/1400146129 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560ec8a3e4e0 con 0x560ec9805400 2024-05-08T05:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.289+0000 7f7492648640 1 -- 172.21.15.45:0/1400146129 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x560ec8a44a80 con 0x560ec9805400 2024-05-08T05:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.289+0000 7f749a920280 1 -- 172.21.15.45:0/1400146129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560ec8a4f4a0 con 0x560ec9805400 2024-05-08T05:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.289+0000 7f7492648640 1 -- 172.21.15.45:0/1400146129 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560ec8a3e680 con 0x560ec9805400 2024-05-08T05:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.289+0000 7f749a920280 1 -- 172.21.15.45:0/1400146129 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ec8a4e960 con 0x560ec9805400 2024-05-08T05:50:07.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.289+0000 7f749a920280 4 main: JournalTool: resolving pool 15 2024-05-08T05:50:07.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.290+0000 7f7492648640 1 -- 172.21.15.45:0/1400146129 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x560ec982cc00 con 0x560ec9805400 2024-05-08T05:50:07.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.291+0000 7f7492648640 1 --2- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x560ec9828c00 0x560ec97f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.291+0000 7f7492648640 1 -- 172.21.15.45:0/1400146129 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6187+0+0 (secure 0 0 0) 0x560ec982aa00 con 0x560ec9805400 2024-05-08T05:50:07.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.291+0000 7f749a920280 4 main: JournalTool: creating IoCtx.. 2024-05-08T05:50:07.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.291+0000 7f749a920280 4 main: Executing for rank 0 2024-05-08T05:50:07.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.291+0000 7f749a920280 4 Filtering by inode '1099511627777' 2024-05-08T05:50:07.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.291+0000 7f749a920280 1 --2- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x560ec9829000 0x560ec97f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.291+0000 7f749a920280 1 -- 172.21.15.45:0/1400146129 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105) -- 0x560ec9829400 con 0x560ec9829000 2024-05-08T05:50:07.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.291+0000 7f7498654640 1 --2- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x560ec9828c00 0x560ec97f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.291+0000 7f7497e53640 1 --2- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x560ec9829000 0x560ec97f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.291+0000 7f7498654640 1 --2- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x560ec9828c00 0x560ec97f2100 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x560ec984bcb0 tx=0x560ec97b8a80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.292+0000 7f7497e53640 1 --2- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x560ec9829000 0x560ec97f2680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.295+0000 7f7497e53640 1 -- 172.21.15.45:0/1400146129 <== osd.6 v2:172.21.15.63:6800/2926759052 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x560ec8b26000 con 0x560ec9829000 2024-05-08T05:50:07.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.295+0000 7f749a920280 4 Pointer 400.00000000 is readable 2024-05-08T05:50:07.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.295+0000 7f749a920280 4 JournalScanner::scan: reading header object '200.00000000' 2024-05-08T05:50:07.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.295+0000 7f749a920280 1 -- 172.21.15.45:0/1400146129 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105) -- 0x560ec9829800 con 0x560ec9829000 2024-05-08T05:50:07.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.295+0000 7f7497e53640 1 -- 172.21.15.45:0/1400146129 <== osd.6 v2:172.21.15.63:6800/2926759052 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x560ec8b26000 con 0x560ec9829000 2024-05-08T05:50:07.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.295+0000 7f749a920280 10 Header 0x400000 0x407d21 0x40809d 2024-05-08T05:50:07.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.295+0000 7f749a920280 10 Starting journal scan from offset 0x407d21 2024-05-08T05:50:07.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.295+0000 7f749a920280 1 -- 172.21.15.45:0/1400146129 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 32033~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105) -- 0x560ec9829c00 con 0x560ec9829000 2024-05-08T05:50:07.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.296+0000 7f7497e53640 1 -- 172.21.15.45:0/1400146129 <== osd.6 v2:172.21.15.63:6800/2926759052 3 ==== osd_op_reply(3 200.00000001 [read 32033~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x560ec8b26000 con 0x560ec9829000 2024-05-08T05:50:07.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.296+0000 7f749a920280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-05-08T05:50:07.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.296+0000 7f749a920280 10 Parsing data, 0x37c bytes available 2024-05-08T05:50:07.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.296+0000 7f749a920280 10 Attempting decode at 0x407d21 2024-05-08T05:50:07.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.296+0000 7f749a920280 10 Consumed 0x37c bytes 2024-05-08T05:50:07.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.296+0000 7f749a920280 10 Valid entry at 0x407d21 2024-05-08T05:50:07.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.296+0000 7f749a920280 1 --2- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x560ec8dc3000 0x560ec97f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.296+0000 7f749a920280 1 -- 172.21.15.45:0/1400146129 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105) -- 0x560ec9829400 con 0x560ec8dc3000 2024-05-08T05:50:07.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.296+0000 7f7498e55640 1 --2- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x560ec8dc3000 0x560ec97f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.296+0000 7f7498e55640 1 --2- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x560ec8dc3000 0x560ec97f2c00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f7498e55640 1 -- 172.21.15.45:0/1400146129 <== osd.7 v2:172.21.15.63:6816/1163469437 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x560ec8b26500 con 0x560ec8dc3000 2024-05-08T05:50:07.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 4 Reached end of journal objects 2024-05-08T05:50:07.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 4 Scanned objects, 0 missing, 1 valid 2024-05-08T05:50:07.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 4 Events scanned, 0 gaps 2024-05-08T05:50:07.302 INFO:teuthology.orchestra.run.smithi045.stdout:Events by type: 2024-05-08T05:50:07.303 INFO:teuthology.orchestra.run.smithi045.stdout:Errors: 0 2024-05-08T05:50:07.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 4 Found 1 valid events 2024-05-08T05:50:07.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 4 Selected 0 events events for processing 2024-05-08T05:50:07.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 4 0 events 2024-05-08T05:50:07.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 -- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x560ec9829000 msgr2=0x560ec97f2680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 --2- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x560ec9829000 0x560ec97f2680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 -- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x560ec8dc3000 msgr2=0x560ec97f2c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 --2- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x560ec8dc3000 0x560ec97f2c00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 -- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x560ec9828c00 msgr2=0x560ec97f2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 --2- 172.21.15.45:0/1400146129 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x560ec9828c00 0x560ec97f2100 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x560ec984bcb0 tx=0x560ec97b8a80 comp rx=0 tx=0).stop 2024-05-08T05:50:07.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 -- 172.21.15.45:0/1400146129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec9805400 msgr2=0x560ec97f1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f7497e53640 1 -- 172.21.15.45:0/1400146129 reap_dead start 2024-05-08T05:50:07.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 --2- 172.21.15.45:0/1400146129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec9805400 0x560ec97f1080 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x560ec9915920 tx=0x560ec97b88a0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 -- 172.21.15.45:0/1400146129 shutdown_connections 2024-05-08T05:50:07.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 --2- 172.21.15.45:0/1400146129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec9805400 0x560ec97f1080 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 -- 172.21.15.45:0/1400146129 >> 172.21.15.45:0/1400146129 conn(0x560ec9805000 msgr2=0x560ec8b13000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:07.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 -- 172.21.15.45:0/1400146129 shutdown_connections 2024-05-08T05:50:07.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 -- 172.21.15.45:0/1400146129 wait complete. 2024-05-08T05:50:07.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 -- 172.21.15.45:0/3189630192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec8a18c00 msgr2=0x560ec97f0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 --2- 172.21.15.45:0/3189630192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec8a18c00 0x560ec97f0000 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x560ec984b7d0 tx=0x560ec97baa80 comp rx=0 tx=0).stop 2024-05-08T05:50:07.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 -- 172.21.15.45:0/3189630192 shutdown_connections 2024-05-08T05:50:07.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 --2- 172.21.15.45:0/3189630192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x560ec8a19c00 0x560ec97f0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 --2- 172.21.15.45:0/3189630192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x560ec8a19800 0x560ec97f0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 --2- 172.21.15.45:0/3189630192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ec8a18c00 0x560ec97f0000 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 -- 172.21.15.45:0/3189630192 >> 172.21.15.45:0/3189630192 conn(0x560ec8a19000 msgr2=0x560ec8b12800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:07.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 -- 172.21.15.45:0/3189630192 shutdown_connections 2024-05-08T05:50:07.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.297+0000 7f749a920280 1 -- 172.21.15.45:0/3189630192 wait complete. 2024-05-08T05:50:07.307 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.069068, result: Events by type: Errors: 0 2024-05-08T05:50:07.307 DEBUG:teuthology.orchestra.run.smithi045:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-05-08T05:50:07.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.390+0000 7f9f6c40e280 1 -- start start 2024-05-08T05:50:07.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.391+0000 7f9f6c40e280 1 Processor -- start 2024-05-08T05:50:07.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.391+0000 7f9f6c40e280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5637315e2c00 0x5637323a8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.391+0000 7f9f6c40e280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5637315e3800 0x5637323a8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.391+0000 7f9f6c40e280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637315e3c00 0x5637323a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.391+0000 7f9f6c40e280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56373175c180 con 0x5637315e3c00 2024-05-08T05:50:07.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.391+0000 7f9f6c40e280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56373175c300 con 0x5637315e3800 2024-05-08T05:50:07.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.391+0000 7f9f6c40e280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56373175c480 con 0x5637315e2c00 2024-05-08T05:50:07.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.391+0000 7f9f6a74c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637315e3c00 0x5637323a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.391+0000 7f9f6a74c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637315e3c00 0x5637323a8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50546/0 (socket says 172.21.15.45:50546) 2024-05-08T05:50:07.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.391+0000 7f9f6a74c640 1 -- 172.21.15.45:0/2860600294 learned_addr learned my addr 172.21.15.45:0/2860600294 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:07.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.391+0000 7f9f69f4b640 1 --2- 172.21.15.45:0/2860600294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5637315e2c00 0x5637323a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.391+0000 7f9f6974a640 1 --2- 172.21.15.45:0/2860600294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5637315e3800 0x5637323a8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.392+0000 7f9f6a74c640 1 -- 172.21.15.45:0/2860600294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5637315e2c00 msgr2=0x5637323a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:07.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.392+0000 7f9f6a74c640 1 --2- 172.21.15.45:0/2860600294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5637315e2c00 0x5637323a8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.392+0000 7f9f6a74c640 1 -- 172.21.15.45:0/2860600294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5637315e3800 msgr2=0x5637323a8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.392+0000 7f9f6a74c640 1 --2- 172.21.15.45:0/2860600294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5637315e3800 0x5637323a8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.392+0000 7f9f6a74c640 1 -- 172.21.15.45:0/2860600294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563731618780 con 0x5637315e3c00 2024-05-08T05:50:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.392+0000 7f9f6a74c640 1 --2- 172.21.15.45:0/2860600294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637315e3c00 0x5637323a8b00 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x563732401800 tx=0x563732384120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb5aa8cb8ca344e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.392+0000 7f9f67746640 1 -- 172.21.15.45:0/2860600294 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563731608680 con 0x5637315e3c00 2024-05-08T05:50:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.392+0000 7f9f67746640 1 -- 172.21.15.45:0/2860600294 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56373160ea80 con 0x5637315e3c00 2024-05-08T05:50:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.392+0000 7f9f67746640 1 -- 172.21.15.45:0/2860600294 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563731608b60 con 0x5637315e3c00 2024-05-08T05:50:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.392+0000 7f9f6c40e280 1 -- 172.21.15.45:0/2860600294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637316183c0 con 0x5637315e3c00 2024-05-08T05:50:07.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.392+0000 7f9f6c40e280 1 -- 172.21.15.45:0/2860600294 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=0}) -- 0x563731619680 con 0x5637315e3c00 2024-05-08T05:50:07.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.392+0000 7f9f6c40e280 4 waiting for MDS map... 2024-05-08T05:50:07.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.392+0000 7f9f67746640 1 -- 172.21.15.45:0/2860600294 <== mon.0 v2:172.21.15.45:3300/0 4 ==== osd_map(105..105 src has 1..105) ==== 6187+0+0 (secure 0 0 0) 0x5637324c8600 con 0x5637315e3c00 2024-05-08T05:50:07.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.392+0000 7f9f67746640 1 -- 172.21.15.45:0/2860600294 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 84) ==== 2885+0+0 (secure 0 0 0) 0x56373163d180 con 0x5637315e3c00 2024-05-08T05:50:07.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.392+0000 7f9f6c40e280 4 Got MDS map 84 2024-05-08T05:50:07.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.392+0000 7f9f6c40e280 10 main: JournalTool::main 2024-05-08T05:50:07.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.393+0000 7f9f6c40e280 4 main: JournalTool: connecting to RADOS... 2024-05-08T05:50:07.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.393+0000 7f9f6c40e280 1 Processor -- start 2024-05-08T05:50:07.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.393+0000 7f9f6c40e280 1 -- start start 2024-05-08T05:50:07.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.394+0000 7f9f6c40e280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637323c7400 0x5637323a9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.394+0000 7f9f6c40e280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5637323c7800 0x5637323a9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.394+0000 7f9f6c40e280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5637323c7c00 0x5637323a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.394+0000 7f9f6c40e280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56373175c600 con 0x5637323c7400 2024-05-08T05:50:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.394+0000 7f9f6c40e280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56373175c780 con 0x5637323c7c00 2024-05-08T05:50:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.394+0000 7f9f6c40e280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56373175c900 con 0x5637323c7800 2024-05-08T05:50:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.394+0000 7f9f6974a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637323c7400 0x5637323a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.394+0000 7f9f6974a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637323c7400 0x5637323a9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50556/0 (socket says 172.21.15.45:50556) 2024-05-08T05:50:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.394+0000 7f9f6974a640 1 -- 172.21.15.45:0/3189006663 learned_addr learned my addr 172.21.15.45:0/3189006663 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.394+0000 7f9f69f4b640 1 --2- 172.21.15.45:0/3189006663 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5637323c7800 0x5637323a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.394+0000 7f9f6974a640 1 --2- 172.21.15.45:0/3189006663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5637323c7c00 0x5637323a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.394+0000 7f9f6974a640 1 -- 172.21.15.45:0/3189006663 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5637323c7800 msgr2=0x5637323a9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.394+0000 7f9f6974a640 1 --2- 172.21.15.45:0/3189006663 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5637323c7800 0x5637323a9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.394+0000 7f9f6974a640 1 -- 172.21.15.45:0/3189006663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5637323c7c00 msgr2=0x5637323a9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.394+0000 7f9f6974a640 1 --2- 172.21.15.45:0/3189006663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5637323c7c00 0x5637323a9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.394+0000 7f9f6974a640 1 -- 172.21.15.45:0/3189006663 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637316194a0 con 0x5637323c7400 2024-05-08T05:50:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.394+0000 7f9f6974a640 1 --2- 172.21.15.45:0/3189006663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637323c7400 0x5637323a9080 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x5637324b6600 tx=0x563732382a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122606dab4801eed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.394+0000 7f9f65742640 1 -- 172.21.15.45:0/3189006663 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5637316089c0 con 0x5637323c7400 2024-05-08T05:50:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.394+0000 7f9f65742640 1 -- 172.21.15.45:0/3189006663 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56373160efc0 con 0x5637323c7400 2024-05-08T05:50:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.394+0000 7f9f65742640 1 -- 172.21.15.45:0/3189006663 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563731608680 con 0x5637323c7400 2024-05-08T05:50:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.395+0000 7f9f6c40e280 1 -- 172.21.15.45:0/3189006663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637323c7400 msgr2=0x5637323a9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.395+0000 7f9f6c40e280 1 --2- 172.21.15.45:0/3189006663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637323c7400 0x5637323a9080 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x5637324b6600 tx=0x563732382a80 comp rx=0 tx=0).stop 2024-05-08T05:50:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.395+0000 7f9f6c40e280 1 -- 172.21.15.45:0/3189006663 shutdown_connections 2024-05-08T05:50:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.395+0000 7f9f6c40e280 1 --2- 172.21.15.45:0/3189006663 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5637323c7800 0x5637323a9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.395+0000 7f9f6c40e280 1 --2- 172.21.15.45:0/3189006663 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5637323c7c00 0x5637323a9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.395+0000 7f9f6c40e280 1 --2- 172.21.15.45:0/3189006663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637323c7400 0x5637323a9080 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.395+0000 7f9f6c40e280 1 -- 172.21.15.45:0/3189006663 >> 172.21.15.45:0/3189006663 conn(0x5637323c7000 msgr2=0x5637316dd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.395+0000 7f9f6c40e280 1 -- 172.21.15.45:0/3189006663 shutdown_connections 2024-05-08T05:50:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.395+0000 7f9f6c40e280 1 -- 172.21.15.45:0/3189006663 wait complete. 2024-05-08T05:50:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.395+0000 7f9f6c40e280 1 Processor -- start 2024-05-08T05:50:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.395+0000 7f9f6c40e280 1 -- start start 2024-05-08T05:50:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.396+0000 7f9f6c40e280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637323c7400 0x5637323a9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.396+0000 7f9f6c40e280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5637323c7800 0x5637323a9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.396+0000 7f9f6c40e280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5637323c7c00 0x5637323a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.396+0000 7f9f6c40e280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56373175db00 con 0x5637323c7400 2024-05-08T05:50:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.396+0000 7f9f6c40e280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56373175d980 con 0x5637323c7c00 2024-05-08T05:50:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.396+0000 7f9f6c40e280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56373175d800 con 0x5637323c7800 2024-05-08T05:50:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.396+0000 7f9f6974a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637323c7400 0x5637323a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.396+0000 7f9f6974a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637323c7400 0x5637323a9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50560/0 (socket says 172.21.15.45:50560) 2024-05-08T05:50:07.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.396+0000 7f9f6974a640 1 -- 172.21.15.45:0/1073201174 learned_addr learned my addr 172.21.15.45:0/1073201174 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.396+0000 7f9f69f4b640 1 --2- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5637323c7800 0x5637323a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.396+0000 7f9f6974a640 1 --2- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5637323c7c00 0x5637323a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.396+0000 7f9f6974a640 1 -- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5637323c7800 msgr2=0x5637323a9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.396+0000 7f9f6974a640 1 --2- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5637323c7800 0x5637323a9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.396+0000 7f9f6974a640 1 -- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5637323c7c00 msgr2=0x5637323a9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.396+0000 7f9f6974a640 1 --2- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5637323c7c00 0x5637323a9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.396+0000 7f9f6974a640 1 -- 172.21.15.45:0/1073201174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563731618780 con 0x5637323c7400 2024-05-08T05:50:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.397+0000 7f9f6974a640 1 --2- 172.21.15.45:0/1073201174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637323c7400 0x5637323a9080 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x5637324b6930 tx=0x563732382600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.397+0000 7f9f63f3f640 1 -- 172.21.15.45:0/1073201174 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563731608b60 con 0x5637323c7400 2024-05-08T05:50:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.397+0000 7f9f63f3f640 1 -- 172.21.15.45:0/1073201174 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56373160ee00 con 0x5637323c7400 2024-05-08T05:50:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.397+0000 7f9f6c40e280 1 -- 172.21.15.45:0/1073201174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5637316183c0 con 0x5637323c7400 2024-05-08T05:50:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.397+0000 7f9f63f3f640 1 -- 172.21.15.45:0/1073201174 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5637316084e0 con 0x5637323c7400 2024-05-08T05:50:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.397+0000 7f9f6c40e280 1 -- 172.21.15.45:0/1073201174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563731619680 con 0x5637323c7400 2024-05-08T05:50:07.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.397+0000 7f9f6c40e280 4 main: JournalTool: resolving pool 15 2024-05-08T05:50:07.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.398+0000 7f9f63f3f640 1 -- 172.21.15.45:0/1073201174 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x5637323dec00 con 0x5637323c7400 2024-05-08T05:50:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.398+0000 7f9f63f3f640 1 --2- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x563731980000 0x5637323aa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.398+0000 7f9f63f3f640 1 -- 172.21.15.45:0/1073201174 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6187+0+0 (secure 0 0 0) 0x563732536600 con 0x5637323c7400 2024-05-08T05:50:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.398+0000 7f9f6c40e280 4 main: JournalTool: creating IoCtx.. 2024-05-08T05:50:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.398+0000 7f9f6c40e280 4 main: Executing for rank 0 2024-05-08T05:50:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.398+0000 7f9f6c40e280 4 Filtering by inode '1099511627778' 2024-05-08T05:50:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.398+0000 7f9f6c40e280 1 --2- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x563731980400 0x5637323aa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.399+0000 7f9f6c40e280 1 -- 172.21.15.45:0/1073201174 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105) -- 0x563731980800 con 0x563731980400 2024-05-08T05:50:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.399+0000 7f9f69f4b640 1 --2- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x563731980000 0x5637323aa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.399+0000 7f9f6974a640 1 --2- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x563731980400 0x5637323aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.399+0000 7f9f69f4b640 1 --2- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x563731980000 0x5637323aa100 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x563732370cf0 tx=0x563731904840 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.399+0000 7f9f6974a640 1 --2- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x563731980400 0x5637323aa680 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.400+0000 7f9f6974a640 1 -- 172.21.15.45:0/1073201174 <== osd.6 v2:172.21.15.63:6800/2926759052 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x5637316f0000 con 0x563731980400 2024-05-08T05:50:07.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.400+0000 7f9f6c40e280 4 Pointer 400.00000000 is readable 2024-05-08T05:50:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.400+0000 7f9f6c40e280 4 JournalScanner::scan: reading header object '200.00000000' 2024-05-08T05:50:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.400+0000 7f9f6c40e280 1 -- 172.21.15.45:0/1073201174 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105) -- 0x563731980c00 con 0x563731980400 2024-05-08T05:50:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.400+0000 7f9f6974a640 1 -- 172.21.15.45:0/1073201174 <== osd.6 v2:172.21.15.63:6800/2926759052 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x5637316f0000 con 0x563731980400 2024-05-08T05:50:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.400+0000 7f9f6c40e280 10 Header 0x400000 0x407d21 0x40809d 2024-05-08T05:50:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.400+0000 7f9f6c40e280 10 Starting journal scan from offset 0x407d21 2024-05-08T05:50:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.400+0000 7f9f6c40e280 1 -- 172.21.15.45:0/1073201174 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 32033~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105) -- 0x563731981000 con 0x563731980400 2024-05-08T05:50:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.400+0000 7f9f6974a640 1 -- 172.21.15.45:0/1073201174 <== osd.6 v2:172.21.15.63:6800/2926759052 3 ==== osd_op_reply(3 200.00000001 [read 32033~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x5637316f0000 con 0x563731980400 2024-05-08T05:50:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.401+0000 7f9f6c40e280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-05-08T05:50:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.401+0000 7f9f6c40e280 10 Parsing data, 0x37c bytes available 2024-05-08T05:50:07.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.401+0000 7f9f6c40e280 10 Attempting decode at 0x407d21 2024-05-08T05:50:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.401+0000 7f9f6c40e280 10 Consumed 0x37c bytes 2024-05-08T05:50:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.401+0000 7f9f6c40e280 10 Valid entry at 0x407d21 2024-05-08T05:50:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.401+0000 7f9f6c40e280 1 --2- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x563731981400 0x5637323aac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.401+0000 7f9f6c40e280 1 -- 172.21.15.45:0/1073201174 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105) -- 0x5637318f8000 con 0x563731981400 2024-05-08T05:50:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.401+0000 7f9f6a74c640 1 --2- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x563731981400 0x5637323aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.401+0000 7f9f6a74c640 1 --2- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x563731981400 0x5637323aac00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.401+0000 7f9f6a74c640 1 -- 172.21.15.45:0/1073201174 <== osd.7 v2:172.21.15.63:6816/1163469437 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5637316f0500 con 0x563731981400 2024-05-08T05:50:07.411 INFO:teuthology.orchestra.run.smithi045.stdout:Events by type: 2024-05-08T05:50:07.411 INFO:teuthology.orchestra.run.smithi045.stdout:Errors: 0 2024-05-08T05:50:07.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 4 Reached end of journal objects 2024-05-08T05:50:07.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 4 Scanned objects, 0 missing, 1 valid 2024-05-08T05:50:07.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 4 Events scanned, 0 gaps 2024-05-08T05:50:07.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 4 Found 1 valid events 2024-05-08T05:50:07.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 4 Selected 0 events events for processing 2024-05-08T05:50:07.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 4 0 events 2024-05-08T05:50:07.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 -- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x563731980400 msgr2=0x5637323aa680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 --2- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x563731980400 0x5637323aa680 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 -- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x563731981400 msgr2=0x5637323aac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 --2- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x563731981400 0x5637323aac00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 -- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x563731980000 msgr2=0x5637323aa100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 --2- 172.21.15.45:0/1073201174 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x563731980000 0x5637323aa100 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x563732370cf0 tx=0x563731904840 comp rx=0 tx=0).stop 2024-05-08T05:50:07.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 -- 172.21.15.45:0/1073201174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637323c7400 msgr2=0x5637323a9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 --2- 172.21.15.45:0/1073201174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637323c7400 0x5637323a9080 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x5637324b6930 tx=0x563732382600 comp rx=0 tx=0).stop 2024-05-08T05:50:07.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f69f4b640 1 -- 172.21.15.45:0/1073201174 reap_dead start 2024-05-08T05:50:07.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f69f4b640 1 -- 172.21.15.45:0/1073201174 reap_dead start 2024-05-08T05:50:07.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 -- 172.21.15.45:0/1073201174 shutdown_connections 2024-05-08T05:50:07.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 -- 172.21.15.45:0/1073201174 >> 172.21.15.45:0/1073201174 conn(0x5637323c7000 msgr2=0x5637316dd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:07.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 -- 172.21.15.45:0/1073201174 shutdown_connections 2024-05-08T05:50:07.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 -- 172.21.15.45:0/1073201174 wait complete. 2024-05-08T05:50:07.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 -- 172.21.15.45:0/2860600294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637315e3c00 msgr2=0x5637323a8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 --2- 172.21.15.45:0/2860600294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637315e3c00 0x5637323a8b00 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x563732401800 tx=0x563732384120 comp rx=0 tx=0).stop 2024-05-08T05:50:07.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 -- 172.21.15.45:0/2860600294 shutdown_connections 2024-05-08T05:50:07.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 --2- 172.21.15.45:0/2860600294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5637315e3c00 0x5637323a8b00 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 --2- 172.21.15.45:0/2860600294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5637315e3800 0x5637323a8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 --2- 172.21.15.45:0/2860600294 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5637315e2c00 0x5637323a8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 -- 172.21.15.45:0/2860600294 >> 172.21.15.45:0/2860600294 conn(0x5637315e3000 msgr2=0x5637316dc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:07.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 -- 172.21.15.45:0/2860600294 shutdown_connections 2024-05-08T05:50:07.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.402+0000 7f9f6c40e280 1 -- 172.21.15.45:0/2860600294 wait complete. 2024-05-08T05:50:07.416 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.108789, result: Events by type: Errors: 0 2024-05-08T05:50:07.416 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-05-08T05:50:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.460+0000 7f3c90771f40 1 Processor -- start 2024-05-08T05:50:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.460+0000 7f3c90771f40 1 -- start start 2024-05-08T05:50:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.461+0000 7f3c90771f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128b5f75a0 0x56128b638d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.461+0000 7f3c90771f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128b5f7980 0x56128b639b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.461+0000 7f3c90771f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128b6c96f0 0x56128b6b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.461+0000 7f3c90771f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56128b623110 con 0x56128b5f7980 2024-05-08T05:50:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.461+0000 7f3c90771f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56128b59cf50 con 0x56128b5f75a0 2024-05-08T05:50:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.461+0000 7f3c90771f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56128b6233f0 con 0x56128b6c96f0 2024-05-08T05:50:07.463 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.461+0000 7f3c8da6c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128b5f7980 0x56128b639b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.461+0000 7f3c8da6c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128b5f7980 0x56128b639b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50580/0 (socket says 172.21.15.45:50580) 2024-05-08T05:50:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.461+0000 7f3c8da6c640 1 -- 172.21.15.45:0/2238727716 learned_addr learned my addr 172.21.15.45:0/2238727716 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.461+0000 7f3c8e26d640 1 --2- 172.21.15.45:0/2238727716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128b5f75a0 0x56128b638d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.461+0000 7f3c8da6c640 1 -- 172.21.15.45:0/2238727716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128b6c96f0 msgr2=0x56128b6b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.461+0000 7f3c8da6c640 1 --2- 172.21.15.45:0/2238727716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128b6c96f0 0x56128b6b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.461+0000 7f3c8da6c640 1 -- 172.21.15.45:0/2238727716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128b5f75a0 msgr2=0x56128b638d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.461+0000 7f3c8da6c640 1 --2- 172.21.15.45:0/2238727716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128b5f75a0 0x56128b638d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.461+0000 7f3c8da6c640 1 -- 172.21.15.45:0/2238727716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56128b6385c0 con 0x56128b5f7980 2024-05-08T05:50:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.461+0000 7f3c8da6c640 1 --2- 172.21.15.45:0/2238727716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128b5f7980 0x56128b639b70 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f3c84025d20 tx=0x7f3c84051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb150f33db172ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.462+0000 7f3c8ca6a640 1 -- 172.21.15.45:0/2238727716 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c84053880 con 0x56128b5f7980 2024-05-08T05:50:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.462+0000 7f3c8ca6a640 1 -- 172.21.15.45:0/2238727716 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c84053a40 con 0x56128b5f7980 2024-05-08T05:50:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.462+0000 7f3c8ca6a640 1 -- 172.21.15.45:0/2238727716 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c84053e30 con 0x56128b5f7980 2024-05-08T05:50:07.464 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.462+0000 7f3c90771f40 1 -- 172.21.15.45:0/2238727716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128b5f7980 msgr2=0x56128b639b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.462+0000 7f3c90771f40 1 --2- 172.21.15.45:0/2238727716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128b5f7980 0x56128b639b70 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f3c84025d20 tx=0x7f3c84051000 comp rx=0 tx=0).stop 2024-05-08T05:50:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.462+0000 7f3c90771f40 1 -- 172.21.15.45:0/2238727716 shutdown_connections 2024-05-08T05:50:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.462+0000 7f3c90771f40 1 --2- 172.21.15.45:0/2238727716 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128b6c96f0 0x56128b6b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.462+0000 7f3c90771f40 1 --2- 172.21.15.45:0/2238727716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128b5f7980 0x56128b639b70 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.462+0000 7f3c90771f40 1 --2- 172.21.15.45:0/2238727716 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128b5f75a0 0x56128b638d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.462+0000 7f3c90771f40 1 -- 172.21.15.45:0/2238727716 >> 172.21.15.45:0/2238727716 conn(0x56128b5f5060 msgr2=0x56128b624860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.462+0000 7f3c90771f40 1 -- 172.21.15.45:0/2238727716 shutdown_connections 2024-05-08T05:50:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.462+0000 7f3c90771f40 1 -- 172.21.15.45:0/2238727716 wait complete. 2024-05-08T05:50:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.463+0000 7f3c90771f40 1 Processor -- start 2024-05-08T05:50:07.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.463+0000 7f3c90771f40 1 -- start start 2024-05-08T05:50:07.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.463+0000 7f3c90771f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128b5f75a0 0x56128b623dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c90771f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128b5f7980 0x56128b756320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c90771f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128b6c96f0 0x56128b6b7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c90771f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56128b751470 con 0x56128b6c96f0 2024-05-08T05:50:07.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c90771f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56128b6388f0 con 0x56128b5f75a0 2024-05-08T05:50:07.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c90771f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56128b6b8520 con 0x56128b5f7980 2024-05-08T05:50:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c8ea6e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128b6c96f0 0x56128b6b7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c8ea6e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128b6c96f0 0x56128b6b7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50588/0 (socket says 172.21.15.45:50588) 2024-05-08T05:50:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c8ea6e640 1 -- 172.21.15.45:0/3433993046 learned_addr learned my addr 172.21.15.45:0/3433993046 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c8e26d640 1 --2- 172.21.15.45:0/3433993046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128b5f75a0 0x56128b623dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c8da6c640 1 --2- 172.21.15.45:0/3433993046 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128b5f7980 0x56128b756320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c8ea6e640 1 -- 172.21.15.45:0/3433993046 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128b5f7980 msgr2=0x56128b756320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c8ea6e640 1 --2- 172.21.15.45:0/3433993046 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56128b5f7980 0x56128b756320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c8ea6e640 1 -- 172.21.15.45:0/3433993046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128b5f75a0 msgr2=0x56128b623dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c8ea6e640 1 --2- 172.21.15.45:0/3433993046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128b5f75a0 0x56128b623dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c8ea6e640 1 -- 172.21.15.45:0/3433993046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56128b5f5690 con 0x56128b6c96f0 2024-05-08T05:50:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c8e26d640 1 --2- 172.21.15.45:0/3433993046 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56128b5f75a0 0x56128b623dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c8ea6e640 1 --2- 172.21.15.45:0/3433993046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128b6c96f0 0x56128b6b7360 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f3c880091d0 tx=0x7f3c88009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c7f7fe640 1 -- 172.21.15.45:0/3433993046 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88014070 con 0x56128b6c96f0 2024-05-08T05:50:07.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c7f7fe640 1 -- 172.21.15.45:0/3433993046 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c88010800 con 0x56128b6c96f0 2024-05-08T05:50:07.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c90771f40 1 -- 172.21.15.45:0/3433993046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56128b621160 con 0x56128b6c96f0 2024-05-08T05:50:07.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c7f7fe640 1 -- 172.21.15.45:0/3433993046 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88010bd0 con 0x56128b6c96f0 2024-05-08T05:50:07.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.464+0000 7f3c90771f40 1 -- 172.21.15.45:0/3433993046 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56128b624d20 con 0x56128b6c96f0 2024-05-08T05:50:07.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.466+0000 7f3c7f7fe640 1 -- 172.21.15.45:0/3433993046 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3c88020350 con 0x56128b6c96f0 2024-05-08T05:50:07.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.466+0000 7f3c7f7fe640 1 --2- 172.21.15.45:0/3433993046 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3c70074ce0 0x7f3c700771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.466+0000 7f3c7f7fe640 1 -- 172.21.15.45:0/3433993046 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6187+0+0 (secure 0 0 0) 0x7f3c880962c0 con 0x56128b6c96f0 2024-05-08T05:50:07.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.467+0000 7f3c8e26d640 1 --2- 172.21.15.45:0/3433993046 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3c70074ce0 0x7f3c700771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.467+0000 7f3c90771f40 1 --2- 172.21.15.45:0/3433993046 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x56128b624300 0x56128b752c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.467+0000 7f3c90771f40 1 -- 172.21.15.45:0/3433993046 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x56128b6b4590 con 0x56128b624300 2024-05-08T05:50:07.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.467+0000 7f3c8da6c640 1 --2- 172.21.15.45:0/3433993046 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x56128b624300 0x56128b752c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.467+0000 7f3c8e26d640 1 --2- 172.21.15.45:0/3433993046 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3c70074ce0 0x7f3c700771a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000d3d0 tx=0x7f3c800024d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.467+0000 7f3c8da6c640 1 --2- 172.21.15.45:0/3433993046 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x56128b624300 0x56128b752c20 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.468+0000 7f3c8da6c640 1 -- 172.21.15.45:0/3433993046 <== osd.1 v2:172.21.15.45:6824/1337179184 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v105'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3c84051070 con 0x56128b624300 2024-05-08T05:50:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.468+0000 7f3c90771f40 1 -- 172.21.15.45:0/3433993046 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x56128b624300 msgr2=0x56128b752c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.469+0000 7f3c90771f40 1 --2- 172.21.15.45:0/3433993046 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x56128b624300 0x56128b752c20 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.469+0000 7f3c90771f40 1 -- 172.21.15.45:0/3433993046 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3c70074ce0 msgr2=0x7f3c700771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.469+0000 7f3c90771f40 1 --2- 172.21.15.45:0/3433993046 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3c70074ce0 0x7f3c700771a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000d3d0 tx=0x7f3c800024d0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.469+0000 7f3c90771f40 1 -- 172.21.15.45:0/3433993046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128b6c96f0 msgr2=0x56128b6b7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.469+0000 7f3c90771f40 1 --2- 172.21.15.45:0/3433993046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56128b6c96f0 0x56128b6b7360 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f3c880091d0 tx=0x7f3c88009690 comp rx=0 tx=0).stop 2024-05-08T05:50:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.469+0000 7f3c8ea6e640 1 -- 172.21.15.45:0/3433993046 reap_dead start 2024-05-08T05:50:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.469+0000 7f3c90771f40 1 -- 172.21.15.45:0/3433993046 shutdown_connections 2024-05-08T05:50:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.469+0000 7f3c90771f40 1 -- 172.21.15.45:0/3433993046 >> 172.21.15.45:0/3433993046 conn(0x56128b5f5060 msgr2=0x56128b636e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.469+0000 7f3c90771f40 1 -- 172.21.15.45:0/3433993046 shutdown_connections 2024-05-08T05:50:07.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.469+0000 7f3c90771f40 1 -- 172.21.15.45:0/3433993046 wait complete. 2024-05-08T05:50:07.473 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-08T05:50:07.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.620+0000 7ff0830a9640 1 Processor -- start 2024-05-08T05:50:07.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.620+0000 7ff0830a9640 1 -- start start 2024-05-08T05:50:07.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.621+0000 7ff0830a9640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff07c06d620 0x7ff07c073800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.621+0000 7ff0830a9640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff07c06da00 0x7ff07c0af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.621+0000 7ff0830a9640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff07c0af800 con 0x7ff07c0b5ee0 2024-05-08T05:50:07.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.621+0000 7ff0830a9640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff07c0afaa0 con 0x7ff07c06d620 2024-05-08T05:50:07.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.621+0000 7ff0830a9640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff07c0b5a50 con 0x7ff07c06da00 2024-05-08T05:50:07.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.621+0000 7ff080e45640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff07c06d620 0x7ff07c073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.621+0000 7ff080e45640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff07c06d620 0x7ff07c073800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50598/0 (socket says 172.21.15.45:50598) 2024-05-08T05:50:07.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.621+0000 7ff080e45640 1 -- 172.21.15.45:0/2949628990 learned_addr learned my addr 172.21.15.45:0/2949628990 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:07.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.621+0000 7ff07bfff640 1 --2- 172.21.15.45:0/2949628990 >> v2:172.21.15.63:3300/0 conn(0x7ff07c06da00 0x7ff07c0af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.621+0000 7ff080e45640 1 -- 172.21.15.45:0/2949628990 >> v2:172.21.15.63:3300/0 conn(0x7ff07c06da00 msgr2=0x7ff07c0af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:07.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.621+0000 7ff080e45640 1 --2- 172.21.15.45:0/2949628990 >> v2:172.21.15.63:3300/0 conn(0x7ff07c06da00 0x7ff07c0af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.621+0000 7ff080e45640 1 -- 172.21.15.45:0/2949628990 >> v1:172.21.15.63:6789/0 conn(0x7ff07c0b5ee0 legacy=0x7ff07c06e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.621+0000 7ff080e45640 1 -- 172.21.15.45:0/2949628990 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07c0b5120 con 0x7ff07c06d620 2024-05-08T05:50:07.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.622+0000 7ff080e45640 1 --2- 172.21.15.45:0/2949628990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff07c06d620 0x7ff07c073800 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0aebe0 tx=0x7ff07002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880d3b2e192151e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.622+0000 7ff07b7fe640 1 -- 172.21.15.45:0/2949628990 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070037070 con 0x7ff07c06d620 2024-05-08T05:50:07.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.622+0000 7ff07b7fe640 1 -- 172.21.15.45:0/2949628990 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff070032440 con 0x7ff07c06d620 2024-05-08T05:50:07.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.622+0000 7ff07b7fe640 1 -- 172.21.15.45:0/2949628990 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070032810 con 0x7ff07c06d620 2024-05-08T05:50:07.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.623+0000 7ff0830a9640 1 -- 172.21.15.45:0/2949628990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff07c06d620 msgr2=0x7ff07c073800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.623+0000 7ff0830a9640 1 --2- 172.21.15.45:0/2949628990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff07c06d620 0x7ff07c073800 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0aebe0 tx=0x7ff07002a150 comp rx=0 tx=0).stop 2024-05-08T05:50:07.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.623+0000 7ff0830a9640 1 -- 172.21.15.45:0/2949628990 shutdown_connections 2024-05-08T05:50:07.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.623+0000 7ff0830a9640 1 --2- 172.21.15.45:0/2949628990 >> v2:172.21.15.63:3300/0 conn(0x7ff07c06da00 0x7ff07c0af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.623+0000 7ff0830a9640 1 --2- 172.21.15.45:0/2949628990 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff07c06d620 0x7ff07c073800 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.623+0000 7ff0830a9640 1 -- 172.21.15.45:0/2949628990 >> 172.21.15.45:0/2949628990 conn(0x7ff07c0728c0 msgr2=0x7ff07c072cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:07.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.623+0000 7ff0830a9640 1 -- 172.21.15.45:0/2949628990 shutdown_connections 2024-05-08T05:50:07.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.623+0000 7ff0830a9640 1 -- 172.21.15.45:0/2949628990 wait complete. 2024-05-08T05:50:07.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.624+0000 7ff0830a9640 1 Processor -- start 2024-05-08T05:50:07.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.624+0000 7ff0830a9640 1 -- start start 2024-05-08T05:50:07.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.625+0000 7ff0830a9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff07c06d620 0x7ff07c1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.625+0000 7ff0830a9640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff07c06da00 0x7ff07c158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.625+0000 7ff0830a9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff07c0b5ee0 0x7ff07c158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.625+0000 7ff0830a9640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c06f030 con 0x7ff07c06da00 2024-05-08T05:50:07.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.625+0000 7ff0830a9640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c0b56e0 con 0x7ff07c0b5ee0 2024-05-08T05:50:07.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.625+0000 7ff0830a9640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff07c06f1b0 con 0x7ff07c06d620 2024-05-08T05:50:07.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.625+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff07c06da00 0x7ff07c158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.625+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff07c06da00 0x7ff07c158660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50602/0 (socket says 172.21.15.45:50602) 2024-05-08T05:50:07.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.625+0000 7ff07bfff640 1 -- 172.21.15.45:0/3854400717 learned_addr learned my addr 172.21.15.45:0/3854400717 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:07.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.625+0000 7ff080e45640 1 --2- 172.21.15.45:0/3854400717 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff07c06d620 0x7ff07c1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.625+0000 7ff081646640 1 --2- 172.21.15.45:0/3854400717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff07c0b5ee0 0x7ff07c158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.625+0000 7ff07bfff640 1 -- 172.21.15.45:0/3854400717 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff07c06d620 msgr2=0x7ff07c1567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.625+0000 7ff07bfff640 1 --2- 172.21.15.45:0/3854400717 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff07c06d620 0x7ff07c1567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.625+0000 7ff07bfff640 1 -- 172.21.15.45:0/3854400717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff07c0b5ee0 msgr2=0x7ff07c158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:07.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.625+0000 7ff07bfff640 1 --2- 172.21.15.45:0/3854400717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff07c0b5ee0 0x7ff07c158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:07.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.625+0000 7ff07bfff640 1 -- 172.21.15.45:0/3854400717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07c156d20 con 0x7ff07c06da00 2024-05-08T05:50:07.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.626+0000 7ff07bfff640 1 --2- 172.21.15.45:0/3854400717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff07c06da00 0x7ff07c158660 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7ff06800e870 tx=0x7ff06800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.626+0000 7ff0797fa640 1 -- 172.21.15.45:0/3854400717 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068017070 con 0x7ff07c06da00 2024-05-08T05:50:07.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.626+0000 7ff0797fa640 1 -- 172.21.15.45:0/3854400717 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff06800c940 con 0x7ff07c06da00 2024-05-08T05:50:07.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.626+0000 7ff0830a9640 1 -- 172.21.15.45:0/3854400717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff07c1575c0 con 0x7ff07c06da00 2024-05-08T05:50:07.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.626+0000 7ff0797fa640 1 -- 172.21.15.45:0/3854400717 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06800cd30 con 0x7ff07c06da00 2024-05-08T05:50:07.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.626+0000 7ff0830a9640 1 -- 172.21.15.45:0/3854400717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff07c157b80 con 0x7ff07c06da00 2024-05-08T05:50:07.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.627+0000 7ff0830a9640 1 -- 172.21.15.45:0/3854400717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff07c06a440 con 0x7ff07c06da00 2024-05-08T05:50:07.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.628+0000 7ff0797fa640 1 -- 172.21.15.45:0/3854400717 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff06801a070 con 0x7ff07c06da00 2024-05-08T05:50:07.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.628+0000 7ff0797fa640 1 --2- 172.21.15.45:0/3854400717 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff0600762d0 0x7ff060078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:07.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.628+0000 7ff0797fa640 1 -- 172.21.15.45:0/3854400717 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6187+0+0 (secure 0 0 0) 0x7ff0680978e0 con 0x7ff07c06da00 2024-05-08T05:50:07.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.628+0000 7ff080e45640 1 --2- 172.21.15.45:0/3854400717 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff0600762d0 0x7ff060078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:07.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.629+0000 7ff080e45640 1 --2- 172.21.15.45:0/3854400717 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff0600762d0 0x7ff060078790 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff070034000 tx=0x7ff07002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:07.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.633+0000 7ff0797fa640 1 -- 172.21.15.45:0/3854400717 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff07c06a440 con 0x7ff07c06da00 2024-05-08T05:50:07.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:07.866+0000 7ff0830a9640 1 -- 172.21.15.45:0/3854400717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7ff07c0693f0 con 0x7ff07c06da00 2024-05-08T05:50:08.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.176+0000 7ff0797fa640 1 -- 172.21.15.45:0/3854400717 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v85) ==== 162+0+0 (secure 0 0 0) 0x7ff068060aa0 con 0x7ff07c06da00 2024-05-08T05:50:08.176 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-08T05:50:08.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.178+0000 7ff0830a9640 1 -- 172.21.15.45:0/3854400717 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff0600762d0 msgr2=0x7ff060078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:08.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.178+0000 7ff0830a9640 1 --2- 172.21.15.45:0/3854400717 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff0600762d0 0x7ff060078790 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff070034000 tx=0x7ff07002e000 comp rx=0 tx=0).stop 2024-05-08T05:50:08.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.178+0000 7ff0830a9640 1 -- 172.21.15.45:0/3854400717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff07c06da00 msgr2=0x7ff07c158660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:08.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.178+0000 7ff0830a9640 1 --2- 172.21.15.45:0/3854400717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff07c06da00 0x7ff07c158660 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7ff06800e870 tx=0x7ff06800ed30 comp rx=0 tx=0).stop 2024-05-08T05:50:08.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.179+0000 7ff0830a9640 1 -- 172.21.15.45:0/3854400717 shutdown_connections 2024-05-08T05:50:08.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.179+0000 7ff0830a9640 1 --2- 172.21.15.45:0/3854400717 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff0600762d0 0x7ff060078790 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:08.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.179+0000 7ff0830a9640 1 --2- 172.21.15.45:0/3854400717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff07c0b5ee0 0x7ff07c158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:08.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.179+0000 7ff0830a9640 1 --2- 172.21.15.45:0/3854400717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff07c06da00 0x7ff07c158660 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:08.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.179+0000 7ff0830a9640 1 --2- 172.21.15.45:0/3854400717 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff07c06d620 0x7ff07c1567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:08.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.179+0000 7ff0830a9640 1 -- 172.21.15.45:0/3854400717 >> 172.21.15.45:0/3854400717 conn(0x7ff07c0728c0 msgr2=0x7ff07c155510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:08.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.179+0000 7ff0830a9640 1 -- 172.21.15.45:0/3854400717 shutdown_connections 2024-05-08T05:50:08.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.179+0000 7ff0830a9640 1 -- 172.21.15.45:0/3854400717 wait complete. 2024-05-08T05:50:08.192 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:50:08.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.299+0000 7f6ea04da640 1 Processor -- start 2024-05-08T05:50:08.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.299+0000 7f6ea04da640 1 -- start start 2024-05-08T05:50:08.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.300+0000 7f6ea04da640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6e9815f420 0x7f6e980d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:08.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.300+0000 7f6ea04da640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f6e980d3ca0 0x7f6e980d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:08.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.300+0000 7f6ea04da640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e981630b0 con 0x7f6e980d1eb0 2024-05-08T05:50:08.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.300+0000 7f6ea04da640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f6e980d2550 con 0x7f6e9815f420 2024-05-08T05:50:08.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.300+0000 7f6ea04da640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f6e980d2c80 con 0x7f6e980d3ca0 2024-05-08T05:50:08.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.300+0000 7f6e9e276640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6e9815f420 0x7f6e980d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:08.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.300+0000 7f6e9e276640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f6e9815f420 0x7f6e980d3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50604/0 (socket says 172.21.15.45:50604) 2024-05-08T05:50:08.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.300+0000 7f6e9e276640 1 -- 172.21.15.45:0/3653862587 learned_addr learned my addr 172.21.15.45:0/3653862587 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:08.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.301+0000 7f6e9da75640 1 --2- 172.21.15.45:0/3653862587 >> v2:172.21.15.63:3300/0 conn(0x7f6e980d3ca0 0x7f6e980d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:08.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.301+0000 7f6e9e276640 1 -- 172.21.15.45:0/3653862587 >> v2:172.21.15.63:3300/0 conn(0x7f6e980d3ca0 msgr2=0x7f6e980d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:08.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.301+0000 7f6e9e276640 1 --2- 172.21.15.45:0/3653862587 >> v2:172.21.15.63:3300/0 conn(0x7f6e980d3ca0 0x7f6e980d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:08.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.301+0000 7f6e9e276640 1 -- 172.21.15.45:0/3653862587 >> v1:172.21.15.63:6789/0 conn(0x7f6e980d1eb0 legacy=0x7f6e980d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:08.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.301+0000 7f6e9e276640 1 -- 172.21.15.45:0/3653862587 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e980d2f60 con 0x7f6e9815f420 2024-05-08T05:50:08.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.301+0000 7f6e9da75640 1 --2- 172.21.15.45:0/3653862587 >> v2:172.21.15.63:3300/0 conn(0x7f6e980d3ca0 0x7f6e980d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:08.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.301+0000 7f6e9e276640 1 --2- 172.21.15.45:0/3653862587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e9815f420 0x7f6e980d3760 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f6e9808f380 tx=0x7f6e9402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19143a1299bb2de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.302+0000 7f6e9d274640 1 -- 172.21.15.45:0/3653862587 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e94037070 con 0x7f6e9815f420 2024-05-08T05:50:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.302+0000 7f6e9d274640 1 -- 172.21.15.45:0/3653862587 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e94032440 con 0x7f6e9815f420 2024-05-08T05:50:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.302+0000 7f6e9d274640 1 -- 172.21.15.45:0/3653862587 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e94032810 con 0x7f6e9815f420 2024-05-08T05:50:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.302+0000 7f6ea04da640 1 -- 172.21.15.45:0/3653862587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e9815f420 msgr2=0x7f6e980d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.302+0000 7f6ea04da640 1 --2- 172.21.15.45:0/3653862587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e9815f420 0x7f6e980d3760 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f6e9808f380 tx=0x7f6e9402a1e0 comp rx=0 tx=0).stop 2024-05-08T05:50:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.302+0000 7f6ea04da640 1 -- 172.21.15.45:0/3653862587 shutdown_connections 2024-05-08T05:50:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.302+0000 7f6ea04da640 1 --2- 172.21.15.45:0/3653862587 >> v2:172.21.15.63:3300/0 conn(0x7f6e980d3ca0 0x7f6e980d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.302+0000 7f6ea04da640 1 --2- 172.21.15.45:0/3653862587 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e9815f420 0x7f6e980d3760 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.302+0000 7f6ea04da640 1 -- 172.21.15.45:0/3653862587 >> 172.21.15.45:0/3653862587 conn(0x7f6e9808eac0 msgr2=0x7f6e9809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:08.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.302+0000 7f6ea04da640 1 -- 172.21.15.45:0/3653862587 shutdown_connections 2024-05-08T05:50:08.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.302+0000 7f6ea04da640 1 -- 172.21.15.45:0/3653862587 wait complete. 2024-05-08T05:50:08.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.303+0000 7f6ea04da640 1 Processor -- start 2024-05-08T05:50:08.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.304+0000 7f6ea04da640 1 -- start start 2024-05-08T05:50:08.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.304+0000 7f6ea04da640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e980d1eb0 0x7f6e980cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:08.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.304+0000 7f6ea04da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e98160270 0x7f6e981f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:08.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.304+0000 7f6ea04da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e980b5720 0x7f6e980b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:08.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.304+0000 7f6ea04da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6e980d53d0 con 0x7f6e980b5720 2024-05-08T05:50:08.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.304+0000 7f6ea04da640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6e980d2830 con 0x7f6e980d1eb0 2024-05-08T05:50:08.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.304+0000 7f6ea04da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6e980d5550 con 0x7f6e98160270 2024-05-08T05:50:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.304+0000 7f6e9ea77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e980b5720 0x7f6e980b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.304+0000 7f6e9ea77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e980b5720 0x7f6e980b5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50618/0 (socket says 172.21.15.45:50618) 2024-05-08T05:50:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.304+0000 7f6e9ea77640 1 -- 172.21.15.45:0/2938108215 learned_addr learned my addr 172.21.15.45:0/2938108215 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.304+0000 7f6e9e276640 1 --2- 172.21.15.45:0/2938108215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e980d1eb0 0x7f6e980cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.304+0000 7f6e9da75640 1 --2- 172.21.15.45:0/2938108215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e98160270 0x7f6e981f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.305+0000 7f6e9ea77640 1 -- 172.21.15.45:0/2938108215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e98160270 msgr2=0x7f6e981f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.305+0000 7f6e9ea77640 1 --2- 172.21.15.45:0/2938108215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e98160270 0x7f6e981f4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.305+0000 7f6e9ea77640 1 -- 172.21.15.45:0/2938108215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e980d1eb0 msgr2=0x7f6e980cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.305+0000 7f6e9ea77640 1 --2- 172.21.15.45:0/2938108215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e980d1eb0 0x7f6e980cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.305+0000 7f6e9ea77640 1 -- 172.21.15.45:0/2938108215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e98161a80 con 0x7f6e980b5720 2024-05-08T05:50:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.305+0000 7f6e9da75640 1 --2- 172.21.15.45:0/2938108215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e98160270 0x7f6e981f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.305+0000 7f6e9ea77640 1 --2- 172.21.15.45:0/2938108215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e980b5720 0x7f6e980b5ba0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000c7b0 tx=0x7f6e9000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.305+0000 7f6e8effd640 1 -- 172.21.15.45:0/2938108215 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90004b60 con 0x7f6e980b5720 2024-05-08T05:50:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.305+0000 7f6e8effd640 1 -- 172.21.15.45:0/2938108215 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e90004d00 con 0x7f6e980b5720 2024-05-08T05:50:08.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.305+0000 7f6ea04da640 1 -- 172.21.15.45:0/2938108215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e981609e0 con 0x7f6e980b5720 2024-05-08T05:50:08.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.305+0000 7f6e8effd640 1 -- 172.21.15.45:0/2938108215 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9000aa90 con 0x7f6e980b5720 2024-05-08T05:50:08.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.305+0000 7f6ea04da640 1 -- 172.21.15.45:0/2938108215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e98160fa0 con 0x7f6e980b5720 2024-05-08T05:50:08.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.306+0000 7f6ea04da640 1 -- 172.21.15.45:0/2938108215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e980693f0 con 0x7f6e980b5720 2024-05-08T05:50:08.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.307+0000 7f6e8effd640 1 -- 172.21.15.45:0/2938108215 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6e9000ac30 con 0x7f6e980b5720 2024-05-08T05:50:08.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.307+0000 7f6e8effd640 1 --2- 172.21.15.45:0/2938108215 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6e7c076300 0x7f6e7c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:08.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.307+0000 7f6e8effd640 1 -- 172.21.15.45:0/2938108215 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6187+0+0 (secure 0 0 0) 0x7f6e90097b80 con 0x7f6e980b5720 2024-05-08T05:50:08.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.308+0000 7f6e9e276640 1 --2- 172.21.15.45:0/2938108215 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6e7c076300 0x7f6e7c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:08.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.308+0000 7f6e9e276640 1 --2- 172.21.15.45:0/2938108215 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6e7c076300 0x7f6e7c0787c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6e94034000 tx=0x7f6e9402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:08.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.312+0000 7f6e8effd640 1 -- 172.21.15.45:0/2938108215 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6e90061390 con 0x7f6e980b5720 2024-05-08T05:50:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.542+0000 7f6ea04da640 1 -- 172.21.15.45:0/2938108215 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e980b4c50 con 0x7f6e980b5720 2024-05-08T05:50:08.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.542+0000 7f6e8effd640 1 -- 172.21.15.45:0/2938108215 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5178 (secure 0 0 0) 0x7f6e90060c90 con 0x7f6e980b5720 2024-05-08T05:50:08.542 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:50:08.543 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3692881261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3692881261},{"type":"v1","addr":"172.21.15.63:6836","nonce":3692881261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7497,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2916399818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2916399818},{"type":"v1","addr":"172.21.15.45:6833","nonce":2916399818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7515,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2978250004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":2978250004},{"type":"v1","addr":"172.21.15.63:6837","nonce":2978250004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:49:19.587326+0000","modified":"2024-05-08T05:50:08.175864+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7806},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7806":{"gid":7806,"name":"a","rank":0,"incarnation":85,"state":"up:replay","state_seq":1,"addr":"172.21.15.45:6835/2729952281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2729952281},{"type":"v1","addr":"172.21.15.45:6835","nonce":2729952281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-08T05:50:08.543 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 85 2024-05-08T05:50:08.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.544+0000 7f6ea04da640 1 -- 172.21.15.45:0/2938108215 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6e7c076300 msgr2=0x7f6e7c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.544+0000 7f6ea04da640 1 --2- 172.21.15.45:0/2938108215 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6e7c076300 0x7f6e7c0787c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6e94034000 tx=0x7f6e9402e000 comp rx=0 tx=0).stop 2024-05-08T05:50:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.544+0000 7f6ea04da640 1 -- 172.21.15.45:0/2938108215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e980b5720 msgr2=0x7f6e980b5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.544+0000 7f6ea04da640 1 --2- 172.21.15.45:0/2938108215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e980b5720 0x7f6e980b5ba0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f6e9000c7b0 tx=0x7f6e9000cc70 comp rx=0 tx=0).stop 2024-05-08T05:50:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.544+0000 7f6ea04da640 1 -- 172.21.15.45:0/2938108215 shutdown_connections 2024-05-08T05:50:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.544+0000 7f6ea04da640 1 --2- 172.21.15.45:0/2938108215 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6e7c076300 0x7f6e7c0787c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.544+0000 7f6ea04da640 1 --2- 172.21.15.45:0/2938108215 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6e98160270 0x7f6e981f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.544+0000 7f6ea04da640 1 --2- 172.21.15.45:0/2938108215 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e980b5720 0x7f6e980b5ba0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.544+0000 7f6ea04da640 1 --2- 172.21.15.45:0/2938108215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e980d1eb0 0x7f6e980cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.544+0000 7f6ea04da640 1 -- 172.21.15.45:0/2938108215 >> 172.21.15.45:0/2938108215 conn(0x7f6e9808eac0 msgr2=0x7f6e980d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.544+0000 7f6ea04da640 1 -- 172.21.15.45:0/2938108215 shutdown_connections 2024-05-08T05:50:08.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:08.544+0000 7f6ea04da640 1 -- 172.21.15.45:0/2938108215 wait complete. 2024-05-08T05:50:08.958 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:49:19.587326+0000', 'modified': '2024-05-08T05:50:08.175864+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 105, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7806}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7806': {'gid': 7806, 'name': 'a', 'rank': 0, 'incarnation': 85, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.45:6835/2729952281', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 2729952281}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 2729952281}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-08T05:50:08.958 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7806:up:replay 2024-05-08T05:50:09.959 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:50:10.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.076+0000 7f65e1445640 1 Processor -- start 2024-05-08T05:50:10.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.076+0000 7f65e1445640 1 -- start start 2024-05-08T05:50:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.077+0000 7f65e1445640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65dc0d1970 0x7f65dc0d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.077+0000 7f65e1445640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f65dc0d4cc0 0x7f65dc0d5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.077+0000 7f65e1445640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65dc1630b0 con 0x7f65dc15f420 2024-05-08T05:50:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.077+0000 7f65e1445640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f65dc0d2330 con 0x7f65dc0d4cc0 2024-05-08T05:50:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.077+0000 7f65e1445640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f65dc0d3e70 con 0x7f65dc0d1970 2024-05-08T05:50:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.077+0000 7f65db7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f65dc0d4cc0 0x7f65dc0d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.077+0000 7f65daffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f65dc15f420 0x7f65dc0d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:48182/0 (socket says 172.21.15.45:48182) 2024-05-08T05:50:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.077+0000 7f65daffd640 1 -- 172.21.15.45:0/529245123 learned_addr learned my addr 172.21.15.45:0/529245123 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:50:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.078+0000 7f65db7fe640 1 -- 172.21.15.45:0/529245123 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65dc0d1970 msgr2=0x7f65dc0d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:10.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.078+0000 7f65da7fc640 1 --2- 172.21.15.45:0/529245123 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65dc0d1970 0x7f65dc0d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.078+0000 7f65db7fe640 1 --2- 172.21.15.45:0/529245123 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65dc0d1970 0x7f65dc0d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.078+0000 7f65db7fe640 1 -- 172.21.15.45:0/529245123 >> v1:172.21.15.45:6789/0 conn(0x7f65dc15f420 legacy=0x7f65dc0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.078+0000 7f65db7fe640 1 -- 172.21.15.45:0/529245123 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65dc0d5670 con 0x7f65dc0d4cc0 2024-05-08T05:50:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.078+0000 7f65da7fc640 1 --2- 172.21.15.45:0/529245123 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65dc0d1970 0x7f65dc0d1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.078+0000 7f65db7fe640 1 --2- 172.21.15.45:0/529245123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65dc0d4cc0 0x7f65dc0d5130 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f65d40097a0 tx=0x7f65d402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcf496b247e5264 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.079+0000 7f65d9ffb640 1 -- 172.21.15.45:0/529245123 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d4037070 con 0x7f65dc0d4cc0 2024-05-08T05:50:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.079+0000 7f65d9ffb640 1 -- 172.21.15.45:0/529245123 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65d4032440 con 0x7f65dc0d4cc0 2024-05-08T05:50:10.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.079+0000 7f65d9ffb640 1 -- 172.21.15.45:0/529245123 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d4032810 con 0x7f65dc0d4cc0 2024-05-08T05:50:10.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.079+0000 7f65e1445640 1 -- 172.21.15.45:0/529245123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65dc0d4cc0 msgr2=0x7f65dc0d5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:10.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.079+0000 7f65e1445640 1 --2- 172.21.15.45:0/529245123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65dc0d4cc0 0x7f65dc0d5130 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f65d40097a0 tx=0x7f65d402a180 comp rx=0 tx=0).stop 2024-05-08T05:50:10.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.079+0000 7f65e1445640 1 -- 172.21.15.45:0/529245123 shutdown_connections 2024-05-08T05:50:10.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.079+0000 7f65e1445640 1 --2- 172.21.15.45:0/529245123 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65dc0d4cc0 0x7f65dc0d5130 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:10.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.079+0000 7f65e1445640 1 --2- 172.21.15.45:0/529245123 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65dc0d1970 0x7f65dc0d1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:10.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.079+0000 7f65e1445640 1 -- 172.21.15.45:0/529245123 >> 172.21.15.45:0/529245123 conn(0x7f65dc08eac0 msgr2=0x7f65dc09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:10.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.079+0000 7f65e1445640 1 -- 172.21.15.45:0/529245123 shutdown_connections 2024-05-08T05:50:10.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.079+0000 7f65e1445640 1 -- 172.21.15.45:0/529245123 wait complete. 2024-05-08T05:50:10.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.081+0000 7f65e1445640 1 Processor -- start 2024-05-08T05:50:10.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.081+0000 7f65e1445640 1 -- start start 2024-05-08T05:50:10.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65e1445640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65dc0d1970 0x7f65dc0b80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65e1445640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65dc0d4cc0 0x7f65dc1f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65e1445640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65dc0ca990 0x7f65dc0b9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65e1445640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc0d2f60 con 0x7f65dc0d1970 2024-05-08T05:50:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65e1445640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc0d2c80 con 0x7f65dc0ca990 2024-05-08T05:50:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65e1445640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f65dc0d3240 con 0x7f65dc0d4cc0 2024-05-08T05:50:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65daffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65dc0d1970 0x7f65dc0b80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65daffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65dc0d1970 0x7f65dc0b80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50636/0 (socket says 172.21.15.45:50636) 2024-05-08T05:50:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65daffd640 1 -- 172.21.15.45:0/1850477653 learned_addr learned my addr 172.21.15.45:0/1850477653 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65da7fc640 1 --2- 172.21.15.45:0/1850477653 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65dc0d4cc0 0x7f65dc1f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65db7fe640 1 --2- 172.21.15.45:0/1850477653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65dc0ca990 0x7f65dc0b9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65daffd640 1 -- 172.21.15.45:0/1850477653 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65dc0d4cc0 msgr2=0x7f65dc1f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65daffd640 1 --2- 172.21.15.45:0/1850477653 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65dc0d4cc0 0x7f65dc1f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65daffd640 1 -- 172.21.15.45:0/1850477653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65dc0ca990 msgr2=0x7f65dc0b9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:10.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65daffd640 1 --2- 172.21.15.45:0/1850477653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65dc0ca990 0x7f65dc0b9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:10.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65daffd640 1 -- 172.21.15.45:0/1850477653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65dc0b87d0 con 0x7f65dc0d1970 2024-05-08T05:50:10.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65da7fc640 1 --2- 172.21.15.45:0/1850477653 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65dc0d4cc0 0x7f65dc1f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:10.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65db7fe640 1 --2- 172.21.15.45:0/1850477653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65dc0ca990 0x7f65dc0b9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:10.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65daffd640 1 --2- 172.21.15.45:0/1850477653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65dc0d1970 0x7f65dc0b80d0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00ca80 tx=0x7f65cc00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:10.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65c37fe640 1 -- 172.21.15.45:0/1850477653 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc017070 con 0x7f65dc0d1970 2024-05-08T05:50:10.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65c37fe640 1 -- 172.21.15.45:0/1850477653 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65cc00ac30 con 0x7f65dc0d1970 2024-05-08T05:50:10.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65e1445640 1 -- 172.21.15.45:0/1850477653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65dc0b8ab0 con 0x7f65dc0d1970 2024-05-08T05:50:10.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.082+0000 7f65c37fe640 1 -- 172.21.15.45:0/1850477653 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc012640 con 0x7f65dc0d1970 2024-05-08T05:50:10.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.083+0000 7f65e1445640 1 -- 172.21.15.45:0/1850477653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65dc0b9070 con 0x7f65dc0d1970 2024-05-08T05:50:10.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.084+0000 7f65e1445640 1 -- 172.21.15.45:0/1850477653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65dc0693f0 con 0x7f65dc0d1970 2024-05-08T05:50:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.084+0000 7f65c37fe640 1 -- 172.21.15.45:0/1850477653 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f65cc01a070 con 0x7f65dc0d1970 2024-05-08T05:50:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.085+0000 7f65c37fe640 1 --2- 172.21.15.45:0/1850477653 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f65c80762d0 0x7f65c8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.085+0000 7f65c37fe640 1 -- 172.21.15.45:0/1850477653 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6187+0+0 (secure 0 0 0) 0x7f65cc097480 con 0x7f65dc0d1970 2024-05-08T05:50:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.085+0000 7f65da7fc640 1 --2- 172.21.15.45:0/1850477653 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f65c80762d0 0x7f65c8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:10.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.086+0000 7f65da7fc640 1 --2- 172.21.15.45:0/1850477653 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f65c80762d0 0x7f65c8078790 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f65d0006fd0 tx=0x7f65d0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:10.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.090+0000 7f65c37fe640 1 -- 172.21.15.45:0/1850477653 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f65dc1630b0 con 0x7f65dc0d1970 2024-05-08T05:50:10.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.321+0000 7f65e1445640 1 -- 172.21.15.45:0/1850477653 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65dc0b7eb0 con 0x7f65dc0d1970 2024-05-08T05:50:10.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.321+0000 7f65c37fe640 1 -- 172.21.15.45:0/1850477653 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5178 (secure 0 0 0) 0x7f65cc00ae20 con 0x7f65dc0d1970 2024-05-08T05:50:10.322 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:50:10.322 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":87,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3692881261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3692881261},{"type":"v1","addr":"172.21.15.63:6836","nonce":3692881261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7497,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2916399818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2916399818},{"type":"v1","addr":"172.21.15.45:6833","nonce":2916399818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7515,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2978250004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":2978250004},{"type":"v1","addr":"172.21.15.63:6837","nonce":2978250004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:49:19.587326+0000","modified":"2024-05-08T05:50:09.181580+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7806},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7806":{"gid":7806,"name":"a","rank":0,"incarnation":85,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.45:6835/2729952281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2729952281},{"type":"v1","addr":"172.21.15.45:6835","nonce":2729952281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-08T05:50:10.322 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 87 2024-05-08T05:50:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.323+0000 7f65e1445640 1 -- 172.21.15.45:0/1850477653 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f65c80762d0 msgr2=0x7f65c8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.323+0000 7f65e1445640 1 --2- 172.21.15.45:0/1850477653 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f65c80762d0 0x7f65c8078790 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f65d0006fd0 tx=0x7f65d0008040 comp rx=0 tx=0).stop 2024-05-08T05:50:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.323+0000 7f65e1445640 1 -- 172.21.15.45:0/1850477653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65dc0d1970 msgr2=0x7f65dc0b80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.323+0000 7f65e1445640 1 --2- 172.21.15.45:0/1850477653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65dc0d1970 0x7f65dc0b80d0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00ca80 tx=0x7f65cc00cf40 comp rx=0 tx=0).stop 2024-05-08T05:50:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.323+0000 7f65e1445640 1 -- 172.21.15.45:0/1850477653 shutdown_connections 2024-05-08T05:50:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.323+0000 7f65e1445640 1 --2- 172.21.15.45:0/1850477653 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f65c80762d0 0x7f65c8078790 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.323+0000 7f65e1445640 1 --2- 172.21.15.45:0/1850477653 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f65dc0d4cc0 0x7f65dc1f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.323+0000 7f65e1445640 1 --2- 172.21.15.45:0/1850477653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65dc0ca990 0x7f65dc0b9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.323+0000 7f65e1445640 1 --2- 172.21.15.45:0/1850477653 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65dc0d1970 0x7f65dc0b80d0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.323+0000 7f65e1445640 1 -- 172.21.15.45:0/1850477653 >> 172.21.15.45:0/1850477653 conn(0x7f65dc08eac0 msgr2=0x7f65dc0b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.323+0000 7f65e1445640 1 -- 172.21.15.45:0/1850477653 shutdown_connections 2024-05-08T05:50:10.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:10.323+0000 7f65e1445640 1 -- 172.21.15.45:0/1850477653 wait complete. 2024-05-08T05:50:10.738 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:49:19.587326+0000', 'modified': '2024-05-08T05:50:09.181580+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 105, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7806}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7806': {'gid': 7806, 'name': 'a', 'rank': 0, 'incarnation': 85, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.45:6835/2729952281', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 2729952281}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 2729952281}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-08T05:50:10.738 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7806:up:rejoin 2024-05-08T05:50:11.739 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:50:11.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.857+0000 7f3f4d71b640 1 Processor -- start 2024-05-08T05:50:11.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.857+0000 7f3f4d71b640 1 -- start start 2024-05-08T05:50:11.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.858+0000 7f3f4d71b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3f481629a0 0x7f3f480d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:11.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.858+0000 7f3f4d71b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f480d0cb0 0x7f3f480d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:11.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.858+0000 7f3f4d71b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f480d3330 con 0x7f3f480d2dc0 2024-05-08T05:50:11.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.858+0000 7f3f4d71b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3f480d2170 con 0x7f3f481629a0 2024-05-08T05:50:11.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.858+0000 7f3f4d71b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3f480d2450 con 0x7f3f480d0cb0 2024-05-08T05:50:11.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.858+0000 7f3f467fc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3f480d2dc0 0x7f3f480c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40676/0 (socket says 172.21.15.45:40676) 2024-05-08T05:50:11.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.858+0000 7f3f467fc640 1 -- 172.21.15.45:0/1243255095 learned_addr learned my addr 172.21.15.45:0/1243255095 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:50:11.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.858+0000 7f3f46ffd640 1 --2- 172.21.15.45:0/1243255095 >> v2:172.21.15.63:3300/0 conn(0x7f3f481629a0 0x7f3f480d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:11.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.858+0000 7f3f477fe640 1 --2- 172.21.15.45:0/1243255095 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f480d0cb0 0x7f3f480d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:11.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.858+0000 7f3f45ffb640 1 -- 172.21.15.45:0/1243255095 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3051778209 0 0) 0x7f3f480d3330 con 0x7f3f480d2dc0 2024-05-08T05:50:11.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.858+0000 7f3f45ffb640 1 -- 172.21.15.45:0/1243255095 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f30003520 con 0x7f3f480d2dc0 2024-05-08T05:50:11.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f45ffb640 1 -- 172.21.15.45:0/1243255095 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4004088466 0 0) 0x7f3f30003520 con 0x7f3f480d2dc0 2024-05-08T05:50:11.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f45ffb640 1 -- 172.21.15.45:0/1243255095 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f480d0cb0 msgr2=0x7f3f480d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:11.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f45ffb640 1 --2- 172.21.15.45:0/1243255095 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f480d0cb0 0x7f3f480d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:11.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f45ffb640 1 -- 172.21.15.45:0/1243255095 >> v2:172.21.15.63:3300/0 conn(0x7f3f481629a0 msgr2=0x7f3f480d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:11.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f45ffb640 1 --2- 172.21.15.45:0/1243255095 >> v2:172.21.15.63:3300/0 conn(0x7f3f481629a0 0x7f3f480d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:11.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f45ffb640 1 -- 172.21.15.45:0/1243255095 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f480d2730 con 0x7f3f480d2dc0 2024-05-08T05:50:11.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f45ffb640 1 -- 172.21.15.45:0/1243255095 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f3f3c0016f0 con 0x7f3f480d2dc0 2024-05-08T05:50:11.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f45ffb640 1 -- 172.21.15.45:0/1243255095 >> v1:172.21.15.45:6789/0 conn(0x7f3f480d2dc0 legacy=0x7f3f480c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:11.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f45ffb640 1 --2- 172.21.15.45:0/1243255095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f30003a00 0x7f3f30003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:11.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f45ffb640 1 --2- 172.21.15.45:0/1243255095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f30004800 0x7f3f30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:11.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f45ffb640 1 --2- 172.21.15.45:0/1243255095 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f480d0cb0 0x7f3f300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:11.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f45ffb640 1 -- 172.21.15.45:0/1243255095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3f480d2170 con 0x7f3f30004800 2024-05-08T05:50:11.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f45ffb640 1 -- 172.21.15.45:0/1243255095 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3f480d2450 con 0x7f3f30003a00 2024-05-08T05:50:11.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f45ffb640 1 -- 172.21.15.45:0/1243255095 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3f3000baf0 con 0x7f3f480d0cb0 2024-05-08T05:50:11.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f467fc640 1 --2- 172.21.15.45:0/1243255095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f30004800 0x7f3f30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:11.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f46ffd640 1 --2- 172.21.15.45:0/1243255095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f30003a00 0x7f3f30003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:11.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f477fe640 1 --2- 172.21.15.45:0/1243255095 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f480d0cb0 0x7f3f300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:11.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f467fc640 1 -- 172.21.15.45:0/1243255095 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f480d0cb0 msgr2=0x7f3f300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:11.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f467fc640 1 --2- 172.21.15.45:0/1243255095 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f480d0cb0 0x7f3f300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:11.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f467fc640 1 -- 172.21.15.45:0/1243255095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f30003a00 msgr2=0x7f3f30003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:11.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f467fc640 1 --2- 172.21.15.45:0/1243255095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f30003a00 0x7f3f30003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:11.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.859+0000 7f3f467fc640 1 -- 172.21.15.45:0/1243255095 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f30003520 con 0x7f3f30004800 2024-05-08T05:50:11.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.860+0000 7f3f467fc640 1 --2- 172.21.15.45:0/1243255095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f30004800 0x7f3f30004c80 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c007fd0 tx=0x7f3f30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66cce1b68ebb53f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:11.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.860+0000 7f3f45ffb640 1 -- 172.21.15.45:0/1243255095 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c007d50 con 0x7f3f30004800 2024-05-08T05:50:11.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.860+0000 7f3f45ffb640 1 -- 172.21.15.45:0/1243255095 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f3c005af0 con 0x7f3f30004800 2024-05-08T05:50:11.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.860+0000 7f3f45ffb640 1 -- 172.21.15.45:0/1243255095 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c007070 con 0x7f3f30004800 2024-05-08T05:50:11.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.861+0000 7f3f4d71b640 1 -- 172.21.15.45:0/1243255095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f30004800 msgr2=0x7f3f30004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:11.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.861+0000 7f3f4d71b640 1 --2- 172.21.15.45:0/1243255095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f30004800 0x7f3f30004c80 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c007fd0 tx=0x7f3f30005830 comp rx=0 tx=0).stop 2024-05-08T05:50:11.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.861+0000 7f3f477fe640 1 -- 172.21.15.45:0/1243255095 reap_dead start 2024-05-08T05:50:11.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.861+0000 7f3f4d71b640 1 -- 172.21.15.45:0/1243255095 shutdown_connections 2024-05-08T05:50:11.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.861+0000 7f3f4d71b640 1 -- 172.21.15.45:0/1243255095 >> 172.21.15.45:0/1243255095 conn(0x7f3f480d4fe0 msgr2=0x7f3f480d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:11.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.861+0000 7f3f4d71b640 1 -- 172.21.15.45:0/1243255095 shutdown_connections 2024-05-08T05:50:11.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.861+0000 7f3f4d71b640 1 -- 172.21.15.45:0/1243255095 wait complete. 2024-05-08T05:50:11.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.862+0000 7f3f4d71b640 1 Processor -- start 2024-05-08T05:50:11.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.862+0000 7f3f4d71b640 1 -- start start 2024-05-08T05:50:11.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.863+0000 7f3f4d71b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f30004800 0x7f3f4814d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:11.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.863+0000 7f3f4d71b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48157390 0x7f3f481577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:11.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.863+0000 7f3f4d71b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f480cdf40 0x7f3f480ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:11.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.863+0000 7f3f4d71b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3f480cf440 con 0x7f3f48157390 2024-05-08T05:50:11.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.863+0000 7f3f4d71b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3f480cf2c0 con 0x7f3f480cdf40 2024-05-08T05:50:11.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.863+0000 7f3f4d71b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3f480d3b00 con 0x7f3f30004800 2024-05-08T05:50:11.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.863+0000 7f3f467fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48157390 0x7f3f481577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:11.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.863+0000 7f3f467fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48157390 0x7f3f481577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59866/0 (socket says 172.21.15.45:59866) 2024-05-08T05:50:11.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.863+0000 7f3f467fc640 1 -- 172.21.15.45:0/251000040 learned_addr learned my addr 172.21.15.45:0/251000040 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:11.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.863+0000 7f3f477fe640 1 --2- 172.21.15.45:0/251000040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f480cdf40 0x7f3f480ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:11.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.863+0000 7f3f46ffd640 1 --2- 172.21.15.45:0/251000040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f30004800 0x7f3f4814d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:11.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.863+0000 7f3f467fc640 1 -- 172.21.15.45:0/251000040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f30004800 msgr2=0x7f3f4814d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:11.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.863+0000 7f3f467fc640 1 --2- 172.21.15.45:0/251000040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f30004800 0x7f3f4814d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:11.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.863+0000 7f3f467fc640 1 -- 172.21.15.45:0/251000040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f480cdf40 msgr2=0x7f3f480ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:11.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.863+0000 7f3f467fc640 1 --2- 172.21.15.45:0/251000040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f480cdf40 0x7f3f480ce3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:11.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.863+0000 7f3f467fc640 1 -- 172.21.15.45:0/251000040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f480b8550 con 0x7f3f48157390 2024-05-08T05:50:11.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.863+0000 7f3f477fe640 1 --2- 172.21.15.45:0/251000040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f480cdf40 0x7f3f480ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.864+0000 7f3f467fc640 1 --2- 172.21.15.45:0/251000040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48157390 0x7f3f481577f0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00c350 tx=0x7f3f480b68d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.864+0000 7f3f2f7fe640 1 -- 172.21.15.45:0/251000040 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c037040 con 0x7f3f48157390 2024-05-08T05:50:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.864+0000 7f3f2f7fe640 1 -- 172.21.15.45:0/251000040 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f3c041070 con 0x7f3f48157390 2024-05-08T05:50:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.864+0000 7f3f4d71b640 1 -- 172.21.15.45:0/251000040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f480b8830 con 0x7f3f48157390 2024-05-08T05:50:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.864+0000 7f3f2f7fe640 1 -- 172.21.15.45:0/251000040 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c007d50 con 0x7f3f48157390 2024-05-08T05:50:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.864+0000 7f3f4d71b640 1 -- 172.21.15.45:0/251000040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f480b8df0 con 0x7f3f48157390 2024-05-08T05:50:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.865+0000 7f3f4d71b640 1 -- 172.21.15.45:0/251000040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f480693f0 con 0x7f3f48157390 2024-05-08T05:50:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.866+0000 7f3f2f7fe640 1 -- 172.21.15.45:0/251000040 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3f3c004a00 con 0x7f3f48157390 2024-05-08T05:50:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.866+0000 7f3f2f7fe640 1 --2- 172.21.15.45:0/251000040 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f340762d0 0x7f3f34078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:11.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.866+0000 7f3f2f7fe640 1 -- 172.21.15.45:0/251000040 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6187+0+0 (secure 0 0 0) 0x7f3f3c0b7510 con 0x7f3f48157390 2024-05-08T05:50:11.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.867+0000 7f3f46ffd640 1 --2- 172.21.15.45:0/251000040 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f340762d0 0x7f3f34078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:11.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.867+0000 7f3f46ffd640 1 --2- 172.21.15.45:0/251000040 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f340762d0 0x7f3f34078790 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009b00 tx=0x7f3f38008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:11.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:11.871+0000 7f3f2f7fe640 1 -- 172.21.15.45:0/251000040 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3f3c0016b0 con 0x7f3f48157390 2024-05-08T05:50:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.102+0000 7f3f4d71b640 1 -- 172.21.15.45:0/251000040 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f480b6ae0 con 0x7f3f48157390 2024-05-08T05:50:12.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.102+0000 7f3f2f7fe640 1 -- 172.21.15.45:0/251000040 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5185 (secure 0 0 0) 0x7f3f3c080b80 con 0x7f3f48157390 2024-05-08T05:50:12.103 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:50:12.103 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":88,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3692881261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3692881261},{"type":"v1","addr":"172.21.15.63:6836","nonce":3692881261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7497,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2916399818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2916399818},{"type":"v1","addr":"172.21.15.45:6833","nonce":2916399818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7515,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2978250004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":2978250004},{"type":"v1","addr":"172.21.15.63:6837","nonce":2978250004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:49:19.587326+0000","modified":"2024-05-08T05:50:11.180312+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7806},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7806":{"gid":7806,"name":"a","rank":0,"incarnation":85,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/2729952281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2729952281},{"type":"v1","addr":"172.21.15.45:6835","nonce":2729952281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7806,"qdb_cluster":[7806]},"id":7}]} 2024-05-08T05:50:12.103 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 88 2024-05-08T05:50:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.104+0000 7f3f4d71b640 1 -- 172.21.15.45:0/251000040 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f340762d0 msgr2=0x7f3f34078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.104+0000 7f3f4d71b640 1 --2- 172.21.15.45:0/251000040 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f340762d0 0x7f3f34078790 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009b00 tx=0x7f3f38008040 comp rx=0 tx=0).stop 2024-05-08T05:50:12.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.104+0000 7f3f4d71b640 1 -- 172.21.15.45:0/251000040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48157390 msgr2=0x7f3f481577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.104+0000 7f3f4d71b640 1 --2- 172.21.15.45:0/251000040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48157390 0x7f3f481577f0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00c350 tx=0x7f3f480b68d0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.104+0000 7f3f4d71b640 1 -- 172.21.15.45:0/251000040 shutdown_connections 2024-05-08T05:50:12.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.104+0000 7f3f4d71b640 1 --2- 172.21.15.45:0/251000040 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3f340762d0 0x7f3f34078790 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.104+0000 7f3f4d71b640 1 --2- 172.21.15.45:0/251000040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f480cdf40 0x7f3f480ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.104+0000 7f3f4d71b640 1 --2- 172.21.15.45:0/251000040 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f48157390 0x7f3f481577f0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.104+0000 7f3f4d71b640 1 --2- 172.21.15.45:0/251000040 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3f30004800 0x7f3f4814d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.104+0000 7f3f4d71b640 1 -- 172.21.15.45:0/251000040 >> 172.21.15.45:0/251000040 conn(0x7f3f480d4fe0 msgr2=0x7f3f480b72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:12.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.104+0000 7f3f4d71b640 1 -- 172.21.15.45:0/251000040 shutdown_connections 2024-05-08T05:50:12.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.104+0000 7f3f4d71b640 1 -- 172.21.15.45:0/251000040 wait complete. 2024-05-08T05:50:12.519 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:49:19.587326+0000', 'modified': '2024-05-08T05:50:11.180312+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 105, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7806}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7806': {'gid': 7806, 'name': 'a', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.45:6835/2729952281', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 2729952281}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 2729952281}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7806, 'qdb_cluster': [7806]} 2024-05-08T05:50:12.519 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T05:50:12.519 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-05-08T05:50:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.625+0000 7fbe33099640 1 Processor -- start 2024-05-08T05:50:12.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.625+0000 7fbe33099640 1 -- start start 2024-05-08T05:50:12.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.626+0000 7fbe33099640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fbe2c06e0f0 0x7fbe2c15c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.626+0000 7fbe33099640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbe2c06e4d0 0x7fbe2c0aec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.626+0000 7fbe33099640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe2c15c920 con 0x7fbe2c072840 2024-05-08T05:50:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.626+0000 7fbe33099640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fbe2c15cc00 con 0x7fbe2c06e4d0 2024-05-08T05:50:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.626+0000 7fbe33099640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fbe2c15cee0 con 0x7fbe2c06e0f0 2024-05-08T05:50:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.626+0000 7fbe31636640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbe2c06e4d0 0x7fbe2c0aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.626+0000 7fbe31636640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fbe2c06e4d0 0x7fbe2c0aec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59882/0 (socket says 172.21.15.45:59882) 2024-05-08T05:50:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.626+0000 7fbe31636640 1 -- 172.21.15.45:0/1859729113 learned_addr learned my addr 172.21.15.45:0/1859729113 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.626+0000 7fbe30e35640 1 --2- 172.21.15.45:0/1859729113 >> v2:172.21.15.63:3300/0 conn(0x7fbe2c06e0f0 0x7fbe2c15c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.626+0000 7fbe31636640 1 -- 172.21.15.45:0/1859729113 >> v2:172.21.15.63:3300/0 conn(0x7fbe2c06e0f0 msgr2=0x7fbe2c15c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.626+0000 7fbe31636640 1 --2- 172.21.15.45:0/1859729113 >> v2:172.21.15.63:3300/0 conn(0x7fbe2c06e0f0 0x7fbe2c15c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.626+0000 7fbe31636640 1 -- 172.21.15.45:0/1859729113 >> v1:172.21.15.45:6789/0 conn(0x7fbe2c072840 legacy=0x7fbe2c0d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.626+0000 7fbe31636640 1 -- 172.21.15.45:0/1859729113 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2c15d1c0 con 0x7fbe2c06e4d0 2024-05-08T05:50:12.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.626+0000 7fbe2b7fe640 1 -- 172.21.15.45:0/1859729113 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1460832040 0 0) 0x7fbe2c15c920 con 0x7fbe2c072840 2024-05-08T05:50:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.627+0000 7fbe31636640 1 --2- 172.21.15.45:0/1859729113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe2c06e4d0 0x7fbe2c0aec30 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c0d1cf0 tx=0x7fbe2402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d422ca1b1fb719 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.627+0000 7fbe2b7fe640 1 -- 172.21.15.45:0/1859729113 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24036070 con 0x7fbe2c06e4d0 2024-05-08T05:50:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.627+0000 7fbe2b7fe640 1 -- 172.21.15.45:0/1859729113 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe24031600 con 0x7fbe2c06e4d0 2024-05-08T05:50:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.627+0000 7fbe2b7fe640 1 -- 172.21.15.45:0/1859729113 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe240319d0 con 0x7fbe2c06e4d0 2024-05-08T05:50:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.628+0000 7fbe33099640 1 -- 172.21.15.45:0/1859729113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe2c06e4d0 msgr2=0x7fbe2c0aec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.628+0000 7fbe33099640 1 --2- 172.21.15.45:0/1859729113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe2c06e4d0 0x7fbe2c0aec30 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c0d1cf0 tx=0x7fbe2402a730 comp rx=0 tx=0).stop 2024-05-08T05:50:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.628+0000 7fbe33099640 1 -- 172.21.15.45:0/1859729113 shutdown_connections 2024-05-08T05:50:12.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.628+0000 7fbe33099640 1 --2- 172.21.15.45:0/1859729113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe2c06e4d0 0x7fbe2c0aec30 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.628+0000 7fbe33099640 1 --2- 172.21.15.45:0/1859729113 >> v2:172.21.15.63:3300/0 conn(0x7fbe2c06e0f0 0x7fbe2c15c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.628+0000 7fbe33099640 1 -- 172.21.15.45:0/1859729113 >> 172.21.15.45:0/1859729113 conn(0x7fbe2c06c3a0 msgr2=0x7fbe2c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:12.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.628+0000 7fbe33099640 1 -- 172.21.15.45:0/1859729113 shutdown_connections 2024-05-08T05:50:12.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.628+0000 7fbe33099640 1 -- 172.21.15.45:0/1859729113 wait complete. 2024-05-08T05:50:12.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.629+0000 7fbe33099640 1 Processor -- start 2024-05-08T05:50:12.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.629+0000 7fbe33099640 1 -- start start 2024-05-08T05:50:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.630+0000 7fbe33099640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe2c06e0f0 0x7fbe2c149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.630+0000 7fbe33099640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe2c06e4d0 0x7fbe2c0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.630+0000 7fbe33099640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe2c072840 0x7fbe2c154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.630+0000 7fbe33099640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c0ae7d0 con 0x7fbe2c06e4d0 2024-05-08T05:50:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.630+0000 7fbe33099640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c0ae650 con 0x7fbe2c06e0f0 2024-05-08T05:50:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.630+0000 7fbe33099640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbe2c1530b0 con 0x7fbe2c072840 2024-05-08T05:50:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.630+0000 7fbe2bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe2c06e4d0 0x7fbe2c0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.630+0000 7fbe2bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe2c06e4d0 0x7fbe2c0cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59892/0 (socket says 172.21.15.45:59892) 2024-05-08T05:50:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.630+0000 7fbe2bfff640 1 -- 172.21.15.45:0/3589283679 learned_addr learned my addr 172.21.15.45:0/3589283679 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.630+0000 7fbe30e35640 1 --2- 172.21.15.45:0/3589283679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe2c06e0f0 0x7fbe2c149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.630+0000 7fbe31636640 1 --2- 172.21.15.45:0/3589283679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe2c072840 0x7fbe2c154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.630+0000 7fbe2bfff640 1 -- 172.21.15.45:0/3589283679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe2c072840 msgr2=0x7fbe2c154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.630+0000 7fbe2bfff640 1 --2- 172.21.15.45:0/3589283679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe2c072840 0x7fbe2c154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.630+0000 7fbe2bfff640 1 -- 172.21.15.45:0/3589283679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe2c06e0f0 msgr2=0x7fbe2c149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.630+0000 7fbe2bfff640 1 --2- 172.21.15.45:0/3589283679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe2c06e0f0 0x7fbe2c149be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.630+0000 7fbe2bfff640 1 -- 172.21.15.45:0/3589283679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2c0d0560 con 0x7fbe2c06e4d0 2024-05-08T05:50:12.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.631+0000 7fbe2bfff640 1 --2- 172.21.15.45:0/3589283679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe2c06e4d0 0x7fbe2c0cfda0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800e640 tx=0x7fbe1800eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.631+0000 7fbe297fa640 1 -- 172.21.15.45:0/3589283679 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18015070 con 0x7fbe2c06e4d0 2024-05-08T05:50:12.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.631+0000 7fbe297fa640 1 -- 172.21.15.45:0/3589283679 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe18005480 con 0x7fbe2c06e4d0 2024-05-08T05:50:12.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.631+0000 7fbe33099640 1 -- 172.21.15.45:0/3589283679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2c157af0 con 0x7fbe2c06e4d0 2024-05-08T05:50:12.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.631+0000 7fbe297fa640 1 -- 172.21.15.45:0/3589283679 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18003810 con 0x7fbe2c06e4d0 2024-05-08T05:50:12.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.631+0000 7fbe33099640 1 -- 172.21.15.45:0/3589283679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe2c155290 con 0x7fbe2c06e4d0 2024-05-08T05:50:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.632+0000 7fbe297fa640 1 -- 172.21.15.45:0/3589283679 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fbe18018070 con 0x7fbe2c06e4d0 2024-05-08T05:50:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.633+0000 7fbe297fa640 1 --2- 172.21.15.45:0/3589283679 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbe100762d0 0x7fbe10078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.633+0000 7fbe297fa640 1 -- 172.21.15.45:0/3589283679 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6187+0+0 (secure 0 0 0) 0x7fbe180999a0 con 0x7fbe2c06e4d0 2024-05-08T05:50:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.633+0000 7fbe30e35640 1 --2- 172.21.15.45:0/3589283679 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbe100762d0 0x7fbe10078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.634+0000 7fbe35db9740 1 Processor -- start 2024-05-08T05:50:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.634+0000 7fbe35db9740 1 -- start start 2024-05-08T05:50:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.634+0000 7fbe30e35640 1 --2- 172.21.15.45:0/3589283679 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbe100762d0 0x7fbe10078790 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fbe20006fd0 tx=0x7fbe20008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.634+0000 7fbe35db9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 0x5582bfd85760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.634+0000 7fbe35db9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd83330 0x5582bfd85ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.634+0000 7fbe35db9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd55bb0 0x5582bfd593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.634+0000 7fbe35db9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5582bfcbf8d0 con 0x5582bfd82f50 2024-05-08T05:50:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.634+0000 7fbe35db9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5582bfc78c10 con 0x5582bfd55bb0 2024-05-08T05:50:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.634+0000 7fbe35db9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5582bfcbb890 con 0x5582bfd83330 2024-05-08T05:50:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.634+0000 7fbe30e35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 0x5582bfd85760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.635+0000 7fbe30e35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 0x5582bfd85760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59908/0 (socket says 172.21.15.45:59908) 2024-05-08T05:50:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.635+0000 7fbe30e35640 1 -- 172.21.15.45:0/1481597346 learned_addr learned my addr 172.21.15.45:0/1481597346 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.635+0000 7fbe31636640 1 --2- 172.21.15.45:0/1481597346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd55bb0 0x5582bfd593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.635+0000 7fbe2bfff640 1 --2- 172.21.15.45:0/1481597346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd83330 0x5582bfd85ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.635+0000 7fbe30e35640 1 -- 172.21.15.45:0/1481597346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd83330 msgr2=0x5582bfd85ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.635+0000 7fbe30e35640 1 --2- 172.21.15.45:0/1481597346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd83330 0x5582bfd85ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.635+0000 7fbe30e35640 1 -- 172.21.15.45:0/1481597346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd55bb0 msgr2=0x5582bfd593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.635+0000 7fbe30e35640 1 --2- 172.21.15.45:0/1481597346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd55bb0 0x5582bfd593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.635+0000 7fbe30e35640 1 -- 172.21.15.45:0/1481597346 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582bfc805d0 con 0x5582bfd82f50 2024-05-08T05:50:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.635+0000 7fbe30e35640 1 --2- 172.21.15.45:0/1481597346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 0x5582bfd85760 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000e8f0 tx=0x7fbe20011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966c1d2fce3265b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.635+0000 7fbe167fc640 1 -- 172.21.15.45:0/1481597346 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20025070 con 0x5582bfd82f50 2024-05-08T05:50:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.635+0000 7fbe167fc640 1 -- 172.21.15.45:0/1481597346 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe20015640 con 0x5582bfd82f50 2024-05-08T05:50:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.635+0000 7fbe167fc640 1 -- 172.21.15.45:0/1481597346 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2000f040 con 0x5582bfd82f50 2024-05-08T05:50:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.636+0000 7fbe35db9740 1 -- 172.21.15.45:0/1481597346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 msgr2=0x5582bfd85760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.636+0000 7fbe35db9740 1 --2- 172.21.15.45:0/1481597346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 0x5582bfd85760 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000e8f0 tx=0x7fbe20011000 comp rx=0 tx=0).stop 2024-05-08T05:50:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.636+0000 7fbe35db9740 1 -- 172.21.15.45:0/1481597346 shutdown_connections 2024-05-08T05:50:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.636+0000 7fbe35db9740 1 --2- 172.21.15.45:0/1481597346 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd83330 0x5582bfd85ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.636+0000 7fbe35db9740 1 --2- 172.21.15.45:0/1481597346 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd55bb0 0x5582bfd593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.636+0000 7fbe35db9740 1 --2- 172.21.15.45:0/1481597346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 0x5582bfd85760 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.636+0000 7fbe35db9740 1 -- 172.21.15.45:0/1481597346 >> 172.21.15.45:0/1481597346 conn(0x5582bfd57e20 msgr2=0x5582bfda0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.636+0000 7fbe35db9740 1 -- 172.21.15.45:0/1481597346 shutdown_connections 2024-05-08T05:50:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.636+0000 7fbe35db9740 1 -- 172.21.15.45:0/1481597346 wait complete. 2024-05-08T05:50:12.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.636+0000 7fbe35db9740 1 -- start start 2024-05-08T05:50:12.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.637+0000 7fbe35db9740 1 Processor -- start 2024-05-08T05:50:12.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.637+0000 7fbe35db9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 0x5582bfda4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.637+0000 7fbe35db9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd83330 0x5582bfd88410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.637+0000 7fbe35db9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd78a10 0x5582bfd78e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.637+0000 7fbe35db9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5582bfd5afd0 con 0x5582bfd82f50 2024-05-08T05:50:12.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.637+0000 7fbe35db9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5582bfc7c760 con 0x5582bfd78a10 2024-05-08T05:50:12.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.637+0000 7fbe35db9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5582bfd5b2b0 con 0x5582bfd83330 2024-05-08T05:50:12.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.637+0000 7fbe35db9740 10 client.0 _ll_register_callbacks cb 0x5582bfcb5cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:50:12.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.638+0000 7fbe30e35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 0x5582bfda4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.638+0000 7fbe30e35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 0x5582bfda4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59922/0 (socket says 172.21.15.45:59922) 2024-05-08T05:50:12.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.638+0000 7fbe30e35640 1 -- 172.21.15.45:0/3647283808 learned_addr learned my addr 172.21.15.45:0/3647283808 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:12.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.638+0000 7fbe31636640 1 --2- 172.21.15.45:0/3647283808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd78a10 0x5582bfd78e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.638+0000 7fbe2bfff640 1 --2- 172.21.15.45:0/3647283808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd83330 0x5582bfd88410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.638+0000 7fbe30e35640 1 -- 172.21.15.45:0/3647283808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd83330 msgr2=0x5582bfd88410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.638+0000 7fbe30e35640 1 --2- 172.21.15.45:0/3647283808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd83330 0x5582bfd88410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.638+0000 7fbe30e35640 1 -- 172.21.15.45:0/3647283808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd78a10 msgr2=0x5582bfd78e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.638+0000 7fbe30e35640 1 --2- 172.21.15.45:0/3647283808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd78a10 0x5582bfd78e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.638+0000 7fbe30e35640 1 -- 172.21.15.45:0/3647283808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582bf9d12d0 con 0x5582bfd82f50 2024-05-08T05:50:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.638+0000 7fbe30e35640 1 --2- 172.21.15.45:0/3647283808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 0x5582bfda4260 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fbe20023040 tx=0x7fbe20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7795fcbc51b1ca4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.638+0000 7fbe0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:50:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.638+0000 7fbe0b7fe640 1 -- 172.21.15.45:0/3647283808 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2002e060 con 0x5582bfd82f50 2024-05-08T05:50:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.638+0000 7fbe0b7fe640 1 -- 172.21.15.45:0/3647283808 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6187+0+0 (secure 0 0 0) 0x7fbe20020400 con 0x5582bfd82f50 2024-05-08T05:50:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.638+0000 7fbe35db9740 1 -- 172.21.15.45:0/3647283808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe20015640 con 0x5582bfd82f50 2024-05-08T05:50:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.638+0000 7fbe0b7fe640 1 -- 172.21.15.45:0/3647283808 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fbe20034050 con 0x5582bfd82f50 2024-05-08T05:50:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.638+0000 7fbe35db9740 10 client.7878 fetch_fsmap learned FSMap version 88 2024-05-08T05:50:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.638+0000 7fbe35db9740 1 -- 172.21.15.45:0/3647283808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x5582bfa9b6b0 con 0x5582bfd82f50 2024-05-08T05:50:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.638+0000 7fbe0b7fe640 1 -- 172.21.15.45:0/3647283808 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe20015640 con 0x5582bfd82f50 2024-05-08T05:50:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.639+0000 7fbe0b7fe640 1 -- 172.21.15.45:0/3647283808 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7fbe20037020 con 0x5582bfd82f50 2024-05-08T05:50:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.639+0000 7fbe35db9740 10 client.7878 fetch_fsmap finished waiting for FSMap version 88 2024-05-08T05:50:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.639+0000 7fbe35db9740 10 client.7878 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T05:50:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.639+0000 7fbe35db9740 20 client.7878 populate_metadata read hostname 'smithi045' 2024-05-08T05:50:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.639+0000 7fbe35db9740 1 --2- 172.21.15.45:0/3647283808 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x5582bfd86760 0x5582bfd86b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.639+0000 7fbe35db9740 4 client.7878 mds_command: new command op to 7806 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:50:12.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.639+0000 7fbe35db9740 1 -- 172.21.15.45:0/3647283808 --> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582bfc43760 con 0x5582bfd86760 2024-05-08T05:50:12.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.639+0000 7fbe2bfff640 1 --2- 172.21.15.45:0/3647283808 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x5582bfd86760 0x5582bfd86b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.639+0000 7fbe0b7fe640 10 client.7878 ms_handle_connect on v2:172.21.15.45:6834/2729952281 2024-05-08T05:50:12.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.639+0000 7fbe2bfff640 1 --2- 172.21.15.45:0/3647283808 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x5582bfd86760 0x5582bfd86b40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c2f91edfd6e5df4 server_cookie=acd7ea45f34e48b8 in_seq=0 out_seq=0 2024-05-08T05:50:12.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.640+0000 7fbe0b7fe640 1 -- 172.21.15.45:0/3647283808 <== mds.0 v2:172.21.15.45:6834/2729952281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5582bfc43760 con 0x5582bfd86760 2024-05-08T05:50:12.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.640+0000 7fbe0b7fe640 10 client.7878 handle_command_reply: tid=0 2024-05-08T05:50:12.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.640+0000 7fbe35db9740 1 client.7878 shutdown 2024-05-08T05:50:12.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.640+0000 7fbe35db9740 1 -- 172.21.15.45:0/3647283808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 msgr2=0x5582bfda4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.640+0000 7fbe35db9740 1 --2- 172.21.15.45:0/3647283808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 0x5582bfda4260 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fbe20023040 tx=0x7fbe20004730 comp rx=0 tx=0).stop 2024-05-08T05:50:12.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.640+0000 7fbe35db9740 1 -- 172.21.15.45:0/3647283808 shutdown_connections 2024-05-08T05:50:12.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.640+0000 7fbe35db9740 1 --2- 172.21.15.45:0/3647283808 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x5582bfd86760 0x5582bfd86b40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.640+0000 7fbe35db9740 1 --2- 172.21.15.45:0/3647283808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd83330 0x5582bfd88410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.640+0000 7fbe35db9740 1 --2- 172.21.15.45:0/3647283808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd78a10 0x5582bfd78e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.640+0000 7fbe35db9740 1 --2- 172.21.15.45:0/3647283808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 0x5582bfda4260 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.640+0000 7fbe0b7fe640 0 client.7878 ms_handle_reset on v2:172.21.15.45:6834/2729952281 2024-05-08T05:50:12.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.640+0000 7fbe35db9740 1 -- 172.21.15.45:0/3647283808 >> 172.21.15.45:0/3647283808 conn(0x5582bfd55bb0 msgr2=0x5582bfd815d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:12.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.640+0000 7fbe35db9740 1 -- 172.21.15.45:0/3647283808 shutdown_connections 2024-05-08T05:50:12.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.640+0000 7fbe35db9740 1 -- 172.21.15.45:0/3647283808 wait complete. 2024-05-08T05:50:12.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.640+0000 7fbe35db9740 20 client.7878 trim_cache size 0 max 16384 2024-05-08T05:50:12.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.659+0000 7fbe35db9740 1 Processor -- start 2024-05-08T05:50:12.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.659+0000 7fbe35db9740 1 -- start start 2024-05-08T05:50:12.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe35db9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd78a10 0x5582bfdc4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe35db9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd82f50 0x5582bfd56090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe35db9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd83330 0x5582bfd595b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe35db9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c1fd910 con 0x5582bfd78a10 2024-05-08T05:50:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe35db9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c1fc910 con 0x5582bfd83330 2024-05-08T05:50:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe35db9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fbe2c1f5c00 con 0x5582bfd82f50 2024-05-08T05:50:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe30e35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd78a10 0x5582bfdc4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe30e35640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd78a10 0x5582bfdc4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59936/0 (socket says 172.21.15.45:59936) 2024-05-08T05:50:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe30e35640 1 -- 172.21.15.45:0/3894040352 learned_addr learned my addr 172.21.15.45:0/3894040352 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe31636640 1 --2- 172.21.15.45:0/3894040352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd83330 0x5582bfd595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe30e35640 1 -- 172.21.15.45:0/3894040352 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd82f50 msgr2=0x5582bfd56090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe2bfff640 1 --2- 172.21.15.45:0/3894040352 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd82f50 0x5582bfd56090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe30e35640 1 --2- 172.21.15.45:0/3894040352 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd82f50 0x5582bfd56090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe30e35640 1 -- 172.21.15.45:0/3894040352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd83330 msgr2=0x5582bfd595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe30e35640 1 --2- 172.21.15.45:0/3894040352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd83330 0x5582bfd595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe30e35640 1 -- 172.21.15.45:0/3894040352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582bfda8ca0 con 0x5582bfd78a10 2024-05-08T05:50:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe31636640 1 --2- 172.21.15.45:0/3894040352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd83330 0x5582bfd595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe2bfff640 1 --2- 172.21.15.45:0/3894040352 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd82f50 0x5582bfd56090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe30e35640 1 --2- 172.21.15.45:0/3894040352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd78a10 0x5582bfdc4b60 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fbe20020810 tx=0x7fbe20027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d182c9067df43d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe15ffb640 1 -- 172.21.15.45:0/3894040352 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20025040 con 0x5582bfd78a10 2024-05-08T05:50:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe15ffb640 1 -- 172.21.15.45:0/3894040352 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe20015440 con 0x5582bfd78a10 2024-05-08T05:50:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.660+0000 7fbe15ffb640 1 -- 172.21.15.45:0/3894040352 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20020900 con 0x5582bfd78a10 2024-05-08T05:50:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.661+0000 7fbe35db9740 1 -- 172.21.15.45:0/3894040352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd78a10 msgr2=0x5582bfdc4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.661+0000 7fbe35db9740 1 --2- 172.21.15.45:0/3894040352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd78a10 0x5582bfdc4b60 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fbe20020810 tx=0x7fbe20027000 comp rx=0 tx=0).stop 2024-05-08T05:50:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.661+0000 7fbe35db9740 1 -- 172.21.15.45:0/3894040352 shutdown_connections 2024-05-08T05:50:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.661+0000 7fbe35db9740 1 --2- 172.21.15.45:0/3894040352 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd82f50 0x5582bfd56090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.661+0000 7fbe35db9740 1 --2- 172.21.15.45:0/3894040352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd83330 0x5582bfd595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.661+0000 7fbe35db9740 1 --2- 172.21.15.45:0/3894040352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd78a10 0x5582bfdc4b60 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.661+0000 7fbe35db9740 1 -- 172.21.15.45:0/3894040352 >> 172.21.15.45:0/3894040352 conn(0x5582bfd55bb0 msgr2=0x5582bfd815d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:12.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.661+0000 7fbe35db9740 1 -- 172.21.15.45:0/3894040352 shutdown_connections 2024-05-08T05:50:12.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.661+0000 7fbe35db9740 1 -- 172.21.15.45:0/3894040352 wait complete. 2024-05-08T05:50:12.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.661+0000 7fbe35db9740 1 -- start start 2024-05-08T05:50:12.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.662+0000 7fbe35db9740 1 Processor -- start 2024-05-08T05:50:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.662+0000 7fbe35db9740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd78a10 0x5582bfe1e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.662+0000 7fbe35db9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 0x5582bfd826c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.662+0000 7fbe35db9740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd83330 0x5582bfe21fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.662+0000 7fbe35db9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5582bfda53a0 con 0x5582bfd82f50 2024-05-08T05:50:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.662+0000 7fbe35db9740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c1fc2c0 con 0x5582bfd78a10 2024-05-08T05:50:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.662+0000 7fbe35db9740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5582bfda20f0 con 0x5582bfd83330 2024-05-08T05:50:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.663+0000 7fbe35db9740 10 client.0 _ll_register_callbacks cb 0x7fbe20023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:50:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.663+0000 7fbe2bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 0x5582bfd826c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.663+0000 7fbe2bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 0x5582bfd826c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59940/0 (socket says 172.21.15.45:59940) 2024-05-08T05:50:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.663+0000 7fbe2bfff640 1 -- 172.21.15.45:0/1291270679 learned_addr learned my addr 172.21.15.45:0/1291270679 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.663+0000 7fbe30e35640 1 --2- 172.21.15.45:0/1291270679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd78a10 0x5582bfe1e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.663+0000 7fbe31636640 1 --2- 172.21.15.45:0/1291270679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd83330 0x5582bfe21fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.663+0000 7fbe2bfff640 1 -- 172.21.15.45:0/1291270679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd83330 msgr2=0x5582bfe21fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.663+0000 7fbe2bfff640 1 --2- 172.21.15.45:0/1291270679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd83330 0x5582bfe21fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.663+0000 7fbe2bfff640 1 -- 172.21.15.45:0/1291270679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd78a10 msgr2=0x5582bfe1e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.663+0000 7fbe2bfff640 1 --2- 172.21.15.45:0/1291270679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd78a10 0x5582bfe1e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.663+0000 7fbe2bfff640 1 -- 172.21.15.45:0/1291270679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582bfd82c00 con 0x5582bfd82f50 2024-05-08T05:50:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.663+0000 7fbe31636640 1 --2- 172.21.15.45:0/1291270679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd83330 0x5582bfe21fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.663+0000 7fbe2bfff640 1 --2- 172.21.15.45:0/1291270679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 0x5582bfd826c0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fbe1805ef90 tx=0x7fbe18069040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73fbb32353a0f52 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.663+0000 7fbe0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:50:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.663+0000 7fbe0b7fe640 1 -- 172.21.15.45:0/1291270679 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1801f040 con 0x5582bfd82f50 2024-05-08T05:50:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.663+0000 7fbe0b7fe640 1 -- 172.21.15.45:0/1291270679 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6187+0+0 (secure 0 0 0) 0x7fbe180690b0 con 0x5582bfd82f50 2024-05-08T05:50:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.663+0000 7fbe35db9740 1 -- 172.21.15.45:0/1291270679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe20015440 con 0x5582bfd82f50 2024-05-08T05:50:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.663+0000 7fbe0b7fe640 1 -- 172.21.15.45:0/1291270679 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fbe18095050 con 0x5582bfd82f50 2024-05-08T05:50:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.663+0000 7fbe35db9740 10 client.7884 fetch_fsmap learned FSMap version 88 2024-05-08T05:50:12.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.663+0000 7fbe35db9740 1 -- 172.21.15.45:0/1291270679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x5582bfd555e0 con 0x5582bfd82f50 2024-05-08T05:50:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.664+0000 7fbe0b7fe640 1 -- 172.21.15.45:0/1291270679 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe20015440 con 0x5582bfd82f50 2024-05-08T05:50:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.664+0000 7fbe0b7fe640 1 -- 172.21.15.45:0/1291270679 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7fbe1800f040 con 0x5582bfd82f50 2024-05-08T05:50:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.664+0000 7fbe35db9740 10 client.7884 fetch_fsmap finished waiting for FSMap version 88 2024-05-08T05:50:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.664+0000 7fbe35db9740 10 client.7884 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-08T05:50:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.664+0000 7fbe35db9740 20 client.7884 populate_metadata read hostname 'smithi045' 2024-05-08T05:50:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.664+0000 7fbe35db9740 1 --2- 172.21.15.45:0/1291270679 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x5582bfda0000 0x5582bfda03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.664+0000 7fbe35db9740 4 client.7884 mds_command: new command op to 7806 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:50:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.664+0000 7fbe35db9740 1 -- 172.21.15.45:0/1291270679 --> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] -- command(tid 0: {"prefix": "status"}) -- 0x5582bfccd4e0 con 0x5582bfda0000 2024-05-08T05:50:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.664+0000 7fbe30e35640 1 --2- 172.21.15.45:0/1291270679 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x5582bfda0000 0x5582bfda03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.664+0000 7fbe0b7fe640 10 client.7884 ms_handle_connect on v2:172.21.15.45:6834/2729952281 2024-05-08T05:50:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.664+0000 7fbe30e35640 1 --2- 172.21.15.45:0/1291270679 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x5582bfda0000 0x5582bfda03e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3072753a93138767 server_cookie=d1ef4684204355fc in_seq=0 out_seq=0 2024-05-08T05:50:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.664+0000 7fbe0b7fe640 1 -- 172.21.15.45:0/1291270679 <== mds.0 v2:172.21.15.45:6834/2729952281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5582bfccd4e0 con 0x5582bfda0000 2024-05-08T05:50:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.664+0000 7fbe0b7fe640 10 client.7884 handle_command_reply: tid=0 2024-05-08T05:50:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.664+0000 7fbe35db9740 1 client.7884 shutdown 2024-05-08T05:50:12.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe35db9740 1 -- 172.21.15.45:0/1291270679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 msgr2=0x5582bfd826c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe35db9740 1 --2- 172.21.15.45:0/1291270679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 0x5582bfd826c0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fbe1805ef90 tx=0x7fbe18069040 comp rx=0 tx=0).stop 2024-05-08T05:50:12.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe35db9740 1 -- 172.21.15.45:0/1291270679 shutdown_connections 2024-05-08T05:50:12.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe35db9740 1 --2- 172.21.15.45:0/1291270679 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x5582bfda0000 0x5582bfda03e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe35db9740 1 --2- 172.21.15.45:0/1291270679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5582bfd83330 0x5582bfe21fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe35db9740 1 --2- 172.21.15.45:0/1291270679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5582bfd82f50 0x5582bfd826c0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe0b7fe640 0 client.7884 ms_handle_reset on v2:172.21.15.45:6834/2729952281 2024-05-08T05:50:12.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe35db9740 1 --2- 172.21.15.45:0/1291270679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5582bfd78a10 0x5582bfe1e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe35db9740 1 -- 172.21.15.45:0/1291270679 >> 172.21.15.45:0/1291270679 conn(0x5582bfd55bb0 msgr2=0x5582bfd815d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:12.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe35db9740 1 -- 172.21.15.45:0/1291270679 shutdown_connections 2024-05-08T05:50:12.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe35db9740 1 -- 172.21.15.45:0/1291270679 wait complete. 2024-05-08T05:50:12.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe35db9740 20 client.7884 trim_cache size 0 max 16384 2024-05-08T05:50:12.671 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:50:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "e9e1170a-fd27-4625-97a9-5d5f104d13a2", 2024-05-08T05:50:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-05-08T05:50:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 7806, 2024-05-08T05:50:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-05-08T05:50:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-05-08T05:50:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-05-08T05:50:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 4.487751373, 2024-05-08T05:50:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 88, 2024-05-08T05:50:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 105, 2024-05-08T05:50:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 105, 2024-05-08T05:50:12.672 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 5.4070571459999996 2024-05-08T05:50:12.672 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:50:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe33099640 1 -- 172.21.15.45:0/3589283679 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbe100762d0 msgr2=0x7fbe10078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe33099640 1 --2- 172.21.15.45:0/3589283679 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbe100762d0 0x7fbe10078790 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fbe20006fd0 tx=0x7fbe20008040 comp rx=0 tx=0).stop 2024-05-08T05:50:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe33099640 1 -- 172.21.15.45:0/3589283679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe2c06e4d0 msgr2=0x7fbe2c0cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe33099640 1 --2- 172.21.15.45:0/3589283679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe2c06e4d0 0x7fbe2c0cfda0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800e640 tx=0x7fbe1800eb00 comp rx=0 tx=0).stop 2024-05-08T05:50:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe33099640 1 -- 172.21.15.45:0/3589283679 shutdown_connections 2024-05-08T05:50:12.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe33099640 1 --2- 172.21.15.45:0/3589283679 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fbe100762d0 0x7fbe10078790 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe33099640 1 --2- 172.21.15.45:0/3589283679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fbe2c072840 0x7fbe2c154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe33099640 1 --2- 172.21.15.45:0/3589283679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe2c06e4d0 0x7fbe2c0cfda0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe33099640 1 --2- 172.21.15.45:0/3589283679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe2c06e0f0 0x7fbe2c149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe33099640 1 -- 172.21.15.45:0/3589283679 >> 172.21.15.45:0/3589283679 conn(0x7fbe2c06c3a0 msgr2=0x7fbe2c073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe33099640 1 -- 172.21.15.45:0/3589283679 shutdown_connections 2024-05-08T05:50:12.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.665+0000 7fbe33099640 1 -- 172.21.15.45:0/3589283679 wait complete. 2024-05-08T05:50:12.676 DEBUG:teuthology.orchestra.run.smithi045:> sudo daemon-helper kill ceph --cluster ceph -w 2024-05-08T05:50:12.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:50:12.679 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-05-08T05:50:12.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.786+0000 7f771e48c640 1 Processor -- start 2024-05-08T05:50:12.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.787+0000 7f771e48c640 1 -- start start 2024-05-08T05:50:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.787+0000 7f771e48c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7718162b70 0x7f77180d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.787+0000 7f771e48c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f77180d38e0 0x7f77180d0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.787+0000 7f771e48c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7718162f50 con 0x7f77180d3480 2024-05-08T05:50:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.787+0000 7f771e48c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f77180d11a0 con 0x7f77180d38e0 2024-05-08T05:50:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.787+0000 7f771e48c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f77180d1f60 con 0x7f7718162b70 2024-05-08T05:50:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.788+0000 7f771ca29640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f77180d38e0 0x7f77180d0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.788+0000 7f771ca29640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f77180d38e0 0x7f77180d0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59954/0 (socket says 172.21.15.45:59954) 2024-05-08T05:50:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.788+0000 7f771ca29640 1 -- 172.21.15.45:0/3253639227 learned_addr learned my addr 172.21.15.45:0/3253639227 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.788+0000 7f7717fff640 1 --2- 172.21.15.45:0/3253639227 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7718162b70 0x7f77180d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.788+0000 7f771ca29640 1 -- 172.21.15.45:0/3253639227 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7718162b70 msgr2=0x7f77180d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.788+0000 7f771ca29640 1 --2- 172.21.15.45:0/3253639227 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7718162b70 0x7f77180d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.788+0000 7f771ca29640 1 -- 172.21.15.45:0/3253639227 >> v1:172.21.15.63:6789/0 conn(0x7f77180d3480 legacy=0x7f77180b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.788+0000 7f771ca29640 1 -- 172.21.15.45:0/3253639227 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77180d3ee0 con 0x7f77180d38e0 2024-05-08T05:50:12.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.788+0000 7f77177fe640 1 -- 172.21.15.45:0/3253639227 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2832488721 0 0) 0x7f7700001110 con 0x7f77180d3480 2024-05-08T05:50:12.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.789+0000 7f771ca29640 1 --2- 172.21.15.45:0/3253639227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77180d38e0 0x7f77180d0c60 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f7710009e50 tx=0x7f771002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa537e7e95e11ef9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.789+0000 7f77177fe640 1 -- 172.21.15.45:0/3253639227 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7710036070 con 0x7f77180d38e0 2024-05-08T05:50:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.789+0000 7f77177fe640 1 -- 172.21.15.45:0/3253639227 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7710031600 con 0x7f77180d38e0 2024-05-08T05:50:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.789+0000 7f77177fe640 1 -- 172.21.15.45:0/3253639227 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77100319d0 con 0x7f77180d38e0 2024-05-08T05:50:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.790+0000 7f771e48c640 1 -- 172.21.15.45:0/3253639227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77180d38e0 msgr2=0x7f77180d0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.790+0000 7f771e48c640 1 --2- 172.21.15.45:0/3253639227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77180d38e0 0x7f77180d0c60 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f7710009e50 tx=0x7f771002a710 comp rx=0 tx=0).stop 2024-05-08T05:50:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.790+0000 7f771e48c640 1 -- 172.21.15.45:0/3253639227 shutdown_connections 2024-05-08T05:50:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.790+0000 7f771e48c640 1 --2- 172.21.15.45:0/3253639227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f77180d38e0 0x7f77180d0c60 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.790+0000 7f771e48c640 1 --2- 172.21.15.45:0/3253639227 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7718162b70 0x7f77180d2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.790+0000 7f771e48c640 1 -- 172.21.15.45:0/3253639227 >> 172.21.15.45:0/3253639227 conn(0x7f77180d5040 msgr2=0x7f77180d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.790+0000 7f771e48c640 1 -- 172.21.15.45:0/3253639227 shutdown_connections 2024-05-08T05:50:12.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.790+0000 7f771e48c640 1 -- 172.21.15.45:0/3253639227 wait complete. 2024-05-08T05:50:12.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.791+0000 7f771e48c640 1 Processor -- start 2024-05-08T05:50:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.791+0000 7f771e48c640 1 -- start start 2024-05-08T05:50:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.792+0000 7f771e48c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77180d3480 0x7f7718160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.792+0000 7f771e48c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77180d38e0 0x7f77180c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.792+0000 7f771e48c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718162b70 0x7f77180cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.792+0000 7f771e48c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f77180b4170 con 0x7f7718162b70 2024-05-08T05:50:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.792+0000 7f771e48c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f77180d2490 con 0x7f77180d38e0 2024-05-08T05:50:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.792+0000 7f771e48c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f77180b42f0 con 0x7f77180d3480 2024-05-08T05:50:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.792+0000 7f771ca29640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718162b70 0x7f77180cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.792+0000 7f771ca29640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718162b70 0x7f77180cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59968/0 (socket says 172.21.15.45:59968) 2024-05-08T05:50:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.792+0000 7f771ca29640 1 -- 172.21.15.45:0/4058962975 learned_addr learned my addr 172.21.15.45:0/4058962975 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.792+0000 7f770ffff640 1 --2- 172.21.15.45:0/4058962975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77180d38e0 0x7f77180c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.793+0000 7f7717fff640 1 --2- 172.21.15.45:0/4058962975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77180d3480 0x7f7718160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.793+0000 7f771ca29640 1 -- 172.21.15.45:0/4058962975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77180d3480 msgr2=0x7f7718160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.793+0000 7f771ca29640 1 --2- 172.21.15.45:0/4058962975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77180d3480 0x7f7718160360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.793+0000 7f771ca29640 1 -- 172.21.15.45:0/4058962975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77180d38e0 msgr2=0x7f77180c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.793+0000 7f771ca29640 1 --2- 172.21.15.45:0/4058962975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77180d38e0 0x7f77180c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.793+0000 7f771ca29640 1 -- 172.21.15.45:0/4058962975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77180caa50 con 0x7f7718162b70 2024-05-08T05:50:12.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.793+0000 7f770ffff640 1 --2- 172.21.15.45:0/4058962975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77180d38e0 0x7f77180c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:12.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.793+0000 7f7717fff640 1 --2- 172.21.15.45:0/4058962975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77180d3480 0x7f7718160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:12.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.793+0000 7f771ca29640 1 --2- 172.21.15.45:0/4058962975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718162b70 0x7f77180cfc10 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f7710009e50 tx=0x7f7710004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.793+0000 7f77157fa640 1 -- 172.21.15.45:0/4058962975 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7710036050 con 0x7f7718162b70 2024-05-08T05:50:12.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.793+0000 7f77157fa640 1 -- 172.21.15.45:0/4058962975 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7710042070 con 0x7f7718162b70 2024-05-08T05:50:12.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.793+0000 7f77157fa640 1 -- 172.21.15.45:0/4058962975 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7710009510 con 0x7f7718162b70 2024-05-08T05:50:12.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.793+0000 7f771e48c640 1 -- 172.21.15.45:0/4058962975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77180c9ed0 con 0x7f7718162b70 2024-05-08T05:50:12.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.793+0000 7f771e48c640 1 -- 172.21.15.45:0/4058962975 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77180ca490 con 0x7f7718162b70 2024-05-08T05:50:12.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.795+0000 7f77157fa640 1 -- 172.21.15.45:0/4058962975 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7710007bf0 con 0x7f7718162b70 2024-05-08T05:50:12.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.796+0000 7f77157fa640 1 --2- 172.21.15.45:0/4058962975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f76f80762d0 0x7f76f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.796+0000 7f77157fa640 1 -- 172.21.15.45:0/4058962975 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6187+0+0 (secure 0 0 0) 0x7f77100b8530 con 0x7f7718162b70 2024-05-08T05:50:12.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.796+0000 7f77211a8740 1 Processor -- start 2024-05-08T05:50:12.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.796+0000 7f77211a8740 1 -- start start 2024-05-08T05:50:12.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.796+0000 7f7717fff640 1 --2- 172.21.15.45:0/4058962975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f76f80762d0 0x7f76f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.797+0000 7f7717fff640 1 --2- 172.21.15.45:0/4058962975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f76f80762d0 0x7f76f8078790 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7708009af0 tx=0x7f7708008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.797+0000 7f77211a8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55814169b220 0x55814164dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.797+0000 7f77211a8740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 0x55814167d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.797+0000 7f77211a8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55814167dc00 0x558141652a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.797+0000 7f77211a8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5581415b6cf0 con 0x55814169b220 2024-05-08T05:50:12.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.797+0000 7f77211a8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55814156fe10 con 0x55814167dc00 2024-05-08T05:50:12.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.797+0000 7f77211a8740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5581415b2cb0 con 0x55814167e340 2024-05-08T05:50:12.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.797+0000 7f7717fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55814169b220 0x55814164dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.797+0000 7f7717fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55814169b220 0x55814164dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59982/0 (socket says 172.21.15.45:59982) 2024-05-08T05:50:12.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.797+0000 7f7717fff640 1 -- 172.21.15.45:0/3716735408 learned_addr learned my addr 172.21.15.45:0/3716735408 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:12.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.797+0000 7f770ffff640 1 --2- 172.21.15.45:0/3716735408 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 0x55814167d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.797+0000 7f771ca29640 1 --2- 172.21.15.45:0/3716735408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55814167dc00 0x558141652a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.797+0000 7f7717fff640 1 -- 172.21.15.45:0/3716735408 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 msgr2=0x55814167d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.797+0000 7f7717fff640 1 --2- 172.21.15.45:0/3716735408 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 0x55814167d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.797+0000 7f7717fff640 1 -- 172.21.15.45:0/3716735408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55814167dc00 msgr2=0x558141652a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.797+0000 7f7717fff640 1 --2- 172.21.15.45:0/3716735408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55814167dc00 0x558141652a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.797+0000 7f7717fff640 1 -- 172.21.15.45:0/3716735408 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581413907e0 con 0x55814169b220 2024-05-08T05:50:12.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.798+0000 7f7717fff640 1 --2- 172.21.15.45:0/3716735408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55814169b220 0x55814164dbb0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f7708011de0 tx=0x7f7708014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3a7d5e7510bf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.798+0000 7f770dffb640 1 -- 172.21.15.45:0/3716735408 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708011e10 con 0x55814169b220 2024-05-08T05:50:12.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.798+0000 7f770dffb640 1 -- 172.21.15.45:0/3716735408 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7708026070 con 0x55814169b220 2024-05-08T05:50:12.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.798+0000 7f770dffb640 1 -- 172.21.15.45:0/3716735408 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708021640 con 0x55814169b220 2024-05-08T05:50:12.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.799+0000 7f77211a8740 1 -- 172.21.15.45:0/3716735408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55814169b220 msgr2=0x55814164dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.799+0000 7f77211a8740 1 --2- 172.21.15.45:0/3716735408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55814169b220 0x55814164dbb0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f7708011de0 tx=0x7f7708014000 comp rx=0 tx=0).stop 2024-05-08T05:50:12.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.799+0000 7f77211a8740 1 -- 172.21.15.45:0/3716735408 shutdown_connections 2024-05-08T05:50:12.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.799+0000 7f77211a8740 1 --2- 172.21.15.45:0/3716735408 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 0x55814167d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.799+0000 7f77211a8740 1 --2- 172.21.15.45:0/3716735408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55814167dc00 0x558141652a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.799+0000 7f77211a8740 1 --2- 172.21.15.45:0/3716735408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55814169b220 0x55814164dbb0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.799+0000 7f77211a8740 1 -- 172.21.15.45:0/3716735408 >> 172.21.15.45:0/3716735408 conn(0x55814164fe20 msgr2=0x5581416986c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:12.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.799+0000 7f77211a8740 1 -- 172.21.15.45:0/3716735408 shutdown_connections 2024-05-08T05:50:12.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.799+0000 7f77211a8740 1 -- 172.21.15.45:0/3716735408 wait complete. 2024-05-08T05:50:12.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.799+0000 7f77211a8740 1 -- start start 2024-05-08T05:50:12.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.799+0000 7f77211a8740 1 Processor -- start 2024-05-08T05:50:12.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.800+0000 7f77211a8740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 0x5581416904c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.800+0000 7f77211a8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55814169b220 0x55814166aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.800+0000 7f77211a8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558141690a90 0x558141690f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.800+0000 7f77211a8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558141651fd0 con 0x55814169b220 2024-05-08T05:50:12.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.800+0000 7f77211a8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x558141573940 con 0x558141690a90 2024-05-08T05:50:12.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.800+0000 7f77211a8740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5581416513c0 con 0x55814167e340 2024-05-08T05:50:12.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.800+0000 7f77211a8740 10 client.0 _ll_register_callbacks cb 0x558141576440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:50:12.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.800+0000 7f770ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55814169b220 0x55814166aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.800+0000 7f770ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55814169b220 0x55814166aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59998/0 (socket says 172.21.15.45:59998) 2024-05-08T05:50:12.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.800+0000 7f770ffff640 1 -- 172.21.15.45:0/2595689928 learned_addr learned my addr 172.21.15.45:0/2595689928 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:12.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.800+0000 7f7717fff640 1 --2- 172.21.15.45:0/2595689928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 0x5581416904c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.800+0000 7f771ca29640 1 --2- 172.21.15.45:0/2595689928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558141690a90 0x558141690f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.801+0000 7f770ffff640 1 -- 172.21.15.45:0/2595689928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 msgr2=0x5581416904c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.801+0000 7f770ffff640 1 --2- 172.21.15.45:0/2595689928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 0x5581416904c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.801+0000 7f770ffff640 1 -- 172.21.15.45:0/2595689928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558141690a90 msgr2=0x558141690f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.801+0000 7f770ffff640 1 --2- 172.21.15.45:0/2595689928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558141690a90 0x558141690f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.801+0000 7f770ffff640 1 -- 172.21.15.45:0/2595689928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581413f0f70 con 0x55814169b220 2024-05-08T05:50:12.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.801+0000 7f771ca29640 1 --2- 172.21.15.45:0/2595689928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558141690a90 0x558141690f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:12.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.801+0000 7f770ffff640 1 --2- 172.21.15.45:0/2595689928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55814169b220 0x55814166aff0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f7700000f60 tx=0x7f7700003da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783f3e621f363390 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.801+0000 7f76f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:50:12.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.801+0000 7f76f6ffd640 1 -- 172.21.15.45:0/2595689928 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770001a060 con 0x55814169b220 2024-05-08T05:50:12.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.801+0000 7f76f6ffd640 1 -- 172.21.15.45:0/2595689928 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6187+0+0 (secure 0 0 0) 0x7f7700001c80 con 0x55814169b220 2024-05-08T05:50:12.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.801+0000 7f77211a8740 1 -- 172.21.15.45:0/2595689928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7708026070 con 0x55814169b220 2024-05-08T05:50:12.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.801+0000 7f76f6ffd640 1 -- 172.21.15.45:0/2595689928 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f7700020050 con 0x55814169b220 2024-05-08T05:50:12.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.801+0000 7f77211a8740 10 client.7896 fetch_fsmap learned FSMap version 88 2024-05-08T05:50:12.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.801+0000 7f77211a8740 1 -- 172.21.15.45:0/2595689928 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x558141288170 con 0x55814169b220 2024-05-08T05:50:12.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.801+0000 7f76f6ffd640 1 -- 172.21.15.45:0/2595689928 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7708026070 con 0x55814169b220 2024-05-08T05:50:12.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.802+0000 7f76f6ffd640 1 -- 172.21.15.45:0/2595689928 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f7700023020 con 0x55814169b220 2024-05-08T05:50:12.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.802+0000 7f77211a8740 10 client.7896 fetch_fsmap finished waiting for FSMap version 88 2024-05-08T05:50:12.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.802+0000 7f77211a8740 10 client.7896 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-08T05:50:12.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.802+0000 7f77211a8740 20 client.7896 populate_metadata read hostname 'smithi045' 2024-05-08T05:50:12.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.802+0000 7f77211a8740 1 --2- 172.21.15.45:0/2595689928 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x55814166bdd0 0x55814169bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.802+0000 7f77211a8740 4 client.7896 mds_command: new command op to 7806 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:50:12.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.802+0000 7f77211a8740 1 -- 172.21.15.45:0/2595689928 --> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581415c4900 con 0x55814166bdd0 2024-05-08T05:50:12.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.802+0000 7f7717fff640 1 --2- 172.21.15.45:0/2595689928 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x55814166bdd0 0x55814169bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.802+0000 7f7717fff640 1 --2- 172.21.15.45:0/2595689928 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x55814166bdd0 0x55814169bae0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d90f9c83bba2d70 server_cookie=faff6896405d129a in_seq=0 out_seq=0 2024-05-08T05:50:12.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.802+0000 7f76f6ffd640 10 client.7896 ms_handle_connect on v2:172.21.15.45:6834/2729952281 2024-05-08T05:50:12.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.803+0000 7f76f6ffd640 1 -- 172.21.15.45:0/2595689928 <== mds.0 v2:172.21.15.45:6834/2729952281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5581415c4900 con 0x55814166bdd0 2024-05-08T05:50:12.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.803+0000 7f76f6ffd640 10 client.7896 handle_command_reply: tid=0 2024-05-08T05:50:12.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.803+0000 7f77211a8740 1 client.7896 shutdown 2024-05-08T05:50:12.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.803+0000 7f77211a8740 1 -- 172.21.15.45:0/2595689928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55814169b220 msgr2=0x55814166aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.803+0000 7f77211a8740 1 --2- 172.21.15.45:0/2595689928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55814169b220 0x55814166aff0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f7700000f60 tx=0x7f7700003da0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.803+0000 7f77211a8740 1 -- 172.21.15.45:0/2595689928 shutdown_connections 2024-05-08T05:50:12.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.803+0000 7f77211a8740 1 --2- 172.21.15.45:0/2595689928 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x55814166bdd0 0x55814169bae0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.803+0000 7f77211a8740 1 --2- 172.21.15.45:0/2595689928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x558141690a90 0x558141690f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.803+0000 7f77211a8740 1 --2- 172.21.15.45:0/2595689928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55814169b220 0x55814166aff0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.803+0000 7f76f6ffd640 0 client.7896 ms_handle_reset on v2:172.21.15.45:6834/2729952281 2024-05-08T05:50:12.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.803+0000 7f77211a8740 1 --2- 172.21.15.45:0/2595689928 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 0x5581416904c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.803+0000 7f77211a8740 1 -- 172.21.15.45:0/2595689928 >> 172.21.15.45:0/2595689928 conn(0x55814167dc00 msgr2=0x55814169a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:12.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.803+0000 7f77211a8740 1 -- 172.21.15.45:0/2595689928 shutdown_connections 2024-05-08T05:50:12.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.803+0000 7f77211a8740 1 -- 172.21.15.45:0/2595689928 wait complete. 2024-05-08T05:50:12.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.803+0000 7f77211a8740 20 client.7896 trim_cache size 0 max 16384 2024-05-08T05:50:12.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.807+0000 7ff44fb0a640 1 Processor -- start 2024-05-08T05:50:12.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.807+0000 7ff44fb0a640 1 -- start start 2024-05-08T05:50:12.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.808+0000 7ff44fb0a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff4480cfdc0 0x7ff4480ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.808+0000 7ff44fb0a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff4480d3a30 0x7ff4480b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.808+0000 7ff44fb0a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff44815d830 con 0x7ff44815d450 2024-05-08T05:50:12.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.808+0000 7ff44fb0a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff4480d1fa0 con 0x7ff4480d3a30 2024-05-08T05:50:12.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.808+0000 7ff44fb0a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff4480ced10 con 0x7ff4480cfdc0 2024-05-08T05:50:12.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.808+0000 7ff44e0a7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff4480d3a30 0x7ff4480b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.808+0000 7ff44e0a7640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff4480d3a30 0x7ff4480b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60002/0 (socket says 172.21.15.45:60002) 2024-05-08T05:50:12.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.808+0000 7ff44e0a7640 1 -- 172.21.15.45:0/1703380538 learned_addr learned my addr 172.21.15.45:0/1703380538 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:12.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.809+0000 7ff44d0a5640 1 --2- 172.21.15.45:0/1703380538 >> v2:172.21.15.63:3300/0 conn(0x7ff4480cfdc0 0x7ff4480ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.809+0000 7ff44e0a7640 1 -- 172.21.15.45:0/1703380538 >> v2:172.21.15.63:3300/0 conn(0x7ff4480cfdc0 msgr2=0x7ff4480ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.809+0000 7ff44e0a7640 1 --2- 172.21.15.45:0/1703380538 >> v2:172.21.15.63:3300/0 conn(0x7ff4480cfdc0 0x7ff4480ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.809+0000 7ff44e0a7640 1 -- 172.21.15.45:0/1703380538 >> v1:172.21.15.63:6789/0 conn(0x7ff44815d450 legacy=0x7ff4480cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.809+0000 7ff44e0a7640 1 -- 172.21.15.45:0/1703380538 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4480b2600 con 0x7ff4480d3a30 2024-05-08T05:50:12.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.809+0000 7ff44d0a5640 1 --2- 172.21.15.45:0/1703380538 >> v2:172.21.15.63:3300/0 conn(0x7ff4480cfdc0 0x7ff4480ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:12.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.809+0000 7ff44e0a7640 1 --2- 172.21.15.45:0/1703380538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4480d3a30 0x7ff4480b30e0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7ff448072e10 tx=0x7ff44002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d3d713d64bd03c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.810+0000 7ff44c8a4640 1 -- 172.21.15.45:0/1703380538 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff440036070 con 0x7ff4480d3a30 2024-05-08T05:50:12.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.810+0000 7ff44c8a4640 1 -- 172.21.15.45:0/1703380538 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff440031600 con 0x7ff4480d3a30 2024-05-08T05:50:12.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.810+0000 7ff44c8a4640 1 -- 172.21.15.45:0/1703380538 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4400319d0 con 0x7ff4480d3a30 2024-05-08T05:50:12.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.810+0000 7ff44fb0a640 1 -- 172.21.15.45:0/1703380538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4480d3a30 msgr2=0x7ff4480b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.810+0000 7ff44fb0a640 1 --2- 172.21.15.45:0/1703380538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4480d3a30 0x7ff4480b30e0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7ff448072e10 tx=0x7ff44002a610 comp rx=0 tx=0).stop 2024-05-08T05:50:12.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.810+0000 7ff44fb0a640 1 -- 172.21.15.45:0/1703380538 shutdown_connections 2024-05-08T05:50:12.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.810+0000 7ff44fb0a640 1 --2- 172.21.15.45:0/1703380538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4480d3a30 0x7ff4480b30e0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.810+0000 7ff44fb0a640 1 --2- 172.21.15.45:0/1703380538 >> v2:172.21.15.63:3300/0 conn(0x7ff4480cfdc0 0x7ff4480ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.810+0000 7ff44fb0a640 1 -- 172.21.15.45:0/1703380538 >> 172.21.15.45:0/1703380538 conn(0x7ff4480d2f10 msgr2=0x7ff4480d3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:12.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.810+0000 7ff44fb0a640 1 -- 172.21.15.45:0/1703380538 shutdown_connections 2024-05-08T05:50:12.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.810+0000 7ff44fb0a640 1 -- 172.21.15.45:0/1703380538 wait complete. 2024-05-08T05:50:12.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.811+0000 7ff44fb0a640 1 Processor -- start 2024-05-08T05:50:12.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.812+0000 7ff44fb0a640 1 -- start start 2024-05-08T05:50:12.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.812+0000 7ff44fb0a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4480cfdc0 0x7ff448159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.812+0000 7ff44fb0a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4480d3a30 0x7ff44815da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.812+0000 7ff44fb0a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff44815d450 0x7ff4480cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.812+0000 7ff44fb0a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff4480b2bc0 con 0x7ff4480cfdc0 2024-05-08T05:50:12.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.812+0000 7ff44fb0a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff4480b28e0 con 0x7ff4480d3a30 2024-05-08T05:50:12.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.812+0000 7ff44fb0a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff4480b3620 con 0x7ff44815d450 2024-05-08T05:50:12.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.812+0000 7ff44d8a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4480cfdc0 0x7ff448159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.812+0000 7ff44d8a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4480cfdc0 0x7ff448159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60006/0 (socket says 172.21.15.45:60006) 2024-05-08T05:50:12.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.812+0000 7ff44d8a6640 1 -- 172.21.15.45:0/1557537917 learned_addr learned my addr 172.21.15.45:0/1557537917 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:12.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.813+0000 7ff44d0a5640 1 --2- 172.21.15.45:0/1557537917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4480d3a30 0x7ff44815da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.813+0000 7ff44e0a7640 1 --2- 172.21.15.45:0/1557537917 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff44815d450 0x7ff4480cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.813+0000 7ff44d8a6640 1 -- 172.21.15.45:0/1557537917 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff44815d450 msgr2=0x7ff4480cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.813+0000 7ff44d8a6640 1 --2- 172.21.15.45:0/1557537917 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff44815d450 0x7ff4480cbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.813+0000 7ff44d8a6640 1 -- 172.21.15.45:0/1557537917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4480d3a30 msgr2=0x7ff44815da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.813+0000 7ff44d8a6640 1 --2- 172.21.15.45:0/1557537917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4480d3a30 0x7ff44815da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.813+0000 7ff44d8a6640 1 -- 172.21.15.45:0/1557537917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff44815e430 con 0x7ff4480cfdc0 2024-05-08T05:50:12.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.813+0000 7ff44d8a6640 1 --2- 172.21.15.45:0/1557537917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4480cfdc0 0x7ff448159810 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7ff44400b3f0 tx=0x7ff44400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.813+0000 7ff43a7fc640 1 -- 172.21.15.45:0/1557537917 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444015020 con 0x7ff4480cfdc0 2024-05-08T05:50:12.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.813+0000 7ff43a7fc640 1 -- 172.21.15.45:0/1557537917 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff444011830 con 0x7ff4480cfdc0 2024-05-08T05:50:12.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.813+0000 7ff44fb0a640 1 -- 172.21.15.45:0/1557537917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff44815e710 con 0x7ff4480cfdc0 2024-05-08T05:50:12.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.813+0000 7ff43a7fc640 1 -- 172.21.15.45:0/1557537917 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444011c20 con 0x7ff4480cfdc0 2024-05-08T05:50:12.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.813+0000 7ff44fb0a640 1 -- 172.21.15.45:0/1557537917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff44815ecd0 con 0x7ff4480cfdc0 2024-05-08T05:50:12.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.814+0000 7ff44fb0a640 1 -- 172.21.15.45:0/1557537917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7ff448069b00 con 0x7ff4480cfdc0 2024-05-08T05:50:12.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.815+0000 7ff43a7fc640 1 -- 172.21.15.45:0/1557537917 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff444018020 con 0x7ff4480cfdc0 2024-05-08T05:50:12.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.815+0000 7ff43a7fc640 1 --2- 172.21.15.45:0/1557537917 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff42c0762d0 0x7ff42c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.815+0000 7ff43a7fc640 1 -- 172.21.15.45:0/1557537917 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6187+0+0 (secure 0 0 0) 0x7ff444097c60 con 0x7ff4480cfdc0 2024-05-08T05:50:12.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.815+0000 7ff43a7fc640 1 -- 172.21.15.45:0/1557537917 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+512 (secure 0 0 0) 0x7ff44401f020 con 0x7ff4480cfdc0 2024-05-08T05:50:12.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.815+0000 7ff44d0a5640 1 --2- 172.21.15.45:0/1557537917 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff42c0762d0 0x7ff42c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.816+0000 7ff44fb0a640 1 -- 172.21.15.45:0/1557537917 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7ff44815c700 con 0x7ff4480cfdc0 2024-05-08T05:50:12.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.816+0000 7ff44d0a5640 1 --2- 172.21.15.45:0/1557537917 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff42c0762d0 0x7ff42c078790 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff4480b2010 tx=0x7ff43c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.823+0000 7f77211a8740 1 Processor -- start 2024-05-08T05:50:12.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.823+0000 7f77211a8740 1 -- start start 2024-05-08T05:50:12.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f77211a8740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 0x55814164d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f77211a8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558141690a90 0x558141653720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f77211a8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55814169b220 0x558141653c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f77211a8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558141691c30 con 0x558141690a90 2024-05-08T05:50:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f77211a8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7718201ff0 con 0x55814169b220 2024-05-08T05:50:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f77211a8740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7718201030 con 0x55814167e340 2024-05-08T05:50:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f770ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558141690a90 0x558141653720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f770ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558141690a90 0x558141653720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60020/0 (socket says 172.21.15.45:60020) 2024-05-08T05:50:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f770ffff640 1 -- 172.21.15.45:0/2309844893 learned_addr learned my addr 172.21.15.45:0/2309844893 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f7717fff640 1 --2- 172.21.15.45:0/2309844893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 0x55814164d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f770ffff640 1 -- 172.21.15.45:0/2309844893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 msgr2=0x55814164d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f771ca29640 1 --2- 172.21.15.45:0/2309844893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55814169b220 0x558141653c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f770ffff640 1 --2- 172.21.15.45:0/2309844893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 0x55814164d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f770ffff640 1 -- 172.21.15.45:0/2309844893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55814169b220 msgr2=0x558141653c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f770ffff640 1 --2- 172.21.15.45:0/2309844893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55814169b220 0x558141653c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f770ffff640 1 -- 172.21.15.45:0/2309844893 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55814169be50 con 0x558141690a90 2024-05-08T05:50:12.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f7717fff640 1 --2- 172.21.15.45:0/2309844893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 0x55814164d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f771ca29640 1 --2- 172.21.15.45:0/2309844893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55814169b220 0x558141653c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f770ffff640 1 --2- 172.21.15.45:0/2309844893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558141690a90 0x558141653720 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f7700005040 tx=0x7f770000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293d070ba63f996c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f770d7fa640 1 -- 172.21.15.45:0/2309844893 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700001c80 con 0x558141690a90 2024-05-08T05:50:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f770d7fa640 1 -- 172.21.15.45:0/2309844893 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7700015990 con 0x558141690a90 2024-05-08T05:50:12.826 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.824+0000 7f770d7fa640 1 -- 172.21.15.45:0/2309844893 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700015d60 con 0x558141690a90 2024-05-08T05:50:12.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.825+0000 7f77211a8740 1 -- 172.21.15.45:0/2309844893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558141690a90 msgr2=0x558141653720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.825+0000 7f77211a8740 1 --2- 172.21.15.45:0/2309844893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558141690a90 0x558141653720 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f7700005040 tx=0x7f770000e000 comp rx=0 tx=0).stop 2024-05-08T05:50:12.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.825+0000 7f77211a8740 1 -- 172.21.15.45:0/2309844893 shutdown_connections 2024-05-08T05:50:12.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.825+0000 7f77211a8740 1 --2- 172.21.15.45:0/2309844893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55814169b220 0x558141653c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.825+0000 7f77211a8740 1 --2- 172.21.15.45:0/2309844893 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558141690a90 0x558141653720 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.825+0000 7f77211a8740 1 --2- 172.21.15.45:0/2309844893 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 0x55814164d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.825+0000 7f77211a8740 1 -- 172.21.15.45:0/2309844893 >> 172.21.15.45:0/2309844893 conn(0x55814167dc00 msgr2=0x55814166ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:12.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.825+0000 7f77211a8740 1 -- 172.21.15.45:0/2309844893 shutdown_connections 2024-05-08T05:50:12.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.825+0000 7f77211a8740 1 -- 172.21.15.45:0/2309844893 wait complete. 2024-05-08T05:50:12.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.826+0000 7f77211a8740 1 -- start start 2024-05-08T05:50:12.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.826+0000 7f77211a8740 1 Processor -- start 2024-05-08T05:50:12.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f77211a8740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 0x55814168e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f77211a8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558141690a90 0x558141653720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f77211a8740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55814169b220 0x558141653c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f77211a8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7718200ba0 con 0x558141690a90 2024-05-08T05:50:12.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f77211a8740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f77181f6570 con 0x55814169b220 2024-05-08T05:50:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f77211a8740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x558141649820 con 0x55814167e340 2024-05-08T05:50:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f77211a8740 10 client.0 _ll_register_callbacks cb 0x7f7700007c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:50:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f770ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558141690a90 0x558141653720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f770ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558141690a90 0x558141653720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60028/0 (socket says 172.21.15.45:60028) 2024-05-08T05:50:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f770ffff640 1 -- 172.21.15.45:0/2281025183 learned_addr learned my addr 172.21.15.45:0/2281025183 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f7717fff640 1 --2- 172.21.15.45:0/2281025183 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 0x55814168e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f771ca29640 1 --2- 172.21.15.45:0/2281025183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55814169b220 0x558141653c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f770ffff640 1 -- 172.21.15.45:0/2281025183 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 msgr2=0x55814168e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f770ffff640 1 --2- 172.21.15.45:0/2281025183 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 0x55814168e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f770ffff640 1 -- 172.21.15.45:0/2281025183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55814169b220 msgr2=0x558141653c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f770ffff640 1 --2- 172.21.15.45:0/2281025183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55814169b220 0x558141653c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f770ffff640 1 -- 172.21.15.45:0/2281025183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55814164d1c0 con 0x558141690a90 2024-05-08T05:50:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f771ca29640 1 --2- 172.21.15.45:0/2281025183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55814169b220 0x558141653c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f770ffff640 1 --2- 172.21.15.45:0/2281025183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558141690a90 0x558141653720 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f7700007ed0 tx=0x7f7700001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4100b5a5272939f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f76f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:50:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f76f6ffd640 1 -- 172.21.15.45:0/2281025183 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700030070 con 0x558141690a90 2024-05-08T05:50:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f76f6ffd640 1 -- 172.21.15.45:0/2281025183 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6187+0+0 (secure 0 0 0) 0x7f770002c850 con 0x558141690a90 2024-05-08T05:50:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.827+0000 7f77211a8740 1 -- 172.21.15.45:0/2281025183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7700015990 con 0x558141690a90 2024-05-08T05:50:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.828+0000 7f76f6ffd640 1 -- 172.21.15.45:0/2281025183 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f7700001890 con 0x558141690a90 2024-05-08T05:50:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.828+0000 7f77211a8740 10 client.7908 fetch_fsmap learned FSMap version 88 2024-05-08T05:50:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.828+0000 7f77211a8740 1 -- 172.21.15.45:0/2281025183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x5581416811f0 con 0x558141690a90 2024-05-08T05:50:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.828+0000 7f76f6ffd640 1 -- 172.21.15.45:0/2281025183 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7700015990 con 0x558141690a90 2024-05-08T05:50:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.828+0000 7f76f6ffd640 1 -- 172.21.15.45:0/2281025183 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f7700023040 con 0x558141690a90 2024-05-08T05:50:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.828+0000 7f77211a8740 10 client.7908 fetch_fsmap finished waiting for FSMap version 88 2024-05-08T05:50:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.828+0000 7f77211a8740 10 client.7908 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-08T05:50:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.828+0000 7f77211a8740 20 client.7908 populate_metadata read hostname 'smithi045' 2024-05-08T05:50:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.828+0000 7f77211a8740 1 --2- 172.21.15.45:0/2281025183 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x558141680db0 0x558141679530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.828+0000 7f77211a8740 4 client.7908 mds_command: new command op to 7806 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-05-08T05:50:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.828+0000 7f77211a8740 1 -- 172.21.15.45:0/2281025183 --> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x5581415be770 con 0x558141680db0 2024-05-08T05:50:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.828+0000 7f7717fff640 1 --2- 172.21.15.45:0/2281025183 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x558141680db0 0x558141679530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.828+0000 7f76f6ffd640 10 client.7908 ms_handle_connect on v2:172.21.15.45:6834/2729952281 2024-05-08T05:50:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.828+0000 7f7717fff640 1 --2- 172.21.15.45:0/2281025183 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x558141680db0 0x558141679530 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb279af9bdc43bf7 server_cookie=c50fd0638e3881e3 in_seq=0 out_seq=0 2024-05-08T05:50:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.829+0000 7f76f6ffd640 1 -- 172.21.15.45:0/2281025183 <== mds.0 v2:172.21.15.45:6834/2729952281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5581415be770 con 0x558141680db0 2024-05-08T05:50:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.829+0000 7f76f6ffd640 10 client.7908 handle_command_reply: tid=0 2024-05-08T05:50:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.829+0000 7f77211a8740 1 client.7908 shutdown 2024-05-08T05:50:12.834 INFO:tasks.ceph.mds.a.smithi045.stderr:2024-05-08T05:50:12.829+0000 7fbecd4e1640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-05-08T05:50:12.835 INFO:tasks.ceph.mds.a.smithi045.stderr:2024-05-08T05:50:12.829+0000 7fbecd4e1640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-05-08T05:50:12.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.829+0000 7f77211a8740 1 -- 172.21.15.45:0/2281025183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558141690a90 msgr2=0x558141653720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.829+0000 7f77211a8740 1 --2- 172.21.15.45:0/2281025183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558141690a90 0x558141653720 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f7700007ed0 tx=0x7f7700001080 comp rx=0 tx=0).stop 2024-05-08T05:50:12.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.829+0000 7f77211a8740 1 -- 172.21.15.45:0/2281025183 shutdown_connections 2024-05-08T05:50:12.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.829+0000 7f77211a8740 1 --2- 172.21.15.45:0/2281025183 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x558141680db0 0x558141679530 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.829+0000 7f77211a8740 1 --2- 172.21.15.45:0/2281025183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55814169b220 0x558141653c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.829+0000 7f77211a8740 1 --2- 172.21.15.45:0/2281025183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558141690a90 0x558141653720 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.829+0000 7f77211a8740 1 --2- 172.21.15.45:0/2281025183 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55814167e340 0x55814168e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.829+0000 7f77211a8740 1 -- 172.21.15.45:0/2281025183 >> 172.21.15.45:0/2281025183 conn(0x55814167dc00 msgr2=0x5581416bc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:12.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.829+0000 7f76f6ffd640 0 client.7908 ms_handle_reset on v2:172.21.15.45:6834/2729952281 2024-05-08T05:50:12.836 INFO:tasks.ceph.mds.a.smithi045.stderr:2024-05-08T05:50:12.829+0000 7fbecd4e1640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-05-08T05:50:12.836 INFO:tasks.ceph.mds.a.smithi045.stderr:2024-05-08T05:50:12.829+0000 7fbecd4e1640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-05-08T05:50:12.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.829+0000 7f77211a8740 1 -- 172.21.15.45:0/2281025183 shutdown_connections 2024-05-08T05:50:12.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.830+0000 7f77211a8740 1 -- 172.21.15.45:0/2281025183 wait complete. 2024-05-08T05:50:12.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.830+0000 7f77211a8740 20 client.7908 trim_cache size 0 max 16384 2024-05-08T05:50:12.837 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:50:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0, 2024-05-08T05:50:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_tag": "e76b6156-c6d9-4618-90fc-52e2d08af961", 2024-05-08T05:50:12.837 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": "asynchronous" 2024-05-08T05:50:12.837 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:50:12.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.831+0000 7f771e48c640 1 -- 172.21.15.45:0/4058962975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f76f80762d0 msgr2=0x7f76f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.831+0000 7f771e48c640 1 --2- 172.21.15.45:0/4058962975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f76f80762d0 0x7f76f8078790 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7708009af0 tx=0x7f7708008040 comp rx=0 tx=0).stop 2024-05-08T05:50:12.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.831+0000 7f771e48c640 1 -- 172.21.15.45:0/4058962975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718162b70 msgr2=0x7f77180cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.831+0000 7f771e48c640 1 --2- 172.21.15.45:0/4058962975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718162b70 0x7f77180cfc10 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f7710009e50 tx=0x7f7710004a40 comp rx=0 tx=0).stop 2024-05-08T05:50:12.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.831+0000 7f771e48c640 1 -- 172.21.15.45:0/4058962975 shutdown_connections 2024-05-08T05:50:12.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.831+0000 7f771e48c640 1 --2- 172.21.15.45:0/4058962975 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f76f80762d0 0x7f76f8078790 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.831+0000 7f771e48c640 1 --2- 172.21.15.45:0/4058962975 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7718162b70 0x7f77180cfc10 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.831+0000 7f771e48c640 1 --2- 172.21.15.45:0/4058962975 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f77180d38e0 0x7f77180c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.831+0000 7f771e48c640 1 --2- 172.21.15.45:0/4058962975 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f77180d3480 0x7f7718160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.831+0000 7f771e48c640 1 -- 172.21.15.45:0/4058962975 >> 172.21.15.45:0/4058962975 conn(0x7f77180d5040 msgr2=0x7f771806d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:12.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.831+0000 7f771e48c640 1 -- 172.21.15.45:0/4058962975 shutdown_connections 2024-05-08T05:50:12.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.831+0000 7f771e48c640 1 -- 172.21.15.45:0/4058962975 wait complete. 2024-05-08T05:50:12.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:50:12.842 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-08T05:50:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.948+0000 7f99c7ce8640 1 Processor -- start 2024-05-08T05:50:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.948+0000 7f99c7ce8640 1 -- start start 2024-05-08T05:50:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.949+0000 7f99c7ce8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f99c00d20c0 0x7f99c0163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.949+0000 7f99c7ce8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99c0071110 0x7f99c006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.949+0000 7f99c7ce8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99c00d1300 con 0x7f99c0070d30 2024-05-08T05:50:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.949+0000 7f99c7ce8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f99c00d10d0 con 0x7f99c00d20c0 2024-05-08T05:50:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.949+0000 7f99c7ce8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f99c0070b80 con 0x7f99c0071110 2024-05-08T05:50:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.949+0000 7f99c5283640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f99c00d20c0 0x7f99c0163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.949+0000 7f99c5283640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f99c00d20c0 0x7f99c0163920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60040/0 (socket says 172.21.15.45:60040) 2024-05-08T05:50:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.949+0000 7f99c5283640 1 -- 172.21.15.45:0/1440386591 learned_addr learned my addr 172.21.15.45:0/1440386591 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.949+0000 7f99c6285640 1 --2- 172.21.15.45:0/1440386591 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99c0071110 0x7f99c006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.949+0000 7f99c5283640 1 -- 172.21.15.45:0/1440386591 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99c0071110 msgr2=0x7f99c006d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.949+0000 7f99c5283640 1 --2- 172.21.15.45:0/1440386591 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99c0071110 0x7f99c006d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.949+0000 7f99c5283640 1 -- 172.21.15.45:0/1440386591 >> v1:172.21.15.63:6789/0 conn(0x7f99c0070d30 legacy=0x7f99c00703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.949+0000 7f99c5283640 1 -- 172.21.15.45:0/1440386591 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c00d15e0 con 0x7f99c00d20c0 2024-05-08T05:50:12.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.950+0000 7f99c5283640 1 --2- 172.21.15.45:0/1440386591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c00d20c0 0x7f99c0163920 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009870 tx=0x7f99b002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6d5d670469649e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.950+0000 7f99c4a82640 1 -- 172.21.15.45:0/1440386591 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0037070 con 0x7f99c00d20c0 2024-05-08T05:50:12.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.950+0000 7f99c4a82640 1 -- 172.21.15.45:0/1440386591 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b0032440 con 0x7f99c00d20c0 2024-05-08T05:50:12.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.950+0000 7f99c4a82640 1 -- 172.21.15.45:0/1440386591 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0032810 con 0x7f99c00d20c0 2024-05-08T05:50:12.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.951+0000 7f99c7ce8640 1 -- 172.21.15.45:0/1440386591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c00d20c0 msgr2=0x7f99c0163920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.951+0000 7f99c7ce8640 1 --2- 172.21.15.45:0/1440386591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c00d20c0 0x7f99c0163920 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009870 tx=0x7f99b002a1c0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.951+0000 7f99c7ce8640 1 -- 172.21.15.45:0/1440386591 shutdown_connections 2024-05-08T05:50:12.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.951+0000 7f99c7ce8640 1 --2- 172.21.15.45:0/1440386591 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99c0071110 0x7f99c006d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.951+0000 7f99c7ce8640 1 --2- 172.21.15.45:0/1440386591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c00d20c0 0x7f99c0163920 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.951+0000 7f99c7ce8640 1 -- 172.21.15.45:0/1440386591 >> 172.21.15.45:0/1440386591 conn(0x7f99c00d2e90 msgr2=0x7f99c00d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:12.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.951+0000 7f99c7ce8640 1 -- 172.21.15.45:0/1440386591 shutdown_connections 2024-05-08T05:50:12.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.951+0000 7f99c7ce8640 1 -- 172.21.15.45:0/1440386591 wait complete. 2024-05-08T05:50:12.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.952+0000 7f99c7ce8640 1 Processor -- start 2024-05-08T05:50:12.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.952+0000 7f99c7ce8640 1 -- start start 2024-05-08T05:50:12.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.953+0000 7f99c7ce8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99c0070d30 0x7f99c015e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.953+0000 7f99c7ce8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c0071110 0x7f99c00b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.953+0000 7f99c7ce8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99c00d20c0 0x7f99c00b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.953+0000 7f99c7ce8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f99c0071560 con 0x7f99c0071110 2024-05-08T05:50:12.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.953+0000 7f99c7ce8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f99c00d1c30 con 0x7f99c00d20c0 2024-05-08T05:50:12.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.953+0000 7f99c7ce8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f99c00716e0 con 0x7f99c0070d30 2024-05-08T05:50:12.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.953+0000 7f99c5283640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c0071110 0x7f99c00b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.953+0000 7f99c5283640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c0071110 0x7f99c00b78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60052/0 (socket says 172.21.15.45:60052) 2024-05-08T05:50:12.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.953+0000 7f99c5283640 1 -- 172.21.15.45:0/3531168820 learned_addr learned my addr 172.21.15.45:0/3531168820 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:12.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.953+0000 7f99c5a84640 1 --2- 172.21.15.45:0/3531168820 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99c0070d30 0x7f99c015e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.953+0000 7f99c5283640 1 -- 172.21.15.45:0/3531168820 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99c0070d30 msgr2=0x7f99c015e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.953+0000 7f99c6285640 1 --2- 172.21.15.45:0/3531168820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99c00d20c0 0x7f99c00b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.953+0000 7f99c5283640 1 --2- 172.21.15.45:0/3531168820 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99c0070d30 0x7f99c015e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.953+0000 7f99c5283640 1 -- 172.21.15.45:0/3531168820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99c00d20c0 msgr2=0x7f99c00b7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.953+0000 7f99c5283640 1 --2- 172.21.15.45:0/3531168820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99c00d20c0 0x7f99c00b7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.953+0000 7f99c5283640 1 -- 172.21.15.45:0/3531168820 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c00b5fe0 con 0x7f99c0071110 2024-05-08T05:50:12.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.953+0000 7f99c5a84640 1 --2- 172.21.15.45:0/3531168820 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99c0070d30 0x7f99c015e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:12.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.953+0000 7f99c6285640 1 --2- 172.21.15.45:0/3531168820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99c00d20c0 0x7f99c00b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:12.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.954+0000 7f99c5283640 1 --2- 172.21.15.45:0/3531168820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c0071110 0x7f99c00b78e0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009870 tx=0x7f99b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.954+0000 7f99b67fc640 1 -- 172.21.15.45:0/3531168820 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0041020 con 0x7f99c0071110 2024-05-08T05:50:12.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.954+0000 7f99b67fc640 1 -- 172.21.15.45:0/3531168820 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b0030970 con 0x7f99c0071110 2024-05-08T05:50:12.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.954+0000 7f99b67fc640 1 -- 172.21.15.45:0/3531168820 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0037040 con 0x7f99c0071110 2024-05-08T05:50:12.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.954+0000 7f99c7ce8640 1 -- 172.21.15.45:0/3531168820 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c00c8420 con 0x7f99c0071110 2024-05-08T05:50:12.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.954+0000 7f99c7ce8640 1 -- 172.21.15.45:0/3531168820 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c00c8cc0 con 0x7f99c0071110 2024-05-08T05:50:12.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.956+0000 7f99b67fc640 1 -- 172.21.15.45:0/3531168820 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f99b0044020 con 0x7f99c0071110 2024-05-08T05:50:12.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.956+0000 7f99b67fc640 1 --2- 172.21.15.45:0/3531168820 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99a40762d0 0x7f99a4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.956+0000 7f99b67fc640 1 -- 172.21.15.45:0/3531168820 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6187+0+0 (secure 0 0 0) 0x7f99b00b81c0 con 0x7f99c0071110 2024-05-08T05:50:12.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.956+0000 7f99c5a84640 1 --2- 172.21.15.45:0/3531168820 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99a40762d0 0x7f99a4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.957+0000 7f99c5a84640 1 --2- 172.21.15.45:0/3531168820 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99a40762d0 0x7f99a4078790 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f99c0161c20 tx=0x7f99bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.957+0000 7f99caa06740 1 Processor -- start 2024-05-08T05:50:12.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.957+0000 7f99caa06740 1 -- start start 2024-05-08T05:50:12.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.958+0000 7f99caa06740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e38160 0x559d78deaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.958+0000 7f99caa06740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e1b280 0x559d78e1a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.958+0000 7f99caa06740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559d78e1ab40 0x559d78def950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.958+0000 7f99caa06740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559d78db7690 con 0x559d78e1b280 2024-05-08T05:50:12.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.958+0000 7f99caa06740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559d78d54f30 con 0x559d78e38160 2024-05-08T05:50:12.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.958+0000 7f99caa06740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x559d78d0db20 con 0x559d78e1ab40 2024-05-08T05:50:12.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.958+0000 7f99c5283640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e1b280 0x559d78e1a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.958+0000 7f99c5283640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e1b280 0x559d78e1a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60068/0 (socket says 172.21.15.45:60068) 2024-05-08T05:50:12.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.958+0000 7f99c5283640 1 -- 172.21.15.45:0/3324627183 learned_addr learned my addr 172.21.15.45:0/3324627183 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:12.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.958+0000 7f99c5a84640 1 --2- 172.21.15.45:0/3324627183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e38160 0x559d78deaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.958+0000 7f99c6285640 1 --2- 172.21.15.45:0/3324627183 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559d78e1ab40 0x559d78def950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.958+0000 7f99c5283640 1 -- 172.21.15.45:0/3324627183 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559d78e1ab40 msgr2=0x559d78def950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.958+0000 7f99c5283640 1 --2- 172.21.15.45:0/3324627183 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559d78e1ab40 0x559d78def950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.958+0000 7f99c5283640 1 -- 172.21.15.45:0/3324627183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e38160 msgr2=0x559d78deaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.958+0000 7f99c5283640 1 --2- 172.21.15.45:0/3324627183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e38160 0x559d78deaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.958+0000 7f99c5283640 1 -- 172.21.15.45:0/3324627183 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d78d05e60 con 0x559d78e1b280 2024-05-08T05:50:12.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.958+0000 7f99c5283640 1 --2- 172.21.15.45:0/3324627183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e1b280 0x559d78e1a600 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f99b00b6480 tx=0x7f99b007d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fd86a1152dfae2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.958+0000 7f99a37fe640 1 -- 172.21.15.45:0/3324627183 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b00809b0 con 0x559d78e1b280 2024-05-08T05:50:12.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.959+0000 7f99a37fe640 1 -- 172.21.15.45:0/3324627183 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b0080d50 con 0x559d78e1b280 2024-05-08T05:50:12.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.959+0000 7f99a37fe640 1 -- 172.21.15.45:0/3324627183 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0081940 con 0x559d78e1b280 2024-05-08T05:50:12.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.959+0000 7f99caa06740 1 -- 172.21.15.45:0/3324627183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e1b280 msgr2=0x559d78e1a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.959+0000 7f99caa06740 1 --2- 172.21.15.45:0/3324627183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e1b280 0x559d78e1a600 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f99b00b6480 tx=0x7f99b007d9c0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.959+0000 7f99caa06740 1 -- 172.21.15.45:0/3324627183 shutdown_connections 2024-05-08T05:50:12.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.959+0000 7f99caa06740 1 --2- 172.21.15.45:0/3324627183 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559d78e1ab40 0x559d78def950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.959+0000 7f99caa06740 1 --2- 172.21.15.45:0/3324627183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e1b280 0x559d78e1a600 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.959+0000 7f99caa06740 1 --2- 172.21.15.45:0/3324627183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e38160 0x559d78deaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.959+0000 7f99caa06740 1 -- 172.21.15.45:0/3324627183 >> 172.21.15.45:0/3324627183 conn(0x559d78decd60 msgr2=0x559d78e35600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:12.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.959+0000 7f99caa06740 1 -- 172.21.15.45:0/3324627183 shutdown_connections 2024-05-08T05:50:12.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.959+0000 7f99caa06740 1 -- 172.21.15.45:0/3324627183 wait complete. 2024-05-08T05:50:12.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.960+0000 7f99caa06740 1 -- start start 2024-05-08T05:50:12.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.960+0000 7f99caa06740 1 Processor -- start 2024-05-08T05:50:12.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.961+0000 7f99caa06740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e1b280 0x559d78e36080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.961+0000 7f99caa06740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e38160 0x559d78e34200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.961+0000 7f99caa06740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559d78e31ce0 0x559d78e34740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.961+0000 7f99caa06740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559d78d115e0 con 0x559d78e1b280 2024-05-08T05:50:12.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.961+0000 7f99caa06740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x559d78d50ef0 con 0x559d78e38160 2024-05-08T05:50:12.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.961+0000 7f99caa06740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x559d78deef10 con 0x559d78e31ce0 2024-05-08T05:50:12.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.961+0000 7f99caa06740 10 client.0 _ll_register_callbacks cb 0x559d78d0c320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:50:12.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.961+0000 7f99c5a84640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e1b280 0x559d78e36080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.961+0000 7f99c5a84640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e1b280 0x559d78e36080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60084/0 (socket says 172.21.15.45:60084) 2024-05-08T05:50:12.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.961+0000 7f99c5a84640 1 -- 172.21.15.45:0/1388944951 learned_addr learned my addr 172.21.15.45:0/1388944951 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:12.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.961+0000 7f99c6285640 1 --2- 172.21.15.45:0/1388944951 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559d78e31ce0 0x559d78e34740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.961+0000 7f99c5283640 1 --2- 172.21.15.45:0/1388944951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e38160 0x559d78e34200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.961+0000 7f99c5a84640 1 -- 172.21.15.45:0/1388944951 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559d78e31ce0 msgr2=0x559d78e34740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.961+0000 7f99c5a84640 1 --2- 172.21.15.45:0/1388944951 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559d78e31ce0 0x559d78e34740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.961+0000 7f99c5a84640 1 -- 172.21.15.45:0/1388944951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e38160 msgr2=0x559d78e34200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.961+0000 7f99c5a84640 1 --2- 172.21.15.45:0/1388944951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e38160 0x559d78e34200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.961+0000 7f99c5a84640 1 -- 172.21.15.45:0/1388944951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d78b307d0 con 0x559d78e1b280 2024-05-08T05:50:12.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.961+0000 7f99c5a84640 1 --2- 172.21.15.45:0/1388944951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e1b280 0x559d78e36080 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00cea0 tx=0x7f99bc004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896edbedf498b550 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.961+0000 7f99a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:50:12.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.962+0000 7f99a0ff9640 1 -- 172.21.15.45:0/1388944951 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc01f420 con 0x559d78e1b280 2024-05-08T05:50:12.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.962+0000 7f99a0ff9640 1 -- 172.21.15.45:0/1388944951 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6187+0+0 (secure 0 0 0) 0x7f99bc01f5c0 con 0x559d78e1b280 2024-05-08T05:50:12.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.962+0000 7f99caa06740 1 -- 172.21.15.45:0/1388944951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99b0080d50 con 0x559d78e1b280 2024-05-08T05:50:12.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.962+0000 7f99a0ff9640 1 -- 172.21.15.45:0/1388944951 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f99bc02c050 con 0x559d78e1b280 2024-05-08T05:50:12.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.962+0000 7f99caa06740 10 client.7920 fetch_fsmap learned FSMap version 88 2024-05-08T05:50:12.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.962+0000 7f99caa06740 1 -- 172.21.15.45:0/1388944951 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x559d78a663f0 con 0x559d78e1b280 2024-05-08T05:50:12.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.962+0000 7f99a0ff9640 1 -- 172.21.15.45:0/1388944951 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b0080d50 con 0x559d78e1b280 2024-05-08T05:50:12.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.962+0000 7f99a0ff9640 1 -- 172.21.15.45:0/1388944951 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f99bc02f020 con 0x559d78e1b280 2024-05-08T05:50:12.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.962+0000 7f99caa06740 10 client.7920 fetch_fsmap finished waiting for FSMap version 88 2024-05-08T05:50:12.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.962+0000 7f99caa06740 10 client.7920 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-08T05:50:12.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.962+0000 7f99caa06740 20 client.7920 populate_metadata read hostname 'smithi045' 2024-05-08T05:50:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.962+0000 7f99caa06740 1 --2- 172.21.15.45:0/1388944951 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x559d78e1bc40 0x559d78e1c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.962+0000 7f99caa06740 4 client.7920 mds_command: new command op to 7806 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:50:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.962+0000 7f99caa06740 1 -- 172.21.15.45:0/1388944951 --> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d78d62b40 con 0x559d78e1bc40 2024-05-08T05:50:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.962+0000 7f99c5283640 1 --2- 172.21.15.45:0/1388944951 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x559d78e1bc40 0x559d78e1c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.963+0000 7f99a0ff9640 10 client.7920 ms_handle_connect on v2:172.21.15.45:6834/2729952281 2024-05-08T05:50:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.963+0000 7f99c5283640 1 --2- 172.21.15.45:0/1388944951 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x559d78e1bc40 0x559d78e1c0d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c04f5f2834c8ac4 server_cookie=ea0801943f22f2f9 in_seq=0 out_seq=0 2024-05-08T05:50:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.963+0000 7f99a0ff9640 1 -- 172.21.15.45:0/1388944951 <== mds.0 v2:172.21.15.45:6834/2729952281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559d78d62b40 con 0x559d78e1bc40 2024-05-08T05:50:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.963+0000 7f99a0ff9640 10 client.7920 handle_command_reply: tid=0 2024-05-08T05:50:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.963+0000 7f99caa06740 1 client.7920 shutdown 2024-05-08T05:50:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.963+0000 7f99caa06740 1 -- 172.21.15.45:0/1388944951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e1b280 msgr2=0x559d78e36080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.963+0000 7f99caa06740 1 --2- 172.21.15.45:0/1388944951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e1b280 0x559d78e36080 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00cea0 tx=0x7f99bc004680 comp rx=0 tx=0).stop 2024-05-08T05:50:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.963+0000 7f99caa06740 1 -- 172.21.15.45:0/1388944951 shutdown_connections 2024-05-08T05:50:12.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.963+0000 7f99caa06740 1 --2- 172.21.15.45:0/1388944951 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x559d78e1bc40 0x559d78e1c0d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.963+0000 7f99caa06740 1 --2- 172.21.15.45:0/1388944951 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559d78e31ce0 0x559d78e34740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.963+0000 7f99caa06740 1 --2- 172.21.15.45:0/1388944951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e38160 0x559d78e34200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.963+0000 7f99a0ff9640 0 client.7920 ms_handle_reset on v2:172.21.15.45:6834/2729952281 2024-05-08T05:50:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.963+0000 7f99caa06740 1 --2- 172.21.15.45:0/1388944951 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e1b280 0x559d78e36080 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.963+0000 7f99caa06740 1 -- 172.21.15.45:0/1388944951 >> 172.21.15.45:0/1388944951 conn(0x559d78e1ab40 msgr2=0x559d78dea230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.963+0000 7f99caa06740 1 -- 172.21.15.45:0/1388944951 shutdown_connections 2024-05-08T05:50:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.964+0000 7f99caa06740 1 -- 172.21.15.45:0/1388944951 wait complete. 2024-05-08T05:50:12.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.964+0000 7f99caa06740 20 client.7920 trim_cache size 0 max 16384 2024-05-08T05:50:12.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.982+0000 7f99caa06740 1 Processor -- start 2024-05-08T05:50:12.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.982+0000 7f99caa06740 1 -- start start 2024-05-08T05:50:12.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.982+0000 7f99caa06740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559d78e1b280 0x559d78e45310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.982+0000 7f99caa06740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e31ce0 0x559d78dee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.982+0000 7f99caa06740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e38160 0x559d78e35400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.982+0000 7f99caa06740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f99c0202000 con 0x559d78e38160 2024-05-08T05:50:12.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.982+0000 7f99caa06740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f99c02010a0 con 0x559d78e31ce0 2024-05-08T05:50:12.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.982+0000 7f99caa06740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f99c0200430 con 0x559d78e1b280 2024-05-08T05:50:12.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.982+0000 7f99c6285640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e38160 0x559d78e35400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.982+0000 7f99c6285640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e38160 0x559d78e35400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60096/0 (socket says 172.21.15.45:60096) 2024-05-08T05:50:12.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.983+0000 7f99c6285640 1 -- 172.21.15.45:0/1980586341 learned_addr learned my addr 172.21.15.45:0/1980586341 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:12.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.983+0000 7f99c5283640 1 --2- 172.21.15.45:0/1980586341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e31ce0 0x559d78dee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.983+0000 7f99c5a84640 1 --2- 172.21.15.45:0/1980586341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559d78e1b280 0x559d78e45310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.983+0000 7f99c6285640 1 -- 172.21.15.45:0/1980586341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559d78e1b280 msgr2=0x559d78e45310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.983+0000 7f99c6285640 1 --2- 172.21.15.45:0/1980586341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559d78e1b280 0x559d78e45310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.983+0000 7f99c6285640 1 -- 172.21.15.45:0/1980586341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e31ce0 msgr2=0x559d78dee3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.983+0000 7f99c6285640 1 --2- 172.21.15.45:0/1980586341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e31ce0 0x559d78dee3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.983+0000 7f99c6285640 1 -- 172.21.15.45:0/1980586341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d78a260f0 con 0x559d78e38160 2024-05-08T05:50:12.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.983+0000 7f99c5283640 1 --2- 172.21.15.45:0/1980586341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e31ce0 0x559d78dee3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:12.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.983+0000 7f99c6285640 1 --2- 172.21.15.45:0/1980586341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e38160 0x559d78e35400 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f99b8007e10 tx=0x7f99b8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7488cbceafd7969e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.983+0000 7f99a2ffd640 1 -- 172.21.15.45:0/1980586341 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b80106a0 con 0x559d78e38160 2024-05-08T05:50:12.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.983+0000 7f99a2ffd640 1 -- 172.21.15.45:0/1980586341 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b8010840 con 0x559d78e38160 2024-05-08T05:50:12.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.983+0000 7f99a2ffd640 1 -- 172.21.15.45:0/1980586341 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8010c30 con 0x559d78e38160 2024-05-08T05:50:12.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.984+0000 7f99caa06740 1 -- 172.21.15.45:0/1980586341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e38160 msgr2=0x559d78e35400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.984+0000 7f99caa06740 1 --2- 172.21.15.45:0/1980586341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e38160 0x559d78e35400 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f99b8007e10 tx=0x7f99b8012820 comp rx=0 tx=0).stop 2024-05-08T05:50:12.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.984+0000 7f99caa06740 1 -- 172.21.15.45:0/1980586341 shutdown_connections 2024-05-08T05:50:12.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.984+0000 7f99caa06740 1 --2- 172.21.15.45:0/1980586341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e38160 0x559d78e35400 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.984+0000 7f99caa06740 1 --2- 172.21.15.45:0/1980586341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e31ce0 0x559d78dee3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.984+0000 7f99caa06740 1 --2- 172.21.15.45:0/1980586341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559d78e1b280 0x559d78e45310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.984+0000 7f99caa06740 1 -- 172.21.15.45:0/1980586341 >> 172.21.15.45:0/1980586341 conn(0x559d78e1ab40 msgr2=0x559d78e4f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:12.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.984+0000 7f99caa06740 1 -- 172.21.15.45:0/1980586341 shutdown_connections 2024-05-08T05:50:12.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.984+0000 7f99caa06740 1 -- 172.21.15.45:0/1980586341 wait complete. 2024-05-08T05:50:12.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.984+0000 7f99caa06740 1 -- start start 2024-05-08T05:50:12.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.984+0000 7f99caa06740 1 Processor -- start 2024-05-08T05:50:12.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.985+0000 7f99caa06740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e1b280 0x559d78e1a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.985+0000 7f99caa06740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e31ce0 0x559d78e2af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.985+0000 7f99caa06740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559d78e38160 0x559d78e351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.985+0000 7f99caa06740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559d78e59cd0 con 0x559d78e31ce0 2024-05-08T05:50:12.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.985+0000 7f99caa06740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f99c0200a80 con 0x559d78e1b280 2024-05-08T05:50:12.990 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.985+0000 7f99caa06740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x559d78e1e800 con 0x559d78e38160 2024-05-08T05:50:12.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.985+0000 7f99caa06740 10 client.0 _ll_register_callbacks cb 0x7f99bc00a440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:50:12.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.985+0000 7f99c5283640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e31ce0 0x559d78e2af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.985+0000 7f99c5283640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e31ce0 0x559d78e2af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60112/0 (socket says 172.21.15.45:60112) 2024-05-08T05:50:12.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.985+0000 7f99c5283640 1 -- 172.21.15.45:0/1702428441 learned_addr learned my addr 172.21.15.45:0/1702428441 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:12.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.985+0000 7f99c5a84640 1 --2- 172.21.15.45:0/1702428441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e1b280 0x559d78e1a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.985+0000 7f99c6285640 1 --2- 172.21.15.45:0/1702428441 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559d78e38160 0x559d78e351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99c5283640 1 -- 172.21.15.45:0/1702428441 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559d78e38160 msgr2=0x559d78e351f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99c5283640 1 --2- 172.21.15.45:0/1702428441 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559d78e38160 0x559d78e351f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99c5283640 1 -- 172.21.15.45:0/1702428441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e1b280 msgr2=0x559d78e1a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:12.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99c5283640 1 --2- 172.21.15.45:0/1702428441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e1b280 0x559d78e1a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99c5283640 1 -- 172.21.15.45:0/1702428441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d78e45ad0 con 0x559d78e31ce0 2024-05-08T05:50:12.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99c5283640 1 --2- 172.21.15.45:0/1702428441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e31ce0 0x559d78e2af90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f99b0035040 tx=0x7f99b003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fcddab8086e051c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:12.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:50:12.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99a0ff9640 1 -- 172.21.15.45:0/1702428441 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b00b5060 con 0x559d78e31ce0 2024-05-08T05:50:12.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99a0ff9640 1 -- 172.21.15.45:0/1702428441 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6187+0+0 (secure 0 0 0) 0x7f99b0089b80 con 0x559d78e31ce0 2024-05-08T05:50:12.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99caa06740 1 -- 172.21.15.45:0/1702428441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99b8010840 con 0x559d78e31ce0 2024-05-08T05:50:12.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99a0ff9640 1 -- 172.21.15.45:0/1702428441 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f99b0080d50 con 0x559d78e31ce0 2024-05-08T05:50:12.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99caa06740 10 client.7926 fetch_fsmap learned FSMap version 88 2024-05-08T05:50:12.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99caa06740 1 -- 172.21.15.45:0/1702428441 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x559d78e45810 con 0x559d78e31ce0 2024-05-08T05:50:12.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99a0ff9640 1 -- 172.21.15.45:0/1702428441 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b8010840 con 0x559d78e31ce0 2024-05-08T05:50:12.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99a0ff9640 1 -- 172.21.15.45:0/1702428441 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f99b003a030 con 0x559d78e31ce0 2024-05-08T05:50:12.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99caa06740 10 client.7926 fetch_fsmap finished waiting for FSMap version 88 2024-05-08T05:50:12.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99caa06740 10 client.7926 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-08T05:50:12.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99caa06740 20 client.7926 populate_metadata read hostname 'smithi045' 2024-05-08T05:50:12.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99caa06740 1 --2- 172.21.15.45:0/1702428441 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x559d78dee300 0x559d78e35730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:12.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99caa06740 4 client.7926 mds_command: new command op to 7806 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:50:12.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.986+0000 7f99caa06740 1 -- 172.21.15.45:0/1702428441 --> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559d78e37dc0 con 0x559d78dee300 2024-05-08T05:50:12.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.987+0000 7f99c5a84640 1 --2- 172.21.15.45:0/1702428441 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x559d78dee300 0x559d78e35730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:12.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.987+0000 7f99c5a84640 1 --2- 172.21.15.45:0/1702428441 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x559d78dee300 0x559d78e35730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75a817df86d00609 server_cookie=e76cd1c494cfefd2 in_seq=0 out_seq=0 2024-05-08T05:50:12.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.987+0000 7f99a0ff9640 10 client.7926 ms_handle_connect on v2:172.21.15.45:6834/2729952281 2024-05-08T05:50:12.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.987+0000 7f99a0ff9640 1 -- 172.21.15.45:0/1702428441 <== mds.0 v2:172.21.15.45:6834/2729952281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x559d78e37dc0 con 0x559d78dee300 2024-05-08T05:50:12.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.987+0000 7f99a0ff9640 10 client.7926 handle_command_reply: tid=0 2024-05-08T05:50:12.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.987+0000 7f99caa06740 1 client.7926 shutdown 2024-05-08T05:50:12.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.987+0000 7f99caa06740 1 -- 172.21.15.45:0/1702428441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e31ce0 msgr2=0x559d78e2af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.987+0000 7f99caa06740 1 --2- 172.21.15.45:0/1702428441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e31ce0 0x559d78e2af90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f99b0035040 tx=0x7f99b003b000 comp rx=0 tx=0).stop 2024-05-08T05:50:12.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.987+0000 7f99caa06740 1 -- 172.21.15.45:0/1702428441 shutdown_connections 2024-05-08T05:50:12.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.987+0000 7f99caa06740 1 --2- 172.21.15.45:0/1702428441 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x559d78dee300 0x559d78e35730 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.987+0000 7f99caa06740 1 --2- 172.21.15.45:0/1702428441 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x559d78e38160 0x559d78e351f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.987+0000 7f99caa06740 1 --2- 172.21.15.45:0/1702428441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559d78e31ce0 0x559d78e2af90 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.987+0000 7f99caa06740 1 --2- 172.21.15.45:0/1702428441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x559d78e1b280 0x559d78e1a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.987+0000 7f99a0ff9640 0 client.7926 ms_handle_reset on v2:172.21.15.45:6834/2729952281 2024-05-08T05:50:12.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.987+0000 7f99caa06740 1 -- 172.21.15.45:0/1702428441 >> 172.21.15.45:0/1702428441 conn(0x559d78e1ab40 msgr2=0x559d78e38a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:12.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.987+0000 7f99caa06740 1 -- 172.21.15.45:0/1702428441 shutdown_connections 2024-05-08T05:50:12.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.987+0000 7f99caa06740 1 -- 172.21.15.45:0/1702428441 wait complete. 2024-05-08T05:50:12.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.987+0000 7f99caa06740 20 client.7926 trim_cache size 0 max 16384 2024-05-08T05:50:12.994 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:50:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T05:50:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": { 2024-05-08T05:50:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "e76b6156-c6d9-4618-90fc-52e2d08af961": { 2024-05-08T05:50:12.994 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/", 2024-05-08T05:50:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "tag": "e76b6156-c6d9-4618-90fc-52e2d08af961", 2024-05-08T05:50:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: "options": "recursive,repair" 2024-05-08T05:50:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:50:12.995 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:50:12.995 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:50:12.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.988+0000 7f99c7ce8640 1 -- 172.21.15.45:0/3531168820 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99a40762d0 msgr2=0x7f99a4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.988+0000 7f99c7ce8640 1 --2- 172.21.15.45:0/3531168820 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99a40762d0 0x7f99a4078790 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f99c0161c20 tx=0x7f99bc008040 comp rx=0 tx=0).stop 2024-05-08T05:50:12.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.988+0000 7f99c7ce8640 1 -- 172.21.15.45:0/3531168820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c0071110 msgr2=0x7f99c00b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:12.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.988+0000 7f99c7ce8640 1 --2- 172.21.15.45:0/3531168820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c0071110 0x7f99c00b78e0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009870 tx=0x7f99b00046c0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.988+0000 7f99c7ce8640 1 -- 172.21.15.45:0/3531168820 shutdown_connections 2024-05-08T05:50:12.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.988+0000 7f99c7ce8640 1 --2- 172.21.15.45:0/3531168820 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99a40762d0 0x7f99a4078790 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.988+0000 7f99c7ce8640 1 --2- 172.21.15.45:0/3531168820 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99c00d20c0 0x7f99c00b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.988+0000 7f99c7ce8640 1 --2- 172.21.15.45:0/3531168820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99c0071110 0x7f99c00b78e0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.988+0000 7f99c7ce8640 1 --2- 172.21.15.45:0/3531168820 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99c0070d30 0x7f99c015e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:12.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.988+0000 7f99c7ce8640 1 -- 172.21.15.45:0/3531168820 >> 172.21.15.45:0/3531168820 conn(0x7f99c00d2e90 msgr2=0x7f99c006ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:12.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.988+0000 7f99c7ce8640 1 -- 172.21.15.45:0/3531168820 shutdown_connections 2024-05-08T05:50:12.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:12.988+0000 7f99c7ce8640 1 -- 172.21.15.45:0/3531168820 wait complete. 2024-05-08T05:50:13.002 INFO:tasks.cephfs.filesystem:scrub status for tag:e76b6156-c6d9-4618-90fc-52e2d08af961 - {'path': '/', 'tag': 'e76b6156-c6d9-4618-90fc-52e2d08af961', 'options': 'recursive,repair'} 2024-05-08T05:50:13.183 INFO:tasks.ceph.mds.a.smithi045.stderr:2024-05-08T05:50:13.183+0000 7fbed14e9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:50:13.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:13.348+0000 7ff43a7fc640 1 -- 172.21.15.45:0/1557537917 <== mon.0 v2:172.21.15.45:3300/0 7 ==== log(8 entries from seq 1 at 2024-05-08T05:50:12.829952+0000) ==== 1407+0+0 (secure 0 0 0) 0x7ff4440960c0 con 0x7ff4480cfdc0 2024-05-08T05:50:13.349 INFO:teuthology.orchestra.run.smithi045.stdout: cluster: 2024-05-08T05:50:13.349 INFO:teuthology.orchestra.run.smithi045.stdout: id: e9e1170a-fd27-4625-97a9-5d5f104d13a2 2024-05-08T05:50:13.349 INFO:teuthology.orchestra.run.smithi045.stdout: health: HEALTH_OK 2024-05-08T05:50:13.349 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:50:13.349 INFO:teuthology.orchestra.run.smithi045.stdout: services: 2024-05-08T05:50:13.349 INFO:teuthology.orchestra.run.smithi045.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2024-05-08T05:50:13.349 INFO:teuthology.orchestra.run.smithi045.stdout: mgr: x(active, since 8m), standbys: y 2024-05-08T05:50:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: mds: 1/1 daemons up, 3 standby 2024-05-08T05:50:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: osd: 8 osds: 8 up (since 8m), 8 in (since 9m) 2024-05-08T05:50:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:50:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: data: 2024-05-08T05:50:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: volumes: 1/1 healthy 2024-05-08T05:50:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: pools: 3 pools, 97 pgs 2024-05-08T05:50:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: objects: 30 objects, 19 MiB 2024-05-08T05:50:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-05-08T05:50:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: pgs: 97 active+clean 2024-05-08T05:50:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:50:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: io: 2024-05-08T05:50:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: client: 1.8 KiB/s rd, 102 B/s wr, 1 op/s rd, 0 op/s wr 2024-05-08T05:50:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:50:13.350 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:50:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:50:12.829952+0000 mds.a [INF] scrub queued for path: / 2024-05-08T05:50:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:50:12.829955+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-05-08T05:50:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:50:12.829960+0000 mds.a [INF] scrub summary: active paths [/] 2024-05-08T05:50:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:50:12.830495+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-05-08T05:50:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:50:12.830500+0000 mds.a [ERR] inode table repaired for inode: 0x10000000001 2024-05-08T05:50:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:50:12.830825+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-05-08T05:50:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:50:12.830829+0000 mds.a [ERR] inode table repaired for inode: 0x10000000002 2024-05-08T05:50:13.351 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:50:12.831063+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-05-08T05:50:15.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:15.355+0000 7ff43a7fc640 1 -- 172.21.15.45:0/1557537917 <== mon.0 v2:172.21.15.45:3300/0 8 ==== log(6 entries from seq 150 at 2024-05-08T05:50:14.578548+0000) ==== 1667+0+0 (secure 0 0 0) 0x7ff444011dc0 con 0x7ff4480cfdc0 2024-05-08T05:50:16.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:16.366+0000 7ff43a7fc640 1 -- 172.21.15.45:0/1557537917 <== mon.0 v2:172.21.15.45:3300/0 9 ==== log(3 entries from seq 941 at 2024-05-08T05:50:15.359983+0000) ==== 821+0+0 (secure 0 0 0) 0x7ff44405d000 con 0x7ff4480cfdc0 2024-05-08T05:50:18.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:18.373+0000 7ff43a7fc640 1 -- 172.21.15.45:0/1557537917 <== mon.0 v2:172.21.15.45:3300/0 10 ==== log(1 entries from seq 9 at 2024-05-08T05:50:17.260403+0000) ==== 189+0+0 (secure 0 0 0) 0x7ff4440696b0 con 0x7ff4480cfdc0 2024-05-08T05:50:18.373 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:50:17.260403+0000 mds.a [INF] scrub summary: idle 2024-05-08T05:50:35.238 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:50:35.242 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:50:43.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:50:43.004 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-05-08T05:50:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.123+0000 7f0e42c6d640 1 Processor -- start 2024-05-08T05:50:43.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.123+0000 7f0e42c6d640 1 -- start start 2024-05-08T05:50:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.124+0000 7f0e42c6d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e3c1629b0 0x7f0e3c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.124+0000 7f0e42c6d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0e3c070d20 0x7f0e3c0711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.124+0000 7f0e42c6d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e3c06e110 con 0x7f0e3c0b2490 2024-05-08T05:50:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.124+0000 7f0e42c6d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f0e3c070a40 con 0x7f0e3c070d20 2024-05-08T05:50:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.124+0000 7f0e42c6d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0e3c06dec0 con 0x7f0e3c1629b0 2024-05-08T05:50:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.124+0000 7f0e4120a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0e3c070d20 0x7f0e3c0711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.124+0000 7f0e4120a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0e3c070d20 0x7f0e3c0711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50368/0 (socket says 172.21.15.45:50368) 2024-05-08T05:50:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.124+0000 7f0e4120a640 1 -- 172.21.15.45:0/649507603 learned_addr learned my addr 172.21.15.45:0/649507603 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.124+0000 7f0e3bfff640 1 --1- 172.21.15.45:0/649507603 >> v1:172.21.15.45:6789/0 conn(0x7f0e3c0b2490 0x7f0e3c070330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60426/0 (socket says 172.21.15.45:60426) 2024-05-08T05:50:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.124+0000 7f0e40a09640 1 --2- 172.21.15.45:0/649507603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e3c1629b0 0x7f0e3c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.124+0000 7f0e4120a640 1 -- 172.21.15.45:0/649507603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e3c1629b0 msgr2=0x7f0e3c06eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.124+0000 7f0e4120a640 1 --2- 172.21.15.45:0/649507603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e3c1629b0 0x7f0e3c06eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.124+0000 7f0e4120a640 1 -- 172.21.15.45:0/649507603 >> v1:172.21.15.45:6789/0 conn(0x7f0e3c0b2490 legacy=0x7f0e3c070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.124+0000 7f0e4120a640 1 -- 172.21.15.45:0/649507603 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3c06e3f0 con 0x7f0e3c070d20 2024-05-08T05:50:43.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.125+0000 7f0e4120a640 1 --2- 172.21.15.45:0/649507603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e3c070d20 0x7f0e3c0711a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009e50 tx=0x7f0e3402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146a28d83996eead server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:43.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.125+0000 7f0e3b7fe640 1 -- 172.21.15.45:0/649507603 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34036070 con 0x7f0e3c070d20 2024-05-08T05:50:43.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.125+0000 7f0e3b7fe640 1 -- 172.21.15.45:0/649507603 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e34031600 con 0x7f0e3c070d20 2024-05-08T05:50:43.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.125+0000 7f0e3b7fe640 1 -- 172.21.15.45:0/649507603 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e340319d0 con 0x7f0e3c070d20 2024-05-08T05:50:43.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.126+0000 7f0e42c6d640 1 -- 172.21.15.45:0/649507603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e3c070d20 msgr2=0x7f0e3c0711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.126+0000 7f0e42c6d640 1 --2- 172.21.15.45:0/649507603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e3c070d20 0x7f0e3c0711a0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009e50 tx=0x7f0e3402a710 comp rx=0 tx=0).stop 2024-05-08T05:50:43.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.126+0000 7f0e42c6d640 1 -- 172.21.15.45:0/649507603 shutdown_connections 2024-05-08T05:50:43.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.126+0000 7f0e42c6d640 1 --2- 172.21.15.45:0/649507603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e3c070d20 0x7f0e3c0711a0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.126+0000 7f0e42c6d640 1 --2- 172.21.15.45:0/649507603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e3c1629b0 0x7f0e3c06eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.126+0000 7f0e42c6d640 1 -- 172.21.15.45:0/649507603 >> 172.21.15.45:0/649507603 conn(0x7f0e3c0b49d0 msgr2=0x7f0e3c0b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:43.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.126+0000 7f0e42c6d640 1 -- 172.21.15.45:0/649507603 shutdown_connections 2024-05-08T05:50:43.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.126+0000 7f0e42c6d640 1 -- 172.21.15.45:0/649507603 wait complete. 2024-05-08T05:50:43.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.127+0000 7f0e42c6d640 1 Processor -- start 2024-05-08T05:50:43.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.127+0000 7f0e42c6d640 1 -- start start 2024-05-08T05:50:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.128+0000 7f0e42c6d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e3c070d20 0x7f0e3c15bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.128+0000 7f0e42c6d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e3c0b2490 0x7f0e3c0b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.128+0000 7f0e42c6d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e3c1629b0 0x7f0e3c160f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.128+0000 7f0e42c6d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c06f410 con 0x7f0e3c0b2490 2024-05-08T05:50:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.128+0000 7f0e42c6d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c06e9b0 con 0x7f0e3c070d20 2024-05-08T05:50:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.128+0000 7f0e42c6d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0e3c0afcb0 con 0x7f0e3c1629b0 2024-05-08T05:50:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.128+0000 7f0e3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e3c0b2490 0x7f0e3c0b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.128+0000 7f0e3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e3c0b2490 0x7f0e3c0b76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50372/0 (socket says 172.21.15.45:50372) 2024-05-08T05:50:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.128+0000 7f0e3bfff640 1 -- 172.21.15.45:0/2312594025 learned_addr learned my addr 172.21.15.45:0/2312594025 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.128+0000 7f0e40a09640 1 --2- 172.21.15.45:0/2312594025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e3c070d20 0x7f0e3c15bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.128+0000 7f0e4120a640 1 --2- 172.21.15.45:0/2312594025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e3c1629b0 0x7f0e3c160f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.128+0000 7f0e3bfff640 1 -- 172.21.15.45:0/2312594025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e3c1629b0 msgr2=0x7f0e3c160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.128+0000 7f0e3bfff640 1 --2- 172.21.15.45:0/2312594025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e3c1629b0 0x7f0e3c160f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.128+0000 7f0e3bfff640 1 -- 172.21.15.45:0/2312594025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e3c070d20 msgr2=0x7f0e3c15bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.128+0000 7f0e3bfff640 1 --2- 172.21.15.45:0/2312594025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e3c070d20 0x7f0e3c15bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.128+0000 7f0e3bfff640 1 -- 172.21.15.45:0/2312594025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3c1f5660 con 0x7f0e3c0b2490 2024-05-08T05:50:43.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.129+0000 7f0e3bfff640 1 --2- 172.21.15.45:0/2312594025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e3c0b2490 0x7f0e3c0b76d0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000c500 tx=0x7f0e3000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:43.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.129+0000 7f0e397fa640 1 -- 172.21.15.45:0/2312594025 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e30018070 con 0x7f0e3c0b2490 2024-05-08T05:50:43.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.129+0000 7f0e397fa640 1 -- 172.21.15.45:0/2312594025 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e30007500 con 0x7f0e3c0b2490 2024-05-08T05:50:43.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.129+0000 7f0e42c6d640 1 -- 172.21.15.45:0/2312594025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3c1f5940 con 0x7f0e3c0b2490 2024-05-08T05:50:43.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.129+0000 7f0e397fa640 1 -- 172.21.15.45:0/2312594025 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e300049e0 con 0x7f0e3c0b2490 2024-05-08T05:50:43.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.129+0000 7f0e42c6d640 1 -- 172.21.15.45:0/2312594025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e3c1f5f00 con 0x7f0e3c0b2490 2024-05-08T05:50:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.131+0000 7f0e397fa640 1 -- 172.21.15.45:0/2312594025 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0e3001b070 con 0x7f0e3c0b2490 2024-05-08T05:50:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.132+0000 7f0e397fa640 1 --2- 172.21.15.45:0/2312594025 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0e280762d0 0x7f0e28078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.132+0000 7f0e397fa640 1 -- 172.21.15.45:0/2312594025 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6274+0+0 (secure 0 0 0) 0x7f0e30096be0 con 0x7f0e3c0b2490 2024-05-08T05:50:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.132+0000 7f0e40a09640 1 --2- 172.21.15.45:0/2312594025 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0e280762d0 0x7f0e28078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.132+0000 7f0e45985740 1 Processor -- start 2024-05-08T05:50:43.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.132+0000 7f0e45985740 1 -- start start 2024-05-08T05:50:43.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.133+0000 7f0e40a09640 1 --2- 172.21.15.45:0/2312594025 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0e280762d0 0x7f0e28078790 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0d0510 tx=0x7f0e2c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:43.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.133+0000 7f0e45985740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 0x555570cf4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.133+0000 7f0e45985740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570d24600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.133+0000 7f0e45985740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d24b40 0x555570cf9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.133+0000 7f0e45985740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555570cc1690 con 0x555570d25280 2024-05-08T05:50:43.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.133+0000 7f0e45985740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555570c5ef30 con 0x555570d24b40 2024-05-08T05:50:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.133+0000 7f0e45985740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x555570c17b20 con 0x555570d42160 2024-05-08T05:50:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.133+0000 7f0e3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570d24600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.133+0000 7f0e3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570d24600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50386/0 (socket says 172.21.15.45:50386) 2024-05-08T05:50:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.133+0000 7f0e3bfff640 1 -- 172.21.15.45:0/1395101247 learned_addr learned my addr 172.21.15.45:0/1395101247 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.133+0000 7f0e40a09640 1 --2- 172.21.15.45:0/1395101247 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 0x555570cf4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.133+0000 7f0e4120a640 1 --2- 172.21.15.45:0/1395101247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d24b40 0x555570cf9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.133+0000 7f0e3bfff640 1 -- 172.21.15.45:0/1395101247 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 msgr2=0x555570cf4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.133+0000 7f0e3bfff640 1 --2- 172.21.15.45:0/1395101247 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 0x555570cf4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.133+0000 7f0e3bfff640 1 -- 172.21.15.45:0/1395101247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d24b40 msgr2=0x555570cf9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.133+0000 7f0e3bfff640 1 --2- 172.21.15.45:0/1395101247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d24b40 0x555570cf9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.133+0000 7f0e3bfff640 1 -- 172.21.15.45:0/1395101247 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555570c0fe60 con 0x555570d25280 2024-05-08T05:50:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.133+0000 7f0e40a09640 1 --2- 172.21.15.45:0/1395101247 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 0x555570cf4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.134+0000 7f0e4120a640 1 --2- 172.21.15.45:0/1395101247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d24b40 0x555570cf9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.134+0000 7f0e3bfff640 1 --2- 172.21.15.45:0/1395101247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570d24600 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f0e30094ed0 tx=0x7f0e3005f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2205ffa6e7b5fc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:43.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.134+0000 7f0e227fc640 1 -- 172.21.15.45:0/1395101247 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e300791f0 con 0x555570d25280 2024-05-08T05:50:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.134+0000 7f0e227fc640 1 -- 172.21.15.45:0/1395101247 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e30060390 con 0x555570d25280 2024-05-08T05:50:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.134+0000 7f0e227fc640 1 -- 172.21.15.45:0/1395101247 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3005fc90 con 0x555570d25280 2024-05-08T05:50:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.134+0000 7f0e45985740 1 -- 172.21.15.45:0/1395101247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 msgr2=0x555570d24600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.134+0000 7f0e45985740 1 --2- 172.21.15.45:0/1395101247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570d24600 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f0e30094ed0 tx=0x7f0e3005f710 comp rx=0 tx=0).stop 2024-05-08T05:50:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.134+0000 7f0e45985740 1 -- 172.21.15.45:0/1395101247 shutdown_connections 2024-05-08T05:50:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.134+0000 7f0e45985740 1 --2- 172.21.15.45:0/1395101247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d24b40 0x555570cf9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.134+0000 7f0e45985740 1 --2- 172.21.15.45:0/1395101247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570d24600 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.134+0000 7f0e45985740 1 --2- 172.21.15.45:0/1395101247 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 0x555570cf4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.134+0000 7f0e45985740 1 -- 172.21.15.45:0/1395101247 >> 172.21.15.45:0/1395101247 conn(0x555570cf6d60 msgr2=0x555570d3f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.135+0000 7f0e45985740 1 -- 172.21.15.45:0/1395101247 shutdown_connections 2024-05-08T05:50:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.135+0000 7f0e45985740 1 -- 172.21.15.45:0/1395101247 wait complete. 2024-05-08T05:50:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.135+0000 7f0e45985740 1 -- start start 2024-05-08T05:50:43.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.135+0000 7f0e45985740 1 Processor -- start 2024-05-08T05:50:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.136+0000 7f0e45985740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570d3dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.136+0000 7f0e45985740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 0x555570d37800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.136+0000 7f0e45985740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d1f9e0 0x555570d40650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.136+0000 7f0e45985740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555570c1b5e0 con 0x555570d25280 2024-05-08T05:50:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.136+0000 7f0e45985740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x555570c5aef0 con 0x555570d1f9e0 2024-05-08T05:50:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.136+0000 7f0e45985740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x555570cf8f10 con 0x555570d42160 2024-05-08T05:50:43.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.136+0000 7f0e45985740 10 client.0 _ll_register_callbacks cb 0x555570c16320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:50:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.136+0000 7f0e40a09640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570d3dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.136+0000 7f0e40a09640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570d3dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50388/0 (socket says 172.21.15.45:50388) 2024-05-08T05:50:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.136+0000 7f0e40a09640 1 -- 172.21.15.45:0/1408004367 learned_addr learned my addr 172.21.15.45:0/1408004367 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.136+0000 7f0e3bfff640 1 --2- 172.21.15.45:0/1408004367 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 0x555570d37800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.137+0000 7f0e4120a640 1 --2- 172.21.15.45:0/1408004367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d1f9e0 0x555570d40650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.137+0000 7f0e40a09640 1 -- 172.21.15.45:0/1408004367 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 msgr2=0x555570d37800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.137+0000 7f0e40a09640 1 --2- 172.21.15.45:0/1408004367 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 0x555570d37800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.137+0000 7f0e40a09640 1 -- 172.21.15.45:0/1408004367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d1f9e0 msgr2=0x555570d40650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.137+0000 7f0e40a09640 1 --2- 172.21.15.45:0/1408004367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d1f9e0 0x555570d40650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.137+0000 7f0e40a09640 1 -- 172.21.15.45:0/1408004367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555570a3a7d0 con 0x555570d25280 2024-05-08T05:50:43.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.137+0000 7f0e4120a640 1 --2- 172.21.15.45:0/1408004367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d1f9e0 0x555570d40650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.137+0000 7f0e40a09640 1 --2- 172.21.15.45:0/1408004367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570d3dd40 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0156f0 tx=0x7f0e2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12df8f305ae58a88 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.137+0000 7f0e1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:50:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.137+0000 7f0e1b7fe640 1 -- 172.21.15.45:0/1408004367 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c025070 con 0x555570d25280 2024-05-08T05:50:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.137+0000 7f0e1b7fe640 1 -- 172.21.15.45:0/1408004367 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 6274+0+0 (secure 0 0 0) 0x7f0e2c014730 con 0x555570d25280 2024-05-08T05:50:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.137+0000 7f0e45985740 1 -- 172.21.15.45:0/1408004367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e30060390 con 0x555570d25280 2024-05-08T05:50:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.137+0000 7f0e1b7fe640 1 -- 172.21.15.45:0/1408004367 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f0e2c030050 con 0x555570d25280 2024-05-08T05:50:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.137+0000 7f0e45985740 10 client.7941 fetch_fsmap learned FSMap version 88 2024-05-08T05:50:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.137+0000 7f0e45985740 1 -- 172.21.15.45:0/1408004367 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x5555709703f0 con 0x555570d25280 2024-05-08T05:50:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.137+0000 7f0e1b7fe640 1 -- 172.21.15.45:0/1408004367 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e30060390 con 0x555570d25280 2024-05-08T05:50:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.137+0000 7f0e1b7fe640 1 -- 172.21.15.45:0/1408004367 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f0e2c033020 con 0x555570d25280 2024-05-08T05:50:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.138+0000 7f0e45985740 10 client.7941 fetch_fsmap finished waiting for FSMap version 88 2024-05-08T05:50:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.138+0000 7f0e45985740 10 client.7941 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-08T05:50:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.138+0000 7f0e45985740 20 client.7941 populate_metadata read hostname 'smithi045' 2024-05-08T05:50:43.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.138+0000 7f0e45985740 1 --2- 172.21.15.45:0/1408004367 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x555570d258d0 0x555570d25cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.138+0000 7f0e45985740 4 client.7941 mds_command: new command op to 7806 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:50:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.138+0000 7f0e45985740 1 -- 172.21.15.45:0/1408004367 --> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555570c6cb40 con 0x555570d258d0 2024-05-08T05:50:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.138+0000 7f0e3bfff640 1 --2- 172.21.15.45:0/1408004367 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x555570d258d0 0x555570d25cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.138+0000 7f0e3bfff640 1 --2- 172.21.15.45:0/1408004367 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x555570d258d0 0x555570d25cd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c702f37e89777ab1 server_cookie=ae753fa3ed78118c in_seq=0 out_seq=0 2024-05-08T05:50:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.138+0000 7f0e1b7fe640 10 client.7941 ms_handle_connect on v2:172.21.15.45:6834/2729952281 2024-05-08T05:50:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.138+0000 7f0e1b7fe640 1 -- 172.21.15.45:0/1408004367 <== mds.0 v2:172.21.15.45:6834/2729952281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555570c6cb40 con 0x555570d258d0 2024-05-08T05:50:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.138+0000 7f0e1b7fe640 10 client.7941 handle_command_reply: tid=0 2024-05-08T05:50:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.139+0000 7f0e45985740 1 client.7941 shutdown 2024-05-08T05:50:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.139+0000 7f0e45985740 1 -- 172.21.15.45:0/1408004367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 msgr2=0x555570d3dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.139+0000 7f0e45985740 1 --2- 172.21.15.45:0/1408004367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570d3dd40 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0156f0 tx=0x7f0e2c0046c0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.139+0000 7f0e45985740 1 -- 172.21.15.45:0/1408004367 shutdown_connections 2024-05-08T05:50:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.139+0000 7f0e45985740 1 --2- 172.21.15.45:0/1408004367 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x555570d258d0 0x555570d25cd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.139+0000 7f0e45985740 1 --2- 172.21.15.45:0/1408004367 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 0x555570d37800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.139+0000 7f0e45985740 1 --2- 172.21.15.45:0/1408004367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d1f9e0 0x555570d40650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.139+0000 7f0e45985740 1 --2- 172.21.15.45:0/1408004367 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570d3dd40 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.139+0000 7f0e45985740 1 -- 172.21.15.45:0/1408004367 >> 172.21.15.45:0/1408004367 conn(0x555570d24b40 msgr2=0x555570cf4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.139+0000 7f0e1b7fe640 0 client.7941 ms_handle_reset on v2:172.21.15.45:6834/2729952281 2024-05-08T05:50:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.139+0000 7f0e45985740 1 -- 172.21.15.45:0/1408004367 shutdown_connections 2024-05-08T05:50:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.139+0000 7f0e45985740 1 -- 172.21.15.45:0/1408004367 wait complete. 2024-05-08T05:50:43.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.139+0000 7f0e45985740 20 client.7941 trim_cache size 0 max 16384 2024-05-08T05:50:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.157+0000 7f0e45985740 1 Processor -- start 2024-05-08T05:50:43.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.157+0000 7f0e45985740 1 -- start start 2024-05-08T05:50:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.158+0000 7f0e45985740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d1f9e0 0x555570d3d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.158+0000 7f0e45985740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570d25c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.158+0000 7f0e45985740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 0x555570d42a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.158+0000 7f0e45985740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c2021d0 con 0x555570d25280 2024-05-08T05:50:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.158+0000 7f0e45985740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c201230 con 0x555570d1f9e0 2024-05-08T05:50:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.158+0000 7f0e45985740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0e3c0c78f0 con 0x555570d42160 2024-05-08T05:50:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.158+0000 7f0e3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570d25c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.158+0000 7f0e3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570d25c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50406/0 (socket says 172.21.15.45:50406) 2024-05-08T05:50:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.158+0000 7f0e3bfff640 1 -- 172.21.15.45:0/3914853228 learned_addr learned my addr 172.21.15.45:0/3914853228 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.158+0000 7f0e3bfff640 1 -- 172.21.15.45:0/3914853228 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 msgr2=0x555570d42a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.158+0000 7f0e3bfff640 1 --2- 172.21.15.45:0/3914853228 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 0x555570d42a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.158+0000 7f0e3bfff640 1 -- 172.21.15.45:0/3914853228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d1f9e0 msgr2=0x555570d3d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:43.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.158+0000 7f0e40a09640 1 --2- 172.21.15.45:0/3914853228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d1f9e0 0x555570d3d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.158+0000 7f0e3bfff640 1 --2- 172.21.15.45:0/3914853228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d1f9e0 0x555570d3d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.158+0000 7f0e3bfff640 1 -- 172.21.15.45:0/3914853228 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555570d267d0 con 0x555570d25280 2024-05-08T05:50:43.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.158+0000 7f0e40a09640 1 --2- 172.21.15.45:0/3914853228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d1f9e0 0x555570d3d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:43.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.159+0000 7f0e3bfff640 1 --2- 172.21.15.45:0/3914853228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570d25c40 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f0e3005bd10 tx=0x7f0e3001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8054874859ab41fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:43.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.159+0000 7f0e21ffb640 1 -- 172.21.15.45:0/3914853228 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e300813c0 con 0x555570d25280 2024-05-08T05:50:43.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.159+0000 7f0e21ffb640 1 -- 172.21.15.45:0/3914853228 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e3005fc90 con 0x555570d25280 2024-05-08T05:50:43.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.159+0000 7f0e21ffb640 1 -- 172.21.15.45:0/3914853228 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e300687d0 con 0x555570d25280 2024-05-08T05:50:43.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.159+0000 7f0e45985740 1 -- 172.21.15.45:0/3914853228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 msgr2=0x555570d25c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.159+0000 7f0e45985740 1 --2- 172.21.15.45:0/3914853228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570d25c40 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f0e3005bd10 tx=0x7f0e3001f000 comp rx=0 tx=0).stop 2024-05-08T05:50:43.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.159+0000 7f0e45985740 1 -- 172.21.15.45:0/3914853228 shutdown_connections 2024-05-08T05:50:43.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.159+0000 7f0e45985740 1 --2- 172.21.15.45:0/3914853228 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 0x555570d42a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.159+0000 7f0e45985740 1 --2- 172.21.15.45:0/3914853228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570d25c40 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.159+0000 7f0e45985740 1 --2- 172.21.15.45:0/3914853228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d1f9e0 0x555570d3d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.159+0000 7f0e45985740 1 -- 172.21.15.45:0/3914853228 >> 172.21.15.45:0/3914853228 conn(0x555570d24b40 msgr2=0x555570d3e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:43.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.159+0000 7f0e45985740 1 -- 172.21.15.45:0/3914853228 shutdown_connections 2024-05-08T05:50:43.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.159+0000 7f0e45985740 1 -- 172.21.15.45:0/3914853228 wait complete. 2024-05-08T05:50:43.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.160+0000 7f0e45985740 1 -- start start 2024-05-08T05:50:43.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.160+0000 7f0e45985740 1 Processor -- start 2024-05-08T05:50:43.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.161+0000 7f0e45985740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d1f9e0 0x555570d35a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.161+0000 7f0e45985740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570dbf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.161+0000 7f0e45985740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 0x555570d26dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.161+0000 7f0e45985740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555570c1d9b0 con 0x555570d25280 2024-05-08T05:50:43.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.161+0000 7f0e45985740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c200be0 con 0x555570d1f9e0 2024-05-08T05:50:43.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.161+0000 7f0e45985740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x555570d1dac0 con 0x555570d42160 2024-05-08T05:50:43.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.161+0000 7f0e45985740 10 client.0 _ll_register_callbacks cb 0x7f0e2c0096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:50:43.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.161+0000 7f0e3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570dbf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.161+0000 7f0e3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570dbf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50408/0 (socket says 172.21.15.45:50408) 2024-05-08T05:50:43.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.161+0000 7f0e3bfff640 1 -- 172.21.15.45:0/762627375 learned_addr learned my addr 172.21.15.45:0/762627375 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:43.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.161+0000 7f0e4120a640 1 --2- 172.21.15.45:0/762627375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 0x555570d26dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.161+0000 7f0e40a09640 1 --2- 172.21.15.45:0/762627375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d1f9e0 0x555570d35a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.161+0000 7f0e3bfff640 1 -- 172.21.15.45:0/762627375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 msgr2=0x555570d26dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:43.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.161+0000 7f0e3bfff640 1 --2- 172.21.15.45:0/762627375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 0x555570d26dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.161+0000 7f0e3bfff640 1 -- 172.21.15.45:0/762627375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d1f9e0 msgr2=0x555570d35a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:43.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.161+0000 7f0e3bfff640 1 --2- 172.21.15.45:0/762627375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d1f9e0 0x555570d35a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.161+0000 7f0e3bfff640 1 -- 172.21.15.45:0/762627375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555709300f0 con 0x555570d25280 2024-05-08T05:50:43.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.161+0000 7f0e4120a640 1 --2- 172.21.15.45:0/762627375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 0x555570d26dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:43.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.161+0000 7f0e40a09640 1 --2- 172.21.15.45:0/762627375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d1f9e0 0x555570d35a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:43.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.162+0000 7f0e3bfff640 1 --2- 172.21.15.45:0/762627375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570dbf000 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f0e3005f990 tx=0x7f0e30000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471e5ae11748b796 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:43.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.162+0000 7f0e1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:50:43.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.162+0000 7f0e1b7fe640 1 -- 172.21.15.45:0/762627375 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e300687d0 con 0x555570d25280 2024-05-08T05:50:43.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.162+0000 7f0e1b7fe640 1 -- 172.21.15.45:0/762627375 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 6274+0+0 (secure 0 0 0) 0x7f0e30004b80 con 0x555570d25280 2024-05-08T05:50:43.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.162+0000 7f0e45985740 1 -- 172.21.15.45:0/762627375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e3005fc90 con 0x555570d25280 2024-05-08T05:50:43.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.162+0000 7f0e1b7fe640 1 -- 172.21.15.45:0/762627375 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f0e300813c0 con 0x555570d25280 2024-05-08T05:50:43.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.162+0000 7f0e45985740 10 client.7947 fetch_fsmap learned FSMap version 88 2024-05-08T05:50:43.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.162+0000 7f0e45985740 1 -- 172.21.15.45:0/762627375 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x5555709300f0 con 0x555570d25280 2024-05-08T05:50:43.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.162+0000 7f0e1b7fe640 1 -- 172.21.15.45:0/762627375 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e3005fc90 con 0x555570d25280 2024-05-08T05:50:43.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.162+0000 7f0e1b7fe640 1 -- 172.21.15.45:0/762627375 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f0e30096040 con 0x555570d25280 2024-05-08T05:50:43.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.162+0000 7f0e45985740 10 client.7947 fetch_fsmap finished waiting for FSMap version 88 2024-05-08T05:50:43.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.162+0000 7f0e45985740 10 client.7947 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-05-08T05:50:43.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.162+0000 7f0e45985740 20 client.7947 populate_metadata read hostname 'smithi045' 2024-05-08T05:50:43.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.162+0000 7f0e45985740 1 --2- 172.21.15.45:0/762627375 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x555570d1d2f0 0x555570d1e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.162+0000 7f0e45985740 4 client.7947 mds_command: new command op to 7806 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T05:50:43.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.162+0000 7f0e45985740 1 -- 172.21.15.45:0/762627375 --> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555570c669b0 con 0x555570d1d2f0 2024-05-08T05:50:43.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.162+0000 7f0e40a09640 1 --2- 172.21.15.45:0/762627375 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x555570d1d2f0 0x555570d1e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.163+0000 7f0e1b7fe640 10 client.7947 ms_handle_connect on v2:172.21.15.45:6834/2729952281 2024-05-08T05:50:43.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.163+0000 7f0e40a09640 1 --2- 172.21.15.45:0/762627375 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x555570d1d2f0 0x555570d1e4f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdff0d016f655d9b server_cookie=ad121c57cbbff828 in_seq=0 out_seq=0 2024-05-08T05:50:43.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.163+0000 7f0e1b7fe640 1 -- 172.21.15.45:0/762627375 <== mds.0 v2:172.21.15.45:6834/2729952281 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555570c669b0 con 0x555570d1d2f0 2024-05-08T05:50:43.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.163+0000 7f0e1b7fe640 10 client.7947 handle_command_reply: tid=0 2024-05-08T05:50:43.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.163+0000 7f0e45985740 1 client.7947 shutdown 2024-05-08T05:50:43.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.163+0000 7f0e45985740 1 -- 172.21.15.45:0/762627375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 msgr2=0x555570dbf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.163+0000 7f0e45985740 1 --2- 172.21.15.45:0/762627375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570dbf000 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f0e3005f990 tx=0x7f0e30000bc0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.163+0000 7f0e45985740 1 -- 172.21.15.45:0/762627375 shutdown_connections 2024-05-08T05:50:43.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.163+0000 7f0e45985740 1 --2- 172.21.15.45:0/762627375 >> [v2:172.21.15.45:6834/2729952281,v1:172.21.15.45:6835/2729952281] conn(0x555570d1d2f0 0x555570d1e4f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.163+0000 7f0e45985740 1 --2- 172.21.15.45:0/762627375 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x555570d42160 0x555570d26dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.163+0000 7f0e45985740 1 --2- 172.21.15.45:0/762627375 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555570d25280 0x555570dbf000 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.163+0000 7f0e1b7fe640 0 client.7947 ms_handle_reset on v2:172.21.15.45:6834/2729952281 2024-05-08T05:50:43.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.163+0000 7f0e45985740 1 --2- 172.21.15.45:0/762627375 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x555570d1f9e0 0x555570d35a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.163+0000 7f0e45985740 1 -- 172.21.15.45:0/762627375 >> 172.21.15.45:0/762627375 conn(0x555570d24b40 msgr2=0x555570cf39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:43.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.163+0000 7f0e45985740 1 -- 172.21.15.45:0/762627375 shutdown_connections 2024-05-08T05:50:43.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.163+0000 7f0e45985740 1 -- 172.21.15.45:0/762627375 wait complete. 2024-05-08T05:50:43.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.163+0000 7f0e45985740 20 client.7947 trim_cache size 0 max 16384 2024-05-08T05:50:43.169 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:50:43.170 INFO:teuthology.orchestra.run.smithi045.stdout: "status": "no active scrubs running", 2024-05-08T05:50:43.170 INFO:teuthology.orchestra.run.smithi045.stdout: "scrubs": {} 2024-05-08T05:50:43.170 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:50:43.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.164+0000 7f0e42c6d640 1 -- 172.21.15.45:0/2312594025 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0e280762d0 msgr2=0x7f0e28078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.164+0000 7f0e42c6d640 1 --2- 172.21.15.45:0/2312594025 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0e280762d0 0x7f0e28078790 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0d0510 tx=0x7f0e2c008040 comp rx=0 tx=0).stop 2024-05-08T05:50:43.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.164+0000 7f0e42c6d640 1 -- 172.21.15.45:0/2312594025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e3c0b2490 msgr2=0x7f0e3c0b76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.164+0000 7f0e42c6d640 1 --2- 172.21.15.45:0/2312594025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e3c0b2490 0x7f0e3c0b76d0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000c500 tx=0x7f0e3000c9c0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.164+0000 7f0e42c6d640 1 -- 172.21.15.45:0/2312594025 shutdown_connections 2024-05-08T05:50:43.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.164+0000 7f0e42c6d640 1 --2- 172.21.15.45:0/2312594025 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0e280762d0 0x7f0e28078790 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.164+0000 7f0e42c6d640 1 --2- 172.21.15.45:0/2312594025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e3c1629b0 0x7f0e3c160f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.164+0000 7f0e42c6d640 1 --2- 172.21.15.45:0/2312594025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e3c0b2490 0x7f0e3c0b76d0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.164+0000 7f0e42c6d640 1 --2- 172.21.15.45:0/2312594025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e3c070d20 0x7f0e3c15bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.164+0000 7f0e42c6d640 1 -- 172.21.15.45:0/2312594025 >> 172.21.15.45:0/2312594025 conn(0x7f0e3c0b49d0 msgr2=0x7f0e3c0b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:43.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.164+0000 7f0e42c6d640 1 -- 172.21.15.45:0/2312594025 shutdown_connections 2024-05-08T05:50:43.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.164+0000 7f0e42c6d640 1 -- 172.21.15.45:0/2312594025 wait complete. 2024-05-08T05:50:43.175 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T05:50:43.276 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T05:50:43.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.382+0000 7f90517e4640 1 Processor -- start 2024-05-08T05:50:43.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.382+0000 7f90517e4640 1 -- start start 2024-05-08T05:50:43.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.383+0000 7f90517e4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f904c06c3a0 0x7f904c0b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.383+0000 7f90517e4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f904c0aedf0 0x7f904c0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.383+0000 7f90517e4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f904c09ff30 con 0x7f904c15d180 2024-05-08T05:50:43.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.383+0000 7f90517e4640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f904c15e360 con 0x7f904c0aedf0 2024-05-08T05:50:43.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.383+0000 7f90517e4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f904c15d750 con 0x7f904c06c3a0 2024-05-08T05:50:43.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.383+0000 7f904a7fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f904c0aedf0 0x7f904c0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.383+0000 7f904b7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f904c15d180 0x7f904c06c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60434/0 (socket says 172.21.15.45:60434) 2024-05-08T05:50:43.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.383+0000 7f904a7fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f904c0aedf0 0x7f904c0b6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50416/0 (socket says 172.21.15.45:50416) 2024-05-08T05:50:43.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.383+0000 7f904b7fe640 1 -- 172.21.15.45:0/3123795201 learned_addr learned my addr 172.21.15.45:0/3123795201 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:50:43.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.383+0000 7f904affd640 1 --2- 172.21.15.45:0/3123795201 >> v2:172.21.15.63:3300/0 conn(0x7f904c06c3a0 0x7f904c0b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.383+0000 7f904a7fc640 1 -- 172.21.15.45:0/3123795201 >> v2:172.21.15.63:3300/0 conn(0x7f904c06c3a0 msgr2=0x7f904c0b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:43.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.383+0000 7f904a7fc640 1 --2- 172.21.15.45:0/3123795201 >> v2:172.21.15.63:3300/0 conn(0x7f904c06c3a0 0x7f904c0b13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.383+0000 7f904a7fc640 1 -- 172.21.15.45:0/3123795201 >> v1:172.21.15.45:6789/0 conn(0x7f904c15d180 legacy=0x7f904c06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.383+0000 7f904a7fc640 1 -- 172.21.15.45:0/3123795201 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c15da30 con 0x7f904c0aedf0 2024-05-08T05:50:43.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.383+0000 7f9049ffb640 1 -- 172.21.15.45:0/3123795201 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 271841099 0 0) 0x7f904c09ff30 con 0x7f904c15d180 2024-05-08T05:50:43.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.383+0000 7f904affd640 1 --2- 172.21.15.45:0/3123795201 >> v2:172.21.15.63:3300/0 conn(0x7f904c06c3a0 0x7f904c0b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:43.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.384+0000 7f904a7fc640 1 --2- 172.21.15.45:0/3123795201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f904c0aedf0 0x7f904c0b6130 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f904c0b1950 tx=0x7f904002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5a7b045f1c31b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:43.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.384+0000 7f9049ffb640 1 -- 172.21.15.45:0/3123795201 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9040037070 con 0x7f904c0aedf0 2024-05-08T05:50:43.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.384+0000 7f9049ffb640 1 -- 172.21.15.45:0/3123795201 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9040032440 con 0x7f904c0aedf0 2024-05-08T05:50:43.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.384+0000 7f9049ffb640 1 -- 172.21.15.45:0/3123795201 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9040032810 con 0x7f904c0aedf0 2024-05-08T05:50:43.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.385+0000 7f90517e4640 1 -- 172.21.15.45:0/3123795201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f904c0aedf0 msgr2=0x7f904c0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.385+0000 7f90517e4640 1 --2- 172.21.15.45:0/3123795201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f904c0aedf0 0x7f904c0b6130 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f904c0b1950 tx=0x7f904002a1a0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.385+0000 7f90517e4640 1 -- 172.21.15.45:0/3123795201 shutdown_connections 2024-05-08T05:50:43.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.385+0000 7f90517e4640 1 --2- 172.21.15.45:0/3123795201 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f904c0aedf0 0x7f904c0b6130 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.385+0000 7f90517e4640 1 --2- 172.21.15.45:0/3123795201 >> v2:172.21.15.63:3300/0 conn(0x7f904c06c3a0 0x7f904c0b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.385+0000 7f90517e4640 1 -- 172.21.15.45:0/3123795201 >> 172.21.15.45:0/3123795201 conn(0x7f904c0b3b10 msgr2=0x7f904c0b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:43.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.385+0000 7f90517e4640 1 -- 172.21.15.45:0/3123795201 shutdown_connections 2024-05-08T05:50:43.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.385+0000 7f90517e4640 1 -- 172.21.15.45:0/3123795201 wait complete. 2024-05-08T05:50:43.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.386+0000 7f90517e4640 1 Processor -- start 2024-05-08T05:50:43.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.386+0000 7f90517e4640 1 -- start start 2024-05-08T05:50:43.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.387+0000 7f90517e4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f904c06c3a0 0x7f904c153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.387+0000 7f90517e4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f904c0aedf0 0x7f904c151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.387+0000 7f90517e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f904c15d180 0x7f904c150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.387+0000 7f90517e4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f904c0b4fc0 con 0x7f904c15d180 2024-05-08T05:50:43.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.387+0000 7f90517e4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f904c15dff0 con 0x7f904c0aedf0 2024-05-08T05:50:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.387+0000 7f90517e4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f904c0b5140 con 0x7f904c06c3a0 2024-05-08T05:50:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.387+0000 7f904b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f904c15d180 0x7f904c150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.387+0000 7f904b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f904c15d180 0x7f904c150c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50428/0 (socket says 172.21.15.45:50428) 2024-05-08T05:50:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.387+0000 7f904b7fe640 1 -- 172.21.15.45:0/597131468 learned_addr learned my addr 172.21.15.45:0/597131468 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.387+0000 7f904affd640 1 --2- 172.21.15.45:0/597131468 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f904c06c3a0 0x7f904c153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.387+0000 7f904a7fc640 1 --2- 172.21.15.45:0/597131468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f904c0aedf0 0x7f904c151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.387+0000 7f904b7fe640 1 -- 172.21.15.45:0/597131468 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f904c06c3a0 msgr2=0x7f904c153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.387+0000 7f904b7fe640 1 --2- 172.21.15.45:0/597131468 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f904c06c3a0 0x7f904c153f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.387+0000 7f904b7fe640 1 -- 172.21.15.45:0/597131468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f904c0aedf0 msgr2=0x7f904c151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.387+0000 7f904b7fe640 1 --2- 172.21.15.45:0/597131468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f904c0aedf0 0x7f904c151370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.387+0000 7f904b7fe640 1 -- 172.21.15.45:0/597131468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c151a70 con 0x7f904c15d180 2024-05-08T05:50:43.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.388+0000 7f904b7fe640 1 --2- 172.21.15.45:0/597131468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f904c15d180 0x7f904c150c80 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f904400c3c0 tx=0x7f904400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:43.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.388+0000 7f90337fe640 1 -- 172.21.15.45:0/597131468 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90440074e0 con 0x7f904c15d180 2024-05-08T05:50:43.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.388+0000 7f90337fe640 1 -- 172.21.15.45:0/597131468 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90440047e0 con 0x7f904c15d180 2024-05-08T05:50:43.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.388+0000 7f90517e4640 1 -- 172.21.15.45:0/597131468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f904c151d50 con 0x7f904c15d180 2024-05-08T05:50:43.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.388+0000 7f90337fe640 1 -- 172.21.15.45:0/597131468 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044004c50 con 0x7f904c15d180 2024-05-08T05:50:43.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.388+0000 7f90517e4640 1 -- 172.21.15.45:0/597131468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f904c06fb50 con 0x7f904c15d180 2024-05-08T05:50:43.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.389+0000 7f90517e4640 1 -- 172.21.15.45:0/597131468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f904c06a8f0 con 0x7f904c15d180 2024-05-08T05:50:43.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.389+0000 7f90337fe640 1 -- 172.21.15.45:0/597131468 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9044011070 con 0x7f904c15d180 2024-05-08T05:50:43.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.390+0000 7f90337fe640 1 --2- 172.21.15.45:0/597131468 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f90380762d0 0x7f9038078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.390+0000 7f90337fe640 1 -- 172.21.15.45:0/597131468 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6274+0+0 (secure 0 0 0) 0x7f9044097330 con 0x7f904c15d180 2024-05-08T05:50:43.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.390+0000 7f904affd640 1 --2- 172.21.15.45:0/597131468 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f90380762d0 0x7f9038078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.391+0000 7f904affd640 1 --2- 172.21.15.45:0/597131468 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f90380762d0 0x7f9038078790 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f903c009f70 tx=0x7f903c00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:43.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.394+0000 7f90337fe640 1 -- 172.21.15.45:0/597131468 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9044060ae0 con 0x7f904c15d180 2024-05-08T05:50:43.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.626+0000 7f90517e4640 1 -- 172.21.15.45:0/597131468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f904c15f3e0 con 0x7f904c15d180 2024-05-08T05:50:43.626 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-05-08T05:50:43.626+0000 7f124de12640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T05:50:43.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.628+0000 7f90337fe640 1 -- 172.21.15.45:0/597131468 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v89) ==== 161+0+0 (secure 0 0 0) 0x7f90440603e0 con 0x7f904c15d180 2024-05-08T05:50:43.628 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T05:50:43.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.632+0000 7f90517e4640 1 -- 172.21.15.45:0/597131468 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f90380762d0 msgr2=0x7f9038078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.632+0000 7f90517e4640 1 --2- 172.21.15.45:0/597131468 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f90380762d0 0x7f9038078790 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f903c009f70 tx=0x7f903c00b040 comp rx=0 tx=0).stop 2024-05-08T05:50:43.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.632+0000 7f90517e4640 1 -- 172.21.15.45:0/597131468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f904c15d180 msgr2=0x7f904c150c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.632+0000 7f90517e4640 1 --2- 172.21.15.45:0/597131468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f904c15d180 0x7f904c150c80 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f904400c3c0 tx=0x7f904400c880 comp rx=0 tx=0).stop 2024-05-08T05:50:43.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.632+0000 7f90517e4640 1 -- 172.21.15.45:0/597131468 shutdown_connections 2024-05-08T05:50:43.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.632+0000 7f90517e4640 1 --2- 172.21.15.45:0/597131468 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f90380762d0 0x7f9038078790 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.632+0000 7f90517e4640 1 --2- 172.21.15.45:0/597131468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f904c15d180 0x7f904c150c80 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.632+0000 7f90517e4640 1 --2- 172.21.15.45:0/597131468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f904c0aedf0 0x7f904c151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.632+0000 7f90517e4640 1 --2- 172.21.15.45:0/597131468 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f904c06c3a0 0x7f904c153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.632+0000 7f90517e4640 1 -- 172.21.15.45:0/597131468 >> 172.21.15.45:0/597131468 conn(0x7f904c0b3b10 msgr2=0x7f904c0b6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:43.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.632+0000 7f90517e4640 1 -- 172.21.15.45:0/597131468 shutdown_connections 2024-05-08T05:50:43.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.632+0000 7f90517e4640 1 -- 172.21.15.45:0/597131468 wait complete. 2024-05-08T05:50:43.633 INFO:tasks.ceph.mds.a.smithi045.stderr: -1170> 2024-05-08T05:50:12.829+0000 7fbecd4e1640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-05-08T05:50:43.634 INFO:tasks.ceph.mds.a.smithi045.stderr: -1167> 2024-05-08T05:50:12.829+0000 7fbecd4e1640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-05-08T05:50:43.634 INFO:tasks.ceph.mds.a.smithi045.stderr: -1127> 2024-05-08T05:50:12.829+0000 7fbecd4e1640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-05-08T05:50:43.634 INFO:tasks.ceph.mds.a.smithi045.stderr: -1124> 2024-05-08T05:50:12.829+0000 7fbecd4e1640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-05-08T05:50:43.634 INFO:tasks.ceph.mds.a.smithi045.stderr: -995> 2024-05-08T05:50:13.183+0000 7fbed14e9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:50:43.644 DEBUG:teuthology.orchestra.run.smithi045:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-05-08T05:50:43.659 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-05-08T05:50:43.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.686+0000 7fc8b9876280 1 -- start start 2024-05-08T05:50:43.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.686+0000 7fc8b9876280 1 Processor -- start 2024-05-08T05:50:43.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.687+0000 7fc8b9876280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d249ec800 0x564d249e5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.687+0000 7fc8b9876280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d23c09c00 0x564d249e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.687+0000 7fc8b9876280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d249edc00 0x564d249e4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.687+0000 7fc8b9876280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564d249f4000 con 0x564d23c09c00 2024-05-08T05:50:43.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.687+0000 7fc8b9876280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564d249f4180 con 0x564d249edc00 2024-05-08T05:50:43.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.687+0000 7fc8b9876280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564d249f4300 con 0x564d249ec800 2024-05-08T05:50:43.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.687+0000 7fc8b7821640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d23c09c00 0x564d249e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.687+0000 7fc8b7821640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d23c09c00 0x564d249e4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50474/0 (socket says 172.21.15.45:50474) 2024-05-08T05:50:43.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.687+0000 7fc8b7821640 1 -- 172.21.15.45:0/2382849568 learned_addr learned my addr 172.21.15.45:0/2382849568 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:43.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.687+0000 7fc8b8823640 1 --2- 172.21.15.45:0/2382849568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d249edc00 0x564d249e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.687+0000 7fc8b8022640 1 --2- 172.21.15.45:0/2382849568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d249ec800 0x564d249e5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.687+0000 7fc8b7821640 1 -- 172.21.15.45:0/2382849568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d249ec800 msgr2=0x564d249e5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:43.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.687+0000 7fc8b7821640 1 --2- 172.21.15.45:0/2382849568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d249ec800 0x564d249e5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.687+0000 7fc8b7821640 1 -- 172.21.15.45:0/2382849568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d249edc00 msgr2=0x564d249e4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:43.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.687+0000 7fc8b7821640 1 --2- 172.21.15.45:0/2382849568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d249edc00 0x564d249e4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.687+0000 7fc8b7821640 1 -- 172.21.15.45:0/2382849568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d23c3e780 con 0x564d23c09c00 2024-05-08T05:50:43.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.687+0000 7fc8b8022640 1 --2- 172.21.15.45:0/2382849568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d249ec800 0x564d249e5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:43.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.687+0000 7fc8b8823640 1 --2- 172.21.15.45:0/2382849568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d249edc00 0x564d249e4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:43.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.687+0000 7fc8b7821640 1 --2- 172.21.15.45:0/2382849568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d23c09c00 0x564d249e4b00 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x564d24a4b350 tx=0x564d249aa180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa9d6c9f003d34c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:43.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.687+0000 7fc8b581d640 1 -- 172.21.15.45:0/2382849568 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564d23c2ed00 con 0x564d23c09c00 2024-05-08T05:50:43.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.688+0000 7fc8b581d640 1 -- 172.21.15.45:0/2382849568 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564d23c34a80 con 0x564d23c09c00 2024-05-08T05:50:43.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.688+0000 7fc8b581d640 1 -- 172.21.15.45:0/2382849568 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564d23c2e680 con 0x564d23c09c00 2024-05-08T05:50:43.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.688+0000 7fc8b9876280 1 -- 172.21.15.45:0/2382849568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d23c3e000 con 0x564d23c09c00 2024-05-08T05:50:43.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.688+0000 7fc8b9876280 1 -- 172.21.15.45:0/2382849568 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=0}) -- 0x564d23c3fa40 con 0x564d23c09c00 2024-05-08T05:50:43.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.688+0000 7fc8b9876280 4 waiting for MDS map... 2024-05-08T05:50:43.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.688+0000 7fc8b581d640 1 -- 172.21.15.45:0/2382849568 <== mon.0 v2:172.21.15.45:3300/0 4 ==== osd_map(108..108 src has 1..108) ==== 6360+0+0 (secure 0 0 0) 0x564d24aec600 con 0x564d23c09c00 2024-05-08T05:50:43.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.688+0000 7fc8b581d640 1 -- 172.21.15.45:0/2382849568 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 89) ==== 2885+0+0 (secure 0 0 0) 0x564d23c63180 con 0x564d23c09c00 2024-05-08T05:50:43.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.688+0000 7fc8b9876280 4 Got MDS map 89 2024-05-08T05:50:43.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.688+0000 7fc8b9876280 10 main: main 2024-05-08T05:50:43.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.688+0000 7fc8b9876280 4 main: connecting to RADOS... 2024-05-08T05:50:43.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.688+0000 7fc8b9876280 1 Processor -- start 2024-05-08T05:50:43.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.688+0000 7fc8b9876280 1 -- start start 2024-05-08T05:50:43.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.689+0000 7fc8b9876280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d249ec000 0x564d249e5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.689+0000 7fc8b9876280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d24aef800 0x564d249e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.689+0000 7fc8b9876280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d24aefc00 0x564d249e6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.689+0000 7fc8b9876280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564d249f4480 con 0x564d249ec000 2024-05-08T05:50:43.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.689+0000 7fc8b9876280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564d249f4600 con 0x564d24aefc00 2024-05-08T05:50:43.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.689+0000 7fc8b9876280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564d249f4780 con 0x564d24aef800 2024-05-08T05:50:43.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.689+0000 7fc8b8823640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d249ec000 0x564d249e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.689+0000 7fc8b8823640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d249ec000 0x564d249e5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50490/0 (socket says 172.21.15.45:50490) 2024-05-08T05:50:43.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.689+0000 7fc8b8823640 1 -- 172.21.15.45:0/2656127839 learned_addr learned my addr 172.21.15.45:0/2656127839 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:43.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.689+0000 7fc8b7821640 1 --2- 172.21.15.45:0/2656127839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d24aefc00 0x564d249e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.689+0000 7fc8b8022640 1 --2- 172.21.15.45:0/2656127839 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d24aef800 0x564d249e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.689+0000 7fc8b8823640 1 -- 172.21.15.45:0/2656127839 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d24aef800 msgr2=0x564d249e5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:43.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.689+0000 7fc8b8823640 1 --2- 172.21.15.45:0/2656127839 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d24aef800 0x564d249e5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.689+0000 7fc8b8823640 1 -- 172.21.15.45:0/2656127839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d24aefc00 msgr2=0x564d249e6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:43.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.689+0000 7fc8b8823640 1 --2- 172.21.15.45:0/2656127839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d24aefc00 0x564d249e6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.689+0000 7fc8b8823640 1 -- 172.21.15.45:0/2656127839 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d23c3f860 con 0x564d249ec000 2024-05-08T05:50:43.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.689+0000 7fc8b8022640 1 --2- 172.21.15.45:0/2656127839 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d24aef800 0x564d249e5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:43.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.690+0000 7fc8b8823640 1 --2- 172.21.15.45:0/2656127839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d249ec000 0x564d249e5600 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x564d23c40ed0 tx=0x564d249a8ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c27beae5a42b99 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:43.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.690+0000 7fc8b3819640 1 -- 172.21.15.45:0/2656127839 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564d23c2eb60 con 0x564d249ec000 2024-05-08T05:50:43.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.690+0000 7fc8b3819640 1 -- 172.21.15.45:0/2656127839 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564d23c34e00 con 0x564d249ec000 2024-05-08T05:50:43.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.690+0000 7fc8b3819640 1 -- 172.21.15.45:0/2656127839 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564d23c2ed00 con 0x564d249ec000 2024-05-08T05:50:43.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.690+0000 7fc8b9876280 1 -- 172.21.15.45:0/2656127839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d249ec000 msgr2=0x564d249e5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.690+0000 7fc8b9876280 1 --2- 172.21.15.45:0/2656127839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d249ec000 0x564d249e5600 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x564d23c40ed0 tx=0x564d249a8ae0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.690+0000 7fc8b9876280 1 -- 172.21.15.45:0/2656127839 shutdown_connections 2024-05-08T05:50:43.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.690+0000 7fc8b9876280 1 --2- 172.21.15.45:0/2656127839 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d24aef800 0x564d249e5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.690+0000 7fc8b9876280 1 --2- 172.21.15.45:0/2656127839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d24aefc00 0x564d249e6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.690+0000 7fc8b9876280 1 --2- 172.21.15.45:0/2656127839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d249ec000 0x564d249e5600 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.690+0000 7fc8b9876280 1 -- 172.21.15.45:0/2656127839 >> 172.21.15.45:0/2656127839 conn(0x564d249ec400 msgr2=0x564d23d01800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:43.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.690+0000 7fc8b9876280 1 -- 172.21.15.45:0/2656127839 shutdown_connections 2024-05-08T05:50:43.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.690+0000 7fc8b9876280 1 -- 172.21.15.45:0/2656127839 wait complete. 2024-05-08T05:50:43.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.691+0000 7fc8b9876280 1 Processor -- start 2024-05-08T05:50:43.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.691+0000 7fc8b9876280 1 -- start start 2024-05-08T05:50:43.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.691+0000 7fc8b9876280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d249ec000 0x564d249e5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.691+0000 7fc8b9876280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d24aef800 0x564d249e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.691+0000 7fc8b9876280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d24aefc00 0x564d249e6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.691+0000 7fc8b9876280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564d249f5c80 con 0x564d24aefc00 2024-05-08T05:50:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.691+0000 7fc8b9876280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564d249f5b00 con 0x564d249ec000 2024-05-08T05:50:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.691+0000 7fc8b9876280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564d249f5980 con 0x564d24aef800 2024-05-08T05:50:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.691+0000 7fc8b7821640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d24aefc00 0x564d249e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.692+0000 7fc8b7821640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d24aefc00 0x564d249e6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50506/0 (socket says 172.21.15.45:50506) 2024-05-08T05:50:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.692+0000 7fc8b7821640 1 -- 172.21.15.45:0/2721149116 learned_addr learned my addr 172.21.15.45:0/2721149116 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.692+0000 7fc8b8823640 1 --2- 172.21.15.45:0/2721149116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d249ec000 0x564d249e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.692+0000 7fc8b8022640 1 --2- 172.21.15.45:0/2721149116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d24aef800 0x564d249e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.692+0000 7fc8b7821640 1 -- 172.21.15.45:0/2721149116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d24aef800 msgr2=0x564d249e5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.692+0000 7fc8b7821640 1 --2- 172.21.15.45:0/2721149116 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d24aef800 0x564d249e5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.692+0000 7fc8b7821640 1 -- 172.21.15.45:0/2721149116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d249ec000 msgr2=0x564d249e5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.692+0000 7fc8b7821640 1 --2- 172.21.15.45:0/2721149116 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d249ec000 0x564d249e5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.692+0000 7fc8b7821640 1 -- 172.21.15.45:0/2721149116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d23c3e780 con 0x564d24aefc00 2024-05-08T05:50:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.692+0000 7fc8b7821640 1 --2- 172.21.15.45:0/2721149116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d24aefc00 0x564d249e6100 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x564d24b03aa0 tx=0x564d249a89c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:43.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.692+0000 7fc8b2016640 1 -- 172.21.15.45:0/2721149116 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564d23c2e820 con 0x564d24aefc00 2024-05-08T05:50:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.692+0000 7fc8b2016640 1 -- 172.21.15.45:0/2721149116 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564d23c341c0 con 0x564d24aefc00 2024-05-08T05:50:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.692+0000 7fc8b9876280 1 -- 172.21.15.45:0/2721149116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564d23c3e000 con 0x564d24aefc00 2024-05-08T05:50:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.692+0000 7fc8b2016640 1 -- 172.21.15.45:0/2721149116 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564d23c2eb60 con 0x564d24aefc00 2024-05-08T05:50:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.692+0000 7fc8b9876280 1 -- 172.21.15.45:0/2721149116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d23c3f680 con 0x564d24aefc00 2024-05-08T05:50:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.692+0000 7fc8b9876280 4 main: resolving pool 15 2024-05-08T05:50:43.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.693+0000 7fc8b2016640 1 -- 172.21.15.45:0/2721149116 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x564d24b56c00 con 0x564d24aefc00 2024-05-08T05:50:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.694+0000 7fc8b2016640 1 --2- 172.21.15.45:0/2721149116 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x564d24b61000 0x564d249e6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.694+0000 7fc8b2016640 1 -- 172.21.15.45:0/2721149116 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6360+0+0 (secure 0 0 0) 0x564d24b54a00 con 0x564d24aefc00 2024-05-08T05:50:43.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.694+0000 7fc8b9876280 4 main: creating IoCtx.. 2024-05-08T05:50:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.694+0000 7fc8b9876280 1 --2- 172.21.15.45:0/2721149116 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x564d24b61400 0x564d249e6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.694+0000 7fc8b9876280 1 -- 172.21.15.45:0/2721149116 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) -- 0x564d24b61800 con 0x564d24b61400 2024-05-08T05:50:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.694+0000 7fc8b8823640 1 --2- 172.21.15.45:0/2721149116 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x564d24b61000 0x564d249e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.694+0000 7fc8b8022640 1 --2- 172.21.15.45:0/2721149116 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x564d24b61400 0x564d249e6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.694+0000 7fc8b8022640 1 --2- 172.21.15.45:0/2721149116 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x564d24b61400 0x564d249e6c00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.695+0000 7fc8b8823640 1 --2- 172.21.15.45:0/2721149116 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x564d24b61000 0x564d249e6680 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x564d23c40ed0 tx=0x564d249a8a20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:43.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.697+0000 7fc8b8022640 1 -- 172.21.15.45:0/2721149116 <== osd.1 v2:172.21.15.45:6824/1337179184 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x564d24996000 con 0x564d24b61400 2024-05-08T05:50:43.702 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:50:43.702 INFO:teuthology.orchestra.run.smithi045.stdout: "0": { 2024-05-08T05:50:43.703 INFO:teuthology.orchestra.run.smithi045.stdout: "data": { 2024-05-08T05:50:43.703 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 5, 2024-05-08T05:50:43.703 INFO:teuthology.orchestra.run.smithi045.stdout: "inotable": { 2024-05-08T05:50:43.703 INFO:teuthology.orchestra.run.smithi045.stdout: "projected_free": [ 2024-05-08T05:50:43.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:50:43.703 INFO:teuthology.orchestra.run.smithi045.stdout: "start": 1099511627779, 2024-05-08T05:50:43.703 INFO:teuthology.orchestra.run.smithi045.stdout: "len": 1099511627773 2024-05-08T05:50:43.703 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:50:43.703 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-05-08T05:50:43.703 INFO:teuthology.orchestra.run.smithi045.stdout: "free": [ 2024-05-08T05:50:43.703 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:50:43.703 INFO:teuthology.orchestra.run.smithi045.stdout: "start": 1099511627779, 2024-05-08T05:50:43.703 INFO:teuthology.orchestra.run.smithi045.stdout: "len": 1099511627773 2024-05-08T05:50:43.703 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:50:43.704 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-05-08T05:50:43.704 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:50:43.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:50:43.704 INFO:teuthology.orchestra.run.smithi045.stdout: "result": 0 2024-05-08T05:50:43.704 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:50:43.704 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:50:43.704 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:50:43.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.697+0000 7fc8b9876280 1 -- 172.21.15.45:0/2721149116 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x564d24b61400 msgr2=0x564d249e6c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.697+0000 7fc8b9876280 1 --2- 172.21.15.45:0/2721149116 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x564d24b61400 0x564d249e6c00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.697+0000 7fc8b9876280 1 -- 172.21.15.45:0/2721149116 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x564d24b61000 msgr2=0x564d249e6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.697+0000 7fc8b9876280 1 --2- 172.21.15.45:0/2721149116 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x564d24b61000 0x564d249e6680 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x564d23c40ed0 tx=0x564d249a8a20 comp rx=0 tx=0).stop 2024-05-08T05:50:43.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.697+0000 7fc8b9876280 1 -- 172.21.15.45:0/2721149116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d24aefc00 msgr2=0x564d249e6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.697+0000 7fc8b9876280 1 --2- 172.21.15.45:0/2721149116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d24aefc00 0x564d249e6100 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x564d24b03aa0 tx=0x564d249a89c0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.697+0000 7fc8b8022640 1 -- 172.21.15.45:0/2721149116 reap_dead start 2024-05-08T05:50:43.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.697+0000 7fc8b9876280 1 -- 172.21.15.45:0/2721149116 shutdown_connections 2024-05-08T05:50:43.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.697+0000 7fc8b9876280 1 -- 172.21.15.45:0/2721149116 >> 172.21.15.45:0/2721149116 conn(0x564d249ec400 msgr2=0x564d23d01800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:43.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.697+0000 7fc8b9876280 1 -- 172.21.15.45:0/2721149116 shutdown_connections 2024-05-08T05:50:43.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.697+0000 7fc8b9876280 1 -- 172.21.15.45:0/2721149116 wait complete. 2024-05-08T05:50:43.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.697+0000 7fc8b9876280 1 -- 172.21.15.45:0/2382849568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d23c09c00 msgr2=0x564d249e4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:43.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.697+0000 7fc8b9876280 1 --2- 172.21.15.45:0/2382849568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d23c09c00 0x564d249e4b00 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x564d24a4b350 tx=0x564d249aa180 comp rx=0 tx=0).stop 2024-05-08T05:50:43.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.697+0000 7fc8b9876280 1 -- 172.21.15.45:0/2382849568 shutdown_connections 2024-05-08T05:50:43.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.697+0000 7fc8b9876280 1 --2- 172.21.15.45:0/2382849568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564d249edc00 0x564d249e4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.697+0000 7fc8b9876280 1 --2- 172.21.15.45:0/2382849568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564d23c09c00 0x564d249e4b00 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.697+0000 7fc8b9876280 1 --2- 172.21.15.45:0/2382849568 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564d249ec800 0x564d249e5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:43.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.697+0000 7fc8b9876280 1 -- 172.21.15.45:0/2382849568 >> 172.21.15.45:0/2382849568 conn(0x564d23c09000 msgr2=0x564d23d00000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:43.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.698+0000 7fc8b9876280 1 -- 172.21.15.45:0/2382849568 shutdown_connections 2024-05-08T05:50:43.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:43.698+0000 7fc8b9876280 1 -- 172.21.15.45:0/2382849568 wait complete. 2024-05-08T05:50:43.707 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.063729, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-05-08T05:50:43.708 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T05:50:43.708 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:50:43.796 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:50:43.841 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T05:50:43.841 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:50:43.841 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:50:43.841 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:50:43.858 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:50:43.858 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:50:43.858 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:50:43.858 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:50:43.878 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:50:43.878 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:50:43.878 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:50:43.878 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:50:43.914 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:50:43.914 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:50:43.914 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:50:43.914 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:50:43.936 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:50:43.936 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:50:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.057+0000 7f2ae9371640 1 Processor -- start 2024-05-08T05:50:44.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.058+0000 7f2ae9371640 1 -- start start 2024-05-08T05:50:44.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.058+0000 7f2ae9371640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae4073800 0x7f2ae41637b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:44.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.058+0000 7f2ae9371640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ae40b65f0 con 0x7f2ae406da00 2024-05-08T05:50:44.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.058+0000 7f2ae9371640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ae40b5120 con 0x7f2ae406d620 2024-05-08T05:50:44.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.058+0000 7f2ae9371640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae40afaa0 con 0x7f2ae4073800 2024-05-08T05:50:44.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae27fc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2ae406da00 0x7f2ae40b5ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60444/0 (socket says 172.21.15.45:60444) 2024-05-08T05:50:44.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae27fc640 1 -- 172.21.15.45:0/336441730 learned_addr learned my addr 172.21.15.45:0/336441730 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:50:44.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae1ffb640 1 -- 172.21.15.45:0/336441730 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2224156513 0 0) 0x7f2ae40b65f0 con 0x7f2ae406da00 2024-05-08T05:50:44.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae1ffb640 1 -- 172.21.15.45:0/336441730 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ad0003520 con 0x7f2ae406da00 2024-05-08T05:50:44.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae37fe640 1 --2- 172.21.15.45:0/336441730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae4073800 0x7f2ae41637b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:44.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae1ffb640 1 -- 172.21.15.45:0/336441730 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4153820548 0 0) 0x7f2ad0003520 con 0x7f2ae406da00 2024-05-08T05:50:44.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae1ffb640 1 -- 172.21.15.45:0/336441730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae4073800 msgr2=0x7f2ae41637b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:44.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae1ffb640 1 --2- 172.21.15.45:0/336441730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae4073800 0x7f2ae41637b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae1ffb640 1 -- 172.21.15.45:0/336441730 >> v1:172.21.15.63:6789/0 conn(0x7f2ae406d620 legacy=0x7f2ae40af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:44.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae1ffb640 1 -- 172.21.15.45:0/336441730 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae40b5400 con 0x7f2ae406da00 2024-05-08T05:50:44.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae1ffb640 1 -- 172.21.15.45:0/336441730 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f2acc0016f0 con 0x7f2ae406da00 2024-05-08T05:50:44.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae1ffb640 1 -- 172.21.15.45:0/336441730 >> v1:172.21.15.45:6789/0 conn(0x7f2ae406da00 legacy=0x7f2ae40b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae1ffb640 1 --2- 172.21.15.45:0/336441730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad0003a30 0x7f2ad0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae1ffb640 1 --2- 172.21.15.45:0/336441730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ad0004800 0x7f2ad0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae1ffb640 1 --2- 172.21.15.45:0/336441730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae4073800 0x7f2ad000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae1ffb640 1 -- 172.21.15.45:0/336441730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae40afaa0 con 0x7f2ad0003a30 2024-05-08T05:50:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae1ffb640 1 -- 172.21.15.45:0/336441730 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad00055e0 con 0x7f2ad0004800 2024-05-08T05:50:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae1ffb640 1 -- 172.21.15.45:0/336441730 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2ad000bf00 con 0x7f2ae4073800 2024-05-08T05:50:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae2ffd640 1 --2- 172.21.15.45:0/336441730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad0003a30 0x7f2ad0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae27fc640 1 --2- 172.21.15.45:0/336441730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ad0004800 0x7f2ad0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.059+0000 7f2ae37fe640 1 --2- 172.21.15.45:0/336441730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae4073800 0x7f2ad000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.060+0000 7f2ae2ffd640 1 -- 172.21.15.45:0/336441730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae4073800 msgr2=0x7f2ad000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.060+0000 7f2ae2ffd640 1 --2- 172.21.15.45:0/336441730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae4073800 0x7f2ad000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.060+0000 7f2ae2ffd640 1 -- 172.21.15.45:0/336441730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ad0004800 msgr2=0x7f2ad0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.060+0000 7f2ae2ffd640 1 --2- 172.21.15.45:0/336441730 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ad0004800 0x7f2ad0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.060+0000 7f2ae2ffd640 1 -- 172.21.15.45:0/336441730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad0003520 con 0x7f2ad0003a30 2024-05-08T05:50:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.060+0000 7f2ae37fe640 1 --2- 172.21.15.45:0/336441730 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae4073800 0x7f2ad000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.060+0000 7f2ae2ffd640 1 --2- 172.21.15.45:0/336441730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad0003a30 0x7f2ad0003e50 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f2adc001e90 tx=0x7f2ad0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147fa9db40d2c2a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.060+0000 7f2ae1ffb640 1 -- 172.21.15.45:0/336441730 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc001ab0 con 0x7f2ad0003a30 2024-05-08T05:50:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.060+0000 7f2ae1ffb640 1 -- 172.21.15.45:0/336441730 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2adc02ae50 con 0x7f2ad0003a30 2024-05-08T05:50:44.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.060+0000 7f2ae1ffb640 1 -- 172.21.15.45:0/336441730 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc003b70 con 0x7f2ad0003a30 2024-05-08T05:50:44.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.061+0000 7f2ae9371640 1 -- 172.21.15.45:0/336441730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad0003a30 msgr2=0x7f2ad0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.061+0000 7f2ae9371640 1 --2- 172.21.15.45:0/336441730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad0003a30 0x7f2ad0003e50 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f2adc001e90 tx=0x7f2ad0005830 comp rx=0 tx=0).stop 2024-05-08T05:50:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.061+0000 7f2ae37fe640 1 -- 172.21.15.45:0/336441730 reap_dead start 2024-05-08T05:50:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.061+0000 7f2ae9371640 1 -- 172.21.15.45:0/336441730 shutdown_connections 2024-05-08T05:50:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.061+0000 7f2ae9371640 1 -- 172.21.15.45:0/336441730 >> 172.21.15.45:0/336441730 conn(0x7f2ae40728c0 msgr2=0x7f2ae4072cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.061+0000 7f2ae9371640 1 -- 172.21.15.45:0/336441730 shutdown_connections 2024-05-08T05:50:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.061+0000 7f2ae9371640 1 -- 172.21.15.45:0/336441730 wait complete. 2024-05-08T05:50:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.062+0000 7f2ae9371640 1 Processor -- start 2024-05-08T05:50:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.062+0000 7f2ae9371640 1 -- start start 2024-05-08T05:50:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.063+0000 7f2ae9371640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad0003a30 0x7f2ae4156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.063+0000 7f2ae9371640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ae4152ff0 0x7f2ae41509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.063+0000 7f2ae9371640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae4153470 0x7f2ae4150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.063+0000 7f2ae9371640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41637b0 con 0x7f2ad0003a30 2024-05-08T05:50:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.063+0000 7f2ae9371640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae40b5a50 con 0x7f2ae4152ff0 2024-05-08T05:50:44.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.063+0000 7f2ae9371640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae4163930 con 0x7f2ae4153470 2024-05-08T05:50:44.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.063+0000 7f2ae2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad0003a30 0x7f2ae4156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:44.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.063+0000 7f2ae2ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad0003a30 0x7f2ae4156be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50520/0 (socket says 172.21.15.45:50520) 2024-05-08T05:50:44.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.063+0000 7f2ae2ffd640 1 -- 172.21.15.45:0/1534588732 learned_addr learned my addr 172.21.15.45:0/1534588732 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:44.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.063+0000 7f2ae37fe640 1 --2- 172.21.15.45:0/1534588732 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae4153470 0x7f2ae4150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:44.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.063+0000 7f2ae27fc640 1 --2- 172.21.15.45:0/1534588732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ae4152ff0 0x7f2ae41509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:44.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.064+0000 7f2ae2ffd640 1 -- 172.21.15.45:0/1534588732 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae4153470 msgr2=0x7f2ae4150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:44.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.064+0000 7f2ae2ffd640 1 --2- 172.21.15.45:0/1534588732 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae4153470 0x7f2ae4150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.064+0000 7f2ae2ffd640 1 -- 172.21.15.45:0/1534588732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ae4152ff0 msgr2=0x7f2ae41509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:44.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.064+0000 7f2ae2ffd640 1 --2- 172.21.15.45:0/1534588732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ae4152ff0 0x7f2ae41509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.064+0000 7f2ae2ffd640 1 -- 172.21.15.45:0/1534588732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae4155b40 con 0x7f2ad0003a30 2024-05-08T05:50:44.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.064+0000 7f2ae2ffd640 1 --2- 172.21.15.45:0/1534588732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad0003a30 0x7f2ae4156be0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f2adc001e90 tx=0x7f2adc0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:44.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.064+0000 7f2adb7fe640 1 -- 172.21.15.45:0/1534588732 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc040030 con 0x7f2ad0003a30 2024-05-08T05:50:44.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.064+0000 7f2adb7fe640 1 -- 172.21.15.45:0/1534588732 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2adc02ac50 con 0x7f2ad0003a30 2024-05-08T05:50:44.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.064+0000 7f2ae9371640 1 -- 172.21.15.45:0/1534588732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae4155e20 con 0x7f2ad0003a30 2024-05-08T05:50:44.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.064+0000 7f2adb7fe640 1 -- 172.21.15.45:0/1534588732 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2adc007df0 con 0x7f2ad0003a30 2024-05-08T05:50:44.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.064+0000 7f2ae9371640 1 -- 172.21.15.45:0/1534588732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae41563e0 con 0x7f2ad0003a30 2024-05-08T05:50:44.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.065+0000 7f2ae9371640 1 -- 172.21.15.45:0/1534588732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae4156820 con 0x7f2ad0003a30 2024-05-08T05:50:44.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.066+0000 7f2adb7fe640 1 -- 172.21.15.45:0/1534588732 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2adc043070 con 0x7f2ad0003a30 2024-05-08T05:50:44.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.066+0000 7f2adb7fe640 1 --2- 172.21.15.45:0/1534588732 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2ac40762d0 0x7f2ac4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:44.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.066+0000 7f2adb7fe640 1 -- 172.21.15.45:0/1534588732 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6360+0+0 (secure 0 0 0) 0x7f2adc0b7900 con 0x7f2ad0003a30 2024-05-08T05:50:44.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.066+0000 7f2ae27fc640 1 --2- 172.21.15.45:0/1534588732 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2ac40762d0 0x7f2ac4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:44.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.067+0000 7f2ae27fc640 1 --2- 172.21.15.45:0/1534588732 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2ac40762d0 0x7f2ac4078790 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2acc001240 tx=0x7f2acc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:44.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.070+0000 7f2adb7fe640 1 -- 172.21.15.45:0/1534588732 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2adc080fd0 con 0x7f2ad0003a30 2024-05-08T05:50:44.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.301+0000 7f2ae9371640 1 -- 172.21.15.45:0/1534588732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ae40693f0 con 0x7f2ad0003a30 2024-05-08T05:50:44.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.302+0000 7f2adb7fe640 1 -- 172.21.15.45:0/1534588732 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+4345 (secure 0 0 0) 0x7f2adc0808d0 con 0x7f2ad0003a30 2024-05-08T05:50:44.302 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:50:44.302 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3692881261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3692881261},{"type":"v1","addr":"172.21.15.63:6836","nonce":3692881261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7497,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2916399818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2916399818},{"type":"v1","addr":"172.21.15.45:6833","nonce":2916399818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7515,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2978250004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":2978250004},{"type":"v1","addr":"172.21.15.63:6837","nonce":2978250004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:49:19.587326+0000","modified":"2024-05-08T05:50:43.627486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-08T05:50:44.302 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 89 2024-05-08T05:50:44.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.304+0000 7f2ae9371640 1 -- 172.21.15.45:0/1534588732 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2ac40762d0 msgr2=0x7f2ac4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:44.304 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.304+0000 7f2ae9371640 1 --2- 172.21.15.45:0/1534588732 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2ac40762d0 0x7f2ac4078790 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2acc001240 tx=0x7f2acc009040 comp rx=0 tx=0).stop 2024-05-08T05:50:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.304+0000 7f2ae9371640 1 -- 172.21.15.45:0/1534588732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad0003a30 msgr2=0x7f2ae4156be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.304+0000 7f2ae9371640 1 --2- 172.21.15.45:0/1534588732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad0003a30 0x7f2ae4156be0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f2adc001e90 tx=0x7f2adc0014a0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.304+0000 7f2ae9371640 1 -- 172.21.15.45:0/1534588732 shutdown_connections 2024-05-08T05:50:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.304+0000 7f2ae9371640 1 --2- 172.21.15.45:0/1534588732 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2ac40762d0 0x7f2ac4078790 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.304+0000 7f2ae9371640 1 --2- 172.21.15.45:0/1534588732 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2ae4153470 0x7f2ae4150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.304+0000 7f2ae9371640 1 --2- 172.21.15.45:0/1534588732 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2ae4152ff0 0x7f2ae41509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.304+0000 7f2ae9371640 1 --2- 172.21.15.45:0/1534588732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ad0003a30 0x7f2ae4156be0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.304+0000 7f2ae9371640 1 -- 172.21.15.45:0/1534588732 >> 172.21.15.45:0/1534588732 conn(0x7f2ae40728c0 msgr2=0x7f2ae4072ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.304+0000 7f2ae9371640 1 -- 172.21.15.45:0/1534588732 shutdown_connections 2024-05-08T05:50:44.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.304+0000 7f2ae9371640 1 -- 172.21.15.45:0/1534588732 wait complete. 2024-05-08T05:50:44.316 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:50:44.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.420+0000 7fcde1b11640 1 Processor -- start 2024-05-08T05:50:44.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.420+0000 7fcde1b11640 1 -- start start 2024-05-08T05:50:44.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.421+0000 7fcde1b11640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcddc1629a0 0x7fcddc0d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:44.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.421+0000 7fcde1b11640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcddc0d2dc0 0x7fcddc0d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.421+0000 7fcde1b11640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcddc0d3330 con 0x7fcddc0c8ed0 2024-05-08T05:50:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.421+0000 7fcde1b11640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fcddc0d2170 con 0x7fcddc0d2dc0 2024-05-08T05:50:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.421+0000 7fcde1b11640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fcddc0d2450 con 0x7fcddc1629a0 2024-05-08T05:50:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.421+0000 7fcddaffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcddc0d2dc0 0x7fcddc0d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.421+0000 7fcddaffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcddc0d2dc0 0x7fcddc0d3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50532/0 (socket says 172.21.15.45:50532) 2024-05-08T05:50:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.421+0000 7fcddaffd640 1 -- 172.21.15.45:0/143895270 learned_addr learned my addr 172.21.15.45:0/143895270 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.421+0000 7fcddb7fe640 1 --2- 172.21.15.45:0/143895270 >> v2:172.21.15.63:3300/0 conn(0x7fcddc1629a0 0x7fcddc0d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.421+0000 7fcddaffd640 1 -- 172.21.15.45:0/143895270 >> v2:172.21.15.63:3300/0 conn(0x7fcddc1629a0 msgr2=0x7fcddc0d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.421+0000 7fcddaffd640 1 --2- 172.21.15.45:0/143895270 >> v2:172.21.15.63:3300/0 conn(0x7fcddc1629a0 0x7fcddc0d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.421+0000 7fcddaffd640 1 -- 172.21.15.45:0/143895270 >> v1:172.21.15.63:6789/0 conn(0x7fcddc0c8ed0 legacy=0x7fcddc0c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:44.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.421+0000 7fcddaffd640 1 -- 172.21.15.45:0/143895270 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcddc0d2730 con 0x7fcddc0d2dc0 2024-05-08T05:50:44.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.423+0000 7fcddaffd640 1 --2- 172.21.15.45:0/143895270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcddc0d2dc0 0x7fcddc0d3b00 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fcddc0768a0 tx=0x7fcdd002a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec4785dce643c91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:44.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.423+0000 7fcdda7fc640 1 -- 172.21.15.45:0/143895270 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0037070 con 0x7fcddc0d2dc0 2024-05-08T05:50:44.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.423+0000 7fcdda7fc640 1 -- 172.21.15.45:0/143895270 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdd0032440 con 0x7fcddc0d2dc0 2024-05-08T05:50:44.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.423+0000 7fcdda7fc640 1 -- 172.21.15.45:0/143895270 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0032810 con 0x7fcddc0d2dc0 2024-05-08T05:50:44.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.424+0000 7fcde1b11640 1 -- 172.21.15.45:0/143895270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcddc0d2dc0 msgr2=0x7fcddc0d3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.424+0000 7fcde1b11640 1 --2- 172.21.15.45:0/143895270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcddc0d2dc0 0x7fcddc0d3b00 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fcddc0768a0 tx=0x7fcdd002a160 comp rx=0 tx=0).stop 2024-05-08T05:50:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.424+0000 7fcde1b11640 1 -- 172.21.15.45:0/143895270 shutdown_connections 2024-05-08T05:50:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.424+0000 7fcde1b11640 1 --2- 172.21.15.45:0/143895270 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcddc0d2dc0 0x7fcddc0d3b00 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.424+0000 7fcde1b11640 1 --2- 172.21.15.45:0/143895270 >> v2:172.21.15.63:3300/0 conn(0x7fcddc1629a0 0x7fcddc0d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.424+0000 7fcde1b11640 1 -- 172.21.15.45:0/143895270 >> 172.21.15.45:0/143895270 conn(0x7fcddc0d4fe0 msgr2=0x7fcddc0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.424+0000 7fcde1b11640 1 -- 172.21.15.45:0/143895270 shutdown_connections 2024-05-08T05:50:44.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.424+0000 7fcde1b11640 1 -- 172.21.15.45:0/143895270 wait complete. 2024-05-08T05:50:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.425+0000 7fcde1b11640 1 Processor -- start 2024-05-08T05:50:44.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.426+0000 7fcde1b11640 1 -- start start 2024-05-08T05:50:44.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.426+0000 7fcde1b11640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcddc0c8ed0 0x7fcddc15b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:44.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.426+0000 7fcde1b11640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcddc0d2dc0 0x7fcddc152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.426+0000 7fcde1b11640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcddc1629a0 0x7fcddc0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.426+0000 7fcde1b11640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc0cf440 con 0x7fcddc1629a0 2024-05-08T05:50:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.426+0000 7fcde1b11640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc0cf2c0 con 0x7fcddc0d2dc0 2024-05-08T05:50:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.426+0000 7fcde1b11640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcddc0d0cb0 con 0x7fcddc0c8ed0 2024-05-08T05:50:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.426+0000 7fcddbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcddc1629a0 0x7fcddc0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.426+0000 7fcddbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcddc1629a0 0x7fcddc0cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50544/0 (socket says 172.21.15.45:50544) 2024-05-08T05:50:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.426+0000 7fcddbfff640 1 -- 172.21.15.45:0/2828363454 learned_addr learned my addr 172.21.15.45:0/2828363454 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.427+0000 7fcddaffd640 1 --2- 172.21.15.45:0/2828363454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcddc0d2dc0 0x7fcddc152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.427+0000 7fcddb7fe640 1 --2- 172.21.15.45:0/2828363454 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcddc0c8ed0 0x7fcddc15b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.427+0000 7fcddbfff640 1 -- 172.21.15.45:0/2828363454 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcddc0c8ed0 msgr2=0x7fcddc15b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:44.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.427+0000 7fcddbfff640 1 --2- 172.21.15.45:0/2828363454 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcddc0c8ed0 0x7fcddc15b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.427+0000 7fcddbfff640 1 -- 172.21.15.45:0/2828363454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcddc0d2dc0 msgr2=0x7fcddc152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.427+0000 7fcddbfff640 1 --2- 172.21.15.45:0/2828363454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcddc0d2dc0 0x7fcddc152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.427+0000 7fcddbfff640 1 -- 172.21.15.45:0/2828363454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcddc0cc250 con 0x7fcddc1629a0 2024-05-08T05:50:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.427+0000 7fcddbfff640 1 --2- 172.21.15.45:0/2828363454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcddc1629a0 0x7fcddc0cd010 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400c7d0 tx=0x7fcdd400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.427+0000 7fcdc3fff640 1 -- 172.21.15.45:0/2828363454 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4003b70 con 0x7fcddc1629a0 2024-05-08T05:50:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.427+0000 7fcdc3fff640 1 -- 172.21.15.45:0/2828363454 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdd4003d30 con 0x7fcddc1629a0 2024-05-08T05:50:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.427+0000 7fcdc3fff640 1 -- 172.21.15.45:0/2828363454 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd400aa90 con 0x7fcddc1629a0 2024-05-08T05:50:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.427+0000 7fcde1b11640 1 -- 172.21.15.45:0/2828363454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcddc0cc530 con 0x7fcddc1629a0 2024-05-08T05:50:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.427+0000 7fcde1b11640 1 -- 172.21.15.45:0/2828363454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcddc15c020 con 0x7fcddc1629a0 2024-05-08T05:50:44.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.428+0000 7fcde1b11640 1 -- 172.21.15.45:0/2828363454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcddc0693f0 con 0x7fcddc1629a0 2024-05-08T05:50:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.429+0000 7fcdc3fff640 1 -- 172.21.15.45:0/2828363454 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcdd400ac30 con 0x7fcddc1629a0 2024-05-08T05:50:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.429+0000 7fcdc3fff640 1 --2- 172.21.15.45:0/2828363454 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcdc80762d0 0x7fcdc8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.429+0000 7fcdc3fff640 1 -- 172.21.15.45:0/2828363454 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6360+0+0 (secure 0 0 0) 0x7fcdd4097600 con 0x7fcddc1629a0 2024-05-08T05:50:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.430+0000 7fcddb7fe640 1 --2- 172.21.15.45:0/2828363454 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcdc80762d0 0x7fcdc8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:44.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.430+0000 7fcddb7fe640 1 --2- 172.21.15.45:0/2828363454 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcdc80762d0 0x7fcdc8078790 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fcddc162380 tx=0x7fcdcc009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:44.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.434+0000 7fcdc3fff640 1 -- 172.21.15.45:0/2828363454 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcdd4060680 con 0x7fcddc1629a0 2024-05-08T05:50:44.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.669+0000 7fcde1b11640 1 -- 172.21.15.45:0/2828363454 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcddc160c60 con 0x7fcddc1629a0 2024-05-08T05:50:44.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.669+0000 7fcdc3fff640 1 -- 172.21.15.45:0/2828363454 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5170 (secure 0 0 0) 0x7fcdd4064980 con 0x7fcddc1629a0 2024-05-08T05:50:44.670 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:50:44.670 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3692881261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3692881261},{"type":"v1","addr":"172.21.15.63:6836","nonce":3692881261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7497,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2916399818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2916399818},{"type":"v1","addr":"172.21.15.45:6833","nonce":2916399818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7515,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2978250004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":2978250004},{"type":"v1","addr":"172.21.15.63:6837","nonce":2978250004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7962,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4201508950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4201508950},{"type":"v1","addr":"172.21.15.45:6835","nonce":4201508950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:49:19.587326+0000","modified":"2024-05-08T05:50:43.627486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-08T05:50:44.670 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 90 2024-05-08T05:50:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.671+0000 7fcde1b11640 1 -- 172.21.15.45:0/2828363454 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcdc80762d0 msgr2=0x7fcdc8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.671+0000 7fcde1b11640 1 --2- 172.21.15.45:0/2828363454 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcdc80762d0 0x7fcdc8078790 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fcddc162380 tx=0x7fcdcc009190 comp rx=0 tx=0).stop 2024-05-08T05:50:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.671+0000 7fcde1b11640 1 -- 172.21.15.45:0/2828363454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcddc1629a0 msgr2=0x7fcddc0cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.671+0000 7fcde1b11640 1 --2- 172.21.15.45:0/2828363454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcddc1629a0 0x7fcddc0cd010 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400c7d0 tx=0x7fcdd400cc90 comp rx=0 tx=0).stop 2024-05-08T05:50:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.671+0000 7fcde1b11640 1 -- 172.21.15.45:0/2828363454 shutdown_connections 2024-05-08T05:50:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.671+0000 7fcde1b11640 1 --2- 172.21.15.45:0/2828363454 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcdc80762d0 0x7fcdc8078790 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.671+0000 7fcde1b11640 1 --2- 172.21.15.45:0/2828363454 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcddc1629a0 0x7fcddc0cd010 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.671+0000 7fcde1b11640 1 --2- 172.21.15.45:0/2828363454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcddc0d2dc0 0x7fcddc152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.671+0000 7fcde1b11640 1 --2- 172.21.15.45:0/2828363454 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcddc0c8ed0 0x7fcddc15b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.671+0000 7fcde1b11640 1 -- 172.21.15.45:0/2828363454 >> 172.21.15.45:0/2828363454 conn(0x7fcddc0d4fe0 msgr2=0x7fcddc06e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.672+0000 7fcde1b11640 1 -- 172.21.15.45:0/2828363454 shutdown_connections 2024-05-08T05:50:44.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.672+0000 7fcde1b11640 1 -- 172.21.15.45:0/2828363454 wait complete. 2024-05-08T05:50:44.683 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:50:44.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.786+0000 7fa21f6a0640 1 Processor -- start 2024-05-08T05:50:44.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.787+0000 7fa21f6a0640 1 -- start start 2024-05-08T05:50:44.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.787+0000 7fa21f6a0640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa2180d1970 0x7fa2180d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:44.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.787+0000 7fa21f6a0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2180d4cc0 0x7fa2180d5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:44.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.787+0000 7fa21f6a0640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2181630b0 con 0x7fa21815f420 2024-05-08T05:50:44.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.787+0000 7fa21f6a0640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa2180d3e70 con 0x7fa2180d1970 2024-05-08T05:50:44.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.787+0000 7fa21f6a0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa2180d5670 con 0x7fa2180d4cc0 2024-05-08T05:50:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.788+0000 7fa21dc3d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2180d4cc0 0x7fa2180d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.788+0000 7fa21cc3b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa2180d1970 0x7fa2180d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:44.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.788+0000 7fa21dc3d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2180d4cc0 0x7fa2180d5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:33650/0 (socket says 172.21.15.45:33650) 2024-05-08T05:50:44.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.788+0000 7fa21dc3d640 1 -- 172.21.15.45:0/2265731011 learned_addr learned my addr 172.21.15.45:0/2265731011 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:44.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.788+0000 7fa21cc3b640 1 -- 172.21.15.45:0/2265731011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2180d4cc0 msgr2=0x7fa2180d5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:44.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.788+0000 7fa21cc3b640 1 --2- 172.21.15.45:0/2265731011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2180d4cc0 0x7fa2180d5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.788+0000 7fa21cc3b640 1 -- 172.21.15.45:0/2265731011 >> v1:172.21.15.63:6789/0 conn(0x7fa21815f420 legacy=0x7fa2180d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:44.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.788+0000 7fa21cc3b640 1 -- 172.21.15.45:0/2265731011 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2180d2550 con 0x7fa2180d1970 2024-05-08T05:50:44.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.788+0000 7fa21dc3d640 1 --2- 172.21.15.45:0/2265731011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2180d4cc0 0x7fa2180d5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:50:44.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.789+0000 7fa21cc3b640 1 --2- 172.21.15.45:0/2265731011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2180d1970 0x7fa2180d1e20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0097d0 tx=0x7fa20c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd1529679c515145 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:44.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.789+0000 7fa217fff640 1 -- 172.21.15.45:0/2265731011 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c037070 con 0x7fa2180d1970 2024-05-08T05:50:44.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.789+0000 7fa217fff640 1 -- 172.21.15.45:0/2265731011 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa20c032440 con 0x7fa2180d1970 2024-05-08T05:50:44.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.789+0000 7fa217fff640 1 -- 172.21.15.45:0/2265731011 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c032810 con 0x7fa2180d1970 2024-05-08T05:50:44.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.790+0000 7fa21f6a0640 1 -- 172.21.15.45:0/2265731011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2180d1970 msgr2=0x7fa2180d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:44.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.790+0000 7fa21f6a0640 1 --2- 172.21.15.45:0/2265731011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2180d1970 0x7fa2180d1e20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0097d0 tx=0x7fa20c02a1c0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.790+0000 7fa21f6a0640 1 -- 172.21.15.45:0/2265731011 shutdown_connections 2024-05-08T05:50:44.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.790+0000 7fa21f6a0640 1 --2- 172.21.15.45:0/2265731011 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2180d4cc0 0x7fa2180d5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.790+0000 7fa21f6a0640 1 --2- 172.21.15.45:0/2265731011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2180d1970 0x7fa2180d1e20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.790+0000 7fa21f6a0640 1 -- 172.21.15.45:0/2265731011 >> 172.21.15.45:0/2265731011 conn(0x7fa21808eac0 msgr2=0x7fa21809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:44.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.790+0000 7fa21f6a0640 1 -- 172.21.15.45:0/2265731011 shutdown_connections 2024-05-08T05:50:44.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.790+0000 7fa21f6a0640 1 -- 172.21.15.45:0/2265731011 wait complete. 2024-05-08T05:50:44.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.791+0000 7fa21f6a0640 1 Processor -- start 2024-05-08T05:50:44.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.791+0000 7fa21f6a0640 1 -- start start 2024-05-08T05:50:44.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.792+0000 7fa21f6a0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2180d1970 0x7fa218154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:44.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.792+0000 7fa21f6a0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2180d4cc0 0x7fa21815d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:44.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.792+0000 7fa21f6a0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa21815ddf0 0x7fa2180b7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:44.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.792+0000 7fa21f6a0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa2180d3240 con 0x7fa21815ddf0 2024-05-08T05:50:44.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.792+0000 7fa21f6a0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa2180d2f60 con 0x7fa2180d1970 2024-05-08T05:50:44.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.792+0000 7fa21f6a0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa2180d2830 con 0x7fa2180d4cc0 2024-05-08T05:50:44.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.792+0000 7fa21dc3d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa21815ddf0 0x7fa2180b7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:44.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.792+0000 7fa21dc3d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa21815ddf0 0x7fa2180b7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50558/0 (socket says 172.21.15.45:50558) 2024-05-08T05:50:44.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.792+0000 7fa21dc3d640 1 -- 172.21.15.45:0/2137731642 learned_addr learned my addr 172.21.15.45:0/2137731642 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:44.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.792+0000 7fa21cc3b640 1 --2- 172.21.15.45:0/2137731642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2180d4cc0 0x7fa21815d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:44.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.792+0000 7fa21d43c640 1 --2- 172.21.15.45:0/2137731642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2180d1970 0x7fa218154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:44.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.792+0000 7fa21dc3d640 1 -- 172.21.15.45:0/2137731642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2180d4cc0 msgr2=0x7fa21815d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:44.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.792+0000 7fa21dc3d640 1 --2- 172.21.15.45:0/2137731642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2180d4cc0 0x7fa21815d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.792+0000 7fa21dc3d640 1 -- 172.21.15.45:0/2137731642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2180d1970 msgr2=0x7fa218154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:44.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.792+0000 7fa21dc3d640 1 --2- 172.21.15.45:0/2137731642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2180d1970 0x7fa218154e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.792+0000 7fa21dc3d640 1 -- 172.21.15.45:0/2137731642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2180b8bd0 con 0x7fa21815ddf0 2024-05-08T05:50:44.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.792+0000 7fa21dc3d640 1 --2- 172.21.15.45:0/2137731642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa21815ddf0 0x7fa2180b7ea0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fa20800efc0 tx=0x7fa20800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:44.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.793+0000 7fa215ffb640 1 -- 172.21.15.45:0/2137731642 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20800cbe0 con 0x7fa21815ddf0 2024-05-08T05:50:44.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.793+0000 7fa215ffb640 1 -- 172.21.15.45:0/2137731642 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa20800cd80 con 0x7fa21815ddf0 2024-05-08T05:50:44.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.793+0000 7fa215ffb640 1 -- 172.21.15.45:0/2137731642 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa208003bb0 con 0x7fa21815ddf0 2024-05-08T05:50:44.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.793+0000 7fa21f6a0640 1 -- 172.21.15.45:0/2137731642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2180b8eb0 con 0x7fa21815ddf0 2024-05-08T05:50:44.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.793+0000 7fa21f6a0640 1 -- 172.21.15.45:0/2137731642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2180b9470 con 0x7fa21815ddf0 2024-05-08T05:50:44.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.794+0000 7fa21f6a0640 1 -- 172.21.15.45:0/2137731642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2180693f0 con 0x7fa21815ddf0 2024-05-08T05:50:44.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.794+0000 7fa215ffb640 1 -- 172.21.15.45:0/2137731642 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa208005d10 con 0x7fa21815ddf0 2024-05-08T05:50:44.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.795+0000 7fa215ffb640 1 --2- 172.21.15.45:0/2137731642 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa2040762d0 0x7fa204078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:44.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.795+0000 7fa215ffb640 1 -- 172.21.15.45:0/2137731642 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6360+0+0 (secure 0 0 0) 0x7fa208097c00 con 0x7fa21815ddf0 2024-05-08T05:50:44.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.795+0000 7fa21d43c640 1 --2- 172.21.15.45:0/2137731642 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa2040762d0 0x7fa204078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:44.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.796+0000 7fa21d43c640 1 --2- 172.21.15.45:0/2137731642 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa2040762d0 0x7fa204078790 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa2180d17d0 tx=0x7fa210009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:44.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.799+0000 7fa215ffb640 1 -- 172.21.15.45:0/2137731642 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa20809e080 con 0x7fa21815ddf0 2024-05-08T05:50:44.994 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.994+0000 7fa21f6a0640 1 -- 172.21.15.45:0/2137731642 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2181629b0 con 0x7fa21815ddf0 2024-05-08T05:50:44.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.995+0000 7fa215ffb640 1 -- 172.21.15.45:0/2137731642 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+17239 (secure 0 0 0) 0x7fa208022020 con 0x7fa21815ddf0 2024-05-08T05:50:44.995 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:50:44.995 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":108,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:50:43.627313+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:49:17.241406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-08T05:49:18.542707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.21","mappings":[{"from":6,"to":7}]},{"pgid":"16.25","mappings":[{"from":5,"to":4}]},{"pgid":"16.39","mappings":[{"from":1,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/2729952281":"2024-05-09T05:50:43.627290+0000","172.21.15.45:6834/2729952281":"2024-05-09T05:50:43.627290+0000","172.21.15.45:6835/878092981":"2024-05-09T05:50:07.216878+0000","172.21.15.45:6834/878092981":"2024-05-09T05:50:07.216878+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:50:44.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.997+0000 7fa21f6a0640 1 -- 172.21.15.45:0/2137731642 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa2040762d0 msgr2=0x7fa204078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:44.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.997+0000 7fa21f6a0640 1 --2- 172.21.15.45:0/2137731642 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa2040762d0 0x7fa204078790 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa2180d17d0 tx=0x7fa210009040 comp rx=0 tx=0).stop 2024-05-08T05:50:44.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.997+0000 7fa21f6a0640 1 -- 172.21.15.45:0/2137731642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa21815ddf0 msgr2=0x7fa2180b7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:44.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.997+0000 7fa21f6a0640 1 --2- 172.21.15.45:0/2137731642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa21815ddf0 0x7fa2180b7ea0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fa20800efc0 tx=0x7fa20800c490 comp rx=0 tx=0).stop 2024-05-08T05:50:44.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.997+0000 7fa21f6a0640 1 -- 172.21.15.45:0/2137731642 shutdown_connections 2024-05-08T05:50:44.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.997+0000 7fa21f6a0640 1 --2- 172.21.15.45:0/2137731642 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa2040762d0 0x7fa204078790 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.997+0000 7fa21f6a0640 1 --2- 172.21.15.45:0/2137731642 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa2180d4cc0 0x7fa21815d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.997+0000 7fa21f6a0640 1 --2- 172.21.15.45:0/2137731642 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa21815ddf0 0x7fa2180b7ea0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.997+0000 7fa21f6a0640 1 --2- 172.21.15.45:0/2137731642 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa2180d1970 0x7fa218154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:44.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.997+0000 7fa21f6a0640 1 -- 172.21.15.45:0/2137731642 >> 172.21.15.45:0/2137731642 conn(0x7fa21808eac0 msgr2=0x7fa2180d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:44.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.997+0000 7fa21f6a0640 1 -- 172.21.15.45:0/2137731642 shutdown_connections 2024-05-08T05:50:44.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:44.997+0000 7fa21f6a0640 1 -- 172.21.15.45:0/2137731642 wait complete. 2024-05-08T05:50:45.009 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T05:50:45.009 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:50:45.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.112+0000 7fc2e8971640 1 Processor -- start 2024-05-08T05:50:45.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.112+0000 7fc2e8971640 1 -- start start 2024-05-08T05:50:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.113+0000 7fc2e8971640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc2e40d33f0 0x7fc2e415d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.113+0000 7fc2e8971640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2e4070b90 0x7fc2e4071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.113+0000 7fc2e8971640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2e40718c0 con 0x7fc2e415ccb0 2024-05-08T05:50:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.113+0000 7fc2e8971640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc2e40afba0 con 0x7fc2e40d33f0 2024-05-08T05:50:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.113+0000 7fc2e8971640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc2e4071ba0 con 0x7fc2e4070b90 2024-05-08T05:50:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.113+0000 7fc2d9d9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc2e40d33f0 0x7fc2e415d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.113+0000 7fc2d9d9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc2e40d33f0 0x7fc2e415d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50572/0 (socket says 172.21.15.45:50572) 2024-05-08T05:50:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.113+0000 7fc2d9d9b640 1 -- 172.21.15.45:0/2329064775 learned_addr learned my addr 172.21.15.45:0/2329064775 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.113+0000 7fc2e2d9d640 1 --2- 172.21.15.45:0/2329064775 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2e4070b90 0x7fc2e4071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.113+0000 7fc2d9d9b640 1 -- 172.21.15.45:0/2329064775 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2e4070b90 msgr2=0x7fc2e4071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.113+0000 7fc2d9d9b640 1 --2- 172.21.15.45:0/2329064775 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2e4070b90 0x7fc2e4071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.113+0000 7fc2d9d9b640 1 -- 172.21.15.45:0/2329064775 >> v1:172.21.15.63:6789/0 conn(0x7fc2e415ccb0 legacy=0x7fc2e40d2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.113+0000 7fc2d9d9b640 1 -- 172.21.15.45:0/2329064775 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e4071e80 con 0x7fc2e40d33f0 2024-05-08T05:50:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.114+0000 7fc2d9d9b640 1 --2- 172.21.15.45:0/2329064775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2e40d33f0 0x7fc2e415d090 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009870 tx=0x7fc2cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5e346b34104167 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.114+0000 7fc2e1d9b640 1 -- 172.21.15.45:0/2329064775 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc037070 con 0x7fc2e40d33f0 2024-05-08T05:50:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.114+0000 7fc2e1d9b640 1 -- 172.21.15.45:0/2329064775 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2cc032440 con 0x7fc2e40d33f0 2024-05-08T05:50:45.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.114+0000 7fc2e1d9b640 1 -- 172.21.15.45:0/2329064775 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc032810 con 0x7fc2e40d33f0 2024-05-08T05:50:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.114+0000 7fc2e8971640 1 -- 172.21.15.45:0/2329064775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2e40d33f0 msgr2=0x7fc2e415d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.114+0000 7fc2e8971640 1 --2- 172.21.15.45:0/2329064775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2e40d33f0 0x7fc2e415d090 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009870 tx=0x7fc2cc02a270 comp rx=0 tx=0).stop 2024-05-08T05:50:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.115+0000 7fc2e8971640 1 -- 172.21.15.45:0/2329064775 shutdown_connections 2024-05-08T05:50:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.115+0000 7fc2e8971640 1 --2- 172.21.15.45:0/2329064775 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2e4070b90 0x7fc2e4071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.115+0000 7fc2e8971640 1 --2- 172.21.15.45:0/2329064775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2e40d33f0 0x7fc2e415d090 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.115+0000 7fc2e8971640 1 -- 172.21.15.45:0/2329064775 >> 172.21.15.45:0/2329064775 conn(0x7fc2e409ba40 msgr2=0x7fc2e409be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.115+0000 7fc2e8971640 1 -- 172.21.15.45:0/2329064775 shutdown_connections 2024-05-08T05:50:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.115+0000 7fc2e8971640 1 -- 172.21.15.45:0/2329064775 wait complete. 2024-05-08T05:50:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.116+0000 7fc2e8971640 1 Processor -- start 2024-05-08T05:50:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.116+0000 7fc2e8971640 1 -- start start 2024-05-08T05:50:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.117+0000 7fc2e8971640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2e4070b90 0x7fc2e415fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.117+0000 7fc2e8971640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2e40d33f0 0x7fc2e41569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.117+0000 7fc2e8971640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2e415ccb0 0x7fc2e40b8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.117+0000 7fc2e8971640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e40cf980 con 0x7fc2e415ccb0 2024-05-08T05:50:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.117+0000 7fc2e8971640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e4071560 con 0x7fc2e4070b90 2024-05-08T05:50:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.117+0000 7fc2e8971640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc2e40cfb00 con 0x7fc2e40d33f0 2024-05-08T05:50:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.117+0000 7fc2e2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2e415ccb0 0x7fc2e40b8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.117+0000 7fc2e2d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2e415ccb0 0x7fc2e40b8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50590/0 (socket says 172.21.15.45:50590) 2024-05-08T05:50:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.117+0000 7fc2e2d9d640 1 -- 172.21.15.45:0/2562690249 learned_addr learned my addr 172.21.15.45:0/2562690249 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.117+0000 7fc2d9d9b640 1 --2- 172.21.15.45:0/2562690249 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2e40d33f0 0x7fc2e41569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.117+0000 7fc2e259c640 1 --2- 172.21.15.45:0/2562690249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2e4070b90 0x7fc2e415fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.117+0000 7fc2e2d9d640 1 -- 172.21.15.45:0/2562690249 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2e40d33f0 msgr2=0x7fc2e41569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.117+0000 7fc2e2d9d640 1 --2- 172.21.15.45:0/2562690249 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2e40d33f0 0x7fc2e41569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.117+0000 7fc2e2d9d640 1 -- 172.21.15.45:0/2562690249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2e4070b90 msgr2=0x7fc2e415fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.117+0000 7fc2e2d9d640 1 --2- 172.21.15.45:0/2562690249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2e4070b90 0x7fc2e415fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.117+0000 7fc2e2d9d640 1 -- 172.21.15.45:0/2562690249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e40ccfc0 con 0x7fc2e415ccb0 2024-05-08T05:50:45.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.118+0000 7fc2e2d9d640 1 --2- 172.21.15.45:0/2562690249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2e415ccb0 0x7fc2e40b8210 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009d30 tx=0x7fc2dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:45.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.118+0000 7fc2db7fe640 1 -- 172.21.15.45:0/2562690249 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc018020 con 0x7fc2e415ccb0 2024-05-08T05:50:45.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.118+0000 7fc2db7fe640 1 -- 172.21.15.45:0/2562690249 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2dc014480 con 0x7fc2e415ccb0 2024-05-08T05:50:45.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.118+0000 7fc2e8971640 1 -- 172.21.15.45:0/2562690249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e40b7450 con 0x7fc2e415ccb0 2024-05-08T05:50:45.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.118+0000 7fc2db7fe640 1 -- 172.21.15.45:0/2562690249 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc014870 con 0x7fc2e415ccb0 2024-05-08T05:50:45.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.118+0000 7fc2e8971640 1 -- 172.21.15.45:0/2562690249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e40b7a10 con 0x7fc2e415ccb0 2024-05-08T05:50:45.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.119+0000 7fc2e8971640 1 -- 172.21.15.45:0/2562690249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e4069b00 con 0x7fc2e415ccb0 2024-05-08T05:50:45.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.119+0000 7fc2db7fe640 1 -- 172.21.15.45:0/2562690249 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc2dc005d10 con 0x7fc2e415ccb0 2024-05-08T05:50:45.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.120+0000 7fc2db7fe640 1 --2- 172.21.15.45:0/2562690249 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc2c40762d0 0x7fc2c4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.120+0000 7fc2db7fe640 1 -- 172.21.15.45:0/2562690249 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6360+0+0 (secure 0 0 0) 0x7fc2dc0980a0 con 0x7fc2e415ccb0 2024-05-08T05:50:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.120+0000 7fc2e259c640 1 --2- 172.21.15.45:0/2562690249 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc2c40762d0 0x7fc2c4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.121+0000 7fc2e259c640 1 --2- 172.21.15.45:0/2562690249 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc2c40762d0 0x7fc2c4078790 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40017c0 tx=0x7fc2d4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:45.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.124+0000 7fc2db7fe640 1 -- 172.21.15.45:0/2562690249 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc2dc022070 con 0x7fc2e415ccb0 2024-05-08T05:50:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.357+0000 7fc2e8971640 1 -- 172.21.15.45:0/2562690249 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2e4070380 con 0x7fc2e415ccb0 2024-05-08T05:50:45.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.358+0000 7fc2db7fe640 1 -- 172.21.15.45:0/2562690249 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5170 (secure 0 0 0) 0x7fc2dc010040 con 0x7fc2e415ccb0 2024-05-08T05:50:45.358 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:50:45.358 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3692881261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3692881261},{"type":"v1","addr":"172.21.15.63:6836","nonce":3692881261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7497,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2916399818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2916399818},{"type":"v1","addr":"172.21.15.45:6833","nonce":2916399818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7515,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2978250004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":2978250004},{"type":"v1","addr":"172.21.15.63:6837","nonce":2978250004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7962,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4201508950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4201508950},{"type":"v1","addr":"172.21.15.45:6835","nonce":4201508950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:49:19.587326+0000","modified":"2024-05-08T05:50:43.627486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-08T05:50:45.358 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 90 2024-05-08T05:50:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.359+0000 7fc2e8971640 1 -- 172.21.15.45:0/2562690249 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc2c40762d0 msgr2=0x7fc2c4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.359+0000 7fc2e8971640 1 --2- 172.21.15.45:0/2562690249 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc2c40762d0 0x7fc2c4078790 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40017c0 tx=0x7fc2d4009040 comp rx=0 tx=0).stop 2024-05-08T05:50:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.360+0000 7fc2e8971640 1 -- 172.21.15.45:0/2562690249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2e415ccb0 msgr2=0x7fc2e40b8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.360+0000 7fc2e8971640 1 --2- 172.21.15.45:0/2562690249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2e415ccb0 0x7fc2e40b8210 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009d30 tx=0x7fc2dc00c490 comp rx=0 tx=0).stop 2024-05-08T05:50:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.360+0000 7fc2e8971640 1 -- 172.21.15.45:0/2562690249 shutdown_connections 2024-05-08T05:50:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.360+0000 7fc2e8971640 1 --2- 172.21.15.45:0/2562690249 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc2c40762d0 0x7fc2c4078790 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.360+0000 7fc2e8971640 1 --2- 172.21.15.45:0/2562690249 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2e40d33f0 0x7fc2e41569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.360+0000 7fc2e8971640 1 --2- 172.21.15.45:0/2562690249 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2e415ccb0 0x7fc2e40b8210 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.360+0000 7fc2e8971640 1 --2- 172.21.15.45:0/2562690249 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2e4070b90 0x7fc2e415fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.360+0000 7fc2e8971640 1 -- 172.21.15.45:0/2562690249 >> 172.21.15.45:0/2562690249 conn(0x7fc2e409ba40 msgr2=0x7fc2e40ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:45.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.360+0000 7fc2e8971640 1 -- 172.21.15.45:0/2562690249 shutdown_connections 2024-05-08T05:50:45.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.360+0000 7fc2e8971640 1 -- 172.21.15.45:0/2562690249 wait complete. 2024-05-08T05:50:45.371 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:50:45.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.474+0000 7f7d45f4e640 1 Processor -- start 2024-05-08T05:50:45.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.474+0000 7f7d45f4e640 1 -- start start 2024-05-08T05:50:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.475+0000 7f7d45f4e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d401638e0 0x7f7d400b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.475+0000 7f7d45f4e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d4006c3a0 0x7f7d4006c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.475+0000 7f7d45f4e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d400b14e0 con 0x7f7d4015f0c0 2024-05-08T05:50:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.475+0000 7f7d45f4e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f7d40163450 con 0x7f7d401638e0 2024-05-08T05:50:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.475+0000 7f7d45f4e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7d401604b0 con 0x7f7d4006c3a0 2024-05-08T05:50:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.475+0000 7f7d3effd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d401638e0 0x7f7d400b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.475+0000 7f7d3effd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f7d401638e0 0x7f7d400b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50592/0 (socket says 172.21.15.45:50592) 2024-05-08T05:50:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.475+0000 7f7d3effd640 1 -- 172.21.15.45:0/1690679880 learned_addr learned my addr 172.21.15.45:0/1690679880 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.476+0000 7f7d3ffff640 1 --2- 172.21.15.45:0/1690679880 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d4006c3a0 0x7f7d4006c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.476+0000 7f7d3effd640 1 -- 172.21.15.45:0/1690679880 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d4006c3a0 msgr2=0x7f7d4006c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.476+0000 7f7d3effd640 1 --2- 172.21.15.45:0/1690679880 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d4006c3a0 0x7f7d4006c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.476+0000 7f7d3effd640 1 -- 172.21.15.45:0/1690679880 >> v1:172.21.15.63:6789/0 conn(0x7f7d4015f0c0 legacy=0x7f7d400b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.476+0000 7f7d3effd640 1 -- 172.21.15.45:0/1690679880 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d40162e00 con 0x7f7d401638e0 2024-05-08T05:50:45.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.476+0000 7f7d3effd640 1 --2- 172.21.15.45:0/1690679880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d401638e0 0x7f7d400b6f40 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009870 tx=0x7f7d3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401fd674d3f2e38c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:45.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.476+0000 7f7d3e7fc640 1 -- 172.21.15.45:0/1690679880 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d34037070 con 0x7f7d401638e0 2024-05-08T05:50:45.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.477+0000 7f7d3e7fc640 1 -- 172.21.15.45:0/1690679880 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d34032440 con 0x7f7d401638e0 2024-05-08T05:50:45.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.477+0000 7f7d3e7fc640 1 -- 172.21.15.45:0/1690679880 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d34032810 con 0x7f7d401638e0 2024-05-08T05:50:45.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.477+0000 7f7d45f4e640 1 -- 172.21.15.45:0/1690679880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d401638e0 msgr2=0x7f7d400b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.477+0000 7f7d45f4e640 1 --2- 172.21.15.45:0/1690679880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d401638e0 0x7f7d400b6f40 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009870 tx=0x7f7d3402a270 comp rx=0 tx=0).stop 2024-05-08T05:50:45.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.477+0000 7f7d45f4e640 1 -- 172.21.15.45:0/1690679880 shutdown_connections 2024-05-08T05:50:45.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.477+0000 7f7d45f4e640 1 --2- 172.21.15.45:0/1690679880 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d4006c3a0 0x7f7d4006c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.477+0000 7f7d45f4e640 1 --2- 172.21.15.45:0/1690679880 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d401638e0 0x7f7d400b6f40 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.477+0000 7f7d45f4e640 1 -- 172.21.15.45:0/1690679880 >> 172.21.15.45:0/1690679880 conn(0x7f7d400b3b10 msgr2=0x7f7d400b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:45.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.477+0000 7f7d45f4e640 1 -- 172.21.15.45:0/1690679880 shutdown_connections 2024-05-08T05:50:45.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.477+0000 7f7d45f4e640 1 -- 172.21.15.45:0/1690679880 wait complete. 2024-05-08T05:50:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.478+0000 7f7d45f4e640 1 Processor -- start 2024-05-08T05:50:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.479+0000 7f7d45f4e640 1 -- start start 2024-05-08T05:50:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.479+0000 7f7d45f4e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d4006c3a0 0x7f7d40159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.479+0000 7f7d45f4e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d4015f0c0 0x7f7d400af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.479+0000 7f7d45f4e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d401638e0 0x7f7d400d0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.479+0000 7f7d45f4e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4015f4a0 con 0x7f7d4006c3a0 2024-05-08T05:50:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.479+0000 7f7d45f4e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7d400b17c0 con 0x7f7d4015f0c0 2024-05-08T05:50:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.479+0000 7f7d45f4e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7d4015f620 con 0x7f7d401638e0 2024-05-08T05:50:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.479+0000 7f7d3f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d4006c3a0 0x7f7d40159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.479+0000 7f7d3f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d4006c3a0 0x7f7d40159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50594/0 (socket says 172.21.15.45:50594) 2024-05-08T05:50:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.479+0000 7f7d3f7fe640 1 -- 172.21.15.45:0/1196570256 learned_addr learned my addr 172.21.15.45:0/1196570256 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.480+0000 7f7d3effd640 1 --2- 172.21.15.45:0/1196570256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d4015f0c0 0x7f7d400af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.480+0000 7f7d3ffff640 1 --2- 172.21.15.45:0/1196570256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d401638e0 0x7f7d400d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.480+0000 7f7d3f7fe640 1 -- 172.21.15.45:0/1196570256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d401638e0 msgr2=0x7f7d400d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.480+0000 7f7d3f7fe640 1 --2- 172.21.15.45:0/1196570256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d401638e0 0x7f7d400d0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.480+0000 7f7d3f7fe640 1 -- 172.21.15.45:0/1196570256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d4015f0c0 msgr2=0x7f7d400af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.480+0000 7f7d3f7fe640 1 --2- 172.21.15.45:0/1196570256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d4015f0c0 0x7f7d400af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.480+0000 7f7d3f7fe640 1 -- 172.21.15.45:0/1196570256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4015a2e0 con 0x7f7d4006c3a0 2024-05-08T05:50:45.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.480+0000 7f7d3f7fe640 1 --2- 172.21.15.45:0/1196570256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d4006c3a0 0x7f7d40159da0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000c420 tx=0x7f7d3000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:45.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.480+0000 7f7d27fff640 1 -- 172.21.15.45:0/1196570256 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d300052d0 con 0x7f7d4006c3a0 2024-05-08T05:50:45.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.480+0000 7f7d27fff640 1 -- 172.21.15.45:0/1196570256 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d30007020 con 0x7f7d4006c3a0 2024-05-08T05:50:45.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.480+0000 7f7d45f4e640 1 -- 172.21.15.45:0/1196570256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d400b0050 con 0x7f7d4006c3a0 2024-05-08T05:50:45.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.480+0000 7f7d27fff640 1 -- 172.21.15.45:0/1196570256 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d300054b0 con 0x7f7d4006c3a0 2024-05-08T05:50:45.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.480+0000 7f7d45f4e640 1 -- 172.21.15.45:0/1196570256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d400b0610 con 0x7f7d4006c3a0 2024-05-08T05:50:45.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.481+0000 7f7d45f4e640 1 -- 172.21.15.45:0/1196570256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d400693f0 con 0x7f7d4006c3a0 2024-05-08T05:50:45.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.482+0000 7f7d27fff640 1 -- 172.21.15.45:0/1196570256 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7d30007410 con 0x7f7d4006c3a0 2024-05-08T05:50:45.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.482+0000 7f7d27fff640 1 --2- 172.21.15.45:0/1196570256 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7d2c0762d0 0x7f7d2c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.482+0000 7f7d27fff640 1 -- 172.21.15.45:0/1196570256 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6360+0+0 (secure 0 0 0) 0x7f7d30096d40 con 0x7f7d4006c3a0 2024-05-08T05:50:45.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.482+0000 7f7d3effd640 1 --2- 172.21.15.45:0/1196570256 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7d2c0762d0 0x7f7d2c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.483+0000 7f7d3effd640 1 --2- 172.21.15.45:0/1196570256 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7d2c0762d0 0x7f7d2c078790 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7d34034000 tx=0x7f7d3402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.487+0000 7f7d27fff640 1 -- 172.21.15.45:0/1196570256 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7d30060420 con 0x7f7d4006c3a0 2024-05-08T05:50:45.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.717+0000 7f7d45f4e640 1 -- 172.21.15.45:0/1196570256 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d4006cf30 con 0x7f7d4006c3a0 2024-05-08T05:50:45.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.718+0000 7f7d27fff640 1 -- 172.21.15.45:0/1196570256 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5170 (secure 0 0 0) 0x7f7d3005fd20 con 0x7f7d4006c3a0 2024-05-08T05:50:45.718 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:50:45.718 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3692881261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3692881261},{"type":"v1","addr":"172.21.15.63:6836","nonce":3692881261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7497,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2916399818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2916399818},{"type":"v1","addr":"172.21.15.45:6833","nonce":2916399818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7515,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2978250004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":2978250004},{"type":"v1","addr":"172.21.15.63:6837","nonce":2978250004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7962,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4201508950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4201508950},{"type":"v1","addr":"172.21.15.45:6835","nonce":4201508950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:49:19.587326+0000","modified":"2024-05-08T05:50:43.627486+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-05-08T05:50:45.719 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 90 2024-05-08T05:50:45.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.720+0000 7f7d45f4e640 1 -- 172.21.15.45:0/1196570256 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7d2c0762d0 msgr2=0x7f7d2c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.720+0000 7f7d45f4e640 1 --2- 172.21.15.45:0/1196570256 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7d2c0762d0 0x7f7d2c078790 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7d34034000 tx=0x7f7d3402e000 comp rx=0 tx=0).stop 2024-05-08T05:50:45.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.720+0000 7f7d45f4e640 1 -- 172.21.15.45:0/1196570256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d4006c3a0 msgr2=0x7f7d40159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.720+0000 7f7d45f4e640 1 --2- 172.21.15.45:0/1196570256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d4006c3a0 0x7f7d40159da0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000c420 tx=0x7f7d3000c8e0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.720+0000 7f7d45f4e640 1 -- 172.21.15.45:0/1196570256 shutdown_connections 2024-05-08T05:50:45.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.720+0000 7f7d45f4e640 1 --2- 172.21.15.45:0/1196570256 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7d2c0762d0 0x7f7d2c078790 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.720+0000 7f7d45f4e640 1 --2- 172.21.15.45:0/1196570256 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7d401638e0 0x7f7d400d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.720+0000 7f7d45f4e640 1 --2- 172.21.15.45:0/1196570256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d4015f0c0 0x7f7d400af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.720+0000 7f7d45f4e640 1 --2- 172.21.15.45:0/1196570256 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d4006c3a0 0x7f7d40159da0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.720+0000 7f7d45f4e640 1 -- 172.21.15.45:0/1196570256 >> 172.21.15.45:0/1196570256 conn(0x7f7d400b3b10 msgr2=0x7f7d400b4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:45.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.720+0000 7f7d45f4e640 1 -- 172.21.15.45:0/1196570256 shutdown_connections 2024-05-08T05:50:45.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.720+0000 7f7d45f4e640 1 -- 172.21.15.45:0/1196570256 wait complete. 2024-05-08T05:50:45.731 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:50:45.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.835+0000 7f16ab4da640 1 Processor -- start 2024-05-08T05:50:45.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.835+0000 7f16ab4da640 1 -- start start 2024-05-08T05:50:45.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.836+0000 7f16ab4da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16a406df40 0x7f16a406d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.836+0000 7f16ab4da640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16a415f830 0x7f16a40d3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.836+0000 7f16ab4da640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16a40d0cb0 con 0x7f16a415f450 2024-05-08T05:50:45.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.836+0000 7f16ab4da640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f16a406d690 con 0x7f16a415f830 2024-05-08T05:50:45.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.836+0000 7f16ab4da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f16a40d0f90 con 0x7f16a406df40 2024-05-08T05:50:45.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.836+0000 7f16a9a77640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16a415f830 0x7f16a40d3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.837+0000 7f16a9a77640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f16a415f830 0x7f16a40d3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50608/0 (socket says 172.21.15.45:50608) 2024-05-08T05:50:45.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.837+0000 7f16a9a77640 1 -- 172.21.15.45:0/2169356383 learned_addr learned my addr 172.21.15.45:0/2169356383 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:45.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.837+0000 7f16a8a75640 1 --2- 172.21.15.45:0/2169356383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16a406df40 0x7f16a406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.837+0000 7f16a9a77640 1 -- 172.21.15.45:0/2169356383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16a406df40 msgr2=0x7f16a406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:45.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.837+0000 7f16a9a77640 1 --2- 172.21.15.45:0/2169356383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16a406df40 0x7f16a406d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.837+0000 7f16a9a77640 1 -- 172.21.15.45:0/2169356383 >> v1:172.21.15.63:6789/0 conn(0x7f16a415f450 legacy=0x7f16a406d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.837+0000 7f16a9a77640 1 -- 172.21.15.45:0/2169356383 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a40d2fe0 con 0x7f16a415f830 2024-05-08T05:50:45.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.838+0000 7f16a9a77640 1 --2- 172.21.15.45:0/2169356383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a415f830 0x7f16a40d3ac0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f169c009e50 tx=0x7f169c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b2d327ddc19a2b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:45.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.838+0000 7f1697fff640 1 -- 172.21.15.45:0/2169356383 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c036070 con 0x7f16a415f830 2024-05-08T05:50:45.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.838+0000 7f1697fff640 1 -- 172.21.15.45:0/2169356383 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f169c031600 con 0x7f16a415f830 2024-05-08T05:50:45.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.838+0000 7f1697fff640 1 -- 172.21.15.45:0/2169356383 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c0319d0 con 0x7f16a415f830 2024-05-08T05:50:45.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.838+0000 7f16ab4da640 1 -- 172.21.15.45:0/2169356383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a415f830 msgr2=0x7f16a40d3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.838+0000 7f16ab4da640 1 --2- 172.21.15.45:0/2169356383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a415f830 0x7f16a40d3ac0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f169c009e50 tx=0x7f169c02a710 comp rx=0 tx=0).stop 2024-05-08T05:50:45.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.838+0000 7f16ab4da640 1 -- 172.21.15.45:0/2169356383 shutdown_connections 2024-05-08T05:50:45.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.838+0000 7f16ab4da640 1 --2- 172.21.15.45:0/2169356383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a415f830 0x7f16a40d3ac0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.838+0000 7f16ab4da640 1 --2- 172.21.15.45:0/2169356383 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16a406df40 0x7f16a406d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.838+0000 7f16ab4da640 1 -- 172.21.15.45:0/2169356383 >> 172.21.15.45:0/2169356383 conn(0x7f16a40d5040 msgr2=0x7f16a40d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:45.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.838+0000 7f16ab4da640 1 -- 172.21.15.45:0/2169356383 shutdown_connections 2024-05-08T05:50:45.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.838+0000 7f16ab4da640 1 -- 172.21.15.45:0/2169356383 wait complete. 2024-05-08T05:50:45.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.840+0000 7f16ab4da640 1 Processor -- start 2024-05-08T05:50:45.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.840+0000 7f16ab4da640 1 -- start start 2024-05-08T05:50:45.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.840+0000 7f16ab4da640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16a406df40 0x7f16a40b7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.840+0000 7f16ab4da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16a415f450 0x7f16a4150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.840+0000 7f16ab4da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a415f830 0x7f16a40b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.840+0000 7f16ab4da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f16a40d0880 con 0x7f16a415f830 2024-05-08T05:50:45.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.840+0000 7f16ab4da640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f16a40d0700 con 0x7f16a406df40 2024-05-08T05:50:45.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.840+0000 7f16ab4da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f16a40d0a00 con 0x7f16a415f450 2024-05-08T05:50:45.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.841+0000 7f16a9a77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a415f830 0x7f16a40b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.841+0000 7f16a9a77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a415f830 0x7f16a40b3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50620/0 (socket says 172.21.15.45:50620) 2024-05-08T05:50:45.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.841+0000 7f16a9a77640 1 -- 172.21.15.45:0/952043174 learned_addr learned my addr 172.21.15.45:0/952043174 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.841+0000 7f16a9276640 1 --2- 172.21.15.45:0/952043174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16a406df40 0x7f16a40b7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.841+0000 7f16a8a75640 1 --2- 172.21.15.45:0/952043174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16a415f450 0x7f16a4150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.841+0000 7f16a9a77640 1 -- 172.21.15.45:0/952043174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16a415f450 msgr2=0x7f16a4150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.841+0000 7f16a9a77640 1 --2- 172.21.15.45:0/952043174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16a415f450 0x7f16a4150260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.841+0000 7f16a9a77640 1 -- 172.21.15.45:0/952043174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16a406df40 msgr2=0x7f16a40b7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.841+0000 7f16a9a77640 1 --2- 172.21.15.45:0/952043174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16a406df40 0x7f16a40b7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.841+0000 7f16a9a77640 1 -- 172.21.15.45:0/952043174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a40b81a0 con 0x7f16a415f830 2024-05-08T05:50:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.841+0000 7f16a9a77640 1 --2- 172.21.15.45:0/952043174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a415f830 0x7f16a40b3620 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f169c034040 tx=0x7f169c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.841+0000 7f1695ffb640 1 -- 172.21.15.45:0/952043174 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c036050 con 0x7f16a415f830 2024-05-08T05:50:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.841+0000 7f1695ffb640 1 -- 172.21.15.45:0/952043174 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f169c042070 con 0x7f16a415f830 2024-05-08T05:50:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.841+0000 7f16ab4da640 1 -- 172.21.15.45:0/952043174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a40ca650 con 0x7f16a415f830 2024-05-08T05:50:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.841+0000 7f1695ffb640 1 -- 172.21.15.45:0/952043174 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c009510 con 0x7f16a415f830 2024-05-08T05:50:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.842+0000 7f16ab4da640 1 -- 172.21.15.45:0/952043174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a40cac10 con 0x7f16a415f830 2024-05-08T05:50:45.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.842+0000 7f16ab4da640 1 -- 172.21.15.45:0/952043174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a40cb3e0 con 0x7f16a415f830 2024-05-08T05:50:45.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.843+0000 7f1695ffb640 1 -- 172.21.15.45:0/952043174 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f169c007c70 con 0x7f16a415f830 2024-05-08T05:50:45.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.844+0000 7f1695ffb640 1 --2- 172.21.15.45:0/952043174 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f16880762d0 0x7f1688078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.844+0000 7f1695ffb640 1 -- 172.21.15.45:0/952043174 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6360+0+0 (secure 0 0 0) 0x7f169c0b86d0 con 0x7f16a415f830 2024-05-08T05:50:45.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.844+0000 7f16a9276640 1 --2- 172.21.15.45:0/952043174 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f16880762d0 0x7f1688078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.844+0000 7f16a9276640 1 --2- 172.21.15.45:0/952043174 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f16880762d0 0x7f1688078790 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f16a00015e0 tx=0x7f16a0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:45.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:45.848+0000 7f1695ffb640 1 -- 172.21.15.45:0/952043174 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f169c081d90 con 0x7f16a415f830 2024-05-08T05:50:46.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.041+0000 7f16ab4da640 1 -- 172.21.15.45:0/952043174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16a40693f0 con 0x7f16a415f830 2024-05-08T05:50:46.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.042+0000 7f1695ffb640 1 -- 172.21.15.45:0/952043174 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+17239 (secure 0 0 0) 0x7f169c081690 con 0x7f16a415f830 2024-05-08T05:50:46.043 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:50:46.043 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":108,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:50:43.627313+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:49:17.241406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-05-08T05:49:18.542707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.21","mappings":[{"from":6,"to":7}]},{"pgid":"16.25","mappings":[{"from":5,"to":4}]},{"pgid":"16.39","mappings":[{"from":1,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/2729952281":"2024-05-09T05:50:43.627290+0000","172.21.15.45:6834/2729952281":"2024-05-09T05:50:43.627290+0000","172.21.15.45:6835/878092981":"2024-05-09T05:50:07.216878+0000","172.21.15.45:6834/878092981":"2024-05-09T05:50:07.216878+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:50:46.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.044+0000 7f16ab4da640 1 -- 172.21.15.45:0/952043174 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f16880762d0 msgr2=0x7f1688078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:46.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.044+0000 7f16ab4da640 1 --2- 172.21.15.45:0/952043174 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f16880762d0 0x7f1688078790 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f16a00015e0 tx=0x7f16a0009040 comp rx=0 tx=0).stop 2024-05-08T05:50:46.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.044+0000 7f16ab4da640 1 -- 172.21.15.45:0/952043174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a415f830 msgr2=0x7f16a40b3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:46.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.044+0000 7f16ab4da640 1 --2- 172.21.15.45:0/952043174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a415f830 0x7f16a40b3620 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f169c034040 tx=0x7f169c004a40 comp rx=0 tx=0).stop 2024-05-08T05:50:46.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.044+0000 7f16ab4da640 1 -- 172.21.15.45:0/952043174 shutdown_connections 2024-05-08T05:50:46.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.044+0000 7f16ab4da640 1 --2- 172.21.15.45:0/952043174 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f16880762d0 0x7f1688078790 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:46.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.044+0000 7f16ab4da640 1 --2- 172.21.15.45:0/952043174 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f16a415f450 0x7f16a4150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:46.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.044+0000 7f16ab4da640 1 --2- 172.21.15.45:0/952043174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f16a415f830 0x7f16a40b3620 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:46.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.044+0000 7f16ab4da640 1 --2- 172.21.15.45:0/952043174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f16a406df40 0x7f16a40b7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:46.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.044+0000 7f16ab4da640 1 -- 172.21.15.45:0/952043174 >> 172.21.15.45:0/952043174 conn(0x7f16a40d5040 msgr2=0x7f16a40b5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:46.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.045+0000 7f16ab4da640 1 -- 172.21.15.45:0/952043174 shutdown_connections 2024-05-08T05:50:46.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.045+0000 7f16ab4da640 1 -- 172.21.15.45:0/952043174 wait complete. 2024-05-08T05:50:46.056 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T05:50:46.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.162+0000 7f5f30d24640 1 Processor -- start 2024-05-08T05:50:46.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.162+0000 7f5f30d24640 1 -- start start 2024-05-08T05:50:46.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.162+0000 7f5f30d24640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5f2c0d0ec0 0x7f5f2c0d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:46.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.162+0000 7f5f30d24640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5f2c0d3b00 0x7f5f2c0b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:46.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.162+0000 7f5f30d24640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f2c0d1760 con 0x7f5f2c15f380 2024-05-08T05:50:46.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.162+0000 7f5f30d24640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f5f2c0d30f0 con 0x7f5f2c0d3b00 2024-05-08T05:50:46.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.162+0000 7f5f30d24640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5f2c0d2370 con 0x7f5f2c0d0ec0 2024-05-08T05:50:46.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.163+0000 7f5f2ad9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f5f2c0d3b00 0x7f5f2c0b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:46.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.163+0000 7f5f2a59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5f2c15f380 0x7f5f2c0d07b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60450/0 (socket says 172.21.15.45:60450) 2024-05-08T05:50:46.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.163+0000 7f5f2a59c640 1 -- 172.21.15.45:0/2313160346 learned_addr learned my addr 172.21.15.45:0/2313160346 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:50:46.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.163+0000 7f5f23fff640 1 --2- 172.21.15.45:0/2313160346 >> v2:172.21.15.63:3300/0 conn(0x7f5f2c0d0ec0 0x7f5f2c0d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:46.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.163+0000 7f5f2ad9d640 1 -- 172.21.15.45:0/2313160346 >> v2:172.21.15.63:3300/0 conn(0x7f5f2c0d0ec0 msgr2=0x7f5f2c0d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:46.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.163+0000 7f5f2ad9d640 1 --2- 172.21.15.45:0/2313160346 >> v2:172.21.15.63:3300/0 conn(0x7f5f2c0d0ec0 0x7f5f2c0d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:46.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.163+0000 7f5f2ad9d640 1 -- 172.21.15.45:0/2313160346 >> v1:172.21.15.45:6789/0 conn(0x7f5f2c15f380 legacy=0x7f5f2c0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:46.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.163+0000 7f5f2ad9d640 1 -- 172.21.15.45:0/2313160346 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2c0d1a40 con 0x7f5f2c0d3b00 2024-05-08T05:50:46.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.163+0000 7f5f29d9b640 1 -- 172.21.15.45:0/2313160346 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2407233201 0 0) 0x7f5f2c0d1760 con 0x7f5f2c15f380 2024-05-08T05:50:46.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.164+0000 7f5f2ad9d640 1 --2- 172.21.15.45:0/2313160346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f2c0d3b00 0x7f5f2c0b7e30 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0b7b00 tx=0x7f5f2402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45233697811e9bcf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:46.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.164+0000 7f5f29d9b640 1 -- 172.21.15.45:0/2313160346 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24007740 con 0x7f5f2c0d3b00 2024-05-08T05:50:46.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.164+0000 7f5f29d9b640 1 -- 172.21.15.45:0/2313160346 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f240078e0 con 0x7f5f2c0d3b00 2024-05-08T05:50:46.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.164+0000 7f5f29d9b640 1 -- 172.21.15.45:0/2313160346 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24007cb0 con 0x7f5f2c0d3b00 2024-05-08T05:50:46.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.164+0000 7f5f30d24640 1 -- 172.21.15.45:0/2313160346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f2c0d3b00 msgr2=0x7f5f2c0b7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:46.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.164+0000 7f5f30d24640 1 --2- 172.21.15.45:0/2313160346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f2c0d3b00 0x7f5f2c0b7e30 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0b7b00 tx=0x7f5f2402a680 comp rx=0 tx=0).stop 2024-05-08T05:50:46.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.164+0000 7f5f30d24640 1 -- 172.21.15.45:0/2313160346 shutdown_connections 2024-05-08T05:50:46.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.164+0000 7f5f30d24640 1 --2- 172.21.15.45:0/2313160346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f2c0d3b00 0x7f5f2c0b7e30 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:46.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.164+0000 7f5f30d24640 1 --2- 172.21.15.45:0/2313160346 >> v2:172.21.15.63:3300/0 conn(0x7f5f2c0d0ec0 0x7f5f2c0d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:46.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.164+0000 7f5f30d24640 1 -- 172.21.15.45:0/2313160346 >> 172.21.15.45:0/2313160346 conn(0x7f5f2c0d4fe0 msgr2=0x7f5f2c0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:46.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.164+0000 7f5f30d24640 1 -- 172.21.15.45:0/2313160346 shutdown_connections 2024-05-08T05:50:46.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.164+0000 7f5f30d24640 1 -- 172.21.15.45:0/2313160346 wait complete. 2024-05-08T05:50:46.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.166+0000 7f5f30d24640 1 Processor -- start 2024-05-08T05:50:46.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.166+0000 7f5f30d24640 1 -- start start 2024-05-08T05:50:46.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f30d24640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f2c0d0ec0 0x7f5f2c0cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:46.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f30d24640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f2c0d3b00 0x7f5f2c15cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:46.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f30d24640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f2c15f380 0x7f5f2c150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:46.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f30d24640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c0d2000 con 0x7f5f2c15f380 2024-05-08T05:50:46.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f30d24640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c0d3f30 con 0x7f5f2c0d0ec0 2024-05-08T05:50:46.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f30d24640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5f2c15f760 con 0x7f5f2c0d3b00 2024-05-08T05:50:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f2ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f2c15f380 0x7f5f2c150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f2ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f2c15f380 0x7f5f2c150330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50646/0 (socket says 172.21.15.45:50646) 2024-05-08T05:50:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f2ad9d640 1 -- 172.21.15.45:0/2135613673 learned_addr learned my addr 172.21.15.45:0/2135613673 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f2a59c640 1 --2- 172.21.15.45:0/2135613673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f2c0d0ec0 0x7f5f2c0cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f23fff640 1 --2- 172.21.15.45:0/2135613673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f2c0d3b00 0x7f5f2c15cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f2ad9d640 1 -- 172.21.15.45:0/2135613673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f2c0d3b00 msgr2=0x7f5f2c15cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f2ad9d640 1 --2- 172.21.15.45:0/2135613673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f2c0d3b00 0x7f5f2c15cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f2ad9d640 1 -- 172.21.15.45:0/2135613673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f2c0d0ec0 msgr2=0x7f5f2c0cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f2ad9d640 1 --2- 172.21.15.45:0/2135613673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f2c0d0ec0 0x7f5f2c0cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f2ad9d640 1 -- 172.21.15.45:0/2135613673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2c0cc770 con 0x7f5f2c15f380 2024-05-08T05:50:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f2ad9d640 1 --2- 172.21.15.45:0/2135613673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f2c15f380 0x7f5f2c150330 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f5f2402ac70 tx=0x7f5f24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f22ffd640 1 -- 172.21.15.45:0/2135613673 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24040070 con 0x7f5f2c15f380 2024-05-08T05:50:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f22ffd640 1 -- 172.21.15.45:0/2135613673 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f24031c40 con 0x7f5f2c15f380 2024-05-08T05:50:46.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f30d24640 1 -- 172.21.15.45:0/2135613673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2c0cca50 con 0x7f5f2c15f380 2024-05-08T05:50:46.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f22ffd640 1 -- 172.21.15.45:0/2135613673 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24009450 con 0x7f5f2c15f380 2024-05-08T05:50:46.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.167+0000 7f5f30d24640 1 -- 172.21.15.45:0/2135613673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f2c0cd010 con 0x7f5f2c15f380 2024-05-08T05:50:46.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.169+0000 7f5f30d24640 1 -- 172.21.15.45:0/2135613673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f2c0693f0 con 0x7f5f2c15f380 2024-05-08T05:50:46.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.169+0000 7f5f22ffd640 1 -- 172.21.15.45:0/2135613673 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5f24039040 con 0x7f5f2c15f380 2024-05-08T05:50:46.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.169+0000 7f5f22ffd640 1 --2- 172.21.15.45:0/2135613673 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5f0c0762d0 0x7f5f0c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:46.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.170+0000 7f5f22ffd640 1 -- 172.21.15.45:0/2135613673 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6360+0+0 (secure 0 0 0) 0x7f5f240b8a50 con 0x7f5f2c15f380 2024-05-08T05:50:46.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.170+0000 7f5f2a59c640 1 --2- 172.21.15.45:0/2135613673 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5f0c0762d0 0x7f5f0c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:46.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.170+0000 7f5f2a59c640 1 --2- 172.21.15.45:0/2135613673 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5f0c0762d0 0x7f5f0c078790 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c000f60 tx=0x7f5f1c007cf0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:46.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.174+0000 7f5f22ffd640 1 -- 172.21.15.45:0/2135613673 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5f24081ad0 con 0x7f5f2c15f380 2024-05-08T05:50:46.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.408+0000 7f5f30d24640 1 -- 172.21.15.45:0/2135613673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5f2c0b7980 con 0x7f5f2c15f380 2024-05-08T05:50:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.641+0000 7f5f22ffd640 1 -- 172.21.15.45:0/2135613673 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) ==== 161+0+0 (secure 0 0 0) 0x7f5f24085dd0 con 0x7f5f2c15f380 2024-05-08T05:50:46.641 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T05:50:46.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.643+0000 7f5f30d24640 1 -- 172.21.15.45:0/2135613673 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5f0c0762d0 msgr2=0x7f5f0c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.643+0000 7f5f30d24640 1 --2- 172.21.15.45:0/2135613673 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5f0c0762d0 0x7f5f0c078790 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c000f60 tx=0x7f5f1c007cf0 comp rx=0 tx=0).stop 2024-05-08T05:50:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.643+0000 7f5f30d24640 1 -- 172.21.15.45:0/2135613673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f2c15f380 msgr2=0x7f5f2c150330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.643+0000 7f5f30d24640 1 --2- 172.21.15.45:0/2135613673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f2c15f380 0x7f5f2c150330 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f5f2402ac70 tx=0x7f5f24004a40 comp rx=0 tx=0).stop 2024-05-08T05:50:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.643+0000 7f5f30d24640 1 -- 172.21.15.45:0/2135613673 shutdown_connections 2024-05-08T05:50:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.643+0000 7f5f30d24640 1 --2- 172.21.15.45:0/2135613673 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5f0c0762d0 0x7f5f0c078790 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.643+0000 7f5f30d24640 1 --2- 172.21.15.45:0/2135613673 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5f2c0d3b00 0x7f5f2c15cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.643+0000 7f5f30d24640 1 --2- 172.21.15.45:0/2135613673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f2c15f380 0x7f5f2c150330 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.643+0000 7f5f30d24640 1 --2- 172.21.15.45:0/2135613673 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f2c0d0ec0 0x7f5f2c0cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.643+0000 7f5f30d24640 1 -- 172.21.15.45:0/2135613673 >> 172.21.15.45:0/2135613673 conn(0x7f5f2c0d4fe0 msgr2=0x7f5f2c0cab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.643+0000 7f5f30d24640 1 -- 172.21.15.45:0/2135613673 shutdown_connections 2024-05-08T05:50:46.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.643+0000 7f5f30d24640 1 -- 172.21.15.45:0/2135613673 wait complete. 2024-05-08T05:50:46.655 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T05:50:46.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.762+0000 7f019cc75640 1 Processor -- start 2024-05-08T05:50:46.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.762+0000 7f019cc75640 1 -- start start 2024-05-08T05:50:46.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.763+0000 7f019cc75640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f01980d1970 0x7f01980d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:46.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.763+0000 7f019cc75640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01981630b0 con 0x7f019815f420 2024-05-08T05:50:46.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.763+0000 7f019cc75640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01980d3e70 con 0x7f01980d4cc0 2024-05-08T05:50:46.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.763+0000 7f019cc75640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f01980d2550 con 0x7f01980d1970 2024-05-08T05:50:46.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.763+0000 7f019659c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f019815f420 0x7f01980d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60462/0 (socket says 172.21.15.45:60462) 2024-05-08T05:50:46.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.763+0000 7f019659c640 1 -- 172.21.15.45:0/853817687 learned_addr learned my addr 172.21.15.45:0/853817687 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:50:46.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f019559a640 1 -- 172.21.15.45:0/853817687 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2171033138 0 0) 0x7f0188001110 con 0x7f019815f420 2024-05-08T05:50:46.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f019559a640 1 -- 172.21.15.45:0/853817687 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0180003520 con 0x7f019815f420 2024-05-08T05:50:46.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f0195d9b640 1 --2- 172.21.15.45:0/853817687 >> v2:172.21.15.63:3300/0 conn(0x7f01980d1970 0x7f01980d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:46.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f019559a640 1 -- 172.21.15.45:0/853817687 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1890774905 0 0) 0x7f0180003520 con 0x7f019815f420 2024-05-08T05:50:46.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f019559a640 1 -- 172.21.15.45:0/853817687 >> v2:172.21.15.63:3300/0 conn(0x7f01980d1970 msgr2=0x7f01980d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:46.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f019559a640 1 --2- 172.21.15.45:0/853817687 >> v2:172.21.15.63:3300/0 conn(0x7f01980d1970 0x7f01980d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:46.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f019559a640 1 -- 172.21.15.45:0/853817687 >> v1:172.21.15.63:6789/0 conn(0x7f01980d4cc0 legacy=0x7f01980d50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:46.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f019559a640 1 -- 172.21.15.45:0/853817687 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01980d2c80 con 0x7f019815f420 2024-05-08T05:50:46.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f019559a640 1 -- 172.21.15.45:0/853817687 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f01880018c0 con 0x7f019815f420 2024-05-08T05:50:46.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f0195d9b640 1 --2- 172.21.15.45:0/853817687 >> v2:172.21.15.63:3300/0 conn(0x7f01980d1970 0x7f01980d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:46.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f019559a640 1 -- 172.21.15.45:0/853817687 >> v1:172.21.15.45:6789/0 conn(0x7f019815f420 legacy=0x7f01980d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:46.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f019559a640 1 --2- 172.21.15.45:0/853817687 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0180003a50 0x7f0180003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:46.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f019559a640 1 --2- 172.21.15.45:0/853817687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0180004800 0x7f0180004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:46.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f019559a640 1 --2- 172.21.15.45:0/853817687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01800052e0 0x7f018000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:46.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f019559a640 1 -- 172.21.15.45:0/853817687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f01980d2550 con 0x7f01800052e0 2024-05-08T05:50:46.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f019559a640 1 -- 172.21.15.45:0/853817687 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f018000c1b0 con 0x7f0180004800 2024-05-08T05:50:46.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f019559a640 1 -- 172.21.15.45:0/853817687 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f018000c3b0 con 0x7f0180003a50 2024-05-08T05:50:46.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f0196d9d640 1 --2- 172.21.15.45:0/853817687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01800052e0 0x7f018000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:46.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f0195d9b640 1 --2- 172.21.15.45:0/853817687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0180004800 0x7f0180004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:46.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f019659c640 1 --2- 172.21.15.45:0/853817687 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0180003a50 0x7f0180003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:46.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f0196d9d640 1 -- 172.21.15.45:0/853817687 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0180003a50 msgr2=0x7f0180003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:46.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f0196d9d640 1 --2- 172.21.15.45:0/853817687 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0180003a50 0x7f0180003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:46.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f0196d9d640 1 -- 172.21.15.45:0/853817687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0180004800 msgr2=0x7f0180004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:46.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f0196d9d640 1 --2- 172.21.15.45:0/853817687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0180004800 0x7f0180004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:46.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.764+0000 7f0196d9d640 1 -- 172.21.15.45:0/853817687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0180003520 con 0x7f01800052e0 2024-05-08T05:50:46.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.765+0000 7f0196d9d640 1 -- 172.21.15.45:0/853817687 reap_dead start 2024-05-08T05:50:46.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.765+0000 7f0196d9d640 1 --2- 172.21.15.45:0/853817687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01800052e0 0x7f018000b9c0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f0190001e60 tx=0x7f0180005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f51973fce101c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:46.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.765+0000 7f019559a640 1 -- 172.21.15.45:0/853817687 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190001ab0 con 0x7f01800052e0 2024-05-08T05:50:46.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.765+0000 7f019559a640 1 -- 172.21.15.45:0/853817687 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f019002ae40 con 0x7f01800052e0 2024-05-08T05:50:46.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.765+0000 7f019559a640 1 -- 172.21.15.45:0/853817687 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190003bb0 con 0x7f01800052e0 2024-05-08T05:50:46.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.766+0000 7f019cc75640 1 -- 172.21.15.45:0/853817687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01800052e0 msgr2=0x7f018000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:46.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.766+0000 7f019cc75640 1 --2- 172.21.15.45:0/853817687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01800052e0 0x7f018000b9c0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f0190001e60 tx=0x7f0180005830 comp rx=0 tx=0).stop 2024-05-08T05:50:46.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.766+0000 7f019cc75640 1 -- 172.21.15.45:0/853817687 shutdown_connections 2024-05-08T05:50:46.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.766+0000 7f019cc75640 1 --2- 172.21.15.45:0/853817687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01800052e0 0x7f018000b9c0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:46.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.766+0000 7f019cc75640 1 -- 172.21.15.45:0/853817687 >> 172.21.15.45:0/853817687 conn(0x7f019808eac0 msgr2=0x7f019809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:46.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.766+0000 7f019cc75640 1 -- 172.21.15.45:0/853817687 shutdown_connections 2024-05-08T05:50:46.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.766+0000 7f019cc75640 1 -- 172.21.15.45:0/853817687 wait complete. 2024-05-08T05:50:46.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.767+0000 7f019cc75640 1 Processor -- start 2024-05-08T05:50:46.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.768+0000 7f019cc75640 1 -- start start 2024-05-08T05:50:46.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.768+0000 7f019cc75640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f01800052e0 0x7f0198159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:46.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.768+0000 7f019cc75640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f019815a380 0x7f01980c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:46.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.768+0000 7f019cc75640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01980c9040 0x7f0198160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:46.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.768+0000 7f019cc75640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f01980d2830 con 0x7f01980c9040 2024-05-08T05:50:46.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.768+0000 7f019cc75640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f01980d3240 con 0x7f019815a380 2024-05-08T05:50:46.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.768+0000 7f019cc75640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f01980d1e20 con 0x7f01800052e0 2024-05-08T05:50:46.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.768+0000 7f0196d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01980c9040 0x7f0198160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:46.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.768+0000 7f0196d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01980c9040 0x7f0198160ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50664/0 (socket says 172.21.15.45:50664) 2024-05-08T05:50:46.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.768+0000 7f0196d9d640 1 -- 172.21.15.45:0/4134764620 learned_addr learned my addr 172.21.15.45:0/4134764620 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:46.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.769+0000 7f019659c640 1 --2- 172.21.15.45:0/4134764620 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f01800052e0 0x7f0198159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:46.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.769+0000 7f0195d9b640 1 --2- 172.21.15.45:0/4134764620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f019815a380 0x7f01980c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:46.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.769+0000 7f0196d9d640 1 -- 172.21.15.45:0/4134764620 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f01800052e0 msgr2=0x7f0198159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:46.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.769+0000 7f0196d9d640 1 --2- 172.21.15.45:0/4134764620 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f01800052e0 0x7f0198159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:46.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.769+0000 7f0196d9d640 1 -- 172.21.15.45:0/4134764620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f019815a380 msgr2=0x7f01980c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:46.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.769+0000 7f0196d9d640 1 --2- 172.21.15.45:0/4134764620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f019815a380 0x7f01980c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:46.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.769+0000 7f0196d9d640 1 -- 172.21.15.45:0/4134764620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019815aaa0 con 0x7f01980c9040 2024-05-08T05:50:46.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.769+0000 7f019659c640 1 --2- 172.21.15.45:0/4134764620 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f01800052e0 0x7f0198159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:46.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.769+0000 7f0196d9d640 1 --2- 172.21.15.45:0/4134764620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01980c9040 0x7f0198160ec0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f0190001370 tx=0x7f01900014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:46.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.769+0000 7f0186ffd640 1 -- 172.21.15.45:0/4134764620 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190007df0 con 0x7f01980c9040 2024-05-08T05:50:46.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.769+0000 7f0186ffd640 1 -- 172.21.15.45:0/4134764620 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f019002ac40 con 0x7f01980c9040 2024-05-08T05:50:46.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.769+0000 7f019cc75640 1 -- 172.21.15.45:0/4134764620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01980c9650 con 0x7f01980c9040 2024-05-08T05:50:46.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.769+0000 7f0186ffd640 1 -- 172.21.15.45:0/4134764620 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01900054e0 con 0x7f01980c9040 2024-05-08T05:50:46.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.769+0000 7f019cc75640 1 -- 172.21.15.45:0/4134764620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01980c9c10 con 0x7f01980c9040 2024-05-08T05:50:46.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.770+0000 7f019cc75640 1 -- 172.21.15.45:0/4134764620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01980c8640 con 0x7f01980c9040 2024-05-08T05:50:46.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.771+0000 7f0186ffd640 1 -- 172.21.15.45:0/4134764620 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0190005680 con 0x7f01980c9040 2024-05-08T05:50:46.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.771+0000 7f0186ffd640 1 --2- 172.21.15.45:0/4134764620 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f01780762d0 0x7f0178078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:46.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.771+0000 7f0186ffd640 1 -- 172.21.15.45:0/4134764620 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6360+0+0 (secure 0 0 0) 0x7f01900b6c70 con 0x7f01980c9040 2024-05-08T05:50:46.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.772+0000 7f019659c640 1 --2- 172.21.15.45:0/4134764620 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f01780762d0 0x7f0178078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:46.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.772+0000 7f019659c640 1 --2- 172.21.15.45:0/4134764620 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f01780762d0 0x7f0178078790 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0188001eb0 tx=0x7f0188009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:46.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:46.776+0000 7f0186ffd640 1 -- 172.21.15.45:0/4134764620 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f01980c8640 con 0x7f01980c9040 2024-05-08T05:50:47.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.006+0000 7f019cc75640 1 -- 172.21.15.45:0/4134764620 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f01980b4c50 con 0x7f01980c9040 2024-05-08T05:50:47.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.647+0000 7f0186ffd640 1 -- 172.21.15.45:0/4134764620 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v92) ==== 104+0+0 (secure 0 0 0) 0x7f0190043060 con 0x7f01980c9040 2024-05-08T05:50:47.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.650+0000 7f019cc75640 1 -- 172.21.15.45:0/4134764620 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f01780762d0 msgr2=0x7f0178078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:47.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.650+0000 7f019cc75640 1 --2- 172.21.15.45:0/4134764620 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f01780762d0 0x7f0178078790 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0188001eb0 tx=0x7f0188009040 comp rx=0 tx=0).stop 2024-05-08T05:50:47.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.650+0000 7f019cc75640 1 -- 172.21.15.45:0/4134764620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01980c9040 msgr2=0x7f0198160ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:47.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.650+0000 7f019cc75640 1 --2- 172.21.15.45:0/4134764620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01980c9040 0x7f0198160ec0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f0190001370 tx=0x7f01900014a0 comp rx=0 tx=0).stop 2024-05-08T05:50:47.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.650+0000 7f019cc75640 1 -- 172.21.15.45:0/4134764620 shutdown_connections 2024-05-08T05:50:47.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.650+0000 7f019cc75640 1 --2- 172.21.15.45:0/4134764620 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f01780762d0 0x7f0178078790 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:47.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.650+0000 7f019cc75640 1 --2- 172.21.15.45:0/4134764620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f01980c9040 0x7f0198160ec0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:47.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.650+0000 7f019cc75640 1 --2- 172.21.15.45:0/4134764620 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f019815a380 0x7f01980c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:47.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.650+0000 7f019cc75640 1 --2- 172.21.15.45:0/4134764620 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f01800052e0 0x7f0198159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:47.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.650+0000 7f019cc75640 1 -- 172.21.15.45:0/4134764620 >> 172.21.15.45:0/4134764620 conn(0x7f019808eac0 msgr2=0x7f0198163290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:47.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.650+0000 7f019cc75640 1 -- 172.21.15.45:0/4134764620 shutdown_connections 2024-05-08T05:50:47.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.650+0000 7f019cc75640 1 -- 172.21.15.45:0/4134764620 wait complete. 2024-05-08T05:50:47.664 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T05:50:47.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.779+0000 7fac21514640 1 Processor -- start 2024-05-08T05:50:47.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.779+0000 7fac21514640 1 -- start start 2024-05-08T05:50:47.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.780+0000 7fac21514640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fac1c0d3670 0x7fac1c0b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:47.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.780+0000 7fac21514640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac1c0d1050 0x7fac1c162920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:47.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.780+0000 7fac21514640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac1c0cfc00 con 0x7fac1c0d3a50 2024-05-08T05:50:47.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.780+0000 7fac21514640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fac1c0d2a10 con 0x7fac1c0d3670 2024-05-08T05:50:47.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.780+0000 7fac21514640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fac1c0d04b0 con 0x7fac1c0d1050 2024-05-08T05:50:47.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.780+0000 7fac1affd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fac1c0d3670 0x7fac1c0b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:47.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.780+0000 7fac1affd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fac1c0d3670 0x7fac1c0b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50672/0 (socket says 172.21.15.45:50672) 2024-05-08T05:50:47.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.780+0000 7fac1a7fc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fac1c0d3a50 0x7fac1c0d0940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60470/0 (socket says 172.21.15.45:60470) 2024-05-08T05:50:47.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.780+0000 7fac1affd640 1 -- 172.21.15.45:0/2935269295 learned_addr learned my addr 172.21.15.45:0/2935269295 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:47.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.780+0000 7fac1affd640 1 -- 172.21.15.45:0/2935269295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac1c0d1050 msgr2=0x7fac1c162920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:47.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.780+0000 7fac1affd640 1 --2- 172.21.15.45:0/2935269295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac1c0d1050 0x7fac1c162920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:47.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.781+0000 7fac1affd640 1 -- 172.21.15.45:0/2935269295 >> v1:172.21.15.45:6789/0 conn(0x7fac1c0d3a50 legacy=0x7fac1c0d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:47.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.781+0000 7fac1affd640 1 -- 172.21.15.45:0/2935269295 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c0cfea0 con 0x7fac1c0d3670 2024-05-08T05:50:47.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.781+0000 7fac1affd640 1 --2- 172.21.15.45:0/2935269295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac1c0d3670 0x7fac1c0b49c0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0d0630 tx=0x7fac0c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e029572e7393755d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:47.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.781+0000 7fac19ffb640 1 -- 172.21.15.45:0/2935269295 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c037070 con 0x7fac1c0d3670 2024-05-08T05:50:47.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.781+0000 7fac19ffb640 1 -- 172.21.15.45:0/2935269295 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac0c032440 con 0x7fac1c0d3670 2024-05-08T05:50:47.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.781+0000 7fac19ffb640 1 -- 172.21.15.45:0/2935269295 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c032810 con 0x7fac1c0d3670 2024-05-08T05:50:47.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.782+0000 7fac21514640 1 -- 172.21.15.45:0/2935269295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac1c0d3670 msgr2=0x7fac1c0b49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:47.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.782+0000 7fac21514640 1 --2- 172.21.15.45:0/2935269295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac1c0d3670 0x7fac1c0b49c0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0d0630 tx=0x7fac0c02a250 comp rx=0 tx=0).stop 2024-05-08T05:50:47.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.782+0000 7fac21514640 1 -- 172.21.15.45:0/2935269295 shutdown_connections 2024-05-08T05:50:47.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.782+0000 7fac21514640 1 --2- 172.21.15.45:0/2935269295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac1c0d1050 0x7fac1c162920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:47.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.782+0000 7fac21514640 1 --2- 172.21.15.45:0/2935269295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac1c0d3670 0x7fac1c0b49c0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:47.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.782+0000 7fac21514640 1 -- 172.21.15.45:0/2935269295 >> 172.21.15.45:0/2935269295 conn(0x7fac1c09e390 msgr2=0x7fac1c0d5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:47.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.782+0000 7fac21514640 1 -- 172.21.15.45:0/2935269295 shutdown_connections 2024-05-08T05:50:47.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.782+0000 7fac21514640 1 -- 172.21.15.45:0/2935269295 wait complete. 2024-05-08T05:50:47.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.783+0000 7fac21514640 1 Processor -- start 2024-05-08T05:50:47.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.783+0000 7fac21514640 1 -- start start 2024-05-08T05:50:47.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.784+0000 7fac21514640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac1c0d1050 0x7fac1c1f85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:47.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.784+0000 7fac21514640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac1c0d3670 0x7fac1c15d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:47.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.784+0000 7fac21514640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac1c0d3a50 0x7fac1c0c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:47.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.784+0000 7fac21514640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c162e60 con 0x7fac1c0d1050 2024-05-08T05:50:47.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.784+0000 7fac21514640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c0d0140 con 0x7fac1c0d3a50 2024-05-08T05:50:47.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.784+0000 7fac21514640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fac1c0b4f00 con 0x7fac1c0d3670 2024-05-08T05:50:47.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.784+0000 7fac1affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac1c0d1050 0x7fac1c1f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:47.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.784+0000 7fac1affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac1c0d1050 0x7fac1c1f85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50686/0 (socket says 172.21.15.45:50686) 2024-05-08T05:50:47.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.784+0000 7fac1affd640 1 -- 172.21.15.45:0/189892925 learned_addr learned my addr 172.21.15.45:0/189892925 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:47.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.785+0000 7fac1a7fc640 1 --2- 172.21.15.45:0/189892925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac1c0d3670 0x7fac1c15d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:47.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.785+0000 7fac1b7fe640 1 --2- 172.21.15.45:0/189892925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac1c0d3a50 0x7fac1c0c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:47.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.785+0000 7fac1affd640 1 -- 172.21.15.45:0/189892925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac1c0d3670 msgr2=0x7fac1c15d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:47.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.785+0000 7fac1affd640 1 --2- 172.21.15.45:0/189892925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac1c0d3670 0x7fac1c15d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:47.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.785+0000 7fac1affd640 1 -- 172.21.15.45:0/189892925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac1c0d3a50 msgr2=0x7fac1c0c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:47.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.785+0000 7fac1affd640 1 --2- 172.21.15.45:0/189892925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac1c0d3a50 0x7fac1c0c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:47.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.785+0000 7fac1affd640 1 -- 172.21.15.45:0/189892925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c1f94d0 con 0x7fac1c0d1050 2024-05-08T05:50:47.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.785+0000 7fac1a7fc640 1 --2- 172.21.15.45:0/189892925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac1c0d3670 0x7fac1c15d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:47.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.785+0000 7fac1b7fe640 1 --2- 172.21.15.45:0/189892925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac1c0d3a50 0x7fac1c0c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:47.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.785+0000 7fac1affd640 1 --2- 172.21.15.45:0/189892925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac1c0d1050 0x7fac1c1f85e0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fac0c02a290 tx=0x7fac0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:47.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.785+0000 7fac137fe640 1 -- 172.21.15.45:0/189892925 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c037040 con 0x7fac1c0d1050 2024-05-08T05:50:47.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.785+0000 7fac137fe640 1 -- 172.21.15.45:0/189892925 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac0c042070 con 0x7fac1c0d1050 2024-05-08T05:50:47.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.785+0000 7fac137fe640 1 -- 172.21.15.45:0/189892925 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c030bb0 con 0x7fac1c0d1050 2024-05-08T05:50:47.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.785+0000 7fac21514640 1 -- 172.21.15.45:0/189892925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1c1f8f10 con 0x7fac1c0d1050 2024-05-08T05:50:47.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.785+0000 7fac21514640 1 -- 172.21.15.45:0/189892925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac1c0cc240 con 0x7fac1c0d1050 2024-05-08T05:50:47.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.786+0000 7fac21514640 1 -- 172.21.15.45:0/189892925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac1c0693f0 con 0x7fac1c0d1050 2024-05-08T05:50:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.787+0000 7fac137fe640 1 -- 172.21.15.45:0/189892925 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fac0c047460 con 0x7fac1c0d1050 2024-05-08T05:50:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.787+0000 7fac137fe640 1 --2- 172.21.15.45:0/189892925 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fabfc0762d0 0x7fabfc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.787+0000 7fac137fe640 1 -- 172.21.15.45:0/189892925 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6360+0+0 (secure 0 0 0) 0x7fac0c0b8510 con 0x7fac1c0d1050 2024-05-08T05:50:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.787+0000 7fac1a7fc640 1 --2- 172.21.15.45:0/189892925 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fabfc0762d0 0x7fabfc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:47.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.788+0000 7fac1a7fc640 1 --2- 172.21.15.45:0/189892925 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fabfc0762d0 0x7fabfc078790 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fac1c1586d0 tx=0x7fac04009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:47.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.792+0000 7fac137fe640 1 -- 172.21.15.45:0/189892925 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fac0c081be0 con 0x7fac1c0d1050 2024-05-08T05:50:47.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:47.983+0000 7fac21514640 1 -- 172.21.15.45:0/189892925 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fac1c0cc660 con 0x7fac1c0d1050 2024-05-08T05:50:48.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.653+0000 7fac137fe640 1 -- 172.21.15.45:0/189892925 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v109) ==== 181+0+0 (secure 0 0 0) 0x7fac0c0814e0 con 0x7fac1c0d1050 2024-05-08T05:50:48.653 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-05-08T05:50:48.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.657+0000 7fac21514640 1 -- 172.21.15.45:0/189892925 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fabfc0762d0 msgr2=0x7fabfc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:48.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.657+0000 7fac21514640 1 --2- 172.21.15.45:0/189892925 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fabfc0762d0 0x7fabfc078790 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fac1c1586d0 tx=0x7fac04009040 comp rx=0 tx=0).stop 2024-05-08T05:50:48.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.657+0000 7fac21514640 1 -- 172.21.15.45:0/189892925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac1c0d1050 msgr2=0x7fac1c1f85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:48.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.657+0000 7fac21514640 1 --2- 172.21.15.45:0/189892925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac1c0d1050 0x7fac1c1f85e0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fac0c02a290 tx=0x7fac0c0046c0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.657+0000 7fac21514640 1 -- 172.21.15.45:0/189892925 shutdown_connections 2024-05-08T05:50:48.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.657+0000 7fac21514640 1 --2- 172.21.15.45:0/189892925 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fabfc0762d0 0x7fabfc078790 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.657+0000 7fac21514640 1 --2- 172.21.15.45:0/189892925 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fac1c0d3670 0x7fac1c15d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.657+0000 7fac21514640 1 --2- 172.21.15.45:0/189892925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac1c0d3a50 0x7fac1c0c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.657+0000 7fac21514640 1 --2- 172.21.15.45:0/189892925 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac1c0d1050 0x7fac1c1f85e0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.657+0000 7fac21514640 1 -- 172.21.15.45:0/189892925 >> 172.21.15.45:0/189892925 conn(0x7fac1c09e390 msgr2=0x7fac1c0d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:48.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.657+0000 7fac21514640 1 -- 172.21.15.45:0/189892925 shutdown_connections 2024-05-08T05:50:48.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.657+0000 7fac21514640 1 -- 172.21.15.45:0/189892925 wait complete. 2024-05-08T05:50:48.672 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T05:50:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.786+0000 7fce7d6da640 1 Processor -- start 2024-05-08T05:50:48.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.786+0000 7fce7d6da640 1 -- start start 2024-05-08T05:50:48.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.787+0000 7fce7d6da640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce78162b70 0x7fce780d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.787+0000 7fce7d6da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fce780d3480 0x7fce780d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.787+0000 7fce7d6da640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fce780b4a40 0x7fce780b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.787+0000 7fce7d6da640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fce780d1f60 con 0x7fce78162b70 2024-05-08T05:50:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.787+0000 7fce7d6da640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fce78162f50 con 0x7fce780b4a40 2024-05-08T05:50:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.787+0000 7fce7d6da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fce780d3ee0 con 0x7fce780d3480 2024-05-08T05:50:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.787+0000 7fce76ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce78162b70 0x7fce780d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.787+0000 7fce76ffd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fce78162b70 0x7fce780d2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50694/0 (socket says 172.21.15.45:50694) 2024-05-08T05:50:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.787+0000 7fce76ffd640 1 -- 172.21.15.45:0/1480079224 learned_addr learned my addr 172.21.15.45:0/1480079224 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.787+0000 7fce777fe640 1 --2- 172.21.15.45:0/1480079224 >> v2:172.21.15.63:3300/0 conn(0x7fce780b4a40 0x7fce780b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.787+0000 7fce767fc640 1 --2- 172.21.15.45:0/1480079224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fce780d3480 0x7fce780d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.787+0000 7fce76ffd640 1 -- 172.21.15.45:0/1480079224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fce780d3480 msgr2=0x7fce780d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.787+0000 7fce76ffd640 1 --2- 172.21.15.45:0/1480079224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fce780d3480 0x7fce780d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.787+0000 7fce76ffd640 1 -- 172.21.15.45:0/1480079224 >> v2:172.21.15.63:3300/0 conn(0x7fce780b4a40 msgr2=0x7fce780b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:48.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.787+0000 7fce76ffd640 1 --2- 172.21.15.45:0/1480079224 >> v2:172.21.15.63:3300/0 conn(0x7fce780b4a40 0x7fce780b4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.787+0000 7fce76ffd640 1 -- 172.21.15.45:0/1480079224 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce780d21b0 con 0x7fce78162b70 2024-05-08T05:50:48.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.787+0000 7fce777fe640 1 --2- 172.21.15.45:0/1480079224 >> v2:172.21.15.63:3300/0 conn(0x7fce780b4a40 0x7fce780b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:48.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.788+0000 7fce76ffd640 1 --2- 172.21.15.45:0/1480079224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce78162b70 0x7fce780d2f40 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fce7808f680 tx=0x7fce6802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a629913d4d74190 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:48.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.788+0000 7fce75ffb640 1 -- 172.21.15.45:0/1480079224 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce68037070 con 0x7fce78162b70 2024-05-08T05:50:48.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.788+0000 7fce75ffb640 1 -- 172.21.15.45:0/1480079224 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce68032440 con 0x7fce78162b70 2024-05-08T05:50:48.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.788+0000 7fce75ffb640 1 -- 172.21.15.45:0/1480079224 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce68032810 con 0x7fce78162b70 2024-05-08T05:50:48.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.789+0000 7fce7d6da640 1 -- 172.21.15.45:0/1480079224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce78162b70 msgr2=0x7fce780d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:48.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.789+0000 7fce7d6da640 1 --2- 172.21.15.45:0/1480079224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce78162b70 0x7fce780d2f40 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fce7808f680 tx=0x7fce6802a1e0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.789+0000 7fce7d6da640 1 -- 172.21.15.45:0/1480079224 shutdown_connections 2024-05-08T05:50:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.789+0000 7fce7d6da640 1 --2- 172.21.15.45:0/1480079224 >> v2:172.21.15.63:3300/0 conn(0x7fce780b4a40 0x7fce780b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.789+0000 7fce7d6da640 1 --2- 172.21.15.45:0/1480079224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fce780d3480 0x7fce780d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.789+0000 7fce7d6da640 1 --2- 172.21.15.45:0/1480079224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce78162b70 0x7fce780d2f40 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.789+0000 7fce7d6da640 1 -- 172.21.15.45:0/1480079224 >> 172.21.15.45:0/1480079224 conn(0x7fce780d5040 msgr2=0x7fce780d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.789+0000 7fce7d6da640 1 -- 172.21.15.45:0/1480079224 shutdown_connections 2024-05-08T05:50:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.789+0000 7fce7d6da640 1 -- 172.21.15.45:0/1480079224 wait complete. 2024-05-08T05:50:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.790+0000 7fce7d6da640 1 Processor -- start 2024-05-08T05:50:48.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.790+0000 7fce7d6da640 1 -- start start 2024-05-08T05:50:48.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.791+0000 7fce7d6da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fce780b4a40 0x7fce7814ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:48.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.791+0000 7fce7d6da640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce780d3480 0x7fce780ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:48.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.791+0000 7fce7d6da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce78162b70 0x7fce780c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:48.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.791+0000 7fce7d6da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fce780b42f0 con 0x7fce78162b70 2024-05-08T05:50:48.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.791+0000 7fce7d6da640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fce780b4170 con 0x7fce780d3480 2024-05-08T05:50:48.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.791+0000 7fce7d6da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fce780b5410 con 0x7fce780b4a40 2024-05-08T05:50:48.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.791+0000 7fce777fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce78162b70 0x7fce780c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:48.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.791+0000 7fce777fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce78162b70 0x7fce780c86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50706/0 (socket says 172.21.15.45:50706) 2024-05-08T05:50:48.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.791+0000 7fce777fe640 1 -- 172.21.15.45:0/3463968896 learned_addr learned my addr 172.21.15.45:0/3463968896 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:48.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.791+0000 7fce767fc640 1 --2- 172.21.15.45:0/3463968896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce780d3480 0x7fce780ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:48.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.791+0000 7fce76ffd640 1 --2- 172.21.15.45:0/3463968896 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fce780b4a40 0x7fce7814ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:48.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.792+0000 7fce777fe640 1 -- 172.21.15.45:0/3463968896 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fce780b4a40 msgr2=0x7fce7814ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:48.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.792+0000 7fce777fe640 1 --2- 172.21.15.45:0/3463968896 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fce780b4a40 0x7fce7814ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.792+0000 7fce777fe640 1 -- 172.21.15.45:0/3463968896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce780d3480 msgr2=0x7fce780ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:48.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.792+0000 7fce777fe640 1 --2- 172.21.15.45:0/3463968896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce780d3480 0x7fce780ce3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.792+0000 7fce777fe640 1 -- 172.21.15.45:0/3463968896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce780cd300 con 0x7fce78162b70 2024-05-08T05:50:48.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.792+0000 7fce76ffd640 1 --2- 172.21.15.45:0/3463968896 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fce780b4a40 0x7fce7814ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:48.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.792+0000 7fce777fe640 1 --2- 172.21.15.45:0/3463968896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce78162b70 0x7fce780c86e0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fce70007e10 tx=0x7fce7000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:48.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.792+0000 7fce5f7fe640 1 -- 172.21.15.45:0/3463968896 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7001a070 con 0x7fce78162b70 2024-05-08T05:50:48.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.792+0000 7fce5f7fe640 1 -- 172.21.15.45:0/3463968896 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce70003b90 con 0x7fce78162b70 2024-05-08T05:50:48.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.792+0000 7fce7d6da640 1 -- 172.21.15.45:0/3463968896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce780cd5e0 con 0x7fce78162b70 2024-05-08T05:50:48.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.792+0000 7fce5f7fe640 1 -- 172.21.15.45:0/3463968896 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70004030 con 0x7fce78162b70 2024-05-08T05:50:48.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.792+0000 7fce7d6da640 1 -- 172.21.15.45:0/3463968896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce780cdba0 con 0x7fce78162b70 2024-05-08T05:50:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.793+0000 7fce7d6da640 1 -- 172.21.15.45:0/3463968896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce780693f0 con 0x7fce78162b70 2024-05-08T05:50:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.794+0000 7fce5f7fe640 1 -- 172.21.15.45:0/3463968896 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fce7001d070 con 0x7fce78162b70 2024-05-08T05:50:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.794+0000 7fce5f7fe640 1 --2- 172.21.15.45:0/3463968896 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fce640762d0 0x7fce64078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.794+0000 7fce5f7fe640 1 -- 172.21.15.45:0/3463968896 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5922+0+0 (secure 0 0 0) 0x7fce70097990 con 0x7fce78162b70 2024-05-08T05:50:48.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.794+0000 7fce76ffd640 1 --2- 172.21.15.45:0/3463968896 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fce640762d0 0x7fce64078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:48.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.795+0000 7fce76ffd640 1 --2- 172.21.15.45:0/3463968896 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fce640762d0 0x7fce64078790 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fce68034000 tx=0x7fce6802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:48.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.799+0000 7fce5f7fe640 1 -- 172.21.15.45:0/3463968896 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fce70010070 con 0x7fce78162b70 2024-05-08T05:50:48.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:48.995+0000 7fce7d6da640 1 -- 172.21.15.45:0/3463968896 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fce780d1950 con 0x7fce78162b70 2024-05-08T05:50:49.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.658+0000 7fce5f7fe640 1 -- 172.21.15.45:0/3463968896 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v110) ==== 169+0+0 (secure 0 0 0) 0x7fce70069070 con 0x7fce78162b70 2024-05-08T05:50:49.658 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-05-08T05:50:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.662+0000 7fce7d6da640 1 -- 172.21.15.45:0/3463968896 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fce640762d0 msgr2=0x7fce64078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.662+0000 7fce7d6da640 1 --2- 172.21.15.45:0/3463968896 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fce640762d0 0x7fce64078790 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fce68034000 tx=0x7fce6802e000 comp rx=0 tx=0).stop 2024-05-08T05:50:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.662+0000 7fce7d6da640 1 -- 172.21.15.45:0/3463968896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce78162b70 msgr2=0x7fce780c86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.662+0000 7fce7d6da640 1 --2- 172.21.15.45:0/3463968896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce78162b70 0x7fce780c86e0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fce70007e10 tx=0x7fce7000d4a0 comp rx=0 tx=0).stop 2024-05-08T05:50:49.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.662+0000 7fce7d6da640 1 -- 172.21.15.45:0/3463968896 shutdown_connections 2024-05-08T05:50:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.662+0000 7fce7d6da640 1 --2- 172.21.15.45:0/3463968896 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fce640762d0 0x7fce64078790 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.662+0000 7fce7d6da640 1 --2- 172.21.15.45:0/3463968896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fce78162b70 0x7fce780c86e0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.662+0000 7fce7d6da640 1 --2- 172.21.15.45:0/3463968896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce780d3480 0x7fce780ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.662+0000 7fce7d6da640 1 --2- 172.21.15.45:0/3463968896 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fce780b4a40 0x7fce7814ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.662+0000 7fce7d6da640 1 -- 172.21.15.45:0/3463968896 >> 172.21.15.45:0/3463968896 conn(0x7fce780d5040 msgr2=0x7fce780b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.662+0000 7fce7d6da640 1 -- 172.21.15.45:0/3463968896 shutdown_connections 2024-05-08T05:50:49.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.662+0000 7fce7d6da640 1 -- 172.21.15.45:0/3463968896 wait complete. 2024-05-08T05:50:49.678 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T05:50:49.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.790+0000 7f28d32a7640 1 Processor -- start 2024-05-08T05:50:49.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.790+0000 7f28d32a7640 1 -- start start 2024-05-08T05:50:49.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.791+0000 7f28d32a7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f28cc070d20 0x7f28cc0711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:49.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.791+0000 7f28d32a7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28cc06e110 con 0x7f28cc1629b0 2024-05-08T05:50:49.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.791+0000 7f28d32a7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28cc06e3f0 con 0x7f28cc0b2490 2024-05-08T05:50:49.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.791+0000 7f28d32a7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f28cc070a40 con 0x7f28cc070d20 2024-05-08T05:50:49.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.791+0000 7f28d1043640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f28cc1629b0 0x7f28cc06eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60480/0 (socket says 172.21.15.45:60480) 2024-05-08T05:50:49.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.791+0000 7f28d1043640 1 -- 172.21.15.45:0/3304020728 learned_addr learned my addr 172.21.15.45:0/3304020728 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:50:49.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28d1844640 1 --2- 172.21.15.45:0/3304020728 >> v2:172.21.15.63:3300/0 conn(0x7f28cc070d20 0x7f28cc0711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:49.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28c3fff640 1 -- 172.21.15.45:0/3304020728 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3170264660 0 0) 0x7f28cc06e110 con 0x7f28cc1629b0 2024-05-08T05:50:49.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28c3fff640 1 -- 172.21.15.45:0/3304020728 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28b8003520 con 0x7f28cc1629b0 2024-05-08T05:50:49.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28c3fff640 1 -- 172.21.15.45:0/3304020728 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3620288844 0 0) 0x7f28b8003520 con 0x7f28cc1629b0 2024-05-08T05:50:49.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28c3fff640 1 -- 172.21.15.45:0/3304020728 >> v2:172.21.15.63:3300/0 conn(0x7f28cc070d20 msgr2=0x7f28cc0711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:49.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28c3fff640 1 --2- 172.21.15.45:0/3304020728 >> v2:172.21.15.63:3300/0 conn(0x7f28cc070d20 0x7f28cc0711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:49.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28c3fff640 1 -- 172.21.15.45:0/3304020728 >> v1:172.21.15.63:6789/0 conn(0x7f28cc0b2490 legacy=0x7f28cc070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:49.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28c3fff640 1 -- 172.21.15.45:0/3304020728 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc06e6d0 con 0x7f28cc1629b0 2024-05-08T05:50:49.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28c3fff640 1 -- 172.21.15.45:0/3304020728 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f28c80016f0 con 0x7f28cc1629b0 2024-05-08T05:50:49.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28c3fff640 1 -- 172.21.15.45:0/3304020728 >> v1:172.21.15.45:6789/0 conn(0x7f28cc1629b0 legacy=0x7f28cc06eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:49.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28c3fff640 1 --2- 172.21.15.45:0/3304020728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28b8003a00 0x7f28b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:49.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28c3fff640 1 --2- 172.21.15.45:0/3304020728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28b8004800 0x7f28b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:49.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28c3fff640 1 --2- 172.21.15.45:0/3304020728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b80051c0 0x7f28b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:49.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28c3fff640 1 -- 172.21.15.45:0/3304020728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc070a40 con 0x7f28b8004800 2024-05-08T05:50:49.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28c3fff640 1 -- 172.21.15.45:0/3304020728 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28b800c060 con 0x7f28b80051c0 2024-05-08T05:50:49.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28c3fff640 1 -- 172.21.15.45:0/3304020728 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f28b800c260 con 0x7f28b8003a00 2024-05-08T05:50:49.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28d0842640 1 --2- 172.21.15.45:0/3304020728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28b8004800 0x7f28b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:49.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28d1844640 1 --2- 172.21.15.45:0/3304020728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b80051c0 0x7f28b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:49.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28d1043640 1 --2- 172.21.15.45:0/3304020728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28b8003a00 0x7f28b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:49.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28d0842640 1 -- 172.21.15.45:0/3304020728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28b8003a00 msgr2=0x7f28b8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:49.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28d0842640 1 --2- 172.21.15.45:0/3304020728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28b8003a00 0x7f28b8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:49.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28d0842640 1 -- 172.21.15.45:0/3304020728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b80051c0 msgr2=0x7f28b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:49.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28d0842640 1 --2- 172.21.15.45:0/3304020728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b80051c0 0x7f28b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:49.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28d0842640 1 -- 172.21.15.45:0/3304020728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b8003520 con 0x7f28b8004800 2024-05-08T05:50:49.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.792+0000 7f28d1844640 1 -- 172.21.15.45:0/3304020728 reap_dead start 2024-05-08T05:50:49.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.793+0000 7f28d0842640 1 --2- 172.21.15.45:0/3304020728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28b8004800 0x7f28b8004c80 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f28bc001810 tx=0x7f28b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67095b4a126cf353 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:49.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.793+0000 7f28c3fff640 1 -- 172.21.15.45:0/3304020728 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc037070 con 0x7f28b8004800 2024-05-08T05:50:49.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.793+0000 7f28c3fff640 1 -- 172.21.15.45:0/3304020728 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28bc02ac00 con 0x7f28b8004800 2024-05-08T05:50:49.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.793+0000 7f28c3fff640 1 -- 172.21.15.45:0/3304020728 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc001430 con 0x7f28b8004800 2024-05-08T05:50:49.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.794+0000 7f28d32a7640 1 -- 172.21.15.45:0/3304020728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28b8004800 msgr2=0x7f28b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:49.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.794+0000 7f28d32a7640 1 --2- 172.21.15.45:0/3304020728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28b8004800 0x7f28b8004c80 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f28bc001810 tx=0x7f28b8005830 comp rx=0 tx=0).stop 2024-05-08T05:50:49.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.794+0000 7f28d32a7640 1 -- 172.21.15.45:0/3304020728 shutdown_connections 2024-05-08T05:50:49.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.794+0000 7f28d32a7640 1 --2- 172.21.15.45:0/3304020728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28b8004800 0x7f28b8004c80 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:49.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.794+0000 7f28d32a7640 1 -- 172.21.15.45:0/3304020728 >> 172.21.15.45:0/3304020728 conn(0x7f28cc0b49d0 msgr2=0x7f28cc0b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:49.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.794+0000 7f28d32a7640 1 -- 172.21.15.45:0/3304020728 shutdown_connections 2024-05-08T05:50:49.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.794+0000 7f28d32a7640 1 -- 172.21.15.45:0/3304020728 wait complete. 2024-05-08T05:50:49.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.795+0000 7f28d32a7640 1 Processor -- start 2024-05-08T05:50:49.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.795+0000 7f28d32a7640 1 -- start start 2024-05-08T05:50:49.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.796+0000 7f28d32a7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b8004800 0x7f28cc15c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:49.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.796+0000 7f28d32a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28cc157f70 0x7f28cc15ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:49.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.796+0000 7f28d32a7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28cc15b240 0x7f28cc0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:49.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.796+0000 7f28d32a7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc0711d0 con 0x7f28cc157f70 2024-05-08T05:50:49.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.796+0000 7f28d32a7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc06dec0 con 0x7f28b8004800 2024-05-08T05:50:49.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.796+0000 7f28d32a7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f28cc071350 con 0x7f28cc15b240 2024-05-08T05:50:49.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.796+0000 7f28d0842640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28cc157f70 0x7f28cc15ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:49.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.796+0000 7f28d0842640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28cc157f70 0x7f28cc15ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50736/0 (socket says 172.21.15.45:50736) 2024-05-08T05:50:49.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.796+0000 7f28d0842640 1 -- 172.21.15.45:0/3500018436 learned_addr learned my addr 172.21.15.45:0/3500018436 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:49.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.796+0000 7f28d0842640 1 -- 172.21.15.45:0/3500018436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28cc15b240 msgr2=0x7f28cc0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:49.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.796+0000 7f28d1043640 1 --2- 172.21.15.45:0/3500018436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b8004800 0x7f28cc15c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:49.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.796+0000 7f28d0842640 1 --2- 172.21.15.45:0/3500018436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28cc15b240 0x7f28cc0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:49.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.796+0000 7f28d0842640 1 -- 172.21.15.45:0/3500018436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b8004800 msgr2=0x7f28cc15c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:49.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.796+0000 7f28d0842640 1 --2- 172.21.15.45:0/3500018436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b8004800 0x7f28cc15c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:49.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.796+0000 7f28d0842640 1 -- 172.21.15.45:0/3500018436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc15bf20 con 0x7f28cc157f70 2024-05-08T05:50:49.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.797+0000 7f28d0842640 1 --2- 172.21.15.45:0/3500018436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28cc157f70 0x7f28cc15ad00 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f28bc000c00 tx=0x7f28bc001110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:49.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.797+0000 7f28c1ffb640 1 -- 172.21.15.45:0/3500018436 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc040020 con 0x7f28cc157f70 2024-05-08T05:50:49.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.797+0000 7f28c1ffb640 1 -- 172.21.15.45:0/3500018436 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28bc038040 con 0x7f28cc157f70 2024-05-08T05:50:49.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.797+0000 7f28d32a7640 1 -- 172.21.15.45:0/3500018436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28cc15b7c0 con 0x7f28cc157f70 2024-05-08T05:50:49.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.797+0000 7f28c1ffb640 1 -- 172.21.15.45:0/3500018436 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc037070 con 0x7f28cc157f70 2024-05-08T05:50:49.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.797+0000 7f28d32a7640 1 -- 172.21.15.45:0/3500018436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28cc15c1c0 con 0x7f28cc157f70 2024-05-08T05:50:49.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.798+0000 7f28d32a7640 1 -- 172.21.15.45:0/3500018436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28cc0693f0 con 0x7f28cc157f70 2024-05-08T05:50:49.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.799+0000 7f28c1ffb640 1 -- 172.21.15.45:0/3500018436 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f28bc043020 con 0x7f28cc157f70 2024-05-08T05:50:49.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.799+0000 7f28c1ffb640 1 --2- 172.21.15.45:0/3500018436 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f28b00762d0 0x7f28b0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:49.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.799+0000 7f28c1ffb640 1 -- 172.21.15.45:0/3500018436 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5421+0+0 (secure 0 0 0) 0x7f28bc0b73f0 con 0x7f28cc157f70 2024-05-08T05:50:49.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.799+0000 7f28d1043640 1 --2- 172.21.15.45:0/3500018436 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f28b00762d0 0x7f28b0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:49.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.800+0000 7f28d1043640 1 --2- 172.21.15.45:0/3500018436 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f28b00762d0 0x7f28b0078790 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f28c8001eb0 tx=0x7f28c8001170 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:49.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.803+0000 7f28c1ffb640 1 -- 172.21.15.45:0/3500018436 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f28bc04a030 con 0x7f28cc157f70 2024-05-08T05:50:49.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.993+0000 7f28d32a7640 1 -- 172.21.15.45:0/3500018436 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f28cc0d3270 con 0x7f28cc157f70 2024-05-08T05:50:49.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.996+0000 7f28c1ffb640 1 -- 172.21.15.45:0/3500018436 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f28bc080e70 con 0x7f28cc157f70 2024-05-08T05:50:49.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.998+0000 7f28d32a7640 1 -- 172.21.15.45:0/3500018436 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f28b00762d0 msgr2=0x7f28b0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:49.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.998+0000 7f28d32a7640 1 --2- 172.21.15.45:0/3500018436 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f28b00762d0 0x7f28b0078790 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f28c8001eb0 tx=0x7f28c8001170 comp rx=0 tx=0).stop 2024-05-08T05:50:49.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.998+0000 7f28d32a7640 1 -- 172.21.15.45:0/3500018436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28cc157f70 msgr2=0x7f28cc15ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:49.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.998+0000 7f28d32a7640 1 --2- 172.21.15.45:0/3500018436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28cc157f70 0x7f28cc15ad00 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f28bc000c00 tx=0x7f28bc001110 comp rx=0 tx=0).stop 2024-05-08T05:50:49.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.998+0000 7f28d32a7640 1 -- 172.21.15.45:0/3500018436 shutdown_connections 2024-05-08T05:50:49.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.998+0000 7f28d32a7640 1 --2- 172.21.15.45:0/3500018436 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f28b00762d0 0x7f28b0078790 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:49.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.998+0000 7f28d32a7640 1 --2- 172.21.15.45:0/3500018436 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f28cc15b240 0x7f28cc0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:49.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.998+0000 7f28d32a7640 1 --2- 172.21.15.45:0/3500018436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28cc157f70 0x7f28cc15ad00 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:49.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.998+0000 7f28d32a7640 1 --2- 172.21.15.45:0/3500018436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28b8004800 0x7f28cc15c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:49.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.998+0000 7f28d32a7640 1 -- 172.21.15.45:0/3500018436 >> 172.21.15.45:0/3500018436 conn(0x7f28cc0b49d0 msgr2=0x7f28cc0d18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:49.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.999+0000 7f28d32a7640 1 -- 172.21.15.45:0/3500018436 shutdown_connections 2024-05-08T05:50:50.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:49.999+0000 7f28d32a7640 1 -- 172.21.15.45:0/3500018436 wait complete. 2024-05-08T05:50:50.010 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-05-08T05:50:50.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.114+0000 7fc8e4146640 1 Processor -- start 2024-05-08T05:50:50.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.114+0000 7fc8e4146640 1 -- start start 2024-05-08T05:50:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.115+0000 7fc8e4146640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc8dc15f420 0x7fc8dc0d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.115+0000 7fc8e4146640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8dc1630b0 con 0x7fc8dc0d2080 2024-05-08T05:50:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.115+0000 7fc8e4146640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8dc0d2550 con 0x7fc8dc0d3ca0 2024-05-08T05:50:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.115+0000 7fc8e4146640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc8dc0d2c80 con 0x7fc8dc15f420 2024-05-08T05:50:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.115+0000 7fc8e1ee2640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc8dc15f420 0x7fc8dc0d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.115+0000 7fc8e26e3640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc8dc0d2080 0x7fc8dc0d4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:60492/0 (socket says 172.21.15.45:60492) 2024-05-08T05:50:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.115+0000 7fc8e26e3640 1 -- 172.21.15.45:0/1003320008 learned_addr learned my addr 172.21.15.45:0/1003320008 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:50:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.116+0000 7fc8e1ee2640 1 -- 172.21.15.45:0/1003320008 >> v1:172.21.15.63:6789/0 conn(0x7fc8dc0d3ca0 legacy=0x7fc8dc0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.116+0000 7fc8e1ee2640 1 -- 172.21.15.45:0/1003320008 >> v1:172.21.15.45:6789/0 conn(0x7fc8dc0d2080 legacy=0x7fc8dc0d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:50.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.116+0000 7fc8e1ee2640 1 -- 172.21.15.45:0/1003320008 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8dc0d2f60 con 0x7fc8dc15f420 2024-05-08T05:50:50.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.117+0000 7fc8e1ee2640 1 --2- 172.21.15.45:0/1003320008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8dc15f420 0x7fc8dc0d3760 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40097a0 tx=0x7fc8d402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e96dddc6b254e73 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:50.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.117+0000 7fc8e0ee0640 1 -- 172.21.15.45:0/1003320008 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4037070 con 0x7fc8dc15f420 2024-05-08T05:50:50.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.117+0000 7fc8e0ee0640 1 -- 172.21.15.45:0/1003320008 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d4032440 con 0x7fc8dc15f420 2024-05-08T05:50:50.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.117+0000 7fc8e0ee0640 1 -- 172.21.15.45:0/1003320008 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4032810 con 0x7fc8dc15f420 2024-05-08T05:50:50.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.117+0000 7fc8e4146640 1 -- 172.21.15.45:0/1003320008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8dc15f420 msgr2=0x7fc8dc0d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:50.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.117+0000 7fc8e4146640 1 --2- 172.21.15.45:0/1003320008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8dc15f420 0x7fc8dc0d3760 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40097a0 tx=0x7fc8d402a230 comp rx=0 tx=0).stop 2024-05-08T05:50:50.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.117+0000 7fc8e4146640 1 -- 172.21.15.45:0/1003320008 shutdown_connections 2024-05-08T05:50:50.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.117+0000 7fc8e4146640 1 --2- 172.21.15.45:0/1003320008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8dc15f420 0x7fc8dc0d3760 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:50.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.117+0000 7fc8e4146640 1 -- 172.21.15.45:0/1003320008 >> 172.21.15.45:0/1003320008 conn(0x7fc8dc08eac0 msgr2=0x7fc8dc09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:50.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.117+0000 7fc8e4146640 1 -- 172.21.15.45:0/1003320008 shutdown_connections 2024-05-08T05:50:50.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.117+0000 7fc8e4146640 1 -- 172.21.15.45:0/1003320008 wait complete. 2024-05-08T05:50:50.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.119+0000 7fc8e4146640 1 Processor -- start 2024-05-08T05:50:50.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.119+0000 7fc8e4146640 1 -- start start 2024-05-08T05:50:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.120+0000 7fc8e4146640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8dc0d2080 0x7fc8dc155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.120+0000 7fc8e4146640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8dc0ccaf0 0x7fc8dc0ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.120+0000 7fc8e4146640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8dc0b53e0 0x7fc8dc0b5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.120+0000 7fc8e4146640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc0d53d0 con 0x7fc8dc0ccaf0 2024-05-08T05:50:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.120+0000 7fc8e4146640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc0d2830 con 0x7fc8dc0b53e0 2024-05-08T05:50:50.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.120+0000 7fc8e4146640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc8dc0d5550 con 0x7fc8dc0d2080 2024-05-08T05:50:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.120+0000 7fc8e16e1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8dc0ccaf0 0x7fc8dc0ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.120+0000 7fc8e16e1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8dc0ccaf0 0x7fc8dc0ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:50758/0 (socket says 172.21.15.45:50758) 2024-05-08T05:50:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.120+0000 7fc8e16e1640 1 -- 172.21.15.45:0/3663418348 learned_addr learned my addr 172.21.15.45:0/3663418348 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.120+0000 7fc8e1ee2640 1 --2- 172.21.15.45:0/3663418348 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8dc0d2080 0x7fc8dc155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.120+0000 7fc8e26e3640 1 --2- 172.21.15.45:0/3663418348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8dc0b53e0 0x7fc8dc0b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.120+0000 7fc8e16e1640 1 -- 172.21.15.45:0/3663418348 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8dc0d2080 msgr2=0x7fc8dc155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.120+0000 7fc8e16e1640 1 --2- 172.21.15.45:0/3663418348 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8dc0d2080 0x7fc8dc155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.120+0000 7fc8e16e1640 1 -- 172.21.15.45:0/3663418348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8dc0b53e0 msgr2=0x7fc8dc0b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.120+0000 7fc8e16e1640 1 --2- 172.21.15.45:0/3663418348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8dc0b53e0 0x7fc8dc0b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.120+0000 7fc8e16e1640 1 -- 172.21.15.45:0/3663418348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8dc0cd640 con 0x7fc8dc0ccaf0 2024-05-08T05:50:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.120+0000 7fc8e16e1640 1 --2- 172.21.15.45:0/3663418348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8dc0ccaf0 0x7fc8dc0ccf70 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fc8d000c420 tx=0x7fc8d000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.120+0000 7fc8da7fc640 1 -- 172.21.15.45:0/3663418348 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0017070 con 0x7fc8dc0ccaf0 2024-05-08T05:50:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.121+0000 7fc8da7fc640 1 -- 172.21.15.45:0/3663418348 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d00052d0 con 0x7fc8dc0ccaf0 2024-05-08T05:50:50.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.121+0000 7fc8da7fc640 1 -- 172.21.15.45:0/3663418348 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0003830 con 0x7fc8dc0ccaf0 2024-05-08T05:50:50.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.121+0000 7fc8e4146640 1 -- 172.21.15.45:0/3663418348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8dc0cd920 con 0x7fc8dc0ccaf0 2024-05-08T05:50:50.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.121+0000 7fc8e4146640 1 -- 172.21.15.45:0/3663418348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8dc0cdee0 con 0x7fc8dc0ccaf0 2024-05-08T05:50:50.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.122+0000 7fc8e4146640 1 -- 172.21.15.45:0/3663418348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8dc0693f0 con 0x7fc8dc0ccaf0 2024-05-08T05:50:50.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.124+0000 7fc8da7fc640 1 -- 172.21.15.45:0/3663418348 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc8d001a070 con 0x7fc8dc0ccaf0 2024-05-08T05:50:50.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.124+0000 7fc8da7fc640 1 --2- 172.21.15.45:0/3663418348 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc8c80762d0 0x7fc8c8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:50.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.125+0000 7fc8da7fc640 1 -- 172.21.15.45:0/3663418348 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5421+0+0 (secure 0 0 0) 0x7fc8d0096bf0 con 0x7fc8dc0ccaf0 2024-05-08T05:50:50.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.125+0000 7fc8e1ee2640 1 --2- 172.21.15.45:0/3663418348 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc8c80762d0 0x7fc8c8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:50.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.125+0000 7fc8e1ee2640 1 --2- 172.21.15.45:0/3663418348 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc8c80762d0 0x7fc8c8078790 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4034000 tx=0x7fc8d402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:50.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.130+0000 7fc8da7fc640 1 -- 172.21.15.45:0/3663418348 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc8d0021020 con 0x7fc8dc0ccaf0 2024-05-08T05:50:50.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.396+0000 7fc8e4146640 1 -- 172.21.15.45:0/3663418348 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7fc8dc0cf2c0 con 0x7fc8dc0ccaf0 2024-05-08T05:50:50.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.659+0000 7fc8da7fc640 1 -- 172.21.15.45:0/3663418348 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v424) ==== 146+0+0 (secure 0 0 0) 0x7fc8d0060750 con 0x7fc8dc0ccaf0 2024-05-08T05:50:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.661+0000 7fc8e4146640 1 -- 172.21.15.45:0/3663418348 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc8c80762d0 msgr2=0x7fc8c8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.661+0000 7fc8e4146640 1 --2- 172.21.15.45:0/3663418348 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc8c80762d0 0x7fc8c8078790 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4034000 tx=0x7fc8d402e000 comp rx=0 tx=0).stop 2024-05-08T05:50:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.661+0000 7fc8e4146640 1 -- 172.21.15.45:0/3663418348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8dc0ccaf0 msgr2=0x7fc8dc0ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.661+0000 7fc8e4146640 1 --2- 172.21.15.45:0/3663418348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8dc0ccaf0 0x7fc8dc0ccf70 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fc8d000c420 tx=0x7fc8d000c8e0 comp rx=0 tx=0).stop 2024-05-08T05:50:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.661+0000 7fc8e4146640 1 -- 172.21.15.45:0/3663418348 shutdown_connections 2024-05-08T05:50:50.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.661+0000 7fc8e4146640 1 --2- 172.21.15.45:0/3663418348 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc8c80762d0 0x7fc8c8078790 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:50.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.661+0000 7fc8e4146640 1 --2- 172.21.15.45:0/3663418348 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8dc0b53e0 0x7fc8dc0b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:50.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.661+0000 7fc8e4146640 1 --2- 172.21.15.45:0/3663418348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8dc0ccaf0 0x7fc8dc0ccf70 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:50.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.661+0000 7fc8e4146640 1 --2- 172.21.15.45:0/3663418348 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8dc0d2080 0x7fc8dc155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:50.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.661+0000 7fc8e4146640 1 -- 172.21.15.45:0/3663418348 >> 172.21.15.45:0/3663418348 conn(0x7fc8dc08eac0 msgr2=0x7fc8dc06dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:50.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.661+0000 7fc8e4146640 1 -- 172.21.15.45:0/3663418348 shutdown_connections 2024-05-08T05:50:50.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.661+0000 7fc8e4146640 1 -- 172.21.15.45:0/3663418348 wait complete. 2024-05-08T05:50:50.673 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-08T05:50:50.673 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-05-08T05:50:50.674 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:50:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.783+0000 7fc0fba7e640 1 Processor -- start 2024-05-08T05:50:50.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.783+0000 7fc0fba7e640 1 -- start start 2024-05-08T05:50:50.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.784+0000 7fc0fba7e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc0f415c340 0x7fc0f4166970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:50.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.784+0000 7fc0fba7e640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0f415c920 con 0x7fc0f4072840 2024-05-08T05:50:50.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.784+0000 7fc0fba7e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0f415cc00 con 0x7fc0f406e0f0 2024-05-08T05:50:50.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.784+0000 7fc0fba7e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc0f415cee0 con 0x7fc0f415c340 2024-05-08T05:50:50.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.784+0000 7fc0fa01b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc0f415c340 0x7fc0f4166970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:50.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.785+0000 7fc0fa01b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc0f415c340 0x7fc0f4166970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54868/0 (socket says 172.21.15.45:54868) 2024-05-08T05:50:50.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.785+0000 7fc0fa01b640 1 -- 172.21.15.45:0/2508386907 learned_addr learned my addr 172.21.15.45:0/2508386907 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:50.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.785+0000 7fc0f9019640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc0f4072840 0x7fc0f406e4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35106/0 (socket says 172.21.15.45:35106) 2024-05-08T05:50:50.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.785+0000 7fc0fa01b640 1 -- 172.21.15.45:0/2508386907 >> v1:172.21.15.63:6789/0 conn(0x7fc0f406e0f0 legacy=0x7fc0f40d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:50.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.785+0000 7fc0fa01b640 1 -- 172.21.15.45:0/2508386907 >> v1:172.21.15.45:6789/0 conn(0x7fc0f4072840 legacy=0x7fc0f406e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:50.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.785+0000 7fc0fa01b640 1 -- 172.21.15.45:0/2508386907 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f415d1c0 con 0x7fc0f415c340 2024-05-08T05:50:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.785+0000 7fc0fa01b640 1 --2- 172.21.15.45:0/2508386907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0f415c340 0x7fc0f4166970 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009e50 tx=0x7fc0ec02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa73774d7b9a830 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.786+0000 7fc0f8818640 1 -- 172.21.15.45:0/2508386907 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec036070 con 0x7fc0f415c340 2024-05-08T05:50:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.786+0000 7fc0f8818640 1 -- 172.21.15.45:0/2508386907 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0ec031600 con 0x7fc0f415c340 2024-05-08T05:50:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.786+0000 7fc0f8818640 1 -- 172.21.15.45:0/2508386907 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec0319d0 con 0x7fc0f415c340 2024-05-08T05:50:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.786+0000 7fc0fba7e640 1 -- 172.21.15.45:0/2508386907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0f415c340 msgr2=0x7fc0f4166970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.786+0000 7fc0fba7e640 1 --2- 172.21.15.45:0/2508386907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0f415c340 0x7fc0f4166970 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009e50 tx=0x7fc0ec02a730 comp rx=0 tx=0).stop 2024-05-08T05:50:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.786+0000 7fc0fba7e640 1 -- 172.21.15.45:0/2508386907 shutdown_connections 2024-05-08T05:50:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.786+0000 7fc0fba7e640 1 --2- 172.21.15.45:0/2508386907 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0f415c340 0x7fc0f4166970 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.786+0000 7fc0fba7e640 1 -- 172.21.15.45:0/2508386907 >> 172.21.15.45:0/2508386907 conn(0x7fc0f406c3a0 msgr2=0x7fc0f406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.786+0000 7fc0fba7e640 1 -- 172.21.15.45:0/2508386907 shutdown_connections 2024-05-08T05:50:50.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.786+0000 7fc0fba7e640 1 -- 172.21.15.45:0/2508386907 wait complete. 2024-05-08T05:50:50.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.788+0000 7fc0fba7e640 1 Processor -- start 2024-05-08T05:50:50.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.788+0000 7fc0fba7e640 1 -- start start 2024-05-08T05:50:50.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.788+0000 7fc0fba7e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0f406e0f0 0x7fc0f415a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:50.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.788+0000 7fc0fba7e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f4072840 0x7fc0f4140300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:50.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.788+0000 7fc0fba7e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0f415c340 0x7fc0f4140840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:50.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.788+0000 7fc0fba7e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f4153230 con 0x7fc0f406e0f0 2024-05-08T05:50:50.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.788+0000 7fc0fba7e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f41530b0 con 0x7fc0f4072840 2024-05-08T05:50:50.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.788+0000 7fc0fba7e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc0f4166eb0 con 0x7fc0f415c340 2024-05-08T05:50:50.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.788+0000 7fc0f981a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0f406e0f0 0x7fc0f415a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:50.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.788+0000 7fc0f981a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0f406e0f0 0x7fc0f415a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54882/0 (socket says 172.21.15.45:54882) 2024-05-08T05:50:50.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.788+0000 7fc0f981a640 1 -- 172.21.15.45:0/3167674398 learned_addr learned my addr 172.21.15.45:0/3167674398 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.789+0000 7fc0f9019640 1 --2- 172.21.15.45:0/3167674398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f4072840 0x7fc0f4140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.789+0000 7fc0fa01b640 1 --2- 172.21.15.45:0/3167674398 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0f415c340 0x7fc0f4140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.789+0000 7fc0f981a640 1 -- 172.21.15.45:0/3167674398 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0f415c340 msgr2=0x7fc0f4140840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.789+0000 7fc0f981a640 1 --2- 172.21.15.45:0/3167674398 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0f415c340 0x7fc0f4140840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.789+0000 7fc0f981a640 1 -- 172.21.15.45:0/3167674398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f4072840 msgr2=0x7fc0f4140300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.789+0000 7fc0f981a640 1 --2- 172.21.15.45:0/3167674398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f4072840 0x7fc0f4140300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.789+0000 7fc0f981a640 1 -- 172.21.15.45:0/3167674398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f414ead0 con 0x7fc0f406e0f0 2024-05-08T05:50:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.789+0000 7fc0f9019640 1 --2- 172.21.15.45:0/3167674398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f4072840 0x7fc0f4140300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.789+0000 7fc0f981a640 1 --2- 172.21.15.45:0/3167674398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0f406e0f0 0x7fc0f415a1d0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc0f000c480 tx=0x7fc0f000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.789+0000 7fc0ea7fc640 1 -- 172.21.15.45:0/3167674398 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f00052d0 con 0x7fc0f406e0f0 2024-05-08T05:50:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.789+0000 7fc0ea7fc640 1 -- 172.21.15.45:0/3167674398 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f0005470 con 0x7fc0f406e0f0 2024-05-08T05:50:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.789+0000 7fc0fba7e640 1 -- 172.21.15.45:0/3167674398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f414edb0 con 0x7fc0f406e0f0 2024-05-08T05:50:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.789+0000 7fc0ea7fc640 1 -- 172.21.15.45:0/3167674398 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f0012870 con 0x7fc0f406e0f0 2024-05-08T05:50:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.789+0000 7fc0fba7e640 1 -- 172.21.15.45:0/3167674398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f414f370 con 0x7fc0f406e0f0 2024-05-08T05:50:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.790+0000 7fc0fba7e640 1 -- 172.21.15.45:0/3167674398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f414f570 con 0x7fc0f406e0f0 2024-05-08T05:50:50.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.791+0000 7fc0ea7fc640 1 -- 172.21.15.45:0/3167674398 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc0f0012a10 con 0x7fc0f406e0f0 2024-05-08T05:50:50.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.791+0000 7fc0ea7fc640 1 --2- 172.21.15.45:0/3167674398 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc0d80762d0 0x7fc0d8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:50.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.791+0000 7fc0ea7fc640 1 -- 172.21.15.45:0/3167674398 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5421+0+0 (secure 0 0 0) 0x7fc0f00970e0 con 0x7fc0f406e0f0 2024-05-08T05:50:50.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.792+0000 7fc0f9019640 1 --2- 172.21.15.45:0/3167674398 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc0d80762d0 0x7fc0d8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.792+0000 7fc0f9019640 1 --2- 172.21.15.45:0/3167674398 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc0d80762d0 0x7fc0d8078790 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc0f415b830 tx=0x7fc0e4007d00 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:50.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.795+0000 7fc0ea7fc640 1 -- 172.21.15.45:0/3167674398 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc0f414f570 con 0x7fc0f406e0f0 2024-05-08T05:50:50.988 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.988+0000 7fc0fba7e640 1 -- 172.21.15.45:0/3167674398 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0f40693f0 con 0x7fc0f406e0f0 2024-05-08T05:50:50.989 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.989+0000 7fc0ea7fc640 1 -- 172.21.15.45:0/3167674398 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+13383 (secure 0 0 0) 0x7fc0f0060bf0 con 0x7fc0f406e0f0 2024-05-08T05:50:50.989 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:50:50.989 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":110,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:50:49.654412+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/2729952281":"2024-05-09T05:50:43.627290+0000","172.21.15.45:6834/2729952281":"2024-05-09T05:50:43.627290+0000","172.21.15.45:6835/878092981":"2024-05-09T05:50:07.216878+0000","172.21.15.45:6834/878092981":"2024-05-09T05:50:07.216878+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:50:50.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.991+0000 7fc0fba7e640 1 -- 172.21.15.45:0/3167674398 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc0d80762d0 msgr2=0x7fc0d8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:50.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.991+0000 7fc0fba7e640 1 --2- 172.21.15.45:0/3167674398 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc0d80762d0 0x7fc0d8078790 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc0f415b830 tx=0x7fc0e4007d00 comp rx=0 tx=0).stop 2024-05-08T05:50:50.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.991+0000 7fc0fba7e640 1 -- 172.21.15.45:0/3167674398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0f406e0f0 msgr2=0x7fc0f415a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.991+0000 7fc0fba7e640 1 --2- 172.21.15.45:0/3167674398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0f406e0f0 0x7fc0f415a1d0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc0f000c480 tx=0x7fc0f000c940 comp rx=0 tx=0).stop 2024-05-08T05:50:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.991+0000 7fc0fba7e640 1 -- 172.21.15.45:0/3167674398 shutdown_connections 2024-05-08T05:50:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.991+0000 7fc0fba7e640 1 --2- 172.21.15.45:0/3167674398 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc0d80762d0 0x7fc0d8078790 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.991+0000 7fc0fba7e640 1 --2- 172.21.15.45:0/3167674398 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc0f415c340 0x7fc0f4140840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.991+0000 7fc0fba7e640 1 --2- 172.21.15.45:0/3167674398 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0f4072840 0x7fc0f4140300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.991+0000 7fc0fba7e640 1 --2- 172.21.15.45:0/3167674398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0f406e0f0 0x7fc0f415a1d0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.991+0000 7fc0fba7e640 1 -- 172.21.15.45:0/3167674398 >> 172.21.15.45:0/3167674398 conn(0x7fc0f406c3a0 msgr2=0x7fc0f406e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.991+0000 7fc0fba7e640 1 -- 172.21.15.45:0/3167674398 shutdown_connections 2024-05-08T05:50:50.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:50.991+0000 7fc0fba7e640 1 -- 172.21.15.45:0/3167674398 wait complete. 2024-05-08T05:50:51.003 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T05:41:18.847604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T05:50:51.003 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T05:50:51.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.109+0000 7fb54b912640 1 Processor -- start 2024-05-08T05:50:51.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.110+0000 7fb54b912640 1 -- start start 2024-05-08T05:50:51.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.110+0000 7fb54b912640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5440d3cd0 0x7fb5440d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.110+0000 7fb54b912640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5440b1c90 con 0x7fb54415f380 2024-05-08T05:50:51.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.110+0000 7fb54b912640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5440d0a50 con 0x7fb5440b3f30 2024-05-08T05:50:51.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.110+0000 7fb54b912640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb5440d0d30 con 0x7fb5440d3cd0 2024-05-08T05:50:51.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb5496ae640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb54415f380 0x7fb5440d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35122/0 (socket says 172.21.15.45:35122) 2024-05-08T05:50:51.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb5496ae640 1 -- 172.21.15.45:0/843786744 learned_addr learned my addr 172.21.15.45:0/843786744 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:50:51.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb548ead640 1 --2- 172.21.15.45:0/843786744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5440d3cd0 0x7fb5440d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb53bfff640 1 -- 172.21.15.45:0/843786744 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4021918750 0 0) 0x7fb5440b1c90 con 0x7fb54415f380 2024-05-08T05:50:51.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb53bfff640 1 -- 172.21.15.45:0/843786744 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb530003520 con 0x7fb54415f380 2024-05-08T05:50:51.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb53bfff640 1 -- 172.21.15.45:0/843786744 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 781825136 0 0) 0x7fb5440d0a50 con 0x7fb5440b3f30 2024-05-08T05:50:51.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb53bfff640 1 -- 172.21.15.45:0/843786744 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5440b1c90 con 0x7fb5440b3f30 2024-05-08T05:50:51.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb53bfff640 1 -- 172.21.15.45:0/843786744 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 893211810 0 0) 0x7fb530003520 con 0x7fb54415f380 2024-05-08T05:50:51.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb53bfff640 1 -- 172.21.15.45:0/843786744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5440d3cd0 msgr2=0x7fb5440d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:51.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb53bfff640 1 --2- 172.21.15.45:0/843786744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5440d3cd0 0x7fb5440d15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb53bfff640 1 -- 172.21.15.45:0/843786744 >> v1:172.21.15.63:6789/0 conn(0x7fb5440b3f30 legacy=0x7fb5440d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb53bfff640 1 -- 172.21.15.45:0/843786744 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5440b1f00 con 0x7fb54415f380 2024-05-08T05:50:51.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb53bfff640 1 -- 172.21.15.45:0/843786744 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fb5400016f0 con 0x7fb54415f380 2024-05-08T05:50:51.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb53bfff640 1 -- 172.21.15.45:0/843786744 >> v1:172.21.15.45:6789/0 conn(0x7fb54415f380 legacy=0x7fb5440d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb53bfff640 1 --2- 172.21.15.45:0/843786744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb530003c10 0x7fb530004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb53bfff640 1 --2- 172.21.15.45:0/843786744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5440d3cd0 0x7fb530004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb53bfff640 1 --2- 172.21.15.45:0/843786744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb530004d40 0x7fb5300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb53bfff640 1 -- 172.21.15.45:0/843786744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb5440d0d30 con 0x7fb530004d40 2024-05-08T05:50:51.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb53bfff640 1 -- 172.21.15.45:0/843786744 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb53000bb50 con 0x7fb530003c10 2024-05-08T05:50:51.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb53bfff640 1 -- 172.21.15.45:0/843786744 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb53000bd50 con 0x7fb5440d3cd0 2024-05-08T05:50:51.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb549eaf640 1 --2- 172.21.15.45:0/843786744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb530004d40 0x7fb5300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.111+0000 7fb5496ae640 1 --2- 172.21.15.45:0/843786744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb530003c10 0x7fb530004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.112+0000 7fb548ead640 1 --2- 172.21.15.45:0/843786744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5440d3cd0 0x7fb530004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.112+0000 7fb549eaf640 1 -- 172.21.15.45:0/843786744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5440d3cd0 msgr2=0x7fb530004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:51.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.112+0000 7fb549eaf640 1 --2- 172.21.15.45:0/843786744 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb5440d3cd0 0x7fb530004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.112+0000 7fb549eaf640 1 -- 172.21.15.45:0/843786744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb530003c10 msgr2=0x7fb530004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:51.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.112+0000 7fb549eaf640 1 --2- 172.21.15.45:0/843786744 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb530003c10 0x7fb530004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.112+0000 7fb549eaf640 1 -- 172.21.15.45:0/843786744 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb530003520 con 0x7fb530004d40 2024-05-08T05:50:51.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.112+0000 7fb549eaf640 1 --2- 172.21.15.45:0/843786744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb530004d40 0x7fb5300051c0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb53c002000 tx=0x7fb530005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad34f7942c030bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:51.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.113+0000 7fb53bfff640 1 -- 172.21.15.45:0/843786744 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c001c20 con 0x7fb530004d40 2024-05-08T05:50:51.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.113+0000 7fb53bfff640 1 -- 172.21.15.45:0/843786744 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb53c006d70 con 0x7fb530004d40 2024-05-08T05:50:51.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.113+0000 7fb53bfff640 1 -- 172.21.15.45:0/843786744 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c003bb0 con 0x7fb530004d40 2024-05-08T05:50:51.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.113+0000 7fb54b912640 1 -- 172.21.15.45:0/843786744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb530004d40 msgr2=0x7fb5300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.113+0000 7fb54b912640 1 --2- 172.21.15.45:0/843786744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb530004d40 0x7fb5300051c0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb53c002000 tx=0x7fb530005830 comp rx=0 tx=0).stop 2024-05-08T05:50:51.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.113+0000 7fb549eaf640 1 -- 172.21.15.45:0/843786744 reap_dead start 2024-05-08T05:50:51.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.113+0000 7fb54b912640 1 -- 172.21.15.45:0/843786744 shutdown_connections 2024-05-08T05:50:51.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.113+0000 7fb54b912640 1 -- 172.21.15.45:0/843786744 >> 172.21.15.45:0/843786744 conn(0x7fb5440d4fe0 msgr2=0x7fb5440d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:51.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.113+0000 7fb54b912640 1 -- 172.21.15.45:0/843786744 shutdown_connections 2024-05-08T05:50:51.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.113+0000 7fb54b912640 1 -- 172.21.15.45:0/843786744 wait complete. 2024-05-08T05:50:51.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.114+0000 7fb54b912640 1 Processor -- start 2024-05-08T05:50:51.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.114+0000 7fb54b912640 1 -- start start 2024-05-08T05:50:51.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.115+0000 7fb54b912640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb530004d40 0x7fb544155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.115+0000 7fb54b912640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5440cb6d0 0x7fb5440cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.115+0000 7fb54b912640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5440cc5f0 0x7fb5440c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.115+0000 7fb54b912640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb5440d06e0 con 0x7fb5440cc5f0 2024-05-08T05:50:51.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.115+0000 7fb54b912640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb5440d1010 con 0x7fb5440cb6d0 2024-05-08T05:50:51.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.115+0000 7fb54b912640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb5440b3430 con 0x7fb530004d40 2024-05-08T05:50:51.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.115+0000 7fb549eaf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5440cc5f0 0x7fb5440c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.115+0000 7fb549eaf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5440cc5f0 0x7fb5440c8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54898/0 (socket says 172.21.15.45:54898) 2024-05-08T05:50:51.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.115+0000 7fb549eaf640 1 -- 172.21.15.45:0/3835387355 learned_addr learned my addr 172.21.15.45:0/3835387355 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:51.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.116+0000 7fb548ead640 1 --2- 172.21.15.45:0/3835387355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5440cb6d0 0x7fb5440cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.116+0000 7fb5496ae640 1 --2- 172.21.15.45:0/3835387355 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb530004d40 0x7fb544155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.116+0000 7fb549eaf640 1 -- 172.21.15.45:0/3835387355 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb530004d40 msgr2=0x7fb544155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.116+0000 7fb549eaf640 1 --2- 172.21.15.45:0/3835387355 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb530004d40 0x7fb544155350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.116+0000 7fb549eaf640 1 -- 172.21.15.45:0/3835387355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5440cb6d0 msgr2=0x7fb5440cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.116+0000 7fb549eaf640 1 --2- 172.21.15.45:0/3835387355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5440cb6d0 0x7fb5440cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.116+0000 7fb549eaf640 1 -- 172.21.15.45:0/3835387355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5440ce820 con 0x7fb5440cc5f0 2024-05-08T05:50:51.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.116+0000 7fb548ead640 1 --2- 172.21.15.45:0/3835387355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5440cb6d0 0x7fb5440cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:51.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.116+0000 7fb549eaf640 1 --2- 172.21.15.45:0/3835387355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5440cc5f0 0x7fb5440c8f90 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb53c02aa30 tx=0x7fb53c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:51.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.116+0000 7fb539ffb640 1 -- 172.21.15.45:0/3835387355 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c007d90 con 0x7fb5440cc5f0 2024-05-08T05:50:51.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.116+0000 7fb539ffb640 1 -- 172.21.15.45:0/3835387355 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb53c040070 con 0x7fb5440cc5f0 2024-05-08T05:50:51.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.116+0000 7fb54b912640 1 -- 172.21.15.45:0/3835387355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5440ceac0 con 0x7fb5440cc5f0 2024-05-08T05:50:51.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.116+0000 7fb539ffb640 1 -- 172.21.15.45:0/3835387355 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c005520 con 0x7fb5440cc5f0 2024-05-08T05:50:51.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.116+0000 7fb54b912640 1 -- 172.21.15.45:0/3835387355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5440cf080 con 0x7fb5440cc5f0 2024-05-08T05:50:51.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.117+0000 7fb54b912640 1 -- 172.21.15.45:0/3835387355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5440693f0 con 0x7fb5440cc5f0 2024-05-08T05:50:51.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.118+0000 7fb539ffb640 1 -- 172.21.15.45:0/3835387355 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb53c039070 con 0x7fb5440cc5f0 2024-05-08T05:50:51.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.118+0000 7fb539ffb640 1 --2- 172.21.15.45:0/3835387355 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb5280762d0 0x7fb528078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.118+0000 7fb539ffb640 1 -- 172.21.15.45:0/3835387355 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5421+0+0 (secure 0 0 0) 0x7fb53c0b70e0 con 0x7fb5440cc5f0 2024-05-08T05:50:51.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.119+0000 7fb5496ae640 1 --2- 172.21.15.45:0/3835387355 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb5280762d0 0x7fb528078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.119+0000 7fb5496ae640 1 --2- 172.21.15.45:0/3835387355 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb5280762d0 0x7fb528078790 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb540001050 tx=0x7fb540009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:51.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.123+0000 7fb539ffb640 1 -- 172.21.15.45:0/3835387355 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb53c080b60 con 0x7fb5440cc5f0 2024-05-08T05:50:51.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.315+0000 7fb54b912640 1 -- 172.21.15.45:0/3835387355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb5440b79a0 con 0x7fb5440cc5f0 2024-05-08T05:50:51.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.315+0000 7fb539ffb640 1 -- 172.21.15.45:0/3835387355 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7fb53c080460 con 0x7fb5440cc5f0 2024-05-08T05:50:51.315 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-05-08T05:50:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.317+0000 7fb54b912640 1 -- 172.21.15.45:0/3835387355 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb5280762d0 msgr2=0x7fb528078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.317+0000 7fb54b912640 1 --2- 172.21.15.45:0/3835387355 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb5280762d0 0x7fb528078790 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb540001050 tx=0x7fb540009040 comp rx=0 tx=0).stop 2024-05-08T05:50:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.317+0000 7fb54b912640 1 -- 172.21.15.45:0/3835387355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5440cc5f0 msgr2=0x7fb5440c8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.317+0000 7fb54b912640 1 --2- 172.21.15.45:0/3835387355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5440cc5f0 0x7fb5440c8f90 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb53c02aa30 tx=0x7fb53c001670 comp rx=0 tx=0).stop 2024-05-08T05:50:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.317+0000 7fb54b912640 1 -- 172.21.15.45:0/3835387355 shutdown_connections 2024-05-08T05:50:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.317+0000 7fb54b912640 1 --2- 172.21.15.45:0/3835387355 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb5280762d0 0x7fb528078790 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.317+0000 7fb54b912640 1 --2- 172.21.15.45:0/3835387355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb5440cc5f0 0x7fb5440c8f90 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.317+0000 7fb54b912640 1 --2- 172.21.15.45:0/3835387355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5440cb6d0 0x7fb5440cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.317+0000 7fb54b912640 1 --2- 172.21.15.45:0/3835387355 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb530004d40 0x7fb544155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.317+0000 7fb54b912640 1 -- 172.21.15.45:0/3835387355 >> 172.21.15.45:0/3835387355 conn(0x7fb5440d4fe0 msgr2=0x7fb5440cfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.317+0000 7fb54b912640 1 -- 172.21.15.45:0/3835387355 shutdown_connections 2024-05-08T05:50:51.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.317+0000 7fb54b912640 1 -- 172.21.15.45:0/3835387355 wait complete. 2024-05-08T05:50:51.328 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-05-08T05:50:51.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.433+0000 7f5c60980640 1 Processor -- start 2024-05-08T05:50:51.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.433+0000 7f5c60980640 1 -- start start 2024-05-08T05:50:51.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.434+0000 7f5c60980640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b2490 0x7f5c5c070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.434+0000 7f5c60980640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5c5c070d20 0x7f5c5c0711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.434+0000 7f5c60980640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c5c06e110 con 0x7f5c5c1629b0 2024-05-08T05:50:51.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.434+0000 7f5c60980640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f5c5c06dec0 con 0x7f5c5c070d20 2024-05-08T05:50:51.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.434+0000 7f5c60980640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5c5c06e3f0 con 0x7f5c5c0b2490 2024-05-08T05:50:51.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.434+0000 7f5c51d9b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b2490 0x7f5c5c070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.434+0000 7f5c5ad9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5c5c070d20 0x7f5c5c0711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.434+0000 7f5c5a59c640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f5c5c1629b0 0x7f5c5c06eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:39284/0 (socket says 172.21.15.45:39284) 2024-05-08T05:50:51.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.434+0000 7f5c5a59c640 1 -- 172.21.15.45:0/2278085432 learned_addr learned my addr 172.21.15.45:0/2278085432 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:50:51.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.434+0000 7f5c59d9b640 1 -- 172.21.15.45:0/2278085432 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3350719681 0 0) 0x7f5c5c06e110 con 0x7f5c5c1629b0 2024-05-08T05:50:51.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.434+0000 7f5c59d9b640 1 -- 172.21.15.45:0/2278085432 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c48003520 con 0x7f5c5c1629b0 2024-05-08T05:50:51.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.434+0000 7f5c51d9b640 1 -- 172.21.15.45:0/2278085432 >> v2:172.21.15.63:3300/0 conn(0x7f5c5c070d20 msgr2=0x7f5c5c0711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:51.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.434+0000 7f5c51d9b640 1 --2- 172.21.15.45:0/2278085432 >> v2:172.21.15.63:3300/0 conn(0x7f5c5c070d20 0x7f5c5c0711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.434+0000 7f5c51d9b640 1 -- 172.21.15.45:0/2278085432 >> v1:172.21.15.63:6789/0 conn(0x7f5c5c1629b0 legacy=0x7f5c5c06eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.434+0000 7f5c51d9b640 1 -- 172.21.15.45:0/2278085432 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c06e6d0 con 0x7f5c5c0b2490 2024-05-08T05:50:51.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.434+0000 7f5c5ad9d640 1 --2- 172.21.15.45:0/2278085432 >> v2:172.21.15.63:3300/0 conn(0x7f5c5c070d20 0x7f5c5c0711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:50:51.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.435+0000 7f5c51d9b640 1 --2- 172.21.15.45:0/2278085432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b2490 0x7f5c5c070330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5c440097a0 tx=0x7f5c4402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4720f1ca6b41d398 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:51.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.435+0000 7f5c59d9b640 1 -- 172.21.15.45:0/2278085432 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44037070 con 0x7f5c5c0b2490 2024-05-08T05:50:51.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.435+0000 7f5c59d9b640 1 -- 172.21.15.45:0/2278085432 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c44032440 con 0x7f5c5c0b2490 2024-05-08T05:50:51.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.435+0000 7f5c59d9b640 1 -- 172.21.15.45:0/2278085432 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c440327f0 con 0x7f5c5c0b2490 2024-05-08T05:50:51.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.436+0000 7f5c60980640 1 -- 172.21.15.45:0/2278085432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b2490 msgr2=0x7f5c5c070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.436+0000 7f5c60980640 1 --2- 172.21.15.45:0/2278085432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b2490 0x7f5c5c070330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5c440097a0 tx=0x7f5c4402a180 comp rx=0 tx=0).stop 2024-05-08T05:50:51.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.436+0000 7f5c60980640 1 -- 172.21.15.45:0/2278085432 shutdown_connections 2024-05-08T05:50:51.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.436+0000 7f5c60980640 1 --2- 172.21.15.45:0/2278085432 >> v2:172.21.15.63:3300/0 conn(0x7f5c5c070d20 0x7f5c5c0711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.436+0000 7f5c60980640 1 --2- 172.21.15.45:0/2278085432 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b2490 0x7f5c5c070330 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.436+0000 7f5c60980640 1 -- 172.21.15.45:0/2278085432 >> 172.21.15.45:0/2278085432 conn(0x7f5c5c0b49d0 msgr2=0x7f5c5c0b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:51.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.436+0000 7f5c60980640 1 -- 172.21.15.45:0/2278085432 shutdown_connections 2024-05-08T05:50:51.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.436+0000 7f5c60980640 1 -- 172.21.15.45:0/2278085432 wait complete. 2024-05-08T05:50:51.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.437+0000 7f5c60980640 1 Processor -- start 2024-05-08T05:50:51.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.437+0000 7f5c60980640 1 -- start start 2024-05-08T05:50:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.438+0000 7f5c60980640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c5c070d20 0x7f5c5c0cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.438+0000 7f5c60980640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b2490 0x7f5c5c0ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.438+0000 7f5c60980640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c5c1629b0 0x7f5c5c15be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.438+0000 7f5c60980640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c0709f0 con 0x7f5c5c070d20 2024-05-08T05:50:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.438+0000 7f5c60980640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c070870 con 0x7f5c5c1629b0 2024-05-08T05:50:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.438+0000 7f5c60980640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5c5c0afcb0 con 0x7f5c5c0b2490 2024-05-08T05:50:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.438+0000 7f5c5a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c5c070d20 0x7f5c5c0cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.438+0000 7f5c5a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c5c070d20 0x7f5c5c0cdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54902/0 (socket says 172.21.15.45:54902) 2024-05-08T05:50:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.438+0000 7f5c5a59c640 1 -- 172.21.15.45:0/3270487712 learned_addr learned my addr 172.21.15.45:0/3270487712 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.438+0000 7f5c51d9b640 1 --2- 172.21.15.45:0/3270487712 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b2490 0x7f5c5c0ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.438+0000 7f5c5ad9d640 1 --2- 172.21.15.45:0/3270487712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c5c1629b0 0x7f5c5c15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.438+0000 7f5c5a59c640 1 -- 172.21.15.45:0/3270487712 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b2490 msgr2=0x7f5c5c0ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.438+0000 7f5c5a59c640 1 --2- 172.21.15.45:0/3270487712 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b2490 0x7f5c5c0ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.438+0000 7f5c5a59c640 1 -- 172.21.15.45:0/3270487712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c5c1629b0 msgr2=0x7f5c5c15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.438+0000 7f5c5a59c640 1 --2- 172.21.15.45:0/3270487712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c5c1629b0 0x7f5c5c15be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.439+0000 7f5c5a59c640 1 -- 172.21.15.45:0/3270487712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c15dd10 con 0x7f5c5c070d20 2024-05-08T05:50:51.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.439+0000 7f5c5a59c640 1 --2- 172.21.15.45:0/3270487712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c5c070d20 0x7f5c5c0cdfa0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00c410 tx=0x7f5c4c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:51.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.439+0000 7f5c537fe640 1 -- 172.21.15.45:0/3270487712 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c017070 con 0x7f5c5c070d20 2024-05-08T05:50:51.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.439+0000 7f5c537fe640 1 -- 172.21.15.45:0/3270487712 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4c005ce0 con 0x7f5c5c070d20 2024-05-08T05:50:51.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.439+0000 7f5c60980640 1 -- 172.21.15.45:0/3270487712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5c15dfb0 con 0x7f5c5c070d20 2024-05-08T05:50:51.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.439+0000 7f5c537fe640 1 -- 172.21.15.45:0/3270487712 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c0072d0 con 0x7f5c5c070d20 2024-05-08T05:50:51.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.439+0000 7f5c60980640 1 -- 172.21.15.45:0/3270487712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c5c0b5f80 con 0x7f5c5c070d20 2024-05-08T05:50:51.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.440+0000 7f5c60980640 1 -- 172.21.15.45:0/3270487712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c5c0b5ac0 con 0x7f5c5c070d20 2024-05-08T05:50:51.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.440+0000 7f5c537fe640 1 -- 172.21.15.45:0/3270487712 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5c4c01a070 con 0x7f5c5c070d20 2024-05-08T05:50:51.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.441+0000 7f5c537fe640 1 --2- 172.21.15.45:0/3270487712 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5c3c0762d0 0x7f5c3c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.441+0000 7f5c537fe640 1 -- 172.21.15.45:0/3270487712 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5421+0+0 (secure 0 0 0) 0x7f5c4c096720 con 0x7f5c5c070d20 2024-05-08T05:50:51.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.441+0000 7f5c51d9b640 1 --2- 172.21.15.45:0/3270487712 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5c3c0762d0 0x7f5c3c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.442+0000 7f5c51d9b640 1 --2- 172.21.15.45:0/3270487712 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5c3c0762d0 0x7f5c3c078790 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5c44034000 tx=0x7f5c4402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:51.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.445+0000 7f5c537fe640 1 -- 172.21.15.45:0/3270487712 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5c5c0b5ac0 con 0x7f5c5c070d20 2024-05-08T05:50:51.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:51.710+0000 7f5c60980640 1 -- 172.21.15.45:0/3270487712 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f5c5c15e180 con 0x7f5c5c070d20 2024-05-08T05:50:52.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.664+0000 7f5c537fe640 1 -- 172.21.15.45:0/3270487712 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v426) ==== 147+0+0 (secure 0 0 0) 0x7f5c4c05fbe0 con 0x7f5c5c070d20 2024-05-08T05:50:52.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.668+0000 7f5c60980640 1 -- 172.21.15.45:0/3270487712 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5c3c0762d0 msgr2=0x7f5c3c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:52.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.668+0000 7f5c60980640 1 --2- 172.21.15.45:0/3270487712 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5c3c0762d0 0x7f5c3c078790 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5c44034000 tx=0x7f5c4402e000 comp rx=0 tx=0).stop 2024-05-08T05:50:52.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.668+0000 7f5c60980640 1 -- 172.21.15.45:0/3270487712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c5c070d20 msgr2=0x7f5c5c0cdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:52.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.668+0000 7f5c60980640 1 --2- 172.21.15.45:0/3270487712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c5c070d20 0x7f5c5c0cdfa0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00c410 tx=0x7f5c4c00c8d0 comp rx=0 tx=0).stop 2024-05-08T05:50:52.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.668+0000 7f5c60980640 1 -- 172.21.15.45:0/3270487712 shutdown_connections 2024-05-08T05:50:52.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.668+0000 7f5c60980640 1 --2- 172.21.15.45:0/3270487712 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f5c3c0762d0 0x7f5c3c078790 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:52.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.669+0000 7f5c60980640 1 --2- 172.21.15.45:0/3270487712 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5c5c0b2490 0x7f5c5c0ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:52.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.669+0000 7f5c60980640 1 --2- 172.21.15.45:0/3270487712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5c5c1629b0 0x7f5c5c15be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:52.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.669+0000 7f5c60980640 1 --2- 172.21.15.45:0/3270487712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c5c070d20 0x7f5c5c0cdfa0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:52.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.669+0000 7f5c60980640 1 -- 172.21.15.45:0/3270487712 >> 172.21.15.45:0/3270487712 conn(0x7f5c5c0b49d0 msgr2=0x7f5c5c1616f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:52.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.669+0000 7f5c60980640 1 -- 172.21.15.45:0/3270487712 shutdown_connections 2024-05-08T05:50:52.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.669+0000 7f5c60980640 1 -- 172.21.15.45:0/3270487712 wait complete. 2024-05-08T05:50:52.683 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-08T05:50:52.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.797+0000 7efc221e5640 1 Processor -- start 2024-05-08T05:50:52.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.797+0000 7efc221e5640 1 -- start start 2024-05-08T05:50:52.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.798+0000 7efc221e5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efc1c06cab0 0x7efc1c0b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:52.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.798+0000 7efc221e5640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc1c1604b0 con 0x7efc1c1638e0 2024-05-08T05:50:52.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.798+0000 7efc221e5640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc1c0b14e0 con 0x7efc1c15f0c0 2024-05-08T05:50:52.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.798+0000 7efc221e5640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7efc1c163450 con 0x7efc1c06cab0 2024-05-08T05:50:52.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.798+0000 7efc1bfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efc1c06cab0 0x7efc1c0b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:52.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.798+0000 7efc13fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7efc1c1638e0 0x7efc1c06c3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35134/0 (socket says 172.21.15.45:35134) 2024-05-08T05:50:52.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.798+0000 7efc1bfff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7efc1c06cab0 0x7efc1c0b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54914/0 (socket says 172.21.15.45:54914) 2024-05-08T05:50:52.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.798+0000 7efc13fff640 1 -- 172.21.15.45:0/2067129253 learned_addr learned my addr 172.21.15.45:0/2067129253 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:50:52.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.798+0000 7efc1bfff640 1 -- 172.21.15.45:0/2067129253 >> v1:172.21.15.63:6789/0 conn(0x7efc1c15f0c0 legacy=0x7efc1c0b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:52.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.799+0000 7efc1bfff640 1 -- 172.21.15.45:0/2067129253 >> v1:172.21.15.45:6789/0 conn(0x7efc1c1638e0 legacy=0x7efc1c06c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:52.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.799+0000 7efc1bfff640 1 -- 172.21.15.45:0/2067129253 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc1c162e00 con 0x7efc1c06cab0 2024-05-08T05:50:52.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.799+0000 7efc1affd640 1 -- 172.21.15.45:0/2067129253 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 797058341 0 0) 0x7efc1c1604b0 con 0x7efc1c1638e0 2024-05-08T05:50:52.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.799+0000 7efc1bfff640 1 --2- 172.21.15.45:0/2067129253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc1c06cab0 0x7efc1c0b6f40 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7efc14009d90 tx=0x7efc1402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f360020e7ad1cf78 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:52.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.799+0000 7efc1affd640 1 -- 172.21.15.45:0/2067129253 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14007740 con 0x7efc1c06cab0 2024-05-08T05:50:52.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.799+0000 7efc1affd640 1 -- 172.21.15.45:0/2067129253 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc140078e0 con 0x7efc1c06cab0 2024-05-08T05:50:52.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.799+0000 7efc1affd640 1 -- 172.21.15.45:0/2067129253 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14007cb0 con 0x7efc1c06cab0 2024-05-08T05:50:52.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.800+0000 7efc221e5640 1 -- 172.21.15.45:0/2067129253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc1c06cab0 msgr2=0x7efc1c0b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:52.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.800+0000 7efc221e5640 1 --2- 172.21.15.45:0/2067129253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc1c06cab0 0x7efc1c0b6f40 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7efc14009d90 tx=0x7efc1402a6f0 comp rx=0 tx=0).stop 2024-05-08T05:50:52.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.800+0000 7efc221e5640 1 -- 172.21.15.45:0/2067129253 shutdown_connections 2024-05-08T05:50:52.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.800+0000 7efc221e5640 1 --2- 172.21.15.45:0/2067129253 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc1c06cab0 0x7efc1c0b6f40 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:52.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.800+0000 7efc221e5640 1 -- 172.21.15.45:0/2067129253 >> 172.21.15.45:0/2067129253 conn(0x7efc1c0b3b10 msgr2=0x7efc1c0b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:52.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.800+0000 7efc221e5640 1 -- 172.21.15.45:0/2067129253 shutdown_connections 2024-05-08T05:50:52.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.800+0000 7efc221e5640 1 -- 172.21.15.45:0/2067129253 wait complete. 2024-05-08T05:50:52.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.801+0000 7efc221e5640 1 Processor -- start 2024-05-08T05:50:52.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.801+0000 7efc221e5640 1 -- start start 2024-05-08T05:50:52.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.802+0000 7efc221e5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc1c06cab0 0x7efc1c14cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:52.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.802+0000 7efc221e5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc1c15f0c0 0x7efc1c0af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.802+0000 7efc221e5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc1c1638e0 0x7efc1c0afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.802+0000 7efc221e5640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c169ee0 con 0x7efc1c1638e0 2024-05-08T05:50:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.802+0000 7efc221e5640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c0b17c0 con 0x7efc1c15f0c0 2024-05-08T05:50:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.802+0000 7efc221e5640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7efc1c16a060 con 0x7efc1c06cab0 2024-05-08T05:50:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.802+0000 7efc1bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc1c1638e0 0x7efc1c0afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.802+0000 7efc1bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc1c1638e0 0x7efc1c0afa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54928/0 (socket says 172.21.15.45:54928) 2024-05-08T05:50:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.802+0000 7efc1bfff640 1 -- 172.21.15.45:0/2024231199 learned_addr learned my addr 172.21.15.45:0/2024231199 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.802+0000 7efc13fff640 1 --2- 172.21.15.45:0/2024231199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc1c15f0c0 0x7efc1c0af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.802+0000 7efc1b7fe640 1 --2- 172.21.15.45:0/2024231199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc1c06cab0 0x7efc1c14cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.802+0000 7efc1bfff640 1 -- 172.21.15.45:0/2024231199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc1c06cab0 msgr2=0x7efc1c14cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.802+0000 7efc1bfff640 1 --2- 172.21.15.45:0/2024231199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc1c06cab0 0x7efc1c14cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.802+0000 7efc1bfff640 1 -- 172.21.15.45:0/2024231199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc1c15f0c0 msgr2=0x7efc1c0af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.802+0000 7efc1bfff640 1 --2- 172.21.15.45:0/2024231199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc1c15f0c0 0x7efc1c0af4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:52.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.802+0000 7efc1bfff640 1 -- 172.21.15.45:0/2024231199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc1c159a20 con 0x7efc1c1638e0 2024-05-08T05:50:52.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.802+0000 7efc13fff640 1 --2- 172.21.15.45:0/2024231199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc1c15f0c0 0x7efc1c0af4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:52.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.803+0000 7efc1bfff640 1 --2- 172.21.15.45:0/2024231199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc1c1638e0 0x7efc1c0afa20 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7efc14009d90 tx=0x7efc14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:52.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.803+0000 7efc18ff9640 1 -- 172.21.15.45:0/2024231199 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc14040070 con 0x7efc1c1638e0 2024-05-08T05:50:52.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.803+0000 7efc18ff9640 1 -- 172.21.15.45:0/2024231199 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc14031c90 con 0x7efc1c1638e0 2024-05-08T05:50:52.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.803+0000 7efc18ff9640 1 -- 172.21.15.45:0/2024231199 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1403c640 con 0x7efc1c1638e0 2024-05-08T05:50:52.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.803+0000 7efc221e5640 1 -- 172.21.15.45:0/2024231199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc1c1589c0 con 0x7efc1c1638e0 2024-05-08T05:50:52.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.803+0000 7efc221e5640 1 -- 172.21.15.45:0/2024231199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc1c159220 con 0x7efc1c1638e0 2024-05-08T05:50:52.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.804+0000 7efc221e5640 1 -- 172.21.15.45:0/2024231199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc1c06a8f0 con 0x7efc1c1638e0 2024-05-08T05:50:52.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.804+0000 7efc18ff9640 1 -- 172.21.15.45:0/2024231199 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efc14039040 con 0x7efc1c1638e0 2024-05-08T05:50:52.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.805+0000 7efc18ff9640 1 --2- 172.21.15.45:0/2024231199 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efbfc0762d0 0x7efbfc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:52.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.805+0000 7efc18ff9640 1 -- 172.21.15.45:0/2024231199 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5421+0+0 (secure 0 0 0) 0x7efc140b8a80 con 0x7efc1c1638e0 2024-05-08T05:50:52.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.805+0000 7efc1b7fe640 1 --2- 172.21.15.45:0/2024231199 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efbfc0762d0 0x7efbfc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:52.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.806+0000 7efc1b7fe640 1 --2- 172.21.15.45:0/2024231199 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efbfc0762d0 0x7efbfc078790 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efc0c000f60 tx=0x7efc0c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:52.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:52.809+0000 7efc18ff9640 1 -- 172.21.15.45:0/2024231199 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efc14082650 con 0x7efc1c1638e0 2024-05-08T05:50:53.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.003+0000 7efc221e5640 1 -- 172.21.15.45:0/2024231199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7efc1c157440 con 0x7efc1c1638e0 2024-05-08T05:50:53.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.006+0000 7efc18ff9640 1 -- 172.21.15.45:0/2024231199 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7efc14081f50 con 0x7efc1c1638e0 2024-05-08T05:50:53.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.008+0000 7efc221e5640 1 -- 172.21.15.45:0/2024231199 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efbfc0762d0 msgr2=0x7efbfc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:53.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.008+0000 7efc221e5640 1 --2- 172.21.15.45:0/2024231199 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efbfc0762d0 0x7efbfc078790 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efc0c000f60 tx=0x7efc0c009040 comp rx=0 tx=0).stop 2024-05-08T05:50:53.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.008+0000 7efc221e5640 1 -- 172.21.15.45:0/2024231199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc1c1638e0 msgr2=0x7efc1c0afa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:53.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.008+0000 7efc221e5640 1 --2- 172.21.15.45:0/2024231199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc1c1638e0 0x7efc1c0afa20 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7efc14009d90 tx=0x7efc14004a40 comp rx=0 tx=0).stop 2024-05-08T05:50:53.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.008+0000 7efc221e5640 1 -- 172.21.15.45:0/2024231199 shutdown_connections 2024-05-08T05:50:53.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.008+0000 7efc221e5640 1 --2- 172.21.15.45:0/2024231199 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7efbfc0762d0 0x7efbfc078790 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:53.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.008+0000 7efc221e5640 1 --2- 172.21.15.45:0/2024231199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc1c1638e0 0x7efc1c0afa20 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:53.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.008+0000 7efc221e5640 1 --2- 172.21.15.45:0/2024231199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc1c15f0c0 0x7efc1c0af4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:53.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.008+0000 7efc221e5640 1 --2- 172.21.15.45:0/2024231199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7efc1c06cab0 0x7efc1c14cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:53.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.008+0000 7efc221e5640 1 -- 172.21.15.45:0/2024231199 >> 172.21.15.45:0/2024231199 conn(0x7efc1c0b3b10 msgr2=0x7efc1c15e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:53.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.008+0000 7efc221e5640 1 -- 172.21.15.45:0/2024231199 shutdown_connections 2024-05-08T05:50:53.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.008+0000 7efc221e5640 1 -- 172.21.15.45:0/2024231199 wait complete. 2024-05-08T05:50:53.020 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T05:50:53.020 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:50:53.058 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:50:53.111 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T05:50:53.111 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:50:53.111 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:50:53.129 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:50:53.129 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:50:53.129 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:50:53.149 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:50:53.149 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:50:53.149 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:50:53.184 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:50:53.185 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:50:53.185 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:50:53.207 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:50:53.207 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-08T05:50:53.207 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:50:53.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.329+0000 7fc9ba514640 1 Processor -- start 2024-05-08T05:50:53.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.329+0000 7fc9ba514640 1 -- start start 2024-05-08T05:50:53.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.330+0000 7fc9ba514640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc9b40d20c0 0x7fc9b4163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:53.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.330+0000 7fc9ba514640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9b40d1300 con 0x7fc9b4070d30 2024-05-08T05:50:53.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.330+0000 7fc9ba514640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9b40d15e0 con 0x7fc9b4071110 2024-05-08T05:50:53.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.330+0000 7fc9ba514640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc9b40d10d0 con 0x7fc9b40d20c0 2024-05-08T05:50:53.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.330+0000 7fc9b37fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc9b40d20c0 0x7fc9b4163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:53.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.330+0000 7fc9b3fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc9b4070d30 0x7fc9b40703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:35138/0 (socket says 172.21.15.45:35138) 2024-05-08T05:50:53.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.330+0000 7fc9b3fff640 1 -- 172.21.15.45:0/4014281572 learned_addr learned my addr 172.21.15.45:0/4014281572 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:50:53.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.330+0000 7fc9b37fe640 1 -- 172.21.15.45:0/4014281572 >> v1:172.21.15.63:6789/0 conn(0x7fc9b4071110 legacy=0x7fc9b406d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:53.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.330+0000 7fc9b37fe640 1 -- 172.21.15.45:0/4014281572 >> v1:172.21.15.45:6789/0 conn(0x7fc9b4070d30 legacy=0x7fc9b40703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:53.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.330+0000 7fc9b37fe640 1 -- 172.21.15.45:0/4014281572 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b40d18c0 con 0x7fc9b40d20c0 2024-05-08T05:50:53.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.331+0000 7fc9b37fe640 1 --2- 172.21.15.45:0/4014281572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9b40d20c0 0x7fc9b4163920 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80097a0 tx=0x7fc9a802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477b76cd577c8e2d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:53.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.331+0000 7fc9b2ffd640 1 -- 172.21.15.45:0/4014281572 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8037070 con 0x7fc9b40d20c0 2024-05-08T05:50:53.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.331+0000 7fc9b2ffd640 1 -- 172.21.15.45:0/4014281572 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9a8032440 con 0x7fc9b40d20c0 2024-05-08T05:50:53.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.331+0000 7fc9b2ffd640 1 -- 172.21.15.45:0/4014281572 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8032810 con 0x7fc9b40d20c0 2024-05-08T05:50:53.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.332+0000 7fc9ba514640 1 -- 172.21.15.45:0/4014281572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9b40d20c0 msgr2=0x7fc9b4163920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:53.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.332+0000 7fc9ba514640 1 --2- 172.21.15.45:0/4014281572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9b40d20c0 0x7fc9b4163920 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fc9a80097a0 tx=0x7fc9a802a230 comp rx=0 tx=0).stop 2024-05-08T05:50:53.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.332+0000 7fc9ba514640 1 -- 172.21.15.45:0/4014281572 shutdown_connections 2024-05-08T05:50:53.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.332+0000 7fc9ba514640 1 --2- 172.21.15.45:0/4014281572 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9b40d20c0 0x7fc9b4163920 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:53.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.332+0000 7fc9ba514640 1 -- 172.21.15.45:0/4014281572 >> 172.21.15.45:0/4014281572 conn(0x7fc9b40d2e90 msgr2=0x7fc9b40d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:53.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.332+0000 7fc9ba514640 1 -- 172.21.15.45:0/4014281572 shutdown_connections 2024-05-08T05:50:53.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.332+0000 7fc9ba514640 1 -- 172.21.15.45:0/4014281572 wait complete. 2024-05-08T05:50:53.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.333+0000 7fc9ba514640 1 Processor -- start 2024-05-08T05:50:53.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.333+0000 7fc9ba514640 1 -- start start 2024-05-08T05:50:53.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.334+0000 7fc9ba514640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9b4070d30 0x7fc9b40ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:53.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.334+0000 7fc9ba514640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9b4071110 0x7fc9b41f9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:53.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.334+0000 7fc9ba514640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9b40d20c0 0x7fc9b40b5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:53.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.334+0000 7fc9ba514640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b40d1c30 con 0x7fc9b4070d30 2024-05-08T05:50:53.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.334+0000 7fc9ba514640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b4070b80 con 0x7fc9b4071110 2024-05-08T05:50:53.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.334+0000 7fc9ba514640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc9b406dbc0 con 0x7fc9b40d20c0 2024-05-08T05:50:53.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.334+0000 7fc9b3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9b4070d30 0x7fc9b40ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:53.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.334+0000 7fc9b3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9b4070d30 0x7fc9b40ca420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54956/0 (socket says 172.21.15.45:54956) 2024-05-08T05:50:53.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.334+0000 7fc9b3fff640 1 -- 172.21.15.45:0/699711394 learned_addr learned my addr 172.21.15.45:0/699711394 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:53.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.334+0000 7fc9b37fe640 1 --2- 172.21.15.45:0/699711394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9b4071110 0x7fc9b41f9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:53.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.334+0000 7fc9b3fff640 1 -- 172.21.15.45:0/699711394 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9b40d20c0 msgr2=0x7fc9b40b5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:53.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.334+0000 7fc9b3fff640 1 --2- 172.21.15.45:0/699711394 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9b40d20c0 0x7fc9b40b5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:53.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.334+0000 7fc9b3fff640 1 -- 172.21.15.45:0/699711394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9b4071110 msgr2=0x7fc9b41f9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:53.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.334+0000 7fc9b3fff640 1 --2- 172.21.15.45:0/699711394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9b4071110 0x7fc9b41f9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:53.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.334+0000 7fc9b3fff640 1 -- 172.21.15.45:0/699711394 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b40caae0 con 0x7fc9b4070d30 2024-05-08T05:50:53.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.334+0000 7fc9b3fff640 1 --2- 172.21.15.45:0/699711394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9b4070d30 0x7fc9b40ca420 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400c500 tx=0x7fc9a400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:53.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.334+0000 7fc9b0ff9640 1 -- 172.21.15.45:0/699711394 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4017070 con 0x7fc9b4070d30 2024-05-08T05:50:53.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.335+0000 7fc9b0ff9640 1 -- 172.21.15.45:0/699711394 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9a4005ce0 con 0x7fc9b4070d30 2024-05-08T05:50:53.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.335+0000 7fc9ba514640 1 -- 172.21.15.45:0/699711394 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b40cadc0 con 0x7fc9b4070d30 2024-05-08T05:50:53.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.335+0000 7fc9b0ff9640 1 -- 172.21.15.45:0/699711394 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a40072f0 con 0x7fc9b4070d30 2024-05-08T05:50:53.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.335+0000 7fc9ba514640 1 -- 172.21.15.45:0/699711394 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b40cb380 con 0x7fc9b4070d30 2024-05-08T05:50:53.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.336+0000 7fc9ba514640 1 -- 172.21.15.45:0/699711394 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9b40693f0 con 0x7fc9b4070d30 2024-05-08T05:50:53.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.336+0000 7fc9b0ff9640 1 -- 172.21.15.45:0/699711394 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc9a401a070 con 0x7fc9b4070d30 2024-05-08T05:50:53.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.337+0000 7fc9b0ff9640 1 --2- 172.21.15.45:0/699711394 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc9a00762d0 0x7fc9a0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:53.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.337+0000 7fc9b0ff9640 1 -- 172.21.15.45:0/699711394 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5421+0+0 (secure 0 0 0) 0x7fc9a4096840 con 0x7fc9b4070d30 2024-05-08T05:50:53.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.337+0000 7fc9b37fe640 1 --2- 172.21.15.45:0/699711394 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc9a00762d0 0x7fc9a0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:53.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.338+0000 7fc9b37fe640 1 --2- 172.21.15.45:0/699711394 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc9a00762d0 0x7fc9a0078790 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8034000 tx=0x7fc9a802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:53.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.341+0000 7fc9b0ff9640 1 -- 172.21.15.45:0/699711394 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc9a405fc60 con 0x7fc9b4070d30 2024-05-08T05:50:53.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.574+0000 7fc9ba514640 1 -- 172.21.15.45:0/699711394 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9b4152f90 con 0x7fc9b4070d30 2024-05-08T05:50:53.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.574+0000 7fc9b0ff9640 1 -- 172.21.15.45:0/699711394 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+3869 (secure 0 0 0) 0x7fc9a4063f60 con 0x7fc9b4070d30 2024-05-08T05:50:53.575 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:50:53.575 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":92,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3692881261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3692881261},{"type":"v1","addr":"172.21.15.63:6836","nonce":3692881261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7497,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2916399818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2916399818},{"type":"v1","addr":"172.21.15.45:6833","nonce":2916399818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":7515,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/2978250004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":2978250004},{"type":"v1","addr":"172.21.15.63:6837","nonce":2978250004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7962,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4201508950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4201508950},{"type":"v1","addr":"172.21.15.45:6835","nonce":4201508950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[]} 2024-05-08T05:50:53.575 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 92 2024-05-08T05:50:53.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.577+0000 7fc9ba514640 1 -- 172.21.15.45:0/699711394 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc9a00762d0 msgr2=0x7fc9a0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:53.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.577+0000 7fc9ba514640 1 --2- 172.21.15.45:0/699711394 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc9a00762d0 0x7fc9a0078790 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8034000 tx=0x7fc9a802e000 comp rx=0 tx=0).stop 2024-05-08T05:50:53.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.577+0000 7fc9ba514640 1 -- 172.21.15.45:0/699711394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9b4070d30 msgr2=0x7fc9b40ca420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:53.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.577+0000 7fc9ba514640 1 --2- 172.21.15.45:0/699711394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9b4070d30 0x7fc9b40ca420 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fc9a400c500 tx=0x7fc9a400c9c0 comp rx=0 tx=0).stop 2024-05-08T05:50:53.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.577+0000 7fc9ba514640 1 -- 172.21.15.45:0/699711394 shutdown_connections 2024-05-08T05:50:53.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.577+0000 7fc9ba514640 1 --2- 172.21.15.45:0/699711394 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc9a00762d0 0x7fc9a0078790 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:53.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.577+0000 7fc9ba514640 1 --2- 172.21.15.45:0/699711394 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc9b40d20c0 0x7fc9b40b5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:53.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.577+0000 7fc9ba514640 1 --2- 172.21.15.45:0/699711394 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc9b4071110 0x7fc9b41f9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:53.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.577+0000 7fc9ba514640 1 --2- 172.21.15.45:0/699711394 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc9b4070d30 0x7fc9b40ca420 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:53.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.577+0000 7fc9ba514640 1 -- 172.21.15.45:0/699711394 >> 172.21.15.45:0/699711394 conn(0x7fc9b40d2e90 msgr2=0x7fc9b41e6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:53.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.577+0000 7fc9ba514640 1 -- 172.21.15.45:0/699711394 shutdown_connections 2024-05-08T05:50:53.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:53.577+0000 7fc9ba514640 1 -- 172.21.15.45:0/699711394 wait complete. 2024-05-08T05:50:53.588 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-08T05:50:53.588 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-08T05:50:53.588 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T05:50:53.589 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:50:53.589 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-08T05:50:53.589 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-08T05:50:53.589 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T05:50:53.589 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:50:53.589 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-08T05:50:53.589 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-08T05:50:53.590 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T05:50:53.590 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:50:53.590 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-08T05:50:53.590 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-08T05:50:53.590 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T05:50:53.590 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:50:53.598 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T05:50:53.598 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-08T05:50:53.602 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T05:50:53.602 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-08T05:50:53.603 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T05:50:53.603 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-08T05:50:53.641 INFO:tasks.ceph.mds.a:Started 2024-05-08T05:50:53.642 DEBUG:teuthology.parallel:result is None 2024-05-08T05:50:53.644 INFO:tasks.ceph.mds.b:Started 2024-05-08T05:50:53.644 DEBUG:teuthology.parallel:result is None 2024-05-08T05:50:53.645 INFO:tasks.ceph.mds.d:Started 2024-05-08T05:50:53.645 DEBUG:teuthology.parallel:result is None 2024-05-08T05:50:53.717 INFO:tasks.ceph.mds.a.smithi045.stdout:starting mds.a at 2024-05-08T05:50:53.731 INFO:tasks.ceph.mds.b.smithi063.stdout:starting mds.b at 2024-05-08T05:50:53.731 INFO:tasks.ceph.mds.d.smithi063.stdout:starting mds.d at 2024-05-08T05:50:58.569 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-05-08T05:50:59.598 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T05:50:59.598 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-08T05:50:59.602 INFO:tasks.ceph.mds.c:Started 2024-05-08T05:50:59.602 DEBUG:teuthology.parallel:result is None 2024-05-08T05:50:59.602 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-08T05:50:59.691 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-05-08T05:50:59.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.721+0000 7f2631a23640 1 Processor -- start 2024-05-08T05:50:59.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.721+0000 7f2631a23640 1 -- start start 2024-05-08T05:50:59.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.721+0000 7f2631a23640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f262c162880 0x7f262c0b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:59.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.721+0000 7f2631a23640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f262c0b60b0 0x7f262c073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.721+0000 7f2631a23640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f262c06c3a0 0x7f262c06c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.721+0000 7f2631a23640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f262c06cd50 con 0x7f262c06c3a0 2024-05-08T05:50:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.721+0000 7f2631a23640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f262c162c60 con 0x7f262c0b60b0 2024-05-08T05:50:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.721+0000 7f2631a23640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f262c0b5c20 con 0x7f262c162880 2024-05-08T05:50:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.722+0000 7f262b7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f262c06c3a0 0x7f262c06c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.722+0000 7f262b7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f262c06c3a0 0x7f262c06c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54988/0 (socket says 172.21.15.45:54988) 2024-05-08T05:50:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.722+0000 7f262b7fe640 1 -- 172.21.15.45:0/1332277407 learned_addr learned my addr 172.21.15.45:0/1332277407 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.722+0000 7f262affd640 1 --2- 172.21.15.45:0/1332277407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f262c162880 0x7f262c0b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.722+0000 7f262a7fc640 1 --2- 172.21.15.45:0/1332277407 >> v2:172.21.15.63:3300/0 conn(0x7f262c0b60b0 0x7f262c073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.722+0000 7f262b7fe640 1 -- 172.21.15.45:0/1332277407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f262c162880 msgr2=0x7f262c0b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.722+0000 7f262b7fe640 1 --2- 172.21.15.45:0/1332277407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f262c162880 0x7f262c0b10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.722+0000 7f262b7fe640 1 -- 172.21.15.45:0/1332277407 >> v2:172.21.15.63:3300/0 conn(0x7f262c0b60b0 msgr2=0x7f262c073ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.722+0000 7f262b7fe640 1 --2- 172.21.15.45:0/1332277407 >> v2:172.21.15.63:3300/0 conn(0x7f262c0b60b0 0x7f262c073ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:59.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.722+0000 7f262b7fe640 1 -- 172.21.15.45:0/1332277407 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262c0b5010 con 0x7f262c06c3a0 2024-05-08T05:50:59.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.722+0000 7f262a7fc640 1 --2- 172.21.15.45:0/1332277407 >> v2:172.21.15.63:3300/0 conn(0x7f262c0b60b0 0x7f262c073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:59.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.723+0000 7f262b7fe640 1 --2- 172.21.15.45:0/1332277407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f262c06c3a0 0x7f262c06c810 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f262c0b5da0 tx=0x7f262402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d4bda9d68febef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:59.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.723+0000 7f2629ffb640 1 -- 172.21.15.45:0/1332277407 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624036070 con 0x7f262c06c3a0 2024-05-08T05:50:59.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.723+0000 7f2629ffb640 1 -- 172.21.15.45:0/1332277407 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2624031600 con 0x7f262c06c3a0 2024-05-08T05:50:59.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.723+0000 7f2629ffb640 1 -- 172.21.15.45:0/1332277407 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26240319d0 con 0x7f262c06c3a0 2024-05-08T05:50:59.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.723+0000 7f2631a23640 1 -- 172.21.15.45:0/1332277407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f262c06c3a0 msgr2=0x7f262c06c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:59.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.723+0000 7f2631a23640 1 --2- 172.21.15.45:0/1332277407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f262c06c3a0 0x7f262c06c810 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f262c0b5da0 tx=0x7f262402a680 comp rx=0 tx=0).stop 2024-05-08T05:50:59.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.724+0000 7f2631a23640 1 -- 172.21.15.45:0/1332277407 shutdown_connections 2024-05-08T05:50:59.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.724+0000 7f2631a23640 1 --2- 172.21.15.45:0/1332277407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f262c06c3a0 0x7f262c06c810 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:59.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.724+0000 7f2631a23640 1 --2- 172.21.15.45:0/1332277407 >> v2:172.21.15.63:3300/0 conn(0x7f262c0b60b0 0x7f262c073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:59.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.724+0000 7f2631a23640 1 --2- 172.21.15.45:0/1332277407 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f262c162880 0x7f262c0b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:59.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.724+0000 7f2631a23640 1 -- 172.21.15.45:0/1332277407 >> 172.21.15.45:0/1332277407 conn(0x7f262c0b39a0 msgr2=0x7f262c0b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:59.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.724+0000 7f2631a23640 1 -- 172.21.15.45:0/1332277407 shutdown_connections 2024-05-08T05:50:59.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.724+0000 7f2631a23640 1 -- 172.21.15.45:0/1332277407 wait complete. 2024-05-08T05:50:59.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.725+0000 7f2631a23640 1 Processor -- start 2024-05-08T05:50:59.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.725+0000 7f2631a23640 1 -- start start 2024-05-08T05:50:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.726+0000 7f2631a23640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f262c06c3a0 0x7f262c1f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.726+0000 7f2631a23640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f262c0b60b0 0x7f262c159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.726+0000 7f2631a23640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f262c162880 0x7f262c1f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.726+0000 7f2631a23640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f262c0b55d0 con 0x7f262c0b60b0 2024-05-08T05:50:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.726+0000 7f2631a23640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f262c0740e0 con 0x7f262c162880 2024-05-08T05:50:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.726+0000 7f2631a23640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f262c0b58b0 con 0x7f262c06c3a0 2024-05-08T05:50:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.726+0000 7f262a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f262c0b60b0 0x7f262c159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.726+0000 7f262a7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f262c0b60b0 0x7f262c159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54998/0 (socket says 172.21.15.45:54998) 2024-05-08T05:50:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.726+0000 7f262a7fc640 1 -- 172.21.15.45:0/172144791 learned_addr learned my addr 172.21.15.45:0/172144791 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:50:59.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.726+0000 7f262b7fe640 1 --2- 172.21.15.45:0/172144791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f262c162880 0x7f262c1f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:59.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.726+0000 7f262affd640 1 --2- 172.21.15.45:0/172144791 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f262c06c3a0 0x7f262c1f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:59.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.726+0000 7f262a7fc640 1 -- 172.21.15.45:0/172144791 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f262c06c3a0 msgr2=0x7f262c1f8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:59.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.726+0000 7f262a7fc640 1 --2- 172.21.15.45:0/172144791 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f262c06c3a0 0x7f262c1f8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:59.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.726+0000 7f262a7fc640 1 -- 172.21.15.45:0/172144791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f262c162880 msgr2=0x7f262c1f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:59.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.726+0000 7f262a7fc640 1 --2- 172.21.15.45:0/172144791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f262c162880 0x7f262c1f4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:59.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.726+0000 7f262a7fc640 1 -- 172.21.15.45:0/172144791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262c1f56d0 con 0x7f262c0b60b0 2024-05-08T05:50:59.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.726+0000 7f262b7fe640 1 --2- 172.21.15.45:0/172144791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f262c162880 0x7f262c1f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:59.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.726+0000 7f262affd640 1 --2- 172.21.15.45:0/172144791 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f262c06c3a0 0x7f262c1f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:59.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.726+0000 7f262a7fc640 1 --2- 172.21.15.45:0/172144791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f262c0b60b0 0x7f262c159b30 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f261400c7c0 tx=0x7f261400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:59.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.727+0000 7f2623fff640 1 -- 172.21.15.45:0/172144791 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614015070 con 0x7f262c0b60b0 2024-05-08T05:50:59.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.727+0000 7f2623fff640 1 -- 172.21.15.45:0/172144791 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2614007a30 con 0x7f262c0b60b0 2024-05-08T05:50:59.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.727+0000 7f2623fff640 1 -- 172.21.15.45:0/172144791 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614007e00 con 0x7f262c0b60b0 2024-05-08T05:50:59.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.727+0000 7f2631a23640 1 -- 172.21.15.45:0/172144791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262c1f59b0 con 0x7f262c0b60b0 2024-05-08T05:50:59.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.727+0000 7f2631a23640 1 -- 172.21.15.45:0/172144791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f262c1f8450 con 0x7f262c0b60b0 2024-05-08T05:50:59.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.728+0000 7f2631a23640 1 -- 172.21.15.45:0/172144791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f262c069b00 con 0x7f262c0b60b0 2024-05-08T05:50:59.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.728+0000 7f2623fff640 1 -- 172.21.15.45:0/172144791 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2614018070 con 0x7f262c0b60b0 2024-05-08T05:50:59.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.729+0000 7f2623fff640 1 --2- 172.21.15.45:0/172144791 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f260c0762d0 0x7f260c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:59.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.729+0000 7f2623fff640 1 -- 172.21.15.45:0/172144791 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5421+0+0 (secure 0 0 0) 0x7f2614097470 con 0x7f262c0b60b0 2024-05-08T05:50:59.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.729+0000 7f262affd640 1 --2- 172.21.15.45:0/172144791 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f260c0762d0 0x7f260c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:59.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.730+0000 7f262affd640 1 --2- 172.21.15.45:0/172144791 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f260c0762d0 0x7f260c078790 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f261c006fd0 tx=0x7f261c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:59.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.733+0000 7f2623fff640 1 -- 172.21.15.45:0/172144791 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2614060ef0 con 0x7f262c0b60b0 2024-05-08T05:50:59.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:50:59.943+0000 7f2631a23640 1 -- 172.21.15.45:0/172144791 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f262c0c8d70 con 0x7f262c0b60b0 2024-05-08T05:51:01.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.027+0000 7f2623fff640 1 -- 172.21.15.45:0/172144791 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v115) ==== 97+0+0 (secure 0 0 0) 0x7f26140607f0 con 0x7f262c0b60b0 2024-05-08T05:51:01.027 INFO:teuthology.orchestra.run.smithi045.stderr: removed all blocklist entries 2024-05-08T05:51:01.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.030+0000 7f2631a23640 1 -- 172.21.15.45:0/172144791 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f260c0762d0 msgr2=0x7f260c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:01.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.030+0000 7f2631a23640 1 --2- 172.21.15.45:0/172144791 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f260c0762d0 0x7f260c078790 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f261c006fd0 tx=0x7f261c008040 comp rx=0 tx=0).stop 2024-05-08T05:51:01.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.030+0000 7f2631a23640 1 -- 172.21.15.45:0/172144791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f262c0b60b0 msgr2=0x7f262c159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:01.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.030+0000 7f2631a23640 1 --2- 172.21.15.45:0/172144791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f262c0b60b0 0x7f262c159b30 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f261400c7c0 tx=0x7f261400cc80 comp rx=0 tx=0).stop 2024-05-08T05:51:01.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.030+0000 7f2631a23640 1 -- 172.21.15.45:0/172144791 shutdown_connections 2024-05-08T05:51:01.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.030+0000 7f2631a23640 1 --2- 172.21.15.45:0/172144791 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f260c0762d0 0x7f260c078790 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.030+0000 7f2631a23640 1 --2- 172.21.15.45:0/172144791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f262c162880 0x7f262c1f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.030+0000 7f2631a23640 1 --2- 172.21.15.45:0/172144791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f262c0b60b0 0x7f262c159b30 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.030+0000 7f2631a23640 1 --2- 172.21.15.45:0/172144791 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f262c06c3a0 0x7f262c1f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.030+0000 7f2631a23640 1 -- 172.21.15.45:0/172144791 >> 172.21.15.45:0/172144791 conn(0x7f262c0b39a0 msgr2=0x7f262c0b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:01.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.030+0000 7f2631a23640 1 -- 172.21.15.45:0/172144791 shutdown_connections 2024-05-08T05:51:01.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.030+0000 7f2631a23640 1 -- 172.21.15.45:0/172144791 wait complete. 2024-05-08T05:51:01.045 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-08T05:51:01.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.155+0000 7f6ca8baf640 1 Processor -- start 2024-05-08T05:51:01.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.155+0000 7f6ca8baf640 1 -- start start 2024-05-08T05:51:01.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.156+0000 7f6ca8baf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ca4072840 0x7f6ca415c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.156+0000 7f6ca8baf640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ca415c920 con 0x7f6ca406e0f0 2024-05-08T05:51:01.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.156+0000 7f6ca8baf640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ca415cc00 con 0x7f6ca406e4d0 2024-05-08T05:51:01.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.156+0000 7f6ca8baf640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca415cee0 con 0x7f6ca4072840 2024-05-08T05:51:01.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.156+0000 7f6ca259c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f6ca406e0f0 0x7f6ca40d0f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52742/0 (socket says 172.21.15.45:52742) 2024-05-08T05:51:01.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.156+0000 7f6ca259c640 1 -- 172.21.15.45:0/2088501909 learned_addr learned my addr 172.21.15.45:0/2088501909 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca1d9b640 1 --2- 172.21.15.45:0/2088501909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ca4072840 0x7f6ca415c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca159a640 1 -- 172.21.15.45:0/2088501909 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1695247808 0 0) 0x7f6ca415c920 con 0x7f6ca406e0f0 2024-05-08T05:51:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca159a640 1 -- 172.21.15.45:0/2088501909 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c8c003520 con 0x7f6ca406e0f0 2024-05-08T05:51:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca159a640 1 -- 172.21.15.45:0/2088501909 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1492918078 0 0) 0x7f6ca415cc00 con 0x7f6ca406e4d0 2024-05-08T05:51:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca159a640 1 -- 172.21.15.45:0/2088501909 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ca415c920 con 0x7f6ca406e4d0 2024-05-08T05:51:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca159a640 1 -- 172.21.15.45:0/2088501909 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 781935135 0 0) 0x7f6c8c003520 con 0x7f6ca406e0f0 2024-05-08T05:51:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca159a640 1 -- 172.21.15.45:0/2088501909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ca4072840 msgr2=0x7f6ca415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca159a640 1 --2- 172.21.15.45:0/2088501909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ca4072840 0x7f6ca415c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca159a640 1 -- 172.21.15.45:0/2088501909 >> v1:172.21.15.63:6789/0 conn(0x7f6ca406e4d0 legacy=0x7f6ca40aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca159a640 1 -- 172.21.15.45:0/2088501909 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca415d1c0 con 0x7f6ca406e0f0 2024-05-08T05:51:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca159a640 1 -- 172.21.15.45:0/2088501909 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f6c940016f0 con 0x7f6ca406e0f0 2024-05-08T05:51:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca159a640 1 -- 172.21.15.45:0/2088501909 >> v1:172.21.15.45:6789/0 conn(0x7f6ca406e0f0 legacy=0x7f6ca40d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca159a640 1 --2- 172.21.15.45:0/2088501909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ca4072840 0x7f6c8c003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca159a640 1 --2- 172.21.15.45:0/2088501909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c8c004210 0x7f6c8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca159a640 1 --2- 172.21.15.45:0/2088501909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c004d90 0x7f6c8c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca159a640 1 -- 172.21.15.45:0/2088501909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca415cee0 con 0x7f6c8c004d90 2024-05-08T05:51:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca159a640 1 -- 172.21.15.45:0/2088501909 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c00bbd0 con 0x7f6c8c004210 2024-05-08T05:51:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca159a640 1 -- 172.21.15.45:0/2088501909 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6c8c00bdd0 con 0x7f6ca4072840 2024-05-08T05:51:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca2d9d640 1 --2- 172.21.15.45:0/2088501909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c004d90 0x7f6c8c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca1d9b640 1 --2- 172.21.15.45:0/2088501909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c8c004210 0x7f6c8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca259c640 1 --2- 172.21.15.45:0/2088501909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ca4072840 0x7f6c8c003c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca2d9d640 1 -- 172.21.15.45:0/2088501909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ca4072840 msgr2=0x7f6c8c003c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca2d9d640 1 --2- 172.21.15.45:0/2088501909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ca4072840 0x7f6c8c003c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca2d9d640 1 -- 172.21.15.45:0/2088501909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c8c004210 msgr2=0x7f6c8c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca2d9d640 1 --2- 172.21.15.45:0/2088501909 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6c8c004210 0x7f6c8c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca2d9d640 1 -- 172.21.15.45:0/2088501909 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8c003520 con 0x7f6c8c004d90 2024-05-08T05:51:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.157+0000 7f6ca259c640 1 --2- 172.21.15.45:0/2088501909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ca4072840 0x7f6c8c003c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.158+0000 7f6ca2d9d640 1 --2- 172.21.15.45:0/2088501909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c004d90 0x7f6c8c005210 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c001be0 tx=0x7f6c8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f444d8edd6ec4b7e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.158+0000 7f6ca159a640 1 -- 172.21.15.45:0/2088501909 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c001c20 con 0x7f6c8c004d90 2024-05-08T05:51:01.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.158+0000 7f6ca159a640 1 -- 172.21.15.45:0/2088501909 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c9c006d70 con 0x7f6c8c004d90 2024-05-08T05:51:01.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.158+0000 7f6ca159a640 1 -- 172.21.15.45:0/2088501909 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c003bb0 con 0x7f6c8c004d90 2024-05-08T05:51:01.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.159+0000 7f6ca8baf640 1 -- 172.21.15.45:0/2088501909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c004d90 msgr2=0x7f6c8c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:01.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.159+0000 7f6ca8baf640 1 --2- 172.21.15.45:0/2088501909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c004d90 0x7f6c8c005210 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c001be0 tx=0x7f6c8c005830 comp rx=0 tx=0).stop 2024-05-08T05:51:01.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.159+0000 7f6ca2d9d640 1 -- 172.21.15.45:0/2088501909 reap_dead start 2024-05-08T05:51:01.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.159+0000 7f6ca8baf640 1 -- 172.21.15.45:0/2088501909 shutdown_connections 2024-05-08T05:51:01.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.159+0000 7f6ca8baf640 1 -- 172.21.15.45:0/2088501909 >> 172.21.15.45:0/2088501909 conn(0x7f6ca406c3a0 msgr2=0x7f6ca406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:01.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.159+0000 7f6ca8baf640 1 -- 172.21.15.45:0/2088501909 shutdown_connections 2024-05-08T05:51:01.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.159+0000 7f6ca8baf640 1 -- 172.21.15.45:0/2088501909 wait complete. 2024-05-08T05:51:01.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.160+0000 7f6ca8baf640 1 Processor -- start 2024-05-08T05:51:01.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.160+0000 7f6ca8baf640 1 -- start start 2024-05-08T05:51:01.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.161+0000 7f6ca8baf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c004d90 0x7f6ca40b3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.161+0000 7f6ca8baf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ca40cfcd0 0x7f6ca41505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.161+0000 7f6ca8baf640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ca40cbf40 0x7f6ca4072c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.161+0000 7f6ca8baf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca40ae7d0 con 0x7f6c8c004d90 2024-05-08T05:51:01.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.161+0000 7f6ca8baf640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca40ae650 con 0x7f6ca40cbf40 2024-05-08T05:51:01.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.161+0000 7f6ca8baf640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca41530b0 con 0x7f6ca40cfcd0 2024-05-08T05:51:01.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.161+0000 7f6ca259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c004d90 0x7f6ca40b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.161+0000 7f6ca259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c004d90 0x7f6ca40b3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37120/0 (socket says 172.21.15.45:37120) 2024-05-08T05:51:01.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.161+0000 7f6ca259c640 1 -- 172.21.15.45:0/3265908453 learned_addr learned my addr 172.21.15.45:0/3265908453 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:01.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.161+0000 7f6ca1d9b640 1 --2- 172.21.15.45:0/3265908453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ca40cfcd0 0x7f6ca41505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.161+0000 7f6ca259c640 1 -- 172.21.15.45:0/3265908453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ca40cfcd0 msgr2=0x7f6ca41505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:01.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.161+0000 7f6ca2d9d640 1 --2- 172.21.15.45:0/3265908453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ca40cbf40 0x7f6ca4072c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.161+0000 7f6ca259c640 1 --2- 172.21.15.45:0/3265908453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ca40cfcd0 0x7f6ca41505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.161+0000 7f6ca259c640 1 -- 172.21.15.45:0/3265908453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ca40cbf40 msgr2=0x7f6ca4072c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:01.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.162+0000 7f6ca259c640 1 --2- 172.21.15.45:0/3265908453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ca40cbf40 0x7f6ca4072c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.162+0000 7f6ca259c640 1 -- 172.21.15.45:0/3265908453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca40cba60 con 0x7f6c8c004d90 2024-05-08T05:51:01.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.162+0000 7f6ca1d9b640 1 --2- 172.21.15.45:0/3265908453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ca40cfcd0 0x7f6ca41505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:01.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.162+0000 7f6ca2d9d640 1 --2- 172.21.15.45:0/3265908453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ca40cbf40 0x7f6ca4072c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:01.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.162+0000 7f6ca259c640 1 --2- 172.21.15.45:0/3265908453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c004d90 0x7f6ca40b3340 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400e790 tx=0x7f6c9400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:01.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.162+0000 7f6c92ffd640 1 -- 172.21.15.45:0/3265908453 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9400eea0 con 0x7f6c8c004d90 2024-05-08T05:51:01.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.162+0000 7f6c92ffd640 1 -- 172.21.15.45:0/3265908453 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c94018070 con 0x7f6c8c004d90 2024-05-08T05:51:01.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.162+0000 7f6ca8baf640 1 -- 172.21.15.45:0/3265908453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca40ce190 con 0x7f6c8c004d90 2024-05-08T05:51:01.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.162+0000 7f6c92ffd640 1 -- 172.21.15.45:0/3265908453 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9400ca90 con 0x7f6c8c004d90 2024-05-08T05:51:01.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.162+0000 7f6ca8baf640 1 -- 172.21.15.45:0/3265908453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca40ce750 con 0x7f6c8c004d90 2024-05-08T05:51:01.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.163+0000 7f6ca8baf640 1 -- 172.21.15.45:0/3265908453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca40693f0 con 0x7f6c8c004d90 2024-05-08T05:51:01.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.163+0000 7f6c92ffd640 1 -- 172.21.15.45:0/3265908453 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6c9400cc30 con 0x7f6c8c004d90 2024-05-08T05:51:01.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.164+0000 7f6c92ffd640 1 --2- 172.21.15.45:0/3265908453 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6c840762d0 0x7f6c84078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.164+0000 7f6c92ffd640 1 -- 172.21.15.45:0/3265908453 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5249+0+0 (secure 0 0 0) 0x7f6c94097180 con 0x7f6c8c004d90 2024-05-08T05:51:01.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.164+0000 7f6ca1d9b640 1 --2- 172.21.15.45:0/3265908453 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6c840762d0 0x7f6c84078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.165+0000 7f6ca1d9b640 1 --2- 172.21.15.45:0/3265908453 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6c840762d0 0x7f6c84078790 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6c980093d0 tx=0x7f6c98008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:01.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.168+0000 7f6c92ffd640 1 -- 172.21.15.45:0/3265908453 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6c94060d30 con 0x7f6c8c004d90 2024-05-08T05:51:01.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.433+0000 7f6ca8baf640 1 -- 172.21.15.45:0/3265908453 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f6ca4151740 con 0x7f6c8c004d90 2024-05-08T05:51:01.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.433+0000 7f6c92ffd640 1 -- 172.21.15.45:0/3265908453 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7f6c94060630 con 0x7f6c8c004d90 2024-05-08T05:51:01.434 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:01.434 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:51:01.434 INFO:teuthology.orchestra.run.smithi045.stdout: "auth_dump": [ 2024-05-08T05:51:01.434 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:51:01.434 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.a", 2024-05-08T05:51:01.434 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmrRgXHBAAX1mgdmOD9JC4S1vJNyLSQw==", 2024-05-08T05:51:01.434 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:51:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:51:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:51:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:51:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:51:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:51:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:51:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.b", 2024-05-08T05:51:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmvn8SKBAAVkXYBbTfu98gl0AptOfJEg==", 2024-05-08T05:51:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:51:01.435 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:51:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:51:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:51:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:51:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:51:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:51:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.c", 2024-05-08T05:51:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmeiJCIxAAr3Br1fJVgR9TW2Q+4tMQBQ==", 2024-05-08T05:51:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:51:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:51:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:51:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:51:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:51:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:51:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:51:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mds.d", 2024-05-08T05:51:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmXdFILxAAxteIxUYeFdQKd/oxHLpeOQ==", 2024-05-08T05:51:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:51:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:51:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:51:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:51:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:51:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:51:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:51:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.0", 2024-05-08T05:51:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBWEDtmLdHHGRAAA1o4Z/lTGERIZhulqVtybw==", 2024-05-08T05:51:01.437 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:51:01.438 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:51:01.438 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:51:01.438 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:51:01.438 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:51:01.438 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:51:01.438 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.1", 2024-05-08T05:51:01.438 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBZEDtmMrRVGBAAeKBtcf5fSQMFE6IStUvokQ==", 2024-05-08T05:51:01.438 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:51:01.438 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:51:01.438 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:51:01.438 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:51:01.438 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:51:01.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:51:01.439 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.2", 2024-05-08T05:51:01.439 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBcEDtmpozQFhAAS6AF6X592Wh/QiBipcOVew==", 2024-05-08T05:51:01.439 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:51:01.439 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:51:01.439 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:51:01.439 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:51:01.439 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:51:01.439 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:51:01.439 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.3", 2024-05-08T05:51:01.439 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBfEDtm47lSFRAAt5tvpRCljxGtAmHQqcFj2A==", 2024-05-08T05:51:01.439 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:51:01.439 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:51:01.440 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:51:01.440 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:51:01.440 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:51:01.440 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:51:01.440 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.4", 2024-05-08T05:51:01.440 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBlEDtmDQ7mGxAA5z7rHZRf7X5g7ZiqEoO2DA==", 2024-05-08T05:51:01.440 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:51:01.440 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:51:01.440 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:51:01.440 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:51:01.440 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:51:01.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:51:01.441 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.5", 2024-05-08T05:51:01.441 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBoEDtmBdi0GhAAVMfg+KEJHRvaiS49PEboOg==", 2024-05-08T05:51:01.441 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:51:01.441 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:51:01.441 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:51:01.441 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:51:01.441 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:51:01.441 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:51:01.441 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.6", 2024-05-08T05:51:01.441 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBrEDtmG2daGRAANQgJFAT6gNE/vPofCv6veA==", 2024-05-08T05:51:01.441 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:51:01.442 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:51:01.442 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:51:01.442 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:51:01.442 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:51:01.442 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:51:01.442 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "osd.7", 2024-05-08T05:51:01.442 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBuEDtm3tmDGRAARsAzm9KIzgKAs4ZtfCVdaQ==", 2024-05-08T05:51:01.442 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:51:01.442 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow profile osd", 2024-05-08T05:51:01.442 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile osd", 2024-05-08T05:51:01.442 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:51:01.443 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:51:01.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:51:01.443 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.0", 2024-05-08T05:51:01.443 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmDcqGMxAALgQaBYp4Zd8uMqswCXvGaw==", 2024-05-08T05:51:01.443 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:51:01.443 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:51:01.443 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow r", 2024-05-08T05:51:01.443 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-08T05:51:01.443 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:51:01.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:51:01.443 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.1", 2024-05-08T05:51:01.444 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBVEDtm6z0GARAARgRma5hPpH6X95pE5NNrQw==", 2024-05-08T05:51:01.444 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:51:01.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:51:01.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:51:01.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:51:01.444 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:51:01.444 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:51:01.444 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:51:01.444 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.2", 2024-05-08T05:51:01.444 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmAMtIORAAJf8oSwHqSzgjsTuqxtQqHQ==", 2024-05-08T05:51:01.444 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:51:01.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:51:01.444 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:51:01.445 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:51:01.445 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:51:01.445 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:51:01.445 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:51:01.445 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.3", 2024-05-08T05:51:01.445 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBVEDtm9qK1BhAAokvkJZmNxT5WER9/+EKIpA==", 2024-05-08T05:51:01.445 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:51:01.445 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow", 2024-05-08T05:51:01.445 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow r", 2024-05-08T05:51:01.445 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow rw", 2024-05-08T05:51:01.446 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow rwx" 2024-05-08T05:51:01.446 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:51:01.446 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:51:01.446 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "client.admin", 2024-05-08T05:51:01.446 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmZ4Q3ABAANcqmwOmt+dNJ5wOiPHHO4Q==", 2024-05-08T05:51:01.446 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:51:01.446 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:51:01.446 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr": "allow *", 2024-05-08T05:51:01.446 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow *", 2024-05-08T05:51:01.446 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:51:01.446 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:51:01.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:51:01.447 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.x", 2024-05-08T05:51:01.447 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmL0XLGBAA1BSFdyiDYV9Je+igOVBvlQ==", 2024-05-08T05:51:01.447 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:51:01.447 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:51:01.447 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-05-08T05:51:01.447 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:51:01.447 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-05-08T05:51:01.447 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:51:01.447 INFO:teuthology.orchestra.run.smithi045.stdout: "entity": "mgr.y", 2024-05-08T05:51:01.447 INFO:teuthology.orchestra.run.smithi045.stdout: "key": "AQBUEDtmfM5OFRAAgdJPUfqmnKU8vRrjGMNwrw==", 2024-05-08T05:51:01.448 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": { 2024-05-08T05:51:01.448 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": "allow *", 2024-05-08T05:51:01.448 INFO:teuthology.orchestra.run.smithi045.stdout: "mon": "allow profile mgr", 2024-05-08T05:51:01.448 INFO:teuthology.orchestra.run.smithi045.stdout: "osd": "allow *" 2024-05-08T05:51:01.448 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.448 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.448 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-05-08T05:51:01.448 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:51:01.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.435+0000 7f6ca8baf640 1 -- 172.21.15.45:0/3265908453 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6c840762d0 msgr2=0x7f6c84078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:01.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.435+0000 7f6ca8baf640 1 --2- 172.21.15.45:0/3265908453 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6c840762d0 0x7f6c84078790 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6c980093d0 tx=0x7f6c98008040 comp rx=0 tx=0).stop 2024-05-08T05:51:01.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.435+0000 7f6ca8baf640 1 -- 172.21.15.45:0/3265908453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c004d90 msgr2=0x7f6ca40b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:01.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.435+0000 7f6ca8baf640 1 --2- 172.21.15.45:0/3265908453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c004d90 0x7f6ca40b3340 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400e790 tx=0x7f6c9400ec50 comp rx=0 tx=0).stop 2024-05-08T05:51:01.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.435+0000 7f6ca8baf640 1 -- 172.21.15.45:0/3265908453 shutdown_connections 2024-05-08T05:51:01.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.435+0000 7f6ca8baf640 1 --2- 172.21.15.45:0/3265908453 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6c840762d0 0x7f6c84078790 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.435+0000 7f6ca8baf640 1 --2- 172.21.15.45:0/3265908453 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f6ca40cfcd0 0x7f6ca41505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.435+0000 7f6ca8baf640 1 --2- 172.21.15.45:0/3265908453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6ca40cbf40 0x7f6ca4072c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.435+0000 7f6ca8baf640 1 --2- 172.21.15.45:0/3265908453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8c004d90 0x7f6ca40b3340 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.435+0000 7f6ca8baf640 1 -- 172.21.15.45:0/3265908453 >> 172.21.15.45:0/3265908453 conn(0x7f6ca406c3a0 msgr2=0x7f6ca4151f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:01.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.436+0000 7f6ca8baf640 1 -- 172.21.15.45:0/3265908453 shutdown_connections 2024-05-08T05:51:01.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.436+0000 7f6ca8baf640 1 -- 172.21.15.45:0/3265908453 wait complete. 2024-05-08T05:51:01.451 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T05:51:01.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.555+0000 7f7b513c1640 1 Processor -- start 2024-05-08T05:51:01.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.555+0000 7f7b513c1640 1 -- start start 2024-05-08T05:51:01.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.556+0000 7f7b513c1640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7b4c15f2d0 0x7f7b4c0b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b513c1640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b4c0d2730 con 0x7f7b4c0d2310 2024-05-08T05:51:01.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b513c1640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b4c1606c0 con 0x7f7b4c0d4fc0 2024-05-08T05:51:01.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b513c1640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7b4c0b5810 con 0x7f7b4c15f2d0 2024-05-08T05:51:01.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b4b7fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f7b4c0d2310 0x7f7b4c0d2c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52748/0 (socket says 172.21.15.45:52748) 2024-05-08T05:51:01.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b4b7fe640 1 -- 172.21.15.45:0/944948066 learned_addr learned my addr 172.21.15.45:0/944948066 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:01.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b4affd640 1 --2- 172.21.15.45:0/944948066 >> v2:172.21.15.63:3300/0 conn(0x7f7b4c15f2d0 0x7f7b4c0b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b49ffb640 1 -- 172.21.15.45:0/944948066 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1757993321 0 0) 0x7f7b4c0d2730 con 0x7f7b4c0d2310 2024-05-08T05:51:01.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b49ffb640 1 -- 172.21.15.45:0/944948066 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b38003520 con 0x7f7b4c0d2310 2024-05-08T05:51:01.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b49ffb640 1 -- 172.21.15.45:0/944948066 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1179187597 0 0) 0x7f7b4c1606c0 con 0x7f7b4c0d4fc0 2024-05-08T05:51:01.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b49ffb640 1 -- 172.21.15.45:0/944948066 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b4c0d2730 con 0x7f7b4c0d4fc0 2024-05-08T05:51:01.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b49ffb640 1 -- 172.21.15.45:0/944948066 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2754090078 0 0) 0x7f7b38003520 con 0x7f7b4c0d2310 2024-05-08T05:51:01.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b49ffb640 1 -- 172.21.15.45:0/944948066 >> v2:172.21.15.63:3300/0 conn(0x7f7b4c15f2d0 msgr2=0x7f7b4c0b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:01.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b49ffb640 1 --2- 172.21.15.45:0/944948066 >> v2:172.21.15.63:3300/0 conn(0x7f7b4c15f2d0 0x7f7b4c0b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b49ffb640 1 -- 172.21.15.45:0/944948066 >> v1:172.21.15.63:6789/0 conn(0x7f7b4c0d4fc0 legacy=0x7f7b4c0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b49ffb640 1 -- 172.21.15.45:0/944948066 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4c0b4f00 con 0x7f7b4c0d2310 2024-05-08T05:51:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b49ffb640 1 -- 172.21.15.45:0/944948066 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f7b3c0016f0 con 0x7f7b4c0d2310 2024-05-08T05:51:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b49ffb640 1 -- 172.21.15.45:0/944948066 >> v1:172.21.15.45:6789/0 conn(0x7f7b4c0d2310 legacy=0x7f7b4c0d2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b49ffb640 1 --2- 172.21.15.45:0/944948066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b38003c80 0x7f7b38004080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b49ffb640 1 --2- 172.21.15.45:0/944948066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b38004800 0x7f7b38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b49ffb640 1 --2- 172.21.15.45:0/944948066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b38005230 0x7f7b3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b49ffb640 1 -- 172.21.15.45:0/944948066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c0b5810 con 0x7f7b38004800 2024-05-08T05:51:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b49ffb640 1 -- 172.21.15.45:0/944948066 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3800c190 con 0x7f7b38005230 2024-05-08T05:51:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.557+0000 7f7b49ffb640 1 -- 172.21.15.45:0/944948066 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7b3800c390 con 0x7f7b38003c80 2024-05-08T05:51:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.558+0000 7f7b4a7fc640 1 --2- 172.21.15.45:0/944948066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b38004800 0x7f7b38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.558+0000 7f7b4b7fe640 1 --2- 172.21.15.45:0/944948066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b38005230 0x7f7b3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.558+0000 7f7b4affd640 1 --2- 172.21.15.45:0/944948066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b38003c80 0x7f7b38004080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.558+0000 7f7b4a7fc640 1 -- 172.21.15.45:0/944948066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b38003c80 msgr2=0x7f7b38004080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.558+0000 7f7b4a7fc640 1 --2- 172.21.15.45:0/944948066 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b38003c80 0x7f7b38004080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.558+0000 7f7b4a7fc640 1 -- 172.21.15.45:0/944948066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b38005230 msgr2=0x7f7b3800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.558+0000 7f7b4a7fc640 1 --2- 172.21.15.45:0/944948066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b38005230 0x7f7b3800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.558+0000 7f7b4a7fc640 1 -- 172.21.15.45:0/944948066 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b38003520 con 0x7f7b38004800 2024-05-08T05:51:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.558+0000 7f7b4b7fe640 1 --2- 172.21.15.45:0/944948066 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b38005230 0x7f7b3800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.558+0000 7f7b4b7fe640 1 -- 172.21.15.45:0/944948066 reap_dead start 2024-05-08T05:51:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.559+0000 7f7b4a7fc640 1 --2- 172.21.15.45:0/944948066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b38004800 0x7f7b38004c80 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f7b34001900 tx=0x7f7b38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6156015aa58f5fbf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.559+0000 7f7b49ffb640 1 -- 172.21.15.45:0/944948066 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34001520 con 0x7f7b38004800 2024-05-08T05:51:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.559+0000 7f7b49ffb640 1 -- 172.21.15.45:0/944948066 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b3402acb0 con 0x7f7b38004800 2024-05-08T05:51:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.559+0000 7f7b49ffb640 1 -- 172.21.15.45:0/944948066 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b34005310 con 0x7f7b38004800 2024-05-08T05:51:01.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.559+0000 7f7b513c1640 1 -- 172.21.15.45:0/944948066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b38004800 msgr2=0x7f7b38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:01.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.559+0000 7f7b513c1640 1 --2- 172.21.15.45:0/944948066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b38004800 0x7f7b38004c80 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f7b34001900 tx=0x7f7b38005830 comp rx=0 tx=0).stop 2024-05-08T05:51:01.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.559+0000 7f7b513c1640 1 -- 172.21.15.45:0/944948066 shutdown_connections 2024-05-08T05:51:01.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.559+0000 7f7b513c1640 1 --2- 172.21.15.45:0/944948066 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b38004800 0x7f7b38004c80 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.559+0000 7f7b513c1640 1 -- 172.21.15.45:0/944948066 >> 172.21.15.45:0/944948066 conn(0x7f7b4c0746d0 msgr2=0x7f7b4c09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:01.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.559+0000 7f7b513c1640 1 -- 172.21.15.45:0/944948066 shutdown_connections 2024-05-08T05:51:01.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.559+0000 7f7b513c1640 1 -- 172.21.15.45:0/944948066 wait complete. 2024-05-08T05:51:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.561+0000 7f7b513c1640 1 Processor -- start 2024-05-08T05:51:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.561+0000 7f7b513c1640 1 -- start start 2024-05-08T05:51:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.561+0000 7f7b513c1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b38004800 0x7f7b4c15c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.561+0000 7f7b513c1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b4c15a2a0 0x7f7b4c15d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.561+0000 7f7b513c1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b4c161710 0x7f7b4c157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.561+0000 7f7b513c1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c0b5ca0 con 0x7f7b38004800 2024-05-08T05:51:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.561+0000 7f7b513c1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4c0b54a0 con 0x7f7b4c15a2a0 2024-05-08T05:51:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.561+0000 7f7b513c1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7b4c0b5e20 con 0x7f7b4c161710 2024-05-08T05:51:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.561+0000 7f7b4affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b38004800 0x7f7b4c15c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.562+0000 7f7b4affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b38004800 0x7f7b4c15c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37150/0 (socket says 172.21.15.45:37150) 2024-05-08T05:51:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.562+0000 7f7b4affd640 1 -- 172.21.15.45:0/2864467429 learned_addr learned my addr 172.21.15.45:0/2864467429 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.562+0000 7f7b4b7fe640 1 --2- 172.21.15.45:0/2864467429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b4c161710 0x7f7b4c157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.562+0000 7f7b4a7fc640 1 --2- 172.21.15.45:0/2864467429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b4c15a2a0 0x7f7b4c15d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.562+0000 7f7b4affd640 1 -- 172.21.15.45:0/2864467429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b4c161710 msgr2=0x7f7b4c157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.562+0000 7f7b4affd640 1 --2- 172.21.15.45:0/2864467429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b4c161710 0x7f7b4c157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.562+0000 7f7b4affd640 1 -- 172.21.15.45:0/2864467429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b4c15a2a0 msgr2=0x7f7b4c15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.562+0000 7f7b4affd640 1 --2- 172.21.15.45:0/2864467429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b4c15a2a0 0x7f7b4c15d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.562+0000 7f7b4affd640 1 -- 172.21.15.45:0/2864467429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4c15cc40 con 0x7f7b38004800 2024-05-08T05:51:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.562+0000 7f7b4a7fc640 1 --2- 172.21.15.45:0/2864467429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b4c15a2a0 0x7f7b4c15d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.562+0000 7f7b4affd640 1 --2- 172.21.15.45:0/2864467429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b38004800 0x7f7b4c15c660 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f7b44007f30 tx=0x7f7b44011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.562+0000 7f7b437fe640 1 -- 172.21.15.45:0/2864467429 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4401a020 con 0x7f7b38004800 2024-05-08T05:51:01.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.562+0000 7f7b437fe640 1 -- 172.21.15.45:0/2864467429 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b44016470 con 0x7f7b38004800 2024-05-08T05:51:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.562+0000 7f7b513c1640 1 -- 172.21.15.45:0/2864467429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4c160950 con 0x7f7b38004800 2024-05-08T05:51:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.562+0000 7f7b437fe640 1 -- 172.21.15.45:0/2864467429 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44016860 con 0x7f7b38004800 2024-05-08T05:51:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.562+0000 7f7b513c1640 1 -- 172.21.15.45:0/2864467429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b4c160f10 con 0x7f7b38004800 2024-05-08T05:51:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.563+0000 7f7b513c1640 1 -- 172.21.15.45:0/2864467429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b4c069b00 con 0x7f7b38004800 2024-05-08T05:51:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.564+0000 7f7b437fe640 1 -- 172.21.15.45:0/2864467429 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7b44005d10 con 0x7f7b38004800 2024-05-08T05:51:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.564+0000 7f7b437fe640 1 --2- 172.21.15.45:0/2864467429 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7b2c0762d0 0x7f7b2c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.564+0000 7f7b437fe640 1 -- 172.21.15.45:0/2864467429 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5249+0+0 (secure 0 0 0) 0x7f7b44098250 con 0x7f7b38004800 2024-05-08T05:51:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.565+0000 7f7b4a7fc640 1 --2- 172.21.15.45:0/2864467429 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7b2c0762d0 0x7f7b2c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.565+0000 7f7b4a7fc640 1 --2- 172.21.15.45:0/2864467429 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7b2c0762d0 0x7f7b2c078790 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7b34031000 tx=0x7f7b3402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:01.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.569+0000 7f7b437fe640 1 -- 172.21.15.45:0/2864467429 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7b44024030 con 0x7f7b38004800 2024-05-08T05:51:01.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.763+0000 7f7b513c1640 1 -- 172.21.15.45:0/2864467429 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7b4c0b4b60 con 0x7f7b38004800 2024-05-08T05:51:01.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.764+0000 7f7b437fe640 1 -- 172.21.15.45:0/2864467429 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v115) ==== 84+0+65 (secure 0 0 0) 0x7f7b44061cd0 con 0x7f7b38004800 2024-05-08T05:51:01.764 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:01.764 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-05-08T05:51:01.764 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-05-08T05:51:01.764 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-05-08T05:51:01.765 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-05-08T05:51:01.765 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-05-08T05:51:01.765 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-05-08T05:51:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.766+0000 7f7b513c1640 1 -- 172.21.15.45:0/2864467429 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7b2c0762d0 msgr2=0x7f7b2c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.766+0000 7f7b513c1640 1 --2- 172.21.15.45:0/2864467429 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7b2c0762d0 0x7f7b2c078790 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7b34031000 tx=0x7f7b3402e000 comp rx=0 tx=0).stop 2024-05-08T05:51:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.766+0000 7f7b513c1640 1 -- 172.21.15.45:0/2864467429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b38004800 msgr2=0x7f7b4c15c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.766+0000 7f7b513c1640 1 --2- 172.21.15.45:0/2864467429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b38004800 0x7f7b4c15c660 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f7b44007f30 tx=0x7f7b44011660 comp rx=0 tx=0).stop 2024-05-08T05:51:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.766+0000 7f7b513c1640 1 -- 172.21.15.45:0/2864467429 shutdown_connections 2024-05-08T05:51:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.766+0000 7f7b513c1640 1 --2- 172.21.15.45:0/2864467429 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7b2c0762d0 0x7f7b2c078790 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.766+0000 7f7b513c1640 1 --2- 172.21.15.45:0/2864467429 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7b4c161710 0x7f7b4c157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.766+0000 7f7b513c1640 1 --2- 172.21.15.45:0/2864467429 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b4c15a2a0 0x7f7b4c15d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.766+0000 7f7b513c1640 1 --2- 172.21.15.45:0/2864467429 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b38004800 0x7f7b4c15c660 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.766+0000 7f7b513c1640 1 -- 172.21.15.45:0/2864467429 >> 172.21.15.45:0/2864467429 conn(0x7f7b4c0746d0 msgr2=0x7f7b4c0d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.766+0000 7f7b513c1640 1 -- 172.21.15.45:0/2864467429 shutdown_connections 2024-05-08T05:51:01.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.766+0000 7f7b513c1640 1 -- 172.21.15.45:0/2864467429 wait complete. 2024-05-08T05:51:01.777 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T05:51:01.778 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T05:51:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.885+0000 7fe757d46640 1 Processor -- start 2024-05-08T05:51:01.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.886+0000 7fe757d46640 1 -- start start 2024-05-08T05:51:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.887+0000 7fe757d46640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe75015f140 0x7fe7500b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.887+0000 7fe757d46640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe75006eb70 con 0x7fe75006ced0 2024-05-08T05:51:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.887+0000 7fe757d46640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7500b4bc0 con 0x7fe7500ae8b0 2024-05-08T05:51:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.887+0000 7fe757d46640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe75006eea0 con 0x7fe75015f140 2024-05-08T05:51:01.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.887+0000 7fe755ae2640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe75015f140 0x7fe7500b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.887+0000 7fe7552e1640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fe75006ced0 0x7fe75006f330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52758/0 (socket says 172.21.15.45:52758) 2024-05-08T05:51:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.887+0000 7fe755ae2640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe75015f140 0x7fe7500b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37162/0 (socket says 172.21.15.45:37162) 2024-05-08T05:51:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.887+0000 7fe7552e1640 1 -- 172.21.15.45:0/709983637 learned_addr learned my addr 172.21.15.45:0/709983637 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.887+0000 7fe755ae2640 1 -- 172.21.15.45:0/709983637 >> v1:172.21.15.63:6789/0 conn(0x7fe7500ae8b0 legacy=0x7fe750072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.887+0000 7fe755ae2640 1 -- 172.21.15.45:0/709983637 >> v1:172.21.15.45:6789/0 conn(0x7fe75006ced0 legacy=0x7fe75006f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.887+0000 7fe755ae2640 1 -- 172.21.15.45:0/709983637 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7500b4ea0 con 0x7fe75015f140 2024-05-08T05:51:01.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.887+0000 7fe754ae0640 1 -- 172.21.15.45:0/709983637 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 529386988 0 0) 0x7fe75006eb70 con 0x7fe75006ced0 2024-05-08T05:51:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.888+0000 7fe755ae2640 1 --2- 172.21.15.45:0/709983637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe75015f140 0x7fe7500b4550 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fe7500b4310 tx=0x7fe74c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72f7840fe457301 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.888+0000 7fe754ae0640 1 -- 172.21.15.45:0/709983637 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c037070 con 0x7fe75015f140 2024-05-08T05:51:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.888+0000 7fe754ae0640 1 -- 172.21.15.45:0/709983637 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe74c02adb0 con 0x7fe75015f140 2024-05-08T05:51:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.888+0000 7fe754ae0640 1 -- 172.21.15.45:0/709983637 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c032660 con 0x7fe75015f140 2024-05-08T05:51:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.889+0000 7fe757d46640 1 -- 172.21.15.45:0/709983637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe75015f140 msgr2=0x7fe7500b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:01.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.889+0000 7fe757d46640 1 --2- 172.21.15.45:0/709983637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe75015f140 0x7fe7500b4550 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fe7500b4310 tx=0x7fe74c02a060 comp rx=0 tx=0).stop 2024-05-08T05:51:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.889+0000 7fe757d46640 1 -- 172.21.15.45:0/709983637 shutdown_connections 2024-05-08T05:51:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.889+0000 7fe757d46640 1 --2- 172.21.15.45:0/709983637 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe75015f140 0x7fe7500b4550 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.889+0000 7fe757d46640 1 -- 172.21.15.45:0/709983637 >> 172.21.15.45:0/709983637 conn(0x7fe7500b3640 msgr2=0x7fe7500b3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.889+0000 7fe757d46640 1 -- 172.21.15.45:0/709983637 shutdown_connections 2024-05-08T05:51:01.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.889+0000 7fe757d46640 1 -- 172.21.15.45:0/709983637 wait complete. 2024-05-08T05:51:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.890+0000 7fe757d46640 1 Processor -- start 2024-05-08T05:51:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.890+0000 7fe757d46640 1 -- start start 2024-05-08T05:51:01.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.891+0000 7fe757d46640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe75006ced0 0x7fe750156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.891+0000 7fe757d46640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7500ae8b0 0x7fe750158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.891+0000 7fe757d46640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe75015f140 0x7fe7501519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.891+0000 7fe757d46640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe7500b5180 con 0x7fe75015f140 2024-05-08T05:51:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.891+0000 7fe757d46640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe75015f520 con 0x7fe7500ae8b0 2024-05-08T05:51:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.891+0000 7fe757d46640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe7500b5300 con 0x7fe75006ced0 2024-05-08T05:51:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.891+0000 7fe7562e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe75015f140 0x7fe7501519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.891+0000 7fe7562e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe75015f140 0x7fe7501519f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37172/0 (socket says 172.21.15.45:37172) 2024-05-08T05:51:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.891+0000 7fe7562e3640 1 -- 172.21.15.45:0/499361447 learned_addr learned my addr 172.21.15.45:0/499361447 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.891+0000 7fe755ae2640 1 --2- 172.21.15.45:0/499361447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe75006ced0 0x7fe750156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.891+0000 7fe7552e1640 1 --2- 172.21.15.45:0/499361447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7500ae8b0 0x7fe750158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.891+0000 7fe7562e3640 1 -- 172.21.15.45:0/499361447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe75006ced0 msgr2=0x7fe750156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.891+0000 7fe7562e3640 1 --2- 172.21.15.45:0/499361447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe75006ced0 0x7fe750156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.891+0000 7fe7562e3640 1 -- 172.21.15.45:0/499361447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7500ae8b0 msgr2=0x7fe750158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:01.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.891+0000 7fe7562e3640 1 --2- 172.21.15.45:0/499361447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7500ae8b0 0x7fe750158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.891+0000 7fe7562e3640 1 -- 172.21.15.45:0/499361447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe750156a90 con 0x7fe75015f140 2024-05-08T05:51:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.891+0000 7fe755ae2640 1 --2- 172.21.15.45:0/499361447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe75006ced0 0x7fe750156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.891+0000 7fe7552e1640 1 --2- 172.21.15.45:0/499361447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7500ae8b0 0x7fe750158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.891+0000 7fe7562e3640 1 --2- 172.21.15.45:0/499361447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe75015f140 0x7fe7501519f0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fe74800b990 tx=0x7fe74800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.891+0000 7fe7467fc640 1 -- 172.21.15.45:0/499361447 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe748019070 con 0x7fe75015f140 2024-05-08T05:51:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.892+0000 7fe7467fc640 1 -- 172.21.15.45:0/499361447 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe74800cba0 con 0x7fe75015f140 2024-05-08T05:51:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.892+0000 7fe757d46640 1 -- 172.21.15.45:0/499361447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe750156d70 con 0x7fe75015f140 2024-05-08T05:51:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.892+0000 7fe7467fc640 1 -- 172.21.15.45:0/499361447 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe748014440 con 0x7fe75015f140 2024-05-08T05:51:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.892+0000 7fe757d46640 1 -- 172.21.15.45:0/499361447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe750157330 con 0x7fe75015f140 2024-05-08T05:51:01.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.892+0000 7fe757d46640 1 -- 172.21.15.45:0/499361447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7500693f0 con 0x7fe75015f140 2024-05-08T05:51:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.893+0000 7fe7467fc640 1 -- 172.21.15.45:0/499361447 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe74801c070 con 0x7fe75015f140 2024-05-08T05:51:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.894+0000 7fe7467fc640 1 --2- 172.21.15.45:0/499361447 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe7340762d0 0x7fe734078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.894+0000 7fe7467fc640 1 -- 172.21.15.45:0/499361447 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5249+0+0 (secure 0 0 0) 0x7fe748097160 con 0x7fe75015f140 2024-05-08T05:51:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.894+0000 7fe755ae2640 1 --2- 172.21.15.45:0/499361447 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe7340762d0 0x7fe734078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:01.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.894+0000 7fe755ae2640 1 --2- 172.21.15.45:0/499361447 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe7340762d0 0x7fe734078790 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe74c034000 tx=0x7fe74c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:01.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:01.898+0000 7fe7467fc640 1 -- 172.21.15.45:0/499361447 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe748060c80 con 0x7fe75015f140 2024-05-08T05:51:02.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:02.089+0000 7fe757d46640 1 -- 172.21.15.45:0/499361447 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe75006d8e0 con 0x7fe75015f140 2024-05-08T05:51:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.040+0000 7fe7467fc640 1 -- 172.21.15.45:0/499361447 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v116) ==== 138+0+0 (secure 0 0 0) 0x7fe748060580 con 0x7fe75015f140 2024-05-08T05:51:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-05-08T05:51:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.043+0000 7fe757d46640 1 -- 172.21.15.45:0/499361447 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe7340762d0 msgr2=0x7fe734078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.043+0000 7fe757d46640 1 --2- 172.21.15.45:0/499361447 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe7340762d0 0x7fe734078790 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe74c034000 tx=0x7fe74c02e000 comp rx=0 tx=0).stop 2024-05-08T05:51:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.044+0000 7fe757d46640 1 -- 172.21.15.45:0/499361447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe75015f140 msgr2=0x7fe7501519f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.044+0000 7fe757d46640 1 --2- 172.21.15.45:0/499361447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe75015f140 0x7fe7501519f0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fe74800b990 tx=0x7fe74800be50 comp rx=0 tx=0).stop 2024-05-08T05:51:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.044+0000 7fe757d46640 1 -- 172.21.15.45:0/499361447 shutdown_connections 2024-05-08T05:51:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.044+0000 7fe757d46640 1 --2- 172.21.15.45:0/499361447 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe7340762d0 0x7fe734078790 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.044+0000 7fe757d46640 1 --2- 172.21.15.45:0/499361447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe75015f140 0x7fe7501519f0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.044+0000 7fe757d46640 1 --2- 172.21.15.45:0/499361447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe7500ae8b0 0x7fe750158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.044+0000 7fe757d46640 1 --2- 172.21.15.45:0/499361447 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe75006ced0 0x7fe750156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:03.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.044+0000 7fe757d46640 1 -- 172.21.15.45:0/499361447 >> 172.21.15.45:0/499361447 conn(0x7fe7500b3640 msgr2=0x7fe7500b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:03.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.044+0000 7fe757d46640 1 -- 172.21.15.45:0/499361447 shutdown_connections 2024-05-08T05:51:03.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.044+0000 7fe757d46640 1 -- 172.21.15.45:0/499361447 wait complete. 2024-05-08T05:51:03.061 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T05:51:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.178+0000 7fb85fdb6640 1 Processor -- start 2024-05-08T05:51:03.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.178+0000 7fb85fdb6640 1 -- start start 2024-05-08T05:51:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.179+0000 7fb85fdb6640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8580b2490 0x7fb858070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.179+0000 7fb85fdb6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb85806e110 con 0x7fb858070d20 2024-05-08T05:51:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.179+0000 7fb85fdb6640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb85806e3f0 con 0x7fb8581629b0 2024-05-08T05:51:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.179+0000 7fb85fdb6640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb85806dec0 con 0x7fb8580b2490 2024-05-08T05:51:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.179+0000 7fb85e353640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb858070d20 0x7fb8580afcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52766/0 (socket says 172.21.15.45:52766) 2024-05-08T05:51:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.179+0000 7fb85e353640 1 -- 172.21.15.45:0/2616234181 learned_addr learned my addr 172.21.15.45:0/2616234181 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.179+0000 7fb85d351640 1 --2- 172.21.15.45:0/2616234181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8580b2490 0x7fb858070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.179+0000 7fb85cb50640 1 -- 172.21.15.45:0/2616234181 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 861357226 0 0) 0x7fb85806e110 con 0x7fb858070d20 2024-05-08T05:51:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.179+0000 7fb85cb50640 1 -- 172.21.15.45:0/2616234181 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb844003520 con 0x7fb858070d20 2024-05-08T05:51:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85cb50640 1 -- 172.21.15.45:0/2616234181 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2876254359 0 0) 0x7fb844003520 con 0x7fb858070d20 2024-05-08T05:51:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85cb50640 1 -- 172.21.15.45:0/2616234181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8580b2490 msgr2=0x7fb858070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:03.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85cb50640 1 --2- 172.21.15.45:0/2616234181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8580b2490 0x7fb858070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85cb50640 1 -- 172.21.15.45:0/2616234181 >> v1:172.21.15.63:6789/0 conn(0x7fb8581629b0 legacy=0x7fb85806eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85cb50640 1 -- 172.21.15.45:0/2616234181 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb85806e6d0 con 0x7fb858070d20 2024-05-08T05:51:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85cb50640 1 -- 172.21.15.45:0/2616234181 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fb850001a70 con 0x7fb858070d20 2024-05-08T05:51:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85cb50640 1 -- 172.21.15.45:0/2616234181 >> v1:172.21.15.45:6789/0 conn(0x7fb858070d20 legacy=0x7fb8580afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85cb50640 1 --2- 172.21.15.45:0/2616234181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8580b2490 0x7fb8440039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85cb50640 1 --2- 172.21.15.45:0/2616234181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb844003f70 0x7fb844004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85cb50640 1 --2- 172.21.15.45:0/2616234181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb844004d40 0x7fb8440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85cb50640 1 -- 172.21.15.45:0/2616234181 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb85806dec0 con 0x7fb844004d40 2024-05-08T05:51:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85cb50640 1 -- 172.21.15.45:0/2616234181 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb84400bad0 con 0x7fb844003f70 2024-05-08T05:51:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85cb50640 1 -- 172.21.15.45:0/2616234181 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb84400bcd0 con 0x7fb8580b2490 2024-05-08T05:51:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85e353640 1 --2- 172.21.15.45:0/2616234181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb844004d40 0x7fb8440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85db52640 1 --2- 172.21.15.45:0/2616234181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8580b2490 0x7fb8440039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:03.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85d351640 1 --2- 172.21.15.45:0/2616234181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb844003f70 0x7fb844004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85e353640 1 -- 172.21.15.45:0/2616234181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8580b2490 msgr2=0x7fb8440039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85e353640 1 --2- 172.21.15.45:0/2616234181 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8580b2490 0x7fb8440039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85e353640 1 -- 172.21.15.45:0/2616234181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb844003f70 msgr2=0x7fb844004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85e353640 1 --2- 172.21.15.45:0/2616234181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb844003f70 0x7fb844004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.180+0000 7fb85e353640 1 -- 172.21.15.45:0/2616234181 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb844003520 con 0x7fb844004d40 2024-05-08T05:51:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.181+0000 7fb85e353640 1 --2- 172.21.15.45:0/2616234181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb844004d40 0x7fb8440051c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb85000a850 tx=0x7fb844005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912c888e40c2c526 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.181+0000 7fb85cb50640 1 -- 172.21.15.45:0/2616234181 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85000a4a0 con 0x7fb844004d40 2024-05-08T05:51:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.181+0000 7fb85cb50640 1 -- 172.21.15.45:0/2616234181 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb85002f070 con 0x7fb844004d40 2024-05-08T05:51:03.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.181+0000 7fb85cb50640 1 -- 172.21.15.45:0/2616234181 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85002bbe0 con 0x7fb844004d40 2024-05-08T05:51:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.182+0000 7fb85fdb6640 1 -- 172.21.15.45:0/2616234181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb844004d40 msgr2=0x7fb8440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.182+0000 7fb85fdb6640 1 --2- 172.21.15.45:0/2616234181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb844004d40 0x7fb8440051c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fb85000a850 tx=0x7fb844005830 comp rx=0 tx=0).stop 2024-05-08T05:51:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.182+0000 7fb85e353640 1 -- 172.21.15.45:0/2616234181 reap_dead start 2024-05-08T05:51:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.182+0000 7fb85fdb6640 1 -- 172.21.15.45:0/2616234181 shutdown_connections 2024-05-08T05:51:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.182+0000 7fb85fdb6640 1 -- 172.21.15.45:0/2616234181 >> 172.21.15.45:0/2616234181 conn(0x7fb8580b49d0 msgr2=0x7fb8580b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:03.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.182+0000 7fb85fdb6640 1 -- 172.21.15.45:0/2616234181 shutdown_connections 2024-05-08T05:51:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.182+0000 7fb85fdb6640 1 -- 172.21.15.45:0/2616234181 wait complete. 2024-05-08T05:51:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.183+0000 7fb85fdb6640 1 Processor -- start 2024-05-08T05:51:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.183+0000 7fb85fdb6640 1 -- start start 2024-05-08T05:51:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb85fdb6640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb844004d40 0x7fb8581512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb85fdb6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8580b5ac0 0x7fb8580b7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb85fdb6640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8580b5f40 0x7fb8580b7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb85fdb6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb8580b0540 con 0x7fb8580b5ac0 2024-05-08T05:51:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb85fdb6640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb8580b03c0 con 0x7fb844004d40 2024-05-08T05:51:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb85fdb6640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb8580b06c0 con 0x7fb8580b5f40 2024-05-08T05:51:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb85d351640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8580b5ac0 0x7fb8580b7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb85d351640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8580b5ac0 0x7fb8580b7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37200/0 (socket says 172.21.15.45:37200) 2024-05-08T05:51:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb85d351640 1 -- 172.21.15.45:0/1489686586 learned_addr learned my addr 172.21.15.45:0/1489686586 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb85db52640 1 --2- 172.21.15.45:0/1489686586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb844004d40 0x7fb8581512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:03.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb85e353640 1 --2- 172.21.15.45:0/1489686586 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8580b5f40 0x7fb8580b7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb85d351640 1 -- 172.21.15.45:0/1489686586 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8580b5f40 msgr2=0x7fb8580b7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb85d351640 1 --2- 172.21.15.45:0/1489686586 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8580b5f40 0x7fb8580b7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb85d351640 1 -- 172.21.15.45:0/1489686586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb844004d40 msgr2=0x7fb8581512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb85d351640 1 --2- 172.21.15.45:0/1489686586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb844004d40 0x7fb8581512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb85d351640 1 -- 172.21.15.45:0/1489686586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb858151800 con 0x7fb8580b5ac0 2024-05-08T05:51:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb85d351640 1 --2- 172.21.15.45:0/1489686586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8580b5ac0 0x7fb8580b7540 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb8480119a0 tx=0x7fb848011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb84e7fc640 1 -- 172.21.15.45:0/1489686586 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84800faf0 con 0x7fb8580b5ac0 2024-05-08T05:51:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb84e7fc640 1 -- 172.21.15.45:0/1489686586 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb84800fc90 con 0x7fb8580b5ac0 2024-05-08T05:51:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb85fdb6640 1 -- 172.21.15.45:0/1489686586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8580b64a0 con 0x7fb8580b5ac0 2024-05-08T05:51:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.184+0000 7fb84e7fc640 1 -- 172.21.15.45:0/1489686586 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb848014440 con 0x7fb8580b5ac0 2024-05-08T05:51:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.185+0000 7fb85fdb6640 1 -- 172.21.15.45:0/1489686586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8580b6a60 con 0x7fb8580b5ac0 2024-05-08T05:51:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.185+0000 7fb85fdb6640 1 -- 172.21.15.45:0/1489686586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8580693f0 con 0x7fb8580b5ac0 2024-05-08T05:51:03.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.186+0000 7fb84e7fc640 1 -- 172.21.15.45:0/1489686586 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb848012040 con 0x7fb8580b5ac0 2024-05-08T05:51:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.187+0000 7fb84e7fc640 1 --2- 172.21.15.45:0/1489686586 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb83c076300 0x7fb83c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.187+0000 7fb84e7fc640 1 -- 172.21.15.45:0/1489686586 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5619+0+0 (secure 0 0 0) 0x7fb848097a00 con 0x7fb8580b5ac0 2024-05-08T05:51:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.187+0000 7fb85db52640 1 --2- 172.21.15.45:0/1489686586 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb83c076300 0x7fb83c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:03.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.187+0000 7fb85db52640 1 --2- 172.21.15.45:0/1489686586 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb83c076300 0x7fb83c0787c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb8540011a0 tx=0x7fb854009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:03.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.192+0000 7fb84e7fc640 1 -- 172.21.15.45:0/1489686586 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb858151800 con 0x7fb8580b5ac0 2024-05-08T05:51:03.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:03.383+0000 7fb85fdb6640 1 -- 172.21.15.45:0/1489686586 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb85806db50 con 0x7fb8580b5ac0 2024-05-08T05:51:04.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.043+0000 7fb84e7fc640 1 -- 172.21.15.45:0/1489686586 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v117) ==== 170+0+0 (secure 0 0 0) 0x7fb848019030 con 0x7fb8580b5ac0 2024-05-08T05:51:04.043 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-05-08T05:51:04.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.046+0000 7fb85fdb6640 1 -- 172.21.15.45:0/1489686586 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb83c076300 msgr2=0x7fb83c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:04.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.046+0000 7fb85fdb6640 1 --2- 172.21.15.45:0/1489686586 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb83c076300 0x7fb83c0787c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb8540011a0 tx=0x7fb854009040 comp rx=0 tx=0).stop 2024-05-08T05:51:04.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.046+0000 7fb85fdb6640 1 -- 172.21.15.45:0/1489686586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8580b5ac0 msgr2=0x7fb8580b7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:04.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.046+0000 7fb85fdb6640 1 --2- 172.21.15.45:0/1489686586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8580b5ac0 0x7fb8580b7540 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb8480119a0 tx=0x7fb848011e60 comp rx=0 tx=0).stop 2024-05-08T05:51:04.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.046+0000 7fb85fdb6640 1 -- 172.21.15.45:0/1489686586 shutdown_connections 2024-05-08T05:51:04.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.046+0000 7fb85fdb6640 1 --2- 172.21.15.45:0/1489686586 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb83c076300 0x7fb83c0787c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:04.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.046+0000 7fb85fdb6640 1 --2- 172.21.15.45:0/1489686586 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8580b5f40 0x7fb8580b7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:04.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.046+0000 7fb85fdb6640 1 --2- 172.21.15.45:0/1489686586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8580b5ac0 0x7fb8580b7540 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:04.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.046+0000 7fb85fdb6640 1 --2- 172.21.15.45:0/1489686586 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb844004d40 0x7fb8581512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:04.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.046+0000 7fb85fdb6640 1 -- 172.21.15.45:0/1489686586 >> 172.21.15.45:0/1489686586 conn(0x7fb8580b49d0 msgr2=0x7fb8580afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:04.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.046+0000 7fb85fdb6640 1 -- 172.21.15.45:0/1489686586 shutdown_connections 2024-05-08T05:51:04.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.046+0000 7fb85fdb6640 1 -- 172.21.15.45:0/1489686586 wait complete. 2024-05-08T05:51:04.074 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T05:51:04.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.186+0000 7ff097609640 1 Processor -- start 2024-05-08T05:51:04.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.187+0000 7ff097609640 1 -- start start 2024-05-08T05:51:04.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.187+0000 7ff097609640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ff09015f4e0 0x7ff0900cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:04.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.187+0000 7ff097609640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff0900d56b0 0x7ff0900d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:04.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.187+0000 7ff097609640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0900d0320 con 0x7ff0900d3aa0 2024-05-08T05:51:04.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.187+0000 7ff097609640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff0900d0600 con 0x7ff0900d56b0 2024-05-08T05:51:04.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.187+0000 7ff097609640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ff0900d08e0 con 0x7ff09015f4e0 2024-05-08T05:51:04.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.188+0000 7ff095ba6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff0900d56b0 0x7ff0900d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:04.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.188+0000 7ff095ba6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff0900d56b0 0x7ff0900d13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37208/0 (socket says 172.21.15.45:37208) 2024-05-08T05:51:04.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.188+0000 7ff095ba6640 1 -- 172.21.15.45:0/1642617333 learned_addr learned my addr 172.21.15.45:0/1642617333 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:04.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.188+0000 7ff0953a5640 1 --2- 172.21.15.45:0/1642617333 >> v2:172.21.15.63:3300/0 conn(0x7ff09015f4e0 0x7ff0900cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:04.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.188+0000 7ff095ba6640 1 -- 172.21.15.45:0/1642617333 >> v2:172.21.15.63:3300/0 conn(0x7ff09015f4e0 msgr2=0x7ff0900cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:04.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.188+0000 7ff095ba6640 1 --2- 172.21.15.45:0/1642617333 >> v2:172.21.15.63:3300/0 conn(0x7ff09015f4e0 0x7ff0900cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:04.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.188+0000 7ff095ba6640 1 -- 172.21.15.45:0/1642617333 >> v1:172.21.15.63:6789/0 conn(0x7ff0900d3aa0 legacy=0x7ff0900d4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:04.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.188+0000 7ff095ba6640 1 -- 172.21.15.45:0/1642617333 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0900d0bc0 con 0x7ff0900d56b0 2024-05-08T05:51:04.189 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.188+0000 7ff0953a5640 1 --2- 172.21.15.45:0/1642617333 >> v2:172.21.15.63:3300/0 conn(0x7ff09015f4e0 0x7ff0900cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:04.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.189+0000 7ff095ba6640 1 --2- 172.21.15.45:0/1642617333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0900d56b0 0x7ff0900d13c0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7ff0900b6230 tx=0x7ff08002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61eb04f7117b8115 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:04.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.189+0000 7ff08ffff640 1 -- 172.21.15.45:0/1642617333 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080036070 con 0x7ff0900d56b0 2024-05-08T05:51:04.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.189+0000 7ff08ffff640 1 -- 172.21.15.45:0/1642617333 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff080031600 con 0x7ff0900d56b0 2024-05-08T05:51:04.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.189+0000 7ff08ffff640 1 -- 172.21.15.45:0/1642617333 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0800319d0 con 0x7ff0900d56b0 2024-05-08T05:51:04.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.189+0000 7ff097609640 1 -- 172.21.15.45:0/1642617333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0900d56b0 msgr2=0x7ff0900d13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:04.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.189+0000 7ff097609640 1 --2- 172.21.15.45:0/1642617333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0900d56b0 0x7ff0900d13c0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7ff0900b6230 tx=0x7ff08002a730 comp rx=0 tx=0).stop 2024-05-08T05:51:04.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.189+0000 7ff097609640 1 -- 172.21.15.45:0/1642617333 shutdown_connections 2024-05-08T05:51:04.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.189+0000 7ff097609640 1 --2- 172.21.15.45:0/1642617333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0900d56b0 0x7ff0900d13c0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:04.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.189+0000 7ff097609640 1 --2- 172.21.15.45:0/1642617333 >> v2:172.21.15.63:3300/0 conn(0x7ff09015f4e0 0x7ff0900cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:04.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.189+0000 7ff097609640 1 -- 172.21.15.45:0/1642617333 >> 172.21.15.45:0/1642617333 conn(0x7ff09008eac0 msgr2=0x7ff09009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:04.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.190+0000 7ff097609640 1 -- 172.21.15.45:0/1642617333 shutdown_connections 2024-05-08T05:51:04.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.190+0000 7ff097609640 1 -- 172.21.15.45:0/1642617333 wait complete. 2024-05-08T05:51:04.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.191+0000 7ff097609640 1 Processor -- start 2024-05-08T05:51:04.192 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.191+0000 7ff097609640 1 -- start start 2024-05-08T05:51:04.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff097609640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0900d3aa0 0x7ff0901626b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:04.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff097609640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0900d56b0 0x7ff090161430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:04.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff097609640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff09015f4e0 0x7ff0900b8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:04.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff097609640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff0900d3f00 con 0x7ff0900d56b0 2024-05-08T05:51:04.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff097609640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff09015f8c0 con 0x7ff0900d3aa0 2024-05-08T05:51:04.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff097609640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff0900d4080 con 0x7ff09015f4e0 2024-05-08T05:51:04.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff094ba4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0900d56b0 0x7ff090161430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:04.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff0953a5640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0900d3aa0 0x7ff0901626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:04.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff094ba4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0900d56b0 0x7ff090161430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37212/0 (socket says 172.21.15.45:37212) 2024-05-08T05:51:04.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff094ba4640 1 -- 172.21.15.45:0/3984385176 learned_addr learned my addr 172.21.15.45:0/3984385176 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:04.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff095ba6640 1 --2- 172.21.15.45:0/3984385176 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff09015f4e0 0x7ff0900b8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:04.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff094ba4640 1 -- 172.21.15.45:0/3984385176 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff09015f4e0 msgr2=0x7ff0900b8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:04.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff094ba4640 1 --2- 172.21.15.45:0/3984385176 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff09015f4e0 0x7ff0900b8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:04.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff094ba4640 1 -- 172.21.15.45:0/3984385176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0900d3aa0 msgr2=0x7ff0901626b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:04.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff094ba4640 1 --2- 172.21.15.45:0/3984385176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0900d3aa0 0x7ff0901626b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:04.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff094ba4640 1 -- 172.21.15.45:0/3984385176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff090161db0 con 0x7ff0900d56b0 2024-05-08T05:51:04.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff094ba4640 1 --2- 172.21.15.45:0/3984385176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0900d56b0 0x7ff090161430 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7ff08400c450 tx=0x7ff08400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:04.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff08dffb640 1 -- 172.21.15.45:0/3984385176 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0840052d0 con 0x7ff0900d56b0 2024-05-08T05:51:04.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff08dffb640 1 -- 172.21.15.45:0/3984385176 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff084005470 con 0x7ff0900d56b0 2024-05-08T05:51:04.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff097609640 1 -- 172.21.15.45:0/3984385176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff090160700 con 0x7ff0900d56b0 2024-05-08T05:51:04.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.192+0000 7ff08dffb640 1 -- 172.21.15.45:0/3984385176 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084012870 con 0x7ff0900d56b0 2024-05-08T05:51:04.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.193+0000 7ff097609640 1 -- 172.21.15.45:0/3984385176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0900b8080 con 0x7ff0900d56b0 2024-05-08T05:51:04.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.194+0000 7ff097609640 1 -- 172.21.15.45:0/3984385176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0900693f0 con 0x7ff0900d56b0 2024-05-08T05:51:04.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.195+0000 7ff08dffb640 1 -- 172.21.15.45:0/3984385176 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff084012a10 con 0x7ff0900d56b0 2024-05-08T05:51:04.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.195+0000 7ff08dffb640 1 --2- 172.21.15.45:0/3984385176 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff07c076300 0x7ff07c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:04.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.195+0000 7ff08dffb640 1 -- 172.21.15.45:0/3984385176 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6001+0+0 (secure 0 0 0) 0x7ff084097110 con 0x7ff0900d56b0 2024-05-08T05:51:04.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.195+0000 7ff0953a5640 1 --2- 172.21.15.45:0/3984385176 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff07c076300 0x7ff07c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:04.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.196+0000 7ff0953a5640 1 --2- 172.21.15.45:0/3984385176 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff07c076300 0x7ff07c0787c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff088006fd0 tx=0x7ff088008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:04.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.200+0000 7ff08dffb640 1 -- 172.21.15.45:0/3984385176 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff084060380 con 0x7ff0900d56b0 2024-05-08T05:51:04.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:04.428+0000 7ff097609640 1 -- 172.21.15.45:0/3984385176 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff0900b3e90 con 0x7ff0900d56b0 2024-05-08T05:51:05.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.057+0000 7ff08dffb640 1 -- 172.21.15.45:0/3984385176 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v96) ==== 174+0+0 (secure 0 0 0) 0x7ff084064680 con 0x7ff0900d56b0 2024-05-08T05:51:05.057 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 17 and data pool 18 2024-05-08T05:51:05.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.063+0000 7ff097609640 1 -- 172.21.15.45:0/3984385176 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff07c076300 msgr2=0x7ff07c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.063+0000 7ff097609640 1 --2- 172.21.15.45:0/3984385176 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff07c076300 0x7ff07c0787c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff088006fd0 tx=0x7ff088008040 comp rx=0 tx=0).stop 2024-05-08T05:51:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.063+0000 7ff097609640 1 -- 172.21.15.45:0/3984385176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0900d56b0 msgr2=0x7ff090161430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.063+0000 7ff097609640 1 --2- 172.21.15.45:0/3984385176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0900d56b0 0x7ff090161430 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7ff08400c450 tx=0x7ff08400c910 comp rx=0 tx=0).stop 2024-05-08T05:51:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.064+0000 7ff097609640 1 -- 172.21.15.45:0/3984385176 shutdown_connections 2024-05-08T05:51:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.064+0000 7ff097609640 1 --2- 172.21.15.45:0/3984385176 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff07c076300 0x7ff07c0787c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.064+0000 7ff097609640 1 --2- 172.21.15.45:0/3984385176 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff09015f4e0 0x7ff0900b8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.064+0000 7ff097609640 1 --2- 172.21.15.45:0/3984385176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0900d56b0 0x7ff090161430 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.064+0000 7ff097609640 1 --2- 172.21.15.45:0/3984385176 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0900d3aa0 0x7ff0901626b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.064+0000 7ff097609640 1 -- 172.21.15.45:0/3984385176 >> 172.21.15.45:0/3984385176 conn(0x7ff09008eac0 msgr2=0x7ff0900d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:05.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.070+0000 7ff097609640 1 -- 172.21.15.45:0/3984385176 shutdown_connections 2024-05-08T05:51:05.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.071+0000 7ff097609640 1 -- 172.21.15.45:0/3984385176 wait complete. 2024-05-08T05:51:05.100 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:51:05.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.208+0000 7fd7288f8640 1 Processor -- start 2024-05-08T05:51:05.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.208+0000 7fd7288f8640 1 -- start start 2024-05-08T05:51:05.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.209+0000 7fd7288f8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd724070d20 0x7fd7240711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.209+0000 7fd7288f8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd72406e110 con 0x7fd7240b2490 2024-05-08T05:51:05.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.209+0000 7fd7288f8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd72406e3f0 con 0x7fd7241629b0 2024-05-08T05:51:05.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.209+0000 7fd7288f8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd724070a40 con 0x7fd724070d20 2024-05-08T05:51:05.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.209+0000 7fd721d9b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd7240b2490 0x7fd724070330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52778/0 (socket says 172.21.15.45:52778) 2024-05-08T05:51:05.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.209+0000 7fd721d9b640 1 -- 172.21.15.45:0/922454131 learned_addr learned my addr 172.21.15.45:0/922454131 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:05.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.209+0000 7fd722d9d640 1 --2- 172.21.15.45:0/922454131 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd724070d20 0x7fd7240711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.209+0000 7fd72159a640 1 -- 172.21.15.45:0/922454131 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1690199350 0 0) 0x7fd72406e110 con 0x7fd7240b2490 2024-05-08T05:51:05.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.209+0000 7fd72159a640 1 -- 172.21.15.45:0/922454131 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd70c003520 con 0x7fd7240b2490 2024-05-08T05:51:05.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.210+0000 7fd72159a640 1 -- 172.21.15.45:0/922454131 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3556546657 0 0) 0x7fd72406e3f0 con 0x7fd7241629b0 2024-05-08T05:51:05.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.210+0000 7fd722d9d640 1 -- 172.21.15.45:0/922454131 >> v1:172.21.15.63:6789/0 conn(0x7fd7241629b0 legacy=0x7fd72406eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.210+0000 7fd722d9d640 1 -- 172.21.15.45:0/922454131 >> v1:172.21.15.45:6789/0 conn(0x7fd7240b2490 legacy=0x7fd724070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.210+0000 7fd722d9d640 1 -- 172.21.15.45:0/922454131 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72406e6d0 con 0x7fd724070d20 2024-05-08T05:51:05.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.210+0000 7fd722d9d640 1 --2- 172.21.15.45:0/922454131 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd724070d20 0x7fd7240711d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009dc0 tx=0x7fd71c02a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22c3173be14a9c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:05.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.211+0000 7fd72159a640 1 -- 172.21.15.45:0/922454131 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c035020 con 0x7fd724070d20 2024-05-08T05:51:05.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.211+0000 7fd72159a640 1 -- 172.21.15.45:0/922454131 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd71c007540 con 0x7fd724070d20 2024-05-08T05:51:05.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.211+0000 7fd72159a640 1 -- 172.21.15.45:0/922454131 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd718001670 con 0x7fd724070d20 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.211+0000 7fd7288f8640 1 -- 172.21.15.45:0/922454131 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd724070d20 msgr2=0x7fd7240711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.211+0000 7fd7288f8640 1 --2- 172.21.15.45:0/922454131 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd724070d20 0x7fd7240711d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009dc0 tx=0x7fd71c02a640 comp rx=0 tx=0).stop 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.211+0000 7fd7288f8640 1 -- 172.21.15.45:0/922454131 shutdown_connections 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.211+0000 7fd7288f8640 1 --2- 172.21.15.45:0/922454131 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd724070d20 0x7fd7240711d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.211+0000 7fd7288f8640 1 -- 172.21.15.45:0/922454131 >> 172.21.15.45:0/922454131 conn(0x7fd7240b49d0 msgr2=0x7fd7240b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.211+0000 7fd7288f8640 1 -- 172.21.15.45:0/922454131 shutdown_connections 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.211+0000 7fd7288f8640 1 -- 172.21.15.45:0/922454131 wait complete. 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.213+0000 7fd7288f8640 1 Processor -- start 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.213+0000 7fd7288f8640 1 -- start start 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.213+0000 7fd7288f8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd724070d20 0x7fd72414c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.213+0000 7fd7288f8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7240b2490 0x7fd7240b7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.213+0000 7fd7288f8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7241629b0 0x7fd72414ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.213+0000 7fd7288f8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd7240afcb0 con 0x7fd724070d20 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.213+0000 7fd7288f8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd72406dec0 con 0x7fd7240b2490 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.213+0000 7fd7288f8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd7240afe30 con 0x7fd7241629b0 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.214+0000 7fd72259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd724070d20 0x7fd72414c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.214+0000 7fd72259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd724070d20 0x7fd72414c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37228/0 (socket says 172.21.15.45:37228) 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.214+0000 7fd72259c640 1 -- 172.21.15.45:0/185473396 learned_addr learned my addr 172.21.15.45:0/185473396 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.214+0000 7fd722d9d640 1 --2- 172.21.15.45:0/185473396 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7241629b0 0x7fd72414ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.214+0000 7fd721d9b640 1 --2- 172.21.15.45:0/185473396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7240b2490 0x7fd7240b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.214+0000 7fd72259c640 1 -- 172.21.15.45:0/185473396 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7241629b0 msgr2=0x7fd72414ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.214+0000 7fd72259c640 1 --2- 172.21.15.45:0/185473396 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7241629b0 0x7fd72414ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.214+0000 7fd72259c640 1 -- 172.21.15.45:0/185473396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7240b2490 msgr2=0x7fd7240b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.214+0000 7fd72259c640 1 --2- 172.21.15.45:0/185473396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7240b2490 0x7fd7240b7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.214+0000 7fd72259c640 1 -- 172.21.15.45:0/185473396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7240b8180 con 0x7fd724070d20 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.214+0000 7fd721d9b640 1 --2- 172.21.15.45:0/185473396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7240b2490 0x7fd7240b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:05.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.214+0000 7fd72259c640 1 --2- 172.21.15.45:0/185473396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd724070d20 0x7fd72414c550 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fd71400c450 tx=0x7fd71400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:05.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.214+0000 7fd712ffd640 1 -- 172.21.15.45:0/185473396 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714017070 con 0x7fd724070d20 2024-05-08T05:51:05.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.214+0000 7fd712ffd640 1 -- 172.21.15.45:0/185473396 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd714005480 con 0x7fd724070d20 2024-05-08T05:51:05.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.214+0000 7fd7288f8640 1 -- 172.21.15.45:0/185473396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7240b8460 con 0x7fd724070d20 2024-05-08T05:51:05.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.214+0000 7fd712ffd640 1 -- 172.21.15.45:0/185473396 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714003830 con 0x7fd724070d20 2024-05-08T05:51:05.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.214+0000 7fd7288f8640 1 -- 172.21.15.45:0/185473396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7240b8a20 con 0x7fd724070d20 2024-05-08T05:51:05.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.215+0000 7fd7288f8640 1 -- 172.21.15.45:0/185473396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7240693f0 con 0x7fd724070d20 2024-05-08T05:51:05.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.216+0000 7fd712ffd640 1 -- 172.21.15.45:0/185473396 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd71401a070 con 0x7fd724070d20 2024-05-08T05:51:05.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.216+0000 7fd712ffd640 1 --2- 172.21.15.45:0/185473396 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7040762d0 0x7fd704078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.217+0000 7fd712ffd640 1 -- 172.21.15.45:0/185473396 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fd7140969f0 con 0x7fd724070d20 2024-05-08T05:51:05.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.217+0000 7fd721d9b640 1 --2- 172.21.15.45:0/185473396 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7040762d0 0x7fd704078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.217+0000 7fd721d9b640 1 --2- 172.21.15.45:0/185473396 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7040762d0 0x7fd704078790 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd7240d04f0 tx=0x7fd718001c30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:05.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.221+0000 7fd712ffd640 1 -- 172.21.15.45:0/185473396 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd71405fae0 con 0x7fd724070d20 2024-05-08T05:51:05.283 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-05-08T05:51:05.301 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:51:05.304 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:51:05.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.414+0000 7fd7288f8640 1 -- 172.21.15.45:0/185473396 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd724076460 con 0x7fd724070d20 2024-05-08T05:51:05.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.416+0000 7fd712ffd640 1 -- 172.21.15.45:0/185473396 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16917 (secure 0 0 0) 0x7fd714068120 con 0x7fd724070d20 2024-05-08T05:51:05.416 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:05.416 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":118,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:51:04.429390+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:51:02.094088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-08T05:51:03.388702+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:51:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.418+0000 7fd7288f8640 1 -- 172.21.15.45:0/185473396 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7040762d0 msgr2=0x7fd704078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.418+0000 7fd7288f8640 1 --2- 172.21.15.45:0/185473396 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7040762d0 0x7fd704078790 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd7240d04f0 tx=0x7fd718001c30 comp rx=0 tx=0).stop 2024-05-08T05:51:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.418+0000 7fd7288f8640 1 -- 172.21.15.45:0/185473396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd724070d20 msgr2=0x7fd72414c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.418+0000 7fd7288f8640 1 --2- 172.21.15.45:0/185473396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd724070d20 0x7fd72414c550 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fd71400c450 tx=0x7fd71400c910 comp rx=0 tx=0).stop 2024-05-08T05:51:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.418+0000 7fd7288f8640 1 -- 172.21.15.45:0/185473396 shutdown_connections 2024-05-08T05:51:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.418+0000 7fd7288f8640 1 --2- 172.21.15.45:0/185473396 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd7040762d0 0x7fd704078790 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.418+0000 7fd7288f8640 1 --2- 172.21.15.45:0/185473396 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd7241629b0 0x7fd72414ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.418+0000 7fd7288f8640 1 --2- 172.21.15.45:0/185473396 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7240b2490 0x7fd7240b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.418+0000 7fd7288f8640 1 --2- 172.21.15.45:0/185473396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd724070d20 0x7fd72414c550 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.418+0000 7fd7288f8640 1 -- 172.21.15.45:0/185473396 >> 172.21.15.45:0/185473396 conn(0x7fd7240b49d0 msgr2=0x7fd724160fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.418+0000 7fd7288f8640 1 -- 172.21.15.45:0/185473396 shutdown_connections 2024-05-08T05:51:05.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.418+0000 7fd7288f8640 1 -- 172.21.15.45:0/185473396 wait complete. 2024-05-08T05:51:05.429 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:51:05.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.533+0000 7f0b47d24640 1 Processor -- start 2024-05-08T05:51:05.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.533+0000 7f0b47d24640 1 -- start start 2024-05-08T05:51:05.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.534+0000 7f0b47d24640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0b400b1050 0x7f0b40071720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.534+0000 7f0b47d24640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0b4015f680 0x7f0b400b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.534+0000 7f0b47d24640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b400b5bf0 0x7f0b4006c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.534+0000 7f0b47d24640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f0b4015fae0 con 0x7f0b4015f680 2024-05-08T05:51:05.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.534+0000 7f0b47d24640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0b40160600 con 0x7f0b400b1050 2024-05-08T05:51:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.534+0000 7f0b47d24640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0b4015fd60 con 0x7f0b400b5bf0 2024-05-08T05:51:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.535+0000 7f0b452bf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0b4015f680 0x7f0b400b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.535+0000 7f0b452bf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0b4015f680 0x7f0b400b56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37230/0 (socket says 172.21.15.45:37230) 2024-05-08T05:51:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.535+0000 7f0b452bf640 1 -- 172.21.15.45:0/2225225191 learned_addr learned my addr 172.21.15.45:0/2225225191 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.535+0000 7f0b462c1640 1 --2- 172.21.15.45:0/2225225191 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b400b5bf0 0x7f0b4006c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.535+0000 7f0b45ac0640 1 --2- 172.21.15.45:0/2225225191 >> v2:172.21.15.63:3300/0 conn(0x7f0b400b1050 0x7f0b40071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.535+0000 7f0b452bf640 1 -- 172.21.15.45:0/2225225191 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b400b5bf0 msgr2=0x7f0b4006c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.535+0000 7f0b452bf640 1 --2- 172.21.15.45:0/2225225191 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b400b5bf0 0x7f0b4006c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.535+0000 7f0b452bf640 1 -- 172.21.15.45:0/2225225191 >> v2:172.21.15.63:3300/0 conn(0x7f0b400b1050 msgr2=0x7f0b40071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.535+0000 7f0b452bf640 1 --2- 172.21.15.45:0/2225225191 >> v2:172.21.15.63:3300/0 conn(0x7f0b400b1050 0x7f0b40071720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.535+0000 7f0b452bf640 1 -- 172.21.15.45:0/2225225191 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b40160040 con 0x7f0b4015f680 2024-05-08T05:51:05.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.536+0000 7f0b452bf640 1 --2- 172.21.15.45:0/2225225191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b4015f680 0x7f0b400b56b0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f0b400b7a30 tx=0x7f0b3402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4087d66729fbf790 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.536+0000 7f0b44abe640 1 -- 172.21.15.45:0/2225225191 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34037070 con 0x7f0b4015f680 2024-05-08T05:51:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.536+0000 7f0b44abe640 1 -- 172.21.15.45:0/2225225191 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b34032440 con 0x7f0b4015f680 2024-05-08T05:51:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.536+0000 7f0b44abe640 1 -- 172.21.15.45:0/2225225191 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34032810 con 0x7f0b4015f680 2024-05-08T05:51:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.536+0000 7f0b47d24640 1 -- 172.21.15.45:0/2225225191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b4015f680 msgr2=0x7f0b400b56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.536+0000 7f0b47d24640 1 --2- 172.21.15.45:0/2225225191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b4015f680 0x7f0b400b56b0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f0b400b7a30 tx=0x7f0b3402a1e0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.536+0000 7f0b47d24640 1 -- 172.21.15.45:0/2225225191 shutdown_connections 2024-05-08T05:51:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.536+0000 7f0b47d24640 1 --2- 172.21.15.45:0/2225225191 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b400b5bf0 0x7f0b4006c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.536+0000 7f0b47d24640 1 --2- 172.21.15.45:0/2225225191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b4015f680 0x7f0b400b56b0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.536+0000 7f0b47d24640 1 --2- 172.21.15.45:0/2225225191 >> v2:172.21.15.63:3300/0 conn(0x7f0b400b1050 0x7f0b40071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.536+0000 7f0b47d24640 1 -- 172.21.15.45:0/2225225191 >> 172.21.15.45:0/2225225191 conn(0x7f0b400b3c40 msgr2=0x7f0b400b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:05.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.536+0000 7f0b47d24640 1 -- 172.21.15.45:0/2225225191 shutdown_connections 2024-05-08T05:51:05.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.537+0000 7f0b47d24640 1 -- 172.21.15.45:0/2225225191 wait complete. 2024-05-08T05:51:05.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.538+0000 7f0b47d24640 1 Processor -- start 2024-05-08T05:51:05.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.538+0000 7f0b47d24640 1 -- start start 2024-05-08T05:51:05.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.538+0000 7f0b47d24640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b400b1050 0x7f0b401462a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.538+0000 7f0b47d24640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b400b5bf0 0x7f0b40149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.538+0000 7f0b47d24640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4015f680 0x7f0b400afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.538+0000 7f0b47d24640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0b400701e0 con 0x7f0b400b5bf0 2024-05-08T05:51:05.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.538+0000 7f0b47d24640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0b40162d10 con 0x7f0b4015f680 2024-05-08T05:51:05.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.538+0000 7f0b47d24640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0b400b1430 con 0x7f0b400b1050 2024-05-08T05:51:05.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.539+0000 7f0b452bf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b400b5bf0 0x7f0b40149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.539+0000 7f0b452bf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b400b5bf0 0x7f0b40149f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37232/0 (socket says 172.21.15.45:37232) 2024-05-08T05:51:05.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.539+0000 7f0b452bf640 1 -- 172.21.15.45:0/2437916612 learned_addr learned my addr 172.21.15.45:0/2437916612 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:05.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.539+0000 7f0b462c1640 1 --2- 172.21.15.45:0/2437916612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4015f680 0x7f0b400afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.539+0000 7f0b45ac0640 1 --2- 172.21.15.45:0/2437916612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b400b1050 0x7f0b401462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.539+0000 7f0b452bf640 1 -- 172.21.15.45:0/2437916612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b400b1050 msgr2=0x7f0b401462a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.539+0000 7f0b452bf640 1 --2- 172.21.15.45:0/2437916612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b400b1050 0x7f0b401462a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.539+0000 7f0b452bf640 1 -- 172.21.15.45:0/2437916612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4015f680 msgr2=0x7f0b400afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.539+0000 7f0b452bf640 1 --2- 172.21.15.45:0/2437916612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4015f680 0x7f0b400afcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.539+0000 7f0b452bf640 1 -- 172.21.15.45:0/2437916612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4014e7d0 con 0x7f0b400b5bf0 2024-05-08T05:51:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.539+0000 7f0b45ac0640 1 --2- 172.21.15.45:0/2437916612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b400b1050 0x7f0b401462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.539+0000 7f0b452bf640 1 --2- 172.21.15.45:0/2437916612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b400b5bf0 0x7f0b40149f80 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f0b3402a220 tx=0x7f0b340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.539+0000 7f0b3e7fc640 1 -- 172.21.15.45:0/2437916612 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34041020 con 0x7f0b400b5bf0 2024-05-08T05:51:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.539+0000 7f0b3e7fc640 1 -- 172.21.15.45:0/2437916612 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b340306c0 con 0x7f0b400b5bf0 2024-05-08T05:51:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.539+0000 7f0b3e7fc640 1 -- 172.21.15.45:0/2437916612 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34037040 con 0x7f0b400b5bf0 2024-05-08T05:51:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.539+0000 7f0b47d24640 1 -- 172.21.15.45:0/2437916612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b401522c0 con 0x7f0b400b5bf0 2024-05-08T05:51:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.539+0000 7f0b47d24640 1 -- 172.21.15.45:0/2437916612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b40152880 con 0x7f0b400b5bf0 2024-05-08T05:51:05.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.540+0000 7f0b47d24640 1 -- 172.21.15.45:0/2437916612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b40069b00 con 0x7f0b400b5bf0 2024-05-08T05:51:05.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.541+0000 7f0b3e7fc640 1 -- 172.21.15.45:0/2437916612 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0b34044020 con 0x7f0b400b5bf0 2024-05-08T05:51:05.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.542+0000 7f0b3e7fc640 1 --2- 172.21.15.45:0/2437916612 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0b2c0762d0 0x7f0b2c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.542+0000 7f0b3e7fc640 1 -- 172.21.15.45:0/2437916612 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f0b340b7f30 con 0x7f0b400b5bf0 2024-05-08T05:51:05.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.542+0000 7f0b45ac0640 1 --2- 172.21.15.45:0/2437916612 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0b2c0762d0 0x7f0b2c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.543 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.542+0000 7f0b45ac0640 1 --2- 172.21.15.45:0/2437916612 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0b2c0762d0 0x7f0b2c078790 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0b38006fd0 tx=0x7f0b38008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:05.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.546+0000 7f0b3e7fc640 1 -- 172.21.15.45:0/2437916612 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0b3404b030 con 0x7f0b400b5bf0 2024-05-08T05:51:05.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.740+0000 7f0b47d24640 1 -- 172.21.15.45:0/2437916612 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b400b0e30 con 0x7f0b400b5bf0 2024-05-08T05:51:05.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.741+0000 7f0b3e7fc640 1 -- 172.21.15.45:0/2437916612 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16917 (secure 0 0 0) 0x7f0b340816f0 con 0x7f0b400b5bf0 2024-05-08T05:51:05.741 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:05.741 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":118,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:51:04.429390+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:51:02.094088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-08T05:51:03.388702+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:51:05.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.743+0000 7f0b47d24640 1 -- 172.21.15.45:0/2437916612 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0b2c0762d0 msgr2=0x7f0b2c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.743+0000 7f0b47d24640 1 --2- 172.21.15.45:0/2437916612 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0b2c0762d0 0x7f0b2c078790 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0b38006fd0 tx=0x7f0b38008040 comp rx=0 tx=0).stop 2024-05-08T05:51:05.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.743+0000 7f0b47d24640 1 -- 172.21.15.45:0/2437916612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b400b5bf0 msgr2=0x7f0b40149f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.743+0000 7f0b47d24640 1 --2- 172.21.15.45:0/2437916612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b400b5bf0 0x7f0b40149f80 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f0b3402a220 tx=0x7f0b340046c0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.743+0000 7f0b47d24640 1 -- 172.21.15.45:0/2437916612 shutdown_connections 2024-05-08T05:51:05.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.743+0000 7f0b47d24640 1 --2- 172.21.15.45:0/2437916612 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0b2c0762d0 0x7f0b2c078790 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.743+0000 7f0b47d24640 1 --2- 172.21.15.45:0/2437916612 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0b4015f680 0x7f0b400afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.743+0000 7f0b47d24640 1 --2- 172.21.15.45:0/2437916612 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0b400b5bf0 0x7f0b40149f80 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.743+0000 7f0b47d24640 1 --2- 172.21.15.45:0/2437916612 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0b400b1050 0x7f0b401462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.743+0000 7f0b47d24640 1 -- 172.21.15.45:0/2437916612 >> 172.21.15.45:0/2437916612 conn(0x7f0b400b3c40 msgr2=0x7f0b400b6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:05.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.743+0000 7f0b47d24640 1 -- 172.21.15.45:0/2437916612 shutdown_connections 2024-05-08T05:51:05.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.743+0000 7f0b47d24640 1 -- 172.21.15.45:0/2437916612 wait complete. 2024-05-08T05:51:05.754 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T05:51:05.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.861+0000 7faa1fc85640 1 Processor -- start 2024-05-08T05:51:05.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.861+0000 7faa1fc85640 1 -- start start 2024-05-08T05:51:05.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.862+0000 7faa1fc85640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa1815bc80 0x7faa1806e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.862+0000 7faa1fc85640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faa1806e8d0 0x7faa18073700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.862+0000 7faa1fc85640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa180b4590 con 0x7faa181524f0 2024-05-08T05:51:05.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.862+0000 7faa1fc85640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7faa180b51a0 con 0x7faa1806e8d0 2024-05-08T05:51:05.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.862+0000 7faa1fc85640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faa18073450 con 0x7faa1815bc80 2024-05-08T05:51:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.862+0000 7faa1e222640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faa1806e8d0 0x7faa18073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.862+0000 7faa1e222640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7faa1806e8d0 0x7faa18073700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37240/0 (socket says 172.21.15.45:37240) 2024-05-08T05:51:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.862+0000 7faa1e222640 1 -- 172.21.15.45:0/2646167049 learned_addr learned my addr 172.21.15.45:0/2646167049 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.862+0000 7faa1d220640 1 --2- 172.21.15.45:0/2646167049 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa1815bc80 0x7faa1806e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.862+0000 7faa1e222640 1 -- 172.21.15.45:0/2646167049 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa1815bc80 msgr2=0x7faa1806e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.862+0000 7faa1e222640 1 --2- 172.21.15.45:0/2646167049 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa1815bc80 0x7faa1806e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.862+0000 7faa1e222640 1 -- 172.21.15.45:0/2646167049 >> v1:172.21.15.63:6789/0 conn(0x7faa181524f0 legacy=0x7faa180b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:05.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.862+0000 7faa1e222640 1 -- 172.21.15.45:0/2646167049 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa180b4870 con 0x7faa1806e8d0 2024-05-08T05:51:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.863+0000 7faa1e222640 1 --2- 172.21.15.45:0/2646167049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1806e8d0 0x7faa18073700 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7faa10009e50 tx=0x7faa1002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9831c1bf74be3b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.863+0000 7faa1ca1f640 1 -- 172.21.15.45:0/2646167049 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10036070 con 0x7faa1806e8d0 2024-05-08T05:51:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.863+0000 7faa1ca1f640 1 -- 172.21.15.45:0/2646167049 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa10031600 con 0x7faa1806e8d0 2024-05-08T05:51:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.863+0000 7faa1ca1f640 1 -- 172.21.15.45:0/2646167049 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa100319d0 con 0x7faa1806e8d0 2024-05-08T05:51:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.863+0000 7faa1fc85640 1 -- 172.21.15.45:0/2646167049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1806e8d0 msgr2=0x7faa18073700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.863+0000 7faa1fc85640 1 --2- 172.21.15.45:0/2646167049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1806e8d0 0x7faa18073700 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7faa10009e50 tx=0x7faa1002a710 comp rx=0 tx=0).stop 2024-05-08T05:51:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.864+0000 7faa1fc85640 1 -- 172.21.15.45:0/2646167049 shutdown_connections 2024-05-08T05:51:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.864+0000 7faa1fc85640 1 --2- 172.21.15.45:0/2646167049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1806e8d0 0x7faa18073700 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.864+0000 7faa1fc85640 1 --2- 172.21.15.45:0/2646167049 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa1815bc80 0x7faa1806e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.864+0000 7faa1fc85640 1 -- 172.21.15.45:0/2646167049 >> 172.21.15.45:0/2646167049 conn(0x7faa1806c3a0 msgr2=0x7faa1806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.864+0000 7faa1fc85640 1 -- 172.21.15.45:0/2646167049 shutdown_connections 2024-05-08T05:51:05.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.864+0000 7faa1fc85640 1 -- 172.21.15.45:0/2646167049 wait complete. 2024-05-08T05:51:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.865+0000 7faa1fc85640 1 Processor -- start 2024-05-08T05:51:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.865+0000 7faa1fc85640 1 -- start start 2024-05-08T05:51:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa1fc85640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa1806e8d0 0x7faa18149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa1fc85640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa181524f0 0x7faa180ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa1fc85640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1815bc80 0x7faa181596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa1fc85640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faa180b4e30 con 0x7faa1815bc80 2024-05-08T05:51:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa1fc85640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7faa180b4330 con 0x7faa181524f0 2024-05-08T05:51:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa1fc85640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7faa181528d0 con 0x7faa1806e8d0 2024-05-08T05:51:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa1e222640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1815bc80 0x7faa181596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa1e222640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1815bc80 0x7faa181596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37256/0 (socket says 172.21.15.45:37256) 2024-05-08T05:51:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa1e222640 1 -- 172.21.15.45:0/1779277213 learned_addr learned my addr 172.21.15.45:0/1779277213 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa1da21640 1 --2- 172.21.15.45:0/1779277213 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa1806e8d0 0x7faa18149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa1d220640 1 --2- 172.21.15.45:0/1779277213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa181524f0 0x7faa180ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa1e222640 1 -- 172.21.15.45:0/1779277213 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa1806e8d0 msgr2=0x7faa18149db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa1e222640 1 --2- 172.21.15.45:0/1779277213 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa1806e8d0 0x7faa18149db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa1e222640 1 -- 172.21.15.45:0/1779277213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa181524f0 msgr2=0x7faa180ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa1e222640 1 --2- 172.21.15.45:0/1779277213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa181524f0 0x7faa180ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa1e222640 1 -- 172.21.15.45:0/1779277213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa180b3420 con 0x7faa1815bc80 2024-05-08T05:51:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa1da21640 1 --2- 172.21.15.45:0/1779277213 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa1806e8d0 0x7faa18149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa1d220640 1 --2- 172.21.15.45:0/1779277213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa181524f0 0x7faa180ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:05.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa1e222640 1 --2- 172.21.15.45:0/1779277213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1815bc80 0x7faa181596c0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7faa10034040 tx=0x7faa10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:05.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa0e7fc640 1 -- 172.21.15.45:0/1779277213 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10036050 con 0x7faa1815bc80 2024-05-08T05:51:05.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa0e7fc640 1 -- 172.21.15.45:0/1779277213 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa10042070 con 0x7faa1815bc80 2024-05-08T05:51:05.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa0e7fc640 1 -- 172.21.15.45:0/1779277213 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10009510 con 0x7faa1815bc80 2024-05-08T05:51:05.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.866+0000 7faa1fc85640 1 -- 172.21.15.45:0/1779277213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa18157100 con 0x7faa1815bc80 2024-05-08T05:51:05.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.867+0000 7faa1fc85640 1 -- 172.21.15.45:0/1779277213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa181576c0 con 0x7faa1815bc80 2024-05-08T05:51:05.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.867+0000 7faa1fc85640 1 -- 172.21.15.45:0/1779277213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa1806f310 con 0x7faa1815bc80 2024-05-08T05:51:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.868+0000 7faa0e7fc640 1 -- 172.21.15.45:0/1779277213 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7faa10007c50 con 0x7faa1815bc80 2024-05-08T05:51:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.869+0000 7faa0e7fc640 1 --2- 172.21.15.45:0/1779277213 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa9fc0762d0 0x7fa9fc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.869+0000 7faa0e7fc640 1 -- 172.21.15.45:0/1779277213 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7faa100b85d0 con 0x7faa1815bc80 2024-05-08T05:51:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.869+0000 7faa1da21640 1 --2- 172.21.15.45:0/1779277213 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa9fc0762d0 0x7fa9fc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.869+0000 7faa1da21640 1 --2- 172.21.15.45:0/1779277213 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa9fc0762d0 0x7fa9fc078790 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7faa1815a2c0 tx=0x7faa14009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:05.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:05.873+0000 7faa0e7fc640 1 -- 172.21.15.45:0/1779277213 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7faa1806f310 con 0x7faa1815bc80 2024-05-08T05:51:06.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:06.104+0000 7faa1fc85640 1 -- 172.21.15.45:0/1779277213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7faa18072220 con 0x7faa1815bc80 2024-05-08T05:51:07.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.081+0000 7faa0e7fc640 1 -- 172.21.15.45:0/1779277213 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v99) ==== 118+0+0 (secure 0 0 0) 0x7faa10081e30 con 0x7faa1815bc80 2024-05-08T05:51:07.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.084+0000 7faa1fc85640 1 -- 172.21.15.45:0/1779277213 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa9fc0762d0 msgr2=0x7fa9fc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:07.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.084+0000 7faa1fc85640 1 --2- 172.21.15.45:0/1779277213 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa9fc0762d0 0x7fa9fc078790 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7faa1815a2c0 tx=0x7faa14009040 comp rx=0 tx=0).stop 2024-05-08T05:51:07.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.084+0000 7faa1fc85640 1 -- 172.21.15.45:0/1779277213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1815bc80 msgr2=0x7faa181596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:07.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.084+0000 7faa1fc85640 1 --2- 172.21.15.45:0/1779277213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1815bc80 0x7faa181596c0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7faa10034040 tx=0x7faa10004a40 comp rx=0 tx=0).stop 2024-05-08T05:51:07.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.084+0000 7faa1fc85640 1 -- 172.21.15.45:0/1779277213 shutdown_connections 2024-05-08T05:51:07.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.084+0000 7faa1fc85640 1 --2- 172.21.15.45:0/1779277213 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa9fc0762d0 0x7fa9fc078790 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:07.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.084+0000 7faa1fc85640 1 --2- 172.21.15.45:0/1779277213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faa1815bc80 0x7faa181596c0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:07.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.084+0000 7faa1fc85640 1 --2- 172.21.15.45:0/1779277213 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa181524f0 0x7faa180ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:07.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.084+0000 7faa1fc85640 1 --2- 172.21.15.45:0/1779277213 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7faa1806e8d0 0x7faa18149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:07.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.084+0000 7faa1fc85640 1 -- 172.21.15.45:0/1779277213 >> 172.21.15.45:0/1779277213 conn(0x7faa1806c3a0 msgr2=0x7faa180d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:07.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.084+0000 7faa1fc85640 1 -- 172.21.15.45:0/1779277213 shutdown_connections 2024-05-08T05:51:07.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.084+0000 7faa1fc85640 1 -- 172.21.15.45:0/1779277213 wait complete. 2024-05-08T05:51:07.097 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-08T05:51:07.098 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-08T05:51:07.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.211+0000 7f359cf7f640 1 Processor -- start 2024-05-08T05:51:07.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.211+0000 7f359cf7f640 1 -- start start 2024-05-08T05:51:07.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.212+0000 7f359cf7f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f359815f380 0x7f35980d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.212+0000 7f359cf7f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35980b1c90 con 0x7f35980b3f30 2024-05-08T05:51:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.212+0000 7f359cf7f640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35980d0a50 con 0x7f35980d2600 2024-05-08T05:51:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.212+0000 7f359cf7f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f35980d2a00 con 0x7f359815f380 2024-05-08T05:51:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.212+0000 7f359659c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f359815f380 0x7f35980d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.212+0000 7f3595d9b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f35980b3f30 0x7f35980b2420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52786/0 (socket says 172.21.15.45:52786) 2024-05-08T05:51:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.212+0000 7f3595d9b640 1 -- 172.21.15.45:0/2070176432 learned_addr learned my addr 172.21.15.45:0/2070176432 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.212+0000 7f359659c640 1 -- 172.21.15.45:0/2070176432 >> v1:172.21.15.63:6789/0 conn(0x7f35980d2600 legacy=0x7f35980b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.212+0000 7f359659c640 1 -- 172.21.15.45:0/2070176432 >> v1:172.21.15.45:6789/0 conn(0x7f35980b3f30 legacy=0x7f35980b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.212+0000 7f359659c640 1 -- 172.21.15.45:0/2070176432 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35980d0d30 con 0x7f359815f380 2024-05-08T05:51:07.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.212+0000 7f359559a640 1 -- 172.21.15.45:0/2070176432 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4093339198 0 0) 0x7f35980b1c90 con 0x7f35980b3f30 2024-05-08T05:51:07.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.213+0000 7f359659c640 1 --2- 172.21.15.45:0/2070176432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359815f380 0x7f35980d35c0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f35880097a0 tx=0x7f358802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29657dd3e6ba890b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.213+0000 7f359559a640 1 -- 172.21.15.45:0/2070176432 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588037070 con 0x7f359815f380 2024-05-08T05:51:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.213+0000 7f359559a640 1 -- 172.21.15.45:0/2070176432 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3588032440 con 0x7f359815f380 2024-05-08T05:51:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.213+0000 7f359559a640 1 -- 172.21.15.45:0/2070176432 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588032810 con 0x7f359815f380 2024-05-08T05:51:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.214+0000 7f359cf7f640 1 -- 172.21.15.45:0/2070176432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359815f380 msgr2=0x7f35980d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.214+0000 7f359cf7f640 1 --2- 172.21.15.45:0/2070176432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359815f380 0x7f35980d35c0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f35880097a0 tx=0x7f358802a180 comp rx=0 tx=0).stop 2024-05-08T05:51:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.214+0000 7f359cf7f640 1 -- 172.21.15.45:0/2070176432 shutdown_connections 2024-05-08T05:51:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.214+0000 7f359cf7f640 1 --2- 172.21.15.45:0/2070176432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359815f380 0x7f35980d35c0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.214+0000 7f359cf7f640 1 -- 172.21.15.45:0/2070176432 >> 172.21.15.45:0/2070176432 conn(0x7f35980d4fe0 msgr2=0x7f35980d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.214+0000 7f359cf7f640 1 -- 172.21.15.45:0/2070176432 shutdown_connections 2024-05-08T05:51:07.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.214+0000 7f359cf7f640 1 -- 172.21.15.45:0/2070176432 wait complete. 2024-05-08T05:51:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.215+0000 7f359cf7f640 1 Processor -- start 2024-05-08T05:51:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.215+0000 7f359cf7f640 1 -- start start 2024-05-08T05:51:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.216+0000 7f359cf7f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35980b3f30 0x7f35980ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.216+0000 7f359cf7f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35980d2600 0x7f35980ba700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.216+0000 7f359cf7f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359815f380 0x7f35980cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.216+0000 7f359cf7f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f35980d1010 con 0x7f359815f380 2024-05-08T05:51:07.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.216+0000 7f359cf7f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f359815f760 con 0x7f35980d2600 2024-05-08T05:51:07.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.216+0000 7f359cf7f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f35980d06e0 con 0x7f35980b3f30 2024-05-08T05:51:07.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.216+0000 7f3596d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359815f380 0x7f35980cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:07.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.216+0000 7f3596d9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359815f380 0x7f35980cdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37266/0 (socket says 172.21.15.45:37266) 2024-05-08T05:51:07.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.216+0000 7f3596d9d640 1 -- 172.21.15.45:0/2986534832 learned_addr learned my addr 172.21.15.45:0/2986534832 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:07.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.216+0000 7f3595d9b640 1 --2- 172.21.15.45:0/2986534832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35980d2600 0x7f35980ba700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:07.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.216+0000 7f359659c640 1 --2- 172.21.15.45:0/2986534832 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35980b3f30 0x7f35980ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:07.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.217+0000 7f3596d9d640 1 -- 172.21.15.45:0/2986534832 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35980b3f30 msgr2=0x7f35980ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:07.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.217+0000 7f3596d9d640 1 --2- 172.21.15.45:0/2986534832 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35980b3f30 0x7f35980ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:07.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.217+0000 7f3596d9d640 1 -- 172.21.15.45:0/2986534832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35980d2600 msgr2=0x7f35980ba700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:07.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.217+0000 7f3596d9d640 1 --2- 172.21.15.45:0/2986534832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35980d2600 0x7f35980ba700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:07.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.217+0000 7f3596d9d640 1 -- 172.21.15.45:0/2986534832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35980b7cf0 con 0x7f359815f380 2024-05-08T05:51:07.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.217+0000 7f3596d9d640 1 --2- 172.21.15.45:0/2986534832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359815f380 0x7f35980cdee0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f359000c780 tx=0x7f359000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.217+0000 7f3586ffd640 1 -- 172.21.15.45:0/2986534832 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3590017070 con 0x7f359815f380 2024-05-08T05:51:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.217+0000 7f3586ffd640 1 -- 172.21.15.45:0/2986534832 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3590003da0 con 0x7f359815f380 2024-05-08T05:51:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.217+0000 7f3586ffd640 1 -- 172.21.15.45:0/2986534832 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35900052b0 con 0x7f359815f380 2024-05-08T05:51:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.217+0000 7f359cf7f640 1 -- 172.21.15.45:0/2986534832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35980b7fd0 con 0x7f359815f380 2024-05-08T05:51:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.217+0000 7f359cf7f640 1 -- 172.21.15.45:0/2986534832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35980bac40 con 0x7f359815f380 2024-05-08T05:51:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.218+0000 7f359cf7f640 1 -- 172.21.15.45:0/2986534832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35980693f0 con 0x7f359815f380 2024-05-08T05:51:07.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.219+0000 7f3586ffd640 1 -- 172.21.15.45:0/2986534832 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f359001a070 con 0x7f359815f380 2024-05-08T05:51:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.219+0000 7f3586ffd640 1 --2- 172.21.15.45:0/2986534832 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3578076300 0x7f35780787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.219+0000 7f3586ffd640 1 -- 172.21.15.45:0/2986534832 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f3590096c50 con 0x7f359815f380 2024-05-08T05:51:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.220+0000 7f359659c640 1 --2- 172.21.15.45:0/2986534832 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3578076300 0x7f35780787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.220+0000 7f359659c640 1 --2- 172.21.15.45:0/2986534832 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3578076300 0x7f35780787c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3588034000 tx=0x7f358802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:07.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.224+0000 7f3586ffd640 1 -- 172.21.15.45:0/2986534832 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f359009c020 con 0x7f359815f380 2024-05-08T05:51:07.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:07.452+0000 7f359cf7f640 1 -- 172.21.15.45:0/2986534832 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f35980d56a0 con 0x7f359815f380 2024-05-08T05:51:08.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.081+0000 7f3586ffd640 1 -- 172.21.15.45:0/2986534832 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v100) ==== 122+0+0 (secure 0 0 0) 0x7f359005fe60 con 0x7f359815f380 2024-05-08T05:51:08.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.084+0000 7f359cf7f640 1 -- 172.21.15.45:0/2986534832 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3578076300 msgr2=0x7f35780787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.084+0000 7f359cf7f640 1 --2- 172.21.15.45:0/2986534832 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3578076300 0x7f35780787c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3588034000 tx=0x7f358802e000 comp rx=0 tx=0).stop 2024-05-08T05:51:08.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.084+0000 7f359cf7f640 1 -- 172.21.15.45:0/2986534832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359815f380 msgr2=0x7f35980cdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.084+0000 7f359cf7f640 1 --2- 172.21.15.45:0/2986534832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359815f380 0x7f35980cdee0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f359000c780 tx=0x7f359000cc40 comp rx=0 tx=0).stop 2024-05-08T05:51:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.084+0000 7f359cf7f640 1 -- 172.21.15.45:0/2986534832 shutdown_connections 2024-05-08T05:51:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.084+0000 7f359cf7f640 1 --2- 172.21.15.45:0/2986534832 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3578076300 0x7f35780787c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.084+0000 7f359cf7f640 1 --2- 172.21.15.45:0/2986534832 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f359815f380 0x7f35980cdee0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.084+0000 7f359cf7f640 1 --2- 172.21.15.45:0/2986534832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35980d2600 0x7f35980ba700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.084+0000 7f359cf7f640 1 --2- 172.21.15.45:0/2986534832 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f35980b3f30 0x7f35980ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.084+0000 7f359cf7f640 1 -- 172.21.15.45:0/2986534832 >> 172.21.15.45:0/2986534832 conn(0x7f35980d4fe0 msgr2=0x7f359815e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.084+0000 7f359cf7f640 1 -- 172.21.15.45:0/2986534832 shutdown_connections 2024-05-08T05:51:08.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.084+0000 7f359cf7f640 1 -- 172.21.15.45:0/2986534832 wait complete. 2024-05-08T05:51:08.099 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-08T05:51:08.099 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:51:08.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.208+0000 7fa63034e640 1 Processor -- start 2024-05-08T05:51:08.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.208+0000 7fa63034e640 1 -- start start 2024-05-08T05:51:08.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.209+0000 7fa63034e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6280d1970 0x7fa6280d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.209+0000 7fa63034e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa6280d4cc0 0x7fa6280d5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.209+0000 7fa63034e640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6281630b0 con 0x7fa62815f420 2024-05-08T05:51:08.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.209+0000 7fa63034e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa6280d2330 con 0x7fa6280d4cc0 2024-05-08T05:51:08.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.209+0000 7fa63034e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa6280d3e70 con 0x7fa6280d1970 2024-05-08T05:51:08.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.210+0000 7fa62e8eb640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa6280d4cc0 0x7fa6280d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.210+0000 7fa62d8e9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6280d1970 0x7fa6280d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.210+0000 7fa62e0ea640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fa62815f420 0x7fa6280d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:36852/0 (socket says 172.21.15.45:36852) 2024-05-08T05:51:08.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.210+0000 7fa62d8e9640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6280d1970 0x7fa6280d1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:42596/0 (socket says 172.21.15.45:42596) 2024-05-08T05:51:08.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.210+0000 7fa62e8eb640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa6280d4cc0 0x7fa6280d5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:43360/0 (socket says 172.21.15.45:43360) 2024-05-08T05:51:08.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.210+0000 7fa62d8e9640 1 -- 172.21.15.45:0/2040516027 learned_addr learned my addr 172.21.15.45:0/2040516027 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:08.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.210+0000 7fa62d8e9640 1 -- 172.21.15.45:0/2040516027 >> v2:172.21.15.63:3300/0 conn(0x7fa6280d4cc0 msgr2=0x7fa6280d5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:08.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.210+0000 7fa62d8e9640 1 --2- 172.21.15.45:0/2040516027 >> v2:172.21.15.63:3300/0 conn(0x7fa6280d4cc0 0x7fa6280d5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.210+0000 7fa62d8e9640 1 -- 172.21.15.45:0/2040516027 >> v1:172.21.15.63:6789/0 conn(0x7fa62815f420 legacy=0x7fa6280d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.210+0000 7fa62d8e9640 1 -- 172.21.15.45:0/2040516027 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6280d5670 con 0x7fa6280d1970 2024-05-08T05:51:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.211+0000 7fa62d8e9640 1 --2- 172.21.15.45:0/2040516027 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6280d1970 0x7fa6280d1df0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009870 tx=0x7fa61c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24a261cd2efcd522 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.211+0000 7fa62d0e8640 1 -- 172.21.15.45:0/2040516027 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c037070 con 0x7fa6280d1970 2024-05-08T05:51:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.211+0000 7fa62d0e8640 1 -- 172.21.15.45:0/2040516027 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa61c032440 con 0x7fa6280d1970 2024-05-08T05:51:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.211+0000 7fa62d0e8640 1 -- 172.21.15.45:0/2040516027 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c032810 con 0x7fa6280d1970 2024-05-08T05:51:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.212+0000 7fa63034e640 1 -- 172.21.15.45:0/2040516027 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6280d1970 msgr2=0x7fa6280d1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.212+0000 7fa63034e640 1 --2- 172.21.15.45:0/2040516027 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6280d1970 0x7fa6280d1df0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009870 tx=0x7fa61c02a270 comp rx=0 tx=0).stop 2024-05-08T05:51:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.212+0000 7fa63034e640 1 -- 172.21.15.45:0/2040516027 shutdown_connections 2024-05-08T05:51:08.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.212+0000 7fa63034e640 1 --2- 172.21.15.45:0/2040516027 >> v2:172.21.15.63:3300/0 conn(0x7fa6280d4cc0 0x7fa6280d5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.212+0000 7fa63034e640 1 --2- 172.21.15.45:0/2040516027 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa6280d1970 0x7fa6280d1df0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.212+0000 7fa63034e640 1 -- 172.21.15.45:0/2040516027 >> 172.21.15.45:0/2040516027 conn(0x7fa62808eac0 msgr2=0x7fa62809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:08.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.212+0000 7fa63034e640 1 -- 172.21.15.45:0/2040516027 shutdown_connections 2024-05-08T05:51:08.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.212+0000 7fa63034e640 1 -- 172.21.15.45:0/2040516027 wait complete. 2024-05-08T05:51:08.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.213+0000 7fa63034e640 1 Processor -- start 2024-05-08T05:51:08.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.213+0000 7fa63034e640 1 -- start start 2024-05-08T05:51:08.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.214+0000 7fa63034e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6280d1970 0x7fa6280ba2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.214+0000 7fa63034e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6280d4cc0 0x7fa62815db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.214+0000 7fa63034e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa62815e070 0x7fa6281e8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.214+0000 7fa63034e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa6280d2f60 con 0x7fa6280d1970 2024-05-08T05:51:08.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.214+0000 7fa63034e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa6280d2c80 con 0x7fa6280d4cc0 2024-05-08T05:51:08.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.214+0000 7fa63034e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa6280d3240 con 0x7fa62815e070 2024-05-08T05:51:08.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.214+0000 7fa62e0ea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6280d1970 0x7fa6280ba2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.214+0000 7fa62e0ea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6280d1970 0x7fa6280ba2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37274/0 (socket says 172.21.15.45:37274) 2024-05-08T05:51:08.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.214+0000 7fa62e0ea640 1 -- 172.21.15.45:0/303447979 learned_addr learned my addr 172.21.15.45:0/303447979 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:08.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.214+0000 7fa62d8e9640 1 --2- 172.21.15.45:0/303447979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6280d4cc0 0x7fa62815db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.214+0000 7fa62e8eb640 1 --2- 172.21.15.45:0/303447979 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa62815e070 0x7fa6281e8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.214+0000 7fa62e0ea640 1 -- 172.21.15.45:0/303447979 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa62815e070 msgr2=0x7fa6281e8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.214+0000 7fa62e0ea640 1 --2- 172.21.15.45:0/303447979 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa62815e070 0x7fa6281e8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.214+0000 7fa62e0ea640 1 -- 172.21.15.45:0/303447979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6280d4cc0 msgr2=0x7fa62815db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.214+0000 7fa62e0ea640 1 --2- 172.21.15.45:0/303447979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6280d4cc0 0x7fa62815db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.214+0000 7fa62e0ea640 1 -- 172.21.15.45:0/303447979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6281e9800 con 0x7fa6280d1970 2024-05-08T05:51:08.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.215+0000 7fa62e0ea640 1 --2- 172.21.15.45:0/303447979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6280d1970 0x7fa6280ba2d0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa61800c3a0 tx=0x7fa61800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:08.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.215+0000 7fa626ffd640 1 -- 172.21.15.45:0/303447979 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6180074e0 con 0x7fa6280d1970 2024-05-08T05:51:08.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.215+0000 7fa626ffd640 1 -- 172.21.15.45:0/303447979 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6180047e0 con 0x7fa6280d1970 2024-05-08T05:51:08.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.215+0000 7fa63034e640 1 -- 172.21.15.45:0/303447979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6281e9ae0 con 0x7fa6280d1970 2024-05-08T05:51:08.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.215+0000 7fa626ffd640 1 -- 172.21.15.45:0/303447979 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa618004bd0 con 0x7fa6280d1970 2024-05-08T05:51:08.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.215+0000 7fa63034e640 1 -- 172.21.15.45:0/303447979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6280b7ee0 con 0x7fa6280d1970 2024-05-08T05:51:08.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.216+0000 7fa63034e640 1 -- 172.21.15.45:0/303447979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6280693f0 con 0x7fa6280d1970 2024-05-08T05:51:08.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.216+0000 7fa626ffd640 1 -- 172.21.15.45:0/303447979 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa61801a020 con 0x7fa6280d1970 2024-05-08T05:51:08.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.217+0000 7fa626ffd640 1 --2- 172.21.15.45:0/303447979 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa6140762d0 0x7fa614078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.217+0000 7fa626ffd640 1 -- 172.21.15.45:0/303447979 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7fa618096db0 con 0x7fa6280d1970 2024-05-08T05:51:08.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.217+0000 7fa62d8e9640 1 --2- 172.21.15.45:0/303447979 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa6140762d0 0x7fa614078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.218+0000 7fa62d8e9640 1 --2- 172.21.15.45:0/303447979 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa6140762d0 0x7fa614078790 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa61c034000 tx=0x7fa61c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:08.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.221+0000 7fa626ffd640 1 -- 172.21.15.45:0/303447979 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa6281630b0 con 0x7fa6280d1970 2024-05-08T05:51:08.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.451+0000 7fa63034e640 1 -- 172.21.15.45:0/303447979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6280b4c50 con 0x7fa6280d1970 2024-05-08T05:51:08.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.452+0000 7fa626ffd640 1 -- 172.21.15.45:0/303447979 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5185 (secure 0 0 0) 0x7fa61805ff30 con 0x7fa6280d1970 2024-05-08T05:51:08.453 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:08.453 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7168,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3882445426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3882445426},{"type":"v1","addr":"172.21.15.63:6836","nonce":3882445426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/3173207720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3173207720},{"type":"v1","addr":"172.21.15.45:6833","nonce":3173207720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4254619360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4254619360},{"type":"v1","addr":"172.21.15.63:6837","nonce":4254619360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:51:04.429358+0000","modified":"2024-05-08T05:51:07.454864+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8112},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8112":{"gid":8112,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1751419054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1751419054},{"type":"v1","addr":"172.21.15.45:6835","nonce":1751419054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8112,"qdb_cluster":[8112]},"id":8}]} 2024-05-08T05:51:08.453 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 100 2024-05-08T05:51:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.454+0000 7fa63034e640 1 -- 172.21.15.45:0/303447979 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa6140762d0 msgr2=0x7fa614078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.454+0000 7fa63034e640 1 --2- 172.21.15.45:0/303447979 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa6140762d0 0x7fa614078790 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa61c034000 tx=0x7fa61c02e000 comp rx=0 tx=0).stop 2024-05-08T05:51:08.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.454+0000 7fa63034e640 1 -- 172.21.15.45:0/303447979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6280d1970 msgr2=0x7fa6280ba2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.454+0000 7fa63034e640 1 --2- 172.21.15.45:0/303447979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6280d1970 0x7fa6280ba2d0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa61800c3a0 tx=0x7fa61800c860 comp rx=0 tx=0).stop 2024-05-08T05:51:08.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.454+0000 7fa63034e640 1 -- 172.21.15.45:0/303447979 shutdown_connections 2024-05-08T05:51:08.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.454+0000 7fa63034e640 1 --2- 172.21.15.45:0/303447979 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa6140762d0 0x7fa614078790 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.454+0000 7fa63034e640 1 --2- 172.21.15.45:0/303447979 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa62815e070 0x7fa6281e8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.454+0000 7fa63034e640 1 --2- 172.21.15.45:0/303447979 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa6280d4cc0 0x7fa62815db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.454+0000 7fa63034e640 1 --2- 172.21.15.45:0/303447979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6280d1970 0x7fa6280ba2d0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.454+0000 7fa63034e640 1 -- 172.21.15.45:0/303447979 >> 172.21.15.45:0/303447979 conn(0x7fa62808eac0 msgr2=0x7fa62806d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:08.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.454+0000 7fa63034e640 1 -- 172.21.15.45:0/303447979 shutdown_connections 2024-05-08T05:51:08.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.454+0000 7fa63034e640 1 -- 172.21.15.45:0/303447979 wait complete. 2024-05-08T05:51:08.466 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:51:08.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.568+0000 7fd4876f8640 1 Processor -- start 2024-05-08T05:51:08.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.569+0000 7fd4876f8640 1 -- start start 2024-05-08T05:51:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.569+0000 7fd4876f8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd48006f060 0x7fd48006e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.570+0000 7fd4876f8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd48006fe30 0x7fd4800cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.570+0000 7fd4876f8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd48015d2a0 con 0x7fd4800d0360 2024-05-08T05:51:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.570+0000 7fd4876f8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fd4800d07c0 con 0x7fd48006f060 2024-05-08T05:51:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.570+0000 7fd4876f8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd48006f440 con 0x7fd48006fe30 2024-05-08T05:51:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.570+0000 7fd485494640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd48006f060 0x7fd48006e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.570+0000 7fd485c95640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd4800d0360 0x7fd4800b3f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52796/0 (socket says 172.21.15.45:52796) 2024-05-08T05:51:08.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.570+0000 7fd485c95640 1 -- 172.21.15.45:0/341518023 learned_addr learned my addr 172.21.15.45:0/341518023 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:08.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.570+0000 7fd485494640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd48006f060 0x7fd48006e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37276/0 (socket says 172.21.15.45:37276) 2024-05-08T05:51:08.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.570+0000 7fd484c93640 1 --2- 172.21.15.45:0/341518023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd48006fe30 0x7fd4800cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.570+0000 7fd485494640 1 -- 172.21.15.45:0/341518023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd48006fe30 msgr2=0x7fd4800cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:08.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.570+0000 7fd485494640 1 --2- 172.21.15.45:0/341518023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd48006fe30 0x7fd4800cfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.570+0000 7fd485494640 1 -- 172.21.15.45:0/341518023 >> v1:172.21.15.45:6789/0 conn(0x7fd4800d0360 legacy=0x7fd4800b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.570+0000 7fd485494640 1 -- 172.21.15.45:0/341518023 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4801609b0 con 0x7fd48006f060 2024-05-08T05:51:08.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.571+0000 7fd485494640 1 --2- 172.21.15.45:0/341518023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd48006f060 0x7fd48006e0f0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0097a0 tx=0x7fd47c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8c68c6ba359477 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:08.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.571+0000 7fd473fff640 1 -- 172.21.15.45:0/341518023 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c037070 con 0x7fd48006f060 2024-05-08T05:51:08.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.571+0000 7fd473fff640 1 -- 172.21.15.45:0/341518023 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd47c032440 con 0x7fd48006f060 2024-05-08T05:51:08.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.571+0000 7fd473fff640 1 -- 172.21.15.45:0/341518023 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c032810 con 0x7fd48006f060 2024-05-08T05:51:08.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.571+0000 7fd4876f8640 1 -- 172.21.15.45:0/341518023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd48006f060 msgr2=0x7fd48006e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.571+0000 7fd4876f8640 1 --2- 172.21.15.45:0/341518023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd48006f060 0x7fd48006e0f0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0097a0 tx=0x7fd47c02a180 comp rx=0 tx=0).stop 2024-05-08T05:51:08.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.571+0000 7fd4876f8640 1 -- 172.21.15.45:0/341518023 shutdown_connections 2024-05-08T05:51:08.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.571+0000 7fd4876f8640 1 --2- 172.21.15.45:0/341518023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd48006fe30 0x7fd4800cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.571+0000 7fd4876f8640 1 --2- 172.21.15.45:0/341518023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd48006f060 0x7fd48006e0f0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.571+0000 7fd4876f8640 1 -- 172.21.15.45:0/341518023 >> 172.21.15.45:0/341518023 conn(0x7fd48006c3a0 msgr2=0x7fd48006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:08.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.572+0000 7fd4876f8640 1 -- 172.21.15.45:0/341518023 shutdown_connections 2024-05-08T05:51:08.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.572+0000 7fd4876f8640 1 -- 172.21.15.45:0/341518023 wait complete. 2024-05-08T05:51:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.573+0000 7fd4876f8640 1 Processor -- start 2024-05-08T05:51:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.573+0000 7fd4876f8640 1 -- start start 2024-05-08T05:51:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.573+0000 7fd4876f8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd48006f060 0x7fd4801587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd4876f8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd48006fe30 0x7fd4800b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd4876f8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4800d0360 0x7fd4800b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd4876f8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd480161250 con 0x7fd4800d0360 2024-05-08T05:51:08.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd4876f8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd480160f70 con 0x7fd48006fe30 2024-05-08T05:51:08.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd4876f8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd480161530 con 0x7fd48006f060 2024-05-08T05:51:08.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd485c95640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4800d0360 0x7fd4800b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd485c95640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4800d0360 0x7fd4800b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37292/0 (socket says 172.21.15.45:37292) 2024-05-08T05:51:08.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd485c95640 1 -- 172.21.15.45:0/2728794234 learned_addr learned my addr 172.21.15.45:0/2728794234 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:08.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd485494640 1 --2- 172.21.15.45:0/2728794234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd48006f060 0x7fd4801587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd484c93640 1 --2- 172.21.15.45:0/2728794234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd48006fe30 0x7fd4800b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd485c95640 1 -- 172.21.15.45:0/2728794234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd48006f060 msgr2=0x7fd4801587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd485c95640 1 --2- 172.21.15.45:0/2728794234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd48006f060 0x7fd4801587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd485c95640 1 -- 172.21.15.45:0/2728794234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd48006fe30 msgr2=0x7fd4800b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd485c95640 1 --2- 172.21.15.45:0/2728794234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd48006fe30 0x7fd4800b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd485c95640 1 -- 172.21.15.45:0/2728794234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd480158d30 con 0x7fd4800d0360 2024-05-08T05:51:08.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd484c93640 1 --2- 172.21.15.45:0/2728794234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd48006fe30 0x7fd4800b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:08.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd485494640 1 --2- 172.21.15.45:0/2728794234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd48006f060 0x7fd4801587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:08.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd485c95640 1 --2- 172.21.15.45:0/2728794234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4800d0360 0x7fd4800b2ce0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fd47800cb10 tx=0x7fd47800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd471ffb640 1 -- 172.21.15.45:0/2728794234 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478017070 con 0x7fd4800d0360 2024-05-08T05:51:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd471ffb640 1 -- 172.21.15.45:0/2728794234 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd47800ac10 con 0x7fd4800d0360 2024-05-08T05:51:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd4876f8640 1 -- 172.21.15.45:0/2728794234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd480072c60 con 0x7fd4800d0360 2024-05-08T05:51:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.574+0000 7fd471ffb640 1 -- 172.21.15.45:0/2728794234 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478012400 con 0x7fd4800d0360 2024-05-08T05:51:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.575+0000 7fd4876f8640 1 -- 172.21.15.45:0/2728794234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd480073500 con 0x7fd4800d0360 2024-05-08T05:51:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.576+0000 7fd4876f8640 1 -- 172.21.15.45:0/2728794234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4800693f0 con 0x7fd4800d0360 2024-05-08T05:51:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.576+0000 7fd471ffb640 1 -- 172.21.15.45:0/2728794234 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd47801a070 con 0x7fd4800d0360 2024-05-08T05:51:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.577+0000 7fd471ffb640 1 --2- 172.21.15.45:0/2728794234 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd4640762d0 0x7fd464078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.577+0000 7fd471ffb640 1 -- 172.21.15.45:0/2728794234 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7fd4780972f0 con 0x7fd4800d0360 2024-05-08T05:51:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.577+0000 7fd485494640 1 --2- 172.21.15.45:0/2728794234 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd4640762d0 0x7fd464078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.577+0000 7fd485494640 1 --2- 172.21.15.45:0/2728794234 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd4640762d0 0x7fd464078790 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd47c034000 tx=0x7fd47c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:08.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.581+0000 7fd471ffb640 1 -- 172.21.15.45:0/2728794234 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd480158d30 con 0x7fd4800d0360 2024-05-08T05:51:08.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.774+0000 7fd4876f8640 1 -- 172.21.15.45:0/2728794234 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4800ade10 con 0x7fd4800d0360 2024-05-08T05:51:08.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.775+0000 7fd471ffb640 1 -- 172.21.15.45:0/2728794234 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16904 (secure 0 0 0) 0x7fd478060500 con 0x7fd4800d0360 2024-05-08T05:51:08.776 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:08.777 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":119,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:51:06.071472+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:51:02.094088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-08T05:51:03.388702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:51:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.777+0000 7fd4876f8640 1 -- 172.21.15.45:0/2728794234 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd4640762d0 msgr2=0x7fd464078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.777+0000 7fd4876f8640 1 --2- 172.21.15.45:0/2728794234 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd4640762d0 0x7fd464078790 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd47c034000 tx=0x7fd47c02e000 comp rx=0 tx=0).stop 2024-05-08T05:51:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.777+0000 7fd4876f8640 1 -- 172.21.15.45:0/2728794234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4800d0360 msgr2=0x7fd4800b2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.777+0000 7fd4876f8640 1 --2- 172.21.15.45:0/2728794234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4800d0360 0x7fd4800b2ce0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fd47800cb10 tx=0x7fd47800a060 comp rx=0 tx=0).stop 2024-05-08T05:51:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.778+0000 7fd4876f8640 1 -- 172.21.15.45:0/2728794234 shutdown_connections 2024-05-08T05:51:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.778+0000 7fd4876f8640 1 --2- 172.21.15.45:0/2728794234 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd4640762d0 0x7fd464078790 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.778+0000 7fd4876f8640 1 --2- 172.21.15.45:0/2728794234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd4800d0360 0x7fd4800b2ce0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.778+0000 7fd4876f8640 1 --2- 172.21.15.45:0/2728794234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd48006fe30 0x7fd4800b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.778+0000 7fd4876f8640 1 --2- 172.21.15.45:0/2728794234 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd48006f060 0x7fd4801587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.778+0000 7fd4876f8640 1 -- 172.21.15.45:0/2728794234 >> 172.21.15.45:0/2728794234 conn(0x7fd48006c3a0 msgr2=0x7fd4800cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:08.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.778+0000 7fd4876f8640 1 -- 172.21.15.45:0/2728794234 shutdown_connections 2024-05-08T05:51:08.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.778+0000 7fd4876f8640 1 -- 172.21.15.45:0/2728794234 wait complete. 2024-05-08T05:51:08.790 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T05:51:08.790 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:51:08.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.893+0000 7f9297820640 1 Processor -- start 2024-05-08T05:51:08.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.893+0000 7f9297820640 1 -- start start 2024-05-08T05:51:08.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.894+0000 7f9297820640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f92900d0ec0 0x7f92900d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.894+0000 7f9297820640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92900d1760 con 0x7f92900d3b00 2024-05-08T05:51:08.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.894+0000 7f9297820640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92900d1a40 con 0x7f929015f380 2024-05-08T05:51:08.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.894+0000 7f9297820640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f92900d30f0 con 0x7f92900d0ec0 2024-05-08T05:51:08.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.894+0000 7f9295dbd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f92900d3b00 0x7f92900b7e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52806/0 (socket says 172.21.15.45:52806) 2024-05-08T05:51:08.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.894+0000 7f9295dbd640 1 -- 172.21.15.45:0/121632358 learned_addr learned my addr 172.21.15.45:0/121632358 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:08.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.894+0000 7f9294dbb640 1 --2- 172.21.15.45:0/121632358 >> v2:172.21.15.63:3300/0 conn(0x7f92900d0ec0 0x7f92900d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.894+0000 7f9287fff640 1 -- 172.21.15.45:0/121632358 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3770092798 0 0) 0x7f92900d1760 con 0x7f92900d3b00 2024-05-08T05:51:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.894+0000 7f9287fff640 1 -- 172.21.15.45:0/121632358 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f927c003520 con 0x7f92900d3b00 2024-05-08T05:51:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.894+0000 7f9287fff640 1 -- 172.21.15.45:0/121632358 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1313660770 0 0) 0x7f927c003520 con 0x7f92900d3b00 2024-05-08T05:51:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.894+0000 7f9287fff640 1 -- 172.21.15.45:0/121632358 >> v2:172.21.15.63:3300/0 conn(0x7f92900d0ec0 msgr2=0x7f92900d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.894+0000 7f9287fff640 1 --2- 172.21.15.45:0/121632358 >> v2:172.21.15.63:3300/0 conn(0x7f92900d0ec0 0x7f92900d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.894+0000 7f9287fff640 1 -- 172.21.15.45:0/121632358 >> v1:172.21.15.63:6789/0 conn(0x7f929015f380 legacy=0x7f92900d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.894+0000 7f9287fff640 1 -- 172.21.15.45:0/121632358 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92900d1d20 con 0x7f92900d3b00 2024-05-08T05:51:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.894+0000 7f9287fff640 1 -- 172.21.15.45:0/121632358 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f92880016f0 con 0x7f92900d3b00 2024-05-08T05:51:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.894+0000 7f9287fff640 1 -- 172.21.15.45:0/121632358 >> v1:172.21.15.45:6789/0 conn(0x7f92900d3b00 legacy=0x7f92900b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.894+0000 7f9287fff640 1 --2- 172.21.15.45:0/121632358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f927c003a50 0x7f927c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.894+0000 7f9287fff640 1 --2- 172.21.15.45:0/121632358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f927c004800 0x7f927c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.895+0000 7f9287fff640 1 --2- 172.21.15.45:0/121632358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c0051c0 0x7f927c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.895+0000 7f9287fff640 1 -- 172.21.15.45:0/121632358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f92900d30f0 con 0x7f927c0051c0 2024-05-08T05:51:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.895+0000 7f9287fff640 1 -- 172.21.15.45:0/121632358 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f927c00c160 con 0x7f927c004800 2024-05-08T05:51:08.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.895+0000 7f9287fff640 1 -- 172.21.15.45:0/121632358 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f927c00c360 con 0x7f927c003a50 2024-05-08T05:51:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.895+0000 7f9295dbd640 1 --2- 172.21.15.45:0/121632358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c0051c0 0x7f927c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.895+0000 7f92955bc640 1 --2- 172.21.15.45:0/121632358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f927c003a50 0x7f927c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.895+0000 7f9294dbb640 1 --2- 172.21.15.45:0/121632358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f927c004800 0x7f927c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.895+0000 7f9295dbd640 1 -- 172.21.15.45:0/121632358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f927c003a50 msgr2=0x7f927c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.895+0000 7f9295dbd640 1 --2- 172.21.15.45:0/121632358 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f927c003a50 0x7f927c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.895+0000 7f9295dbd640 1 -- 172.21.15.45:0/121632358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f927c004800 msgr2=0x7f927c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.895+0000 7f9295dbd640 1 --2- 172.21.15.45:0/121632358 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f927c004800 0x7f927c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.895+0000 7f9295dbd640 1 -- 172.21.15.45:0/121632358 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927c003520 con 0x7f927c0051c0 2024-05-08T05:51:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.896+0000 7f9295dbd640 1 -- 172.21.15.45:0/121632358 reap_dead start 2024-05-08T05:51:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.896+0000 7f9295dbd640 1 --2- 172.21.15.45:0/121632358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c0051c0 0x7f927c00b9c0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f928800c210 tx=0x7f927c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e1882b93e0f9a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.896+0000 7f9287fff640 1 -- 172.21.15.45:0/121632358 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9288007d30 con 0x7f927c0051c0 2024-05-08T05:51:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.896+0000 7f9287fff640 1 -- 172.21.15.45:0/121632358 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f928802e070 con 0x7f927c0051c0 2024-05-08T05:51:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.896+0000 7f9287fff640 1 -- 172.21.15.45:0/121632358 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9288005d30 con 0x7f927c0051c0 2024-05-08T05:51:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.896+0000 7f9297820640 1 -- 172.21.15.45:0/121632358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c0051c0 msgr2=0x7f927c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.896+0000 7f9297820640 1 --2- 172.21.15.45:0/121632358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c0051c0 0x7f927c00b9c0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f928800c210 tx=0x7f927c005830 comp rx=0 tx=0).stop 2024-05-08T05:51:08.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.896+0000 7f9297820640 1 -- 172.21.15.45:0/121632358 shutdown_connections 2024-05-08T05:51:08.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.896+0000 7f9297820640 1 --2- 172.21.15.45:0/121632358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c0051c0 0x7f927c00b9c0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.896+0000 7f9297820640 1 -- 172.21.15.45:0/121632358 >> 172.21.15.45:0/121632358 conn(0x7f92900d4fe0 msgr2=0x7f92900d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:08.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.896+0000 7f9297820640 1 -- 172.21.15.45:0/121632358 shutdown_connections 2024-05-08T05:51:08.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.896+0000 7f9297820640 1 -- 172.21.15.45:0/121632358 wait complete. 2024-05-08T05:51:08.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.898+0000 7f9297820640 1 Processor -- start 2024-05-08T05:51:08.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.898+0000 7f9297820640 1 -- start start 2024-05-08T05:51:08.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.898+0000 7f9297820640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f927c0051c0 0x7f92900a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.898+0000 7f9297820640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9290150110 0x7f929014cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.898+0000 7f9297820640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f929014d050 0x7f92901626b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.898+0000 7f9297820640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f92900d3f30 con 0x7f929014d050 2024-05-08T05:51:08.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.898+0000 7f9297820640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f92900d2370 con 0x7f927c0051c0 2024-05-08T05:51:08.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.898+0000 7f9297820640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f929015f760 con 0x7f9290150110 2024-05-08T05:51:08.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.898+0000 7f9295dbd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f929014d050 0x7f92901626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.899+0000 7f9295dbd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f929014d050 0x7f92901626b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37318/0 (socket says 172.21.15.45:37318) 2024-05-08T05:51:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.899+0000 7f9295dbd640 1 -- 172.21.15.45:0/644945838 learned_addr learned my addr 172.21.15.45:0/644945838 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.899+0000 7f92955bc640 1 --2- 172.21.15.45:0/644945838 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f927c0051c0 0x7f92900a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.899+0000 7f9295dbd640 1 -- 172.21.15.45:0/644945838 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9290150110 msgr2=0x7f929014cb10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T05:51:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.899+0000 7f9295dbd640 1 --2- 172.21.15.45:0/644945838 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9290150110 0x7f929014cb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.899+0000 7f9295dbd640 1 -- 172.21.15.45:0/644945838 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f927c0051c0 msgr2=0x7f92900a0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.899+0000 7f9295dbd640 1 --2- 172.21.15.45:0/644945838 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f927c0051c0 0x7f92900a0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.899+0000 7f9295dbd640 1 -- 172.21.15.45:0/644945838 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f929014e4d0 con 0x7f929014d050 2024-05-08T05:51:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.899+0000 7f9295dbd640 1 --2- 172.21.15.45:0/644945838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f929014d050 0x7f92901626b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f928800c330 tx=0x7f92900cfa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.899+0000 7f9285ffb640 1 -- 172.21.15.45:0/644945838 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9288007270 con 0x7f929014d050 2024-05-08T05:51:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.899+0000 7f9285ffb640 1 -- 172.21.15.45:0/644945838 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9288007410 con 0x7f929014d050 2024-05-08T05:51:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.899+0000 7f9297820640 1 -- 172.21.15.45:0/644945838 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9290150570 con 0x7f929014d050 2024-05-08T05:51:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.899+0000 7f9285ffb640 1 -- 172.21.15.45:0/644945838 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9288007d10 con 0x7f929014d050 2024-05-08T05:51:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.900+0000 7f9297820640 1 -- 172.21.15.45:0/644945838 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92900a18a0 con 0x7f929014d050 2024-05-08T05:51:08.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.901+0000 7f9297820640 1 -- 172.21.15.45:0/644945838 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92900693f0 con 0x7f929014d050 2024-05-08T05:51:08.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.901+0000 7f9285ffb640 1 -- 172.21.15.45:0/644945838 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9288033470 con 0x7f929014d050 2024-05-08T05:51:08.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.902+0000 7f9285ffb640 1 --2- 172.21.15.45:0/644945838 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f92740762d0 0x7f9274078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.902+0000 7f9285ffb640 1 -- 172.21.15.45:0/644945838 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f92880b7100 con 0x7f929014d050 2024-05-08T05:51:08.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.902+0000 7f92955bc640 1 --2- 172.21.15.45:0/644945838 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f92740762d0 0x7f9274078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.903+0000 7f92955bc640 1 --2- 172.21.15.45:0/644945838 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f92740762d0 0x7f9274078790 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f928c000f80 tx=0x7f928c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:08.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:08.907+0000 7f9285ffb640 1 -- 172.21.15.45:0/644945838 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9288045060 con 0x7f929014d050 2024-05-08T05:51:09.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.095+0000 7f9297820640 1 -- 172.21.15.45:0/644945838 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f92900763f0 con 0x7f92740762d0 2024-05-08T05:51:09.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.102+0000 7f9285ffb640 1 -- 172.21.15.45:0/644945838 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250600 (secure 0 0 0) 0x7f92900763f0 con 0x7f92740762d0 2024-05-08T05:51:09.106 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:09.108 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:51:09.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.105+0000 7f9297820640 1 -- 172.21.15.45:0/644945838 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f92740762d0 msgr2=0x7f9274078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:09.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.105+0000 7f9297820640 1 --2- 172.21.15.45:0/644945838 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f92740762d0 0x7f9274078790 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f928c000f80 tx=0x7f928c009040 comp rx=0 tx=0).stop 2024-05-08T05:51:09.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.105+0000 7f9297820640 1 -- 172.21.15.45:0/644945838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f929014d050 msgr2=0x7f92901626b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:09.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.105+0000 7f9297820640 1 --2- 172.21.15.45:0/644945838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f929014d050 0x7f92901626b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f928800c330 tx=0x7f92900cfa50 comp rx=0 tx=0).stop 2024-05-08T05:51:09.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.105+0000 7f9297820640 1 -- 172.21.15.45:0/644945838 shutdown_connections 2024-05-08T05:51:09.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.105+0000 7f9297820640 1 --2- 172.21.15.45:0/644945838 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f92740762d0 0x7f9274078790 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.105+0000 7f9297820640 1 --2- 172.21.15.45:0/644945838 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9290150110 0x7f929014cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.105+0000 7f9297820640 1 --2- 172.21.15.45:0/644945838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f929014d050 0x7f92901626b0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.105+0000 7f9297820640 1 --2- 172.21.15.45:0/644945838 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f927c0051c0 0x7f92900a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.105+0000 7f9297820640 1 -- 172.21.15.45:0/644945838 >> 172.21.15.45:0/644945838 conn(0x7f92900d4fe0 msgr2=0x7f92900ca450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:09.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.105+0000 7f9297820640 1 -- 172.21.15.45:0/644945838 shutdown_connections 2024-05-08T05:51:09.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.105+0000 7f9297820640 1 -- 172.21.15.45:0/644945838 wait complete. 2024-05-08T05:51:09.117 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2024-05-08T05:51:08.974825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592036,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":133,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262108,"kb_used_data":12640,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712612,"statfs":{"total":773094113280,"available":772825714688,"internally_reserved":0,"allocated":12943360,"data_stored":6641833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1668,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.928070"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.073391+0000","last_change":"2024-05-08T05:51:05.069476+0000","last_active":"2024-05-08T05:51:05.073391+0000","last_peered":"2024-05-08T05:51:05.073391+0000","last_clean":"2024-05-08T05:51:05.073391+0000","last_became_active":"2024-05-08T05:51:05.069131+0000","last_became_peered":"2024-05-08T05:51:05.069131+0000","last_unstale":"2024-05-08T05:51:05.073391+0000","last_undegraded":"2024-05-08T05:51:05.073391+0000","last_fullsized":"2024-05-08T05:51:05.073391+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:05:09.239164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093231+0000","last_change":"2024-05-08T05:51:05.084139+0000","last_active":"2024-05-08T05:51:05.093231+0000","last_peered":"2024-05-08T05:51:05.093231+0000","last_clean":"2024-05-08T05:51:05.093231+0000","last_became_active":"2024-05-08T05:51:05.081830+0000","last_became_peered":"2024-05-08T05:51:05.081830+0000","last_unstale":"2024-05-08T05:51:05.093231+0000","last_undegraded":"2024-05-08T05:51:05.093231+0000","last_fullsized":"2024-05-08T05:51:05.093231+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:37.715909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093594+0000","last_change":"2024-05-08T05:51:05.090876+0000","last_active":"2024-05-08T05:51:05.093594+0000","last_peered":"2024-05-08T05:51:05.093594+0000","last_clean":"2024-05-08T05:51:05.093594+0000","last_became_active":"2024-05-08T05:51:05.089329+0000","last_became_peered":"2024-05-08T05:51:05.089329+0000","last_unstale":"2024-05-08T05:51:05.093594+0000","last_undegraded":"2024-05-08T05:51:05.093594+0000","last_fullsized":"2024-05-08T05:51:05.093594+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:51:43.350274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052577+0000","last_change":"2024-05-08T05:51:04.052577+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052577+0000","last_undegraded":"2024-05-08T05:51:04.052577+0000","last_fullsized":"2024-05-08T05:51:04.052577+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.077282+0000","last_change":"2024-05-08T05:51:05.072296+0000","last_active":"2024-05-08T05:51:05.077282+0000","last_peered":"2024-05-08T05:51:05.077282+0000","last_clean":"2024-05-08T05:51:05.077282+0000","last_became_active":"2024-05-08T05:51:05.072009+0000","last_became_peered":"2024-05-08T05:51:05.072009+0000","last_unstale":"2024-05-08T05:51:05.077282+0000","last_undegraded":"2024-05-08T05:51:05.077282+0000","last_fullsized":"2024-05-08T05:51:05.077282+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:12:02.641413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066407+0000","last_change":"2024-05-08T05:51:05.082847+0000","last_active":"2024-05-08T05:51:07.066407+0000","last_peered":"2024-05-08T05:51:07.066407+0000","last_clean":"2024-05-08T05:51:07.066407+0000","last_became_active":"2024-05-08T05:51:05.082267+0000","last_became_peered":"2024-05-08T05:51:05.082267+0000","last_unstale":"2024-05-08T05:51:07.066407+0000","last_undegraded":"2024-05-08T05:51:07.066407+0000","last_fullsized":"2024-05-08T05:51:07.066407+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:16:17.521040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066220+0000","last_change":"2024-05-08T05:51:05.087173+0000","last_active":"2024-05-08T05:51:07.066220+0000","last_peered":"2024-05-08T05:51:07.066220+0000","last_clean":"2024-05-08T05:51:07.066220+0000","last_became_active":"2024-05-08T05:51:05.086914+0000","last_became_peered":"2024-05-08T05:51:05.086914+0000","last_unstale":"2024-05-08T05:51:07.066220+0000","last_undegraded":"2024-05-08T05:51:07.066220+0000","last_fullsized":"2024-05-08T05:51:07.066220+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:42:24.823847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093326+0000","last_change":"2024-05-08T05:51:05.085921+0000","last_active":"2024-05-08T05:51:05.093326+0000","last_peered":"2024-05-08T05:51:05.093326+0000","last_clean":"2024-05-08T05:51:05.093326+0000","last_became_active":"2024-05-08T05:51:05.083800+0000","last_became_peered":"2024-05-08T05:51:05.083800+0000","last_unstale":"2024-05-08T05:51:05.093326+0000","last_undegraded":"2024-05-08T05:51:05.093326+0000","last_fullsized":"2024-05-08T05:51:05.093326+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:47.453010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.053739+0000","last_change":"2024-05-08T05:51:04.053739+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.053739+0000","last_undegraded":"2024-05-08T05:51:04.053739+0000","last_fullsized":"2024-05-08T05:51:04.053739+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.075989+0000","last_change":"2024-05-08T05:51:05.071111+0000","last_active":"2024-05-08T05:51:05.075989+0000","last_peered":"2024-05-08T05:51:05.075989+0000","last_clean":"2024-05-08T05:51:05.075989+0000","last_became_active":"2024-05-08T05:51:05.070823+0000","last_became_peered":"2024-05-08T05:51:05.070823+0000","last_unstale":"2024-05-08T05:51:05.075989+0000","last_undegraded":"2024-05-08T05:51:05.075989+0000","last_fullsized":"2024-05-08T05:51:05.075989+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:18.549853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.073515+0000","last_change":"2024-05-08T05:51:05.070718+0000","last_active":"2024-05-08T05:51:05.073515+0000","last_peered":"2024-05-08T05:51:05.073515+0000","last_clean":"2024-05-08T05:51:05.073515+0000","last_became_active":"2024-05-08T05:51:05.070422+0000","last_became_peered":"2024-05-08T05:51:05.070422+0000","last_unstale":"2024-05-08T05:51:05.073515+0000","last_undegraded":"2024-05-08T05:51:05.073515+0000","last_fullsized":"2024-05-08T05:51:05.073515+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:29.725291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066591+0000","last_change":"2024-05-08T05:51:05.076336+0000","last_active":"2024-05-08T05:51:07.066591+0000","last_peered":"2024-05-08T05:51:07.066591+0000","last_clean":"2024-05-08T05:51:07.066591+0000","last_became_active":"2024-05-08T05:51:05.075648+0000","last_became_peered":"2024-05-08T05:51:05.075648+0000","last_unstale":"2024-05-08T05:51:07.066591+0000","last_undegraded":"2024-05-08T05:51:07.066591+0000","last_fullsized":"2024-05-08T05:51:07.066591+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:27.438502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088120+0000","last_change":"2024-05-08T05:51:05.079977+0000","last_active":"2024-05-08T05:51:05.088120+0000","last_peered":"2024-05-08T05:51:05.088120+0000","last_clean":"2024-05-08T05:51:05.088120+0000","last_became_active":"2024-05-08T05:51:05.078938+0000","last_became_peered":"2024-05-08T05:51:05.078938+0000","last_unstale":"2024-05-08T05:51:05.088120+0000","last_undegraded":"2024-05-08T05:51:05.088120+0000","last_fullsized":"2024-05-08T05:51:05.088120+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:00.252213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093291+0000","last_change":"2024-05-08T05:51:05.086072+0000","last_active":"2024-05-08T05:51:05.093291+0000","last_peered":"2024-05-08T05:51:05.093291+0000","last_clean":"2024-05-08T05:51:05.093291+0000","last_became_active":"2024-05-08T05:51:05.084475+0000","last_became_peered":"2024-05-08T05:51:05.084475+0000","last_unstale":"2024-05-08T05:51:05.093291+0000","last_undegraded":"2024-05-08T05:51:05.093291+0000","last_fullsized":"2024-05-08T05:51:05.093291+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:08:48.269564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088031+0000","last_change":"2024-05-08T05:51:05.083505+0000","last_active":"2024-05-08T05:51:05.088031+0000","last_peered":"2024-05-08T05:51:05.088031+0000","last_clean":"2024-05-08T05:51:05.088031+0000","last_became_active":"2024-05-08T05:51:05.083236+0000","last_became_peered":"2024-05-08T05:51:05.083236+0000","last_unstale":"2024-05-08T05:51:05.088031+0000","last_undegraded":"2024-05-08T05:51:05.088031+0000","last_fullsized":"2024-05-08T05:51:05.088031+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:20.213009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066209+0000","last_change":"2024-05-08T05:51:05.077087+0000","last_active":"2024-05-08T05:51:07.066209+0000","last_peered":"2024-05-08T05:51:07.066209+0000","last_clean":"2024-05-08T05:51:07.066209+0000","last_became_active":"2024-05-08T05:51:05.076743+0000","last_became_peered":"2024-05-08T05:51:05.076743+0000","last_unstale":"2024-05-08T05:51:07.066209+0000","last_undegraded":"2024-05-08T05:51:07.066209+0000","last_fullsized":"2024-05-08T05:51:07.066209+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:03.278751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088248+0000","last_change":"2024-05-08T05:51:05.081532+0000","last_active":"2024-05-08T05:51:05.088248+0000","last_peered":"2024-05-08T05:51:05.088248+0000","last_clean":"2024-05-08T05:51:05.088248+0000","last_became_active":"2024-05-08T05:51:05.081228+0000","last_became_peered":"2024-05-08T05:51:05.081228+0000","last_unstale":"2024-05-08T05:51:05.088248+0000","last_undegraded":"2024-05-08T05:51:05.088248+0000","last_fullsized":"2024-05-08T05:51:05.088248+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:38:53.162197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093376+0000","last_change":"2024-05-08T05:51:05.086943+0000","last_active":"2024-05-08T05:51:05.093376+0000","last_peered":"2024-05-08T05:51:05.093376+0000","last_clean":"2024-05-08T05:51:05.093376+0000","last_became_active":"2024-05-08T05:51:05.086221+0000","last_became_peered":"2024-05-08T05:51:05.086221+0000","last_unstale":"2024-05-08T05:51:05.093376+0000","last_undegraded":"2024-05-08T05:51:05.093376+0000","last_fullsized":"2024-05-08T05:51:05.093376+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:34:12.399560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.056046+0000","last_change":"2024-05-08T05:51:04.056046+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.056046+0000","last_undegraded":"2024-05-08T05:51:04.056046+0000","last_fullsized":"2024-05-08T05:51:04.056046+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078597+0000","last_change":"2024-05-08T05:51:05.080761+0000","last_active":"2024-05-08T05:51:06.078597+0000","last_peered":"2024-05-08T05:51:06.078597+0000","last_clean":"2024-05-08T05:51:06.078597+0000","last_became_active":"2024-05-08T05:51:05.080381+0000","last_became_peered":"2024-05-08T05:51:05.080381+0000","last_unstale":"2024-05-08T05:51:06.078597+0000","last_undegraded":"2024-05-08T05:51:06.078597+0000","last_fullsized":"2024-05-08T05:51:06.078597+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:42.792089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.051557+0000","last_change":"2024-05-08T05:51:04.051557+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.051557+0000","last_undegraded":"2024-05-08T05:51:04.051557+0000","last_fullsized":"2024-05-08T05:51:04.051557+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.084370+0000","last_change":"2024-05-08T05:51:05.079566+0000","last_active":"2024-05-08T05:51:05.084370+0000","last_peered":"2024-05-08T05:51:05.084370+0000","last_clean":"2024-05-08T05:51:05.084370+0000","last_became_active":"2024-05-08T05:51:05.079256+0000","last_became_peered":"2024-05-08T05:51:05.079256+0000","last_unstale":"2024-05-08T05:51:05.084370+0000","last_undegraded":"2024-05-08T05:51:05.084370+0000","last_fullsized":"2024-05-08T05:51:05.084370+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:51:07.851943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.050947+0000","last_change":"2024-05-08T05:51:04.050947+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.050947+0000","last_undegraded":"2024-05-08T05:51:04.050947+0000","last_fullsized":"2024-05-08T05:51:04.050947+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052838+0000","last_change":"2024-05-08T05:51:04.052838+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052838+0000","last_undegraded":"2024-05-08T05:51:04.052838+0000","last_fullsized":"2024-05-08T05:51:04.052838+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.053724+0000","last_change":"2024-05-08T05:51:04.053724+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.053724+0000","last_undegraded":"2024-05-08T05:51:04.053724+0000","last_fullsized":"2024-05-08T05:51:04.053724+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078673+0000","last_change":"2024-05-08T05:51:05.081486+0000","last_active":"2024-05-08T05:51:06.078673+0000","last_peered":"2024-05-08T05:51:06.078673+0000","last_clean":"2024-05-08T05:51:06.078673+0000","last_became_active":"2024-05-08T05:51:05.081143+0000","last_became_peered":"2024-05-08T05:51:05.081143+0000","last_unstale":"2024-05-08T05:51:06.078673+0000","last_undegraded":"2024-05-08T05:51:06.078673+0000","last_fullsized":"2024-05-08T05:51:06.078673+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:44:56.396392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.055539+0000","last_change":"2024-05-08T05:51:04.055539+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.055539+0000","last_undegraded":"2024-05-08T05:51:04.055539+0000","last_fullsized":"2024-05-08T05:51:04.055539+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066746+0000","last_change":"2024-05-08T05:51:05.081306+0000","last_active":"2024-05-08T05:51:07.066746+0000","last_peered":"2024-05-08T05:51:07.066746+0000","last_clean":"2024-05-08T05:51:07.066746+0000","last_became_active":"2024-05-08T05:51:05.081023+0000","last_became_peered":"2024-05-08T05:51:05.081023+0000","last_unstale":"2024-05-08T05:51:07.066746+0000","last_undegraded":"2024-05-08T05:51:07.066746+0000","last_fullsized":"2024-05-08T05:51:07.066746+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:20:04.797865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078729+0000","last_change":"2024-05-08T05:51:05.082212+0000","last_active":"2024-05-08T05:51:06.078729+0000","last_peered":"2024-05-08T05:51:06.078729+0000","last_clean":"2024-05-08T05:51:06.078729+0000","last_became_active":"2024-05-08T05:51:05.081847+0000","last_became_peered":"2024-05-08T05:51:05.081847+0000","last_unstale":"2024-05-08T05:51:06.078729+0000","last_undegraded":"2024-05-08T05:51:06.078729+0000","last_fullsized":"2024-05-08T05:51:06.078729+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:10.568249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093267+0000","last_change":"2024-05-08T05:51:05.085821+0000","last_active":"2024-05-08T05:51:05.093267+0000","last_peered":"2024-05-08T05:51:05.093267+0000","last_clean":"2024-05-08T05:51:05.093267+0000","last_became_active":"2024-05-08T05:51:05.083777+0000","last_became_peered":"2024-05-08T05:51:05.083777+0000","last_unstale":"2024-05-08T05:51:05.093267+0000","last_undegraded":"2024-05-08T05:51:05.093267+0000","last_fullsized":"2024-05-08T05:51:05.093267+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:22:46.959500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.051392+0000","last_change":"2024-05-08T05:51:04.051392+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.051392+0000","last_undegraded":"2024-05-08T05:51:04.051392+0000","last_fullsized":"2024-05-08T05:51:04.051392+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.050670+0000","last_change":"2024-05-08T05:51:04.050670+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.050670+0000","last_undegraded":"2024-05-08T05:51:04.050670+0000","last_fullsized":"2024-05-08T05:51:04.050670+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.076195+0000","last_change":"2024-05-08T05:51:05.072621+0000","last_active":"2024-05-08T05:51:05.076195+0000","last_peered":"2024-05-08T05:51:05.076195+0000","last_clean":"2024-05-08T05:51:05.076195+0000","last_became_active":"2024-05-08T05:51:05.072317+0000","last_became_peered":"2024-05-08T05:51:05.072317+0000","last_unstale":"2024-05-08T05:51:05.076195+0000","last_undegraded":"2024-05-08T05:51:05.076195+0000","last_fullsized":"2024-05-08T05:51:05.076195+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:05:09.239164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.064918+0000","last_change":"2024-05-08T05:51:04.063764+0000","last_active":"2024-05-08T05:51:04.064918+0000","last_peered":"2024-05-08T05:51:04.064918+0000","last_clean":"2024-05-08T05:51:04.064918+0000","last_became_active":"2024-05-08T05:51:04.063484+0000","last_became_peered":"2024-05-08T05:51:04.063484+0000","last_unstale":"2024-05-08T05:51:04.064918+0000","last_undegraded":"2024-05-08T05:51:04.064918+0000","last_fullsized":"2024-05-08T05:51:04.064918+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:17.541863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066655+0000","last_change":"2024-05-08T05:51:05.084520+0000","last_active":"2024-05-08T05:51:07.066655+0000","last_peered":"2024-05-08T05:51:07.066655+0000","last_clean":"2024-05-08T05:51:07.066655+0000","last_became_active":"2024-05-08T05:51:05.084198+0000","last_became_peered":"2024-05-08T05:51:05.084198+0000","last_unstale":"2024-05-08T05:51:07.066655+0000","last_undegraded":"2024-05-08T05:51:07.066655+0000","last_fullsized":"2024-05-08T05:51:07.066655+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:08:36.961131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.063174+0000","last_change":"2024-05-08T05:51:04.057409+0000","last_active":"2024-05-08T05:51:05.063174+0000","last_peered":"2024-05-08T05:51:05.063174+0000","last_clean":"2024-05-08T05:51:05.063174+0000","last_became_active":"2024-05-08T05:51:04.057127+0000","last_became_peered":"2024-05-08T05:51:04.057127+0000","last_unstale":"2024-05-08T05:51:05.063174+0000","last_undegraded":"2024-05-08T05:51:05.063174+0000","last_fullsized":"2024-05-08T05:51:05.063174+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:33.984066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052424+0000","last_change":"2024-05-08T05:51:04.052424+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052424+0000","last_undegraded":"2024-05-08T05:51:04.052424+0000","last_fullsized":"2024-05-08T05:51:04.052424+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.065887+0000","last_change":"2024-05-08T05:51:04.063770+0000","last_active":"2024-05-08T05:51:04.065887+0000","last_peered":"2024-05-08T05:51:04.065887+0000","last_clean":"2024-05-08T05:51:04.065887+0000","last_became_active":"2024-05-08T05:51:04.063471+0000","last_became_peered":"2024-05-08T05:51:04.063471+0000","last_unstale":"2024-05-08T05:51:04.065887+0000","last_undegraded":"2024-05-08T05:51:04.065887+0000","last_fullsized":"2024-05-08T05:51:04.065887+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:14:14.017450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.053316+0000","last_change":"2024-05-08T05:51:04.053316+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.053316+0000","last_undegraded":"2024-05-08T05:51:04.053316+0000","last_fullsized":"2024-05-08T05:51:04.053316+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.095608+0000","last_change":"2024-05-08T05:51:04.060756+0000","last_active":"2024-05-08T05:51:05.095608+0000","last_peered":"2024-05-08T05:51:05.095608+0000","last_clean":"2024-05-08T05:51:05.095608+0000","last_became_active":"2024-05-08T05:51:04.060480+0000","last_became_peered":"2024-05-08T05:51:04.060480+0000","last_unstale":"2024-05-08T05:51:05.095608+0000","last_undegraded":"2024-05-08T05:51:05.095608+0000","last_fullsized":"2024-05-08T05:51:05.095608+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:58:38.791116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.076073+0000","last_change":"2024-05-08T05:51:05.072513+0000","last_active":"2024-05-08T05:51:05.076073+0000","last_peered":"2024-05-08T05:51:05.076073+0000","last_clean":"2024-05-08T05:51:05.076073+0000","last_became_active":"2024-05-08T05:51:05.072191+0000","last_became_peered":"2024-05-08T05:51:05.072191+0000","last_unstale":"2024-05-08T05:51:05.076073+0000","last_undegraded":"2024-05-08T05:51:05.076073+0000","last_fullsized":"2024-05-08T05:51:05.076073+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:27.347106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.062425+0000","last_change":"2024-05-08T05:51:04.058250+0000","last_active":"2024-05-08T05:51:05.062425+0000","last_peered":"2024-05-08T05:51:05.062425+0000","last_clean":"2024-05-08T05:51:05.062425+0000","last_became_active":"2024-05-08T05:51:04.057765+0000","last_became_peered":"2024-05-08T05:51:04.057765+0000","last_unstale":"2024-05-08T05:51:05.062425+0000","last_undegraded":"2024-05-08T05:51:05.062425+0000","last_fullsized":"2024-05-08T05:51:05.062425+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:46:01.704267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.084362+0000","last_change":"2024-05-08T05:51:05.079919+0000","last_active":"2024-05-08T05:51:05.084362+0000","last_peered":"2024-05-08T05:51:05.084362+0000","last_clean":"2024-05-08T05:51:05.084362+0000","last_became_active":"2024-05-08T05:51:05.079552+0000","last_became_peered":"2024-05-08T05:51:05.079552+0000","last_unstale":"2024-05-08T05:51:05.084362+0000","last_undegraded":"2024-05-08T05:51:05.084362+0000","last_fullsized":"2024-05-08T05:51:05.084362+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:38:53.162197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066334+0000","last_change":"2024-05-08T05:51:04.069265+0000","last_active":"2024-05-08T05:51:07.066334+0000","last_peered":"2024-05-08T05:51:07.066334+0000","last_clean":"2024-05-08T05:51:07.066334+0000","last_became_active":"2024-05-08T05:51:04.068997+0000","last_became_peered":"2024-05-08T05:51:04.068997+0000","last_unstale":"2024-05-08T05:51:07.066334+0000","last_undegraded":"2024-05-08T05:51:07.066334+0000","last_fullsized":"2024-05-08T05:51:07.066334+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:55.959876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.084003+0000","last_change":"2024-05-08T05:51:05.079354+0000","last_active":"2024-05-08T05:51:05.084003+0000","last_peered":"2024-05-08T05:51:05.084003+0000","last_clean":"2024-05-08T05:51:05.084003+0000","last_became_active":"2024-05-08T05:51:05.079042+0000","last_became_peered":"2024-05-08T05:51:05.079042+0000","last_unstale":"2024-05-08T05:51:05.084003+0000","last_undegraded":"2024-05-08T05:51:05.084003+0000","last_fullsized":"2024-05-08T05:51:05.084003+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:20:04.111530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.065847+0000","last_change":"2024-05-08T05:51:04.058859+0000","last_active":"2024-05-08T05:51:04.065847+0000","last_peered":"2024-05-08T05:51:04.065847+0000","last_clean":"2024-05-08T05:51:04.065847+0000","last_became_active":"2024-05-08T05:51:04.058518+0000","last_became_peered":"2024-05-08T05:51:04.058518+0000","last_unstale":"2024-05-08T05:51:04.065847+0000","last_undegraded":"2024-05-08T05:51:04.065847+0000","last_fullsized":"2024-05-08T05:51:04.065847+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:02:28.473412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.087245+0000","last_change":"2024-05-08T05:51:05.077404+0000","last_active":"2024-05-08T05:51:05.087245+0000","last_peered":"2024-05-08T05:51:05.087245+0000","last_clean":"2024-05-08T05:51:05.087245+0000","last_became_active":"2024-05-08T05:51:05.077093+0000","last_became_peered":"2024-05-08T05:51:05.077093+0000","last_unstale":"2024-05-08T05:51:05.087245+0000","last_undegraded":"2024-05-08T05:51:05.087245+0000","last_fullsized":"2024-05-08T05:51:05.087245+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:12:02.641413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066301+0000","last_change":"2024-05-08T05:51:04.067859+0000","last_active":"2024-05-08T05:51:07.066301+0000","last_peered":"2024-05-08T05:51:07.066301+0000","last_clean":"2024-05-08T05:51:07.066301+0000","last_became_active":"2024-05-08T05:51:04.067586+0000","last_became_peered":"2024-05-08T05:51:04.067586+0000","last_unstale":"2024-05-08T05:51:07.066301+0000","last_undegraded":"2024-05-08T05:51:07.066301+0000","last_fullsized":"2024-05-08T05:51:07.066301+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:35:04.353051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.055148+0000","last_change":"2024-05-08T05:51:04.055148+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.055148+0000","last_undegraded":"2024-05-08T05:51:04.055148+0000","last_fullsized":"2024-05-08T05:51:04.055148+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.070847+0000","last_change":"2024-05-08T05:51:04.069859+0000","last_active":"2024-05-08T05:51:04.070847+0000","last_peered":"2024-05-08T05:51:04.070847+0000","last_clean":"2024-05-08T05:51:04.070847+0000","last_became_active":"2024-05-08T05:51:04.069575+0000","last_became_peered":"2024-05-08T05:51:04.069575+0000","last_unstale":"2024-05-08T05:51:04.070847+0000","last_undegraded":"2024-05-08T05:51:04.070847+0000","last_fullsized":"2024-05-08T05:51:04.070847+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:05:08.231174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.051301+0000","last_change":"2024-05-08T05:51:04.051301+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.051301+0000","last_undegraded":"2024-05-08T05:51:04.051301+0000","last_fullsized":"2024-05-08T05:51:04.051301+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.067379+0000","last_change":"2024-05-08T05:51:04.064523+0000","last_active":"2024-05-08T05:51:04.067379+0000","last_peered":"2024-05-08T05:51:04.067379+0000","last_clean":"2024-05-08T05:51:04.067379+0000","last_became_active":"2024-05-08T05:51:04.064236+0000","last_became_peered":"2024-05-08T05:51:04.064236+0000","last_unstale":"2024-05-08T05:51:04.067379+0000","last_undegraded":"2024-05-08T05:51:04.067379+0000","last_fullsized":"2024-05-08T05:51:04.067379+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:58:38.791116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093624+0000","last_change":"2024-05-08T05:51:05.090316+0000","last_active":"2024-05-08T05:51:05.093624+0000","last_peered":"2024-05-08T05:51:05.093624+0000","last_clean":"2024-05-08T05:51:05.093624+0000","last_became_active":"2024-05-08T05:51:05.084493+0000","last_became_peered":"2024-05-08T05:51:05.084493+0000","last_unstale":"2024-05-08T05:51:05.093624+0000","last_undegraded":"2024-05-08T05:51:05.093624+0000","last_fullsized":"2024-05-08T05:51:05.093624+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:16:24.027328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.066003+0000","last_change":"2024-05-08T05:51:04.059742+0000","last_active":"2024-05-08T05:51:04.066003+0000","last_peered":"2024-05-08T05:51:04.066003+0000","last_clean":"2024-05-08T05:51:04.066003+0000","last_became_active":"2024-05-08T05:51:04.059468+0000","last_became_peered":"2024-05-08T05:51:04.059468+0000","last_unstale":"2024-05-08T05:51:04.066003+0000","last_undegraded":"2024-05-08T05:51:04.066003+0000","last_fullsized":"2024-05-08T05:51:04.066003+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:35.193080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052749+0000","last_change":"2024-05-08T05:51:04.052749+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052749+0000","last_undegraded":"2024-05-08T05:51:04.052749+0000","last_fullsized":"2024-05-08T05:51:04.052749+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"118'6","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.101220+0000","last_change":"2024-05-08T05:51:04.066429+0000","last_active":"2024-05-08T05:51:05.101220+0000","last_peered":"2024-05-08T05:51:05.101220+0000","last_clean":"2024-05-08T05:51:05.101220+0000","last_became_active":"2024-05-08T05:51:04.065698+0000","last_became_peered":"2024-05-08T05:51:04.065698+0000","last_unstale":"2024-05-08T05:51:05.101220+0000","last_undegraded":"2024-05-08T05:51:05.101220+0000","last_fullsized":"2024-05-08T05:51:05.101220+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:28.960211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.054481+0000","last_change":"2024-05-08T05:51:04.054481+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.054481+0000","last_undegraded":"2024-05-08T05:51:04.054481+0000","last_fullsized":"2024-05-08T05:51:04.054481+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.094391+0000","last_change":"2024-05-08T05:51:04.067759+0000","last_active":"2024-05-08T05:51:05.094391+0000","last_peered":"2024-05-08T05:51:05.094391+0000","last_clean":"2024-05-08T05:51:05.094391+0000","last_became_active":"2024-05-08T05:51:04.067484+0000","last_became_peered":"2024-05-08T05:51:04.067484+0000","last_unstale":"2024-05-08T05:51:05.094391+0000","last_undegraded":"2024-05-08T05:51:05.094391+0000","last_fullsized":"2024-05-08T05:51:05.094391+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:44.431356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088282+0000","last_change":"2024-05-08T05:51:05.080720+0000","last_active":"2024-05-08T05:51:05.088282+0000","last_peered":"2024-05-08T05:51:05.088282+0000","last_clean":"2024-05-08T05:51:05.088282+0000","last_became_active":"2024-05-08T05:51:05.080428+0000","last_became_peered":"2024-05-08T05:51:05.080428+0000","last_unstale":"2024-05-08T05:51:05.088282+0000","last_undegraded":"2024-05-08T05:51:05.088282+0000","last_fullsized":"2024-05-08T05:51:05.088282+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:51:07.851943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.070039+0000","last_change":"2024-05-08T05:51:04.065712+0000","last_active":"2024-05-08T05:51:05.070039+0000","last_peered":"2024-05-08T05:51:05.070039+0000","last_clean":"2024-05-08T05:51:05.070039+0000","last_became_active":"2024-05-08T05:51:04.065421+0000","last_became_peered":"2024-05-08T05:51:04.065421+0000","last_unstale":"2024-05-08T05:51:05.070039+0000","last_undegraded":"2024-05-08T05:51:05.070039+0000","last_fullsized":"2024-05-08T05:51:05.070039+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:44:48.375896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078591+0000","last_change":"2024-05-08T05:51:05.084846+0000","last_active":"2024-05-08T05:51:06.078591+0000","last_peered":"2024-05-08T05:51:06.078591+0000","last_clean":"2024-05-08T05:51:06.078591+0000","last_became_active":"2024-05-08T05:51:05.084544+0000","last_became_peered":"2024-05-08T05:51:05.084544+0000","last_unstale":"2024-05-08T05:51:06.078591+0000","last_undegraded":"2024-05-08T05:51:06.078591+0000","last_fullsized":"2024-05-08T05:51:06.078591+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:30.552529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.068303+0000","last_change":"2024-05-08T05:51:04.067872+0000","last_active":"2024-05-08T05:51:05.068303+0000","last_peered":"2024-05-08T05:51:05.068303+0000","last_clean":"2024-05-08T05:51:05.068303+0000","last_became_active":"2024-05-08T05:51:04.067380+0000","last_became_peered":"2024-05-08T05:51:04.067380+0000","last_unstale":"2024-05-08T05:51:05.068303+0000","last_undegraded":"2024-05-08T05:51:05.068303+0000","last_fullsized":"2024-05-08T05:51:05.068303+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:21.223822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052885+0000","last_change":"2024-05-08T05:51:04.052885+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052885+0000","last_undegraded":"2024-05-08T05:51:04.052885+0000","last_fullsized":"2024-05-08T05:51:04.052885+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.063576+0000","last_change":"2024-05-08T05:51:04.070032+0000","last_active":"2024-05-08T05:51:05.063576+0000","last_peered":"2024-05-08T05:51:05.063576+0000","last_clean":"2024-05-08T05:51:05.063576+0000","last_became_active":"2024-05-08T05:51:04.069748+0000","last_became_peered":"2024-05-08T05:51:04.069748+0000","last_unstale":"2024-05-08T05:51:05.063576+0000","last_undegraded":"2024-05-08T05:51:05.063576+0000","last_fullsized":"2024-05-08T05:51:05.063576+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:46:01.704267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.084392+0000","last_change":"2024-05-08T05:51:05.078677+0000","last_active":"2024-05-08T05:51:05.084392+0000","last_peered":"2024-05-08T05:51:05.084392+0000","last_clean":"2024-05-08T05:51:05.084392+0000","last_became_active":"2024-05-08T05:51:05.078364+0000","last_became_peered":"2024-05-08T05:51:05.078364+0000","last_unstale":"2024-05-08T05:51:05.084392+0000","last_undegraded":"2024-05-08T05:51:05.084392+0000","last_fullsized":"2024-05-08T05:51:05.084392+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:00.252213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078677+0000","last_change":"2024-05-08T05:51:04.067612+0000","last_active":"2024-05-08T05:51:06.078677+0000","last_peered":"2024-05-08T05:51:06.078677+0000","last_clean":"2024-05-08T05:51:06.078677+0000","last_became_active":"2024-05-08T05:51:04.067268+0000","last_became_peered":"2024-05-08T05:51:04.067268+0000","last_unstale":"2024-05-08T05:51:06.078677+0000","last_undegraded":"2024-05-08T05:51:06.078677+0000","last_fullsized":"2024-05-08T05:51:06.078677+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:28:11.738033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066899+0000","last_change":"2024-05-08T05:51:05.073728+0000","last_active":"2024-05-08T05:51:07.066899+0000","last_peered":"2024-05-08T05:51:07.066899+0000","last_clean":"2024-05-08T05:51:07.066899+0000","last_became_active":"2024-05-08T05:51:05.073415+0000","last_became_peered":"2024-05-08T05:51:05.073415+0000","last_unstale":"2024-05-08T05:51:07.066899+0000","last_undegraded":"2024-05-08T05:51:07.066899+0000","last_fullsized":"2024-05-08T05:51:07.066899+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:52.612093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.069657+0000","last_change":"2024-05-08T05:51:04.066627+0000","last_active":"2024-05-08T05:51:04.069657+0000","last_peered":"2024-05-08T05:51:04.069657+0000","last_clean":"2024-05-08T05:51:04.069657+0000","last_became_active":"2024-05-08T05:51:04.066321+0000","last_became_peered":"2024-05-08T05:51:04.066321+0000","last_unstale":"2024-05-08T05:51:04.069657+0000","last_undegraded":"2024-05-08T05:51:04.069657+0000","last_fullsized":"2024-05-08T05:51:04.069657+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:54.640197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052213+0000","last_change":"2024-05-08T05:51:04.052213+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052213+0000","last_undegraded":"2024-05-08T05:51:04.052213+0000","last_fullsized":"2024-05-08T05:51:04.052213+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.072448+0000","last_change":"2024-05-08T05:51:04.072005+0000","last_active":"2024-05-08T05:51:04.072448+0000","last_peered":"2024-05-08T05:51:04.072448+0000","last_clean":"2024-05-08T05:51:04.072448+0000","last_became_active":"2024-05-08T05:51:04.071776+0000","last_became_peered":"2024-05-08T05:51:04.071776+0000","last_unstale":"2024-05-08T05:51:04.072448+0000","last_undegraded":"2024-05-08T05:51:04.072448+0000","last_fullsized":"2024-05-08T05:51:04.072448+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:21.223822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066166+0000","last_change":"2024-05-08T05:51:05.074599+0000","last_active":"2024-05-08T05:51:07.066166+0000","last_peered":"2024-05-08T05:51:07.066166+0000","last_clean":"2024-05-08T05:51:07.066166+0000","last_became_active":"2024-05-08T05:51:05.073942+0000","last_became_peered":"2024-05-08T05:51:05.073942+0000","last_unstale":"2024-05-08T05:51:07.066166+0000","last_undegraded":"2024-05-08T05:51:07.066166+0000","last_fullsized":"2024-05-08T05:51:07.066166+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:30.124962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066177+0000","last_change":"2024-05-08T05:51:04.069082+0000","last_active":"2024-05-08T05:51:07.066177+0000","last_peered":"2024-05-08T05:51:07.066177+0000","last_clean":"2024-05-08T05:51:07.066177+0000","last_became_active":"2024-05-08T05:51:04.068801+0000","last_became_peered":"2024-05-08T05:51:04.068801+0000","last_unstale":"2024-05-08T05:51:07.066177+0000","last_undegraded":"2024-05-08T05:51:07.066177+0000","last_fullsized":"2024-05-08T05:51:07.066177+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:57:45.457442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.092747+0000","last_change":"2024-05-08T05:51:05.083611+0000","last_active":"2024-05-08T05:51:05.092747+0000","last_peered":"2024-05-08T05:51:05.092747+0000","last_clean":"2024-05-08T05:51:05.092747+0000","last_became_active":"2024-05-08T05:51:05.083328+0000","last_became_peered":"2024-05-08T05:51:05.083328+0000","last_unstale":"2024-05-08T05:51:05.092747+0000","last_undegraded":"2024-05-08T05:51:05.092747+0000","last_fullsized":"2024-05-08T05:51:05.092747+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:55.131091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.065846+0000","last_change":"2024-05-08T05:51:04.063770+0000","last_active":"2024-05-08T05:51:04.065846+0000","last_peered":"2024-05-08T05:51:04.065846+0000","last_clean":"2024-05-08T05:51:04.065846+0000","last_became_active":"2024-05-08T05:51:04.063509+0000","last_became_peered":"2024-05-08T05:51:04.063509+0000","last_unstale":"2024-05-08T05:51:04.065846+0000","last_undegraded":"2024-05-08T05:51:04.065846+0000","last_fullsized":"2024-05-08T05:51:04.065846+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:49:24.745045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093539+0000","last_change":"2024-05-08T05:51:05.090152+0000","last_active":"2024-05-08T05:51:05.093539+0000","last_peered":"2024-05-08T05:51:05.093539+0000","last_clean":"2024-05-08T05:51:05.093539+0000","last_became_active":"2024-05-08T05:51:05.089828+0000","last_became_peered":"2024-05-08T05:51:05.089828+0000","last_unstale":"2024-05-08T05:51:05.093539+0000","last_undegraded":"2024-05-08T05:51:05.093539+0000","last_fullsized":"2024-05-08T05:51:05.093539+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:34:19.612717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.069013+0000","last_change":"2024-05-08T05:51:04.069833+0000","last_active":"2024-05-08T05:51:05.069013+0000","last_peered":"2024-05-08T05:51:05.069013+0000","last_clean":"2024-05-08T05:51:05.069013+0000","last_became_active":"2024-05-08T05:51:04.069539+0000","last_became_peered":"2024-05-08T05:51:04.069539+0000","last_unstale":"2024-05-08T05:51:05.069013+0000","last_undegraded":"2024-05-08T05:51:05.069013+0000","last_fullsized":"2024-05-08T05:51:05.069013+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:36:49.817010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052495+0000","last_change":"2024-05-08T05:51:04.052495+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052495+0000","last_undegraded":"2024-05-08T05:51:04.052495+0000","last_fullsized":"2024-05-08T05:51:04.052495+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.065891+0000","last_change":"2024-05-08T05:51:04.063987+0000","last_active":"2024-05-08T05:51:04.065891+0000","last_peered":"2024-05-08T05:51:04.065891+0000","last_clean":"2024-05-08T05:51:04.065891+0000","last_became_active":"2024-05-08T05:51:04.063646+0000","last_became_peered":"2024-05-08T05:51:04.063646+0000","last_unstale":"2024-05-08T05:51:04.065891+0000","last_undegraded":"2024-05-08T05:51:04.065891+0000","last_fullsized":"2024-05-08T05:51:04.065891+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:58:37.220465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066817+0000","last_change":"2024-05-08T05:51:05.075930+0000","last_active":"2024-05-08T05:51:07.066817+0000","last_peered":"2024-05-08T05:51:07.066817+0000","last_clean":"2024-05-08T05:51:07.066817+0000","last_became_active":"2024-05-08T05:51:05.075052+0000","last_became_peered":"2024-05-08T05:51:05.075052+0000","last_unstale":"2024-05-08T05:51:07.066817+0000","last_undegraded":"2024-05-08T05:51:07.066817+0000","last_fullsized":"2024-05-08T05:51:07.066817+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:27:28.653248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066775+0000","last_change":"2024-05-08T05:51:04.063919+0000","last_active":"2024-05-08T05:51:07.066775+0000","last_peered":"2024-05-08T05:51:07.066775+0000","last_clean":"2024-05-08T05:51:07.066775+0000","last_became_active":"2024-05-08T05:51:04.063610+0000","last_became_peered":"2024-05-08T05:51:04.063610+0000","last_unstale":"2024-05-08T05:51:07.066775+0000","last_undegraded":"2024-05-08T05:51:07.066775+0000","last_fullsized":"2024-05-08T05:51:07.066775+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:26.855446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.076159+0000","last_change":"2024-05-08T05:51:05.073177+0000","last_active":"2024-05-08T05:51:05.076159+0000","last_peered":"2024-05-08T05:51:05.076159+0000","last_clean":"2024-05-08T05:51:05.076159+0000","last_became_active":"2024-05-08T05:51:05.072104+0000","last_became_peered":"2024-05-08T05:51:05.072104+0000","last_unstale":"2024-05-08T05:51:05.076159+0000","last_undegraded":"2024-05-08T05:51:05.076159+0000","last_fullsized":"2024-05-08T05:51:05.076159+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:29.725291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.070325+0000","last_change":"2024-05-08T05:51:04.069690+0000","last_active":"2024-05-08T05:51:04.070325+0000","last_peered":"2024-05-08T05:51:04.070325+0000","last_clean":"2024-05-08T05:51:04.070325+0000","last_became_active":"2024-05-08T05:51:04.069380+0000","last_became_peered":"2024-05-08T05:51:04.069380+0000","last_unstale":"2024-05-08T05:51:04.070325+0000","last_undegraded":"2024-05-08T05:51:04.070325+0000","last_fullsized":"2024-05-08T05:51:04.070325+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:26.339116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.053043+0000","last_change":"2024-05-08T05:51:04.053043+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.053043+0000","last_undegraded":"2024-05-08T05:51:04.053043+0000","last_fullsized":"2024-05-08T05:51:04.053043+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.070852+0000","last_change":"2024-05-08T05:51:04.070027+0000","last_active":"2024-05-08T05:51:04.070852+0000","last_peered":"2024-05-08T05:51:04.070852+0000","last_clean":"2024-05-08T05:51:04.070852+0000","last_became_active":"2024-05-08T05:51:04.069331+0000","last_became_peered":"2024-05-08T05:51:04.069331+0000","last_unstale":"2024-05-08T05:51:04.070852+0000","last_undegraded":"2024-05-08T05:51:04.070852+0000","last_fullsized":"2024-05-08T05:51:04.070852+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:28.717301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.079017+0000","last_change":"2024-05-08T05:51:05.079710+0000","last_active":"2024-05-08T05:51:06.079017+0000","last_peered":"2024-05-08T05:51:06.079017+0000","last_clean":"2024-05-08T05:51:06.079017+0000","last_became_active":"2024-05-08T05:51:05.079405+0000","last_became_peered":"2024-05-08T05:51:05.079405+0000","last_unstale":"2024-05-08T05:51:06.079017+0000","last_undegraded":"2024-05-08T05:51:06.079017+0000","last_fullsized":"2024-05-08T05:51:06.079017+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:57:15.216780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066813+0000","last_change":"2024-05-08T05:51:04.066485+0000","last_active":"2024-05-08T05:51:07.066813+0000","last_peered":"2024-05-08T05:51:07.066813+0000","last_clean":"2024-05-08T05:51:07.066813+0000","last_became_active":"2024-05-08T05:51:04.066192+0000","last_became_peered":"2024-05-08T05:51:04.066192+0000","last_unstale":"2024-05-08T05:51:07.066813+0000","last_undegraded":"2024-05-08T05:51:07.066813+0000","last_fullsized":"2024-05-08T05:51:07.066813+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:54:27.233544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066370+0000","last_change":"2024-05-08T05:51:05.074917+0000","last_active":"2024-05-08T05:51:07.066370+0000","last_peered":"2024-05-08T05:51:07.066370+0000","last_clean":"2024-05-08T05:51:07.066370+0000","last_became_active":"2024-05-08T05:51:05.074339+0000","last_became_peered":"2024-05-08T05:51:05.074339+0000","last_unstale":"2024-05-08T05:51:07.066370+0000","last_undegraded":"2024-05-08T05:51:07.066370+0000","last_fullsized":"2024-05-08T05:51:07.066370+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:22:04.619439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.062382+0000","last_change":"2024-05-08T05:51:04.057289+0000","last_active":"2024-05-08T05:51:05.062382+0000","last_peered":"2024-05-08T05:51:05.062382+0000","last_clean":"2024-05-08T05:51:05.062382+0000","last_became_active":"2024-05-08T05:51:04.056950+0000","last_became_peered":"2024-05-08T05:51:04.056950+0000","last_unstale":"2024-05-08T05:51:05.062382+0000","last_undegraded":"2024-05-08T05:51:05.062382+0000","last_fullsized":"2024-05-08T05:51:05.062382+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:17.541863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.079392+0000","last_change":"2024-05-08T05:51:05.076808+0000","last_active":"2024-05-08T05:51:05.079392+0000","last_peered":"2024-05-08T05:51:05.079392+0000","last_clean":"2024-05-08T05:51:05.079392+0000","last_became_active":"2024-05-08T05:51:05.076532+0000","last_became_peered":"2024-05-08T05:51:05.076532+0000","last_unstale":"2024-05-08T05:51:05.079392+0000","last_undegraded":"2024-05-08T05:51:05.079392+0000","last_fullsized":"2024-05-08T05:51:05.079392+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:12.526046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.062570+0000","last_change":"2024-05-08T05:51:04.054314+0000","last_active":"2024-05-08T05:51:05.062570+0000","last_peered":"2024-05-08T05:51:05.062570+0000","last_clean":"2024-05-08T05:51:05.062570+0000","last_became_active":"2024-05-08T05:51:04.053754+0000","last_became_peered":"2024-05-08T05:51:04.053754+0000","last_unstale":"2024-05-08T05:51:05.062570+0000","last_undegraded":"2024-05-08T05:51:05.062570+0000","last_fullsized":"2024-05-08T05:51:05.062570+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:26.339116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.053473+0000","last_change":"2024-05-08T05:51:04.053473+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.053473+0000","last_undegraded":"2024-05-08T05:51:04.053473+0000","last_fullsized":"2024-05-08T05:51:04.053473+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066301+0000","last_change":"2024-05-08T05:51:04.064412+0000","last_active":"2024-05-08T05:51:07.066301+0000","last_peered":"2024-05-08T05:51:07.066301+0000","last_clean":"2024-05-08T05:51:07.066301+0000","last_became_active":"2024-05-08T05:51:04.064134+0000","last_became_peered":"2024-05-08T05:51:04.064134+0000","last_unstale":"2024-05-08T05:51:07.066301+0000","last_undegraded":"2024-05-08T05:51:07.066301+0000","last_fullsized":"2024-05-08T05:51:07.066301+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:37:09.043074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088681+0000","last_change":"2024-05-08T05:51:05.087096+0000","last_active":"2024-05-08T05:51:05.088681+0000","last_peered":"2024-05-08T05:51:05.088681+0000","last_clean":"2024-05-08T05:51:05.088681+0000","last_became_active":"2024-05-08T05:51:05.085170+0000","last_became_peered":"2024-05-08T05:51:05.085170+0000","last_unstale":"2024-05-08T05:51:05.088681+0000","last_undegraded":"2024-05-08T05:51:05.088681+0000","last_fullsized":"2024-05-08T05:51:05.088681+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:20.213009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.066642+0000","last_change":"2024-05-08T05:51:04.065454+0000","last_active":"2024-05-08T05:51:04.066642+0000","last_peered":"2024-05-08T05:51:04.066642+0000","last_clean":"2024-05-08T05:51:04.066642+0000","last_became_active":"2024-05-08T05:51:04.065175+0000","last_became_peered":"2024-05-08T05:51:04.065175+0000","last_unstale":"2024-05-08T05:51:04.066642+0000","last_undegraded":"2024-05-08T05:51:04.066642+0000","last_fullsized":"2024-05-08T05:51:04.066642+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:01:19.756722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.051048+0000","last_change":"2024-05-08T05:51:04.051048+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.051048+0000","last_undegraded":"2024-05-08T05:51:04.051048+0000","last_fullsized":"2024-05-08T05:51:04.051048+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"118'8","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.096717+0000","last_change":"2024-05-08T05:51:04.068279+0000","last_active":"2024-05-08T05:51:05.096717+0000","last_peered":"2024-05-08T05:51:05.096717+0000","last_clean":"2024-05-08T05:51:05.096717+0000","last_became_active":"2024-05-08T05:51:04.067953+0000","last_became_peered":"2024-05-08T05:51:04.067953+0000","last_unstale":"2024-05-08T05:51:05.096717+0000","last_undegraded":"2024-05-08T05:51:05.096717+0000","last_fullsized":"2024-05-08T05:51:05.096717+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:54.640197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":288,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.055816+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:51:04.055816+0000","last_peered":"2024-05-08T05:51:04.055816+0000","last_clean":"2024-05-08T05:51:04.055816+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:51:04.055816+0000","last_undegraded":"2024-05-08T05:51:04.055816+0000","last_fullsized":"2024-05-08T05:51:04.055816+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":1668,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":3302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542265,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2048000,"data_stored":1266440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542266,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1454080,"data_stored":676072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542265,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1454080,"data_stored":676072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":684851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1482752,"data_stored":684885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1478656,"data_stored":689931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1470464,"data_stored":683261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2076672,"data_stored":1280321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:51:09.158 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:51:09.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.260+0000 7facf8e25640 1 Processor -- start 2024-05-08T05:51:09.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.260+0000 7facf8e25640 1 -- start start 2024-05-08T05:51:09.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.261+0000 7facf8e25640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7facf415ce90 0x7facf4070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:09.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.261+0000 7facf8e25640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facf406d3b0 0x7facf406d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:09.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.261+0000 7facf8e25640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facf4071280 con 0x7facf40733e0 2024-05-08T05:51:09.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.261+0000 7facf8e25640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7facf406dd60 con 0x7facf415ce90 2024-05-08T05:51:09.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.261+0000 7facf8e25640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7facf40d1390 con 0x7facf406d3b0 2024-05-08T05:51:09.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.261+0000 7facf259c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7facf415ce90 0x7facf4070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:09.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.261+0000 7facf259c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7facf415ce90 0x7facf4070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37334/0 (socket says 172.21.15.45:37334) 2024-05-08T05:51:09.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.261+0000 7facf259c640 1 -- 172.21.15.45:0/3471569379 learned_addr learned my addr 172.21.15.45:0/3471569379 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:09.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.261+0000 7facf2d9d640 1 --2- 172.21.15.45:0/3471569379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facf406d3b0 0x7facf406d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:09.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.261+0000 7facf259c640 1 -- 172.21.15.45:0/3471569379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facf406d3b0 msgr2=0x7facf406d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:09.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.261+0000 7facf259c640 1 --2- 172.21.15.45:0/3471569379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facf406d3b0 0x7facf406d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.261+0000 7facf259c640 1 -- 172.21.15.45:0/3471569379 >> v1:172.21.15.63:6789/0 conn(0x7facf40733e0 legacy=0x7facf40d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:09.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.261+0000 7facf259c640 1 -- 172.21.15.45:0/3471569379 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf40cff00 con 0x7facf415ce90 2024-05-08T05:51:09.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.262+0000 7facf259c640 1 --2- 172.21.15.45:0/3471569379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facf415ce90 0x7facf4070d40 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7face4009870 tx=0x7face402a1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f2bc545a8bbf2b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:09.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.262+0000 7facf1d9b640 1 -- 172.21.15.45:0/3471569379 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4037070 con 0x7facf415ce90 2024-05-08T05:51:09.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.262+0000 7facf1d9b640 1 -- 172.21.15.45:0/3471569379 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face4032440 con 0x7facf415ce90 2024-05-08T05:51:09.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.262+0000 7facf1d9b640 1 -- 172.21.15.45:0/3471569379 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4032810 con 0x7facf415ce90 2024-05-08T05:51:09.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.263+0000 7facf8e25640 1 -- 172.21.15.45:0/3471569379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facf415ce90 msgr2=0x7facf4070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:09.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.263+0000 7facf8e25640 1 --2- 172.21.15.45:0/3471569379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facf415ce90 0x7facf4070d40 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7face4009870 tx=0x7face402a1b0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.263+0000 7facf8e25640 1 -- 172.21.15.45:0/3471569379 shutdown_connections 2024-05-08T05:51:09.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.263+0000 7facf8e25640 1 --2- 172.21.15.45:0/3471569379 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facf406d3b0 0x7facf406d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.263+0000 7facf8e25640 1 --2- 172.21.15.45:0/3471569379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facf415ce90 0x7facf4070d40 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.263+0000 7facf8e25640 1 -- 172.21.15.45:0/3471569379 >> 172.21.15.45:0/3471569379 conn(0x7facf40728c0 msgr2=0x7facf4072cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:09.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.263+0000 7facf8e25640 1 -- 172.21.15.45:0/3471569379 shutdown_connections 2024-05-08T05:51:09.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.263+0000 7facf8e25640 1 -- 172.21.15.45:0/3471569379 wait complete. 2024-05-08T05:51:09.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.264+0000 7facf8e25640 1 Processor -- start 2024-05-08T05:51:09.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.264+0000 7facf8e25640 1 -- start start 2024-05-08T05:51:09.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.265+0000 7facf8e25640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facf406d3b0 0x7facf41440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:09.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.265+0000 7facf8e25640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facf40733e0 0x7facf4155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.265+0000 7facf8e25640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facf415ce90 0x7facf4156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.265+0000 7facf8e25640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7facf40d0480 con 0x7facf40733e0 2024-05-08T05:51:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.265+0000 7facf8e25640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7facf40d01a0 con 0x7facf406d3b0 2024-05-08T05:51:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.265+0000 7facf8e25640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7facf40d0760 con 0x7facf415ce90 2024-05-08T05:51:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.265+0000 7facebfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facf40733e0 0x7facf4155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.265+0000 7facebfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facf40733e0 0x7facf4155fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37336/0 (socket says 172.21.15.45:37336) 2024-05-08T05:51:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.265+0000 7facebfff640 1 -- 172.21.15.45:0/256162866 learned_addr learned my addr 172.21.15.45:0/256162866 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.265+0000 7facf2d9d640 1 --2- 172.21.15.45:0/256162866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facf415ce90 0x7facf4156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.265+0000 7facf259c640 1 --2- 172.21.15.45:0/256162866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facf406d3b0 0x7facf41440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.266+0000 7facebfff640 1 -- 172.21.15.45:0/256162866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facf415ce90 msgr2=0x7facf4156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.266+0000 7facebfff640 1 --2- 172.21.15.45:0/256162866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facf415ce90 0x7facf4156510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.266+0000 7facebfff640 1 -- 172.21.15.45:0/256162866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facf406d3b0 msgr2=0x7facf41440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.266+0000 7facebfff640 1 --2- 172.21.15.45:0/256162866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facf406d3b0 0x7facf41440c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.266+0000 7facebfff640 1 -- 172.21.15.45:0/256162866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf415bc00 con 0x7facf40733e0 2024-05-08T05:51:09.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.266+0000 7facf2d9d640 1 --2- 172.21.15.45:0/256162866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facf415ce90 0x7facf4156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:09.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.266+0000 7facf259c640 1 --2- 172.21.15.45:0/256162866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facf406d3b0 0x7facf41440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:09.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.266+0000 7facebfff640 1 --2- 172.21.15.45:0/256162866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facf40733e0 0x7facf4155fd0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7facdc00c430 tx=0x7facdc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:09.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.266+0000 7faceaffd640 1 -- 172.21.15.45:0/256162866 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc011790 con 0x7facf40733e0 2024-05-08T05:51:09.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.266+0000 7faceaffd640 1 -- 172.21.15.45:0/256162866 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facdc017070 con 0x7facf40733e0 2024-05-08T05:51:09.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.266+0000 7facf8e25640 1 -- 172.21.15.45:0/256162866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf4153b20 con 0x7facf40733e0 2024-05-08T05:51:09.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.266+0000 7faceaffd640 1 -- 172.21.15.45:0/256162866 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc011bb0 con 0x7facf40733e0 2024-05-08T05:51:09.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.266+0000 7facf8e25640 1 -- 172.21.15.45:0/256162866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf41540e0 con 0x7facf40733e0 2024-05-08T05:51:09.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.267+0000 7facf8e25640 1 -- 172.21.15.45:0/256162866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf41542e0 con 0x7facf40733e0 2024-05-08T05:51:09.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.267+0000 7faceaffd640 1 -- 172.21.15.45:0/256162866 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7facdc01c460 con 0x7facf40733e0 2024-05-08T05:51:09.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.268+0000 7faceaffd640 1 --2- 172.21.15.45:0/256162866 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7facd4076300 0x7facd40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:09.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.268+0000 7faceaffd640 1 -- 172.21.15.45:0/256162866 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7facdc097610 con 0x7facf40733e0 2024-05-08T05:51:09.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.268+0000 7facf259c640 1 --2- 172.21.15.45:0/256162866 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7facd4076300 0x7facd40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:09.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.269+0000 7facf259c640 1 --2- 172.21.15.45:0/256162866 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7facd4076300 0x7facd40787c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7face4034000 tx=0x7face402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:09.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.272+0000 7faceaffd640 1 -- 172.21.15.45:0/256162866 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7facf41542e0 con 0x7facf40733e0 2024-05-08T05:51:09.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.461+0000 7facf8e25640 1 -- 172.21.15.45:0/256162866 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7facf40693f0 con 0x7facd4076300 2024-05-08T05:51:09.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.468+0000 7faceaffd640 1 -- 172.21.15.45:0/256162866 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250600 (secure 0 0 0) 0x7facf40693f0 con 0x7facd4076300 2024-05-08T05:51:09.471 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:09.473 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:51:09.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.471+0000 7facf8e25640 1 -- 172.21.15.45:0/256162866 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7facd4076300 msgr2=0x7facd40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:09.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.471+0000 7facf8e25640 1 --2- 172.21.15.45:0/256162866 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7facd4076300 0x7facd40787c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7face4034000 tx=0x7face402e000 comp rx=0 tx=0).stop 2024-05-08T05:51:09.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.471+0000 7facf8e25640 1 -- 172.21.15.45:0/256162866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facf40733e0 msgr2=0x7facf4155fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:09.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.471+0000 7facf8e25640 1 --2- 172.21.15.45:0/256162866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facf40733e0 0x7facf4155fd0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7facdc00c430 tx=0x7facdc00c8f0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.471+0000 7facf8e25640 1 -- 172.21.15.45:0/256162866 shutdown_connections 2024-05-08T05:51:09.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.471+0000 7facf8e25640 1 --2- 172.21.15.45:0/256162866 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7facd4076300 0x7facd40787c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.471+0000 7facf8e25640 1 --2- 172.21.15.45:0/256162866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7facf415ce90 0x7facf4156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.471+0000 7facf8e25640 1 --2- 172.21.15.45:0/256162866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7facf40733e0 0x7facf4155fd0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.471+0000 7facf8e25640 1 --2- 172.21.15.45:0/256162866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7facf406d3b0 0x7facf41440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.471+0000 7facf8e25640 1 -- 172.21.15.45:0/256162866 >> 172.21.15.45:0/256162866 conn(0x7facf40728c0 msgr2=0x7facf40b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:09.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.471+0000 7facf8e25640 1 -- 172.21.15.45:0/256162866 shutdown_connections 2024-05-08T05:51:09.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.471+0000 7facf8e25640 1 -- 172.21.15.45:0/256162866 wait complete. 2024-05-08T05:51:09.483 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2024-05-08T05:51:08.974825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592036,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":133,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262108,"kb_used_data":12640,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712612,"statfs":{"total":773094113280,"available":772825714688,"internally_reserved":0,"allocated":12943360,"data_stored":6641833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1668,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.928070"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.073391+0000","last_change":"2024-05-08T05:51:05.069476+0000","last_active":"2024-05-08T05:51:05.073391+0000","last_peered":"2024-05-08T05:51:05.073391+0000","last_clean":"2024-05-08T05:51:05.073391+0000","last_became_active":"2024-05-08T05:51:05.069131+0000","last_became_peered":"2024-05-08T05:51:05.069131+0000","last_unstale":"2024-05-08T05:51:05.073391+0000","last_undegraded":"2024-05-08T05:51:05.073391+0000","last_fullsized":"2024-05-08T05:51:05.073391+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:05:09.239164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093231+0000","last_change":"2024-05-08T05:51:05.084139+0000","last_active":"2024-05-08T05:51:05.093231+0000","last_peered":"2024-05-08T05:51:05.093231+0000","last_clean":"2024-05-08T05:51:05.093231+0000","last_became_active":"2024-05-08T05:51:05.081830+0000","last_became_peered":"2024-05-08T05:51:05.081830+0000","last_unstale":"2024-05-08T05:51:05.093231+0000","last_undegraded":"2024-05-08T05:51:05.093231+0000","last_fullsized":"2024-05-08T05:51:05.093231+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:37.715909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093594+0000","last_change":"2024-05-08T05:51:05.090876+0000","last_active":"2024-05-08T05:51:05.093594+0000","last_peered":"2024-05-08T05:51:05.093594+0000","last_clean":"2024-05-08T05:51:05.093594+0000","last_became_active":"2024-05-08T05:51:05.089329+0000","last_became_peered":"2024-05-08T05:51:05.089329+0000","last_unstale":"2024-05-08T05:51:05.093594+0000","last_undegraded":"2024-05-08T05:51:05.093594+0000","last_fullsized":"2024-05-08T05:51:05.093594+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:51:43.350274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052577+0000","last_change":"2024-05-08T05:51:04.052577+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052577+0000","last_undegraded":"2024-05-08T05:51:04.052577+0000","last_fullsized":"2024-05-08T05:51:04.052577+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.077282+0000","last_change":"2024-05-08T05:51:05.072296+0000","last_active":"2024-05-08T05:51:05.077282+0000","last_peered":"2024-05-08T05:51:05.077282+0000","last_clean":"2024-05-08T05:51:05.077282+0000","last_became_active":"2024-05-08T05:51:05.072009+0000","last_became_peered":"2024-05-08T05:51:05.072009+0000","last_unstale":"2024-05-08T05:51:05.077282+0000","last_undegraded":"2024-05-08T05:51:05.077282+0000","last_fullsized":"2024-05-08T05:51:05.077282+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:12:02.641413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066407+0000","last_change":"2024-05-08T05:51:05.082847+0000","last_active":"2024-05-08T05:51:07.066407+0000","last_peered":"2024-05-08T05:51:07.066407+0000","last_clean":"2024-05-08T05:51:07.066407+0000","last_became_active":"2024-05-08T05:51:05.082267+0000","last_became_peered":"2024-05-08T05:51:05.082267+0000","last_unstale":"2024-05-08T05:51:07.066407+0000","last_undegraded":"2024-05-08T05:51:07.066407+0000","last_fullsized":"2024-05-08T05:51:07.066407+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:16:17.521040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066220+0000","last_change":"2024-05-08T05:51:05.087173+0000","last_active":"2024-05-08T05:51:07.066220+0000","last_peered":"2024-05-08T05:51:07.066220+0000","last_clean":"2024-05-08T05:51:07.066220+0000","last_became_active":"2024-05-08T05:51:05.086914+0000","last_became_peered":"2024-05-08T05:51:05.086914+0000","last_unstale":"2024-05-08T05:51:07.066220+0000","last_undegraded":"2024-05-08T05:51:07.066220+0000","last_fullsized":"2024-05-08T05:51:07.066220+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:42:24.823847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093326+0000","last_change":"2024-05-08T05:51:05.085921+0000","last_active":"2024-05-08T05:51:05.093326+0000","last_peered":"2024-05-08T05:51:05.093326+0000","last_clean":"2024-05-08T05:51:05.093326+0000","last_became_active":"2024-05-08T05:51:05.083800+0000","last_became_peered":"2024-05-08T05:51:05.083800+0000","last_unstale":"2024-05-08T05:51:05.093326+0000","last_undegraded":"2024-05-08T05:51:05.093326+0000","last_fullsized":"2024-05-08T05:51:05.093326+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:47.453010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.053739+0000","last_change":"2024-05-08T05:51:04.053739+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.053739+0000","last_undegraded":"2024-05-08T05:51:04.053739+0000","last_fullsized":"2024-05-08T05:51:04.053739+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.075989+0000","last_change":"2024-05-08T05:51:05.071111+0000","last_active":"2024-05-08T05:51:05.075989+0000","last_peered":"2024-05-08T05:51:05.075989+0000","last_clean":"2024-05-08T05:51:05.075989+0000","last_became_active":"2024-05-08T05:51:05.070823+0000","last_became_peered":"2024-05-08T05:51:05.070823+0000","last_unstale":"2024-05-08T05:51:05.075989+0000","last_undegraded":"2024-05-08T05:51:05.075989+0000","last_fullsized":"2024-05-08T05:51:05.075989+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:18.549853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.073515+0000","last_change":"2024-05-08T05:51:05.070718+0000","last_active":"2024-05-08T05:51:05.073515+0000","last_peered":"2024-05-08T05:51:05.073515+0000","last_clean":"2024-05-08T05:51:05.073515+0000","last_became_active":"2024-05-08T05:51:05.070422+0000","last_became_peered":"2024-05-08T05:51:05.070422+0000","last_unstale":"2024-05-08T05:51:05.073515+0000","last_undegraded":"2024-05-08T05:51:05.073515+0000","last_fullsized":"2024-05-08T05:51:05.073515+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:29.725291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066591+0000","last_change":"2024-05-08T05:51:05.076336+0000","last_active":"2024-05-08T05:51:07.066591+0000","last_peered":"2024-05-08T05:51:07.066591+0000","last_clean":"2024-05-08T05:51:07.066591+0000","last_became_active":"2024-05-08T05:51:05.075648+0000","last_became_peered":"2024-05-08T05:51:05.075648+0000","last_unstale":"2024-05-08T05:51:07.066591+0000","last_undegraded":"2024-05-08T05:51:07.066591+0000","last_fullsized":"2024-05-08T05:51:07.066591+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:27.438502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088120+0000","last_change":"2024-05-08T05:51:05.079977+0000","last_active":"2024-05-08T05:51:05.088120+0000","last_peered":"2024-05-08T05:51:05.088120+0000","last_clean":"2024-05-08T05:51:05.088120+0000","last_became_active":"2024-05-08T05:51:05.078938+0000","last_became_peered":"2024-05-08T05:51:05.078938+0000","last_unstale":"2024-05-08T05:51:05.088120+0000","last_undegraded":"2024-05-08T05:51:05.088120+0000","last_fullsized":"2024-05-08T05:51:05.088120+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:00.252213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093291+0000","last_change":"2024-05-08T05:51:05.086072+0000","last_active":"2024-05-08T05:51:05.093291+0000","last_peered":"2024-05-08T05:51:05.093291+0000","last_clean":"2024-05-08T05:51:05.093291+0000","last_became_active":"2024-05-08T05:51:05.084475+0000","last_became_peered":"2024-05-08T05:51:05.084475+0000","last_unstale":"2024-05-08T05:51:05.093291+0000","last_undegraded":"2024-05-08T05:51:05.093291+0000","last_fullsized":"2024-05-08T05:51:05.093291+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:08:48.269564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088031+0000","last_change":"2024-05-08T05:51:05.083505+0000","last_active":"2024-05-08T05:51:05.088031+0000","last_peered":"2024-05-08T05:51:05.088031+0000","last_clean":"2024-05-08T05:51:05.088031+0000","last_became_active":"2024-05-08T05:51:05.083236+0000","last_became_peered":"2024-05-08T05:51:05.083236+0000","last_unstale":"2024-05-08T05:51:05.088031+0000","last_undegraded":"2024-05-08T05:51:05.088031+0000","last_fullsized":"2024-05-08T05:51:05.088031+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:20.213009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066209+0000","last_change":"2024-05-08T05:51:05.077087+0000","last_active":"2024-05-08T05:51:07.066209+0000","last_peered":"2024-05-08T05:51:07.066209+0000","last_clean":"2024-05-08T05:51:07.066209+0000","last_became_active":"2024-05-08T05:51:05.076743+0000","last_became_peered":"2024-05-08T05:51:05.076743+0000","last_unstale":"2024-05-08T05:51:07.066209+0000","last_undegraded":"2024-05-08T05:51:07.066209+0000","last_fullsized":"2024-05-08T05:51:07.066209+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:03.278751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088248+0000","last_change":"2024-05-08T05:51:05.081532+0000","last_active":"2024-05-08T05:51:05.088248+0000","last_peered":"2024-05-08T05:51:05.088248+0000","last_clean":"2024-05-08T05:51:05.088248+0000","last_became_active":"2024-05-08T05:51:05.081228+0000","last_became_peered":"2024-05-08T05:51:05.081228+0000","last_unstale":"2024-05-08T05:51:05.088248+0000","last_undegraded":"2024-05-08T05:51:05.088248+0000","last_fullsized":"2024-05-08T05:51:05.088248+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:38:53.162197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093376+0000","last_change":"2024-05-08T05:51:05.086943+0000","last_active":"2024-05-08T05:51:05.093376+0000","last_peered":"2024-05-08T05:51:05.093376+0000","last_clean":"2024-05-08T05:51:05.093376+0000","last_became_active":"2024-05-08T05:51:05.086221+0000","last_became_peered":"2024-05-08T05:51:05.086221+0000","last_unstale":"2024-05-08T05:51:05.093376+0000","last_undegraded":"2024-05-08T05:51:05.093376+0000","last_fullsized":"2024-05-08T05:51:05.093376+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:34:12.399560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.056046+0000","last_change":"2024-05-08T05:51:04.056046+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.056046+0000","last_undegraded":"2024-05-08T05:51:04.056046+0000","last_fullsized":"2024-05-08T05:51:04.056046+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078597+0000","last_change":"2024-05-08T05:51:05.080761+0000","last_active":"2024-05-08T05:51:06.078597+0000","last_peered":"2024-05-08T05:51:06.078597+0000","last_clean":"2024-05-08T05:51:06.078597+0000","last_became_active":"2024-05-08T05:51:05.080381+0000","last_became_peered":"2024-05-08T05:51:05.080381+0000","last_unstale":"2024-05-08T05:51:06.078597+0000","last_undegraded":"2024-05-08T05:51:06.078597+0000","last_fullsized":"2024-05-08T05:51:06.078597+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:42.792089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.051557+0000","last_change":"2024-05-08T05:51:04.051557+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.051557+0000","last_undegraded":"2024-05-08T05:51:04.051557+0000","last_fullsized":"2024-05-08T05:51:04.051557+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.084370+0000","last_change":"2024-05-08T05:51:05.079566+0000","last_active":"2024-05-08T05:51:05.084370+0000","last_peered":"2024-05-08T05:51:05.084370+0000","last_clean":"2024-05-08T05:51:05.084370+0000","last_became_active":"2024-05-08T05:51:05.079256+0000","last_became_peered":"2024-05-08T05:51:05.079256+0000","last_unstale":"2024-05-08T05:51:05.084370+0000","last_undegraded":"2024-05-08T05:51:05.084370+0000","last_fullsized":"2024-05-08T05:51:05.084370+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:51:07.851943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.050947+0000","last_change":"2024-05-08T05:51:04.050947+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.050947+0000","last_undegraded":"2024-05-08T05:51:04.050947+0000","last_fullsized":"2024-05-08T05:51:04.050947+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052838+0000","last_change":"2024-05-08T05:51:04.052838+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052838+0000","last_undegraded":"2024-05-08T05:51:04.052838+0000","last_fullsized":"2024-05-08T05:51:04.052838+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.053724+0000","last_change":"2024-05-08T05:51:04.053724+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.053724+0000","last_undegraded":"2024-05-08T05:51:04.053724+0000","last_fullsized":"2024-05-08T05:51:04.053724+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078673+0000","last_change":"2024-05-08T05:51:05.081486+0000","last_active":"2024-05-08T05:51:06.078673+0000","last_peered":"2024-05-08T05:51:06.078673+0000","last_clean":"2024-05-08T05:51:06.078673+0000","last_became_active":"2024-05-08T05:51:05.081143+0000","last_became_peered":"2024-05-08T05:51:05.081143+0000","last_unstale":"2024-05-08T05:51:06.078673+0000","last_undegraded":"2024-05-08T05:51:06.078673+0000","last_fullsized":"2024-05-08T05:51:06.078673+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:44:56.396392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.055539+0000","last_change":"2024-05-08T05:51:04.055539+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.055539+0000","last_undegraded":"2024-05-08T05:51:04.055539+0000","last_fullsized":"2024-05-08T05:51:04.055539+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066746+0000","last_change":"2024-05-08T05:51:05.081306+0000","last_active":"2024-05-08T05:51:07.066746+0000","last_peered":"2024-05-08T05:51:07.066746+0000","last_clean":"2024-05-08T05:51:07.066746+0000","last_became_active":"2024-05-08T05:51:05.081023+0000","last_became_peered":"2024-05-08T05:51:05.081023+0000","last_unstale":"2024-05-08T05:51:07.066746+0000","last_undegraded":"2024-05-08T05:51:07.066746+0000","last_fullsized":"2024-05-08T05:51:07.066746+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:20:04.797865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078729+0000","last_change":"2024-05-08T05:51:05.082212+0000","last_active":"2024-05-08T05:51:06.078729+0000","last_peered":"2024-05-08T05:51:06.078729+0000","last_clean":"2024-05-08T05:51:06.078729+0000","last_became_active":"2024-05-08T05:51:05.081847+0000","last_became_peered":"2024-05-08T05:51:05.081847+0000","last_unstale":"2024-05-08T05:51:06.078729+0000","last_undegraded":"2024-05-08T05:51:06.078729+0000","last_fullsized":"2024-05-08T05:51:06.078729+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:10.568249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093267+0000","last_change":"2024-05-08T05:51:05.085821+0000","last_active":"2024-05-08T05:51:05.093267+0000","last_peered":"2024-05-08T05:51:05.093267+0000","last_clean":"2024-05-08T05:51:05.093267+0000","last_became_active":"2024-05-08T05:51:05.083777+0000","last_became_peered":"2024-05-08T05:51:05.083777+0000","last_unstale":"2024-05-08T05:51:05.093267+0000","last_undegraded":"2024-05-08T05:51:05.093267+0000","last_fullsized":"2024-05-08T05:51:05.093267+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:22:46.959500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.051392+0000","last_change":"2024-05-08T05:51:04.051392+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.051392+0000","last_undegraded":"2024-05-08T05:51:04.051392+0000","last_fullsized":"2024-05-08T05:51:04.051392+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.050670+0000","last_change":"2024-05-08T05:51:04.050670+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.050670+0000","last_undegraded":"2024-05-08T05:51:04.050670+0000","last_fullsized":"2024-05-08T05:51:04.050670+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.076195+0000","last_change":"2024-05-08T05:51:05.072621+0000","last_active":"2024-05-08T05:51:05.076195+0000","last_peered":"2024-05-08T05:51:05.076195+0000","last_clean":"2024-05-08T05:51:05.076195+0000","last_became_active":"2024-05-08T05:51:05.072317+0000","last_became_peered":"2024-05-08T05:51:05.072317+0000","last_unstale":"2024-05-08T05:51:05.076195+0000","last_undegraded":"2024-05-08T05:51:05.076195+0000","last_fullsized":"2024-05-08T05:51:05.076195+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:05:09.239164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.064918+0000","last_change":"2024-05-08T05:51:04.063764+0000","last_active":"2024-05-08T05:51:04.064918+0000","last_peered":"2024-05-08T05:51:04.064918+0000","last_clean":"2024-05-08T05:51:04.064918+0000","last_became_active":"2024-05-08T05:51:04.063484+0000","last_became_peered":"2024-05-08T05:51:04.063484+0000","last_unstale":"2024-05-08T05:51:04.064918+0000","last_undegraded":"2024-05-08T05:51:04.064918+0000","last_fullsized":"2024-05-08T05:51:04.064918+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:17.541863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066655+0000","last_change":"2024-05-08T05:51:05.084520+0000","last_active":"2024-05-08T05:51:07.066655+0000","last_peered":"2024-05-08T05:51:07.066655+0000","last_clean":"2024-05-08T05:51:07.066655+0000","last_became_active":"2024-05-08T05:51:05.084198+0000","last_became_peered":"2024-05-08T05:51:05.084198+0000","last_unstale":"2024-05-08T05:51:07.066655+0000","last_undegraded":"2024-05-08T05:51:07.066655+0000","last_fullsized":"2024-05-08T05:51:07.066655+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:08:36.961131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.063174+0000","last_change":"2024-05-08T05:51:04.057409+0000","last_active":"2024-05-08T05:51:05.063174+0000","last_peered":"2024-05-08T05:51:05.063174+0000","last_clean":"2024-05-08T05:51:05.063174+0000","last_became_active":"2024-05-08T05:51:04.057127+0000","last_became_peered":"2024-05-08T05:51:04.057127+0000","last_unstale":"2024-05-08T05:51:05.063174+0000","last_undegraded":"2024-05-08T05:51:05.063174+0000","last_fullsized":"2024-05-08T05:51:05.063174+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:33.984066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052424+0000","last_change":"2024-05-08T05:51:04.052424+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052424+0000","last_undegraded":"2024-05-08T05:51:04.052424+0000","last_fullsized":"2024-05-08T05:51:04.052424+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.065887+0000","last_change":"2024-05-08T05:51:04.063770+0000","last_active":"2024-05-08T05:51:04.065887+0000","last_peered":"2024-05-08T05:51:04.065887+0000","last_clean":"2024-05-08T05:51:04.065887+0000","last_became_active":"2024-05-08T05:51:04.063471+0000","last_became_peered":"2024-05-08T05:51:04.063471+0000","last_unstale":"2024-05-08T05:51:04.065887+0000","last_undegraded":"2024-05-08T05:51:04.065887+0000","last_fullsized":"2024-05-08T05:51:04.065887+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:14:14.017450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.053316+0000","last_change":"2024-05-08T05:51:04.053316+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.053316+0000","last_undegraded":"2024-05-08T05:51:04.053316+0000","last_fullsized":"2024-05-08T05:51:04.053316+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.095608+0000","last_change":"2024-05-08T05:51:04.060756+0000","last_active":"2024-05-08T05:51:05.095608+0000","last_peered":"2024-05-08T05:51:05.095608+0000","last_clean":"2024-05-08T05:51:05.095608+0000","last_became_active":"2024-05-08T05:51:04.060480+0000","last_became_peered":"2024-05-08T05:51:04.060480+0000","last_unstale":"2024-05-08T05:51:05.095608+0000","last_undegraded":"2024-05-08T05:51:05.095608+0000","last_fullsized":"2024-05-08T05:51:05.095608+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:58:38.791116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.076073+0000","last_change":"2024-05-08T05:51:05.072513+0000","last_active":"2024-05-08T05:51:05.076073+0000","last_peered":"2024-05-08T05:51:05.076073+0000","last_clean":"2024-05-08T05:51:05.076073+0000","last_became_active":"2024-05-08T05:51:05.072191+0000","last_became_peered":"2024-05-08T05:51:05.072191+0000","last_unstale":"2024-05-08T05:51:05.076073+0000","last_undegraded":"2024-05-08T05:51:05.076073+0000","last_fullsized":"2024-05-08T05:51:05.076073+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:27.347106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.062425+0000","last_change":"2024-05-08T05:51:04.058250+0000","last_active":"2024-05-08T05:51:05.062425+0000","last_peered":"2024-05-08T05:51:05.062425+0000","last_clean":"2024-05-08T05:51:05.062425+0000","last_became_active":"2024-05-08T05:51:04.057765+0000","last_became_peered":"2024-05-08T05:51:04.057765+0000","last_unstale":"2024-05-08T05:51:05.062425+0000","last_undegraded":"2024-05-08T05:51:05.062425+0000","last_fullsized":"2024-05-08T05:51:05.062425+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:46:01.704267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.084362+0000","last_change":"2024-05-08T05:51:05.079919+0000","last_active":"2024-05-08T05:51:05.084362+0000","last_peered":"2024-05-08T05:51:05.084362+0000","last_clean":"2024-05-08T05:51:05.084362+0000","last_became_active":"2024-05-08T05:51:05.079552+0000","last_became_peered":"2024-05-08T05:51:05.079552+0000","last_unstale":"2024-05-08T05:51:05.084362+0000","last_undegraded":"2024-05-08T05:51:05.084362+0000","last_fullsized":"2024-05-08T05:51:05.084362+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:38:53.162197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066334+0000","last_change":"2024-05-08T05:51:04.069265+0000","last_active":"2024-05-08T05:51:07.066334+0000","last_peered":"2024-05-08T05:51:07.066334+0000","last_clean":"2024-05-08T05:51:07.066334+0000","last_became_active":"2024-05-08T05:51:04.068997+0000","last_became_peered":"2024-05-08T05:51:04.068997+0000","last_unstale":"2024-05-08T05:51:07.066334+0000","last_undegraded":"2024-05-08T05:51:07.066334+0000","last_fullsized":"2024-05-08T05:51:07.066334+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:55.959876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.084003+0000","last_change":"2024-05-08T05:51:05.079354+0000","last_active":"2024-05-08T05:51:05.084003+0000","last_peered":"2024-05-08T05:51:05.084003+0000","last_clean":"2024-05-08T05:51:05.084003+0000","last_became_active":"2024-05-08T05:51:05.079042+0000","last_became_peered":"2024-05-08T05:51:05.079042+0000","last_unstale":"2024-05-08T05:51:05.084003+0000","last_undegraded":"2024-05-08T05:51:05.084003+0000","last_fullsized":"2024-05-08T05:51:05.084003+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:20:04.111530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.065847+0000","last_change":"2024-05-08T05:51:04.058859+0000","last_active":"2024-05-08T05:51:04.065847+0000","last_peered":"2024-05-08T05:51:04.065847+0000","last_clean":"2024-05-08T05:51:04.065847+0000","last_became_active":"2024-05-08T05:51:04.058518+0000","last_became_peered":"2024-05-08T05:51:04.058518+0000","last_unstale":"2024-05-08T05:51:04.065847+0000","last_undegraded":"2024-05-08T05:51:04.065847+0000","last_fullsized":"2024-05-08T05:51:04.065847+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:02:28.473412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.087245+0000","last_change":"2024-05-08T05:51:05.077404+0000","last_active":"2024-05-08T05:51:05.087245+0000","last_peered":"2024-05-08T05:51:05.087245+0000","last_clean":"2024-05-08T05:51:05.087245+0000","last_became_active":"2024-05-08T05:51:05.077093+0000","last_became_peered":"2024-05-08T05:51:05.077093+0000","last_unstale":"2024-05-08T05:51:05.087245+0000","last_undegraded":"2024-05-08T05:51:05.087245+0000","last_fullsized":"2024-05-08T05:51:05.087245+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:12:02.641413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066301+0000","last_change":"2024-05-08T05:51:04.067859+0000","last_active":"2024-05-08T05:51:07.066301+0000","last_peered":"2024-05-08T05:51:07.066301+0000","last_clean":"2024-05-08T05:51:07.066301+0000","last_became_active":"2024-05-08T05:51:04.067586+0000","last_became_peered":"2024-05-08T05:51:04.067586+0000","last_unstale":"2024-05-08T05:51:07.066301+0000","last_undegraded":"2024-05-08T05:51:07.066301+0000","last_fullsized":"2024-05-08T05:51:07.066301+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:35:04.353051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.055148+0000","last_change":"2024-05-08T05:51:04.055148+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.055148+0000","last_undegraded":"2024-05-08T05:51:04.055148+0000","last_fullsized":"2024-05-08T05:51:04.055148+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.070847+0000","last_change":"2024-05-08T05:51:04.069859+0000","last_active":"2024-05-08T05:51:04.070847+0000","last_peered":"2024-05-08T05:51:04.070847+0000","last_clean":"2024-05-08T05:51:04.070847+0000","last_became_active":"2024-05-08T05:51:04.069575+0000","last_became_peered":"2024-05-08T05:51:04.069575+0000","last_unstale":"2024-05-08T05:51:04.070847+0000","last_undegraded":"2024-05-08T05:51:04.070847+0000","last_fullsized":"2024-05-08T05:51:04.070847+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:05:08.231174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.051301+0000","last_change":"2024-05-08T05:51:04.051301+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.051301+0000","last_undegraded":"2024-05-08T05:51:04.051301+0000","last_fullsized":"2024-05-08T05:51:04.051301+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.067379+0000","last_change":"2024-05-08T05:51:04.064523+0000","last_active":"2024-05-08T05:51:04.067379+0000","last_peered":"2024-05-08T05:51:04.067379+0000","last_clean":"2024-05-08T05:51:04.067379+0000","last_became_active":"2024-05-08T05:51:04.064236+0000","last_became_peered":"2024-05-08T05:51:04.064236+0000","last_unstale":"2024-05-08T05:51:04.067379+0000","last_undegraded":"2024-05-08T05:51:04.067379+0000","last_fullsized":"2024-05-08T05:51:04.067379+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:58:38.791116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093624+0000","last_change":"2024-05-08T05:51:05.090316+0000","last_active":"2024-05-08T05:51:05.093624+0000","last_peered":"2024-05-08T05:51:05.093624+0000","last_clean":"2024-05-08T05:51:05.093624+0000","last_became_active":"2024-05-08T05:51:05.084493+0000","last_became_peered":"2024-05-08T05:51:05.084493+0000","last_unstale":"2024-05-08T05:51:05.093624+0000","last_undegraded":"2024-05-08T05:51:05.093624+0000","last_fullsized":"2024-05-08T05:51:05.093624+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:16:24.027328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.066003+0000","last_change":"2024-05-08T05:51:04.059742+0000","last_active":"2024-05-08T05:51:04.066003+0000","last_peered":"2024-05-08T05:51:04.066003+0000","last_clean":"2024-05-08T05:51:04.066003+0000","last_became_active":"2024-05-08T05:51:04.059468+0000","last_became_peered":"2024-05-08T05:51:04.059468+0000","last_unstale":"2024-05-08T05:51:04.066003+0000","last_undegraded":"2024-05-08T05:51:04.066003+0000","last_fullsized":"2024-05-08T05:51:04.066003+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:35.193080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052749+0000","last_change":"2024-05-08T05:51:04.052749+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052749+0000","last_undegraded":"2024-05-08T05:51:04.052749+0000","last_fullsized":"2024-05-08T05:51:04.052749+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"118'6","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.101220+0000","last_change":"2024-05-08T05:51:04.066429+0000","last_active":"2024-05-08T05:51:05.101220+0000","last_peered":"2024-05-08T05:51:05.101220+0000","last_clean":"2024-05-08T05:51:05.101220+0000","last_became_active":"2024-05-08T05:51:04.065698+0000","last_became_peered":"2024-05-08T05:51:04.065698+0000","last_unstale":"2024-05-08T05:51:05.101220+0000","last_undegraded":"2024-05-08T05:51:05.101220+0000","last_fullsized":"2024-05-08T05:51:05.101220+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:28.960211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.054481+0000","last_change":"2024-05-08T05:51:04.054481+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.054481+0000","last_undegraded":"2024-05-08T05:51:04.054481+0000","last_fullsized":"2024-05-08T05:51:04.054481+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.094391+0000","last_change":"2024-05-08T05:51:04.067759+0000","last_active":"2024-05-08T05:51:05.094391+0000","last_peered":"2024-05-08T05:51:05.094391+0000","last_clean":"2024-05-08T05:51:05.094391+0000","last_became_active":"2024-05-08T05:51:04.067484+0000","last_became_peered":"2024-05-08T05:51:04.067484+0000","last_unstale":"2024-05-08T05:51:05.094391+0000","last_undegraded":"2024-05-08T05:51:05.094391+0000","last_fullsized":"2024-05-08T05:51:05.094391+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:44.431356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088282+0000","last_change":"2024-05-08T05:51:05.080720+0000","last_active":"2024-05-08T05:51:05.088282+0000","last_peered":"2024-05-08T05:51:05.088282+0000","last_clean":"2024-05-08T05:51:05.088282+0000","last_became_active":"2024-05-08T05:51:05.080428+0000","last_became_peered":"2024-05-08T05:51:05.080428+0000","last_unstale":"2024-05-08T05:51:05.088282+0000","last_undegraded":"2024-05-08T05:51:05.088282+0000","last_fullsized":"2024-05-08T05:51:05.088282+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:51:07.851943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.070039+0000","last_change":"2024-05-08T05:51:04.065712+0000","last_active":"2024-05-08T05:51:05.070039+0000","last_peered":"2024-05-08T05:51:05.070039+0000","last_clean":"2024-05-08T05:51:05.070039+0000","last_became_active":"2024-05-08T05:51:04.065421+0000","last_became_peered":"2024-05-08T05:51:04.065421+0000","last_unstale":"2024-05-08T05:51:05.070039+0000","last_undegraded":"2024-05-08T05:51:05.070039+0000","last_fullsized":"2024-05-08T05:51:05.070039+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:44:48.375896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078591+0000","last_change":"2024-05-08T05:51:05.084846+0000","last_active":"2024-05-08T05:51:06.078591+0000","last_peered":"2024-05-08T05:51:06.078591+0000","last_clean":"2024-05-08T05:51:06.078591+0000","last_became_active":"2024-05-08T05:51:05.084544+0000","last_became_peered":"2024-05-08T05:51:05.084544+0000","last_unstale":"2024-05-08T05:51:06.078591+0000","last_undegraded":"2024-05-08T05:51:06.078591+0000","last_fullsized":"2024-05-08T05:51:06.078591+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:30.552529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.068303+0000","last_change":"2024-05-08T05:51:04.067872+0000","last_active":"2024-05-08T05:51:05.068303+0000","last_peered":"2024-05-08T05:51:05.068303+0000","last_clean":"2024-05-08T05:51:05.068303+0000","last_became_active":"2024-05-08T05:51:04.067380+0000","last_became_peered":"2024-05-08T05:51:04.067380+0000","last_unstale":"2024-05-08T05:51:05.068303+0000","last_undegraded":"2024-05-08T05:51:05.068303+0000","last_fullsized":"2024-05-08T05:51:05.068303+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:21.223822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052885+0000","last_change":"2024-05-08T05:51:04.052885+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052885+0000","last_undegraded":"2024-05-08T05:51:04.052885+0000","last_fullsized":"2024-05-08T05:51:04.052885+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.063576+0000","last_change":"2024-05-08T05:51:04.070032+0000","last_active":"2024-05-08T05:51:05.063576+0000","last_peered":"2024-05-08T05:51:05.063576+0000","last_clean":"2024-05-08T05:51:05.063576+0000","last_became_active":"2024-05-08T05:51:04.069748+0000","last_became_peered":"2024-05-08T05:51:04.069748+0000","last_unstale":"2024-05-08T05:51:05.063576+0000","last_undegraded":"2024-05-08T05:51:05.063576+0000","last_fullsized":"2024-05-08T05:51:05.063576+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:46:01.704267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.084392+0000","last_change":"2024-05-08T05:51:05.078677+0000","last_active":"2024-05-08T05:51:05.084392+0000","last_peered":"2024-05-08T05:51:05.084392+0000","last_clean":"2024-05-08T05:51:05.084392+0000","last_became_active":"2024-05-08T05:51:05.078364+0000","last_became_peered":"2024-05-08T05:51:05.078364+0000","last_unstale":"2024-05-08T05:51:05.084392+0000","last_undegraded":"2024-05-08T05:51:05.084392+0000","last_fullsized":"2024-05-08T05:51:05.084392+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:00.252213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078677+0000","last_change":"2024-05-08T05:51:04.067612+0000","last_active":"2024-05-08T05:51:06.078677+0000","last_peered":"2024-05-08T05:51:06.078677+0000","last_clean":"2024-05-08T05:51:06.078677+0000","last_became_active":"2024-05-08T05:51:04.067268+0000","last_became_peered":"2024-05-08T05:51:04.067268+0000","last_unstale":"2024-05-08T05:51:06.078677+0000","last_undegraded":"2024-05-08T05:51:06.078677+0000","last_fullsized":"2024-05-08T05:51:06.078677+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:28:11.738033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066899+0000","last_change":"2024-05-08T05:51:05.073728+0000","last_active":"2024-05-08T05:51:07.066899+0000","last_peered":"2024-05-08T05:51:07.066899+0000","last_clean":"2024-05-08T05:51:07.066899+0000","last_became_active":"2024-05-08T05:51:05.073415+0000","last_became_peered":"2024-05-08T05:51:05.073415+0000","last_unstale":"2024-05-08T05:51:07.066899+0000","last_undegraded":"2024-05-08T05:51:07.066899+0000","last_fullsized":"2024-05-08T05:51:07.066899+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:52.612093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.069657+0000","last_change":"2024-05-08T05:51:04.066627+0000","last_active":"2024-05-08T05:51:04.069657+0000","last_peered":"2024-05-08T05:51:04.069657+0000","last_clean":"2024-05-08T05:51:04.069657+0000","last_became_active":"2024-05-08T05:51:04.066321+0000","last_became_peered":"2024-05-08T05:51:04.066321+0000","last_unstale":"2024-05-08T05:51:04.069657+0000","last_undegraded":"2024-05-08T05:51:04.069657+0000","last_fullsized":"2024-05-08T05:51:04.069657+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:54.640197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052213+0000","last_change":"2024-05-08T05:51:04.052213+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052213+0000","last_undegraded":"2024-05-08T05:51:04.052213+0000","last_fullsized":"2024-05-08T05:51:04.052213+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.072448+0000","last_change":"2024-05-08T05:51:04.072005+0000","last_active":"2024-05-08T05:51:04.072448+0000","last_peered":"2024-05-08T05:51:04.072448+0000","last_clean":"2024-05-08T05:51:04.072448+0000","last_became_active":"2024-05-08T05:51:04.071776+0000","last_became_peered":"2024-05-08T05:51:04.071776+0000","last_unstale":"2024-05-08T05:51:04.072448+0000","last_undegraded":"2024-05-08T05:51:04.072448+0000","last_fullsized":"2024-05-08T05:51:04.072448+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:21.223822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066166+0000","last_change":"2024-05-08T05:51:05.074599+0000","last_active":"2024-05-08T05:51:07.066166+0000","last_peered":"2024-05-08T05:51:07.066166+0000","last_clean":"2024-05-08T05:51:07.066166+0000","last_became_active":"2024-05-08T05:51:05.073942+0000","last_became_peered":"2024-05-08T05:51:05.073942+0000","last_unstale":"2024-05-08T05:51:07.066166+0000","last_undegraded":"2024-05-08T05:51:07.066166+0000","last_fullsized":"2024-05-08T05:51:07.066166+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:30.124962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066177+0000","last_change":"2024-05-08T05:51:04.069082+0000","last_active":"2024-05-08T05:51:07.066177+0000","last_peered":"2024-05-08T05:51:07.066177+0000","last_clean":"2024-05-08T05:51:07.066177+0000","last_became_active":"2024-05-08T05:51:04.068801+0000","last_became_peered":"2024-05-08T05:51:04.068801+0000","last_unstale":"2024-05-08T05:51:07.066177+0000","last_undegraded":"2024-05-08T05:51:07.066177+0000","last_fullsized":"2024-05-08T05:51:07.066177+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:57:45.457442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.092747+0000","last_change":"2024-05-08T05:51:05.083611+0000","last_active":"2024-05-08T05:51:05.092747+0000","last_peered":"2024-05-08T05:51:05.092747+0000","last_clean":"2024-05-08T05:51:05.092747+0000","last_became_active":"2024-05-08T05:51:05.083328+0000","last_became_peered":"2024-05-08T05:51:05.083328+0000","last_unstale":"2024-05-08T05:51:05.092747+0000","last_undegraded":"2024-05-08T05:51:05.092747+0000","last_fullsized":"2024-05-08T05:51:05.092747+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:55.131091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.065846+0000","last_change":"2024-05-08T05:51:04.063770+0000","last_active":"2024-05-08T05:51:04.065846+0000","last_peered":"2024-05-08T05:51:04.065846+0000","last_clean":"2024-05-08T05:51:04.065846+0000","last_became_active":"2024-05-08T05:51:04.063509+0000","last_became_peered":"2024-05-08T05:51:04.063509+0000","last_unstale":"2024-05-08T05:51:04.065846+0000","last_undegraded":"2024-05-08T05:51:04.065846+0000","last_fullsized":"2024-05-08T05:51:04.065846+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:49:24.745045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093539+0000","last_change":"2024-05-08T05:51:05.090152+0000","last_active":"2024-05-08T05:51:05.093539+0000","last_peered":"2024-05-08T05:51:05.093539+0000","last_clean":"2024-05-08T05:51:05.093539+0000","last_became_active":"2024-05-08T05:51:05.089828+0000","last_became_peered":"2024-05-08T05:51:05.089828+0000","last_unstale":"2024-05-08T05:51:05.093539+0000","last_undegraded":"2024-05-08T05:51:05.093539+0000","last_fullsized":"2024-05-08T05:51:05.093539+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:34:19.612717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.069013+0000","last_change":"2024-05-08T05:51:04.069833+0000","last_active":"2024-05-08T05:51:05.069013+0000","last_peered":"2024-05-08T05:51:05.069013+0000","last_clean":"2024-05-08T05:51:05.069013+0000","last_became_active":"2024-05-08T05:51:04.069539+0000","last_became_peered":"2024-05-08T05:51:04.069539+0000","last_unstale":"2024-05-08T05:51:05.069013+0000","last_undegraded":"2024-05-08T05:51:05.069013+0000","last_fullsized":"2024-05-08T05:51:05.069013+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:36:49.817010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052495+0000","last_change":"2024-05-08T05:51:04.052495+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052495+0000","last_undegraded":"2024-05-08T05:51:04.052495+0000","last_fullsized":"2024-05-08T05:51:04.052495+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.065891+0000","last_change":"2024-05-08T05:51:04.063987+0000","last_active":"2024-05-08T05:51:04.065891+0000","last_peered":"2024-05-08T05:51:04.065891+0000","last_clean":"2024-05-08T05:51:04.065891+0000","last_became_active":"2024-05-08T05:51:04.063646+0000","last_became_peered":"2024-05-08T05:51:04.063646+0000","last_unstale":"2024-05-08T05:51:04.065891+0000","last_undegraded":"2024-05-08T05:51:04.065891+0000","last_fullsized":"2024-05-08T05:51:04.065891+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:58:37.220465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066817+0000","last_change":"2024-05-08T05:51:05.075930+0000","last_active":"2024-05-08T05:51:07.066817+0000","last_peered":"2024-05-08T05:51:07.066817+0000","last_clean":"2024-05-08T05:51:07.066817+0000","last_became_active":"2024-05-08T05:51:05.075052+0000","last_became_peered":"2024-05-08T05:51:05.075052+0000","last_unstale":"2024-05-08T05:51:07.066817+0000","last_undegraded":"2024-05-08T05:51:07.066817+0000","last_fullsized":"2024-05-08T05:51:07.066817+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:27:28.653248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066775+0000","last_change":"2024-05-08T05:51:04.063919+0000","last_active":"2024-05-08T05:51:07.066775+0000","last_peered":"2024-05-08T05:51:07.066775+0000","last_clean":"2024-05-08T05:51:07.066775+0000","last_became_active":"2024-05-08T05:51:04.063610+0000","last_became_peered":"2024-05-08T05:51:04.063610+0000","last_unstale":"2024-05-08T05:51:07.066775+0000","last_undegraded":"2024-05-08T05:51:07.066775+0000","last_fullsized":"2024-05-08T05:51:07.066775+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:26.855446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.076159+0000","last_change":"2024-05-08T05:51:05.073177+0000","last_active":"2024-05-08T05:51:05.076159+0000","last_peered":"2024-05-08T05:51:05.076159+0000","last_clean":"2024-05-08T05:51:05.076159+0000","last_became_active":"2024-05-08T05:51:05.072104+0000","last_became_peered":"2024-05-08T05:51:05.072104+0000","last_unstale":"2024-05-08T05:51:05.076159+0000","last_undegraded":"2024-05-08T05:51:05.076159+0000","last_fullsized":"2024-05-08T05:51:05.076159+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:29.725291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.070325+0000","last_change":"2024-05-08T05:51:04.069690+0000","last_active":"2024-05-08T05:51:04.070325+0000","last_peered":"2024-05-08T05:51:04.070325+0000","last_clean":"2024-05-08T05:51:04.070325+0000","last_became_active":"2024-05-08T05:51:04.069380+0000","last_became_peered":"2024-05-08T05:51:04.069380+0000","last_unstale":"2024-05-08T05:51:04.070325+0000","last_undegraded":"2024-05-08T05:51:04.070325+0000","last_fullsized":"2024-05-08T05:51:04.070325+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:26.339116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.053043+0000","last_change":"2024-05-08T05:51:04.053043+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.053043+0000","last_undegraded":"2024-05-08T05:51:04.053043+0000","last_fullsized":"2024-05-08T05:51:04.053043+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.070852+0000","last_change":"2024-05-08T05:51:04.070027+0000","last_active":"2024-05-08T05:51:04.070852+0000","last_peered":"2024-05-08T05:51:04.070852+0000","last_clean":"2024-05-08T05:51:04.070852+0000","last_became_active":"2024-05-08T05:51:04.069331+0000","last_became_peered":"2024-05-08T05:51:04.069331+0000","last_unstale":"2024-05-08T05:51:04.070852+0000","last_undegraded":"2024-05-08T05:51:04.070852+0000","last_fullsized":"2024-05-08T05:51:04.070852+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:28.717301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.079017+0000","last_change":"2024-05-08T05:51:05.079710+0000","last_active":"2024-05-08T05:51:06.079017+0000","last_peered":"2024-05-08T05:51:06.079017+0000","last_clean":"2024-05-08T05:51:06.079017+0000","last_became_active":"2024-05-08T05:51:05.079405+0000","last_became_peered":"2024-05-08T05:51:05.079405+0000","last_unstale":"2024-05-08T05:51:06.079017+0000","last_undegraded":"2024-05-08T05:51:06.079017+0000","last_fullsized":"2024-05-08T05:51:06.079017+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:57:15.216780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066813+0000","last_change":"2024-05-08T05:51:04.066485+0000","last_active":"2024-05-08T05:51:07.066813+0000","last_peered":"2024-05-08T05:51:07.066813+0000","last_clean":"2024-05-08T05:51:07.066813+0000","last_became_active":"2024-05-08T05:51:04.066192+0000","last_became_peered":"2024-05-08T05:51:04.066192+0000","last_unstale":"2024-05-08T05:51:07.066813+0000","last_undegraded":"2024-05-08T05:51:07.066813+0000","last_fullsized":"2024-05-08T05:51:07.066813+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:54:27.233544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066370+0000","last_change":"2024-05-08T05:51:05.074917+0000","last_active":"2024-05-08T05:51:07.066370+0000","last_peered":"2024-05-08T05:51:07.066370+0000","last_clean":"2024-05-08T05:51:07.066370+0000","last_became_active":"2024-05-08T05:51:05.074339+0000","last_became_peered":"2024-05-08T05:51:05.074339+0000","last_unstale":"2024-05-08T05:51:07.066370+0000","last_undegraded":"2024-05-08T05:51:07.066370+0000","last_fullsized":"2024-05-08T05:51:07.066370+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:22:04.619439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.062382+0000","last_change":"2024-05-08T05:51:04.057289+0000","last_active":"2024-05-08T05:51:05.062382+0000","last_peered":"2024-05-08T05:51:05.062382+0000","last_clean":"2024-05-08T05:51:05.062382+0000","last_became_active":"2024-05-08T05:51:04.056950+0000","last_became_peered":"2024-05-08T05:51:04.056950+0000","last_unstale":"2024-05-08T05:51:05.062382+0000","last_undegraded":"2024-05-08T05:51:05.062382+0000","last_fullsized":"2024-05-08T05:51:05.062382+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:17.541863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.079392+0000","last_change":"2024-05-08T05:51:05.076808+0000","last_active":"2024-05-08T05:51:05.079392+0000","last_peered":"2024-05-08T05:51:05.079392+0000","last_clean":"2024-05-08T05:51:05.079392+0000","last_became_active":"2024-05-08T05:51:05.076532+0000","last_became_peered":"2024-05-08T05:51:05.076532+0000","last_unstale":"2024-05-08T05:51:05.079392+0000","last_undegraded":"2024-05-08T05:51:05.079392+0000","last_fullsized":"2024-05-08T05:51:05.079392+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:12.526046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.062570+0000","last_change":"2024-05-08T05:51:04.054314+0000","last_active":"2024-05-08T05:51:05.062570+0000","last_peered":"2024-05-08T05:51:05.062570+0000","last_clean":"2024-05-08T05:51:05.062570+0000","last_became_active":"2024-05-08T05:51:04.053754+0000","last_became_peered":"2024-05-08T05:51:04.053754+0000","last_unstale":"2024-05-08T05:51:05.062570+0000","last_undegraded":"2024-05-08T05:51:05.062570+0000","last_fullsized":"2024-05-08T05:51:05.062570+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:26.339116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.053473+0000","last_change":"2024-05-08T05:51:04.053473+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.053473+0000","last_undegraded":"2024-05-08T05:51:04.053473+0000","last_fullsized":"2024-05-08T05:51:04.053473+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066301+0000","last_change":"2024-05-08T05:51:04.064412+0000","last_active":"2024-05-08T05:51:07.066301+0000","last_peered":"2024-05-08T05:51:07.066301+0000","last_clean":"2024-05-08T05:51:07.066301+0000","last_became_active":"2024-05-08T05:51:04.064134+0000","last_became_peered":"2024-05-08T05:51:04.064134+0000","last_unstale":"2024-05-08T05:51:07.066301+0000","last_undegraded":"2024-05-08T05:51:07.066301+0000","last_fullsized":"2024-05-08T05:51:07.066301+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:37:09.043074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088681+0000","last_change":"2024-05-08T05:51:05.087096+0000","last_active":"2024-05-08T05:51:05.088681+0000","last_peered":"2024-05-08T05:51:05.088681+0000","last_clean":"2024-05-08T05:51:05.088681+0000","last_became_active":"2024-05-08T05:51:05.085170+0000","last_became_peered":"2024-05-08T05:51:05.085170+0000","last_unstale":"2024-05-08T05:51:05.088681+0000","last_undegraded":"2024-05-08T05:51:05.088681+0000","last_fullsized":"2024-05-08T05:51:05.088681+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:20.213009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.066642+0000","last_change":"2024-05-08T05:51:04.065454+0000","last_active":"2024-05-08T05:51:04.066642+0000","last_peered":"2024-05-08T05:51:04.066642+0000","last_clean":"2024-05-08T05:51:04.066642+0000","last_became_active":"2024-05-08T05:51:04.065175+0000","last_became_peered":"2024-05-08T05:51:04.065175+0000","last_unstale":"2024-05-08T05:51:04.066642+0000","last_undegraded":"2024-05-08T05:51:04.066642+0000","last_fullsized":"2024-05-08T05:51:04.066642+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:01:19.756722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.051048+0000","last_change":"2024-05-08T05:51:04.051048+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.051048+0000","last_undegraded":"2024-05-08T05:51:04.051048+0000","last_fullsized":"2024-05-08T05:51:04.051048+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"118'8","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.096717+0000","last_change":"2024-05-08T05:51:04.068279+0000","last_active":"2024-05-08T05:51:05.096717+0000","last_peered":"2024-05-08T05:51:05.096717+0000","last_clean":"2024-05-08T05:51:05.096717+0000","last_became_active":"2024-05-08T05:51:04.067953+0000","last_became_peered":"2024-05-08T05:51:04.067953+0000","last_unstale":"2024-05-08T05:51:05.096717+0000","last_undegraded":"2024-05-08T05:51:05.096717+0000","last_fullsized":"2024-05-08T05:51:05.096717+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:54.640197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":288,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.055816+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:51:04.055816+0000","last_peered":"2024-05-08T05:51:04.055816+0000","last_clean":"2024-05-08T05:51:04.055816+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:51:04.055816+0000","last_undegraded":"2024-05-08T05:51:04.055816+0000","last_fullsized":"2024-05-08T05:51:04.055816+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":1668,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":3302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542265,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2048000,"data_stored":1266440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542266,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1454080,"data_stored":676072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542265,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1454080,"data_stored":676072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":684851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1482752,"data_stored":684885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1478656,"data_stored":689931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1470464,"data_stored":683261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2076672,"data_stored":1280321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:51:09.487 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:51:09.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.627+0000 7f49e8504640 1 Processor -- start 2024-05-08T05:51:09.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.627+0000 7f49e8504640 1 -- start start 2024-05-08T05:51:09.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.628+0000 7f49e8504640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49e006e0f0 0x7f49e00cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:09.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.628+0000 7f49e8504640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f49e00d0360 0x7f49e00b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:09.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.628+0000 7f49e8504640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49e015d2a0 con 0x7f49e006f060 2024-05-08T05:51:09.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.628+0000 7f49e8504640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f49e006f440 con 0x7f49e00d0360 2024-05-08T05:51:09.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.628+0000 7f49e8504640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f49e01609b0 con 0x7f49e006e0f0 2024-05-08T05:51:09.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.628+0000 7f49e62a0640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f49e006f060 0x7f49e006fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52816/0 (socket says 172.21.15.45:52816) 2024-05-08T05:51:09.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.628+0000 7f49e62a0640 1 -- 172.21.15.45:0/1616945857 learned_addr learned my addr 172.21.15.45:0/1616945857 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:09.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.628+0000 7f49e5a9f640 1 --2- 172.21.15.45:0/1616945857 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49e006e0f0 0x7f49e00cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:09.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.628+0000 7f49e6aa1640 1 --2- 172.21.15.45:0/1616945857 >> v2:172.21.15.63:3300/0 conn(0x7f49e00d0360 0x7f49e00b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:09.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.628+0000 7f49e529e640 1 -- 172.21.15.45:0/1616945857 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 103576602 0 0) 0x7f49e015d2a0 con 0x7f49e006f060 2024-05-08T05:51:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.628+0000 7f49e529e640 1 -- 172.21.15.45:0/1616945857 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49cc003520 con 0x7f49e006f060 2024-05-08T05:51:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e529e640 1 -- 172.21.15.45:0/1616945857 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2784760026 0 0) 0x7f49cc003520 con 0x7f49e006f060 2024-05-08T05:51:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e529e640 1 -- 172.21.15.45:0/1616945857 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49e006e0f0 msgr2=0x7f49e00cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e529e640 1 --2- 172.21.15.45:0/1616945857 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49e006e0f0 0x7f49e00cfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e529e640 1 -- 172.21.15.45:0/1616945857 >> v2:172.21.15.63:3300/0 conn(0x7f49e00d0360 msgr2=0x7f49e00b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e529e640 1 --2- 172.21.15.45:0/1616945857 >> v2:172.21.15.63:3300/0 conn(0x7f49e00d0360 0x7f49e00b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e529e640 1 -- 172.21.15.45:0/1616945857 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e0160c90 con 0x7f49e006f060 2024-05-08T05:51:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e529e640 1 -- 172.21.15.45:0/1616945857 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f49d40016f0 con 0x7f49e006f060 2024-05-08T05:51:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e529e640 1 -- 172.21.15.45:0/1616945857 >> v1:172.21.15.45:6789/0 conn(0x7f49e006f060 legacy=0x7f49e006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e529e640 1 --2- 172.21.15.45:0/1616945857 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49e006e0f0 0x7f49cc003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e529e640 1 --2- 172.21.15.45:0/1616945857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49cc003fd0 0x7f49cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e529e640 1 --2- 172.21.15.45:0/1616945857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49cc004d40 0x7f49cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e529e640 1 -- 172.21.15.45:0/1616945857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f49e006f440 con 0x7f49cc004d40 2024-05-08T05:51:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e529e640 1 -- 172.21.15.45:0/1616945857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f49e01609b0 con 0x7f49cc003fd0 2024-05-08T05:51:09.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e529e640 1 -- 172.21.15.45:0/1616945857 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f49cc00ba10 con 0x7f49e006e0f0 2024-05-08T05:51:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e6aa1640 1 --2- 172.21.15.45:0/1616945857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49cc004d40 0x7f49cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e62a0640 1 --2- 172.21.15.45:0/1616945857 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49e006e0f0 0x7f49cc003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e5a9f640 1 --2- 172.21.15.45:0/1616945857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49cc003fd0 0x7f49cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e6aa1640 1 -- 172.21.15.45:0/1616945857 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49e006e0f0 msgr2=0x7f49cc003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e6aa1640 1 --2- 172.21.15.45:0/1616945857 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49e006e0f0 0x7f49cc003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e6aa1640 1 -- 172.21.15.45:0/1616945857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49cc003fd0 msgr2=0x7f49cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e6aa1640 1 --2- 172.21.15.45:0/1616945857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49cc003fd0 0x7f49cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.629+0000 7f49e6aa1640 1 -- 172.21.15.45:0/1616945857 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49cc003520 con 0x7f49cc004d40 2024-05-08T05:51:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.630+0000 7f49e6aa1640 1 --2- 172.21.15.45:0/1616945857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49cc004d40 0x7f49cc0051c0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f49d800e170 tx=0x7f49cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b3462f0cbd11fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.630+0000 7f49e529e640 1 -- 172.21.15.45:0/1616945857 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8034020 con 0x7f49cc004d40 2024-05-08T05:51:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.630+0000 7f49e529e640 1 -- 172.21.15.45:0/1616945857 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d8030430 con 0x7f49cc004d40 2024-05-08T05:51:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.630+0000 7f49e529e640 1 -- 172.21.15.45:0/1616945857 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8009c10 con 0x7f49cc004d40 2024-05-08T05:51:09.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.631+0000 7f49e8504640 1 -- 172.21.15.45:0/1616945857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49cc004d40 msgr2=0x7f49cc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:09.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.631+0000 7f49e8504640 1 --2- 172.21.15.45:0/1616945857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49cc004d40 0x7f49cc0051c0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f49d800e170 tx=0x7f49cc005830 comp rx=0 tx=0).stop 2024-05-08T05:51:09.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.631+0000 7f49e6aa1640 1 -- 172.21.15.45:0/1616945857 reap_dead start 2024-05-08T05:51:09.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.631+0000 7f49e8504640 1 -- 172.21.15.45:0/1616945857 shutdown_connections 2024-05-08T05:51:09.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.631+0000 7f49e8504640 1 -- 172.21.15.45:0/1616945857 >> 172.21.15.45:0/1616945857 conn(0x7f49e006c3a0 msgr2=0x7f49e006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:09.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.631+0000 7f49e8504640 1 -- 172.21.15.45:0/1616945857 shutdown_connections 2024-05-08T05:51:09.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.631+0000 7f49e8504640 1 -- 172.21.15.45:0/1616945857 wait complete. 2024-05-08T05:51:09.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.632+0000 7f49e8504640 1 Processor -- start 2024-05-08T05:51:09.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.632+0000 7f49e8504640 1 -- start start 2024-05-08T05:51:09.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.633+0000 7f49e8504640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49cc004d40 0x7f49e014f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:09.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.633+0000 7f49e8504640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e00cb6d0 0x7f49e00739e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:09.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.633+0000 7f49e8504640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49e015ee40 0x7f49e015f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:09.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.633+0000 7f49e8504640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f49e0161530 con 0x7f49e00cb6d0 2024-05-08T05:51:09.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.633+0000 7f49e8504640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f49e0161250 con 0x7f49cc004d40 2024-05-08T05:51:09.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.633+0000 7f49e8504640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f49e0161810 con 0x7f49e015ee40 2024-05-08T05:51:09.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.633+0000 7f49e5a9f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e00cb6d0 0x7f49e00739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:09.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.633+0000 7f49e5a9f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e00cb6d0 0x7f49e00739e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37354/0 (socket says 172.21.15.45:37354) 2024-05-08T05:51:09.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.633+0000 7f49e5a9f640 1 -- 172.21.15.45:0/1539267296 learned_addr learned my addr 172.21.15.45:0/1539267296 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.633+0000 7f49e62a0640 1 --2- 172.21.15.45:0/1539267296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49cc004d40 0x7f49e014f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.633+0000 7f49e6aa1640 1 --2- 172.21.15.45:0/1539267296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49e015ee40 0x7f49e015f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.633+0000 7f49e5a9f640 1 -- 172.21.15.45:0/1539267296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49e015ee40 msgr2=0x7f49e015f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.633+0000 7f49e5a9f640 1 --2- 172.21.15.45:0/1539267296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49e015ee40 0x7f49e015f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.633+0000 7f49e5a9f640 1 -- 172.21.15.45:0/1539267296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49cc004d40 msgr2=0x7f49e014f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.633+0000 7f49e5a9f640 1 --2- 172.21.15.45:0/1539267296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49cc004d40 0x7f49e014f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.633+0000 7f49e5a9f640 1 -- 172.21.15.45:0/1539267296 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e015b1b0 con 0x7f49e00cb6d0 2024-05-08T05:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.633+0000 7f49e5a9f640 1 --2- 172.21.15.45:0/1539267296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e00cb6d0 0x7f49e00739e0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0118e0 tx=0x7f49dc011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.633+0000 7f49d2ffd640 1 -- 172.21.15.45:0/1539267296 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc00f9d0 con 0x7f49e00cb6d0 2024-05-08T05:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.634+0000 7f49d2ffd640 1 -- 172.21.15.45:0/1539267296 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49dc00fb70 con 0x7f49e00cb6d0 2024-05-08T05:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.634+0000 7f49e8504640 1 -- 172.21.15.45:0/1539267296 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e015b490 con 0x7f49e00cb6d0 2024-05-08T05:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.634+0000 7f49d2ffd640 1 -- 172.21.15.45:0/1539267296 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc014440 con 0x7f49e00cb6d0 2024-05-08T05:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.634+0000 7f49e8504640 1 -- 172.21.15.45:0/1539267296 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e015ba50 con 0x7f49e00cb6d0 2024-05-08T05:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.635+0000 7f49e8504640 1 -- 172.21.15.45:0/1539267296 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49e015be30 con 0x7f49e00cb6d0 2024-05-08T05:51:09.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.635+0000 7f49d2ffd640 1 -- 172.21.15.45:0/1539267296 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f49dc012040 con 0x7f49e00cb6d0 2024-05-08T05:51:09.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.636+0000 7f49d2ffd640 1 --2- 172.21.15.45:0/1539267296 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f49c4076300 0x7f49c40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:09.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.636+0000 7f49d2ffd640 1 -- 172.21.15.45:0/1539267296 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f49dc097ba0 con 0x7f49e00cb6d0 2024-05-08T05:51:09.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.636+0000 7f49e62a0640 1 --2- 172.21.15.45:0/1539267296 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f49c4076300 0x7f49c40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:09.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.636+0000 7f49e62a0640 1 --2- 172.21.15.45:0/1539267296 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f49c4076300 0x7f49c40787c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f49d4001bb0 tx=0x7f49d4001b40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:09.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.640+0000 7f49d2ffd640 1 -- 172.21.15.45:0/1539267296 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f49e015be30 con 0x7f49e00cb6d0 2024-05-08T05:51:09.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.829+0000 7f49e8504640 1 -- 172.21.15.45:0/1539267296 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f49e00693f0 con 0x7f49c4076300 2024-05-08T05:51:09.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.836+0000 7f49d2ffd640 1 -- 172.21.15.45:0/1539267296 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250600 (secure 0 0 0) 0x7f49e00693f0 con 0x7f49c4076300 2024-05-08T05:51:09.839 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:09.840 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:51:09.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.839+0000 7f49e8504640 1 -- 172.21.15.45:0/1539267296 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f49c4076300 msgr2=0x7f49c40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:09.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.839+0000 7f49e8504640 1 --2- 172.21.15.45:0/1539267296 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f49c4076300 0x7f49c40787c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f49d4001bb0 tx=0x7f49d4001b40 comp rx=0 tx=0).stop 2024-05-08T05:51:09.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.839+0000 7f49e8504640 1 -- 172.21.15.45:0/1539267296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e00cb6d0 msgr2=0x7f49e00739e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:09.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.839+0000 7f49e8504640 1 --2- 172.21.15.45:0/1539267296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e00cb6d0 0x7f49e00739e0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0118e0 tx=0x7f49dc011da0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.839+0000 7f49e8504640 1 -- 172.21.15.45:0/1539267296 shutdown_connections 2024-05-08T05:51:09.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.839+0000 7f49e8504640 1 --2- 172.21.15.45:0/1539267296 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f49c4076300 0x7f49c40787c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.839+0000 7f49e8504640 1 --2- 172.21.15.45:0/1539267296 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f49e015ee40 0x7f49e015f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.839+0000 7f49e8504640 1 --2- 172.21.15.45:0/1539267296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e00cb6d0 0x7f49e00739e0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.839+0000 7f49e8504640 1 --2- 172.21.15.45:0/1539267296 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49cc004d40 0x7f49e014f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.839+0000 7f49e8504640 1 -- 172.21.15.45:0/1539267296 >> 172.21.15.45:0/1539267296 conn(0x7f49e006c3a0 msgr2=0x7f49e00cec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:09.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.839+0000 7f49e8504640 1 -- 172.21.15.45:0/1539267296 shutdown_connections 2024-05-08T05:51:09.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.839+0000 7f49e8504640 1 -- 172.21.15.45:0/1539267296 wait complete. 2024-05-08T05:51:09.851 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2024-05-08T05:51:08.974825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592036,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":133,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262108,"kb_used_data":12640,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712612,"statfs":{"total":773094113280,"available":772825714688,"internally_reserved":0,"allocated":12943360,"data_stored":6641833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1668,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.928070"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.073391+0000","last_change":"2024-05-08T05:51:05.069476+0000","last_active":"2024-05-08T05:51:05.073391+0000","last_peered":"2024-05-08T05:51:05.073391+0000","last_clean":"2024-05-08T05:51:05.073391+0000","last_became_active":"2024-05-08T05:51:05.069131+0000","last_became_peered":"2024-05-08T05:51:05.069131+0000","last_unstale":"2024-05-08T05:51:05.073391+0000","last_undegraded":"2024-05-08T05:51:05.073391+0000","last_fullsized":"2024-05-08T05:51:05.073391+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:05:09.239164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093231+0000","last_change":"2024-05-08T05:51:05.084139+0000","last_active":"2024-05-08T05:51:05.093231+0000","last_peered":"2024-05-08T05:51:05.093231+0000","last_clean":"2024-05-08T05:51:05.093231+0000","last_became_active":"2024-05-08T05:51:05.081830+0000","last_became_peered":"2024-05-08T05:51:05.081830+0000","last_unstale":"2024-05-08T05:51:05.093231+0000","last_undegraded":"2024-05-08T05:51:05.093231+0000","last_fullsized":"2024-05-08T05:51:05.093231+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:37.715909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093594+0000","last_change":"2024-05-08T05:51:05.090876+0000","last_active":"2024-05-08T05:51:05.093594+0000","last_peered":"2024-05-08T05:51:05.093594+0000","last_clean":"2024-05-08T05:51:05.093594+0000","last_became_active":"2024-05-08T05:51:05.089329+0000","last_became_peered":"2024-05-08T05:51:05.089329+0000","last_unstale":"2024-05-08T05:51:05.093594+0000","last_undegraded":"2024-05-08T05:51:05.093594+0000","last_fullsized":"2024-05-08T05:51:05.093594+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:51:43.350274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052577+0000","last_change":"2024-05-08T05:51:04.052577+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052577+0000","last_undegraded":"2024-05-08T05:51:04.052577+0000","last_fullsized":"2024-05-08T05:51:04.052577+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.077282+0000","last_change":"2024-05-08T05:51:05.072296+0000","last_active":"2024-05-08T05:51:05.077282+0000","last_peered":"2024-05-08T05:51:05.077282+0000","last_clean":"2024-05-08T05:51:05.077282+0000","last_became_active":"2024-05-08T05:51:05.072009+0000","last_became_peered":"2024-05-08T05:51:05.072009+0000","last_unstale":"2024-05-08T05:51:05.077282+0000","last_undegraded":"2024-05-08T05:51:05.077282+0000","last_fullsized":"2024-05-08T05:51:05.077282+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:12:02.641413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066407+0000","last_change":"2024-05-08T05:51:05.082847+0000","last_active":"2024-05-08T05:51:07.066407+0000","last_peered":"2024-05-08T05:51:07.066407+0000","last_clean":"2024-05-08T05:51:07.066407+0000","last_became_active":"2024-05-08T05:51:05.082267+0000","last_became_peered":"2024-05-08T05:51:05.082267+0000","last_unstale":"2024-05-08T05:51:07.066407+0000","last_undegraded":"2024-05-08T05:51:07.066407+0000","last_fullsized":"2024-05-08T05:51:07.066407+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:16:17.521040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066220+0000","last_change":"2024-05-08T05:51:05.087173+0000","last_active":"2024-05-08T05:51:07.066220+0000","last_peered":"2024-05-08T05:51:07.066220+0000","last_clean":"2024-05-08T05:51:07.066220+0000","last_became_active":"2024-05-08T05:51:05.086914+0000","last_became_peered":"2024-05-08T05:51:05.086914+0000","last_unstale":"2024-05-08T05:51:07.066220+0000","last_undegraded":"2024-05-08T05:51:07.066220+0000","last_fullsized":"2024-05-08T05:51:07.066220+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:42:24.823847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093326+0000","last_change":"2024-05-08T05:51:05.085921+0000","last_active":"2024-05-08T05:51:05.093326+0000","last_peered":"2024-05-08T05:51:05.093326+0000","last_clean":"2024-05-08T05:51:05.093326+0000","last_became_active":"2024-05-08T05:51:05.083800+0000","last_became_peered":"2024-05-08T05:51:05.083800+0000","last_unstale":"2024-05-08T05:51:05.093326+0000","last_undegraded":"2024-05-08T05:51:05.093326+0000","last_fullsized":"2024-05-08T05:51:05.093326+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:47.453010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.053739+0000","last_change":"2024-05-08T05:51:04.053739+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.053739+0000","last_undegraded":"2024-05-08T05:51:04.053739+0000","last_fullsized":"2024-05-08T05:51:04.053739+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.075989+0000","last_change":"2024-05-08T05:51:05.071111+0000","last_active":"2024-05-08T05:51:05.075989+0000","last_peered":"2024-05-08T05:51:05.075989+0000","last_clean":"2024-05-08T05:51:05.075989+0000","last_became_active":"2024-05-08T05:51:05.070823+0000","last_became_peered":"2024-05-08T05:51:05.070823+0000","last_unstale":"2024-05-08T05:51:05.075989+0000","last_undegraded":"2024-05-08T05:51:05.075989+0000","last_fullsized":"2024-05-08T05:51:05.075989+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:18.549853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.073515+0000","last_change":"2024-05-08T05:51:05.070718+0000","last_active":"2024-05-08T05:51:05.073515+0000","last_peered":"2024-05-08T05:51:05.073515+0000","last_clean":"2024-05-08T05:51:05.073515+0000","last_became_active":"2024-05-08T05:51:05.070422+0000","last_became_peered":"2024-05-08T05:51:05.070422+0000","last_unstale":"2024-05-08T05:51:05.073515+0000","last_undegraded":"2024-05-08T05:51:05.073515+0000","last_fullsized":"2024-05-08T05:51:05.073515+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:29.725291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066591+0000","last_change":"2024-05-08T05:51:05.076336+0000","last_active":"2024-05-08T05:51:07.066591+0000","last_peered":"2024-05-08T05:51:07.066591+0000","last_clean":"2024-05-08T05:51:07.066591+0000","last_became_active":"2024-05-08T05:51:05.075648+0000","last_became_peered":"2024-05-08T05:51:05.075648+0000","last_unstale":"2024-05-08T05:51:07.066591+0000","last_undegraded":"2024-05-08T05:51:07.066591+0000","last_fullsized":"2024-05-08T05:51:07.066591+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:27.438502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088120+0000","last_change":"2024-05-08T05:51:05.079977+0000","last_active":"2024-05-08T05:51:05.088120+0000","last_peered":"2024-05-08T05:51:05.088120+0000","last_clean":"2024-05-08T05:51:05.088120+0000","last_became_active":"2024-05-08T05:51:05.078938+0000","last_became_peered":"2024-05-08T05:51:05.078938+0000","last_unstale":"2024-05-08T05:51:05.088120+0000","last_undegraded":"2024-05-08T05:51:05.088120+0000","last_fullsized":"2024-05-08T05:51:05.088120+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:00.252213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093291+0000","last_change":"2024-05-08T05:51:05.086072+0000","last_active":"2024-05-08T05:51:05.093291+0000","last_peered":"2024-05-08T05:51:05.093291+0000","last_clean":"2024-05-08T05:51:05.093291+0000","last_became_active":"2024-05-08T05:51:05.084475+0000","last_became_peered":"2024-05-08T05:51:05.084475+0000","last_unstale":"2024-05-08T05:51:05.093291+0000","last_undegraded":"2024-05-08T05:51:05.093291+0000","last_fullsized":"2024-05-08T05:51:05.093291+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:08:48.269564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088031+0000","last_change":"2024-05-08T05:51:05.083505+0000","last_active":"2024-05-08T05:51:05.088031+0000","last_peered":"2024-05-08T05:51:05.088031+0000","last_clean":"2024-05-08T05:51:05.088031+0000","last_became_active":"2024-05-08T05:51:05.083236+0000","last_became_peered":"2024-05-08T05:51:05.083236+0000","last_unstale":"2024-05-08T05:51:05.088031+0000","last_undegraded":"2024-05-08T05:51:05.088031+0000","last_fullsized":"2024-05-08T05:51:05.088031+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:20.213009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066209+0000","last_change":"2024-05-08T05:51:05.077087+0000","last_active":"2024-05-08T05:51:07.066209+0000","last_peered":"2024-05-08T05:51:07.066209+0000","last_clean":"2024-05-08T05:51:07.066209+0000","last_became_active":"2024-05-08T05:51:05.076743+0000","last_became_peered":"2024-05-08T05:51:05.076743+0000","last_unstale":"2024-05-08T05:51:07.066209+0000","last_undegraded":"2024-05-08T05:51:07.066209+0000","last_fullsized":"2024-05-08T05:51:07.066209+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:03.278751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088248+0000","last_change":"2024-05-08T05:51:05.081532+0000","last_active":"2024-05-08T05:51:05.088248+0000","last_peered":"2024-05-08T05:51:05.088248+0000","last_clean":"2024-05-08T05:51:05.088248+0000","last_became_active":"2024-05-08T05:51:05.081228+0000","last_became_peered":"2024-05-08T05:51:05.081228+0000","last_unstale":"2024-05-08T05:51:05.088248+0000","last_undegraded":"2024-05-08T05:51:05.088248+0000","last_fullsized":"2024-05-08T05:51:05.088248+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:38:53.162197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093376+0000","last_change":"2024-05-08T05:51:05.086943+0000","last_active":"2024-05-08T05:51:05.093376+0000","last_peered":"2024-05-08T05:51:05.093376+0000","last_clean":"2024-05-08T05:51:05.093376+0000","last_became_active":"2024-05-08T05:51:05.086221+0000","last_became_peered":"2024-05-08T05:51:05.086221+0000","last_unstale":"2024-05-08T05:51:05.093376+0000","last_undegraded":"2024-05-08T05:51:05.093376+0000","last_fullsized":"2024-05-08T05:51:05.093376+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:34:12.399560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.056046+0000","last_change":"2024-05-08T05:51:04.056046+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.056046+0000","last_undegraded":"2024-05-08T05:51:04.056046+0000","last_fullsized":"2024-05-08T05:51:04.056046+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078597+0000","last_change":"2024-05-08T05:51:05.080761+0000","last_active":"2024-05-08T05:51:06.078597+0000","last_peered":"2024-05-08T05:51:06.078597+0000","last_clean":"2024-05-08T05:51:06.078597+0000","last_became_active":"2024-05-08T05:51:05.080381+0000","last_became_peered":"2024-05-08T05:51:05.080381+0000","last_unstale":"2024-05-08T05:51:06.078597+0000","last_undegraded":"2024-05-08T05:51:06.078597+0000","last_fullsized":"2024-05-08T05:51:06.078597+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:42.792089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.051557+0000","last_change":"2024-05-08T05:51:04.051557+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.051557+0000","last_undegraded":"2024-05-08T05:51:04.051557+0000","last_fullsized":"2024-05-08T05:51:04.051557+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.084370+0000","last_change":"2024-05-08T05:51:05.079566+0000","last_active":"2024-05-08T05:51:05.084370+0000","last_peered":"2024-05-08T05:51:05.084370+0000","last_clean":"2024-05-08T05:51:05.084370+0000","last_became_active":"2024-05-08T05:51:05.079256+0000","last_became_peered":"2024-05-08T05:51:05.079256+0000","last_unstale":"2024-05-08T05:51:05.084370+0000","last_undegraded":"2024-05-08T05:51:05.084370+0000","last_fullsized":"2024-05-08T05:51:05.084370+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:51:07.851943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.050947+0000","last_change":"2024-05-08T05:51:04.050947+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.050947+0000","last_undegraded":"2024-05-08T05:51:04.050947+0000","last_fullsized":"2024-05-08T05:51:04.050947+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052838+0000","last_change":"2024-05-08T05:51:04.052838+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052838+0000","last_undegraded":"2024-05-08T05:51:04.052838+0000","last_fullsized":"2024-05-08T05:51:04.052838+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.053724+0000","last_change":"2024-05-08T05:51:04.053724+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.053724+0000","last_undegraded":"2024-05-08T05:51:04.053724+0000","last_fullsized":"2024-05-08T05:51:04.053724+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078673+0000","last_change":"2024-05-08T05:51:05.081486+0000","last_active":"2024-05-08T05:51:06.078673+0000","last_peered":"2024-05-08T05:51:06.078673+0000","last_clean":"2024-05-08T05:51:06.078673+0000","last_became_active":"2024-05-08T05:51:05.081143+0000","last_became_peered":"2024-05-08T05:51:05.081143+0000","last_unstale":"2024-05-08T05:51:06.078673+0000","last_undegraded":"2024-05-08T05:51:06.078673+0000","last_fullsized":"2024-05-08T05:51:06.078673+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:44:56.396392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.055539+0000","last_change":"2024-05-08T05:51:04.055539+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.055539+0000","last_undegraded":"2024-05-08T05:51:04.055539+0000","last_fullsized":"2024-05-08T05:51:04.055539+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066746+0000","last_change":"2024-05-08T05:51:05.081306+0000","last_active":"2024-05-08T05:51:07.066746+0000","last_peered":"2024-05-08T05:51:07.066746+0000","last_clean":"2024-05-08T05:51:07.066746+0000","last_became_active":"2024-05-08T05:51:05.081023+0000","last_became_peered":"2024-05-08T05:51:05.081023+0000","last_unstale":"2024-05-08T05:51:07.066746+0000","last_undegraded":"2024-05-08T05:51:07.066746+0000","last_fullsized":"2024-05-08T05:51:07.066746+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:20:04.797865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078729+0000","last_change":"2024-05-08T05:51:05.082212+0000","last_active":"2024-05-08T05:51:06.078729+0000","last_peered":"2024-05-08T05:51:06.078729+0000","last_clean":"2024-05-08T05:51:06.078729+0000","last_became_active":"2024-05-08T05:51:05.081847+0000","last_became_peered":"2024-05-08T05:51:05.081847+0000","last_unstale":"2024-05-08T05:51:06.078729+0000","last_undegraded":"2024-05-08T05:51:06.078729+0000","last_fullsized":"2024-05-08T05:51:06.078729+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:10.568249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093267+0000","last_change":"2024-05-08T05:51:05.085821+0000","last_active":"2024-05-08T05:51:05.093267+0000","last_peered":"2024-05-08T05:51:05.093267+0000","last_clean":"2024-05-08T05:51:05.093267+0000","last_became_active":"2024-05-08T05:51:05.083777+0000","last_became_peered":"2024-05-08T05:51:05.083777+0000","last_unstale":"2024-05-08T05:51:05.093267+0000","last_undegraded":"2024-05-08T05:51:05.093267+0000","last_fullsized":"2024-05-08T05:51:05.093267+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:22:46.959500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.051392+0000","last_change":"2024-05-08T05:51:04.051392+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.051392+0000","last_undegraded":"2024-05-08T05:51:04.051392+0000","last_fullsized":"2024-05-08T05:51:04.051392+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.050670+0000","last_change":"2024-05-08T05:51:04.050670+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.050670+0000","last_undegraded":"2024-05-08T05:51:04.050670+0000","last_fullsized":"2024-05-08T05:51:04.050670+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.076195+0000","last_change":"2024-05-08T05:51:05.072621+0000","last_active":"2024-05-08T05:51:05.076195+0000","last_peered":"2024-05-08T05:51:05.076195+0000","last_clean":"2024-05-08T05:51:05.076195+0000","last_became_active":"2024-05-08T05:51:05.072317+0000","last_became_peered":"2024-05-08T05:51:05.072317+0000","last_unstale":"2024-05-08T05:51:05.076195+0000","last_undegraded":"2024-05-08T05:51:05.076195+0000","last_fullsized":"2024-05-08T05:51:05.076195+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:05:09.239164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.064918+0000","last_change":"2024-05-08T05:51:04.063764+0000","last_active":"2024-05-08T05:51:04.064918+0000","last_peered":"2024-05-08T05:51:04.064918+0000","last_clean":"2024-05-08T05:51:04.064918+0000","last_became_active":"2024-05-08T05:51:04.063484+0000","last_became_peered":"2024-05-08T05:51:04.063484+0000","last_unstale":"2024-05-08T05:51:04.064918+0000","last_undegraded":"2024-05-08T05:51:04.064918+0000","last_fullsized":"2024-05-08T05:51:04.064918+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:17.541863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066655+0000","last_change":"2024-05-08T05:51:05.084520+0000","last_active":"2024-05-08T05:51:07.066655+0000","last_peered":"2024-05-08T05:51:07.066655+0000","last_clean":"2024-05-08T05:51:07.066655+0000","last_became_active":"2024-05-08T05:51:05.084198+0000","last_became_peered":"2024-05-08T05:51:05.084198+0000","last_unstale":"2024-05-08T05:51:07.066655+0000","last_undegraded":"2024-05-08T05:51:07.066655+0000","last_fullsized":"2024-05-08T05:51:07.066655+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:08:36.961131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.063174+0000","last_change":"2024-05-08T05:51:04.057409+0000","last_active":"2024-05-08T05:51:05.063174+0000","last_peered":"2024-05-08T05:51:05.063174+0000","last_clean":"2024-05-08T05:51:05.063174+0000","last_became_active":"2024-05-08T05:51:04.057127+0000","last_became_peered":"2024-05-08T05:51:04.057127+0000","last_unstale":"2024-05-08T05:51:05.063174+0000","last_undegraded":"2024-05-08T05:51:05.063174+0000","last_fullsized":"2024-05-08T05:51:05.063174+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:33.984066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052424+0000","last_change":"2024-05-08T05:51:04.052424+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052424+0000","last_undegraded":"2024-05-08T05:51:04.052424+0000","last_fullsized":"2024-05-08T05:51:04.052424+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.065887+0000","last_change":"2024-05-08T05:51:04.063770+0000","last_active":"2024-05-08T05:51:04.065887+0000","last_peered":"2024-05-08T05:51:04.065887+0000","last_clean":"2024-05-08T05:51:04.065887+0000","last_became_active":"2024-05-08T05:51:04.063471+0000","last_became_peered":"2024-05-08T05:51:04.063471+0000","last_unstale":"2024-05-08T05:51:04.065887+0000","last_undegraded":"2024-05-08T05:51:04.065887+0000","last_fullsized":"2024-05-08T05:51:04.065887+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:14:14.017450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.053316+0000","last_change":"2024-05-08T05:51:04.053316+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.053316+0000","last_undegraded":"2024-05-08T05:51:04.053316+0000","last_fullsized":"2024-05-08T05:51:04.053316+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.095608+0000","last_change":"2024-05-08T05:51:04.060756+0000","last_active":"2024-05-08T05:51:05.095608+0000","last_peered":"2024-05-08T05:51:05.095608+0000","last_clean":"2024-05-08T05:51:05.095608+0000","last_became_active":"2024-05-08T05:51:04.060480+0000","last_became_peered":"2024-05-08T05:51:04.060480+0000","last_unstale":"2024-05-08T05:51:05.095608+0000","last_undegraded":"2024-05-08T05:51:05.095608+0000","last_fullsized":"2024-05-08T05:51:05.095608+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:58:38.791116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.076073+0000","last_change":"2024-05-08T05:51:05.072513+0000","last_active":"2024-05-08T05:51:05.076073+0000","last_peered":"2024-05-08T05:51:05.076073+0000","last_clean":"2024-05-08T05:51:05.076073+0000","last_became_active":"2024-05-08T05:51:05.072191+0000","last_became_peered":"2024-05-08T05:51:05.072191+0000","last_unstale":"2024-05-08T05:51:05.076073+0000","last_undegraded":"2024-05-08T05:51:05.076073+0000","last_fullsized":"2024-05-08T05:51:05.076073+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:27.347106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.062425+0000","last_change":"2024-05-08T05:51:04.058250+0000","last_active":"2024-05-08T05:51:05.062425+0000","last_peered":"2024-05-08T05:51:05.062425+0000","last_clean":"2024-05-08T05:51:05.062425+0000","last_became_active":"2024-05-08T05:51:04.057765+0000","last_became_peered":"2024-05-08T05:51:04.057765+0000","last_unstale":"2024-05-08T05:51:05.062425+0000","last_undegraded":"2024-05-08T05:51:05.062425+0000","last_fullsized":"2024-05-08T05:51:05.062425+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:46:01.704267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.084362+0000","last_change":"2024-05-08T05:51:05.079919+0000","last_active":"2024-05-08T05:51:05.084362+0000","last_peered":"2024-05-08T05:51:05.084362+0000","last_clean":"2024-05-08T05:51:05.084362+0000","last_became_active":"2024-05-08T05:51:05.079552+0000","last_became_peered":"2024-05-08T05:51:05.079552+0000","last_unstale":"2024-05-08T05:51:05.084362+0000","last_undegraded":"2024-05-08T05:51:05.084362+0000","last_fullsized":"2024-05-08T05:51:05.084362+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:38:53.162197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066334+0000","last_change":"2024-05-08T05:51:04.069265+0000","last_active":"2024-05-08T05:51:07.066334+0000","last_peered":"2024-05-08T05:51:07.066334+0000","last_clean":"2024-05-08T05:51:07.066334+0000","last_became_active":"2024-05-08T05:51:04.068997+0000","last_became_peered":"2024-05-08T05:51:04.068997+0000","last_unstale":"2024-05-08T05:51:07.066334+0000","last_undegraded":"2024-05-08T05:51:07.066334+0000","last_fullsized":"2024-05-08T05:51:07.066334+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:55.959876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.084003+0000","last_change":"2024-05-08T05:51:05.079354+0000","last_active":"2024-05-08T05:51:05.084003+0000","last_peered":"2024-05-08T05:51:05.084003+0000","last_clean":"2024-05-08T05:51:05.084003+0000","last_became_active":"2024-05-08T05:51:05.079042+0000","last_became_peered":"2024-05-08T05:51:05.079042+0000","last_unstale":"2024-05-08T05:51:05.084003+0000","last_undegraded":"2024-05-08T05:51:05.084003+0000","last_fullsized":"2024-05-08T05:51:05.084003+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:20:04.111530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.065847+0000","last_change":"2024-05-08T05:51:04.058859+0000","last_active":"2024-05-08T05:51:04.065847+0000","last_peered":"2024-05-08T05:51:04.065847+0000","last_clean":"2024-05-08T05:51:04.065847+0000","last_became_active":"2024-05-08T05:51:04.058518+0000","last_became_peered":"2024-05-08T05:51:04.058518+0000","last_unstale":"2024-05-08T05:51:04.065847+0000","last_undegraded":"2024-05-08T05:51:04.065847+0000","last_fullsized":"2024-05-08T05:51:04.065847+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:02:28.473412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.087245+0000","last_change":"2024-05-08T05:51:05.077404+0000","last_active":"2024-05-08T05:51:05.087245+0000","last_peered":"2024-05-08T05:51:05.087245+0000","last_clean":"2024-05-08T05:51:05.087245+0000","last_became_active":"2024-05-08T05:51:05.077093+0000","last_became_peered":"2024-05-08T05:51:05.077093+0000","last_unstale":"2024-05-08T05:51:05.087245+0000","last_undegraded":"2024-05-08T05:51:05.087245+0000","last_fullsized":"2024-05-08T05:51:05.087245+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:12:02.641413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066301+0000","last_change":"2024-05-08T05:51:04.067859+0000","last_active":"2024-05-08T05:51:07.066301+0000","last_peered":"2024-05-08T05:51:07.066301+0000","last_clean":"2024-05-08T05:51:07.066301+0000","last_became_active":"2024-05-08T05:51:04.067586+0000","last_became_peered":"2024-05-08T05:51:04.067586+0000","last_unstale":"2024-05-08T05:51:07.066301+0000","last_undegraded":"2024-05-08T05:51:07.066301+0000","last_fullsized":"2024-05-08T05:51:07.066301+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:35:04.353051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.055148+0000","last_change":"2024-05-08T05:51:04.055148+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.055148+0000","last_undegraded":"2024-05-08T05:51:04.055148+0000","last_fullsized":"2024-05-08T05:51:04.055148+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.070847+0000","last_change":"2024-05-08T05:51:04.069859+0000","last_active":"2024-05-08T05:51:04.070847+0000","last_peered":"2024-05-08T05:51:04.070847+0000","last_clean":"2024-05-08T05:51:04.070847+0000","last_became_active":"2024-05-08T05:51:04.069575+0000","last_became_peered":"2024-05-08T05:51:04.069575+0000","last_unstale":"2024-05-08T05:51:04.070847+0000","last_undegraded":"2024-05-08T05:51:04.070847+0000","last_fullsized":"2024-05-08T05:51:04.070847+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:05:08.231174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.051301+0000","last_change":"2024-05-08T05:51:04.051301+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.051301+0000","last_undegraded":"2024-05-08T05:51:04.051301+0000","last_fullsized":"2024-05-08T05:51:04.051301+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.067379+0000","last_change":"2024-05-08T05:51:04.064523+0000","last_active":"2024-05-08T05:51:04.067379+0000","last_peered":"2024-05-08T05:51:04.067379+0000","last_clean":"2024-05-08T05:51:04.067379+0000","last_became_active":"2024-05-08T05:51:04.064236+0000","last_became_peered":"2024-05-08T05:51:04.064236+0000","last_unstale":"2024-05-08T05:51:04.067379+0000","last_undegraded":"2024-05-08T05:51:04.067379+0000","last_fullsized":"2024-05-08T05:51:04.067379+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:58:38.791116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093624+0000","last_change":"2024-05-08T05:51:05.090316+0000","last_active":"2024-05-08T05:51:05.093624+0000","last_peered":"2024-05-08T05:51:05.093624+0000","last_clean":"2024-05-08T05:51:05.093624+0000","last_became_active":"2024-05-08T05:51:05.084493+0000","last_became_peered":"2024-05-08T05:51:05.084493+0000","last_unstale":"2024-05-08T05:51:05.093624+0000","last_undegraded":"2024-05-08T05:51:05.093624+0000","last_fullsized":"2024-05-08T05:51:05.093624+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:16:24.027328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.066003+0000","last_change":"2024-05-08T05:51:04.059742+0000","last_active":"2024-05-08T05:51:04.066003+0000","last_peered":"2024-05-08T05:51:04.066003+0000","last_clean":"2024-05-08T05:51:04.066003+0000","last_became_active":"2024-05-08T05:51:04.059468+0000","last_became_peered":"2024-05-08T05:51:04.059468+0000","last_unstale":"2024-05-08T05:51:04.066003+0000","last_undegraded":"2024-05-08T05:51:04.066003+0000","last_fullsized":"2024-05-08T05:51:04.066003+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:35.193080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052749+0000","last_change":"2024-05-08T05:51:04.052749+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052749+0000","last_undegraded":"2024-05-08T05:51:04.052749+0000","last_fullsized":"2024-05-08T05:51:04.052749+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"118'6","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.101220+0000","last_change":"2024-05-08T05:51:04.066429+0000","last_active":"2024-05-08T05:51:05.101220+0000","last_peered":"2024-05-08T05:51:05.101220+0000","last_clean":"2024-05-08T05:51:05.101220+0000","last_became_active":"2024-05-08T05:51:04.065698+0000","last_became_peered":"2024-05-08T05:51:04.065698+0000","last_unstale":"2024-05-08T05:51:05.101220+0000","last_undegraded":"2024-05-08T05:51:05.101220+0000","last_fullsized":"2024-05-08T05:51:05.101220+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:28.960211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.054481+0000","last_change":"2024-05-08T05:51:04.054481+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.054481+0000","last_undegraded":"2024-05-08T05:51:04.054481+0000","last_fullsized":"2024-05-08T05:51:04.054481+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.094391+0000","last_change":"2024-05-08T05:51:04.067759+0000","last_active":"2024-05-08T05:51:05.094391+0000","last_peered":"2024-05-08T05:51:05.094391+0000","last_clean":"2024-05-08T05:51:05.094391+0000","last_became_active":"2024-05-08T05:51:04.067484+0000","last_became_peered":"2024-05-08T05:51:04.067484+0000","last_unstale":"2024-05-08T05:51:05.094391+0000","last_undegraded":"2024-05-08T05:51:05.094391+0000","last_fullsized":"2024-05-08T05:51:05.094391+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:44.431356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088282+0000","last_change":"2024-05-08T05:51:05.080720+0000","last_active":"2024-05-08T05:51:05.088282+0000","last_peered":"2024-05-08T05:51:05.088282+0000","last_clean":"2024-05-08T05:51:05.088282+0000","last_became_active":"2024-05-08T05:51:05.080428+0000","last_became_peered":"2024-05-08T05:51:05.080428+0000","last_unstale":"2024-05-08T05:51:05.088282+0000","last_undegraded":"2024-05-08T05:51:05.088282+0000","last_fullsized":"2024-05-08T05:51:05.088282+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:51:07.851943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.070039+0000","last_change":"2024-05-08T05:51:04.065712+0000","last_active":"2024-05-08T05:51:05.070039+0000","last_peered":"2024-05-08T05:51:05.070039+0000","last_clean":"2024-05-08T05:51:05.070039+0000","last_became_active":"2024-05-08T05:51:04.065421+0000","last_became_peered":"2024-05-08T05:51:04.065421+0000","last_unstale":"2024-05-08T05:51:05.070039+0000","last_undegraded":"2024-05-08T05:51:05.070039+0000","last_fullsized":"2024-05-08T05:51:05.070039+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:44:48.375896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078591+0000","last_change":"2024-05-08T05:51:05.084846+0000","last_active":"2024-05-08T05:51:06.078591+0000","last_peered":"2024-05-08T05:51:06.078591+0000","last_clean":"2024-05-08T05:51:06.078591+0000","last_became_active":"2024-05-08T05:51:05.084544+0000","last_became_peered":"2024-05-08T05:51:05.084544+0000","last_unstale":"2024-05-08T05:51:06.078591+0000","last_undegraded":"2024-05-08T05:51:06.078591+0000","last_fullsized":"2024-05-08T05:51:06.078591+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:30.552529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.068303+0000","last_change":"2024-05-08T05:51:04.067872+0000","last_active":"2024-05-08T05:51:05.068303+0000","last_peered":"2024-05-08T05:51:05.068303+0000","last_clean":"2024-05-08T05:51:05.068303+0000","last_became_active":"2024-05-08T05:51:04.067380+0000","last_became_peered":"2024-05-08T05:51:04.067380+0000","last_unstale":"2024-05-08T05:51:05.068303+0000","last_undegraded":"2024-05-08T05:51:05.068303+0000","last_fullsized":"2024-05-08T05:51:05.068303+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:21.223822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052885+0000","last_change":"2024-05-08T05:51:04.052885+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052885+0000","last_undegraded":"2024-05-08T05:51:04.052885+0000","last_fullsized":"2024-05-08T05:51:04.052885+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.063576+0000","last_change":"2024-05-08T05:51:04.070032+0000","last_active":"2024-05-08T05:51:05.063576+0000","last_peered":"2024-05-08T05:51:05.063576+0000","last_clean":"2024-05-08T05:51:05.063576+0000","last_became_active":"2024-05-08T05:51:04.069748+0000","last_became_peered":"2024-05-08T05:51:04.069748+0000","last_unstale":"2024-05-08T05:51:05.063576+0000","last_undegraded":"2024-05-08T05:51:05.063576+0000","last_fullsized":"2024-05-08T05:51:05.063576+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:46:01.704267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.084392+0000","last_change":"2024-05-08T05:51:05.078677+0000","last_active":"2024-05-08T05:51:05.084392+0000","last_peered":"2024-05-08T05:51:05.084392+0000","last_clean":"2024-05-08T05:51:05.084392+0000","last_became_active":"2024-05-08T05:51:05.078364+0000","last_became_peered":"2024-05-08T05:51:05.078364+0000","last_unstale":"2024-05-08T05:51:05.084392+0000","last_undegraded":"2024-05-08T05:51:05.084392+0000","last_fullsized":"2024-05-08T05:51:05.084392+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:00.252213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078677+0000","last_change":"2024-05-08T05:51:04.067612+0000","last_active":"2024-05-08T05:51:06.078677+0000","last_peered":"2024-05-08T05:51:06.078677+0000","last_clean":"2024-05-08T05:51:06.078677+0000","last_became_active":"2024-05-08T05:51:04.067268+0000","last_became_peered":"2024-05-08T05:51:04.067268+0000","last_unstale":"2024-05-08T05:51:06.078677+0000","last_undegraded":"2024-05-08T05:51:06.078677+0000","last_fullsized":"2024-05-08T05:51:06.078677+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:28:11.738033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066899+0000","last_change":"2024-05-08T05:51:05.073728+0000","last_active":"2024-05-08T05:51:07.066899+0000","last_peered":"2024-05-08T05:51:07.066899+0000","last_clean":"2024-05-08T05:51:07.066899+0000","last_became_active":"2024-05-08T05:51:05.073415+0000","last_became_peered":"2024-05-08T05:51:05.073415+0000","last_unstale":"2024-05-08T05:51:07.066899+0000","last_undegraded":"2024-05-08T05:51:07.066899+0000","last_fullsized":"2024-05-08T05:51:07.066899+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:52.612093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.069657+0000","last_change":"2024-05-08T05:51:04.066627+0000","last_active":"2024-05-08T05:51:04.069657+0000","last_peered":"2024-05-08T05:51:04.069657+0000","last_clean":"2024-05-08T05:51:04.069657+0000","last_became_active":"2024-05-08T05:51:04.066321+0000","last_became_peered":"2024-05-08T05:51:04.066321+0000","last_unstale":"2024-05-08T05:51:04.069657+0000","last_undegraded":"2024-05-08T05:51:04.069657+0000","last_fullsized":"2024-05-08T05:51:04.069657+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:54.640197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052213+0000","last_change":"2024-05-08T05:51:04.052213+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052213+0000","last_undegraded":"2024-05-08T05:51:04.052213+0000","last_fullsized":"2024-05-08T05:51:04.052213+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.072448+0000","last_change":"2024-05-08T05:51:04.072005+0000","last_active":"2024-05-08T05:51:04.072448+0000","last_peered":"2024-05-08T05:51:04.072448+0000","last_clean":"2024-05-08T05:51:04.072448+0000","last_became_active":"2024-05-08T05:51:04.071776+0000","last_became_peered":"2024-05-08T05:51:04.071776+0000","last_unstale":"2024-05-08T05:51:04.072448+0000","last_undegraded":"2024-05-08T05:51:04.072448+0000","last_fullsized":"2024-05-08T05:51:04.072448+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:21.223822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066166+0000","last_change":"2024-05-08T05:51:05.074599+0000","last_active":"2024-05-08T05:51:07.066166+0000","last_peered":"2024-05-08T05:51:07.066166+0000","last_clean":"2024-05-08T05:51:07.066166+0000","last_became_active":"2024-05-08T05:51:05.073942+0000","last_became_peered":"2024-05-08T05:51:05.073942+0000","last_unstale":"2024-05-08T05:51:07.066166+0000","last_undegraded":"2024-05-08T05:51:07.066166+0000","last_fullsized":"2024-05-08T05:51:07.066166+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:30.124962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066177+0000","last_change":"2024-05-08T05:51:04.069082+0000","last_active":"2024-05-08T05:51:07.066177+0000","last_peered":"2024-05-08T05:51:07.066177+0000","last_clean":"2024-05-08T05:51:07.066177+0000","last_became_active":"2024-05-08T05:51:04.068801+0000","last_became_peered":"2024-05-08T05:51:04.068801+0000","last_unstale":"2024-05-08T05:51:07.066177+0000","last_undegraded":"2024-05-08T05:51:07.066177+0000","last_fullsized":"2024-05-08T05:51:07.066177+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:57:45.457442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.092747+0000","last_change":"2024-05-08T05:51:05.083611+0000","last_active":"2024-05-08T05:51:05.092747+0000","last_peered":"2024-05-08T05:51:05.092747+0000","last_clean":"2024-05-08T05:51:05.092747+0000","last_became_active":"2024-05-08T05:51:05.083328+0000","last_became_peered":"2024-05-08T05:51:05.083328+0000","last_unstale":"2024-05-08T05:51:05.092747+0000","last_undegraded":"2024-05-08T05:51:05.092747+0000","last_fullsized":"2024-05-08T05:51:05.092747+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:55.131091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.065846+0000","last_change":"2024-05-08T05:51:04.063770+0000","last_active":"2024-05-08T05:51:04.065846+0000","last_peered":"2024-05-08T05:51:04.065846+0000","last_clean":"2024-05-08T05:51:04.065846+0000","last_became_active":"2024-05-08T05:51:04.063509+0000","last_became_peered":"2024-05-08T05:51:04.063509+0000","last_unstale":"2024-05-08T05:51:04.065846+0000","last_undegraded":"2024-05-08T05:51:04.065846+0000","last_fullsized":"2024-05-08T05:51:04.065846+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:49:24.745045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093539+0000","last_change":"2024-05-08T05:51:05.090152+0000","last_active":"2024-05-08T05:51:05.093539+0000","last_peered":"2024-05-08T05:51:05.093539+0000","last_clean":"2024-05-08T05:51:05.093539+0000","last_became_active":"2024-05-08T05:51:05.089828+0000","last_became_peered":"2024-05-08T05:51:05.089828+0000","last_unstale":"2024-05-08T05:51:05.093539+0000","last_undegraded":"2024-05-08T05:51:05.093539+0000","last_fullsized":"2024-05-08T05:51:05.093539+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:34:19.612717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.069013+0000","last_change":"2024-05-08T05:51:04.069833+0000","last_active":"2024-05-08T05:51:05.069013+0000","last_peered":"2024-05-08T05:51:05.069013+0000","last_clean":"2024-05-08T05:51:05.069013+0000","last_became_active":"2024-05-08T05:51:04.069539+0000","last_became_peered":"2024-05-08T05:51:04.069539+0000","last_unstale":"2024-05-08T05:51:05.069013+0000","last_undegraded":"2024-05-08T05:51:05.069013+0000","last_fullsized":"2024-05-08T05:51:05.069013+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:36:49.817010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052495+0000","last_change":"2024-05-08T05:51:04.052495+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052495+0000","last_undegraded":"2024-05-08T05:51:04.052495+0000","last_fullsized":"2024-05-08T05:51:04.052495+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.065891+0000","last_change":"2024-05-08T05:51:04.063987+0000","last_active":"2024-05-08T05:51:04.065891+0000","last_peered":"2024-05-08T05:51:04.065891+0000","last_clean":"2024-05-08T05:51:04.065891+0000","last_became_active":"2024-05-08T05:51:04.063646+0000","last_became_peered":"2024-05-08T05:51:04.063646+0000","last_unstale":"2024-05-08T05:51:04.065891+0000","last_undegraded":"2024-05-08T05:51:04.065891+0000","last_fullsized":"2024-05-08T05:51:04.065891+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:58:37.220465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066817+0000","last_change":"2024-05-08T05:51:05.075930+0000","last_active":"2024-05-08T05:51:07.066817+0000","last_peered":"2024-05-08T05:51:07.066817+0000","last_clean":"2024-05-08T05:51:07.066817+0000","last_became_active":"2024-05-08T05:51:05.075052+0000","last_became_peered":"2024-05-08T05:51:05.075052+0000","last_unstale":"2024-05-08T05:51:07.066817+0000","last_undegraded":"2024-05-08T05:51:07.066817+0000","last_fullsized":"2024-05-08T05:51:07.066817+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:27:28.653248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066775+0000","last_change":"2024-05-08T05:51:04.063919+0000","last_active":"2024-05-08T05:51:07.066775+0000","last_peered":"2024-05-08T05:51:07.066775+0000","last_clean":"2024-05-08T05:51:07.066775+0000","last_became_active":"2024-05-08T05:51:04.063610+0000","last_became_peered":"2024-05-08T05:51:04.063610+0000","last_unstale":"2024-05-08T05:51:07.066775+0000","last_undegraded":"2024-05-08T05:51:07.066775+0000","last_fullsized":"2024-05-08T05:51:07.066775+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:26.855446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.076159+0000","last_change":"2024-05-08T05:51:05.073177+0000","last_active":"2024-05-08T05:51:05.076159+0000","last_peered":"2024-05-08T05:51:05.076159+0000","last_clean":"2024-05-08T05:51:05.076159+0000","last_became_active":"2024-05-08T05:51:05.072104+0000","last_became_peered":"2024-05-08T05:51:05.072104+0000","last_unstale":"2024-05-08T05:51:05.076159+0000","last_undegraded":"2024-05-08T05:51:05.076159+0000","last_fullsized":"2024-05-08T05:51:05.076159+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:29.725291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.070325+0000","last_change":"2024-05-08T05:51:04.069690+0000","last_active":"2024-05-08T05:51:04.070325+0000","last_peered":"2024-05-08T05:51:04.070325+0000","last_clean":"2024-05-08T05:51:04.070325+0000","last_became_active":"2024-05-08T05:51:04.069380+0000","last_became_peered":"2024-05-08T05:51:04.069380+0000","last_unstale":"2024-05-08T05:51:04.070325+0000","last_undegraded":"2024-05-08T05:51:04.070325+0000","last_fullsized":"2024-05-08T05:51:04.070325+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:26.339116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.053043+0000","last_change":"2024-05-08T05:51:04.053043+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.053043+0000","last_undegraded":"2024-05-08T05:51:04.053043+0000","last_fullsized":"2024-05-08T05:51:04.053043+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.070852+0000","last_change":"2024-05-08T05:51:04.070027+0000","last_active":"2024-05-08T05:51:04.070852+0000","last_peered":"2024-05-08T05:51:04.070852+0000","last_clean":"2024-05-08T05:51:04.070852+0000","last_became_active":"2024-05-08T05:51:04.069331+0000","last_became_peered":"2024-05-08T05:51:04.069331+0000","last_unstale":"2024-05-08T05:51:04.070852+0000","last_undegraded":"2024-05-08T05:51:04.070852+0000","last_fullsized":"2024-05-08T05:51:04.070852+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:28.717301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.079017+0000","last_change":"2024-05-08T05:51:05.079710+0000","last_active":"2024-05-08T05:51:06.079017+0000","last_peered":"2024-05-08T05:51:06.079017+0000","last_clean":"2024-05-08T05:51:06.079017+0000","last_became_active":"2024-05-08T05:51:05.079405+0000","last_became_peered":"2024-05-08T05:51:05.079405+0000","last_unstale":"2024-05-08T05:51:06.079017+0000","last_undegraded":"2024-05-08T05:51:06.079017+0000","last_fullsized":"2024-05-08T05:51:06.079017+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:57:15.216780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066813+0000","last_change":"2024-05-08T05:51:04.066485+0000","last_active":"2024-05-08T05:51:07.066813+0000","last_peered":"2024-05-08T05:51:07.066813+0000","last_clean":"2024-05-08T05:51:07.066813+0000","last_became_active":"2024-05-08T05:51:04.066192+0000","last_became_peered":"2024-05-08T05:51:04.066192+0000","last_unstale":"2024-05-08T05:51:07.066813+0000","last_undegraded":"2024-05-08T05:51:07.066813+0000","last_fullsized":"2024-05-08T05:51:07.066813+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:54:27.233544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066370+0000","last_change":"2024-05-08T05:51:05.074917+0000","last_active":"2024-05-08T05:51:07.066370+0000","last_peered":"2024-05-08T05:51:07.066370+0000","last_clean":"2024-05-08T05:51:07.066370+0000","last_became_active":"2024-05-08T05:51:05.074339+0000","last_became_peered":"2024-05-08T05:51:05.074339+0000","last_unstale":"2024-05-08T05:51:07.066370+0000","last_undegraded":"2024-05-08T05:51:07.066370+0000","last_fullsized":"2024-05-08T05:51:07.066370+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:22:04.619439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.062382+0000","last_change":"2024-05-08T05:51:04.057289+0000","last_active":"2024-05-08T05:51:05.062382+0000","last_peered":"2024-05-08T05:51:05.062382+0000","last_clean":"2024-05-08T05:51:05.062382+0000","last_became_active":"2024-05-08T05:51:04.056950+0000","last_became_peered":"2024-05-08T05:51:04.056950+0000","last_unstale":"2024-05-08T05:51:05.062382+0000","last_undegraded":"2024-05-08T05:51:05.062382+0000","last_fullsized":"2024-05-08T05:51:05.062382+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:17.541863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.079392+0000","last_change":"2024-05-08T05:51:05.076808+0000","last_active":"2024-05-08T05:51:05.079392+0000","last_peered":"2024-05-08T05:51:05.079392+0000","last_clean":"2024-05-08T05:51:05.079392+0000","last_became_active":"2024-05-08T05:51:05.076532+0000","last_became_peered":"2024-05-08T05:51:05.076532+0000","last_unstale":"2024-05-08T05:51:05.079392+0000","last_undegraded":"2024-05-08T05:51:05.079392+0000","last_fullsized":"2024-05-08T05:51:05.079392+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:12.526046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.062570+0000","last_change":"2024-05-08T05:51:04.054314+0000","last_active":"2024-05-08T05:51:05.062570+0000","last_peered":"2024-05-08T05:51:05.062570+0000","last_clean":"2024-05-08T05:51:05.062570+0000","last_became_active":"2024-05-08T05:51:04.053754+0000","last_became_peered":"2024-05-08T05:51:04.053754+0000","last_unstale":"2024-05-08T05:51:05.062570+0000","last_undegraded":"2024-05-08T05:51:05.062570+0000","last_fullsized":"2024-05-08T05:51:05.062570+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:26.339116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.053473+0000","last_change":"2024-05-08T05:51:04.053473+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.053473+0000","last_undegraded":"2024-05-08T05:51:04.053473+0000","last_fullsized":"2024-05-08T05:51:04.053473+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066301+0000","last_change":"2024-05-08T05:51:04.064412+0000","last_active":"2024-05-08T05:51:07.066301+0000","last_peered":"2024-05-08T05:51:07.066301+0000","last_clean":"2024-05-08T05:51:07.066301+0000","last_became_active":"2024-05-08T05:51:04.064134+0000","last_became_peered":"2024-05-08T05:51:04.064134+0000","last_unstale":"2024-05-08T05:51:07.066301+0000","last_undegraded":"2024-05-08T05:51:07.066301+0000","last_fullsized":"2024-05-08T05:51:07.066301+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:37:09.043074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088681+0000","last_change":"2024-05-08T05:51:05.087096+0000","last_active":"2024-05-08T05:51:05.088681+0000","last_peered":"2024-05-08T05:51:05.088681+0000","last_clean":"2024-05-08T05:51:05.088681+0000","last_became_active":"2024-05-08T05:51:05.085170+0000","last_became_peered":"2024-05-08T05:51:05.085170+0000","last_unstale":"2024-05-08T05:51:05.088681+0000","last_undegraded":"2024-05-08T05:51:05.088681+0000","last_fullsized":"2024-05-08T05:51:05.088681+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:20.213009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.066642+0000","last_change":"2024-05-08T05:51:04.065454+0000","last_active":"2024-05-08T05:51:04.066642+0000","last_peered":"2024-05-08T05:51:04.066642+0000","last_clean":"2024-05-08T05:51:04.066642+0000","last_became_active":"2024-05-08T05:51:04.065175+0000","last_became_peered":"2024-05-08T05:51:04.065175+0000","last_unstale":"2024-05-08T05:51:04.066642+0000","last_undegraded":"2024-05-08T05:51:04.066642+0000","last_fullsized":"2024-05-08T05:51:04.066642+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:01:19.756722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.051048+0000","last_change":"2024-05-08T05:51:04.051048+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.051048+0000","last_undegraded":"2024-05-08T05:51:04.051048+0000","last_fullsized":"2024-05-08T05:51:04.051048+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"118'8","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.096717+0000","last_change":"2024-05-08T05:51:04.068279+0000","last_active":"2024-05-08T05:51:05.096717+0000","last_peered":"2024-05-08T05:51:05.096717+0000","last_clean":"2024-05-08T05:51:05.096717+0000","last_became_active":"2024-05-08T05:51:04.067953+0000","last_became_peered":"2024-05-08T05:51:04.067953+0000","last_unstale":"2024-05-08T05:51:05.096717+0000","last_undegraded":"2024-05-08T05:51:05.096717+0000","last_fullsized":"2024-05-08T05:51:05.096717+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:54.640197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":288,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.055816+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:51:04.055816+0000","last_peered":"2024-05-08T05:51:04.055816+0000","last_clean":"2024-05-08T05:51:04.055816+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:51:04.055816+0000","last_undegraded":"2024-05-08T05:51:04.055816+0000","last_fullsized":"2024-05-08T05:51:04.055816+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":1668,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":3302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542265,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2048000,"data_stored":1266440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542266,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1454080,"data_stored":676072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542265,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1454080,"data_stored":676072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":684851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1482752,"data_stored":684885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1478656,"data_stored":689931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1470464,"data_stored":683261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2076672,"data_stored":1280321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:51:09.855 INFO:tasks.ceph.ceph_manager.ceph:PG 18.3c is not active+clean 2024-05-08T05:51:09.855 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.052577+0000', 'last_change': '2024-05-08T05:51:04.052577+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.052577+0000', 'last_undegraded': '2024-05-08T05:51:04.052577+0000', 'last_fullsized': '2024-05-08T05:51:04.052577+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T05:51:09.856 INFO:tasks.ceph.ceph_manager.ceph:PG 18.37 is not active+clean 2024-05-08T05:51:09.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.053739+0000', 'last_change': '2024-05-08T05:51:04.053739+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.053739+0000', 'last_undegraded': '2024-05-08T05:51:04.053739+0000', 'last_fullsized': '2024-05-08T05:51:04.053739+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T05:51:09.856 INFO:tasks.ceph.ceph_manager.ceph:PG 18.2d is not active+clean 2024-05-08T05:51:09.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.056046+0000', 'last_change': '2024-05-08T05:51:04.056046+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.056046+0000', 'last_undegraded': '2024-05-08T05:51:04.056046+0000', 'last_fullsized': '2024-05-08T05:51:04.056046+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T05:51:09.856 INFO:tasks.ceph.ceph_manager.ceph:PG 18.2b is not active+clean 2024-05-08T05:51:09.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.051557+0000', 'last_change': '2024-05-08T05:51:04.051557+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.051557+0000', 'last_undegraded': '2024-05-08T05:51:04.051557+0000', 'last_fullsized': '2024-05-08T05:51:04.051557+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T05:51:09.856 INFO:tasks.ceph.ceph_manager.ceph:PG 18.29 is not active+clean 2024-05-08T05:51:09.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.050947+0000', 'last_change': '2024-05-08T05:51:04.050947+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.050947+0000', 'last_undegraded': '2024-05-08T05:51:04.050947+0000', 'last_fullsized': '2024-05-08T05:51:04.050947+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T05:51:09.856 INFO:tasks.ceph.ceph_manager.ceph:PG 18.28 is not active+clean 2024-05-08T05:51:09.856 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.052838+0000', 'last_change': '2024-05-08T05:51:04.052838+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.052838+0000', 'last_undegraded': '2024-05-08T05:51:04.052838+0000', 'last_fullsized': '2024-05-08T05:51:04.052838+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:51:09.856 INFO:tasks.ceph.ceph_manager.ceph:PG 18.27 is not active+clean 2024-05-08T05:51:09.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.053724+0000', 'last_change': '2024-05-08T05:51:04.053724+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.053724+0000', 'last_undegraded': '2024-05-08T05:51:04.053724+0000', 'last_fullsized': '2024-05-08T05:51:04.053724+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:51:09.857 INFO:tasks.ceph.ceph_manager.ceph:PG 18.25 is not active+clean 2024-05-08T05:51:09.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.055539+0000', 'last_change': '2024-05-08T05:51:04.055539+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.055539+0000', 'last_undegraded': '2024-05-08T05:51:04.055539+0000', 'last_fullsized': '2024-05-08T05:51:04.055539+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T05:51:09.857 INFO:tasks.ceph.ceph_manager.ceph:PG 18.21 is not active+clean 2024-05-08T05:51:09.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.051392+0000', 'last_change': '2024-05-08T05:51:04.051392+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.051392+0000', 'last_undegraded': '2024-05-08T05:51:04.051392+0000', 'last_fullsized': '2024-05-08T05:51:04.051392+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:51:09.857 INFO:tasks.ceph.ceph_manager.ceph:PG 18.20 is not active+clean 2024-05-08T05:51:09.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.050670+0000', 'last_change': '2024-05-08T05:51:04.050670+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.050670+0000', 'last_undegraded': '2024-05-08T05:51:04.050670+0000', 'last_fullsized': '2024-05-08T05:51:04.050670+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:51:09.857 INFO:tasks.ceph.ceph_manager.ceph:PG 18.1e is not active+clean 2024-05-08T05:51:09.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.052424+0000', 'last_change': '2024-05-08T05:51:04.052424+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.052424+0000', 'last_undegraded': '2024-05-08T05:51:04.052424+0000', 'last_fullsized': '2024-05-08T05:51:04.052424+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:51:09.857 INFO:tasks.ceph.ceph_manager.ceph:PG 18.1f is not active+clean 2024-05-08T05:51:09.857 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.053316+0000', 'last_change': '2024-05-08T05:51:04.053316+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.053316+0000', 'last_undegraded': '2024-05-08T05:51:04.053316+0000', 'last_fullsized': '2024-05-08T05:51:04.053316+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T05:51:09.858 INFO:tasks.ceph.ceph_manager.ceph:PG 18.14 is not active+clean 2024-05-08T05:51:09.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.055148+0000', 'last_change': '2024-05-08T05:51:04.055148+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.055148+0000', 'last_undegraded': '2024-05-08T05:51:04.055148+0000', 'last_fullsized': '2024-05-08T05:51:04.055148+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T05:51:09.858 INFO:tasks.ceph.ceph_manager.ceph:PG 18.15 is not active+clean 2024-05-08T05:51:09.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.051301+0000', 'last_change': '2024-05-08T05:51:04.051301+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.051301+0000', 'last_undegraded': '2024-05-08T05:51:04.051301+0000', 'last_fullsized': '2024-05-08T05:51:04.051301+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:51:09.858 INFO:tasks.ceph.ceph_manager.ceph:PG 18.17 is not active+clean 2024-05-08T05:51:09.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.052749+0000', 'last_change': '2024-05-08T05:51:04.052749+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.052749+0000', 'last_undegraded': '2024-05-08T05:51:04.052749+0000', 'last_fullsized': '2024-05-08T05:51:04.052749+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:51:09.858 INFO:tasks.ceph.ceph_manager.ceph:PG 18.10 is not active+clean 2024-05-08T05:51:09.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.054481+0000', 'last_change': '2024-05-08T05:51:04.054481+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.054481+0000', 'last_undegraded': '2024-05-08T05:51:04.054481+0000', 'last_fullsized': '2024-05-08T05:51:04.054481+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-05-08T05:51:09.858 INFO:tasks.ceph.ceph_manager.ceph:PG 18.c is not active+clean 2024-05-08T05:51:09.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.052885+0000', 'last_change': '2024-05-08T05:51:04.052885+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.052885+0000', 'last_undegraded': '2024-05-08T05:51:04.052885+0000', 'last_fullsized': '2024-05-08T05:51:04.052885+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:51:09.858 INFO:tasks.ceph.ceph_manager.ceph:PG 18.f is not active+clean 2024-05-08T05:51:09.858 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.052213+0000', 'last_change': '2024-05-08T05:51:04.052213+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.052213+0000', 'last_undegraded': '2024-05-08T05:51:04.052213+0000', 'last_fullsized': '2024-05-08T05:51:04.052213+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:51:09.859 INFO:tasks.ceph.ceph_manager.ceph:PG 18.b is not active+clean 2024-05-08T05:51:09.859 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.052495+0000', 'last_change': '2024-05-08T05:51:04.052495+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.052495+0000', 'last_undegraded': '2024-05-08T05:51:04.052495+0000', 'last_fullsized': '2024-05-08T05:51:04.052495+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-05-08T05:51:09.859 INFO:tasks.ceph.ceph_manager.ceph:PG 18.6 is not active+clean 2024-05-08T05:51:09.859 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.053043+0000', 'last_change': '2024-05-08T05:51:04.053043+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.053043+0000', 'last_undegraded': '2024-05-08T05:51:04.053043+0000', 'last_fullsized': '2024-05-08T05:51:04.053043+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T05:51:09.859 INFO:tasks.ceph.ceph_manager.ceph:PG 18.2 is not active+clean 2024-05-08T05:51:09.859 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.053473+0000', 'last_change': '2024-05-08T05:51:04.053473+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.053473+0000', 'last_undegraded': '2024-05-08T05:51:04.053473+0000', 'last_fullsized': '2024-05-08T05:51:04.053473+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T05:51:09.859 INFO:tasks.ceph.ceph_manager.ceph:PG 18.13 is not active+clean 2024-05-08T05:51:09.859 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '18.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 117, 'state': 'creating+peering', 'last_fresh': '2024-05-08T05:51:04.051048+0000', 'last_change': '2024-05-08T05:51:04.051048+0000', 'last_active': '2024-05-08T05:51:04.041342+0000', 'last_peered': '2024-05-08T05:51:04.041342+0000', 'last_clean': '2024-05-08T05:51:04.041342+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-05-08T05:51:04.051048+0000', 'last_undegraded': '2024-05-08T05:51:04.051048+0000', 'last_fullsized': '2024-05-08T05:51:04.051048+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'last_clean_scrub_stamp': '2024-05-08T05:51:04.041342+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-05-08T05:51:09.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T05:51:09.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.994+0000 7fa06cc9b640 1 Processor -- start 2024-05-08T05:51:09.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.994+0000 7fa06cc9b640 1 -- start start 2024-05-08T05:51:09.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.994+0000 7fa06cc9b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa068070c70 0x7fa0680d3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:09.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.994+0000 7fa06cc9b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0680cee00 con 0x7fa0680d0d40 2024-05-08T05:51:09.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.994+0000 7fa06cc9b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa068161c40 con 0x7fa06815d2f0 2024-05-08T05:51:09.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.994+0000 7fa06cc9b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa0680cd9e0 con 0x7fa068070c70 2024-05-08T05:51:09.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.995+0000 7fa05ffff640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa068070c70 0x7fa0680d3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:09.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.995+0000 7fa066d9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa0680d0d40 0x7fa068160e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52830/0 (socket says 172.21.15.45:52830) 2024-05-08T05:51:09.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.995+0000 7fa066d9d640 1 -- 172.21.15.45:0/351871293 learned_addr learned my addr 172.21.15.45:0/351871293 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:09.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.995+0000 7fa065d9b640 1 -- 172.21.15.45:0/351871293 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4282089443 0 0) 0x7fa0680cee00 con 0x7fa0680d0d40 2024-05-08T05:51:09.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.995+0000 7fa065d9b640 1 -- 172.21.15.45:0/351871293 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa054003520 con 0x7fa0680d0d40 2024-05-08T05:51:09.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.995+0000 7fa05ffff640 1 -- 172.21.15.45:0/351871293 >> v1:172.21.15.63:6789/0 conn(0x7fa06815d2f0 legacy=0x7fa0680ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:09.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.995+0000 7fa05ffff640 1 -- 172.21.15.45:0/351871293 >> v1:172.21.15.45:6789/0 conn(0x7fa0680d0d40 legacy=0x7fa068160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:09.997 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.995+0000 7fa05ffff640 1 -- 172.21.15.45:0/351871293 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0680711e0 con 0x7fa068070c70 2024-05-08T05:51:09.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.996+0000 7fa05ffff640 1 --2- 172.21.15.45:0/351871293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa068070c70 0x7fa0680d3960 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fa0500097a0 tx=0x7fa05002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a06a0b2ea266917 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:09.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.996+0000 7fa065d9b640 1 -- 172.21.15.45:0/351871293 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050037070 con 0x7fa068070c70 2024-05-08T05:51:09.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.996+0000 7fa065d9b640 1 -- 172.21.15.45:0/351871293 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa050032440 con 0x7fa068070c70 2024-05-08T05:51:09.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.996+0000 7fa065d9b640 1 -- 172.21.15.45:0/351871293 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050032810 con 0x7fa068070c70 2024-05-08T05:51:09.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.996+0000 7fa06cc9b640 1 -- 172.21.15.45:0/351871293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa068070c70 msgr2=0x7fa0680d3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:09.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.996+0000 7fa06cc9b640 1 --2- 172.21.15.45:0/351871293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa068070c70 0x7fa0680d3960 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fa0500097a0 tx=0x7fa05002a230 comp rx=0 tx=0).stop 2024-05-08T05:51:09.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.996+0000 7fa06cc9b640 1 -- 172.21.15.45:0/351871293 shutdown_connections 2024-05-08T05:51:09.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.996+0000 7fa06cc9b640 1 --2- 172.21.15.45:0/351871293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa068070c70 0x7fa0680d3960 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:09.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.996+0000 7fa06cc9b640 1 -- 172.21.15.45:0/351871293 >> 172.21.15.45:0/351871293 conn(0x7fa0680d2e40 msgr2=0x7fa0680d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:09.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.996+0000 7fa06cc9b640 1 -- 172.21.15.45:0/351871293 shutdown_connections 2024-05-08T05:51:09.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.996+0000 7fa06cc9b640 1 -- 172.21.15.45:0/351871293 wait complete. 2024-05-08T05:51:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.998+0000 7fa06cc9b640 1 Processor -- start 2024-05-08T05:51:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.998+0000 7fa06cc9b640 1 -- start start 2024-05-08T05:51:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa06cc9b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa068070c70 0x7fa0680cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa06cc9b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0680d0d40 0x7fa068149110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa06cc9b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa06815d2f0 0x7fa0680d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa06cc9b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0680cdef0 con 0x7fa0680d0d40 2024-05-08T05:51:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa06cc9b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa0680714c0 con 0x7fa068070c70 2024-05-08T05:51:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa06cc9b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa0680ce1d0 con 0x7fa06815d2f0 2024-05-08T05:51:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa05ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0680d0d40 0x7fa068149110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa05ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0680d0d40 0x7fa068149110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37366/0 (socket says 172.21.15.45:37366) 2024-05-08T05:51:10.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa05ffff640 1 -- 172.21.15.45:0/2067688782 learned_addr learned my addr 172.21.15.45:0/2067688782 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa06659c640 1 --2- 172.21.15.45:0/2067688782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa068070c70 0x7fa0680cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa066d9d640 1 --2- 172.21.15.45:0/2067688782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa06815d2f0 0x7fa0680d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa05ffff640 1 -- 172.21.15.45:0/2067688782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa06815d2f0 msgr2=0x7fa0680d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa05ffff640 1 --2- 172.21.15.45:0/2067688782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa06815d2f0 0x7fa0680d21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa05ffff640 1 -- 172.21.15.45:0/2067688782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa068070c70 msgr2=0x7fa0680cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa05ffff640 1 --2- 172.21.15.45:0/2067688782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa068070c70 0x7fa0680cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa05ffff640 1 -- 172.21.15.45:0/2067688782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0681e8510 con 0x7fa0680d0d40 2024-05-08T05:51:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa05ffff640 1 --2- 172.21.15.45:0/2067688782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0680d0d40 0x7fa068149110 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fa050035040 tx=0x7fa0500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa05effd640 1 -- 172.21.15.45:0/2067688782 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050037040 con 0x7fa0680d0d40 2024-05-08T05:51:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa05effd640 1 -- 172.21.15.45:0/2067688782 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa050030500 con 0x7fa0680d0d40 2024-05-08T05:51:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa06cc9b640 1 -- 172.21.15.45:0/2067688782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0681e87f0 con 0x7fa0680d0d40 2024-05-08T05:51:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa05effd640 1 -- 172.21.15.45:0/2067688782 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050030910 con 0x7fa0680d0d40 2024-05-08T05:51:10.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:09.999+0000 7fa06cc9b640 1 -- 172.21.15.45:0/2067688782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0681e8db0 con 0x7fa0680d0d40 2024-05-08T05:51:10.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.000+0000 7fa06cc9b640 1 -- 172.21.15.45:0/2067688782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0680693f0 con 0x7fa0680d0d40 2024-05-08T05:51:10.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.001+0000 7fa05effd640 1 -- 172.21.15.45:0/2067688782 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa050044020 con 0x7fa0680d0d40 2024-05-08T05:51:10.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.002+0000 7fa05effd640 1 --2- 172.21.15.45:0/2067688782 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa0480762d0 0x7fa048078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:10.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.002+0000 7fa05effd640 1 -- 172.21.15.45:0/2067688782 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7fa0500b79e0 con 0x7fa0680d0d40 2024-05-08T05:51:10.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.002+0000 7fa06659c640 1 --2- 172.21.15.45:0/2067688782 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa0480762d0 0x7fa048078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:10.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.002+0000 7fa06659c640 1 --2- 172.21.15.45:0/2067688782 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa0480762d0 0x7fa048078790 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa0580017c0 tx=0x7fa058009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:10.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.006+0000 7fa05effd640 1 -- 172.21.15.45:0/2067688782 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa0500811a0 con 0x7fa0680d0d40 2024-05-08T05:51:10.031 INFO:tasks.ceph.mds.c.smithi045.stderr:2024-05-08T05:51:10.030+0000 7facdd29f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:51:10.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.273+0000 7fa06cc9b640 1 -- 172.21.15.45:0/2067688782 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fa0680b8dc0 con 0x7fa0680d0d40 2024-05-08T05:51:10.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.273+0000 7fa05effd640 1 -- 172.21.15.45:0/2067688782 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1263 (secure 0 0 0) 0x7fa050080aa0 con 0x7fa0680d0d40 2024-05-08T05:51:10.274 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:10.274 INFO:teuthology.orchestra.run.smithi045.stdout:{"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":601,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":119,"num_osds":8,"num_up_osds":8,"osd_up_since":1715146878,"num_in_osds":8,"osd_in_since":1715146870,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":75},{"state_name":"creating+peering","count":22}],"num_pgs":97,"num_pools":3,"num_objects":17,"data_bytes":592036,"bytes_used":268398592,"bytes_avail":772825714688,"bytes_total":773094113280,"inactive_pgs_ratio":0.22680412232875824,"write_bytes_sec":1900,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":100,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8112}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-05-08T05:50:10.961146+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T05:51:10.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.275+0000 7fa06cc9b640 1 -- 172.21.15.45:0/2067688782 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa0480762d0 msgr2=0x7fa048078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:10.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.275+0000 7fa06cc9b640 1 --2- 172.21.15.45:0/2067688782 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa0480762d0 0x7fa048078790 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa0580017c0 tx=0x7fa058009040 comp rx=0 tx=0).stop 2024-05-08T05:51:10.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.275+0000 7fa06cc9b640 1 -- 172.21.15.45:0/2067688782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0680d0d40 msgr2=0x7fa068149110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:10.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.275+0000 7fa06cc9b640 1 --2- 172.21.15.45:0/2067688782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0680d0d40 0x7fa068149110 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fa050035040 tx=0x7fa0500046c0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.275+0000 7fa06cc9b640 1 -- 172.21.15.45:0/2067688782 shutdown_connections 2024-05-08T05:51:10.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.275+0000 7fa06cc9b640 1 --2- 172.21.15.45:0/2067688782 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa0480762d0 0x7fa048078790 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.275+0000 7fa06cc9b640 1 --2- 172.21.15.45:0/2067688782 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa06815d2f0 0x7fa0680d21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.275+0000 7fa06cc9b640 1 --2- 172.21.15.45:0/2067688782 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa0680d0d40 0x7fa068149110 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.275+0000 7fa06cc9b640 1 --2- 172.21.15.45:0/2067688782 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa068070c70 0x7fa0680cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.275+0000 7fa06cc9b640 1 -- 172.21.15.45:0/2067688782 >> 172.21.15.45:0/2067688782 conn(0x7fa0680d2e40 msgr2=0x7fa0680b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:10.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.275+0000 7fa06cc9b640 1 -- 172.21.15.45:0/2067688782 shutdown_connections 2024-05-08T05:51:10.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.275+0000 7fa06cc9b640 1 -- 172.21.15.45:0/2067688782 wait complete. 2024-05-08T05:51:10.287 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-05-08T05:51:10.287 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:51:10.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.391+0000 7f646195f640 1 Processor -- start 2024-05-08T05:51:10.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.391+0000 7f646195f640 1 -- start start 2024-05-08T05:51:10.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.392+0000 7f646195f640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f645c15bc50 0x7f645c0b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:10.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.392+0000 7f646195f640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f645c0711f0 con 0x7f645c06f540 2024-05-08T05:51:10.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.392+0000 7f646195f640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f645c06fa40 con 0x7f645c0732c0 2024-05-08T05:51:10.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.392+0000 7f646195f640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f645c06fd20 con 0x7f645c15bc50 2024-05-08T05:51:10.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.392+0000 7f645affd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f645c15bc50 0x7f645c0b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:10.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.392+0000 7f645a7fc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f645c06f540 0x7f645c070ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:52844/0 (socket says 172.21.15.45:52844) 2024-05-08T05:51:10.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.392+0000 7f645a7fc640 1 -- 172.21.15.45:0/2726536022 learned_addr learned my addr 172.21.15.45:0/2726536022 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:10.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.393+0000 7f645affd640 1 -- 172.21.15.45:0/2726536022 >> v1:172.21.15.63:6789/0 conn(0x7f645c0732c0 legacy=0x7f645c0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:10.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.393+0000 7f645affd640 1 -- 172.21.15.45:0/2726536022 >> v1:172.21.15.45:6789/0 conn(0x7f645c06f540 legacy=0x7f645c070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:10.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.393+0000 7f645affd640 1 -- 172.21.15.45:0/2726536022 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c070000 con 0x7f645c15bc50 2024-05-08T05:51:10.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.393+0000 7f6459ffb640 1 -- 172.21.15.45:0/2726536022 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 867202871 0 0) 0x7f645c0711f0 con 0x7f645c06f540 2024-05-08T05:51:10.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.393+0000 7f645affd640 1 --2- 172.21.15.45:0/2726536022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f645c15bc50 0x7f645c0b5ba0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f6454009d90 tx=0x7f645402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3010cfc97f30e159 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:10.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.394+0000 7f6459ffb640 1 -- 172.21.15.45:0/2726536022 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454007740 con 0x7f645c15bc50 2024-05-08T05:51:10.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.394+0000 7f6459ffb640 1 -- 172.21.15.45:0/2726536022 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64540078e0 con 0x7f645c15bc50 2024-05-08T05:51:10.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.394+0000 7f6459ffb640 1 -- 172.21.15.45:0/2726536022 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454007cb0 con 0x7f645c15bc50 2024-05-08T05:51:10.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.394+0000 7f646195f640 1 -- 172.21.15.45:0/2726536022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f645c15bc50 msgr2=0x7f645c0b5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:10.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.394+0000 7f646195f640 1 --2- 172.21.15.45:0/2726536022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f645c15bc50 0x7f645c0b5ba0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f6454009d90 tx=0x7f645402a660 comp rx=0 tx=0).stop 2024-05-08T05:51:10.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.394+0000 7f646195f640 1 -- 172.21.15.45:0/2726536022 shutdown_connections 2024-05-08T05:51:10.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.394+0000 7f646195f640 1 --2- 172.21.15.45:0/2726536022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f645c15bc50 0x7f645c0b5ba0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.394+0000 7f646195f640 1 -- 172.21.15.45:0/2726536022 >> 172.21.15.45:0/2726536022 conn(0x7f645c0b3c40 msgr2=0x7f645c0b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:10.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.394+0000 7f646195f640 1 -- 172.21.15.45:0/2726536022 shutdown_connections 2024-05-08T05:51:10.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.394+0000 7f646195f640 1 -- 172.21.15.45:0/2726536022 wait complete. 2024-05-08T05:51:10.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.395+0000 7f646195f640 1 Processor -- start 2024-05-08T05:51:10.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.396+0000 7f646195f640 1 -- start start 2024-05-08T05:51:10.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.396+0000 7f646195f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f645c06f540 0x7f645c152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:10.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.396+0000 7f646195f640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f645c0732c0 0x7f645c1e8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:10.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.396+0000 7f646195f640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f645c15bc50 0x7f645c1e8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:10.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.396+0000 7f646195f640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f645c0b60e0 con 0x7f645c06f540 2024-05-08T05:51:10.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.396+0000 7f646195f640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f645c0705c0 con 0x7f645c0732c0 2024-05-08T05:51:10.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.396+0000 7f646195f640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f645c0b6260 con 0x7f645c15bc50 2024-05-08T05:51:10.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.396+0000 7f645affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f645c06f540 0x7f645c152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:10.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.396+0000 7f645affd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f645c06f540 0x7f645c152fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37374/0 (socket says 172.21.15.45:37374) 2024-05-08T05:51:10.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.397+0000 7f645affd640 1 -- 172.21.15.45:0/2760087354 learned_addr learned my addr 172.21.15.45:0/2760087354 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:10.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.397+0000 7f645a7fc640 1 --2- 172.21.15.45:0/2760087354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f645c0732c0 0x7f645c1e8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:10.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.397+0000 7f645b7fe640 1 --2- 172.21.15.45:0/2760087354 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f645c15bc50 0x7f645c1e8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:10.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.397+0000 7f645affd640 1 -- 172.21.15.45:0/2760087354 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f645c15bc50 msgr2=0x7f645c1e8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:10.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.397+0000 7f645affd640 1 --2- 172.21.15.45:0/2760087354 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f645c15bc50 0x7f645c1e8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.397+0000 7f645affd640 1 -- 172.21.15.45:0/2760087354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f645c0732c0 msgr2=0x7f645c1e8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:10.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.397+0000 7f645affd640 1 --2- 172.21.15.45:0/2760087354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f645c0732c0 0x7f645c1e8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.397+0000 7f645affd640 1 -- 172.21.15.45:0/2760087354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c15d8e0 con 0x7f645c06f540 2024-05-08T05:51:10.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.397+0000 7f645affd640 1 --2- 172.21.15.45:0/2760087354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f645c06f540 0x7f645c152fe0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f6454009d90 tx=0x7f6454004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.397+0000 7f6453fff640 1 -- 172.21.15.45:0/2760087354 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454040070 con 0x7f645c06f540 2024-05-08T05:51:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.397+0000 7f6453fff640 1 -- 172.21.15.45:0/2760087354 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6454031c40 con 0x7f645c06f540 2024-05-08T05:51:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.397+0000 7f646195f640 1 -- 172.21.15.45:0/2760087354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645c153aa0 con 0x7f645c06f540 2024-05-08T05:51:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.397+0000 7f6453fff640 1 -- 172.21.15.45:0/2760087354 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645403c420 con 0x7f645c06f540 2024-05-08T05:51:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.397+0000 7f646195f640 1 -- 172.21.15.45:0/2760087354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645c153520 con 0x7f645c06f540 2024-05-08T05:51:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.398+0000 7f646195f640 1 -- 172.21.15.45:0/2760087354 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f645c069b00 con 0x7f645c06f540 2024-05-08T05:51:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.399+0000 7f6453fff640 1 -- 172.21.15.45:0/2760087354 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6454039040 con 0x7f645c06f540 2024-05-08T05:51:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.399+0000 7f6453fff640 1 --2- 172.21.15.45:0/2760087354 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f643c0762d0 0x7f643c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.400+0000 7f6453fff640 1 -- 172.21.15.45:0/2760087354 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f64540b8c00 con 0x7f645c06f540 2024-05-08T05:51:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.400+0000 7f645a7fc640 1 --2- 172.21.15.45:0/2760087354 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f643c0762d0 0x7f643c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.400+0000 7f645a7fc640 1 --2- 172.21.15.45:0/2760087354 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f643c0762d0 0x7f643c078790 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6444007f30 tx=0x7f6444007ec0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:10.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.404+0000 7f6453fff640 1 -- 172.21.15.45:0/2760087354 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f64540bf080 con 0x7f645c06f540 2024-05-08T05:51:10.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.596+0000 7f646195f640 1 -- 172.21.15.45:0/2760087354 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f645c074690 con 0x7f643c0762d0 2024-05-08T05:51:10.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.603+0000 7f6453fff640 1 -- 172.21.15.45:0/2760087354 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250600 (secure 0 0 0) 0x7f645c074690 con 0x7f643c0762d0 2024-05-08T05:51:10.606 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:10.607 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:51:10.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.606+0000 7f646195f640 1 -- 172.21.15.45:0/2760087354 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f643c0762d0 msgr2=0x7f643c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:10.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.606+0000 7f646195f640 1 --2- 172.21.15.45:0/2760087354 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f643c0762d0 0x7f643c078790 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6444007f30 tx=0x7f6444007ec0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.606+0000 7f646195f640 1 -- 172.21.15.45:0/2760087354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f645c06f540 msgr2=0x7f645c152fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:10.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.606+0000 7f646195f640 1 --2- 172.21.15.45:0/2760087354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f645c06f540 0x7f645c152fe0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f6454009d90 tx=0x7f6454004a40 comp rx=0 tx=0).stop 2024-05-08T05:51:10.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.606+0000 7f646195f640 1 -- 172.21.15.45:0/2760087354 shutdown_connections 2024-05-08T05:51:10.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.606+0000 7f646195f640 1 --2- 172.21.15.45:0/2760087354 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f643c0762d0 0x7f643c078790 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.606+0000 7f646195f640 1 --2- 172.21.15.45:0/2760087354 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f645c15bc50 0x7f645c1e8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.606+0000 7f646195f640 1 --2- 172.21.15.45:0/2760087354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f645c0732c0 0x7f645c1e8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.606+0000 7f646195f640 1 --2- 172.21.15.45:0/2760087354 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f645c06f540 0x7f645c152fe0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:10.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.606+0000 7f646195f640 1 -- 172.21.15.45:0/2760087354 >> 172.21.15.45:0/2760087354 conn(0x7f645c0b3c40 msgr2=0x7f645c0b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:10.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.606+0000 7f646195f640 1 -- 172.21.15.45:0/2760087354 shutdown_connections 2024-05-08T05:51:10.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:10.606+0000 7f646195f640 1 -- 172.21.15.45:0/2760087354 wait complete. 2024-05-08T05:51:10.618 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2024-05-08T05:51:08.974825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592036,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":133,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262108,"kb_used_data":12640,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712612,"statfs":{"total":773094113280,"available":772825714688,"internally_reserved":0,"allocated":12943360,"data_stored":6641833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1668,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.928070"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.073391+0000","last_change":"2024-05-08T05:51:05.069476+0000","last_active":"2024-05-08T05:51:05.073391+0000","last_peered":"2024-05-08T05:51:05.073391+0000","last_clean":"2024-05-08T05:51:05.073391+0000","last_became_active":"2024-05-08T05:51:05.069131+0000","last_became_peered":"2024-05-08T05:51:05.069131+0000","last_unstale":"2024-05-08T05:51:05.073391+0000","last_undegraded":"2024-05-08T05:51:05.073391+0000","last_fullsized":"2024-05-08T05:51:05.073391+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:05:09.239164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093231+0000","last_change":"2024-05-08T05:51:05.084139+0000","last_active":"2024-05-08T05:51:05.093231+0000","last_peered":"2024-05-08T05:51:05.093231+0000","last_clean":"2024-05-08T05:51:05.093231+0000","last_became_active":"2024-05-08T05:51:05.081830+0000","last_became_peered":"2024-05-08T05:51:05.081830+0000","last_unstale":"2024-05-08T05:51:05.093231+0000","last_undegraded":"2024-05-08T05:51:05.093231+0000","last_fullsized":"2024-05-08T05:51:05.093231+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:37.715909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093594+0000","last_change":"2024-05-08T05:51:05.090876+0000","last_active":"2024-05-08T05:51:05.093594+0000","last_peered":"2024-05-08T05:51:05.093594+0000","last_clean":"2024-05-08T05:51:05.093594+0000","last_became_active":"2024-05-08T05:51:05.089329+0000","last_became_peered":"2024-05-08T05:51:05.089329+0000","last_unstale":"2024-05-08T05:51:05.093594+0000","last_undegraded":"2024-05-08T05:51:05.093594+0000","last_fullsized":"2024-05-08T05:51:05.093594+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:51:43.350274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052577+0000","last_change":"2024-05-08T05:51:04.052577+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052577+0000","last_undegraded":"2024-05-08T05:51:04.052577+0000","last_fullsized":"2024-05-08T05:51:04.052577+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.077282+0000","last_change":"2024-05-08T05:51:05.072296+0000","last_active":"2024-05-08T05:51:05.077282+0000","last_peered":"2024-05-08T05:51:05.077282+0000","last_clean":"2024-05-08T05:51:05.077282+0000","last_became_active":"2024-05-08T05:51:05.072009+0000","last_became_peered":"2024-05-08T05:51:05.072009+0000","last_unstale":"2024-05-08T05:51:05.077282+0000","last_undegraded":"2024-05-08T05:51:05.077282+0000","last_fullsized":"2024-05-08T05:51:05.077282+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:12:02.641413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066407+0000","last_change":"2024-05-08T05:51:05.082847+0000","last_active":"2024-05-08T05:51:07.066407+0000","last_peered":"2024-05-08T05:51:07.066407+0000","last_clean":"2024-05-08T05:51:07.066407+0000","last_became_active":"2024-05-08T05:51:05.082267+0000","last_became_peered":"2024-05-08T05:51:05.082267+0000","last_unstale":"2024-05-08T05:51:07.066407+0000","last_undegraded":"2024-05-08T05:51:07.066407+0000","last_fullsized":"2024-05-08T05:51:07.066407+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:16:17.521040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066220+0000","last_change":"2024-05-08T05:51:05.087173+0000","last_active":"2024-05-08T05:51:07.066220+0000","last_peered":"2024-05-08T05:51:07.066220+0000","last_clean":"2024-05-08T05:51:07.066220+0000","last_became_active":"2024-05-08T05:51:05.086914+0000","last_became_peered":"2024-05-08T05:51:05.086914+0000","last_unstale":"2024-05-08T05:51:07.066220+0000","last_undegraded":"2024-05-08T05:51:07.066220+0000","last_fullsized":"2024-05-08T05:51:07.066220+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:42:24.823847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093326+0000","last_change":"2024-05-08T05:51:05.085921+0000","last_active":"2024-05-08T05:51:05.093326+0000","last_peered":"2024-05-08T05:51:05.093326+0000","last_clean":"2024-05-08T05:51:05.093326+0000","last_became_active":"2024-05-08T05:51:05.083800+0000","last_became_peered":"2024-05-08T05:51:05.083800+0000","last_unstale":"2024-05-08T05:51:05.093326+0000","last_undegraded":"2024-05-08T05:51:05.093326+0000","last_fullsized":"2024-05-08T05:51:05.093326+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:47.453010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.053739+0000","last_change":"2024-05-08T05:51:04.053739+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.053739+0000","last_undegraded":"2024-05-08T05:51:04.053739+0000","last_fullsized":"2024-05-08T05:51:04.053739+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.075989+0000","last_change":"2024-05-08T05:51:05.071111+0000","last_active":"2024-05-08T05:51:05.075989+0000","last_peered":"2024-05-08T05:51:05.075989+0000","last_clean":"2024-05-08T05:51:05.075989+0000","last_became_active":"2024-05-08T05:51:05.070823+0000","last_became_peered":"2024-05-08T05:51:05.070823+0000","last_unstale":"2024-05-08T05:51:05.075989+0000","last_undegraded":"2024-05-08T05:51:05.075989+0000","last_fullsized":"2024-05-08T05:51:05.075989+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:18.549853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.073515+0000","last_change":"2024-05-08T05:51:05.070718+0000","last_active":"2024-05-08T05:51:05.073515+0000","last_peered":"2024-05-08T05:51:05.073515+0000","last_clean":"2024-05-08T05:51:05.073515+0000","last_became_active":"2024-05-08T05:51:05.070422+0000","last_became_peered":"2024-05-08T05:51:05.070422+0000","last_unstale":"2024-05-08T05:51:05.073515+0000","last_undegraded":"2024-05-08T05:51:05.073515+0000","last_fullsized":"2024-05-08T05:51:05.073515+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:29.725291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066591+0000","last_change":"2024-05-08T05:51:05.076336+0000","last_active":"2024-05-08T05:51:07.066591+0000","last_peered":"2024-05-08T05:51:07.066591+0000","last_clean":"2024-05-08T05:51:07.066591+0000","last_became_active":"2024-05-08T05:51:05.075648+0000","last_became_peered":"2024-05-08T05:51:05.075648+0000","last_unstale":"2024-05-08T05:51:07.066591+0000","last_undegraded":"2024-05-08T05:51:07.066591+0000","last_fullsized":"2024-05-08T05:51:07.066591+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:27.438502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088120+0000","last_change":"2024-05-08T05:51:05.079977+0000","last_active":"2024-05-08T05:51:05.088120+0000","last_peered":"2024-05-08T05:51:05.088120+0000","last_clean":"2024-05-08T05:51:05.088120+0000","last_became_active":"2024-05-08T05:51:05.078938+0000","last_became_peered":"2024-05-08T05:51:05.078938+0000","last_unstale":"2024-05-08T05:51:05.088120+0000","last_undegraded":"2024-05-08T05:51:05.088120+0000","last_fullsized":"2024-05-08T05:51:05.088120+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:00.252213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093291+0000","last_change":"2024-05-08T05:51:05.086072+0000","last_active":"2024-05-08T05:51:05.093291+0000","last_peered":"2024-05-08T05:51:05.093291+0000","last_clean":"2024-05-08T05:51:05.093291+0000","last_became_active":"2024-05-08T05:51:05.084475+0000","last_became_peered":"2024-05-08T05:51:05.084475+0000","last_unstale":"2024-05-08T05:51:05.093291+0000","last_undegraded":"2024-05-08T05:51:05.093291+0000","last_fullsized":"2024-05-08T05:51:05.093291+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:08:48.269564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088031+0000","last_change":"2024-05-08T05:51:05.083505+0000","last_active":"2024-05-08T05:51:05.088031+0000","last_peered":"2024-05-08T05:51:05.088031+0000","last_clean":"2024-05-08T05:51:05.088031+0000","last_became_active":"2024-05-08T05:51:05.083236+0000","last_became_peered":"2024-05-08T05:51:05.083236+0000","last_unstale":"2024-05-08T05:51:05.088031+0000","last_undegraded":"2024-05-08T05:51:05.088031+0000","last_fullsized":"2024-05-08T05:51:05.088031+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:20.213009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066209+0000","last_change":"2024-05-08T05:51:05.077087+0000","last_active":"2024-05-08T05:51:07.066209+0000","last_peered":"2024-05-08T05:51:07.066209+0000","last_clean":"2024-05-08T05:51:07.066209+0000","last_became_active":"2024-05-08T05:51:05.076743+0000","last_became_peered":"2024-05-08T05:51:05.076743+0000","last_unstale":"2024-05-08T05:51:07.066209+0000","last_undegraded":"2024-05-08T05:51:07.066209+0000","last_fullsized":"2024-05-08T05:51:07.066209+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:03.278751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088248+0000","last_change":"2024-05-08T05:51:05.081532+0000","last_active":"2024-05-08T05:51:05.088248+0000","last_peered":"2024-05-08T05:51:05.088248+0000","last_clean":"2024-05-08T05:51:05.088248+0000","last_became_active":"2024-05-08T05:51:05.081228+0000","last_became_peered":"2024-05-08T05:51:05.081228+0000","last_unstale":"2024-05-08T05:51:05.088248+0000","last_undegraded":"2024-05-08T05:51:05.088248+0000","last_fullsized":"2024-05-08T05:51:05.088248+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:38:53.162197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093376+0000","last_change":"2024-05-08T05:51:05.086943+0000","last_active":"2024-05-08T05:51:05.093376+0000","last_peered":"2024-05-08T05:51:05.093376+0000","last_clean":"2024-05-08T05:51:05.093376+0000","last_became_active":"2024-05-08T05:51:05.086221+0000","last_became_peered":"2024-05-08T05:51:05.086221+0000","last_unstale":"2024-05-08T05:51:05.093376+0000","last_undegraded":"2024-05-08T05:51:05.093376+0000","last_fullsized":"2024-05-08T05:51:05.093376+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:34:12.399560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.056046+0000","last_change":"2024-05-08T05:51:04.056046+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.056046+0000","last_undegraded":"2024-05-08T05:51:04.056046+0000","last_fullsized":"2024-05-08T05:51:04.056046+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078597+0000","last_change":"2024-05-08T05:51:05.080761+0000","last_active":"2024-05-08T05:51:06.078597+0000","last_peered":"2024-05-08T05:51:06.078597+0000","last_clean":"2024-05-08T05:51:06.078597+0000","last_became_active":"2024-05-08T05:51:05.080381+0000","last_became_peered":"2024-05-08T05:51:05.080381+0000","last_unstale":"2024-05-08T05:51:06.078597+0000","last_undegraded":"2024-05-08T05:51:06.078597+0000","last_fullsized":"2024-05-08T05:51:06.078597+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:42.792089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.051557+0000","last_change":"2024-05-08T05:51:04.051557+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.051557+0000","last_undegraded":"2024-05-08T05:51:04.051557+0000","last_fullsized":"2024-05-08T05:51:04.051557+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.084370+0000","last_change":"2024-05-08T05:51:05.079566+0000","last_active":"2024-05-08T05:51:05.084370+0000","last_peered":"2024-05-08T05:51:05.084370+0000","last_clean":"2024-05-08T05:51:05.084370+0000","last_became_active":"2024-05-08T05:51:05.079256+0000","last_became_peered":"2024-05-08T05:51:05.079256+0000","last_unstale":"2024-05-08T05:51:05.084370+0000","last_undegraded":"2024-05-08T05:51:05.084370+0000","last_fullsized":"2024-05-08T05:51:05.084370+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:51:07.851943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.050947+0000","last_change":"2024-05-08T05:51:04.050947+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.050947+0000","last_undegraded":"2024-05-08T05:51:04.050947+0000","last_fullsized":"2024-05-08T05:51:04.050947+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052838+0000","last_change":"2024-05-08T05:51:04.052838+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052838+0000","last_undegraded":"2024-05-08T05:51:04.052838+0000","last_fullsized":"2024-05-08T05:51:04.052838+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.053724+0000","last_change":"2024-05-08T05:51:04.053724+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.053724+0000","last_undegraded":"2024-05-08T05:51:04.053724+0000","last_fullsized":"2024-05-08T05:51:04.053724+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078673+0000","last_change":"2024-05-08T05:51:05.081486+0000","last_active":"2024-05-08T05:51:06.078673+0000","last_peered":"2024-05-08T05:51:06.078673+0000","last_clean":"2024-05-08T05:51:06.078673+0000","last_became_active":"2024-05-08T05:51:05.081143+0000","last_became_peered":"2024-05-08T05:51:05.081143+0000","last_unstale":"2024-05-08T05:51:06.078673+0000","last_undegraded":"2024-05-08T05:51:06.078673+0000","last_fullsized":"2024-05-08T05:51:06.078673+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:44:56.396392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.055539+0000","last_change":"2024-05-08T05:51:04.055539+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.055539+0000","last_undegraded":"2024-05-08T05:51:04.055539+0000","last_fullsized":"2024-05-08T05:51:04.055539+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066746+0000","last_change":"2024-05-08T05:51:05.081306+0000","last_active":"2024-05-08T05:51:07.066746+0000","last_peered":"2024-05-08T05:51:07.066746+0000","last_clean":"2024-05-08T05:51:07.066746+0000","last_became_active":"2024-05-08T05:51:05.081023+0000","last_became_peered":"2024-05-08T05:51:05.081023+0000","last_unstale":"2024-05-08T05:51:07.066746+0000","last_undegraded":"2024-05-08T05:51:07.066746+0000","last_fullsized":"2024-05-08T05:51:07.066746+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:20:04.797865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078729+0000","last_change":"2024-05-08T05:51:05.082212+0000","last_active":"2024-05-08T05:51:06.078729+0000","last_peered":"2024-05-08T05:51:06.078729+0000","last_clean":"2024-05-08T05:51:06.078729+0000","last_became_active":"2024-05-08T05:51:05.081847+0000","last_became_peered":"2024-05-08T05:51:05.081847+0000","last_unstale":"2024-05-08T05:51:06.078729+0000","last_undegraded":"2024-05-08T05:51:06.078729+0000","last_fullsized":"2024-05-08T05:51:06.078729+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:10.568249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093267+0000","last_change":"2024-05-08T05:51:05.085821+0000","last_active":"2024-05-08T05:51:05.093267+0000","last_peered":"2024-05-08T05:51:05.093267+0000","last_clean":"2024-05-08T05:51:05.093267+0000","last_became_active":"2024-05-08T05:51:05.083777+0000","last_became_peered":"2024-05-08T05:51:05.083777+0000","last_unstale":"2024-05-08T05:51:05.093267+0000","last_undegraded":"2024-05-08T05:51:05.093267+0000","last_fullsized":"2024-05-08T05:51:05.093267+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:22:46.959500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.051392+0000","last_change":"2024-05-08T05:51:04.051392+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.051392+0000","last_undegraded":"2024-05-08T05:51:04.051392+0000","last_fullsized":"2024-05-08T05:51:04.051392+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.050670+0000","last_change":"2024-05-08T05:51:04.050670+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.050670+0000","last_undegraded":"2024-05-08T05:51:04.050670+0000","last_fullsized":"2024-05-08T05:51:04.050670+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.076195+0000","last_change":"2024-05-08T05:51:05.072621+0000","last_active":"2024-05-08T05:51:05.076195+0000","last_peered":"2024-05-08T05:51:05.076195+0000","last_clean":"2024-05-08T05:51:05.076195+0000","last_became_active":"2024-05-08T05:51:05.072317+0000","last_became_peered":"2024-05-08T05:51:05.072317+0000","last_unstale":"2024-05-08T05:51:05.076195+0000","last_undegraded":"2024-05-08T05:51:05.076195+0000","last_fullsized":"2024-05-08T05:51:05.076195+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:05:09.239164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.064918+0000","last_change":"2024-05-08T05:51:04.063764+0000","last_active":"2024-05-08T05:51:04.064918+0000","last_peered":"2024-05-08T05:51:04.064918+0000","last_clean":"2024-05-08T05:51:04.064918+0000","last_became_active":"2024-05-08T05:51:04.063484+0000","last_became_peered":"2024-05-08T05:51:04.063484+0000","last_unstale":"2024-05-08T05:51:04.064918+0000","last_undegraded":"2024-05-08T05:51:04.064918+0000","last_fullsized":"2024-05-08T05:51:04.064918+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:17.541863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066655+0000","last_change":"2024-05-08T05:51:05.084520+0000","last_active":"2024-05-08T05:51:07.066655+0000","last_peered":"2024-05-08T05:51:07.066655+0000","last_clean":"2024-05-08T05:51:07.066655+0000","last_became_active":"2024-05-08T05:51:05.084198+0000","last_became_peered":"2024-05-08T05:51:05.084198+0000","last_unstale":"2024-05-08T05:51:07.066655+0000","last_undegraded":"2024-05-08T05:51:07.066655+0000","last_fullsized":"2024-05-08T05:51:07.066655+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:08:36.961131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.063174+0000","last_change":"2024-05-08T05:51:04.057409+0000","last_active":"2024-05-08T05:51:05.063174+0000","last_peered":"2024-05-08T05:51:05.063174+0000","last_clean":"2024-05-08T05:51:05.063174+0000","last_became_active":"2024-05-08T05:51:04.057127+0000","last_became_peered":"2024-05-08T05:51:04.057127+0000","last_unstale":"2024-05-08T05:51:05.063174+0000","last_undegraded":"2024-05-08T05:51:05.063174+0000","last_fullsized":"2024-05-08T05:51:05.063174+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:33.984066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052424+0000","last_change":"2024-05-08T05:51:04.052424+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052424+0000","last_undegraded":"2024-05-08T05:51:04.052424+0000","last_fullsized":"2024-05-08T05:51:04.052424+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.065887+0000","last_change":"2024-05-08T05:51:04.063770+0000","last_active":"2024-05-08T05:51:04.065887+0000","last_peered":"2024-05-08T05:51:04.065887+0000","last_clean":"2024-05-08T05:51:04.065887+0000","last_became_active":"2024-05-08T05:51:04.063471+0000","last_became_peered":"2024-05-08T05:51:04.063471+0000","last_unstale":"2024-05-08T05:51:04.065887+0000","last_undegraded":"2024-05-08T05:51:04.065887+0000","last_fullsized":"2024-05-08T05:51:04.065887+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:14:14.017450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.053316+0000","last_change":"2024-05-08T05:51:04.053316+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.053316+0000","last_undegraded":"2024-05-08T05:51:04.053316+0000","last_fullsized":"2024-05-08T05:51:04.053316+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.095608+0000","last_change":"2024-05-08T05:51:04.060756+0000","last_active":"2024-05-08T05:51:05.095608+0000","last_peered":"2024-05-08T05:51:05.095608+0000","last_clean":"2024-05-08T05:51:05.095608+0000","last_became_active":"2024-05-08T05:51:04.060480+0000","last_became_peered":"2024-05-08T05:51:04.060480+0000","last_unstale":"2024-05-08T05:51:05.095608+0000","last_undegraded":"2024-05-08T05:51:05.095608+0000","last_fullsized":"2024-05-08T05:51:05.095608+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:58:38.791116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.076073+0000","last_change":"2024-05-08T05:51:05.072513+0000","last_active":"2024-05-08T05:51:05.076073+0000","last_peered":"2024-05-08T05:51:05.076073+0000","last_clean":"2024-05-08T05:51:05.076073+0000","last_became_active":"2024-05-08T05:51:05.072191+0000","last_became_peered":"2024-05-08T05:51:05.072191+0000","last_unstale":"2024-05-08T05:51:05.076073+0000","last_undegraded":"2024-05-08T05:51:05.076073+0000","last_fullsized":"2024-05-08T05:51:05.076073+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:27.347106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.062425+0000","last_change":"2024-05-08T05:51:04.058250+0000","last_active":"2024-05-08T05:51:05.062425+0000","last_peered":"2024-05-08T05:51:05.062425+0000","last_clean":"2024-05-08T05:51:05.062425+0000","last_became_active":"2024-05-08T05:51:04.057765+0000","last_became_peered":"2024-05-08T05:51:04.057765+0000","last_unstale":"2024-05-08T05:51:05.062425+0000","last_undegraded":"2024-05-08T05:51:05.062425+0000","last_fullsized":"2024-05-08T05:51:05.062425+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:46:01.704267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.084362+0000","last_change":"2024-05-08T05:51:05.079919+0000","last_active":"2024-05-08T05:51:05.084362+0000","last_peered":"2024-05-08T05:51:05.084362+0000","last_clean":"2024-05-08T05:51:05.084362+0000","last_became_active":"2024-05-08T05:51:05.079552+0000","last_became_peered":"2024-05-08T05:51:05.079552+0000","last_unstale":"2024-05-08T05:51:05.084362+0000","last_undegraded":"2024-05-08T05:51:05.084362+0000","last_fullsized":"2024-05-08T05:51:05.084362+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:38:53.162197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066334+0000","last_change":"2024-05-08T05:51:04.069265+0000","last_active":"2024-05-08T05:51:07.066334+0000","last_peered":"2024-05-08T05:51:07.066334+0000","last_clean":"2024-05-08T05:51:07.066334+0000","last_became_active":"2024-05-08T05:51:04.068997+0000","last_became_peered":"2024-05-08T05:51:04.068997+0000","last_unstale":"2024-05-08T05:51:07.066334+0000","last_undegraded":"2024-05-08T05:51:07.066334+0000","last_fullsized":"2024-05-08T05:51:07.066334+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:55.959876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.084003+0000","last_change":"2024-05-08T05:51:05.079354+0000","last_active":"2024-05-08T05:51:05.084003+0000","last_peered":"2024-05-08T05:51:05.084003+0000","last_clean":"2024-05-08T05:51:05.084003+0000","last_became_active":"2024-05-08T05:51:05.079042+0000","last_became_peered":"2024-05-08T05:51:05.079042+0000","last_unstale":"2024-05-08T05:51:05.084003+0000","last_undegraded":"2024-05-08T05:51:05.084003+0000","last_fullsized":"2024-05-08T05:51:05.084003+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:20:04.111530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.065847+0000","last_change":"2024-05-08T05:51:04.058859+0000","last_active":"2024-05-08T05:51:04.065847+0000","last_peered":"2024-05-08T05:51:04.065847+0000","last_clean":"2024-05-08T05:51:04.065847+0000","last_became_active":"2024-05-08T05:51:04.058518+0000","last_became_peered":"2024-05-08T05:51:04.058518+0000","last_unstale":"2024-05-08T05:51:04.065847+0000","last_undegraded":"2024-05-08T05:51:04.065847+0000","last_fullsized":"2024-05-08T05:51:04.065847+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:02:28.473412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.087245+0000","last_change":"2024-05-08T05:51:05.077404+0000","last_active":"2024-05-08T05:51:05.087245+0000","last_peered":"2024-05-08T05:51:05.087245+0000","last_clean":"2024-05-08T05:51:05.087245+0000","last_became_active":"2024-05-08T05:51:05.077093+0000","last_became_peered":"2024-05-08T05:51:05.077093+0000","last_unstale":"2024-05-08T05:51:05.087245+0000","last_undegraded":"2024-05-08T05:51:05.087245+0000","last_fullsized":"2024-05-08T05:51:05.087245+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:12:02.641413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066301+0000","last_change":"2024-05-08T05:51:04.067859+0000","last_active":"2024-05-08T05:51:07.066301+0000","last_peered":"2024-05-08T05:51:07.066301+0000","last_clean":"2024-05-08T05:51:07.066301+0000","last_became_active":"2024-05-08T05:51:04.067586+0000","last_became_peered":"2024-05-08T05:51:04.067586+0000","last_unstale":"2024-05-08T05:51:07.066301+0000","last_undegraded":"2024-05-08T05:51:07.066301+0000","last_fullsized":"2024-05-08T05:51:07.066301+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:35:04.353051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.055148+0000","last_change":"2024-05-08T05:51:04.055148+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.055148+0000","last_undegraded":"2024-05-08T05:51:04.055148+0000","last_fullsized":"2024-05-08T05:51:04.055148+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.070847+0000","last_change":"2024-05-08T05:51:04.069859+0000","last_active":"2024-05-08T05:51:04.070847+0000","last_peered":"2024-05-08T05:51:04.070847+0000","last_clean":"2024-05-08T05:51:04.070847+0000","last_became_active":"2024-05-08T05:51:04.069575+0000","last_became_peered":"2024-05-08T05:51:04.069575+0000","last_unstale":"2024-05-08T05:51:04.070847+0000","last_undegraded":"2024-05-08T05:51:04.070847+0000","last_fullsized":"2024-05-08T05:51:04.070847+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:05:08.231174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.051301+0000","last_change":"2024-05-08T05:51:04.051301+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.051301+0000","last_undegraded":"2024-05-08T05:51:04.051301+0000","last_fullsized":"2024-05-08T05:51:04.051301+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.067379+0000","last_change":"2024-05-08T05:51:04.064523+0000","last_active":"2024-05-08T05:51:04.067379+0000","last_peered":"2024-05-08T05:51:04.067379+0000","last_clean":"2024-05-08T05:51:04.067379+0000","last_became_active":"2024-05-08T05:51:04.064236+0000","last_became_peered":"2024-05-08T05:51:04.064236+0000","last_unstale":"2024-05-08T05:51:04.067379+0000","last_undegraded":"2024-05-08T05:51:04.067379+0000","last_fullsized":"2024-05-08T05:51:04.067379+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:58:38.791116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093624+0000","last_change":"2024-05-08T05:51:05.090316+0000","last_active":"2024-05-08T05:51:05.093624+0000","last_peered":"2024-05-08T05:51:05.093624+0000","last_clean":"2024-05-08T05:51:05.093624+0000","last_became_active":"2024-05-08T05:51:05.084493+0000","last_became_peered":"2024-05-08T05:51:05.084493+0000","last_unstale":"2024-05-08T05:51:05.093624+0000","last_undegraded":"2024-05-08T05:51:05.093624+0000","last_fullsized":"2024-05-08T05:51:05.093624+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:16:24.027328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.066003+0000","last_change":"2024-05-08T05:51:04.059742+0000","last_active":"2024-05-08T05:51:04.066003+0000","last_peered":"2024-05-08T05:51:04.066003+0000","last_clean":"2024-05-08T05:51:04.066003+0000","last_became_active":"2024-05-08T05:51:04.059468+0000","last_became_peered":"2024-05-08T05:51:04.059468+0000","last_unstale":"2024-05-08T05:51:04.066003+0000","last_undegraded":"2024-05-08T05:51:04.066003+0000","last_fullsized":"2024-05-08T05:51:04.066003+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:35.193080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052749+0000","last_change":"2024-05-08T05:51:04.052749+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052749+0000","last_undegraded":"2024-05-08T05:51:04.052749+0000","last_fullsized":"2024-05-08T05:51:04.052749+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"118'6","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.101220+0000","last_change":"2024-05-08T05:51:04.066429+0000","last_active":"2024-05-08T05:51:05.101220+0000","last_peered":"2024-05-08T05:51:05.101220+0000","last_clean":"2024-05-08T05:51:05.101220+0000","last_became_active":"2024-05-08T05:51:04.065698+0000","last_became_peered":"2024-05-08T05:51:04.065698+0000","last_unstale":"2024-05-08T05:51:05.101220+0000","last_undegraded":"2024-05-08T05:51:05.101220+0000","last_fullsized":"2024-05-08T05:51:05.101220+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:28.960211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.054481+0000","last_change":"2024-05-08T05:51:04.054481+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.054481+0000","last_undegraded":"2024-05-08T05:51:04.054481+0000","last_fullsized":"2024-05-08T05:51:04.054481+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.094391+0000","last_change":"2024-05-08T05:51:04.067759+0000","last_active":"2024-05-08T05:51:05.094391+0000","last_peered":"2024-05-08T05:51:05.094391+0000","last_clean":"2024-05-08T05:51:05.094391+0000","last_became_active":"2024-05-08T05:51:04.067484+0000","last_became_peered":"2024-05-08T05:51:04.067484+0000","last_unstale":"2024-05-08T05:51:05.094391+0000","last_undegraded":"2024-05-08T05:51:05.094391+0000","last_fullsized":"2024-05-08T05:51:05.094391+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:44.431356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088282+0000","last_change":"2024-05-08T05:51:05.080720+0000","last_active":"2024-05-08T05:51:05.088282+0000","last_peered":"2024-05-08T05:51:05.088282+0000","last_clean":"2024-05-08T05:51:05.088282+0000","last_became_active":"2024-05-08T05:51:05.080428+0000","last_became_peered":"2024-05-08T05:51:05.080428+0000","last_unstale":"2024-05-08T05:51:05.088282+0000","last_undegraded":"2024-05-08T05:51:05.088282+0000","last_fullsized":"2024-05-08T05:51:05.088282+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:51:07.851943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.070039+0000","last_change":"2024-05-08T05:51:04.065712+0000","last_active":"2024-05-08T05:51:05.070039+0000","last_peered":"2024-05-08T05:51:05.070039+0000","last_clean":"2024-05-08T05:51:05.070039+0000","last_became_active":"2024-05-08T05:51:04.065421+0000","last_became_peered":"2024-05-08T05:51:04.065421+0000","last_unstale":"2024-05-08T05:51:05.070039+0000","last_undegraded":"2024-05-08T05:51:05.070039+0000","last_fullsized":"2024-05-08T05:51:05.070039+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:44:48.375896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078591+0000","last_change":"2024-05-08T05:51:05.084846+0000","last_active":"2024-05-08T05:51:06.078591+0000","last_peered":"2024-05-08T05:51:06.078591+0000","last_clean":"2024-05-08T05:51:06.078591+0000","last_became_active":"2024-05-08T05:51:05.084544+0000","last_became_peered":"2024-05-08T05:51:05.084544+0000","last_unstale":"2024-05-08T05:51:06.078591+0000","last_undegraded":"2024-05-08T05:51:06.078591+0000","last_fullsized":"2024-05-08T05:51:06.078591+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:30.552529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.068303+0000","last_change":"2024-05-08T05:51:04.067872+0000","last_active":"2024-05-08T05:51:05.068303+0000","last_peered":"2024-05-08T05:51:05.068303+0000","last_clean":"2024-05-08T05:51:05.068303+0000","last_became_active":"2024-05-08T05:51:04.067380+0000","last_became_peered":"2024-05-08T05:51:04.067380+0000","last_unstale":"2024-05-08T05:51:05.068303+0000","last_undegraded":"2024-05-08T05:51:05.068303+0000","last_fullsized":"2024-05-08T05:51:05.068303+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:21.223822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052885+0000","last_change":"2024-05-08T05:51:04.052885+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052885+0000","last_undegraded":"2024-05-08T05:51:04.052885+0000","last_fullsized":"2024-05-08T05:51:04.052885+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.063576+0000","last_change":"2024-05-08T05:51:04.070032+0000","last_active":"2024-05-08T05:51:05.063576+0000","last_peered":"2024-05-08T05:51:05.063576+0000","last_clean":"2024-05-08T05:51:05.063576+0000","last_became_active":"2024-05-08T05:51:04.069748+0000","last_became_peered":"2024-05-08T05:51:04.069748+0000","last_unstale":"2024-05-08T05:51:05.063576+0000","last_undegraded":"2024-05-08T05:51:05.063576+0000","last_fullsized":"2024-05-08T05:51:05.063576+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:46:01.704267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.084392+0000","last_change":"2024-05-08T05:51:05.078677+0000","last_active":"2024-05-08T05:51:05.084392+0000","last_peered":"2024-05-08T05:51:05.084392+0000","last_clean":"2024-05-08T05:51:05.084392+0000","last_became_active":"2024-05-08T05:51:05.078364+0000","last_became_peered":"2024-05-08T05:51:05.078364+0000","last_unstale":"2024-05-08T05:51:05.084392+0000","last_undegraded":"2024-05-08T05:51:05.084392+0000","last_fullsized":"2024-05-08T05:51:05.084392+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:00.252213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078677+0000","last_change":"2024-05-08T05:51:04.067612+0000","last_active":"2024-05-08T05:51:06.078677+0000","last_peered":"2024-05-08T05:51:06.078677+0000","last_clean":"2024-05-08T05:51:06.078677+0000","last_became_active":"2024-05-08T05:51:04.067268+0000","last_became_peered":"2024-05-08T05:51:04.067268+0000","last_unstale":"2024-05-08T05:51:06.078677+0000","last_undegraded":"2024-05-08T05:51:06.078677+0000","last_fullsized":"2024-05-08T05:51:06.078677+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:28:11.738033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066899+0000","last_change":"2024-05-08T05:51:05.073728+0000","last_active":"2024-05-08T05:51:07.066899+0000","last_peered":"2024-05-08T05:51:07.066899+0000","last_clean":"2024-05-08T05:51:07.066899+0000","last_became_active":"2024-05-08T05:51:05.073415+0000","last_became_peered":"2024-05-08T05:51:05.073415+0000","last_unstale":"2024-05-08T05:51:07.066899+0000","last_undegraded":"2024-05-08T05:51:07.066899+0000","last_fullsized":"2024-05-08T05:51:07.066899+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:52.612093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.069657+0000","last_change":"2024-05-08T05:51:04.066627+0000","last_active":"2024-05-08T05:51:04.069657+0000","last_peered":"2024-05-08T05:51:04.069657+0000","last_clean":"2024-05-08T05:51:04.069657+0000","last_became_active":"2024-05-08T05:51:04.066321+0000","last_became_peered":"2024-05-08T05:51:04.066321+0000","last_unstale":"2024-05-08T05:51:04.069657+0000","last_undegraded":"2024-05-08T05:51:04.069657+0000","last_fullsized":"2024-05-08T05:51:04.069657+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:54.640197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052213+0000","last_change":"2024-05-08T05:51:04.052213+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052213+0000","last_undegraded":"2024-05-08T05:51:04.052213+0000","last_fullsized":"2024-05-08T05:51:04.052213+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.072448+0000","last_change":"2024-05-08T05:51:04.072005+0000","last_active":"2024-05-08T05:51:04.072448+0000","last_peered":"2024-05-08T05:51:04.072448+0000","last_clean":"2024-05-08T05:51:04.072448+0000","last_became_active":"2024-05-08T05:51:04.071776+0000","last_became_peered":"2024-05-08T05:51:04.071776+0000","last_unstale":"2024-05-08T05:51:04.072448+0000","last_undegraded":"2024-05-08T05:51:04.072448+0000","last_fullsized":"2024-05-08T05:51:04.072448+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:21.223822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066166+0000","last_change":"2024-05-08T05:51:05.074599+0000","last_active":"2024-05-08T05:51:07.066166+0000","last_peered":"2024-05-08T05:51:07.066166+0000","last_clean":"2024-05-08T05:51:07.066166+0000","last_became_active":"2024-05-08T05:51:05.073942+0000","last_became_peered":"2024-05-08T05:51:05.073942+0000","last_unstale":"2024-05-08T05:51:07.066166+0000","last_undegraded":"2024-05-08T05:51:07.066166+0000","last_fullsized":"2024-05-08T05:51:07.066166+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:30.124962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066177+0000","last_change":"2024-05-08T05:51:04.069082+0000","last_active":"2024-05-08T05:51:07.066177+0000","last_peered":"2024-05-08T05:51:07.066177+0000","last_clean":"2024-05-08T05:51:07.066177+0000","last_became_active":"2024-05-08T05:51:04.068801+0000","last_became_peered":"2024-05-08T05:51:04.068801+0000","last_unstale":"2024-05-08T05:51:07.066177+0000","last_undegraded":"2024-05-08T05:51:07.066177+0000","last_fullsized":"2024-05-08T05:51:07.066177+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:57:45.457442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.092747+0000","last_change":"2024-05-08T05:51:05.083611+0000","last_active":"2024-05-08T05:51:05.092747+0000","last_peered":"2024-05-08T05:51:05.092747+0000","last_clean":"2024-05-08T05:51:05.092747+0000","last_became_active":"2024-05-08T05:51:05.083328+0000","last_became_peered":"2024-05-08T05:51:05.083328+0000","last_unstale":"2024-05-08T05:51:05.092747+0000","last_undegraded":"2024-05-08T05:51:05.092747+0000","last_fullsized":"2024-05-08T05:51:05.092747+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:55.131091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.065846+0000","last_change":"2024-05-08T05:51:04.063770+0000","last_active":"2024-05-08T05:51:04.065846+0000","last_peered":"2024-05-08T05:51:04.065846+0000","last_clean":"2024-05-08T05:51:04.065846+0000","last_became_active":"2024-05-08T05:51:04.063509+0000","last_became_peered":"2024-05-08T05:51:04.063509+0000","last_unstale":"2024-05-08T05:51:04.065846+0000","last_undegraded":"2024-05-08T05:51:04.065846+0000","last_fullsized":"2024-05-08T05:51:04.065846+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:49:24.745045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093539+0000","last_change":"2024-05-08T05:51:05.090152+0000","last_active":"2024-05-08T05:51:05.093539+0000","last_peered":"2024-05-08T05:51:05.093539+0000","last_clean":"2024-05-08T05:51:05.093539+0000","last_became_active":"2024-05-08T05:51:05.089828+0000","last_became_peered":"2024-05-08T05:51:05.089828+0000","last_unstale":"2024-05-08T05:51:05.093539+0000","last_undegraded":"2024-05-08T05:51:05.093539+0000","last_fullsized":"2024-05-08T05:51:05.093539+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:34:19.612717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.069013+0000","last_change":"2024-05-08T05:51:04.069833+0000","last_active":"2024-05-08T05:51:05.069013+0000","last_peered":"2024-05-08T05:51:05.069013+0000","last_clean":"2024-05-08T05:51:05.069013+0000","last_became_active":"2024-05-08T05:51:04.069539+0000","last_became_peered":"2024-05-08T05:51:04.069539+0000","last_unstale":"2024-05-08T05:51:05.069013+0000","last_undegraded":"2024-05-08T05:51:05.069013+0000","last_fullsized":"2024-05-08T05:51:05.069013+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:36:49.817010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.052495+0000","last_change":"2024-05-08T05:51:04.052495+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.052495+0000","last_undegraded":"2024-05-08T05:51:04.052495+0000","last_fullsized":"2024-05-08T05:51:04.052495+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.065891+0000","last_change":"2024-05-08T05:51:04.063987+0000","last_active":"2024-05-08T05:51:04.065891+0000","last_peered":"2024-05-08T05:51:04.065891+0000","last_clean":"2024-05-08T05:51:04.065891+0000","last_became_active":"2024-05-08T05:51:04.063646+0000","last_became_peered":"2024-05-08T05:51:04.063646+0000","last_unstale":"2024-05-08T05:51:04.065891+0000","last_undegraded":"2024-05-08T05:51:04.065891+0000","last_fullsized":"2024-05-08T05:51:04.065891+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:58:37.220465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066817+0000","last_change":"2024-05-08T05:51:05.075930+0000","last_active":"2024-05-08T05:51:07.066817+0000","last_peered":"2024-05-08T05:51:07.066817+0000","last_clean":"2024-05-08T05:51:07.066817+0000","last_became_active":"2024-05-08T05:51:05.075052+0000","last_became_peered":"2024-05-08T05:51:05.075052+0000","last_unstale":"2024-05-08T05:51:07.066817+0000","last_undegraded":"2024-05-08T05:51:07.066817+0000","last_fullsized":"2024-05-08T05:51:07.066817+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:27:28.653248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066775+0000","last_change":"2024-05-08T05:51:04.063919+0000","last_active":"2024-05-08T05:51:07.066775+0000","last_peered":"2024-05-08T05:51:07.066775+0000","last_clean":"2024-05-08T05:51:07.066775+0000","last_became_active":"2024-05-08T05:51:04.063610+0000","last_became_peered":"2024-05-08T05:51:04.063610+0000","last_unstale":"2024-05-08T05:51:07.066775+0000","last_undegraded":"2024-05-08T05:51:07.066775+0000","last_fullsized":"2024-05-08T05:51:07.066775+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:26.855446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.076159+0000","last_change":"2024-05-08T05:51:05.073177+0000","last_active":"2024-05-08T05:51:05.076159+0000","last_peered":"2024-05-08T05:51:05.076159+0000","last_clean":"2024-05-08T05:51:05.076159+0000","last_became_active":"2024-05-08T05:51:05.072104+0000","last_became_peered":"2024-05-08T05:51:05.072104+0000","last_unstale":"2024-05-08T05:51:05.076159+0000","last_undegraded":"2024-05-08T05:51:05.076159+0000","last_fullsized":"2024-05-08T05:51:05.076159+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:29.725291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.070325+0000","last_change":"2024-05-08T05:51:04.069690+0000","last_active":"2024-05-08T05:51:04.070325+0000","last_peered":"2024-05-08T05:51:04.070325+0000","last_clean":"2024-05-08T05:51:04.070325+0000","last_became_active":"2024-05-08T05:51:04.069380+0000","last_became_peered":"2024-05-08T05:51:04.069380+0000","last_unstale":"2024-05-08T05:51:04.070325+0000","last_undegraded":"2024-05-08T05:51:04.070325+0000","last_fullsized":"2024-05-08T05:51:04.070325+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:26.339116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.053043+0000","last_change":"2024-05-08T05:51:04.053043+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.053043+0000","last_undegraded":"2024-05-08T05:51:04.053043+0000","last_fullsized":"2024-05-08T05:51:04.053043+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.070852+0000","last_change":"2024-05-08T05:51:04.070027+0000","last_active":"2024-05-08T05:51:04.070852+0000","last_peered":"2024-05-08T05:51:04.070852+0000","last_clean":"2024-05-08T05:51:04.070852+0000","last_became_active":"2024-05-08T05:51:04.069331+0000","last_became_peered":"2024-05-08T05:51:04.069331+0000","last_unstale":"2024-05-08T05:51:04.070852+0000","last_undegraded":"2024-05-08T05:51:04.070852+0000","last_fullsized":"2024-05-08T05:51:04.070852+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:28.717301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.079017+0000","last_change":"2024-05-08T05:51:05.079710+0000","last_active":"2024-05-08T05:51:06.079017+0000","last_peered":"2024-05-08T05:51:06.079017+0000","last_clean":"2024-05-08T05:51:06.079017+0000","last_became_active":"2024-05-08T05:51:05.079405+0000","last_became_peered":"2024-05-08T05:51:05.079405+0000","last_unstale":"2024-05-08T05:51:06.079017+0000","last_undegraded":"2024-05-08T05:51:06.079017+0000","last_fullsized":"2024-05-08T05:51:06.079017+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:57:15.216780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066813+0000","last_change":"2024-05-08T05:51:04.066485+0000","last_active":"2024-05-08T05:51:07.066813+0000","last_peered":"2024-05-08T05:51:07.066813+0000","last_clean":"2024-05-08T05:51:07.066813+0000","last_became_active":"2024-05-08T05:51:04.066192+0000","last_became_peered":"2024-05-08T05:51:04.066192+0000","last_unstale":"2024-05-08T05:51:07.066813+0000","last_undegraded":"2024-05-08T05:51:07.066813+0000","last_fullsized":"2024-05-08T05:51:07.066813+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:54:27.233544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066370+0000","last_change":"2024-05-08T05:51:05.074917+0000","last_active":"2024-05-08T05:51:07.066370+0000","last_peered":"2024-05-08T05:51:07.066370+0000","last_clean":"2024-05-08T05:51:07.066370+0000","last_became_active":"2024-05-08T05:51:05.074339+0000","last_became_peered":"2024-05-08T05:51:05.074339+0000","last_unstale":"2024-05-08T05:51:07.066370+0000","last_undegraded":"2024-05-08T05:51:07.066370+0000","last_fullsized":"2024-05-08T05:51:07.066370+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:22:04.619439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.062382+0000","last_change":"2024-05-08T05:51:04.057289+0000","last_active":"2024-05-08T05:51:05.062382+0000","last_peered":"2024-05-08T05:51:05.062382+0000","last_clean":"2024-05-08T05:51:05.062382+0000","last_became_active":"2024-05-08T05:51:04.056950+0000","last_became_peered":"2024-05-08T05:51:04.056950+0000","last_unstale":"2024-05-08T05:51:05.062382+0000","last_undegraded":"2024-05-08T05:51:05.062382+0000","last_fullsized":"2024-05-08T05:51:05.062382+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:17.541863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.079392+0000","last_change":"2024-05-08T05:51:05.076808+0000","last_active":"2024-05-08T05:51:05.079392+0000","last_peered":"2024-05-08T05:51:05.079392+0000","last_clean":"2024-05-08T05:51:05.079392+0000","last_became_active":"2024-05-08T05:51:05.076532+0000","last_became_peered":"2024-05-08T05:51:05.076532+0000","last_unstale":"2024-05-08T05:51:05.079392+0000","last_undegraded":"2024-05-08T05:51:05.079392+0000","last_fullsized":"2024-05-08T05:51:05.079392+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:12.526046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.062570+0000","last_change":"2024-05-08T05:51:04.054314+0000","last_active":"2024-05-08T05:51:05.062570+0000","last_peered":"2024-05-08T05:51:05.062570+0000","last_clean":"2024-05-08T05:51:05.062570+0000","last_became_active":"2024-05-08T05:51:04.053754+0000","last_became_peered":"2024-05-08T05:51:04.053754+0000","last_unstale":"2024-05-08T05:51:05.062570+0000","last_undegraded":"2024-05-08T05:51:05.062570+0000","last_fullsized":"2024-05-08T05:51:05.062570+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:26.339116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.053473+0000","last_change":"2024-05-08T05:51:04.053473+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.053473+0000","last_undegraded":"2024-05-08T05:51:04.053473+0000","last_fullsized":"2024-05-08T05:51:04.053473+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066301+0000","last_change":"2024-05-08T05:51:04.064412+0000","last_active":"2024-05-08T05:51:07.066301+0000","last_peered":"2024-05-08T05:51:07.066301+0000","last_clean":"2024-05-08T05:51:07.066301+0000","last_became_active":"2024-05-08T05:51:04.064134+0000","last_became_peered":"2024-05-08T05:51:04.064134+0000","last_unstale":"2024-05-08T05:51:07.066301+0000","last_undegraded":"2024-05-08T05:51:07.066301+0000","last_fullsized":"2024-05-08T05:51:07.066301+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:37:09.043074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088681+0000","last_change":"2024-05-08T05:51:05.087096+0000","last_active":"2024-05-08T05:51:05.088681+0000","last_peered":"2024-05-08T05:51:05.088681+0000","last_clean":"2024-05-08T05:51:05.088681+0000","last_became_active":"2024-05-08T05:51:05.085170+0000","last_became_peered":"2024-05-08T05:51:05.085170+0000","last_unstale":"2024-05-08T05:51:05.088681+0000","last_undegraded":"2024-05-08T05:51:05.088681+0000","last_fullsized":"2024-05-08T05:51:05.088681+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:20.213009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.066642+0000","last_change":"2024-05-08T05:51:04.065454+0000","last_active":"2024-05-08T05:51:04.066642+0000","last_peered":"2024-05-08T05:51:04.066642+0000","last_clean":"2024-05-08T05:51:04.066642+0000","last_became_active":"2024-05-08T05:51:04.065175+0000","last_became_peered":"2024-05-08T05:51:04.065175+0000","last_unstale":"2024-05-08T05:51:04.066642+0000","last_undegraded":"2024-05-08T05:51:04.066642+0000","last_fullsized":"2024-05-08T05:51:04.066642+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:01:19.756722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-05-08T05:51:04.051048+0000","last_change":"2024-05-08T05:51:04.051048+0000","last_active":"2024-05-08T05:51:04.041342+0000","last_peered":"2024-05-08T05:51:04.041342+0000","last_clean":"2024-05-08T05:51:04.041342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-05-08T05:51:04.051048+0000","last_undegraded":"2024-05-08T05:51:04.051048+0000","last_fullsized":"2024-05-08T05:51:04.051048+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"118'8","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.096717+0000","last_change":"2024-05-08T05:51:04.068279+0000","last_active":"2024-05-08T05:51:05.096717+0000","last_peered":"2024-05-08T05:51:05.096717+0000","last_clean":"2024-05-08T05:51:05.096717+0000","last_became_active":"2024-05-08T05:51:04.067953+0000","last_became_peered":"2024-05-08T05:51:04.067953+0000","last_unstale":"2024-05-08T05:51:05.096717+0000","last_undegraded":"2024-05-08T05:51:05.096717+0000","last_fullsized":"2024-05-08T05:51:05.096717+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:54.640197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":288,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-05-08T05:51:04.055816+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:51:04.055816+0000","last_peered":"2024-05-08T05:51:04.055816+0000","last_clean":"2024-05-08T05:51:04.055816+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:51:04.055816+0000","last_undegraded":"2024-05-08T05:51:04.055816+0000","last_fullsized":"2024-05-08T05:51:04.055816+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":1668,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":3302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542265,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2048000,"data_stored":1266440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542266,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1454080,"data_stored":676072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542265,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1454080,"data_stored":676072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":684851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1482752,"data_stored":684885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1478656,"data_stored":689931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1470464,"data_stored":683261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2076672,"data_stored":1280321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:51:13.623 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:51:13.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.739+0000 7f9e49912640 1 Processor -- start 2024-05-08T05:51:13.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.739+0000 7f9e49912640 1 -- start start 2024-05-08T05:51:13.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.739+0000 7f9e49912640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9e440d20c0 0x7f9e44163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:13.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.739+0000 7f9e49912640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e440d1300 con 0x7f9e44071110 2024-05-08T05:51:13.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.739+0000 7f9e49912640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e440d15e0 con 0x7f9e44070d30 2024-05-08T05:51:13.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.739+0000 7f9e49912640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f9e440d10d0 con 0x7f9e440d20c0 2024-05-08T05:51:13.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.740+0000 7f9e427fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9e440d20c0 0x7f9e44163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:13.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.740+0000 7f9e427fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9e440d20c0 0x7f9e44163920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45658/0 (socket says 172.21.15.45:45658) 2024-05-08T05:51:13.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.740+0000 7f9e437fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9e44071110 0x7f9e4406d4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40520/0 (socket says 172.21.15.45:40520) 2024-05-08T05:51:13.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.740+0000 7f9e427fc640 1 -- 172.21.15.45:0/2848574295 learned_addr learned my addr 172.21.15.45:0/2848574295 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:13.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.740+0000 7f9e427fc640 1 -- 172.21.15.45:0/2848574295 >> v1:172.21.15.63:6789/0 conn(0x7f9e44070d30 legacy=0x7f9e440703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:13.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.740+0000 7f9e427fc640 1 -- 172.21.15.45:0/2848574295 >> v1:172.21.15.45:6789/0 conn(0x7f9e44071110 legacy=0x7f9e4406d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:13.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.740+0000 7f9e427fc640 1 -- 172.21.15.45:0/2848574295 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e440d18c0 con 0x7f9e440d20c0 2024-05-08T05:51:13.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.740+0000 7f9e41ffb640 1 -- 172.21.15.45:0/2848574295 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1937269316 0 0) 0x7f9e440d1300 con 0x7f9e44071110 2024-05-08T05:51:13.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.741+0000 7f9e427fc640 1 --2- 172.21.15.45:0/2848574295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e440d20c0 0x7f9e44163920 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009870 tx=0x7f9e2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bad058209d68236 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:13.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.741+0000 7f9e41ffb640 1 -- 172.21.15.45:0/2848574295 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c037070 con 0x7f9e440d20c0 2024-05-08T05:51:13.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.741+0000 7f9e41ffb640 1 -- 172.21.15.45:0/2848574295 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e2c032440 con 0x7f9e440d20c0 2024-05-08T05:51:13.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.741+0000 7f9e41ffb640 1 -- 172.21.15.45:0/2848574295 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c032810 con 0x7f9e440d20c0 2024-05-08T05:51:13.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.741+0000 7f9e49912640 1 -- 172.21.15.45:0/2848574295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e440d20c0 msgr2=0x7f9e44163920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:13.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.741+0000 7f9e49912640 1 --2- 172.21.15.45:0/2848574295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e440d20c0 0x7f9e44163920 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009870 tx=0x7f9e2c02a270 comp rx=0 tx=0).stop 2024-05-08T05:51:13.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.742+0000 7f9e49912640 1 -- 172.21.15.45:0/2848574295 shutdown_connections 2024-05-08T05:51:13.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.742+0000 7f9e49912640 1 --2- 172.21.15.45:0/2848574295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e440d20c0 0x7f9e44163920 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:13.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.742+0000 7f9e49912640 1 -- 172.21.15.45:0/2848574295 >> 172.21.15.45:0/2848574295 conn(0x7f9e440d2e90 msgr2=0x7f9e440d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:13.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.742+0000 7f9e49912640 1 -- 172.21.15.45:0/2848574295 shutdown_connections 2024-05-08T05:51:13.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.742+0000 7f9e49912640 1 -- 172.21.15.45:0/2848574295 wait complete. 2024-05-08T05:51:13.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.743+0000 7f9e49912640 1 Processor -- start 2024-05-08T05:51:13.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.743+0000 7f9e49912640 1 -- start start 2024-05-08T05:51:13.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.744+0000 7f9e49912640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e44070d30 0x7f9e440b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:13.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.744+0000 7f9e49912640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e44071110 0x7f9e440b3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:13.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.744+0000 7f9e49912640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e440d20c0 0x7f9e4415ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:13.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.744+0000 7f9e49912640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9e440d1c30 con 0x7f9e440d20c0 2024-05-08T05:51:13.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.744+0000 7f9e49912640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9e44070b80 con 0x7f9e44070d30 2024-05-08T05:51:13.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.744+0000 7f9e49912640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9e4406dbc0 con 0x7f9e44071110 2024-05-08T05:51:13.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.744+0000 7f9e437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e440d20c0 0x7f9e4415ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:13.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.744+0000 7f9e437fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e440d20c0 0x7f9e4415ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45660/0 (socket says 172.21.15.45:45660) 2024-05-08T05:51:13.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.744+0000 7f9e437fe640 1 -- 172.21.15.45:0/1510000264 learned_addr learned my addr 172.21.15.45:0/1510000264 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:13.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.744+0000 7f9e42ffd640 1 --2- 172.21.15.45:0/1510000264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e44070d30 0x7f9e440b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:13.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.744+0000 7f9e427fc640 1 --2- 172.21.15.45:0/1510000264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e44071110 0x7f9e440b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:13.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.744+0000 7f9e437fe640 1 -- 172.21.15.45:0/1510000264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e44071110 msgr2=0x7f9e440b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:13.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.744+0000 7f9e437fe640 1 --2- 172.21.15.45:0/1510000264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e44071110 0x7f9e440b3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:13.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.744+0000 7f9e437fe640 1 -- 172.21.15.45:0/1510000264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e44070d30 msgr2=0x7f9e440b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:13.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.744+0000 7f9e437fe640 1 --2- 172.21.15.45:0/1510000264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e44070d30 0x7f9e440b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:13.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.744+0000 7f9e437fe640 1 -- 172.21.15.45:0/1510000264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e440b3900 con 0x7f9e440d20c0 2024-05-08T05:51:13.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.745+0000 7f9e437fe640 1 --2- 172.21.15.45:0/1510000264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e440d20c0 0x7f9e4415ca30 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400c7d0 tx=0x7f9e3400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:13.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.745+0000 7f9e3bfff640 1 -- 172.21.15.45:0/1510000264 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e34018070 con 0x7f9e440d20c0 2024-05-08T05:51:13.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.745+0000 7f9e3bfff640 1 -- 172.21.15.45:0/1510000264 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e3400a5e0 con 0x7f9e440d20c0 2024-05-08T05:51:13.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.745+0000 7f9e49912640 1 -- 172.21.15.45:0/1510000264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e440b3be0 con 0x7f9e440d20c0 2024-05-08T05:51:13.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.745+0000 7f9e3bfff640 1 -- 172.21.15.45:0/1510000264 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3400aa10 con 0x7f9e440d20c0 2024-05-08T05:51:13.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.745+0000 7f9e49912640 1 -- 172.21.15.45:0/1510000264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e44153cc0 con 0x7f9e440d20c0 2024-05-08T05:51:13.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.746+0000 7f9e49912640 1 -- 172.21.15.45:0/1510000264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e440b4100 con 0x7f9e440d20c0 2024-05-08T05:51:13.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.746+0000 7f9e3bfff640 1 -- 172.21.15.45:0/1510000264 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9e3401b070 con 0x7f9e440d20c0 2024-05-08T05:51:13.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.747+0000 7f9e3bfff640 1 --2- 172.21.15.45:0/1510000264 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9e240762d0 0x7f9e24078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:13.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.747+0000 7f9e3bfff640 1 -- 172.21.15.45:0/1510000264 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f9e34096d90 con 0x7f9e440d20c0 2024-05-08T05:51:13.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.747+0000 7f9e42ffd640 1 --2- 172.21.15.45:0/1510000264 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9e240762d0 0x7f9e24078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:13.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.748+0000 7f9e42ffd640 1 --2- 172.21.15.45:0/1510000264 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9e240762d0 0x7f9e24078790 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c004ea0 tx=0x7f9e3c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:13.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.752+0000 7f9e3bfff640 1 -- 172.21.15.45:0/1510000264 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9e440b4100 con 0x7f9e440d20c0 2024-05-08T05:51:13.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.939+0000 7f9e49912640 1 -- 172.21.15.45:0/1510000264 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e4406fc30 con 0x7f9e240762d0 2024-05-08T05:51:13.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.947+0000 7f9e3bfff640 1 -- 172.21.15.45:0/1510000264 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252449 (secure 0 0 0) 0x7f9e4406fc30 con 0x7f9e240762d0 2024-05-08T05:51:13.950 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:13.951 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:51:13.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.949+0000 7f9e49912640 1 -- 172.21.15.45:0/1510000264 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9e240762d0 msgr2=0x7f9e24078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:13.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.949+0000 7f9e49912640 1 --2- 172.21.15.45:0/1510000264 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9e240762d0 0x7f9e24078790 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c004ea0 tx=0x7f9e3c009040 comp rx=0 tx=0).stop 2024-05-08T05:51:13.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.949+0000 7f9e49912640 1 -- 172.21.15.45:0/1510000264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e440d20c0 msgr2=0x7f9e4415ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:13.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.949+0000 7f9e49912640 1 --2- 172.21.15.45:0/1510000264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e440d20c0 0x7f9e4415ca30 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400c7d0 tx=0x7f9e3400cc90 comp rx=0 tx=0).stop 2024-05-08T05:51:13.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.949+0000 7f9e49912640 1 -- 172.21.15.45:0/1510000264 shutdown_connections 2024-05-08T05:51:13.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.949+0000 7f9e49912640 1 --2- 172.21.15.45:0/1510000264 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9e240762d0 0x7f9e24078790 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:13.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.949+0000 7f9e49912640 1 --2- 172.21.15.45:0/1510000264 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9e44071110 0x7f9e440b3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:13.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.949+0000 7f9e49912640 1 --2- 172.21.15.45:0/1510000264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9e440d20c0 0x7f9e4415ca30 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:13.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.949+0000 7f9e49912640 1 --2- 172.21.15.45:0/1510000264 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e44070d30 0x7f9e440b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:13.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.949+0000 7f9e49912640 1 -- 172.21.15.45:0/1510000264 >> 172.21.15.45:0/1510000264 conn(0x7f9e440d2e90 msgr2=0x7f9e441542d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:13.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.949+0000 7f9e49912640 1 -- 172.21.15.45:0/1510000264 shutdown_connections 2024-05-08T05:51:13.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:13.949+0000 7f9e49912640 1 -- 172.21.15.45:0/1510000264 wait complete. 2024-05-08T05:51:13.961 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-05-08T05:51:12.975899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262212,"kb_used_data":12740,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712508,"statfs":{"total":773094113280,"available":772825608192,"internally_reserved":0,"allocated":13045760,"data_stored":6684928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002011"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.073391+0000","last_change":"2024-05-08T05:51:05.069476+0000","last_active":"2024-05-08T05:51:05.073391+0000","last_peered":"2024-05-08T05:51:05.073391+0000","last_clean":"2024-05-08T05:51:05.073391+0000","last_became_active":"2024-05-08T05:51:05.069131+0000","last_became_peered":"2024-05-08T05:51:05.069131+0000","last_unstale":"2024-05-08T05:51:05.073391+0000","last_undegraded":"2024-05-08T05:51:05.073391+0000","last_fullsized":"2024-05-08T05:51:05.073391+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:05:09.239164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093231+0000","last_change":"2024-05-08T05:51:05.084139+0000","last_active":"2024-05-08T05:51:05.093231+0000","last_peered":"2024-05-08T05:51:05.093231+0000","last_clean":"2024-05-08T05:51:05.093231+0000","last_became_active":"2024-05-08T05:51:05.081830+0000","last_became_peered":"2024-05-08T05:51:05.081830+0000","last_unstale":"2024-05-08T05:51:05.093231+0000","last_undegraded":"2024-05-08T05:51:05.093231+0000","last_fullsized":"2024-05-08T05:51:05.093231+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:37.715909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093594+0000","last_change":"2024-05-08T05:51:05.090876+0000","last_active":"2024-05-08T05:51:05.093594+0000","last_peered":"2024-05-08T05:51:05.093594+0000","last_clean":"2024-05-08T05:51:05.093594+0000","last_became_active":"2024-05-08T05:51:05.089329+0000","last_became_peered":"2024-05-08T05:51:05.089329+0000","last_unstale":"2024-05-08T05:51:05.093594+0000","last_undegraded":"2024-05-08T05:51:05.093594+0000","last_fullsized":"2024-05-08T05:51:05.093594+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:51:43.350274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.561508+0000","last_change":"2024-05-08T05:51:05.085405+0000","last_active":"2024-05-08T05:51:06.561508+0000","last_peered":"2024-05-08T05:51:06.561508+0000","last_clean":"2024-05-08T05:51:06.561508+0000","last_became_active":"2024-05-08T05:51:05.085053+0000","last_became_peered":"2024-05-08T05:51:05.085053+0000","last_unstale":"2024-05-08T05:51:06.561508+0000","last_undegraded":"2024-05-08T05:51:06.561508+0000","last_fullsized":"2024-05-08T05:51:06.561508+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:24:05.256088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.077282+0000","last_change":"2024-05-08T05:51:05.072296+0000","last_active":"2024-05-08T05:51:05.077282+0000","last_peered":"2024-05-08T05:51:05.077282+0000","last_clean":"2024-05-08T05:51:05.077282+0000","last_became_active":"2024-05-08T05:51:05.072009+0000","last_became_peered":"2024-05-08T05:51:05.072009+0000","last_unstale":"2024-05-08T05:51:05.077282+0000","last_undegraded":"2024-05-08T05:51:05.077282+0000","last_fullsized":"2024-05-08T05:51:05.077282+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:12:02.641413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066407+0000","last_change":"2024-05-08T05:51:05.082847+0000","last_active":"2024-05-08T05:51:07.066407+0000","last_peered":"2024-05-08T05:51:07.066407+0000","last_clean":"2024-05-08T05:51:07.066407+0000","last_became_active":"2024-05-08T05:51:05.082267+0000","last_became_peered":"2024-05-08T05:51:05.082267+0000","last_unstale":"2024-05-08T05:51:07.066407+0000","last_undegraded":"2024-05-08T05:51:07.066407+0000","last_fullsized":"2024-05-08T05:51:07.066407+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:16:17.521040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066220+0000","last_change":"2024-05-08T05:51:05.087173+0000","last_active":"2024-05-08T05:51:07.066220+0000","last_peered":"2024-05-08T05:51:07.066220+0000","last_clean":"2024-05-08T05:51:07.066220+0000","last_became_active":"2024-05-08T05:51:05.086914+0000","last_became_peered":"2024-05-08T05:51:05.086914+0000","last_unstale":"2024-05-08T05:51:07.066220+0000","last_undegraded":"2024-05-08T05:51:07.066220+0000","last_fullsized":"2024-05-08T05:51:07.066220+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:42:24.823847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093326+0000","last_change":"2024-05-08T05:51:05.085921+0000","last_active":"2024-05-08T05:51:05.093326+0000","last_peered":"2024-05-08T05:51:05.093326+0000","last_clean":"2024-05-08T05:51:05.093326+0000","last_became_active":"2024-05-08T05:51:05.083800+0000","last_became_peered":"2024-05-08T05:51:05.083800+0000","last_unstale":"2024-05-08T05:51:05.093326+0000","last_undegraded":"2024-05-08T05:51:05.093326+0000","last_fullsized":"2024-05-08T05:51:05.093326+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:05:47.453010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.561893+0000","last_change":"2024-05-08T05:51:05.084941+0000","last_active":"2024-05-08T05:51:06.561893+0000","last_peered":"2024-05-08T05:51:06.561893+0000","last_clean":"2024-05-08T05:51:06.561893+0000","last_became_active":"2024-05-08T05:51:05.084309+0000","last_became_peered":"2024-05-08T05:51:05.084309+0000","last_unstale":"2024-05-08T05:51:06.561893+0000","last_undegraded":"2024-05-08T05:51:06.561893+0000","last_fullsized":"2024-05-08T05:51:06.561893+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:08.030369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.075989+0000","last_change":"2024-05-08T05:51:05.071111+0000","last_active":"2024-05-08T05:51:05.075989+0000","last_peered":"2024-05-08T05:51:05.075989+0000","last_clean":"2024-05-08T05:51:05.075989+0000","last_became_active":"2024-05-08T05:51:05.070823+0000","last_became_peered":"2024-05-08T05:51:05.070823+0000","last_unstale":"2024-05-08T05:51:05.075989+0000","last_undegraded":"2024-05-08T05:51:05.075989+0000","last_fullsized":"2024-05-08T05:51:05.075989+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:18.549853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.073515+0000","last_change":"2024-05-08T05:51:05.070718+0000","last_active":"2024-05-08T05:51:05.073515+0000","last_peered":"2024-05-08T05:51:05.073515+0000","last_clean":"2024-05-08T05:51:05.073515+0000","last_became_active":"2024-05-08T05:51:05.070422+0000","last_became_peered":"2024-05-08T05:51:05.070422+0000","last_unstale":"2024-05-08T05:51:05.073515+0000","last_undegraded":"2024-05-08T05:51:05.073515+0000","last_fullsized":"2024-05-08T05:51:05.073515+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:29.725291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066591+0000","last_change":"2024-05-08T05:51:05.076336+0000","last_active":"2024-05-08T05:51:07.066591+0000","last_peered":"2024-05-08T05:51:07.066591+0000","last_clean":"2024-05-08T05:51:07.066591+0000","last_became_active":"2024-05-08T05:51:05.075648+0000","last_became_peered":"2024-05-08T05:51:05.075648+0000","last_unstale":"2024-05-08T05:51:07.066591+0000","last_undegraded":"2024-05-08T05:51:07.066591+0000","last_fullsized":"2024-05-08T05:51:07.066591+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:27.438502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088120+0000","last_change":"2024-05-08T05:51:05.079977+0000","last_active":"2024-05-08T05:51:05.088120+0000","last_peered":"2024-05-08T05:51:05.088120+0000","last_clean":"2024-05-08T05:51:05.088120+0000","last_became_active":"2024-05-08T05:51:05.078938+0000","last_became_peered":"2024-05-08T05:51:05.078938+0000","last_unstale":"2024-05-08T05:51:05.088120+0000","last_undegraded":"2024-05-08T05:51:05.088120+0000","last_fullsized":"2024-05-08T05:51:05.088120+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:00.252213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093291+0000","last_change":"2024-05-08T05:51:05.086072+0000","last_active":"2024-05-08T05:51:05.093291+0000","last_peered":"2024-05-08T05:51:05.093291+0000","last_clean":"2024-05-08T05:51:05.093291+0000","last_became_active":"2024-05-08T05:51:05.084475+0000","last_became_peered":"2024-05-08T05:51:05.084475+0000","last_unstale":"2024-05-08T05:51:05.093291+0000","last_undegraded":"2024-05-08T05:51:05.093291+0000","last_fullsized":"2024-05-08T05:51:05.093291+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:08:48.269564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088031+0000","last_change":"2024-05-08T05:51:05.083505+0000","last_active":"2024-05-08T05:51:05.088031+0000","last_peered":"2024-05-08T05:51:05.088031+0000","last_clean":"2024-05-08T05:51:05.088031+0000","last_became_active":"2024-05-08T05:51:05.083236+0000","last_became_peered":"2024-05-08T05:51:05.083236+0000","last_unstale":"2024-05-08T05:51:05.088031+0000","last_undegraded":"2024-05-08T05:51:05.088031+0000","last_fullsized":"2024-05-08T05:51:05.088031+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:20.213009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066209+0000","last_change":"2024-05-08T05:51:05.077087+0000","last_active":"2024-05-08T05:51:07.066209+0000","last_peered":"2024-05-08T05:51:07.066209+0000","last_clean":"2024-05-08T05:51:07.066209+0000","last_became_active":"2024-05-08T05:51:05.076743+0000","last_became_peered":"2024-05-08T05:51:05.076743+0000","last_unstale":"2024-05-08T05:51:07.066209+0000","last_undegraded":"2024-05-08T05:51:07.066209+0000","last_fullsized":"2024-05-08T05:51:07.066209+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:38:03.278751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088248+0000","last_change":"2024-05-08T05:51:05.081532+0000","last_active":"2024-05-08T05:51:05.088248+0000","last_peered":"2024-05-08T05:51:05.088248+0000","last_clean":"2024-05-08T05:51:05.088248+0000","last_became_active":"2024-05-08T05:51:05.081228+0000","last_became_peered":"2024-05-08T05:51:05.081228+0000","last_unstale":"2024-05-08T05:51:05.088248+0000","last_undegraded":"2024-05-08T05:51:05.088248+0000","last_fullsized":"2024-05-08T05:51:05.088248+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:38:53.162197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093376+0000","last_change":"2024-05-08T05:51:05.086943+0000","last_active":"2024-05-08T05:51:05.093376+0000","last_peered":"2024-05-08T05:51:05.093376+0000","last_clean":"2024-05-08T05:51:05.093376+0000","last_became_active":"2024-05-08T05:51:05.086221+0000","last_became_peered":"2024-05-08T05:51:05.086221+0000","last_unstale":"2024-05-08T05:51:05.093376+0000","last_undegraded":"2024-05-08T05:51:05.093376+0000","last_fullsized":"2024-05-08T05:51:05.093376+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:34:12.399560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.083730+0000","last_change":"2024-05-08T05:51:05.077701+0000","last_active":"2024-05-08T05:51:06.083730+0000","last_peered":"2024-05-08T05:51:06.083730+0000","last_clean":"2024-05-08T05:51:06.083730+0000","last_became_active":"2024-05-08T05:51:05.077416+0000","last_became_peered":"2024-05-08T05:51:05.077416+0000","last_unstale":"2024-05-08T05:51:06.083730+0000","last_undegraded":"2024-05-08T05:51:06.083730+0000","last_fullsized":"2024-05-08T05:51:06.083730+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:45.439346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078597+0000","last_change":"2024-05-08T05:51:05.080761+0000","last_active":"2024-05-08T05:51:06.078597+0000","last_peered":"2024-05-08T05:51:06.078597+0000","last_clean":"2024-05-08T05:51:06.078597+0000","last_became_active":"2024-05-08T05:51:05.080381+0000","last_became_peered":"2024-05-08T05:51:05.080381+0000","last_unstale":"2024-05-08T05:51:06.078597+0000","last_undegraded":"2024-05-08T05:51:06.078597+0000","last_fullsized":"2024-05-08T05:51:06.078597+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:42.792089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.561491+0000","last_change":"2024-05-08T05:51:05.088902+0000","last_active":"2024-05-08T05:51:06.561491+0000","last_peered":"2024-05-08T05:51:06.561491+0000","last_clean":"2024-05-08T05:51:06.561491+0000","last_became_active":"2024-05-08T05:51:05.088619+0000","last_became_peered":"2024-05-08T05:51:05.088619+0000","last_unstale":"2024-05-08T05:51:06.561491+0000","last_undegraded":"2024-05-08T05:51:06.561491+0000","last_fullsized":"2024-05-08T05:51:06.561491+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:04.219768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.084370+0000","last_change":"2024-05-08T05:51:05.079566+0000","last_active":"2024-05-08T05:51:05.084370+0000","last_peered":"2024-05-08T05:51:05.084370+0000","last_clean":"2024-05-08T05:51:05.084370+0000","last_became_active":"2024-05-08T05:51:05.079256+0000","last_became_peered":"2024-05-08T05:51:05.079256+0000","last_unstale":"2024-05-08T05:51:05.084370+0000","last_undegraded":"2024-05-08T05:51:05.084370+0000","last_fullsized":"2024-05-08T05:51:05.084370+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:51:07.851943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.561520+0000","last_change":"2024-05-08T05:51:05.080989+0000","last_active":"2024-05-08T05:51:06.561520+0000","last_peered":"2024-05-08T05:51:06.561520+0000","last_clean":"2024-05-08T05:51:06.561520+0000","last_became_active":"2024-05-08T05:51:05.080716+0000","last_became_peered":"2024-05-08T05:51:05.080716+0000","last_unstale":"2024-05-08T05:51:06.561520+0000","last_undegraded":"2024-05-08T05:51:06.561520+0000","last_fullsized":"2024-05-08T05:51:06.561520+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:06:19.692304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.347800+0000","last_change":"2024-05-08T05:51:05.076925+0000","last_active":"2024-05-08T05:51:07.347800+0000","last_peered":"2024-05-08T05:51:07.347800+0000","last_clean":"2024-05-08T05:51:07.347800+0000","last_became_active":"2024-05-08T05:51:05.076630+0000","last_became_peered":"2024-05-08T05:51:05.076630+0000","last_unstale":"2024-05-08T05:51:07.347800+0000","last_undegraded":"2024-05-08T05:51:07.347800+0000","last_fullsized":"2024-05-08T05:51:07.347800+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:22.231812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.348281+0000","last_change":"2024-05-08T05:51:05.082567+0000","last_active":"2024-05-08T05:51:07.348281+0000","last_peered":"2024-05-08T05:51:07.348281+0000","last_clean":"2024-05-08T05:51:07.348281+0000","last_became_active":"2024-05-08T05:51:05.082154+0000","last_became_peered":"2024-05-08T05:51:05.082154+0000","last_unstale":"2024-05-08T05:51:07.348281+0000","last_undegraded":"2024-05-08T05:51:07.348281+0000","last_fullsized":"2024-05-08T05:51:07.348281+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:12.526046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078673+0000","last_change":"2024-05-08T05:51:05.081486+0000","last_active":"2024-05-08T05:51:06.078673+0000","last_peered":"2024-05-08T05:51:06.078673+0000","last_clean":"2024-05-08T05:51:06.078673+0000","last_became_active":"2024-05-08T05:51:05.081143+0000","last_became_peered":"2024-05-08T05:51:05.081143+0000","last_unstale":"2024-05-08T05:51:06.078673+0000","last_undegraded":"2024-05-08T05:51:06.078673+0000","last_fullsized":"2024-05-08T05:51:06.078673+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:44:56.396392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.083756+0000","last_change":"2024-05-08T05:51:05.076935+0000","last_active":"2024-05-08T05:51:06.083756+0000","last_peered":"2024-05-08T05:51:06.083756+0000","last_clean":"2024-05-08T05:51:06.083756+0000","last_became_active":"2024-05-08T05:51:05.076660+0000","last_became_peered":"2024-05-08T05:51:05.076660+0000","last_unstale":"2024-05-08T05:51:06.083756+0000","last_undegraded":"2024-05-08T05:51:06.083756+0000","last_fullsized":"2024-05-08T05:51:06.083756+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:36:50.825000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066746+0000","last_change":"2024-05-08T05:51:05.081306+0000","last_active":"2024-05-08T05:51:07.066746+0000","last_peered":"2024-05-08T05:51:07.066746+0000","last_clean":"2024-05-08T05:51:07.066746+0000","last_became_active":"2024-05-08T05:51:05.081023+0000","last_became_peered":"2024-05-08T05:51:05.081023+0000","last_unstale":"2024-05-08T05:51:07.066746+0000","last_undegraded":"2024-05-08T05:51:07.066746+0000","last_fullsized":"2024-05-08T05:51:07.066746+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:20:04.797865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078729+0000","last_change":"2024-05-08T05:51:05.082212+0000","last_active":"2024-05-08T05:51:06.078729+0000","last_peered":"2024-05-08T05:51:06.078729+0000","last_clean":"2024-05-08T05:51:06.078729+0000","last_became_active":"2024-05-08T05:51:05.081847+0000","last_became_peered":"2024-05-08T05:51:05.081847+0000","last_unstale":"2024-05-08T05:51:06.078729+0000","last_undegraded":"2024-05-08T05:51:06.078729+0000","last_fullsized":"2024-05-08T05:51:06.078729+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:10.568249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093267+0000","last_change":"2024-05-08T05:51:05.085821+0000","last_active":"2024-05-08T05:51:05.093267+0000","last_peered":"2024-05-08T05:51:05.093267+0000","last_clean":"2024-05-08T05:51:05.093267+0000","last_became_active":"2024-05-08T05:51:05.083777+0000","last_became_peered":"2024-05-08T05:51:05.083777+0000","last_unstale":"2024-05-08T05:51:05.093267+0000","last_undegraded":"2024-05-08T05:51:05.093267+0000","last_fullsized":"2024-05-08T05:51:05.093267+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:22:46.959500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.347965+0000","last_change":"2024-05-08T05:51:05.078612+0000","last_active":"2024-05-08T05:51:07.347965+0000","last_peered":"2024-05-08T05:51:07.347965+0000","last_clean":"2024-05-08T05:51:07.347965+0000","last_became_active":"2024-05-08T05:51:05.078288+0000","last_became_peered":"2024-05-08T05:51:05.078288+0000","last_unstale":"2024-05-08T05:51:07.347965+0000","last_undegraded":"2024-05-08T05:51:07.347965+0000","last_fullsized":"2024-05-08T05:51:07.347965+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:55.648187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.347799+0000","last_change":"2024-05-08T05:51:05.077654+0000","last_active":"2024-05-08T05:51:07.347799+0000","last_peered":"2024-05-08T05:51:07.347799+0000","last_clean":"2024-05-08T05:51:07.347799+0000","last_became_active":"2024-05-08T05:51:05.077381+0000","last_became_peered":"2024-05-08T05:51:05.077381+0000","last_unstale":"2024-05-08T05:51:07.347799+0000","last_undegraded":"2024-05-08T05:51:07.347799+0000","last_fullsized":"2024-05-08T05:51:07.347799+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:01:20.764712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.076195+0000","last_change":"2024-05-08T05:51:05.072621+0000","last_active":"2024-05-08T05:51:05.076195+0000","last_peered":"2024-05-08T05:51:05.076195+0000","last_clean":"2024-05-08T05:51:05.076195+0000","last_became_active":"2024-05-08T05:51:05.072317+0000","last_became_peered":"2024-05-08T05:51:05.072317+0000","last_unstale":"2024-05-08T05:51:05.076195+0000","last_undegraded":"2024-05-08T05:51:05.076195+0000","last_fullsized":"2024-05-08T05:51:05.076195+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:05:09.239164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"118'4","reported_seq":18,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.083258+0000","last_change":"2024-05-08T05:51:04.063764+0000","last_active":"2024-05-08T05:51:06.083258+0000","last_peered":"2024-05-08T05:51:06.083258+0000","last_clean":"2024-05-08T05:51:06.083258+0000","last_became_active":"2024-05-08T05:51:04.063484+0000","last_became_peered":"2024-05-08T05:51:04.063484+0000","last_unstale":"2024-05-08T05:51:06.083258+0000","last_undegraded":"2024-05-08T05:51:06.083258+0000","last_fullsized":"2024-05-08T05:51:06.083258+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:13:59.244223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066655+0000","last_change":"2024-05-08T05:51:05.084520+0000","last_active":"2024-05-08T05:51:07.066655+0000","last_peered":"2024-05-08T05:51:07.066655+0000","last_clean":"2024-05-08T05:51:07.066655+0000","last_became_active":"2024-05-08T05:51:05.084198+0000","last_became_peered":"2024-05-08T05:51:05.084198+0000","last_unstale":"2024-05-08T05:51:07.066655+0000","last_undegraded":"2024-05-08T05:51:07.066655+0000","last_fullsized":"2024-05-08T05:51:07.066655+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:08:36.961131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.063174+0000","last_change":"2024-05-08T05:51:04.057409+0000","last_active":"2024-05-08T05:51:05.063174+0000","last_peered":"2024-05-08T05:51:05.063174+0000","last_clean":"2024-05-08T05:51:05.063174+0000","last_became_active":"2024-05-08T05:51:04.057127+0000","last_became_peered":"2024-05-08T05:51:04.057127+0000","last_unstale":"2024-05-08T05:51:05.063174+0000","last_undegraded":"2024-05-08T05:51:05.063174+0000","last_fullsized":"2024-05-08T05:51:05.063174+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:33.984066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.348107+0000","last_change":"2024-05-08T05:51:05.079626+0000","last_active":"2024-05-08T05:51:07.348107+0000","last_peered":"2024-05-08T05:51:07.348107+0000","last_clean":"2024-05-08T05:51:07.348107+0000","last_became_active":"2024-05-08T05:51:05.079329+0000","last_became_peered":"2024-05-08T05:51:05.079329+0000","last_unstale":"2024-05-08T05:51:07.348107+0000","last_undegraded":"2024-05-08T05:51:07.348107+0000","last_fullsized":"2024-05-08T05:51:07.348107+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:36:50.825000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.561714+0000","last_change":"2024-05-08T05:51:04.063770+0000","last_active":"2024-05-08T05:51:06.561714+0000","last_peered":"2024-05-08T05:51:06.561714+0000","last_clean":"2024-05-08T05:51:06.561714+0000","last_became_active":"2024-05-08T05:51:04.063471+0000","last_became_peered":"2024-05-08T05:51:04.063471+0000","last_unstale":"2024-05-08T05:51:06.561714+0000","last_undegraded":"2024-05-08T05:51:06.561714+0000","last_fullsized":"2024-05-08T05:51:06.561714+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:11.518056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.561891+0000","last_change":"2024-05-08T05:51:05.089687+0000","last_active":"2024-05-08T05:51:06.561891+0000","last_peered":"2024-05-08T05:51:06.561891+0000","last_clean":"2024-05-08T05:51:06.561891+0000","last_became_active":"2024-05-08T05:51:05.089358+0000","last_became_peered":"2024-05-08T05:51:05.089358+0000","last_unstale":"2024-05-08T05:51:06.561891+0000","last_undegraded":"2024-05-08T05:51:06.561891+0000","last_fullsized":"2024-05-08T05:51:06.561891+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:11:54.376923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.095608+0000","last_change":"2024-05-08T05:51:04.060756+0000","last_active":"2024-05-08T05:51:05.095608+0000","last_peered":"2024-05-08T05:51:05.095608+0000","last_clean":"2024-05-08T05:51:05.095608+0000","last_became_active":"2024-05-08T05:51:04.060480+0000","last_became_peered":"2024-05-08T05:51:04.060480+0000","last_unstale":"2024-05-08T05:51:05.095608+0000","last_undegraded":"2024-05-08T05:51:05.095608+0000","last_fullsized":"2024-05-08T05:51:05.095608+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:58:38.791116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.076073+0000","last_change":"2024-05-08T05:51:05.072513+0000","last_active":"2024-05-08T05:51:05.076073+0000","last_peered":"2024-05-08T05:51:05.076073+0000","last_clean":"2024-05-08T05:51:05.076073+0000","last_became_active":"2024-05-08T05:51:05.072191+0000","last_became_peered":"2024-05-08T05:51:05.072191+0000","last_unstale":"2024-05-08T05:51:05.076073+0000","last_undegraded":"2024-05-08T05:51:05.076073+0000","last_fullsized":"2024-05-08T05:51:05.076073+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:27.347106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.062425+0000","last_change":"2024-05-08T05:51:04.058250+0000","last_active":"2024-05-08T05:51:05.062425+0000","last_peered":"2024-05-08T05:51:05.062425+0000","last_clean":"2024-05-08T05:51:05.062425+0000","last_became_active":"2024-05-08T05:51:04.057765+0000","last_became_peered":"2024-05-08T05:51:04.057765+0000","last_unstale":"2024-05-08T05:51:05.062425+0000","last_undegraded":"2024-05-08T05:51:05.062425+0000","last_fullsized":"2024-05-08T05:51:05.062425+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:46:01.704267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.084362+0000","last_change":"2024-05-08T05:51:05.079919+0000","last_active":"2024-05-08T05:51:05.084362+0000","last_peered":"2024-05-08T05:51:05.084362+0000","last_clean":"2024-05-08T05:51:05.084362+0000","last_became_active":"2024-05-08T05:51:05.079552+0000","last_became_peered":"2024-05-08T05:51:05.079552+0000","last_unstale":"2024-05-08T05:51:05.084362+0000","last_undegraded":"2024-05-08T05:51:05.084362+0000","last_fullsized":"2024-05-08T05:51:05.084362+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:38:53.162197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066334+0000","last_change":"2024-05-08T05:51:04.069265+0000","last_active":"2024-05-08T05:51:07.066334+0000","last_peered":"2024-05-08T05:51:07.066334+0000","last_clean":"2024-05-08T05:51:07.066334+0000","last_became_active":"2024-05-08T05:51:04.068997+0000","last_became_peered":"2024-05-08T05:51:04.068997+0000","last_unstale":"2024-05-08T05:51:07.066334+0000","last_undegraded":"2024-05-08T05:51:07.066334+0000","last_fullsized":"2024-05-08T05:51:07.066334+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:55.959876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.084003+0000","last_change":"2024-05-08T05:51:05.079354+0000","last_active":"2024-05-08T05:51:05.084003+0000","last_peered":"2024-05-08T05:51:05.084003+0000","last_clean":"2024-05-08T05:51:05.084003+0000","last_became_active":"2024-05-08T05:51:05.079042+0000","last_became_peered":"2024-05-08T05:51:05.079042+0000","last_unstale":"2024-05-08T05:51:05.084003+0000","last_undegraded":"2024-05-08T05:51:05.084003+0000","last_fullsized":"2024-05-08T05:51:05.084003+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:20:04.111530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.348007+0000","last_change":"2024-05-08T05:51:04.058859+0000","last_active":"2024-05-08T05:51:07.348007+0000","last_peered":"2024-05-08T05:51:07.348007+0000","last_clean":"2024-05-08T05:51:07.348007+0000","last_became_active":"2024-05-08T05:51:04.058518+0000","last_became_peered":"2024-05-08T05:51:04.058518+0000","last_unstale":"2024-05-08T05:51:07.348007+0000","last_undegraded":"2024-05-08T05:51:07.348007+0000","last_fullsized":"2024-05-08T05:51:07.348007+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:17.541863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.087245+0000","last_change":"2024-05-08T05:51:05.077404+0000","last_active":"2024-05-08T05:51:05.087245+0000","last_peered":"2024-05-08T05:51:05.087245+0000","last_clean":"2024-05-08T05:51:05.087245+0000","last_became_active":"2024-05-08T05:51:05.077093+0000","last_became_peered":"2024-05-08T05:51:05.077093+0000","last_unstale":"2024-05-08T05:51:05.087245+0000","last_undegraded":"2024-05-08T05:51:05.087245+0000","last_fullsized":"2024-05-08T05:51:05.087245+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:12:02.641413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066301+0000","last_change":"2024-05-08T05:51:04.067859+0000","last_active":"2024-05-08T05:51:07.066301+0000","last_peered":"2024-05-08T05:51:07.066301+0000","last_clean":"2024-05-08T05:51:07.066301+0000","last_became_active":"2024-05-08T05:51:04.067586+0000","last_became_peered":"2024-05-08T05:51:04.067586+0000","last_unstale":"2024-05-08T05:51:07.066301+0000","last_undegraded":"2024-05-08T05:51:07.066301+0000","last_fullsized":"2024-05-08T05:51:07.066301+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:35:04.353051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.083704+0000","last_change":"2024-05-08T05:51:05.078481+0000","last_active":"2024-05-08T05:51:06.083704+0000","last_peered":"2024-05-08T05:51:06.083704+0000","last_clean":"2024-05-08T05:51:06.083704+0000","last_became_active":"2024-05-08T05:51:05.078205+0000","last_became_peered":"2024-05-08T05:51:05.078205+0000","last_unstale":"2024-05-08T05:51:06.083704+0000","last_undegraded":"2024-05-08T05:51:06.083704+0000","last_fullsized":"2024-05-08T05:51:06.083704+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:22.231812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.083394+0000","last_change":"2024-05-08T05:51:04.069859+0000","last_active":"2024-05-08T05:51:06.083394+0000","last_peered":"2024-05-08T05:51:06.083394+0000","last_clean":"2024-05-08T05:51:06.083394+0000","last_became_active":"2024-05-08T05:51:04.069575+0000","last_became_peered":"2024-05-08T05:51:04.069575+0000","last_unstale":"2024-05-08T05:51:06.083394+0000","last_undegraded":"2024-05-08T05:51:06.083394+0000","last_fullsized":"2024-05-08T05:51:06.083394+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:51:06.843953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.348125+0000","last_change":"2024-05-08T05:51:05.084952+0000","last_active":"2024-05-08T05:51:07.348125+0000","last_peered":"2024-05-08T05:51:07.348125+0000","last_clean":"2024-05-08T05:51:07.348125+0000","last_became_active":"2024-05-08T05:51:05.084626+0000","last_became_peered":"2024-05-08T05:51:05.084626+0000","last_unstale":"2024-05-08T05:51:07.348125+0000","last_undegraded":"2024-05-08T05:51:07.348125+0000","last_fullsized":"2024-05-08T05:51:07.348125+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:29.968201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.347979+0000","last_change":"2024-05-08T05:51:04.064523+0000","last_active":"2024-05-08T05:51:07.347979+0000","last_peered":"2024-05-08T05:51:07.347979+0000","last_clean":"2024-05-08T05:51:07.347979+0000","last_became_active":"2024-05-08T05:51:04.064236+0000","last_became_peered":"2024-05-08T05:51:04.064236+0000","last_unstale":"2024-05-08T05:51:07.347979+0000","last_undegraded":"2024-05-08T05:51:07.347979+0000","last_fullsized":"2024-05-08T05:51:07.347979+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:46:01.704267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093624+0000","last_change":"2024-05-08T05:51:05.090316+0000","last_active":"2024-05-08T05:51:05.093624+0000","last_peered":"2024-05-08T05:51:05.093624+0000","last_clean":"2024-05-08T05:51:05.093624+0000","last_became_active":"2024-05-08T05:51:05.084493+0000","last_became_peered":"2024-05-08T05:51:05.084493+0000","last_unstale":"2024-05-08T05:51:05.093624+0000","last_undegraded":"2024-05-08T05:51:05.093624+0000","last_fullsized":"2024-05-08T05:51:05.093624+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:16:24.027328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"118'3","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.348115+0000","last_change":"2024-05-08T05:51:04.059742+0000","last_active":"2024-05-08T05:51:07.348115+0000","last_peered":"2024-05-08T05:51:07.348115+0000","last_clean":"2024-05-08T05:51:07.348115+0000","last_became_active":"2024-05-08T05:51:04.059468+0000","last_became_peered":"2024-05-08T05:51:04.059468+0000","last_unstale":"2024-05-08T05:51:07.348115+0000","last_undegraded":"2024-05-08T05:51:07.348115+0000","last_fullsized":"2024-05-08T05:51:07.348115+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:33.984066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.348387+0000","last_change":"2024-05-08T05:51:05.078549+0000","last_active":"2024-05-08T05:51:07.348387+0000","last_peered":"2024-05-08T05:51:07.348387+0000","last_clean":"2024-05-08T05:51:07.348387+0000","last_became_active":"2024-05-08T05:51:05.078230+0000","last_became_peered":"2024-05-08T05:51:05.078230+0000","last_unstale":"2024-05-08T05:51:07.348387+0000","last_undegraded":"2024-05-08T05:51:07.348387+0000","last_fullsized":"2024-05-08T05:51:07.348387+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:20:04.111530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"118'6","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.101220+0000","last_change":"2024-05-08T05:51:04.066429+0000","last_active":"2024-05-08T05:51:05.101220+0000","last_peered":"2024-05-08T05:51:05.101220+0000","last_clean":"2024-05-08T05:51:05.101220+0000","last_became_active":"2024-05-08T05:51:04.065698+0000","last_became_peered":"2024-05-08T05:51:04.065698+0000","last_unstale":"2024-05-08T05:51:05.101220+0000","last_undegraded":"2024-05-08T05:51:05.101220+0000","last_fullsized":"2024-05-08T05:51:05.101220+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:28.960211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.083467+0000","last_change":"2024-05-08T05:51:05.080999+0000","last_active":"2024-05-08T05:51:06.083467+0000","last_peered":"2024-05-08T05:51:06.083467+0000","last_clean":"2024-05-08T05:51:06.083467+0000","last_became_active":"2024-05-08T05:51:05.080604+0000","last_became_peered":"2024-05-08T05:51:05.080604+0000","last_unstale":"2024-05-08T05:51:06.083467+0000","last_undegraded":"2024-05-08T05:51:06.083467+0000","last_fullsized":"2024-05-08T05:51:06.083467+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:55.648187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.094391+0000","last_change":"2024-05-08T05:51:04.067759+0000","last_active":"2024-05-08T05:51:05.094391+0000","last_peered":"2024-05-08T05:51:05.094391+0000","last_clean":"2024-05-08T05:51:05.094391+0000","last_became_active":"2024-05-08T05:51:04.067484+0000","last_became_peered":"2024-05-08T05:51:04.067484+0000","last_unstale":"2024-05-08T05:51:05.094391+0000","last_undegraded":"2024-05-08T05:51:05.094391+0000","last_fullsized":"2024-05-08T05:51:05.094391+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:44.431356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088282+0000","last_change":"2024-05-08T05:51:05.080720+0000","last_active":"2024-05-08T05:51:05.088282+0000","last_peered":"2024-05-08T05:51:05.088282+0000","last_clean":"2024-05-08T05:51:05.088282+0000","last_became_active":"2024-05-08T05:51:05.080428+0000","last_became_peered":"2024-05-08T05:51:05.080428+0000","last_unstale":"2024-05-08T05:51:05.088282+0000","last_undegraded":"2024-05-08T05:51:05.088282+0000","last_fullsized":"2024-05-08T05:51:05.088282+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:51:07.851943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.070039+0000","last_change":"2024-05-08T05:51:04.065712+0000","last_active":"2024-05-08T05:51:05.070039+0000","last_peered":"2024-05-08T05:51:05.070039+0000","last_clean":"2024-05-08T05:51:05.070039+0000","last_became_active":"2024-05-08T05:51:04.065421+0000","last_became_peered":"2024-05-08T05:51:04.065421+0000","last_unstale":"2024-05-08T05:51:05.070039+0000","last_undegraded":"2024-05-08T05:51:05.070039+0000","last_fullsized":"2024-05-08T05:51:05.070039+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:44:48.375896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078591+0000","last_change":"2024-05-08T05:51:05.084846+0000","last_active":"2024-05-08T05:51:06.078591+0000","last_peered":"2024-05-08T05:51:06.078591+0000","last_clean":"2024-05-08T05:51:06.078591+0000","last_became_active":"2024-05-08T05:51:05.084544+0000","last_became_peered":"2024-05-08T05:51:05.084544+0000","last_unstale":"2024-05-08T05:51:06.078591+0000","last_undegraded":"2024-05-08T05:51:06.078591+0000","last_fullsized":"2024-05-08T05:51:06.078591+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:30.552529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.068303+0000","last_change":"2024-05-08T05:51:04.067872+0000","last_active":"2024-05-08T05:51:05.068303+0000","last_peered":"2024-05-08T05:51:05.068303+0000","last_clean":"2024-05-08T05:51:05.068303+0000","last_became_active":"2024-05-08T05:51:04.067380+0000","last_became_peered":"2024-05-08T05:51:04.067380+0000","last_unstale":"2024-05-08T05:51:05.068303+0000","last_undegraded":"2024-05-08T05:51:05.068303+0000","last_fullsized":"2024-05-08T05:51:05.068303+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:21.223822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.348208+0000","last_change":"2024-05-08T05:51:05.081155+0000","last_active":"2024-05-08T05:51:07.348208+0000","last_peered":"2024-05-08T05:51:07.348208+0000","last_clean":"2024-05-08T05:51:07.348208+0000","last_became_active":"2024-05-08T05:51:05.080832+0000","last_became_peered":"2024-05-08T05:51:05.080832+0000","last_unstale":"2024-05-08T05:51:07.348208+0000","last_undegraded":"2024-05-08T05:51:07.348208+0000","last_fullsized":"2024-05-08T05:51:07.348208+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:44:49.383886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.063576+0000","last_change":"2024-05-08T05:51:04.070032+0000","last_active":"2024-05-08T05:51:05.063576+0000","last_peered":"2024-05-08T05:51:05.063576+0000","last_clean":"2024-05-08T05:51:05.063576+0000","last_became_active":"2024-05-08T05:51:04.069748+0000","last_became_peered":"2024-05-08T05:51:04.069748+0000","last_unstale":"2024-05-08T05:51:05.063576+0000","last_undegraded":"2024-05-08T05:51:05.063576+0000","last_fullsized":"2024-05-08T05:51:05.063576+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:46:01.704267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.084392+0000","last_change":"2024-05-08T05:51:05.078677+0000","last_active":"2024-05-08T05:51:05.084392+0000","last_peered":"2024-05-08T05:51:05.084392+0000","last_clean":"2024-05-08T05:51:05.084392+0000","last_became_active":"2024-05-08T05:51:05.078364+0000","last_became_peered":"2024-05-08T05:51:05.078364+0000","last_unstale":"2024-05-08T05:51:05.084392+0000","last_undegraded":"2024-05-08T05:51:05.084392+0000","last_fullsized":"2024-05-08T05:51:05.084392+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:14:00.252213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.078677+0000","last_change":"2024-05-08T05:51:04.067612+0000","last_active":"2024-05-08T05:51:06.078677+0000","last_peered":"2024-05-08T05:51:06.078677+0000","last_clean":"2024-05-08T05:51:06.078677+0000","last_became_active":"2024-05-08T05:51:04.067268+0000","last_became_peered":"2024-05-08T05:51:04.067268+0000","last_unstale":"2024-05-08T05:51:06.078677+0000","last_undegraded":"2024-05-08T05:51:06.078677+0000","last_fullsized":"2024-05-08T05:51:06.078677+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:28:11.738033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066899+0000","last_change":"2024-05-08T05:51:05.073728+0000","last_active":"2024-05-08T05:51:07.066899+0000","last_peered":"2024-05-08T05:51:07.066899+0000","last_clean":"2024-05-08T05:51:07.066899+0000","last_became_active":"2024-05-08T05:51:05.073415+0000","last_became_peered":"2024-05-08T05:51:05.073415+0000","last_unstale":"2024-05-08T05:51:07.066899+0000","last_undegraded":"2024-05-08T05:51:07.066899+0000","last_fullsized":"2024-05-08T05:51:07.066899+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:23:52.612093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.561768+0000","last_change":"2024-05-08T05:51:04.066627+0000","last_active":"2024-05-08T05:51:06.561768+0000","last_peered":"2024-05-08T05:51:06.561768+0000","last_clean":"2024-05-08T05:51:06.561768+0000","last_became_active":"2024-05-08T05:51:04.066321+0000","last_became_peered":"2024-05-08T05:51:04.066321+0000","last_unstale":"2024-05-08T05:51:06.561768+0000","last_undegraded":"2024-05-08T05:51:06.561768+0000","last_fullsized":"2024-05-08T05:51:06.561768+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:20:03.103540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.348445+0000","last_change":"2024-05-08T05:51:05.080306+0000","last_active":"2024-05-08T05:51:07.348445+0000","last_peered":"2024-05-08T05:51:07.348445+0000","last_clean":"2024-05-08T05:51:07.348445+0000","last_became_active":"2024-05-08T05:51:05.080046+0000","last_became_peered":"2024-05-08T05:51:05.080046+0000","last_unstale":"2024-05-08T05:51:07.348445+0000","last_undegraded":"2024-05-08T05:51:07.348445+0000","last_fullsized":"2024-05-08T05:51:07.348445+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:55.131091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.561544+0000","last_change":"2024-05-08T05:51:04.072005+0000","last_active":"2024-05-08T05:51:06.561544+0000","last_peered":"2024-05-08T05:51:06.561544+0000","last_clean":"2024-05-08T05:51:06.561544+0000","last_became_active":"2024-05-08T05:51:04.071776+0000","last_became_peered":"2024-05-08T05:51:04.071776+0000","last_unstale":"2024-05-08T05:51:06.561544+0000","last_undegraded":"2024-05-08T05:51:06.561544+0000","last_fullsized":"2024-05-08T05:51:06.561544+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:36.707919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066166+0000","last_change":"2024-05-08T05:51:05.074599+0000","last_active":"2024-05-08T05:51:07.066166+0000","last_peered":"2024-05-08T05:51:07.066166+0000","last_clean":"2024-05-08T05:51:07.066166+0000","last_became_active":"2024-05-08T05:51:05.073942+0000","last_became_peered":"2024-05-08T05:51:05.073942+0000","last_unstale":"2024-05-08T05:51:07.066166+0000","last_undegraded":"2024-05-08T05:51:07.066166+0000","last_fullsized":"2024-05-08T05:51:07.066166+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:13:30.124962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066177+0000","last_change":"2024-05-08T05:51:04.069082+0000","last_active":"2024-05-08T05:51:07.066177+0000","last_peered":"2024-05-08T05:51:07.066177+0000","last_clean":"2024-05-08T05:51:07.066177+0000","last_became_active":"2024-05-08T05:51:04.068801+0000","last_became_peered":"2024-05-08T05:51:04.068801+0000","last_unstale":"2024-05-08T05:51:07.066177+0000","last_undegraded":"2024-05-08T05:51:07.066177+0000","last_fullsized":"2024-05-08T05:51:07.066177+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:57:45.457442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.092747+0000","last_change":"2024-05-08T05:51:05.083611+0000","last_active":"2024-05-08T05:51:05.092747+0000","last_peered":"2024-05-08T05:51:05.092747+0000","last_clean":"2024-05-08T05:51:05.092747+0000","last_became_active":"2024-05-08T05:51:05.083328+0000","last_became_peered":"2024-05-08T05:51:05.083328+0000","last_unstale":"2024-05-08T05:51:05.092747+0000","last_undegraded":"2024-05-08T05:51:05.092747+0000","last_fullsized":"2024-05-08T05:51:05.092747+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:55.131091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.561199+0000","last_change":"2024-05-08T05:51:04.063770+0000","last_active":"2024-05-08T05:51:06.561199+0000","last_peered":"2024-05-08T05:51:06.561199+0000","last_clean":"2024-05-08T05:51:06.561199+0000","last_became_active":"2024-05-08T05:51:04.063509+0000","last_became_peered":"2024-05-08T05:51:04.063509+0000","last_unstale":"2024-05-08T05:51:06.561199+0000","last_undegraded":"2024-05-08T05:51:06.561199+0000","last_fullsized":"2024-05-08T05:51:06.561199+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:54.123101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.093539+0000","last_change":"2024-05-08T05:51:05.090152+0000","last_active":"2024-05-08T05:51:05.093539+0000","last_peered":"2024-05-08T05:51:05.093539+0000","last_clean":"2024-05-08T05:51:05.093539+0000","last_became_active":"2024-05-08T05:51:05.089828+0000","last_became_peered":"2024-05-08T05:51:05.089828+0000","last_unstale":"2024-05-08T05:51:05.093539+0000","last_undegraded":"2024-05-08T05:51:05.093539+0000","last_fullsized":"2024-05-08T05:51:05.093539+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:34:19.612717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.069013+0000","last_change":"2024-05-08T05:51:04.069833+0000","last_active":"2024-05-08T05:51:05.069013+0000","last_peered":"2024-05-08T05:51:05.069013+0000","last_clean":"2024-05-08T05:51:05.069013+0000","last_became_active":"2024-05-08T05:51:04.069539+0000","last_became_peered":"2024-05-08T05:51:04.069539+0000","last_unstale":"2024-05-08T05:51:05.069013+0000","last_undegraded":"2024-05-08T05:51:05.069013+0000","last_fullsized":"2024-05-08T05:51:05.069013+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:36:49.817010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.348040+0000","last_change":"2024-05-08T05:51:05.079322+0000","last_active":"2024-05-08T05:51:07.348040+0000","last_peered":"2024-05-08T05:51:07.348040+0000","last_clean":"2024-05-08T05:51:07.348040+0000","last_became_active":"2024-05-08T05:51:05.078999+0000","last_became_peered":"2024-05-08T05:51:05.078999+0000","last_unstale":"2024-05-08T05:51:07.348040+0000","last_undegraded":"2024-05-08T05:51:07.348040+0000","last_fullsized":"2024-05-08T05:51:07.348040+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:45.439346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.561082+0000","last_change":"2024-05-08T05:51:04.063987+0000","last_active":"2024-05-08T05:51:06.561082+0000","last_peered":"2024-05-08T05:51:06.561082+0000","last_clean":"2024-05-08T05:51:06.561082+0000","last_became_active":"2024-05-08T05:51:04.063646+0000","last_became_peered":"2024-05-08T05:51:04.063646+0000","last_unstale":"2024-05-08T05:51:06.561082+0000","last_undegraded":"2024-05-08T05:51:06.561082+0000","last_fullsized":"2024-05-08T05:51:06.561082+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:28.960211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066817+0000","last_change":"2024-05-08T05:51:05.075930+0000","last_active":"2024-05-08T05:51:07.066817+0000","last_peered":"2024-05-08T05:51:07.066817+0000","last_clean":"2024-05-08T05:51:07.066817+0000","last_became_active":"2024-05-08T05:51:05.075052+0000","last_became_peered":"2024-05-08T05:51:05.075052+0000","last_unstale":"2024-05-08T05:51:07.066817+0000","last_undegraded":"2024-05-08T05:51:07.066817+0000","last_fullsized":"2024-05-08T05:51:07.066817+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:27:28.653248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066775+0000","last_change":"2024-05-08T05:51:04.063919+0000","last_active":"2024-05-08T05:51:07.066775+0000","last_peered":"2024-05-08T05:51:07.066775+0000","last_clean":"2024-05-08T05:51:07.066775+0000","last_became_active":"2024-05-08T05:51:04.063610+0000","last_became_peered":"2024-05-08T05:51:04.063610+0000","last_unstale":"2024-05-08T05:51:07.066775+0000","last_undegraded":"2024-05-08T05:51:07.066775+0000","last_fullsized":"2024-05-08T05:51:07.066775+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:26.855446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.076159+0000","last_change":"2024-05-08T05:51:05.073177+0000","last_active":"2024-05-08T05:51:05.076159+0000","last_peered":"2024-05-08T05:51:05.076159+0000","last_clean":"2024-05-08T05:51:05.076159+0000","last_became_active":"2024-05-08T05:51:05.072104+0000","last_became_peered":"2024-05-08T05:51:05.072104+0000","last_unstale":"2024-05-08T05:51:05.076159+0000","last_undegraded":"2024-05-08T05:51:05.076159+0000","last_fullsized":"2024-05-08T05:51:05.076159+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:29.725291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"119'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.083591+0000","last_change":"2024-05-08T05:51:04.069690+0000","last_active":"2024-05-08T05:51:06.083591+0000","last_peered":"2024-05-08T05:51:06.083591+0000","last_clean":"2024-05-08T05:51:06.083591+0000","last_became_active":"2024-05-08T05:51:04.069380+0000","last_became_peered":"2024-05-08T05:51:04.069380+0000","last_unstale":"2024-05-08T05:51:06.083591+0000","last_undegraded":"2024-05-08T05:51:06.083591+0000","last_fullsized":"2024-05-08T05:51:06.083591+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:38:52.154207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.561904+0000","last_change":"2024-05-08T05:51:05.086167+0000","last_active":"2024-05-08T05:51:06.561904+0000","last_peered":"2024-05-08T05:51:06.561904+0000","last_clean":"2024-05-08T05:51:06.561904+0000","last_became_active":"2024-05-08T05:51:05.085865+0000","last_became_peered":"2024-05-08T05:51:05.085865+0000","last_unstale":"2024-05-08T05:51:06.561904+0000","last_undegraded":"2024-05-08T05:51:06.561904+0000","last_fullsized":"2024-05-08T05:51:06.561904+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:29:35.863971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.083805+0000","last_change":"2024-05-08T05:51:04.070027+0000","last_active":"2024-05-08T05:51:06.083805+0000","last_peered":"2024-05-08T05:51:06.083805+0000","last_clean":"2024-05-08T05:51:06.083805+0000","last_became_active":"2024-05-08T05:51:04.069331+0000","last_became_peered":"2024-05-08T05:51:04.069331+0000","last_unstale":"2024-05-08T05:51:06.083805+0000","last_undegraded":"2024-05-08T05:51:06.083805+0000","last_fullsized":"2024-05-08T05:51:06.083805+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:12:01.633423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.079017+0000","last_change":"2024-05-08T05:51:05.079710+0000","last_active":"2024-05-08T05:51:06.079017+0000","last_peered":"2024-05-08T05:51:06.079017+0000","last_clean":"2024-05-08T05:51:06.079017+0000","last_became_active":"2024-05-08T05:51:05.079405+0000","last_became_peered":"2024-05-08T05:51:05.079405+0000","last_unstale":"2024-05-08T05:51:06.079017+0000","last_undegraded":"2024-05-08T05:51:06.079017+0000","last_fullsized":"2024-05-08T05:51:06.079017+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:57:15.216780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066813+0000","last_change":"2024-05-08T05:51:04.066485+0000","last_active":"2024-05-08T05:51:07.066813+0000","last_peered":"2024-05-08T05:51:07.066813+0000","last_clean":"2024-05-08T05:51:07.066813+0000","last_became_active":"2024-05-08T05:51:04.066192+0000","last_became_peered":"2024-05-08T05:51:04.066192+0000","last_unstale":"2024-05-08T05:51:07.066813+0000","last_undegraded":"2024-05-08T05:51:07.066813+0000","last_fullsized":"2024-05-08T05:51:07.066813+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:54:27.233544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066370+0000","last_change":"2024-05-08T05:51:05.074917+0000","last_active":"2024-05-08T05:51:07.066370+0000","last_peered":"2024-05-08T05:51:07.066370+0000","last_clean":"2024-05-08T05:51:07.066370+0000","last_became_active":"2024-05-08T05:51:05.074339+0000","last_became_peered":"2024-05-08T05:51:05.074339+0000","last_unstale":"2024-05-08T05:51:07.066370+0000","last_undegraded":"2024-05-08T05:51:07.066370+0000","last_fullsized":"2024-05-08T05:51:07.066370+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:22:04.619439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.062382+0000","last_change":"2024-05-08T05:51:04.057289+0000","last_active":"2024-05-08T05:51:05.062382+0000","last_peered":"2024-05-08T05:51:05.062382+0000","last_clean":"2024-05-08T05:51:05.062382+0000","last_became_active":"2024-05-08T05:51:04.056950+0000","last_became_peered":"2024-05-08T05:51:04.056950+0000","last_unstale":"2024-05-08T05:51:05.062382+0000","last_undegraded":"2024-05-08T05:51:05.062382+0000","last_fullsized":"2024-05-08T05:51:05.062382+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:34:17.541863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.079392+0000","last_change":"2024-05-08T05:51:05.076808+0000","last_active":"2024-05-08T05:51:05.079392+0000","last_peered":"2024-05-08T05:51:05.079392+0000","last_clean":"2024-05-08T05:51:05.079392+0000","last_became_active":"2024-05-08T05:51:05.076532+0000","last_became_peered":"2024-05-08T05:51:05.076532+0000","last_unstale":"2024-05-08T05:51:05.079392+0000","last_undegraded":"2024-05-08T05:51:05.079392+0000","last_fullsized":"2024-05-08T05:51:05.079392+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:12.526046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.062570+0000","last_change":"2024-05-08T05:51:04.054314+0000","last_active":"2024-05-08T05:51:05.062570+0000","last_peered":"2024-05-08T05:51:05.062570+0000","last_clean":"2024-05-08T05:51:05.062570+0000","last_became_active":"2024-05-08T05:51:04.053754+0000","last_became_peered":"2024-05-08T05:51:04.053754+0000","last_unstale":"2024-05-08T05:51:05.062570+0000","last_undegraded":"2024-05-08T05:51:05.062570+0000","last_fullsized":"2024-05-08T05:51:05.062570+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:26.339116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.561282+0000","last_change":"2024-05-08T05:51:05.076675+0000","last_active":"2024-05-08T05:51:06.561282+0000","last_peered":"2024-05-08T05:51:06.561282+0000","last_clean":"2024-05-08T05:51:06.561282+0000","last_became_active":"2024-05-08T05:51:05.076351+0000","last_became_peered":"2024-05-08T05:51:05.076351+0000","last_unstale":"2024-05-08T05:51:06.561282+0000","last_undegraded":"2024-05-08T05:51:06.561282+0000","last_fullsized":"2024-05-08T05:51:06.561282+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:03:06.656017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:07.066301+0000","last_change":"2024-05-08T05:51:04.064412+0000","last_active":"2024-05-08T05:51:07.066301+0000","last_peered":"2024-05-08T05:51:07.066301+0000","last_clean":"2024-05-08T05:51:07.066301+0000","last_became_active":"2024-05-08T05:51:04.064134+0000","last_became_peered":"2024-05-08T05:51:04.064134+0000","last_unstale":"2024-05-08T05:51:07.066301+0000","last_undegraded":"2024-05-08T05:51:07.066301+0000","last_fullsized":"2024-05-08T05:51:07.066301+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:37:09.043074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.088681+0000","last_change":"2024-05-08T05:51:05.087096+0000","last_active":"2024-05-08T05:51:05.088681+0000","last_peered":"2024-05-08T05:51:05.088681+0000","last_clean":"2024-05-08T05:51:05.088681+0000","last_became_active":"2024-05-08T05:51:05.085170+0000","last_became_peered":"2024-05-08T05:51:05.085170+0000","last_unstale":"2024-05-08T05:51:05.088681+0000","last_undegraded":"2024-05-08T05:51:05.088681+0000","last_fullsized":"2024-05-08T05:51:05.088681+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:20.213009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.561096+0000","last_change":"2024-05-08T05:51:04.065454+0000","last_active":"2024-05-08T05:51:06.561096+0000","last_peered":"2024-05-08T05:51:06.561096+0000","last_clean":"2024-05-08T05:51:06.561096+0000","last_became_active":"2024-05-08T05:51:04.065175+0000","last_became_peered":"2024-05-08T05:51:04.065175+0000","last_unstale":"2024-05-08T05:51:06.561096+0000","last_undegraded":"2024-05-08T05:51:06.561096+0000","last_fullsized":"2024-05-08T05:51:06.561096+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:44:48.375896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.561490+0000","last_change":"2024-05-08T05:51:05.089415+0000","last_active":"2024-05-08T05:51:06.561490+0000","last_peered":"2024-05-08T05:51:06.561490+0000","last_clean":"2024-05-08T05:51:06.561490+0000","last_became_active":"2024-05-08T05:51:05.089090+0000","last_became_peered":"2024-05-08T05:51:05.089090+0000","last_unstale":"2024-05-08T05:51:06.561490+0000","last_undegraded":"2024-05-08T05:51:06.561490+0000","last_fullsized":"2024-05-08T05:51:06.561490+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:04.041342+0000","last_clean_scrub_stamp":"2024-05-08T05:51:04.041342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:43:23.481433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"118'8","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-05-08T05:51:05.096717+0000","last_change":"2024-05-08T05:51:04.068279+0000","last_active":"2024-05-08T05:51:05.096717+0000","last_peered":"2024-05-08T05:51:05.096717+0000","last_clean":"2024-05-08T05:51:05.096717+0000","last_became_active":"2024-05-08T05:51:04.067953+0000","last_became_peered":"2024-05-08T05:51:04.067953+0000","last_unstale":"2024-05-08T05:51:05.096717+0000","last_undegraded":"2024-05-08T05:51:05.096717+0000","last_fullsized":"2024-05-08T05:51:05.096717+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:51:03.033352+0000","last_clean_scrub_stamp":"2024-05-08T05:51:03.033352+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:48:54.640197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":292,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-05-08T05:51:06.561370+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:51:06.561370+0000","last_peered":"2024-05-08T05:51:06.561370+0000","last_clean":"2024-05-08T05:51:06.561370+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:51:06.561370+0000","last_undegraded":"2024-05-08T05:51:06.561370+0000","last_fullsized":"2024-05-08T05:51:06.561370+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542266,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33220,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338620,"statfs":{"total":96636764160,"available":96602746880,"internally_reserved":0,"allocated":2088960,"data_stored":1281075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542267,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32636,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1482752,"data_stored":690513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542266,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":690091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1478656,"data_stored":684851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1482752,"data_stored":684885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1478656,"data_stored":689931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1470464,"data_stored":683261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2076672,"data_stored":1280321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:51:13.965 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T05:51:13.965 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-08T05:51:14.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.114+0000 7f2eb1b1c640 1 Processor -- start 2024-05-08T05:51:14.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.114+0000 7f2eb1b1c640 1 -- start start 2024-05-08T05:51:14.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eb1b1c640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f2eac0d33f0 0x7f2eac15d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eb1b1c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eac0718c0 con 0x7f2eac070b90 2024-05-08T05:51:14.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eb1b1c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eac071ba0 con 0x7f2eac15ccb0 2024-05-08T05:51:14.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eb1b1c640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2eac0afba0 con 0x7f2eac0d33f0 2024-05-08T05:51:14.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eabfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2eac070b90 0x7f2eac071000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40534/0 (socket says 172.21.15.45:40534) 2024-05-08T05:51:14.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eabfff640 1 -- 172.21.15.45:0/2098562850 learned_addr learned my addr 172.21.15.45:0/2098562850 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:14.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2ea3fff640 1 --2- 172.21.15.45:0/2098562850 >> v2:172.21.15.63:3300/0 conn(0x7f2eac0d33f0 0x7f2eac15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eaaffd640 1 -- 172.21.15.45:0/2098562850 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2232629952 0 0) 0x7f2eac0718c0 con 0x7f2eac070b90 2024-05-08T05:51:14.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eaaffd640 1 -- 172.21.15.45:0/2098562850 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e98003520 con 0x7f2eac070b90 2024-05-08T05:51:14.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eaaffd640 1 -- 172.21.15.45:0/2098562850 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1847488461 0 0) 0x7f2e98003520 con 0x7f2eac070b90 2024-05-08T05:51:14.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eaaffd640 1 -- 172.21.15.45:0/2098562850 >> v2:172.21.15.63:3300/0 conn(0x7f2eac0d33f0 msgr2=0x7f2eac15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:14.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eaaffd640 1 --2- 172.21.15.45:0/2098562850 >> v2:172.21.15.63:3300/0 conn(0x7f2eac0d33f0 0x7f2eac15d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eaaffd640 1 -- 172.21.15.45:0/2098562850 >> v1:172.21.15.63:6789/0 conn(0x7f2eac15ccb0 legacy=0x7f2eac0d2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eaaffd640 1 -- 172.21.15.45:0/2098562850 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eac071e80 con 0x7f2eac070b90 2024-05-08T05:51:14.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eaaffd640 1 -- 172.21.15.45:0/2098562850 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f2ea4001a70 con 0x7f2eac070b90 2024-05-08T05:51:14.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eaaffd640 1 -- 172.21.15.45:0/2098562850 >> v1:172.21.15.45:6789/0 conn(0x7f2eac070b90 legacy=0x7f2eac071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eaaffd640 1 --2- 172.21.15.45:0/2098562850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e98003a50 0x7f2e98003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eaaffd640 1 --2- 172.21.15.45:0/2098562850 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e98004800 0x7f2e98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eaaffd640 1 --2- 172.21.15.45:0/2098562850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e980052e0 0x7f2e9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eaaffd640 1 -- 172.21.15.45:0/2098562850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9800c180 con 0x7f2e980052e0 2024-05-08T05:51:14.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eaaffd640 1 -- 172.21.15.45:0/2098562850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9800c330 con 0x7f2e98003a50 2024-05-08T05:51:14.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eaaffd640 1 -- 172.21.15.45:0/2098562850 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2e9800c530 con 0x7f2e98004800 2024-05-08T05:51:14.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eabfff640 1 --2- 172.21.15.45:0/2098562850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e980052e0 0x7f2e9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2ea3fff640 1 --2- 172.21.15.45:0/2098562850 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e98004800 0x7f2e98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eab7fe640 1 --2- 172.21.15.45:0/2098562850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e98003a50 0x7f2e98003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eabfff640 1 -- 172.21.15.45:0/2098562850 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e98004800 msgr2=0x7f2e98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:14.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eabfff640 1 --2- 172.21.15.45:0/2098562850 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e98004800 0x7f2e98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eabfff640 1 -- 172.21.15.45:0/2098562850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e98003a50 msgr2=0x7f2e98003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:14.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eabfff640 1 --2- 172.21.15.45:0/2098562850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e98003a50 0x7f2e98003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.115+0000 7f2eabfff640 1 -- 172.21.15.45:0/2098562850 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e98003520 con 0x7f2e980052e0 2024-05-08T05:51:14.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.116+0000 7f2eabfff640 1 -- 172.21.15.45:0/2098562850 reap_dead start 2024-05-08T05:51:14.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.116+0000 7f2eabfff640 1 --2- 172.21.15.45:0/2098562850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e980052e0 0x7f2e9800b9c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4002230 tx=0x7f2e98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532b60063cae714a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:14.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.116+0000 7f2eaaffd640 1 -- 172.21.15.45:0/2098562850 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea400a480 con 0x7f2e980052e0 2024-05-08T05:51:14.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.116+0000 7f2eaaffd640 1 -- 172.21.15.45:0/2098562850 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea402f070 con 0x7f2e980052e0 2024-05-08T05:51:14.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.116+0000 7f2eaaffd640 1 -- 172.21.15.45:0/2098562850 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea402bba0 con 0x7f2e980052e0 2024-05-08T05:51:14.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.116+0000 7f2eb1b1c640 1 -- 172.21.15.45:0/2098562850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e980052e0 msgr2=0x7f2e9800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.116+0000 7f2eb1b1c640 1 --2- 172.21.15.45:0/2098562850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e980052e0 0x7f2e9800b9c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4002230 tx=0x7f2e98005830 comp rx=0 tx=0).stop 2024-05-08T05:51:14.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.116+0000 7f2eb1b1c640 1 -- 172.21.15.45:0/2098562850 shutdown_connections 2024-05-08T05:51:14.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.116+0000 7f2eb1b1c640 1 --2- 172.21.15.45:0/2098562850 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e980052e0 0x7f2e9800b9c0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.117+0000 7f2eb1b1c640 1 -- 172.21.15.45:0/2098562850 >> 172.21.15.45:0/2098562850 conn(0x7f2eac09ba40 msgr2=0x7f2eac09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:14.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.117+0000 7f2eb1b1c640 1 -- 172.21.15.45:0/2098562850 shutdown_connections 2024-05-08T05:51:14.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.117+0000 7f2eb1b1c640 1 -- 172.21.15.45:0/2098562850 wait complete. 2024-05-08T05:51:14.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.118+0000 7f2eb1b1c640 1 Processor -- start 2024-05-08T05:51:14.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.118+0000 7f2eb1b1c640 1 -- start start 2024-05-08T05:51:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.118+0000 7f2eb1b1c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e980052e0 0x7f2eac0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.118+0000 7f2eb1b1c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2eac06d4b0 0x7f2eac0c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.119+0000 7f2eb1b1c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eac06d930 0x7f2eac0ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.119+0000 7f2eb1b1c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac15d210 con 0x7f2eac06d930 2024-05-08T05:51:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.119+0000 7f2eb1b1c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac15d090 con 0x7f2eac06d4b0 2024-05-08T05:51:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.119+0000 7f2eb1b1c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2eac15d390 con 0x7f2e980052e0 2024-05-08T05:51:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.119+0000 7f2eabfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eac06d930 0x7f2eac0ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.119+0000 7f2eabfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eac06d930 0x7f2eac0ca440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45682/0 (socket says 172.21.15.45:45682) 2024-05-08T05:51:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.119+0000 7f2eabfff640 1 -- 172.21.15.45:0/2333664807 learned_addr learned my addr 172.21.15.45:0/2333664807 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.119+0000 7f2ea3fff640 1 --2- 172.21.15.45:0/2333664807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2eac06d4b0 0x7f2eac0c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.119+0000 7f2eab7fe640 1 --2- 172.21.15.45:0/2333664807 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e980052e0 0x7f2eac0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.119+0000 7f2eabfff640 1 -- 172.21.15.45:0/2333664807 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e980052e0 msgr2=0x7f2eac0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.119+0000 7f2eabfff640 1 --2- 172.21.15.45:0/2333664807 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e980052e0 0x7f2eac0cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.119+0000 7f2eabfff640 1 -- 172.21.15.45:0/2333664807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2eac06d4b0 msgr2=0x7f2eac0c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.119+0000 7f2eabfff640 1 --2- 172.21.15.45:0/2333664807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2eac06d4b0 0x7f2eac0c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.119+0000 7f2eabfff640 1 -- 172.21.15.45:0/2333664807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eac06de90 con 0x7f2eac06d930 2024-05-08T05:51:14.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.119+0000 7f2eabfff640 1 --2- 172.21.15.45:0/2333664807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eac06d930 0x7f2eac0ca440 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400a8f0 tx=0x7f2ea4001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:14.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.119+0000 7f2ea8ff9640 1 -- 172.21.15.45:0/2333664807 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea400a460 con 0x7f2eac06d930 2024-05-08T05:51:14.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.119+0000 7f2ea8ff9640 1 -- 172.21.15.45:0/2333664807 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea4034470 con 0x7f2eac06d930 2024-05-08T05:51:14.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.119+0000 7f2eb1b1c640 1 -- 172.21.15.45:0/2333664807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eac06e170 con 0x7f2eac06d930 2024-05-08T05:51:14.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.119+0000 7f2ea8ff9640 1 -- 172.21.15.45:0/2333664807 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4034820 con 0x7f2eac06d930 2024-05-08T05:51:14.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.119+0000 7f2eb1b1c640 1 -- 172.21.15.45:0/2333664807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eac06e730 con 0x7f2eac06d930 2024-05-08T05:51:14.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.120+0000 7f2eb1b1c640 1 -- 172.21.15.45:0/2333664807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eac069b00 con 0x7f2eac06d930 2024-05-08T05:51:14.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.121+0000 7f2ea8ff9640 1 -- 172.21.15.45:0/2333664807 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2ea40349c0 con 0x7f2eac06d930 2024-05-08T05:51:14.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.122+0000 7f2ea8ff9640 1 --2- 172.21.15.45:0/2333664807 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2e8c076300 0x7f2e8c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.122+0000 7f2ea8ff9640 1 -- 172.21.15.45:0/2333664807 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f2ea40b7550 con 0x7f2eac06d930 2024-05-08T05:51:14.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.122+0000 7f2eab7fe640 1 --2- 172.21.15.45:0/2333664807 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2e8c076300 0x7f2e8c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.122+0000 7f2eab7fe640 1 --2- 172.21.15.45:0/2333664807 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2e8c076300 0x7f2e8c0787c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0017a0 tx=0x7f2e9c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:14.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.126+0000 7f2ea8ff9640 1 -- 172.21.15.45:0/2333664807 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2ea4080d20 con 0x7f2eac06d930 2024-05-08T05:51:14.390 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.389+0000 7f2eb1b1c640 1 -- 172.21.15.45:0/2333664807 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f2eac070380 con 0x7f2eac06d930 2024-05-08T05:51:14.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.391+0000 7f2ea8ff9640 1 -- 172.21.15.45:0/2333664807 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f2ea4080620 con 0x7f2eac06d930 2024-05-08T05:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-05-08T05:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBUEDtmDcqGMxAALgQaBYp4Zd8uMqswCXvGaw== 2024-05-08T05:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: caps mds = "allow" 2024-05-08T05:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: caps mon = "allow r" 2024-05-08T05:51:14.393 INFO:teuthology.orchestra.run.smithi045.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-08T05:51:14.393 INFO:teuthology.orchestra.run.smithi045.stderr:updated caps for client.0 2024-05-08T05:51:14.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.394+0000 7f2eb1b1c640 1 -- 172.21.15.45:0/2333664807 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2e8c076300 msgr2=0x7f2e8c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.394+0000 7f2eb1b1c640 1 --2- 172.21.15.45:0/2333664807 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2e8c076300 0x7f2e8c0787c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0017a0 tx=0x7f2e9c009040 comp rx=0 tx=0).stop 2024-05-08T05:51:14.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.394+0000 7f2eb1b1c640 1 -- 172.21.15.45:0/2333664807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eac06d930 msgr2=0x7f2eac0ca440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.394+0000 7f2eb1b1c640 1 --2- 172.21.15.45:0/2333664807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eac06d930 0x7f2eac0ca440 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400a8f0 tx=0x7f2ea4001360 comp rx=0 tx=0).stop 2024-05-08T05:51:14.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.394+0000 7f2eb1b1c640 1 -- 172.21.15.45:0/2333664807 shutdown_connections 2024-05-08T05:51:14.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.394+0000 7f2eb1b1c640 1 --2- 172.21.15.45:0/2333664807 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2e8c076300 0x7f2e8c0787c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.394+0000 7f2eb1b1c640 1 --2- 172.21.15.45:0/2333664807 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eac06d930 0x7f2eac0ca440 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.394+0000 7f2eb1b1c640 1 --2- 172.21.15.45:0/2333664807 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2eac06d4b0 0x7f2eac0c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.394+0000 7f2eb1b1c640 1 --2- 172.21.15.45:0/2333664807 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2e980052e0 0x7f2eac0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.394+0000 7f2eb1b1c640 1 -- 172.21.15.45:0/2333664807 >> 172.21.15.45:0/2333664807 conn(0x7f2eac09ba40 msgr2=0x7f2eac15c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:14.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.394+0000 7f2eb1b1c640 1 -- 172.21.15.45:0/2333664807 shutdown_connections 2024-05-08T05:51:14.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.394+0000 7f2eb1b1c640 1 -- 172.21.15.45:0/2333664807 wait complete. 2024-05-08T05:51:14.406 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:51:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.509+0000 7fb834fe4640 1 Processor -- start 2024-05-08T05:51:14.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.509+0000 7fb834fe4640 1 -- start start 2024-05-08T05:51:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.510+0000 7fb834fe4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8300d20c0 0x7fb830163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.510+0000 7fb834fe4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8300d1300 con 0x7fb830070d30 2024-05-08T05:51:14.514 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.510+0000 7fb834fe4640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8300d15e0 con 0x7fb830071110 2024-05-08T05:51:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.510+0000 7fb834fe4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb8300d10d0 con 0x7fb8300d20c0 2024-05-08T05:51:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.510+0000 7fb82e59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fb830070d30 0x7fb8300703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40542/0 (socket says 172.21.15.45:40542) 2024-05-08T05:51:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.510+0000 7fb82e59c640 1 -- 172.21.15.45:0/1833946378 learned_addr learned my addr 172.21.15.45:0/1833946378 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.510+0000 7fb827fff640 1 --2- 172.21.15.45:0/1833946378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8300d20c0 0x7fb830163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.510+0000 7fb82dd9b640 1 -- 172.21.15.45:0/1833946378 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2036821278 0 0) 0x7fb8300d1300 con 0x7fb830070d30 2024-05-08T05:51:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.510+0000 7fb82dd9b640 1 -- 172.21.15.45:0/1833946378 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb81c003520 con 0x7fb830070d30 2024-05-08T05:51:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.510+0000 7fb82dd9b640 1 -- 172.21.15.45:0/1833946378 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2066651801 0 0) 0x7fb8300d15e0 con 0x7fb830071110 2024-05-08T05:51:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.510+0000 7fb82dd9b640 1 -- 172.21.15.45:0/1833946378 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8300d1300 con 0x7fb830071110 2024-05-08T05:51:14.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.510+0000 7fb82dd9b640 1 -- 172.21.15.45:0/1833946378 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3505748035 0 0) 0x7fb81c003520 con 0x7fb830070d30 2024-05-08T05:51:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.510+0000 7fb82dd9b640 1 -- 172.21.15.45:0/1833946378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8300d20c0 msgr2=0x7fb830163920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.510+0000 7fb82dd9b640 1 --2- 172.21.15.45:0/1833946378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8300d20c0 0x7fb830163920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.510+0000 7fb82dd9b640 1 -- 172.21.15.45:0/1833946378 >> v1:172.21.15.63:6789/0 conn(0x7fb830071110 legacy=0x7fb83006d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.510+0000 7fb82dd9b640 1 -- 172.21.15.45:0/1833946378 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8300d18c0 con 0x7fb830070d30 2024-05-08T05:51:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.510+0000 7fb82dd9b640 1 -- 172.21.15.45:0/1833946378 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fb8200016f0 con 0x7fb830070d30 2024-05-08T05:51:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.510+0000 7fb82dd9b640 1 -- 172.21.15.45:0/1833946378 >> v1:172.21.15.45:6789/0 conn(0x7fb830070d30 legacy=0x7fb8300703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.511+0000 7fb82dd9b640 1 --2- 172.21.15.45:0/1833946378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb81c003c10 0x7fb81c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.511+0000 7fb82dd9b640 1 --2- 172.21.15.45:0/1833946378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb81c004800 0x7fb81c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.511+0000 7fb82dd9b640 1 --2- 172.21.15.45:0/1833946378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8300d20c0 0x7fb81c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.511+0000 7fb82dd9b640 1 -- 172.21.15.45:0/1833946378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c00bb50 con 0x7fb81c004800 2024-05-08T05:51:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.511+0000 7fb82dd9b640 1 -- 172.21.15.45:0/1833946378 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c00bcd0 con 0x7fb81c003c10 2024-05-08T05:51:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.511+0000 7fb82dd9b640 1 -- 172.21.15.45:0/1833946378 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb81c00bea0 con 0x7fb8300d20c0 2024-05-08T05:51:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.511+0000 7fb82e59c640 1 --2- 172.21.15.45:0/1833946378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb81c003c10 0x7fb81c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.511+0000 7fb827fff640 1 --2- 172.21.15.45:0/1833946378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb81c004800 0x7fb81c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.511+0000 7fb82ed9d640 1 --2- 172.21.15.45:0/1833946378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8300d20c0 0x7fb81c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.511+0000 7fb827fff640 1 -- 172.21.15.45:0/1833946378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8300d20c0 msgr2=0x7fb81c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.511+0000 7fb827fff640 1 --2- 172.21.15.45:0/1833946378 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb8300d20c0 0x7fb81c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.511+0000 7fb827fff640 1 -- 172.21.15.45:0/1833946378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb81c003c10 msgr2=0x7fb81c004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.511+0000 7fb827fff640 1 --2- 172.21.15.45:0/1833946378 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb81c003c10 0x7fb81c004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.511+0000 7fb827fff640 1 -- 172.21.15.45:0/1833946378 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81c003520 con 0x7fb81c004800 2024-05-08T05:51:14.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.512+0000 7fb827fff640 1 --2- 172.21.15.45:0/1833946378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb81c004800 0x7fb81c004c80 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fb818009af0 tx=0x7fb81c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be790a67ff530aa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.512+0000 7fb82dd9b640 1 -- 172.21.15.45:0/1833946378 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818036070 con 0x7fb81c004800 2024-05-08T05:51:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.512+0000 7fb82dd9b640 1 -- 172.21.15.45:0/1833946378 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb818031400 con 0x7fb81c004800 2024-05-08T05:51:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.512+0000 7fb82dd9b640 1 -- 172.21.15.45:0/1833946378 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818009690 con 0x7fb81c004800 2024-05-08T05:51:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.512+0000 7fb834fe4640 1 -- 172.21.15.45:0/1833946378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb81c004800 msgr2=0x7fb81c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.512+0000 7fb834fe4640 1 --2- 172.21.15.45:0/1833946378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb81c004800 0x7fb81c004c80 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fb818009af0 tx=0x7fb81c005830 comp rx=0 tx=0).stop 2024-05-08T05:51:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.512+0000 7fb82ed9d640 1 -- 172.21.15.45:0/1833946378 reap_dead start 2024-05-08T05:51:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.512+0000 7fb834fe4640 1 -- 172.21.15.45:0/1833946378 shutdown_connections 2024-05-08T05:51:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.512+0000 7fb834fe4640 1 -- 172.21.15.45:0/1833946378 >> 172.21.15.45:0/1833946378 conn(0x7fb8300d2e90 msgr2=0x7fb8300d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.512+0000 7fb834fe4640 1 -- 172.21.15.45:0/1833946378 shutdown_connections 2024-05-08T05:51:14.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.513+0000 7fb834fe4640 1 -- 172.21.15.45:0/1833946378 wait complete. 2024-05-08T05:51:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb834fe4640 1 Processor -- start 2024-05-08T05:51:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb834fe4640 1 -- start start 2024-05-08T05:51:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb834fe4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb81c004800 0x7fb8301f5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb834fe4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8300c6bf0 0x7fb8300ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb834fe4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8300c8e70 0x7fb8300b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb834fe4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb8300d1c30 con 0x7fb8300c8e70 2024-05-08T05:51:14.521 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb834fe4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb830070b80 con 0x7fb8300c6bf0 2024-05-08T05:51:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb834fe4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb830163920 con 0x7fb81c004800 2024-05-08T05:51:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb82ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8300c8e70 0x7fb8300b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb82ed9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8300c8e70 0x7fb8300b7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45698/0 (socket says 172.21.15.45:45698) 2024-05-08T05:51:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb82ed9d640 1 -- 172.21.15.45:0/651701737 learned_addr learned my addr 172.21.15.45:0/651701737 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb82e59c640 1 --2- 172.21.15.45:0/651701737 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb81c004800 0x7fb8301f5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb827fff640 1 --2- 172.21.15.45:0/651701737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8300c6bf0 0x7fb8300ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb82ed9d640 1 -- 172.21.15.45:0/651701737 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb81c004800 msgr2=0x7fb8301f5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb82ed9d640 1 --2- 172.21.15.45:0/651701737 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb81c004800 0x7fb8301f5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb82ed9d640 1 -- 172.21.15.45:0/651701737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8300c6bf0 msgr2=0x7fb8300ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb82ed9d640 1 --2- 172.21.15.45:0/651701737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8300c6bf0 0x7fb8300ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb82ed9d640 1 -- 172.21.15.45:0/651701737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8300c97a0 con 0x7fb8300c8e70 2024-05-08T05:51:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb82ed9d640 1 --2- 172.21.15.45:0/651701737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8300c8e70 0x7fb8300b7ed0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fb8300d2e40 tx=0x7fb82800ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb826ffd640 1 -- 172.21.15.45:0/651701737 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb828003b80 con 0x7fb8300c8e70 2024-05-08T05:51:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb826ffd640 1 -- 172.21.15.45:0/651701737 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb828018070 con 0x7fb8300c8e70 2024-05-08T05:51:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb834fe4640 1 -- 172.21.15.45:0/651701737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8300b5fe0 con 0x7fb8300c8e70 2024-05-08T05:51:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb826ffd640 1 -- 172.21.15.45:0/651701737 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8280050d0 con 0x7fb8300c8e70 2024-05-08T05:51:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.515+0000 7fb834fe4640 1 -- 172.21.15.45:0/651701737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8300b65a0 con 0x7fb8300c8e70 2024-05-08T05:51:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.516+0000 7fb834fe4640 1 -- 172.21.15.45:0/651701737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb830163d10 con 0x7fb8300c8e70 2024-05-08T05:51:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.517+0000 7fb826ffd640 1 -- 172.21.15.45:0/651701737 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb828005270 con 0x7fb8300c8e70 2024-05-08T05:51:14.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.517+0000 7fb826ffd640 1 --2- 172.21.15.45:0/651701737 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb810076300 0x7fb8100787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.518+0000 7fb826ffd640 1 -- 172.21.15.45:0/651701737 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7fb828097690 con 0x7fb8300c8e70 2024-05-08T05:51:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.518+0000 7fb82e59c640 1 --2- 172.21.15.45:0/651701737 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb810076300 0x7fb8100787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.518+0000 7fb82e59c640 1 --2- 172.21.15.45:0/651701737 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb810076300 0x7fb8100787c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb820001050 tx=0x7fb820001be0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:14.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.523+0000 7fb826ffd640 1 -- 172.21.15.45:0/651701737 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb830163d10 con 0x7fb8300c8e70 2024-05-08T05:51:14.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.751+0000 7fb834fe4640 1 -- 172.21.15.45:0/651701737 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8300693f0 con 0x7fb8300c8e70 2024-05-08T05:51:14.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.752+0000 7fb826ffd640 1 -- 172.21.15.45:0/651701737 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5185 (secure 0 0 0) 0x7fb828060770 con 0x7fb8300c8e70 2024-05-08T05:51:14.754 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:14.754 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7168,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3882445426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3882445426},{"type":"v1","addr":"172.21.15.63:6836","nonce":3882445426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/3173207720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3173207720},{"type":"v1","addr":"172.21.15.45:6833","nonce":3173207720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4254619360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4254619360},{"type":"v1","addr":"172.21.15.63:6837","nonce":4254619360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:51:04.429358+0000","modified":"2024-05-08T05:51:07.454864+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8112},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8112":{"gid":8112,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.45:6835/1751419054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":1751419054},{"type":"v1","addr":"172.21.15.45:6835","nonce":1751419054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8112,"qdb_cluster":[8112]},"id":8}]} 2024-05-08T05:51:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 100 2024-05-08T05:51:14.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.754+0000 7fb834fe4640 1 -- 172.21.15.45:0/651701737 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb810076300 msgr2=0x7fb8100787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.754+0000 7fb834fe4640 1 --2- 172.21.15.45:0/651701737 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb810076300 0x7fb8100787c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb820001050 tx=0x7fb820001be0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.754+0000 7fb834fe4640 1 -- 172.21.15.45:0/651701737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8300c8e70 msgr2=0x7fb8300b7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.754+0000 7fb834fe4640 1 --2- 172.21.15.45:0/651701737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8300c8e70 0x7fb8300b7ed0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fb8300d2e40 tx=0x7fb82800ff70 comp rx=0 tx=0).stop 2024-05-08T05:51:14.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.754+0000 7fb834fe4640 1 -- 172.21.15.45:0/651701737 shutdown_connections 2024-05-08T05:51:14.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.754+0000 7fb834fe4640 1 --2- 172.21.15.45:0/651701737 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb810076300 0x7fb8100787c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.754+0000 7fb834fe4640 1 --2- 172.21.15.45:0/651701737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8300c8e70 0x7fb8300b7ed0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.754+0000 7fb834fe4640 1 --2- 172.21.15.45:0/651701737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb8300c6bf0 0x7fb8300ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.754+0000 7fb834fe4640 1 --2- 172.21.15.45:0/651701737 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb81c004800 0x7fb8301f5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.754+0000 7fb834fe4640 1 -- 172.21.15.45:0/651701737 >> 172.21.15.45:0/651701737 conn(0x7fb8300d2e90 msgr2=0x7fb83006ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:14.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.754+0000 7fb834fe4640 1 -- 172.21.15.45:0/651701737 shutdown_connections 2024-05-08T05:51:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:14.754+0000 7fb834fe4640 1 -- 172.21.15.45:0/651701737 wait complete. 2024-05-08T05:51:15.169 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 100, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:51:04.429358+0000', 'modified': '2024-05-08T05:51:07.454864+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8112}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8112': {'gid': 8112, 'name': 'c', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.45:6835/1751419054', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 1751419054}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 1751419054}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8112, 'qdb_cluster': [8112]} 2024-05-08T05:51:15.169 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T05:51:15.170 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T05:51:15.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.278+0000 7f05c76cf640 1 Processor -- start 2024-05-08T05:51:15.279 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.279+0000 7f05c76cf640 1 -- start start 2024-05-08T05:51:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.279+0000 7f05c76cf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f05c00d20c0 0x7f05c0163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.279+0000 7f05c76cf640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f05c0071110 0x7f05c006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.279+0000 7f05c76cf640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05c00d1300 con 0x7f05c0070d30 2024-05-08T05:51:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.279+0000 7f05c76cf640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f05c00d10d0 con 0x7f05c00d20c0 2024-05-08T05:51:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.279+0000 7f05c76cf640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f05c0070b80 con 0x7f05c0071110 2024-05-08T05:51:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.280+0000 7f05c4c6a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f05c00d20c0 0x7f05c0163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.280+0000 7f05c4c6a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f05c00d20c0 0x7f05c0163920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45704/0 (socket says 172.21.15.45:45704) 2024-05-08T05:51:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.280+0000 7f05c4c6a640 1 -- 172.21.15.45:0/3864671016 learned_addr learned my addr 172.21.15.45:0/3864671016 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:15.280 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.280+0000 7f05c5c6c640 1 --2- 172.21.15.45:0/3864671016 >> v2:172.21.15.63:3300/0 conn(0x7f05c0071110 0x7f05c006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.280+0000 7f05c4c6a640 1 -- 172.21.15.45:0/3864671016 >> v2:172.21.15.63:3300/0 conn(0x7f05c0071110 msgr2=0x7f05c006d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:15.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.280+0000 7f05c4c6a640 1 --2- 172.21.15.45:0/3864671016 >> v2:172.21.15.63:3300/0 conn(0x7f05c0071110 0x7f05c006d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.280+0000 7f05c4c6a640 1 -- 172.21.15.45:0/3864671016 >> v1:172.21.15.63:6789/0 conn(0x7f05c0070d30 legacy=0x7f05c00703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:15.281 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.280+0000 7f05c4c6a640 1 -- 172.21.15.45:0/3864671016 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c00d15e0 con 0x7f05c00d20c0 2024-05-08T05:51:15.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.281+0000 7f05c4c6a640 1 --2- 172.21.15.45:0/3864671016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c00d20c0 0x7f05c0163920 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f05c00d2e40 tx=0x7f05b402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3d0f741c929a0a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:15.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.281+0000 7f05bffff640 1 -- 172.21.15.45:0/3864671016 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4037070 con 0x7f05c00d20c0 2024-05-08T05:51:15.282 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.281+0000 7f05bffff640 1 -- 172.21.15.45:0/3864671016 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b4032440 con 0x7f05c00d20c0 2024-05-08T05:51:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.281+0000 7f05bffff640 1 -- 172.21.15.45:0/3864671016 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4032810 con 0x7f05c00d20c0 2024-05-08T05:51:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.281+0000 7f05c76cf640 1 -- 172.21.15.45:0/3864671016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c00d20c0 msgr2=0x7f05c0163920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.281+0000 7f05c76cf640 1 --2- 172.21.15.45:0/3864671016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c00d20c0 0x7f05c0163920 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f05c00d2e40 tx=0x7f05b402a1e0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.281+0000 7f05c76cf640 1 -- 172.21.15.45:0/3864671016 shutdown_connections 2024-05-08T05:51:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.281+0000 7f05c76cf640 1 --2- 172.21.15.45:0/3864671016 >> v2:172.21.15.63:3300/0 conn(0x7f05c0071110 0x7f05c006d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.281+0000 7f05c76cf640 1 --2- 172.21.15.45:0/3864671016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c00d20c0 0x7f05c0163920 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.281+0000 7f05c76cf640 1 -- 172.21.15.45:0/3864671016 >> 172.21.15.45:0/3864671016 conn(0x7f05c00d2e90 msgr2=0x7f05c00d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.281+0000 7f05c76cf640 1 -- 172.21.15.45:0/3864671016 shutdown_connections 2024-05-08T05:51:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.281+0000 7f05c76cf640 1 -- 172.21.15.45:0/3864671016 wait complete. 2024-05-08T05:51:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.283+0000 7f05c76cf640 1 Processor -- start 2024-05-08T05:51:15.283 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.283+0000 7f05c76cf640 1 -- start start 2024-05-08T05:51:15.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.283+0000 7f05c76cf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c0070d30 0x7f05c0154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.283+0000 7f05c76cf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05c0071110 0x7f05c00b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.283+0000 7f05c76cf640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05c00d20c0 0x7f05c0162cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.283+0000 7f05c76cf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f05c0071520 con 0x7f05c0070d30 2024-05-08T05:51:15.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.283+0000 7f05c76cf640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f05c00d1c30 con 0x7f05c00d20c0 2024-05-08T05:51:15.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.283+0000 7f05c76cf640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f05c00716a0 con 0x7f05c0071110 2024-05-08T05:51:15.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.283+0000 7f05c546b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c0070d30 0x7f05c0154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.283+0000 7f05c546b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c0070d30 0x7f05c0154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45710/0 (socket says 172.21.15.45:45710) 2024-05-08T05:51:15.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.283+0000 7f05c546b640 1 -- 172.21.15.45:0/2004137306 learned_addr learned my addr 172.21.15.45:0/2004137306 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:15.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.284+0000 7f05c5c6c640 1 --2- 172.21.15.45:0/2004137306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05c00d20c0 0x7f05c0162cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.286 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.284+0000 7f05c4c6a640 1 --2- 172.21.15.45:0/2004137306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05c0071110 0x7f05c00b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.284+0000 7f05c546b640 1 -- 172.21.15.45:0/2004137306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05c0071110 msgr2=0x7f05c00b6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.284+0000 7f05c546b640 1 --2- 172.21.15.45:0/2004137306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05c0071110 0x7f05c00b6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.284+0000 7f05c546b640 1 -- 172.21.15.45:0/2004137306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05c00d20c0 msgr2=0x7f05c0162cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.284+0000 7f05c546b640 1 --2- 172.21.15.45:0/2004137306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05c00d20c0 0x7f05c0162cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.284+0000 7f05c546b640 1 -- 172.21.15.45:0/2004137306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c00b7b70 con 0x7f05c0070d30 2024-05-08T05:51:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.284+0000 7f05c5c6c640 1 --2- 172.21.15.45:0/2004137306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05c00d20c0 0x7f05c0162cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.284+0000 7f05c546b640 1 --2- 172.21.15.45:0/2004137306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c0070d30 0x7f05c0154180 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f05b000ba50 tx=0x7f05b000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.284+0000 7f05bdffb640 1 -- 172.21.15.45:0/2004137306 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0019070 con 0x7f05c0070d30 2024-05-08T05:51:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.284+0000 7f05bdffb640 1 -- 172.21.15.45:0/2004137306 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b000cc20 con 0x7f05c0070d30 2024-05-08T05:51:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.284+0000 7f05bdffb640 1 -- 172.21.15.45:0/2004137306 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0014680 con 0x7f05c0070d30 2024-05-08T05:51:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.284+0000 7f05c76cf640 1 -- 172.21.15.45:0/2004137306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c00b8530 con 0x7f05c0070d30 2024-05-08T05:51:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.284+0000 7f05c76cf640 1 -- 172.21.15.45:0/2004137306 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c00b8af0 con 0x7f05c0070d30 2024-05-08T05:51:15.287 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.286+0000 7f05bdffb640 1 -- 172.21.15.45:0/2004137306 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f05b001c070 con 0x7f05c0070d30 2024-05-08T05:51:15.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.286+0000 7f05bdffb640 1 --2- 172.21.15.45:0/2004137306 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f05ac0762d0 0x7f05ac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.286+0000 7f05bdffb640 1 -- 172.21.15.45:0/2004137306 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f05b00973d0 con 0x7f05c0070d30 2024-05-08T05:51:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.287+0000 7f05c4c6a640 1 --2- 172.21.15.45:0/2004137306 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f05ac0762d0 0x7f05ac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.287+0000 7f05c4c6a640 1 --2- 172.21.15.45:0/2004137306 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f05ac0762d0 0x7f05ac078790 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f05b4034000 tx=0x7f05b402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.287+0000 7f05ca3e1740 1 Processor -- start 2024-05-08T05:51:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.287+0000 7f05ca3e1740 1 -- start start 2024-05-08T05:51:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.288+0000 7f05ca3e1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938df50 0x5628e9390760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.288+0000 7f05ca3e1740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5628e938e330 0x5628e9390ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.288+0000 7f05ca3e1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e9360bb0 0x5628e93643c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.288+0000 7f05ca3e1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5628e92ca8d0 con 0x5628e938df50 2024-05-08T05:51:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.288+0000 7f05ca3e1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5628e9283c10 con 0x5628e9360bb0 2024-05-08T05:51:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.288+0000 7f05ca3e1740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5628e92c6890 con 0x5628e938e330 2024-05-08T05:51:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.288+0000 7f05c546b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938df50 0x5628e9390760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.288+0000 7f05c546b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938df50 0x5628e9390760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45726/0 (socket says 172.21.15.45:45726) 2024-05-08T05:51:15.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.288+0000 7f05c546b640 1 -- 172.21.15.45:0/863649784 learned_addr learned my addr 172.21.15.45:0/863649784 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.288+0000 7f05c4c6a640 1 --2- 172.21.15.45:0/863649784 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5628e938e330 0x5628e9390ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.288+0000 7f05c5c6c640 1 --2- 172.21.15.45:0/863649784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e9360bb0 0x5628e93643c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.288+0000 7f05c546b640 1 -- 172.21.15.45:0/863649784 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5628e938e330 msgr2=0x5628e9390ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.288+0000 7f05c546b640 1 --2- 172.21.15.45:0/863649784 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5628e938e330 0x5628e9390ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.288+0000 7f05c546b640 1 -- 172.21.15.45:0/863649784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e9360bb0 msgr2=0x5628e93643c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.288+0000 7f05c546b640 1 --2- 172.21.15.45:0/863649784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e9360bb0 0x5628e93643c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.288+0000 7f05c546b640 1 -- 172.21.15.45:0/863649784 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628e928b5d0 con 0x5628e938df50 2024-05-08T05:51:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.288+0000 7f05c5c6c640 1 --2- 172.21.15.45:0/863649784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e9360bb0 0x5628e93643c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.289+0000 7f05c546b640 1 --2- 172.21.15.45:0/863649784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938df50 0x5628e9390760 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f05b005cb40 tx=0x7f05b005c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d97d3229f22d2b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.289+0000 7f05a6ffd640 1 -- 172.21.15.45:0/863649784 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0064850 con 0x5628e938df50 2024-05-08T05:51:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.289+0000 7f05a6ffd640 1 -- 172.21.15.45:0/863649784 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b0069270 con 0x5628e938df50 2024-05-08T05:51:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.289+0000 7f05a6ffd640 1 -- 172.21.15.45:0/863649784 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b000ce10 con 0x5628e938df50 2024-05-08T05:51:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.289+0000 7f05ca3e1740 1 -- 172.21.15.45:0/863649784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938df50 msgr2=0x5628e9390760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:15.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.289+0000 7f05ca3e1740 1 --2- 172.21.15.45:0/863649784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938df50 0x5628e9390760 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f05b005cb40 tx=0x7f05b005c4c0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.289+0000 7f05ca3e1740 1 -- 172.21.15.45:0/863649784 shutdown_connections 2024-05-08T05:51:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.289+0000 7f05ca3e1740 1 --2- 172.21.15.45:0/863649784 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5628e938e330 0x5628e9390ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.289+0000 7f05ca3e1740 1 --2- 172.21.15.45:0/863649784 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e9360bb0 0x5628e93643c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.289+0000 7f05ca3e1740 1 --2- 172.21.15.45:0/863649784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938df50 0x5628e9390760 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.289+0000 7f05ca3e1740 1 -- 172.21.15.45:0/863649784 >> 172.21.15.45:0/863649784 conn(0x5628e9362e20 msgr2=0x5628e93ab760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.289+0000 7f05ca3e1740 1 -- 172.21.15.45:0/863649784 shutdown_connections 2024-05-08T05:51:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.289+0000 7f05ca3e1740 1 -- 172.21.15.45:0/863649784 wait complete. 2024-05-08T05:51:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.290+0000 7f05ca3e1740 1 -- start start 2024-05-08T05:51:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.290+0000 7f05ca3e1740 1 Processor -- start 2024-05-08T05:51:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.291+0000 7f05ca3e1740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5628e938df50 0x5628e93b1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.291+0000 7f05ca3e1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e938e330 0x5628e93adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.291+0000 7f05ca3e1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e93ae3f0 0x5628e938aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.291+0000 7f05ca3e1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5628e9365fd0 con 0x5628e93ae3f0 2024-05-08T05:51:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.291+0000 7f05ca3e1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5628e9287760 con 0x5628e938e330 2024-05-08T05:51:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.291+0000 7f05ca3e1740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5628e93662b0 con 0x5628e938df50 2024-05-08T05:51:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.291+0000 7f05ca3e1740 10 client.0 _ll_register_callbacks cb 0x5628e92c0cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:51:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.291+0000 7f05c5c6c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e93ae3f0 0x5628e938aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.291+0000 7f05c5c6c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e93ae3f0 0x5628e938aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45734/0 (socket says 172.21.15.45:45734) 2024-05-08T05:51:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.291+0000 7f05c5c6c640 1 -- 172.21.15.45:0/3703370651 learned_addr learned my addr 172.21.15.45:0/3703370651 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.291+0000 7f05c546b640 1 --2- 172.21.15.45:0/3703370651 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5628e938df50 0x5628e93b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.291+0000 7f05c4c6a640 1 --2- 172.21.15.45:0/3703370651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e938e330 0x5628e93adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.291+0000 7f05c5c6c640 1 -- 172.21.15.45:0/3703370651 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5628e938df50 msgr2=0x5628e93b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.291+0000 7f05c5c6c640 1 --2- 172.21.15.45:0/3703370651 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5628e938df50 0x5628e93b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.291+0000 7f05c5c6c640 1 -- 172.21.15.45:0/3703370651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e938e330 msgr2=0x5628e93adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.291+0000 7f05c5c6c640 1 --2- 172.21.15.45:0/3703370651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e938e330 0x5628e93adeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.291+0000 7f05c5c6c640 1 -- 172.21.15.45:0/3703370651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628e8fdc2d0 con 0x5628e93ae3f0 2024-05-08T05:51:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.291+0000 7f05c546b640 1 --2- 172.21.15.45:0/3703370651 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5628e938df50 0x5628e93b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.291+0000 7f05c4c6a640 1 --2- 172.21.15.45:0/3703370651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e938e330 0x5628e93adeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.292+0000 7f05c5c6c640 1 --2- 172.21.15.45:0/3703370651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e93ae3f0 0x5628e938aff0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f05b800bfb0 tx=0x7f05b80116f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5e29566061f794 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.292+0000 7f059bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:51:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.292+0000 7f059bfff640 1 -- 172.21.15.45:0/3703370651 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8019070 con 0x5628e93ae3f0 2024-05-08T05:51:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.292+0000 7f059bfff640 1 -- 172.21.15.45:0/3703370651 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f05b8015650 con 0x5628e93ae3f0 2024-05-08T05:51:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.292+0000 7f05ca3e1740 1 -- 172.21.15.45:0/3703370651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05b0069270 con 0x5628e93ae3f0 2024-05-08T05:51:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.292+0000 7f059bfff640 1 -- 172.21.15.45:0/3703370651 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f05b8024050 con 0x5628e93ae3f0 2024-05-08T05:51:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.292+0000 7f05ca3e1740 10 client.8259 fetch_fsmap learned FSMap version 100 2024-05-08T05:51:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.292+0000 7f05ca3e1740 1 -- 172.21.15.45:0/3703370651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x5628e90a66b0 con 0x5628e93ae3f0 2024-05-08T05:51:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.292+0000 7f059bfff640 1 -- 172.21.15.45:0/3703370651 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b0069270 con 0x5628e93ae3f0 2024-05-08T05:51:15.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.292+0000 7f059bfff640 1 -- 172.21.15.45:0/3703370651 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7f05b8027020 con 0x5628e93ae3f0 2024-05-08T05:51:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.292+0000 7f05ca3e1740 10 client.8259 fetch_fsmap finished waiting for FSMap version 100 2024-05-08T05:51:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.292+0000 7f05ca3e1740 10 client.8259 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T05:51:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.292+0000 7f05ca3e1740 20 client.8259 populate_metadata read hostname 'smithi045' 2024-05-08T05:51:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.292+0000 7f05ca3e1740 1 --2- 172.21.15.45:0/3703370651 >> [v2:172.21.15.45:6834/1751419054,v1:172.21.15.45:6835/1751419054] conn(0x5628e9393bd0 0x5628e9391970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.292+0000 7f05ca3e1740 4 client.8259 mds_command: new command op to 8112 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:51:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.292+0000 7f05ca3e1740 1 -- 172.21.15.45:0/3703370651 --> [v2:172.21.15.45:6834/1751419054,v1:172.21.15.45:6835/1751419054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628e924e760 con 0x5628e9393bd0 2024-05-08T05:51:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.292+0000 7f05c546b640 1 --2- 172.21.15.45:0/3703370651 >> [v2:172.21.15.45:6834/1751419054,v1:172.21.15.45:6835/1751419054] conn(0x5628e9393bd0 0x5628e9391970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.293+0000 7f05c546b640 1 --2- 172.21.15.45:0/3703370651 >> [v2:172.21.15.45:6834/1751419054,v1:172.21.15.45:6835/1751419054] conn(0x5628e9393bd0 0x5628e9391970 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a89be67b84048c57 server_cookie=aaab4be17d5b0ff0 in_seq=0 out_seq=0 2024-05-08T05:51:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.293+0000 7f059bfff640 10 client.8259 ms_handle_connect on v2:172.21.15.45:6834/1751419054 2024-05-08T05:51:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.293+0000 7f059bfff640 1 -- 172.21.15.45:0/3703370651 <== mds.0 v2:172.21.15.45:6834/1751419054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5628e924e760 con 0x5628e9393bd0 2024-05-08T05:51:15.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.293+0000 7f059bfff640 10 client.8259 handle_command_reply: tid=0 2024-05-08T05:51:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.293+0000 7f05ca3e1740 1 client.8259 shutdown 2024-05-08T05:51:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.293+0000 7f05ca3e1740 1 -- 172.21.15.45:0/3703370651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e93ae3f0 msgr2=0x5628e938aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.293+0000 7f05ca3e1740 1 --2- 172.21.15.45:0/3703370651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e93ae3f0 0x5628e938aff0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f05b800bfb0 tx=0x7f05b80116f0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.294+0000 7f05ca3e1740 1 -- 172.21.15.45:0/3703370651 shutdown_connections 2024-05-08T05:51:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.294+0000 7f05ca3e1740 1 --2- 172.21.15.45:0/3703370651 >> [v2:172.21.15.45:6834/1751419054,v1:172.21.15.45:6835/1751419054] conn(0x5628e9393bd0 0x5628e9391970 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.294+0000 7f05ca3e1740 1 --2- 172.21.15.45:0/3703370651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e93ae3f0 0x5628e938aff0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.294+0000 7f05ca3e1740 1 --2- 172.21.15.45:0/3703370651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e938e330 0x5628e93adeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.294+0000 7f059bfff640 0 client.8259 ms_handle_reset on v2:172.21.15.45:6834/1751419054 2024-05-08T05:51:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.294+0000 7f05ca3e1740 1 --2- 172.21.15.45:0/3703370651 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5628e938df50 0x5628e93b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.294+0000 7f05ca3e1740 1 -- 172.21.15.45:0/3703370651 >> 172.21.15.45:0/3703370651 conn(0x5628e9360bb0 msgr2=0x5628e9361860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:15.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.294+0000 7f05ca3e1740 1 -- 172.21.15.45:0/3703370651 shutdown_connections 2024-05-08T05:51:15.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.294+0000 7f05ca3e1740 1 -- 172.21.15.45:0/3703370651 wait complete. 2024-05-08T05:51:15.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.294+0000 7f05ca3e1740 20 client.8259 trim_cache size 0 max 16384 2024-05-08T05:51:15.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.313+0000 7f05ca3e1740 1 Processor -- start 2024-05-08T05:51:15.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.313+0000 7f05ca3e1740 1 -- start start 2024-05-08T05:51:15.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.313+0000 7f05ca3e1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938df50 0x5628e93a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.313+0000 7f05ca3e1740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5628e938e330 0x5628e93adc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.313+0000 7f05ca3e1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e93ae3f0 0x5628e9361780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.313+0000 7f05ca3e1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f05c00ccb50 con 0x5628e938df50 2024-05-08T05:51:15.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.313+0000 7f05ca3e1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f05c0158f70 con 0x5628e93ae3f0 2024-05-08T05:51:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.313+0000 7f05ca3e1740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f05c00cc500 con 0x5628e938e330 2024-05-08T05:51:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.313+0000 7f05c546b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938df50 0x5628e93a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.313+0000 7f05c546b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938df50 0x5628e93a8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45740/0 (socket says 172.21.15.45:45740) 2024-05-08T05:51:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.313+0000 7f05c546b640 1 -- 172.21.15.45:0/4105220745 learned_addr learned my addr 172.21.15.45:0/4105220745 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.314+0000 7f05c546b640 1 -- 172.21.15.45:0/4105220745 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5628e938e330 msgr2=0x5628e93adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.314+0000 7f05c546b640 1 --2- 172.21.15.45:0/4105220745 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5628e938e330 0x5628e93adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.314+0000 7f05c546b640 1 -- 172.21.15.45:0/4105220745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e93ae3f0 msgr2=0x5628e9361780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.314+0000 7f05c546b640 1 --2- 172.21.15.45:0/4105220745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e93ae3f0 0x5628e9361780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.314+0000 7f05c546b640 1 -- 172.21.15.45:0/4105220745 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628e93b18c0 con 0x5628e938df50 2024-05-08T05:51:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.314+0000 7f05c546b640 1 --2- 172.21.15.45:0/4105220745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938df50 0x5628e93a8a80 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f05b005cb40 tx=0x7f05b0097000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9fcc8801daf8935 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.314+0000 7f05a67fc640 1 -- 172.21.15.45:0/4105220745 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0069270 con 0x5628e938df50 2024-05-08T05:51:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.314+0000 7f05a67fc640 1 -- 172.21.15.45:0/4105220745 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b0064850 con 0x5628e938df50 2024-05-08T05:51:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.314+0000 7f05a67fc640 1 -- 172.21.15.45:0/4105220745 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0068de0 con 0x5628e938df50 2024-05-08T05:51:15.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.315+0000 7f05ca3e1740 1 -- 172.21.15.45:0/4105220745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938df50 msgr2=0x5628e93a8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:15.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.315+0000 7f05ca3e1740 1 --2- 172.21.15.45:0/4105220745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938df50 0x5628e93a8a80 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f05b005cb40 tx=0x7f05b0097000 comp rx=0 tx=0).stop 2024-05-08T05:51:15.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.315+0000 7f05ca3e1740 1 -- 172.21.15.45:0/4105220745 shutdown_connections 2024-05-08T05:51:15.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.315+0000 7f05ca3e1740 1 --2- 172.21.15.45:0/4105220745 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5628e938e330 0x5628e93adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.315+0000 7f05ca3e1740 1 --2- 172.21.15.45:0/4105220745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e93ae3f0 0x5628e9361780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.315+0000 7f05ca3e1740 1 --2- 172.21.15.45:0/4105220745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938df50 0x5628e93a8a80 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.315+0000 7f05ca3e1740 1 -- 172.21.15.45:0/4105220745 >> 172.21.15.45:0/4105220745 conn(0x5628e9360bb0 msgr2=0x5628e9366be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:15.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.315+0000 7f05ca3e1740 1 -- 172.21.15.45:0/4105220745 shutdown_connections 2024-05-08T05:51:15.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.315+0000 7f05ca3e1740 1 -- 172.21.15.45:0/4105220745 wait complete. 2024-05-08T05:51:15.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.315+0000 7f05ca3e1740 1 -- start start 2024-05-08T05:51:15.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.315+0000 7f05ca3e1740 1 Processor -- start 2024-05-08T05:51:15.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.316+0000 7f05ca3e1740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5628e938df50 0x5628e93ac170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.316+0000 7f05ca3e1740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938e330 0x5628e94290d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.316+0000 7f05ca3e1740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e93ae3f0 0x5628e9391e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.316+0000 7f05ca3e1740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5628e9393a50 con 0x5628e938e330 2024-05-08T05:51:15.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.316+0000 7f05ca3e1740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5628e93b1ae0 con 0x5628e93ae3f0 2024-05-08T05:51:15.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.316+0000 7f05ca3e1740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5628e93940a0 con 0x5628e938df50 2024-05-08T05:51:15.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.316+0000 7f05ca3e1740 10 client.0 _ll_register_callbacks cb 0x7f05b800b280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:51:15.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.316+0000 7f05c4c6a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938e330 0x5628e94290d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.316+0000 7f05c4c6a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938e330 0x5628e94290d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45750/0 (socket says 172.21.15.45:45750) 2024-05-08T05:51:15.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.316+0000 7f05c4c6a640 1 -- 172.21.15.45:0/3570442316 learned_addr learned my addr 172.21.15.45:0/3570442316 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:15.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.317+0000 7f05c546b640 1 --2- 172.21.15.45:0/3570442316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5628e938df50 0x5628e93ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.317+0000 7f05c5c6c640 1 --2- 172.21.15.45:0/3570442316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e93ae3f0 0x5628e9391e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.317+0000 7f05c4c6a640 1 -- 172.21.15.45:0/3570442316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5628e938df50 msgr2=0x5628e93ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:15.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.317+0000 7f05c4c6a640 1 --2- 172.21.15.45:0/3570442316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5628e938df50 0x5628e93ac170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.317+0000 7f05c4c6a640 1 -- 172.21.15.45:0/3570442316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e93ae3f0 msgr2=0x5628e9391e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.317+0000 7f05c4c6a640 1 --2- 172.21.15.45:0/3570442316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e93ae3f0 0x5628e9391e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.317+0000 7f05c4c6a640 1 -- 172.21.15.45:0/3570442316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628e9390a50 con 0x5628e938e330 2024-05-08T05:51:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.317+0000 7f05c5c6c640 1 --2- 172.21.15.45:0/3570442316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e93ae3f0 0x5628e9391e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.317+0000 7f05c4c6a640 1 --2- 172.21.15.45:0/3570442316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938e330 0x5628e94290d0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f05b40090b0 tx=0x7f05b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72fc8192734d2a0a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.317+0000 7f059bfff640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:51:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.317+0000 7f059bfff640 1 -- 172.21.15.45:0/3570442316 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4037040 con 0x5628e938e330 2024-05-08T05:51:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.317+0000 7f059bfff640 1 -- 172.21.15.45:0/3570442316 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f05b403eac0 con 0x5628e938e330 2024-05-08T05:51:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.317+0000 7f05ca3e1740 1 -- 172.21.15.45:0/3570442316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05b0064850 con 0x5628e938e330 2024-05-08T05:51:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.317+0000 7f059bfff640 1 -- 172.21.15.45:0/3570442316 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f05b4054050 con 0x5628e938e330 2024-05-08T05:51:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.317+0000 7f05ca3e1740 10 client.8265 fetch_fsmap learned FSMap version 100 2024-05-08T05:51:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.317+0000 7f05ca3e1740 1 -- 172.21.15.45:0/3570442316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x5628e938ae80 con 0x5628e938e330 2024-05-08T05:51:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.317+0000 7f059bfff640 1 -- 172.21.15.45:0/3570442316 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b0064850 con 0x5628e938e330 2024-05-08T05:51:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.317+0000 7f059bfff640 1 -- 172.21.15.45:0/3570442316 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7f05b4057020 con 0x5628e938e330 2024-05-08T05:51:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f05ca3e1740 10 client.8265 fetch_fsmap finished waiting for FSMap version 100 2024-05-08T05:51:15.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f05ca3e1740 10 client.8265 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T05:51:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f05ca3e1740 20 client.8265 populate_metadata read hostname 'smithi045' 2024-05-08T05:51:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f05ca3e1740 1 --2- 172.21.15.45:0/3570442316 >> [v2:172.21.15.45:6834/1751419054,v1:172.21.15.45:6835/1751419054] conn(0x5628e935fa40 0x5628e935fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f05ca3e1740 4 client.8265 mds_command: new command op to 8112 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:51:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f05ca3e1740 1 -- 172.21.15.45:0/3570442316 --> [v2:172.21.15.45:6834/1751419054,v1:172.21.15.45:6835/1751419054] -- command(tid 0: {"prefix": "status"}) -- 0x5628e92d84e0 con 0x5628e935fa40 2024-05-08T05:51:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f05c546b640 1 --2- 172.21.15.45:0/3570442316 >> [v2:172.21.15.45:6834/1751419054,v1:172.21.15.45:6835/1751419054] conn(0x5628e935fa40 0x5628e935fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f059bfff640 10 client.8265 ms_handle_connect on v2:172.21.15.45:6834/1751419054 2024-05-08T05:51:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f05c546b640 1 --2- 172.21.15.45:0/3570442316 >> [v2:172.21.15.45:6834/1751419054,v1:172.21.15.45:6835/1751419054] conn(0x5628e935fa40 0x5628e935fe20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e48b01b73212e076 server_cookie=943567d02b91d5b1 in_seq=0 out_seq=0 2024-05-08T05:51:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f059bfff640 1 -- 172.21.15.45:0/3570442316 <== mds.0 v2:172.21.15.45:6834/1751419054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x5628e92d84e0 con 0x5628e935fa40 2024-05-08T05:51:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f059bfff640 10 client.8265 handle_command_reply: tid=0 2024-05-08T05:51:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f05ca3e1740 1 client.8265 shutdown 2024-05-08T05:51:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f05ca3e1740 1 -- 172.21.15.45:0/3570442316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938e330 msgr2=0x5628e94290d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f05ca3e1740 1 --2- 172.21.15.45:0/3570442316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938e330 0x5628e94290d0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f05b40090b0 tx=0x7f05b40046c0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f05ca3e1740 1 -- 172.21.15.45:0/3570442316 shutdown_connections 2024-05-08T05:51:15.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f05ca3e1740 1 --2- 172.21.15.45:0/3570442316 >> [v2:172.21.15.45:6834/1751419054,v1:172.21.15.45:6835/1751419054] conn(0x5628e935fa40 0x5628e935fe20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f05ca3e1740 1 --2- 172.21.15.45:0/3570442316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5628e93ae3f0 0x5628e9391e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f05ca3e1740 1 --2- 172.21.15.45:0/3570442316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5628e938e330 0x5628e94290d0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f059bfff640 0 client.8265 ms_handle_reset on v2:172.21.15.45:6834/1751419054 2024-05-08T05:51:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f05ca3e1740 1 --2- 172.21.15.45:0/3570442316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5628e938df50 0x5628e93ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f05ca3e1740 1 -- 172.21.15.45:0/3570442316 >> 172.21.15.45:0/3570442316 conn(0x5628e9360bb0 msgr2=0x5628e93cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.318+0000 7f05ca3e1740 1 -- 172.21.15.45:0/3570442316 shutdown_connections 2024-05-08T05:51:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.319+0000 7f05ca3e1740 1 -- 172.21.15.45:0/3570442316 wait complete. 2024-05-08T05:51:15.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.319+0000 7f05ca3e1740 20 client.8265 trim_cache size 0 max 16384 2024-05-08T05:51:15.325 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:51:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "e9e1170a-fd27-4625-97a9-5d5f104d13a2", 2024-05-08T05:51:15.325 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-05-08T05:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 8112, 2024-05-08T05:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-05-08T05:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-05-08T05:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-05-08T05:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 10.260434889000001, 2024-05-08T05:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 100, 2024-05-08T05:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 119, 2024-05-08T05:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 119, 2024-05-08T05:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 15.625371608 2024-05-08T05:51:15.326 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:51:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.319+0000 7f05c76cf640 1 -- 172.21.15.45:0/2004137306 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f05ac0762d0 msgr2=0x7f05ac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.319+0000 7f05c76cf640 1 --2- 172.21.15.45:0/2004137306 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f05ac0762d0 0x7f05ac078790 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f05b4034000 tx=0x7f05b402e000 comp rx=0 tx=0).stop 2024-05-08T05:51:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.319+0000 7f05c76cf640 1 -- 172.21.15.45:0/2004137306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c0070d30 msgr2=0x7f05c0154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.319+0000 7f05c76cf640 1 --2- 172.21.15.45:0/2004137306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c0070d30 0x7f05c0154180 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f05b000ba50 tx=0x7f05b000bf10 comp rx=0 tx=0).stop 2024-05-08T05:51:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.319+0000 7f05c76cf640 1 -- 172.21.15.45:0/2004137306 shutdown_connections 2024-05-08T05:51:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.319+0000 7f05c76cf640 1 --2- 172.21.15.45:0/2004137306 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f05ac0762d0 0x7f05ac078790 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.319+0000 7f05c76cf640 1 --2- 172.21.15.45:0/2004137306 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f05c0071110 0x7f05c00b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.319+0000 7f05c76cf640 1 --2- 172.21.15.45:0/2004137306 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f05c00d20c0 0x7f05c0162cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.319+0000 7f05c76cf640 1 --2- 172.21.15.45:0/2004137306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c0070d30 0x7f05c0154180 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.319+0000 7f05c76cf640 1 -- 172.21.15.45:0/2004137306 >> 172.21.15.45:0/2004137306 conn(0x7f05c00d2e90 msgr2=0x7f05c01612f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:15.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.319+0000 7f05c76cf640 1 -- 172.21.15.45:0/2004137306 shutdown_connections 2024-05-08T05:51:15.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.319+0000 7f05c76cf640 1 -- 172.21.15.45:0/2004137306 wait complete. 2024-05-08T05:51:15.330 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T05:51:15.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.435+0000 7fcbc2447640 1 Processor -- start 2024-05-08T05:51:15.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.435+0000 7fcbc2447640 1 -- start start 2024-05-08T05:51:15.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.436+0000 7fcbc2447640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcbbc06f060 0x7fcbbc06e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.436+0000 7fcbc2447640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc06fe30 0x7fcbbc0cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.436+0000 7fcbc2447640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbbc15d2a0 con 0x7fcbbc0d0360 2024-05-08T05:51:15.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.436+0000 7fcbc2447640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fcbbc0d07c0 con 0x7fcbbc06f060 2024-05-08T05:51:15.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.436+0000 7fcbc2447640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcbbc06f440 con 0x7fcbbc06fe30 2024-05-08T05:51:15.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.436+0000 7fcbc09e4640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fcbbc0d0360 0x7fcbbc0b3f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40548/0 (socket says 172.21.15.45:40548) 2024-05-08T05:51:15.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.436+0000 7fcbc09e4640 1 -- 172.21.15.45:0/476877855 learned_addr learned my addr 172.21.15.45:0/476877855 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:15.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.436+0000 7fcbbbfff640 1 --2- 172.21.15.45:0/476877855 >> v2:172.21.15.63:3300/0 conn(0x7fcbbc06f060 0x7fcbbc06e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.436+0000 7fcbbb7fe640 1 --2- 172.21.15.45:0/476877855 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc06fe30 0x7fcbbc0cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.436+0000 7fcbbaffd640 1 -- 172.21.15.45:0/476877855 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4232367626 0 0) 0x7fcbbc15d2a0 con 0x7fcbbc0d0360 2024-05-08T05:51:15.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.436+0000 7fcbbaffd640 1 -- 172.21.15.45:0/476877855 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcba4003520 con 0x7fcbbc0d0360 2024-05-08T05:51:15.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbbaffd640 1 -- 172.21.15.45:0/476877855 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4114700596 0 0) 0x7fcba4003520 con 0x7fcbbc0d0360 2024-05-08T05:51:15.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbbaffd640 1 -- 172.21.15.45:0/476877855 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc06fe30 msgr2=0x7fcbbc0cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:15.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbbaffd640 1 --2- 172.21.15.45:0/476877855 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc06fe30 0x7fcbbc0cfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbbaffd640 1 -- 172.21.15.45:0/476877855 >> v2:172.21.15.63:3300/0 conn(0x7fcbbc06f060 msgr2=0x7fcbbc06e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:15.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbbaffd640 1 --2- 172.21.15.45:0/476877855 >> v2:172.21.15.63:3300/0 conn(0x7fcbbc06f060 0x7fcbbc06e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbbaffd640 1 -- 172.21.15.45:0/476877855 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbbc1609b0 con 0x7fcbbc0d0360 2024-05-08T05:51:15.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbbaffd640 1 -- 172.21.15.45:0/476877855 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fcbb4001a70 con 0x7fcbbc0d0360 2024-05-08T05:51:15.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbbaffd640 1 -- 172.21.15.45:0/476877855 >> v1:172.21.15.45:6789/0 conn(0x7fcbbc0d0360 legacy=0x7fcbbc0b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:15.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbbaffd640 1 --2- 172.21.15.45:0/476877855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba4003a00 0x7fcba4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbbaffd640 1 --2- 172.21.15.45:0/476877855 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc06fe30 0x7fcba4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbbaffd640 1 --2- 172.21.15.45:0/476877855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcba4004d40 0x7fcba40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbbaffd640 1 -- 172.21.15.45:0/476877855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc06f440 con 0x7fcba4004d40 2024-05-08T05:51:15.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbbaffd640 1 -- 172.21.15.45:0/476877855 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcba400b9c0 con 0x7fcba4003a00 2024-05-08T05:51:15.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbbaffd640 1 -- 172.21.15.45:0/476877855 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcba400bb70 con 0x7fcbbc06fe30 2024-05-08T05:51:15.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbc09e4640 1 --2- 172.21.15.45:0/476877855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcba4004d40 0x7fcba40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbbbfff640 1 --2- 172.21.15.45:0/476877855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba4003a00 0x7fcba4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbbb7fe640 1 --2- 172.21.15.45:0/476877855 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc06fe30 0x7fcba4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbc09e4640 1 -- 172.21.15.45:0/476877855 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc06fe30 msgr2=0x7fcba4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:15.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbc09e4640 1 --2- 172.21.15.45:0/476877855 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc06fe30 0x7fcba4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbc09e4640 1 -- 172.21.15.45:0/476877855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba4003a00 msgr2=0x7fcba4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:15.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbc09e4640 1 --2- 172.21.15.45:0/476877855 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcba4003a00 0x7fcba4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.437+0000 7fcbc09e4640 1 -- 172.21.15.45:0/476877855 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba4003520 con 0x7fcba4004d40 2024-05-08T05:51:15.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.438+0000 7fcbc09e4640 1 --2- 172.21.15.45:0/476877855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcba4004d40 0x7fcba40051c0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400a920 tx=0x7fcba4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09aba13d9f51a30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:15.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.438+0000 7fcbbaffd640 1 -- 172.21.15.45:0/476877855 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb400a4c0 con 0x7fcba4004d40 2024-05-08T05:51:15.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.438+0000 7fcbbaffd640 1 -- 172.21.15.45:0/476877855 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb402f070 con 0x7fcba4004d40 2024-05-08T05:51:15.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.438+0000 7fcbbaffd640 1 -- 172.21.15.45:0/476877855 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb402bc50 con 0x7fcba4004d40 2024-05-08T05:51:15.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.438+0000 7fcbc2447640 1 -- 172.21.15.45:0/476877855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcba4004d40 msgr2=0x7fcba40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:15.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.438+0000 7fcbc2447640 1 --2- 172.21.15.45:0/476877855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcba4004d40 0x7fcba40051c0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400a920 tx=0x7fcba4005830 comp rx=0 tx=0).stop 2024-05-08T05:51:15.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.438+0000 7fcbc09e4640 1 -- 172.21.15.45:0/476877855 reap_dead start 2024-05-08T05:51:15.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.439+0000 7fcbc2447640 1 -- 172.21.15.45:0/476877855 shutdown_connections 2024-05-08T05:51:15.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.439+0000 7fcbc2447640 1 -- 172.21.15.45:0/476877855 >> 172.21.15.45:0/476877855 conn(0x7fcbbc06c3a0 msgr2=0x7fcbbc06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:15.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.439+0000 7fcbc2447640 1 -- 172.21.15.45:0/476877855 shutdown_connections 2024-05-08T05:51:15.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.439+0000 7fcbc2447640 1 -- 172.21.15.45:0/476877855 wait complete. 2024-05-08T05:51:15.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.440+0000 7fcbc2447640 1 Processor -- start 2024-05-08T05:51:15.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.440+0000 7fcbc2447640 1 -- start start 2024-05-08T05:51:15.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.441+0000 7fcbc2447640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbbc150890 0x7fcbbc15e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.441+0000 7fcbc2447640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbbc15f560 0x7fcbbc15f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.441+0000 7fcbc2447640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc1f6460 0x7fcbbc1f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.441+0000 7fcbc2447640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc161250 con 0x7fcbbc15f560 2024-05-08T05:51:15.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.441+0000 7fcbc2447640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc160f70 con 0x7fcbbc150890 2024-05-08T05:51:15.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.441+0000 7fcbc2447640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcbbc161530 con 0x7fcbbc1f6460 2024-05-08T05:51:15.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.441+0000 7fcbbb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbbc15f560 0x7fcbbc15f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.441+0000 7fcbbb7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbbc15f560 0x7fcbbc15f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45774/0 (socket says 172.21.15.45:45774) 2024-05-08T05:51:15.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.441+0000 7fcbbb7fe640 1 -- 172.21.15.45:0/3118322679 learned_addr learned my addr 172.21.15.45:0/3118322679 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:15.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.441+0000 7fcbbbfff640 1 --2- 172.21.15.45:0/3118322679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbbc150890 0x7fcbbc15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.441+0000 7fcbc09e4640 1 --2- 172.21.15.45:0/3118322679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc1f6460 0x7fcbbc1f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.441+0000 7fcbbb7fe640 1 -- 172.21.15.45:0/3118322679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc1f6460 msgr2=0x7fcbbc1f68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:15.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.441+0000 7fcbbb7fe640 1 --2- 172.21.15.45:0/3118322679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc1f6460 0x7fcbbc1f68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.441+0000 7fcbbb7fe640 1 -- 172.21.15.45:0/3118322679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbbc150890 msgr2=0x7fcbbc15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:15.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.441+0000 7fcbbb7fe640 1 --2- 172.21.15.45:0/3118322679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbbc150890 0x7fcbbc15e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.441+0000 7fcbbb7fe640 1 -- 172.21.15.45:0/3118322679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbbc150c70 con 0x7fcbbc15f560 2024-05-08T05:51:15.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.441+0000 7fcbbb7fe640 1 --2- 172.21.15.45:0/3118322679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbbc15f560 0x7fcbbc15f9e0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00119d0 tx=0x7fcbb0011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:15.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.442+0000 7fcbb8ff9640 1 -- 172.21.15.45:0/3118322679 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb000f9d0 con 0x7fcbbc15f560 2024-05-08T05:51:15.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.442+0000 7fcbb8ff9640 1 -- 172.21.15.45:0/3118322679 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb000fb70 con 0x7fcbbc15f560 2024-05-08T05:51:15.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.442+0000 7fcbb8ff9640 1 -- 172.21.15.45:0/3118322679 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0014440 con 0x7fcbbc15f560 2024-05-08T05:51:15.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.442+0000 7fcbc2447640 1 -- 172.21.15.45:0/3118322679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbbc1f2e90 con 0x7fcbbc15f560 2024-05-08T05:51:15.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.442+0000 7fcbc2447640 1 -- 172.21.15.45:0/3118322679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbbc0c6010 con 0x7fcbbc15f560 2024-05-08T05:51:15.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.443+0000 7fcbc2447640 1 -- 172.21.15.45:0/3118322679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbbc068cb0 con 0x7fcbbc15f560 2024-05-08T05:51:15.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.443+0000 7fcbb8ff9640 1 -- 172.21.15.45:0/3118322679 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcbb0012040 con 0x7fcbbc15f560 2024-05-08T05:51:15.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.444+0000 7fcbb8ff9640 1 --2- 172.21.15.45:0/3118322679 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcba80762d0 0x7fcba8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:15.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.444+0000 7fcbb8ff9640 1 -- 172.21.15.45:0/3118322679 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7fcbb0097c50 con 0x7fcbbc15f560 2024-05-08T05:51:15.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.444+0000 7fcbbbfff640 1 --2- 172.21.15.45:0/3118322679 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcba80762d0 0x7fcba8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:15.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.444+0000 7fcbbbfff640 1 --2- 172.21.15.45:0/3118322679 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcba80762d0 0x7fcba8078790 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009bc0 tx=0x7fcbac009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:15.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.448+0000 7fcbb8ff9640 1 -- 172.21.15.45:0/3118322679 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcbbc068cb0 con 0x7fcbbc15f560 2024-05-08T05:51:15.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.677+0000 7fcbc2447640 1 -- 172.21.15.45:0/3118322679 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcbbc0693f0 con 0x7fcbbc15f560 2024-05-08T05:51:15.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.678+0000 7fcbb8ff9640 1 -- 172.21.15.45:0/3118322679 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) ==== 53+0+73 (secure 0 0 0) 0x7fcbb009d020 con 0x7fcbbc15f560 2024-05-08T05:51:15.678 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T05:51:15.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.680+0000 7fcbc2447640 1 -- 172.21.15.45:0/3118322679 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcba80762d0 msgr2=0x7fcba8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:15.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.680+0000 7fcbc2447640 1 --2- 172.21.15.45:0/3118322679 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcba80762d0 0x7fcba8078790 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009bc0 tx=0x7fcbac009190 comp rx=0 tx=0).stop 2024-05-08T05:51:15.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.680+0000 7fcbc2447640 1 -- 172.21.15.45:0/3118322679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbbc15f560 msgr2=0x7fcbbc15f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:15.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.680+0000 7fcbc2447640 1 --2- 172.21.15.45:0/3118322679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbbc15f560 0x7fcbbc15f9e0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00119d0 tx=0x7fcbb0011dd0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.680+0000 7fcbc2447640 1 -- 172.21.15.45:0/3118322679 shutdown_connections 2024-05-08T05:51:15.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.680+0000 7fcbc2447640 1 --2- 172.21.15.45:0/3118322679 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcba80762d0 0x7fcba8078790 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.680+0000 7fcbc2447640 1 --2- 172.21.15.45:0/3118322679 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcbbc1f6460 0x7fcbbc1f68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.680+0000 7fcbc2447640 1 --2- 172.21.15.45:0/3118322679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbbc15f560 0x7fcbbc15f9e0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.680+0000 7fcbc2447640 1 --2- 172.21.15.45:0/3118322679 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcbbc150890 0x7fcbbc15e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:15.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.680+0000 7fcbc2447640 1 -- 172.21.15.45:0/3118322679 >> 172.21.15.45:0/3118322679 conn(0x7fcbbc06c3a0 msgr2=0x7fcbbc15c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:15.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.680+0000 7fcbc2447640 1 -- 172.21.15.45:0/3118322679 shutdown_connections 2024-05-08T05:51:15.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15.680+0000 7fcbc2447640 1 -- 172.21.15.45:0/3118322679 wait complete. 2024-05-08T05:51:15.691 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T05:51:15.691 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:51:15.691 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-05-08T05:51:15.691 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T05:51:15.691 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:51:15.691 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T05:51:15.691 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T05:51:15.691 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:51:15.691 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:51:15.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:15.692 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-05-08T05:51:15.708 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:51:15.708 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:51:15.708 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:51:15.708 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:15.708 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-05-08T05:51:15.709 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:15.709 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:51:15.709 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:51:15.709 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-05-08T05:51:15.709 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:51:15.709 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:51:15.709 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-05-08T05:51:15.709 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:51:15.709 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-05-08T05:51:15.709 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-05-08T05:51:15.709 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:51:15.709 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42149sec preferred_lft 42149sec 2024-05-08T05:51:15.710 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-05-08T05:51:15.710 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:15.710 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:51:15.710 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-05-08T05:51:15.710 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-05-08T05:51:15.710 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:51:15.710 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T05:51:15.710 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T05:51:15.710 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:15.710 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c80a:aff:fe94:c77/64 scope link 2024-05-08T05:51:15.710 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:15.711 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:51:15.711 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 82:f3:eb:6a:57:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:51:15.711 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::80f3:ebff:fe6a:57ba/64 scope link 2024-05-08T05:51:15.711 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:15.711 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:51:15.711 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T05:51:15.711 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::14a2:64ff:feda:3a9c/64 scope link 2024-05-08T05:51:15.711 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:15.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:15.712 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:51:15.773 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:51:15.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:15.775 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-05-08T05:51:15.832 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-05-08T05:51:15.832 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:51:15.832 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:51:15.832 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:51:15.832 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:51:15.832 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:51:15.833 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:51:15.833 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:51:15.833 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:51:15.833 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:51:15.833 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:51:15.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:51:15.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:15.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:15.962 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:51:15.978 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T05:51:15.978 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T05:51:15.978 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:51:15.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:15.979 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-05-08T05:51:16.044 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:51:16.044 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T05:51:16.101 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:51:16.101 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:51:16.101 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:51:16.156 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T05:51:16.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:16.157 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:51:16.242 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:51:16.242 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:51:16.253 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:51:16.253 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:51:16.253 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-08T05:51:16.253 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@e9e1170a-fd27-4625-97a9-5d5f104d13a2.cephfs=/ 2024-05-08T05:51:16.253 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.63:3300/172.21.15.63:3301" will pass to kernel 2024-05-08T05:51:16.254 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T05:51:16.254 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:51:16.254 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:51:16.254 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:51:16.254 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:51:16.259 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:51:16.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:16.259 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:51:16.286 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:51:16.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:51:16.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:16.391 INFO:teuthology.orchestra.run.smithi045.stdout:client8274 2024-05-08T05:51:16.392 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:51:16.392 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:51:16.408 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:16.408 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:16.408 DEBUG:teuthology.orchestra.run.smithi045:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T05:51:16.408 DEBUG:teuthology.orchestra.run.smithi045:> parent = os.path.join(mount_point, "parent") 2024-05-08T05:51:16.408 DEBUG:teuthology.orchestra.run.smithi045:> os.mkdir(parent) 2024-05-08T05:51:16.409 DEBUG:teuthology.orchestra.run.smithi045:> flushed = os.path.join(parent, "flushed") 2024-05-08T05:51:16.409 DEBUG:teuthology.orchestra.run.smithi045:> os.mkdir(flushed) 2024-05-08T05:51:16.409 DEBUG:teuthology.orchestra.run.smithi045:> for f in ["alpha", "bravo", "charlie"]: 2024-05-08T05:51:16.409 DEBUG:teuthology.orchestra.run.smithi045:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-05-08T05:51:16.409 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:16.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:16.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:16.535 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-08T05:51:16.596 INFO:teuthology.orchestra.run.smithi045.stdout:./ 2024-05-08T05:51:16.596 INFO:teuthology.orchestra.run.smithi045.stdout:./parent 2024-05-08T05:51:16.596 INFO:teuthology.orchestra.run.smithi045.stdout:./parent/flushed 2024-05-08T05:51:16.597 INFO:teuthology.orchestra.run.smithi045.stdout:./parent/flushed/charlie 2024-05-08T05:51:16.597 INFO:teuthology.orchestra.run.smithi045.stdout:./parent/flushed/alpha 2024-05-08T05:51:16.597 INFO:teuthology.orchestra.run.smithi045.stdout:./parent/flushed/bravo 2024-05-08T05:51:16.597 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:16.597 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:16.597 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:16.597 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:16.597 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-08T05:51:16.597 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:16.704 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:51:16.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:16.713 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:16.713 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:16.714 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:16.714 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:16.714 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-08T05:51:16.714 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:16.822 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627776 2024-05-08T05:51:16.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:16.830 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:16.830 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:16.831 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:16.831 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:16.831 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-08T05:51:16.831 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:16.939 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627777 2024-05-08T05:51:16.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:16.948 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:16.948 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:16.948 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:16.948 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:16.948 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-08T05:51:16.948 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:17.056 INFO:teuthology.orchestra.run.smithi045.stdout:1099511628280 2024-05-08T05:51:17.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:17.066 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:17.066 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:17.066 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:17.066 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:17.066 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-08T05:51:17.066 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:17.173 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627778 2024-05-08T05:51:17.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:17.182 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:17.182 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:17.182 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:17.183 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:17.183 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-05-08T05:51:17.183 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:17.291 INFO:teuthology.orchestra.run.smithi045.stdout:1099511628279 2024-05-08T05:51:17.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:17.302 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:51:17.302 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:51:17.359 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:51:17.359 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:51:17.416 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:51:17.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.562+0000 7fb41106b640 1 Processor -- start 2024-05-08T05:51:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.562+0000 7fb41106b640 1 -- start start 2024-05-08T05:51:17.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.562+0000 7fb41106b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb40c1638e0 0x7fb40c0b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.562+0000 7fb41106b640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb40c06c3a0 0x7fb40c06c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.562+0000 7fb41106b640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb40c0b14e0 con 0x7fb40c15f0c0 2024-05-08T05:51:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.562+0000 7fb41106b640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fb40c163450 con 0x7fb40c06c3a0 2024-05-08T05:51:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.562+0000 7fb41106b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb40c1604b0 con 0x7fb40c1638e0 2024-05-08T05:51:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.563+0000 7fb40b59e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb40c06c3a0 0x7fb40c06c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.563+0000 7fb40b59e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fb40c06c3a0 0x7fb40c06c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45776/0 (socket says 172.21.15.45:45776) 2024-05-08T05:51:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.563+0000 7fb40b59e640 1 -- 172.21.15.45:0/2857244239 learned_addr learned my addr 172.21.15.45:0/2857244239 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.563+0000 7fb40a59c640 1 --2- 172.21.15.45:0/2857244239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb40c1638e0 0x7fb40c0b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.563+0000 7fb40b59e640 1 -- 172.21.15.45:0/2857244239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb40c1638e0 msgr2=0x7fb40c0b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.563+0000 7fb40b59e640 1 --2- 172.21.15.45:0/2857244239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb40c1638e0 0x7fb40c0b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.563+0000 7fb40b59e640 1 -- 172.21.15.45:0/2857244239 >> v1:172.21.15.63:6789/0 conn(0x7fb40c15f0c0 legacy=0x7fb40c0b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:17.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.563+0000 7fb40b59e640 1 -- 172.21.15.45:0/2857244239 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c162e00 con 0x7fb40c06c3a0 2024-05-08T05:51:17.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.564+0000 7fb40b59e640 1 --2- 172.21.15.45:0/2857244239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c06c3a0 0x7fb40c06c830 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fb404009e50 tx=0x7fb40402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc88bd687877d3e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:17.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.564+0000 7fb409d9b640 1 -- 172.21.15.45:0/2857244239 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404036070 con 0x7fb40c06c3a0 2024-05-08T05:51:17.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.564+0000 7fb409d9b640 1 -- 172.21.15.45:0/2857244239 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb404031600 con 0x7fb40c06c3a0 2024-05-08T05:51:17.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.564+0000 7fb409d9b640 1 -- 172.21.15.45:0/2857244239 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4040319d0 con 0x7fb40c06c3a0 2024-05-08T05:51:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.564+0000 7fb41106b640 1 -- 172.21.15.45:0/2857244239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c06c3a0 msgr2=0x7fb40c06c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.564+0000 7fb41106b640 1 --2- 172.21.15.45:0/2857244239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c06c3a0 0x7fb40c06c830 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fb404009e50 tx=0x7fb40402a710 comp rx=0 tx=0).stop 2024-05-08T05:51:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.564+0000 7fb41106b640 1 -- 172.21.15.45:0/2857244239 shutdown_connections 2024-05-08T05:51:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.564+0000 7fb41106b640 1 --2- 172.21.15.45:0/2857244239 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c06c3a0 0x7fb40c06c830 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.564+0000 7fb41106b640 1 --2- 172.21.15.45:0/2857244239 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb40c1638e0 0x7fb40c0b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.564+0000 7fb41106b640 1 -- 172.21.15.45:0/2857244239 >> 172.21.15.45:0/2857244239 conn(0x7fb40c0b3b10 msgr2=0x7fb40c0b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.564+0000 7fb41106b640 1 -- 172.21.15.45:0/2857244239 shutdown_connections 2024-05-08T05:51:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.564+0000 7fb41106b640 1 -- 172.21.15.45:0/2857244239 wait complete. 2024-05-08T05:51:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.566+0000 7fb41106b640 1 Processor -- start 2024-05-08T05:51:17.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.566+0000 7fb41106b640 1 -- start start 2024-05-08T05:51:17.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb41106b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb40c06c3a0 0x7fb40c14bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:17.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb41106b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c15f0c0 0x7fb40c161a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:17.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb41106b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb40c1638e0 0x7fb40c161f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:17.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb41106b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c15f4a0 con 0x7fb40c15f0c0 2024-05-08T05:51:17.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb41106b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c0b17c0 con 0x7fb40c1638e0 2024-05-08T05:51:17.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb41106b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fb40c15f620 con 0x7fb40c06c3a0 2024-05-08T05:51:17.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb40a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c15f0c0 0x7fb40c161a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:17.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb40a59c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c15f0c0 0x7fb40c161a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45784/0 (socket says 172.21.15.45:45784) 2024-05-08T05:51:17.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb40a59c640 1 -- 172.21.15.45:0/2670276316 learned_addr learned my addr 172.21.15.45:0/2670276316 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:17.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb40ad9d640 1 --2- 172.21.15.45:0/2670276316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb40c06c3a0 0x7fb40c14bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:17.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb40b59e640 1 --2- 172.21.15.45:0/2670276316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb40c1638e0 0x7fb40c161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:17.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb40a59c640 1 -- 172.21.15.45:0/2670276316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb40c06c3a0 msgr2=0x7fb40c14bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:17.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb40a59c640 1 --2- 172.21.15.45:0/2670276316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb40c06c3a0 0x7fb40c14bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb40a59c640 1 -- 172.21.15.45:0/2670276316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb40c1638e0 msgr2=0x7fb40c161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb40a59c640 1 --2- 172.21.15.45:0/2670276316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb40c1638e0 0x7fb40c161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb40a59c640 1 -- 172.21.15.45:0/2670276316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c14f480 con 0x7fb40c15f0c0 2024-05-08T05:51:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb40b59e640 1 --2- 172.21.15.45:0/2670276316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb40c1638e0 0x7fb40c161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb40a59c640 1 --2- 172.21.15.45:0/2670276316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c15f0c0 0x7fb40c161a30 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fb40000d5d0 tx=0x7fb40000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb3f37fe640 1 -- 172.21.15.45:0/2670276316 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4000125f0 con 0x7fb40c15f0c0 2024-05-08T05:51:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb3f37fe640 1 -- 172.21.15.45:0/2670276316 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb400019070 con 0x7fb40c15f0c0 2024-05-08T05:51:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb41106b640 1 -- 172.21.15.45:0/2670276316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40c14c840 con 0x7fb40c15f0c0 2024-05-08T05:51:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb3f37fe640 1 -- 172.21.15.45:0/2670276316 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4000129f0 con 0x7fb40c15f0c0 2024-05-08T05:51:17.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.567+0000 7fb41106b640 1 -- 172.21.15.45:0/2670276316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40c14ce00 con 0x7fb40c15f0c0 2024-05-08T05:51:17.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.568+0000 7fb41106b640 1 -- 172.21.15.45:0/2670276316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb40c06a8f0 con 0x7fb40c15f0c0 2024-05-08T05:51:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.569+0000 7fb3f37fe640 1 -- 172.21.15.45:0/2670276316 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb400012b90 con 0x7fb40c15f0c0 2024-05-08T05:51:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.570+0000 7fb3f37fe640 1 --2- 172.21.15.45:0/2670276316 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb3f80762d0 0x7fb3f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.570+0000 7fb3f37fe640 1 -- 172.21.15.45:0/2670276316 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7fb400097b80 con 0x7fb40c15f0c0 2024-05-08T05:51:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.570+0000 7fb40ad9d640 1 --2- 172.21.15.45:0/2670276316 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb3f80762d0 0x7fb3f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:17.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.571+0000 7fb40ad9d640 1 --2- 172.21.15.45:0/2670276316 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb3f80762d0 0x7fb3f8078790 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0017c0 tx=0x7fb3fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:17.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.574+0000 7fb3f37fe640 1 -- 172.21.15.45:0/2670276316 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb400061350 con 0x7fb40c15f0c0 2024-05-08T05:51:17.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.778+0000 7fb41106b640 1 -- 172.21.15.45:0/2670276316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb40c0693f0 con 0x7fb40c15f0c0 2024-05-08T05:51:17.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.778+0000 7fb3f37fe640 1 -- 172.21.15.45:0/2670276316 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v119) ==== 80+0+0 (secure 0 0 0) 0x7fb400060c50 con 0x7fb40c15f0c0 2024-05-08T05:51:17.778 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-05-08T05:51:17.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.780+0000 7fb41106b640 1 -- 172.21.15.45:0/2670276316 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb3f80762d0 msgr2=0x7fb3f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:17.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.780+0000 7fb41106b640 1 --2- 172.21.15.45:0/2670276316 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb3f80762d0 0x7fb3f8078790 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0017c0 tx=0x7fb3fc009040 comp rx=0 tx=0).stop 2024-05-08T05:51:17.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.780+0000 7fb41106b640 1 -- 172.21.15.45:0/2670276316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c15f0c0 msgr2=0x7fb40c161a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:17.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.780+0000 7fb41106b640 1 --2- 172.21.15.45:0/2670276316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c15f0c0 0x7fb40c161a30 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fb40000d5d0 tx=0x7fb40000da90 comp rx=0 tx=0).stop 2024-05-08T05:51:17.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.780+0000 7fb41106b640 1 -- 172.21.15.45:0/2670276316 shutdown_connections 2024-05-08T05:51:17.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.780+0000 7fb41106b640 1 --2- 172.21.15.45:0/2670276316 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fb3f80762d0 0x7fb3f8078790 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:17.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.780+0000 7fb41106b640 1 --2- 172.21.15.45:0/2670276316 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb40c1638e0 0x7fb40c161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:17.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.780+0000 7fb41106b640 1 --2- 172.21.15.45:0/2670276316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb40c15f0c0 0x7fb40c161a30 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:17.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.780+0000 7fb41106b640 1 --2- 172.21.15.45:0/2670276316 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fb40c06c3a0 0x7fb40c14bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:17.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.780+0000 7fb41106b640 1 -- 172.21.15.45:0/2670276316 >> 172.21.15.45:0/2670276316 conn(0x7fb40c0b3b10 msgr2=0x7fb40c160b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:17.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.780+0000 7fb41106b640 1 -- 172.21.15.45:0/2670276316 shutdown_connections 2024-05-08T05:51:17.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:17.780+0000 7fb41106b640 1 -- 172.21.15.45:0/2670276316 wait complete. 2024-05-08T05:51:17.791 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T05:51:17.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:17.791 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T05:51:17.843 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:51:17.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:17.843 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T05:51:17.874 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-05-08T05:51:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.019+0000 7f8669340640 1 Processor -- start 2024-05-08T05:51:18.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.019+0000 7f8669340640 1 -- start start 2024-05-08T05:51:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.020+0000 7f8669340640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f86640d3cd0 0x7f86640cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.020+0000 7f8669340640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86640d3330 con 0x7f86640c9ee0 2024-05-08T05:51:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.020+0000 7f8669340640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86640d2170 con 0x7f86641629a0 2024-05-08T05:51:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.020+0000 7f8669340640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f86640d2450 con 0x7f86640d3cd0 2024-05-08T05:51:18.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.020+0000 7f86637fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f86640c9ee0 0x7f86640d01e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40550/0 (socket says 172.21.15.45:40550) 2024-05-08T05:51:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.020+0000 7f86637fe640 1 -- 172.21.15.45:0/440263858 learned_addr learned my addr 172.21.15.45:0/440263858 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f86627fc640 1 --2- 172.21.15.45:0/440263858 >> v2:172.21.15.63:3300/0 conn(0x7f86640d3cd0 0x7f86640cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f8661ffb640 1 -- 172.21.15.45:0/440263858 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4090545704 0 0) 0x7f86640d3330 con 0x7f86640c9ee0 2024-05-08T05:51:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f8661ffb640 1 -- 172.21.15.45:0/440263858 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8650003520 con 0x7f86640c9ee0 2024-05-08T05:51:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f8661ffb640 1 -- 172.21.15.45:0/440263858 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1701649665 0 0) 0x7f8650003520 con 0x7f86640c9ee0 2024-05-08T05:51:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f8661ffb640 1 -- 172.21.15.45:0/440263858 >> v2:172.21.15.63:3300/0 conn(0x7f86640d3cd0 msgr2=0x7f86640cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f8661ffb640 1 --2- 172.21.15.45:0/440263858 >> v2:172.21.15.63:3300/0 conn(0x7f86640d3cd0 0x7f86640cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f8661ffb640 1 -- 172.21.15.45:0/440263858 >> v1:172.21.15.63:6789/0 conn(0x7f86641629a0 legacy=0x7f86640d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f8661ffb640 1 -- 172.21.15.45:0/440263858 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86640d2730 con 0x7f86640c9ee0 2024-05-08T05:51:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f8661ffb640 1 -- 172.21.15.45:0/440263858 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f86540016f0 con 0x7f86640c9ee0 2024-05-08T05:51:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f8661ffb640 1 -- 172.21.15.45:0/440263858 >> v1:172.21.15.45:6789/0 conn(0x7f86640c9ee0 legacy=0x7f86640d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f8661ffb640 1 --2- 172.21.15.45:0/440263858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8650003a50 0x7f8650003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f8661ffb640 1 --2- 172.21.15.45:0/440263858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8650004800 0x7f8650004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f8661ffb640 1 --2- 172.21.15.45:0/440263858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86500053c0 0x7f865000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f8661ffb640 1 -- 172.21.15.45:0/440263858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f86640d2450 con 0x7f86500053c0 2024-05-08T05:51:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f8661ffb640 1 -- 172.21.15.45:0/440263858 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f865000c230 con 0x7f8650004800 2024-05-08T05:51:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f8661ffb640 1 -- 172.21.15.45:0/440263858 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f865000c430 con 0x7f8650003a50 2024-05-08T05:51:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f86637fe640 1 --2- 172.21.15.45:0/440263858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86500053c0 0x7f865000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f8662ffd640 1 --2- 172.21.15.45:0/440263858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8650003a50 0x7f8650003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f86627fc640 1 --2- 172.21.15.45:0/440263858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8650004800 0x7f8650004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f86637fe640 1 -- 172.21.15.45:0/440263858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8650003a50 msgr2=0x7f8650003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f86637fe640 1 --2- 172.21.15.45:0/440263858 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8650003a50 0x7f8650003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f86637fe640 1 -- 172.21.15.45:0/440263858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8650004800 msgr2=0x7f8650004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f86637fe640 1 --2- 172.21.15.45:0/440263858 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8650004800 0x7f8650004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.021+0000 7f86637fe640 1 -- 172.21.15.45:0/440263858 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8650003520 con 0x7f86500053c0 2024-05-08T05:51:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.022+0000 7f86637fe640 1 -- 172.21.15.45:0/440263858 reap_dead start 2024-05-08T05:51:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.022+0000 7f86637fe640 1 --2- 172.21.15.45:0/440263858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86500053c0 0x7f865000b9c0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f865400c240 tx=0x7f8650005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75413b92468bb6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.022+0000 7f8661ffb640 1 -- 172.21.15.45:0/440263858 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654007ce0 con 0x7f86500053c0 2024-05-08T05:51:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.022+0000 7f8661ffb640 1 -- 172.21.15.45:0/440263858 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8654005b10 con 0x7f86500053c0 2024-05-08T05:51:18.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.022+0000 7f8661ffb640 1 -- 172.21.15.45:0/440263858 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654007070 con 0x7f86500053c0 2024-05-08T05:51:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.023+0000 7f8669340640 1 -- 172.21.15.45:0/440263858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86500053c0 msgr2=0x7f865000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:18.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.023+0000 7f8669340640 1 --2- 172.21.15.45:0/440263858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86500053c0 0x7f865000b9c0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f865400c240 tx=0x7f8650005830 comp rx=0 tx=0).stop 2024-05-08T05:51:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.023+0000 7f8669340640 1 -- 172.21.15.45:0/440263858 shutdown_connections 2024-05-08T05:51:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.023+0000 7f8669340640 1 --2- 172.21.15.45:0/440263858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86500053c0 0x7f865000b9c0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.023+0000 7f8669340640 1 -- 172.21.15.45:0/440263858 >> 172.21.15.45:0/440263858 conn(0x7f86640d4fe0 msgr2=0x7f86640d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.023+0000 7f8669340640 1 -- 172.21.15.45:0/440263858 shutdown_connections 2024-05-08T05:51:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.023+0000 7f8669340640 1 -- 172.21.15.45:0/440263858 wait complete. 2024-05-08T05:51:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.024+0000 7f8669340640 1 Processor -- start 2024-05-08T05:51:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.024+0000 7f8669340640 1 -- start start 2024-05-08T05:51:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.025+0000 7f8669340640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86500053c0 0x7f8664157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.025+0000 7f8669340640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86640ce280 0x7f86640ce700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.025+0000 7f8669340640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86641590d0 0x7f8664159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.025+0000 7f8669340640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f86640cf440 con 0x7f86641590d0 2024-05-08T05:51:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.025+0000 7f8669340640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86640cf2c0 con 0x7f86640ce280 2024-05-08T05:51:18.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.025+0000 7f8669340640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f86640d2dc0 con 0x7f86500053c0 2024-05-08T05:51:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.025+0000 7f86637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86641590d0 0x7f8664159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.025+0000 7f86637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86641590d0 0x7f8664159580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45810/0 (socket says 172.21.15.45:45810) 2024-05-08T05:51:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.025+0000 7f86637fe640 1 -- 172.21.15.45:0/3118251847 learned_addr learned my addr 172.21.15.45:0/3118251847 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.025+0000 7f8662ffd640 1 --2- 172.21.15.45:0/3118251847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86500053c0 0x7f8664157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.025+0000 7f86637fe640 1 -- 172.21.15.45:0/3118251847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86500053c0 msgr2=0x7f8664157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.025+0000 7f86637fe640 1 --2- 172.21.15.45:0/3118251847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86500053c0 0x7f8664157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.025+0000 7f86637fe640 1 -- 172.21.15.45:0/3118251847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86640ce280 msgr2=0x7f86640ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.025+0000 7f86627fc640 1 --2- 172.21.15.45:0/3118251847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86640ce280 0x7f86640ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.026+0000 7f86637fe640 1 --2- 172.21.15.45:0/3118251847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86640ce280 0x7f86640ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.026+0000 7f86637fe640 1 -- 172.21.15.45:0/3118251847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8664159ac0 con 0x7f86641590d0 2024-05-08T05:51:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.026+0000 7f86637fe640 1 --2- 172.21.15.45:0/3118251847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86641590d0 0x7f8664159580 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f865400c360 tx=0x7f86640d2340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.026+0000 7f865b7fe640 1 -- 172.21.15.45:0/3118251847 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654001700 con 0x7f86641590d0 2024-05-08T05:51:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.026+0000 7f865b7fe640 1 -- 172.21.15.45:0/3118251847 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8654040070 con 0x7f86641590d0 2024-05-08T05:51:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.026+0000 7f8669340640 1 -- 172.21.15.45:0/3118251847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8664156950 con 0x7f86641590d0 2024-05-08T05:51:18.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.026+0000 7f865b7fe640 1 -- 172.21.15.45:0/3118251847 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86540047e0 con 0x7f86641590d0 2024-05-08T05:51:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.026+0000 7f8669340640 1 -- 172.21.15.45:0/3118251847 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f866415a010 con 0x7f86641590d0 2024-05-08T05:51:18.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.028+0000 7f865b7fe640 1 -- 172.21.15.45:0/3118251847 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f865403a070 con 0x7f86641590d0 2024-05-08T05:51:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.028+0000 7f865b7fe640 1 --2- 172.21.15.45:0/3118251847 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8644076300 0x7f86440787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.028+0000 7f865b7fe640 1 -- 172.21.15.45:0/3118251847 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f86540b6ec0 con 0x7f86641590d0 2024-05-08T05:51:18.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.028+0000 7f8662ffd640 1 --2- 172.21.15.45:0/3118251847 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8644076300 0x7f86440787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.029+0000 7f8662ffd640 1 --2- 172.21.15.45:0/3118251847 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8644076300 0x7f86440787c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f865c003ef0 tx=0x7f865c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.029+0000 7f866c05a740 1 Processor -- start 2024-05-08T05:51:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.029+0000 7f866c05a740 1 -- start start 2024-05-08T05:51:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.030+0000 7f866c05a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654bb962160 0x5654bb914af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.030+0000 7f866c05a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb945280 0x5654bb944600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.030+0000 7f866c05a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb944b40 0x5654bb919950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.030+0000 7f866c05a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5654bb8e1690 con 0x5654bb944b40 2024-05-08T05:51:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.030+0000 7f866c05a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5654bb87ef30 con 0x5654bb962160 2024-05-08T05:51:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.030+0000 7f866c05a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5654bb837b20 con 0x5654bb945280 2024-05-08T05:51:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.030+0000 7f86637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb944b40 0x5654bb919950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.030+0000 7f86637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb944b40 0x5654bb919950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45814/0 (socket says 172.21.15.45:45814) 2024-05-08T05:51:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.030+0000 7f86637fe640 1 -- 172.21.15.45:0/949962648 learned_addr learned my addr 172.21.15.45:0/949962648 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.030+0000 7f8662ffd640 1 --2- 172.21.15.45:0/949962648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654bb962160 0x5654bb914af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.030+0000 7f86627fc640 1 --2- 172.21.15.45:0/949962648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb945280 0x5654bb944600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.030+0000 7f86637fe640 1 -- 172.21.15.45:0/949962648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb945280 msgr2=0x5654bb944600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.030+0000 7f86637fe640 1 --2- 172.21.15.45:0/949962648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb945280 0x5654bb944600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.030+0000 7f86637fe640 1 -- 172.21.15.45:0/949962648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654bb962160 msgr2=0x5654bb914af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.030+0000 7f86637fe640 1 --2- 172.21.15.45:0/949962648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654bb962160 0x5654bb914af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.030+0000 7f86637fe640 1 -- 172.21.15.45:0/949962648 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654bb82fe60 con 0x5654bb944b40 2024-05-08T05:51:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.030+0000 7f86637fe640 1 --2- 172.21.15.45:0/949962648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb944b40 0x5654bb919950 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f865407fe90 tx=0x7f865402ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6906b837cc375a84 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.031+0000 7f8658ff9640 1 -- 172.21.15.45:0/949962648 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86540b3070 con 0x5654bb944b40 2024-05-08T05:51:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.031+0000 7f8658ff9640 1 -- 172.21.15.45:0/949962648 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f865407fa10 con 0x5654bb944b40 2024-05-08T05:51:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.031+0000 7f8658ff9640 1 -- 172.21.15.45:0/949962648 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654088ad0 con 0x5654bb944b40 2024-05-08T05:51:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.031+0000 7f866c05a740 1 -- 172.21.15.45:0/949962648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb944b40 msgr2=0x5654bb919950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.031+0000 7f866c05a740 1 --2- 172.21.15.45:0/949962648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb944b40 0x5654bb919950 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f865407fe90 tx=0x7f865402ad10 comp rx=0 tx=0).stop 2024-05-08T05:51:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.031+0000 7f866c05a740 1 -- 172.21.15.45:0/949962648 shutdown_connections 2024-05-08T05:51:18.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.031+0000 7f866c05a740 1 --2- 172.21.15.45:0/949962648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb945280 0x5654bb944600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.031+0000 7f866c05a740 1 --2- 172.21.15.45:0/949962648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb944b40 0x5654bb919950 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.031+0000 7f866c05a740 1 --2- 172.21.15.45:0/949962648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654bb962160 0x5654bb914af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.031+0000 7f866c05a740 1 -- 172.21.15.45:0/949962648 >> 172.21.15.45:0/949962648 conn(0x5654bb916d60 msgr2=0x5654bb95f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.031+0000 7f866c05a740 1 -- 172.21.15.45:0/949962648 shutdown_connections 2024-05-08T05:51:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.031+0000 7f866c05a740 1 -- 172.21.15.45:0/949962648 wait complete. 2024-05-08T05:51:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.032+0000 7f866c05a740 1 -- start start 2024-05-08T05:51:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.032+0000 7f866c05a740 1 Processor -- start 2024-05-08T05:51:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f866c05a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654bb945280 0x5654bb934e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f866c05a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb962160 0x5654bb96f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f866c05a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb93ff70 0x5654bb9472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f866c05a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5654bb83b5e0 con 0x5654bb93ff70 2024-05-08T05:51:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f866c05a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5654bb87aef0 con 0x5654bb945280 2024-05-08T05:51:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f866c05a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5654bb87ef30 con 0x5654bb962160 2024-05-08T05:51:18.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f866c05a740 10 client.0 _ll_register_callbacks cb 0x5654bb836320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:51:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f86637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb93ff70 0x5654bb9472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f86637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb93ff70 0x5654bb9472c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45822/0 (socket says 172.21.15.45:45822) 2024-05-08T05:51:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f86637fe640 1 -- 172.21.15.45:0/4059470547 learned_addr learned my addr 172.21.15.45:0/4059470547 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f8662ffd640 1 --2- 172.21.15.45:0/4059470547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654bb945280 0x5654bb934e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f86627fc640 1 --2- 172.21.15.45:0/4059470547 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb962160 0x5654bb96f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f86637fe640 1 -- 172.21.15.45:0/4059470547 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb962160 msgr2=0x5654bb96f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f86637fe640 1 --2- 172.21.15.45:0/4059470547 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb962160 0x5654bb96f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f86637fe640 1 -- 172.21.15.45:0/4059470547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654bb945280 msgr2=0x5654bb934e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f86637fe640 1 --2- 172.21.15.45:0/4059470547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654bb945280 0x5654bb934e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f86637fe640 1 -- 172.21.15.45:0/4059470547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654bb65a7d0 con 0x5654bb93ff70 2024-05-08T05:51:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f86627fc640 1 --2- 172.21.15.45:0/4059470547 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb962160 0x5654bb96f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f86637fe640 1 --2- 172.21.15.45:0/4059470547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb93ff70 0x5654bb9472c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f86540b52b0 tx=0x7f8654083000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c697089eb5453 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:18.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f8641ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:51:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.033+0000 7f8641ffb640 1 -- 172.21.15.45:0/4059470547 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86540b3040 con 0x5654bb93ff70 2024-05-08T05:51:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.034+0000 7f8641ffb640 1 -- 172.21.15.45:0/4059470547 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f86540888d0 con 0x5654bb93ff70 2024-05-08T05:51:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.034+0000 7f866c05a740 1 -- 172.21.15.45:0/4059470547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f865407fa10 con 0x5654bb93ff70 2024-05-08T05:51:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.034+0000 7f8641ffb640 1 -- 172.21.15.45:0/4059470547 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f865407ff00 con 0x5654bb93ff70 2024-05-08T05:51:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.034+0000 7f866c05a740 10 client.8292 fetch_fsmap learned FSMap version 100 2024-05-08T05:51:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.034+0000 7f866c05a740 1 -- 172.21.15.45:0/4059470547 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x5654bb5903f0 con 0x5654bb93ff70 2024-05-08T05:51:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.034+0000 7f8641ffb640 1 -- 172.21.15.45:0/4059470547 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f865407fa10 con 0x5654bb93ff70 2024-05-08T05:51:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.034+0000 7f8641ffb640 1 -- 172.21.15.45:0/4059470547 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7f865407f160 con 0x5654bb93ff70 2024-05-08T05:51:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.034+0000 7f866c05a740 10 client.8292 fetch_fsmap finished waiting for FSMap version 100 2024-05-08T05:51:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.034+0000 7f866c05a740 10 client.8292 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-05-08T05:51:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.034+0000 7f866c05a740 20 client.8292 populate_metadata read hostname 'smithi045' 2024-05-08T05:51:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.034+0000 7f866c05a740 1 --2- 172.21.15.45:0/4059470547 >> [v2:172.21.15.45:6834/1751419054,v1:172.21.15.45:6835/1751419054] conn(0x5654bb947800 0x5654bb919870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.034+0000 7f866c05a740 4 client.8292 mds_command: new command op to 8112 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:51:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.034+0000 7f866c05a740 1 -- 172.21.15.45:0/4059470547 --> [v2:172.21.15.45:6834/1751419054,v1:172.21.15.45:6835/1751419054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654bb88cb40 con 0x5654bb947800 2024-05-08T05:51:18.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.034+0000 7f8662ffd640 1 --2- 172.21.15.45:0/4059470547 >> [v2:172.21.15.45:6834/1751419054,v1:172.21.15.45:6835/1751419054] conn(0x5654bb947800 0x5654bb919870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.035+0000 7f8641ffb640 10 client.8292 ms_handle_connect on v2:172.21.15.45:6834/1751419054 2024-05-08T05:51:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.035+0000 7f8662ffd640 1 --2- 172.21.15.45:0/4059470547 >> [v2:172.21.15.45:6834/1751419054,v1:172.21.15.45:6835/1751419054] conn(0x5654bb947800 0x5654bb919870 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0bab2129f1ab9e9 server_cookie=ff720831b12bcc9d in_seq=0 out_seq=0 2024-05-08T05:51:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.035+0000 7f8641ffb640 1 -- 172.21.15.45:0/4059470547 <== mds.0 v2:172.21.15.45:6834/1751419054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5654bb88cb40 con 0x5654bb947800 2024-05-08T05:51:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.035+0000 7f8641ffb640 10 client.8292 handle_command_reply: tid=0 2024-05-08T05:51:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.035+0000 7f866c05a740 1 client.8292 shutdown 2024-05-08T05:51:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.035+0000 7f866c05a740 1 -- 172.21.15.45:0/4059470547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb93ff70 msgr2=0x5654bb9472c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.035+0000 7f866c05a740 1 --2- 172.21.15.45:0/4059470547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb93ff70 0x5654bb9472c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f86540b52b0 tx=0x7f8654083000 comp rx=0 tx=0).stop 2024-05-08T05:51:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.035+0000 7f866c05a740 1 -- 172.21.15.45:0/4059470547 shutdown_connections 2024-05-08T05:51:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.035+0000 7f866c05a740 1 --2- 172.21.15.45:0/4059470547 >> [v2:172.21.15.45:6834/1751419054,v1:172.21.15.45:6835/1751419054] conn(0x5654bb947800 0x5654bb919870 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.035+0000 7f866c05a740 1 --2- 172.21.15.45:0/4059470547 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb962160 0x5654bb96f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.035+0000 7f866c05a740 1 --2- 172.21.15.45:0/4059470547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb93ff70 0x5654bb9472c0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.035+0000 7f866c05a740 1 --2- 172.21.15.45:0/4059470547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654bb945280 0x5654bb934e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.035+0000 7f8641ffb640 0 client.8292 ms_handle_reset on v2:172.21.15.45:6834/1751419054 2024-05-08T05:51:18.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.035+0000 7f866c05a740 1 -- 172.21.15.45:0/4059470547 >> 172.21.15.45:0/4059470547 conn(0x5654bb944b40 msgr2=0x5654bb914230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.035+0000 7f866c05a740 1 -- 172.21.15.45:0/4059470547 shutdown_connections 2024-05-08T05:51:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.035+0000 7f866c05a740 1 -- 172.21.15.45:0/4059470547 wait complete. 2024-05-08T05:51:18.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.035+0000 7f866c05a740 20 client.8292 trim_cache size 0 max 16384 2024-05-08T05:51:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.051+0000 7f866c05a740 1 Processor -- start 2024-05-08T05:51:18.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.051+0000 7f866c05a740 1 -- start start 2024-05-08T05:51:18.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.051+0000 7f866c05a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb93ff70 0x5654bb96f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.051+0000 7f866c05a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654bb945280 0x5654bb9790c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.051+0000 7f866c05a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb962160 0x5654bb91a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.051+0000 7f866c05a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8664202510 con 0x5654bb962160 2024-05-08T05:51:18.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.051+0000 7f866c05a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f86642014f0 con 0x5654bb945280 2024-05-08T05:51:18.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.051+0000 7f866c05a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f86640cdc70 con 0x5654bb93ff70 2024-05-08T05:51:18.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.051+0000 7f86637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb962160 0x5654bb91a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.051+0000 7f86637fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb962160 0x5654bb91a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45838/0 (socket says 172.21.15.45:45838) 2024-05-08T05:51:18.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.051+0000 7f86637fe640 1 -- 172.21.15.45:0/682782811 learned_addr learned my addr 172.21.15.45:0/682782811 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:18.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.052+0000 7f86637fe640 1 -- 172.21.15.45:0/682782811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb93ff70 msgr2=0x5654bb96f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:18.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.052+0000 7f86637fe640 1 --2- 172.21.15.45:0/682782811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb93ff70 0x5654bb96f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.052+0000 7f86637fe640 1 -- 172.21.15.45:0/682782811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654bb945280 msgr2=0x5654bb9790c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:18.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.052+0000 7f86637fe640 1 --2- 172.21.15.45:0/682782811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654bb945280 0x5654bb9790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.052+0000 7f86637fe640 1 -- 172.21.15.45:0/682782811 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654bb919be0 con 0x5654bb962160 2024-05-08T05:51:18.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.052+0000 7f86637fe640 1 --2- 172.21.15.45:0/682782811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb962160 0x5654bb91a7e0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f865407fe90 tx=0x7f8654097000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90e2a3cfdb8dbc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.052+0000 7f8643fff640 1 -- 172.21.15.45:0/682782811 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86540b3040 con 0x5654bb962160 2024-05-08T05:51:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.052+0000 7f8643fff640 1 -- 172.21.15.45:0/682782811 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f865407fa10 con 0x5654bb962160 2024-05-08T05:51:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.052+0000 7f8643fff640 1 -- 172.21.15.45:0/682782811 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86540842b0 con 0x5654bb962160 2024-05-08T05:51:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.052+0000 7f866c05a740 1 -- 172.21.15.45:0/682782811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb962160 msgr2=0x5654bb91a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.052+0000 7f866c05a740 1 --2- 172.21.15.45:0/682782811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb962160 0x5654bb91a7e0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f865407fe90 tx=0x7f8654097000 comp rx=0 tx=0).stop 2024-05-08T05:51:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.053+0000 7f866c05a740 1 -- 172.21.15.45:0/682782811 shutdown_connections 2024-05-08T05:51:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.053+0000 7f866c05a740 1 --2- 172.21.15.45:0/682782811 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb962160 0x5654bb91a7e0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.053+0000 7f866c05a740 1 --2- 172.21.15.45:0/682782811 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654bb945280 0x5654bb9790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.053+0000 7f866c05a740 1 --2- 172.21.15.45:0/682782811 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb93ff70 0x5654bb96f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.053+0000 7f866c05a740 1 -- 172.21.15.45:0/682782811 >> 172.21.15.45:0/682782811 conn(0x5654bb944b40 msgr2=0x5654bb944100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.053+0000 7f866c05a740 1 -- 172.21.15.45:0/682782811 shutdown_connections 2024-05-08T05:51:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.053+0000 7f866c05a740 1 -- 172.21.15.45:0/682782811 wait complete. 2024-05-08T05:51:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.053+0000 7f866c05a740 1 -- start start 2024-05-08T05:51:18.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.053+0000 7f866c05a740 1 Processor -- start 2024-05-08T05:51:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.054+0000 7f866c05a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb93ff70 0x5654bb96f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.054+0000 7f866c05a740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb945280 0x5654bb932bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.054+0000 7f866c05a740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654bb962160 0x5654bb9551d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.054+0000 7f866c05a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5654bb8e1690 con 0x5654bb93ff70 2024-05-08T05:51:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.054+0000 7f866c05a740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8664200e80 con 0x5654bb962160 2024-05-08T05:51:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.054+0000 7f866c05a740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5654bb947ef0 con 0x5654bb945280 2024-05-08T05:51:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.054+0000 7f866c05a740 10 client.0 _ll_register_callbacks cb 0x7f86540b1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:51:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.054+0000 7f8662ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb93ff70 0x5654bb96f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.054+0000 7f8662ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb93ff70 0x5654bb96f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45850/0 (socket says 172.21.15.45:45850) 2024-05-08T05:51:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.054+0000 7f8662ffd640 1 -- 172.21.15.45:0/775786092 learned_addr learned my addr 172.21.15.45:0/775786092 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.054+0000 7f86627fc640 1 --2- 172.21.15.45:0/775786092 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb945280 0x5654bb932bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.054+0000 7f86637fe640 1 --2- 172.21.15.45:0/775786092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654bb962160 0x5654bb9551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.054+0000 7f8662ffd640 1 -- 172.21.15.45:0/775786092 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb945280 msgr2=0x5654bb932bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.054+0000 7f8662ffd640 1 --2- 172.21.15.45:0/775786092 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb945280 0x5654bb932bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.054+0000 7f8662ffd640 1 -- 172.21.15.45:0/775786092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654bb962160 msgr2=0x5654bb9551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:18.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.054+0000 7f8662ffd640 1 --2- 172.21.15.45:0/775786092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654bb962160 0x5654bb9551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.054+0000 7f8662ffd640 1 -- 172.21.15.45:0/775786092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654bb5500f0 con 0x5654bb93ff70 2024-05-08T05:51:18.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.054+0000 7f86627fc640 1 --2- 172.21.15.45:0/775786092 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb945280 0x5654bb932bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:18.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.055+0000 7f8662ffd640 1 --2- 172.21.15.45:0/775786092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb93ff70 0x5654bb96f470 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x5654bb5903f0 tx=0x7f865c001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec94de549105971 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:18.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.055+0000 7f8641ffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:51:18.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.055+0000 7f8641ffb640 1 -- 172.21.15.45:0/775786092 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c006b50 con 0x5654bb93ff70 2024-05-08T05:51:18.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.055+0000 7f8641ffb640 1 -- 172.21.15.45:0/775786092 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f865c006cf0 con 0x5654bb93ff70 2024-05-08T05:51:18.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.055+0000 7f866c05a740 1 -- 172.21.15.45:0/775786092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f865407fa10 con 0x5654bb93ff70 2024-05-08T05:51:18.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.055+0000 7f8641ffb640 1 -- 172.21.15.45:0/775786092 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f865c007050 con 0x5654bb93ff70 2024-05-08T05:51:18.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.055+0000 7f866c05a740 10 client.8298 fetch_fsmap learned FSMap version 100 2024-05-08T05:51:18.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.055+0000 7f866c05a740 1 -- 172.21.15.45:0/775786092 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x5654bb933130 con 0x5654bb93ff70 2024-05-08T05:51:18.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.055+0000 7f8641ffb640 1 -- 172.21.15.45:0/775786092 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f865407fa10 con 0x5654bb93ff70 2024-05-08T05:51:18.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.055+0000 7f8641ffb640 1 -- 172.21.15.45:0/775786092 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 100) ==== 3494+0+0 (secure 0 0 0) 0x7f865c0074c0 con 0x5654bb93ff70 2024-05-08T05:51:18.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.055+0000 7f866c05a740 10 client.8298 fetch_fsmap finished waiting for FSMap version 100 2024-05-08T05:51:18.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.055+0000 7f866c05a740 10 client.8298 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-05-08T05:51:18.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.055+0000 7f866c05a740 20 client.8298 populate_metadata read hostname 'smithi045' 2024-05-08T05:51:18.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.055+0000 7f866c05a740 1 --2- 172.21.15.45:0/775786092 >> [v2:172.21.15.45:6834/1751419054,v1:172.21.15.45:6835/1751419054] conn(0x5654bb955710 0x5654bb93ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.055+0000 7f866c05a740 4 client.8298 mds_command: new command op to 8112 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:51:18.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.055+0000 7f866c05a740 1 -- 172.21.15.45:0/775786092 --> [v2:172.21.15.45:6834/1751419054,v1:172.21.15.45:6835/1751419054] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5654bb8869b0 con 0x5654bb955710 2024-05-08T05:51:18.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.055+0000 7f86627fc640 1 --2- 172.21.15.45:0/775786092 >> [v2:172.21.15.45:6834/1751419054,v1:172.21.15.45:6835/1751419054] conn(0x5654bb955710 0x5654bb93ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.056+0000 7f86627fc640 1 --2- 172.21.15.45:0/775786092 >> [v2:172.21.15.45:6834/1751419054,v1:172.21.15.45:6835/1751419054] conn(0x5654bb955710 0x5654bb93ec90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34a76d59655b21ae server_cookie=15ded52fd98187f6 in_seq=0 out_seq=0 2024-05-08T05:51:18.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.056+0000 7f8641ffb640 10 client.8298 ms_handle_connect on v2:172.21.15.45:6834/1751419054 2024-05-08T05:51:18.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.064+0000 7f8641ffb640 1 -- 172.21.15.45:0/775786092 <== mds.0 v2:172.21.15.45:6834/1751419054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5654bb8869b0 con 0x5654bb955710 2024-05-08T05:51:18.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.064+0000 7f8641ffb640 10 client.8298 handle_command_reply: tid=0 2024-05-08T05:51:18.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.064+0000 7f866c05a740 1 client.8298 shutdown 2024-05-08T05:51:18.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.064+0000 7f866c05a740 1 -- 172.21.15.45:0/775786092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb93ff70 msgr2=0x5654bb96f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:18.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.064+0000 7f866c05a740 1 --2- 172.21.15.45:0/775786092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb93ff70 0x5654bb96f470 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x5654bb5903f0 tx=0x7f865c001690 comp rx=0 tx=0).stop 2024-05-08T05:51:18.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.064+0000 7f866c05a740 1 -- 172.21.15.45:0/775786092 shutdown_connections 2024-05-08T05:51:18.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.064+0000 7f866c05a740 1 --2- 172.21.15.45:0/775786092 >> [v2:172.21.15.45:6834/1751419054,v1:172.21.15.45:6835/1751419054] conn(0x5654bb955710 0x5654bb93ec90 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.064+0000 7f866c05a740 1 --2- 172.21.15.45:0/775786092 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5654bb945280 0x5654bb932bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.064+0000 7f866c05a740 1 --2- 172.21.15.45:0/775786092 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5654bb962160 0x5654bb9551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.064+0000 7f866c05a740 1 --2- 172.21.15.45:0/775786092 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5654bb93ff70 0x5654bb96f470 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.064+0000 7f8641ffb640 0 client.8298 ms_handle_reset on v2:172.21.15.45:6834/1751419054 2024-05-08T05:51:18.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.064+0000 7f866c05a740 1 -- 172.21.15.45:0/775786092 >> 172.21.15.45:0/775786092 conn(0x5654bb944b40 msgr2=0x5654bb95f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:18.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.064+0000 7f866c05a740 1 -- 172.21.15.45:0/775786092 shutdown_connections 2024-05-08T05:51:18.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.064+0000 7f866c05a740 1 -- 172.21.15.45:0/775786092 wait complete. 2024-05-08T05:51:18.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.064+0000 7f866c05a740 20 client.8298 trim_cache size 0 max 16384 2024-05-08T05:51:18.114 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:51:18.114 INFO:teuthology.orchestra.run.smithi045.stdout: "message": "", 2024-05-08T05:51:18.114 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0 2024-05-08T05:51:18.115 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:51:18.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.065+0000 7f8669340640 1 -- 172.21.15.45:0/3118251847 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8644076300 msgr2=0x7f86440787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:18.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.065+0000 7f8669340640 1 --2- 172.21.15.45:0/3118251847 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8644076300 0x7f86440787c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f865c003ef0 tx=0x7f865c009040 comp rx=0 tx=0).stop 2024-05-08T05:51:18.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.065+0000 7f8669340640 1 -- 172.21.15.45:0/3118251847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86641590d0 msgr2=0x7f8664159580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:18.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.065+0000 7f8669340640 1 --2- 172.21.15.45:0/3118251847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86641590d0 0x7f8664159580 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f865400c360 tx=0x7f86640d2340 comp rx=0 tx=0).stop 2024-05-08T05:51:18.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.065+0000 7f8669340640 1 -- 172.21.15.45:0/3118251847 shutdown_connections 2024-05-08T05:51:18.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.065+0000 7f8669340640 1 --2- 172.21.15.45:0/3118251847 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8644076300 0x7f86440787c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.065+0000 7f8669340640 1 --2- 172.21.15.45:0/3118251847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f86641590d0 0x7f8664159580 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.065+0000 7f8669340640 1 --2- 172.21.15.45:0/3118251847 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f86640ce280 0x7f86640ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.065+0000 7f8669340640 1 --2- 172.21.15.45:0/3118251847 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f86500053c0 0x7f8664157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.065+0000 7f8669340640 1 -- 172.21.15.45:0/3118251847 >> 172.21.15.45:0/3118251847 conn(0x7f86640d4fe0 msgr2=0x7f86640cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:18.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.065+0000 7f8669340640 1 -- 172.21.15.45:0/3118251847 shutdown_connections 2024-05-08T05:51:18.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.065+0000 7f8669340640 1 -- 172.21.15.45:0/3118251847 wait complete. 2024-05-08T05:51:18.118 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T05:51:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.221+0000 7fd3323b6640 1 Processor -- start 2024-05-08T05:51:18.221 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.221+0000 7fd3323b6640 1 -- start start 2024-05-08T05:51:18.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.222+0000 7fd3323b6640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fd32c0cf8a0 0x7fd32c15e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.222+0000 7fd3323b6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd32c0d39b0 0x7fd32c0cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.222+0000 7fd3323b6640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd32c0d01c0 con 0x7fd32c0d0f10 2024-05-08T05:51:18.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.222+0000 7fd3323b6640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fd32c15d5f0 con 0x7fd32c0d39b0 2024-05-08T05:51:18.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.222+0000 7fd3323b6640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fd32c161920 con 0x7fd32c0cf8a0 2024-05-08T05:51:18.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.222+0000 7fd32b7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd32c0d39b0 0x7fd32c0cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.222+0000 7fd330953640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fd32c0d0f10 0x7fd32c0d1360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40552/0 (socket says 172.21.15.45:40552) 2024-05-08T05:51:18.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.222+0000 7fd32b7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fd32c0d39b0 0x7fd32c0cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45854/0 (socket says 172.21.15.45:45854) 2024-05-08T05:51:18.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.222+0000 7fd330953640 1 -- 172.21.15.45:0/760403685 learned_addr learned my addr 172.21.15.45:0/760403685 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:18.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.223+0000 7fd32bfff640 1 --2- 172.21.15.45:0/760403685 >> v2:172.21.15.63:3300/0 conn(0x7fd32c0cf8a0 0x7fd32c15e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.223+0000 7fd32b7fe640 1 -- 172.21.15.45:0/760403685 >> v2:172.21.15.63:3300/0 conn(0x7fd32c0cf8a0 msgr2=0x7fd32c15e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:18.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.223+0000 7fd32b7fe640 1 --2- 172.21.15.45:0/760403685 >> v2:172.21.15.63:3300/0 conn(0x7fd32c0cf8a0 0x7fd32c15e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.223+0000 7fd32b7fe640 1 -- 172.21.15.45:0/760403685 >> v1:172.21.15.45:6789/0 conn(0x7fd32c0d0f10 legacy=0x7fd32c0d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:18.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.223+0000 7fd32b7fe640 1 -- 172.21.15.45:0/760403685 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32c160d10 con 0x7fd32c0d39b0 2024-05-08T05:51:18.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.223+0000 7fd32affd640 1 -- 172.21.15.45:0/760403685 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2385891850 0 0) 0x7fd32c0d01c0 con 0x7fd32c0d0f10 2024-05-08T05:51:18.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.223+0000 7fd32bfff640 1 --2- 172.21.15.45:0/760403685 >> v2:172.21.15.63:3300/0 conn(0x7fd32c0cf8a0 0x7fd32c15e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:18.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.223+0000 7fd32b7fe640 1 --2- 172.21.15.45:0/760403685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32c0d39b0 0x7fd32c0cfc80 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0701d0 tx=0x7fd32002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4a126d94db1040 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:18.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.224+0000 7fd32affd640 1 -- 172.21.15.45:0/760403685 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320037070 con 0x7fd32c0d39b0 2024-05-08T05:51:18.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.224+0000 7fd32affd640 1 -- 172.21.15.45:0/760403685 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd320032440 con 0x7fd32c0d39b0 2024-05-08T05:51:18.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.224+0000 7fd32affd640 1 -- 172.21.15.45:0/760403685 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320032810 con 0x7fd32c0d39b0 2024-05-08T05:51:18.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.224+0000 7fd3323b6640 1 -- 172.21.15.45:0/760403685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32c0d39b0 msgr2=0x7fd32c0cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:18.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.224+0000 7fd3323b6640 1 --2- 172.21.15.45:0/760403685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32c0d39b0 0x7fd32c0cfc80 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0701d0 tx=0x7fd32002a1e0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.224+0000 7fd3323b6640 1 -- 172.21.15.45:0/760403685 shutdown_connections 2024-05-08T05:51:18.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.224+0000 7fd3323b6640 1 --2- 172.21.15.45:0/760403685 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32c0d39b0 0x7fd32c0cfc80 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.224+0000 7fd3323b6640 1 --2- 172.21.15.45:0/760403685 >> v2:172.21.15.63:3300/0 conn(0x7fd32c0cf8a0 0x7fd32c15e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.224+0000 7fd3323b6640 1 -- 172.21.15.45:0/760403685 >> 172.21.15.45:0/760403685 conn(0x7fd32c0d2e90 msgr2=0x7fd32c0d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:18.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.224+0000 7fd3323b6640 1 -- 172.21.15.45:0/760403685 shutdown_connections 2024-05-08T05:51:18.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.224+0000 7fd3323b6640 1 -- 172.21.15.45:0/760403685 wait complete. 2024-05-08T05:51:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.225+0000 7fd3323b6640 1 Processor -- start 2024-05-08T05:51:18.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.226+0000 7fd3323b6640 1 -- start start 2024-05-08T05:51:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.226+0000 7fd3323b6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32c0cf8a0 0x7fd32c155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.226+0000 7fd3323b6640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd32c0d0f10 0x7fd32c1e8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.226+0000 7fd3323b6640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd32c0d39b0 0x7fd32c1e89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.226+0000 7fd3323b6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c1612d0 con 0x7fd32c0cf8a0 2024-05-08T05:51:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.226+0000 7fd3323b6640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c0d0700 con 0x7fd32c0d0f10 2024-05-08T05:51:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.226+0000 7fd3323b6640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fd32c1615b0 con 0x7fd32c0d39b0 2024-05-08T05:51:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.226+0000 7fd32bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32c0cf8a0 0x7fd32c155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.226+0000 7fd32bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32c0cf8a0 0x7fd32c155020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45864/0 (socket says 172.21.15.45:45864) 2024-05-08T05:51:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.226+0000 7fd32bfff640 1 -- 172.21.15.45:0/3242590500 learned_addr learned my addr 172.21.15.45:0/3242590500 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.227+0000 7fd32bfff640 1 -- 172.21.15.45:0/3242590500 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd32c0d39b0 msgr2=0x7fd32c1e89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.227+0000 7fd32bfff640 1 --2- 172.21.15.45:0/3242590500 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd32c0d39b0 0x7fd32c1e89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.227+0000 7fd32bfff640 1 -- 172.21.15.45:0/3242590500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd32c0d0f10 msgr2=0x7fd32c1e8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.227+0000 7fd32bfff640 1 --2- 172.21.15.45:0/3242590500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd32c0d0f10 0x7fd32c1e8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.227+0000 7fd32bfff640 1 -- 172.21.15.45:0/3242590500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32c0b6ba0 con 0x7fd32c0cf8a0 2024-05-08T05:51:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.227+0000 7fd32bfff640 1 --2- 172.21.15.45:0/3242590500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32c0cf8a0 0x7fd32c155020 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00c7f0 tx=0x7fd31c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:18.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.227+0000 7fd328ff9640 1 -- 172.21.15.45:0/3242590500 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c015070 con 0x7fd32c0cf8a0 2024-05-08T05:51:18.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.227+0000 7fd328ff9640 1 -- 172.21.15.45:0/3242590500 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31c007820 con 0x7fd32c0cf8a0 2024-05-08T05:51:18.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.227+0000 7fd3323b6640 1 -- 172.21.15.45:0/3242590500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32c0b6e80 con 0x7fd32c0cf8a0 2024-05-08T05:51:18.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.227+0000 7fd328ff9640 1 -- 172.21.15.45:0/3242590500 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c007bd0 con 0x7fd32c0cf8a0 2024-05-08T05:51:18.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.227+0000 7fd3323b6640 1 -- 172.21.15.45:0/3242590500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd32c0b7440 con 0x7fd32c0cf8a0 2024-05-08T05:51:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.228+0000 7fd3323b6640 1 -- 172.21.15.45:0/3242590500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd32c0b7820 con 0x7fd32c0cf8a0 2024-05-08T05:51:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.229+0000 7fd328ff9640 1 -- 172.21.15.45:0/3242590500 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd31c018070 con 0x7fd32c0cf8a0 2024-05-08T05:51:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.229+0000 7fd328ff9640 1 --2- 172.21.15.45:0/3242590500 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd3180762d0 0x7fd318078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.229+0000 7fd328ff9640 1 -- 172.21.15.45:0/3242590500 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7fd31c097890 con 0x7fd32c0cf8a0 2024-05-08T05:51:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.229+0000 7fd32b7fe640 1 --2- 172.21.15.45:0/3242590500 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd3180762d0 0x7fd318078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:18.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.230+0000 7fd32b7fe640 1 --2- 172.21.15.45:0/3242590500 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd3180762d0 0x7fd318078790 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd320034000 tx=0x7fd32002c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:18.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.234+0000 7fd328ff9640 1 -- 172.21.15.45:0/3242590500 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd32c0b7820 con 0x7fd32c0cf8a0 2024-05-08T05:51:18.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.467+0000 7fd3323b6640 1 -- 172.21.15.45:0/3242590500 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd32c0693f0 con 0x7fd32c0cf8a0 2024-05-08T05:51:18.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.467+0000 7fd328ff9640 1 -- 172.21.15.45:0/3242590500 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) ==== 53+0+73 (secure 0 0 0) 0x7fd31c061060 con 0x7fd32c0cf8a0 2024-05-08T05:51:18.467 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T05:51:18.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.469+0000 7fd3323b6640 1 -- 172.21.15.45:0/3242590500 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd3180762d0 msgr2=0x7fd318078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:18.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.469+0000 7fd3323b6640 1 --2- 172.21.15.45:0/3242590500 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd3180762d0 0x7fd318078790 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd320034000 tx=0x7fd32002c000 comp rx=0 tx=0).stop 2024-05-08T05:51:18.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.469+0000 7fd3323b6640 1 -- 172.21.15.45:0/3242590500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32c0cf8a0 msgr2=0x7fd32c155020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:18.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.469+0000 7fd3323b6640 1 --2- 172.21.15.45:0/3242590500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32c0cf8a0 0x7fd32c155020 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00c7f0 tx=0x7fd31c00ccb0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.469+0000 7fd3323b6640 1 -- 172.21.15.45:0/3242590500 shutdown_connections 2024-05-08T05:51:18.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.469+0000 7fd3323b6640 1 --2- 172.21.15.45:0/3242590500 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fd3180762d0 0x7fd318078790 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.469+0000 7fd3323b6640 1 --2- 172.21.15.45:0/3242590500 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fd32c0d39b0 0x7fd32c1e89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.469+0000 7fd3323b6640 1 --2- 172.21.15.45:0/3242590500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd32c0d0f10 0x7fd32c1e8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.469+0000 7fd3323b6640 1 --2- 172.21.15.45:0/3242590500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd32c0cf8a0 0x7fd32c155020 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:18.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.469+0000 7fd3323b6640 1 -- 172.21.15.45:0/3242590500 >> 172.21.15.45:0/3242590500 conn(0x7fd32c0d2e90 msgr2=0x7fd32c1603f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:18.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.469+0000 7fd3323b6640 1 -- 172.21.15.45:0/3242590500 shutdown_connections 2024-05-08T05:51:18.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18.469+0000 7fd3323b6640 1 -- 172.21.15.45:0/3242590500 wait complete. 2024-05-08T05:51:18.480 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T05:51:18.480 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:51:18.480 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-05-08T05:51:18.480 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T05:51:18.480 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:51:18.480 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T05:51:18.480 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T05:51:18.480 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:51:18.480 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:51:18.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:18.481 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-05-08T05:51:18.497 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:51:18.497 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:51:18.497 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:51:18.497 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:18.497 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-05-08T05:51:18.497 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:18.497 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:51:18.497 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:51:18.497 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-05-08T05:51:18.497 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:51:18.497 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:51:18.497 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-05-08T05:51:18.498 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:51:18.498 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-05-08T05:51:18.498 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-05-08T05:51:18.498 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:51:18.498 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42146sec preferred_lft 42146sec 2024-05-08T05:51:18.498 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-05-08T05:51:18.498 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:18.498 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:51:18.498 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-05-08T05:51:18.498 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-05-08T05:51:18.498 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:51:18.498 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T05:51:18.498 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T05:51:18.498 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:18.499 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c80a:aff:fe94:c77/64 scope link 2024-05-08T05:51:18.499 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:18.499 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:51:18.499 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 82:f3:eb:6a:57:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:51:18.499 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::80f3:ebff:fe6a:57ba/64 scope link 2024-05-08T05:51:18.499 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:18.499 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:51:18.499 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T05:51:18.499 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::14a2:64ff:feda:3a9c/64 scope link 2024-05-08T05:51:18.499 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:18.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:18.499 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:51:18.561 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:51:18.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:18.563 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-05-08T05:51:18.620 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-05-08T05:51:18.621 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:51:18.621 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:51:18.621 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:51:18.621 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:51:18.621 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:51:18.621 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:51:18.621 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:51:18.621 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:51:18.621 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:51:18.621 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:51:18.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:51:18.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:18.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:18.751 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:51:18.767 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T05:51:18.767 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T05:51:18.767 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:51:18.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:18.767 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-05-08T05:51:18.832 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:51:18.833 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T05:51:18.889 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:51:18.889 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:51:18.889 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:51:18.945 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T05:51:18.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:18.945 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:51:19.030 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:51:19.030 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:51:19.038 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:51:19.039 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:51:19.039 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-08T05:51:19.039 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@e9e1170a-fd27-4625-97a9-5d5f104d13a2.cephfs=/ 2024-05-08T05:51:19.039 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.63:3300/172.21.15.63:3301" will pass to kernel 2024-05-08T05:51:19.039 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T05:51:19.039 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:51:19.039 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:51:19.039 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:51:19.040 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:51:19.041 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:51:19.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:19.041 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:51:19.067 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:51:19.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:51:19.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:19.165 INFO:teuthology.orchestra.run.smithi045.stdout:client7309 2024-05-08T05:51:19.165 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:51:19.165 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:51:19.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:19.181 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-05-08T05:51:19.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:19.242 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-05-08T05:51:19.321 INFO:teuthology.orchestra.run.smithi045.stderr:8+0 records in 2024-05-08T05:51:19.321 INFO:teuthology.orchestra.run.smithi045.stderr:8+0 records out 2024-05-08T05:51:19.321 INFO:teuthology.orchestra.run.smithi045.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0233238 s, 360 MB/s 2024-05-08T05:51:19.323 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:19.323 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:19.323 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:19.323 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:19.323 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-08T05:51:19.323 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:19.431 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627779 2024-05-08T05:51:19.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:19.440 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:19.440 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:19.440 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:19.440 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:19.440 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-08T05:51:19.441 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:19.547 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627780 2024-05-08T05:51:19.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:19.556 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:51:19.556 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:51:19.613 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:51:19.613 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:51:19.670 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:51:19.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.815+0000 7eff5c608640 1 Processor -- start 2024-05-08T05:51:19.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.815+0000 7eff5c608640 1 -- start start 2024-05-08T05:51:19.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.816+0000 7eff5c608640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff540b2490 0x7eff54070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.816+0000 7eff5c608640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7eff54070d20 0x7eff540711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.816+0000 7eff5c608640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff5406e110 con 0x7eff541629b0 2024-05-08T05:51:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.816+0000 7eff5c608640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7eff5406dec0 con 0x7eff54070d20 2024-05-08T05:51:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.816+0000 7eff5c608640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7eff5406e3f0 con 0x7eff540b2490 2024-05-08T05:51:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.816+0000 7eff5aba5640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7eff54070d20 0x7eff540711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.816+0000 7eff59ba3640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff540b2490 0x7eff54070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.816+0000 7eff5a3a4640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7eff541629b0 0x7eff5406eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:44332/0 (socket says 172.21.15.45:44332) 2024-05-08T05:51:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.816+0000 7eff5a3a4640 1 -- 172.21.15.45:0/2658769587 learned_addr learned my addr 172.21.15.45:0/2658769587 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.816+0000 7eff593a2640 1 -- 172.21.15.45:0/2658769587 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 668895028 0 0) 0x7eff5406e110 con 0x7eff541629b0 2024-05-08T05:51:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.816+0000 7eff593a2640 1 -- 172.21.15.45:0/2658769587 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff40003520 con 0x7eff541629b0 2024-05-08T05:51:19.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.816+0000 7eff59ba3640 1 -- 172.21.15.45:0/2658769587 >> v2:172.21.15.63:3300/0 conn(0x7eff54070d20 msgr2=0x7eff540711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:19.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.816+0000 7eff59ba3640 1 --2- 172.21.15.45:0/2658769587 >> v2:172.21.15.63:3300/0 conn(0x7eff54070d20 0x7eff540711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:19.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.816+0000 7eff59ba3640 1 -- 172.21.15.45:0/2658769587 >> v1:172.21.15.63:6789/0 conn(0x7eff541629b0 legacy=0x7eff5406eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:19.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.816+0000 7eff59ba3640 1 -- 172.21.15.45:0/2658769587 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5406e6d0 con 0x7eff540b2490 2024-05-08T05:51:19.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.816+0000 7eff5aba5640 1 --2- 172.21.15.45:0/2658769587 >> v2:172.21.15.63:3300/0 conn(0x7eff54070d20 0x7eff540711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.817+0000 7eff59ba3640 1 --2- 172.21.15.45:0/2658769587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff540b2490 0x7eff54070330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7eff440097a0 tx=0x7eff4402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56e4bd732b123d9e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.817+0000 7eff593a2640 1 -- 172.21.15.45:0/2658769587 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44037070 con 0x7eff540b2490 2024-05-08T05:51:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.817+0000 7eff593a2640 1 -- 172.21.15.45:0/2658769587 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff44032440 con 0x7eff540b2490 2024-05-08T05:51:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.817+0000 7eff593a2640 1 -- 172.21.15.45:0/2658769587 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff440327f0 con 0x7eff540b2490 2024-05-08T05:51:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.818+0000 7eff5c608640 1 -- 172.21.15.45:0/2658769587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff540b2490 msgr2=0x7eff54070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.818+0000 7eff5c608640 1 --2- 172.21.15.45:0/2658769587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff540b2490 0x7eff54070330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7eff440097a0 tx=0x7eff4402a180 comp rx=0 tx=0).stop 2024-05-08T05:51:19.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.818+0000 7eff5c608640 1 -- 172.21.15.45:0/2658769587 shutdown_connections 2024-05-08T05:51:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.818+0000 7eff5c608640 1 --2- 172.21.15.45:0/2658769587 >> v2:172.21.15.63:3300/0 conn(0x7eff54070d20 0x7eff540711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.818+0000 7eff5c608640 1 --2- 172.21.15.45:0/2658769587 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff540b2490 0x7eff54070330 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.818+0000 7eff5c608640 1 -- 172.21.15.45:0/2658769587 >> 172.21.15.45:0/2658769587 conn(0x7eff540b49d0 msgr2=0x7eff540b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.818+0000 7eff5c608640 1 -- 172.21.15.45:0/2658769587 shutdown_connections 2024-05-08T05:51:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.818+0000 7eff5c608640 1 -- 172.21.15.45:0/2658769587 wait complete. 2024-05-08T05:51:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.819+0000 7eff5c608640 1 Processor -- start 2024-05-08T05:51:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.819+0000 7eff5c608640 1 -- start start 2024-05-08T05:51:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.820+0000 7eff5c608640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff54070d20 0x7eff541531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.820+0000 7eff5c608640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff540b2490 0x7eff540b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.820+0000 7eff5c608640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff541629b0 0x7eff540b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.820+0000 7eff5c608640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7eff540709f0 con 0x7eff541629b0 2024-05-08T05:51:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.820+0000 7eff5c608640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7eff54070870 con 0x7eff540b2490 2024-05-08T05:51:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.820+0000 7eff5c608640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7eff540afcb0 con 0x7eff54070d20 2024-05-08T05:51:19.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.820+0000 7eff5aba5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff541629b0 0x7eff540b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.820+0000 7eff5aba5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff541629b0 0x7eff540b1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45874/0 (socket says 172.21.15.45:45874) 2024-05-08T05:51:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.820+0000 7eff5aba5640 1 -- 172.21.15.45:0/2444380075 learned_addr learned my addr 172.21.15.45:0/2444380075 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.820+0000 7eff59ba3640 1 --2- 172.21.15.45:0/2444380075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff540b2490 0x7eff540b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.820+0000 7eff5a3a4640 1 --2- 172.21.15.45:0/2444380075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff54070d20 0x7eff541531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.820+0000 7eff5aba5640 1 -- 172.21.15.45:0/2444380075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff54070d20 msgr2=0x7eff541531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.820+0000 7eff5aba5640 1 --2- 172.21.15.45:0/2444380075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff54070d20 0x7eff541531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.820+0000 7eff5aba5640 1 -- 172.21.15.45:0/2444380075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff540b2490 msgr2=0x7eff540b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.820+0000 7eff5aba5640 1 --2- 172.21.15.45:0/2444380075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff540b2490 0x7eff540b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.820+0000 7eff5aba5640 1 -- 172.21.15.45:0/2444380075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff54153720 con 0x7eff541629b0 2024-05-08T05:51:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.820+0000 7eff5aba5640 1 --2- 172.21.15.45:0/2444380075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff541629b0 0x7eff540b1a70 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00efc0 tx=0x7eff4c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.821+0000 7eff4affd640 1 -- 172.21.15.45:0/2444380075 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c018030 con 0x7eff541629b0 2024-05-08T05:51:19.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.821+0000 7eff4affd640 1 -- 172.21.15.45:0/2444380075 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff4c00cde0 con 0x7eff541629b0 2024-05-08T05:51:19.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.821+0000 7eff5c608640 1 -- 172.21.15.45:0/2444380075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff540b5ac0 con 0x7eff541629b0 2024-05-08T05:51:19.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.821+0000 7eff4affd640 1 -- 172.21.15.45:0/2444380075 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c003bb0 con 0x7eff541629b0 2024-05-08T05:51:19.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.821+0000 7eff5c608640 1 -- 172.21.15.45:0/2444380075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff540b6080 con 0x7eff541629b0 2024-05-08T05:51:19.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.822+0000 7eff5c608640 1 -- 172.21.15.45:0/2444380075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff540693f0 con 0x7eff541629b0 2024-05-08T05:51:19.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.822+0000 7eff4affd640 1 -- 172.21.15.45:0/2444380075 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7eff4c005d10 con 0x7eff541629b0 2024-05-08T05:51:19.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.823+0000 7eff4affd640 1 --2- 172.21.15.45:0/2444380075 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7eff380762d0 0x7eff38078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:19.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.823+0000 7eff4affd640 1 -- 172.21.15.45:0/2444380075 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7eff4c098080 con 0x7eff541629b0 2024-05-08T05:51:19.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.823+0000 7eff5a3a4640 1 --2- 172.21.15.45:0/2444380075 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7eff380762d0 0x7eff38078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:19.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.824+0000 7eff5a3a4640 1 --2- 172.21.15.45:0/2444380075 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7eff380762d0 0x7eff38078790 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7eff50005880 tx=0x7eff50006aa0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:19.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:19.828+0000 7eff4affd640 1 -- 172.21.15.45:0/2444380075 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7eff4c022070 con 0x7eff541629b0 2024-05-08T05:51:20.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.030+0000 7eff5c608640 1 -- 172.21.15.45:0/2444380075 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7eff540b39f0 con 0x7eff541629b0 2024-05-08T05:51:20.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.030+0000 7eff4affd640 1 -- 172.21.15.45:0/2444380075 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v119) ==== 80+0+0 (secure 0 0 0) 0x7eff4c010040 con 0x7eff541629b0 2024-05-08T05:51:20.030 INFO:teuthology.orchestra.run.smithi045.stderr:listed 0 entries 2024-05-08T05:51:20.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.032+0000 7eff5c608640 1 -- 172.21.15.45:0/2444380075 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7eff380762d0 msgr2=0x7eff38078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:20.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.032+0000 7eff5c608640 1 --2- 172.21.15.45:0/2444380075 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7eff380762d0 0x7eff38078790 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7eff50005880 tx=0x7eff50006aa0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.032+0000 7eff5c608640 1 -- 172.21.15.45:0/2444380075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff541629b0 msgr2=0x7eff540b1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:20.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.032+0000 7eff5c608640 1 --2- 172.21.15.45:0/2444380075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff541629b0 0x7eff540b1a70 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00efc0 tx=0x7eff4c009f90 comp rx=0 tx=0).stop 2024-05-08T05:51:20.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.032+0000 7eff5c608640 1 -- 172.21.15.45:0/2444380075 shutdown_connections 2024-05-08T05:51:20.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.032+0000 7eff5c608640 1 --2- 172.21.15.45:0/2444380075 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7eff380762d0 0x7eff38078790 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.032+0000 7eff5c608640 1 --2- 172.21.15.45:0/2444380075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7eff541629b0 0x7eff540b1a70 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.032+0000 7eff5c608640 1 --2- 172.21.15.45:0/2444380075 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff540b2490 0x7eff540b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.032+0000 7eff5c608640 1 --2- 172.21.15.45:0/2444380075 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7eff54070d20 0x7eff541531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.032+0000 7eff5c608640 1 -- 172.21.15.45:0/2444380075 >> 172.21.15.45:0/2444380075 conn(0x7eff540b49d0 msgr2=0x7eff540b0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:20.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.032+0000 7eff5c608640 1 -- 172.21.15.45:0/2444380075 shutdown_connections 2024-05-08T05:51:20.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.032+0000 7eff5c608640 1 -- 172.21.15.45:0/2444380075 wait complete. 2024-05-08T05:51:20.043 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T05:51:20.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:20.043 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T05:51:20.120 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:51:20.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:20.120 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T05:51:20.147 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:51:20.148 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:51:20.204 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:51:20.204 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:51:20.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:20.204 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T05:51:20.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:51:20.261 INFO:teuthology.orchestra.run.smithi045.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-08T05:51:20.261 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T05:51:20.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.406+0000 7f49182a7640 1 Processor -- start 2024-05-08T05:51:20.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.406+0000 7f49182a7640 1 -- start start 2024-05-08T05:51:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.407+0000 7f49182a7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f49100d33f0 0x7f491015d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.407+0000 7f49182a7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49100718c0 con 0x7f491015ccb0 2024-05-08T05:51:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.407+0000 7f49182a7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4910071ba0 con 0x7f4910070b90 2024-05-08T05:51:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.407+0000 7f49182a7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f49100afba0 con 0x7f49100d33f0 2024-05-08T05:51:20.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.407+0000 7f4916043640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f491015ccb0 0x7f49100d2ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:40556/0 (socket says 172.21.15.45:40556) 2024-05-08T05:51:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.407+0000 7f4916043640 1 -- 172.21.15.45:0/773988653 learned_addr learned my addr 172.21.15.45:0/773988653 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.407+0000 7f4915842640 1 --2- 172.21.15.45:0/773988653 >> v2:172.21.15.63:3300/0 conn(0x7f49100d33f0 0x7f491015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.407+0000 7f4915041640 1 -- 172.21.15.45:0/773988653 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1446100783 0 0) 0x7f49100718c0 con 0x7f491015ccb0 2024-05-08T05:51:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.407+0000 7f4915041640 1 -- 172.21.15.45:0/773988653 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48fc003520 con 0x7f491015ccb0 2024-05-08T05:51:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.407+0000 7f4915842640 1 -- 172.21.15.45:0/773988653 >> v1:172.21.15.63:6789/0 conn(0x7f4910070b90 legacy=0x7f4910071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.407+0000 7f4915842640 1 -- 172.21.15.45:0/773988653 >> v1:172.21.15.45:6789/0 conn(0x7f491015ccb0 legacy=0x7f49100d2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.407+0000 7f4915842640 1 -- 172.21.15.45:0/773988653 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4910071e80 con 0x7f49100d33f0 2024-05-08T05:51:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.408+0000 7f4915842640 1 --2- 172.21.15.45:0/773988653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49100d33f0 0x7f491015d090 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f49000097a0 tx=0x7f490002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=865ca70540422bcf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.408+0000 7f4915041640 1 -- 172.21.15.45:0/773988653 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900037070 con 0x7f49100d33f0 2024-05-08T05:51:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.408+0000 7f4915041640 1 -- 172.21.15.45:0/773988653 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4900032440 con 0x7f49100d33f0 2024-05-08T05:51:20.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.408+0000 7f4915041640 1 -- 172.21.15.45:0/773988653 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900032810 con 0x7f49100d33f0 2024-05-08T05:51:20.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.409+0000 7f49182a7640 1 -- 172.21.15.45:0/773988653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49100d33f0 msgr2=0x7f491015d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:20.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.409+0000 7f49182a7640 1 --2- 172.21.15.45:0/773988653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49100d33f0 0x7f491015d090 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f49000097a0 tx=0x7f490002a1a0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.409+0000 7f49182a7640 1 -- 172.21.15.45:0/773988653 shutdown_connections 2024-05-08T05:51:20.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.409+0000 7f49182a7640 1 --2- 172.21.15.45:0/773988653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49100d33f0 0x7f491015d090 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.409+0000 7f49182a7640 1 -- 172.21.15.45:0/773988653 >> 172.21.15.45:0/773988653 conn(0x7f491009ba40 msgr2=0x7f491009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:20.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.409+0000 7f49182a7640 1 -- 172.21.15.45:0/773988653 shutdown_connections 2024-05-08T05:51:20.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.409+0000 7f49182a7640 1 -- 172.21.15.45:0/773988653 wait complete. 2024-05-08T05:51:20.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.410+0000 7f49182a7640 1 Processor -- start 2024-05-08T05:51:20.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.410+0000 7f49182a7640 1 -- start start 2024-05-08T05:51:20.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.411+0000 7f49182a7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4910070b90 0x7f49100cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:20.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.411+0000 7f49182a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49100d33f0 0x7f491014bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:20.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.411+0000 7f49182a7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f491015ccb0 0x7f491014c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:20.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.411+0000 7f49182a7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f49100b11c0 con 0x7f49100d33f0 2024-05-08T05:51:20.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.411+0000 7f49182a7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f49100b1040 con 0x7f491015ccb0 2024-05-08T05:51:20.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.411+0000 7f49182a7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f49100b1340 con 0x7f4910070b90 2024-05-08T05:51:20.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.411+0000 7f4915842640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49100d33f0 0x7f491014bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:20.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.411+0000 7f4915842640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49100d33f0 0x7f491014bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45882/0 (socket says 172.21.15.45:45882) 2024-05-08T05:51:20.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.411+0000 7f4915842640 1 -- 172.21.15.45:0/2177879404 learned_addr learned my addr 172.21.15.45:0/2177879404 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:20.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.411+0000 7f4916043640 1 --2- 172.21.15.45:0/2177879404 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4910070b90 0x7f49100cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:20.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.411+0000 7f4916844640 1 --2- 172.21.15.45:0/2177879404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f491015ccb0 0x7f491014c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:20.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.411+0000 7f4915842640 1 -- 172.21.15.45:0/2177879404 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4910070b90 msgr2=0x7f49100cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:20.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.411+0000 7f4915842640 1 --2- 172.21.15.45:0/2177879404 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4910070b90 0x7f49100cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.411+0000 7f4915842640 1 -- 172.21.15.45:0/2177879404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f491015ccb0 msgr2=0x7f491014c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:20.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.411+0000 7f4915842640 1 --2- 172.21.15.45:0/2177879404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f491015ccb0 0x7f491014c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.411+0000 7f4915842640 1 -- 172.21.15.45:0/2177879404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49100c9fa0 con 0x7f49100d33f0 2024-05-08T05:51:20.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.412+0000 7f4915842640 1 --2- 172.21.15.45:0/2177879404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49100d33f0 0x7f491014bdc0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f4900035040 tx=0x7f49000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:20.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.412+0000 7f4906ffd640 1 -- 172.21.15.45:0/2177879404 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900037040 con 0x7f49100d33f0 2024-05-08T05:51:20.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.412+0000 7f4906ffd640 1 -- 172.21.15.45:0/2177879404 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4900030500 con 0x7f49100d33f0 2024-05-08T05:51:20.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.412+0000 7f49182a7640 1 -- 172.21.15.45:0/2177879404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49100ca210 con 0x7f49100d33f0 2024-05-08T05:51:20.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.412+0000 7f4906ffd640 1 -- 172.21.15.45:0/2177879404 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900030910 con 0x7f49100d33f0 2024-05-08T05:51:20.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.412+0000 7f49182a7640 1 -- 172.21.15.45:0/2177879404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49100ca7d0 con 0x7f49100d33f0 2024-05-08T05:51:20.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.413+0000 7f49182a7640 1 -- 172.21.15.45:0/2177879404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4910069b00 con 0x7f49100d33f0 2024-05-08T05:51:20.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.413+0000 7f4906ffd640 1 -- 172.21.15.45:0/2177879404 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4900044020 con 0x7f49100d33f0 2024-05-08T05:51:20.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.414+0000 7f4906ffd640 1 --2- 172.21.15.45:0/2177879404 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f48f40762d0 0x7f48f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:20.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.414+0000 7f4906ffd640 1 -- 172.21.15.45:0/2177879404 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6101+0+0 (secure 0 0 0) 0x7f49000b78e0 con 0x7f49100d33f0 2024-05-08T05:51:20.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.414+0000 7f4916043640 1 --2- 172.21.15.45:0/2177879404 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f48f40762d0 0x7f48f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:20.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.415+0000 7f4916043640 1 --2- 172.21.15.45:0/2177879404 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f48f40762d0 0x7f48f4078790 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f490c005880 tx=0x7f490c007cd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:20.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.418+0000 7f4906ffd640 1 -- 172.21.15.45:0/2177879404 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4900080a70 con 0x7f49100d33f0 2024-05-08T05:51:20.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.650+0000 7f49182a7640 1 -- 172.21.15.45:0/2177879404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f49100d0d10 con 0x7f49100d33f0 2024-05-08T05:51:20.650 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-05-08T05:51:20.650+0000 7f124de12640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T05:51:20.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.652+0000 7f4906ffd640 1 -- 172.21.15.45:0/2177879404 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v101) ==== 161+0+0 (secure 0 0 0) 0x7f4900084d70 con 0x7f49100d33f0 2024-05-08T05:51:20.652 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T05:51:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.655+0000 7f49182a7640 1 -- 172.21.15.45:0/2177879404 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f48f40762d0 msgr2=0x7f48f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.655+0000 7f49182a7640 1 --2- 172.21.15.45:0/2177879404 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f48f40762d0 0x7f48f4078790 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f490c005880 tx=0x7f490c007cd0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.655+0000 7f49182a7640 1 -- 172.21.15.45:0/2177879404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49100d33f0 msgr2=0x7f491014bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.655+0000 7f49182a7640 1 --2- 172.21.15.45:0/2177879404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49100d33f0 0x7f491014bdc0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f4900035040 tx=0x7f49000046c0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.655+0000 7f49182a7640 1 -- 172.21.15.45:0/2177879404 shutdown_connections 2024-05-08T05:51:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.655+0000 7f49182a7640 1 --2- 172.21.15.45:0/2177879404 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f48f40762d0 0x7f48f4078790 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.655+0000 7f49182a7640 1 --2- 172.21.15.45:0/2177879404 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f491015ccb0 0x7f491014c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.655+0000 7f49182a7640 1 --2- 172.21.15.45:0/2177879404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49100d33f0 0x7f491014bdc0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.655+0000 7f49182a7640 1 --2- 172.21.15.45:0/2177879404 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4910070b90 0x7f49100cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.655+0000 7f49182a7640 1 -- 172.21.15.45:0/2177879404 >> 172.21.15.45:0/2177879404 conn(0x7f491009ba40 msgr2=0x7f49100b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.655+0000 7f49182a7640 1 -- 172.21.15.45:0/2177879404 shutdown_connections 2024-05-08T05:51:20.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.655+0000 7f49182a7640 1 -- 172.21.15.45:0/2177879404 wait complete. 2024-05-08T05:51:20.659 INFO:tasks.ceph.mds.c.smithi045.stderr: -8370> 2024-05-08T05:51:10.030+0000 7facdd29f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:51:20.668 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T05:51:20.670 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T05:51:20.693 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-05-08T05:51:20.715 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-05-08T05:51:20.715 INFO:teuthology.orchestra.run.smithi045.stdout: chdir = "" 2024-05-08T05:51:20.715 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T05:51:20.715 INFO:teuthology.orchestra.run.smithi045.stdout: auth supported = cephx 2024-05-08T05:51:20.715 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.715 INFO:teuthology.orchestra.run.smithi045.stdout: filestore xattr use omap = true 2024-05-08T05:51:20.715 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.715 INFO:teuthology.orchestra.run.smithi045.stdout: mon clock drift allowed = 1.000 2024-05-08T05:51:20.715 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.715 INFO:teuthology.orchestra.run.smithi045.stdout: osd crush chooseleaf type = 0 2024-05-08T05:51:20.715 INFO:teuthology.orchestra.run.smithi045.stdout: auth debug = true 2024-05-08T05:51:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on old message = true 2024-05-08T05:51:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on bug = true 2024-05-08T05:51:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T05:51:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: mon pg warn max object skew = 0 2024-05-08T05:51:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T05:51:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T05:51:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default size = 2 2024-05-08T05:51:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow primary affinity = true 2024-05-08T05:51:20.716 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow pg remap = true 2024-05-08T05:51:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on legacy crush tunables = false 2024-05-08T05:51:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on crush straw calc version zero = false 2024-05-08T05:51:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on no sortbitwise = false 2024-05-08T05:51:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on osd down out interval zero = false 2024-05-08T05:51:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on too few osds = false 2024-05-08T05:51:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T05:51:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T05:51:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: mon_allow_pool_size_one = true 2024-05-08T05:51:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T05:51:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: osd default data pool replay window = 5 2024-05-08T05:51:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: mon allow pool delete = true 2024-05-08T05:51:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.717 INFO:teuthology.orchestra.run.smithi045.stdout: mon cluster log file level = debug 2024-05-08T05:51:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: debug asserts on shutdown = true 2024-05-08T05:51:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: mon health detail to clog = false 2024-05-08T05:51:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: mon host = "172.21.15.45,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-08T05:51:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on skipped message = False 2024-05-08T05:51:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = e9e1170a-fd27-4625-97a9-5d5f104d13a2 2024-05-08T05:51:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.718 INFO:teuthology.orchestra.run.smithi045.stdout:[osd] 2024-05-08T05:51:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: osd journal size = 100 2024-05-08T05:51:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub load threshold = 5.0 2024-05-08T05:51:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub max interval = 600 2024-05-08T05:51:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: osd mclock profile = high_recovery_ops 2024-05-08T05:51:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.718 INFO:teuthology.orchestra.run.smithi045.stdout: osd recover clone overlap = true 2024-05-08T05:51:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: osd recovery max chunk = 1048576 2024-05-08T05:51:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug shutdown = true 2024-05-08T05:51:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug op order = true 2024-05-08T05:51:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify stray on activate = true 2024-05-08T05:51:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug trim objects = true 2024-05-08T05:51:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: osd open classes on start = true 2024-05-08T05:51:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug pg log writeout = true 2024-05-08T05:51:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: osd deep scrub update digest min age = 30 2024-05-08T05:51:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: osd map max advance = 10 2024-05-08T05:51:20.719 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: journal zero on create = true 2024-05-08T05:51:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: filestore ondisk finisher threads = 3 2024-05-08T05:51:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: filestore apply finisher threads = 3 2024-05-08T05:51:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: bdev debug aio = true 2024-05-08T05:51:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug misdirected ops = true 2024-05-08T05:51:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore block size = 96636764160 2024-05-08T05:51:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore fsck on mount = True 2024-05-08T05:51:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluefs = 20 2024-05-08T05:51:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluestore = 20 2024-05-08T05:51:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:51:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: debug osd = 20 2024-05-08T05:51:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: debug rocksdb = 10 2024-05-08T05:51:20.720 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T05:51:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd full ratio = 0.9 2024-05-08T05:51:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd nearfull ratio = 0.8 2024-05-08T05:51:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: osd failsafe full ratio = 0.95 2024-05-08T05:51:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: osd objectstore = bluestore 2024-05-08T05:51:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-05-08T05:51:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.721 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr] 2024-05-08T05:51:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:51:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: debug mgr = 20 2024-05-08T05:51:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-05-08T05:51:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-05-08T05:51:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-05-08T05:51:20.721 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-05-08T05:51:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: mgr/telemetry/nag = false 2024-05-08T05:51:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: debug client = 20 2024-05-08T05:51:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.722 INFO:teuthology.orchestra.run.smithi045.stdout:[mon] 2024-05-08T05:51:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:51:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-05-08T05:51:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: debug paxos = 20 2024-05-08T05:51:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-05-08T05:51:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: mon data avail warn = 5 2024-05-08T05:51:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: mon mgr mkfs grace = 240 2024-05-08T05:51:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-05-08T05:51:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd reporter subtree level = osd 2024-05-08T05:51:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd prime pg temp = true 2024-05-08T05:51:20.722 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-05-08T05:51:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T05:51:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T05:51:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T05:51:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T05:51:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T05:51:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T05:51:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: # 1m isn't quite enough 2024-05-08T05:51:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: mon_down_mkfs_grace = 2m 2024-05-08T05:51:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.723 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_filestore_osds = false 2024-05-08T05:51:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: mon down mkfs grace = 300 2024-05-08T05:51:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: mon op complaint time = 120 2024-05-08T05:51:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.724 INFO:teuthology.orchestra.run.smithi045.stdout:[client] 2024-05-08T05:51:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: rgw cache enabled = true 2024-05-08T05:51:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable ops log = true 2024-05-08T05:51:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable usage log = true 2024-05-08T05:51:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T05:51:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T05:51:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: client mount timeout = 600 2024-05-08T05:51:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: debug client = 20 2024-05-08T05:51:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:51:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = "" 2024-05-08T05:51:20.724 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-05-08T05:51:20.725 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-05-08T05:51:20.725 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.a] 2024-05-08T05:51:20.725 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.b] 2024-05-08T05:51:20.725 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.c] 2024-05-08T05:51:20.725 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a] 2024-05-08T05:51:20.725 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c] 2024-05-08T05:51:20.725 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b] 2024-05-08T05:51:20.725 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d] 2024-05-08T05:51:20.725 INFO:teuthology.orchestra.run.smithi045.stdout:[mds] 2024-05-08T05:51:20.725 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds = 20 2024-05-08T05:51:20.725 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds balancer = 20 2024-05-08T05:51:20.725 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:51:20.725 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug frag = True 2024-05-08T05:51:20.726 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug scatterstat = True 2024-05-08T05:51:20.726 INFO:teuthology.orchestra.run.smithi045.stdout: mds op complaint time = 180 2024-05-08T05:51:20.726 INFO:teuthology.orchestra.run.smithi045.stdout: mds verify scatter = False 2024-05-08T05:51:20.726 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-05-08T05:51:20.726 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-05-08T05:51:20.726 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-05-08T05:51:20.726 INFO:teuthology.orchestra.run.smithi063.stdout:[global] 2024-05-08T05:51:20.727 INFO:teuthology.orchestra.run.smithi063.stdout: chdir = "" 2024-05-08T05:51:20.727 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T05:51:20.727 INFO:teuthology.orchestra.run.smithi063.stdout: auth supported = cephx 2024-05-08T05:51:20.727 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.727 INFO:teuthology.orchestra.run.smithi063.stdout: filestore xattr use omap = true 2024-05-08T05:51:20.727 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.727 INFO:teuthology.orchestra.run.smithi063.stdout: mon clock drift allowed = 1.000 2024-05-08T05:51:20.727 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.727 INFO:teuthology.orchestra.run.smithi063.stdout: osd crush chooseleaf type = 0 2024-05-08T05:51:20.727 INFO:teuthology.orchestra.run.smithi063.stdout: auth debug = true 2024-05-08T05:51:20.727 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.727 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on old message = true 2024-05-08T05:51:20.727 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on bug = true 2024-05-08T05:51:20.728 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.728 INFO:teuthology.orchestra.run.smithi063.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T05:51:20.728 INFO:teuthology.orchestra.run.smithi063.stdout: mon pg warn max object skew = 0 2024-05-08T05:51:20.728 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.728 INFO:teuthology.orchestra.run.smithi063.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T05:51:20.728 INFO:teuthology.orchestra.run.smithi063.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T05:51:20.728 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.728 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default size = 2 2024-05-08T05:51:20.728 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.728 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow primary affinity = true 2024-05-08T05:51:20.728 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow pg remap = true 2024-05-08T05:51:20.728 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on legacy crush tunables = false 2024-05-08T05:51:20.728 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on crush straw calc version zero = false 2024-05-08T05:51:20.729 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on no sortbitwise = false 2024-05-08T05:51:20.729 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on osd down out interval zero = false 2024-05-08T05:51:20.729 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on too few osds = false 2024-05-08T05:51:20.729 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T05:51:20.729 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T05:51:20.729 INFO:teuthology.orchestra.run.smithi063.stdout: mon_allow_pool_size_one = true 2024-05-08T05:51:20.729 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.729 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T05:51:20.729 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.729 INFO:teuthology.orchestra.run.smithi063.stdout: osd default data pool replay window = 5 2024-05-08T05:51:20.729 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.729 INFO:teuthology.orchestra.run.smithi063.stdout: mon allow pool delete = true 2024-05-08T05:51:20.729 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.730 INFO:teuthology.orchestra.run.smithi063.stdout: mon cluster log file level = debug 2024-05-08T05:51:20.730 INFO:teuthology.orchestra.run.smithi063.stdout: debug asserts on shutdown = true 2024-05-08T05:51:20.730 INFO:teuthology.orchestra.run.smithi063.stdout: mon health detail to clog = false 2024-05-08T05:51:20.730 INFO:teuthology.orchestra.run.smithi063.stdout: mon host = "172.21.15.45,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-08T05:51:20.730 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on skipped message = False 2024-05-08T05:51:20.730 INFO:teuthology.orchestra.run.smithi063.stdout: fsid = e9e1170a-fd27-4625-97a9-5d5f104d13a2 2024-05-08T05:51:20.730 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.730 INFO:teuthology.orchestra.run.smithi063.stdout:[osd] 2024-05-08T05:51:20.730 INFO:teuthology.orchestra.run.smithi063.stdout: osd journal size = 100 2024-05-08T05:51:20.730 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.730 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub load threshold = 5.0 2024-05-08T05:51:20.730 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub max interval = 600 2024-05-08T05:51:20.730 INFO:teuthology.orchestra.run.smithi063.stdout: osd mclock profile = high_recovery_ops 2024-05-08T05:51:20.731 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.731 INFO:teuthology.orchestra.run.smithi063.stdout: osd recover clone overlap = true 2024-05-08T05:51:20.731 INFO:teuthology.orchestra.run.smithi063.stdout: osd recovery max chunk = 1048576 2024-05-08T05:51:20.731 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.731 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug shutdown = true 2024-05-08T05:51:20.731 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug op order = true 2024-05-08T05:51:20.731 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug verify stray on activate = true 2024-05-08T05:51:20.731 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug trim objects = true 2024-05-08T05:51:20.731 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.731 INFO:teuthology.orchestra.run.smithi063.stdout: osd open classes on start = true 2024-05-08T05:51:20.731 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug pg log writeout = true 2024-05-08T05:51:20.731 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.731 INFO:teuthology.orchestra.run.smithi063.stdout: osd deep scrub update digest min age = 30 2024-05-08T05:51:20.731 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.731 INFO:teuthology.orchestra.run.smithi063.stdout: osd map max advance = 10 2024-05-08T05:51:20.732 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.732 INFO:teuthology.orchestra.run.smithi063.stdout: journal zero on create = true 2024-05-08T05:51:20.732 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.732 INFO:teuthology.orchestra.run.smithi063.stdout: filestore ondisk finisher threads = 3 2024-05-08T05:51:20.732 INFO:teuthology.orchestra.run.smithi063.stdout: filestore apply finisher threads = 3 2024-05-08T05:51:20.732 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.732 INFO:teuthology.orchestra.run.smithi063.stdout: bdev debug aio = true 2024-05-08T05:51:20.732 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug misdirected ops = true 2024-05-08T05:51:20.732 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore block size = 96636764160 2024-05-08T05:51:20.732 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore fsck on mount = True 2024-05-08T05:51:20.732 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluefs = 20 2024-05-08T05:51:20.732 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluestore = 20 2024-05-08T05:51:20.732 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:51:20.732 INFO:teuthology.orchestra.run.smithi063.stdout: debug osd = 20 2024-05-08T05:51:20.732 INFO:teuthology.orchestra.run.smithi063.stdout: debug rocksdb = 10 2024-05-08T05:51:20.732 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T05:51:20.733 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd full ratio = 0.9 2024-05-08T05:51:20.733 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd nearfull ratio = 0.8 2024-05-08T05:51:20.733 INFO:teuthology.orchestra.run.smithi063.stdout: osd failsafe full ratio = 0.95 2024-05-08T05:51:20.733 INFO:teuthology.orchestra.run.smithi063.stdout: osd objectstore = bluestore 2024-05-08T05:51:20.733 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-08T05:51:20.733 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.733 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr] 2024-05-08T05:51:20.733 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:51:20.733 INFO:teuthology.orchestra.run.smithi063.stdout: debug mgr = 20 2024-05-08T05:51:20.733 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-08T05:51:20.733 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-08T05:51:20.733 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-08T05:51:20.733 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-08T05:51:20.733 INFO:teuthology.orchestra.run.smithi063.stdout: mgr/telemetry/nag = false 2024-05-08T05:51:20.734 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-08T05:51:20.734 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.734 INFO:teuthology.orchestra.run.smithi063.stdout:[mon] 2024-05-08T05:51:20.734 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:51:20.734 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-08T05:51:20.734 INFO:teuthology.orchestra.run.smithi063.stdout: debug paxos = 20 2024-05-08T05:51:20.734 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-08T05:51:20.734 INFO:teuthology.orchestra.run.smithi063.stdout: mon data avail warn = 5 2024-05-08T05:51:20.734 INFO:teuthology.orchestra.run.smithi063.stdout: mon mgr mkfs grace = 240 2024-05-08T05:51:20.734 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-08T05:51:20.734 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd reporter subtree level = osd 2024-05-08T05:51:20.734 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd prime pg temp = true 2024-05-08T05:51:20.734 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-08T05:51:20.734 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.735 INFO:teuthology.orchestra.run.smithi063.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T05:51:20.735 INFO:teuthology.orchestra.run.smithi063.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T05:51:20.735 INFO:teuthology.orchestra.run.smithi063.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T05:51:20.735 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.735 INFO:teuthology.orchestra.run.smithi063.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T05:51:20.735 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T05:51:20.735 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T05:51:20.735 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.735 INFO:teuthology.orchestra.run.smithi063.stdout: # 1m isn't quite enough 2024-05-08T05:51:20.735 INFO:teuthology.orchestra.run.smithi063.stdout: mon_down_mkfs_grace = 2m 2024-05-08T05:51:20.735 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.735 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_filestore_osds = false 2024-05-08T05:51:20.735 INFO:teuthology.orchestra.run.smithi063.stdout: mon down mkfs grace = 300 2024-05-08T05:51:20.735 INFO:teuthology.orchestra.run.smithi063.stdout: mon op complaint time = 120 2024-05-08T05:51:20.735 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.736 INFO:teuthology.orchestra.run.smithi063.stdout:[client] 2024-05-08T05:51:20.736 INFO:teuthology.orchestra.run.smithi063.stdout: rgw cache enabled = true 2024-05-08T05:51:20.736 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable ops log = true 2024-05-08T05:51:20.736 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable usage log = true 2024-05-08T05:51:20.736 INFO:teuthology.orchestra.run.smithi063.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T05:51:20.736 INFO:teuthology.orchestra.run.smithi063.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T05:51:20.736 INFO:teuthology.orchestra.run.smithi063.stdout: client mount timeout = 600 2024-05-08T05:51:20.736 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-08T05:51:20.736 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:51:20.736 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = "" 2024-05-08T05:51:20.736 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-08T05:51:20.736 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-08T05:51:20.736 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.a] 2024-05-08T05:51:20.736 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.b] 2024-05-08T05:51:20.736 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.c] 2024-05-08T05:51:20.736 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.a] 2024-05-08T05:51:20.737 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.c] 2024-05-08T05:51:20.737 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.b] 2024-05-08T05:51:20.737 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.d] 2024-05-08T05:51:20.737 INFO:teuthology.orchestra.run.smithi063.stdout:[mds] 2024-05-08T05:51:20.737 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds = 20 2024-05-08T05:51:20.737 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds balancer = 20 2024-05-08T05:51:20.737 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:51:20.737 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug frag = True 2024-05-08T05:51:20.737 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug scatterstat = True 2024-05-08T05:51:20.737 INFO:teuthology.orchestra.run.smithi063.stdout: mds op complaint time = 180 2024-05-08T05:51:20.737 INFO:teuthology.orchestra.run.smithi063.stdout: mds verify scatter = False 2024-05-08T05:51:20.737 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-08T05:51:20.737 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-08T05:51:20.737 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-08T05:51:20.740 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T05:51:20.781 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T05:51:20.824 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-05-08T05:51:20.824 INFO:teuthology.orchestra.run.smithi045.stdout: chdir = "" 2024-05-08T05:51:20.824 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T05:51:20.824 INFO:teuthology.orchestra.run.smithi045.stdout: auth supported = cephx 2024-05-08T05:51:20.824 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.824 INFO:teuthology.orchestra.run.smithi045.stdout: filestore xattr use omap = true 2024-05-08T05:51:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: mon clock drift allowed = 1.000 2024-05-08T05:51:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: osd crush chooseleaf type = 0 2024-05-08T05:51:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: auth debug = true 2024-05-08T05:51:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on old message = true 2024-05-08T05:51:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on bug = true 2024-05-08T05:51:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T05:51:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: mon pg warn max object skew = 0 2024-05-08T05:51:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.825 INFO:teuthology.orchestra.run.smithi045.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T05:51:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T05:51:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default size = 2 2024-05-08T05:51:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow primary affinity = true 2024-05-08T05:51:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow pg remap = true 2024-05-08T05:51:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on legacy crush tunables = false 2024-05-08T05:51:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on crush straw calc version zero = false 2024-05-08T05:51:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on no sortbitwise = false 2024-05-08T05:51:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on osd down out interval zero = false 2024-05-08T05:51:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on too few osds = false 2024-05-08T05:51:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T05:51:20.826 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T05:51:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: mon_allow_pool_size_one = true 2024-05-08T05:51:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T05:51:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: osd default data pool replay window = 5 2024-05-08T05:51:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: mon allow pool delete = true 2024-05-08T05:51:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: mon cluster log file level = debug 2024-05-08T05:51:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: debug asserts on shutdown = true 2024-05-08T05:51:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: mon health detail to clog = false 2024-05-08T05:51:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: mon host = "172.21.15.45,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-08T05:51:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on skipped message = False 2024-05-08T05:51:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = e9e1170a-fd27-4625-97a9-5d5f104d13a2 2024-05-08T05:51:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.827 INFO:teuthology.orchestra.run.smithi045.stdout:[osd] 2024-05-08T05:51:20.827 INFO:teuthology.orchestra.run.smithi045.stdout: osd journal size = 100 2024-05-08T05:51:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub load threshold = 5.0 2024-05-08T05:51:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub max interval = 600 2024-05-08T05:51:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: osd mclock profile = high_recovery_ops 2024-05-08T05:51:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: osd recover clone overlap = true 2024-05-08T05:51:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: osd recovery max chunk = 1048576 2024-05-08T05:51:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug shutdown = true 2024-05-08T05:51:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug op order = true 2024-05-08T05:51:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify stray on activate = true 2024-05-08T05:51:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug trim objects = true 2024-05-08T05:51:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: osd open classes on start = true 2024-05-08T05:51:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug pg log writeout = true 2024-05-08T05:51:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.828 INFO:teuthology.orchestra.run.smithi045.stdout: osd deep scrub update digest min age = 30 2024-05-08T05:51:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: osd map max advance = 10 2024-05-08T05:51:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: journal zero on create = true 2024-05-08T05:51:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: filestore ondisk finisher threads = 3 2024-05-08T05:51:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: filestore apply finisher threads = 3 2024-05-08T05:51:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: bdev debug aio = true 2024-05-08T05:51:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug misdirected ops = true 2024-05-08T05:51:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore block size = 96636764160 2024-05-08T05:51:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore fsck on mount = True 2024-05-08T05:51:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluefs = 20 2024-05-08T05:51:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluestore = 20 2024-05-08T05:51:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:51:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: debug osd = 20 2024-05-08T05:51:20.829 INFO:teuthology.orchestra.run.smithi045.stdout: debug rocksdb = 10 2024-05-08T05:51:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T05:51:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd full ratio = 0.9 2024-05-08T05:51:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd nearfull ratio = 0.8 2024-05-08T05:51:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: osd failsafe full ratio = 0.95 2024-05-08T05:51:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: osd objectstore = bluestore 2024-05-08T05:51:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-05-08T05:51:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.830 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr] 2024-05-08T05:51:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:51:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: debug mgr = 20 2024-05-08T05:51:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-05-08T05:51:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-05-08T05:51:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-05-08T05:51:20.830 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-05-08T05:51:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: mgr/telemetry/nag = false 2024-05-08T05:51:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: debug client = 20 2024-05-08T05:51:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.831 INFO:teuthology.orchestra.run.smithi045.stdout:[mon] 2024-05-08T05:51:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:51:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-05-08T05:51:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: debug paxos = 20 2024-05-08T05:51:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-05-08T05:51:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: mon data avail warn = 5 2024-05-08T05:51:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: mon mgr mkfs grace = 240 2024-05-08T05:51:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-05-08T05:51:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd reporter subtree level = osd 2024-05-08T05:51:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd prime pg temp = true 2024-05-08T05:51:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-05-08T05:51:20.831 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T05:51:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T05:51:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T05:51:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T05:51:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T05:51:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T05:51:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: # 1m isn't quite enough 2024-05-08T05:51:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: mon_down_mkfs_grace = 2m 2024-05-08T05:51:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_filestore_osds = false 2024-05-08T05:51:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: mon down mkfs grace = 300 2024-05-08T05:51:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: mon op complaint time = 120 2024-05-08T05:51:20.832 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:20.833 INFO:teuthology.orchestra.run.smithi045.stdout:[client] 2024-05-08T05:51:20.833 INFO:teuthology.orchestra.run.smithi045.stdout: rgw cache enabled = true 2024-05-08T05:51:20.833 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable ops log = true 2024-05-08T05:51:20.833 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable usage log = true 2024-05-08T05:51:20.833 INFO:teuthology.orchestra.run.smithi045.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T05:51:20.833 INFO:teuthology.orchestra.run.smithi045.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T05:51:20.833 INFO:teuthology.orchestra.run.smithi045.stdout: client mount timeout = 600 2024-05-08T05:51:20.833 INFO:teuthology.orchestra.run.smithi045.stdout: debug client = 20 2024-05-08T05:51:20.833 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:51:20.833 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = "" 2024-05-08T05:51:20.833 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-05-08T05:51:20.833 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-05-08T05:51:20.833 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.a] 2024-05-08T05:51:20.833 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.b] 2024-05-08T05:51:20.833 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.c] 2024-05-08T05:51:20.833 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a] 2024-05-08T05:51:20.834 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c] 2024-05-08T05:51:20.834 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b] 2024-05-08T05:51:20.834 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d] 2024-05-08T05:51:20.834 INFO:teuthology.orchestra.run.smithi045.stdout:[mds] 2024-05-08T05:51:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds = 20 2024-05-08T05:51:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds balancer = 20 2024-05-08T05:51:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:51:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug frag = True 2024-05-08T05:51:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug scatterstat = False 2024-05-08T05:51:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: mds op complaint time = 180 2024-05-08T05:51:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: mds verify scatter = False 2024-05-08T05:51:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-05-08T05:51:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-05-08T05:51:20.834 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-05-08T05:51:20.835 INFO:teuthology.orchestra.run.smithi063.stdout:[global] 2024-05-08T05:51:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: chdir = "" 2024-05-08T05:51:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T05:51:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: auth supported = cephx 2024-05-08T05:51:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: filestore xattr use omap = true 2024-05-08T05:51:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.835 INFO:teuthology.orchestra.run.smithi063.stdout: mon clock drift allowed = 1.000 2024-05-08T05:51:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: osd crush chooseleaf type = 0 2024-05-08T05:51:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: auth debug = true 2024-05-08T05:51:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on old message = true 2024-05-08T05:51:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on bug = true 2024-05-08T05:51:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T05:51:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: mon pg warn max object skew = 0 2024-05-08T05:51:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T05:51:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T05:51:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default size = 2 2024-05-08T05:51:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.836 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow primary affinity = true 2024-05-08T05:51:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow pg remap = true 2024-05-08T05:51:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on legacy crush tunables = false 2024-05-08T05:51:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on crush straw calc version zero = false 2024-05-08T05:51:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on no sortbitwise = false 2024-05-08T05:51:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on osd down out interval zero = false 2024-05-08T05:51:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on too few osds = false 2024-05-08T05:51:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T05:51:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T05:51:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: mon_allow_pool_size_one = true 2024-05-08T05:51:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T05:51:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: osd default data pool replay window = 5 2024-05-08T05:51:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: mon allow pool delete = true 2024-05-08T05:51:20.837 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: mon cluster log file level = debug 2024-05-08T05:51:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: debug asserts on shutdown = true 2024-05-08T05:51:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: mon health detail to clog = false 2024-05-08T05:51:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: mon host = "172.21.15.45,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-08T05:51:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on skipped message = False 2024-05-08T05:51:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: fsid = e9e1170a-fd27-4625-97a9-5d5f104d13a2 2024-05-08T05:51:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.838 INFO:teuthology.orchestra.run.smithi063.stdout:[osd] 2024-05-08T05:51:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: osd journal size = 100 2024-05-08T05:51:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub load threshold = 5.0 2024-05-08T05:51:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub max interval = 600 2024-05-08T05:51:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: osd mclock profile = high_recovery_ops 2024-05-08T05:51:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.838 INFO:teuthology.orchestra.run.smithi063.stdout: osd recover clone overlap = true 2024-05-08T05:51:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: osd recovery max chunk = 1048576 2024-05-08T05:51:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug shutdown = true 2024-05-08T05:51:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug op order = true 2024-05-08T05:51:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug verify stray on activate = true 2024-05-08T05:51:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug trim objects = true 2024-05-08T05:51:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: osd open classes on start = true 2024-05-08T05:51:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug pg log writeout = true 2024-05-08T05:51:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: osd deep scrub update digest min age = 30 2024-05-08T05:51:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: osd map max advance = 10 2024-05-08T05:51:20.839 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: journal zero on create = true 2024-05-08T05:51:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: filestore ondisk finisher threads = 3 2024-05-08T05:51:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: filestore apply finisher threads = 3 2024-05-08T05:51:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: bdev debug aio = true 2024-05-08T05:51:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug misdirected ops = true 2024-05-08T05:51:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore block size = 96636764160 2024-05-08T05:51:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore fsck on mount = True 2024-05-08T05:51:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluefs = 20 2024-05-08T05:51:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluestore = 20 2024-05-08T05:51:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:51:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: debug osd = 20 2024-05-08T05:51:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: debug rocksdb = 10 2024-05-08T05:51:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T05:51:20.840 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd full ratio = 0.9 2024-05-08T05:51:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd nearfull ratio = 0.8 2024-05-08T05:51:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: osd failsafe full ratio = 0.95 2024-05-08T05:51:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: osd objectstore = bluestore 2024-05-08T05:51:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-08T05:51:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.841 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr] 2024-05-08T05:51:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:51:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: debug mgr = 20 2024-05-08T05:51:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-08T05:51:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-08T05:51:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-08T05:51:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-08T05:51:20.841 INFO:teuthology.orchestra.run.smithi063.stdout: mgr/telemetry/nag = false 2024-05-08T05:51:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-08T05:51:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.842 INFO:teuthology.orchestra.run.smithi063.stdout:[mon] 2024-05-08T05:51:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:51:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-08T05:51:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: debug paxos = 20 2024-05-08T05:51:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-08T05:51:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: mon data avail warn = 5 2024-05-08T05:51:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: mon mgr mkfs grace = 240 2024-05-08T05:51:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-08T05:51:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd reporter subtree level = osd 2024-05-08T05:51:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd prime pg temp = true 2024-05-08T05:51:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-08T05:51:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.842 INFO:teuthology.orchestra.run.smithi063.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T05:51:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T05:51:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T05:51:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T05:51:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T05:51:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T05:51:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: # 1m isn't quite enough 2024-05-08T05:51:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: mon_down_mkfs_grace = 2m 2024-05-08T05:51:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_filestore_osds = false 2024-05-08T05:51:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: mon down mkfs grace = 300 2024-05-08T05:51:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: mon op complaint time = 120 2024-05-08T05:51:20.843 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:20.843 INFO:teuthology.orchestra.run.smithi063.stdout:[client] 2024-05-08T05:51:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: rgw cache enabled = true 2024-05-08T05:51:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable ops log = true 2024-05-08T05:51:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable usage log = true 2024-05-08T05:51:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T05:51:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T05:51:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: client mount timeout = 600 2024-05-08T05:51:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-08T05:51:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:51:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = "" 2024-05-08T05:51:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-08T05:51:20.844 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-08T05:51:20.844 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.a] 2024-05-08T05:51:20.844 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.b] 2024-05-08T05:51:20.844 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.c] 2024-05-08T05:51:20.844 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.a] 2024-05-08T05:51:20.845 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.c] 2024-05-08T05:51:20.845 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.b] 2024-05-08T05:51:20.845 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.d] 2024-05-08T05:51:20.845 INFO:teuthology.orchestra.run.smithi063.stdout:[mds] 2024-05-08T05:51:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds = 20 2024-05-08T05:51:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds balancer = 20 2024-05-08T05:51:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:51:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug frag = True 2024-05-08T05:51:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug scatterstat = False 2024-05-08T05:51:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: mds op complaint time = 180 2024-05-08T05:51:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: mds verify scatter = False 2024-05-08T05:51:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-08T05:51:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-08T05:51:20.845 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-08T05:51:20.847 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-05-08T05:51:20.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.931+0000 7f8b0551ef40 1 Processor -- start 2024-05-08T05:51:20.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.932+0000 7f8b0551ef40 1 -- start start 2024-05-08T05:51:20.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.932+0000 7f8b0551ef40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5580f54b8da0 0x5580f54b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:20.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.932+0000 7f8b0551ef40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580f54cd8f0 0x5580f5562f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:20.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.932+0000 7f8b0551ef40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580f54cdcd0 0x5580f54d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:20.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.932+0000 7f8b0551ef40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5580f53a6f50 con 0x5580f54cdcd0 2024-05-08T05:51:20.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.932+0000 7f8b0551ef40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5580f54b9730 con 0x5580f54cd8f0 2024-05-08T05:51:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.932+0000 7f8b0551ef40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5580f54b5720 con 0x5580f54b8da0 2024-05-08T05:51:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.932+0000 7f8b02d4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580f54cdcd0 0x5580f54d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.932+0000 7f8b02d4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580f54cdcd0 0x5580f54d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51158/0 (socket says 172.21.15.45:51158) 2024-05-08T05:51:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.932+0000 7f8b02d4b640 1 -- 172.21.15.45:0/3786651525 learned_addr learned my addr 172.21.15.45:0/3786651525 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.932+0000 7f8b0254a640 1 --2- 172.21.15.45:0/3786651525 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5580f54b8da0 0x5580f54b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.932+0000 7f8b01d49640 1 --2- 172.21.15.45:0/3786651525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580f54cd8f0 0x5580f5562f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.933+0000 7f8b02d4b640 1 -- 172.21.15.45:0/3786651525 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5580f54b8da0 msgr2=0x5580f54b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.933+0000 7f8b02d4b640 1 --2- 172.21.15.45:0/3786651525 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5580f54b8da0 0x5580f54b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.933+0000 7f8b02d4b640 1 -- 172.21.15.45:0/3786651525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580f54cd8f0 msgr2=0x5580f5562f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.933+0000 7f8b02d4b640 1 --2- 172.21.15.45:0/3786651525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580f54cd8f0 0x5580f5562f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.933+0000 7f8b02d4b640 1 -- 172.21.15.45:0/3786651525 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580f53fef50 con 0x5580f54cdcd0 2024-05-08T05:51:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.933+0000 7f8b02d4b640 1 --2- 172.21.15.45:0/3786651525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580f54cdcd0 0x5580f54d24d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0098a0 tx=0x7f8afc005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436776ffedce7017 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.933+0000 7f8b00d47640 1 -- 172.21.15.45:0/3786651525 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc008930 con 0x5580f54cdcd0 2024-05-08T05:51:20.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.933+0000 7f8b00d47640 1 -- 172.21.15.45:0/3786651525 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8afc018070 con 0x5580f54cdcd0 2024-05-08T05:51:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.933+0000 7f8b00d47640 1 -- 172.21.15.45:0/3786651525 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc008d10 con 0x5580f54cdcd0 2024-05-08T05:51:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.933+0000 7f8b0551ef40 1 -- 172.21.15.45:0/3786651525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580f54cdcd0 msgr2=0x5580f54d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.933+0000 7f8b0551ef40 1 --2- 172.21.15.45:0/3786651525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580f54cdcd0 0x5580f54d24d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0098a0 tx=0x7f8afc005a00 comp rx=0 tx=0).stop 2024-05-08T05:51:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.934+0000 7f8b0551ef40 1 -- 172.21.15.45:0/3786651525 shutdown_connections 2024-05-08T05:51:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.934+0000 7f8b0551ef40 1 --2- 172.21.15.45:0/3786651525 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580f54cdcd0 0x5580f54d24d0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.934+0000 7f8b0551ef40 1 --2- 172.21.15.45:0/3786651525 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580f54cd8f0 0x5580f5562f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.934+0000 7f8b0551ef40 1 --2- 172.21.15.45:0/3786651525 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5580f54b8da0 0x5580f54b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.934+0000 7f8b0551ef40 1 -- 172.21.15.45:0/3786651525 >> 172.21.15.45:0/3786651525 conn(0x5580f53fe7e0 msgr2=0x5580f54d1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.934+0000 7f8b0551ef40 1 -- 172.21.15.45:0/3786651525 shutdown_connections 2024-05-08T05:51:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.934+0000 7f8b0551ef40 1 -- 172.21.15.45:0/3786651525 wait complete. 2024-05-08T05:51:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.934+0000 7f8b0551ef40 1 Processor -- start 2024-05-08T05:51:20.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.934+0000 7f8b0551ef40 1 -- start start 2024-05-08T05:51:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.935+0000 7f8b0551ef40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580f54b8da0 0x5580f54cf020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.935+0000 7f8b0551ef40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5580f54cd8f0 0x5580f54ac600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.935+0000 7f8b0551ef40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580f54cdcd0 0x5580f54acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:20.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.935+0000 7f8b0551ef40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5580f54b6070 con 0x5580f54cdcd0 2024-05-08T05:51:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.935+0000 7f8b0551ef40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5580f54d1080 con 0x5580f54b8da0 2024-05-08T05:51:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.935+0000 7f8b0551ef40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5580f54d0af0 con 0x5580f54cd8f0 2024-05-08T05:51:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.935+0000 7f8b02d4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580f54cdcd0 0x5580f54acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.935+0000 7f8b02d4b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580f54cdcd0 0x5580f54acb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51170/0 (socket says 172.21.15.45:51170) 2024-05-08T05:51:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.935+0000 7f8b02d4b640 1 -- 172.21.15.45:0/55736605 learned_addr learned my addr 172.21.15.45:0/55736605 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.935+0000 7f8b0254a640 1 --2- 172.21.15.45:0/55736605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580f54b8da0 0x5580f54cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.935+0000 7f8b01d49640 1 --2- 172.21.15.45:0/55736605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5580f54cd8f0 0x5580f54ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.935+0000 7f8b02d4b640 1 -- 172.21.15.45:0/55736605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5580f54cd8f0 msgr2=0x5580f54ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.935+0000 7f8b02d4b640 1 --2- 172.21.15.45:0/55736605 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5580f54cd8f0 0x5580f54ac600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.935+0000 7f8b02d4b640 1 -- 172.21.15.45:0/55736605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580f54b8da0 msgr2=0x5580f54cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.935+0000 7f8b02d4b640 1 --2- 172.21.15.45:0/55736605 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5580f54b8da0 0x5580f54cf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.935+0000 7f8b02d4b640 1 -- 172.21.15.45:0/55736605 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580f5444700 con 0x5580f54cdcd0 2024-05-08T05:51:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.936+0000 7f8b02d4b640 1 --2- 172.21.15.45:0/55736605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580f54cdcd0 0x5580f54acb40 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x5580f54b8b30 tx=0x7f8afc003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.936+0000 7f8afb7fe640 1 -- 172.21.15.45:0/55736605 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc026470 con 0x5580f54cdcd0 2024-05-08T05:51:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.936+0000 7f8afb7fe640 1 -- 172.21.15.45:0/55736605 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8afc021070 con 0x5580f54cdcd0 2024-05-08T05:51:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.936+0000 7f8afb7fe640 1 -- 172.21.15.45:0/55736605 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc026870 con 0x5580f54cdcd0 2024-05-08T05:51:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.936+0000 7f8b0551ef40 1 -- 172.21.15.45:0/55736605 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5580f54449a0 con 0x5580f54cdcd0 2024-05-08T05:51:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.936+0000 7f8b0551ef40 1 -- 172.21.15.45:0/55736605 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580f5444c40 con 0x5580f54cdcd0 2024-05-08T05:51:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.937+0000 7f8afb7fe640 1 -- 172.21.15.45:0/55736605 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8afc013420 con 0x5580f54cdcd0 2024-05-08T05:51:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.938+0000 7f8afb7fe640 1 --2- 172.21.15.45:0/55736605 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8ae4074cb0 0x7f8ae4077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.938+0000 7f8afb7fe640 1 -- 172.21.15.45:0/55736605 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f8afc097db0 con 0x5580f54cdcd0 2024-05-08T05:51:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.938+0000 7f8b0551ef40 1 --2- 172.21.15.45:0/55736605 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x5580f54cf560 0x5580f54b3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.938+0000 7f8b0254a640 1 --2- 172.21.15.45:0/55736605 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8ae4074cb0 0x7f8ae4077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.938+0000 7f8b0551ef40 1 -- 172.21.15.45:0/55736605 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:1 17.7 17:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x5580f54b3ac0 con 0x5580f54cf560 2024-05-08T05:51:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.938+0000 7f8b01d49640 1 --2- 172.21.15.45:0/55736605 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x5580f54cf560 0x5580f54b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.938+0000 7f8b0254a640 1 --2- 172.21.15.45:0/55736605 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8ae4074cb0 0x7f8ae4077170 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8af40044f0 tx=0x7f8af4006cc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:20.941 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.938+0000 7f8b01d49640 1 --2- 172.21.15.45:0/55736605 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x5580f54cf560 0x5580f54b3580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.942+0000 7f8b01d49640 1 -- 172.21.15.45:0/55736605 <== osd.0 v2:172.21.15.45:6800/3734454393 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v120'5 uv5 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8aec05c040 con 0x5580f54cf560 2024-05-08T05:51:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.942+0000 7f8b0551ef40 1 -- 172.21.15.45:0/55736605 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x5580f54cf560 msgr2=0x5580f54b3580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.942+0000 7f8b0551ef40 1 --2- 172.21.15.45:0/55736605 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x5580f54cf560 0x5580f54b3580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.942+0000 7f8b0551ef40 1 -- 172.21.15.45:0/55736605 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8ae4074cb0 msgr2=0x7f8ae4077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.942+0000 7f8b0551ef40 1 --2- 172.21.15.45:0/55736605 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8ae4074cb0 0x7f8ae4077170 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8af40044f0 tx=0x7f8af4006cc0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.942+0000 7f8b0551ef40 1 -- 172.21.15.45:0/55736605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580f54cdcd0 msgr2=0x5580f54acb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.942+0000 7f8b0551ef40 1 --2- 172.21.15.45:0/55736605 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5580f54cdcd0 0x5580f54acb40 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x5580f54b8b30 tx=0x7f8afc003020 comp rx=0 tx=0).stop 2024-05-08T05:51:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.942+0000 7f8b02d4b640 1 -- 172.21.15.45:0/55736605 reap_dead start 2024-05-08T05:51:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.942+0000 7f8b0551ef40 1 -- 172.21.15.45:0/55736605 shutdown_connections 2024-05-08T05:51:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.942+0000 7f8b0551ef40 1 -- 172.21.15.45:0/55736605 >> 172.21.15.45:0/55736605 conn(0x5580f53fe7e0 msgr2=0x5580f54d1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.942+0000 7f8b0551ef40 1 -- 172.21.15.45:0/55736605 shutdown_connections 2024-05-08T05:51:20.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:20.942+0000 7f8b0551ef40 1 -- 172.21.15.45:0/55736605 wait complete. 2024-05-08T05:51:20.945 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-08T05:51:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.052+0000 7f73ab293640 1 Processor -- start 2024-05-08T05:51:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.052+0000 7f73ab293640 1 -- start start 2024-05-08T05:51:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.053+0000 7f73ab293640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f73a4163460 0x7f73a40d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.053+0000 7f73ab293640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73a40d2bb0 0x7f73a415ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.053+0000 7f73ab293640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73a40d3320 con 0x7f73a40b6590 2024-05-08T05:51:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.053+0000 7f73ab293640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f73a40d3600 con 0x7f73a4163460 2024-05-08T05:51:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.053+0000 7f73ab293640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f73a40d38e0 con 0x7f73a40d2bb0 2024-05-08T05:51:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.053+0000 7f73a9830640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73a40d2bb0 0x7f73a415ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.053+0000 7f73a882e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f73a4163460 0x7f73a40d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.053+0000 7f73a9830640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73a40d2bb0 0x7f73a415ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:34252/0 (socket says 172.21.15.45:34252) 2024-05-08T05:51:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.053+0000 7f73a902f640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f73a40b6590 0x7f73a4162d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:59284/0 (socket says 172.21.15.45:59284) 2024-05-08T05:51:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.053+0000 7f73a882e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f73a4163460 0x7f73a40d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:47344/0 (socket says 172.21.15.45:47344) 2024-05-08T05:51:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.053+0000 7f73a9830640 1 -- 172.21.15.45:0/2925245505 learned_addr learned my addr 172.21.15.45:0/2925245505 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.054+0000 7f73a882e640 1 -- 172.21.15.45:0/2925245505 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73a40d2bb0 msgr2=0x7f73a415ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.054+0000 7f73a882e640 1 --2- 172.21.15.45:0/2925245505 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73a40d2bb0 0x7f73a415ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:21.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.054+0000 7f73a882e640 1 -- 172.21.15.45:0/2925245505 >> v1:172.21.15.63:6789/0 conn(0x7f73a40b6590 legacy=0x7f73a4162d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:21.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.054+0000 7f739bfff640 1 -- 172.21.15.45:0/2925245505 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3095738665 0 0) 0x7f73a40d3320 con 0x7f73a40b6590 2024-05-08T05:51:21.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.054+0000 7f73a882e640 1 -- 172.21.15.45:0/2925245505 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a40d3bc0 con 0x7f73a4163460 2024-05-08T05:51:21.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.054+0000 7f73a882e640 1 --2- 172.21.15.45:0/2925245505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73a4163460 0x7f73a40d0bd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f73940097a0 tx=0x7f739402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31b87433c2c8e955 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:21.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.055+0000 7f739bfff640 1 -- 172.21.15.45:0/2925245505 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394037070 con 0x7f73a4163460 2024-05-08T05:51:21.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.055+0000 7f739bfff640 1 -- 172.21.15.45:0/2925245505 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7394032440 con 0x7f73a4163460 2024-05-08T05:51:21.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.055+0000 7f739bfff640 1 -- 172.21.15.45:0/2925245505 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394032810 con 0x7f73a4163460 2024-05-08T05:51:21.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.055+0000 7f73ab293640 1 -- 172.21.15.45:0/2925245505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73a4163460 msgr2=0x7f73a40d0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:21.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.055+0000 7f73ab293640 1 --2- 172.21.15.45:0/2925245505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73a4163460 0x7f73a40d0bd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f73940097a0 tx=0x7f739402a180 comp rx=0 tx=0).stop 2024-05-08T05:51:21.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.055+0000 7f73ab293640 1 -- 172.21.15.45:0/2925245505 shutdown_connections 2024-05-08T05:51:21.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.055+0000 7f73ab293640 1 --2- 172.21.15.45:0/2925245505 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73a40d2bb0 0x7f73a415ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:21.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.055+0000 7f73ab293640 1 --2- 172.21.15.45:0/2925245505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73a4163460 0x7f73a40d0bd0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:21.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.055+0000 7f73ab293640 1 -- 172.21.15.45:0/2925245505 >> 172.21.15.45:0/2925245505 conn(0x7f73a40d4fe0 msgr2=0x7f73a40d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:21.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.055+0000 7f73ab293640 1 -- 172.21.15.45:0/2925245505 shutdown_connections 2024-05-08T05:51:21.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.055+0000 7f73ab293640 1 -- 172.21.15.45:0/2925245505 wait complete. 2024-05-08T05:51:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.056+0000 7f73ab293640 1 Processor -- start 2024-05-08T05:51:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.057+0000 7f73ab293640 1 -- start start 2024-05-08T05:51:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.057+0000 7f73ab293640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73a40b6590 0x7f73a415f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.057+0000 7f73ab293640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a40d2bb0 0x7f73a414e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.057+0000 7f73ab293640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73a4163460 0x7f73a414e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.057+0000 7f73ab293640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f73a415f170 con 0x7f73a40d2bb0 2024-05-08T05:51:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.057+0000 7f73ab293640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f73a40d3040 con 0x7f73a4163460 2024-05-08T05:51:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.057+0000 7f73ab293640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f73a415f2f0 con 0x7f73a40b6590 2024-05-08T05:51:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.057+0000 7f73a882e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a40d2bb0 0x7f73a414e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.057+0000 7f73a882e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a40d2bb0 0x7f73a414e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51182/0 (socket says 172.21.15.45:51182) 2024-05-08T05:51:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.057+0000 7f73a882e640 1 -- 172.21.15.45:0/2133394303 learned_addr learned my addr 172.21.15.45:0/2133394303 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.057+0000 7f73a9830640 1 --2- 172.21.15.45:0/2133394303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73a4163460 0x7f73a414e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:21.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.057+0000 7f73a902f640 1 --2- 172.21.15.45:0/2133394303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73a40b6590 0x7f73a415f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.058+0000 7f73a882e640 1 -- 172.21.15.45:0/2133394303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73a40b6590 msgr2=0x7f73a415f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.058+0000 7f73a882e640 1 --2- 172.21.15.45:0/2133394303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73a40b6590 0x7f73a415f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.058+0000 7f73a882e640 1 -- 172.21.15.45:0/2133394303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73a4163460 msgr2=0x7f73a414e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.058+0000 7f73a882e640 1 --2- 172.21.15.45:0/2133394303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73a4163460 0x7f73a414e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.058+0000 7f73a882e640 1 -- 172.21.15.45:0/2133394303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a41625a0 con 0x7f73a40d2bb0 2024-05-08T05:51:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.058+0000 7f73a882e640 1 --2- 172.21.15.45:0/2133394303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a40d2bb0 0x7f73a414e000 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f7394035040 tx=0x7f73940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.058+0000 7f7399ffb640 1 -- 172.21.15.45:0/2133394303 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394037040 con 0x7f73a40d2bb0 2024-05-08T05:51:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.058+0000 7f7399ffb640 1 -- 172.21.15.45:0/2133394303 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7394030500 con 0x7f73a40d2bb0 2024-05-08T05:51:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.058+0000 7f7399ffb640 1 -- 172.21.15.45:0/2133394303 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394030910 con 0x7f73a40d2bb0 2024-05-08T05:51:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.058+0000 7f73ab293640 1 -- 172.21.15.45:0/2133394303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a4162880 con 0x7f73a40d2bb0 2024-05-08T05:51:21.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.058+0000 7f73ab293640 1 -- 172.21.15.45:0/2133394303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a40c91b0 con 0x7f73a40d2bb0 2024-05-08T05:51:21.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.059+0000 7f73ab293640 1 -- 172.21.15.45:0/2133394303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a40693f0 con 0x7f73a40d2bb0 2024-05-08T05:51:21.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.060+0000 7f7399ffb640 1 -- 172.21.15.45:0/2133394303 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7394044020 con 0x7f73a40d2bb0 2024-05-08T05:51:21.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.061+0000 7f7399ffb640 1 --2- 172.21.15.45:0/2133394303 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f73880762d0 0x7f7388078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:21.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.061+0000 7f7399ffb640 1 -- 172.21.15.45:0/2133394303 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f73940b7a30 con 0x7f73a40d2bb0 2024-05-08T05:51:21.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.061+0000 7f73a902f640 1 --2- 172.21.15.45:0/2133394303 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f73880762d0 0x7f7388078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:21.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.061+0000 7f73a902f640 1 --2- 172.21.15.45:0/2133394303 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f73880762d0 0x7f7388078790 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f73a0000f60 tx=0x7f73a0007cf0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:21.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.065+0000 7f7399ffb640 1 -- 172.21.15.45:0/2133394303 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7394080ad0 con 0x7f73a40d2bb0 2024-05-08T05:51:21.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.298+0000 7f73ab293640 1 -- 172.21.15.45:0/2133394303 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f73a40d4260 con 0x7f73a40d2bb0 2024-05-08T05:51:21.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.653+0000 7f7399ffb640 1 -- 172.21.15.45:0/2133394303 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v102) ==== 162+0+0 (secure 0 0 0) 0x7f7394084dd0 con 0x7f73a40d2bb0 2024-05-08T05:51:21.654 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-08T05:51:21.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.656+0000 7f73ab293640 1 -- 172.21.15.45:0/2133394303 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f73880762d0 msgr2=0x7f7388078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:21.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.656+0000 7f73ab293640 1 --2- 172.21.15.45:0/2133394303 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f73880762d0 0x7f7388078790 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f73a0000f60 tx=0x7f73a0007cf0 comp rx=0 tx=0).stop 2024-05-08T05:51:21.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.656+0000 7f73ab293640 1 -- 172.21.15.45:0/2133394303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a40d2bb0 msgr2=0x7f73a414e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:21.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.656+0000 7f73ab293640 1 --2- 172.21.15.45:0/2133394303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a40d2bb0 0x7f73a414e000 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f7394035040 tx=0x7f73940046c0 comp rx=0 tx=0).stop 2024-05-08T05:51:21.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.656+0000 7f73ab293640 1 -- 172.21.15.45:0/2133394303 shutdown_connections 2024-05-08T05:51:21.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.656+0000 7f73ab293640 1 --2- 172.21.15.45:0/2133394303 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f73880762d0 0x7f7388078790 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:21.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.656+0000 7f73ab293640 1 --2- 172.21.15.45:0/2133394303 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73a4163460 0x7f73a414e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:21.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.656+0000 7f73ab293640 1 --2- 172.21.15.45:0/2133394303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f73a40d2bb0 0x7f73a414e000 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:21.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.656+0000 7f73ab293640 1 --2- 172.21.15.45:0/2133394303 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f73a40b6590 0x7f73a415f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:21.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.656+0000 7f73ab293640 1 -- 172.21.15.45:0/2133394303 >> 172.21.15.45:0/2133394303 conn(0x7f73a40d4fe0 msgr2=0x7f73a40d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:21.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.656+0000 7f73ab293640 1 -- 172.21.15.45:0/2133394303 shutdown_connections 2024-05-08T05:51:21.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.656+0000 7f73ab293640 1 -- 172.21.15.45:0/2133394303 wait complete. 2024-05-08T05:51:21.671 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:51:21.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.776+0000 7fc287b0a640 1 Processor -- start 2024-05-08T05:51:21.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.776+0000 7fc287b0a640 1 -- start start 2024-05-08T05:51:21.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.777+0000 7fc287b0a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc28015f420 0x7fc2800d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:21.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.777+0000 7fc287b0a640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc2800d3ca0 0x7fc2800d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:21.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.777+0000 7fc287b0a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2800d1eb0 0x7fc2800d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:21.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.777+0000 7fc287b0a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc2801630b0 con 0x7fc28015f420 2024-05-08T05:51:21.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.777+0000 7fc287b0a640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc2800d2550 con 0x7fc2800d3ca0 2024-05-08T05:51:21.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.777+0000 7fc287b0a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc2800d2c80 con 0x7fc2800d1eb0 2024-05-08T05:51:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.777+0000 7fc2858a6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc28015f420 0x7fc2800d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.777+0000 7fc2858a6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc28015f420 0x7fc2800d3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51196/0 (socket says 172.21.15.45:51196) 2024-05-08T05:51:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.777+0000 7fc2858a6640 1 -- 172.21.15.45:0/1898187430 learned_addr learned my addr 172.21.15.45:0/1898187430 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.777+0000 7fc2858a6640 1 -- 172.21.15.45:0/1898187430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2800d1eb0 msgr2=0x7fc2800d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.777+0000 7fc2860a7640 1 --2- 172.21.15.45:0/1898187430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2800d1eb0 0x7fc2800d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.777+0000 7fc2850a5640 1 --2- 172.21.15.45:0/1898187430 >> v2:172.21.15.63:3300/0 conn(0x7fc2800d3ca0 0x7fc2800d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.777+0000 7fc2858a6640 1 --2- 172.21.15.45:0/1898187430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2800d1eb0 0x7fc2800d4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.777+0000 7fc2858a6640 1 -- 172.21.15.45:0/1898187430 >> v2:172.21.15.63:3300/0 conn(0x7fc2800d3ca0 msgr2=0x7fc2800d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.777+0000 7fc2858a6640 1 --2- 172.21.15.45:0/1898187430 >> v2:172.21.15.63:3300/0 conn(0x7fc2800d3ca0 0x7fc2800d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.777+0000 7fc2858a6640 1 -- 172.21.15.45:0/1898187430 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2800d2f60 con 0x7fc28015f420 2024-05-08T05:51:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.778+0000 7fc2858a6640 1 --2- 172.21.15.45:0/1898187430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc28015f420 0x7fc2800d3760 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fc28008f380 tx=0x7fc27c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171703c07392f07b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:21.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.778+0000 7fc2848a4640 1 -- 172.21.15.45:0/1898187430 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c037070 con 0x7fc28015f420 2024-05-08T05:51:21.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.778+0000 7fc2848a4640 1 -- 172.21.15.45:0/1898187430 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc27c032440 con 0x7fc28015f420 2024-05-08T05:51:21.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.778+0000 7fc2848a4640 1 -- 172.21.15.45:0/1898187430 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c032810 con 0x7fc28015f420 2024-05-08T05:51:21.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.779+0000 7fc287b0a640 1 -- 172.21.15.45:0/1898187430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc28015f420 msgr2=0x7fc2800d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.779+0000 7fc287b0a640 1 --2- 172.21.15.45:0/1898187430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc28015f420 0x7fc2800d3760 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fc28008f380 tx=0x7fc27c02a1e0 comp rx=0 tx=0).stop 2024-05-08T05:51:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.779+0000 7fc287b0a640 1 -- 172.21.15.45:0/1898187430 shutdown_connections 2024-05-08T05:51:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.779+0000 7fc287b0a640 1 --2- 172.21.15.45:0/1898187430 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2800d1eb0 0x7fc2800d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.779+0000 7fc287b0a640 1 --2- 172.21.15.45:0/1898187430 >> v2:172.21.15.63:3300/0 conn(0x7fc2800d3ca0 0x7fc2800d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.779+0000 7fc287b0a640 1 --2- 172.21.15.45:0/1898187430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc28015f420 0x7fc2800d3760 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.779+0000 7fc287b0a640 1 -- 172.21.15.45:0/1898187430 >> 172.21.15.45:0/1898187430 conn(0x7fc28008eac0 msgr2=0x7fc28009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.779+0000 7fc287b0a640 1 -- 172.21.15.45:0/1898187430 shutdown_connections 2024-05-08T05:51:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.779+0000 7fc287b0a640 1 -- 172.21.15.45:0/1898187430 wait complete. 2024-05-08T05:51:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.780+0000 7fc287b0a640 1 Processor -- start 2024-05-08T05:51:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.780+0000 7fc287b0a640 1 -- start start 2024-05-08T05:51:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.781+0000 7fc287b0a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2800d1eb0 0x7fc280152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.781+0000 7fc287b0a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2800cc0e0 0x7fc2801612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:21.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.781+0000 7fc287b0a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2800cc560 0x7fc280161820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.781+0000 7fc287b0a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc2800d5200 con 0x7fc2800d1eb0 2024-05-08T05:51:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.781+0000 7fc287b0a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc2800d2830 con 0x7fc2800cc560 2024-05-08T05:51:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.781+0000 7fc287b0a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc2800d5380 con 0x7fc2800cc0e0 2024-05-08T05:51:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.781+0000 7fc2858a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2800d1eb0 0x7fc280152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.781+0000 7fc2858a6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2800d1eb0 0x7fc280152650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51208/0 (socket says 172.21.15.45:51208) 2024-05-08T05:51:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.781+0000 7fc2858a6640 1 -- 172.21.15.45:0/258774866 learned_addr learned my addr 172.21.15.45:0/258774866 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.781+0000 7fc2858a6640 1 -- 172.21.15.45:0/258774866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2800cc0e0 msgr2=0x7fc2801612e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.781+0000 7fc2858a6640 1 --2- 172.21.15.45:0/258774866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2800cc0e0 0x7fc2801612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.781+0000 7fc2858a6640 1 -- 172.21.15.45:0/258774866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2800cc560 msgr2=0x7fc280161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.781+0000 7fc2858a6640 1 --2- 172.21.15.45:0/258774866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2800cc560 0x7fc280161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.781+0000 7fc2858a6640 1 -- 172.21.15.45:0/258774866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2800ccaf0 con 0x7fc2800d1eb0 2024-05-08T05:51:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.781+0000 7fc2858a6640 1 --2- 172.21.15.45:0/258774866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2800d1eb0 0x7fc280152650 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc27c000c00 tx=0x7fc27c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:21.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.782+0000 7fc2767fc640 1 -- 172.21.15.45:0/258774866 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c037040 con 0x7fc2800d1eb0 2024-05-08T05:51:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.782+0000 7fc2767fc640 1 -- 172.21.15.45:0/258774866 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc27c042070 con 0x7fc2800d1eb0 2024-05-08T05:51:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.782+0000 7fc287b0a640 1 -- 172.21.15.45:0/258774866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2800ccdd0 con 0x7fc2800d1eb0 2024-05-08T05:51:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.782+0000 7fc2767fc640 1 -- 172.21.15.45:0/258774866 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c030bb0 con 0x7fc2800d1eb0 2024-05-08T05:51:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.782+0000 7fc287b0a640 1 -- 172.21.15.45:0/258774866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2800cd390 con 0x7fc2800d1eb0 2024-05-08T05:51:21.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.783+0000 7fc287b0a640 1 -- 172.21.15.45:0/258774866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2800693f0 con 0x7fc2800d1eb0 2024-05-08T05:51:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.783+0000 7fc2767fc640 1 -- 172.21.15.45:0/258774866 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc27c047460 con 0x7fc2800d1eb0 2024-05-08T05:51:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.784+0000 7fc2767fc640 1 --2- 172.21.15.45:0/258774866 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc264076300 0x7fc2640787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.784+0000 7fc2767fc640 1 -- 172.21.15.45:0/258774866 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7fc27c0b8440 con 0x7fc2800d1eb0 2024-05-08T05:51:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.784+0000 7fc2850a5640 1 --2- 172.21.15.45:0/258774866 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc264076300 0x7fc2640787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:21.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.785+0000 7fc2850a5640 1 --2- 172.21.15.45:0/258774866 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc264076300 0x7fc2640787c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc2800b37b0 tx=0x7fc270008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:21.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:21.789+0000 7fc2767fc640 1 -- 172.21.15.45:0/258774866 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc27c081bc0 con 0x7fc2800d1eb0 2024-05-08T05:51:22.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:22.020+0000 7fc287b0a640 1 -- 172.21.15.45:0/258774866 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2800b4c50 con 0x7fc2800d1eb0 2024-05-08T05:51:22.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:22.020+0000 7fc2767fc640 1 -- 172.21.15.45:0/258774866 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5178 (secure 0 0 0) 0x7fc27c0814c0 con 0x7fc2800d1eb0 2024-05-08T05:51:22.020 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:22.020 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":103,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7168,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3882445426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3882445426},{"type":"v1","addr":"172.21.15.63:6836","nonce":3882445426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/3173207720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3173207720},{"type":"v1","addr":"172.21.15.45:6833","nonce":3173207720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4254619360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4254619360},{"type":"v1","addr":"172.21.15.63:6837","nonce":4254619360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:51:04.429358+0000","modified":"2024-05-08T05:51:21.656948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8319":{"gid":8319,"name":"c","rank":0,"incarnation":103,"state":"up:replay","state_seq":1,"addr":"172.21.15.45:6835/453198328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":453198328},{"type":"v1","addr":"172.21.15.45:6835","nonce":453198328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-08T05:51:22.021 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 103 2024-05-08T05:51:22.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:22.022+0000 7fc287b0a640 1 -- 172.21.15.45:0/258774866 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc264076300 msgr2=0x7fc2640787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:22.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:22.022+0000 7fc287b0a640 1 --2- 172.21.15.45:0/258774866 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc264076300 0x7fc2640787c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc2800b37b0 tx=0x7fc270008040 comp rx=0 tx=0).stop 2024-05-08T05:51:22.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:22.022+0000 7fc287b0a640 1 -- 172.21.15.45:0/258774866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2800d1eb0 msgr2=0x7fc280152650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:22.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:22.022+0000 7fc287b0a640 1 --2- 172.21.15.45:0/258774866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2800d1eb0 0x7fc280152650 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc27c000c00 tx=0x7fc27c0046c0 comp rx=0 tx=0).stop 2024-05-08T05:51:22.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:22.022+0000 7fc287b0a640 1 -- 172.21.15.45:0/258774866 shutdown_connections 2024-05-08T05:51:22.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:22.022+0000 7fc287b0a640 1 --2- 172.21.15.45:0/258774866 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc264076300 0x7fc2640787c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:22.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:22.022+0000 7fc287b0a640 1 --2- 172.21.15.45:0/258774866 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc2800cc0e0 0x7fc2801612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:22.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:22.022+0000 7fc287b0a640 1 --2- 172.21.15.45:0/258774866 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc2800cc560 0x7fc280161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:22.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:22.022+0000 7fc287b0a640 1 --2- 172.21.15.45:0/258774866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc2800d1eb0 0x7fc280152650 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:22.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:22.022+0000 7fc287b0a640 1 -- 172.21.15.45:0/258774866 >> 172.21.15.45:0/258774866 conn(0x7fc28008eac0 msgr2=0x7fc28006dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:22.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:22.023+0000 7fc287b0a640 1 -- 172.21.15.45:0/258774866 shutdown_connections 2024-05-08T05:51:22.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:22.023+0000 7fc287b0a640 1 -- 172.21.15.45:0/258774866 wait complete. 2024-05-08T05:51:22.436 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:51:04.429358+0000', 'modified': '2024-05-08T05:51:21.656948+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 120, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8319}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8319': {'gid': 8319, 'name': 'c', 'rank': 0, 'incarnation': 103, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.45:6835/453198328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 453198328}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 453198328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-08T05:51:22.436 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8319:up:replay 2024-05-08T05:51:23.437 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:51:23.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.554+0000 7ff1c192c640 1 Processor -- start 2024-05-08T05:51:23.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.554+0000 7ff1c192c640 1 -- start start 2024-05-08T05:51:23.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.555+0000 7ff1c192c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff1bc06e0f0 0x7ff1bc0cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:23.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.555+0000 7ff1c192c640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1bc15d2a0 con 0x7ff1bc06f060 2024-05-08T05:51:23.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.555+0000 7ff1c192c640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1bc1609b0 con 0x7ff1bc0d0360 2024-05-08T05:51:23.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.555+0000 7ff1c192c640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff1bc0d07c0 con 0x7ff1bc06e0f0 2024-05-08T05:51:23.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.555+0000 7ff1ba7fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff1bc06e0f0 0x7ff1bc0cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:23.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.555+0000 7ff1baffd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff1bc06f060 0x7ff1bc06fe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49574/0 (socket says 172.21.15.45:49574) 2024-05-08T05:51:23.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.555+0000 7ff1ba7fc640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff1bc06e0f0 0x7ff1bc0cfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51218/0 (socket says 172.21.15.45:51218) 2024-05-08T05:51:23.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.555+0000 7ff1baffd640 1 -- 172.21.15.45:0/3764509434 learned_addr learned my addr 172.21.15.45:0/3764509434 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:23.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.555+0000 7ff1ba7fc640 1 -- 172.21.15.45:0/3764509434 >> v1:172.21.15.63:6789/0 conn(0x7ff1bc0d0360 legacy=0x7ff1bc0b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:23.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.555+0000 7ff1ba7fc640 1 -- 172.21.15.45:0/3764509434 >> v1:172.21.15.45:6789/0 conn(0x7ff1bc06f060 legacy=0x7ff1bc06fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:23.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.555+0000 7ff1ba7fc640 1 -- 172.21.15.45:0/3764509434 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1bc160c90 con 0x7ff1bc06e0f0 2024-05-08T05:51:23.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.556+0000 7ff1ba7fc640 1 --2- 172.21.15.45:0/3764509434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1bc06e0f0 0x7ff1bc0cfe20 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00097a0 tx=0x7ff1b002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b813939f3a48cd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:23.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.556+0000 7ff1b9ffb640 1 -- 172.21.15.45:0/3764509434 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0037070 con 0x7ff1bc06e0f0 2024-05-08T05:51:23.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.556+0000 7ff1b9ffb640 1 -- 172.21.15.45:0/3764509434 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b0032440 con 0x7ff1bc06e0f0 2024-05-08T05:51:23.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.556+0000 7ff1b9ffb640 1 -- 172.21.15.45:0/3764509434 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0032810 con 0x7ff1bc06e0f0 2024-05-08T05:51:23.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.557+0000 7ff1c192c640 1 -- 172.21.15.45:0/3764509434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1bc06e0f0 msgr2=0x7ff1bc0cfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:23.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.557+0000 7ff1c192c640 1 --2- 172.21.15.45:0/3764509434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1bc06e0f0 0x7ff1bc0cfe20 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00097a0 tx=0x7ff1b002a230 comp rx=0 tx=0).stop 2024-05-08T05:51:23.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.557+0000 7ff1c192c640 1 -- 172.21.15.45:0/3764509434 shutdown_connections 2024-05-08T05:51:23.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.557+0000 7ff1c192c640 1 --2- 172.21.15.45:0/3764509434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1bc06e0f0 0x7ff1bc0cfe20 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.557+0000 7ff1c192c640 1 -- 172.21.15.45:0/3764509434 >> 172.21.15.45:0/3764509434 conn(0x7ff1bc06c3a0 msgr2=0x7ff1bc06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:23.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.557+0000 7ff1c192c640 1 -- 172.21.15.45:0/3764509434 shutdown_connections 2024-05-08T05:51:23.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.557+0000 7ff1c192c640 1 -- 172.21.15.45:0/3764509434 wait complete. 2024-05-08T05:51:23.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.558+0000 7ff1c192c640 1 Processor -- start 2024-05-08T05:51:23.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.558+0000 7ff1c192c640 1 -- start start 2024-05-08T05:51:23.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.559+0000 7ff1c192c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1bc06e0f0 0x7ff1bc09e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.559+0000 7ff1c192c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1bc06f060 0x7ff1bc147950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.559+0000 7ff1c192c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1bc0d0360 0x7ff1bc147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.559+0000 7ff1c192c640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff1bc161250 con 0x7ff1bc06f060 2024-05-08T05:51:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.559+0000 7ff1c192c640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff1bc06f440 con 0x7ff1bc06e0f0 2024-05-08T05:51:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.559+0000 7ff1c192c640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff1bc161530 con 0x7ff1bc0d0360 2024-05-08T05:51:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.559+0000 7ff1ba7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1bc06f060 0x7ff1bc147950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.559+0000 7ff1ba7fc640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1bc06f060 0x7ff1bc147950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51224/0 (socket says 172.21.15.45:51224) 2024-05-08T05:51:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.559+0000 7ff1ba7fc640 1 -- 172.21.15.45:0/1831101643 learned_addr learned my addr 172.21.15.45:0/1831101643 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.559+0000 7ff1baffd640 1 --2- 172.21.15.45:0/1831101643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1bc06e0f0 0x7ff1bc09e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.560+0000 7ff1bb7fe640 1 --2- 172.21.15.45:0/1831101643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1bc0d0360 0x7ff1bc147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.560+0000 7ff1ba7fc640 1 -- 172.21.15.45:0/1831101643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1bc0d0360 msgr2=0x7ff1bc147e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.560+0000 7ff1ba7fc640 1 --2- 172.21.15.45:0/1831101643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1bc0d0360 0x7ff1bc147e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.560+0000 7ff1ba7fc640 1 -- 172.21.15.45:0/1831101643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1bc06e0f0 msgr2=0x7ff1bc09e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.560+0000 7ff1ba7fc640 1 --2- 172.21.15.45:0/1831101643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1bc06e0f0 0x7ff1bc09e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.560+0000 7ff1ba7fc640 1 -- 172.21.15.45:0/1831101643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1bc09cf90 con 0x7ff1bc06f060 2024-05-08T05:51:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.560+0000 7ff1baffd640 1 --2- 172.21.15.45:0/1831101643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1bc06e0f0 0x7ff1bc09e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.560+0000 7ff1ba7fc640 1 --2- 172.21.15.45:0/1831101643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1bc06f060 0x7ff1bc147950 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0035040 tx=0x7ff1b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.560+0000 7ff1a3fff640 1 -- 172.21.15.45:0/1831101643 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0037040 con 0x7ff1bc06f060 2024-05-08T05:51:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.560+0000 7ff1a3fff640 1 -- 172.21.15.45:0/1831101643 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b0030770 con 0x7ff1bc06f060 2024-05-08T05:51:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.560+0000 7ff1c192c640 1 -- 172.21.15.45:0/1831101643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1bc09d270 con 0x7ff1bc06f060 2024-05-08T05:51:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.560+0000 7ff1a3fff640 1 -- 172.21.15.45:0/1831101643 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0030b80 con 0x7ff1bc06f060 2024-05-08T05:51:23.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.560+0000 7ff1c192c640 1 -- 172.21.15.45:0/1831101643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1bc09d830 con 0x7ff1bc06f060 2024-05-08T05:51:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.561+0000 7ff1c192c640 1 -- 172.21.15.45:0/1831101643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1bc0693f0 con 0x7ff1bc06f060 2024-05-08T05:51:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.562+0000 7ff1a3fff640 1 -- 172.21.15.45:0/1831101643 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff1b0044020 con 0x7ff1bc06f060 2024-05-08T05:51:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.562+0000 7ff1a3fff640 1 --2- 172.21.15.45:0/1831101643 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff1a80762d0 0x7ff1a8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.562+0000 7ff1a3fff640 1 -- 172.21.15.45:0/1831101643 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7ff1b00b7d90 con 0x7ff1bc06f060 2024-05-08T05:51:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.563+0000 7ff1baffd640 1 --2- 172.21.15.45:0/1831101643 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff1a80762d0 0x7ff1a8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:23.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.563+0000 7ff1baffd640 1 --2- 172.21.15.45:0/1831101643 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff1a80762d0 0x7ff1a8078790 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac000f60 tx=0x7ff1ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:23.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.567+0000 7ff1a3fff640 1 -- 172.21.15.45:0/1831101643 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff1b0032e20 con 0x7ff1bc06f060 2024-05-08T05:51:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.800+0000 7ff1c192c640 1 -- 172.21.15.45:0/1831101643 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1bc153c60 con 0x7ff1bc06f060 2024-05-08T05:51:23.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.800+0000 7ff1a3fff640 1 -- 172.21.15.45:0/1831101643 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5178 (secure 0 0 0) 0x7ff1b0081480 con 0x7ff1bc06f060 2024-05-08T05:51:23.801 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:23.801 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":105,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7168,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3882445426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3882445426},{"type":"v1","addr":"172.21.15.63:6836","nonce":3882445426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/3173207720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3173207720},{"type":"v1","addr":"172.21.15.45:6833","nonce":3173207720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4254619360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4254619360},{"type":"v1","addr":"172.21.15.63:6837","nonce":4254619360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:51:04.429358+0000","modified":"2024-05-08T05:51:22.664444+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8319":{"gid":8319,"name":"c","rank":0,"incarnation":103,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.45:6835/453198328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":453198328},{"type":"v1","addr":"172.21.15.45:6835","nonce":453198328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-08T05:51:23.801 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 105 2024-05-08T05:51:23.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.802+0000 7ff1c192c640 1 -- 172.21.15.45:0/1831101643 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff1a80762d0 msgr2=0x7ff1a8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:23.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.802+0000 7ff1c192c640 1 --2- 172.21.15.45:0/1831101643 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff1a80762d0 0x7ff1a8078790 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac000f60 tx=0x7ff1ac009040 comp rx=0 tx=0).stop 2024-05-08T05:51:23.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.802+0000 7ff1c192c640 1 -- 172.21.15.45:0/1831101643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1bc06f060 msgr2=0x7ff1bc147950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.802+0000 7ff1c192c640 1 --2- 172.21.15.45:0/1831101643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1bc06f060 0x7ff1bc147950 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0035040 tx=0x7ff1b00046c0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.802+0000 7ff1c192c640 1 -- 172.21.15.45:0/1831101643 shutdown_connections 2024-05-08T05:51:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.802+0000 7ff1c192c640 1 --2- 172.21.15.45:0/1831101643 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff1a80762d0 0x7ff1a8078790 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.802+0000 7ff1c192c640 1 --2- 172.21.15.45:0/1831101643 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff1bc0d0360 0x7ff1bc147e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.802+0000 7ff1c192c640 1 --2- 172.21.15.45:0/1831101643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1bc06f060 0x7ff1bc147950 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.802+0000 7ff1c192c640 1 --2- 172.21.15.45:0/1831101643 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1bc06e0f0 0x7ff1bc09e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.802+0000 7ff1c192c640 1 -- 172.21.15.45:0/1831101643 >> 172.21.15.45:0/1831101643 conn(0x7ff1bc06c3a0 msgr2=0x7ff1bc072db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.802+0000 7ff1c192c640 1 -- 172.21.15.45:0/1831101643 shutdown_connections 2024-05-08T05:51:23.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:23.802+0000 7ff1c192c640 1 -- 172.21.15.45:0/1831101643 wait complete. 2024-05-08T05:51:24.217 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:51:04.429358+0000', 'modified': '2024-05-08T05:51:22.664444+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 120, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8319}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8319': {'gid': 8319, 'name': 'c', 'rank': 0, 'incarnation': 103, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.45:6835/453198328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 453198328}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 453198328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-08T05:51:24.217 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8319:up:rejoin 2024-05-08T05:51:25.218 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:51:25.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.336+0000 7f1e90659640 1 Processor -- start 2024-05-08T05:51:25.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.336+0000 7f1e90659640 1 -- start start 2024-05-08T05:51:25.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.337+0000 7f1e90659640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e8815f380 0x7f1e880d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:25.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.337+0000 7f1e90659640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1e880d3cd0 0x7f1e880b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:25.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.337+0000 7f1e90659640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e880d1760 con 0x7f1e880d0cf0 2024-05-08T05:51:25.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.337+0000 7f1e90659640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1e880d30f0 con 0x7f1e880d3cd0 2024-05-08T05:51:25.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.337+0000 7f1e90659640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1e880d1150 con 0x7f1e8815f380 2024-05-08T05:51:25.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.337+0000 7f1e8dbf4640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f1e880d0cf0 0x7f1e880d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49584/0 (socket says 172.21.15.45:49584) 2024-05-08T05:51:25.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.337+0000 7f1e8dbf4640 1 -- 172.21.15.45:0/2033839341 learned_addr learned my addr 172.21.15.45:0/2033839341 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:25.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.337+0000 7f1e8e3f5640 1 --2- 172.21.15.45:0/2033839341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e8815f380 0x7f1e880d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:25.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.337+0000 7f1e8ebf6640 1 --2- 172.21.15.45:0/2033839341 >> v2:172.21.15.63:3300/0 conn(0x7f1e880d3cd0 0x7f1e880b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:25.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.337+0000 7f1e8d3f3640 1 -- 172.21.15.45:0/2033839341 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1697309952 0 0) 0x7f1e880d1760 con 0x7f1e880d0cf0 2024-05-08T05:51:25.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.337+0000 7f1e8d3f3640 1 -- 172.21.15.45:0/2033839341 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e74003520 con 0x7f1e880d0cf0 2024-05-08T05:51:25.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8d3f3640 1 -- 172.21.15.45:0/2033839341 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2516263945 0 0) 0x7f1e74003520 con 0x7f1e880d0cf0 2024-05-08T05:51:25.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8d3f3640 1 -- 172.21.15.45:0/2033839341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e8815f380 msgr2=0x7f1e880d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:25.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8d3f3640 1 --2- 172.21.15.45:0/2033839341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e8815f380 0x7f1e880d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:25.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8d3f3640 1 -- 172.21.15.45:0/2033839341 >> v2:172.21.15.63:3300/0 conn(0x7f1e880d3cd0 msgr2=0x7f1e880b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:25.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8d3f3640 1 --2- 172.21.15.45:0/2033839341 >> v2:172.21.15.63:3300/0 conn(0x7f1e880d3cd0 0x7f1e880b7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8d3f3640 1 -- 172.21.15.45:0/2033839341 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e880d1a40 con 0x7f1e880d0cf0 2024-05-08T05:51:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8d3f3640 1 -- 172.21.15.45:0/2033839341 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f1e7c0016f0 con 0x7f1e880d0cf0 2024-05-08T05:51:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8d3f3640 1 -- 172.21.15.45:0/2033839341 >> v1:172.21.15.45:6789/0 conn(0x7f1e880d0cf0 legacy=0x7f1e880d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8d3f3640 1 --2- 172.21.15.45:0/2033839341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e74003a50 0x7f1e74003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8d3f3640 1 --2- 172.21.15.45:0/2033839341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e8815f380 0x7f1e74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8d3f3640 1 --2- 172.21.15.45:0/2033839341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e74004d40 0x7f1e740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8d3f3640 1 -- 172.21.15.45:0/2033839341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1e880d30f0 con 0x7f1e74004d40 2024-05-08T05:51:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8d3f3640 1 -- 172.21.15.45:0/2033839341 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7400b9c0 con 0x7f1e74003a50 2024-05-08T05:51:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8d3f3640 1 -- 172.21.15.45:0/2033839341 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1e7400bb70 con 0x7f1e8815f380 2024-05-08T05:51:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8ebf6640 1 --2- 172.21.15.45:0/2033839341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e74004d40 0x7f1e740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8e3f5640 1 --2- 172.21.15.45:0/2033839341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e74003a50 0x7f1e74003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8dbf4640 1 --2- 172.21.15.45:0/2033839341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e8815f380 0x7f1e74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8ebf6640 1 -- 172.21.15.45:0/2033839341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e8815f380 msgr2=0x7f1e74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8ebf6640 1 --2- 172.21.15.45:0/2033839341 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e8815f380 0x7f1e74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:25.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8ebf6640 1 -- 172.21.15.45:0/2033839341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e74003a50 msgr2=0x7f1e74003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:25.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8ebf6640 1 --2- 172.21.15.45:0/2033839341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e74003a50 0x7f1e74003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:25.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8ebf6640 1 -- 172.21.15.45:0/2033839341 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e74003520 con 0x7f1e74004d40 2024-05-08T05:51:25.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.338+0000 7f1e8e3f5640 1 --2- 172.21.15.45:0/2033839341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e74003a50 0x7f1e74003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:25.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.339+0000 7f1e8ebf6640 1 --2- 172.21.15.45:0/2033839341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e74004d40 0x7f1e740051c0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009f90 tx=0x7f1e74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d42976e28be44e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:25.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.339+0000 7f1e8d3f3640 1 -- 172.21.15.45:0/2033839341 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80009c10 con 0x7f1e74004d40 2024-05-08T05:51:25.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.339+0000 7f1e8d3f3640 1 -- 172.21.15.45:0/2033839341 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e80036070 con 0x7f1e74004d40 2024-05-08T05:51:25.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.339+0000 7f1e8d3f3640 1 -- 172.21.15.45:0/2033839341 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80031880 con 0x7f1e74004d40 2024-05-08T05:51:25.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.340+0000 7f1e90659640 1 -- 172.21.15.45:0/2033839341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e74004d40 msgr2=0x7f1e740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:25.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.340+0000 7f1e90659640 1 --2- 172.21.15.45:0/2033839341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e74004d40 0x7f1e740051c0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009f90 tx=0x7f1e74005830 comp rx=0 tx=0).stop 2024-05-08T05:51:25.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.340+0000 7f1e8ebf6640 1 -- 172.21.15.45:0/2033839341 reap_dead start 2024-05-08T05:51:25.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.340+0000 7f1e90659640 1 -- 172.21.15.45:0/2033839341 shutdown_connections 2024-05-08T05:51:25.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.340+0000 7f1e90659640 1 -- 172.21.15.45:0/2033839341 >> 172.21.15.45:0/2033839341 conn(0x7f1e880d4fe0 msgr2=0x7f1e880d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:25.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.340+0000 7f1e90659640 1 -- 172.21.15.45:0/2033839341 shutdown_connections 2024-05-08T05:51:25.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.340+0000 7f1e90659640 1 -- 172.21.15.45:0/2033839341 wait complete. 2024-05-08T05:51:25.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.341+0000 7f1e90659640 1 Processor -- start 2024-05-08T05:51:25.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.341+0000 7f1e90659640 1 -- start start 2024-05-08T05:51:25.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.342+0000 7f1e90659640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e74004d40 0x7f1e88151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:25.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.342+0000 7f1e90659640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e88161fb0 0x7f1e8806ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:25.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.342+0000 7f1e90659640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e88162430 0x7f1e8806e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:25.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.342+0000 7f1e90659640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1e880d2000 con 0x7f1e88162430 2024-05-08T05:51:25.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.342+0000 7f1e90659640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1e880d2370 con 0x7f1e88161fb0 2024-05-08T05:51:25.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.342+0000 7f1e90659640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1e8815f760 con 0x7f1e74004d40 2024-05-08T05:51:25.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.342+0000 7f1e8ebf6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e88162430 0x7f1e8806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:25.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.342+0000 7f1e8ebf6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e88162430 0x7f1e8806e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51242/0 (socket says 172.21.15.45:51242) 2024-05-08T05:51:25.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.342+0000 7f1e8ebf6640 1 -- 172.21.15.45:0/4280333991 learned_addr learned my addr 172.21.15.45:0/4280333991 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:25.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.342+0000 7f1e8dbf4640 1 --2- 172.21.15.45:0/4280333991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e88161fb0 0x7f1e8806ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:25.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.342+0000 7f1e8e3f5640 1 --2- 172.21.15.45:0/4280333991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e74004d40 0x7f1e88151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:25.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.342+0000 7f1e8ebf6640 1 -- 172.21.15.45:0/4280333991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e74004d40 msgr2=0x7f1e88151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:25.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.342+0000 7f1e8ebf6640 1 --2- 172.21.15.45:0/4280333991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e74004d40 0x7f1e88151590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:25.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.342+0000 7f1e8ebf6640 1 -- 172.21.15.45:0/4280333991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e88161fb0 msgr2=0x7f1e8806ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:25.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.342+0000 7f1e8ebf6640 1 --2- 172.21.15.45:0/4280333991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e88161fb0 0x7f1e8806ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:25.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.342+0000 7f1e8ebf6640 1 -- 172.21.15.45:0/4280333991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e88161a40 con 0x7f1e88162430 2024-05-08T05:51:25.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.343+0000 7f1e8dbf4640 1 --2- 172.21.15.45:0/4280333991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e88161fb0 0x7f1e8806ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:25.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.343+0000 7f1e8e3f5640 1 --2- 172.21.15.45:0/4280333991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e74004d40 0x7f1e88151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:25.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.343+0000 7f1e8ebf6640 1 --2- 172.21.15.45:0/4280333991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e88162430 0x7f1e8806e330 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009f90 tx=0x7f1e80009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:25.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.343+0000 7f1e7affd640 1 -- 172.21.15.45:0/4280333991 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e84009690 con 0x7f1e88162430 2024-05-08T05:51:25.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.343+0000 7f1e7affd640 1 -- 172.21.15.45:0/4280333991 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e80003990 con 0x7f1e88162430 2024-05-08T05:51:25.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.343+0000 7f1e90659640 1 -- 172.21.15.45:0/4280333991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e880a17c0 con 0x7f1e88162430 2024-05-08T05:51:25.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.343+0000 7f1e7affd640 1 -- 172.21.15.45:0/4280333991 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80009c10 con 0x7f1e88162430 2024-05-08T05:51:25.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.343+0000 7f1e90659640 1 -- 172.21.15.45:0/4280333991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e880a1d80 con 0x7f1e88162430 2024-05-08T05:51:25.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.344+0000 7f1e90659640 1 -- 172.21.15.45:0/4280333991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e88163b60 con 0x7f1e88162430 2024-05-08T05:51:25.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.345+0000 7f1e7affd640 1 -- 172.21.15.45:0/4280333991 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1e80005d10 con 0x7f1e88162430 2024-05-08T05:51:25.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.345+0000 7f1e7affd640 1 --2- 172.21.15.45:0/4280333991 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1e6c0762d0 0x7f1e6c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:25.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.345+0000 7f1e7affd640 1 -- 172.21.15.45:0/4280333991 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f1e800b87a0 con 0x7f1e88162430 2024-05-08T05:51:25.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.345+0000 7f1e8e3f5640 1 --2- 172.21.15.45:0/4280333991 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1e6c0762d0 0x7f1e6c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:25.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.346+0000 7f1e8e3f5640 1 --2- 172.21.15.45:0/4280333991 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1e6c0762d0 0x7f1e6c078790 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009ce0 tx=0x7f1e84008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:25.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.350+0000 7f1e7affd640 1 -- 172.21.15.45:0/4280333991 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1e88163b60 con 0x7f1e88162430 2024-05-08T05:51:25.582 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.583+0000 7f1e90659640 1 -- 172.21.15.45:0/4280333991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e880a1f80 con 0x7f1e88162430 2024-05-08T05:51:25.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.583+0000 7f1e7affd640 1 -- 172.21.15.45:0/4280333991 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5185 (secure 0 0 0) 0x7f1e880a1f80 con 0x7f1e88162430 2024-05-08T05:51:25.583 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:25.583 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":106,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7168,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3882445426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3882445426},{"type":"v1","addr":"172.21.15.63:6836","nonce":3882445426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/3173207720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3173207720},{"type":"v1","addr":"172.21.15.45:6833","nonce":3173207720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4254619360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4254619360},{"type":"v1","addr":"172.21.15.63:6837","nonce":4254619360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:51:04.429358+0000","modified":"2024-05-08T05:51:24.668156+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8319":{"gid":8319,"name":"c","rank":0,"incarnation":103,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/453198328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":453198328},{"type":"v1","addr":"172.21.15.45:6835","nonce":453198328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8319,"qdb_cluster":[8319]},"id":8}]} 2024-05-08T05:51:25.584 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 106 2024-05-08T05:51:25.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.585+0000 7f1e90659640 1 -- 172.21.15.45:0/4280333991 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1e6c0762d0 msgr2=0x7f1e6c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:25.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.585+0000 7f1e90659640 1 --2- 172.21.15.45:0/4280333991 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1e6c0762d0 0x7f1e6c078790 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009ce0 tx=0x7f1e84008040 comp rx=0 tx=0).stop 2024-05-08T05:51:25.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.585+0000 7f1e90659640 1 -- 172.21.15.45:0/4280333991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e88162430 msgr2=0x7f1e8806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:25.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.585+0000 7f1e90659640 1 --2- 172.21.15.45:0/4280333991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e88162430 0x7f1e8806e330 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f1e80009f90 tx=0x7f1e80009750 comp rx=0 tx=0).stop 2024-05-08T05:51:25.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.585+0000 7f1e90659640 1 -- 172.21.15.45:0/4280333991 shutdown_connections 2024-05-08T05:51:25.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.585+0000 7f1e90659640 1 --2- 172.21.15.45:0/4280333991 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1e6c0762d0 0x7f1e6c078790 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:25.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.585+0000 7f1e90659640 1 --2- 172.21.15.45:0/4280333991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e88162430 0x7f1e8806e330 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:25.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.585+0000 7f1e90659640 1 --2- 172.21.15.45:0/4280333991 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e88161fb0 0x7f1e8806ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:25.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.585+0000 7f1e90659640 1 --2- 172.21.15.45:0/4280333991 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1e74004d40 0x7f1e88151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:25.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.585+0000 7f1e90659640 1 -- 172.21.15.45:0/4280333991 >> 172.21.15.45:0/4280333991 conn(0x7f1e880d4fe0 msgr2=0x7f1e880d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:25.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.585+0000 7f1e90659640 1 -- 172.21.15.45:0/4280333991 shutdown_connections 2024-05-08T05:51:25.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:25.585+0000 7f1e90659640 1 -- 172.21.15.45:0/4280333991 wait complete. 2024-05-08T05:51:25.999 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 106, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:51:04.429358+0000', 'modified': '2024-05-08T05:51:24.668156+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 120, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8319}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8319': {'gid': 8319, 'name': 'c', 'rank': 0, 'incarnation': 103, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.45:6835/453198328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 453198328}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 453198328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8319, 'qdb_cluster': [8319]} 2024-05-08T05:51:25.999 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T05:51:25.999 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T05:51:26.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.110+0000 7f5115b2b640 1 Processor -- start 2024-05-08T05:51:26.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.110+0000 7f5115b2b640 1 -- start start 2024-05-08T05:51:26.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.111+0000 7f5115b2b640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f5110070d30 0x7f51100d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.111+0000 7f5115b2b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5110163920 0x7f51100703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.111+0000 7f5115b2b640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5110163d80 con 0x7f5110071110 2024-05-08T05:51:26.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.111+0000 7f5115b2b640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f51100d10d0 con 0x7f5110070d30 2024-05-08T05:51:26.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.111+0000 7f5115b2b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5110070b80 con 0x7f5110163920 2024-05-08T05:51:26.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.111+0000 7f510ffff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f5110071110 0x7f511006d4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49598/0 (socket says 172.21.15.45:49598) 2024-05-08T05:51:26.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.111+0000 7f510ffff640 1 -- 172.21.15.45:0/3518306618 learned_addr learned my addr 172.21.15.45:0/3518306618 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:26.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.111+0000 7f510f7fe640 1 --2- 172.21.15.45:0/3518306618 >> v2:172.21.15.63:3300/0 conn(0x7f5110070d30 0x7f51100d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.111+0000 7f510effd640 1 --2- 172.21.15.45:0/3518306618 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5110163920 0x7f51100703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.111+0000 7f510e7fc640 1 -- 172.21.15.45:0/3518306618 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2363802480 0 0) 0x7f5110163d80 con 0x7f5110071110 2024-05-08T05:51:26.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510e7fc640 1 -- 172.21.15.45:0/3518306618 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50f8003520 con 0x7f5110071110 2024-05-08T05:51:26.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510e7fc640 1 -- 172.21.15.45:0/3518306618 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2772144296 0 0) 0x7f50f8003520 con 0x7f5110071110 2024-05-08T05:51:26.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510e7fc640 1 -- 172.21.15.45:0/3518306618 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5110163920 msgr2=0x7f51100703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:26.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510e7fc640 1 --2- 172.21.15.45:0/3518306618 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5110163920 0x7f51100703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510e7fc640 1 -- 172.21.15.45:0/3518306618 >> v2:172.21.15.63:3300/0 conn(0x7f5110070d30 msgr2=0x7f51100d20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:26.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510e7fc640 1 --2- 172.21.15.45:0/3518306618 >> v2:172.21.15.63:3300/0 conn(0x7f5110070d30 0x7f51100d20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510e7fc640 1 -- 172.21.15.45:0/3518306618 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51100708e0 con 0x7f5110071110 2024-05-08T05:51:26.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510e7fc640 1 -- 172.21.15.45:0/3518306618 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f5108001a70 con 0x7f5110071110 2024-05-08T05:51:26.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510e7fc640 1 -- 172.21.15.45:0/3518306618 >> v1:172.21.15.45:6789/0 conn(0x7f5110071110 legacy=0x7f511006d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510e7fc640 1 --2- 172.21.15.45:0/3518306618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50f8003a00 0x7f50f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510e7fc640 1 --2- 172.21.15.45:0/3518306618 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5110163920 0x7f50f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510e7fc640 1 --2- 172.21.15.45:0/3518306618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f8004d40 0x7f50f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510e7fc640 1 -- 172.21.15.45:0/3518306618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5110070b80 con 0x7f50f8003a00 2024-05-08T05:51:26.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510e7fc640 1 -- 172.21.15.45:0/3518306618 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50f800b9c0 con 0x7f50f8004d40 2024-05-08T05:51:26.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510e7fc640 1 -- 172.21.15.45:0/3518306618 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f50f800bb70 con 0x7f5110163920 2024-05-08T05:51:26.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510f7fe640 1 --2- 172.21.15.45:0/3518306618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50f8003a00 0x7f50f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510effd640 1 --2- 172.21.15.45:0/3518306618 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5110163920 0x7f50f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510ffff640 1 --2- 172.21.15.45:0/3518306618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f8004d40 0x7f50f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510f7fe640 1 -- 172.21.15.45:0/3518306618 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5110163920 msgr2=0x7f50f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:26.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510f7fe640 1 --2- 172.21.15.45:0/3518306618 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f5110163920 0x7f50f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510f7fe640 1 -- 172.21.15.45:0/3518306618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f8004d40 msgr2=0x7f50f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:26.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510f7fe640 1 --2- 172.21.15.45:0/3518306618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f8004d40 0x7f50f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510f7fe640 1 -- 172.21.15.45:0/3518306618 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f8003520 con 0x7f50f8003a00 2024-05-08T05:51:26.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.112+0000 7f510ffff640 1 --2- 172.21.15.45:0/3518306618 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50f8004d40 0x7f50f80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:26.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.113+0000 7f510f7fe640 1 --2- 172.21.15.45:0/3518306618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50f8003a00 0x7f50f8003e20 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f5100009ac0 tx=0x7f50f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875e554966eb9a25 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:26.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.113+0000 7f510e7fc640 1 -- 172.21.15.45:0/3518306618 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100036070 con 0x7f50f8003a00 2024-05-08T05:51:26.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.113+0000 7f510e7fc640 1 -- 172.21.15.45:0/3518306618 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5100031400 con 0x7f50f8003a00 2024-05-08T05:51:26.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.113+0000 7f510e7fc640 1 -- 172.21.15.45:0/3518306618 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100009690 con 0x7f50f8003a00 2024-05-08T05:51:26.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.114+0000 7f5115b2b640 1 -- 172.21.15.45:0/3518306618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50f8003a00 msgr2=0x7f50f8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.114+0000 7f5115b2b640 1 --2- 172.21.15.45:0/3518306618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50f8003a00 0x7f50f8003e20 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f5100009ac0 tx=0x7f50f8005830 comp rx=0 tx=0).stop 2024-05-08T05:51:26.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.114+0000 7f510ffff640 1 -- 172.21.15.45:0/3518306618 reap_dead start 2024-05-08T05:51:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.114+0000 7f5115b2b640 1 -- 172.21.15.45:0/3518306618 shutdown_connections 2024-05-08T05:51:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.114+0000 7f5115b2b640 1 -- 172.21.15.45:0/3518306618 >> 172.21.15.45:0/3518306618 conn(0x7f51100d2e90 msgr2=0x7f51100d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.114+0000 7f5115b2b640 1 -- 172.21.15.45:0/3518306618 shutdown_connections 2024-05-08T05:51:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.114+0000 7f5115b2b640 1 -- 172.21.15.45:0/3518306618 wait complete. 2024-05-08T05:51:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.115+0000 7f5115b2b640 1 Processor -- start 2024-05-08T05:51:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.115+0000 7f5115b2b640 1 -- start start 2024-05-08T05:51:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.116+0000 7f5115b2b640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50f8003a00 0x7f51100c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.116+0000 7f5115b2b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f511015e8b0 0x7f51100cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.116+0000 7f5115b2b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f511015dce0 0x7f511015e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.116+0000 7f5115b2b640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f51100d15e0 con 0x7f511015e8b0 2024-05-08T05:51:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.116+0000 7f5115b2b640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f51100d1c30 con 0x7f511015dce0 2024-05-08T05:51:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.116+0000 7f5115b2b640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f51100d18c0 con 0x7f50f8003a00 2024-05-08T05:51:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.116+0000 7f510effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f511015e8b0 0x7f51100cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.116+0000 7f510effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f511015e8b0 0x7f51100cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51254/0 (socket says 172.21.15.45:51254) 2024-05-08T05:51:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.116+0000 7f510effd640 1 -- 172.21.15.45:0/1352839790 learned_addr learned my addr 172.21.15.45:0/1352839790 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.116+0000 7f510effd640 1 -- 172.21.15.45:0/1352839790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50f8003a00 msgr2=0x7f51100c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.116+0000 7f510effd640 1 --2- 172.21.15.45:0/1352839790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50f8003a00 0x7f51100c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.116+0000 7f510effd640 1 -- 172.21.15.45:0/1352839790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f511015dce0 msgr2=0x7f511015e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.116+0000 7f510effd640 1 --2- 172.21.15.45:0/1352839790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f511015dce0 0x7f511015e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.116+0000 7f510effd640 1 -- 172.21.15.45:0/1352839790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511015e6a0 con 0x7f511015e8b0 2024-05-08T05:51:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.117+0000 7f510effd640 1 --2- 172.21.15.45:0/1352839790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f511015e8b0 0x7f51100cd730 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f51040119a0 tx=0x7f5104011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.117+0000 7f50f7fff640 1 -- 172.21.15.45:0/1352839790 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510400fa90 con 0x7f511015e8b0 2024-05-08T05:51:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.117+0000 7f50f7fff640 1 -- 172.21.15.45:0/1352839790 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f510400fc30 con 0x7f511015e8b0 2024-05-08T05:51:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.117+0000 7f5115b2b640 1 -- 172.21.15.45:0/1352839790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51100c7570 con 0x7f511015e8b0 2024-05-08T05:51:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.117+0000 7f50f7fff640 1 -- 172.21.15.45:0/1352839790 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104014440 con 0x7f511015e8b0 2024-05-08T05:51:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.117+0000 7f5115b2b640 1 -- 172.21.15.45:0/1352839790 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51100c95f0 con 0x7f511015e8b0 2024-05-08T05:51:26.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.118+0000 7f50f7fff640 1 -- 172.21.15.45:0/1352839790 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5104012040 con 0x7f511015e8b0 2024-05-08T05:51:26.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.119+0000 7f50f7fff640 1 --2- 172.21.15.45:0/1352839790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50fc0762d0 0x7f50fc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.119+0000 7f50f7fff640 1 -- 172.21.15.45:0/1352839790 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f5104097cf0 con 0x7f511015e8b0 2024-05-08T05:51:26.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.119+0000 7f510f7fe640 1 --2- 172.21.15.45:0/1352839790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50fc0762d0 0x7f50fc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.120+0000 7f5118848740 1 Processor -- start 2024-05-08T05:51:26.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.120+0000 7f5118848740 1 -- start start 2024-05-08T05:51:26.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.120+0000 7f510f7fe640 1 --2- 172.21.15.45:0/1352839790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50fc0762d0 0x7f50fc078790 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5100033000 tx=0x7f510002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:26.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.121+0000 7f5118848740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f5b367f50 0x561f5b36a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.121+0000 7f5118848740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561f5b368330 0x561f5b36aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.121+0000 7f5118848740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b33abb0 0x561f5b33e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.121+0000 7f5118848740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561f5b2a48d0 con 0x561f5b33abb0 2024-05-08T05:51:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.121+0000 7f5118848740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561f5b25dc10 con 0x561f5b367f50 2024-05-08T05:51:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.121+0000 7f5118848740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x561f5b2a0890 con 0x561f5b368330 2024-05-08T05:51:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.121+0000 7f510ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b33abb0 0x561f5b33e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.121+0000 7f510ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b33abb0 0x561f5b33e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51262/0 (socket says 172.21.15.45:51262) 2024-05-08T05:51:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.121+0000 7f510ffff640 1 -- 172.21.15.45:0/2854526332 learned_addr learned my addr 172.21.15.45:0/2854526332 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.121+0000 7f510f7fe640 1 --2- 172.21.15.45:0/2854526332 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f5b367f50 0x561f5b36a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.121+0000 7f510effd640 1 --2- 172.21.15.45:0/2854526332 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561f5b368330 0x561f5b36aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.121+0000 7f510ffff640 1 -- 172.21.15.45:0/2854526332 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561f5b368330 msgr2=0x561f5b36aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.121+0000 7f510ffff640 1 --2- 172.21.15.45:0/2854526332 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561f5b368330 0x561f5b36aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.121+0000 7f510ffff640 1 -- 172.21.15.45:0/2854526332 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f5b367f50 msgr2=0x561f5b36a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:26.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.121+0000 7f510ffff640 1 --2- 172.21.15.45:0/2854526332 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f5b367f50 0x561f5b36a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.121+0000 7f510ffff640 1 -- 172.21.15.45:0/2854526332 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f5b2655d0 con 0x561f5b33abb0 2024-05-08T05:51:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.121+0000 7f510ffff640 1 --2- 172.21.15.45:0/2854526332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b33abb0 0x561f5b33e3c0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f5110073710 tx=0x7f5108007050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa06c0171a42073e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.121+0000 7f50f57fa640 1 -- 172.21.15.45:0/2854526332 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108019070 con 0x561f5b33abb0 2024-05-08T05:51:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.121+0000 7f50f57fa640 1 -- 172.21.15.45:0/2854526332 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5108007980 con 0x561f5b33abb0 2024-05-08T05:51:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.121+0000 7f50f57fa640 1 -- 172.21.15.45:0/2854526332 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51080146a0 con 0x561f5b33abb0 2024-05-08T05:51:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.122+0000 7f5118848740 1 -- 172.21.15.45:0/2854526332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b33abb0 msgr2=0x561f5b33e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.122+0000 7f5118848740 1 --2- 172.21.15.45:0/2854526332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b33abb0 0x561f5b33e3c0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f5110073710 tx=0x7f5108007050 comp rx=0 tx=0).stop 2024-05-08T05:51:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.122+0000 7f5118848740 1 -- 172.21.15.45:0/2854526332 shutdown_connections 2024-05-08T05:51:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.122+0000 7f5118848740 1 --2- 172.21.15.45:0/2854526332 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561f5b368330 0x561f5b36aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.122+0000 7f5118848740 1 --2- 172.21.15.45:0/2854526332 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b33abb0 0x561f5b33e3c0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.122+0000 7f5118848740 1 --2- 172.21.15.45:0/2854526332 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f5b367f50 0x561f5b36a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.122+0000 7f5118848740 1 -- 172.21.15.45:0/2854526332 >> 172.21.15.45:0/2854526332 conn(0x561f5b33ce20 msgr2=0x561f5b385760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.122+0000 7f5118848740 1 -- 172.21.15.45:0/2854526332 shutdown_connections 2024-05-08T05:51:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.122+0000 7f5118848740 1 -- 172.21.15.45:0/2854526332 wait complete. 2024-05-08T05:51:26.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.123+0000 7f5118848740 1 -- start start 2024-05-08T05:51:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.123+0000 7f5118848740 1 Processor -- start 2024-05-08T05:51:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f5118848740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f5b367f50 0x561f5b380600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f5118848740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561f5b368330 0x561f5b33b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f5118848740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b380b40 0x561f5b33bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f5118848740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561f5b33ffd0 con 0x561f5b380b40 2024-05-08T05:51:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f5118848740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561f5b261760 con 0x561f5b367f50 2024-05-08T05:51:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f5118848740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x561f5b3402b0 con 0x561f5b368330 2024-05-08T05:51:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f5118848740 10 client.0 _ll_register_callbacks cb 0x561f5b29acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:51:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f510ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b380b40 0x561f5b33bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f510ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b380b40 0x561f5b33bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51278/0 (socket says 172.21.15.45:51278) 2024-05-08T05:51:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f510ffff640 1 -- 172.21.15.45:0/3158919576 learned_addr learned my addr 172.21.15.45:0/3158919576 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f510effd640 1 --2- 172.21.15.45:0/3158919576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561f5b368330 0x561f5b33b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f510f7fe640 1 --2- 172.21.15.45:0/3158919576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f5b367f50 0x561f5b380600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f510ffff640 1 -- 172.21.15.45:0/3158919576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561f5b368330 msgr2=0x561f5b33b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f510ffff640 1 --2- 172.21.15.45:0/3158919576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561f5b368330 0x561f5b33b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f510ffff640 1 -- 172.21.15.45:0/3158919576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f5b367f50 msgr2=0x561f5b380600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f510ffff640 1 --2- 172.21.15.45:0/3158919576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f5b367f50 0x561f5b380600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f510ffff640 1 -- 172.21.15.45:0/3158919576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f5afb62d0 con 0x561f5b380b40 2024-05-08T05:51:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f510ffff640 1 --2- 172.21.15.45:0/3158919576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b380b40 0x561f5b33bda0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x561f5b3424b0 tx=0x7f5108011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc405910582ffe8f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f50ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:51:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f50ee7fc640 1 -- 172.21.15.45:0/3158919576 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108023030 con 0x561f5b380b40 2024-05-08T05:51:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.124+0000 7f50ee7fc640 1 -- 172.21.15.45:0/3158919576 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f510800f050 con 0x561f5b380b40 2024-05-08T05:51:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.125+0000 7f5118848740 1 -- 172.21.15.45:0/3158919576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5108007980 con 0x561f5b380b40 2024-05-08T05:51:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.125+0000 7f50ee7fc640 1 -- 172.21.15.45:0/3158919576 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7f510802c050 con 0x561f5b380b40 2024-05-08T05:51:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.125+0000 7f5118848740 10 client.8361 fetch_fsmap learned FSMap version 106 2024-05-08T05:51:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.125+0000 7f5118848740 1 -- 172.21.15.45:0/3158919576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) -- 0x561f5b0806b0 con 0x561f5b380b40 2024-05-08T05:51:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.125+0000 7f50ee7fc640 1 -- 172.21.15.45:0/3158919576 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5108007980 con 0x561f5b380b40 2024-05-08T05:51:26.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.125+0000 7f50ee7fc640 1 -- 172.21.15.45:0/3158919576 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 106) ==== 3494+0+0 (secure 0 0 0) 0x7f510802f020 con 0x561f5b380b40 2024-05-08T05:51:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.125+0000 7f5118848740 10 client.8361 fetch_fsmap finished waiting for FSMap version 106 2024-05-08T05:51:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.125+0000 7f5118848740 10 client.8361 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T05:51:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.125+0000 7f5118848740 20 client.8361 populate_metadata read hostname 'smithi045' 2024-05-08T05:51:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.125+0000 7f5118848740 1 --2- 172.21.15.45:0/3158919576 >> [v2:172.21.15.45:6834/453198328,v1:172.21.15.45:6835/453198328] conn(0x561f5b360290 0x561f5b3a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.125+0000 7f5118848740 4 client.8361 mds_command: new command op to 8319 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:51:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.125+0000 7f5118848740 1 -- 172.21.15.45:0/3158919576 --> [v2:172.21.15.45:6834/453198328,v1:172.21.15.45:6835/453198328] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f5b228760 con 0x561f5b360290 2024-05-08T05:51:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.125+0000 7f510f7fe640 1 --2- 172.21.15.45:0/3158919576 >> [v2:172.21.15.45:6834/453198328,v1:172.21.15.45:6835/453198328] conn(0x561f5b360290 0x561f5b3a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.125+0000 7f510f7fe640 1 --2- 172.21.15.45:0/3158919576 >> [v2:172.21.15.45:6834/453198328,v1:172.21.15.45:6835/453198328] conn(0x561f5b360290 0x561f5b3a3ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=285ef20b414e44e3 server_cookie=ada9d6171ba866f9 in_seq=0 out_seq=0 2024-05-08T05:51:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.125+0000 7f50ee7fc640 10 client.8361 ms_handle_connect on v2:172.21.15.45:6834/453198328 2024-05-08T05:51:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.126+0000 7f50ee7fc640 1 -- 172.21.15.45:0/3158919576 <== mds.0 v2:172.21.15.45:6834/453198328 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561f5b228760 con 0x561f5b360290 2024-05-08T05:51:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.126+0000 7f50ee7fc640 10 client.8361 handle_command_reply: tid=0 2024-05-08T05:51:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.126+0000 7f5118848740 1 client.8361 shutdown 2024-05-08T05:51:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.126+0000 7f5118848740 1 -- 172.21.15.45:0/3158919576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b380b40 msgr2=0x561f5b33bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.126+0000 7f5118848740 1 --2- 172.21.15.45:0/3158919576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b380b40 0x561f5b33bda0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x561f5b3424b0 tx=0x7f5108011000 comp rx=0 tx=0).stop 2024-05-08T05:51:26.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.126+0000 7f5118848740 1 -- 172.21.15.45:0/3158919576 shutdown_connections 2024-05-08T05:51:26.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.126+0000 7f5118848740 1 --2- 172.21.15.45:0/3158919576 >> [v2:172.21.15.45:6834/453198328,v1:172.21.15.45:6835/453198328] conn(0x561f5b360290 0x561f5b3a3ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.126+0000 7f5118848740 1 --2- 172.21.15.45:0/3158919576 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561f5b368330 0x561f5b33b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.126+0000 7f5118848740 1 --2- 172.21.15.45:0/3158919576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b380b40 0x561f5b33bda0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.126+0000 7f50ee7fc640 0 client.8361 ms_handle_reset on v2:172.21.15.45:6834/453198328 2024-05-08T05:51:26.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.126+0000 7f5118848740 1 --2- 172.21.15.45:0/3158919576 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f5b367f50 0x561f5b380600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.126+0000 7f5118848740 1 -- 172.21.15.45:0/3158919576 >> 172.21.15.45:0/3158919576 conn(0x561f5b33abb0 msgr2=0x561f5b3672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:26.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.126+0000 7f5118848740 1 -- 172.21.15.45:0/3158919576 shutdown_connections 2024-05-08T05:51:26.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.126+0000 7f5118848740 1 -- 172.21.15.45:0/3158919576 wait complete. 2024-05-08T05:51:26.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.127+0000 7f5118848740 20 client.8361 trim_cache size 0 max 16384 2024-05-08T05:51:26.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.145+0000 7f5118848740 1 Processor -- start 2024-05-08T05:51:26.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.145+0000 7f5118848740 1 -- start start 2024-05-08T05:51:26.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.146+0000 7f5118848740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b367f50 0x561f5b360860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.146+0000 7f5118848740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f5b368330 0x561f5b3888a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.146+0000 7f5118848740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561f5b380b40 0x561f5b388de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.146+0000 7f5118848740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f51102015d0 con 0x561f5b367f50 2024-05-08T05:51:26.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.146+0000 7f5118848740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f51100cfb50 con 0x561f5b368330 2024-05-08T05:51:26.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.146+0000 7f5118848740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f5110201010 con 0x561f5b380b40 2024-05-08T05:51:26.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.146+0000 7f510f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b367f50 0x561f5b360860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.146+0000 7f510f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b367f50 0x561f5b360860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51292/0 (socket says 172.21.15.45:51292) 2024-05-08T05:51:26.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.146+0000 7f510f7fe640 1 -- 172.21.15.45:0/1551429153 learned_addr learned my addr 172.21.15.45:0/1551429153 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:26.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.146+0000 7f510f7fe640 1 -- 172.21.15.45:0/1551429153 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561f5b380b40 msgr2=0x561f5b388de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:26.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.146+0000 7f510f7fe640 1 --2- 172.21.15.45:0/1551429153 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561f5b380b40 0x561f5b388de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.146+0000 7f510f7fe640 1 -- 172.21.15.45:0/1551429153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f5b368330 msgr2=0x561f5b3888a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:26.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.146+0000 7f510f7fe640 1 --2- 172.21.15.45:0/1551429153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f5b368330 0x561f5b3888a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.147+0000 7f510f7fe640 1 -- 172.21.15.45:0/1551429153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f5b35dc20 con 0x561f5b367f50 2024-05-08T05:51:26.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.147+0000 7f510f7fe640 1 --2- 172.21.15.45:0/1551429153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b367f50 0x561f5b360860 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f5100009be0 tx=0x7f510002f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c2fd9f5a28a9ee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:26.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.147+0000 7f50f4ff9640 1 -- 172.21.15.45:0/1551429153 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100036040 con 0x561f5b367f50 2024-05-08T05:51:26.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.147+0000 7f50f4ff9640 1 -- 172.21.15.45:0/1551429153 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5100004340 con 0x561f5b367f50 2024-05-08T05:51:26.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.147+0000 7f50f4ff9640 1 -- 172.21.15.45:0/1551429153 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100003810 con 0x561f5b367f50 2024-05-08T05:51:26.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.147+0000 7f5118848740 1 -- 172.21.15.45:0/1551429153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b367f50 msgr2=0x561f5b360860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.147+0000 7f5118848740 1 --2- 172.21.15.45:0/1551429153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b367f50 0x561f5b360860 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f5100009be0 tx=0x7f510002f000 comp rx=0 tx=0).stop 2024-05-08T05:51:26.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.148+0000 7f5118848740 1 -- 172.21.15.45:0/1551429153 shutdown_connections 2024-05-08T05:51:26.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.148+0000 7f5118848740 1 --2- 172.21.15.45:0/1551429153 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561f5b380b40 0x561f5b388de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.148+0000 7f5118848740 1 --2- 172.21.15.45:0/1551429153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f5b368330 0x561f5b3888a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.148+0000 7f5118848740 1 --2- 172.21.15.45:0/1551429153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b367f50 0x561f5b360860 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.148+0000 7f5118848740 1 -- 172.21.15.45:0/1551429153 >> 172.21.15.45:0/1551429153 conn(0x561f5b33abb0 msgr2=0x561f5b36a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:26.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.148+0000 7f5118848740 1 -- 172.21.15.45:0/1551429153 shutdown_connections 2024-05-08T05:51:26.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.148+0000 7f5118848740 1 -- 172.21.15.45:0/1551429153 wait complete. 2024-05-08T05:51:26.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.148+0000 7f5118848740 1 -- start start 2024-05-08T05:51:26.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.148+0000 7f5118848740 1 Processor -- start 2024-05-08T05:51:26.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.149+0000 7f5118848740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f5b367f50 0x561f5b402ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.149+0000 7f5118848740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561f5b368330 0x561f5b340be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.149+0000 7f5118848740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b380b40 0x561f5b35c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.149+0000 7f5118848740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561f5b381360 con 0x561f5b380b40 2024-05-08T05:51:26.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.149+0000 7f5118848740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x561f5b3621f0 con 0x561f5b367f50 2024-05-08T05:51:26.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.149+0000 7f5118848740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x561f5b361d00 con 0x561f5b368330 2024-05-08T05:51:26.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.149+0000 7f5118848740 10 client.0 _ll_register_callbacks cb 0x7f5108007b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:51:26.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.149+0000 7f510ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b380b40 0x561f5b35c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.149+0000 7f510ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b380b40 0x561f5b35c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51294/0 (socket says 172.21.15.45:51294) 2024-05-08T05:51:26.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.149+0000 7f510ffff640 1 -- 172.21.15.45:0/1082593444 learned_addr learned my addr 172.21.15.45:0/1082593444 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:26.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.149+0000 7f510ffff640 1 -- 172.21.15.45:0/1082593444 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561f5b368330 msgr2=0x561f5b340be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.149+0000 7f510ffff640 1 --2- 172.21.15.45:0/1082593444 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561f5b368330 0x561f5b340be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.149+0000 7f510f7fe640 1 --2- 172.21.15.45:0/1082593444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f5b367f50 0x561f5b402ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.149+0000 7f510ffff640 1 -- 172.21.15.45:0/1082593444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f5b367f50 msgr2=0x561f5b402ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.149+0000 7f510ffff640 1 --2- 172.21.15.45:0/1082593444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f5b367f50 0x561f5b402ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.149+0000 7f510ffff640 1 -- 172.21.15.45:0/1082593444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f5b360f70 con 0x561f5b380b40 2024-05-08T05:51:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.150+0000 7f510ffff640 1 --2- 172.21.15.45:0/1082593444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b380b40 0x561f5b35c600 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x561f5b33ff00 tx=0x7f5108000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1cb082189e2cbd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.150+0000 7f50ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:51:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.150+0000 7f50ee7fc640 1 -- 172.21.15.45:0/1082593444 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108023030 con 0x561f5b380b40 2024-05-08T05:51:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.150+0000 7f50ee7fc640 1 -- 172.21.15.45:0/1082593444 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f510800f070 con 0x561f5b380b40 2024-05-08T05:51:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.150+0000 7f5118848740 1 -- 172.21.15.45:0/1082593444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5100004340 con 0x561f5b380b40 2024-05-08T05:51:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.150+0000 7f50ee7fc640 1 -- 172.21.15.45:0/1082593444 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7f5108038050 con 0x561f5b380b40 2024-05-08T05:51:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.150+0000 7f5118848740 10 client.8367 fetch_fsmap learned FSMap version 106 2024-05-08T05:51:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.150+0000 7f5118848740 1 -- 172.21.15.45:0/1082593444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) -- 0x561f5b36dab0 con 0x561f5b380b40 2024-05-08T05:51:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.150+0000 7f50ee7fc640 1 -- 172.21.15.45:0/1082593444 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5100004340 con 0x561f5b380b40 2024-05-08T05:51:26.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.150+0000 7f50ee7fc640 1 -- 172.21.15.45:0/1082593444 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 106) ==== 3494+0+0 (secure 0 0 0) 0x7f510803b020 con 0x561f5b380b40 2024-05-08T05:51:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.150+0000 7f5118848740 10 client.8367 fetch_fsmap finished waiting for FSMap version 106 2024-05-08T05:51:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.150+0000 7f5118848740 10 client.8367 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T05:51:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.150+0000 7f5118848740 20 client.8367 populate_metadata read hostname 'smithi045' 2024-05-08T05:51:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.150+0000 7f5118848740 1 --2- 172.21.15.45:0/1082593444 >> [v2:172.21.15.45:6834/453198328,v1:172.21.15.45:6835/453198328] conn(0x561f5b33b640 0x561f5b3850e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.150+0000 7f5118848740 4 client.8367 mds_command: new command op to 8319 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:51:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.150+0000 7f5118848740 1 -- 172.21.15.45:0/1082593444 --> [v2:172.21.15.45:6834/453198328,v1:172.21.15.45:6835/453198328] -- command(tid 0: {"prefix": "status"}) -- 0x561f5b2b24e0 con 0x561f5b33b640 2024-05-08T05:51:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.150+0000 7f510f7fe640 1 --2- 172.21.15.45:0/1082593444 >> [v2:172.21.15.45:6834/453198328,v1:172.21.15.45:6835/453198328] conn(0x561f5b33b640 0x561f5b3850e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.151+0000 7f510f7fe640 1 --2- 172.21.15.45:0/1082593444 >> [v2:172.21.15.45:6834/453198328,v1:172.21.15.45:6835/453198328] conn(0x561f5b33b640 0x561f5b3850e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48d931d53349b7e4 server_cookie=1bfcb47167629bab in_seq=0 out_seq=0 2024-05-08T05:51:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.151+0000 7f50ee7fc640 10 client.8367 ms_handle_connect on v2:172.21.15.45:6834/453198328 2024-05-08T05:51:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.151+0000 7f50ee7fc640 1 -- 172.21.15.45:0/1082593444 <== mds.0 v2:172.21.15.45:6834/453198328 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x561f5b2b24e0 con 0x561f5b33b640 2024-05-08T05:51:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.151+0000 7f50ee7fc640 10 client.8367 handle_command_reply: tid=0 2024-05-08T05:51:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.151+0000 7f5118848740 1 client.8367 shutdown 2024-05-08T05:51:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.151+0000 7f5118848740 1 -- 172.21.15.45:0/1082593444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b380b40 msgr2=0x561f5b35c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.151+0000 7f5118848740 1 --2- 172.21.15.45:0/1082593444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b380b40 0x561f5b35c600 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x561f5b33ff00 tx=0x7f5108000f40 comp rx=0 tx=0).stop 2024-05-08T05:51:26.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.151+0000 7f5118848740 1 -- 172.21.15.45:0/1082593444 shutdown_connections 2024-05-08T05:51:26.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.151+0000 7f5118848740 1 --2- 172.21.15.45:0/1082593444 >> [v2:172.21.15.45:6834/453198328,v1:172.21.15.45:6835/453198328] conn(0x561f5b33b640 0x561f5b3850e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.151+0000 7f5118848740 1 --2- 172.21.15.45:0/1082593444 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x561f5b368330 0x561f5b340be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.151+0000 7f5118848740 1 --2- 172.21.15.45:0/1082593444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561f5b380b40 0x561f5b35c600 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.151+0000 7f5118848740 1 --2- 172.21.15.45:0/1082593444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x561f5b367f50 0x561f5b402ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.151+0000 7f5118848740 1 -- 172.21.15.45:0/1082593444 >> 172.21.15.45:0/1082593444 conn(0x561f5b33abb0 msgr2=0x561f5b360860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:26.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.151+0000 7f50ee7fc640 0 client.8367 ms_handle_reset on v2:172.21.15.45:6834/453198328 2024-05-08T05:51:26.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.151+0000 7f5118848740 1 -- 172.21.15.45:0/1082593444 shutdown_connections 2024-05-08T05:51:26.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.151+0000 7f5118848740 1 -- 172.21.15.45:0/1082593444 wait complete. 2024-05-08T05:51:26.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.151+0000 7f5118848740 20 client.8367 trim_cache size 0 max 16384 2024-05-08T05:51:26.157 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:51:26.158 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "e9e1170a-fd27-4625-97a9-5d5f104d13a2", 2024-05-08T05:51:26.158 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-05-08T05:51:26.158 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 8319, 2024-05-08T05:51:26.158 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-05-08T05:51:26.158 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-05-08T05:51:26.158 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-05-08T05:51:26.158 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 4.4904486610000003, 2024-05-08T05:51:26.158 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 106, 2024-05-08T05:51:26.158 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 120, 2024-05-08T05:51:26.158 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 120, 2024-05-08T05:51:26.158 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 5.4556422830000004 2024-05-08T05:51:26.158 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:51:26.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.152+0000 7f5115b2b640 1 -- 172.21.15.45:0/1352839790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50fc0762d0 msgr2=0x7f50fc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.152+0000 7f5115b2b640 1 --2- 172.21.15.45:0/1352839790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50fc0762d0 0x7f50fc078790 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5100033000 tx=0x7f510002e000 comp rx=0 tx=0).stop 2024-05-08T05:51:26.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.152+0000 7f5115b2b640 1 -- 172.21.15.45:0/1352839790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f511015e8b0 msgr2=0x7f51100cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.152+0000 7f5115b2b640 1 --2- 172.21.15.45:0/1352839790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f511015e8b0 0x7f51100cd730 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f51040119a0 tx=0x7f5104011e60 comp rx=0 tx=0).stop 2024-05-08T05:51:26.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.152+0000 7f5115b2b640 1 -- 172.21.15.45:0/1352839790 shutdown_connections 2024-05-08T05:51:26.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.152+0000 7f5115b2b640 1 --2- 172.21.15.45:0/1352839790 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50fc0762d0 0x7f50fc078790 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.152+0000 7f5115b2b640 1 --2- 172.21.15.45:0/1352839790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f511015dce0 0x7f511015e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.152+0000 7f5115b2b640 1 --2- 172.21.15.45:0/1352839790 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f511015e8b0 0x7f51100cd730 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.152+0000 7f5115b2b640 1 --2- 172.21.15.45:0/1352839790 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50f8003a00 0x7f51100c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.152+0000 7f5115b2b640 1 -- 172.21.15.45:0/1352839790 >> 172.21.15.45:0/1352839790 conn(0x7f51100d2e90 msgr2=0x7f5110160db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:26.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.152+0000 7f5115b2b640 1 -- 172.21.15.45:0/1352839790 shutdown_connections 2024-05-08T05:51:26.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.152+0000 7f5115b2b640 1 -- 172.21.15.45:0/1352839790 wait complete. 2024-05-08T05:51:26.163 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T05:51:26.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.266+0000 7f9dd4fa4640 1 Processor -- start 2024-05-08T05:51:26.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.266+0000 7f9dd4fa4640 1 -- start start 2024-05-08T05:51:26.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.267+0000 7f9dd4fa4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f9dd015f380 0x7f9dd00d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.267+0000 7f9dd4fa4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9dd00d0cf0 0x7f9dd00d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.267+0000 7f9dd4fa4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dd00d1760 con 0x7f9dd00d3b00 2024-05-08T05:51:26.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.267+0000 7f9dd4fa4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd00d30f0 con 0x7f9dd015f380 2024-05-08T05:51:26.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.267+0000 7f9dd4fa4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd00d1150 con 0x7f9dd00d0cf0 2024-05-08T05:51:26.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.267+0000 7f9dced9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9dd00d3b00 0x7f9dd0162e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49604/0 (socket says 172.21.15.45:49604) 2024-05-08T05:51:26.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.267+0000 7f9dced9d640 1 -- 172.21.15.45:0/3088109603 learned_addr learned my addr 172.21.15.45:0/3088109603 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:26.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.267+0000 7f9dc7fff640 1 --2- 172.21.15.45:0/3088109603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9dd00d0cf0 0x7f9dd00d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.267+0000 7f9dce59c640 1 --2- 172.21.15.45:0/3088109603 >> v2:172.21.15.63:3300/0 conn(0x7f9dd015f380 0x7f9dd00d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.267+0000 7f9dcdd9b640 1 -- 172.21.15.45:0/3088109603 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1763815567 0 0) 0x7f9dd00d1760 con 0x7f9dd00d3b00 2024-05-08T05:51:26.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.267+0000 7f9dcdd9b640 1 -- 172.21.15.45:0/3088109603 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9dbc003520 con 0x7f9dd00d3b00 2024-05-08T05:51:26.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.267+0000 7f9dcdd9b640 1 -- 172.21.15.45:0/3088109603 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2286502517 0 0) 0x7f9dbc003520 con 0x7f9dd00d3b00 2024-05-08T05:51:26.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.267+0000 7f9dcdd9b640 1 -- 172.21.15.45:0/3088109603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9dd00d0cf0 msgr2=0x7f9dd00d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:26.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.267+0000 7f9dcdd9b640 1 --2- 172.21.15.45:0/3088109603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9dd00d0cf0 0x7f9dd00d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.267+0000 7f9dcdd9b640 1 -- 172.21.15.45:0/3088109603 >> v2:172.21.15.63:3300/0 conn(0x7f9dd015f380 msgr2=0x7f9dd00d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:26.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.267+0000 7f9dcdd9b640 1 --2- 172.21.15.45:0/3088109603 >> v2:172.21.15.63:3300/0 conn(0x7f9dd015f380 0x7f9dd00d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.267+0000 7f9dcdd9b640 1 -- 172.21.15.45:0/3088109603 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd00d1a40 con 0x7f9dd00d3b00 2024-05-08T05:51:26.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.267+0000 7f9dcdd9b640 1 -- 172.21.15.45:0/3088109603 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f9dc8001a70 con 0x7f9dd00d3b00 2024-05-08T05:51:26.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.268+0000 7f9dcdd9b640 1 -- 172.21.15.45:0/3088109603 >> v1:172.21.15.45:6789/0 conn(0x7f9dd00d3b00 legacy=0x7f9dd0162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.268+0000 7f9dcdd9b640 1 --2- 172.21.15.45:0/3088109603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dbc003a00 0x7f9dbc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.268+0000 7f9dcdd9b640 1 --2- 172.21.15.45:0/3088109603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9dd00d0cf0 0x7f9dbc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.268+0000 7f9dcdd9b640 1 --2- 172.21.15.45:0/3088109603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dbc004d40 0x7f9dbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.268+0000 7f9dcdd9b640 1 -- 172.21.15.45:0/3088109603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd00d30f0 con 0x7f9dbc003a00 2024-05-08T05:51:26.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.268+0000 7f9dcdd9b640 1 -- 172.21.15.45:0/3088109603 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc00b9c0 con 0x7f9dbc004d40 2024-05-08T05:51:26.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.268+0000 7f9dcdd9b640 1 -- 172.21.15.45:0/3088109603 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9dbc00bb70 con 0x7f9dd00d0cf0 2024-05-08T05:51:26.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.268+0000 7f9dce59c640 1 --2- 172.21.15.45:0/3088109603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dbc003a00 0x7f9dbc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.268+0000 7f9dc7fff640 1 --2- 172.21.15.45:0/3088109603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9dd00d0cf0 0x7f9dbc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.268+0000 7f9dced9d640 1 --2- 172.21.15.45:0/3088109603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dbc004d40 0x7f9dbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.268+0000 7f9dce59c640 1 -- 172.21.15.45:0/3088109603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9dd00d0cf0 msgr2=0x7f9dbc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:26.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.268+0000 7f9dce59c640 1 --2- 172.21.15.45:0/3088109603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9dd00d0cf0 0x7f9dbc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.268+0000 7f9dce59c640 1 -- 172.21.15.45:0/3088109603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dbc004d40 msgr2=0x7f9dbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:26.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.268+0000 7f9dce59c640 1 --2- 172.21.15.45:0/3088109603 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dbc004d40 0x7f9dbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.268+0000 7f9dce59c640 1 -- 172.21.15.45:0/3088109603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dbc003520 con 0x7f9dbc003a00 2024-05-08T05:51:26.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.268+0000 7f9dc7fff640 1 --2- 172.21.15.45:0/3088109603 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9dd00d0cf0 0x7f9dbc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:26.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.269+0000 7f9dce59c640 1 --2- 172.21.15.45:0/3088109603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dbc003a00 0x7f9dbc003e20 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0009ac0 tx=0x7f9dbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee93455dd314ee07 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:26.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.269+0000 7f9dcdd9b640 1 -- 172.21.15.45:0/3088109603 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0036070 con 0x7f9dbc003a00 2024-05-08T05:51:26.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.269+0000 7f9dcdd9b640 1 -- 172.21.15.45:0/3088109603 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc0031400 con 0x7f9dbc003a00 2024-05-08T05:51:26.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.269+0000 7f9dcdd9b640 1 -- 172.21.15.45:0/3088109603 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0009690 con 0x7f9dbc003a00 2024-05-08T05:51:26.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.269+0000 7f9dd4fa4640 1 -- 172.21.15.45:0/3088109603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dbc003a00 msgr2=0x7f9dbc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.269+0000 7f9dd4fa4640 1 --2- 172.21.15.45:0/3088109603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dbc003a00 0x7f9dbc003e20 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0009ac0 tx=0x7f9dbc005830 comp rx=0 tx=0).stop 2024-05-08T05:51:26.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.269+0000 7f9dced9d640 1 -- 172.21.15.45:0/3088109603 reap_dead start 2024-05-08T05:51:26.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.269+0000 7f9dd4fa4640 1 -- 172.21.15.45:0/3088109603 shutdown_connections 2024-05-08T05:51:26.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.269+0000 7f9dd4fa4640 1 -- 172.21.15.45:0/3088109603 >> 172.21.15.45:0/3088109603 conn(0x7f9dd00d4fe0 msgr2=0x7f9dd00d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:26.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.269+0000 7f9dd4fa4640 1 -- 172.21.15.45:0/3088109603 shutdown_connections 2024-05-08T05:51:26.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.269+0000 7f9dd4fa4640 1 -- 172.21.15.45:0/3088109603 wait complete. 2024-05-08T05:51:26.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.271+0000 7f9dd4fa4640 1 Processor -- start 2024-05-08T05:51:26.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.271+0000 7f9dd4fa4640 1 -- start start 2024-05-08T05:51:26.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.271+0000 7f9dd4fa4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dbc003a00 0x7f9dd015da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.271+0000 7f9dd4fa4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dd00b5a10 0x7f9dd00ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.271+0000 7f9dd4fa4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9dd00b5e90 0x7f9dd00ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.271+0000 7f9dd4fa4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd00b8e40 con 0x7f9dd00b5a10 2024-05-08T05:51:26.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.271+0000 7f9dd4fa4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd00d2370 con 0x7f9dbc003a00 2024-05-08T05:51:26.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.271+0000 7f9dd4fa4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd00d2000 con 0x7f9dd00b5e90 2024-05-08T05:51:26.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.272+0000 7f9dc7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dd00b5a10 0x7f9dd00ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.272+0000 7f9dc7fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dd00b5a10 0x7f9dd00ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51308/0 (socket says 172.21.15.45:51308) 2024-05-08T05:51:26.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.272+0000 7f9dc7fff640 1 -- 172.21.15.45:0/2131029483 learned_addr learned my addr 172.21.15.45:0/2131029483 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:26.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.272+0000 7f9dce59c640 1 --2- 172.21.15.45:0/2131029483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dbc003a00 0x7f9dd015da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.272+0000 7f9dced9d640 1 --2- 172.21.15.45:0/2131029483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9dd00b5e90 0x7f9dd00ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.272+0000 7f9dc7fff640 1 -- 172.21.15.45:0/2131029483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9dd00b5e90 msgr2=0x7f9dd00ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.272+0000 7f9dc7fff640 1 --2- 172.21.15.45:0/2131029483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9dd00b5e90 0x7f9dd00ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.272+0000 7f9dc7fff640 1 -- 172.21.15.45:0/2131029483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dbc003a00 msgr2=0x7f9dd015da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.272+0000 7f9dc7fff640 1 --2- 172.21.15.45:0/2131029483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dbc003a00 0x7f9dd015da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.272+0000 7f9dc7fff640 1 -- 172.21.15.45:0/2131029483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd00b4410 con 0x7f9dd00b5a10 2024-05-08T05:51:26.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.272+0000 7f9dce59c640 1 --2- 172.21.15.45:0/2131029483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dbc003a00 0x7f9dd015da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:26.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.272+0000 7f9dc7fff640 1 --2- 172.21.15.45:0/2131029483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dd00b5a10 0x7f9dd00ce280 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f9db80116d0 tx=0x7f9db8011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:26.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.272+0000 7f9dc6ffd640 1 -- 172.21.15.45:0/2131029483 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db800f9c0 con 0x7f9dd00b5a10 2024-05-08T05:51:26.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.272+0000 7f9dc6ffd640 1 -- 172.21.15.45:0/2131029483 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db800fb60 con 0x7f9dd00b5a10 2024-05-08T05:51:26.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.272+0000 7f9dd4fa4640 1 -- 172.21.15.45:0/2131029483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd00b46f0 con 0x7f9dd00b5a10 2024-05-08T05:51:26.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.272+0000 7f9dc6ffd640 1 -- 172.21.15.45:0/2131029483 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db8015460 con 0x7f9dd00b5a10 2024-05-08T05:51:26.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.272+0000 7f9dd4fa4640 1 -- 172.21.15.45:0/2131029483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd00b4cb0 con 0x7f9dd00b5a10 2024-05-08T05:51:26.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.273+0000 7f9dd4fa4640 1 -- 172.21.15.45:0/2131029483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd00693f0 con 0x7f9dd00b5a10 2024-05-08T05:51:26.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.274+0000 7f9dc6ffd640 1 -- 172.21.15.45:0/2131029483 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9db8015600 con 0x7f9dd00b5a10 2024-05-08T05:51:26.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.274+0000 7f9dc6ffd640 1 --2- 172.21.15.45:0/2131029483 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9db00762d0 0x7f9db0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.275+0000 7f9dc6ffd640 1 -- 172.21.15.45:0/2131029483 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f9db8098190 con 0x7f9dd00b5a10 2024-05-08T05:51:26.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.275+0000 7f9dce59c640 1 --2- 172.21.15.45:0/2131029483 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9db00762d0 0x7f9db0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.275+0000 7f9dce59c640 1 --2- 172.21.15.45:0/2131029483 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9db00762d0 0x7f9db0078790 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0033000 tx=0x7f9dc0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:26.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.279+0000 7f9dc6ffd640 1 -- 172.21.15.45:0/2131029483 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9db8019030 con 0x7f9dd00b5a10 2024-05-08T05:51:26.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.509+0000 7f9dd4fa4640 1 -- 172.21.15.45:0/2131029483 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9dd00cf5c0 con 0x7f9dd00b5a10 2024-05-08T05:51:26.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.509+0000 7f9dc6ffd640 1 -- 172.21.15.45:0/2131029483 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v106) ==== 53+0+73 (secure 0 0 0) 0x7f9db80619a0 con 0x7f9dd00b5a10 2024-05-08T05:51:26.509 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T05:51:26.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.511+0000 7f9dd4fa4640 1 -- 172.21.15.45:0/2131029483 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9db00762d0 msgr2=0x7f9db0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.511+0000 7f9dd4fa4640 1 --2- 172.21.15.45:0/2131029483 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9db00762d0 0x7f9db0078790 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0033000 tx=0x7f9dc0008030 comp rx=0 tx=0).stop 2024-05-08T05:51:26.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.511+0000 7f9dd4fa4640 1 -- 172.21.15.45:0/2131029483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dd00b5a10 msgr2=0x7f9dd00ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.511+0000 7f9dd4fa4640 1 --2- 172.21.15.45:0/2131029483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dd00b5a10 0x7f9dd00ce280 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f9db80116d0 tx=0x7f9db8011b90 comp rx=0 tx=0).stop 2024-05-08T05:51:26.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.511+0000 7f9dd4fa4640 1 -- 172.21.15.45:0/2131029483 shutdown_connections 2024-05-08T05:51:26.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.511+0000 7f9dd4fa4640 1 --2- 172.21.15.45:0/2131029483 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9db00762d0 0x7f9db0078790 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.511+0000 7f9dd4fa4640 1 --2- 172.21.15.45:0/2131029483 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9dd00b5e90 0x7f9dd00ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.511+0000 7f9dd4fa4640 1 --2- 172.21.15.45:0/2131029483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9dd00b5a10 0x7f9dd00ce280 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.511+0000 7f9dd4fa4640 1 --2- 172.21.15.45:0/2131029483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9dbc003a00 0x7f9dd015da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.511+0000 7f9dd4fa4640 1 -- 172.21.15.45:0/2131029483 >> 172.21.15.45:0/2131029483 conn(0x7f9dd00d4fe0 msgr2=0x7f9dd00d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:26.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.511+0000 7f9dd4fa4640 1 -- 172.21.15.45:0/2131029483 shutdown_connections 2024-05-08T05:51:26.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26.511+0000 7f9dd4fa4640 1 -- 172.21.15.45:0/2131029483 wait complete. 2024-05-08T05:51:26.522 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T05:51:26.522 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:51:26.523 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-05-08T05:51:26.523 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T05:51:26.523 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:51:26.523 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T05:51:26.523 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T05:51:26.523 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:51:26.523 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:51:26.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:26.524 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-05-08T05:51:26.541 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:51:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:51:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:51:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-05-08T05:51:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:26.541 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:51:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:51:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-05-08T05:51:26.542 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:51:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:51:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-05-08T05:51:26.542 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:51:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-05-08T05:51:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-05-08T05:51:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:51:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42138sec preferred_lft 42138sec 2024-05-08T05:51:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-05-08T05:51:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:26.542 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:51:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-05-08T05:51:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-05-08T05:51:26.543 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:51:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T05:51:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T05:51:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c80a:aff:fe94:c77/64 scope link 2024-05-08T05:51:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:26.543 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:51:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 82:f3:eb:6a:57:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:51:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::80f3:ebff:fe6a:57ba/64 scope link 2024-05-08T05:51:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:26.543 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:51:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T05:51:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::14a2:64ff:feda:3a9c/64 scope link 2024-05-08T05:51:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:26.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:26.544 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:51:26.606 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:51:26.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:26.608 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-05-08T05:51:26.660 INFO:tasks.ceph.mds.c.smithi045.stderr:2024-05-08T05:51:26.660+0000 7f4d07da5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:51:26.664 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-05-08T05:51:26.665 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:51:26.665 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:51:26.666 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:51:26.666 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:51:26.667 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:51:26.667 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:51:26.667 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:51:26.667 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:51:26.667 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:51:26.667 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:51:26.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:51:26.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:26.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:26.795 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:51:26.811 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T05:51:26.812 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T05:51:26.812 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:51:26.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:26.812 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-05-08T05:51:26.878 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:51:26.878 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T05:51:26.934 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:51:26.935 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:51:26.935 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:51:26.990 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T05:51:26.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:26.991 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:51:27.075 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:51:27.076 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:51:27.088 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:51:27.088 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:51:27.088 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-08T05:51:27.088 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@e9e1170a-fd27-4625-97a9-5d5f104d13a2.cephfs=/ 2024-05-08T05:51:27.088 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.63:3300/172.21.15.63:3301" will pass to kernel 2024-05-08T05:51:27.088 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T05:51:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:51:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:51:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:51:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:51:27.090 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:51:27.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:27.090 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:51:27.116 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:51:27.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:51:27.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:27.214 INFO:teuthology.orchestra.run.smithi045.stdout:client7384 2024-05-08T05:51:27.215 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:51:27.215 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:51:27.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:27.231 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-08T05:51:27.292 INFO:teuthology.orchestra.run.smithi045.stdout:./ 2024-05-08T05:51:27.292 INFO:teuthology.orchestra.run.smithi045.stdout:./parent 2024-05-08T05:51:27.292 INFO:teuthology.orchestra.run.smithi045.stdout:./parent/flushed 2024-05-08T05:51:27.292 INFO:teuthology.orchestra.run.smithi045.stdout:./parent/flushed/charlie 2024-05-08T05:51:27.292 INFO:teuthology.orchestra.run.smithi045.stdout:./parent/flushed/alpha 2024-05-08T05:51:27.292 INFO:teuthology.orchestra.run.smithi045.stdout:./parent/unflushed 2024-05-08T05:51:27.292 INFO:teuthology.orchestra.run.smithi045.stdout:./parent/unflushed/jfile 2024-05-08T05:51:27.293 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:27.293 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:27.293 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:27.293 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:27.293 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-08T05:51:27.293 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:27.400 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:51:27.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:27.409 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:27.409 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:27.409 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:27.409 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:27.409 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-08T05:51:27.409 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:27.517 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627776 2024-05-08T05:51:27.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:27.526 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:27.526 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:27.526 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:27.526 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:27.526 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-08T05:51:27.526 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:27.633 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627777 2024-05-08T05:51:27.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:27.643 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:27.643 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:27.643 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:27.643 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:27.643 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-08T05:51:27.643 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:27.750 INFO:teuthology.orchestra.run.smithi045.stdout:1099511628280 2024-05-08T05:51:27.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:27.759 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:27.759 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:27.759 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:27.759 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:27.760 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-08T05:51:27.760 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:27.867 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627778 2024-05-08T05:51:27.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:27.876 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:27.876 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:27.876 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:27.876 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:27.876 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-08T05:51:27.876 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:27.983 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627779 2024-05-08T05:51:27.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:27.993 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:27.993 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:27.993 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:27.993 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:27.993 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-08T05:51:27.993 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:28.102 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627780 2024-05-08T05:51:28.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:28.111 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:51:28.111 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:51:28.168 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:51:28.168 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:51:28.224 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:51:28.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.368+0000 7f99ae86d640 1 Processor -- start 2024-05-08T05:51:28.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.368+0000 7f99ae86d640 1 -- start start 2024-05-08T05:51:28.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.369+0000 7f99ae86d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f99a815f0c0 0x7f99a80b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:28.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.369+0000 7f99ae86d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f99a81638e0 0x7f99a80b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:28.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.369+0000 7f99ae86d640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99a80b1260 con 0x7f99a806c3a0 2024-05-08T05:51:28.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.369+0000 7f99ae86d640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f99a8163450 con 0x7f99a81638e0 2024-05-08T05:51:28.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.369+0000 7f99ae86d640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f99a81604b0 con 0x7f99a815f0c0 2024-05-08T05:51:28.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.369+0000 7f99a77fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f99a81638e0 0x7f99a80b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:28.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.369+0000 7f99ace0a640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f99a806c3a0 0x7f99a8073170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49616/0 (socket says 172.21.15.45:49616) 2024-05-08T05:51:28.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.369+0000 7f99ace0a640 1 -- 172.21.15.45:0/3721595171 learned_addr learned my addr 172.21.15.45:0/3721595171 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.370+0000 7f99a7fff640 1 --2- 172.21.15.45:0/3721595171 >> v2:172.21.15.63:3300/0 conn(0x7f99a815f0c0 0x7f99a80b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.370+0000 7f99a77fe640 1 -- 172.21.15.45:0/3721595171 >> v2:172.21.15.63:3300/0 conn(0x7f99a815f0c0 msgr2=0x7f99a80b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.370+0000 7f99a77fe640 1 --2- 172.21.15.45:0/3721595171 >> v2:172.21.15.63:3300/0 conn(0x7f99a815f0c0 0x7f99a80b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.370+0000 7f99a77fe640 1 -- 172.21.15.45:0/3721595171 >> v1:172.21.15.45:6789/0 conn(0x7f99a806c3a0 legacy=0x7f99a8073170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.370+0000 7f99a77fe640 1 -- 172.21.15.45:0/3721595171 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a80b14e0 con 0x7f99a81638e0 2024-05-08T05:51:28.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.370+0000 7f99a6ffd640 1 -- 172.21.15.45:0/3721595171 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3019426730 0 0) 0x7f99a0001490 con 0x7f99a806c3a0 2024-05-08T05:51:28.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.370+0000 7f99a77fe640 1 --2- 172.21.15.45:0/3721595171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a81638e0 0x7f99a80b0d20 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f99a8071200 tx=0x7f999c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07f2a0377445faf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:28.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.371+0000 7f99a6ffd640 1 -- 172.21.15.45:0/3721595171 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c037070 con 0x7f99a81638e0 2024-05-08T05:51:28.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.371+0000 7f99a6ffd640 1 -- 172.21.15.45:0/3721595171 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f999c032440 con 0x7f99a81638e0 2024-05-08T05:51:28.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.371+0000 7f99a6ffd640 1 -- 172.21.15.45:0/3721595171 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c032810 con 0x7f99a81638e0 2024-05-08T05:51:28.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.371+0000 7f99ae86d640 1 -- 172.21.15.45:0/3721595171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a81638e0 msgr2=0x7f99a80b0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:28.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.371+0000 7f99ae86d640 1 --2- 172.21.15.45:0/3721595171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a81638e0 0x7f99a80b0d20 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f99a8071200 tx=0x7f999c02a1a0 comp rx=0 tx=0).stop 2024-05-08T05:51:28.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.371+0000 7f99ae86d640 1 -- 172.21.15.45:0/3721595171 shutdown_connections 2024-05-08T05:51:28.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.371+0000 7f99ae86d640 1 --2- 172.21.15.45:0/3721595171 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a81638e0 0x7f99a80b0d20 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:28.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.371+0000 7f99ae86d640 1 --2- 172.21.15.45:0/3721595171 >> v2:172.21.15.63:3300/0 conn(0x7f99a815f0c0 0x7f99a80b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:28.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.371+0000 7f99ae86d640 1 -- 172.21.15.45:0/3721595171 >> 172.21.15.45:0/3721595171 conn(0x7f99a80b3b10 msgr2=0x7f99a80b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:28.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.371+0000 7f99ae86d640 1 -- 172.21.15.45:0/3721595171 shutdown_connections 2024-05-08T05:51:28.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.371+0000 7f99ae86d640 1 -- 172.21.15.45:0/3721595171 wait complete. 2024-05-08T05:51:28.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.372+0000 7f99ae86d640 1 Processor -- start 2024-05-08T05:51:28.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.373+0000 7f99ae86d640 1 -- start start 2024-05-08T05:51:28.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.373+0000 7f99ae86d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a806c3a0 0x7f99a8155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.373+0000 7f99ae86d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a815f0c0 0x7f99a80d2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.373+0000 7f99ae86d640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99a81638e0 0x7f99a81562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.373+0000 7f99ae86d640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f99a80b17c0 con 0x7f99a815f0c0 2024-05-08T05:51:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.373+0000 7f99ae86d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f99a81630e0 con 0x7f99a806c3a0 2024-05-08T05:51:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.373+0000 7f99ae86d640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f99a806c7e0 con 0x7f99a81638e0 2024-05-08T05:51:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.373+0000 7f99a77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a815f0c0 0x7f99a80d2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.373+0000 7f99a77fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a815f0c0 0x7f99a80d2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51324/0 (socket says 172.21.15.45:51324) 2024-05-08T05:51:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.373+0000 7f99a77fe640 1 -- 172.21.15.45:0/2139148874 learned_addr learned my addr 172.21.15.45:0/2139148874 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.374+0000 7f99a77fe640 1 -- 172.21.15.45:0/2139148874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99a81638e0 msgr2=0x7f99a81562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.374+0000 7f99a77fe640 1 --2- 172.21.15.45:0/2139148874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99a81638e0 0x7f99a81562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.374+0000 7f99a77fe640 1 -- 172.21.15.45:0/2139148874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a806c3a0 msgr2=0x7f99a8155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.374+0000 7f99a77fe640 1 --2- 172.21.15.45:0/2139148874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a806c3a0 0x7f99a8155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.374+0000 7f99a77fe640 1 -- 172.21.15.45:0/2139148874 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a8154550 con 0x7f99a815f0c0 2024-05-08T05:51:28.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.374+0000 7f99a77fe640 1 --2- 172.21.15.45:0/2139148874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a815f0c0 0x7f99a80d2530 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f999c009890 tx=0x7f999c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.374+0000 7f99a4ff9640 1 -- 172.21.15.45:0/2139148874 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c037040 con 0x7f99a815f0c0 2024-05-08T05:51:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.374+0000 7f99a4ff9640 1 -- 172.21.15.45:0/2139148874 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f999c042070 con 0x7f99a815f0c0 2024-05-08T05:51:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.374+0000 7f99ae86d640 1 -- 172.21.15.45:0/2139148874 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a8154830 con 0x7f99a815f0c0 2024-05-08T05:51:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.374+0000 7f99a4ff9640 1 -- 172.21.15.45:0/2139148874 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c030bd0 con 0x7f99a815f0c0 2024-05-08T05:51:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.374+0000 7f99ae86d640 1 -- 172.21.15.45:0/2139148874 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a81539f0 con 0x7f99a815f0c0 2024-05-08T05:51:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.375+0000 7f99ae86d640 1 -- 172.21.15.45:0/2139148874 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a806a8f0 con 0x7f99a815f0c0 2024-05-08T05:51:28.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.376+0000 7f99a4ff9640 1 -- 172.21.15.45:0/2139148874 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f999c047460 con 0x7f99a815f0c0 2024-05-08T05:51:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.376+0000 7f99a4ff9640 1 --2- 172.21.15.45:0/2139148874 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99940762d0 0x7f9994078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.376+0000 7f99a4ff9640 1 -- 172.21.15.45:0/2139148874 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f999c0b8360 con 0x7f99a815f0c0 2024-05-08T05:51:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.377+0000 7f99a7fff640 1 --2- 172.21.15.45:0/2139148874 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99940762d0 0x7f9994078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:28.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.377+0000 7f99a7fff640 1 --2- 172.21.15.45:0/2139148874 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99940762d0 0x7f9994078790 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f99980098f0 tx=0x7f9998008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:28.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.381+0000 7f99a4ff9640 1 -- 172.21.15.45:0/2139148874 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f999c032e20 con 0x7f99a815f0c0 2024-05-08T05:51:28.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.584+0000 7f99ae86d640 1 -- 172.21.15.45:0/2139148874 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f99a80693f0 con 0x7f99a815f0c0 2024-05-08T05:51:28.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.584+0000 7f99a4ff9640 1 -- 172.21.15.45:0/2139148874 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v120) ==== 80+0+122 (secure 0 0 0) 0x7f999c081b70 con 0x7f99a815f0c0 2024-05-08T05:51:28.584 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6835/1751419054 2024-05-09T05:51:20.651495+0000 2024-05-08T05:51:28.584 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6834/1751419054 2024-05-09T05:51:20.651495+0000 2024-05-08T05:51:28.585 INFO:teuthology.orchestra.run.smithi045.stderr:listed 2 entries 2024-05-08T05:51:28.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.586+0000 7f99ae86d640 1 -- 172.21.15.45:0/2139148874 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99940762d0 msgr2=0x7f9994078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:28.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.586+0000 7f99ae86d640 1 --2- 172.21.15.45:0/2139148874 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99940762d0 0x7f9994078790 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f99980098f0 tx=0x7f9998008040 comp rx=0 tx=0).stop 2024-05-08T05:51:28.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.586+0000 7f99ae86d640 1 -- 172.21.15.45:0/2139148874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a815f0c0 msgr2=0x7f99a80d2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:28.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.586+0000 7f99ae86d640 1 --2- 172.21.15.45:0/2139148874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a815f0c0 0x7f99a80d2530 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f999c009890 tx=0x7f999c0046c0 comp rx=0 tx=0).stop 2024-05-08T05:51:28.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.586+0000 7f99ae86d640 1 -- 172.21.15.45:0/2139148874 shutdown_connections 2024-05-08T05:51:28.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.586+0000 7f99ae86d640 1 --2- 172.21.15.45:0/2139148874 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f99940762d0 0x7f9994078790 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:28.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.586+0000 7f99ae86d640 1 --2- 172.21.15.45:0/2139148874 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f99a81638e0 0x7f99a81562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:28.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.586+0000 7f99ae86d640 1 --2- 172.21.15.45:0/2139148874 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99a815f0c0 0x7f99a80d2530 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:28.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.586+0000 7f99ae86d640 1 --2- 172.21.15.45:0/2139148874 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a806c3a0 0x7f99a8155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:28.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.587+0000 7f99ae86d640 1 -- 172.21.15.45:0/2139148874 >> 172.21.15.45:0/2139148874 conn(0x7f99a80b3b10 msgr2=0x7f99a8161500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:28.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.587+0000 7f99ae86d640 1 -- 172.21.15.45:0/2139148874 shutdown_connections 2024-05-08T05:51:28.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.587+0000 7f99ae86d640 1 -- 172.21.15.45:0/2139148874 wait complete. 2024-05-08T05:51:28.597 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T05:51:28.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:28.598 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T05:51:28.627 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:51:28.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:28.627 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T05:51:28.684 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:51:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.829+0000 7fe4b2020640 1 Processor -- start 2024-05-08T05:51:28.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.829+0000 7fe4b2020640 1 -- start start 2024-05-08T05:51:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.829+0000 7fe4b2020640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe4ac15f420 0x7fe4ac0d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.829+0000 7fe4b2020640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe4ac0d3ca0 0x7fe4ac0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.829+0000 7fe4b2020640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4ac1630b0 con 0x7fe4ac0d1eb0 2024-05-08T05:51:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.829+0000 7fe4b2020640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe4ac0d2550 con 0x7fe4ac15f420 2024-05-08T05:51:28.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.829+0000 7fe4b2020640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe4ac0d2c80 con 0x7fe4ac0d3ca0 2024-05-08T05:51:28.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.830+0000 7fe4ab7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe4ac15f420 0x7fe4ac0d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:28.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.830+0000 7fe4ab7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe4ac15f420 0x7fe4ac0d3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51334/0 (socket says 172.21.15.45:51334) 2024-05-08T05:51:28.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.830+0000 7fe4ab7fe640 1 -- 172.21.15.45:0/688833541 learned_addr learned my addr 172.21.15.45:0/688833541 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:28.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.830+0000 7fe4abfff640 1 --1- 172.21.15.45:0/688833541 >> v1:172.21.15.45:6789/0 conn(0x7fe4ac0d1eb0 0x7fe4ac0d4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49630/0 (socket says 172.21.15.45:49630) 2024-05-08T05:51:28.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.830+0000 7fe4aaffd640 1 --2- 172.21.15.45:0/688833541 >> v2:172.21.15.63:3300/0 conn(0x7fe4ac0d3ca0 0x7fe4ac0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:28.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.830+0000 7fe4ab7fe640 1 -- 172.21.15.45:0/688833541 >> v2:172.21.15.63:3300/0 conn(0x7fe4ac0d3ca0 msgr2=0x7fe4ac0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:28.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.830+0000 7fe4ab7fe640 1 --2- 172.21.15.45:0/688833541 >> v2:172.21.15.63:3300/0 conn(0x7fe4ac0d3ca0 0x7fe4ac0d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:28.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.830+0000 7fe4ab7fe640 1 -- 172.21.15.45:0/688833541 >> v1:172.21.15.45:6789/0 conn(0x7fe4ac0d1eb0 legacy=0x7fe4ac0d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:28.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.830+0000 7fe4ab7fe640 1 -- 172.21.15.45:0/688833541 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ac0d2f60 con 0x7fe4ac15f420 2024-05-08T05:51:28.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.830+0000 7fe4aa7fc640 1 -- 172.21.15.45:0/688833541 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3415624783 0 0) 0x7fe4a4001490 con 0x7fe4ac0d1eb0 2024-05-08T05:51:28.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.831+0000 7fe4ab7fe640 1 --2- 172.21.15.45:0/688833541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4ac15f420 0x7fe4ac0d3760 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac08f380 tx=0x7fe49c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c2a76b6ff436d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:28.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.831+0000 7fe4aa7fc640 1 -- 172.21.15.45:0/688833541 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c037070 con 0x7fe4ac15f420 2024-05-08T05:51:28.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.831+0000 7fe4aa7fc640 1 -- 172.21.15.45:0/688833541 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe49c032440 con 0x7fe4ac15f420 2024-05-08T05:51:28.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.831+0000 7fe4aa7fc640 1 -- 172.21.15.45:0/688833541 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c032810 con 0x7fe4ac15f420 2024-05-08T05:51:28.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.831+0000 7fe4b2020640 1 -- 172.21.15.45:0/688833541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4ac15f420 msgr2=0x7fe4ac0d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:28.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.831+0000 7fe4b2020640 1 --2- 172.21.15.45:0/688833541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4ac15f420 0x7fe4ac0d3760 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac08f380 tx=0x7fe49c02a270 comp rx=0 tx=0).stop 2024-05-08T05:51:28.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.831+0000 7fe4b2020640 1 -- 172.21.15.45:0/688833541 shutdown_connections 2024-05-08T05:51:28.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.831+0000 7fe4b2020640 1 --2- 172.21.15.45:0/688833541 >> v2:172.21.15.63:3300/0 conn(0x7fe4ac0d3ca0 0x7fe4ac0d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:28.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.831+0000 7fe4b2020640 1 --2- 172.21.15.45:0/688833541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4ac15f420 0x7fe4ac0d3760 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:28.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.831+0000 7fe4b2020640 1 -- 172.21.15.45:0/688833541 >> 172.21.15.45:0/688833541 conn(0x7fe4ac08eac0 msgr2=0x7fe4ac09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.832+0000 7fe4b2020640 1 -- 172.21.15.45:0/688833541 shutdown_connections 2024-05-08T05:51:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.832+0000 7fe4b2020640 1 -- 172.21.15.45:0/688833541 wait complete. 2024-05-08T05:51:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.833+0000 7fe4b2020640 1 Processor -- start 2024-05-08T05:51:28.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.833+0000 7fe4b2020640 1 -- start start 2024-05-08T05:51:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4b2020640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4ac0d1eb0 0x7fe4ac158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4b2020640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4ac0c9a20 0x7fe4ac15f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4b2020640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4ac0c9ea0 0x7fe4ac0caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4b2020640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac0d53d0 con 0x7fe4ac0c9a20 2024-05-08T05:51:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4b2020640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac0d2830 con 0x7fe4ac0d1eb0 2024-05-08T05:51:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4b2020640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe4ac0d5550 con 0x7fe4ac0c9ea0 2024-05-08T05:51:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4aaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4ac0c9a20 0x7fe4ac15f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4aaffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4ac0c9a20 0x7fe4ac15f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51338/0 (socket says 172.21.15.45:51338) 2024-05-08T05:51:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4aaffd640 1 -- 172.21.15.45:0/4145599735 learned_addr learned my addr 172.21.15.45:0/4145599735 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4ab7fe640 1 --2- 172.21.15.45:0/4145599735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4ac0d1eb0 0x7fe4ac158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4abfff640 1 --2- 172.21.15.45:0/4145599735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4ac0c9ea0 0x7fe4ac0caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4aaffd640 1 -- 172.21.15.45:0/4145599735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4ac0c9ea0 msgr2=0x7fe4ac0caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4aaffd640 1 --2- 172.21.15.45:0/4145599735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4ac0c9ea0 0x7fe4ac0caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:28.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4aaffd640 1 -- 172.21.15.45:0/4145599735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4ac0d1eb0 msgr2=0x7fe4ac158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:28.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4aaffd640 1 --2- 172.21.15.45:0/4145599735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4ac0d1eb0 0x7fe4ac158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:28.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4aaffd640 1 -- 172.21.15.45:0/4145599735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ac0cc280 con 0x7fe4ac0c9a20 2024-05-08T05:51:28.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4aaffd640 1 --2- 172.21.15.45:0/4145599735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4ac0c9a20 0x7fe4ac15f860 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fe49400d800 tx=0x7fe49400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:28.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4a3fff640 1 -- 172.21.15.45:0/4145599735 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe494004020 con 0x7fe4ac0c9a20 2024-05-08T05:51:28.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4a3fff640 1 -- 172.21.15.45:0/4145599735 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4940041c0 con 0x7fe4ac0c9a20 2024-05-08T05:51:28.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4a3fff640 1 -- 172.21.15.45:0/4145599735 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe494004390 con 0x7fe4ac0c9a20 2024-05-08T05:51:28.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.834+0000 7fe4b2020640 1 -- 172.21.15.45:0/4145599735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4ac0cc560 con 0x7fe4ac0c9a20 2024-05-08T05:51:28.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.835+0000 7fe4b2020640 1 -- 172.21.15.45:0/4145599735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4ac0ccb20 con 0x7fe4ac0c9a20 2024-05-08T05:51:28.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.835+0000 7fe4b2020640 1 -- 172.21.15.45:0/4145599735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4ac0693f0 con 0x7fe4ac0c9a20 2024-05-08T05:51:28.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.836+0000 7fe4a3fff640 1 -- 172.21.15.45:0/4145599735 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe49401b020 con 0x7fe4ac0c9a20 2024-05-08T05:51:28.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.837+0000 7fe4a3fff640 1 --2- 172.21.15.45:0/4145599735 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe48c0762d0 0x7fe48c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:28.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.837+0000 7fe4a3fff640 1 -- 172.21.15.45:0/4145599735 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7fe494097420 con 0x7fe4ac0c9a20 2024-05-08T05:51:28.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.837+0000 7fe4ab7fe640 1 --2- 172.21.15.45:0/4145599735 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe48c0762d0 0x7fe48c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:28.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.837+0000 7fe4ab7fe640 1 --2- 172.21.15.45:0/4145599735 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe48c0762d0 0x7fe48c078790 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe49c034000 tx=0x7fe49c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:28.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:28.841+0000 7fe4a3fff640 1 -- 172.21.15.45:0/4145599735 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe4940605e0 con 0x7fe4ac0c9a20 2024-05-08T05:51:29.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.077+0000 7fe4b2020640 1 -- 172.21.15.45:0/4145599735 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4ac0d15b0 con 0x7fe4ac0c9a20 2024-05-08T05:51:29.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.078+0000 7fe4a3fff640 1 -- 172.21.15.45:0/4145599735 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5185 (secure 0 0 0) 0x7fe4940648e0 con 0x7fe4ac0c9a20 2024-05-08T05:51:29.078 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:29.078 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":106,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7168,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3882445426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3882445426},{"type":"v1","addr":"172.21.15.63:6836","nonce":3882445426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/3173207720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3173207720},{"type":"v1","addr":"172.21.15.45:6833","nonce":3173207720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4254619360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4254619360},{"type":"v1","addr":"172.21.15.63:6837","nonce":4254619360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:51:04.429358+0000","modified":"2024-05-08T05:51:24.668156+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8319":{"gid":8319,"name":"c","rank":0,"incarnation":103,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/453198328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":453198328},{"type":"v1","addr":"172.21.15.45:6835","nonce":453198328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8319,"qdb_cluster":[8319]},"id":8}]} 2024-05-08T05:51:29.078 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 106 2024-05-08T05:51:29.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.080+0000 7fe4b2020640 1 -- 172.21.15.45:0/4145599735 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe48c0762d0 msgr2=0x7fe48c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.080+0000 7fe4b2020640 1 --2- 172.21.15.45:0/4145599735 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe48c0762d0 0x7fe48c078790 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe49c034000 tx=0x7fe49c02e000 comp rx=0 tx=0).stop 2024-05-08T05:51:29.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.080+0000 7fe4b2020640 1 -- 172.21.15.45:0/4145599735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4ac0c9a20 msgr2=0x7fe4ac15f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.080+0000 7fe4b2020640 1 --2- 172.21.15.45:0/4145599735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4ac0c9a20 0x7fe4ac15f860 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fe49400d800 tx=0x7fe49400dcc0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.080+0000 7fe4b2020640 1 -- 172.21.15.45:0/4145599735 shutdown_connections 2024-05-08T05:51:29.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.080+0000 7fe4b2020640 1 --2- 172.21.15.45:0/4145599735 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe48c0762d0 0x7fe48c078790 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.080+0000 7fe4b2020640 1 --2- 172.21.15.45:0/4145599735 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe4ac0c9ea0 0x7fe4ac0caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.080+0000 7fe4b2020640 1 --2- 172.21.15.45:0/4145599735 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4ac0c9a20 0x7fe4ac15f860 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.080+0000 7fe4b2020640 1 --2- 172.21.15.45:0/4145599735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4ac0d1eb0 0x7fe4ac158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.080+0000 7fe4b2020640 1 -- 172.21.15.45:0/4145599735 >> 172.21.15.45:0/4145599735 conn(0x7fe4ac08eac0 msgr2=0x7fe4ac06cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:29.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.080+0000 7fe4b2020640 1 -- 172.21.15.45:0/4145599735 shutdown_connections 2024-05-08T05:51:29.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.080+0000 7fe4b2020640 1 -- 172.21.15.45:0/4145599735 wait complete. 2024-05-08T05:51:29.092 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parent mytag123 2024-05-08T05:51:29.218 INFO:teuthology.orchestra.run.smithi045.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-05-08T05:51:29.219 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-05-08T05:51:29.219 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-05-08T05:51:29.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.263+0000 7f66da971f40 1 Processor -- start 2024-05-08T05:51:29.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.263+0000 7f66da971f40 1 -- start start 2024-05-08T05:51:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.264+0000 7f66da971f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed98db2f50 0x55ed98dc48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.264+0000 7f66da971f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed98e58560 0x55ed98dc5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.264+0000 7f66da971f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ed98e58940 0x55ed98e54360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.264+0000 7f66da971f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ed98d85ff0 con 0x55ed98db2f50 2024-05-08T05:51:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.264+0000 7f66da971f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ed98d2af50 con 0x55ed98e58560 2024-05-08T05:51:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.264+0000 7f66da971f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55ed98d86de0 con 0x55ed98e58940 2024-05-08T05:51:29.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.264+0000 7f66d846d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed98db2f50 0x55ed98dc48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.264+0000 7f66d846d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed98db2f50 0x55ed98dc48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51360/0 (socket says 172.21.15.45:51360) 2024-05-08T05:51:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.264+0000 7f66d846d640 1 -- 172.21.15.45:0/2944868257 learned_addr learned my addr 172.21.15.45:0/2944868257 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.264+0000 7f66d846d640 1 -- 172.21.15.45:0/2944868257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ed98e58940 msgr2=0x55ed98e54360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.264+0000 7f66d846d640 1 --2- 172.21.15.45:0/2944868257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ed98e58940 0x55ed98e54360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.264+0000 7f66d846d640 1 -- 172.21.15.45:0/2944868257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed98e58560 msgr2=0x55ed98dc5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.264+0000 7f66cffff640 1 --2- 172.21.15.45:0/2944868257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed98e58560 0x55ed98dc5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.264+0000 7f66d846d640 1 --2- 172.21.15.45:0/2944868257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed98e58560 0x55ed98dc5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.265+0000 7f66d846d640 1 -- 172.21.15.45:0/2944868257 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed98db2d20 con 0x55ed98db2f50 2024-05-08T05:51:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.265+0000 7f66cffff640 1 --2- 172.21.15.45:0/2944868257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed98e58560 0x55ed98dc5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.265+0000 7f66d846d640 1 --2- 172.21.15.45:0/2944868257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed98db2f50 0x55ed98dc48d0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f66c800a770 tx=0x7f66c800a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa2a0825d28a31b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.265+0000 7f66d746b640 1 -- 172.21.15.45:0/2944868257 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8019070 con 0x55ed98db2f50 2024-05-08T05:51:29.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.265+0000 7f66d746b640 1 -- 172.21.15.45:0/2944868257 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66c8007600 con 0x55ed98db2f50 2024-05-08T05:51:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.265+0000 7f66d746b640 1 -- 172.21.15.45:0/2944868257 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c80079d0 con 0x55ed98db2f50 2024-05-08T05:51:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.265+0000 7f66da971f40 1 -- 172.21.15.45:0/2944868257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed98db2f50 msgr2=0x55ed98dc48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.265+0000 7f66da971f40 1 --2- 172.21.15.45:0/2944868257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed98db2f50 0x55ed98dc48d0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f66c800a770 tx=0x7f66c800a860 comp rx=0 tx=0).stop 2024-05-08T05:51:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.265+0000 7f66da971f40 1 -- 172.21.15.45:0/2944868257 shutdown_connections 2024-05-08T05:51:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.265+0000 7f66da971f40 1 --2- 172.21.15.45:0/2944868257 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ed98e58940 0x55ed98e54360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.265+0000 7f66da971f40 1 --2- 172.21.15.45:0/2944868257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed98e58560 0x55ed98dc5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.265+0000 7f66da971f40 1 --2- 172.21.15.45:0/2944868257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed98db2f50 0x55ed98dc48d0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.265+0000 7f66da971f40 1 -- 172.21.15.45:0/2944868257 >> 172.21.15.45:0/2944868257 conn(0x55ed98db0b40 msgr2=0x55ed98db0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.265+0000 7f66da971f40 1 -- 172.21.15.45:0/2944868257 shutdown_connections 2024-05-08T05:51:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.265+0000 7f66da971f40 1 -- 172.21.15.45:0/2944868257 wait complete. 2024-05-08T05:51:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.266+0000 7f66da971f40 1 Processor -- start 2024-05-08T05:51:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.266+0000 7f66da971f40 1 -- start start 2024-05-08T05:51:29.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.267+0000 7f66da971f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed98db2f50 0x55ed98e4a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.267+0000 7f66da971f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ed98e58560 0x55ed98ee7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.267+0000 7f66da971f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed98e58940 0x55ed98e5a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.267+0000 7f66da971f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ed98e57370 con 0x55ed98e58940 2024-05-08T05:51:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.267+0000 7f66da971f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ed98db27f0 con 0x55ed98db2f50 2024-05-08T05:51:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.267+0000 7f66da971f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55ed98db2a40 con 0x55ed98e58560 2024-05-08T05:51:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.267+0000 7f66d8c6e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed98e58940 0x55ed98e5a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.267+0000 7f66d8c6e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed98e58940 0x55ed98e5a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51368/0 (socket says 172.21.15.45:51368) 2024-05-08T05:51:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.267+0000 7f66d8c6e640 1 -- 172.21.15.45:0/3524450752 learned_addr learned my addr 172.21.15.45:0/3524450752 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.267+0000 7f66d846d640 1 --2- 172.21.15.45:0/3524450752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed98db2f50 0x55ed98e4a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.267+0000 7f66cffff640 1 --2- 172.21.15.45:0/3524450752 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ed98e58560 0x55ed98ee7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.267+0000 7f66d8c6e640 1 -- 172.21.15.45:0/3524450752 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ed98e58560 msgr2=0x55ed98ee7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.267+0000 7f66d8c6e640 1 --2- 172.21.15.45:0/3524450752 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ed98e58560 0x55ed98ee7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.267+0000 7f66d8c6e640 1 -- 172.21.15.45:0/3524450752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed98db2f50 msgr2=0x55ed98e4a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.267+0000 7f66d8c6e640 1 --2- 172.21.15.45:0/3524450752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ed98db2f50 0x55ed98e4a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.267+0000 7f66d8c6e640 1 -- 172.21.15.45:0/3524450752 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed98d82b30 con 0x55ed98e58940 2024-05-08T05:51:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.267+0000 7f66d8c6e640 1 --2- 172.21.15.45:0/3524450752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed98e58940 0x55ed98e5a520 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f66d0006fc0 tx=0x7f66d000b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.267+0000 7f66d6469640 1 -- 172.21.15.45:0/3524450752 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0011620 con 0x55ed98e58940 2024-05-08T05:51:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.268+0000 7f66d6469640 1 -- 172.21.15.45:0/3524450752 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d0016070 con 0x55ed98e58940 2024-05-08T05:51:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.268+0000 7f66da971f40 1 -- 172.21.15.45:0/3524450752 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ed98e4a570 con 0x55ed98e58940 2024-05-08T05:51:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.268+0000 7f66d6469640 1 -- 172.21.15.45:0/3524450752 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0011a60 con 0x55ed98e58940 2024-05-08T05:51:29.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.268+0000 7f66da971f40 1 -- 172.21.15.45:0/3524450752 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed98d830b0 con 0x55ed98e58940 2024-05-08T05:51:29.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:51:29.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.269+0000 7f66d6469640 1 -- 172.21.15.45:0/3524450752 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f66d0011c00 con 0x55ed98e58940 2024-05-08T05:51:29.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.270+0000 7f66d6469640 1 --2- 172.21.15.45:0/3524450752 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f66b8074ce0 0x7f66b80771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.270+0000 7f66d6469640 1 -- 172.21.15.45:0/3524450752 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f66d0095fd0 con 0x55ed98e58940 2024-05-08T05:51:29.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.270+0000 7f66d846d640 1 --2- 172.21.15.45:0/3524450752 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f66b8074ce0 0x7f66b80771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.270+0000 7f66da971f40 1 --2- 172.21.15.45:0/3524450752 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55ed98d83b90 0x55ed98d83f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.270+0000 7f66da971f40 1 -- 172.21.15.45:0/3524450752 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:1 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x55ed98e4dd40 con 0x55ed98d83b90 2024-05-08T05:51:29.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.270+0000 7f66cffff640 1 --2- 172.21.15.45:0/3524450752 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55ed98d83b90 0x55ed98d83f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.270+0000 7f66d846d640 1 --2- 172.21.15.45:0/3524450752 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f66b8074ce0 0x7f66b80771a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f66c800a770 tx=0x7f66c800c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.270+0000 7f66cffff640 1 --2- 172.21.15.45:0/3524450752 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55ed98d83b90 0x55ed98d83f70 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.271+0000 7f66cffff640 1 -- 172.21.15.45:0/3524450752 <== osd.1 v2:172.21.15.45:6824/1337179184 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f66c0054870 con 0x55ed98d83b90 2024-05-08T05:51:29.275 INFO:teuthology.orchestra.run.smithi045.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-05-08T05:51:29.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.271+0000 7f66da971f40 1 -- 172.21.15.45:0/3524450752 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55ed98d83b90 msgr2=0x55ed98d83f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.271+0000 7f66da971f40 1 --2- 172.21.15.45:0/3524450752 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55ed98d83b90 0x55ed98d83f70 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.271+0000 7f66da971f40 1 -- 172.21.15.45:0/3524450752 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f66b8074ce0 msgr2=0x7f66b80771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.271+0000 7f66da971f40 1 --2- 172.21.15.45:0/3524450752 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f66b8074ce0 0x7f66b80771a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f66c800a770 tx=0x7f66c800c000 comp rx=0 tx=0).stop 2024-05-08T05:51:29.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.271+0000 7f66da971f40 1 -- 172.21.15.45:0/3524450752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed98e58940 msgr2=0x55ed98e5a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.271+0000 7f66da971f40 1 --2- 172.21.15.45:0/3524450752 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ed98e58940 0x55ed98e5a520 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f66d0006fc0 tx=0x7f66d000b4b0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.271+0000 7f66d8c6e640 1 -- 172.21.15.45:0/3524450752 reap_dead start 2024-05-08T05:51:29.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.271+0000 7f66da971f40 1 -- 172.21.15.45:0/3524450752 shutdown_connections 2024-05-08T05:51:29.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.271+0000 7f66da971f40 1 -- 172.21.15.45:0/3524450752 >> 172.21.15.45:0/3524450752 conn(0x55ed98db0b40 msgr2=0x55ed98e568f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:29.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.271+0000 7f66da971f40 1 -- 172.21.15.45:0/3524450752 shutdown_connections 2024-05-08T05:51:29.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.271+0000 7f66da971f40 1 -- 172.21.15.45:0/3524450752 wait complete. 2024-05-08T05:51:29.277 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-05-08T05:51:29.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.321+0000 7f6e14c0cf40 1 Processor -- start 2024-05-08T05:51:29.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.321+0000 7f6e14c0cf40 1 -- start start 2024-05-08T05:51:29.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.322+0000 7f6e14c0cf40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb7617cf50 0x55cb7618e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.322+0000 7f6e14c0cf40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cb76222560 0x55cb76224f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.322+0000 7f6e14c0cf40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb76222940 0x55cb76225480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.322+0000 7f6e14c0cf40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cb7614fff0 con 0x55cb76222940 2024-05-08T05:51:29.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.322+0000 7f6e14c0cf40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cb760f4f50 con 0x55cb7617cf50 2024-05-08T05:51:29.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.322+0000 7f6e14c0cf40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55cb76150de0 con 0x55cb76222560 2024-05-08T05:51:29.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.322+0000 7f6e12242640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb76222940 0x55cb76225480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.322+0000 7f6e12242640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb76222940 0x55cb76225480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51382/0 (socket says 172.21.15.45:51382) 2024-05-08T05:51:29.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.322+0000 7f6e12242640 1 -- 172.21.15.45:0/2088851402 learned_addr learned my addr 172.21.15.45:0/2088851402 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:29.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.322+0000 7f6e11a41640 1 --2- 172.21.15.45:0/2088851402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb7617cf50 0x55cb7618e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.322+0000 7f6e11240640 1 --2- 172.21.15.45:0/2088851402 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cb76222560 0x55cb76224f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.322+0000 7f6e12242640 1 -- 172.21.15.45:0/2088851402 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cb76222560 msgr2=0x55cb76224f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:29.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.322+0000 7f6e12242640 1 --2- 172.21.15.45:0/2088851402 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cb76222560 0x55cb76224f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.322+0000 7f6e12242640 1 -- 172.21.15.45:0/2088851402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb7617cf50 msgr2=0x55cb7618e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:29.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.322+0000 7f6e12242640 1 --2- 172.21.15.45:0/2088851402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb7617cf50 0x55cb7618e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.322+0000 7f6e12242640 1 -- 172.21.15.45:0/2088851402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb7621de20 con 0x55cb76222940 2024-05-08T05:51:29.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.322+0000 7f6e12242640 1 --2- 172.21.15.45:0/2088851402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb76222940 0x55cb76225480 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f6e04002660 tx=0x7f6e04002b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadd802dbca1121b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.322+0000 7f6e0bfff640 1 -- 172.21.15.45:0/2088851402 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e04016070 con 0x55cb76222940 2024-05-08T05:51:29.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.322+0000 7f6e0bfff640 1 -- 172.21.15.45:0/2088851402 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0400f730 con 0x55cb76222940 2024-05-08T05:51:29.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.322+0000 7f6e0bfff640 1 -- 172.21.15.45:0/2088851402 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0400fb20 con 0x55cb76222940 2024-05-08T05:51:29.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.323+0000 7f6e14c0cf40 1 -- 172.21.15.45:0/2088851402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb76222940 msgr2=0x55cb76225480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.323+0000 7f6e14c0cf40 1 --2- 172.21.15.45:0/2088851402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb76222940 0x55cb76225480 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f6e04002660 tx=0x7f6e04002b20 comp rx=0 tx=0).stop 2024-05-08T05:51:29.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.323+0000 7f6e14c0cf40 1 -- 172.21.15.45:0/2088851402 shutdown_connections 2024-05-08T05:51:29.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.323+0000 7f6e14c0cf40 1 --2- 172.21.15.45:0/2088851402 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cb76222560 0x55cb76224f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.323+0000 7f6e14c0cf40 1 --2- 172.21.15.45:0/2088851402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb76222940 0x55cb76225480 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.323+0000 7f6e14c0cf40 1 --2- 172.21.15.45:0/2088851402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb7617cf50 0x55cb7618e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.323+0000 7f6e14c0cf40 1 -- 172.21.15.45:0/2088851402 >> 172.21.15.45:0/2088851402 conn(0x55cb7617ab40 msgr2=0x55cb7617a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:29.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.323+0000 7f6e14c0cf40 1 -- 172.21.15.45:0/2088851402 shutdown_connections 2024-05-08T05:51:29.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.323+0000 7f6e14c0cf40 1 -- 172.21.15.45:0/2088851402 wait complete. 2024-05-08T05:51:29.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.324+0000 7f6e14c0cf40 1 Processor -- start 2024-05-08T05:51:29.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.324+0000 7f6e14c0cf40 1 -- start start 2024-05-08T05:51:29.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.324+0000 7f6e14c0cf40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb7617cf50 0x55cb7621e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.324+0000 7f6e14c0cf40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cb76222560 0x55cb76225130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.324+0000 7f6e14c0cf40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb76222940 0x55cb762be600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.324+0000 7f6e14c0cf40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55cb7617b320 con 0x55cb7617cf50 2024-05-08T05:51:29.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.324+0000 7f6e14c0cf40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55cb7621dbd0 con 0x55cb76222940 2024-05-08T05:51:29.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.324+0000 7f6e14c0cf40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55cb7621a2c0 con 0x55cb76222560 2024-05-08T05:51:29.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.324+0000 7f6e11a41640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb7617cf50 0x55cb7621e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.324+0000 7f6e11a41640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb7617cf50 0x55cb7621e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51386/0 (socket says 172.21.15.45:51386) 2024-05-08T05:51:29.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.324+0000 7f6e11a41640 1 -- 172.21.15.45:0/197836806 learned_addr learned my addr 172.21.15.45:0/197836806 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:29.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.325+0000 7f6e11240640 1 --2- 172.21.15.45:0/197836806 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cb76222560 0x55cb76225130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.325+0000 7f6e12242640 1 --2- 172.21.15.45:0/197836806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb76222940 0x55cb762be600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.325+0000 7f6e11a41640 1 -- 172.21.15.45:0/197836806 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cb76222560 msgr2=0x55cb76225130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.325+0000 7f6e11a41640 1 --2- 172.21.15.45:0/197836806 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55cb76222560 0x55cb76225130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.325+0000 7f6e11a41640 1 -- 172.21.15.45:0/197836806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb76222940 msgr2=0x55cb762be600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.325+0000 7f6e11a41640 1 --2- 172.21.15.45:0/197836806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb76222940 0x55cb762be600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.325+0000 7f6e11a41640 1 -- 172.21.15.45:0/197836806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb7621fad0 con 0x55cb7617cf50 2024-05-08T05:51:29.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.325+0000 7f6e12242640 1 --2- 172.21.15.45:0/197836806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55cb76222940 0x55cb762be600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:29.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.325+0000 7f6e11a41640 1 --2- 172.21.15.45:0/197836806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb7617cf50 0x55cb7621e050 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c000c00 tx=0x7f6e0c003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.325+0000 7f6e0affd640 1 -- 172.21.15.45:0/197836806 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c06c070 con 0x55cb7617cf50 2024-05-08T05:51:29.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.325+0000 7f6e0affd640 1 -- 172.21.15.45:0/197836806 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0c052b70 con 0x55cb7617cf50 2024-05-08T05:51:29.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.325+0000 7f6e14c0cf40 1 -- 172.21.15.45:0/197836806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cb762b61c0 con 0x55cb7617cf50 2024-05-08T05:51:29.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.325+0000 7f6e0affd640 1 -- 172.21.15.45:0/197836806 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c056070 con 0x55cb7617cf50 2024-05-08T05:51:29.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.325+0000 7f6e14c0cf40 1 -- 172.21.15.45:0/197836806 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb76225670 con 0x55cb7617cf50 2024-05-08T05:51:29.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.327+0000 7f6e0affd640 1 -- 172.21.15.45:0/197836806 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6e0c06f070 con 0x55cb7617cf50 2024-05-08T05:51:29.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.327+0000 7f6e0affd640 1 --2- 172.21.15.45:0/197836806 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6df4074ce0 0x7f6df40771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.327+0000 7f6e0affd640 1 -- 172.21.15.45:0/197836806 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f6e0c0da740 con 0x55cb7617cf50 2024-05-08T05:51:29.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.327+0000 7f6e14c0cf40 1 --2- 172.21.15.45:0/197836806 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x55cb762bf050 0x55cb76220270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.327+0000 7f6e11240640 1 --2- 172.21.15.45:0/197836806 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6df4074ce0 0x7f6df40771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.327+0000 7f6e14c0cf40 1 -- 172.21.15.45:0/197836806 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:1 18.12 18:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x55cb762207b0 con 0x55cb762bf050 2024-05-08T05:51:29.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.328+0000 7f6e12242640 1 --2- 172.21.15.45:0/197836806 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x55cb762bf050 0x55cb76220270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.328+0000 7f6e12242640 1 --2- 172.21.15.45:0/197836806 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x55cb762bf050 0x55cb76220270 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.328+0000 7f6e11240640 1 --2- 172.21.15.45:0/197836806 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6df4074ce0 0x7f6df40771a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x55cb7618e0c0 tx=0x7f6dfc00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.328+0000 7f6e12242640 1 -- 172.21.15.45:0/197836806 <== osd.2 v2:172.21.15.45:6808/785834340 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f6e0400f070 con 0x55cb762bf050 2024-05-08T05:51:29.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.328+0000 7f6e14c0cf40 1 -- 172.21.15.45:0/197836806 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x55cb762bf050 msgr2=0x55cb76220270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.328+0000 7f6e14c0cf40 1 --2- 172.21.15.45:0/197836806 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x55cb762bf050 0x55cb76220270 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.328+0000 7f6e14c0cf40 1 -- 172.21.15.45:0/197836806 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6df4074ce0 msgr2=0x7f6df40771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.328+0000 7f6e14c0cf40 1 --2- 172.21.15.45:0/197836806 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f6df4074ce0 0x7f6df40771a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x55cb7618e0c0 tx=0x7f6dfc00b040 comp rx=0 tx=0).stop 2024-05-08T05:51:29.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.328+0000 7f6e14c0cf40 1 -- 172.21.15.45:0/197836806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb7617cf50 msgr2=0x55cb7621e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.328+0000 7f6e14c0cf40 1 --2- 172.21.15.45:0/197836806 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55cb7617cf50 0x55cb7621e050 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c000c00 tx=0x7f6e0c003020 comp rx=0 tx=0).stop 2024-05-08T05:51:29.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.328+0000 7f6e12242640 1 -- 172.21.15.45:0/197836806 reap_dead start 2024-05-08T05:51:29.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.328+0000 7f6e14c0cf40 1 -- 172.21.15.45:0/197836806 shutdown_connections 2024-05-08T05:51:29.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.328+0000 7f6e14c0cf40 1 -- 172.21.15.45:0/197836806 >> 172.21.15.45:0/197836806 conn(0x55cb7617ab40 msgr2=0x55cb76196200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:29.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.328+0000 7f6e14c0cf40 1 -- 172.21.15.45:0/197836806 shutdown_connections 2024-05-08T05:51:29.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.328+0000 7f6e14c0cf40 1 -- 172.21.15.45:0/197836806 wait complete. 2024-05-08T05:51:29.333 INFO:teuthology.orchestra.run.smithi045.stdout:mytag123 2024-05-08T05:51:29.334 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-05-08T05:51:29.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.378+0000 7fe52b3e6f40 1 Processor -- start 2024-05-08T05:51:29.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.378+0000 7fe52b3e6f40 1 -- start start 2024-05-08T05:51:29.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.379+0000 7fe52b3e6f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e445245f60 0x55e445248470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.379+0000 7fe52b3e6f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e44528f980 0x55e4453b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.379+0000 7fe52b3e6f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44531d130 0x55e4453b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.379+0000 7fe52b3e6f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e44531d720 con 0x55e44531d130 2024-05-08T05:51:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.379+0000 7fe52b3e6f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e44528fd60 con 0x55e445245f60 2024-05-08T05:51:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.379+0000 7fe52b3e6f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e44531da00 con 0x55e44528f980 2024-05-08T05:51:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.379+0000 7fe5296e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44531d130 0x55e4453b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.379+0000 7fe5296e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44531d130 0x55e4453b12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51406/0 (socket says 172.21.15.45:51406) 2024-05-08T05:51:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.379+0000 7fe5296e3640 1 -- 172.21.15.45:0/1573560902 learned_addr learned my addr 172.21.15.45:0/1573560902 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.379+0000 7fe528ee2640 1 --2- 172.21.15.45:0/1573560902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e445245f60 0x55e445248470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.379+0000 7fe5286e1640 1 --2- 172.21.15.45:0/1573560902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e44528f980 0x55e4453b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.379+0000 7fe5296e3640 1 -- 172.21.15.45:0/1573560902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e44528f980 msgr2=0x55e4453b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.379+0000 7fe5296e3640 1 --2- 172.21.15.45:0/1573560902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e44528f980 0x55e4453b0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.379+0000 7fe5296e3640 1 -- 172.21.15.45:0/1573560902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e445245f60 msgr2=0x55e445248470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:29.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.379+0000 7fe5296e3640 1 --2- 172.21.15.45:0/1573560902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e445245f60 0x55e445248470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.379+0000 7fe5296e3640 1 -- 172.21.15.45:0/1573560902 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4451b0350 con 0x55e44531d130 2024-05-08T05:51:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.379+0000 7fe5296e3640 1 --2- 172.21.15.45:0/1573560902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44531d130 0x55e4453b12f0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe5200259f0 tx=0x7fe520050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00af004dbfe6027 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.379+0000 7fe5276df640 1 -- 172.21.15.45:0/1573560902 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520051ba0 con 0x55e44531d130 2024-05-08T05:51:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.379+0000 7fe5276df640 1 -- 172.21.15.45:0/1573560902 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe520051d40 con 0x55e44531d130 2024-05-08T05:51:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.379+0000 7fe5276df640 1 -- 172.21.15.45:0/1573560902 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52005a670 con 0x55e44531d130 2024-05-08T05:51:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.380+0000 7fe52b3e6f40 1 -- 172.21.15.45:0/1573560902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44531d130 msgr2=0x55e4453b12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.380+0000 7fe52b3e6f40 1 --2- 172.21.15.45:0/1573560902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44531d130 0x55e4453b12f0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe5200259f0 tx=0x7fe520050000 comp rx=0 tx=0).stop 2024-05-08T05:51:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.380+0000 7fe52b3e6f40 1 -- 172.21.15.45:0/1573560902 shutdown_connections 2024-05-08T05:51:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.380+0000 7fe52b3e6f40 1 --2- 172.21.15.45:0/1573560902 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e44528f980 0x55e4453b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.380+0000 7fe52b3e6f40 1 --2- 172.21.15.45:0/1573560902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44531d130 0x55e4453b12f0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.380+0000 7fe52b3e6f40 1 --2- 172.21.15.45:0/1573560902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e445245f60 0x55e445248470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.380+0000 7fe52b3e6f40 1 -- 172.21.15.45:0/1573560902 >> 172.21.15.45:0/1573560902 conn(0x55e445246b30 msgr2=0x55e44528e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.380+0000 7fe52b3e6f40 1 -- 172.21.15.45:0/1573560902 shutdown_connections 2024-05-08T05:51:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.380+0000 7fe52b3e6f40 1 -- 172.21.15.45:0/1573560902 wait complete. 2024-05-08T05:51:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.380+0000 7fe52b3e6f40 1 Processor -- start 2024-05-08T05:51:29.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.381+0000 7fe52b3e6f40 1 -- start start 2024-05-08T05:51:29.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.381+0000 7fe52b3e6f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e445245f60 0x55e44530df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.381+0000 7fe52b3e6f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e44528f980 0x55e44530a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.381+0000 7fe52b3e6f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44531d130 0x55e44528d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.381+0000 7fe52b3e6f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e44531ef80 con 0x55e44531d130 2024-05-08T05:51:29.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.381+0000 7fe52b3e6f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e445273600 con 0x55e445245f60 2024-05-08T05:51:29.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.381+0000 7fe52b3e6f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e44531f260 con 0x55e44528f980 2024-05-08T05:51:29.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.381+0000 7fe5296e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44531d130 0x55e44528d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.381+0000 7fe5296e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44531d130 0x55e44528d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51422/0 (socket says 172.21.15.45:51422) 2024-05-08T05:51:29.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.381+0000 7fe5296e3640 1 -- 172.21.15.45:0/1751455822 learned_addr learned my addr 172.21.15.45:0/1751455822 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:29.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.381+0000 7fe528ee2640 1 --2- 172.21.15.45:0/1751455822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e445245f60 0x55e44530df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.382+0000 7fe5286e1640 1 --2- 172.21.15.45:0/1751455822 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e44528f980 0x55e44530a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.382+0000 7fe5296e3640 1 -- 172.21.15.45:0/1751455822 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e44528f980 msgr2=0x55e44530a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.382+0000 7fe5296e3640 1 --2- 172.21.15.45:0/1751455822 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e44528f980 0x55e44530a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.382+0000 7fe5296e3640 1 -- 172.21.15.45:0/1751455822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e445245f60 msgr2=0x55e44530df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.382+0000 7fe5296e3640 1 --2- 172.21.15.45:0/1751455822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e445245f60 0x55e44530df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.382+0000 7fe5296e3640 1 -- 172.21.15.45:0/1751455822 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e445248a20 con 0x55e44531d130 2024-05-08T05:51:29.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.382+0000 7fe528ee2640 1 --2- 172.21.15.45:0/1751455822 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e445245f60 0x55e44530df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.382+0000 7fe5296e3640 1 --2- 172.21.15.45:0/1751455822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44531d130 0x55e44528d3a0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fe52000a190 tx=0x7fe520003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.382+0000 7fe5266dd640 1 -- 172.21.15.45:0/1751455822 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5200519a0 con 0x55e44531d130 2024-05-08T05:51:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.382+0000 7fe5266dd640 1 -- 172.21.15.45:0/1751455822 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe520051b40 con 0x55e44531d130 2024-05-08T05:51:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.382+0000 7fe52b3e6f40 1 -- 172.21.15.45:0/1751455822 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e445248d00 con 0x55e44531d130 2024-05-08T05:51:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.382+0000 7fe5266dd640 1 -- 172.21.15.45:0/1751455822 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52005ce60 con 0x55e44531d130 2024-05-08T05:51:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.382+0000 7fe52b3e6f40 1 -- 172.21.15.45:0/1751455822 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e44531bab0 con 0x55e44531d130 2024-05-08T05:51:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.384+0000 7fe5266dd640 1 -- 172.21.15.45:0/1751455822 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe520058020 con 0x55e44531d130 2024-05-08T05:51:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.384+0000 7fe5266dd640 1 --2- 172.21.15.45:0/1751455822 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe514074cb0 0x7fe514077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.384+0000 7fe5266dd640 1 -- 172.21.15.45:0/1751455822 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7fe5200dbc90 con 0x55e44531d130 2024-05-08T05:51:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.384+0000 7fe528ee2640 1 --2- 172.21.15.45:0/1751455822 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe514074cb0 0x7fe514077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.384+0000 7fe52b3e6f40 1 --2- 172.21.15.45:0/1751455822 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x55e44530af70 0x55e44530b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.384+0000 7fe52b3e6f40 1 -- 172.21.15.45:0/1751455822 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:1 18.19 18:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x55e44530b890 con 0x55e44530af70 2024-05-08T05:51:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.385+0000 7fe5286e1640 1 --2- 172.21.15.45:0/1751455822 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x55e44530af70 0x55e44530b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.385+0000 7fe528ee2640 1 --2- 172.21.15.45:0/1751455822 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe514074cb0 0x7fe514077170 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe5180096e0 tx=0x7fe518005d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.388 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.385+0000 7fe5286e1640 1 --2- 172.21.15.45:0/1751455822 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x55e44530af70 0x55e44530b350 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.385+0000 7fe5286e1640 1 -- 172.21.15.45:0/1751455822 <== osd.4 v2:172.21.15.63:6824/2212879151 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7fe51c009690 con 0x55e44530af70 2024-05-08T05:51:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.385+0000 7fe52b3e6f40 1 -- 172.21.15.45:0/1751455822 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x55e44530af70 msgr2=0x55e44530b350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.385+0000 7fe52b3e6f40 1 --2- 172.21.15.45:0/1751455822 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x55e44530af70 0x55e44530b350 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.385+0000 7fe52b3e6f40 1 -- 172.21.15.45:0/1751455822 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe514074cb0 msgr2=0x7fe514077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.385+0000 7fe52b3e6f40 1 --2- 172.21.15.45:0/1751455822 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe514074cb0 0x7fe514077170 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe5180096e0 tx=0x7fe518005d30 comp rx=0 tx=0).stop 2024-05-08T05:51:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.385+0000 7fe52b3e6f40 1 -- 172.21.15.45:0/1751455822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44531d130 msgr2=0x55e44528d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.385+0000 7fe52b3e6f40 1 --2- 172.21.15.45:0/1751455822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e44531d130 0x55e44528d3a0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fe52000a190 tx=0x7fe520003330 comp rx=0 tx=0).stop 2024-05-08T05:51:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.385+0000 7fe5296e3640 1 -- 172.21.15.45:0/1751455822 reap_dead start 2024-05-08T05:51:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.385+0000 7fe52b3e6f40 1 -- 172.21.15.45:0/1751455822 shutdown_connections 2024-05-08T05:51:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.385+0000 7fe52b3e6f40 1 -- 172.21.15.45:0/1751455822 >> 172.21.15.45:0/1751455822 conn(0x55e445246b30 msgr2=0x55e445317500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.386+0000 7fe52b3e6f40 1 -- 172.21.15.45:0/1751455822 shutdown_connections 2024-05-08T05:51:29.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.386+0000 7fe52b3e6f40 1 -- 172.21.15.45:0/1751455822 wait complete. 2024-05-08T05:51:29.390 INFO:teuthology.orchestra.run.smithi045.stdout:mytag123 2024-05-08T05:51:29.391 DEBUG:teuthology.orchestra.run.smithi045:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-05-08T05:51:29.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.435+0000 7f1eb1c28f40 1 Processor -- start 2024-05-08T05:51:29.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.435+0000 7f1eb1c28f40 1 -- start start 2024-05-08T05:51:29.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.436+0000 7f1eb1c28f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dff0de8130 0x55dff0e181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.436+0000 7f1eb1c28f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff0e32ab0 0x55dff0f52c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.436+0000 7f1eb1c28f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dff0ebe580 0x55dff0f53170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.436+0000 7f1eb1c28f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dff0d90f50 con 0x55dff0de8130 2024-05-08T05:51:29.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.436+0000 7f1eb1c28f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dff0d94f90 con 0x55dff0e32ab0 2024-05-08T05:51:29.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.436+0000 7f1eb1c28f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55dff0eb1930 con 0x55dff0ebe580 2024-05-08T05:51:29.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.436+0000 7f1eaf722640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dff0de8130 0x55dff0e181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.436+0000 7f1eaf722640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dff0de8130 0x55dff0e181f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51444/0 (socket says 172.21.15.45:51444) 2024-05-08T05:51:29.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.436+0000 7f1eaf722640 1 -- 172.21.15.45:0/644446561 learned_addr learned my addr 172.21.15.45:0/644446561 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:29.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.436+0000 7f1eaff23640 1 --2- 172.21.15.45:0/644446561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dff0ebe580 0x55dff0f53170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.436+0000 7f1eaef21640 1 --2- 172.21.15.45:0/644446561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff0e32ab0 0x55dff0f52c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.436+0000 7f1eaf722640 1 -- 172.21.15.45:0/644446561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dff0ebe580 msgr2=0x55dff0f53170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:29.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.436+0000 7f1eaf722640 1 --2- 172.21.15.45:0/644446561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dff0ebe580 0x55dff0f53170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.436+0000 7f1eaf722640 1 -- 172.21.15.45:0/644446561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff0e32ab0 msgr2=0x55dff0f52c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:29.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.436+0000 7f1eaf722640 1 --2- 172.21.15.45:0/644446561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff0e32ab0 0x55dff0f52c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.436+0000 7f1eaf722640 1 -- 172.21.15.45:0/644446561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff0e17240 con 0x55dff0de8130 2024-05-08T05:51:29.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.436+0000 7f1eaf722640 1 --2- 172.21.15.45:0/644446561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dff0de8130 0x55dff0e181f0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0001660 tx=0x7f1ea0001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c082a7328156820e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.436+0000 7f1eadf1f640 1 -- 172.21.15.45:0/644446561 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0016070 con 0x55dff0de8130 2024-05-08T05:51:29.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.436+0000 7f1eadf1f640 1 -- 172.21.15.45:0/644446561 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea000f710 con 0x55dff0de8130 2024-05-08T05:51:29.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.436+0000 7f1eadf1f640 1 -- 172.21.15.45:0/644446561 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea000fae0 con 0x55dff0de8130 2024-05-08T05:51:29.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.437+0000 7f1eb1c28f40 1 -- 172.21.15.45:0/644446561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dff0de8130 msgr2=0x55dff0e181f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.437+0000 7f1eb1c28f40 1 --2- 172.21.15.45:0/644446561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dff0de8130 0x55dff0e181f0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0001660 tx=0x7f1ea0001b20 comp rx=0 tx=0).stop 2024-05-08T05:51:29.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.437+0000 7f1eb1c28f40 1 -- 172.21.15.45:0/644446561 shutdown_connections 2024-05-08T05:51:29.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.437+0000 7f1eb1c28f40 1 --2- 172.21.15.45:0/644446561 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dff0ebe580 0x55dff0f53170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.437+0000 7f1eb1c28f40 1 --2- 172.21.15.45:0/644446561 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff0e32ab0 0x55dff0f52c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.437+0000 7f1eb1c28f40 1 --2- 172.21.15.45:0/644446561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dff0de8130 0x55dff0e181f0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.437+0000 7f1eb1c28f40 1 -- 172.21.15.45:0/644446561 >> 172.21.15.45:0/644446561 conn(0x55dff0de8b30 msgr2=0x55dff0e174e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:29.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.437+0000 7f1eb1c28f40 1 -- 172.21.15.45:0/644446561 shutdown_connections 2024-05-08T05:51:29.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.437+0000 7f1eb1c28f40 1 -- 172.21.15.45:0/644446561 wait complete. 2024-05-08T05:51:29.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.437+0000 7f1eb1c28f40 1 Processor -- start 2024-05-08T05:51:29.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.437+0000 7f1eb1c28f40 1 -- start start 2024-05-08T05:51:29.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.438+0000 7f1eb1c28f40 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff0de8130 0x55dff0e183e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.438+0000 7f1eb1c28f40 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dff0e32ab0 0x55dff0f5ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.438+0000 7f1eb1c28f40 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dff0ebe580 0x55dff0f5b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.438+0000 7f1eb1c28f40 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55dff0e14260 con 0x55dff0ebe580 2024-05-08T05:51:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.438+0000 7f1eb1c28f40 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55dff0e16ad0 con 0x55dff0de8130 2024-05-08T05:51:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.438+0000 7f1eb1c28f40 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55dff0e14540 con 0x55dff0e32ab0 2024-05-08T05:51:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.438+0000 7f1eaff23640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dff0ebe580 0x55dff0f5b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.438+0000 7f1eaff23640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dff0ebe580 0x55dff0f5b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51454/0 (socket says 172.21.15.45:51454) 2024-05-08T05:51:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.438+0000 7f1eaff23640 1 -- 172.21.15.45:0/3926146006 learned_addr learned my addr 172.21.15.45:0/3926146006 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.438+0000 7f1eaef21640 1 --2- 172.21.15.45:0/3926146006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dff0e32ab0 0x55dff0f5ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.438+0000 7f1eaf722640 1 --2- 172.21.15.45:0/3926146006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff0de8130 0x55dff0e183e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.439+0000 7f1eaff23640 1 -- 172.21.15.45:0/3926146006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dff0e32ab0 msgr2=0x55dff0f5ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.439+0000 7f1eaff23640 1 --2- 172.21.15.45:0/3926146006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dff0e32ab0 0x55dff0f5ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.439+0000 7f1eaff23640 1 -- 172.21.15.45:0/3926146006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff0de8130 msgr2=0x55dff0e183e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.439+0000 7f1eaff23640 1 --2- 172.21.15.45:0/3926146006 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55dff0de8130 0x55dff0e183e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.439+0000 7f1eaff23640 1 -- 172.21.15.45:0/3926146006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff0eaedc0 con 0x55dff0ebe580 2024-05-08T05:51:29.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.439+0000 7f1eaef21640 1 --2- 172.21.15.45:0/3926146006 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55dff0e32ab0 0x55dff0f5ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:29.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.439+0000 7f1eaff23640 1 --2- 172.21.15.45:0/3926146006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dff0ebe580 0x55dff0f5b180 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8001eb0 tx=0x7f1ea800df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.439+0000 7f1eacf1d640 1 -- 172.21.15.45:0/3926146006 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8014020 con 0x55dff0ebe580 2024-05-08T05:51:29.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.439+0000 7f1eacf1d640 1 -- 172.21.15.45:0/3926146006 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea8010410 con 0x55dff0ebe580 2024-05-08T05:51:29.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.439+0000 7f1eacf1d640 1 -- 172.21.15.45:0/3926146006 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8008030 con 0x55dff0ebe580 2024-05-08T05:51:29.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.439+0000 7f1eb1c28f40 1 -- 172.21.15.45:0/3926146006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dff0eba880 con 0x55dff0ebe580 2024-05-08T05:51:29.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.439+0000 7f1eb1c28f40 1 -- 172.21.15.45:0/3926146006 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff0ebb130 con 0x55dff0ebe580 2024-05-08T05:51:29.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.440+0000 7f1eacf1d640 1 -- 172.21.15.45:0/3926146006 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1ea8017020 con 0x55dff0ebe580 2024-05-08T05:51:29.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.441+0000 7f1eacf1d640 1 --2- 172.21.15.45:0/3926146006 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1e9c074cb0 0x7f1e9c077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.441+0000 7f1eacf1d640 1 -- 172.21.15.45:0/3926146006 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f1ea80963a0 con 0x55dff0ebe580 2024-05-08T05:51:29.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.441+0000 7f1eb1c28f40 1 --2- 172.21.15.45:0/3926146006 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55dff0eb12e0 0x55dff0eb7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.441+0000 7f1eaf722640 1 --2- 172.21.15.45:0/3926146006 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1e9c074cb0 0x7f1e9c077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.441+0000 7f1eb1c28f40 1 -- 172.21.15.45:0/3926146006 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:1 18.5 18:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120) -- 0x55dff0eaf330 con 0x55dff0eb12e0 2024-05-08T05:51:29.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.441+0000 7f1eaef21640 1 --2- 172.21.15.45:0/3926146006 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55dff0eb12e0 0x55dff0eb7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.442+0000 7f1eaf722640 1 --2- 172.21.15.45:0/3926146006 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1e9c074cb0 0x7f1e9c077170 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0001b60 tx=0x7f1ea0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.442+0000 7f1eaef21640 1 --2- 172.21.15.45:0/3926146006 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55dff0eb12e0 0x55dff0eb7390 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.442+0000 7f1eaef21640 1 -- 172.21.15.45:0/3926146006 <== osd.1 v2:172.21.15.45:6824/1337179184 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f1ea4058950 con 0x55dff0eb12e0 2024-05-08T05:51:29.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.442+0000 7f1eb1c28f40 1 -- 172.21.15.45:0/3926146006 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55dff0eb12e0 msgr2=0x55dff0eb7390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.442+0000 7f1eb1c28f40 1 --2- 172.21.15.45:0/3926146006 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55dff0eb12e0 0x55dff0eb7390 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.442+0000 7f1eb1c28f40 1 -- 172.21.15.45:0/3926146006 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1e9c074cb0 msgr2=0x7f1e9c077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.442+0000 7f1eb1c28f40 1 --2- 172.21.15.45:0/3926146006 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1e9c074cb0 0x7f1e9c077170 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0001b60 tx=0x7f1ea0008030 comp rx=0 tx=0).stop 2024-05-08T05:51:29.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.442+0000 7f1eb1c28f40 1 -- 172.21.15.45:0/3926146006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dff0ebe580 msgr2=0x55dff0f5b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.442+0000 7f1eb1c28f40 1 --2- 172.21.15.45:0/3926146006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55dff0ebe580 0x55dff0f5b180 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8001eb0 tx=0x7f1ea800df90 comp rx=0 tx=0).stop 2024-05-08T05:51:29.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.442+0000 7f1eaff23640 1 -- 172.21.15.45:0/3926146006 reap_dead start 2024-05-08T05:51:29.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.442+0000 7f1eb1c28f40 1 -- 172.21.15.45:0/3926146006 shutdown_connections 2024-05-08T05:51:29.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.442+0000 7f1eb1c28f40 1 -- 172.21.15.45:0/3926146006 >> 172.21.15.45:0/3926146006 conn(0x55dff0de8b30 msgr2=0x55dff0e13850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:29.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.442+0000 7f1eb1c28f40 1 -- 172.21.15.45:0/3926146006 shutdown_connections 2024-05-08T05:51:29.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.442+0000 7f1eb1c28f40 1 -- 172.21.15.45:0/3926146006 wait complete. 2024-05-08T05:51:29.447 INFO:teuthology.orchestra.run.smithi045.stdout:mytag123 2024-05-08T05:51:29.448 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-05-08T05:51:29.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.554+0000 7fea25111640 1 Processor -- start 2024-05-08T05:51:29.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.554+0000 7fea25111640 1 -- start start 2024-05-08T05:51:29.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.554+0000 7fea25111640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea2006cab0 0x7fea200b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.554+0000 7fea25111640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea201604b0 con 0x7fea2015f0c0 2024-05-08T05:51:29.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea25111640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea200b14e0 con 0x7fea201638e0 2024-05-08T05:51:29.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea25111640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea20163450 con 0x7fea2006cab0 2024-05-08T05:51:29.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1ed9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fea2015f0c0 0x7fea200b0d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49660/0 (socket says 172.21.15.45:49660) 2024-05-08T05:51:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1ed9d640 1 -- 172.21.15.45:0/3093285518 learned_addr learned my addr 172.21.15.45:0/3093285518 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1f59e640 1 --2- 172.21.15.45:0/3093285518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea2006cab0 0x7fea200b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1e59c640 1 -- 172.21.15.45:0/3093285518 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 116026458 0 0) 0x7fea201604b0 con 0x7fea2015f0c0 2024-05-08T05:51:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1e59c640 1 -- 172.21.15.45:0/3093285518 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea0c003520 con 0x7fea2015f0c0 2024-05-08T05:51:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1e59c640 1 -- 172.21.15.45:0/3093285518 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 300711665 0 0) 0x7fea200b14e0 con 0x7fea201638e0 2024-05-08T05:51:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1e59c640 1 -- 172.21.15.45:0/3093285518 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea201604b0 con 0x7fea201638e0 2024-05-08T05:51:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1e59c640 1 -- 172.21.15.45:0/3093285518 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1067221764 0 0) 0x7fea0c003520 con 0x7fea2015f0c0 2024-05-08T05:51:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1e59c640 1 -- 172.21.15.45:0/3093285518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea2006cab0 msgr2=0x7fea200b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1e59c640 1 --2- 172.21.15.45:0/3093285518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea2006cab0 0x7fea200b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1e59c640 1 -- 172.21.15.45:0/3093285518 >> v1:172.21.15.63:6789/0 conn(0x7fea201638e0 legacy=0x7fea2006c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1e59c640 1 -- 172.21.15.45:0/3093285518 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea20162e00 con 0x7fea2015f0c0 2024-05-08T05:51:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1e59c640 1 -- 172.21.15.45:0/3093285518 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7fea100016f0 con 0x7fea2015f0c0 2024-05-08T05:51:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1e59c640 1 -- 172.21.15.45:0/3093285518 >> v1:172.21.15.45:6789/0 conn(0x7fea2015f0c0 legacy=0x7fea200b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1e59c640 1 --2- 172.21.15.45:0/3093285518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea0c003c10 0x7fea0c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1e59c640 1 --2- 172.21.15.45:0/3093285518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea2006cab0 0x7fea0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1e59c640 1 --2- 172.21.15.45:0/3093285518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea0c004d40 0x7fea0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1e59c640 1 -- 172.21.15.45:0/3093285518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea20163450 con 0x7fea0c004d40 2024-05-08T05:51:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1e59c640 1 -- 172.21.15.45:0/3093285518 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c00bb50 con 0x7fea0c003c10 2024-05-08T05:51:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1e59c640 1 -- 172.21.15.45:0/3093285518 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea0c00bd50 con 0x7fea2006cab0 2024-05-08T05:51:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.555+0000 7fea1f59e640 1 --2- 172.21.15.45:0/3093285518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea0c004d40 0x7fea0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.556+0000 7fea1ed9d640 1 --2- 172.21.15.45:0/3093285518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea0c003c10 0x7fea0c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.556+0000 7fea17fff640 1 --2- 172.21.15.45:0/3093285518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea2006cab0 0x7fea0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.556+0000 7fea1f59e640 1 -- 172.21.15.45:0/3093285518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea2006cab0 msgr2=0x7fea0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.556+0000 7fea1f59e640 1 --2- 172.21.15.45:0/3093285518 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea2006cab0 0x7fea0c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.556+0000 7fea1f59e640 1 -- 172.21.15.45:0/3093285518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea0c003c10 msgr2=0x7fea0c004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.556+0000 7fea1f59e640 1 --2- 172.21.15.45:0/3093285518 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea0c003c10 0x7fea0c004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.556+0000 7fea1f59e640 1 -- 172.21.15.45:0/3093285518 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0c003520 con 0x7fea0c004d40 2024-05-08T05:51:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.556+0000 7fea1f59e640 1 --2- 172.21.15.45:0/3093285518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea0c004d40 0x7fea0c0051c0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fea18009f90 tx=0x7fea0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4afa1fd5ac138b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.557+0000 7fea1e59c640 1 -- 172.21.15.45:0/3093285518 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18009c10 con 0x7fea0c004d40 2024-05-08T05:51:29.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.557+0000 7fea1e59c640 1 -- 172.21.15.45:0/3093285518 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea18036070 con 0x7fea0c004d40 2024-05-08T05:51:29.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.557+0000 7fea1e59c640 1 -- 172.21.15.45:0/3093285518 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18031850 con 0x7fea0c004d40 2024-05-08T05:51:29.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.557+0000 7fea25111640 1 -- 172.21.15.45:0/3093285518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea0c004d40 msgr2=0x7fea0c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.557+0000 7fea25111640 1 --2- 172.21.15.45:0/3093285518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea0c004d40 0x7fea0c0051c0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fea18009f90 tx=0x7fea0c005830 comp rx=0 tx=0).stop 2024-05-08T05:51:29.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.557+0000 7fea1f59e640 1 -- 172.21.15.45:0/3093285518 reap_dead start 2024-05-08T05:51:29.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.557+0000 7fea25111640 1 -- 172.21.15.45:0/3093285518 shutdown_connections 2024-05-08T05:51:29.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.557+0000 7fea25111640 1 -- 172.21.15.45:0/3093285518 >> 172.21.15.45:0/3093285518 conn(0x7fea200b3b10 msgr2=0x7fea200b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:29.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.557+0000 7fea25111640 1 -- 172.21.15.45:0/3093285518 shutdown_connections 2024-05-08T05:51:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.557+0000 7fea25111640 1 -- 172.21.15.45:0/3093285518 wait complete. 2024-05-08T05:51:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.558+0000 7fea25111640 1 Processor -- start 2024-05-08T05:51:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.559+0000 7fea25111640 1 -- start start 2024-05-08T05:51:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.559+0000 7fea25111640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea0c004d40 0x7fea200b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.559+0000 7fea25111640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea2015c290 0x7fea2015c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.559+0000 7fea25111640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea200ca6f0 0x7fea200cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.559+0000 7fea25111640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea200b6f40 con 0x7fea200ca6f0 2024-05-08T05:51:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.559+0000 7fea25111640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea200b17c0 con 0x7fea0c004d40 2024-05-08T05:51:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.559+0000 7fea25111640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea200b70c0 con 0x7fea2015c290 2024-05-08T05:51:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.559+0000 7fea1f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea200ca6f0 0x7fea200cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.559+0000 7fea1f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea200ca6f0 0x7fea200cab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51474/0 (socket says 172.21.15.45:51474) 2024-05-08T05:51:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.559+0000 7fea1f59e640 1 -- 172.21.15.45:0/4157503262 learned_addr learned my addr 172.21.15.45:0/4157503262 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.560+0000 7fea1ed9d640 1 --2- 172.21.15.45:0/4157503262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea0c004d40 0x7fea200b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.560+0000 7fea17fff640 1 --2- 172.21.15.45:0/4157503262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea2015c290 0x7fea2015c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.560+0000 7fea1f59e640 1 -- 172.21.15.45:0/4157503262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea2015c290 msgr2=0x7fea2015c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.560+0000 7fea1f59e640 1 --2- 172.21.15.45:0/4157503262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea2015c290 0x7fea2015c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.560+0000 7fea1f59e640 1 -- 172.21.15.45:0/4157503262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea0c004d40 msgr2=0x7fea200b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.560+0000 7fea1f59e640 1 --2- 172.21.15.45:0/4157503262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea0c004d40 0x7fea200b00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.560+0000 7fea1f59e640 1 -- 172.21.15.45:0/4157503262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea200cb440 con 0x7fea200ca6f0 2024-05-08T05:51:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.560+0000 7fea1ed9d640 1 --2- 172.21.15.45:0/4157503262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea0c004d40 0x7fea200b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.560+0000 7fea1f59e640 1 --2- 172.21.15.45:0/4157503262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea200ca6f0 0x7fea200cab70 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fea18033040 tx=0x7fea18009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.560+0000 7fea177fe640 1 -- 172.21.15.45:0/4157503262 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18009c10 con 0x7fea200ca6f0 2024-05-08T05:51:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.560+0000 7fea177fe640 1 -- 172.21.15.45:0/4157503262 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea18003990 con 0x7fea200ca6f0 2024-05-08T05:51:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.560+0000 7fea25111640 1 -- 172.21.15.45:0/4157503262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea201f83c0 con 0x7fea200ca6f0 2024-05-08T05:51:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.560+0000 7fea177fe640 1 -- 172.21.15.45:0/4157503262 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18003d80 con 0x7fea200ca6f0 2024-05-08T05:51:29.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.560+0000 7fea25111640 1 -- 172.21.15.45:0/4157503262 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea201f8980 con 0x7fea200ca6f0 2024-05-08T05:51:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.562+0000 7fea177fe640 1 -- 172.21.15.45:0/4157503262 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fea18005d10 con 0x7fea200ca6f0 2024-05-08T05:51:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.562+0000 7fea177fe640 1 --2- 172.21.15.45:0/4157503262 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fea000762d0 0x7fea00078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.562+0000 7fea177fe640 1 -- 172.21.15.45:0/4157503262 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7fea180b8da0 con 0x7fea200ca6f0 2024-05-08T05:51:29.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.563+0000 7fea1ed9d640 1 --2- 172.21.15.45:0/4157503262 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fea000762d0 0x7fea00078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.563+0000 7fea27e21740 1 Processor -- start 2024-05-08T05:51:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.563+0000 7fea1ed9d640 1 --2- 172.21.15.45:0/4157503262 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fea000762d0 0x7fea00078790 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fea10001eb0 tx=0x7fea10001150 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.563+0000 7fea27e21740 1 -- start start 2024-05-08T05:51:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.564+0000 7fea27e21740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02dbeb160 0x55e02db9daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.564+0000 7fea27e21740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02dbce280 0x55e02dbcd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.564+0000 7fea27e21740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbcdb40 0x55e02dba2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.564+0000 7fea27e21740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e02db6a690 con 0x55e02dbcdb40 2024-05-08T05:51:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.564+0000 7fea27e21740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e02db07f30 con 0x55e02dbeb160 2024-05-08T05:51:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.564+0000 7fea27e21740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e02dac0b20 con 0x55e02dbce280 2024-05-08T05:51:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.564+0000 7fea1f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbcdb40 0x55e02dba2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.564+0000 7fea1f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbcdb40 0x55e02dba2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51478/0 (socket says 172.21.15.45:51478) 2024-05-08T05:51:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.564+0000 7fea1f59e640 1 -- 172.21.15.45:0/2846236216 learned_addr learned my addr 172.21.15.45:0/2846236216 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:29.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.564+0000 7fea17fff640 1 --2- 172.21.15.45:0/2846236216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02dbce280 0x55e02dbcd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.564+0000 7fea1ed9d640 1 --2- 172.21.15.45:0/2846236216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02dbeb160 0x55e02db9daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.564+0000 7fea1f59e640 1 -- 172.21.15.45:0/2846236216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02dbce280 msgr2=0x55e02dbcd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.564+0000 7fea1f59e640 1 --2- 172.21.15.45:0/2846236216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02dbce280 0x55e02dbcd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.564+0000 7fea1f59e640 1 -- 172.21.15.45:0/2846236216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02dbeb160 msgr2=0x55e02db9daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.564+0000 7fea1f59e640 1 --2- 172.21.15.45:0/2846236216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02dbeb160 0x55e02db9daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.564+0000 7fea1f59e640 1 -- 172.21.15.45:0/2846236216 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e02dab8e60 con 0x55e02dbcdb40 2024-05-08T05:51:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.564+0000 7fea1ed9d640 1 --2- 172.21.15.45:0/2846236216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02dbeb160 0x55e02db9daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.564+0000 7fea1f59e640 1 --2- 172.21.15.45:0/2846236216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbcdb40 0x55e02dba2950 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fea1807de90 tx=0x7fea1807de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883153ba1c1cb902 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.565+0000 7fea14ff9640 1 -- 172.21.15.45:0/2846236216 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18081e10 con 0x55e02dbcdb40 2024-05-08T05:51:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.565+0000 7fea14ff9640 1 -- 172.21.15.45:0/2846236216 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea18036060 con 0x55e02dbcdb40 2024-05-08T05:51:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.565+0000 7fea14ff9640 1 -- 172.21.15.45:0/2846236216 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea180861c0 con 0x55e02dbcdb40 2024-05-08T05:51:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.565+0000 7fea27e21740 1 -- 172.21.15.45:0/2846236216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbcdb40 msgr2=0x55e02dba2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.565+0000 7fea27e21740 1 --2- 172.21.15.45:0/2846236216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbcdb40 0x55e02dba2950 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fea1807de90 tx=0x7fea1807de00 comp rx=0 tx=0).stop 2024-05-08T05:51:29.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.565+0000 7fea27e21740 1 -- 172.21.15.45:0/2846236216 shutdown_connections 2024-05-08T05:51:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.565+0000 7fea27e21740 1 --2- 172.21.15.45:0/2846236216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02dbce280 0x55e02dbcd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.565+0000 7fea27e21740 1 --2- 172.21.15.45:0/2846236216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbcdb40 0x55e02dba2950 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.565+0000 7fea27e21740 1 --2- 172.21.15.45:0/2846236216 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02dbeb160 0x55e02db9daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.565+0000 7fea27e21740 1 -- 172.21.15.45:0/2846236216 >> 172.21.15.45:0/2846236216 conn(0x55e02db9fd60 msgr2=0x55e02dbe8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.565+0000 7fea27e21740 1 -- 172.21.15.45:0/2846236216 shutdown_connections 2024-05-08T05:51:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.565+0000 7fea27e21740 1 -- 172.21.15.45:0/2846236216 wait complete. 2024-05-08T05:51:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.566+0000 7fea27e21740 1 -- start start 2024-05-08T05:51:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.566+0000 7fea27e21740 1 Processor -- start 2024-05-08T05:51:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.567+0000 7fea27e21740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02dbce280 0x55e02dbc8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.567+0000 7fea27e21740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbeb160 0x55e02dba2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.567+0000 7fea27e21740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02dbc8c50 0x55e02dba2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.567+0000 7fea27e21740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e02dac45e0 con 0x55e02dbeb160 2024-05-08T05:51:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.567+0000 7fea27e21740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55e02db03ef0 con 0x55e02dbce280 2024-05-08T05:51:29.573 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.567+0000 7fea27e21740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e02dac0b20 con 0x55e02dbc8c50 2024-05-08T05:51:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.567+0000 7fea27e21740 10 client.0 _ll_register_callbacks cb 0x55e02dabf320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:51:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.567+0000 7fea17fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbeb160 0x55e02dba2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.567+0000 7fea17fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbeb160 0x55e02dba2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51484/0 (socket says 172.21.15.45:51484) 2024-05-08T05:51:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.567+0000 7fea17fff640 1 -- 172.21.15.45:0/1621890646 learned_addr learned my addr 172.21.15.45:0/1621890646 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.567+0000 7fea1ed9d640 1 --2- 172.21.15.45:0/1621890646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02dbce280 0x55e02dbc8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.567+0000 7fea1f59e640 1 --2- 172.21.15.45:0/1621890646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02dbc8c50 0x55e02dba2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.567+0000 7fea17fff640 1 -- 172.21.15.45:0/1621890646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02dbc8c50 msgr2=0x55e02dba2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.567+0000 7fea17fff640 1 --2- 172.21.15.45:0/1621890646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02dbc8c50 0x55e02dba2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.567+0000 7fea17fff640 1 -- 172.21.15.45:0/1621890646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02dbce280 msgr2=0x55e02dbc8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.567+0000 7fea17fff640 1 --2- 172.21.15.45:0/1621890646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02dbce280 0x55e02dbc8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.567+0000 7fea17fff640 1 -- 172.21.15.45:0/1621890646 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e02d8e37d0 con 0x55e02dbeb160 2024-05-08T05:51:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.567+0000 7fea17fff640 1 --2- 172.21.15.45:0/1621890646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbeb160 0x55e02dba2840 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fea08003c40 tx=0x7fea08001830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50a068df64355f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.567+0000 7fe9fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:51:29.574 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.568+0000 7fe9fdffb640 1 -- 172.21.15.45:0/1621890646 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08015470 con 0x55e02dbeb160 2024-05-08T05:51:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.568+0000 7fe9fdffb640 1 -- 172.21.15.45:0/1621890646 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7fea08015610 con 0x55e02dbeb160 2024-05-08T05:51:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.568+0000 7fea27e21740 1 -- 172.21.15.45:0/1621890646 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea18036060 con 0x55e02dbeb160 2024-05-08T05:51:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.568+0000 7fe9fdffb640 1 -- 172.21.15.45:0/1621890646 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7fea08022050 con 0x55e02dbeb160 2024-05-08T05:51:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.568+0000 7fea27e21740 10 client.8442 fetch_fsmap learned FSMap version 106 2024-05-08T05:51:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.568+0000 7fea27e21740 1 -- 172.21.15.45:0/1621890646 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) -- 0x55e02d8193f0 con 0x55e02dbeb160 2024-05-08T05:51:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.568+0000 7fe9fdffb640 1 -- 172.21.15.45:0/1621890646 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea18036060 con 0x55e02dbeb160 2024-05-08T05:51:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.568+0000 7fe9fdffb640 1 -- 172.21.15.45:0/1621890646 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 106) ==== 3494+0+0 (secure 0 0 0) 0x7fea08025020 con 0x55e02dbeb160 2024-05-08T05:51:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.568+0000 7fea27e21740 10 client.8442 fetch_fsmap finished waiting for FSMap version 106 2024-05-08T05:51:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.568+0000 7fea27e21740 10 client.8442 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-05-08T05:51:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.568+0000 7fea27e21740 20 client.8442 populate_metadata read hostname 'smithi045' 2024-05-08T05:51:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.568+0000 7fea27e21740 1 --2- 172.21.15.45:0/1621890646 >> [v2:172.21.15.45:6834/453198328,v1:172.21.15.45:6835/453198328] conn(0x55e02dc029b0 0x55e02dc04da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.568+0000 7fea27e21740 4 client.8442 mds_command: new command op to 8319 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:51:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.568+0000 7fea27e21740 1 -- 172.21.15.45:0/1621890646 --> [v2:172.21.15.45:6834/453198328,v1:172.21.15.45:6835/453198328] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e02db15b40 con 0x55e02dc029b0 2024-05-08T05:51:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.568+0000 7fea1ed9d640 1 --2- 172.21.15.45:0/1621890646 >> [v2:172.21.15.45:6834/453198328,v1:172.21.15.45:6835/453198328] conn(0x55e02dc029b0 0x55e02dc04da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.568+0000 7fea1ed9d640 1 --2- 172.21.15.45:0/1621890646 >> [v2:172.21.15.45:6834/453198328,v1:172.21.15.45:6835/453198328] conn(0x55e02dc029b0 0x55e02dc04da0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=693351b8aba8bcfc server_cookie=8f04dabf5359981a in_seq=0 out_seq=0 2024-05-08T05:51:29.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.568+0000 7fe9fdffb640 10 client.8442 ms_handle_connect on v2:172.21.15.45:6834/453198328 2024-05-08T05:51:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.569+0000 7fe9fdffb640 1 -- 172.21.15.45:0/1621890646 <== mds.0 v2:172.21.15.45:6834/453198328 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e02db15b40 con 0x55e02dc029b0 2024-05-08T05:51:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.569+0000 7fe9fdffb640 10 client.8442 handle_command_reply: tid=0 2024-05-08T05:51:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.569+0000 7fea27e21740 1 client.8442 shutdown 2024-05-08T05:51:29.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.569+0000 7fea27e21740 1 -- 172.21.15.45:0/1621890646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbeb160 msgr2=0x55e02dba2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.569+0000 7fea27e21740 1 --2- 172.21.15.45:0/1621890646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbeb160 0x55e02dba2840 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fea08003c40 tx=0x7fea08001830 comp rx=0 tx=0).stop 2024-05-08T05:51:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.569+0000 7fea27e21740 1 -- 172.21.15.45:0/1621890646 shutdown_connections 2024-05-08T05:51:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.569+0000 7fea27e21740 1 --2- 172.21.15.45:0/1621890646 >> [v2:172.21.15.45:6834/453198328,v1:172.21.15.45:6835/453198328] conn(0x55e02dc029b0 0x55e02dc04da0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.569+0000 7fea27e21740 1 --2- 172.21.15.45:0/1621890646 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02dbc8c50 0x55e02dba2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.569+0000 7fe9fdffb640 0 client.8442 ms_handle_reset on v2:172.21.15.45:6834/453198328 2024-05-08T05:51:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.569+0000 7fea27e21740 1 --2- 172.21.15.45:0/1621890646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbeb160 0x55e02dba2840 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.569+0000 7fea27e21740 1 --2- 172.21.15.45:0/1621890646 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02dbce280 0x55e02dbc8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.569+0000 7fea27e21740 1 -- 172.21.15.45:0/1621890646 >> 172.21.15.45:0/1621890646 conn(0x55e02dbcdb40 msgr2=0x55e02dba1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.569+0000 7fea27e21740 1 -- 172.21.15.45:0/1621890646 shutdown_connections 2024-05-08T05:51:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.569+0000 7fea27e21740 1 -- 172.21.15.45:0/1621890646 wait complete. 2024-05-08T05:51:29.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.569+0000 7fea27e21740 20 client.8442 trim_cache size 0 max 16384 2024-05-08T05:51:29.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.585+0000 7fea27e21740 1 Processor -- start 2024-05-08T05:51:29.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.585+0000 7fea27e21740 1 -- start start 2024-05-08T05:51:29.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.585+0000 7fea27e21740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02dbc8c50 0x55e02dba3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.585+0000 7fea27e21740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbce280 0x55e02db9d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.585+0000 7fea27e21740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02dbeb160 0x55e02db9d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.585+0000 7fea27e21740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea20202670 con 0x55e02dbce280 2024-05-08T05:51:29.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.585+0000 7fea27e21740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea20201690 con 0x55e02dbc8c50 2024-05-08T05:51:29.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.585+0000 7fea27e21740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fea20200a20 con 0x55e02dbeb160 2024-05-08T05:51:29.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.585+0000 7fea17fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbce280 0x55e02db9d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.585+0000 7fea17fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbce280 0x55e02db9d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51500/0 (socket says 172.21.15.45:51500) 2024-05-08T05:51:29.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.586+0000 7fea17fff640 1 -- 172.21.15.45:0/3344632920 learned_addr learned my addr 172.21.15.45:0/3344632920 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:29.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.586+0000 7fea17fff640 1 -- 172.21.15.45:0/3344632920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02dbeb160 msgr2=0x55e02db9d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:29.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.586+0000 7fea17fff640 1 --2- 172.21.15.45:0/3344632920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02dbeb160 0x55e02db9d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.586+0000 7fea17fff640 1 -- 172.21.15.45:0/3344632920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02dbc8c50 msgr2=0x55e02dba3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:29.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.586+0000 7fea17fff640 1 --2- 172.21.15.45:0/3344632920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02dbc8c50 0x55e02dba3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.586+0000 7fea17fff640 1 -- 172.21.15.45:0/3344632920 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e02dc05110 con 0x55e02dbce280 2024-05-08T05:51:29.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.586+0000 7fea17fff640 1 --2- 172.21.15.45:0/3344632920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbce280 0x55e02db9d2c0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fea08005470 tx=0x7fea0800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d596848fc5237 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.586+0000 7fe9fffff640 1 -- 172.21.15.45:0/3344632920 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08015670 con 0x55e02dbce280 2024-05-08T05:51:29.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.586+0000 7fe9fffff640 1 -- 172.21.15.45:0/3344632920 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea0801a070 con 0x55e02dbce280 2024-05-08T05:51:29.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.586+0000 7fe9fffff640 1 -- 172.21.15.45:0/3344632920 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08026630 con 0x55e02dbce280 2024-05-08T05:51:29.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.587+0000 7fea27e21740 1 -- 172.21.15.45:0/3344632920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbce280 msgr2=0x55e02db9d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.587+0000 7fea27e21740 1 --2- 172.21.15.45:0/3344632920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbce280 0x55e02db9d2c0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fea08005470 tx=0x7fea0800f000 comp rx=0 tx=0).stop 2024-05-08T05:51:29.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.587+0000 7fea27e21740 1 -- 172.21.15.45:0/3344632920 shutdown_connections 2024-05-08T05:51:29.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.587+0000 7fea27e21740 1 --2- 172.21.15.45:0/3344632920 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02dbeb160 0x55e02db9d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.587+0000 7fea27e21740 1 --2- 172.21.15.45:0/3344632920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbce280 0x55e02db9d2c0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.587+0000 7fea27e21740 1 --2- 172.21.15.45:0/3344632920 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02dbc8c50 0x55e02dba3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.587+0000 7fea27e21740 1 -- 172.21.15.45:0/3344632920 >> 172.21.15.45:0/3344632920 conn(0x55e02dbcdb40 msgr2=0x55e02dbc84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:29.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.587+0000 7fea27e21740 1 -- 172.21.15.45:0/3344632920 shutdown_connections 2024-05-08T05:51:29.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.587+0000 7fea27e21740 1 -- 172.21.15.45:0/3344632920 wait complete. 2024-05-08T05:51:29.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.587+0000 7fea27e21740 1 -- start start 2024-05-08T05:51:29.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.587+0000 7fea27e21740 1 Processor -- start 2024-05-08T05:51:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.588+0000 7fea27e21740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02dbc8c50 0x55e02dc65fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.588+0000 7fea27e21740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbce280 0x55e02dba26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.588+0000 7fea27e21740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02dbeb160 0x55e02dbe81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.588+0000 7fea27e21740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e02db6a690 con 0x55e02dbce280 2024-05-08T05:51:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.588+0000 7fea27e21740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fea20201040 con 0x55e02dbeb160 2024-05-08T05:51:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.588+0000 7fea27e21740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55e02db07f30 con 0x55e02dbc8c50 2024-05-08T05:51:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.588+0000 7fea27e21740 10 client.0 _ll_register_callbacks cb 0x7fea08003820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:51:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.588+0000 7fea17fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbce280 0x55e02dba26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.588+0000 7fea17fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbce280 0x55e02dba26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51510/0 (socket says 172.21.15.45:51510) 2024-05-08T05:51:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.588+0000 7fea17fff640 1 -- 172.21.15.45:0/3616488926 learned_addr learned my addr 172.21.15.45:0/3616488926 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.588+0000 7fea1ed9d640 1 --2- 172.21.15.45:0/3616488926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02dbc8c50 0x55e02dc65fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.588+0000 7fea1f59e640 1 --2- 172.21.15.45:0/3616488926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02dbeb160 0x55e02dbe81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.588+0000 7fea17fff640 1 -- 172.21.15.45:0/3616488926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02dbc8c50 msgr2=0x55e02dc65fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.588+0000 7fea17fff640 1 --2- 172.21.15.45:0/3616488926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02dbc8c50 0x55e02dc65fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.588+0000 7fea17fff640 1 -- 172.21.15.45:0/3616488926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02dbeb160 msgr2=0x55e02dbe81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.588+0000 7fea17fff640 1 --2- 172.21.15.45:0/3616488926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02dbeb160 0x55e02dbe81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.589+0000 7fea17fff640 1 -- 172.21.15.45:0/3616488926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e02dbc82f0 con 0x55e02dbce280 2024-05-08T05:51:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.589+0000 7fea17fff640 1 --2- 172.21.15.45:0/3616488926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbce280 0x55e02dba26f0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fea08003c40 tx=0x7fea080012f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba52fa8752328dfb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.589+0000 7fe9fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:51:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.589+0000 7fe9fdffb640 1 -- 172.21.15.45:0/3616488926 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0801fbf0 con 0x55e02dbce280 2024-05-08T05:51:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.589+0000 7fe9fdffb640 1 -- 172.21.15.45:0/3616488926 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7fea0801fd90 con 0x55e02dbce280 2024-05-08T05:51:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.589+0000 7fea27e21740 1 -- 172.21.15.45:0/3616488926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea0801a070 con 0x55e02dbce280 2024-05-08T05:51:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.589+0000 7fe9fdffb640 1 -- 172.21.15.45:0/3616488926 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=106) ==== 24+0+0 (secure 0 0 0) 0x7fea0803f050 con 0x55e02dbce280 2024-05-08T05:51:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.589+0000 7fea27e21740 10 client.8448 fetch_fsmap learned FSMap version 106 2024-05-08T05:51:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.589+0000 7fea27e21740 1 -- 172.21.15.45:0/3616488926 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=106,monmap=2+}) -- 0x55e02d7d90f0 con 0x55e02dbce280 2024-05-08T05:51:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.589+0000 7fe9fdffb640 1 -- 172.21.15.45:0/3616488926 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea0801a070 con 0x55e02dbce280 2024-05-08T05:51:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.589+0000 7fe9fdffb640 1 -- 172.21.15.45:0/3616488926 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 106) ==== 3494+0+0 (secure 0 0 0) 0x7fea08042020 con 0x55e02dbce280 2024-05-08T05:51:29.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.589+0000 7fea27e21740 10 client.8448 fetch_fsmap finished waiting for FSMap version 106 2024-05-08T05:51:29.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.589+0000 7fea27e21740 10 client.8448 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-05-08T05:51:29.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.589+0000 7fea27e21740 20 client.8448 populate_metadata read hostname 'smithi045' 2024-05-08T05:51:29.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.589+0000 7fea27e21740 1 --2- 172.21.15.45:0/3616488926 >> [v2:172.21.15.45:6834/453198328,v1:172.21.15.45:6835/453198328] conn(0x55e02dba2c30 0x55e02dbe86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.589+0000 7fea27e21740 4 client.8448 mds_command: new command op to 8319 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:51:29.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.589+0000 7fea27e21740 1 -- 172.21.15.45:0/3616488926 --> [v2:172.21.15.45:6834/453198328,v1:172.21.15.45:6835/453198328] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e02db0f9b0 con 0x55e02dba2c30 2024-05-08T05:51:29.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.589+0000 7fea1ed9d640 1 --2- 172.21.15.45:0/3616488926 >> [v2:172.21.15.45:6834/453198328,v1:172.21.15.45:6835/453198328] conn(0x55e02dba2c30 0x55e02dbe86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.590+0000 7fe9fdffb640 10 client.8448 ms_handle_connect on v2:172.21.15.45:6834/453198328 2024-05-08T05:51:29.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.590+0000 7fea1ed9d640 1 --2- 172.21.15.45:0/3616488926 >> [v2:172.21.15.45:6834/453198328,v1:172.21.15.45:6835/453198328] conn(0x55e02dba2c30 0x55e02dbe86e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cd704805d777b11 server_cookie=ab8e516a6acfc573 in_seq=0 out_seq=0 2024-05-08T05:51:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.600+0000 7fe9fdffb640 1 -- 172.21.15.45:0/3616488926 <== mds.0 v2:172.21.15.45:6834/453198328 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e02db0f9b0 con 0x55e02dba2c30 2024-05-08T05:51:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.600+0000 7fe9fdffb640 10 client.8448 handle_command_reply: tid=0 2024-05-08T05:51:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.600+0000 7fea27e21740 1 client.8448 shutdown 2024-05-08T05:51:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.600+0000 7fea27e21740 1 -- 172.21.15.45:0/3616488926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbce280 msgr2=0x55e02dba26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.600+0000 7fea27e21740 1 --2- 172.21.15.45:0/3616488926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbce280 0x55e02dba26f0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fea08003c40 tx=0x7fea080012f0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fea27e21740 1 -- 172.21.15.45:0/3616488926 shutdown_connections 2024-05-08T05:51:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fea27e21740 1 --2- 172.21.15.45:0/3616488926 >> [v2:172.21.15.45:6834/453198328,v1:172.21.15.45:6835/453198328] conn(0x55e02dba2c30 0x55e02dbe86e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fea27e21740 1 --2- 172.21.15.45:0/3616488926 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55e02dbeb160 0x55e02dbe81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fea27e21740 1 --2- 172.21.15.45:0/3616488926 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e02dbce280 0x55e02dba26f0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fe9fdffb640 0 client.8448 ms_handle_reset on v2:172.21.15.45:6834/453198328 2024-05-08T05:51:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fea27e21740 1 --2- 172.21.15.45:0/3616488926 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e02dbc8c50 0x55e02dc65fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fea27e21740 1 -- 172.21.15.45:0/3616488926 >> 172.21.15.45:0/3616488926 conn(0x55e02dbcdb40 msgr2=0x55e02dc0a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fea27e21740 1 -- 172.21.15.45:0/3616488926 shutdown_connections 2024-05-08T05:51:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fea27e21740 1 -- 172.21.15.45:0/3616488926 wait complete. 2024-05-08T05:51:29.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fea27e21740 20 client.8448 trim_cache size 0 max 16384 2024-05-08T05:51:29.602 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:51:29.602 INFO:teuthology.orchestra.run.smithi045.stdout: "message": "", 2024-05-08T05:51:29.603 INFO:teuthology.orchestra.run.smithi045.stdout: "return_code": 0 2024-05-08T05:51:29.603 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:51:29.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fea25111640 1 -- 172.21.15.45:0/4157503262 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fea000762d0 msgr2=0x7fea00078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fea25111640 1 --2- 172.21.15.45:0/4157503262 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fea000762d0 0x7fea00078790 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fea10001eb0 tx=0x7fea10001150 comp rx=0 tx=0).stop 2024-05-08T05:51:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fea25111640 1 -- 172.21.15.45:0/4157503262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea200ca6f0 msgr2=0x7fea200cab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fea25111640 1 --2- 172.21.15.45:0/4157503262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea200ca6f0 0x7fea200cab70 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fea18033040 tx=0x7fea18009750 comp rx=0 tx=0).stop 2024-05-08T05:51:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fea25111640 1 -- 172.21.15.45:0/4157503262 shutdown_connections 2024-05-08T05:51:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fea25111640 1 --2- 172.21.15.45:0/4157503262 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fea000762d0 0x7fea00078790 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fea25111640 1 --2- 172.21.15.45:0/4157503262 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fea2015c290 0x7fea2015c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fea25111640 1 --2- 172.21.15.45:0/4157503262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea200ca6f0 0x7fea200cab70 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fea25111640 1 --2- 172.21.15.45:0/4157503262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fea0c004d40 0x7fea200b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fea25111640 1 -- 172.21.15.45:0/4157503262 >> 172.21.15.45:0/4157503262 conn(0x7fea200b3b10 msgr2=0x7fea200d2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.601+0000 7fea25111640 1 -- 172.21.15.45:0/4157503262 shutdown_connections 2024-05-08T05:51:29.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.602+0000 7fea25111640 1 -- 172.21.15.45:0/4157503262 wait complete. 2024-05-08T05:51:29.612 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T05:51:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.718+0000 7f2955baf640 1 Processor -- start 2024-05-08T05:51:29.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.718+0000 7f2955baf640 1 -- start start 2024-05-08T05:51:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.719+0000 7f2955baf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29501629a0 0x7f29500d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.719+0000 7f2955baf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29500d0cb0 0x7f29500d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.719+0000 7f2955baf640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29500d3330 con 0x7f29500d2dc0 2024-05-08T05:51:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.719+0000 7f2955baf640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f29500d2170 con 0x7f29501629a0 2024-05-08T05:51:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.719+0000 7f2955baf640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f29500d2450 con 0x7f29500d0cb0 2024-05-08T05:51:29.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.719+0000 7f294f7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29501629a0 0x7f29500d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.719+0000 7f294f7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f29501629a0 0x7f29500d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51522/0 (socket says 172.21.15.45:51522) 2024-05-08T05:51:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.719+0000 7f294effd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f29500d2dc0 0x7f29500c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49668/0 (socket says 172.21.15.45:49668) 2024-05-08T05:51:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.719+0000 7f294f7fe640 1 -- 172.21.15.45:0/1037662529 learned_addr learned my addr 172.21.15.45:0/1037662529 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.719+0000 7f294ffff640 1 --2- 172.21.15.45:0/1037662529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29500d0cb0 0x7f29500d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.719+0000 7f294f7fe640 1 -- 172.21.15.45:0/1037662529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29500d0cb0 msgr2=0x7f29500d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.719+0000 7f294f7fe640 1 --2- 172.21.15.45:0/1037662529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29500d0cb0 0x7f29500d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.719+0000 7f294f7fe640 1 -- 172.21.15.45:0/1037662529 >> v1:172.21.15.45:6789/0 conn(0x7f29500d2dc0 legacy=0x7f29500c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.719+0000 7f294f7fe640 1 -- 172.21.15.45:0/1037662529 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29500d2730 con 0x7f29501629a0 2024-05-08T05:51:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.719+0000 7f294e7fc640 1 -- 172.21.15.45:0/1037662529 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2208465477 0 0) 0x7f29500d3330 con 0x7f29500d2dc0 2024-05-08T05:51:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.719+0000 7f294ffff640 1 --2- 172.21.15.45:0/1037662529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29500d0cb0 0x7f29500d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.720+0000 7f294f7fe640 1 --2- 172.21.15.45:0/1037662529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29501629a0 0x7f29500d35c0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f2940009870 tx=0x7f294002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f553d870d202f67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.720+0000 7f294e7fc640 1 -- 172.21.15.45:0/1037662529 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940037070 con 0x7f29501629a0 2024-05-08T05:51:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.720+0000 7f294e7fc640 1 -- 172.21.15.45:0/1037662529 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2940032440 con 0x7f29501629a0 2024-05-08T05:51:29.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.720+0000 7f294e7fc640 1 -- 172.21.15.45:0/1037662529 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940032810 con 0x7f29501629a0 2024-05-08T05:51:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.721+0000 7f2955baf640 1 -- 172.21.15.45:0/1037662529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29501629a0 msgr2=0x7f29500d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.721+0000 7f2955baf640 1 --2- 172.21.15.45:0/1037662529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29501629a0 0x7f29500d35c0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f2940009870 tx=0x7f294002a130 comp rx=0 tx=0).stop 2024-05-08T05:51:29.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.721+0000 7f2955baf640 1 -- 172.21.15.45:0/1037662529 shutdown_connections 2024-05-08T05:51:29.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.721+0000 7f2955baf640 1 --2- 172.21.15.45:0/1037662529 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29500d0cb0 0x7f29500d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.721+0000 7f2955baf640 1 --2- 172.21.15.45:0/1037662529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29501629a0 0x7f29500d35c0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.721+0000 7f2955baf640 1 -- 172.21.15.45:0/1037662529 >> 172.21.15.45:0/1037662529 conn(0x7f29500d4fe0 msgr2=0x7f29500d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:29.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.721+0000 7f2955baf640 1 -- 172.21.15.45:0/1037662529 shutdown_connections 2024-05-08T05:51:29.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.721+0000 7f2955baf640 1 -- 172.21.15.45:0/1037662529 wait complete. 2024-05-08T05:51:29.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.722+0000 7f2955baf640 1 Processor -- start 2024-05-08T05:51:29.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.722+0000 7f2955baf640 1 -- start start 2024-05-08T05:51:29.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.723+0000 7f2955baf640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29500d0cb0 0x7f29500ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.723+0000 7f2955baf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29500d2dc0 0x7f29501f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.723+0000 7f2955baf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29501629a0 0x7f29500b97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.723+0000 7f2955baf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f29500cf440 con 0x7f29500d2dc0 2024-05-08T05:51:29.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.723+0000 7f2955baf640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f29500cf2c0 con 0x7f29500d0cb0 2024-05-08T05:51:29.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.723+0000 7f2955baf640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f29500c95e0 con 0x7f29501629a0 2024-05-08T05:51:29.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.723+0000 7f294effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29500d2dc0 0x7f29501f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.723+0000 7f294effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29500d2dc0 0x7f29501f4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51524/0 (socket says 172.21.15.45:51524) 2024-05-08T05:51:29.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.723+0000 7f294effd640 1 -- 172.21.15.45:0/3863650987 learned_addr learned my addr 172.21.15.45:0/3863650987 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:29.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.723+0000 7f294f7fe640 1 --2- 172.21.15.45:0/3863650987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29500d0cb0 0x7f29500ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.723+0000 7f294ffff640 1 --2- 172.21.15.45:0/3863650987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29501629a0 0x7f29500b97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.723+0000 7f294effd640 1 -- 172.21.15.45:0/3863650987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29501629a0 msgr2=0x7f29500b97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.723+0000 7f294effd640 1 --2- 172.21.15.45:0/3863650987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29501629a0 0x7f29500b97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.723+0000 7f294effd640 1 -- 172.21.15.45:0/3863650987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29500d0cb0 msgr2=0x7f29500ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.723+0000 7f294effd640 1 --2- 172.21.15.45:0/3863650987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29500d0cb0 0x7f29500ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.723+0000 7f294effd640 1 -- 172.21.15.45:0/3863650987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29501f51a0 con 0x7f29500d2dc0 2024-05-08T05:51:29.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.723+0000 7f294f7fe640 1 --2- 172.21.15.45:0/3863650987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29500d0cb0 0x7f29500ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:29.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.724+0000 7f294effd640 1 --2- 172.21.15.45:0/3863650987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29500d2dc0 0x7f29501f4c60 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f294400c490 tx=0x7f294400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.724+0000 7f2937fff640 1 -- 172.21.15.45:0/3863650987 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944018070 con 0x7f29500d2dc0 2024-05-08T05:51:29.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.724+0000 7f2937fff640 1 -- 172.21.15.45:0/3863650987 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2944007500 con 0x7f29500d2dc0 2024-05-08T05:51:29.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.724+0000 7f2955baf640 1 -- 172.21.15.45:0/3863650987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29501f5430 con 0x7f29500d2dc0 2024-05-08T05:51:29.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.724+0000 7f2937fff640 1 -- 172.21.15.45:0/3863650987 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944004a20 con 0x7f29500d2dc0 2024-05-08T05:51:29.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.724+0000 7f2955baf640 1 -- 172.21.15.45:0/3863650987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29501f59f0 con 0x7f29500d2dc0 2024-05-08T05:51:29.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.725+0000 7f2955baf640 1 -- 172.21.15.45:0/3863650987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29500693f0 con 0x7f29500d2dc0 2024-05-08T05:51:29.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.725+0000 7f2937fff640 1 -- 172.21.15.45:0/3863650987 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f294401b070 con 0x7f29500d2dc0 2024-05-08T05:51:29.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.726+0000 7f2937fff640 1 --2- 172.21.15.45:0/3863650987 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f293c076300 0x7f293c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.726+0000 7f2937fff640 1 -- 172.21.15.45:0/3863650987 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6187+0+0 (secure 0 0 0) 0x7f29440969e0 con 0x7f29500d2dc0 2024-05-08T05:51:29.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.726+0000 7f294f7fe640 1 --2- 172.21.15.45:0/3863650987 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f293c076300 0x7f293c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.727+0000 7f294f7fe640 1 --2- 172.21.15.45:0/3863650987 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f293c076300 0x7f293c0787c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2940034000 tx=0x7f294002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.730+0000 7f2937fff640 1 -- 172.21.15.45:0/3863650987 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f29440601f0 con 0x7f29500d2dc0 2024-05-08T05:51:29.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.961+0000 7f2955baf640 1 -- 172.21.15.45:0/3863650987 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f29500cb830 con 0x7f29500d2dc0 2024-05-08T05:51:29.961 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-05-08T05:51:29.961+0000 7f124de12640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T05:51:29.963 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.963+0000 7f2937fff640 1 -- 172.21.15.45:0/3863650987 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v107) ==== 161+0+0 (secure 0 0 0) 0x7f294405faf0 con 0x7f29500d2dc0 2024-05-08T05:51:29.963 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T05:51:29.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.966+0000 7f2955baf640 1 -- 172.21.15.45:0/3863650987 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f293c076300 msgr2=0x7f293c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.966+0000 7f2955baf640 1 --2- 172.21.15.45:0/3863650987 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f293c076300 0x7f293c0787c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2940034000 tx=0x7f294002e000 comp rx=0 tx=0).stop 2024-05-08T05:51:29.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.966+0000 7f2955baf640 1 -- 172.21.15.45:0/3863650987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29500d2dc0 msgr2=0x7f29501f4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.966+0000 7f2955baf640 1 --2- 172.21.15.45:0/3863650987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29500d2dc0 0x7f29501f4c60 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f294400c490 tx=0x7f294400c950 comp rx=0 tx=0).stop 2024-05-08T05:51:29.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.966+0000 7f2955baf640 1 -- 172.21.15.45:0/3863650987 shutdown_connections 2024-05-08T05:51:29.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.966+0000 7f2955baf640 1 --2- 172.21.15.45:0/3863650987 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f293c076300 0x7f293c0787c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.966+0000 7f2955baf640 1 --2- 172.21.15.45:0/3863650987 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f29501629a0 0x7f29500b97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.966+0000 7f2955baf640 1 --2- 172.21.15.45:0/3863650987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f29500d2dc0 0x7f29501f4c60 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.966+0000 7f2955baf640 1 --2- 172.21.15.45:0/3863650987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f29500d0cb0 0x7f29500ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.966+0000 7f2955baf640 1 -- 172.21.15.45:0/3863650987 >> 172.21.15.45:0/3863650987 conn(0x7f29500d4fe0 msgr2=0x7f29500d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:29.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.966+0000 7f2955baf640 1 -- 172.21.15.45:0/3863650987 shutdown_connections 2024-05-08T05:51:29.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:29.966+0000 7f2955baf640 1 -- 172.21.15.45:0/3863650987 wait complete. 2024-05-08T05:51:29.968 INFO:tasks.ceph.mds.c.smithi045.stderr: -1672> 2024-05-08T05:51:26.660+0000 7f4d07da5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:51:29.978 DEBUG:teuthology.orchestra.run.smithi045:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force --yes-i-really-really-mean-it 2024-05-08T05:51:29.995 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-05-08T05:51:30.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.023+0000 7f96cbb17280 1 -- start start 2024-05-08T05:51:30.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.023+0000 7f96cbb17280 1 Processor -- start 2024-05-08T05:51:30.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.024+0000 7f96cbb17280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648276ca400 0x5648276f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.024+0000 7f96cbb17280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56482692dc00 0x5648276f7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.024+0000 7f96cbb17280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5648276ca800 0x5648276f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.024+0000 7f96cbb17280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564827700000 con 0x5648276ca400 2024-05-08T05:51:30.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.024+0000 7f96cbb17280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564827700180 con 0x56482692dc00 2024-05-08T05:51:30.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.024+0000 7f96cbb17280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564827700300 con 0x5648276ca800 2024-05-08T05:51:30.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.024+0000 7f96ca2c5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648276ca400 0x5648276f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.024+0000 7f96ca2c5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648276ca400 0x5648276f6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51552/0 (socket says 172.21.15.45:51552) 2024-05-08T05:51:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.024+0000 7f96ca2c5640 1 -- 172.21.15.45:0/4082852475 learned_addr learned my addr 172.21.15.45:0/4082852475 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.024+0000 7f96c9ac4640 1 --2- 172.21.15.45:0/4082852475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56482692dc00 0x5648276f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.024+0000 7f96caac6640 1 --2- 172.21.15.45:0/4082852475 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5648276ca800 0x5648276f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.024+0000 7f96ca2c5640 1 -- 172.21.15.45:0/4082852475 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5648276ca800 msgr2=0x5648276f6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.025+0000 7f96ca2c5640 1 --2- 172.21.15.45:0/4082852475 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5648276ca800 0x5648276f6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.025+0000 7f96ca2c5640 1 -- 172.21.15.45:0/4082852475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56482692dc00 msgr2=0x5648276f7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.025+0000 7f96ca2c5640 1 --2- 172.21.15.45:0/4082852475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56482692dc00 0x5648276f7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.025+0000 7f96ca2c5640 1 -- 172.21.15.45:0/4082852475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564826962780 con 0x5648276ca400 2024-05-08T05:51:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.025+0000 7f96c9ac4640 1 --2- 172.21.15.45:0/4082852475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56482692dc00 0x5648276f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.025+0000 7f96ca2c5640 1 --2- 172.21.15.45:0/4082852475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648276ca400 0x5648276f6b00 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x56482774f7d0 tx=0x5648276ce180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3412527670e66 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.025+0000 7f96c7ac0640 1 -- 172.21.15.45:0/4082852475 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564826952d00 con 0x5648276ca400 2024-05-08T05:51:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.025+0000 7f96c7ac0640 1 -- 172.21.15.45:0/4082852475 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564826958700 con 0x5648276ca400 2024-05-08T05:51:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.025+0000 7f96c7ac0640 1 -- 172.21.15.45:0/4082852475 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564826952680 con 0x5648276ca400 2024-05-08T05:51:30.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.025+0000 7f96cbb17280 1 -- 172.21.15.45:0/4082852475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564826962000 con 0x5648276ca400 2024-05-08T05:51:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.025+0000 7f96cbb17280 1 -- 172.21.15.45:0/4082852475 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=0}) -- 0x564826963680 con 0x5648276ca400 2024-05-08T05:51:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.025+0000 7f96cbb17280 4 waiting for MDS map... 2024-05-08T05:51:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.025+0000 7f96c7ac0640 1 -- 172.21.15.45:0/4082852475 <== mon.0 v2:172.21.15.45:3300/0 4 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x564827804600 con 0x5648276ca400 2024-05-08T05:51:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.025+0000 7f96c7ac0640 1 -- 172.21.15.45:0/4082852475 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 107) ==== 2885+0+0 (secure 0 0 0) 0x564826987180 con 0x5648276ca400 2024-05-08T05:51:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.025+0000 7f96cbb17280 4 Got MDS map 107 2024-05-08T05:51:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.025+0000 7f96cbb17280 10 main: JournalTool::main 2024-05-08T05:51:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.025+0000 7f96cbb17280 4 main: JournalTool: connecting to RADOS... 2024-05-08T05:51:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.026+0000 7f96cbb17280 1 Processor -- start 2024-05-08T05:51:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.026+0000 7f96cbb17280 1 -- start start 2024-05-08T05:51:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.026+0000 7f96cbb17280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564827801c00 0x5648276f6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.026+0000 7f96cbb17280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564827874000 0x5648276f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.026+0000 7f96cbb17280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564827874400 0x5648276f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.026+0000 7f96cbb17280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564827700480 con 0x564827874400 2024-05-08T05:51:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.026+0000 7f96cbb17280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564827700600 con 0x564827874000 2024-05-08T05:51:30.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.026+0000 7f96cbb17280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564827700780 con 0x564827801c00 2024-05-08T05:51:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.027+0000 7f96c9ac4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564827874400 0x5648276f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.027+0000 7f96c9ac4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564827874400 0x5648276f7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51566/0 (socket says 172.21.15.45:51566) 2024-05-08T05:51:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.027+0000 7f96c9ac4640 1 -- 172.21.15.45:0/3393771240 learned_addr learned my addr 172.21.15.45:0/3393771240 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.027+0000 7f96ca2c5640 1 --2- 172.21.15.45:0/3393771240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564827874000 0x5648276f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.027+0000 7f96caac6640 1 --2- 172.21.15.45:0/3393771240 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564827801c00 0x5648276f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.027+0000 7f96c9ac4640 1 -- 172.21.15.45:0/3393771240 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564827801c00 msgr2=0x5648276f6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.027+0000 7f96c9ac4640 1 --2- 172.21.15.45:0/3393771240 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564827801c00 0x5648276f6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.027+0000 7f96c9ac4640 1 -- 172.21.15.45:0/3393771240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564827874000 msgr2=0x5648276f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.027+0000 7f96c9ac4640 1 --2- 172.21.15.45:0/3393771240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564827874000 0x5648276f7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.027+0000 7f96c9ac4640 1 -- 172.21.15.45:0/3393771240 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564826963860 con 0x564827874400 2024-05-08T05:51:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.027+0000 7f96c9ac4640 1 --2- 172.21.15.45:0/3393771240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564827874400 0x5648276f7b80 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x56482781c510 tx=0x5648276cc480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99835b8c97d3c51f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.027+0000 7f96c5abc640 1 -- 172.21.15.45:0/3393771240 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564826952d00 con 0x564827874400 2024-05-08T05:51:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.027+0000 7f96c5abc640 1 -- 172.21.15.45:0/3393771240 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564826958e00 con 0x564827874400 2024-05-08T05:51:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.027+0000 7f96c5abc640 1 -- 172.21.15.45:0/3393771240 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564826952680 con 0x564827874400 2024-05-08T05:51:30.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.028+0000 7f96cbb17280 1 -- 172.21.15.45:0/3393771240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564827874400 msgr2=0x5648276f7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.028+0000 7f96cbb17280 1 --2- 172.21.15.45:0/3393771240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564827874400 0x5648276f7b80 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x56482781c510 tx=0x5648276cc480 comp rx=0 tx=0).stop 2024-05-08T05:51:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.028+0000 7f96cbb17280 1 -- 172.21.15.45:0/3393771240 shutdown_connections 2024-05-08T05:51:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.028+0000 7f96cbb17280 1 --2- 172.21.15.45:0/3393771240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564827874400 0x5648276f7b80 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.028+0000 7f96cbb17280 1 --2- 172.21.15.45:0/3393771240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564827874000 0x5648276f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.028+0000 7f96cbb17280 1 --2- 172.21.15.45:0/3393771240 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x564827801c00 0x5648276f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.028+0000 7f96cbb17280 1 -- 172.21.15.45:0/3393771240 >> 172.21.15.45:0/3393771240 conn(0x5648276ca000 msgr2=0x564826a25000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.028+0000 7f96cbb17280 1 -- 172.21.15.45:0/3393771240 shutdown_connections 2024-05-08T05:51:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.028+0000 7f96cbb17280 1 -- 172.21.15.45:0/3393771240 wait complete. 2024-05-08T05:51:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.028+0000 7f96cbb17280 1 Processor -- start 2024-05-08T05:51:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.028+0000 7f96cbb17280 1 -- start start 2024-05-08T05:51:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.029+0000 7f96cbb17280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564827801c00 0x5648276f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.029+0000 7f96cbb17280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56482770c000 0x5648276f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.029+0000 7f96cbb17280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56482770c400 0x5648276f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.029+0000 7f96cbb17280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564827701c80 con 0x56482770c400 2024-05-08T05:51:30.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.029+0000 7f96cbb17280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564827701b00 con 0x564827801c00 2024-05-08T05:51:30.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.029+0000 7f96cbb17280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564827701980 con 0x56482770c000 2024-05-08T05:51:30.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.029+0000 7f96c9ac4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56482770c400 0x5648276f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.029+0000 7f96c9ac4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56482770c400 0x5648276f7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51568/0 (socket says 172.21.15.45:51568) 2024-05-08T05:51:30.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.029+0000 7f96c9ac4640 1 -- 172.21.15.45:0/358097808 learned_addr learned my addr 172.21.15.45:0/358097808 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:30.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.029+0000 7f96caac6640 1 --2- 172.21.15.45:0/358097808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564827801c00 0x5648276f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.029+0000 7f96ca2c5640 1 --2- 172.21.15.45:0/358097808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56482770c000 0x5648276f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.029+0000 7f96c9ac4640 1 -- 172.21.15.45:0/358097808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56482770c000 msgr2=0x5648276f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.029+0000 7f96c9ac4640 1 --2- 172.21.15.45:0/358097808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56482770c000 0x5648276f7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.029+0000 7f96c9ac4640 1 -- 172.21.15.45:0/358097808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564827801c00 msgr2=0x5648276f6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.029+0000 7f96c9ac4640 1 --2- 172.21.15.45:0/358097808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564827801c00 0x5648276f6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.029+0000 7f96c9ac4640 1 -- 172.21.15.45:0/358097808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648269634a0 con 0x56482770c400 2024-05-08T05:51:30.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.029+0000 7f96c9ac4640 1 --2- 172.21.15.45:0/358097808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56482770c400 0x5648276f7b80 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x56482781d740 tx=0x5648276ccae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.030+0000 7f96c42b9640 1 -- 172.21.15.45:0/358097808 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564826952820 con 0x56482770c400 2024-05-08T05:51:30.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.030+0000 7f96c42b9640 1 -- 172.21.15.45:0/358097808 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564826958c40 con 0x56482770c400 2024-05-08T05:51:30.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.030+0000 7f96cbb17280 1 -- 172.21.15.45:0/358097808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564826962960 con 0x56482770c400 2024-05-08T05:51:30.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.030+0000 7f96c42b9640 1 -- 172.21.15.45:0/358097808 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564826952d00 con 0x56482770c400 2024-05-08T05:51:30.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.030+0000 7f96cbb17280 1 -- 172.21.15.45:0/358097808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648269623c0 con 0x56482770c400 2024-05-08T05:51:30.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.030+0000 7f96cbb17280 4 main: JournalTool: resolving pool 17 2024-05-08T05:51:30.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.031+0000 7f96c42b9640 1 -- 172.21.15.45:0/358097808 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x564827874c00 con 0x56482770c400 2024-05-08T05:51:30.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.031+0000 7f96c42b9640 1 --2- 172.21.15.45:0/358097808 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x564826cc8000 0x5648276f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.031+0000 7f96c42b9640 1 -- 172.21.15.45:0/358097808 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x56482787aa00 con 0x56482770c400 2024-05-08T05:51:30.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.031+0000 7f96cbb17280 4 main: JournalTool: creating IoCtx.. 2024-05-08T05:51:30.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.031+0000 7f96cbb17280 4 main: Executing for rank 0 2024-05-08T05:51:30.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.031+0000 7f96caac6640 1 --2- 172.21.15.45:0/358097808 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x564826cc8000 0x5648276f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.032+0000 7f96caac6640 1 --2- 172.21.15.45:0/358097808 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x564826cc8000 0x5648276f8100 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x5648276ba2a0 tx=0x564826cfa000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.032+0000 7f96cbb17280 1 -- start start 2024-05-08T05:51:30.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.032+0000 7f96cbb17280 1 Processor -- start 2024-05-08T05:51:30.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96cbb17280 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56482770cc00 0x5648276f8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96cbb17280 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56482770d000 0x5648276f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96cbb17280 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56482770d400 0x5648276f9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96cbb17280 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564827701680 con 0x56482770d000 2024-05-08T05:51:30.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96cbb17280 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x564827701500 con 0x56482770d400 2024-05-08T05:51:30.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96cbb17280 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x564827700480 con 0x56482770cc00 2024-05-08T05:51:30.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96ca2c5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56482770d000 0x5648276f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96ca2c5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56482770d000 0x5648276f8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51570/0 (socket says 172.21.15.45:51570) 2024-05-08T05:51:30.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96ca2c5640 1 -- 172.21.15.45:0/1698073836 learned_addr learned my addr 172.21.15.45:0/1698073836 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:30.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96caac6640 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56482770cc00 0x5648276f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96c9ac4640 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56482770d400 0x5648276f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96ca2c5640 1 -- 172.21.15.45:0/1698073836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56482770cc00 msgr2=0x5648276f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:30.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96ca2c5640 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56482770cc00 0x5648276f8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96ca2c5640 1 -- 172.21.15.45:0/1698073836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56482770d400 msgr2=0x5648276f9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:30.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96ca2c5640 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56482770d400 0x5648276f9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96ca2c5640 1 -- 172.21.15.45:0/1698073836 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564826963860 con 0x56482770d000 2024-05-08T05:51:30.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96ca2c5640 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56482770d000 0x5648276f8c00 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x5648278aad20 tx=0x5648276cc8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55ab30595d6fb6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96c12b3640 1 -- 172.21.15.45:0/1698073836 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564826952b60 con 0x56482770d000 2024-05-08T05:51:30.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96c12b3640 1 -- 172.21.15.45:0/1698073836 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x564826958540 con 0x56482770d000 2024-05-08T05:51:30.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96c12b3640 1 -- 172.21.15.45:0/1698073836 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5648278bc1a0 con 0x56482770d000 2024-05-08T05:51:30.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96cbb17280 1 -- 172.21.15.45:0/1698073836 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564826962780 con 0x56482770d000 2024-05-08T05:51:30.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96cbb17280 1 -- 172.21.15.45:0/1698073836 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=0}) -- 0x564826962000 con 0x56482770d000 2024-05-08T05:51:30.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96cbb17280 4 waiting for MDS map... 2024-05-08T05:51:30.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.033+0000 7f96c12b3640 1 -- 172.21.15.45:0/1698073836 <== mon.0 v2:172.21.15.45:3300/0 4 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x564826d22600 con 0x56482770d000 2024-05-08T05:51:30.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.034+0000 7f96c12b3640 1 -- 172.21.15.45:0/1698073836 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 107) ==== 2885+0+0 (secure 0 0 0) 0x564826987500 con 0x56482770d000 2024-05-08T05:51:30.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.034+0000 7f96cbb17280 4 Got MDS map 107 2024-05-08T05:51:30.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.034+0000 7f96cbb17280 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x56482770d800 0x5648276f9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.034+0000 7f96cbb17280 1 -- 172.21.15.45:0/1698073836 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:1 17.f 17:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x56482770dc00 con 0x56482770d800 2024-05-08T05:51:30.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.034+0000 7f96caac6640 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x56482770d800 0x5648276f9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.034+0000 7f96caac6640 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x56482770d800 0x5648276f9700 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=320b039a9a07d0ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.037+0000 7f96caac6640 1 -- 172.21.15.45:0/1698073836 <== osd.1 v2:172.21.15.45:6824/1337179184 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v121'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x564826a3a000 con 0x56482770d800 2024-05-08T05:51:30.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.037+0000 7f96cbb17280 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x564826cc9c00 0x564826d3c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.037+0000 7f96cbb17280 1 -- 172.21.15.45:0/1698073836 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:2 17.14 17:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564826d40000 con 0x564826cc9c00 2024-05-08T05:51:30.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.037+0000 7f96c9ac4640 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x564826cc9c00 0x564826d3c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.037+0000 7f96c9ac4640 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x564826cc9c00 0x564826d3c000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=8d9b401ee0b2c2c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.041+0000 7f96c9ac4640 1 -- 172.21.15.45:0/1698073836 <== osd.3 v2:172.21.15.45:6816/1259225446 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v121'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x564826a3a500 con 0x564826cc9c00 2024-05-08T05:51:30.043 INFO:teuthology.orchestra.run.smithi045.stdout:writing EResetJournal entry 2024-05-08T05:51:30.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.041+0000 7f96cbb17280 1 -- 172.21.15.45:0/1698073836 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:3 17.14 17:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564826d40400 con 0x564826cc9c00 2024-05-08T05:51:30.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.041+0000 7f96cbb17280 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x564826d40800 0x564826d3c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.041+0000 7f96cbb17280 1 -- 172.21.15.45:0/1698073836 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:4 17.1b 17:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564826d40c00 con 0x564826d40800 2024-05-08T05:51:30.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.041+0000 7f96cbb17280 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x564826d41000 0x564826d3cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.041+0000 7f96cbb17280 1 -- 172.21.15.45:0/1698073836 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:5 17.b 17:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564826d41400 con 0x564826d41000 2024-05-08T05:51:30.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.041+0000 7f96cbb17280 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x564826d41800 0x564826d3d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.041+0000 7f96cbb17280 1 -- 172.21.15.45:0/1698073836 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:6 17.1f 17:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564826d41c00 con 0x564826d41800 2024-05-08T05:51:30.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.041+0000 7f96cbb17280 1 -- 172.21.15.45:0/1698073836 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:7 17.1a 17:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564826d4e000 con 0x564826d41000 2024-05-08T05:51:30.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.041+0000 7f96cbb17280 1 -- 172.21.15.45:0/1698073836 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:8 17.1e 17:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x564826d4e400 con 0x56482770d800 2024-05-08T05:51:30.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.041+0000 7f96ca2c5640 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x564826d41000 0x564826d3cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.041+0000 7f96caac6640 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x564826d40800 0x564826d3c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.041+0000 7f96c9ac4640 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x564826d41800 0x564826d3d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.042+0000 7f96ca2c5640 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x564826d41000 0x564826d3cb00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=57e47ff455887aee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.042+0000 7f96c9ac4640 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x564826d41800 0x564826d3d080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=4b59ae852c3bd555 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.042+0000 7f96c9ac4640 1 -- 172.21.15.45:0/1698073836 <== osd.3 v2:172.21.15.45:6816/1259225446 2 ==== osd_op_reply(3 200.00000001 [delete] v121'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x564826a3a500 con 0x564826cc9c00 2024-05-08T05:51:30.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.042+0000 7f96caac6640 1 -- 172.21.15.45:0/1698073836 <== osd.1 v2:172.21.15.45:6824/1337179184 2 ==== osd_op_reply(8 200.00000006 [delete] v121'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x564826a3a000 con 0x56482770d800 2024-05-08T05:51:30.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.043+0000 7f96caac6640 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x564826d40800 0x564826d3c580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=9bbb1ad242d1e3a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.043+0000 7f96ca2c5640 1 -- 172.21.15.45:0/1698073836 <== osd.0 v2:172.21.15.45:6800/3734454393 1 ==== osd_op_reply(5 200.00000003 [delete] v121'6 uv5 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x564826a3a780 con 0x564826d41000 2024-05-08T05:51:30.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.044+0000 7f96ca2c5640 1 -- 172.21.15.45:0/1698073836 <== osd.0 v2:172.21.15.45:6800/3734454393 2 ==== osd_op_reply(7 200.00000005 [delete] v121'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x564826a3a780 con 0x564826d41000 2024-05-08T05:51:30.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.044+0000 7f96caac6640 1 -- 172.21.15.45:0/1698073836 <== osd.4 v2:172.21.15.63:6824/2212879151 1 ==== osd_op_reply(4 200.00000002 [delete] v121'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x564826a3a000 con 0x564826d40800 2024-05-08T05:51:30.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.044+0000 7f96bfab0640 1 -- 172.21.15.45:0/1698073836 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:9 17.14 17:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x56482770dc00 con 0x564826cc9c00 2024-05-08T05:51:30.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96c9ac4640 1 -- 172.21.15.45:0/1698073836 <== osd.6 v2:172.21.15.63:6800/2926759052 1 ==== osd_op_reply(6 200.00000004 [delete] v121'9 uv8 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x564826a3a500 con 0x564826d41800 2024-05-08T05:51:30.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96c9ac4640 1 -- 172.21.15.45:0/1698073836 <== osd.3 v2:172.21.15.45:6816/1259225446 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v121'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x564826a3a500 con 0x564826cc9c00 2024-05-08T05:51:30.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96cbb17280 4 Successfully wrote new journal pointer and header for rank 8:0 2024-05-08T05:51:30.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96cbb17280 1 -- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x564826d41000 msgr2=0x564826d3cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96cbb17280 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x564826d41000 0x564826d3cb00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96cbb17280 1 -- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x56482770d800 msgr2=0x5648276f9700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96cbb17280 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x56482770d800 0x5648276f9700 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96cbb17280 1 -- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x564826cc9c00 msgr2=0x564826d3c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96cbb17280 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x564826cc9c00 0x564826d3c000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96cbb17280 1 -- 172.21.15.45:0/1698073836 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x564826d40800 msgr2=0x564826d3c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96cbb17280 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x564826d40800 0x564826d3c580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96cbb17280 1 -- 172.21.15.45:0/1698073836 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x564826d41800 msgr2=0x564826d3d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96cbb17280 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x564826d41800 0x564826d3d080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96cbb17280 1 -- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56482770d000 msgr2=0x5648276f8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96cbb17280 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56482770d000 0x5648276f8c00 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x5648278aad20 tx=0x5648276cc8a0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96cbb17280 1 -- 172.21.15.45:0/1698073836 shutdown_connections 2024-05-08T05:51:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96cbb17280 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x564826d41000 0x564826d3cb00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96ca2c5640 1 -- 172.21.15.45:0/1698073836 reap_dead start 2024-05-08T05:51:30.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96cbb17280 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x564826d40800 0x564826d3c580 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96cbb17280 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x564826cc9c00 0x564826d3c000 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.046+0000 7f96cbb17280 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x564826d41800 0x564826d3d080 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x56482770d800 0x5648276f9700 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56482770d400 0x5648276f9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56482770d000 0x5648276f8c00 secure :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x5648278aad20 tx=0x5648276cc8a0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 --2- 172.21.15.45:0/1698073836 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56482770cc00 0x5648276f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 -- 172.21.15.45:0/1698073836 >> 172.21.15.45:0/1698073836 conn(0x56482770c800 msgr2=0x564826a24800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96ca2c5640 1 -- 172.21.15.45:0/1698073836 reap_dead start 2024-05-08T05:51:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 -- 172.21.15.45:0/1698073836 shutdown_connections 2024-05-08T05:51:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 -- 172.21.15.45:0/1698073836 wait complete. 2024-05-08T05:51:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 -- 172.21.15.45:0/358097808 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x564826cc8000 msgr2=0x5648276f8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 --2- 172.21.15.45:0/358097808 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x564826cc8000 0x5648276f8100 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x5648276ba2a0 tx=0x564826cfa000 comp rx=0 tx=0).stop 2024-05-08T05:51:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 -- 172.21.15.45:0/358097808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56482770c400 msgr2=0x5648276f7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 --2- 172.21.15.45:0/358097808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56482770c400 0x5648276f7b80 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x56482781d740 tx=0x5648276ccae0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 -- 172.21.15.45:0/358097808 shutdown_connections 2024-05-08T05:51:30.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 --2- 172.21.15.45:0/358097808 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x564826cc8000 0x5648276f8100 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 --2- 172.21.15.45:0/358097808 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56482770c000 0x5648276f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 --2- 172.21.15.45:0/358097808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56482770c400 0x5648276f7b80 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 --2- 172.21.15.45:0/358097808 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x564827801c00 0x5648276f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 -- 172.21.15.45:0/358097808 >> 172.21.15.45:0/358097808 conn(0x5648276ca000 msgr2=0x564826a25000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:30.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 -- 172.21.15.45:0/358097808 shutdown_connections 2024-05-08T05:51:30.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 -- 172.21.15.45:0/358097808 wait complete. 2024-05-08T05:51:30.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 -- 172.21.15.45:0/4082852475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648276ca400 msgr2=0x5648276f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 --2- 172.21.15.45:0/4082852475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648276ca400 0x5648276f6b00 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x56482774f7d0 tx=0x5648276ce180 comp rx=0 tx=0).stop 2024-05-08T05:51:30.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 -- 172.21.15.45:0/4082852475 shutdown_connections 2024-05-08T05:51:30.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 --2- 172.21.15.45:0/4082852475 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5648276ca800 0x5648276f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 --2- 172.21.15.45:0/4082852475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56482692dc00 0x5648276f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 --2- 172.21.15.45:0/4082852475 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5648276ca400 0x5648276f6b00 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.047+0000 7f96cbb17280 1 -- 172.21.15.45:0/4082852475 >> 172.21.15.45:0/4082852475 conn(0x56482692d000 msgr2=0x564826a26000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:30.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.048+0000 7f96cbb17280 1 -- 172.21.15.45:0/4082852475 shutdown_connections 2024-05-08T05:51:30.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.048+0000 7f96cbb17280 1 -- 172.21.15.45:0/4082852475 wait complete. 2024-05-08T05:51:30.054 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force', '--yes-i-really-really-mean-it'] in time 0:00:00.075861, result: writing EResetJournal entry 2024-05-08T05:51:30.054 DEBUG:teuthology.orchestra.run.smithi045:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-05-08T05:51:30.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.132+0000 7fc7b5b22740 1 -- start start 2024-05-08T05:51:30.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.132+0000 7fc7b5b22740 1 Processor -- start 2024-05-08T05:51:30.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.132+0000 7fc7b5b22740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5595dce2ac00 0x5595ddbf2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.132+0000 7fc7b5b22740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5595dce2b800 0x5595ddbf2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b5b22740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5595dce2bc00 0x5595ddbf2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b5b22740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5595dcf2e180 con 0x5595dce2bc00 2024-05-08T05:51:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b5b22740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5595dcf2e300 con 0x5595dce2b800 2024-05-08T05:51:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b5b22740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5595dcf2e480 con 0x5595dce2ac00 2024-05-08T05:51:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b427d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5595dce2bc00 0x5595ddbf2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b427d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5595dce2bc00 0x5595ddbf2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51586/0 (socket says 172.21.15.45:51586) 2024-05-08T05:51:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b427d640 1 -- 172.21.15.45:0/1349169661 learned_addr learned my addr 172.21.15.45:0/1349169661 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b3a7c640 1 --2- 172.21.15.45:0/1349169661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5595dce2ac00 0x5595ddbf2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b327b640 1 --2- 172.21.15.45:0/1349169661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5595dce2b800 0x5595ddbf2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b427d640 1 -- 172.21.15.45:0/1349169661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5595dce2ac00 msgr2=0x5595ddbf2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b427d640 1 --2- 172.21.15.45:0/1349169661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5595dce2ac00 0x5595ddbf2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b427d640 1 -- 172.21.15.45:0/1349169661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5595dce2b800 msgr2=0x5595ddbf2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b427d640 1 --2- 172.21.15.45:0/1349169661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5595dce2b800 0x5595ddbf2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b427d640 1 -- 172.21.15.45:0/1349169661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595dce57680 con 0x5595dce2bc00 2024-05-08T05:51:30.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b3a7c640 1 --2- 172.21.15.45:0/1349169661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5595dce2ac00 0x5595ddbf2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b427d640 1 --2- 172.21.15.45:0/1349169661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5595dce2bc00 0x5595ddbf2b00 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x5595ddc3fbc0 tx=0x5595ddbd0600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0613c247de57649 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b1277640 1 -- 172.21.15.45:0/1349169661 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5595dce509c0 con 0x5595dce2bc00 2024-05-08T05:51:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b1277640 1 -- 172.21.15.45:0/1349169661 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5595dce72e00 con 0x5595dce2bc00 2024-05-08T05:51:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b1277640 1 -- 172.21.15.45:0/1349169661 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5595dce50680 con 0x5595dce2bc00 2024-05-08T05:51:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b5b22740 1 -- 172.21.15.45:0/1349169661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595dce574a0 con 0x5595dce2bc00 2024-05-08T05:51:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b5b22740 1 -- 172.21.15.45:0/1349169661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5595dce57a40 con 0x5595dce2bc00 2024-05-08T05:51:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.133+0000 7fc7b5b22740 4 waiting for MDS map... 2024-05-08T05:51:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.134+0000 7fc7b1277640 1 -- 172.21.15.45:0/1349169661 <== mon.0 v2:172.21.15.45:3300/0 4 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x5595ddd0c600 con 0x5595dce2bc00 2024-05-08T05:51:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.134+0000 7fc7b1277640 1 -- 172.21.15.45:0/1349169661 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 107) ==== 2885+0+0 (secure 0 0 0) 0x5595dce83180 con 0x5595dce2bc00 2024-05-08T05:51:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.134+0000 7fc7b5b22740 4 Got MDS map 107 2024-05-08T05:51:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.134+0000 7fc7b5b22740 4 datascan.main: Using metadata pool output 2024-05-08T05:51:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.134+0000 7fc7b5b22740 4 datascan.main: connecting to RADOS... 2024-05-08T05:51:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.134+0000 7fc7b5b22740 1 Processor -- start 2024-05-08T05:51:30.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.134+0000 7fc7b5b22740 1 -- start start 2024-05-08T05:51:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.135+0000 7fc7b5b22740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5595ddd50000 0x5595ddbf3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.135+0000 7fc7b5b22740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5595ddd50400 0x5595ddbf3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.135+0000 7fc7b5b22740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5595ddd50800 0x5595ddbf3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.135+0000 7fc7b5b22740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5595dcf2e600 con 0x5595ddd50800 2024-05-08T05:51:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.135+0000 7fc7b5b22740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5595dcf2e780 con 0x5595ddd50400 2024-05-08T05:51:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.135+0000 7fc7b5b22740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5595dcf2e900 con 0x5595ddd50000 2024-05-08T05:51:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.135+0000 7fc7b327b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5595ddd50800 0x5595ddbf3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.135+0000 7fc7b327b640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5595ddd50800 0x5595ddbf3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51590/0 (socket says 172.21.15.45:51590) 2024-05-08T05:51:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.135+0000 7fc7b327b640 1 -- 172.21.15.45:0/1278227062 learned_addr learned my addr 172.21.15.45:0/1278227062 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.135+0000 7fc7b327b640 1 -- 172.21.15.45:0/1278227062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5595ddd50000 msgr2=0x5595ddbf3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.135+0000 7fc7b327b640 1 --2- 172.21.15.45:0/1278227062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5595ddd50000 0x5595ddbf3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.135+0000 7fc7b327b640 1 -- 172.21.15.45:0/1278227062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5595ddd50400 msgr2=0x5595ddbf3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.135+0000 7fc7b327b640 1 --2- 172.21.15.45:0/1278227062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5595ddd50400 0x5595ddbf3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.135+0000 7fc7b327b640 1 -- 172.21.15.45:0/1278227062 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595dce57860 con 0x5595ddd50800 2024-05-08T05:51:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.135+0000 7fc7b327b640 1 --2- 172.21.15.45:0/1278227062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5595ddd50800 0x5595ddbf3b80 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x5595ddd071a0 tx=0x5595ddcfcc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce6a54741407100 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.136+0000 7fc7af273640 1 -- 172.21.15.45:0/1278227062 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5595dce509c0 con 0x5595ddd50800 2024-05-08T05:51:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.136+0000 7fc7af273640 1 -- 172.21.15.45:0/1278227062 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5595dce72540 con 0x5595ddd50800 2024-05-08T05:51:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.136+0000 7fc7af273640 1 -- 172.21.15.45:0/1278227062 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5595dce50680 con 0x5595ddd50800 2024-05-08T05:51:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.136+0000 7fc7b5b22740 1 -- 172.21.15.45:0/1278227062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5595ddd50800 msgr2=0x5595ddbf3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.136+0000 7fc7b5b22740 1 --2- 172.21.15.45:0/1278227062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5595ddd50800 0x5595ddbf3b80 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x5595ddd071a0 tx=0x5595ddcfcc00 comp rx=0 tx=0).stop 2024-05-08T05:51:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.136+0000 7fc7b5b22740 1 -- 172.21.15.45:0/1278227062 shutdown_connections 2024-05-08T05:51:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.136+0000 7fc7b5b22740 1 --2- 172.21.15.45:0/1278227062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5595ddd50800 0x5595ddbf3b80 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.136+0000 7fc7b5b22740 1 --2- 172.21.15.45:0/1278227062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5595ddd50400 0x5595ddbf3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.136+0000 7fc7b5b22740 1 --2- 172.21.15.45:0/1278227062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5595ddd50000 0x5595ddbf3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.136+0000 7fc7b5b22740 1 -- 172.21.15.45:0/1278227062 >> 172.21.15.45:0/1278227062 conn(0x5595ddc03c00 msgr2=0x5595ddba9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.136+0000 7fc7b5b22740 1 -- 172.21.15.45:0/1278227062 shutdown_connections 2024-05-08T05:51:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.136+0000 7fc7b5b22740 1 -- 172.21.15.45:0/1278227062 wait complete. 2024-05-08T05:51:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.137+0000 7fc7b5b22740 1 Processor -- start 2024-05-08T05:51:30.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.137+0000 7fc7b5b22740 1 -- start start 2024-05-08T05:51:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.137+0000 7fc7b5b22740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5595ddd50000 0x5595ddbf3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.137+0000 7fc7b5b22740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5595ddd50400 0x5595ddbf3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.137+0000 7fc7b5b22740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5595ddd50800 0x5595ddbf3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.137+0000 7fc7b5b22740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5595dcf2fc80 con 0x5595ddd50400 2024-05-08T05:51:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.137+0000 7fc7b5b22740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x5595dcf2fb00 con 0x5595ddd50800 2024-05-08T05:51:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.137+0000 7fc7b5b22740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x5595dcf2f980 con 0x5595ddd50000 2024-05-08T05:51:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.137+0000 7fc7b3a7c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5595ddd50400 0x5595ddbf3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.137+0000 7fc7b3a7c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5595ddd50400 0x5595ddbf3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51602/0 (socket says 172.21.15.45:51602) 2024-05-08T05:51:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.137+0000 7fc7b3a7c640 1 -- 172.21.15.45:0/473461767 learned_addr learned my addr 172.21.15.45:0/473461767 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.137+0000 7fc7b327b640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5595ddd50000 0x5595ddbf3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.138+0000 7fc7b327b640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5595ddd50800 0x5595ddbf3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.138+0000 7fc7b3a7c640 1 -- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5595ddd50000 msgr2=0x5595ddbf3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.138+0000 7fc7b3a7c640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5595ddd50000 0x5595ddbf3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.138+0000 7fc7b3a7c640 1 -- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5595ddd50800 msgr2=0x5595ddbf3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.138+0000 7fc7b3a7c640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5595ddd50800 0x5595ddbf3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.138+0000 7fc7b3a7c640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595dce572c0 con 0x5595ddd50400 2024-05-08T05:51:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.138+0000 7fc7b327b640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5595ddd50800 0x5595ddbf3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.138+0000 7fc7b3a7c640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5595ddd50400 0x5595ddbf3600 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x5595ddd07e30 tx=0x5595ddcfcd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.138+0000 7fc7ada70640 1 -- 172.21.15.45:0/473461767 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5595dce509c0 con 0x5595ddd50400 2024-05-08T05:51:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.138+0000 7fc7ada70640 1 -- 172.21.15.45:0/473461767 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5595dce72a80 con 0x5595ddd50400 2024-05-08T05:51:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.138+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5595dce570e0 con 0x5595ddd50400 2024-05-08T05:51:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.138+0000 7fc7ada70640 1 -- 172.21.15.45:0/473461767 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5595dce50680 con 0x5595ddd50400 2024-05-08T05:51:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.138+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595dce563c0 con 0x5595ddd50400 2024-05-08T05:51:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.138+0000 7fc7b5b22740 4 datascan.init: resolving metadata pool 17 2024-05-08T05:51:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.139+0000 7fc7ada70640 1 -- 172.21.15.45:0/473461767 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x5595ddd58c00 con 0x5595ddd50400 2024-05-08T05:51:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.140+0000 7fc7ada70640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x5595ddd88c00 0x5595ddbf4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.140+0000 7fc7ada70640 1 -- 172.21.15.45:0/473461767 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x5595ddd57000 con 0x5595ddd50400 2024-05-08T05:51:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.140+0000 7fc7b5b22740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-08T05:51:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.140+0000 7fc7b5b22740 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-05-08T05:51:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.140+0000 7fc7b5b22740 4 datascan.main: opening data pool 'cephfs_data' 2024-05-08T05:51:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.140+0000 7fc7b5b22740 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x5595ddd50c00 0x5595ddbf4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.140+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:1 18.0 18:00000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595ddd51000 con 0x5595ddd50c00 2024-05-08T05:51:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.140+0000 7fc7b327b640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x5595ddd88c00 0x5595ddbf4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.140+0000 7fc7b327b640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x5595ddd50c00 0x5595ddbf4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.140+0000 7fc7b327b640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x5595ddd50c00 0x5595ddbf4680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.140+0000 7fc7b327b640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x5595ddd88c00 0x5595ddbf4100 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x5595ddd072f0 tx=0x5595ddbd0780 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.140+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.0 v2:172.21.15.45:6800/3734454393 1 ==== osd_op_reply(1 [pgnls start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595ddd50c00 2024-05-08T05:51:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.141+0000 7fc7aea72640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x5595ddd89000 0x5595dd1e4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.141+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:2 18.20 18:04000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595ddd89400 con 0x5595ddd89000 2024-05-08T05:51:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.141+0000 7fc7b427d640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x5595ddd89000 0x5595dd1e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.141+0000 7fc7b427d640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x5595ddd89000 0x5595dd1e4000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.143+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.5 v2:172.21.15.63:6808/2395798255 1 ==== osd_op_reply(2 [pgnls start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd89000 2024-05-08T05:51:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.143+0000 7fc7af273640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x5595ddd89c00 0x5595dd1e4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.143+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:3 18.10 18:08000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ec000 con 0x5595ddd89c00 2024-05-08T05:51:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.144+0000 7fc7b3a7c640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x5595ddd89c00 0x5595dd1e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.144+0000 7fc7b3a7c640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x5595ddd89c00 0x5595dd1e4580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.144+0000 7fc7b3a7c640 1 -- 172.21.15.45:0/473461767 <== osd.6 v2:172.21.15.63:6800/2926759052 1 ==== osd_op_reply(3 [pgnls start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc780 con 0x5595ddd89c00 2024-05-08T05:51:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.144+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:4 18.30 18:0c000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595ddd89800 con 0x5595ddd50c00 2024-05-08T05:51:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.145+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.0 v2:172.21.15.45:6800/3734454393 2 ==== osd_op_reply(4 [pgnls start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595ddd50c00 2024-05-08T05:51:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.145+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:5 18.8 18:10000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ec400 con 0x5595ddd50c00 2024-05-08T05:51:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.145+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.0 v2:172.21.15.45:6800/3734454393 3 ==== osd_op_reply(5 [pgnls start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595ddd50c00 2024-05-08T05:51:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.145+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:6 18.28 18:14000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595ddd89800 con 0x5595ddd89000 2024-05-08T05:51:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.145+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.5 v2:172.21.15.63:6808/2395798255 2 ==== osd_op_reply(6 [pgnls start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd89000 2024-05-08T05:51:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.145+0000 7fc7af273640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x5595dd1ec400 0x5595dd1e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.145+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:7 18.18 18:18000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ec800 con 0x5595dd1ec400 2024-05-08T05:51:30.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.145+0000 7fc7b327b640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x5595dd1ec400 0x5595dd1e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.146+0000 7fc7b327b640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x5595dd1ec400 0x5595dd1e4b00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.146+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.1 v2:172.21.15.45:6824/1337179184 1 ==== osd_op_reply(7 [pgnls start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1ec400 2024-05-08T05:51:30.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.146+0000 7fc7aea72640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x5595ddd51c00 0x5595dd1e5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.146+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:8 18.38 18:1c000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595ddd51000 con 0x5595ddd51c00 2024-05-08T05:51:30.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.146+0000 7fc7b427d640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x5595ddd51c00 0x5595dd1e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.146+0000 7fc7b427d640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x5595ddd51c00 0x5595dd1e5600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.146+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.3 v2:172.21.15.45:6816/1259225446 1 ==== osd_op_reply(8 [pgnls start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd51c00 2024-05-08T05:51:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.146+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:9 18.4 18:20000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ecc00 con 0x5595ddd50c00 2024-05-08T05:51:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.147+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.0 v2:172.21.15.45:6800/3734454393 4 ==== osd_op_reply(9 [pgnls start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595ddd50c00 2024-05-08T05:51:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.147+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:10 18.24 18:24000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ed000 con 0x5595ddd50c00 2024-05-08T05:51:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.147+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.0 v2:172.21.15.45:6800/3734454393 5 ==== osd_op_reply(10 [pgnls start_epoch 121 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595ddd50c00 2024-05-08T05:51:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.147+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:11 18.14 18:28000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1fe000 con 0x5595ddd89c00 2024-05-08T05:51:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.147+0000 7fc7b3a7c640 1 -- 172.21.15.45:0/473461767 <== osd.6 v2:172.21.15.63:6800/2926759052 2 ==== osd_op_reply(11 [pgnls start_epoch 121 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc780 con 0x5595ddd89c00 2024-05-08T05:51:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.147+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:12 18.34 18:2c000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ed400 con 0x5595ddd50c00 2024-05-08T05:51:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.148+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.0 v2:172.21.15.45:6800/3734454393 6 ==== osd_op_reply(12 [pgnls start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595ddd50c00 2024-05-08T05:51:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.148+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:13 18.c 18:30000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1fe400 con 0x5595ddd89000 2024-05-08T05:51:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.148+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.5 v2:172.21.15.63:6808/2395798255 3 ==== osd_op_reply(13 [pgnls start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd89000 2024-05-08T05:51:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.148+0000 7fc7aea72640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x5595dd1ed800 0x5595dd1e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.148+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:14 18.2c 18:34000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1edc00 con 0x5595dd1ed800 2024-05-08T05:51:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.148+0000 7fc7b3a7c640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x5595dd1ed800 0x5595dd1e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.148+0000 7fc7b3a7c640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x5595dd1ed800 0x5595dd1e5b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.149+0000 7fc7b3a7c640 1 -- 172.21.15.45:0/473461767 <== osd.2 v2:172.21.15.45:6808/785834340 1 ==== osd_op_reply(14 [pgnls start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc780 con 0x5595dd1ed800 2024-05-08T05:51:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.149+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:15 18.1c 18:38000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1fe800 con 0x5595dd1ec400 2024-05-08T05:51:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.149+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.1 v2:172.21.15.45:6824/1337179184 2 ==== osd_op_reply(15 [pgnls start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1ec400 2024-05-08T05:51:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.149+0000 7fc7aea72640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x5595dd1fe800 0x5595dd1e6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.149+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:16 18.3c 18:3c000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ec800 con 0x5595dd1fe800 2024-05-08T05:51:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.149+0000 7fc7b327b640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x5595dd1fe800 0x5595dd1e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.149+0000 7fc7b327b640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x5595dd1fe800 0x5595dd1e6100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.150+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.7 v2:172.21.15.63:6816/1163469437 1 ==== osd_op_reply(16 [pgnls start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1fe800 2024-05-08T05:51:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.150+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:17 18.2 18:40000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1fec00 con 0x5595dd1fe800 2024-05-08T05:51:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.150+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.7 v2:172.21.15.63:6816/1163469437 2 ==== osd_op_reply(17 [pgnls start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1fe800 2024-05-08T05:51:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.150+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:18 18.22 18:44000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ecc00 con 0x5595ddd51c00 2024-05-08T05:51:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.150+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.3 v2:172.21.15.45:6816/1259225446 2 ==== osd_op_reply(18 [pgnls start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd51c00 2024-05-08T05:51:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.150+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:19 18.12 18:48000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1fe400 con 0x5595dd1ed800 2024-05-08T05:51:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.151+0000 7fc7b3a7c640 1 -- 172.21.15.45:0/473461767 <== osd.2 v2:172.21.15.45:6808/785834340 2 ==== osd_op_reply(19 [pgnls start_epoch 121 out=81b] v120'22 uv22 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5595ddbbc780 con 0x5595dd1ed800 2024-05-08T05:51:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.151+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:20 18.12 18:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1fe000 con 0x5595dd1ed800 2024-05-08T05:51:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.151+0000 7fc7b3a7c640 1 -- 172.21.15.45:0/473461767 <== osd.2 v2:172.21.15.45:6808/785834340 3 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5595ddbbc780 con 0x5595dd1ed800 2024-05-08T05:51:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.151+0000 7fc7b5b22740 10 datascan.operator(): handling object 1099511627778.0 2024-05-08T05:51:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.151+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:21 18.12 18:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1fe400 con 0x5595dd1ed800 2024-05-08T05:51:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.152+0000 7fc7b3a7c640 1 -- 172.21.15.45:0/473461767 <== osd.2 v2:172.21.15.45:6808/785834340 4 ==== osd_op_reply(21 10000000002.00000000 [call] v121'23 uv23 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5595ddbbc780 con 0x5595dd1ed800 2024-05-08T05:51:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.152+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:22 18.32 18:4c000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ff000 con 0x5595ddd51c00 2024-05-08T05:51:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.153+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.3 v2:172.21.15.45:6816/1259225446 3 ==== osd_op_reply(22 [pgnls start_epoch 121 out=49b] v119'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd51c00 2024-05-08T05:51:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.153+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:23 18.a 18:50000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ecc00 con 0x5595ddd51c00 2024-05-08T05:51:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.153+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.3 v2:172.21.15.45:6816/1259225446 4 ==== osd_op_reply(23 [pgnls start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd51c00 2024-05-08T05:51:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.153+0000 7fc7aea72640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x5595dd1ed000 0x5595dd1e6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.153+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:24 18.2a 18:54000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ed400 con 0x5595dd1ed000 2024-05-08T05:51:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.153+0000 7fc7b427d640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x5595dd1ed000 0x5595dd1e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.153+0000 7fc7b427d640 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x5595dd1ed000 0x5595dd1e6680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.154+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.4 v2:172.21.15.63:6824/2212879151 1 ==== osd_op_reply(24 [pgnls start_epoch 121 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595dd1ed000 2024-05-08T05:51:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.154+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:25 18.1a 18:58000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595ddd51000 con 0x5595ddd51c00 2024-05-08T05:51:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.154+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.3 v2:172.21.15.45:6816/1259225446 5 ==== osd_op_reply(25 [pgnls start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd51c00 2024-05-08T05:51:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.154+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:26 18.3a 18:5c000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ff800 con 0x5595ddd50c00 2024-05-08T05:51:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.154+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.0 v2:172.21.15.45:6800/3734454393 7 ==== osd_op_reply(26 [pgnls start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595ddd50c00 2024-05-08T05:51:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.154+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:27 18.6 18:60000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595ddd89400 con 0x5595dd1fe800 2024-05-08T05:51:30.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.155+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.7 v2:172.21.15.63:6816/1163469437 3 ==== osd_op_reply(27 [pgnls start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1fe800 2024-05-08T05:51:30.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.155+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:28 18.26 18:64000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ffc00 con 0x5595dd1ed800 2024-05-08T05:51:30.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.155+0000 7fc7b3a7c640 1 -- 172.21.15.45:0/473461767 <== osd.2 v2:172.21.15.45:6808/785834340 5 ==== osd_op_reply(28 [pgnls start_epoch 121 out=49b] v119'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc780 con 0x5595dd1ed800 2024-05-08T05:51:30.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.155+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:29 18.16 18:68000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595ddd89800 con 0x5595ddd51c00 2024-05-08T05:51:30.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.155+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.3 v2:172.21.15.45:6816/1259225446 6 ==== osd_op_reply(29 [pgnls start_epoch 121 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd51c00 2024-05-08T05:51:30.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.155+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:30 18.36 18:6c000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd212000 con 0x5595dd1ec400 2024-05-08T05:51:30.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.156+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.1 v2:172.21.15.45:6824/1337179184 3 ==== osd_op_reply(30 [pgnls start_epoch 121 out=81b] v119'33 uv17 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1ec400 2024-05-08T05:51:30.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.156+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:31 18.36 18:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ff400 con 0x5595dd1ec400 2024-05-08T05:51:30.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.156+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.1 v2:172.21.15.45:6824/1337179184 4 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1ec400 2024-05-08T05:51:30.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.156+0000 7fc7b5b22740 10 datascan.operator(): handling object 1099511628279.0 2024-05-08T05:51:30.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.156+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:32 18.36 18:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ec000 con 0x5595dd1ec400 2024-05-08T05:51:30.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.157+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.1 v2:172.21.15.45:6824/1337179184 5 ==== osd_op_reply(32 100000001f7.00000000 [call] v121'34 uv34 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1ec400 2024-05-08T05:51:30.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.157+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:33 18.e 18:70000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1edc00 con 0x5595ddd50c00 2024-05-08T05:51:30.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.157+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.0 v2:172.21.15.45:6800/3734454393 8 ==== osd_op_reply(33 [pgnls start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595ddd50c00 2024-05-08T05:51:30.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.157+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:34 18.2e 18:74000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd212400 con 0x5595ddd51c00 2024-05-08T05:51:30.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.158+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.3 v2:172.21.15.45:6816/1259225446 7 ==== osd_op_reply(34 [pgnls start_epoch 121 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd51c00 2024-05-08T05:51:30.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.158+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:35 18.1e 18:78000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595ddd89800 con 0x5595ddd89000 2024-05-08T05:51:30.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.158+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.5 v2:172.21.15.63:6808/2395798255 4 ==== osd_op_reply(35 [pgnls start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd89000 2024-05-08T05:51:30.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.158+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:36 18.3e 18:7c000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd212800 con 0x5595ddd51c00 2024-05-08T05:51:30.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.158+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.3 v2:172.21.15.45:6816/1259225446 8 ==== osd_op_reply(36 [pgnls start_epoch 121 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd51c00 2024-05-08T05:51:30.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.158+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:37 18.1 18:80000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595ddd89400 con 0x5595ddd51c00 2024-05-08T05:51:30.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.158+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.3 v2:172.21.15.45:6816/1259225446 9 ==== osd_op_reply(37 [pgnls start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd51c00 2024-05-08T05:51:30.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.158+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:38 18.21 18:84000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd212c00 con 0x5595ddd89000 2024-05-08T05:51:30.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.159+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.5 v2:172.21.15.63:6808/2395798255 5 ==== osd_op_reply(38 [pgnls start_epoch 121 out=49b] v119'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd89000 2024-05-08T05:51:30.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.159+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:39 18.11 18:88000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ecc00 con 0x5595dd1ec400 2024-05-08T05:51:30.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.159+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.1 v2:172.21.15.45:6824/1337179184 6 ==== osd_op_reply(39 [pgnls start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1ec400 2024-05-08T05:51:30.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.159+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:40 18.31 18:8c000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd213000 con 0x5595dd1ed000 2024-05-08T05:51:30.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.160+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.4 v2:172.21.15.63:6824/2212879151 2 ==== osd_op_reply(40 [pgnls start_epoch 121 out=81b] v119'15 uv8 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5595ddbbc500 con 0x5595dd1ed000 2024-05-08T05:51:30.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.160+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:41 18.31 18:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5595dd212400 con 0x5595dd1ed000 2024-05-08T05:51:30.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.160+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.4 v2:172.21.15.63:6824/2212879151 3 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5595ddbbc500 con 0x5595dd1ed000 2024-05-08T05:51:30.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.160+0000 7fc7b5b22740 10 datascan.operator(): handling object 1099511627780.1 2024-05-08T05:51:30.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.160+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:42 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x5595dd212000 con 0x5595dd1ec400 2024-05-08T05:51:30.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.161+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.1 v2:172.21.15.45:6824/1337179184 7 ==== osd_op_reply(42 10000000004.00000000 [call] v121'14 uv14 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1ec400 2024-05-08T05:51:30.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.161+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:43 18.9 18:90000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd213c00 con 0x5595ddd51c00 2024-05-08T05:51:30.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.162+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.3 v2:172.21.15.45:6816/1259225446 10 ==== osd_op_reply(43 [pgnls start_epoch 121 out=49b] v119'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd51c00 2024-05-08T05:51:30.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.162+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:44 18.29 18:94000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd213400 con 0x5595dd1fe800 2024-05-08T05:51:30.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.162+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.7 v2:172.21.15.63:6816/1163469437 4 ==== osd_op_reply(44 [pgnls start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1fe800 2024-05-08T05:51:30.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.162+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:45 18.19 18:98000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ed400 con 0x5595dd1ed000 2024-05-08T05:51:30.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.162+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.4 v2:172.21.15.63:6824/2212879151 4 ==== osd_op_reply(45 [pgnls start_epoch 121 out=81b] v120'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5595ddbbc500 con 0x5595dd1ed000 2024-05-08T05:51:30.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.163+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:46 18.19 18:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ff000 con 0x5595dd1ed000 2024-05-08T05:51:30.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.163+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.4 v2:172.21.15.63:6824/2212879151 5 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5595ddbbc500 con 0x5595dd1ed000 2024-05-08T05:51:30.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.163+0000 7fc7b5b22740 10 datascan.operator(): handling object 1099511628280.0 2024-05-08T05:51:30.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.163+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:47 18.19 18:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1fec00 con 0x5595dd1ed000 2024-05-08T05:51:30.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.164+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.4 v2:172.21.15.63:6824/2212879151 6 ==== osd_op_reply(47 100000001f8.00000000 [call] v121'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5595ddbbc500 con 0x5595dd1ed000 2024-05-08T05:51:30.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.164+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:48 18.39 18:9c000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ff800 con 0x5595ddd50c00 2024-05-08T05:51:30.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.165+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.0 v2:172.21.15.45:6800/3734454393 9 ==== osd_op_reply(48 [pgnls start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595ddd50c00 2024-05-08T05:51:30.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.165+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:49 18.5 18:a0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ec800 con 0x5595dd1ec400 2024-05-08T05:51:30.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.165+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.1 v2:172.21.15.45:6824/1337179184 8 ==== osd_op_reply(49 [pgnls start_epoch 121 out=81b] v121'14 uv14 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1ec400 2024-05-08T05:51:30.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.165+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:50 18.5 18:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ff400 con 0x5595dd1ec400 2024-05-08T05:51:30.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.165+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.1 v2:172.21.15.45:6824/1337179184 9 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1ec400 2024-05-08T05:51:30.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.165+0000 7fc7b5b22740 10 datascan.operator(): handling object 1099511627780.0 2024-05-08T05:51:30.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.165+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:51 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x5595dd212000 con 0x5595dd1ec400 2024-05-08T05:51:30.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.166+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.1 v2:172.21.15.45:6824/1337179184 10 ==== osd_op_reply(51 10000000004.00000000 [call] v121'15 uv15 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1ec400 2024-05-08T05:51:30.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.166+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:52 18.25 18:a4000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595ddd89800 con 0x5595ddd89c00 2024-05-08T05:51:30.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.167+0000 7fc7b3a7c640 1 -- 172.21.15.45:0/473461767 <== osd.6 v2:172.21.15.63:6800/2926759052 3 ==== osd_op_reply(52 [pgnls start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc780 con 0x5595ddd89c00 2024-05-08T05:51:30.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.167+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:53 18.15 18:a8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595ddd51000 con 0x5595ddd89000 2024-05-08T05:51:30.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.167+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.5 v2:172.21.15.63:6808/2395798255 6 ==== osd_op_reply(53 [pgnls start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd89000 2024-05-08T05:51:30.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.167+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:54 18.35 18:ac000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd213800 con 0x5595dd1ed000 2024-05-08T05:51:30.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.168+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.4 v2:172.21.15.63:6824/2212879151 7 ==== osd_op_reply(54 [pgnls start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595dd1ed000 2024-05-08T05:51:30.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.168+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:55 18.d 18:b0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd213400 con 0x5595dd1ed000 2024-05-08T05:51:30.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.168+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.4 v2:172.21.15.63:6824/2212879151 8 ==== osd_op_reply(55 [pgnls start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595dd1ed000 2024-05-08T05:51:30.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.168+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:56 18.2d 18:b4000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ee000 con 0x5595ddd89c00 2024-05-08T05:51:30.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.168+0000 7fc7b3a7c640 1 -- 172.21.15.45:0/473461767 <== osd.6 v2:172.21.15.63:6800/2926759052 4 ==== osd_op_reply(56 [pgnls start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc780 con 0x5595ddd89c00 2024-05-08T05:51:30.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.168+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:57 18.1d 18:b8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd212800 con 0x5595ddd50c00 2024-05-08T05:51:30.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.169+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.0 v2:172.21.15.45:6800/3734454393 10 ==== osd_op_reply(57 [pgnls start_epoch 121 out=49b] v119'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595ddd50c00 2024-05-08T05:51:30.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.169+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:58 18.3d 18:bc000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ee400 con 0x5595ddd51c00 2024-05-08T05:51:30.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.169+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.3 v2:172.21.15.45:6816/1259225446 11 ==== osd_op_reply(58 [pgnls start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd51c00 2024-05-08T05:51:30.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.169+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:59 18.3 18:c0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd212c00 con 0x5595dd1ec400 2024-05-08T05:51:30.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.169+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.1 v2:172.21.15.45:6824/1337179184 11 ==== osd_op_reply(59 [pgnls start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1ec400 2024-05-08T05:51:30.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.169+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:60 18.23 18:c4000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ee800 con 0x5595dd1ed800 2024-05-08T05:51:30.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.170+0000 7fc7b3a7c640 1 -- 172.21.15.45:0/473461767 <== osd.2 v2:172.21.15.45:6808/785834340 6 ==== osd_op_reply(60 [pgnls start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc780 con 0x5595dd1ed800 2024-05-08T05:51:30.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.170+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:61 18.13 18:c8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd213000 con 0x5595dd1fe800 2024-05-08T05:51:30.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.170+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.7 v2:172.21.15.63:6816/1163469437 5 ==== osd_op_reply(61 [pgnls start_epoch 121 out=49b] v119'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1fe800 2024-05-08T05:51:30.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.170+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:62 18.33 18:cc000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1eec00 con 0x5595dd1ec400 2024-05-08T05:51:30.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.170+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.1 v2:172.21.15.45:6824/1337179184 12 ==== osd_op_reply(62 [pgnls start_epoch 121 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1ec400 2024-05-08T05:51:30.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.170+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:63 18.b 18:d0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd212400 con 0x5595ddd89000 2024-05-08T05:51:30.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.170+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.5 v2:172.21.15.63:6808/2395798255 7 ==== osd_op_reply(63 [pgnls start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd89000 2024-05-08T05:51:30.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.170+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:64 18.2b 18:d4000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ef000 con 0x5595dd1fe800 2024-05-08T05:51:30.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.171+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.7 v2:172.21.15.63:6816/1163469437 6 ==== osd_op_reply(64 [pgnls start_epoch 121 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1fe800 2024-05-08T05:51:30.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.171+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:65 18.1b 18:d8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd213c00 con 0x5595dd1ec400 2024-05-08T05:51:30.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.171+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.1 v2:172.21.15.45:6824/1337179184 13 ==== osd_op_reply(65 [pgnls start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1ec400 2024-05-08T05:51:30.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.171+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:66 18.3b 18:dc000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ef400 con 0x5595dd1ed000 2024-05-08T05:51:30.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.171+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.4 v2:172.21.15.63:6824/2212879151 9 ==== osd_op_reply(66 [pgnls start_epoch 121 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595dd1ed000 2024-05-08T05:51:30.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.171+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:67 18.7 18:e0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ff400 con 0x5595dd1ed800 2024-05-08T05:51:30.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.172+0000 7fc7b3a7c640 1 -- 172.21.15.45:0/473461767 <== osd.2 v2:172.21.15.45:6808/785834340 7 ==== osd_op_reply(67 [pgnls start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc780 con 0x5595dd1ed800 2024-05-08T05:51:30.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.172+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:68 18.27 18:e4000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ef800 con 0x5595ddd89000 2024-05-08T05:51:30.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.172+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.5 v2:172.21.15.63:6808/2395798255 8 ==== osd_op_reply(68 [pgnls start_epoch 121 out=49b] v119'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd89000 2024-05-08T05:51:30.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.172+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:69 18.17 18:e8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ff000 con 0x5595ddd89000 2024-05-08T05:51:30.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.172+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.5 v2:172.21.15.63:6808/2395798255 9 ==== osd_op_reply(69 [pgnls start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd89000 2024-05-08T05:51:30.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.172+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:70 18.37 18:ec000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1efc00 con 0x5595dd1fe800 2024-05-08T05:51:30.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.172+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.7 v2:172.21.15.63:6816/1163469437 7 ==== osd_op_reply(70 [pgnls start_epoch 121 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1fe800 2024-05-08T05:51:30.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.173+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:71 18.f 18:f0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1fec00 con 0x5595ddd89000 2024-05-08T05:51:30.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.173+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.5 v2:172.21.15.63:6808/2395798255 10 ==== osd_op_reply(71 [pgnls start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595ddd89000 2024-05-08T05:51:30.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.173+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:72 18.2f 18:f4000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1eec00 con 0x5595dd1ec400 2024-05-08T05:51:30.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.173+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.1 v2:172.21.15.45:6824/1337179184 14 ==== osd_op_reply(72 [pgnls start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1ec400 2024-05-08T05:51:30.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.173+0000 7fc7af273640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:73 18.1f 18:f8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1fe000 con 0x5595dd1fe800 2024-05-08T05:51:30.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.173+0000 7fc7b327b640 1 -- 172.21.15.45:0/473461767 <== osd.7 v2:172.21.15.63:6816/1163469437 8 ==== osd_op_reply(73 [pgnls start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc000 con 0x5595dd1fe800 2024-05-08T05:51:30.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.173+0000 7fc7aea72640 1 -- 172.21.15.45:0/473461767 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:74 18.3f 18:fc000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x5595dd1ef000 con 0x5595dd1ed000 2024-05-08T05:51:30.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b427d640 1 -- 172.21.15.45:0/473461767 <== osd.4 v2:172.21.15.63:6824/2212879151 10 ==== osd_op_reply(74 [pgnls start_epoch 121 out=49b] v119'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5595ddbbc500 con 0x5595dd1ed000 2024-05-08T05:51:30.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x5595ddd50c00 msgr2=0x5595ddbf4680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x5595ddd50c00 0x5595ddbf4680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x5595dd1ec400 msgr2=0x5595dd1e4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x5595dd1ec400 0x5595dd1e4b00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x5595dd1ed800 msgr2=0x5595dd1e5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x5595dd1ed800 0x5595dd1e5b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x5595ddd51c00 msgr2=0x5595dd1e5600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x5595ddd51c00 0x5595dd1e5600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b3a7c640 1 -- 172.21.15.45:0/473461767 reap_dead start 2024-05-08T05:51:30.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x5595dd1ed000 msgr2=0x5595dd1e6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b3a7c640 1 -- 172.21.15.45:0/473461767 reap_dead start 2024-05-08T05:51:30.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x5595dd1ed000 0x5595dd1e6680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x5595ddd89000 msgr2=0x5595dd1e4000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x5595ddd89000 0x5595dd1e4000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x5595ddd89c00 msgr2=0x5595dd1e4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x5595ddd89c00 0x5595dd1e4580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x5595dd1fe800 msgr2=0x5595dd1e6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x5595dd1fe800 0x5595dd1e6100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x5595ddd88c00 msgr2=0x5595ddbf4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x5595ddd88c00 0x5595ddbf4100 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x5595ddd072f0 tx=0x5595ddbd0780 comp rx=0 tx=0).stop 2024-05-08T05:51:30.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5595ddd50400 msgr2=0x5595ddbf3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 --2- 172.21.15.45:0/473461767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5595ddd50400 0x5595ddbf3600 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x5595ddd07e30 tx=0x5595ddcfcd20 comp rx=0 tx=0).stop 2024-05-08T05:51:30.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b3a7c640 1 -- 172.21.15.45:0/473461767 reap_dead start 2024-05-08T05:51:30.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b3a7c640 1 -- 172.21.15.45:0/473461767 reap_dead start 2024-05-08T05:51:30.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 shutdown_connections 2024-05-08T05:51:30.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 >> 172.21.15.45:0/473461767 conn(0x5595ddc03c00 msgr2=0x5595ddba9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:30.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 shutdown_connections 2024-05-08T05:51:30.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.174+0000 7fc7b5b22740 1 -- 172.21.15.45:0/473461767 wait complete. 2024-05-08T05:51:30.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.175+0000 7fc7b5b22740 1 -- 172.21.15.45:0/1349169661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5595dce2bc00 msgr2=0x5595ddbf2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.175+0000 7fc7b5b22740 1 --2- 172.21.15.45:0/1349169661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5595dce2bc00 0x5595ddbf2b00 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x5595ddc3fbc0 tx=0x5595ddbd0600 comp rx=0 tx=0).stop 2024-05-08T05:51:30.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.175+0000 7fc7b5b22740 1 -- 172.21.15.45:0/1349169661 shutdown_connections 2024-05-08T05:51:30.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.175+0000 7fc7b5b22740 1 --2- 172.21.15.45:0/1349169661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5595dce2bc00 0x5595ddbf2b00 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.175+0000 7fc7b5b22740 1 --2- 172.21.15.45:0/1349169661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x5595dce2b800 0x5595ddbf2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.175+0000 7fc7b5b22740 1 --2- 172.21.15.45:0/1349169661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x5595dce2ac00 0x5595ddbf2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.175+0000 7fc7b5b22740 1 -- 172.21.15.45:0/1349169661 >> 172.21.15.45:0/1349169661 conn(0x5595dce2b000 msgr2=0x5595ddba8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:30.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.175+0000 7fc7b5b22740 1 -- 172.21.15.45:0/1349169661 shutdown_connections 2024-05-08T05:51:30.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.175+0000 7fc7b5b22740 1 -- 172.21.15.45:0/1349169661 wait complete. 2024-05-08T05:51:30.181 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.126742, result: 2024-05-08T05:51:30.181 DEBUG:teuthology.orchestra.run.smithi045:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-05-08T05:51:30.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.264+0000 7fb3a882d740 1 -- start start 2024-05-08T05:51:30.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.264+0000 7fb3a882d740 1 Processor -- start 2024-05-08T05:51:30.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.264+0000 7fb3a882d740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ce1cff0c00 0x55ce1ddc4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.264+0000 7fb3a882d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1cff1800 0x55ce1ddc4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.264+0000 7fb3a882d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce1cff1c00 0x55ce1ddc4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.264+0000 7fb3a882d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ce1dddc180 con 0x55ce1cff1800 2024-05-08T05:51:30.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.264+0000 7fb3a882d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ce1dddc300 con 0x55ce1cff1c00 2024-05-08T05:51:30.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.264+0000 7fb3a882d740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55ce1dddc480 con 0x55ce1cff0c00 2024-05-08T05:51:30.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.264+0000 7fb3a5959640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1cff1800 0x55ce1ddc4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.264+0000 7fb3a5959640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1cff1800 0x55ce1ddc4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51622/0 (socket says 172.21.15.45:51622) 2024-05-08T05:51:30.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.264+0000 7fb3a5959640 1 -- 172.21.15.45:0/4169064468 learned_addr learned my addr 172.21.15.45:0/4169064468 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:30.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.265+0000 7fb3a695b640 1 --2- 172.21.15.45:0/4169064468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce1cff1c00 0x55ce1ddc4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.265+0000 7fb3a615a640 1 --2- 172.21.15.45:0/4169064468 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ce1cff0c00 0x55ce1ddc4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.265+0000 7fb3a5959640 1 -- 172.21.15.45:0/4169064468 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ce1cff0c00 msgr2=0x55ce1ddc4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:30.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.265+0000 7fb3a5959640 1 --2- 172.21.15.45:0/4169064468 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ce1cff0c00 0x55ce1ddc4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.265+0000 7fb3a5959640 1 -- 172.21.15.45:0/4169064468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce1cff1c00 msgr2=0x55ce1ddc4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:30.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.265+0000 7fb3a5959640 1 --2- 172.21.15.45:0/4169064468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce1cff1c00 0x55ce1ddc4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.265+0000 7fb3a5959640 1 -- 172.21.15.45:0/4169064468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce1d01d680 con 0x55ce1cff1800 2024-05-08T05:51:30.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.265+0000 7fb3a5959640 1 --2- 172.21.15.45:0/4169064468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1cff1800 0x55ce1ddc4580 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x55ce1de11bc0 tx=0x55ce1d0c0120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34eaa3bd5d9e06db server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.265+0000 7fb3a3955640 1 -- 172.21.15.45:0/4169064468 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ce1d016680 con 0x55ce1cff1800 2024-05-08T05:51:30.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.265+0000 7fb3a3955640 1 -- 172.21.15.45:0/4169064468 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ce1d0381c0 con 0x55ce1cff1800 2024-05-08T05:51:30.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.265+0000 7fb3a3955640 1 -- 172.21.15.45:0/4169064468 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ce1d016b60 con 0x55ce1cff1800 2024-05-08T05:51:30.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.265+0000 7fb3a882d740 1 -- 172.21.15.45:0/4169064468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce1d01d860 con 0x55ce1cff1800 2024-05-08T05:51:30.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.265+0000 7fb3a882d740 1 -- 172.21.15.45:0/4169064468 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55ce1d01d2c0 con 0x55ce1cff1800 2024-05-08T05:51:30.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.265+0000 7fb3a882d740 4 waiting for MDS map... 2024-05-08T05:51:30.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.265+0000 7fb3a3955640 1 -- 172.21.15.45:0/4169064468 <== mon.0 v2:172.21.15.45:3300/0 4 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x55ce1ded2a00 con 0x55ce1cff1800 2024-05-08T05:51:30.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.265+0000 7fb3a3955640 1 -- 172.21.15.45:0/4169064468 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 107) ==== 2885+0+0 (secure 0 0 0) 0x55ce1d049180 con 0x55ce1cff1800 2024-05-08T05:51:30.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.265+0000 7fb3a882d740 4 Got MDS map 107 2024-05-08T05:51:30.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.266+0000 7fb3a882d740 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-05-08T05:51:30.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.266+0000 7fb3a882d740 4 datascan.main: Using metadata pool output 2024-05-08T05:51:30.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.266+0000 7fb3a882d740 4 datascan.main: connecting to RADOS... 2024-05-08T05:51:30.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.266+0000 7fb3a882d740 1 Processor -- start 2024-05-08T05:51:30.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.266+0000 7fb3a882d740 1 -- start start 2024-05-08T05:51:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a882d740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ce1ded5400 0x55ce1ddc5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a882d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1ded5800 0x55ce1ddc5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a882d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce1ded5c00 0x55ce1ddc5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a882d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ce1dddc600 con 0x55ce1ded5800 2024-05-08T05:51:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a882d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ce1dddc780 con 0x55ce1ded5c00 2024-05-08T05:51:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a882d740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55ce1dddc900 con 0x55ce1ded5400 2024-05-08T05:51:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a615a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1ded5800 0x55ce1ddc5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a615a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1ded5800 0x55ce1ddc5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51626/0 (socket says 172.21.15.45:51626) 2024-05-08T05:51:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a615a640 1 -- 172.21.15.45:0/3328948025 learned_addr learned my addr 172.21.15.45:0/3328948025 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a5959640 1 --2- 172.21.15.45:0/3328948025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce1ded5c00 0x55ce1ddc5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a695b640 1 --2- 172.21.15.45:0/3328948025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ce1ded5400 0x55ce1ddc5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a615a640 1 -- 172.21.15.45:0/3328948025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ce1ded5400 msgr2=0x55ce1ddc5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a615a640 1 --2- 172.21.15.45:0/3328948025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ce1ded5400 0x55ce1ddc5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a615a640 1 -- 172.21.15.45:0/3328948025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce1ded5c00 msgr2=0x55ce1ddc5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:30.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a615a640 1 --2- 172.21.15.45:0/3328948025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce1ded5c00 0x55ce1ddc5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a615a640 1 -- 172.21.15.45:0/3328948025 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce1d01d4a0 con 0x55ce1ded5800 2024-05-08T05:51:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a5959640 1 --2- 172.21.15.45:0/3328948025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce1ded5c00 0x55ce1ddc5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a695b640 1 --2- 172.21.15.45:0/3328948025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ce1ded5400 0x55ce1ddc5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a615a640 1 --2- 172.21.15.45:0/3328948025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1ded5800 0x55ce1ddc5600 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x55ce1dee8f60 tx=0x55ce1deed800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2b703634fdd4f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a1951640 1 -- 172.21.15.45:0/3328948025 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ce1d016680 con 0x55ce1ded5800 2024-05-08T05:51:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a1951640 1 -- 172.21.15.45:0/3328948025 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ce1d038e00 con 0x55ce1ded5800 2024-05-08T05:51:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.267+0000 7fb3a1951640 1 -- 172.21.15.45:0/3328948025 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ce1d016b60 con 0x55ce1ded5800 2024-05-08T05:51:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.268+0000 7fb3a882d740 1 -- 172.21.15.45:0/3328948025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1ded5800 msgr2=0x55ce1ddc5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.268+0000 7fb3a882d740 1 --2- 172.21.15.45:0/3328948025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1ded5800 0x55ce1ddc5600 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x55ce1dee8f60 tx=0x55ce1deed800 comp rx=0 tx=0).stop 2024-05-08T05:51:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.268+0000 7fb3a882d740 1 -- 172.21.15.45:0/3328948025 shutdown_connections 2024-05-08T05:51:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.268+0000 7fb3a882d740 1 --2- 172.21.15.45:0/3328948025 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce1ded5c00 0x55ce1ddc5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.268+0000 7fb3a882d740 1 --2- 172.21.15.45:0/3328948025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1ded5800 0x55ce1ddc5600 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.268+0000 7fb3a882d740 1 --2- 172.21.15.45:0/3328948025 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ce1ded5400 0x55ce1ddc5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.268+0000 7fb3a882d740 1 -- 172.21.15.45:0/3328948025 >> 172.21.15.45:0/3328948025 conn(0x55ce1ded5000 msgr2=0x55ce1dd6f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.268+0000 7fb3a882d740 1 -- 172.21.15.45:0/3328948025 shutdown_connections 2024-05-08T05:51:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.268+0000 7fb3a882d740 1 -- 172.21.15.45:0/3328948025 wait complete. 2024-05-08T05:51:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.268+0000 7fb3a882d740 1 Processor -- start 2024-05-08T05:51:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.268+0000 7fb3a882d740 1 -- start start 2024-05-08T05:51:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.269+0000 7fb3a882d740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce1ded5400 0x55ce1ddc5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.269+0000 7fb3a882d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1ded5800 0x55ce1ddc5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.269+0000 7fb3a882d740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ce1ded5c00 0x55ce1ddc5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.269+0000 7fb3a882d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ce1ddddc80 con 0x55ce1ded5800 2024-05-08T05:51:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.269+0000 7fb3a882d740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55ce1ddddb00 con 0x55ce1ded5400 2024-05-08T05:51:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.269+0000 7fb3a882d740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55ce1dddd980 con 0x55ce1ded5c00 2024-05-08T05:51:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.269+0000 7fb3a615a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1ded5800 0x55ce1ddc5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.269+0000 7fb3a615a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1ded5800 0x55ce1ddc5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51638/0 (socket says 172.21.15.45:51638) 2024-05-08T05:51:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.269+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 learned_addr learned my addr 172.21.15.45:0/1956270364 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:30.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.269+0000 7fb3a695b640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce1ded5400 0x55ce1ddc5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.269+0000 7fb3a5959640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ce1ded5c00 0x55ce1ddc5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.269+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ce1ded5c00 msgr2=0x55ce1ddc5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.269+0000 7fb3a615a640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ce1ded5c00 0x55ce1ddc5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.269+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce1ded5400 msgr2=0x55ce1ddc5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.269+0000 7fb3a615a640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce1ded5400 0x55ce1ddc5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.269+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce1d01d680 con 0x55ce1ded5800 2024-05-08T05:51:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.270+0000 7fb3a615a640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1ded5800 0x55ce1ddc5600 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x55ce1dee9c20 tx=0x55ce1dd97f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.270+0000 7fb3a014e640 1 -- 172.21.15.45:0/1956270364 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ce1d0164e0 con 0x55ce1ded5800 2024-05-08T05:51:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.270+0000 7fb3a014e640 1 -- 172.21.15.45:0/1956270364 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ce1d038c40 con 0x55ce1ded5800 2024-05-08T05:51:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.270+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ce1d01d860 con 0x55ce1ded5800 2024-05-08T05:51:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.270+0000 7fb3a014e640 1 -- 172.21.15.45:0/1956270364 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ce1d016680 con 0x55ce1ded5800 2024-05-08T05:51:30.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.270+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce1d01d0e0 con 0x55ce1ded5800 2024-05-08T05:51:30.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.270+0000 7fb3a882d740 4 datascan.init: resolving metadata pool 17 2024-05-08T05:51:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.271+0000 7fb3a014e640 1 -- 172.21.15.45:0/1956270364 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x55ce1df42c00 con 0x55ce1ded5800 2024-05-08T05:51:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.271+0000 7fb3a014e640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x55ce1df48c00 0x55ce1ddc6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.272+0000 7fb3a014e640 1 -- 172.21.15.45:0/1956270364 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x55ce1df40a00 con 0x55ce1ded5800 2024-05-08T05:51:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.272+0000 7fb3a882d740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-08T05:51:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.272+0000 7fb3a882d740 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-05-08T05:51:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.272+0000 7fb3a882d740 4 datascan.main: opening data pool 'cephfs_data' 2024-05-08T05:51:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.272+0000 7fb3a882d740 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x55ce1df49000 0x55ce1ddc6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.272+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:1 17.14 17:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55ce1df49400 con 0x55ce1df49000 2024-05-08T05:51:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.272+0000 7fb3a695b640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x55ce1df48c00 0x55ce1ddc6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.272+0000 7fb3a5959640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x55ce1df49000 0x55ce1ddc6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.272+0000 7fb3a5959640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x55ce1df49000 0x55ce1ddc6680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.272+0000 7fb3a695b640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x55ce1df48c00 0x55ce1ddc6100 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55ce1dd98270 tx=0x55ce1dd96c60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.272+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.3 v2:172.21.15.45:6816/1259225446 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv33 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1df49000 2024-05-08T05:51:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.272+0000 7fb3a882d740 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55ce1df49800 0x55ce1ddc6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.272+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:2 17.1f 17:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55ce1df49c00 con 0x55ce1df49800 2024-05-08T05:51:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.273+0000 7fb3a695b640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55ce1df49800 0x55ce1ddc6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.273+0000 7fb3a695b640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55ce1df49800 0x55ce1ddc6c00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.273+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.6 v2:172.21.15.63:6800/2926759052 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1df49800 2024-05-08T05:51:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.273+0000 7fb3a882d740 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x55ce1d3b2000 0x55ce1ddc7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.273+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:3 18.0 18:00000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b2400 con 0x55ce1d3b2000 2024-05-08T05:51:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.274+0000 7fb3a615a640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x55ce1d3b2000 0x55ce1ddc7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.274+0000 7fb3a615a640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x55ce1d3b2000 0x55ce1ddc7180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.274+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.0 v2:172.21.15.45:6800/3734454393 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b2000 2024-05-08T05:51:30.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.274+0000 7fb3a1150640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55ce1d3b3800 0x55ce1d3be000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.274+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:4 18.20 18:04000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b3c00 con 0x55ce1d3b3800 2024-05-08T05:51:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.274+0000 7fb3a5959640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55ce1d3b3800 0x55ce1d3be000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.275+0000 7fb3a5959640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55ce1d3b3800 0x55ce1d3be000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.275+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.5 v2:172.21.15.63:6808/2395798255 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b3800 2024-05-08T05:51:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.275+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:5 18.10 18:08000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b4000 con 0x55ce1df49800 2024-05-08T05:51:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.275+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.6 v2:172.21.15.63:6800/2926759052 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1df49800 2024-05-08T05:51:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.275+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:6 18.30 18:0c000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b4400 con 0x55ce1d3b2000 2024-05-08T05:51:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.276+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.0 v2:172.21.15.45:6800/3734454393 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b2000 2024-05-08T05:51:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.276+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:7 18.8 18:10000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b5000 con 0x55ce1d3b2000 2024-05-08T05:51:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.276+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.0 v2:172.21.15.45:6800/3734454393 3 ==== osd_op_reply(7 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b2000 2024-05-08T05:51:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.276+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:8 18.28 18:14000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b4800 con 0x55ce1d3b3800 2024-05-08T05:51:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.276+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.5 v2:172.21.15.63:6808/2395798255 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b3800 2024-05-08T05:51:30.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.276+0000 7fb3a1951640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55ce1d3b5400 0x55ce1d3be580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.276+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:9 18.18 18:18000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b5800 con 0x55ce1d3b5400 2024-05-08T05:51:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.276+0000 7fb3a695b640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55ce1d3b5400 0x55ce1d3be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.277+0000 7fb3a695b640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55ce1d3b5400 0x55ce1d3be580 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.277+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.1 v2:172.21.15.45:6824/1337179184 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1d3b5400 2024-05-08T05:51:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.277+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:10 18.38 18:1c000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b4c00 con 0x55ce1df49000 2024-05-08T05:51:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.277+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.3 v2:172.21.15.45:6816/1259225446 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1df49000 2024-05-08T05:51:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.277+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:11 18.4 18:20000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b5c00 con 0x55ce1d3b2000 2024-05-08T05:51:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.277+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.0 v2:172.21.15.45:6800/3734454393 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b2000 2024-05-08T05:51:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.277+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:12 18.24 18:24000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1dddf400 con 0x55ce1d3b2000 2024-05-08T05:51:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.277+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.0 v2:172.21.15.45:6800/3734454393 5 ==== osd_op_reply(12 [pgnls-filter start_epoch 121 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b2000 2024-05-08T05:51:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.278+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:13 18.14 18:28000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b4400 con 0x55ce1df49800 2024-05-08T05:51:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.278+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.6 v2:172.21.15.63:6800/2926759052 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 121 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1df49800 2024-05-08T05:51:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.278+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:14 18.34 18:2c000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b2400 con 0x55ce1d3b2000 2024-05-08T05:51:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.278+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.0 v2:172.21.15.45:6800/3734454393 6 ==== osd_op_reply(14 [pgnls-filter start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b2000 2024-05-08T05:51:30.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.278+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:15 18.c 18:30000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1df49400 con 0x55ce1d3b3800 2024-05-08T05:51:30.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.278+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.5 v2:172.21.15.63:6808/2395798255 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b3800 2024-05-08T05:51:30.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.278+0000 7fb3a1150640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x55ce1d3b3c00 0x55ce1d3beb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.278+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:16 18.2c 18:34000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1ded4800 con 0x55ce1d3b3c00 2024-05-08T05:51:30.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.279+0000 7fb3a615a640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x55ce1d3b3c00 0x55ce1d3beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.279+0000 7fb3a615a640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x55ce1d3b3c00 0x55ce1d3beb00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.279+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.2 v2:172.21.15.45:6808/785834340 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b3c00 2024-05-08T05:51:30.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.279+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:17 18.1c 18:38000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1df49c00 con 0x55ce1d3b5400 2024-05-08T05:51:30.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.279+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.1 v2:172.21.15.45:6824/1337179184 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1d3b5400 2024-05-08T05:51:30.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.279+0000 7fb3a1150640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x55ce1d3b5000 0x55ce1d3bf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.279+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:18 18.3c 18:3c000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b5c00 con 0x55ce1d3b5000 2024-05-08T05:51:30.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.280+0000 7fb3a5959640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x55ce1d3b5000 0x55ce1d3bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.280+0000 7fb3a5959640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x55ce1d3b5000 0x55ce1d3bf080 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.280+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.7 v2:172.21.15.63:6816/1163469437 1 ==== osd_op_reply(18 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b5000 2024-05-08T05:51:30.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.280+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:19 18.2 18:40000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1dddf000 con 0x55ce1d3b5000 2024-05-08T05:51:30.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.280+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.7 v2:172.21.15.63:6816/1163469437 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b5000 2024-05-08T05:51:30.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.281+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:20 18.22 18:44000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b4800 con 0x55ce1df49000 2024-05-08T05:51:30.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.281+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.3 v2:172.21.15.45:6816/1259225446 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1df49000 2024-05-08T05:51:30.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.281+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:21 18.12 18:48000000::::head [pgnls-filter start_epoch 121 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b5800 con 0x55ce1d3b3c00 2024-05-08T05:51:30.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.281+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.2 v2:172.21.15.45:6808/785834340 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 121 out=81b] v121'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b3c00 2024-05-08T05:51:30.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.281+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:22 18.0 18:00000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b2800 con 0x55ce1d3b2000 2024-05-08T05:51:30.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.281+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.0 v2:172.21.15.45:6800/3734454393 7 ==== osd_op_reply(22 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b2000 2024-05-08T05:51:30.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.281+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:23 18.20 18:04000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b2400 con 0x55ce1d3b3800 2024-05-08T05:51:30.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.282+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.5 v2:172.21.15.63:6808/2395798255 4 ==== osd_op_reply(23 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b3800 2024-05-08T05:51:30.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.282+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:24 18.10 18:08000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b4c00 con 0x55ce1df49800 2024-05-08T05:51:30.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.282+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.6 v2:172.21.15.63:6800/2926759052 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1df49800 2024-05-08T05:51:30.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.282+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:25 18.30 18:0c000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b2800 con 0x55ce1d3b2000 2024-05-08T05:51:30.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.282+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.0 v2:172.21.15.45:6800/3734454393 8 ==== osd_op_reply(25 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b2000 2024-05-08T05:51:30.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.282+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:26 18.8 18:10000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b4000 con 0x55ce1d3b2000 2024-05-08T05:51:30.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.283+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.0 v2:172.21.15.45:6800/3734454393 9 ==== osd_op_reply(26 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b2000 2024-05-08T05:51:30.297 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.283+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:27 18.28 18:14000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b5800 con 0x55ce1d3b3800 2024-05-08T05:51:30.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.283+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.5 v2:172.21.15.63:6808/2395798255 5 ==== osd_op_reply(27 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b3800 2024-05-08T05:51:30.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.283+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:28 18.18 18:18000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b5c00 con 0x55ce1d3b5400 2024-05-08T05:51:30.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.283+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.1 v2:172.21.15.45:6824/1337179184 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1d3b5400 2024-05-08T05:51:30.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.283+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:29 18.38 18:1c000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1ded4800 con 0x55ce1df49000 2024-05-08T05:51:30.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.283+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.3 v2:172.21.15.45:6816/1259225446 4 ==== osd_op_reply(29 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1df49000 2024-05-08T05:51:30.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.283+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:30 18.4 18:20000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b4800 con 0x55ce1d3b2000 2024-05-08T05:51:30.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.284+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.0 v2:172.21.15.45:6800/3734454393 10 ==== osd_op_reply(30 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b2000 2024-05-08T05:51:30.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.284+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:31 18.24 18:24000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1dddf400 con 0x55ce1d3b2000 2024-05-08T05:51:30.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.284+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.0 v2:172.21.15.45:6800/3734454393 11 ==== osd_op_reply(31 [pgnls-filter start_epoch 121 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b2000 2024-05-08T05:51:30.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.284+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:32 18.14 18:28000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b5800 con 0x55ce1df49800 2024-05-08T05:51:30.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.284+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.6 v2:172.21.15.63:6800/2926759052 5 ==== osd_op_reply(32 [pgnls-filter start_epoch 121 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1df49800 2024-05-08T05:51:30.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.284+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:33 18.34 18:2c000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d4000 con 0x55ce1d3b2000 2024-05-08T05:51:30.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.284+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.0 v2:172.21.15.45:6800/3734454393 12 ==== osd_op_reply(33 [pgnls-filter start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b2000 2024-05-08T05:51:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.284+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:34 18.c 18:30000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b2400 con 0x55ce1d3b3800 2024-05-08T05:51:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.285+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.5 v2:172.21.15.63:6808/2395798255 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b3800 2024-05-08T05:51:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.285+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:35 18.2c 18:34000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d4400 con 0x55ce1d3b3c00 2024-05-08T05:51:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.285+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.2 v2:172.21.15.45:6808/785834340 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b3c00 2024-05-08T05:51:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.285+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:36 18.1c 18:38000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1dddf000 con 0x55ce1d3b5400 2024-05-08T05:51:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.285+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.1 v2:172.21.15.45:6824/1337179184 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1d3b5400 2024-05-08T05:51:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.285+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:37 18.3c 18:3c000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d4800 con 0x55ce1d3b5000 2024-05-08T05:51:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.286+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.7 v2:172.21.15.63:6816/1163469437 3 ==== osd_op_reply(37 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b5000 2024-05-08T05:51:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.286+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:38 18.2 18:40000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1df49400 con 0x55ce1d3b5000 2024-05-08T05:51:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.286+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.7 v2:172.21.15.63:6816/1163469437 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b5000 2024-05-08T05:51:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.286+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:39 18.22 18:44000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d4c00 con 0x55ce1df49000 2024-05-08T05:51:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.286+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.3 v2:172.21.15.45:6816/1259225446 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1df49000 2024-05-08T05:51:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.286+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:40 18.12 18:48000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b4400 con 0x55ce1d3b3c00 2024-05-08T05:51:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.287+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.2 v2:172.21.15.45:6808/785834340 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 121 out=49b] v121'23 uv23 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b3c00 2024-05-08T05:51:30.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.287+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:41 18.32 18:4c000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d5000 con 0x55ce1df49000 2024-05-08T05:51:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.287+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.3 v2:172.21.15.45:6816/1259225446 6 ==== osd_op_reply(41 [pgnls-filter start_epoch 121 out=49b] v119'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1df49000 2024-05-08T05:51:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.287+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:42 18.a 18:50000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b4c00 con 0x55ce1df49000 2024-05-08T05:51:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.287+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.3 v2:172.21.15.45:6816/1259225446 7 ==== osd_op_reply(42 [pgnls-filter start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1df49000 2024-05-08T05:51:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.287+0000 7fb3a1951640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x55ce1d3d5400 0x55ce1d3bf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.287+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:43 18.2a 18:54000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d5800 con 0x55ce1d3d5400 2024-05-08T05:51:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.287+0000 7fb3a695b640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x55ce1d3d5400 0x55ce1d3bf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.288+0000 7fb3a695b640 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x55ce1d3d5400 0x55ce1d3bf600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.288+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.4 v2:172.21.15.63:6824/2212879151 1 ==== osd_op_reply(43 [pgnls-filter start_epoch 121 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1d3d5400 2024-05-08T05:51:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.288+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:44 18.1a 18:58000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b5c00 con 0x55ce1df49000 2024-05-08T05:51:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.288+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.3 v2:172.21.15.45:6816/1259225446 8 ==== osd_op_reply(44 [pgnls-filter start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1df49000 2024-05-08T05:51:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.288+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:45 18.3a 18:5c000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d4800 con 0x55ce1d3b2000 2024-05-08T05:51:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.288+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.0 v2:172.21.15.45:6800/3734454393 13 ==== osd_op_reply(45 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b2000 2024-05-08T05:51:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.288+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:46 18.6 18:60000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b5800 con 0x55ce1d3b5000 2024-05-08T05:51:30.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.289+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.7 v2:172.21.15.63:6816/1163469437 5 ==== osd_op_reply(46 [pgnls-filter start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b5000 2024-05-08T05:51:30.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.289+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:47 18.26 18:64000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1dddf000 con 0x55ce1d3b3c00 2024-05-08T05:51:30.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.289+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.2 v2:172.21.15.45:6808/785834340 5 ==== osd_op_reply(47 [pgnls-filter start_epoch 121 out=49b] v119'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b3c00 2024-05-08T05:51:30.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.289+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:48 18.16 18:68000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b4000 con 0x55ce1df49000 2024-05-08T05:51:30.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.289+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.3 v2:172.21.15.45:6816/1259225446 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 121 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1df49000 2024-05-08T05:51:30.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.289+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:49 18.36 18:6c000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1dddf400 con 0x55ce1d3b5400 2024-05-08T05:51:30.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.290+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.1 v2:172.21.15.45:6824/1337179184 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 121 out=81b] v121'34 uv34 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1d3b5400 2024-05-08T05:51:30.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.290+0000 7fb3a882d740 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-05-08T05:51:30.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.290+0000 7fb3a882d740 10 datascan.operator(): handling object 100000001f7.0 2024-05-08T05:51:30.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.290+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:50 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b3000 con 0x55ce1d3b5400 2024-05-08T05:51:30.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.290+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.1 v2:172.21.15.45:6824/1337179184 6 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) ==== 416+0+194 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1d3b5400 2024-05-08T05:51:30.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.290+0000 7fb3a882d740 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-05-08T05:51:30.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.290+0000 7fb3a882d740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-05-08T05:51:30.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.290+0000 7fb3a882d740 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-05-08T05:51:30.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.290+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:51 17.7 17:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b3400 con 0x55ce1d3b2000 2024-05-08T05:51:30.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.290+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.0 v2:172.21.15.45:6800/3734454393 14 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b2000 2024-05-08T05:51:30.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.290+0000 7fb3a882d740 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-05-08T05:51:30.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.290+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:52 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d4c00 con 0x55ce1df49000 2024-05-08T05:51:30.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.291+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.3 v2:172.21.15.45:6816/1259225446 10 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=585b] v0'0 uv12 ondisk = 0) ==== 164+0+585 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1df49000 2024-05-08T05:51:30.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.291+0000 7fb3a882d740 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-05-08T05:51:30.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.291+0000 7fb3a882d740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-05-08T05:51:30.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.291+0000 7fb3a882d740 20 datascan.get_frag_of: frag is * 2024-05-08T05:51:30.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.291+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:53 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d5000 con 0x55ce1d3b2000 2024-05-08T05:51:30.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.291+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.0 v2:172.21.15.45:6800/3734454393 15 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b2000 2024-05-08T05:51:30.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.291+0000 7fb3a882d740 10 datascan.find_or_create_dirfrag: read_version = 8 2024-05-08T05:51:30.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.291+0000 7fb3a882d740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-05-08T05:51:30.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.291+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:54 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d4000 con 0x55ce1d3b2000 2024-05-08T05:51:30.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.291+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.0 v2:172.21.15.45:6800/3734454393 16 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b2000 2024-05-08T05:51:30.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.291+0000 7fb3a882d740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-05-08T05:51:30.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.291+0000 7fb3a882d740 20 datascan.read_dentry: bravo_head not found in result 2024-05-08T05:51:30.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.291+0000 7fb3a882d740 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-05-08T05:51:30.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.291+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:55 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d4400 con 0x55ce1d3b2000 2024-05-08T05:51:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.295+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.0 v2:172.21.15.45:6800/3734454393 17 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v121'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b2000 2024-05-08T05:51:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.295+0000 7fb3a882d740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-05-08T05:51:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.295+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:56 18.e 18:70000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d5c00 con 0x55ce1d3b2000 2024-05-08T05:51:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.295+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.0 v2:172.21.15.45:6800/3734454393 18 ==== osd_op_reply(56 [pgnls-filter start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b2000 2024-05-08T05:51:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.295+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:57 18.2e 18:74000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b4c00 con 0x55ce1df49000 2024-05-08T05:51:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.296+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.3 v2:172.21.15.45:6816/1259225446 11 ==== osd_op_reply(57 [pgnls-filter start_epoch 121 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1df49000 2024-05-08T05:51:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.296+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:58 18.1e 18:78000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b2800 con 0x55ce1d3b3800 2024-05-08T05:51:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.296+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.5 v2:172.21.15.63:6808/2395798255 7 ==== osd_op_reply(58 [pgnls-filter start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b3800 2024-05-08T05:51:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.296+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:59 18.3e 18:7c000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b4800 con 0x55ce1df49000 2024-05-08T05:51:30.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.296+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.3 v2:172.21.15.45:6816/1259225446 12 ==== osd_op_reply(59 [pgnls-filter start_epoch 121 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1df49000 2024-05-08T05:51:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.296+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:60 18.1 18:80000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1df49c00 con 0x55ce1df49000 2024-05-08T05:51:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.297+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.3 v2:172.21.15.45:6816/1259225446 13 ==== osd_op_reply(60 [pgnls-filter start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1df49000 2024-05-08T05:51:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.297+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:61 18.21 18:84000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1df49400 con 0x55ce1d3b3800 2024-05-08T05:51:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.297+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.5 v2:172.21.15.63:6808/2395798255 8 ==== osd_op_reply(61 [pgnls-filter start_epoch 121 out=49b] v119'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b3800 2024-05-08T05:51:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.297+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:62 18.11 18:88000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1dddf000 con 0x55ce1d3b5400 2024-05-08T05:51:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.297+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.1 v2:172.21.15.45:6824/1337179184 7 ==== osd_op_reply(62 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1d3b5400 2024-05-08T05:51:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.297+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:63 18.31 18:8c000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b2400 con 0x55ce1d3d5400 2024-05-08T05:51:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.298+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.4 v2:172.21.15.63:6824/2212879151 2 ==== osd_op_reply(63 [pgnls-filter start_epoch 121 out=49b] v119'15 uv8 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1d3d5400 2024-05-08T05:51:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.298+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:64 18.9 18:90000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b5c00 con 0x55ce1df49000 2024-05-08T05:51:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.298+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.3 v2:172.21.15.45:6816/1259225446 14 ==== osd_op_reply(64 [pgnls-filter start_epoch 121 out=49b] v119'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1df49000 2024-05-08T05:51:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.298+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:65 18.29 18:94000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1ded4800 con 0x55ce1d3b5000 2024-05-08T05:51:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.298+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.7 v2:172.21.15.63:6816/1163469437 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b5000 2024-05-08T05:51:30.313 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.298+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:66 18.19 18:98000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b5800 con 0x55ce1d3d5400 2024-05-08T05:51:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.299+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.4 v2:172.21.15.63:6824/2212879151 3 ==== osd_op_reply(66 [pgnls-filter start_epoch 121 out=49b] v121'24 uv24 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1d3d5400 2024-05-08T05:51:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.299+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:67 18.39 18:9c000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d5800 con 0x55ce1d3b2000 2024-05-08T05:51:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.299+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.0 v2:172.21.15.45:6800/3734454393 19 ==== osd_op_reply(67 [pgnls-filter start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b2000 2024-05-08T05:51:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.299+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:68 18.5 18:a0000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b4000 con 0x55ce1d3b5400 2024-05-08T05:51:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.299+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.1 v2:172.21.15.45:6824/1337179184 8 ==== osd_op_reply(68 [pgnls-filter start_epoch 121 out=49b] v121'15 uv15 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1d3b5400 2024-05-08T05:51:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.299+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:69 18.25 18:a4000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b3000 con 0x55ce1df49800 2024-05-08T05:51:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.300+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.6 v2:172.21.15.63:6800/2926759052 6 ==== osd_op_reply(69 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1df49800 2024-05-08T05:51:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.300+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:70 18.15 18:a8000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b4c00 con 0x55ce1d3b3800 2024-05-08T05:51:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.300+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.5 v2:172.21.15.63:6808/2395798255 9 ==== osd_op_reply(70 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b3800 2024-05-08T05:51:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.300+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:71 18.35 18:ac000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b2c00 con 0x55ce1d3d5400 2024-05-08T05:51:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.300+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.4 v2:172.21.15.63:6824/2212879151 4 ==== osd_op_reply(71 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1d3d5400 2024-05-08T05:51:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.300+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:72 18.d 18:b0000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b4800 con 0x55ce1d3d5400 2024-05-08T05:51:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.301+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.4 v2:172.21.15.63:6824/2212879151 5 ==== osd_op_reply(72 [pgnls-filter start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1d3d5400 2024-05-08T05:51:30.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.301+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:73 18.2d 18:b4000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b2400 con 0x55ce1df49800 2024-05-08T05:51:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.301+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.6 v2:172.21.15.63:6800/2926759052 7 ==== osd_op_reply(73 [pgnls-filter start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1df49800 2024-05-08T05:51:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.301+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:74 18.1d 18:b8000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b4400 con 0x55ce1d3b2000 2024-05-08T05:51:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.301+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.0 v2:172.21.15.45:6800/3734454393 20 ==== osd_op_reply(74 [pgnls-filter start_epoch 121 out=49b] v119'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b2000 2024-05-08T05:51:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.301+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:75 18.3d 18:bc000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b2800 con 0x55ce1df49000 2024-05-08T05:51:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.302+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.3 v2:172.21.15.45:6816/1259225446 15 ==== osd_op_reply(75 [pgnls-filter start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1df49000 2024-05-08T05:51:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.302+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:76 18.3 18:c0000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1df49c00 con 0x55ce1d3b5400 2024-05-08T05:51:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.302+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.1 v2:172.21.15.45:6824/1337179184 9 ==== osd_op_reply(76 [pgnls-filter start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1d3b5400 2024-05-08T05:51:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.302+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:77 18.23 18:c4000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b3400 con 0x55ce1d3b3c00 2024-05-08T05:51:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.302+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.2 v2:172.21.15.45:6808/785834340 6 ==== osd_op_reply(77 [pgnls-filter start_epoch 121 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b3c00 2024-05-08T05:51:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.302+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:78 18.13 18:c8000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d4c00 con 0x55ce1d3b5000 2024-05-08T05:51:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.303+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.7 v2:172.21.15.63:6816/1163469437 7 ==== osd_op_reply(78 [pgnls-filter start_epoch 121 out=49b] v119'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b5000 2024-05-08T05:51:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.303+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:79 18.33 18:cc000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b5800 con 0x55ce1d3b5400 2024-05-08T05:51:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.303+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.1 v2:172.21.15.45:6824/1337179184 10 ==== osd_op_reply(79 [pgnls-filter start_epoch 121 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1d3b5400 2024-05-08T05:51:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.303+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:80 18.b 18:d0000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d4800 con 0x55ce1d3b3800 2024-05-08T05:51:30.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.303+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.5 v2:172.21.15.63:6808/2395798255 10 ==== osd_op_reply(80 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b3800 2024-05-08T05:51:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.303+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:81 18.2b 18:d4000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1dddf400 con 0x55ce1d3b5000 2024-05-08T05:51:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.303+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.7 v2:172.21.15.63:6816/1163469437 8 ==== osd_op_reply(81 [pgnls-filter start_epoch 121 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b5000 2024-05-08T05:51:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.303+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:82 18.1b 18:d8000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d5000 con 0x55ce1d3b5400 2024-05-08T05:51:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.304+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.1 v2:172.21.15.45:6824/1337179184 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1d3b5400 2024-05-08T05:51:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.304+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:83 18.3b 18:dc000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1dddf000 con 0x55ce1d3d5400 2024-05-08T05:51:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.304+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.4 v2:172.21.15.63:6824/2212879151 6 ==== osd_op_reply(83 [pgnls-filter start_epoch 121 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1d3d5400 2024-05-08T05:51:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.304+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:84 18.7 18:e0000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d4000 con 0x55ce1d3b3c00 2024-05-08T05:51:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.304+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 <== osd.2 v2:172.21.15.45:6808/785834340 7 ==== osd_op_reply(84 [pgnls-filter start_epoch 121 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82780 con 0x55ce1d3b3c00 2024-05-08T05:51:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.304+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:85 18.27 18:e4000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d5000 con 0x55ce1d3b3800 2024-05-08T05:51:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.305+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.5 v2:172.21.15.63:6808/2395798255 11 ==== osd_op_reply(85 [pgnls-filter start_epoch 121 out=49b] v119'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b3800 2024-05-08T05:51:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.305+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:86 18.17 18:e8000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d4400 con 0x55ce1d3b3800 2024-05-08T05:51:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.305+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.5 v2:172.21.15.63:6808/2395798255 12 ==== osd_op_reply(86 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b3800 2024-05-08T05:51:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.305+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:87 18.37 18:ec000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b4000 con 0x55ce1d3b5000 2024-05-08T05:51:30.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.305+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.7 v2:172.21.15.63:6816/1163469437 9 ==== osd_op_reply(87 [pgnls-filter start_epoch 121 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b5000 2024-05-08T05:51:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.305+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:88 18.f 18:f0000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d4c00 con 0x55ce1d3b3800 2024-05-08T05:51:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.306+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.5 v2:172.21.15.63:6808/2395798255 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 121 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b3800 2024-05-08T05:51:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.306+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:89 18.2f 18:f4000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b4800 con 0x55ce1d3b5400 2024-05-08T05:51:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.306+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.1 v2:172.21.15.45:6824/1337179184 12 ==== osd_op_reply(89 [pgnls-filter start_epoch 121 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1d3b5400 2024-05-08T05:51:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.306+0000 7fb3a1150640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:90 18.1f 18:f8000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3d4800 con 0x55ce1d3b5000 2024-05-08T05:51:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.306+0000 7fb3a5959640 1 -- 172.21.15.45:0/1956270364 <== osd.7 v2:172.21.15.63:6816/1163469437 10 ==== osd_op_reply(90 [pgnls-filter start_epoch 121 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82000 con 0x55ce1d3b5000 2024-05-08T05:51:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.306+0000 7fb3a1951640 1 -- 172.21.15.45:0/1956270364 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:91 18.3f 18:fc000000::::head [pgnls-filter start_epoch 121 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55ce1d3b5800 con 0x55ce1d3d5400 2024-05-08T05:51:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a695b640 1 -- 172.21.15.45:0/1956270364 <== osd.4 v2:172.21.15.63:6824/2212879151 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 121 out=49b] v119'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ce1dd82500 con 0x55ce1d3d5400 2024-05-08T05:51:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x55ce1d3b2000 msgr2=0x55ce1ddc7180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x55ce1d3b2000 0x55ce1ddc7180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55ce1d3b5400 msgr2=0x55ce1d3be580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55ce1d3b5400 0x55ce1d3be580 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x55ce1d3b3c00 msgr2=0x55ce1d3beb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x55ce1d3b3c00 0x55ce1d3beb00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x55ce1df49000 msgr2=0x55ce1ddc6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x55ce1df49000 0x55ce1ddc6680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 reap_dead start 2024-05-08T05:51:30.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x55ce1d3d5400 msgr2=0x55ce1d3bf600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x55ce1d3d5400 0x55ce1d3bf600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55ce1d3b3800 msgr2=0x55ce1d3be000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55ce1d3b3800 0x55ce1d3be000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55ce1df49800 msgr2=0x55ce1ddc6c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55ce1df49800 0x55ce1ddc6c00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x55ce1d3b5000 msgr2=0x55ce1d3bf080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x55ce1d3b5000 0x55ce1d3bf080 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a615a640 1 -- 172.21.15.45:0/1956270364 reap_dead start 2024-05-08T05:51:30.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x55ce1df48c00 msgr2=0x55ce1ddc6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x55ce1df48c00 0x55ce1ddc6100 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55ce1dd98270 tx=0x55ce1dd96c60 comp rx=0 tx=0).stop 2024-05-08T05:51:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1ded5800 msgr2=0x55ce1ddc5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1ded5800 0x55ce1ddc5600 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x55ce1dee9c20 tx=0x55ce1dd97f80 comp rx=0 tx=0).stop 2024-05-08T05:51:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 shutdown_connections 2024-05-08T05:51:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x55ce1df48c00 0x55ce1ddc6100 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 --2- 172.21.15.45:0/1956270364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1ded5800 0x55ce1ddc5600 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 >> 172.21.15.45:0/1956270364 conn(0x55ce1ded5000 msgr2=0x55ce1dd6f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 shutdown_connections 2024-05-08T05:51:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 -- 172.21.15.45:0/1956270364 wait complete. 2024-05-08T05:51:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 -- 172.21.15.45:0/4169064468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1cff1800 msgr2=0x55ce1ddc4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 --2- 172.21.15.45:0/4169064468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1cff1800 0x55ce1ddc4580 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x55ce1de11bc0 tx=0x55ce1d0c0120 comp rx=0 tx=0).stop 2024-05-08T05:51:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 -- 172.21.15.45:0/4169064468 shutdown_connections 2024-05-08T05:51:30.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 --2- 172.21.15.45:0/4169064468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55ce1cff1c00 0x55ce1ddc4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 --2- 172.21.15.45:0/4169064468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ce1cff1800 0x55ce1ddc4580 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 --2- 172.21.15.45:0/4169064468 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55ce1cff0c00 0x55ce1ddc4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 -- 172.21.15.45:0/4169064468 >> 172.21.15.45:0/4169064468 conn(0x55ce1cff1000 msgr2=0x55ce1dd6e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:30.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 -- 172.21.15.45:0/4169064468 shutdown_connections 2024-05-08T05:51:30.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.307+0000 7fb3a882d740 1 -- 172.21.15.45:0/4169064468 wait complete. 2024-05-08T05:51:30.320 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.138944, result: 2024-05-08T05:51:30.320 DEBUG:teuthology.orchestra.run.smithi045:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-05-08T05:51:30.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.394+0000 7f2434927740 1 -- start start 2024-05-08T05:51:30.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.394+0000 7f2434927740 1 Processor -- start 2024-05-08T05:51:30.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.394+0000 7f2434927740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5a9878c00 0x55f5aa648000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f2434927740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5a9879800 0x55f5aa648580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f2434927740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f5a9879c00 0x55f5aa648b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f2434927740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f5aa670180 con 0x55f5a9878c00 2024-05-08T05:51:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f2434927740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f5aa670300 con 0x55f5a9879800 2024-05-08T05:51:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f2434927740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f5aa670480 con 0x55f5a9879c00 2024-05-08T05:51:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f24327ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5a9878c00 0x55f5aa648000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f24327ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5a9878c00 0x55f5aa648000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51648/0 (socket says 172.21.15.45:51648) 2024-05-08T05:51:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f24327ba640 1 -- 172.21.15.45:0/1982119801 learned_addr learned my addr 172.21.15.45:0/1982119801 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f2431fb9640 1 --2- 172.21.15.45:0/1982119801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5a9879800 0x55f5aa648580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f2432fbb640 1 --2- 172.21.15.45:0/1982119801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f5a9879c00 0x55f5aa648b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f24327ba640 1 -- 172.21.15.45:0/1982119801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f5a9879c00 msgr2=0x55f5aa648b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f24327ba640 1 --2- 172.21.15.45:0/1982119801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f5a9879c00 0x55f5aa648b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f24327ba640 1 -- 172.21.15.45:0/1982119801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5a9879800 msgr2=0x55f5aa648580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f24327ba640 1 --2- 172.21.15.45:0/1982119801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5a9879800 0x55f5aa648580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f24327ba640 1 -- 172.21.15.45:0/1982119801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5a98a5680 con 0x55f5a9878c00 2024-05-08T05:51:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f24327ba640 1 --2- 172.21.15.45:0/1982119801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5a9878c00 0x55f5aa648000 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x55f5aa6a7c50 tx=0x55f5a9948120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee685f3c52ee578 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f242ffb5640 1 -- 172.21.15.45:0/1982119801 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f5a989e680 con 0x55f5a9878c00 2024-05-08T05:51:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f242ffb5640 1 -- 172.21.15.45:0/1982119801 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55f5a98c01c0 con 0x55f5a9878c00 2024-05-08T05:51:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f2434927740 1 -- 172.21.15.45:0/1982119801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5a98a5a40 con 0x55f5a9878c00 2024-05-08T05:51:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f2434927740 1 -- 172.21.15.45:0/1982119801 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55f5a98a52c0 con 0x55f5a9878c00 2024-05-08T05:51:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f2434927740 4 waiting for MDS map... 2024-05-08T05:51:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f242ffb5640 1 -- 172.21.15.45:0/1982119801 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f5a989e9c0 con 0x55f5a9878c00 2024-05-08T05:51:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.395+0000 7f242ffb5640 1 -- 172.21.15.45:0/1982119801 <== mon.0 v2:172.21.15.45:3300/0 4 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x55f5aa656600 con 0x55f5a9878c00 2024-05-08T05:51:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.396+0000 7f242ffb5640 1 -- 172.21.15.45:0/1982119801 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 107) ==== 2885+0+0 (secure 0 0 0) 0x55f5a98d1180 con 0x55f5a9878c00 2024-05-08T05:51:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.396+0000 7f2434927740 4 Got MDS map 107 2024-05-08T05:51:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.396+0000 7f2434927740 4 datascan.main: Using metadata pool output 2024-05-08T05:51:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.396+0000 7f2434927740 4 datascan.main: connecting to RADOS... 2024-05-08T05:51:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.396+0000 7f2434927740 1 Processor -- start 2024-05-08T05:51:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.396+0000 7f2434927740 1 -- start start 2024-05-08T05:51:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.397+0000 7f2434927740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5aa667400 0x55f5aa649080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.397+0000 7f2434927740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f5aa667800 0x55f5aa649600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.397+0000 7f2434927740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5aa667c00 0x55f5aa649b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.397+0000 7f2434927740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f5aa670600 con 0x55f5aa667400 2024-05-08T05:51:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.397+0000 7f2434927740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f5aa670780 con 0x55f5aa667c00 2024-05-08T05:51:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.397+0000 7f2434927740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f5aa670900 con 0x55f5aa667800 2024-05-08T05:51:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.397+0000 7f2432fbb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5aa667400 0x55f5aa649080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.397+0000 7f2432fbb640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5aa667400 0x55f5aa649080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51656/0 (socket says 172.21.15.45:51656) 2024-05-08T05:51:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.397+0000 7f2432fbb640 1 -- 172.21.15.45:0/863557271 learned_addr learned my addr 172.21.15.45:0/863557271 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.397+0000 7f24327ba640 1 --2- 172.21.15.45:0/863557271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f5aa667800 0x55f5aa649600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.397+0000 7f2431fb9640 1 --2- 172.21.15.45:0/863557271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5aa667c00 0x55f5aa649b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.397+0000 7f2432fbb640 1 -- 172.21.15.45:0/863557271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f5aa667800 msgr2=0x55f5aa649600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.397+0000 7f2432fbb640 1 --2- 172.21.15.45:0/863557271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f5aa667800 0x55f5aa649600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.397+0000 7f2432fbb640 1 -- 172.21.15.45:0/863557271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5aa667c00 msgr2=0x55f5aa649b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.397+0000 7f2432fbb640 1 --2- 172.21.15.45:0/863557271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5aa667c00 0x55f5aa649b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.415 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.397+0000 7f2432fbb640 1 -- 172.21.15.45:0/863557271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5a98a54a0 con 0x55f5aa667400 2024-05-08T05:51:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.397+0000 7f24327ba640 1 --2- 172.21.15.45:0/863557271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f5aa667800 0x55f5aa649600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.397+0000 7f2432fbb640 1 --2- 172.21.15.45:0/863557271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5aa667400 0x55f5aa649080 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x55f5aa76cba0 tx=0x55f5aa61ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b731bed0e908e484 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.397+0000 7f242dfb1640 1 -- 172.21.15.45:0/863557271 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f5a989e680 con 0x55f5aa667400 2024-05-08T05:51:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.397+0000 7f242dfb1640 1 -- 172.21.15.45:0/863557271 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55f5a98c0e00 con 0x55f5aa667400 2024-05-08T05:51:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.398+0000 7f242dfb1640 1 -- 172.21.15.45:0/863557271 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f5a989e9c0 con 0x55f5aa667400 2024-05-08T05:51:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.398+0000 7f2434927740 1 -- 172.21.15.45:0/863557271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5aa667400 msgr2=0x55f5aa649080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.398+0000 7f2434927740 1 --2- 172.21.15.45:0/863557271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5aa667400 0x55f5aa649080 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x55f5aa76cba0 tx=0x55f5aa61ea80 comp rx=0 tx=0).stop 2024-05-08T05:51:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.398+0000 7f2434927740 1 -- 172.21.15.45:0/863557271 shutdown_connections 2024-05-08T05:51:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.398+0000 7f2434927740 1 --2- 172.21.15.45:0/863557271 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f5aa667800 0x55f5aa649600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.398+0000 7f2434927740 1 --2- 172.21.15.45:0/863557271 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5aa667c00 0x55f5aa649b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.398+0000 7f2434927740 1 --2- 172.21.15.45:0/863557271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5aa667400 0x55f5aa649080 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.398+0000 7f2434927740 1 -- 172.21.15.45:0/863557271 >> 172.21.15.45:0/863557271 conn(0x55f5aa667000 msgr2=0x55f5aa5f7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.398+0000 7f2434927740 1 -- 172.21.15.45:0/863557271 shutdown_connections 2024-05-08T05:51:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.398+0000 7f2434927740 1 -- 172.21.15.45:0/863557271 wait complete. 2024-05-08T05:51:30.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.398+0000 7f2434927740 1 Processor -- start 2024-05-08T05:51:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.399+0000 7f2434927740 1 -- start start 2024-05-08T05:51:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.399+0000 7f2434927740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5aa667400 0x55f5aa649080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.399+0000 7f2434927740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5aa667800 0x55f5aa649600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.399+0000 7f2434927740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f5aa667c00 0x55f5aa649b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.399+0000 7f2434927740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f5aa671980 con 0x55f5aa667800 2024-05-08T05:51:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.399+0000 7f2434927740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x55f5aa671800 con 0x55f5aa667400 2024-05-08T05:51:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.399+0000 7f2434927740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x55f5aa671680 con 0x55f5aa667c00 2024-05-08T05:51:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.399+0000 7f24327ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5aa667800 0x55f5aa649600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.399+0000 7f24327ba640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5aa667800 0x55f5aa649600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51658/0 (socket says 172.21.15.45:51658) 2024-05-08T05:51:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.399+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 learned_addr learned my addr 172.21.15.45:0/927209153 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.399+0000 7f2432fbb640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5aa667400 0x55f5aa649080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.399+0000 7f2431fb9640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f5aa667c00 0x55f5aa649b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.400+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f5aa667c00 msgr2=0x55f5aa649b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.400+0000 7f24327ba640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f5aa667c00 0x55f5aa649b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.400+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5aa667400 msgr2=0x55f5aa649080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.400+0000 7f24327ba640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5aa667400 0x55f5aa649080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.400+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5a98a5680 con 0x55f5aa667800 2024-05-08T05:51:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.400+0000 7f24327ba640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5aa667800 0x55f5aa649600 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x55f5aa76d8f0 tx=0x55f5aa61eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.400+0000 7f242c7ae640 1 -- 172.21.15.45:0/927209153 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f5a989eb60 con 0x55f5aa667800 2024-05-08T05:51:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.400+0000 7f242c7ae640 1 -- 172.21.15.45:0/927209153 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55f5a98c0e00 con 0x55f5aa667800 2024-05-08T05:51:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.400+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f5a98a5a40 con 0x55f5aa667800 2024-05-08T05:51:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.400+0000 7f242c7ae640 1 -- 172.21.15.45:0/927209153 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f5a989e680 con 0x55f5aa667800 2024-05-08T05:51:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.400+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5a98a50e0 con 0x55f5aa667800 2024-05-08T05:51:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.400+0000 7f2434927740 4 datascan.init: resolving metadata pool 17 2024-05-08T05:51:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.401+0000 7f242c7ae640 1 -- 172.21.15.45:0/927209153 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x55f5aa7d6c00 con 0x55f5aa667800 2024-05-08T05:51:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.402+0000 7f242c7ae640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x55f5aa7db000 0x55f5aa64a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.402+0000 7f242c7ae640 1 -- 172.21.15.45:0/927209153 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x55f5aa7d4a00 con 0x55f5aa667800 2024-05-08T05:51:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.402+0000 7f2434927740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-05-08T05:51:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.402+0000 7f2434927740 4 datascan.main: resolving metadata pool 17 2024-05-08T05:51:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.402+0000 7f2434927740 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x55f5aa7db400 0x55f5aa64a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.402+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa7db800 con 0x55f5aa7db400 2024-05-08T05:51:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.402+0000 7f2432fbb640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x55f5aa7db000 0x55f5aa64a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.402+0000 7f2431fb9640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x55f5aa7db400 0x55f5aa64a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.403+0000 7f2432fbb640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x55f5aa7db000 0x55f5aa64a100 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x55f5aa76cba0 tx=0x55f5aa61e600 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.403+0000 7f2431fb9640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x55f5aa7db400 0x55f5aa64a680 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.403+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.7 v2:172.21.15.63:6816/1163469437 1 ==== osd_op_reply(1 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7db400 2024-05-08T05:51:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.403+0000 7f2431fb9640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x55f5aa7db800 0x55f5a9c2a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.403+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:2 17.10 17:08000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2e000 con 0x55f5aa7db800 2024-05-08T05:51:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.403+0000 7f2432fbb640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x55f5aa7db800 0x55f5a9c2a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.403+0000 7f2432fbb640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x55f5aa7db800 0x55f5a9c2a000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.404+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 1 ==== osd_op_reply(2 [pgnls start_epoch 121 out=177b] v120'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.404+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:3 17.8 17:10000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2e000 con 0x55f5aa7db400 2024-05-08T05:51:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.404+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.7 v2:172.21.15.63:6816/1163469437 2 ==== osd_op_reply(3 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7db400 2024-05-08T05:51:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.404+0000 7f2431fb9640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x55f5a9c2e000 0x55f5a9c2a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.404+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:4 17.18 17:18000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2e400 con 0x55f5a9c2e000 2024-05-08T05:51:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.404+0000 7f24327ba640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x55f5a9c2e000 0x55f5a9c2a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.404+0000 7f24327ba640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x55f5a9c2e000 0x55f5a9c2a580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.405+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.0 v2:172.21.15.45:6800/3734454393 1 ==== osd_op_reply(4 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2e000 2024-05-08T05:51:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.405+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:5 17.4 17:20000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2e400 con 0x55f5a9c2e000 2024-05-08T05:51:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.405+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.0 v2:172.21.15.45:6800/3734454393 2 ==== osd_op_reply(5 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2e000 2024-05-08T05:51:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.405+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:6 17.14 17:28000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2e400 con 0x55f5aa7db800 2024-05-08T05:51:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.405+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 2 ==== osd_op_reply(6 [pgnls start_epoch 121 out=149b] v121'40 uv40 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.405+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:7 17.c 17:30000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2e400 con 0x55f5aa7db400 2024-05-08T05:51:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.406+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.7 v2:172.21.15.63:6816/1163469437 3 ==== osd_op_reply(7 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7db400 2024-05-08T05:51:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.406+0000 7f2431fb9640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55f5a9c2e400 0x55f5a9c2ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.406+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:8 17.1c 17:38000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2e800 con 0x55f5a9c2e400 2024-05-08T05:51:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.406+0000 7f2431fb9640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55f5a9c2e400 0x55f5a9c2ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.406+0000 7f2431fb9640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55f5a9c2e400 0x55f5a9c2ab00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.406+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.1 v2:172.21.15.45:6824/1337179184 1 ==== osd_op_reply(8 [pgnls start_epoch 121 out=73b] v120'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5a9c2e400 2024-05-08T05:51:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.406+0000 7f2431fb9640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x55f5a9c2e800 0x55f5a9c2b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.406+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:9 17.2 17:40000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2ec00 con 0x55f5a9c2e800 2024-05-08T05:51:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.406+0000 7f2432fbb640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x55f5a9c2e800 0x55f5a9c2b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.407+0000 7f2432fbb640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x55f5a9c2e800 0x55f5a9c2b080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.407+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.4 v2:172.21.15.63:6824/2212879151 1 ==== osd_op_reply(9 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5a9c2e800 2024-05-08T05:51:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.407+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:10 17.12 17:48000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2ec00 con 0x55f5aa7db800 2024-05-08T05:51:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.407+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 3 ==== osd_op_reply(10 [pgnls start_epoch 121 out=81b] v120'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.407+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:11 17.a 17:50000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2ec00 con 0x55f5aa7db400 2024-05-08T05:51:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.408+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.7 v2:172.21.15.63:6816/1163469437 4 ==== osd_op_reply(11 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7db400 2024-05-08T05:51:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.408+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:12 17.1a 17:58000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2ec00 con 0x55f5a9c2e000 2024-05-08T05:51:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.408+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.0 v2:172.21.15.45:6800/3734454393 3 ==== osd_op_reply(12 [pgnls start_epoch 121 out=49b] v121'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2e000 2024-05-08T05:51:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.408+0000 7f24327ba640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55f5a9c2ec00 0x55f5a9c2c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.408+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:13 17.6 17:60000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa7da800 con 0x55f5a9c2ec00 2024-05-08T05:51:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.408+0000 7f24327ba640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55f5a9c2ec00 0x55f5a9c2c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.408+0000 7f24327ba640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55f5a9c2ec00 0x55f5a9c2c100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.409+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.6 v2:172.21.15.63:6800/2926759052 1 ==== osd_op_reply(13 [pgnls start_epoch 121 out=77b] v120'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2ec00 2024-05-08T05:51:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.409+0000 7f24327ba640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55f5aa7da800 0x55f5a9c2c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.409+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:14 17.16 17:68000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa7dac00 con 0x55f5aa7da800 2024-05-08T05:51:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.409+0000 7f2431fb9640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55f5aa7da800 0x55f5a9c2c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.409+0000 7f2431fb9640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55f5aa7da800 0x55f5a9c2c680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.410+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.5 v2:172.21.15.63:6808/2395798255 1 ==== osd_op_reply(14 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7da800 2024-05-08T05:51:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.410+0000 7f2431fb9640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x55f5aa7dac00 0x55f5a9c2b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.410+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:15 17.e 17:70000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2f000 con 0x55f5aa7dac00 2024-05-08T05:51:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.410+0000 7f2432fbb640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x55f5aa7dac00 0x55f5a9c2b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.410+0000 7f2432fbb640 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x55f5aa7dac00 0x55f5a9c2b600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.410+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.2 v2:172.21.15.45:6808/785834340 1 ==== osd_op_reply(15 [pgnls start_epoch 121 out=73b] v120'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7dac00 2024-05-08T05:51:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.410+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:16 17.1e 17:78000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2f000 con 0x55f5a9c2e400 2024-05-08T05:51:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.411+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.1 v2:172.21.15.45:6824/1337179184 2 ==== osd_op_reply(16 [pgnls start_epoch 121 out=49b] v121'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5a9c2e400 2024-05-08T05:51:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.411+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:17 17.1 17:80000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2f000 con 0x55f5a9c2e000 2024-05-08T05:51:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.411+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.0 v2:172.21.15.45:6800/3734454393 4 ==== osd_op_reply(17 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2e000 2024-05-08T05:51:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.411+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:18 17.11 17:88000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2f000 con 0x55f5aa7db800 2024-05-08T05:51:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.411+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 4 ==== osd_op_reply(18 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.411+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:19 17.9 17:90000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2f000 con 0x55f5aa7db800 2024-05-08T05:51:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.411+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 5 ==== osd_op_reply(19 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.411+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:20 17.19 17:98000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2f000 con 0x55f5aa7da800 2024-05-08T05:51:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.412+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.5 v2:172.21.15.63:6808/2395798255 2 ==== osd_op_reply(20 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7da800 2024-05-08T05:51:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.412+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:21 17.5 17:a0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2f000 con 0x55f5a9c2ec00 2024-05-08T05:51:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.412+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.6 v2:172.21.15.63:6800/2926759052 2 ==== osd_op_reply(21 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2ec00 2024-05-08T05:51:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.412+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:22 17.15 17:a8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2f000 con 0x55f5aa7da800 2024-05-08T05:51:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.412+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.5 v2:172.21.15.63:6808/2395798255 3 ==== osd_op_reply(22 [pgnls start_epoch 121 out=97b] v120'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7da800 2024-05-08T05:51:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.412+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:23 17.d 17:b0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2f000 con 0x55f5aa7db400 2024-05-08T05:51:30.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.412+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.7 v2:172.21.15.63:6816/1163469437 5 ==== osd_op_reply(23 [pgnls start_epoch 121 out=74b] v118'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7db400 2024-05-08T05:51:30.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.413+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:24 17.1d 17:b8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2f000 con 0x55f5aa7db400 2024-05-08T05:51:30.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.413+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.7 v2:172.21.15.63:6816/1163469437 6 ==== osd_op_reply(24 [pgnls start_epoch 121 out=73b] v120'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7db400 2024-05-08T05:51:30.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.413+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:25 17.3 17:c0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2f000 con 0x55f5a9c2e800 2024-05-08T05:51:30.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.413+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.4 v2:172.21.15.63:6824/2212879151 2 ==== osd_op_reply(25 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5a9c2e800 2024-05-08T05:51:30.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.413+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:26 17.13 17:c8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2f000 con 0x55f5aa7db800 2024-05-08T05:51:30.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.413+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 6 ==== osd_op_reply(26 [pgnls start_epoch 121 out=98b] v120'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.413+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:27 17.b 17:d0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2f000 con 0x55f5a9c2e000 2024-05-08T05:51:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.414+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.0 v2:172.21.15.45:6800/3734454393 5 ==== osd_op_reply(27 [pgnls start_epoch 121 out=76b] v121'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2e000 2024-05-08T05:51:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.414+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:28 17.1b 17:d8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2f000 con 0x55f5a9c2e800 2024-05-08T05:51:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.414+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.4 v2:172.21.15.63:6824/2212879151 3 ==== osd_op_reply(28 [pgnls start_epoch 121 out=49b] v121'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5a9c2e800 2024-05-08T05:51:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.414+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:29 17.7 17:e0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2f000 con 0x55f5a9c2e000 2024-05-08T05:51:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.414+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.0 v2:172.21.15.45:6800/3734454393 6 ==== osd_op_reply(29 [pgnls start_epoch 121 out=105b] v121'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2e000 2024-05-08T05:51:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.414+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:30 17.17 17:e8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2f000 con 0x55f5a9c2ec00 2024-05-08T05:51:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.415+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.6 v2:172.21.15.63:6800/2926759052 3 ==== osd_op_reply(30 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2ec00 2024-05-08T05:51:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.415+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:31 17.f 17:f0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2f000 con 0x55f5a9c2e400 2024-05-08T05:51:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.415+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.1 v2:172.21.15.45:6824/1337179184 3 ==== osd_op_reply(31 [pgnls start_epoch 121 out=73b] v121'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5a9c2e400 2024-05-08T05:51:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.415+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:32 17.1f 17:f8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2f000 con 0x55f5a9c2ec00 2024-05-08T05:51:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.415+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.6 v2:172.21.15.63:6800/2926759052 4 ==== osd_op_reply(32 [pgnls start_epoch 121 out=101b] v121'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2ec00 2024-05-08T05:51:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.415+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-05-08T05:51:30.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.415+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:33 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa7dbc00 con 0x55f5aa7db800 2024-05-08T05:51:30.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.416+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 7 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.416+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-05-08T05:51:30.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.416+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:34 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2f800 con 0x55f5aa7db800 2024-05-08T05:51:30.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.416+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 8 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.416+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-05-08T05:51:30.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.416+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:35 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2fc00 con 0x55f5aa7db800 2024-05-08T05:51:30.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.416+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 9 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) ==== 164+0+1169 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.416+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-05-08T05:51:30.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.416+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:36 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2f800 con 0x55f5aa7db800 2024-05-08T05:51:30.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.417+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 10 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.417+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-05-08T05:51:30.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.417+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:37 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c2fc00 con 0x55f5aa7db800 2024-05-08T05:51:30.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.417+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 11 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.417+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-05-08T05:51:30.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.417+0000 7f2434927740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-08T05:51:30.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.417+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-05-08T05:51:30.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.417+0000 7f2434927740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-08T05:51:30.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.417+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-05-08T05:51:30.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.417+0000 7f2434927740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-08T05:51:30.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.417+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-05-08T05:51:30.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.417+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:38 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa7dbc00 con 0x55f5aa7db800 2024-05-08T05:51:30.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.417+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 12 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv35 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.417+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-05-08T05:51:30.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.417+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:39 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5a9c2e400 2024-05-08T05:51:30.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.417+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.1 v2:172.21.15.45:6824/1337179184 4 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5a9c2e400 2024-05-08T05:51:30.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.418+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-05-08T05:51:30.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.418+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:40 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675000 con 0x55f5aa7db800 2024-05-08T05:51:30.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.418+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 13 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=584b] v0'0 uv3 ondisk = 0) ==== 164+0+584 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.418+0000 7f2434927740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-05-08T05:51:30.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.418+0000 7f2434927740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-08T05:51:30.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.418+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-05-08T05:51:30.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.418+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:41 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c52000 con 0x55f5aa7dac00 2024-05-08T05:51:30.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.418+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.2 v2:172.21.15.45:6808/785834340 2 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7dac00 2024-05-08T05:51:30.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.418+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-05-08T05:51:30.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.418+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:42 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c52400 con 0x55f5aa7da800 2024-05-08T05:51:30.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.419+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.5 v2:172.21.15.63:6808/2395798255 4 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7da800 2024-05-08T05:51:30.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.419+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-05-08T05:51:30.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.419+0000 7f2434927740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-08T05:51:30.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.419+0000 7f2434927740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-05-08T05:51:30.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.419+0000 7f2434927740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-08T05:51:30.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.419+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-05-08T05:51:30.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.419+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:43 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c52800 con 0x55f5aa7db400 2024-05-08T05:51:30.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.419+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.7 v2:172.21.15.63:6816/1163469437 7 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7db400 2024-05-08T05:51:30.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.419+0000 7f2434927740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-05-08T05:51:30.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.419+0000 7f2434927740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-08T05:51:30.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.419+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-05-08T05:51:30.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.419+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:44 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c52c00 con 0x55f5aa7db800 2024-05-08T05:51:30.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.419+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 14 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.419+0000 7f2434927740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-05-08T05:51:30.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.419+0000 7f2434927740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-08T05:51:30.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.419+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-05-08T05:51:30.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.419+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:45 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c53000 con 0x55f5a9c2e000 2024-05-08T05:51:30.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.420+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.0 v2:172.21.15.45:6800/3734454393 7 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2e000 2024-05-08T05:51:30.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.420+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-05-08T05:51:30.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.420+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:46 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c53400 con 0x55f5a9c2e000 2024-05-08T05:51:30.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.420+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.0 v2:172.21.15.45:6800/3734454393 8 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1667b] v0'0 uv9 ondisk = 0) ==== 164+0+1667 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2e000 2024-05-08T05:51:30.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.420+0000 7f2434927740 20 datascan.scan_links: injected ino detected 2024-05-08T05:51:30.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.420+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-05-08T05:51:30.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.420+0000 7f2434927740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-08T05:51:30.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.420+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-05-08T05:51:30.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.420+0000 7f2434927740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-08T05:51:30.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.420+0000 7f2434927740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-05-08T05:51:30.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.420+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:47 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c53800 con 0x55f5a9c2ec00 2024-05-08T05:51:30.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.420+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.6 v2:172.21.15.63:6800/2926759052 5 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2ec00 2024-05-08T05:51:30.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.420+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:48 17.0 17:00000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c53c00 con 0x55f5aa7db400 2024-05-08T05:51:30.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.421+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.7 v2:172.21.15.63:6816/1163469437 8 ==== osd_op_reply(48 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7db400 2024-05-08T05:51:30.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.421+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:49 17.10 17:08000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5aa7db800 2024-05-08T05:51:30.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.421+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 15 ==== osd_op_reply(49 [pgnls start_epoch 121 out=177b] v120'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.421+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:50 17.8 17:10000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5aa7db400 2024-05-08T05:51:30.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.422+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.7 v2:172.21.15.63:6816/1163469437 9 ==== osd_op_reply(50 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7db400 2024-05-08T05:51:30.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.422+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:51 17.18 17:18000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5a9c2e000 2024-05-08T05:51:30.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.422+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.0 v2:172.21.15.45:6800/3734454393 9 ==== osd_op_reply(51 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2e000 2024-05-08T05:51:30.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.422+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:52 17.4 17:20000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5a9c2e000 2024-05-08T05:51:30.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.422+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.0 v2:172.21.15.45:6800/3734454393 10 ==== osd_op_reply(52 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2e000 2024-05-08T05:51:30.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.422+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:53 17.14 17:28000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5aa7db800 2024-05-08T05:51:30.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.422+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 16 ==== osd_op_reply(53 [pgnls start_epoch 121 out=149b] v121'40 uv40 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.422+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:54 17.c 17:30000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5aa7db400 2024-05-08T05:51:30.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.423+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.7 v2:172.21.15.63:6816/1163469437 10 ==== osd_op_reply(54 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7db400 2024-05-08T05:51:30.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.423+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:55 17.1c 17:38000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5a9c2e400 2024-05-08T05:51:30.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.423+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.1 v2:172.21.15.45:6824/1337179184 5 ==== osd_op_reply(55 [pgnls start_epoch 121 out=73b] v120'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5a9c2e400 2024-05-08T05:51:30.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.423+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:56 17.2 17:40000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5a9c2e800 2024-05-08T05:51:30.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.423+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.4 v2:172.21.15.63:6824/2212879151 4 ==== osd_op_reply(56 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5a9c2e800 2024-05-08T05:51:30.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.423+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:57 17.12 17:48000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5aa7db800 2024-05-08T05:51:30.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.424+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 17 ==== osd_op_reply(57 [pgnls start_epoch 121 out=81b] v120'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.424+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:58 17.a 17:50000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5aa7db400 2024-05-08T05:51:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.424+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.7 v2:172.21.15.63:6816/1163469437 11 ==== osd_op_reply(58 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7db400 2024-05-08T05:51:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.424+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:59 17.1a 17:58000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5a9c2e000 2024-05-08T05:51:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.424+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.0 v2:172.21.15.45:6800/3734454393 11 ==== osd_op_reply(59 [pgnls start_epoch 121 out=49b] v121'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2e000 2024-05-08T05:51:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.424+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:60 17.6 17:60000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5a9c2ec00 2024-05-08T05:51:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.425+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.6 v2:172.21.15.63:6800/2926759052 6 ==== osd_op_reply(60 [pgnls start_epoch 121 out=77b] v120'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2ec00 2024-05-08T05:51:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.425+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:61 17.16 17:68000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5aa7da800 2024-05-08T05:51:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.425+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.5 v2:172.21.15.63:6808/2395798255 5 ==== osd_op_reply(61 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7da800 2024-05-08T05:51:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.425+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:62 17.e 17:70000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5aa7dac00 2024-05-08T05:51:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.425+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.2 v2:172.21.15.45:6808/785834340 3 ==== osd_op_reply(62 [pgnls start_epoch 121 out=73b] v120'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7dac00 2024-05-08T05:51:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.425+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:63 17.1e 17:78000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5a9c2e400 2024-05-08T05:51:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.425+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.1 v2:172.21.15.45:6824/1337179184 6 ==== osd_op_reply(63 [pgnls start_epoch 121 out=49b] v121'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5a9c2e400 2024-05-08T05:51:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.425+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:64 17.1 17:80000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5a9c2e000 2024-05-08T05:51:30.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.426+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.0 v2:172.21.15.45:6800/3734454393 12 ==== osd_op_reply(64 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2e000 2024-05-08T05:51:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.426+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:65 17.11 17:88000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5aa7db800 2024-05-08T05:51:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.426+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 18 ==== osd_op_reply(65 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.426+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:66 17.9 17:90000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5aa7db800 2024-05-08T05:51:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.426+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 19 ==== osd_op_reply(66 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.426+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:67 17.19 17:98000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5aa7da800 2024-05-08T05:51:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.426+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.5 v2:172.21.15.63:6808/2395798255 6 ==== osd_op_reply(67 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7da800 2024-05-08T05:51:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.426+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5a9c2ec00 2024-05-08T05:51:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.427+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.6 v2:172.21.15.63:6800/2926759052 7 ==== osd_op_reply(68 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2ec00 2024-05-08T05:51:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.427+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:69 17.15 17:a8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5aa7da800 2024-05-08T05:51:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.427+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.5 v2:172.21.15.63:6808/2395798255 7 ==== osd_op_reply(69 [pgnls start_epoch 121 out=97b] v120'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7da800 2024-05-08T05:51:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.427+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:70 17.d 17:b0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5aa7db400 2024-05-08T05:51:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.427+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.7 v2:172.21.15.63:6816/1163469437 12 ==== osd_op_reply(70 [pgnls start_epoch 121 out=74b] v118'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7db400 2024-05-08T05:51:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.428+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:71 17.1d 17:b8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5aa7db400 2024-05-08T05:51:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.428+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.7 v2:172.21.15.63:6816/1163469437 13 ==== osd_op_reply(71 [pgnls start_epoch 121 out=73b] v120'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7db400 2024-05-08T05:51:30.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.428+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:72 17.3 17:c0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5a9c2e800 2024-05-08T05:51:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.428+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.4 v2:172.21.15.63:6824/2212879151 5 ==== osd_op_reply(72 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5a9c2e800 2024-05-08T05:51:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.428+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:73 17.13 17:c8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5aa7db800 2024-05-08T05:51:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.428+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 20 ==== osd_op_reply(73 [pgnls start_epoch 121 out=98b] v120'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.428+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:74 17.b 17:d0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5a9c2e000 2024-05-08T05:51:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.429+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.0 v2:172.21.15.45:6800/3734454393 13 ==== osd_op_reply(74 [pgnls start_epoch 121 out=76b] v121'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2e000 2024-05-08T05:51:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.429+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- osd_op(unknown.0.0:75 17.1b 17:d8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5a9c2e800 2024-05-08T05:51:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.429+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.4 v2:172.21.15.63:6824/2212879151 6 ==== osd_op_reply(75 [pgnls start_epoch 121 out=49b] v121'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5a9c2e800 2024-05-08T05:51:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.429+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:76 17.7 17:e0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5a9c2e000 2024-05-08T05:51:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.429+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.0 v2:172.21.15.45:6800/3734454393 14 ==== osd_op_reply(76 [pgnls start_epoch 121 out=105b] v121'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2e000 2024-05-08T05:51:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.429+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:77 17.17 17:e8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5a9c2ec00 2024-05-08T05:51:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.430+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.6 v2:172.21.15.63:6800/2926759052 8 ==== osd_op_reply(77 [pgnls start_epoch 121 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2ec00 2024-05-08T05:51:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.430+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:78 17.f 17:f0000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5a9c2e400 2024-05-08T05:51:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.430+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.1 v2:172.21.15.45:6824/1337179184 7 ==== osd_op_reply(78 [pgnls start_epoch 121 out=73b] v121'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5a9c2e400 2024-05-08T05:51:30.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.430+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:79 17.1f 17:f8000000::::head [pgnls start_epoch 121 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5a9c2ec00 2024-05-08T05:51:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.430+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.6 v2:172.21.15.63:6800/2926759052 9 ==== osd_op_reply(79 [pgnls start_epoch 121 out=101b] v121'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2ec00 2024-05-08T05:51:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.430+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-05-08T05:51:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.430+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:80 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c52400 con 0x55f5aa7db800 2024-05-08T05:51:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.431+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 21 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.431+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-05-08T05:51:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.431+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:81 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c52800 con 0x55f5aa7db800 2024-05-08T05:51:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.431+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 22 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.431+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-05-08T05:51:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.431+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:82 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c53c00 con 0x55f5aa7db800 2024-05-08T05:51:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.432+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 23 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) ==== 164+0+1169 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.432+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-05-08T05:51:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.432+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:83 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c52000 con 0x55f5aa7db800 2024-05-08T05:51:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.432+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 24 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.432+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-05-08T05:51:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.432+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:84 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c52c00 con 0x55f5aa7db800 2024-05-08T05:51:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.432+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 25 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.432+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-05-08T05:51:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.432+0000 7f2434927740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-05-08T05:51:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.432+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-05-08T05:51:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.432+0000 7f2434927740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-05-08T05:51:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.432+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-05-08T05:51:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.432+0000 7f2434927740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-05-08T05:51:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.432+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-05-08T05:51:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.432+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:85 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675000 con 0x55f5aa7db800 2024-05-08T05:51:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.433+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 26 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv35 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.433+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-05-08T05:51:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.433+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- osd_op(unknown.0.0:86 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c58000 con 0x55f5a9c2e400 2024-05-08T05:51:30.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.433+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.1 v2:172.21.15.45:6824/1337179184 8 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5a9c2e400 2024-05-08T05:51:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.433+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-05-08T05:51:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.433+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:87 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c58400 con 0x55f5aa7db800 2024-05-08T05:51:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.433+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 27 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=584b] v0'0 uv3 ondisk = 0) ==== 164+0+584 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.433+0000 7f2434927740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-05-08T05:51:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.433+0000 7f2434927740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-05-08T05:51:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.433+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-05-08T05:51:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.433+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- osd_op(unknown.0.0:88 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c58800 con 0x55f5aa7dac00 2024-05-08T05:51:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.433+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.2 v2:172.21.15.45:6808/785834340 4 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7dac00 2024-05-08T05:51:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.434+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-05-08T05:51:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.434+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- osd_op(unknown.0.0:89 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c58c00 con 0x55f5aa7da800 2024-05-08T05:51:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.434+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.5 v2:172.21.15.63:6808/2395798255 8 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7da800 2024-05-08T05:51:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.434+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-05-08T05:51:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.434+0000 7f2434927740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-05-08T05:51:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.434+0000 7f2434927740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-05-08T05:51:30.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.434+0000 7f2434927740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-05-08T05:51:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.434+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-05-08T05:51:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.434+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:90 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c59000 con 0x55f5aa7db400 2024-05-08T05:51:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.434+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.7 v2:172.21.15.63:6816/1163469437 14 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7db400 2024-05-08T05:51:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.434+0000 7f2434927740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-05-08T05:51:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.434+0000 7f2434927740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-05-08T05:51:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.434+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-05-08T05:51:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.434+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:91 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c59400 con 0x55f5aa7db800 2024-05-08T05:51:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.435+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 28 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.435+0000 7f2434927740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-05-08T05:51:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.435+0000 7f2434927740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-05-08T05:51:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.435+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-05-08T05:51:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.435+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:92 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c59800 con 0x55f5a9c2e000 2024-05-08T05:51:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.435+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.0 v2:172.21.15.45:6800/3734454393 15 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2e000 2024-05-08T05:51:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.435+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-05-08T05:51:30.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.435+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:93 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c59c00 con 0x55f5a9c2e000 2024-05-08T05:51:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.435+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.0 v2:172.21.15.45:6800/3734454393 16 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1667b] v0'0 uv9 ondisk = 0) ==== 164+0+1667 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2e000 2024-05-08T05:51:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.435+0000 7f2434927740 20 datascan.scan_links: injected ino detected 2024-05-08T05:51:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.435+0000 7f2434927740 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-05-08T05:51:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.435+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-05-08T05:51:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.435+0000 7f2434927740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-05-08T05:51:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.435+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-05-08T05:51:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.435+0000 7f2434927740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-05-08T05:51:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.435+0000 7f2434927740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-05-08T05:51:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.435+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- osd_op(unknown.0.0:94 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c59800 con 0x55f5a9c2ec00 2024-05-08T05:51:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.436+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.6 v2:172.21.15.63:6800/2926759052 10 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2ec00 2024-05-08T05:51:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.436+0000 7f2434927740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-05-08T05:51:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.436+0000 7f2434927740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-05-08T05:51:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.436+0000 7f2434927740 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-05-08T05:51:30.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.436+0000 7f2434927740 10 datascan.scan_links: processing 1 injected_inos 2024-05-08T05:51:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.436+0000 7f2434927740 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-05-08T05:51:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.436+0000 7f2434927740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-05-08T05:51:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.436+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:95 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c58000 con 0x55f5a9c2e000 2024-05-08T05:51:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.436+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.0 v2:172.21.15.45:6800/3734454393 17 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2e000 2024-05-08T05:51:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.436+0000 7f2434927740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-05-08T05:51:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.436+0000 7f2434927740 20 datascan.scan_links: first is now 2 2024-05-08T05:51:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.436+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- osd_op(unknown.0.0:96 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c58c00 con 0x55f5a9c2e000 2024-05-08T05:51:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.437+0000 7f24327ba640 1 -- 172.21.15.45:0/927209153 <== osd.0 v2:172.21.15.45:6800/3734454393 18 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v121'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55f5aa60a780 con 0x55f5a9c2e000 2024-05-08T05:51:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.437+0000 7f2434927740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-05-08T05:51:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.437+0000 7f2434927740 10 datascan.scan_links: updating inotable 2024-05-08T05:51:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.437+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:97 17.13 17:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5a9c59000 con 0x55f5aa7db800 2024-05-08T05:51:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.438+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 29 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.438+0000 7f2434927740 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-05-08T05:51:30.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.438+0000 7f2434927740 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-05-08T05:51:30.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.438+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- osd_op(unknown.0.0:98 17.13 17:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675400 con 0x55f5aa7db800 2024-05-08T05:51:30.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2432fbb640 1 -- 172.21.15.45:0/927209153 <== osd.3 v2:172.21.15.45:6816/1259225446 30 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v121'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x55f5aa60a500 con 0x55f5aa7db800 2024-05-08T05:51:30.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2434927740 10 datascan.scan_links: updating snaptable 2024-05-08T05:51:30.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- osd_op(unknown.0.0:99 17.d 17:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x55f5aa675000 con 0x55f5aa7db400 2024-05-08T05:51:30.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 <== osd.7 v2:172.21.15.63:6816/1163469437 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x55f5aa60a000 con 0x55f5aa7db400 2024-05-08T05:51:30.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x55f5a9c2e000 msgr2=0x55f5a9c2a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2434927740 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x55f5a9c2e000 0x55f5a9c2a580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55f5a9c2e400 msgr2=0x55f5a9c2ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2434927740 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x55f5a9c2e400 0x55f5a9c2ab00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x55f5aa7dac00 msgr2=0x55f5a9c2b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2434927740 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x55f5aa7dac00 0x55f5a9c2b600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x55f5aa7db800 msgr2=0x55f5a9c2a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2434927740 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x55f5aa7db800 0x55f5a9c2a000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x55f5a9c2e800 msgr2=0x55f5a9c2b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2434927740 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x55f5a9c2e800 0x55f5a9c2b080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55f5aa7da800 msgr2=0x55f5a9c2c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2434927740 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x55f5aa7da800 0x55f5a9c2c680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55f5a9c2ec00 msgr2=0x55f5a9c2c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2434927740 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x55f5a9c2ec00 0x55f5a9c2c100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x55f5aa7db400 msgr2=0x55f5aa64a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 reap_dead start 2024-05-08T05:51:30.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2434927740 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x55f5aa7db400 0x55f5aa64a680 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 reap_dead start 2024-05-08T05:51:30.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.439+0000 7f2431fb9640 1 -- 172.21.15.45:0/927209153 reap_dead start 2024-05-08T05:51:30.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.440+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x55f5aa7db000 msgr2=0x55f5aa64a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.440+0000 7f2434927740 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x55f5aa7db000 0x55f5aa64a100 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x55f5aa76cba0 tx=0x55f5aa61e600 comp rx=0 tx=0).stop 2024-05-08T05:51:30.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.440+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5aa667800 msgr2=0x55f5aa649600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.440+0000 7f2434927740 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5aa667800 0x55f5aa649600 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x55f5aa76d8f0 tx=0x55f5aa61eae0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.440+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 shutdown_connections 2024-05-08T05:51:30.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.440+0000 7f2434927740 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x55f5aa7db000 0x55f5aa64a100 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.440+0000 7f2434927740 1 --2- 172.21.15.45:0/927209153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5aa667800 0x55f5aa649600 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.440+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 >> 172.21.15.45:0/927209153 conn(0x55f5aa667000 msgr2=0x55f5aa5f7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:30.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.440+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 shutdown_connections 2024-05-08T05:51:30.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.440+0000 7f2434927740 1 -- 172.21.15.45:0/927209153 wait complete. 2024-05-08T05:51:30.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.440+0000 7f2434927740 1 -- 172.21.15.45:0/1982119801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5a9878c00 msgr2=0x55f5aa648000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.440+0000 7f2434927740 1 --2- 172.21.15.45:0/1982119801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5a9878c00 0x55f5aa648000 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x55f5aa6a7c50 tx=0x55f5a9948120 comp rx=0 tx=0).stop 2024-05-08T05:51:30.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.440+0000 7f2434927740 1 -- 172.21.15.45:0/1982119801 shutdown_connections 2024-05-08T05:51:30.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.440+0000 7f2434927740 1 --2- 172.21.15.45:0/1982119801 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55f5a9879c00 0x55f5aa648b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.440+0000 7f2434927740 1 --2- 172.21.15.45:0/1982119801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x55f5a9879800 0x55f5aa648580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.440+0000 7f2434927740 1 --2- 172.21.15.45:0/1982119801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f5a9878c00 0x55f5aa648000 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.440+0000 7f2434927740 1 -- 172.21.15.45:0/1982119801 >> 172.21.15.45:0/1982119801 conn(0x55f5a9879000 msgr2=0x55f5aa5f6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:30.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.440+0000 7f2434927740 1 -- 172.21.15.45:0/1982119801 shutdown_connections 2024-05-08T05:51:30.462 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.440+0000 7f2434927740 1 -- 172.21.15.45:0/1982119801 wait complete. 2024-05-08T05:51:30.463 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.142483, result: 2024-05-08T05:51:30.464 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T05:51:30.487 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T05:51:30.529 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-05-08T05:51:30.529 INFO:teuthology.orchestra.run.smithi045.stdout: chdir = "" 2024-05-08T05:51:30.530 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T05:51:30.530 INFO:teuthology.orchestra.run.smithi045.stdout: auth supported = cephx 2024-05-08T05:51:30.530 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.530 INFO:teuthology.orchestra.run.smithi045.stdout: filestore xattr use omap = true 2024-05-08T05:51:30.530 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.530 INFO:teuthology.orchestra.run.smithi045.stdout: mon clock drift allowed = 1.000 2024-05-08T05:51:30.530 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.530 INFO:teuthology.orchestra.run.smithi045.stdout: osd crush chooseleaf type = 0 2024-05-08T05:51:30.530 INFO:teuthology.orchestra.run.smithi045.stdout: auth debug = true 2024-05-08T05:51:30.530 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.530 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on old message = true 2024-05-08T05:51:30.530 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on bug = true 2024-05-08T05:51:30.530 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.531 INFO:teuthology.orchestra.run.smithi045.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T05:51:30.531 INFO:teuthology.orchestra.run.smithi045.stdout: mon pg warn max object skew = 0 2024-05-08T05:51:30.531 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.531 INFO:teuthology.orchestra.run.smithi045.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T05:51:30.531 INFO:teuthology.orchestra.run.smithi045.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T05:51:30.531 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.531 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default size = 2 2024-05-08T05:51:30.531 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.531 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow primary affinity = true 2024-05-08T05:51:30.531 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow pg remap = true 2024-05-08T05:51:30.531 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on legacy crush tunables = false 2024-05-08T05:51:30.531 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on crush straw calc version zero = false 2024-05-08T05:51:30.531 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on no sortbitwise = false 2024-05-08T05:51:30.531 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on osd down out interval zero = false 2024-05-08T05:51:30.532 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on too few osds = false 2024-05-08T05:51:30.532 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T05:51:30.532 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T05:51:30.532 INFO:teuthology.orchestra.run.smithi045.stdout: mon_allow_pool_size_one = true 2024-05-08T05:51:30.532 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.532 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T05:51:30.532 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.532 INFO:teuthology.orchestra.run.smithi045.stdout: osd default data pool replay window = 5 2024-05-08T05:51:30.532 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.532 INFO:teuthology.orchestra.run.smithi045.stdout: mon allow pool delete = true 2024-05-08T05:51:30.532 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.532 INFO:teuthology.orchestra.run.smithi045.stdout: mon cluster log file level = debug 2024-05-08T05:51:30.532 INFO:teuthology.orchestra.run.smithi045.stdout: debug asserts on shutdown = true 2024-05-08T05:51:30.532 INFO:teuthology.orchestra.run.smithi045.stdout: mon health detail to clog = false 2024-05-08T05:51:30.533 INFO:teuthology.orchestra.run.smithi045.stdout: mon host = "172.21.15.45,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-08T05:51:30.533 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on skipped message = False 2024-05-08T05:51:30.533 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = e9e1170a-fd27-4625-97a9-5d5f104d13a2 2024-05-08T05:51:30.533 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.533 INFO:teuthology.orchestra.run.smithi045.stdout:[osd] 2024-05-08T05:51:30.533 INFO:teuthology.orchestra.run.smithi045.stdout: osd journal size = 100 2024-05-08T05:51:30.533 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.533 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub load threshold = 5.0 2024-05-08T05:51:30.533 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub max interval = 600 2024-05-08T05:51:30.533 INFO:teuthology.orchestra.run.smithi045.stdout: osd mclock profile = high_recovery_ops 2024-05-08T05:51:30.533 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.533 INFO:teuthology.orchestra.run.smithi045.stdout: osd recover clone overlap = true 2024-05-08T05:51:30.533 INFO:teuthology.orchestra.run.smithi045.stdout: osd recovery max chunk = 1048576 2024-05-08T05:51:30.533 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.533 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug shutdown = true 2024-05-08T05:51:30.534 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug op order = true 2024-05-08T05:51:30.534 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify stray on activate = true 2024-05-08T05:51:30.534 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug trim objects = true 2024-05-08T05:51:30.534 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.534 INFO:teuthology.orchestra.run.smithi045.stdout: osd open classes on start = true 2024-05-08T05:51:30.534 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug pg log writeout = true 2024-05-08T05:51:30.534 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.534 INFO:teuthology.orchestra.run.smithi045.stdout: osd deep scrub update digest min age = 30 2024-05-08T05:51:30.534 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.534 INFO:teuthology.orchestra.run.smithi045.stdout: osd map max advance = 10 2024-05-08T05:51:30.534 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.534 INFO:teuthology.orchestra.run.smithi045.stdout: journal zero on create = true 2024-05-08T05:51:30.534 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.535 INFO:teuthology.orchestra.run.smithi045.stdout: filestore ondisk finisher threads = 3 2024-05-08T05:51:30.535 INFO:teuthology.orchestra.run.smithi045.stdout: filestore apply finisher threads = 3 2024-05-08T05:51:30.535 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.535 INFO:teuthology.orchestra.run.smithi045.stdout: bdev debug aio = true 2024-05-08T05:51:30.535 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug misdirected ops = true 2024-05-08T05:51:30.535 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore block size = 96636764160 2024-05-08T05:51:30.535 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore fsck on mount = True 2024-05-08T05:51:30.535 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluefs = 20 2024-05-08T05:51:30.535 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluestore = 20 2024-05-08T05:51:30.535 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:51:30.535 INFO:teuthology.orchestra.run.smithi045.stdout: debug osd = 20 2024-05-08T05:51:30.535 INFO:teuthology.orchestra.run.smithi045.stdout: debug rocksdb = 10 2024-05-08T05:51:30.535 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T05:51:30.535 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd full ratio = 0.9 2024-05-08T05:51:30.535 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd nearfull ratio = 0.8 2024-05-08T05:51:30.536 INFO:teuthology.orchestra.run.smithi045.stdout: osd failsafe full ratio = 0.95 2024-05-08T05:51:30.536 INFO:teuthology.orchestra.run.smithi045.stdout: osd objectstore = bluestore 2024-05-08T05:51:30.536 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-05-08T05:51:30.536 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.536 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr] 2024-05-08T05:51:30.536 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:51:30.536 INFO:teuthology.orchestra.run.smithi045.stdout: debug mgr = 20 2024-05-08T05:51:30.536 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-05-08T05:51:30.536 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-05-08T05:51:30.536 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-05-08T05:51:30.536 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-05-08T05:51:30.536 INFO:teuthology.orchestra.run.smithi045.stdout: mgr/telemetry/nag = false 2024-05-08T05:51:30.536 INFO:teuthology.orchestra.run.smithi045.stdout: debug client = 20 2024-05-08T05:51:30.537 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.537 INFO:teuthology.orchestra.run.smithi045.stdout:[mon] 2024-05-08T05:51:30.537 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:51:30.537 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-05-08T05:51:30.537 INFO:teuthology.orchestra.run.smithi045.stdout: debug paxos = 20 2024-05-08T05:51:30.537 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-05-08T05:51:30.537 INFO:teuthology.orchestra.run.smithi045.stdout: mon data avail warn = 5 2024-05-08T05:51:30.537 INFO:teuthology.orchestra.run.smithi045.stdout: mon mgr mkfs grace = 240 2024-05-08T05:51:30.537 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-05-08T05:51:30.537 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd reporter subtree level = osd 2024-05-08T05:51:30.537 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd prime pg temp = true 2024-05-08T05:51:30.537 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-05-08T05:51:30.537 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.537 INFO:teuthology.orchestra.run.smithi045.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T05:51:30.537 INFO:teuthology.orchestra.run.smithi045.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T05:51:30.538 INFO:teuthology.orchestra.run.smithi045.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T05:51:30.538 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.538 INFO:teuthology.orchestra.run.smithi045.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T05:51:30.538 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T05:51:30.538 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T05:51:30.538 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.538 INFO:teuthology.orchestra.run.smithi045.stdout: # 1m isn't quite enough 2024-05-08T05:51:30.538 INFO:teuthology.orchestra.run.smithi045.stdout: mon_down_mkfs_grace = 2m 2024-05-08T05:51:30.538 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.538 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_filestore_osds = false 2024-05-08T05:51:30.538 INFO:teuthology.orchestra.run.smithi045.stdout: mon down mkfs grace = 300 2024-05-08T05:51:30.538 INFO:teuthology.orchestra.run.smithi045.stdout: mon op complaint time = 120 2024-05-08T05:51:30.538 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.538 INFO:teuthology.orchestra.run.smithi045.stdout:[client] 2024-05-08T05:51:30.539 INFO:teuthology.orchestra.run.smithi045.stdout: rgw cache enabled = true 2024-05-08T05:51:30.539 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable ops log = true 2024-05-08T05:51:30.539 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable usage log = true 2024-05-08T05:51:30.539 INFO:teuthology.orchestra.run.smithi045.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T05:51:30.539 INFO:teuthology.orchestra.run.smithi045.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T05:51:30.539 INFO:teuthology.orchestra.run.smithi045.stdout: client mount timeout = 600 2024-05-08T05:51:30.539 INFO:teuthology.orchestra.run.smithi045.stdout: debug client = 20 2024-05-08T05:51:30.539 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:51:30.539 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = "" 2024-05-08T05:51:30.539 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-05-08T05:51:30.539 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-05-08T05:51:30.539 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.a] 2024-05-08T05:51:30.539 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.b] 2024-05-08T05:51:30.539 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.c] 2024-05-08T05:51:30.539 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a] 2024-05-08T05:51:30.540 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c] 2024-05-08T05:51:30.540 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b] 2024-05-08T05:51:30.540 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d] 2024-05-08T05:51:30.540 INFO:teuthology.orchestra.run.smithi045.stdout:[mds] 2024-05-08T05:51:30.540 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds = 20 2024-05-08T05:51:30.540 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds balancer = 20 2024-05-08T05:51:30.540 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:51:30.540 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug frag = True 2024-05-08T05:51:30.540 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug scatterstat = False 2024-05-08T05:51:30.540 INFO:teuthology.orchestra.run.smithi045.stdout: mds op complaint time = 180 2024-05-08T05:51:30.540 INFO:teuthology.orchestra.run.smithi045.stdout: mds verify scatter = True 2024-05-08T05:51:30.540 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-05-08T05:51:30.540 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-05-08T05:51:30.540 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-05-08T05:51:30.541 INFO:teuthology.orchestra.run.smithi063.stdout:[global] 2024-05-08T05:51:30.541 INFO:teuthology.orchestra.run.smithi063.stdout: chdir = "" 2024-05-08T05:51:30.541 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T05:51:30.541 INFO:teuthology.orchestra.run.smithi063.stdout: auth supported = cephx 2024-05-08T05:51:30.541 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.541 INFO:teuthology.orchestra.run.smithi063.stdout: filestore xattr use omap = true 2024-05-08T05:51:30.541 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.541 INFO:teuthology.orchestra.run.smithi063.stdout: mon clock drift allowed = 1.000 2024-05-08T05:51:30.542 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.542 INFO:teuthology.orchestra.run.smithi063.stdout: osd crush chooseleaf type = 0 2024-05-08T05:51:30.542 INFO:teuthology.orchestra.run.smithi063.stdout: auth debug = true 2024-05-08T05:51:30.542 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.542 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on old message = true 2024-05-08T05:51:30.542 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on bug = true 2024-05-08T05:51:30.542 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.542 INFO:teuthology.orchestra.run.smithi063.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T05:51:30.542 INFO:teuthology.orchestra.run.smithi063.stdout: mon pg warn max object skew = 0 2024-05-08T05:51:30.542 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.542 INFO:teuthology.orchestra.run.smithi063.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T05:51:30.542 INFO:teuthology.orchestra.run.smithi063.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T05:51:30.542 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.542 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default size = 2 2024-05-08T05:51:30.543 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.543 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow primary affinity = true 2024-05-08T05:51:30.543 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow pg remap = true 2024-05-08T05:51:30.543 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on legacy crush tunables = false 2024-05-08T05:51:30.543 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on crush straw calc version zero = false 2024-05-08T05:51:30.543 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on no sortbitwise = false 2024-05-08T05:51:30.543 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on osd down out interval zero = false 2024-05-08T05:51:30.543 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on too few osds = false 2024-05-08T05:51:30.543 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T05:51:30.543 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T05:51:30.543 INFO:teuthology.orchestra.run.smithi063.stdout: mon_allow_pool_size_one = true 2024-05-08T05:51:30.543 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.543 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T05:51:30.543 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.543 INFO:teuthology.orchestra.run.smithi063.stdout: osd default data pool replay window = 5 2024-05-08T05:51:30.544 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.544 INFO:teuthology.orchestra.run.smithi063.stdout: mon allow pool delete = true 2024-05-08T05:51:30.544 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.544 INFO:teuthology.orchestra.run.smithi063.stdout: mon cluster log file level = debug 2024-05-08T05:51:30.544 INFO:teuthology.orchestra.run.smithi063.stdout: debug asserts on shutdown = true 2024-05-08T05:51:30.544 INFO:teuthology.orchestra.run.smithi063.stdout: mon health detail to clog = false 2024-05-08T05:51:30.544 INFO:teuthology.orchestra.run.smithi063.stdout: mon host = "172.21.15.45,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-08T05:51:30.544 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on skipped message = False 2024-05-08T05:51:30.544 INFO:teuthology.orchestra.run.smithi063.stdout: fsid = e9e1170a-fd27-4625-97a9-5d5f104d13a2 2024-05-08T05:51:30.544 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.544 INFO:teuthology.orchestra.run.smithi063.stdout:[osd] 2024-05-08T05:51:30.544 INFO:teuthology.orchestra.run.smithi063.stdout: osd journal size = 100 2024-05-08T05:51:30.544 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.544 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub load threshold = 5.0 2024-05-08T05:51:30.545 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub max interval = 600 2024-05-08T05:51:30.545 INFO:teuthology.orchestra.run.smithi063.stdout: osd mclock profile = high_recovery_ops 2024-05-08T05:51:30.545 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.545 INFO:teuthology.orchestra.run.smithi063.stdout: osd recover clone overlap = true 2024-05-08T05:51:30.545 INFO:teuthology.orchestra.run.smithi063.stdout: osd recovery max chunk = 1048576 2024-05-08T05:51:30.545 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.545 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug shutdown = true 2024-05-08T05:51:30.545 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug op order = true 2024-05-08T05:51:30.545 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug verify stray on activate = true 2024-05-08T05:51:30.545 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug trim objects = true 2024-05-08T05:51:30.545 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.545 INFO:teuthology.orchestra.run.smithi063.stdout: osd open classes on start = true 2024-05-08T05:51:30.545 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug pg log writeout = true 2024-05-08T05:51:30.545 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.545 INFO:teuthology.orchestra.run.smithi063.stdout: osd deep scrub update digest min age = 30 2024-05-08T05:51:30.546 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.546 INFO:teuthology.orchestra.run.smithi063.stdout: osd map max advance = 10 2024-05-08T05:51:30.546 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.546 INFO:teuthology.orchestra.run.smithi063.stdout: journal zero on create = true 2024-05-08T05:51:30.546 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.546 INFO:teuthology.orchestra.run.smithi063.stdout: filestore ondisk finisher threads = 3 2024-05-08T05:51:30.546 INFO:teuthology.orchestra.run.smithi063.stdout: filestore apply finisher threads = 3 2024-05-08T05:51:30.546 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.546 INFO:teuthology.orchestra.run.smithi063.stdout: bdev debug aio = true 2024-05-08T05:51:30.546 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug misdirected ops = true 2024-05-08T05:51:30.546 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore block size = 96636764160 2024-05-08T05:51:30.546 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore fsck on mount = True 2024-05-08T05:51:30.546 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluefs = 20 2024-05-08T05:51:30.546 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluestore = 20 2024-05-08T05:51:30.547 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:51:30.547 INFO:teuthology.orchestra.run.smithi063.stdout: debug osd = 20 2024-05-08T05:51:30.547 INFO:teuthology.orchestra.run.smithi063.stdout: debug rocksdb = 10 2024-05-08T05:51:30.547 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T05:51:30.547 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd full ratio = 0.9 2024-05-08T05:51:30.547 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd nearfull ratio = 0.8 2024-05-08T05:51:30.547 INFO:teuthology.orchestra.run.smithi063.stdout: osd failsafe full ratio = 0.95 2024-05-08T05:51:30.547 INFO:teuthology.orchestra.run.smithi063.stdout: osd objectstore = bluestore 2024-05-08T05:51:30.547 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-08T05:51:30.547 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.547 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr] 2024-05-08T05:51:30.547 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:51:30.547 INFO:teuthology.orchestra.run.smithi063.stdout: debug mgr = 20 2024-05-08T05:51:30.547 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-08T05:51:30.548 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-08T05:51:30.548 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-08T05:51:30.548 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-08T05:51:30.548 INFO:teuthology.orchestra.run.smithi063.stdout: mgr/telemetry/nag = false 2024-05-08T05:51:30.548 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-08T05:51:30.548 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.548 INFO:teuthology.orchestra.run.smithi063.stdout:[mon] 2024-05-08T05:51:30.548 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:51:30.548 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-08T05:51:30.548 INFO:teuthology.orchestra.run.smithi063.stdout: debug paxos = 20 2024-05-08T05:51:30.548 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-08T05:51:30.548 INFO:teuthology.orchestra.run.smithi063.stdout: mon data avail warn = 5 2024-05-08T05:51:30.548 INFO:teuthology.orchestra.run.smithi063.stdout: mon mgr mkfs grace = 240 2024-05-08T05:51:30.548 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-08T05:51:30.548 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd reporter subtree level = osd 2024-05-08T05:51:30.549 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd prime pg temp = true 2024-05-08T05:51:30.549 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-08T05:51:30.549 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.549 INFO:teuthology.orchestra.run.smithi063.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T05:51:30.549 INFO:teuthology.orchestra.run.smithi063.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T05:51:30.549 INFO:teuthology.orchestra.run.smithi063.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T05:51:30.549 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.549 INFO:teuthology.orchestra.run.smithi063.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T05:51:30.549 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T05:51:30.549 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T05:51:30.549 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.549 INFO:teuthology.orchestra.run.smithi063.stdout: # 1m isn't quite enough 2024-05-08T05:51:30.549 INFO:teuthology.orchestra.run.smithi063.stdout: mon_down_mkfs_grace = 2m 2024-05-08T05:51:30.549 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.549 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_filestore_osds = false 2024-05-08T05:51:30.550 INFO:teuthology.orchestra.run.smithi063.stdout: mon down mkfs grace = 300 2024-05-08T05:51:30.550 INFO:teuthology.orchestra.run.smithi063.stdout: mon op complaint time = 120 2024-05-08T05:51:30.550 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.550 INFO:teuthology.orchestra.run.smithi063.stdout:[client] 2024-05-08T05:51:30.550 INFO:teuthology.orchestra.run.smithi063.stdout: rgw cache enabled = true 2024-05-08T05:51:30.550 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable ops log = true 2024-05-08T05:51:30.550 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable usage log = true 2024-05-08T05:51:30.550 INFO:teuthology.orchestra.run.smithi063.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T05:51:30.550 INFO:teuthology.orchestra.run.smithi063.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T05:51:30.550 INFO:teuthology.orchestra.run.smithi063.stdout: client mount timeout = 600 2024-05-08T05:51:30.550 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-08T05:51:30.550 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:51:30.550 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = "" 2024-05-08T05:51:30.550 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-08T05:51:30.551 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-08T05:51:30.551 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.a] 2024-05-08T05:51:30.551 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.b] 2024-05-08T05:51:30.551 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.c] 2024-05-08T05:51:30.551 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.a] 2024-05-08T05:51:30.551 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.c] 2024-05-08T05:51:30.551 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.b] 2024-05-08T05:51:30.551 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.d] 2024-05-08T05:51:30.551 INFO:teuthology.orchestra.run.smithi063.stdout:[mds] 2024-05-08T05:51:30.551 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds = 20 2024-05-08T05:51:30.551 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds balancer = 20 2024-05-08T05:51:30.551 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:51:30.551 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug frag = True 2024-05-08T05:51:30.551 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug scatterstat = False 2024-05-08T05:51:30.552 INFO:teuthology.orchestra.run.smithi063.stdout: mds op complaint time = 180 2024-05-08T05:51:30.552 INFO:teuthology.orchestra.run.smithi063.stdout: mds verify scatter = True 2024-05-08T05:51:30.552 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-08T05:51:30.552 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-08T05:51:30.552 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-08T05:51:30.554 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T05:51:30.599 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T05:51:30.642 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-05-08T05:51:30.642 INFO:teuthology.orchestra.run.smithi045.stdout: chdir = "" 2024-05-08T05:51:30.642 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T05:51:30.643 INFO:teuthology.orchestra.run.smithi045.stdout: auth supported = cephx 2024-05-08T05:51:30.643 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.643 INFO:teuthology.orchestra.run.smithi045.stdout: filestore xattr use omap = true 2024-05-08T05:51:30.643 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.643 INFO:teuthology.orchestra.run.smithi045.stdout: mon clock drift allowed = 1.000 2024-05-08T05:51:30.643 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.643 INFO:teuthology.orchestra.run.smithi045.stdout: osd crush chooseleaf type = 0 2024-05-08T05:51:30.643 INFO:teuthology.orchestra.run.smithi045.stdout: auth debug = true 2024-05-08T05:51:30.643 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.643 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on old message = true 2024-05-08T05:51:30.643 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on bug = true 2024-05-08T05:51:30.643 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.643 INFO:teuthology.orchestra.run.smithi045.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T05:51:30.643 INFO:teuthology.orchestra.run.smithi045.stdout: mon pg warn max object skew = 0 2024-05-08T05:51:30.644 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.644 INFO:teuthology.orchestra.run.smithi045.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T05:51:30.644 INFO:teuthology.orchestra.run.smithi045.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T05:51:30.644 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.644 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default size = 2 2024-05-08T05:51:30.644 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.644 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow primary affinity = true 2024-05-08T05:51:30.644 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd allow pg remap = true 2024-05-08T05:51:30.644 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on legacy crush tunables = false 2024-05-08T05:51:30.644 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on crush straw calc version zero = false 2024-05-08T05:51:30.644 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on no sortbitwise = false 2024-05-08T05:51:30.644 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on osd down out interval zero = false 2024-05-08T05:51:30.644 INFO:teuthology.orchestra.run.smithi045.stdout: mon warn on too few osds = false 2024-05-08T05:51:30.644 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T05:51:30.645 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T05:51:30.645 INFO:teuthology.orchestra.run.smithi045.stdout: mon_allow_pool_size_one = true 2024-05-08T05:51:30.645 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.645 INFO:teuthology.orchestra.run.smithi045.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T05:51:30.645 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.645 INFO:teuthology.orchestra.run.smithi045.stdout: osd default data pool replay window = 5 2024-05-08T05:51:30.645 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.645 INFO:teuthology.orchestra.run.smithi045.stdout: mon allow pool delete = true 2024-05-08T05:51:30.645 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.645 INFO:teuthology.orchestra.run.smithi045.stdout: mon cluster log file level = debug 2024-05-08T05:51:30.645 INFO:teuthology.orchestra.run.smithi045.stdout: debug asserts on shutdown = true 2024-05-08T05:51:30.646 INFO:teuthology.orchestra.run.smithi045.stdout: mon health detail to clog = false 2024-05-08T05:51:30.646 INFO:teuthology.orchestra.run.smithi045.stdout: mon host = "172.21.15.45,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-08T05:51:30.646 INFO:teuthology.orchestra.run.smithi045.stdout: ms die on skipped message = False 2024-05-08T05:51:30.646 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = e9e1170a-fd27-4625-97a9-5d5f104d13a2 2024-05-08T05:51:30.646 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.646 INFO:teuthology.orchestra.run.smithi045.stdout:[osd] 2024-05-08T05:51:30.646 INFO:teuthology.orchestra.run.smithi045.stdout: osd journal size = 100 2024-05-08T05:51:30.646 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.646 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub load threshold = 5.0 2024-05-08T05:51:30.646 INFO:teuthology.orchestra.run.smithi045.stdout: osd scrub max interval = 600 2024-05-08T05:51:30.646 INFO:teuthology.orchestra.run.smithi045.stdout: osd mclock profile = high_recovery_ops 2024-05-08T05:51:30.646 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.646 INFO:teuthology.orchestra.run.smithi045.stdout: osd recover clone overlap = true 2024-05-08T05:51:30.647 INFO:teuthology.orchestra.run.smithi045.stdout: osd recovery max chunk = 1048576 2024-05-08T05:51:30.647 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.647 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug shutdown = true 2024-05-08T05:51:30.647 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug op order = true 2024-05-08T05:51:30.647 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug verify stray on activate = true 2024-05-08T05:51:30.647 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug trim objects = true 2024-05-08T05:51:30.647 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.647 INFO:teuthology.orchestra.run.smithi045.stdout: osd open classes on start = true 2024-05-08T05:51:30.647 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug pg log writeout = true 2024-05-08T05:51:30.647 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.647 INFO:teuthology.orchestra.run.smithi045.stdout: osd deep scrub update digest min age = 30 2024-05-08T05:51:30.647 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.647 INFO:teuthology.orchestra.run.smithi045.stdout: osd map max advance = 10 2024-05-08T05:51:30.647 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.648 INFO:teuthology.orchestra.run.smithi045.stdout: journal zero on create = true 2024-05-08T05:51:30.648 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.648 INFO:teuthology.orchestra.run.smithi045.stdout: filestore ondisk finisher threads = 3 2024-05-08T05:51:30.648 INFO:teuthology.orchestra.run.smithi045.stdout: filestore apply finisher threads = 3 2024-05-08T05:51:30.648 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.648 INFO:teuthology.orchestra.run.smithi045.stdout: bdev debug aio = true 2024-05-08T05:51:30.648 INFO:teuthology.orchestra.run.smithi045.stdout: osd debug misdirected ops = true 2024-05-08T05:51:30.648 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore block size = 96636764160 2024-05-08T05:51:30.648 INFO:teuthology.orchestra.run.smithi045.stdout: bluestore fsck on mount = True 2024-05-08T05:51:30.648 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluefs = 20 2024-05-08T05:51:30.648 INFO:teuthology.orchestra.run.smithi045.stdout: debug bluestore = 20 2024-05-08T05:51:30.648 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:51:30.648 INFO:teuthology.orchestra.run.smithi045.stdout: debug osd = 20 2024-05-08T05:51:30.648 INFO:teuthology.orchestra.run.smithi045.stdout: debug rocksdb = 10 2024-05-08T05:51:30.648 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T05:51:30.649 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd full ratio = 0.9 2024-05-08T05:51:30.649 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd nearfull ratio = 0.8 2024-05-08T05:51:30.649 INFO:teuthology.orchestra.run.smithi045.stdout: osd failsafe full ratio = 0.95 2024-05-08T05:51:30.649 INFO:teuthology.orchestra.run.smithi045.stdout: osd objectstore = bluestore 2024-05-08T05:51:30.649 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-05-08T05:51:30.649 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.649 INFO:teuthology.orchestra.run.smithi045.stdout:[mgr] 2024-05-08T05:51:30.649 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:51:30.649 INFO:teuthology.orchestra.run.smithi045.stdout: debug mgr = 20 2024-05-08T05:51:30.649 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-05-08T05:51:30.649 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-05-08T05:51:30.649 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-05-08T05:51:30.649 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-05-08T05:51:30.649 INFO:teuthology.orchestra.run.smithi045.stdout: mgr/telemetry/nag = false 2024-05-08T05:51:30.649 INFO:teuthology.orchestra.run.smithi045.stdout: debug client = 20 2024-05-08T05:51:30.650 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.650 INFO:teuthology.orchestra.run.smithi045.stdout:[mon] 2024-05-08T05:51:30.650 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:51:30.650 INFO:teuthology.orchestra.run.smithi045.stdout: debug mon = 20 2024-05-08T05:51:30.650 INFO:teuthology.orchestra.run.smithi045.stdout: debug paxos = 20 2024-05-08T05:51:30.650 INFO:teuthology.orchestra.run.smithi045.stdout: debug auth = 20 2024-05-08T05:51:30.650 INFO:teuthology.orchestra.run.smithi045.stdout: mon data avail warn = 5 2024-05-08T05:51:30.650 INFO:teuthology.orchestra.run.smithi045.stdout: mon mgr mkfs grace = 240 2024-05-08T05:51:30.650 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min pgs per osd = 4 2024-05-08T05:51:30.650 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd reporter subtree level = osd 2024-05-08T05:51:30.650 INFO:teuthology.orchestra.run.smithi045.stdout: mon osd prime pg temp = true 2024-05-08T05:51:30.650 INFO:teuthology.orchestra.run.smithi045.stdout: mon reweight min bytes per osd = 10 2024-05-08T05:51:30.650 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.650 INFO:teuthology.orchestra.run.smithi045.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T05:51:30.651 INFO:teuthology.orchestra.run.smithi045.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T05:51:30.651 INFO:teuthology.orchestra.run.smithi045.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T05:51:30.651 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.651 INFO:teuthology.orchestra.run.smithi045.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T05:51:30.651 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T05:51:30.651 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T05:51:30.651 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.651 INFO:teuthology.orchestra.run.smithi045.stdout: # 1m isn't quite enough 2024-05-08T05:51:30.651 INFO:teuthology.orchestra.run.smithi045.stdout: mon_down_mkfs_grace = 2m 2024-05-08T05:51:30.651 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.651 INFO:teuthology.orchestra.run.smithi045.stdout: mon_warn_on_filestore_osds = false 2024-05-08T05:51:30.651 INFO:teuthology.orchestra.run.smithi045.stdout: mon down mkfs grace = 300 2024-05-08T05:51:30.651 INFO:teuthology.orchestra.run.smithi045.stdout: mon op complaint time = 120 2024-05-08T05:51:30.651 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:30.651 INFO:teuthology.orchestra.run.smithi045.stdout:[client] 2024-05-08T05:51:30.652 INFO:teuthology.orchestra.run.smithi045.stdout: rgw cache enabled = true 2024-05-08T05:51:30.652 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable ops log = true 2024-05-08T05:51:30.652 INFO:teuthology.orchestra.run.smithi045.stdout: rgw enable usage log = true 2024-05-08T05:51:30.652 INFO:teuthology.orchestra.run.smithi045.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T05:51:30.652 INFO:teuthology.orchestra.run.smithi045.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T05:51:30.652 INFO:teuthology.orchestra.run.smithi045.stdout: client mount timeout = 600 2024-05-08T05:51:30.652 INFO:teuthology.orchestra.run.smithi045.stdout: debug client = 20 2024-05-08T05:51:30.652 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:51:30.652 INFO:teuthology.orchestra.run.smithi045.stdout: pid file = "" 2024-05-08T05:51:30.652 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-05-08T05:51:30.652 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-05-08T05:51:30.652 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.a] 2024-05-08T05:51:30.652 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.b] 2024-05-08T05:51:30.652 INFO:teuthology.orchestra.run.smithi045.stdout:[mon.c] 2024-05-08T05:51:30.652 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a] 2024-05-08T05:51:30.653 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c] 2024-05-08T05:51:30.653 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b] 2024-05-08T05:51:30.653 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d] 2024-05-08T05:51:30.653 INFO:teuthology.orchestra.run.smithi045.stdout:[mds] 2024-05-08T05:51:30.653 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds = 20 2024-05-08T05:51:30.653 INFO:teuthology.orchestra.run.smithi045.stdout: debug mds balancer = 20 2024-05-08T05:51:30.653 INFO:teuthology.orchestra.run.smithi045.stdout: debug ms = 1 2024-05-08T05:51:30.653 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug frag = True 2024-05-08T05:51:30.653 INFO:teuthology.orchestra.run.smithi045.stdout: mds debug scatterstat = True 2024-05-08T05:51:30.653 INFO:teuthology.orchestra.run.smithi045.stdout: mds op complaint time = 180 2024-05-08T05:51:30.653 INFO:teuthology.orchestra.run.smithi045.stdout: mds verify scatter = True 2024-05-08T05:51:30.653 INFO:teuthology.orchestra.run.smithi045.stdout: osd op complaint time = 180 2024-05-08T05:51:30.653 INFO:teuthology.orchestra.run.smithi045.stdout: rados mon op timeout = 900 2024-05-08T05:51:30.653 INFO:teuthology.orchestra.run.smithi045.stdout: rados osd op timeout = 900 2024-05-08T05:51:30.654 INFO:teuthology.orchestra.run.smithi063.stdout:[global] 2024-05-08T05:51:30.654 INFO:teuthology.orchestra.run.smithi063.stdout: chdir = "" 2024-05-08T05:51:30.654 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T05:51:30.654 INFO:teuthology.orchestra.run.smithi063.stdout: auth supported = cephx 2024-05-08T05:51:30.655 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.655 INFO:teuthology.orchestra.run.smithi063.stdout: filestore xattr use omap = true 2024-05-08T05:51:30.655 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.655 INFO:teuthology.orchestra.run.smithi063.stdout: mon clock drift allowed = 1.000 2024-05-08T05:51:30.655 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.655 INFO:teuthology.orchestra.run.smithi063.stdout: osd crush chooseleaf type = 0 2024-05-08T05:51:30.655 INFO:teuthology.orchestra.run.smithi063.stdout: auth debug = true 2024-05-08T05:51:30.655 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.655 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on old message = true 2024-05-08T05:51:30.655 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on bug = true 2024-05-08T05:51:30.655 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.655 INFO:teuthology.orchestra.run.smithi063.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T05:51:30.655 INFO:teuthology.orchestra.run.smithi063.stdout: mon pg warn max object skew = 0 2024-05-08T05:51:30.656 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.656 INFO:teuthology.orchestra.run.smithi063.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T05:51:30.656 INFO:teuthology.orchestra.run.smithi063.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T05:51:30.656 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.656 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default size = 2 2024-05-08T05:51:30.656 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.656 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow primary affinity = true 2024-05-08T05:51:30.656 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow pg remap = true 2024-05-08T05:51:30.656 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on legacy crush tunables = false 2024-05-08T05:51:30.656 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on crush straw calc version zero = false 2024-05-08T05:51:30.656 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on no sortbitwise = false 2024-05-08T05:51:30.656 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on osd down out interval zero = false 2024-05-08T05:51:30.656 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on too few osds = false 2024-05-08T05:51:30.656 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T05:51:30.657 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T05:51:30.657 INFO:teuthology.orchestra.run.smithi063.stdout: mon_allow_pool_size_one = true 2024-05-08T05:51:30.657 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.657 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T05:51:30.657 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.657 INFO:teuthology.orchestra.run.smithi063.stdout: osd default data pool replay window = 5 2024-05-08T05:51:30.657 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.657 INFO:teuthology.orchestra.run.smithi063.stdout: mon allow pool delete = true 2024-05-08T05:51:30.657 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.657 INFO:teuthology.orchestra.run.smithi063.stdout: mon cluster log file level = debug 2024-05-08T05:51:30.657 INFO:teuthology.orchestra.run.smithi063.stdout: debug asserts on shutdown = true 2024-05-08T05:51:30.657 INFO:teuthology.orchestra.run.smithi063.stdout: mon health detail to clog = false 2024-05-08T05:51:30.657 INFO:teuthology.orchestra.run.smithi063.stdout: mon host = "172.21.15.45,172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790]" 2024-05-08T05:51:30.657 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on skipped message = False 2024-05-08T05:51:30.658 INFO:teuthology.orchestra.run.smithi063.stdout: fsid = e9e1170a-fd27-4625-97a9-5d5f104d13a2 2024-05-08T05:51:30.658 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.658 INFO:teuthology.orchestra.run.smithi063.stdout:[osd] 2024-05-08T05:51:30.658 INFO:teuthology.orchestra.run.smithi063.stdout: osd journal size = 100 2024-05-08T05:51:30.658 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.658 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub load threshold = 5.0 2024-05-08T05:51:30.658 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub max interval = 600 2024-05-08T05:51:30.659 INFO:teuthology.orchestra.run.smithi063.stdout: osd mclock profile = high_recovery_ops 2024-05-08T05:51:30.659 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.659 INFO:teuthology.orchestra.run.smithi063.stdout: osd recover clone overlap = true 2024-05-08T05:51:30.659 INFO:teuthology.orchestra.run.smithi063.stdout: osd recovery max chunk = 1048576 2024-05-08T05:51:30.659 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.659 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug shutdown = true 2024-05-08T05:51:30.659 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug op order = true 2024-05-08T05:51:30.659 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug verify stray on activate = true 2024-05-08T05:51:30.660 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug trim objects = true 2024-05-08T05:51:30.660 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.660 INFO:teuthology.orchestra.run.smithi063.stdout: osd open classes on start = true 2024-05-08T05:51:30.660 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug pg log writeout = true 2024-05-08T05:51:30.660 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.660 INFO:teuthology.orchestra.run.smithi063.stdout: osd deep scrub update digest min age = 30 2024-05-08T05:51:30.661 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.661 INFO:teuthology.orchestra.run.smithi063.stdout: osd map max advance = 10 2024-05-08T05:51:30.661 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.661 INFO:teuthology.orchestra.run.smithi063.stdout: journal zero on create = true 2024-05-08T05:51:30.661 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.661 INFO:teuthology.orchestra.run.smithi063.stdout: filestore ondisk finisher threads = 3 2024-05-08T05:51:30.662 INFO:teuthology.orchestra.run.smithi063.stdout: filestore apply finisher threads = 3 2024-05-08T05:51:30.662 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.662 INFO:teuthology.orchestra.run.smithi063.stdout: bdev debug aio = true 2024-05-08T05:51:30.662 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug misdirected ops = true 2024-05-08T05:51:30.662 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore block size = 96636764160 2024-05-08T05:51:30.662 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore fsck on mount = True 2024-05-08T05:51:30.662 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluefs = 20 2024-05-08T05:51:30.663 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluestore = 20 2024-05-08T05:51:30.663 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:51:30.663 INFO:teuthology.orchestra.run.smithi063.stdout: debug osd = 20 2024-05-08T05:51:30.663 INFO:teuthology.orchestra.run.smithi063.stdout: debug rocksdb = 10 2024-05-08T05:51:30.663 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T05:51:30.663 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd full ratio = 0.9 2024-05-08T05:51:30.664 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd nearfull ratio = 0.8 2024-05-08T05:51:30.664 INFO:teuthology.orchestra.run.smithi063.stdout: osd failsafe full ratio = 0.95 2024-05-08T05:51:30.664 INFO:teuthology.orchestra.run.smithi063.stdout: osd objectstore = bluestore 2024-05-08T05:51:30.664 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-08T05:51:30.664 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.664 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr] 2024-05-08T05:51:30.664 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:51:30.664 INFO:teuthology.orchestra.run.smithi063.stdout: debug mgr = 20 2024-05-08T05:51:30.665 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-08T05:51:30.665 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-08T05:51:30.665 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-08T05:51:30.665 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-08T05:51:30.665 INFO:teuthology.orchestra.run.smithi063.stdout: mgr/telemetry/nag = false 2024-05-08T05:51:30.665 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-08T05:51:30.666 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.666 INFO:teuthology.orchestra.run.smithi063.stdout:[mon] 2024-05-08T05:51:30.666 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:51:30.666 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-08T05:51:30.666 INFO:teuthology.orchestra.run.smithi063.stdout: debug paxos = 20 2024-05-08T05:51:30.666 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-08T05:51:30.667 INFO:teuthology.orchestra.run.smithi063.stdout: mon data avail warn = 5 2024-05-08T05:51:30.667 INFO:teuthology.orchestra.run.smithi063.stdout: mon mgr mkfs grace = 240 2024-05-08T05:51:30.667 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-08T05:51:30.667 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd reporter subtree level = osd 2024-05-08T05:51:30.667 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd prime pg temp = true 2024-05-08T05:51:30.667 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-08T05:51:30.668 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.668 INFO:teuthology.orchestra.run.smithi063.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T05:51:30.668 INFO:teuthology.orchestra.run.smithi063.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T05:51:30.668 INFO:teuthology.orchestra.run.smithi063.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T05:51:30.668 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.668 INFO:teuthology.orchestra.run.smithi063.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T05:51:30.668 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T05:51:30.668 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T05:51:30.669 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.669 INFO:teuthology.orchestra.run.smithi063.stdout: # 1m isn't quite enough 2024-05-08T05:51:30.669 INFO:teuthology.orchestra.run.smithi063.stdout: mon_down_mkfs_grace = 2m 2024-05-08T05:51:30.669 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.669 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_filestore_osds = false 2024-05-08T05:51:30.669 INFO:teuthology.orchestra.run.smithi063.stdout: mon down mkfs grace = 300 2024-05-08T05:51:30.669 INFO:teuthology.orchestra.run.smithi063.stdout: mon op complaint time = 120 2024-05-08T05:51:30.669 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-08T05:51:30.669 INFO:teuthology.orchestra.run.smithi063.stdout:[client] 2024-05-08T05:51:30.669 INFO:teuthology.orchestra.run.smithi063.stdout: rgw cache enabled = true 2024-05-08T05:51:30.669 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable ops log = true 2024-05-08T05:51:30.669 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable usage log = true 2024-05-08T05:51:30.669 INFO:teuthology.orchestra.run.smithi063.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T05:51:30.670 INFO:teuthology.orchestra.run.smithi063.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T05:51:30.670 INFO:teuthology.orchestra.run.smithi063.stdout: client mount timeout = 600 2024-05-08T05:51:30.670 INFO:teuthology.orchestra.run.smithi063.stdout: debug client = 20 2024-05-08T05:51:30.670 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:51:30.670 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = "" 2024-05-08T05:51:30.670 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-08T05:51:30.670 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-08T05:51:30.670 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.a] 2024-05-08T05:51:30.670 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.b] 2024-05-08T05:51:30.670 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.c] 2024-05-08T05:51:30.670 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.a] 2024-05-08T05:51:30.670 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.c] 2024-05-08T05:51:30.671 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.b] 2024-05-08T05:51:30.671 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.d] 2024-05-08T05:51:30.671 INFO:teuthology.orchestra.run.smithi063.stdout:[mds] 2024-05-08T05:51:30.671 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds = 20 2024-05-08T05:51:30.671 INFO:teuthology.orchestra.run.smithi063.stdout: debug mds balancer = 20 2024-05-08T05:51:30.671 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-08T05:51:30.671 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug frag = True 2024-05-08T05:51:30.671 INFO:teuthology.orchestra.run.smithi063.stdout: mds debug scatterstat = True 2024-05-08T05:51:30.671 INFO:teuthology.orchestra.run.smithi063.stdout: mds op complaint time = 180 2024-05-08T05:51:30.671 INFO:teuthology.orchestra.run.smithi063.stdout: mds verify scatter = True 2024-05-08T05:51:30.671 INFO:teuthology.orchestra.run.smithi063.stdout: osd op complaint time = 180 2024-05-08T05:51:30.671 INFO:teuthology.orchestra.run.smithi063.stdout: rados mon op timeout = 900 2024-05-08T05:51:30.671 INFO:teuthology.orchestra.run.smithi063.stdout: rados osd op timeout = 900 2024-05-08T05:51:30.673 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-05-08T05:51:30.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.819+0000 7f9f911c8640 1 Processor -- start 2024-05-08T05:51:30.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.819+0000 7f9f911c8640 1 -- start start 2024-05-08T05:51:30.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.820+0000 7f9f911c8640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9f8c0d56b0 0x7f9f8c16a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.820+0000 7f9f911c8640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f8c0d0320 con 0x7f9f8c15f4e0 2024-05-08T05:51:30.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.820+0000 7f9f911c8640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f8c0d0600 con 0x7f9f8c0cfcd0 2024-05-08T05:51:30.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.820+0000 7f9f911c8640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f9f8c0d41b0 con 0x7f9f8c0d56b0 2024-05-08T05:51:30.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.820+0000 7f9f8b59e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9f8c0d56b0 0x7f9f8c16a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.820+0000 7f9f8ad9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f9f8c15f4e0 0x7f9f8c0d3aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46094/0 (socket says 172.21.15.45:46094) 2024-05-08T05:51:30.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.820+0000 7f9f8b59e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f9f8c0d56b0 0x7f9f8c16a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39616/0 (socket says 172.21.15.45:39616) 2024-05-08T05:51:30.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.820+0000 7f9f8ad9d640 1 -- 172.21.15.45:0/2083564186 learned_addr learned my addr 172.21.15.45:0/2083564186 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:30.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.821+0000 7f9f8b59e640 1 -- 172.21.15.45:0/2083564186 >> v1:172.21.15.63:6789/0 conn(0x7f9f8c0cfcd0 legacy=0x7f9f8c0d4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.821+0000 7f9f8b59e640 1 -- 172.21.15.45:0/2083564186 >> v1:172.21.15.45:6789/0 conn(0x7f9f8c15f4e0 legacy=0x7f9f8c0d3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.821+0000 7f9f8b59e640 1 -- 172.21.15.45:0/2083564186 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c0d08e0 con 0x7f9f8c0d56b0 2024-05-08T05:51:30.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.821+0000 7f9f89d9b640 1 -- 172.21.15.45:0/2083564186 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3345533885 0 0) 0x7f9f8c0d0320 con 0x7f9f8c15f4e0 2024-05-08T05:51:30.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.821+0000 7f9f8b59e640 1 --2- 172.21.15.45:0/2083564186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f8c0d56b0 0x7f9f8c16a080 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009d90 tx=0x7f9f7c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae537fc91e8c1ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.822+0000 7f9f89d9b640 1 -- 172.21.15.45:0/2083564186 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c007740 con 0x7f9f8c0d56b0 2024-05-08T05:51:30.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.822+0000 7f9f89d9b640 1 -- 172.21.15.45:0/2083564186 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f7c0078e0 con 0x7f9f8c0d56b0 2024-05-08T05:51:30.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.822+0000 7f9f89d9b640 1 -- 172.21.15.45:0/2083564186 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c007cb0 con 0x7f9f8c0d56b0 2024-05-08T05:51:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.822+0000 7f9f911c8640 1 -- 172.21.15.45:0/2083564186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f8c0d56b0 msgr2=0x7f9f8c16a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.822+0000 7f9f911c8640 1 --2- 172.21.15.45:0/2083564186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f8c0d56b0 0x7f9f8c16a080 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009d90 tx=0x7f9f7c02a6f0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.822+0000 7f9f911c8640 1 -- 172.21.15.45:0/2083564186 shutdown_connections 2024-05-08T05:51:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.822+0000 7f9f911c8640 1 --2- 172.21.15.45:0/2083564186 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f8c0d56b0 0x7f9f8c16a080 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.822+0000 7f9f911c8640 1 -- 172.21.15.45:0/2083564186 >> 172.21.15.45:0/2083564186 conn(0x7f9f8c08eac0 msgr2=0x7f9f8c09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.822+0000 7f9f911c8640 1 -- 172.21.15.45:0/2083564186 shutdown_connections 2024-05-08T05:51:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.822+0000 7f9f911c8640 1 -- 172.21.15.45:0/2083564186 wait complete. 2024-05-08T05:51:30.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.824+0000 7f9f911c8640 1 Processor -- start 2024-05-08T05:51:30.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.824+0000 7f9f911c8640 1 -- start start 2024-05-08T05:51:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.824+0000 7f9f911c8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f8c0cfcd0 0x7f9f8c15b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.824+0000 7f9f911c8640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f8c0d56b0 0x7f9f8c0ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.824+0000 7f9f911c8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f8c15f4e0 0x7f9f8c15fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.824+0000 7f9f911c8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c15f8c0 con 0x7f9f8c15f4e0 2024-05-08T05:51:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.824+0000 7f9f911c8640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c0d0ea0 con 0x7f9f8c0cfcd0 2024-05-08T05:51:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.824+0000 7f9f911c8640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f9f8c0d23d0 con 0x7f9f8c0d56b0 2024-05-08T05:51:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.825+0000 7f9f8b59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f8c15f4e0 0x7f9f8c15fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.825+0000 7f9f8b59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f8c15f4e0 0x7f9f8c15fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39626/0 (socket says 172.21.15.45:39626) 2024-05-08T05:51:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.825+0000 7f9f8b59e640 1 -- 172.21.15.45:0/3886400314 learned_addr learned my addr 172.21.15.45:0/3886400314 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:30.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.825+0000 7f9f8a59c640 1 --2- 172.21.15.45:0/3886400314 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f8c0d56b0 0x7f9f8c0ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.825+0000 7f9f8ad9d640 1 --2- 172.21.15.45:0/3886400314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f8c0cfcd0 0x7f9f8c15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.825+0000 7f9f8b59e640 1 -- 172.21.15.45:0/3886400314 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f8c0d56b0 msgr2=0x7f9f8c0ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.825+0000 7f9f8b59e640 1 --2- 172.21.15.45:0/3886400314 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f8c0d56b0 0x7f9f8c0ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.825+0000 7f9f8b59e640 1 -- 172.21.15.45:0/3886400314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f8c0cfcd0 msgr2=0x7f9f8c15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.825+0000 7f9f8b59e640 1 --2- 172.21.15.45:0/3886400314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f8c0cfcd0 0x7f9f8c15b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.825+0000 7f9f8b59e640 1 -- 172.21.15.45:0/3886400314 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c15bac0 con 0x7f9f8c15f4e0 2024-05-08T05:51:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.825+0000 7f9f8b59e640 1 --2- 172.21.15.45:0/3886400314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f8c15f4e0 0x7f9f8c15fa40 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009d90 tx=0x7f9f7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.825+0000 7f9f837fe640 1 -- 172.21.15.45:0/3886400314 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c040070 con 0x7f9f8c15f4e0 2024-05-08T05:51:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.825+0000 7f9f837fe640 1 -- 172.21.15.45:0/3886400314 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f7c031c90 con 0x7f9f8c15f4e0 2024-05-08T05:51:30.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.825+0000 7f9f837fe640 1 -- 172.21.15.45:0/3886400314 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c03c640 con 0x7f9f8c15f4e0 2024-05-08T05:51:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.825+0000 7f9f911c8640 1 -- 172.21.15.45:0/3886400314 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8c0cda10 con 0x7f9f8c15f4e0 2024-05-08T05:51:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.826+0000 7f9f911c8640 1 -- 172.21.15.45:0/3886400314 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f8c0cdfd0 con 0x7f9f8c15f4e0 2024-05-08T05:51:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.826+0000 7f9f911c8640 1 -- 172.21.15.45:0/3886400314 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f8c0693f0 con 0x7f9f8c15f4e0 2024-05-08T05:51:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.827+0000 7f9f837fe640 1 -- 172.21.15.45:0/3886400314 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9f7c039040 con 0x7f9f8c15f4e0 2024-05-08T05:51:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.828+0000 7f9f837fe640 1 --2- 172.21.15.45:0/3886400314 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9f6c0762d0 0x7f9f6c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:30.829 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.828+0000 7f9f837fe640 1 -- 172.21.15.45:0/3886400314 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f9f7c0b8ae0 con 0x7f9f8c15f4e0 2024-05-08T05:51:30.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.828+0000 7f9f8ad9d640 1 --2- 172.21.15.45:0/3886400314 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9f6c0762d0 0x7f9f6c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:30.830 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.829+0000 7f9f8ad9d640 1 --2- 172.21.15.45:0/3886400314 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9f6c0762d0 0x7f9f6c078790 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9f84000f60 tx=0x7f9f84007d20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:30.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:30.832+0000 7f9f837fe640 1 -- 172.21.15.45:0/3886400314 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9f7c082220 con 0x7f9f8c15f4e0 2024-05-08T05:51:31.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:31.066+0000 7f9f911c8640 1 -- 172.21.15.45:0/3886400314 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f9f8c15eb20 con 0x7f9f8c15f4e0 2024-05-08T05:51:31.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:31.972+0000 7f9f837fe640 1 -- 172.21.15.45:0/3886400314 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v109) ==== 162+0+0 (secure 0 0 0) 0x7f9f7c081b20 con 0x7f9f8c15f4e0 2024-05-08T05:51:31.973 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked joinable; MDS may join as newly active. 2024-05-08T05:51:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:31.975+0000 7f9f911c8640 1 -- 172.21.15.45:0/3886400314 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9f6c0762d0 msgr2=0x7f9f6c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:31.975+0000 7f9f911c8640 1 --2- 172.21.15.45:0/3886400314 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9f6c0762d0 0x7f9f6c078790 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9f84000f60 tx=0x7f9f84007d20 comp rx=0 tx=0).stop 2024-05-08T05:51:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:31.975+0000 7f9f911c8640 1 -- 172.21.15.45:0/3886400314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f8c15f4e0 msgr2=0x7f9f8c15fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:31.975+0000 7f9f911c8640 1 --2- 172.21.15.45:0/3886400314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f8c15f4e0 0x7f9f8c15fa40 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009d90 tx=0x7f9f7c004a40 comp rx=0 tx=0).stop 2024-05-08T05:51:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:31.976+0000 7f9f911c8640 1 -- 172.21.15.45:0/3886400314 shutdown_connections 2024-05-08T05:51:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:31.976+0000 7f9f911c8640 1 --2- 172.21.15.45:0/3886400314 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f9f6c0762d0 0x7f9f6c078790 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:31.976+0000 7f9f911c8640 1 --2- 172.21.15.45:0/3886400314 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f9f8c0d56b0 0x7f9f8c0ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:31.976+0000 7f9f911c8640 1 --2- 172.21.15.45:0/3886400314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9f8c15f4e0 0x7f9f8c15fa40 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:31.976+0000 7f9f911c8640 1 --2- 172.21.15.45:0/3886400314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9f8c0cfcd0 0x7f9f8c15b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:31.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:31.976+0000 7f9f911c8640 1 -- 172.21.15.45:0/3886400314 >> 172.21.15.45:0/3886400314 conn(0x7f9f8c08eac0 msgr2=0x7f9f8c0d3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:31.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:31.976+0000 7f9f911c8640 1 -- 172.21.15.45:0/3886400314 shutdown_connections 2024-05-08T05:51:31.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:31.976+0000 7f9f911c8640 1 -- 172.21.15.45:0/3886400314 wait complete. 2024-05-08T05:51:31.992 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:51:32.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.102+0000 7f3067b24640 1 Processor -- start 2024-05-08T05:51:32.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.102+0000 7f3067b24640 1 -- start start 2024-05-08T05:51:32.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.103+0000 7f3067b24640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f306015f420 0x7f30600d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:32.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.103+0000 7f3067b24640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30600d3ca0 0x7f30600d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:32.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.103+0000 7f3067b24640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f30600d1eb0 0x7f30600d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:32.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.103+0000 7f3067b24640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f30600d22f0 con 0x7f306015f420 2024-05-08T05:51:32.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.103+0000 7f3067b24640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f30601630b0 con 0x7f30600d1eb0 2024-05-08T05:51:32.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.103+0000 7f3067b24640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f30600d2550 con 0x7f30600d3ca0 2024-05-08T05:51:32.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.103+0000 7f30658c0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f306015f420 0x7f30600d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:32.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.103+0000 7f30658c0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f306015f420 0x7f30600d3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39642/0 (socket says 172.21.15.45:39642) 2024-05-08T05:51:32.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.103+0000 7f30658c0640 1 -- 172.21.15.45:0/3633664863 learned_addr learned my addr 172.21.15.45:0/3633664863 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:32.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.103+0000 7f30650bf640 1 --2- 172.21.15.45:0/3633664863 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30600d3ca0 0x7f30600d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:32.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.103+0000 7f30660c1640 1 --2- 172.21.15.45:0/3633664863 >> v2:172.21.15.63:3300/0 conn(0x7f30600d1eb0 0x7f30600d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:32.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.103+0000 7f30658c0640 1 -- 172.21.15.45:0/3633664863 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30600d3ca0 msgr2=0x7f30600d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:32.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.103+0000 7f30658c0640 1 --2- 172.21.15.45:0/3633664863 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30600d3ca0 0x7f30600d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.103+0000 7f30658c0640 1 -- 172.21.15.45:0/3633664863 >> v2:172.21.15.63:3300/0 conn(0x7f30600d1eb0 msgr2=0x7f30600d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:32.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.103+0000 7f30658c0640 1 --2- 172.21.15.45:0/3633664863 >> v2:172.21.15.63:3300/0 conn(0x7f30600d1eb0 0x7f30600d4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.103+0000 7f30658c0640 1 -- 172.21.15.45:0/3633664863 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30600d2c80 con 0x7f306015f420 2024-05-08T05:51:32.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.103+0000 7f30650bf640 1 --2- 172.21.15.45:0/3633664863 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30600d3ca0 0x7f30600d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:32.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.103+0000 7f30660c1640 1 --2- 172.21.15.45:0/3633664863 >> v2:172.21.15.63:3300/0 conn(0x7f30600d1eb0 0x7f30600d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:32.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.104+0000 7f30658c0640 1 --2- 172.21.15.45:0/3633664863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f306015f420 0x7f30600d3760 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f30600d17a0 tx=0x7f305c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f223d0e652462de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:32.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.104+0000 7f30648be640 1 -- 172.21.15.45:0/3633664863 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c037070 con 0x7f306015f420 2024-05-08T05:51:32.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.104+0000 7f30648be640 1 -- 172.21.15.45:0/3633664863 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f305c032440 con 0x7f306015f420 2024-05-08T05:51:32.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.104+0000 7f30648be640 1 -- 172.21.15.45:0/3633664863 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c032810 con 0x7f306015f420 2024-05-08T05:51:32.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.105+0000 7f3067b24640 1 -- 172.21.15.45:0/3633664863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f306015f420 msgr2=0x7f30600d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:32.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.105+0000 7f3067b24640 1 --2- 172.21.15.45:0/3633664863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f306015f420 0x7f30600d3760 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f30600d17a0 tx=0x7f305c02a1e0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.105+0000 7f3067b24640 1 -- 172.21.15.45:0/3633664863 shutdown_connections 2024-05-08T05:51:32.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.105+0000 7f3067b24640 1 --2- 172.21.15.45:0/3633664863 >> v2:172.21.15.63:3300/0 conn(0x7f30600d1eb0 0x7f30600d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.105+0000 7f3067b24640 1 --2- 172.21.15.45:0/3633664863 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30600d3ca0 0x7f30600d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.105+0000 7f3067b24640 1 --2- 172.21.15.45:0/3633664863 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f306015f420 0x7f30600d3760 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.105+0000 7f3067b24640 1 -- 172.21.15.45:0/3633664863 >> 172.21.15.45:0/3633664863 conn(0x7f306008eac0 msgr2=0x7f306009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:32.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.105+0000 7f3067b24640 1 -- 172.21.15.45:0/3633664863 shutdown_connections 2024-05-08T05:51:32.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.105+0000 7f3067b24640 1 -- 172.21.15.45:0/3633664863 wait complete. 2024-05-08T05:51:32.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.106+0000 7f3067b24640 1 Processor -- start 2024-05-08T05:51:32.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.106+0000 7f3067b24640 1 -- start start 2024-05-08T05:51:32.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.107+0000 7f3067b24640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30600d1eb0 0x7f30600b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:32.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.107+0000 7f3067b24640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30600cabf0 0x7f30600cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:32.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.107+0000 7f3067b24640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30601f45e0 0x7f30601f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:32.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.107+0000 7f3067b24640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f30600d2830 con 0x7f30600cabf0 2024-05-08T05:51:32.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.107+0000 7f3067b24640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f30600d3240 con 0x7f30601f45e0 2024-05-08T05:51:32.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.107+0000 7f3067b24640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f30600d5200 con 0x7f30600d1eb0 2024-05-08T05:51:32.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.107+0000 7f30650bf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30600cabf0 0x7f30600cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:32.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.107+0000 7f30650bf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30600cabf0 0x7f30600cb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39646/0 (socket says 172.21.15.45:39646) 2024-05-08T05:51:32.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.107+0000 7f30650bf640 1 -- 172.21.15.45:0/2623809359 learned_addr learned my addr 172.21.15.45:0/2623809359 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:32.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.107+0000 7f30660c1640 1 --2- 172.21.15.45:0/2623809359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30601f45e0 0x7f30601f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:32.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.107+0000 7f30658c0640 1 --2- 172.21.15.45:0/2623809359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30600d1eb0 0x7f30600b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:32.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.107+0000 7f30650bf640 1 -- 172.21.15.45:0/2623809359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30600d1eb0 msgr2=0x7f30600b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:32.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.107+0000 7f30650bf640 1 --2- 172.21.15.45:0/2623809359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30600d1eb0 0x7f30600b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.107+0000 7f30650bf640 1 -- 172.21.15.45:0/2623809359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30601f45e0 msgr2=0x7f30601f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:32.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.107+0000 7f30650bf640 1 --2- 172.21.15.45:0/2623809359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30601f45e0 0x7f30601f4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.107+0000 7f30650bf640 1 -- 172.21.15.45:0/2623809359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30601537e0 con 0x7f30600cabf0 2024-05-08T05:51:32.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.108+0000 7f30650bf640 1 --2- 172.21.15.45:0/2623809359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30600cabf0 0x7f30600cb070 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f305000c820 tx=0x7f305000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:32.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.108+0000 7f30567fc640 1 -- 172.21.15.45:0/2623809359 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050015070 con 0x7f30600cabf0 2024-05-08T05:51:32.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.108+0000 7f30567fc640 1 -- 172.21.15.45:0/2623809359 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3050007a30 con 0x7f30600cabf0 2024-05-08T05:51:32.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.108+0000 7f3067b24640 1 -- 172.21.15.45:0/2623809359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30600cb8d0 con 0x7f30600cabf0 2024-05-08T05:51:32.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.108+0000 7f30567fc640 1 -- 172.21.15.45:0/2623809359 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050007e00 con 0x7f30600cabf0 2024-05-08T05:51:32.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.108+0000 7f3067b24640 1 -- 172.21.15.45:0/2623809359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30600cc460 con 0x7f30600cabf0 2024-05-08T05:51:32.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.109+0000 7f3067b24640 1 -- 172.21.15.45:0/2623809359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30600693f0 con 0x7f30600cabf0 2024-05-08T05:51:32.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.109+0000 7f30567fc640 1 -- 172.21.15.45:0/2623809359 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3050018070 con 0x7f30600cabf0 2024-05-08T05:51:32.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.110+0000 7f30567fc640 1 --2- 172.21.15.45:0/2623809359 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3044076300 0x7f30440787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:32.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.110+0000 7f30567fc640 1 -- 172.21.15.45:0/2623809359 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f3050097700 con 0x7f30600cabf0 2024-05-08T05:51:32.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.110+0000 7f30658c0640 1 --2- 172.21.15.45:0/2623809359 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3044076300 0x7f30440787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:32.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.111+0000 7f30658c0640 1 --2- 172.21.15.45:0/2623809359 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3044076300 0x7f30440787c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f305c034000 tx=0x7f305c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:32.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.114+0000 7f30567fc640 1 -- 172.21.15.45:0/2623809359 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f30500607d0 con 0x7f30600cabf0 2024-05-08T05:51:32.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.343+0000 7f3067b24640 1 -- 172.21.15.45:0/2623809359 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30600b3170 con 0x7f30600cabf0 2024-05-08T05:51:32.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.343+0000 7f30567fc640 1 -- 172.21.15.45:0/2623809359 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5181 (secure 0 0 0) 0x7f3050064ad0 con 0x7f30600cabf0 2024-05-08T05:51:32.344 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:32.344 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":110,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7168,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3882445426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3882445426},{"type":"v1","addr":"172.21.15.63:6836","nonce":3882445426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/3173207720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3173207720},{"type":"v1","addr":"172.21.15.45:6833","nonce":3173207720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4254619360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4254619360},{"type":"v1","addr":"172.21.15.63:6837","nonce":4254619360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:51:04.429358+0000","modified":"2024-05-08T05:51:31.976710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8463":{"gid":8463,"name":"c","rank":0,"incarnation":110,"state":"up:replay","state_seq":1,"addr":"172.21.15.45:6835/4071581846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4071581846},{"type":"v1","addr":"172.21.15.45:6835","nonce":4071581846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-08T05:51:32.344 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 110 2024-05-08T05:51:32.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.345+0000 7f3067b24640 1 -- 172.21.15.45:0/2623809359 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3044076300 msgr2=0x7f30440787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:32.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.345+0000 7f3067b24640 1 --2- 172.21.15.45:0/2623809359 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3044076300 0x7f30440787c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f305c034000 tx=0x7f305c02e000 comp rx=0 tx=0).stop 2024-05-08T05:51:32.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.345+0000 7f3067b24640 1 -- 172.21.15.45:0/2623809359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30600cabf0 msgr2=0x7f30600cb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:32.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.345+0000 7f3067b24640 1 --2- 172.21.15.45:0/2623809359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30600cabf0 0x7f30600cb070 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f305000c820 tx=0x7f305000cc20 comp rx=0 tx=0).stop 2024-05-08T05:51:32.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.345+0000 7f3067b24640 1 -- 172.21.15.45:0/2623809359 shutdown_connections 2024-05-08T05:51:32.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.345+0000 7f3067b24640 1 --2- 172.21.15.45:0/2623809359 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3044076300 0x7f30440787c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.345+0000 7f3067b24640 1 --2- 172.21.15.45:0/2623809359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30601f45e0 0x7f30601f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.345+0000 7f3067b24640 1 --2- 172.21.15.45:0/2623809359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30600cabf0 0x7f30600cb070 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.345+0000 7f3067b24640 1 --2- 172.21.15.45:0/2623809359 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f30600d1eb0 0x7f30600b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.346+0000 7f3067b24640 1 -- 172.21.15.45:0/2623809359 >> 172.21.15.45:0/2623809359 conn(0x7f306008eac0 msgr2=0x7f3060163230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:32.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.346+0000 7f3067b24640 1 -- 172.21.15.45:0/2623809359 shutdown_connections 2024-05-08T05:51:32.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:32.346+0000 7f3067b24640 1 -- 172.21.15.45:0/2623809359 wait complete. 2024-05-08T05:51:32.760 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 110, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:51:04.429358+0000', 'modified': '2024-05-08T05:51:31.976710+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 121, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8463}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8463': {'gid': 8463, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.45:6835/4071581846', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 4071581846}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 4071581846}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-08T05:51:32.760 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8463:up:replay 2024-05-08T05:51:33.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:51:33.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.878+0000 7f43ef464640 1 Processor -- start 2024-05-08T05:51:33.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.878+0000 7f43ef464640 1 -- start start 2024-05-08T05:51:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.879+0000 7f43ef464640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43e80d0ec0 0x7f43e80d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.879+0000 7f43ef464640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f43e80d3b00 0x7f43e80b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.879+0000 7f43ef464640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43e80d1760 con 0x7f43e815f380 2024-05-08T05:51:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.879+0000 7f43ef464640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f43e80d30f0 con 0x7f43e80d3b00 2024-05-08T05:51:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.879+0000 7f43ef464640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f43e80d2370 con 0x7f43e80d0ec0 2024-05-08T05:51:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.879+0000 7f43ed200640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f43e815f380 0x7f43e80d07b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46098/0 (socket says 172.21.15.45:46098) 2024-05-08T05:51:33.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.880+0000 7f43ed200640 1 -- 172.21.15.45:0/658616216 learned_addr learned my addr 172.21.15.45:0/658616216 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.880+0000 7f43eda01640 1 --2- 172.21.15.45:0/658616216 >> v2:172.21.15.45:3300/0 conn(0x7f43e80d3b00 0x7f43e80b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.880+0000 7f43ec9ff640 1 --2- 172.21.15.45:0/658616216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43e80d0ec0 0x7f43e80d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.880+0000 7f43e7fff640 1 -- 172.21.15.45:0/658616216 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3041366919 0 0) 0x7f43e80d1760 con 0x7f43e815f380 2024-05-08T05:51:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.880+0000 7f43e7fff640 1 -- 172.21.15.45:0/658616216 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43d4003520 con 0x7f43e815f380 2024-05-08T05:51:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.880+0000 7f43eda01640 1 -- 172.21.15.45:0/658616216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43e80d0ec0 msgr2=0x7f43e80d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.880+0000 7f43eda01640 1 --2- 172.21.15.45:0/658616216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43e80d0ec0 0x7f43e80d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.880+0000 7f43eda01640 1 -- 172.21.15.45:0/658616216 >> v1:172.21.15.45:6789/0 conn(0x7f43e815f380 legacy=0x7f43e80d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:33.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.880+0000 7f43eda01640 1 -- 172.21.15.45:0/658616216 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e80d1a40 con 0x7f43e80d3b00 2024-05-08T05:51:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.881+0000 7f43eda01640 1 --2- 172.21.15.45:0/658616216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e80d3b00 0x7f43e80b7e30 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f43e00097a0 tx=0x7f43e002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3065281dc0c5a79d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.881+0000 7f43e7fff640 1 -- 172.21.15.45:0/658616216 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e0037070 con 0x7f43e80d3b00 2024-05-08T05:51:33.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.881+0000 7f43e7fff640 1 -- 172.21.15.45:0/658616216 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43e0032440 con 0x7f43e80d3b00 2024-05-08T05:51:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.881+0000 7f43e7fff640 1 -- 172.21.15.45:0/658616216 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e0032810 con 0x7f43e80d3b00 2024-05-08T05:51:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.882+0000 7f43ef464640 1 -- 172.21.15.45:0/658616216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e80d3b00 msgr2=0x7f43e80b7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.882+0000 7f43ef464640 1 --2- 172.21.15.45:0/658616216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e80d3b00 0x7f43e80b7e30 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f43e00097a0 tx=0x7f43e002a180 comp rx=0 tx=0).stop 2024-05-08T05:51:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.882+0000 7f43ef464640 1 -- 172.21.15.45:0/658616216 shutdown_connections 2024-05-08T05:51:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.882+0000 7f43ef464640 1 --2- 172.21.15.45:0/658616216 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e80d3b00 0x7f43e80b7e30 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.882+0000 7f43ef464640 1 --2- 172.21.15.45:0/658616216 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43e80d0ec0 0x7f43e80d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.882+0000 7f43ef464640 1 -- 172.21.15.45:0/658616216 >> 172.21.15.45:0/658616216 conn(0x7f43e80d4fe0 msgr2=0x7f43e80d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.882+0000 7f43ef464640 1 -- 172.21.15.45:0/658616216 shutdown_connections 2024-05-08T05:51:33.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.882+0000 7f43ef464640 1 -- 172.21.15.45:0/658616216 wait complete. 2024-05-08T05:51:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.883+0000 7f43ef464640 1 Processor -- start 2024-05-08T05:51:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.883+0000 7f43ef464640 1 -- start start 2024-05-08T05:51:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.884+0000 7f43ef464640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43e80d0ec0 0x7f43e80ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.884+0000 7f43ef464640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e80d3b00 0x7f43e80b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.884+0000 7f43ef464640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43e815f380 0x7f43e80b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.884+0000 7f43ef464640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f43e815f760 con 0x7f43e80d3b00 2024-05-08T05:51:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.884+0000 7f43ef464640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f43e80d2000 con 0x7f43e80d0ec0 2024-05-08T05:51:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.884+0000 7f43ef464640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f43e80d2800 con 0x7f43e815f380 2024-05-08T05:51:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.884+0000 7f43ec9ff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e80d3b00 0x7f43e80b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.884+0000 7f43ec9ff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e80d3b00 0x7f43e80b5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39658/0 (socket says 172.21.15.45:39658) 2024-05-08T05:51:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.884+0000 7f43ec9ff640 1 -- 172.21.15.45:0/3253607579 learned_addr learned my addr 172.21.15.45:0/3253607579 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.884+0000 7f43ed200640 1 --2- 172.21.15.45:0/3253607579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43e80d0ec0 0x7f43e80ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.884+0000 7f43ec9ff640 1 -- 172.21.15.45:0/3253607579 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43e815f380 msgr2=0x7f43e80b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.884+0000 7f43ec9ff640 1 --2- 172.21.15.45:0/3253607579 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43e815f380 0x7f43e80b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.884+0000 7f43ec9ff640 1 -- 172.21.15.45:0/3253607579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43e80d0ec0 msgr2=0x7f43e80ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.884+0000 7f43ec9ff640 1 --2- 172.21.15.45:0/3253607579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43e80d0ec0 0x7f43e80ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.884+0000 7f43ec9ff640 1 -- 172.21.15.45:0/3253607579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e81f95f0 con 0x7f43e80d3b00 2024-05-08T05:51:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.884+0000 7f43ec9ff640 1 --2- 172.21.15.45:0/3253607579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e80d3b00 0x7f43e80b5e60 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f43d000e9b0 tx=0x7f43d000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.885+0000 7f43e5ffb640 1 -- 172.21.15.45:0/3253607579 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0017070 con 0x7f43e80d3b00 2024-05-08T05:51:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.885+0000 7f43e5ffb640 1 -- 172.21.15.45:0/3253607579 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43d000c9a0 con 0x7f43e80d3b00 2024-05-08T05:51:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.885+0000 7f43e5ffb640 1 -- 172.21.15.45:0/3253607579 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d000cd70 con 0x7f43e80d3b00 2024-05-08T05:51:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.885+0000 7f43ef464640 1 -- 172.21.15.45:0/3253607579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e809eda0 con 0x7f43e80d3b00 2024-05-08T05:51:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.885+0000 7f43ef464640 1 -- 172.21.15.45:0/3253607579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e809f360 con 0x7f43e80d3b00 2024-05-08T05:51:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.886+0000 7f43ef464640 1 -- 172.21.15.45:0/3253607579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e80693f0 con 0x7f43e80d3b00 2024-05-08T05:51:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.886+0000 7f43e5ffb640 1 -- 172.21.15.45:0/3253607579 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f43d001a070 con 0x7f43e80d3b00 2024-05-08T05:51:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.887+0000 7f43e5ffb640 1 --2- 172.21.15.45:0/3253607579 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f43c80762d0 0x7f43c8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.887+0000 7f43e5ffb640 1 -- 172.21.15.45:0/3253607579 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f43d0097be0 con 0x7f43e80d3b00 2024-05-08T05:51:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.887+0000 7f43ed200640 1 --2- 172.21.15.45:0/3253607579 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f43c80762d0 0x7f43c8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.888+0000 7f43ed200640 1 --2- 172.21.15.45:0/3253607579 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f43c80762d0 0x7f43c8078790 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f43d80015a0 tx=0x7f43d8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:33.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:33.891+0000 7f43e5ffb640 1 -- 172.21.15.45:0/3253607579 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f43d0061300 con 0x7f43e80d3b00 2024-05-08T05:51:34.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:34.127+0000 7f43ef464640 1 -- 172.21.15.45:0/3253607579 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43e815eaa0 con 0x7f43e80d3b00 2024-05-08T05:51:34.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:34.128+0000 7f43e5ffb640 1 -- 172.21.15.45:0/3253607579 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5181 (secure 0 0 0) 0x7f43d0060c00 con 0x7f43e80d3b00 2024-05-08T05:51:34.128 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:34.128 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7168,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3882445426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3882445426},{"type":"v1","addr":"172.21.15.63:6836","nonce":3882445426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/3173207720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3173207720},{"type":"v1","addr":"172.21.15.45:6833","nonce":3173207720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4254619360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4254619360},{"type":"v1","addr":"172.21.15.63:6837","nonce":4254619360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:51:04.429358+0000","modified":"2024-05-08T05:51:32.982158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8463":{"gid":8463,"name":"c","rank":0,"incarnation":110,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.45:6835/4071581846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4071581846},{"type":"v1","addr":"172.21.15.45:6835","nonce":4071581846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-05-08T05:51:34.128 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 112 2024-05-08T05:51:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:34.130+0000 7f43ef464640 1 -- 172.21.15.45:0/3253607579 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f43c80762d0 msgr2=0x7f43c8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:34.130+0000 7f43ef464640 1 --2- 172.21.15.45:0/3253607579 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f43c80762d0 0x7f43c8078790 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f43d80015a0 tx=0x7f43d8009040 comp rx=0 tx=0).stop 2024-05-08T05:51:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:34.130+0000 7f43ef464640 1 -- 172.21.15.45:0/3253607579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e80d3b00 msgr2=0x7f43e80b5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:34.130+0000 7f43ef464640 1 --2- 172.21.15.45:0/3253607579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e80d3b00 0x7f43e80b5e60 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f43d000e9b0 tx=0x7f43d000ee70 comp rx=0 tx=0).stop 2024-05-08T05:51:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:34.130+0000 7f43ef464640 1 -- 172.21.15.45:0/3253607579 shutdown_connections 2024-05-08T05:51:34.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:34.130+0000 7f43ef464640 1 --2- 172.21.15.45:0/3253607579 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f43c80762d0 0x7f43c8078790 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:34.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:34.130+0000 7f43ef464640 1 --2- 172.21.15.45:0/3253607579 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f43e815f380 0x7f43e80b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:34.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:34.130+0000 7f43ef464640 1 --2- 172.21.15.45:0/3253607579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43e80d3b00 0x7f43e80b5e60 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:34.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:34.130+0000 7f43ef464640 1 --2- 172.21.15.45:0/3253607579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f43e80d0ec0 0x7f43e80ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:34.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:34.130+0000 7f43ef464640 1 -- 172.21.15.45:0/3253607579 >> 172.21.15.45:0/3253607579 conn(0x7f43e80d4fe0 msgr2=0x7f43e806cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:34.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:34.130+0000 7f43ef464640 1 -- 172.21.15.45:0/3253607579 shutdown_connections 2024-05-08T05:51:34.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:34.130+0000 7f43ef464640 1 -- 172.21.15.45:0/3253607579 wait complete. 2024-05-08T05:51:34.544 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:51:04.429358+0000', 'modified': '2024-05-08T05:51:32.982158+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 121, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8463}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8463': {'gid': 8463, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.45:6835/4071581846', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 4071581846}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 4071581846}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-05-08T05:51:34.544 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8463:up:rejoin 2024-05-08T05:51:35.358 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:51:35.361 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:51:35.545 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:51:35.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.661+0000 7f27f40c4640 1 Processor -- start 2024-05-08T05:51:35.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.662+0000 7f27f40c4640 1 -- start start 2024-05-08T05:51:35.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.662+0000 7f27f40c4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f27ec0b21f0 0x7f27ec06e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:35.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.662+0000 7f27f40c4640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27ec0d0500 con 0x7f27ec162f50 2024-05-08T05:51:35.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f40c4640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27ec06da60 con 0x7f27ec0cfea0 2024-05-08T05:51:35.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f40c4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f27ec06d810 con 0x7f27ec0b21f0 2024-05-08T05:51:35.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f165f640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f27ec162f50 0x7f27ec0cf790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46102/0 (socket says 172.21.15.45:46102) 2024-05-08T05:51:35.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f165f640 1 -- 172.21.15.45:0/3264434295 learned_addr learned my addr 172.21.15.45:0/3264434295 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:35.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f1e60640 1 --2- 172.21.15.45:0/3264434295 >> v2:172.21.15.63:3300/0 conn(0x7f27ec0b21f0 0x7f27ec06e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:35.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f0e5e640 1 -- 172.21.15.45:0/3264434295 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3682405499 0 0) 0x7f27ec0d0500 con 0x7f27ec162f50 2024-05-08T05:51:35.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f0e5e640 1 -- 172.21.15.45:0/3264434295 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27d8003520 con 0x7f27ec162f50 2024-05-08T05:51:35.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f0e5e640 1 -- 172.21.15.45:0/3264434295 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3257272658 0 0) 0x7f27d8003520 con 0x7f27ec162f50 2024-05-08T05:51:35.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f0e5e640 1 -- 172.21.15.45:0/3264434295 >> v2:172.21.15.63:3300/0 conn(0x7f27ec0b21f0 msgr2=0x7f27ec06e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:35.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f0e5e640 1 --2- 172.21.15.45:0/3264434295 >> v2:172.21.15.63:3300/0 conn(0x7f27ec0b21f0 0x7f27ec06e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f0e5e640 1 -- 172.21.15.45:0/3264434295 >> v1:172.21.15.63:6789/0 conn(0x7f27ec0cfea0 legacy=0x7f27ec0cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:35.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f0e5e640 1 -- 172.21.15.45:0/3264434295 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ec06dd40 con 0x7f27ec162f50 2024-05-08T05:51:35.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f0e5e640 1 -- 172.21.15.45:0/3264434295 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f27dc0016f0 con 0x7f27ec162f50 2024-05-08T05:51:35.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f0e5e640 1 -- 172.21.15.45:0/3264434295 >> v1:172.21.15.45:6789/0 conn(0x7f27ec162f50 legacy=0x7f27ec0cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:35.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f0e5e640 1 --2- 172.21.15.45:0/3264434295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27d8003a50 0x7f27d8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:35.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f0e5e640 1 --2- 172.21.15.45:0/3264434295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d8004800 0x7f27d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:35.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f0e5e640 1 --2- 172.21.15.45:0/3264434295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27d80053c0 0x7f27d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:35.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f0e5e640 1 -- 172.21.15.45:0/3264434295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec06d810 con 0x7f27d8004800 2024-05-08T05:51:35.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f0e5e640 1 -- 172.21.15.45:0/3264434295 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f27d800c230 con 0x7f27d80053c0 2024-05-08T05:51:35.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f0e5e640 1 -- 172.21.15.45:0/3264434295 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f27d800c430 con 0x7f27d8003a50 2024-05-08T05:51:35.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.663+0000 7f27f165f640 1 --2- 172.21.15.45:0/3264434295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d8004800 0x7f27d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:35.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.664+0000 7f27f1e60640 1 --2- 172.21.15.45:0/3264434295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27d8003a50 0x7f27d8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:35.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.664+0000 7f27f2661640 1 --2- 172.21.15.45:0/3264434295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27d80053c0 0x7f27d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:35.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.664+0000 7f27f165f640 1 -- 172.21.15.45:0/3264434295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27d8003a50 msgr2=0x7f27d8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:35.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.664+0000 7f27f165f640 1 --2- 172.21.15.45:0/3264434295 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27d8003a50 0x7f27d8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.664+0000 7f27f165f640 1 -- 172.21.15.45:0/3264434295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27d80053c0 msgr2=0x7f27d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:35.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.664+0000 7f27f165f640 1 --2- 172.21.15.45:0/3264434295 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27d80053c0 0x7f27d800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.664+0000 7f27f165f640 1 -- 172.21.15.45:0/3264434295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d8003520 con 0x7f27d8004800 2024-05-08T05:51:35.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.664+0000 7f27f2661640 1 -- 172.21.15.45:0/3264434295 reap_dead start 2024-05-08T05:51:35.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.665+0000 7f27f165f640 1 --2- 172.21.15.45:0/3264434295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d8004800 0x7f27d8004c80 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00c240 tx=0x7f27d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2e7fe304e387d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:35.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.665+0000 7f27f0e5e640 1 -- 172.21.15.45:0/3264434295 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc007d70 con 0x7f27d8004800 2024-05-08T05:51:35.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.665+0000 7f27f0e5e640 1 -- 172.21.15.45:0/3264434295 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27dc005af0 con 0x7f27d8004800 2024-05-08T05:51:35.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.665+0000 7f27f0e5e640 1 -- 172.21.15.45:0/3264434295 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc007070 con 0x7f27d8004800 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.665+0000 7f27f40c4640 1 -- 172.21.15.45:0/3264434295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d8004800 msgr2=0x7f27d8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.665+0000 7f27f40c4640 1 --2- 172.21.15.45:0/3264434295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d8004800 0x7f27d8004c80 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00c240 tx=0x7f27d8005830 comp rx=0 tx=0).stop 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.665+0000 7f27f40c4640 1 -- 172.21.15.45:0/3264434295 shutdown_connections 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.665+0000 7f27f40c4640 1 --2- 172.21.15.45:0/3264434295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27d8004800 0x7f27d8004c80 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.665+0000 7f27f40c4640 1 -- 172.21.15.45:0/3264434295 >> 172.21.15.45:0/3264434295 conn(0x7f27ec09ba40 msgr2=0x7f27ec09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.665+0000 7f27f40c4640 1 -- 172.21.15.45:0/3264434295 shutdown_connections 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.665+0000 7f27f40c4640 1 -- 172.21.15.45:0/3264434295 wait complete. 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.667+0000 7f27f40c4640 1 Processor -- start 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.667+0000 7f27f40c4640 1 -- start start 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.667+0000 7f27f40c4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27d8004800 0x7f27ec14c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:35.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.667+0000 7f27f40c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27ec0c9630 0x7f27ec0c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:35.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.667+0000 7f27f40c4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27ec0d32a0 0x7f27ec148940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:35.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.667+0000 7f27f40c4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec1633e0 con 0x7f27ec0c9630 2024-05-08T05:51:35.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.667+0000 7f27f40c4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec06e0b0 con 0x7f27d8004800 2024-05-08T05:51:35.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.667+0000 7f27f40c4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f27ec163560 con 0x7f27ec0d32a0 2024-05-08T05:51:35.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.668+0000 7f27f165f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27ec0c9630 0x7f27ec0c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:35.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.668+0000 7f27f165f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27ec0c9630 0x7f27ec0c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39674/0 (socket says 172.21.15.45:39674) 2024-05-08T05:51:35.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.668+0000 7f27f165f640 1 -- 172.21.15.45:0/2296054499 learned_addr learned my addr 172.21.15.45:0/2296054499 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:35.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.668+0000 7f27f2661640 1 --2- 172.21.15.45:0/2296054499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27ec0d32a0 0x7f27ec148940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:35.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.668+0000 7f27f1e60640 1 --2- 172.21.15.45:0/2296054499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27d8004800 0x7f27ec14c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:35.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.668+0000 7f27f165f640 1 -- 172.21.15.45:0/2296054499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27ec0d32a0 msgr2=0x7f27ec148940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:35.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.668+0000 7f27f165f640 1 --2- 172.21.15.45:0/2296054499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27ec0d32a0 0x7f27ec148940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.668+0000 7f27f165f640 1 -- 172.21.15.45:0/2296054499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27d8004800 msgr2=0x7f27ec14c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:35.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.668+0000 7f27f165f640 1 --2- 172.21.15.45:0/2296054499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27d8004800 0x7f27ec14c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.668+0000 7f27f165f640 1 -- 172.21.15.45:0/2296054499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ec0c6ea0 con 0x7f27ec0c9630 2024-05-08T05:51:35.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.668+0000 7f27f165f640 1 --2- 172.21.15.45:0/2296054499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27ec0c9630 0x7f27ec0c6960 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00c240 tx=0x7f27ec06dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:35.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.668+0000 7f27e27fc640 1 -- 172.21.15.45:0/2296054499 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc037040 con 0x7f27ec0c9630 2024-05-08T05:51:35.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.668+0000 7f27e27fc640 1 -- 172.21.15.45:0/2296054499 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27dc041070 con 0x7f27ec0c9630 2024-05-08T05:51:35.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.668+0000 7f27f40c4640 1 -- 172.21.15.45:0/2296054499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27ec0c8870 con 0x7f27ec0c9630 2024-05-08T05:51:35.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.668+0000 7f27e27fc640 1 -- 172.21.15.45:0/2296054499 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc0016f0 con 0x7f27ec0c9630 2024-05-08T05:51:35.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.668+0000 7f27f40c4640 1 -- 172.21.15.45:0/2296054499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27ec0c8e30 con 0x7f27ec0c9630 2024-05-08T05:51:35.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.669+0000 7f27f40c4640 1 -- 172.21.15.45:0/2296054499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27ec069b00 con 0x7f27ec0c9630 2024-05-08T05:51:35.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.670+0000 7f27e27fc640 1 -- 172.21.15.45:0/2296054499 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f27dc0047e0 con 0x7f27ec0c9630 2024-05-08T05:51:35.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.671+0000 7f27e27fc640 1 --2- 172.21.15.45:0/2296054499 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f27d00762d0 0x7f27d0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:35.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.671+0000 7f27e27fc640 1 -- 172.21.15.45:0/2296054499 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f27dc0b73f0 con 0x7f27ec0c9630 2024-05-08T05:51:35.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.671+0000 7f27f1e60640 1 --2- 172.21.15.45:0/2296054499 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f27d00762d0 0x7f27d0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:35.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.672+0000 7f27f1e60640 1 --2- 172.21.15.45:0/2296054499 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f27d00762d0 0x7f27d0078790 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0520f0 tx=0x7f27e8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:35.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.675+0000 7f27e27fc640 1 -- 172.21.15.45:0/2296054499 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f27dc080ba0 con 0x7f27ec0c9630 2024-05-08T05:51:35.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.908+0000 7f27f40c4640 1 -- 172.21.15.45:0/2296054499 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27ec070a30 con 0x7f27ec0c9630 2024-05-08T05:51:35.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.908+0000 7f27e27fc640 1 -- 172.21.15.45:0/2296054499 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5188 (secure 0 0 0) 0x7f27dc0804a0 con 0x7f27ec0c9630 2024-05-08T05:51:35.908 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:35.908 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":113,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7168,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3882445426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3882445426},{"type":"v1","addr":"172.21.15.63:6836","nonce":3882445426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/3173207720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3173207720},{"type":"v1","addr":"172.21.15.45:6833","nonce":3173207720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4254619360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4254619360},{"type":"v1","addr":"172.21.15.63:6837","nonce":4254619360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:51:04.429358+0000","modified":"2024-05-08T05:51:34.985085+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8463":{"gid":8463,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/4071581846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4071581846},{"type":"v1","addr":"172.21.15.45:6835","nonce":4071581846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8463,"qdb_cluster":[8463]},"id":8}]} 2024-05-08T05:51:35.909 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 113 2024-05-08T05:51:35.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.910+0000 7f27f40c4640 1 -- 172.21.15.45:0/2296054499 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f27d00762d0 msgr2=0x7f27d0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:35.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.910+0000 7f27f40c4640 1 --2- 172.21.15.45:0/2296054499 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f27d00762d0 0x7f27d0078790 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0520f0 tx=0x7f27e8008040 comp rx=0 tx=0).stop 2024-05-08T05:51:35.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.910+0000 7f27f40c4640 1 -- 172.21.15.45:0/2296054499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27ec0c9630 msgr2=0x7f27ec0c6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:35.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.910+0000 7f27f40c4640 1 --2- 172.21.15.45:0/2296054499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27ec0c9630 0x7f27ec0c6960 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00c240 tx=0x7f27ec06dc30 comp rx=0 tx=0).stop 2024-05-08T05:51:35.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.910+0000 7f27f40c4640 1 -- 172.21.15.45:0/2296054499 shutdown_connections 2024-05-08T05:51:35.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.910+0000 7f27f40c4640 1 --2- 172.21.15.45:0/2296054499 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f27d00762d0 0x7f27d0078790 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.910+0000 7f27f40c4640 1 --2- 172.21.15.45:0/2296054499 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f27ec0d32a0 0x7f27ec148940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.911+0000 7f27f40c4640 1 --2- 172.21.15.45:0/2296054499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27ec0c9630 0x7f27ec0c6960 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.911+0000 7f27f40c4640 1 --2- 172.21.15.45:0/2296054499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27d8004800 0x7f27ec14c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.911 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.911+0000 7f27f40c4640 1 -- 172.21.15.45:0/2296054499 >> 172.21.15.45:0/2296054499 conn(0x7f27ec09ba40 msgr2=0x7f27ec0d1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:35.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.911+0000 7f27f40c4640 1 -- 172.21.15.45:0/2296054499 shutdown_connections 2024-05-08T05:51:35.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:35.911+0000 7f27f40c4640 1 -- 172.21.15.45:0/2296054499 wait complete. 2024-05-08T05:51:35.970 INFO:tasks.ceph.mds.c.smithi045.stderr:2024-05-08T05:51:35.970+0000 7ff259610640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:51:36.327 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 113, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:51:04.429358+0000', 'modified': '2024-05-08T05:51:34.985085+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 121, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8463}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8463': {'gid': 8463, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.45:6835/4071581846', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6834', 'nonce': 4071581846}, {'type': 'v1', 'addr': '172.21.15.45:6835', 'nonce': 4071581846}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8463, 'qdb_cluster': [8463]} 2024-05-08T05:51:36.327 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T05:51:36.327 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T05:51:36.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.438+0000 7f2035ba4640 1 Processor -- start 2024-05-08T05:51:36.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.438+0000 7f2035ba4640 1 -- start start 2024-05-08T05:51:36.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.439+0000 7f2035ba4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f203015f430 0x7f2030073e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.439+0000 7f2035ba4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f203006e6f0 0x7f203006eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.439+0000 7f2035ba4640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f203015f810 con 0x7f20300b5f60 2024-05-08T05:51:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.439+0000 7f2035ba4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f2030072d80 con 0x7f203006e6f0 2024-05-08T05:51:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.439+0000 7f2035ba4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2030073060 con 0x7f203015f430 2024-05-08T05:51:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.439+0000 7f202ffff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f203006e6f0 0x7f203006eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.439+0000 7f202f7fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f203015f430 0x7f2030073e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.439+0000 7f202effd640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f20300b5f60 0x7f2030072480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:33484/0 (socket says 172.21.15.45:33484) 2024-05-08T05:51:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.439+0000 7f202f7fe640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f203015f430 0x7f2030073e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:52010/0 (socket says 172.21.15.45:52010) 2024-05-08T05:51:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.439+0000 7f202ffff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f203006e6f0 0x7f203006eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:60380/0 (socket says 172.21.15.45:60380) 2024-05-08T05:51:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.439+0000 7f202effd640 1 -- 172.21.15.45:0/3331343600 learned_addr learned my addr 172.21.15.45:0/3331343600 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.439+0000 7f202f7fe640 1 -- 172.21.15.45:0/3331343600 >> v2:172.21.15.63:3300/0 conn(0x7f203006e6f0 msgr2=0x7f203006eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:36.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.439+0000 7f202f7fe640 1 --2- 172.21.15.45:0/3331343600 >> v2:172.21.15.63:3300/0 conn(0x7f203006e6f0 0x7f203006eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.439+0000 7f202f7fe640 1 -- 172.21.15.45:0/3331343600 >> v1:172.21.15.63:6789/0 conn(0x7f20300b5f60 legacy=0x7f2030072480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.440+0000 7f202f7fe640 1 -- 172.21.15.45:0/3331343600 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2030073340 con 0x7f203015f430 2024-05-08T05:51:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.439+0000 7f202e7fc640 1 -- 172.21.15.45:0/3331343600 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 36654688 0 0) 0x7f203015f810 con 0x7f20300b5f60 2024-05-08T05:51:36.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.440+0000 7f202ffff640 1 --2- 172.21.15.45:0/3331343600 >> v2:172.21.15.63:3300/0 conn(0x7f203006e6f0 0x7f203006eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:36.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.440+0000 7f202f7fe640 1 --2- 172.21.15.45:0/3331343600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f203015f430 0x7f2030073e20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f20200097a0 tx=0x7f202002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b425d0216c8f0e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:36.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.440+0000 7f202e7fc640 1 -- 172.21.15.45:0/3331343600 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020037070 con 0x7f203015f430 2024-05-08T05:51:36.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.440+0000 7f202e7fc640 1 -- 172.21.15.45:0/3331343600 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f202002ae50 con 0x7f203015f430 2024-05-08T05:51:36.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.441+0000 7f202e7fc640 1 -- 172.21.15.45:0/3331343600 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020032660 con 0x7f203015f430 2024-05-08T05:51:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.441+0000 7f2035ba4640 1 -- 172.21.15.45:0/3331343600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f203015f430 msgr2=0x7f2030073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.441+0000 7f2035ba4640 1 --2- 172.21.15.45:0/3331343600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f203015f430 0x7f2030073e20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f20200097a0 tx=0x7f202002a100 comp rx=0 tx=0).stop 2024-05-08T05:51:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.441+0000 7f2035ba4640 1 -- 172.21.15.45:0/3331343600 shutdown_connections 2024-05-08T05:51:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.441+0000 7f2035ba4640 1 --2- 172.21.15.45:0/3331343600 >> v2:172.21.15.63:3300/0 conn(0x7f203006e6f0 0x7f203006eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.441+0000 7f2035ba4640 1 --2- 172.21.15.45:0/3331343600 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f203015f430 0x7f2030073e20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.441+0000 7f2035ba4640 1 -- 172.21.15.45:0/3331343600 >> 172.21.15.45:0/3331343600 conn(0x7f20300b2cc0 msgr2=0x7f20300b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.441+0000 7f2035ba4640 1 -- 172.21.15.45:0/3331343600 shutdown_connections 2024-05-08T05:51:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.441+0000 7f2035ba4640 1 -- 172.21.15.45:0/3331343600 wait complete. 2024-05-08T05:51:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.442+0000 7f2035ba4640 1 Processor -- start 2024-05-08T05:51:36.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.443+0000 7f2035ba4640 1 -- start start 2024-05-08T05:51:36.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.443+0000 7f2035ba4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f203006e6f0 0x7f203015abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.443+0000 7f2035ba4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20300b5f60 0x7f20301587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.443+0000 7f2035ba4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f203015f430 0x7f20300b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.443+0000 7f2035ba4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f20300cf150 con 0x7f20300b5f60 2024-05-08T05:51:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.443+0000 7f2035ba4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f20300cefd0 con 0x7f203015f430 2024-05-08T05:51:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.443+0000 7f2035ba4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f20300cf2d0 con 0x7f203006e6f0 2024-05-08T05:51:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.443+0000 7f202effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20300b5f60 0x7f20301587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.444+0000 7f202effd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20300b5f60 0x7f20301587c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39682/0 (socket says 172.21.15.45:39682) 2024-05-08T05:51:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.444+0000 7f202effd640 1 -- 172.21.15.45:0/376924870 learned_addr learned my addr 172.21.15.45:0/376924870 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.444+0000 7f202ffff640 1 --2- 172.21.15.45:0/376924870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f203015f430 0x7f20300b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.444+0000 7f202f7fe640 1 --2- 172.21.15.45:0/376924870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f203006e6f0 0x7f203015abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.444+0000 7f202effd640 1 -- 172.21.15.45:0/376924870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f203006e6f0 msgr2=0x7f203015abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.444+0000 7f202effd640 1 --2- 172.21.15.45:0/376924870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f203006e6f0 0x7f203015abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.444+0000 7f202effd640 1 -- 172.21.15.45:0/376924870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f203015f430 msgr2=0x7f20300b15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:36.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.444+0000 7f202effd640 1 --2- 172.21.15.45:0/376924870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f203015f430 0x7f20300b15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.444+0000 7f202effd640 1 -- 172.21.15.45:0/376924870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f203015b130 con 0x7f20300b5f60 2024-05-08T05:51:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.444+0000 7f202ffff640 1 --2- 172.21.15.45:0/376924870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f203015f430 0x7f20300b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.444+0000 7f202effd640 1 --2- 172.21.15.45:0/376924870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20300b5f60 0x7f20301587c0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f202400c480 tx=0x7f202400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.444+0000 7f2017fff640 1 -- 172.21.15.45:0/376924870 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2024018070 con 0x7f20300b5f60 2024-05-08T05:51:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.444+0000 7f2017fff640 1 -- 172.21.15.45:0/376924870 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2024007500 con 0x7f20300b5f60 2024-05-08T05:51:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.444+0000 7f2035ba4640 1 -- 172.21.15.45:0/376924870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2030159210 con 0x7f20300b5f60 2024-05-08T05:51:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.444+0000 7f2017fff640 1 -- 172.21.15.45:0/376924870 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2024004a20 con 0x7f20300b5f60 2024-05-08T05:51:36.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.444+0000 7f2035ba4640 1 -- 172.21.15.45:0/376924870 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20301597d0 con 0x7f20300b5f60 2024-05-08T05:51:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.446+0000 7f2017fff640 1 -- 172.21.15.45:0/376924870 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f202401b070 con 0x7f20300b5f60 2024-05-08T05:51:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.447+0000 7f2017fff640 1 --2- 172.21.15.45:0/376924870 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f201c0762d0 0x7f201c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.447+0000 7f2017fff640 1 -- 172.21.15.45:0/376924870 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f2024096ac0 con 0x7f20300b5f60 2024-05-08T05:51:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.447+0000 7f202f7fe640 1 --2- 172.21.15.45:0/376924870 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f201c0762d0 0x7f201c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.447+0000 7f2038999740 1 Processor -- start 2024-05-08T05:51:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.447+0000 7f2038999740 1 -- start start 2024-05-08T05:51:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.448+0000 7f202f7fe640 1 --2- 172.21.15.45:0/376924870 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f201c0762d0 0x7f201c078790 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2020034000 tx=0x7f202002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.448+0000 7f2038999740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f61af50 0x56186f61d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.448+0000 7f2038999740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56186f61b330 0x56186f61dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.448+0000 7f2038999740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56186f5edbb0 0x56186f5f13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.448+0000 7f2038999740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56186f5578d0 con 0x56186f61af50 2024-05-08T05:51:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.448+0000 7f2038999740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56186f510c10 con 0x56186f5edbb0 2024-05-08T05:51:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.448+0000 7f2038999740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56186f553890 con 0x56186f61b330 2024-05-08T05:51:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.448+0000 7f202f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f61af50 0x56186f61d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.449+0000 7f202f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f61af50 0x56186f61d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39686/0 (socket says 172.21.15.45:39686) 2024-05-08T05:51:36.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.449+0000 7f202f7fe640 1 -- 172.21.15.45:0/4169409560 learned_addr learned my addr 172.21.15.45:0/4169409560 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.449+0000 7f202ffff640 1 --2- 172.21.15.45:0/4169409560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56186f5edbb0 0x56186f5f13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.449+0000 7f202effd640 1 --2- 172.21.15.45:0/4169409560 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56186f61b330 0x56186f61dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.449+0000 7f202f7fe640 1 -- 172.21.15.45:0/4169409560 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56186f61b330 msgr2=0x56186f61dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.449+0000 7f202f7fe640 1 --2- 172.21.15.45:0/4169409560 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56186f61b330 0x56186f61dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.449+0000 7f202f7fe640 1 -- 172.21.15.45:0/4169409560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56186f5edbb0 msgr2=0x56186f5f13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.449+0000 7f202f7fe640 1 --2- 172.21.15.45:0/4169409560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56186f5edbb0 0x56186f5f13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.449+0000 7f202f7fe640 1 -- 172.21.15.45:0/4169409560 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56186f5185d0 con 0x56186f61af50 2024-05-08T05:51:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.449+0000 7f202f7fe640 1 --2- 172.21.15.45:0/4169409560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f61af50 0x56186f61d760 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f202002a6f0 tx=0x7f202002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c884c0a9e51121b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.449+0000 7f20157fa640 1 -- 172.21.15.45:0/4169409560 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020037040 con 0x56186f61af50 2024-05-08T05:51:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.449+0000 7f20157fa640 1 -- 172.21.15.45:0/4169409560 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f202004d070 con 0x56186f61af50 2024-05-08T05:51:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.449+0000 7f20157fa640 1 -- 172.21.15.45:0/4169409560 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202003d690 con 0x56186f61af50 2024-05-08T05:51:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.450+0000 7f2038999740 1 -- 172.21.15.45:0/4169409560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f61af50 msgr2=0x56186f61d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.450+0000 7f2038999740 1 --2- 172.21.15.45:0/4169409560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f61af50 0x56186f61d760 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f202002a6f0 tx=0x7f202002c000 comp rx=0 tx=0).stop 2024-05-08T05:51:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.450+0000 7f2038999740 1 -- 172.21.15.45:0/4169409560 shutdown_connections 2024-05-08T05:51:36.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.450+0000 7f2038999740 1 --2- 172.21.15.45:0/4169409560 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56186f61b330 0x56186f61dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.450+0000 7f2038999740 1 --2- 172.21.15.45:0/4169409560 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56186f5edbb0 0x56186f5f13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.450+0000 7f2038999740 1 --2- 172.21.15.45:0/4169409560 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f61af50 0x56186f61d760 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.450+0000 7f2038999740 1 -- 172.21.15.45:0/4169409560 >> 172.21.15.45:0/4169409560 conn(0x56186f5efe20 msgr2=0x56186f638760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.450+0000 7f2038999740 1 -- 172.21.15.45:0/4169409560 shutdown_connections 2024-05-08T05:51:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.450+0000 7f2038999740 1 -- 172.21.15.45:0/4169409560 wait complete. 2024-05-08T05:51:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.451+0000 7f2038999740 1 -- start start 2024-05-08T05:51:36.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.451+0000 7f2038999740 1 Processor -- start 2024-05-08T05:51:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.451+0000 7f2038999740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56186f61af50 0x56186f64c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.451+0000 7f2038999740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56186f61b330 0x56186f63c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.451+0000 7f2038999740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f6106e0 0x56186f610b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.451+0000 7f2038999740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56186f5f2fd0 con 0x56186f6106e0 2024-05-08T05:51:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.451+0000 7f2038999740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x56186f514760 con 0x56186f61af50 2024-05-08T05:51:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.451+0000 7f2038999740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56186f5f32b0 con 0x56186f61b330 2024-05-08T05:51:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.452+0000 7f2038999740 10 client.0 _ll_register_callbacks cb 0x56186f54dcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:51:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.452+0000 7f202ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f6106e0 0x56186f610b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.452+0000 7f202ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f6106e0 0x56186f610b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39688/0 (socket says 172.21.15.45:39688) 2024-05-08T05:51:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.452+0000 7f202ffff640 1 -- 172.21.15.45:0/1315917616 learned_addr learned my addr 172.21.15.45:0/1315917616 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.452+0000 7f202f7fe640 1 --2- 172.21.15.45:0/1315917616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56186f61af50 0x56186f64c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.452+0000 7f202effd640 1 --2- 172.21.15.45:0/1315917616 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56186f61b330 0x56186f63c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.452+0000 7f202ffff640 1 -- 172.21.15.45:0/1315917616 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56186f61b330 msgr2=0x56186f63c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.452+0000 7f202ffff640 1 --2- 172.21.15.45:0/1315917616 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56186f61b330 0x56186f63c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.452+0000 7f202ffff640 1 -- 172.21.15.45:0/1315917616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56186f61af50 msgr2=0x56186f64c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.452+0000 7f202ffff640 1 --2- 172.21.15.45:0/1315917616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56186f61af50 0x56186f64c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.452+0000 7f202ffff640 1 -- 172.21.15.45:0/1315917616 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56186f2692d0 con 0x56186f6106e0 2024-05-08T05:51:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.452+0000 7f202ffff640 1 --2- 172.21.15.45:0/1315917616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f6106e0 0x56186f610b60 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f2028007dd0 tx=0x7f2028012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a166a2282cab1e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.452+0000 7f200e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:51:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.452+0000 7f200e7fc640 1 -- 172.21.15.45:0/1315917616 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202801c020 con 0x56186f6106e0 2024-05-08T05:51:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.452+0000 7f200e7fc640 1 -- 172.21.15.45:0/1315917616 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f2028018670 con 0x56186f6106e0 2024-05-08T05:51:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.452+0000 7f2038999740 1 -- 172.21.15.45:0/1315917616 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f202004d070 con 0x56186f6106e0 2024-05-08T05:51:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.452+0000 7f200e7fc640 1 -- 172.21.15.45:0/1315917616 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f2028026050 con 0x56186f6106e0 2024-05-08T05:51:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.452+0000 7f2038999740 10 client.8556 fetch_fsmap learned FSMap version 113 2024-05-08T05:51:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.452+0000 7f2038999740 1 -- 172.21.15.45:0/1315917616 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x56186f3336b0 con 0x56186f6106e0 2024-05-08T05:51:36.458 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.453+0000 7f200e7fc640 1 -- 172.21.15.45:0/1315917616 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f202004d070 con 0x56186f6106e0 2024-05-08T05:51:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.453+0000 7f200e7fc640 1 -- 172.21.15.45:0/1315917616 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 113) ==== 3494+0+0 (secure 0 0 0) 0x7f2028029020 con 0x56186f6106e0 2024-05-08T05:51:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.453+0000 7f2038999740 10 client.8556 fetch_fsmap finished waiting for FSMap version 113 2024-05-08T05:51:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.453+0000 7f2038999740 10 client.8556 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T05:51:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.453+0000 7f2038999740 20 client.8556 populate_metadata read hostname 'smithi045' 2024-05-08T05:51:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.453+0000 7f2038999740 1 --2- 172.21.15.45:0/1315917616 >> [v2:172.21.15.45:6834/4071581846,v1:172.21.15.45:6835/4071581846] conn(0x56186f61ec70 0x56186f61f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.453+0000 7f2038999740 4 client.8556 mds_command: new command op to 8463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:51:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.453+0000 7f2038999740 1 -- 172.21.15.45:0/1315917616 --> [v2:172.21.15.45:6834/4071581846,v1:172.21.15.45:6835/4071581846] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56186f4db760 con 0x56186f61ec70 2024-05-08T05:51:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.453+0000 7f202f7fe640 1 --2- 172.21.15.45:0/1315917616 >> [v2:172.21.15.45:6834/4071581846,v1:172.21.15.45:6835/4071581846] conn(0x56186f61ec70 0x56186f61f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.453+0000 7f200e7fc640 10 client.8556 ms_handle_connect on v2:172.21.15.45:6834/4071581846 2024-05-08T05:51:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.453+0000 7f202f7fe640 1 --2- 172.21.15.45:0/1315917616 >> [v2:172.21.15.45:6834/4071581846,v1:172.21.15.45:6835/4071581846] conn(0x56186f61ec70 0x56186f61f050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ef4dce35fd0f7ba server_cookie=fcae12061d8e5eb4 in_seq=0 out_seq=0 2024-05-08T05:51:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.454+0000 7f200e7fc640 1 -- 172.21.15.45:0/1315917616 <== mds.0 v2:172.21.15.45:6834/4071581846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56186f4db760 con 0x56186f61ec70 2024-05-08T05:51:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.454+0000 7f200e7fc640 10 client.8556 handle_command_reply: tid=0 2024-05-08T05:51:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.454+0000 7f2038999740 1 client.8556 shutdown 2024-05-08T05:51:36.459 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.454+0000 7f2038999740 1 -- 172.21.15.45:0/1315917616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f6106e0 msgr2=0x56186f610b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:36.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.454+0000 7f2038999740 1 --2- 172.21.15.45:0/1315917616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f6106e0 0x56186f610b60 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f2028007dd0 tx=0x7f2028012610 comp rx=0 tx=0).stop 2024-05-08T05:51:36.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.454+0000 7f2038999740 1 -- 172.21.15.45:0/1315917616 shutdown_connections 2024-05-08T05:51:36.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.454+0000 7f2038999740 1 --2- 172.21.15.45:0/1315917616 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56186f61b330 0x56186f63c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.454+0000 7f2038999740 1 --2- 172.21.15.45:0/1315917616 >> [v2:172.21.15.45:6834/4071581846,v1:172.21.15.45:6835/4071581846] conn(0x56186f61ec70 0x56186f61f050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.454+0000 7f2038999740 1 --2- 172.21.15.45:0/1315917616 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f6106e0 0x56186f610b60 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.454+0000 7f2038999740 1 --2- 172.21.15.45:0/1315917616 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56186f61af50 0x56186f64c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.454+0000 7f200e7fc640 0 client.8556 ms_handle_reset on v2:172.21.15.45:6834/4071581846 2024-05-08T05:51:36.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.454+0000 7f2038999740 1 -- 172.21.15.45:0/1315917616 >> 172.21.15.45:0/1315917616 conn(0x56186f5edbb0 msgr2=0x56186f5ee860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:36.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.454+0000 7f2038999740 1 -- 172.21.15.45:0/1315917616 shutdown_connections 2024-05-08T05:51:36.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.454+0000 7f2038999740 1 -- 172.21.15.45:0/1315917616 wait complete. 2024-05-08T05:51:36.460 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.454+0000 7f2038999740 20 client.8556 trim_cache size 0 max 16384 2024-05-08T05:51:36.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.474+0000 7f2038999740 1 Processor -- start 2024-05-08T05:51:36.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.474+0000 7f2038999740 1 -- start start 2024-05-08T05:51:36.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.475+0000 7f2038999740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f6106e0 0x56186f638000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.475+0000 7f2038999740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56186f61af50 0x56186f5e8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.475+0000 7f2038999740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56186f61b330 0x56186f6195b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.475+0000 7f2038999740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2030201e10 con 0x56186f6106e0 2024-05-08T05:51:36.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.475+0000 7f2038999740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2030200e50 con 0x56186f61af50 2024-05-08T05:51:36.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.475+0000 7f2038999740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2030150350 con 0x56186f61b330 2024-05-08T05:51:36.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.475+0000 7f202f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f6106e0 0x56186f638000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.475+0000 7f202f7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f6106e0 0x56186f638000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39698/0 (socket says 172.21.15.45:39698) 2024-05-08T05:51:36.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.475+0000 7f202f7fe640 1 -- 172.21.15.45:0/948701167 learned_addr learned my addr 172.21.15.45:0/948701167 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:36.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.475+0000 7f202f7fe640 1 -- 172.21.15.45:0/948701167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56186f61b330 msgr2=0x56186f6195b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:36.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.475+0000 7f202f7fe640 1 --2- 172.21.15.45:0/948701167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56186f61b330 0x56186f6195b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.475+0000 7f202f7fe640 1 -- 172.21.15.45:0/948701167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56186f61af50 msgr2=0x56186f5e8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:36.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.475+0000 7f202f7fe640 1 --2- 172.21.15.45:0/948701167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56186f61af50 0x56186f5e8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.475+0000 7f202f7fe640 1 -- 172.21.15.45:0/948701167 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56186f61ad20 con 0x56186f6106e0 2024-05-08T05:51:36.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.475+0000 7f202f7fe640 1 --2- 172.21.15.45:0/948701167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f6106e0 0x56186f638000 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f202002a6f0 tx=0x7f2020009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfe3d58e4e50664 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:36.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.475+0000 7f2014ff9640 1 -- 172.21.15.45:0/948701167 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202005f070 con 0x56186f6106e0 2024-05-08T05:51:36.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.475+0000 7f2014ff9640 1 -- 172.21.15.45:0/948701167 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f202004d040 con 0x56186f6106e0 2024-05-08T05:51:36.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.475+0000 7f2014ff9640 1 -- 172.21.15.45:0/948701167 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202003dc20 con 0x56186f6106e0 2024-05-08T05:51:36.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.476+0000 7f2038999740 1 -- 172.21.15.45:0/948701167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f6106e0 msgr2=0x56186f638000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:36.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.476+0000 7f2038999740 1 --2- 172.21.15.45:0/948701167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f6106e0 0x56186f638000 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f202002a6f0 tx=0x7f2020009190 comp rx=0 tx=0).stop 2024-05-08T05:51:36.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.476+0000 7f2038999740 1 -- 172.21.15.45:0/948701167 shutdown_connections 2024-05-08T05:51:36.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.476+0000 7f2038999740 1 --2- 172.21.15.45:0/948701167 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56186f61b330 0x56186f6195b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.476+0000 7f2038999740 1 --2- 172.21.15.45:0/948701167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56186f61af50 0x56186f5e8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.476+0000 7f2038999740 1 --2- 172.21.15.45:0/948701167 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f6106e0 0x56186f638000 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.476+0000 7f2038999740 1 -- 172.21.15.45:0/948701167 >> 172.21.15.45:0/948701167 conn(0x56186f5edbb0 msgr2=0x56186f61da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:36.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.476+0000 7f2038999740 1 -- 172.21.15.45:0/948701167 shutdown_connections 2024-05-08T05:51:36.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.476+0000 7f2038999740 1 -- 172.21.15.45:0/948701167 wait complete. 2024-05-08T05:51:36.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.477+0000 7f2038999740 1 -- start start 2024-05-08T05:51:36.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.477+0000 7f2038999740 1 Processor -- start 2024-05-08T05:51:36.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f2038999740 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56186f6106e0 0x56186f619780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f2038999740 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56186f61af50 0x56186f64a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f2038999740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f61b330 0x56186f64ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f2038999740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56186f64c860 con 0x56186f61b330 2024-05-08T05:51:36.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f2038999740 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2030200920 con 0x56186f6106e0 2024-05-08T05:51:36.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f2038999740 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x56186f63ae40 con 0x56186f61af50 2024-05-08T05:51:36.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f2038999740 10 client.0 _ll_register_callbacks cb 0x7f2028009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:51:36.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f202ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f61b330 0x56186f64ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f202ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f61b330 0x56186f64ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39706/0 (socket says 172.21.15.45:39706) 2024-05-08T05:51:36.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f202ffff640 1 -- 172.21.15.45:0/2962162751 learned_addr learned my addr 172.21.15.45:0/2962162751 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:36.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f202effd640 1 --2- 172.21.15.45:0/2962162751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56186f61af50 0x56186f64a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f202f7fe640 1 --2- 172.21.15.45:0/2962162751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56186f6106e0 0x56186f619780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f202ffff640 1 -- 172.21.15.45:0/2962162751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56186f61af50 msgr2=0x56186f64a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:36.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f202ffff640 1 --2- 172.21.15.45:0/2962162751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56186f61af50 0x56186f64a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f202ffff640 1 -- 172.21.15.45:0/2962162751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56186f6106e0 msgr2=0x56186f619780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:36.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f202ffff640 1 --2- 172.21.15.45:0/2962162751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56186f6106e0 0x56186f619780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f202ffff640 1 -- 172.21.15.45:0/2962162751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56186f5ed780 con 0x56186f61b330 2024-05-08T05:51:36.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f202ffff640 1 --2- 172.21.15.45:0/2962162751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f61b330 0x56186f64ac20 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f2028002230 tx=0x7f2028004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6d062cf5b0343d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:36.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f200e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-05-08T05:51:36.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f200e7fc640 1 -- 172.21.15.45:0/2962162751 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202801c040 con 0x56186f61b330 2024-05-08T05:51:36.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f200e7fc640 1 -- 172.21.15.45:0/2962162751 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f2028018a80 con 0x56186f61b330 2024-05-08T05:51:36.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.478+0000 7f2038999740 1 -- 172.21.15.45:0/2962162751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f202004d040 con 0x56186f61b330 2024-05-08T05:51:36.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.479+0000 7f200e7fc640 1 -- 172.21.15.45:0/2962162751 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f2028037050 con 0x56186f61b330 2024-05-08T05:51:36.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.479+0000 7f2038999740 10 client.8562 fetch_fsmap learned FSMap version 113 2024-05-08T05:51:36.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.479+0000 7f2038999740 1 -- 172.21.15.45:0/2962162751 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x56186f64b160 con 0x56186f61b330 2024-05-08T05:51:36.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.479+0000 7f200e7fc640 1 -- 172.21.15.45:0/2962162751 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f202004d040 con 0x56186f61b330 2024-05-08T05:51:36.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.479+0000 7f200e7fc640 1 -- 172.21.15.45:0/2962162751 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 113) ==== 3494+0+0 (secure 0 0 0) 0x7f202803a020 con 0x56186f61b330 2024-05-08T05:51:36.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.479+0000 7f2038999740 10 client.8562 fetch_fsmap finished waiting for FSMap version 113 2024-05-08T05:51:36.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.479+0000 7f2038999740 10 client.8562 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T05:51:36.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.479+0000 7f2038999740 20 client.8562 populate_metadata read hostname 'smithi045' 2024-05-08T05:51:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.479+0000 7f2038999740 1 --2- 172.21.15.45:0/2962162751 >> [v2:172.21.15.45:6834/4071581846,v1:172.21.15.45:6835/4071581846] conn(0x56186f62e860 0x56186f6b9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.479+0000 7f2038999740 4 client.8562 mds_command: new command op to 8463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:51:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.479+0000 7f2038999740 1 -- 172.21.15.45:0/2962162751 --> [v2:172.21.15.45:6834/4071581846,v1:172.21.15.45:6835/4071581846] -- command(tid 0: {"prefix": "status"}) -- 0x56186f5654e0 con 0x56186f62e860 2024-05-08T05:51:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.479+0000 7f202f7fe640 1 --2- 172.21.15.45:0/2962162751 >> [v2:172.21.15.45:6834/4071581846,v1:172.21.15.45:6835/4071581846] conn(0x56186f62e860 0x56186f6b9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.479+0000 7f200e7fc640 10 client.8562 ms_handle_connect on v2:172.21.15.45:6834/4071581846 2024-05-08T05:51:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.479+0000 7f202f7fe640 1 --2- 172.21.15.45:0/2962162751 >> [v2:172.21.15.45:6834/4071581846,v1:172.21.15.45:6835/4071581846] conn(0x56186f62e860 0x56186f6b9b20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eafda49bb8a0d0e server_cookie=92e7fc9f2d59976 in_seq=0 out_seq=0 2024-05-08T05:51:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.480+0000 7f200e7fc640 1 -- 172.21.15.45:0/2962162751 <== mds.0 v2:172.21.15.45:6834/4071581846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56186f5654e0 con 0x56186f62e860 2024-05-08T05:51:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.480+0000 7f200e7fc640 10 client.8562 handle_command_reply: tid=0 2024-05-08T05:51:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.480+0000 7f2038999740 1 client.8562 shutdown 2024-05-08T05:51:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.480+0000 7f2038999740 1 -- 172.21.15.45:0/2962162751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f61b330 msgr2=0x56186f64ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.480+0000 7f2038999740 1 --2- 172.21.15.45:0/2962162751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f61b330 0x56186f64ac20 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f2028002230 tx=0x7f2028004ab0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.480+0000 7f2038999740 1 -- 172.21.15.45:0/2962162751 shutdown_connections 2024-05-08T05:51:36.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.480+0000 7f2038999740 1 --2- 172.21.15.45:0/2962162751 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x56186f61af50 0x56186f64a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.480+0000 7f2038999740 1 --2- 172.21.15.45:0/2962162751 >> [v2:172.21.15.45:6834/4071581846,v1:172.21.15.45:6835/4071581846] conn(0x56186f62e860 0x56186f6b9b20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.480+0000 7f2038999740 1 --2- 172.21.15.45:0/2962162751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56186f61b330 0x56186f64ac20 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.480+0000 7f2038999740 1 --2- 172.21.15.45:0/2962162751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x56186f6106e0 0x56186f619780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.480+0000 7f200e7fc640 0 client.8562 ms_handle_reset on v2:172.21.15.45:6834/4071581846 2024-05-08T05:51:36.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.480+0000 7f2038999740 1 -- 172.21.15.45:0/2962162751 >> 172.21.15.45:0/2962162751 conn(0x56186f5edbb0 msgr2=0x56186f63ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:36.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.480+0000 7f2038999740 1 -- 172.21.15.45:0/2962162751 shutdown_connections 2024-05-08T05:51:36.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.480+0000 7f2038999740 1 -- 172.21.15.45:0/2962162751 wait complete. 2024-05-08T05:51:36.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.480+0000 7f2038999740 20 client.8562 trim_cache size 0 max 16384 2024-05-08T05:51:36.486 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:51:36.486 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "e9e1170a-fd27-4625-97a9-5d5f104d13a2", 2024-05-08T05:51:36.487 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-05-08T05:51:36.487 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 8463, 2024-05-08T05:51:36.487 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-05-08T05:51:36.487 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-05-08T05:51:36.487 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-05-08T05:51:36.487 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 4.502401603, 2024-05-08T05:51:36.487 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 113, 2024-05-08T05:51:36.487 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 121, 2024-05-08T05:51:36.487 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 121, 2024-05-08T05:51:36.487 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 6.4828120949999999 2024-05-08T05:51:36.487 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:51:36.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.480+0000 7f2035ba4640 1 -- 172.21.15.45:0/376924870 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f201c0762d0 msgr2=0x7f201c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:36.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.480+0000 7f2035ba4640 1 --2- 172.21.15.45:0/376924870 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f201c0762d0 0x7f201c078790 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2020034000 tx=0x7f202002e000 comp rx=0 tx=0).stop 2024-05-08T05:51:36.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.480+0000 7f2035ba4640 1 -- 172.21.15.45:0/376924870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20300b5f60 msgr2=0x7f20301587c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:36.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.480+0000 7f2035ba4640 1 --2- 172.21.15.45:0/376924870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20300b5f60 0x7f20301587c0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f202400c480 tx=0x7f202400c940 comp rx=0 tx=0).stop 2024-05-08T05:51:36.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.481+0000 7f2035ba4640 1 -- 172.21.15.45:0/376924870 shutdown_connections 2024-05-08T05:51:36.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.481+0000 7f2035ba4640 1 --2- 172.21.15.45:0/376924870 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f201c0762d0 0x7f201c078790 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.481+0000 7f2035ba4640 1 --2- 172.21.15.45:0/376924870 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f203015f430 0x7f20300b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.481+0000 7f2035ba4640 1 --2- 172.21.15.45:0/376924870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20300b5f60 0x7f20301587c0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.481+0000 7f2035ba4640 1 --2- 172.21.15.45:0/376924870 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f203006e6f0 0x7f203015abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.481+0000 7f2035ba4640 1 -- 172.21.15.45:0/376924870 >> 172.21.15.45:0/376924870 conn(0x7f20300b2cc0 msgr2=0x7f203015bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:36.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.481+0000 7f2035ba4640 1 -- 172.21.15.45:0/376924870 shutdown_connections 2024-05-08T05:51:36.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.481+0000 7f2035ba4640 1 -- 172.21.15.45:0/376924870 wait complete. 2024-05-08T05:51:36.492 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-08T05:51:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.595+0000 7f01124a7640 1 Processor -- start 2024-05-08T05:51:36.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.595+0000 7f01124a7640 1 -- start start 2024-05-08T05:51:36.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.596+0000 7f01124a7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c06f060 0x7f010c06e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.596+0000 7f01124a7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f010c0d0530 0x7f010c0b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.596+0000 7f01124a7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f010c15d2a0 con 0x7f010c06fe30 2024-05-08T05:51:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.596+0000 7f01124a7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f010c06f440 con 0x7f010c0d0530 2024-05-08T05:51:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.596+0000 7f01124a7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f010c1609b0 con 0x7f010c06f060 2024-05-08T05:51:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.596+0000 7f0110a44640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f010c0d0530 0x7f010c0b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.596+0000 7f010bfff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c06f060 0x7f010c06e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.596+0000 7f010bfff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c06f060 0x7f010c06e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:52080/0 (socket says 172.21.15.45:52080) 2024-05-08T05:51:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.596+0000 7f0110a44640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f010c0d0530 0x7f010c0b3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:60420/0 (socket says 172.21.15.45:60420) 2024-05-08T05:51:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.596+0000 7f010bfff640 1 -- 172.21.15.45:0/643917372 learned_addr learned my addr 172.21.15.45:0/643917372 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.596+0000 7f010b7fe640 1 --1- 172.21.15.45:0/643917372 >> v1:172.21.15.63:6789/0 conn(0x7f010c06fe30 0x7f010c0cfe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:33500/0 (socket says 172.21.15.45:33500) 2024-05-08T05:51:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.596+0000 7f0110a44640 1 -- 172.21.15.45:0/643917372 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c06f060 msgr2=0x7f010c06e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.596+0000 7f0110a44640 1 --2- 172.21.15.45:0/643917372 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c06f060 0x7f010c06e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.596+0000 7f0110a44640 1 -- 172.21.15.45:0/643917372 >> v1:172.21.15.63:6789/0 conn(0x7f010c06fe30 legacy=0x7f010c0cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.596+0000 7f0110a44640 1 -- 172.21.15.45:0/643917372 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010c160c90 con 0x7f010c0d0530 2024-05-08T05:51:36.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.596+0000 7f010bfff640 1 --2- 172.21.15.45:0/643917372 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c06f060 0x7f010c06e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.597+0000 7f0110a44640 1 --2- 172.21.15.45:0/643917372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f010c0d0530 0x7f010c0b3f00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0104009d90 tx=0x7f010402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d9b8d844862263c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.597+0000 7f010affd640 1 -- 172.21.15.45:0/643917372 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104007740 con 0x7f010c0d0530 2024-05-08T05:51:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.597+0000 7f010affd640 1 -- 172.21.15.45:0/643917372 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01040078e0 con 0x7f010c0d0530 2024-05-08T05:51:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.598+0000 7f010affd640 1 -- 172.21.15.45:0/643917372 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104007cb0 con 0x7f010c0d0530 2024-05-08T05:51:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.598+0000 7f01124a7640 1 -- 172.21.15.45:0/643917372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f010c0d0530 msgr2=0x7f010c0b3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.598+0000 7f01124a7640 1 --2- 172.21.15.45:0/643917372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f010c0d0530 0x7f010c0b3f00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0104009d90 tx=0x7f010402a660 comp rx=0 tx=0).stop 2024-05-08T05:51:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.598+0000 7f01124a7640 1 -- 172.21.15.45:0/643917372 shutdown_connections 2024-05-08T05:51:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.598+0000 7f01124a7640 1 --2- 172.21.15.45:0/643917372 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f010c0d0530 0x7f010c0b3f00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.598+0000 7f01124a7640 1 --2- 172.21.15.45:0/643917372 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c06f060 0x7f010c06e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.598+0000 7f01124a7640 1 -- 172.21.15.45:0/643917372 >> 172.21.15.45:0/643917372 conn(0x7f010c06c3a0 msgr2=0x7f010c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.598+0000 7f01124a7640 1 -- 172.21.15.45:0/643917372 shutdown_connections 2024-05-08T05:51:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.598+0000 7f01124a7640 1 -- 172.21.15.45:0/643917372 wait complete. 2024-05-08T05:51:36.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.599+0000 7f01124a7640 1 Processor -- start 2024-05-08T05:51:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.600+0000 7f01124a7640 1 -- start start 2024-05-08T05:51:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.600+0000 7f01124a7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f010c06f060 0x7f010c0b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.600+0000 7f01124a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c06fe30 0x7f010c15a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.600+0000 7f01124a7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c0d0530 0x7f010c15d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.600+0000 7f01124a7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f010c161530 con 0x7f010c06fe30 2024-05-08T05:51:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.600+0000 7f01124a7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f010c161250 con 0x7f010c06f060 2024-05-08T05:51:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.600+0000 7f01124a7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f010c161810 con 0x7f010c0d0530 2024-05-08T05:51:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.600+0000 7f010b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c06fe30 0x7f010c15a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.600+0000 7f010b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c06fe30 0x7f010c15a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39722/0 (socket says 172.21.15.45:39722) 2024-05-08T05:51:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.600+0000 7f010b7fe640 1 -- 172.21.15.45:0/2250137260 learned_addr learned my addr 172.21.15.45:0/2250137260 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.601+0000 7f010bfff640 1 --2- 172.21.15.45:0/2250137260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f010c06f060 0x7f010c0b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.601+0000 7f0110a44640 1 --2- 172.21.15.45:0/2250137260 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c0d0530 0x7f010c15d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.601+0000 7f010b7fe640 1 -- 172.21.15.45:0/2250137260 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c0d0530 msgr2=0x7f010c15d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.601+0000 7f010b7fe640 1 --2- 172.21.15.45:0/2250137260 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c0d0530 0x7f010c15d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.601+0000 7f010b7fe640 1 -- 172.21.15.45:0/2250137260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f010c06f060 msgr2=0x7f010c0b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.601+0000 7f010b7fe640 1 --2- 172.21.15.45:0/2250137260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f010c06f060 0x7f010c0b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.601+0000 7f010b7fe640 1 -- 172.21.15.45:0/2250137260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010c0b0fc0 con 0x7f010c06fe30 2024-05-08T05:51:36.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.601+0000 7f0110a44640 1 --2- 172.21.15.45:0/2250137260 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c0d0530 0x7f010c15d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.601+0000 7f010b7fe640 1 --2- 172.21.15.45:0/2250137260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c06fe30 0x7f010c15a650 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f010000c500 tx=0x7f010000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.601+0000 7f0108ff9640 1 -- 172.21.15.45:0/2250137260 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100018070 con 0x7f010c06fe30 2024-05-08T05:51:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.601+0000 7f0108ff9640 1 -- 172.21.15.45:0/2250137260 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0100007500 con 0x7f010c06fe30 2024-05-08T05:51:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.601+0000 7f0108ff9640 1 -- 172.21.15.45:0/2250137260 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01000049e0 con 0x7f010c06fe30 2024-05-08T05:51:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.601+0000 7f01124a7640 1 -- 172.21.15.45:0/2250137260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010c0b12a0 con 0x7f010c06fe30 2024-05-08T05:51:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.601+0000 7f01124a7640 1 -- 172.21.15.45:0/2250137260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f010c0b1b60 con 0x7f010c06fe30 2024-05-08T05:51:36.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.602+0000 7f01124a7640 1 -- 172.21.15.45:0/2250137260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f010c0b1980 con 0x7f010c06fe30 2024-05-08T05:51:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.603+0000 7f0108ff9640 1 -- 172.21.15.45:0/2250137260 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f010001b070 con 0x7f010c06fe30 2024-05-08T05:51:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.603+0000 7f0108ff9640 1 --2- 172.21.15.45:0/2250137260 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f00f80762d0 0x7f00f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.603+0000 7f0108ff9640 1 -- 172.21.15.45:0/2250137260 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f0100096b00 con 0x7f010c06fe30 2024-05-08T05:51:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.604+0000 7f010bfff640 1 --2- 172.21.15.45:0/2250137260 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f00f80762d0 0x7f00f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:36.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.604+0000 7f010bfff640 1 --2- 172.21.15.45:0/2250137260 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f00f80762d0 0x7f00f8078790 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f00fc006ca0 tx=0x7f00fc009240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:36.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.608+0000 7f0108ff9640 1 -- 172.21.15.45:0/2250137260 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f010c0b1980 con 0x7f010c06fe30 2024-05-08T05:51:36.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.840+0000 7f01124a7640 1 -- 172.21.15.45:0/2250137260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f010c0b1d60 con 0x7f010c06fe30 2024-05-08T05:51:36.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.840+0000 7f0108ff9640 1 -- 172.21.15.45:0/2250137260 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v113) ==== 53+0+73 (secure 0 0 0) 0x7f010c0b1d60 con 0x7f010c06fe30 2024-05-08T05:51:36.840 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-08T05:51:36.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.842+0000 7f01124a7640 1 -- 172.21.15.45:0/2250137260 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f00f80762d0 msgr2=0x7f00f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:36.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.842+0000 7f01124a7640 1 --2- 172.21.15.45:0/2250137260 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f00f80762d0 0x7f00f8078790 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f00fc006ca0 tx=0x7f00fc009240 comp rx=0 tx=0).stop 2024-05-08T05:51:36.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.842+0000 7f01124a7640 1 -- 172.21.15.45:0/2250137260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c06fe30 msgr2=0x7f010c15a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:36.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.842+0000 7f01124a7640 1 --2- 172.21.15.45:0/2250137260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c06fe30 0x7f010c15a650 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f010000c500 tx=0x7f010000c9c0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.842+0000 7f01124a7640 1 -- 172.21.15.45:0/2250137260 shutdown_connections 2024-05-08T05:51:36.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.842+0000 7f01124a7640 1 --2- 172.21.15.45:0/2250137260 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f00f80762d0 0x7f00f8078790 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.842+0000 7f01124a7640 1 --2- 172.21.15.45:0/2250137260 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f010c0d0530 0x7f010c15d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.842+0000 7f01124a7640 1 --2- 172.21.15.45:0/2250137260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f010c06fe30 0x7f010c15a650 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.842+0000 7f01124a7640 1 --2- 172.21.15.45:0/2250137260 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f010c06f060 0x7f010c0b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:36.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.842+0000 7f01124a7640 1 -- 172.21.15.45:0/2250137260 >> 172.21.15.45:0/2250137260 conn(0x7f010c06c3a0 msgr2=0x7f010c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:36.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.842+0000 7f01124a7640 1 -- 172.21.15.45:0/2250137260 shutdown_connections 2024-05-08T05:51:36.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:36.842+0000 7f01124a7640 1 -- 172.21.15.45:0/2250137260 wait complete. 2024-05-08T05:51:36.853 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T05:51:36.853 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:51:36.853 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-05-08T05:51:36.853 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T05:51:36.853 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:51:36.853 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T05:51:36.853 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-05-08T05:51:36.853 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:51:36.853 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:51:36.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:36.854 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-05-08T05:51:36.870 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:51:36.870 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:51:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:51:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-05-08T05:51:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:36.871 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:51:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:51:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-05-08T05:51:36.871 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:51:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-05-08T05:51:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-05-08T05:51:36.871 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:51:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-05-08T05:51:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-05-08T05:51:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:51:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 42128sec preferred_lft 42128sec 2024-05-08T05:51:36.871 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-05-08T05:51:36.872 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:36.872 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:51:36.872 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-05-08T05:51:36.872 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-05-08T05:51:36.872 INFO:teuthology.orchestra.run.smithi045.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-08T05:51:36.872 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff 2024-05-08T05:51:36.872 INFO:teuthology.orchestra.run.smithi045.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-08T05:51:36.872 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:36.872 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::c80a:aff:fe94:c77/64 scope link 2024-05-08T05:51:36.872 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:36.872 INFO:teuthology.orchestra.run.smithi045.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:51:36.872 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 82:f3:eb:6a:57:ba brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:51:36.872 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::80f3:ebff:fe6a:57ba/64 scope link 2024-05-08T05:51:36.872 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:36.872 INFO:teuthology.orchestra.run.smithi045.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-08T05:51:36.872 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 16:a2:64:da:3a:9c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T05:51:36.873 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::14a2:64ff:feda:3a9c/64 scope link 2024-05-08T05:51:36.873 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:51:36.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:36.873 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:51:36.935 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:51:36.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:36.937 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-05-08T05:51:36.994 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-05-08T05:51:36.994 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:51:36.994 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:51:36.994 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:51:36.995 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:51:36.995 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:51:36.995 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-05-08T05:51:36.995 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:51:36.995 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:51:36.995 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:51:36.995 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-05-08T05:51:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:51:37.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:37.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:37.127 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:51:37.142 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T05:51:37.143 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T05:51:37.143 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:51:37.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:37.143 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-05-08T05:51:37.209 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:51:37.209 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T05:51:37.265 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:51:37.265 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:51:37.265 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:51:37.320 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-05-08T05:51:37.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:37.321 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:51:37.406 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:51:37.406 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:51:37.413 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-08T05:51:37.413 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-08T05:51:37.414 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.63:3300,172.21.15.63:3301" 2024-05-08T05:51:37.414 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@e9e1170a-fd27-4625-97a9-5d5f104d13a2.cephfs=/ 2024-05-08T05:51:37.414 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.45:3300/172.21.15.63:3300/172.21.15.63:3301" will pass to kernel 2024-05-08T05:51:37.414 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T05:51:37.414 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:51:37.414 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:51:37.414 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:51:37.414 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:51:37.416 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:51:37.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:37.416 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:51:37.442 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:51:37.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:51:37.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:37.540 INFO:teuthology.orchestra.run.smithi045.stdout:client8568 2024-05-08T05:51:37.540 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:51:37.540 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:51:37.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:37.556 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-05-08T05:51:37.616 INFO:teuthology.orchestra.run.smithi045.stdout:./ 2024-05-08T05:51:37.616 INFO:teuthology.orchestra.run.smithi045.stdout:./parent 2024-05-08T05:51:37.616 INFO:teuthology.orchestra.run.smithi045.stdout:./parent/flushed 2024-05-08T05:51:37.616 INFO:teuthology.orchestra.run.smithi045.stdout:./parent/flushed/charlie 2024-05-08T05:51:37.617 INFO:teuthology.orchestra.run.smithi045.stdout:./parent/flushed/alpha 2024-05-08T05:51:37.617 INFO:teuthology.orchestra.run.smithi045.stdout:./parent/flushed/bravo 2024-05-08T05:51:37.617 INFO:teuthology.orchestra.run.smithi045.stdout:./parent/unflushed 2024-05-08T05:51:37.617 INFO:teuthology.orchestra.run.smithi045.stdout:./parent/unflushed/jfile 2024-05-08T05:51:37.617 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:37.617 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:37.617 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:37.617 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:37.617 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-05-08T05:51:37.617 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:37.725 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-05-08T05:51:37.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:37.734 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:37.734 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:37.734 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:37.734 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:37.734 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-05-08T05:51:37.734 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:37.841 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627776 2024-05-08T05:51:37.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:37.850 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:37.850 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:37.850 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:37.850 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:37.850 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-05-08T05:51:37.850 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:37.957 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627777 2024-05-08T05:51:37.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:37.966 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:37.966 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:37.966 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:37.967 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:37.967 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-05-08T05:51:37.967 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:38.074 INFO:teuthology.orchestra.run.smithi045.stdout:1099511628280 2024-05-08T05:51:38.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:38.083 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:38.083 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:38.083 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:38.083 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:38.084 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-05-08T05:51:38.084 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:38.191 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627778 2024-05-08T05:51:38.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:38.200 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:38.201 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:38.201 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:38.201 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:38.201 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-05-08T05:51:38.201 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:38.309 INFO:teuthology.orchestra.run.smithi045.stdout:1099511628279 2024-05-08T05:51:38.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:38.318 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:38.318 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:38.319 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:38.319 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:38.319 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-05-08T05:51:38.319 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:38.426 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627779 2024-05-08T05:51:38.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:38.436 DEBUG:teuthology.orchestra.run.smithi045:> stdin-killer -- python3 -c ' 2024-05-08T05:51:38.436 DEBUG:teuthology.orchestra.run.smithi045:> import os 2024-05-08T05:51:38.436 DEBUG:teuthology.orchestra.run.smithi045:> import stat 2024-05-08T05:51:38.436 DEBUG:teuthology.orchestra.run.smithi045:> 2024-05-08T05:51:38.436 DEBUG:teuthology.orchestra.run.smithi045:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-05-08T05:51:38.436 DEBUG:teuthology.orchestra.run.smithi045:> ' 2024-05-08T05:51:38.544 INFO:teuthology.orchestra.run.smithi045.stdout:1099511627780 2024-05-08T05:51:38.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:51:38.553 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-08T05:51:38.553 DEBUG:teuthology.orchestra.run.smithi045:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:51:38.636 DEBUG:teuthology.orchestra.run.smithi063:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-08T05:51:38.691 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-08T05:51:38.691 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:51:38.692 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:51:38.692 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:51:38.708 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:51:38.709 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:51:38.765 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T05:51:38.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.909+0000 7f0336faf640 1 Processor -- start 2024-05-08T05:51:38.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.909+0000 7f0336faf640 1 -- start start 2024-05-08T05:51:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.910+0000 7f0336faf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03300c9ee0 0x7f03300d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.910+0000 7f0336faf640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03300d3330 con 0x7f03301629a0 2024-05-08T05:51:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.910+0000 7f0336faf640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03300d2170 con 0x7f03300d3cd0 2024-05-08T05:51:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.910+0000 7f0336faf640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f03300d2450 con 0x7f03300c9ee0 2024-05-08T05:51:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.910+0000 7f0334d4b640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f03301629a0 0x7f03300d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46108/0 (socket says 172.21.15.45:46108) 2024-05-08T05:51:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.910+0000 7f0334d4b640 1 -- 172.21.15.45:0/3697014471 learned_addr learned my addr 172.21.15.45:0/3697014471 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.910+0000 7f032f7fe640 1 -- 172.21.15.45:0/3697014471 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 102598330 0 0) 0x7f03300d3330 con 0x7f03301629a0 2024-05-08T05:51:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f032f7fe640 1 -- 172.21.15.45:0/3697014471 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0318003520 con 0x7f03301629a0 2024-05-08T05:51:38.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f033554c640 1 --2- 172.21.15.45:0/3697014471 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03300c9ee0 0x7f03300d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f032f7fe640 1 -- 172.21.15.45:0/3697014471 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1375110858 0 0) 0x7f0318003520 con 0x7f03301629a0 2024-05-08T05:51:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f032f7fe640 1 -- 172.21.15.45:0/3697014471 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03300c9ee0 msgr2=0x7f03300d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f032f7fe640 1 --2- 172.21.15.45:0/3697014471 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03300c9ee0 0x7f03300d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f032f7fe640 1 -- 172.21.15.45:0/3697014471 >> v1:172.21.15.63:6789/0 conn(0x7f03300d3cd0 legacy=0x7f03300cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f032f7fe640 1 -- 172.21.15.45:0/3697014471 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03300d2730 con 0x7f03301629a0 2024-05-08T05:51:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f032f7fe640 1 -- 172.21.15.45:0/3697014471 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f03200016f0 con 0x7f03301629a0 2024-05-08T05:51:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f032f7fe640 1 -- 172.21.15.45:0/3697014471 >> v1:172.21.15.45:6789/0 conn(0x7f03301629a0 legacy=0x7f03300d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f032f7fe640 1 --2- 172.21.15.45:0/3697014471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0318003a30 0x7f0318003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f032f7fe640 1 --2- 172.21.15.45:0/3697014471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0318004800 0x7f0318004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f032f7fe640 1 --2- 172.21.15.45:0/3697014471 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03300c9ee0 0x7f031800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f032f7fe640 1 -- 172.21.15.45:0/3697014471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f03300d2450 con 0x7f0318004800 2024-05-08T05:51:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f032f7fe640 1 -- 172.21.15.45:0/3697014471 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f03180055e0 con 0x7f0318003a30 2024-05-08T05:51:38.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f032f7fe640 1 -- 172.21.15.45:0/3697014471 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f031800bf00 con 0x7f03300c9ee0 2024-05-08T05:51:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f032ffff640 1 --2- 172.21.15.45:0/3697014471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0318004800 0x7f0318004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f0334d4b640 1 --2- 172.21.15.45:0/3697014471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0318003a30 0x7f0318003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f033554c640 1 --2- 172.21.15.45:0/3697014471 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03300c9ee0 0x7f031800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f032ffff640 1 -- 172.21.15.45:0/3697014471 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03300c9ee0 msgr2=0x7f031800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f032ffff640 1 --2- 172.21.15.45:0/3697014471 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03300c9ee0 0x7f031800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f032ffff640 1 -- 172.21.15.45:0/3697014471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0318003a30 msgr2=0x7f0318003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f032ffff640 1 --2- 172.21.15.45:0/3697014471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0318003a30 0x7f0318003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f032ffff640 1 -- 172.21.15.45:0/3697014471 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0318003520 con 0x7f0318004800 2024-05-08T05:51:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f033554c640 1 --2- 172.21.15.45:0/3697014471 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f03300c9ee0 0x7f031800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.911+0000 7f0334d4b640 1 --2- 172.21.15.45:0/3697014471 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0318003a30 0x7f0318003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.912+0000 7f032ffff640 1 --2- 172.21.15.45:0/3697014471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0318004800 0x7f0318004c80 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f0324001790 tx=0x7f0318005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5ca82ad2d98bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.912+0000 7f032f7fe640 1 -- 172.21.15.45:0/3697014471 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03240013b0 con 0x7f0318004800 2024-05-08T05:51:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.912+0000 7f032f7fe640 1 -- 172.21.15.45:0/3697014471 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03240052d0 con 0x7f0318004800 2024-05-08T05:51:38.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.912+0000 7f032f7fe640 1 -- 172.21.15.45:0/3697014471 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032402aae0 con 0x7f0318004800 2024-05-08T05:51:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.913+0000 7f0336faf640 1 -- 172.21.15.45:0/3697014471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0318004800 msgr2=0x7f0318004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.913+0000 7f0336faf640 1 --2- 172.21.15.45:0/3697014471 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0318004800 0x7f0318004c80 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f0324001790 tx=0x7f0318005830 comp rx=0 tx=0).stop 2024-05-08T05:51:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.913+0000 7f033554c640 1 -- 172.21.15.45:0/3697014471 reap_dead start 2024-05-08T05:51:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.913+0000 7f0336faf640 1 -- 172.21.15.45:0/3697014471 shutdown_connections 2024-05-08T05:51:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.913+0000 7f0336faf640 1 -- 172.21.15.45:0/3697014471 >> 172.21.15.45:0/3697014471 conn(0x7f03300d4fe0 msgr2=0x7f03300d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.913+0000 7f0336faf640 1 -- 172.21.15.45:0/3697014471 shutdown_connections 2024-05-08T05:51:38.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.913+0000 7f0336faf640 1 -- 172.21.15.45:0/3697014471 wait complete. 2024-05-08T05:51:38.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.914+0000 7f0336faf640 1 Processor -- start 2024-05-08T05:51:38.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.914+0000 7f0336faf640 1 -- start start 2024-05-08T05:51:38.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.915+0000 7f0336faf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0318004800 0x7f03300ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:38.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.915+0000 7f0336faf640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03300cd920 0x7f03300b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:38.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.915+0000 7f0336faf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03300cdda0 0x7f03300b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:38.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.915+0000 7f0336faf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f03300cf440 con 0x7f03300cdda0 2024-05-08T05:51:38.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.915+0000 7f0336faf640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f03300cf2c0 con 0x7f03300cd920 2024-05-08T05:51:38.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.915+0000 7f0336faf640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f03300d2dc0 con 0x7f0318004800 2024-05-08T05:51:38.919 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.915+0000 7f033554c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03300cdda0 0x7f03300b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.915+0000 7f033554c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03300cdda0 0x7f03300b7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39736/0 (socket says 172.21.15.45:39736) 2024-05-08T05:51:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.915+0000 7f033554c640 1 -- 172.21.15.45:0/2642302687 learned_addr learned my addr 172.21.15.45:0/2642302687 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.915+0000 7f032ffff640 1 --2- 172.21.15.45:0/2642302687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03300cd920 0x7f03300b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.915+0000 7f0334d4b640 1 --2- 172.21.15.45:0/2642302687 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0318004800 0x7f03300ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.915+0000 7f033554c640 1 -- 172.21.15.45:0/2642302687 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0318004800 msgr2=0x7f03300ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.915+0000 7f033554c640 1 --2- 172.21.15.45:0/2642302687 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0318004800 0x7f03300ce8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.915+0000 7f033554c640 1 -- 172.21.15.45:0/2642302687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03300cd920 msgr2=0x7f03300b6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.915+0000 7f033554c640 1 --2- 172.21.15.45:0/2642302687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03300cd920 0x7f03300b6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.915+0000 7f033554c640 1 -- 172.21.15.45:0/2642302687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0330155d30 con 0x7f03300cdda0 2024-05-08T05:51:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.915+0000 7f032ffff640 1 --2- 172.21.15.45:0/2642302687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03300cd920 0x7f03300b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.916+0000 7f033554c640 1 --2- 172.21.15.45:0/2642302687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03300cdda0 0x7f03300b7420 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f03300b7b70 tx=0x7f0328004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.916+0000 7f032d7fa640 1 -- 172.21.15.45:0/2642302687 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328007c50 con 0x7f03300cdda0 2024-05-08T05:51:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.916+0000 7f032d7fa640 1 -- 172.21.15.45:0/2642302687 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0328019070 con 0x7f03300cdda0 2024-05-08T05:51:38.920 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.916+0000 7f0336faf640 1 -- 172.21.15.45:0/2642302687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03300cc0b0 con 0x7f03300cdda0 2024-05-08T05:51:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.916+0000 7f032d7fa640 1 -- 172.21.15.45:0/2642302687 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328014650 con 0x7f03300cdda0 2024-05-08T05:51:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.916+0000 7f0336faf640 1 -- 172.21.15.45:0/2642302687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03300cc670 con 0x7f03300cdda0 2024-05-08T05:51:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.917+0000 7f0336faf640 1 -- 172.21.15.45:0/2642302687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03300693f0 con 0x7f03300cdda0 2024-05-08T05:51:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.918+0000 7f032d7fa640 1 -- 172.21.15.45:0/2642302687 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f03280147f0 con 0x7f03300cdda0 2024-05-08T05:51:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.918+0000 7f032d7fa640 1 --2- 172.21.15.45:0/2642302687 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f031c076300 0x7f031c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.918+0000 7f032d7fa640 1 -- 172.21.15.45:0/2642302687 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f0328097df0 con 0x7f03300cdda0 2024-05-08T05:51:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.918+0000 7f0334d4b640 1 --2- 172.21.15.45:0/2642302687 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f031c076300 0x7f031c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:38.921 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.919+0000 7f0334d4b640 1 --2- 172.21.15.45:0/2642302687 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f031c076300 0x7f031c0787c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0320000fb0 tx=0x7f0320009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:38.922 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:38.922+0000 7f032d7fa640 1 -- 172.21.15.45:0/2642302687 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0328061510 con 0x7f03300cdda0 2024-05-08T05:51:39.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.126+0000 7f0336faf640 1 -- 172.21.15.45:0/2642302687 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f03300d3ab0 con 0x7f03300cdda0 2024-05-08T05:51:39.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.126+0000 7f032d7fa640 1 -- 172.21.15.45:0/2642302687 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v121) ==== 80+0+242 (secure 0 0 0) 0x7f0328060e10 con 0x7f03300cdda0 2024-05-08T05:51:39.127 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6835/453198328 2024-05-09T05:51:29.962310+0000 2024-05-08T05:51:39.127 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6834/453198328 2024-05-09T05:51:29.962310+0000 2024-05-08T05:51:39.127 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6835/1751419054 2024-05-09T05:51:20.651495+0000 2024-05-08T05:51:39.127 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6834/1751419054 2024-05-09T05:51:20.651495+0000 2024-05-08T05:51:39.127 INFO:teuthology.orchestra.run.smithi045.stderr:listed 4 entries 2024-05-08T05:51:39.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.128+0000 7f0336faf640 1 -- 172.21.15.45:0/2642302687 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f031c076300 msgr2=0x7f031c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:39.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.128+0000 7f0336faf640 1 --2- 172.21.15.45:0/2642302687 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f031c076300 0x7f031c0787c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0320000fb0 tx=0x7f0320009040 comp rx=0 tx=0).stop 2024-05-08T05:51:39.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.128+0000 7f0336faf640 1 -- 172.21.15.45:0/2642302687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03300cdda0 msgr2=0x7f03300b7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:39.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.128+0000 7f0336faf640 1 --2- 172.21.15.45:0/2642302687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03300cdda0 0x7f03300b7420 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f03300b7b70 tx=0x7f0328004840 comp rx=0 tx=0).stop 2024-05-08T05:51:39.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.128+0000 7f0336faf640 1 -- 172.21.15.45:0/2642302687 shutdown_connections 2024-05-08T05:51:39.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.128+0000 7f0336faf640 1 --2- 172.21.15.45:0/2642302687 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f031c076300 0x7f031c0787c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.128+0000 7f0336faf640 1 --2- 172.21.15.45:0/2642302687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f03300cdda0 0x7f03300b7420 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.128+0000 7f0336faf640 1 --2- 172.21.15.45:0/2642302687 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f03300cd920 0x7f03300b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.128+0000 7f0336faf640 1 --2- 172.21.15.45:0/2642302687 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0318004800 0x7f03300ce8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.128+0000 7f0336faf640 1 -- 172.21.15.45:0/2642302687 >> 172.21.15.45:0/2642302687 conn(0x7f03300d4fe0 msgr2=0x7f03300ca2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:39.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.128+0000 7f0336faf640 1 -- 172.21.15.45:0/2642302687 shutdown_connections 2024-05-08T05:51:39.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.128+0000 7f0336faf640 1 -- 172.21.15.45:0/2642302687 wait complete. 2024-05-08T05:51:39.139 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T05:51:39.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:39.140 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T05:51:39.168 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:51:39.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:39.168 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T05:51:39.225 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:51:39.226 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:51:39.226 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:51:39.246 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:51:39.246 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:51:39.246 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:51:39.247 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:51:39.281 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:51:39.282 INFO:tasks.cephfs.mount:Terminating background process 2024-05-08T05:51:39.282 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:51:39.282 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:51:39.306 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:51:39.306 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:51:39.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.425+0000 7f1171874640 1 Processor -- start 2024-05-08T05:51:39.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.425+0000 7f1171874640 1 -- start start 2024-05-08T05:51:39.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.426+0000 7f1171874640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f116c15ccb0 0x7f116c15d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:39.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.426+0000 7f1171874640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f116c070b90 0x7f116c071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:39.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.426+0000 7f1171874640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f116c0718c0 con 0x7f116c0d2ce0 2024-05-08T05:51:39.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.426+0000 7f1171874640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f116c0afba0 con 0x7f116c15ccb0 2024-05-08T05:51:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.426+0000 7f1171874640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f116c071ba0 con 0x7f116c070b90 2024-05-08T05:51:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.426+0000 7f116affd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f116c15ccb0 0x7f116c15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.426+0000 7f116affd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f116c15ccb0 0x7f116c15d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39752/0 (socket says 172.21.15.45:39752) 2024-05-08T05:51:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.426+0000 7f116affd640 1 -- 172.21.15.45:0/3540777218 learned_addr learned my addr 172.21.15.45:0/3540777218 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.427+0000 7f116affd640 1 -- 172.21.15.45:0/3540777218 >> v2:172.21.15.63:3300/0 conn(0x7f116c070b90 msgr2=0x7f116c071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.427+0000 7f116affd640 1 --2- 172.21.15.45:0/3540777218 >> v2:172.21.15.63:3300/0 conn(0x7f116c070b90 0x7f116c071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.427+0000 7f116affd640 1 -- 172.21.15.45:0/3540777218 >> v1:172.21.15.63:6789/0 conn(0x7f116c0d2ce0 legacy=0x7f116c0d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.427+0000 7f116affd640 1 -- 172.21.15.45:0/3540777218 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116c071e80 con 0x7f116c15ccb0 2024-05-08T05:51:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.427+0000 7f116affd640 1 --2- 172.21.15.45:0/3540777218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f116c15ccb0 0x7f116c15d090 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f116c0d3830 tx=0x7f115c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd01163eb0f0b71 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.427+0000 7f1169ffb640 1 -- 172.21.15.45:0/3540777218 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c037070 con 0x7f116c15ccb0 2024-05-08T05:51:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.427+0000 7f1169ffb640 1 -- 172.21.15.45:0/3540777218 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f115c032440 con 0x7f116c15ccb0 2024-05-08T05:51:39.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.427+0000 7f1169ffb640 1 -- 172.21.15.45:0/3540777218 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c032810 con 0x7f116c15ccb0 2024-05-08T05:51:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.428+0000 7f1171874640 1 -- 172.21.15.45:0/3540777218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f116c15ccb0 msgr2=0x7f116c15d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:39.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.428+0000 7f1171874640 1 --2- 172.21.15.45:0/3540777218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f116c15ccb0 0x7f116c15d090 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f116c0d3830 tx=0x7f115c02a140 comp rx=0 tx=0).stop 2024-05-08T05:51:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.428+0000 7f1171874640 1 -- 172.21.15.45:0/3540777218 shutdown_connections 2024-05-08T05:51:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.428+0000 7f1171874640 1 --2- 172.21.15.45:0/3540777218 >> v2:172.21.15.63:3300/0 conn(0x7f116c070b90 0x7f116c071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.428+0000 7f1171874640 1 --2- 172.21.15.45:0/3540777218 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f116c15ccb0 0x7f116c15d090 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.428+0000 7f1171874640 1 -- 172.21.15.45:0/3540777218 >> 172.21.15.45:0/3540777218 conn(0x7f116c09ba40 msgr2=0x7f116c09be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.428+0000 7f1171874640 1 -- 172.21.15.45:0/3540777218 shutdown_connections 2024-05-08T05:51:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.428+0000 7f1171874640 1 -- 172.21.15.45:0/3540777218 wait complete. 2024-05-08T05:51:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.429+0000 7f1171874640 1 Processor -- start 2024-05-08T05:51:39.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.429+0000 7f1171874640 1 -- start start 2024-05-08T05:51:39.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.430+0000 7f1171874640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f116c070b90 0x7f116c0b7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:39.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.430+0000 7f1171874640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f116c0d2ce0 0x7f116c15fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:39.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.430+0000 7f1171874640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f116c15ccb0 0x7f116c15d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:39.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.430+0000 7f1171874640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f116c0cf980 con 0x7f116c15ccb0 2024-05-08T05:51:39.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.430+0000 7f1171874640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f116c071540 con 0x7f116c070b90 2024-05-08T05:51:39.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.430+0000 7f1171874640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f116c0cfb00 con 0x7f116c0d2ce0 2024-05-08T05:51:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.430+0000 7f116b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f116c15ccb0 0x7f116c15d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.430+0000 7f116b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f116c15ccb0 0x7f116c15d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39764/0 (socket says 172.21.15.45:39764) 2024-05-08T05:51:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.430+0000 7f116b7fe640 1 -- 172.21.15.45:0/2922526478 learned_addr learned my addr 172.21.15.45:0/2922526478 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.430+0000 7f116affd640 1 --2- 172.21.15.45:0/2922526478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f116c070b90 0x7f116c0b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.431+0000 7f116a7fc640 1 --2- 172.21.15.45:0/2922526478 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f116c0d2ce0 0x7f116c15fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.431+0000 7f116b7fe640 1 -- 172.21.15.45:0/2922526478 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f116c0d2ce0 msgr2=0x7f116c15fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.431+0000 7f116b7fe640 1 --2- 172.21.15.45:0/2922526478 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f116c0d2ce0 0x7f116c15fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.431+0000 7f116b7fe640 1 -- 172.21.15.45:0/2922526478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f116c070b90 msgr2=0x7f116c0b7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.431+0000 7f116b7fe640 1 --2- 172.21.15.45:0/2922526478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f116c070b90 0x7f116c0b7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.431+0000 7f116b7fe640 1 -- 172.21.15.45:0/2922526478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116c0b83a0 con 0x7f116c15ccb0 2024-05-08T05:51:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.431+0000 7f116a7fc640 1 --2- 172.21.15.45:0/2922526478 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f116c0d2ce0 0x7f116c15fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.431+0000 7f116b7fe640 1 --2- 172.21.15.45:0/2922526478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f116c15ccb0 0x7f116c15d880 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f116400b9a0 tx=0x7f116400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.431+0000 7f1153fff640 1 -- 172.21.15.45:0/2922526478 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164019070 con 0x7f116c15ccb0 2024-05-08T05:51:39.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.431+0000 7f1153fff640 1 -- 172.21.15.45:0/2922526478 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f116400cbc0 con 0x7f116c15ccb0 2024-05-08T05:51:39.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.431+0000 7f1171874640 1 -- 172.21.15.45:0/2922526478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f116c0b7b00 con 0x7f116c15ccb0 2024-05-08T05:51:39.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.431+0000 7f1153fff640 1 -- 172.21.15.45:0/2922526478 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164014440 con 0x7f116c15ccb0 2024-05-08T05:51:39.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.431+0000 7f1171874640 1 -- 172.21.15.45:0/2922526478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f116c0b80c0 con 0x7f116c15ccb0 2024-05-08T05:51:39.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.432+0000 7f1171874640 1 -- 172.21.15.45:0/2922526478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f116c069b00 con 0x7f116c15ccb0 2024-05-08T05:51:39.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.433+0000 7f1153fff640 1 -- 172.21.15.45:0/2922526478 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f116401c070 con 0x7f116c15ccb0 2024-05-08T05:51:39.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.433+0000 7f1153fff640 1 --2- 172.21.15.45:0/2922526478 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f11580762d0 0x7f1158078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:39.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.433+0000 7f1153fff640 1 -- 172.21.15.45:0/2922526478 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f1164097570 con 0x7f116c15ccb0 2024-05-08T05:51:39.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.433+0000 7f116affd640 1 --2- 172.21.15.45:0/2922526478 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f11580762d0 0x7f1158078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:39.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.434+0000 7f116affd640 1 --2- 172.21.15.45:0/2922526478 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f11580762d0 0x7f1158078790 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f115c034000 tx=0x7f115c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:39.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.438+0000 7f1153fff640 1 -- 172.21.15.45:0/2922526478 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1164060c90 con 0x7f116c15ccb0 2024-05-08T05:51:39.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.669+0000 7f1171874640 1 -- 172.21.15.45:0/2922526478 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f116c1600b0 con 0x7f116c15ccb0 2024-05-08T05:51:39.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.670+0000 7f1153fff640 1 -- 172.21.15.45:0/2922526478 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5188 (secure 0 0 0) 0x7f1164060590 con 0x7f116c15ccb0 2024-05-08T05:51:39.670 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:39.670 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":113,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7168,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3882445426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3882445426},{"type":"v1","addr":"172.21.15.63:6836","nonce":3882445426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/3173207720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3173207720},{"type":"v1","addr":"172.21.15.45:6833","nonce":3173207720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4254619360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4254619360},{"type":"v1","addr":"172.21.15.63:6837","nonce":4254619360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:51:04.429358+0000","modified":"2024-05-08T05:51:34.985085+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8463":{"gid":8463,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/4071581846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4071581846},{"type":"v1","addr":"172.21.15.45:6835","nonce":4071581846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8463,"qdb_cluster":[8463]},"id":8}]} 2024-05-08T05:51:39.671 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 113 2024-05-08T05:51:39.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.672+0000 7f1171874640 1 -- 172.21.15.45:0/2922526478 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f11580762d0 msgr2=0x7f1158078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:39.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.672+0000 7f1171874640 1 --2- 172.21.15.45:0/2922526478 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f11580762d0 0x7f1158078790 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f115c034000 tx=0x7f115c02e000 comp rx=0 tx=0).stop 2024-05-08T05:51:39.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.672+0000 7f1171874640 1 -- 172.21.15.45:0/2922526478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f116c15ccb0 msgr2=0x7f116c15d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:39.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.672+0000 7f1171874640 1 --2- 172.21.15.45:0/2922526478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f116c15ccb0 0x7f116c15d880 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f116400b9a0 tx=0x7f116400be60 comp rx=0 tx=0).stop 2024-05-08T05:51:39.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.672+0000 7f1171874640 1 -- 172.21.15.45:0/2922526478 shutdown_connections 2024-05-08T05:51:39.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.672+0000 7f1171874640 1 --2- 172.21.15.45:0/2922526478 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f11580762d0 0x7f1158078790 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.672+0000 7f1171874640 1 --2- 172.21.15.45:0/2922526478 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f116c0d2ce0 0x7f116c15fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.672+0000 7f1171874640 1 --2- 172.21.15.45:0/2922526478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f116c15ccb0 0x7f116c15d880 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.672+0000 7f1171874640 1 --2- 172.21.15.45:0/2922526478 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f116c070b90 0x7f116c0b7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.672+0000 7f1171874640 1 -- 172.21.15.45:0/2922526478 >> 172.21.15.45:0/2922526478 conn(0x7f116c09ba40 msgr2=0x7f116c0cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:39.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.672+0000 7f1171874640 1 -- 172.21.15.45:0/2922526478 shutdown_connections 2024-05-08T05:51:39.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.672+0000 7f1171874640 1 -- 172.21.15.45:0/2922526478 wait complete. 2024-05-08T05:51:39.684 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:51:39.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.787+0000 7fcd8f6a7640 1 Processor -- start 2024-05-08T05:51:39.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.788+0000 7fcd8f6a7640 1 -- start start 2024-05-08T05:51:39.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.788+0000 7fcd8f6a7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcd8815ccb0 0x7fcd8815d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:39.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.788+0000 7fcd8f6a7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd88070b90 0x7fcd88071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:39.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.788+0000 7fcd8f6a7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd880718c0 con 0x7fcd880d2ce0 2024-05-08T05:51:39.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.788+0000 7fcd8f6a7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fcd880afba0 con 0x7fcd8815ccb0 2024-05-08T05:51:39.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.788+0000 7fcd8f6a7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcd88071ba0 con 0x7fcd88070b90 2024-05-08T05:51:39.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.789+0000 7fcd8dc44640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd88070b90 0x7fcd88071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:39.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.789+0000 7fcd8d443640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fcd8815ccb0 0x7fcd8815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:39.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.789+0000 7fcd8cc42640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7fcd880d2ce0 0x7fcd880d3120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:33518/0 (socket says 172.21.15.45:33518) 2024-05-08T05:51:39.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.789+0000 7fcd8cc42640 1 -- 172.21.15.45:0/817406192 learned_addr learned my addr 172.21.15.45:0/817406192 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:39.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.789+0000 7fcd8d443640 1 -- 172.21.15.45:0/817406192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd88070b90 msgr2=0x7fcd88071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:39.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.789+0000 7fcd8d443640 1 --2- 172.21.15.45:0/817406192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd88070b90 0x7fcd88071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.789+0000 7fcd8d443640 1 -- 172.21.15.45:0/817406192 >> v1:172.21.15.63:6789/0 conn(0x7fcd880d2ce0 legacy=0x7fcd880d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:39.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.789+0000 7fcd8d443640 1 -- 172.21.15.45:0/817406192 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd88071e80 con 0x7fcd8815ccb0 2024-05-08T05:51:39.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.790+0000 7fcd8d443640 1 --2- 172.21.15.45:0/817406192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd8815ccb0 0x7fcd8815d090 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcd800097a0 tx=0x7fcd8002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a4bf448f7f654ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:39.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.790+0000 7fcd87fff640 1 -- 172.21.15.45:0/817406192 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80037070 con 0x7fcd8815ccb0 2024-05-08T05:51:39.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.790+0000 7fcd87fff640 1 -- 172.21.15.45:0/817406192 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd80032440 con 0x7fcd8815ccb0 2024-05-08T05:51:39.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.790+0000 7fcd87fff640 1 -- 172.21.15.45:0/817406192 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80032810 con 0x7fcd8815ccb0 2024-05-08T05:51:39.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.791+0000 7fcd8f6a7640 1 -- 172.21.15.45:0/817406192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd8815ccb0 msgr2=0x7fcd8815d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:39.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.791+0000 7fcd8f6a7640 1 --2- 172.21.15.45:0/817406192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd8815ccb0 0x7fcd8815d090 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcd800097a0 tx=0x7fcd8002a180 comp rx=0 tx=0).stop 2024-05-08T05:51:39.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.791+0000 7fcd8f6a7640 1 -- 172.21.15.45:0/817406192 shutdown_connections 2024-05-08T05:51:39.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.791+0000 7fcd8f6a7640 1 --2- 172.21.15.45:0/817406192 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd88070b90 0x7fcd88071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.791+0000 7fcd8f6a7640 1 --2- 172.21.15.45:0/817406192 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd8815ccb0 0x7fcd8815d090 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.791+0000 7fcd8f6a7640 1 -- 172.21.15.45:0/817406192 >> 172.21.15.45:0/817406192 conn(0x7fcd8809ba40 msgr2=0x7fcd8809be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:39.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.791+0000 7fcd8f6a7640 1 -- 172.21.15.45:0/817406192 shutdown_connections 2024-05-08T05:51:39.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.791+0000 7fcd8f6a7640 1 -- 172.21.15.45:0/817406192 wait complete. 2024-05-08T05:51:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.792+0000 7fcd8f6a7640 1 Processor -- start 2024-05-08T05:51:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.792+0000 7fcd8f6a7640 1 -- start start 2024-05-08T05:51:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.793+0000 7fcd8f6a7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd88070b90 0x7fcd8815d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.793+0000 7fcd8f6a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd880d2ce0 0x7fcd881f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.793+0000 7fcd8f6a7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd8815ccb0 0x7fcd8815f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.793+0000 7fcd8f6a7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcd880cf980 con 0x7fcd880d2ce0 2024-05-08T05:51:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.793+0000 7fcd8f6a7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcd88071560 con 0x7fcd88070b90 2024-05-08T05:51:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.793+0000 7fcd8f6a7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcd880cfb00 con 0x7fcd8815ccb0 2024-05-08T05:51:39.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.793+0000 7fcd8cc42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd880d2ce0 0x7fcd881f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.793+0000 7fcd8cc42640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd880d2ce0 0x7fcd881f2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39770/0 (socket says 172.21.15.45:39770) 2024-05-08T05:51:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.793+0000 7fcd8cc42640 1 -- 172.21.15.45:0/1152647373 learned_addr learned my addr 172.21.15.45:0/1152647373 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.793+0000 7fcd8d443640 1 --2- 172.21.15.45:0/1152647373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd88070b90 0x7fcd8815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.793+0000 7fcd8dc44640 1 --2- 172.21.15.45:0/1152647373 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd8815ccb0 0x7fcd8815f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.793+0000 7fcd8cc42640 1 -- 172.21.15.45:0/1152647373 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd8815ccb0 msgr2=0x7fcd8815f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.793+0000 7fcd8cc42640 1 --2- 172.21.15.45:0/1152647373 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd8815ccb0 0x7fcd8815f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.793+0000 7fcd8cc42640 1 -- 172.21.15.45:0/1152647373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd88070b90 msgr2=0x7fcd8815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.793+0000 7fcd8cc42640 1 --2- 172.21.15.45:0/1152647373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd88070b90 0x7fcd8815d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.793+0000 7fcd8cc42640 1 -- 172.21.15.45:0/1152647373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8815da10 con 0x7fcd880d2ce0 2024-05-08T05:51:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.793+0000 7fcd8d443640 1 --2- 172.21.15.45:0/1152647373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd88070b90 0x7fcd8815d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.793+0000 7fcd8cc42640 1 --2- 172.21.15.45:0/1152647373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd880d2ce0 0x7fcd881f2e10 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00c4d0 tx=0x7fcd7c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.793+0000 7fcd85ffb640 1 -- 172.21.15.45:0/1152647373 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c017070 con 0x7fcd880d2ce0 2024-05-08T05:51:39.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.793+0000 7fcd85ffb640 1 -- 172.21.15.45:0/1152647373 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd7c005d00 con 0x7fcd880d2ce0 2024-05-08T05:51:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.794+0000 7fcd8f6a7640 1 -- 172.21.15.45:0/1152647373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd8815dcf0 con 0x7fcd880d2ce0 2024-05-08T05:51:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.794+0000 7fcd85ffb640 1 -- 172.21.15.45:0/1152647373 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c0072b0 con 0x7fcd880d2ce0 2024-05-08T05:51:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.794+0000 7fcd8f6a7640 1 -- 172.21.15.45:0/1152647373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd88158e30 con 0x7fcd880d2ce0 2024-05-08T05:51:39.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.795+0000 7fcd8f6a7640 1 -- 172.21.15.45:0/1152647373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd880d39d0 con 0x7fcd880d2ce0 2024-05-08T05:51:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.795+0000 7fcd85ffb640 1 -- 172.21.15.45:0/1152647373 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcd7c01a070 con 0x7fcd880d2ce0 2024-05-08T05:51:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.796+0000 7fcd85ffb640 1 --2- 172.21.15.45:0/1152647373 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcd740762d0 0x7fcd74078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.796+0000 7fcd85ffb640 1 -- 172.21.15.45:0/1152647373 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7fcd7c096a90 con 0x7fcd880d2ce0 2024-05-08T05:51:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.796+0000 7fcd8d443640 1 --2- 172.21.15.45:0/1152647373 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcd740762d0 0x7fcd74078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:39.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.797+0000 7fcd8d443640 1 --2- 172.21.15.45:0/1152647373 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcd740762d0 0x7fcd74078790 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcd80034000 tx=0x7fcd8002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:39.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:39.800+0000 7fcd85ffb640 1 -- 172.21.15.45:0/1152647373 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcd880d39d0 con 0x7fcd880d2ce0 2024-05-08T05:51:40.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.030+0000 7fcd8f6a7640 1 -- 172.21.15.45:0/1152647373 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd88069b00 con 0x7fcd880d2ce0 2024-05-08T05:51:40.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.031+0000 7fcd85ffb640 1 -- 172.21.15.45:0/1152647373 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5188 (secure 0 0 0) 0x7fcd7c05fb60 con 0x7fcd880d2ce0 2024-05-08T05:51:40.031 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:40.032 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":113,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7168,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3882445426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3882445426},{"type":"v1","addr":"172.21.15.63:6836","nonce":3882445426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/3173207720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3173207720},{"type":"v1","addr":"172.21.15.45:6833","nonce":3173207720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4254619360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4254619360},{"type":"v1","addr":"172.21.15.63:6837","nonce":4254619360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:51:04.429358+0000","modified":"2024-05-08T05:51:34.985085+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8463":{"gid":8463,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/4071581846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4071581846},{"type":"v1","addr":"172.21.15.45:6835","nonce":4071581846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8463,"qdb_cluster":[8463]},"id":8}]} 2024-05-08T05:51:40.032 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 113 2024-05-08T05:51:40.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.033+0000 7fcd8f6a7640 1 -- 172.21.15.45:0/1152647373 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcd740762d0 msgr2=0x7fcd74078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:40.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.033+0000 7fcd8f6a7640 1 --2- 172.21.15.45:0/1152647373 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcd740762d0 0x7fcd74078790 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcd80034000 tx=0x7fcd8002e000 comp rx=0 tx=0).stop 2024-05-08T05:51:40.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.033+0000 7fcd8f6a7640 1 -- 172.21.15.45:0/1152647373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd880d2ce0 msgr2=0x7fcd881f2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:40.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.033+0000 7fcd8f6a7640 1 --2- 172.21.15.45:0/1152647373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd880d2ce0 0x7fcd881f2e10 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00c4d0 tx=0x7fcd7c00c990 comp rx=0 tx=0).stop 2024-05-08T05:51:40.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.033+0000 7fcd8f6a7640 1 -- 172.21.15.45:0/1152647373 shutdown_connections 2024-05-08T05:51:40.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.033+0000 7fcd8f6a7640 1 --2- 172.21.15.45:0/1152647373 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcd740762d0 0x7fcd74078790 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.033+0000 7fcd8f6a7640 1 --2- 172.21.15.45:0/1152647373 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd8815ccb0 0x7fcd8815f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.033+0000 7fcd8f6a7640 1 --2- 172.21.15.45:0/1152647373 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd880d2ce0 0x7fcd881f2e10 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.033+0000 7fcd8f6a7640 1 --2- 172.21.15.45:0/1152647373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd88070b90 0x7fcd8815d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.033+0000 7fcd8f6a7640 1 -- 172.21.15.45:0/1152647373 >> 172.21.15.45:0/1152647373 conn(0x7fcd8809ba40 msgr2=0x7fcd880d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:40.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.033+0000 7fcd8f6a7640 1 -- 172.21.15.45:0/1152647373 shutdown_connections 2024-05-08T05:51:40.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.033+0000 7fcd8f6a7640 1 -- 172.21.15.45:0/1152647373 wait complete. 2024-05-08T05:51:40.044 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:51:40.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.148+0000 7ffb93254640 1 Processor -- start 2024-05-08T05:51:40.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.148+0000 7ffb93254640 1 -- start start 2024-05-08T05:51:40.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.149+0000 7ffb93254640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffb8c070d30 0x7ffb8c0d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:40.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.149+0000 7ffb93254640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7ffb8c071110 0x7ffb8c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:40.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.149+0000 7ffb93254640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb8c0d1300 con 0x7ffb8c163920 2024-05-08T05:51:40.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.149+0000 7ffb93254640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ffb8c0d10d0 con 0x7ffb8c070d30 2024-05-08T05:51:40.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.149+0000 7ffb93254640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7ffb8c163db0 con 0x7ffb8c071110 2024-05-08T05:51:40.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.149+0000 7ffb90ff0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ffb8c070d30 0x7ffb8c0d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:40.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.149+0000 7ffb8bfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ffb8c163920 0x7ffb8c0703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46122/0 (socket says 172.21.15.45:46122) 2024-05-08T05:51:40.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.149+0000 7ffb8bfff640 1 -- 172.21.15.45:0/873533416 learned_addr learned my addr 172.21.15.45:0/873533416 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:40.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.150+0000 7ffb90ff0640 1 -- 172.21.15.45:0/873533416 >> v2:172.21.15.63:3300/0 conn(0x7ffb8c071110 msgr2=0x7ffb8c06d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:40.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.150+0000 7ffb90ff0640 1 --2- 172.21.15.45:0/873533416 >> v2:172.21.15.63:3300/0 conn(0x7ffb8c071110 0x7ffb8c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.150+0000 7ffb90ff0640 1 -- 172.21.15.45:0/873533416 >> v1:172.21.15.45:6789/0 conn(0x7ffb8c163920 legacy=0x7ffb8c0703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:40.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.150+0000 7ffb90ff0640 1 -- 172.21.15.45:0/873533416 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8c0d15e0 con 0x7ffb8c070d30 2024-05-08T05:51:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.150+0000 7ffb90ff0640 1 --2- 172.21.15.45:0/873533416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb8c070d30 0x7ffb8c0d20c0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c073540 tx=0x7ffb7c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8f1c9c130473ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.150+0000 7ffb8b7fe640 1 -- 172.21.15.45:0/873533416 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c037070 con 0x7ffb8c070d30 2024-05-08T05:51:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.151+0000 7ffb8b7fe640 1 -- 172.21.15.45:0/873533416 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb7c032440 con 0x7ffb8c070d30 2024-05-08T05:51:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.151+0000 7ffb8b7fe640 1 -- 172.21.15.45:0/873533416 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c032810 con 0x7ffb8c070d30 2024-05-08T05:51:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.151+0000 7ffb93254640 1 -- 172.21.15.45:0/873533416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb8c070d30 msgr2=0x7ffb8c0d20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.151+0000 7ffb93254640 1 --2- 172.21.15.45:0/873533416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb8c070d30 0x7ffb8c0d20c0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c073540 tx=0x7ffb7c02a1a0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.151+0000 7ffb93254640 1 -- 172.21.15.45:0/873533416 shutdown_connections 2024-05-08T05:51:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.151+0000 7ffb93254640 1 --2- 172.21.15.45:0/873533416 >> v2:172.21.15.63:3300/0 conn(0x7ffb8c071110 0x7ffb8c06d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.151+0000 7ffb93254640 1 --2- 172.21.15.45:0/873533416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb8c070d30 0x7ffb8c0d20c0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.151+0000 7ffb93254640 1 -- 172.21.15.45:0/873533416 >> 172.21.15.45:0/873533416 conn(0x7ffb8c0d2e90 msgr2=0x7ffb8c0d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:40.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.151+0000 7ffb93254640 1 -- 172.21.15.45:0/873533416 shutdown_connections 2024-05-08T05:51:40.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.151+0000 7ffb93254640 1 -- 172.21.15.45:0/873533416 wait complete. 2024-05-08T05:51:40.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.152+0000 7ffb93254640 1 Processor -- start 2024-05-08T05:51:40.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.152+0000 7ffb93254640 1 -- start start 2024-05-08T05:51:40.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.153+0000 7ffb93254640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb8c070d30 0x7ffb8c156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:40.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.153+0000 7ffb93254640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb8c071110 0x7ffb8c0cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:40.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.153+0000 7ffb93254640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb8c163920 0x7ffb8c06e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:40.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.153+0000 7ffb93254640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c0d1c30 con 0x7ffb8c070d30 2024-05-08T05:51:40.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.153+0000 7ffb93254640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c070b80 con 0x7ffb8c071110 2024-05-08T05:51:40.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.153+0000 7ffb93254640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ffb8c071560 con 0x7ffb8c163920 2024-05-08T05:51:40.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.153+0000 7ffb90ff0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb8c070d30 0x7ffb8c156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:40.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.153+0000 7ffb90ff0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb8c070d30 0x7ffb8c156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39786/0 (socket says 172.21.15.45:39786) 2024-05-08T05:51:40.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.153+0000 7ffb90ff0640 1 -- 172.21.15.45:0/1479089617 learned_addr learned my addr 172.21.15.45:0/1479089617 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:40.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.154+0000 7ffb8bfff640 1 --2- 172.21.15.45:0/1479089617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb8c071110 0x7ffb8c0cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:40.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.154+0000 7ffb90ff0640 1 -- 172.21.15.45:0/1479089617 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb8c163920 msgr2=0x7ffb8c06e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:40.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.154+0000 7ffb917f1640 1 --2- 172.21.15.45:0/1479089617 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb8c163920 0x7ffb8c06e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:40.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.154+0000 7ffb90ff0640 1 --2- 172.21.15.45:0/1479089617 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb8c163920 0x7ffb8c06e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.154+0000 7ffb90ff0640 1 -- 172.21.15.45:0/1479089617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb8c071110 msgr2=0x7ffb8c0cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:40.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.154+0000 7ffb90ff0640 1 --2- 172.21.15.45:0/1479089617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb8c071110 0x7ffb8c0cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.154+0000 7ffb90ff0640 1 -- 172.21.15.45:0/1479089617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8c15c020 con 0x7ffb8c070d30 2024-05-08T05:51:40.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.154+0000 7ffb8bfff640 1 --2- 172.21.15.45:0/1479089617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb8c071110 0x7ffb8c0cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:40.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.154+0000 7ffb917f1640 1 --2- 172.21.15.45:0/1479089617 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb8c163920 0x7ffb8c06e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:40.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.154+0000 7ffb90ff0640 1 --2- 172.21.15.45:0/1479089617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb8c070d30 0x7ffb8c156680 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c000c00 tx=0x7ffb7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:40.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.154+0000 7ffb897fa640 1 -- 172.21.15.45:0/1479089617 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c041070 con 0x7ffb8c070d30 2024-05-08T05:51:40.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.154+0000 7ffb897fa640 1 -- 172.21.15.45:0/1479089617 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb7c03c890 con 0x7ffb8c070d30 2024-05-08T05:51:40.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.154+0000 7ffb93254640 1 -- 172.21.15.45:0/1479089617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb8c15c300 con 0x7ffb8c070d30 2024-05-08T05:51:40.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.154+0000 7ffb897fa640 1 -- 172.21.15.45:0/1479089617 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c037040 con 0x7ffb8c070d30 2024-05-08T05:51:40.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.154+0000 7ffb93254640 1 -- 172.21.15.45:0/1479089617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb8c15c8c0 con 0x7ffb8c070d30 2024-05-08T05:51:40.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.155+0000 7ffb93254640 1 -- 172.21.15.45:0/1479089617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb8c0693f0 con 0x7ffb8c070d30 2024-05-08T05:51:40.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.156+0000 7ffb897fa640 1 -- 172.21.15.45:0/1479089617 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ffb7c044070 con 0x7ffb8c070d30 2024-05-08T05:51:40.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.156+0000 7ffb897fa640 1 --2- 172.21.15.45:0/1479089617 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ffb780762d0 0x7ffb78078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:40.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.157+0000 7ffb897fa640 1 -- 172.21.15.45:0/1479089617 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7ffb7c0b84e0 con 0x7ffb8c070d30 2024-05-08T05:51:40.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.157+0000 7ffb8bfff640 1 --2- 172.21.15.45:0/1479089617 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ffb780762d0 0x7ffb78078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:40.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.157+0000 7ffb8bfff640 1 --2- 172.21.15.45:0/1479089617 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ffb780762d0 0x7ffb78078790 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c1f8be0 tx=0x7ffb80009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:40.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.161+0000 7ffb897fa640 1 -- 172.21.15.45:0/1479089617 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ffb7c081b70 con 0x7ffb8c070d30 2024-05-08T05:51:40.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.352+0000 7ffb93254640 1 -- 172.21.15.45:0/1479089617 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb8c0d3d70 con 0x7ffb8c070d30 2024-05-08T05:51:40.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.353+0000 7ffb897fa640 1 -- 172.21.15.45:0/1479089617 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+17161 (secure 0 0 0) 0x7ffb7c081470 con 0x7ffb8c070d30 2024-05-08T05:51:40.353 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:40.353 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":121,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:51:29.962334+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:51:02.094088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-08T05:51:03.388702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/453198328":"2024-05-09T05:51:29.962310+0000","172.21.15.45:6834/453198328":"2024-05-09T05:51:29.962310+0000","172.21.15.45:6835/1751419054":"2024-05-09T05:51:20.651495+0000","172.21.15.45:6834/1751419054":"2024-05-09T05:51:20.651495+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:51:40.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.355+0000 7ffb93254640 1 -- 172.21.15.45:0/1479089617 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ffb780762d0 msgr2=0x7ffb78078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:40.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.355+0000 7ffb93254640 1 --2- 172.21.15.45:0/1479089617 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ffb780762d0 0x7ffb78078790 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c1f8be0 tx=0x7ffb80009040 comp rx=0 tx=0).stop 2024-05-08T05:51:40.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.355+0000 7ffb93254640 1 -- 172.21.15.45:0/1479089617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb8c070d30 msgr2=0x7ffb8c156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:40.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.355+0000 7ffb93254640 1 --2- 172.21.15.45:0/1479089617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb8c070d30 0x7ffb8c156680 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c000c00 tx=0x7ffb7c0046c0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.355+0000 7ffb93254640 1 -- 172.21.15.45:0/1479089617 shutdown_connections 2024-05-08T05:51:40.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.355+0000 7ffb93254640 1 --2- 172.21.15.45:0/1479089617 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ffb780762d0 0x7ffb78078790 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.355+0000 7ffb93254640 1 --2- 172.21.15.45:0/1479089617 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ffb8c163920 0x7ffb8c06e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.355+0000 7ffb93254640 1 --2- 172.21.15.45:0/1479089617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffb8c071110 0x7ffb8c0cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.355+0000 7ffb93254640 1 --2- 172.21.15.45:0/1479089617 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb8c070d30 0x7ffb8c156680 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.355+0000 7ffb93254640 1 -- 172.21.15.45:0/1479089617 >> 172.21.15.45:0/1479089617 conn(0x7ffb8c0d2e90 msgr2=0x7ffb8c0d3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:40.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.355+0000 7ffb93254640 1 -- 172.21.15.45:0/1479089617 shutdown_connections 2024-05-08T05:51:40.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.355+0000 7ffb93254640 1 -- 172.21.15.45:0/1479089617 wait complete. 2024-05-08T05:51:40.366 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T05:51:40.366 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:51:40.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.470+0000 7f45c04da640 1 Processor -- start 2024-05-08T05:51:40.470 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.470+0000 7f45c04da640 1 -- start start 2024-05-08T05:51:40.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.471+0000 7f45c04da640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f45b80d20c0 0x7f45b8163920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:40.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.471+0000 7f45c04da640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45b80d1300 con 0x7f45b8070d30 2024-05-08T05:51:40.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.471+0000 7f45c04da640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45b80d15e0 con 0x7f45b8071110 2024-05-08T05:51:40.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.471+0000 7f45c04da640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f45b80d10d0 con 0x7f45b80d20c0 2024-05-08T05:51:40.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.471+0000 7f45bda75640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f45b80d20c0 0x7f45b8163920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:40.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.471+0000 7f45be276640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f45b8070d30 0x7f45b80703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:46124/0 (socket says 172.21.15.45:46124) 2024-05-08T05:51:40.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.471+0000 7f45bda75640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f45b80d20c0 0x7f45b8163920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39802/0 (socket says 172.21.15.45:39802) 2024-05-08T05:51:40.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.471+0000 7f45be276640 1 -- 172.21.15.45:0/3348285154 learned_addr learned my addr 172.21.15.45:0/3348285154 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:40.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.472+0000 7f45bda75640 1 -- 172.21.15.45:0/3348285154 >> v1:172.21.15.63:6789/0 conn(0x7f45b8071110 legacy=0x7f45b806d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:40.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.472+0000 7f45bda75640 1 -- 172.21.15.45:0/3348285154 >> v1:172.21.15.45:6789/0 conn(0x7f45b8070d30 legacy=0x7f45b80703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:40.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.472+0000 7f45bd274640 1 -- 172.21.15.45:0/3348285154 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 881737404 0 0) 0x7f45b80d1300 con 0x7f45b8070d30 2024-05-08T05:51:40.472 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.472+0000 7f45bda75640 1 -- 172.21.15.45:0/3348285154 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b80d18c0 con 0x7f45b80d20c0 2024-05-08T05:51:40.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.472+0000 7f45bda75640 1 --2- 172.21.15.45:0/3348285154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45b80d20c0 0x7f45b8163920 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0097a0 tx=0x7f45ac02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1993887463796b10 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:40.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.472+0000 7f45bd274640 1 -- 172.21.15.45:0/3348285154 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac037070 con 0x7f45b80d20c0 2024-05-08T05:51:40.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.472+0000 7f45bd274640 1 -- 172.21.15.45:0/3348285154 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45ac032440 con 0x7f45b80d20c0 2024-05-08T05:51:40.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.472+0000 7f45bd274640 1 -- 172.21.15.45:0/3348285154 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac032810 con 0x7f45b80d20c0 2024-05-08T05:51:40.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.473+0000 7f45c04da640 1 -- 172.21.15.45:0/3348285154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45b80d20c0 msgr2=0x7f45b8163920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:40.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.473+0000 7f45c04da640 1 --2- 172.21.15.45:0/3348285154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45b80d20c0 0x7f45b8163920 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0097a0 tx=0x7f45ac02a230 comp rx=0 tx=0).stop 2024-05-08T05:51:40.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.473+0000 7f45c04da640 1 -- 172.21.15.45:0/3348285154 shutdown_connections 2024-05-08T05:51:40.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.473+0000 7f45c04da640 1 --2- 172.21.15.45:0/3348285154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45b80d20c0 0x7f45b8163920 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.473+0000 7f45c04da640 1 -- 172.21.15.45:0/3348285154 >> 172.21.15.45:0/3348285154 conn(0x7f45b80d2e90 msgr2=0x7f45b80d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:40.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.473+0000 7f45c04da640 1 -- 172.21.15.45:0/3348285154 shutdown_connections 2024-05-08T05:51:40.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.473+0000 7f45c04da640 1 -- 172.21.15.45:0/3348285154 wait complete. 2024-05-08T05:51:40.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.474+0000 7f45c04da640 1 Processor -- start 2024-05-08T05:51:40.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.474+0000 7f45c04da640 1 -- start start 2024-05-08T05:51:40.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.475+0000 7f45c04da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45b8070d30 0x7f45b80cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:40.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.475+0000 7f45c04da640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45b8071110 0x7f45b80b7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:40.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.475+0000 7f45c04da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45b80d20c0 0x7f45b80b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:40.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.475+0000 7f45c04da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f45b80d1c30 con 0x7f45b80d20c0 2024-05-08T05:51:40.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.475+0000 7f45c04da640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f45b8070b80 con 0x7f45b8071110 2024-05-08T05:51:40.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.475+0000 7f45c04da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f45b806dbc0 con 0x7f45b8070d30 2024-05-08T05:51:40.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.475+0000 7f45bea77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45b80d20c0 0x7f45b80b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:40.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.475+0000 7f45bea77640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45b80d20c0 0x7f45b80b7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39808/0 (socket says 172.21.15.45:39808) 2024-05-08T05:51:40.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.475+0000 7f45bea77640 1 -- 172.21.15.45:0/1216753852 learned_addr learned my addr 172.21.15.45:0/1216753852 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:40.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.475+0000 7f45be276640 1 --2- 172.21.15.45:0/1216753852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45b8070d30 0x7f45b80cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:40.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.475+0000 7f45bda75640 1 --2- 172.21.15.45:0/1216753852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45b8071110 0x7f45b80b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:40.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.475+0000 7f45bea77640 1 -- 172.21.15.45:0/1216753852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45b8070d30 msgr2=0x7f45b80cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:40.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.475+0000 7f45bea77640 1 --2- 172.21.15.45:0/1216753852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45b8070d30 0x7f45b80cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.475+0000 7f45bea77640 1 -- 172.21.15.45:0/1216753852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45b8071110 msgr2=0x7f45b80b7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:40.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.475+0000 7f45bea77640 1 --2- 172.21.15.45:0/1216753852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45b8071110 0x7f45b80b7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.475+0000 7f45bea77640 1 -- 172.21.15.45:0/1216753852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b80cc020 con 0x7f45b80d20c0 2024-05-08T05:51:40.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.476+0000 7f45bea77640 1 --2- 172.21.15.45:0/1216753852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45b80d20c0 0x7f45b80b7b50 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f45b000c7a0 tx=0x7f45b000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:40.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.476+0000 7f45b6ffd640 1 -- 172.21.15.45:0/1216753852 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0003b70 con 0x7f45b80d20c0 2024-05-08T05:51:40.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.476+0000 7f45b6ffd640 1 -- 172.21.15.45:0/1216753852 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b0003d10 con 0x7f45b80d20c0 2024-05-08T05:51:40.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.476+0000 7f45c04da640 1 -- 172.21.15.45:0/1216753852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b80cc300 con 0x7f45b80d20c0 2024-05-08T05:51:40.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.476+0000 7f45b6ffd640 1 -- 172.21.15.45:0/1216753852 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b000aa70 con 0x7f45b80d20c0 2024-05-08T05:51:40.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.476+0000 7f45c04da640 1 -- 172.21.15.45:0/1216753852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b80cc8c0 con 0x7f45b80d20c0 2024-05-08T05:51:40.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.477+0000 7f45c04da640 1 -- 172.21.15.45:0/1216753852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b806fc30 con 0x7f45b80d20c0 2024-05-08T05:51:40.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.477+0000 7f45b6ffd640 1 -- 172.21.15.45:0/1216753852 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f45b000ac10 con 0x7f45b80d20c0 2024-05-08T05:51:40.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.478+0000 7f45b6ffd640 1 --2- 172.21.15.45:0/1216753852 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f45a40762d0 0x7f45a4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:40.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.478+0000 7f45b6ffd640 1 -- 172.21.15.45:0/1216753852 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f45b0097780 con 0x7f45b80d20c0 2024-05-08T05:51:40.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.478+0000 7f45be276640 1 --2- 172.21.15.45:0/1216753852 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f45a40762d0 0x7f45a4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:40.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.479+0000 7f45be276640 1 --2- 172.21.15.45:0/1216753852 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f45a40762d0 0x7f45a4078790 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f45a8007fd0 tx=0x7f45a8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:40.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.482+0000 7f45b6ffd640 1 -- 172.21.15.45:0/1216753852 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f45b806fc30 con 0x7f45b80d20c0 2024-05-08T05:51:40.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.715+0000 7f45c04da640 1 -- 172.21.15.45:0/1216753852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45b80693f0 con 0x7f45b80d20c0 2024-05-08T05:51:40.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.715+0000 7f45b6ffd640 1 -- 172.21.15.45:0/1216753852 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5188 (secure 0 0 0) 0x7f45b0060ea0 con 0x7f45b80d20c0 2024-05-08T05:51:40.716 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:40.716 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":113,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7168,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3882445426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3882445426},{"type":"v1","addr":"172.21.15.63:6836","nonce":3882445426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/3173207720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3173207720},{"type":"v1","addr":"172.21.15.45:6833","nonce":3173207720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4254619360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4254619360},{"type":"v1","addr":"172.21.15.63:6837","nonce":4254619360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:51:04.429358+0000","modified":"2024-05-08T05:51:34.985085+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8463":{"gid":8463,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/4071581846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4071581846},{"type":"v1","addr":"172.21.15.45:6835","nonce":4071581846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8463,"qdb_cluster":[8463]},"id":8}]} 2024-05-08T05:51:40.716 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 113 2024-05-08T05:51:40.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.717+0000 7f45c04da640 1 -- 172.21.15.45:0/1216753852 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f45a40762d0 msgr2=0x7f45a4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:40.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.717+0000 7f45c04da640 1 --2- 172.21.15.45:0/1216753852 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f45a40762d0 0x7f45a4078790 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f45a8007fd0 tx=0x7f45a8009040 comp rx=0 tx=0).stop 2024-05-08T05:51:40.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.717+0000 7f45c04da640 1 -- 172.21.15.45:0/1216753852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45b80d20c0 msgr2=0x7f45b80b7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:40.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.717+0000 7f45c04da640 1 --2- 172.21.15.45:0/1216753852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45b80d20c0 0x7f45b80b7b50 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f45b000c7a0 tx=0x7f45b000cc60 comp rx=0 tx=0).stop 2024-05-08T05:51:40.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.717+0000 7f45c04da640 1 -- 172.21.15.45:0/1216753852 shutdown_connections 2024-05-08T05:51:40.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.717+0000 7f45c04da640 1 --2- 172.21.15.45:0/1216753852 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f45a40762d0 0x7f45a4078790 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.717+0000 7f45c04da640 1 --2- 172.21.15.45:0/1216753852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45b80d20c0 0x7f45b80b7b50 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.717+0000 7f45c04da640 1 --2- 172.21.15.45:0/1216753852 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f45b8071110 0x7f45b80b7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.717+0000 7f45c04da640 1 --2- 172.21.15.45:0/1216753852 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f45b8070d30 0x7f45b80cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.717+0000 7f45c04da640 1 -- 172.21.15.45:0/1216753852 >> 172.21.15.45:0/1216753852 conn(0x7f45b80d2e90 msgr2=0x7f45b80b5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:40.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.717+0000 7f45c04da640 1 -- 172.21.15.45:0/1216753852 shutdown_connections 2024-05-08T05:51:40.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.717+0000 7f45c04da640 1 -- 172.21.15.45:0/1216753852 wait complete. 2024-05-08T05:51:40.728 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:51:40.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.832+0000 7f5103d24640 1 Processor -- start 2024-05-08T05:51:40.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.833+0000 7f5103d24640 1 -- start start 2024-05-08T05:51:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.833+0000 7f5103d24640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50fc0d1970 0x7f50fc0d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.834+0000 7f5103d24640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50fc0d4cc0 0x7f50fc0d5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.834+0000 7f5103d24640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50fc1630b0 con 0x7f50fc15f420 2024-05-08T05:51:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.834+0000 7f5103d24640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f50fc0d3e70 con 0x7f50fc0d1970 2024-05-08T05:51:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.834+0000 7f5103d24640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f50fc0d5670 con 0x7f50fc0d4cc0 2024-05-08T05:51:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.834+0000 7f51012bf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50fc0d1970 0x7f50fc0d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.834+0000 7f5101ac0640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f50fc15f420 0x7f50fc0d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38718/0 (socket says 172.21.15.45:38718) 2024-05-08T05:51:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.834+0000 7f5101ac0640 1 -- 172.21.15.45:0/2562435936 learned_addr learned my addr 172.21.15.45:0/2562435936 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.834+0000 7f51012bf640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f50fc0d1970 0x7f50fc0d1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54240/0 (socket says 172.21.15.45:54240) 2024-05-08T05:51:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.834+0000 7f51022c1640 1 --2- 172.21.15.45:0/2562435936 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50fc0d4cc0 0x7f50fc0d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:40.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.834+0000 7f51012bf640 1 -- 172.21.15.45:0/2562435936 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50fc0d4cc0 msgr2=0x7f50fc0d5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.834+0000 7f51012bf640 1 --2- 172.21.15.45:0/2562435936 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50fc0d4cc0 0x7f50fc0d5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.834+0000 7f51012bf640 1 -- 172.21.15.45:0/2562435936 >> v1:172.21.15.45:6789/0 conn(0x7f50fc15f420 legacy=0x7f50fc0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.834+0000 7f51012bf640 1 -- 172.21.15.45:0/2562435936 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50fc0d2550 con 0x7f50fc0d1970 2024-05-08T05:51:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.835+0000 7f51012bf640 1 --2- 172.21.15.45:0/2562435936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50fc0d1970 0x7f50fc0d1e20 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f50f00097d0 tx=0x7f50f002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2183addd1208fe8f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.835+0000 7f5100abe640 1 -- 172.21.15.45:0/2562435936 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0037070 con 0x7f50fc0d1970 2024-05-08T05:51:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.835+0000 7f5100abe640 1 -- 172.21.15.45:0/2562435936 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f0032440 con 0x7f50fc0d1970 2024-05-08T05:51:40.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.835+0000 7f5100abe640 1 -- 172.21.15.45:0/2562435936 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0032810 con 0x7f50fc0d1970 2024-05-08T05:51:40.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.835+0000 7f5103d24640 1 -- 172.21.15.45:0/2562435936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50fc0d1970 msgr2=0x7f50fc0d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:40.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.835+0000 7f5103d24640 1 --2- 172.21.15.45:0/2562435936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50fc0d1970 0x7f50fc0d1e20 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f50f00097d0 tx=0x7f50f002a1c0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.836+0000 7f5103d24640 1 -- 172.21.15.45:0/2562435936 shutdown_connections 2024-05-08T05:51:40.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.836+0000 7f5103d24640 1 --2- 172.21.15.45:0/2562435936 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50fc0d4cc0 0x7f50fc0d5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.836+0000 7f5103d24640 1 --2- 172.21.15.45:0/2562435936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50fc0d1970 0x7f50fc0d1e20 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.836+0000 7f5103d24640 1 -- 172.21.15.45:0/2562435936 >> 172.21.15.45:0/2562435936 conn(0x7f50fc08eac0 msgr2=0x7f50fc09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:40.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.836+0000 7f5103d24640 1 -- 172.21.15.45:0/2562435936 shutdown_connections 2024-05-08T05:51:40.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.836+0000 7f5103d24640 1 -- 172.21.15.45:0/2562435936 wait complete. 2024-05-08T05:51:40.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.837+0000 7f5103d24640 1 Processor -- start 2024-05-08T05:51:40.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.837+0000 7f5103d24640 1 -- start start 2024-05-08T05:51:40.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f5103d24640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50fc0d1970 0x7f50fc0d0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:40.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f5103d24640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50fc0d4cc0 0x7f50fc161870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:40.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f5103d24640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50fc161db0 0x7f50fc1f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:40.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f5103d24640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc0d3240 con 0x7f50fc161db0 2024-05-08T05:51:40.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f5103d24640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc0d2f60 con 0x7f50fc0d1970 2024-05-08T05:51:40.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f5103d24640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f50fc0d2830 con 0x7f50fc0d4cc0 2024-05-08T05:51:40.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f51022c1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50fc161db0 0x7f50fc1f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:40.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f51022c1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50fc161db0 0x7f50fc1f77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54250/0 (socket says 172.21.15.45:54250) 2024-05-08T05:51:40.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f51022c1640 1 -- 172.21.15.45:0/3442097559 learned_addr learned my addr 172.21.15.45:0/3442097559 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:40.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f51012bf640 1 --2- 172.21.15.45:0/3442097559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50fc0d4cc0 0x7f50fc161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:40.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f5101ac0640 1 --2- 172.21.15.45:0/3442097559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50fc0d1970 0x7f50fc0d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:40.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f51022c1640 1 -- 172.21.15.45:0/3442097559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50fc0d4cc0 msgr2=0x7f50fc161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:40.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f51022c1640 1 --2- 172.21.15.45:0/3442097559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50fc0d4cc0 0x7f50fc161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f51022c1640 1 -- 172.21.15.45:0/3442097559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50fc0d1970 msgr2=0x7f50fc0d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:40.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f51022c1640 1 --2- 172.21.15.45:0/3442097559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50fc0d1970 0x7f50fc0d0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:40.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f51022c1640 1 -- 172.21.15.45:0/3442097559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50fc162240 con 0x7f50fc161db0 2024-05-08T05:51:40.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f5101ac0640 1 --2- 172.21.15.45:0/3442097559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50fc0d1970 0x7f50fc0d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:40.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f51012bf640 1 --2- 172.21.15.45:0/3442097559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50fc0d4cc0 0x7f50fc161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:40.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f51022c1640 1 --2- 172.21.15.45:0/3442097559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50fc161db0 0x7f50fc1f77f0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00d830 tx=0x7f50ec00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:40.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f50fa7fc640 1 -- 172.21.15.45:0/3442097559 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec00b6c0 con 0x7f50fc161db0 2024-05-08T05:51:40.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f50fa7fc640 1 -- 172.21.15.45:0/3442097559 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ec019070 con 0x7f50fc161db0 2024-05-08T05:51:40.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f5103d24640 1 -- 172.21.15.45:0/3442097559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50fc1f8800 con 0x7f50fc161db0 2024-05-08T05:51:40.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f50fa7fc640 1 -- 172.21.15.45:0/3442097559 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec00bac0 con 0x7f50fc161db0 2024-05-08T05:51:40.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.838+0000 7f5103d24640 1 -- 172.21.15.45:0/3442097559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50fc1f8dc0 con 0x7f50fc161db0 2024-05-08T05:51:40.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.839+0000 7f5103d24640 1 -- 172.21.15.45:0/3442097559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50fc0693f0 con 0x7f50fc161db0 2024-05-08T05:51:40.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.840+0000 7f50fa7fc640 1 -- 172.21.15.45:0/3442097559 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f50ec01e460 con 0x7f50fc161db0 2024-05-08T05:51:40.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.841+0000 7f50fa7fc640 1 --2- 172.21.15.45:0/3442097559 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50e80762d0 0x7f50e8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:40.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.841+0000 7f50fa7fc640 1 -- 172.21.15.45:0/3442097559 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f50ec097e80 con 0x7f50fc161db0 2024-05-08T05:51:40.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.841+0000 7f5101ac0640 1 --2- 172.21.15.45:0/3442097559 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50e80762d0 0x7f50e8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:40.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.841+0000 7f5101ac0640 1 --2- 172.21.15.45:0/3442097559 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50e80762d0 0x7f50e8078790 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f50f40012e0 tx=0x7f50f4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:40.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:40.845+0000 7f50fa7fc640 1 -- 172.21.15.45:0/3442097559 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f50fc162240 con 0x7f50fc161db0 2024-05-08T05:51:41.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.085+0000 7f5103d24640 1 -- 172.21.15.45:0/3442097559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50fc0b4c50 con 0x7f50fc161db0 2024-05-08T05:51:41.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.085+0000 7f50fa7fc640 1 -- 172.21.15.45:0/3442097559 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5188 (secure 0 0 0) 0x7f50ec061630 con 0x7f50fc161db0 2024-05-08T05:51:41.085 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:41.085 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":113,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7168,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3882445426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3882445426},{"type":"v1","addr":"172.21.15.63:6836","nonce":3882445426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/3173207720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3173207720},{"type":"v1","addr":"172.21.15.45:6833","nonce":3173207720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4254619360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4254619360},{"type":"v1","addr":"172.21.15.63:6837","nonce":4254619360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:51:04.429358+0000","modified":"2024-05-08T05:51:34.985085+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8463":{"gid":8463,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":4,"addr":"172.21.15.45:6835/4071581846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4071581846},{"type":"v1","addr":"172.21.15.45:6835","nonce":4071581846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8463,"qdb_cluster":[8463]},"id":8}]} 2024-05-08T05:51:41.086 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 113 2024-05-08T05:51:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.087+0000 7f5103d24640 1 -- 172.21.15.45:0/3442097559 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50e80762d0 msgr2=0x7f50e8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.087+0000 7f5103d24640 1 --2- 172.21.15.45:0/3442097559 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50e80762d0 0x7f50e8078790 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f50f40012e0 tx=0x7f50f4009040 comp rx=0 tx=0).stop 2024-05-08T05:51:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.087+0000 7f5103d24640 1 -- 172.21.15.45:0/3442097559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50fc161db0 msgr2=0x7f50fc1f77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.087+0000 7f5103d24640 1 --2- 172.21.15.45:0/3442097559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50fc161db0 0x7f50fc1f77f0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00d830 tx=0x7f50ec00dcf0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.087+0000 7f5103d24640 1 -- 172.21.15.45:0/3442097559 shutdown_connections 2024-05-08T05:51:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.087+0000 7f5103d24640 1 --2- 172.21.15.45:0/3442097559 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f50e80762d0 0x7f50e8078790 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.087+0000 7f5103d24640 1 --2- 172.21.15.45:0/3442097559 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f50fc0d4cc0 0x7f50fc161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.087+0000 7f5103d24640 1 --2- 172.21.15.45:0/3442097559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50fc161db0 0x7f50fc1f77f0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.087+0000 7f5103d24640 1 --2- 172.21.15.45:0/3442097559 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f50fc0d1970 0x7f50fc0d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.087+0000 7f5103d24640 1 -- 172.21.15.45:0/3442097559 >> 172.21.15.45:0/3442097559 conn(0x7f50fc08eac0 msgr2=0x7f50fc163290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.087+0000 7f5103d24640 1 -- 172.21.15.45:0/3442097559 shutdown_connections 2024-05-08T05:51:41.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.087+0000 7f5103d24640 1 -- 172.21.15.45:0/3442097559 wait complete. 2024-05-08T05:51:41.099 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:51:41.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.202+0000 7fc4b9e47640 1 Processor -- start 2024-05-08T05:51:41.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.202+0000 7fc4b9e47640 1 -- start start 2024-05-08T05:51:41.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.203+0000 7fc4b9e47640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc4b40d3cd0 0x7fc4b40cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:41.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.203+0000 7fc4b9e47640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4b40c9ee0 0x7fc4b40d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:41.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.203+0000 7fc4b9e47640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4b40d3330 con 0x7fc4b41629a0 2024-05-08T05:51:41.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.203+0000 7fc4b9e47640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc4b40d2170 con 0x7fc4b40d3cd0 2024-05-08T05:51:41.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.203+0000 7fc4b9e47640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b40d2450 con 0x7fc4b40c9ee0 2024-05-08T05:51:41.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.203+0000 7fc4b37fe640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc4b41629a0 0x7fc4b40d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38726/0 (socket says 172.21.15.45:38726) 2024-05-08T05:51:41.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.203+0000 7fc4b37fe640 1 -- 172.21.15.45:0/1627390185 learned_addr learned my addr 172.21.15.45:0/1627390185 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:41.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.204+0000 7fc4b3fff640 1 --2- 172.21.15.45:0/1627390185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4b40c9ee0 0x7fc4b40d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:41.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.204+0000 7fc4b2ffd640 1 --2- 172.21.15.45:0/1627390185 >> v2:172.21.15.63:3300/0 conn(0x7fc4b40d3cd0 0x7fc4b40cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:41.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.204+0000 7fc4b27fc640 1 -- 172.21.15.45:0/1627390185 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2737218763 0 0) 0x7fc4b40d3330 con 0x7fc4b41629a0 2024-05-08T05:51:41.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.204+0000 7fc4b27fc640 1 -- 172.21.15.45:0/1627390185 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc49c003520 con 0x7fc4b41629a0 2024-05-08T05:51:41.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.204+0000 7fc4b27fc640 1 -- 172.21.15.45:0/1627390185 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 891751993 0 0) 0x7fc49c003520 con 0x7fc4b41629a0 2024-05-08T05:51:41.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.204+0000 7fc4b2ffd640 1 -- 172.21.15.45:0/1627390185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4b40c9ee0 msgr2=0x7fc4b40d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:41.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.204+0000 7fc4b2ffd640 1 --2- 172.21.15.45:0/1627390185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4b40c9ee0 0x7fc4b40d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.204+0000 7fc4b2ffd640 1 -- 172.21.15.45:0/1627390185 >> v1:172.21.15.45:6789/0 conn(0x7fc4b41629a0 legacy=0x7fc4b40d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:41.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.204+0000 7fc4b2ffd640 1 -- 172.21.15.45:0/1627390185 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b40d2730 con 0x7fc4b40d3cd0 2024-05-08T05:51:41.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.205+0000 7fc4b2ffd640 1 --2- 172.21.15.45:0/1627390185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b40d3cd0 0x7fc4b40cfca0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc4a80097a0 tx=0x7fc4a802a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea0398047bd800fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:41.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.205+0000 7fc4b27fc640 1 -- 172.21.15.45:0/1627390185 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8037070 con 0x7fc4b40d3cd0 2024-05-08T05:51:41.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.205+0000 7fc4b27fc640 1 -- 172.21.15.45:0/1627390185 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a802aa90 con 0x7fc4b40d3cd0 2024-05-08T05:51:41.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.205+0000 7fc4b27fc640 1 -- 172.21.15.45:0/1627390185 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a40016f0 con 0x7fc4b40d3cd0 2024-05-08T05:51:41.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.205+0000 7fc4b9e47640 1 -- 172.21.15.45:0/1627390185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b40d3cd0 msgr2=0x7fc4b40cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:41.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.205+0000 7fc4b9e47640 1 --2- 172.21.15.45:0/1627390185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b40d3cd0 0x7fc4b40cfca0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc4a80097a0 tx=0x7fc4a802a000 comp rx=0 tx=0).stop 2024-05-08T05:51:41.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.205+0000 7fc4b9e47640 1 -- 172.21.15.45:0/1627390185 shutdown_connections 2024-05-08T05:51:41.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.205+0000 7fc4b9e47640 1 --2- 172.21.15.45:0/1627390185 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4b40c9ee0 0x7fc4b40d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.205+0000 7fc4b9e47640 1 --2- 172.21.15.45:0/1627390185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b40d3cd0 0x7fc4b40cfca0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.205+0000 7fc4b9e47640 1 -- 172.21.15.45:0/1627390185 >> 172.21.15.45:0/1627390185 conn(0x7fc4b40d4fe0 msgr2=0x7fc4b40d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:41.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.206+0000 7fc4b9e47640 1 -- 172.21.15.45:0/1627390185 shutdown_connections 2024-05-08T05:51:41.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.206+0000 7fc4b9e47640 1 -- 172.21.15.45:0/1627390185 wait complete. 2024-05-08T05:51:41.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.207+0000 7fc4b9e47640 1 Processor -- start 2024-05-08T05:51:41.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.207+0000 7fc4b9e47640 1 -- start start 2024-05-08T05:51:41.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc4b9e47640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b40c9ee0 0x7fc4b4154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:41.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc4b9e47640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4b40d3cd0 0x7fc4b40ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:41.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc4b9e47640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4b41629a0 0x7fc4b40b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:41.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc4b9e47640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b40cf440 con 0x7fc4b41629a0 2024-05-08T05:51:41.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc4b9e47640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b40cf2c0 con 0x7fc4b40c9ee0 2024-05-08T05:51:41.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc4b9e47640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b40d2dc0 con 0x7fc4b40d3cd0 2024-05-08T05:51:41.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc4b3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4b41629a0 0x7fc4b40b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:41.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc4b3fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4b41629a0 0x7fc4b40b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54266/0 (socket says 172.21.15.45:54266) 2024-05-08T05:51:41.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc4b3fff640 1 -- 172.21.15.45:0/1560053703 learned_addr learned my addr 172.21.15.45:0/1560053703 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:41.210 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc4b37fe640 1 --2- 172.21.15.45:0/1560053703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b40c9ee0 0x7fc4b4154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc4b2ffd640 1 --2- 172.21.15.45:0/1560053703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4b40d3cd0 0x7fc4b40ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc4b3fff640 1 -- 172.21.15.45:0/1560053703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4b40d3cd0 msgr2=0x7fc4b40ba6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc4b3fff640 1 --2- 172.21.15.45:0/1560053703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4b40d3cd0 0x7fc4b40ba6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc4b3fff640 1 -- 172.21.15.45:0/1560053703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b40c9ee0 msgr2=0x7fc4b4154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc4b3fff640 1 --2- 172.21.15.45:0/1560053703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b40c9ee0 0x7fc4b4154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc4b3fff640 1 -- 172.21.15.45:0/1560053703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b40b87a0 con 0x7fc4b41629a0 2024-05-08T05:51:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc4b2ffd640 1 --2- 172.21.15.45:0/1560053703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4b40d3cd0 0x7fc4b40ba6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc4b3fff640 1 --2- 172.21.15.45:0/1560053703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4b41629a0 0x7fc4b40b32f0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00efc0 tx=0x7fc4ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc49bfff640 1 -- 172.21.15.45:0/1560053703 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac018020 con 0x7fc4b41629a0 2024-05-08T05:51:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc49bfff640 1 -- 172.21.15.45:0/1560053703 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4ac014480 con 0x7fc4b41629a0 2024-05-08T05:51:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc4b9e47640 1 -- 172.21.15.45:0/1560053703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b4154880 con 0x7fc4b41629a0 2024-05-08T05:51:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc49bfff640 1 -- 172.21.15.45:0/1560053703 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac014870 con 0x7fc4b41629a0 2024-05-08T05:51:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.208+0000 7fc4b9e47640 1 -- 172.21.15.45:0/1560053703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b40b8d40 con 0x7fc4b41629a0 2024-05-08T05:51:41.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.209+0000 7fc4b9e47640 1 -- 172.21.15.45:0/1560053703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b40693f0 con 0x7fc4b41629a0 2024-05-08T05:51:41.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.210+0000 7fc49bfff640 1 -- 172.21.15.45:0/1560053703 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc4ac005d10 con 0x7fc4b41629a0 2024-05-08T05:51:41.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.211+0000 7fc49bfff640 1 --2- 172.21.15.45:0/1560053703 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc4a00762d0 0x7fc4a0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:41.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.211+0000 7fc49bfff640 1 -- 172.21.15.45:0/1560053703 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7fc4ac098350 con 0x7fc4b41629a0 2024-05-08T05:51:41.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.211+0000 7fc4b37fe640 1 --2- 172.21.15.45:0/1560053703 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc4a00762d0 0x7fc4a0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:41.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.211+0000 7fc4b37fe640 1 --2- 172.21.15.45:0/1560053703 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc4a00762d0 0x7fc4a0078790 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40017d0 tx=0x7fc4a4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:41.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.215+0000 7fc49bfff640 1 -- 172.21.15.45:0/1560053703 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc4ac022070 con 0x7fc4b41629a0 2024-05-08T05:51:41.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.410+0000 7fc4b9e47640 1 -- 172.21.15.45:0/1560053703 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4b4162150 con 0x7fc4b41629a0 2024-05-08T05:51:41.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.411+0000 7fc49bfff640 1 -- 172.21.15.45:0/1560053703 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+17161 (secure 0 0 0) 0x7fc4ac010040 con 0x7fc4b41629a0 2024-05-08T05:51:41.411 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:41.411 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":121,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:51:29.962334+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:51:02.094088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-05-08T05:51:03.388702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/453198328":"2024-05-09T05:51:29.962310+0000","172.21.15.45:6834/453198328":"2024-05-09T05:51:29.962310+0000","172.21.15.45:6835/1751419054":"2024-05-09T05:51:20.651495+0000","172.21.15.45:6834/1751419054":"2024-05-09T05:51:20.651495+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:51:41.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.413+0000 7fc4b9e47640 1 -- 172.21.15.45:0/1560053703 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc4a00762d0 msgr2=0x7fc4a0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:41.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.413+0000 7fc4b9e47640 1 --2- 172.21.15.45:0/1560053703 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc4a00762d0 0x7fc4a0078790 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40017d0 tx=0x7fc4a4009040 comp rx=0 tx=0).stop 2024-05-08T05:51:41.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.413+0000 7fc4b9e47640 1 -- 172.21.15.45:0/1560053703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4b41629a0 msgr2=0x7fc4b40b32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:41.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.413+0000 7fc4b9e47640 1 --2- 172.21.15.45:0/1560053703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4b41629a0 0x7fc4b40b32f0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00efc0 tx=0x7fc4ac00c490 comp rx=0 tx=0).stop 2024-05-08T05:51:41.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.413+0000 7fc4b9e47640 1 -- 172.21.15.45:0/1560053703 shutdown_connections 2024-05-08T05:51:41.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.413+0000 7fc4b9e47640 1 --2- 172.21.15.45:0/1560053703 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc4a00762d0 0x7fc4a0078790 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.413+0000 7fc4b9e47640 1 --2- 172.21.15.45:0/1560053703 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc4b40d3cd0 0x7fc4b40ba6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.413+0000 7fc4b9e47640 1 --2- 172.21.15.45:0/1560053703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc4b41629a0 0x7fc4b40b32f0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.413+0000 7fc4b9e47640 1 --2- 172.21.15.45:0/1560053703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc4b40c9ee0 0x7fc4b4154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.413+0000 7fc4b9e47640 1 -- 172.21.15.45:0/1560053703 >> 172.21.15.45:0/1560053703 conn(0x7fc4b40d4fe0 msgr2=0x7fc4b40b6560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:41.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.413+0000 7fc4b9e47640 1 -- 172.21.15.45:0/1560053703 shutdown_connections 2024-05-08T05:51:41.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.413+0000 7fc4b9e47640 1 -- 172.21.15.45:0/1560053703 wait complete. 2024-05-08T05:51:41.425 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T05:51:41.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.530+0000 7f662f459640 1 Processor -- start 2024-05-08T05:51:41.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.530+0000 7f662f459640 1 -- start start 2024-05-08T05:51:41.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.531+0000 7f662f459640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f662815f2d0 0x7f66280b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:41.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.531+0000 7f662f459640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f66280d2310 0x7f66280d5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:41.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.531+0000 7f662f459640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66280d2730 con 0x7f66280d4fc0 2024-05-08T05:51:41.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.531+0000 7f662f459640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f66280d5960 con 0x7f66280d2310 2024-05-08T05:51:41.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.531+0000 7f662f459640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f66280b5810 con 0x7f662815f2d0 2024-05-08T05:51:41.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.531+0000 7f662d9f6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f66280d2310 0x7f66280d5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:41.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.531+0000 7f662c9f4640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f66280d4fc0 0x7f66280d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38742/0 (socket says 172.21.15.45:38742) 2024-05-08T05:51:41.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.531+0000 7f662d9f6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f66280d2310 0x7f66280d5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54274/0 (socket says 172.21.15.45:54274) 2024-05-08T05:51:41.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.531+0000 7f662c9f4640 1 -- 172.21.15.45:0/3695191496 learned_addr learned my addr 172.21.15.45:0/3695191496 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:41.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.531+0000 7f662d1f5640 1 --2- 172.21.15.45:0/3695191496 >> v2:172.21.15.63:3300/0 conn(0x7f662815f2d0 0x7f66280b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:41.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.531+0000 7f662d9f6640 1 -- 172.21.15.45:0/3695191496 >> v2:172.21.15.63:3300/0 conn(0x7f662815f2d0 msgr2=0x7f66280b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:41.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.531+0000 7f662d9f6640 1 --2- 172.21.15.45:0/3695191496 >> v2:172.21.15.63:3300/0 conn(0x7f662815f2d0 0x7f66280b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.531+0000 7f662d9f6640 1 -- 172.21.15.45:0/3695191496 >> v1:172.21.15.45:6789/0 conn(0x7f66280d4fc0 legacy=0x7f66280d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:41.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.531+0000 7f661ffff640 1 -- 172.21.15.45:0/3695191496 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 640047959 0 0) 0x7f66280d2730 con 0x7f66280d4fc0 2024-05-08T05:51:41.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.531+0000 7f662d9f6640 1 -- 172.21.15.45:0/3695191496 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66281606c0 con 0x7f66280d2310 2024-05-08T05:51:41.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.531+0000 7f662d1f5640 1 --2- 172.21.15.45:0/3695191496 >> v2:172.21.15.63:3300/0 conn(0x7f662815f2d0 0x7f66280b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:41.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.532+0000 7f662d9f6640 1 --2- 172.21.15.45:0/3695191496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66280d2310 0x7f66280d5420 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f66280d4f90 tx=0x7f662002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7f24af56883234 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:41.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.532+0000 7f661ffff640 1 -- 172.21.15.45:0/3695191496 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6620007740 con 0x7f66280d2310 2024-05-08T05:51:41.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.532+0000 7f661ffff640 1 -- 172.21.15.45:0/3695191496 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66200078e0 con 0x7f66280d2310 2024-05-08T05:51:41.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.532+0000 7f661ffff640 1 -- 172.21.15.45:0/3695191496 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6620007cb0 con 0x7f66280d2310 2024-05-08T05:51:41.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.533+0000 7f662f459640 1 -- 172.21.15.45:0/3695191496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66280d2310 msgr2=0x7f66280d5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:41.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.533+0000 7f662f459640 1 --2- 172.21.15.45:0/3695191496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66280d2310 0x7f66280d5420 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f66280d4f90 tx=0x7f662002a680 comp rx=0 tx=0).stop 2024-05-08T05:51:41.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.533+0000 7f662f459640 1 -- 172.21.15.45:0/3695191496 shutdown_connections 2024-05-08T05:51:41.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.533+0000 7f662f459640 1 --2- 172.21.15.45:0/3695191496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66280d2310 0x7f66280d5420 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.533+0000 7f662f459640 1 --2- 172.21.15.45:0/3695191496 >> v2:172.21.15.63:3300/0 conn(0x7f662815f2d0 0x7f66280b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.533+0000 7f662f459640 1 -- 172.21.15.45:0/3695191496 >> 172.21.15.45:0/3695191496 conn(0x7f66280746d0 msgr2=0x7f662809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:41.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.533+0000 7f662f459640 1 -- 172.21.15.45:0/3695191496 shutdown_connections 2024-05-08T05:51:41.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.533+0000 7f662f459640 1 -- 172.21.15.45:0/3695191496 wait complete. 2024-05-08T05:51:41.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.534+0000 7f662f459640 1 Processor -- start 2024-05-08T05:51:41.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.534+0000 7f662f459640 1 -- start start 2024-05-08T05:51:41.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.535+0000 7f662f459640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66280d2310 0x7f6628151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:41.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.535+0000 7f662f459640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f66280d4fc0 0x7f66280cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:41.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.535+0000 7f662f459640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f662815f2d0 0x7f66280cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:41.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.535+0000 7f662f459640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f66280b54a0 con 0x7f662815f2d0 2024-05-08T05:51:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.535+0000 7f662f459640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f66280b51c0 con 0x7f66280d2310 2024-05-08T05:51:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.535+0000 7f662f459640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f66280d2c00 con 0x7f66280d4fc0 2024-05-08T05:51:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.535+0000 7f662d9f6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f662815f2d0 0x7f66280cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.535+0000 7f662d9f6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f662815f2d0 0x7f66280cc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54286/0 (socket says 172.21.15.45:54286) 2024-05-08T05:51:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.535+0000 7f662d9f6640 1 -- 172.21.15.45:0/1231253320 learned_addr learned my addr 172.21.15.45:0/1231253320 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.535+0000 7f662d1f5640 1 --2- 172.21.15.45:0/1231253320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66280d2310 0x7f6628151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.535+0000 7f662c9f4640 1 --2- 172.21.15.45:0/1231253320 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f66280d4fc0 0x7f66280cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.535+0000 7f662d9f6640 1 -- 172.21.15.45:0/1231253320 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f66280d4fc0 msgr2=0x7f66280cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.535+0000 7f662d9f6640 1 --2- 172.21.15.45:0/1231253320 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f66280d4fc0 0x7f66280cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.535+0000 7f662d9f6640 1 -- 172.21.15.45:0/1231253320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66280d2310 msgr2=0x7f6628151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.535+0000 7f662d9f6640 1 --2- 172.21.15.45:0/1231253320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66280d2310 0x7f6628151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.535+0000 7f662d9f6640 1 -- 172.21.15.45:0/1231253320 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66280ccb50 con 0x7f662815f2d0 2024-05-08T05:51:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.535+0000 7f662c9f4640 1 --2- 172.21.15.45:0/1231253320 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f66280d4fc0 0x7f66280cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:41.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.535+0000 7f662d9f6640 1 --2- 172.21.15.45:0/1231253320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f662815f2d0 0x7f66280cc100 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f662002ac70 tx=0x7f6620004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.535+0000 7f661dffb640 1 -- 172.21.15.45:0/1231253320 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6620040070 con 0x7f662815f2d0 2024-05-08T05:51:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.536+0000 7f661dffb640 1 -- 172.21.15.45:0/1231253320 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6620031bc0 con 0x7f662815f2d0 2024-05-08T05:51:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.536+0000 7f662f459640 1 -- 172.21.15.45:0/1231253320 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66280cce30 con 0x7f662815f2d0 2024-05-08T05:51:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.536+0000 7f661dffb640 1 -- 172.21.15.45:0/1231253320 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6620045460 con 0x7f662815f2d0 2024-05-08T05:51:41.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.536+0000 7f662f459640 1 -- 172.21.15.45:0/1231253320 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66280cd3f0 con 0x7f662815f2d0 2024-05-08T05:51:41.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.537+0000 7f662f459640 1 -- 172.21.15.45:0/1231253320 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66280cd8e0 con 0x7f662815f2d0 2024-05-08T05:51:41.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.537+0000 7f661dffb640 1 -- 172.21.15.45:0/1231253320 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6620039040 con 0x7f662815f2d0 2024-05-08T05:51:41.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.538+0000 7f661dffb640 1 --2- 172.21.15.45:0/1231253320 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f660c076300 0x7f660c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:41.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.538+0000 7f661dffb640 1 -- 172.21.15.45:0/1231253320 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6273+0+0 (secure 0 0 0) 0x7f66200b8fb0 con 0x7f662815f2d0 2024-05-08T05:51:41.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.538+0000 7f662d1f5640 1 --2- 172.21.15.45:0/1231253320 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f660c076300 0x7f660c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:41.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.539+0000 7f662d1f5640 1 --2- 172.21.15.45:0/1231253320 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f660c076300 0x7f660c0787c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6624006fd0 tx=0x7f6624008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:41.542 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.543+0000 7f661dffb640 1 -- 172.21.15.45:0/1231253320 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6620082760 con 0x7f662815f2d0 2024-05-08T05:51:41.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.772+0000 7f662f459640 1 -- 172.21.15.45:0/1231253320 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6628069b00 con 0x7f662815f2d0 2024-05-08T05:51:41.773 INFO:tasks.ceph.mon.a.smithi045.stderr:2024-05-08T05:51:41.773+0000 7f124de12640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T05:51:41.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.775+0000 7f661dffb640 1 -- 172.21.15.45:0/1231253320 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v114) ==== 161+0+0 (secure 0 0 0) 0x7f6620082060 con 0x7f662815f2d0 2024-05-08T05:51:41.774 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T05:51:41.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.777+0000 7f662f459640 1 -- 172.21.15.45:0/1231253320 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f660c076300 msgr2=0x7f660c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:41.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.777+0000 7f662f459640 1 --2- 172.21.15.45:0/1231253320 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f660c076300 0x7f660c0787c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6624006fd0 tx=0x7f6624008040 comp rx=0 tx=0).stop 2024-05-08T05:51:41.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.777+0000 7f662f459640 1 -- 172.21.15.45:0/1231253320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f662815f2d0 msgr2=0x7f66280cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:41.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.777+0000 7f662f459640 1 --2- 172.21.15.45:0/1231253320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f662815f2d0 0x7f66280cc100 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f662002ac70 tx=0x7f6620004a40 comp rx=0 tx=0).stop 2024-05-08T05:51:41.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.777+0000 7f662f459640 1 -- 172.21.15.45:0/1231253320 shutdown_connections 2024-05-08T05:51:41.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.777+0000 7f662f459640 1 --2- 172.21.15.45:0/1231253320 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f660c076300 0x7f660c0787c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.777+0000 7f662f459640 1 --2- 172.21.15.45:0/1231253320 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f66280d4fc0 0x7f66280cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.778+0000 7f662f459640 1 --2- 172.21.15.45:0/1231253320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f662815f2d0 0x7f66280cc100 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.778+0000 7f662f459640 1 --2- 172.21.15.45:0/1231253320 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f66280d2310 0x7f6628151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.778+0000 7f662f459640 1 -- 172.21.15.45:0/1231253320 >> 172.21.15.45:0/1231253320 conn(0x7f66280746d0 msgr2=0x7f66280d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:41.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.778+0000 7f662f459640 1 -- 172.21.15.45:0/1231253320 shutdown_connections 2024-05-08T05:51:41.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.778+0000 7f662f459640 1 -- 172.21.15.45:0/1231253320 wait complete. 2024-05-08T05:51:41.779 INFO:tasks.ceph.mds.c.smithi045.stderr: -1199> 2024-05-08T05:51:35.970+0000 7ff259610640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:51:41.789 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T05:51:41.805 INFO:tasks.ceph.mds.c.smithi045.stdout:starting mds.c at 2024-05-08T05:51:41.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.896+0000 7fec10454640 1 Processor -- start 2024-05-08T05:51:41.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.896+0000 7fec10454640 1 -- start start 2024-05-08T05:51:41.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.897+0000 7fec10454640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec081629a0 0x7fec080d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:41.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.897+0000 7fec10454640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fec080d0cb0 0x7fec080d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:41.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.897+0000 7fec10454640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec080d3330 con 0x7fec080d2dc0 2024-05-08T05:51:41.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.897+0000 7fec10454640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fec080d2170 con 0x7fec080d0cb0 2024-05-08T05:51:41.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.897+0000 7fec10454640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fec080d2450 con 0x7fec081629a0 2024-05-08T05:51:41.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.897+0000 7fec0e9f1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fec080d0cb0 0x7fec080d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:41.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.897+0000 7fec0d9ef640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fec080d2dc0 0x7fec080c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38746/0 (socket says 172.21.15.45:38746) 2024-05-08T05:51:41.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.897+0000 7fec0e9f1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fec080d0cb0 0x7fec080d3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54316/0 (socket says 172.21.15.45:54316) 2024-05-08T05:51:41.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.897+0000 7fec0d9ef640 1 -- 172.21.15.45:0/1910872769 learned_addr learned my addr 172.21.15.45:0/1910872769 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:41.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.898+0000 7fec0e1f0640 1 --2- 172.21.15.45:0/1910872769 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec081629a0 0x7fec080d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:41.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.898+0000 7fec0e9f1640 1 -- 172.21.15.45:0/1910872769 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec081629a0 msgr2=0x7fec080d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:41.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.898+0000 7fec0e9f1640 1 --2- 172.21.15.45:0/1910872769 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec081629a0 0x7fec080d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.898+0000 7fec0e9f1640 1 -- 172.21.15.45:0/1910872769 >> v1:172.21.15.45:6789/0 conn(0x7fec080d2dc0 legacy=0x7fec080c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:41.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.898+0000 7fec0e9f1640 1 -- 172.21.15.45:0/1910872769 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec080d2730 con 0x7fec080d0cb0 2024-05-08T05:51:41.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.898+0000 7fec0e1f0640 1 --2- 172.21.15.45:0/1910872769 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec081629a0 0x7fec080d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:41.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.898+0000 7fec0e9f1640 1 --2- 172.21.15.45:0/1910872769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec080d0cb0 0x7fec080d3b00 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fec00009d90 tx=0x7fec0002a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9241db27fa5b832 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:41.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.899+0000 7fec0d1ee640 1 -- 172.21.15.45:0/1910872769 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec00007740 con 0x7fec080d0cb0 2024-05-08T05:51:41.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.899+0000 7fec0d1ee640 1 -- 172.21.15.45:0/1910872769 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec000078e0 con 0x7fec080d0cb0 2024-05-08T05:51:41.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.899+0000 7fec0d1ee640 1 -- 172.21.15.45:0/1910872769 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec00007cb0 con 0x7fec080d0cb0 2024-05-08T05:51:41.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.899+0000 7fec10454640 1 -- 172.21.15.45:0/1910872769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec080d0cb0 msgr2=0x7fec080d3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:41.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.899+0000 7fec10454640 1 --2- 172.21.15.45:0/1910872769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec080d0cb0 0x7fec080d3b00 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fec00009d90 tx=0x7fec0002a530 comp rx=0 tx=0).stop 2024-05-08T05:51:41.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.899+0000 7fec10454640 1 -- 172.21.15.45:0/1910872769 shutdown_connections 2024-05-08T05:51:41.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.899+0000 7fec10454640 1 --2- 172.21.15.45:0/1910872769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec080d0cb0 0x7fec080d3b00 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.899+0000 7fec10454640 1 --2- 172.21.15.45:0/1910872769 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec081629a0 0x7fec080d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.899+0000 7fec10454640 1 -- 172.21.15.45:0/1910872769 >> 172.21.15.45:0/1910872769 conn(0x7fec080d4fe0 msgr2=0x7fec080d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:41.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.899+0000 7fec10454640 1 -- 172.21.15.45:0/1910872769 shutdown_connections 2024-05-08T05:51:41.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.899+0000 7fec10454640 1 -- 172.21.15.45:0/1910872769 wait complete. 2024-05-08T05:51:41.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.901+0000 7fec10454640 1 Processor -- start 2024-05-08T05:51:41.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.901+0000 7fec10454640 1 -- start start 2024-05-08T05:51:41.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.901+0000 7fec10454640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec080d0cb0 0x7fec0815d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:41.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.901+0000 7fec10454640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec080d2dc0 0x7fec081f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:41.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.901+0000 7fec10454640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec081629a0 0x7fec081f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:41.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.901+0000 7fec10454640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fec080cf440 con 0x7fec081629a0 2024-05-08T05:51:41.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.901+0000 7fec10454640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fec080cf2c0 con 0x7fec080d0cb0 2024-05-08T05:51:41.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.901+0000 7fec10454640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fec080c95e0 con 0x7fec080d2dc0 2024-05-08T05:51:41.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.902+0000 7fec0e9f1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec081629a0 0x7fec081f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:41.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.902+0000 7fec0e9f1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec081629a0 0x7fec081f53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54332/0 (socket says 172.21.15.45:54332) 2024-05-08T05:51:41.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.902+0000 7fec0e9f1640 1 -- 172.21.15.45:0/1632792742 learned_addr learned my addr 172.21.15.45:0/1632792742 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:41.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.902+0000 7fec0d9ef640 1 --2- 172.21.15.45:0/1632792742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec080d2dc0 0x7fec081f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:41.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.902+0000 7fec0e1f0640 1 --2- 172.21.15.45:0/1632792742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec080d0cb0 0x7fec0815d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:41.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.902+0000 7fec0e9f1640 1 -- 172.21.15.45:0/1632792742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec080d2dc0 msgr2=0x7fec081f4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:41.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.902+0000 7fec0e9f1640 1 --2- 172.21.15.45:0/1632792742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec080d2dc0 0x7fec081f4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.902+0000 7fec0e9f1640 1 -- 172.21.15.45:0/1632792742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec080d0cb0 msgr2=0x7fec0815d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:41.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.902+0000 7fec0e9f1640 1 --2- 172.21.15.45:0/1632792742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec080d0cb0 0x7fec0815d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:41.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.902+0000 7fec0e9f1640 1 -- 172.21.15.45:0/1632792742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec080a1960 con 0x7fec081629a0 2024-05-08T05:51:41.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.902+0000 7fec0d9ef640 1 --2- 172.21.15.45:0/1632792742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec080d2dc0 0x7fec081f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:41.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.902+0000 7fec0e1f0640 1 --2- 172.21.15.45:0/1632792742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec080d0cb0 0x7fec0815d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:41.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.902+0000 7fec0e9f1640 1 --2- 172.21.15.45:0/1632792742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec081629a0 0x7fec081f53d0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fec00009eb0 tx=0x7fec00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:41.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.902+0000 7febfeffd640 1 -- 172.21.15.45:0/1632792742 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec00040070 con 0x7fec081629a0 2024-05-08T05:51:41.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.902+0000 7febfeffd640 1 -- 172.21.15.45:0/1632792742 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec00031c60 con 0x7fec081629a0 2024-05-08T05:51:41.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.902+0000 7febfeffd640 1 -- 172.21.15.45:0/1632792742 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0003c660 con 0x7fec081629a0 2024-05-08T05:51:41.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.902+0000 7fec10454640 1 -- 172.21.15.45:0/1632792742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec080a1c40 con 0x7fec081629a0 2024-05-08T05:51:41.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.902+0000 7fec10454640 1 -- 172.21.15.45:0/1632792742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec080b8df0 con 0x7fec081629a0 2024-05-08T05:51:41.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.903+0000 7fec10454640 1 -- 172.21.15.45:0/1632792742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec080693f0 con 0x7fec081629a0 2024-05-08T05:51:41.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.904+0000 7febfeffd640 1 -- 172.21.15.45:0/1632792742 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fec00039040 con 0x7fec081629a0 2024-05-08T05:51:41.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.905+0000 7febfeffd640 1 --2- 172.21.15.45:0/1632792742 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7febec0762d0 0x7febec078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:41.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.905+0000 7febfeffd640 1 -- 172.21.15.45:0/1632792742 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6359+0+0 (secure 0 0 0) 0x7fec000b8df0 con 0x7fec081629a0 2024-05-08T05:51:41.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.905+0000 7fec0e1f0640 1 --2- 172.21.15.45:0/1632792742 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7febec0762d0 0x7febec078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:41.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.906+0000 7fec0e1f0640 1 --2- 172.21.15.45:0/1632792742 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7febec0762d0 0x7febec078790 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fec04006fd0 tx=0x7fec04008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:41.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:41.909+0000 7febfeffd640 1 -- 172.21.15.45:0/1632792742 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fec000824c0 con 0x7fec081629a0 2024-05-08T05:51:42.143 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.143+0000 7fec10454640 1 -- 172.21.15.45:0/1632792742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fec080a2160 con 0x7fec081629a0 2024-05-08T05:51:42.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.776+0000 7febfeffd640 1 -- 172.21.15.45:0/1632792742 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v115) ==== 104+0+0 (secure 0 0 0) 0x7fec00081dc0 con 0x7fec081629a0 2024-05-08T05:51:42.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.779+0000 7fec10454640 1 -- 172.21.15.45:0/1632792742 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7febec0762d0 msgr2=0x7febec078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:42.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.779+0000 7fec10454640 1 --2- 172.21.15.45:0/1632792742 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7febec0762d0 0x7febec078790 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fec04006fd0 tx=0x7fec04008040 comp rx=0 tx=0).stop 2024-05-08T05:51:42.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.779+0000 7fec10454640 1 -- 172.21.15.45:0/1632792742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec081629a0 msgr2=0x7fec081f53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:42.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.779+0000 7fec10454640 1 --2- 172.21.15.45:0/1632792742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec081629a0 0x7fec081f53d0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fec00009eb0 tx=0x7fec00004a40 comp rx=0 tx=0).stop 2024-05-08T05:51:42.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.779+0000 7fec10454640 1 -- 172.21.15.45:0/1632792742 shutdown_connections 2024-05-08T05:51:42.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.779+0000 7fec10454640 1 --2- 172.21.15.45:0/1632792742 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7febec0762d0 0x7febec078790 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.779+0000 7fec10454640 1 --2- 172.21.15.45:0/1632792742 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec080d2dc0 0x7fec081f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.779+0000 7fec10454640 1 --2- 172.21.15.45:0/1632792742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec081629a0 0x7fec081f53d0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.779+0000 7fec10454640 1 --2- 172.21.15.45:0/1632792742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec080d0cb0 0x7fec0815d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.779+0000 7fec10454640 1 -- 172.21.15.45:0/1632792742 >> 172.21.15.45:0/1632792742 conn(0x7fec080d4fe0 msgr2=0x7fec080cae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:42.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.779+0000 7fec10454640 1 -- 172.21.15.45:0/1632792742 shutdown_connections 2024-05-08T05:51:42.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.779+0000 7fec10454640 1 -- 172.21.15.45:0/1632792742 wait complete. 2024-05-08T05:51:42.793 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T05:51:42.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.952+0000 7f07eb6a0640 1 Processor -- start 2024-05-08T05:51:42.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.953+0000 7f07eb6a0640 1 -- start start 2024-05-08T05:51:42.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.953+0000 7f07eb6a0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f07e40d0c60 0x7f07e416a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:42.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.953+0000 7f07eb6a0640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07e4162f50 con 0x7f07e4162b70 2024-05-08T05:51:42.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.953+0000 7f07eb6a0640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07e40d3ee0 con 0x7f07e40b5a50 2024-05-08T05:51:42.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.953+0000 7f07eb6a0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f07e40d1f60 con 0x7f07e40d0c60 2024-05-08T05:51:42.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.954+0000 7f07e9c3d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f07e40d0c60 0x7f07e416a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:42.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.954+0000 7f07e9c3d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f07e40d0c60 0x7f07e416a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54344/0 (socket says 172.21.15.45:54344) 2024-05-08T05:51:42.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.954+0000 7f07e9c3d640 1 -- 172.21.15.45:0/3583548359 learned_addr learned my addr 172.21.15.45:0/3583548359 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:42.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.954+0000 7f07e943c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f07e4162b70 0x7f07e40d2f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38748/0 (socket says 172.21.15.45:38748) 2024-05-08T05:51:42.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.954+0000 7f07e9c3d640 1 -- 172.21.15.45:0/3583548359 >> v1:172.21.15.63:6789/0 conn(0x7f07e40b5a50 legacy=0x7f07e40d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:42.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.954+0000 7f07e9c3d640 1 -- 172.21.15.45:0/3583548359 >> v1:172.21.15.45:6789/0 conn(0x7f07e4162b70 legacy=0x7f07e40d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:42.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.954+0000 7f07e9c3d640 1 -- 172.21.15.45:0/3583548359 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e40d21b0 con 0x7f07e40d0c60 2024-05-08T05:51:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.955+0000 7f07e9c3d640 1 --2- 172.21.15.45:0/3583548359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e40d0c60 0x7f07e416a1e0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009870 tx=0x7f07dc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd73edf1b680e89 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.955+0000 7f07e3fff640 1 -- 172.21.15.45:0/3583548359 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc037070 con 0x7f07e40d0c60 2024-05-08T05:51:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.955+0000 7f07e3fff640 1 -- 172.21.15.45:0/3583548359 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07dc02ae50 con 0x7f07e40d0c60 2024-05-08T05:51:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.955+0000 7f07e3fff640 1 -- 172.21.15.45:0/3583548359 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc032860 con 0x7f07e40d0c60 2024-05-08T05:51:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.955+0000 7f07eb6a0640 1 -- 172.21.15.45:0/3583548359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e40d0c60 msgr2=0x7f07e416a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.955+0000 7f07eb6a0640 1 --2- 172.21.15.45:0/3583548359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e40d0c60 0x7f07e416a1e0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009870 tx=0x7f07dc02a300 comp rx=0 tx=0).stop 2024-05-08T05:51:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.955+0000 7f07eb6a0640 1 -- 172.21.15.45:0/3583548359 shutdown_connections 2024-05-08T05:51:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.955+0000 7f07eb6a0640 1 --2- 172.21.15.45:0/3583548359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e40d0c60 0x7f07e416a1e0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.955+0000 7f07eb6a0640 1 -- 172.21.15.45:0/3583548359 >> 172.21.15.45:0/3583548359 conn(0x7f07e40d5040 msgr2=0x7f07e40d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:42.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.956+0000 7f07eb6a0640 1 -- 172.21.15.45:0/3583548359 shutdown_connections 2024-05-08T05:51:42.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.956+0000 7f07eb6a0640 1 -- 172.21.15.45:0/3583548359 wait complete. 2024-05-08T05:51:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.957+0000 7f07eb6a0640 1 Processor -- start 2024-05-08T05:51:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.957+0000 7f07eb6a0640 1 -- start start 2024-05-08T05:51:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.957+0000 7f07eb6a0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07e40b5a50 0x7f07e40ba630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.957+0000 7f07eb6a0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07e40d0c60 0x7f07e415a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.957+0000 7f07eb6a0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e4162b70 0x7f07e415e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.957+0000 7f07eb6a0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f07e40b4170 con 0x7f07e4162b70 2024-05-08T05:51:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.957+0000 7f07eb6a0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f07e40d1090 con 0x7f07e40b5a50 2024-05-08T05:51:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.957+0000 7f07eb6a0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f07e40b42f0 con 0x7f07e40d0c60 2024-05-08T05:51:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.958+0000 7f07e9c3d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e4162b70 0x7f07e415e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.958+0000 7f07e9c3d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e4162b70 0x7f07e415e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54358/0 (socket says 172.21.15.45:54358) 2024-05-08T05:51:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.958+0000 7f07e9c3d640 1 -- 172.21.15.45:0/1868408854 learned_addr learned my addr 172.21.15.45:0/1868408854 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.958+0000 7f07e943c640 1 --2- 172.21.15.45:0/1868408854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07e40b5a50 0x7f07e40ba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:42.959 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.958+0000 7f07e8c3b640 1 --2- 172.21.15.45:0/1868408854 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07e40d0c60 0x7f07e415a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.958+0000 7f07e9c3d640 1 -- 172.21.15.45:0/1868408854 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07e40d0c60 msgr2=0x7f07e415a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.958+0000 7f07e9c3d640 1 --2- 172.21.15.45:0/1868408854 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07e40d0c60 0x7f07e415a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.958+0000 7f07e9c3d640 1 -- 172.21.15.45:0/1868408854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07e40b5a50 msgr2=0x7f07e40ba630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.958+0000 7f07e9c3d640 1 --2- 172.21.15.45:0/1868408854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07e40b5a50 0x7f07e40ba630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.958+0000 7f07e9c3d640 1 -- 172.21.15.45:0/1868408854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e40cd300 con 0x7f07e4162b70 2024-05-08T05:51:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.958+0000 7f07e9c3d640 1 --2- 172.21.15.45:0/1868408854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e4162b70 0x7f07e415e210 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009870 tx=0x7f07dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.958+0000 7f07e1ffb640 1 -- 172.21.15.45:0/1868408854 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc041070 con 0x7f07e4162b70 2024-05-08T05:51:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.958+0000 7f07e1ffb640 1 -- 172.21.15.45:0/1868408854 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07dc03c890 con 0x7f07e4162b70 2024-05-08T05:51:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.958+0000 7f07e1ffb640 1 -- 172.21.15.45:0/1868408854 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc037040 con 0x7f07e4162b70 2024-05-08T05:51:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.958+0000 7f07eb6a0640 1 -- 172.21.15.45:0/1868408854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e415c9b0 con 0x7f07e4162b70 2024-05-08T05:51:42.960 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.959+0000 7f07eb6a0640 1 -- 172.21.15.45:0/1868408854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e415cc90 con 0x7f07e4162b70 2024-05-08T05:51:42.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.960+0000 7f07eb6a0640 1 -- 172.21.15.45:0/1868408854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e40693f0 con 0x7f07e4162b70 2024-05-08T05:51:42.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.960+0000 7f07e1ffb640 1 -- 172.21.15.45:0/1868408854 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f07dc044070 con 0x7f07e4162b70 2024-05-08T05:51:42.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.961+0000 7f07e1ffb640 1 --2- 172.21.15.45:0/1868408854 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07d00762d0 0x7f07d0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:42.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.961+0000 7f07e1ffb640 1 -- 172.21.15.45:0/1868408854 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6359+0+0 (secure 0 0 0) 0x7f07dc0b81b0 con 0x7f07e4162b70 2024-05-08T05:51:42.961 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.961+0000 7f07e943c640 1 --2- 172.21.15.45:0/1868408854 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07d00762d0 0x7f07d0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:42.962 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.961+0000 7f07e943c640 1 --2- 172.21.15.45:0/1868408854 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07d00762d0 0x7f07d0078790 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f07d40012e0 tx=0x7f07d4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:42.965 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:42.965+0000 7f07e1ffb640 1 -- 172.21.15.45:0/1868408854 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f07dc081910 con 0x7f07e4162b70 2024-05-08T05:51:43.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:43.159+0000 7f07eb6a0640 1 -- 172.21.15.45:0/1868408854 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f07e40b29c0 con 0x7f07e4162b70 2024-05-08T05:51:44.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.008+0000 7f07e1ffb640 1 -- 172.21.15.45:0/1868408854 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v123) ==== 181+0+0 (secure 0 0 0) 0x7f07dc081210 con 0x7f07e4162b70 2024-05-08T05:51:44.009 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-05-08T05:51:44.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.012+0000 7f07eb6a0640 1 -- 172.21.15.45:0/1868408854 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07d00762d0 msgr2=0x7f07d0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:44.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.012+0000 7f07eb6a0640 1 --2- 172.21.15.45:0/1868408854 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07d00762d0 0x7f07d0078790 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f07d40012e0 tx=0x7f07d4009040 comp rx=0 tx=0).stop 2024-05-08T05:51:44.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.012+0000 7f07eb6a0640 1 -- 172.21.15.45:0/1868408854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e4162b70 msgr2=0x7f07e415e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:44.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.012+0000 7f07eb6a0640 1 --2- 172.21.15.45:0/1868408854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e4162b70 0x7f07e415e210 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009870 tx=0x7f07dc0046c0 comp rx=0 tx=0).stop 2024-05-08T05:51:44.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.013+0000 7f07eb6a0640 1 -- 172.21.15.45:0/1868408854 shutdown_connections 2024-05-08T05:51:44.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.013+0000 7f07eb6a0640 1 --2- 172.21.15.45:0/1868408854 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f07d00762d0 0x7f07d0078790 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:44.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.013+0000 7f07eb6a0640 1 --2- 172.21.15.45:0/1868408854 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f07e40d0c60 0x7f07e415a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:44.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.013+0000 7f07eb6a0640 1 --2- 172.21.15.45:0/1868408854 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f07e4162b70 0x7f07e415e210 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:44.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.013+0000 7f07eb6a0640 1 --2- 172.21.15.45:0/1868408854 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f07e40b5a50 0x7f07e40ba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:44.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.013+0000 7f07eb6a0640 1 -- 172.21.15.45:0/1868408854 >> 172.21.15.45:0/1868408854 conn(0x7f07e40d5040 msgr2=0x7f07e4161e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:44.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.013+0000 7f07eb6a0640 1 -- 172.21.15.45:0/1868408854 shutdown_connections 2024-05-08T05:51:44.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.014+0000 7f07eb6a0640 1 -- 172.21.15.45:0/1868408854 wait complete. 2024-05-08T05:51:44.028 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T05:51:44.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.143+0000 7f2b4250a640 1 Processor -- start 2024-05-08T05:51:44.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.143+0000 7f2b4250a640 1 -- start start 2024-05-08T05:51:44.146 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.144+0000 7f2b4250a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2b3c0b5a50 0x7f2b3c0d3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.144+0000 7f2b4250a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b3c0cfc50 0x7f2b3c0d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.144+0000 7f2b4250a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b3c162f50 con 0x7f2b3c162b70 2024-05-08T05:51:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.144+0000 7f2b4250a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f2b3c0d1f60 con 0x7f2b3c0b5a50 2024-05-08T05:51:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.144+0000 7f2b4250a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b3c0d3ee0 con 0x7f2b3c0cfc50 2024-05-08T05:51:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.144+0000 7f2b3b7fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f2b3c0b5a50 0x7f2b3c0d3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.144+0000 7f2b3bfff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f2b3c162b70 0x7f2b3c0d2f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38760/0 (socket says 172.21.15.45:38760) 2024-05-08T05:51:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.145+0000 7f2b3bfff640 1 -- 172.21.15.45:0/3962971224 learned_addr learned my addr 172.21.15.45:0/3962971224 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.145+0000 7f2b40aa7640 1 --2- 172.21.15.45:0/3962971224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b3c0cfc50 0x7f2b3c0d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.145+0000 7f2b3b7fe640 1 -- 172.21.15.45:0/3962971224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b3c0cfc50 msgr2=0x7f2b3c0d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.145+0000 7f2b3b7fe640 1 --2- 172.21.15.45:0/3962971224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b3c0cfc50 0x7f2b3c0d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.145+0000 7f2b3b7fe640 1 -- 172.21.15.45:0/3962971224 >> v1:172.21.15.45:6789/0 conn(0x7f2b3c162b70 legacy=0x7f2b3c0d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.145+0000 7f2b3b7fe640 1 -- 172.21.15.45:0/3962971224 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3c0d21b0 con 0x7f2b3c0b5a50 2024-05-08T05:51:44.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.145+0000 7f2b3affd640 1 -- 172.21.15.45:0/3962971224 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3576146438 0 0) 0x7f2b2c001110 con 0x7f2b3c162b70 2024-05-08T05:51:44.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.146+0000 7f2b3b7fe640 1 --2- 172.21.15.45:0/3962971224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c0b5a50 0x7f2b3c0d3650 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f2b300097a0 tx=0x7f2b3002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c61ac5cafa8b6a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:44.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.146+0000 7f2b3affd640 1 -- 172.21.15.45:0/3962971224 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b30037070 con 0x7f2b3c0b5a50 2024-05-08T05:51:44.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.146+0000 7f2b3affd640 1 -- 172.21.15.45:0/3962971224 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b30032440 con 0x7f2b3c0b5a50 2024-05-08T05:51:44.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.146+0000 7f2b3affd640 1 -- 172.21.15.45:0/3962971224 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b30032810 con 0x7f2b3c0b5a50 2024-05-08T05:51:44.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.146+0000 7f2b4250a640 1 -- 172.21.15.45:0/3962971224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c0b5a50 msgr2=0x7f2b3c0d3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:44.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.146+0000 7f2b4250a640 1 --2- 172.21.15.45:0/3962971224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c0b5a50 0x7f2b3c0d3650 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f2b300097a0 tx=0x7f2b3002a180 comp rx=0 tx=0).stop 2024-05-08T05:51:44.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.146+0000 7f2b4250a640 1 -- 172.21.15.45:0/3962971224 shutdown_connections 2024-05-08T05:51:44.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.146+0000 7f2b4250a640 1 --2- 172.21.15.45:0/3962971224 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b3c0cfc50 0x7f2b3c0d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:44.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.146+0000 7f2b4250a640 1 --2- 172.21.15.45:0/3962971224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c0b5a50 0x7f2b3c0d3650 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:44.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.146+0000 7f2b4250a640 1 -- 172.21.15.45:0/3962971224 >> 172.21.15.45:0/3962971224 conn(0x7f2b3c0d5040 msgr2=0x7f2b3c0d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:44.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.146+0000 7f2b4250a640 1 -- 172.21.15.45:0/3962971224 shutdown_connections 2024-05-08T05:51:44.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.146+0000 7f2b4250a640 1 -- 172.21.15.45:0/3962971224 wait complete. 2024-05-08T05:51:44.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.148+0000 7f2b4250a640 1 Processor -- start 2024-05-08T05:51:44.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.148+0000 7f2b4250a640 1 -- start start 2024-05-08T05:51:44.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b4250a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c0b5a50 0x7f2b3c1f2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:44.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b4250a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b3c0cfc50 0x7f2b3c14b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:44.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b4250a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b3c162b70 0x7f2b3c0cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:44.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b4250a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c0b4170 con 0x7f2b3c0b5a50 2024-05-08T05:51:44.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b4250a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c0d3b90 con 0x7f2b3c162b70 2024-05-08T05:51:44.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b4250a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f2b3c0b42f0 con 0x7f2b3c0cfc50 2024-05-08T05:51:44.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c0b5a50 0x7f2b3c1f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:44.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b3bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c0b5a50 0x7f2b3c1f2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54380/0 (socket says 172.21.15.45:54380) 2024-05-08T05:51:44.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b3bfff640 1 -- 172.21.15.45:0/629962680 learned_addr learned my addr 172.21.15.45:0/629962680 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:44.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b40aa7640 1 --2- 172.21.15.45:0/629962680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b3c162b70 0x7f2b3c0cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b3b7fe640 1 --2- 172.21.15.45:0/629962680 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b3c0cfc50 0x7f2b3c14b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b3bfff640 1 -- 172.21.15.45:0/629962680 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b3c0cfc50 msgr2=0x7f2b3c14b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b3bfff640 1 --2- 172.21.15.45:0/629962680 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b3c0cfc50 0x7f2b3c14b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b3bfff640 1 -- 172.21.15.45:0/629962680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b3c162b70 msgr2=0x7f2b3c0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b3bfff640 1 --2- 172.21.15.45:0/629962680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b3c162b70 0x7f2b3c0cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b3bfff640 1 -- 172.21.15.45:0/629962680 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3c14b980 con 0x7f2b3c0b5a50 2024-05-08T05:51:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b40aa7640 1 --2- 172.21.15.45:0/629962680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b3c162b70 0x7f2b3c0cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b3bfff640 1 --2- 172.21.15.45:0/629962680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c0b5a50 0x7f2b3c1f2380 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00c730 tx=0x7f2b2c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b38ff9640 1 -- 172.21.15.45:0/629962680 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c018070 con 0x7f2b3c0b5a50 2024-05-08T05:51:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b38ff9640 1 -- 172.21.15.45:0/629962680 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b2c0070d0 con 0x7f2b3c0b5a50 2024-05-08T05:51:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b4250a640 1 -- 172.21.15.45:0/629962680 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b3c14bc60 con 0x7f2b3c0b5a50 2024-05-08T05:51:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b38ff9640 1 -- 172.21.15.45:0/629962680 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c007500 con 0x7f2b3c0b5a50 2024-05-08T05:51:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.149+0000 7f2b4250a640 1 -- 172.21.15.45:0/629962680 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b3c1f28c0 con 0x7f2b3c0b5a50 2024-05-08T05:51:44.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.150+0000 7f2b4250a640 1 -- 172.21.15.45:0/629962680 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b3c0b9ff0 con 0x7f2b3c0b5a50 2024-05-08T05:51:44.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.151+0000 7f2b38ff9640 1 -- 172.21.15.45:0/629962680 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2b2c01b070 con 0x7f2b3c0b5a50 2024-05-08T05:51:44.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.151+0000 7f2b38ff9640 1 --2- 172.21.15.45:0/629962680 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b28076300 0x7f2b280787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:44.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.152+0000 7f2b38ff9640 1 -- 172.21.15.45:0/629962680 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5921+0+0 (secure 0 0 0) 0x7f2b2c0969a0 con 0x7f2b3c0b5a50 2024-05-08T05:51:44.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.152+0000 7f2b3b7fe640 1 --2- 172.21.15.45:0/629962680 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b28076300 0x7f2b280787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:44.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.152+0000 7f2b3b7fe640 1 --2- 172.21.15.45:0/629962680 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b28076300 0x7f2b280787c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2b30034000 tx=0x7f2b3002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:44.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.156+0000 7f2b38ff9640 1 -- 172.21.15.45:0/629962680 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2b3c162f50 con 0x7f2b3c0b5a50 2024-05-08T05:51:44.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:44.351+0000 7f2b4250a640 1 -- 172.21.15.45:0/629962680 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2b3c0693f0 con 0x7f2b3c0b5a50 2024-05-08T05:51:45.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.016+0000 7f2b38ff9640 1 -- 172.21.15.45:0/629962680 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v124) ==== 169+0+0 (secure 0 0 0) 0x7f2b2c0602b0 con 0x7f2b3c0b5a50 2024-05-08T05:51:45.017 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-05-08T05:51:45.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.021+0000 7f2b4250a640 1 -- 172.21.15.45:0/629962680 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b28076300 msgr2=0x7f2b280787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:45.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.021+0000 7f2b4250a640 1 --2- 172.21.15.45:0/629962680 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b28076300 0x7f2b280787c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2b30034000 tx=0x7f2b3002e000 comp rx=0 tx=0).stop 2024-05-08T05:51:45.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.021+0000 7f2b4250a640 1 -- 172.21.15.45:0/629962680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c0b5a50 msgr2=0x7f2b3c1f2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:45.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.021+0000 7f2b4250a640 1 --2- 172.21.15.45:0/629962680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c0b5a50 0x7f2b3c1f2380 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00c730 tx=0x7f2b2c00cbf0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.021+0000 7f2b4250a640 1 -- 172.21.15.45:0/629962680 shutdown_connections 2024-05-08T05:51:45.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.021+0000 7f2b4250a640 1 --2- 172.21.15.45:0/629962680 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f2b28076300 0x7f2b280787c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.021+0000 7f2b4250a640 1 --2- 172.21.15.45:0/629962680 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f2b3c0cfc50 0x7f2b3c14b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.021+0000 7f2b4250a640 1 --2- 172.21.15.45:0/629962680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b3c162b70 0x7f2b3c0cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.021+0000 7f2b4250a640 1 --2- 172.21.15.45:0/629962680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b3c0b5a50 0x7f2b3c1f2380 secure :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00c730 tx=0x7f2b2c00cbf0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.021+0000 7f2b4250a640 1 -- 172.21.15.45:0/629962680 >> 172.21.15.45:0/629962680 conn(0x7f2b3c0d5040 msgr2=0x7f2b3c0d1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:45.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.021+0000 7f2b4250a640 1 -- 172.21.15.45:0/629962680 shutdown_connections 2024-05-08T05:51:45.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.021+0000 7f2b4250a640 1 -- 172.21.15.45:0/629962680 wait complete. 2024-05-08T05:51:45.036 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-08T05:51:45.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.149+0000 7f8201178640 1 Processor -- start 2024-05-08T05:51:45.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.150+0000 7f8201178640 1 -- start start 2024-05-08T05:51:45.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.150+0000 7f8201178640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f81fc0d3cd0 0x7f81fc16a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:45.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.150+0000 7f8201178640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81fc0d1760 con 0x7f81fc0d0ec0 2024-05-08T05:51:45.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.150+0000 7f8201178640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81fc0d1a40 con 0x7f81fc15f380 2024-05-08T05:51:45.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.150+0000 7f8201178640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f81fc0d30f0 con 0x7f81fc0d3cd0 2024-05-08T05:51:45.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81fa59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f81fc0d0ec0 0x7f81fc0d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38770/0 (socket says 172.21.15.45:38770) 2024-05-08T05:51:45.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81fa59c640 1 -- 172.21.15.45:0/1849455385 learned_addr learned my addr 172.21.15.45:0/1849455385 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:45.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81f9d9b640 1 -- 172.21.15.45:0/1849455385 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3987821926 0 0) 0x7f81fc0d1760 con 0x7f81fc0d0ec0 2024-05-08T05:51:45.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81fb59e640 1 --2- 172.21.15.45:0/1849455385 >> v2:172.21.15.63:3300/0 conn(0x7f81fc0d3cd0 0x7f81fc16a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:45.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81f9d9b640 1 -- 172.21.15.45:0/1849455385 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81e8003520 con 0x7f81fc0d0ec0 2024-05-08T05:51:45.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81f9d9b640 1 -- 172.21.15.45:0/1849455385 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1942400220 0 0) 0x7f81e8003520 con 0x7f81fc0d0ec0 2024-05-08T05:51:45.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81f9d9b640 1 -- 172.21.15.45:0/1849455385 >> v2:172.21.15.63:3300/0 conn(0x7f81fc0d3cd0 msgr2=0x7f81fc16a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:45.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81f9d9b640 1 --2- 172.21.15.45:0/1849455385 >> v2:172.21.15.63:3300/0 conn(0x7f81fc0d3cd0 0x7f81fc16a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81f9d9b640 1 -- 172.21.15.45:0/1849455385 >> v1:172.21.15.63:6789/0 conn(0x7f81fc15f380 legacy=0x7f81fc0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:45.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81f9d9b640 1 -- 172.21.15.45:0/1849455385 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc0d1d20 con 0x7f81fc0d0ec0 2024-05-08T05:51:45.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81f9d9b640 1 -- 172.21.15.45:0/1849455385 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f81e40016f0 con 0x7f81fc0d0ec0 2024-05-08T05:51:45.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81f9d9b640 1 -- 172.21.15.45:0/1849455385 >> v1:172.21.15.45:6789/0 conn(0x7f81fc0d0ec0 legacy=0x7f81fc0d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:45.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81f9d9b640 1 --2- 172.21.15.45:0/1849455385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81e8003aa0 0x7f81e8003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:45.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81f9d9b640 1 --2- 172.21.15.45:0/1849455385 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81e8004800 0x7f81e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:45.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81f9d9b640 1 --2- 172.21.15.45:0/1849455385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e8005210 0x7f81e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81f9d9b640 1 -- 172.21.15.45:0/1849455385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc0d30f0 con 0x7f81e8003aa0 2024-05-08T05:51:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81f9d9b640 1 -- 172.21.15.45:0/1849455385 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f81e800c160 con 0x7f81e8005210 2024-05-08T05:51:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81f9d9b640 1 -- 172.21.15.45:0/1849455385 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f81e800c360 con 0x7f81e8004800 2024-05-08T05:51:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81fad9d640 1 --2- 172.21.15.45:0/1849455385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81e8003aa0 0x7f81e8003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81fb59e640 1 --2- 172.21.15.45:0/1849455385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e8005210 0x7f81e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.151+0000 7f81fa59c640 1 --2- 172.21.15.45:0/1849455385 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81e8004800 0x7f81e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.152+0000 7f81fad9d640 1 -- 172.21.15.45:0/1849455385 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81e8004800 msgr2=0x7f81e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.152+0000 7f81fad9d640 1 --2- 172.21.15.45:0/1849455385 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81e8004800 0x7f81e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.152+0000 7f81fad9d640 1 -- 172.21.15.45:0/1849455385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e8005210 msgr2=0x7f81e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.152+0000 7f81fad9d640 1 --2- 172.21.15.45:0/1849455385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e8005210 0x7f81e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.152+0000 7f81fad9d640 1 -- 172.21.15.45:0/1849455385 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e8003520 con 0x7f81e8003aa0 2024-05-08T05:51:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.152+0000 7f81fb59e640 1 -- 172.21.15.45:0/1849455385 reap_dead start 2024-05-08T05:51:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.152+0000 7f81fad9d640 1 --2- 172.21.15.45:0/1849455385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81e8003aa0 0x7f81e8003ec0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f81ec001e60 tx=0x7f81e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39835d0cd8c46d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:45.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.153+0000 7f81f9d9b640 1 -- 172.21.15.45:0/1849455385 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec001ab0 con 0x7f81e8003aa0 2024-05-08T05:51:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.153+0000 7f81f9d9b640 1 -- 172.21.15.45:0/1849455385 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81ec02adc0 con 0x7f81e8003aa0 2024-05-08T05:51:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.153+0000 7f81f9d9b640 1 -- 172.21.15.45:0/1849455385 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec003b70 con 0x7f81e8003aa0 2024-05-08T05:51:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.153+0000 7f8201178640 1 -- 172.21.15.45:0/1849455385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81e8003aa0 msgr2=0x7f81e8003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.153+0000 7f8201178640 1 --2- 172.21.15.45:0/1849455385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81e8003aa0 0x7f81e8003ec0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f81ec001e60 tx=0x7f81e8005830 comp rx=0 tx=0).stop 2024-05-08T05:51:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.153+0000 7f8201178640 1 -- 172.21.15.45:0/1849455385 shutdown_connections 2024-05-08T05:51:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.153+0000 7f8201178640 1 --2- 172.21.15.45:0/1849455385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81e8003aa0 0x7f81e8003ec0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.153+0000 7f8201178640 1 -- 172.21.15.45:0/1849455385 >> 172.21.15.45:0/1849455385 conn(0x7f81fc0d4fe0 msgr2=0x7f81fc0d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.153+0000 7f8201178640 1 -- 172.21.15.45:0/1849455385 shutdown_connections 2024-05-08T05:51:45.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.153+0000 7f8201178640 1 -- 172.21.15.45:0/1849455385 wait complete. 2024-05-08T05:51:45.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.155+0000 7f8201178640 1 Processor -- start 2024-05-08T05:51:45.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.155+0000 7f8201178640 1 -- start start 2024-05-08T05:51:45.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.155+0000 7f8201178640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e8003aa0 0x7f81fc1543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:45.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.155+0000 7f8201178640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81fc0ba330 0x7f81fc0cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:45.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.155+0000 7f8201178640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81fc0ba7b0 0x7f81fc0ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:45.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.155+0000 7f8201178640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc15f760 con 0x7f81fc0ba7b0 2024-05-08T05:51:45.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.155+0000 7f8201178640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f81fc0d2370 con 0x7f81e8003aa0 2024-05-08T05:51:45.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.155+0000 7f8201178640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f81fc0d2800 con 0x7f81fc0ba330 2024-05-08T05:51:45.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.156+0000 7f81fb59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81fc0ba7b0 0x7f81fc0ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:45.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.156+0000 7f81fb59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81fc0ba7b0 0x7f81fc0ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54402/0 (socket says 172.21.15.45:54402) 2024-05-08T05:51:45.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.156+0000 7f81fb59e640 1 -- 172.21.15.45:0/1051815402 learned_addr learned my addr 172.21.15.45:0/1051815402 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:45.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.156+0000 7f81fad9d640 1 --2- 172.21.15.45:0/1051815402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e8003aa0 0x7f81fc1543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:45.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.156+0000 7f81fa59c640 1 --2- 172.21.15.45:0/1051815402 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81fc0ba330 0x7f81fc0cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:45.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.156+0000 7f81fb59e640 1 -- 172.21.15.45:0/1051815402 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81fc0ba330 msgr2=0x7f81fc0cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:45.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.156+0000 7f81fb59e640 1 --2- 172.21.15.45:0/1051815402 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81fc0ba330 0x7f81fc0cdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.156+0000 7f81fb59e640 1 -- 172.21.15.45:0/1051815402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e8003aa0 msgr2=0x7f81fc1543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:45.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.156+0000 7f81fb59e640 1 --2- 172.21.15.45:0/1051815402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e8003aa0 0x7f81fc1543e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.156+0000 7f81fb59e640 1 -- 172.21.15.45:0/1051815402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81fc0a2d60 con 0x7f81fc0ba7b0 2024-05-08T05:51:45.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.156+0000 7f81fb59e640 1 --2- 172.21.15.45:0/1051815402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81fc0ba7b0 0x7f81fc0ce0a0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f81fc08f140 tx=0x7f81f4011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:45.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.156+0000 7f81f37fe640 1 -- 172.21.15.45:0/1051815402 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4004030 con 0x7f81fc0ba7b0 2024-05-08T05:51:45.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.156+0000 7f81f37fe640 1 -- 172.21.15.45:0/1051815402 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81f401b070 con 0x7f81fc0ba7b0 2024-05-08T05:51:45.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.156+0000 7f8201178640 1 -- 172.21.15.45:0/1051815402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81fc0a21e0 con 0x7f81fc0ba7b0 2024-05-08T05:51:45.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.156+0000 7f81f37fe640 1 -- 172.21.15.45:0/1051815402 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4004430 con 0x7f81fc0ba7b0 2024-05-08T05:51:45.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.156+0000 7f8201178640 1 -- 172.21.15.45:0/1051815402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81fc0a27a0 con 0x7f81fc0ba7b0 2024-05-08T05:51:45.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.157+0000 7f8201178640 1 -- 172.21.15.45:0/1051815402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81fc0693f0 con 0x7f81fc0ba7b0 2024-05-08T05:51:45.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.158+0000 7f81f37fe640 1 -- 172.21.15.45:0/1051815402 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f81f4012040 con 0x7f81fc0ba7b0 2024-05-08T05:51:45.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.158+0000 7f81f37fe640 1 --2- 172.21.15.45:0/1051815402 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f81dc076300 0x7f81dc0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:45.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.158+0000 7f81f37fe640 1 -- 172.21.15.45:0/1051815402 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f81f4097340 con 0x7f81fc0ba7b0 2024-05-08T05:51:45.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.159+0000 7f81fad9d640 1 --2- 172.21.15.45:0/1051815402 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f81dc076300 0x7f81dc0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:45.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.159+0000 7f81fad9d640 1 --2- 172.21.15.45:0/1051815402 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f81dc076300 0x7f81dc0787c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f81ec032000 tx=0x7f81ec02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:45.162 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.163+0000 7f81f37fe640 1 -- 172.21.15.45:0/1051815402 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f81f4060d50 con 0x7f81fc0ba7b0 2024-05-08T05:51:45.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.356+0000 7f8201178640 1 -- 172.21.15.45:0/1051815402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f81fc0ccaf0 con 0x7f81fc0ba7b0 2024-05-08T05:51:45.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.359+0000 7f81f37fe640 1 -- 172.21.15.45:0/1051815402 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f81f4060650 con 0x7f81fc0ba7b0 2024-05-08T05:51:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.362+0000 7f8201178640 1 -- 172.21.15.45:0/1051815402 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f81dc076300 msgr2=0x7f81dc0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.362+0000 7f8201178640 1 --2- 172.21.15.45:0/1051815402 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f81dc076300 0x7f81dc0787c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f81ec032000 tx=0x7f81ec02e000 comp rx=0 tx=0).stop 2024-05-08T05:51:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.362+0000 7f8201178640 1 -- 172.21.15.45:0/1051815402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81fc0ba7b0 msgr2=0x7f81fc0ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.362+0000 7f8201178640 1 --2- 172.21.15.45:0/1051815402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81fc0ba7b0 0x7f81fc0ce0a0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f81fc08f140 tx=0x7f81f4011cf0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.362+0000 7f8201178640 1 -- 172.21.15.45:0/1051815402 shutdown_connections 2024-05-08T05:51:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.362+0000 7f8201178640 1 --2- 172.21.15.45:0/1051815402 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f81dc076300 0x7f81dc0787c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.362+0000 7f8201178640 1 --2- 172.21.15.45:0/1051815402 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f81fc0ba330 0x7f81fc0cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.362+0000 7f8201178640 1 --2- 172.21.15.45:0/1051815402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f81fc0ba7b0 0x7f81fc0ce0a0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.362+0000 7f8201178640 1 --2- 172.21.15.45:0/1051815402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81e8003aa0 0x7f81fc1543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.362+0000 7f8201178640 1 -- 172.21.15.45:0/1051815402 >> 172.21.15.45:0/1051815402 conn(0x7f81fc0d4fe0 msgr2=0x7f81fc06ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:45.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.362+0000 7f8201178640 1 -- 172.21.15.45:0/1051815402 shutdown_connections 2024-05-08T05:51:45.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.362+0000 7f8201178640 1 -- 172.21.15.45:0/1051815402 wait complete. 2024-05-08T05:51:45.373 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-05-08T05:51:45.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.476+0000 7fa1b32a0640 1 Processor -- start 2024-05-08T05:51:45.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.477+0000 7fa1b32a0640 1 -- start start 2024-05-08T05:51:45.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.477+0000 7fa1b32a0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1ac15f420 0x7fa1ac0d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:45.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.477+0000 7fa1b32a0640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa1ac0d3ca0 0x7fa1ac0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:45.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.477+0000 7fa1b32a0640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1ac0d1eb0 0x7fa1ac0d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:45.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.477+0000 7fa1b32a0640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa1ac0d22f0 con 0x7fa1ac0d1eb0 2024-05-08T05:51:45.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.477+0000 7fa1b32a0640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa1ac1630b0 con 0x7fa1ac0d3ca0 2024-05-08T05:51:45.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.477+0000 7fa1b32a0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa1ac0d2550 con 0x7fa1ac15f420 2024-05-08T05:51:45.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.478+0000 7fa1b183d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1ac0d1eb0 0x7fa1ac0d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:45.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.478+0000 7fa1b183d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa1ac0d1eb0 0x7fa1ac0d4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54404/0 (socket says 172.21.15.45:54404) 2024-05-08T05:51:45.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.478+0000 7fa1b183d640 1 -- 172.21.15.45:0/3898114014 learned_addr learned my addr 172.21.15.45:0/3898114014 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:45.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.478+0000 7fa1b183d640 1 -- 172.21.15.45:0/3898114014 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1ac15f420 msgr2=0x7fa1ac0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:45.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.478+0000 7fa1b083b640 1 --2- 172.21.15.45:0/3898114014 >> v2:172.21.15.63:3300/0 conn(0x7fa1ac0d3ca0 0x7fa1ac0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:45.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.478+0000 7fa1b103c640 1 --2- 172.21.15.45:0/3898114014 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1ac15f420 0x7fa1ac0d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:45.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.478+0000 7fa1b183d640 1 --2- 172.21.15.45:0/3898114014 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1ac15f420 0x7fa1ac0d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.478+0000 7fa1b183d640 1 -- 172.21.15.45:0/3898114014 >> v2:172.21.15.63:3300/0 conn(0x7fa1ac0d3ca0 msgr2=0x7fa1ac0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:45.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.478+0000 7fa1b183d640 1 --2- 172.21.15.45:0/3898114014 >> v2:172.21.15.63:3300/0 conn(0x7fa1ac0d3ca0 0x7fa1ac0d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.478+0000 7fa1b183d640 1 -- 172.21.15.45:0/3898114014 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ac0d2c80 con 0x7fa1ac0d1eb0 2024-05-08T05:51:45.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.478+0000 7fa1b083b640 1 --2- 172.21.15.45:0/3898114014 >> v2:172.21.15.63:3300/0 conn(0x7fa1ac0d3ca0 0x7fa1ac0d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:45.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.478+0000 7fa1b103c640 1 --2- 172.21.15.45:0/3898114014 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1ac15f420 0x7fa1ac0d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:45.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.479+0000 7fa1b183d640 1 --2- 172.21.15.45:0/3898114014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ac0d1eb0 0x7fa1ac0d4cc0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0b7b90 tx=0x7fa1a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7badd331b53f85 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:45.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.479+0000 7fa1abfff640 1 -- 172.21.15.45:0/3898114014 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4037070 con 0x7fa1ac0d1eb0 2024-05-08T05:51:45.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.479+0000 7fa1abfff640 1 -- 172.21.15.45:0/3898114014 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a4032440 con 0x7fa1ac0d1eb0 2024-05-08T05:51:45.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.479+0000 7fa1abfff640 1 -- 172.21.15.45:0/3898114014 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4032810 con 0x7fa1ac0d1eb0 2024-05-08T05:51:45.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.479+0000 7fa1b32a0640 1 -- 172.21.15.45:0/3898114014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ac0d1eb0 msgr2=0x7fa1ac0d4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:45.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.479+0000 7fa1b32a0640 1 --2- 172.21.15.45:0/3898114014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ac0d1eb0 0x7fa1ac0d4cc0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0b7b90 tx=0x7fa1a402a1e0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.479+0000 7fa1b32a0640 1 -- 172.21.15.45:0/3898114014 shutdown_connections 2024-05-08T05:51:45.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.479+0000 7fa1b32a0640 1 --2- 172.21.15.45:0/3898114014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ac0d1eb0 0x7fa1ac0d4cc0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.479+0000 7fa1b32a0640 1 --2- 172.21.15.45:0/3898114014 >> v2:172.21.15.63:3300/0 conn(0x7fa1ac0d3ca0 0x7fa1ac0d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.479+0000 7fa1b32a0640 1 --2- 172.21.15.45:0/3898114014 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1ac15f420 0x7fa1ac0d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.479+0000 7fa1b32a0640 1 -- 172.21.15.45:0/3898114014 >> 172.21.15.45:0/3898114014 conn(0x7fa1ac08eac0 msgr2=0x7fa1ac09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:45.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.480+0000 7fa1b32a0640 1 -- 172.21.15.45:0/3898114014 shutdown_connections 2024-05-08T05:51:45.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.480+0000 7fa1b32a0640 1 -- 172.21.15.45:0/3898114014 wait complete. 2024-05-08T05:51:45.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.481+0000 7fa1b32a0640 1 Processor -- start 2024-05-08T05:51:45.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.481+0000 7fa1b32a0640 1 -- start start 2024-05-08T05:51:45.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.481+0000 7fa1b32a0640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ac0d1eb0 0x7fa1ac15a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:45.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.481+0000 7fa1b32a0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1ac0c9ea0 0x7fa1ac0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:45.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.481+0000 7fa1b32a0640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1ac0ca320 0x7fa1ac0ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:45.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.481+0000 7fa1b32a0640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac0d2830 con 0x7fa1ac0d1eb0 2024-05-08T05:51:45.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.481+0000 7fa1b32a0640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ac0d3240 con 0x7fa1ac0ca320 2024-05-08T05:51:45.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.481+0000 7fa1b32a0640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa1ac0d5200 con 0x7fa1ac0c9ea0 2024-05-08T05:51:45.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.482+0000 7fa1b103c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ac0d1eb0 0x7fa1ac15a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:45.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.482+0000 7fa1b103c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ac0d1eb0 0x7fa1ac15a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54406/0 (socket says 172.21.15.45:54406) 2024-05-08T05:51:45.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.482+0000 7fa1b103c640 1 -- 172.21.15.45:0/1028694007 learned_addr learned my addr 172.21.15.45:0/1028694007 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:45.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.482+0000 7fa1b183d640 1 --2- 172.21.15.45:0/1028694007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1ac0ca320 0x7fa1ac0ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:45.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.482+0000 7fa1b083b640 1 --2- 172.21.15.45:0/1028694007 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1ac0c9ea0 0x7fa1ac0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.482+0000 7fa1b103c640 1 -- 172.21.15.45:0/1028694007 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1ac0c9ea0 msgr2=0x7fa1ac0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.482+0000 7fa1b103c640 1 --2- 172.21.15.45:0/1028694007 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1ac0c9ea0 0x7fa1ac0ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.482+0000 7fa1b103c640 1 -- 172.21.15.45:0/1028694007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1ac0ca320 msgr2=0x7fa1ac0ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.482+0000 7fa1b103c640 1 --2- 172.21.15.45:0/1028694007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1ac0ca320 0x7fa1ac0ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.482+0000 7fa1b103c640 1 -- 172.21.15.45:0/1028694007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ac0c8b20 con 0x7fa1ac0d1eb0 2024-05-08T05:51:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.482+0000 7fa1b103c640 1 --2- 172.21.15.45:0/1028694007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ac0d1eb0 0x7fa1ac15a0b0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00ce30 tx=0x7fa19c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.482+0000 7fa1a9ffb640 1 -- 172.21.15.45:0/1028694007 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c00f890 con 0x7fa1ac0d1eb0 2024-05-08T05:51:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.482+0000 7fa1a9ffb640 1 -- 172.21.15.45:0/1028694007 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa19c00fa30 con 0x7fa1ac0d1eb0 2024-05-08T05:51:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.482+0000 7fa1b32a0640 1 -- 172.21.15.45:0/1028694007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ac0c8e00 con 0x7fa1ac0d1eb0 2024-05-08T05:51:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.482+0000 7fa1a9ffb640 1 -- 172.21.15.45:0/1028694007 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c00fe00 con 0x7fa1ac0d1eb0 2024-05-08T05:51:45.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.482+0000 7fa1b32a0640 1 -- 172.21.15.45:0/1028694007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ac0c93c0 con 0x7fa1ac0d1eb0 2024-05-08T05:51:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.483+0000 7fa1b32a0640 1 -- 172.21.15.45:0/1028694007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1ac0693f0 con 0x7fa1ac0d1eb0 2024-05-08T05:51:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.484+0000 7fa1a9ffb640 1 -- 172.21.15.45:0/1028694007 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa19c018400 con 0x7fa1ac0d1eb0 2024-05-08T05:51:45.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.485+0000 7fa1a9ffb640 1 --2- 172.21.15.45:0/1028694007 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1980762d0 0x7fa198078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.485+0000 7fa1a9ffb640 1 -- 172.21.15.45:0/1028694007 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7fa19c098390 con 0x7fa1ac0d1eb0 2024-05-08T05:51:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.485+0000 7fa1b083b640 1 --2- 172.21.15.45:0/1028694007 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1980762d0 0x7fa198078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:45.487 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.485+0000 7fa1b083b640 1 --2- 172.21.15.45:0/1028694007 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1980762d0 0x7fa198078790 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac06e200 tx=0x7fa1a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:45.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.489+0000 7fa1a9ffb640 1 -- 172.21.15.45:0/1028694007 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa19c014030 con 0x7fa1ac0d1eb0 2024-05-08T05:51:45.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:45.760+0000 7fa1b32a0640 1 -- 172.21.15.45:0/1028694007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7fa1ac0d4fe0 con 0x7fa1ac0d1eb0 2024-05-08T05:51:46.018 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:46.018+0000 7fa1a9ffb640 1 -- 172.21.15.45:0/1028694007 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v474) ==== 144+0+0 (secure 0 0 0) 0x7fa19c00ae20 con 0x7fa1ac0d1eb0 2024-05-08T05:51:46.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:46.020+0000 7fa1b32a0640 1 -- 172.21.15.45:0/1028694007 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1980762d0 msgr2=0x7fa198078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:46.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:46.020+0000 7fa1b32a0640 1 --2- 172.21.15.45:0/1028694007 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1980762d0 0x7fa198078790 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac06e200 tx=0x7fa1a0008040 comp rx=0 tx=0).stop 2024-05-08T05:51:46.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:46.020+0000 7fa1b32a0640 1 -- 172.21.15.45:0/1028694007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ac0d1eb0 msgr2=0x7fa1ac15a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:46.020+0000 7fa1b32a0640 1 --2- 172.21.15.45:0/1028694007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ac0d1eb0 0x7fa1ac15a0b0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00ce30 tx=0x7fa19c00a650 comp rx=0 tx=0).stop 2024-05-08T05:51:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:46.020+0000 7fa1b32a0640 1 -- 172.21.15.45:0/1028694007 shutdown_connections 2024-05-08T05:51:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:46.020+0000 7fa1b32a0640 1 --2- 172.21.15.45:0/1028694007 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa1980762d0 0x7fa198078790 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:46.020+0000 7fa1b32a0640 1 --2- 172.21.15.45:0/1028694007 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa1ac0c9ea0 0x7fa1ac0ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:46.020+0000 7fa1b32a0640 1 --2- 172.21.15.45:0/1028694007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1ac0ca320 0x7fa1ac0ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:46.020+0000 7fa1b32a0640 1 --2- 172.21.15.45:0/1028694007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa1ac0d1eb0 0x7fa1ac15a0b0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:46.020+0000 7fa1b32a0640 1 -- 172.21.15.45:0/1028694007 >> 172.21.15.45:0/1028694007 conn(0x7fa1ac08eac0 msgr2=0x7fa1ac06d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:46.020+0000 7fa1b32a0640 1 -- 172.21.15.45:0/1028694007 shutdown_connections 2024-05-08T05:51:46.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:46.020+0000 7fa1b32a0640 1 -- 172.21.15.45:0/1028694007 wait complete. 2024-05-08T05:51:46.031 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-05-08T05:51:46.032 INFO:tasks.cephfs_test_runner: 2024-05-08T05:51:46.032 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-08T05:51:46.032 INFO:tasks.cephfs_test_runner:Ran 7 tests in 584.521s 2024-05-08T05:51:46.032 INFO:tasks.cephfs_test_runner: 2024-05-08T05:51:46.032 INFO:tasks.cephfs_test_runner:OK 2024-05-08T05:51:46.033 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-08T05:51:46.042 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-08T05:51:46.146 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-08T05:51:46.146 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:51:46.146 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:51:46.164 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:51:46.164 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-08T05:51:46.164 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:51:46.221 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057103,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3276 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249452k,nr_inodes=812363,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498912k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-08T05:51:46.221 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:51:46.221 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:51:46.241 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:51:46.242 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-08T05:51:46.242 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:51:46.301 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056334,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15642 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497680k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-08T05:51:46.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:46.301 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:51:46.318 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-08T05:51:46.319 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T05:51:46.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:46.319 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-08T05:51:46.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:46.387 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:51:46.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:46.455 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-05-08T05:51:46.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:46.550 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-05-08T05:51:46.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:46.566 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-05-08T05:51:46.635 INFO:teuthology.orchestra.run.smithi045.stderr:Cannot find device "ceph-brx" 2024-05-08T05:51:46.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:51:46.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:46.638 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-08T05:51:46.659 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-08T05:51:46.659 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-08T05:51:46.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:46.660 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-08T05:51:46.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:46.730 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-08T05:51:46.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:46.800 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link delete ceph-brx 2024-05-08T05:51:46.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:46.897 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-05-08T05:51:46.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:51:46.913 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link delete ceph-brx 2024-05-08T05:51:46.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:51:46.986 INFO:teuthology.orchestra.run.smithi063.stderr:Cannot find device "ceph-brx" 2024-05-08T05:51:46.986 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-08T05:51:46.998 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T05:51:46.998 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:51:47.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.107+0000 7fef9894e640 1 Processor -- start 2024-05-08T05:51:47.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.107+0000 7fef9894e640 1 -- start start 2024-05-08T05:51:47.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.108+0000 7fef9894e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef941524f0 0x7fef940b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:47.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.108+0000 7fef9894e640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fef9415bc80 0x7fef9406e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:47.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.108+0000 7fef9894e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef9406e8d0 0x7fef94073700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:47.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.108+0000 7fef9894e640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fef940b51a0 con 0x7fef9406e8d0 2024-05-08T05:51:47.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.108+0000 7fef9894e640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fef940b4590 con 0x7fef9415bc80 2024-05-08T05:51:47.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.108+0000 7fef9894e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fef940b4870 con 0x7fef941524f0 2024-05-08T05:51:47.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.108+0000 7fef92d9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef9406e8d0 0x7fef94073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:47.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.108+0000 7fef92d9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fef9406e8d0 0x7fef94073700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54410/0 (socket says 172.21.15.45:54410) 2024-05-08T05:51:47.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.108+0000 7fef92d9d640 1 -- 172.21.15.45:0/4112128219 learned_addr learned my addr 172.21.15.45:0/4112128219 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:47.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.108+0000 7fef89d9b640 1 --2- 172.21.15.45:0/4112128219 >> v2:172.21.15.63:3300/0 conn(0x7fef9415bc80 0x7fef9406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:47.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.108+0000 7fef9259c640 1 --2- 172.21.15.45:0/4112128219 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef941524f0 0x7fef940b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:47.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.108+0000 7fef92d9d640 1 -- 172.21.15.45:0/4112128219 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef941524f0 msgr2=0x7fef940b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:47.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.108+0000 7fef92d9d640 1 --2- 172.21.15.45:0/4112128219 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef941524f0 0x7fef940b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.108+0000 7fef92d9d640 1 -- 172.21.15.45:0/4112128219 >> v2:172.21.15.63:3300/0 conn(0x7fef9415bc80 msgr2=0x7fef9406e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:47.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.108+0000 7fef92d9d640 1 --2- 172.21.15.45:0/4112128219 >> v2:172.21.15.63:3300/0 conn(0x7fef9415bc80 0x7fef9406e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.108+0000 7fef92d9d640 1 -- 172.21.15.45:0/4112128219 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef940b4b50 con 0x7fef9406e8d0 2024-05-08T05:51:47.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.109+0000 7fef92d9d640 1 --2- 172.21.15.45:0/4112128219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9406e8d0 0x7fef94073700 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fef940d14e0 tx=0x7fef8c02c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f37313c24274732 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:47.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.109+0000 7fef91d9b640 1 -- 172.21.15.45:0/4112128219 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c00b2e0 con 0x7fef9406e8d0 2024-05-08T05:51:47.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.109+0000 7fef91d9b640 1 -- 172.21.15.45:0/4112128219 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef8c009600 con 0x7fef9406e8d0 2024-05-08T05:51:47.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.109+0000 7fef91d9b640 1 -- 172.21.15.45:0/4112128219 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c00b480 con 0x7fef9406e8d0 2024-05-08T05:51:47.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.110+0000 7fef9894e640 1 -- 172.21.15.45:0/4112128219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9406e8d0 msgr2=0x7fef94073700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:47.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.110+0000 7fef9894e640 1 --2- 172.21.15.45:0/4112128219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9406e8d0 0x7fef94073700 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fef940d14e0 tx=0x7fef8c02c8a0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.110+0000 7fef9894e640 1 -- 172.21.15.45:0/4112128219 shutdown_connections 2024-05-08T05:51:47.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.110+0000 7fef9894e640 1 --2- 172.21.15.45:0/4112128219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9406e8d0 0x7fef94073700 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.110+0000 7fef9894e640 1 --2- 172.21.15.45:0/4112128219 >> v2:172.21.15.63:3300/0 conn(0x7fef9415bc80 0x7fef9406e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.110+0000 7fef9894e640 1 --2- 172.21.15.45:0/4112128219 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef941524f0 0x7fef940b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.110+0000 7fef9894e640 1 -- 172.21.15.45:0/4112128219 >> 172.21.15.45:0/4112128219 conn(0x7fef9406c3a0 msgr2=0x7fef9406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:47.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.110+0000 7fef9894e640 1 -- 172.21.15.45:0/4112128219 shutdown_connections 2024-05-08T05:51:47.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.110+0000 7fef9894e640 1 -- 172.21.15.45:0/4112128219 wait complete. 2024-05-08T05:51:47.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.111+0000 7fef9894e640 1 Processor -- start 2024-05-08T05:51:47.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.112+0000 7fef9894e640 1 -- start start 2024-05-08T05:51:47.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.112+0000 7fef9894e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9406e8d0 0x7fef94147630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:47.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.112+0000 7fef9894e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef941524f0 0x7fef940cda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:47.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.112+0000 7fef9894e640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef9415bc80 0x7fef9414a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:47.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.112+0000 7fef9894e640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fef940b5630 con 0x7fef9406e8d0 2024-05-08T05:51:47.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.112+0000 7fef9894e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fef941528d0 con 0x7fef941524f0 2024-05-08T05:51:47.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.112+0000 7fef9894e640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fef94148280 con 0x7fef9415bc80 2024-05-08T05:51:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.112+0000 7fef9259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9406e8d0 0x7fef94147630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.112+0000 7fef9259c640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9406e8d0 0x7fef94147630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54416/0 (socket says 172.21.15.45:54416) 2024-05-08T05:51:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.112+0000 7fef9259c640 1 -- 172.21.15.45:0/1150076072 learned_addr learned my addr 172.21.15.45:0/1150076072 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.113+0000 7fef92d9d640 1 --2- 172.21.15.45:0/1150076072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef9415bc80 0x7fef9414a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.113+0000 7fef89d9b640 1 --2- 172.21.15.45:0/1150076072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef941524f0 0x7fef940cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.113+0000 7fef9259c640 1 -- 172.21.15.45:0/1150076072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef9415bc80 msgr2=0x7fef9414a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.113+0000 7fef9259c640 1 --2- 172.21.15.45:0/1150076072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef9415bc80 0x7fef9414a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.113+0000 7fef9259c640 1 -- 172.21.15.45:0/1150076072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef941524f0 msgr2=0x7fef940cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.113+0000 7fef9259c640 1 --2- 172.21.15.45:0/1150076072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef941524f0 0x7fef940cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.113+0000 7fef9259c640 1 -- 172.21.15.45:0/1150076072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef94148560 con 0x7fef9406e8d0 2024-05-08T05:51:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.113+0000 7fef89d9b640 1 --2- 172.21.15.45:0/1150076072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef941524f0 0x7fef940cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.113+0000 7fef9259c640 1 --2- 172.21.15.45:0/1150076072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9406e8d0 0x7fef94147630 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fef8400d800 tx=0x7fef8400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.113+0000 7fef8b7fe640 1 -- 172.21.15.45:0/1150076072 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84018020 con 0x7fef9406e8d0 2024-05-08T05:51:47.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.113+0000 7fef8b7fe640 1 -- 172.21.15.45:0/1150076072 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef840096a0 con 0x7fef9406e8d0 2024-05-08T05:51:47.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.113+0000 7fef9894e640 1 -- 172.21.15.45:0/1150076072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef94148840 con 0x7fef9406e8d0 2024-05-08T05:51:47.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.113+0000 7fef8b7fe640 1 -- 172.21.15.45:0/1150076072 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84009a70 con 0x7fef9406e8d0 2024-05-08T05:51:47.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.113+0000 7fef9894e640 1 -- 172.21.15.45:0/1150076072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef940ce400 con 0x7fef9406e8d0 2024-05-08T05:51:47.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.114+0000 7fef9894e640 1 -- 172.21.15.45:0/1150076072 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef940693f0 con 0x7fef9406e8d0 2024-05-08T05:51:47.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.115+0000 7fef8b7fe640 1 -- 172.21.15.45:0/1150076072 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fef8401b020 con 0x7fef9406e8d0 2024-05-08T05:51:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.115+0000 7fef8b7fe640 1 --2- 172.21.15.45:0/1150076072 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef740762d0 0x7fef74078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.115+0000 7fef8b7fe640 1 -- 172.21.15.45:0/1150076072 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7fef84097bf0 con 0x7fef9406e8d0 2024-05-08T05:51:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.116+0000 7fef89d9b640 1 --2- 172.21.15.45:0/1150076072 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef740762d0 0x7fef74078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:47.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.116+0000 7fef89d9b640 1 --2- 172.21.15.45:0/1150076072 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef740762d0 0x7fef74078790 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fef7c006fd0 tx=0x7fef7c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:47.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.120+0000 7fef8b7fe640 1 -- 172.21.15.45:0/1150076072 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fef84022020 con 0x7fef9406e8d0 2024-05-08T05:51:47.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.307+0000 7fef9894e640 1 -- 172.21.15.45:0/1150076072 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef94072220 con 0x7fef740762d0 2024-05-08T05:51:47.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.308+0000 7fef8b7fe640 1 -- 172.21.15.45:0/1150076072 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13399 (secure 0 0 0) 0x7fef94072220 con 0x7fef740762d0 2024-05-08T05:51:47.308 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:47.308 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:51:47.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.310+0000 7fef9894e640 1 -- 172.21.15.45:0/1150076072 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef740762d0 msgr2=0x7fef74078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:47.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.310+0000 7fef9894e640 1 --2- 172.21.15.45:0/1150076072 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef740762d0 0x7fef74078790 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fef7c006fd0 tx=0x7fef7c008040 comp rx=0 tx=0).stop 2024-05-08T05:51:47.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.310+0000 7fef9894e640 1 -- 172.21.15.45:0/1150076072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9406e8d0 msgr2=0x7fef94147630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:47.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.310+0000 7fef9894e640 1 --2- 172.21.15.45:0/1150076072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9406e8d0 0x7fef94147630 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fef8400d800 tx=0x7fef8400dcc0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.310+0000 7fef9894e640 1 -- 172.21.15.45:0/1150076072 shutdown_connections 2024-05-08T05:51:47.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.310+0000 7fef9894e640 1 --2- 172.21.15.45:0/1150076072 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fef740762d0 0x7fef74078790 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.310+0000 7fef9894e640 1 --2- 172.21.15.45:0/1150076072 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fef9415bc80 0x7fef9414a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.310+0000 7fef9894e640 1 --2- 172.21.15.45:0/1150076072 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fef941524f0 0x7fef940cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.310+0000 7fef9894e640 1 --2- 172.21.15.45:0/1150076072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fef9406e8d0 0x7fef94147630 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.310+0000 7fef9894e640 1 -- 172.21.15.45:0/1150076072 >> 172.21.15.45:0/1150076072 conn(0x7fef9406c3a0 msgr2=0x7fef940ade70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:47.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.310+0000 7fef9894e640 1 -- 172.21.15.45:0/1150076072 shutdown_connections 2024-05-08T05:51:47.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.310+0000 7fef9894e640 1 -- 172.21.15.45:0/1150076072 wait complete. 2024-05-08T05:51:47.321 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-05-08T05:51:46.984195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278896,"kb_used_data":29460,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695824,"statfs":{"total":773094113280,"available":772808523776,"internally_reserved":0,"allocated":30167040,"data_stored":23632035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390399,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-267,"num_write":-170,"num_write_kb":-8299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001284"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":301,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-05-08T05:51:44.888547+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:51:44.888547+0000","last_peered":"2024-05-08T05:51:44.888547+0000","last_clean":"2024-05-08T05:51:44.888547+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:51:44.888547+0000","last_undegraded":"2024-05-08T05:51:44.888547+0000","last_fullsized":"2024-05-08T05:51:44.888547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542273,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33260,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338580,"statfs":{"total":96636764160,"available":96602705920,"internally_reserved":0,"allocated":2129920,"data_stored":1300759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542274,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36760,"kb_used_data":5580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335080,"statfs":{"total":96636764160,"available":96599121920,"internally_reserved":0,"allocated":5713920,"data_stored":4904494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542273,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":5584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":5718016,"data_stored":4904072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574976,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36776,"kb_used_data":5596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335064,"statfs":{"total":96636764160,"available":96599105536,"internally_reserved":0,"allocated":5730304,"data_stored":4905041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574975,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1531904,"data_stored":710759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574975,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1519616,"data_stored":709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":5572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":5705728,"data_stored":4897294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33248,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338592,"statfs":{"total":96636764160,"available":96602718208,"internally_reserved":0,"allocated":2117632,"data_stored":1300003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:51:47.321 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:51:47.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.426+0000 7f1495ea7640 1 Processor -- start 2024-05-08T05:51:47.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.426+0000 7f1495ea7640 1 -- start start 2024-05-08T05:51:47.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.426+0000 7f1495ea7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f14900b21f0 0x7f149006e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:47.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.426+0000 7f1495ea7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1490162f50 0x7f14901633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:47.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.427+0000 7f1495ea7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14900d0500 con 0x7f14900cf790 2024-05-08T05:51:47.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.427+0000 7f1495ea7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f14901638f0 con 0x7f14900b21f0 2024-05-08T05:51:47.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.427+0000 7f1495ea7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f149006d810 con 0x7f1490162f50 2024-05-08T05:51:47.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.427+0000 7f148effd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1490162f50 0x7f14901633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:47.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.427+0000 7f148f7fe640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f14900b21f0 0x7f149006e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:47.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.427+0000 7f148effd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1490162f50 0x7f14901633b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:35070/0 (socket says 172.21.15.45:35070) 2024-05-08T05:51:47.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.427+0000 7f148ffff640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f14900cf790 0x7f14900cfc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:51718/0 (socket says 172.21.15.45:51718) 2024-05-08T05:51:47.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.427+0000 7f148effd640 1 -- 172.21.15.45:0/1247295748 learned_addr learned my addr 172.21.15.45:0/1247295748 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:47.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.427+0000 7f148effd640 1 -- 172.21.15.45:0/1247295748 >> v2:172.21.15.63:3300/0 conn(0x7f14900b21f0 msgr2=0x7f149006e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:47.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.427+0000 7f148effd640 1 --2- 172.21.15.45:0/1247295748 >> v2:172.21.15.63:3300/0 conn(0x7f14900b21f0 0x7f149006e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.427+0000 7f148effd640 1 -- 172.21.15.45:0/1247295748 >> v1:172.21.15.63:6789/0 conn(0x7f14900cf790 legacy=0x7f14900cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:47.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.427+0000 7f148effd640 1 -- 172.21.15.45:0/1247295748 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149006da60 con 0x7f1490162f50 2024-05-08T05:51:47.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.427+0000 7f148f7fe640 1 --2- 172.21.15.45:0/1247295748 >> v2:172.21.15.63:3300/0 conn(0x7f14900b21f0 0x7f149006e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:47.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.428+0000 7f148effd640 1 --2- 172.21.15.45:0/1247295748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1490162f50 0x7f14901633b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1484009870 tx=0x7f148402a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6af162686f23a1a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:47.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.428+0000 7f148e7fc640 1 -- 172.21.15.45:0/1247295748 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484037070 con 0x7f1490162f50 2024-05-08T05:51:47.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.428+0000 7f148e7fc640 1 -- 172.21.15.45:0/1247295748 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1484032440 con 0x7f1490162f50 2024-05-08T05:51:47.429 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.428+0000 7f148e7fc640 1 -- 172.21.15.45:0/1247295748 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484032810 con 0x7f1490162f50 2024-05-08T05:51:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.429+0000 7f1495ea7640 1 -- 172.21.15.45:0/1247295748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1490162f50 msgr2=0x7f14901633b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.429+0000 7f1495ea7640 1 --2- 172.21.15.45:0/1247295748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1490162f50 0x7f14901633b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1484009870 tx=0x7f148402a130 comp rx=0 tx=0).stop 2024-05-08T05:51:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.429+0000 7f1495ea7640 1 -- 172.21.15.45:0/1247295748 shutdown_connections 2024-05-08T05:51:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.429+0000 7f1495ea7640 1 --2- 172.21.15.45:0/1247295748 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1490162f50 0x7f14901633b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.429+0000 7f1495ea7640 1 --2- 172.21.15.45:0/1247295748 >> v2:172.21.15.63:3300/0 conn(0x7f14900b21f0 0x7f149006e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.429+0000 7f1495ea7640 1 -- 172.21.15.45:0/1247295748 >> 172.21.15.45:0/1247295748 conn(0x7f149009ba40 msgr2=0x7f149009be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.429+0000 7f1495ea7640 1 -- 172.21.15.45:0/1247295748 shutdown_connections 2024-05-08T05:51:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.429+0000 7f1495ea7640 1 -- 172.21.15.45:0/1247295748 wait complete. 2024-05-08T05:51:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.430+0000 7f1495ea7640 1 Processor -- start 2024-05-08T05:51:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.430+0000 7f1495ea7640 1 -- start start 2024-05-08T05:51:47.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.431+0000 7f1495ea7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14900b21f0 0x7f14901f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.431+0000 7f1495ea7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14900cf790 0x7f14900b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.431+0000 7f1495ea7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1490162f50 0x7f14900b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.431+0000 7f1495ea7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f149006ea80 con 0x7f1490162f50 2024-05-08T05:51:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.431+0000 7f1495ea7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f149006e0b0 con 0x7f14900cf790 2024-05-08T05:51:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.431+0000 7f1495ea7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f14900b25d0 con 0x7f14900b21f0 2024-05-08T05:51:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.431+0000 7f148ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1490162f50 0x7f14900b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.431+0000 7f148ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1490162f50 0x7f14900b07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54424/0 (socket says 172.21.15.45:54424) 2024-05-08T05:51:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.431+0000 7f148ffff640 1 -- 172.21.15.45:0/4212419022 learned_addr learned my addr 172.21.15.45:0/4212419022 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.431+0000 7f148f7fe640 1 --2- 172.21.15.45:0/4212419022 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14900b21f0 0x7f14901f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.431+0000 7f148effd640 1 --2- 172.21.15.45:0/4212419022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14900cf790 0x7f14900b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.431+0000 7f148ffff640 1 -- 172.21.15.45:0/4212419022 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14900b21f0 msgr2=0x7f14901f45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.431+0000 7f148ffff640 1 --2- 172.21.15.45:0/4212419022 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14900b21f0 0x7f14901f45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.431+0000 7f148ffff640 1 -- 172.21.15.45:0/4212419022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14900cf790 msgr2=0x7f14900b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:47.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.431+0000 7f148ffff640 1 --2- 172.21.15.45:0/4212419022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14900cf790 0x7f14900b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.431+0000 7f148ffff640 1 -- 172.21.15.45:0/4212419022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14901f4b20 con 0x7f1490162f50 2024-05-08T05:51:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.431+0000 7f148f7fe640 1 --2- 172.21.15.45:0/4212419022 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14900b21f0 0x7f14901f45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.431+0000 7f148ffff640 1 --2- 172.21.15.45:0/4212419022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1490162f50 0x7f14900b07f0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f148800c500 tx=0x7f148800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.432+0000 7f1477fff640 1 -- 172.21.15.45:0/4212419022 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1488018070 con 0x7f1490162f50 2024-05-08T05:51:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.432+0000 7f1477fff640 1 -- 172.21.15.45:0/4212419022 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1488007500 con 0x7f1490162f50 2024-05-08T05:51:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.432+0000 7f1495ea7640 1 -- 172.21.15.45:0/4212419022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14900b41e0 con 0x7f1490162f50 2024-05-08T05:51:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.432+0000 7f1477fff640 1 -- 172.21.15.45:0/4212419022 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14880076b0 con 0x7f1490162f50 2024-05-08T05:51:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.432+0000 7f1495ea7640 1 -- 172.21.15.45:0/4212419022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14900b47a0 con 0x7f1490162f50 2024-05-08T05:51:47.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.433+0000 7f1495ea7640 1 -- 172.21.15.45:0/4212419022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1490069b00 con 0x7f1490162f50 2024-05-08T05:51:47.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.433+0000 7f1477fff640 1 -- 172.21.15.45:0/4212419022 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f148801b070 con 0x7f1490162f50 2024-05-08T05:51:47.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.434+0000 7f1477fff640 1 --2- 172.21.15.45:0/4212419022 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f147c0762d0 0x7f147c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:47.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.434+0000 7f1477fff640 1 -- 172.21.15.45:0/4212419022 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f1488096730 con 0x7f1490162f50 2024-05-08T05:51:47.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.434+0000 7f148f7fe640 1 --2- 172.21.15.45:0/4212419022 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f147c0762d0 0x7f147c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:47.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.435+0000 7f148f7fe640 1 --2- 172.21.15.45:0/4212419022 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f147c0762d0 0x7f147c078790 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1480009fd0 tx=0x7f148000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:47.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.438+0000 7f1477fff640 1 -- 172.21.15.45:0/4212419022 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f148805fb90 con 0x7f1490162f50 2024-05-08T05:51:47.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.626+0000 7f1495ea7640 1 -- 172.21.15.45:0/4212419022 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1490070a30 con 0x7f147c0762d0 2024-05-08T05:51:47.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.627+0000 7f1477fff640 1 -- 172.21.15.45:0/4212419022 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13399 (secure 0 0 0) 0x7f1490070a30 con 0x7f147c0762d0 2024-05-08T05:51:47.627 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:47.627 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:51:47.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.629+0000 7f1495ea7640 1 -- 172.21.15.45:0/4212419022 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f147c0762d0 msgr2=0x7f147c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:47.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.629+0000 7f1495ea7640 1 --2- 172.21.15.45:0/4212419022 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f147c0762d0 0x7f147c078790 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1480009fd0 tx=0x7f148000b040 comp rx=0 tx=0).stop 2024-05-08T05:51:47.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.629+0000 7f1495ea7640 1 -- 172.21.15.45:0/4212419022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1490162f50 msgr2=0x7f14900b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:47.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.629+0000 7f1495ea7640 1 --2- 172.21.15.45:0/4212419022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1490162f50 0x7f14900b07f0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f148800c500 tx=0x7f148800c9c0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.629+0000 7f1495ea7640 1 -- 172.21.15.45:0/4212419022 shutdown_connections 2024-05-08T05:51:47.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.629+0000 7f1495ea7640 1 --2- 172.21.15.45:0/4212419022 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f147c0762d0 0x7f147c078790 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.629+0000 7f1495ea7640 1 --2- 172.21.15.45:0/4212419022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1490162f50 0x7f14900b07f0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.629+0000 7f1495ea7640 1 --2- 172.21.15.45:0/4212419022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14900cf790 0x7f14900b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.629+0000 7f1495ea7640 1 --2- 172.21.15.45:0/4212419022 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f14900b21f0 0x7f14901f45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.629+0000 7f1495ea7640 1 -- 172.21.15.45:0/4212419022 >> 172.21.15.45:0/4212419022 conn(0x7f149009ba40 msgr2=0x7f149015f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:47.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.629+0000 7f1495ea7640 1 -- 172.21.15.45:0/4212419022 shutdown_connections 2024-05-08T05:51:47.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.629+0000 7f1495ea7640 1 -- 172.21.15.45:0/4212419022 wait complete. 2024-05-08T05:51:47.640 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-05-08T05:51:46.984195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278896,"kb_used_data":29460,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695824,"statfs":{"total":773094113280,"available":772808523776,"internally_reserved":0,"allocated":30167040,"data_stored":23632035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390399,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-267,"num_write":-170,"num_write_kb":-8299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001284"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":301,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-05-08T05:51:44.888547+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:51:44.888547+0000","last_peered":"2024-05-08T05:51:44.888547+0000","last_clean":"2024-05-08T05:51:44.888547+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:51:44.888547+0000","last_undegraded":"2024-05-08T05:51:44.888547+0000","last_fullsized":"2024-05-08T05:51:44.888547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542273,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33260,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338580,"statfs":{"total":96636764160,"available":96602705920,"internally_reserved":0,"allocated":2129920,"data_stored":1300759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542274,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36760,"kb_used_data":5580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335080,"statfs":{"total":96636764160,"available":96599121920,"internally_reserved":0,"allocated":5713920,"data_stored":4904494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542273,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":5584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":5718016,"data_stored":4904072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574976,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36776,"kb_used_data":5596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335064,"statfs":{"total":96636764160,"available":96599105536,"internally_reserved":0,"allocated":5730304,"data_stored":4905041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574975,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1531904,"data_stored":710759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574975,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1519616,"data_stored":709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":5572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":5705728,"data_stored":4897294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33248,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338592,"statfs":{"total":96636764160,"available":96602718208,"internally_reserved":0,"allocated":2117632,"data_stored":1300003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:51:47.641 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T05:51:47.641 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:51:47.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.745+0000 7fa4f3fac640 1 Processor -- start 2024-05-08T05:51:47.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.745+0000 7fa4f3fac640 1 -- start start 2024-05-08T05:51:47.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.746+0000 7fa4f3fac640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa4ec167c30 0x7fa4ec0d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:47.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.746+0000 7fa4f3fac640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4ec0d31d0 con 0x7fa4ec0b4770 2024-05-08T05:51:47.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.746+0000 7fa4f3fac640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4ec0d34b0 con 0x7fa4ec0d53d0 2024-05-08T05:51:47.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.746+0000 7fa4f3fac640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa4ec0d3790 con 0x7fa4ec167c30 2024-05-08T05:51:47.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.746+0000 7fa4f2549640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa4ec167c30 0x7fa4ec0d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:47.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.746+0000 7fa4f1d48640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fa4ec0b4770 0x7fa4ec0d4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38776/0 (socket says 172.21.15.45:38776) 2024-05-08T05:51:47.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.746+0000 7fa4f1d48640 1 -- 172.21.15.45:0/2406060749 learned_addr learned my addr 172.21.15.45:0/2406060749 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:47.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.746+0000 7fa4f2549640 1 -- 172.21.15.45:0/2406060749 >> v1:172.21.15.63:6789/0 conn(0x7fa4ec0d53d0 legacy=0x7fa4ec0cfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:47.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.746+0000 7fa4f2549640 1 -- 172.21.15.45:0/2406060749 >> v1:172.21.15.45:6789/0 conn(0x7fa4ec0b4770 legacy=0x7fa4ec0d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:47.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.746+0000 7fa4f2549640 1 -- 172.21.15.45:0/2406060749 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec0d3a70 con 0x7fa4ec167c30 2024-05-08T05:51:47.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.746+0000 7fa4f0d46640 1 -- 172.21.15.45:0/2406060749 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3893632224 0 0) 0x7fa4ec0d31d0 con 0x7fa4ec0b4770 2024-05-08T05:51:47.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.747+0000 7fa4f2549640 1 --2- 172.21.15.45:0/2406060749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ec167c30 0x7fa4ec0d0410 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009d90 tx=0x7fa4e402a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10a60acbc7bdfec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:47.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.747+0000 7fa4f0d46640 1 -- 172.21.15.45:0/2406060749 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4007740 con 0x7fa4ec167c30 2024-05-08T05:51:47.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.747+0000 7fa4f0d46640 1 -- 172.21.15.45:0/2406060749 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e40078e0 con 0x7fa4ec167c30 2024-05-08T05:51:47.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.747+0000 7fa4f0d46640 1 -- 172.21.15.45:0/2406060749 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4007cb0 con 0x7fa4ec167c30 2024-05-08T05:51:47.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.748+0000 7fa4f3fac640 1 -- 172.21.15.45:0/2406060749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ec167c30 msgr2=0x7fa4ec0d0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:47.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.748+0000 7fa4f3fac640 1 --2- 172.21.15.45:0/2406060749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ec167c30 0x7fa4ec0d0410 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009d90 tx=0x7fa4e402a530 comp rx=0 tx=0).stop 2024-05-08T05:51:47.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.748+0000 7fa4f3fac640 1 -- 172.21.15.45:0/2406060749 shutdown_connections 2024-05-08T05:51:47.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.748+0000 7fa4f3fac640 1 --2- 172.21.15.45:0/2406060749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ec167c30 0x7fa4ec0d0410 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.748+0000 7fa4f3fac640 1 -- 172.21.15.45:0/2406060749 >> 172.21.15.45:0/2406060749 conn(0x7fa4ec08eac0 msgr2=0x7fa4ec09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:47.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.748+0000 7fa4f3fac640 1 -- 172.21.15.45:0/2406060749 shutdown_connections 2024-05-08T05:51:47.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.748+0000 7fa4f3fac640 1 -- 172.21.15.45:0/2406060749 wait complete. 2024-05-08T05:51:47.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.749+0000 7fa4f3fac640 1 Processor -- start 2024-05-08T05:51:47.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.749+0000 7fa4f3fac640 1 -- start start 2024-05-08T05:51:47.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.750+0000 7fa4f3fac640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4ec0b4770 0x7fa4ec159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:47.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.750+0000 7fa4f3fac640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4ec0d53d0 0x7fa4ec0ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:47.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.750+0000 7fa4f3fac640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ec167c30 0x7fa4ec0b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:47.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.750+0000 7fa4f3fac640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec0d2df0 con 0x7fa4ec167c30 2024-05-08T05:51:47.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.750+0000 7fa4f3fac640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec168040 con 0x7fa4ec0b4770 2024-05-08T05:51:47.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.750+0000 7fa4f3fac640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa4ec0d2f70 con 0x7fa4ec0d53d0 2024-05-08T05:51:47.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.750+0000 7fa4f2549640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ec167c30 0x7fa4ec0b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:47.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.750+0000 7fa4f2549640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ec167c30 0x7fa4ec0b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54440/0 (socket says 172.21.15.45:54440) 2024-05-08T05:51:47.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.750+0000 7fa4f2549640 1 -- 172.21.15.45:0/2056768199 learned_addr learned my addr 172.21.15.45:0/2056768199 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:47.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.750+0000 7fa4f1547640 1 --2- 172.21.15.45:0/2056768199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4ec0d53d0 0x7fa4ec0ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:47.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.750+0000 7fa4f1d48640 1 --2- 172.21.15.45:0/2056768199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4ec0b4770 0x7fa4ec159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:47.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.751+0000 7fa4f2549640 1 -- 172.21.15.45:0/2056768199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4ec0d53d0 msgr2=0x7fa4ec0ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:47.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.751+0000 7fa4f2549640 1 --2- 172.21.15.45:0/2056768199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4ec0d53d0 0x7fa4ec0ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.751+0000 7fa4f2549640 1 -- 172.21.15.45:0/2056768199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4ec0b4770 msgr2=0x7fa4ec159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:47.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.751+0000 7fa4f2549640 1 --2- 172.21.15.45:0/2056768199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4ec0b4770 0x7fa4ec159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.751+0000 7fa4f2549640 1 -- 172.21.15.45:0/2056768199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec159e50 con 0x7fa4ec167c30 2024-05-08T05:51:47.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.751+0000 7fa4f2549640 1 --2- 172.21.15.45:0/2056768199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ec167c30 0x7fa4ec0b1c40 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009d90 tx=0x7fa4e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:47.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.751+0000 7fa4e27fc640 1 -- 172.21.15.45:0/2056768199 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4040070 con 0x7fa4ec167c30 2024-05-08T05:51:47.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.751+0000 7fa4e27fc640 1 -- 172.21.15.45:0/2056768199 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e4031c40 con 0x7fa4ec167c30 2024-05-08T05:51:47.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.751+0000 7fa4e27fc640 1 -- 172.21.15.45:0/2056768199 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4003990 con 0x7fa4ec167c30 2024-05-08T05:51:47.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.751+0000 7fa4f3fac640 1 -- 172.21.15.45:0/2056768199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ec0cc2f0 con 0x7fa4ec167c30 2024-05-08T05:51:47.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.751+0000 7fa4f3fac640 1 -- 172.21.15.45:0/2056768199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ec0cc8b0 con 0x7fa4ec167c30 2024-05-08T05:51:47.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.752+0000 7fa4f3fac640 1 -- 172.21.15.45:0/2056768199 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4ec0693f0 con 0x7fa4ec167c30 2024-05-08T05:51:47.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.753+0000 7fa4e27fc640 1 -- 172.21.15.45:0/2056768199 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa4e4039040 con 0x7fa4ec167c30 2024-05-08T05:51:47.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.753+0000 7fa4e27fc640 1 --2- 172.21.15.45:0/2056768199 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa4d0076300 0x7fa4d00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:47.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.753+0000 7fa4e27fc640 1 -- 172.21.15.45:0/2056768199 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7fa4e40b8370 con 0x7fa4ec167c30 2024-05-08T05:51:47.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.753+0000 7fa4f1d48640 1 --2- 172.21.15.45:0/2056768199 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa4d0076300 0x7fa4d00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:47.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.754+0000 7fa4f1d48640 1 --2- 172.21.15.45:0/2056768199 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa4d0076300 0x7fa4d00787c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8000f60 tx=0x7fa4e8007cf0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:47.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.757+0000 7fa4e27fc640 1 -- 172.21.15.45:0/2056768199 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa4e40817d0 con 0x7fa4ec167c30 2024-05-08T05:51:47.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.946+0000 7fa4f3fac640 1 -- 172.21.15.45:0/2056768199 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa4ec0cf440 con 0x7fa4d0076300 2024-05-08T05:51:47.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.947+0000 7fa4e27fc640 1 -- 172.21.15.45:0/2056768199 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13399 (secure 0 0 0) 0x7fa4ec0cf440 con 0x7fa4d0076300 2024-05-08T05:51:47.948 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:47.948 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:51:47.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.949+0000 7fa4f3fac640 1 -- 172.21.15.45:0/2056768199 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa4d0076300 msgr2=0x7fa4d00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:47.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.949+0000 7fa4f3fac640 1 --2- 172.21.15.45:0/2056768199 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa4d0076300 0x7fa4d00787c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8000f60 tx=0x7fa4e8007cf0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.949+0000 7fa4f3fac640 1 -- 172.21.15.45:0/2056768199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ec167c30 msgr2=0x7fa4ec0b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:47.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.949+0000 7fa4f3fac640 1 --2- 172.21.15.45:0/2056768199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ec167c30 0x7fa4ec0b1c40 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009d90 tx=0x7fa4e4004a40 comp rx=0 tx=0).stop 2024-05-08T05:51:47.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.949+0000 7fa4f3fac640 1 -- 172.21.15.45:0/2056768199 shutdown_connections 2024-05-08T05:51:47.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.949+0000 7fa4f3fac640 1 --2- 172.21.15.45:0/2056768199 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa4d0076300 0x7fa4d00787c0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.949+0000 7fa4f3fac640 1 --2- 172.21.15.45:0/2056768199 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa4ec0d53d0 0x7fa4ec0ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.949+0000 7fa4f3fac640 1 --2- 172.21.15.45:0/2056768199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa4ec167c30 0x7fa4ec0b1c40 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.949+0000 7fa4f3fac640 1 --2- 172.21.15.45:0/2056768199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4ec0b4770 0x7fa4ec159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:47.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.949+0000 7fa4f3fac640 1 -- 172.21.15.45:0/2056768199 >> 172.21.15.45:0/2056768199 conn(0x7fa4ec08eac0 msgr2=0x7fa4ec0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:47.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.949+0000 7fa4f3fac640 1 -- 172.21.15.45:0/2056768199 shutdown_connections 2024-05-08T05:51:47.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:47.949+0000 7fa4f3fac640 1 -- 172.21.15.45:0/2056768199 wait complete. 2024-05-08T05:51:47.961 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-05-08T05:51:46.984195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278896,"kb_used_data":29460,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695824,"statfs":{"total":773094113280,"available":772808523776,"internally_reserved":0,"allocated":30167040,"data_stored":23632035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390399,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-267,"num_write":-170,"num_write_kb":-8299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001284"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":301,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-05-08T05:51:44.888547+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:51:44.888547+0000","last_peered":"2024-05-08T05:51:44.888547+0000","last_clean":"2024-05-08T05:51:44.888547+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:51:44.888547+0000","last_undegraded":"2024-05-08T05:51:44.888547+0000","last_fullsized":"2024-05-08T05:51:44.888547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542273,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33260,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338580,"statfs":{"total":96636764160,"available":96602705920,"internally_reserved":0,"allocated":2129920,"data_stored":1300759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542274,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36760,"kb_used_data":5580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335080,"statfs":{"total":96636764160,"available":96599121920,"internally_reserved":0,"allocated":5713920,"data_stored":4904494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542273,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":5584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":5718016,"data_stored":4904072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574976,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36776,"kb_used_data":5596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335064,"statfs":{"total":96636764160,"available":96599105536,"internally_reserved":0,"allocated":5730304,"data_stored":4905041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574975,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1531904,"data_stored":710759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574975,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1519616,"data_stored":709613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":5572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":5705728,"data_stored":4897294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33248,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338592,"statfs":{"total":96636764160,"available":96602718208,"internally_reserved":0,"allocated":2117632,"data_stored":1300003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:51:47.962 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:51:48.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.065+0000 7f38085e4640 1 Processor -- start 2024-05-08T05:51:48.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.065+0000 7f38085e4640 1 -- start start 2024-05-08T05:51:48.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.066+0000 7f38085e4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f380015f420 0x7f38000d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:48.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.066+0000 7f38085e4640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f38000d2080 0x7f38000d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:48.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.066+0000 7f38085e4640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38001630b0 con 0x7f38000d3ca0 2024-05-08T05:51:48.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.066+0000 7f38085e4640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f38000d2550 con 0x7f38000d2080 2024-05-08T05:51:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.066+0000 7f38085e4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f38000d2c80 con 0x7f380015f420 2024-05-08T05:51:48.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.067+0000 7f3806b81640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f38000d2080 0x7f38000d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.067+0000 7f3806380640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f380015f420 0x7f38000d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.067+0000 7f3806380640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f380015f420 0x7f38000d3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:35090/0 (socket says 172.21.15.45:35090) 2024-05-08T05:51:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.067+0000 7f3806380640 1 -- 172.21.15.45:0/702923144 learned_addr learned my addr 172.21.15.45:0/702923144 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.067+0000 7f3806b81640 1 -- 172.21.15.45:0/702923144 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f380015f420 msgr2=0x7f38000d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.067+0000 7f3806b81640 1 --2- 172.21.15.45:0/702923144 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f380015f420 0x7f38000d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.067+0000 7f3806b81640 1 -- 172.21.15.45:0/702923144 >> v1:172.21.15.63:6789/0 conn(0x7f38000d3ca0 legacy=0x7f38000d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.067+0000 7f3806b81640 1 -- 172.21.15.45:0/702923144 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38000d2f60 con 0x7f38000d2080 2024-05-08T05:51:48.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.067+0000 7f3806380640 1 --2- 172.21.15.45:0/702923144 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f380015f420 0x7f38000d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:48.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.068+0000 7f3806b81640 1 --2- 172.21.15.45:0/702923144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38000d2080 0x7f38000d4cc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009d90 tx=0x7f37f802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ae450839b3435fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:48.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.068+0000 7f380537e640 1 -- 172.21.15.45:0/702923144 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f8007740 con 0x7f38000d2080 2024-05-08T05:51:48.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.068+0000 7f380537e640 1 -- 172.21.15.45:0/702923144 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37f80078e0 con 0x7f38000d2080 2024-05-08T05:51:48.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.068+0000 7f380537e640 1 -- 172.21.15.45:0/702923144 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f8007cb0 con 0x7f38000d2080 2024-05-08T05:51:48.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.069+0000 7f38085e4640 1 -- 172.21.15.45:0/702923144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38000d2080 msgr2=0x7f38000d4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:48.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.069+0000 7f38085e4640 1 --2- 172.21.15.45:0/702923144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38000d2080 0x7f38000d4cc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009d90 tx=0x7f37f802a660 comp rx=0 tx=0).stop 2024-05-08T05:51:48.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.069+0000 7f38085e4640 1 -- 172.21.15.45:0/702923144 shutdown_connections 2024-05-08T05:51:48.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.069+0000 7f38085e4640 1 --2- 172.21.15.45:0/702923144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38000d2080 0x7f38000d4cc0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:48.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.069+0000 7f38085e4640 1 --2- 172.21.15.45:0/702923144 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f380015f420 0x7f38000d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:48.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.069+0000 7f38085e4640 1 -- 172.21.15.45:0/702923144 >> 172.21.15.45:0/702923144 conn(0x7f380008eac0 msgr2=0x7f380009e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:48.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.069+0000 7f38085e4640 1 -- 172.21.15.45:0/702923144 shutdown_connections 2024-05-08T05:51:48.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.069+0000 7f38085e4640 1 -- 172.21.15.45:0/702923144 wait complete. 2024-05-08T05:51:48.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.070+0000 7f38085e4640 1 Processor -- start 2024-05-08T05:51:48.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.070+0000 7f38085e4640 1 -- start start 2024-05-08T05:51:48.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.071+0000 7f38085e4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38000d2080 0x7f3800159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:48.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.071+0000 7f38085e4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38000cefb0 0x7f38001e9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:48.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.071+0000 7f38085e4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3800161210 0x7f3800161690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:48.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.071+0000 7f38085e4640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f38000d5200 con 0x7f38000cefb0 2024-05-08T05:51:48.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.071+0000 7f38085e4640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f38000d2830 con 0x7f3800161210 2024-05-08T05:51:48.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.071+0000 7f38085e4640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f38000d5380 con 0x7f38000d2080 2024-05-08T05:51:48.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.071+0000 7f3805b7f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38000cefb0 0x7f38001e9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:48.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.071+0000 7f3805b7f640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38000cefb0 0x7f38001e9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54446/0 (socket says 172.21.15.45:54446) 2024-05-08T05:51:48.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.071+0000 7f3805b7f640 1 -- 172.21.15.45:0/2505756520 learned_addr learned my addr 172.21.15.45:0/2505756520 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:48.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.071+0000 7f3806380640 1 --2- 172.21.15.45:0/2505756520 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38000d2080 0x7f3800159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:48.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.071+0000 7f3806b81640 1 --2- 172.21.15.45:0/2505756520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3800161210 0x7f3800161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:48.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.071+0000 7f3805b7f640 1 -- 172.21.15.45:0/2505756520 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38000d2080 msgr2=0x7f3800159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:48.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.071+0000 7f3805b7f640 1 --2- 172.21.15.45:0/2505756520 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38000d2080 0x7f3800159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:48.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.071+0000 7f3805b7f640 1 -- 172.21.15.45:0/2505756520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3800161210 msgr2=0x7f3800161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:48.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.071+0000 7f3805b7f640 1 --2- 172.21.15.45:0/2505756520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3800161210 0x7f3800161690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:48.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.071+0000 7f3805b7f640 1 -- 172.21.15.45:0/2505756520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38000ccac0 con 0x7f38000cefb0 2024-05-08T05:51:48.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.072+0000 7f3805b7f640 1 --2- 172.21.15.45:0/2505756520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38000cefb0 0x7f38001e9590 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f37f000c450 tx=0x7f37f000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:48.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.072+0000 7f37f6ffd640 1 -- 172.21.15.45:0/2505756520 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f00052d0 con 0x7f38000cefb0 2024-05-08T05:51:48.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.072+0000 7f37f6ffd640 1 -- 172.21.15.45:0/2505756520 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37f0005470 con 0x7f38000cefb0 2024-05-08T05:51:48.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.072+0000 7f38085e4640 1 -- 172.21.15.45:0/2505756520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380015a810 con 0x7f38000cefb0 2024-05-08T05:51:48.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.072+0000 7f37f6ffd640 1 -- 172.21.15.45:0/2505756520 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f0012870 con 0x7f38000cefb0 2024-05-08T05:51:48.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.072+0000 7f38085e4640 1 -- 172.21.15.45:0/2505756520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38001e9f70 con 0x7f38000cefb0 2024-05-08T05:51:48.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.073+0000 7f38085e4640 1 -- 172.21.15.45:0/2505756520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38000693f0 con 0x7f38000cefb0 2024-05-08T05:51:48.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.074+0000 7f37f6ffd640 1 -- 172.21.15.45:0/2505756520 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f37f0012a10 con 0x7f38000cefb0 2024-05-08T05:51:48.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.074+0000 7f37f6ffd640 1 --2- 172.21.15.45:0/2505756520 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f37e4076300 0x7f37e40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:48.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.074+0000 7f37f6ffd640 1 -- 172.21.15.45:0/2505756520 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f37f00970e0 con 0x7f38000cefb0 2024-05-08T05:51:48.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.074+0000 7f3806380640 1 --2- 172.21.15.45:0/2505756520 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f37e4076300 0x7f37e40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:48.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.075+0000 7f3806380640 1 --2- 172.21.15.45:0/2505756520 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f37e4076300 0x7f37e40787c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f38001f90d0 tx=0x7f37fc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:48.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.078+0000 7f37f6ffd640 1 -- 172.21.15.45:0/2505756520 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f38001630b0 con 0x7f38000cefb0 2024-05-08T05:51:48.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.271+0000 7f38085e4640 1 -- 172.21.15.45:0/2505756520 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3800163ba0 con 0x7f38000cefb0 2024-05-08T05:51:48.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.272+0000 7f37f6ffd640 1 -- 172.21.15.45:0/2505756520 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) ==== 74+0+13513 (secure 0 0 0) 0x7f37f0060b00 con 0x7f38000cefb0 2024-05-08T05:51:48.272 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:48.272 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":124,"fsid":"e9e1170a-fd27-4625-97a9-5d5f104d13a2","created":"2024-05-08T05:41:08.308636+0000","modified":"2024-05-08T05:51:45.010104+0000","last_up_change":"2024-05-08T05:41:18.996355+0000","last_in_change":"2024-05-08T05:41:10.475782+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:41:18.847604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"402a8cef-9d73-41d2-b171-5a8140c79a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6801","nonce":3734454393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6803","nonce":3734454393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6807","nonce":3734454393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3734454393},{"type":"v1","addr":"172.21.15.45:6805","nonce":3734454393}]},"public_addr":"172.21.15.45:6801/3734454393","cluster_addr":"172.21.15.45:6803/3734454393","heartbeat_back_addr":"172.21.15.45:6807/3734454393","heartbeat_front_addr":"172.21.15.45:6805/3734454393","state":["exists","up"]},{"osd":1,"uuid":"7cf7f6a4-d457-432d-b5d9-1a69592abc90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6825","nonce":1337179184}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6827","nonce":1337179184}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6831","nonce":1337179184}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1337179184},{"type":"v1","addr":"172.21.15.45:6829","nonce":1337179184}]},"public_addr":"172.21.15.45:6825/1337179184","cluster_addr":"172.21.15.45:6827/1337179184","heartbeat_back_addr":"172.21.15.45:6831/1337179184","heartbeat_front_addr":"172.21.15.45:6829/1337179184","state":["exists","up"]},{"osd":2,"uuid":"68484847-6fbd-4594-8374-a1a3e959f247","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6809","nonce":785834340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6811","nonce":785834340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6815","nonce":785834340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":785834340},{"type":"v1","addr":"172.21.15.45:6813","nonce":785834340}]},"public_addr":"172.21.15.45:6809/785834340","cluster_addr":"172.21.15.45:6811/785834340","heartbeat_back_addr":"172.21.15.45:6815/785834340","heartbeat_front_addr":"172.21.15.45:6813/785834340","state":["exists","up"]},{"osd":3,"uuid":"663351b5-c96a-470b-b9c8-3d1f345c75a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6817","nonce":1259225446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6819","nonce":1259225446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6823","nonce":1259225446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1259225446},{"type":"v1","addr":"172.21.15.45:6821","nonce":1259225446}]},"public_addr":"172.21.15.45:6817/1259225446","cluster_addr":"172.21.15.45:6819/1259225446","heartbeat_back_addr":"172.21.15.45:6823/1259225446","heartbeat_front_addr":"172.21.15.45:6821/1259225446","state":["exists","up"]},{"osd":4,"uuid":"92decff9-1aa1-4089-9bb4-0c24366e8def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6825","nonce":2212879151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6827","nonce":2212879151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6831","nonce":2212879151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":2212879151},{"type":"v1","addr":"172.21.15.63:6829","nonce":2212879151}]},"public_addr":"172.21.15.63:6825/2212879151","cluster_addr":"172.21.15.63:6827/2212879151","heartbeat_back_addr":"172.21.15.63:6831/2212879151","heartbeat_front_addr":"172.21.15.63:6829/2212879151","state":["exists","up"]},{"osd":5,"uuid":"19f59f65-8cab-4bc4-a68d-07aab3fa695a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6809","nonce":2395798255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6811","nonce":2395798255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6815","nonce":2395798255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":2395798255},{"type":"v1","addr":"172.21.15.63:6813","nonce":2395798255}]},"public_addr":"172.21.15.63:6809/2395798255","cluster_addr":"172.21.15.63:6811/2395798255","heartbeat_back_addr":"172.21.15.63:6815/2395798255","heartbeat_front_addr":"172.21.15.63:6813/2395798255","state":["exists","up"]},{"osd":6,"uuid":"37dfc950-fffc-42d3-8d74-c27574554eee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6801","nonce":2926759052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6803","nonce":2926759052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6807","nonce":2926759052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":2926759052},{"type":"v1","addr":"172.21.15.63:6805","nonce":2926759052}]},"public_addr":"172.21.15.63:6801/2926759052","cluster_addr":"172.21.15.63:6803/2926759052","heartbeat_back_addr":"172.21.15.63:6807/2926759052","heartbeat_front_addr":"172.21.15.63:6805/2926759052","state":["exists","up"]},{"osd":7,"uuid":"adf18b78-2e46-4ad0-84ce-296c5c431ebb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6817","nonce":1163469437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6819","nonce":1163469437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6823","nonce":1163469437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1163469437},{"type":"v1","addr":"172.21.15.63:6821","nonce":1163469437}]},"public_addr":"172.21.15.63:6817/1163469437","cluster_addr":"172.21.15.63:6819/1163469437","heartbeat_back_addr":"172.21.15.63:6823/1163469437","heartbeat_front_addr":"172.21.15.63:6821/1163469437","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.612091+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.632727+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.623202+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.606219+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.699195+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.619762+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.611368+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:41:15.685029+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6835/453198328":"2024-05-09T05:51:29.962310+0000","172.21.15.45:6835/4071581846":"2024-05-09T05:51:41.774063+0000","172.21.15.45:6834/453198328":"2024-05-09T05:51:29.962310+0000","172.21.15.45:6835/1751419054":"2024-05-09T05:51:20.651495+0000","172.21.15.45:6834/4071581846":"2024-05-09T05:51:41.774063+0000","172.21.15.45:6834/1751419054":"2024-05-09T05:51:20.651495+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:51:48.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.274+0000 7f38085e4640 1 -- 172.21.15.45:0/2505756520 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f37e4076300 msgr2=0x7f37e40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:48.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.274+0000 7f38085e4640 1 --2- 172.21.15.45:0/2505756520 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f37e4076300 0x7f37e40787c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f38001f90d0 tx=0x7f37fc008040 comp rx=0 tx=0).stop 2024-05-08T05:51:48.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.274+0000 7f38085e4640 1 -- 172.21.15.45:0/2505756520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38000cefb0 msgr2=0x7f38001e9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:48.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.274+0000 7f38085e4640 1 --2- 172.21.15.45:0/2505756520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38000cefb0 0x7f38001e9590 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f37f000c450 tx=0x7f37f000c910 comp rx=0 tx=0).stop 2024-05-08T05:51:48.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.274+0000 7f38085e4640 1 -- 172.21.15.45:0/2505756520 shutdown_connections 2024-05-08T05:51:48.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.274+0000 7f38085e4640 1 --2- 172.21.15.45:0/2505756520 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f37e4076300 0x7f37e40787c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:48.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.274+0000 7f38085e4640 1 --2- 172.21.15.45:0/2505756520 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3800161210 0x7f3800161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:48.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.274+0000 7f38085e4640 1 --2- 172.21.15.45:0/2505756520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38000cefb0 0x7f38001e9590 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:48.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.274+0000 7f38085e4640 1 --2- 172.21.15.45:0/2505756520 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f38000d2080 0x7f3800159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:48.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.274+0000 7f38085e4640 1 -- 172.21.15.45:0/2505756520 >> 172.21.15.45:0/2505756520 conn(0x7f380008eac0 msgr2=0x7f380006e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:48.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.274+0000 7f38085e4640 1 -- 172.21.15.45:0/2505756520 shutdown_connections 2024-05-08T05:51:48.275 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:48.274+0000 7f38085e4640 1 -- 172.21.15.45:0/2505756520 wait complete. 2024-05-08T05:51:49.286 INFO:tasks.ceph:Scrubbing osd.0 2024-05-08T05:51:49.286 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-08T05:51:49.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.396+0000 7f1f2c358640 1 Processor -- start 2024-05-08T05:51:49.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.396+0000 7f1f2c358640 1 -- start start 2024-05-08T05:51:49.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.397+0000 7f1f2c358640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1f240cfd80 0x7f1f24164ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:49.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.397+0000 7f1f2c358640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f2415d680 0x7f1f24167540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:49.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.397+0000 7f1f2c358640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f240cec40 con 0x7f1f2415d2a0 2024-05-08T05:51:49.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.397+0000 7f1f2c358640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f1f240cef20 con 0x7f1f240cfd80 2024-05-08T05:51:49.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.397+0000 7f1f2c358640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1f240c6b10 con 0x7f1f2415d680 2024-05-08T05:51:49.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.397+0000 7f1f298f3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1f240cfd80 0x7f1f24164ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:49.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.397+0000 7f1f2a8f5640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f2415d680 0x7f1f24167540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:49.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.397+0000 7f1f2a0f4640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f1f2415d2a0 0x7f1f240cf670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:51752/0 (socket says 172.21.15.45:51752) 2024-05-08T05:51:49.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.397+0000 7f1f298f3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f1f240cfd80 0x7f1f24164ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:44000/0 (socket says 172.21.15.45:44000) 2024-05-08T05:51:49.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.397+0000 7f1f2a0f4640 1 -- 172.21.15.45:0/823979023 learned_addr learned my addr 172.21.15.45:0/823979023 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:49.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.397+0000 7f1f2a8f5640 1 -- 172.21.15.45:0/823979023 >> v2:172.21.15.63:3300/0 conn(0x7f1f240cfd80 msgr2=0x7f1f24164ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:49.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.397+0000 7f1f2a8f5640 1 --2- 172.21.15.45:0/823979023 >> v2:172.21.15.63:3300/0 conn(0x7f1f240cfd80 0x7f1f24164ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.397+0000 7f1f2a8f5640 1 -- 172.21.15.45:0/823979023 >> v1:172.21.15.63:6789/0 conn(0x7f1f2415d2a0 legacy=0x7f1f240cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:49.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.397+0000 7f1f2a8f5640 1 -- 172.21.15.45:0/823979023 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f240c6df0 con 0x7f1f2415d680 2024-05-08T05:51:49.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.397+0000 7f1f290f2640 1 -- 172.21.15.45:0/823979023 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3711575284 0 0) 0x7f1f240cec40 con 0x7f1f2415d2a0 2024-05-08T05:51:49.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.397+0000 7f1f298f3640 1 --2- 172.21.15.45:0/823979023 >> v2:172.21.15.63:3300/0 conn(0x7f1f240cfd80 0x7f1f24164ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:49.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.398+0000 7f1f2a8f5640 1 --2- 172.21.15.45:0/823979023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f2415d680 0x7f1f24167540 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c009d90 tx=0x7f1f1c02a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d58c569a8af10de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:49.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.398+0000 7f1f290f2640 1 -- 172.21.15.45:0/823979023 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c007740 con 0x7f1f2415d680 2024-05-08T05:51:49.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.398+0000 7f1f290f2640 1 -- 172.21.15.45:0/823979023 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f1c0078e0 con 0x7f1f2415d680 2024-05-08T05:51:49.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.398+0000 7f1f290f2640 1 -- 172.21.15.45:0/823979023 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c007cb0 con 0x7f1f2415d680 2024-05-08T05:51:49.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.399+0000 7f1f2c358640 1 -- 172.21.15.45:0/823979023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f2415d680 msgr2=0x7f1f24167540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:49.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.399+0000 7f1f2c358640 1 --2- 172.21.15.45:0/823979023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f2415d680 0x7f1f24167540 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c009d90 tx=0x7f1f1c02a6f0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.399+0000 7f1f2c358640 1 -- 172.21.15.45:0/823979023 shutdown_connections 2024-05-08T05:51:49.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.399+0000 7f1f2c358640 1 --2- 172.21.15.45:0/823979023 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f2415d680 0x7f1f24167540 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.399+0000 7f1f2c358640 1 --2- 172.21.15.45:0/823979023 >> v2:172.21.15.63:3300/0 conn(0x7f1f240cfd80 0x7f1f24164ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.399+0000 7f1f2c358640 1 -- 172.21.15.45:0/823979023 >> 172.21.15.45:0/823979023 conn(0x7f1f240d2f20 msgr2=0x7f1f240d3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:49.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.399+0000 7f1f2c358640 1 -- 172.21.15.45:0/823979023 shutdown_connections 2024-05-08T05:51:49.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.399+0000 7f1f2c358640 1 -- 172.21.15.45:0/823979023 wait complete. 2024-05-08T05:51:49.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.400+0000 7f1f2c358640 1 Processor -- start 2024-05-08T05:51:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.401+0000 7f1f2c358640 1 -- start start 2024-05-08T05:51:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.401+0000 7f1f2c358640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f240cfd80 0x7f1f241e53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.401+0000 7f1f2c358640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f2415d2a0 0x7f1f24159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.401+0000 7f1f2c358640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f2415d680 0x7f1f24159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.401+0000 7f1f2c358640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1f240c9780 con 0x7f1f2415d2a0 2024-05-08T05:51:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.401+0000 7f1f2c358640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f1f240c9600 con 0x7f1f240cfd80 2024-05-08T05:51:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.401+0000 7f1f2c358640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f1f240c9900 con 0x7f1f2415d680 2024-05-08T05:51:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.401+0000 7f1f298f3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f2415d2a0 0x7f1f24159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.401+0000 7f1f298f3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f2415d2a0 0x7f1f24159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54452/0 (socket says 172.21.15.45:54452) 2024-05-08T05:51:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.401+0000 7f1f298f3640 1 -- 172.21.15.45:0/2655408648 learned_addr learned my addr 172.21.15.45:0/2655408648 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.401+0000 7f1f2a0f4640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f240cfd80 0x7f1f241e53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.402+0000 7f1f2a8f5640 1 --2- 172.21.15.45:0/2655408648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f2415d680 0x7f1f24159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:49.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.402+0000 7f1f298f3640 1 -- 172.21.15.45:0/2655408648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f2415d680 msgr2=0x7f1f24159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.402+0000 7f1f298f3640 1 --2- 172.21.15.45:0/2655408648 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f1f2415d680 0x7f1f24159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.402+0000 7f1f298f3640 1 -- 172.21.15.45:0/2655408648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f240cfd80 msgr2=0x7f1f241e53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.402+0000 7f1f298f3640 1 --2- 172.21.15.45:0/2655408648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f240cfd80 0x7f1f241e53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.402+0000 7f1f298f3640 1 -- 172.21.15.45:0/2655408648 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f240caa60 con 0x7f1f2415d2a0 2024-05-08T05:51:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.402+0000 7f1f298f3640 1 --2- 172.21.15.45:0/2655408648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f2415d2a0 0x7f1f24159720 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f1f1400ebe0 tx=0x7f1f1400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.402+0000 7f1f1affd640 1 -- 172.21.15.45:0/2655408648 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14017070 con 0x7f1f2415d2a0 2024-05-08T05:51:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.402+0000 7f1f1affd640 1 -- 172.21.15.45:0/2655408648 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f1400cc30 con 0x7f1f2415d2a0 2024-05-08T05:51:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.402+0000 7f1f2c358640 1 -- 172.21.15.45:0/2655408648 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f240cad00 con 0x7f1f2415d2a0 2024-05-08T05:51:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.402+0000 7f1f1affd640 1 -- 172.21.15.45:0/2655408648 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14013470 con 0x7f1f2415d2a0 2024-05-08T05:51:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.402+0000 7f1f2c358640 1 -- 172.21.15.45:0/2655408648 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f241e79e0 con 0x7f1f2415d2a0 2024-05-08T05:51:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.403+0000 7f1f2c358640 1 -- 172.21.15.45:0/2655408648 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1f241e5900 con 0x7f1f2415d2a0 2024-05-08T05:51:49.405 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.404+0000 7f1f1affd640 1 -- 172.21.15.45:0/2655408648 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1f1401a070 con 0x7f1f2415d2a0 2024-05-08T05:51:49.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.404+0000 7f1f1affd640 1 --2- 172.21.15.45:0/2655408648 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1f080762d0 0x7f1f08078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:49.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.404+0000 7f1f1affd640 1 -- 172.21.15.45:0/2655408648 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f1f14097500 con 0x7f1f2415d2a0 2024-05-08T05:51:49.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.405+0000 7f1f1affd640 1 --2- 172.21.15.45:0/2655408648 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x7f1f0807c8b0 0x7f1f0807ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:49.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.405+0000 7f1f2a0f4640 1 --2- 172.21.15.45:0/2655408648 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1f080762d0 0x7f1f08078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:49.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.405+0000 7f1f1affd640 1 -- 172.21.15.45:0/2655408648 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f14008eb0 con 0x7f1f0807c8b0 2024-05-08T05:51:49.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.405+0000 7f1f1affd640 1 -- 172.21.15.45:0/2655408648 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7f1f1409e050 con 0x7f1f2415d2a0 2024-05-08T05:51:49.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.405+0000 7f1f2a8f5640 1 --2- 172.21.15.45:0/2655408648 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x7f1f0807c8b0 0x7f1f0807ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:49.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.405+0000 7f1f2a8f5640 1 --2- 172.21.15.45:0/2655408648 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x7f1f0807c8b0 0x7f1f0807ecf0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:49.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.405+0000 7f1f2a0f4640 1 --2- 172.21.15.45:0/2655408648 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1f080762d0 0x7f1f08078790 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1f20000f60 tx=0x7f1f20007cf0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:49.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.406+0000 7f1f1affd640 1 -- 172.21.15.45:0/2655408648 <== osd.0 v2:172.21.15.45:6800/3734454393 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f14008eb0 con 0x7f1f0807c8b0 2024-05-08T05:51:49.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.419+0000 7f1f2c358640 1 -- 172.21.15.45:0/2655408648 --> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1f24159c60 con 0x7f1f0807c8b0 2024-05-08T05:51:49.419 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.419+0000 7f1f1affd640 1 -- 172.21.15.45:0/2655408648 <== osd.0 v2:172.21.15.45:6800/3734454393 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1f1c003a90 con 0x7f1f0807c8b0 2024-05-08T05:51:49.419 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:51:49.419 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T05:51:49.419 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:51:49.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.419+0000 7f1f2c358640 1 -- 172.21.15.45:0/2655408648 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x7f1f0807c8b0 msgr2=0x7f1f0807ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:49.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.419+0000 7f1f2c358640 1 --2- 172.21.15.45:0/2655408648 >> [v2:172.21.15.45:6800/3734454393,v1:172.21.15.45:6801/3734454393] conn(0x7f1f0807c8b0 0x7f1f0807ecf0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.419+0000 7f1f2c358640 1 -- 172.21.15.45:0/2655408648 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1f080762d0 msgr2=0x7f1f08078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:49.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.419+0000 7f1f2c358640 1 --2- 172.21.15.45:0/2655408648 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f1f080762d0 0x7f1f08078790 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1f20000f60 tx=0x7f1f20007cf0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.419+0000 7f1f2c358640 1 -- 172.21.15.45:0/2655408648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f2415d2a0 msgr2=0x7f1f24159720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:49.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.419+0000 7f1f2c358640 1 --2- 172.21.15.45:0/2655408648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f2415d2a0 0x7f1f24159720 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f1f1400ebe0 tx=0x7f1f1400c520 comp rx=0 tx=0).stop 2024-05-08T05:51:49.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.419+0000 7f1f2a8f5640 1 -- 172.21.15.45:0/2655408648 reap_dead start 2024-05-08T05:51:49.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.419+0000 7f1f2c358640 1 -- 172.21.15.45:0/2655408648 shutdown_connections 2024-05-08T05:51:49.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.419+0000 7f1f2c358640 1 -- 172.21.15.45:0/2655408648 >> 172.21.15.45:0/2655408648 conn(0x7f1f240d2f20 msgr2=0x7f1f240ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:49.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.419+0000 7f1f2c358640 1 -- 172.21.15.45:0/2655408648 shutdown_connections 2024-05-08T05:51:49.421 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.420+0000 7f1f2c358640 1 -- 172.21.15.45:0/2655408648 wait complete. 2024-05-08T05:51:49.430 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-08T05:51:49.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.535+0000 7fcd610da640 1 Processor -- start 2024-05-08T05:51:49.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.535+0000 7fcd610da640 1 -- start start 2024-05-08T05:51:49.544 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.536+0000 7fcd610da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5c071720 0x7fcd5c15f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:49.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.536+0000 7fcd610da640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcd5c06c3a0 0x7fcd5c06c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:49.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.536+0000 7fcd610da640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd5c160600 con 0x7fcd5c0b1050 2024-05-08T05:51:49.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.536+0000 7fcd610da640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fcd5c071b80 con 0x7fcd5c06c3a0 2024-05-08T05:51:49.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.536+0000 7fcd610da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcd5c06cd50 con 0x7fcd5c071720 2024-05-08T05:51:49.545 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.536+0000 7fcd5b59e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcd5c06c3a0 0x7fcd5c06c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:49.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.536+0000 7fcd5b59e640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fcd5c06c3a0 0x7fcd5c06c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54454/0 (socket says 172.21.15.45:54454) 2024-05-08T05:51:49.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.536+0000 7fcd5b59e640 1 -- 172.21.15.45:0/3503584778 learned_addr learned my addr 172.21.15.45:0/3503584778 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:49.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.536+0000 7fcd53fff640 1 --2- 172.21.15.45:0/3503584778 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5c071720 0x7fcd5c15f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:49.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.536+0000 7fcd5b59e640 1 -- 172.21.15.45:0/3503584778 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5c071720 msgr2=0x7fcd5c15f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:49.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.536+0000 7fcd5b59e640 1 --2- 172.21.15.45:0/3503584778 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5c071720 0x7fcd5c15f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.536+0000 7fcd5b59e640 1 -- 172.21.15.45:0/3503584778 >> v1:172.21.15.63:6789/0 conn(0x7fcd5c0b1050 legacy=0x7fcd5c0b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:49.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.536+0000 7fcd5b59e640 1 -- 172.21.15.45:0/3503584778 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c15fd60 con 0x7fcd5c06c3a0 2024-05-08T05:51:49.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.536+0000 7fcd5a59c640 1 -- 172.21.15.45:0/3503584778 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3376590811 0 0) 0x7fcd5c160600 con 0x7fcd5c0b1050 2024-05-08T05:51:49.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.537+0000 7fcd5b59e640 1 --2- 172.21.15.45:0/3503584778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd5c06c3a0 0x7fcd5c06c810 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009e50 tx=0x7fcd5402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b5fc29cea6273b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:49.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.537+0000 7fcd5a59c640 1 -- 172.21.15.45:0/3503584778 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54036070 con 0x7fcd5c06c3a0 2024-05-08T05:51:49.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.537+0000 7fcd5a59c640 1 -- 172.21.15.45:0/3503584778 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd54031600 con 0x7fcd5c06c3a0 2024-05-08T05:51:49.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.537+0000 7fcd5a59c640 1 -- 172.21.15.45:0/3503584778 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd540319d0 con 0x7fcd5c06c3a0 2024-05-08T05:51:49.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.538+0000 7fcd610da640 1 -- 172.21.15.45:0/3503584778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd5c06c3a0 msgr2=0x7fcd5c06c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:49.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.538+0000 7fcd610da640 1 --2- 172.21.15.45:0/3503584778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd5c06c3a0 0x7fcd5c06c810 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009e50 tx=0x7fcd5402a710 comp rx=0 tx=0).stop 2024-05-08T05:51:49.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.538+0000 7fcd610da640 1 -- 172.21.15.45:0/3503584778 shutdown_connections 2024-05-08T05:51:49.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.538+0000 7fcd610da640 1 --2- 172.21.15.45:0/3503584778 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd5c06c3a0 0x7fcd5c06c810 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.538+0000 7fcd610da640 1 --2- 172.21.15.45:0/3503584778 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5c071720 0x7fcd5c15f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.538+0000 7fcd610da640 1 -- 172.21.15.45:0/3503584778 >> 172.21.15.45:0/3503584778 conn(0x7fcd5c0b3c40 msgr2=0x7fcd5c0b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:49.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.538+0000 7fcd610da640 1 -- 172.21.15.45:0/3503584778 shutdown_connections 2024-05-08T05:51:49.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.538+0000 7fcd610da640 1 -- 172.21.15.45:0/3503584778 wait complete. 2024-05-08T05:51:49.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.539+0000 7fcd610da640 1 Processor -- start 2024-05-08T05:51:49.548 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.539+0000 7fcd610da640 1 -- start start 2024-05-08T05:51:49.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.540+0000 7fcd610da640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd5c06c3a0 0x7fcd5c154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:49.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.540+0000 7fcd610da640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd5c071720 0x7fcd5c1f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:49.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.540+0000 7fcd610da640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5c0b1050 0x7fcd5c161800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:49.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.540+0000 7fcd610da640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c162d10 con 0x7fcd5c06c3a0 2024-05-08T05:51:49.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.540+0000 7fcd610da640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c160320 con 0x7fcd5c071720 2024-05-08T05:51:49.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.540+0000 7fcd610da640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fcd5c0b5dc0 con 0x7fcd5c0b1050 2024-05-08T05:51:49.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.540+0000 7fcd5ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd5c06c3a0 0x7fcd5c154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:49.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.540+0000 7fcd5ad9d640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd5c06c3a0 0x7fcd5c154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54468/0 (socket says 172.21.15.45:54468) 2024-05-08T05:51:49.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.540+0000 7fcd5ad9d640 1 -- 172.21.15.45:0/2090492418 learned_addr learned my addr 172.21.15.45:0/2090492418 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:49.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.540+0000 7fcd53fff640 1 --2- 172.21.15.45:0/2090492418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd5c071720 0x7fcd5c1f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:49.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.540+0000 7fcd5b59e640 1 --2- 172.21.15.45:0/2090492418 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5c0b1050 0x7fcd5c161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:49.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.541+0000 7fcd5ad9d640 1 -- 172.21.15.45:0/2090492418 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5c0b1050 msgr2=0x7fcd5c161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:49.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.541+0000 7fcd5ad9d640 1 --2- 172.21.15.45:0/2090492418 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5c0b1050 0x7fcd5c161800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.541+0000 7fcd5ad9d640 1 -- 172.21.15.45:0/2090492418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd5c071720 msgr2=0x7fcd5c1f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:49.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.541+0000 7fcd5ad9d640 1 --2- 172.21.15.45:0/2090492418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd5c071720 0x7fcd5c1f5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.541+0000 7fcd5ad9d640 1 -- 172.21.15.45:0/2090492418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c15a5d0 con 0x7fcd5c06c3a0 2024-05-08T05:51:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.541+0000 7fcd5ad9d640 1 --2- 172.21.15.45:0/2090492418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd5c06c3a0 0x7fcd5c154290 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00c530 tx=0x7fcd4c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.541+0000 7fcd537fe640 1 -- 172.21.15.45:0/2090492418 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c017070 con 0x7fcd5c06c3a0 2024-05-08T05:51:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.541+0000 7fcd537fe640 1 -- 172.21.15.45:0/2090492418 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4c0035f0 con 0x7fcd5c06c3a0 2024-05-08T05:51:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.541+0000 7fcd610da640 1 -- 172.21.15.45:0/2090492418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5c15a8b0 con 0x7fcd5c06c3a0 2024-05-08T05:51:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.541+0000 7fcd537fe640 1 -- 172.21.15.45:0/2090492418 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c0039e0 con 0x7fcd5c06c3a0 2024-05-08T05:51:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.541+0000 7fcd610da640 1 -- 172.21.15.45:0/2090492418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5c15ae70 con 0x7fcd5c06c3a0 2024-05-08T05:51:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.542+0000 7fcd610da640 1 -- 172.21.15.45:0/2090492418 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd5c069b00 con 0x7fcd5c06c3a0 2024-05-08T05:51:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.543+0000 7fcd537fe640 1 -- 172.21.15.45:0/2090492418 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcd4c01a070 con 0x7fcd5c06c3a0 2024-05-08T05:51:49.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.544+0000 7fcd537fe640 1 --2- 172.21.15.45:0/2090492418 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcd3c0762d0 0x7fcd3c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.544+0000 7fcd537fe640 1 -- 172.21.15.45:0/2090492418 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7fcd4c0968c0 con 0x7fcd5c06c3a0 2024-05-08T05:51:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.544+0000 7fcd53fff640 1 --2- 172.21.15.45:0/2090492418 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcd3c0762d0 0x7fcd3c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.544+0000 7fcd53fff640 1 --2- 172.21.15.45:0/2090492418 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcd3c0762d0 0x7fcd3c078790 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0b0960 tx=0x7fcd44009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:49.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.548+0000 7fcd537fe640 1 -- 172.21.15.45:0/2090492418 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcd4c05fca0 con 0x7fcd5c06c3a0 2024-05-08T05:51:49.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.739+0000 7fcd610da640 1 -- 172.21.15.45:0/2090492418 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fcd5c06d290 con 0x7fcd3c0762d0 2024-05-08T05:51:49.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.739+0000 7fcd537fe640 1 -- 172.21.15.45:0/2090492418 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcd5c06d290 con 0x7fcd3c0762d0 2024-05-08T05:51:49.739 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 0 to deep-scrub 2024-05-08T05:51:49.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.741+0000 7fcd610da640 1 -- 172.21.15.45:0/2090492418 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcd3c0762d0 msgr2=0x7fcd3c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:49.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.741+0000 7fcd610da640 1 --2- 172.21.15.45:0/2090492418 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcd3c0762d0 0x7fcd3c078790 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0b0960 tx=0x7fcd44009190 comp rx=0 tx=0).stop 2024-05-08T05:51:49.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.741+0000 7fcd610da640 1 -- 172.21.15.45:0/2090492418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd5c06c3a0 msgr2=0x7fcd5c154290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:49.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.741+0000 7fcd610da640 1 --2- 172.21.15.45:0/2090492418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd5c06c3a0 0x7fcd5c154290 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00c530 tx=0x7fcd4c00c9f0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.741+0000 7fcd610da640 1 -- 172.21.15.45:0/2090492418 shutdown_connections 2024-05-08T05:51:49.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.741+0000 7fcd610da640 1 --2- 172.21.15.45:0/2090492418 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fcd3c0762d0 0x7fcd3c078790 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.741+0000 7fcd610da640 1 --2- 172.21.15.45:0/2090492418 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fcd5c0b1050 0x7fcd5c161800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.741+0000 7fcd610da640 1 --2- 172.21.15.45:0/2090492418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd5c071720 0x7fcd5c1f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.741+0000 7fcd610da640 1 --2- 172.21.15.45:0/2090492418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcd5c06c3a0 0x7fcd5c154290 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.741+0000 7fcd610da640 1 -- 172.21.15.45:0/2090492418 >> 172.21.15.45:0/2090492418 conn(0x7fcd5c0b3c40 msgr2=0x7fcd5c0b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:49.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.741+0000 7fcd610da640 1 -- 172.21.15.45:0/2090492418 shutdown_connections 2024-05-08T05:51:49.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.741+0000 7fcd610da640 1 -- 172.21.15.45:0/2090492418 wait complete. 2024-05-08T05:51:49.752 INFO:tasks.ceph:Scrubbing osd.1 2024-05-08T05:51:49.752 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-08T05:51:49.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.860+0000 7f0e1dfaf640 1 Processor -- start 2024-05-08T05:51:49.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.860+0000 7f0e1dfaf640 1 -- start start 2024-05-08T05:51:49.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.861+0000 7f0e1dfaf640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0e1815c7d0 0x7f0e180737e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:49.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.861+0000 7f0e1dfaf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e18073d20 0x7f0e180b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:49.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.861+0000 7f0e1dfaf640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e18072270 con 0x7f0e1815c3f0 2024-05-08T05:51:49.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.861+0000 7f0e1dfaf640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f0e18072550 con 0x7f0e1815c7d0 2024-05-08T05:51:49.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.861+0000 7f0e1dfaf640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0e18072830 con 0x7f0e18073d20 2024-05-08T05:51:49.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.862+0000 7f0e16ffd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0e1815c7d0 0x7f0e180737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:49.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.862+0000 7f0e177fe640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f0e1815c3f0 0x7f0e1806ea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:51780/0 (socket says 172.21.15.45:51780) 2024-05-08T05:51:49.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.862+0000 7f0e16ffd640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f0e1815c7d0 0x7f0e180737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:44020/0 (socket says 172.21.15.45:44020) 2024-05-08T05:51:49.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.862+0000 7f0e177fe640 1 -- 172.21.15.45:0/4001662894 learned_addr learned my addr 172.21.15.45:0/4001662894 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:49.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.862+0000 7f0e17fff640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e18073d20 0x7f0e180b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:49.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.862+0000 7f0e17fff640 1 -- 172.21.15.45:0/4001662894 >> v2:172.21.15.63:3300/0 conn(0x7f0e1815c7d0 msgr2=0x7f0e180737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:49.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.862+0000 7f0e17fff640 1 --2- 172.21.15.45:0/4001662894 >> v2:172.21.15.63:3300/0 conn(0x7f0e1815c7d0 0x7f0e180737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.862+0000 7f0e17fff640 1 -- 172.21.15.45:0/4001662894 >> v1:172.21.15.63:6789/0 conn(0x7f0e1815c3f0 legacy=0x7f0e1806ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:49.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.862+0000 7f0e17fff640 1 -- 172.21.15.45:0/4001662894 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e18072b10 con 0x7f0e18073d20 2024-05-08T05:51:49.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.862+0000 7f0e16ffd640 1 --2- 172.21.15.45:0/4001662894 >> v2:172.21.15.63:3300/0 conn(0x7f0e1815c7d0 0x7f0e180737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:49.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.863+0000 7f0e17fff640 1 --2- 172.21.15.45:0/4001662894 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e18073d20 0x7f0e180b3c20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000adc0 tx=0x7f0e1002b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=237bfe9eb5235d03 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:49.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.863+0000 7f0e167fc640 1 -- 172.21.15.45:0/4001662894 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10008740 con 0x7f0e18073d20 2024-05-08T05:51:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.863+0000 7f0e167fc640 1 -- 172.21.15.45:0/4001662894 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e100088e0 con 0x7f0e18073d20 2024-05-08T05:51:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.863+0000 7f0e167fc640 1 -- 172.21.15.45:0/4001662894 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10008cb0 con 0x7f0e18073d20 2024-05-08T05:51:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.863+0000 7f0e1dfaf640 1 -- 172.21.15.45:0/4001662894 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e18073d20 msgr2=0x7f0e180b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.863+0000 7f0e1dfaf640 1 --2- 172.21.15.45:0/4001662894 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e18073d20 0x7f0e180b3c20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0e1000adc0 tx=0x7f0e1002b710 comp rx=0 tx=0).stop 2024-05-08T05:51:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.863+0000 7f0e1dfaf640 1 -- 172.21.15.45:0/4001662894 shutdown_connections 2024-05-08T05:51:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.863+0000 7f0e1dfaf640 1 --2- 172.21.15.45:0/4001662894 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e18073d20 0x7f0e180b3c20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.863+0000 7f0e1dfaf640 1 --2- 172.21.15.45:0/4001662894 >> v2:172.21.15.63:3300/0 conn(0x7f0e1815c7d0 0x7f0e180737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.863+0000 7f0e1dfaf640 1 -- 172.21.15.45:0/4001662894 >> 172.21.15.45:0/4001662894 conn(0x7f0e1806c3a0 msgr2=0x7f0e1806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.863+0000 7f0e1dfaf640 1 -- 172.21.15.45:0/4001662894 shutdown_connections 2024-05-08T05:51:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.863+0000 7f0e1dfaf640 1 -- 172.21.15.45:0/4001662894 wait complete. 2024-05-08T05:51:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.865+0000 7f0e1dfaf640 1 Processor -- start 2024-05-08T05:51:49.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.865+0000 7f0e1dfaf640 1 -- start start 2024-05-08T05:51:49.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.865+0000 7f0e1dfaf640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e18073d20 0x7f0e1814ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:49.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.865+0000 7f0e1dfaf640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e1815c3f0 0x7f0e1809ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:49.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.865+0000 7f0e1dfaf640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e1815c7d0 0x7f0e180aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:49.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.865+0000 7f0e1dfaf640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1806f110 con 0x7f0e1815c7d0 2024-05-08T05:51:49.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.865+0000 7f0e1dfaf640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0e18073310 con 0x7f0e1815c3f0 2024-05-08T05:51:49.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.865+0000 7f0e1dfaf640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0e180d0bf0 con 0x7f0e18073d20 2024-05-08T05:51:49.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.866+0000 7f0e17fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e1815c7d0 0x7f0e180aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:49.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.866+0000 7f0e16ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e1815c3f0 0x7f0e1809ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:49.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.866+0000 7f0e16ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e1815c3f0 0x7f0e1809ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:44028/0 (socket says 172.21.15.45:44028) 2024-05-08T05:51:49.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.866+0000 7f0e16ffd640 1 -- 172.21.15.45:0/2061793061 learned_addr learned my addr 172.21.15.45:0/2061793061 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:49.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.866+0000 7f0e17fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e1815c7d0 0x7f0e180aedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54484/0 (socket says 172.21.15.45:54484) 2024-05-08T05:51:49.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.866+0000 7f0e177fe640 1 --2- 172.21.15.45:0/2061793061 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e18073d20 0x7f0e1814ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:49.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.866+0000 7f0e16ffd640 1 -- 172.21.15.45:0/2061793061 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e18073d20 msgr2=0x7f0e1814ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:49.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.866+0000 7f0e16ffd640 1 --2- 172.21.15.45:0/2061793061 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0e18073d20 0x7f0e1814ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.866+0000 7f0e16ffd640 1 -- 172.21.15.45:0/2061793061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e1815c7d0 msgr2=0x7f0e180aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:49.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.866+0000 7f0e16ffd640 1 --2- 172.21.15.45:0/2061793061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0e1815c7d0 0x7f0e180aedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.866+0000 7f0e16ffd640 1 -- 172.21.15.45:0/2061793061 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1809c390 con 0x7f0e1815c3f0 2024-05-08T05:51:49.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.866+0000 7f0e16ffd640 1 --2- 172.21.15.45:0/2061793061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e1815c3f0 0x7f0e1809ba40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00e8e0 tx=0x7f0e0c00eda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:49.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.866+0000 7f0dfffff640 1 -- 172.21.15.45:0/2061793061 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c017070 con 0x7f0e1815c3f0 2024-05-08T05:51:49.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.866+0000 7f0dfffff640 1 -- 172.21.15.45:0/2061793061 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e0c00ca00 con 0x7f0e1815c3f0 2024-05-08T05:51:49.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.866+0000 7f0e1dfaf640 1 -- 172.21.15.45:0/2061793061 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1809c670 con 0x7f0e1815c3f0 2024-05-08T05:51:49.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.866+0000 7f0dfffff640 1 -- 172.21.15.45:0/2061793061 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c00cdb0 con 0x7f0e1815c3f0 2024-05-08T05:51:49.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.866+0000 7f0e1dfaf640 1 -- 172.21.15.45:0/2061793061 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1809cc30 con 0x7f0e1815c3f0 2024-05-08T05:51:49.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.867+0000 7f0e1dfaf640 1 -- 172.21.15.45:0/2061793061 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0e180693f0 con 0x7f0e1815c3f0 2024-05-08T05:51:49.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.868+0000 7f0dfffff640 1 -- 172.21.15.45:0/2061793061 <== mon.1 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0e0c01a070 con 0x7f0e1815c3f0 2024-05-08T05:51:49.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.869+0000 7f0dfffff640 1 --2- 172.21.15.45:0/2061793061 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0e04076300 0x7f0e040787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:49.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.869+0000 7f0dfffff640 1 -- 172.21.15.45:0/2061793061 <== mon.1 v2:172.21.15.63:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f0e0c097b90 con 0x7f0e1815c3f0 2024-05-08T05:51:49.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.869+0000 7f0e177fe640 1 --2- 172.21.15.45:0/2061793061 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0e04076300 0x7f0e040787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:49.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.869+0000 7f0dfffff640 1 --2- 172.21.15.45:0/2061793061 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x7f0e0407c8e0 0x7f0e0407ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:49.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.869+0000 7f0dfffff640 1 -- 172.21.15.45:0/2061793061 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e0407f3f0 con 0x7f0e0407c8e0 2024-05-08T05:51:49.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.869+0000 7f0dfffff640 1 -- 172.21.15.45:0/2061793061 <== mon.1 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7f0e0c097ff0 con 0x7f0e1815c3f0 2024-05-08T05:51:49.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.869+0000 7f0e17fff640 1 --2- 172.21.15.45:0/2061793061 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x7f0e0407c8e0 0x7f0e0407ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:49.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.869+0000 7f0e177fe640 1 --2- 172.21.15.45:0/2061793061 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0e04076300 0x7f0e040787c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0e08000f60 tx=0x7f0e08009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:49.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.869+0000 7f0e17fff640 1 --2- 172.21.15.45:0/2061793061 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x7f0e0407c8e0 0x7f0e0407ed20 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:49.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.870+0000 7f0dfffff640 1 -- 172.21.15.45:0/2061793061 <== osd.1 v2:172.21.15.45:6824/1337179184 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e0407f3f0 con 0x7f0e0407c8e0 2024-05-08T05:51:49.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.883+0000 7f0e1dfaf640 1 -- 172.21.15.45:0/2061793061 --> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0e180b2690 con 0x7f0e0407c8e0 2024-05-08T05:51:49.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.883+0000 7f0dfffff640 1 -- 172.21.15.45:0/2061793061 <== osd.1 v2:172.21.15.45:6824/1337179184 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0e180b2690 con 0x7f0e0407c8e0 2024-05-08T05:51:49.884 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:51:49.884 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T05:51:49.884 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:51:49.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.884+0000 7f0e1dfaf640 1 -- 172.21.15.45:0/2061793061 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x7f0e0407c8e0 msgr2=0x7f0e0407ed20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:49.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.884+0000 7f0e1dfaf640 1 --2- 172.21.15.45:0/2061793061 >> [v2:172.21.15.45:6824/1337179184,v1:172.21.15.45:6825/1337179184] conn(0x7f0e0407c8e0 0x7f0e0407ed20 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.884+0000 7f0e1dfaf640 1 -- 172.21.15.45:0/2061793061 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0e04076300 msgr2=0x7f0e040787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:49.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.884+0000 7f0e1dfaf640 1 --2- 172.21.15.45:0/2061793061 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0e04076300 0x7f0e040787c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0e08000f60 tx=0x7f0e08009040 comp rx=0 tx=0).stop 2024-05-08T05:51:49.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.884+0000 7f0e1dfaf640 1 -- 172.21.15.45:0/2061793061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e1815c3f0 msgr2=0x7f0e1809ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:49.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.884+0000 7f0e1dfaf640 1 --2- 172.21.15.45:0/2061793061 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e1815c3f0 0x7f0e1809ba40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00e8e0 tx=0x7f0e0c00eda0 comp rx=0 tx=0).stop 2024-05-08T05:51:49.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.884+0000 7f0e17fff640 1 -- 172.21.15.45:0/2061793061 reap_dead start 2024-05-08T05:51:49.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.884+0000 7f0e1dfaf640 1 -- 172.21.15.45:0/2061793061 shutdown_connections 2024-05-08T05:51:49.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.884+0000 7f0e1dfaf640 1 -- 172.21.15.45:0/2061793061 >> 172.21.15.45:0/2061793061 conn(0x7f0e1806c3a0 msgr2=0x7f0e18154380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:49.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.884+0000 7f0e1dfaf640 1 -- 172.21.15.45:0/2061793061 shutdown_connections 2024-05-08T05:51:49.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.884+0000 7f0e1dfaf640 1 -- 172.21.15.45:0/2061793061 wait complete. 2024-05-08T05:51:49.895 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-08T05:51:49.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.999+0000 7f9386607640 1 Processor -- start 2024-05-08T05:51:49.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:49.999+0000 7f9386607640 1 -- start start 2024-05-08T05:51:50.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.000+0000 7f9386607640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f93801638e0 0x7f93800b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.000+0000 7f9386607640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f938006c3a0 0x7f938006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.000+0000 7f9386607640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93800b14e0 con 0x7f938015f0c0 2024-05-08T05:51:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.000+0000 7f9386607640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f9380163450 con 0x7f938006c3a0 2024-05-08T05:51:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.000+0000 7f9386607640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f938006cd20 con 0x7f93801638e0 2024-05-08T05:51:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.000+0000 7f9384ba4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f938006c3a0 0x7f938006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.000+0000 7f9384ba4640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f938006c3a0 0x7f938006c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54486/0 (socket says 172.21.15.45:54486) 2024-05-08T05:51:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.000+0000 7f9384ba4640 1 -- 172.21.15.45:0/647453924 learned_addr learned my addr 172.21.15.45:0/647453924 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.000+0000 7f937f7fe640 1 --2- 172.21.15.45:0/647453924 >> v2:172.21.15.63:3300/0 conn(0x7f93801638e0 0x7f93800b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.000+0000 7f9384ba4640 1 -- 172.21.15.45:0/647453924 >> v2:172.21.15.63:3300/0 conn(0x7f93801638e0 msgr2=0x7f93800b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.000+0000 7f9384ba4640 1 --2- 172.21.15.45:0/647453924 >> v2:172.21.15.63:3300/0 conn(0x7f93801638e0 0x7f93800b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.000+0000 7f9384ba4640 1 -- 172.21.15.45:0/647453924 >> v1:172.21.15.63:6789/0 conn(0x7f938015f0c0 legacy=0x7f93800b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.000+0000 7f9384ba4640 1 -- 172.21.15.45:0/647453924 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93801604b0 con 0x7f938006c3a0 2024-05-08T05:51:50.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.001+0000 7f9384ba4640 1 --2- 172.21.15.45:0/647453924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f938006c3a0 0x7f938006c7e0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f938006ea70 tx=0x7f937802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8deb87f98cac57bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:50.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.001+0000 7f937effd640 1 -- 172.21.15.45:0/647453924 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9378036070 con 0x7f938006c3a0 2024-05-08T05:51:50.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.001+0000 7f937effd640 1 -- 172.21.15.45:0/647453924 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9378031600 con 0x7f938006c3a0 2024-05-08T05:51:50.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.001+0000 7f937effd640 1 -- 172.21.15.45:0/647453924 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93780319d0 con 0x7f938006c3a0 2024-05-08T05:51:50.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.002+0000 7f9386607640 1 -- 172.21.15.45:0/647453924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f938006c3a0 msgr2=0x7f938006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.002+0000 7f9386607640 1 --2- 172.21.15.45:0/647453924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f938006c3a0 0x7f938006c7e0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f938006ea70 tx=0x7f937802a730 comp rx=0 tx=0).stop 2024-05-08T05:51:50.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.002+0000 7f9386607640 1 -- 172.21.15.45:0/647453924 shutdown_connections 2024-05-08T05:51:50.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.002+0000 7f9386607640 1 --2- 172.21.15.45:0/647453924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f938006c3a0 0x7f938006c7e0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.002+0000 7f9386607640 1 --2- 172.21.15.45:0/647453924 >> v2:172.21.15.63:3300/0 conn(0x7f93801638e0 0x7f93800b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.002+0000 7f9386607640 1 -- 172.21.15.45:0/647453924 >> 172.21.15.45:0/647453924 conn(0x7f93800b3b10 msgr2=0x7f93800b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:50.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.002+0000 7f9386607640 1 -- 172.21.15.45:0/647453924 shutdown_connections 2024-05-08T05:51:50.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.002+0000 7f9386607640 1 -- 172.21.15.45:0/647453924 wait complete. 2024-05-08T05:51:50.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.003+0000 7f9386607640 1 Processor -- start 2024-05-08T05:51:50.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.003+0000 7f9386607640 1 -- start start 2024-05-08T05:51:50.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.004+0000 7f9386607640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f938006c3a0 0x7f9380146870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.004+0000 7f9386607640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f938015f0c0 0x7f93800b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.004+0000 7f9386607640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f93801638e0 0x7f93800b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.004+0000 7f9386607640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f93800b17c0 con 0x7f938006c3a0 2024-05-08T05:51:50.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.004+0000 7f9386607640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f93801630e0 con 0x7f938015f0c0 2024-05-08T05:51:50.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.004+0000 7f9386607640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f938015f4a0 con 0x7f93801638e0 2024-05-08T05:51:50.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.004+0000 7f937ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f938006c3a0 0x7f9380146870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.004+0000 7f937ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f938006c3a0 0x7f9380146870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54494/0 (socket says 172.21.15.45:54494) 2024-05-08T05:51:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.004+0000 7f937ffff640 1 -- 172.21.15.45:0/2254880728 learned_addr learned my addr 172.21.15.45:0/2254880728 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.004+0000 7f937f7fe640 1 --2- 172.21.15.45:0/2254880728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f938015f0c0 0x7f93800b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.004+0000 7f9384ba4640 1 --2- 172.21.15.45:0/2254880728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f93801638e0 0x7f93800b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.004+0000 7f937ffff640 1 -- 172.21.15.45:0/2254880728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f93801638e0 msgr2=0x7f93800b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.004+0000 7f937ffff640 1 --2- 172.21.15.45:0/2254880728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f93801638e0 0x7f93800b2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.004+0000 7f937ffff640 1 -- 172.21.15.45:0/2254880728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f938015f0c0 msgr2=0x7f93800b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.004+0000 7f937ffff640 1 --2- 172.21.15.45:0/2254880728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f938015f0c0 0x7f93800b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.004+0000 7f937ffff640 1 -- 172.21.15.45:0/2254880728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93801f6430 con 0x7f938006c3a0 2024-05-08T05:51:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.004+0000 7f937f7fe640 1 --2- 172.21.15.45:0/2254880728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f938015f0c0 0x7f93800b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.005+0000 7f937ffff640 1 --2- 172.21.15.45:0/2254880728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f938006c3a0 0x7f9380146870 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f937000c480 tx=0x7f937000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.005+0000 7f937cff9640 1 -- 172.21.15.45:0/2254880728 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93700052d0 con 0x7f938006c3a0 2024-05-08T05:51:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.005+0000 7f937cff9640 1 -- 172.21.15.45:0/2254880728 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9370005470 con 0x7f938006c3a0 2024-05-08T05:51:50.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.005+0000 7f9386607640 1 -- 172.21.15.45:0/2254880728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93801470e0 con 0x7f938006c3a0 2024-05-08T05:51:50.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.005+0000 7f937cff9640 1 -- 172.21.15.45:0/2254880728 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9370012870 con 0x7f938006c3a0 2024-05-08T05:51:50.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.005+0000 7f9386607640 1 -- 172.21.15.45:0/2254880728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93801f56f0 con 0x7f938006c3a0 2024-05-08T05:51:50.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.006+0000 7f9386607640 1 -- 172.21.15.45:0/2254880728 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f938006a8f0 con 0x7f938006c3a0 2024-05-08T05:51:50.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.006+0000 7f937cff9640 1 -- 172.21.15.45:0/2254880728 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9370012a10 con 0x7f938006c3a0 2024-05-08T05:51:50.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.007+0000 7f937cff9640 1 --2- 172.21.15.45:0/2254880728 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f936c0762d0 0x7f936c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.007+0000 7f937cff9640 1 -- 172.21.15.45:0/2254880728 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f9370097060 con 0x7f938006c3a0 2024-05-08T05:51:50.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.007+0000 7f937f7fe640 1 --2- 172.21.15.45:0/2254880728 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f936c0762d0 0x7f936c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.008+0000 7f937f7fe640 1 --2- 172.21.15.45:0/2254880728 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f936c0762d0 0x7f936c078790 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f938015c6d0 tx=0x7f9374009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:50.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.011+0000 7f937cff9640 1 -- 172.21.15.45:0/2254880728 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9370060440 con 0x7f938006c3a0 2024-05-08T05:51:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.205+0000 7f9386607640 1 -- 172.21.15.45:0/2254880728 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f93800693f0 con 0x7f936c0762d0 2024-05-08T05:51:50.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.205+0000 7f937cff9640 1 -- 172.21.15.45:0/2254880728 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f93800693f0 con 0x7f936c0762d0 2024-05-08T05:51:50.206 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 1 to deep-scrub 2024-05-08T05:51:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.207+0000 7f9386607640 1 -- 172.21.15.45:0/2254880728 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f936c0762d0 msgr2=0x7f936c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.207+0000 7f9386607640 1 --2- 172.21.15.45:0/2254880728 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f936c0762d0 0x7f936c078790 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f938015c6d0 tx=0x7f9374009190 comp rx=0 tx=0).stop 2024-05-08T05:51:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.207+0000 7f9386607640 1 -- 172.21.15.45:0/2254880728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f938006c3a0 msgr2=0x7f9380146870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.207+0000 7f9386607640 1 --2- 172.21.15.45:0/2254880728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f938006c3a0 0x7f9380146870 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f937000c480 tx=0x7f937000c940 comp rx=0 tx=0).stop 2024-05-08T05:51:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.207+0000 7f9386607640 1 -- 172.21.15.45:0/2254880728 shutdown_connections 2024-05-08T05:51:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.207+0000 7f9386607640 1 --2- 172.21.15.45:0/2254880728 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f936c0762d0 0x7f936c078790 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.207+0000 7f9386607640 1 --2- 172.21.15.45:0/2254880728 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f93801638e0 0x7f93800b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.207+0000 7f9386607640 1 --2- 172.21.15.45:0/2254880728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f938015f0c0 0x7f93800b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.207+0000 7f9386607640 1 --2- 172.21.15.45:0/2254880728 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f938006c3a0 0x7f9380146870 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.207+0000 7f9386607640 1 -- 172.21.15.45:0/2254880728 >> 172.21.15.45:0/2254880728 conn(0x7f93800b3b10 msgr2=0x7f9380160880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.207+0000 7f9386607640 1 -- 172.21.15.45:0/2254880728 shutdown_connections 2024-05-08T05:51:50.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.207+0000 7f9386607640 1 -- 172.21.15.45:0/2254880728 wait complete. 2024-05-08T05:51:50.218 INFO:tasks.ceph:Scrubbing osd.2 2024-05-08T05:51:50.218 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-08T05:51:50.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.326+0000 7fa95b559640 1 Processor -- start 2024-05-08T05:51:50.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.326+0000 7fa95b559640 1 -- start start 2024-05-08T05:51:50.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.327+0000 7fa95b559640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa95415c3f0 0x7fa95406ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.327+0000 7fa95b559640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fa95415c7d0 0x7fa9540737e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.327+0000 7fa95b559640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa95406ef40 0x7fa954073d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.327+0000 7fa95b559640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fa954072270 con 0x7fa95406ef40 2024-05-08T05:51:50.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.327+0000 7fa95b559640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fa954072550 con 0x7fa95415c7d0 2024-05-08T05:51:50.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.327+0000 7fa95b559640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa954072830 con 0x7fa95415c3f0 2024-05-08T05:51:50.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.327+0000 7fa959af6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa95406ef40 0x7fa954073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.327+0000 7fa959af6640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fa95406ef40 0x7fa954073d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54502/0 (socket says 172.21.15.45:54502) 2024-05-08T05:51:50.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.327+0000 7fa959af6640 1 -- 172.21.15.45:0/3153225062 learned_addr learned my addr 172.21.15.45:0/3153225062 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:50.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.328+0000 7fa958af4640 1 --2- 172.21.15.45:0/3153225062 >> v2:172.21.15.63:3300/0 conn(0x7fa95415c7d0 0x7fa9540737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.328+0000 7fa9592f5640 1 --2- 172.21.15.45:0/3153225062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa95415c3f0 0x7fa95406ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.328+0000 7fa959af6640 1 -- 172.21.15.45:0/3153225062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa95415c3f0 msgr2=0x7fa95406ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:50.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.328+0000 7fa959af6640 1 --2- 172.21.15.45:0/3153225062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa95415c3f0 0x7fa95406ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.328+0000 7fa959af6640 1 -- 172.21.15.45:0/3153225062 >> v2:172.21.15.63:3300/0 conn(0x7fa95415c7d0 msgr2=0x7fa9540737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:50.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.328+0000 7fa959af6640 1 --2- 172.21.15.45:0/3153225062 >> v2:172.21.15.63:3300/0 conn(0x7fa95415c7d0 0x7fa9540737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.328+0000 7fa959af6640 1 -- 172.21.15.45:0/3153225062 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa954072b10 con 0x7fa95406ef40 2024-05-08T05:51:50.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.328+0000 7fa9592f5640 1 --2- 172.21.15.45:0/3153225062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa95415c3f0 0x7fa95406ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:50.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.328+0000 7fa959af6640 1 --2- 172.21.15.45:0/3153225062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95406ef40 0x7fa954073d20 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fa9540d1480 tx=0x7fa94c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cc89e9a1b4afcf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:50.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.329+0000 7fa94bfff640 1 -- 172.21.15.45:0/3153225062 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c037070 con 0x7fa95406ef40 2024-05-08T05:51:50.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.329+0000 7fa94bfff640 1 -- 172.21.15.45:0/3153225062 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa94c032440 con 0x7fa95406ef40 2024-05-08T05:51:50.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.329+0000 7fa94bfff640 1 -- 172.21.15.45:0/3153225062 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c032810 con 0x7fa95406ef40 2024-05-08T05:51:50.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.329+0000 7fa95b559640 1 -- 172.21.15.45:0/3153225062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95406ef40 msgr2=0x7fa954073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.329+0000 7fa95b559640 1 --2- 172.21.15.45:0/3153225062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95406ef40 0x7fa954073d20 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fa9540d1480 tx=0x7fa94c02a270 comp rx=0 tx=0).stop 2024-05-08T05:51:50.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.329+0000 7fa95b559640 1 -- 172.21.15.45:0/3153225062 shutdown_connections 2024-05-08T05:51:50.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.329+0000 7fa95b559640 1 --2- 172.21.15.45:0/3153225062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95406ef40 0x7fa954073d20 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.329+0000 7fa95b559640 1 --2- 172.21.15.45:0/3153225062 >> v2:172.21.15.63:3300/0 conn(0x7fa95415c7d0 0x7fa9540737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.329+0000 7fa95b559640 1 --2- 172.21.15.45:0/3153225062 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa95415c3f0 0x7fa95406ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.329+0000 7fa95b559640 1 -- 172.21.15.45:0/3153225062 >> 172.21.15.45:0/3153225062 conn(0x7fa95406c3a0 msgr2=0x7fa95406c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:50.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.329+0000 7fa95b559640 1 -- 172.21.15.45:0/3153225062 shutdown_connections 2024-05-08T05:51:50.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.329+0000 7fa95b559640 1 -- 172.21.15.45:0/3153225062 wait complete. 2024-05-08T05:51:50.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.330+0000 7fa95b559640 1 Processor -- start 2024-05-08T05:51:50.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.331+0000 7fa95b559640 1 -- start start 2024-05-08T05:51:50.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.331+0000 7fa95b559640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa95406ef40 0x7fa95414b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.331+0000 7fa95b559640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa95415c3f0 0x7fa954157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.331+0000 7fa95b559640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95415c7d0 0x7fa9541f3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.331+0000 7fa95b559640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa9540d0bf0 con 0x7fa95415c7d0 2024-05-08T05:51:50.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.331+0000 7fa95b559640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fa954073310 con 0x7fa95406ef40 2024-05-08T05:51:50.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.331+0000 7fa95b559640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fa9540d0d70 con 0x7fa95415c3f0 2024-05-08T05:51:50.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.331+0000 7fa959af6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95415c7d0 0x7fa9541f3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.331+0000 7fa959af6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95415c7d0 0x7fa9541f3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54514/0 (socket says 172.21.15.45:54514) 2024-05-08T05:51:50.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.331+0000 7fa959af6640 1 -- 172.21.15.45:0/2893710846 learned_addr learned my addr 172.21.15.45:0/2893710846 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:50.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.332+0000 7fa9592f5640 1 --2- 172.21.15.45:0/2893710846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa95406ef40 0x7fa95414b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.332+0000 7fa958af4640 1 --2- 172.21.15.45:0/2893710846 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa95415c3f0 0x7fa954157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.332+0000 7fa959af6640 1 -- 172.21.15.45:0/2893710846 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa95415c3f0 msgr2=0x7fa954157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.332+0000 7fa959af6640 1 --2- 172.21.15.45:0/2893710846 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa95415c3f0 0x7fa954157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.332+0000 7fa959af6640 1 -- 172.21.15.45:0/2893710846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa95406ef40 msgr2=0x7fa95414b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.332+0000 7fa959af6640 1 --2- 172.21.15.45:0/2893710846 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa95406ef40 0x7fa95414b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.332+0000 7fa959af6640 1 -- 172.21.15.45:0/2893710846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa954158310 con 0x7fa95415c7d0 2024-05-08T05:51:50.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.332+0000 7fa958af4640 1 --2- 172.21.15.45:0/2893710846 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fa95415c3f0 0x7fa954157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:50.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.332+0000 7fa959af6640 1 --2- 172.21.15.45:0/2893710846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95415c7d0 0x7fa9541f3400 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fa94c02a2b0 tx=0x7fa94c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:50.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.332+0000 7fa949ffb640 1 -- 172.21.15.45:0/2893710846 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c041020 con 0x7fa95415c7d0 2024-05-08T05:51:50.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.332+0000 7fa949ffb640 1 -- 172.21.15.45:0/2893710846 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa94c03d640 con 0x7fa95415c7d0 2024-05-08T05:51:50.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.332+0000 7fa95b559640 1 -- 172.21.15.45:0/2893710846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa954156250 con 0x7fa95415c7d0 2024-05-08T05:51:50.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.332+0000 7fa949ffb640 1 -- 172.21.15.45:0/2893710846 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c037040 con 0x7fa95415c7d0 2024-05-08T05:51:50.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.332+0000 7fa95b559640 1 -- 172.21.15.45:0/2893710846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa954156810 con 0x7fa95415c7d0 2024-05-08T05:51:50.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.333+0000 7fa95b559640 1 -- 172.21.15.45:0/2893710846 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa95406fc80 con 0x7fa95415c7d0 2024-05-08T05:51:50.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.334+0000 7fa949ffb640 1 -- 172.21.15.45:0/2893710846 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa94c044020 con 0x7fa95415c7d0 2024-05-08T05:51:50.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.334+0000 7fa949ffb640 1 --2- 172.21.15.45:0/2893710846 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa9380762d0 0x7fa938078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.334+0000 7fa949ffb640 1 -- 172.21.15.45:0/2893710846 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7fa94c0b7c20 con 0x7fa95415c7d0 2024-05-08T05:51:50.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.335+0000 7fa949ffb640 1 --2- 172.21.15.45:0/2893710846 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x7fa93807c8b0 0x7fa93807ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.335+0000 7fa949ffb640 1 -- 172.21.15.45:0/2893710846 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa94c032440 con 0x7fa93807c8b0 2024-05-08T05:51:50.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.335+0000 7fa949ffb640 1 -- 172.21.15.45:0/2893710846 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7fa94c0ba320 con 0x7fa95415c7d0 2024-05-08T05:51:50.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.335+0000 7fa9592f5640 1 --2- 172.21.15.45:0/2893710846 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa9380762d0 0x7fa938078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.335+0000 7fa958af4640 1 --2- 172.21.15.45:0/2893710846 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x7fa93807c8b0 0x7fa93807ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.335+0000 7fa958af4640 1 --2- 172.21.15.45:0/2893710846 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x7fa93807c8b0 0x7fa93807ecf0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:50.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.335+0000 7fa9592f5640 1 --2- 172.21.15.45:0/2893710846 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa9380762d0 0x7fa938078790 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa950003fd0 tx=0x7fa950003ce0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:50.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.336+0000 7fa949ffb640 1 -- 172.21.15.45:0/2893710846 <== osd.2 v2:172.21.15.45:6808/785834340 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa94c032440 con 0x7fa93807c8b0 2024-05-08T05:51:50.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.349+0000 7fa95b559640 1 -- 172.21.15.45:0/2893710846 --> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa9540693f0 con 0x7fa93807c8b0 2024-05-08T05:51:50.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.349+0000 7fa949ffb640 1 -- 172.21.15.45:0/2893710846 <== osd.2 v2:172.21.15.45:6808/785834340 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa9540693f0 con 0x7fa93807c8b0 2024-05-08T05:51:50.349 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:51:50.349 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T05:51:50.349 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:51:50.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.349+0000 7fa95b559640 1 -- 172.21.15.45:0/2893710846 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x7fa93807c8b0 msgr2=0x7fa93807ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.349+0000 7fa95b559640 1 --2- 172.21.15.45:0/2893710846 >> [v2:172.21.15.45:6808/785834340,v1:172.21.15.45:6809/785834340] conn(0x7fa93807c8b0 0x7fa93807ecf0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.350+0000 7fa95b559640 1 -- 172.21.15.45:0/2893710846 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa9380762d0 msgr2=0x7fa938078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.350+0000 7fa95b559640 1 --2- 172.21.15.45:0/2893710846 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fa9380762d0 0x7fa938078790 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa950003fd0 tx=0x7fa950003ce0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.350+0000 7fa95b559640 1 -- 172.21.15.45:0/2893710846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95415c7d0 msgr2=0x7fa9541f3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.350+0000 7fa95b559640 1 --2- 172.21.15.45:0/2893710846 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa95415c7d0 0x7fa9541f3400 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fa94c02a2b0 tx=0x7fa94c0046c0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.350+0000 7fa959af6640 1 -- 172.21.15.45:0/2893710846 reap_dead start 2024-05-08T05:51:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.350+0000 7fa95b559640 1 -- 172.21.15.45:0/2893710846 shutdown_connections 2024-05-08T05:51:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.350+0000 7fa95b559640 1 -- 172.21.15.45:0/2893710846 >> 172.21.15.45:0/2893710846 conn(0x7fa95406c3a0 msgr2=0x7fa954155790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.350+0000 7fa95b559640 1 -- 172.21.15.45:0/2893710846 shutdown_connections 2024-05-08T05:51:50.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.350+0000 7fa95b559640 1 -- 172.21.15.45:0/2893710846 wait complete. 2024-05-08T05:51:50.360 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-08T05:51:50.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.465+0000 7f42a5054640 1 Processor -- start 2024-05-08T05:51:50.465 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.465+0000 7f42a5054640 1 -- start start 2024-05-08T05:51:50.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.466+0000 7f42a5054640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f42a01524f0 0x7f42a00b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.466+0000 7f42a5054640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42a006eaa0 0x7f42a0073700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.466+0000 7f42a5054640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42a00b4590 con 0x7f42a015bc80 2024-05-08T05:51:50.467 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.466+0000 7f42a5054640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f42a00b51a0 con 0x7f42a01524f0 2024-05-08T05:51:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.466+0000 7f42a5054640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f42a0073450 con 0x7f42a006eaa0 2024-05-08T05:51:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.466+0000 7f429ed9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f42a01524f0 0x7f42a00b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.466+0000 7f429e59c640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f42a015bc80 0x7f42a006e390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:38778/0 (socket says 172.21.15.45:38778) 2024-05-08T05:51:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.466+0000 7f429ed9d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f42a01524f0 0x7f42a00b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54520/0 (socket says 172.21.15.45:54520) 2024-05-08T05:51:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.466+0000 7f429e59c640 1 -- 172.21.15.45:0/1103663909 learned_addr learned my addr 172.21.15.45:0/1103663909 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.466+0000 7f429f59e640 1 --2- 172.21.15.45:0/1103663909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42a006eaa0 0x7f42a0073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.466+0000 7f429ed9d640 1 -- 172.21.15.45:0/1103663909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42a006eaa0 msgr2=0x7f42a0073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.466+0000 7f429ed9d640 1 --2- 172.21.15.45:0/1103663909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42a006eaa0 0x7f42a0073700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.466+0000 7f429ed9d640 1 -- 172.21.15.45:0/1103663909 >> v1:172.21.15.45:6789/0 conn(0x7f42a015bc80 legacy=0x7f42a006e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.468 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.466+0000 7f429dd9b640 1 -- 172.21.15.45:0/1103663909 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2074192571 0 0) 0x7f42a00b4590 con 0x7f42a015bc80 2024-05-08T05:51:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.466+0000 7f429ed9d640 1 -- 172.21.15.45:0/1103663909 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a00b4870 con 0x7f42a01524f0 2024-05-08T05:51:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.467+0000 7f429ed9d640 1 --2- 172.21.15.45:0/1103663909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a01524f0 0x7f42a00b3c20 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f429000a940 tx=0x7f429002b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6dfa2edcd2cdaa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.467+0000 7f429dd9b640 1 -- 172.21.15.45:0/1103663909 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429000b040 con 0x7f42a01524f0 2024-05-08T05:51:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.467+0000 7f429dd9b640 1 -- 172.21.15.45:0/1103663909 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4290030710 con 0x7f42a01524f0 2024-05-08T05:51:50.469 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.467+0000 7f429dd9b640 1 -- 172.21.15.45:0/1103663909 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290030ae0 con 0x7f42a01524f0 2024-05-08T05:51:50.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.468+0000 7f42a5054640 1 -- 172.21.15.45:0/1103663909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a01524f0 msgr2=0x7f42a00b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.468+0000 7f42a5054640 1 --2- 172.21.15.45:0/1103663909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a01524f0 0x7f42a00b3c20 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f429000a940 tx=0x7f429002b550 comp rx=0 tx=0).stop 2024-05-08T05:51:50.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.468+0000 7f42a5054640 1 -- 172.21.15.45:0/1103663909 shutdown_connections 2024-05-08T05:51:50.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.468+0000 7f42a5054640 1 --2- 172.21.15.45:0/1103663909 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42a006eaa0 0x7f42a0073700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.468+0000 7f42a5054640 1 --2- 172.21.15.45:0/1103663909 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a01524f0 0x7f42a00b3c20 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.468+0000 7f42a5054640 1 -- 172.21.15.45:0/1103663909 >> 172.21.15.45:0/1103663909 conn(0x7f42a006c3a0 msgr2=0x7f42a006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:50.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.468+0000 7f42a5054640 1 -- 172.21.15.45:0/1103663909 shutdown_connections 2024-05-08T05:51:50.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.468+0000 7f42a5054640 1 -- 172.21.15.45:0/1103663909 wait complete. 2024-05-08T05:51:50.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.469+0000 7f42a5054640 1 Processor -- start 2024-05-08T05:51:50.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.469+0000 7f42a5054640 1 -- start start 2024-05-08T05:51:50.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.470+0000 7f42a5054640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42a006eaa0 0x7f42a00cfba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.470+0000 7f42a5054640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42a01524f0 0x7f42a01517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.470+0000 7f42a5054640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a015bc80 0x7f42a00b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.470+0000 7f42a5054640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f42a01528d0 con 0x7f42a015bc80 2024-05-08T05:51:50.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.470+0000 7f42a5054640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f42a00b4e30 con 0x7f42a006eaa0 2024-05-08T05:51:50.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.470+0000 7f42a5054640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f42a00b5630 con 0x7f42a01524f0 2024-05-08T05:51:50.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.470+0000 7f429f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a015bc80 0x7f42a00b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.470+0000 7f429f59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a015bc80 0x7f42a00b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54532/0 (socket says 172.21.15.45:54532) 2024-05-08T05:51:50.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.470+0000 7f429f59e640 1 -- 172.21.15.45:0/1716632960 learned_addr learned my addr 172.21.15.45:0/1716632960 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:50.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.470+0000 7f429ed9d640 1 --2- 172.21.15.45:0/1716632960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42a006eaa0 0x7f42a00cfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.470+0000 7f429e59c640 1 --2- 172.21.15.45:0/1716632960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42a01524f0 0x7f42a01517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.471+0000 7f429f59e640 1 -- 172.21.15.45:0/1716632960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42a01524f0 msgr2=0x7f42a01517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.471+0000 7f429f59e640 1 --2- 172.21.15.45:0/1716632960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42a01524f0 0x7f42a01517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.471+0000 7f429f59e640 1 -- 172.21.15.45:0/1716632960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42a006eaa0 msgr2=0x7f42a00cfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.471+0000 7f429f59e640 1 --2- 172.21.15.45:0/1716632960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42a006eaa0 0x7f42a00cfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.471+0000 7f429f59e640 1 -- 172.21.15.45:0/1716632960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a00b1ee0 con 0x7f42a015bc80 2024-05-08T05:51:50.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.471+0000 7f429f59e640 1 --2- 172.21.15.45:0/1716632960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a015bc80 0x7f42a00b1640 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f429800efc0 tx=0x7f429800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:50.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.471+0000 7f42977fe640 1 -- 172.21.15.45:0/1716632960 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4298018020 con 0x7f42a015bc80 2024-05-08T05:51:50.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.471+0000 7f42977fe640 1 -- 172.21.15.45:0/1716632960 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4298014480 con 0x7f42a015bc80 2024-05-08T05:51:50.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.471+0000 7f42a5054640 1 -- 172.21.15.45:0/1716632960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a00cf0c0 con 0x7f42a015bc80 2024-05-08T05:51:50.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.471+0000 7f42977fe640 1 -- 172.21.15.45:0/1716632960 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4298014870 con 0x7f42a015bc80 2024-05-08T05:51:50.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.471+0000 7f42a5054640 1 -- 172.21.15.45:0/1716632960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a00cf680 con 0x7f42a015bc80 2024-05-08T05:51:50.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.472+0000 7f42a5054640 1 -- 172.21.15.45:0/1716632960 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42a00cf880 con 0x7f42a015bc80 2024-05-08T05:51:50.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.473+0000 7f42977fe640 1 -- 172.21.15.45:0/1716632960 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4298005d10 con 0x7f42a015bc80 2024-05-08T05:51:50.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.473+0000 7f42977fe640 1 --2- 172.21.15.45:0/1716632960 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f42800762d0 0x7f4280078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.473+0000 7f42977fe640 1 -- 172.21.15.45:0/1716632960 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f4298098200 con 0x7f42a015bc80 2024-05-08T05:51:50.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.473+0000 7f429ed9d640 1 --2- 172.21.15.45:0/1716632960 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f42800762d0 0x7f4280078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.474+0000 7f429ed9d640 1 --2- 172.21.15.45:0/1716632960 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f42800762d0 0x7f4280078790 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4290035000 tx=0x7f429000c040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:50.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.478+0000 7f42977fe640 1 -- 172.21.15.45:0/1716632960 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f42a00cf880 con 0x7f42a015bc80 2024-05-08T05:51:50.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.667+0000 7f42a5054640 1 -- 172.21.15.45:0/1716632960 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f42a0151d40 con 0x7f42800762d0 2024-05-08T05:51:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.668+0000 7f42977fe640 1 -- 172.21.15.45:0/1716632960 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f42a0151d40 con 0x7f42800762d0 2024-05-08T05:51:50.668 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 2 to deep-scrub 2024-05-08T05:51:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.670+0000 7f42a5054640 1 -- 172.21.15.45:0/1716632960 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f42800762d0 msgr2=0x7f4280078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.670+0000 7f42a5054640 1 --2- 172.21.15.45:0/1716632960 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f42800762d0 0x7f4280078790 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4290035000 tx=0x7f429000c040 comp rx=0 tx=0).stop 2024-05-08T05:51:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.670+0000 7f42a5054640 1 -- 172.21.15.45:0/1716632960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a015bc80 msgr2=0x7f42a00b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.670+0000 7f42a5054640 1 --2- 172.21.15.45:0/1716632960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a015bc80 0x7f42a00b1640 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f429800efc0 tx=0x7f429800c490 comp rx=0 tx=0).stop 2024-05-08T05:51:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.670+0000 7f42a5054640 1 -- 172.21.15.45:0/1716632960 shutdown_connections 2024-05-08T05:51:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.670+0000 7f42a5054640 1 --2- 172.21.15.45:0/1716632960 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f42800762d0 0x7f4280078790 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.670+0000 7f42a5054640 1 --2- 172.21.15.45:0/1716632960 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f42a01524f0 0x7f42a01517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.670+0000 7f42a5054640 1 --2- 172.21.15.45:0/1716632960 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f42a015bc80 0x7f42a00b1640 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.670+0000 7f42a5054640 1 --2- 172.21.15.45:0/1716632960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f42a006eaa0 0x7f42a00cfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.670+0000 7f42a5054640 1 -- 172.21.15.45:0/1716632960 >> 172.21.15.45:0/1716632960 conn(0x7f42a006c3a0 msgr2=0x7f42a0072c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:50.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.670+0000 7f42a5054640 1 -- 172.21.15.45:0/1716632960 shutdown_connections 2024-05-08T05:51:50.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.670+0000 7f42a5054640 1 -- 172.21.15.45:0/1716632960 wait complete. 2024-05-08T05:51:50.681 INFO:tasks.ceph:Scrubbing osd.3 2024-05-08T05:51:50.681 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-08T05:51:50.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.788+0000 7f96e75b6640 1 Processor -- start 2024-05-08T05:51:50.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.788+0000 7f96e75b6640 1 -- start start 2024-05-08T05:51:50.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.789+0000 7f96e75b6640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f96e0162b70 0x7f96e00d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.789+0000 7f96e75b6640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96e00d3910 0x7f96e00d0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.789+0000 7f96e75b6640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96e0162f50 con 0x7f96e00d3480 2024-05-08T05:51:50.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.789+0000 7f96e75b6640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f96e00d11a0 con 0x7f96e0162b70 2024-05-08T05:51:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.789+0000 7f96e75b6640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f96e00d1f60 con 0x7f96e00d3910 2024-05-08T05:51:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.790+0000 7f96e5352640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f96e0162b70 0x7f96e00d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.790+0000 7f96e4b51640 1 --1- >> v1:172.21.15.63:6789/0 conn(0x7f96e00d3480 0x7f96e00b4a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.63:6789/0 says I am v1:172.21.15.45:60096/0 (socket says 172.21.15.45:60096) 2024-05-08T05:51:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.790+0000 7f96e5352640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f96e0162b70 0x7f96e00d2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:40762/0 (socket says 172.21.15.45:40762) 2024-05-08T05:51:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.790+0000 7f96e4b51640 1 -- 172.21.15.45:0/948841918 learned_addr learned my addr 172.21.15.45:0/948841918 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.790+0000 7f96e5b53640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96e00d3910 0x7f96e00d0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.790+0000 7f96e5352640 1 -- 172.21.15.45:0/948841918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96e00d3910 msgr2=0x7f96e00d0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.790+0000 7f96e5352640 1 --2- 172.21.15.45:0/948841918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96e00d3910 0x7f96e00d0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.790+0000 7f96e5352640 1 -- 172.21.15.45:0/948841918 >> v1:172.21.15.63:6789/0 conn(0x7f96e00d3480 legacy=0x7f96e00b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.790+0000 7f96e5352640 1 -- 172.21.15.45:0/948841918 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e00d3ee0 con 0x7f96e0162b70 2024-05-08T05:51:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.790+0000 7f96dffff640 1 -- 172.21.15.45:0/948841918 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1514661971 0 0) 0x7f96d4001110 con 0x7f96e00d3480 2024-05-08T05:51:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.790+0000 7f96e5b53640 1 --2- 172.21.15.45:0/948841918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96e00d3910 0x7f96e00d0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:50.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.791+0000 7f96e5352640 1 --2- 172.21.15.45:0/948841918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e0162b70 0x7f96e00d2f40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f96d80097a0 tx=0x7f96d802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a1e3aeba4e43913 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:50.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.791+0000 7f96dffff640 1 -- 172.21.15.45:0/948841918 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8037070 con 0x7f96e0162b70 2024-05-08T05:51:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.791+0000 7f96dffff640 1 -- 172.21.15.45:0/948841918 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d8032440 con 0x7f96e0162b70 2024-05-08T05:51:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.791+0000 7f96dffff640 1 -- 172.21.15.45:0/948841918 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8032810 con 0x7f96e0162b70 2024-05-08T05:51:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.791+0000 7f96e75b6640 1 -- 172.21.15.45:0/948841918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e0162b70 msgr2=0x7f96e00d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.791+0000 7f96e75b6640 1 --2- 172.21.15.45:0/948841918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e0162b70 0x7f96e00d2f40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f96d80097a0 tx=0x7f96d802a180 comp rx=0 tx=0).stop 2024-05-08T05:51:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.791+0000 7f96e75b6640 1 -- 172.21.15.45:0/948841918 shutdown_connections 2024-05-08T05:51:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.791+0000 7f96e75b6640 1 --2- 172.21.15.45:0/948841918 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96e00d3910 0x7f96e00d0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.791+0000 7f96e75b6640 1 --2- 172.21.15.45:0/948841918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e0162b70 0x7f96e00d2f40 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.791+0000 7f96e75b6640 1 -- 172.21.15.45:0/948841918 >> 172.21.15.45:0/948841918 conn(0x7f96e00d5040 msgr2=0x7f96e00d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.791+0000 7f96e75b6640 1 -- 172.21.15.45:0/948841918 shutdown_connections 2024-05-08T05:51:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.791+0000 7f96e75b6640 1 -- 172.21.15.45:0/948841918 wait complete. 2024-05-08T05:51:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.793+0000 7f96e75b6640 1 Processor -- start 2024-05-08T05:51:50.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.793+0000 7f96e75b6640 1 -- start start 2024-05-08T05:51:50.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.793+0000 7f96e75b6640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96e00d3480 0x7f96e015e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.793+0000 7f96e75b6640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96e00d3910 0x7f96e01587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.793+0000 7f96e75b6640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e0162b70 0x7f96e01570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.793+0000 7f96e75b6640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f96e00b4170 con 0x7f96e00d3910 2024-05-08T05:51:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.793+0000 7f96e75b6640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f96e00d2490 con 0x7f96e0162b70 2024-05-08T05:51:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.793+0000 7f96e75b6640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f96e00b42f0 con 0x7f96e00d3480 2024-05-08T05:51:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.793+0000 7f96e4b51640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96e00d3910 0x7f96e01587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.794+0000 7f96e4b51640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96e00d3910 0x7f96e01587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43488/0 (socket says 172.21.15.45:43488) 2024-05-08T05:51:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.794+0000 7f96e5352640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96e00d3480 0x7f96e015e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.794+0000 7f96e4b51640 1 -- 172.21.15.45:0/1004485291 learned_addr learned my addr 172.21.15.45:0/1004485291 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.794+0000 7f96e5b53640 1 --2- 172.21.15.45:0/1004485291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e0162b70 0x7f96e01570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.794+0000 7f96e5352640 1 -- 172.21.15.45:0/1004485291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e0162b70 msgr2=0x7f96e01570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.794+0000 7f96e5352640 1 --2- 172.21.15.45:0/1004485291 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96e0162b70 0x7f96e01570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.794+0000 7f96e5352640 1 -- 172.21.15.45:0/1004485291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96e00d3910 msgr2=0x7f96e01587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.794+0000 7f96e5352640 1 --2- 172.21.15.45:0/1004485291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96e00d3910 0x7f96e01587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.794+0000 7f96e5352640 1 -- 172.21.15.45:0/1004485291 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e0158f10 con 0x7f96e00d3480 2024-05-08T05:51:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.794+0000 7f96e4b51640 1 --2- 172.21.15.45:0/1004485291 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96e00d3910 0x7f96e01587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:51:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.794+0000 7f96e5352640 1 --2- 172.21.15.45:0/1004485291 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96e00d3480 0x7f96e015e6f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f96d8031000 tx=0x7f96d80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.794+0000 7f96ddffb640 1 -- 172.21.15.45:0/1004485291 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8041030 con 0x7f96e00d3480 2024-05-08T05:51:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.794+0000 7f96ddffb640 1 -- 172.21.15.45:0/1004485291 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d8030440 con 0x7f96e00d3480 2024-05-08T05:51:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.794+0000 7f96e75b6640 1 -- 172.21.15.45:0/1004485291 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e01591f0 con 0x7f96e00d3480 2024-05-08T05:51:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.794+0000 7f96ddffb640 1 -- 172.21.15.45:0/1004485291 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8037040 con 0x7f96e00d3480 2024-05-08T05:51:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.795+0000 7f96e75b6640 1 -- 172.21.15.45:0/1004485291 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e01597b0 con 0x7f96e00d3480 2024-05-08T05:51:50.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.795+0000 7f96e75b6640 1 -- 172.21.15.45:0/1004485291 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f96e00693f0 con 0x7f96e00d3480 2024-05-08T05:51:50.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.796+0000 7f96ddffb640 1 -- 172.21.15.45:0/1004485291 <== mon.2 v2:172.21.15.63:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f96d8044070 con 0x7f96e00d3480 2024-05-08T05:51:50.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.797+0000 7f96ddffb640 1 --2- 172.21.15.45:0/1004485291 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f96cc0762d0 0x7f96cc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.797+0000 7f96ddffb640 1 -- 172.21.15.45:0/1004485291 <== mon.2 v2:172.21.15.63:3301/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f96d80b8540 con 0x7f96e00d3480 2024-05-08T05:51:50.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.797+0000 7f96e4b51640 1 --2- 172.21.15.45:0/1004485291 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f96cc0762d0 0x7f96cc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.797+0000 7f96ddffb640 1 --2- 172.21.15.45:0/1004485291 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x7f96cc07c8b0 0x7f96cc07ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.797+0000 7f96ddffb640 1 -- 172.21.15.45:0/1004485291 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f96d8040e80 con 0x7f96cc07c8b0 2024-05-08T05:51:50.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.797+0000 7f96ddffb640 1 -- 172.21.15.45:0/1004485291 <== mon.2 v2:172.21.15.63:3301/0 6 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7f96d80b88f0 con 0x7f96e00d3480 2024-05-08T05:51:50.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.797+0000 7f96e5b53640 1 --2- 172.21.15.45:0/1004485291 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x7f96cc07c8b0 0x7f96cc07ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.797+0000 7f96e4b51640 1 --2- 172.21.15.45:0/1004485291 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f96cc0762d0 0x7f96cc078790 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f96e01f8bc0 tx=0x7f96d4007bd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:50.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.797+0000 7f96e5b53640 1 --2- 172.21.15.45:0/1004485291 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x7f96cc07c8b0 0x7f96cc07ecf0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:50.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.798+0000 7f96ddffb640 1 -- 172.21.15.45:0/1004485291 <== osd.3 v2:172.21.15.45:6816/1259225446 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96d8040e80 con 0x7f96cc07c8b0 2024-05-08T05:51:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.811+0000 7f96e75b6640 1 -- 172.21.15.45:0/1004485291 --> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f96e015bb20 con 0x7f96cc07c8b0 2024-05-08T05:51:50.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.811+0000 7f96ddffb640 1 -- 172.21.15.45:0/1004485291 <== osd.3 v2:172.21.15.45:6816/1259225446 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f96e015bb20 con 0x7f96cc07c8b0 2024-05-08T05:51:50.811 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:51:50.812 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T05:51:50.812 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:51:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.812+0000 7f96e75b6640 1 -- 172.21.15.45:0/1004485291 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x7f96cc07c8b0 msgr2=0x7f96cc07ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.812+0000 7f96e75b6640 1 --2- 172.21.15.45:0/1004485291 >> [v2:172.21.15.45:6816/1259225446,v1:172.21.15.45:6817/1259225446] conn(0x7f96cc07c8b0 0x7f96cc07ecf0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.812+0000 7f96e75b6640 1 -- 172.21.15.45:0/1004485291 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f96cc0762d0 msgr2=0x7f96cc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.812+0000 7f96e75b6640 1 --2- 172.21.15.45:0/1004485291 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f96cc0762d0 0x7f96cc078790 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f96e01f8bc0 tx=0x7f96d4007bd0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.812+0000 7f96e75b6640 1 -- 172.21.15.45:0/1004485291 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96e00d3480 msgr2=0x7f96e015e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.812+0000 7f96e75b6640 1 --2- 172.21.15.45:0/1004485291 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f96e00d3480 0x7f96e015e6f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f96d8031000 tx=0x7f96d80046c0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.812+0000 7f96e5b53640 1 -- 172.21.15.45:0/1004485291 reap_dead start 2024-05-08T05:51:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.812+0000 7f96e75b6640 1 -- 172.21.15.45:0/1004485291 shutdown_connections 2024-05-08T05:51:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.812+0000 7f96e75b6640 1 -- 172.21.15.45:0/1004485291 >> 172.21.15.45:0/1004485291 conn(0x7f96e00d5040 msgr2=0x7f96e00b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.812+0000 7f96e75b6640 1 -- 172.21.15.45:0/1004485291 shutdown_connections 2024-05-08T05:51:50.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.812+0000 7f96e75b6640 1 -- 172.21.15.45:0/1004485291 wait complete. 2024-05-08T05:51:50.822 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-08T05:51:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.927+0000 7f8a286a7640 1 Processor -- start 2024-05-08T05:51:50.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.927+0000 7f8a286a7640 1 -- start start 2024-05-08T05:51:50.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.928+0000 7f8a286a7640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f8a2006cab0 0x7f8a200b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.928+0000 7f8a286a7640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a201604b0 con 0x7f8a2015f0c0 2024-05-08T05:51:50.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.928+0000 7f8a286a7640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a200b14e0 con 0x7f8a201638e0 2024-05-08T05:51:50.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.928+0000 7f8a286a7640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f8a20163450 con 0x7f8a2006cab0 2024-05-08T05:51:50.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.928+0000 7f8a26443640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f8a2015f0c0 0x7f8a200b0d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49934/0 (socket says 172.21.15.45:49934) 2024-05-08T05:51:50.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.928+0000 7f8a26443640 1 -- 172.21.15.45:0/3210903689 learned_addr learned my addr 172.21.15.45:0/3210903689 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:50.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.928+0000 7f8a26c44640 1 --2- 172.21.15.45:0/3210903689 >> v2:172.21.15.63:3300/0 conn(0x7f8a2006cab0 0x7f8a200b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.928+0000 7f8a25441640 1 -- 172.21.15.45:0/3210903689 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3105271876 0 0) 0x7f8a201604b0 con 0x7f8a2015f0c0 2024-05-08T05:51:50.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.928+0000 7f8a25441640 1 -- 172.21.15.45:0/3210903689 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a0c003520 con 0x7f8a2015f0c0 2024-05-08T05:51:50.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.928+0000 7f8a25441640 1 -- 172.21.15.45:0/3210903689 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1806019587 0 0) 0x7f8a200b14e0 con 0x7f8a201638e0 2024-05-08T05:51:50.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.928+0000 7f8a25441640 1 -- 172.21.15.45:0/3210903689 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a201604b0 con 0x7f8a201638e0 2024-05-08T05:51:50.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.928+0000 7f8a26c44640 1 -- 172.21.15.45:0/3210903689 >> v1:172.21.15.63:6789/0 conn(0x7f8a201638e0 legacy=0x7f8a2006c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.928+0000 7f8a26c44640 1 -- 172.21.15.45:0/3210903689 >> v1:172.21.15.45:6789/0 conn(0x7f8a2015f0c0 legacy=0x7f8a200b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.929+0000 7f8a26c44640 1 -- 172.21.15.45:0/3210903689 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a20162e00 con 0x7f8a2006cab0 2024-05-08T05:51:50.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.929+0000 7f8a26c44640 1 --2- 172.21.15.45:0/3210903689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a2006cab0 0x7f8a200b6f40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009d90 tx=0x7f8a1802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb5961b144d793b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:50.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.929+0000 7f8a25441640 1 -- 172.21.15.45:0/3210903689 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18007740 con 0x7f8a2006cab0 2024-05-08T05:51:50.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.929+0000 7f8a25441640 1 -- 172.21.15.45:0/3210903689 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a180078e0 con 0x7f8a2006cab0 2024-05-08T05:51:50.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.929+0000 7f8a25441640 1 -- 172.21.15.45:0/3210903689 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18007cb0 con 0x7f8a2006cab0 2024-05-08T05:51:50.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.930+0000 7f8a286a7640 1 -- 172.21.15.45:0/3210903689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a2006cab0 msgr2=0x7f8a200b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.930+0000 7f8a286a7640 1 --2- 172.21.15.45:0/3210903689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a2006cab0 0x7f8a200b6f40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009d90 tx=0x7f8a1802a6f0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.930+0000 7f8a286a7640 1 -- 172.21.15.45:0/3210903689 shutdown_connections 2024-05-08T05:51:50.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.930+0000 7f8a286a7640 1 --2- 172.21.15.45:0/3210903689 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a2006cab0 0x7f8a200b6f40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.930+0000 7f8a286a7640 1 -- 172.21.15.45:0/3210903689 >> 172.21.15.45:0/3210903689 conn(0x7f8a200b3b10 msgr2=0x7f8a200b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:50.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.930+0000 7f8a286a7640 1 -- 172.21.15.45:0/3210903689 shutdown_connections 2024-05-08T05:51:50.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.930+0000 7f8a286a7640 1 -- 172.21.15.45:0/3210903689 wait complete. 2024-05-08T05:51:50.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.931+0000 7f8a286a7640 1 Processor -- start 2024-05-08T05:51:50.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.931+0000 7f8a286a7640 1 -- start start 2024-05-08T05:51:50.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.932+0000 7f8a286a7640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a2006cab0 0x7f8a200b2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.932+0000 7f8a286a7640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a2015f0c0 0x7f8a200b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.932+0000 7f8a286a7640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a201638e0 0x7f8a200aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.932+0000 7f8a286a7640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8a20169ee0 con 0x7f8a201638e0 2024-05-08T05:51:50.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.932+0000 7f8a286a7640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f8a200b17c0 con 0x7f8a2015f0c0 2024-05-08T05:51:50.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.932+0000 7f8a286a7640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f8a2016a060 con 0x7f8a2006cab0 2024-05-08T05:51:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.932+0000 7f8a26c44640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a201638e0 0x7f8a200aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.932+0000 7f8a26c44640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a201638e0 0x7f8a200aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43490/0 (socket says 172.21.15.45:43490) 2024-05-08T05:51:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.932+0000 7f8a26c44640 1 -- 172.21.15.45:0/948424304 learned_addr learned my addr 172.21.15.45:0/948424304 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.932+0000 7f8a26443640 1 --2- 172.21.15.45:0/948424304 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a2006cab0 0x7f8a200b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.932+0000 7f8a25c42640 1 --2- 172.21.15.45:0/948424304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a2015f0c0 0x7f8a200b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.932+0000 7f8a26c44640 1 -- 172.21.15.45:0/948424304 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a2006cab0 msgr2=0x7f8a200b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.932+0000 7f8a26c44640 1 --2- 172.21.15.45:0/948424304 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a2006cab0 0x7f8a200b2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.933+0000 7f8a26c44640 1 -- 172.21.15.45:0/948424304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a2015f0c0 msgr2=0x7f8a200b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.933+0000 7f8a26c44640 1 --2- 172.21.15.45:0/948424304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a2015f0c0 0x7f8a200b2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.933+0000 7f8a26c44640 1 -- 172.21.15.45:0/948424304 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2014f4d0 con 0x7f8a201638e0 2024-05-08T05:51:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.933+0000 7f8a26443640 1 --2- 172.21.15.45:0/948424304 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a2006cab0 0x7f8a200b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:50.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.933+0000 7f8a26c44640 1 --2- 172.21.15.45:0/948424304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a201638e0 0x7f8a200aedf0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009eb0 tx=0x7f8a18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.933+0000 7f8a16ffd640 1 -- 172.21.15.45:0/948424304 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18040070 con 0x7f8a201638e0 2024-05-08T05:51:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.933+0000 7f8a16ffd640 1 -- 172.21.15.45:0/948424304 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a18031bc0 con 0x7f8a201638e0 2024-05-08T05:51:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.933+0000 7f8a286a7640 1 -- 172.21.15.45:0/948424304 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a2014f770 con 0x7f8a201638e0 2024-05-08T05:51:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.933+0000 7f8a16ffd640 1 -- 172.21.15.45:0/948424304 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18009450 con 0x7f8a201638e0 2024-05-08T05:51:50.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.933+0000 7f8a286a7640 1 -- 172.21.15.45:0/948424304 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a200b3870 con 0x7f8a201638e0 2024-05-08T05:51:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.934+0000 7f8a286a7640 1 -- 172.21.15.45:0/948424304 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a2014fb60 con 0x7f8a201638e0 2024-05-08T05:51:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.935+0000 7f8a16ffd640 1 -- 172.21.15.45:0/948424304 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8a18039040 con 0x7f8a201638e0 2024-05-08T05:51:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.935+0000 7f8a16ffd640 1 --2- 172.21.15.45:0/948424304 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8a040762d0 0x7f8a04078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.935+0000 7f8a16ffd640 1 -- 172.21.15.45:0/948424304 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f8a180b8220 con 0x7f8a201638e0 2024-05-08T05:51:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.935+0000 7f8a26443640 1 --2- 172.21.15.45:0/948424304 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8a040762d0 0x7f8a04078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:50.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.936+0000 7f8a26443640 1 --2- 172.21.15.45:0/948424304 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8a040762d0 0x7f8a04078790 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c005880 tx=0x7f8a1c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:50.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:50.939+0000 7f8a16ffd640 1 -- 172.21.15.45:0/948424304 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8a18081cd0 con 0x7f8a201638e0 2024-05-08T05:51:51.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.131+0000 7f8a286a7640 1 -- 172.21.15.45:0/948424304 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f8a2006a8f0 con 0x7f8a040762d0 2024-05-08T05:51:51.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.132+0000 7f8a16ffd640 1 -- 172.21.15.45:0/948424304 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8a2006a8f0 con 0x7f8a040762d0 2024-05-08T05:51:51.132 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 3 to deep-scrub 2024-05-08T05:51:51.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.134+0000 7f8a286a7640 1 -- 172.21.15.45:0/948424304 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8a040762d0 msgr2=0x7f8a04078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.134+0000 7f8a286a7640 1 --2- 172.21.15.45:0/948424304 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8a040762d0 0x7f8a04078790 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c005880 tx=0x7f8a1c009040 comp rx=0 tx=0).stop 2024-05-08T05:51:51.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.134+0000 7f8a286a7640 1 -- 172.21.15.45:0/948424304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a201638e0 msgr2=0x7f8a200aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.134+0000 7f8a286a7640 1 --2- 172.21.15.45:0/948424304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a201638e0 0x7f8a200aedf0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009eb0 tx=0x7f8a18004a40 comp rx=0 tx=0).stop 2024-05-08T05:51:51.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.134+0000 7f8a286a7640 1 -- 172.21.15.45:0/948424304 shutdown_connections 2024-05-08T05:51:51.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.134+0000 7f8a286a7640 1 --2- 172.21.15.45:0/948424304 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f8a040762d0 0x7f8a04078790 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.134+0000 7f8a286a7640 1 --2- 172.21.15.45:0/948424304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a201638e0 0x7f8a200aedf0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.134+0000 7f8a286a7640 1 --2- 172.21.15.45:0/948424304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8a2015f0c0 0x7f8a200b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.134+0000 7f8a286a7640 1 --2- 172.21.15.45:0/948424304 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f8a2006cab0 0x7f8a200b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.134+0000 7f8a286a7640 1 -- 172.21.15.45:0/948424304 >> 172.21.15.45:0/948424304 conn(0x7f8a200b3b10 msgr2=0x7f8a201574e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:51.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.134+0000 7f8a286a7640 1 -- 172.21.15.45:0/948424304 shutdown_connections 2024-05-08T05:51:51.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.134+0000 7f8a286a7640 1 -- 172.21.15.45:0/948424304 wait complete. 2024-05-08T05:51:51.145 INFO:tasks.ceph:Scrubbing osd.4 2024-05-08T05:51:51.145 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-08T05:51:51.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.253+0000 7fe6fc447640 1 Processor -- start 2024-05-08T05:51:51.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.253+0000 7fe6fc447640 1 -- start start 2024-05-08T05:51:51.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.254+0000 7fe6fc447640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe6f415d2a0 0x7fe6f40cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.254+0000 7fe6fc447640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fe6f40c9600 0x7fe6f40cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.254+0000 7fe6fc447640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6f40cec40 con 0x7fe6f415d680 2024-05-08T05:51:51.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.254+0000 7fe6fc447640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fe6f40cef20 con 0x7fe6f415d2a0 2024-05-08T05:51:51.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.254+0000 7fe6fc447640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fe6f40c6b10 con 0x7fe6f40c9600 2024-05-08T05:51:51.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.254+0000 7fe6fa1e3640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe6f415d2a0 0x7fe6f40cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.254+0000 7fe6fa1e3640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fe6f415d2a0 0x7fe6f40cf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43496/0 (socket says 172.21.15.45:43496) 2024-05-08T05:51:51.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.254+0000 7fe6fa1e3640 1 -- 172.21.15.45:0/2502007860 learned_addr learned my addr 172.21.15.45:0/2502007860 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:51.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.254+0000 7fe6f99e2640 1 --1- 172.21.15.45:0/2502007860 >> v1:172.21.15.45:6789/0 conn(0x7fe6f415d680 0x7fe6f40d0d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49944/0 (socket says 172.21.15.45:49944) 2024-05-08T05:51:51.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.254+0000 7fe6fa1e3640 1 -- 172.21.15.45:0/2502007860 >> v2:172.21.15.63:3300/0 conn(0x7fe6f40c9600 msgr2=0x7fe6f40cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:51.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.254+0000 7fe6fa9e4640 1 --2- 172.21.15.45:0/2502007860 >> v2:172.21.15.63:3300/0 conn(0x7fe6f40c9600 0x7fe6f40cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.254+0000 7fe6fa1e3640 1 --2- 172.21.15.45:0/2502007860 >> v2:172.21.15.63:3300/0 conn(0x7fe6f40c9600 0x7fe6f40cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.254+0000 7fe6fa1e3640 1 -- 172.21.15.45:0/2502007860 >> v1:172.21.15.45:6789/0 conn(0x7fe6f415d680 legacy=0x7fe6f40d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.254+0000 7fe6fa1e3640 1 -- 172.21.15.45:0/2502007860 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f40c6df0 con 0x7fe6f415d2a0 2024-05-08T05:51:51.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.254+0000 7fe6fa9e4640 1 --2- 172.21.15.45:0/2502007860 >> v2:172.21.15.63:3300/0 conn(0x7fe6f40c9600 0x7fe6f40cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:51.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.254+0000 7fe6f91e1640 1 -- 172.21.15.45:0/2502007860 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2238805935 0 0) 0x7fe6f40cec40 con 0x7fe6f415d680 2024-05-08T05:51:51.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.255+0000 7fe6fa1e3640 1 --2- 172.21.15.45:0/2502007860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6f415d2a0 0x7fe6f40cf670 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40cf2d0 tx=0x7fe6f002e600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d95cd2e15807a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:51.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.255+0000 7fe6f91e1640 1 -- 172.21.15.45:0/2502007860 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f003a070 con 0x7fe6f415d2a0 2024-05-08T05:51:51.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.255+0000 7fe6f91e1640 1 -- 172.21.15.45:0/2502007860 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f0033710 con 0x7fe6f415d2a0 2024-05-08T05:51:51.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.255+0000 7fe6f91e1640 1 -- 172.21.15.45:0/2502007860 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0033ae0 con 0x7fe6f415d2a0 2024-05-08T05:51:51.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.256+0000 7fe6fc447640 1 -- 172.21.15.45:0/2502007860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6f415d2a0 msgr2=0x7fe6f40cf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.256+0000 7fe6fc447640 1 --2- 172.21.15.45:0/2502007860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6f415d2a0 0x7fe6f40cf670 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40cf2d0 tx=0x7fe6f002e600 comp rx=0 tx=0).stop 2024-05-08T05:51:51.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.256+0000 7fe6fc447640 1 -- 172.21.15.45:0/2502007860 shutdown_connections 2024-05-08T05:51:51.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.256+0000 7fe6fc447640 1 --2- 172.21.15.45:0/2502007860 >> v2:172.21.15.63:3300/0 conn(0x7fe6f40c9600 0x7fe6f40cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.256+0000 7fe6fc447640 1 --2- 172.21.15.45:0/2502007860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6f415d2a0 0x7fe6f40cf670 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.256+0000 7fe6fc447640 1 -- 172.21.15.45:0/2502007860 >> 172.21.15.45:0/2502007860 conn(0x7fe6f40d2f20 msgr2=0x7fe6f40d3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:51.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.256+0000 7fe6fc447640 1 -- 172.21.15.45:0/2502007860 shutdown_connections 2024-05-08T05:51:51.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.256+0000 7fe6fc447640 1 -- 172.21.15.45:0/2502007860 wait complete. 2024-05-08T05:51:51.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.257+0000 7fe6fc447640 1 Processor -- start 2024-05-08T05:51:51.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.257+0000 7fe6fc447640 1 -- start start 2024-05-08T05:51:51.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.258+0000 7fe6fc447640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6f40c9600 0x7fe6f40caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.258+0000 7fe6fc447640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6f415d2a0 0x7fe6f40b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.258+0000 7fe6fc447640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6f415d680 0x7fe6f415ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.258+0000 7fe6fc447640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f40ce870 con 0x7fe6f40c9600 2024-05-08T05:51:51.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.258+0000 7fe6fc447640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f40ce6f0 con 0x7fe6f415d680 2024-05-08T05:51:51.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.258+0000 7fe6fc447640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fe6f40ce9f0 con 0x7fe6f415d2a0 2024-05-08T05:51:51.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.258+0000 7fe6fa1e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6f40c9600 0x7fe6f40caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.258+0000 7fe6fa1e3640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6f40c9600 0x7fe6f40caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43498/0 (socket says 172.21.15.45:43498) 2024-05-08T05:51:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.258+0000 7fe6fa1e3640 1 -- 172.21.15.45:0/1837657065 learned_addr learned my addr 172.21.15.45:0/1837657065 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.258+0000 7fe6f99e2640 1 --2- 172.21.15.45:0/1837657065 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6f415d2a0 0x7fe6f40b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.258+0000 7fe6fa9e4640 1 --2- 172.21.15.45:0/1837657065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6f415d680 0x7fe6f415ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.258+0000 7fe6fa1e3640 1 -- 172.21.15.45:0/1837657065 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6f415d2a0 msgr2=0x7fe6f40b7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.258+0000 7fe6fa1e3640 1 --2- 172.21.15.45:0/1837657065 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6f415d2a0 0x7fe6f40b7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.258+0000 7fe6fa1e3640 1 -- 172.21.15.45:0/1837657065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6f415d680 msgr2=0x7fe6f415ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.258+0000 7fe6fa1e3640 1 --2- 172.21.15.45:0/1837657065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6f415d680 0x7fe6f415ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.259+0000 7fe6fa1e3640 1 -- 172.21.15.45:0/1837657065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f409f750 con 0x7fe6f40c9600 2024-05-08T05:51:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.259+0000 7fe6fa9e4640 1 --2- 172.21.15.45:0/1837657065 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe6f415d680 0x7fe6f415ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.259+0000 7fe6f99e2640 1 --2- 172.21.15.45:0/1837657065 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fe6f415d2a0 0x7fe6f40b7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.259+0000 7fe6fa1e3640 1 --2- 172.21.15.45:0/1837657065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6f40c9600 0x7fe6f40caa00 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fe6f002e640 tx=0x7fe6f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.259+0000 7fe6eaffd640 1 -- 172.21.15.45:0/1837657065 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0044020 con 0x7fe6f40c9600 2024-05-08T05:51:51.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.259+0000 7fe6eaffd640 1 -- 172.21.15.45:0/1837657065 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f0035990 con 0x7fe6f40c9600 2024-05-08T05:51:51.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.259+0000 7fe6fc447640 1 -- 172.21.15.45:0/1837657065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f40caf40 con 0x7fe6f40c9600 2024-05-08T05:51:51.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.259+0000 7fe6eaffd640 1 -- 172.21.15.45:0/1837657065 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f003a040 con 0x7fe6f40c9600 2024-05-08T05:51:51.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.259+0000 7fe6fc447640 1 -- 172.21.15.45:0/1837657065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f409ff80 con 0x7fe6f40c9600 2024-05-08T05:51:51.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.260+0000 7fe6fc447640 1 -- 172.21.15.45:0/1837657065 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe6f4069b00 con 0x7fe6f40c9600 2024-05-08T05:51:51.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.261+0000 7fe6eaffd640 1 -- 172.21.15.45:0/1837657065 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe6f0047020 con 0x7fe6f40c9600 2024-05-08T05:51:51.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.261+0000 7fe6eaffd640 1 --2- 172.21.15.45:0/1837657065 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe6d80762d0 0x7fe6d8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.261+0000 7fe6eaffd640 1 -- 172.21.15.45:0/1837657065 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7fe6f00bc140 con 0x7fe6f40c9600 2024-05-08T05:51:51.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.261+0000 7fe6eaffd640 1 --2- 172.21.15.45:0/1837657065 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x7fe6d807c8b0 0x7fe6d807ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.261+0000 7fe6eaffd640 1 -- 172.21.15.45:0/1837657065 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe6f0007e80 con 0x7fe6d807c8b0 2024-05-08T05:51:51.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.261+0000 7fe6eaffd640 1 -- 172.21.15.45:0/1837657065 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7fe6f00c3050 con 0x7fe6f40c9600 2024-05-08T05:51:51.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.261+0000 7fe6f99e2640 1 --2- 172.21.15.45:0/1837657065 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe6d80762d0 0x7fe6d8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.262+0000 7fe6fa9e4640 1 --2- 172.21.15.45:0/1837657065 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x7fe6d807c8b0 0x7fe6d807ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.262+0000 7fe6f99e2640 1 --2- 172.21.15.45:0/1837657065 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe6d80762d0 0x7fe6d8078790 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe6f415bdd0 tx=0x7fe6e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:51.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.262+0000 7fe6fa9e4640 1 --2- 172.21.15.45:0/1837657065 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x7fe6d807c8b0 0x7fe6d807ecf0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:51.264 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.263+0000 7fe6eaffd640 1 -- 172.21.15.45:0/1837657065 <== osd.4 v2:172.21.15.63:6824/2212879151 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6f0007e80 con 0x7fe6d807c8b0 2024-05-08T05:51:51.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.276+0000 7fe6fc447640 1 -- 172.21.15.45:0/1837657065 --> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe6f40cd500 con 0x7fe6d807c8b0 2024-05-08T05:51:51.276 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.276+0000 7fe6eaffd640 1 -- 172.21.15.45:0/1837657065 <== osd.4 v2:172.21.15.63:6824/2212879151 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe6f40cd500 con 0x7fe6d807c8b0 2024-05-08T05:51:51.276 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:51:51.276 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T05:51:51.277 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:51:51.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.277+0000 7fe6fc447640 1 -- 172.21.15.45:0/1837657065 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x7fe6d807c8b0 msgr2=0x7fe6d807ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.277 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.277+0000 7fe6fc447640 1 --2- 172.21.15.45:0/1837657065 >> [v2:172.21.15.63:6824/2212879151,v1:172.21.15.63:6825/2212879151] conn(0x7fe6d807c8b0 0x7fe6d807ecf0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.277+0000 7fe6fc447640 1 -- 172.21.15.45:0/1837657065 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe6d80762d0 msgr2=0x7fe6d8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.277+0000 7fe6fc447640 1 --2- 172.21.15.45:0/1837657065 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fe6d80762d0 0x7fe6d8078790 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe6f415bdd0 tx=0x7fe6e4009040 comp rx=0 tx=0).stop 2024-05-08T05:51:51.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.277+0000 7fe6fc447640 1 -- 172.21.15.45:0/1837657065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6f40c9600 msgr2=0x7fe6f40caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.277+0000 7fe6fc447640 1 --2- 172.21.15.45:0/1837657065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6f40c9600 0x7fe6f40caa00 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fe6f002e640 tx=0x7fe6f00046c0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.277+0000 7fe6fa9e4640 1 -- 172.21.15.45:0/1837657065 reap_dead start 2024-05-08T05:51:51.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.277+0000 7fe6fc447640 1 -- 172.21.15.45:0/1837657065 shutdown_connections 2024-05-08T05:51:51.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.277+0000 7fe6fc447640 1 -- 172.21.15.45:0/1837657065 >> 172.21.15.45:0/1837657065 conn(0x7fe6f40d2f20 msgr2=0x7fe6f40716a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:51.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.277+0000 7fe6fc447640 1 -- 172.21.15.45:0/1837657065 shutdown_connections 2024-05-08T05:51:51.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.277+0000 7fe6fc447640 1 -- 172.21.15.45:0/1837657065 wait complete. 2024-05-08T05:51:51.287 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-08T05:51:51.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.392+0000 7fec6136a640 1 Processor -- start 2024-05-08T05:51:51.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.392+0000 7fec6136a640 1 -- start start 2024-05-08T05:51:51.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.393+0000 7fec6136a640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fec5c0d5bc0 0x7fec5c0d3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.393+0000 7fec6136a640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec5c0d2c80 con 0x7fec5c15f100 2024-05-08T05:51:51.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.393+0000 7fec6136a640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec5c0d2f60 con 0x7fec5c162c60 2024-05-08T05:51:51.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.393+0000 7fec6136a640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fec5c0d35b0 con 0x7fec5c0d5bc0 2024-05-08T05:51:51.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.393+0000 7fec5affd640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fec5c15f100 0x7fec5c0d07e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49958/0 (socket says 172.21.15.45:49958) 2024-05-08T05:51:51.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.393+0000 7fec5affd640 1 -- 172.21.15.45:0/3947390543 learned_addr learned my addr 172.21.15.45:0/3947390543 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:51.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.393+0000 7fec5a7fc640 1 --2- 172.21.15.45:0/3947390543 >> v2:172.21.15.45:3300/0 conn(0x7fec5c0d5bc0 0x7fec5c0d3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.393+0000 7fec59ffb640 1 -- 172.21.15.45:0/3947390543 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2622504912 0 0) 0x7fec5c0d2c80 con 0x7fec5c15f100 2024-05-08T05:51:51.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.393+0000 7fec59ffb640 1 -- 172.21.15.45:0/3947390543 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec48003520 con 0x7fec5c15f100 2024-05-08T05:51:51.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.393+0000 7fec5a7fc640 1 -- 172.21.15.45:0/3947390543 >> v1:172.21.15.63:6789/0 conn(0x7fec5c162c60 legacy=0x7fec5c0b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:51.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.393+0000 7fec5a7fc640 1 -- 172.21.15.45:0/3947390543 >> v1:172.21.15.45:6789/0 conn(0x7fec5c15f100 legacy=0x7fec5c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.394 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.393+0000 7fec5a7fc640 1 -- 172.21.15.45:0/3947390543 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5c0d3240 con 0x7fec5c0d5bc0 2024-05-08T05:51:51.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.394+0000 7fec5a7fc640 1 --2- 172.21.15.45:0/3947390543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec5c0d5bc0 0x7fec5c0d3a40 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fec440097d0 tx=0x7fec4402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf2e8c2d756e4ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:51.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.394+0000 7fec59ffb640 1 -- 172.21.15.45:0/3947390543 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec44037070 con 0x7fec5c0d5bc0 2024-05-08T05:51:51.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.394+0000 7fec59ffb640 1 -- 172.21.15.45:0/3947390543 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec44032440 con 0x7fec5c0d5bc0 2024-05-08T05:51:51.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.394+0000 7fec59ffb640 1 -- 172.21.15.45:0/3947390543 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec44032810 con 0x7fec5c0d5bc0 2024-05-08T05:51:51.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.395+0000 7fec6136a640 1 -- 172.21.15.45:0/3947390543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec5c0d5bc0 msgr2=0x7fec5c0d3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.395+0000 7fec6136a640 1 --2- 172.21.15.45:0/3947390543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec5c0d5bc0 0x7fec5c0d3a40 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fec440097d0 tx=0x7fec4402a250 comp rx=0 tx=0).stop 2024-05-08T05:51:51.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.395+0000 7fec6136a640 1 -- 172.21.15.45:0/3947390543 shutdown_connections 2024-05-08T05:51:51.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.395+0000 7fec6136a640 1 --2- 172.21.15.45:0/3947390543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec5c0d5bc0 0x7fec5c0d3a40 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.395+0000 7fec6136a640 1 -- 172.21.15.45:0/3947390543 >> 172.21.15.45:0/3947390543 conn(0x7fec5c09e3e0 msgr2=0x7fec5c0d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:51.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.395+0000 7fec6136a640 1 -- 172.21.15.45:0/3947390543 shutdown_connections 2024-05-08T05:51:51.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.395+0000 7fec6136a640 1 -- 172.21.15.45:0/3947390543 wait complete. 2024-05-08T05:51:51.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.396+0000 7fec6136a640 1 Processor -- start 2024-05-08T05:51:51.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.396+0000 7fec6136a640 1 -- start start 2024-05-08T05:51:51.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.397+0000 7fec6136a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec5c0d5bc0 0x7fec5c0ba200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.397+0000 7fec6136a640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec5c15f100 0x7fec5c0ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.397+0000 7fec6136a640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec5c162c60 0x7fec5c1537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.397+0000 7fec6136a640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c0d3f80 con 0x7fec5c162c60 2024-05-08T05:51:51.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.397+0000 7fec6136a640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c15f4e0 con 0x7fec5c0d5bc0 2024-05-08T05:51:51.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.397+0000 7fec6136a640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fec5c0cde00 con 0x7fec5c15f100 2024-05-08T05:51:51.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.397+0000 7fec5b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec5c162c60 0x7fec5c1537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.397+0000 7fec5b7fe640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec5c162c60 0x7fec5c1537b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43504/0 (socket says 172.21.15.45:43504) 2024-05-08T05:51:51.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.397+0000 7fec5b7fe640 1 -- 172.21.15.45:0/1062290702 learned_addr learned my addr 172.21.15.45:0/1062290702 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:51.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.397+0000 7fec5affd640 1 --2- 172.21.15.45:0/1062290702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec5c0d5bc0 0x7fec5c0ba200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.397+0000 7fec5a7fc640 1 --2- 172.21.15.45:0/1062290702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec5c15f100 0x7fec5c0ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.397+0000 7fec5b7fe640 1 -- 172.21.15.45:0/1062290702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec5c15f100 msgr2=0x7fec5c0ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.397+0000 7fec5b7fe640 1 --2- 172.21.15.45:0/1062290702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec5c15f100 0x7fec5c0ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.397+0000 7fec5b7fe640 1 -- 172.21.15.45:0/1062290702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec5c0d5bc0 msgr2=0x7fec5c0ba200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.397+0000 7fec5b7fe640 1 --2- 172.21.15.45:0/1062290702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec5c0d5bc0 0x7fec5c0ba200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.397+0000 7fec5b7fe640 1 -- 172.21.15.45:0/1062290702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5c0cef20 con 0x7fec5c162c60 2024-05-08T05:51:51.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.397+0000 7fec5a7fc640 1 --2- 172.21.15.45:0/1062290702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec5c15f100 0x7fec5c0ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:51.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.397+0000 7fec5affd640 1 --2- 172.21.15.45:0/1062290702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec5c0d5bc0 0x7fec5c0ba200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:51.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.398+0000 7fec5b7fe640 1 --2- 172.21.15.45:0/1062290702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec5c162c60 0x7fec5c1537b0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fec5400b970 tx=0x7fec5400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:51.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.398+0000 7fec537fe640 1 -- 172.21.15.45:0/1062290702 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec54019070 con 0x7fec5c162c60 2024-05-08T05:51:51.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.398+0000 7fec537fe640 1 -- 172.21.15.45:0/1062290702 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec5400cb40 con 0x7fec5c162c60 2024-05-08T05:51:51.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.398+0000 7fec6136a640 1 -- 172.21.15.45:0/1062290702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5c0cd1f0 con 0x7fec5c162c60 2024-05-08T05:51:51.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.398+0000 7fec537fe640 1 -- 172.21.15.45:0/1062290702 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec54014440 con 0x7fec5c162c60 2024-05-08T05:51:51.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.398+0000 7fec6136a640 1 -- 172.21.15.45:0/1062290702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec5c0cd7b0 con 0x7fec5c162c60 2024-05-08T05:51:51.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.399+0000 7fec6136a640 1 -- 172.21.15.45:0/1062290702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec5c0693f0 con 0x7fec5c162c60 2024-05-08T05:51:51.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.399+0000 7fec537fe640 1 -- 172.21.15.45:0/1062290702 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fec5401c070 con 0x7fec5c162c60 2024-05-08T05:51:51.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.400+0000 7fec537fe640 1 --2- 172.21.15.45:0/1062290702 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fec3c0762d0 0x7fec3c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.400+0000 7fec537fe640 1 -- 172.21.15.45:0/1062290702 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7fec540972a0 con 0x7fec5c162c60 2024-05-08T05:51:51.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.400+0000 7fec5affd640 1 --2- 172.21.15.45:0/1062290702 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fec3c0762d0 0x7fec3c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.401+0000 7fec5affd640 1 --2- 172.21.15.45:0/1062290702 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fec3c0762d0 0x7fec3c078790 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fec4c005880 tx=0x7fec4c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:51.404 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.405+0000 7fec537fe640 1 -- 172.21.15.45:0/1062290702 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fec54060cc0 con 0x7fec5c162c60 2024-05-08T05:51:51.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.597+0000 7fec6136a640 1 -- 172.21.15.45:0/1062290702 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fec5c0b5c00 con 0x7fec3c0762d0 2024-05-08T05:51:51.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.598+0000 7fec537fe640 1 -- 172.21.15.45:0/1062290702 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fec5c0b5c00 con 0x7fec3c0762d0 2024-05-08T05:51:51.598 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 4 to deep-scrub 2024-05-08T05:51:51.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.600+0000 7fec6136a640 1 -- 172.21.15.45:0/1062290702 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fec3c0762d0 msgr2=0x7fec3c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.600+0000 7fec6136a640 1 --2- 172.21.15.45:0/1062290702 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fec3c0762d0 0x7fec3c078790 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fec4c005880 tx=0x7fec4c009040 comp rx=0 tx=0).stop 2024-05-08T05:51:51.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.600+0000 7fec6136a640 1 -- 172.21.15.45:0/1062290702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec5c162c60 msgr2=0x7fec5c1537b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.600+0000 7fec6136a640 1 --2- 172.21.15.45:0/1062290702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec5c162c60 0x7fec5c1537b0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fec5400b970 tx=0x7fec5400be30 comp rx=0 tx=0).stop 2024-05-08T05:51:51.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.600+0000 7fec6136a640 1 -- 172.21.15.45:0/1062290702 shutdown_connections 2024-05-08T05:51:51.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.600+0000 7fec6136a640 1 --2- 172.21.15.45:0/1062290702 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fec3c0762d0 0x7fec3c078790 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.600+0000 7fec6136a640 1 --2- 172.21.15.45:0/1062290702 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fec5c15f100 0x7fec5c0ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.600+0000 7fec6136a640 1 --2- 172.21.15.45:0/1062290702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec5c162c60 0x7fec5c1537b0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.600+0000 7fec6136a640 1 --2- 172.21.15.45:0/1062290702 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fec5c0d5bc0 0x7fec5c0ba200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.600+0000 7fec6136a640 1 -- 172.21.15.45:0/1062290702 >> 172.21.15.45:0/1062290702 conn(0x7fec5c09e3e0 msgr2=0x7fec5c0b4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:51.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.600+0000 7fec6136a640 1 -- 172.21.15.45:0/1062290702 shutdown_connections 2024-05-08T05:51:51.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.600+0000 7fec6136a640 1 -- 172.21.15.45:0/1062290702 wait complete. 2024-05-08T05:51:51.611 INFO:tasks.ceph:Scrubbing osd.5 2024-05-08T05:51:51.611 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-08T05:51:51.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.719+0000 7f7bc7124640 1 Processor -- start 2024-05-08T05:51:51.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.719+0000 7f7bc7124640 1 -- start start 2024-05-08T05:51:51.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.720+0000 7f7bc7124640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7bc0159920 0x7f7bc00ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.720+0000 7f7bc7124640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7bc00aeb90 0x7f7bc00d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.720+0000 7f7bc7124640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bc00afa00 con 0x7f7bc00d0df0 2024-05-08T05:51:51.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.720+0000 7f7bc7124640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f7bc00d1260 con 0x7f7bc00aeb90 2024-05-08T05:51:51.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.720+0000 7f7bc7124640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc00af160 con 0x7f7bc0159920 2024-05-08T05:51:51.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.720+0000 7f7bc4ec0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7bc0159920 0x7f7bc00ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.720+0000 7f7bbffff640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f7bc00aeb90 0x7f7bc00d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.720+0000 7f7bc4ec0640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7bc0159920 0x7f7bc00ae650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3301/0 says I am v2:172.21.15.45:34942/0 (socket says 172.21.15.45:34942) 2024-05-08T05:51:51.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.720+0000 7f7bc4ec0640 1 -- 172.21.15.45:0/1883159929 learned_addr learned my addr 172.21.15.45:0/1883159929 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:51.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.721+0000 7f7bc4ec0640 1 -- 172.21.15.45:0/1883159929 >> v2:172.21.15.63:3300/0 conn(0x7f7bc00aeb90 msgr2=0x7f7bc00d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:51.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.721+0000 7f7bc4ec0640 1 --2- 172.21.15.45:0/1883159929 >> v2:172.21.15.63:3300/0 conn(0x7f7bc00aeb90 0x7f7bc00d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.721+0000 7f7bc4ec0640 1 -- 172.21.15.45:0/1883159929 >> v1:172.21.15.63:6789/0 conn(0x7f7bc00d0df0 legacy=0x7f7bc00b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.721+0000 7f7bc4ec0640 1 -- 172.21.15.45:0/1883159929 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc00af440 con 0x7f7bc0159920 2024-05-08T05:51:51.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.721+0000 7f7bc4ec0640 1 --2- 172.21.15.45:0/1883159929 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7bc0159920 0x7f7bc00ae650 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009870 tx=0x7f7bb402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a197ef36f556bc32 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:51.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.722+0000 7f7bbf7fe640 1 -- 172.21.15.45:0/1883159929 <== mon.2 v2:172.21.15.63:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4037070 con 0x7f7bc0159920 2024-05-08T05:51:51.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.722+0000 7f7bbf7fe640 1 -- 172.21.15.45:0/1883159929 <== mon.2 v2:172.21.15.63:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bb4032440 con 0x7f7bc0159920 2024-05-08T05:51:51.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.722+0000 7f7bbf7fe640 1 -- 172.21.15.45:0/1883159929 <== mon.2 v2:172.21.15.63:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4032810 con 0x7f7bc0159920 2024-05-08T05:51:51.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.722+0000 7f7bc7124640 1 -- 172.21.15.45:0/1883159929 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7bc0159920 msgr2=0x7f7bc00ae650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.722+0000 7f7bc7124640 1 --2- 172.21.15.45:0/1883159929 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7bc0159920 0x7f7bc00ae650 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009870 tx=0x7f7bb402a1c0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.722+0000 7f7bc7124640 1 -- 172.21.15.45:0/1883159929 shutdown_connections 2024-05-08T05:51:51.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.722+0000 7f7bc7124640 1 --2- 172.21.15.45:0/1883159929 >> v2:172.21.15.63:3300/0 conn(0x7f7bc00aeb90 0x7f7bc00d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.722+0000 7f7bc7124640 1 --2- 172.21.15.45:0/1883159929 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7bc0159920 0x7f7bc00ae650 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.722+0000 7f7bc7124640 1 -- 172.21.15.45:0/1883159929 >> 172.21.15.45:0/1883159929 conn(0x7f7bc006c3a0 msgr2=0x7f7bc006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:51.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.722+0000 7f7bc7124640 1 -- 172.21.15.45:0/1883159929 shutdown_connections 2024-05-08T05:51:51.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.722+0000 7f7bc7124640 1 -- 172.21.15.45:0/1883159929 wait complete. 2024-05-08T05:51:51.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.723+0000 7f7bc7124640 1 Processor -- start 2024-05-08T05:51:51.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.723+0000 7f7bc7124640 1 -- start start 2024-05-08T05:51:51.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.724+0000 7f7bc7124640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7bc00aeb90 0x7f7bc0153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.724+0000 7f7bc7124640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7bc00d0df0 0x7f7bc01456c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.724+0000 7f7bc7124640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bc0159920 0x7f7bc015b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.724+0000 7f7bc7124640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc0159d00 con 0x7f7bc0159920 2024-05-08T05:51:51.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.724+0000 7f7bc7124640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc006e190 con 0x7f7bc00aeb90 2024-05-08T05:51:51.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.724+0000 7f7bc7124640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc015c5e0 con 0x7f7bc00d0df0 2024-05-08T05:51:51.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.724+0000 7f7bc56c1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bc0159920 0x7f7bc015b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.724+0000 7f7bc56c1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bc0159920 0x7f7bc015b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43506/0 (socket says 172.21.15.45:43506) 2024-05-08T05:51:51.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.724+0000 7f7bc56c1640 1 -- 172.21.15.45:0/2879276661 learned_addr learned my addr 172.21.15.45:0/2879276661 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:51.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.724+0000 7f7bc4ec0640 1 --2- 172.21.15.45:0/2879276661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7bc00aeb90 0x7f7bc0153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.724+0000 7f7bbffff640 1 --2- 172.21.15.45:0/2879276661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7bc00d0df0 0x7f7bc01456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.725+0000 7f7bc56c1640 1 -- 172.21.15.45:0/2879276661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7bc00d0df0 msgr2=0x7f7bc01456c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.725+0000 7f7bc56c1640 1 --2- 172.21.15.45:0/2879276661 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f7bc00d0df0 0x7f7bc01456c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.725+0000 7f7bc56c1640 1 -- 172.21.15.45:0/2879276661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7bc00aeb90 msgr2=0x7f7bc0153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.725+0000 7f7bc56c1640 1 --2- 172.21.15.45:0/2879276661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7bc00aeb90 0x7f7bc0153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.725+0000 7f7bc56c1640 1 -- 172.21.15.45:0/2879276661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc015a4c0 con 0x7f7bc0159920 2024-05-08T05:51:51.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.725+0000 7f7bc56c1640 1 --2- 172.21.15.45:0/2879276661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bc0159920 0x7f7bc015b560 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800c7a0 tx=0x7f7bb800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:51.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.725+0000 7f7bbd7fa640 1 -- 172.21.15.45:0/2879276661 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8003b70 con 0x7f7bc0159920 2024-05-08T05:51:51.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.725+0000 7f7bbd7fa640 1 -- 172.21.15.45:0/2879276661 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bb8003d10 con 0x7f7bc0159920 2024-05-08T05:51:51.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.725+0000 7f7bc7124640 1 -- 172.21.15.45:0/2879276661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc015a7a0 con 0x7f7bc0159920 2024-05-08T05:51:51.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.725+0000 7f7bbd7fa640 1 -- 172.21.15.45:0/2879276661 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb800aa70 con 0x7f7bc0159920 2024-05-08T05:51:51.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.725+0000 7f7bc7124640 1 -- 172.21.15.45:0/2879276661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc015ad60 con 0x7f7bc0159920 2024-05-08T05:51:51.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.726+0000 7f7bc7124640 1 -- 172.21.15.45:0/2879276661 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7bc00693f0 con 0x7f7bc0159920 2024-05-08T05:51:51.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.727+0000 7f7bbd7fa640 1 -- 172.21.15.45:0/2879276661 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7bb800ac10 con 0x7f7bc0159920 2024-05-08T05:51:51.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.727+0000 7f7bbd7fa640 1 --2- 172.21.15.45:0/2879276661 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7ba40762d0 0x7f7ba4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.727+0000 7f7bbd7fa640 1 -- 172.21.15.45:0/2879276661 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f7bb8097300 con 0x7f7bc0159920 2024-05-08T05:51:51.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.728+0000 7f7bbd7fa640 1 --2- 172.21.15.45:0/2879276661 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x7f7ba407c8b0 0x7f7ba407ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.728+0000 7f7bbd7fa640 1 -- 172.21.15.45:0/2879276661 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ba407f3c0 con 0x7f7ba407c8b0 2024-05-08T05:51:51.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.728+0000 7f7bbd7fa640 1 -- 172.21.15.45:0/2879276661 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7f7bb809e050 con 0x7f7bc0159920 2024-05-08T05:51:51.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.728+0000 7f7bc4ec0640 1 --2- 172.21.15.45:0/2879276661 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7ba40762d0 0x7f7ba4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.728+0000 7f7bbffff640 1 --2- 172.21.15.45:0/2879276661 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x7f7ba407c8b0 0x7f7ba407ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.728+0000 7f7bc4ec0640 1 --2- 172.21.15.45:0/2879276661 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7ba40762d0 0x7f7ba4078790 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4034000 tx=0x7f7bb402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:51.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.728+0000 7f7bbffff640 1 --2- 172.21.15.45:0/2879276661 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x7f7ba407c8b0 0x7f7ba407ecf0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:51.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.729+0000 7f7bbd7fa640 1 -- 172.21.15.45:0/2879276661 <== osd.5 v2:172.21.15.63:6808/2395798255 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ba407f3c0 con 0x7f7ba407c8b0 2024-05-08T05:51:51.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.742+0000 7f7bc7124640 1 -- 172.21.15.45:0/2879276661 --> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7bc00b0a30 con 0x7f7ba407c8b0 2024-05-08T05:51:51.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.743+0000 7f7bbd7fa640 1 -- 172.21.15.45:0/2879276661 <== osd.5 v2:172.21.15.63:6808/2395798255 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7bc00b0a30 con 0x7f7ba407c8b0 2024-05-08T05:51:51.743 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:51:51.743 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T05:51:51.743 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:51:51.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.743+0000 7f7bc7124640 1 -- 172.21.15.45:0/2879276661 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x7f7ba407c8b0 msgr2=0x7f7ba407ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.743+0000 7f7bc7124640 1 --2- 172.21.15.45:0/2879276661 >> [v2:172.21.15.63:6808/2395798255,v1:172.21.15.63:6809/2395798255] conn(0x7f7ba407c8b0 0x7f7ba407ecf0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.743+0000 7f7bc7124640 1 -- 172.21.15.45:0/2879276661 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7ba40762d0 msgr2=0x7f7ba4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.743+0000 7f7bc7124640 1 --2- 172.21.15.45:0/2879276661 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f7ba40762d0 0x7f7ba4078790 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4034000 tx=0x7f7bb402e000 comp rx=0 tx=0).stop 2024-05-08T05:51:51.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.743+0000 7f7bc7124640 1 -- 172.21.15.45:0/2879276661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bc0159920 msgr2=0x7f7bc015b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.743+0000 7f7bc7124640 1 --2- 172.21.15.45:0/2879276661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bc0159920 0x7f7bc015b560 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800c7a0 tx=0x7f7bb800cc60 comp rx=0 tx=0).stop 2024-05-08T05:51:51.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.743+0000 7f7bc56c1640 1 -- 172.21.15.45:0/2879276661 reap_dead start 2024-05-08T05:51:51.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.744+0000 7f7bc7124640 1 -- 172.21.15.45:0/2879276661 shutdown_connections 2024-05-08T05:51:51.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.744+0000 7f7bc7124640 1 -- 172.21.15.45:0/2879276661 >> 172.21.15.45:0/2879276661 conn(0x7f7bc006c3a0 msgr2=0x7f7bc00729e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:51.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.744+0000 7f7bc7124640 1 -- 172.21.15.45:0/2879276661 shutdown_connections 2024-05-08T05:51:51.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.744+0000 7f7bc7124640 1 -- 172.21.15.45:0/2879276661 wait complete. 2024-05-08T05:51:51.754 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-08T05:51:51.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.859+0000 7f3d6ebdd640 1 Processor -- start 2024-05-08T05:51:51.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.859+0000 7f3d6ebdd640 1 -- start start 2024-05-08T05:51:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.860+0000 7f3d6ebdd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d6815f420 0x7f3d680d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.860+0000 7f3d6ebdd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d680d2080 0x7f3d680d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.860+0000 7f3d6ebdd640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d681630b0 con 0x7f3d680d3ca0 2024-05-08T05:51:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.860+0000 7f3d6ebdd640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f3d680d2550 con 0x7f3d6815f420 2024-05-08T05:51:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.860+0000 7f3d6ebdd640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3d680d2c80 con 0x7f3d680d2080 2024-05-08T05:51:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.860+0000 7f3d6c979640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3d6815f420 0x7f3d680d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.860+0000 7f3d67fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3d680d3ca0 0x7f3d680d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49966/0 (socket says 172.21.15.45:49966) 2024-05-08T05:51:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.860+0000 7f3d67fff640 1 -- 172.21.15.45:0/2889151217 learned_addr learned my addr 172.21.15.45:0/2889151217 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.861+0000 7f3d6d17a640 1 --2- 172.21.15.45:0/2889151217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d680d2080 0x7f3d680d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.861+0000 7f3d6c979640 1 -- 172.21.15.45:0/2889151217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d680d2080 msgr2=0x7f3d680d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.861+0000 7f3d6c979640 1 --2- 172.21.15.45:0/2889151217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d680d2080 0x7f3d680d4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.861+0000 7f3d6c979640 1 -- 172.21.15.45:0/2889151217 >> v1:172.21.15.45:6789/0 conn(0x7f3d680d3ca0 legacy=0x7f3d680d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.861+0000 7f3d677fe640 1 -- 172.21.15.45:0/2889151217 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1527424701 0 0) 0x7f3d5c001110 con 0x7f3d680d3ca0 2024-05-08T05:51:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.861+0000 7f3d6c979640 1 -- 172.21.15.45:0/2889151217 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d680d2f60 con 0x7f3d6815f420 2024-05-08T05:51:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.861+0000 7f3d6c979640 1 --2- 172.21.15.45:0/2889151217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6815f420 0x7f3d680d3760 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f3d580097a0 tx=0x7f3d5802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d9713c5fcd0a14 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.862+0000 7f3d677fe640 1 -- 172.21.15.45:0/2889151217 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58037070 con 0x7f3d6815f420 2024-05-08T05:51:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.862+0000 7f3d677fe640 1 -- 172.21.15.45:0/2889151217 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d58032440 con 0x7f3d6815f420 2024-05-08T05:51:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.862+0000 7f3d677fe640 1 -- 172.21.15.45:0/2889151217 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58032810 con 0x7f3d6815f420 2024-05-08T05:51:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.862+0000 7f3d6ebdd640 1 -- 172.21.15.45:0/2889151217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6815f420 msgr2=0x7f3d680d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.862+0000 7f3d6ebdd640 1 --2- 172.21.15.45:0/2889151217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6815f420 0x7f3d680d3760 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f3d580097a0 tx=0x7f3d5802a180 comp rx=0 tx=0).stop 2024-05-08T05:51:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.862+0000 7f3d6ebdd640 1 -- 172.21.15.45:0/2889151217 shutdown_connections 2024-05-08T05:51:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.862+0000 7f3d6ebdd640 1 --2- 172.21.15.45:0/2889151217 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d680d2080 0x7f3d680d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.862+0000 7f3d6ebdd640 1 --2- 172.21.15.45:0/2889151217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6815f420 0x7f3d680d3760 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.862+0000 7f3d6ebdd640 1 -- 172.21.15.45:0/2889151217 >> 172.21.15.45:0/2889151217 conn(0x7f3d6808eac0 msgr2=0x7f3d6809e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.862+0000 7f3d6ebdd640 1 -- 172.21.15.45:0/2889151217 shutdown_connections 2024-05-08T05:51:51.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.862+0000 7f3d6ebdd640 1 -- 172.21.15.45:0/2889151217 wait complete. 2024-05-08T05:51:51.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.863+0000 7f3d6ebdd640 1 Processor -- start 2024-05-08T05:51:51.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.864+0000 7f3d6ebdd640 1 -- start start 2024-05-08T05:51:51.870 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.864+0000 7f3d6ebdd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d680d2080 0x7f3d681f2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.864+0000 7f3d6ebdd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d681f5c40 0x7f3d680caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.864+0000 7f3d6ebdd640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d680caf50 0x7f3d681f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.864+0000 7f3d6ebdd640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3d680d5200 con 0x7f3d680d2080 2024-05-08T05:51:51.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.864+0000 7f3d6ebdd640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3d680d2830 con 0x7f3d681f5c40 2024-05-08T05:51:51.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.864+0000 7f3d6ebdd640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3d680d5380 con 0x7f3d680caf50 2024-05-08T05:51:51.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.865+0000 7f3d6c979640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d680d2080 0x7f3d681f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.865+0000 7f3d6c979640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d680d2080 0x7f3d681f2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43532/0 (socket says 172.21.15.45:43532) 2024-05-08T05:51:51.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.865+0000 7f3d6c979640 1 -- 172.21.15.45:0/3512003650 learned_addr learned my addr 172.21.15.45:0/3512003650 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:51.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.865+0000 7f3d6c979640 1 -- 172.21.15.45:0/3512003650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d680caf50 msgr2=0x7f3d681f87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.865+0000 7f3d6d17a640 1 --2- 172.21.15.45:0/3512003650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d680caf50 0x7f3d681f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.865+0000 7f3d6c979640 1 --2- 172.21.15.45:0/3512003650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d680caf50 0x7f3d681f87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.872 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.865+0000 7f3d6c979640 1 -- 172.21.15.45:0/3512003650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d681f5c40 msgr2=0x7f3d680caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:51.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.865+0000 7f3d67fff640 1 --2- 172.21.15.45:0/3512003650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d681f5c40 0x7f3d680caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.865+0000 7f3d6c979640 1 --2- 172.21.15.45:0/3512003650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d681f5c40 0x7f3d680caa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:51.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.865+0000 7f3d6c979640 1 -- 172.21.15.45:0/3512003650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d680c8f50 con 0x7f3d680d2080 2024-05-08T05:51:51.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.865+0000 7f3d6c979640 1 --2- 172.21.15.45:0/3512003650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d680d2080 0x7f3d681f2750 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f3d580098c0 tx=0x7f3d580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:51.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.865+0000 7f3d657fa640 1 -- 172.21.15.45:0/3512003650 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58037050 con 0x7f3d680d2080 2024-05-08T05:51:51.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.865+0000 7f3d657fa640 1 -- 172.21.15.45:0/3512003650 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d58038040 con 0x7f3d680d2080 2024-05-08T05:51:51.873 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.865+0000 7f3d6ebdd640 1 -- 172.21.15.45:0/3512003650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d680c9230 con 0x7f3d680d2080 2024-05-08T05:51:51.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.865+0000 7f3d657fa640 1 -- 172.21.15.45:0/3512003650 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58004490 con 0x7f3d680d2080 2024-05-08T05:51:51.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.865+0000 7f3d6ebdd640 1 -- 172.21.15.45:0/3512003650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d680c97f0 con 0x7f3d680d2080 2024-05-08T05:51:51.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.866+0000 7f3d6ebdd640 1 -- 172.21.15.45:0/3512003650 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d680693f0 con 0x7f3d680d2080 2024-05-08T05:51:51.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.867+0000 7f3d657fa640 1 -- 172.21.15.45:0/3512003650 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3d58030900 con 0x7f3d680d2080 2024-05-08T05:51:51.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.867+0000 7f3d657fa640 1 --2- 172.21.15.45:0/3512003650 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3d540762d0 0x7f3d54078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:51.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.867+0000 7f3d657fa640 1 -- 172.21.15.45:0/3512003650 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f3d580b8170 con 0x7f3d680d2080 2024-05-08T05:51:51.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.868+0000 7f3d67fff640 1 --2- 172.21.15.45:0/3512003650 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3d540762d0 0x7f3d54078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:51.874 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.868+0000 7f3d67fff640 1 --2- 172.21.15.45:0/3512003650 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3d540762d0 0x7f3d54078790 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c000f60 tx=0x7f3d5c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:51.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:51.872+0000 7f3d657fa640 1 -- 172.21.15.45:0/3512003650 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3d58081a70 con 0x7f3d680d2080 2024-05-08T05:51:52.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.062+0000 7f3d6ebdd640 1 -- 172.21.15.45:0/3512003650 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f3d680cb330 con 0x7f3d540762d0 2024-05-08T05:51:52.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.063+0000 7f3d657fa640 1 -- 172.21.15.45:0/3512003650 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3d680cb330 con 0x7f3d540762d0 2024-05-08T05:51:52.063 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 5 to deep-scrub 2024-05-08T05:51:52.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.065+0000 7f3d6ebdd640 1 -- 172.21.15.45:0/3512003650 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3d540762d0 msgr2=0x7f3d54078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.065+0000 7f3d6ebdd640 1 --2- 172.21.15.45:0/3512003650 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3d540762d0 0x7f3d54078790 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c000f60 tx=0x7f3d5c009040 comp rx=0 tx=0).stop 2024-05-08T05:51:52.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.065+0000 7f3d6ebdd640 1 -- 172.21.15.45:0/3512003650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d680d2080 msgr2=0x7f3d681f2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.065+0000 7f3d6ebdd640 1 --2- 172.21.15.45:0/3512003650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d680d2080 0x7f3d681f2750 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f3d580098c0 tx=0x7f3d580046c0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.065+0000 7f3d6ebdd640 1 -- 172.21.15.45:0/3512003650 shutdown_connections 2024-05-08T05:51:52.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.065+0000 7f3d6ebdd640 1 --2- 172.21.15.45:0/3512003650 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3d540762d0 0x7f3d54078790 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.065+0000 7f3d6ebdd640 1 --2- 172.21.15.45:0/3512003650 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d680caf50 0x7f3d681f87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.065+0000 7f3d6ebdd640 1 --2- 172.21.15.45:0/3512003650 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d681f5c40 0x7f3d680caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.065+0000 7f3d6ebdd640 1 --2- 172.21.15.45:0/3512003650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d680d2080 0x7f3d681f2750 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.065+0000 7f3d6ebdd640 1 -- 172.21.15.45:0/3512003650 >> 172.21.15.45:0/3512003650 conn(0x7f3d6808eac0 msgr2=0x7f3d6806d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:52.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.065+0000 7f3d6ebdd640 1 -- 172.21.15.45:0/3512003650 shutdown_connections 2024-05-08T05:51:52.066 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.065+0000 7f3d6ebdd640 1 -- 172.21.15.45:0/3512003650 wait complete. 2024-05-08T05:51:52.076 INFO:tasks.ceph:Scrubbing osd.6 2024-05-08T05:51:52.076 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-08T05:51:52.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.183+0000 7ff4e8c08640 1 Processor -- start 2024-05-08T05:51:52.184 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.184+0000 7ff4e8c08640 1 -- start start 2024-05-08T05:51:52.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.184+0000 7ff4e8c08640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff4e4162b70 0x7ff4e40d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.184+0000 7ff4e8c08640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4e4162f50 con 0x7ff4e40d3910 2024-05-08T05:51:52.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.184+0000 7ff4e8c08640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4e40d3ee0 con 0x7ff4e40d3480 2024-05-08T05:51:52.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.184+0000 7ff4e8c08640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7ff4e40d1f60 con 0x7ff4e4162b70 2024-05-08T05:51:52.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.185+0000 7ff4e259c640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7ff4e4162b70 0x7ff4e40d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.185+0000 7ff4e2d9d640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7ff4e40d3910 0x7ff4e40b5150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49970/0 (socket says 172.21.15.45:49970) 2024-05-08T05:51:52.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.185+0000 7ff4e2d9d640 1 -- 172.21.15.45:0/655639573 learned_addr learned my addr 172.21.15.45:0/655639573 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:52.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.185+0000 7ff4e259c640 1 -- 172.21.15.45:0/655639573 >> v1:172.21.15.63:6789/0 conn(0x7ff4e40d3480 legacy=0x7ff4e40b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:52.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.185+0000 7ff4e1d9b640 1 -- 172.21.15.45:0/655639573 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3033514252 0 0) 0x7ff4dc001490 con 0x7ff4e40d3910 2024-05-08T05:51:52.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.185+0000 7ff4e259c640 1 -- 172.21.15.45:0/655639573 >> v1:172.21.15.45:6789/0 conn(0x7ff4e40d3910 legacy=0x7ff4e40b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.185+0000 7ff4e259c640 1 -- 172.21.15.45:0/655639573 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e40d21b0 con 0x7ff4e4162b70 2024-05-08T05:51:52.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.186+0000 7ff4e259c640 1 --2- 172.21.15.45:0/655639573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e4162b70 0x7ff4e40d2f40 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40097a0 tx=0x7ff4d402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d721f3d08f9798ee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:52.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.186+0000 7ff4e1d9b640 1 -- 172.21.15.45:0/655639573 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4037070 con 0x7ff4e4162b70 2024-05-08T05:51:52.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.186+0000 7ff4e1d9b640 1 -- 172.21.15.45:0/655639573 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d4032440 con 0x7ff4e4162b70 2024-05-08T05:51:52.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.186+0000 7ff4e1d9b640 1 -- 172.21.15.45:0/655639573 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4032810 con 0x7ff4e4162b70 2024-05-08T05:51:52.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.186+0000 7ff4e8c08640 1 -- 172.21.15.45:0/655639573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e4162b70 msgr2=0x7ff4e40d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.186+0000 7ff4e8c08640 1 --2- 172.21.15.45:0/655639573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e4162b70 0x7ff4e40d2f40 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40097a0 tx=0x7ff4d402a230 comp rx=0 tx=0).stop 2024-05-08T05:51:52.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.186+0000 7ff4e8c08640 1 -- 172.21.15.45:0/655639573 shutdown_connections 2024-05-08T05:51:52.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.186+0000 7ff4e8c08640 1 --2- 172.21.15.45:0/655639573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e4162b70 0x7ff4e40d2f40 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.186+0000 7ff4e8c08640 1 -- 172.21.15.45:0/655639573 >> 172.21.15.45:0/655639573 conn(0x7ff4e40d5040 msgr2=0x7ff4e40d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:52.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.186+0000 7ff4e8c08640 1 -- 172.21.15.45:0/655639573 shutdown_connections 2024-05-08T05:51:52.188 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.186+0000 7ff4e8c08640 1 -- 172.21.15.45:0/655639573 wait complete. 2024-05-08T05:51:52.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.188+0000 7ff4e8c08640 1 Processor -- start 2024-05-08T05:51:52.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.188+0000 7ff4e8c08640 1 -- start start 2024-05-08T05:51:52.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.188+0000 7ff4e8c08640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4e40d3480 0x7ff4e40ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.188+0000 7ff4e8c08640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e40d3910 0x7ff4e40d1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.188+0000 7ff4e8c08640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4e4162b70 0x7ff4e40d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.188+0000 7ff4e8c08640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e40b42f0 con 0x7ff4e40d3910 2024-05-08T05:51:52.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.188+0000 7ff4e8c08640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e40b4170 con 0x7ff4e40d3480 2024-05-08T05:51:52.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.188+0000 7ff4e8c08640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e40b5860 con 0x7ff4e4162b70 2024-05-08T05:51:52.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.189+0000 7ff4dbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e40d3910 0x7ff4e40d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.189+0000 7ff4dbfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e40d3910 0x7ff4e40d1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43536/0 (socket says 172.21.15.45:43536) 2024-05-08T05:51:52.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.189+0000 7ff4dbfff640 1 -- 172.21.15.45:0/476751029 learned_addr learned my addr 172.21.15.45:0/476751029 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:52.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.189+0000 7ff4e259c640 1 --2- 172.21.15.45:0/476751029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4e40d3480 0x7ff4e40ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.189+0000 7ff4dbfff640 1 -- 172.21.15.45:0/476751029 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4e4162b70 msgr2=0x7ff4e40d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.189+0000 7ff4dbfff640 1 --2- 172.21.15.45:0/476751029 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7ff4e4162b70 0x7ff4e40d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.189+0000 7ff4dbfff640 1 -- 172.21.15.45:0/476751029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4e40d3480 msgr2=0x7ff4e40ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.189+0000 7ff4dbfff640 1 --2- 172.21.15.45:0/476751029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4e40d3480 0x7ff4e40ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.189+0000 7ff4dbfff640 1 -- 172.21.15.45:0/476751029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e415f5a0 con 0x7ff4e40d3910 2024-05-08T05:51:52.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.189+0000 7ff4e259c640 1 --2- 172.21.15.45:0/476751029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff4e40d3480 0x7ff4e40ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:52.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.189+0000 7ff4dbfff640 1 --2- 172.21.15.45:0/476751029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e40d3910 0x7ff4e40d1170 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00c530 tx=0x7ff4cc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:52.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.189+0000 7ff4daffd640 1 -- 172.21.15.45:0/476751029 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc0052d0 con 0x7ff4e40d3910 2024-05-08T05:51:52.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.189+0000 7ff4daffd640 1 -- 172.21.15.45:0/476751029 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4cc005470 con 0x7ff4e40d3910 2024-05-08T05:51:52.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.189+0000 7ff4e8c08640 1 -- 172.21.15.45:0/476751029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e415f880 con 0x7ff4e40d3910 2024-05-08T05:51:52.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.189+0000 7ff4daffd640 1 -- 172.21.15.45:0/476751029 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc012870 con 0x7ff4e40d3910 2024-05-08T05:51:52.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.189+0000 7ff4e8c08640 1 -- 172.21.15.45:0/476751029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e415fe40 con 0x7ff4e40d3910 2024-05-08T05:51:52.191 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.190+0000 7ff4e8c08640 1 -- 172.21.15.45:0/476751029 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff4e40693f0 con 0x7ff4e40d3910 2024-05-08T05:51:52.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.191+0000 7ff4daffd640 1 -- 172.21.15.45:0/476751029 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff4cc012a10 con 0x7ff4e40d3910 2024-05-08T05:51:52.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.191+0000 7ff4daffd640 1 --2- 172.21.15.45:0/476751029 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff4c40762d0 0x7ff4c4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.191+0000 7ff4daffd640 1 -- 172.21.15.45:0/476751029 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7ff4cc097160 con 0x7ff4e40d3910 2024-05-08T05:51:52.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.192+0000 7ff4daffd640 1 --2- 172.21.15.45:0/476751029 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x7ff4c407c8b0 0x7ff4c407ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.192+0000 7ff4daffd640 1 -- 172.21.15.45:0/476751029 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff4cc014e50 con 0x7ff4c407c8b0 2024-05-08T05:51:52.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.192+0000 7ff4daffd640 1 -- 172.21.15.45:0/476751029 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7ff4cc09e050 con 0x7ff4e40d3910 2024-05-08T05:51:52.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.192+0000 7ff4e259c640 1 --2- 172.21.15.45:0/476751029 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff4c40762d0 0x7ff4c4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.192+0000 7ff4e2d9d640 1 --2- 172.21.15.45:0/476751029 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x7ff4c407c8b0 0x7ff4c407ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.192+0000 7ff4e259c640 1 --2- 172.21.15.45:0/476751029 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff4c40762d0 0x7ff4c4078790 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4034000 tx=0x7ff4d402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:52.193 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.192+0000 7ff4e2d9d640 1 --2- 172.21.15.45:0/476751029 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x7ff4c407c8b0 0x7ff4c407ecf0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:52.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.193+0000 7ff4daffd640 1 -- 172.21.15.45:0/476751029 <== osd.6 v2:172.21.15.63:6800/2926759052 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff4cc014e50 con 0x7ff4c407c8b0 2024-05-08T05:51:52.206 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.206+0000 7ff4e8c08640 1 -- 172.21.15.45:0/476751029 --> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff4e40cafb0 con 0x7ff4c407c8b0 2024-05-08T05:51:52.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.207+0000 7ff4daffd640 1 -- 172.21.15.45:0/476751029 <== osd.6 v2:172.21.15.63:6800/2926759052 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff4e40cafb0 con 0x7ff4c407c8b0 2024-05-08T05:51:52.207 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:51:52.207 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T05:51:52.207 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:51:52.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.207+0000 7ff4e8c08640 1 -- 172.21.15.45:0/476751029 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x7ff4c407c8b0 msgr2=0x7ff4c407ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.207+0000 7ff4e8c08640 1 --2- 172.21.15.45:0/476751029 >> [v2:172.21.15.63:6800/2926759052,v1:172.21.15.63:6801/2926759052] conn(0x7ff4c407c8b0 0x7ff4c407ecf0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.207+0000 7ff4e8c08640 1 -- 172.21.15.45:0/476751029 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff4c40762d0 msgr2=0x7ff4c4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.207+0000 7ff4e8c08640 1 --2- 172.21.15.45:0/476751029 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7ff4c40762d0 0x7ff4c4078790 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4034000 tx=0x7ff4d402e000 comp rx=0 tx=0).stop 2024-05-08T05:51:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.207+0000 7ff4e8c08640 1 -- 172.21.15.45:0/476751029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e40d3910 msgr2=0x7ff4e40d1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.207+0000 7ff4e8c08640 1 --2- 172.21.15.45:0/476751029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff4e40d3910 0x7ff4e40d1170 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00c530 tx=0x7ff4cc00c9f0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.207+0000 7ff4e2d9d640 1 -- 172.21.15.45:0/476751029 reap_dead start 2024-05-08T05:51:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.207+0000 7ff4e8c08640 1 -- 172.21.15.45:0/476751029 shutdown_connections 2024-05-08T05:51:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.207+0000 7ff4e8c08640 1 -- 172.21.15.45:0/476751029 >> 172.21.15.45:0/476751029 conn(0x7ff4e40d5040 msgr2=0x7ff4e4161e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.207+0000 7ff4e8c08640 1 -- 172.21.15.45:0/476751029 shutdown_connections 2024-05-08T05:51:52.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.207+0000 7ff4e8c08640 1 -- 172.21.15.45:0/476751029 wait complete. 2024-05-08T05:51:52.219 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-08T05:51:52.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.323+0000 7f3a63428640 1 Processor -- start 2024-05-08T05:51:52.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.324+0000 7f3a63428640 1 -- start start 2024-05-08T05:51:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.324+0000 7f3a63428640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3a5c0d53d0 0x7f3a5c0d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.324+0000 7f3a63428640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a5c1630b0 con 0x7f3a5c0d1970 2024-05-08T05:51:52.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.324+0000 7f3a63428640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a5c0d3e70 con 0x7f3a5c15f420 2024-05-08T05:51:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.324+0000 7f3a63428640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f3a5c0d2550 con 0x7f3a5c0d53d0 2024-05-08T05:51:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.325+0000 7f3a619c5640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f3a5c0d53d0 0x7f3a5c0d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.325+0000 7f3a609c3640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f3a5c0d1970 0x7f3a5c0d4cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:49986/0 (socket says 172.21.15.45:49986) 2024-05-08T05:51:52.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.325+0000 7f3a609c3640 1 -- 172.21.15.45:0/200315992 learned_addr learned my addr 172.21.15.45:0/200315992 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.325+0000 7f3a619c5640 1 -- 172.21.15.45:0/200315992 >> v1:172.21.15.63:6789/0 conn(0x7f3a5c15f420 legacy=0x7f3a5c0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.325+0000 7f3a619c5640 1 -- 172.21.15.45:0/200315992 >> v1:172.21.15.45:6789/0 conn(0x7f3a5c0d1970 legacy=0x7f3a5c0d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.325+0000 7f3a619c5640 1 -- 172.21.15.45:0/200315992 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5c0d2c80 con 0x7f3a5c0d53d0 2024-05-08T05:51:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.325+0000 7f3a5bfff640 1 -- 172.21.15.45:0/200315992 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1781112623 0 0) 0x7f3a50001110 con 0x7f3a5c0d1970 2024-05-08T05:51:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.326+0000 7f3a619c5640 1 --2- 172.21.15.45:0/200315992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a5c0d53d0 0x7f3a5c0d1e20 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c0097a0 tx=0x7f3a4c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac9a55ccdeb82bc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.326+0000 7f3a5bfff640 1 -- 172.21.15.45:0/200315992 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c037070 con 0x7f3a5c0d53d0 2024-05-08T05:51:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.326+0000 7f3a5bfff640 1 -- 172.21.15.45:0/200315992 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a4c032440 con 0x7f3a5c0d53d0 2024-05-08T05:51:52.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.326+0000 7f3a5bfff640 1 -- 172.21.15.45:0/200315992 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c032810 con 0x7f3a5c0d53d0 2024-05-08T05:51:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.326+0000 7f3a63428640 1 -- 172.21.15.45:0/200315992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a5c0d53d0 msgr2=0x7f3a5c0d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.326+0000 7f3a63428640 1 --2- 172.21.15.45:0/200315992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a5c0d53d0 0x7f3a5c0d1e20 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c0097a0 tx=0x7f3a4c02a180 comp rx=0 tx=0).stop 2024-05-08T05:51:52.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.326+0000 7f3a63428640 1 -- 172.21.15.45:0/200315992 shutdown_connections 2024-05-08T05:51:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.326+0000 7f3a63428640 1 --2- 172.21.15.45:0/200315992 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a5c0d53d0 0x7f3a5c0d1e20 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.326+0000 7f3a63428640 1 -- 172.21.15.45:0/200315992 >> 172.21.15.45:0/200315992 conn(0x7f3a5c08eac0 msgr2=0x7f3a5c09e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.326+0000 7f3a63428640 1 -- 172.21.15.45:0/200315992 shutdown_connections 2024-05-08T05:51:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.326+0000 7f3a63428640 1 -- 172.21.15.45:0/200315992 wait complete. 2024-05-08T05:51:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.328+0000 7f3a63428640 1 Processor -- start 2024-05-08T05:51:52.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.328+0000 7f3a63428640 1 -- start start 2024-05-08T05:51:52.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.328+0000 7f3a63428640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a5c0d1970 0x7f3a5c1f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.328+0000 7f3a63428640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a5c0d53d0 0x7f3a5c15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.328+0000 7f3a63428640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a5c0caae0 0x7f3a5c0cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.328+0000 7f3a63428640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c0d2830 con 0x7f3a5c0d1970 2024-05-08T05:51:52.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.328+0000 7f3a63428640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c0d3240 con 0x7f3a5c0d53d0 2024-05-08T05:51:52.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.328+0000 7f3a63428640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3a5c0cfb20 con 0x7f3a5c0caae0 2024-05-08T05:51:52.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.329+0000 7f3a611c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a5c0d1970 0x7f3a5c1f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.329+0000 7f3a611c4640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a5c0d1970 0x7f3a5c1f5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43554/0 (socket says 172.21.15.45:43554) 2024-05-08T05:51:52.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.329+0000 7f3a611c4640 1 -- 172.21.15.45:0/3513179658 learned_addr learned my addr 172.21.15.45:0/3513179658 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:52.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.329+0000 7f3a611c4640 1 -- 172.21.15.45:0/3513179658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a5c0caae0 msgr2=0x7f3a5c0cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.329+0000 7f3a609c3640 1 --2- 172.21.15.45:0/3513179658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a5c0d53d0 0x7f3a5c15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.329+0000 7f3a619c5640 1 --2- 172.21.15.45:0/3513179658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a5c0caae0 0x7f3a5c0cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.329+0000 7f3a611c4640 1 --2- 172.21.15.45:0/3513179658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a5c0caae0 0x7f3a5c0cfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.329+0000 7f3a611c4640 1 -- 172.21.15.45:0/3513179658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a5c0d53d0 msgr2=0x7f3a5c15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.329+0000 7f3a611c4640 1 --2- 172.21.15.45:0/3513179658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a5c0d53d0 0x7f3a5c15d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.329+0000 7f3a611c4640 1 -- 172.21.15.45:0/3513179658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5c15dc00 con 0x7f3a5c0d1970 2024-05-08T05:51:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.329+0000 7f3a611c4640 1 --2- 172.21.15.45:0/3513179658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a5c0d1970 0x7f3a5c1f5fe0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400ba80 tx=0x7f3a5400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.329+0000 7f3a59ffb640 1 -- 172.21.15.45:0/3513179658 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5400cc00 con 0x7f3a5c0d1970 2024-05-08T05:51:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.329+0000 7f3a59ffb640 1 -- 172.21.15.45:0/3513179658 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a5400cda0 con 0x7f3a5c0d1970 2024-05-08T05:51:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.329+0000 7f3a63428640 1 -- 172.21.15.45:0/3513179658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5c0cb530 con 0x7f3a5c0d1970 2024-05-08T05:51:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.329+0000 7f3a59ffb640 1 -- 172.21.15.45:0/3513179658 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54003bb0 con 0x7f3a5c0d1970 2024-05-08T05:51:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.329+0000 7f3a63428640 1 -- 172.21.15.45:0/3513179658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5c1f57e0 con 0x7f3a5c0d1970 2024-05-08T05:51:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.330+0000 7f3a63428640 1 -- 172.21.15.45:0/3513179658 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a5c0693f0 con 0x7f3a5c0d1970 2024-05-08T05:51:52.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.331+0000 7f3a59ffb640 1 -- 172.21.15.45:0/3513179658 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3a54014440 con 0x7f3a5c0d1970 2024-05-08T05:51:52.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.331+0000 7f3a59ffb640 1 --2- 172.21.15.45:0/3513179658 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3a480762d0 0x7f3a48078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.332+0000 7f3a59ffb640 1 -- 172.21.15.45:0/3513179658 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f3a54097420 con 0x7f3a5c0d1970 2024-05-08T05:51:52.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.332+0000 7f3a609c3640 1 --2- 172.21.15.45:0/3513179658 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3a480762d0 0x7f3a48078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.332+0000 7f3a609c3640 1 --2- 172.21.15.45:0/3513179658 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3a480762d0 0x7f3a48078790 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3a50000f60 tx=0x7f3a50009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:52.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.336+0000 7f3a59ffb640 1 -- 172.21.15.45:0/3513179658 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3a5c15dc00 con 0x7f3a5c0d1970 2024-05-08T05:51:52.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.528+0000 7f3a63428640 1 -- 172.21.15.45:0/3513179658 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f3a5c0b4c50 con 0x7f3a480762d0 2024-05-08T05:51:52.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.529+0000 7f3a59ffb640 1 -- 172.21.15.45:0/3513179658 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3a5c0b4c50 con 0x7f3a480762d0 2024-05-08T05:51:52.530 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 6 to deep-scrub 2024-05-08T05:51:52.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.531+0000 7f3a63428640 1 -- 172.21.15.45:0/3513179658 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3a480762d0 msgr2=0x7f3a48078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.531+0000 7f3a63428640 1 --2- 172.21.15.45:0/3513179658 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3a480762d0 0x7f3a48078790 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f3a50000f60 tx=0x7f3a50009040 comp rx=0 tx=0).stop 2024-05-08T05:51:52.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.531+0000 7f3a63428640 1 -- 172.21.15.45:0/3513179658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a5c0d1970 msgr2=0x7f3a5c1f5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.531+0000 7f3a63428640 1 --2- 172.21.15.45:0/3513179658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a5c0d1970 0x7f3a5c1f5fe0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400ba80 tx=0x7f3a5400bf40 comp rx=0 tx=0).stop 2024-05-08T05:51:52.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.531+0000 7f3a63428640 1 -- 172.21.15.45:0/3513179658 shutdown_connections 2024-05-08T05:51:52.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.531+0000 7f3a63428640 1 --2- 172.21.15.45:0/3513179658 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3a480762d0 0x7f3a48078790 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.531+0000 7f3a63428640 1 --2- 172.21.15.45:0/3513179658 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3a5c0caae0 0x7f3a5c0cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.531+0000 7f3a63428640 1 --2- 172.21.15.45:0/3513179658 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a5c0d53d0 0x7f3a5c15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.531+0000 7f3a63428640 1 --2- 172.21.15.45:0/3513179658 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3a5c0d1970 0x7f3a5c1f5fe0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.531+0000 7f3a63428640 1 -- 172.21.15.45:0/3513179658 >> 172.21.15.45:0/3513179658 conn(0x7f3a5c08eac0 msgr2=0x7f3a5c15e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:52.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.531+0000 7f3a63428640 1 -- 172.21.15.45:0/3513179658 shutdown_connections 2024-05-08T05:51:52.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.531+0000 7f3a63428640 1 -- 172.21.15.45:0/3513179658 wait complete. 2024-05-08T05:51:52.542 INFO:tasks.ceph:Scrubbing osd.7 2024-05-08T05:51:52.542 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-08T05:51:52.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.650+0000 7f0d66a20640 1 Processor -- start 2024-05-08T05:51:52.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.650+0000 7f0d66a20640 1 -- start start 2024-05-08T05:51:52.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.651+0000 7f0d66a20640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0d600d0c60 0x7f0d6016a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.651+0000 7f0d66a20640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d60162f50 con 0x7f0d600b5a50 2024-05-08T05:51:52.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.651+0000 7f0d66a20640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d600d3ee0 con 0x7f0d60162b70 2024-05-08T05:51:52.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.651+0000 7f0d66a20640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f0d600d1f60 con 0x7f0d600d0c60 2024-05-08T05:51:52.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.651+0000 7f0d64fbd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0d600d0c60 0x7f0d6016a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.651+0000 7f0d64fbd640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f0d600d0c60 0x7f0d6016a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43570/0 (socket says 172.21.15.45:43570) 2024-05-08T05:51:52.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.651+0000 7f0d57fff640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f0d600b5a50 0x7f0d600d3650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50002/0 (socket says 172.21.15.45:50002) 2024-05-08T05:51:52.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.651+0000 7f0d64fbd640 1 -- 172.21.15.45:0/2519165559 learned_addr learned my addr 172.21.15.45:0/2519165559 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:52.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.651+0000 7f0d64fbd640 1 -- 172.21.15.45:0/2519165559 >> v1:172.21.15.63:6789/0 conn(0x7f0d60162b70 legacy=0x7f0d600d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.651+0000 7f0d64fbd640 1 -- 172.21.15.45:0/2519165559 >> v1:172.21.15.45:6789/0 conn(0x7f0d600b5a50 legacy=0x7f0d600d3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.651+0000 7f0d64fbd640 1 -- 172.21.15.45:0/2519165559 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d600d21b0 con 0x7f0d600d0c60 2024-05-08T05:51:52.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.652+0000 7f0d64fbd640 1 --2- 172.21.15.45:0/2519165559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d600d0c60 0x7f0d6016a1e0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009e50 tx=0x7f0d5802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6e33e9466c925f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:52.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.652+0000 7f0d5f7fe640 1 -- 172.21.15.45:0/2519165559 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d58036070 con 0x7f0d600d0c60 2024-05-08T05:51:52.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.652+0000 7f0d5f7fe640 1 -- 172.21.15.45:0/2519165559 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d58031600 con 0x7f0d600d0c60 2024-05-08T05:51:52.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.652+0000 7f0d5f7fe640 1 -- 172.21.15.45:0/2519165559 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d580319f0 con 0x7f0d600d0c60 2024-05-08T05:51:52.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.652+0000 7f0d66a20640 1 -- 172.21.15.45:0/2519165559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d600d0c60 msgr2=0x7f0d6016a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.652+0000 7f0d66a20640 1 --2- 172.21.15.45:0/2519165559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d600d0c60 0x7f0d6016a1e0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009e50 tx=0x7f0d5802a7c0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.653+0000 7f0d66a20640 1 -- 172.21.15.45:0/2519165559 shutdown_connections 2024-05-08T05:51:52.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.653+0000 7f0d66a20640 1 --2- 172.21.15.45:0/2519165559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d600d0c60 0x7f0d6016a1e0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.653+0000 7f0d66a20640 1 -- 172.21.15.45:0/2519165559 >> 172.21.15.45:0/2519165559 conn(0x7f0d600d5040 msgr2=0x7f0d600d5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:52.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.653+0000 7f0d66a20640 1 -- 172.21.15.45:0/2519165559 shutdown_connections 2024-05-08T05:51:52.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.653+0000 7f0d66a20640 1 -- 172.21.15.45:0/2519165559 wait complete. 2024-05-08T05:51:52.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.654+0000 7f0d66a20640 1 Processor -- start 2024-05-08T05:51:52.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.654+0000 7f0d66a20640 1 -- start start 2024-05-08T05:51:52.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.654+0000 7f0d66a20640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d600b5a50 0x7f0d6014aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.654+0000 7f0d66a20640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d600d0c60 0x7f0d600ba170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.654+0000 7f0d66a20640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d60162b70 0x7f0d600ba6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.654+0000 7f0d66a20640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0d600b4170 con 0x7f0d600b5a50 2024-05-08T05:51:52.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.654+0000 7f0d66a20640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f0d600d1090 con 0x7f0d60162b70 2024-05-08T05:51:52.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.654+0000 7f0d66a20640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f0d600b42f0 con 0x7f0d600d0c60 2024-05-08T05:51:52.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.655+0000 7f0d5ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d600b5a50 0x7f0d6014aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.655+0000 7f0d5ffff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d600b5a50 0x7f0d6014aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43582/0 (socket says 172.21.15.45:43582) 2024-05-08T05:51:52.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.655+0000 7f0d5ffff640 1 -- 172.21.15.45:0/724118722 learned_addr learned my addr 172.21.15.45:0/724118722 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:52.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.655+0000 7f0d64fbd640 1 --2- 172.21.15.45:0/724118722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d60162b70 0x7f0d600ba6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.655+0000 7f0d5ffff640 1 -- 172.21.15.45:0/724118722 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d600d0c60 msgr2=0x7f0d600ba170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.655+0000 7f0d57fff640 1 --2- 172.21.15.45:0/724118722 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d600d0c60 0x7f0d600ba170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.657 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.655+0000 7f0d5ffff640 1 --2- 172.21.15.45:0/724118722 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f0d600d0c60 0x7f0d600ba170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.655+0000 7f0d5ffff640 1 -- 172.21.15.45:0/724118722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d60162b70 msgr2=0x7f0d600ba6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.655+0000 7f0d5ffff640 1 --2- 172.21.15.45:0/724118722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d60162b70 0x7f0d600ba6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.655+0000 7f0d5ffff640 1 -- 172.21.15.45:0/724118722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6014afc0 con 0x7f0d600b5a50 2024-05-08T05:51:52.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.655+0000 7f0d64fbd640 1 --2- 172.21.15.45:0/724118722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d60162b70 0x7f0d600ba6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:52.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.655+0000 7f0d5ffff640 1 --2- 172.21.15.45:0/724118722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d600b5a50 0x7f0d6014aa80 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000c3d0 tx=0x7f0d5000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:52.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.655+0000 7f0d5d7fa640 1 -- 172.21.15.45:0/724118722 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50017070 con 0x7f0d600b5a50 2024-05-08T05:51:52.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.655+0000 7f0d5d7fa640 1 -- 172.21.15.45:0/724118722 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d500052d0 con 0x7f0d600b5a50 2024-05-08T05:51:52.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.655+0000 7f0d66a20640 1 -- 172.21.15.45:0/724118722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d6014cf40 con 0x7f0d600b5a50 2024-05-08T05:51:52.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.655+0000 7f0d5d7fa640 1 -- 172.21.15.45:0/724118722 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50003830 con 0x7f0d600b5a50 2024-05-08T05:51:52.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.655+0000 7f0d66a20640 1 -- 172.21.15.45:0/724118722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d600b7270 con 0x7f0d600b5a50 2024-05-08T05:51:52.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.656+0000 7f0d66a20640 1 -- 172.21.15.45:0/724118722 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0d600babf0 con 0x7f0d600b5a50 2024-05-08T05:51:52.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.657+0000 7f0d5d7fa640 1 -- 172.21.15.45:0/724118722 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0d5001a070 con 0x7f0d600b5a50 2024-05-08T05:51:52.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.658+0000 7f0d5d7fa640 1 --2- 172.21.15.45:0/724118722 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0d400762d0 0x7f0d40078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.658+0000 7f0d5d7fa640 1 -- 172.21.15.45:0/724118722 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f0d50096870 con 0x7f0d600b5a50 2024-05-08T05:51:52.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.658+0000 7f0d57fff640 1 --2- 172.21.15.45:0/724118722 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0d400762d0 0x7f0d40078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.658+0000 7f0d5d7fa640 1 --2- 172.21.15.45:0/724118722 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x7f0d4007c8b0 0x7f0d4007ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.658+0000 7f0d5d7fa640 1 -- 172.21.15.45:0/724118722 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0d5001ee50 con 0x7f0d4007c8b0 2024-05-08T05:51:52.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.658+0000 7f0d5d7fa640 1 -- 172.21.15.45:0/724118722 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7f0d5009b2a0 con 0x7f0d600b5a50 2024-05-08T05:51:52.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.658+0000 7f0d64fbd640 1 --2- 172.21.15.45:0/724118722 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x7f0d4007c8b0 0x7f0d4007ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.658+0000 7f0d57fff640 1 --2- 172.21.15.45:0/724118722 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0d400762d0 0x7f0d40078790 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0d600c8670 tx=0x7f0d48009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:52.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.658+0000 7f0d64fbd640 1 --2- 172.21.15.45:0/724118722 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x7f0d4007c8b0 0x7f0d4007ecf0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:52.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.659+0000 7f0d5d7fa640 1 -- 172.21.15.45:0/724118722 <== osd.7 v2:172.21.15.63:6816/1163469437 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0d5001ee50 con 0x7f0d4007c8b0 2024-05-08T05:51:52.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.672+0000 7f0d66a20640 1 -- 172.21.15.45:0/724118722 --> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0d600693f0 con 0x7f0d4007c8b0 2024-05-08T05:51:52.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.673+0000 7f0d5d7fa640 1 -- 172.21.15.45:0/724118722 <== osd.7 v2:172.21.15.63:6816/1163469437 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0d600693f0 con 0x7f0d4007c8b0 2024-05-08T05:51:52.673 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-08T05:51:52.673 INFO:teuthology.orchestra.run.smithi045.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T05:51:52.673 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-08T05:51:52.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.673+0000 7f0d66a20640 1 -- 172.21.15.45:0/724118722 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x7f0d4007c8b0 msgr2=0x7f0d4007ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.673+0000 7f0d66a20640 1 --2- 172.21.15.45:0/724118722 >> [v2:172.21.15.63:6816/1163469437,v1:172.21.15.63:6817/1163469437] conn(0x7f0d4007c8b0 0x7f0d4007ecf0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.673+0000 7f0d66a20640 1 -- 172.21.15.45:0/724118722 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0d400762d0 msgr2=0x7f0d40078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.673+0000 7f0d66a20640 1 --2- 172.21.15.45:0/724118722 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f0d400762d0 0x7f0d40078790 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0d600c8670 tx=0x7f0d48009040 comp rx=0 tx=0).stop 2024-05-08T05:51:52.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.673+0000 7f0d66a20640 1 -- 172.21.15.45:0/724118722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d600b5a50 msgr2=0x7f0d6014aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.673+0000 7f0d66a20640 1 --2- 172.21.15.45:0/724118722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d600b5a50 0x7f0d6014aa80 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000c3d0 tx=0x7f0d5000c890 comp rx=0 tx=0).stop 2024-05-08T05:51:52.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.673+0000 7f0d64fbd640 1 -- 172.21.15.45:0/724118722 reap_dead start 2024-05-08T05:51:52.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.673+0000 7f0d66a20640 1 -- 172.21.15.45:0/724118722 shutdown_connections 2024-05-08T05:51:52.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.673+0000 7f0d66a20640 1 -- 172.21.15.45:0/724118722 >> 172.21.15.45:0/724118722 conn(0x7f0d600d5040 msgr2=0x7f0d600ce120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:52.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.674+0000 7f0d66a20640 1 -- 172.21.15.45:0/724118722 shutdown_connections 2024-05-08T05:51:52.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.674+0000 7f0d66a20640 1 -- 172.21.15.45:0/724118722 wait complete. 2024-05-08T05:51:52.684 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-08T05:51:52.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.790+0000 7f4c1ace1640 1 Processor -- start 2024-05-08T05:51:52.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.790+0000 7f4c1ace1640 1 -- start start 2024-05-08T05:51:52.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.791+0000 7f4c1ace1640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4c14070d30 0x7f4c140d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.791+0000 7f4c1ace1640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f4c14071110 0x7f4c1406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.791+0000 7f4c1ace1640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c140d1300 con 0x7f4c14163920 2024-05-08T05:51:52.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.791+0000 7f4c1ace1640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7f4c140d10d0 con 0x7f4c14070d30 2024-05-08T05:51:52.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.791+0000 7f4c1ace1640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f4c14163db0 con 0x7f4c14071110 2024-05-08T05:51:52.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.791+0000 7f4c18a7d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4c14070d30 0x7f4c140d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.791+0000 7f4c18a7d640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7f4c14070d30 0x7f4c140d20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43590/0 (socket says 172.21.15.45:43590) 2024-05-08T05:51:52.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.791+0000 7f4c18a7d640 1 -- 172.21.15.45:0/2118865233 learned_addr learned my addr 172.21.15.45:0/2118865233 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:52.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.791+0000 7f4c1927e640 1 --2- 172.21.15.45:0/2118865233 >> v2:172.21.15.63:3300/0 conn(0x7f4c14071110 0x7f4c1406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.791+0000 7f4c18a7d640 1 -- 172.21.15.45:0/2118865233 >> v2:172.21.15.63:3300/0 conn(0x7f4c14071110 msgr2=0x7f4c1406d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:52.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.791+0000 7f4c18a7d640 1 --2- 172.21.15.45:0/2118865233 >> v2:172.21.15.63:3300/0 conn(0x7f4c14071110 0x7f4c1406d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.791+0000 7f4c18a7d640 1 -- 172.21.15.45:0/2118865233 >> v1:172.21.15.63:6789/0 conn(0x7f4c14163920 legacy=0x7f4c140703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:52.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.791+0000 7f4c18a7d640 1 -- 172.21.15.45:0/2118865233 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c140d15e0 con 0x7f4c14070d30 2024-05-08T05:51:52.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.791+0000 7f4c1927e640 1 --2- 172.21.15.45:0/2118865233 >> v2:172.21.15.63:3300/0 conn(0x7f4c14071110 0x7f4c1406d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:52.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.792+0000 7f4c18a7d640 1 --2- 172.21.15.45:0/2118865233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c14070d30 0x7f4c140d20c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f4c14073540 tx=0x7f4c1002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae9d3adceea4c17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:52.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.792+0000 7f4c0b7fe640 1 -- 172.21.15.45:0/2118865233 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10037070 con 0x7f4c14070d30 2024-05-08T05:51:52.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.792+0000 7f4c0b7fe640 1 -- 172.21.15.45:0/2118865233 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c10032440 con 0x7f4c14070d30 2024-05-08T05:51:52.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.792+0000 7f4c0b7fe640 1 -- 172.21.15.45:0/2118865233 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10032810 con 0x7f4c14070d30 2024-05-08T05:51:52.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.793+0000 7f4c1ace1640 1 -- 172.21.15.45:0/2118865233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c14070d30 msgr2=0x7f4c140d20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.793+0000 7f4c1ace1640 1 --2- 172.21.15.45:0/2118865233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c14070d30 0x7f4c140d20c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f4c14073540 tx=0x7f4c1002a1e0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.793+0000 7f4c1ace1640 1 -- 172.21.15.45:0/2118865233 shutdown_connections 2024-05-08T05:51:52.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.793+0000 7f4c1ace1640 1 --2- 172.21.15.45:0/2118865233 >> v2:172.21.15.63:3300/0 conn(0x7f4c14071110 0x7f4c1406d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.793+0000 7f4c1ace1640 1 --2- 172.21.15.45:0/2118865233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c14070d30 0x7f4c140d20c0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.793+0000 7f4c1ace1640 1 -- 172.21.15.45:0/2118865233 >> 172.21.15.45:0/2118865233 conn(0x7f4c140d2e90 msgr2=0x7f4c140d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:52.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.793+0000 7f4c1ace1640 1 -- 172.21.15.45:0/2118865233 shutdown_connections 2024-05-08T05:51:52.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.793+0000 7f4c1ace1640 1 -- 172.21.15.45:0/2118865233 wait complete. 2024-05-08T05:51:52.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.794+0000 7f4c1ace1640 1 Processor -- start 2024-05-08T05:51:52.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.794+0000 7f4c1ace1640 1 -- start start 2024-05-08T05:51:52.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.795+0000 7f4c1ace1640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c14070d30 0x7f4c140c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.795+0000 7f4c1ace1640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c14071110 0x7f4c141f9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.795+0000 7f4c1ace1640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c14163920 0x7f4c141f9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.795+0000 7f4c1ace1640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c140d1c30 con 0x7f4c14071110 2024-05-08T05:51:52.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.795+0000 7f4c1ace1640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f4c14070b80 con 0x7f4c14163920 2024-05-08T05:51:52.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.795+0000 7f4c1ace1640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f4c14071560 con 0x7f4c14070d30 2024-05-08T05:51:52.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.795+0000 7f4c0bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c14071110 0x7f4c141f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.795+0000 7f4c0bfff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c14071110 0x7f4c141f9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43602/0 (socket says 172.21.15.45:43602) 2024-05-08T05:51:52.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.795+0000 7f4c0bfff640 1 -- 172.21.15.45:0/3248640108 learned_addr learned my addr 172.21.15.45:0/3248640108 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:52.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.795+0000 7f4c18a7d640 1 --2- 172.21.15.45:0/3248640108 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c14070d30 0x7f4c140c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.795+0000 7f4c1927e640 1 --2- 172.21.15.45:0/3248640108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c14163920 0x7f4c141f9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.795+0000 7f4c0bfff640 1 -- 172.21.15.45:0/3248640108 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c14070d30 msgr2=0x7f4c140c69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.795+0000 7f4c0bfff640 1 --2- 172.21.15.45:0/3248640108 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c14070d30 0x7f4c140c69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.795+0000 7f4c0bfff640 1 -- 172.21.15.45:0/3248640108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c14163920 msgr2=0x7f4c141f9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.795+0000 7f4c0bfff640 1 --2- 172.21.15.45:0/3248640108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c14163920 0x7f4c141f9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.795+0000 7f4c0bfff640 1 -- 172.21.15.45:0/3248640108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c140d01a0 con 0x7f4c14071110 2024-05-08T05:51:52.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.796+0000 7f4c0bfff640 1 --2- 172.21.15.45:0/3248640108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c14071110 0x7f4c141f9100 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400c430 tx=0x7f4c0400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:52.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.796+0000 7f4c097fa640 1 -- 172.21.15.45:0/3248640108 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c04011790 con 0x7f4c14071110 2024-05-08T05:51:52.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.796+0000 7f4c097fa640 1 -- 172.21.15.45:0/3248640108 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c04017070 con 0x7f4c14071110 2024-05-08T05:51:52.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.796+0000 7f4c1ace1640 1 -- 172.21.15.45:0/3248640108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c140b6ed0 con 0x7f4c14071110 2024-05-08T05:51:52.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.796+0000 7f4c097fa640 1 -- 172.21.15.45:0/3248640108 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c04011b70 con 0x7f4c14071110 2024-05-08T05:51:52.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.796+0000 7f4c1ace1640 1 -- 172.21.15.45:0/3248640108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c140ccc50 con 0x7f4c14071110 2024-05-08T05:51:52.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.797+0000 7f4c1ace1640 1 -- 172.21.15.45:0/3248640108 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c140ccee0 con 0x7f4c14071110 2024-05-08T05:51:52.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.797+0000 7f4c097fa640 1 -- 172.21.15.45:0/3248640108 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4c04011d10 con 0x7f4c14071110 2024-05-08T05:51:52.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.798+0000 7f4c097fa640 1 --2- 172.21.15.45:0/3248640108 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4c00076300 0x7f4c000787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.799 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.798+0000 7f4c097fa640 1 -- 172.21.15.45:0/3248640108 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f4c040971f0 con 0x7f4c14071110 2024-05-08T05:51:52.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.798+0000 7f4c18a7d640 1 --2- 172.21.15.45:0/3248640108 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4c00076300 0x7f4c000787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.799+0000 7f4c18a7d640 1 --2- 172.21.15.45:0/3248640108 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4c00076300 0x7f4c000787c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4c10034000 tx=0x7f4c1002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:52.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.802+0000 7f4c097fa640 1 -- 172.21.15.45:0/3248640108 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4c040605c0 con 0x7f4c14071110 2024-05-08T05:51:52.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.992+0000 7f4c1ace1640 1 -- 172.21.15.45:0/3248640108 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f4c140693f0 con 0x7f4c00076300 2024-05-08T05:51:52.992 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.992+0000 7f4c097fa640 1 -- 172.21.15.45:0/3248640108 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4c140693f0 con 0x7f4c00076300 2024-05-08T05:51:52.993 INFO:teuthology.orchestra.run.smithi045.stderr:instructed osd(s) 7 to deep-scrub 2024-05-08T05:51:52.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.995+0000 7f4c1ace1640 1 -- 172.21.15.45:0/3248640108 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4c00076300 msgr2=0x7f4c000787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.995+0000 7f4c1ace1640 1 --2- 172.21.15.45:0/3248640108 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4c00076300 0x7f4c000787c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4c10034000 tx=0x7f4c1002e000 comp rx=0 tx=0).stop 2024-05-08T05:51:52.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.995+0000 7f4c1ace1640 1 -- 172.21.15.45:0/3248640108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c14071110 msgr2=0x7f4c141f9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.995+0000 7f4c1ace1640 1 --2- 172.21.15.45:0/3248640108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c14071110 0x7f4c141f9100 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400c430 tx=0x7f4c0400c8f0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.995+0000 7f4c1ace1640 1 -- 172.21.15.45:0/3248640108 shutdown_connections 2024-05-08T05:51:52.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.995+0000 7f4c1ace1640 1 --2- 172.21.15.45:0/3248640108 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f4c00076300 0x7f4c000787c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.995+0000 7f4c1ace1640 1 --2- 172.21.15.45:0/3248640108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4c14163920 0x7f4c141f9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.995+0000 7f4c1ace1640 1 --2- 172.21.15.45:0/3248640108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c14071110 0x7f4c141f9100 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.995+0000 7f4c1ace1640 1 --2- 172.21.15.45:0/3248640108 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f4c14070d30 0x7f4c140c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.995+0000 7f4c1ace1640 1 -- 172.21.15.45:0/3248640108 >> 172.21.15.45:0/3248640108 conn(0x7f4c140d2e90 msgr2=0x7f4c1406eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:52.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.995+0000 7f4c1ace1640 1 -- 172.21.15.45:0/3248640108 shutdown_connections 2024-05-08T05:51:52.996 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:52.995+0000 7f4c1ace1640 1 -- 172.21.15.45:0/3248640108 wait complete. 2024-05-08T05:51:53.007 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:51:53.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.112+0000 7f257ef51640 1 Processor -- start 2024-05-08T05:51:53.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.112+0000 7f257ef51640 1 -- start start 2024-05-08T05:51:53.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f257ef51640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f257815f380 0x7f25780d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f257ef51640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25780d1760 con 0x7f25780d3cd0 2024-05-08T05:51:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f257ef51640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25780d1a40 con 0x7f25780d0cf0 2024-05-08T05:51:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f257ef51640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f25780d30f0 con 0x7f257815f380 2024-05-08T05:51:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f257d4ee640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7f25780d3cd0 0x7f25780b7e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:50012/0 (socket says 172.21.15.45:50012) 2024-05-08T05:51:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f257d4ee640 1 -- 172.21.15.45:0/3926667158 learned_addr learned my addr 172.21.15.45:0/3926667158 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:51:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f257cced640 1 --2- 172.21.15.45:0/3926667158 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f257815f380 0x7f25780d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f25777fe640 1 -- 172.21.15.45:0/3926667158 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3371169608 0 0) 0x7f25780d1760 con 0x7f25780d3cd0 2024-05-08T05:51:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f25777fe640 1 -- 172.21.15.45:0/3926667158 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2568003520 con 0x7f25780d3cd0 2024-05-08T05:51:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f25777fe640 1 -- 172.21.15.45:0/3926667158 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 862389149 0 0) 0x7f25780d1a40 con 0x7f25780d0cf0 2024-05-08T05:51:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f25777fe640 1 -- 172.21.15.45:0/3926667158 --> v1:172.21.15.63:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25780d1760 con 0x7f25780d0cf0 2024-05-08T05:51:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f25777fe640 1 -- 172.21.15.45:0/3926667158 <== mon.0 v1:172.21.15.45:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3858216685 0 0) 0x7f2568003520 con 0x7f25780d3cd0 2024-05-08T05:51:53.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f25777fe640 1 -- 172.21.15.45:0/3926667158 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f257815f380 msgr2=0x7f25780d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f25777fe640 1 --2- 172.21.15.45:0/3926667158 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f257815f380 0x7f25780d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f25777fe640 1 -- 172.21.15.45:0/3926667158 >> v1:172.21.15.63:6789/0 conn(0x7f25780d0cf0 legacy=0x7f25780d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f25777fe640 1 -- 172.21.15.45:0/3926667158 --> v1:172.21.15.45:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25780d1d20 con 0x7f25780d3cd0 2024-05-08T05:51:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f25777fe640 1 -- 172.21.15.45:0/3926667158 <== mon.0 v1:172.21.15.45:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2414100598 0 0) 0x7f25700019f0 con 0x7f25780d3cd0 2024-05-08T05:51:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f25777fe640 1 -- 172.21.15.45:0/3926667158 >> v1:172.21.15.45:6789/0 conn(0x7f25780d3cd0 legacy=0x7f25780b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f25777fe640 1 --2- 172.21.15.45:0/3926667158 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f257815f380 0x7f2568003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f25777fe640 1 --2- 172.21.15.45:0/3926667158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2568004210 0x7f2568004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f25777fe640 1 --2- 172.21.15.45:0/3926667158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2568004d90 0x7f2568005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f25777fe640 1 -- 172.21.15.45:0/3926667158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f25780d30f0 con 0x7f2568004210 2024-05-08T05:51:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f25777fe640 1 -- 172.21.15.45:0/3926667158 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f256800bbd0 con 0x7f2568004d90 2024-05-08T05:51:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.113+0000 7f25777fe640 1 -- 172.21.15.45:0/3926667158 --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f256800bdd0 con 0x7f257815f380 2024-05-08T05:51:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.114+0000 7f2577fff640 1 --2- 172.21.15.45:0/3926667158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2568004210 0x7f2568004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.114+0000 7f257cced640 1 --2- 172.21.15.45:0/3926667158 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f257815f380 0x7f2568003c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.114+0000 7f257d4ee640 1 --2- 172.21.15.45:0/3926667158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2568004d90 0x7f2568005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:53.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.114+0000 7f2577fff640 1 -- 172.21.15.45:0/3926667158 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f257815f380 msgr2=0x7f2568003c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:53.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.114+0000 7f2577fff640 1 --2- 172.21.15.45:0/3926667158 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f257815f380 0x7f2568003c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:53.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.114+0000 7f2577fff640 1 -- 172.21.15.45:0/3926667158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2568004d90 msgr2=0x7f2568005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:51:53.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.114+0000 7f2577fff640 1 --2- 172.21.15.45:0/3926667158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2568004d90 0x7f2568005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:53.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.114+0000 7f2577fff640 1 -- 172.21.15.45:0/3926667158 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2568003520 con 0x7f2568004210 2024-05-08T05:51:53.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.114+0000 7f257d4ee640 1 --2- 172.21.15.45:0/3926667158 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2568004d90 0x7f2568005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:53.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.114+0000 7f2577fff640 1 --2- 172.21.15.45:0/3926667158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2568004210 0x7f2568004800 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f25640018a0 tx=0x7f2568005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5971482652abf0f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:53.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.115+0000 7f25777fe640 1 -- 172.21.15.45:0/3926667158 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25640014c0 con 0x7f2568004210 2024-05-08T05:51:53.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.115+0000 7f25777fe640 1 -- 172.21.15.45:0/3926667158 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f256402acf0 con 0x7f2568004210 2024-05-08T05:51:53.118 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.115+0000 7f25777fe640 1 -- 172.21.15.45:0/3926667158 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2564005310 con 0x7f2568004210 2024-05-08T05:51:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.115+0000 7f257ef51640 1 -- 172.21.15.45:0/3926667158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2568004210 msgr2=0x7f2568004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.115+0000 7f257ef51640 1 --2- 172.21.15.45:0/3926667158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2568004210 0x7f2568004800 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f25640018a0 tx=0x7f2568005830 comp rx=0 tx=0).stop 2024-05-08T05:51:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.115+0000 7f257d4ee640 1 -- 172.21.15.45:0/3926667158 reap_dead start 2024-05-08T05:51:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.115+0000 7f257ef51640 1 -- 172.21.15.45:0/3926667158 shutdown_connections 2024-05-08T05:51:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.115+0000 7f257ef51640 1 -- 172.21.15.45:0/3926667158 >> 172.21.15.45:0/3926667158 conn(0x7f25780d4fe0 msgr2=0x7f25780d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.115+0000 7f257ef51640 1 -- 172.21.15.45:0/3926667158 shutdown_connections 2024-05-08T05:51:53.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.115+0000 7f257ef51640 1 -- 172.21.15.45:0/3926667158 wait complete. 2024-05-08T05:51:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.116+0000 7f257ef51640 1 Processor -- start 2024-05-08T05:51:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.116+0000 7f257ef51640 1 -- start start 2024-05-08T05:51:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.117+0000 7f257ef51640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2568004210 0x7f25781f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.117+0000 7f257ef51640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25781584e0 0x7f257815b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.117+0000 7f257ef51640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f257814e250 0x7f25780cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.117+0000 7f257ef51640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f25780d2370 con 0x7f25781584e0 2024-05-08T05:51:53.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.117+0000 7f257ef51640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f257815f760 con 0x7f2568004210 2024-05-08T05:51:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.117+0000 7f257ef51640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f25780d2800 con 0x7f257814e250 2024-05-08T05:51:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.117+0000 7f2577fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25781584e0 0x7f257815b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.117+0000 7f2577fff640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25781584e0 0x7f257815b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43616/0 (socket says 172.21.15.45:43616) 2024-05-08T05:51:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.117+0000 7f2577fff640 1 -- 172.21.15.45:0/1731794122 learned_addr learned my addr 172.21.15.45:0/1731794122 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:51:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.118+0000 7f257cced640 1 --2- 172.21.15.45:0/1731794122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2568004210 0x7f25781f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.118+0000 7f257d4ee640 1 --2- 172.21.15.45:0/1731794122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f257814e250 0x7f25780cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.118+0000 7f2577fff640 1 -- 172.21.15.45:0/1731794122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f257814e250 msgr2=0x7f25780cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.118+0000 7f2577fff640 1 --2- 172.21.15.45:0/1731794122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f257814e250 0x7f25780cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.118+0000 7f2577fff640 1 -- 172.21.15.45:0/1731794122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2568004210 msgr2=0x7f25781f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.118+0000 7f2577fff640 1 --2- 172.21.15.45:0/1731794122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2568004210 0x7f25781f52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.118+0000 7f2577fff640 1 -- 172.21.15.45:0/1731794122 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257815a4d0 con 0x7f25781584e0 2024-05-08T05:51:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.118+0000 7f257cced640 1 --2- 172.21.15.45:0/1731794122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2568004210 0x7f25781f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:53.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.118+0000 7f2577fff640 1 --2- 172.21.15.45:0/1731794122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25781584e0 0x7f257815b920 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f25640019c0 tx=0x7f25640012f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:53.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.118+0000 7f25757fa640 1 -- 172.21.15.45:0/1731794122 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2564007a10 con 0x7f25781584e0 2024-05-08T05:51:53.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.118+0000 7f25757fa640 1 -- 172.21.15.45:0/1731794122 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25640075e0 con 0x7f25781584e0 2024-05-08T05:51:53.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.118+0000 7f257ef51640 1 -- 172.21.15.45:0/1731794122 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25781f5810 con 0x7f25781584e0 2024-05-08T05:51:53.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.118+0000 7f25757fa640 1 -- 172.21.15.45:0/1731794122 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2564006a10 con 0x7f25781584e0 2024-05-08T05:51:53.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.118+0000 7f257ef51640 1 -- 172.21.15.45:0/1731794122 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25780cd750 con 0x7f25781584e0 2024-05-08T05:51:53.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.119+0000 7f257ef51640 1 -- 172.21.15.45:0/1731794122 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25780693f0 con 0x7f25781584e0 2024-05-08T05:51:53.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.120+0000 7f25757fa640 1 -- 172.21.15.45:0/1731794122 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2564006bb0 con 0x7f25781584e0 2024-05-08T05:51:53.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.120+0000 7f25757fa640 1 --2- 172.21.15.45:0/1731794122 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f255c0762d0 0x7f255c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:53.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.121+0000 7f25757fa640 1 -- 172.21.15.45:0/1731794122 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f25640b6c80 con 0x7f25781584e0 2024-05-08T05:51:53.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.121+0000 7f257cced640 1 --2- 172.21.15.45:0/1731794122 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f255c0762d0 0x7f255c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:53.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.121+0000 7f257cced640 1 --2- 172.21.15.45:0/1731794122 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f255c0762d0 0x7f255c078790 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f256c009ae0 tx=0x7f256c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:53.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.125+0000 7f25757fa640 1 -- 172.21.15.45:0/1731794122 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2564080780 con 0x7f25781584e0 2024-05-08T05:51:53.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.312+0000 7f257ef51640 1 -- 172.21.15.45:0/1731794122 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f257808cfd0 con 0x7f255c0762d0 2024-05-08T05:51:53.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.313+0000 7f25757fa640 1 -- 172.21.15.45:0/1731794122 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13449 (secure 0 0 0) 0x7f257808cfd0 con 0x7f255c0762d0 2024-05-08T05:51:53.313 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:51:53.313 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:51:53.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.314+0000 7f257ef51640 1 -- 172.21.15.45:0/1731794122 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f255c0762d0 msgr2=0x7f255c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:53.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.314+0000 7f257ef51640 1 --2- 172.21.15.45:0/1731794122 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f255c0762d0 0x7f255c078790 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f256c009ae0 tx=0x7f256c008040 comp rx=0 tx=0).stop 2024-05-08T05:51:53.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.315+0000 7f257ef51640 1 -- 172.21.15.45:0/1731794122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25781584e0 msgr2=0x7f257815b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:53.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.315+0000 7f257ef51640 1 --2- 172.21.15.45:0/1731794122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25781584e0 0x7f257815b920 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f25640019c0 tx=0x7f25640012f0 comp rx=0 tx=0).stop 2024-05-08T05:51:53.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.315+0000 7f257ef51640 1 -- 172.21.15.45:0/1731794122 shutdown_connections 2024-05-08T05:51:53.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.315+0000 7f257ef51640 1 --2- 172.21.15.45:0/1731794122 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f255c0762d0 0x7f255c078790 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:53.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.315+0000 7f257ef51640 1 --2- 172.21.15.45:0/1731794122 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f257814e250 0x7f25780cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:53.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.315+0000 7f257ef51640 1 --2- 172.21.15.45:0/1731794122 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f25781584e0 0x7f257815b920 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:53.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.315+0000 7f257ef51640 1 --2- 172.21.15.45:0/1731794122 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2568004210 0x7f25781f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:53.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.315+0000 7f257ef51640 1 -- 172.21.15.45:0/1731794122 >> 172.21.15.45:0/1731794122 conn(0x7f25780d4fe0 msgr2=0x7f25780b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:53.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.315+0000 7f257ef51640 1 -- 172.21.15.45:0/1731794122 shutdown_connections 2024-05-08T05:51:53.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:51:53.315+0000 7f257ef51640 1 -- 172.21.15.45:0/1731794122 wait complete. 2024-05-08T05:51:53.326 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-05-08T05:51:52.985423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":266876,"kb_used_data":13256,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754707844,"statfs":{"total":773094113280,"available":772820832256,"internally_reserved":0,"allocated":13574144,"data_stored":6959168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8388625,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":-32,"num_read_kb":-13,"num_write":-29,"num_write_kb":-8195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.974012"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":303,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-05-08T05:51:46.248278+0000","last_change":"2024-05-08T05:41:20.478822+0000","last_active":"2024-05-08T05:51:46.248278+0000","last_peered":"2024-05-08T05:51:46.248278+0000","last_clean":"2024-05-08T05:51:46.248278+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:51:46.248278+0000","last_undegraded":"2024-05-08T05:51:46.248278+0000","last_fullsized":"2024-05-08T05:51:46.248278+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:41:18.996355+0000","last_clean_scrub_stamp":"2024-05-08T05:41:18.996355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:53.411771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542274,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2142208,"data_stored":1312672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542275,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32692,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1548288,"data_stored":722304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542274,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32692,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1548288,"data_stored":722304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574977,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32692,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1548288,"data_stored":722304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32692,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1548288,"data_stored":722304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32696,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339144,"statfs":{"total":96636764160,"available":96603283456,"internally_reserved":0,"allocated":1548288,"data_stored":722304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32700,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1548288,"data_stored":722304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574976,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37436,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334404,"statfs":{"total":96636764160,"available":96598429696,"internally_reserved":0,"allocated":2142208,"data_stored":1312672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:51:53.329 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-08T05:41:18.996355+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=5, tm_min=41, tm_sec=18, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=5, tm_min=51, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T05:51:53.329 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T05:52:05.432 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:52:05.436 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:52:13.330 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T05:52:13.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.447+0000 7f685b538640 1 Processor -- start 2024-05-08T05:52:13.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.447+0000 7f685b538640 1 -- start start 2024-05-08T05:52:13.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.448+0000 7f685b538640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f685415f380 0x7f68540d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.448+0000 7f685b538640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f68540d0cf0 0x7f68540d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.448+0000 7f685b538640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68540d1760 con 0x7f68540d3b00 2024-05-08T05:52:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.448+0000 7f685b538640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f68540d30f0 con 0x7f68540d0cf0 2024-05-08T05:52:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.448+0000 7f685b538640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f68540d1150 con 0x7f685415f380 2024-05-08T05:52:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.448+0000 7f6858ad3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f68540d0cf0 0x7f68540d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.448+0000 7f68592d4640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f685415f380 0x7f68540d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.448+0000 7f6858ad3640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f68540d0cf0 0x7f68540d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:35024/0 (socket says 172.21.15.45:35024) 2024-05-08T05:52:13.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.449+0000 7f6858ad3640 1 -- 172.21.15.45:0/1751690757 learned_addr learned my addr 172.21.15.45:0/1751690757 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:52:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.449+0000 7f6858ad3640 1 -- 172.21.15.45:0/1751690757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f685415f380 msgr2=0x7f68540d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:52:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.449+0000 7f6858ad3640 1 --2- 172.21.15.45:0/1751690757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f685415f380 0x7f68540d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.449+0000 7f6858ad3640 1 -- 172.21.15.45:0/1751690757 >> v1:172.21.15.63:6789/0 conn(0x7f68540d3b00 legacy=0x7f6854162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.449+0000 7f684bfff640 1 -- 172.21.15.45:0/1751690757 <== mon.1 v1:172.21.15.63:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1057181971 0 0) 0x7f68540d1760 con 0x7f68540d3b00 2024-05-08T05:52:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.449+0000 7f6858ad3640 1 -- 172.21.15.45:0/1751690757 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68540d1a40 con 0x7f68540d0cf0 2024-05-08T05:52:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.449+0000 7f68592d4640 1 --2- 172.21.15.45:0/1751690757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f685415f380 0x7f68540d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:52:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.450+0000 7f6858ad3640 1 --2- 172.21.15.45:0/1751690757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68540d0cf0 0x7f68540d35c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6844009870 tx=0x7f684402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d5f327f345c3723 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.450+0000 7f684bfff640 1 -- 172.21.15.45:0/1751690757 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6844037070 con 0x7f68540d0cf0 2024-05-08T05:52:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.450+0000 7f684bfff640 1 -- 172.21.15.45:0/1751690757 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6844032440 con 0x7f68540d0cf0 2024-05-08T05:52:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.450+0000 7f684bfff640 1 -- 172.21.15.45:0/1751690757 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6844032810 con 0x7f68540d0cf0 2024-05-08T05:52:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.450+0000 7f685b538640 1 -- 172.21.15.45:0/1751690757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68540d0cf0 msgr2=0x7f68540d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.450+0000 7f685b538640 1 --2- 172.21.15.45:0/1751690757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68540d0cf0 0x7f68540d35c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6844009870 tx=0x7f684402a250 comp rx=0 tx=0).stop 2024-05-08T05:52:13.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.451+0000 7f685b538640 1 -- 172.21.15.45:0/1751690757 shutdown_connections 2024-05-08T05:52:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.451+0000 7f685b538640 1 --2- 172.21.15.45:0/1751690757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68540d0cf0 0x7f68540d35c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.451+0000 7f685b538640 1 --2- 172.21.15.45:0/1751690757 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f685415f380 0x7f68540d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.451+0000 7f685b538640 1 -- 172.21.15.45:0/1751690757 >> 172.21.15.45:0/1751690757 conn(0x7f68540d4fe0 msgr2=0x7f68540d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.451+0000 7f685b538640 1 -- 172.21.15.45:0/1751690757 shutdown_connections 2024-05-08T05:52:13.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.451+0000 7f685b538640 1 -- 172.21.15.45:0/1751690757 wait complete. 2024-05-08T05:52:13.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.452+0000 7f685b538640 1 Processor -- start 2024-05-08T05:52:13.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.452+0000 7f685b538640 1 -- start start 2024-05-08T05:52:13.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.453+0000 7f685b538640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68540d0cf0 0x7f685409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.453+0000 7f685b538640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68540d3b00 0x7f6854159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.453+0000 7f685b538640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f685415f380 0x7f68540b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.453+0000 7f685b538640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f68540b8e40 con 0x7f685415f380 2024-05-08T05:52:13.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.453+0000 7f685b538640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f68540d2370 con 0x7f68540d3b00 2024-05-08T05:52:13.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.453+0000 7f685b538640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f68540d2000 con 0x7f68540d0cf0 2024-05-08T05:52:13.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.453+0000 7f6859ad5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f685415f380 0x7f68540b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.453+0000 7f6859ad5640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f685415f380 0x7f68540b6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56296/0 (socket says 172.21.15.45:56296) 2024-05-08T05:52:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.453+0000 7f6859ad5640 1 -- 172.21.15.45:0/1159231864 learned_addr learned my addr 172.21.15.45:0/1159231864 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:52:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.453+0000 7f6858ad3640 1 --2- 172.21.15.45:0/1159231864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68540d3b00 0x7f6854159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.453+0000 7f68592d4640 1 --2- 172.21.15.45:0/1159231864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68540d0cf0 0x7f685409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.453+0000 7f6859ad5640 1 -- 172.21.15.45:0/1159231864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68540d0cf0 msgr2=0x7f685409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.453+0000 7f6859ad5640 1 --2- 172.21.15.45:0/1159231864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68540d0cf0 0x7f685409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.453+0000 7f6859ad5640 1 -- 172.21.15.45:0/1159231864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68540d3b00 msgr2=0x7f6854159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.453+0000 7f6859ad5640 1 --2- 172.21.15.45:0/1159231864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68540d3b00 0x7f6854159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.453+0000 7f6859ad5640 1 -- 172.21.15.45:0/1159231864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68540a2bf0 con 0x7f685415f380 2024-05-08T05:52:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.453+0000 7f6859ad5640 1 --2- 172.21.15.45:0/1159231864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f685415f380 0x7f68540b6e50 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f684c00c770 tx=0x7f684c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.454+0000 7f6849ffb640 1 -- 172.21.15.45:0/1159231864 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c003da0 con 0x7f685415f380 2024-05-08T05:52:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.454+0000 7f6849ffb640 1 -- 172.21.15.45:0/1159231864 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f684c005070 con 0x7f685415f380 2024-05-08T05:52:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.454+0000 7f685b538640 1 -- 172.21.15.45:0/1159231864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685415a200 con 0x7f685415f380 2024-05-08T05:52:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.454+0000 7f6849ffb640 1 -- 172.21.15.45:0/1159231864 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c005460 con 0x7f685415f380 2024-05-08T05:52:13.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.454+0000 7f685b538640 1 -- 172.21.15.45:0/1159231864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f685409f6c0 con 0x7f685415f380 2024-05-08T05:52:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.455+0000 7f685b538640 1 -- 172.21.15.45:0/1159231864 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68540693f0 con 0x7f685415f380 2024-05-08T05:52:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.456+0000 7f6849ffb640 1 -- 172.21.15.45:0/1159231864 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f684c005600 con 0x7f685415f380 2024-05-08T05:52:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.456+0000 7f6849ffb640 1 --2- 172.21.15.45:0/1159231864 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f68380762d0 0x7f6838078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.456+0000 7f6849ffb640 1 -- 172.21.15.45:0/1159231864 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f684c0973c0 con 0x7f685415f380 2024-05-08T05:52:13.456 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.456+0000 7f68592d4640 1 --2- 172.21.15.45:0/1159231864 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f68380762d0 0x7f6838078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.457 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.457+0000 7f68592d4640 1 --2- 172.21.15.45:0/1159231864 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f68380762d0 0x7f6838078790 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6850009a30 tx=0x7f6850008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:13.461 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.461+0000 7f6849ffb640 1 -- 172.21.15.45:0/1159231864 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f684c060d60 con 0x7f685415f380 2024-05-08T05:52:13.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.648+0000 7f685b538640 1 -- 172.21.15.45:0/1159231864 --> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f685408cfd0 con 0x7f68380762d0 2024-05-08T05:52:13.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.649+0000 7f6849ffb640 1 -- 172.21.15.45:0/1159231864 <== mgr.4099 v2:172.21.15.63:6832/2860796916 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13364 (secure 0 0 0) 0x7f685408cfd0 con 0x7f68380762d0 2024-05-08T05:52:13.649 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:52:13.649 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-05-08T05:52:13.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.651+0000 7f685b538640 1 -- 172.21.15.45:0/1159231864 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f68380762d0 msgr2=0x7f6838078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.651+0000 7f685b538640 1 --2- 172.21.15.45:0/1159231864 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f68380762d0 0x7f6838078790 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6850009a30 tx=0x7f6850008040 comp rx=0 tx=0).stop 2024-05-08T05:52:13.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.651+0000 7f685b538640 1 -- 172.21.15.45:0/1159231864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f685415f380 msgr2=0x7f68540b6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.651+0000 7f685b538640 1 --2- 172.21.15.45:0/1159231864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f685415f380 0x7f68540b6e50 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f684c00c770 tx=0x7f684c00cc30 comp rx=0 tx=0).stop 2024-05-08T05:52:13.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.651+0000 7f685b538640 1 -- 172.21.15.45:0/1159231864 shutdown_connections 2024-05-08T05:52:13.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.651+0000 7f685b538640 1 --2- 172.21.15.45:0/1159231864 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f68380762d0 0x7f6838078790 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.651+0000 7f685b538640 1 --2- 172.21.15.45:0/1159231864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f685415f380 0x7f68540b6e50 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.651+0000 7f685b538640 1 --2- 172.21.15.45:0/1159231864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68540d3b00 0x7f6854159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.651+0000 7f685b538640 1 --2- 172.21.15.45:0/1159231864 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f68540d0cf0 0x7f685409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.651+0000 7f685b538640 1 -- 172.21.15.45:0/1159231864 >> 172.21.15.45:0/1159231864 conn(0x7f68540d4fe0 msgr2=0x7f68540d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:13.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.652+0000 7f685b538640 1 -- 172.21.15.45:0/1159231864 shutdown_connections 2024-05-08T05:52:13.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.652+0000 7f685b538640 1 -- 172.21.15.45:0/1159231864 wait complete. 2024-05-08T05:52:13.663 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-05-08T05:52:12.989626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":266860,"kb_used_data":13256,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754707860,"statfs":{"total":773094113280,"available":772820848640,"internally_reserved":0,"allocated":13574144,"data_stored":6959168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002851"},"pg_stats":[{"pgid":"1.0","version":"124'74","reported_seq":314,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-05-08T05:51:53.084816+0000","last_change":"2024-05-08T05:51:53.084849+0000","last_active":"2024-05-08T05:51:53.084816+0000","last_peered":"2024-05-08T05:51:53.084816+0000","last_clean":"2024-05-08T05:51:53.084816+0000","last_became_active":"2024-05-08T05:41:20.477744+0000","last_became_peered":"2024-05-08T05:41:20.477744+0000","last_unstale":"2024-05-08T05:51:53.084816+0000","last_undegraded":"2024-05-08T05:51:53.084816+0000","last_fullsized":"2024-05-08T05:51:53.084816+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"124'74","last_scrub_stamp":"2024-05-08T05:51:53.084727+0000","last_deep_scrub":"124'74","last_deep_scrub_stamp":"2024-05-08T05:51:53.084727+0000","last_clean_scrub_stamp":"2024-05-08T05:51:53.084727+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:26.200103+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542278,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2142208,"data_stored":1312672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542279,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32692,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1548288,"data_stored":722304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542278,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32692,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1548288,"data_stored":722304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32692,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1548288,"data_stored":722304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574980,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32692,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1548288,"data_stored":722304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32696,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339144,"statfs":{"total":96636764160,"available":96603283456,"internally_reserved":0,"allocated":1548288,"data_stored":722304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32700,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1548288,"data_stored":722304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574980,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37428,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334412,"statfs":{"total":96636764160,"available":96598437888,"internally_reserved":0,"allocated":2142208,"data_stored":1312672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:52:13.664 DEBUG:teuthology.orchestra.run.smithi045:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-08T05:52:13.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.767+0000 7fc8ad932640 1 Processor -- start 2024-05-08T05:52:13.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.767+0000 7fc8ad932640 1 -- start start 2024-05-08T05:52:13.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.768+0000 7fc8ad932640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7fc8a81606d0 0x7fc8a80cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.768+0000 7fc8ad932640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc8a8071a00 0x7fc8a80ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.768+0000 7fc8ad932640 1 -- --> v1:172.21.15.45:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8a80d15c0 con 0x7fc8a80cdf00 2024-05-08T05:52:13.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.768+0000 7fc8ad932640 1 -- --> v2:172.21.15.45:3300/0 -- mon_getmap magic: 0 -- 0x7fc8a80ce8d0 con 0x7fc8a8071a00 2024-05-08T05:52:13.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.768+0000 7fc8ad932640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7fc8a80d18a0 con 0x7fc8a81606d0 2024-05-08T05:52:13.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.768+0000 7fc8a77fe640 1 --2- >> v2:172.21.15.45:3300/0 conn(0x7fc8a8071a00 0x7fc8a80ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.768+0000 7fc8a67fc640 1 --1- >> v1:172.21.15.45:6789/0 conn(0x7fc8a80cdf00 0x7fc8a80cf580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.45:6789/0 says I am v1:172.21.15.45:59406/0 (socket says 172.21.15.45:59406) 2024-05-08T05:52:13.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.768+0000 7fc8a67fc640 1 -- 172.21.15.45:0/3470376463 learned_addr learned my addr 172.21.15.45:0/3470376463 (peer_addr_for_me v1:172.21.15.45:0/0) 2024-05-08T05:52:13.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.768+0000 7fc8a6ffd640 1 --2- 172.21.15.45:0/3470376463 >> v2:172.21.15.63:3300/0 conn(0x7fc8a81606d0 0x7fc8a80cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.768+0000 7fc8a5ffb640 1 -- 172.21.15.45:0/3470376463 <== mon.0 v1:172.21.15.45:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1288088231 0 0) 0x7fc8a80d15c0 con 0x7fc8a80cdf00 2024-05-08T05:52:13.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.768+0000 7fc8a5ffb640 1 -- 172.21.15.45:0/3470376463 --> v1:172.21.15.45:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc890003520 con 0x7fc8a80cdf00 2024-05-08T05:52:13.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.768+0000 7fc8a77fe640 1 -- 172.21.15.45:0/3470376463 >> v2:172.21.15.63:3300/0 conn(0x7fc8a81606d0 msgr2=0x7fc8a80cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:52:13.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.768+0000 7fc8a77fe640 1 --2- 172.21.15.45:0/3470376463 >> v2:172.21.15.63:3300/0 conn(0x7fc8a81606d0 0x7fc8a80cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.768+0000 7fc8a77fe640 1 -- 172.21.15.45:0/3470376463 >> v1:172.21.15.45:6789/0 conn(0x7fc8a80cdf00 legacy=0x7fc8a80cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.768+0000 7fc8a77fe640 1 -- 172.21.15.45:0/3470376463 --> v2:172.21.15.45:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a80d1b80 con 0x7fc8a8071a00 2024-05-08T05:52:13.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.769+0000 7fc8a77fe640 1 --2- 172.21.15.45:0/3470376463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a8071a00 0x7fc8a80ce390 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80731f0 tx=0x7fc8a002a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3473fcae03d4cd66 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:13.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.769+0000 7fc8a5ffb640 1 -- 172.21.15.45:0/3470376463 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0007740 con 0x7fc8a8071a00 2024-05-08T05:52:13.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.769+0000 7fc8a5ffb640 1 -- 172.21.15.45:0/3470376463 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a00078e0 con 0x7fc8a8071a00 2024-05-08T05:52:13.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.769+0000 7fc8a5ffb640 1 -- 172.21.15.45:0/3470376463 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0007cb0 con 0x7fc8a8071a00 2024-05-08T05:52:13.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.770+0000 7fc8ad932640 1 -- 172.21.15.45:0/3470376463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a8071a00 msgr2=0x7fc8a80ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.770+0000 7fc8ad932640 1 --2- 172.21.15.45:0/3470376463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a8071a00 0x7fc8a80ce390 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80731f0 tx=0x7fc8a002a560 comp rx=0 tx=0).stop 2024-05-08T05:52:13.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.770+0000 7fc8ad932640 1 -- 172.21.15.45:0/3470376463 shutdown_connections 2024-05-08T05:52:13.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.770+0000 7fc8ad932640 1 --2- 172.21.15.45:0/3470376463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a8071a00 0x7fc8a80ce390 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.770+0000 7fc8ad932640 1 --2- 172.21.15.45:0/3470376463 >> v2:172.21.15.63:3300/0 conn(0x7fc8a81606d0 0x7fc8a80cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.770+0000 7fc8ad932640 1 -- 172.21.15.45:0/3470376463 >> 172.21.15.45:0/3470376463 conn(0x7fc8a80b49d0 msgr2=0x7fc8a80b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:13.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.770+0000 7fc8ad932640 1 -- 172.21.15.45:0/3470376463 shutdown_connections 2024-05-08T05:52:13.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.770+0000 7fc8ad932640 1 -- 172.21.15.45:0/3470376463 wait complete. 2024-05-08T05:52:13.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.771+0000 7fc8ad932640 1 Processor -- start 2024-05-08T05:52:13.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.771+0000 7fc8ad932640 1 -- start start 2024-05-08T05:52:13.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.772+0000 7fc8ad932640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a8071a00 0x7fc8a81583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.772+0000 7fc8ad932640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8a80cdf00 0x7fc8a80cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.772+0000 7fc8ad932640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8a81606d0 0x7fc8a80b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.772+0000 7fc8ad932640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a80cfc90 con 0x7fc8a8071a00 2024-05-08T05:52:13.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.772+0000 7fc8ad932640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a80b5710 con 0x7fc8a80cdf00 2024-05-08T05:52:13.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.772+0000 7fc8ad932640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7fc8a80cfe10 con 0x7fc8a81606d0 2024-05-08T05:52:13.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.772+0000 7fc8a6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a8071a00 0x7fc8a81583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.772+0000 7fc8a6ffd640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a8071a00 0x7fc8a81583a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56308/0 (socket says 172.21.15.45:56308) 2024-05-08T05:52:13.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.772+0000 7fc8a6ffd640 1 -- 172.21.15.45:0/277229352 learned_addr learned my addr 172.21.15.45:0/277229352 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:52:13.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.772+0000 7fc8a67fc640 1 --2- 172.21.15.45:0/277229352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8a80cdf00 0x7fc8a80cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.772+0000 7fc8a6ffd640 1 -- 172.21.15.45:0/277229352 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8a81606d0 msgr2=0x7fc8a80b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.772+0000 7fc8a77fe640 1 --2- 172.21.15.45:0/277229352 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8a81606d0 0x7fc8a80b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.772+0000 7fc8a6ffd640 1 --2- 172.21.15.45:0/277229352 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8a81606d0 0x7fc8a80b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.772+0000 7fc8a6ffd640 1 -- 172.21.15.45:0/277229352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8a80cdf00 msgr2=0x7fc8a80cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.772+0000 7fc8a6ffd640 1 --2- 172.21.15.45:0/277229352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8a80cdf00 0x7fc8a80cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.772+0000 7fc8a6ffd640 1 -- 172.21.15.45:0/277229352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a80cd080 con 0x7fc8a8071a00 2024-05-08T05:52:13.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.772+0000 7fc8a67fc640 1 --2- 172.21.15.45:0/277229352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8a80cdf00 0x7fc8a80cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:52:13.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.772+0000 7fc8a77fe640 1 --2- 172.21.15.45:0/277229352 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8a81606d0 0x7fc8a80b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:52:13.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.773+0000 7fc8a6ffd640 1 --2- 172.21.15.45:0/277229352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a8071a00 0x7fc8a81583a0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fc89800e870 tx=0x7fc89800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:13.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.773+0000 7fc88ffff640 1 -- 172.21.15.45:0/277229352 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc898017070 con 0x7fc8a8071a00 2024-05-08T05:52:13.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.773+0000 7fc88ffff640 1 -- 172.21.15.45:0/277229352 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc89800c940 con 0x7fc8a8071a00 2024-05-08T05:52:13.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.773+0000 7fc8ad932640 1 -- 172.21.15.45:0/277229352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a8159250 con 0x7fc8a8071a00 2024-05-08T05:52:13.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.773+0000 7fc88ffff640 1 -- 172.21.15.45:0/277229352 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89800cd30 con 0x7fc8a8071a00 2024-05-08T05:52:13.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.773+0000 7fc8ad932640 1 -- 172.21.15.45:0/277229352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a8159810 con 0x7fc8a8071a00 2024-05-08T05:52:13.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.774+0000 7fc8ad932640 1 -- 172.21.15.45:0/277229352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a8159040 con 0x7fc8a8071a00 2024-05-08T05:52:13.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.775+0000 7fc88ffff640 1 -- 172.21.15.45:0/277229352 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc89801a070 con 0x7fc8a8071a00 2024-05-08T05:52:13.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.775+0000 7fc88ffff640 1 --2- 172.21.15.45:0/277229352 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc894076300 0x7fc8940787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.775+0000 7fc88ffff640 1 -- 172.21.15.45:0/277229352 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7fc898097780 con 0x7fc8a8071a00 2024-05-08T05:52:13.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.775+0000 7fc8a67fc640 1 --2- 172.21.15.45:0/277229352 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc894076300 0x7fc8940787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.776+0000 7fc8a67fc640 1 --2- 172.21.15.45:0/277229352 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc894076300 0x7fc8940787c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc89c001220 tx=0x7fc89c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:13.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.779+0000 7fc88ffff640 1 -- 172.21.15.45:0/277229352 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc8980611a0 con 0x7fc8a8071a00 2024-05-08T05:52:13.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.972+0000 7fc8ad932640 1 -- 172.21.15.45:0/277229352 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fc8a80693f0 con 0x7fc8a8071a00 2024-05-08T05:52:13.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.975+0000 7fc88ffff640 1 -- 172.21.15.45:0/277229352 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) ==== 123+0+0 (secure 0 0 0) 0x7fc898060aa0 con 0x7fc8a8071a00 2024-05-08T05:52:13.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.976+0000 7fc88ffff640 1 -- 172.21.15.45:0/277229352 <== mon.0 v2:172.21.15.45:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fc8980605b0 con 0x7fc8a8071a00 2024-05-08T05:52:13.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.978+0000 7fc8ad932640 1 -- 172.21.15.45:0/277229352 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc894076300 msgr2=0x7fc8940787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.978+0000 7fc8ad932640 1 --2- 172.21.15.45:0/277229352 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc894076300 0x7fc8940787c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc89c001220 tx=0x7fc89c009040 comp rx=0 tx=0).stop 2024-05-08T05:52:13.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.978+0000 7fc8ad932640 1 -- 172.21.15.45:0/277229352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a8071a00 msgr2=0x7fc8a81583a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.978+0000 7fc8ad932640 1 --2- 172.21.15.45:0/277229352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a8071a00 0x7fc8a81583a0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fc89800e870 tx=0x7fc89800ed30 comp rx=0 tx=0).stop 2024-05-08T05:52:13.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.978+0000 7fc8ad932640 1 -- 172.21.15.45:0/277229352 shutdown_connections 2024-05-08T05:52:13.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.978+0000 7fc8ad932640 1 --2- 172.21.15.45:0/277229352 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7fc894076300 0x7fc8940787c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.978+0000 7fc8ad932640 1 --2- 172.21.15.45:0/277229352 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7fc8a81606d0 0x7fc8a80b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.978+0000 7fc8ad932640 1 --2- 172.21.15.45:0/277229352 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc8a80cdf00 0x7fc8a80cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.978+0000 7fc8ad932640 1 --2- 172.21.15.45:0/277229352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc8a8071a00 0x7fc8a81583a0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.978+0000 7fc8ad932640 1 -- 172.21.15.45:0/277229352 >> 172.21.15.45:0/277229352 conn(0x7fc8a80b49d0 msgr2=0x7fc8a815e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:13.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.978+0000 7fc8ad932640 1 -- 172.21.15.45:0/277229352 shutdown_connections 2024-05-08T05:52:13.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:13.978+0000 7fc8ad932640 1 -- 172.21.15.45:0/277229352 wait complete. 2024-05-08T05:52:13.989 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T05:52:13.989 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T05:52:14.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.092+0000 7f3d11189640 1 Processor -- start 2024-05-08T05:52:14.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.092+0000 7f3d11189640 1 -- start start 2024-05-08T05:52:14.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.093+0000 7f3d11189640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3d0c1629b0 0x7f3d0c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:14.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.093+0000 7f3d11189640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d0c0b2490 0x7f3d0c070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:14.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.093+0000 7f3d11189640 1 -- --> v1:172.21.15.63:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d0c06e110 con 0x7f3d0c070d20 2024-05-08T05:52:14.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.093+0000 7f3d11189640 1 -- --> v2:172.21.15.63:3300/0 -- mon_getmap magic: 0 -- 0x7f3d0c06dec0 con 0x7f3d0c1629b0 2024-05-08T05:52:14.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.093+0000 7f3d11189640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3d0c06e3f0 con 0x7f3d0c0b2490 2024-05-08T05:52:14.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.094+0000 7f3d0ad9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3d0c1629b0 0x7f3d0c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:14.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.094+0000 7f3d0a59c640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d0c0b2490 0x7f3d0c070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:14.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.094+0000 7f3d0ad9d640 1 --2- >> v2:172.21.15.63:3300/0 conn(0x7f3d0c1629b0 0x7f3d0c06eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.45:35062/0 (socket says 172.21.15.45:35062) 2024-05-08T05:52:14.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.094+0000 7f3d0ad9d640 1 -- 172.21.15.45:0/664642828 learned_addr learned my addr 172.21.15.45:0/664642828 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:52:14.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.094+0000 7f3d0ad9d640 1 -- 172.21.15.45:0/664642828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d0c0b2490 msgr2=0x7f3d0c070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:52:14.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.094+0000 7f3d0ad9d640 1 --2- 172.21.15.45:0/664642828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d0c0b2490 0x7f3d0c070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:14.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.094+0000 7f3d0ad9d640 1 -- 172.21.15.45:0/664642828 >> v1:172.21.15.63:6789/0 conn(0x7f3d0c070d20 legacy=0x7f3d0c0afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:14.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.094+0000 7f3d0ad9d640 1 -- 172.21.15.45:0/664642828 --> v2:172.21.15.63:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0c06e6d0 con 0x7f3d0c1629b0 2024-05-08T05:52:14.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.094+0000 7f3d0a59c640 1 --2- 172.21.15.45:0/664642828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d0c0b2490 0x7f3d0c070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:52:14.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.095+0000 7f3d0ad9d640 1 --2- 172.21.15.45:0/664642828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d0c1629b0 0x7f3d0c06eed0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009e50 tx=0x7f3cfc02a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34e47777b32daa3e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:14.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.095+0000 7f3d09d9b640 1 -- 172.21.15.45:0/664642828 <== mon.1 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc036070 con 0x7f3d0c1629b0 2024-05-08T05:52:14.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.095+0000 7f3d09d9b640 1 -- 172.21.15.45:0/664642828 <== mon.1 v2:172.21.15.63:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3cfc031600 con 0x7f3d0c1629b0 2024-05-08T05:52:14.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.095+0000 7f3d09d9b640 1 -- 172.21.15.45:0/664642828 <== mon.1 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc0319f0 con 0x7f3d0c1629b0 2024-05-08T05:52:14.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.095+0000 7f3d11189640 1 -- 172.21.15.45:0/664642828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d0c1629b0 msgr2=0x7f3d0c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:14.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.095+0000 7f3d11189640 1 --2- 172.21.15.45:0/664642828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d0c1629b0 0x7f3d0c06eed0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc009e50 tx=0x7f3cfc02a730 comp rx=0 tx=0).stop 2024-05-08T05:52:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.096+0000 7f3d11189640 1 -- 172.21.15.45:0/664642828 shutdown_connections 2024-05-08T05:52:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.096+0000 7f3d11189640 1 --2- 172.21.15.45:0/664642828 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d0c0b2490 0x7f3d0c070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.096+0000 7f3d11189640 1 --2- 172.21.15.45:0/664642828 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d0c1629b0 0x7f3d0c06eed0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.096+0000 7f3d11189640 1 -- 172.21.15.45:0/664642828 >> 172.21.15.45:0/664642828 conn(0x7f3d0c0b49d0 msgr2=0x7f3d0c0b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.096+0000 7f3d11189640 1 -- 172.21.15.45:0/664642828 shutdown_connections 2024-05-08T05:52:14.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.096+0000 7f3d11189640 1 -- 172.21.15.45:0/664642828 wait complete. 2024-05-08T05:52:14.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.097+0000 7f3d11189640 1 Processor -- start 2024-05-08T05:52:14.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.097+0000 7f3d11189640 1 -- start start 2024-05-08T05:52:14.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.098+0000 7f3d11189640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d0c070d20 0x7f3d0c1f7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:14.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.098+0000 7f3d11189640 1 --2- >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d0c0b2490 0x7f3d0c0b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:14.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.098+0000 7f3d11189640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d0c1629b0 0x7f3d0c0b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:14.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.098+0000 7f3d11189640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c070870 con 0x7f3d0c1629b0 2024-05-08T05:52:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.098+0000 7f3d11189640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c06f410 con 0x7f3d0c070d20 2024-05-08T05:52:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.098+0000 7f3d11189640 1 -- --> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] -- mon_getmap magic: 0 -- 0x7f3d0c0709f0 con 0x7f3d0c0b2490 2024-05-08T05:52:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.098+0000 7f3d0b59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d0c1629b0 0x7f3d0c0b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.098+0000 7f3d0b59e640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d0c1629b0 0x7f3d0c0b17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56316/0 (socket says 172.21.15.45:56316) 2024-05-08T05:52:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.098+0000 7f3d0b59e640 1 -- 172.21.15.45:0/2947826897 learned_addr learned my addr 172.21.15.45:0/2947826897 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-08T05:52:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.098+0000 7f3d0ad9d640 1 --2- 172.21.15.45:0/2947826897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d0c070d20 0x7f3d0c1f7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.098+0000 7f3d0a59c640 1 --2- 172.21.15.45:0/2947826897 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d0c0b2490 0x7f3d0c0b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.098+0000 7f3d0b59e640 1 -- 172.21.15.45:0/2947826897 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d0c0b2490 msgr2=0x7f3d0c0b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.098+0000 7f3d0b59e640 1 --2- 172.21.15.45:0/2947826897 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d0c0b2490 0x7f3d0c0b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.098+0000 7f3d0b59e640 1 -- 172.21.15.45:0/2947826897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d0c070d20 msgr2=0x7f3d0c1f7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.098+0000 7f3d0b59e640 1 --2- 172.21.15.45:0/2947826897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d0c070d20 0x7f3d0c1f7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.099+0000 7f3d0b59e640 1 -- 172.21.15.45:0/2947826897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0c0c7b50 con 0x7f3d0c1629b0 2024-05-08T05:52:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.099+0000 7f3d0b59e640 1 --2- 172.21.15.45:0/2947826897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d0c1629b0 0x7f3d0c0b17e0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400c400 tx=0x7f3d0400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:14.100 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.099+0000 7f3cf37fe640 1 -- 172.21.15.45:0/2947826897 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04017070 con 0x7f3d0c1629b0 2024-05-08T05:52:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.099+0000 7f3cf37fe640 1 -- 172.21.15.45:0/2947826897 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3d040052d0 con 0x7f3d0c1629b0 2024-05-08T05:52:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.099+0000 7f3d11189640 1 -- 172.21.15.45:0/2947826897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0c1f7f40 con 0x7f3d0c1629b0 2024-05-08T05:52:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.099+0000 7f3cf37fe640 1 -- 172.21.15.45:0/2947826897 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04003810 con 0x7f3d0c1629b0 2024-05-08T05:52:14.101 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.099+0000 7f3d11189640 1 -- 172.21.15.45:0/2947826897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d0c0c7df0 con 0x7f3d0c1629b0 2024-05-08T05:52:14.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.100+0000 7f3d11189640 1 -- 172.21.15.45:0/2947826897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d0c0693f0 con 0x7f3d0c1629b0 2024-05-08T05:52:14.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.101+0000 7f3cf37fe640 1 -- 172.21.15.45:0/2947826897 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3d0401a070 con 0x7f3d0c1629b0 2024-05-08T05:52:14.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.101+0000 7f3cf37fe640 1 --2- 172.21.15.45:0/2947826897 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3cf80762d0 0x7f3cf8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:14.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.101+0000 7f3cf37fe640 1 -- 172.21.15.45:0/2947826897 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5507+0+0 (secure 0 0 0) 0x7f3d040969c0 con 0x7f3d0c1629b0 2024-05-08T05:52:14.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.101+0000 7f3d0ad9d640 1 --2- 172.21.15.45:0/2947826897 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3cf80762d0 0x7f3cf8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:14.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.102+0000 7f3d0ad9d640 1 --2- 172.21.15.45:0/2947826897 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3cf80762d0 0x7f3cf8078790 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc033000 tx=0x7f3cfc009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:14.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.105+0000 7f3cf37fe640 1 -- 172.21.15.45:0/2947826897 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3d04098e20 con 0x7f3d0c1629b0 2024-05-08T05:52:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.333+0000 7f3d11189640 1 -- 172.21.15.45:0/2947826897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d0c076460 con 0x7f3d0c1629b0 2024-05-08T05:52:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.334+0000 7f3cf37fe640 1 -- 172.21.15.45:0/2947826897 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+3871 (secure 0 0 0) 0x7f3d04068350 con 0x7f3d0c1629b0 2024-05-08T05:52:14.334 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-08T05:52:14.334 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":115,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7168,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6836/3882445426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6834","nonce":3882445426},{"type":"v1","addr":"172.21.15.63:6836","nonce":3882445426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8094,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/3173207720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":3173207720},{"type":"v1","addr":"172.21.15.45:6833","nonce":3173207720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8106,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6837/4254619360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6835","nonce":4254619360},{"type":"v1","addr":"172.21.15.63:6837","nonce":4254619360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8631,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4235592862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4235592862},{"type":"v1","addr":"172.21.15.45:6835","nonce":4235592862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115}],"filesystems":[]} 2024-05-08T05:52:14.334 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 115 2024-05-08T05:52:14.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.335+0000 7f3d11189640 1 -- 172.21.15.45:0/2947826897 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3cf80762d0 msgr2=0x7f3cf8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:14.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.336+0000 7f3d11189640 1 --2- 172.21.15.45:0/2947826897 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3cf80762d0 0x7f3cf8078790 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc033000 tx=0x7f3cfc009000 comp rx=0 tx=0).stop 2024-05-08T05:52:14.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.336+0000 7f3d11189640 1 -- 172.21.15.45:0/2947826897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d0c1629b0 msgr2=0x7f3d0c0b17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:14.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.336+0000 7f3d11189640 1 --2- 172.21.15.45:0/2947826897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d0c1629b0 0x7f3d0c0b17e0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400c400 tx=0x7f3d0400c8c0 comp rx=0 tx=0).stop 2024-05-08T05:52:14.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.336+0000 7f3d11189640 1 -- 172.21.15.45:0/2947826897 shutdown_connections 2024-05-08T05:52:14.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.336+0000 7f3d11189640 1 --2- 172.21.15.45:0/2947826897 >> [v2:172.21.15.63:6832/2860796916,v1:172.21.15.63:6833/2860796916] conn(0x7f3cf80762d0 0x7f3cf8078790 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:14.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.336+0000 7f3d11189640 1 --2- 172.21.15.45:0/2947826897 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x7f3d0c0b2490 0x7f3d0c0b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:14.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.336+0000 7f3d11189640 1 --2- 172.21.15.45:0/2947826897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d0c1629b0 0x7f3d0c0b17e0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:14.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.336+0000 7f3d11189640 1 --2- 172.21.15.45:0/2947826897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3d0c070d20 0x7f3d0c1f7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.336+0000 7f3d11189640 1 -- 172.21.15.45:0/2947826897 >> 172.21.15.45:0/2947826897 conn(0x7f3d0c0b49d0 msgr2=0x7f3d0c160470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.336+0000 7f3d11189640 1 -- 172.21.15.45:0/2947826897 shutdown_connections 2024-05-08T05:52:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-08T05:52:14.336+0000 7f3d11189640 1 -- 172.21.15.45:0/2947826897 wait complete. 2024-05-08T05:52:14.348 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-08T05:52:14.348 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-08T05:52:14.349 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-08T05:52:14.349 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:52:14.357 INFO:tasks.ceph.mds.a:Stopped 2024-05-08T05:52:14.358 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-08T05:52:14.358 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:52:14.413 INFO:tasks.ceph.mds.c:Stopped 2024-05-08T05:52:14.434 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-08T05:52:14.434 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:52:14.447 INFO:tasks.ceph.mds.b:Stopped 2024-05-08T05:52:14.448 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-08T05:52:14.448 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:52:14.500 INFO:tasks.ceph.mds.d:Stopped 2024-05-08T05:52:14.501 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-08T05:52:14.501 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-08T05:52:14.501 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:52:14.532 INFO:tasks.ceph.osd.0:Stopped 2024-05-08T05:52:14.532 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-08T05:52:14.532 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:52:14.600 INFO:tasks.ceph.osd.1:Stopped 2024-05-08T05:52:14.601 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-08T05:52:14.601 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:52:14.676 INFO:tasks.ceph.osd.2:Stopped 2024-05-08T05:52:14.676 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-08T05:52:14.676 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:52:14.744 INFO:tasks.ceph.osd.3:Stopped 2024-05-08T05:52:14.745 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-08T05:52:14.745 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:52:14.784 INFO:tasks.ceph.osd.4:Stopped 2024-05-08T05:52:14.784 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-08T05:52:14.784 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:52:14.851 INFO:tasks.ceph.osd.5:Stopped 2024-05-08T05:52:14.851 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-08T05:52:14.851 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:52:14.919 INFO:tasks.ceph.osd.6:Stopped 2024-05-08T05:52:14.920 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-08T05:52:14.920 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:52:14.994 INFO:tasks.ceph.osd.7:Stopped 2024-05-08T05:52:14.994 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-08T05:52:14.994 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-08T05:52:14.995 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:52:15.021 INFO:tasks.ceph.mgr.y:Stopped 2024-05-08T05:52:15.021 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-08T05:52:15.021 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:52:15.064 INFO:tasks.ceph.mgr.x:Stopped 2024-05-08T05:52:15.064 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-08T05:52:15.065 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T05:52:15.065 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:52:15.077 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T05:52:15.077 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-08T05:52:15.077 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:52:15.118 INFO:tasks.ceph.mon.b:Stopped 2024-05-08T05:52:15.119 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T05:52:15.119 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:52:15.171 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T05:52:15.171 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-08T05:52:15.171 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-05-08T05:52:15.209 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:45:40.482071+0000 mds.a (mds.0) 1 : cluster [WRN] Corrupt dentry 'file_to_be_damaged' in dir frag 0x10000000000: Invalid tag char 'd' pos 0: Malformed input [buffer:3](/subdir) 2024-05-08T05:52:15.209 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-05-08T05:52:15.210 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-05-08T05:52:15.286 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-05-08T05:52:15.361 INFO:teuthology.orchestra.run.smithi045.stdout:2024-05-08T05:47:16.158078+0000 mds.b (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-05-08T05:52:15.361 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:52:15.361 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-08T05:52:15.573 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:52:15.573 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-08T05:52:15.655 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:52:15.655 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-08T05:52:15.704 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:52:15.704 DEBUG:teuthology.orchestra.run.smithi045:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-08T05:52:15.754 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-08T05:52:15.755 DEBUG:teuthology.orchestra.run.smithi063:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-08T05:52:15.947 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-08T05:52:15.947 DEBUG:teuthology.orchestra.run.smithi063:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-08T05:52:16.053 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-08T05:52:16.053 DEBUG:teuthology.orchestra.run.smithi063:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-08T05:52:16.112 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-08T05:52:16.112 DEBUG:teuthology.orchestra.run.smithi063:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-08T05:52:16.168 INFO:tasks.ceph:Archiving mon data... 2024-05-08T05:52:16.169 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/jcollin-2024-05-08_05:22:48-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697310/data/mon.a.tgz 2024-05-08T05:52:16.169 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-05-08T05:52:16.186 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.S64Jvwb1tb 2024-05-08T05:52:16.187 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.S64Jvwb1tb 2024-05-08T05:52:16.333 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.S64Jvwb1tb 2024-05-08T05:52:16.418 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.S64Jvwb1tb is 389KB 2024-05-08T05:52:16.516 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.S64Jvwb1tb 2024-05-08T05:52:16.532 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/jcollin-2024-05-08_05:22:48-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697310/data/mon.b.tgz 2024-05-08T05:52:16.533 DEBUG:teuthology.orchestra.run.smithi063:> mktemp 2024-05-08T05:52:16.552 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/tmp.SGpsXUDAY0 2024-05-08T05:52:16.553 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.SGpsXUDAY0 2024-05-08T05:52:16.713 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0666 /tmp/tmp.SGpsXUDAY0 2024-05-08T05:52:16.800 DEBUG:teuthology.orchestra.remote:smithi063:/tmp/tmp.SGpsXUDAY0 is 390KB 2024-05-08T05:52:16.903 DEBUG:teuthology.orchestra.run.smithi063:> rm -fr /tmp/tmp.SGpsXUDAY0 2024-05-08T05:52:16.919 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/jcollin-2024-05-08_05:22:48-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697310/data/mon.c.tgz 2024-05-08T05:52:16.919 DEBUG:teuthology.orchestra.run.smithi063:> mktemp 2024-05-08T05:52:16.975 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/tmp.tnFSJLo2Gi 2024-05-08T05:52:16.975 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.tnFSJLo2Gi 2024-05-08T05:52:17.116 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0666 /tmp/tmp.tnFSJLo2Gi 2024-05-08T05:52:17.201 DEBUG:teuthology.orchestra.remote:smithi063:/tmp/tmp.tnFSJLo2Gi is 390KB 2024-05-08T05:52:17.298 DEBUG:teuthology.orchestra.run.smithi063:> rm -fr /tmp/tmp.tnFSJLo2Gi 2024-05-08T05:52:17.314 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-08T05:52:17.314 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T05:52:17.317 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T05:52:17.401 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T05:52:17.437 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T05:52:17.437 INFO:tasks.ceph:Archiving crash dumps... 2024-05-08T05:52:17.439 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/crash to /home/teuthworker/archive/jcollin-2024-05-08_05:22:48-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697310/remote/smithi045/crash 2024-05-08T05:52:17.440 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T05:52:17.469 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/lib/ceph/crash to /home/teuthworker/archive/jcollin-2024-05-08_05:22:48-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697310/remote/smithi063/crash 2024-05-08T05:52:17.470 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T05:52:17.503 INFO:tasks.ceph:Shutting down logrotate 2024-05-08T05:52:17.503 DEBUG:teuthology.orchestra.run.smithi045:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:52:17.514 DEBUG:teuthology.orchestra.run.smithi063:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-08T05:52:17.615 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-08T05:52:17.643 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-08T05:52:17.668 INFO:tasks.ceph:Compressing logs... 2024-05-08T05:52:17.669 DEBUG:teuthology.orchestra.run.smithi045:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T05:52:17.686 DEBUG:teuthology.orchestra.run.smithi063:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T05:52:17.757 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35556.log 2024-05-08T05:52:17.757 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-08T05:52:17.757 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.35556.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-08T05:52:17.757 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35556.log.gz 2024-05-08T05:52:17.757 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-08T05:52:17.757 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-08T05:52:17.757 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-08T05:52:17.757 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-08T05:52:17.758 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35476.log 2024-05-08T05:52:17.758 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42130.log 2024-05-08T05:52:17.758 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.35476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42528.log 2024-05-08T05:52:17.758 INFO:teuthology.orchestra.run.smithi063.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35476.log.gz 2024-05-08T05:52:17.758 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.1.42130.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42130.log.gz 2024-05-08T05:52:17.758 INFO:teuthology.orchestra.run.smithi063.stderr:gzip 92.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-08T05:52:17.758 INFO:teuthology.orchestra.run.smithi063.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35516.log 2024-05-08T05:52:17.758 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.3.42528.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42528.log.gz 2024-05-08T05:52:17.758 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35727.log 2024-05-08T05:52:17.758 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.35516.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-08T05:52:17.758 INFO:teuthology.orchestra.run.smithi063.stderr: 87.9%/var/log/ceph/ceph-client.admin.35727.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-08T05:52:17.758 INFO:teuthology.orchestra.run.smithi063.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35516.log.gz 2024-05-08T05:52:17.758 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-08T05:52:17.758 INFO:teuthology.orchestra.run.smithi063.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.35727.log.gz 2024-05-08T05:52:17.758 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35600.log 2024-05-08T05:52:17.759 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-08T05:52:17.759 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.35600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35687.log 88.1% 2024-05-08T05:52:17.759 INFO:teuthology.orchestra.run.smithi063.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35600.log.gz 2024-05-08T05:52:17.764 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35767.log 2024-05-08T05:52:17.764 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.35687.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35687.log.gz 2024-05-08T05:52:17.780 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35647.log 2024-05-08T05:52:17.780 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.35767.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35767.log.gz 2024-05-08T05:52:17.780 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-08T05:52:17.780 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-client.admin.35647.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35647.log.gz 2024-05-08T05:52:17.780 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-08T05:52:17.860 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: 92.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-08T05:52:17.867 INFO:teuthology.orchestra.run.smithi063.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-08T05:52:17.893 INFO:teuthology.orchestra.run.smithi063.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-08T05:52:17.922 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66677.log 2024-05-08T05:52:17.922 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71641.log 2024-05-08T05:52:17.922 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49644.log 2024-05-08T05:52:17.922 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.71641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59113.log 2024-05-08T05:52:17.922 INFO:teuthology.orchestra.run.smithi045.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.66677.log.gz 2024-05-08T05:52:17.922 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68601.log 2024-05-08T05:52:17.922 INFO:teuthology.orchestra.run.smithi045.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.71641.log.gz 2024-05-08T05:52:17.922 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49644.log.gz 2024-05-08T05:52:17.922 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42881.log 2024-05-08T05:52:17.922 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59113.log.gz 2024-05-08T05:52:17.922 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.68601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69096.log 2024-05-08T05:52:17.923 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42881.log: gzip -5 88.8% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.68601.log.gz 2024-05-08T05:52:17.923 INFO:teuthology.orchestra.run.smithi045.stderr: /var/log/ceph/ceph-client.admin.62289.log 2024-05-08T05:52:17.923 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69096.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42881.log.gz 2024-05-08T05:52:17.923 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47207.log 2024-05-08T05:52:17.923 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69096.log.gz 2024-05-08T05:52:17.923 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50500.log 2024-05-08T05:52:17.923 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62289.log.gz 2024-05-08T05:52:17.923 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43227.log 2024-05-08T05:52:17.923 INFO:teuthology.orchestra.run.smithi045.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47207.log.gz 2024-05-08T05:52:17.923 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54776.log 2024-05-08T05:52:17.923 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43227.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50500.log.gz 2024-05-08T05:52:17.923 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71496.log 2024-05-08T05:52:17.923 INFO:teuthology.orchestra.run.smithi045.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43227.log.gz 2024-05-08T05:52:17.923 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65229.log 2024-05-08T05:52:17.924 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54776.log.gz 2024-05-08T05:52:17.924 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.71496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59837.log 2024-05-08T05:52:17.924 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.65229.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.71496.log.gz 2024-05-08T05:52:17.924 INFO:teuthology.orchestra.run.smithi045.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54692.log 2024-05-08T05:52:17.924 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62510.log 2024-05-08T05:52:17.924 INFO:teuthology.orchestra.run.smithi045.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.65229.log.gz 2024-05-08T05:52:17.924 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54692.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59837.log.gz 2024-05-08T05:52:17.924 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45992.log 2024-05-08T05:52:17.924 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54692.log.gz 2024-05-08T05:52:17.924 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63234.log 2024-05-08T05:52:17.924 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45992.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62510.log.gz 2024-05-08T05:52:17.924 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68216.log 2024-05-08T05:52:17.924 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45992.log.gz 2024-05-08T05:52:17.924 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.63234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63395.log 2024-05-08T05:52:17.924 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.68216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50416.log 2024-05-08T05:52:17.925 INFO:teuthology.orchestra.run.smithi045.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.63234.log.gz 2024-05-08T05:52:17.925 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.63395.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68216.log.gz 2024-05-08T05:52:17.925 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52834.log 2024-05-08T05:52:17.925 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63395.log.gz 2024-05-08T05:52:17.925 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63126.log 2024-05-08T05:52:17.925 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50416.log.gz 2024-05-08T05:52:17.925 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46537.log 2024-05-08T05:52:17.925 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52834.log.gz 2024-05-08T05:52:17.925 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55047.log 2024-05-08T05:52:17.925 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.63126.log: /var/log/ceph/ceph-client.admin.46537.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46537.log.gz 2024-05-08T05:52:17.925 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68928.log 2024-05-08T05:52:17.925 INFO:teuthology.orchestra.run.smithi045.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.63126.log.gz 2024-05-08T05:52:17.925 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61351.log 2024-05-08T05:52:17.925 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55047.log.gz 2024-05-08T05:52:17.925 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.68928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46411.log 2024-05-08T05:52:17.925 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68928.log.gz 2024-05-08T05:52:17.926 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67627.log 2024-05-08T05:52:17.926 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.61351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61351.log.gz 2024-05-08T05:52:17.926 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60862.log 2024-05-08T05:52:17.926 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67627.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46411.log.gzgzip 2024-05-08T05:52:17.926 INFO:teuthology.orchestra.run.smithi045.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.70634.log 2024-05-08T05:52:17.926 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67627.log.gz 2024-05-08T05:52:17.926 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63573.log 2024-05-08T05:52:17.926 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60862.log: /var/log/ceph/ceph-client.admin.70634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71618.log 2024-05-08T05:52:17.926 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.63573.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70634.log.gz 2024-05-08T05:52:17.926 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71422.log 2024-05-08T05:52:17.926 INFO:teuthology.orchestra.run.smithi045.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.60862.log.gz 2024-05-08T05:52:17.926 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63573.log.gz 2024-05-08T05:52:17.926 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.71618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64935.log 2024-05-08T05:52:17.926 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71618.log.gz 2024-05-08T05:52:17.927 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.71422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55770.log 2024-05-08T05:52:17.927 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.64935.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.71422.log.gz 2024-05-08T05:52:17.927 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59795.log 2024-05-08T05:52:17.927 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55770.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64935.log.gz 2024-05-08T05:52:17.927 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41688.log 2024-05-08T05:52:17.927 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55770.log.gz 2024-05-08T05:52:17.927 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48631.log 2024-05-08T05:52:17.927 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59795.log.gz 2024-05-08T05:52:17.927 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67838.log 2024-05-08T05:52:17.927 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41688.log: /var/log/ceph/ceph-client.admin.48631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52270.log 2024-05-08T05:52:17.927 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41688.log.gz 2024-05-08T05:52:17.927 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73720.log 2024-05-08T05:52:17.927 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48631.log.gz 2024-05-08T05:52:17.927 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67838.log.gz 2024-05-08T05:52:17.928 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74408.log 2024-05-08T05:52:17.928 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52270.log.gz 2024-05-08T05:52:17.928 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70139.log 2024-05-08T05:52:17.928 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.74408.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73720.log.gz 2024-05-08T05:52:17.928 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49549.log 2024-05-08T05:52:17.928 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74408.log.gz 2024-05-08T05:52:17.928 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.70139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67066.log 2024-05-08T05:52:17.928 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70139.log.gz 2024-05-08T05:52:17.928 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53090.log 2024-05-08T05:52:17.928 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43567.log 2024-05-08T05:52:17.928 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49549.log.gz 2024-05-08T05:52:17.928 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67066.log.gz 2024-05-08T05:52:17.928 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74266.log 2024-05-08T05:52:17.928 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47610.log 2024-05-08T05:52:17.929 INFO:teuthology.orchestra.run.smithi045.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53090.log.gz 2024-05-08T05:52:17.929 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43567.log.gz 2024-05-08T05:52:17.929 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.74266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44183.log 2024-05-08T05:52:17.929 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9%/var/log/ceph/ceph-client.admin.47610.log: -- replaced with /var/log/ceph/ceph-client.admin.74266.log.gz 2024-05-08T05:52:17.929 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64251.log 2024-05-08T05:52:17.929 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44183.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47610.log.gz 2024-05-08T05:52:17.929 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54734.log 2024-05-08T05:52:17.929 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.64251.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44183.log.gz 2024-05-08T05:52:17.929 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61597.log 2024-05-08T05:52:17.929 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54734.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64251.log.gz 2024-05-08T05:52:17.929 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55432.log 2024-05-08T05:52:17.929 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54734.log.gz 2024-05-08T05:52:17.929 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67880.log 2024-05-08T05:52:17.930 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52611.log 2024-05-08T05:52:17.930 INFO:teuthology.orchestra.run.smithi045.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.61597.log.gz 2024-05-08T05:52:17.930 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55432.log.gz 2024-05-08T05:52:17.930 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44557.log 2024-05-08T05:52:17.930 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67880.log.gz 2024-05-08T05:52:17.930 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51640.log 2024-05-08T05:52:17.930 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62894.log 2024-05-08T05:52:17.930 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52611.log.gz 2024-05-08T05:52:17.930 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44557.log.gz 2024-05-08T05:52:17.930 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51808.log 2024-05-08T05:52:17.930 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51640.log.gz 2024-05-08T05:52:17.930 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.62894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48570.log 2024-05-08T05:52:17.930 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62894.log.gz 2024-05-08T05:52:17.930 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45011.log 2024-05-08T05:52:17.931 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51808.log.gz 2024-05-08T05:52:17.931 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50692.log 2024-05-08T05:52:17.931 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48570.log.gz 2024-05-08T05:52:17.931 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52018.log 2024-05-08T05:52:17.931 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45011.log.gz 2024-05-08T05:52:17.931 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72784.log 2024-05-08T05:52:17.931 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50692.log.gz 2024-05-08T05:52:17.931 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68696.log 2024-05-08T05:52:17.931 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72784.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52018.log.gz 2024-05-08T05:52:17.931 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56022.log 2024-05-08T05:52:17.931 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72784.log.gz 2024-05-08T05:52:17.931 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.68696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64542.log 2024-05-08T05:52:17.931 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68696.log.gz 2024-05-08T05:52:17.931 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55215.log 2024-05-08T05:52:17.931 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56022.log.gz/var/log/ceph/ceph-client.admin.64542.log: 2024-05-08T05:52:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68006.log 2024-05-08T05:52:17.932 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64542.log.gz 2024-05-08T05:52:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55215.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-08T05:52:17.932 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55215.log.gz 2024-05-08T05:52:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.68006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53446.log 2024-05-08T05:52:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68006.log.gz 2024-05-08T05:52:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57473.log 2024-05-08T05:52:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.53446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46327.log 2024-05-08T05:52:17.932 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53446.log.gz 2024-05-08T05:52:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52060.log 2024-05-08T05:52:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55642.log 2024-05-08T05:52:17.932 INFO:teuthology.orchestra.run.smithi045.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.57473.log.gz 2024-05-08T05:52:17.932 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46327.log.gz 2024-05-08T05:52:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47119.log 2024-05-08T05:52:17.932 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43222.log 2024-05-08T05:52:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55642.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52060.log.gz 2024-05-08T05:52:17.933 INFO:teuthology.orchestra.run.smithi045.stderr: 86.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-08T05:52:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60802.log 2024-05-08T05:52:17.933 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55642.log.gz 2024-05-08T05:52:17.933 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47119.log.gz 2024-05-08T05:52:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43566.log 2024-05-08T05:52:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60802.log: gzip 87.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.43222.log.gz 2024-05-08T05:52:17.933 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.73181.log 2024-05-08T05:52:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43566.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.60802.log.gz -5 2024-05-08T05:52:17.933 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54419.log 2024-05-08T05:52:17.933 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43566.log.gz 2024-05-08T05:52:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47484.log 2024-05-08T05:52:17.933 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73181.log.gz 2024-05-08T05:52:17.933 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50880.log 2024-05-08T05:52:17.933 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54419.log.gz 2024-05-08T05:52:17.934 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65295.log 2024-05-08T05:52:17.934 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.50880.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47484.log.gz 2024-05-08T05:52:17.934 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50062.log 2024-05-08T05:52:17.934 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50880.log.gz 2024-05-08T05:52:17.934 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.65295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62169.log 2024-05-08T05:52:17.934 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50062.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65295.log.gz 2024-05-08T05:52:17.934 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71962.log 2024-05-08T05:52:17.934 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62169.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50062.log.gz 2024-05-08T05:52:17.934 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42755.log 2024-05-08T05:52:17.934 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62169.log.gz 2024-05-08T05:52:17.934 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.71962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41772.log 2024-05-08T05:52:17.934 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71962.log.gz 2024-05-08T05:52:17.934 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71908.log 2024-05-08T05:52:17.934 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42755.log.gz 2024-05-08T05:52:17.935 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46495.log 2024-05-08T05:52:17.935 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41772.log.gz 2024-05-08T05:52:17.935 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.71908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41286.log 2024-05-08T05:52:17.935 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46495.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71908.log.gz 2024-05-08T05:52:17.935 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45659.log 2024-05-08T05:52:17.935 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46495.log.gz 2024-05-08T05:52:17.935 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41374.log 2024-05-08T05:52:17.935 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41286.log.gz 2024-05-08T05:52:17.935 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69885.log 2024-05-08T05:52:17.935 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41374.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45659.log.gz 2024-05-08T05:52:17.935 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50230.log 2024-05-08T05:52:17.935 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41374.log.gz 2024-05-08T05:52:17.935 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43562.log 2024-05-08T05:52:17.935 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47736.log 2024-05-08T05:52:17.935 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50230.log.gz 2024-05-08T05:52:17.936 INFO:teuthology.orchestra.run.smithi045.stderr: 91.0%/var/log/ceph/ceph-client.admin.43562.log: -- replaced with /var/log/ceph/ceph-client.admin.69885.log.gz 2024-05-08T05:52:17.936 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42629.log 2024-05-08T05:52:17.936 INFO:teuthology.orchestra.run.smithi045.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43562.log.gz 2024-05-08T05:52:17.936 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47736.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-08T05:52:17.936 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55896.log 2024-05-08T05:52:17.936 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47736.log.gz 2024-05-08T05:52:17.936 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.2.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42629.log.gz 2024-05-08T05:52:17.936 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58884.log 2024-05-08T05:52:17.936 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62312.log 2024-05-08T05:52:17.936 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.58884.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55896.log.gz 2024-05-08T05:52:17.936 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51398.log 2024-05-08T05:52:17.936 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58884.log.gz 2024-05-08T05:52:17.936 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.62312.log: -- /var/log/ceph/ceph-client.admin.73054.log 2024-05-08T05:52:17.936 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47652.log 2024-05-08T05:52:17.937 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73054.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51398.log.gz 2024-05-08T05:52:17.937 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64497.log 2024-05-08T05:52:17.937 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73054.log.gz 2024-05-08T05:52:17.937 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67280.log 2024-05-08T05:52:17.937 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47652.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.62312.log.gz 2024-05-08T05:52:17.937 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.64497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61097.log 2024-05-08T05:52:17.937 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47652.log.gz 2024-05-08T05:52:17.937 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64497.log.gz 2024-05-08T05:52:17.937 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52228.log 2024-05-08T05:52:17.937 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52750.log 2024-05-08T05:52:17.937 INFO:teuthology.orchestra.run.smithi045.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.67280.log.gz 2024-05-08T05:52:17.937 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41880.log 2024-05-08T05:52:17.937 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52228.log.gz 2024-05-08T05:52:17.937 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47778.log 2024-05-08T05:52:17.937 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52750.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.61097.log.gz 2024-05-08T05:52:17.937 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60487.log 2024-05-08T05:52:17.938 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52750.log.gz 2024-05-08T05:52:17.938 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41880.log.gz 2024-05-08T05:52:17.938 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74224.log 2024-05-08T05:52:17.938 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47778.log.gz 2024-05-08T05:52:17.938 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.60487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74182.log 2024-05-08T05:52:17.938 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60487.log.gz 2024-05-08T05:52:17.938 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.74224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63970.log 2024-05-08T05:52:17.938 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74224.log.gz 2024-05-08T05:52:17.938 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.74182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49370.log 2024-05-08T05:52:17.938 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.63970.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74182.log.gz 2024-05-08T05:52:17.938 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60683.log 2024-05-08T05:52:17.938 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49370.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63970.log.gz 2024-05-08T05:52:17.938 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57797.log 2024-05-08T05:52:17.938 INFO:teuthology.orchestra.run.smithi045.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49370.log.gz 2024-05-08T05:52:17.938 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64637.log 2024-05-08T05:52:17.939 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57797.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60683.log.gz 2024-05-08T05:52:17.939 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55514.log 2024-05-08T05:52:17.939 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57797.log.gz 2024-05-08T05:52:17.939 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.64637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65019.log 2024-05-08T05:52:17.939 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64637.log.gz 2024-05-08T05:52:17.939 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53048.log 2024-05-08T05:52:17.939 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.65019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51976.log 2024-05-08T05:52:17.939 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55514.log.gz 2024-05-08T05:52:17.939 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53048.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65019.log.gz 2024-05-08T05:52:17.939 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57046.log 2024-05-08T05:52:17.939 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53048.log.gz 2024-05-08T05:52:17.939 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47163.log 2024-05-08T05:52:17.939 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57046.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51976.log.gz 2024-05-08T05:52:17.939 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61055.log 2024-05-08T05:52:17.939 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57046.log.gz 2024-05-08T05:52:17.939 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42923.log 2024-05-08T05:52:17.939 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47163.log.gz 2024-05-08T05:52:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57539.log 2024-05-08T05:52:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42923.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61055.log.gz 2024-05-08T05:52:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73456.log 2024-05-08T05:52:17.940 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42923.log.gz 2024-05-08T05:52:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57539.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-08T05:52:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43231.log 2024-05-08T05:52:17.940 INFO:teuthology.orchestra.run.smithi045.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.57539.log.gz 2024-05-08T05:52:17.940 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73456.log.gz 2024-05-08T05:52:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51892.log 2024-05-08T05:52:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71018.log 2024-05-08T05:52:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43231.log: /var/log/ceph/ceph-client.admin.51892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56581.log 2024-05-08T05:52:17.940 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43231.log.gz 2024-05-08T05:52:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.71018.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51892.log.gz 2024-05-08T05:52:17.940 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.71018.log.gz 2024-05-08T05:52:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41244.log 2024-05-08T05:52:17.940 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62071.log 2024-05-08T05:52:17.941 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56581.log.gz/var/log/ceph/ceph-client.admin.41244.log: 2024-05-08T05:52:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41520.log 2024-05-08T05:52:17.941 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41244.log.gz 2024-05-08T05:52:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74326.log 2024-05-08T05:52:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62071.log: /var/log/ceph/ceph-client.admin.41520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67585.log 2024-05-08T05:52:17.941 INFO:teuthology.orchestra.run.smithi045.stderr: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-08T05:52:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.74326.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41520.log.gz 2024-05-08T05:52:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58206.log 2024-05-08T05:52:17.941 INFO:teuthology.orchestra.run.smithi045.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.62071.log.gz 2024-05-08T05:52:17.941 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74326.log.gz 2024-05-08T05:52:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73498.log 2024-05-08T05:52:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.58206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55089.log 2024-05-08T05:52:17.941 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67585.log.gz 2024-05-08T05:52:17.941 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73498.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58206.log.gz 2024-05-08T05:52:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60825.log 2024-05-08T05:52:17.942 INFO:teuthology.orchestra.run.smithi045.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73498.log.gz 2024-05-08T05:52:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45315.log 2024-05-08T05:52:17.942 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55089.log.gz 2024-05-08T05:52:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67670.log 2024-05-08T05:52:17.942 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60825.log.gz/var/log/ceph/ceph-client.2.45315.log: 2024-05-08T05:52:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43230.log 2024-05-08T05:52:17.942 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45315.log.gz 2024-05-08T05:52:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70317.log 2024-05-08T05:52:17.942 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67670.log.gz 2024-05-08T05:52:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55812.log 2024-05-08T05:52:17.942 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43230.log.gz 2024-05-08T05:52:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53132.log 2024-05-08T05:52:17.942 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55812.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70317.log.gz 2024-05-08T05:52:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71552.log 2024-05-08T05:52:17.943 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55812.log.gz 2024-05-08T05:52:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65187.log 2024-05-08T05:52:17.943 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53132.log.gz 2024-05-08T05:52:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.71552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63489.log 2024-05-08T05:52:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.65187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56892.log 2024-05-08T05:52:17.943 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65187.log.gz 2024-05-08T05:52:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.63489.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.71552.log.gz 2024-05-08T05:52:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59623.log 2024-05-08T05:52:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63699.log 2024-05-08T05:52:17.943 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63489.log.gz 2024-05-08T05:52:17.943 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56892.log: /var/log/ceph/ceph-client.admin.59623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59419.log 2024-05-08T05:52:17.943 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56892.log.gz 2024-05-08T05:52:17.943 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59623.log.gz 2024-05-08T05:52:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.63699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49507.log 2024-05-08T05:52:17.944 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63699.log.gz 2024-05-08T05:52:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35373.log 2024-05-08T05:52:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49507.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59419.log.gz 2024-05-08T05:52:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43224.log 2024-05-08T05:52:17.944 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49507.log.gz 2024-05-08T05:52:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.35373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66389.log 2024-05-08T05:52:17.944 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35373.log.gz 2024-05-08T05:52:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46082.log 2024-05-08T05:52:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66389.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43224.log.gz 2024-05-08T05:52:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44267.log 2024-05-08T05:52:17.944 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66389.log.gz 2024-05-08T05:52:17.944 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73888.log 2024-05-08T05:52:17.944 INFO:teuthology.orchestra.run.smithi045.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46082.log.gz 2024-05-08T05:52:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52922.log 2024-05-08T05:52:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73888.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44267.log.gz 2024-05-08T05:52:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54818.log 2024-05-08T05:52:17.945 INFO:teuthology.orchestra.run.smithi045.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73888.log.gz 2024-05-08T05:52:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49889.log 2024-05-08T05:52:17.945 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52922.log.gz 2024-05-08T05:52:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54353.log 2024-05-08T05:52:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49889.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54818.log.gz 2024-05-08T05:52:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56723.log 2024-05-08T05:52:17.945 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49889.log.gz 2024-05-08T05:52:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59879.log 2024-05-08T05:52:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74056.log 2024-05-08T05:52:17.945 INFO:teuthology.orchestra.run.smithi045.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54353.log.gz 2024-05-08T05:52:17.945 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59879.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.56723.log.gz 2024-05-08T05:52:17.946 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49327.log 2024-05-08T05:52:17.946 INFO:teuthology.orchestra.run.smithi045.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.59879.log.gz 2024-05-08T05:52:17.946 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.74056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51850.log 2024-05-08T05:52:17.946 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74056.log.gz 2024-05-08T05:52:17.946 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52186.log 2024-05-08T05:52:17.946 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67020.log 2024-05-08T05:52:17.946 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49327.log.gz 2024-05-08T05:52:17.946 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51850.log.gz 2024-05-08T05:52:17.946 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42839.log 2024-05-08T05:52:17.946 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52186.log.gz 2024-05-08T05:52:17.946 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54650.log 2024-05-08T05:52:17.946 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53986.log 2024-05-08T05:52:17.946 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67020.log.gz 2024-05-08T05:52:17.946 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54650.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42839.log.gz 2024-05-08T05:52:17.947 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73593.log 2024-05-08T05:52:17.947 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53986.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54650.log.gz 2024-05-08T05:52:17.947 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61447.log 2024-05-08T05:52:17.947 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71342.log 2024-05-08T05:52:17.947 INFO:teuthology.orchestra.run.smithi045.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53986.log.gz 2024-05-08T05:52:17.947 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61447.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73593.log.gz 2024-05-08T05:52:17.947 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65483.log 2024-05-08T05:52:17.947 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66033.log 2024-05-08T05:52:17.947 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.71342.log: /var/log/ceph/ceph-client.0.65483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49098.log 2024-05-08T05:52:17.947 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65483.log.gz 2024-05-08T05:52:17.947 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% 91.0% -- replaced with /var/log/ceph/ceph-client.admin.71342.log.gz 2024-05-08T05:52:17.947 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61447.log.gz 2024-05-08T05:52:17.947 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.66033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59239.log 2024-05-08T05:52:17.947 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66033.log.gz 2024-05-08T05:52:17.947 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61879.log 2024-05-08T05:52:17.948 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49098.log.gz 2024-05-08T05:52:17.948 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59239.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-08T05:52:17.948 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59239.log.gz 2024-05-08T05:52:17.948 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74366.log 2024-05-08T05:52:17.948 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41120.log 2024-05-08T05:52:17.948 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.74366.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.61879.log.gz 2024-05-08T05:52:17.948 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44099.log 2024-05-08T05:52:17.948 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64030.log 2024-05-08T05:52:17.948 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41120.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74366.log.gz 2024-05-08T05:52:17.948 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44099.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41120.log.gz 2024-05-08T05:52:17.948 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59581.log 2024-05-08T05:52:17.948 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44099.log.gz 2024-05-08T05:52:17.948 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62029.log 2024-05-08T05:52:17.948 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.64030.log: /var/log/ceph/ceph-client.admin.59581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60047.log 2024-05-08T05:52:17.949 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64030.log.gz 2024-05-08T05:52:17.949 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62029.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59581.log.gz 2024-05-08T05:52:17.949 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46159.log 2024-05-08T05:52:17.949 INFO:teuthology.orchestra.run.smithi045.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62029.log.gz 2024-05-08T05:52:17.949 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59029.log 2024-05-08T05:52:17.949 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60047.log.gz 2024-05-08T05:52:17.949 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47568.log 2024-05-08T05:52:17.949 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46159.log: /var/log/ceph/ceph-client.admin.59029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43007.log 2024-05-08T05:52:17.949 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46159.log.gz 2024-05-08T05:52:17.949 INFO:teuthology.orchestra.run.smithi045.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59029.log.gz 2024-05-08T05:52:17.949 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53192.log 2024-05-08T05:52:17.949 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47568.log.gz 2024-05-08T05:52:17.949 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45449.log 2024-05-08T05:52:17.949 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43007.log: /var/log/ceph/ceph-client.admin.53192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54963.log 2024-05-08T05:52:17.949 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43007.log.gz 2024-05-08T05:52:17.950 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68886.log 2024-05-08T05:52:17.950 INFO:teuthology.orchestra.run.smithi045.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53192.log.gz 2024-05-08T05:52:17.950 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59071.log 2024-05-08T05:52:17.950 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45449.log.gz 2024-05-08T05:52:17.950 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44309.log 2024-05-08T05:52:17.950 INFO:teuthology.orchestra.run.smithi045.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54963.log.gz 2024-05-08T05:52:17.950 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.68886.log: /var/log/ceph/ceph-client.admin.59071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51682.log 2024-05-08T05:52:17.950 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68886.log.gz 2024-05-08T05:52:17.950 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44309.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59071.log.gz 2024-05-08T05:52:17.950 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64413.log 2024-05-08T05:52:17.950 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44309.log.gz 2024-05-08T05:52:17.950 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73097.log 2024-05-08T05:52:17.950 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.64413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66502.log 2024-05-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51682.log.gz 2024-05-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73097.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64413.log.gz 2024-05-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54860.log 2024-05-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.73097.log.gz 2024-05-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50314.log 2024-05-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47820.log 2024-05-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54860.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.66502.log.gz 2024-05-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59963.log 2024-05-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54860.log.gz 2024-05-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50314.log.gz 2024-05-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43882.log 2024-05-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47820.log.gz 2024-05-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45743.log 2024-05-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59963.log: /var/log/ceph/ceph-client.admin.43882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72068.log 2024-05-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi045.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59963.log.gz 2024-05-08T05:52:17.952 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45743.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43882.log.gz 2024-05-08T05:52:17.952 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72868.log 2024-05-08T05:52:17.952 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45743.log.gz 2024-05-08T05:52:17.952 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72068.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-08T05:52:17.952 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59197.log 2024-05-08T05:52:17.952 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72068.log.gz 2024-05-08T05:52:17.952 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mds.c.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72868.log.gz 2024-05-08T05:52:17.952 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43560.log 2024-05-08T05:52:17.952 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52445.log 2024-05-08T05:52:17.952 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59197.log.gz 2024-05-08T05:52:17.952 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46285.log 2024-05-08T05:52:17.952 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52445.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43560.log.gz 2024-05-08T05:52:17.952 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62354.log 2024-05-08T05:52:17.952 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52445.log.gz 2024-05-08T05:52:17.952 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54920.log 2024-05-08T05:52:17.953 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46285.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46285.log.gz 2024-05-08T05:52:17.953 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59665.log 2024-05-08T05:52:17.953 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69560.log 2024-05-08T05:52:17.953 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54920.log: /var/log/ceph/ceph-client.admin.59665.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49200.log 2024-05-08T05:52:17.953 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54920.log.gz 2024-05-08T05:52:17.953 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.69560.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59665.log.gz 2024-05-08T05:52:17.953 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69054.log 2024-05-08T05:52:17.953 INFO:teuthology.orchestra.run.smithi045.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69560.log.gz 2024-05-08T05:52:17.953 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73678.log 2024-05-08T05:52:17.953 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49200.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.62354.log.gz 2024-05-08T05:52:17.953 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69054.log: gzip -5 --verbose -- 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49200.log.gz 2024-05-08T05:52:17.953 INFO:teuthology.orchestra.run.smithi045.stderr: /var/log/ceph/ceph-client.admin.68557.log 2024-05-08T05:52:17.953 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73678.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69054.log.gz 2024-05-08T05:52:17.953 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58581.log 2024-05-08T05:52:17.953 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73678.log.gz 2024-05-08T05:52:17.954 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48668.log 2024-05-08T05:52:17.954 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.68557.log: /var/log/ceph/ceph-client.admin.58581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67192.log 2024-05-08T05:52:17.954 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68557.log.gz 2024-05-08T05:52:17.954 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48668.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58581.log.gz 2024-05-08T05:52:17.954 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47526.log 2024-05-08T05:52:17.954 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49978.log 2024-05-08T05:52:17.954 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48668.log.gz 2024-05-08T05:52:17.954 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67192.log: /var/log/ceph/ceph-client.admin.47526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66978.log 2024-05-08T05:52:17.954 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49978.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47526.log.gz 2024-05-08T05:52:17.954 INFO:teuthology.orchestra.run.smithi045.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.67192.log.gz 2024-05-08T05:52:17.954 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61771.log 2024-05-08T05:52:17.954 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49978.log.gz 2024-05-08T05:52:17.955 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52569.log 2024-05-08T05:52:17.955 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66978.log: /var/log/ceph/ceph-client.admin.61771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54608.log 2024-05-08T05:52:17.955 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66978.log.gz 2024-05-08T05:52:17.955 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61771.log.gz 2024-05-08T05:52:17.955 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52569.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-08T05:52:17.955 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54608.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52569.log.gz 2024-05-08T05:52:17.955 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64977.log 2024-05-08T05:52:17.955 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54608.log.gz 2024-05-08T05:52:17.955 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-08T05:52:17.955 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47035.log 2024-05-08T05:52:17.955 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.64977.log: /var/log/ceph/ceph-mgr.y.log: gzip -5 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64977.log.gz 2024-05-08T05:52:17.955 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41458.log 2024-05-08T05:52:17.955 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49934.log 88.2% 2024-05-08T05:52:17.955 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47035.log.gz 2024-05-08T05:52:17.956 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42494.log 2024-05-08T05:52:17.956 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41458.log.gz 2024-05-08T05:52:17.956 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69138.log 2024-05-08T05:52:17.956 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49934.log.gz/var/log/ceph/ceph-client.admin.42494.log: 2024-05-08T05:52:17.956 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68473.log 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42494.log.gz 2024-05-08T05:52:17.956 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-05-08T05:52:17.956 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63039.log 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69138.log.gz 2024-05-08T05:52:17.956 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-05-08T05:52:17.956 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.68473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45575.log 2024-05-08T05:52:17.956 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.68473.log.gz 2024-05-08T05:52:17.956 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.63039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60005.log 2024-05-08T05:52:17.956 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.45575.log: /var/log/ceph/ceph-client.admin.56938.log 2024-05-08T05:52:17.956 INFO:teuthology.orchestra.run.smithi045.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.63039.log.gz 2024-05-08T05:52:17.956 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45575.log.gz 2024-05-08T05:52:17.956 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60005.log: gzip -5 --verbose 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60005.log.gz 2024-05-08T05:52:17.957 INFO:teuthology.orchestra.run.smithi045.stderr: -- /var/log/ceph/ceph-client.admin.48742.log 2024-05-08T05:52:17.957 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45533.log 2024-05-08T05:52:17.957 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56938.log.gz 2024-05-08T05:52:17.957 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48742.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-08T05:52:17.957 INFO:teuthology.orchestra.run.smithi045.stderr: 90.2%/var/log/ceph/ceph-client.admin.45533.log: -- replaced with /var/log/ceph/ceph-client.admin.48742.log.gz 2024-05-08T05:52:17.957 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51332.log 2024-05-08T05:52:17.957 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45533.log.gz 2024-05-08T05:52:17.957 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58926.log 2024-05-08T05:52:17.957 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65789.log 2024-05-08T05:52:17.957 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51332.log: /var/log/ceph/ceph-client.admin.58926.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58926.log.gz 2024-05-08T05:52:17.957 INFO:teuthology.orchestra.run.smithi045.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.51332.log.gz 2024-05-08T05:52:17.957 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44141.log 2024-05-08T05:52:17.957 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.65789.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.65789.log.gz 2024-05-08T05:52:17.958 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55352.log 2024-05-08T05:52:17.958 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44141.log: 94.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64809.log 2024-05-08T05:52:17.958 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-05-08T05:52:17.958 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44141.log.gz 2024-05-08T05:52:17.958 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55352.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55352.log.gz 2024-05-08T05:52:17.958 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56106.log 2024-05-08T05:52:17.958 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.64809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61513.log 2024-05-08T05:52:17.958 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6%/var/log/ceph/ceph-client.admin.56106.log: -- replaced with /var/log/ceph/ceph-client.admin.64809.log.gz 2024-05-08T05:52:17.958 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66503.log 2024-05-08T05:52:17.958 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56106.log.gz 2024-05-08T05:52:17.958 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69306.log 2024-05-08T05:52:17.958 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61513.log: /var/log/ceph/ceph-client.admin.66503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46782.log 2024-05-08T05:52:17.958 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69306.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66503.log.gz 2024-05-08T05:52:17.959 INFO:teuthology.orchestra.run.smithi045.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61513.log.gz 2024-05-08T05:52:17.959 INFO:teuthology.orchestra.run.smithi045.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.69306.log.gz 2024-05-08T05:52:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48972.log 2024-05-08T05:52:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:gzip/var/log/ceph/ceph-client.admin.46782.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.71791.log 2024-05-08T05:52:17.959 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1%/var/log/ceph/ceph-client.admin.48972.log: -- replaced with /var/log/ceph/ceph-client.admin.46782.log.gz 2024-05-08T05:52:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42797.log 2024-05-08T05:52:17.959 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48972.log.gz 2024-05-08T05:52:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71707.log 2024-05-08T05:52:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.71791.log: /var/log/ceph/ceph-client.admin.42797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56214.log 2024-05-08T05:52:17.959 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42797.log.gz 2024-05-08T05:52:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.71707.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.71791.log.gz 2024-05-08T05:52:17.959 INFO:teuthology.orchestra.run.smithi045.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.71707.log.gz 2024-05-08T05:52:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65766.log 2024-05-08T05:52:17.959 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53619.log 2024-05-08T05:52:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.65766.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56214.log.gz 2024-05-08T05:52:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46993.log 2024-05-08T05:52:17.960 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65766.log.gz 2024-05-08T05:52:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45617.log 2024-05-08T05:52:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53619.log: /var/log/ceph/ceph-client.admin.46993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53672.log 2024-05-08T05:52:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45617.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46993.log.gz 2024-05-08T05:52:17.960 INFO:teuthology.orchestra.run.smithi045.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.53619.log.gz 2024-05-08T05:52:17.960 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45617.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41604.log 2024-05-08T05:52:17.960 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-05-08T05:52:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52527.log 2024-05-08T05:52:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43876.log 2024-05-08T05:52:17.960 INFO:teuthology.orchestra.run.smithi045.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.53672.log.gz 2024-05-08T05:52:17.960 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41604.log: /var/log/ceph/ceph-client.admin.52527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68756.log 2024-05-08T05:52:17.960 INFO:teuthology.orchestra.run.smithi045.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41604.log.gz 2024-05-08T05:52:17.961 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52527.log.gz 2024-05-08T05:52:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70580.log 2024-05-08T05:52:17.961 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43876.log.gz 2024-05-08T05:52:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.68756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54514.log 2024-05-08T05:52:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64114.log 2024-05-08T05:52:17.961 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68756.log.gz 2024-05-08T05:52:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54514.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60149.log 2024-05-08T05:52:17.961 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70580.log.gz 2024-05-08T05:52:17.961 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54514.log.gz 2024-05-08T05:52:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73414.log 2024-05-08T05:52:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.64114.log: /var/log/ceph/ceph-client.admin.60149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68411.log 2024-05-08T05:52:17.961 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64114.log.gz 2024-05-08T05:52:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73414.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60149.log.gz 2024-05-08T05:52:17.961 INFO:teuthology.orchestra.run.smithi045.stderr:gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73414.log.gz 2024-05-08T05:52:17.962 INFO:teuthology.orchestra.run.smithi045.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59753.log 2024-05-08T05:52:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.68411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56402.log 2024-05-08T05:52:17.962 INFO:teuthology.orchestra.run.smithi045.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.68411.log.gz 2024-05-08T05:52:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35651.log 2024-05-08T05:52:17.962 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59753.log.gz 2024-05-08T05:52:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.56402.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56402.log.gz 2024-05-08T05:52:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51934.log 2024-05-08T05:52:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:gzip/var/log/ceph/ceph-client.admin.35651.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.73972.log 2024-05-08T05:52:17.962 INFO:teuthology.orchestra.run.smithi045.stderr: 86.9%/var/log/ceph/ceph-client.admin.51934.log: -- replaced with /var/log/ceph/ceph-client.admin.35651.log.gz 2024-05-08T05:52:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72364.log 2024-05-08T05:52:17.962 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51934.log.gz 2024-05-08T05:52:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68048.log/var/log/ceph/ceph-client.admin.73972.log: 2024-05-08T05:52:17.962 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.72364.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72364.log.gz 2024-05-08T05:52:17.963 INFO:teuthology.orchestra.run.smithi045.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73972.log.gz 2024-05-08T05:52:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56064.log 2024-05-08T05:52:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51546.log 2024-05-08T05:52:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.68048.log: gzip -5 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68048.log.gz 2024-05-08T05:52:17.963 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose --/var/log/ceph/ceph-client.admin.56064.log: /var/log/ceph/ceph-client.admin.49140.log 2024-05-08T05:52:17.963 INFO:teuthology.orchestra.run.smithi045.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56064.log.gz 2024-05-08T05:52:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52699.log 2024-05-08T05:52:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50584.log 2024-05-08T05:52:17.963 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51546.log.gz/var/log/ceph/ceph-client.admin.49140.log: 2024-05-08T05:52:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52699.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49140.log.gz 2024-05-08T05:52:17.963 INFO:teuthology.orchestra.run.smithi045.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52699.log.gz 2024-05-08T05:52:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64455.log 2024-05-08T05:52:17.963 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52964.log 2024-05-08T05:52:17.964 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50584.log.gz/var/log/ceph/ceph-client.admin.64455.log: 2024-05-08T05:52:17.964 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50356.log 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64455.log.gz 2024-05-08T05:52:17.964 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-05-08T05:52:17.964 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52964.log: gzip -5 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52964.log.gz 2024-05-08T05:52:17.964 INFO:teuthology.orchestra.run.smithi045.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55131.log 2024-05-08T05:52:17.964 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66924.log 2024-05-08T05:52:17.964 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50356.log.gz 2024-05-08T05:52:17.964 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46701.log 2024-05-08T05:52:17.964 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55131.log.gz 2024-05-08T05:52:17.964 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72110.log 2024-05-08T05:52:17.964 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66924.log.gz 2024-05-08T05:52:17.964 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61813.log 2024-05-08T05:52:17.964 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46701.log.gz 2024-05-08T05:52:17.964 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47694.log 2024-05-08T05:52:17.964 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63928.log 2024-05-08T05:52:17.965 INFO:teuthology.orchestra.run.smithi045.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.72110.log.gz 2024-05-08T05:52:17.965 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73846.log 2024-05-08T05:52:17.965 INFO:teuthology.orchestra.run.smithi045.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.61813.log.gz 2024-05-08T05:52:17.965 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47694.log.gz/var/log/ceph/ceph-client.admin.63928.log: 2024-05-08T05:52:17.965 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60299.log 2024-05-08T05:52:17.965 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43877.log 2024-05-08T05:52:17.965 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63928.log.gz 2024-05-08T05:52:17.965 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60299.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73846.log.gz 2024-05-08T05:52:17.965 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57689.log 2024-05-08T05:52:17.965 INFO:teuthology.orchestra.run.smithi045.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.60299.log.gz 2024-05-08T05:52:17.965 INFO:teuthology.orchestra.run.smithi045.stderr:gzip/var/log/ceph/ceph-client.admin.43877.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45701.log 2024-05-08T05:52:17.965 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57689.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43877.log.gz 2024-05-08T05:52:17.966 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67964.log 2024-05-08T05:52:17.966 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57689.log.gz 2024-05-08T05:52:17.966 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5/var/log/ceph/ceph-client.admin.45701.log: --verbose -- /var/log/ceph/ceph-client.admin.50020.log 2024-05-08T05:52:17.966 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67964.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45701.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-08T05:52:17.966 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-05-08T05:52:17.966 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67964.log.gz 2024-05-08T05:52:17.966 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63741.log 2024-05-08T05:52:17.966 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50020.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51161.log 2024-05-08T05:52:17.966 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50020.log.gz/var/log/ceph/ceph-client.admin.63741.log: 2024-05-08T05:52:17.966 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63741.log.gz 2024-05-08T05:52:17.966 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64072.log 2024-05-08T05:52:17.966 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61963.log 2024-05-08T05:52:17.966 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51161.log.gz 2024-05-08T05:52:17.966 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.64072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74098.log 2024-05-08T05:52:17.967 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61963.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64072.log.gz 2024-05-08T05:52:17.967 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53576.log 2024-05-08T05:52:17.967 INFO:teuthology.orchestra.run.smithi045.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61963.log.gz 2024-05-08T05:52:17.967 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.74098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57623.log 2024-05-08T05:52:17.967 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53576.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74098.log.gz 2024-05-08T05:52:17.967 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48508.log 2024-05-08T05:52:17.967 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53576.log.gz 2024-05-08T05:52:17.967 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58164.log 2024-05-08T05:52:17.967 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48508.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48508.log.gz 2024-05-08T05:52:17.967 INFO:teuthology.orchestra.run.smithi045.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57623.log.gz 2024-05-08T05:52:17.967 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67796.log 2024-05-08T05:52:17.967 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.58164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64767.log 2024-05-08T05:52:17.967 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58164.log.gz 2024-05-08T05:52:17.967 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41416.log 2024-05-08T05:52:17.968 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.64767.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67796.log.gz 2024-05-08T05:52:17.968 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64767.log.gz 2024-05-08T05:52:17.968 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72649.log 2024-05-08T05:52:17.968 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71459.log 2024-05-08T05:52:17.968 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41416.log.gz 2024-05-08T05:52:17.968 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70722.log 2024-05-08T05:52:17.968 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.71459.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72649.log.gz 2024-05-08T05:52:17.968 INFO:teuthology.orchestra.run.smithi045.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.71459.log.gz 2024-05-08T05:52:17.968 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59323.log 2024-05-08T05:52:17.968 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46824.log 2024-05-08T05:52:17.968 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.70722.log.gz 2024-05-08T05:52:17.968 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59323.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60929.log 2024-05-08T05:52:17.968 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59323.log.gz 2024-05-08T05:52:17.968 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46824.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46824.log.gz 2024-05-08T05:52:17.968 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65627.log 2024-05-08T05:52:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73540.log 2024-05-08T05:52:17.969 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60929.log.gz 2024-05-08T05:52:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.65627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59921.log 2024-05-08T05:52:17.969 INFO:teuthology.orchestra.run.smithi045.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65627.log.gz 2024-05-08T05:52:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73540.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73540.log.gz 2024-05-08T05:52:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63886.log 2024-05-08T05:52:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54052.log 2024-05-08T05:52:17.969 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59921.log.gz 2024-05-08T05:52:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.63886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43565.log 2024-05-08T05:52:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54052.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63886.log.gz 2024-05-08T05:52:17.969 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54052.log.gz 2024-05-08T05:52:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45491.log 2024-05-08T05:52:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50626.log 2024-05-08T05:52:17.969 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43565.log.gz 2024-05-08T05:52:17.969 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41814.log 2024-05-08T05:52:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50626.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45491.log.gz 2024-05-08T05:52:17.970 INFO:teuthology.orchestra.run.smithi045.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50626.log.gz 2024-05-08T05:52:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64351.log 2024-05-08T05:52:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41730.log 2024-05-08T05:52:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.64351.log: 91.1% 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64351.log.gz 2024-05-08T05:52:17.970 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41814.log.gz 2024-05-08T05:52:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49847.log 2024-05-08T05:52:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51010.log 2024-05-08T05:52:17.970 INFO:teuthology.orchestra.run.smithi045.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41730.log.gz 2024-05-08T05:52:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49805.log 2024-05-08T05:52:17.970 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49847.log.gz 2024-05-08T05:52:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51010.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51010.log.gz 2024-05-08T05:52:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57731.log 2024-05-08T05:52:17.970 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50272.log 2024-05-08T05:52:17.970 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49805.log.gz 2024-05-08T05:52:17.971 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53878.log 2024-05-08T05:52:17.971 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50272.log: 91.3% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50272.log.gz 2024-05-08T05:52:17.971 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57731.log.gz 2024-05-08T05:52:17.971 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66868.log 2024-05-08T05:52:17.971 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51724.log 2024-05-08T05:52:17.971 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66868.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53878.log.gz 2024-05-08T05:52:17.971 INFO:teuthology.orchestra.run.smithi045.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.66868.log.gz 2024-05-08T05:52:17.971 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47400.log 2024-05-08T05:52:17.971 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48888.log 2024-05-08T05:52:17.971 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51724.log.gz 2024-05-08T05:52:17.971 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66590.log 2024-05-08T05:52:17.971 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48888.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47400.log.gz 2024-05-08T05:52:17.971 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48888.log.gz 2024-05-08T05:52:17.971 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60191.log 2024-05-08T05:52:17.971 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62457.log 2024-05-08T05:52:17.971 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60191.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.66590.log.gz 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60191.log.gz 2024-05-08T05:52:17.972 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-05-08T05:52:17.972 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47258.log 2024-05-08T05:52:17.972 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72994.log 2024-05-08T05:52:17.972 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62457.log.gz 2024-05-08T05:52:17.972 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52655.log 2024-05-08T05:52:17.972 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72994.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47258.log.gz 2024-05-08T05:52:17.972 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72994.log.gzgzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29344.log 2024-05-08T05:52:17.972 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-05-08T05:52:17.972 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52655.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52655.log.gz 2024-05-08T05:52:17.972 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67108.log 2024-05-08T05:52:17.972 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52365.log 2024-05-08T05:52:17.972 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph.tmp-client.admin.29344.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29344.log.gz 2024-05-08T05:52:17.972 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62247.log 2024-05-08T05:52:17.972 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52365.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.67108.log.gz 2024-05-08T05:52:17.972 INFO:teuthology.orchestra.run.smithi045.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52365.log.gz 2024-05-08T05:52:17.973 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66826.log 2024-05-08T05:52:17.973 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46951.log 2024-05-08T05:52:17.973 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62247.log: /var/log/ceph/ceph-client.admin.66826.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62247.log.gz 2024-05-08T05:52:17.973 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69222.log 2024-05-08T05:52:17.973 INFO:teuthology.orchestra.run.smithi045.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.66826.log.gz 2024-05-08T05:52:17.973 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46951.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46951.log.gz 2024-05-08T05:52:17.973 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62706.log 2024-05-08T05:52:17.973 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53794.log 2024-05-08T05:52:17.973 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69222.log: /var/log/ceph/ceph-client.admin.62706.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69222.log.gz 2024-05-08T05:52:17.973 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62706.log.gz 2024-05-08T05:52:17.973 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66766.log 2024-05-08T05:52:17.973 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.53794.log: /var/log/ceph/ceph-client.0.58394.log 2024-05-08T05:52:17.973 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66766.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53794.log.gz 2024-05-08T05:52:17.973 INFO:teuthology.orchestra.run.smithi045.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.66766.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66743.log 2024-05-08T05:52:17.973 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-05-08T05:52:17.974 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.58394.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58394.log.gz 2024-05-08T05:52:17.974 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43880.log 2024-05-08T05:52:17.974 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63192.log 2024-05-08T05:52:17.974 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66743.log.gz 2024-05-08T05:52:17.974 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46866.log 2024-05-08T05:52:17.974 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.63192.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43880.log.gz 2024-05-08T05:52:17.974 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63192.log.gz 2024-05-08T05:52:17.974 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58842.log 2024-05-08T05:52:17.974 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62640.log 2024-05-08T05:52:17.974 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46866.log.gz 2024-05-08T05:52:17.974 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.58842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52144.log 2024-05-08T05:52:17.974 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62640.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58842.log.gz 2024-05-08T05:52:17.974 INFO:teuthology.orchestra.run.smithi045.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.62640.log.gz 2024-05-08T05:52:17.974 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70427.log 2024-05-08T05:52:17.975 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43547.log 2024-05-08T05:52:17.975 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52144.log.gz 2024-05-08T05:52:17.975 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50104.log 2024-05-08T05:52:17.975 INFO:teuthology.orchestra.run.smithi045.stderr: 87.3%/var/log/ceph/ceph-client.admin.43547.log: -- replaced with /var/log/ceph/ceph-client.admin.70427.log.gz 2024-05-08T05:52:17.975 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43547.log.gz 2024-05-08T05:52:17.975 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72176.log 2024-05-08T05:52:17.975 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51033.log 2024-05-08T05:52:17.975 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50104.log.gz 2024-05-08T05:52:17.975 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53728.log 2024-05-08T05:52:17.975 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51033.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72176.log.gz 2024-05-08T05:52:17.975 INFO:teuthology.orchestra.run.smithi045.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.51033.log.gz 2024-05-08T05:52:17.975 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65145.log 2024-05-08T05:52:17.975 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53944.log 2024-05-08T05:52:17.975 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.65145.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.53728.log.gz 2024-05-08T05:52:17.975 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65145.log.gz 2024-05-08T05:52:17.976 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41562.log 2024-05-08T05:52:17.976 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47301.log 2024-05-08T05:52:17.976 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53944.log.gz 2024-05-08T05:52:17.976 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51160.log 2024-05-08T05:52:17.976 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47301.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41562.log.gz 2024-05-08T05:52:17.976 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47301.log.gz 2024-05-08T05:52:17.976 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64851.log 2024-05-08T05:52:17.976 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45914.log 2024-05-08T05:52:17.976 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.64851.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.51160.log.gz 2024-05-08T05:52:17.976 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64851.log.gz 2024-05-08T05:52:17.976 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42713.log 2024-05-08T05:52:17.976 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.45914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64893.log 2024-05-08T05:52:17.976 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45914.log.gz 2024-05-08T05:52:17.976 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67193.log 2024-05-08T05:52:17.976 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.64893.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42713.log.gz 2024-05-08T05:52:17.977 INFO:teuthology.orchestra.run.smithi045.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64893.log.gz 2024-05-08T05:52:17.977 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47928.log 2024-05-08T05:52:17.977 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68844.log 2024-05-08T05:52:17.977 INFO:teuthology.orchestra.run.smithi045.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67193.log.gz 2024-05-08T05:52:17.977 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65103.log 2024-05-08T05:52:17.977 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.68844.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47928.log.gz 2024-05-08T05:52:17.977 INFO:teuthology.orchestra.run.smithi045.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68844.log.gz 2024-05-08T05:52:17.977 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52792.log 2024-05-08T05:52:17.977 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.65103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69180.log 2024-05-08T05:52:17.977 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65103.log.gz 2024-05-08T05:52:17.977 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52792.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61163.log 2024-05-08T05:52:17.977 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52792.log.gz 2024-05-08T05:52:17.977 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64156.log 2024-05-08T05:52:17.977 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69180.log: /var/log/ceph/ceph-client.admin.61163.log: 87.6% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69180.log.gz 2024-05-08T05:52:17.977 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61163.log.gz 2024-05-08T05:52:17.977 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51086.log 2024-05-08T05:52:17.978 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43229.log 2024-05-08T05:52:17.978 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.64156.log: /var/log/ceph/ceph-client.admin.51086.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64156.log.gz 2024-05-08T05:52:17.978 INFO:teuthology.orchestra.run.smithi045.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.51086.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56832.log 2024-05-08T05:52:17.978 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-05-08T05:52:17.978 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73930.log 2024-05-08T05:52:17.978 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56832.log: /var/log/ceph/ceph-client.admin.43229.log: 89.3% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43229.log.gz 2024-05-08T05:52:17.978 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56832.log.gz 2024-05-08T05:52:17.978 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64721.log 2024-05-08T05:52:17.978 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46909.log 2024-05-08T05:52:17.978 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73930.log: /var/log/ceph/ceph-client.admin.64721.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73930.log.gz 2024-05-08T05:52:17.978 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64721.log.gz 2024-05-08T05:52:17.978 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48116.log 2024-05-08T05:52:17.978 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61705.log 2024-05-08T05:52:17.978 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46909.log: /var/log/ceph/ceph-client.0.48116.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48116.log.gz 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46909.log.gz 2024-05-08T05:52:17.979 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-05-08T05:52:17.979 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73223.log 2024-05-08T05:52:17.979 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64586.log 2024-05-08T05:52:17.979 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61705.log: /var/log/ceph/ceph-client.admin.73223.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73223.log.gz 2024-05-08T05:52:17.979 INFO:teuthology.orchestra.run.smithi045.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.61705.log.gz 2024-05-08T05:52:17.979 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65845.log 2024-05-08T05:52:17.979 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.64586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58098.log 2024-05-08T05:52:17.979 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64586.log.gz 2024-05-08T05:52:17.979 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.65845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49014.log 2024-05-08T05:52:17.979 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.58098.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65845.log.gz 2024-05-08T05:52:17.979 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 91.1% --verbose -- /var/log/ceph/ceph-client.admin.48930.log 2024-05-08T05:52:17.979 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58098.log.gz 2024-05-08T05:52:17.979 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51766.log 2024-05-08T05:52:17.979 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49014.log: /var/log/ceph/ceph-client.admin.48930.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49014.log.gz 2024-05-08T05:52:17.980 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47862.log 2024-05-08T05:52:17.980 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48930.log.gz 2024-05-08T05:52:17.980 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51766.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51766.log.gz 2024-05-08T05:52:17.980 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49056.log 2024-05-08T05:52:17.980 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43563.log 2024-05-08T05:52:17.980 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47862.log: /var/log/ceph/ceph-client.admin.49056.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49056.log.gz 2024-05-08T05:52:17.980 INFO:teuthology.orchestra.run.smithi045.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47862.log.gz 2024-05-08T05:52:17.980 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70764.log 2024-05-08T05:52:17.980 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56148.log 2024-05-08T05:52:17.980 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43563.log: /var/log/ceph/ceph-client.admin.70764.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43563.log.gz 2024-05-08T05:52:17.980 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50188.log 2024-05-08T05:52:17.980 INFO:teuthology.orchestra.run.smithi045.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.70764.log.gz/var/log/ceph/ceph-client.admin.56148.log: 2024-05-08T05:52:17.980 INFO:teuthology.orchestra.run.smithi045.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.56148.log.gz 2024-05-08T05:52:17.980 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64679.log 2024-05-08T05:52:17.981 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61663.log 2024-05-08T05:52:17.981 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50188.log: /var/log/ceph/ceph-client.admin.64679.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50188.log.gz 2024-05-08T05:52:17.981 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64679.log.gz 2024-05-08T05:52:17.981 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48431.log 2024-05-08T05:52:17.981 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61663.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61663.log.gz 2024-05-08T05:52:17.981 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67453.log 2024-05-08T05:52:17.981 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60760.log 2024-05-08T05:52:17.981 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.48431.log: /var/log/ceph/ceph-client.admin.67453.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48431.log.gz 2024-05-08T05:52:17.981 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42552.log 2024-05-08T05:52:17.981 INFO:teuthology.orchestra.run.smithi045.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.67453.log.gz 2024-05-08T05:52:17.981 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60760.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60760.log.gz 2024-05-08T05:52:17.981 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62973.log 2024-05-08T05:52:17.981 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67712.log 2024-05-08T05:52:17.981 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42552.log: /var/log/ceph/ceph-client.admin.62973.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42552.log.gz 2024-05-08T05:52:17.981 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51248.log 2024-05-08T05:52:17.982 INFO:teuthology.orchestra.run.smithi045.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.62973.log.gz 2024-05-08T05:52:17.982 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67712.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67712.log.gz 2024-05-08T05:52:17.982 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48779.log 2024-05-08T05:52:17.982 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57905.log 2024-05-08T05:52:17.982 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51248.log: /var/log/ceph/ceph-client.admin.48779.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48779.log.gz 2024-05-08T05:52:17.982 INFO:teuthology.orchestra.run.smithi045.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51248.log.gz 2024-05-08T05:52:17.982 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71265.log 2024-05-08T05:52:17.982 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72742.log 2024-05-08T05:52:17.982 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57905.log: /var/log/ceph/ceph-client.admin.71265.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71265.log.gz 2024-05-08T05:52:17.982 INFO:teuthology.orchestra.run.smithi045.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.57905.log.gz 2024-05-08T05:52:17.982 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46201.log 2024-05-08T05:52:17.982 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56779.log 2024-05-08T05:52:17.982 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72742.log: /var/log/ceph/ceph-client.admin.46201.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72742.log.gz 2024-05-08T05:52:17.982 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46201.log.gz 2024-05-08T05:52:17.983 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56700.log 2024-05-08T05:52:17.983 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56779.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56779.log.gz 2024-05-08T05:52:18.044 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59499.log 2024-05-08T05:52:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68174.log 2024-05-08T05:52:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56700.log: /var/log/ceph/ceph-client.admin.59499.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56700.log.gz 2024-05-08T05:52:18.045 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59499.log.gz 2024-05-08T05:52:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43228.log 2024-05-08T05:52:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.68174.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68174.log.gz 2024-05-08T05:52:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59281.log 2024-05-08T05:52:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55257.log 2024-05-08T05:52:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43228.log: /var/log/ceph/ceph-client.admin.59281.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43228.log.gz 2024-05-08T05:52:18.045 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59281.log.gz 2024-05-08T05:52:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67430.log 2024-05-08T05:52:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55257.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55257.log.gz 2024-05-08T05:52:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58472.log 2024-05-08T05:52:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47077.log 2024-05-08T05:52:18.045 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67430.log: /var/log/ceph/ceph-client.admin.58472.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67430.log.gz 2024-05-08T05:52:18.046 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42319.log 2024-05-08T05:52:18.046 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58472.log.gz 2024-05-08T05:52:18.046 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47077.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47077.log.gz 2024-05-08T05:52:18.046 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48705.log 2024-05-08T05:52:18.046 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69808.log 2024-05-08T05:52:18.046 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42319.log: /var/log/ceph/ceph-client.admin.48705.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.48705.log.gz 2024-05-08T05:52:18.046 INFO:teuthology.orchestra.run.smithi045.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.42319.log.gz 2024-05-08T05:52:18.046 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68090.log 2024-05-08T05:52:18.046 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50458.log 2024-05-08T05:52:18.046 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69808.log.gz 2024-05-08T05:52:18.046 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.68090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58758.log 2024-05-08T05:52:18.046 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50458.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68090.log.gz 2024-05-08T05:52:18.046 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50458.log.gz 2024-05-08T05:52:18.046 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49420.log 2024-05-08T05:52:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57990.log 2024-05-08T05:52:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.58758.log: /var/log/ceph/ceph-client.admin.49420.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58758.log.gz 2024-05-08T05:52:18.047 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49420.log.gz 2024-05-08T05:52:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52880.log 2024-05-08T05:52:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43226.log 2024-05-08T05:52:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57990.log: /var/log/ceph/ceph-client.admin.52880.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52880.log.gz 2024-05-08T05:52:18.047 INFO:teuthology.orchestra.run.smithi045.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57990.log.gz 2024-05-08T05:52:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68132.log 2024-05-08T05:52:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74140.log 2024-05-08T05:52:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43226.log: /var/log/ceph/ceph-client.admin.68132.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43226.log.gz 2024-05-08T05:52:18.047 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68132.log.gz 2024-05-08T05:52:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55005.log 2024-05-08T05:52:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60233.log 2024-05-08T05:52:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55005.log: /var/log/ceph/ceph-client.admin.74140.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74140.log.gz 2024-05-08T05:52:18.047 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55005.log.gz 2024-05-08T05:52:18.047 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58716.log 2024-05-08T05:52:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72952.log 2024-05-08T05:52:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60233.log: /var/log/ceph/ceph-client.admin.58716.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58716.log.gz 2024-05-08T05:52:18.048 INFO:teuthology.orchestra.run.smithi045.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.60233.log.gz 2024-05-08T05:52:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71385.log 2024-05-08T05:52:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55686.log 2024-05-08T05:52:18.048 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72952.log.gz 2024-05-08T05:52:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.71385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67364.log 2024-05-08T05:52:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55686.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.71385.log.gz 2024-05-08T05:52:18.048 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55686.log.gz 2024-05-08T05:52:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58986.log 2024-05-08T05:52:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54160.log 2024-05-08T05:52:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67364.log: /var/log/ceph/ceph-client.admin.58986.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58986.log.gz 2024-05-08T05:52:18.048 INFO:teuthology.orchestra.run.smithi045.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.67364.log.gz 2024-05-08T05:52:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41202.log 2024-05-08T05:52:18.048 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51421.log 2024-05-08T05:52:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41202.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.54160.log.gz 2024-05-08T05:52:18.049 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41202.log.gz 2024-05-08T05:52:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62552.log 2024-05-08T05:52:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.51421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49243.log 2024-05-08T05:52:18.049 INFO:teuthology.orchestra.run.smithi045.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51421.log.gz 2024-05-08T05:52:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54311.log 2024-05-08T05:52:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49243.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62552.log.gz 2024-05-08T05:52:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65061.log 2024-05-08T05:52:18.049 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49243.log.gz 2024-05-08T05:52:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50542.log 2024-05-08T05:52:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.65061.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54311.log.gz 2024-05-08T05:52:18.049 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65061.log.gz 2024-05-08T05:52:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53006.log 2024-05-08T05:52:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70830.log 2024-05-08T05:52:18.049 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50542.log: /var/log/ceph/ceph-client.admin.53006.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50542.log.gz 2024-05-08T05:52:18.049 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53006.log.gz 2024-05-08T05:52:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47351.log 2024-05-08T05:52:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62396.log 2024-05-08T05:52:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70830.log: /var/log/ceph/ceph-client.admin.47351.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.70830.log.gz 2024-05-08T05:52:18.050 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47351.log.gz 2024-05-08T05:52:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57234.log 2024-05-08T05:52:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53258.log 2024-05-08T05:52:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.62396.log: /var/log/ceph/ceph-client.0.57234.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57234.log.gz 2024-05-08T05:52:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44351.log 2024-05-08T05:52:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.53258.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53258.log.gz 2024-05-08T05:52:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47442.log 2024-05-08T05:52:18.050 INFO:teuthology.orchestra.run.smithi045.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.62396.log.gz/var/log/ceph/ceph-client.admin.44351.log: 2024-05-08T05:52:18.050 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46369.log 2024-05-08T05:52:18.050 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44351.log.gz 2024-05-08T05:52:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72004.log 2024-05-08T05:52:18.050 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.47442.log: /var/log/ceph/ceph-client.admin.46369.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47442.log.gz 2024-05-08T05:52:18.051 INFO:teuthology.orchestra.run.smithi045.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46369.log.gz 2024-05-08T05:52:18.051 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68515.log 2024-05-08T05:52:18.051 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46621.log 2024-05-08T05:52:18.051 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.72004.log: /var/log/ceph/ceph-client.admin.68515.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72004.log.gz 2024-05-08T05:52:18.051 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68515.log.gz 2024-05-08T05:52:18.051 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69951.log 2024-05-08T05:52:18.051 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44876.log 2024-05-08T05:52:18.051 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46621.log: /var/log/ceph/ceph-client.admin.69951.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46621.log.gz 2024-05-08T05:52:18.051 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69951.log.gz 2024-05-08T05:52:18.051 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52102.log 2024-05-08T05:52:18.051 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59709.log 2024-05-08T05:52:18.051 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.0.44876.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44876.log.gz 2024-05-08T05:52:18.051 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.52102.log: 92.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-08T05:52:18.052 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65704.log 2024-05-08T05:52:18.052 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52102.log.gz 2024-05-08T05:52:18.052 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.59709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50146.log 2024-05-08T05:52:18.052 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.65704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61013.log 2024-05-08T05:52:18.052 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59709.log.gz 2024-05-08T05:52:18.052 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65704.log.gz 2024-05-08T05:52:18.052 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.50146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57839.log 2024-05-08T05:52:18.052 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.61013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73635.log 2024-05-08T05:52:18.052 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50146.log.gz 2024-05-08T05:52:18.052 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.57839.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61013.log.gz 2024-05-08T05:52:18.052 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46453.log 2024-05-08T05:52:18.052 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73635.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.57839.log.gz 2024-05-08T05:52:18.052 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73635.log.gz 2024-05-08T05:52:18.052 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-08T05:52:18.052 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74014.log 2024-05-08T05:52:18.052 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60107.log 2024-05-08T05:52:18.053 INFO:teuthology.orchestra.run.smithi045.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46453.log.gz 2024-05-08T05:52:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.74014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73804.log 2024-05-08T05:52:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60107.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74014.log.gz 2024-05-08T05:52:18.053 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60107.log.gz 2024-05-08T05:52:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55173.log 2024-05-08T05:52:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55938.log 2024-05-08T05:52:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73804.log: /var/log/ceph/ceph-client.admin.55173.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73804.log.gz 2024-05-08T05:52:18.053 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55173.log.gz 2024-05-08T05:52:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63844.log 2024-05-08T05:52:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42671.log 2024-05-08T05:52:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.63844.log: /var/log/ceph/ceph-client.admin.55938.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55938.log.gz 2024-05-08T05:52:18.053 INFO:teuthology.orchestra.run.smithi045.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63844.log.gz 2024-05-08T05:52:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57396.log 2024-05-08T05:52:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42965.log 2024-05-08T05:52:18.053 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.42671.log: /var/log/ceph/ceph-client.admin.57396.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42671.log.gz 2024-05-08T05:52:18.053 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57396.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56658.log 2024-05-08T05:52:18.053 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-05-08T05:52:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54094.log 2024-05-08T05:52:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56658.log: /var/log/ceph/ceph-client.admin.42965.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42965.log.gz 2024-05-08T05:52:18.054 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56658.log.gz 2024-05-08T05:52:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68798.log 2024-05-08T05:52:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41646.log 2024-05-08T05:52:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54094.log: /var/log/ceph/ceph-client.admin.68798.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68798.log.gz 2024-05-08T05:52:18.054 INFO:teuthology.orchestra.run.smithi045.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.54094.log.gz 2024-05-08T05:52:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40943.log 2024-05-08T05:52:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55854.log 2024-05-08T05:52:18.054 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41646.log.gz 2024-05-08T05:52:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.40943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43881.log 2024-05-08T05:52:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55854.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40943.log.gz 2024-05-08T05:52:18.054 INFO:teuthology.orchestra.run.smithi045.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55854.log.gz 2024-05-08T05:52:18.054 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73139.log 2024-05-08T05:52:18.055 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58800.log 2024-05-08T05:52:18.055 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43881.log: /var/log/ceph/ceph-client.admin.73139.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43881.log.gz 2024-05-08T05:52:18.055 INFO:teuthology.orchestra.run.smithi045.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73139.log.gz 2024-05-08T05:52:18.055 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63657.log 2024-05-08T05:52:18.055 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73762.log 2024-05-08T05:52:18.055 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.58800.log: /var/log/ceph/ceph-client.admin.63657.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.58800.log.gz 2024-05-08T05:52:18.055 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63657.log.gz 2024-05-08T05:52:18.055 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43763.log 2024-05-08T05:52:18.055 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63615.log 2024-05-08T05:52:18.055 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.73762.log: /var/log/ceph/ceph-client.admin.43763.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73762.log.gz 2024-05-08T05:52:18.055 INFO:teuthology.orchestra.run.smithi045.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43763.log.gz 2024-05-08T05:52:18.055 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43883.log 2024-05-08T05:52:18.055 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63531.log 2024-05-08T05:52:18.055 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.63615.log: /var/log/ceph/ceph-client.admin.43883.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63615.log.gz 2024-05-08T05:52:18.055 INFO:teuthology.orchestra.run.smithi045.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43883.log.gz 2024-05-08T05:52:18.056 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68645.log 2024-05-08T05:52:18.056 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44225.log 2024-05-08T05:52:18.056 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.63531.log: /var/log/ceph/ceph-client.admin.68645.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63531.log.gz 2024-05-08T05:52:18.056 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68645.log.gz 2024-05-08T05:52:18.056 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59155.log 2024-05-08T05:52:18.056 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49724.log 2024-05-08T05:52:18.056 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44225.log: /var/log/ceph/ceph-client.admin.59155.log: 89.1% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44225.log.gz 2024-05-08T05:52:18.056 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59155.log.gz 2024-05-08T05:52:18.056 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68970.log 2024-05-08T05:52:18.056 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43564.log 2024-05-08T05:52:18.056 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.49724.log: /var/log/ceph/ceph-client.admin.68970.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49724.log.gz 2024-05-08T05:52:18.056 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.68970.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.49285.log 2024-05-08T05:52:18.056 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-05-08T05:52:18.056 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55980.log 2024-05-08T05:52:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43564.log: /var/log/ceph/ceph-client.admin.49285.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43564.log.gz 2024-05-08T05:52:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63801.log 2024-05-08T05:52:18.057 INFO:teuthology.orchestra.run.smithi045.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49285.log.gz 2024-05-08T05:52:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70524.log 2024-05-08T05:52:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.63801.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55980.log.gz 2024-05-08T05:52:18.057 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63801.log.gz 2024-05-08T05:52:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60971.log 2024-05-08T05:52:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.70524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68311.log 2024-05-08T05:52:18.057 INFO:teuthology.orchestra.run.smithi045.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.70524.log.gz 2024-05-08T05:52:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.60971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56980.log 2024-05-08T05:52:18.057 INFO:teuthology.orchestra.run.smithi045.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.60971.log.gz 2024-05-08T05:52:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.68311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54245.log 2024-05-08T05:52:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.56980.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68311.log.gz 2024-05-08T05:52:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63300.log 2024-05-08T05:52:18.057 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.54245.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.56980.log.gz 2024-05-08T05:52:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43903.log 2024-05-08T05:52:18.058 INFO:teuthology.orchestra.run.smithi045.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54245.log.gz 2024-05-08T05:52:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.63300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62598.log 2024-05-08T05:52:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43903.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63300.log.gz 2024-05-08T05:52:18.058 INFO:teuthology.orchestra.run.smithi045.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43903.log.gz 2024-05-08T05:52:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58674.log 2024-05-08T05:52:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.62598.log: -- /var/log/ceph/ceph-client.admin.69264.log 2024-05-08T05:52:18.058 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62598.log.gz 2024-05-08T05:52:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.58674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41330.log 2024-05-08T05:52:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69264.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58674.log.gz 2024-05-08T05:52:18.058 INFO:teuthology.orchestra.run.smithi045.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69264.log.gz 2024-05-08T05:52:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35740.log 2024-05-08T05:52:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70676.log 2024-05-08T05:52:18.058 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.41330.log: /var/log/ceph/ceph-client.admin.35740.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41330.log.gz 2024-05-08T05:52:18.059 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35740.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46579.log 2024-05-08T05:52:18.059 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-05-08T05:52:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71749.log 2024-05-08T05:52:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.46579.log: /var/log/ceph/ceph-client.admin.70676.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70676.log.gz 2024-05-08T05:52:18.059 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46579.log.gz 2024-05-08T05:52:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49464.log 2024-05-08T05:52:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69012.log 2024-05-08T05:52:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.71749.log: /var/log/ceph/ceph-client.admin.49464.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49464.log.gz 2024-05-08T05:52:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46243.log 2024-05-08T05:52:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69012.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.71749.log.gz 2024-05-08T05:52:18.059 INFO:teuthology.orchestra.run.smithi045.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69012.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58056.log 2024-05-08T05:52:18.059 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-05-08T05:52:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55556.log 2024-05-08T05:52:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.58056.log: /var/log/ceph/ceph-client.admin.46243.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46243.log.gz 2024-05-08T05:52:18.059 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58056.log.gz 2024-05-08T05:52:18.059 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41024.log 2024-05-08T05:52:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43049.log 2024-05-08T05:52:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55556.log: /var/log/ceph/ceph-client.admin.41024.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55556.log.gz 2024-05-08T05:52:18.060 INFO:teuthology.orchestra.run.smithi045.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41024.log.gz 2024-05-08T05:52:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45803.log 2024-05-08T05:52:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66312.log 2024-05-08T05:52:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.43049.log: /var/log/ceph/ceph-client.admin.45803.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43049.log.gz 2024-05-08T05:52:18.060 INFO:teuthology.orchestra.run.smithi045.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45803.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55728.log 2024-05-08T05:52:18.060 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-05-08T05:52:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55598.log 2024-05-08T05:52:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.66312.log: /var/log/ceph/ceph-client.admin.55728.log: 89.3% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66312.log.gz 2024-05-08T05:52:18.060 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55728.log.gz 2024-05-08T05:52:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72826.log 2024-05-08T05:52:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44393.log 2024-05-08T05:52:18.060 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.55598.log: /var/log/ceph/ceph-client.admin.72826.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55598.log.gz 2024-05-08T05:52:18.060 INFO:teuthology.orchestra.run.smithi045.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72826.log.gz 2024-05-08T05:52:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72910.log 2024-05-08T05:52:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67922.log 2024-05-08T05:52:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.44393.log: /var/log/ceph/ceph-client.admin.72910.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72910.log.gz 2024-05-08T05:52:18.061 INFO:teuthology.orchestra.run.smithi045.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.44393.log.gz 2024-05-08T05:52:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67754.log 2024-05-08T05:52:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69372.log 2024-05-08T05:52:18.061 INFO:teuthology.orchestra.run.smithi045.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67922.log.gz 2024-05-08T05:52:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.67754.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67754.log.gz 2024-05-08T05:52:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ceph-client.admin.69372.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69372.log.gz 2024-05-08T05:52:18.121 INFO:teuthology.orchestra.run.smithi045.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-08T05:52:18.129 INFO:teuthology.orchestra.run.smithi063.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-08T05:52:18.147 INFO:teuthology.orchestra.run.smithi063.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-08T05:52:18.510 INFO:teuthology.orchestra.run.smithi045.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-08T05:52:18.976 INFO:teuthology.orchestra.run.smithi045.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-08T05:52:19.011 INFO:teuthology.orchestra.run.smithi063.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-08T05:52:19.032 INFO:teuthology.orchestra.run.smithi063.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-08T05:52:19.076 INFO:teuthology.orchestra.run.smithi063.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-08T05:52:19.108 INFO:teuthology.orchestra.run.smithi063.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-08T05:52:19.110 INFO:teuthology.orchestra.run.smithi063.stderr: 2024-05-08T05:52:19.110 INFO:teuthology.orchestra.run.smithi063.stderr:real 0m1.380s 2024-05-08T05:52:19.111 INFO:teuthology.orchestra.run.smithi063.stderr:user 0m6.085s 2024-05-08T05:52:19.111 INFO:teuthology.orchestra.run.smithi063.stderr:sys 0m0.217s 2024-05-08T05:52:19.201 INFO:teuthology.orchestra.run.smithi045.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-08T05:52:19.326 INFO:teuthology.orchestra.run.smithi045.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-08T05:52:19.330 INFO:teuthology.orchestra.run.smithi045.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-08T05:52:19.332 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-05-08T05:52:19.332 INFO:teuthology.orchestra.run.smithi045.stderr:real 0m1.633s 2024-05-08T05:52:19.332 INFO:teuthology.orchestra.run.smithi045.stderr:user 0m6.863s 2024-05-08T05:52:19.332 INFO:teuthology.orchestra.run.smithi045.stderr:sys 0m0.560s 2024-05-08T05:52:19.332 INFO:tasks.ceph:Archiving logs... 2024-05-08T05:52:19.332 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/jcollin-2024-05-08_05:22:48-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697310/remote/smithi045/log 2024-05-08T05:52:19.334 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T05:52:20.835 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/log/ceph to /home/teuthworker/archive/jcollin-2024-05-08_05:22:48-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697310/remote/smithi063/log 2024-05-08T05:52:20.836 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T05:52:21.692 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T05:52:21.702 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T05:52:21.702 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T05:52:21.705 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T05:52:21.790 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T05:52:21.799 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T05:52:21.800 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T05:52:21.803 DEBUG:teuthology.orchestra.run.smithi063:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T05:52:21.817 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-05-08T05:52:21.820 INFO:teuthology.orchestra.run.smithi063.stderr:bash: line 1: ntpq: command not found 2024-05-08T05:52:21.822 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T05:52:21.822 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-05-08T05:52:21.822 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 +204us[ +218us] +/- 43ms 2024-05-08T05:52:21.822 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 1 -326us[ -312us] +/- 69ms 2024-05-08T05:52:21.822 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 -5631ns[+8118ns] +/- 66ms 2024-05-08T05:52:21.822 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:52:21.824 INFO:teuthology.orchestra.run.smithi063.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T05:52:21.824 INFO:teuthology.orchestra.run.smithi063.stdout:=============================================================================== 2024-05-08T05:52:21.824 INFO:teuthology.orchestra.run.smithi063.stdout:^* hv01.front.sepia.ceph.com 3 6 377 3 +190us[ +190us] +/- 43ms 2024-05-08T05:52:21.825 INFO:teuthology.orchestra.run.smithi063.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 3 -311us[ -311us] +/- 69ms 2024-05-08T05:52:21.825 INFO:teuthology.orchestra.run.smithi063.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 -5003ns[-5003ns] +/- 66ms 2024-05-08T05:52:21.825 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:52:21.825 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T05:52:21.851 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T05:52:21.851 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T05:52:21.927 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T05:52:22.018 DEBUG:teuthology.orchestra.run.smithi063:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T05:52:22.092 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T05:52:22.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:52:22.125 DEBUG:teuthology.orchestra.run.smithi063:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T05:52:22.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:52:22.157 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T05:52:22.167 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T05:52:22.236 INFO:teuthology.task.internal:Duration was 1102.367893 seconds 2024-05-08T05:52:22.237 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T05:52:22.262 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T05:52:22.262 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T05:52:22.265 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T05:52:22.306 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T05:52:22.309 INFO:teuthology.orchestra.run.smithi063.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T05:52:22.600 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T05:52:22.600 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2024-05-08T05:52:22.600 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T05:52:22.624 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi063.front.sepia.ceph.com 2024-05-08T05:52:22.624 DEBUG:teuthology.orchestra.run.smithi063:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T05:52:22.671 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T05:52:22.672 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T05:52:22.674 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T05:52:22.741 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T05:52:22.741 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T05:52:22.744 DEBUG:teuthology.orchestra.run.smithi063:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T05:52:22.888 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T05:52:22.900 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T05:52:22.900 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T05:52:22.925 DEBUG:teuthology.orchestra.run.smithi063:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T05:52:22.952 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T05:52:22.963 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T05:52:22.968 DEBUG:teuthology.orchestra.run.smithi063:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T05:52:22.990 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2024-05-08T05:52:23.015 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = core 2024-05-08T05:52:23.029 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T05:52:23.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:52:23.059 DEBUG:teuthology.orchestra.run.smithi063:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T05:52:23.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:52:23.085 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T05:52:23.096 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T05:52:23.097 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/jcollin-2024-05-08_05:22:48-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697310/remote/smithi045 2024-05-08T05:52:23.097 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T05:52:23.136 DEBUG:teuthology.misc:Transferring archived files from smithi063:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/jcollin-2024-05-08_05:22:48-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697310/remote/smithi063 2024-05-08T05:52:23.137 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T05:52:23.170 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T05:52:23.170 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T05:52:23.173 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T05:52:23.221 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T05:52:23.234 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T05:52:23.234 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T05:52:23.314 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T05:52:23.314 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T05:52:23.316 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T05:52:23.330 INFO:teuthology.orchestra.run.smithi045.stdout: 265295 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 05:52 /home/ubuntu/cephtest 2024-05-08T05:52:23.332 INFO:teuthology.orchestra.run.smithi063.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 05:52 /home/ubuntu/cephtest 2024-05-08T05:52:23.333 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T05:52:23.421 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T05:52:23.527 INFO:teuthology.nuke:Checking targets against current locks 2024-05-08T05:52:23.554 DEBUG:teuthology.nuke:shortname: smithi045 2024-05-08T05:52:23.555 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T05:52:23.581 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-05-08_05:22:48-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697310', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 05:25:06.792692', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNg+jL2TFZLqEtYhSHQewN38uACCZFO7eXUwqrpHlx/FTXOWQj+xZ3w69kzHZOiA4tYvSJvJbpniXpITjUVOnWA='} 2024-05-08T05:52:23.584 DEBUG:teuthology.nuke:shortname: smithi063 2024-05-08T05:52:23.585 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T05:52:23.610 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi063.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/jcollin-2024-05-08_05:22:48-fs:functional-wip-jcollin-testing-07052024-distro-default-smithi/7697310', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 05:25:06.794254', 'locked_by': 'scheduled_jcollin@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJLilVd7/rNRhWYw6MU9+3Ckro0GQceTbVPKn/c5mxyp28sk/YF12goWlIH7ePrfPdpZnK6SAh44avQEtMLye4g='} 2024-05-08T05:52:23.642 INFO:teuthology.orchestra.console.smithi045:Power off 2024-05-08T05:52:23.643 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-08T05:52:23.666 DEBUG:teuthology.orchestra.console.smithi045:power off output: Chassis Power Control: Down/Off 2024-05-08T05:52:23.666 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T05:52:23.682 INFO:teuthology.orchestra.console.smithi063:Power off 2024-05-08T05:52:23.683 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-08T05:52:23.694 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2024-05-08T05:52:23.704 DEBUG:teuthology.orchestra.console.smithi063:power off output: Chassis Power Control: Down/Off 2024-05-08T05:52:23.704 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T05:52:23.730 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is on 2024-05-08T05:52:27.699 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T05:52:27.731 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T05:52:27.810 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2024-05-08T05:52:27.840 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is on 2024-05-08T05:52:31.814 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T05:52:31.842 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T05:52:31.925 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is off 2024-05-08T05:52:31.952 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is off 2024-05-08T05:52:32.026 INFO:teuthology.orchestra.console.smithi045:Power off completed 2024-05-08T05:52:32.053 INFO:teuthology.orchestra.console.smithi063:Power off completed 2024-05-08T05:52:32.158 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} duration: 1102.3678934574127 failure_reason: '"2024-05-08T05:47:16.158078+0000 mds.b (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir)" in cluster log' flavor: default owner: scheduled_jcollin@teuthology success: false 2024-05-08T05:52:32.158 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T05:52:32.243 INFO:teuthology.run:FAIL